Storage rules [VC 21.3 GEN]

Storage rules are a way of controlling the availability of files. The rules describe where files of different types are stored. Settings include a minimum number of storages, specific storages and priorities for how suited a storage is for a particular type. A rule can be applied on a specific item, collection, library or shape tag. To further filter which shapes that the rules applies to, a shape tag can be set. Files can be named using storage name rules.

A storage rule can also describe where files should not be stored, in which case files will be eagerly removed. The default is otherwise to start removing files once the high watermark on a storage has been reached. A rule can specify specific storages or storage groups, or that all other storages should be excluded by using the “all” qualifier.

Negative rules do not work for storages of type ARCHIVE

Storage rules on collections will also be inherited to items in sub-collections. See Inherited rule example.

Files can also be load balanced across multiple storages.

Resolving storage rules

If a minimum number of storages has been set and an insufficient amount of specific storages are given, priorities are used to pick a suitable storage. The different priority criteria can be seen in the table below. The criteria type is given together with an integer describing its priority, where a lower number means that it is more important than an entry with a higher number.

Which rules apply?

Certain rules takes precedence over other rules. There are three things that factors into this decision process (ordered according to their importance):

1. The precedence given to the rule.

2. The type of the entity the rule is applied to.

3. Whether the rule is set to a certain shape tag or not.

Below a table of available precedence values can be seen, ordered from most important to least important.

Below a table of the difference entity types can be seen, ordered from most important to least important.

So for example a rule with the precedence value HIGHEST, that is applied to a certain shape tag on an item will always take precedence over any other rule.

How are storage rules applied?

Since a shape can have 0 or more shape tags, there can be some ambiguity between the rules. Below a basic algorithm, that describes how the rules are applied, can be seen.

1. Start out with an empty set of storages, S.

2. Add all storages, given in the specific rules, to S.

3. If S is empty, add in storages specified in the generic rule.

4. Set the minimum required storages, n, to equal the highest number specified in the specific rules and the generic rule.

5. If the size of S is less than n:

   1. Retrieve the priorities from one of the specific rules.

   2. If no specific rule specified any priorities, use the generic rule.

   3. If the generic rule did not specify any priorities, use some system default priorities.

   4. Attempt to fill S using the priorities.

Examples

Simple rule example

Setting a simple rule on a item, dictating that the item’s original shape should exist on at least two storages, and one of them must be storage VX-3

PUT /item/VX-28/storage-rule/original
Content-type: application/xml

<StorageRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <storageCount>2</storageCount>
  <storage>VX-3</storage>
</StorageRuleDocument>

Negative rule example

Setting a simple rule on a item, dictating that the item’s original shape should exist on at least two storages, and one of them must be storage VX-3, and it must not exist on storage VX-2.

PUT /item/VX-28/storage-rule/original
Content-type: application/xml

<StorageRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <storageCount>2</storageCount>
  <storage>VX-3</storage>
  <not>
    <storage>VX-2</storage>
  </not>
</StorageRuleDocument>

Load balancing example

The pool element can be used to specify the storages that files could be stored on. This can be used to spread files across multiple selected storages.

For example, to specify that a file should exist on at least two of storages VX-1, VX-2 and VX-3:

<StorageRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <storageCount>2</storageCount>
  <pool>
    <storage>VX-1</storage>
    <storage>VX-2</storage>
    <storage>VX-3</storage>
  </pool>
</StorageRuleDocument>

Or to have one copy on S3 (storage VX-1 in this example) and one copy on a local storage:

<StorageRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <storageCount>2</storageCount>
  <storage>VX-1</storage>
  <pool>
    <group>local-storage-pool</group>
  </pool>
</StorageRuleDocument>

Inherited rule example

Storage rules on collections by default only applies to the items in the collection and does not apply for items that exist in any sub-collections.

To change so that a collection storage rule applies to all items in it and all items in any sub-collections, recursively, use:

<StorageRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <storageCount>1</storageCount>
  <inherited>true</inherited>
  <storage>VX-1</storage>
</StorageRuleDocument>