Automatic import [VC 21.3.1 GEN]
A storage can be configured to automatically import new files/image sequences that are detected. Auto-import rules define what transcodes that should be performed as well as what metadata to be used if none can be found. Metadata can automatically be found if it shares the same filename and has the extension
.xml, for example
Auto-import rules can also use Import settings to set up access control lists by setting the optional
Import using a specific transcoder resource
To use a specific transcoder resource during auto import, specify its resource id in the auto import rule using the
PUT /storage/VX-2/auto-import Content-Type: application/xml <AutoImportRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine"> <tag>myflvtag</tag> <resourceId>VX-1</resourceId> </AutoImportRuleDocument>
You may also specify which transcoder resource to use by setting the
defaultTranscoder configuration property.
Setting a user for jobs started as a result of an auto import rule
The default behavior is that jobs started from an auto import rule will not have a user set. This can be changed by setting the
user element in the rule:
PUT /storage/VX-2/auto-import Content-Type: application/xml <AutoImportRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine"> <tag>myflvtag</tag> <user>firstname.lastname@example.org</user> </AutoImportRuleDocument>
Importing with a metadata file of an external format
Vidispine also supports auto imports with a metadata XML file that is of a different format than the native Vidispine MetadataDocument. This is achieved by associating a Metadata projections (XSLT transformation) with the auto import rule. First, create the projection, then set the auto import rule:
PUT /storage/VX-2/auto-import Content-Type: application/xml <AutoImportRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine"> <tag>myflvtag</tag> <projection>myProjection</projection> </AutoImportRuleDocument>
projection element contains a Projection id.
Any auto imports from this storage will then first transform the supplied XML file using the specified projection.
Disable automatic import rules
Auto-import rules can be disabled by setting the
enabled field to
false, or by using the disable rule request. When a rule is disabled then no new auto-import jobs will be created for new files discovered on the storage.
Jobs will be created once the rule is enabled again. By default rules are enabled.
If the admin only wants the user to be able to import during specific hours a program like cron could be used:
0 0 * * * curl -X PUT -uadmin:admin http://localhost:8080/API/storage/VX-1/auto-import/enable 0 5 * * * curl -X PUT -uadmin:admin http://localhost:8080/API/storage/VX-1/auto-import/disable
Sidecar auto import
If auto-import is enabled, sidecar files are by default identified by file extension and imported as metadata to files with matching filenames. See Importing sidecar files for the supported sidecar formats which are automatically identified. The AutoImportRuleDocument contains two fields determining how sidecar are handled by auto-import:
False by default. If set to true, files with a sidecar file extension are not imported in any way.
Extensions can be specified here to be excluded from being treated as sidecar files, and may instead be imported as new items.
If there are several files in a storage with the same file name (prefix) with different extensions, only one will be imported as an item and others will be either imported as sidecars or ignored. This can be changed by setting the system-wide configuration property
groupImportableFiles to false. Multiple files with the same prefix can then be auto-imported as individual items.
Multiple sidecar files can be imported to the same item. If
groupImportableFiles is false, one sidecar file may be auto-imported multiple times as metadata for different items with the same file prefix.
Title as metadata
The AutoImportRuleDocument contains a field
fileNameAsTitle. Setting this property to
true means that the “title” fields of all single files imported form this storage will be set to their file names.
Applying file name filters to auto import rules
There are two kinds of filename filters that can be applied to auto import rules:
- Exclusion filters
Used to exclude files from being auto imported. This can be useful when the OS creates files automatically, e.g.
Thumbs.dbon Windows or
.DS_Storefiles on Mac OS. Note that the expression must match the entire path, not only a part of the path.
- Shape tag filters
These can be used to transcode the imported file using a specific shape tag when a file name follows a certain pattern. You might want files ending in
.tiffto be transcoded using the tag
The filters are specified in the XML document you use to create/update the auto import rule.
<AutoImportRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine"> <metadata> <timespan start="-INF" end="+INF"> <field> <name>title</name> <value>This is an auto-imported item.</value> </field> </timespan> </metadata> <tag>generictag</tag> <excludeFilter> <pattern>.*\.DS_Store</pattern> <excludeFilter> <shapeTagFilter> <pattern>.*\.tiff</pattern> <tag>lowimage</tag> </shapeTagFilter> <shapeTagFilter> <pattern>.*\.mxf</pattern> <tag>lowvideo</tag> </shapeTagFilter> </AutoImportRuleDocument>
This rule will exclude any file ending with
.DS_Store. Any files ending with
.tiff will be imported with the shape tag
lowimage, and any files ending in
.mxf will be imported with the shape tag
lowvideo. All files will be imported with the shape tag
Auto import of image sequences
Deprecated since version 4.6: Define a sequence pattern on the storage and use an auto-import rule without a sequence definition instead.
Image sequences can be auto detected and imported if their file names match the predefined regex in AutoImportRuleDocument. The elements in the document are:
Defines the file name pattern, and it is mandatory.
Defines the metadata file name pattern.
The matching group in the regex should be used as the id of the file sequence.
The matching group in the regex that should represent the position of a file in a sequence.
A sequence is considered as completed after a certain timeout (in seconds). The default timeout is 60 seconds.
<AutoImportRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine"> <tag>mp4</tag> <metadata> <timespan end="+INF" start="-INF"> <field> <name>title</name> <value>auto-imported item.</value> </field> </timespan> </metadata> <sequenceDefinition> <sequenceMetadata> <regex>(.*)-metadata.xml</regex> <idGroup>1</idGroup> </sequenceMetadata> <fileSequence> <regex>(.*)-([0-9]+).(dpx|tga|png|jpg)</regex> <idGroup>1</idGroup> <numGroup>2</numGroup> <timeout>10</timeout> <!-- seconds--> </fileSequence> </sequenceDefinition> </AutoImportRuleDocument>
Given a storage with the above import rule, with the files:
foo-metadata.xml foo-001.dpx foo-002.dpx foo-002.dpx
Then these would be recognized as a sequence
foo-metadata.xml as the metadata.