Transfer Files Between VidiCore Storages [ENT 21.3 IG]

Motivation

VidiCore-driven file transfers are possible via storages rules (which would be implicitly) or explicitly via an transfer jobs. To transfer files from within a VidiFlow agent explicit transfer job are the preferred option. The API for starting transfer jobs is described here: Move/copy/delete files from a storage.

Copying and moving files usually is orchestrated by a VidiFlow workflow. We highly recommend to use workflows for this purpose. The following section describes how VidiFlow is driving VidiCore for transferring files and can serve as a blueprint for initiating such a process directly on the VidiCore API.

Workflow

Let’s assume an item ITEM-VX-13 with an original shape where the Op1a MXF file is located on storage STORAGE-VX-4. To copy this file from storage STORAGE-VX-4 to storage STORAGE-VX-7 this API call is required:

POST /API/storage/VX-4/file/VX-53/storage/VX-19?move=false&filename=transfer.mxf

The call will return a job ID that can be used to monitor the status of the copy operation.

The (optional) filename parameter defines how the file is named on the destination storage. If the file already exist the transfer job will fail. The filename parameter may contain subfolders (delimited by forward slashes) that are created automatically if they do not exist.

If a shape has several files (e.g. for MXF OpAtom material) each file needs to be transferred in a separate transfer job. To get all files for a shape use this API call:

GET /API/item/VX-17/shape/VX-4/file

If you do not specify a destination file name VidiCore will try to use the source file’s name as destination file name. However, if there already exists a file with that name on the destination storage, VidiCore will choose a different name – which would not be the expected behaviour. To avoid this you should always specify the destination filename on the API level. The file name of the source file can be retrieved with an API call to the file resource of the source file (the path element contains the current filename).

VidiCore will transfer the source file to the target storage. The storage method to use is determined internally in VidiCore. If a storage is locally accessible (which only applies to the local file system) VidiCore directly copies the file via OS commands. For all other storage methods VidiCore Server will copy the file on a byte-by-byte basis by reading it from the source storage and writing it to the target storage.

VidiCore Agents (VSAs) can be used for distributing network load of file transfer operations onto different servers.