Table of contents

SyncPlanet API

The service is accessible at different levels using the programming interface:

The servers are based on damas-core (data management system core) which is a free and open source project providing generic CRUD operations . You can visit the Environment Setup Guide to setup a Python, Javascript or Shell scripting environment.

Types

A server stores objects made of key-values pairs in the JSON format, describing objects of different types:

The reserved _id key is the object’s primary key, being a unique identifier and index of the object.

File

A file is indexed using its path, as a unique identifier, under the _id key. A few other keys are used to control the synchronisation system (see the list below). Any number of custom keys can be added without limitation, this might be useful for an integration in a custom asset management system.

{
  "_id": "/file/path",         /* UNIX path format (String) */
  "comment": "<text>",         /* Text comment of user when published (String) */
  "origin": "<storagename>",      /* Storage unit name where the file is coming from  (String)*/
  "sync_priority": 7,          /* Positive or negative, 0 is default normal priority (Number) */ 
  "syncEverywhere": true       /* overide default behavior in globalParameters (Boolean) */
}

_id

String
File's unique identifier, using UNIX path format.

Note

The _id key has a path relative from the project's root directory as value

Note

A path always starts with a slash (/) character

Note

Slash (/) characters are used as sub level delimiters

Note

A folder can be indexed using a trailing slash, see Packages

comment

String
User's text comment, specified at publish.

Note

If the file was discovered by an scan agent, the comment will be:

scanned at <storagename>
or
modified at <storagename>

origin

String
The name of the storage on which the file was created, and where it has to be retrieved.

sync_priority

Number
Order preference to process this file. The files with a higher sync_priority number are processed first, default is 0 (normal priority) and it can be negative.

syncEverywhere

Boolean
Does this file has to be propagated everywhere? (on every other site that is was originated from?)

Note

To select specific targets : put syncEverywhere to False and use specific keys for each site: 

{
    "syncEverywhere": False,
    "sync_studio1": True,
    "sync_studio2": False,
    "sync_studio3": True
}

This will propagate the file to studio1 and studio3 but not to studio2. Not specifying sync_studio2 is assumed False.
Format is:
sync_<studioname>: Boolean

your_custom_key

String|Number|Array|Boolean|Object

Tip

Use the JSON file object to store any key-value pair useful for your asset management system

File Publish

When some files are created and need to be propagated on several storages, a JSON object is provided at publish time with the keys described above, so the synchronisation service can process the file specified by the _id key. You can use the generic create method to insert new objects in the database using one of the available clients (Python, Javascript, or command line). You can visit the Environment Setup Guide to setup a Python, Javascript or Shell scripting environment.

New Keys at Synchronisation Process Time

Some keys are created and updated on each file as the synchronisation process runs. These keys are useful for the monitoring interfaces, to get the file's history log, or to debug the file's synchronisation:

{
  "_id": "/file/path",
  [...]
  "file_mtime": 1567529676000,      /* Number,  modification time on the origin file system (epoch time in milliseconds) */
  "file_size": 1286230019,          /* Number,  number of bytes on the origin file system */
  "modified_site5": 15676534821582, /* Number,  a modification was scanned on site5 */
  "syncError_site2": 23,            /* Number,  error encountered on site2 (rsync exit status code) */
  "syncLastAttempt_site2": 1567516933000, /* Number, time of last failed attempt on site2 */
  "synced_site1": 15675564912347,   /* Number,  successful delivery time on site1 */
  "synced_site3": 15675564591231,   /* Number,  successful delivery time on site3 */
  "synced_site4": 15675565175645,   /* Number,  successful delivery time on site4 */
  "syncing_site6": 1754897397000    /* the service is propagating this file on site6 since epoch (Number) */
}

New Keys at Creation

The author and time keys are automatically inserted by the tracker when a new file is created:

{
  "_id": "/file/path",
  [...]
  "author": "<@username>",  /* String, authenticated user who did the publish */
  "time": 1567529676000     /* Number, time of the element at creation, milliseconds EPOCH */
}

author

String
The authenticated user name of the author who did the publish.

Note

When a file was discovered by the automatic file system scan, the username support is used

Note

When the server is in authorMode, only authors can edit the object, along with admins

Note

The author key can be an array of usernames

time

Number
The time at publish. It is used to display and sort the elements in the interfaces. Epoch milliseconds: Number of milliseconds since January 1st 1970.

Note

The time key is also used to compare the synced_* keys so the system knows if a synchronized copy is outdated (yellow color is then used instead of green)

Optional Keys

If you need a multi-site locking system please contact us.

{
  "lock": "<@username>"            /* String,  username who is editing the file */
}

User

The users are objects wearing some specific keys: username, password, class plus optional keys.

{
"_id": "@username",               /* String, unique user name identifying the user in the database */
"class": "editor",                /* String, a user class for permissions. Could be guest, user, editor, admin  */  
"email": "usermail@address.com",  /* String */
"fullname": "Firstname Lastname", /* String */
"password": "13d3a2a16c0cd2f7bf115d471999377e" /* String, hashed password */
}

The Authentication page gives more details about the authentication username/password mechanism.

Team

Not documented yet

Hub

Read the dedicated hub programming section

Storage

Read the dedicated storage programming section

Channel

Read the dedicated channel programming section

Generic function lists and objects

CRUD Function List

A set of generic CRUD functions is provided to create, read, modify and delete elements:

User Management

Search queries

Function Details

lock

Lock file(s) for edition. Sets a lock key on elements with current authenticated username as value)

lock( identifiers [, callback] )

parameters

return values

# Python
# lock one asset
project.lock('/project/path/to/file')
# True

# lock multiple assets in 1 request
project.lock(['/project/path/to/file1', '/project/another_file_path'])
# True

Publish

Add files to the index

publish( elements [, callback] )

Many options are provided at publish time, allowing Folder Packages and Destination Filtering.

Parameters

return values

Same specifications as /api/create, except that it is expecting specific keys and a child element is created to keep track of the states upon publish

When we publish a file that is already tracked, the function returns an error

Examples

{
  "_id": "/project/path/new_file",
  "comment": "text",
  "origin": "storagename"
}

Child node :

{
  "_id": "55ae0b1ed81e88357d77d0e9",
  "#parent" : "/project/path/new_file",
  "comment": "text",
  "origin": "storagename"
}

optional keys (these keys are not mandatory but could ease multi sites configurations and version control):

In a multi-site environment, the origin and _id path are used to retrieve the file from the source server.

Unlock

Unlock a locked asset

unlock ( identifiers [, callback] )

Parameters

return values

If the asset is not locked or locked for someone else (lock key value != authenticated user name) it returns false. If it was successfully unlocked, returns true.

Comment

Add a comment to one or several element(s)

comment ( elements [, callback] )

Parameters

Return values

Sets author key on elements with the authenticated username as value, as well as time key with the current time at creation.

# Python
# single parent id
>>> project.comment({"#parent" : "asset_id", "comment" : "text"})
{u'author' : u'username', u'time' : 1480588505449, u'#parent' : u'asset_id', u'comment' : u'text'}

# multiple parent ids
>>> project.comment({"#parent" : ["asset_id1", "asset_id2"], "comment" : "text"})
[{u'author' : u'username', u'time' : 1480588505449, u'#parent' : u'asset_id1', u'comment' : u'text'}, {u'author' : u'username', u'time' : 1480588505449, u'#parent' : u'asset_id2', u'comment' : u'text'}]

// Javascript
damas.comment({'#parent' : "asset_id", comment : "text"});
>> Object { author: "damas", time: 1480588505449, '#parent': "asset_id", comment: "text" }

// comment using an asynchronous call
damas.comment({'#parent' : "asset_id", comment : "text"}, function (node) {
    // asynchronous mode
    console.log(node.time);
});

SyncPlanet-specific older documentation

Global Parameters

The global configuration parameters are accessible through a JSON element that is used by the synchronisation agents.

{
"_id": "rsyncGlobalParameters", /* generic parameter */
"enableScans": true,            /* parameter for scans */
"excludePatternRules": "",      /* parameter for scans */
"excludedFiles":       "",      /* parameter for scans */
"ignoreExistingFiles": false,   /* parameter for scans */
"includePatternRules": "",      /* parameter for scans */
"rsaPublicKey":        "",      /* key to install under volumes' .ssh/authorized_keys */
"sleepBetweenScans": 60,        /* parameter for scans */
"syncEverywhere": true,         /* [Destination Filtering](Destination-Filtering) */
"time": 1555711910004           /* generic parameter */
}

Volumes

Each volume is described as an element which holds its configuration and options to control its behavior.

{
"_id": "sit/inthebox/mice/amopix",  /* the volume identifier */
"agentStartTime": 1599731439000,    /* agent time when launched */
"agentStatus": "on",                /* status of the agent, on / off / undefined */
"agentVersion": "1.0.5",            /* version of the running agent */
"author": "support",                /* generic parameter */
"enableDiscovery": true,             /* Boolean, filesystem scan switch */
"enableDiscoveryPackages": true,     /* Boolean, filesystem scan switch */
"enableDiscoveryUpdatedFiles": true, /* Boolean, filesystem scan switch */
"enableDiscoveryDeletedFiles": true, /* Boolean, filesystem scan switch */
"enableEmission": true,              /* Boolean, file emission switch */
"enableReception": true,             /* Boolean, file reception switch */
"error": 0,                    /* Number, error encountered */
"exit_emission": 0,            /* Number, error encountered */ 
"exit_reception": 0,           /* Number, error encountered */ 
"exit_scan": 0,                /* Number, error encountered */ 
"last_emission": 1571918493000,   /* Number, last emission */
"last_reception": 1571910212000,  /* Number, last reception */
"last_scan": 1571918421000,       /* Number, last filesystem scan */
"lastConnectionAttempt": 1600942773000,      /* Number, time of last connection attempt */
"lastConnectionStatusChange": 1600512139000, /* Number, time of previous status */
"lastConnectionSuccess": 1600942773000,      /* Number, last successful connection to the volume */
"last_emission": 1600942573000,   /* Number, last file emission time */
"last_reception": 1598770303000,  /* Number, last file reception time */
"last_scan": 1600942965000,       /* Number, last volume scan time */
"name": "amopix",             /* String, volume's name */
"notifiedEmails": ""          /* String, comma separated email addresses */
"rsync_args": "",             /* String, arguments for rsync */
"rsync_env": "",              /* String, environment for rsync */
"scan_duration": 14,          /* Number, last filesystem scan duration */
"stderr_emission": "",        /* String, stderr if emission error */
"stderr_scan": "",            /* String, stderr if scan error */
"syncAmountEmission":  10,    /* Number, Number of elements uploaded each time */
"syncAmountReception": 10,    /* Number, Number of elements downloaded each time */
"time": 1563285980221,        /* generic parameter */
"url": "ssh://user@adress:port/project/" /* String, URL of the volume */
}