Moxion to ShotGrid Entity Mapping

Overview

Moxion has the capability of pushing Moxion Assets into ShotGrid Entities. Moxion Assets contain a rich data structure, including a large set of metadata.

A Moxion Asset is a single flat object, which often needs to be restructured into multiple, hierarchical ShotGrid Entities. 

For instance, Moxion may have metadata about 'Scene', 'Shoot Day' and 'Unit' in its metadata fields, but ShotGrid may choose model this as three 'Scene', 'ShootDay' and 'Unit' high-level Entities, which are linked to a ShotGrid Asset as necessary. 

The mapping functionality is intended to handle the process of transforming a Moxion Asset into multiple, interrelated ShotGrid entities.

Key Functionality of the Mapping System

• Splits a single Moxion Asset into a collection of interlinked ShotGrid Entities, in a flexible and configurable fashion.
• Describes the ShotGrid Entities to be targeted: what they are called and how they are to be located amongst the existing data.
• Describes how the metadata fields from the Moxion Asset map into ShotGrid fields. Uses the Java Expression Language (JEXL) as a powerful, simple way to do any data transformations or cleanup.
• Describes the relationships between the different ShotGrid Entities, and how these links are to be constructed when translating Moxion to ShotGrid.
• Creates or updates ShotGrid Entities as appropriate. This only applies to Entities ('records') - this functionality will not attempt to create missing Entity Types ('tables').
• Updates the schema of an Entity Type to add any missing Fields.
• Mappings are configured on a per-project basis with a JSON mapping file.
• Allows optional upload of thumbnail images to any ShotGrid Entity.
• Allows optional upload of video clips to Version entities.

This mapping format also has provision to simply and quickly push every field in the Moxion Asset metadata into a single, flat ShotGrid Entity. 

This covers off the traditional use case of 'send Moxion Asset some nominated ShotGrid Entity designed to receive Moxion Assets'. 

This function can be used in combination with the smart mapping features mentioned above.

Example Mapping

Mappings are expressed in JSON format. 

Any field in this file can be a JEXL expression, meaning that variables and logical functions can be used where needed. 

JEXL expressions start with a dollar sign.

Note that a JSON schema is available to make creating these files easier. 

Also, this example is intended to be a short sample, and it is not very realistic or exhaustive. 

The MoxionClip Entity used in this example is fictional.

See below for a mapping example that does the following:

• Takes a Moxion Asset and - when it is pushed to ShotGrid - transforms it to a ShotGrid MoxionClip Entity.
• Looks up a MoxionClip Entity by way of ShotGrid sg_moxion_id == Moxion's Asset ID.
• If an existing MoxionClip is found, updates it. If no MoxionClip exists, create a fresh one.
• Set the fields in the ShotGrid MoxionClip as follows:
• Moxion ID (sg_moxion_id) = Moxion's Asset ID
• Code (code) = Moxion's Clip name
• Description (sg_description) = Moxion's metadata description
• This is just a short sample. In real life, the fields can be very numerous, and will probably be transformed, cleaned up or set to reasonable defaults by JEXL expressions.
• Uploads a thumbnail to the MoxionClip.
• Finally, link the 'sg_shoot_day' entity field in the MoxionClip back to an appropriate ShotGrid ShootDay entity (looked up where ShotGrid ShootDay.code = Moxion's metadata shoot_day field).

Note that this file also contains a mapping for a Moxion Asset to ShotGrid 'ShootDay' Entity. 

This mapping is also run for every Moxion Asset pushed, and serves to create a ShootDay record if one does not exist (or update one if it does).

This example shows only a simple link between two ShotGrid Entities. 

In a realistic scenario, a much larger web of interrelated Entities could be created and maintained via this format. 

For instance, adding a 'Scene' and 'Take' entity could be achieved by modifying this file to add entity mappings for Scene and Take, followed by adding link definitions for MoxionClip.scene and MoxionClip.take .

Example Mapping JSON

"entityMappings": [
    {
      "entityName": "MoxionClip",           // name of the ShotGrid entity  
      "uploadThumbnail": "true",            // send a thumbnail to this entity
      "key": [                              // unique key to use when checking  
                                            // for existing
        {
          "field": "Moxion ID",
          "value": "$id"
        }
      ],
      "fields": [                           // fields to update in MoxionClip
        {
          "field": "Moxion ID",             // Use field label or ShotGrid name
          "value": "$id"                    // (one of many predefined vars)
        },
        {
          "field": "Moxion Link",           
          "value": "$moxionLink"
        },
        {
          "field": "code",
          "value": "$assetName"
        },
        {
          "field": "description",
          "value": "$metadata.description"
        }
      ],
      "links": [                            // link MoxionClip to a ShootDay
                                            // entity
        {
          "entityName": "ShootDay",
          "field": "shoot day",             // set MoxionClip.ShootDay
          "key": [                          // lookup criteria for ShootDay
            {
              "field": "code", 
              "value": "$metadata.shoot_day"
            }
          ]
        }
      ]
    },
    {
      "entityName": "ShootDay",            // Entity 2: ShootDay
      "key": [                             
        {
          "field": "code",
          "value": "$metadata.shoot_day"
        }
      ],
      "fields": [
        {
          "field": "code",
          "value": "$metadata.shoot_day"
        },
        {
          "field": "description",
          "value": "
<h5>Predefined JEXL variables </h5>
<p>
	 These variables may be used anywhere in the mapping file:
</p>
<table>
<thead>
<tr>
	<th>
		 Jexl variable
	</th>
	<th>
		 Example
	</th>
	<th>
		 Comments
	</th>
</tr>
</thead>
<tbody>
<tr>
	<td>
		 id
	</td>
	<td>
		 7154824B7B <br>
		 341C2195D4 <br>
		 90BA6B8D8920
	</td>
	<td>
		 Moxion’s Unique ID for the asset
	</td>
</tr>
<tr>
	<td>
		 assetName
	</td>
	<td>
		 301-26AL-2b
	</td>
	<td>
		 Name of the asset, as it appears in Moxion
	</td>
</tr>
<tr>
	<td>
		 projectId
	</td>
	<td>
		 154
	</td>
	<td>
		 ShotGrid project ID we are pushing to.
	</td>
</tr>
<tr>
	<td>
		 uploadThumbnails
	</td>
	<td>
		 true
	</td>
	<td>
		 Is thumbnail upload enabled for this project?
	</td>
</tr>
<tr>
	<td>
		 uploadVideos
	</td>
	<td>
		 false
	</td>
	<td>
		 Is video upload enabled for this project?
	</td>
</tr>
<tr>
	<td>
		 entityCode
	</td>
	<td>
		 301-26AL-2b
	</td>
	<td>
		 Business key of the clip made up of scene, shot, take and camera. Potentially useful as a unique identifier.
	</td>
</tr>
<tr>
	<td>
		 assetViewerLink
	</td>
	<td>
		 <a href="http://moxion">http://moxion</a>. <br>
		 shotgunstudio. <br>
		 comdashboard <br>
		 /project/batch/ <br>
		 E98AD5F75BF0 <br>
		 79128B6287FF <br>
		 7C0906E2/ <br>
		 asset/ <br>
		 7154824B7B3 <br>
		 41C2195D49 <br>
		 0BA6B8D8920/<br>
	</td>
	<td>
		 Link to the Asset within the Moxion application. Login to the app will be required to view.
	</td>
</tr>
<tr>
	<td>
		 moxion_tags
	</td>
	<td>
		 VFX, CGI
	</td>
	<td>
		 The tags associated with this asset in Moxion. Separated by commas.
	</td>
</tr>
<tr>
	<td>
		 moxion_folder_path
	</td>
	<td>
		 My Project > My Folder
	</td>
	<td>
		 The full path to the Moxion folder where the asset is stored. Folders are separated by " > "
	</td>
</tr>
<tr>
	<td>
		 moxion_folder
	</td>
	<td>
		 My Folder
	</td>
	<td>
		 The name of the Moxion folder where the asset is stored.
	</td>
</tr>
</tbody>
</table>
<p>
	 In addition, we have a field called 'metadata', which contains the metadata for the asset in question.
</p>
<h5>The Metadata Field</h5>
<p>
	 The metadata map is a list of key/value pairs, which represents all the metadata held against an asset. 
</p>
<p>
	 The metadata fields may vary from asset to asset, depending on what has been provided to Moxion. 
</p>
<p>
	 Any metadata fields may be unset (ie null).
</p>
<p>
	 Metadata field names provided to Moxion are all sanitized to lowercase with underscores (snake case). 
</p>
<p>
	 For example, a metadata field called 'Camera TC' in the ALE file will be available through the Jexl variable '$metadata.camera_tc'.
</p>
<p>
	 Shout out to our ShotGrid wizard Mark Howard.
</p>Day ' + metadata.shoot_day"
        }
      ]
    }
  ]
}

Predefined JEXL variables 

These variables may be used anywhere in the mapping file:

In addition, we have a field called 'metadata', which contains the metadata for the asset in question.

The Metadata Field

The metadata map is a list of key/value pairs, which represents all the metadata held against an asset. 

The metadata fields may vary from asset to asset, depending on what has been provided to Moxion. 

Any metadata fields may be unset (ie null).

Metadata field names provided to Moxion are all sanitized to lowercase with underscores (snake case). 

For example, a metadata field called 'Camera TC' in the ALE file will be available through the Jexl variable '$metadata.camera_tc'.

Shout out to our ShotGrid wizard Mark Howard.