Moxion to ShotGrid Entity Mapping

Overview

Moxion has the capability of pushing Moxion Assets into ShotGrid Entities. Moxion Assets contain a rich data structure, including a large set of metadata.

A Moxion Asset is a single flat object, which often needs to be restructured into multiple, hierarchical ShotGrid Entities. 

For instance, Moxion may have metadata about 'Scene', 'Shoot Day' and 'Unit' in its metadata fields, but ShotGrid may choose model this as three 'Scene', 'ShootDay' and 'Unit' high-level Entities, which are linked to a ShotGrid Asset as necessary. 

The mapping functionality is intended to handle the process of transforming a Moxion Asset into multiple, interrelated ShotGrid entities.

Key Functionality of the Mapping System

• Splits a single Moxion Asset into a collection of interlinked ShotGrid Entities, in a flexible and configurable fashion.
• Describes the ShotGrid Entities to be targeted: what they are called and how they are to be located amongst the existing data.
• Describes how the metadata fields from the Moxion Asset map into ShotGrid fields. Uses the Java Expression Language (JEXL) as a powerful, simple way to do any data transformations or cleanup.
• Describes the relationships between the different ShotGrid Entities, and how these links are to be constructed when translating Moxion to ShotGrid.
• Creates or updates ShotGrid Entities as appropriate. This only applies to Entities ('records') - this functionality will not attempt to create missing Entity Types ('tables').
• Updates the schema of an Entity Type to add any missing Fields.
• Mappings are configured on a per-project basis with a JSON mapping file.
• Allows optional upload of thumbnail images to any ShotGrid Entity.
• Allows optional upload of video clips to Version entities.

This mapping format also has provision to simply and quickly push every field in the Moxion Asset metadata into a single, flat ShotGrid Entity. 

This covers off the traditional use case of 'send Moxion Asset some nominated ShotGrid Entity designed to receive Moxion Assets'. 

This function can be used in combination with the smart mapping features mentioned above.

Example Mapping

Mappings are expressed in JSON format. 

Any field in this file can be a JEXL expression, meaning that variables and logical functions can be used where needed. 

JEXL expressions start with a dollar sign.

Note that a JSON schema is available to make creating these files easier. 

Also, this example is intended to be a short sample, and it is not very realistic or exhaustive. 

The MoxionClip Entity used in this example is fictional.

See below for a mapping example that does the following:

• Takes a Moxion Asset and - when it is pushed to ShotGrid - transforms it to a ShotGrid MoxionClip Entity.
• Looks up a MoxionClip Entity by way of ShotGrid sg_moxion_id == Moxion's Asset ID.
• If an existing MoxionClip is found, updates it. If no MoxionClip exists, create a fresh one.
• Set the fields in the ShotGrid MoxionClip as follows:
• Moxion ID (sg_moxion_id) = Moxion's Asset ID
• Code (code) = Moxion's Clip name
• Description (sg_description) = Moxion's metadata description
• This is just a short sample. In real life, the fields can be very numerous, and will probably be transformed, cleaned up or set to reasonable defaults by JEXL expressions.
• Uploads a thumbnail to the MoxionClip.
• Finally, link the 'sg_shoot_day' entity field in the MoxionClip back to an appropriate ShotGrid ShootDay entity (looked up where Shotgun ShootDay.code = Moxion's metadata shoot_day field).

Note that this file also contains a mapping for a Moxion Asset to ShotGrid 'ShootDay' Entity. 

This mapping is also run for every Moxion Asset pushed, and serves to create a ShootDay record if one does not exist (or update one if it does).

This example shows only a simple link between two ShotGrid Entities. 

In a realistic scenario, a much larger web of interrelated Entities could be created and maintained via this format. 

For instance, adding a 'Scene' and 'Take' entity could be achieved by modifying this file to add entity mappings for Scene and Take, followed by adding link definitions for MoxionClip.scene and MoxionClip.take .

Example Mapping JSON

"entityMappings": [
    {
      "entityName": "MoxionClip",           // name of the Shotgun entity  
      "uploadThumbnail": "true",            // send a thumbnail to this entity
      "key": [                              // unique key to use when checking  
                                            // for existing
        {
          "field": "Moxion ID",
          "value": "$id"
        }
      ],
      "fields": [                           // fields to update in MoxionClip
        {
          "field": "Moxion ID",             // Use field label or Shotgun name
          "value": "$id"                    // (one of many predefined vars)
        },
        {
          "field": "Moxion Link",           
          "value": "$moxionLink"
        },
        {
          "field": "code",
          "value": "$assetName"
        },
        {
          "field": "description",
          "value": "$metadata.description"
        }
      ],
      "links": [                            // link MoxionClip to a ShootDay
                                            // entity
        {
          "entityName": "ShootDay",
          "field": "shoot day",             // set MoxionClip.ShootDay
          "key": [                          // lookup criteria for ShootDay
            {
              "field": "code", 
              "value": "$metadata.shoot_day"
            }
          ]
        }
      ]
    },
    {
      "entityName": "ShootDay",            // Entity 2: ShootDay
      "key": [                             
        {
          "field": "code",
          "value": "$metadata.shoot_day"
        }
      ],
      "fields": [
        {
          "field": "code",
          "value": "$metadata.shoot_day"
        },
        {
          "field": "description",
          "value": "$'Day ' + metadata.shoot_day"
        }
      ]
    }
  ]
}

Predefined JEXL variables 

These variables may be used anywhere in the mapping file:

In addition, we have a field called 'metadata', which contains the metadata for the asset in question.

The Metadata Field

The metadata map is a list of key/value pairs, which represents all the metadata held against an asset. 

The metadata fields may vary from asset to asset, depending on what has been provided to Moxion. 

Any metadata fields may be unset (ie null).

Metadata field names provided to Moxion are all sanitized to lowercase with underscores (snake case). 

For example, a metadata field called 'Camera TC' in the ALE file will be available through the Jexl variable '$metadata.camera_tc'.

Shout out to our Shotgun wizard Mark Howard.