Workflow Automation Use Case: Secure Upload

Use the Automation Engine to check every file before uploading it onto your secure shared storage environment

Date
Author
Filip Milovanovic
Post-production expert,
ELEMENTS
Category
Workflow

The year is 2062 in the Orbit City, technology has advanced so far and has become so reliable and dependable that George Jetson’s job as a professional index operator has been reduced to the very simple task of pushing a button. However, as easy as George’s boss Mr. Spacely, thinks this job is, George actually isn’t too good at it. This of course is only an example used to highlight that humans really aren’t designed for repetitive tasks. Luckily though this is where computers shine.

Through the development of the Automation Engine, ELEMENTS has equipped every system with the tools to easily automate a number of post-production processes through an intuitive user interface. Thanks to a large offering of pre-implemented steps users can manage files on the system, transcode footage, send notifications and even execute custom Python and Bash scripts. In this blog we will showcase one interesting Automation use case, as created and used by one of our customers.

Upload-checking Automation

This creative workflow is designed to check the files that are being copied onto the production storage for expected formats and potential viruses. Depending on the status of these checks, an action will be executed, and the selected users notified of the results.
In this case, the customer manages two separate file systems with different security standards. One of these contains “normal” post-production data including files, footage, plugins and other potentially unsafe data that can be uploaded through the process of ingesting or simply being copied by the user. The other file system is considered a high-security area and is accessed by users form a different department. The management of access permissions is done through two Active Directory instances. Often, footage and other data must be copied from the less secure to the more secure file system. Furthermore, this transfer must be initiated by users that do not have access to the secure file system. Luckily, the Automation Engine has all the tools needed to accomplish this task.

Extension check

The first step to be executed upon starting the Automation job is a simple extension check of the selected files. Only certain (in this case eight different) file types are allowed. This puts rigid guidelines on the type of copied files and reduces the risk of transferring malicious software. This step is achieved by using the Python step offered by the Automation Engine and populated with the following code (please note that this is example code and may not be directly applicable to your use case):

import os
import subprocess

extensions = 'jpg,tiff,mov,mp4,dpx,png,dmg,pkg'.split(',')
bad_types = 'application/x-dosexec,application/zip,application/vnd.openxmlformats-officedocument'.split(',')

def check(path):
    if os.path.splitext(path)[1][1:].lower() not in extensions:
        raise RuntimeError('File extension not allowed for ' + path)
    type = subprocess.check_output(['file', '-i', path]).decode()
    if any(t in type for t in bad_types):
        raise RuntimeError('File type not allowed for ' + path)

if not os.path.isdir(input):
    check(input)

for root, dirs, files in os.walk(input):
    for file in files:
        path = os.path.join(root, file)
        check(path)

If one or multiple files with unpermitted file extensions are selected, an error will be raised and the files in question listed in the log. As a consequence, the format-check step will return an error and a push notification will be sent in the next step, informing selected user(s) about the failed job for the selected file. Obviously, simply checking for the extension of the file is not very secure. An additional virus scan is also required due to the fact that users can easily cheat the check by simply renaming the file extension.

Virus scan

Due to the high security nature of the receiving file system and the fact that more than just media formats are allowed to be copied, a virus scan should be performed after the extension check. The antivirus scan is realized through the integration of Sophos services into the ELEMENTS environment.After enabling this feature, a virus scan step will automatically be available in the Automation Engine. Besides adding the Virus scan step to the workflow, no other actions are needed.

Upon finishing, the Virus scan step will result in one of three possible states:
Success – the scan was completed successfully, and the scanned file is safe.
Warning – the scan was completed successfully, however, the file contains a virus recognized by the Sophos antivirus software.
Error – the scan could not be completed for the file.

Action

Depending on the outcome of the Antivirus scan, a different action will be executed. If the file is safe, this step will finish successfully and allow the files to be copied onto the secure file system. If, however the step delivers a warning or an error, the files will be moved into quarantine. Alternatively, any other step can be executed, such as repeating the scan, deleting the file etc. 

Notifications

In this Automation Job, every outcome will deliver a push notification to the ELEMENTS Client of the selected users. The user will be informed about the status of the file upload in the form of an OS notification on any workstation in the network on which the selected user is logged in. Alternatively, email notifications and even Slack messages are supported by the Automation Engine.

Job permissions

The Automation Engine offers multiple levels of user permissions. On the lowest level, users and groups can be granted starting or management permission for the Jobs, or the access to the Automation Engine can be restricted altogether. Additionally, for every individual Job, starting permissions can be further restricted to any number of chosen users.

In the Advanced tab of the Jobs settings, a security context can be specified. In the examined Automation this setting is set to root – this allows users to use the Secure Upload Job to copy files onto a file system, which they do not have access to.

Job initiation

There are four different ways to start any Automation Job:

Media Library – select any number of assets in the Media Library and start the Automation in the Jobs dropdown menu. Alternatively, the Automation can also be displayed as a button in the Media Library’s button bar, making the Job easier to find. Individual Automation Jobs can be enabled or disabled for every Media Root separately.

MacOS Finder or Windows Explorer – start Automations straight from the context menu of the macOS Finder or Windows Explorer from any workspace mounted via the ELEMENTS Client. This option should be enabled by the user in the Advanced tab of the Automation Jobs settings.

File Manager – management of an ELEMENTS system is done through an easy-to-understand, browser-based UI. Here, the File Manager section allows for quick and intuitive control over the files on the system. User can’t just copy, move, delete and compress files, but can also start Automation Jobs. Just like in the Media Library, the user can select any number of files or folders and start the Automation in the Jobs dropdown menu.

Scheduling – finally, Jobs can also be scheduled to start at fixed intervals – every hour, day, month etc. The Crontab support also allows users to further specify when to run the job, for instance, only on specific days of the week.

Do you have a cool use case for the Automation Engine that you would like to share with us? We would love to hear about it.

Workflow

Using the OpenAI Transcription Engine to Generate Subtitles

Workflow

Exploring the New Scene Cut Detection Features of DaVinci Resolve and Adobe Premiere Pro

Workflow

Automatic Transcription with DaVinci Resolve’s Speech-To-Text Function and Text-based Video Editing

Glossar

COBIT

COBIT ist ein international anerkanntes Rahmenwerk für das Management und die Governance von Informationstechnologie. Es bietet ein umfassendes Regelwerk von Prinzipien, Praktiken und analytischen Instrumenten und Modellen zur Steuerung der unternehmensweiten IT.