Today, cloud storage already plays an important role in many media workflows, especially in relation to use cases when enabling worldwide collaboration and archiving footage. When it comes to cloud archiving, one of the biggest challenges is the balancing act between the cost and the data retrieval time. Simply put, the user wants to store the footage as cheaply as possible, but at the same time still be able to retrieve it quickly. Until now, this wasn’t an easy thing to do as the user would usually only have the choice between two kinds of different storage classes. The S3 Standard storage class, which is designed for frequently accessed data due to its low latency and high throughput, and the S3 Glacier Deep Archive which is the lowest-cost storage class, designed for long-term data retention. Though Glacier seems like an obvious choice for archiving footage, its retrieval time of several hours and sometimes even days makes it impractical for many media use cases. The newest AWS storage class may very well be the solution we’ve all been waiting for.
Amazon S3 Glacier Instant Retrieval
Announced at the beginning of December 2021, this storage class has been specifically created with media workflows in mind. It offers a cost-effective cloud storage option for rarely accessed, long-lived data that requires milliseconds retrieval. Though the retrieval time is the same as the S3 Standard-IA, Amazon claims that it can be up to two-thirds cheaper:
„With S3 Glacier Instant Retrieval, you can save up to 68% on storage costs compared to the S3 Standard-Infrequent Access (S3 Standard-IA) storage class, when you access your data once per quarter.“
Keep in mind that “when you access your data once per quarter” is probably the important part, as it is the access that will almost certainly produce the bulk of the cost. Therefore, the Glacier Instant Retrieval class is well suited for media workflows where large amounts of footage needs to be stored but rarely and infrequently accessed. This makes it well suited for all different kinds of media archives such as raw footage, stock footage and finished projects. One more use case that requires large amounts of long-term storage space that may need to be accessed quickly is backup. The Glacier Instant Retrieval class might be a good solution for an off-site backup, especially for the smaller facilities that may not have the required independent site.
All in all, Amazon has quite a compelling offer of cloud storage classes. However, no matter which class you choose, you can only expect an appropriately named “Bucket” with very limited capabilities. Making this cloud storage usable in your day-to-day workflows is where ELEMENTS truly shines.
Integration
Connecting the ELEMENTS environment with your Amazon cloud is as easy as entering your AWS credentials in the Integrations tab of the ELEMENTS Web UI. This process requires no technical knowledge or additional licenses. Clicking the Save button will automatically enable the cloud features and add them to the ELEMENTS tools. This integration means that you don’t have to hand out cloud access credentials to anyone and can remain in full control of all files that are being uploaded.
Automation Engine
The ELEMENTS Automation Engine is a powerful tool that allows you to drastically increase productivity in your media workflows, all while approving accountability and minimising human error. Build automations with any number of available tasks, such as: transcoding, archiving, Media Library control, e-mail and Slack notifications or even run custom Python and Shell scripts.
Upon connecting your ELEMENTS environment to your AWS account, the Automation Engine will be expanded with four new steps:
- AWS copy – With this step, selected items can be copied into the chosen destination in the cloud. If wanted, a list of files and folders can be excluded from the input and additional options selected.
- AWS delete – Use this step to delete files and folders located in the AWS cloud.
- AWS sync – With this step, selected files or the content of selected folders will be synched with the cloud destination folder. If required, a list of files and folders can be excluded from the input and additional options selected.
- AWS SNS message – Use this step to send an automated AWS SNS message (Amazon Simple Notification Service). This functionality allows you to send application-to-application (A2A) and application-to-person (A2P) AWS notifications.
The ELEMENTS Automation Engine allows you to precisely control which users or groups are allowed to start which automations. Automations can be scheduled to run in the chosen interval or be triggered from the Media Library or even from the context menu of the macOS Finder or Windows Explorer. This approach allows you to use your AWS cloud in a more intuitive and secure manner.
To better understand how this flexible tool can help your workflows check out the Consolidate Media into Cloud automation that has been created by one of our customers. When a user triggers this clever automation with any Premiere Pro project, all the media used in it, along with the project itself, will be uploaded into the cloud and the editor will be notified. This automation speeds up and simplifies the media consolidation process greatly.
Media Library
The Media Library, our browser-based Media Asset Manager (MAM) is a true game-changer when it comes to an intuitive and efficient usage of AWS S3 cloud storage. Not only does it provide users with a graphical interface for effortless team-collaboration and media management, but it also allows you to archive footage to AWS S3 with a simple mouse click, all while enjoying the following valuable benefits:
- Preview the footage located in the AWS cloud – After a clip has been moved to the cloud, a lightweight proxy preview file remains in the Media Library. This allows you to screen the full clip and decide if it needs to be restored. This functionality can greatly reduce the costs associated with cloud data retrieval.
- Create editing timelines with archived footage – The integrated Rough-cut Editor allows you to edit footage even before it has been restored from the cloud. This means that you can outline the story and see exactly which clips need to be restored. After the footage has been retrieved from the cloud, the high-res version of the created sequence can be exported directly from the editor or simply imported into Adobe Premiere Pro, Avid Media Composer or DaVinci Resolve.
- Use the collaboration functions on the archived footage – Users can add comments, tags, custom fields and use all other collaboration features on the archived footage.
- Use the powerful search function to find the archived footage – The granular search function of the Media Library allows you to combine multiple search criteria to find just the right footage. Search by clip name, added comments, creation date, editing project, tags and other metadata. This allows you to build an indexed archive and lets you maintain an overview of all the archived files.
- Retrieve footage with a single click of a button – Restoring footage is as easy as archiving it. Simply select any number of clips to be restored and start the fully automated retrieval process. If required, you can be informed about the progress via email, client notifications or even Slack messages.
Conclusion
The recent announcement of the Amazon S3 Glacier Instant Retrieval storage class opens up new possibilities for utilizing cloud storage for media archive, backups and other purposes requiring rare, yet immediate access. This cloud storage class enables users to store large amounts of data cost-efficiently without having to sacrifice the file retrieval speed – perfect for media workflows.
However, an efficient cloud workflow requires more than just cloud storage. It requires a platform that unifies all steps of your workflow – This is exactly what ELEMENTS tools is capable of. The ELEMENTS Media Library is a powerful collaboration platform that helps you get the most out of your AWS S3 cloud storage. It allows you to build a well-organised cloud-based media archive with capable collaboration tools while at the same time allowing you to greatly decrease the costs associated with cloud data retrieval.
