Bulletproof solution for 4K HDR framestack workflows

In recent years, we have witnessed a meteoric rise in video on demand platforms, that have revolutionised the way we consume content, through high-quality video productions. But the weight of these high-spec expectations unavoidably rests with the production companies, and in recent years, we have seen these companies regularly encountering one particular challenge – 4K 16-bit workflows.

Different steps in the post-production process have different requirements when it comes to the working footage. While editing for example, focuses primarily on the storyline and doesn’t necessarily demand detail-rich footage, the requirements of other departments may be much higher. Compositing is one of the processes that requires the highest possible quality footage in order to be able to produce realistic-looking visual effects. Colour grading is another notoriously demanding post-production process, which benefits form uncompressed footage with as much colour information as possible. To cater to these demands, formats such as DPX and OpenEXR are often used. These so called framestack formats provide every frame of the video as a single, often uncompressed file.

Challenges

Working with framestack formats presents three main challenges, summarised by these straightforward questions:

– How should I store the media files to avoid performance issues?

– How can I solve the transmission bottleneck in the most efficient manner?

– How can I avoid getting overrun by huge files?

Solving the transmission challenge

The obvious challenge with any video format that has been set up to preserve as much information as possible is going to be its bitrate. Every frame of a 4K UHD DPX stream is about 50 MB in size. This means that the bitrate of a 24 fps video ads up to about 1.2 GB per second. In other words, it is roughly the same as 200 XDCAM 50Mb streams. The widely adopted 10 Gbps ethernet infrastructure cannot reliably transport footage with these kinds of high bitrates.

To solve this problem, many would gravitate towards switching to a 40 Gbps ethernet connection to the clients. But this switch would be a costly and complicated one. This is due to the fact that, apart from needing new network interface cards, you would probably require new cabling.

The most efficient solution in this case is probably not the 40GbE but rather the 25GbE connection. A 25GbE easily provides the required bandwidth and when upgrading from 10GbE, users can simply reuse the existing cabling. Simply put, with 25GbE you gain the benefit of it being up to 2.5 times faster than the 10GbE connections whilst using much of the same infrastructure. At the same time, it provides greater port density, has a lower power requirement and a smaller footprint compared to 40GbE solutions. 

If you are thinking about tackling the transmission challenge by using the Ethernets best-known alternative, the Fibre-Channel, keep this in mind: Delivering this performance over standard NAS protocols opposed to the Fibre-Channel means that you can continue to enjoy benefits typical for a NAS, such as its security features, ease of maintenance and lower infrastructure cost.

Solving the storage performance challenge

Even when the connection bandwidth challenge is resolved, the issue of hard drives not being able to deliver DPX frames in a timely manner still exists. This is due to the fact that the head of the HDD has to physically visit the place on the disk on which each frame is stored. This becomes vastly more challenging if the files are scattered on the disk and also if the frames are not in sequence.

Flash-based storage is much better suited to this job and this is mainly due to the read/write speeds as well as the way in which the data is accessed. For instance, one NVMe drive can offer read and write as high as 3.5 GB per second according to tests conducted by the industry leader – Micron. For comparison, a 7200 RPM SATA drive manages around 100MB per second depending on its age, condition, and the level of fragmentation. The second reason that speaks in favour of flash-based storage is that NVMe drives have virtually no seek times – all storage blocks on it can be accessed at roughly the same speed. This means that you will never have to worry about fragmentation or have to keep frames in sequence. Here at ELEMENTS, we recognised the amazing benefits of flash technology early on  and have been integrating it into media workflows for many years now. We have used this valuable experience to develop one of the very first all-NVMe storage in the industry – the ELEMENTS BOLT.

With more than 25 GB per second of internal performance in a single 2U chassis, it is fast enough for 10 16-bit 4K DPX streams and beyond. ELEMENTS BOLT provides up to 300 Terabyte of capacity in a single 2U chassis and can be scaled to over 1.2 Petabyte in a 8U setup.

Solving the capacity challenge

Beside making a smooth playback challenging, high bitrates inevitably produce large file sizes. 10 minutes of previously mentioned 4K 16-bit DPX with a 1.2 GBps bitrate will consume about 720 GB of your storage. For a full hour, be ready to set aside about 4.3 Terabytes. When working with this kind of media, you must make sure that you have high capacity storage as well as a proper archiving workflow. The ELEMENTS BOLT offers solutions to both of these issues Create the most-efficient storage environment by seamlessly combining the NVMe-based BOLT with hard drive-based systems and profit from both extreme performance and cost-efficient storage. Our Workflow Automation Engine allows you to quickly move whole projects from one storage pool to another with just a couple clicks. Use the intuitive UI to automate your archiving to the cloud, nearline storage or cost-efficient LTO tapes, or use it to create complex custom workflows with pre-implemented steps such as transcoding, scripting, notifications and many more.

Summary

The NVMe technology offers solutions to many issues that a modern post-production facility faces. Its implementation allows your company to gain a massive competitive edge and will support your most demanding workflows for many years to come. In the last few years, the availability of flash-based storage media has grown so much, that investing in NVMe-based storage such as the ELEMENTS BOLT makes a lot of sense. Coupling it with a 25 Gigabit connection instead of the 40 Gigabit one provides your clients with the performance needed for the most demanding workflows while achieving substantial cost-saving.