Are SSDs the future of shared storage?

As resolutions creep up towards 8K, the demands on post facilities’ storage are intensifying. The data rate of 4K (UHD, in TV terms) can be anywhere from 30MBps for 25fps XAVC to 2.5 GBps for uncompressed 16-bit RGB at 50fps, and those demands are only set to increase as higher resolutions enter the mainstream. At the level of local and direct attach storage, high speed SSDs are seen as the easiest way to achieve the data rates needed – but are they up to the task when it comes to shared storage?

By Liz Sunter January 14, 2020

Before we get into this, we need to acknowledge that not all SSDs are created equal. Recent advances in the protocols on SSDs and the NAND cells they use to store data mean that latest generation of SSDs are faster, bigger and more economical than their predecessors.

“The performance of your SSD depends on three things: the type of NAND it relies on, the protocol on the SSD, and how you’re connecting to it,” said our Storage Product Specialist, Tristan Beamish. “If we take desktop storage as an example, using a SATAIII SSD and USB 3.1 Gen 1 will give you write speeds up to 510MBps. Switching to USB3.1 Gen 2 will boost that to around 560MBps. But get a next generation NVMe SSD and hook it up to your PC over Thunderbolt 3, and you’ll see speeds of up to 2800MBps. You can see a similar performance jump in Apple notebooks: the next-gen NVMe storage in the latest Apple models reads and writes data four times faster than the SATA SSD storage it replaced, and is ten times faster at locating data.”

This performance hike holds true for shared storage, too – in fact, SSDs are even faster in RAID configurations (more capacity means more NAND, which means more paths available to the controller). The drives themselves are also increasing in capacity and resiliency thanks to the arrival of 3D NAND. Denser and able to withstand more rewrites than its predecessors, it narrows that gap between the cost per bit of SSD and HDD storage, making large-scale shared SSD storage economically viable.

So, is it time to replace your entire storage setup with SSDs? Not quite…

Introducing hybrid storage

SSDs are still relatively expensive and low capacity, so many facilities (and manufacturers) take a hybrid approach, combining SSDs and HDDs to maximise performance gains (faster read/write speeds, responsivity, support for more concurrent streams) while keeping costs down.

Some solutions, like OpenDrives’ Summit or Ardis Technologies’ hybrid DDP option, combine SSDs and HDDs in a single unit. In this workflow, your data is held on the HDDs until it’s needed, at which point it’s automatically moved to the ultra-responsive SSD layer. Once you’re done, the media is returned to the HDD layer, ensuring that your fastest storage is always available for active projects.

In a more modular storage environment, like Avid NEXIS, Quantum StorNext or an EditShare EFS setup, you can purchase SSD-only modules to add to your storage. These can then act as your SSD cache, or be assigned to the specific users and groups who need them most.

Developing a hierarchal storage workflow

The key to pulling off a hybrid approach is careful, hierarchal management of your available storage. Whether you’re combining SSDs and spinning disks in a single solution, or expanding a modular system, it’s important to develop a hierarchical storage workflow in which inactive media is quickly moved from the SSD layer to the HDD layer, and then to a larger, nearline HDD array if it’s not currently in use (you could also offload inactive media to the cloud). Equally, it’s important that retrieving media and restoring it to the SSD layer is a simple, seamless process, or your editors are going to have a very inefficient and unproductive user experience.

Manufacturers are aware that hybrid solutions require more delicate handling of your offline versus nearline storage arrangements, and many have built in management tools that will help you establish this workflow and hide it from users, so they have a seamless editing experience and don’t have to worry about which storage volume they’re actually using.

What SSD storage options are on the table?

While SSDs are currently far less dominant in shared storage than they are in workstations and desktop drives, Quantum, Rohde & Schwarz, EditShare, OpenDrives, DDP and Avid all provide all-SSD or SSD-caching shared storage systems. These hybrid solutions deliver the low latency and high IOPS your end users expect, while ensuring you have the capacity needed to store high resolution footage at a reasonable price. Here are a few of our favourites…

EditShare’s EFS SSD

EditShare’s unique file system and management toolkit have always made for very efficient storage performance, and the same is true of their EFS SSD units. Specifically designed for uncompressed DPX, DNxHR 444 and ProRes 4444, EFS SSD employs a newly developed architecture to deliver SSD storage that can scale from 8TB to over 5PB. It can hit the 1.8-2.4GBps data rate required for DPX content, and speeds up the movement of files to and from client workstations by replacing SMB and AFP protocols with their high-performing, multi-thread Native Client solution.

A fully loaded, 24-drive EFS SSD unit is capable of top speeds 5500MBps, but if you’d rather start smaller, an eight drive unit can still hit 3000MBps – enough to move two 10-bit DPX files. A single unit can support a few workstations on a dedicated 10 or 40GbE network but, unusually for an SSD solution, the EFS SSD can scale up to 5PB, and EditShare suggest you use it to replace your entire online storage.

OpenDrives’ Summit and Apex

Designed to support workgroups where multiple editors, colourists and VFX artists working concurrently on projects, Summit is format, resolution and codec agnostic and can generate 11GBps aggregate throughput, while Apex is an SSD-only module designed to act as group storage for editors in hi-res, high-throughout workflows.

Ardis Technologies’ microDDP

Working in ENG or OB environments? Ardis have a DDP solution tailor made for you, combining eight SSDs of 1, 2 or 4TB to deliver a compact unit that has enough throughput to support editing during ingest. The 10GbE version can hit data rates of up to 2200MBps!

There are also SSD-only and hybrid storage modules available as part of their standard DDP range.

Quantum F-Series

Quantum’s F-Series uses NVMe flash drives to deliver extremely fast read and write speeds and a huge amount of parallel processing. Designed specifically for video editors, it uses RDMA (remote direct memory access) networking technology to allow faster transfers between workstations and the NVMe storage devices and deliver more predictable network performance.

How can we help?

Our experienced networking team can help you design, install, manage and support complex storage environments. Whether you’re expanding an existing solution or starting from scratch, we can help you to develop a hierarchical workflow that covers your online, nearline, archive and backup storage – even if it’s spread across multiple sites. We’ll ensure that your end users have a seamless working experience, and your technical team aren’t overburdened with new tools and processes. We can even host your storage in our datacentre, or help you develop a cloud first workflow. To find out more or book a demo at our Soho experience centre, get in touch using the form below.

Want to get to grips with the next generation of shared storage? Fill in the form below, or get in touch with the team on 03332 409 210 or at broadcast@Jigsaw24.com. For the latest news, follow us on LinkedIn, Facebook and Twitter.