Storage for Cloud-based Workflows

Tom Coughlin, Coughlin Associates, Inc.,

The outbreak of COVID-19 resulted in the cancellation of the 2020 NAB show as a physical event in Las Vegas.  Instead, various vendors who would have had exhibits and presented on digital storage systems and software for various media and entertainment applications moved to a Virtual Event, starting in late April and into June 2020, including the NAB Show Express (

In light of the move to remote work by many industry professionals, cloud-based workflows have become particularly relevant. The current quarantine has accelerated the trend to cloud-based workflows, which will likely continue even after we can work together again.  Without the cloud, most M&E professionals would be without a job.

Cloud-based workflows were growing in popularity even before the COVID-19 pandemic.  At the 2020 HPA Retreat, leading companies offering cloud workflow options as well as experienced directors, created a live cloud-based workflow short video, The Lost Lederhosen.

In my 2019 annual report on digital Storage in Media and Entertainment, I projected significant growth in cloud storage to support media and entertainment (see below[1]).  In the 2020 report the growth in cloud storage will be even higher due to increased usage after the onset of the pandemic, and based upon that experience, faster growth in remote work and cloud strorage.  The pandemic has served as an accelerator of cloud usage.


This article looks at developments for cloud-based workflows, and particularly digital storage solutions to support these workflows.  Note that although many companies who would have exhibited at the 2020 NAB are participating in virtual events, these events are spread out over time, from April through June 2020.  I will talk in this article about things I found out about at the time of writing.

Amazon Web Services (AWS) had its own NAB virtual event, focusing on remote workloads from content creation and post production through distribution.

Many M&E companies have been using AWS services including Turner, Untold, the Rock and Roll Hall of Fame, Fox, HBO, Hotstar and Eurosport.  AWS was one of 5 companies given an Engineering EMMY award for cloud0based media supply chain for content ingest, management and delivery.

AWS is offering three new services to help with latency sensitive media workloads.  These are AWS Local Zones, AWS Outposts and AWS Wavelength.  Local Zones provide low latencies by being closer to your end users with AWS services.  AWS outposts brings a rack of AWS product into your data center for a hybrid cloud experience on-premise or in the cloud.  AWS wavelength enables mobile app developers to deliver apps with single-digit millisecond latencies.

AWS is offering virtual workstations for Windows or Linux that includes access to NVIDIA T4 Tensor Core CPUs and NVIDIA Quadro workstations at the same cost.  It is also offering rendering on AWS either as hybrid or full public cloud using AWS Thinkbox Deadline or your preferred render management solution.  Both products are offered for paying as you use it.

In 2019 Fox said that it would use AWS for cable and satellite broadcasts using AWS Outposts in some of its production facilities and an AWS local zone.  The image below shows what the future of live broadcast using AWS could look as the production control room moves into the cloud.

AWS also discussed low latency content distribution using AWS Elemental MediaStore (media optimized storage and origin source).  For NAB 2020 AWS offered Elemental Live chunked transfer, DRM support and server-side Ad insertion.  The company also said that its Elemental MediaConvert and Accelerated Transcoding can make more complex AV1 encoding possible today.  The figure below shows how the AWS Elemental MediaLive, on-site media ingest box, can provide live video access.

Qumulo provides hybrid cloud file storage and data services and offers some virtual NAB videos.  The M&E industry has been one of Qumulo’s target markets.  The company announced that Adobe Premiere Pro and After Effects, in tandem with Qumulo’s file services, enable creative teams to create and edit video footage using cloud storage with the same levels of performance, access and functionality as workstations in the studio.  Qumulo’s CloudStudio allows moving projects that were traditionally hardware-bound in a physical production site to the public cloud on both the AWS and GCP platforms.

In a briefing for analysts Qumulo talked more about how Qumulo, Adobe  and Teradici can provide collaborative hybrid cloud editing as shown in the figure below.  This combination was able to provide limitless scaling, high performance collaborative video editing, visual effects and burst rendering with analytics and visibility using Qumulo analytic tools.

Quantum announced enhancements with its StorNext file system and data management software which are designed to make cloud content more accessible, with significantly improved read and write speeds for any cloud and object store. The new StorNext 6.4 features help enable hybrid-cloud and multi-cloud use cases, delivering greater flexibility for media and entertainment and other data intensive environments.

StorNext 6.4 incorporates self-describing objects to make cloud content more easily accessible, enabling new hybrid-cloud workflows. The client writes files into StorNext file system, then based on policy, StorNext copies files to the public or private cloud, with the option to include additional object metadata. Non-StorNext clients and cloud-resident processes may now access objects directly, leveraging the new extended metadata.  In addition, StorNext 6.4’s multi-threaded put/get operations provide a 5X to 7X improvement over single threaded operations.

NetApp is doing its own virtual NAB event on June 2.  Their event will showcase their solutions, with partners to offer new levels of performance and efficiency and create a media data fabric that supports M&E workflows.

Dell Technologies also had a virtual event that showcased their compute and storage tools for media and entertainment workflows as well as demonstrations with Adobe with Dell Isilon storage for enabling collaborative workflows.  Their on-demand video talked about the importance of metadata access and an Adobe expert talked about how Dell Isilon helped with Adobe’s Productions (part of Premiere) workflow.  Note that in 2020 the company has plans for Isilon server and cloud platform with OneFS.Next.  The company said in April that it expected to be able to say more in a couple of months.

Dell also had a slide talking about their data first strategy.  This is a comprehensive strategy to move data between clouds, private, multi-cloud and public clouds.

Dell is doing considerable work on IP-based workflows (SMPTE 2110) with its products.  During one of their on-line recordings an expert from IABM pointed out that M&E professionals have turned to using archives because it became difficult to get new footage with actors.  Dell works with Greymeta to provide easy access to archived data through AI generated metadata using their Iris platform.  Below is a figure from their on-line presentation that is a high-level view of Dell’s storage platform and software plans.

Avid provided online resources for doing M&E work remotely.  Marquis Broadcast is offering remote work options combining Avid Nexis storage with wasabi cloud storage (for backup) and collaborative work.

Spectra Logic is well established in the M&E industry as a provider of archive storage.  At their virtual NAB presentations they were showing advanced uses for their BlackPearl object storage gateway.  BlackPearl is the basis of Spectra’s converged storage system that includes connection to public and hybrid cloud, multi-site storage as well as HDD-based object storage and magnetic tape library storage.

Riobroker, introduced at the 2019 NAB is a data mover and connectivity engine.  This allows adding metadata and indexing with content and provides a migration engine to Black Pearl and partial file recovery.  This includes movement of data to or from a public cloud.  Adding RioBroker nodes can be added to increase availability and capacity with a global name space.  Spectra’s StorCycle allows educated management of all your connected storage assets.

Wasabi has made M&E one of its focused markets for the company’s low-cost cloud storage.  The company works with several channel partners to deliver cloud storage as part of their projects.  The company says that any 3rd party AWS S3-compatible application or platform should work with Wasabi storage.  The company says that 200+ applications are listed as Wasabi interoperable as shown below.  The company says its use of a modern purpose-built file system with leading hardware technology offers exabyte-scale storage.

The company has a single tier of high-performance storage for $5.99/TB/mo with no egress charges and no charge for API calls.  In 2020 Wasabi offered pay-as-you-go storage at $5.99 per month or reserved storage capacity from 50 TB to 10 PB in fixed increments over 1,3 or 5 years with payment up front.  The company says its storage is secure and with high durability and availability and includes Motion Picture Association of America compliance.

Wasabi has co-location facility at 1 Wilshire in LA with a 100 TB Wasabi Ball transfer appliance for easy ingest of larger amounts of content.  The company also has storage on the US East Coast as well as Europe (Amsterdam) and Asia (Japan).  In addition Wasabi is open to 1 and 10 GbE dedicated connections to their cloud storage.

Backblaze, another low-cost cloud storage company targeting the M&E space, announced it now supports the massive S3 ecosystem with the release of new, S3 Compatible APIs.

This means that content creators can easily move from other cloud storage vendors to Backblaze less expensive B2 cloud storage.  BackBlaze’s launch is supported by IBM Aspera for fast data transfers and streaming across distances and Quantum works with Backblaze for capturing, creating and sharing digital content.   Recently Backblaze said that they had more than an Exabyte of storage in their cloud.

Object Matrix provides object storage for media and entertainment applications.  The company says that “Object Matrix is focused on providing solutions that enable creative and production teams to self-serve access to content from work or remotely from anywhere.”  It is promoting self-serve access to content from on-line archives to help with remote M&E workflows and collaboration.

Editshare made its Flow remote media management platform free until July 1st in order to help creative professional working from home.  The company had a major focus on providing an on ramp to video production in the cloud at the 2020 NAB.  This included new EFS and Flow technology to support end-to-end production in the cloud with deeper integration with creative tools like Adobe Premiere Pro for collaborative workflows and practical applications of AI to enrich archives.  EFSv is a new platform offering virtualized video-editing and storage that the company says helps customers seamlessly transition from on-premise workflows to optimized remote production workflows in the cloud.

The company said that “EFS 2020 powers faster EditShare storage nodes and networks on-premise, in the cloud and in hybrid configurations. Fully compatible with Flow 2020, EFS 2020 enables media organizations to build extensive collaborative workflows, shielding creative personnel from the underlying technical complexity while equipping technical teams with a comprehensive set of media management tools.”

Scale Logic discussed its cloud-enabled NAS that can work with major post-production tools using the company’s new NVMe based NX2/ZX with onboard Sync, Backup and Archive to a local tape library or to a cloud library.

Various other companies offered cloud workflow related offerings including Masstech offering help for including cloud storage in workflows and facilitating remote editing.  The company said that this product enables remote editing with an all in one appliance for small, medium and large video production workflows.

The COVID pandemic has accelerated trends in remote media and entertainment workflows, with remote collaboration keeping M&E professionals and their employers in business.   Local storage products aren’t going away but there is a greater need to share and analyze content in order to enable remote workflows.  Cloud-based tools will become an essential part of future media projects, increasing the use of hybrid and public cloud storage.

About the Author

Tom Coughlin, President, Coughlin Associates is a digital storage analyst and business and technology consultant.  He has over 39 years in the data storage industry with engineering and management positions at several companies.  Coughlin Associates consults, publishes books and market and technology reports and puts on digital storage-oriented events.  He is a regular storage and memory contributor for and M&E organization websites.  He is an IEEE Fellow, Past-President of IEEE-USA and is active with SNIA and SMPTE. For more information on Tom Coughlin and his publications and activities go to


[1] 2019 Digital Storage in Media and Entertainment Report, Coughlin Associates, 2019,

Broadcast Beat - Production Industry Resource