By Adrian Herrera, Vice President of Marketing, Caringo
The proliferation of broadband, cloud and mobile devices has permanently changed the access patterns of everybody in the new digital economy. Now, storage and the assets stored are expected to be available anywhere and from any device. Oh…and no one wants to pay for it.
This is becoming a particularly challenging issue in the Media and Entertainment (M&E) space because of exponentially increasing file sizes, rapidly changing consumer viewing patterns (long-tail, VOD, cord cutting, binge watching, etc.) and the resulting expectation of content producers for assets to remain online and instantly available in perpetuity. M&E organizations that can satisfy these requests get to keep their customers and viewers. Those who can’t will ultimately lose their customers and viewers. So how do you turn the expectation of “free” storage and “instant” access from a significant risk to your business into a competitive advantage? The answer—as in many problems driven by paradigm shifts—is in the underlying infrastructure. The non-sexy storage layer.
The late Frank Zappa once said, “Change is not only necessary, it’s inevitable.” As such, disruptive technologies like object storage will become commonplace in the M&E industry in the future. Currently, the efficiencies of object storage are not being utilized because workflows, processes, and applications were designed for incumbent technologies like NAS and tape. This, however, is changing.
To understand why it’s changing, we need to understand why NAS and tape are no longer viable as the only storage technologies in the M&E data center. NAS is really too expensive and too difficult to maintain. Even when it is given away for free, on-going backup costs, maintenance costs, data center footprint and limits in scale will ultimately inhibit your ability to provide long-term storage for large digital asset libraries. This reality turned many in the M&E industry to tape, primarily as an extremely low-cost form of archival storage. But costs aside, not being able to categorize, search and instantly deliver content now puts those using tape at a competitive disadvantage.
Both NAS and tape will always have a place in the M&E data center. NAS or file-system-based technologies designed for I/O are necessary when many people are collaborating on large files on a local area network (LAN). Here, performance and file locking are critical. And, there will always be a place for low-cost tertiary backups on tape in a comprehensive preservation strategy that includes disaster recovery (DR). That said, the majority of digital assets in the near future will be stored on object storage. Recent advancements in object storage and in editing platforms, asset management and other existing applications and workflows are enabling you, the M&E IT professional, to take advantage of the many efficiencies inherent in object storage. In other words, you can now move your digital machines (aka, your internal processes) around your digital factory to take full advantage of object storage.
This leads to the question of why object storage is so different. Object storage actually combines storage, web serving, and data protection in a nice, neat software-based package. Object storage runs on commodity hardware, moving the vast majority of your asset library to a more reasonable cost curve. Proprietary hardware, backup, web serving, content delivery and operational costs are all reduced. Said another way, CAPEX and OPEX are reduced while consolidating many complex technologies onto a platform that scales to petabytes and that can encompass various locations. Historically, the challenge has been the need to develop to a proprietary API or modify existing workflows and applications. But, there have been significant advancements in supporting standard protocols like CIFS/SMB and NFS. Almost all major software vendors in the M&E space now support the S3 API. This means that you can now leverage object storage, the same technology that enables the largest clouds, secure in your data center, and often with no changes to your existing workflows or processes.
That said, not all object stores are created equal. There are a few solutions still based on file systems and therefore inherit those file system limitations. In your evaluation, you should look for the following characteristics that are indicators of a truly efficient object storage platform:
- Hard drive space and every bay in your server chassis should be used for your content. Solutions that need significant portions of hard drive for software or a certain number of drive bays for databases will ultimately be difficult to manage at scale, costing you up to 20–30% in server and hard drive capacity for the vendor’s software.
- The ability to add capacity and resources by booting servers from bare metal should also be one of your considerations. Think of how rapidly your assets are growing at 4K. Then, imagine when 8K hits a few years down the road, requiring 3 times the storage for the same length of clip. If you have to hassle with OS installs and patches to add more capacity and performance, you will be burning precious staff hours and incurring more operational costs.
- Look for the ability to Write/Read/Edit files via HTTP, S3 or NFS interchangeably, and ask your vendor to prove it. Underlying this requirement is where the actual file namespace sits. There are solutions on the market that create YOSS (yet another storage silo) rather that eliminating storage silos.
- Integrated search is also critical, with customizable metadata stored with the object. Be wary of solutions that require a metadata database. Ask any application developer with a large database about administration and protection of a database and they will undoubtedly have some horror story to share. There is a reason DBAs are hard to find.
As the pioneer in the field of object storage, the Caringo team has been doing this for over 12 years, and the M&E field is a topic that I blog on regularly.
Adrian “AJ” Herrera has over 20 years of experience bringing innovative storage, cloud and media software and services to market. With an MBA from San Diego State University and a BS in Information Systems from Chapman University, AJ serves as Caringo’s VP of Marketing.
- JVC Flexes Its Muscles For Strongman Competition Livestreams - November 22, 2021
- CONNECT ramps up its NMS SaaS offer with a new cloud-powered solution - November 18, 2021
- Chulalongkorn University Uses Blackmagic Design Workflow for Online and Streaming Classes - August 25, 2020
