Upstart NooBaa Debuts Frictionless Storage for Unstructured Data

bucket_grey overview_grey

NooBaa, Inc., an Israeli based ‘frictionless’ storage software company, recently made its formal debut at VMworld 2016. Founded in 2013 by storage and networking veterans Yuval Dimnik (CEO) and Guy Margalit (CTO), the company is currently seed funded through Israeli investors Jerusalem Venture Partners ( and Ourcrowd (, and is seeking Series A funding later this year.

NooBaa claims that its software solution, which will be available in early 2017, is an SDS (Software-Defined Storage) model dedicated initially to unstructured data, and can utilize up to many thousands of off-the-shelf servers of any type or configuration and pool their resources to provision storage capacity. The reason why the company is focused initially on unstructured data is because it already consumes most of the capacity in an enterprise datacenter, and the emerging workloads like Splunk, Hadoop, and IoT (Internet of Things) are accelerating the problem.

NooBaa has plenty of company in targeting unstructured data and retention-oriented workloads such as Splunk and general archival. Traditional storage vendors also positioning their products for these workloads include scale-out NAS (Isilon, Qumulo) as well as object storage (Cleversafe/IBM, CEPH, Caringo). Like NooBaa, some of these support the RESTful S3-compatible API for data I/O, but none have delivered the unique architecture or capabilities of the NooBaa system, as yet.

NooBaa’s ability to intelligently spread information across a heterogeneous pool of resources, dynamically adjusting distribution when nodes are added and decommissioned, means it provisions storage that promises to be reliable, resilient and secure  ̶  no matter the state of the underlying hardware.

Because NooBaa’s software-only solution is hardware agnostic, the software can provision storage by pooling the capacity locked in a large enterprise’s many hundreds or thousands of servers and desktops, both within the datacenter and out of it  ̶  devices that are largely underutilized but are a sunk capital expense.

Their elegant solution allows a datacenter with storage capacity challenges to create a secure storage system across any compute resource available, including heterogeneous, cloud, and even shared hosts. Data placement across heterogeneous capacity resources is optimized through a novel control plane architecture using machine learning technologies, which will be discussed later in this article.

With no upfront cost for storage hardware and a claimed 15-minute deployment time with near-zero configuration, NooBaa’s disruptive technology brings the ‘one click away’ attraction that has underpinned the go-to-market advantages long held by public cloud service providers.

If NooBaa’s scale-out storage solution’s claims prove true, multiple benefits await the datacenter, including:

  • The solution deploys from download to production in 15 minutes.
  • Accommodates and takes advantage of completely heterogeneous capacity resources.
  • Can seamlessly incorporate new technologies over the lifecycle without administrative intervention.
  • Can utilize shared resources across disparate hosts, playing nicely with other host workloads.
  • Can easily incorporate resources from the public cloud, multi-cloud, or multi-WAN in a single system.
  • NooBaa is the only clustered storage solution that can shrink.

The bottom-line of this innovative solution for the datacenter are improved resource utilization, and topology flexibility that delivers purportedly the lowest-TCO (Total Cost of Ownership) storage architecture. Specifically, NooBaa’s solution can always use the lowest-cost vendors and media, greatly reducing up-front time and costs, minimizing lifecycle and opportunity cost resulting from vendor lock-in. This would be a boon to IT administrators because it allows them to deliver more responsive solutions to business users, respond immediately to ‘cloud-shift’, all the while taking credit for improving datacenter efficiency.

NooBaa’s ‘Aha’ Moment

Before founding NooBaa, Dimnik, a veteran of Exanet and Dell, saw first-hand the immense costs that customers routinely paid for either evaluation or production implementation of traditional scale-out storage systems. Rigid configurations, vendor lock-in, firmware dependencies, networking sensitivities, and long learning-curves all turned storage and its management into a daunting project, even for modern SDS systems. To Dimnik, the answer seemed obvious; Cloud storage. The Cloud model promises its users numerous benefits including: instant-on, elastic up and down, pay-as-you-go, and eliminating 90% of the concern for hardware, infrastructure, networking and all the other headaches associated with on-premise storage. The ‘Aha’ moment for Dimnik was when he realized that it was possible to deliver these cloud-like attributes in an on-premise solution.

According to Dimnik, NooBaa’s SDS architecture purposely avoided current systems’ dependencies on rigid requirements around hardware configurations, performance consistency, media choices, and networking.

Instead, NooBaa’s approach takes into account the assumption that storage capacity is lying everywhere in the average datacenter, with aggregate utilization running at 20% or less. Its product offering would have to guarantee consistent performance, reliability and security across hundreds or thousands of servers and devices, all with different characteristics, located inside and outside the datacenter.

Expanding on NooBaa’s unique approach, Dimnik noted, “It’s naive to think that private cloud is just about virtualization. It should be about coordinating disparate resources to deliver QoS, regardless of whether they’re virtual, physical or containerized. Why can’t a Rasberry Pi be a viable enterprise resource? Our solution to this challenge lies in treating the whole storage ecosystem as a huge orchestration problem rather than a traditional storage approach. In doing so, we can support any capacity resource located anywhere on the network by using big data concepts like machine learning to manage data placement. Once you approach the problem in those terms, then an entirely new architecture emerges, particularly in the area of control-system design, which leads to massive benefits.”

Challenges That NooBaa Solves

The main challenge IT datacenter administrators face is how to retain and scale large data collections on a limited budget, without the lock-in and rigidity of traditional storage. Every year the industry sees new application drivers piling onto this problem. Ultimately, this is a question of reducing cost. Although media choices and sheet-metal always matter in the math, it turns out that the costs of inflexibility and lock-in are massive contributors to the TCO equation. This is the cost-savings frontier that NooBaa has addressed.

Mike Davis, Chief Marketing Officer (CMO) of NooBaa, explained how the company’s ‘intelligent’ technology addresses current storage management limitations. According to Davis, NooBaa solves the following storage and infrastructure challenges.

Lock-in: Lock-in imposes risk to the enterprise. When IT doesn’t have the flexibility to negotiate amongst vendors, and can’t take advantage of emerging technologies down the road, then they lose the ability to respond to business needs, get left behind when new technologies become cost effective, and face painful forklift upgrades in the future. NooBaa eliminates lock-in.

Dedicated resources: Existing solutions require dedicated resources for storage, rack-space, and networking. The truth is that most enterprises are horribly underutilized across their capacity landscape, with capacity ‘stranded’ in existing hosts. NooBaa finds and utilizes stranded capacity, even on shared hosts where NooBaa plays nicely with other applications. Most NooBaa customers find that deployment is a zero-capex experience, fulfilled by the bootstrapping of existing and stranded capacity.

Storage adoption is painful and expensive: Evaluating and deploying traditional storage is hard, with vendor dependencies such as mandatory installation services, lengthy debugging and integration, and weeks-long evaluation projects. NooBaa installs instantly, by the customer, with a 15 minutes-to-production design goal.

Storage infrastructure challenges: Hybrid deployments and WAN flexibility are huge challenges for existing storage solutions. With NooBaa, a node in the cloud is as good as a node in the local rack, or a node in a remote datacenter. Each has different geolocations and characteristics, but NooBaa’s advanced intelligence is designed to handle that. This means that a NooBaa customer can easily and transparently move data to and from the cloud, across multiple clouds, and across WAN resources.

True hybrid technology: Unlike other so-called “hybrid” solutions that statically provide cache-to-cloud or DR-to-cloud, NooBaa can arbitrarily consume cloud VMs and cloud storage services alongside private resources for scaling, blending, migration efforts, overflow capability, and DR. Customers may also adopt multi-cloud models, adjusting placement policies to achieve cost goals or to align data with cloud application deployments.

Optimized for Unstructured Data: The NooBaa architecture is optimized for files. Data-path design, dedupe capabilities, caching logic, metadata systems, and RESTful semantics are tuned for file applications. Splunk, enterprise archival, and media archival are excellent examples of NooBaa use-cases. Customers seeking SAN or file-system access can utilize various 3rd party gateway solutions.

Automated intelligence: NooBaa uses machine learning to autonomously classify storage resources, then intelligently direct data placement. NooBaa algorithms can optimize for reliability, capacity utilization, and performance through methods including wide-striping, preemptive data localization, and by aligning data IO (aka heat-map) with resource attributes. Customers benefit from autonomous machine learning through a massively simplified administrative experience that eliminates the need for complex administrative tasks such as the configuration of new capacity or the setup and tuning of tiering policies.

NooBaa’s Secret Sauce

Up until recently, datacenter computing and storage capacity were expanded in ‘forklift upgrades’ with expensive dedicated hardware boxes. This paradigm changed when cloud infrastructure vendors like Amazon, Aliyun, Google, Microsoft, CenturyLink, IBM, AT&T, Rackspace, Verizon and the like, developed virtualized computing systems and software-defined storage protocols  ̶  allowing them to scale massively on commodity hardware and with relatively automated administration of individual servers. However, storage for unstructured data is still largely provisioned as it has been for decades. This is a crucial limitation for smaller datacenters’ scaling abilities of unstructured data and why NooBaa’s potential solution for optimally utilizing disparate storage resources could save the M&E industry tens of billions of dollars in the years to come.

In discussing NooBaa’s ‘Secret Sauce’, CTO Guy Margalit added that NooBaa re-architected the storage stack to break apart metadata, placement intelligence, and cluster management from the storage nodes. Said Margalit, “In our NooBaa system, there is a central highly-available ‘brain’ (The NooBaa Core) that uses machine learning algorithms to control data placement across heterogeneous storage nodes. These nodes can be physical or virtual, large or small, fast or slow, with different OS, different media, using DAS or external storage. NooBaa not only can consume completely heterogeneous resources, but will actually take advantage of heterogeneity to optimize for performance, economics, and reliability.” Specifically, Margalit noted that NooBaa uses machine learning to match hot-data with high-quality resources, to push cold data to low-cost resources, and to optimize data geolocation for best application performance. In addition, the NooBaa Core actively manages resource security and trust, supports content-specific optimizations, manages extended metadata, and provides a rich management interface.

Looking deeper into NooBaa’s claim to be the most agile, lowest cost architecture ever invented, Margalit continued, “We say that not only because customers can deploy us with zero upfront expense, but because NooBaa eliminates many administrative and opex costs imposed by other SDS architectures, and allows customers to leverage capacity resources that they’ve already paid for.”

The main features that benefit users include:

  • Deploys from download to production in 15 minutes.
  • Recruits any Linux or Windows host as a storage node, virtual or physical.
  • Can leverage stranded capacity in existing hosts (e.g. underutilized Hadoop clusters, ESX hosts, etc.).
  • Can recruit any new capacity resource over the lifecycle, regardless of technology.
  • Fully hybrid and multi-cloud capable.
  • Automated DR (Disaster Recovery), tiering, balancing, and tuning based on machine learning.
  • Uses VMware ESX to host the NooBaa core control-plane and data is accessed through an Amazon S3 compatible interface.
  • Global deduplication, compression, audit, and encryption are always-on. Erasure coding is also on the roadmap.
  • Ability to pool nodes, implement per-bucket DR policies.
  • Highly reliable through data fragmentation and redundancy.
  • Highly scalable clustering and capable of high throughput performance.
  • Free license with enterprise support subscription for large-capacity customers.
  • Detects malicious activity.
  • Encryption of data-path and at rest (AES-256).
  • 2-layer data integrity checks.
  • Unlimited WAN placement options.

NooBaa’s Sales and Pricing Model:

According to Davis, the company supports customers through direct and channel sales. Interested customers can download the product directly from without going through direct sales. The company’s Community Edition is free to download and install. It supports unlimited provisioned capacity and a maximum of 20TB of files stored.

Currently, their proposed enterprise pricing structure will be about $15/TB/month, based on logical data stored (not provisioned capacity). “This is the same model as Amazon S3, but about half the price, and does not include IO charges. But we also know some customers will express a preference for a traditional license, so we’ll be able to accommodate that as well,” stated Davis.

Final Thoughts

The late Frank Zappa said it best, “Change is not only necessary; it’s inevitable.” The IT landscape is every changing and the onset of start-ups like NooBaa will have a dramatic impact on how the M&E industry and the vendors that support it evolves. According to IDC, Forrester and Bernard Golden, some likely scenarios will be: 1) Legacy vendors that don’t deliver on tomorrow’s solutions will be out of business or merged/acquired within the next five years; 2) Major cloud service providers (6 – 8) will continue to dominate as users standardize on the leaders, with the remaining providers fighting over scraps; 3) Big data will be an important area for enterprise IT; 4) Enterprises turn into software companies and IT will be leading the way; and last, 5) Developers will be in hot demand as the difference between ‘enterprise IT’ and ‘technology vendor’ will blur as both seek to implement technology solutions that form the basis of how their company operates  ̶  each battling for a limited pool of next-generation talent. By 2017, over 50 percent of organizations’ IT spending will be for 3rd platform technologies, solutions, and services, rising to over 60 percent by 2020.

If these predictions come to past, organizations already reeling from industry consolidation and technology overload, will look toward reducing hardware independence in their datacenter and its associated capex and opex costs. In this future, innovative companies like NooBaa will become pioneering enablers that will help set a new paradigm and benchmark on how we think storage should be managed.

As CEO Dimnik concluded, “We’d like to inflict a lot of pain on the traditional storage industry.”

I couldn’t agree more and recommend taking a closer look at NooBaa and their technology.


Contributing Editor at Broadcast Beat/AV Beat
Curtis Chan is President and CEO/Founder and Managing Partner of two high technology Brand Marketing and Public Relations companies, CHAN & ASSOCIATES and COGNITIVE IMPACT, whose clients range from the Global Fortune 500 to venture capital funded startups.

His agencies’ services, for over a quarter century, have helped garner early stage and growth companies significant brand exposure, resulting in either acquisition or IPO, with a total valuation of over $2.5 Billion.

Mr. Chan is a serial entrepreneur, philanthropist, author, mentor and angel investor. He played key executive roles early on in helping to pioneer and usher in both professional digital audio and digital video technologies for the recording, broadcast, cinematic and post production industries. He also played key executive roles in helping to develop and market data storage technologies. His expansive background spans four decades in ever increasing executive roles in operations, business development, engineering and sales/marketing in the information technology, media & entertainment, storage and networking, and other related high technology industries.

He is an active mentor and senior advisor to many established and VC backed start-up companies and incubators, a market/technology analyst, and has over three decades of experience in brand development, management consulting, company turnarounds, creative advertising and public relations.

A graduate of California Polytechnic State University (Cal Poly), San Luis Obispo; Mr. Chan recently ended his 14th year as Vice President for the Fullerton College Foundation; 22nd year as an Entrepreneur-in-Residence and mentor for the Small Business Institute at California State University Fullerton and incubators. Additionally, he is a volunteer member of DECA (Delta Epsilon Chi and Distributive Education Clubs of America); an advisor to the College of Engineering's Global Waste Research Institute, Cal Poly, San Luis Obispo and a member of the President’s Green and Gold Society; and advisory council member to Pepperdine University's Most Fundable Companies program. Passionate about hospice care, he was the past Board President for the Healing Hearts Association; and is a regularly featured speaker at many MBA and Doctorate level lectures around the country. He has presented and published over 30 papers worldwide, is a book co-author, an Editorial Board Member for the Society of Motion Picture and Television Engineers, past Senior Editor for Broadcast Beat and AV Beat Magazines, Computer Technology Review, and is a regularly featured Contributing Editor to many US and international trade publications for over 35 years.

View Cognitive Impact services at Curtis can be reached at; Office: (714) 620-4993
Curtis Chan
Broadcast Beat - Production Industry Resource