The Promise of Enterprise Hybrid Cloud File Storage - FILE STORAGE AT THE CROSSROADS - Qumulo
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
WHITEPAPER The Promise of Enterprise Hybrid Cloud File Storage FILE STORAGE AT THE CROSSROADS August 12, 2019
WHITEPAPER
Table of Contents
Executive Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
A Perfect Storm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Scale-Across File Storage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Re-Thinking The File Storage Industry. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
The Limitations Of Legacy Storage Solutions.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
The Challenges of Object Storage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Inadequacy of Cloud-Based File Solutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Qumulo Enterprise-Proven Hybrid Cloud Storage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
How Qumulo Works. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
The Qumulo File System.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Real-Time Analytics.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Real-Time Quotas.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Audit.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Snapshots.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Continous Replication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Scalable Block Store (SBS). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
The Promise of Enterprise Hybrid Cloud File Storage 2
WHITEPAPER
Executive Summary
Large-scale file storage has reached a tipping point. The amount of unstructured
data has been growing steadily, and its growth has accelerated to the point that
companies wonder how they will be able to manage the ever-increasing scale of their
digital assets. In addition, the global reach of public clouds are creating new demand
for the mobility of file data.
As a result, new requirements for file storage are emerging for enterprises,
underscoring the need for a scale-across file storage system. Such a system would
have no upper limit on the number of files it could manage, regardless of size, and it
would be able to run anywhere, on-prem or in the cloud, or both.
Qumulo offers an enterprise-proven, highly scalable, hybrid cloud file storage system
that can span the data center and the cloud. It scales to billions of files, costs less,
and has a lower TCO than legacy storage solutions. Qumulo also provides the highest
performant file storage system on-prem and in the cloud. With built-in real-time
analytics, administrators can easily manage data no matter the file size or where it’s
located globally.
Qumulo’s continuous replication enables data to move where it’s needed, when it’s
needed; for example, between on-prem clusters and clusters running in the cloud,
or between cloud clusters. Qumulo’s software runs on Intel® Xeon® Gold based
industry-standard hardware and was designed from the ground up to meet today’s
requirements for scale. Qumulo offers the world’s first scale-across file storage
system, allowing modern enterprises to easily store and manage files numbering in
the billions, in any operating environment, anywhere in the world.
A Perfect Storm
IDC predicts that the amount of data deployed in public clouds, private clouds, and
on-prem for file services is expected to reach 45.5 exabytes, 10.6 exabytes and 57.3
exabytes by 2022 respectively.1 An exabyte is a million terabytes. To put that in
perspective, you can store 341 billion three-minute MP3s in an exabyte - that is a lot
of music.
One drop of human blood
creates enough data to fill Machine-generated data, virtually all of which is file-based, is one of the primary
factors behind this dramatic acceleration of data growth. Life sciences researchers,
an entire laptop computer,
for example, who are developing the latest medical breakthroughs, use vast amounts
and some research projects of file data for genome sequences and share that data with colleagues around the
require a million drops. world. Oil and gas companies’ greatest assets are their file-based seismic data used
for natural gas and oil discovery. Every movie and television program we watch is
produced on computers and stored digitally as files. Text-based log files—data about
machines, created by machines—are proliferating at an ever-increasing rate. The
increasing need for security and safety monitoring has caused video surveillance
cameras and security devices to be pervasive across many public and private
organizations, resulting in an extraordinary amount of unstructured data.
1
Worldwide File-Based Storage Forecast, 2018–2022: Storage by Deployment Location, IDC December, 2018
Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.
The Promise of Enterprise Hybrid Cloud File Storage 3
WHITEPAPER
There is also a trend toward higher-resolution digital assets. Uncompressed 4K
and 8K video is the new standard in media and entertainment. The resolution for
video and images created by digital sensors and scientific equipment is constantly
increasing. Higher-resolution causes file sizes to grow more than linearly. By
doubling the resolution of a digital photograph, it’s size increases by four times.
As organizations demand more fidelity from digital assets, storage requirements
continue to grow.
The continued rise in data volumes is paralleled by the advent of the public cloud. Its
arrival overturned many basic assumptions about how storage should work.
The rise of the public cloud signalled that compute resources and global reach were
now achievable without building data centers across the world. Consequently, new
ways of working have arrived and are here to stay. All businesses realize that, in the
future, they will no longer be running their workloads out of single, self-managed data
centers. Instead, they will be moving to multiple data centers, with one or more in the
public cloud, or completely in the cloud. This flexibility will help them adapt to a world
with geographically-dispersed employees and business partners. Companies will
focus their resources on their core business lines instead of on IT expenditures. Most
will improve their disaster recovery and business continuity plans, and many will do
this by taking advantage of the cloud.
Users of legacy scale-up and scale-out file systems, long considered workhorses
of file data, find that those systems are often inadequate for a future shaped by
tremendous amounts of unstructured data. A core part of this problem is that the
metadata within large file systems—their directory structures and file attributes—has
itself become unmanageable.
Legacy solutions often rely on brute force to provide insight into the storage system,
and brute force has been defeated by scale. For example, tree walks - the sequential
processes that scan nested directories as part of routine management tasks - have
become computationally infeasible. Brute force methods are fundamental to the way
legacy file systems are designed and cannot be fixed with patches.
Against this backdrop of profound change, file storage users still need to maintain and
safely manage large-scale, complex workflows that rely on collaborations between
many distinct software programs, operating systems, and individuals. Moreover, the
traditional buying criteria of price, performance, ease-of-use, and reliability remain as
important as ever, no matter how much the landscape has changed.
The storage industry finds itself at a crossroads, which includes both new challenges
and new opportunities. Without innovation among storage providers, users of large-
scale file storage will continue to struggle to understand what is going on inside their
systems. They will struggle to cope with massive amounts of data. They will struggle
to meet the demands for global reach, with few viable options for file data that span
both the data center and the cloud.
The Promise of Enterprise Hybrid Cloud File Storage 4
WHITEPAPER
Scale-Across File Storage
Traditionally, companies face two problems when deploying file-based storage
systems: they need to scale both capacity and performance simultaneously. In the
world where the growth of unstructured data is unrelenting, scale is no longer limited
Today, capacity means to these two axes. New criteria for scale have emerged, including the number and size
more than terabytes of of files stored, the ability to control enormous amounts data in real-time, to distribute
raw storage. data globally, and the flexibility to leverage on-prem, hybrid, or cloud deployments.
These requirements define a new market category called scale-across file storage.
Scale-across file storage scales to billions of files. The notion that capacity is only
measured in terms of bytes of raw storage is giving way to a broader understanding
that capacity is just as often defined by the number of files that can be stored. Modern
file-based workflows include a mix of large and small files, especially if they involve
any amount of machine-generated data. As legacy file systems reach the limits in the
number of files they can effectively store, buyers can no longer assume that they will
have adequate file capacity.
Scale-across file storage works across operating environments, including on-
prem data centers, as well as private and public clouds. Proprietary hardware is
increasingly a dead end for users of large-scale file storage. Today’s businesses need
flexibility and choice. They want to store files in data centers, in private clouds and/or
public clouds, opting for one or the other based on business decisions rather than on
the technical limitations of their storage platform.
The Promise of Enterprise Hybrid Cloud File Storage 5
WHITEPAPER
Companies want to benefit from the rapid technical and economic advantages of
standard hardware, such as denser drives and lower-cost components. They want
to reduce the complexity of hardware maintenance through standardization and
streamlined configurations. The trend of software-defined storage on standard
hardware will only continue. Users responsible for large amounts of data require their
storage systems to run on a variety of operating environments and not be locked in to
proprietary hardware.
Scale-across file storage scales across geographic locations with data mobility.
Today’s businesses are global. Their file-based storage systems must now scale
across geographic locations. This may involve multiple data centers, private clouds
and almost certainly public clouds. A piece-meal approach and a label that says
“cloud-ready” simply won’t work. True mobility and global reach are now required.
Scale-across file storage provides real-time visibility and control. As the
number of files being managed today has grown to billion-file scale, the ability to
control storage resources in real-time has become an urgent requirement. Storage
administrators must be able to instantly monitor all aspects of system performance
and capacity, regardless of the size of the storage system.
Scale-across file storage gives access to rapid innovation. Modern file storage
needs a simple, elegant design and advanced engineering. Companies that develop
scale-across file storage will leverage Agile development processes that emphasize
rapid release cycles and continual access to innovation. Three-year update cycles, a
result of cumbersome “waterfall” development processes, are a relic of the past that
customers can no longer tolerate.
As the needs of lines-of-business surpass what central IT can provide in a reasonable
time frame, accessing cloud-based resources has become a requirement. A flexible,
on-demand usage model is a hallmark of the cloud. However, the shift to cloud has
stranded users of large-scale file storage, who often have no effective way to harness
the power that the cloud offers. A file system that is enterprise-proven and can scale-
across to the cloud is required.
Re-Thinking The File Storage Industry
Legacy scale-up and scale-out file systems are not capable of meeting the emerging
requirements of managing storage on-prem and/or in the cloud at scale. The
engineers who designed them 20 years ago never anticipated the number of files and
directories, and mixed file sizes, that characterize modern workloads. They could also
not foresee cloud computing.
The Promise of Enterprise Hybrid Cloud File Storage 6WHITEPAPER
THE LIMITATIONS OF LEGACY STORAGE SYSTEMS
Qumulo often hears from organizations that they’re in a scalability crisis as the
growth of their unstructured data is rapidly outpacing the design assumptions of
their existing storage solutions. These legacy solutions are difficult to install, difficult
to maintain, and inefficient. Putting in one of these systems is usually a service
engagement that can take a week, assuming the person doing the installation is
experienced. These systems often have many inherent limitations, such as volume
sizes and the number of inodes (inodes store the attributes and disk block location of
the object’s data), which interact and make it challenging to avoid bottlenecks.
Legacy systems are expensive, and their inefficiency adds even more to their total
cost. Generally, only 70 to 80 percent of the provisioned storage capacity is actually
available. System performance suffers if the disk gets any fuller. Another problem is
that legacy systems were not designed for the higher drive densities that are now
available. Rebuild times in the event of a failed disk can stretch into days.
Finally, traditional storage systems offer no visibility into an organization’s data.
Getting information about how the system is being used is often clumsy and
slow. It can take so long to get the information that it is outdated even before the
administrator sees it.
THE CHALLENGES OF OBJECT STORAGE
Object storage allows for very large systems with petabytes of data and billions of
objects, and works well for its intended use. In fact, it was the default that object
storage technologies were the solution to the scale and geo-distribution challenges of
unstructured storage. Cloud providers believed wholeheartedly in object storage.
Adopting object storage in use cases for which it was never intended is a poor
technical fit. In order to achieve their scale and geo-distribution properties, some
object stores have intentionally traded off features many users need and expect,
including transactional consistency, modification of objects (e.g. files), fine-grained
access control, and use of standard protocols such as NFS and SMB, to name a few.
Object storage also does not handle the problem of organizing data. Instead, users
are encouraged to index the data themselves in some sort of external database. This
may suffice for the storage needs of stand-alone applications, but it complicates
collaboration between applications, and between humans and those applications.
Modern workflows almost always involve applications that were developed
independently but work together by exchanging file-based data, an interop scenario
Object stores have that is simply not possible with object storage.
intentionally sacrificed
A surprising amount of valuable business logic is encoded in the directory structure
features users need of enterprise file systems. The need for file storage at scale remains compelling.
and expect. Qumulo’s software provides the scalability benefits of object without sacrificing
features.
The Promise of Enterprise Hybrid Cloud File Storage 7WHITEPAPER
INADEQUACY OF CLOUD-BASED FILE SOLUTIONS
While there is tremendous demand for running file-based workloads in the cloud,
There are no mature existing solutions for unstructured file management in the cloud are often inadequate.
These solutions are either sold by the cloud providers themselves or by legacy
products among existing
storage vendors. In the first case, the solutions are immature. In the second, they
cloud-based file solutions. apply 1990’s technology to 21st century problems.
For example, cloud-only file systems are limited by the fact that they don’t connect
with a company’s on-prem data center in any way. Further, they lack important
enterprise features, such as support for the Server Message Block (SMB) protocol,
quotas, snapshots, replication and audit that are needed for modern file-based
workflows in data-intensive industries.
The efforts of legacy storage vendors to pivot to the cloud have resulted in solutions
with limited capacity and limited scalable performance. This inflexibility negates the
very reason businesses are turning to the cloud - the ease of adding more compute
power.
None of the legacy solutions provide real-time visibility and control of the data in the
cloud, which leads to over-provisioning of capacity, performance, or both. In general,
current solutions for file storage in the cloud are piecemeal approaches that address
only parts of the problem. Customers are stranded in their attempts to integrate file-
based workloads with the cloud.
Qumulo’s Enterprise-Proven Hybrid Cloud Storage
Qumulo was founded in 2012, as the crisis in file storage was beginning to reach its
tipping point. A group of storage pioneers, the inventors of scale-out NAS, joined
forces and formed a different kind of storage company, one that would address these
new requirements head-on. The result of their work, and of the team they assembled,
is Qumulo, which developed the world’s first scale-across file storage system.
Qumulo’s enterprise-proven, hybrid cloud file storage system spans the data center,
the private clouds and/or public clouds. It scales to billions of files, costs less, and
has a lower TCO than legacy storage solutions. It is also the highest performance
file storage system on-prem and in the cloud. Real-time analytics let administrators
easily access and manage data regardless of size or location. Qumulo’s continuous
replication enables data to move where it’s needed, when it’s needed; for example,
between on-prem clusters and clusters running in the cloud or between clusters
running on different cloud instances.
The Promise of Enterprise Hybrid Cloud File Storage 8WHITEPAPER
With Qumulo’s file storage, cloud instances or computing nodes with Intel® Xeon®
Gold based standard hardware work together to form clusters that provide scalable
performance a single, unified file system. Qumulo clusters work together to form a
globally distributed, highly connected, storage solution tied together with continuous
replication.
Qumulo clusters work
together to form a
globally distributed but
highly connected storage
solution tied together with
continuous replication.
Customers interact with Qumulo clusters using industry-standard file protocols such
as NFS and SMB, the Qumulo REST API and a web-based graphical user interface
(GUI) for storage administrators. Below is an example of the GUI.
Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.
The Promise of Enterprise Hybrid Cloud File Storage 9WHITEPAPER
Qumulo’s software has a unique ability to scale. Here are some of the capabilities that
set our file system apart from legacy file storage solutions.
Qumulo scales to billions of files. With Qumulo, you can use any mix of large
and small files, and store as many files as you need. There is no practical limit with
Qumulo’s advanced file system. Many of Qumulo customers have data in excess
of a billion files. This is in stark contrast to legacy scale-out storage systems which
were not designed to handle modern workflows with mixed file sizes, which become
very inefficient when there are many small files. This is because these legacy
systems are based on a decades-old design that forces them to mirror (or double
mirror, sometimes even triple mirror) files under a 128KB threshold. Qumulo is
vastly more efficient at representing and protecting small files than legacy scale-out
NAS, typically requiring one-third of the storage capacity and half of the protection
overhead.
We developed a fundamentally different approach to data protection, protecting at
the block level versus the file level. Working at the block level rather than the file level
using our custom erasure coding makes it possible to protect data effectively without
having to create a one-to-one copy of the entire data volume.
Qumulo provides the highest performance. Qumulo is the highest performance
file storage system whether on-prem and/or in the cloud. It provides twice the price
performance compared to legacy storage systems. In the data center, Qumulo’s file
system is optimized for Intel® Xeon® Gold based standard hardware with Intel®
SSD Data Center Family for NVMe, SSDs and HDDs, which cost less than proprietary
hardware. In the cloud, Qumulo’s software intelligently trades off between low-
latency block resources and higher-latency, lower-cost block options.
“Our research organization falls between the cracks for most storage vendors, with
giant imaging sets and millions of tiny genetic sequencing scraps. Finding a system
that reasonably handled all our complex workflows was difficult, and in the end only
Qumulo was the right fit.”
— Bill Kupiec, IT Manager, Department of Embryology, Carnegie Institution for Science
Qumulo has lower cost. Qumulo’s file system costs less and has a lower TCO
than legacy storage solutions on a capacity basis, as measured by cost-per-usable
terabyte. Qumulo’s cost advantage comes from its efficient use of storage capacity
and its use of Intel® Xeon® Gold based standard hardware.
Qumulo’s cost efficiencies also make it extremely reliable. Storage system reliability is
usually measured in terms of mean time to data loss (MTTDL). MTTDL is the average
number of years a given cluster will survive before there’s a hardware failure that
causes a significant loss of data. At a minimum, MTTDLs should be measured in the
tens of thousands of years.
While some variables that affect reliability can’t be controlled by the storage system,
one that can is the reprotect time, or how long it takes to recover data if a disk fails.
Reprotect times matter because the longer it takes to reprotect the cluster, the more
vulnerable the cluster is to other failures and the poorer the MTTDL. As disks become
denser, data volumes increase, and clusters grow, a legacy storage system’s reprotect
times can turn into weeks.
Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.
The Promise of Enterprise Hybrid Cloud File Storage 10WHITEPAPER
Qumulo uses sophisticated data protection techniques that enable the fastest
reprotect times in the industry. They are measured in hours, not days or weeks.
When reprotect times are fast, reliability increases. Better reliability means that
administrators can greatly reduce the level of redundancy they need to achieve target
MTTDL standards, which in turn increases storage efficiency and lowers cost.
“With critical high-profile projects, you want to know exactly what you’re going to be
leaning on for successful delivery. When the ‘La La Land’ project came around it was
make or break, and we were never down for a moment. Qumulo is our rock, allowing
us to focus on the visual effects with absolute confidence that the data is safe.”
— Tim LeDoux, Founder/VFX Supervisor, Crafty Apes
Qumulo also takes into account the drop in performance that occurs when a disk
failure happens and needs to be rebuilt. Qumulo, with I/O Assurance, automatically
adjusts all users’ performance so no one person or application experiences a
significant performance degradation.
Qumulo makes 100 percent of user-provisioned capacity available for user files,
in contrast to legacy scale-up and scale-out NAS that only recommend using 70 to
80 percent to ensure consistent performance. In addition to this 20 to 30 percent,
legacy vendors often require additional capacity reserved for data protection or for
administration. Further, Qumulo’s software can safely run at 2-drive protection where
others require 3-drive protection given our leading-edge restripe and rebalance
performance. The difference between 2-drive and 3-drive protection can be up to 15
percent of raw capacity. Certain vendors also have a “small file tax,” where managing
small files less than 128K in size adds to the problem of not being able to use all of
your storage.
Qumulo has real-time analytics that tell you what’s happening in your file
system instantly. Analytics is an integral part of the Qumulo file system; it is not an
afterthought. Instead of running multiple commands, parsing through pages of log
files, and running separate programs, an administrator can simply look at the GUI and
understand what’s happening. For example, an administrator can immediately see if a
process or user is hogging system resources and, in real-time, apply a capacity quota.
Qumulo gives you the freedom to store and access your data anywhere. Qumulo
is hardware-independent and can run both in the data center and/or in the cloud,
while still offering the same interface and capabilities to users, whether they are on-
prem, off-prem, or spanning both. Administrators have the freedom to take advantage
of the compute resources that the cloud offers, and then move data back to their data
centers as needed.
Qumulo has industry-leading support. Many storage customers are dissatisfied
with the support they receive from their vendors. They find them to be unresponsive
and reactive rather than proactive. Qumulo offers responsive, personal, customer
support, with one of the highest Net Promoter Scores (NPS) in the industry.
Qumulo has simple subscription pricing. Businesses often feel they are being held
hostage by the high cost of their existing storage solutions. If they want to upgrade
their hardware after three years, they’re forced to throw out software licenses
associated with their legacy hardware. Even if they wish to run their storage systems
The Promise of Enterprise Hybrid Cloud File Storage 11WHITEPAPER
for seven years instead of three, their vendor forces them to replace their hardware by
way of exorbitant support quotes. Pricing is complicated and figuring out how much
a system will cost is far from straightforward. In contrast, Qumulo’s pricing is based
on a single, simple subscription service that covers everything, including software,
updates and support.
“For a critical digital media archive, Qumulo is the safest place I can think to put it,
short of directly in a backup vault. Soon we won’t need anything else but backup,
high-speed virtual storage, and Qumulo.”
— Joel Hsia, Assistant Head for Systems Development, Marriott Library, University of Utah
Qumulo provides cloud-based monitoring and trends. A Qumulo software
subscription includes cloud-based monitoring that proactively detects potential
problems, such as disk failures. Administrators can also access the Qumulo trends
service, which provides historical data about how the system is being used. This
information can help lower costs and optimize workflows.
Qumulo provides access to innovation. Qumulo follows Agile and other modern
It turns out only 14 percent development practices, which means it has many small releases that steadily improve
of B2B companies have the product and keep it on the leading edge of what’s possible. This is in contrast
a customer-centric to legacy storage vendors that have infrequent releases that can keep customers
waiting years for improvements.
culture. Qumulo is all
about customer feedback Qumulo has no hardware lock-in. Qumulo uses Intel® Xeon® based standard
and constantly evolves hardware provided by Qumulo or by partners such as HPE and Dell. In the cloud,
our file system can use a range of instances within AWS or GCP that you can pick
its offerings to match according to your capacity and performance requirements.
customer needs. Forbes,
August 20192
Qumulo’s Intel® Xeon® based hardware platforms ensure that you can get the
perfect solution for your needs. Qumulo’s NVMe-based system provides capacity and
sustained performance, the hybrid SSD/Disk-based system provides the performance
of flash at the price of disk, and the active archive solution provides incredible density.
Qumulo provides a fully programmable REST API. Customers get programmatic
access to any feature or administrative setting in Qumulo. The Qumulo REST API is
built for developers. The API is suitable for DevOps and Agile operating approaches,
which are how modern application stacks are constructed and managed, particularly
in the cloud. For example, you can use tools such as Terraform and CloudFormation to
automatically spin-up Qumulo clusters in the cloud.
“Managing data with Qumulo is so simple it’s hard to describe the impact. It has given
us tremendous ROI in terms of time saved and problems eliminated, and having that
reliable storage we can finally trust makes us eager to use it more broadly throughout
the company.”
— John Beck, IT Manager, Hyundai MOBIS
2
“100 Of The Most Customer-Centric Companies”, Blake Morgan, Forbes, June 30, 2019
Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.
The Promise of Enterprise Hybrid Cloud File Storage 12WHITEPAPER
Qumulo offers out-of-the-box simplicity. It might seem obvious that storage
administrators want a system that is easy to install and easy to manage. They have
better ways to spend their time. Unfortunately, legacy storage systems can take
days to set up and configure. For data center installations, Qumulo’s file system is
extremely simple to install. Once the nodes are racked and cabled, all an administrator
has to do is sign the end-user license agreement, name the cluster, set up an admin
name and password, and perhaps enter some IP addresses. Installation is painless.
“Qumulo allows us to move file-based data sets to a Qumulo cluster on AWS, complete
our analysis, and move the artifact back to our on-prem Qumulo storage cluster, saving
us time and money. The flexibility for us to move our file-based data where we need it to
be is something that nobody else in the market can provide at scale.”
— Tyrone Grandison, CIO, Institute for Health Metrics and Evaluation (IHME)
From the moment Qumulo’s software is unboxed to when it can start serving data is
a matter of hours, not days. It is also extremely easy to create a Qumulo cluster in the
public cloud.
“Qumulo customer care is absolutely phenomenal – the best support I’ve seen from any
vendor. It’s been a real pleasure to deal with Qumulo.”
— Nathan Larsen, Director of IT, Sinclair Oil Corporation
The Promise of Enterprise Hybrid Cloud File Storage 13WHITEPAPER
How Qumulo Works
Qumulo is a new kind of storage company, based entirely on advanced software
and modern development practices. Intel based industry standard hardware running
advanced, distributed software is the basis of modern, low-cost, scalable computing.
This is just as true for file storage at large scale as it is for search engines and social
media platforms.
Qumulo’s file system is unique in how it approaches the problems of scalability. Its
design implements principles similar to those used by modern, large-scale, distributed
databases. The result is a file system with unmatched scale characteristics.
THE QUMULO FILE SYSTEM
For massively scalable files and directories, Qumulo’s file system makes extensive
use of index data structures known as B-trees. B-trees minimize the amount of
I/O required for each operation as the amount of data increases. With B-trees as a
foundation, the computational cost of reading or inserting data blocks grows very
slowly as the amount of data increases.
REAL-TIME ANALYTICS WITH QUMULO
When people are introduced to Qumulo’s real-time analytics and watch them perform
at scale, the first question is usually, “How can it be that fast?”. The breakthrough
performance of Qumulo’s analytics is that it continually maintains up-to-date
metadata summaries for each directory. It uses the file system’s B-trees to collect
information about the file system as changes occur. Various metadata fields are
summarized inside the file system to create a virtual index. The performance analytics
that you see in the GUI, and can pull out with the REST API, are based on sampling
“We use the same Agile methodology at Sinclair, and I’ve seen first-hand its ability to
drive good products into production so much faster than with traditional 18-month
monolithic releases. Given Qumulo’s existing lead on its competitors, I knew that fast
development pace would help keep it out in front of our needs.”
— Nathan Larsen, Director of IT, Sinclair Oil Corporation
The Promise of Enterprise Hybrid Cloud File Storage 14WHITEPAPER
mechanisms that are enabled by Qumulo’s metadata aggregation. In contrast,
metadata queries in legacy storage appliances are answered outside of the core file
system by an unrelated software component.
REAL-TIME QUOTAS
Just as real-time aggregation of metadata enables Qumulo’s real-time analytics, it also
enables real-time capacity quotas. Quotas allow administrators to specify how much
capacity a given directory is allowed to use for files.
Qumulo’s quotas are deployed immediately and do not have to be provisioned.
They are enforced in real-time, and changes to their capacities are immediately
implemented. Quotas can be specified at any level of the directory tree.
AUDIT
Qumulo’s auditing capability is easy to set-up and integrates with standard
monitoring systems for enhanced security. Audit will track all events and actions with
your data and can scale from thousands to millions of IOPS with minimal performance
impact.
SNAPSHOTS
Snapshots let system administrators capture the state of a file system or directory at
a given point in time. If a file or directory is modified or deleted unintentionally, users
or administrators can revert it to its saved state. Snapshots in Qumulo’s file system
have an extremely efficient and scalable implementation. A single Qumulo cluster can
have a virtually unlimited number of concurrent snapshots without performance or
capacity degradation.
When people are CONTINUOUS REPLICATION
introduced to Qumulo’s Qumulo provides continuous replication across storage clusters, whether on-prem
or in the cloud. Once a replication relationship between a source cluster and a target
real-time analytics and
cluster has been established and synchronized, Qumulo’s software automatically
watch them perform at keeps data consistent. There’s no need to manage the complex job queues for
scale, their first question replication associated with legacy storage appliances.
is usually, “How can it be
Continuous replication in Qumulo’s file system leverages advanced snapshot
that fast?” capabilities to ensure consistent data replicas. With Qumulo snapshots, a replica on
the target cluster reproduces the state of the source directory at exact moments in
time. Qumulo replication relationships can be established on a per-directory basis for
maximum flexibility.
In the event of a disaster, Qumulo will get you back to a consistent known state with
minimal impact to the business. Qumulo’s software is able to to failover to point in
time snapshot efficiently by only considering new data that has written to the source.
No tree walk is required. Qumulo also preserves the configuration after fail-back and
enables the replication and fail-back of local users.
The Promise of Enterprise Hybrid Cloud File Storage 15WHITEPAPER
SCALABLE BLOCK STORE (SBS)
The Qumulo file system sits on top of a transactional virtual layer of protected storage
blocks called the Scalable Block Store (SBS). Instead of a system where every file
must figure out its protection for itself, data protection exists beneath the file system,
at the block level. Qumulo’s block-based protection, as implemented by SBS, provides
Instead of a system where
outstanding performance in environments that have petabytes of data and workloads
every file must figure out with mixed file sizes. SBS has many benefits, including:
its protection for itself, data
protection in Qumulo’s • Fast rebuild times in case of a failed disk drive;
software exists beneath • The ability to continue normal file operations during rebuild operations;
the file system, at the
block level. • No performance degradation due to contention between normal file writes and
rebuild writes;
• Equal storage efficiency for small files and for large files;
• Timely, accurate reporting of usable space;
• Efficient transactions that allow Qumulo clusters to scale to many hundreds of
nodes; and
• The ability to balance performance during rebuilds.
The virtualized protected block functionality of SBS is a huge advantage for the
Qumulo file system. In legacy storage systems that do not have SBS, protection
occurs on a file-by-file basis or using fixed RAID groups, which introduces many
difficult problems such as long rebuild times, inefficient storage of small files, and
costly and inefficient management of disk layouts.
Conclusion
At Qumulo, we believe that file data is the engine of innovation and that it fuels the
growth and long-term profitability of modern enterprises. File data is more important
than ever, and there are new requirements for how file storage must scale.
Qumulo opens new possibilities for its customers. With Qumulo’s file system, meeting
the release date of a major animated motion picture gets easier. Qumulo’s technology
makes it possible to achieve medical breakthroughs from multi-petabyte experimental
datasets. With Qumulo, identifying security threats in a billion-file network log can be
a daily reality. Determining when an event or intrusion happened that might involve
thousands of video files is now possible.
“Right now, Qumulo is the closest thing to an Apple unboxing, setup, and support
experience in the storage world.”
— High-level executive, Top U.S.-Based Mobile Carrier
The Promise of Enterprise Hybrid Cloud File Storage 16WHITEPAPER
At Qumulo, we believe that file data becomes transformative when it gives people
the freedom to collaborate, to innovate, and to create. The needs of our customers,
who are leaders and innovators in so many industries, are the sole drivers of our
aggressive product roadmap.
In a enterprise-proven modern file storage system, unparalleled reliability, scale and
performance are table stakes. A great storage system goes beyond that and gives
companies the global access and data insight they need to make their own dreams of
greatness come true. A great file storage system moves data where it’s needed, when
it’s needed, and at massive scale, and it does these things at lower cost, with higher
performance, more reliability and greater ease of use than other systems. Qumulo is
a different kind of storage company. As the creators of the world’s most advanced
file storage system, our own team of innovators puts what we believe into practice
every day. Scale-across file storage that supports massive scale is our vision and our
passion.
“I’ve worked with many different vendors, and while I’ve learned to expect problems I’ve
also learned no one’s going to knock themselves out to help me. Qumulo is the complete
opposite. I’ve never had so many smart people working so hard to curve the product
toward what we’re trying to do.”
— Tim LeDoux, Founder/VFX Supervisor, Crafty Apes
ABOUT QUMULO
Qumulo’s hybrid cloud file storage delivers real-time visibility, scale and control of
data across on-prem and cloud. Qumulo customers understand storage at a granular
level; programmatically configure and manage usage, capacity and performance; and
are continuously delighted with new capabilities, 100 percent usable capacity, and
direct access to experts. For more information visit www.qumulo.com
The Promise of Enterprise Hybrid Cloud File Storage 17You can also read
