The data management company is using automation to help enterprises move their unstructured data in such scenarios as IoT, health care, and acquisitions.
In a business world where the data growth is the norm, unstructured data presents a particular challenge.
First, there is a lot of unstructured data – images, videos, texts, emails, social media content, data from Internet of Things (IoT) devices, streaming data, and more. Gartner estimates that 80% to 90% of data being created is unstructured.
It also doesn’t tend to live in corporate spreadsheets, databases or data warehouses that run on servers in central data centers. Unstructured data often is found outside the firewalls, in the cloud and at the edge. Managing it all isn’t easy.
“Machines and applications generate data in a variety of formats and sizes and data is growing fast across data centers and in the cloud,” Krishna Subramanian, co-founder and president of Komprise, which offers a platform for managing and moving unstructured data, told Digital CxO.
A Huge Amount of Data
Subramanian said analyst estimates say the world will store 200 zettabytes of by 2025, half of it being stored in the cloud. About 463 exabytes of data will be created each day around the world – more than 212 million DVDs worth of data. A recent Komprise study found that enterprises spend more than 30% of their IT budgets on data storage.
“Now imagine having to store and protect all this data for decades, and you can understand how this problem gets exponentially more complex over time,” she said.
That complexity grows when it’s time to move the unstructured data. There not only is a lot of it, but it’s distributed across many locations and storage technologies, making it difficult to move to the right place at the right time, such as to lower cost storage when it’s no longer in active use or to AI or other big data analytics tools for research efforts.
“Leveraging data for new uses is a top opportunity and challenge in most enterprises today,” Subramanian said.
Komprise this week unveiled a major update of its Elastic Data Migration, a tool that can be used as part of its Intelligent Data Management platform or as a standalone offering, that is aimed at three particularly difficult challenges when moving large-scale file and object data.
“The new capabilities we added expand the addressable use cases beyond hybrid cloud to include edge, multi-site consolidation and migrations across storage architectures,” she said. “The three new use cases we added are to support devices that are constantly churning out data such as IoT and medical, consolidations due to M&A or other transformations, and migrations that may occur offline or through a different tool first and then through Komprise.”
Cutovers and Consolidation
The first addresses that “cutover” period of migrations, when source and destinations shares are set to “read-only” and users and applications can’t write data. This is a problem for environments where machine-generated data – think connected medical devices or IoT sensors – can’t stop creating data and must be able to always write it.
With the latest update, Komprise allows data to be added, modified or deleted on the destination shares during the cutover period.
The other two uses cases include when an enterprise begins a preloaded migration using one tool –open-source or offline data transfer technology like AWS’s Snowball or Azure Data Box or mirrored using offerings from storage vendors – and then chooses Komprise to continue the migration to the final destination. Organizations can now complete the migration without having to move the data again.
In scenarios like an acquisition, data center consolidation, or cloud transformation, an enterprise may need to combine multiple sources in a single migration project to the same destination. Komprise now automates the process.
“The common thread across these use cases is that they represent non-traditional scenarios in which data is piling up at the edge, across sites, and across storage architectures,” Subramanian said, adding that “Komprise simplifies data mobility by adding automation to find data wherever it is piling up – whether in data centers or continuously at the edge by IoT and health care equipment—and then analyze the data and index it and move it systematically so an IT person does not have to deal with the headaches.”
