CONTRIBUTOR
Managing the massive amount of data being generated is always a challenge, and it’s not getting any easier.
Start with the data itself. Not only will there by 181 zettabytes of it generated in 2025 – up from about 120ZB expected this year – but it is widely scattered and siloed in disparate storage systems, within the multicloud, out at the edge and in on-premises data centers.
At the same time, the bulk of the data – about 80% to 90% – is unstructured. This creates its own set of trials for data and storage administrators. Structured data can be more easily collated and plugged into spreadsheets with rows and columns and into databases. It’s easier to store and analyze with easy-to-use tools and languages. Think of the business data normally found in Excel spreadsheets.
Then there is unstructured data, the photos, videos, social media posts, texts, audio files, behavioral and mobile data, data from the internet of things, and so on. This creates two problems for enterprises data managers and storage administrators, according to Krishna Subramanian, co-founder, president and COO of Komprise, which offers a service for managing unstructured data.
The first is they have to handle the huge amount of data that is growing more than 20% a year with budgets that that are only increasing about 5% or less.
Organizations “have to squeeze more data into less resources, and this is a challenge because often IT does not even know what this data is or who created it or why it is piling up in so many places,” Subramanian told Digital CxO.
Secondly, they need to make it easier for users to pull business value from all this data through using AI, machine learning and analytics, but the “unstructured data can be billions of files of different types and sizes scattered everywhere; finding the right data and moving it to the right place at the right time is time-consuming and error prone,” she said.
Surfacing the Data Management Challenges
A lot these concerns emerged earlier this month in Komprise’s annual study of the unstructured data management space, where the top challenge – at 47% – was moving data without disrupting users and applications, only a step ahead of preparing for AI and cloud services (46%). In addition, optimizing costs was second on the list of the key top data storage priorities, behind preparing data for AI.
According to the study, 73% of organizations are spending more than 30% of their IT budgets on data storage and backups.
The Campbell, California-based company has been trying to chip away thorny issues of trying to make data more manageable and less expensive for enterprises. In March, the company rolled out Komprise Analysis, a software-as-a-service (SaaS) offering aimed at letting organizations analyze data regardless of the object or file stores it sits in. The company pitched it as a tool for getting greater visibility into data that the business might not be ready to move.
First Data Visibility, Now Movement
This week, Komprise unveiled another service that brings the mobility into the picture, an important step for companies that are shifting away from simply storing data to delivering data services, according to Subramanian. Storage Insights enables organizations to use more than two dozen metrics to see what data is being stored where, how much it costs, and the amount of free and used space across systems from such vendors and cloud providers as Dell EMC, Pure Storage, Amazon Web Services and Microsoft Azure.
The service then leverages the company’s Transparent Move Technology to shift data where it makes the most business and fiscal sense at that moment. It’s difficult to say that some data is better for on-prem environments and other data better suited for the cloud, because data has different uses and performance needs throughout its lifecycle, Subramanian said.
“What you really need is something that analyzes data usage and based on whether the data is hot/active or cold, it moves the data to either higher performance tiers on-premises or in the cloud,” she said. “When the data becomes cold or no longer needed for active use, Komprise can transparently tier it to less expensive capacity storage.”
AI and Security
Storage Insights also includes security capabilities, such as spotting ransomware attacks and other cyberthreats on data stores by detecting anomalous activity and setting alerts when such unusual activity occurs.
With generative AI seeing rapid adoption in enterprises, demand for data security is rising from data managers and security admins. Komprise’s survey showed that 90% of companies let employees use the technology, but 66% are worried about governance concerns like preventing security and privacy violations, the threat of unethical; or inaccurate outputs based on a lack of information about the source of the data, and corporate data leaking from AI models.
Because Storage Insights can show what data moved where – including to an AI service – it can address some of the governance concerns. The service also can show who moved the data, how it was used, and the source of the data, she said.
