You’ve probably heard of that reality TV show called Britain’s Biggest Hoarders. It features people who compulsively acquire stuff and are unwilling or unable to discard it.

A lot of companies are behaving the same way. They’ve become data hoarders. And, like those unfortunate people on the TV show, they’ll soon find themselves flooded with data and struggling with the cost of managing it—if they aren’t already.

It gets messy

Compulsive hoarding
Compulsive hoarding – Wikimedia Commons / Grap

Companies are brought to this point by the strong belief, now widespread in business, that the worst thing you can do with a piece of data is to throw it away. It’s an understandable impulse. You want to keep all your data because you never know which innocent piece of information might suddenly become hugely important, today or tomorrow.

Data accumulation is now a problem for companies both large and small. Not long ago, 500 terabytes of data used to be solely concern for a Fortune 500 company. Today, it is a problem for a lot of small and midsize companies as well.

For example, one of our healthcare customers stores medical images such as CT scans and mammograms for multiple radiology offices. They recently upgraded to high-resolution 3D mammography, which meant their storage requirements for each image went from a few gigabytes to tens of gigabytes. In a short space of time, they went from an organisation dealing with data on a few hundred TB-scale to one needing to manage storage on a petabyte scale.

Solutions that worked perfectly fine just five years ago simply don’t work any longer. In these “old days,” companies would just drop data into their existing storage infrastructure. But when your data is growing at 120 percent per year, you can’t do that because you’ll end up either swapping out your entire storage infrastructure every year or two, or adding disparate storage siloes to address data growth. Either option is bad for business in terms of cost, complexity, cohesion and continuity.

Today, companies need to quickly figure out which information is critical to their business, and which information can be neglected. They need to understand which data should be pushed to the cloud, so it’s always available, and which data can be stored locally.

The answer to this challenge lies in self-organising storage that applies intelligence and, specifically, machine learning to the management of information. In this scenario, real-time analytics done by the storage system itself decide what the optimal placement for data is and what the optimal protection for any element of information within a dataset may be. This is the only way people are going to keep up with the explosive growth of data that has reached a scale and size that humans can no longer handle effectively.

The future where machine-learning algorithms go through the content of your data and establish relationships has already begun. As a result, organisations are able to organise data in its proper context and ensure that datasets with similar context are matched together, thus making it easier to manage and make sense of mountains of information.

To tackle your growing data issue, my advice is to start with digestible chunks. Don’t try to pretend you know what the world of technology and data storage is going to bring you next year. Don’t go out and buy a storage system that you hope will be sufficient five years from now. The digital world moves far too fast and is far too fluid for that kind of guesswork.

Instead, start small with a strategy that scales over time. Take sensible steps so you can test and prove as you go. You’ll be light on your feet and ready to respond as your business changes, as the type of data you’re storing changes, as the technologies in the market change, as regulations change, as your policy changes.

So, go ahead and hoard your data. But be smart about it. Be nimble and flexible. Go with a system that’s scalable and requires a relatively small investment now, so you can develop with agility and accuracy in the years ahead.

Andy Zollo is vice president of Sales at StorageCraft EMEA