Managing big data storage – options for 2018

Data volumes across businesses of all sizes are growing at an average of 40% per annum, and IDC predicts that worldwide, this will almost double every two years – between now and 2020. This trend is presenting real challenges for IT departments, as they are spending an inordinate amount of time and money on their storage infrastructures.

Rather than firefighting data growth, new approaches are required, so valuable IT resources can be liberated to develop and implement business solutions – which drive productivity and have a positive impact on the bottom line.

[clickToTweet tweet=”Storage requirements for 2018 and beyond – Rather than firefighting data growth, new approaches are required” quote=”Storage requirements for 2018 and beyond – Rather than firefighting data growth, new approaches are required”]

Influencing factors

There are many trends, such as BYOD and the consumerisation of IT, that are driving exponential data growth, as they’re creating increased user expectations – which demand ever-present, information availability. However, despite the best attempts of IT teams to cater for this, they can be left struggling – and frequently experience such headaches as, storage sprawl, spiralling storage costs and an unending cycle of capacity purchases. They face information management and governance challenges too – and due to the complexities of data protection – these tasks can quickly become unworkable and drain resources even further.

Getting storage and archiving right is also key to ensuring GDPR compliance. Our recent UK data archiving survey of IT executives, across 150 UK medium to large organisations, revealed that poor practices are creating GDPR risks. Of those questioned, 70% admitted that they don’t have effective processes and systems to swiftly submit data to regulators – and risk being exposed to penalties.

The research also indicated that only 30% of those surveyed claimed their staff are able to find all types of structured and unstructured data quickly and easily – so are currently unprepared to cope with the demands GDPR compliance. To avoid the risks of GDPR non-compliance and other legal claims, organisations urgently need to better understand the lifecycle of their data – so it can then be correctly stored, managed, discovered and protected.

So, what’s the answer? To keep pace with this change and growth, storage technologies need to be more agile, scalable and flexible to address the myriad of business requirements that are now an everyday norm. Thankfully, traditional on-premise infrastructure is evolving to embrace the many business benefits of cloud technology, which can positively impact on the way business can store, archive and retrieve data.

On-premise or cloud?

Businesses now face clear data storage options – either in-house, cloud or both. Although cloud storage provides well-documented cost savings, efficiencies and liberated IT operations, decision making should involve thoroughly examining which option best caters for specific business environments.


Almost every business has invested in some form of in-house, data storage – be it a manual process, or a more agile solution that can meet both present and future business requirements. On-premise storage scores highly by providing fast access between data and applications, so businesses have the option of using dedicated high-performance disks. Although these functions are available in the cloud too, the input/output rate is not always as fast.

Ultimately, on-premise, high-performance storage holds many advantages for businesses that experience high-performance workloads, large databases, and performance-dependent virtual machines – where the applications are local.


With unstructured data growing faster than ever, storing it in the cloud is the better option. This is because, although unstructured data doesn’t necessarily have the same performance requirements as tier 1 data, it is capacity intensive. Cloud storage works well as files of this type are accessed less in the longer term, due to the scale and resilience of the associated workloads. By using storage in the cloud, also has the added business advantage that management and maintenance of an underlying hardware platform is no longer required.

What to demand from cloud

The new breed of more mature cloud storage solutions have progressed from traditional services  – and now offer complete archive capabilities, with the security, scalability and integrity of big vendor-owned data centres. They offer fixed operational costs – without owning, running or managing any infrastructure – with minimal upfront investment, and guarantees of a simple transition to the cloud. The best solutions should also deliver an intuitive user experience and the freedom to access from anywhere, across all devices.

There are several non-negotiable, key features that businesses should expect from cloud storage options.

There are several non-negotiable, key features that businesses should expect from cloud storage options. Firstly, the cost of migrating data into cloud from on-premises should be minimal. Secondly, businesses should only consider solutions that can archive and search across all data – including emails, files, IM and social media.

Another key feature to look for is the customisation of policy configuration – including archive, retain and delete data using specific grouping – or policies that suit business requirements. Finally, businesses should only consider solutions backed by enterprise-class cloud infrastructure and data protection – so critical data is extensively covered against outages, loss or corruption.

+ posts

CIF Presents TWF – Professor Sue Black


Related articles

How Businesses Should Tackle Big Data Challenges

In today's data-driven landscape, Big Data plays a pivotal...

UK IP Benefits and How to Get One

There are many reasons why you may get a...

Navigating the Landscape of AI Adoption in Business

In today's rapidly evolving technological landscape, the integration of...

Three Ways to Strengthen API Security

APIs (Application Programming Interfaces) are a critical driver of...

A Comprehensive Guide To The Cloud Native Database [2024]

Databases are crucial for storing and managing important information....

Subscribe to our Newsletter