The march of Object Storage

We have started to see a number of vendors make more positive noise towards Object and Virtualised storage by adding new functions or whole new products in the storage space. This lead me to think – why?

[easy-tweet tweet=”#ObjectStorage is the way of the future, @Anke_Philipp tells you why” user=”comparethecloud”]

Well it’s simple, we are producing so much data today that the current setups will not hold together in the mid to long term. So they are having to address this today, and as they say, you need to look back to move forward.

We are producing so much Data today that the current setups will not hold together

Brief History of Data Storage in the last 15 years

The way we store data has evolved immensely since the turn of the millennium. Fifteen years ago we had a lot less data, and it was mainly stored in a local or direct attached model. By the mid 2000s we started to see a move to scale-out storage driven by the need for greater data protection/DR, mixed environments, virtualisation etc. but there had been an increase to a lot more data and a lot of silos of storage. Now we are in the cloud era and it was this drive to cloud based services and storage – plus the hard learned lessons of the passed years – that has lead us to re-examine how we store our data. Add to this Big Data, Analytics, more Virtualisation, VDI, personal data and the fact we wish to access it 24/7, things are becoming hard to manage and control.

It’s this drive to cloud based services and storage that is leading us to re-examine how we store so much data

So why you ask: Object Storage?

The biggest problem with traditional approaches is scalability. NAS lacks the ability to scale as a single system, especially in exabyte and larger environments we are seeing today. Today’s SANs are already complex when they are deployed with a file system layer on top. Scaling them out makes the problem a lot worse, and then there is the big one: cost. Prices are out of control; in some cases £1k per TB when it is £100 in PC world for the same basic disk! If this is enabling tier one vendors to run there own private jets, well then you have to look very closely at the whole storage story!

How is Object Storage different from NAS or SAN?

Well first, Object Storage is not new, it certainly pre-dates Amazon’s S3. Object Storage is essentially just a different way of storing, organising and accessing data on disk. An Object Storage platform provides a storage infrastructure to store files with lots of metadata added to them. The back-end architecture of an object storage platform is designed to present all the storage nodes as one single pool. With Object Storage, there is no file system hierarchy. The architecture of the platform, and its new data protection schemes (vs. RAID, the de-facto data protection scheme for SAN and NAS), allow this pool to scale virtually to an unlimited size, while keeping the system simple to manage. Plus it’s not limited to location so data can be spread worldwide or local – your choice.

With Object Storage, there is no file system hierarchy

Users access object storage through applications that typically use a REST API (an internet protocol, optimized for online applications). This makes object storage ideal for all online / cloud environments but not limited too as we are now starting to see new applications object aware.

[easy-tweet tweet=”REST API access makes #objectstorage ideal for all online / cloud environments” user=”anke_philipp”]

When objects are stored, an identifier is created to locate the object in the pool; applications can very quickly retrieve the right data for the users through the object identifier or by querying the metadata (information about the objects, like the name, when it was created, by who etc.). I think of it as the concierge service in a hotel, you give them your bag they give you a ticket the bag goes off to be stored safely, you come back handover your ticket and there’s your bag. This approach enables significantly faster access and much less overhead than locating a file through a traditional file system.

Current generation of object storage platforms are designed with openness and flexibility in mind. The industry has learned some tough lessons from using proprietary systems. One initiative to prevent vendor lock-in, is SNIA’s cloud data management interface (CDMI). This is a set of pre-defined RESTful HTTP operations for assessing the capabilities of the cloud storage system, allocating and accessing containers and objects, managing users and groups, implementing access control, attaching metadata, billing etc.

[quote_box_center]

Object Storage Summary

• Data is stored as objects in one large, scalable pool of storage.
• Objects are stored with metadata – information about the object.
• An Object ID is stored, to locate the data.
• REST (API) is the standard interface, simple commands used by applications.
• Objects are immutable; edits are saved as a new object.

[/quote_box_center]

So… will we see the march of object-based storage? I think YES! This is because we are going to produce so much more data which we will need store, but we will expect the costs to be lower. Added to this, with people becoming more open to their data being dispersed to different geographically spread data centres, and the law catching up with regards to data governance – it is only going to move one way and that is forward (at least until the new great leap in storage technology comes along).