Since the invention of the television, we as human beings have become accustomed to trusting the output of a screen. Military campaigns always have a media and propaganda element as part of the planning. The capture of the national broadcaster is always key in any rebellion or coup d’etat.

However, the issue is trust.

[easy-tweet tweet=”Why do we tend to almost hypnotically fall for the messaging laid before us? #BigData #Security”]

Why do we tend to almost hypnotically fall for the messaging laid before us? This is especially the case when it is using the right mix of colours and aims for confirmation bias (the part of our mind that makes instant judgments at speed).

The Myth of the Data Scientist

The word scientist, in most of our minds, conjures up a man in a white lab coat that deals in certainty and fact. A scientist discovers new frontiers whilst carefully documenting an outcome and sharing it with the greater world. The problem though is the so called ‘data scientist’ who is building these algorithms and analytical insight. For example, predictions of the Iraq War through to using Big Data to predict flu outbreaks. Does the wisdom inherent make the data analysis real?

So what is a data scientist?

Generally, a data scientist is a system developer or statistician; or as we are finding all too often, someone who has done a course in a lab using one of the main open source distributions, who has appointed themselves as the solver of all problems. Not unlike the snake oil salesman of Wild West times.

The similarity to ERP Promises of old 

Do a reasonable amount of googling and you will see the promises of ERP systems from many years ago that offered unparalleled insight and analytics to businesses. Unfortunately, with many of these promises the enterprise usually follows like sheep, based upon a high profile success case that incurs pressure from the board to adapt or die.

So what is the big issue?

The big issue is that companies small and large are betting on becoming the next Amazon or Netflix. The marketing hype and science, combined with the dark Machiavellian public relation types, have the general market believing that Big Data will save the world. Whilst there are big data and analytics projects in use today within both the scientific and machine learning community that can be hailed as a success, we have to temper the hype with caution.

[easy-tweet tweet=”Big Data has a chance to make a meaningful impact, but only if the rhetoric is calmed down” hashtags=”security, big data, “]

Big Data has a chance to make a meaningful impact, but only if the rhetoric is calmed down – the pressure cooker slowed – so that this technology has a chance to shine. Yes, we understand that storage vendors and cloud providers want to espouse Big Data benefits, but let’s not go through another round of hyperbole as we did with cloud.

Our tips for a Big Data project:

  • Start small, go with a little data and analyse with an open mind
  • Think of an outcome you are aiming for and reverse the project plan from there
  • Do not get sucked in by the rhetorical, a title is meaningless without the human occupying it being able to communicate clearly
  • Choose the vendor you will go with wisely and make sure that they offer the cradle to grave support and consultancy, do not get stuck with point solutions
  • Always check data regulations when handling sensitive or personal data that is going to an external cloud provider
  • Everyone makes mistakes, those that claim never to or have elevated their status to demi-gods need to be avoided like the plague
  • If something sounds too good to be true then it probably is
  • Always check backup and data retention policies, check terms and conditions of engagement with a lawyer – watch out for the liability clauses
  • Avoid the blame game where possible, limit the scope to as few external parties as possible
  • Cost of everything and value of nothing, make sure that costs are accurately reflected. Do not become a never ending subscription with a low entry point to get you onto a platform