Critical.
Authoritative.
Strategic.
Subscribe to CIO Magazine »

The promise and lie of big data

Look beyond the hype

The term big data is everywhere these days, there’s no doubt about it. When I meet with CIOs, big data is often named as one of the top trends shaping their IT agendas for 2012, along with cloud and mobility.

So what exactly is big data and why is it big news now?

The best formal definition of big data I have seen is data that takes more than one server to store or process. For me, when I think of big data, I think of a world of possibilities to dig deeper into information to bring additional insight to the business. For example, imagine analysing a data set that includes public transport statistics with your own Web store to determine if people shop on their mobile phone during peak hour commuting.

Like the term ‘cloud’, technology vendors have realised the marketability of ‘big data’ and have applied it to almost any solution they can. From big data databases and big data backup services, to cloud based big data analysis systems, not a week goes past without a new solution being promoted as helping companies enable big data.

The truth is that at the moment unleashing the essence of big data into small, medium and even enterprise businesses is hard. Most big data jobs are spawned from internal business intelligence projects where the business is trying to understand purchasing habits of their customers, work out the most effective shipping routes to get stock to stores, or trying to predict the stock market. Businesses have been analysing data in this way for many years but due to the cloud — and a few other technology movements such as NoSQL — the velocity of data growth has placed a new perspective on this.

The failure of big data at the moment is still in what it takes to get there. Many businesses believe that they can purchase a new Web 2.0 Big Data solution and in a few days have shiny reports and graphs walking them through the spending habits of their customers — which is a picture that is currently being painted by the marketing teams of most big technology vendors. However, what they fail to tell you is that data analysis and big data have spawned a new role which some IT teams need to fill; the data scientist. These people have been around for many years, writing the algorithms that power everything from our traffic lights, how our air traffic control system prioritises landing schedules through to the systems that power background checks for new mortgages.

To get to this promised land of big data where our dashboards, reports and business decisions are being made from solid data, we need to bring data scientists onboard, help them to understand the business and how it works, and then let them loose on all the data they can get their hands on. From there they may use tools from the major vendors (e.g. EMC, Microsoft, Oracle), or they may use tools that have been developed by open source companies (Hadoop, Google BigQuery), or they might look at moving these workloads to the cloud (think Amazon, Rackspace, or others).

Big data isn’t a technology or strategy that is implemented by the IT team, it needs to be thought about from the highest levels within the business starting with the question: ‘How can we better utilise the data we have to make better business decisions?’

This style of project needs executive backing, both in support and budget to ensure it gets off the ground and continues over time.

There are many businesses out there that spend their time developing dashboards, reports and data analysis systems that can help and really ensure that goals are reached, and IT budgets aren’t depleted by purchasing a ‘big data ready product’.

Big data is new and for many business (and IT teams) it’s still an unfulfilled promise that the business may be driving them towards. Before you embark on a big data project, have a really good look at what you’re buying into and ensure it’s something that can actually help your business — not drain time and budget.

Rhys Evans is national practice manager Enterprise Information Systems at Thomas Duryea Consulting.

Related Articles

  1. From CMO to CEO: Anametrix’s Pelin Thorogood

    CMO

Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

Comments are now closed.
Related Whitepapers
Latest Stories
Community Comments
Latest Blog Posts
Whitepapers
  • Advanced Malware Exposed
    This handbook shines a light on the dark corners of advanced malware, both to educate as well as to spark renewed efforts against these stealthy and persistent threats. By understanding the tools being used by criminals, we can better defend our nations, our critical infrastructures and our citizens. This ebook will provide readers with a new understanding of the rapidly developing cyber threat landscape and practical insights into how they can protect their data and computing infrastructures. Download now.
    Learn more »
  • A Holistic Approach to your BYOD Challenge
    More and more enterprises are seeing significant benefits from allowing employees to choose the device they use to get their jobs done, and are adopting bring your own device (BYOD) initiatives. While the BYOD trend increases flexibility and productivity, it introduces a host of new challenges for your IT administrators. Click for more!
    Learn more »
  • Real-Time Protection Against Malware Infection
    Malware is at such high levels (more than 60 million unique samples per year) that protecting an endpoint with traditional antivirus software, has become futile. More than 100,000 new types of malware are now released every day, and antivirus vendors are racing to add new protection features to try to keep their protection levels up. Read more.
    Learn more »
All whitepapers
rhs_login_lockGet exclusive access to Invitation only events CIO, reports & analysis.
Recent comments