Critical.
Authoritative.
Strategic.
Subscribe to CIO Magazine »

Deploying big data networks

Automation of servers and systems such as Hadoop will help ease the big data journey, says a principal networking engineer.

So you’ve got approval for a big data project and addressed the top four challenges. Now the final hurdle is deploying the big data network.

Speaking at the IDC Asia-Pacific Business Analytics conference in Sydney recently, Arista Networks Australia principal engineer Lincoln Dale shared his tips with delegates for creating and deploying big data networks.

Invest in big data now for competitive advantage: IDC

“If we look at what the network looks like behind big data, what we tend to have is racks and racks of servers with built-in storage,” he said.

“The key here is that big data networks are not built like enterprise IT networks."

According to Dale, with a big data network CIOs and network engineers need to automate the infrastructure as much as possible.

“For example, adding a new rack or server should be a case of someone stacking it and turning it on,” he said.

“You don’t want a system administrator spending a couple of days putting an image [of a server] on the network. If something fails, you don’t want it to take out the entire [big data] cluster.”

Dale added that technology such as open source distributed computing platform Hadoop can handle big data projects and potential system failures.

“A key point of big data is that performance matters. You can do unstructured queries and data mining in ways to find out things,” he said.

Data growth

Turning to the future of big data, Dale cited Moore’s Law — a law which was created in 1970 and predicts the amount of transistors on integrated circuits doubles every two years.

According to Dale, Moore's Law also applies to digital data. For example, IDC’s Worldwide Storage Services Opportunity for Big Data 2012-2016 global report found that digital data is doubling approximately every two years, with 1.8 trillion gigabytes of data created in 2011 alone.

“The challenge is that a lot of this data loses value over time so the sooner you can reach an outcome from a given piece of data it has more value,” he said.

For example, if a consumer is on an online shopping website and navigates away from the site before coming back, this shows they are on other e-commerce sites looking for better prices.

“You can potentially make use of big data analytics to offer the consumer a better price sooner which could result in a sale,” he said.

Follow Hamish Barwick on Twitter: @HamishBarwick

Follow CIO Australia on Twitter and Like us on Facebook… Twitter: @CIO_Australia, Facebook: CIO Australia, or take part in the CIO conversation on LinkedIn: CIO Australia

Related Articles

  1. From CMO to CEO: Anametrix’s Pelin Thorogood

    CMO

Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

References show all
Comments are now closed.
Related Whitepapers
Latest Stories
Community Comments
Latest Blog Posts
Whitepapers
  • Saving Time and Money with Savvy Use of Flash in Automated Storage Tiering
    In a sluggish economy, getting the best ROI on every IT dollar spent is the top priority for almost every business. Storage budgets in most IT environments continue to remain flat or are capped as a percentage of the overall IT spend, while data storage requirements continue to grow at an unsustainable pace. Download now to learn about the benefits of using flash in automated storage tiering.
    Learn more »
  • HP Helps NEC Reduce Network Management Costs and Gain Efficiencies
    NEC wanted to reduce network management costs, while increasing network visibility, decreasing mean-time-to-repair, improving stability and mitigating the risk of downtime. Download today to hear from Cameron Craig, Senior department manager of NEC on what approach they took and why.
    Learn more »
  • Leading Through Connections – Insights from the Global Chief Executive Officer Study
    IBM’s 2012 Global CEO study follows face-to-face discussions with more than 1,700 CEOs and senior public sector leaders from around the globe. The findings examine how CEOs are responding to the complexity of increasingly interconnected organisations, markets, societies and governments. For example, almost one-quarter of CEOs say their organisations operate below par in terms of driving value from data. CEOs have expressed frustration about their inability to capitalise on available information. This is because: “The time available to capture, interpret and act on information is getting shorter and shorter.” CEO, Chemicals and Petroleum, United States Given the need for deeper business insight, the best performing organisations are more adept at converting complex data into insights, and insights into action. Download Entire Report Now.
    Learn more »
All whitepapers
rhs_login_lockGet exclusive access to Invitation only events CIO, reports & analysis.
Recent comments