Deploying big data networks
- 02 October, 2012 10:27
- Comments
So you’ve got approval for a big data project and addressed the top four challenges. Now the final hurdle is deploying the big data network.
Speaking at the IDC Asia-Pacific Business Analytics conference in Sydney recently, Arista Networks Australia principal engineer Lincoln Dale shared his tips with delegates for creating and deploying big data networks.
Invest in big data now for competitive advantage: IDC
“If we look at what the network looks like behind big data, what we tend to have is racks and racks of servers with built-in storage,” he said.
“The key here is that big data networks are not built like enterprise IT networks."
According to Dale, with a big data network CIOs and network engineers need to automate the infrastructure as much as possible.
“For example, adding a new rack or server should be a case of someone stacking it and turning it on,” he said.
“You don’t want a system administrator spending a couple of days putting an image [of a server] on the network. If something fails, you don’t want it to take out the entire [big data] cluster.”
Dale added that technology such as open source distributed computing platform Hadoop can handle big data projects and potential system failures.
“A key point of big data is that performance matters. You can do unstructured queries and data mining in ways to find out things,” he said.
Data growth
Turning to the future of big data, Dale cited Moore’s Law — a law which was created in 1970 and predicts the amount of transistors on integrated circuits doubles every two years.
According to Dale, Moore's Law also applies to digital data. For example, IDC’s Worldwide Storage Services Opportunity for Big Data 2012-2016 global report found that digital data is doubling approximately every two years, with 1.8 trillion gigabytes of data created in 2011 alone.
“The challenge is that a lot of this data loses value over time so the sooner you can reach an outcome from a given piece of data it has more value,” he said.
For example, if a consumer is on an online shopping website and navigates away from the site before coming back, this shows they are on other e-commerce sites looking for better prices.
“You can potentially make use of big data analytics to offer the consumer a better price sooner which could result in a sale,” he said.
Follow Hamish Barwick on Twitter: @HamishBarwick
Follow CIO Australia on Twitter and Like us on Facebook… Twitter: @CIO_Australia, Facebook: CIO Australia, or take part in the CIO conversation on LinkedIn: CIO Australia
Recommended
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.
- Bookmark this page
- Share this article
- Got more on this story? Email CIO
- Follow CIO on twitter
-
Larry Page wants to see your medical records
-
Dual-Persona Smartphones Not a BYOD Panacea
-
After two-year hiatus, EFF accepts bitcoin donations again
-
CIOs struggle to deliver timely mobile business apps: survey
-
Spiceworks' free management software gets integrated MDM
-
ESG Whitepaper: Integrated Computing Platform Survey
Data centres, servers, storage and more are being combined for simplified management and cost savings. In this survey, ESG looks at the current and future trends surrounding today’s integrated computing solutions. Download to find out how organisations are more likely to see commit IT budgets to the purchase of integrated solutions. Read more. -
Pathways Advanced ICT Leadership Development Program Course Outline and Big 6 2013
Developed by the CIO executive Council in conjunction with Rob Livingstone Advisory, Pathways Advanced is a 12-month CIO delivered, small group, mentor based professional leadership development program. Pathways Advanced brings together best practice, thought leadership and business insights for today’s most promising ICT professionals -
New Demands for Real-time Threat Management
Many organisations are evaluating a new security model based upon IT risk management best practices. This is a good idea, but not enough for today’s dynamic and malevolent threat landscape. To keep up with IT changes and external threats, large organisations need to embrace two new security practices: real-time risk management for day-to-day security adjustments and real-time threat management to detect and remediate sophisticated, stealthy, and damaging security breaches (i.e., advanced persistent threats, or APTs). Learn more.
Get exclusive access to Invitation only events CIO, reports & analysis. 
