Critical.
Authoritative.
Strategic.
Subscribe to CIO Magazine »

Operation Clean Data

Page:

Cleaning dirty data is not just a matter of mastering the technical challenges. It requires making sure your staff is working closely with the business every step of the way.

In the early hours of March 20, 2003, British soldiers, sailors and airmen joined US forces in the invasion of Iraq and the toppling of Saddam Hussein. Thus far, they have played a vital role in rebuilding Basra and the critical Persian Gulf port of Umm Qasr. Massive shipments of military materiel were essential to their success, and basically, anything that wasn't a vehicle, live ammunition or fresh provisions (which have different supply lines) began its journey to the Gulf from England's military warehouses. In the few weeks prior to the invasion of Iraq, these depots sent by ship or air 3169 6-metre shipping containers to the Gulf, along with almost 22,000 1-metre pallets.

Getting these shipments to the Gulf was a logistical nightmare that would have been far more fraught had the British defence ministry not embarked four years ago on a £6 million effort to pull together three separate supply chains: This involved reconciling some 850 different information systems, and integrating three inventory management systems and 15 remote systems.

The biggest foe in this massive integration effort was not Saddam Hussein, but dirty or disparate data. To one system, stock number 99 000 1111 was a 24-hour, cold-climate ration pack. To another system, the same number referred to an electronic radio valve. And if hungry troops were sent radio valves instead of rations, the invasion and rebuilding of Iraq wouldn't have gone very far.

Dirty data has long been a CIO's bugbear. But in today's wired world, the costs and consequences of inaccurate information are rising exponentially. Muddled mailing lists are one thing, missing military materiel quite another. Throw in the complications arising from merging different data sets, as in the aftermath of a merger or acquisition, and the difficulties of data cleansing multiply. For this article, we interviewed seasoned data-cleaning veterans from organizations as diverse as the British Ministry of Defence, the US Census Bureau and Cendant, a real estate and hospitality conglomerate. But the lessons learned contain two common themes: How to surmount the technical challenges of cleaning data, and how to align IT staff with the business side to ensure that the task gets done right.

Page:

Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

More about: Avis, Bill, Billion, Cendant, Defence Department, HIS Limited, Inventory Management Systems, IT People, NATO, PLUS, Sparkle

Comments

Post new comment

The content of this field is kept private and will not be shown publicly.
Users posting comments agree to the CIO comments policy.
Login or register to link comments to your user profile, or you may also post a comment without being logged in.
Related Whitepapers
Latest Stories
Community Comments
Latest Blog Posts
Whitepapers
  • Web 2.0 in the Workplace Today
    More than a decade after the term ‘Web 2.0’ was coined, many businesses are still nowhere near to taking full advantage of the collaborative technologies the term refers to. Undoubtedly, confidence is growing in relation to using tools such as Facebook, Skype, Twitter, and indeed many more organisations are using such technology now compared to even just a couple of years ago. But the fact remains that a worrying amount of businesses seem to be operating a ‘lockdown’ approach – an approach that I’m sure many Board-level staff know is simply not good for business in the long-term.
    Learn more »
  • Enhancing Decision-Making, Cost-Efficiency, and Profitability With Predictive Analytics
    Today’s managers must always look at the past, present, and future. They need reports on past performance to improve operational efficiency. Business intelligence (BI) platforms such as Information Builders WebFOCUS, are providing a unified decision-support environment where managers can retrieve and analyze data about past, present, and future activities. In this paper, we will discuss the incorporation of predictive modeling capabilities into the WebFOCUS BI platform, and highlight how this advanced functionality can dramatically improve decision-making, thus reducing risk and costs while increasing revenue and profits.
    Learn more »
  • IDC Insight: V-Ray Gives Symantec NetBackup a Competitive Advantage Today and into the Future
    Over a decade ago, Veritas software announced NetBackup FlashBackup to address the millions of small files problem, which had been and often remains the nemesis to fast and efficient backup of large file servers. Today, the FlashBackup technology is used to provide a logical understanding of what is stored with a VMDK- or VHD-image-level backup, without the necessity to install an agent inside each virtual machine. Read more.
    Learn more »
All whitepapers
rhs_login_lockGet exclusive access to Invitation only events CIO, reports & analysis.