Big Just Got Bigger

Suppose you came to work one day, took off your jacket, loosened your tie, sat down and found a letter on your desk from corporate counsel advising you that a court has just ruled that corporations are now responsible for retaining all business-related phone conversations for one calendar year.

Under this letter is a memo from the PR department (labelled URGENT) advising you that marketing's plan to use face recognition technology to identify prime customers entering the store may be getting some negative reviews in the press, with phrases like invasion of privacy and Big Brother being tossed around.

And under that is a directive from management asking for a technical analysis of a market simulator that may be able to predict to 5 per cent accuracy the response to a new product of single parents between the ages of 29 and 33 living in Sydney's eastern suburbs with incomes in the third decile.

Who are you? You're a petabyte CIO, the person responsible for developing and maintaining the science-fiction-like applications that will be running on tomorrow's immense storage capacities. Currently, petabyte responsibilities are mostly restricted to the IT departments of universities, research organisations, and microbiology and genetics labs. But the law of technological adoption ("If you build it, they will use it") says that sooner or later most CIOs will be crossing that line, discovering a new world of applications, responsibilities, costs and problems. So it's time to start thinking about . . .

PETABYTE POWER

Petabyte levels of storage will make possible three new categories of applications. One group depends on retaining and processing vast amounts of visual data, especially data from video cams. Imagine cameras trained on the sales floor, recording the minute-by-minute flow of customer traffic. Imagine that data feeding an application that analyses the relative effectiveness of a given product placement or the impact of a markdown. Marketing might be interested in learning how the proportion of couples to singles entering the store changes during the course of a two-week promotional campaign, or at what time of day the number of women shopping with kids rises and at what time it falls. With petabyte levels of storage, HR will be able to crunch a few months' worth of video camera data to flag personnel responsible for traffic bottlenecks on the sales floor.

A second category of petabyte potential is in supporting the transition to device networks. If the first generation of networks connected people to data and to each other, the second will do the same with both physical (counters, meters, cameras, motors, switches, telephones, digital printers) and virtual devices (applications and program objects). The great virtue of device networks is that they allow any interested constituency to have remote access to any link in the production cycle. CNN, for example, is digitising and networking all its production equipment so that pagers, mobile phones, PDAs, desktops and Web sites will all have equal, continuous and simultaneous access to programming.

In many industries, machines already keep maintenance informed about their operating condition, allowing them to be repaired just before they are about to fail. But if all the machines in a production line could be fully networked, management would be able to switch an entire production process to a single desired configuration - change the car's bench seats, for example, to buckets, or its colour from bottle-green to battleship grey - from half a world away. Once manufacturers can do that reliably, the dream of on-demand manufacturing truly has been achieved.

However, properly managing the thousands of sensor-actuated loops that form device networks requires retaining the history of their states, often in their raw, unsummarised form, for months and possibly years. To do that, you need to be able to store data in petabytes.

Finally, petabyte levels of storage would allow simulations and predictive models of enormous complexity. For instance, retail managers currently worry about how they can persuade a casual visitor to make a purchase. While this is an important problem, even more critical is how to turn a casual buyer into a loyal, recurring one. Over the long term, this second kind of conversion can deliver even more value to the enterprise.

But brand loyalty doesn't happen overnight, points out Richard Winter, president of the Winter Corporation, a US-based consultancy specialising in the architecture of very large databases. "[Transforming] someone into a repeat customer means presenting her with just the right information or opportunity at the right time," he says. "Knowing what to present can mean retaining huge amounts of information on that customer - what they've looked at, checked prices on, asked about, what they've not looked at - over long periods. Often the relationship needs to be followed from the point the customer first appears. Right now, that's impossible because raw, unsummarised, clickstream and transaction data is generally discarded after 30 to 60 days."

The reason it's discarded is because heretofore it was impossible to store. Petabyte levels of storage will change that.

Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

• Bookmark this page
• Share this article
  •  
  • facebook
  • slashdot
  • digg
  • Reddit
  • stumbleupon
  • linkedin
  • twitter
• Got more on this story? Email CIO
• Follow CIO on twitter

• Share
  Share this article

  •  google+
  • facebookfacebook
  • slashdotslashdot
  • diggdigg
  • RedditReddit
  • stumbleuponstumbleupon
  • linkedinlinkedin
  • twittertwitter
• print
• email
• Bookmark