Signs of Lifecycle
- 06 November, 2006 11:19
Like most CIOs, the Australian Broadcasting Corporation's Colin Knowles recognizes the importance of information lifecycle management. What distinguishes Knowles's role as director of technology and distribution at the national broadcaster is how the strategic importance of managing information over its lifecycle forms a fundamental pillar of the organization's business.
With a number of disparate information lifecycle management (ILM) projects behind it, eight years ago the ABC began a concerted effort to unify the classification of its information when it recognized the need to digitize its tape-bound archives. For Knowles, the move has resulted in the new-found luxury of using standard IT storage infrastructure for traditional broadcast content, but also ushered in a new era of how coherent ILM has become integral to business processes. So the ABC moved away from tape archiving and began betting its business on ILM.
Knowles says that if you do not have an ILM strategy "you're in big trouble" because the organization is not going to be practising good governance, and is "probably not going to be worrying about business survival". Businesses and vendors can talk about ILM and think of a "grandiose scheme" but Knowles says a lot of ILM has been done for many years in different ways, albeit in a micro fashion.
"In a lot of organizations, and we're not unique, you have a lot of information about what's happened in the past but it's stashed away and is not adequately accessible," he says. "We've been trying to fix that over the last few years. We've tried to enshrine some of that information in our procedures and practices. Information management is about making that available in a readily digestible form. And of course it's becoming easier."
Recent research by the Information Storage Industry Centre (ISIC) at the University of California San Diego into end-user perspectives of ILM revealed three key drivers for ILM initiatives, which differed depending on the level of operational management within the organization.
To gain a better understanding of how senior technology managers address ILM, Dr James Short conducted 12 interviews with CIOs and CTOs across nine organizations. Short's findings revealed three distinct categories for how the respondents viewed ILM within their organization: a storage-centric, or utility, view; a compliance-driven, records management view; and a customer-focused, business intelligence (BI) perspective.
Despite the divergent viewpoints of ILM by CIOs, both Knowles and Short agree ILM is a moving target that has not yet matured to the point where vendors can offer a packaged solution.
"We are in early days [and] have a long way to go before the diverse vendor offerings and product platforms can be simplified," Short says. "User companies do not have their arms around the problem at this point, and vendor economics would not favour standardized offerings even if they could be reasonably produced at this time."
Knowles goes a step further to say he does not think there will ever be an "out of the box" ILM solution because the characteristics of all organizations are different. Instead, Knowles leans towards the idea of a toolbox, which is easy to adapt to the intricacies of individual businesses. "The most successful applications, apart from mundane ones, are really just toolboxes that allow you to create your applications around it," he says.
In addition to the dearth of turnkey solutions, another key finding of the ISIC survey positions ILM as an ill-defined concept or set of practices. ILM is a process to support business goals and is seen as a "natural" progression of ideas and products coming into the market. To help alleviate the infrastructure uncertainty, Short recommends CIOs look at software middleware providers, not storage vendors, because this is where "the rubber hits the road" with respect to rationalizing the divergent vendor ILM offerings at the point of use.
More Is More
The most significant development in the storage tier of ILM is the "flattening" of the infrastructure required to manage the information to the extent that the distinction between operational and warehouse storage environments has become blurred. For the ABC this means the elimination of manual tape handling, dramatically reduced information retrieval times and the ability to make automatic replication copies as its content ages.
"The warehouse is an integral part of ILM anyway," Knowles says. "It sits in the background and comes forward when you need it. If you look at a lot of corporate data, you would keep it there whenever you had space and eventually you would pension it off on usage, and so forth."
Short, having also conducted research on warehouses, says the data supports the idea that warehouses have grown to the point that in many companies there is little distinction between warehouses and operational level data, driven in part by the declining cost of storage and the fact that it is easier to buy another storage rack than it is to go down the path of an enterprise data management strategy.
Consequently all organizations are seeing an explosion of data storage because "it's easy and cheap", Knowles says, adding that storage capacity as a means for ILM is equivalent to building a shed at the back of your house - you keep throwing things into it but eventually you will have to do a clean-up. "You think it doesn't cost you much to store it but it takes you a few days to clean the shed," he says. The ABC is now distinguishing between discretionary and non-discretionary data. It will manage the former as a business obligation, and charge users for the latter with the rationale that people will not store data unnecessarily if they have to pay for it.
How to store information efficiently is the fundamental challenge of storage-tier ILM, because "we can't forever keep adding more and more", Knowles says. Even in the context of the ABC's video and audio material, it could go to the extreme of keeping every camera tape ever taken but much of it may be redundant, even in the long term. Conversely, Knowles urges CIOs to be aware of how the value of information may change over time.
"Our natural history people say we do take a lot of footage but if the animal disappears it may become very valuable," he says. "A lot of the value of information comes from re-purposing it. Corporate data has value because of the other things you do with it. It's looking for trends long term and looking at data a totally different way. If I look at the data from a different angle, does it tell me something different?"
Knowles says part of designing an information model is about trying to segment storage in a way that delivers "what makes sense" if the organization wants to look at it in different ways. "What does it cost me to produce Radio National? I can't tell you that if I don't collect that for a particular service," he says. As an example, the ABC recently completed an exercise to better classify its assets. It had records of how old the assets were, but did not know precisely what they were being used for. Variables like which programs made use of the assets at what periods of time, how they all related to each other and the utilization factor of most of the material remained a mystery.
"We are now looking at that from different perspectives and saying, let's look at it in content streams, let's look at it from platform streams," Knowles says. "Then we can make decisions on how important it is for us to replace the asset, or is it time for us to reappraise the way we deliver?"
Short agrees, saying an equally important variable to consider in ILM policy is the type of information rather than just the application or the business function that created it. "Defining information types is more important in dealing with unstructured rather than structured data," he says. "For the latter, if I know the application and the business unit, I have most of what I need. In unstructured data, I really need more, and a schema that fits strategy to information types makes a lot of sense."
Going on Record
The ISIC study revealed records management as another core driver for ILM initiatives, particularly among companies most directly affected by evolving security and compliance regulations, such as financial institutions. According to Short's research, the distinction between records management and ILM is narrowing as ILM's footprint continues to include records and content, which require information policy management. Short says while the responsibility for implementing records control is delegated several levels below the CIO, the ultimate responsibility often ends up with IT management.
At the ABC, Knowles has established a total records information management strategy that integrates with its archives. "If we take the governance requirements for records as an example, from the time you create a record you need to be able to track that record through until there is some formal, appropriate time to destroy it," he says. "If it is a board record it will probably be reserved forever. On the other hand if it's an order for a pencil it probably has a seven-year life. And if it's administrative, like 'someone has arrived at front desk', it probably has a life span of a year."
The ABC is now attempting to align all of its business to the same model as it begins to build things, like its new e-mail system. If ILM has been implemented correctly, the e-mail system will manage the preservation of the information for the time it needs to be preserved. "That isn't simple because there is a lot of noise in the e-mail system," Knowles says. "Do you really want to have all the e-mails the organization has created and have them stashed away in case you need them? Or do you properly file those into an information storage system?"
The ABC's strategy, one which Knowles believes is appropriate for e-mail, is to put messages in information storage boxes - equivalent to an electronic filing system. This automatically classifies that material into the sorts of categories of information the organization has, including determining how long the record should be kept. "The problem with an e-mail system is I have this big bulk of stuff, [but] if I have it classified I now have what I need to preserve," Knowles says.
By extending this philosophy to all of its content - regardless if the information is stored on disk or tape - the ABC can now apply business rules around its ILM infrastructure. "ABC2 is being produced with a very modest $2 million per year and you couldn't do that with traditional TV," Knowles says. "If data is going to be played back from the ABC the next day it will sit in disk but if it is going to be played back in two weeks it will be held on tape because it's very cheap."
Turning on the Indicators
A successful ILM strategy may result in universally-accessible and appropriately-recorded data, but Short's research also uncovered business intelligence as a third view of ILM initiatives, with a particular focus on an improved customer experience.
Acknowledging that the top-level business intelligence (BI) view of ILM has been in demand by executives since he was completing his postgraduate studies in the 1970s, Knowles says BI is still probably not done too well. While some organizations have managed to define their requirements down to the point where they have useful business intelligence, he says, most businesses have great trouble grappling with defining what the real key indicators are specific for their business.
"In particular sectors where it has been developed well, I think it serves a greatly useful purpose because you have got the indicators like what are the three or four things that really matter, and tell me how the business is going," Knowles says. "We don't get the same clarity as driving a car in driving our business. Many of the indicators are too slow because they are lagging, in which case by the time you've taken corrective action it's too late."
To avoid the strategic information quagmire, the ABC's approach to data warehousing and business intelligence is to determine what it can extract to project forward into its system so it has a genuine management tool that is responsive in giving clear indications of "the current state of play". In essence, a BI platform must be reliable so it can be trusted and, in Knowles's words, tells the organization "we are seeing a deviation here, something might be happening, somebody else can go and look at a more detailed level of information in the organization".
For CIOs to see the full circle of BI relating back to ILM, Knowles says organizations have to "strike" what is the real value of the information and what is the value of the intelligence because the former is only good when it becomes useful intelligence. "So it's measuring the value of that to the business, and every business is going to be different," he says. "Is that going to add value to my business? And being able to articulate that to the users of that material. Each of those users has a different value that they attach to that particular piece of information. It's the collective component of all of those values that adds to the reason why I do it in the first place."
With the rapid convergence of operational and archival storage technologies, increasing records management and compliance regulation, and a relentless quest for more relevant business intelligence, ILM is quickly becoming the next important patch under the CIO's umbrella.
Regarding what CIOs should expect the cost of an ILM implementation to be, Knowles is pragmatic. "I don't think you can answer it, not simply, because it's core for our business."
SIDEBAR: Data Classification Speed Bumpsby Jerome Wendt
Classitying data is becoming the hot topic in storage. Viewed as a prerequisite for information lifecycle management (ILM) and simple to implement, it makes the dream of ILM a more attainable goal. However, there are some items users need to consider in order to keep this dream from becoming a harsh reality.
First, accessing all of the data you want to classify is never as easy as vendors make it sound. Discovering data using agent-less and agent-based approaches may look simple on a PowerPoint presentation, but introduce a few firewalls, battle hardened systems and database administrators and internal corporate barriers and the data that seemed so accessible at the outset of the project now might as well be stored in a vault.
Second, once the tool is deployed and accessing data, what exactly will it do and how will it perform? Determining file and database type by its file name extension is one thing; cracking each of them open to examine and index their contents for specific data is quite another. This type of activity almost surely introduces performance overhead on the server and possibly impacts application performance. Also determine how often, deep and efficiently these scans and indexes will run. Scanning an entire 300MB file to locate only a couple of changed bytes may not be the best use of system resources.
Finally, ask yourself what types of files and databases will it classify? For example, if it only scans and indexes 63 percent of the files and databases in your environment, is that enough? And if not, will you need more than one tool to meet your objectives?
Data classification is another necessary step in the ongoing evolution of storage management, but before anyone heads too far down the path of ILM, make sure that your plan has more thought behind it than some vendor's PowerPoint presentation.
SIDEBAR: Building a Storage Transformation Road Map
The basic components of a storage strategy road map by Jim Damoulakis
Building a storage road map begins with three elements: a starting point, a destination and an understanding of the gap between points A and B.
A complete picture requires an honest assessment of four areas: people, process, technology and business demands. Typically, organizations have identifiable strengths and weaknesses in each of these areas. From this assessment several strategic initiatives should emerge and a series of projects associated with each initiative identified.
Essentially, the road map is a prioritized timeline of these projects and activities designed to first stabilize key problem areas, then to optimize high priority functions, and finally improve overall manageability of the environment. Prioritization of individual projects should be based on the following criteria.
Business value: Consideration must be given to the most pressing current problem areas, as well as longer-term service and efficiency improvements. In the world of storage this might include storage availability and recoverability issues, improving processes such as provisioning, or increasing utilization.
Cost: It is inevitable that some items of high importance will be more costly to achieve than others. Budget realities that need to be weighed may result in pushing some projects further out on the timeline.
Level of effort: How difficult is the project to implement - how long will it take, what resources are required, what is the overall impact from a change management perspective.
Organizational readiness: Is the organizational structure in place - the right people in the right roles - to ensure project success. This is an area where a surprising number of companies fall short. They invest in technology, but do not establish the organizational functions needed to maximize the value of their investment.
Internal and external dependencies: A project may depend on the completion of another project on the road map or an event external to the storage road map - a data centre expansion, for example.
Any road map is highly customized to an organization and should be reviewed and adjusted annually, as required.