Will legacy data kill your Cloud migration?

Rob Livingstone

Rob is a respected and experienced CIO, with more than three decades of industry and ICT experience. Over the last 16 years he has held the CIO role at several multinationals, most recently Ricoh. He is the owner of Rob Livingstone Advisory and a Fellow of University of Technology, Sydney. Rob delivers the Pathways Advanced and Business ICT leadership programs in conjunction with the CIO Executive Council.

You never know how much ‘stuff’ you really own, until you have to move house. Likewise, moving any enterprise IT system from one platform to another is challenging when it comes to managing legacy data. It is even more challenging when it comes to shifting this data into the Cloud.

If you, as a CIO, are surrounded by influential business stakeholders asking why you have not moved everything to the Cloud yet, send them a copy of this article!

Many organisations have hoards of accumulated legacy data on their on-premise IT infrastructure. Some of the data may need to be stored for a defined period of time due to record retention legislation. Other data may have no value. Managing legacy data presents its own set of unique challenges.

Broadly speaking, legacy data falls into three categories:

• Structured data include sales, shipping, payroll and accounting transactional data. It is typically precisely defined, and is ‘locked-up’ within enterprise applications such as CRM, ERP, payroll, leasing and finance systems and so on. It consists of many tables, typically in the forms of relational databases.

• Semi-structured data, is tagged or labelled so that it can be grouped or categorised. An e-mail message, for example, does not fit into a pre-defined data format, but is tagged with sender, date and so on, for sorting and categorisation.

• Unstructured data includes the myriad of separate files, often in the form of Word, Excel, images, multimedia files and others, spread across network drives and various servers, PCs and network storage systems. It has no master catalogue or management interface. Some call it the organisational data ‘black hole’.

Considerations in shifting legacy data to the Cloud

In the case of structured data, there are no universal data interchange standards. Organisations must map and transform legacy data to a format that is acceptable to the Cloud system. Even then, there may not be an exact match. Specialised Cloud based data-mapping and migration software is now available, but it adds cost and additional complexity to the overall exercise.

Regardless of migration path, the conversion can still be wrapped up in technical complexity. Due to the standardised nature of most software-as-a-service (SaaS) systems, your ability to modify databases in order to accept your legacy data is severely limited, although you can often map data to extension or custom fields.

Full data conversion can be difficult and costly, bound by the restriction of the platform’s integration offering (if it exists, at all) and application programming interfaces (APIs).

The reality is that your on-premise database system is rarely compatible with Cloud-based SaaS, unless it is from the same vendor. Ask yourself: What conversion process needs to be implemented? Have you realistically costed the migration effort, and if so, what does it mean for the project’s bottom line? The migration of your structured data could be a showstopper to your Cloud plans.

The most pervasive form of semi-structured data is email, and almost all email systems have standard export and import tools. This is not a major headache for the average enterprise.

Unstructured data, however, often represents the largest volume of data. Paradoxically, this can be the easiest to shift into the Cloud; essentially a case of ‘cut-and-paste’ to Cloud storage. But the time taken to shift large volumes of legacy data over the internet can be hours, days or weeks, depending on your network and internet speed.

As with any change to complex enterprise applications and databases, the devil is in the detail and comprehensive due diligence is key. If the primary driver in moving to the Cloud is cost, test with absolute certainty that you will be able to shut down the outgoing system.

Tags: legacy, databases, integration, migration, Cloud