Recently, I've been consulting with a mid-sized organisation who are embarking on an initiative to implement a new inventory system.

With just a few days in there have been some serious lessons learned by the organisation on how to approach data migration in the right way.

Some recent data migration lessons

For a company of this size the new target system will have a dramatic effect on how they not only manage their operations but provide new services that they hope to roll out in the face of some serious competition in the sector. Prior to the new system the company had a mix of different databases, spreadsheets and even local paper records out in the engineering centres.

All that will change in the next few months when the new system comes online. However, they have the small task of the data migration to contend with first.

What has interested me most about engaging with this company is the perception they had about the data migration prior to our consultation. Data migration was seen simply as an IT technique, bolted on at the end, tool driven, tech-focused, limited business impact and highly tactical. I think a lot of this perception was driven by data volumes, which by industry standards, are indeed quite small. However, there is a tremendous amount of complexity in the way this organisation does business. What's more, in the new target system there will be many more demands placed upon the data.

As we assessed the data it was also clear there were major data quality issues that had not been addressed for many years.

So, we've put together a plan and we're moving forward but I think there are a few lessons that are probably worth sharing with other organisations who may find themselves in a similar position:

Data migration is a strategic enabler - treat it like one

Data migration may look tactical in nature but it has the ability to completely destroy any credibility in the strategic system you are about to implement if you get it wrong.

Consider data migration a strategic weapon and move it up the priority and awareness stack.

Joining the party late is very unfashionable

It is extremely difficult to make improvements on a data migration when the main programme is in full swing. Data migration should also be viewed as a separate project that should ideally commence with the start of the main programme.

Leave it late and your system go-live is almost guaranteed to slip.

Data quality is mandatory

The company in question did not invest in a data quality framework. Notice, I use the word framework, not tools because tools don't create solutions. In their words - "We didn't want to do a big data cleanse" highlighted their lack of experience in how data quality tools and skills should really be adopted on a migration project.

This is a big topic that we've covered in numerous places on the site but there needs to be a complete DQ framework in place from day 1 of the migration otherwise every stage of the project is open to risk, delay or failure.

Get help at the start

It's often tempting to ask for data migration expert assistance part way through the main programme as your target application won't really take shape until that point. However, specialists arriving mid-way through the project are akin to a lone policeman showing up at a teenage end-of-term party that has got way out of hand. No-one wants to stop, no-one wants to take responsibility and those who planned it are wondering how much trouble they will face from their seniors.

If you have doubts about your capabilities to deliver the project then seek expert advice right from the outset, before the migration has even been planned.

Complexity is more important than volume

Although the business in question have low volumes of data, their business model is complex. Most businesses look to new target systems as a means of changing or improving their current business model. This change often places pressures on the legacy data to perform amazing new feats of business agility that sadly it isn't fit for.

Assess the complexity of your underlying data models and relationships within the data, both in the target and source systems. This has to be fully understood, data quality management is critical here for measuring the gaps and defects. Many companies mistakenly view a small data load as a simple migration and fail to factor in the complexity dimension.

Conclusion

All of these lessons point to the fact that data migration should be granted strategic importance.

Ensure that your organization understands the critical role it plays in bringing high-quality applications into existence and work harder to increase the awareness of this much misunderstood discipline.

Useful Resources

6 tips for "selling" a best-practice data migration

Promoting DQ Tools for Data Migration Projects

The data migration go-live strategy - what is it and why does it matter?

Data Migration Survey of Global 2000 Organisations

Does your data migration strategy have an effective plan for systems retirement?

8 Data Migration Resolutions for 2009

Data migration project checklist - a template for more effective data migration planning

Extreme data Pruning - How to Cut Effort, Complexity and Cost in your next Data Migration

Six Misconceptions about Data Migration