The Problem with Data Migration Scoping
Failed data migration projects are often identified in post mortem as having failed because they were "poorly scoped".
What is this poor scoping and how can it be prevented?
In this best-practice article by John Platten we are offered simple techniques to help reduce the problems associated with data migration scoping.
It's always tempting as technologists to look at scoping problems in terms of tables or systems that were missed or were more complex than expected. This is our home ground in which we are trained and experienced and the language with which we feel most comfortable.
Let us assume that we have some preventative measure in mind from past experience or perhaps from other articles on Data Migration Pro.
What shall we say to the client about these countermeasures when we first commence the project?
"We must perform some initial data discovery because there is a risk we might miss important systems, tables and complexity, leading to a scoping problem".
How does that sound to you as a fellow data professional? Fair enough I should think?
Now let’s consider two similar statements from the worlds of medicine and airline construction:
"People should stop smoking because it can damage their DNA leading to cells mutating and replicating in an out of control manner".
"The windows and doors in aeroplanes should have rounded corners because square corners lead to concentrations of stress".
These statements are quite interesting scientifically (well, they are to me), they both took considerable research, experience and expenditure to uncover and they both accurately depict the underlying cause of an issue and the relevant preventative measure.
But how would you actually feel about them as a smoker or an airline passenger?
Suppose I had said instead:
"Smoking causes cancer and cancer can kill the smoker".
"Aeroplanes with square windows can crash and kill their passengers ".
That would more likely make you sit up and take notice, possibly to the extent of checking the windows for rounded corners the next time you take a flight. (Don’t worry. All pressurised jet aircraft now have these). More to the point, it was the severity of the symptoms in both of these cases that led to considerable effort and expenditure to be deployed on investigating cures and mitigations.
So let us put our own case in data migration more strongly and directly:
"Failure to take adequate preventative measures in data migration can cause cost overruns, time overruns, missed targets, complete application project failure and in some cases actual commercial damage to the client and their reputation".
The point is clear - the client’s opinion must take the following path if you are to involve and convince them of why mitigation is necessary.
This is true even when your opposite number, the client migration manager, is already a convert from their own reading or experience. In these welcome cases you will still often need to take your case jointly to the project board in order to have funding released.
The Language of Failure
While client managers may talk in terms of a scope failure during migration post mortem, the data migration professional (assuming that one was hired), is more likely to quote the following causes:
"I warned the client in advance but they did not listen".
"There was insufficient study before plans were laid and budgets were set".
"They would not put time and money into risk mitigation".
Here is the rub: This is often still your fault.
Clients hire data migration professionals not only to execute data migration but to ensure it is problem free. Once the hire has been accepted the migration professional must deliver that outcome including engaging the business in adequate risk mitigation.
The case for countermeasures must be clearly made in the client’s own terms, emphasising not tables, data, profiling and other technical matters but the consequences of poor prediction of cost, duration and expertise requirements on business outcome and ultimately commercial success.
Identify the reason the client business is migrating between applications in the first place.
What potential benefits does the wider project claim in its own business case document and how might these be undermined were the migration to fail?
Is the migration tied to a specific date of opportunity such as a year end and what happens if you miss it?
Does everyone have to stay hired till the next window in an entire quarter’s time?
How much would that cost in consultant fees?
What would the business landscape look like were the company forced back onto their old system by project failure or worse still caught in the middle unable to go forwards or back for an extended period?
Can the business actually run for several weeks without a system while the mess is sorted out or will it be missing deliveries and turning away orders?
Project sponsors will only release funds into a mitigation exercise when they can see a clear business case. When we fail to achieve this as migration professionals we should perhaps look first at our own approach, persuasiveness and ability to communicate with the business, rather than leap to suggest the client was deaf, headstrong or uninterested in our proposals.
Surely, the worst migration failure of all is to have best practice solutions in your pocket yet fail to create an environment in which they can be deployed.
In summary, to avoid The Problem with Scoping:
Put down those data maps and schemas
Stop talking to your sponsors about profiling, tables, systems and interfaces
Have a difficult but necessary conversation on funding early investigation
Begin your case with the business consequences of failure
Make this relevant to the specific migrated application and business concerned
If they don’t listen at first, recheck the clarity and relevance of your business case
Above all persevere