8 data migration benefits of logical and conceptual data models

The use of logical and conceptual data models is often a rarity in data migration projects but when adopted correctly they can create significant benefits for both the business and technical community.

In this post we take a closer look at some of the reasons why they can prove so useful.

What are logical and conceptual models?

Most people involved in the more technical aspect of data migration will be well versed in the use of physical data models. Physical models give us a blueprint of how data it is to be stored and related in a particular database system. For this reason, physical models are often far too complex for our needs, particularly in the earlier analysis and discovery phases of a migration.

In contrast, logical and conceptual models provide greater insight into the underlying relationships of the data at a level that is meaningful to the business. (For more information on the difference between physical and logical models, click here).

Logical Models

Logical models provide some of the vital information we need about the entities that matter to the business but without the "noise" of a physical model.

Here are some of the typical elements we record in a logical model:

• Business name of an entity
• Relationships between entities
• Unique key for an entity
• List of attributes

Conceptual Models

The conceptual model has to be one of the most under utilised yet most beneficial resources in any data migration.

A conceptual model works by recording very high level business objects and defining their relationships. For example, we may begin with a simple model that records the relationships between subjects such as:

• Employees
• Customers
• Products
• Equipment etc.

Why are logical and conceptual data models so vital to data migration projects?

At this point some of you may be wondering why these high level models can add value in such a technically focused activity as data migration?

Here are some of the many ways these models can add value to a data migration project:

1. Structuring exploratory business workshops: Every data migration has a learning curve where the project team begins to learn more about how the legacy (and target) businesses operate. By starting your workshop with a discovery activity to create a conceptual data model you immediately focus the attention on data plus you get all parties actively discussing the legacy and target environments at a level that does not descend into minute technical detail.
2. Resolving"turf-wars" and political issues early in the project: By creating a more business focused model we can identify where ownership conflicts may be taking place. In any migration there are political sensitivities as major business change is often a driver for the migration process. By creating higher-level maps we can quickly ascertain where there are overlaps of data ownership that may lead to issues when we wish to decommission the legacy environment.
3. A useful tool for scoping the legacy landscape and allocating appropriate resources: By creating high level business objects we get a far more manageable view of what data is involved in our migration. If we are only presented with a physical model, the sheer complexity and lack of information can delay the scoping process considerably.
4. Great for partitioning workload and "chunking" the migration: Another byproduct of the scoping exercise is that we are more able to allocate the appropriate resources. With our project broken down into higher level subject areas we can identify the necessary expertise for each domain and create more accurate plans for each area.
5. Helps find "data gaps" and "hidden" data stores early in the project: By creating an initial conceptual model then a lower-level logical models we can identify very quickly where there are gaps in the legacy datastores that may need further investigation. For example, our business analysts may indicate that the business manages a relationship between a particular service and a customer segment that does not appear to be found in the physical systems in scope. Very often this data is held in a private store in the form of local paper records or spreadsheets etc.
6. Provides more focused business process analysis: Opinion is mixed on whether business process analysis should form part of a data migration project but there is no escaping the fact that a deep understanding of both the legacy and target environments is highly beneficial to delivering a "fit-for-purpose" migration. By identifying the entity relationships at a more business-focused level it makes deeper analysis of the business processes far easier and relevant.
7. Helps to prioritise migration design and build: Progressive and incremental migrations are now increasingly favoured over the traditional "big-bang" approach. Progressive migrations need a great deal of focus on what data to migrate so that the business can gain maximum advantage. By creating conceptual and logical data models we can more easily converse with the business to understand which data items are critical to success on the target platform and which items can be delayed.
8. Helps to align target application build with the target migration design: Very few migrations have a "firmed-up" target structure at project kick-off. Most target schemas can vary significantly throughout the migration build and this can cause severe delays to the migration and obvious risk. By creating a common logical data model that is version controlled and jointly agreed between both parties it provides a much easier means of communication than complex spreadsheets or design documents.

Have you used logical or conceptual data models in your data migration projects? Were they a help or a hindrance? Why not share your views below?

Useful Resources

Conceptual, logical and physical data models explained

Conceptual schema - Wikipedia

Logical data model - Wikipedia

See all posts in: Data Migration Methodology, Data Migration Techniques

Comments

[Tom Spoors and Philippe Claeys - Rever sa]

This is a very clear post that outlines well data migration issues as we see them at Rever sa (spin-off from Namur University, Belgium) and we agree with the ideas and sentiments expressed. At Rever we belive that we add value to the whole migration process through our software's capacity to reconstruct these models through an automated analysis of all the technical elements related to the database to be migrated. This includes a description of the database (DDL), programs and JCL accessing the database and the data themselves…

What appears to be tedious and repetitive (perhaps even unrealistic) in the approach proposed in the article, becomes simple and quick using REVER technology.

Using this model-driven technology REVER have carried out several migrations for the Ministry of Finance in Belgium, the State of Oklahoma in the USA and BRED in France.
In migration that don’t use models IT staff are limited to a one-to-one migration that means that the result might emulate the columns in the source database but will not be able to take advantage of the full set of features offered in an up-to-date RDBMS. As a result, the migrated system is not compliant with the rules and semantics expected by the users of the target database. Additionally, during migration, there is a period where the system may be frozen causing significant business risk. After migration there is a risk of poor database quality, poor data quality and the system can be difficult to maintain and upgrade. It is also difficult to give new programs access to the data in the system due to a lack of documentation.

In migration systems driven by models it is possible to ensure a redesign of the database to fit within the constraints of the target DBMS. An understanding of the semantic is clearly required but the result is that there is a database that can take full advantage of the features offered in the target RDBMS ensuring that the migrated database complies with the rules and semantic expected by users of the target database. Additionally, during the migration; risks are reduced to a minimum and there is a choice between a ‘phased’ or ‘big-bang’ approach. A reduced downtime for the target database means the new development work can start more quickly. The coherence of the database can be ensured through the ability to integrate new functions or constraints onto the target database. The resulting database is: a high-quality native database, compliant with the target DBMS that is easy to maintain and upgrade thanks to the selection of tools available to simplify maintenance and upgradeability. Tom Spoors and Philippe Claeys (Rever sa)

http://www.rever.eu/rever_new/en