Introduction

Data migration is the process of transferring data from one system to the other, which can necessitate changing the storage type, formats, database, application or computer systems.

Reasons for embarking on a data migration project might be due to a need to upgrade hardware platforms or servers, change of storage equipment, database upgrade or migration to a new software application. It is a complex undertaking and it’s a branch of IT in which the processes and software tools being used are continually evolving.

Making mistakes in a data migration project can lead to loss of data and this might be very costly for a business and organization embarking on the project. Nonetheless, if choices are made carefully and the project is adequately planned, the migration project would be completed in a timely manner without any problem afterwards.

Data Migration can be in the form of

• Storage Migration: with the current advancement in storage technologies, it is not uncommon for businesses to choose to take advantage of more efficient and effective storage technologies, thus, moving data from one hardware storage platform to another, which can be local disk, iSCSI or Fibre Channel (FC). This would necessitate the movement of blocks of data from one storage medium to another, the format of the data is usually not changed in this process.
• Database Migration: this is the most common form of migration, it entails the upgrade of a database version or moving from one database vendor to another. A database migration project might lead to change in the data format which may or may not affect the behavior of the application and as such a validation process is required at the end of the migration procedure. A vendor change can be from Mysql, Sybase or Oracle to SQL Server.
• Application Migration: every software application has, and can operates its own specific data model or schema, changing the application vendor can trigger a migration project, say changing a business’ ERP vendor from SAP to Sage will definitely require migration of the data from the existing application to the new application. Application migration can also be in the form of application environment change such as moving an application from an on premise server to a cloud provider’s environment.

Data Migration Methodology

Data migration usually involves four steps or processes


Assessment: this phase involves information gathering about the migration environment, a thorough assessment of the data to be migrated, its current format and the new format it will take after migration, risks assessment is also done to evaluate any challenges that might be faced in the process. At the end of this phase, a report is usually generated to determine the cost, risks and time frames of the migration.

Checklist

• Evaluate and analyze the data to be migrated
• Evaluate the current and future environments
• Define appropriate standards

Planning: this process usually involves the determining the migration requirements, time schedules, while identifying the current and future environments. At the stage, the hardware or software needed to successfully carry out the migration is identified. It is a good practice for the IT staff to carry the business owners along in the planning process, this might help determine potential problems.

Checklist

• Document Plan
• Notify users of the migration schedule
• Set a realistic time frame
• Prepare a test plan

Migration: This is the actual migration phase also known as “Move”, at the stage the IT staff(s) involved in the migration would configure the software tools and or hardware required for the migration. Software tools can speed up the process while reducing or eliminating business downtime.

Checklist

• Revise migration plan
• Configure migration environment
• Perform backup or replication of data to be migrated
• Initiate migration process
• Prepare migration report

Validation: data validation is required upon completion of the migration to ensure the integrity of the data. A migration statistics and reports should be prepared, this should include what worked and what didn’t during the migration. The steps used in the migration should be duly documented to simplify future migrations of the same pattern. Acceptance testing should be done by the business owners. Verify database schema and validate that the data was migrated correctly

Checklist

• Test data to ensure integrity
• Perform roll back if data corruption occurs

Choosing appropriate data migration software

The choice of the migration software should take into consideration the OS environments, source and target hardware platforms and the database vendors and versions in the case of a database migration project.

A key requirement to consider is the ability to roll back the migration should in case something goes wrong. This is a great concern in the eventuality of a problem or an error, the migration software should be able to terminate, restart or roll back entirely the process.

The performance of the software should be considered when making choice, how quickly the data is copied from the source environment to the target environment. Performance should be balanced against system overhead and network bandwidth.

Storage Migration software tools

• Rsync: its open source and provides fast incremental file transfer.
• IBM Softek Transparent Data Migration Facility can migrate data at the volume or block level, supports multiple platforms.
• SanXfer: can migrate server data volumes and is optimized for cloud environments.
• Hitachi Data Systems Universal Replicator: provides storage array-based replication.

Database Migration software tools
Most database vendors ship migration tools with their products, nevertheless, there are some readily available commercial software tools

• Microsoft SQL Server Migration Assistant (SSMA) for migrating MySQL or Oracle database to SQL Server.
• MySQL Workbench Wizard for easy database migration to MySQL.
• Oracle SQL Developer provide tool for migrating non Oracle databases to Oracle.
• EDB Migration Toolkit for database migration to Postgres.
• There are other commercial products used for database migration such as Flyway, SwisSQL, ESF Database Migration Toolkit, Ispirer’s SQLways

Application Migration Software tools

Application migration is usually complex and complicated and as such application vendors normally provide custom tools to migrate data from one application to the other,

Data Migration Best Practices

• Migration projects are time consuming and as such a proper planning is crucial to the successful implementation of the project.
• It is important to understand the data being migrated, the current and future storage as well as the current and future form or format.
• A duplication or replication of the data is advised before migration. Implement data migration policies so data is moved in an orderly manner.
• A proper shutdown or downtime plan should be put in place for duration of the migration project, with which the business would not have access to the data.
• Accuracy of the migrated data should be ensured using test and validation procedures.
• Adequate documentation of the entire process could simplify related future projects.

Conclusion

Data migration is often a very complex process, thus a good working cooperation is required among the IT staffs, the business owners and any stakeholder involved in the migration. Following best practices and choosing the suitable software for the project can facilitate smooth and timely completion of the project.