Migration

Relevant stakeholders such as the system owner, key users and representatives from business units which will need and depend on the information should consulted when considering a migration project. Engaging with appropriate stakeholders will enable technical and recordkeeping issues to be identified and addressed. Stakeholder needs and preferences will inform what, when and how to migrate and ensure that records are available when needed.

Involving stakeholders may help to reduce the volume of data to be migrated and enable migration to focus on records that continue to be needed or are required to be maintained. For example, a business unit or system owner will be able to identify records that are in current use and records that are not in use and can remain in the decommissioned system or be stored offline until their retention period ends.

Information technology staff will need to be fully involved in the project. This may be internal staff only or could also involve external service providers.

Understanding the functionality of the system and how it is configured helps to ensure the integrity and continuity of records during migration. When records are migrated between systems, it is likely that there will be structural, technical or configuration differences which may impact records.

To minimise risk to records, gain an understanding of:

• the export and import capabilities of both systems, including any limitations or data transformation requirements
• how metadata is captured, stored, and managed in both systems
• how files and objects are captured and managed across both systems
• how relationships between objects, files, metadata and other record-related relationships are managed across both systems, and how they will be mapped to ensure the contextual integrity of records
• any information types or formats that the source system uses which cannot be captured or managed by the replacement system
• work required to maintain or replicate system integrations
• technical requirements for migrating access controls, workflows and audit and action logs.

Careful consideration and planning is needed to ensure successful migration. A comprehensive migration plan should be in place to outline the required steps, responsibilities and timeframes. Identifying and managing the risks associated with migration will support the success of your migration project.

The migration plan should be approved by the appropriate senior officer/executive and include:

• which records and metadata are to be transferred
• the end location for migrated records
• how metadata will be mapped to the schema used in the new drive or system
• any format and metadata transformations that may need to take place
• any data cleaning activities that need to take place
• risks, mitigation strategies and recovery plans
• how migration will be tested and validated to ensure records and metadata have been transferred successfully
• plans for source record management or disposal and system decommissioning
• roles and responsibilities for migration tasks
• quality control and checking processes during and at the conclusion of the migration
• the milestones for sign-off at different points in the project and the officers/committees authorised to do this,

The migration plan is a key document and any changes to the project should be reflected in it.

Understanding what records are held, where they are held and if there are any high value or high-risk records that need to be prioritised will help to establish what pre-migration work needs to take place.

This could include activities such as:

• pre-migration testing
• data cleaning including removing duplication, out of date or incorrect information
• data validation
• data and metadata mapping and transformation.

The value and usefulness of records is reduced if contextual metadata such as creation dates, author and authorisation details, version information or relationships to other records is lost. Identifying key information and preparing your records can help solve potential issues before the migration begins.

Information can be lost if one of the systems does not have the ability to import or to export records, data formats are incompatible, metadata schemas do not match, or if the migration is interrupted.Records must be tested and validated after migration to ensure they are complete and accurate.

Example: Where multiple metadata fields in the existing system are combined into one metadata field in the new system, such as First Name and Last Name combined into a Full Name field, metadata transformations should be completed to prevent information loss.

The accuracy and integrity of data can be compromised, impacting their trustworthiness and value. This could be due to changes in the metadata associated with a file or object being changed because of the migration in the way that changes its context.

Example: When records are migrated, it is common for the create date of records or files to be replaced with the migration date. Pre- and post-migration testing must be conducted to ensure records are transferred in a way that protects the integrity of the data.

The information may not display properly or the system may not be able to process the information correctly due to differences in the way the systems handle data. Differences in the structure or quality of the data may result in the records being displayed or processed differently in the importing system. 

Example: Changes in the format of dates (i.e. DDMMYY to YYYYMMDD) will need to be validated to ensure they display correctly. These differences may cause the importing system to default to an inaccurate date if the information is incomplete. For best results, conduct data quality improvements and amendments prior to migration.

Locating accurate information may be difficult due to business processes and procedures not being updated to reflect the new system. This can lead to broken links in documentation and confusion regarding who is responsible for maintaining what information.

Example: Changes to system functionality that affect the way information is located must be documented to ensure that stakeholders are able to access and use information when needed.

Pre-migration testing should check for potential issues and confirm that records and associated metadata can be migrated in a way that maintains their completeness, reliability and authenticity. Testing should be informed by your risk assessment to ensure risks are minimised. Where possible, pre-migration testing should occur in a test environment that is as close to the live environment as possible.

Once migration occurs, you must establish that records have been successfully migrated before the old system can be decommissioned metadata.

Testing should demonstrate that records:

• are complete, with no missing content
• can be viewed successfully
• are searchable and useable
• have complete and reliable metadata
• were not corrupted during the migration.

Migration tools can be used to assist with the migration process in some instances. For example, obtaining a comparison report of what was migrated compared with what was in the prior system can help identify any missing content. All records and metadata should be retained in the source system until the integrity and reliability of the migrated content in the destination system has been confirmed.

The details of your migration project should be carefully documented. Any changes to the data should be recorded along with any alteration, loss of functionality or change to the structure, content or appearance of records that may occur during the migration process. Thorough documentation of the project will support the reliability and authenticity of the migrated records.