Database migration has always been a critical step in cloud migrations. On-premise databases are often big and perform a critical functions with interdependencies with other systems. Even small databases can be difficult to migrate in some cases.
How much effort is needed to move database to the cloud? What is involved?
This will be the focus of this blog post, which is to provide an overview of the process of database migration from on-premise to cloud.
Let’s start with the type of database migration: Will it be Homogenous or Heterogeneous migration?
Homogenous Migration
When source database on premise and selected target database in the cloud are the same, this is called Homogenous migration.
Example: migrating Oracle database on-premise to same Oracle database in the cloud.
Heterogeneous Migration
When source database on premise is different from selected target database engines in the cloud, this is called Heterogeneous migration.
Example: Migrating from Oracle database to PostgreSQL database or a cloud vendor’s propriety database.
The Homogenous migration is the simpler and less risky option. Heterogeneous migration, involves a complex conversion process, which makes this option more challenging. This is why many customers start with Homogenous database migration, just to get the cloud and once there, they start exploring other types of databases.
Database Migration Options
There are different application migration strategies to the cloud, each dictating a different database migration option. Below are the different application migration strategies with matching database migration options:
– Re-Host. This is also known as lift and shift approach.
An example of this would be a migrating a traditional 3 tier web application (front, middle and data tier), from on-premise hosted virtual machines to cloud-hosted virtual machines. The database is migrated to the same database engine in the cloud. There is little or no change to data structure, code or application source code. This minimizes the effort, risk and time involved in the migration.
– Re-Factor. This approach offers deeper integration with cloud provider services. Using the same 3 tier application, the front end and middle trier will remain the same but the database tier is where the refactoring is applied. The database often is broken into different types of databases such as SQL and NoSQL. In order to take advantage of the cloud’s service provider database as a service and other advanced services, the data structure, code, and application source code is often modified.
– Re-Architect. Many organisation are re-architecting their traditional n-tier applications to take advantage of microservices architecture and containers. The traditional database is often replaced with cloud database offerings such as Dynamo DB or Cosmos DB to gain better performance and availability. This approach requires significant modifications to the data structure, code, and application source code.
– Re-Build. Organisations in this scenario, will often elevate their applications up the cloud technology stack to utilise serverless platform for their n-tier application’s front and middle tiers. The database tier in this approach will be redesigned with significant changes to the database structure and rewriting of the database code and application source code.
– Replace. In this approach, organisations that adopt SaaS applications will need to discard their old data structure, code, and application source code, for the new SaaS provider’s newly built database.
Steps to take for migrating the database to the cloud
1) Upload
In the initial stage, data is copied from on-premise to the cloud over the network. There are several options for bulk upload the data to cloud service provider:
- Internet. Uploading data over the internet might not necessarily be the best option, as it might not be suitable for large volumes of data, due to limited bandwidth.
- Direct connection. This offers the fastest option for moving data by connecting on-premise data centre directly to cloud data centre. Although this alternative is faster than the previous option, it might still not be fast enough for very large volumes of data.
- Physical Media. This is the fastest option for massive data transfer. Data is copied to physical media devices such as disks, appliances or tapes and shipped to the cloud service provider.
2) Import
Once the data is uploaded to cloud storage, it will be imported into the cloud database.
3) Synchronise
At this stage, an older copy of the on-premise database is in the cloud data centre. The next step, is to synchronise, on-premise database with cloud database through the use of a Change Data Capture (CDC) solution (vendor or 3rd party). There are two common approaches:
- Master/Read Replication. Synchronisation is setup to replicate database from on-premise master database to replica database in cloud data centre. All writes are done in master database on premise, then it is synced with cloud replica read only database. Once both databases are fully in sync, the role of master and replica is switched, making cloud database the master. The downtime in the approach is minimal.
- Master/Master Replication. In this option, once the cloud version is in sync with on premise database, a bi-direction replication is setup between both databases, making each a master. The write and read operations will happen in either database. There is no downtime with this option, if done correctly. However, it is also the most complex of all the migration options.
4) Cutover
This the final stage where gradual applications connectivity to on premise database is migrated to cloud database and on premise database is switched off. It is important to note, there is a need to have a plan B, in case something goes wrong after cutover.
Database migration to the cloud can be challenging. It requires high level of expertise in the cloud platform, networking, security, database, etc. But the rewards of running database in the cloud, can outweigh the challenges.
I hope you have found the post informative and thank you for reading.
Nick
