One of my clients has a Microsoft Failover Cluster built from 3 physical servers with Windows Server 2008 R2. They want to reduce costs, so decided to move to VMware. The cluster runs 4 instances of Progress databases which serving as the backend of a critical business application, so the move must be done with the least possible downtime (as usual :)). I evaluated 3 distinct scenarios to assess downtime, task complexity, resource requirements and risks.
Extend the existing Failover Cluster with VMs and evict physical nodes
This is the least complex scenario as it simply extends the running cluster with 3 new nodes. Only 3 new VMs with Progress has to be deployed to ESXi and downtime only necessary to fail over from physical nodes. In theory, there is no risks involved to this operation. Here is the official VMware guide how you have to configure your VMs in such a scenario.
Build a new Failover Cluster and migrate cluster resources from old cluster
With the built-in cluster migration tool it is possible to migrate many types of resource groups to a new cluster. For this scenario we need to build a new 3 node 2008 R2 cluster, add cluster disks as RDM to the new nodes, migrate resource groups and stop old cluster. This definatelly requires more efforts and uncertainty lays on Progress resource migration capability.
Convert P2V
This scenario involves VMware vCenter Converter to convert a powered-on physical machines to virtual machines. VMware Converter will convert the operating system drive of each node and the operating system partition of each drive. Data and quorum drives will be mapped as RDMs. Since original server settings remains need careful coordination of bringing converted VMs online, adding RDMs to its configuration and shutting down converted physical servers. After the conversion the servers optionally can be re-named to align with the standard naming convention which needs to evict, rename and add back each node one-by-one. Firewall ports needs to be open for successful operation.
I approached this project carefully, since the last attempt of the migration was failed and this time I planned extra tests before move on with migration. Last time the File Share resource on this cluster stuck to the virtual node and was unable to fail over to other nodes in the cluster. At the end the File Share resource migrated to a newly built cluster. The root cause has never been identified.
The production cluster must be prepeared properly, no mistakes allowed, because it would affect availability of the application. I devided the steps into 5 phases:
- Preparation on ESXi side
- Deploying new VMs
- Join the first VM to the cluster
- Join remaining VMs to the cluster
- Migration
First I had to collected all LUNs used by the Failover Cluster that later added to ESXi cluster. To avoid unexpected downtimes the newly built virtual cluster nodes must be placed on ESXi hosts, which aren’t hosting any other VMs configured with RDM. In case of an unexpected issue with an unresponsive ESXi host we can safely reboot it without affecting other production clusters.
In the second phase I deployed 3 VMs on ESXi cluster. The CPU and memory configuration was identical to physical servers. Later this can be reviewed and amend as necessary.
The third phase was the first point where we interacted with the production cluster. The new VM configured with one of the LUNs from the physical cluster as an RDM. Here we did cluster validation tests before join the new VM to the cluster. And right here we ran into an error reported in the Storage tests area, more specifically the MPIO-based disk validation failed. I suspected that this is because the VM is not yet joined to the cluster, but the customer wanted a full explanation and I wasn’t 100% sure, so I asked Microsoft and VMware to confirm our configuration. They found it correct at the and I also found out that this is expected, because of the different layers where the MPIO managed. So we moved ahead and added the new VM to the physical cluster without running storage validation tests. This went without issues and we could successfully failed over a test resource to the newly joined node.
The fourth phase is basically the same as the third, we added the remaining new VMs and repeated the same tests.
Finally in the fifth phase we failed over all resources to VMs and evicted the physical nodes from the cluster.