Search code examples
vmwarevirtualizationkvmesxihypervisor

What happens under live migration


I want to understand what happens under the hood in a live migration for execution of my final year project

According to my understanding ,with two host sharing a common storage via SAN

1)When a vm is migrated from one host to another host,the VM files are transferred from one ESXI to another ,but the question is they have a shared storage so how are they going to transfer.

2)VMDK,snapshots files are transferred during live migration

Now I have questions

1)Only VMDK,.vmx files are transferred

2)with VMotion the memory pages are transferred,so what are this memory pages,are they files ,or what are they physically

3)Where is the code for migration present,in hypervisor or VCenter

4)Can we get a stacktrace for vm ,hypervisor during a migration and if yes how would that be possible (I tried a strace to get a basic on how a VM (ubuntu) would call a hypervisor but that only gives me till the linux system and not beyond that )

Can anyone please guide me on this .


Solution

  • VMotion overview

    Phase 1: Guest Trace Phase

    The guest VM is staged for migration during this phase. Traces are placed on the guest memory pages to track any modifications by the guest during the migration. Tracing all of the memory can cause a brief, noticeable drop in workload throughput. The impact is generally proportional to the overall size of guest memory.

    Phase 2: Precopy Phase

    Because the virtual machine continues to run and actively modify its memory state on the source host during this phase, the memory contents of the virtual machine are copied from the source vSphere host to the destination vSphere host in an iterative process. The first iteration copies all of the memory. Subsequent iterations copy only the memory pages that were modified during the previous iteration. The number of precopy iterations and the number of memory pages copied during each iteration depend on how actively the memory is changed on the source vSphere host, due to the guest’s ongoing operations. The bulk of vMotion network transfer is done during this phase—without taking any significant number of CPU cycles directly from the guest. One would still observe an impact on guest performance, because the write trace fires during the precopy phase will cause a slight slowdown in page writes.

    Phase 3: Switchover Phase

    During this final phase, the virtual machine is momentarily quiesced on the source vSphere host, the last set of memory changes are copied to the target vSphere host, and the virtual machine is resumed on the target vSphere host. The guest briefly pauses processing during this step. Although the duration of this phase is generally less than a second, it is the most likely phase where the largest impact on guest performance (an abrupt, temporary increase of latency) is observed. The impact depends on a variety of factors not limited to but including network infrastructure, shared storage configuration, host hardware, vSphere version, and dynamic guest workload.

    From my experience, I would say I am always loosing at least 1 ping during Phase 3.

    Regarding your questions:

    1) All data is transferred over TCP/IP network. NO .vmdk is transferred unless it's Storage VMotion. All details you can find in the documentation

    2) .nvram is VMware VM memory file. All the list of VMware VM file types can be validated here

    3) All the logic is in hypervisor. vSphere Client/ vCenter are management products. VMware has proprietary code base, so I don't think you can get actual source code. At the same time, you are welcome to check ESXi cli documentation. VMotion invokation due to licensing restrictions can be done only via client.

    4) Guest OS (in your case Ubuntu) is not aware of the fact the it uses virtual hardware at all. There is NO way for guest OS to track migration or any other VMware kernel/vmfs activity in general.