Rebooting a hung HVM in XenServer majentis technologies

http://majentis.com/2011/03/07/rebooting-a-hung-hvm-in-xen...

better technology solutions RSS Feeds

Home Prole Our Clients Blog

Rebooting a hung HVM in XenServer

Posted by admin in Blog Mar 7 No comments

Citrix XenServer has a problem that seems to have become signicantly more prominent in the latest 5.6 FP1 release. HVM (Hardware Virtual Machine) is a true virtualized server. It uses the Intel VT-x and AMD-V extensions to provide virtualization.. XenServer provides a QEMU based emulation layer for devices. PVM (Para-Virtualized Machine) is a virtualized server that requires modied kernel that is Xen aware to run. This kernel has special ties into Xens ABI, and typically runs faster than an HVM. Windows based VMs are para-virtualized by default. Since Citrix/Xen cannot modify the Windows kernel, they use specic device drivers to provide the para-virtualization layer. Linux based VMs can be PVMs or HVMs, based on their support for Xen. Since Citrix has access to the Linux kernel source, they can directly modify (or distributions can modify) the kernel to talk to the Xen ABI. What this basically ends up doing, is XenServer only supports para-virtualization with specic Linux distributions. If you want to run a distribution that is not supported by Citrix, it by default becomes an HVM. The hung HVM issue Ive seen only affects Linux HVMs. The Linux HVM essentially locks, becoming non-responsive to pings, local console, basically everything. Using the tools provided by XenServer (XenCenter or command line) have no effect. The VM cannot be shutdown or rebooted. If this was a real machine, you could pull the power cord and deal with the results later. At least you could get the machine up and running again. My past efforts to reboot a hung HVM was to migrate or shutdown all the VMs running on the XenServer, and rebooting the XenServer itself. Then bringing the VMs back. This was a task that would usually have to wait until after business hours. The hit on server downtime and impact on users was usually too great to do the reboot during the day. A discussion on the Citrix forums nally brought a solution to light. Its entirely command line based, can be done during business hours, and doesnt affect any other running VM. A good solution. 1. Find the UUID of the hung VM. You can do this via the command line with xe vm-list or via XenCenter. 2. Find the Domain ID of the hung VM. Run list_domains from the command line, and match the UUID with the ID number
id | uuid | state R B B B B H

0 | 2fe455fe-3185-4abc-bff6-a3e9a04680b0 | 47 | 267227f3-a59e-dafe-b183-82210cf51ec4 | 59 | 298817fb-8a3e-7501-11e0-045a8aa860ff | 60 | 46e3d5aa-2f02-dfdc-b053-9a8ac56ec5d1 | 61 | 16cf3204-eb17-5a12-e8d0-c72087bda690 | 62 | 1f9053b5-c6ca-40bb-504e-3017c37e7281 |

1 de 3

27/10/11 10:35

Rebooting a hung HVM in XenServer majentis technologies

http://majentis.com/2011/03/07/rebooting-a-hung-hvm-in-xen...

63 | ddaec491-097a-e271-362b-f2f985e26e4a | 65 | 55f3b225-4f65-d1ea-aa19-add44c5acce7 | 66 | 7adef6fd-9171-5426-b333-6fb1b57b8e60 | 67 | 6046dc13-f70b-8398-56fb-069c22440a7c | 68 | f201cd94-a501-00c2-d21e-8c2f03ea167b | B

B H B B H

In our case, UUID 1f9053b5-c6ca-40bb-504e-3017c37e7281 is hung, which is Domain ID 62. 3. Run destroy_domain on the Domain ID.
# /opt/xensource/debug/destroy_domain -domid 62

4. The VM will still show itself as running, so now, we need to reboot it.
# xe vm-reboot name-label='name of the VM' --force

5. The VM is now rebooted, and you can bring it up as if you had just pulled the plug. That is, check for some disk corruption, etc.

Thanks go to the Citrix Forum folks for helping us come up with a solution. If you enjoyed this article please consider sharing it!

Post comment
Name (required) Mail (will not be published) (required) Website Comment

Pages
Information Technology Server Virtualization Software Management and Development Our Clients

Tags
Backups

Citrix
Home

GlusterFS iSCSI mdadm RAID

XenServer

2009 Webfolio. All Right Reserved.

2 de 3

27/10/11 10:35

Rebooting a hung HVM in XenServer majentis technologies

http://majentis.com/2011/03/07/rebooting-a-hung-hvm-in-xen...

Prole Our Clients Theme brought to you by Site5 | Experts In Reseller Hosting

3 de 3

27/10/11 10:35