I inherited a xen server that is setup to have all the VM images hosted on an iSCSI mounted NAS array. We been experiencing a random (about every 2-3 days) issue where xen would crash all the VMs, leaving nothing but the Domain0 running. What appears to be happening is something causes the iSCI mount to hiccup. Running "vgchange -a y" and restarting all the VMs brings everything up. Nothing appears to be wrong with the NAS array - there are a dozen other servers attached to it that never have a problem. The xend log does not have anything useful in it and I'm at a loss to figure out what is causing this. The only suggestion I've heard is maybe the memory usage is too high and it is causing the box to be unstable. If anyone has any suggestions or any additional logs I should be looking at, I'd really appreciate it.
Host OS: CentOS 5.5
Xen kernel: xen.gz-2.6.18-194.11.4.el5
iSCSI libraries: iscsi-initiator-utils-6.2.0.871-0.16.el5
Memory on server: 32G
Total memory allocated for VMs running paravirt: 19,384 M
Total memory allocated for VMs running HVM: 2,688 M
Results of xm top:
xentop - 10:11:06 Xen 3.1.2-194.11.4.el5
39 domains: 1 running, 38 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown
Mem: 25165116k total, 25150528k used, 14588k free CPUs: 8 @ 1995MHz
NAME STATE CPU(sec) CPU(%) MEM(k) MEM(%) MAXMEM(k) MAXMEM(%) VCPUS NETS NETTX(k) NETRX(k) VBDS VBD_OO VBD_RD VBD_WR SSID
Domain-0 -----r1583 17.1 3220540 12.8 no limit n/a 8 32 1932 32747 0 0 0 0 0
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users