Re: RHEL5.5, 32-bit VM repeatedly locks up due to kvmclock

> > > On 04/23/2010 03:39 PM, Zachary Amsden wrote: > > On 04/23/2010 10:39 AM, Brian Jackson wrote: > >> On Friday 23 April 2010 12:08:22 David S. Ahern wrote: > >> > >>> After a few days of debugging I think kvmclock is the source of lockups > >>> for a RHEL5.5-based VM. The VM works fine on one host, but repeatedly > >>> locks up on another. > >>> > >>> Server 1 - VM locks up repeatedly > >>> -- DL580 G5 > >>> -- 4 quad-core X7350 processors at 2.93GHz > >>> -- 48GB RAM > >>> > >>> Server 2 - VM works just fine > >>> -- DL380 G6 > >>> -- 2 quad-core E5540 processors at 2.53GHz > >>> -- 24GB RAM > >>> > >>> Both host servers are running Fedora Core 12, 2.6.32.11-99.fc12.x86_64 > >>> kernel. I have tried various versions of qemu-kvm -- the version in > >>> FC-12 and the version for FC-12 in virt-preview. In both cases the > >>> qemu-kvm command line is identical. > >>> > >>> VM > >>> - RHEL5.5, PAE kernel (also tried standard 32-bit) > >>> - 2 vcpus > >>> - 3GB RAM > >>> - virtio network and disk > >>> > >>> When the VM locks up both vcpu threads are spinning at 100%. Changing > >>> the clocksource to jiffies appears to have addressed the problem. > >>> > >> > >> Does changing the guest to -smp 1 help? > >> > >> > > > > Based on our current understanding of the problem, it should help, but > > it may not prevent the problem entirely. > > > > There are three issues with kvmclock due to sampling: > > > > 1) smp clock alignment may be slightly off due to timing conditions > > 2) kvmclock is resampled at each switch of vcpu to another pcpu > > 3) kvmclock granularity exceeds that of kernel timespec, which means > > sampling errors may show even on UP > > > > Recommend using a different clocksource (tsc is great if you have stable > > TSC and don't migrate across different-speed machines) until we have all > > the fixes in place. > > That's my plan for now. As I recall jiffies was the default in early > RHEL5 versions. Not sure what that means hardware wise. > > The biggest problem for me is that RHEL5.5 defaults to kvmclock; I'll > find some workaround for it.