Re: [PATCH] increase ple_gap default to 64

Previous thread: Network Virtual Terminal by Rodolfo Giometti on Monday, January 3, 2011 - 8:15 am. (2 messages)

Next thread: by gavin willock on Monday, January 3, 2011 - 8:52 am. (1 message)
From: Rik van Riel
Date: Monday, January 3, 2011 - 8:19 am

On some CPUs, a ple_gap of 41 is simply insufficient to ever trigger
PLE exits, even with the minimalistic PLE test from kvm-unit-tests.

http://git.kernel.org/?p=virt/kvm/kvm-unit-tests.git;a=commitdiff;h=eda71b28fa122203e3...

For example, the Xeon X5670 CPU needs a ple_gap of at least 48 in
order to get pause loop exits:

# modprobe kvm_intel ple_gap=47
# taskset 1 /usr/local/bin/qemu-system-x86_64 -device testdev,chardev=log -chardev stdio,id=log -kernel x86/vmexit.flat -append ple-round-robin -smp 2
VNC server running on `::1:5900'
enabling apic
enabling apic
ple-round-robin 58298446
# rmmod kvm_intel
# modprobe kvm_intel ple_gap=48
# taskset 1 /usr/local/bin/qemu-system-x86_64 -device testdev,chardev=log -chardev stdio,id=log -kernel x86/vmexit.flat -append ple-round-robin -smp 2
VNC server running on `::1:5900'
enabling apic
enabling apic
ple-round-robin 36616

Increase the ple_gap to 64 to be on the safe side.  

Is this enough for a CPU with HT that has a busy sibling thread, or 
should it be even larger?   On the X5670, loading up the sibling thread
with an infinite loop does not seem to increase the required ple_gap.

Signed-off-by: Rik van Riel <riel@redhat.com>

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 81fcbe9..0e38b8e 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -90,14 +90,14 @@ module_param(vmm_exclusive, bool, S_IRUGO);
  * These 2 parameters are used to config the controls for Pause-Loop Exiting:
  * ple_gap:    upper bound on the amount of time between two successive
  *             executions of PAUSE in a loop. Also indicate if ple enabled.
- *             According to test, this time is usually small than 41 cycles.
+ *             According to test, this time is usually smaller than 64 cycles.
  * ple_window: upper bound on the amount of time a guest is allowed to execute
  *             in a PAUSE loop. Tests indicate that most spinlocks are held for
  *             less than 2^12 cycles
  * ...
From: Zhai, Edwin
Date: Monday, January 3, 2011 - 8:21 pm

Riel,
Thanks for your patch. I have changed the ple_gap to 128 on xen side, 
but forget the patch for KVM:(

A little bit big is no harm, but more perf data is better.



--

From: Rik van Riel
Date: Tuesday, January 4, 2011 - 7:18 am

So should I resend the patch with the ple_gap default
changed to 128, or are you willing to ack the current
patch?

-- 
All rights reversed
--

From: Avi Kivity
Date: Tuesday, January 4, 2011 - 7:29 am

I think 128 is safer given than 41 was too low.  We have to take into 
account newer cpus and slower spin loops.  If the spin loop does a cache 
ping-pong (which would be a bad, bad possible, implementation), even 128 
might be too low.

-- 
error compiling committee.c: too many arguments to function

--

From: Zhai, Edwin
Date: Tuesday, January 4, 2011 - 7:35 am

Agree with Avi. Let us use 128 at this point.
Thanks,
edwin
--

Previous thread: Network Virtual Terminal by Rodolfo Giometti on Monday, January 3, 2011 - 8:15 am. (2 messages)

Next thread: by gavin willock on Monday, January 3, 2011 - 8:52 am. (1 message)