Re: [PATCH] netfilter: use per-cpu spinlock rather than RCU (v3)

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Stephen Hemminger
Date: Wednesday, April 15, 2009 - 1:55 pm

Looks like there is some recursive path into ip_tables that makes the
per-cpu spinlock break.  I get lockup's with KVM networking.

Suggestions?

[ 1931.788787] virbr3: port 1(vnet2) entering forwarding state
[ 2106.068500] Pid: 12397, comm: kvm Not tainted 2.6.30-rc1 #6
[ 2106.068501] Call Trace:
[ 2106.068503]  <IRQ>  [<ffffffff80393a09>] ? _raw_spin_lock+0xdd/0x105
[ 2106.068513]  [<ffffffffa024041d>] ? ipt_do_table+0x102/0x5f1 [ip_tables]
[ 2106.068516]  [<ffffffff8048d4e0>] ? skb_checksum+0x4c/0x257
[ 2106.068521]  [<ffffffffa025f1fc>] ? manip_pkt+0x80/0xf4 [nf_nat]
[ 2106.068525]  [<ffffffffa025f432>] ? nf_nat_icmp_reply_translation+0x1c2/0x245 [nf_nat]
[ 2106.068528]  [<ffffffff8048f632>] ? __alloc_skb+0x6b/0x13d
[ 2106.068536]  [<ffffffffa0246cfe>] ? nf_conntrack_in+0x45e/0x5ac [nf_conntrack]
[ 2106.068540]  [<ffffffffa01602b1>] ? nf_nat_fn+0xc1/0x14d [iptable_nat]
[ 2106.068544]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2106.068547]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2106.068550]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2106.068552]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2106.068555]  [<ffffffff80393925>] ? _raw_spin_unlock+0x8b/0x92
[ 2106.068557]  [<ffffffff804ba8c7>] ? __ip_local_out+0x98/0x9a
[ 2106.068559]  [<ffffffff804ba8d2>] ? ip_local_out+0x9/0x1f
[ 2106.068562]  [<ffffffff804babb4>] ? ip_push_pending_frames+0x2cc/0x33e
[ 2106.068566]  [<ffffffff804dac79>] ? icmp_send+0x559/0x588
[ 2106.068569]  [<ffffffff8022d3a0>] ? task_rq_lock+0x46/0x79
[ 2106.068571]  [<ffffffff8023004f>] ? enqueue_task_fair+0x23b/0x293
[ 2106.068575]  [<ffffffffa00f5083>] ? reject_tg+0x41/0x30e [ipt_REJECT]
[ 2106.068578]  [<ffffffffa024084f>] ? ipt_do_table+0x534/0x5f1 [ip_tables]
[ 2106.068581]  [<ffffffff804b37a0>] ? rt_intern_hash+0x46f/0x48a
[ 2106.068584]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2106.068587]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2106.068589]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2106.068591]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2106.068594]  [<ffffffff804b8070>] ? ip_forward+0x2a1/0x321
[ 2106.068596]  [<ffffffff804b69c6>] ? ip_rcv_finish+0x31e/0x338
[ 2106.068598]  [<ffffffff804b6c15>] ? ip_rcv+0x235/0x27a
[ 2106.068601]  [<ffffffff804957ee>] ? netif_receive_skb+0x4b6/0x4f3
[ 2106.068604]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2106.068610]  [<ffffffffa0228a24>] ? br_handle_frame_finish+0x110/0x148 [bridge]
[ 2106.068616]  [<ffffffffa022cdf9>] ? br_nf_pre_routing_finish+0x300/0x30f [bridge]
[ 2106.068619]  [<ffffffff804b01ff>] ? nf_hook_slow+0xf3/0x104
[ 2106.068625]  [<ffffffffa022caf9>] ? br_nf_pre_routing_finish+0x0/0x30f [bridge]
[ 2106.068631]  [<ffffffffa022d51e>] ? br_nf_pre_routing+0x716/0x730 [bridge]
[ 2106.068633]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2106.068639]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2106.068641]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2106.068647]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2106.068652]  [<ffffffffa0228bf8>] ? br_handle_frame+0x19c/0x1c0 [bridge]
[ 2106.068655]  [<ffffffff804956d2>] ? netif_receive_skb+0x39a/0x4f3
[ 2106.068657]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2106.068660]  [<ffffffff804958aa>] ? process_backlog+0x7f/0xb4
[ 2106.068663]  [<ffffffff80493b65>] ? net_rx_action+0xa8/0x1d5
[ 2106.068665]  [<ffffffff80493c4f>] ? net_rx_action+0x192/0x1d5
[ 2106.068668]  [<ffffffff8023f041>] ? __do_softirq+0xac/0x173
[ 2106.068671]  [<ffffffff8020c1cc>] ? call_softirq+0x1c/0x28
[ 2106.068672]  <EOI>  [<ffffffff8020d3be>] ? do_softirq+0x3a/0x82
[ 2106.068676]  [<ffffffff80495e89>] ? netif_rx_ni+0x19/0x1d
[ 2106.068680]  [<ffffffffa026d4ae>] ? tun_chr_aio_write+0x347/0x3a6 [tun]
[ 2106.068684]  [<ffffffffa026d167>] ? tun_chr_aio_write+0x0/0x3a6 [tun]
[ 2106.068687]  [<ffffffff802b480d>] ? do_sync_readv_writev+0xc0/0x107
[ 2106.068691]  [<ffffffff8024d8bf>] ? autoremove_wake_function+0x0/0x2e
[ 2106.068701]  [<ffffffffa01eea62>] ? kvm_vcpu_ioctl+0x5eb/0x5fe [kvm]
[ 2106.068704]  [<ffffffff802b4699>] ? rw_copy_check_uvector+0x6d/0xe4
[ 2106.068706]  [<ffffffff802b4f22>] ? do_readv_writev+0xb2/0x18b
[ 2106.068709]  [<ffffffff802b5a80>] ? fget_light+0x46/0xd9
[ 2106.068711]  [<ffffffff802b5121>] ? sys_writev+0x45/0x93
[ 2106.068714]  [<ffffffff8020afc2>] ? system_call_fastpath+0x16/0x1b
[ 2171.106240] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2171.106283] CPU 6:
[ 2171.106284] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2171.106322] Pid: 12397, comm: kvm Not tainted 2.6.30-rc1 #6 System Product Name
[ 2171.106324] RIP: 0010:[<ffffffff8039039f>]  [<ffffffff8039039f>] delay_tsc+0x26/0x57
[ 2171.106330] RSP: 0018:ffff8800281a34f8  EFLAGS: 00000202
[ 2171.106332] RAX: 00000547850e39df RBX: 0000000000000006 RCX: 00000000850e39df
[ 2171.106333] RDX: 00000000850e39df RSI: 0000000000000001 RDI: 0000000000000001
[ 2171.106335] RBP: ffffffff8020bbd3 R08: ffffffff850e39a4 R09: 0000000000013a4b
[ 2171.106337] R10: ffffffff8023a5e7 R11: ffffffff80390379 R12: ffff8800281a3470
[ 2171.106339] R13: 0000000000000006 R14: 0000000000000001 R15: ffffffff8021d470
[ 2171.106341] FS:  00007fa315516950(0000) GS:ffff8800281a0000(0000) knlGS:0000000000000000
[ 2171.106343] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
[ 2171.106344] CR2: 00000000b7ff1000 CR3: 000000015cec1000 CR4: 00000000000026e0
[ 2171.106346] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 2171.106348] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 2171.106349] Call Trace:
[ 2171.106350]  <IRQ>  [<ffffffff8039039f>] ? delay_tsc+0x26/0x57
[ 2171.106356]  [<ffffffff803903cf>] ? delay_tsc+0x56/0x57
[ 2171.106359]  [<ffffffff803939c4>] ? _raw_spin_lock+0x98/0x105
[ 2171.106364]  [<ffffffffa024041d>] ? ipt_do_table+0x102/0x5f1 [ip_tables]
[ 2171.106367]  [<ffffffff8048d4e0>] ? skb_checksum+0x4c/0x257
[ 2171.106372]  [<ffffffffa025f1fc>] ? manip_pkt+0x80/0xf4 [nf_nat]
[ 2171.106376]  [<ffffffffa025f432>] ? nf_nat_icmp_reply_translation+0x1c2/0x245 [nf_nat]
[ 2171.106379]  [<ffffffff8048f632>] ? __alloc_skb+0x6b/0x13d
[ 2171.106387]  [<ffffffffa0246cfe>] ? nf_conntrack_in+0x45e/0x5ac [nf_conntrack]
[ 2171.106391]  [<ffffffffa01602b1>] ? nf_nat_fn+0xc1/0x14d [iptable_nat]
[ 2171.106395]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2171.106398]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2171.106401]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2171.106403]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2171.106405]  [<ffffffff80393925>] ? _raw_spin_unlock+0x8b/0x92
[ 2171.106408]  [<ffffffff804ba8c7>] ? __ip_local_out+0x98/0x9a
[ 2171.106410]  [<ffffffff804ba8d2>] ? ip_local_out+0x9/0x1f
[ 2171.106412]  [<ffffffff804babb4>] ? ip_push_pending_frames+0x2cc/0x33e
[ 2171.106416]  [<ffffffff804dac79>] ? icmp_send+0x559/0x588
[ 2171.106418]  [<ffffffff8022d3a0>] ? task_rq_lock+0x46/0x79
[ 2171.106421]  [<ffffffff8023004f>] ? enqueue_task_fair+0x23b/0x293
[ 2171.106424]  [<ffffffffa00f5083>] ? reject_tg+0x41/0x30e [ipt_REJECT]
[ 2171.106428]  [<ffffffffa024084f>] ? ipt_do_table+0x534/0x5f1 [ip_tables]
[ 2171.106431]  [<ffffffff804b37a0>] ? rt_intern_hash+0x46f/0x48a
[ 2171.106434]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2171.106436]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2171.106439]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2171.106441]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2171.106443]  [<ffffffff804b8070>] ? ip_forward+0x2a1/0x321
[ 2171.106445]  [<ffffffff804b69c6>] ? ip_rcv_finish+0x31e/0x338
[ 2171.106448]  [<ffffffff804b6c15>] ? ip_rcv+0x235/0x27a
[ 2171.106451]  [<ffffffff804957ee>] ? netif_receive_skb+0x4b6/0x4f3
[ 2171.106453]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2171.106460]  [<ffffffffa0228a24>] ? br_handle_frame_finish+0x110/0x148 [bridge]
[ 2171.106466]  [<ffffffffa022cdf9>] ? br_nf_pre_routing_finish+0x300/0x30f [bridge]
[ 2171.106469]  [<ffffffff804b01ff>] ? nf_hook_slow+0xf3/0x104
[ 2171.106474]  [<ffffffffa022caf9>] ? br_nf_pre_routing_finish+0x0/0x30f [bridge]
[ 2171.106480]  [<ffffffffa022d51e>] ? br_nf_pre_routing+0x716/0x730 [bridge]
[ 2171.106483]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2171.106488]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2171.106491]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2171.106496]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2171.106502]  [<ffffffffa0228bf8>] ? br_handle_frame+0x19c/0x1c0 [bridge]
[ 2171.106505]  [<ffffffff804956d2>] ? netif_receive_skb+0x39a/0x4f3
[ 2171.106507]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2171.106510]  [<ffffffff804958aa>] ? process_backlog+0x7f/0xb4
[ 2171.106512]  [<ffffffff80493b65>] ? net_rx_action+0xa8/0x1d5
[ 2171.106515]  [<ffffffff80493c4f>] ? net_rx_action+0x192/0x1d5
[ 2171.106518]  [<ffffffff8023f041>] ? __do_softirq+0xac/0x173
[ 2171.106521]  [<ffffffff8020c1cc>] ? call_softirq+0x1c/0x28
[ 2171.106522]  <EOI>  [<ffffffff8020d3be>] ? do_softirq+0x3a/0x82
[ 2171.106526]  [<ffffffff80495e89>] ? netif_rx_ni+0x19/0x1d
[ 2171.106530]  [<ffffffffa026d4ae>] ? tun_chr_aio_write+0x347/0x3a6 [tun]
[ 2171.106533]  [<ffffffffa026d167>] ? tun_chr_aio_write+0x0/0x3a6 [tun]
[ 2171.106536]  [<ffffffff802b480d>] ? do_sync_readv_writev+0xc0/0x107
[ 2171.106539]  [<ffffffff8024d8bf>] ? autoremove_wake_function+0x0/0x2e
[ 2171.106549]  [<ffffffffa01eea62>] ? kvm_vcpu_ioctl+0x5eb/0x5fe [kvm]
[ 2171.106552]  [<ffffffff802b4699>] ? rw_copy_check_uvector+0x6d/0xe4
[ 2171.106554]  [<ffffffff802b4f22>] ? do_readv_writev+0xb2/0x18b
[ 2171.106556]  [<ffffffff802b5a80>] ? fget_light+0x46/0xd9
[ 2171.106559]  [<ffffffff802b5121>] ? sys_writev+0x45/0x93
[ 2171.106563]  [<ffffffff8020afc2>] ? system_call_fastpath+0x16/0x1b
[ 2236.604439] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2236.604494] CPU 6:
[ 2236.604496] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2236.604543] Pid: 12397, comm: kvm Not tainted 2.6.30-rc1 #6 System Product Name
[ 2236.604545] RIP: 0010:[<ffffffff80210d97>]  [<ffffffff80210d97>] native_read_tsc+0x2/0x11
[ 2236.604551] RSP: 0018:ffff8800281a34f0  EFLAGS: 00000202
[ 2236.604552] RAX: 000000004ae54875 RBX: 0000000000000006 RCX: 000000004ae54852
[ 2236.604554] RDX: 0000000000000570 RSI: 0000000000000001 RDI: 0000000000000001
[ 2236.604556] RBP: ffffffff8020bbd3 R08: 000000004ae54837 R09: 0000000000013a4b
[ 2236.604558] R10: ffffffff8023a5e7 R11: ffffffff80390379 R12: ffff8800281a3470
[ 2236.604561] R13: 0000000000000006 R14: 000001fc89170ac9 R15: ffffffff8021d470
[ 2236.604563] FS:  00007fa315516950(0000) GS:ffff8800281a0000(0000) knlGS:0000000000000000
[ 2236.604566] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
[ 2236.604568] CR2: 00000000b7ff1000 CR3: 000000015cec1000 CR4: 00000000000026e0
[ 2236.604570] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 2236.604573] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 2236.604575] Call Trace:
[ 2236.604577]  <IRQ>  [<ffffffff8039039f>] ? delay_tsc+0x26/0x57
[ 2236.604586]  [<ffffffff80390382>] ? delay_tsc+0x9/0x57
[ 2236.604590]  [<ffffffff803939c4>] ? _raw_spin_lock+0x98/0x105
[ 2236.604595]  [<ffffffffa024041d>] ? ipt_do_table+0x102/0x5f1 [ip_tables]
[ 2236.604598]  [<ffffffff8048d4e0>] ? skb_checksum+0x4c/0x257
[ 2236.604603]  [<ffffffffa025f1fc>] ? manip_pkt+0x80/0xf4 [nf_nat]
[ 2236.604607]  [<ffffffffa025f432>] ? nf_nat_icmp_reply_translation+0x1c2/0x245 [nf_nat]
[ 2236.604610]  [<ffffffff8048f632>] ? __alloc_skb+0x6b/0x13d
[ 2236.604618]  [<ffffffffa0246cfe>] ? nf_conntrack_in+0x45e/0x5ac [nf_conntrack]
[ 2236.604622]  [<ffffffffa01602b1>] ? nf_nat_fn+0xc1/0x14d [iptable_nat]
[ 2236.604626]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2236.604629]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2236.604632]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2236.604634]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2236.604636]  [<ffffffff80393925>] ? _raw_spin_unlock+0x8b/0x92
[ 2236.604639]  [<ffffffff804ba8c7>] ? __ip_local_out+0x98/0x9a
[ 2236.604641]  [<ffffffff804ba8d2>] ? ip_local_out+0x9/0x1f
[ 2236.604643]  [<ffffffff804babb4>] ? ip_push_pending_frames+0x2cc/0x33e
[ 2236.604647]  [<ffffffff804dac79>] ? icmp_send+0x559/0x588
[ 2236.604650]  [<ffffffff8022d3a0>] ? task_rq_lock+0x46/0x79
[ 2236.604653]  [<ffffffff8023004f>] ? enqueue_task_fair+0x23b/0x293
[ 2236.604656]  [<ffffffffa00f5083>] ? reject_tg+0x41/0x30e [ipt_REJECT]
[ 2236.604660]  [<ffffffffa024084f>] ? ipt_do_table+0x534/0x5f1 [ip_tables]
[ 2236.604663]  [<ffffffff804b37a0>] ? rt_intern_hash+0x46f/0x48a
[ 2236.604666]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2236.604668]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2236.604670]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2236.604672]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2236.604675]  [<ffffffff804b8070>] ? ip_forward+0x2a1/0x321
[ 2236.604677]  [<ffffffff804b69c6>] ? ip_rcv_finish+0x31e/0x338
[ 2236.604679]  [<ffffffff804b6c15>] ? ip_rcv+0x235/0x27a
[ 2236.604682]  [<ffffffff804957ee>] ? netif_receive_skb+0x4b6/0x4f3
[ 2236.604685]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2236.604691]  [<ffffffffa0228a24>] ? br_handle_frame_finish+0x110/0x148 [bridge]
[ 2236.604697]  [<ffffffffa022cdf9>] ? br_nf_pre_routing_finish+0x300/0x30f [bridge]
[ 2236.604700]  [<ffffffff804b01ff>] ? nf_hook_slow+0xf3/0x104
[ 2236.604706]  [<ffffffffa022caf9>] ? br_nf_pre_routing_finish+0x0/0x30f [bridge]
[ 2236.604711]  [<ffffffffa022d51e>] ? br_nf_pre_routing+0x716/0x730 [bridge]
[ 2236.604714]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2236.604719]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2236.604722]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2236.604727]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2236.604733]  [<ffffffffa0228bf8>] ? br_handle_frame+0x19c/0x1c0 [bridge]
[ 2236.604736]  [<ffffffff804956d2>] ? netif_receive_skb+0x39a/0x4f3
[ 2236.604738]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2236.604741]  [<ffffffff804958aa>] ? process_backlog+0x7f/0xb4
[ 2236.604743]  [<ffffffff80493b65>] ? net_rx_action+0xa8/0x1d5
[ 2236.604746]  [<ffffffff80493c4f>] ? net_rx_action+0x192/0x1d5
[ 2236.604748]  [<ffffffff8023f041>] ? __do_softirq+0xac/0x173
[ 2236.604751]  [<ffffffff8020c1cc>] ? call_softirq+0x1c/0x28
[ 2236.604752]  <EOI>  [<ffffffff8020d3be>] ? do_softirq+0x3a/0x82
[ 2236.604756]  [<ffffffff80495e89>] ? netif_rx_ni+0x19/0x1d
[ 2236.604760]  [<ffffffffa026d4ae>] ? tun_chr_aio_write+0x347/0x3a6 [tun]
[ 2236.604763]  [<ffffffffa026d167>] ? tun_chr_aio_write+0x0/0x3a6 [tun]
[ 2236.604766]  [<ffffffff802b480d>] ? do_sync_readv_writev+0xc0/0x107
[ 2236.604769]  [<ffffffff8024d8bf>] ? autoremove_wake_function+0x0/0x2e
[ 2236.604779]  [<ffffffffa01eea62>] ? kvm_vcpu_ioctl+0x5eb/0x5fe [kvm]
[ 2236.604782]  [<ffffffff802b4699>] ? rw_copy_check_uvector+0x6d/0xe4
[ 2236.604784]  [<ffffffff802b4f22>] ? do_readv_writev+0xb2/0x18b
[ 2236.604787]  [<ffffffff802b5a80>] ? fget_light+0x46/0xd9
[ 2236.604789]  [<ffffffff802b5121>] ? sys_writev+0x45/0x93
[ 2236.604792]  [<ffffffff8020afc2>] ? system_call_fastpath+0x16/0x1b
[ 2302.102637] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2302.102680] CPU 6:
[ 2302.102681] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2302.102719] Pid: 12397, comm: kvm Not tainted 2.6.30-rc1 #6 System Product Name
[ 2302.102721] RIP: 0010:[<ffffffff80210d95>]  [<ffffffff80210d95>] native_read_tsc+0x0/0x11
[ 2302.102727] RSP: 0018:ffff8800281a34f0  EFLAGS: 00000206
[ 2302.102728] RAX: 0000059910bc579f RBX: 0000000000000006 RCX: 0000000010bc579f
[ 2302.102730] RDX: 0000000010bc579f RSI: 0000000000000001 RDI: 0000000000000001
[ 2302.102732] RBP: ffffffff8020bbd3 R08: 0000000010bc5776 R09: 0000000000013a4b
[ 2302.102733] R10: ffffffff8023a5e7 R11: ffffffff80390379 R12: ffff8800281a3470
[ 2302.102735] R13: 0000000000000006 R14: 0000000000000001 R15: ffffffff8021d470
[ 2302.102737] FS:  00007fa315516950(0000) GS:ffff8800281a0000(0000) knlGS:0000000000000000
[ 2302.102739] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
[ 2302.102741] CR2: 00000000b7ff1000 CR3: 000000015cec1000 CR4: 00000000000026e0
[ 2302.102742] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 2302.102744] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 2302.102746] Call Trace:
[ 2302.102747]  <IRQ>  [<ffffffff8039039f>] ? delay_tsc+0x26/0x57
[ 2302.102754]  [<ffffffff803903dc>] ? __delay+0x0/0xa
[ 2302.102757]  [<ffffffff803939c4>] ? _raw_spin_lock+0x98/0x105
[ 2302.102762]  [<ffffffffa024041d>] ? ipt_do_table+0x102/0x5f1 [ip_tables]
[ 2302.102765]  [<ffffffff8048d4e0>] ? skb_checksum+0x4c/0x257
[ 2302.102769]  [<ffffffffa025f1fc>] ? manip_pkt+0x80/0xf4 [nf_nat]
[ 2302.102773]  [<ffffffffa025f432>] ? nf_nat_icmp_reply_translation+0x1c2/0x245 [nf_nat]
[ 2302.102776]  [<ffffffff8048f632>] ? __alloc_skb+0x6b/0x13d
[ 2302.102784]  [<ffffffffa0246cfe>] ? nf_conntrack_in+0x45e/0x5ac [nf_conntrack]
[ 2302.102788]  [<ffffffffa01602b1>] ? nf_nat_fn+0xc1/0x14d [iptable_nat]
[ 2302.102791]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2302.102794]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2302.102797]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2302.102799]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2302.102801]  [<ffffffff80393925>] ? _raw_spin_unlock+0x8b/0x92
[ 2302.102804]  [<ffffffff804ba8c7>] ? __ip_local_out+0x98/0x9a
[ 2302.102806]  [<ffffffff804ba8d2>] ? ip_local_out+0x9/0x1f
[ 2302.102808]  [<ffffffff804babb4>] ? ip_push_pending_frames+0x2cc/0x33e
[ 2302.102812]  [<ffffffff804dac79>] ? icmp_send+0x559/0x588
[ 2302.102814]  [<ffffffff8022d3a0>] ? task_rq_lock+0x46/0x79
[ 2302.102817]  [<ffffffff8023004f>] ? enqueue_task_fair+0x23b/0x293
[ 2302.102820]  [<ffffffffa00f5083>] ? reject_tg+0x41/0x30e [ipt_REJECT]
[ 2302.102824]  [<ffffffffa024084f>] ? ipt_do_table+0x534/0x5f1 [ip_tables]
[ 2302.102827]  [<ffffffff804b37a0>] ? rt_intern_hash+0x46f/0x48a
[ 2302.102830]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2302.102832]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2302.102835]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2302.102837]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2302.102839]  [<ffffffff804b8070>] ? ip_forward+0x2a1/0x321
[ 2302.102841]  [<ffffffff804b69c6>] ? ip_rcv_finish+0x31e/0x338
[ 2302.102844]  [<ffffffff804b6c15>] ? ip_rcv+0x235/0x27a
[ 2302.102846]  [<ffffffff804957ee>] ? netif_receive_skb+0x4b6/0x4f3
[ 2302.102849]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2302.102855]  [<ffffffffa0228a24>] ? br_handle_frame_finish+0x110/0x148 [bridge]
[ 2302.102862]  [<ffffffffa022cdf9>] ? br_nf_pre_routing_finish+0x300/0x30f [bridge]
[ 2302.102864]  [<ffffffff804b01ff>] ? nf_hook_slow+0xf3/0x104
[ 2302.102870]  [<ffffffffa022caf9>] ? br_nf_pre_routing_finish+0x0/0x30f [bridge]
[ 2302.102876]  [<ffffffffa022d51e>] ? br_nf_pre_routing+0x716/0x730 [bridge]
[ 2302.102878]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2302.102884]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2302.102886]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2302.102892]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2302.102897]  [<ffffffffa0228bf8>] ? br_handle_frame+0x19c/0x1c0 [bridge]
[ 2302.102900]  [<ffffffff804956d2>] ? netif_receive_skb+0x39a/0x4f3
[ 2302.102903]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2302.102905]  [<ffffffff804958aa>] ? process_backlog+0x7f/0xb4
[ 2302.102908]  [<ffffffff80493b65>] ? net_rx_action+0xa8/0x1d5
[ 2302.102910]  [<ffffffff80493c4f>] ? net_rx_action+0x192/0x1d5
[ 2302.102913]  [<ffffffff8023f041>] ? __do_softirq+0xac/0x173
[ 2302.102915]  [<ffffffff8020c1cc>] ? call_softirq+0x1c/0x28
[ 2302.102917]  <EOI>  [<ffffffff8020d3be>] ? do_softirq+0x3a/0x82
[ 2302.102921]  [<ffffffff80495e89>] ? netif_rx_ni+0x19/0x1d
[ 2302.102924]  [<ffffffffa026d4ae>] ? tun_chr_aio_write+0x347/0x3a6 [tun]
[ 2302.102928]  [<ffffffffa026d167>] ? tun_chr_aio_write+0x0/0x3a6 [tun]
[ 2302.102930]  [<ffffffff802b480d>] ? do_sync_readv_writev+0xc0/0x107
[ 2302.102934]  [<ffffffff8024d8bf>] ? autoremove_wake_function+0x0/0x2e
[ 2302.102944]  [<ffffffffa01eea62>] ? kvm_vcpu_ioctl+0x5eb/0x5fe [kvm]
[ 2302.102946]  [<ffffffff802b4699>] ? rw_copy_check_uvector+0x6d/0xe4
[ 2302.102948]  [<ffffffff802b4f22>] ? do_readv_writev+0xb2/0x18b
[ 2302.102951]  [<ffffffff802b5a80>] ? fget_light+0x46/0xd9
[ 2302.102953]  [<ffffffff802b5121>] ? sys_writev+0x45/0x93
[ 2302.102956]  [<ffffffff8020afc2>] ? system_call_fastpath+0x16/0x1b
[ 2367.600835] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2367.600879] CPU 6:
[ 2367.600880] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2367.600918] Pid: 12397, comm: kvm Not tainted 2.6.30-rc1 #6 System Product Name
[ 2367.600920] RIP: 0010:[<ffffffff80210d97>]  [<ffffffff80210d97>] native_read_tsc+0x2/0x11
[ 2367.600925] RSP: 0018:ffff8800281a34f0  EFLAGS: 00000202
[ 2367.600927] RAX: 00000000d69365df RBX: 0000000000000006 RCX: 00000000d69365bc
[ 2367.600929] RDX: 00000000000005c1 RSI: 0000000000000001 RDI: 0000000000000001
[ 2367.600930] RBP: ffffffff8020bbd3 R08: ffffffffd69365a4 R09: 0000000000013a4b
[ 2367.600932] R10: ffffffff8023a5e7 R11: ffffffff80390379 R12: ffff8800281a3470
[ 2367.600934] R13: 0000000000000006 R14: 0000000000000001 R15: ffffffff8021d470
[ 2367.600936] FS:  00007fa315516950(0000) GS:ffff8800281a0000(0000) knlGS:0000000000000000
[ 2367.600938] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
[ 2367.600939] CR2: 00000000b7ff1000 CR3: 000000015cec1000 CR4: 00000000000026e0
[ 2367.600941] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 2367.600943] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 2367.600944] Call Trace:
[ 2367.600946]  <IRQ>  [<ffffffff8039039f>] ? delay_tsc+0x26/0x57
[ 2367.600952]  [<ffffffff80390380>] ? delay_tsc+0x7/0x57
[ 2367.600956]  [<ffffffff803939c4>] ? _raw_spin_lock+0x98/0x105
[ 2367.600961]  [<ffffffffa024041d>] ? ipt_do_table+0x102/0x5f1 [ip_tables]
[ 2367.600964]  [<ffffffff8048d4e0>] ? skb_checksum+0x4c/0x257
[ 2367.600969]  [<ffffffffa025f1fc>] ? manip_pkt+0x80/0xf4 [nf_nat]
[ 2367.600973]  [<ffffffffa025f432>] ? nf_nat_icmp_reply_translation+0x1c2/0x245 [nf_nat]
[ 2367.600976]  [<ffffffff8048f632>] ? __alloc_skb+0x6b/0x13d
[ 2367.600984]  [<ffffffffa0246cfe>] ? nf_conntrack_in+0x45e/0x5ac [nf_conntrack]
[ 2367.600987]  [<ffffffffa01602b1>] ? nf_nat_fn+0xc1/0x14d [iptable_nat]
[ 2367.600991]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2367.600994]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2367.600997]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2367.600999]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2367.601001]  [<ffffffff80393925>] ? _raw_spin_unlock+0x8b/0x92
[ 2367.601004]  [<ffffffff804ba8c7>] ? __ip_local_out+0x98/0x9a
[ 2367.601006]  [<ffffffff804ba8d2>] ? ip_local_out+0x9/0x1f
[ 2367.601009]  [<ffffffff804babb4>] ? ip_push_pending_frames+0x2cc/0x33e
[ 2367.601012]  [<ffffffff804dac79>] ? icmp_send+0x559/0x588
[ 2367.601015]  [<ffffffff8022d3a0>] ? task_rq_lock+0x46/0x79
[ 2367.601017]  [<ffffffff8023004f>] ? enqueue_task_fair+0x23b/0x293
[ 2367.601021]  [<ffffffffa00f5083>] ? reject_tg+0x41/0x30e [ipt_REJECT]
[ 2367.601024]  [<ffffffffa024084f>] ? ipt_do_table+0x534/0x5f1 [ip_tables]
[ 2367.601027]  [<ffffffff804b37a0>] ? rt_intern_hash+0x46f/0x48a
[ 2367.601030]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2367.601032]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2367.601035]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2367.601037]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2367.601039]  [<ffffffff804b8070>] ? ip_forward+0x2a1/0x321
[ 2367.601041]  [<ffffffff804b69c6>] ? ip_rcv_finish+0x31e/0x338
[ 2367.601044]  [<ffffffff804b6c15>] ? ip_rcv+0x235/0x27a
[ 2367.601047]  [<ffffffff804957ee>] ? netif_receive_skb+0x4b6/0x4f3
[ 2367.601049]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2367.601056]  [<ffffffffa0228a24>] ? br_handle_frame_finish+0x110/0x148 [bridge]
[ 2367.601062]  [<ffffffffa022cdf9>] ? br_nf_pre_routing_finish+0x300/0x30f [bridge]
[ 2367.601065]  [<ffffffff804b01ff>] ? nf_hook_slow+0xf3/0x104
[ 2367.601070]  [<ffffffffa022caf9>] ? br_nf_pre_routing_finish+0x0/0x30f [bridge]
[ 2367.601076]  [<ffffffffa022d51e>] ? br_nf_pre_routing+0x716/0x730 [bridge]
[ 2367.601079]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2367.601084]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2367.601087]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2367.601092]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2367.601098]  [<ffffffffa0228bf8>] ? br_handle_frame+0x19c/0x1c0 [bridge]
[ 2367.601100]  [<ffffffff804956d2>] ? netif_receive_skb+0x39a/0x4f3
[ 2367.601103]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2367.601106]  [<ffffffff804958aa>] ? process_backlog+0x7f/0xb4
[ 2367.601108]  [<ffffffff80493b65>] ? net_rx_action+0xa8/0x1d5
[ 2367.601111]  [<ffffffff80493c4f>] ? net_rx_action+0x192/0x1d5
[ 2367.601114]  [<ffffffff8023f041>] ? __do_softirq+0xac/0x173
[ 2367.601116]  [<ffffffff8020c1cc>] ? call_softirq+0x1c/0x28
[ 2367.601118]  <EOI>  [<ffffffff8020d3be>] ? do_softirq+0x3a/0x82
[ 2367.601122]  [<ffffffff80495e89>] ? netif_rx_ni+0x19/0x1d
[ 2367.601126]  [<ffffffffa026d4ae>] ? tun_chr_aio_write+0x347/0x3a6 [tun]
[ 2367.601129]  [<ffffffffa026d167>] ? tun_chr_aio_write+0x0/0x3a6 [tun]
[ 2367.601132]  [<ffffffff802b480d>] ? do_sync_readv_writev+0xc0/0x107
[ 2367.601136]  [<ffffffff8024d8bf>] ? autoremove_wake_function+0x0/0x2e
[ 2367.601145]  [<ffffffffa01eea62>] ? kvm_vcpu_ioctl+0x5eb/0x5fe [kvm]
[ 2367.601148]  [<ffffffff802b4699>] ? rw_copy_check_uvector+0x6d/0xe4
[ 2367.601150]  [<ffffffff802b4f22>] ? do_readv_writev+0xb2/0x18b
[ 2367.601153]  [<ffffffff802b5a80>] ? fget_light+0x46/0xd9
[ 2367.601155]  [<ffffffff802b5121>] ? sys_writev+0x45/0x93
[ 2367.601158]  [<ffffffff8020afc2>] ? system_call_fastpath+0x16/0x1b
[ 2433.099031] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2433.099074] CPU 6:
[ 2433.099076] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2433.099114] Pid: 12397, comm: kvm Not tainted 2.6.30-rc1 #6 System Product Name
[ 2433.099115] RIP: 0010:[<ffffffff80210d97>]  [<ffffffff80210d97>] native_read_tsc+0x2/0x11
[ 2433.099121] RSP: 0018:ffff8800281a34f0  EFLAGS: 00000292
[ 2433.099123] RAX: 000000009c6a7306 RBX: ffff8800281b2730 RCX: 000000009c6a72ee
[ 2433.099124] RDX: 00000000000005ea RSI: 0000000000000001 RDI: 0000000000000001
[ 2433.099126] RBP: ffffffff8020bbd3 R08: ffffffff9c6a72ee R09: 0000000000013a4b
[ 2433.099128] R10: ffffffff8023a5e7 R11: ffffffff80390379 R12: ffff8800281a3470
[ 2433.099130] R13: 0000000000000006 R14: 0000000000000001 R15: ffffffff8021d470
[ 2433.099132] FS:  00007fa315516950(0000) GS:ffff8800281a0000(0000) knlGS:0000000000000000
[ 2433.099134] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
[ 2433.099136] CR2: 00000000b7ff1000 CR3: 000000015cec1000 CR4: 00000000000026e0
[ 2433.099137] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 2433.099139] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 2433.099140] Call Trace:
[ 2433.099142]  <IRQ>  [<ffffffff80390393>] ? delay_tsc+0x1a/0x57
[ 2433.099149]  [<ffffffff803939b1>] ? _raw_spin_lock+0x85/0x105
[ 2433.099152]  [<ffffffff803939c4>] ? _raw_spin_lock+0x98/0x105
[ 2433.099156]  [<ffffffffa024041d>] ? ipt_do_table+0x102/0x5f1 [ip_tables]
[ 2433.099159]  [<ffffffff8048d4e0>] ? skb_checksum+0x4c/0x257
[ 2433.099164]  [<ffffffffa025f1fc>] ? manip_pkt+0x80/0xf4 [nf_nat]
[ 2433.099168]  [<ffffffffa025f432>] ? nf_nat_icmp_reply_translation+0x1c2/0x245 [nf_nat]
[ 2433.099171]  [<ffffffff8048f632>] ? __alloc_skb+0x6b/0x13d
[ 2433.099179]  [<ffffffffa0246cfe>] ? nf_conntrack_in+0x45e/0x5ac [nf_conntrack]
[ 2433.099183]  [<ffffffffa01602b1>] ? nf_nat_fn+0xc1/0x14d [iptable_nat]
[ 2433.099187]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2433.099190]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2433.099193]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2433.099195]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2433.099197]  [<ffffffff80393925>] ? _raw_spin_unlock+0x8b/0x92
[ 2433.099200]  [<ffffffff804ba8c7>] ? __ip_local_out+0x98/0x9a
[ 2433.099202]  [<ffffffff804ba8d2>] ? ip_local_out+0x9/0x1f
[ 2433.099205]  [<ffffffff804babb4>] ? ip_push_pending_frames+0x2cc/0x33e
[ 2433.099208]  [<ffffffff804dac79>] ? icmp_send+0x559/0x588
[ 2433.099211]  [<ffffffff8022d3a0>] ? task_rq_lock+0x46/0x79
[ 2433.099214]  [<ffffffff8023004f>] ? enqueue_task_fair+0x23b/0x293
[ 2433.099217]  [<ffffffffa00f5083>] ? reject_tg+0x41/0x30e [ipt_REJECT]
[ 2433.099221]  [<ffffffffa024084f>] ? ipt_do_table+0x534/0x5f1 [ip_tables]
[ 2433.099224]  [<ffffffff804b37a0>] ? rt_intern_hash+0x46f/0x48a
[ 2433.099227]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2433.099229]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2433.099231]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2433.099234]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2433.099236]  [<ffffffff804b8070>] ? ip_forward+0x2a1/0x321
[ 2433.099238]  [<ffffffff804b69c6>] ? ip_rcv_finish+0x31e/0x338
[ 2433.099241]  [<ffffffff804b6c15>] ? ip_rcv+0x235/0x27a
[ 2433.099244]  [<ffffffff804957ee>] ? netif_receive_skb+0x4b6/0x4f3
[ 2433.099246]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2433.099252]  [<ffffffffa0228a24>] ? br_handle_frame_finish+0x110/0x148 [bridge]
[ 2433.099259]  [<ffffffffa022cdf9>] ? br_nf_pre_routing_finish+0x300/0x30f [bridge]
[ 2433.099261]  [<ffffffff804b01ff>] ? nf_hook_slow+0xf3/0x104
[ 2433.099267]  [<ffffffffa022caf9>] ? br_nf_pre_routing_finish+0x0/0x30f [bridge]
[ 2433.099273]  [<ffffffffa022d51e>] ? br_nf_pre_routing+0x716/0x730 [bridge]
[ 2433.099275]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2433.099281]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2433.099284]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2433.099289]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2433.099295]  [<ffffffffa0228bf8>] ? br_handle_frame+0x19c/0x1c0 [bridge]
[ 2433.099297]  [<ffffffff804956d2>] ? netif_receive_skb+0x39a/0x4f3
[ 2433.099300]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2433.099302]  [<ffffffff804958aa>] ? process_backlog+0x7f/0xb4
[ 2433.099305]  [<ffffffff80493b65>] ? net_rx_action+0xa8/0x1d5
[ 2433.099307]  [<ffffffff80493c4f>] ? net_rx_action+0x192/0x1d5
[ 2433.099310]  [<ffffffff8023f041>] ? __do_softirq+0xac/0x173
[ 2433.099313]  [<ffffffff8020c1cc>] ? call_softirq+0x1c/0x28
[ 2433.099314]  <EOI>  [<ffffffff8020d3be>] ? do_softirq+0x3a/0x82
[ 2433.099318]  [<ffffffff80495e89>] ? netif_rx_ni+0x19/0x1d
[ 2433.099322]  [<ffffffffa026d4ae>] ? tun_chr_aio_write+0x347/0x3a6 [tun]
[ 2433.099325]  [<ffffffffa026d167>] ? tun_chr_aio_write+0x0/0x3a6 [tun]
[ 2433.099328]  [<ffffffff802b480d>] ? do_sync_readv_writev+0xc0/0x107
[ 2433.099331]  [<ffffffff8024d8bf>] ? autoremove_wake_function+0x0/0x2e
[ 2433.099341]  [<ffffffffa01eea62>] ? kvm_vcpu_ioctl+0x5eb/0x5fe [kvm]
[ 2433.099344]  [<ffffffff802b4699>] ? rw_copy_check_uvector+0x6d/0xe4
[ 2433.099346]  [<ffffffff802b4f22>] ? do_readv_writev+0xb2/0x18b
[ 2433.099348]  [<ffffffff802b5a80>] ? fget_light+0x46/0xd9
[ 2433.099351]  [<ffffffff802b5121>] ? sys_writev+0x45/0x93
[ 2433.099354]  [<ffffffff8020afc2>] ? system_call_fastpath+0x16/0x1b
Apr 15 08:28:38 nehalam syslogd 1.5.0#5ubuntu3: restart.
[ 2498.597229] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2498.597272] CPU 6:
[ 2498.597274] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2498.597312] Pid: 12397, comm: kvm Not tainted 2.6.30-rc1 #6 System Product Name
[ 2498.597313] RIP: 0010:[<ffffffff80210d97>]  [<ffffffff80210d97>] native_read_tsc+0x2/0x11
[ 2498.597319] RSP: 0018:ffff8800281a34f0  EFLAGS: 00000202
[ 2498.597321] RAX: 00000000624181c5 RBX: 0000000000000006 RCX: 00000000624181a2
[ 2498.597323] RDX: 0000000000000613 RSI: 0000000000000001 RDI: 0000000000000001
[ 2498.597324] RBP: ffffffff8020bbd3 R08: 000000006241818a R09: 0000000000013a4b
[ 2498.597326] R10: ffffffff8023a5e7 R11: ffffffff80390379 R12: ffff8800281a3470
[ 2498.597328] R13: 0000000000000006 R14: 0000000000000001 R15: ffffffff8021d470
[ 2498.597330] FS:  00007fa315516950(0000) GS:ffff8800281a0000(0000) knlGS:0000000000000000
[ 2498.597332] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
[ 2498.597333] CR2: 00000000b7ff1000 CR3: 000000015cec1000 CR4: 00000000000026e0
[ 2498.597335] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 2498.597337] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 2498.597338] Call Trace:
[ 2498.597340]  <IRQ>  [<ffffffff8039039f>] ? delay_tsc+0x26/0x57
[ 2498.597347]  [<ffffffff803939c4>] ? _raw_spin_lock+0x98/0x105
[ 2498.597349]  [<ffffffff803939c4>] ? _raw_spin_lock+0x98/0x105
[ 2498.597354]  [<ffffffffa024041d>] ? ipt_do_table+0x102/0x5f1 [ip_tables]
[ 2498.597357]  [<ffffffff8048d4e0>] ? skb_checksum+0x4c/0x257
[ 2498.597362]  [<ffffffffa025f1fc>] ? manip_pkt+0x80/0xf4 [nf_nat]
[ 2498.597366]  [<ffffffffa025f432>] ? nf_nat_icmp_reply_translation+0x1c2/0x245 [nf_nat]
[ 2498.597369]  [<ffffffff8048f632>] ? __alloc_skb+0x6b/0x13d
[ 2498.597377]  [<ffffffffa0246cfe>] ? nf_conntrack_in+0x45e/0x5ac [nf_conntrack]
[ 2498.597381]  [<ffffffffa01602b1>] ? nf_nat_fn+0xc1/0x14d [iptable_nat]
[ 2498.597385]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2498.597388]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2498.597390]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2498.597393]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2498.597395]  [<ffffffff80393925>] ? _raw_spin_unlock+0x8b/0x92
[ 2498.597398]  [<ffffffff804ba8c7>] ? __ip_local_out+0x98/0x9a
[ 2498.597400]  [<ffffffff804ba8d2>] ? ip_local_out+0x9/0x1f
[ 2498.597402]  [<ffffffff804babb4>] ? ip_push_pending_frames+0x2cc/0x33e
[ 2498.597406]  [<ffffffff804dac79>] ? icmp_send+0x559/0x588
[ 2498.597409]  [<ffffffff8022d3a0>] ? task_rq_lock+0x46/0x79
[ 2498.597411]  [<ffffffff8023004f>] ? enqueue_task_fair+0x23b/0x293
[ 2498.597415]  [<ffffffffa00f5083>] ? reject_tg+0x41/0x30e [ipt_REJECT]
[ 2498.597418]  [<ffffffffa024084f>] ? ipt_do_table+0x534/0x5f1 [ip_tables]
[ 2498.597421]  [<ffffffff804b37a0>] ? rt_intern_hash+0x46f/0x48a
[ 2498.597424]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2498.597426]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2498.597429]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2498.597431]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2498.597433]  [<ffffffff804b8070>] ? ip_forward+0x2a1/0x321
[ 2498.597436]  [<ffffffff804b69c6>] ? ip_rcv_finish+0x31e/0x338
[ 2498.597438]  [<ffffffff804b6c15>] ? ip_rcv+0x235/0x27a
[ 2498.597441]  [<ffffffff804957ee>] ? netif_receive_skb+0x4b6/0x4f3
[ 2498.597443]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2498.597450]  [<ffffffffa0228a24>] ? br_handle_frame_finish+0x110/0x148 [bridge]
[ 2498.597456]  [<ffffffffa022cdf9>] ? br_nf_pre_routing_finish+0x300/0x30f [bridge]
[ 2498.597458]  [<ffffffff804b01ff>] ? nf_hook_slow+0xf3/0x104
[ 2498.597464]  [<ffffffffa022caf9>] ? br_nf_pre_routing_finish+0x0/0x30f [bridge]
[ 2498.597470]  [<ffffffffa022d51e>] ? br_nf_pre_routing+0x716/0x730 [bridge]
[ 2498.597472]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2498.597478]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2498.597480]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2498.597486]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2498.597491]  [<ffffffffa0228bf8>] ? br_handle_frame+0x19c/0x1c0 [bridge]
[ 2498.597494]  [<ffffffff804956d2>] ? netif_receive_skb+0x39a/0x4f3
[ 2498.597497]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2498.597499]  [<ffffffff804958aa>] ? process_backlog+0x7f/0xb4
[ 2498.597502]  [<ffffffff80493b65>] ? net_rx_action+0xa8/0x1d5
[ 2498.597504]  [<ffffffff80493c4f>] ? net_rx_action+0x192/0x1d5
[ 2498.597507]  [<ffffffff8023f041>] ? __do_softirq+0xac/0x173
[ 2498.597509]  [<ffffffff8020c1cc>] ? call_softirq+0x1c/0x28
[ 2498.597511]  <EOI>  [<ffffffff8020d3be>] ? do_softirq+0x3a/0x82
[ 2498.597515]  [<ffffffff80495e89>] ? netif_rx_ni+0x19/0x1d
[ 2498.597519]  [<ffffffffa026d4ae>] ? tun_chr_aio_write+0x347/0x3a6 [tun]
[ 2498.597522]  [<ffffffffa026d167>] ? tun_chr_aio_write+0x0/0x3a6 [tun]
[ 2498.597525]  [<ffffffff802b480d>] ? do_sync_readv_writev+0xc0/0x107
[ 2498.597529]  [<ffffffff8024d8bf>] ? autoremove_wake_function+0x0/0x2e
[ 2498.597539]  [<ffffffffa01eea62>] ? kvm_vcpu_ioctl+0x5eb/0x5fe [kvm]
[ 2498.597542]  [<ffffffff802b4699>] ? rw_copy_check_uvector+0x6d/0xe4
[ 2498.597544]  [<ffffffff802b4f22>] ? do_readv_writev+0xb2/0x18b
[ 2498.597546]  [<ffffffff802b5a80>] ? fget_light+0x46/0xd9
[ 2498.597549]  [<ffffffff802b5121>] ? sys_writev+0x45/0x93
[ 2498.597552]  [<ffffffff8020afc2>] ? system_call_fastpath+0x16/0x1b
[ 2564.096427] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2564.096470] CPU 6:
[ 2564.096472] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2564.096512] Pid: 12397, comm: kvm Not tainted 2.6.30-rc1 #6 System Product Name
[ 2564.096513] RIP: 0010:[<ffffffff80390382>]  [<ffffffff80390382>] delay_tsc+0x9/0x57
[ 2564.096519] RSP: 0018:ffff8800281a3530  EFLAGS: 00000202
[ 2564.096521] RAX: 0000000000000000 RBX: ffff8800281b2730 RCX: 00000000284159f4
[ 2564.096523] RDX: 000000000000c300 RSI: 0000000000000001 RDI: 0000000000000001
[ 2564.096525] RBP: ffffffff8020bbd3 R08: 00000000284159f4 R09: 0000000000013a4b
[ 2564.096526] R10: ffffffff8023a5e7 R11: ffffffff80390379 R12: ffff8800281a34b0
[ 2564.096528] R13: 000000002471c8ef R14: 0000000028188f77 R15: ffffffff8021d470
[ 2564.096530] FS:  00007fa315516950(0000) GS:ffff8800281a0000(0000) knlGS:0000000000000000
[ 2564.096532] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
[ 2564.096534] CR2: 00000000b7ff1000 CR3: 000000015cec1000 CR4: 00000000000026e0
[ 2564.096536] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 2564.096537] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 2564.096539] Call Trace:
[ 2564.096540]  <IRQ>  [<ffffffff803939c4>] ? _raw_spin_lock+0x98/0x105
[ 2564.096549]  [<ffffffffa024041d>] ? ipt_do_table+0x102/0x5f1 [ip_tables]
[ 2564.096552]  [<ffffffff8048d4e0>] ? skb_checksum+0x4c/0x257
[ 2564.096557]  [<ffffffffa025f1fc>] ? manip_pkt+0x80/0xf4 [nf_nat]
[ 2564.096561]  [<ffffffffa025f432>] ? nf_nat_icmp_reply_translation+0x1c2/0x245 [nf_nat]
[ 2564.096564]  [<ffffffff8048f632>] ? __alloc_skb+0x6b/0x13d
[ 2564.096572]  [<ffffffffa0246cfe>] ? nf_conntrack_in+0x45e/0x5ac [nf_conntrack]
[ 2564.096576]  [<ffffffffa01602b1>] ? nf_nat_fn+0xc1/0x14d [iptable_nat]
[ 2564.096580]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2564.096583]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2564.096585]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2564.096588]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2564.096590]  [<ffffffff80393925>] ? _raw_spin_unlock+0x8b/0x92
[ 2564.096593]  [<ffffffff804ba8c7>] ? __ip_local_out+0x98/0x9a
[ 2564.096595]  [<ffffffff804ba8d2>] ? ip_local_out+0x9/0x1f
[ 2564.096597]  [<ffffffff804babb4>] ? ip_push_pending_frames+0x2cc/0x33e
[ 2564.096601]  [<ffffffff804dac79>] ? icmp_send+0x559/0x588
[ 2564.096604]  [<ffffffff8022d3a0>] ? task_rq_lock+0x46/0x79
[ 2564.096606]  [<ffffffff8023004f>] ? enqueue_task_fair+0x23b/0x293
[ 2564.096610]  [<ffffffffa00f5083>] ? reject_tg+0x41/0x30e [ipt_REJECT]
[ 2564.096613]  [<ffffffffa024084f>] ? ipt_do_table+0x534/0x5f1 [ip_tables]
[ 2564.096616]  [<ffffffff804b37a0>] ? rt_intern_hash+0x46f/0x48a
[ 2564.096619]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2564.096622]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2564.096624]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2564.096626]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2564.096629]  [<ffffffff804b8070>] ? ip_forward+0x2a1/0x321
[ 2564.096631]  [<ffffffff804b69c6>] ? ip_rcv_finish+0x31e/0x338
[ 2564.096633]  [<ffffffff804b6c15>] ? ip_rcv+0x235/0x27a
[ 2564.096636]  [<ffffffff804957ee>] ? netif_receive_skb+0x4b6/0x4f3
[ 2564.096639]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2564.096645]  [<ffffffffa0228a24>] ? br_handle_frame_finish+0x110/0x148 [bridge]
[ 2564.096651]  [<ffffffffa022cdf9>] ? br_nf_pre_routing_finish+0x300/0x30f [bridge]
[ 2564.096654]  [<ffffffff804b01ff>] ? nf_hook_slow+0xf3/0x104
[ 2564.096660]  [<ffffffffa022caf9>] ? br_nf_pre_routing_finish+0x0/0x30f [bridge]
[ 2564.096666]  [<ffffffffa022d51e>] ? br_nf_pre_routing+0x716/0x730 [bridge]
[ 2564.096668]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2564.096674]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2564.096676]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2564.096682]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2564.096687]  [<ffffffffa0228bf8>] ? br_handle_frame+0x19c/0x1c0 [bridge]
[ 2564.096690]  [<ffffffff804956d2>] ? netif_receive_skb+0x39a/0x4f3
[ 2564.096693]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2564.096695]  [<ffffffff804958aa>] ? process_backlog+0x7f/0xb4
[ 2564.096698]  [<ffffffff80493b65>] ? net_rx_action+0xa8/0x1d5
[ 2564.096700]  [<ffffffff80493c4f>] ? net_rx_action+0x192/0x1d5
[ 2564.096704]  [<ffffffff8023f041>] ? __do_softirq+0xac/0x173
[ 2564.096707]  [<ffffffff8020c1cc>] ? call_softirq+0x1c/0x28
[ 2564.096708]  <EOI>  [<ffffffff8020d3be>] ? do_softirq+0x3a/0x82
[ 2564.096712]  [<ffffffff80495e89>] ? netif_rx_ni+0x19/0x1d
[ 2564.096716]  [<ffffffffa026d4ae>] ? tun_chr_aio_write+0x347/0x3a6 [tun]
[ 2564.096720]  [<ffffffffa026d167>] ? tun_chr_aio_write+0x0/0x3a6 [tun]
[ 2564.096723]  [<ffffffff802b480d>] ? do_sync_readv_writev+0xc0/0x107
[ 2564.096727]  [<ffffffff8024d8bf>] ? autoremove_wake_function+0x0/0x2e
[ 2564.096736]  [<ffffffffa01eea62>] ? kvm_vcpu_ioctl+0x5eb/0x5fe [kvm]
[ 2564.096739]  [<ffffffff802b4699>] ? rw_copy_check_uvector+0x6d/0xe4
[ 2564.096741]  [<ffffffff802b4f22>] ? do_readv_writev+0xb2/0x18b
[ 2564.096744]  [<ffffffff802b5a80>] ? fget_light+0x46/0xd9
[ 2564.096746]  [<ffffffff802b5121>] ? sys_writev+0x45/0x93
[ 2564.096749]  [<ffffffff8020afc2>] ? system_call_fastpath+0x16/0x1b
[ 2629.594624] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2629.594668] CPU 6:
[ 2629.594669] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2629.594707] Pid: 12397, comm: kvm Not tainted 2.6.30-rc1 #6 System Product Name
[ 2629.594709] RIP: 0010:[<ffffffff803903c5>]  [<ffffffff803903c5>] delay_tsc+0x4c/0x57
[ 2629.594715] RSP: 0018:ffff8800281a34f8  EFLAGS: 00000206
[ 2629.594717] RAX: 0000000000000012 RBX: 0000000000000006 RCX: 00000000ee1867ed
[ 2629.594718] RDX: 00000000ee1867ed RSI: 0000000000000001 RDI: 0000000000000001
[ 2629.594720] RBP: ffffffff8020bbd3 R08: ffffffffee1867ed R09: 0000000000013a4b
[ 2629.594722] R10: ffffffff8023a5e7 R11: ffffffff80390379 R12: ffff8800281a3470
[ 2629.594724] R13: 0000000000000006 R14: 0000000000000001 R15: ffffffff8021d470
[ 2629.594726] FS:  00007fa315516950(0000) GS:ffff8800281a0000(0000) knlGS:0000000000000000
[ 2629.594728] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
[ 2629.594729] CR2: 00000000b7ff1000 CR3: 000000015cec1000 CR4: 00000000000026e0
[ 2629.594731] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 2629.594733] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 2629.594735] Call Trace:
[ 2629.594736]  <IRQ>  [<ffffffff8039039f>] ? delay_tsc+0x26/0x57
[ 2629.594742]  [<ffffffff803939a9>] ? _raw_spin_lock+0x7d/0x105
[ 2629.594745]  [<ffffffff803939c4>] ? _raw_spin_lock+0x98/0x105
[ 2629.594749]  [<ffffffffa024041d>] ? ipt_do_table+0x102/0x5f1 [ip_tables]
[ 2629.594752]  [<ffffffff8048d4e0>] ? skb_checksum+0x4c/0x257
[ 2629.594757]  [<ffffffffa025f1fc>] ? manip_pkt+0x80/0xf4 [nf_nat]
[ 2629.594761]  [<ffffffffa025f432>] ? nf_nat_icmp_reply_translation+0x1c2/0x245 [nf_nat]
[ 2629.594764]  [<ffffffff8048f632>] ? __alloc_skb+0x6b/0x13d
[ 2629.594772]  [<ffffffffa0246cfe>] ? nf_conntrack_in+0x45e/0x5ac [nf_conntrack]
[ 2629.594776]  [<ffffffffa01602b1>] ? nf_nat_fn+0xc1/0x14d [iptable_nat]
[ 2629.594780]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2629.594783]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2629.594786]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2629.594788]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2629.594791]  [<ffffffff80393925>] ? _raw_spin_unlock+0x8b/0x92
[ 2629.594793]  [<ffffffff804ba8c7>] ? __ip_local_out+0x98/0x9a
[ 2629.594796]  [<ffffffff804ba8d2>] ? ip_local_out+0x9/0x1f
[ 2629.594798]  [<ffffffff804babb4>] ? ip_push_pending_frames+0x2cc/0x33e
[ 2629.594801]  [<ffffffff804dac79>] ? icmp_send+0x559/0x588
[ 2629.594804]  [<ffffffff8022d3a0>] ? task_rq_lock+0x46/0x79
[ 2629.594806]  [<ffffffff8023004f>] ? enqueue_task_fair+0x23b/0x293
[ 2629.594810]  [<ffffffffa00f5083>] ? reject_tg+0x41/0x30e [ipt_REJECT]
[ 2629.594814]  [<ffffffffa024084f>] ? ipt_do_table+0x534/0x5f1 [ip_tables]
[ 2629.594817]  [<ffffffff804b37a0>] ? rt_intern_hash+0x46f/0x48a
[ 2629.594820]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2629.594822]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2629.594824]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2629.594827]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2629.594829]  [<ffffffff804b8070>] ? ip_forward+0x2a1/0x321
[ 2629.594831]  [<ffffffff804b69c6>] ? ip_rcv_finish+0x31e/0x338
[ 2629.594834]  [<ffffffff804b6c15>] ? ip_rcv+0x235/0x27a
[ 2629.594837]  [<ffffffff804957ee>] ? netif_receive_skb+0x4b6/0x4f3
[ 2629.594839]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2629.594845]  [<ffffffffa0228a24>] ? br_handle_frame_finish+0x110/0x148 [bridge]
[ 2629.594852]  [<ffffffffa022cdf9>] ? br_nf_pre_routing_finish+0x300/0x30f [bridge]
[ 2629.594854]  [<ffffffff804b01ff>] ? nf_hook_slow+0xf3/0x104
[ 2629.594860]  [<ffffffffa022caf9>] ? br_nf_pre_routing_finish+0x0/0x30f [bridge]
[ 2629.594866]  [<ffffffffa022d51e>] ? br_nf_pre_routing+0x716/0x730 [bridge]
[ 2629.594868]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2629.594874]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2629.594877]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2629.594882]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2629.594888]  [<ffffffffa0228bf8>] ? br_handle_frame+0x19c/0x1c0 [bridge]
[ 2629.594890]  [<ffffffff804956d2>] ? netif_receive_skb+0x39a/0x4f3
[ 2629.594893]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2629.594896]  [<ffffffff804958aa>] ? process_backlog+0x7f/0xb4
[ 2629.594898]  [<ffffffff80493b65>] ? net_rx_action+0xa8/0x1d5
[ 2629.594901]  [<ffffffff80493c4f>] ? net_rx_action+0x192/0x1d5
[ 2629.594904]  [<ffffffff8023f041>] ? __do_softirq+0xac/0x173
[ 2629.594906]  [<ffffffff8020c1cc>] ? call_softirq+0x1c/0x28
[ 2629.594908]  <EOI>  [<ffffffff8020d3be>] ? do_softirq+0x3a/0x82
[ 2629.594912]  [<ffffffff80495e89>] ? netif_rx_ni+0x19/0x1d
[ 2629.594916]  [<ffffffffa026d4ae>] ? tun_chr_aio_write+0x347/0x3a6 [tun]
[ 2629.594919]  [<ffffffffa026d167>] ? tun_chr_aio_write+0x0/0x3a6 [tun]
[ 2629.594922]  [<ffffffff802b480d>] ? do_sync_readv_writev+0xc0/0x107
[ 2629.594925]  [<ffffffff8024d8bf>] ? autoremove_wake_function+0x0/0x2e
[ 2629.594935]  [<ffffffffa01eea62>] ? kvm_vcpu_ioctl+0x5eb/0x5fe [kvm]
[ 2629.594937]  [<ffffffff802b4699>] ? rw_copy_check_uvector+0x6d/0xe4
[ 2629.594940]  [<ffffffff802b4f22>] ? do_readv_writev+0xb2/0x18b
[ 2629.594942]  [<ffffffff802b5a80>] ? fget_light+0x46/0xd9
[ 2629.594945]  [<ffffffff802b5121>] ? sys_writev+0x45/0x93
[ 2629.594948]  [<ffffffff8020afc2>] ? system_call_fastpath+0x16/0x1b
[ 2695.092824] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2695.092867] CPU 6:
[ 2695.092868] Modules linked in: tun binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm pci_slot cpufreq_powersave cpufreq_ondemand cpufreq_conservative cpufreq_userspace snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse snd pcspkr serio_raw evdev wmi rtc_cmos soundcore snd_page_alloc sg sr_mod cdrom hid_belkin usbhid mvsas pata_marvell pata_acpi libsas ata_piix ahci scsi_transport_sas sky2 igb dca floppy fuse
[ 2695.092907] Pid: 12397, comm: kvm Not tainted 2.6.30-rc1 #6 System Product Name
[ 2695.092909] RIP: 0010:[<ffffffff803903a7>]  [<ffffffff803903a7>] delay_tsc+0x2e/0x57
[ 2695.092915] RSP: 0018:ffff8800281a34f8  EFLAGS: 00000202
[ 2695.092917] RAX: ffffffffb3ef768f RBX: 0000000000000006 RCX: 00000000b3ef768f
[ 2695.092918] RDX: 00000000b3ef768f RSI: 0000000000000001 RDI: 0000000000000001
[ 2695.092920] RBP: ffffffff8020bbd3 R08: ffffffffb3ef768f R09: 0000000000013a4b
[ 2695.092922] R10: ffffffff8023a5e7 R11: ffffffff80390379 R12: ffff8800281a3470
[ 2695.092924] R13: 0000000000000006 R14: 0000000000000001 R15: ffffffff8021d470
[ 2695.092926] FS:  00007fa315516950(0000) GS:ffff8800281a0000(0000) knlGS:0000000000000000
[ 2695.092928] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
[ 2695.092930] CR2: 00000000b7ff1000 CR3: 000000015cec1000 CR4: 00000000000026e0
[ 2695.092931] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 2695.092933] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 2695.092935] Call Trace:
[ 2695.092936]  <IRQ>  [<ffffffff8039039f>] ? delay_tsc+0x26/0x57
[ 2695.092943]  [<ffffffff803939c4>] ? _raw_spin_lock+0x98/0x105
[ 2695.092945]  [<ffffffff803939c4>] ? _raw_spin_lock+0x98/0x105
[ 2695.092950]  [<ffffffffa024041d>] ? ipt_do_table+0x102/0x5f1 [ip_tables]
[ 2695.092953]  [<ffffffff8048d4e0>] ? skb_checksum+0x4c/0x257
[ 2695.092958]  [<ffffffffa025f1fc>] ? manip_pkt+0x80/0xf4 [nf_nat]
[ 2695.092962]  [<ffffffffa025f432>] ? nf_nat_icmp_reply_translation+0x1c2/0x245 [nf_nat]
[ 2695.092965]  [<ffffffff8048f632>] ? __alloc_skb+0x6b/0x13d
[ 2695.092973]  [<ffffffffa0246cfe>] ? nf_conntrack_in+0x45e/0x5ac [nf_conntrack]
[ 2695.092977]  [<ffffffffa01602b1>] ? nf_nat_fn+0xc1/0x14d [iptable_nat]
[ 2695.092981]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2695.092984]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2695.092986]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2695.092988]  [<ffffffff804b8ed0>] ? dst_output+0x0/0xb
[ 2695.092991]  [<ffffffff80393925>] ? _raw_spin_unlock+0x8b/0x92
[ 2695.092993]  [<ffffffff804ba8c7>] ? __ip_local_out+0x98/0x9a
[ 2695.092996]  [<ffffffff804ba8d2>] ? ip_local_out+0x9/0x1f
[ 2695.092998]  [<ffffffff804babb4>] ? ip_push_pending_frames+0x2cc/0x33e
[ 2695.093001]  [<ffffffff804dac79>] ? icmp_send+0x559/0x588
[ 2695.093004]  [<ffffffff8022d3a0>] ? task_rq_lock+0x46/0x79
[ 2695.093007]  [<ffffffff8023004f>] ? enqueue_task_fair+0x23b/0x293
[ 2695.093010]  [<ffffffffa00f5083>] ? reject_tg+0x41/0x30e [ipt_REJECT]
[ 2695.093014]  [<ffffffffa024084f>] ? ipt_do_table+0x534/0x5f1 [ip_tables]
[ 2695.093017]  [<ffffffff804b37a0>] ? rt_intern_hash+0x46f/0x48a
[ 2695.093020]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2695.093022]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2695.093025]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2695.093027]  [<ffffffff804b7d94>] ? ip_forward_finish+0x0/0x3b
[ 2695.093030]  [<ffffffff804b8070>] ? ip_forward+0x2a1/0x321
[ 2695.093032]  [<ffffffff804b69c6>] ? ip_rcv_finish+0x31e/0x338
[ 2695.093034]  [<ffffffff804b6c15>] ? ip_rcv+0x235/0x27a
[ 2695.093037]  [<ffffffff804957ee>] ? netif_receive_skb+0x4b6/0x4f3
[ 2695.093040]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2695.093046]  [<ffffffffa0228a24>] ? br_handle_frame_finish+0x110/0x148 [bridge]
[ 2695.093052]  [<ffffffffa022cdf9>] ? br_nf_pre_routing_finish+0x300/0x30f [bridge]
[ 2695.093055]  [<ffffffff804b01ff>] ? nf_hook_slow+0xf3/0x104
[ 2695.093061]  [<ffffffffa022caf9>] ? br_nf_pre_routing_finish+0x0/0x30f [bridge]
[ 2695.093067]  [<ffffffffa022d51e>] ? br_nf_pre_routing+0x716/0x730 [bridge]
[ 2695.093069]  [<ffffffff804b00d0>] ? nf_iterate+0x41/0x7d
[ 2695.093075]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2695.093078]  [<ffffffff804b0195>] ? nf_hook_slow+0x89/0x104
[ 2695.093083]  [<ffffffffa0228914>] ? br_handle_frame_finish+0x0/0x148 [bridge]
[ 2695.093089]  [<ffffffffa0228bf8>] ? br_handle_frame+0x19c/0x1c0 [bridge]
[ 2695.093092]  [<ffffffff804956d2>] ? netif_receive_skb+0x39a/0x4f3
[ 2695.093094]  [<ffffffff8049548d>] ? netif_receive_skb+0x155/0x4f3
[ 2695.093097]  [<ffffffff804958aa>] ? process_backlog+0x7f/0xb4
[ 2695.093100]  [<ffffffff80493b65>] ? net_rx_action+0xa8/0x1d5
[ 2695.093102]  [<ffffffff80493c4f>] ? net_rx_action+0x192/0x1d5
[ 2695.093105]  [<ffffffff8023f041>] ? __do_softirq+0xac/0x173
[ 2695.093108]  [<ffffffff8020c1cc>] ? call_softirq+0x1c/0x28
[ 2695.093109]  <EOI>  [<ffffffff8020d3be>] ? do_softirq+0x3a/0x82
[ 2695.093113]  [<ffffffff80495e89>] ? netif_rx_ni+0x19/0x1d
[ 2695.093117]  [<ffffffffa026d4ae>] ? tun_chr_aio_write+0x347/0x3a6 [tun]
[ 2695.093120]  [<ffffffffa026d167>] ? tun_chr_aio_write+0x0/0x3a6 [tun]
[ 2695.093124]  [<ffffffff802b480d>] ? do_sync_readv_writev+0xc0/0x107
[ 2695.093128]  [<ffffffff8024d8bf>] ? autoremove_wake_function+0x0/0x2e
[ 2695.093138]  [<ffffffffa01eea62>] ? kvm_vcpu_ioctl+0x5eb/0x5fe [kvm]
[ 2695.093141]  [<ffffffff802b4699>] ? rw_copy_check_uvector+0x6d/0xe4
[ 2695.093143]  [<ffffffff802b4f22>] ? do_readv_writev+0xb2/0x18b
[ 2695.093146]  [<ffffffff802b5a80>] ? fget_light+0x46/0xd9
[ 2695.093148]  [<ffffffff802b5121>] ? sys_writev+0x45/0x93
[ 2695.093151]  [
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
Re: iptables very slow after commit 784544739a25c30637397a ..., Paul E. McKenney, (Fri Apr 10, 10:42 pm)
Re: iptables very slow after commit 784544739a25c30637397a ..., Stephen Hemminger, (Sat Apr 11, 8:05 am)
Re: iptables very slow after commit 784544739a25c30637397a ..., Stephen Hemminger, (Sat Apr 11, 8:07 am)
Re: iptables very slow after commit 784544739a25c30637397a ..., Stephen Hemminger, (Sat Apr 11, 8:50 am)
Re: iptables very slow after commit 784544739a25c30637397a ..., Paul E. McKenney, (Sat Apr 11, 10:43 am)
Re: iptables very slow after commit 784544739a25c30637397a ..., Paul E. McKenney, (Sat Apr 11, 10:48 am)
Re: iptables very slow after commit 784544739a25c30637397a ..., Arkadiusz Miskiewicz, (Sat Apr 11, 11:32 am)
Re: iptables very slow after commit 784544739a25c30637397a ..., Evgeniy Polyakov, (Sun Apr 12, 12:23 am)
Re: iptables very slow after commit 784544739a25c30637397a ..., Stephen Hemminger, (Sun Apr 12, 9:06 am)
Re: iptables very slow after commit 784544739a25c30637397a ..., Paul E. McKenney, (Sun Apr 12, 10:30 am)
Re: iptables very slow after commit 784544739a25c30637397a ..., Paul E. McKenney, (Sun Apr 12, 10:31 am)
[PATCH] netfilter: use per-cpu spinlock rather than RCU, Stephen Hemminger, (Mon Apr 13, 9:53 am)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RCU, Stephen Hemminger, (Mon Apr 13, 11:11 am)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RCU, Martin Josefsson, (Mon Apr 13, 12:06 pm)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RCU, Linus Torvalds, (Mon Apr 13, 12:17 pm)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RCU, Stephen Hemminger, (Mon Apr 13, 4:20 pm)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RCU, Patrick McHardy, (Tue Apr 14, 5:27 am)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RCU, Stephen Hemminger, (Tue Apr 14, 7:45 am)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RCU, Stephen Hemminger, (Tue Apr 14, 10:19 am)
[PATCH] netfilter: use per-cpu spinlock rather than RCU (v2), Stephen Hemminger, (Tue Apr 14, 11:17 am)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RC ..., Stephen Hemminger, (Tue Apr 14, 2:11 pm)
[PATCH] netfilter: use per-cpu spinlock rather than RCU (v3), Stephen Hemminger, (Tue Apr 14, 2:13 pm)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RC ..., Stephen Hemminger, (Wed Apr 15, 9:31 am)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RC ..., Stephen Hemminger, (Wed Apr 15, 1:55 pm)
[PATCH] netfilter: use per-cpu rwlock rather than RCU (v4), Stephen Hemminger, (Wed Apr 15, 2:57 pm)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RC ..., Stephen Hemminger, (Wed Apr 15, 5:01 pm)
[PATCH] netfilter: use per-cpu spinlock and RCU (v5), Stephen Hemminger, (Wed Apr 15, 5:45 pm)
Re: [PATCH] netfilter: use per-cpu spinlock and RCU (v5), Eric Dumazet, (Wed Apr 15, 10:01 pm)
Re: [PATCH] netfilter: use per-cpu spinlock and RCU (v5), Patrick McHardy, (Thu Apr 16, 6:53 am)
Re: [PATCH] netfilter: use per-cpu spinlock and RCU (v5), Paul E. McKenney, (Thu Apr 16, 7:47 am)
[PATCH] netfilter: use per-cpu recursive spinlock (v6), Eric Dumazet, (Thu Apr 16, 9:10 am)
Re: [PATCH] netfilter: use per-cpu recursive spinlock (v6), Linus Torvalds, (Thu Apr 16, 9:37 am)
Re: [PATCH] netfilter: use per-cpu recursive spinlock (v6), Patrick McHardy, (Thu Apr 16, 9:59 am)
Re: [PATCH] netfilter: use per-cpu recursive spinlock (v6), Paul E. McKenney, (Thu Apr 16, 10:58 am)
[PATCH[] netfilter: use per-cpu reader-writer lock (v0.7), Stephen Hemminger, (Thu Apr 16, 1:49 pm)
[PATCH] netfilter: per-cpu spin-lock with recursion (v0.8), Stephen Hemminger, (Thu Apr 16, 4:52 pm)
Re: [PATCH] netfilter: use per-cpu recursive spinlock (v6), Paul E. McKenney, (Thu Apr 16, 5:13 pm)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RC ..., Mathieu Desnoyers, (Thu Apr 16, 7:19 pm)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RC ..., Stephen Hemminger, (Thu Apr 16, 9:50 pm)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RC ..., Paul E. McKenney, (Thu Apr 16, 10:05 pm)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RC ..., Paul E. McKenney, (Thu Apr 16, 10:08 pm)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RC ..., Paul E. McKenney, (Thu Apr 16, 10:40 pm)
Re: [PATCH] netfilter: use per-cpu spinlock rather than RC ..., Mathieu Desnoyers, (Thu Apr 16, 10:44 pm)
[PATCH] netfilter: use per-cpu recursive lock (v10), Stephen Hemminger, (Mon Apr 20, 10:34 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v10), Paul E. McKenney, (Mon Apr 20, 11:21 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v10), Eric Dumazet, (Mon Apr 20, 11:25 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v10), Stephen Hemminger, (Mon Apr 20, 1:32 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v10), Stephen Hemminger, (Mon Apr 20, 1:42 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v10), Paul E. McKenney, (Mon Apr 20, 2:05 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v10), Paul Mackerras, (Mon Apr 20, 2:23 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v10), Paul E. McKenney, (Mon Apr 20, 2:58 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v10), Paul Mackerras, (Mon Apr 20, 3:41 pm)
[PATCH] netfilter: use per-cpu recursive lock (v11), Stephen Hemminger, (Mon Apr 20, 4:01 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v10), Paul E. McKenney, (Mon Apr 20, 4:44 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Lai Jiangshan, (Mon Apr 20, 8:41 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Eric Dumazet, (Mon Apr 20, 8:56 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Stephen Hemminger, (Mon Apr 20, 9:15 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Eric Dumazet, (Mon Apr 20, 9:59 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Lai Jiangshan, (Mon Apr 20, 10:22 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Lai Jiangshan, (Mon Apr 20, 10:34 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Stephen Hemminger, (Mon Apr 20, 10:45 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Lai Jiangshan, (Mon Apr 20, 10:46 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Lai Jiangshan, (Mon Apr 20, 11:52 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Evgeniy Polyakov, (Tue Apr 21, 1:16 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Lai Jiangshan, (Tue Apr 21, 1:42 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), David Miller, (Tue Apr 21, 1:49 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Eric Dumazet, (Tue Apr 21, 1:55 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Evgeniy Polyakov, (Tue Apr 21, 2:22 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Lai Jiangshan, (Tue Apr 21, 2:34 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Linus Torvalds, (Tue Apr 21, 9:13 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Paul E. McKenney, (Tue Apr 21, 9:37 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Stephen Hemminger, (Tue Apr 21, 9:43 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Linus Torvalds, (Tue Apr 21, 9:50 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Ingo Molnar, (Tue Apr 21, 11:02 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Stephen Hemminger, (Tue Apr 21, 11:15 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Paul E. McKenney, (Tue Apr 21, 11:34 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Ingo Molnar, (Tue Apr 21, 12:10 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Ingo Molnar, (Tue Apr 21, 12:39 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Eric Dumazet, (Tue Apr 21, 12:46 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Linus Torvalds, (Tue Apr 21, 1:14 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Stephen Hemminger, (Tue Apr 21, 2:04 pm)
[PATCH] netfilter: use per-cpu recursive lock (v13), Stephen Hemminger, (Tue Apr 21, 2:39 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v13), Paul E. McKenney, (Tue Apr 21, 9:17 pm)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Ingo Molnar, (Wed Apr 22, 12:35 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Eric Dumazet, (Wed Apr 22, 1:53 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Jarek Poplawski, (Wed Apr 22, 3:13 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Jarek Poplawski, (Wed Apr 22, 4:39 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v13), Eric Dumazet, (Wed Apr 22, 7:57 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Linus Torvalds, (Wed Apr 22, 8:19 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v13), Linus Torvalds, (Wed Apr 22, 8:32 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Eric Dumazet, (Wed Apr 22, 9:57 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Linus Torvalds, (Wed Apr 22, 10:18 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Ingo Molnar, (Wed Apr 22, 10:48 am)
Re: [PATCH] netfilter: use per-cpu recursive lock (v11), Jarek Poplawski, (Wed Apr 22, 1:46 pm)
[PATCH] netfilter: use per-CPU recursive lock {XIV}, Stephen Hemminger, (Thu Apr 23, 9:09 pm)
Re: [PATCH] netfilter: use per-CPU recursive lock {XIV}, Eric Dumazet, (Thu Apr 23, 9:58 pm)
Re: [PATCH] netfilter: use per-CPU recursive lock {XIV}, Patrick McHardy, (Fri Apr 24, 8:33 am)
Re: [PATCH] netfilter: use per-CPU recursive lock {XIV}, Stephen Hemminger, (Fri Apr 24, 9:18 am)
Re: [PATCH] netfilter: use per-CPU recursive lock {XIV}, Jarek Poplawski, (Fri Apr 24, 1:43 pm)
[PATCH] netfilter: iptables no lockdep is needed.., Stephen Hemminger, (Sat Apr 25, 1:30 pm)
Re: [PATCH] netfilter: iptables no lockdep is needed.., Jarek Poplawski, (Sun Apr 26, 1:18 am)
[PATCH] netfilter: use per-CPU recursive lock {XV}, Eric Dumazet, (Sun Apr 26, 11:24 am)
Re: [PATCH] netfilter: use per-CPU recursive lock {XV}, Mathieu Desnoyers, (Sun Apr 26, 11:56 am)
Re: [PATCH] netfilter: use per-CPU recursive lock {XV}, Mathieu Desnoyers, (Sun Apr 26, 12:31 pm)
Re: [PATCH] netfilter: use per-CPU recursive lock {XV}, Eric Dumazet, (Sun Apr 26, 1:55 pm)
Re: [PATCH] netfilter: use per-CPU recursive lock {XV}, Mathieu Desnoyers, (Sun Apr 26, 2:39 pm)
Re: [PATCH] netfilter: use per-CPU recursive lock {XV}, Stephen Hemminger, (Sun Apr 26, 2:57 pm)
Re: [PATCH] netfilter: use per-CPU recursive lock {XV}, Mathieu Desnoyers, (Sun Apr 26, 3:32 pm)
Re: [PATCH] netfilter: use per-CPU recursive lock {XV}, Peter Zijlstra, (Mon Apr 27, 10:44 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Stephen Hemminger, (Mon Apr 27, 11:30 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Ingo Molnar, (Mon Apr 27, 11:54 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Stephen Hemminger, (Mon Apr 27, 12:06 pm)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Linus Torvalds, (Mon Apr 27, 12:46 pm)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Linus Torvalds, (Mon Apr 27, 12:48 pm)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Evgeniy Polyakov, (Mon Apr 27, 1:36 pm)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Linus Torvalds, (Mon Apr 27, 1:58 pm)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Stephen Hemminger, (Mon Apr 27, 2:40 pm)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Linus Torvalds, (Mon Apr 27, 3:24 pm)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Linus Torvalds, (Mon Apr 27, 4:01 pm)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Linus Torvalds, (Mon Apr 27, 4:03 pm)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Linus Torvalds, (Mon Apr 27, 4:32 pm)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Eric Dumazet, (Mon Apr 27, 11:58 pm)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Peter Zijlstra, (Tue Apr 28, 12:41 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Jan Engelhardt, (Tue Apr 28, 12:42 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, David Miller, (Tue Apr 28, 4:53 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Ingo Molnar, (Tue Apr 28, 5:40 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, David Miller, (Tue Apr 28, 6:43 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Mathieu Desnoyers, (Tue Apr 28, 6:52 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Paul E. McKenney, (Tue Apr 28, 7:22 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, David Miller, (Tue Apr 28, 7:37 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Mathieu Desnoyers, (Tue Apr 28, 7:49 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, David Miller, (Tue Apr 28, 8:00 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Linus Torvalds, (Tue Apr 28, 8:09 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Paul E. McKenney, (Tue Apr 28, 8:42 am)
[PATCH] netfilter: revised locking for x_tables, Stephen Hemminger, (Tue Apr 28, 9:24 am)
Re: [PATCH] netfilter: revised locking for x_tables, Linus Torvalds, (Tue Apr 28, 9:50 am)
Re: [PATCH] netfilter: revised locking for x_tables, Linus Torvalds, (Tue Apr 28, 9:55 am)
Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}, Christoph Lameter, (Tue Apr 28, 10:35 am)
Re: [PATCH] netfilter: revised locking for x_tables, David Miller, (Tue Apr 28, 10:37 pm)
Re: [PATCH] netfilter: revised locking for x_tables, Jeff Chua, (Wed Apr 29, 8:26 pm)
Re: [PATCH] netfilter: revised locking for x_tables, David Miller, (Wed Apr 29, 8:31 pm)
[PATCH] netfilter: use likely() in xt_info_rdlock_bh(), Eric Dumazet, (Fri May 1, 1:38 am)