FIXED: intel i/oat dma from memory to PCI-E BAR resulting into "BUG: unable to handle kernel paging request at..."

Submitted by liora
on March 25, 2010 - 3:07pm

OK, found the cause!
i had to use ioremap instead of phys_to_virt...
Now it works!

Hi All,
I'm trying to use/hack the dmatest driver as a starting point to do intel ioat based dma between memory and PCIe card BAR. to simplify things, I used the vga card prefetchable BAR (address 0xd3800000 taken from lspci -vv) as a test destination target. To test this approach I wrote random bytes into the relevant mapping in /dev/mem and saw some nice garbage pixels on the screen... :) ) - now my next goal was to achieve the same garbage pixels, but this time with dma. I have added the following line

thread->dsts[0] = phys_to_virt(0xd3800000);

just before the iterations while (line 286 (http://lxr.linux.no/linux+v2.6.32/drivers/dma/dmatest.c#L286) on dmatest.c kernel version 2.6.32.6) effectively overwriting the destination address the driver is testing with, with the vga card address (virtual).

after insmoding the driver I get an oops of the following form:
BUG: unable to handle kernel paging request at 93800000

(ox93800000 is the virtual mem address of bus address 0xd3800000)
the dma size was of 2kB (less than a page size) in order to not deal with page boundaries.

Does anybody have a clue as to why I get this kernel crash ?
I guess I'm doing something naively wrong related to paging, but since I'm targeting a pci BAR and not the physical memory I don't see the error message relevance...
I tried to look for examples of i/oat dma code (between mem and pci-e BAR) but couldn't find anything simple enough to learn from...

[ 9744.007835] dmaengine:__dma_request_channel: success (dma0chan0)
[ 9744.008289] dmatest: Started 1 threads using dma0chan0
[ 9744.008305] LIOR: thread->dsts[0] = 93800000
[ 9744.008332] BUG: unable to handle kernel paging request at 93800000
[ 9744.008338] IP: [] dmatest_func+0x4b7/0xc7c [dmatest]
[ 9744.008347] *pdpt = 0000000000879001 *pde = 0000000000000000
[ 9744.008352] Oops: 0002 [#1] SMP
[ 9744.008356] last sysfs file: /sys/devices/pci0000:00/0000:00:1d.7/usb2/2-0:1.0/uevent
[ 9744.008361] Modules linked in: dmatest binfmt_misc ppdev lp iptable_filter psmouse ioatdma parport bnx2 ip_tables serio_raw dca x_tables joydev dcdbas usbhid ses enclosure megaraid_sas
[ 9744.008376]
[ 9744.008380] Pid: 25207, comm: dma0chan0-copy0 Not tainted (2.6.32.6 #3) PowerEdge R610
[ 9744.008383] EIP: 0060:[] EFLAGS: 00010202 CPU: 5
[ 9744.008386] EIP is at dmatest_func+0x4b7/0xc7c [dmatest]
[ 9744.008389] EAX: 00000000 EBX: 93800000 ECX: c3b51757 EDX: 0000001f
[ 9744.008391] ESI: 00000741 EDI: 000000a5 EBP: c41abfb8 ESP: c41abedc
[ 9744.008394] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 9744.008397] Process dma0chan0-copy0 (pid: 25207, ti=c41aa000 task=c39a1960 task.ti=c41aa000)
[ 9744.008400] Stack:
[ 9744.008402] c41abee8 00000286 c39a1960 c41abf04 c013d538 00000001 00000286 00000001
[ 9744.008408] <0> c445a1d0 f75b65e0 c41abfb8 f8336593 f833724c 93800000 c3efaf40 c41abf34
[ 9744.008415] <0> c0139e41 00000000 f749b2c0 05a67a59 c08716a0 c39a1960 c41abfb8 00000018
[ 9744.008422] Call Trace:
[ 9744.008433] [] ? set_user_nice+0xb8/0x110
[ 9744.008437] [] ? dmatest_func+0x303/0xc7c [dmatest]
[ 9744.008442] [] ? finish_task_switch+0x41/0xb0
[ 9744.008447] [] ? dmatest_func+0x0/0xc7c [dmatest]
[ 9744.008455] [] ? kthread+0x6c/0x80
[ 9744.008459] [] ? kthread+0x0/0x80
[ 9744.008466] [] ? kernel_thread_helper+0x7/0x10
[ 9744.008468] Code: 0f 84 8d 00 00 00 8b 75 94 03 75 a4 89 7d b0 8b 7d 94 89 75 b8 8d b4 26 00 00 00 00 31 c0 85 ff 74 13 66 90 89 c2 f7 d2 83 e2 1f <88> 14 03 83 c0 01 39 c7 77 ef 39 f7 89 f8 73 22 8d 0c 3b 89 fa
[ 9744.008505] EIP: [] dmatest_func+0x4b7/0xc7c [dmatest] SS:ESP 0068:c41abedc
[ 9744.008510] CR2: 0000000093800000
[ 9744.008515] ---[ end trace 11a3553055c8b122 ]---