Re: [00/41] Large Blocksize Support V7 (adds memmap support)

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Goswin von Brederlow
Date: Saturday, September 22, 2007 - 10:50 pm

mel@skynet.ie (Mel Gorman) writes:


Two things to both of you respectively.

Why should we try to stay out of the pte slab? Isn't the slab exactly
made for this thing? To efficiently handle a large number of equal
size objects for quick allocation and dealocation? If it is a locking
problem then there should be a per cpu cache of ptes. Say 0-32
ptes. If you run out you allocate 16 from slab. When you overflow you
free 16 (which would give you your 64k allocations but in multiple
objects).

As for the wastage. Every pte can map 2MB on amd64, 4MB on i386, 8MB
on sparc (?). A 64k pte chunk would be 32MB, 64MB and 32MB (?)
respectively. For the sbrk() and mmap() usage from glibc malloc() that
would be fine as they grow linear and the mmap() call in glibc could
be made to align to those chunks. But for a programm like rtorrent
using mmap to bring in chunks of a 4GB file this looks desasterous.


Personally I would really go with a per cpu cache. When mapping a page
reserve 4 tables. Then you walk the tree and add entries as
needed. And last you release 0-4 unused entries to the cache.

MfG
        Goswin
-
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Mon Sep 10, 11:03 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Tue Sep 11, 5:05 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Tue Sep 11, 9:02 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Tue Sep 11, 9:47 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Maxim Levitsky, (Tue Sep 11, 11:25 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Tue Sep 11, 12:20 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 1:01 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 1:03 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 1:07 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 1:11 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 1:13 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 1:41 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 1:42 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 2:35 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 2:41 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 2:48 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 2:52 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Tue Sep 11, 4:26 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 5:00 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 11, 5:04 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Wed Sep 12, 1:20 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Martin J. Bligh, (Wed Sep 12, 7:29 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Wed Sep 12, 4:06 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Wed Sep 12, 4:17 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Thu Sep 13, 7:38 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Fri Sep 14, 9:10 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Fri Sep 14, 10:52 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Fri Sep 14, 11:08 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Fri Sep 14, 11:15 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Fri Sep 14, 5:31 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Fri Sep 14, 5:33 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sat Sep 15, 5:14 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Sat Sep 15, 8:51 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sat Sep 15, 1:14 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Sat Sep 15, 3:30 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sun Sep 16, 6:54 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Sun Sep 16, 8:08 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Linus Torvalds, (Sun Sep 16, 11:15 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Linus Torvalds, (Sun Sep 16, 11:44 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Sun Sep 16, 11:50 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Sun Sep 16, 2:31 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sun Sep 16, 2:58 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sun Sep 16, 3:06 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sun Sep 16, 3:38 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sun Sep 16, 3:48 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sun Sep 16, 3:51 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sun Sep 16, 3:56 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Mon Sep 17, 3:00 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Mon Sep 17, 3:03 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Mon Sep 17, 3:05 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Mon Sep 17, 3:10 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Mon Sep 17, 3:21 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Linus Torvalds, (Tue Sep 18, 11:30 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Tue Sep 18, 12:18 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Tue Sep 18, 12:31 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Linus Torvalds, (Tue Sep 18, 12:44 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 18, 1:36 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Tue Sep 18, 1:42 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Wed Sep 19, 7:04 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Thu Sep 20, 7:54 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Thu Sep 20, 11:07 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Thu Sep 20, 11:11 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sat Sep 22, 10:50 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sat Sep 22, 11:22 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sat Sep 22, 11:49 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Goswin von Brederlow, (Sat Sep 22, 11:56 pm)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Andrea Arcangeli, (Mon Sep 24, 8:39 am)
Re: [00/41] Large Blocksize Support V7 (adds memmap support), Christoph Lameter, (Mon Sep 24, 2:13 pm)