Christoph Lameter wrote:

quoted text
> Fundamentally there is no way to avoid complex queueing on free() unless 
> one directly frees the object. This is serialized in SLUB by taking a page 
> lock. 

iirc profiling analysis showed that the problem was the page lock
serialization (in particular the slab_lock() in __slab_free). That
was on 2.6.24.2

quoted text
> Howver, the "slow" case in SLUB is still much less complex 
> than comparable processing in SLAB. It is quite fast.

Well in the benchmark it is slower.


quoted text
> SLAB freeing can avoid taking a lock if
> 
> 1. We can establish that the object is node local (trivial if !NUMA 
> otherwise we need to get the node information from the page struct and 
> compare to the current node).

Ignoring NUMA is no option unfortunately. And with integrated memory
controller many of the remote CPU frees are off node.

quoted text
> The main issue for SLAB vs. SLUB on free is likely the !NUMA case in which 
> SLAB can avoid the overhead of the node check (which does not exist in 
> SLUB) and in which case we can always immediately batch the object (if 
> there is space). The additional overhead in SLUB is mainly one 
> atomic instruction over the SLAB fastpath.

I think the problem is that this atomic operation thrashes cache lines
around. Really counting cycles on instructions is not that interesting,
but minimizing the cache thrashing is. And for that it looks like slub
is worse.

quoted text
> So I think that the free need to  stay as is. The disadvantages in terms 
> of the complexity of handling the objects and expiring them and the issue 
> of having to take per node locks in SLAB makes it hard to justify adding a 
> queue for free in SLUB. Maybe someone has an inspiration on how to do this 
> effective that is better than my attempts which always ultimately ended 
> implementing code that thad the same issues that we have in SLAB.

What is the big problem of having a batched free queue? If the expiry
is done at a good bounded time (e.g. on interrupt exit or similar)
locally on the CPU it shouldn't be a big issue, should it?

-Andi
--
unsubscribe notice
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/