defragmentation

Linux: Memory Compaction

Submitted by Jeremy
on April 2, 2010 - 5:11pm
Linux news

Mel Gorman posted the seventh version of his Memory Compaction patches asking, "are there any further obstacles to merging?" The patches, first posted in May of 2007, provide a mechanism for moving GFP_MOVABLE pages into a smaller number of pageblocks, reducing externally fragmented memory. Mel explains that 'compaction' is another method of defragmenting memory, "for example, lumpy reclaim is a form of defragmentation as was slub 'defragmentation' (really a form of targeted reclaim). Hence, this is called 'compaction' to distinguish it from other forms of defragmentation."

The core compaction patch explains that memory is compacted in a zone by relocating movable pages towards the end of the zone:

"A single compaction run involves a migration scanner and a free scanner. Both scanners operate on pageblock-sized areas in the zone. The migration scanner starts at the bottom of the zone and searches for all movable pages within each area, isolating them onto a private list called migratelist. The free scanner starts at the top of the zone and searches for suitable areas and consumes the free pages within making them available for the migration scanner. The pages isolated for migration are then migrated to the newly isolated free pages."

Linux: Slab Defragmentation

Submitted by Jeremy
on September 4, 2007 - 7:59am
Linux news

"Slab defragmentation is mainly an issue if Linux is used as a fileserver and large amounts of dentries, inodes and buffer heads accumulate," Christoph Lameter explained when posting the fifth version of his patchset. He continued, "in some load situations the slabs become very sparsely populated so that a lot of memory is wasted by slabs that only contain one or a few objects. In extreme cases the performance of a machine will become sluggish since we are continually running reclaim. Slab defragmentation adds the capability to recover wasted memory." Christoph noted that the patch is difficult to validate and measure because, "activities are only performed when special load situations are encountered." He then pointed to updatedb as something that typically triggers slab defragmentation on his systems:

"Updatedb scans all files on the system which causes a high inode and dentry use. After updatedb is complete we need to go back to the regular use patterns (typical on my machine: kernel compiles). Those need the memory now for different purposes. The inodes and dentries used for updatedb will gradually be aged by the dentry/inode reclaim algorithm which will free up the dentries and inode entries randomly through the slabs that were allocated. As a result the slabs will become sparsely populated. If they become empty then they can be freed but a lot of them will remain sparsely populated. That is where slab defrag comes in: It removes the slabs with just a few entries reclaiming more memory for other uses."