KernelTrap - Rik van Riel

Quote: Linux Will Have Similar Problems

Jeremy — Mon, 18 Aug 2008 17:28:00 +0000

"If web browsers, office suites and mail clients on Windows have certain kinds of vulnerabilities, it is safe to assume that the same programs on Linux will have similar problems."

Quote: Data Integrity

Jeremy — Tue, 15 Jan 2008 21:29:29 +0000

"You can't play fast and loose with data integrity."

Quote: Good For Development, Bad For Business

Jeremy — Fri, 02 Nov 2007 04:35:07 +0000

"Exposing bugs is good for development, bad for business."

Memory Management Improvements

Jeremy — Sat, 27 Oct 2007 10:04:59 +0000

A recent report on the lkml suggested improved IO/writeback performance in the recently released 2.6.24-rc1 kernel compared to the earlier 2.6.19.2 and 2.6.22.6 kernels. Credit was given to some patches by Peter Zijlstra. Ingo Molnar replied, "wow, really nice results! Peter does know how to make stuff fast :) Now lets pick up some of Peter's other, previously discarded patches as well :-)" He pointed to several patches "as a starter", then quipped, "I think the MM should get out of deep-feature-freeze mode - there's tons of room to improve :-/"

Andrew Morton replied, "kidding. We merged about 265 MM patches in 2.6.24-rc1: 482 files changed, 8071 insertions(+), 5142 deletions(-)". He added, "a lot of that was new functionality. That's easier to add than things which change long-standing functionality." Of the patches Ingo pointed to, Peter noted he was currently working on polishing the swap-over-NFS patch, "will post that one again, soonish.... Esp. after Linus professed liking to have swap over NFS." Rik van Riel also replied regarding rewriting the page replacement code, "at the moment I only have the basic 'plumbing' of the split VM working and am fixing some bugs in that. Expect a patch series with that soon, so you guys can review that code and tell me where to beat it into shape some more :)"

Quote: What Is It?

Jeremy — Thu, 25 Oct 2007 04:31:14 +0000

"That's very nice, but ... what is it? It would be really helpful if each patch series came with some kind of description of what problem the patches try to solve and how it solves it :)"

Signaling When Out of Memory

Jeremy — Fri, 19 Oct 2007 13:29:00 +0000

The previous 2.4 Linux kernel maintainer, Marcelo Tossati, resurrected a discussion on adding support for out of memory notifications to the Linux kernel. He explained, "AIX contains the SIGDANGER signal to notify applications to free up some unused cached memory," then noting, "there have been a few discussions on implementing such an idea on Linux, but nothing concrete has been achieved." In a request for discussion, Marcelo added, "on the kernel side Rik suggested two notification points: 'about to swap' (for desktop scenarios) and 'about to OOM' (for embedded-like scenarios)." Rik van Riel explained:

"The first threshold - 'we are about to swap' - means the application frees memory that it can. Eg. free()d memory that glibc has not yet given back to the kernel, or JVM running the garbage collector, or ...

"The second threshold - 'we are out of memory' - means that the first approach has failed and the system needs to do something else. On an embedded system, I would expect some application to exit or maybe restart itself."

Student Kernel Projects

Jeremy — Mon, 15 Oct 2007 06:54:40 +0000

"The kernel newbies community often gets inquiries from CS students who need a project for their studies and would like to do something with the Linux kernel, but would also like their code to be useful to the community afterwards," explained Rik van Riel in a posting titled "WANTED: kernel projects for CS students". He offered a link to a Kernel Newbies wiki page titled "KernelProjects" adding, "if you have ideas on what projects would be useful, please add them to this page (or email me)". Rik explained that he was assembling a list of projects on that page that meet the following criteria:

"Are self contained enough that the students can implement the project by themselves, since that is often a university requirement; are self contained enough that Linux could merge the code (maybe with additional changes) after the student has been working on it for a few months; are large enough to qualify as a student project, luckily there is flexibility here since we get inquiries for anything from 6 week projects to 6 month projects."

Virtual Machine Time Accounting

Jeremy — Wed, 12 Sep 2007 17:18:52 +0000

"The aim of these four patches is to introduce Virtual Machine time accounting," began Laurent Vivier. He described the first two patches as:

"1) As recent CPUs introduce a third running state, after 'user' and 'system', we need a new field, 'guest', in cpustat to store the time used by the CPU to run virtual CPU. Modify /proc/stat to display this new field.

"2) Like for cpustat, introduce the 'gtime' (guest time of the task) and 'cgtime' (guest time of the task children) fields for the tasks. Modify signal_struct and task_struct. Modify /proc/<pid>/stat to display these new fields."

Both Ingo Molnar and Rik van Riel responded favorably to the patch. Ingo replied, "the concept certainly looks sane to me," adding, "I'd suggest inclusion into 2.6.24." Regarding concerns that the new information at the end of the line could break utilities such as top or ps, Rik assured that it would not, "we have added numbers to the cpu lines in /proc/stat since early 2.6. All the programs parsing /proc/stat should just scan for a number of numbers from the start of the line, without trying to scan for the terminating newline."

Improving kswapd

Jeremy — Fri, 07 Sep 2007 02:56:51 +0000

"The current VM can get itself into trouble fairly easily on systems with a small ZONE_HIGHMEM, which is common on i686 computers with 1GB of memory," Rik van Riel said explaining a small patch to cmscan.c. He continued, "on one side, page_alloc() will allocate down to zone->pages_low, while on the other side, kswapd() and balance_pgdat() will try to free memory from every zone, until every zone has more free pages than zone->pages_high." He noted that highmem could be filled up with "page tables, ramfs, vmalloc allocations and other unswappable things quite easily and without many bad side effects, since we still have a huge ZONE_NORMAL to do future allocations from. However, as long as the number of free pages in the highmem zone is below zone->pages_high, kswapd will continue swapping things out from ZONE_NORMAL, too! Sami Farin managed to get his system into a stage where kswapd had freed about 700MB of low memory and was still 'going strong'." He described his patch:

"The attached patch will make kswapd stop paging out data from zones when there is more than enough memory free. We do go above zone->pages_high in order to keep pressure between zones equal in normal circumstances, but the patch should prevent the kind of excesses that made Sami's computer totally unusable."