KernelTrap - -mm

Quote: Perform Some Serious MM Testing

Jeremy — Thu, 12 Jun 2008 19:59:43 +0000

"This is a bugfixed version of 2.6.26-rc5-mm2, which was a bugfixed version of 2.6.26-rc5-mm1. None of the git trees were repulled for -mm3 (and nor were they repulled for -mm2). The aim here is to get all the stupid bugs out of the way so that some serious MM testing can be performed. Please perform some serious MM testing."

Budget Fair Queuing IO Scheduler

Jeremy — Tue, 15 Apr 2008 18:02:59 +0000

"We are working [on] a new I/O scheduler based on CFQ, aiming at improved predictability and fairness of the service, while maintaining the high throughput it already provides," began Fabio Checconi, announcing the BFQ I/O scheduler. "The Budget Fair Queueing (BFQ) scheduler turns the CFQ Round-Robin scheduling policy of time slices into a fair queuing scheduling of sector budgets," he continued, "more precisely, each task is assigned a budget measured in number of sectors instead of amount of time, and budgets are scheduled using a slightly modified version of WF2Q+. The budget assigned to each task varies over time as a function of its behaviour. However, one can set the maximum value of the budget that BFQ can assign to any task." Fabio went on to explain:

"The time-based allocation of the disk service in CFQ, while having the desirable effect of implicitly charging each application for the seek time it incurs, suffers from unfairness problems also towards processes making the best possible use of the disk bandwidth. In fact, even if the same time slice is assigned to two processes, they may get a different throughput each, as a function of the positions on the disk of their requests. On the contrary, BFQ can provide strong guarantees on bandwidth distribution because the assigned budgets are measured in number of sectors. Moreover, due to its Round Robin policy, CFQ is characterized by an O(N) worst-case delay (jitter) in request completion time, where N is the number of tasks competing for the disk. On the contrary, given the accurate service distribution of the internal WF2Q+ scheduler, BFQ exhibits O(1) delay."

Jens Axboe reacted favorably, "Fabio, I've merged the scheduler for some testing. Overall the code looks great, you've done a good job!" He noted that the scheduler should soon appear in the -mm tree, and that it was worth considering merging the two I/O schedulers together.

Distributed Storage Subsystem Feature Complete

Jeremy — Thu, 08 Nov 2007 17:25:01 +0000

"I'm pleased to announce [the] 7'th and final release of the distributed storage subsystem (DST)," Evgeniy Polyakov stated, completing the TODO list on the project's web page. He titled the release, "squizzed black-out of the dancing back-aching hippo", noting, "it clearly shows my condition". New features in this release include checksum support, extended auto-configuration for detecting and auto-enabling checksums if supported by the remote host, new sysfs files for marking a given node as clean (in-sync) or dirty (not-in-sync), and numerous bug fixes.

Evgeniy released the first version of his distributed storage subsystem in July of 2007. In September he explained that this was the first step in a larger distributed filesystem project he's planning. In late October, Andrew Morton noted that the work looked ready to be merged into his -mm kernel.

Distributed Storage Subsystem Headed For -mm

Jeremy — Sun, 28 Oct 2007 03:50:03 +0000

Andrew Morton responded favorably to Evgeniy Polyakov's most recent release of his distributed storage subsystem, "I went back and re-read last month's discussion and I'm not seeing any reason why we shouldn't start thinking about merging this." He then asked, "how close is it to that stage? A peek at your development blog indicates that things are still changing at a moderate rate?" Evgeniy replied:

"I completed storage layer development itself, the only remaining todo item is to implement [a] new redundancy algorithm, but I did not see major demand on that, so it will stay for now with low priority. I will use DST as a transport layer for [a] distributed filesystem, and probably that will require additional features, I have no clean design so far, but right now I have nothing in the pipe to commit to DST."

2.6.23-mm1, "Working a Bit Better"

Jeremy — Sat, 13 Oct 2007 09:07:31 +0000

Andrew Morton posted his first -mm patchset against the recently released 2.6.23 kernel, preparing for a big merge of patches bound for inclusion in the upcoming 2.6.24 kernel. He noted:

"I've been largely avoiding applying anything since rc8-mm2 in an attempt to stabilise things for the 2.6.23 merge.

"But that didn't stop all the subsystem maintainers from going nuts, with the usual accuracy. We're up to a 37MB diff now, but it seems to be working a bit better."

Merging From -mm in 2.6.24

Jeremy — Mon, 01 Oct 2007 19:56:01 +0000

With the official release of the 2.6.23 kernel expected any day now, Andrew Morton posted his -mm merge plans for the 2.6.24 kernel. The current Linux kernel development model is to open up the mainline kernel for significant merges during the two weeks following a major kernel release. Thus, during the two weeks following the imminent release of the 2.6.23 kernel, subsystem maintainers will push their latest trees to Linus' mainline tree. Andrew Morton will also push many of the patches he collects in his -mm tree to Linus' mainline tree during these two weeks, as detailed in his email. At the end of the merge window, 2.6.24-rc1 will be released and the stabilization process begins, though in reality significant merges also often slip in between -rc1 and -rc2. A series of -rc kernels will be released, eventually leading to a stable 2.6.24 kernel two or three months after the process started, and it all starts again.

Read-only Bind Mounts

Jeremy — Mon, 24 Sep 2007 08:59:46 +0000

"This feature allows a read-only view into a read-write filesystem. In the process of doing that, it also provides infrastructure for keeping track of the number of writers to any given mount," Dave Hansen began, describing his "read-only bind mounts" patches. He continued, "this has a number of uses. It allows chroots to have parts of filesystems writable. It will be useful for containers in the future because users may have root inside a container, but should not be allowed to write to some filesystems. This also replaces patches that vserver has had out of the tree for several years. It allows security enhancements by making sure that parts of your filesystem [are] read-only (such as when you don't trust your FTP server), when you don't want to have entire new filesystems mounted, or when you want atime selectively updated."

Christoph Hellwig was interested in seeing the patches get some more testing, "I still think we really want this in -mm. As we've seen at the kernel summit there's a pretty desperate need for it." Andrew Morton noted that the "unprivileged mounts" code was working in the same area, but described that work as "a bit stuck." He suggested, "it sounds like a better approach would be for me to merge the r/o bind mounts code and to drop (or maybe rework) the unprivileged mounts patches." Dave explained that they don't collide much, to which Andrew's reply suggested that the read-only mount patches would be merged into the -mm kernel soon.

Tiny Linux Redux

Jeremy — Thu, 20 Sep 2007 09:49:48 +0000

"Recently, the CE Linux forum has been working to revive the Linux-tiny project," stated Tim Bird on the Linux Kernel mailing list, adding that Michael Opdenacker has been selected as the project's new primary maintainer. The project's website explains:

"The linux-tiny patchset is a series of patches against the 2.6 mainline Linux kernel to reduce its memory and disk footprint, as well as to add features to aid working on small systems. Target users are developers of embedded system and users of small or legacy machines such as 386s and handhelds."

Andrew Morton suggested that patches should be sent to him to be merged into his -mm tree, aiming for inclusion in the mainline kernel, "seriously, putting this stuff into some private patch collection should be a complete last resort - you should only do this with patches which you (and the rest of us) agree have no hope of ever getting into mainline." Michael, the project's new maintainer, agreed, "you're completely right... The patches should all aim at being included into mainline or die." Tim added, "the patchkit gives a place for things to live while they are out of mainline, and still have multiple people use and work on them. Optimally the duration of being out-of-mainline would be short, but my experience is that sometimes what an embedded developer considers reasonable to hack off the kernel is not considered so reasonable by other developers (even with config options)."

-mm Instability

Jeremy — Wed, 19 Sep 2007 11:03:58 +0000

Following Andrew Morton's recent comment, "this just isn't working any more," Miles Lane asked, "what can be done to reduce the huge number of build fixes required to release an MM tree?" Andrew jokingly replied, "my mind turns to cattle prods." Regarding the suggestion that he could publicly list the offenders he quipped, "I could name names, but it would look like 'grep @ MAINTAINERS' ;))" He continued to say, "I don't think much can be done about it, really," going on to explain:

"See, what I do is to merge probably hundreds of patches into the -mm-only part of the tree and then, after a few days, get down and compile-test it all, then fix it, then runtime test it all, then fix that. Because it is vastly more efficient to do all this work against hundreds of patches than it is to do it against one patch at a time, no?

"And guess what? All the other maintainers do the same thing: someone sends you a patch, it looks good, so you commit it. After you've committed a decent batch of patches, get in there and test it all. Problem is, I often will get in there and do all that testing before the subsystem-tree owner has done his testing."