KernelTrap - 2.6.23

Multiple Stable 2.6.23 Releases

Jeremy — Sun, 18 Nov 2007 19:22:10 +0000

"Ok, I've been slacking on the -stable front for a bit here, and didn't realize how far behind I've gotten. Everyone has been sending patches in, which is great, but now we are facing a HUGE 114 patch release," began Greg Kroah-Hartman. He continued:

"As there's no real way that everyone can review all of these patches, I've decided to split them up into 6 different categories, and will be sending patches out in these categories for review. If people can just glance over the ones in the areas they care about, I would really appreciate it."

The stable review resulted in six stable 2.6.23.y releases. The first, 2.6.23.2, contained bug fixes for the core kernel code. 2.6.23.3 contained bug fixes for architecture specific issues. 2.6.23.4 contained bug fixes for the core networking and wireless code. 2.6.23.5 contained bug fixes for networking drivers. 2.6.23.6 contained bug fixes for non-networking drivers. 2.6.23.7 contained file system bug fixes. These releases were followed by 2.6.23.8 containing a couple security fixes.

Scheduler Fixes

Jeremy — Mon, 12 Nov 2007 16:25:35 +0000

Ingo Molnar sent a merge request to Linus Torvalds for the latest CFS fixes. CFS, the Completely Fair Scheduler, was merged into the mainline Linux kernel in July of 2007. It was first included in the 2.6.23 kernel, released in October of 2007. The scheduler appears to be quickly stabilizing, visible in the minimal assortment of fixes contained in the latest source code push. Ingo Molnar summarized the changes:

"There are two cross-subsystem groups of fixes: three commits that resolve a KVM build fix on !SMP - acked by Avi to go in via the scheduler git tree because it changes a central include file. The other one is a powerpc CPU time accounting regression fix from Paul Mackerras.

"The remaining 14 commits: one crash fix (only triggerable via the new control-groups filesystem), a delay-accounting regression fix, two performance regression fixes, a latency fix, two small scheduling-behavior regression fixes and seven cleanups."

Who Sponsors Kernel Development

Jeremy — Tue, 16 Oct 2007 01:05:52 +0000

In a recent discussion on the Linux Kernel mailing list, it was asked "which companies are helping develop the kernel?" One response pointed out that that a recent article on lwn.net summarized this information nicely. Others pointed to a paper maintained by Greg KH, who responded:

"Yeah, but my paper didn't really track companies very well. The lwn.net article is the best, and below is my version of who did things in 2.6.23. Note, the lack of a company is not an indicator that they did nothing, just that I could not easily determine someone worked for them. I'll try to send out my 'who are you working for' emails in a week or so to see if I can further categorize the 'unknowns'."

According to Greg's email, organizations that contributed more than 100 changesets to the recently released 2.6.23 kernel included: Red Hat with 827 changesets (11.7%), IBM with 557 changesets (7.9%), the Linux Foundation with 528 changesets (7.5%), Novell with 449 changesets (6.3%), Intel with 242 changesets (3.4%), Oracle with 158 changesets (2.2%), MIPS Technologies with 143 changesets (2%), Nokia with 133 changesets (1.9%), and NetApp with 119 changesets (1.7%). Greg noted that there were a total of 7,075 changesets from 992 developers working for 126 different employers. 843 of the changesets (11.9%) were contributed by individuals reporting no sponsor for their work.

2.6.23-mm1, "Working a Bit Better"

Jeremy — Sat, 13 Oct 2007 09:07:31 +0000

Andrew Morton posted his first -mm patchset against the recently released 2.6.23 kernel, preparing for a big merge of patches bound for inclusion in the upcoming 2.6.24 kernel. He noted:

"I've been largely avoiding applying anything since rc8-mm2 in an attempt to stabilise things for the 2.6.23 merge.

"But that didn't stop all the subsystem maintainers from going nuts, with the usual accuracy. We're up to a 37MB diff now, but it seems to be working a bit better."

Measuring Process Scheduler Performance

Jeremy — Wed, 10 Oct 2007 13:02:52 +0000

"As far as my testsystem goes, v2.6.23 beats v2.6.22.9 in sysbench," explained Ingo Molnar in response to a posting showing the opposite results. He referred to his own testing results and explained:

"As you can see it in the graph, v2.6.23 schedules much more consistently too. [ v2.6.22 has a small (but potentially statistically insignificant) edge at 4-6 clients, and CFS has a slightly better peak (which is statistically insignificant)."

Ingo noted that he was nuable to find information as to how the other benchmark was generated, "there are no .configs or other testing details at or around that URL that i could use to reproduce their result precisely, so at least a minimal bugreport would be nice." He then offered some tips on how sysbench works and some suggested tunings, "sysbench is a pretty 'batched' workload: it benefits most from batchy scheduling: the client doing as much work as it can, then server doing as much work as it can - and so on. The longer the client can work the more cache-efficient the workload is. Any round-trip to the server due to pesky preemption only blows up the cache footprint of the workload and gives lower throughput."

2.6.23 Kernel, "Finally"

Jeremy — Tue, 09 Oct 2007 18:08:11 +0000

"Finally. Yeah, it got delayed, not because of any huge issues, but because of various bugfixes trickling in and causing me to reset my 'release clock' all the time. But it's out there now, and hopefully better for the wait," Linus Torvalds said announcing the 2.6.23 kernel. He noted few changes since the last release candidate, "not a whole lot of changes since -rc9, although there's a few updates to mips, sparc64 and blackfin in there. Ignoring those arch updates, there's basically a number of mostly one-liners (mostly in drivers, but there's some networking fixes and soem VFS/VM fixes there too)." Source level changes can be viewed via the gitweb interface. A nice overview of all changes can be found at Kernel Newbies. Linus went on to describe his plan going forward:

"I want this to be what people look at for a few days, but expect the x86 merge to go ahead after that. So far, all indications are still that it's going to be all smooth sailing, but hey, those indicators seem to always say that, and only after the fact do people notice any problems ;)"

Compiler Optimization Bugs and World Domination

Jeremy — Fri, 05 Oct 2007 09:09:36 +0000

A bug report filed by Ingo Molnar regarding a procfs crash in the recently released 2.6.23-rc9 kernel was quickly tracked down by Linus Torvalds as a compiler bug. The bug was ultimately determined to be from a compiler optimization generated with an older version of GCC. Ingo was skeptical at first, "it's 4.0.2. Not the latest & greatest but I've been using it for 2 years and this would be the first time it miscompiles a 32-bit kernel out of tens of thousands of successful kernel bootups." Linus replied, "I am 100% sure. I can look at the disassembly, and point to the fact that your Oops happens on code that is simply totally bogus." He continued on to offer an interesting review of the crash, explaining line by line what should have been generated versus what actually was, causing the crash. In the end, Ingo switched to a distribution compiled GCC 4.1.2 and confirmed that the crash went away, "so you are completely right, it's a compiler bug in 4.0.2."

During the thread, Linus suggested that the optimization made by the compiler wasn't "legal", to which Alan Cox retorted, "pedant: valid. Almost all optimizations are legal, nobody has yet written laws about compilers. Sorry but I'm forever fixing misuse of the word 'illegal' in printks, docs and the like and it gets annoying after a bit." Linus playfully responded, "heh. When I'm ruler of the universe, it *will* be illegal. I'm just getting a bit ahead of myself." When asked how long until he expected to be ruler, Linus added, "I'm working on it, I'm working on it. I'm just as frustrated as you are. It turns out to be a non-trivial problem."

2.6.23-rc9, Avoiding Brown Paper Bags

Jeremy — Tue, 02 Oct 2007 05:53:28 +0000

"I said I was hoping that -rc8 was the last -rc, and I hate doing this, but we've had more changes since -rc8 than we had in -rc8. And while most of them are pretty trivial, I really couldn't face doing a 2.6.23 release and take the risk of some really stupid brown-paper-bag thing," Linus Torvalds said, announcing the 2.6.23-rc9 kernel. He added, "so there's a final -rc out there, and right now my plan is to make this series really short, and release 2.6.23 in a few days. So please do give it a last good testing, and holler about any issues you find!" He went on to warn developers that the first thing planned for 2.6.24 was to merge the unified x86 architecture:

"This is also a good time to warn about the fact that we're doing the x86 merge very soon (as in the next day or two) after 2.6.23 is out, so if you have pending patches for the next series that touch arch/i386 or x86-64, you should get in touch with Thomas Gleixner and Ingo Molnar, who are the keepers of the merge scripts, and will help you prepare..

"Doing it as early as possible in the 2.6.24-rc4 series (basically I'll do it first thing) will mean that we'll have the maximum amount of time to sort out any issues, and the thing is, Thomas and Ingo already have a tree ready to go, so people can check their work against that, and don't need to think that they have to do any fixups after it his *my* tree. It would be much better if everybody was just ready for it, and not taken by surprise."

Merging From -mm in 2.6.24

Jeremy — Mon, 01 Oct 2007 19:56:01 +0000

With the official release of the 2.6.23 kernel expected any day now, Andrew Morton posted his -mm merge plans for the 2.6.24 kernel. The current Linux kernel development model is to open up the mainline kernel for significant merges during the two weeks following a major kernel release. Thus, during the two weeks following the imminent release of the 2.6.23 kernel, subsystem maintainers will push their latest trees to Linus' mainline tree. Andrew Morton will also push many of the patches he collects in his -mm tree to Linus' mainline tree during these two weeks, as detailed in his email. At the end of the merge window, 2.6.24-rc1 will be released and the stabilization process begins, though in reality significant merges also often slip in between -rc1 and -rc2. A series of -rc kernels will be released, eventually leading to a stable 2.6.24 kernel two or three months after the process started, and it all starts again.