"Ok, it's out there, ready for your enjoyment," Linus Torvalds said, announcing the 2.6.25-rc3 kernel. He summarized the changes:
"As usual, most of the updates are in architecture and drivers, with the dirstat showing about 37% in arch (and that's with rename detection: there's some file movement in arch/xtensa that would bring it up to 43% if you looked at it as a traditional diff) and almost 50% in drivers. Much of the include file stuff is also architecture-related updates. The driver updates are mostly fairly spread out, but some of it comes from a couple of new drivers: the mvsas SCSI driver, a new adt7473 driver, and a couple of new watchdog drivers."
Linus continued, "if you ignore the architecture-specific stuff and drivers, the rest is mostly in networking, some Documentation updates, and a few filesystem updates (mainly efs and xfs). Anyway, the upshot of it all? Quite frankly, it's all over the place. The changes in -rc3 are bigger than -rc2, probably mostly because we had some more time (-rc2 was a couple of days early because of the long weekend in the US), but hopefully also because people have started to find regressions." Among the bug fixes, he highlighted, "we had a nasty SLUB corruption issue in -rc2 that is fixed (not that very many people probably saw it), and we've hopfully fixed a number of regressions in networking and suspend/resume."
User account creation has been temporarily disabled on KernelTrap.org due to account abuse by web spammers. I am working on a solution to automatically detect and block these spam accounts, but until I have time to finish the effort I will be leaving user accounts disabled. I apologize for the inconvenience.
Issues reported during the suspend-to-disk process lead Linux creator Linus Torvalds to suggest, "please - just make the f*cking suspend-to-disk use other routines already. 99% of all hardware needs to do exactly *nothing* on suspend-to-disk, and the ones that really do need things tend to need to not do a whole lot." He went on to explain why sharing the code path for suspend-to-disk and freezing to RAM is wrong:
"For example, the 'freeze' action for USB (which is one of the hardest things to suspend) should literally be something like just setting the controller STOP bit, and waiting for it to have stopped. The 'unfreeze' should be to just clear the stop bit, while the 'restart' should be just a controller reset to use the current memory image. NONE OF THIS HAS ABSOLUTELY ANYTHING TO DO WITH SUSPEND. It never did. I've told people so for years. Maybe actually seeing the problems will make people realize."
Linus also noted another advantage to having separate code paths for the two actions, "the other issue is that I've long wanted to make sure that when people fix suspend-to-ram, they don't screw up suspend-to-disk by mistake and vice versa." During the discussion, Rafael Wysocki noted that he would be fixing this up presently, "I'm already convinced, really. :-)"
"These patches add local caching for network filesystems such as NFS," began David Howells describing an updated set of thirty-seven patches to introduce FS-Cache. When asked how the patches affect performance, he noted that this was dependent on the use case, highlighting issues when dealing with lots of metadata, "getting metadata from the local disk fs is slower than pulling it across an unshared gigabit ethernet from a server that already has it in memory."
David continued "these points don't mean that fscache is no use, just that you have to consider carefully whether it's of use to *you* given your particular situation, and that depends on various factors," adding, "note that currently FS-Caching is disabled for individual NFS files opened for writing as there's no way to handle the coherency problems thereby introduced." He concluded with a number of simple performance benchmarks.
"To quote you a number of years ago: 'Linux is evolution, not intelligent design'," noted Greg KH, quoting Linux creator Linus Torvalds. Linus expanded on the statement, "evolution often does odd (and 'suboptimal') things exactly because it does incremental changes that DO NOT BREAK at any point." He continued, "in other words, exactly *because* evolution requires 'bisectability' (any non-viable point in between is a dead end by definition) and does things incrementally, it doesn't do big flips." When alternative examples in evolution were pointed out, Linus suggested that the kernel was much simpler than a mammal and more similar to bacteria:
"In bacteria (and viruses), duplication of DNA/RNA is a big cost of living in general, and as a result there is *much* less junk DNA. So in an evolutionary sense, it's much closer to what the kernel should have (with occasional duplication of code and interfaces to allow new functionality, but rather aggressive pruning of the excess baggage). In other words, all of these choices are a matter of 'balance'. In some areas, excess code is not a sufficient downside, and we keep even broken source code around with no actual function, 'just because' (or rather, because the cost of carrying it around is so small that nobody cares)."
"Ok, this kernel is a winner," began Linux creator Linus Torvalds, playfully announcing the 2.6.25-rc2 kernel which gained the name "Funky Weasel is Jiggy wit it". He continued:
"Just to show how _much_ of a winner it is, it's been awarded a coveted 'weasel' series name, which should tell you just how good it's going to be. It's a name revered in Linux kernel history, and as such this brings back the good old days where if you find a bug, you're almost certainly simply mistaken, and you probably just did something wrong. But hey, you can try to prove me wrong. I dare you."
Linus went on to describe some of the changes using 'git dirstat', "in particular, it shows that almost exactly half of the updates are to drivers, with network drivers alone being a third of the whole patch. And of the remaining half, about half was architecture updates, notably to SH." He then noted, "I'm optimistic that this release cycle won't be anywhere near the pain of what 24 was, which is why I'm just going to go off for the long weekend and stay at the beach."
"Andrew [Morton] was looking for someone to run a linux-next tree that just contained the subsystem git and quilt trees for 2.6.x+1 and I (in a moment of madness) volunteered. So, this is to announce the creating of such a tree," began Stephen Rothwell, resulting in a lengthy thread discussing the current Linux kernel development process. In a follow up email announcing the first linux-next release, Stephen went on to explain, "it has two branches - master and stable. Stable is currently just Linus' tree and will never rebase. Master will rebase on an almost daily basis (maybe slower at the start)." He then detailed the master branch:
"The tree consists of subsystem git and quilt trees. Currently, the quilt trees are integrated by importing them into appropriately based git branches and then merging those branches. This has the advantage that any conflict resolution will onlt have to happen once at the merge point rather than, possibly, several times during the series. However, I am considering just applying the quilt trees on top of the current tree to get a result more like Linus' tree - we will see. The git trees are obviously just merged."
One of the goals of the new tree is to get subsystem maintainers more involved in managing merge conflicts by quickly notifying all involved when things break, and automatically dropping the offenders until build problems are fixed. Andrew plans to base his -mm kernel on the new linux-next tree, providing a stabler test branch for code before it's merged into Linus' mainline kernel tree.
Patches for a much publicized Linux kernel local root exploit were released today as 2.6.24.2, 2.6.23.16, and 2.6.22.18. The latest bug, labeled as CVE-2008-0600, was introduced by the vmsplice() system call and added into the 2.6 kernel in 2.6.17. It is the third in a series of root exploits surrounding the same system call, the two earlier bugs being CVE-2008-0009 and CVE-2008-0010. Easily obtained exploits exist for both the older CVE-2008-0010 which affected the 2.6.23 and 2.6.24 kernels, and the latest CVE-2008-0600, allowing a local non-root user to gain root permissions.
"Ok, it's a bloody large -rc (as was 24-rc1, for that matter), probably because the 2.6.24 release cycle dragged out, so people had a lot of things pending," noted Linus Torvalds, announcing the 2.6.25-rc1 kernel. He added, "the full diff is something like 11MB and 1.4M lines of diffs, with the bulk of the stuff being in architecture updates and drivers." Linus continued:
"Just to have some fun, I did trivial statistics, and of the 1.4M lines of diffs, about 38% - 530k lines - were in architecture files (400k+ lines of diffs in arch/, 100k+ lines of diffs in include/asm-*), and another big chunk is in drivers (including sound) at about 44% - 610k lines - of changes. The rest comes in much smaller, but still noticeable is networking (8% - 110k lines), with filesystems at 4%, and documentation at about 2%. The remaining crumbles being spread out mostly over block layer, crypto, kernel core, and security layer updates (ie SElinux and smack)."
Linus highlighted a few of the changes, including, "the Intel graphics driver is starting to do suspend/resume natively (ie even without X support), which is a welcome sign of the times and may help some people; lots of cleanups from the x86 merge (making more and more use of common files), but also the big page attribute stuff is in and caused a fair amount of churn, and while most of the issues should have been very obvious and all got fixed, this is definitely one of those things that we want a lot of very wide testing of to make sure nothing regressed; fair number of changes to things like the legacy IDE drivers too, and a totally new driver for the very common PCIE version of the Intel e1000 network card etc; and I've probably totally forgotten about tons of other stuff I should have mentioned, but the point is that not only do we have lots of new core, we do have a fair amout of changes to basic stuff that can actually affect perfectly bog-standard hardware setups. So give it all a good testing."
"While this is probably one of the last days of the merge window, please still consider pulling the 'kgdb light' git tree," began Ingo Molnar, explaining:
"This is a slimmed-down and cleaned up version of KGDB that i've created out of the original patches that we submitted two weeks ago. I went over the kgdb patches with Thomas and we cut out everything that we did not like, and cleaned up the result. KGDB is still just as functional as it was before (i tested it on 32-bit and 64-bit x86) - and any desired extra capability or complexity should be added as a delta improvement, not in this initial merge."
Ingo noted that the previous merge request modified 41 files, while this new merge request modifies only 22 files. Among the changes, he highlighted, "removed _all_ critical path impact, even if KGDB is enabled and active; removed all the lowlevel serial drivers; added a redesigned and cleaned up version of the 'KGDB over polled consoles' approach; removed the longjump code; removed the module symbol hacks; removed the GTOD/clocksource hacks; removed the softlockup hacks; removed the toplevel Makefile changes; removed the might_sleep scheduler hack; and did lots of other cleanups and rewrites as well." Ingo summarized, "as a result, this kgdb series has _obviously_ zero impact on the kernel, because it just does not touch any dangerous codepath. From this point on KGDB can evolve in small, well-controlled baby steps, as all other kernel code as well. And the resulting kgdb is still very functional: it can still break into a kernel (via SysRq-G), can catch crashes, can single-step, etc. It's already a quite usable first step."
"With a lot of help from Ingo Molnar and Pekka Enberg over the last couple of weeks, we've been able to produce a new version of kmemcheck!" announced Vegard Nossum, adding, "the current version of the patch boots on real hardware, but we've seen freezes on some machines, so it's not perfect yet. (In other words, this patch is HIGHLY experimental, and run at your own risk, etc.)". He also offered a high level summary of the patch:
"kmemcheck is a patch to the linux kernel that detects use of uninitialized memory. It does this by trapping every read and write to memory that was allocated dynamically (e.g. using kmalloc()). If a memory address is read that has not previously been written to, a message is printed to the kernel log."
Ingo Molnar credited the new patch with already finding 4 kernel bugs, and offered some more insights into how the patch works, and why it's useful, "it should also be made clear that not only does kmemcheck consume half of the RAM to do byte granular tracking of the other half of RAM, it's also slow, very slow, because almost every kernel-space instruction will generate a pagefault and then it will be single-stepped and it takes a debug fault as well. That's of course totally crazy, but that's also OK and it's what makes the feature so interesting and powerful."
"I wasn't planning on releasing v0.12 yet, and it was supposed to have some initial support for multiple devices. But, I have made a number of performance fixes and small bug fixes, and I wanted to get them out there before the (destabilizing) work on multiple-devices took over," explained Chris Mason regarding the 0.12 release of his new btrfs filesytem. Btrfs was first announced in June of 2007, as an alpha-quality filesystem offering checksumming of all files and metadata, extent based file storage, efficient packing of small files, dynamic inode allocation, writable snapshots, object level mirroring and striping, and fast offline filesystem checks, among other features. The project's website explains, "Linux has a wealth of filesystems to choose from, but we are facing a number of challenges with scaling to the large storage subsystems that are becoming common in today's data centers. Filesystems need to scale in their ability to address and manage large storage, and also in their ability to detect, repair and tolerate errors in the data stored on disk." Regarding the latest release, Chris offered:
"So, here's v0.12. It comes with a shiny new disk format (sorry), but the gain is dramatically better random writes to existing files. In testing here, the random write phase of tiobench went from 1MB/s to 30MB/s. The fix was to change the way back references for file extents were hashed."
It was recently pointed out that most of the x86 architecture patches had been merged into the mainline 2.6.25 kernel, except for the kgdb patches. Linus Torvalds replied, "I won't even consider pulling it unless it's offered as a separate tree, not mixed up with other things. At that point I can give a look." He continued:
"That said, I explained to Ingo why I'm not particularly interested in it. I don't think that 'developer-centric' debugging is really even remotely our problem, and that I'm personally a lot more interested in infrastructure that helps normal users give better bug-reports. And kgdb isn't even _remotely_ it.
"So I'd merge a patch that puts oops information (or the whole console printout) in the Intel management stuff in a heartbeat. That code is likely much grottier than any kgdb thing will ever be (Intel really screwed up the interface and made it some insane XML thing), but it's also fundamentally more important - if it means that normal users can give oops reports after they happened in X (or, these days, probably more commonly during suspend/resume) and the machine just died."
Joseph Myers announced the availability of GCC 4.2.3 saying, "GCC 4.2.3 is a bug-fix release, containing fixes for regressions in GCC 4.2.2 relative to previous GCC releases." He adds, "as always, a vast number of people contributed to this GCC release -- far too many to thank individually!"
GCC is the GNU Compiler Collection which includes C, C++, Objective-C, Fortran, Java, and Ada compilers. Download GCC 4.2.3 from your nearest gcc.gnu.org mirror.
"I re-ran some statistics the other day on our kernel development rate, and changed my formula after Andrew accused me of severely undercounting the rate of change," noted Greg KH during a discussion about the stability of the Linux kernel while undergoing significant changes. He continued, "turns out that as of 2.6.24-rc8 for the 2.6.24 kernel release we did: lines added per day, 4945; lines removed per day, 2006; lines modified per day, 1702". Greg added:
"And note, that is real stuff, not renames or file moves at all, git handles not reporting that. That's for the 99 days that it took to do 2.6.24-rc8 (I need to re-run the scripts now that 2.6.24 is out.) It's fricken scarily amazing that things are still working at all... Just something to make you all sleep well at night :)"