Joseph Myers announced the availability of GCC 4.2.3 saying, "GCC 4.2.3 is a bug-fix release, containing fixes for regressions in GCC 4.2.2 relative to previous GCC releases." He adds, "as always, a vast number of people contributed to this GCC release -- far too many to thank individually!"
"I re-ran some statistics the other day on our kernel development rate, and changed my formula after Andrew accused me of severely undercounting the rate of change," noted Greg KH during a discussion about the stability of the Linux kernel while undergoing significant changes. He continued, "turns out that as of 2.6.24-rc8 for the 2.6.24 kernel release we did: lines added per day, 4945; lines removed per day, 2006; lines modified per day, 1702". Greg added:
"And note, that is real stuff, not renames or file moves at all, git handles not reporting that. That's for the 99 days that it took to do 2.6.24-rc8 (I need to re-run the scripts now that 2.6.24 is out.) It's fricken scarily amazing that things are still working at all... Just something to make you all sleep well at night :)"
"The latest feature release GIT 1.5.4 is available at the usual places," began Git maintainer Junio Hamano. He continued, "it has been an unusually long cycle. 5 months since the last feature release 1.5.3 was really a bit too long. But I hope it was worth waiting for. Thanks everybody for working hard to improve it." He noted that there were 165 contributers resulting in 684 changed files, included 70,435 insertions and 28,984 deletions.
The Git distributed version control system was originally written by Linus Torvalds in April of 2005 to temporarily replace BitKeeper, which he had been using to manage kernel source code since February of 2002. Junio Hamano took over maintainership of Git a few months later, in July of 2005, and the tool has long since become quite popular outside of even Linux kernel development. Regarding the latest stable release, Junio highlighted some of the changes, including:
"Comes with much improved gitk, with i18n; comes with git-gui 0.9.2 with i18n; progress displays from many commands are a lot nicer to the eye; rename detection of diff family while detecting exact matches has been greatly optimized; 'git diff' sometimes did not quote paths with funny characters properly; various Perforce importer updates; 'git clean' has been rewritten in C; 'git push' learned --dry-run option to show what would happen if a push is run; 'cvs' is recognized as a synonym for 'git cvsserver', so that CVS users can be switched to git just by changing their login shell; 'git add -i' UI has been colorized; 'git commit' has been rewritten in C; 'git bisect' learned 'skip' action to mark untestable commits; 'git svn' wasted way too much disk to record revision mappings between svn and git, a new representation that is much more compact for this information has been introduced to correct this; in addition there are quite a few internal clean-ups."
Ingo Molnar summarized his pull request for changes to the x86 architecture bound for mainline inclusion in 2.6.25 noting, "it's not a small merge, it consists of 908 commits from 96 individual arch/x86 developers (!)". He continued, "a number of core files are changed as well: most notably percpu, debugging details, timers, the firewire remote debugging patch and ... the KGDB remote debugging stub in kernel/kgdb.c." He went on to detail the extent of the testing this tree has received, "in the past few weeks tens of thousands of random x86.git bzImages were successfully built and booted on a number of (commodity) 32-bit and
64-bit testsystems - and there has been a fair amount of test exposure on -mm as well." Regarding the remote kernel debugger, Ingo explained:
"We tested KGDB to be merge-worthy within the x86 architecture (the only supported architecture for now) and it's better to have kernel/kgdb.c than arch/x86/kernel/kgdb.c. The code is reasonably clean and the user-space exposure is small - the only real exposure is the decades-old remote GDB protocol. We are happy to fix up any further cleanliness comments that people might have - but we really wanted to start somewhere and get this thing moving. As an added bonus: finally a kernel debugger that can be read without puking too much ;-) [anyone remember KDB?]"
Avi Kivity summarized the kvm patches bound for the 2.6.25 kernel:
"Changes include performance and scalability improvements, completion of the portability work (though no new architectures are supported with this submission), support for new hardware features, using general userspace memory for kvm (which enables swapping guest memory as well as sharing memory among guests), as well as the usual cleanups and incremental fixes."
The Kernel-based Virtual Machine project, kvm, was started in mid-2006, and has been part of the Linux kernel since the 2.6.20 release in February of 2007. The recent changes can be browsed with gitweb.
"As you probably know there is a trend in enterprise computing towards networked storage. This is illustrated by the emergence during the past few years of standards like SRP (SCSI RDMA Protocol), iSCSI (Internet SCSI) and iSER (iSCSI Extensions for RDMA)," began Bart Van Assche, proposing that SCST be merged into the mainline kernel. He suggested that while similar to the STGT project which has been part of the mainline kernel since 2.6.20, "SCST is superior to STGT with respect to features, performance, maturity, stability, and number of existing target drivers. Unfortunately the SCST kernel code lives outside the kernel tree, which makes SCST harder to use than STGT."
SCSI subsystem maintainer, James Bottomley, was not convinced, explaining:
"The two target architectures perform essentially identical functions, so there's only really room for one in the kernel. Right at the moment, it's STGT. Problems in STGT come from the user<->kernel boundary which can be mitigated in a variety of ways. The fact that the figures are pretty much comparable on non IB networks shows this. I really need a whole lot more evidence than at worst a 20% performance difference on IB to pull one implementation out and replace it with another. Particularly as there's no real evidence that STGT can't be tweaked to recover the 20% even on IB."
Prefacing a series of 196 patches, Greg KH noted, "due to the low level nature of these patches, and because they touch so many different parts of the kernel, a number of the subsystem maintainers have asked me to get them in first to make merging other trees easier." Linus Torvalds agreed and quickly merged the patches into his tree. Greg summarized the many changes:
"They can be broken down into these major areas: Documentation updates (language translations and fixes, as well as kobject and kset documentation updates.); major kset/kobject/ktype rework and fixes; struct bus_type has been reworked to now handle the lifetime rules properly, as the kobject is properly dynamic; struct driver has also been reworked, and now the lifetime issues are resolved; the block subsystem has been converted to use struct device now, and not 'raw' kobjects; nozomi driver is added; lots of class_device conversions to use struct device instead."
Ingo Molnar posted a merge request for the latest git scheduler tree summarizing, "it contains various enhancements to the scheduler - find the full shortlog is below. 96 commits from 19 authors - scheduler developers have been busy again. :-/" He added, "the scheduling behavior of the kernel to normal users should not change over v2.6.24, but there are a good number of new features and enhancements under the hood." Ingo went on to list a number of these new features, including:
"Various instrumentation and debugging enhancements from Arjan van de Ven; Peter Zijlstra's RT time limit and RT throttling code for the RT scheduling class; Paul E. McKenney's preemptible RCU code; refcount based CPU-hotplug rework by Gautham R Shenoy; there's serious interest in running RT tasks on enterprise-class hardware, so Steven Rostedt and Gregory Haskins wrote a large number of enhancements to the RT scheduling class and load-balancer; Peter Zijlstra's high-resolution scheduler tick code; [...] and a good number of other, smaller enhancements."
"The release is out there (both git trees and as tarballs/patches), and for the next week many kernel developers will be at (or flying into/out of) LCA in Melbourne, so let's hope it's a good one," said Linus Torvalds, announcing the 2.6.24 Linux kernel. He noted, "nothing earth-shattering happened since -rc8". Source level changes can be viewed via the gitweb interface. A nice overview of all changes can be found at Kernel Newbies.
In a followup email, Linus added:
"Since I already had two kernel developers asking about the merge window and whether people (including me) traveling will impact it, the plan right now is to keep the impact pretty minimal. So yes, it will probably extend the window from the regular two weeks, but *hopefully* not by more than a few days."
"I'm happy to announce that I've implemented a Block I/O bandwidth controller," began Ryo Tsuruta, explaining that it was intended to be used in a cgroup or virtual machine environment, implemented as a device-mapper driver. He detailed a token-based implementation in which dm-band passes out to the various groups, "a group passes on I/O requests that its job issues to the underlying layer so long as it has tokens left, while requests are blocked if there aren't any tokens left in the group. One token is consumed each time the group passes on a request. Dm-band will refill groups with tokens once all of groups that have requests on a given physical device use up their tokens." Ryo explained:
"Dm-band is an I/O bandwidth controller implemented as a device-mapper driver. Several jobs using the same physical device have to share the bandwidth of the device. Dm-band gives bandwidth to each job according to its weight, which each job can set its own value to. At this time, a job is a group of processes with the same pid or pgrp or uid. There is also a plan to make it support cgroup. A job can also be a virtual machine such as KVM or Xen."
"The following patches have been in the -mm tree for a while, and I plan to push them to Linus when the 2.6.25 merge window opens," began Theodore Ts'o, offering the patches for review before they are merged. He explained that the patches introduce some of the final changes to the ext4 on-disk format, "ext4, shouldn't be deployed to production systems yet, although we do salute those who are willing to be guinea pigs and play with this code!" He continued:
"With this patch series, it is expected that [the] ext4 format should be settling down. We still have delayed allocation and online defrag which aren't quite ready to merge, but those shouldn't affect the on-disk format. I don't expect any other on-disk format changes to show up after this point, but I've been wrong before.... any such changes would have to have a Really Good Reason, though."
The final 2.6.24 Linux kernel is expected any day now, so the various subsystem maintainers have begun summarizing what changes are expected to be merged into the mainline kernel during the 2.6.25 merge window. Ingo Molnar spoke to changes for the x86 architecture, "there are 763 commits in x86.git so far, from more than 90 contributors, so it would be difficult to mention and credit every contribution in this mail." Along with a lengthy list of other changes, he included:
"Continued, intense arch/x86 unification and cleanup work by lots of people; FIFO ticket spinlocks for better spinlock scalability; 'regset' generalizations - the most important step towards utrace support (==next-gen ptrace); support for more than 255 CPUs [up to 4096 - in theory up to 65535]; almost complete 64-bit paravirt guest support; KGDB support on x86, finally!"
"Yes, I know ... another tree, just what everyone wants," quipped James Bottomley, announcing his new merge candidate (-mc) tree:
"This one has a specific purpose: It's my tree tracking everyone else's git and quilt trees so I get early warning if there are going to be any merge issues. However, it struck me it might be useful to anyone wishing to track what's going upstream more closely."
James noted that his new tree is available in git, and being automatically built each night. "As you can see from the reverts and the skips, we have trouble even now (and that's after I fixed up most of the failures in SCSI). ACPI and the x86 trees clash hideously, so I kicked out x86. Jens' block tree has two patches which clash with Bart's ide quilt. Greg actually has one patch in his tree that clashes with one of mine." He also noted, "this tree is currently very storage centric (i.e. I haven't included net trees or quilts because I didn't think they'd be likely to clash with my SCSI trees). However, if it could be more generally useful, I could add other trees and quilts to it."
"Slow servers, Skipping audio, Jerky video --everyone knows the symptoms of latency. But to know what's really going on in the system, what's causing the latency, and how to fix it... those are difficult questions without good answers right now," began Arjan van de Van, announcing version 0.1 of LatencyTop, "a tool for developers to visualize system latencies." He continued:
"LatencyTOP is a Linux tool for software developers (both kernel and userspace), aimed at identifying where system latency occurs, and what kind of operation/action is causing the latency to happen. By identifying this, developers can then change the code to avoid the worst latency hiccups.
"There are many types and causes of latency, and LatencyTOP focus on type that causes audio skipping and desktop stutters. Specifically, LatencyTOP focuses on the cases where the applications want to run and execute useful code, but there's some resource that's not currently available (and the kernel then blocks the process). This is done both on a system level and on a per process level, so that you can see what's happening to the system, and which process is suffering and/or causing the delays."
"'Const' has *never* been about the thing not being modified. Forget all that claptrap. C does not have such a notion," began Linus Torvalds, responding to a query about why kfree() takes a const pointer. He continued, "'const' is a pointer type issue, and is meant to make certain mis-uses more visible at compile time. It has *no* other meaning, and anybody who thinks it has is just setting himself up for problems." He offered two explanations, beginning with simple C semantics, "from a very obvious and very *real* caller perspective, 'free()' really doesn't change the thing the pointer points to. It does something totally different: it makes the *pointer* itself invalid." He then added his second reason, "anything that *can* take a const pointer should always do so. Why? Because we want the types to be as tight as possible, and normal code should need as few casts as possible." When it was pointed out that GCC 4.2 displays warnings when casting a const pointer to a non-const, Linus replied:
"Either don't use a broken compiler (casting a const pointer to a non-const is definitely not a bug), or cast to 'unsigned long' (if it still complains, now the compiler is not just stupid, it's broken). The whole point of memory management is that we know how pointers work, and understand that they have a *bit* representation, not just the C semantics."