Linux

Linux: LibATA PATA Status

Submitted by Jeremy
on July 5, 2007 - 10:44am
Linux news

Alan Cox posted an updated LibATA PATA (IDE) status report on the lkml. Improved from a previous status report [story] he noted, "current kernels now support HPA (Host Protected Area) but default to honouring it. Probably a wrong default for PATA but we need to decide the right way to expose it nicely." He went on to note, "no PATA hotplug support yet. Need warmplug helpers for some chipsets (eg some intel ICH) to avoid risk of hangs."

Later in the report he listed around 40 chipsets describing LibATA's PATA support for each, ranging from "rock solid" for ATIIXP and "solid" for AMD, TRIFLEX, MARVELL, MPIIX, OLDPIIX, NETCELL, RZ1000, SERVERWORKS, SIL680, and VIA, to "still experimental" for NS87410 and "no idea" for IT8213. When asked about support for PowerPC drivers, Alan replied, "I'm not aware of anyone having done any PPC ports yet, although a couple of people have asked and said they would look at it. Currently our coverage is incomplete for some embedded and obscure platforms, of which the macintrash is probably the least 'obscure'."

AirCard 875U with Ubuntu Linux

Submitted by Jeremy
on July 3, 2007 - 8:16pm

Having arrived in San Francisco for two months, I needed to find a reliable source for getting connected to the Internet. There was a selection of unsecured WiFi connections available from my new apartment, but all proved to be unreliable at best.

Linux: Suspend2 Becomes TuxOnIce

Submitted by Jeremy
on July 3, 2007 - 10:59am
Linux news

The 'Suspend2' project [story] has been renamed to 'TuxOnIce' Nigel Cunningham announced on the lkml, "this is for a couple of reasons: In recent discussions on LKML, the point was made that the word 'Suspend' is confusing. It is used to refer to both suspending to disk and suspending to ram. Life will be simpler if we more clearly differentiate the two. The name Suspend2 came about a couple of years ago when we made the 2.0 release and started self-hosting. If we ever get to a 3.0 release, the name could become even more confusing! (And there are already problems with people confusing the name with swsusp and talking about uswsusp as version 3!)."

Some concern was expressed that this move indicated that the rewritten suspend infrastructure was not being targetted for core inclusion. Nigel explained that this was not the case, "as far as I know, the desire on Rafael's part and mine is still to get at least some of the functionality merged. The only reason this isn't happening yet is that we're both busy with other aspects of the work. Rafael is focussing on infrastructure issues, I'm focussing on minimising the diff and finishing off cleaning up and adding comments to functions." Rafael Wysocki agreed, "my first target is to introduce a framework allowing drivers to have different callbacks for hibernation and suspend. Unfortunately, that turned out to require quite a lot of work - and time - to do."

Linux: 2.6.22 Coming Soon

Submitted by Jeremy
on July 2, 2007 - 9:56pm
Linux news

Linux creator Linus Torvalds released the 2.6.22-rc7 kernel saying, "it's hopefully (almost certainly) the last -rc before the final 2.6.22 release, and we should be in pretty good shape. The flow of patches has really slowed down and the regression list has shrunk a lot." He briefly summarized the changes in this latest release candidate, "the patches are mostly trivial fixes, a few new device ID's, and the appended shortlog really does pretty much explain it," adding, "final testing always appreciated, of course".

The previous stable kernel, 2.6.21, was released a little over two months ago on April 25'th [story]. An overview of all the changes merged into the latest version of the kernel is maintained in the Kernel Newbies wiki. Included in the list of changes are the SLUB allocator which replaced the slab allocator, a new wireless stack, a new firewire stack [story], and support for the Blackfin architecture. Source level changes can be tracked via the gitweb interface to Linus' kernel tree.

Linux: Rewriting the Buffer Layer

Submitted by Jeremy
on June 24, 2007 - 12:54pm
Linux news

Posting a series of three patches, Nick Piggin [interview] announced that he was working on a rewrite of the buffer layer which he calls fsblock, "the name is fsblock because it basically ties the fs layer to the block layer." As to just what the buffer layer is, Nick explained, "the buffer layer is a layer between the pagecache and the block device for block based filesystems. It keeps a translation between logical offset and physical block number, as well as meta information such as locks, dirtyness, and IO status of each block. This information is tracked via the buffer_head structure." Before listing the improvements introduced by his rewrite, Nicked offered a justification for the effort:

"Why rewrite the buffer layer? Lots of people have had a desire to completely rip out the buffer layer, but we can't do that because it does actually serve a useful purpose. Why the bad rap? Because the code is old and crufty, and buffer_head is an awful name. It must be among the oldest code in the core fs/vm, and the main reason is because of the inertia of so many and such complex filesystems."

Linux: Kernel Documentation and Translations

Submitted by Jeremy
on June 24, 2007 - 5:43am
Linux news

Following a recent patch that translated Documentation/HOWTO into Japanese [story], a new patch offered a translation of the same document into Chinese. Li Yang noted, "currently Chinese involvement in Linux kernel is very low, especially [compared to China's large] population base. Language could be the main obstacle. Hopefully this document will help more Chinese to contribute to Linux kernel."

Linux creator Linus Torvalds noted that he was happy to see translations, "I think that the policies and processes parts of the documentation are things that make total sense to encourage translation of, because it's entirely possible that those are interesting and valid even to the people who aren't necessarily directly involved in the actual coding, and may well be relevant to managers etc who may not be _directly_ involed with the rest of the kernel developers." He then noted that he didn't see any reason to include the translations with the kernel proper, "that said, I don't think that merging the result into the standard kernel makes sense - like it or not, right now English ends up being required to be part of actually getting things into the 'standard' kernel, and as such, at _some_ point there has to be a connection point that switches over to English, and trying to make the translations be an in-kernel thing is thus kind of pointless." Rob Landley noted that he has started putting together a documentation site on which he will include these translations, "send me translations (preferably in HTML format), and I'll put 'em up. (I've already got the one that started this thread.)"

Linux: Introducing Bugs

Submitted by Jeremy
on June 20, 2007 - 3:15pm
Linux news

In another thread discussing the tracking of kernel regressions [story], Linux creator Linus Torvalds noted that the kernel is evolving so quickly it is inevitable that bugs will be introduced. He used a git query to determine that there are an average of over 65 patches being committed every single day, "that translates to five hundred commits a week, two _thousand_ commits per month, and 25 thousand commits per year. As a fairly constant stream. Will mistakes happen? Hell *yes*." He continued on to add, "and I'd argue that any flow that tries to 'guarantee' that mistakes don't happen is broken. It's a sure-fire way to just frustrate people, simply because it assumes a level of perfection in maintainers and developers that isn't possible." He then offered a number of calculations based on the number of lines of code added in the past 100 days, summarizing:

"Even by the most *stringent* reasonable rules, we add a new bug every four days. That's just something that people need to accept. The people who say 'we must never introduce a regression' aren't living on planet earth, they are living in some wonderful world of Blarney, where mistakes don't happen, developers are perfect, hardware is perfect, and maintainers always catch things."

Linux: Linus on the GPL, BSD, Tivo and the FSF

Submitted by Jeremy
on June 15, 2007 - 7:11am
Linux news

A lengthy debate that began with a suggestion to dual license the Linux kernel under the GPLv2 and the GPLv3 [story] continues on the Linux Kernel Mailing List. Throughout the ongoing thread Linux creator Linus Torvalds has spoken out on the GPLv2, the upcoming GPLv3, the BSD license, Tivo, the Free Software Foundation, and much more. During the discussion, he was asked we he chose the GPLv2 over the BSD license when he's obviously not a big fan of the FSF. Linus explained:

"Because I think the GPLv2 is a great license. And I don't like the FSF's radical world-view, but I am able to separate the license (the GPLv2) from the author and source of the license (rms and the FSF). Why do people always confuse the two? The GPLv2 stands on its own. The fact that I disagree with the FSF on how to act has _zero_ relevance for my choice of license.

"[...] But for a project I actually care about, I would never choose the BSD license. The license doesn't encode my fundamental beliefs of 'fairness'. I think the BSD license encourages a 'everybody for himself' mentality, and doesn't encourage people to work together, and to merge."

Linux: Btrfs, File Data and Metadata Checksums

Submitted by Jeremy
on June 14, 2007 - 8:33am
Linux news

Chris Mason announced an early alpha release of his new Btrfs filesystem, "after the last FS summit, I started working on a new filesystem that maintains checksums of all file data and metadata." He listed the following features as "mostly implemented": "extent based file storage (2^64 max file size), space efficient packing of small files, space efficient indexed directories, dynamic inode allocation, writable snapshots, subvolumes (separate internal filesystem roots), checksums on data and metadata (multiple algorithms available), very fast offline filesystem check". He listed the following features as yet to be implemented: "object level mirroring and striping, strong integration with device mapper for multiple device support, online filesystem check, efficient incremental backup and FS mirroring". Regarding the current state of the project, Chris said:

"The current status is a very early alpha state, and the kernel code weighs in at a sparsely commented 10,547 lines. I'm releasing now in hopes of finding people interested in testing, benchmarking, documenting, and contributing to the code. I've gotten this far pretty quickly, and plan on continuing to knock off the features as fast as I can. Hopefully I'll manage a release every few weeks or so. The disk format will probably change in some major way every couple of releases."

Linux: Dual-Licensing the Kernel

Submitted by Jeremy
on June 12, 2007 - 3:19am
Linux news

"I was impressed in the sense that it was a hell of a lot better than the disaster that were the earlier drafts," Linus Torvalds explained in reply to a comment suggesting that he was impressed with the final draft of the GPLv3. He went on to add, "I still think GPLv2 is simply the better license." The discussion began with a suggestion that the Linux kernel be dual-licensed GPLv2 and GPLv3. Linus noted, "I consider dual-licensing unlikely (and technically quite hard), but at least _possible_ in theory. I have yet to see any actual *reasons* for licensing under the GPLv3, though. All I've heard are shrill voices about 'tivoization' (which I expressly think is ok) and panicked worries about Novell-MS (which seems way overblown, and quite frankly, the argument seems to not so much be about the Novell deal, as about an excuse to push the GPLv3)." In a followup email, Linus added:

"Btw, if Sun really _is_ going to release OpenSolaris under GPLv3, that _may_ be a good reason. I don't think the GPLv3 is as good a license as v2, but on the other hand, I'm pragmatic, and if we can avoid having two kernels with two different licenses and the friction that causes, I at least see the _reason_ for GPLv3. As it is, I don't really see a reason at all."

Linux: Translating Kernel Documentation

Submitted by Jeremy
on June 11, 2007 - 3:36am
Linux news

The translation of a some kernel documentation into Japanese led to a discussion as to whether or not it was appropriate to include translated documentation with the kernel source code. One concern that was expressed was that as the number of included translations grows, so would the size of the kernel. Another concern was the liklihood that as time passes the various translations might become out of date. Jesper Juhl suggested one workaround, "since the common language of most kernel contributors is english I personally feel that we should stick to just that one language in the tree and then perhaps keep translations on a website somewhere. So the authoritative docs stay in the tree, in english, so that as many contributors as possible can read and update them."

Greg KH noted that there were a number of files in the kernel that change infrequently and that he would like to see included, "I really do want to see a translated copy of the HOWTO, stable-api-nonsense.txt, and possibly a few other files in the main kernel tree (SubmittingPatches, CodingStyle, and SubmittingDrivers might all be good canidates for this.) These files change relatively infrequently (the HOWTO file has had only 7 changes in 1 and 1/2 years, and they were very minor ones) and should be easy for the translators to keep up with."

Linux: Compacting Memory

Submitted by Jeremy
on May 31, 2007 - 11:24am
Linux news

Mel Gorman offered a first release of a patchset that compacts memory, "this is a prototype for compacting memory to reduce external fragmentation so that free memory exists as fewer, but larger contiguous blocks. Rather than being a full defragmentation solution, this focuses exclusively on pages that are movable via the page migration mechanism." He notes that the patchset is currently incomplete, and at this time memory is only compacted manually, not automatically, "this version of the patchset is mainly concerned with getting the compaction mechanism correct." Mel goes on to describe how it works:

"A single compaction run involves two scanners operating within a zone - a migration and a free scanner. The migration scanner starts at the beginning of a zone and finds all movable pages within one pageblock_nr_pages-sized area and isolates them on a migratepages list. The free scanner begins at the end of the zone and searches on a per-area basis for enough free pages to migrate all the pages on the migratepages list. As each area is respectively migrated or exhausted of free pages, the scanners are advanced one area. A compaction run completes within a zone when the two scanners meet."

Linux: Rethinking Suspend and Resume

Submitted by Jeremy
on May 26, 2007 - 2:10am
Linux news

What started as the review of a bug report grew into an interesting debate as Linus Torvalds slammed the current suspend and resume [story] design in the Linux Kernel, "why the HELL cannot you realize that kernel threads are different? The right thing to do is AND HAS ALWAYS BEEN, to stop and start user threads only around the whole thing. Don't touch those kernel threads. Stop freezing them." Later in the discussion, Linus noted that he had no interest in Suspend to Disk (STD), and was only interested in a working Suspend to Ram (STR) implementation. He noted that complexity introduced by STD was infecting the STR logic, and that the two should be completely separated, "what irritates me is that STR really shouldn't have _had_ that bug at all. The only reason STR had the same bug as STD was exactly the fact that the two features are too closely inter-twined in the kernel. That irritates me hugely. We had a bug we should never had had! We had a bug because people are sharing code that shouldn't be shared! We had a bug because of code that makes no sense in the first place!" Linus noted that he doesn't use laptops much, but still likes STR on his desktop, "STR means they are quiet and don't waste energy when I don't use them, but they're instantly available when I care." He then went on to point to design flaws in the freezer:

"I actually don't think that processes should be frozen really at all. I agree that filesystems have to be frozen (and I think that checkpointing of the filesystem or block device is 'too clever'), but I just don't think that has anything to do with freezing processes. So I'd actually much prefer to freeze at the VFS (and socket layers, etc), and make sure that anybody who tries to write or do something else that we cannot do until resuming, will just be blocked (or perhaps just buffered)!"

Linux: Announcing the 2.6.22-rc3 Kernel

Submitted by Jeremy
on May 26, 2007 - 1:28am
Linux news

In a humorous announcement for the latest release candidate of the upcoming 2.6.22 Linux kernel, Linus Torvalds noted that there were updates to the ARM, SH and Blackfin architectures. He also noted fixes to USB suspend, infiband, and the network stack, as well as updates to ATA, DVB and MMC, and network drivers. Noting that a three-day weekend was starting in the US he said, "so what's a pasty white nerd to do? You can't go out on the beach, because the goodlooking people will laugh at you, and kick sand in your face. I'm not bitter." Linus continued:

"But now you _can_ do something: you can download the latest -rc kernel, and smile smugly to yourself, knowing that you are running the latest and greatest on your machine. And suddenly it doesn't even matter that summer is coming, because you can just sit in the basement, and close the blinds, and bask in the warm light from your LCD, rather than the harsh glare of the daystar.."

Further information about what's new and changed in the upcoming 2.6.22 kernel can be found in the KernelNewbies wiki. The latest -rc can be downloaded from the Linux Kernel Archives [story], and the source changes can be browsed online using the gitweb interface.

Linux: The Case Against Crash Dumps

Submitted by Jeremy
on May 25, 2007 - 2:43pm
Linux news

In a recent lkml thread the concept of dumping an image of the kernel's memory to swap when the kernel hits a bug was discussed. Linus Torvalds pointed out that such a feature wasn't useful to an operating system like Linux that can ran on such a diverse assortment of computers, "yes, in a controlled environment, dumping the whole memory image to disk may be the right thing to do. BUT: in a controlled environment, you'll never get the kind of usage that Linux gets. Why do you think Linux (and Windows, for that matter) took away a lot of the market from traditional UNIX?" He went on to explain that there are systems where swap is not larger than the size of the core so collecting a crash dump would not be possible, that Linux instead tries to acknowledge bugs without crashing, and quite often the bug is actually in the drivers, "writing to disk when the biggest problem is a driver to begin with is INSANE." Comparing Linux to Solaris he added, "so the fact is, Solaris is crap, and to a large degree Solaris is crap exactly _because_ it assumes that it runs in a 'controlled environment'."

Alan Cox went on to point out that there are also privacy issues, "there is an additional factor - dumps contain data which variously is - copyright third parties, protected by privacy laws, just personally private, security sensitive (eg browser history) and so on. The only reasons you can get dumps back in the hands of vendors is because there are strong formal agreements controlling where they go and what is done with them." He went on to note that dump utilities are also not user friendly, "diskdump (and even more so netdump) are useful in the hands of a developer crashing their own box just like kgdb, but not in the the normal and rational end user response of 'its broken, hit reset'". Linus heartily agread, and suggested that anyone willing to use kernel dumps would be better off debugging through a firewire connection, " if you've ever picked through a kernel dump after-the-fact, I just bet you could have done equally well with firewire, and it would have had _zero_ impact on your kernel image. Now, contrast that with kdump, and ask yourself: which one do you think is worth concentrating effort on?"