In a string of emails, Julian Elischer announced that Kernel Scheduled Entities - Milestone 3 (KSE-MIII) have been merged into the -current FreeBSD source tree. Julian notes that a few bugs still linger, but "at least we still seem to have basic functionality."

The KSE project is a major effort to allow for multi-threaded applications to scale and perform better, especially on SMP servers. The effort involves a considerable amount of re-working the various internal kernel data structures, and though not actually considered part of FreeBSD's "next generation" symmetric multiprocessing project (SMPng), each project greatly enhances the other.

This third milestone makes it possible for a multi-threaded program to run on a single CPU with full thread concurrency, during which any of the threads can voluntarily yield to the others for any of several reasons. The upcoming fourth milestone will allow a multi-threaded process to efficiently run multiple threads on separate CPUs.

Dave Hansen posted a patch to the lkml this morning, prefacing with the comment, "For those of you who heard the talk at OLS Friday morning, this patch won't be too much of a surprise. But for the rest of you..." He went on to describe an effort the patch would help in understanding how the big kernel lock (BKL) is used:

"The BKL's "magical" properties of allowing recursive holds on a single cpu and its release-on-sleep semantics make it really hard to replace with new locking schemes. Before we can really remove it, we must first characterize the places where it is used in these crazy ways."

His full email follows, further describing what he's found so far using the supplied patch. He also includes sample patches for the tty and ext3 code saying, "Don't take too much stock in these, they're just a demonstration and not nearly complete."

The FreeBSD core team elections are over, and the new core team decided. The nine member team is comprised of five new members (John Baldwin, Jun Kuriyama, Mark Murray, Murray Stokely and Wes Peters) and four returning members (Greg Lehey, Warner Losh, Robert Watson and Peter Wemm). Not returning (now alumni) are David Greenman and Doug Rabson.

A document on the FreeBSD website explains, "The FreeBSD core team constitutes the project's ``Board of Directors'', responsible for deciding the project's overall goals and direction as well as managing specific areas of the FreeBSD project landscape." The new team members take office Monday, July 8th, 2002.

Rik van Riel [earlier interview] recently announced the decision to move the ever helpful #kernelnewbies IRC channel to a new network. He explained:

"Due to a possible incompatibility between one of the OPN admins soliciting money on a regular basis and the policies of the network where kernelnewbies.org is hosted we have decided to move the #kernelnewbies IRC channel to another network."

You can now find #kernelnewbies on irc.oftc.net. Rik describes kernelnewbies as "a project meant to help people learn about operating system development by providing information and operating a mailing list and IRC channel where current and future developers can help each other."

After the recent hole found in OpenSSH [earlier story], the OpenBSD home page has been updated with a new slogan: "One remote hole in the default install, in nearly 6 years!" All in all, not a bad track record... (Previously the tagline had read "Five years without a remote hole in the default install!")

Check out the OpenBSD errata page to see the various security fixes applied over time, as well as a list of other known problems.

LWN has posted a report on the first day of the Ottawa Kernel Summit. They discussed a range of topics, from the AMD x86-64 port; to the absolution of modules; to the 2.5 VM (Linus says rmap will be in 2.5).

Jeremy adds: Also find LWN's excellent day two summary of the Kernel Summit. Among many other interesting conversations, Linus agreed to a feature freeze for the 2.5 kernel on October 31, 2002.

OpenSSH 3.4 was released today, resolving an input validation error that affects versions of sshd from 2.3.1 through 3.3. According to the 3.4 release announcement, the input validation error "can result in an integer overflow and privilege escalation." 3.4 follows the release of OpenSSH 3.3 by five days, and according to OpenBSD creator Theo de Raadt [earlier interview], "while dealing with this hole, Markus Niels and I found and fixed a wide variety of other issues. 3.4 contains changes which we think will matter significantly for security."

The 3.4 release was made earlier than planned to make the fix available prior to ISS disclosing the actual vulnerability. It is recommended you upgrade to this latest release.

Update: (06/26) An updated advisory has been added to the end of this story. Included are patches that can be applied to older versions of OpenSSH. An upgrade is still advised "because OpenSSH 3.4 adds checks for a class of potential bugs."

Kenneth Merry recently announced, "I'm planning on checking in the zero copy sockets code Tuesday evening, MDT." He has a web page set up with quite a bit of information for the curious, including a full changelog. The main zero copy patch was written by Drew Gallatin and is mentioned by several of these documents.

The "Zero Copy" patch removes the copying of buffers from the user process into the kernel when sending packets, and the copying of buffers from the kernel into the user process when recieving packets, offering a performance gain. Details on how this is accomplished can be found in the FAQ on the page linked above. The FAQ also explains that there is still another copy that happens, "The DMA or copy from the kernel into the NIC, or from the NIC into the kernel is not the copy that is being eliminated. In fact you can't eliminate that copy without taking packet processing out of the kernel altogether. (i.e. the kernel has to see the packet headers in order to determine what to do with the payload)"

Marc-Christian Petersen originated the WOLK project in March of 2002. WOLK is the Working Overloaded Linux Kernel, a large set of nearly 450 useful Linux kernel patches applied against the current stable 2.4 tree. The project has recently expanded to offer a second 'secure' patchset, this one against the older stable 2.2 tree.

In this interview, Marc-Christian Petersen tells the history behind WOLK and discusses many of the patches included.

Linux kernel 2.4 maintainer Marcelo Tosatti announced the first release candidate for 2.4.19 today (2.4.19-rc1) from Ottawa, Canada. In his own words, "Directly from OLS you're getting the first release candidate. Please test it extensively." Matthias Andree followed with a changelog summary.

Shortly after Marcelo's release, Stelian Pop sent a patch for a lingering typo in 2.4.19-rc1 that affects anyone compiling "when one does not have a floppy device..." The patch fixes a typo in the file init/do_mounts.c, changing the incorrect "CONFIG_BLOCK_DEV_RAM" to "CONFIG_BLK_DEV_RAM". Without this fix, the function change_floppy() is ifdef'd out, resulting in the error "undefined reference to `change_floppy'".

A recent conversation on the lkml about clusters [earlier story] continued on to look at the scalability of Linux. Linus Torvalds pointed out that current efforts were focused on current commodities:

"So because SMP hardware is cheap and efficient, all reasonable scalability work is done on SMP. And the fringe is just that - fringe. The numa/cluster fringe tends to try to use SMP approaches because they know they are a minority, and they want to try to leverage off the commodity. And it will continue to be this way for the forseeable future. People should just accept the fact."

Larry McVoy [earlier interview] argued that current scalibility efforts using multithreading were destined to cause serious problems in the future.

"The real point is that multi threading screws up your kernel. All the Linux hackers are going through the learning curve on threading and think I'm an alarmist or a nut. After Linux works on a 64 way box, I suspect that the majority of them will secretly admit that threading does screw up the kernel but at that point it's far too late."

In some recent performance tests between different Linux kernel versions, Luis Pedro was surprised to find that the older 2.2.20 kernel outperformed the newer 2.4.18 kernel. His email to the lkml was to find out what causes this performance downgrade.

Nathan Straz was the first to reply, stating flatly, "Benchmarks are evil. Sure they are useful at times, but for the most part they get misused." Daniel Phillips disagreed, explaining, "There's no sense denying evidence that 2.2 outperforms 2.4 under certain workloads. Instead we should just be more determined to root out all those problems and deal with them." He went on to argue that there's no inherent reason for 2.4 to be slower, "however, some practical issues, such as IO scheduling still remain and are being actively worked on."

If interested in doing some benchmarking of your own, you may wish to look at the Linux Benchmark Suite. The formentioned email exchange follows.

Jorg Braun recently released "NetBSD Live!", a CD that boots NetBSD 1.5.2/i386 directly from CD-ROM. Without any hard drive installation, you can now boot NetBSD 1.5.2, complete with your choice of desktops between KDE2, WindowMaker and tvm. Several useful applications are also installed on the CD, including all KDE 2.2.2 applications (such as KOffice), Gimp, Mozilla and AbiWord. Available for download are the CD image and some cover art. Find more details here. The full release announcement follows.

The question of NFS server performance amongst the many server types was raised recently on the FreeBSD hackers mailing list. Terry Lambert pointed out that all client implementations are tuned differently, explaining, "If you're asking about a server and not a client, then you would be better of asking about the particular client by name vs. each of the possible server choices."

Matt Simerson offered another opinion, citing tests he performed two years ago. At that time, he concluded that the FreeBSD NFS implementation "solidly outperformed every other NFS server", though acknowledging that "its NFS is missing locking support".

The question has been asked before, and the answer is always the same: It's a lot more complicated than it sounds.

Adi Zaimi recently asked on the lkml about the possibilty of allowing for live kernel upgrades, giving the ability to upgrade from one kernel to another without rebooting and interrupting services.

This time, however, Rob Landley responded in a series of very informative emails explaining just how complicated of a prospect it is to perform live kernel upgrades. In response to the question, he first says, "Thought about, yes. At length. That's why it hasn't been done. :)" He then goes on to point out many of the details that complicate such efforts. Rob points out two projects of interest to anyone willing to attempt live upgrades. One project is working on suspending a live kernel to restart it later. The other is working on a "two kernel monte", allowing one to boot from one kernel to another. Rob's full emails follow.