The posting describes how the build process was in pre-5.0/current times (e.g. 4.0), what things change in 5.0/current now, and what doesn't change.

[Tags: kmod, miniroot]

• Google Summer of Code is over for some time, but apparantly no final report has emerged so far (shame!). Still, a number of individual status reports came by on the official lists:
  • Zach Wegner's sysinst project
  • Adam Burkepile's subfile project (PDF)
  • Martin Schütte's syslogd project
  • Jeremy Morse's DVB project
  • Mark Dopiera's hurd translators project
  • Adam Hamsik's lvm project

  I know of at least one other project (uvc) that has completed but that I didn't see a report here - maybe I've missed it. Anyways, GSoC was another big success this year. Thanks, Google!

• Speaking of Adam Hamsik and Logical Volume Management (LVM), Adam has continued his work in that are, and he has written a device mapping library that interacts with his kernel driver. This allows to interact with his GSoC project without using any GPL code! See Adam's posting for more details.

• Force 10 Networks, producer of 10gbit switches that use an operating system based on NetBSD, have added a new feature as part of their FTOS operating system: VirtualView, which provides virtualization of Force 10 based equipment. From the xchange article: ``Force10 Networks Inc. this week introduced VirtualView software for benchmaking, troubleshooting and managing virtualized environments based on Force10 gear.''

  More information is available from the Force 10 Networks homepage, plus in articles by fibresystems.org, light reading, Zycko, and SmartBrief.

• Following the latest hype in portable computers, NetBSD has created a netbook page that intends to list models and the extent to which they are supported. Your contributions are most welcome here! (Contact me for sending updates and hardware :-)

• Zafer Aydogan has made RSS feeds available for CVS commits to single files - see his mail to netbsd-users for more details.

• New security advisory were released that I've missed in my last update:
  • NetBSD Security Advisory 2008-011: ICMPv6 MLD query
  • NetBSD Security Advisory 2008-012: Denial of service issues in racoon(8)

• A project that's been ongoing for quite some time is the move from "old-school" loadable kernel modules (LKMs) to new-style kernel modules. Important changes include the fact that modules can be either linked into the kernel at build time, or loaded into the kernel at runtime from the same file. Also, the bootloader was modified to load modules after the kernel, e.g. for a RAM-disk like the one that is used by the INSTALL kernel.

  In the same line, some parts are starting to be moved out of the GENERIC kernel, and installed as modules that can be loaded by the new framework then. The start is made by POSIX semaphores as a first step and proof-of concept, even if some details are still under hot debate, e.g. what the file system layout for modules is, and if the belong to the kernel and its build process, or to the userland.

• While talking about splitting the kernel into modules, Antti Kantee has continued his work to move parts of the kernel into userspace, in particular running file system code as userland in his RUMP, and puffs and (Re)FUSE works. The idea is to provide the interfaces that file systems need in the userland, and the result is that you can run code that used to run inside the kernel in userland now.

  Another subsystem running in the kernel that could be moved to userland by providing appropriate interfaces with the rest of the kernel is the network stack, and Antti has moved just that to the userland. See Antti's mail to tech-net@ for more information on this impressive work.

• NetBSD has shipped XFree in previous releases, and people who wanted to use X.org had to install it from pkgsrc. That's all fine, but to get a modern X, one had to compile things, as no precompiled binary packages are made available for many platforms. This is changing now, and NetBSD is getting X.org integrated via a reachover infrastructure which is also enabled for crosscompiling.

  The "user interface" for this is still in flux, but after some detour ("build.sh -V MKXORG=yes", without -x), "build.sh -x" now builds whatever X is considered the default for the platform. Some platforms already default to use X.org as X, and more will come, as changes that were made to NetBSD's copy of XFree are adopted to X.org.

  Platforms that use X.org by default now are macppc (see here and here), sparc sparc64, shark, amd64 and i386.

  As X.org is at Revision 7 now, it's installed in /usr/X11R7, which will lead to a lot of interesting effects. pkgsrc is already prepared for the new layout, but there are still many minor details that will need adjusting to the new directory. If you find one, post your patches to tech-x11.

• Besides the GNU C compiler, there's the BSD-licensed Portable C Compiler around for some time now. It doesn't offer the same support as its GNU cousin yet, but this may change now: The BSD Fund is currently doing a fund drive to get money to enhance PCC. The goal is to raise $12,000US to improve support for core compiler functionality as well as support for C99, gcc compatibility and the amd64 architecture. See the project page for further details.

• The NetBSD 5.0 release cycle has started! There's a netbsd-5 branch in CVS, daily binaries are available for testing, and some of the highlights of the upcoming release include file system journalling for FFS via WAPBL, and X.org.

  To help testing of NetBSD on Cobalt machines, Izumi Tsutsui has made a NetBSD 5.0_BETA based version of the Cobalt restore CD available. Enjoy!

• As the final point today, a word on NUMA support from Christoph Egger. Non-Uniform Memory Access is needed in massive parallel systems where some nodes have RAM more tightly associated than others, where the RAM is further away, resulting in different access times for different regions of memory. In order to support this, Christop Egger has made first steps.

  His example implementation uses information from ACPI, and shows some heavy dmesg-pr0n from a 16-core machine with four sockets. Yumm!

• Kernel Preemption: Andrew Doran has continued his work towards fine-grained locking, and he has proposed a patch to implement kernel preemption, i.e. that in a realtime environment, high-priority processes can interrupt system calls running inside the kernel.

  Handling the Floating Point Unit (FPU) was added later on -- the FPU needs special attention as saving and restoring is expensive, and doesn't need to be done in many cases. But if a program uses it, care must be taken to handle the case. The exact handling is explained by Christoph Egger.

  While there, Christoph also outlined the roadmap for getting realtime support in NetBSD - there are still a number of bits missing, but being able to preempt the kernel is a good first step!

• Fine-grained socket locking: In order to allow fine-grained locking (instead of blocking all other processes from entering the kernel, as is done in the "biglock" SMP approach), many kernel subsystems need to be changed. The socket system is the core part of interprocess communication, and Andrew Doran has changed it to use fine-grained locking now.

  In that context, the question of what code still runs with the biglock held, and Andrew gave an overview where more work is needed: some file systems (lfs, ext2fs, nfs), most of the drivers, protocols like TCP/IP, Veriexec, and some machine-dependent parts.

  Veriexec-Hacker Brett Lymn added details on the status of Veriexec with respect to its transition towards fine-grained locking.

• Kernel modules and ramdisk: A change in kernel modules was proposed some time ago, and Andrew Doran has used this scheme now to unify the way many ports handle the install media: There, the kernel loaded contains a ramdisk (miniroot) image inside the kernel, which is then used as root-filesystem for the kernel, containing the install tools.

  In order to split things and eventually use a stock GENERIC kernel for both running and installing, Andy has changed the x86 boot process to load the miniroot as a kernel module.

  When booting it may be useful to select one of several ramdisks: one for installing, and one for resuing the system, For this, the recently introduced boot.cfg file was extended to handle kernel modules in the boot menu.

  Izumi Tsutsui has made an ISO with all changes for testing available.

• Device File System (devfs): Another area of the kernel where a lot of work is currently being done by Matt Fleming is NetBSD's device driver infrastructure, esp. under aspects of dynamic attaching, detaching, and suspending (power management!). To talk to the various drivers, device nodes in the /dev directory are kept right now, but those are static and need to be updated when a new driver is added. Matt is working on a Device Filesystem (devfs) that dynamically created /dev from the list of devices inside the kernel. The fileysstem will also handle dynamic creation and deletion of nodes, and as an important case it will also keep permissions across reboots, if someone changes permissions manually.

  The work is at a very mature point right now and needs some testing - see Matt's mail to the tech-kern list for more information!

• Testing driver attachment: While talking about testing of device drivers, David Young has reminded driver developers to test individual drivers' detachment and re-attachment, suspension and resumption after changes. He has also posted a how-to for those tests, using drvctl(8). (The manpage needs some updating, sorry -- UTSL :-)