• NetBSD's "let's move kernel parts to the userland" RUMP project is still under heavy development, and in order to make testing of compatibility after kernel changes easier, a new command "rumptest" was added to build.sh: ``Basically you say:

  	    ./build.sh ${yourargs} tools ; ./build.sh ${yourargs} rumptest
  	

  Where yourargs are what have you, e.g. '-U -u -o -O /objs'.

  The latter builds only the rump kernel libs and uses some ld+awk magic to figure out if things go right or not. This is to avoid having to install headers and build libs (which is too slow since a full build is too slow). The magic is not a substitute for a full build, but it is n+1 times faster and works probably 99.9% of the time.

  The scheme uses a number of predefined component sets (e.g. tmpfs+vfs+rumpkern) to test linkage. They are currently listed in build.sh. This area probably needs some work in the future. It would be nice to autogenerate the combinations somehow.

  If things go well, you get something like this:

          ===> Rump build&link tests successful
          ===> build.sh ended:   Wed Nov 18 20:10:59 EET 2009 

  '' See Antti's Antti's mail to tech-kern: on how to tell if things didn't go so well, and what to do in that case.

• According to Wikipedia, ``Non-Uniform Memory Access or Non-Uniform Memory Architecture (NUMA) is a computer memory design used in multiprocessors, where the memory access time depends on the memory location relative to a processor. Under NUMA, a processor can access its own local memory faster than non-local memory, that is, memory local to another processor or memory shared between processors.''

  Supporting NUMA in a contemporary (i.e.: Intel centric) SMP-enabled operating system requires following a bunch of standards, two of which are parsing of two tables, the System Resource Affinity Table (SRAT) and the System Locality Information Table (SLIT). Both tables are accessible via the Advanced Configuration and Power Interface (ACPI), and according to the German-language Wikipedia, the SRAT is used to assign local memory to local threads to boost their performance, and the SLIT defines the "distance" of the nodes among themselves, which is used to determine the "nearest" memory if local memory is not enough.

  Now, Christop Egger has posted patches to add an ACPI SLIT parser and an ACPI SRAT parser. See the two postings for dmesg pr0n from his tests on an 8-node system.

• Staying with ACPI and Christoph Egger, he found that even though the ACPI spec defines an ACPI device for fans, BIOS vendors and OEMs do their own thing. To accommodate things like the fan sensor found in the ACPI Thermal Zone in his HP Pavillion DV9700 laptop he has proposed a driver to extend the acpitz(4) driver with fan information. That way, envstat(8) can be used to display the ran's RPMs:

  [acpitz0]
    Processor Thermal Zone:     56.000   95.000                       degC
                       fan:       2840                                 RPM 

• Staying with driver games, iMil writes me that there's documentation on getting DRI, AIGLX, Composite and Compiz going with NetBSD 5.0 available in the O(ther)NetBSD Wiki now.

  The documentation covers how to enable the Direct Rendering Manager (DRI), setting up and configuring Modular X.org, assuring that everything's in place, and how to get Compitz going. Mmm, wobbly windows at last! :-)

• While we're talking funky desktop stuff: Marc Balmer has submitted a patch to get touchpanel support for ums(4). ums(4) is for USB mice, and in contrast to mice, touch panels need to deal with absolute numbers, not relative numbers.

• Back to the guts of the kernel, another patch suggested by Christop Egger was for adding x2apic. What is x2apic? X2APIC is ``an Intel-only feature but can also be found in virtual environments with support for CPU apic id's > 0xff.

  I.e. Xen 4.0 (not yet released) supports 128 CPUs in HVM guests with the CPUs enumerated with even apic id's. That means you need x2apic for the 128th CPU :) ''

• While speaking of Xen: Xen 4.0 is coming soon, and there's a call to help testing it on NetBSD!

  Install Mercurial, check out latest Xen sources, apply a bunch of patches, build and install. Examples of commands are given, in addition to changes required for /boot.cfg etc.

  Report your findings to port-xen!

• Last one for today: Michal Gladecki, Editor-in-Chief of BSD Magazine writes: ``We are happy to announce that BSD Magazine is transforming into a free monthly online publication. The online version of BSD Magazine will stay in the same quality and form. It will look like the BSD magazine one is familiar and comfortable with. Please sign up to our newsletter at www.bsdmag.org and get every issue straight to your inbox. Also, you can now download any of the previous issues from our website. The first online issue -- 2/2010 -- is coming out in February. Please spread the word about BSD Magazine. '' Click!

Unrelated, I've been playing with git a bit over the past few days, and wile I have a number of questions building up (which will be subject to tech-repository or so), what I can say today is that the speed of "git pull" with NetBSD's git repository and my 1MBit DSL line reminds me a lot of the times when I used SUP with my 56k modem - it took forever, too. :-(

[Tags: acpi, acpitz, aiglx, Articles, bsdmac, compiz, dmesg, dri, numa, rump, slit, touchpanel, ums, x2apic, xen]

• Web-based firewall log reporting and analysis tool Webfwlog 0.94 released: ``Webfwlog is a flexible web-based firewall log analyzer and reporting tool. It supports standard system logs for linux, FreeBSD, OpenBSD, NetBSD, Solaris, Irix, OS X, etc. as well as Windows XP. Supported log file formats are netfilter, ipfilter, ipfw, ipchains and Windows XP. [...]

  With Webfwlog you can design reports to use on your logged data in whatever configuration you desire. Included are example reports as a starting point. You can sort a report with a single click, "drill-down" on the reports all the way to the packet level, and save your reports for later use. You can also create a link directly to any saved report.''

  See the webfwlog homepage for more information.

• Git copies of cvs modules available - spz@ writes that ``htdocs, othersrc, pkgsrc, src and xsrc are now available as (bare) git repository copies at http://ftp.netbsd.org/pub/NetBSD/misc/repositories/git. These repositories are currently updated every 30 minutes.

  They are not authoritative and are not meant to replace cvs, but if you want to track your development that is not yet fit for committing into cvs head or sending in as a PR, and want to use git for that, they may serve as a convenient starting point or reference.''

• Support for writing file systems on sparse disk images - File system hacker Antti Kantee writes: ``Yesterday I wanted to write to a file system which was on a sparse disk image. Normally I would have just used a rump mount, but since the file system was not at offset = 0 in the file, this failed. vnd wasn't helpful either: it allowed me to mount the image and then I got weird errors when writing.

  I just finished adding disklabel support to the various rump_fs utilities. Due to them using the option parsing code of the real mount_fs utilities, I decided to signal the label number with a magic parameter at the end of the device (yes, this really simplifies things currently).

  Let's assume you're looking at wd0.img. Here's how it works:

  === SNIP ===
  golem> disklabel wd0.img
  [...]
  16 partitions:
  #        size    offset     fstype [fsize bsize cpg/sgs]
   a:    719712        63     4.2BSD   1024  8192     0  # (Cyl.      0*-    714*)
   b:     66657    719775       swap                     # (Cyl.    714*-    780*)
   c:    786369        63     unused      0     0        # (Cyl.      0*-    780*)
   d:    786432         0     unused      0     0        # (Cyl.      0 -    780*)
  
  golem> rump_ffs wd0.img%PART:a% /puffs
  rump_ffs: "wd0.img" is a non-resolved or relative path.
  rump_ffs: using "/usr/home/pooka/wd0.img" instead.
  golem> df /puffs
  Filesystem                      1K-blocks       Used      Avail %Cap Mounted on
  /usr/home/pooka/wd0.img%PART:a%     338471     256211      65337  79% /puffs
  === SNIP ===

  So you specify the partition number as %PART:n%, where is n is obviously the partition number. Other than that, things work as usual. No vnconfig etc. necessary. ''

• New ACPI driver: acpismbus(4) - call for testers - Paul Goyette writes that ``At the request of some folks off-list, I have created a driver for ACPI's SMBus Control Method Interface. Basically, this is an acpi-based wrapper to provide device-independant access to i2c/SMBus controllers. (See spec at http://smbus.org/specs/smbus_cmi10.pdf for further info.)

  To see if your system can use this driver, check the output of acpidump for the existence of a Device(SMB0). If you have an SMB device, I'd appreciate it if you could help test this driver!''

  Continue with Paul's mail to learn on how to test the driver, and what limitations there are currently.

An alternative approach was suggested by Jeremy Reed, which hints at the powerd(8) script hooks that can be set to act on corresponting ACPI Thermal Zone (acpitz) events. See Jeremy Reed's blog posting for a few more ideas, plus a hint that this may become documented even better in the future.

While there: At his website, Jeremy has a number of open questions around NetBSD, and I guess answering them would be good for mankind. Any takers?

[Tags: acpi, cpu, fan]

Here are the talks on NetBSD and/or from NetBSD developers:

• Yvan Vanhullebus: IPSec tools: past, present and future (PDF, mp3)
• Martin Schuette: Improved NetBSD Syslogd (PDF, mp3)
• Michael Dexter: Zen and the Art of Multiplicity Maintenance: An applied survey of BSD-licensed multiplicity strategies from chroot to mult (PDF, mp3)
• Hauke Fath: Managing BSD desktop clients - Fencing in the herd (PDF, mp3)
• Joerg Sonnenberger: Sleeping beauty - NetBSD on Modern Laptops (PDF, mp3)
• Antti Kantee: Converting kernel file systems to services (PDF, mp3)

• In order to re-initialize x86 machines' video/VGA state after suspend and resume, some BIOS functions can be used. This needs to be done in real mode(?), which is a bit hard to do from an operating system kernel that runs in protected mode. To help doing so, a x86 CPU emulator was added to NetBSD some time ago, to help run VGA bios for ACPI resume. Now Joerg has added a sysctl that does just this, assuming your kernel has the VGA_POST options -- set machdep.acpi_vbios_reset=2

• Inside the kernel, data sent/received through the network stack is stored in chains of mbufs. So far, the mbufs were also used to store socket options, i.e. data describing further how the sending/receiving is done. This was split out into a separate struct sockopt by Ian 'plunky' Hibbert now. For more information, see sockopt(9).

• Hans 'woodstock' Rosenfeld has added a new accalerated driver for SPX graphics boards found in some VAXstations, which replaces the old and broken lcspx driver. The work is based on work by Blaz Antonic.

• The simonb-wapbl branch was merged: ``Add Wasabi System's WAPBL (Write Ahead Physical Block Logging) journaling code. Originally written by Darrin B. Jewell while at Wasabi and updated to -current by Antti Kantee, Andy Doran, Greg Oster and Simon Burge.'' This makes NetBSD the first second (see update below) BSD operating system that has a working file system with journaling (not counting LFS, which again and again has issues). Mmm, no more fsck! :-) See my other posts for more on journaling / wapbl.

  Update: James Mansion wrote me to that NetBSD's not the first BSD to have journaling, and I think he's right: DragonflyBSD's HAMMER file system apparently offers similar functionality: ``HAMMER implement an instant-mount capability and will recover information on a cluster-by-cluster basis as it is being accessed.''

• Accept filters were ported from FreeBSD by Coyote Point Systems, and integrated into NetBSD by Thor Lancelot Simon. What are accept filters? According to the accept_filter(9) manpage, they ``allow an application to request that the kernel pre-process incoming connections.'' Pre-defined filters are available with accf_data(9) and accf_http(9). The latter makes sure that the application's accept(2) call only sees the connection if there's a valid HTTP header, moving parts of the parsing from userland (httpd) to the kernel.

• Work is underway for crossbuilds of modular X.org. This is done via src/external/mit/xorg, which needs xsrc/external/mit. The results will be installed in /usr/X11R7(!). (XXX Where can I find more about this?)

• Gregory McGarry is working to get the tree compiled with PCC instead of GCC. This is still ongoing.

• nvi was updated from version 1.79 to 1.81. The most important part of this update is that internationalization is now handled by default.

• Following a bigger masterplan, new 3rd party software packages are now imported into src/external/${license}, which will replace src/dist, src/crypto/dist and src/gnu/dist in the long run. Packages will be moved on upgrades only, existing packages are not being moved just for the sake of moving them.

• Adam Hamsik is working on getting Logical Volume Management (LVM) going in NetBSD. He has adapted Linux' "device mapper" kernel-interface as part of his Google Summer-of-Code project, and with the help of the (GPL'd) Linux tools, things are looking pretty good. More on this in a separate post. This work is currently happening on the haad-dm branch.

• In the context of his work on UDF, Reinoud has added routines for speeding up directory handling by using hash gables. Lookup of files was O(n*n) and is now O(1) even for file creation. See my other blog posting for details and impressive numbers.

• Perry Metzger is working to make binary builds identical. This is useful for binary diffs between releases/builds, e.g. when providing binary patches for updates and security fixes. Areas where this had an impact on are C++ programs and various bootloaders (which had a builder, build date, etc. in it so far).

• EHCI (USB) can now do high speed isochronous support. This was developed by Jeremy Morse as part of his Google Summer-of-Code "dvb" project this year, it is useful for fast transfer of data that comes in steady streams, e.g. from video cards.

• fsck_ffs(8) now has options -x and -X (just like dump) that create a file system snapshot via fss(4), and then operates on the snapshot. This allows "fsck_ffs -n" to work on a snapshot of a read/write mounted file system, and avoid errors related to file system activity. Can be made permanent for the nightly script by setting run_fsck_flags="-X" in /etc/daily.conf. This was brought to you by our Xen-hacker Manuel Bouyer. :-)

• Christoph Badura: Gaols - Implementing Jails Under the kauth Frameworki (paper)
• Yuji IMAI, Takahiro KUROSAWA, Koichi SUZUKI, Eiichi MURAMOTO, Katsuomi HAMAJIMA, Hajimu UMEMOTO, and Nobuo KAWAGUTI: BSD implementations of XCAST6 (paper)
• Antti Kantee: Send and Receive of File System Protocols: Userspace Approach With puffs (paper)
• Kristaps Džonsons: Logical Resource Isolation in the NetBSD Kernel (paper)
• Alistair Crooks: A Portable iSCSI Initiator (paper)
• Jörg Sonnenberger, Jared D. McNeill: Sleeping Beauty---NetBSD on Modern Laptops(slides, paper)

Catching up on NetBSD's source-changes mailing list from Feb 21st 2008 to Mar 1st 2008, here's what people not following things closely may find interesting:

• Chris Gilbert made process on the chris-arm-intr-rework branch, which aims at reworking the ARM interrupt code to provide a shared set of interrupt routines that can be used across all ARM based machines.

• The PowerPC routines for bcopy, memcmp, memcpy and memmove were updated to use tuned versions that were written by IBM and released under a 3-clause BSD license as part of the perflib project, see http://sourceforge.net/projects/ppcperflib/.

• Keiichi Shima has imported the Mobile IPv6 code developed by the SHISA project. The work can be found on the "keiichi-mipv6" branch.

• Matt fleming has started work on a device file system that supports dynamic device nodes, which is of relevance when loading kernel modules for device drivers, so you don't have to run /dev/MAKEDEV manually. The work is on the "mjf-devfs2" branch.

• Nick Hudson is currently working on syncing NetBSD's WiFi (net80211) code with FreeBSD, so that these two operating systems (and possibly others) can share a common codebase. The work currently affects only the kernel, but this may extend to the userland eventually.

• The glob(3) wildcard extension routines got a new options GLOB_NO_DOTDIRS added, which hides . and .. even if the pattern would otherwise include them.

• Juan Romero Pardines has ported the mfi(4) driver from OpenBSD. The driver supports disk controllers with PowerPC IOSs such as the LSI SAS1078 and the Dell PERC6.

• The boot-only ISOs built for releases included several kernels (with ACPI, without ACPI) for some time now. Stephen Borrill has added a menu to make kernel selection easier. Mmm, user friendliness!

• The machine-independent versions of strcspn(3), strpbrk(3) and strpspn(3) were improved from O(mn) to O(n+m) based on ideas from DragonFlyBSD.

• Power management hooks for suspend, shutdown and resume were added to many device drivers, to support the new power management framework by Jared McNeill.

• A whole lot of changes were pulled up from NetBSD's development branch into the release branches for NetBSD 3.x and 4.x.

• Andre Pettersson sent mail about a web design contest for www.NetBSD.se. Besides eternal fame, the winner will get $150US. Check the mail for some requirements and where to send submissions!

• Mike Volokhov pointed me at CUWiN, the the Champaign-Urbana Community Wireless Network, which is ``a world-renowned coalition of wireless developers and community volunteers committed to providing low-cost, do-it-yourself, community-controlled alternatives to contemporary broadband models.''

  Of course they do use NetBSD, and the package is available in precompiled form for various Soekris machines, plus source (of course).

• James Hartley made a Cobalt Restore CD that's based on NetBSD 4.0_BETA2, and thanks to Andreas Fuchs the CDs are now available as torrents or ISOs plus a mirror. Of you're new to the CD, be sure not to miss out on documentation.

• The latest podcast at pofacs.de, the german-language podcast for "alternative computersystems", has an introduction of NetBSD by Stefan Schumacher, among other things.

• Jared McNeill keeps on whacking the ACPI code into shape, and he pointed me at an update to his status report. Testers are still every welcome, please see the previous entry for more information!

People interested to help out with the project can follow the instructions and report back to Jared.

For further support, Jared would appreciate some hardware for extending his work. Right now, an amd64-based laptop (e.g. IBM/Lenovo T60 or X60) would be needed for non-i386/64bit development. If you have some spare amd64 hardware or spare cash to support The NetBSD Foundation to fund the corresponding hardware, talk to Jared, too!

[Tags: acpi]