So the blame was on the RPI-USB-to-serial cable, and I looked into making my own. I have a standard USB-to-serial adapter, but that one cannot be connected pin-by-pin to the RPI, as they use different voltage levels. While the "standard" serial UART protocol (RS-323) encodes 0s and 1s as 5V and -5V, the RPI uses TTL voltage between 0 and 3.3V - a simple 1:1 connection sounds like a bad idea. Converters like the MAX3232 are available: it connects the TX and RX lines and adjusts the voltage levels. To operate, power and ground are provided by the RPI. After frobbing this together (see picture 2) I found out that - surprise - the problem was not in the cable either, as my home-made cable also did not work for input, only for output.

After some detour (frobbing the image-build process to make me an image that not only has working DHCP and SSH, but that actually let's me log in), I got the crucial hint what was going on:

Having lines for sending and receiving data is nice, but what if one end's receiver is full and wants to signal that? This is called "flow control", and apparently with no lines for that, hardware flow control is not an option. In theory there is a software solution (sending XON/XOFF bytes), which requires equal settings on both sides. Apparently the RPI doesn't do that either, and with the default settings on my NetBSD host to wait for an XON before sending data, things didn't match up - no XON from the RPI, no data going to it. Plain and simple - if you know what's going on.

The fix was easy, and thanks go to Jean-Baka Domelevo Entfellner for the hint: disable flow control in my terminal program (kermit). So with a simple "set flow none", things work like a charm now. Voila, another hardware problem fixed in software! :)

How to look at serial interface parameters from NetBSD? Use the stty(1) command. Here's the default output for my USB-to-serial interface attached to ttyU0:

# stty -f /dev/ttyU0
ispeed 0 baud; ospeed 9600 baud;
lflags: echoe echoke echoctl
oflags: onocr onlret
cflags: cs8 -parenb

# stty -f /dev/ttyU0
speed 115200 baud;
lflags: -icanon -isig -iexten -echo echoe echoke echoctl
iflags: -icrnl -ixon -ixany ignbrk ignpar
oflags: -opost onocr onlret
cflags: cs8 -parenb clocal

For reference sake, here is my .kermitrc that I use:

# cat .kermrc
# Raspberry PI
set carrier-watch off
set line /dev/ttyU0
set flow none                   # disable hardware flow control
set speed 115200
#set speed 9600

#connect

This change does not only affect the Raspberry Pi, but also other machines that have a Synopsis USB like the OpenBlocks 600, as KIYOHARA Takashi lets us know.

[Tags: dmesg, raspberrypi, usb]

Juraj Sipos started his project originally on FreeBSD (and still continues that), but there's a variant based on NetBSD 5.1.2 now, too. The system comes up read-write, so you can make changes to it - as Juraj writes on the webpage: ``With this USB image I can now use NetBSD the same way as if it was installed on a hard drive.''

Oh, and for those impatient to try, the root password is "netbsd5".

[Tags: live-cd, mahesha, usb]

Please join in and help test the code, and send your feedback to the lists. If no serious issues come up, the code will be merged within a week.

See Matt's posting to tech-kern for more details, inclusing diffs and links for amd64 and i386 GENERIC (+usbmp) kernels.

Further information on the state of the code - what is and what is not converted yet - can be found in the TODO.usbmp file.

[Tags: smp, usb]

The files are available at http://ftp.netbsd.org/pub/NetBSD/arch/cobalt/restore-cd/5.1.2/. Citing from the announcement: ``The only changes from 5.1.1 version are CHANGES file and 5.1.2 binaries, so the following instructions are same as 5.1.1 ones:

restorecd-5.1.2-20120205.iso.gz is a gzipped RestoreCD ISO9660 image as prior releases.

restoreusb-5.1.2-20120205.img.gz is a new "RestoreUSB" image which has almost identical functions with RestoreCD but is intended to be burned into USB memory sticks for USB bootable PCs.

You can write the image using gzip(1) + dd(1) on Unix like OSes, or you can also use "Rawrite32" utility on MS Windows: http://www.NetBSD.org/~martin/rawrite32/index.html

To use the RestoreUSB for cobalt installation, write the image into >=512MB USB memory stick (or USB HDD etc.) and boot your PC from it, then all other procedures are same as RestoreCD. You no longer have to burn a coaster for every installation ;-)

See also "Restore CD Howto" for actual installation procedures: http://www.NetBSD.org/ports/cobalt/restorecd-howto.html: (though RestoreUSB is not mentioned yet) and see files in .tar.gz archive for more details.''

[Tags: cobalt, usb]

To get removable devices automatically mounted and a file browser pop up, add the following to /usr/pkg/etc/PolicyKit/PolicyKit.conf:

        <match action="org.freedesktop.hal.storage.mount-removable">
                <return result="yes" />
        </match>

A similar thing can be achieved for fixed disks, by adding the following data:

        <match action="org.freedesktop.hal.storage.mount-fixed">
                <return result="yes" />
        </match> 

With RUMP, PUFFS and the like, Antti is set on a mission to define interfaces between kernel components, and move those kernel components from inside the kernel to the userland. As a result, the (once) kernel-code is now running as "simple" userland processes that have all the benefits like memory protection and not taking down the whole system in case of an error. Examples that Antti has been bashing on started with moving file systems to the userland, which resulted in PUFFS and ReFUSE, an implementation of the Linux inferface of File systems in USErland (FUSE). Not stopping at file systems (which are traditionally used for accessing data on storage media, like e.g. for ntfs-3g), Antti went further and moved into getting network services like SSHFS going - in userland, based on FUSE.

Not stopping there, more recent works include:

• USB device driver support, to run NetBSD's USB stack in userland - and with it a bunch of USB device drivers as well, e.g. storage (umass) and wireless adapters.

• Running NFS in userspace, and with it the underlying TCP/IP stack? Well, we've had nfsd(8) for quite some time, but this time it's the same code running in userland as in kernel. The thread also answers questions on why we want to virtualize nfsd+networking with a process and why do we want to do that by reusing kernel code.

Sounds impossible? Ye fear not, it's been done! Arnaud 'stacktic' Ysmal already has ported RUMP to FreeBSD and Linux, the work is available via pkgsrc/misc/rump, and there is also Arnaud's page on Rump on non-NetBSD Operating Systems for more information.

Whew... lots of research and development going on in this area, and - getting back to the initial topic - we may well see an operating system in the future that moves from the monolithic to a microkernel approach, and it may or may not be called NetBSD. Fact is, that a lot of research is going on in that area, on NetBSD, here and now. Hats off, Antti!

[Tags: fuse, microkernel, nfs, ntfs-3f, puffs, refuse, rump, sshfs, tcp/ip, ukfs, usb]

• In order to re-initialize x86 machines' video/VGA state after suspend and resume, some BIOS functions can be used. This needs to be done in real mode(?), which is a bit hard to do from an operating system kernel that runs in protected mode. To help doing so, a x86 CPU emulator was added to NetBSD some time ago, to help run VGA bios for ACPI resume. Now Joerg has added a sysctl that does just this, assuming your kernel has the VGA_POST options -- set machdep.acpi_vbios_reset=2

• Inside the kernel, data sent/received through the network stack is stored in chains of mbufs. So far, the mbufs were also used to store socket options, i.e. data describing further how the sending/receiving is done. This was split out into a separate struct sockopt by Ian 'plunky' Hibbert now. For more information, see sockopt(9).

• Hans 'woodstock' Rosenfeld has added a new accalerated driver for SPX graphics boards found in some VAXstations, which replaces the old and broken lcspx driver. The work is based on work by Blaz Antonic.

• The simonb-wapbl branch was merged: ``Add Wasabi System's WAPBL (Write Ahead Physical Block Logging) journaling code. Originally written by Darrin B. Jewell while at Wasabi and updated to -current by Antti Kantee, Andy Doran, Greg Oster and Simon Burge.'' This makes NetBSD the first second (see update below) BSD operating system that has a working file system with journaling (not counting LFS, which again and again has issues). Mmm, no more fsck! :-) See my other posts for more on journaling / wapbl.

  Update: James Mansion wrote me to that NetBSD's not the first BSD to have journaling, and I think he's right: DragonflyBSD's HAMMER file system apparently offers similar functionality: ``HAMMER implement an instant-mount capability and will recover information on a cluster-by-cluster basis as it is being accessed.''

• Accept filters were ported from FreeBSD by Coyote Point Systems, and integrated into NetBSD by Thor Lancelot Simon. What are accept filters? According to the accept_filter(9) manpage, they ``allow an application to request that the kernel pre-process incoming connections.'' Pre-defined filters are available with accf_data(9) and accf_http(9). The latter makes sure that the application's accept(2) call only sees the connection if there's a valid HTTP header, moving parts of the parsing from userland (httpd) to the kernel.

• Work is underway for crossbuilds of modular X.org. This is done via src/external/mit/xorg, which needs xsrc/external/mit. The results will be installed in /usr/X11R7(!). (XXX Where can I find more about this?)

• Gregory McGarry is working to get the tree compiled with PCC instead of GCC. This is still ongoing.

• nvi was updated from version 1.79 to 1.81. The most important part of this update is that internationalization is now handled by default.

• Following a bigger masterplan, new 3rd party software packages are now imported into src/external/${license}, which will replace src/dist, src/crypto/dist and src/gnu/dist in the long run. Packages will be moved on upgrades only, existing packages are not being moved just for the sake of moving them.

• Adam Hamsik is working on getting Logical Volume Management (LVM) going in NetBSD. He has adapted Linux' "device mapper" kernel-interface as part of his Google Summer-of-Code project, and with the help of the (GPL'd) Linux tools, things are looking pretty good. More on this in a separate post. This work is currently happening on the haad-dm branch.

• In the context of his work on UDF, Reinoud has added routines for speeding up directory handling by using hash gables. Lookup of files was O(n*n) and is now O(1) even for file creation. See my other blog posting for details and impressive numbers.

• Perry Metzger is working to make binary builds identical. This is useful for binary diffs between releases/builds, e.g. when providing binary patches for updates and security fixes. Areas where this had an impact on are C++ programs and various bootloaders (which had a builder, build date, etc. in it so far).

• EHCI (USB) can now do high speed isochronous support. This was developed by Jeremy Morse as part of his Google Summer-of-Code "dvb" project this year, it is useful for fast transfer of data that comes in steady streams, e.g. from video cards.

• fsck_ffs(8) now has options -x and -X (just like dump) that create a file system snapshot via fss(4), and then operates on the snapshot. This allows "fsck_ffs -n" to work on a snapshot of a read/write mounted file system, and avoid errors related to file system activity. Can be made permanent for the nightly script by setting run_fsck_flags="-X" in /etc/daily.conf. This was brought to you by our Xen-hacker Manuel Bouyer. :-)

• wscons: Expansion for wstablet in NetBSD
  Student: Jason W. Beaudoin

  To support tables for the wscons console driver, a number of changes to the wscons API have been proposed in the student's status report. While there are a number of similarities with the wsmouse interface, there are also a number of differences that need to be worked around, e.g. absolute vs. relative coordinates. Many of the proposed changes are implemented, and we're looking forward to complete this project successfully within time.

  Status report, project page, project proposal

• subfiles: Subfile Support for NetBSD
  Student: Adam Burkepile

  Subfile allow to associate data with a regular file, just like regular allow to associate data with a directory. New internal data structures were defined to identify subfiles within the file system, and tools like newfs and dumpfs were adjusted. An API is being designed to access subfiles, and work to realize the assorted functions is being approached.

  Status report part 1 / 2, project page, project proposal

• hurdt: Hurd translators
  Student: Marek Dopiera

  Translators are programs which provide filesystems in user space functionality. This is provided via NetBSD's RUMP interface, and additional system calls and file system operations have been defined to activate the server processes when access to such a "translator" is made. Currently, the translators are only implemented in NetBSD's ext2 file system, as this allows testing of interaction with Hurd - Hurd's support for FFS seems non-working, unfortunately. The project's under busy development, and we're looking forward to see the final results.

  Status report, project page, project proposal

• lvm: Write and improve NetBSD LVM driver
  Student: Adam Hamsik

  This project implements the Linux LVM API (libdevmapper) on NetBSD, to allow using Linux' lvm2tools for logical volume management. The project's making excellent progress, there is an ISO image (see URL in status report) as well as a qemu image available for testing, and latest reports show that linear volumes can be configured and mounted already.

  Status report, project page, project proposal

• uvc: Add support for UVC devices (USB web-cams)
  Student: Patrick Mahoney

  A kernel driver was developed to read data from webcams using a custom API, work to implement the video2linux API is under way. Current challenges lie in NetBSD's USB stack, which lacks support for isochronous transfers, which is used by many (but not all) webcams. The project has made excellent progress so far, and we're positive that the project will be a success.

  Status report, project page, project proposal

• dvb: DVB drivers and kernel framework
  Student: Jeremy Morse

  This project implements a driver for Digital Video Broadcasting to supplement bktr(4)'s TV card support. So far, a driver and an API to transport data from the kernel to userland was implemented, matching LinuxTV. The project's making good progress, even due to conflicts with the academic schedule.

  Status report, project page, project proposal

• install-tool: Customizable install tool for NetBSD
  Student: Zachary Wegner

  NetBSD's current installer, sysinst, is being split into frontend and backend parts, with a configuration file building the interface between the two parts. Untangling the current mix of user interaction and install operation are ongoing, with challenges like request of install media (think floppy #42) and network configuration. Also, a parser for the configuration file was written, and work not started yet is the frontend creating the configuration file for the backend.

  Status report, project page, project proposal

• fs-utils: File system access utilities
  Student: Ysmal Arnaud

  This project is using NetBSD's RUMP and the ukfs library to access a file system image from userland. So far, makefs(1) can generate a file system image, and it can now be manipulated as well. Both a "file system console" as shell to operate on the image as well as separate tools for single operations have been designed. Many of the "normal" userland tools' functionality like ls(1), cp(1) and rm(1) were implemented. This project has made excellent progress so far, see the status report and project page.

  Status report, project page, project proposal

• cwrapper: pkgsrc: rewrite wrapper framework in C
  Student: Amitai Schlair

  The student was distracted by travel obligations for some time, but work has started in pkgsrc now. Current work includes a set of ATF tests to cover the usage of the existing pkgsrc wrapper framework and a design plan for the new wrapper implementation. The actual wrapper implementation remains to be written, and we're looking forward to see the it happen, including integration into pkgsrc plus benchmarks on the increase in speed.

  Status report, project page, project proposal

• atfify: Converting remaining regression tests to the Automatic Testing Framework
  Student: Lukasz Strzygowski

  The Automated Testing Framework was added to NetBSD as a result of last year's Summer of Code. This year, all the remaining regression tests from src/regress are being converted to ATF. Test suites for tools (awk, grep, make, ...) and libraries (libm, libpthread, ...) were converted so far. Tests for libc are currently being converted, and kernel tests are next. We're looking forward to get all of src/regress changed to ATF!

  Status report, project page, project proposal

• ext3: Implement Ext3 file system support
  Student: Rus-Rebreanu Alin-Florin

  This project intends to implement journaling in the file system by reusing Wasbi's wapbl code. Unfortunately, little has happend to reach this goal (to say the least), and the student got AWOL. Interested parties are welcome to try out ext2fuse (which also does ext3, despite the name) for now.

  Project page, project proposal

• packet-classes: Create an in-kernel API for "packet classes"
  Student: Anish Babu

  Nothing has happened in this project as well. I hear about communication problems, and it remains to see if things move forward here. :-(

  Project page, project proposal

• teredo: Implementation of RFC4380 (Teredo) in NetBSD
  Student: Arnaud Lacombe

  The project aims at creating a Teredo client, server and relay via a kernel pseudo device and assorted userland tools. The project is currently still in the analysis and design phase, which will give it little time for completion -- as for other students, this is due to a clash with the academic schedule of the student. We're holding up hopes that the set goals can be met at least partially.

  Project page, project proposal

• syslogd: Improve syslogd
  Student: Martin Schütte

  The project aims at implementing the upcoming IETF standards for syslog transport over TLS (instead of UDP), a more formal format of the messages themselv for easier automatic parsing, and signing of messages to assert authentication, integrity and correct sequencing of syslog messages. The first part of this is already working, the second part is development. We're looking forward to see this project succeed, as it will be of benefit not only to NetBSD.

  Project page, project proposal

Update: Fixed a typo (thanks tron!), got an update on the ext3 project, and added a link to the status report of the cwrapper project, which I've missed (sorry!).

[Tags: atf, dvb, ext3, google-soc, hurd, lvm, pkgsrc, rump, subfile, sysinst, syslog, teredo, ukfs, usb, uvc, wscons, wstablet]

Pierrick Brossin pointed me at the new NetBSD on a Stick page, which is ``a project devoted to installing the NetBSD operating system onto a USB based storage device.''

Besides general information on the project and why you would want to put NetBSD on a USB stick, the page has an install guide that describes how to get NetBSD on a memory stick.

[Tags: usb]