This document was published in the June 2000 Daemonnews issue, it was slightly updated since then.


Ext2fs rev 1 changes in NetBSD
Hubert Feyrer, Manuel Bouyer; May 2000

Manuel Bouyer has made changes to the NetBSD ext2fs to include changes for revision 1 of the Linux Extended2 filesystem. Here's an introduction on what ext2fs is, an overview of the first revision's changes, and a short history of NetBSD's GPL-free ext2fs implementation.

Why so many filesystems?

Running on a broad range of hardware platforms not only means being able to talk to the hardware. Often it's necessary to access programs and files installed on machines prior to the transition to another operating system, or even after. While changing to (or even playing with) another operating system, copying all data over to the new filesystem format is often not an option. And even if so, some method of accessing data is required.

Given that systems may have their own filesystem format to store data in their "native" format, accessibility via an appropriate filesystem is required to get to the data. It is the filesystem's responsibility to understand the layout of the data on the disk, and to make it accessible for reading and writing.

Besides native filesystems like FFS and LFS, NetBSD comes with a wide range of file systems which ensure it can read and/or write disks containing data in the following filesystems:

The filesystem used natively in NetBSD is the Berkeley FFS. The name was changed from Unix File System (UFS) to avoid the trademarked term "Unix". See [1] for more information on the Berkeley FFS.

There were some changes to the ext2fs code used in NetBSD, we'll talk about that here. After a general overview of ext2fs, we'll outline the changes made in ext2fs revision 1.

What is ext2fs?

The Second Extended Filesystem (ext2fs) is used as the Linux operating system's native filesystem format today. Ext2fs is based on the Minix filesystem, and several improvements were made to make it fit today's data storage requirements, using semantics and concepts found then (and today) in the traditional Berkeley based Fast Filesystem (FFS).

Ext2fs uses an on-disk structure similar to the FFS one: the superblock and inode have roughly the same members (some of them only differ by size, e.g uid and gid are 32 bits in FFS and 16 in ext2fs), ext2fs directory entries are identical to the 4.2BSD FFS one, the only difference being that ext2fs doesn't include the null character at the end of string where FFS does. 4.4BSD FFS includes the file type in directory entries; NetBSD supports both. The main difference between ext2fs and FFS is the block allocation policy: ext2fs doesn't have fragments and tries to always allocate blocks continously where FFS has a upper bound on the number of contigous blocks allocated for a file.

NetBSD already supports several FFS-derived file systems (FFS, LFS and MFS), with some functionality (inode manipulation, file lookup, ...) handled by shared code. The idea was to handle ext2fs as another FFS-derived file system. Most of the code was derived from FFS; it was just a matter of changing the structure name, and the name of structure members (and of course the name of the function). The only part that was written from scratch is the block allocation policy.

Please refer to [2] for more details on ext2fs.

Changes in ext2fs rev. 1

Ext2fs rev 1 has bitfields in the superblock indicating which features the ext2fs has. They are categorized in three classes: 'compat' are features which can be ignored by kernels which don't support it; 'rocompat' are features which can be ignored by kernels which don't support them if the filesystem is mounted read-only; and 'incompat' are features which prevent the filesystem from being mounted at all if the kernel doesn't support one of them.

As of linux-2.2.14 the only 'compat' feature defined so far is 'prealloc', which affects the way blocks are pre-allocated on write; No support for this optimisation is present in NetBSD so far.

Three 'rocompat' features are defined:

Linux-2.2.14 supports sparse_super and largefile; NetBSD only supports 'sparse_super' (lacking acces to 64bit Linux systems, it's not known if linux-2.2.14 completely supports largefile, and mke2fs 1.18 doesn't support it either, at least it's not documented in the man page).

Two 'incompat' features are defined:

As of linux-2.2.14/e2fsprogs 1.18 only 'filetype' is supported; NetBSD supports it as well.

History of a non-GPL'd ext2fs implementation

After reading the ext2fs specs[3], the NetBSD ext2fs started with a 'cp -r ffs ext2fs' and then (after renaming all functions/defines) adapting it to ext2fs: no fragments, different block allocation policy, minor changes in inodes and directory formats (ext2fs is really close to FFS), byte swapping on big-endian systems, deleting the 'file type in directory entries' code. That code was added back now, three years later, in Rev. 1 of ext2fs.

Note that the allocation policy doesn't have to be strictly identical to the Linux one; only the filesystem layout matters, and the specs were enough for this. By working strictly according to the ext2fs specs and not looking at/using the GPL'ed code, it was possible to get a completely GPL-free ext2fs implementation which fits the NetBSD requirement to keep GPL'ed code out of the kernel.

References

[1]
M. McKusick, K. Bostic, M. Karels, J. Quarterman: The Design and Implementation of the 4.4BSD Operating System; Addison-Wesley Publishing Company, Reading, MA. April 1996, ISBN 0-201-54979-4.
[2]
R. Card, T. Ts'o, S. Tweedie: Design and Implementation of the Second Extended Filesystem; http://web.mit.edu/tytso/www/linux/ext2intro.html.
[3]
A collection of ext2fs related documents can be found at ftp://sunsite.unc.edu/pub/Linux/system/filesystems/ext2/.

Updates

This text was written for DaemonNews.
$Id: ext2fs-rev1.html,v 1.9 2006/02/20 15:19:51 feyrer Exp feh39068 $