This document was written for Daemonnews and published in the May 2004 DaemonNews issue .



Report of pkgsrcCon 2004
Hubert Feyrer, May 2004

The Event

Over the last couple of months, the infrastructure of pkgsrc, the NetBSD (and others :-) packages system underwent quite some changes. To not only communicate via email and chat, a call was made to all people interested in pkgsrc development to join in for a pkgsrcCon, where things were presentations on details were given and discussed face to face.

The Place

The meeting happened at the weekend of May 1st (April 30th to May 2nd) at the Institute for Discrete Mathmatics and Geometry at Vienna University of Technology in Vienna, Austria's capital city. The place was choosen because many pkgsrc developers are located in Europe, and Vienna is fairly central to Europe and also has good medium and long distance infrastructure for travelers from abroad.

The People

More than thirty people registered and participated in the Con. Many of the were developers working on NetBSD and pkgsrc, but we also had interested NetBSD users and visitors from other open source projects as well who wanted to know more about pkgsrc. All in all, people came in from many different countries, among them Finland, Sweden, England, France, Germany, Poland, Austria, Slovakia, Hungary and the United States. The Conference was jointly planned by Johnny C. Lam and Thomas Klausner, who also did a lot of the preparations and talks. Great work, guys!

The Talks

So what was on the schedule? I'll try to give a rough summary for each of the talks, see www.pkgsrcCon.org.
pkgsrc-wip:
Starting the Con, Thomas Klausner gave an overview on pkgsrc-wip, the "Work In Progress" repository located on Source Forge that is intended to give non-NetBSD-developers a chance to work on packages with other people, getting comments and help and to finalize them so they can be moved into pkgsrc. For more information, check out the Source Force pkgsrc-wip project.

Things I Think Are Broken And That I'd Like To Fix:
Johnny Lam gave a list of things that he thought are not optimal in the pkgsrc infrastructure, and that need work. Items included here were breaking up bsd.pkg.mk into smaller parts that are used for each stage (fetch, extract, ...) and making LOCALBASE=/usr work.

buildlink3, Part 1: Philosophy and Methodology:
Making more developers understand the concepts behind buildlink was one of the main objectives of the pkgsrcCon, and Johnny Lam started by giving an overview of how it evolved. The original idea of chroot'ed builds proved too difficult, so by creating a small shadow-tree with only the headers and libraries of packages really intended to be used as dependencies is built. There were several steps to the goal of buildlink3. The system as it is now supports these goals as well as abstraction of compilers, and pkgviews. The former of these allows pkgsrc to build with non-gcc like SunWorks on Solaris or the MIPS-Pro compiler on Iris, the latter got it's own talks.

buildlink3, Part 2: Implementation:
After giving an overview of what buildlink3 is and does (and will do, eventually), there was an in-depth session that walked throu many of the wrapper scripts involved and invoked by the buildlink setup and compile process, discussion of the workarounds needed to make (and keep) libtool and pkg-config happy, integration of functionality provided by the operating system that doesn't need to be pulled in as package as well as important parts of the bsd.buildlink3.mk Makefile.

Using pkgsrc on Solaris, Linux and NetBSD:
This was an introduction to pkgsrc for users who didn't know what pkgsrc is. As there were many pkgsrc developers listening, the goal was not so much telling them what they already knew, but presenting it to them so they were in a position to start advocaging pkgsrc themselves. Topics covered were reasons and scenarios in which pkgsrc is useful, how to get pkgsrc going on non-NetBSD platforms using Linux as an example, and giving an example of how to use pkgsrc without root privileves - a step building up confidence for users who want to try pkgsrc for the first time.

Cross-building packages:
Krister Waldfridsson started the third pkgsrcCon day with a presentation of his work on cross-building packages. The basic is to have an emulator that fully emulates a StrongARM CPU, and that runs NetBSD/shark (or some other port of NetBSD based on the StrongARM CPU). As this is very slow, a number of speed optimisations are made e.g. by not running the NetBSD/shark compiler under emulation, but by running a crosscompiler built using NetBSD's cross-toolchain natively. With all these optimisations, the emulator running NetBSD/shark in userland on a 500MHz PC compiled pkgsrc packages about at the same speed as the PC compiled native packages. This very nice approach is still work-in-progress, but we can look forward for it to get to a state where it will be made public.

Introduction to Package Views:
Alistair Crooks gave the next presentation. First giving an overall run-down of the pkgsrc history, he then introduced the idea of "pkgviews" in pkgsrc. The concept is to not install packages directly into a target directory, but into some "depot" and then make links to the target directory. That way, two versions of the same package can be installed at the same time, allowing easier testing, up- and down(!)grading.

pkgviews:
After everyone was upto-date regarding the concept of pkgviews, Johnny Lam illustrated that pkgviews was one of the design goals of the buildlink3 framework, and what the interesting internals are. Starting by creating the symlink tree and issues involved, discussion went over to shared files and configuration files, design decisions that were possible and solutions that were chosen.

Roadmap For The Future:
Alistair Crooks rounded up the discussion by introducing various features of other packaging systems (RPM, FreeBSD, ...), then compared them to the ones in pkgsrc. Agreeing that number of packages is not everything ("size doesn't matter" :-), he went into further steps that we will take in pkgsrc in the near, mediate and far future. Items included were maintaining of the stable pkgsrc branches, completion of buildlink3 and pkgviews migration, more robust binary pkg handling as well as better updating, and various smaller projects including finding someone to eat all the choccolate Al got. :)

The Rest & The Future

Between the talks, many small discussion arose around many problems and ongoing changes in pkgsrc and NetBSD. Saturday afternoon was used for PGP key-signing to further extend the web of trust of NetBSD developers. All in all, the event was a very good chance to discuss many issues face to face, and the general concensus is to repeat it next year. So be sure to book your plane tickets early! :-)
(c) Copyright 20020110 Hubert Feyrer
$Id: pkgsrcCon2004.html,v 1.3 2004/05/06 23:07:45 feyrer Exp $