Usually I cross-compile g4u from Mac OS X, but for the fun of it I did it on NetBSD (7.0-stable branch, amd64 architecture in VMware Fusion) this time. After waiting forever on the CVS checkout, I found that empty directories were not removed - that's what you get if you have -P in your ~/.cvsrc file.

I already had the hint that the "g4u-build" script needed a change to have "G4U_BUILD_KERNEL=true".

From there, things went almost smooth: building indicated a few files that popped up "variable may be used uninitialized" errors, and which -- thanks to -Werror -- bombed out the build. Fixing was easy, and I have no idea why that built for me on the release. I have sent a patch with the required changes to the g4u-help mailing list. (After fixing that I apparently got unsubscribed from my own support mailing list - thank you very much, Sourceforge ;)).

After those little hassles, the build worked fine, and gave me the floppy disk and ISO images that I expected:

>       ls -l `pwd`/g4u*fs
>       -rw-r--r--  2 feyrer  staff  1474560 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u1.fs
>       -rw-r--r--  2 feyrer  staff  1474560 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u2.fs
>       -rw-r--r--  2 feyrer  staff  1474560 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u3.fs
>       -rw-r--r--  2 feyrer  staff  1474560 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u4.fs
>       ls -l `pwd`/g4u.iso
>       -rw-r--r--  2 feyrer  staff  6567936 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u.iso
>       ls -l `pwd`/g4u-kernel.gz
>       -rw-r?r--  1 feyrer  staff  6035680 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u-kernel.gz 

About g4u: g4u ("ghosting for unix") is a NetBSD-based bootfloppy/CD-ROM that allows easy cloning of PC harddisks to deploy a common setup on a number of PCs using FTP. The floppy/CD offers two functions. The first is to upload the compressed image of a local harddisk to a FTP server, the other is to restore that image via FTP, uncompress it and write it back to disk. Network configuration is fetched via DHCP. As the harddisk is processed as an image, any filesystem and operating system can be deployed using g4u. Easy cloning of local disks as well as partitions is also supported.

The past: When I started g4u, I had the task to install a number of lab machines with a dual-boot of Windows NT and NetBSD. The hype was about Microsoft's "Zero Administration Kit" (ZAK) then, but that did barely work for the Windows part - file transfers were slow, depended on the clients' hardware a lot (requiring fiddling with MS DOS network driver disks), and on the ZAK server the files for installing happened do disappear for no good reason every now and then. Not working well, and leaving out NetBSD (and everything elase), I created g4u. This gave me the (relative) pain of getting things working once, but with the option to easily add network drivers as they appeared in NetBSD (and oh they did!), plus allowed me to install any operating system.

The present: We've used g4u successfully in our labs then, booting from CDROM. I also got many donations from public and private instituations plus comanies from many sectors, indicating that g4u does make a difference.

In the mean time, the world has changed, and CDROMs aren't used that much any more. Network boot and USB sticks are today's devices of choice, cloning of a full disk without knowing its structure has both advantages but also disadvantages, and g4u's user interface is still command-line based with not much space for automation. For storage, FTP servers are nice and fast, but alternatives like SSH/SFTP, NFS, iSCSI and SMB for remote storage plus local storage (back to fun with filesystems, anyone? avoiding this was why g4u was created in the first place!) should be considered these days. Further aspects include integrity (checksums), confidentiality (encryption). This leaves a number of open points to address either by future releases, or by other products.

The future: At this point, my time budget for g4u is very limited. I welcome people to contribute to g4u - g4u is Open Source for a reason. Feel free to get back to me for any changes that you want to contribute!

The changes: Major changes in g4u 2.6 include:

• Make this build with NetBSD-current sources as of 2017-04-17 (shortly before netbsd-8 release branch), binaries were cross-compiled from Mac OS X 10.10
• Many new drivers, bugfixes and improvements from NetBSD-current (see beta1 and beta2 announcements)
• Go back to keeping the disk image inside the kernel as ramdisk, do not load it as separate module. Less error prone, and allows to boot the g4u (NetBSD) kernel from a single file e.g. via PXE (Testing and documentation updates welcome!)
• Actually DO provide the g4u (NetBSD) kernel with the embedded g4u disk image from now on, as separate file, g4u-kernel.gz
• In addition to MD5, add SHA512 checksums

Enjoy!

[Tags: g4u, Releases]

g4u ("ghosting for unix") is a NetBSD-based bootfloppy/CD-ROM that allows easy cloning of PC harddisks to deploy a common setup on a number of PCs using FTP. The floppy/CD offers two functions. The first is to upload the compressed image of a local harddisk to a FTP server, the other is to restore that image via FTP, uncompress it and write it back to disk. Network configuration is fetched via DHCP. As the harddisk is processed as an image, any filesystem and operating system can be deployed using g4u. Easy cloning of local disks as well as partitions is also supported.

For more information, see http://www.feyrer.de/g4u/.

Changes in 2.6beta2 include:

• Make this build with NetBSD-current sources as of 2017-04-17, binaries were cross-compiled from Mac OS X 10.10
• Go back to keeping the disk image inside the kernel as ramdisk, do not load it as separate module. Less error prone, and allows to boot the g4u (NetBSD) kernel from a single file e.g. via PXE (Testing and documentation updates welcome!)
• Actually DO provide the g4u (NetBSD) kernel with the embedded g4u disk image from now on, as separate file, g4u-kernel.gz
• Put all object files into one object directory. This may need more cleanup in the future. Feedback from people building g4u welcome!
• Disable verbose device messages on Microchannel (MCA) machines
• New drivers:
  • Toshiba Dynabook hotkeys
  • Intel S1200,C2000 (non-pch) SMBus storage controller
  • Non-Volatile Memory Express (NVMe) storage controllers and devices
  • Intel Wireless WiFi Link 7xxx PCI network
  • Intel 8259x 10 gigabit PCI network
  • Realtek 8188CE/8192CE 802.11b/g/n PCI network
  • VMware VMXNET3
  • Marvell Libertas PCMCIA network
  • ASIX AX88178a/AX88179 based USB network adapters
  • Realtek RTS5209/RTS5229 Card Reader
• Happy 18th Birthday, g4u!

• The g4u 2.6beta2 floppy images (zipped/ uncompressed floppy one, floppy two, floppy three, and floppy four)
• The g4u 2.6beta2 ISO CD image (zipped/uncompressed)
• The g4u/NetBSD kernel for PXE boot (gzipped)
• The g4u 2.6beta2 source code
• Some md5 checksums:

  MD5 (g4u-2.6beta2-1.fs) = 8214c1ded55e78aed2f34d3e318dc75d
  MD5 (g4u-2.6beta2-2.fs) = 8b55f3fe6a4806c9a65a3d176463e1bc
  MD5 (g4u-2.6beta2-3.fs) = 0f9629353ddd470d17c5b4fd4ec7fd8e
  MD5 (g4u-2.6beta2-4.fs) = c25a955e5d876d64943fd03f957ed4c9
  MD5 (g4u-2.6beta2.iso) = acabb0fb73d4a2f9062fb070b4ddd825
  MD5 (g4u-2.6beta2.tgz) = f8387d5bf146f98dda040dfcc1c53e44
  MD5 (g4u-2.6beta2.fs.zip) = 8ef324c71ae7b000708e69019e55ef4e
  MD5 (g4u-2.6beta2.iso.zip) = 1f4bd37784405350ee3e8e6c75a6b6b6
  

I'd like to push out version 2.6 within the next few weeks. Please test and let me know if there are any showstoppers!

Full list of news in g4u 2.6beta1:

• Make this build with NetBSD-current sources as of 2013-10-20
• Move back from a ramdisk that's loaded from a separate file back to a ramdisk that's part of the kernel image. This allows easier netbooting for those people who want it - added back by popular demand :-)
• Added more kernel buffer space, to hold all kernel messages for dmesg, even on machines with large ACPI tables (Hello VMware Fusion!)
• New drivers:
  • LSILogic 9x9 and 53c1030 (Fusion-MPT) PCI SCSI
  • LSI Logic Fusion-MPT II PCI SCSI
  • Atheros AR9k (802.11a/g/n) PCI Wireless
  • Marvell PCI Libertas Wireless
  • Atheros AR9k (802.11a/g/n) PC-Card Wireless
  • Broadcom BCM43xx PC-Card Wireless
  • Atheros AR9002U USB Wireless
  • Ralink Technology RT2500USB 802.11a/b/g USB Wireless
  • Ralink Technology RT(2[78]|30)00 802.11a/b/g/n USB Wireless
  • Realtek RTL8187/RTL8187B 802.11b/g USB Wireless
  • Realtek RTL8188CU/RTL8192CU 802.11b/g/n USB Wireless
  • Intel Atom E6xx PCI-LPC

• The g4u 2.6beta1 ISO CD image (zipped/uncompressed)
• The g4u 2.6beta1 source
• The g4u 2.6beta1 floppy images (zipped/ uncompressed floppy one, floppy two and floppy three)
• Some md5 checksums:

  MD5 (g4u-2.6beta1-1.fs) = 264d5b27f208f1053777774d0c2b87a7
  MD5 (g4u-2.6beta1-2.fs) = 533b37fbeeb2d537928de61b25eb2ade
  MD5 (g4u-2.6beta1-3.fs) = 5abe1327326fcdac18bf4d68566f3e9b
  MD5 (g4u-2.6beta1.iso) = 37596efb42c014eb049e0027db059bd9
  MD5 (g4u-2.6beta1.tgz) = acd5907a7fb2cc8fbd4645a1b9db5272
  MD5 (g4u-2.6beta1.fs.zip) = 9b8da62fac57dcd38a4e3376189a087d
  MD5 (g4u-2.6beta1.iso.zip) = 44cb8d01407b17f9d9839740b5a0d2d9
  

Looking forward for your opinions!''

[Tags: g4u]

``First, my problem was I had just replaced my motherboard on my custom build PC. Once I got Windows 7 64-bit loaded and everything working, I sat up to clone my system drive. The drive is a 500GB Seagate Barracude, SATA 2 drive. Cloning locally to an identical drive.

When booting into g4u, my transfer speeds were extremely slow. Normally, my 500 Gb clones take only about 90 minutes. But this was still working after over 6 hours. The g4u transfer speed was reporting only 1.5 Mb/sec.

I shut things down, and went into my system BIOS. I noticed that the SATA mode was set to "IDE Mode" for my drives. I changed this to "AHCI Mode" and continued to boot into g4u. This worked to fix the transfer speeds, and my clone finished normally. Getting 83 Mb/sec.

Once the drive was finished, I attempted to boot into Windows, but it would not boot. I had to change my BIOS back to "IDE Mode", then Windows behaved normally.

Upon researching this, I am now learning that you should enable AHCI Mode BEFORE installing Windows for it to work. Apparently, if Windows is not installed while using AHCI Mode, it disables the drivers for AHCI on the system drive. So if you later enable AHCI in your BIOS as I did, Windows will not have the driver loaded. I saw there is a fix on the Microsoft web site, but I haven't attempted to try it yet.

If someone else runs into a similar problem, hopefully this will help you.''

FWIW, g4u-2.5beta1 is based on NetBSD-current from January 2012, so checking your BIOS may help anyone seeing bad disk performance out there.
(Emphasizes in the text added by me)

[Tags: g4u, performance]

Remember that this is a test release, so your feedback is wanted - either to me in person, or to the g4u-help mailing list. Thanks!

Here's a full list of changes:

• New commands "uploadmbr" and "slurpmbr" to backup and restore the master boot record, which includes the partition table. Required to restore a partition to an empty disk.
• New command "copymbr" to copy the MBR from one disk to another, similar to "copydisk"
• New command "wipedisk" to write the disk full with 0-bytes once from start (sector 0) to end (last sector)
• Enable command line history/editing by forcing /bin/sh to be built without -DSMALL (ugly hack... there be lots of dragons!)
• When setting up a fresh compile tree, g4u patches are now applied automagically without aborting the build
• Error detection was now enabled in the dd(1) command, which is the core of g4u (surprise!). With that, disks with broken/unreadable sectors should now be copied, skipping the unreadable sectors and copying the rest. This affects a number of programs: copydisk, copypart, uploaddisk, uploadpart. BEWARE: I wasn't able to actually test this as I do not have a disk with bad sectors here. Please report back your experiences!!!
• Make this build with NetBSD-current sources as of 2012-01-12
• New drivers added to the kernel:
  • RDC PMX-1000 IDE controllers
  • Intel SCH IDE controllers
  • TOSHIBA PICCOLO controllers
  • Attansic/Atheros L1C/L2C Ethernet
  • Broadcom BCM43xx wireless
  • Agere/LSI ET1310/ET1301 Gigabit Ethernet
  • RDC R6040 10/100 Ethernet
  • USB LCDs and USB-VGA adaptors, e.g.:
  • DisplayLink DL-1x0/1x5
  • Option N.V. Wireless USB WAN modems
  • Microsoft RNDIS specifications USB ethernet
  • Atheros AR9001U USB Wifi
  • Intersil PrismGT USB Wifi
  • Virtio PCI, memory balloon, disk & network devices
  • ... and many more that slipped past QA
• ... and any driver updates, optimizations and bug fixes and other enhancements from NetBSD-current

• ISO
• compressed ISO
• First floppy
• 2nd floppy
• 3rd floppy
• all 3 floppies
• sources

Details include setting up a TFTP server for loading the bootloader and getting DHCP sending out the right files.

[Tags: g4u, pxe]

To go on from there, I found that my NetBSD VM's only disk was too small to do anything useful. Options for enlarging that came to mind:

1. NFS - see 'panic' above, no go.
2. Adding another (virtual) disk - easily doable, but I felt like not adding one
3. Extending the existing disk - adventure time!

1. Enlarge the last file system on disk
2. Fix the partition table to add another partition for the new space

For those in a similar situation, here are the steps to use the newly gained space on an enlarged (virtual) disk:

1. Prepare: save the old output of "dmesg" (/var/run/dmesg.boot is OK)

2. Enlarge - VMware Fusion wants a shutdown for that, you cannot suspend the machine

3. After booting, run a diff on the saved "dmesg" output, to learn what the old and new size of the disk is, in sectors. My diff looks like this, note the size change in sectors:

   -wd0: 10240 MB, 22192 cyl, 15 head, 63 sec, 512 bytes/sect x 20971520 sectors
   +wd0: 20480 MB, 44384 cyl, 15 head, 63 sec, 512 bytes/sect x 41943040 sectors 

4. Backup the existing/old disklabel, just in case: disklabel wd0 >disklabel.BAK

5. Edit the disklabel: disklabel -e wd0

6. In the editor, adjust the disk size in sectors from 20971520 to 41943040:

   total sectors: 41943040 

7. Partition 'd' is the full disk on i386/amd64, it starts at sector 0 and is 41943040 sectors big

   #        size    offset     fstype [fsize bsize cpg/sgs]
    d:  41943040         0     unused      0     0        # (Cyl.      0 -  44384*)
   

8. Partition 'c' is the NetBSD part of the disk. As this VM only has NetBSD, all the usable space is used. Note that "usable" space excludes the first 63 sectors of the disk (mbr etc.), i.e. it is 41943040 - 63 = 41942977 sectors:

   #        size    offset     fstype [fsize bsize cpg/sgs]
    c:  41942977        63     unused      0     0        # (Cyl.      0*-  44384*)
   

9. After this everything is in sync with the new disk again, and the remaining/new space can be used for new partition 'e'. As the new space starts where the disk used to end, its offset is the old size, 20971520 sectors.

   The size of the new partition expands from the offset sector 20971520 to the end of the disk at sector 41943040, i.e. the partition size is:

   % expr 41943040 - 20971520
   20971520
   

   In total, this gives for the new partition:

   #        size    offset     fstype [fsize bsize cpg/sgs]
    e:  20971520  20971520     4.2BSD   2048 16384     0  # (Cyl.  22192*-  44384*)
   

10. Last, create file system, mount and populate:

    # newfs /dev/rwd0e
    # mkdir /disk2
    # echo '/dev/wd0e /disk2 ffs rw,log 2 2' >>/etc/fstab
    # mount /disk2
    # cd /usr ; pax -rw -pe -v stuff /disk2
    # rm -fr stuff ; ln -s /disk2/stuff .
    

P.S.: I'm offering choccolate to anyone fixing crossbuilding of NetBSD-current from Mac OS X. Any takers?

[Tags: disklabel, fusion, g4u, vmware]