Here’s my notes for ZFS root installation with FreeBSD 8.1. (amd64 on my system)
Boot up the FreeBSD install disk, you’ll need one with the Fixit system. In the examples listed, I had 2 disks, ad4 and ad6. Yours may be different, replace as necessary.
1. Add the modules on boot, hit “6” at the BTX loader menu, then enter these lines at the “OK” prompt.
load ahci.ko load opensolaris.ko load zfs.ko boot
2. At the sysinstall menu, go to Fixit.
3. Create the GPT disks and partitions.
- Delete any partions already on the disk, and destroy. (only if needed) On my disks, I had 2 partitions created from another OS installation. “-i 2†indicates the partition number. You may need to add a third or more.
gpart delete -i 2 ad4 gpart delete -i 1 ad4 gpart delete -i 2 ad6 gpart delete -i 1 ad6 gpart destroy ad4 gpart destroy ad6
- Create the gpt disk.
gpart create -s gpt ad4 gpart create -s gpt ad6
- (NOTE: “gpart show†and “gpart show –l†will display the partitions.
- You may want to layout different partitions, but we’ll just create a boot, swap, and zfs partition here, with GPT labels. (swap is better left off ZFS for crash dumps)
gpart add -s 128 -t freebsd-boot -l boot0 ad4 gpart add -s 128 -t freebsd-boot -l boot1 ad6 gpart add -s 8388608 -t freebsd-swap -l swap0 ad4 gpart add -s 8388608 -t freebsd-swap -l swap1 ad6 gpart add -t freebsd-zfs -l disk0 ad4 gpart add -t freebsd-zfs -l disk1 ad6
-
Next, add the bootcode to the disk.
gpart bootcode -b /dist/boot/pmbr -p /dist/boot/gptzfsboot -i 1 ad4 gpart bootcode -b /dist/boot/pmbr -p /dist/boot/gptzfsboot -i 1 ad6
4. OK, now we’ve got some GTP disks setup for a mirror and boot.
IMPORTANT!!! Make sure you don’t skip this step, create the /boot/zfs directory for the zpool.cache.
mkdir /boot/zfs
5. Now we can start the ZFS fun! On the FreeBSD wiki and on other sites I used as reference, there may be listed other options and settings for your ZFS layout. Here, I am going to keep it very simple, but I’ll list out my options later.
Also, you may prefer different names or even separate location/directory/ZFS filesystem for your root pool. My preference is to call the root pool “rpool†and place the FreeBSD system in the ROOT filesystem. (looks like this: rpool/ROOT) In addition, my preference is to place “/home†outside of the ROOT fs. Which allows is to separate system from data and management of snapshots is more flexible. But this is totally a preference thing. One of the beautiful things about ZFS is the flexibility and ease of use.
- Let’s create the pool. Notice the gpt/disk# items, which correspond to your gpt labels. Also, we’ll disable a mountpoint on rpool.
zpool create rpool mirror gpt/disk0 gpt/disk1 zfs set mountpoint=none rpool
- Add the ROOT filesystem. Again, here you may prefer to add all ZFS filesystems for places like /usr, /var, /tmp, and other system locations. I don’t recommend it, but in this example we’ll keep it all in one filesystem, except /home.
zfs create -o mountpoint=/mnt rpool/ROOT zfs create -o mountpoint=/mnt/home rpool/home
(OPTIONAL file systems)
zfs create rpool/ROOT/usr zfs create -o compression=lzjb -o setuid=off rpool/ROOT/usr/ports zfs create -o compression=off -o exec=off -o setuid=off rpool/ROOT/usr/ports/distfiles zfs create -o compression=off -o exec=off -o setuid=off rpool/ROOT/usr/ports/packages zfs create -o compression=lzjb -o setuid=off rpool/ROOT/usr/src zfs create rpool/ROOT/var zfs create -o exec=on -o setuid=off rpool/ROOT/tmp
6. Install the base system. We’ll be intsalling the system into the /mnt directory for now, but we’ll change those mounts later.
-
cd /dist/8.1* export DESTDIR=/mnt for dir in base catpages dict doc games info lib32 manpages ports; do (cd $dir ; ./install.sh) ; done cd src ; ./install.sh all cd ../kernels ; ./install.sh generic cd /mnt/boot ; cp -Rlp GENERIC/* /mnt/boot/kernel/
7. Edit /mnt/boot/loader.conf and add these lines.
ahci_load="YES" zfs_load="YES" vfs.root.mountfrom="zfs:rpool/ROOT"
8. Edit /mnt/etc/rc.conf and add these lines. I have re0 as my network interface, your’s may be different.
zfs_enable="YES" ifconfig_re0="DHCP" hostname="systemname"
9. Edit /mnt/etc/fstab and add these lines. Setup your swap.
/dev/gpt/swap0 none swap sw 0 0 /dev/gpt/swap1 none swap sw 0 0
10. Copy the zpool.cache. IMPORTANT!!!!! Don’t forget this step!!
cp /boot/zfs/zpool.cache /mnt/boot/zfs/zpool.cache
11. Export this…
export LD_LIBRARY_PATH=/dist/lib
12. Unmount ZFS filesystems and set the correct mountpoints for the new root to boot. If you setup other/optional zfs filesystems, then you’ll want to set their correct mount points, too.
zfs umount -a zfs set mountpoint=legacy rpool/ROOT zfs set mountpoint=/home rpool/home
13. Set the bootfs property of the pool to rpool/ROOT
zpool set bootfs=rpool/ROOT rpool
That’s about it! Well, actually, you’ll still need to do a lot of post install stuff. (see the wiki (set passwd, time zone, etc)) This is only a very base load of FreeBSD, root doesn’t even have a password.
Keep in mind, at the time I wrote this, there is a bug in the zfs boot loader, where you can only boot from the first disk in the mirror. Kind of reduces the usefulness of a root mirror, that’s for sure. However, there is a patch that you can compile a new zfs loader with. There’s a procedure to do so, and I’ll write up a how-to on that sooner than later.
Here are the site’s I used as reference, as well as the forum post on the root mirror boot issue.
http://www.b0rken.org/freebsd/zfs.html