This is an updated guide showing how you can install a full ZFS system using FreeBSD9.
In this guide I will demonstrate how you can install a fully functional full ZFS FreeBSD9 using a GPT scheme. We will also use ZFS for SWAP
You can use this as a reference guide for a single or mirror installation.
(1) Boot from a FreeBSD9 installation DVD or memstick and choose “Live CD”.
(2) Create the necessary partitions on the disk(s) and add ZFS aware boot code.
a) For a single disk installation.
gpart create -s gpt ada0 gpart add -b 34 -s 94 -t freebsd-boot ada0 gpart add -t freebsd-zfs -l disk0 ada0 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
b) Repeat the procedure for the second drive if you want a mirror installation.
gpart create -s gpt ada1 gpart add -b 34 -s 94 -t freebsd-boot ada1 gpart add -t freebsd-zfs -l disk1 ada1 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1
(3) Create the pool.(ignore any warnings regarding mounting)
a) For a single disk installation.
zpool create zroot /dev/gpt/disk0
b) For a mirror installation.
zpool create zroot mirror /dev/gpt/disk0 /dev/gpt/disk1
(4) Set bootfs property, checksums and mountpoints.
zpool set bootfs=zroot zroot zfs set checksum=fletcher4 zroot zfs set mountpoint=/mnt zroot
(5)) At this point export and import the pool while preserving zroot.cache in /var/tmp.
zpool export zroot zpool import -o cachefile=/var/tmp/zpool.cache zroot
(6) Create appropriate filesystems (feel free to improvise!).
zfs create zroot/usr zfs create zroot/usr/home zfs create zroot/var zfs create -o compression=on -o exec=on -o setuid=off zroot/tmp zfs create -o compression=lzjb -o setuid=off zroot/usr/ports zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/distfiles zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/packages zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/usr/src zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/crash zfs create -o exec=off -o setuid=off zroot/var/db zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/db/pkg zfs create -o exec=off -o setuid=off zroot/var/empty zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log zfs create -o compression=gzip -o exec=off -o setuid=off zroot/var/mail zfs create -o exec=off -o setuid=off zroot/var/run zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/tmp
(7) Add swap space and disable checksums. In this case I add 4GB of swap.
zfs create -V 4G zroot/swap zfs set org.freebsd:swap=on zroot/swap zfs set checksum=off zroot/swap
(8) Create a symlink to /home and fix some permissions.
chmod 1777 /mnt/tmp cd /mnt ; ln -s usr/home home chmod 1777 /mnt/var/tmp
(9) Instal FreeBSD.
sh cd /usr/freebsd-dist export DESTDIR=/mnt for file in base.txz lib32.txz kernel.txz doc.txz ports.txz src.txz; do (cat $file | tar --unlink -xpJf - -C ${DESTDIR:-/}); done
(10) Copy zpool.cache (very important!!!)
cp /var/tmp/zpool.cache /mnt/boot/zfs/zpool.cache
(11) Create the rc.conf, loader.conf and an empty fstab (otherwise the system will complain).
echo 'zfs_enable="YES"' >> /mnt/etc/rc.conf echo 'zfs_load="YES"' >> /mnt/boot/loader.conf echo 'vfs.root.mountfrom="zfs:zroot"' >> /mnt/boot/loader.conf touch /mnt/etc/fstab
(12) Unmount everything and fix mountpoints for system boot.
zfs set readonly=on zroot/var/empty zfs umount -af zfs set mountpoint=legacy zroot zfs set mountpoint=/tmp zroot/tmp zfs set mountpoint=/usr zroot/usr zfs set mountpoint=/var zroot/var
Reboot, adjust time zone info, add a password for root, add a user and enjoy!!!
If you are looking for a 4K optimized guide see my next guide!
Great post – saved me alot of googling!
Thanks for this. Glad to see the recent 9.0 builds made this simpler again
Very nice tutorial. Flawlessly works for me.
I had to use
# zfs umount -af
at the end to get /mnt unmounted.
Is there a working howto for ms-dos partitioned disk without the need of having a non-zfs boot partition?
You mean dual booting ?
That you require an MBR scheme.
Yes, I mean MBR disk and dualboot with Windows 7.
One year ago I was able to do that only with UFS boot. Can I have MBR + Windows 7 + ZFS (without UFS)?
Nothing that really works, sorry.
FreeBSD 9.0 should mirror the first four GPT partitions in the MBR. Before that, you could do the same thing manually (that’s what I do). So, yes, you can dual boot Windows 7 on the same GPT disk used by FreeBSD. That will require a boot manager, I use GRUB because I’ve got a triple boot with some Linux distro.
After 10th step one must exit the shell to umount /mnt
Do the potential problems with zvols and swap space (as listed here http://lists.freebsd.org/pipermail/freebsd-current/2007-September/076831.html) still apply or has this been resolved in the version of zfs provided in 9.0?
As far as I am aware, ZFS v28 has solved these issues.
I don’t have deadlocks any more with FreeBSD 9.0 and a zvol swap, but swapping is still ways slower than on dedicated freebsd-swap partitions. The best thing I could do to my computer was to buy 8G of RAM instead of 2, ZFS is memory hungry.
Thanks so much for this blog post. Concise yet thorough. Very helpful!
Thanks for the very nice writeup!
One question regarding step 3: why is the boot partition created at an uneven 4k-offset (-b 34 = 4.25 4k blocks)? It is not a big issue because if (!) the disk reports physical block size correctly ZFS will “correctly” allocate its structures anyways. However, wouldn’t it be just nicer to simply go for, e.g., -b 40, to always be on 4k boundaries, even on those 4k drives that report 512byte blocks?
Hi Michael,
You are correct but it is a bit more complicated. We would have to use gnop for this. I will update the guide as soon as I get a couple of 4K sector drives in my hands.
You are right – if you have one of these nasty 4k drives that report 512byte physical sectors, you actually need the gnop trick anyways for ZFS to get it right – then you can also use the -b with gpart.
No. Without proper alignment, the use of gnop is useless. So please update your guide to -b 40 for freebsd-boot and -b 2048 for freebsd-swap.
Hi Knarf,
I am planning on updating this as soon as I get a pair of those drives. However, I don’t think that just a:
gpart add -b 40 -s 64k -t freebsd-boot ada0
is enough.
It’s enough for alignment, but the gnop part is more than desirable if you want the zpool to use the optimal minimum blocksize.
According to the latest gpart(8) manpage (see r230059), you should use :
/sbin/gpart add -b 34 -s 94 -t freebsd-boot ada0
For all readers, please also see the official wiki :
http://wiki.freebsd.org/RootOnZFS
Quoting from man pages:
A size of 15 blocks (7680 bytes) would be sufficient for booting from UFS but let’s use 128 blocks (64 KB) here in this example, in order to reserve some space for potential future need (e.g. from a ZFS partition).
You’re not quoting the latest man page
With -s 94, “the next partition will be aligned on a 64 kB boundary without the need to specify an explicit offset or alignment.”. So that will work for 4k-sectors disks.
http://freshbsd.org/commit/freebsd/r230059
Nothing changed in the code, only a man page adjustment.
Thanks for the valuable input. The guides have been updated to reflect the recent changes.
timesaver! thnx!
I’m beginner in FreeBSD. I’ve done everything step by step like in Your tutorial but after 12 and reboot I got “Missing boot loader”. I’m sure I’ve done point 2 and got something like “Successfull boot install”. What can be wrong here?
Make sure that step 2 is done and that you are actually booting from the disk that you partitioned.
I have simple way for zpool.cache
Create mdmfs for zpool.cache file
# mdmfs -s 1M md /boot/zfs
And copy zpool.cache to root
# cp /boot/zfs/zpool.cache /mnt/boot/zfs/zpool.cache