ZFS Quickstart

ZFS has a truly unique combination of features

ZFS is able to distribute data across multiple devices, and this format is portable. Unlike traditional RAID, only dataset records are copied, making recovery very efficient. ZFS does away with the need for a volume manager, so there are no platform-specific layers such as BSD disklabels, or LVM. (BTRFS also has these capabilities, but is Linux-only.)
Native encryption, which allows send/receive of datasets even without unlocking and mounting a volume.
By setting volmode raw block devices can be provisioned for virtual machines, or across the network by way of iSCSI.

Rocky/Alma Linux Install

dnf install https://zfsonlinux.org/epel/zfs-release-2-2.el9.noarch.rpm
dnf config-manager --disable zfs
dnf config-manager --enable zfs-kmod
dnf install zfs
echo zfs > /etc/modules-load.d/zfs.conf

FreeBSD

# /etc/rc.conf
zfs_enable="YES"

ZFS does not require a partition table, but initializing a disk with GUID partition map will avoid spurious warnings and make it clear what kind of file system is on a device

geom disk list                 # List block devices
gpart destroy -F nda1          # Delete partition data
gpart create -s gpt nda1       # New GUID partition table
gpart add -t freebsd-zfs nda1  # Create and label partition

Create zpool and new volume on first partition

zpool create -O compression=lz4 zpool2 /dev/nda1p1
zfs create -o mountpoint=/ci zpool2/ci

Automated Snapshot Managment

To automate snapshot retention make a new periodic snapshot and prune anything that is more than N days old

#!/bin/sh -e

today=$(date +"%Y-%m-%d")

for fs in zpool2/ci; do
   zfs snapshot $fs@$today

   for snap in $(zfs list -t snapshot -H -o name $fs | sort -r | tail +30); do
      zfs destroy $snap
   done
done

Run daily

15  20  *  *  * /usr/local/bin/zfs-snap.sh

Replication

Since ZFS send and receive use a pipe, they work equally well locally or over SSH. The destination is writable by default, but if the goal is to send incremental updates make it read-only and disable atime

zfs set readonly=on zpool2
zfs set atime=off zpool2
zfs send -v  zpool1/backup@2026-04-02 | zfs receive zpool2/received

Send the deleta by naming the last and current snapshot

zfs send -vi @2026-04-02 zpool1/backup@2026-04-03 | zfs receive zpool2/received

NFS Export

While any local mount can be added to /etc/exports ZFS sharing allows mount points to be automatically by setting the shrenfs property on each volume

$ doas zfs sharenfs='-network 192.168.2.0/24' zpool2/ci
$ zfs get sharenfs zpool2/ci
NAME       PROPERTY  VALUE                    SOURCE
zpool2/ci  sharenfs  -network 192.168.2.0/24  local

To unshare a pool or volume

$ doas zfs set sharenfs=off zpool2

On FreeBSD, rpcbind must also be enabled. Optionally allow clients so connect without the resvport option.

# /etc/rc.conf
rpcbind_enable="YES"
nfs_reserved_port_only="NO"

Applications using flock(2) will hang if locking is not also enabled

# /etc/rc.conf
rpc_lockd_enable="YES"
rpc_statd_enable="YES"

Virtual Machines

When running KVM or Bhyave, ZFS can provide a device that can be attached to a virtual machine directly. This is referred to as a zvol

zfs create -sV 100G -o volmode=dev zpool2/vm/mykube2

Encryption

For manual unlock

zfs create -o encryption=on -o keylocation=prompt -o keyformat=passphrase zroot/home
zfs set setuid=off zroot/home
zfs set devices=off zroot/home
zfs set mountpoint=/home zroot/home

Then in rc.local

zfs load-key -r zroot/home
zfs mount zroot/home

Import all Pools after Install

zpool import -a

Formulas

Create a mirror

zpool create zpool0 /dev/ada0p1
zpool attach zpool0 /dev/ada0p1 /dev/ada1p1
zpool status

Add spare

zpool add zpool0 spare /dev/ada2p1

Replace disk

zpool replace zpool0 /dev/ada1p1 /dev/ada2p1
zpool detach zpool0 /dev/ada1p1
zpool add zpool0 spare /dev/ada1p1

Create RAID-Z pool

zpool create zpool0 raidz /dev/ada0p1 /dev/ada1p1 /dev/ada2p1

Tuning for PostgreSQL

The OpenZFS Workload Tuning pages indicates that full_page_writes can be disabled since there is not need to guard against torn pages on ZFS. This parameter will likely lead to corruption if the database is replicated to a non-ZFS volume.

Configuring Oracle Solaris ZFS for an Oracle Database (September 2020) provides a detailed guide that could be translated for PostgreSQL. Setting recordsize, logbias, primarycache and compression for various parts is var too complex to validate. Instead, the solution needs to simple enough to gain a consensus view.

Eric Radman : a Journal