In a previous article we saw how easy is to use ZFS send/receive to perform differential backups using incremental snapshots. Today, we will see how we can automate the procedure in order to send incremental snapshots to a different machine over ssh.
There are two prerequisites for this procedure to work. First, the machine that is receiving the snapshots must be running the same or higher ZFS version from the machine that is being backed up. Second, we must either have root access to the receiving machine or an account that has been delegated with create, receive ZFS properties.
First lets prepare both host and target machines. We need to create and send our first full snapshot from our host to a ZFS folder on the target.
The host:
core2duo# zfs snapshot -r zroot/usr/src@-2012-01-10 core2duo# zfs list NAME USED AVAIL REFER MOUNTPOINT zroot/usr/src 349M 64.4G 349M /usr/src zroot/usr/src@-2012-01-10 0 - 349M - zroot/usr/src/mytest 31K 64.4G 31K /usr/src/mytest zroot/usr/src/mytest@-2012-01-10 0 - 31K -
The target:
hp# zfs create tank/test hp# zfs list NAME USED AVAIL REFER MOUNTPOINT tank/test 40.0K 747G 40.0K /tank/test
Now, lets send our first full snapshot:
core2duo# zfs send -R zroot/usr/src@-2012-01-10 | ssh root@hp zfs receive -Fduv tank/test Password: receiving full stream of zroot/usr/src@-2012-01-10 into tank/test/usr/src@-2012-01-10 received 732MB stream in 72 seconds (10.2MB/sec) receiving full stream of zroot/usr/src/mytest@-2012-01-10 into tank/test/usr/src/mytest@-2012-01-10 received 47.4KB stream in 1 seconds (47.4KB/sec)
Now, lets create a new snapshot and send them both incremental.
core2duo# zfs snapshot -r zroot/usr/src@-2012-01-11 core2duo# zfs send -R -i zroot/usr/src@-2012-01-10 zroot/usr/src@-2012-01-11 | ssh root@hp zfs receive -Fduv tank/test receiving full stream of zroot/usr/src@-2012-01-10 into tank/test/usr/src@-2012-01-10 received 732MB stream in 72 seconds (10.2MB/sec) receiving full stream of zroot/usr/src/mytest@-2012-01-10 into tank/test/usr/src/mytest@-2012-01-10 received 47.4KB stream in 1 seconds (47.4KB/sec)
Notice that I am using some special switches in my send/receive commands.
When sending, -R allows me to send the snapshots, their children and their properties.
When receiving, -F forces a rollback to the most recent snapshot, -d maintains my naming scheme and -u makes sure that the associated file systems do not get mounted.
Lets now see what we have done so far, first the host:
core2duo# zfs list NAME USED AVAIL REFER MOUNTPOINT zroot/usr/src 349M 64.4G 349M /usr/src zroot/usr/src@-2012-01-11 0 - 349M - zroot/usr/src/mytest 31K 64.4G 31K /usr/src/mytest zroot/usr/src/mytest@-2012-01-11 0 - 31K -
And the target:
hp# zfs list NAME USED AVAIL REFER MOUNTPOINT tank/test 364M 747G 41.3K /tank/test tank/test/usr 364M 747G 40.0K /tank/test/usr tank/test/usr/src 364M 747G 364M /tank/test/usr/src tank/test/usr/src@-2012-01-10 1.33K - 364M - tank/test/usr/src@-2012-01-11 0 - 364M - tank/test/usr/src/mytest 65.3K 747G 40.0K /tank/test/usr/src/mytest tank/test/usr/src/mytest@-2012-01-10 25.3K - 40.0K - tank/test/usr/src/mytest@-2012-01-11 0 - 40.0K -
Looks like it is working. The only thing left is to somehow automate the procedure. Let’s create a script that will create a snapshot which we will call it today, check if there was a snapshot yesterday and send them incrementally to a remote host.
#!/bin/sh pool="zroot/usr/src" destination="tank/test" host="10.10.10.4" today=`date +"$type-%Y-%m-%d"` yesterday=`date -v -1d +"$type-%Y-%m-%d"` # create today snapshot snapshot_today="$pool@$today" # look for a snapshot with this name if zfs list -H -o name -t snapshot | sort | grep "$snapshot_today$" > /dev/null; then echo " snapshot, $snapshot_today, already exists" exit 1 else echo " taking todays snapshot, $snapshot_today" zfs snapshot -r $snapshot_today fi # look for yesterday snapshot snapshot_yesterday="$pool@$yesterday" if zfs list -H -o name -t snapshot | sort | grep "$snapshot_yesterday$" > /dev/null; then echo " yesterday snapshot, $snapshot_yesterday, exists lets proceed with backup" zfs send -R -i $snapshot_yesterday $snapshot_today | ssh root@$host zfs receive -Fduv $destination echo " backup complete destroying yesterday snapshot" zfs destroy -r $snapshot_yesterday exit 0 else echo " missing yesterday snapshot aborting, $snapshot_yesterday" exit 1 fi
pool is the ZFS pool we want to backup
destination is the destination pool that will receive the backup
host is our backup host!
With a cronjob in place this would be a nice candidate for a daily ZFS incremental backup policy. Just make sure you use public/private key for ssh authentication.
