incusd/storage/zfs: Fix ZFS CreateVolume deletes pre-existing data on failure

[mrstux]

The issue was first reported by TrueNAS 25.04 beta test users. TrueNAS 25.04 includes Incus as a major feature

When the ZFS driver's CreateVolume function is unable to create the desired volume due to the existence of a pre-existing dataset, it would fail and proceed to recursively destroy the pre-existing dataset, potentially leading to unexpected data loss.

This has occurred to users who have been experimenting with trying to "import" their existing zvols into an incus zfs storage pool.

All other drivers were surveyed and they do not erase pre-existing data on failure, unless they first succeed in creating a volume or directory. The user is then responsible for correcting the situation in order to enable the CreateVolume call to succeed

The fix splits and moves the current revert.add(DeleteVolume) to only be used after a dataset that needs to be destroyed on reversion has been created or restored.

---

The issue and resolution are demonstrated below

Setup:

# create zfs storage pool
incus storage create zfsPool zfs
>Storage pool zfsPool created

# create pre-existing data (yes, users should not do this)
zfs create zfsPool/containers/lxc-blowaway
zfs create zfsPool/custom/default_blowaway_fs
zfs create zfsPool/custom/default_blowaway_block
zfs create zfsPool/virtual-machines/vm-blowaway
zfs create zfsPool/virtual-machines/vm-blowaway.block

# verify dataset existance
zfs list -o name -r zfsPool | grep blowaway
>zfsPool/containers/lxc-blowaway
>zfsPool/custom/default_blowaway_block
>zfsPool/custom/default_blowaway_fs
>zfsPool/virtual-machines/vm-blowaway
>zfsPool/virtual-machines/vm-blowaway.block

Without fix:

stux@incusdev:~/git/incus$ incus storage volume create zfsPool blowaway_fs
Error: Failed to run: zfs create -o mountpoint=legacy -o canmount=noauto zfsPool/custom/default_blowaway_fs: exit status 1 (cannot create 'zfsPool/custom/default_blowaway_fs': dataset already exists)

stux@incusdev:~/git/incus$ incus storage volume create zfsPool blowaway_block --type block
Error: Failed to run: zfs create -s -V 10737418240 -o volmode=none -o incus:content_type=block -o sync=disabled zfsPool/custom/default_blowaway_block: exit status 1 (cannot create 'zfsPool/custom/default_blowaway_block': dataset already exists)

stux@incusdev:~/git/incus$ incus create --empty lxc-blowaway --storage zfsPool
Creating lxc-blowaway
Error: Failed creating instance: Failed to run: zfs create -o mountpoint=legacy -o canmount=noauto zfsPool/containers/lxc-blowaway: exit status 1 (cannot create 'zfsPool/containers/lxc-blowaway': dataset already exists)

stux@incusdev:~/git/incus$ incus create --empty vm-blowaway --vm --storage zfsPool
Creating vm-blowaway
Error: Failed creating instance: Failed to run: zfs create -s -V 10737418240 -o volmode=none -o primarycache=metadata -o secondarycache=metadata -o sync=disabled zfsPool/virtual-machines/vm-blowaway.block: exit status 1 (cannot create 'zfsPool/virtual-machines/vm-blowaway.block': dataset already exists)

root@incusdev:~# zfs list -o name -r zfsPool | grep blowaway
root@incusdev:~# 

The pre-existing datasets are unexpectedly erased, and its important to note that none of the other Incus storage drivers behave this way.

With the fix:

stux@incusdev:~/git/incus$ incus storage volume create zfsPool blowaway_fs
Error: Failed to run: zfs create -o mountpoint=legacy -o canmount=noauto zfsPool/custom/default_blowaway_fs: exit status 1 (cannot create 'zfsPool/custom/default_blowaway_fs': dataset already exists)

stux@incusdev:~/git/incus$ incus storage volume create zfsPool blowaway_block --type block
Error: Failed to run: zfs create -s -V 10737418240 -o volmode=none -o incus:content_type=block -o sync=disabled zfsPool/custom/default_blowaway_block: exit status 1 (cannot create 'zfsPool/custom/default_blowaway_block': dataset already exists)

stux@incusdev:~/git/incus$ incus create --empty lxc-blowaway --storage zfsPool
Creating lxc-blowaway
Error: Failed creating instance: Failed to run: zfs create -o mountpoint=legacy -o canmount=noauto zfsPool/containers/lxc-blowaway: exit status 1 (cannot create 'zfsPool/containers/lxc-blowaway': dataset already exists)

stux@incusdev:~/git/incus$ incus create --empty vm-blowaway --vm --storage zfsPool
Creating vm-blowaway
Error: Failed creating instance: Failed to run: zfs create -s -V 10737418240 -o volmode=none -o primarycache=metadata -o secondarycache=metadata -o sync=disabled zfsPool/virtual-machines/vm-blowaway.block: exit status 1 (cannot create 'zfsPool/virtual-machines/vm-blowaway.block': dataset already exists)

root@incusdev:~# zfs list -o name -r zfsPool | grep blowaway
zfsPool/containers/lxc-blowaway
zfsPool/custom/default_blowaway_block
zfsPool/custom/default_blowaway_fs
zfsPool/virtual-machines/vm-blowaway
zfsPool/virtual-machines/vm-blowaway.block

The pre-existing datasets are not erased.

Cleanup:

stux@incusdev:~/git/incus$ incus storage rm zfsPool
Storage pool zfsPool deleted