zfs_dataset
zfs_dataset manages one ZFS filesystem dataset on the node and its
properties, such as its compression and its quota. It runs the program
zfs. A dataset lives in a ZFS pool, which must exist: a pool takes whole
disks and is created once for each server, so it stays a shell task with
a guard.
# The pool, once: zpool list succeeds when the pool exists.
- shell:
script: zpool create tank mirror /dev/sdb /dev/sdc
skip_if_this_command_succeeds: zpool list tank
- zfs_dataset:
name: tank/clickhouse
properties:
compression: lz4
recordsize: 128K
Set up
ZFS is two things: the programs zfs and zpool, and a kernel module.
Installing ZFS's programs and kernel module on a node differs by
distribution.
Ubuntu ships the kernel module with its kernel:
- when: {equals: ["{{ .node.distribution.id }}", ubuntu]}
apt_package: {packages: [{name: zfsutils-linux, version: newest_when_first_installed}]}
Debian has the packages in the section contrib of its own
repositories, which the node's package sources must list. A package builds
the kernel module on the node, for the kernel that runs, which takes about
five minutes:
- when: {equals: ["{{ .node.distribution.id }}", debian]}
apt_package:
packages:
- {name: "linux-headers-{{ .node.kernel.release }}", version: newest_when_first_installed}
- {name: zfs-dkms, version: newest_when_first_installed}
- {name: zfsutils-linux, version: newest_when_first_installed}
Rocky Linux and AlmaLinux take ZFS from the OpenZFS project's
repository, which a small package sets up. The repository zfs-kmod holds
the kernel module already built; the repository zfs, which is switched on
as installed, would build it on the node. OpenZFS builds both only for
x86_64, so an ARM64 server of the Red Hat family needs ZFS built from
OpenZFS's source, which this page does not cover. The first task installs
two plugins of dnf that the tasks after it need:
- when: {matches: ["{{ .node.distribution.id }}", "^(rocky|almalinux)$"]}
dnf_package: {packages: [{name: dnf-plugins-core, version: newest_when_first_installed}, {name: python3-dnf-plugin-versionlock, version: newest_when_first_installed}]}
- when:
all:
- matches: ["{{ .node.distribution.id }}", "^(rocky|almalinux)$"]
- number_equals: ["{{ .node.distribution.major_version }}", 9]
dnf_package:
packages:
- {name: zfs-release, version: "3-0.el9", package_file_url: https://zfsonlinux.org/epel/zfs-release-3-0.el9.noarch.rpm, package_file_sha256: 4954819e3d50522db0860647e5f093c143e46b3d8eba9a082fe4e5d8bbc0791e}
- when:
all:
- matches: ["{{ .node.distribution.id }}", "^(rocky|almalinux)$"]
- number_equals: ["{{ .node.distribution.major_version }}", 10]
dnf_package:
packages:
- {name: zfs-release, version: "3-0.el10", package_file_url: https://zfsonlinux.org/epel/zfs-release-3-0.el10.noarch.rpm, package_file_sha256: c2527e7d20aaa64cd77f1f1ca86b34123de6b710ab38e8f6a00206918efbc992}
- when: {matches: ["{{ .node.distribution.id }}", "^(rocky|almalinux)$"]}
dnf_repository: {existing_repository: zfs, enabled: false}
- when: {matches: ["{{ .node.distribution.id }}", "^(rocky|almalinux)$"]}
dnf_repository: {existing_repository: zfs-kmod, enabled: true}
- when: {matches: ["{{ .node.distribution.id }}", "^(rocky|almalinux)$"]}
dnf_package: {packages: [{name: zfs, version: newest_when_first_installed}]}
No service has to be started: a zfs command loads the kernel module by
itself.
Secure Boot. A node with Secure Boot on loads only a kernel module
that carries a signature it trusts. Ubuntu signs its ZFS module. On the
other distributions the kernel refuses the module (Key was rejected by service on Debian 13 and Rocky Linux 9) until the node's owner enrolls a
key that signs the module, or switches Secure Boot off.
Debian 12. ZFS installs as on Debian 13.
For sharenfs, the node also needs an NFS server: the package
nfs-kernel-server on Debian and Ubuntu, and nfs-utils on the Red Hat
family. Without one, ZFS stores the property and cannot share the
filesystem.
Inputs
| Input | Meaning | When omitted |
|---|---|---|
name |
The dataset as ZFS names it: the pool, then the datasets below it, with / between them, such as tank/clickhouse. |
Required. |
state |
present or deleted_with_all_its_data. deleted_with_all_its_data accepts only name. |
present |
properties |
ZFS properties with their values as text, each under the name that zfs get prints, such as compression. Only the listed properties are managed; a property that the task does not list keeps its value. Quote a value that YAML would read as something else, such as "off". |
The module sets no property. |
With state: present:
- A missing dataset is created with its properties (
zfs create). Its parent must exist: fortank/clickhousethe pooltank, fortank/clickhouse/logsthe datasettank/clickhouse. - An existing dataset gets each listed property whose value in effect
differs (
zfs set). A size is compared as a number of bytes, so128Kequals131072and1Gequals1024M. - A property of your own has a colon in its name, such as
com.example:owner. Its value is compared as text.
Write a property's long name. ZFS takes some properties also by a
second, shorter name, such as compress for compression and recsize
for recordsize. The module refuses the second name and names the long
one, before anything changes.
ZFS stores some values in another form than the one written, and the
form differs between ZFS versions: posixacl comes back as posix, and
gzip-6 as gzip. The module reads a property again after it set it.
When the stored value differs from the task's, the task fails and names the
form that ZFS stores, so that you write that form once. Otherwise the task
would set the property on every run.
With state: deleted_with_all_its_data, the module deletes the dataset
(zfs destroy) and everything in it. A dataset that has child datasets or
snapshots fails the task, which names them: the module never deletes
them along. A dataset that does not exist is unchanged.
The module manages filesystem datasets. A volume or a snapshot named by
the task's name input fails the task.
Outputs
| Output | Meaning |
|---|---|
created |
true when the task created the dataset, and in a dryrun when a run would create it. Otherwise false. |
A later node task reads the created output as .node_tasks.<id>.created.
What a dryrun reports
would_changewith what a run would do:would create the dataset tank/clickhouse with compression=lz4, recordsize=128K, each property that would change with its old and its new value, such aswould set compression of the dataset tank/clickhouse from off to lz4, orwould delete the dataset tank/clickhouse with all its data.unchangedwhen the dataset has the listed values.- A failure, with the same reason as in a run, for a property name that ZFS does not know, for a property's second name, and for a volume or a snapshot of the task's name. No node task changes such a state.
not_checkedfor a state that an earlier node task of the same playbook may change, because in a dryrun that task has changed nothing yet: while the programzfsis missing, while the pool or the parent dataset does not exist, while ZFS's kernel module is not loaded and cannot be loaded, and fordeleted_with_all_its_datawhile the dataset has child datasets or snapshots. A run fails once with the same reason; while the kernel module is not loaded, MMX tries a run again.- What a dryrun cannot know: whether ZFS takes a value, whether ZFS stores a value in another form, and whether a dataset is in use when a run wants to delete it.
- What its reads leave on the node: a
zfscommand loads ZFS's kernel module when the module is installed and not loaded yet, also in a dryrun. The module stays loaded.
Examples
Two datasets: one for ClickHouse, with compression and a record size that suits its files, and one with a quota, without access times, and with a property of your own:
- zfs_dataset:
name: tank/clickhouse
properties:
compression: lz4
recordsize: 128K
- zfs_dataset:
name: tank/backups
properties:
quota: 500G
atime: "off"
"com.example:owner": storage-team
A dataset below another one. The task for the parent dataset
tank/clickhouse comes before the task for tank/clickhouse/logs:
- zfs_dataset:
name: tank/clickhouse
- zfs_dataset:
name: tank/clickhouse/logs
properties:
compression: zstd
A dataset is deleted with everything in it, after the dataset below it:
- zfs_dataset:
name: tank/old-clickhouse/logs
state: deleted_with_all_its_data
- zfs_dataset:
name: tank/old-clickhouse
state: deleted_with_all_its_data
Errors and what they mean
Every reason starts with zfs_dataset. How often MMX tries a task that
fails depends on what failed:
- Once, in a dryrun too, when no node task changes what is in the way: an input that breaks a rule, an unknown property name, or a name that is a volume.
- Once in a run, and not checked in a dryrun, when an earlier node task may change what is in the way: it creates the pool or the parent dataset, or deletes the child datasets.
- Once, when the task changed the node and then failed, such as for a dataset that ZFS created and could not share: another try would find the dataset and report success.
- Again, when another command of ZFS failed.
| The reason holds | What it means, and what to do |
|---|---|
the program zfs is not installed; install the package ... |
ZFS's programs are not on the node. The reason names the package for each family of distributions. Install ZFS in an earlier node task; see Set up. |
zfs_dataset.state: deleted_with_all_its_data accepts only name; remove properties |
A task with state: deleted_with_all_its_data holds properties. Remove that input. |
the ZFS pool tank does not exist |
The node has no pool of that name. Create the pool in an earlier node task, with a guarded shell task as above. |
the parent dataset tank/clickhouse does not exist |
The dataset above the task's dataset is missing. Give the parent its own zfs_dataset task before this one. |
ZFS's kernel module is not loaded on this node, and the zfs program could not load it |
The programs are installed, and the kernel module is missing or the kernel refuses it. Check modprobe zfs on the node. With Secure Boot on, see Set up. After a kernel update, the module must fit the new kernel. |
properties: ZFS on this node knows no property ... |
A name in properties is no ZFS property, for example a typing mistake or a property of a newer ZFS. Nothing was changed. Correct the name; zfs get all <dataset> lists the properties. A property of your own needs a colon in its name. |
properties: compress is a second name of the ZFS property compression; write compression in the task |
The task names a property by its second, shorter name. Nothing was changed. Write the long name, which zfs get prints. |
ZFS took the value posixacl for the property acltype ... and stores it as posix |
ZFS keeps the value in another form than the task writes. The property is set, and after a creation the dataset exists and the task reports created: true. Write the stored form in the task, as the reason says. |
setting ... on the dataset ...: and ZFS's words |
ZFS refused a value, or the property can be set only when a dataset is created ('casesensitivity' is readonly). No property of the task was set. Read ZFS's words. A property of that kind cannot change on an existing dataset. MMX reports changed for the command that ran although ZFS refused it. |
ZFS created the dataset ... and then failed: and ZFS's words |
ZFS created the dataset and could then not mount or share it, for example with sharenfs and no NFS server on the node with ZFS 2.2 and 2.3. The dataset exists, and the task reports created: true. Remove the cause. The next run finds the dataset and compares its properties. |
... is a volume; the module manages filesystem datasets |
The name is a ZFS volume or another kind of dataset. The module leaves it alone; manage it with a shell task. |
the dataset ... has child datasets or snapshots, so it stays |
deleted_with_all_its_data for a dataset that has datasets or snapshots below it. The reason names the first five and counts the rest. Nothing was deleted. Delete each child dataset in an earlier node task, with its own zfs_dataset task, and each snapshot with zfs destroy. |
deleting the dataset ...: cannot unmount '...': pool or dataset is busy |
A program uses the dataset: it has a file open there, or works in one of its directories. Nothing was deleted. Stop the program, then run again. |
A reason that ends with The dataset ... was created |
A check right after the creation failed: ZFS stores a value in another form, or a command of ZFS that reads failed. The task reports created: true: another try would find the dataset and report that the task created nothing. |
A reason can end with ; ZFS warned: and ZFS's words. ZFS then stored the
value and could not apply it at once, for example cannot share ... for
sharenfs on a node without an NFS server. ZFS 2.4 also creates a dataset
with such a sharenfs and only warns, where ZFS 2.2 and 2.3 report an
error after the creation.
Tested on
Distributions: Ubuntu 24.04 and 26.04, Debian 13, Rocky Linux 9, AlmaLinux 10.
The suite tests/integration/zfs_datasets.py runs the whole test where the
tasks under Set up install ZFS in under two minutes, and on
Debian 13 with its build of the kernel module. It runs in a virtual
machine, with Secure Boot off, on a pool that is a file:
| Distribution | ZFS | Kernel module | The tasks under "Set up" take |
|---|---|---|---|
| Ubuntu 24.04 | 2.2.2 | comes with the kernel | about 25 seconds |
| Ubuntu 26.04 | 2.4.1 | comes with the kernel | about 30 seconds |
| Rocky Linux 9 | 2.2.11 | prebuilt, from the OpenZFS project's repository zfs-kmod |
about 90 seconds |
| AlmaLinux 10 | 2.2.11 | prebuilt, from the same repository | about 60 seconds |
| Debian 13 | 2.3.9 | built on the node by the package zfs-dkms |
about 5 minutes |
sharenfs with an NFS server on the node was tried by hand on Ubuntu
24.04 (nfs-kernel-server) and Rocky Linux 9 (nfs-utils): ZFS then
shares a new dataset without a message.
On Debian 12 the suite tests the module without ZFS: a dryrun reports the
task not_checked, and a run fails and names the program and its package.
It does the same on Rocky Linux 9 and AlmaLinux 10 ARM64 servers, for which
OpenZFS publishes no packages.
The suite does not install ZFS there, because the build of the kernel
module takes six to seven minutes. ZFS 2.1.11 was built and installed by
hand once on Debian 12, and the output of the zfs commands that the
module is built on was the same there.
A build on the node breaks whenever the distribution's kernel is newer than the ZFS version supports, which a test suite cannot foresee.
Limits on each distribution says what is limited for this module on Debian 12, Debian 13, Rocky Linux 9, and AlmaLinux 10, and why.
Why it works this way
- Pools stay a guarded
shelltask. Creating a pool takes whole disks and happens once for each server. A dataset is what changes over a server's life. - Only the listed properties are managed. ZFS has dozens of properties with good defaults, and a dataset inherits values from its parent. A task that lists two properties says nothing about the others.
- A value is compared as ZFS stores it. The module asks
zfs gettwice, for exact numbers and for the form a person writes, and takes a value as in effect when it equals either one or is the same number of bytes. A list of ZFS's own other names for a value differs between ZFS versions, so the module keeps none: it fails once and names the stored form. The module can learn that form only after the change, so this failure keeps its change and its outputcreated. - A property has one name in a task. ZFS 2.1.5 and 2.2.2 took
compressandrecsizeinzfs create,zfs set, andzfs get, andzfs getprintedcompressionandrecordsize. With both names allowed, two tasks could manage one property under two names. So the module takes the name thatzfs getprints and names it for the other. - The module looks again after a failed
zfs create. ZFS versions 2.2 and 2.3 exit with an error after they created a dataset that they could not share or mount. Without the second look, the task would report that nothing exists. The task then fails without another try: another try would find the dataset with its properties and report success, and nobody would read ZFS's message. - An unknown property name fails before anything changes. ZFS answers a question about several properties with no line at all when it does not know one of them. For a new dataset the module asks the parent dataset about the names first, so a dryrun finds the mistake too.
deleted_with_all_its_datanever deletes child datasets or snapshots along. They are data that the task does not name. The module lists them before it asks ZFS. A playbook deletes the child datasets and then their parent; in its dryrun the children are still there, so the dryrun reports the parent's task asnot_checked, and only a run fails.- A missing pool is no error of the dataset's task in a dryrun. A
playbook that creates the pool and then its datasets must pass its first
dryrun, so the task is
not_checkedthere. - A value holds no tab and no line break. The module reads values from
zfs get, which separates its columns with tabs and prints one line for each property.
This page is official_extra_modules/zfs_dataset/README.md in the MMX repository.