Freebsd installation zfs

Putting ZFS in a separate partition allows the same disk to have other partitions for other purposes. In particular, it allows adding partitions with bootcode and file systems needed for booting. This allows booting from disks that are also members of a pool. Using partitions also allows the administrator to under-provision the disks, using less than the full capacity.

If a future replacement disk of the same nominal size as the original actually has a slightly smaller capacity, the smaller partition will still fit, using the replacement disk. Destroy a pool that is no longer needed to reuse the disks. Destroying a pool requires unmounting the file systems in that pool first. If any dataset is in use, the unmount operation fails without destroying the pool.

Force the pool destruction with -f. This can cause undefined behavior in applications which had open files on those datasets. Two ways exist for adding disks to a zpool: attaching a disk to an existing vdev with zpool attach , or adding vdevs to the pool with zpool add. Some vdev types allow adding disks to the vdev after creation. A pool created with a single disk lacks redundancy.

It can detect corruption but can not repair it, because there is no other copy of the data. The copies property may be able to recover from a small failure such as a bad sector, but does not provide the same level of protection as mirroring or RAID-Z. Starting with a pool consisting of a single disk vdev, use zpool attach to add a new disk to the vdev, creating a mirror.

Also use zpool attach to add new disks to a mirror group, increasing redundancy and read performance. When partitioning the disks used for the pool, replicate the layout of the first disk on to the second. Use gpart backup and gpart restore to make this process easier. Upgrade the single disk stripe vdev ada0p3 to a mirror by attaching ada1p3 :. When adding disks to the existing vdev is not an option, as for RAID-Z, an alternative method is to add another vdev to the pool.

Adding vdevs provides higher performance by distributing writes across the vdevs. Each vdev provides its own redundancy. Adding a non-redundant vdev to a pool containing mirror or RAID-Z vdevs risks the data on the entire pool.

Distributing writes means a failure of the non-redundant disk will result in the loss of a fraction of every block written to the pool. ZFS stripes data across each of the vdevs.

For example, with two mirror vdevs, this is effectively a RAID 10 that stripes writes across two sets of mirrors. Having vdevs with different amounts of free space will lower performance, as more data writes go to the less full vdev. Attach a second mirror group ada2p3 and ada3p3 to the existing mirror:. Removing vdevs from a pool is impossible and removal of disks from a mirror is exclusive if there is enough remaining redundancy. If a single disk remains in a mirror group, that group ceases to be a mirror and becomes a stripe, risking the entire pool if that remaining disk fails.

Pool status is important. If a drive goes offline or ZFS detects a read, write, or checksum error, the corresponding error count increases. The status output shows the configuration and status of each device in the pool and the status of the entire pool. Actions to take and details about the last scrub are also shown.

When detecting an error, ZFS increases the read, write, or checksum error counts. Clear the error message and reset the counts with zpool clear mypool. Clearing the error state can be important for automated scripts that alert the administrator when the pool encounters an error. Without clearing old errors, the scripts may fail to report further errors. It may be desirable to replace one disk with a different disk. When replacing a working disk, the process keeps the old disk online during the replacement.

The pool never enters a degraded state, reducing the risk of data loss. Running zpool replace copies the data from the old disk to the new one. After the operation completes, ZFS disconnects the old disk from the vdev. If the new disk is larger than the old disk, it may be possible to grow the zpool, using the new space.

See Growing a Pool. When a disk in a pool fails, the vdev to which the disk belongs enters the degraded state. The data is still available, but with reduced performance because ZFS computes missing data from the available redundancy. To restore the vdev to a fully functional state, replace the failed physical device. ZFS is then instructed to begin the resilver operation. ZFS recomputes data on the failed device from available redundancy and writes it to the replacement device.

After completion, the vdev returns to online status. If the vdev does not have any redundancy, or if devices have failed and there is not enough redundancy to compensate, the pool enters the faulted state. Unless enough devices can reconnect the pool becomes inoperative requiring a data restore from backups.

When replacing a failed disk, the name of the failed disk changes to the GUID of the new disk. A new device name parameter for zpool replace is not required if the replacement device has the same device name. Routinely scrub pools, ideally at least once every month. The scrub operation is disk-intensive and will reduce performance while running. Avoid high-demand periods when scheduling scrub or use vfs. The checksums stored with data blocks enable the file system to self-heal. This feature will automatically repair data whose checksum does not match the one recorded on another device that is part of the storage pool.

For example, a mirror configuration with two disks where one drive is starting to malfunction and cannot properly store the data any more.

This is worse when the data was not accessed for a long time, as with long term archive storage. Traditional file systems need to run commands that check and repair the data like fsck 8. These commands take time, and in severe cases, an administrator has to decide which repair operation to perform. When ZFS detects a data block with a mismatched checksum, it tries to read the data from the mirror disk. If that disk can provide the correct data, ZFS will give that to the application and correct the data on the disk with the wrong checksum.

This happens without any interaction from a system administrator during normal pool operation. Copy some important data to the pool to protect from data errors using the self-healing feature and create a checksum of the pool for later comparison. Simulate data corruption by writing random data to the beginning of one of the disks in the mirror.

To keep ZFS from healing the data when detected, export the pool before the corruption and import it again afterwards. This is a dangerous operation that can destroy vital data, shown here for demonstration alone. Do not try it during normal operation of a storage pool. Nor should this intentional corruption example run on any disk with a file system not using ZFS on another partition in it. Do not use any other disk device names other than the ones that are part of the pool. Ensure proper backups of the pool exist and test them before running the command!

The pool status shows that one device has experienced an error. Note that applications reading data from the pool did not receive any incorrect data. ZFS provided data from the ada0 device with the correct checksums. To find the device with the wrong checksum, look for one whose CKSUM column contains a nonzero value.

ZFS detected the error and handled it by using the redundancy present in the unaffected ada0 mirror disk. A checksum comparison with the original one will reveal whether the pool is consistent again. Generate checksums before and after the intentional tampering while the pool data still matches. This shows how ZFS is capable of detecting and correcting any errors automatically when the checksums differ.

Note this is possible with enough redundancy present in the pool. A pool consisting of a single device has no self-healing capabilities. That is also the reason why checksums are so important in ZFS; do not disable them for any reason.

ZFS requires no fsck 8 or similar file system consistency check program to detect and correct this, and keeps the pool available while there is a problem. A scrub operation is now required to overwrite the corrupted data on ada1.

The scrub operation reads data from ada0 and rewrites any data with a wrong checksum on ada1 , shown by the repairing output from zpool status. After the operation is complete, the pool status changes to:. After the scrubbing operation completes with all the data synchronized from ada0 to ada1 , clear the error messages from the pool status by running zpool clear. The smallest device in each vdev limits the usable size of a redundant pool.

Replace the smallest device with a larger device. After completing a replace or resilver operation, the pool can grow to use the capacity of the new device. For example, consider a mirror of a 1 TB drive and a 2 TB drive. The usable space is 1 TB. When replacing the 1 TB drive with another 2 TB drive, the resilvering process copies the existing data onto the new drive.

Start expansion by using zpool online -e on each device. After expanding all devices, the extra space becomes available to the pool. Export pools before moving them to another system. ZFS unmounts all datasets, marking each device as exported but still locked to prevent use by other disks.

This allows pools to be imported on other machines, other operating systems that support ZFS, and even different hardware architectures with some caveats, see zpool 8. When a dataset has open files, use zpool export -f to force exporting the pool.

Use this with caution. The datasets are forcibly unmounted, potentially resulting in unexpected behavior by the applications which had open files on those datasets. Importing a pool automatically mounts the datasets. If this is undesired behavior, use zpool import -N to prevent it. If the pool was last used on a different system and was not properly exported, force the import using zpool import -f. After upgrading FreeBSD, or if importing a pool from a system using an older version, manually upgrade the pool to the latest ZFS version to support newer features.

Consider whether the pool may ever need importing on an older system before upgrading. Upgrading is a one-way process. Upgrade older pools is possible, but downgrading pools with newer features is not. The newer features of ZFS will not be available until zpool upgrade has completed.

Use zpool upgrade -v to see what new features the upgrade provides, as well as which features are already supported. Update the boot code on systems that boot from a pool to support the new pool version. Use gpart bootcode on the partition that contains the boot code. Two types of bootcode are available, depending on way the system boots: GPT the most common option and EFI for more modern systems.

Apply the bootcode to all bootable disks in the pool. See gpart 8 for more information. ZFS records commands that change the pool, including creating datasets, changing properties, or replacing a disk. History is not kept in a log file, but is part of the pool itself.

The command to review this history is aptly named zpool history :. The output shows zpool and zfs commands altering the pool in some way along with a timestamp. Commands like zfs list are not included. When specifying no pool name, ZFS displays history of all pools. Show more details by adding -l. Showing history records in a long format, including information like the name of the user who issued the command and the hostname on which the change happened. The hostname display becomes important when exporting the pool from one system and importing on another.

Combine both options to zpool history to give the most detailed information possible for any given pool. Pool history provides valuable information when tracking down the actions performed or when needing more detailed output for debugging. By default, ZFS monitors and displays all pools in the system. Provide a pool name to limit monitoring to that pool. A basic example:. The next statistic line prints after each interval.

Give a second number on the command line after the interval to specify the total number of statistics to display. Each device in the pool appears with a statistics line. This is useful for seeing read and write operations performed on each device, and can help determine if any individual device is slowing down the pool. This example shows a mirrored pool with two devices:. ZFS can split a pool consisting of one or more mirror vdevs into two pools.

Unless otherwise specified, ZFS detaches the last member of each mirror and creates a new pool containing the same data. Be sure to make a dry run of the operation with -n first. This displays the details of the requested operation without actually performing it. This helps confirm that the operation will do what the user intends. The zfs utility can create, destroy, and manage all existing ZFS datasets within a pool.

To manage the pool itself, use zpool. Unlike traditional disks and volume managers, space in ZFS is not preallocated. With traditional file systems, after partitioning and assigning the space, there is no way to add a new file system without adding a new disk. With ZFS, creating new file systems is possible at any time.

Each dataset has properties including features like compression, deduplication, caching, and quotas, as well as other useful properties like readonly, case sensitivity, network file sharing, and a mount point. Nesting datasets within each other is possible and child datasets will inherit properties from their ancestors.

Delegate , replicate , snapshot , jail allows administering and destroying each dataset as a unit. Creating a separate dataset for each different type or set of files has advantages.

The drawbacks to having a large number of datasets are that some commands like zfs list will be slower, and that mounting of hundreds or even thousands of datasets will slow the FreeBSD boot process. Destroying a dataset is much quicker than deleting the files on the dataset, as it does not involve scanning the files and updating the corresponding metadata. In modern versions of ZFS, zfs destroy is asynchronous, and the free space might take minutes to appear in the pool.

Use zpool get freeing poolname to see the freeing property, that shows which datasets are having their blocks freed in the background. If there are child datasets, like snapshots or other datasets, destroying the parent is impossible. To destroy a dataset and its children, use -r to recursively destroy the dataset and its children.

Use -n -v to list datasets and snapshots destroyed by this operation, without actually destroy anything. Space reclaimed by destroying snapshots is also shown. A volume is a special dataset type.

This allows using the volume for other file systems, to back the disks of a virtual machine, or to make it available to other network hosts using protocols like iSCSI or HAST.

Format a volume with any file system or without a file system to store raw data. To the user, a volume appears to be a regular disk. Putting ordinary file systems on these zvols provides features that ordinary disks or file systems do not have. For example, using the compression property on a MB volume allows creation of a compressed FAT file system. Destroying a volume is much the same as destroying a regular file system dataset.

The operation is nearly instantaneous, but it may take minutes to reclaim the free space in the background. To change the name of a dataset, use zfs rename. To change the parent of a dataset, use this command as well. Renaming a dataset to have a different parent dataset will change the value of those properties inherited from the parent dataset. Renaming a dataset unmounts then remounts it in the new location inherited from the new parent dataset. To prevent this behavior, use -u. Renaming snapshots uses the same command.

Due to the nature of snapshots, rename cannot change their parent dataset. To rename a recursive snapshot, specify -r ; this will also rename all snapshots with the same name in child datasets.

Each ZFS dataset has properties that control its behavior. Most properties are automatically inherited from the parent dataset, but can be overridden locally. Most properties have a limited set of valid values, zfs get will display each possible property and valid values. Using zfs inherit reverts most properties to their inherited values. User-defined properties are also possible. They become part of the dataset configuration and provide further information about the dataset or its contents.

To distinguish these custom properties from the ones supplied as part of ZFS, use a colon : to create a custom namespace for the property. To remove a custom property, use zfs inherit with -r. Setting these defines if and how ZFS shares datasets on the network. To get the current status of a share, enter:.

Set other options for sharing datasets through NFS, such as -alldirs , -maproot and -network. To set options on a dataset shared through NFS, enter:.

Snapshots are one of the most powerful features of ZFS. A snapshot provides a read-only, point-in-time copy of the dataset. If no snapshots exist, ZFS reclaims space for future use when data is rewritten or deleted. Snapshots preserve disk space by recording just the differences between the current dataset and a previous version. Allowing snapshots on whole datasets, not on individual files or directories. A snapshot from a dataset duplicates everything contained in it.

This includes the file system properties, files, directories, permissions, and so on. Snapshots use no extra space when first created, but consume space as the blocks they reference change. Recursive snapshots taken with -r create snapshots with the same name on the dataset and its children, providing a consistent moment-in-time snapshot of the file systems. This can be important when an application has files on related datasets or that depend upon each other.

Without snapshots, a backup would have copies of the files from different points in time. Snapshots in ZFS provide a variety of features that even other file systems with snapshot functionality lack. A typical example of snapshot use is as a quick way of backing up the current state of the file system when performing a risky action like a software installation or a system upgrade.

If the action fails, rolling back to the snapshot returns the system to the same state when creating the snapshot. If the upgrade was successful, delete the snapshot to free up space. Without snapshots, a failed upgrade often requires restoring backups, which is tedious, time consuming, and may require downtime during which the system is unusable.

Rolling back to snapshots is fast, even while the system is running in normal operation, with little or no downtime. The time savings are enormous with multi-terabyte storage systems considering the time required to copy the data from backup.

Snapshots are not a replacement for a complete backup of a pool, but offer a quick and easy way to store a dataset copy at a specific time. To create snapshots, use zfs snapshot dataset snapshotname. Adding -r creates a snapshot recursively, with the same name on all child datasets. Snapshots are not shown by a normal zfs list operation. To list snapshots, append -t snapshot to zfs list. Compare the snapshot to the original dataset:.

Displaying both the dataset and the snapshot together reveals how snapshots work in COW fashion. They save the changes delta made and not the complete file system contents all over again. This means that snapshots take little space when making changes. Observe space usage even more by copying a file to the dataset, then creating a second snapshot:. The second snapshot contains the changes to the dataset after the copy operation.

This yields enormous space savings. ZFS provides a built-in command to compare the differences in content between two snapshots.

This is helpful with a lot of snapshots taken over time when the user wants to see how the file system has changed over time. For example, zfs diff lets a user find the latest snapshot that still contains a file deleted by accident.

Doing this for the two snapshots created in the previous section yields this output:. The first column shows the change type:. Comparing two snapshots is helpful when using the ZFS replication feature to transfer a dataset to a different host for backup purposes. A backup administrator can compare two snapshots received from the sending host and determine the actual changes in the dataset. See the Replication section for more information.

When at least one snapshot is available, roll back to it at any time. Most often this is the case when the current state of the dataset is no longer and if preferring an older version.

Scenarios such as local development tests gone wrong, botched system updates hampering the system functionality, or the need to restore deleted files or directories are all too common occurrences.

To roll back a snapshot, use zfs rollback snapshotname. If a lot of changes are present, the operation will take a long time. During that time, the dataset always remains in a consistent state, much like a database that conforms to ACID principles is performing a rollback. This is happening while the dataset is live and accessible without requiring a downtime.

Once the snapshot rolled back, the dataset has the same state as it had when the snapshot was originally taken. Rolling back to a snapshot discards all other data in that dataset not part of the snapshot. Taking a snapshot of the current state of the dataset before rolling back to a previous one is a good idea when requiring some data later.

This way, the user can roll back and forth between snapshots without losing data that is still valuable. In the first example, roll back a snapshot because of a careless rm operation that removes too much data than intended. At this point, the user notices the removal of extra files and wants them back. ZFS provides an easy way to get them back using rollbacks, when performing snapshots of important data on a regular basis.

To get the files back and start over from the last snapshot, issue the command:. The rollback operation restored the dataset to the state of the last snapshot. Rolling back to a snapshot taken much earlier with other snapshots taken afterwards is also possible. When trying to do this, ZFS will issue this warning:.

This warning means that snapshots exist between the current state of the dataset and the snapshot to which the user wants to roll back. To complete the rollback delete these snapshots.

ZFS cannot track all the changes between different states of the dataset, because snapshots are read-only.

ZFS will not delete the affected snapshots unless the user specifies -r to confirm that this is the desired action. If that is the intention, and understanding the consequences of losing all intermediate snapshots, issue the command:. The output from zfs list -t snapshot confirms the removal of the intermediate snapshots as a result of zfs rollback -r. Snapshots live in a hidden directory under the parent dataset:.

By default, these directories will not show even when executing a standard ls -a. The property named snapdir controls whether these hidden directories show up in a directory listing. Setting the property to visible allows them to appear in the output of ls and other commands that deal with directory contents. Restore individual files to a previous state by copying them from the snapshot back to the parent dataset. The directory structure below. The next example shows how to restore a file from the hidden.

Even if the the snapdir property is set to hidden, running ls. The administrator decides whether to display these directories. This is a per-dataset setting. Copying files or directories from this hidden. Trying it the other way around results in this error:. The error reminds the user that snapshots are read-only and cannot change after creation. Copying files into and removing them from snapshot directories are both disallowed because that would change the state of the dataset they represent.

Snapshots consume space based on how much the parent file system has changed since the time of the snapshot. The written property of a snapshot tracks the space the snapshot uses. To destroy snapshots and reclaim the space, use zfs destroy dataset snapshot. Important notes: 1 This tutorial assumes you have the OS you want to dual-boot with already installed on your drive, and that you already have freed up some disk space.

As always, make sure to disable Secure Boot and Fast Boot. Change it according to your setup, if needed. Run gpart show to be sure. I personally recommend rEFInd, but I won't detail how to install it here. Just gonna show you how the respective entries should look like in each case. This tutorial is not an unsafe procedure if you understand what you're doing especially in regards to selecting the correct disk where you want to install.

Anyway, do it at your own risk! Proceed installing as usual, until you reach the "Partitioning" stage. Here I share some sample entries to guide you a bit. Last edited: Feb 12, What can i do? Click to expand Hmm what if you issue this command: df -h Could you share the output? Hello patovm04,. Which advantage do you have to install FreeBSD with root on zfs when you dual boot from the same disk? Im just wondering.. Hmm it's difficult to help you if you're not following the steps to the letter for instance, you are running bash which isn't even part of FreeBSD's live install media.

Thanks to the OP for the excellent tutorial. Following the instructions I was able to get dual boot going without any problems. I did however notice a couple of things that need correction, should one follow the steps by the letter. The tmpfs mount command should be skipped. Select 'No' for the manual configuration. Once it's complete, log in with your root user and password.

This feature is only available to subscribers. Get your subscription here. Log in or Sign up. On this page 1. Requirements 2. Setup Base Installation 3. ZFS Configuration 4. Base Installation and Setup Root Password 5. Setup Networking 6.