Volume Chains

One of the new features I was really excited about when looking into the features of Proxmox 9 VE Server was the snapshot capabilities on thick provisioned LUNs with iSCSI and NFS. I think for many VMware admins, it really made traditional SAN technologies a no go in previous versions of Proxmox to not be able to create snapshots. It basically forced you over to Ceph storage and I don’t think admins wanted to bite that off along with Proxmox at the same time.

Introduction to Volume-chains

In legacy versions of Proxmox, when you take a snapshot of a VM or container, the system stores a point-in-time copy of the VM’s virtual disk state. How your snapshot that is created is implemented depends heavily on the storage backend. Let me give you a few examples:

  • ZFS, LVM-Thin, and Ceph RBD have native snapshot support

  • File-based storage (like NFS/CIFS) used QCOW2 overlays for snapshots

  • iSCSI LUNs have historically been tricky and snapshots usually required complex storage-level features or just weren’t supported at all which was mostly the latter

With Snapshots as Volume Chains feature in Proxmox VE 9, Proxmox moves away from the one-disk-per-VM approach for snapshotting on specific backend storage technologies. Instead, now, it creates a chain of volumes (base + delta files) to represent different points in time. This is similar to what we are used to with the VMware vSphere world. Each snapshot is a new delta volume linked to its parent. This is also similar to how QCOW2 works with backing files.

For iSCSI, this means Proxmox can manage snapshots without depending entirely on the SAN’s native snapshot features. This gives you snapshots even on simpler or generic iSCSI targets as I will show you below.

To exploit Volume-Chains PVE uses LVM storage. In general LVM is a lightweight software layer that sits on top of hard disks and partitions. It can be used to divide available disk space into smaller logical volumes. This particular use case places LVM on top of a large iSCSI LUN (Logical Unit Number) or a SAN (Storage Area Network) connected via Fibre Channel. This allows you to easily manage the space on the iSCSI LUN, which would otherwise be impossible because the iSCSI specification does not define a management interface for space allocation.

the confuguration option of snapshot-as-volume-chain is documented as follows:

Set this flag to enable snapshot support for virtual machines on LVM with a volume backing chain. With this setting, taking a snapshot persists the current state under the snapshot’s name and starts a new volume backed by the snapshot.

A volume based on a snapshot references its parent snapshot volume as its backing volume and records only the differences to that backing volume. Snapshot volumes are currently thick-provisioned LVM logical volumes, but the underlying block storage may provide thin provisioning.

This design avoids issues with native LVM snapshots, such as significant input/output (I/O) penalties and unexpected, dangerous behavior when running out of pre-allocated space.

Snapshots as volume chains provide vendor-agnostic support for snapshots on any storage system that supports block storage. This includes iSCSI and Fibre Channel-attached SANs.

Note that, although this feature relies on qcow2, it only uses qcow2’s ability to layer multiple volumes in a backing chain, not qcow2’s snapshot functionality. The snapshot functionality is managed by the PVE storage system.

Enabling or disabling this flag only affects newly created virtual disk volumes.

Why this matters

Until now, if you used iSCSI with Proxmox, snapshots were either not supported or they depended on your SAN and its snapshot features. That was fine for enterprise arrays with native LUN snapshots. But for those of us with home labs, this is not so great for budget-friendly setups. Most of us are using FreeNAS/TrueNAS, commodity NAS devices like Synology or something else for iSCSI targets.

Now, with Proxmox 9’s volume-chain approach:

Snapshots are now storage agnostic – As long as the iSCSI target supports standard LUNs, Proxmox can manage its own snapshot chains which is great Clone operations become easier – linked clones can be created from snapshot points without full-volume duplication Incremental backups are more efficient – Proxmox can reference only the changed blocks between snapshots Testing and rollbacks are faster – spin up a quick snapshot before testing, then revert in just a few seconds