CetraFS: A VMFS Principle for Proxmox, Read in the Code

Within a single week, two projects appeared with the same goal: bring VMware's VMFS principle to Proxmox, meaning several nodes that share one disk and keep the VM images directly on it. On 24 October Virtualization Howto presented both in one article: MeshXFS, which we covered on 21 October, and the new CetraFS. The MeshXFS post was about a locking layer on XFS; this one is about CetraFS, a format of its own with SCSI locks on the LUN. We read it in its repository and check whether it fits our storage.

CetraFS is a filesystem for shared block storage: several Linux or Proxmox nodes mount the same iSCSI or Fibre Channel LUN at the same time and keep VM images directly on it. The nodes coordinate through atomic SCSI commands (COMPARE AND WRITE) on the drive itself, with no network protocol in the data path. The project presents itself as an open alternative to VMFS and calls itself beta, meaning before version 1.0.

Updated on 5 November 2026. Since the first version of 25 October the project has published five releases (0.8.2 to 0.9.3, the latest on 2 November). Version number, disk format, test scope and the maturity label are brought in line with today's status page. The measurements on our cluster are unchanged from 25 October.

At a glance

Figures from the project are vendor claims that we have not re-measured. The last row is our own measurement.

How CetraFS coordinates the nodes

CetraFS relies on a single mechanism. Every contested decision goes through SCSI COMPARE AND WRITE, an atomic compare-and-swap command on a 512-byte record directly on the drive: which zone a node may claim, who holds a file open, who owns an entry in the node registry. The README compares this with the ATS principle of VMFS ("the same idea as VMFS's ATS"). A distributed lock manager, Corosync and a second network are not needed in the data path. Node liveness runs through a heartbeat every 15 seconds with a stale window after it.

That window shows that the author built against real hardware. The comment in include/cetrafs.h records that the earlier setting of four beats, meaning 60 seconds, was wrong for real multipath storage; the default is now eight beats, meaning 120 seconds. The reason is in the README and the header: when all paths of a multipath setup fail, dm-multipath holds I/O in a queue for no_path_retry times the polling interval instead of returning an error at once, which is 90 seconds for the default setting of an HPE 3PAR. During that time the node cannot renew its heartbeat either. With a shorter stale window the neighbours would declare the node dead and take over its zones while its writes still sit in the queue; those writes would then land on foreign territory. 120 seconds survives the 90. The value can be set between 15 and 240 seconds, but not below three heartbeat beats.

What the repo says besides the idea

According to the GitHub API, the public repo cetrafs/cetrafs was created on 22 October 2026 at 12:24 UTC. Release 0.8.1 with the Debian packages (cetrafs-dkms, cetrafs-tools) followed the same day at 17:33 UTC. Then came 0.8.2 on 26 October, 0.9 and 0.9.1 on 29 October, 0.9.2 on 1 November and 0.9.3 on 2 November. The author is Massimo Cetra (GitHub account ctrix, registered in 2011, blog ctrix.it). The release notes now name outside contributors: most of 0.9.2 comes from a review of the whole code base by Matteo Croce, according to the release notes. The Proxmox plugin pve-storage-cetrafs lives in a repo of its own. A repo of this age is a snapshot. According to the account at Virtualization Howto, the author says the filesystem's architecture predates today's AI coding tools and that he used AI assistance for later development, debugging and tests. We could not fetch the Reddit post itself (bot wall), so we cite only that account. For MeshXFS, by contrast, the author describes an AI agent loop as the origin of the code.

The licence sits in the LICENSE file even though GitHub shows "NOASSERTION" at the top, and it is split in two. The kernel module under kmod/ is GPL-2.0-or-later with no commercial variant; "none can be offered" is the literal wording. The tools under userspace/ are AGPL-3.0-or-later or a commercial licence, which the project offers to anyone for whom the AGPL's source-disclosure duty does not fit. Which variant applies to your use is for a lawyer to decide; this post gives no legal advice.

In two places the project itself says where its diligence ends. The README states that packaging skips make check during the package build: "a built package is not a tested one". The plugin readme says the CI build produces the package in the nocheck profile and does not run the tests.

How mature the project is

The status page (as of 2 November 2026) lists these verification tiers: 243 suites with 2,257 assertions on loop devices, a crash-consistency rig (dm-log-writes, dm-flakey, power cut without a prior flush), fuzzing of the mount and fsck parsers, soak tests and a multi-node suite against a real shared LUN. Its first fully green run was on a three-node arm64 Proxmox cluster on an HPE 3PAR over 32 Gb Fibre Channel multipath; a second, with version 0.7.4, ran on a three-node amd64 cluster on a commodity iSCSI LUN: 27 scenarios, passed three times in a row. The page itself limits the claim: there is test evidence but not yet years of production use, and some race scenarios have run only a handful of times on real hardware.

The loop-device tests only simulate multi-node cases. According to the README, only the ctest/ suite exercises the paths between nodes on a real LUN; the file docs/PERFORMANCE.md says of make check: "proves correctness; it cannot prove speed". The throughput figures on the project page are vendor claims: 613 to 710 MB/s O_DIRECT write throughput on a 3PAR LUN, about 2.3 times GFS2 on the same array according to the project. We have not re-measured them.

The disk format is the biggest warning sign. Version 0.8.1 carried format 0x0008, the current 0.9.3 carries 0x0009. From 0.9.2 on, CetraFS no longer mounts the old format 0x0008: such a volume must first be grown with 0.9.1 or reformatted. There is no in-place migration; a format change means moving the data off, reformatting and restoring. On maturity the project has moved on: release 0.8.1 still called itself alpha, while the project page and the status page now say beta.

Comparison with MeshXFS

Both projects fill the same gap from opposite directions. MeshXFS extends XFS with a locking layer that runs over TCP, over SCSI commands or over DRBD state. That makes it the only one of the two with a documented path for pairs on DRBD in dual-primary mode, built for DRBD 8.4 and at most two nodes. CetraFS builds a format of its own for VM storage only and knows only SCSI COMPARE AND WRITE on the LUN. The comparable NVMe command is not wired up according to the status page, so an NVMe device works for a single node only. For multi-node production use CetraFS requires an external cluster manager with host fencing (STONITH) and does not see that as its job. Maturity of both: pre-1.0. MeshXFS publishes its defect list as a file (data/defects.json); CetraFS relies on the status page, the reformat warning and the advice to keep independent backups. MeshXFS also withdrew v0.90.131 after publication because a bug in the on-disk metadata had been found; v0.90.132 of 21 October is the corrected version with support for Proxmox VE 9.2 (release page).

Does it fit our clusters

Our own measurement of 25 October 2026, read-only from the admin relay against the LINSTOR controller: seven storage nodes online, 97 resources, storage pools of type LVM Thin plus diskless attachments. The picture on the nodes is the same as on 21 October: no SAN target, no iSCSI presentation to several nodes, replication through LINSTOR with DRBD using protocol C. A shared SCSI LUN on which the CAW probe could even start does not exist here.

Then there is the version question we measured on 25 October: all seven nodes run kernel 7.0.2-6-pve on pve-manager 9.2.21. CetraFS lists 7.0.14 and "Proxmox VE 7.0.14-*-pve" as supported. The status page says other kernels are unverified and may fail to build or load. And the fencing component the project requires for multi-node use does not exist here in that form: the LINSTOR tiebreaker settles quorum questions for replication and does not power off hosts.

Our verdict matches the one for MeshXFS: watch it, do not build it into a running cluster. The requirements the project names fit a real SAN project. Anyone without one cannot even mount the filesystem on two nodes for a trial. Three points decide whether a test is worth it: whether a build is confirmed for your kernel family, whether multi-node evidence from commodity iSCSI targets goes beyond the single run with 0.7.4, and whether Proxmox closes the gap itself. Virtualization Howto raises that last question on 24 October; it is a community voice, not a statement by Proxmox. Anyone with a SAN measures the CAW probe from the project documentation on the array first, before any node sees the module. For help sorting storage options for your environment, see IT operations and infrastructure; more storage posts are in the hub Storage, backup and replication.

Further reading

• CetraFS repo, read on 5 Nov 2026: LICENSE with the GPL/AGPL-commercial split, include/cetrafs.h with the heartbeat and stale-window values, docs/TESTING.md with the layered test plan
• Release 0.8.1, published 22 Oct 2026 17:33 UTC, with install notes and the alpha notice
• Project page and status page with platform matrix, verification list and exclusion list
• Proxmox plugin repo (pve-storage-cetrafs)
• Reddit r/ProxmoxVE: post by the project author, not retrievable for us (bot wall), used only through the account at Virtualization Howto
• Virtualization Howto, article of 24 Oct 2026 comparing both projects and recounting the Reddit discussion
• MeshXFS releases: v0.90.132 of 21 Oct 2026 with Proxmox VE 9.2 support, v0.90.131 withdrawn