Storage Journey, Part 2: Vitastor — fast, but young

After the latency experiences with Ceph (Part 1), we evaluated Vitastor: a distributed block storage system that architecturally resembles Ceph but is consistently optimized for low latency — designed for NVMe and fast networks. Our evaluation was not about theoretical benchmarks but a concrete question: can we noticeably reduce the latency of our Ceph setup without compromising the robustness that productive customer data demands? The short answer: yes, latency drops — but the overall picture is more nuanced once operations, support, and long-term planning come into play.

What we liked

• Latency: With NVMe backends and 10/25-GbE networks, noticeably lower write latency than in our Ceph setup.
• Leanness: Less overhead per node, simpler mental model.
• Block-focused: Exactly what VM storage needs — no feature bloat.
• Erasure Coding: Supports both replication and EC — interesting if you want to trade capacity efficiency for CPU load.

The architecture gets by with fewer moving parts, which directly reduces administration overhead. Anyone who has ever debugged an intricate Ceph crush map appreciates that. The monitoring also feels tidier: a few clear metrics instead of a jungle of internal states that first need interpreting. It is especially in write latency — the value most likely to surface as perceptible delay under VM operation — that Vitastor showed its strength.

Where we became cautious

Despite all the speed, in storage, one thing ultimately counts: trust.

• Ecosystem & Maturity: Smaller community, less field operational experience compared to Ceph or DRBD.
• Tooling & Integration: Fewer ready-made paths for monitoring, backup, and Proxmox integration.
• Risk: For productive customer data, we wanted a solution with a long track record and commercial support path.

We were particularly missing reliable long-term experience around rebuild behavior after a disk or node failure. That is exactly when something goes wrong — and when mature tooling and a community that has worked through the problem before really count. The question of trained local partners also had no satisfactory answer for us — and on a weekend outage, a reachable support contact is often the decisive factor.

Where Vitastor still fits

Vitastor is not a bad system — quite the opposite. For lab environments, development clusters, or workloads where latency is the decisive criterion and an occasional manual intervention is acceptable, it is a genuine option. Anyone who relies purely on NVMe, runs a small team, and has clear boundaries on usage gets a lot of performance per euro. The threshold to productive high availability was just not yet reached for us.

The honest assessment

Vitastor is technically impressive and very fast for the right hardware (pure NVMe, fast network). For an experiment-friendly setup, it’s a clear recommendation to explore. For our productive, highly available platform, we ultimately chose the mature, support-capable variant: DRBD/LINSTOR — Part 3.