Cloudflare Meerkat: Building Consensus Without a Leader

Cloudflare Meerkat: Building Consensus Without a Leader

Cloudflare just published details on Meerkat, their experimental consensus service built on top of QuePaxa, a 2023 algorithm from EPFL researchers. On the surface, this looks like an internal infrastructure story. But the implications run deeper than that, especially if you build or depend on systems that need strong consistency at a global scale.

The Problem with Raft in the Wild

Most engineers who have touched distributed systems have at least heard of Raft. It is widely used, well-documented, and relatively easy to reason about. But Raft has a structural weakness: it depends on a single leader to accept writes. If that leader crashes, or if the network degrades enough that other nodes cannot hear from it, the system stalls until a timeout triggers a new election.

In a well-controlled data center environment, timeouts are manageable. You tune them, test them, and mostly forget about them. But Cloudflare operates across 330+ data centers spread over the actual Internet, where latency spikes, link cuts, and partial failures are not edge cases. They are Tuesday. Cloudflare explicitly notes they have experienced multiple real incidents caused by unavailable leaders in Raft-style systems. That is not a theoretical concern. That is production pain.

This is a pattern I find genuinely interesting because it reveals how algorithms that work well in academic settings or controlled deployments can develop real friction at global scale. The algorithm was not wrong. The operating environment just exposed assumptions baked into its design.

What QuePaxa Does Differently

QuePaxa removes the concept of a fixed leader entirely. Every replica can propose writes at all times. Progress is never gated on a timeout. If a majority of replicas are alive and can communicate with each other, the system remains available for both reads and writes.

This is a meaningful shift. Instead of one node being the bottleneck and single point of failure for writes, any node can participate. The consensus algorithm ensures that for each slot in the log, only one value gets decided, even when multiple replicas are racing to fill that slot with different proposals. If a majority already decided a value for a given slot, any late proposal gets rejected and the proposing replica is forced to catch up.

The linearizability guarantee follows naturally from this log structure. Even reads go through the log, which prevents stale reads from replicas that might have missed recent decisions. If you read from a replica that is slightly behind, the act of submitting that read as a log event forces it to reconcile with the rest of the cluster before returning a result.

For developers building on top of systems like this, the practical upside is significant. You get to reason about your data like it lives on a single machine. No need to mentally model race conditions between replicas or wonder whether a replica serving your read has seen the last write. That cognitive load reduction is underrated.

What This Means for the Industry

Cloudflare says this will be the first industrial deployment of QuePaxa at global scale. That is worth paying attention to. Academic algorithms often sit on shelves for years before someone builds production infrastructure on them. When a company at Cloudflare’s scale commits to deploying something new, it tends to accelerate adoption and surface real-world tradeoffs that academic papers cannot fully anticipate.

If Meerkat works well in production, it gives other infrastructure teams a concrete reference point. The argument for tolerating Raft’s leader election issues gets weaker when a viable leaderless alternative exists and has been proven at scale. Teams managing globally distributed systems, whether in fintech, gaming, or anywhere with strict consistency requirements, will be watching.

I also think this points to a broader trend worth tracking in the distributed-systems space: the gap between what consensus algorithms assume about networks and what production networks actually look like is closing. More teams are willing to invest in algorithms tailored for adversarial network conditions rather than patching around the limitations of older ones.

Meerkat is still internal and experimental. Right now it is scoped to small pieces of control-plane state like database leadership assignments and resource placement data. But the architecture, where applications like key-value stores and leasing systems sit on top of the consensus log, is designed to be extensible. The same foundation could support broader use cases over time.

For anyone building in the cloud infrastructure space, the pattern Cloudflare is establishing here, decouple the consensus mechanism from the application logic, layer linearizable reads through the log, and eliminate single points of failure from write paths, is a template worth studying carefully.

The real question is not whether leaderless consensus works in theory, but whether the operational complexity of running something newer and less battle-tested than Raft is worth the availability gains, and that answer will only come from production data that does not exist yet.

Read Next