6 posts tagged with "Distributed Patterns"

Most of this blog has gone deep on one thing at a time — compare-and-swap, multi-key transactions, native data structures, streams, watch, the op-log underneath. Each post answers "how does this work." This one answers a different, more practical question: when you have a problem in front of you, which one do you reach for?

That question matters because the wrong choice is rarely a crash. It's a system that works in the demo and then loses an update under load, or funnels a thousand writers through one key, or uses a fire-and-forget notification where it needed a durable log. The primitives don't stop you from using them wrong. So this is a field guide: the mental model that ties them together, a decision table, a walk down each branch with its honest fit and honest edge, and a flowchart at the end.

Keyspace notifications are one of Redis's most useful features and one of its most quietly fragile. The idea is lovely: every time a key is written, deleted, or expires, the server publishes an event on a well-known pub/sub channel, and anyone who cares can PSUBSCRIBE and react — invalidate a cache, wake a worker, refresh a projection. No polling, no GET in a loop asking "has it changed yet."

The fragility is in the fine print. Redis keyspace notifications are fire-and-forget pub/sub, emitted by whichever node processed the command, to subscribers connected to that same node. In a clustered deployment that means a subscriber only hears about writes that happened to land on the node it's attached to, and if that node fails over, the subscription — and the stream of events it was carrying — goes with it. The feature works beautifully on a single box and gets subtle fast once there's more than one.

Zaris speaks the exact same protocol — CONFIG SET notify-keyspace-events, the __keyspace@0__ and __keyevent@0__ channels, a vanilla PSUBSCRIBE from any Redis client in any language. But the plumbing underneath is not a reimplementation of Redis's notifier. It's a thin bridge onto a feature Zaris already had: a native, cluster-wide, replicated watch. That changes what the notifications are worth.

A companion post, Race-Free Coordination on Optimistic CAS, made the case that a single primitive — a store-owned version plus a version-guarded conditional write — carries a remarkable amount of coordination. It also drew a hard line: CAS is atomic on one key. Move money from account A to account B and a single compare-and-swap can't cover both; the honest advice was to model the transfer as a state machine on one key, or reach for a saga.

That line is real, but it isn't the whole story. Zaris has a genuine multi-key transaction — BeginTransactionAsync, a staged read/write set, CommitAsync — and it commits across keys and across nodes with an all-or-nothing guarantee. This post is about what that transaction actually is under the hood (a client-coordinated two-phase commit over optimistic versions), precisely what it guarantees, and the one subtlety that decides whether your cross-key invariant actually holds.

It's tempting to judge a data store by the length of its command list. Redis has INCR, SETNX, MULTI/EXEC, WATCH, Lua, SCAN, pub/sub — a verb for every occasion. The Zaris .NET client, by contrast, looks almost austere. There is no server-side atomic increment. There is no general multi-key transaction that runs arbitrary logic. There is no prefix scan. Native pub/sub exists only through the RESP front-end, not the typed .NET client.

What the .NET client gives you instead is one small, sharp coordination primitive: optimistic versioned compare-and-swap. This post is about how far that one primitive actually goes — because the honest answer is: surprisingly far. A rate limiter, a wallet that never goes negative under a thousand concurrent debits, a job queue where exactly one worker claims each job, and a feature-flag ruleset that never loses an update under concurrent admin edits. All of them, from the same primitive, with no extra server-side machinery.

Here is a fact about any partitioned key-value store that the glossy diagrams tend to skip: the node that owns your key will move. A node fails and its partitions fail over to their replicas. You scale out and partitions migrate to the new member. A preferred primary comes back and ownership fails back to it. Every one of those events means that the node your client was happily talking to a millisecond ago is, right now, the wrong node for some set of keys.

The naive client experience of that moment is ugly: a connection reset, a generic "server error," or — worse — a write that lands on a node that is no longer the active owner and quietly goes nowhere. The Zaris .NET client is built so that none of that reaches your code. An ownership change becomes a typed status, the client reroutes to the new owner, and the retry is bounded by both an attempt budget and a wall-clock deadline. In the common case your PutAsync just takes a few milliseconds longer and returns Ok. This post is about the machinery that makes that true, and — just as importantly — about the cases it deliberately does not paper over.

If your .NET services already talk to Redis, moving to Zaris doesn't have to be a rewrite. Zaris is a distributed, in-memory, replicated key/value store built natively for .NET — and as of 2.0.0 it speaks two protocols at once: its own efficient binary protocol for first-class .NET clients, and a Redis-compatible RESP2 front-end that any Redis client in any language can point at.

That gives you two honest migration paths, and they're not mutually exclusive. You can start with the zero-change RESP path to get Zaris under load today, then adopt the native client where it pays off. This post walks both, and gives you a clear rule for picking.