Partitioning and replication

A single node can hold only so much data and serve only so many requests. Clustron Zaris scales past that limit by splitting the keyspace into partitions, spreading partitions across nodes, and replicating each partition so that losing a node does not lose data.

This page explains how a key reaches a node, how the replication factor determines availability under node loss, how writes and reads are routed, and what happens during a failure.

From a key to a node

Routing is deterministic and happens in three steps. Because it is deterministic, any client computes the same route for the same key without coordinating.

1. Key to segment. Every key is hashed into one of a fixed number of segments (hash slots). Segments never move relative to keys, so a given key always lands in the same segment.
2. Segment to partition. Segments are grouped into partitions. A partition is the unit of ownership and replication.
3. Partition to node. Each partition is owned by exactly one node at a time — its primary — plus zero or more replicas on other nodes.

Segments, not individual keys, are the thing mapped to partitions. That is why rebalancing moves segments between nodes rather than rehashing keys: keys come along with their segment untouched.

Partition count

You do not usually set an absolute partition count. You set partitions per node, which defaults to 1, and Clustron Zaris derives the total as partitions per node × live nodes. The total is recalculated on every membership change, so it tracks the shape of the cluster rather than freezing at creation time.

• Production topology keeps the default of one partition per node.
• Local testing raises partitions per node (for example, 4) so you can observe partition-aware behavior — segment routing and scale-out rebalancing — on a single machine without running multiple real machines.

Each machine runs partitions per node × replication factor node processes, so both dials affect how many processes come up per machine.

Partition code words

Partitions are identified by a stable code word — Amber, Onyx, Jade, Opal, Topaz, and so on — rather than a bare number. The word is derived from the partition's stable, monotonic serial, so the same partition keeps the same word in every process and across restarts and ownership changes. The word list is deliberately non-geographic (gems, metals, trees, birds, weather, stars, and the like) and is collision-free within a 500-word cycle.

This matters when you operate a cluster. In the console and in diagnostics, you can follow Amber as it moves from one node to another during a failover or rebalance, instead of correlating anonymous partition indexes across snapshots.

tip

The Nodes page shows exactly which partitions each node holds — which it owns as primary and which it keeps as a replica — by code word.

The replication factor

The replication factor (RF) is the number of copies of each partition that exist. It is the main dial for availability under node loss. You set it when you create a store.

• RF = 1 — one copy. Fast and compact, but a node loss means data loss. Use it only for caches you can rebuild.
• RF = 2 — a primary plus one replica. This is the recommended value for production, and it survives one machine failure. (It is not the configuration default: the config property defaults to 1, and New-ZrStore requires -ReplicationFactor explicitly.)
• RF = 3 — a primary plus two replicas. Survives two simultaneous machine failures.

Copies of the same partition are always placed on different machines, so losing one machine can never take out both a primary and its replica. Clustron Zaris enforces this: for production you need at least as many machines as the replication factor, and it warns (single machine) or refuses (scale-in) when that would be violated.

A worked example

Consider replication factor 2, one partition per server (PartitionsPerNode = 1), and two servers — machine-a and machine-b. A server is a machine running one manager; the nodes are the store processes it runs. Each server runs PartitionsPerNode × ReplicationFactor = 2 node processes, so there are four nodes in total, holding the two partitions' four copies.

Every partition has one primary, which serves reads and writes, and one replica, which is a hot standby; writes land on the primary and stream to the replica. Here Amber's primary is a node on machine-a and its replica a node on machine-b, while Onyx's primary is a node on machine-b and its replica a node on machine-a. Each server therefore runs one primary and one replica — a node is primary for the partition it leads and replica for another. Because a partition's two copies always sit on different servers, losing a server (or a single node) takes out at most one copy of any partition, and its replica is promoted to keep serving.

Two rules follow from this, and they govern how a store grows:

• Nodes per server = PartitionsPerNode × ReplicationFactor. The server is the machine; the node processes are what actually hold partition copies. This number is the same on every server.
• Adding a server adds partitions. PartitionsPerNode is uniform across servers, so the total partition count is PartitionsPerNode × servers. With PartitionsPerNode = 1, adding a third server introduces a third partition. See Elastic scaling.

Reads, writes, and consistency

Writes and reads have different default routing, which you can tune per store.

Writes always go to the primary. When the write is durable depends on the store's replication mode:

Replication mode	When the primary acknowledges	Trade-off
Async (default)	Immediately, after writing locally; replication happens in the background	Lowest latency, but a replica can lag the primary, so a failover may lose the most recent writes
Sync	After replicas acknowledge, per the write-quorum policy	Higher latency, stronger durability under node loss

Sync mode uses a write-quorum policy: All (default — every replica must acknowledge), Majority (floor(RF/2)+1 replicas), or BestEffort (replica failures are logged but do not fail the write).

Reads go to the primary by default. A store can opt in to replica reads (off by default) for read-heavy workloads that tolerate slightly staler data.

For the full model, see Consistency.

Failure and recovery

Losing one copy

When a primary is lost — a node crash or a machine reboot — its replica takes over and the cluster rebuilds a fresh replica to restore the configured replication factor.

The promoted replica keeps serving while the cluster rebuilds a new replica on a healthy machine. Until that rebuild completes, any partition below its replica target is under-replicated. Apart from a brief re-route — the client refreshes its routing map on Moved / OwnershipChanging — a single-copy loss is transparent.

Losing every copy of a partition

Zaris is an in-memory store: a partition's data lives only in the RAM of the nodes that hold its copies. If every member of a partition group goes down at once — the primary and all of its replicas — that partition has no surviving copy, and its data is lost. A replication factor makes this correlated failure rarer, not impossible: raising RF and spreading copies across more machines lowers the odds, but nothing restores in-memory data once the last copy is gone.

When it happens, the cluster does not immediately rewrite the topology, because most total outages are transient — a rack blip, or a rolling restart that briefly takes a whole group down. Instead the partition is marked unavailable and a grace window starts (runtime.partitionGroupEvictionWindowSeconds, default 30 seconds, configurable):

• If any member returns within the window, the countdown is cancelled and the partition recovers on that node — no topology change.
• If the window expires with no member back, the cluster evicts the gone partition group: the leader regenerates the partition map from the live members only, folding the dead partition's segments onto surviving nodes. That keyspace now has a fresh, empty home, so new writes for those keys succeed again on a live node. The original data does not come back.

Set the window in the store configuration's runtime section — raise it to wait longer for a flapping group, lower it to restore write availability sooner:

{ "runtime": { "partitionGroupEvictionWindowSeconds": 60 } }

warning

Losing every copy of a partition is data loss for that partition's keys — there is no disk copy to restore from. Choose a replication factor and spread machines across fault domains so a single failure cannot take out a whole partition group, keep your system of record durable elsewhere, and use TTLs so a folded partition repopulates naturally. See High availability and disaster recovery.

The grace window trades recovery patience against how soon the keyspace becomes writable again: a longer window waits longer for a flapping group to return before folding it; a shorter one restores write availability for those keys sooner but gives up on the original data earlier.

warning

With the default Async replication, a replica can trail its primary. If the primary fails before the trailing writes replicate, those writes are lost on promotion. Use Sync replication when a failover must not lose recent writes.

Next steps

• Elastic scaling
• Cluster architecture
• Consistency