Leader election

Many coordination problems reduce to "exactly one node must do this." A singleton scheduled job, a work coordinator, the owner of an authoritative decision — all need the cluster to agree on one member, and to agree again, quickly and consistently, when that member fails.

Nodus answers this with leader election, reached through node.Leadership (ILeadership). This page explains the strategies, the epoch counter that keeps terms distinct, the tiebreaker that resolves collisions, and the failover behavior.

The leadership handle

The handle exposes the current leader, whether it is this node, and the current term, plus an event that fires on every change.

ClusterNode? leader = node.Leadership.Current;   // who leads now (null if none yet)
bool iAmLeader      = node.Leadership.IsSelf;     // is it me?
int epoch           = node.Leadership.Epoch;      // the current election term

node.Leadership.Changed += (leader, epoch) =>
    Console.WriteLine($"leader is now {leader.NodeId} (epoch {epoch})");

The most common pattern is to gate leader-only work on IsSelf:

if (node.Leadership.IsSelf)
    await RunTheSingletonJob();

Bully-style strategies

Both shipped strategies are bully-style: a single highest-priority node wins deterministically. Every member, looking at the same set of candidates, independently computes the same winner — so the cluster converges without a multi-round vote. The strategies differ only in the priority function.

You select the strategy in config:

var config = new NodusConfig
{
    ClusterId = "orders",
    Election = new ElectionOptions
    {
        Strategy = ElectionStrategyKind.OldestNode,   // or HighestId
    },
    // ... cluster definition ...
};

Highest-id (ElectionStrategyKind.HighestId)

The reachable member with the highest NodeId (ordinal comparison) wins.

• It is fully deterministic and clock-independent — the id is a fixed string, so the ordering never depends on wall-clock time.
• The caveat: a freshly (re)joined high-id node can win while still knowledge-poor — it becomes leader the instant it appears, before it has caught up on cluster state.

Oldest-node (ElectionStrategyKind.OldestNode)

The member that has been running the longest wins — the one with the smallest non-zero StartedAtTicks (earliest process start), with NodeId as the tiebreak.

• Each node stamps StartedAtTicks (a DateTime.UtcNow.Ticks marker) once at startup. A restart produces a newer, larger value, so a just-restarted node is the youngest.
• This directly avoids the highest-id caveat: a knowledge-poor node that just (re)joined will not reclaim leadership while it is empty, because it is the youngest candidate.
• StartedAtTicks is a cross-node wall-clock value, so ordering is clock-skew sensitive. Two mitigations keep it deterministic: unstamped nodes (StartedAtTicks == 0, such as legacy peers) are treated as youngest, and exact ties fall back to the lowest NodeId.

The two strategies compare as follows:

	Highest-id	Oldest-node
Priority	Largest NodeId	Smallest non-zero StartedAtTicks
Clock-dependent?	No	Yes (skew-tolerant, id tiebreak)
Knowledge-poor node can win?	Yes	No — youngest never wins alone
Tiebreak	—	Lowest NodeId

Prefer oldest-node for stateful coordination

If losing and regaining leadership means re-establishing state — a map, a lease table, a work assignment — oldest-node is the safer default: it keeps a just-restarted, empty node from grabbing leadership at the worst possible moment. Highest-id is a good fit when leadership carries no state and pure determinism is what you want.

Switching strategies is a config change plus a restart. Because leader selection is a black box to whatever layers on top, higher-level products like Zaris are unaffected by the choice. The default is oldest-node (ElectionOptions.Strategy).

The epoch counter

Every leadership term carries an epoch — a monotonically increasing integer. When leadership changes, the epoch advances, so every node can tell a new term from an old one and reject stale leadership claims.

The epoch is the single most useful signal for correctness: a message or decision stamped with epoch N is obsolete the moment the cluster reaches epoch N+1. You read it via node.Leadership.Epoch and receive it on every Changed event.

Equal-epoch tiebreaker

Two nodes can momentarily both believe they lead at the same epoch — for example, a network partition heals and two sub-clusters that each elected a solo leader rejoin. Highest-id and oldest-node resolve most cases, but for an equal-epoch collision Nodus applies an application-supplied tiebreaker.

At equal epoch, the node whose probe reports the larger nodeCount wins; ties are then broken by the older (smaller) createdAtTicks — the established cluster beats a freshly-minted solo map. You supply your product's authoritative-state backing so the collision resolves deterministically instead of ping-ponging:

node.Leadership.SetTiebreaker(() =>
{
    // return this node's (nodeCount, createdAtTicks) for the equal-epoch comparison
    int nodeCount       = myState.KnownNodeCount;
    long createdAtTicks = myState.MapCreatedAtUtcTicks;
    return (nodeCount, createdAtTicks);
});

The engine reads only those two scalars — it never sees your underlying state model — so the tiebreaker stays a black-box, product-agnostic seam. If you do not set one, election still functions; the tiebreaker exists to resolve the specific equal-epoch, solo-vs-solo case cleanly.

What happens on leader loss

When the current leader goes away — a graceful leave or a detected failure — the survivors converge on a new leader.

The sequence is:

1. The leader's departure is detected — immediately if it left gracefully, or after the suspicion window (20 s) if it was killed and its heartbeats simply stopped.
2. It is removed from active membership, and MemberLeft fires.
3. Each survivor runs the election over the remaining reachable members. Because the strategy is deterministic, they all pick the same winner.
4. The epoch advances and Leadership.Changed fires on every node with the new leader and new epoch.

A membership join can also trigger a re-election, so the cluster keeps leadership current as its shape changes. The ClusterCoordination sample demonstrates the whole cycle: kill the leader mid-computation and watch a survivor take over at a higher epoch and continue the work.

There can be a brief leaderless gap

Between a leader failing and the survivors completing a new election, node.Leadership.Current may be null and no node reports IsSelf. Design leader-only work to tolerate this short gap — check IsSelf at the point of action rather than caching "I am the leader" indefinitely.

Next steps

• Membership — the set of candidates an election runs over.
• Health and gossip — how leader loss is detected in the first place.
• Subscribing to events — reacting to LeaderChanged in code.