Skip to main content

Querying health and membership

Reacting to change (subscribing to events) is half the job; the other half is asking the current state directly — who is in the cluster right now, who can I reach, when did I last hear from that peer. This how-to covers the read side, through node.Cluster (IClusterMembership) and node.Health (IClusterHealth).

For the concepts behind alive versus reachable and the failure detector, see Health and gossip.

Who is in the cluster

node.Cluster.Self is this node; node.Cluster.Members is the current active membership, and it excludes Self.

ClusterNode self = node.Cluster.Self;                       // this node
IReadOnlyList<ClusterNode> others = node.Cluster.Members; // active members, EXCLUDING self

Console.WriteLine($"I am {self.NodeId}; {others.Count} other members");

To enumerate the whole cluster, append this node.

var everyone = node.Cluster.Members
.Select(m => m.NodeId)
.Append(node.NodeId)
.Distinct()
.OrderBy(id => id)
.ToList();

To look up a specific member or filter by role, use Find and MembersInRole.

ClusterNode? n2 = node.Cluster.Find("node-2");                 // null if not an active member
var collectors = node.Cluster.MembersInRole("metrics-collector");

Alive versus reachable

Nodus distinguishes two liveness facts, and you use both. A peer is alive when it has sent you traffic, and reachable when you can open an outbound connection to it.

bool live      = node.Cluster.IsAlive("node-2");      // we have RECEIVED from node-2
bool reachable = node.Cluster.IsReachable("node-2"); // we can DIAL node-2

IReadOnlyList<string> dialable = node.Cluster.ReachableNodeIds; // all currently reachable ids
MemberTrue whenCost
IsAlive(id)The peer has sent us traffic (it is live).O(1)
IsReachable(id)An outbound connection to the peer succeeded within the reachability window.O(1), never opens a socket
ReachableNodeIdsEnumerates everything currently dialable, independent of liveness.O(n)

A peer can be reachable before it is alive — you can connect to it before it has said anything. Prefer IsAlive when choosing a target you expect to respond; use IsReachable or ReachableNodeIds when you need to know what you could contact, such as a would-be solo leader checking whether any peer exists.

When you last heard from a peer

LastHeartbeatUtc returns the UTC time of the last heartbeat from a peer, or null for a peer you have never heard from and for self (a node does not heartbeat itself). It powers per-peer heartbeat-freshness in diagnostics.

DateTime? last = node.Cluster.LastHeartbeatUtc("node-2");
if (last is { } t)
{
var age = DateTime.UtcNow - t;
Console.WriteLine($"last heard from node-2 {age.TotalSeconds:F0}s ago");
if (age > TimeSpan.FromSeconds(15))
Console.WriteLine("node-2 is going quiet — may be evicted soon");
}
else
{
Console.WriteLine("never heard from node-2 (or it is self)");
}

The health snapshot

node.Health rolls liveness into a small snapshot: Local for this node, Peers for each known peer, and Metrics for cluster-level counters.

NodeHealthStatus local = node.Health.Local;
Console.WriteLine($"{local.NodeId}: alive={local.IsAlive}, lastSeen={local.LastSeenUtc:HH:mm:ss}");

foreach (NodeHealthStatus p in node.Health.Peers)
Console.WriteLine($" {p.NodeId}: {(p.IsAlive ? "alive" : "down")}");

ClusterMetrics m = node.Health.Metrics;
Console.WriteLine($"active peers: {m.ActivePeers}");

The types and their fields:

TypeFields
NodeHealthStatusNodeId, IsAlive, LastSeenUtc, Roles
ClusterMetricsActivePeers, TotalMessagesSent, AvgLatencyMs
What node.Health reports today

In this release the health adapter is a minimal snapshot sourced from live membership. Local.IsAlive is true with LastSeenUtc set to now, peer liveness comes from membership, and ClusterMetrics populates only ActivePeersTotalMessagesSent and AvgLatencyMs stay at 0. For accurate per-peer liveness, prefer node.Cluster.IsAlive and node.Cluster.LastHeartbeatUtc.

Reacting to topology changes

Combine the read side with the events so your view updates as the cluster changes. A common pattern keeps a leader-only worker in step with membership.

// react to changes
node.Cluster.MemberJoined += _ => RecomputeAssignments();
node.Cluster.MemberLeft += _ => RecomputeAssignments();
node.Leadership.Changed += (_, _) => RecomputeAssignments();

void RecomputeAssignments()
{
if (!node.Leadership.IsSelf) return; // only the leader assigns work

var liveMembers = node.Cluster.Members
.Where(m => node.Cluster.IsAlive(m.NodeId))
.Select(m => m.NodeId)
.Append(node.NodeId) // the leader works too
.Distinct()
.OrderBy(id => id)
.ToList();

Assign(liveMembers);
}

This is the shape of the ClusterCoordination sample's leader loop: on every topology change, the current leader re-derives the live member set and reassigns work, so a join is absorbed and a departure is covered without special-casing failover.

Check liveness at the point of use

Membership can change between the moment you read Members and the moment you send. For work that must reach a live node, filter with IsAlive at the point of dispatch and wrap the send in a try/catch — a peer can leave mid-operation.

Registering a peer you know about

When a peer exists but is not yet admitted — for example in a dynamic peer set — register it with RegisterKnownPeer so it can be dialed early.

node.Cluster.RegisterKnownPeer(new ClusterNode
{
NodeId = "node-4", Host = "10.0.0.4", Port = 7604, ClusterId = "orders",
});

Next steps