Skip to main content

Sending messages

This how-to covers the fire-and-forget side of the messaging plane: sending a payload to one node or to every member, with both the typed helpers and full control over the wire message. For the ask-and-wait pattern, see Request/reply.

All of it hangs off node.Messaging (IMessaging). For the transport and the two API levels behind it, see Messaging.

Point-to-point (typed)​

To send a serializable payload to one named node, call SendAsync<T> with the target node id.

public sealed class WorkItem
{
    public int Id { get; set; }
    public long From { get; set; }
    public long To { get; set; }
}

await node.Messaging.SendAsync("node-2", new WorkItem { Id = 42, From = 1000, To = 2000 });

On the receiving node, register a handler with On<T>. The handler receives the payload and the sender's node id.

node.Messaging.On<WorkItem>((item, from) =>
{
    Console.WriteLine($"received work {item.Id} [{item.From}..{item.To}] from {from}");
    return Task.CompletedTask;
});

Broadcast (typed)​

To send to every member at once, call BroadcastAsync<T>.

await node.Messaging.BroadcastAsync(new CacheInvalidation { Key = "user:42" });

Every member that registered a matching handler receives it. Broadcast is a fan-out over the direct connections Nodus already maintains — a convenience over looping SendAsync yourself. It targets members only.

The one-typed-handler rule​

On<T> registers into a single typed-message slot. Every call maps to the same underlying client-message channel, so a later On<T> replaces the earlier one. A node has one typed handler at a time.

The idiomatic solution is a single envelope type with a discriminator field, branching inside one handler.

public sealed class CoordinationMessage
{
    public string Kind { get; set; } = "";     // "assign" | "result"
    public int ChunkId { get; set; }
    public long From { get; set; }
    public long To { get; set; }
    public string WorkerId { get; set; } = "";
    public long Primes { get; set; }
}

node.Messaging.On<CoordinationMessage>(async (msg, from) =>
{
    switch (msg.Kind)
    {
        case "assign":
            var primes = CountPrimes(msg.From, msg.To);
            await node.Messaging.SendAsync(from, new CoordinationMessage
            {
                Kind = "result", ChunkId = msg.ChunkId, WorkerId = node.NodeId, Primes = primes,
            });
            break;

        case "result" when node.Leadership.IsSelf:
            Aggregate(msg.WorkerId, msg.Primes);
            break;
    }
});

This is the pattern from the ClusterCoordination sample: the leader sends "assign", workers reply "result", and one envelope type carries both.

When you need genuinely separate handlers

If a discriminator envelope does not fit — you want independent handlers keyed by distinct message types — drop to low-level IMessageHandler instances registered with AddHandler, which the router dispatches by the Type (message-type) string. This is shown below and in Request/reply.

Raw wire control​

When you need the message's type, correlation id, or the priority plane, build a Message explicitly. CreateMessage<T> stamps it with this node's id and the configured serializer, and assigns a fresh correlation id.

Message msg = node.Messaging.CreateMessage("orders.reserve", new ReserveRequest { Sku = "ABC", Qty = 3 });

// queued send (bulk traffic)
await node.Messaging.SendAsync("node-2", msg);

// priority plane — bypasses the outbound queue, for small latency-sensitive control messages
await node.Messaging.SendImmediateAsync("node-2", msg);

To receive raw messages by type, register an IMessageHandler. Its Type property is the MessageType the router dispatches it for.

public sealed class ReserveHandler : IMessageHandler
{
    public string Type => "orders.reserve";
    public MessageDispatchMode DispatchMode => MessageDispatchMode.Inline;

    public Task HandleAsync(Message message)
    {
        Console.WriteLine($"reserve request from {message.SenderId}, corr={message.CorrelationId}");
        // deserialize message.Payload as needed
        return Task.CompletedTask;
    }
}

await node.Messaging.AddHandler(new ReserveHandler());

MessageDispatchMode tells the router how to run the handler. Choose Inline for quick handlers and DataPath for anything that does real work, so a slow handler never stalls the socket.

ModeUse for
InlineFast, non-blocking handlers — awaited on the receive loop.
FastPathCritical cluster-control messages — the priority fast path.
DataPathHeavy work — offloaded to the data dispatcher so the receive loop keeps draining.

Rules and gotchas​

The messaging plane has a few behaviors worth knowing before you rely on it.

  • Only members send. SendAsync<T>, SendAsync(nodeId, message), and BroadcastAsync throw NotSupportedException from a node without the member role.

  • Members excludes self. To reach the whole cluster including yourself by hand, append node.NodeId when you enumerate node.Cluster.Members.

  • Handle in-flight failures. A send to a peer that has just left can throw. Wrap sends to specific nodes in try/catch when a peer may be departing, as the coordination sample does:

    try
    {
        await node.Messaging.SendAsync(target, work);
    }
    catch (Exception ex)
    {
        Console.WriteLine($"send to {target} failed: {ex.Message}");
    }

  • Immediate versus queued. Use SendImmediateAsync only for small, urgent control messages. Bulk payloads belong on the queued path (SendAsync).

Next steps​