Streams

Zaris 2.0.0 adds native Streams — an append-only log of entries under a key, the analogue of Redis Streams. Unlike pub/sub, a stream persists its entries, so consumers can read history, resume where they left off, and coordinate work through consumer groups with per-message acknowledgement and redelivery. Streams are a first-class client API (client.Streams) and are the fifth native collection kind alongside hashes, lists, sets, and sorted sets.

The same engine backs the Redis-protocol front-end: the X* commands map straight onto it, so a Redis client in any language can produce to and consume from the same stream a native .NET service uses.

The model

• Entries and IDs. Each entry is a set of field/value pairs stamped with a monotonically increasing ID of the form <milliseconds>-<sequence> (for example 1700000000000-0). IDs never regress, even across trims and deletes.
• The stream persists while empty. Only deleting the key removes a stream; XDEL/XTRIM never do, and an emptied stream keeps its ID history (matching Redis).
• Consumer groups track a shared cursor plus a per-consumer pending-entries list (PEL) of delivered-but-unacknowledged IDs, so work can be distributed across competing consumers and redelivered if a consumer dies.

Append and read (native)

// Append an entry (auto-generated ID returned)
var id = await client.Streams.AddAsync("events", new[]
{
    ("type", "signup"),
    ("user", "42"),
});

// How many entries, and read a range
var len   = await client.Streams.LengthAsync("events");
var range = await client.Streams.RangeAsync("events", from: "-", to: "+");

// Read entries after an ID (non-blocking on the native client)
var newer = await client.Streams.ReadAsync("events", afterId: id);

Representative native operations mirror the Redis command families: append (AddAsync), length (LengthAsync), range (RangeAsync/reverse), read-after (ReadAsync), delete (DeleteAsync), trim (TrimAsync), set-last-ID (SetIdAsync), and introspection (InfoAsync). Consumer-group methods cover create/destroy/create-consumer/set-ID, group read, acknowledge, pending (summary and extended), and claim/auto-claim.

Consumer groups

A consumer group lets several workers share a stream, each getting a disjoint slice of new entries and acknowledging what it has processed:

// Create a group reading only new entries ("$")
await client.Streams.GroupCreateAsync("events", group: "workers", startId: "$");

// A worker reads its next batch as ">", processes, then acknowledges
var batch = await client.Streams.ReadGroupAsync("events", group: "workers", consumer: "w1", count: 10);
foreach (var entry in batch)
{
    Process(entry);
    await client.Streams.AckAsync("events", group: "workers", entry.Id);
}

Unacknowledged entries stay in the group's pending list; another consumer can inspect them with a pending query and take them over with claim/auto-claim (transfer after a minimum idle time) — the standard pattern for recovering work from a crashed consumer.

Blocking reads

Blocking XREAD and XREADGROUP (BLOCK <ms>, BLOCK 0 to wait indefinitely) are available over the RESP front-end: the call parks until an entry arrives or the timeout elapses, woken by a cluster-wide signal on the next XADD. On the native client there is no blocking read API — poll ReadAsync/ReadGroupAsync on your own cadence, or drive blocking reads through the RESP endpoint.

RESP command mapping

Over the RESP front-end the Stream command family maps onto the native engine:

Group	Commands
Log	XADD (auto / explicit / NOMKSTREAM, inline MAXLEN/MINID trim), XLEN, XRANGE, XREVRANGE, XREAD (+ BLOCK), XDEL, XTRIM, XSETID, XINFO STREAM
Consumer groups	XGROUP CREATE/DESTROY/CREATECONSUMER/DELCONSUMER/SETID, XREADGROUP (+ BLOCK), XACK, XPENDING (summary + extended), XCLAIM, XAUTOCLAIM, XINFO GROUPS/CONSUMERS

Validated end to end against unmodified StackExchange.Redis over a real socket (19/19), including cross-node produce/consume.

Scaling caveat: keep streams bounded

Like the other native collections, a stream is stored as one value and re-serialized on every XADD, so append cost is O(stream size). This is fine for streams of modest size, but an unbounded stream grows without limit and each append gets more expensive. Bound growth with a capped MAXLEN (or MINID) on XADD, or trim periodically with XTRIM, so the working set stays small. (A segmented log that makes append O(1) is future work.)

When to use a stream vs pub/sub

• Stream — you need entries to persist, consumers to resume from a cursor, at-least-once delivery with acknowledgement, or work distributed across a consumer group. Choose a stream for task queues and event logs.
• Pub/sub — you need lightweight, fire-and-forget fan-out to whoever is currently listening, with no history. Choose pub/sub for live notifications.

See also

• Pub/sub — fire-and-forget messaging without persistence.
• Data structures — the other four native collection types and the whole-value model.
• Redis protocol (RESP) — driving streams (including blocking reads) from any language.