Big Values, Small Pauses: Large-Object Chunking in Zaris

There's a number every .NET performance engineer eventually memorizes: 85,000 bytes. Any single allocation at or above roughly that size doesn't go on the normal small-object heap — it goes on the Large Object Heap (LOH). The LOH is a different animal. It's only collected during a gen-2 (full) garbage collection, and by default it isn't compacted. That combination is fine for a big buffer you allocate once and keep. It is not fine for a low-latency in-memory store that repeatedly writes and overwrites large values.

Picture the workload: big serialized blobs, large JSON documents, images, protocol payloads — stored, updated, replaced, over and over. Every one of those large writes lands a fresh object on the LOH. Because the LOH isn't compacted, freed slots don't merge back into contiguous space; they fragment. And because reclaiming any of it means a full gen-2 collection, the pauses get longer and land more often exactly as the working set grows. For a store whose entire pitch is low latency, that's the wrong kind of surprise.

Zaris ships an opt-in feature to sidestep the LOH entirely for these values. It's off by default, and this post is about what it does, how it works, and — just as important — when you should and shouldn't turn it on.

The idea: never allocate the big object

When large-object chunking is enabled, Zaris stops storing a big plain value as one contiguous byte[]. Instead it splits the value into a jagged array of smaller pieces — a byte[][] where every chunk sits comfortably below the ~85KB LOH threshold.

The consequence is the whole point: no single piece is large enough to be a large object, so none of them land on the LOH. The value lives entirely on the normal small-object heap, which is compacted and is collected in cheap gen-0/gen-1 passes far more often than gen-2. You've taken a value that used to be a recurring source of LOH fragmentation and gen-2 pressure and turned it into ordinary, well-behaved heap traffic.

without chunking:  value ──▶ one 400 KB byte[]  ──▶ Large Object Heap (gen-2 only, no compaction)

with chunking:     value ──▶ byte[][] { 64KB, 64KB, 64KB, ... }  ──▶ small-object heap (gen-0/1, compacted)

Reads pay almost nothing for this. Reassembly on read is a single pass: Zaris stitches the pieces back into the contiguous value the caller expects, so the calling code sees exactly the value it stored. The chunked layout is an internal storage detail, not something that leaks into your API.

Memory reporting stays honest

A fair worry: if a value is secretly stored as a dozen sub-arrays plus the bookkeeping to track them, does memory reporting suddenly become confusing? No. Zaris continues to report a value's logical size — what you actually stored — not the physical chunked layout. A 400KB value reads as a 400KB value in your accounting. The chunking is a garbage-collection tactic, not an accounting change, and we deliberately kept it invisible to the numbers you reason about.

Turning it on

Two configuration knobs control the feature:

{
  "memory": {
    "chunking": {
      "enabled": true
    },
    "lohThresholdBytes": 85000
  }
}

• memory.chunking.enabled switches the behavior on. Left at its default, Zaris stores values the ordinary contiguous way.
• memory.lohThresholdBytes sets the size at which a value gets chunked. Set it at or below the runtime's LOH boundary so that anything big enough to become a large object is split before it can.

In native-client terms, nothing about your calls changes — you write a big value the same way you always have, and chunking (if enabled) happens underneath:

// Same call whether or not chunking is enabled — the split happens in the store.
byte[] bigPayload = SerializeSnapshot(order);   // e.g. ~400 KB
await client.PutAsync("snapshot:order:42", bigPayload);

// Read reassembles the pieces in a single pass; caller sees the whole value.
ReadResult<byte[]> result = await client.GetAsync<byte[]>("snapshot:order:42");

An honest caveat about scope

This is a targeted tool, and it's worth being precise about its edges. Chunking is aimed at large plain byte values. A jagged byte[][] survives the storage path cleanly, which is exactly why plain large byte payloads are the sweet spot. It is not a general-purpose compression scheme — the bytes aren't made smaller, they're just laid out differently — and it is not a change to how custom typed objects are handled. Think of it as one specific instrument for one specific problem: GC pressure from hot, large, plain values. Reach for it when that's your problem, and leave it in the drawer otherwise.

Why it's off by default

If it helps, why not always on? Because most values in most stores are small — well under the LOH threshold — and for those values chunking does nothing useful. What it does add is real: splitting on write, tracking the pieces, and stitching them back on read all cost a little CPU and a little bookkeeping per value. That overhead is trivially worth paying for a genuinely large value you overwrite constantly, and pure waste for a 200-byte session token.

So Zaris makes it a deliberate choice rather than a silent default. Turning it on is a statement: "this deployment has hot, large values, and I'd rather trade a bit of per-value bookkeeping for fewer gen-2 pauses and no LOH fragmentation." That's a great trade for a cache of large serialized aggregates or an image store. It's a bad trade for a store full of small keys, which is why the default assumes the common case.

When to enable it

Reach for large-object chunking when both of these are true:

• Your values are genuinely large — routinely at or above the ~85KB LOH threshold (big blobs, large JSON, images, serialized snapshots).
• They're frequently updated — hot values that get overwritten, not written-once-and-forgotten. This is where LOH fragmentation and gen-2 pressure actually accumulate.

If you're seeing lengthening gen-2 pauses and rising LOH size on a workload that matches that description, flip memory.chunking.enabled on and set memory.lohThresholdBytes to keep values off the LOH. If your values are small, leave it off — you'd only be paying bookkeeping for a problem you don't have. Like most good performance knobs, its value is entirely in knowing when not to turn it.