One Atomic Hop: Distributed Rate Limiting Without the Read-Modify-Write Race

Every API eventually needs to say no. Too many password attempts, too many writes from one tenant, too many calls against a metered upstream — at some point you have to count requests inside a time window and reject the ones over the line. On a single process that's a dictionary and a lock. The moment you run two instances behind a load balancer, it stops being that.

The naive move is to keep the limiter in each instance's memory. It's fast and it's wrong: with four instances and a limit of 100 requests a minute, a client that spreads its traffic evenly gets 400 through. You haven't built a 100/min limiter, you've built four of them.

So the state has to be shared. And the instant it's shared, the tempting shape — read the current count, add one, write it back — becomes a classic read-modify-write race. Two instances read 99, both compute 100, both write 100, both allow the request. The count is now wrong and the limit leaked. Redis users reach for INCR or a Lua script to make the step atomic. This post shows the same guarantee on Clustron Zaris with its native counter API: a server-side atomic increment, with the window's TTL applied in the same call, so the whole decision is one round trip and there is no window where two callers can disagree.

Why the counter has to be atomic, and server-side

A fixed-window limiter is almost embarrassingly simple once the increment is atomic. For a given subject (a user id, an API key, a tenant) and a given window (say, the current 10-second bucket), you keep one integer. Each request increments it by one. If the value after your increment is within the limit, you allow the request; otherwise you reject it. When the window rolls over, you want a fresh counter — which means the old one should simply disappear.

Two properties make or break this:

• The increment and the read-back are one operation. If "add one" and "tell me the new value" are separate calls, two instances interleave and the count drifts. You need the store to increment and return the resulting value as a single atomic step, decided by whoever owns that key — never reconstructed on the client.
• The window expires on its own. You do not want a sweeper job deleting stale buckets, and you do not want a bucket living forever after its 10 seconds are up. The counter should carry its own time-to-live so it evaporates when the window closes.

Zaris's counter API gives you both in one method. In a cluster, a key has exactly one active partition owner at a time, and every increment for that key is applied on that owner in order. That's what makes the increment atomic across all your app instances: they aren't racing on a value they each hold a copy of — they're each sending a delta to the one node that owns the number.

The connection string and wiring

Nothing about rate limiting is special at the connection layer. Point the app at a store the way you would for anything else — one connection string, resolved by name:

// appsettings.json
{
  "ConnectionStrings": {
    "RateLimit": "zariss://zaris-a.example.com:7861,zaris-b.example.com:7861/?store=ratelimit&token=env:ZARIS_TOKEN"
  }
}

zariss:// is the TLS scheme (the extra s, same convention as https); list more than one host and the client fails over between them. Register the client from configuration and you get an IZarisClientProvider you can resolve a store from:

using Clustron.Zaris.Client.DependencyInjection;

builder.Services.AddClustronZarisFromConnectionStrings(builder.Configuration);

The limiter

The whole decision is one call. AddAsync increments the counter by one, creates it if it doesn't exist, and — because we pass a Ttl — stamps the window's lifetime onto the key. It returns the value after the increment, so the allow/deny test is a single comparison with no second round trip:

using Clustron.Zaris.Abstractions;
using Clustron.Zaris.Client;

public sealed class FixedWindowRateLimiter
{
    private readonly IZarisClientProvider _provider;
    private const string Store = "RateLimit";
    private const int Limit = 100;                             // requests per window
    private static readonly TimeSpan Window = TimeSpan.FromSeconds(10);

    public FixedWindowRateLimiter(IZarisClientProvider provider) => _provider = provider;

    public async Task<bool> IsAllowedAsync(string subject, CancellationToken ct = default)
    {
        var client  = await _provider.GetAsync(Store);
        var counters = ((IZaris)client).Counters;

        var result = await counters.AddAsync(
            WindowKey(subject),
            delta: 1,
            new CounterOptions { Ttl = Window },               // window expires on its own
            ct);

        if (!result.IsSuccess)
            return false;                                       // fail closed: store said no

        return result.Value.Current <= Limit;                   // value *after* my increment
    }

    // All requests in the same 10s bucket hash to the same key; the next bucket is a new key.
    private static string WindowKey(string subject)
    {
        long bucket = DateTimeOffset.UtcNow.ToUnixTimeSeconds() / (long)Window.TotalSeconds;
        return $"rate:{subject}:{bucket}";
    }
}

Three things are worth pausing on:

• The window key rolls itself over. The bucket number is just wall-clock seconds divided by the window length, so every request in the same 10 seconds lands on the same key and the next window is a brand-new key. Combined with the TTL, old buckets clean themselves up — there is no reset job.
• The TTL rides along with the first increment. The call that creates the counter is also the call that sets its expiry, so there is no gap where a freshly-created counter is immortal because a separate "set expiry" call hasn't landed yet.
• Current is the post-increment value. result.Value.Current is the count including this request, so <= Limit is exactly right: the 100th request sees 100 and is allowed; the 101st sees 101 and is rejected.

Dropping it in front of a minimal API is a two-liner:

app.Use(async (ctx, next) =>
{
    var subject = ctx.User.Identity?.Name ?? ctx.Connection.RemoteIpAddress!.ToString();
    if (!await limiter.IsAllowedAsync(subject, ctx.RequestAborted))
    {
        ctx.Response.StatusCode = StatusCodes.Status429TooManyRequests;
        ctx.Response.Headers.RetryAfter = "10";
        return;
    }
    await next();
});

Letting the server reject for you

You can also push the limit into the store. CounterOptions.MaxValue makes the increment itself fail once the bound is crossed, so the counter never climbs past the ceiling and an over-limit request comes back as a non-success status rather than a number you compare:

var result = await counters.AddAsync(
    WindowKey(subject), 1,
    new CounterOptions { Ttl = Window, MaxValue = Limit }, ct);

return result.IsSuccess;   // false once the window is full

Both styles are correct; the difference is whether you want the count to keep rising past the limit (useful if you log how far over a client went) or to stop dead at the ceiling.

What actually happens on a request

Each instance sends an add-one for the same window key to the one node that owns it. That node applies the increments one after another — there is no interleaving to lose — and hands each caller back the resulting count. The caller compares it to the limit locally. The number lives in exactly one place, so all three instances are counting against the same total, and the TTL means the bucket disappears when its window ends.

Trade-offs, and when to reach for something else

It's a fixed window, so it tolerates an edge burst. A client can send the full quota in the last instant of one window and the full quota in the first instant of the next, briefly doing 2× the rate across the boundary. For most protect-the-backend cases that is fine. If you need the boundary smoothed, a sliding-window-log or a token bucket is the next step up — both are buildable on the same counter and CAS primitives, at the cost of more state per subject.

Each check is a network round trip. One atomic hop to the owner is cheap — the Zaris client routes straight to the partition owner, so it's a single hop, not a fan-out — but it is not free the way an in-memory check is. At extreme request rates you can add a tiny per-instance pre-filter (a local token bucket sized to your share of the limit) in front of the shared counter, and only consult the store when the local allowance runs low. That trades a little precision for a lot fewer round trips.

Counts are owner-serialized, not globally linearizable reads. The increment is atomic and ordered on the owner, which is exactly what a limiter needs. If you also read the counter out-of-band and need the freshest committed value, pass RequireConsistentRead on the read — but the limiter path above never needs it, because the value it acts on is the one the owner just returned from the increment.

When to use this versus the RESP front-end. If you're migrating an app that already does INCR mykey / EXPIRE mykey, Zaris speaks the Redis protocol and those commands work unchanged. The native Counters API shown here is the first-class .NET surface: it folds the expiry into the increment, enforces bounds server-side with MaxValue, and returns a typed result instead of an integer you have to interpret. Reach for RESP when you're porting; reach for the native API when you're writing new .NET code.

Wrap-up

A correct distributed rate limiter comes down to one requirement: the count must be incremented and read as a single atomic step, owned in one place, and it must forget old windows on its own. Keep the limiter in process and it leaks by a factor of your instance count. Build it on a shared GET-then-SET and it races. Zaris collapses the whole decision into one AddAsync call — atomic increment, window TTL, optional ceiling, typed result — so the limiter you ship is the limit you meant, no lock and no Lua required.

The counter API is the same one behind metrics, quotas, and inventory counts; a rate limiter is just its most familiar face. If you want to see it run end to end, the RateLimiter sample in the Zaris samples repository is this post in executable form.