Distributed Rate Limiter

SharedRoute

A Go rate limiter that stays correct under 200 concurrent goroutines and stays up when Redis doesn't — verified with k6 chaos tests, including a live Redis kill.

The problem

Rate limiters are easy to write and easy to get wrong. Under real concurrency, naive check-then-set logic double-counts or double-allows; and when the shared cache layer dies, most implementations either hard-fail every request or silently stop limiting. ShardRoute was built to be provably correct under concurrent load and to degrade gracefully — fail-open with bounded risk — when Redis is killed mid-traffic.

Architecture & approach

1. Token-bucket state lives in Redis; all read-modify-write logic is pushed into Lua scripts so each decision executes atomically server-side — no race window between check and consume.

2. A Go service fronts Redis over gRPC and HTTP, sharding keys so hot tenants don't serialize behind one another.

3. Correctness was verified head-on: 200 concurrent goroutines hammering one bucket, with exactly 100 requests allowed and 100 rejected — the bucket's math held under contention.

4. Chaos testing with k6: Redis was killed live during a load run. The service fell back to a local fail-open path and maintained 96.8% uptime through the outage instead of collapsing.

5. Moving decisions into Lua and off the request hot path cut P99 latency 81%, from 95ms to 18ms.

Stack

• Go
• Redis
• Lua
• gRPC
• HTTP
• k6