Architecture: Where Flags Live
Server, client, edge: how feature flag systems are actually built, what the trade-offs are, and why the hot path matters more than the dashboard.
The dashboard is the part of a feature flag system people see. The hot path, the few milliseconds between your code asking “is this on?” and getting an answer, is the part they don’t, and it’s where every interesting architectural decision gets made.
This chapter is a tour of those decisions.
The two halves of a flag system
Every flag system splits into a control plane and a data plane:
- The control plane is what dashboards, APIs, and audit logs talk to. It is the source of truth. It’s slow, transactional, and rarely on a critical path.
- The data plane is what your SDK talks to ten thousand times a second. It has to be fast, available, and eventually consistent with the control plane.
Most flag-system pain comes from gluing those two halves together. How quickly does a dashboard change show up in production? How do you guarantee it shows up everywhere? What happens when the data plane can’t reach the control plane?
The rest of this chapter is about how teams answer those three questions.
Server-side vs client-side evaluation
The most consequential choice is where the decision is made.
Server-side SDKs
Your backend asks the flag system whether a feature is on, then returns rendered content (or an API response) to the user. The user never sees the flag rules.
- Pros: Rules and traffic data stay private. Updates take effect on the next request. No bloat in the client bundle. Targeting can use server-only data (DB lookups, internal IDs).
- Cons: Every request needs an evaluation, either local in the server process or via a network call. Latency budget is brutal.
Client-side SDKs
The browser, mobile app, or game client evaluates flags locally using rules pulled from the flag service.
- Pros: Zero per-evaluation latency once rules are loaded. The client can render conditional UI without a server round-trip.
- Cons: Rules and segments are visible to anyone with devtools. Flag values are stale until you re-fetch. App startup must wait for (or gracefully handle) the rules payload.
Most real applications use both. Server SDKs gate backend behavior; a thin client SDK handles UI variants. The rules of the road:
- Anything sensitive (pricing, plan logic, security gates), server-side only.
- Anything visual (rendering A vs B), fine on the client.
- A flag that needs to mean the same thing on backend and frontend, evaluate once on the server, pass the decision down.
Push vs pull: how rules reach the SDK
Once you’ve decided where evaluation happens, you have to decide how the SDK gets the rules.
Polling
The SDK fetches rules on an interval, every 30 seconds, every minute. Simple, robust, easy to debug. Cheap if your edge cache is doing its job.
The downside is staleness: change a flag in the dashboard and you wait up to one polling interval before every server has the new value.
Streaming (push)
The SDK opens a long-lived connection, SSE or WebSocket, and the flag service pushes updates as they happen. Sub-second propagation. Beautiful when it works.
Streaming is more expensive on the server side, harder to scale, and introduces a class of “stuck connection” bugs that polling does not have. A real production system usually does streaming with polling as a safety net, push for speed, poll periodically to catch missed messages.
Edge cache
Rather than every SDK fetching from a central API, rules are pushed to an edge cache (Cloudflare KV, Fastly Compute, regional R2 buckets). SDKs pull from the nearest edge POP. Latency drops from ~80ms to ~5ms. The control plane is responsible for invalidating the edge after a write.
For the math on why edge eval matters for global apps, see Edge computing meets feature flags.
The hot path target
The number you should care about is the p99 evaluation latency from inside your app. Not the average. Not the dashboard’s “we returned in 3ms” graph. The actual p99 inside your code.
A reasonable target:
| Where | p99 latency |
|---|---|
| Local evaluation in-process | <0.1ms |
| Edge cache fetch (warm) | <5ms |
| Origin fetch (cold start) | <50ms |
| Streaming reconnect (worst case) | <2s |
If you blow these numbers, your engineers will start working around the flag system, caching values, hardcoding bypasses, adding if (process.env.FAST_PATH) short-circuits. A flag system that’s bypassed is worse than no flag system, because it lies to your dashboard.
Fail-open or fail-closed?
When the flag service is unreachable, DNS hiccup, expired cert, your own bug, the SDK has to return something. There are three sane defaults:
- Default off. Safest for new features. Worst for already-shipped features that depend on flags.
- Default on. Risky for new features. Necessary for kill switches whose “off” position breaks the system.
- Last-known-good cache. Use whatever the SDK saw last. Almost always the right answer in practice.
The SDK should also expose this explicitly: let the caller pass a default per call. Defensive code beats clever defaults.
const showAi = await flags.get("ai-summarizer", user, { default: false });That default is what runs when the network melts. Pick it on purpose, not by accident.
A working mental model: the “two latencies”
The two numbers that define the user-visible behavior of a flag system are:
- Eval latency, how long does a single check take? (Microseconds, ideally.)
- Propagation latency, how long after I change a flag does every server see the new value? (Seconds, ideally.)
A good flag system optimizes both. A bad one trades one for the other and pretends it’s a feature. (“We don’t need streaming because our SDK polls every 30 seconds.”) Watch out for that framing.
Where ShipSilently sits
ShipSilently keeps rules in D1 (control plane), caches a compiled rule blob in Cloudflare KV (data plane), and uses a Durable Object to broadcast invalidations over SSE to active SDKs. SDKs evaluate locally, eval is in-process, and stay current via stream-with-polling-fallback. The numbers above are the targets we hold ourselves to.
You do not have to build it this way. The mental model is what matters. Pick your two latencies, pick your fail mode, and write them down before you write the SDK.
Previous: Chapter 1, What is a feature flag? · Next: Chapter 3, Targeting and rollouts