Resilience
A flag service that fails badly is worse than no flag service. This page documents precisely what each SDK does when things go wrong, so you can reason about your own blast radius.
The guarantee
A failed request never downgrades a value the SDK already holds. The local cache is only ever replaced by a successful response. On any failure the last-known-good value is served instead. Your compile-time default is the floor, not the fallback.
401 and 403 too, which is
deliberate. Key rotations race caches, and an auth blip is not a reason to
flip every flag in production back to its default.
Failure matrix
Behavior of evaluate() and evaluateAll():
| Condition | Behavior | Reported as |
|---|---|---|
| Network error / timeout | Serve cached value, else your default | error_fallback |
5xx | Serve cached value, else your default | error_fallback |
401 / 403 | Serve cached value, else your default. Warning logged, deduplicated to once a minute | error_fallback |
404 (unknown flag) | Definitive. Return your default immediately, no cache lookup | flag_not_found |
402 (plan gate on streaming) | Fall back to polling for the life of the subscription | state polling |
429 | Serve cached value. Honor Retry-After before retrying | error_fallback |
evaluate() and evaluateAll() make exactly one
attempt, because they sit on your request path and a retry storm there is a
worse outage than a stale flag. Retrying is the job of the
stream/poll loop.
The cold-start caveat
Last-known-good only helps once something is known. A process that has never completed a successful evaluation has an empty cache, so a failure at that moment serves your compile-time defaults. Two consequences worth designing around:
- Your defaults are a real code path. Pick values that are safe to run in production, not placeholders.
- Short-lived isolates start cold every time. Serverless functions and Workers do not inherit a warm cache from a previous invocation. If that matters, use the Edge SDK with a KV-backed blob, which is warm on the first read.
Reconnection
stream() reconnects with full-jitter exponential backoff and
never gives up, so recovering from an outage never needs an
application restart. While SSE is down, polling keeps values fresh, a
dropped stream is a degradation, not an interruption.
- Backoff starts at
baseDelayMs(1s) and is capped atmaxDelayMs(60s), with full jitter. - The server sends a heartbeat every 30s. A watchdog treats silence past
heartbeatTimeoutMs(90s, two missed beats) as a dead connection and reconnects. - A connection that stays healthy resets the backoff ladder, so an hour-later blip retries fast rather than waiting a minute.
Connection states
| State | Meaning | Actionable? |
|---|---|---|
streaming | Live SSE connection. | No |
reconnecting | Stream lost; polling covers the gap while SSE retries. | No — values are still fresh |
polling | Terminal for this subscription: free plan, no EventSource, or retry.enabled: false. | Only if you expected streaming |
reconnecting. It is the SDK
working as designed. Alert on polling when you are on a plan
that includes streaming — that one means you will not see changes for up to
a full polling interval.
Making failures visible
The SDKs are deliberately quiet: they degrade instead of throwing, and warnings are deduplicated to at most one line per minute per failure class so an outage cannot flood your logs. The trade-off is that a misconfigured key can look like "the flag is just off". Wire up the hooks:
import { ShipSilentlyClient } from '@shipsilently/node';
export const flags = new ShipSilentlyClient({
apiKey: process.env.SHIPSILENTLY_KEY!,
});
flags.stream({ userId: 'system' }, () => {}, {
onStateChange: (state) => {
// 'polling' on a streaming plan means changes are up to 30s late.
if (state === 'polling') metrics.increment('shipsilently.degraded');
},
});
The Edge SDK takes an onError callback
for the same purpose. It defaults to a no-op, so pass one that logs, or
provider failures are entirely invisible.
Go specifics
The Go client follows the same model with one difference worth knowing:
Evaluate returns (value, error), and on a
cache hit after a failure it returns the cached value with a
nil error. A non-nil error therefore means "no cached value
either", and the returned value is your default.
DisableCache: true turns this off and restores strict
error-per-failure behavior.
Production checklist
- Every default value is a safe, tested code path.
- The client is created once at startup, not per request, so the cache survives.
- Analytics are flushed before serverless handlers return.
onStateChange/onErrorfeed your metrics.- Long-running services stream; short-lived ones call
evaluateAll()once per request. - Kill-switch behavior is verified in staging, not assumed. See Kill Switches.