Most Teams Use Feature Flags Wrong
They wire up LaunchDarkly or Unleash, use it for two A/B tests, then forget about it.
Meanwhile, their production is full of if (isNewCheckoutEnabled) blocks that nobody remembers how to toggle.
Feature flags are not primarily an experimentation tool. They're a reliability tool.
The Real Value
Feature flags let you separate deploy from release. You ship code to production cold, then turn it on gradually for real users.
When things break, you flip the switch back in 10 seconds. No rollback, no redeploy, no PR reverts.
The Four Reliability Patterns
1. Kill Switches
Every risky new feature ships behind a kill switch:
if (featureFlags.isEnabled('new_payment_flow', userId)) {
return newPaymentFlow();
}
return legacyPaymentFlow();
When the new flow has a bug, you don't rollback. You flip the flag.
2. Gradual Rollouts
new_search_algorithm:
rollout_percentage: 1 # Start at 1% of users
rules:
- if: "user.tier == 'internal'"
enabled: true # Internal users always see it
Deploy to 1%, watch metrics, go to 5%, watch, 25%, 50%, 100%. Takes 2-4 hours per rollout instead of a single risky deploy.
3. Circuit Breakers
external_recommendations_service:
enabled: true
automatic_disable_if:
error_rate_above: 5%
for_minutes: 5
If a downstream service starts failing, the flag auto-disables that feature. Your product degrades gracefully instead of crashing.
4. Load Shedding
expensive_realtime_dashboard:
enabled_when:
cpu_utilization_below: 70%
active_users_below: 50000
Under load, disable non-critical features to preserve the critical path.
The Anti-Pattern: Permanent Flags
After a feature is 100% rolled out, the flag should be deleted within 2 weeks. Every flag left in the codebase is technical debt.
Flag hygiene rules:
- Every flag has an expiration date (90 days max)
- Every flag has an owner in CODEOWNERS
- CI fails if a flag is older than 180 days
- Monthly flag cleanup is part of standard operations
We track "flag count" as a reliability metric. If it grows unbounded, we're doing it wrong.
The Architecture
A solid feature flag system has three parts:
1. Definition store
- Source of truth for all flags
- Versioned in Git or a managed service (LaunchDarkly, Unleash, GrowthBook)
- Audit log for every change
2. Client SDK
- In-app flag evaluation
- Falls back to defaults if the service is unreachable
- Caches decisions for 60 seconds
- Emits telemetry for flag usage
3. Admin interface
- Change flags without deploying code
- See current state across environments
- Role-based access (not everyone can flip prod flags)
- Approval workflow for high-risk flags
Evaluating at the Right Layer
Flags can live at multiple layers:
CDN edge — use for marketing experiments
Load balancer — use for blue/green deploys
App server — use for feature experiments
Database — use for schema migrations
The deeper the layer, the faster the rollout. CDN flags flip in seconds. Database flags take minutes to propagate.
The Reliability Metric
Track: mean time to mitigate (MTTM).
If your team can mitigate an incident in under 30 seconds via a feature flag flip, that's a win. If you have to redeploy to mitigate, your reliability is bottlenecked by deploy time.
Good teams: MTTM under 60 seconds
Great teams: MTTM under 15 seconds
Common Gotchas
- Stale flags skew A/B results — clean them up after experiments
- Flags without defaults cause prod outages — every flag must have a safe fallback
- Flag flips mid-request cause weird bugs — evaluate at request start, cache for the request lifetime
- Nested flags (flags inside flags) are impossible to reason about — avoid
A Reliability-First Flag Strategy
Start simple:
- Every new feature ships behind a kill switch
- Gradual rollouts for anything touching the critical path
- Circuit breakers for external dependencies
- Flag cleanup is a monthly ritual
- Track MTTM and optimize it
Feature flags are the most underrated reliability tool in modern engineering. Treat them that way.
Written by Dr. Samson Tanimawo
BSc · MSc · MBA · PhD
Founder & CEO, Nova AI Ops. https://novaaiops.com
Top comments (3)
The 'no one remembers how to toggle' line is the part nobody budgets for. Kill switches pay for themselves the first time you flip one during an incident, but six months later half the flags in the codebase are mysteries nobody wants to touch. We started requiring an expiry date and an owner on every flag at creation time, and cleanup stopped being a quarterly argument. Next step for us is automating the audit with AI - flag what's stale, who's touched it, what's safe to delete. Curious if anyone's actually doing that yet.
The "six months later half the flags are mysteries" part matches what we see. In the migrations we've helped with lately, the slow part wasn't moving flags over, it was the team cleaning up YEARS of old ones first.
To your question: GrowthBook is doing about half of it. GrowthBook has stale flag detection today, but it's a heuristic — is there a live experiment attached, has anyone touched it recently, that sort of thing. You can also ask our MCP for your stale flags (and why they are stale), which is closer to what you're describing but still ends in a list someone has to act on.
The part we don't do yet is the "safe to delete" verdict. We already pull in whether a flag is still referenced in your codebase (through the GitHub integration), but right now that just shows up on the flag page, it doesn't feed into the staleness call. So the pieces exist, they just aren't talking to each other yet. This exact gap came up in our eng sync on Friday, so it's on our radar, just no roadmap date yet.
The thing I'm less sure about is trust. Finding stale flags is the easy part. Getting a team to let an agent archive one (or a hundred) is harder. I think we will get there, though. With enough guardrails and testing.
Required expiry plus owner assigned at creation is a good idea and not something we force today. What happens on your side when the expiry passes — does the flag get disabled, or does the owner just get pinged?
Agreed