What causes flaky tests, and how do you systematically fix them instead of just re-running CI?
A flaky test passes or fails inconsistently on the same code, and it’s almost never actually random, it’s a real bug in the test (or the system under test) that only surfaces under certain timing or ordering conditions. The most common causes are unhandled asynchronicity (asserting before an async operation finishes), shared mutable state between tests (one test’s leftover data affecting another), and reliance on real timing or ordering (like wall-clock sleeps or the actual order tests run in) instead of explicit synchronization.
Why re-running CI is the wrong fix
Re-running masks the symptom without addressing the cause, and worse, it trains the team to stop trusting red CI runs, so a genuinely broken test (or worse, a genuine bug that flakes rather than fails consistently) starts getting waved through as “probably just flaky.” The cost compounds: every flaky test erodes confidence in the whole suite.
Best practice
Replace fixed sleeps with explicit waits for the actual condition you care about (an element appearing, a promise resolving), ensure each test sets up and tears down its own isolated state rather than depending on leftover state from another test, and run suspect tests repeatedly in isolation and in random order to reproduce the flake deterministically before attempting a fix.
Edge case interviewers probe for
How do you debug a test that only flakes in CI but never locally? This usually points to environment differences, CI often runs with less CPU/memory or higher parallelism, which exposes race conditions and timing assumptions that never manifest on a fast, unloaded local machine. Reproducing it means simulating that resource pressure, not just re-running locally.
Common mistake
Adding a longer sleep or a retry wrapper around a flaky test to make it “pass” without ever finding the actual race condition. This just makes the test slower and less reliable at scale, since the underlying timing bug is still there, only harder to catch.
What the interviewer is checking
Whether you treat test flakiness as a real engineering signal worth root-causing, and whether you know the concrete categories (async races, shared state, environment-dependent timing) rather than shrugging it off as inherent randomness.
Imagine checking if a cake is done by opening the oven at exactly the 20-minute mark every time, no matter what. Most days it happens to be done by then, but some days your oven runs a little cooler and it isn’t ready yet, so your “is it done” check sometimes says no even though nothing about the recipe changed. That’s a flaky test: you’re checking at a fixed time instead of actually checking the real condition (is it golden brown and firm), so the result depends on unrelated timing instead of the thing you actually care about.
The fix isn’t to just try again and hope it passes this time, it’s to change your check from “wait exactly 20 minutes” to “wait until it’s actually done,” which is reliable no matter how the oven happens to be running that day.
Why interviewers ask this
Flaky tests are a near-universal pain point on real teams, and this question reveals whether a candidate has actually debugged one or just tolerates the symptom.
What a strong answer signals
That you can name specific root-cause categories (races, shared state, environment differences) instead of treating flakiness as unavoidable randomness.
Common follow-ups
- How would you find which test is causing shared-state pollution?
- What’s the difference between a flaky test and a genuinely non-deterministic system?
- How do you decide when to quarantine a flaky test versus fixing it immediately?
Advanced variation
“How would you build tooling to detect flaky tests automatically?” expects discussing running the suite repeatedly (and in randomized order) in CI to surface inconsistent results, then tracking a flakiness rate per test over time.
A test suite had a handful of tests that failed roughly one run in ten, always in CI, never locally. The team’s habit was to just re-run the pipeline. Running the suite with tests deliberately randomized and in parallel locally reproduced the failures, revealing two tests wrote to the same database row without cleanup, so whichever ran second saw unexpected leftover state. Giving each test its own isolated fixture data eliminated the flake completely, and CI stopped needing manual re-runs.
// Flaky: fixed sleep assumes the async save always finishes in 200ms
await save(user);
await sleep(200);
expect(getUser(user.id).name).toBe('Alice');
// Fixed: wait for the actual condition, no fixed timing assumption
await save(user);
await waitFor(() => getUser(user.id) !== null);
expect(getUser(user.id).name).toBe('Alice');
// Flaky: shared row across tests with no isolation
beforeEach(() => db.reset({ scope: 'per-test' }));- 1Flaky tests almost always have a real, findable cause, not genuine randomness.
- 2Fixed sleeps and shared mutable state between tests are the two most common culprits.
- 3Re-running CI to get a pass masks the bug and erodes trust in the whole suite.
- 4CI-only flakiness often points to resource pressure or parallelism exposing timing assumptions.
- 5Explicit waits for real conditions, and per-test isolated state, eliminate most flakiness at the source.