The four golden signals, from Google’s SRE book, are latency (how long requests take, and critically, tracked separately for successful versus failed requests, since a fast error is not the same as a fast success), traffic (how much demand the system is under, requests per second, or something domain-specific like concurrent sessions), errors (the rate of requests failing, explicit failures and also degraded responses that technically returned 200 but were wrong), and saturation (how full the system is relative to its capacity, CPU, memory, disk, or a queue depth, whichever resource is closest to being the actual bottleneck).
Why these four specifically
Together they answer “is the system healthy” from both the user’s perspective (latency, errors) and the system’s own internal capacity perspective (traffic, saturation), which means most real incidents show up in at least one of them, so alerting on all four gives good coverage without needing to track dozens of ad-hoc metrics.
Best practice for alerting
Alert on symptoms visible to users (high latency, high error rate) with a page (wake someone up), since those directly represent user pain happening right now. Alert on saturation as a warning, not necessarily an immediate page, since high CPU alone doesn’t always mean users are affected yet, it’s a leading indicator, not a symptom. Use burn-rate alerting tied to an SLO (like error budget consumption) rather than static thresholds where possible, since a fixed “alert if errors exceed 1%” threshold doesn’t account for normal traffic variation.
Edge case interviewers probe for
Ask what happens when latency looks fine but error rate is climbing. That’s often actually good news operationally (failing fast rather than hanging), but bad for users, which is exactly why you need multiple signals together: latency alone would miss this, error rate alone would miss a system that’s technically succeeding but painfully slow.
Common mistake
Setting every alert to page immediately regardless of signal type, which causes alert fatigue, or the opposite mistake, treating saturation metrics as unimportant because “nothing’s broken yet,” missing the chance to react before saturation actually causes latency and errors.
What the interviewer is checking
Whether you think about observability as answering specific operational questions (is anyone in pain right now, are we about to run out of capacity) rather than just collecting metrics because they’re easy to collect.
Imagine you run a small restaurant and want to know if tonight is going badly. Latency is how long a table waits for their food. Traffic is how many tables you’re serving right now. Errors are how many dishes get sent back to the kitchen. Saturation is how close your kitchen is to being completely overwhelmed, running out of oven space or having every burner in use.
You’d want an urgent alert (page the manager immediately) if food is taking forever or dishes keep getting sent back, since customers are unhappy right now. But if the kitchen is getting busy (high saturation) and nothing’s gone wrong yet, that’s more of a “heads up, get ready” signal than a five-alarm fire, useful to know early so you can act before it turns into slow food and returned dishes.
Why interviewers ask this
It’s a foundational SRE concept that reveals whether a candidate thinks systematically about observability instead of monitoring metrics ad hoc.
What a strong answer signals
That you distinguish user-facing symptoms from internal leading indicators, and match alert urgency (page vs. dashboard) to which one you’re looking at.
Common follow-ups
- What’s an error budget and how does it relate to alerting?
- How would you track latency percentiles instead of just an average?
- What’s the danger of alerting on averages instead of percentiles?
Advanced variation
“Design an SLO and alerting policy for a payments API,” which expects tying golden signals to a concrete SLO (like 99.9% of requests under 300ms) and burn-rate-based paging.
An API team originally alerted on average latency, which stayed low even during a real incident because most requests were fast while a subset were timing out entirely and getting excluded from the average once they failed. Switching to p95 and p99 latency percentiles, plus a separate error-rate alert, caught the same class of incident within minutes instead of it going unnoticed until customer complaints arrived.
- 1The four golden signals are latency, traffic, errors, and saturation.
- 2Latency should be tracked separately for successful versus failed requests.
- 3Latency and errors are user-facing symptoms; traffic and saturation are more often leading indicators.
- 4Page immediately on user-facing symptoms; treat saturation as an earlier warning, not always an emergency.
- 5Use percentiles, not averages, for latency, since averages can hide a painful tail of slow or failing requests.