An alert is a request for human attention. If it does not explain why attention is needed or what to do next, it is closer to a notification than an alert.
Five useful fields
Every alert should answer:
- What is happening?
- How bad is it?
- Who is affected?
- When did it start?
- What is the safest next action?
Checkout errors above 5% for 10 minutes
Impact: payments may fail for new orders
Started: 14:22 UTC
Next: check payment-provider status, then review deploy 8f31
Runbook: /runbooks/checkout-errorsSeverity needs meaning
| Level | Meaning | Expected response |
|---|---|---|
| Info | Worth recording | No immediate action |
| Warning | Needs review soon | Handle during working hours |
| High | User impact is likely | Respond now |
| Critical | Major function unavailable | Escalate and coordinate |
Avoid alerting on symptoms alone
CPU usage may be high because traffic is healthy. A better alert combines a resource signal with an impact signal, such as elevated latency or failed requests.
The best alert is one that can be acknowledged confidently, investigated quickly, and closed with a clear explanation.