An SLO is a target for a service level indicator (SLI) - for example, 99.9% of requests succeed over 30 days. The error budget is what is left: 0.1% of requests may fail before you have spent it. That single number aligns reliability and release velocity.
Define the SLI as a query
Pick an SLI that reflects user experience - availability or latency - and express it in PromQL.
# availability SLI: fraction of successful requests
sum(rate(http_requests_total{status!~"5.."}[30d]))
/
sum(rate(http_requests_total[30d]))Alert on burn rate, not raw errors
Do not page on every 5xx. Page when you are burning the budget fast enough to exhaust it soon. A multi-window burn-rate alert catches real problems while staying quiet during blips.
- alert: ErrorBudgetBurnFast
expr: |
(1 - (sum(rate(http_requests_total{status!~"5.."}[1h])) / sum(rate(http_requests_total[1h])))) > (14.4 * 0.001)
for: 5m
labels: { severity: page }When the error budget is spent, reliability work wins over new features - automatically, by agreement, not by argument.
SLOs give teams a shared, principled way to say 'not yet' to a risky change - and permission to ship fast while the budget is healthy.