Security with Prometheus is mostly correct, boring defaults applied consistently. Here is the short list that removes most of the risk.
Safe defaults
- Put Prometheus behind auth/proxy; it has no built-in authentication
- Scope exporters and avoid exposing internal metrics publicly
Wire it into the pipeline
Unit-test alert rules in CI with promtool.
promtool check rules rules.yml
promtool test rules tests.ymlReinforcing habits
- Keep label cardinality low and stable
- Alert on symptoms (latency, error ratio) via SLOs, not noisy causes
- Use recording rules for expensive dashboards and remote-write for long-term storage
Want senior help putting Prometheus to work - or a team bootcamp on it? CodeWizard runs Israel's largest DevOps community and embeds named senior engineers with a transparent hours log. Book a free consult or explore the Academy.