This is a complete, hands-on Prometheus cheat sheet: install, configure, operate, automate and secure it - with copy-paste commands grouped by task. Bookmark it for day-to-day work, incidents and interviews alike.
Prometheus scrapes metrics from HTTP endpoints, stores them as time series, and lets you query and alert on them with PromQL - the backbone of Kubernetes-era monitoring.
Why you reach for it
- Pull-based scraping with service discovery fits dynamic infrastructure
- PromQL makes rich, dimensional queries and alerts possible
- A huge ecosystem of exporters for almost everything
Installation
docker run -p 9090:9090 -v $PWD/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus
# open http://localhost:9090Configuration reference
Scrape config plus an alert rule.
# prometheus.yml
scrape_configs:
- job_name: app
static_configs: [{ targets: ['app:3000'] }]
---
# rules.yml
groups:
- name: app
rules:
- alert: HighErrorRate
expr: rate(http_requests_total{status=~"5.."}[5m]) > 0.05
for: 10m
labels: { severity: page }
annotations: { summary: "5xx error rate above 5%" }Core commands you use daily
Useful PromQL queries.
# 95th percentile latency
histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket[5m])) by (le))
# error ratio
sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))PromQL quick reference
rate(http_requests_total[5m]) # per-second rate
sum by (code) (rate(http_requests_total[5m])) # grouped
histogram_quantile(0.95, sum(rate(dur_bucket[5m])) by (le)) # p95
increase(errors_total[1h]) # count over window
predict_linear(disk_free[6h], 4*3600) < 0 # will disk fill?Validate & query
promtool check config prometheus.yml
promtool check rules rules.yml
promtool test rules tests.yml
curl -s 'http://localhost:9090/api/v1/query?query=up' | jqUse it in CI/CD
Unit-test alert rules in CI with promtool.
promtool check rules rules.yml
promtool test rules tests.ymlBest-practice checklist
- Keep label cardinality low and stable
- Alert on symptoms (latency, error ratio) via SLOs, not noisy causes
- Use recording rules for expensive dashboards and remote-write for long-term storage
Common pitfalls to avoid
- High-cardinality labels (user IDs, request IDs) exploding memory
- Treating Prometheus as long-term storage without Thanos/Cortex
- Alerting on raw values instead of rates/ratios tied to user impact
Security quick wins
- Put Prometheus behind auth/proxy; it has no built-in authentication
- Scope exporters and avoid exposing internal metrics publicly
Works well with
- Grafana - The open dashboarding layer for metrics, logs and traces, with alerting across many data sources.
- Grafana Loki - Prometheus-style log aggregation that indexes labels, not full text - cheap and Grafana-native.
- OpenTelemetry - The vendor-neutral standard for traces, metrics and logs - instrument once, export anywhere.
A cheat sheet is muscle memory on paper - keep it close until the commands live in your fingers.
Want senior help putting Prometheus to work - or a team bootcamp on it? CodeWizard runs Israel's largest DevOps community and embeds named senior engineers with a transparent hours log. Book a free consult or explore the Academy.