← All insightsObservability

Monitoring and observability for Prometheus

Nir Geier, CodeWizard · 20 May 2026 · 3 min read · Intermediate

You cannot operate what you cannot see. Here is how to get useful signals out of Prometheus and turn them into alerts that mean something.

Prometheus scrapes metrics from HTTP endpoints, stores them as time series, and lets you query and alert on them with PromQL - the backbone of Kubernetes-era monitoring.

What to watch

  • Keep label cardinality low and stable
  • Alert on symptoms (latency, error ratio) via SLOs, not noisy causes
  • Use recording rules for expensive dashboards and remote-write for long-term storage
  • Track error rate and latency, not just up/down
  • Alert on symptoms tied to user impact

Operate and inspect

Useful PromQL queries.

promql
# 95th percentile latency
histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket[5m])) by (le))
# error ratio
sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))

Keep it healthy

  • Put Prometheus behind auth/proxy; it has no built-in authentication
  • Scope exporters and avoid exposing internal metrics publicly

Want senior help putting Prometheus to work - or a team bootcamp on it? CodeWizard runs Israel's largest DevOps community and embeds named senior engineers with a transparent hours log. Book a free consult or explore the Academy.

Like where this is going?

Get hands-on help from the engineers who write these posts.

Book a free consult