Most Grafana Loki pain is self-inflicted and predictable. Here are the mistakes we see most, and the fix for each.
Mistake 1: High-cardinality labels (like request_id) destroying performance
Fix: Keep labels low-cardinality; filter within the stream using LogQL. Make it the default in your templates and enforce it in review or CI so the mistake cannot come back.
Mistake 2: Pushing huge log volumes without retention or limits
Fix: Set per-tenant retention and ingestion limits. Make it the default in your templates and enforce it in review or CI so the mistake cannot come back.
Mistake 3: Expecting full-text indexing - Loki filters, it doesn't index content
Fix: Structure logs (logfmt/JSON) so LogQL parsers work well. Make it the default in your templates and enforce it in review or CI so the mistake cannot come back.
A safer baseline
Promtail scraping local logs and shipping to Loki.
server: { http_listen_port: 9080 }
clients: [{ url: http://loki:3100/loki/api/v1/push }]
scrape_configs:
- job_name: system
static_configs:
- targets: [localhost]
labels: { job: varlogs, __path__: /var/log/*log }Want senior help putting Grafana Loki to work - or a team bootcamp on it? CodeWizard runs Israel's largest DevOps community and embeds named senior engineers with a transparent hours log. Book a free consult or explore the Academy.