Filters
How to search
  • Plain words find names, texts and examples.
  • "two words" finds the exact phrase.
  • --name or -x finds an option.
  • #word looks only inside commands.
  • @grep finds lessons and labs that use grep.
  • Filters: cat:network · in:labs · in:notes · level:rhcsa · ch:8 · root:yes. Put ! before a word to leave it out. OR between words means one of them.
  • ↑ ↓ to move · Enter to open · Esc to close · / to search

logcli

Loki stores logs cheaply by indexing only labels; logcli queries them with LogQL from the terminal.

Level
DevSecOps service
Group
Monitoring and logging
Needs root
No
Package
loki
Also called
loki

What it does

Grafana Loki is "Prometheus for logs": agents (Grafana Alloy, Promtail, Fluent Bit) send log lines with labels (namespace, app, host); Loki indexes only the labels and keeps the text compressed in object storage (Azure Blob, S3). You search with LogQL: a label selector {app="shop"} plus filters (|= "error") and parsers (| json). You use it in Grafana's Explore view or with logcli.

How to write it

logcli [--addr=URL] query 'LOGQL' [--since=1h] [--limit=N] [--tail]
logcli labels [NAME]

Main parts

Commands

query 'LOGQL'
Search logs.
instant-query 'LOGQL'
Metric query at one time.
labels [NAME]
List labels / values of one label.
series 'SELECTOR'
List streams.
stats 'SELECTOR'
Size of the data.

Options 7

Most used

Flags

--addr=URL / LOKI_ADDR
Loki address.
--since=1h / --from --to
Time range.
--limit=N
Max lines (default 30).
-t, --tail
Follow new lines (like tail -f).
-o default|raw|jsonl
Output format.
-q
Quiet (no query info).
--org-id=TENANT
Tenant (multi-tenant Loki).

Examples 10

$ run as a normal user# run as root (or with sudo)

  1. 1Set the address

    export LOKI_ADDR=http://loki.monitoring.svc:3100
    logcli labels
    logcli labels namespace
  2. 2Find errors of one app

    logcli query '{namespace="shop", app="api"} |= "error"' --since=1h --limit=50
  3. 3Follow logs live

    logcli query '{namespace="shop"}' --tail
  4. 4Parse JSON logs and filter by a field

    logcli query '{app="api"} | json | status >= 500 | line_format "{{.method}} {{.path}} {{.status}}"' --since=30m
  5. 5Count errors per app (metric query)

    logcli instant-query 'sum by (app) (count_over_time({namespace="shop"} |= "error" [5m]))'
  6. 6Find failed SSH logins on all hosts

    logcli query '{job="systemd-journal", unit="sshd.service"} |~ "Failed password|Invalid user"' --since=24h
  7. 7Exclude noise

    logcli query '{app="ingress-nginx"} != "/health" != "/metrics"' --since=15m
  8. 8Save raw lines to a file

    logcli query '{app="api"}' --since=2h --limit=5000 -o raw > api.log
  9. 9Install Loki with Helm (single binary, lab)

    helm repo add grafana https://grafana.github.io/helm-charts
    helm install loki grafana/loki -n monitoring -f loki-values.yaml
  10. 10Alert on log lines (Loki ruler)

    cat rules/shop.yml
    groups:
      - name: shop-logs
        rules:
          - alert: ManyErrors
            expr: sum(rate({namespace="shop"} |= "error" [5m])) > 1

Watch out

  • Do not use high-cardinality labels (user ID, request ID); put them in the text and filter.
  • Logs can contain personal data and secrets — set retention and limit who can query.

Files

/etc/loki/config.yaml
Loki server config (path can differ).

Tips to remember

  • In Grafana, link logs to traces and metrics with the same labels (app, namespace).

Exit codes

0Success.
1Error.