Monitor, log and load test

Prometheus scrapes numbers from every target every 15–30 s; promtool checks its config and rules.

$promtool check config prometheus.yml
$promtool check rules alerts.yml
$curl -s 'http://prometheus:9090/api/v1/query?query=up' | jq '.data.result[] | [.metric.instance, .value[1]]'

node_exporter gives CPU, memory, disk and network numbers of a Linux server on port 9100.

$curl -s localhost:9100/metrics | grep node_filesystem_avail_bytes

Grafana shows metrics, logs and traces in dashboards; grafana-cli installs plugins and resets the admin password.

#grafana-cli plugins install grafana-piechart-panel
#systemctl restart grafana-server

Loki stores logs with labels; logcli queries them from the terminal.

$logcli query '{namespace="shop", app="shop"} |= "ERROR"' --since 1h
$logcli query 'sum by (status) (count_over_time({app="nginx"} | json [5m]))'

otelcol receives traces, metrics and logs (OTLP), processes them and sends them to any backend.

$otelcol --config /etc/otelcol/config.yaml validate
$otelcol --config /etc/otelcol/config.yaml

k6 runs scripted load tests with pass/fail thresholds; ab is a quick one-URL benchmark.

$k6 run -e BASE_URL=https://staging.example.com load.js
$ab -k -n 2000 -c 50 https://staging.example.com/

Good alerts tell you users have a problem; the dashboards then show why.