Make your server "production-ready". Part 2
Make your server "production-ready". Part 5
Make your server "production-ready". Part 4
Make your server "production-ready". Part 3
Metrics tell you that something's wrong, logs tell you why. If you have to SSH into the server and dig through docker logs across ten containers every time something breaks, you're losing valuable time. In this tutorial you'll extend the monitoring stack from Part 1 with centralized logging using Grafana Loki: all container logs end up searchable in one place, right next to your metrics in Grafana.
Two containers are added to the existing stack: Loki stores logs, and Grafana Alloy automatically collects them from all Docker containers. In Grafana, you can then search all your logs with LogQL — including filters by container, time range, and error message.
The classic ELK stack (Elasticsearch, Logstash, Kibana) is powerful, but overkill for a single VPS: Elasticsearch alone wants several GB of RAM. Loki takes a different approach — instead of indexing the entire log content, it only indexes labels like container name and host. This makes it dramatically leaner: the entire logging stack here runs on around 200–400 MB RAM. And since Grafana is already running, you don't need to learn a new interface, logs and metrics live in the same dashboard.
The division of labor is the same as with Prometheus: Loki is the database, Grafana Alloy is the collector that gathers logs and ships them to Loki, and Grafana is the interface for searching and visualizing.
Older tutorials use Promtail as the log collector. Promtail has been officially deprecated by Grafana (end of life: March 2026) and is being replaced by Grafana Alloy. For new setups, you should use Alloy directly — which is exactly what we do here.
Switch to your monitoring directory from Part 1 and create the configuration:
Create loki/loki-config.yml:
Two settings matter here: retention_period: 30d automatically deletes logs after 30 days (the compactor handles this), and the local filesystem is used as storage, completely sufficient for a single VPS, with no S3 bucket or external dependencies required.
Alloy automatically discovers all running containers via the Docker socket and reads their logs without you needing to change anything on your existing containers. Create alloy/config.alloy:
Adjust the host label to your server's name. Once you're later shipping logs from multiple servers into the same Loki instance, this is how you tell them apart.
Add two services under services: in your existing ~/monitoring/docker-compose.yml:
And add the new volume under volumes::
Then start it:
Loki needs a few seconds after starting before it's ready. A quick check:
Once this returns ready, Loki is accepting logs.
Alloy reads logs from the JSON files Docker writes for each container — and those grow without limit until the disk fills up. Since logs now live centrally in Loki anyway, you can keep the local Docker logs small. Create or extend /etc/docker/daemon.json:
Then run sudo systemctl restart docker. Note: restarting the Docker daemon restarts all containers — plan accordingly. This setting only applies to newly created containers; existing ones pick it up after a docker compose up -d --force-recreate.
Open Grafana and add the new data source:
http://loki:3100 as the URL (container name, just like with Prometheus)Then switch to Explore, select Loki as the data source, and try your first query. LogQL works similarly to PromQL — filter by labels first, then search the text:
shows all logs from the Grafana container. It gets more interesting with filters:
searches the logs of all containers on your server for "error". And with a rate function, you can turn logs back into metrics:
counts error messages per container over the last five minutes — perfect as a panel next to your CPU and RAM graphs from Part 1.
The real payoff shows up when the two work together: open your Node Exporter dashboard from Part 1, add a new panel, select Loki as the data source, and choose Logs as the visualization. With the query {host="my-vps"} |= "error", you can now see directly underneath the CPU graph which errors occurred at the moment of a load spike. This is exactly why you build centralized logging: cause and effect at a glance, no SSH session required.
Alerts on logs are also possible, under Alerting in Grafana, you can create a rule based on a LogQL query, for example when a container's error rate crosses a threshold. The notification then goes through the same Alertmanager and contact points as your metric alerts from Part 1.
So far we've only collected container logs. If you also want SSH logins, kernel messages, and other system logs centralized, add this to alloy/config.alloy:
And add two additional volumes to the Alloy service in the compose file:
After docker compose up -d --force-recreate alloy, you'll find the system logs in Grafana under the label job="journal" — handy for, say, keeping an eye on failed SSH login attempts.
Loki and Alloy together run at around 200–400 MB RAM on a typical VPS. Disk space needs depend on log volume: Loki compresses heavily, and a small server with a handful of containers usually stays well under 1 GB with 30 days of retention. If your log volume grows, reduce the retention_period first before upgrading the server.
Your server now has both: metrics that warn you, and logs that tell you why. All container logs (and optionally system logs) are centrally searchable, automatically rotated, and cleaned up after 30 days — with just two additional containers.
In the next part of our "Production-Ready Server" series, we'll compare Nginx, Caddy, and Traefik — which reverse proxy fits which setup, and when is it worth switching?
No. Alloy automatically discovers all containers via the Docker socket and reads their logs. No labels or logging-driver changes are needed on your existing containers.
Yes. Just install Alloy on each additional server and point its loki.write block at the URL of your central Loki. Important: secure the connection via VPN, a private network, or a reverse proxy with authentication, since Loki has no built-in access protection with auth_enabled: false.
Check the chain from the start: Is Alloy running (docker compose ps)? Does Loki report status ready at /ready? The most common causes are an unmounted Docker socket in the Alloy container and a typo in the data source's Loki URL.
VPS on AMD EPYC with NVMe SSD, full control, hosted in Germany.
Configure VPS
Promtail is deprecated (end of life March 2026) and is being replaced by Grafana Alloy. The concepts are identical, only the configuration language differs — new setups should build directly on Alloy.
cd ~/monitoring
mkdir -p loki alloyauth_enabled: false
server:
http_listen_port: 3100
common:
instance_addr: 127.0.0.1
path_prefix: /loki
storage:
filesystem:
chunks_directory: /loki/chunks
rules_directory: /loki/rules
replication_factor: 1
ring:
kvstore:
store: inmemory
schema_config:
configs:
- from: "2024-04-01"
store: tsdb
object_store: filesystem
schema: v13
index:
prefix: index_
period: 24h
limits_config:
retention_period: 30d
compactor:
working_directory: /loki/compactor
retention_enabled: true
delete_request_store: filesystem loki:
image: grafana/loki:latest
container_name: loki
restart: unless-stopped
volumes:
- ./loki:/etc/loki
- loki-data:/loki
command:
- '-config.file=/etc/loki/loki-config.yml'
ports:
- "127.0.0.1:3100:3100"
networks:
- monitoring
alloy:
image: grafana/alloy:latest
container_name: alloy
restart: unless-stopped
volumes:
- ./alloy/config.alloy:/etc/alloy/config.alloy:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
command:
- 'run'
- '/etc/alloy/config.alloy'
networks:
- monitoringvolumes:
prometheus-data:
grafana-data:
loki-data:docker compose up -d
docker compose pscurl -s http://localhost:3100/ready{
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "3"
}
} - /var/log/journal:/var/log/journal:ro
- /etc/machine-id:/etc/machine-id:ro// Automatically discover all running Docker containers
discovery.docker "containers" {
host = "unix:///var/run/docker.sock"
}
// Use the container name as a clean label
discovery.relabel "containers" {
targets = discovery.docker.containers.targets
rule {
source_labels = ["__meta_docker_container_name"]
regex = "/(.*)"
target_label = "container"
}
}
// Read logs from the discovered containers
loki.source.docker "containers" {
host = "unix:///var/run/docker.sock"
targets = discovery.relabel.containers.output
labels = { host = "my-vps" }
forward_to = [loki.write.local.receiver]
}
// Forward to Loki
loki.write "local" {
endpoint {
url = "http://loki:3100/loki/api/v1/push"
}
}{container="grafana"}{host="my-vps"} |= "error"sum by (container) (count_over_time({host="my-vps"} |= "error" [5m]))loki.source.journal "system" {
labels = { host = "my-vps", job = "journal" }
forward_to = [loki.write.local.receiver]
}