Knowledge Base

Diagnosing systemd services and restart loops

Author: DEWOLF.IT

Use effective unit configuration, journal messages and explicit dependencies to troubleshoot Linux services.

This English edition covers the main steps. See the Dutch article for the full detail and examples.

Find the first failure and the effective unit

Inspect the loaded unit and any drop-ins. Read the journal around the first failure before focusing on the final restart-limit message.

systemctl status example-app.service
systemctl cat example-app.service
journalctl -b -u example-app.service

Reconstruct the execution context

Check the service user, working directory, environment, executable, mounted paths and sandboxing. Compare with the environment used in a manual test. Starting successfully and becoming ready for traffic can happen at different times.

Model dependencies and recovery deliberately

After= orders units and does not pull them in. Wants= and Requires= describe different dependency strengths. Use RequiresMountsFor= for paths the service needs. A bounded restart policy can recover from transient faults without hiding a persistent one.

[Unit]
StartLimitIntervalSec=300
StartLimitBurst=5

[Service]
Restart=on-failure
RestartSec=10s

Verify service behaviour

Validate the unit, reload configuration, apply the change and verify the service from its consumer's side. Inspect the logs and test restart and boot dependencies separately.

See the systemd.service manual.