Disk Full Before Morning

Prometheus has no retention.size flag against a fixed disk budget and is heading for a crash-loop.

A advanced Prometheus & Grafana challenge worth 20 points. Solve it hands-on in a real Prometheus & Grafana environment in your browser - no local setup, no fake shells.

The Challenge

Scenario: Prometheus's data directory sits on a disk with a fixed, small budget, but nobody ever set --storage.tsdb.retention.size. Left alone, it will fill the disk overnight and crash-loop by morning.

Your tasks:

  1. Check the current disk budget available to Prometheus's data directory.
  2. Set --storage.tsdb.retention.size to roughly 80-85% of that budget, per Prometheus's own guidance, leaving headroom for in-progress compaction.
  3. Restart Prometheus with the new flag and confirm it's actually applied.

Commands to learn: --storage.tsdb.retention.size, df, checking a running process's flags.

Run check when done.