The Silent Outage

A target has been down for hours and nobody noticed, because nothing alerts on up == 0.

A intermediate Prometheus & Grafana challenge worth 10 points. Solve it hands-on in a real Prometheus & Grafana environment in your browser - no local setup, no fake shells.

The Challenge

Scenario: A scrape target has been down for hours. Nobody caught it, because - as the saying goes - up doesn't belong to any one service team, so nobody ever wrote the rule watching it. This is a real, well-documented monitoring blind spot.

Your tasks:

1. Check /api/v1/targets and confirm a target really is down.
2. Add an alerting rule that fires on up == 0, with no job filter, so it covers every target platform-wide.
3. Reload Prometheus and confirm via /api/v1/alerts that the rule is pending or firing for the down target.

Commands to learn: the up metric, alerting rule expr and for:, curl localhost:9090/api/v1/alerts.

Run check when done.