Linux OOM Kill Forensics: Tracing a Production Memory Crisis

The Incident

A payment webhook service goes silent at 3:14 AM. It auto-restarts, appears healthy, then crashes again 40 minutes later. By morning this has happened four times.

---

Step 1: Confirm It Is an OOM Kill

dmesg -T | grep -i "killed process|out of memory|oom"

[2026-08-20 03:14:52] Out of memory: Kill process 18234 (java) score 892 or sacrifice child
[2026-08-20 03:14:52] Killed process 18234 (java) total-vm:8492132kB, anon-rss:7891204kB

Confirmed. The process consumed 7.8 GB of RSS — the host has 8 GB.

---

Step 2: Check Current OOM Scores

for pid in $(ps -eo pid --no-header | head -20); do
  echo "$(cat /proc/$pid/oom_score 2>/dev/null) $pid $(cat /proc/$pid/comm 2>/dev/null)"
done | sort -rn | head -10

---

Step 3: Watch the Restarted Process

pgrep -f "payment-service"
watch -n 2 "cat /proc/18456/status | grep -E 'VmRSS|VmSize|VmSwap'"

VmRSS:   7891204 kB
VmSwap:        0 kB

7.8 GB RSS, no swap. When it hits the limit the kernel has no buffer — it kills immediately.

---

Step 4: Identify the Root Cause

journalctl -u payment-service --since "2026-08-20 02:00" --until "2026-08-20 03:15"

At 02:47 AM: 340 concurrent webhook requests versus the normal 40. Each triggers an XML parser that allocates large buffers. The parser pool had no memory ceiling and no JVM heap limit was set.

---

The Fix

1. JVM heap limit: Add -Xmx4g -Xms2g to the startup command. The JVM throws OutOfMemoryError and the service restarts cleanly rather than being killed by the kernel mid-transaction.

2. Swap as a buffer:

fallocate -l 4G /swapfile && chmod 600 /swapfile
mkswap /swapfile && swapon /swapfile
echo '/swapfile none swap sw 0 0' >> /etc/fstab

3. systemd cgroup limit:

[Service]
MemoryMax=5G
MemorySwapMax=2G

---

Post-Mortem Summary

Root cause: unbounded XML parser allocation under traffic spike. Contributing: no JVM heap cap, no swap. Fix: application-level heap limit + swap buffer + cgroup ceiling with clean restart semantics.

Practice OOM forensics in the ShellGenius Linux Labs — the oom-kill-forensics challenge has the exact tools used here.