How to Tell If a Server Is About to Run Out of Memory

By the time the OOM killer starts shooting down processes, it's already too late. Your app is down, your users are angry, and you're staring at kernel logs trying to piece together what happened. Memory exhaustion rarely arrives without warning — the problem is knowing what to look for before the system tips over the edge.

Why Free Memory Is a Misleading Number

The first instinct is to run 'free -h' and check how much memory is available. But on Linux, the number labeled 'free' is almost always smaller than what's actually usable. The kernel aggressively uses spare RAM for disk caching, which is a good thing — it speeds up I/O. What you actually care about is the 'available' column, not 'free'.

Run this to see the distinction clearly:

• free -h — look at the 'available' column on the second row, not 'free'
• cat /proc/meminfo | grep -E 'MemTotal|MemFree|MemAvailable|Buffers|Cached' — raw numbers direct from the kernel
• vmstat -s | head -10 — a quick summary of memory allocation

A server with 200 MB 'free' but 4 GB 'available' is healthy. A server with 200 MB 'available' out of 8 GB total is close to the edge, regardless of what the 'free' column says.

The Real Warning Signs to Watch

Low available memory is the headline metric, but a few other signals are often more telling — especially because they show up earlier.

• Swap usage climbing — if swap was at 0 yesterday and is now at 500 MB, something is consuming memory faster than expected. Check with 'free -h' or 'swapon --show'.
• High swap activity (not just usage) — a system swapping constantly is effectively thrashing. 'vmstat 1 5' shows 'si' (swap in) and 'so' (swap out) per second. Anything above a few hundred KB/s sustained is a red flag.
• OOM killer entries in logs — run 'dmesg | grep -i oom' or 'grep -i 'out of memory' /var/log/syslog'. These mean the kernel already had to kill something.
• Slab cache growing unchecked — kernel slab memory can leak. Check 'cat /proc/meminfo | grep Slab' and watch it over time.
• Process memory trending up — if a specific process RSS is growing hour over hour, that's a memory leak in progress.

Finding Which Process Is the Culprit

Once you suspect memory pressure, you need to know who's responsible. These commands help you identify the heaviest consumers:

• ps aux --sort=-%mem | head -15 — top 15 processes by memory percentage
• ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head -20 — more detail including parent PID
• cat /proc/<PID>/status | grep -E 'VmRSS|VmSwap' — check a specific process's resident and swapped memory
• smem -r -k 2>/dev/null | head -20 — if smem is installed, gives proportional set size (PSS), which is more accurate than RSS for shared memory

Pay special attention to the RSS (Resident Set Size) trend over time, not just the current snapshot. A process sitting at 1 GB RSS isn't necessarily a problem — a process that was at 200 MB two hours ago and is now at 1 GB definitely is.

Setting Up Proactive Alerts Before You Hit the Wall

Spot checks are not a monitoring strategy. You want to know when available memory crosses a threshold, not discover it during an incident. Here are practical approaches:

• Simple bash check: set a cron job to alert when available memory drops below a threshold — 'awk '/MemAvailable/ {if ($2 < 524288) print "WARNING: Low memory: " $2 " kB"}' /proc/meminfo' (this example triggers below 512 MB available)
• Use node_exporter + Prometheus alert rule — 'node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes < 0.1' fires when available memory falls below 10% of total.
• Set vm.swappiness appropriately — a lower value (e.g., 10) means the kernel prefers to keep data in RAM longer before swapping. Edit /etc/sysctl.conf: 'vm.swappiness=10', then 'sysctl -p' to apply.
• Check oom_score for critical processes — 'cat /proc/<PID>/oom_score' shows how likely the OOM killer is to target that process. Lower is safer. You can protect critical services by writing -1000 to /proc/<PID>/oom_score_adj.

Responding Before It Becomes a Crisis

If you catch memory pressure early, you have options. Drop the page cache without restarting anything: 'sync; echo 1 > /proc/sys/vm/drop_caches' — this clears cached pages and is safe to run on a live system (though it will briefly slow I/O as the cache warms back up). Restart the leaking process during a low-traffic window rather than waiting for the OOM killer to do it at the worst possible moment. If you're consistently running above 80–85% memory utilization, that's a capacity signal, not something to tune your way out of permanently.

This kind of continuous watching — tracking available memory trends, swap activity, and per-process RSS over time — is exactly where Opservo helps. Instead of setting up your own cron jobs and Prometheus rules, it monitors these signals automatically and flags anomalies in plain language before they become outages.

The OOM killer is the kernel's last resort. Your job is to act on warnings before it ever gets involved.

Memory exhaustion is one of the most predictable causes of server downtime — the pattern almost always shows up hours before the crash. Free memory drops, swap grows, a process's RSS keeps climbing. None of it is subtle if you know what to look at and you're actually watching.