ExitOnOutOfMemoryError: Why Your JVM Should Fail Fast

$ curl -s -w ' [http %{http_code}]\n' http://127.0.0.1:8089/health
ok [http 200]

$ curl -s http://127.0.0.1:8089/stats
queued=20 processed=3 consumerAlive=false

Green health check. Twenty jobs waiting. Three ever processed. The one thread that does the work died with an OutOfMemoryError, the JVM shrugged, and the HTTP server on a different thread kept answering "ok" to every probe you've got. Nothing restarts a process that says it's fine.

That's the problem -XX:+ExitOnOutOfMemoryError exists for, and it's the flag I'd put on every production JVM I run. But it doesn't catch every OutOfMemoryError, and I only found that out because I tested each one instead of trusting the docs page. Two of the most common ones sail straight past it.

Everything below ran on OpenJDK 21.0.12.1 (Ubuntu build), x86_64, 2 cores, no container. I had no Docker daemon and no Kubernetes cluster this session, so the pod parts are commands and YAML, not output.

Everything green, nothing working

The block at the top is real output from the lab in the next sections. Here's what the log of that same JVM looked like, which is the only place the death is visible:

Exception in thread "rules-consumer" java.lang.OutOfMemoryError: Java heap space
	at RulesHost.lambda$main$0(RulesHost.java:20)
	at java.base/java.lang.Thread.runWith(Thread.java:1596)
	at java.base/java.lang.Thread.run(Thread.java:1583)

One stack trace, written once, to stderr. After that the process looks exactly like a healthy one: jcmd <pid> Thread.print shows zero threads called rules-consumer, the PID is alive, the port is open, CPU is idle. Idle is the tell, and nobody alerts on idle.

I'd call this a zombie JVM. It isn't down, so nothing restarts it, and it isn't serving, so users get timeouts. It stays that way until a human notices.

Which OutOfMemoryErrors the flag actually sees

Before the lab, the answer to the question I couldn't find anywhere with measured output: for which errors does ExitOnOutOfMemoryError fire?

I wrote one class, Modes, that triggers each OutOfMemoryError I know how to trigger, and ran each with and without the flag. Same JDK, -Xmx64m unless noted.

Read the bottom three twice. If your pod dies from direct buffers or from thread exhaustion, this flag does nothing for you. Those two are the ones Netty services and thread-per-request services hit, so the gap isn't exotic.

I didn't chase the thread path to the line in HotSpot. The direct-buffer one I can explain from earlier work: that message is thrown by Java code in java.nio.Bits, not by the VM's allocator, so it never goes through the VM's OOM reporting. I haven't tested compressed class space or Map failed here. I'd expect the first to behave like Metaspace and I wouldn't assume anything about the second.

That's also the first reason I don't trust a bare flag. Which brings me to the lab.

Build a zombie in forty lines

Pinned: OpenJDK 21.0.12.1, single-file Java, no dependencies. A rules engine with one consumer thread draining a queue, and an HTTP server (com.sun.net.httpserver) with /health, /submit and /stats.

import com.sun.net.httpserver.HttpServer;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;

public class RulesHost {
    static final BlockingQueue<String> queue = new LinkedBlockingQueue<>();
    static final AtomicInteger processed = new AtomicInteger();
    static Thread consumer;

    public static void main(String[] args) throws Exception {
        consumer = new Thread(() -> {
            while (true) {
                try {
                    String kind = queue.take();
                    if (kind.equals("big")) {
                        List<byte[]> scratch = new ArrayList<>();
                        for (int i = 0; i < 200; i++) scratch.add(new byte[1 << 20]); // a rule set that explodes: the OOM
                    }
                    processed.incrementAndGet();
                } catch (InterruptedException | RuntimeException e) { // catches Exception, not Error: the usual shape
                    System.err.println("job failed: " + e);
                }
            }
        }, "rules-consumer");
        consumer.start();

        HttpServer s = HttpServer.create(new InetSocketAddress("127.0.0.1", 8089), 0);
        s.createContext("/health", x -> reply(x, 200, "ok"));
        s.createContext("/submit", x -> { queue.add(x.getRequestURI().getQuery().split("=")[1]); reply(x, 202, "queued"); });
        s.createContext("/stats", x -> reply(x, 200, "queued=" + queue.size() + " processed=" + processed.get()
                + " consumerAlive=" + consumer.isAlive()));
        s.start();
        System.out.println("up");
    }
    static void reply(com.sun.net.httpserver.HttpExchange x, int code, String body) throws java.io.IOException {
        byte[] b = body.getBytes(); x.sendResponseHeaders(code, b.length);
        try (OutputStream o = x.getResponseBody()) { o.write(b); }
    }
}

Step 1, compile and run with a heap small enough to make 200 MB impossible:

javac -d out RulesHost.java
java -Xmx64m -cp out RulesHost &

Step 2, feed it three normal jobs, then the poison one, then twenty more:

for i in 1 2 3; do curl -s "http://127.0.0.1:8089/submit?kind=small"; done
curl -s "http://127.0.0.1:8089/stats"           # queued=0 processed=3 consumerAlive=true
curl -s "http://127.0.0.1:8089/submit?kind=big"
for i in $(seq 20); do curl -s "http://127.0.0.1:8089/submit?kind=small"; done
curl -s "http://127.0.0.1:8089/health"          # ok
curl -s "http://127.0.0.1:8089/stats"           # queued=20 processed=3 consumerAlive=false

You've reproduced it when /stats says consumerAlive=false and /health still says ok. Takes a couple of seconds. (If your machine has curl pointed at a proxy, add --noproxy '*'; mine did.)

Step 3, same program, one flag:

java -Xmx64m -XX:+ExitOnOutOfMemoryError -cp out RulesHost &
curl -s "http://127.0.0.1:8089/submit?kind=big"
curl -s "http://127.0.0.1:8089/health"          # curl: connection refused (http 000)
wait $!; echo $?                                # 3

Terminating due to java.lang.OutOfMemoryError: Java heap space
exit=3

What I shrank: -Xmx64m (production would be gigabytes). What the confirmation signal is: exit code 3 and that "Terminating due to" line, within about two seconds. Teardown is kill on the first JVM, which I left running on purpose for the jcmd check. Don't forget it. It's a dead worker holding a port.

The one-paragraph version

The JVM treats an OutOfMemoryError as an ordinary Throwable. Whichever thread was allocating gets it, that thread unwinds, and everything else carries on. If the thread was a worker nobody supervises, you now have a process that's up and useless. -XX:+ExitOnOutOfMemoryError makes the VM terminate the whole process the moment it raises certain OOMs, so your orchestrator restarts a fresh one. Certain, not all. See the table.

Following it into HotSpot

The decision isn't made by your code, which is why the app that swallowed the error in my matrix still exited with 3. It's made at the point where the VM raises the error, in report_java_out_of_memory in src/hotspot/share/utilities/debug.cpp (tag jdk-21+35). Read in order, what it does:

1. A once-only guard. An atomic compare-and-swap on a static counter means only the first OOM in the process's life runs the rest. Remember that if you ever expect a second one to do something.
2. If HeapDumpOnOutOfMemoryError is on, dump the heap.
3. If OnOutOfMemoryError has a command, run it.
4. If CrashOnOutOfMemoryError is on, print Aborting due to java.lang.OutOfMemoryError: ... and call the fatal-error path.
5. If ExitOnOutOfMemoryError is on, print Terminating due to ... and call os::_exit(3).

I confirmed the order in the lab rather than trusting my reading. With all of -XX:+HeapDumpOnOutOfMemoryError, an OnOutOfMemoryError command that runs ls on the dump directory, and the exit flag, the output was:

java.lang.OutOfMemoryError: Java heap space
Dumping heap to /home/claude/lab/dumps/java_pid1230.hprof ...
Heap dump file created [35991399 bytes in 0.024 secs]
#   Executing /bin/sh -c "ls -l /home/claude/lab/dumps"...
-rw------- 1 root root 35991399 Oct 27 01:06 java_pid1230.hprof
Terminating due to java.lang.OutOfMemoryError: Java heap space
exit=3

The hook saw a finished dump, and only then did the process die. So the exit flag is safe to combine with the dump flag; it doesn't truncate the dump. (Dump setup, and the ways it still goes missing, is in HeapDumpOnOutOfMemoryError: production setup.)

Two details that surprised me. os::_exit skips everything: I registered a shutdown hook, and on a plain uncaught OOM it ran (SHUTDOWN HOOK RAN), but with the flag it never did. No hooks, no flushed buffers, nothing graceful. And OnOutOfMemoryError="a; b" isn't one shell command. HotSpot splits on the semicolon and executes each piece in its own /bin/sh -c, which the log shows plainly, one Executing line per piece. %p does expand to the PID there.

The reason threads and direct buffers escape is the interesting part, and it took me longer than I'd like to admit. I'd assumed "OutOfMemoryError" was one code path. It's several: some raised by the VM's allocator, which funnels through the reporting function, and some raised by Java library code or by thread startup, which don't. Getting from "the flag works" to "the flag works for these four" ate more than a day of re-runs and source reading, and it fits in one sentence: the flag guards the VM's own front door, and some errors come in the side window.

I couldn't find a JBS issue that frames that as a bug, and I'm not claiming it's one. If you know of one, tell me. The practical upshot is the same either way.

Pick the flag: exit, crash, or hook

Four flags, compared below, all in the JDK since 8u92 (the release notes famously misnamed them, which JDK-8154713 fixed).

CrashOnOutOfMemoryError printed this on my machine, and the hs_err file's header carries fatal error: OutOfMemory encountered: Java heap space from debug.cpp:271:

Aborting due to java.lang.OutOfMemoryError: Java heap space
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (debug.cpp:271), pid=1174, tid=1175
#  fatal error: OutOfMemory encountered: Java heap space

My rule: exit by default; crash only when you have somewhere to put core dumps and someone who'll read them (a 56 KB hs_err file for a Java-heap OOM told me nothing the heap dump wouldn't). And OnOutOfMemoryError="kill -9 %p", the one from old blog posts, is a worse ExitOnOutOfMemoryError: it spawns a shell while the JVM is out of memory.

Costs, because there are some. Every in-flight request on that JVM dies with it, including the 200 healthy ones next to the one that blew up. Nothing drains. A poison message that your broker redelivers will crash every replacement too, and Kubernetes will back you off into CrashLoopBackOff while it does. The flag doesn't fix that. It surfaces it in minutes instead of leaving it as a silent backlog.

And none of this is a fix for the leak or for the undersized heap. Raising -Xmx over a leak buys you hours, not a solution; check heap-after-GC first (finding the leak).

A JVM with an owner: probes, restart policy, no cleverness

The flag gets you a clean death. Something has to turn it into a restart, and something has to cover the errors the flag misses.

# Kubernetes: not run this session, no cluster. Standard fields.
spec:
  restartPolicy: Always          # default for a Deployment; exit 3 → container restarts
  containers:
    - name: rules-host
      env:
        - name: JAVA_TOOL_OPTIONS
          value: >-
            -XX:+ExitOnOutOfMemoryError
            -XX:+HeapDumpOnOutOfMemoryError
            -XX:HeapDumpPath=/dumps
      volumeMounts: [{ name: dumps, mountPath: /dumps }]
      livenessProbe:
        httpGet: { path: /live, port: 8089 }   # an endpoint you add: must test the worker, not "server is up"
        periodSeconds: 10
        failureThreshold: 3

Three things in there are doing real work. The dump path points at a mounted volume, because the container's own writable layer vanishes when the container restarts (stated from how Kubernetes works, not run here). The exit flag and the dump flag are both set, and per the order above they cooperate. And the liveness endpoint is a different endpoint from the one in my lab.

That last point is the second half of the answer. My /health returned "ok" if the HTTP server could answer, which says nothing about the consumer. A probe that answers from the same thread pool as the work proves nothing. Make the liveness check read something the worker owns: a "last job finished at" timestamp, or consumer.isAlive() (the consumerAlive field in my own /stats caught the dead thread; the probe path didn't look at it). That covers the unlisted cases too, the native-thread and direct-buffer failures where the flag stayed quiet. For those, the answer is still the right size limits and alerting; see unable to create native thread and the direct-buffer article, OutOfMemoryError: Direct buffer memory.

The other structural fix is boring: don't let a bare thread own critical work. A supervised executor with a handler that exits on Error is a few lines, and it survives the cases the flag misses. Catching Throwable and calling System.exit yourself is crude, but you control which errors count. (Error vs Exception goes into when catching Error is defensible.)

What to alert on

Alert on restarts, not on the flag. Specifically: container restart count above zero in the last hour, last exit code 3 on the pod (that's the signature of this flag; OOMKilled and 137 is the kernel doing it, a different problem, covered in OOMKilled in Kubernetes), and queue depth rising while processed count stays flat. That last pair would have flagged my lab in about a minute. More signals live in alerting before OutOfMemoryError.

Test it the way I did: a staging pod with a tiny heap, one poison request, and a check that the pod's lastState shows exit code 3 within seconds. If it doesn't, your flag isn't reaching the JVM (this happens when JAVA_TOOL_OPTIONS isn't what the entrypoint script actually uses).

Version safety: verified on 21.0.12.1 only. The flags exist since 8u92. I did not run JDK 17, 25, or any other collector than G1 (heap case) and Parallel (overhead case), so the "which OOMs trigger it" table is a 21 table. Re-run Modes on your target JDK before relying on the bottom rows; it takes five minutes.

The short list

• A thread dying of OutOfMemoryError doesn't stop the JVM. Your health check will keep saying "ok".
• -XX:+ExitOnOutOfMemoryError makes the VM exit with code 3, immediately, no shutdown hooks. Combine it with the heap dump flag; the dump finishes first.
• On 21.0.12.1 it fires for heap, GC overhead, array size and Metaspace. It does not fire for direct buffer memory, unable to create native thread, or an OOM you throw yourself.
• A liveness probe has to look at the worker, not at the web server.
• CrashOnOutOfMemoryError is for when you'll actually read the core. Otherwise, exit.

Questions I get about failing fast

Does ExitOnOutOfMemoryError write a heap dump first?
Only if -XX:+HeapDumpOnOutOfMemoryError is also set. In the lab the dump was complete before the exit (a 36 MB file, 0.024 s). Point it at storage that survives a restart.

What exit code does ExitOnOutOfMemoryError use?
3, on 21.0.12.1, via _exit. CrashOnOutOfMemoryError gave 134 (SIGABRT). A kernel kill shows as 137, which is a different mechanism entirely.

Is it safe to catch OutOfMemoryError instead?
Mostly not. The JVM may have failed halfway through allocating anything, and which thread gets the error is often not the thread that caused it. With the exit flag set, catching doesn't prevent the exit anyway; the VM decides before your catch runs. Details in how the JVM throws an OutOfMemoryError.

Why didn't the JVM exit on "unable to create native thread"?
In my run the flag wasn't applied to that error, and the same goes for direct buffer memory. Treat both as needing their own limits and their own alert. Overview of every message: OutOfMemoryError: every message.