I
A teammate of mine once spent two days chasing a service that kept getting killed. Every dashboard said the memory was fine. It sat at 60%, comfortably. And every few hours the thing died anyway. The problem was that he was watching the wrong memory.
This is probably the single most common misunderstanding in Java operations, and it costs teams real money and real sleep. So here's the whole thing explained without assuming you've read the JVM tuning manual.
Think of your container as an apartment with a strict size limit. Say 512 MB. Go one square foot over and the landlord, in this case the Linux kernel, throws you out. No warning, no negotiation. That's an OOM kill.
Now, most people assume the apartment holds one big room: the heap. The heap is where your application's objects live, the user records, the JSON payloads, the cached data. It's the part you control with the famous -Xmx setting, and it's the part every dashboard shows you.
But the apartment also has a hallway, a closet, and a utility room.
Java needs those too:
None of that counts toward
So here's the rule that solves 80% of these incidents: give the heap no more than 70% of your container limit. For a 512 MB container, the heap gets about 320 MB, and the other 190 MB covers everything above. Set -Xmx to 500 MB in a 512 MB container and you will be OOM-killed. It's not a question of if.
In the monolith era, you had one Java process, one heap, and usually one person whose job included watching it. If it got a little fat, you gave it more memory and moved on.
Now you might have a hundred services. Each one has a memory ceiling; each one can die on its own, and when one dies mid-request, the services calling it retry. Those retries pile up. One badly sized service can take down a whole business domain in about ninety seconds.
The cost side changed too. Nobody ever audited the 16 GB you handed the monolith. But over-provision twenty services by 512 MB each and you're renting 10 GB of RAM that does absolutely nothing, every hour, forever. That shows up on a bill somebody eventually reads.
Java cleans up after itself. A background process called the garbage collector finds objects nobody is using anymore and reclaims that space. Two things about it are worth understanding.
First: most objects die almost immediately. Roughly 98% of the objects your service creates are garbage within milliseconds: temporary strings, parsed request bodies, throwaway calculations. Java is very good at cleaning these up cheaply. It keeps them in a separate area and sweeps it constantly.
The objects that survive get moved into a longer-term area. Cleaning that up is expensive, and it's usually what causes visible problems.
Second: cleaning up sometimes pauses your application. Depending on which garbage collector you use, requests can freeze for anywhere from under a millisecond to several seconds while cleanup happens. If you have latency commitments, this is the thing that breaks them.
You get to choose which collector to use, and most teams never make that choice on purpose.
Here's the short version:
There are others (Shenandoah, Epsilon), but they're for narrower situations. G1 has been the default since Java 9, and it's the right answer for most services. Move to ZGC only if you can point to a latency number you're missing; it buys you shorter pauses by spending more CPU, and that's a bad trade if pauses aren't hurting you.
There's no universal right heap size. It depends on how much your service allocates and how long that data sticks around.
But you can get close:
Then work backwards to the container limit. Take your heap number and add roughly 400–500 MB for a typical Spring Boot service to cover Metaspace, code cache, threads, buffers, and OS overhead, then add 15% breathing room. A 1 GB heap usually wants about a 1.75 GB container.
Then, and this is the part everybody skips, run a load test and check. Watch heap usage, how often collection runs, and how long the pauses are. Tune until garbage collection uses less than 5% of CPU and pauses fit inside whatever your users will tolerate. A heap size you guessed is a heap size that will surprise you in production.
Two settings worth knowing beyond -Xmx:
Setting the minimum equal to the maximum stops Java from slowly asking the OS for more memory during a traffic spike, which is exactly the moment you can't afford the delay.
A memory leak in Java means something is holding a reference to data that should have been thrown away, so the garbage collector can't touch it. Memory creeps up over days, and then the service falls over at the worst possible time.
In my experience, almost every one is one of these four:
When you suspect a leak, the process is boring and it works: confirm memory is climbing after cleanup runs (not just at peak), capture a memory snapshot, take a second one thirty minutes later, and compare. Whatever is growing between the two snapshots is your culprit. Fix it, then run the service under load for an hour and confirm the line goes flat.
Set this on every service today, so you have evidence when it happens:
Both are nearly free, and they're the difference between diagnosing a 3 AM outage and guessing at it.
You don't need an elaborate setup. Micrometer collects Java memory metrics automatically, Prometheus stores them, Grafana draws them.
That's it, and it's an afternoon of work.
Alert on five things:
Put a link to a runbook in every alert. Nobody woken at 3 AM remembers a checklist.
Give the heap 70% of the container, not 95%. Set the minimum equal to the maximum. Cap Metaspace. Turn on garbage collection logging and automatic memory dumps before you need them. Use G1 unless you can name the latency number that requires ZGC. Load test instead of guessing. Watch memory-after-cleanup, not peak memory.
And if you do one thing after reading this: turn on garbage collection logging in your staging environment today, open the log in a GC analysis tool, and look at what your service has actually been doing. It's rarely what people assume.
This is probably the single most common misunderstanding in Java operations, and it costs teams real money and real sleep. So here's the whole thing explained without assuming you've read the JVM tuning manual.
The apartment analogy
Think of your container as an apartment with a strict size limit. Say 512 MB. Go one square foot over and the landlord, in this case the Linux kernel, throws you out. No warning, no negotiation. That's an OOM kill.
Now, most people assume the apartment holds one big room: the heap. The heap is where your application's objects live, the user records, the JSON payloads, the cached data. It's the part you control with the famous -Xmx setting, and it's the part every dashboard shows you.
But the apartment also has a hallway, a closet, and a utility room.
Java needs those too:
- Metaspace, where Java stores information about your code itself: which classes exist, what methods they have. By default, this has no size limit at all.
- Code cache: Java watches which parts of your program run most, then compiles those into faster machine code. It has to put that somewhere.
- Thread stacks: every thread handling a request gets its own small workspace, roughly half a megabyte. Two hundred threads is 100 MB before your application stores a single thing.
- Direct memory: buffers used for network and file operations, sitting outside the normal heap.
None of that counts toward
-Xmx. None of it shows on a heap dashboard. And all of it gets you evicted.
So here's the rule that solves 80% of these incidents: give the heap no more than 70% of your container limit. For a 512 MB container, the heap gets about 320 MB, and the other 190 MB covers everything above. Set -Xmx to 500 MB in a 512 MB container and you will be OOM-killed. It's not a question of if.
Why this matters more than it used to
In the monolith era, you had one Java process, one heap, and usually one person whose job included watching it. If it got a little fat, you gave it more memory and moved on.
Now you might have a hundred services. Each one has a memory ceiling; each one can die on its own, and when one dies mid-request, the services calling it retry. Those retries pile up. One badly sized service can take down a whole business domain in about ninety seconds.
The cost side changed too. Nobody ever audited the 16 GB you handed the monolith. But over-provision twenty services by 512 MB each and you're renting 10 GB of RAM that does absolutely nothing, every hour, forever. That shows up on a bill somebody eventually reads.
Garbage collection, in one section
Java cleans up after itself. A background process called the garbage collector finds objects nobody is using anymore and reclaims that space. Two things about it are worth understanding.
First: most objects die almost immediately. Roughly 98% of the objects your service creates are garbage within milliseconds: temporary strings, parsed request bodies, throwaway calculations. Java is very good at cleaning these up cheaply. It keeps them in a separate area and sweeps it constantly.
The objects that survive get moved into a longer-term area. Cleaning that up is expensive, and it's usually what causes visible problems.
Second: cleaning up sometimes pauses your application. Depending on which garbage collector you use, requests can freeze for anywhere from under a millisecond to several seconds while cleanup happens. If you have latency commitments, this is the thing that breaks them.
You get to choose which collector to use, and most teams never make that choice on purpose.
Here's the short version:
Collector | Pause length | Use it when |
G1 | Around 200ms, tunable | Almost always. This is the sane default for a normal web service. |
ZGC | Under 1ms | You have a real latency requirement , trading, real-time bidding, anything where a 200ms freeze is a business problem. |
Parallel | 50ms to 500ms | Batch jobs, where total throughput matters and nobody's waiting on a response. |
Serial | Up to seconds | Tiny command-line tools. Not your service. |
There are others (Shenandoah, Epsilon), but they're for narrower situations. G1 has been the default since Java 9, and it's the right answer for most services. Move to ZGC only if you can point to a latency number you're missing; it buys you shorter pauses by spending more CPU, and that's a bad trade if pauses aren't hurting you.
Sizing without guessing
There's no universal right heap size. It depends on how much your service allocates and how long that data sticks around.
But you can get close:
- Small API doing simple work: 256–512 MB heap
- Typical Spring Boot service: 512 MB – 1 GB
- Heavy service holding a lot in memory: 1–2 GB
- Batch job: size it to the data it processes, usually 2 GB and up
Then work backwards to the container limit. Take your heap number and add roughly 400–500 MB for a typical Spring Boot service to cover Metaspace, code cache, threads, buffers, and OS overhead, then add 15% breathing room. A 1 GB heap usually wants about a 1.75 GB container.
Then, and this is the part everybody skips, run a load test and check. Watch heap usage, how often collection runs, and how long the pauses are. Tune until garbage collection uses less than 5% of CPU and pauses fit inside whatever your users will tolerate. A heap size you guessed is a heap size that will surprise you in production.
Two settings worth knowing beyond -Xmx:
Xms384m -Xmx384mset the minimum equal to the maximumXX:MaxMetaspaceSize=128mput a ceiling on the "no limit by default" part
Setting the minimum equal to the maximum stops Java from slowly asking the OS for more memory during a traffic spike, which is exactly the moment you can't afford the delay.
Four leaks that cause most of the incidents
A memory leak in Java means something is holding a reference to data that should have been thrown away, so the garbage collector can't touch it. Memory creeps up over days, and then the service falls over at the worst possible time.
In my experience, almost every one is one of these four:
- A cache with no expiry. Somebody used a plain map as a cache. Things go in. Nothing ever comes out. It grows until the service dies. Use a real cache library with a size limit and a time-to-live.
- A class-loading leak. Some frameworks and plugin systems load code dynamically and never release it. This one is nasty because it fills Metaspace, not the heap , so your heap dashboard stays green right up until the pod dies.
- A forgotten listener. A component signs up for notifications, gets destroyed, and never unsubscribes. The notification system is still holding onto it, so it can never be cleaned up.
- Thread-local data that's never cleared. A request stores something on its thread, finishes, and the thread goes back into the pool without clearing it. Those threads get reused for the life of the process, so the data effectively lives forever.
When you suspect a leak, the process is boring and it works: confirm memory is climbing after cleanup runs (not just at peak), capture a memory snapshot, take a second one thirty minutes later, and compare. Whatever is growing between the two snapshots is your culprit. Fix it, then run the service under load for an hour and confirm the line goes flat.
Set this on every service today, so you have evidence when it happens:
XX:+HeapDumpOnOutOfMemoryErrorXX:HeapDumpPath=/dumps/Xlog:gc*:file=/logs/gc.log
Both are nearly free, and they're the difference between diagnosing a 3 AM outage and guessing at it.
What to actually watch
You don't need an elaborate setup. Micrometer collects Java memory metrics automatically, Prometheus stores them, Grafana draws them.
That's it, and it's an afternoon of work.
Alert on five things:
- Heap above 85% of its maximum
- Garbage collection eating more than 10% of CPU
- Pause times above what your users tolerate
- Any full, everything-stops collection happening at all
- Memory after cleanup trending upward; this is the leak signal, and it's the one people forget
Put a link to a runbook in every alert. Nobody woken at 3 AM remembers a checklist.
The short version
Give the heap 70% of the container, not 95%. Set the minimum equal to the maximum. Cap Metaspace. Turn on garbage collection logging and automatic memory dumps before you need them. Use G1 unless you can name the latency number that requires ZGC. Load test instead of guessing. Watch memory-after-cleanup, not peak memory.
And if you do one thing after reading this: turn on garbage collection logging in your staging environment today, open the log in a GC analysis tool, and look at what your service has actually been doing. It's rarely what people assume.