JVM: Solving OutOfMemoryError with less Memory

At work we have 6 web applications (WAR) deployed in Glassfish v2.

In production we experienced sporadic java.lang.OutOfMemoryError: Java heap space under high load. We where sure that we did not have a classic Java memory leak since the used HEAP space decreased after some time and returned to normal. We suspected that the problem was related to our use of EHCache (which stores cached objects in HEAP space).

(Note: This blog post is a summary of two days of research - we tried many things and many numbers - on several more or less identical servers - so the numbers and values in this post is approximately correct - but you get the point)

The JVM, and therefor GlassFish, was running with max 768 MB of HEAP space and 385 MB of PermGEN space.

To reduce the possibility of getting OutOfMemoryError until we had worked out the issue, we decided to increase the maximum HEAP space the JVM could allocate (-Xmx).

GlassFish runs on a 4 core 32 bit server with Windows Server 2003 with 8 GB of RAM.

We increased the the max HEAP memory size to 1 GB (-Xmx=1024M). We started GlassFish with no applications deployed. Then deployed one application after the other - 6 WARs. All applications where deployed without problem and our Apps run fine. After some time the JVM suddenly died.

We found a crash JVM crashdump. We didn't read it to carefully, but it talked about OutOfMemoryError. We did some more research and found out that it had died before the HEAP space had reached 1GB. We thought a solution was to set the initial HEAP size that the JVM should initialize. We told the JVM to initialize all the HEAP at startup (-Xms=1024M).

So now we had 1024MB HEAP and 385MB PermGEN which is a total of 1409MB.

When we then again started GlassFish (with no apps deployed), The JVM and GlassFish started up just fine. So we started to deploy applications - one by one. When GlassFish was in the middle of deploying the second application the JVM died.. So by allocating more memory up front, the JVM died with OutOfMemoryError earlier..

After a lot of research and reading this great post: http://www.codingthearchitecture.com/2008/01/14/jvm_lies_the_outofmemory_myth.html, this is how we concluded:

We took a closer look at the JVM crash dump:

java.lang.OutOfMemoryError: requested 884680 bytes for Chunk::new. Out of swap space?

It also says that the JVM crashed in this thread:

0x5be76800 JavaThread "CompilerThread1" daemon [_thread_in_native, id=6764, stack(0x5c1a0000,0x5c1f0000)]

We had configured the JVM to use a lot of memory for HEAP and PermGen. A Windows process can use max 2 GB total. The internals of the JVM (e.g its JIT compiler) needs its own memory, so do the DLLs loaded. Since so much of those 2GB was already allocated for HEAP/Permgen, windows said NO when the JVM asked for more memory inside CompilerThread1. When this happened, the JVM crashed with java.lang.OutOfMemoryError?: requested 884680 bytes for Chunk::new. Out of swap space?

Solution:

Tell the JVM to use LESS memory..