Maintenance - Tuning Tip for Spring Boot (if using germain APM < 8.5.15)
Issue
Our Spring Boot service (used for Session Replay) was crashing running out of memory generating this exception:
Exception:2019-01-26 17:44:00,006 INFO SessionTrackingService - Processed 130 sessions. - [pool-2-thread-1] 2019-01-26 17:44:03,733 ERROR FileBackedStorageEngine - Failed to allocate a direct buffer for a FileChannel write. Consider increasing the -XX:MaxDirectMemorySize property to allow enough space for the FileChannel transfer buffers - [Ehcache [_default_]-1147] 2019-01-26 17:44:04,633 ERROR FileBackedStorageEngine - Failed to allocate a direct buffer for a FileChannel write. Consider increasing the -XX:MaxDirectMemorySize property to allow enough space for the FileChannel transfer buffers - [Ehcache [_default_]-1148] 2019-01-26 17:44:58,060 INFO SessionCache - Flushing stale sessions before: 2019-01-26T16:44:58.060 - [pool-2-thread-1] 201
Tuning solution
Please apply this tuning recommendation to your environment if you are on Germain 8.5.14 or prior:
Add parameter -Djdk.nio.maxCachedBufferSize=262144 to the session services to prevent them from caching large files in memory in these files:
Windows: $germain/.../ServiceDistribution/src/main/bin/GermainSessionTrackingServices.xml
Unix: $germain/.../ServiceDistribution/src/main/bin/session-tracking/GermainSessionTrackingServices.xml
Java options would look like this:
.../...
JAVA_OPTS="-Xms2g -Xmx4g -XX:+CrashOnOutOfMemoryError -Djdk.nio.maxCachedBufferSize=262144"
.../...