SoftRefLRUPolicyMSPerMB in JVM Builds

This is a post in my JVM Args for Builds series where I break down how these settings affect JVM-based build systems. A number of these settings are complicated and all have compounding effects on each other as the JVM is a complex machine, but used the right way can be the difference between having a high-value feedback build system versus a high-cost one.

This property defines how fast soft references can be evicted by the JVM. As they are soft references, if they are evicted the program that depends on them will just have to spend time and resources recreating them. In the OpenJDK docs: [[1]]

Sets the amount of time (in milliseconds) a softly reachable object is kept active on the heap after the last time it was referenced. The default value is one second of lifetime per free megabyte in the heap. The -XX:SoftRefLRUPolicyMSPerMB option accepts integer values representing milliseconds per one megabyte of the current heap size (for Java HotSpot Client VM) or the maximum possible heap size (for Java HotSpot Server VM). This difference means that the Client VM tends to flush soft references rather than grow the heap, whereas the Server VM tends to grow the heap rather than flush soft references. In the latter case, the value of the -Xmx option has a significant effect on how quickly soft references are garbage collected.

The Problem in Server VMs with Large Heaps

Gradle and Kotlin daemons use the Server VM, create a ton of soft references, and have pretty large heap requirements compared to typical JVM apps. Those conditions paired with the default 1000 means that with a fairly minimal Android memory heap of 2GB means no attempt to release soft references will be made until 34 minutes! Hopefully your builds are faster than that, but this sure isn't helping if they are. So what does it look like if we lower this value?

Thankfully lowering this value is straightforward and in my testing on small and large projects across Kotlin, Dagger, KSP, Compose, Room, SQLDelight, KMP, KSP, etc I haven't seen any negative side effect.

What happens if we set it to zero? Well then soft references can be evicted immediately, but I have seen a slight performance degradation in that scenario.

Build System Recommendations

We should generally keep this value low. 1 seems like the best value for every Android project I've looked at so far, but that doesn't mean its the optimal setting for all Server VM projects. Your particular results on this change will vary - when I was at Hinge we saw a few % of total memory usage reduction when running 8-12gb heaps.

So you should be setting this in your JVM args:

-XX:SoftRefLRUPolicyMSPerMB=1

That way for a 1GB heap you'll flush soft references after 1 second, 8GB heap after 8 seconds. It will still scale with the size of the heap, but if we're talking about reclaiming memory in seconds instead of hours with no other performance impact then we're doing something right.

Thank You

Łukasz Wasylkowski for the inspiration years ago to look into this!

References

1. Advanced Garbage Collection Options [[1]]

[[1]]: Advanced Garbage Collection Options https://docs.oracle.com/javase/8/docs/technotes/tools/unix/java.html#BABFAFAE