Java Memory calculator as standalone command

[jonnybbb]

Hi there,
I stumbled across the memory calculator in the https://github.com/paketo-buildpacks/libjvm repository and I am wondering if it can be reused in a docker image outside the buildpack ecosystem?

[dmikusa]

We don't package or distribute it that way, but I don't see why not. We bundle the memory calculator and some other "helpers" into a single Go binary that gets installed into containers generated by Paketo Buildpacks. You can see the code here, where we're bundling this into the binary, https://github.com/paketo-buildpacks/libjvm/blob/main/cmd/helper/main.go#L45-L50. There are other helpers you could include, but you don't have to.

The binary just needs to run before your application. With buildpacks, that happens automatically, but you could do something like helper && your-app or bash -c 'helper && your-app' for your start command and make sure it runs before your app.

Essentially, if there is a problem the binary will fail so your app will fail to start. When successful, the binary sets values in the env variable JAVA_TOOL_OPTIONS, so your process should inherit those values, and the JVM will read and apply them.

[jonnybbb]

Thanks for the quick reply - that sounds promising. I am wondering if I can build the cmd/helper/main.go and then use it to run the other helpers by providing an argument like helper memory-calculator && your-app

[dmikusa]

The cmd/helper/main.go will run all of our helpers. For efficiency, they get compiled into one binary and run all in one execution. Some helpers will no-op, depending on their criteria for actually doing something. For example, some only run for pre-Java 9, and some only run post-Java 9.

You can modify the source code & recompile the helper binary to include any set of helpers you want to run. If you wanted to trigger specific helpers based on input arguments, you could do that but the present binary does not work that way.

[jonnybbb]

This works great so far. Any idea why it fails to write to /dev/fd/3

Calculating JVM memory based on 39894916K available memory
For more information on this calculation, see https://paketo.io/docs/reference/java-reference/#memory-calculator
Calculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx39369244K -XX:MaxMetaspaceSize=13671K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 39894916K, Thread Count: 250, Loaded Class Count: 0, Headroom: 0%)
unable to write environment
write /dev/fd/3: bad file descriptor

[jonnybbb]

./memory-calculator 3>&1

Does the trick

[dmikusa]

Yes, sorry. Forgot about that, and misspoke slightly before. The binary is what's called an exec.d process in the buildpack spec, see https://github.com/buildpacks/spec/blob/main/buildpack.md#execd-output-toml.

It expects a third output stream. STDOUT & STDERR work the same, and the third output stream is where it writes the environment variables that it wants to be applied. I forgot about this earlier and misspoke. The binary cannot set env variables directly, so it writes them in this third output stream. The buildpack launcher would then take these and apply them before starting the actual application.

So when I said you could helper && your-app and that's not quite right. The env variables that the helper wants to set, i.e. JAVA_TOOL_OPTIONS would be written to that third stream not set as an env. You'd need a slightly more complicated wrapper script to apply those and then start your application.