Ability to configure ZGC to get its full potential

[estigma88]

To get the full advantage of ZGC, they recommend several setups as they stated in their documentation:

Setting the Heap Size:

This seems already done in the built pack using the memory calculator.

Returning Unused Memory to the Operating System:

This is configurable using JVM flags, but there is a note:

Note:Allowing the GC to commit and uncommit memory while the application is running could have a negative impact on the latency of Java threads. If extremely low latency is the main reason for running with ZGC, consider running with the same value for -Xmx and -Xms, and use -XX:+AlwaysPreTouch to page in memory before the application starts.

I am not sure if the memory calculator sets both -Xmx and -Xms as same in the current built pack, or if it is possible to tell the built pack to do that

Using Large Pages:

They stated:

Configuring ZGC to use large pages will generally yield better performance (in terms of throughput, latency and start up time) and comes with no real disadvantage, except that it's slightly more complicated to setup. The setup process typically requires root privileges, which is why it's not enabled by default.

The tricky part is the root privileges and also when to apply those changes in the OS. There seems to be also a relationship between the heap size and the large pages configuration.

Describe the Enhancement

Offer ways to setup ZGC options through env variables or similar

Possible Solution

Seems like there are setups easy to handle and other not that easy, and perhaps this is not the right pack for some of them

Motivation

Simplifying the ZGC setup to get its full potential will help people to switch to it easily. There are several studies cases where companies switched to ZGC and get a lot of good behaviours, for instance, Netflix

[dmikusa]

I am not sure if the memory calculator sets both -Xmx and -Xms as same in the current built pack, or if it is possible to tell the built pack to do that

tl;dr - Yes, the memory calculator will set both of these args.

A little background. A general assumption of the memory calculator is that your app is running inside a container where you're given a memory limit and that you want to get as much use out of that memory as possible (i.e. get the most value out of the memory you have available, as often you are paying based on the amount of memory you're given).

Given this, the memory calculator will attempt to configure the JVM so that it can use and take advantage of all the memory you have available. Part of this is configuring -Xmx to be a large as possible without going over your memory limit, and the other part is using -Xms so that the JVM eagerly grabs the memory that you're entitled to.

If the memory calculator isn't setting an argument that you need, like -XX:+AlwaysPreTouch, you an always add that to JAVA_TOOL_OPTIONS and the JVM will pick it up from there.

[dmikusa]

The tricky part is the root privileges and also when to apply those changes in the OS. There seems to be also a relationship between the heap size and the large pages configuration.

Yes, this is really tricky, not just for buildpacks but for running any containerized workload.

From the CNB standpoint. Buildpacks do not run as root, so the change could not be done by a buildpack. The CNB specification does have the concept of an Extension, which is similar to a buildpack, but will run as root. That could in theory, make this change, but it only has root privileges during the build (and a small part of the build lifecycle). When your application actually runs, there is no way to get root permissions. I have a suspicion that root privilege where the app actually runs is what's required here, and that's not something you'll get with a buildpack-generated image. Your application will run as a very limited user, which is done intentionally to follow good security practices.

I mentioned this isn't just a buildpacks problem, though, but a general container image problem. You may not be able to make the changes you need from inside a container. Containers run on the same system share a kernel, so if you need to change kernel configuration, that's not happening from inside the container. That would need to be done by an admin for your container orchestration platform.

I suspect that this portion of enabling ZGC would need to be something you coordinate with your platform orchestration team. For example, if you run on K8s. You could talk with your K8s admin team about making the kernel changes you need to a set of nodes and then allowing your workloads to run on those nodes. That would have the settings already applied, you would just need to signal your pods to run on those nodes.