VM Option -XX:hashCode=2 not working as expected

[guekho64]

I understand that every JVM implementation has different options, and that each one of them may or may not add compatibility for every option that any other JVM has, aside from the standard ones that are reglamentary, like -help.

That being said, I'd like to know if Eclipse OpenJ9 has support for using the -XX:hashCode option, and if that's the case, what could be causing the JVM to not preserve hash codes between different executions of a program.

I admit my scenario it's quite odd, as I use this option to mantain consistency of the output from Object#hashCode between executions of a determined program, being able to store the results for later use.

Hotspot JVM outputs the desired results, as expected, but Eclipse J9's won't mantaint consistency between executions.

I've already read the docs, but they don't mention anything about -XX:hashCode. I went searching into the source code hosted in this repo, trying to find something about this, with no success.

Also I've alredy added some additional java options to ensure -XX:hashCode isn't being ignored by the JVM, being those:

-XX:hashCode=2 -XX:+PositiveIdentityHash -XX:-IgnoreUnrecognizedVMOptions -XX:+UnlockExperimentalVMOptions

Is Eclipse OpenJ9 able to reproduce Hotspot JVM behavior? Or am I searching something unfindable?

[DanHeidinga]

OpenJ9 doesn't support the -XX:hashCode option.

I use this option to mantain consistency of the output from Object#hashCode between executions of a determined program, being able to store the results for later use.

Can you expand on what it does and why it's important to your use case? Would a JVMTI agent be a more portable choice to control the Object#hashCode value?

[pdbain-ibm]

OpenJ9's identity hashcode is based on the location of an object in the heap, which is non-deterministic and will generally vary from run to run. Are you calling theObject.hashCode() or System.identityHashCode(theObject)?

[guekho64]

@pdbain-ibm I've alredy tried both, with no success. Under normal circumstances, I only make use of theObject.hashCode().

@DanHeidinga The main reason I rely in -XX:hashCode, it's because I'm working on a certain API, and I'm trying to make some changes to it.

Unfortunately, because it's an API, I wasn't the only one who wrote some code based on it, so compatibility it's important. Here's an example of why I'm using -XX:hashCode

Code

public class BaseObject {
    public BaseObject (int objectID) {

        private int id = objectID;

        /* Original code made use of objectID, provided by any developer using the API,
            but there weren't any kind of restrictions on the value provided to the constructor,
            so, many times, different developers would use the same number, causing
            incompatibilities between their extensions. */

        // I had to override the value provided to the method, effectively ignoring the original value
        id = myValue();

        // Then, I had to register wich objectID was assigned to this object
        registerObject(this.get_id());
        

Once all the Objects who extended BaseObject were constructed, the registry was simply passed to a method that converted all the values inside to strings, saving the results into a file.

Now, here comes the interesting part: Because ID's were used later by both parts (API and Developer) to identify Objects and assign some specific properties to each one of them, I had to redirect each one of the custom ID's to the original Objects, but there wasn't a reliable way to tell which Object was which if there were two identical ID's.

That's why I used Object.hashCode() to determine the real identity of a determined Object, because, as far as -XX:hashCode=2 was added to the JVM arguments, Object1.hashCode() would always return a determined value, alongside other parameters I made use of, like the name of the package of the caller class and stuff like that, to ensure Objects would never have the same ID...

In fact, this is a little bit more complex, as the source code of the API is lost and the only way I was able to do all the aforementioned stuff was through ASM, obviously involving bytecode manipulation and things like that...

[JasonFengJ9]

@guekho64 Hotspot JVM -XX:hashCode=2 appears just return a constant 1 for most objects hashCode() except String objects from my quick experiments. So if your target objects are not String, Object1.hashCode() results could be hardcoded as 1 or just be ignored. If String objects have to be handled, my thought is to use the String values directly just like package name and other staff you mentioned.
With these modification, I hope your program can adopt OpenJ9 JVM and also doesn't require -XX:hashCode=2 -XX:+UnlockExperimentalVMOptions.

[guekho64]

You're almost right. In fact, -XX:hashCode should return 1 when asking for an Object's hashCode, but, as far as I've seen, I believe that when Objects have more properties inside of them, such as Inner Classes, Methods, etc, the actual hashCode returned isn't 1, as it should be, or at least as docs say...

So, whether it's a bug or not, I make use of that property made available by -XX:hashCode=2, and you could even see it by yourself:

Just create a random Object, and if you wish, add some extra ingredients to it, such as some methods, maybe one inner class, or some random values...and invoke hashCode() on it to see what value you get!

[JasonFengJ9]

@guekho64 what Hotspot JVM version/platform do you run against the program?

[guekho64]

@JasonFengJ9 sorry for the late response. Here's what java -version outputs:

openjdk version "1.7.0_201"
OpenJDK Runtime Environment (Zulu 7.25.0.5-linux64) (build 1.7.0_201-b02)
OpenJDK 64-Bit Server VM (Zulu 7.25.0.5-linux64) (build 24.201-b02, mixed mode)

Although I've been using different JVM's since I started with this project of mine, including:

• OpenJDK 7/8
• Zulu JDK 7/8
• OpenJDK 8 (w/Shenandoah GC Patches)
• Oracle's JDK 7/8
• OpenJDK 8 (w/Eclipse J9's JVM)

And I haven't seen nothing out of usual when running my program with the discussed JVM option...

In fact, the program runs even better with Eclipse J9's JVM than with Hotspot, but there's this little detail that keeps me from using it for my project: The lack of support from this JVM for using the -XX:hashCode option, or a way to force the JVM to not scramble hashCodes every time the program is executed.

I know that there are many reasons to not preserve hash codes from execution to execution, like security and alike, and as such, I see 3 possibilities to solve this particular issue:

• Include support for -XX:hashCode JVM option, so user can decide whether or not hashCodes should change. (Like Hotspot does)

• Provide patches for users who might want this functionality...

And the last one:

• If none of the above is viable, maybe just giving out some clues on how to modify the source code should suffice, so people could just create custom builds of the JVM

Anyways, this is a personal project, so, in case that Eclipse OpenJ9 developers reject having official support for this feature, I can deal with it, just having a custom build of this JVM would be enough for me...although It may be useful for someone else, but I recognize that not always is that easy to add random features requested by people...

Another option could be to go back to Hotspot, but It'll be hard now that I know about this new virtual machine implementation, sigh...

[JasonFengJ9]

@guekho64 glad to hear that "In fact, the program runs even better with Eclipse J9's JVM than with Hotspot".
To preserve hash codes from execution to execution, a quick thought is to provide java.util. Random alike mechanism to generate pseudorandom numbers within java.lang.Object.hashCode() via natives.
Will let you know when such a patch is available for test.

[guekho64]

Thanks for letting me know! I'll look forward to it.

Something else...

Just as a side note: I'm no expert about this JVM, but I bet that a new JVM Option would be the way to go rather than a new method, 'cause that way you would avoid modifying the `Object` class, not to mention that the new method would be an exclusive method from this virtual machine, and code using this new method will have problems finding it if ever run on another machine, with a chance that it'll be running a Hotspot based JVM...

(Maybe options go here? https://github.com/eclipse/openj9/blob/838c53334b93b636da3019586170a3bc200647a2/runtime/oti/jvminit.h)

Another option that I didn't take in account earlier could be to add a special annotation, so objects that must have a constant hash between executions will be marked with this annotation. Obviously this isn't so "friendly" in terms of compatibility, as the annotiation would be available only in this JVM...

I've always been worried about compatibility issues, as I mostly work with API's and extensions, where every piece of code should work flawlessly without causing problems with someones else's code, so I apologize If i'm asking too much about this. I trust that, whatever decision is taken, it'll be the most efficient one for the project

[JasonFengJ9]

@guekho64 regarding the side note, you shouldn't have to modify the program to invoke a new method, either a -XX:hashCode JVM option or a system property will be needed to use the feature requested.
Note: a patch will be a jar file (for a java only solution) or a changeset (native code modification) to build a custom OpenJ9 JVM.
Thanks for your trust on OpenJ9 project!

[JasonFengJ9]

@guekho64 considering this is your personal project, I would propose a Java only patch to generate hash codes via java.util.Random within java.lang.Object. This function can be enabled via a system property preserveHashCode.

Important note: This patch doesn't apply to String objects which are handled differently by natives (clarification: String hashcode is calculated based on the String content hence its hashcode already is preserved from execution to execution). In addition footprint is going to be increased and the performance might be affected as well.

If your program can't tolerate those limitations, a native implementation need to be investigated instead.

Attached jar file is for OpenJ9 Java 8 and can be applied via -Xbootclasspath/p:/path/to/patch-openj9-java8.zip along with -DpreserveHashCode=true.
patch-openj9-java8.zip

[guekho64]

Thanks for putting your own effort into this!

I'm gonna try this patch as soon as possible, since I'm not in my developing environment right now, but I'll let you know if this worked at all or not.

How hard would it be to implement this behavior for String's too? As I said before, I use some Strings alongside other factors to determine which Object is which, and they're really useful because they contain the name of the class who invoked the method, and those names have very little chances to be repeated by someone else (Unless it's done on purpose...), so you could say that this acts as a way to confirm (Even more) if an Object it's the one being search...

A little hit on performance shouldn't be something to worry about, unless it really affects the overall speed.

If necessary, I offer you my help to try to implement this.

[JasonFengJ9]

@guekho64 no problem, it's our job to help out OpenJ9 usages, :)
Just to clarify about String hashcode, actually it was calculated based on the String content hence its hashcode already is preserved from execution to execution even without this patch.
So please give this patch a try and see if it unblocks your program from adapting OpenJ9 jvm.
I don't believe the speed will be a big hit but you will find out if it is noticeable for your program.

From the specification perspective (https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Object.html#hashCode()), This (hashcode) integer need not remain consistent from one execution of an application to another execution of the same application. , preserving hashcode from execution to execution is not required by Java spec.

Note: I don't expect this patch to be accepted into OpenJ9 repo due to the footprint increasing. A proper implementation would be made within natives.

OpenJ9 project always welcome contributions, big or small. If you would like to take this implementation, I (and others) will help. Thanks for your interest.