jdk_lang test failures

[smlambert]

Test results from the openjdk regression test group run against the nightly openjdk8-openj9 build at AdoptOpenJDK: Test results: passed: 428; failed: 34

https://ci.adoptopenjdk.net/view/work%20in%20progress/job/test_personal/183/testReport/junit/
https://ci.adoptopenjdk.net/view/work%20in%20progress/job/test_personal/lastSuccessfulBuild/artifact/jvmtest/openjdk_regression/work/java/lang/

java/lang/Class/forName/NonJavaNames.sh
java/lang/Class/forName/arrayClass/ExceedMaxDim.java

java/lang/ProcessBuilder/Basic.java
java/lang/SecurityManager/CheckPackageAccess.java

java/lang/String/CaseInsensitiveComparator.java
java/lang/StringBuffer/Exceptions.java
java/lang/StringBuffer/TestSynchronization.java
java/lang/StringBuilder/Exceptions.java

java/lang/System/Versions.java
java/lang/Thread/ITLConstructor.java
java/lang/Thread/UncaughtExceptions.sh
java/lang/ThreadGroup/SetMaxPriority.java
java/lang/Throwable/SuppressedExceptions.java

java/lang/annotation/AnnotationsInheritanceOrderRedefinitionTest.java
java/lang/annotation/loaderLeak/Main.java

java/lang/invoke/ArrayConstructorTest.java
java/lang/invoke/CustomizedLambdaFormTest.java
java/lang/invoke/ExplicitCastArgumentsTest.java
java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java
java/lang/invoke/LFCaching/LFSingleThreadCachingTest.java
java/lang/invoke/LambdaFormTest.java
java/lang/invoke/MethodHandles/TestCatchException.java
java/lang/invoke/MethodHandlesTest.java
java/lang/invoke/PrivateInvokeTest.java
java/lang/invoke/ProtectedMemberDifferentPackage/Test.java
java/lang/invoke/RicochetTest.java
java/lang/invoke/VarargsArrayTest.java
java/lang/invoke/lambda/LambdaStackTrace.java

java/lang/ref/EarlyTimeout.java
java/lang/ref/NullQueue.java
java/lang/ref/OOMEInReferenceHandler.java
java/lang/reflect/DefaultMethodMembers/FilterNotMostSpecific.java

---

As links above expire, steps to reproduce locally (on x86-64_linux machine and assume cloned into /tmp):

Get test materials and SDK:
cd /tmp
git clone https://github.com/AdoptOpenJDK/openjdk-tests.git
cd /tmp/openjdk-tests
./get.sh -s /tmp/openjdk8_j9_nightlybuild -t /tmp/openjdk-tests -p x64_linux -v openjdk8-openj9 -r nightly

Update the excludes file to reinclude the test you want to retry:
/tmp/openjdk-tests/openjdk_regression/ProblemList_openjdk8-openj9.txt

Configure/compile/execute tests:
cd /tmp/openjdk-tests/TestConfig
export JAVA_BIN=/tmp/openjdk8_j9_nightlybuild/jre/bin
export SPEC=linux_x86-64_cmprssptrs
export JAVA_VERSION=SE80
export BUILD_LIST=openjdk_regression
make -f run_configure.mk
make compile
make jdk_lang

---

[pdbain-ibm]

Taking a look at this...

[smlambert]

I suspect a few of these tests may not applicable to openj9. If there are some that appear that we should exclude them, we will add them to the exclude list at AdoptOpenJDK.

Example: output from java/lang/System/Versions.java, test appears to expect a different java.class.version

Stacktrace
Execution failed: `main' threw exception: java.lang.Exception: Forgot to update java.class.version?
Standard Output
java.class.version=52.0
java.version=1.8.0-internal
java.vm.version=2.9
java.runtime.version=1.8.0-internal-jenkins_2018_02_06_14_01-b00
java.specification.version=1.8
majorVersion=52
minorVersion=0

Standard Error
java.lang.Exception: Forgot to update java.class.version?
at Versions.checkClassVersion(Versions.java:69)
at Versions.main(Versions.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.javatest.regtest.agent.MainActionHelper$SameVMRunnable.run(MainActionHelper.java:229)
at java.lang.Thread.run(Thread.java:811)

[pshipton]

@smlambert 52.0 is correct for Java 8. The test creates a class file containing only the class version and expects the VM to throw UnsupportedClassVersionError when the version is incorrect, even when the rest of the class file is missing. I expect OpenJ9 throws a different error.

We'll have to look at each of these tests one by one to see if any have uncovered an actual problem.

[pshipton]

OpenJ9 throws ClassFormatError JVMCFRE113 unexpected EOF; class=ClassVersionTest, offset=8 since the class file is not valid. I think the test is relying on undefined implementation details.

[sophia-guo]

Similar issue:
sun/misc/Version/Version.java
vm/verifier/VerifyProtectedConstructor.java
java/lang/invoke/RevealDirectTest.java

https://ci.adoptopenjdk.net/view/work%20in%20progress/job/test_personal/297/artifact/jvmtest/openjdk_regression/work/sun/misc/Version/Version.jtr/*view*/

[DanHeidinga]

Note: if the test is expecting UnsupportedClassVersionError & OpenJ9 is throwing ClassFormatError, it should be a straightforward change to match the behaviour as UCVE is a subclass of CFE.

https://docs.oracle.com/javase/9/docs/api/java/lang/UnsupportedClassVersionError.html

[pshipton]

@DanHeidinga are you suggesting to change the VM in order to pass this test?

Exception in thread "main" java.lang.ClassFormatError: JVMCFRE113 unexpected EOF; class=ClassVersionTest, offset=8
        at java.lang.ClassLoader.defineClassImpl(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:377)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:679)
        at java.net.URLClassLoader.access$400(URLClassLoader.java:86)
        at java.net.URLClassLoader$ClassFinder.run(URLClassLoader.java:1090)
        at java.security.AccessController.doPrivileged(AccessController.java:732)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:586)
        at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:933)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:878)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:861)
        at java.lang.Class.forNameImpl(Native Method)
        at java.lang.Class.forName(Class.java:367)

[DanHeidinga]

Yes - its arguably more clear to throw UCVE than CFE in this case and should be straightforward to do.

[pshipton]

Created #1282

33 failures remaining to analyze.

[pdbain-ibm]

I'll take a look at issue #1282.