CRIU failing on cent7-ppcle

[pshipton]

https://openj9-jenkins.osuosl.org/job/Test_openjdk11_j9_sanity.functional_ppc64le_linux_Nightly_testList_1/603 - cent7-ppcle-2

Seems all CRIU tests failed on this machine. This is due to ibmruntimes/openj9-openjdk-jdk11#658
Disabled the machine.

 [OUT] Pre-checkpoint
 [OUT] Performing CRIUSupport.checkpointJVM(), current thread name: main, Thu Sep 14 04:54:11 ADT 2023, System.currentTimeMillis(): 1694678051195, System.nanoTime(): 11639303229211012
 [OUT] CRIU needs to have the CAP_SYS_ADMIN or the CAP_CHECKPOINT_RESTORE capability: 
 [OUT] setcap cap_checkpoint_restore+eip criu
 [OUT] Exception in thread "main" org.eclipse.openj9.criu.SystemCheckpointException: Could not dump the JVM processes, err=-52
 [OUT] 	at openj9.criu/org.eclipse.openj9.criu.CRIUSupport.checkpointJVMImpl(Native Method)

[pshipton]

@AdamBrousseau @TobiAjila

[AdamBrousseau]

related #18135 (comment)
Last time I checked the outside plinux machines had still not been upgraded to 3.18. Maybe a miscommunication of the readiness. I assume that is why these are failing.
Seems that still may be the case.
https://openj9-jenkins.osuosl.org/job/all-nodes-matching-labels/85/console

[AdamBrousseau]

ibmruntimes/openj9-openjdk-jdk11#658

Enable CRIU support by default on ppc64le builds
Pete: do you want this change included in 0.41?
Tobi: Yes. Should be a no-op since Adam manually turns it on for milestone and release builds

[tajila]

Maybe a miscommunication of the readiness. I assume that is why these are failing.

Okay, I thought Marcus said that work was complete.

[tajila]

If not we will need to revert the following:
#18138
ibmruntimes/openj9-openjdk-jdk#605
and others

[pshipton]

It may just be the one machine, which I've now disabled.

[pshipton]

Also on cent7-ppcle-1
I expect all cent7-ppcle machines.

https://openj9-jenkins.osuosl.org/job/Test_openjdk11_j9_sanity.functional_ppc64le_linux_Nightly_testList_0/605

[pshipton]

All cent7-ppcle have been disabled.

[cjjdespres]

I think this is happening in the JITServer nightly tests as well, e.g. last night in https://hyc-runtimes-jenkins.swg-devops.com/job/Test_openjdk17_j9_sanity.functional_ppc64le_linux_jit_Personal/715/ on a rhel7 vm and https://hyc-runtimes-jenkins.swg-devops.com/job/Test_openjdk17_j9_sanity.functional_ppc64le_linux_jit_Personal/713/ on a cent8 vm a couple of days ago.

[pshipton]

The cent7-ppcle machines are the build machines, I've removed the ci.role.test labels and re-enabled the machines.

[SajinaKandy]

The host cent7-ppcle-3 fails Jenkins test with the same error:
CRIU needs to have the CAP_SYS_ADMIN or the CAP_CHECKPOINT_RESTORE capability
https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_sanity.functional_ppc64le_linux_jit_Personal_testList_1/115/console