[libc] Add ${CMAKE_CROSSCOMPILING_EMULATOR} to custom test cmdlines

[mikhailramalho]

${CMAKE_CROSSCOMPILING_EMULATOR} will be used in the new rv32 buildbot and is prepended automatically when we call add_custom_target in CMake, except when we use a custom command.

There are two places where custom commands are used in libc, so we explicitly add the ${CMAKE_CROSSCOMPILING_EMULATOR} variable there. Other systems that don't use ${CMAKE_CROSSCOMPILING_EMULATOR} are unaffected

[llvmbot]

@llvm/pr-subscribers-libc

Changes

${CMAKE_CROSSCOMPILING_EMULATOR} will be used in the new rv32 buildbot and is prepended automatically when we call add_custom_target in CMake, except when we use a custom command.

There are two places where custom commands are used in libc, so we explicitly add the ${CMAKE_CROSSCOMPILING_EMULATOR} variable there. Other systems that don't use ${CMAKE_CROSSCOMPILING_EMULATOR} are unaffected

Full diff: https://github.com/llvm/llvm-project/pull/66565.diff

1 Files Affected:

• (modified) libc/cmake/modules/LLVMLibCTestRules.cmake (+2-2)

diff --git a/libc/cmake/modules/LLVMLibCTestRules.cmake b/libc/cmake/modules/LLVMLibCTestRules.cmake
index e1286b4ab963189..552cbd739628deb 100644
--- a/libc/cmake/modules/LLVMLibCTestRules.cmake
+++ b/libc/cmake/modules/LLVMLibCTestRules.cmake
@@ -561,7 +561,7 @@ function(add_integration_test test_name)
   # to expand the list (by including the option COMMAND_EXPAND_LISTS). This
   # makes `add_custom_target` construct the correct command and execute it.
   set(test_cmd
-      ${INTEGRATION_TEST_ENV}
+      ${CMAKE_CROSSCOMPILING_EMULATOR} ${INTEGRATION_TEST_ENV}
       $<$<BOOL:${LIBC_TARGET_ARCHITECTURE_IS_GPU}>:${gpu_loader_exe}>
       ${INTEGRATION_TEST_LOADER_ARGS}
       $<TARGET_FILE:${fq_build_target_name}> ${INTEGRATION_TEST_ARGS})
@@ -728,7 +728,7 @@ function(add_libc_hermetic_test test_name)
 
   set(test_cmd ${HERMETIC_TEST_ENV}
       $<$<BOOL:${LIBC_TARGET_ARCHITECTURE_IS_GPU}>:${gpu_loader_exe}> ${HERMETIC_TEST_LOADER_ARGS}
-      $<TARGET_FILE:${fq_build_target_name}> ${HERMETIC_TEST_ARGS})
+      ${CMAKE_CROSSCOMPILING_EMULATOR} $<TARGET_FILE:${fq_build_target_name}> ${HERMETIC_TEST_ARGS})
   add_custom_target(
     ${fq_target_name}
     COMMAND ${test_cmd}

[jhuber6]

We do the following for the GPU build.

$> clang main.c --target=amdgcn-amd-amdhsa -mcpu=gfx1030 crt1.o -lc -o exe
$> ./emulator <optional arguments> exe

This looks very similar to cross-compiling by design. I'm thinking it would be good to unify this approach with the GPU build. This is just me thinking out loud.

[mikhailramalho]

That sounds like a good idea, but how do you handle env variables in these cases?

Currently, the script I wrote expects <env variables> <full path to binary> <args> to work properly... We need the full path to the binary to scp any test data present in the binary directory; we then copy the binary to a temp dir in the qemu-system and call ssh foo@bar <env variables> ~/<temp dir>/<binary> <args>.

(I should probably upload the script I'm using somewhere).

[jhuber6]

I don't use any env vars, but presumably it would be handled just by the loader. E.g. if I do env foo ./amdhsa_loader <args> <image> it would read foo from the environment and pass it to the image it launches.

[mikhailramalho]

do you know does it detects the foo env variable in the loader? Or will it just send every env var on the system to the image?

[jhuber6]

Every single one, it clones the current user environment.

[jhuber6]

https://github.com/llvm/llvm-project/blob/main/libc/utils/gpu/loader/Loader.h#L92 here's the code for it, pretty much just uses the envp passed in from the main function.

[mikhailramalho]

that actually makes things easier for me.

What do you think about dropping gpu_loader_exe and using just CMAKE_CROSSCOMPILING_EMULATOR instead?

[jhuber6]

Would this be an argument passed in the cmake invocation? Because the gpu_loader_exe is built internally so it's not known until CMake runs. We could potentially just set CMAKE_CROSSCOMPILING_EMULATOR manually inside of libc? Maybe that would override something the user set, unsure.

[mikhailramalho]

I just ran some tests here and it works both ways: either you set it during the cmake config via cmd line or you can set it using set(CMAKE_CROSSCOMPILING_EMULATOR "foobar").

My only concern is that it doesn't tell the user that it overwrote an argument passed via cmd line, but I guess we can always throw a warning there.

I'll update this PR to put $CMAKE_CROSSCOMPILING_EMULATOR right after the $gpu_loader_exe for now, and send all environment variables as the loader does. If you can test using only $CMAKE_CROSSCOMPILING_EMULATOR and it does work, we can send another PR to remove $gpu_loader_exe later.

[jhuber6]

SG, I'll test it once this lands.

[nickdesaulniers]

Do you have any instructions on how this was used? I'd like to try to revive some testing via qemu.