Skip to content

LLVM commit a79995ca6004 breaks as-instr and as-option for ARCH=arm on older kernels #1878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nathanchance opened this issue Jun 21, 2023 · 3 comments
Labels
[BUG] llvm (main) A bug in an unreleased version of LLVM (this label is appropriate for regressions) [FIXED][LLVM] main This bug was only present and fixed in an unreleased version of LLVM

Comments

@nathanchance
Copy link
Member

CI reports several build failures due to the smc instruction when building ARCH=arm multi_v7_defconfig with LLVM 17 on linux-5.4.y and earlier.

https://github.com/ClangBuiltLinux/continuous-integration2/actions/runs/5306568615/jobs/9604484659

$ make -skj"$(nproc)" ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- LLVM=1 LLVM_IAS=0 mrproper multi_v7_defconfig drivers/firmware/qcom_scm-32.o
/tmp/qcom_scm-32-c413c1.s: Assembler messages:
/tmp/qcom_scm-32-c413c1.s:56: Error: selected processor does not support `smc #0' in ARM mode
/tmp/qcom_scm-32-c413c1.s:70: Error: selected processor does not support `smc #0' in ARM mode
/tmp/qcom_scm-32-c413c1.s:178: Error: selected processor does not support `smc #0' in ARM mode
/tmp/qcom_scm-32-c413c1.s:408: Error: selected processor does not support `smc #0' in ARM mode
/tmp/qcom_scm-32-c413c1.s:562: Error: selected processor does not support `smc #0' in ARM mode
/tmp/qcom_scm-32-c413c1.s:947: Error: selected processor does not support `smc #0' in ARM mode
/tmp/qcom_scm-32-c413c1.s:1087: Error: selected processor does not support `smc #0' in ARM mode
/tmp/qcom_scm-32-c413c1.s:1134: Error: selected processor does not support `smc #0' in ARM mode
clang: error: assembler command failed with exit code 1 (use -v to see invocation)

I bisected this to llvm/llvm-project@a79995c:

# bad: [8e1f820bb4eadf5c0704818f6063e0db1006e32d] [dsymutil] Update DWARF 5 tests after 8119ab9bf78b
# good: [168fa3181614fc7bc33e0b8c588e3ec74b3f65b5] [libc] Fix some tests on NVPTX due to insufficient stack size
git bisect start '8e1f820bb4eadf5c0704818f6063e0db1006e32d' '168fa3181614fc7bc33e0b8c588e3ec74b3f65b5'
# good: [a48fe898857c95a063fa6c201343dca969bc098a] [mlir][ArmSME] Add initial dialect with basic lowering of vector.transfer write to zero
git bisect good a48fe898857c95a063fa6c201343dca969bc098a
# good: [aee3a9f5bcdc13de4bb1b3eb04845b172fffb23c] [libc++][NFC] clang-format new_handler.cpp
git bisect good aee3a9f5bcdc13de4bb1b3eb04845b172fffb23c
# good: [11ebe3d906558d93a607347de472e7718127f409] [RISCV] relaxDwarfCallFrameFragment: remove unneeded relocations for relaxation
git bisect good 11ebe3d906558d93a607347de472e7718127f409
# good: [0a246a0c723dc94b77e83279d76c30010cdc2fc0] [LV] Use VPValues when creating GEP with all invariant indices.
git bisect good 0a246a0c723dc94b77e83279d76c30010cdc2fc0
# bad: [0356ceedf2e90459597adfda19db2c52faf7aacd] [DebugInfo] Change DWARFDebugAbbrev initialization
git bisect bad 0356ceedf2e90459597adfda19db2c52faf7aacd
# bad: [8a403166aa617d66bf420ec11350e9e0f4d1f02f] [RISCV] Treat __riscv_vsetvl_*(-1) as vlmax.
git bisect bad 8a403166aa617d66bf420ec11350e9e0f4d1f02f
# bad: [61c9052cec10a80f9a48d2da9e443629089c9a88] [libc] Add LIBC_INLINE_VAR for inline variables
git bisect bad 61c9052cec10a80f9a48d2da9e443629089c9a88
# bad: [c8f4ba374b10b9851a580ad73de8bd4d8e40be60] [NFC] Autogenerate CodeGen/ARM/vlddup.ll
git bisect bad c8f4ba374b10b9851a580ad73de8bd4d8e40be60
# bad: [a79995ca6004082774a87f7a58ab6be5343364b7] [Driver] Allow warning for unclaimed TargetSpecific options
git bisect bad a79995ca6004082774a87f7a58ab6be5343364b7
# good: [ec77747fbdca901e0fded58f940dae62e0f6b726] [NFC] Autogenerate CodeGen/ARM/sub-cmp-peephole.ll
git bisect good ec77747fbdca901e0fded58f940dae62e0f6b726
# first bad commit: [a79995ca6004082774a87f7a58ab6be5343364b7] [Driver] Allow warning for unclaimed TargetSpecific options

Comparing the command of drivers/firmware/qcom_scm-32.o before and after the change reveals -DREQUIRES_SEC is getting dropped:

diff --git a/tmp/.psub.q4ut5Iov1j b/tmp/.psub.NTsX2cJIis
index c9fa288ba7ef..ae3237ee51f9 100644
--- a/tmp/.psub.q4ut5Iov1j
+++ b/tmp/.psub.NTsX2cJIis
@@ -1 +1 @@
-  ccache /mnt/nvme/tmp/cvise.KRtce4djrq/install/llvm-good/bin/clang -Wp,-MD,drivers/firmware/.qcom_scm-32.o.d -nostdinc -isystem /mnt/nvme/tmp/cvise.KRtce4djrq/install/llvm-good/lib/clang/17/include -I./arch/arm/include -I./arch/arm/include/generated  -I./include -I./arch/arm/include/uapi -I./arch/arm/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h -include ./include/linux/compiler_types.h -D__KERNEL__ -mlittle-endian -Qunused-arguments -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror=implicit-function-declaration -Werror=implicit-int -Werror=return-type -Wno-format-security -std=gnu89 --target=arm-linux-gnueabi --prefix=/home/nathan/cbl/toolchains/binutils-latest/bin/arm-linux-gnueabi- --gcc-toolchain=/home/nathan/cbl/toolchains/binutils/2023-06-20_22-22-26-00a854287778c852191df9f9815c5d601daa5ae1 -no-integrated-as -Werror=unknown-warning-option -fno-dwarf2-cfi-asm -mabi=aapcs-linux -mfpu=vfp -funwind-tables -meabi gnu -marm -Wa,-mno-warn-deprecated -D__LINUX_ARM_ARCH__=7 -march=armv7-a -msoft-float -Uarm -fno-delete-null-pointer-checks -Wno-frame-address -Wno-address-of-packed-member -O2 -Wframe-larger-than=1024 -fstack-protector-strong -Wno-format-invalid-specifier -Wno-gnu -Wno-tautological-compare -mno-global-merge -Wno-unused-but-set-variable -Wno-unused-const-variable -fomit-frame-pointer -Wdeclaration-after-statement -Wvla -Wno-pointer-sign -Wno-array-bounds -fno-strict-overflow -fno-merge-all-constants -fno-stack-check -Werror=date-time -Werror=incompatible-pointer-types -fmacro-prefix-map=./= -Wno-initializer-overrides -Wno-format -Wno-sign-compare -Wno-format-zero-length -Wno-pointer-to-enum-cast -Wno-unaligned-access -Wno-cast-function-type-strict -DREQUIRES_SEC=1 -march=armv7-a    -DKBUILD_BASENAME='"qcom_scm_32"' -DKBUILD_MODNAME='"qcom_scm_32"' -c -o drivers/firmware/qcom_scm-32.o drivers/firmware/qcom_scm-32.c
+  ccache /mnt/nvme/tmp/cvise.KRtce4djrq/install/llvm-bad/bin/clang -Wp,-MD,drivers/firmware/.qcom_scm-32.o.d -nostdinc -isystem /mnt/nvme/tmp/cvise.KRtce4djrq/install/llvm-bad/lib/clang/17/include -I./arch/arm/include -I./arch/arm/include/generated  -I./include -I./arch/arm/include/uapi -I./arch/arm/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h -include ./include/linux/compiler_types.h -D__KERNEL__ -mlittle-endian -Qunused-arguments -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror=implicit-function-declaration -Werror=implicit-int -Werror=return-type -Wno-format-security -std=gnu89 --target=arm-linux-gnueabi --prefix=/home/nathan/cbl/toolchains/binutils-latest/bin/arm-linux-gnueabi- --gcc-toolchain=/home/nathan/cbl/toolchains/binutils/2023-06-20_22-22-26-00a854287778c852191df9f9815c5d601daa5ae1 -no-integrated-as -Werror=unknown-warning-option -fno-dwarf2-cfi-asm -mabi=aapcs-linux -mfpu=vfp -funwind-tables -meabi gnu -marm -Wa,-mno-warn-deprecated -D__LINUX_ARM_ARCH__=7 -march=armv7-a -msoft-float -Uarm -fno-delete-null-pointer-checks -Wno-frame-address -Wno-address-of-packed-member -O2 -Wframe-larger-than=1024 -fstack-protector-strong -Wno-format-invalid-specifier -Wno-gnu -Wno-tautological-compare -mno-global-merge -Wno-unused-but-set-variable -Wno-unused-const-variable -fomit-frame-pointer -Wdeclaration-after-statement -Wvla -Wno-pointer-sign -Wno-array-bounds -fno-strict-overflow -fno-merge-all-constants -fno-stack-check -Werror=date-time -Werror=incompatible-pointer-types -fmacro-prefix-map=./= -Wno-initializer-overrides -Wno-format -Wno-sign-compare -Wno-format-zero-length -Wno-pointer-to-enum-cast -Wno-unaligned-access -Wno-cast-function-type-strict -march=armv7-a    -DKBUILD_BASENAME='"qcom_scm_32"' -DKBUILD_MODNAME='"qcom_scm_32"' -c -o drivers/firmware/qcom_scm-32.o drivers/firmware/qcom_scm-32.c

Dumping the as-instr invocation from drivers/firmware/Makefile reveals:

clang: error: unsupported option '-mabi=' for target 'arm-linux-gnueabi'

-mabi=aapcs-linux is added to both KBUILD_CFLAGS and KBUILD_AFLAGS but it errors with -x assembler, which as-instr and as-option use on older kernels, whereas the actual .S build step will use -x assembler-with-cpp internally.

$ clang --target=arm-linux-gnueabi -mabi=aapcs-linux -x c -c -o /dev/null /dev/null

$ clang --target=arm-linux-gnueabi -mabi=aapcs-linux -x assembler -c -o /dev/null /dev/null
clang: error: unsupported option '-mabi=' for target 'arm-linux-gnueabi'

$ clang --target=arm-linux-gnueabi -mabi=aapcs-linux -x assembler-with-cpp -c -o /dev/null /dev/null

The error above is not visible with 5.10 and newer due to commit 0224898152bd ("firmware: qcom_scm-32: Use SMC arch wrappers"), which may be backportable as a solution to that one error. As mentioned above, there are other errors on older branches for similar reasons, so it may be better to fix this in a more universal way.

We may be able to move older releases to using -x assembler-with-cpp like we have with newer versions:

diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
index 5d247d8f1e04..9b37744addda 100644
--- a/scripts/Kbuild.include
+++ b/scripts/Kbuild.include
@@ -108,7 +108,7 @@ as-option = $(call try-run,\
 # Usage: cflags-y += $(call as-instr,instr,option1,option2)

 as-instr = $(call try-run,\
-       printf "%b\n" "$(1)" | $(CC) $(KBUILD_AFLAGS) -c -x assembler -o "$$TMP" -,$(2),$(3))
+       printf "%b\n" "$(1)" | $(CC) $(KBUILD_CPPFLAGS) $(KBUILD_AFLAGS) -c -x assembler-with-cpp -o "$$TMP" -,$(2),$(3))

 # __cc-option
 # Usage: MY_CFLAGS += $(call __cc-option,$(CC),$(MY_CFLAGS),-march=winchip-c6,-march=i586)

But I see another internal error on 5.4, there must have been some additional Kbuild refactoring done at some point to avoid this on newer releases:

<built-in>:1:10: fatal error: 'asm/unified.h' file not found
    1 | #include "asm/unified.h"
      |          ^~~~~~~~~~~~~~~
1 error generated.

which comes from -include asm/unified.h in arch/arm/Makefile's KBUILD_AFLAGS. I will continue to look at ways to workaround this on the Linux side but it would be nice if -mabi= would not error with -x assembler in the first place, although I understand the error is pointing out that the flag does nothing for that particular target (even though GCC won't warn).

cc @MaskRay

@nathanchance nathanchance added the [BUG] llvm (main) A bug in an unreleased version of LLVM (this label is appropriate for regressions) label Jun 21, 2023
@MaskRay
Copy link
Member

MaskRay commented Jun 24, 2023

The -mabi= option for assembler seems strange. GCC translates -mabi={apcs-gnu,atpcs} to gas -meabi=gnu and other -mabi= values to -meabi=5. To the best of my knowledge, LLVM integrated assembler don't support setting e_flags to any value other than EF_ARM_EABI_VER5.

LLVM integrated assembler does not handle -mabi= and there was a warning before.
https://reviews.llvm.org/D153691 will change back the error to a warning.
It's still recommended to drop -mabi= option for -x assembler.

@nathanchance
Copy link
Member Author

Thanks, llvm/llvm-project@fe5bab5 helps avoid this issue when using the integrated assembler but it does not appear to resolve the issue when using -fno-integrated-as.

$ clang --version | head -1
ClangBuiltLinux clang version 17.0.0 (https://github.com/llvm/llvm-project cb1911a2db14b02a3f7855723d9138d26c41e66f)

$ clang --target=arm-linux-gnueabi -mabi=aapcs-linux -x assembler -c -o /dev/null /dev/null
clang: warning: argument unused during compilation: '-mabi=aapcs-linux' [-Wunused-command-line-argument]

$ clang --target=arm-linux-gnueabi -fno-integrated-as -mabi=aapcs-linux -x assembler -c -o /dev/null /dev/null
clang: error: unsupported option '-mabi=' for target 'arm-linux-gnueabi'

@nathanchance nathanchance reopened this Jun 26, 2023
@MaskRay
Copy link
Member

MaskRay commented Jul 7, 2023

Sorry for missing the notification. I don't have a good personal filtering system for https://github.com/notifications and it is full of items (mostly llvm-project issues), so I easily miss notifications that I should pay attention to...

$ clang --target=arm-linux-gnueabi -fno-integrated-as -mabi=aapcs-linux -x assembler -c -o /dev/null /dev/null
clang: error: unsupported option '-mabi=' for target 'arm-linux-gnueabi'

Sent https://reviews.llvm.org/D154736 to downgrade the error to clang: warning: argument unused during compilation: '-mabi=aapcs-linux' [-Wunused-command-line-argument]

@nickdesaulniers nickdesaulniers added the [FIXED][LLVM] main This bug was only present and fixed in an unreleased version of LLVM label Jul 10, 2023
Chenyang-L pushed a commit to intel/llvm that referenced this issue Jul 11, 2023
Previously, Clang Driver reported a warning when assembler input was assembled
with the -mabi= option. D152856 added TargetSpecific to -mabi= option and
reported an error for such a case. This change restores the previous behavior by
reporting a warning.

GCC translates -mabi={apcs-gnu,atpcs} to gas -meabi=gnu and other -mabi= values
to -meabi=5. We don't support setting e_flags to any value other than
EF_ARM_EABI_VER5.

Close ClangBuiltLinux/linux#1878

Reviewed By: michaelplatings

Differential Revision: https://reviews.llvm.org/D153691
veselypeta pushed a commit to veselypeta/cherillvm that referenced this issue Sep 2, 2024
Previously, Clang Driver reported a warning when assembler input was assembled
with the -mabi= option. D152856 added TargetSpecific to -mabi= option and
reported an error for such a case. This change restores the previous behavior by
reporting a warning.

GCC translates -mabi={apcs-gnu,atpcs} to gas -meabi=gnu and other -mabi= values
to -meabi=5. We don't support setting e_flags to any value other than
EF_ARM_EABI_VER5.

Close ClangBuiltLinux/linux#1878

Reviewed By: michaelplatings

Differential Revision: https://reviews.llvm.org/D153691
veselypeta pushed a commit to veselypeta/cherillvm that referenced this issue Sep 4, 2024
…ted-as

Similar to D153691, but for `-x assembler -fno-integrated-as`.

Close ClangBuiltLinux/linux#1878

Reviewed By: michaelplatings

Differential Revision: https://reviews.llvm.org/D154736
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[BUG] llvm (main) A bug in an unreleased version of LLVM (this label is appropriate for regressions) [FIXED][LLVM] main This bug was only present and fixed in an unreleased version of LLVM
Projects
None yet
Development

No branches or pull requests

3 participants