Revert "Pin SHAs for container images"

[richlander]

Reverts #113185

Reverting enables us to safely test Clang 20 being added to these images per dotnet/dotnet-buildtools-prereqs-docker#1369

[Copilot]

PR Overview

This PR reverts the previous change that pinned container image SHAs in the pipeline file, allowing the use of unpinned image tags to facilitate testing of Clang 20 additions.

• Reverts SHA pinning in container image references
• Enables testing of updated images with Clang 20 support

Reviewed Changes

File	Description
eng/pipelines/common/templates/pipeline-with-resources.yml	Removed SHA pins from container image definitions

Copilot reviewed 1 out of 1 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (1)

eng/pipelines/common/templates/pipeline-with-resources.yml:20

• Removing the SHA pin from container image references may introduce non-determinism in builds if the image is updated unexpectedly. Consider verifying through tests or safeguards that these images remain compatible and stable.

image: mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net10.0-cross-arm

[richlander]

/azp run runtime-libraries-coreclr outerloop-linux

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[am11]

Noticed here https://dev.azure.com/dnceng-public/public/_build/results?buildId=971589&view=logs&jobId=62b70ecb-649d-5f05-1138-74c7e6e4705f&j=2f68d9b0-7564-5966-3317-2e2aa7d356c0&t=eee86911-880c-5ea0-1825-5072a102d343, it's still picking up llvm 19 images:

  -- The C compiler identification is Clang 19.1.6
  -- The CXX compiler identification is Clang 19.1.6
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done

dotnet/dotnet-buildtools-prereqs-docker#1369 (comment)

[richlander]

Odd. I opened this PR after seeing this change pushed: dotnet/versions@917ff98. That diff suggests (to me) that there was a source change not just a re-build. I think it was from this change: dotnet/dotnet-buildtools-prereqs-docker@4ac71d0. That's strange since that was 2 days ago.

Hopefully the expected changes are published today. If not, someone will need to take a look at the publish pipeline.

@mthalman

[mthalman]

The changes from dotnet/dotnet-buildtools-prereqs-docker#1369 didn't get properly published because crossdeps-llvm was never rebuilt: dotnet/dotnet-buildtools-prereqs-docker#1369 (comment). I've started a new build about an hour ago to fix that. These builds take like 7 hrs so it'll be a while.

[richlander]

Thanks @mthalman! Nothing is ever simple around here!

[richlander]

This looks better:

$ docker run --rm mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net10.0-cross-amd64-musl clang --version
clang version 20.1.0
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin

Per: dotnet/versions@25648fc

[richlander]

/azp run runtime-libraries-coreclr outerloop-linux

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[richlander]

Looks like our conservative approach with pinning was warranted and saved a bunch repo-wide hassle.

There are a few patterns that are now disallowed (at least by default) by the compiler, I'm guessing.

Lots of errors like:

 In file included from <built-in>:1:
  In file included from /__w/1/s/artifacts/obj/coreclr/linux.x86.Checked/vm/wks/CMakeFiles/cee_wks_mergeable.dir/cmake_pch.hxx:5:
  In file included from /__w/1/s/src/coreclr/vm/common.h:302:
  In file included from /__w/1/s/src/coreclr/vm/ceeload.h:40:
  In file included from /__w/1/s/src/coreclr/vm/readytoruninfo.h:14:
  In file included from /__w/1/s/src/coreclr/vm/nativeformatreader.h:15:
  In file included from /usr/local/lib/clang/20/include/emmintrin.h:17:
  /usr/local/lib/clang/20/include/xmmintrin.h:148:3: error: statement not allowed in constexpr function
    148 |   return (__m128)((__v4sf)__a - (__v4sf)__b);
        |   ^
  /__w/1/s/src/coreclr/inc/debugreturn.h:101:16: note: expanded from macro 'return'
    101 | #define return for (;1;__ReturnOK::safe_to_return()) return
        |                ^

[mthalman]

What is the status of getting this working? By keeping runtime pinned on digests, it's important that blocking issues from newer images are addressed in a timely manner so that newer updates to the images (outside of the clang changes) can be incorporated.

[mthalman]

Ok, I see that #113279 has the proposed fix.

[richlander]

/azp run runtime-libraries-coreclr outerloop-linux

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[richlander]

These look like infra related issues. Fair @sbomer @jkoritzinsky?

[sbomer]

Yes, the failures look like timeouts. LGTM!

[richlander]

FYI: The implication of this revert is that we're now pulling new build images with clang 20, where we were previously pinned to images including clang 19.

https://discourse.llvm.org/t/llvm-20-1-0-released/85122

[MichalStrehovsky]

The docker toolset update broke native AOT on ARM32. I submitted a revert of this at #113598 to confirm; ARM32 works again with a revert.

Filed #113609 for tracking, it's not clear to me why unaligned memory access worked before and now stopped working.

It is my understanding we're snapping dotnet/runtime for Preview 3 today.