[tracking] Publish .NET assemblies in Webcil files

[lambdageek]

Summary

Webcil is a new container format for .NET assemblies that looks less like a normal Windows PE (Portable Executable) .dll file.
A .NET assembly in the Webcil format is a normal binary WebAssembly module with a particular set of exports that will copy a "Webcil payload" to WebAssembly linear memory.
It has a new .webcil extension a standard application/wasm MIME type and .wasm extension and the detailed format is specified in webcil.md.

There are two primary motivations for webcil:

1. In some restricted environments, users may be prevented from downloading .dll or PE files, and some AV tools may quarantine .dll files found in browser caches.
2. By using a new container, the runtime may be able to find additional flexibility in how we bundle and serve .NET assemblies. A goal is for the container to be invisible to higher layers of the runtime - unlike .netmodules in .NET Framework, as far as most of the runtime is concerned, we're still loading ordinary Ecma335 assemblies and any partioning is not visible at the metadata level. But at the loader level, the runtime may actually receive .webcil files that encapsulate one, part of one, multiple, or parts of multiple assemblies. (Version 0.0 of webcil only supports one assembly per .webcil container)

.NET 8 Preview 1

The goal is to make Webcil work in the wasm-experimental workload with the wasmbrowser template

The initial runtime work was done in #79416

• Update MIME types for .webcil
  • in xharness Serve .webcil as application/octet-stream xharness#989
  • and dotnet SDK webserver configs BlazorWasmSdk: Serve .webcil as application/octet-stream sdk#30077
  • aspnetcore UseBlazorFrameworkFiles UseBlazorFrameworkFiles: add webcil mime type mapping aspnetcore#46244
• Implement WebcilReader.ReadPdbChecksumDebugDirectoryData and refactor DebugStore AssemblyInfo.FromWebcilReader and AssemblyInfo.FromPEReader to share PDB logic (maybe using a new IDebugInfoProvider adapter to cover the common parts of PE and Webcil) BrowserDebugProxy: unify debug metadata reading for PE and Webcil #81099

Later in .NET8

• Implement webcil in blazor bundling [browser] Enable webcil in Wasm SDK #84977
  • Implement a standalone WebcilGenerator MSBuild task for Blazor lazy loading scenarios [browser] Fix webcil lazy loading and satellite assemblies in Wasm SDK #85335
  • Add a lazy loading + satellite assemblies WBT test (BugRegression_60479_WithRazorClassLib)
  • Update ComputeBlazorPublishAssets and elsewhere in the BlazorWasmSDK
  • Update various places where file extension is checked (tasks, BlazorWebAssemblyLazyLoad, AssemblyLoader.LoadAssembliesAsync)
• [wasm] Webcil-in-WebAssembly #85932
• Make webcil-in-wasm the default packaging for standalone .NET WebAssembly apps (dotnet new wasmbrowser) and Blazor WebAssembly apps (dotnet new blazorwasm) [wasm] Set WasmEnableWebcil to true by default #86330
• Enable debugger tests with Webcil
• Gather feedback and identify further mitigations for restricted environments
• publish a Microsoft.NET.WebAssembly.Webcil nuget to the transport feed #86597

Post .NET 8

• Take advantage of webcil custom loader to optimize wasm bundles. (Define a multi-assembly .webcil container. Define a partial-assembly webcil container. Etc.)

Other

• Create and upstream a magic(5) entry for webcil #86522

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Follow-up to #79416

• Enable debugger tests with Webcil
• Implement a standalone WebcilGenerator MSBuild task for Blazor lazy loading scenarios
• Gather feedback and identify further mitigations for restricted environments
• Take advantage of webcil custom loader to optimize wasm bundles. (Define a multi-assembly .webcil container. Define a partial-assembly webcil container. Etc.)

Author:	lambdageek
Assignees:	-
Labels:	arch-wasm
Milestone:	-

[lambdageek]

Code review nits:

• use strlen(LITERAL) instead of 4 or 7. For example here: [mono][wasm] Bundle assemblies as WebCIL #79416 (comment)
• In WasmAppBuilder, use Utils.CopyIfDifferent(string src, string dst, bool useHash) to place the webcil file in the final output path and add to fileWrites; FileCopyChecked doesn't do a checksum or anything like that.
  [mono][wasm] Bundle assemblies as WebCIL #79416 (comment)
• Need to update the various web servers to serve .webcil as octet-stream [mono][wasm] Bundle assemblies as WebCIL #79416 (comment)
  • here https://github.com/dotnet/sdk/blob/ea03dc7f05bfbaac843dc2ac09e26a484acfe8f8/src/BlazorWasmSdk/Targets/BlazorWasm.web.config#L13
  • and here

    runtime/src/mono/wasm/host/WebServerStartup.cs

    Line 85 in 12f65cd

    foreach (string extn in new string[] { ".dll", ".pdb", ".dat", ".blat" })

[lambdageek]

• Also need to add mime type to xharness https://github.com/dotnet/xharness/blob/main/src/Microsoft.DotNet.XHarness.CLI/Commands/WebServer.cs#L119