Unable to instrument module incl. simple sample - no work around

[nwoolls]

Hello there! I hope this is not just noise as this seems like a fairly common issue. In fact I've reported it in the past and was able to work around the issue years ago.

However, today on another project, I have run into it again and none of the reported workarounds (.csproj settings, command line parameters, copying DLLs) seem to work.

.NET Version

dotnet --version
7.0.102

Steps to Reproduce

Execute from CLI:

mkdir coverlet-fails-to-resolve-mvc-razor
cd coverlet-fails-to-resolve-mvc-razor

mkdir webapp
pushd webapp
dotnet new webapp
popd

mkdir xunit-tests
pushd xunit-tests
dotnet new xunit
dotnet add reference ../webapp/webapp.csproj
dotnet add package coverlet.msbuild
popd

dotnet new sln
dotnet sln add webapp/webapp.csproj
dotnet sln add xunit-tests/xunit-tests.csproj

Add the following class to the web application project:

using Microsoft.AspNetCore.Mvc.Razor;

namespace webapp;

public static class Class1
{
    public static IMvcBuilder AddLocalization(this IMvcBuilder mvcBuilder,
        LanguageViewLocationExpanderFormat viewLocationExpanderFormat = LanguageViewLocationExpanderFormat.Suffix)
    {
        return mvcBuilder;
    }
}

Add the following class to the xUnit project:

using webapp;
using Microsoft.AspNetCore.Builder;
using Microsoft.Extensions.DependencyInjection;

namespace xunit_tests;

public class Class1Tests
{
    [Fact]
    public void AddLocalizationSucceeds()
    {
        // arrange
        WebApplicationBuilder webApplicationBuilder = WebApplication.CreateBuilder();
        IMvcBuilder mvcBuilder = webApplicationBuilder.Services.AddControllersWithViews();
        
        // act
        IMvcBuilder result = mvcBuilder.AddLocalization();

        // assert
        Assert.NotNull(result);
    }
}

Execute from the CLI:

dotnet test /p:CollectCoverage=true

Expected Result

  Determining projects to restore...
  All projects are up-to-date for restore.
  webapp -> /Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/webapp/bin/Debug/net7.0/webapp.dll
  xunit-tests -> /Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/bin/Debug/net7.0/xunit-tests.dll
Test run for /Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/bin/Debug/net7.0/xunit-tests.dll (.NETCoreApp,Version=v7.0)
Microsoft (R) Test Execution Command Line Tool Version 17.4.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     2, Skipped:     0, Total:     2, Duration: 226 ms - xunit-tests.dll (net7.0)

Calculating coverage result...
  Generating report '/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/coverage.json'

+--------+-------+--------+--------+
| Module | Line  | Branch | Method |
+--------+-------+--------+--------+
| webapp | 5.66% | 0%     | 6.25%  |
+--------+-------+--------+--------+

+---------+-------+--------+--------+
|         | Line  | Branch | Method |
+---------+-------+--------+--------+
| Total   | 5.66% | 0%     | 6.25%  |
+---------+-------+--------+--------+
| Average | 5.66% | 0%     | 6.25%  |
+---------+-------+--------+--------+

Actual Result

  Determining projects to restore...
  All projects are up-to-date for restore.
  webapp -> /Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/webapp/bin/Debug/net7.0/webapp.dll
  xunit-tests -> /Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/bin/Debug/net7.0/xunit-tests.dll
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning : [coverlet] Unable to instrument module: /Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/bin/Debug/net7.0/webapp.dll [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning : Coverlet.Core.Exceptions.CecilAssemblyResolutionException: AssemblyResolutionException for 'Microsoft.AspNetCore.Mvc.Razor, Version=7.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. Try to add <PreserveCompilationContext>true</PreserveCompilationContext> to test projects </PropertyGroup> or pass '/p:CopyLocalLockFileAssemblies=true' option to the 'dotnet test' command-line [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :  ---> Mono.Cecil.AssemblyResolutionException: Failed to resolve assembly: 'Microsoft.AspNetCore.Mvc.Razor, Version=7.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60' [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    --- End of inner exception stack trace --- [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Coverlet.Core.Instrumentation.NetstandardAwareAssemblyResolver.TryWithCustomResolverOnDotNetCore(AssemblyNameReference name) in /_/src/coverlet.core/Instrumentation/CecilAssemblyResolver.cs:line 215 [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Coverlet.Core.Instrumentation.NetstandardAwareAssemblyResolver.Resolve(AssemblyNameReference name) in /_/src/coverlet.core/Instrumentation/CecilAssemblyResolver.cs:line 127 [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.MetadataResolver.Resolve(TypeReference type) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.ModuleDefinition.Resolve(TypeReference type) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.TypeReference.Resolve() [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.Mixin.CheckedResolve(TypeReference self) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.MetadataBuilder.GetConstantType(TypeReference constant_type, Object constant) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.MetadataBuilder.AddConstant(IConstantProvider owner, TypeReference type) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.MetadataBuilder.AddParameter(UInt16 sequence, ParameterDefinition parameter, ParamTable table) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.MetadataBuilder.AddParameters(MethodDefinition method) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.MetadataBuilder.AddMethod(MethodDefinition method) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.MetadataBuilder.AddMethods(TypeDefinition type) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.MetadataBuilder.AddType(TypeDefinition type) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.MetadataBuilder.AddTypes() [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.MetadataBuilder.BuildTypes() [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.MetadataBuilder.BuildModule() [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.MetadataBuilder.BuildMetadata() [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.ModuleWriter.<>c.<BuildMetadata>b__2_0(MetadataBuilder builder, MetadataReader _) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.ModuleDefinition.Read[TItem,TRet](TItem item, Func`3 read) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.ModuleWriter.BuildMetadata(ModuleDefinition module, MetadataBuilder metadata) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.ModuleWriter.Write(ModuleDefinition module, Disposable`1 stream, WriterParameters parameters) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.ModuleWriter.WriteModule(ModuleDefinition module, Disposable`1 stream, WriterParameters parameters) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Mono.Cecil.ModuleDefinition.Write(Stream stream, WriterParameters parameters) [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Coverlet.Core.Instrumentation.Instrumenter.InstrumentModule() in /_/src/coverlet.core/Instrumentation/Instrumenter.cs:line 337 [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Coverlet.Core.Instrumentation.Instrumenter.Instrument() in /_/src/coverlet.core/Instrumentation/Instrumenter.cs:line 153 [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
/Users/jdoe/.nuget/packages/coverlet.msbuild/3.2.0/build/coverlet.msbuild.targets(39,5): warning :    at Coverlet.Core.Coverage.PrepareModules() in /_/src/coverlet.core/Coverage.cs:line 135 [/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/xunit-tests.csproj]
Test run for /Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/bin/Debug/net7.0/xunit-tests.dll (.NETCoreApp,Version=v7.0)
Microsoft (R) Test Execution Command Line Tool Version 17.4.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     2, Skipped:     0, Total:     2, Duration: 238 ms - xunit-tests.dll (net7.0)

Calculating coverage result...
  Generating report '/Users/jdoe/src/dotnet/coverlet-fails-to-resolve-mvc-razor/xunit-tests/coverage.json'

+--------+------+--------+--------+
| Module | Line | Branch | Method |
+--------+------+--------+--------+

+---------+------+--------+--------+
|         | Line | Branch | Method |
+---------+------+--------+--------+
| Total   | 0%   | 0%     | 0%     |
+---------+------+--------+--------+
| Average | 0%   | 0%     | 0%     |
+---------+------+--------+--------+

Attempted Work Arounds

I have tried the recommended work arounds such as:

• Adding <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies> to the .csproj file(s)
• Adding <PreserveCompilationContext>true</PreserveCompilationContext> to the .csproj file(s)
• Using the /p:CopyLocalLockFileAssemblies=true command line argument
• Manually copying the DLL w/ webapp/bin/Debug/net7.0/refs/Microsoft.AspNetCore.Razor.dll xunit-tests/bin/Debug/net7.0

Unfortunately none of these resolve the issue.

[nwoolls]

OK - so I was copying the wrong DLL (Microsoft.AspNetCore.Razor.dll vs Microsoft.AspNetCore.Mvc.Razor.dll). If I do the following:

cp webapp/bin/Debug/net7.0/refs/Microsoft.AspNetCore.Mvc.Razor.dll xunit-tests/bin/Debug/net7.0

That does allow coverage to be collected.

Is this still the required / recommended solution? Is there any way to automate this w/ MSBuild?

[nwoolls]

FWIW, adding the following to the xUnit .csproj file seems to work around the issue. Is there a better way to handle this?

<!-- 
  Begin fix / work around for error(s) generating code coverage using e.g. Coverlet or AltCover.
  
  See: https://github.com/coverlet-coverage/coverlet/issues/1459
  See: https://github.com/SteveGilham/altcover/issues/179
-->
<PropertyGroup>
  <PreserveCompilationReferences>true</PreserveCompilationReferences>
</PropertyGroup>
<Target Name="CopyReferencedAssemblies" AfterTargets="Build">
  <ItemGroup>
    <ReferencedAssemblies Include="$(OutputPath)refs/*.dll"/>
  </ItemGroup>
  <Copy SourceFiles="@(ReferencedAssemblies)" DestinationFolder="$(OutputPath)" ContinueOnError="false" />
</Target>
<!-- End fix / work around for error(s) generating code coverage using e.g. Coverlet or AltCover. -->

[nwoolls]

Again FWIW, I was able to reproduce this w/ AltCover and the author was able to remedy it with a source code fix, in case that helps with your library as well.

SteveGilham/altcover#179