Access violation while finding shadow stack pointer in nativeaot/SmokeTests/UnitTests

[jakobbotsch]

When I run this test locally in checked on win-x64 I get the following failure:

===== Running test BasicThreading.Run =====
FinalizeTest passed
===== Test BasicThreading.Run succeeded =====
===== Running test Delegates.Run =====
Testing delegates to value types...OK
Testing delegates to virtual methods...OK
Testing delegates to interface methods...OK
Testing static open and closed delegates...OK
Testing multicast delegates...OK
Testing dynamic invoke...OK
Testing LINQ Expressions...
Testing default interface methods...
===== Test Delegates.Run succeeded =====
===== Running test Generics.Run =====
Base.GMethod1<System.Int32>(1,2)
Base.IMethod1<System.Int32>(3,4)
Base.IMethod1<System.Int32>(5,6)
Base.IMethod1<System.Int32>(7,8)
====================
Derived.GMethod1<System.Int32>(1,2)
Derived.IFoo<string>.IMethod1<System.Int32>(3,4)
Derived.IFoo<string>.IMethod1<System.Int32>(5,6)
Base.IMethod1<System.Int32>(7,8)
====================
Derived.GMethod1<System.Int32>(1,2)
Derived.IFoo<string>.IMethod1<System.Int32>(3,4)
Derived.IFoo<string>.IMethod1<System.Int32>(5,6)
SuperDerived.IFoo<int>.IMethod1<System.Int32>(7,8)
====================
GenBase<System.Byte>.GMethod1<System.Int32>(1,2)
GenBase<System.Byte>.IMethod1<System.Int32>(3,4)
GenBase<System.Byte>.IMethod1<System.Int32>(5,6)
GenBase<System.Byte>.IMethod1<System.Int32>(7,8)
====================
GenDerived<System.Byte>.GMethod1<System.Int32>(1,2)
GenDerived<System.Byte>.IFoo<string>.IMethod1<System.Int32>(3,4)
GenDerived<System.Byte>.IFoo<string>.IMethod1<System.Int32>(5,6)
GenBase<System.Byte>.IMethod1<System.Int32>(7,8)
====================
GenDerived<System.String>.GMethod1<System.Int32>(1,2)
GenDerived<System.String>.IFoo<string>.IMethod1<System.Int32>(3,4)
GenDerived<System.String>.IFoo<string>.IMethod1<System.Int32>(5,6)
GenBase<System.String>.IMethod1<System.Int32>(7,8)
====================
GenDerived<System.Byte>.GMethod1<System.Int32>(1,2)
GenDerived<System.Byte>.IFoo<string>.IMethod1<System.Int32>(3,4)
GenDerived<System.Byte>.IFoo<string>.IMethod1<System.Int32>(5,6)
GenSuperDerived<System.Byte>.IFoo<int>.IMethod1<System.Int32>(7,8)
====================
MyStruct1.IFoo<string>.IMethod1<System.Int32>(1,2)
MyStruct1.IFoo<string>.IMethod1<System.Int32>(3,4)
MyStruct1.IFoo<int>.IMethod1<System.Int32>(5,6)
====================
MyStruct2.IFoo<string>.IMethod1<System.Int32>(1,2)
MyStruct2.IFoo<string>.IMethod1<System.Int32>(3,4)
MyStruct2.IMethod1<System.Int32>(5,6)
====================
MyStruct3.IMethod1<System.Int32>(1,2)
MyStruct3.IMethod1<System.Int32>(3,4)
MyStruct3.IFoo<int>.IMethod1<System.Int32>(5,6)
====================
AnotherBaseClass.IFaceGVMethod1
BarImplementor:123
YahooDerived:456
Covariant<String>.ICovariantGVM<Exception>
Accepted
Foo<ClassType> cctor
Foo<StructType> cctor
Bar cctor
Testing static fields on type Foo<ClassType> ...
Testing static fields on type Foo<StructType> ...
Testing static fields on type Bar ...
Testing instance fields on type Foo<ClassType> ...
Testing instance fields on type Foo<StructType> ...
Testing instance fields on type Bar ...
Foo<Object> cctor
Foo<ClassType2> cctor
Testing dynamic invoke stubs...
GenericDerived<System.Object>.Get<System.String>
--------------------------------------------------
Debug Assertion Violation

Message: Hardware exception raised inside the runtime.

Expression: 'ASSERT_UNCONDITIONALLY'

File: C:\dev\dotnet\runtime2\src\coreclr\nativeaot\Runtime\EHHelpers.cpp, Line: 651

If I attach VS then I see an access violation on the cmp in this loop:

runtime/src/coreclr/nativeaot/Runtime/amd64/ExceptionHandling.asm

Lines 539 to 547 in 0dd2a62

	;; Find the shadow stack pointer for the frame we are going to restore to.
	;; The SSP we search is pointing to the return address of the frame represented
	;; by the passed in context. So we search for the instruction pointer from
	;; the context and return one slot up from there.
	;; (Same logic as in GetSSPForFrameOnCurrentStack)
	xor r11, r11
	@@: inc r11
	cmp [r9 + r11 * 8 - 8], r10
	jne @b

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

[jkotas]

@VSadov This looks like a regression from CET enablement. Could you please take a look?

[VSadov]

Will take a look. In theory we should be able to find the frame to which we are returning.
Otherwise CET will not let us continue anyways.

[VSadov]

I am able to reproduce this

[VSadov]

This is related to turning HW exceptions into a managed throw.

    if (translateToManagedException)
    {
        pExPtrs->ContextRecord->SetIp(PCODEToPINSTR((PCODE)&RhpThrowHwEx));
        pExPtrs->ContextRecord->SetArg0Reg(faultCode);
        pExPtrs->ContextRecord->SetArg1Reg(faultingIP);

        return EXCEPTION_CONTINUE_EXECUTION;
    }

When we perform the code above the shadow stack is formed as if the fault IP performed a call into OS exception dispatch and matches our view of the stack.

+		[0x00000000]	0x00007ffb7a1a9b5c {Inside ntdll.dll!RtlpCallVectoredHandlers(void)} {0x850fed8548e88b44}	unsigned __int64 *
+		[0x00000001]	0x00007ffb7a182376 {Inside ntdll.dll!RtlDispatchException(void)} {0x000002bb850fc084}	unsigned __int64 *
+		[0x00000002]	0x00007ffb7a1d143e {Inside ntdll.dll!KiUserExceptionDispatch()} {0x33cc8b480c74c084}	unsigned __int64 *
+		[0x00000003]	0x00007ff64fe1d722 {UnitTests.exe!UnitTests_Generics_TestNativeLayoutGeneration__Run(void), Line 2367} {...}	unsigned __int64 *
+		[0x00000004]	0x00007ff64fe103ed {UnitTests.exe!UnitTests_Generics__Run(void), Line 67} {0xd619e80000d59ee8}	unsigned __int64 *
+		[0x00000005]	0x00007ff64fe16228 {UnitTests.exe!UnitTests_Program____Main___g__RunTest_0_0(void), Line 23} {0xb60fc1940f64f883}	unsigned __int64 *
+		[0x00000006]	0x00007ff64fe160c9 {UnitTests.exe!UnitTests_Program___Main__(void), Line 7} {0x8548204b8b48f023}	unsigned __int64 *
+		[0x00000007]	0x00007ff64fe6007f {Inside UnitTests.exe!UnitTests__Module___StartupCodeMain(void)} {0x074e303d8d48f08b}	unsigned __int64 *
+		[0x00000008]	0x00007ff64fc13400 {UnitTests.exe!wmain(int argc, wchar_t * * argv), Line 225} {0x48ffffffffb805eb}	unsigned __int64 *

However when we get into RhpCallCatchFunclet the shadow stack get unwound just above that and then our exception dispatch is pushed

+		[0x00000000]	0x00007ff64fd9eac7 {UnitTests.exe!S_P_CoreLib_System_Runtime_EH__DispatchEx(void), Line 934} {0x480012e3320d8d48}	unsigned __int64 *
+		[0x00000001]	0x00007ff64fd9e73b {UnitTests.exe!S_P_CoreLib_System_Runtime_EH__RhThrowHwEx(void), Line 620} {0x5e5d5b28c4834890}	unsigned __int64 *
+		[0x00000002]	0x00007ff64fc18b1b {UnitTests.exe!unsigned char RhpThrowHwEx2} {0x448d4800401f0fcc}	unsigned __int64 *
+		[0x00000003]	0x00007ff64fe103ed {UnitTests.exe!UnitTests_Generics__Run(void), Line 67} {0xd619e80000d59ee8}	unsigned __int64 *
+		[0x00000004]	0x00007ff64fe16228 {UnitTests.exe!UnitTests_Program____Main___g__RunTest_0_0(void), Line 23} {0xb60fc1940f64f883}	unsigned __int64 *
+		[0x00000005]	0x00007ff64fe160c9 {UnitTests.exe!UnitTests_Program___Main__(void), Line 7} {0x8548204b8b48f023}	unsigned __int64 *
+		[0x00000006]	0x00007ff64fe6007f {Inside UnitTests.exe!UnitTests__Module___StartupCodeMain(void)} {0x074e303d8d48f08b}	unsigned __int64 *
+		[0x00000007]	0x00007ff64fc13400 {UnitTests.exe!wmain(int argc, wchar_t * * argv), Line 225} {0x48ffffffffb805eb}	unsigned __int64 *

Notice that UnitTests_Generics_TestNativeLayoutGeneration__Run is missing. That is where the AV happened.
We still beleive that it is on stack.

>	UnitTests.exe!RhpCallCatchFunclet() Line 493	Unknown	Symbols loaded.
 	UnitTests.exe!S_P_CoreLib_System_Runtime_EH__DispatchEx() Line 934	Unknown	Symbols loaded.
 	UnitTests.exe!S_P_CoreLib_System_Runtime_EH__RhThrowHwEx() Line 620	Unknown	Symbols loaded.
 	UnitTests.exe!RhpThrowHwEx() Line 102	Unknown	Symbols loaded.
 	UnitTests.exe!UnitTests_Generics_TestNativeLayoutGeneration__Run() Line 2367	Unknown	Symbols loaded.
 	UnitTests.exe!UnitTests_Generics__Run() Line 67	Unknown	Symbols loaded.
 	UnitTests.exe!UnitTests_Program____Main___g__RunTest_0_0() Line 23	Unknown	Symbols loaded.
 	UnitTests.exe!UnitTests_Program___Main__() Line 7	Unknown	Symbols loaded.
 	UnitTests.exe!UnitTests__Module___StartupCodeMain()	Unknown	Symbols loaded.
 	UnitTests.exe!wmain(int argc=0x00000001, wchar_t * * argv=0x0000029bf0b499d0) Line 225	C++	Symbols loaded.

Basically shadow stack does not match the actual. In particular the faulting IP is not on the shadow stack. That is why we cannot find it when we want to unwind SSP to the frame just above the fault location.

I guess the only way to handle this situation is to perform the same search for the faulting IP on the shadow stack as we do in RhpCallCatchFunclet, but earlier - while still inside the vectored exception handler (when the IP is still on the stack) and pass the point to which SSP should be unwound all the way to RhpCallCatchFunclet

[VSadov]

Or we can somehow figure that we are in a fault throwing a HW exception and search the SSP for RhpThrowHwEx2. The frame that we need will be just above that.