"(signal: 11, SIGSEGV: invalid memory reference)" when compiling regex crate on recent nightlies

[dvdplm]

After the LVI hardening PR landed on June 8 2020 the rustc nightlies fail to compile the regex crate for the x86_64-fortanix-unknown-sgx target in our project.

I have tested with nightly-2020-06-07 (fails) and nightly-2020-06-09 (compiles).

Will update with a minimal repro. The regex crate on its own compiles just fine, so it's likely a specific set of features.

[dvdplm]

Repro

1. run cargo new regex-on-sgx-bug-repro
2. edit the Cargo.toml file to look like this:

[package]
name = "regex-on-sgx-bug-repro"
version = "0.1.0"
edition = "2018"

[dependencies]
regex = { version = "1" }

3. compile in release mode against the x86_64-fortanix-unknown-sgx target with a rustc after June 8:

cargo build --target=x86_64-fortanix-unknown-sgx --release

4. notice the compiler error:

   Compiling regex v1.3.9
error: could not compile `regex`.

Caused by:
  process didn't exit successfully: `rustc --crate-name regex /home/dvdplm/.cargo/registry/src/github.jpy.wang-1ecc6299db9ec823/regex-1.3.9/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="aho-corasick"' --cfg 'feature="default"' --cfg 'feature="memchr"' --cfg 'feature="perf"' --cfg 'feature="perf-cache"' --cfg 'feature="perf-dfa"' --cfg 'feature="perf-inline"' --cfg 'feature="perf-literal"' --cfg 'feature="std"' --cfg 'feature="thread_local"' --cfg 'feature="unicode"' --cfg 'feature="unicode-age"' --cfg 'feature="unicode-bool"' --cfg 'feature="unicode-case"' --cfg 'feature="unicode-gencat"' --cfg 'feature="unicode-perl"' --cfg 'feature="unicode-script"' --cfg 'feature="unicode-segment"' -C metadata=31a18be4b57f1346 -C extra-filename=-31a18be4b57f1346 --out-dir /home/dvdplm/work/regex-on-sgx-bug-repro/target/x86_64-fortanix-unknown-sgx/release/deps --target x86_64-fortanix-unknown-sgx -L dependency=/home/dvdplm/work/regex-on-sgx-bug-repro/target/x86_64-fortanix-unknown-sgx/release/deps -L dependency=/home/dvdplm/work/regex-on-sgx-bug-repro/target/release/deps --extern aho_corasick=/home/dvdplm/work/regex-on-sgx-bug-repro/target/x86_64-fortanix-unknown-sgx/release/deps/libaho_corasick-112ba46620bbe885.rmeta --extern memchr=/home/dvdplm/work/regex-on-sgx-bug-repro/target/x86_64-fortanix-unknown-sgx/release/deps/libmemchr-0c80a0008d259313.rmeta --extern regex_syntax=/home/dvdplm/work/regex-on-sgx-bug-repro/target/x86_64-fortanix-unknown-sgx/release/deps/libregex_syntax-850a153c4694c3ef.rmeta --extern thread_local=/home/dvdplm/work/regex-on-sgx-bug-repro/target/x86_64-fortanix-unknown-sgx/release/deps/libthread_local-df4459ab27a7d484.rmeta --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)

[AdrianCX]

Quick debug:

gdb --args rustc --crate-name regex /home/adrian/.cargo/registry/src/github.jpy.wang-1ecc6299db9ec823/regex-1.3.9/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="aho-corasick"' --cfg 'feature="default"' --cfg 'feature="memchr"' --cfg 'feature="perf"' --cfg 'feature="perf-cache"' --cfg 'feature="perf-dfa"' --cfg 'feature="perf-inline"' --cfg 'feature="perf-literal"' --cfg 'feature="std"' --cfg 'feature="thread_local"' --cfg 'feature="unicode"' --cfg 'feature="unicode-age"' --cfg 'feature="unicode-bool"' --cfg 'feature="unicode-case"' --cfg 'feature="unicode-gencat"' --cfg 'feature="unicode-perl"' --cfg 'feature="unicode-script"' --cfg 'feature="unicode-segment"' -C metadata=cfb51a7a22bedcb3 -C extra-filename=-cfb51a7a22bedcb3 --out-dir /home/adrian/work/segv/regex-on-sgx-bug-repro/target/x86_64-fortanix-unknown-sgx/release/deps --target x86_64-fortanix-unknown-sgx -L dependency=/home/adrian/work/segv/regex-on-sgx-bug-repro/target/x86_64-fortanix-unknown-sgx/release/deps -L dependency=/home/adrian/work/segv/regex-on-sgx-bug-repro/target/release/deps --extern aho_corasick=/home/adrian/work/segv/regex-on-sgx-bug-repro/target/x86_64-fortanix-unknown-sgx/release/deps/libaho_corasick-21b2878f0311f15d.rmeta --extern memchr=/home/adrian/work/segv/regex-on-sgx-bug-repro/target/x86_64-fortanix-unknown-sgx/release/deps/libmemchr-69a44d1854317b68.rmeta --extern regex_syntax=/home/adrian/work/segv/regex-on-sgx-bug-repro/target/x86_64-fortanix-unknown-sgx/release/deps/libregex_syntax-3ba9e18e48174a50.rmeta --extern thread_local=/home/adrian/work/segv/regex-on-sgx-bug-repro/target/x86_64-fortanix-unknown-sgx/release/deps/libthread_local-4c21b716971c1938.rmeta --cap-lints allow

Stack trace:

#0  0x00007ffff27d0fc2 in (anonymous namespace)::X86LoadValueInjectionRetHardeningPass::runOnMachineFunction(llvm::MachineFunction&) ()
   from /home/adrian/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/../lib/libLLVM-10-rust-1.46.0-nightly.so
#1  0x00007ffff061903e in llvm::MachineFunctionPass::runOnFunction(llvm::Function&) () from /home/adrian/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/../lib/libLLVM-10-rust-1.46.0-nightly.so
#2  0x00007ffff0412b8f in llvm::FPPassManager::runOnFunction(llvm::Function&) () from /home/adrian/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/../lib/libLLVM-10-rust-1.46.0-nightly.so
#3  0x00007ffff0412fc3 in llvm::FPPassManager::runOnModule(llvm::Module&) () from /home/adrian/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/../lib/libLLVM-10-rust-1.46.0-nightly.so
#4  0x00007ffff0413760 in llvm::legacy::PassManagerImpl::run(llvm::Module&) () from /home/adrian/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/../lib/libLLVM-10-rust-1.46.0-nightly.so
#5  0x00007ffff465c42f in LLVMRustWriteOutputFile () from /home/adrian/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-5e50657ff02a973e.so
#6  0x00007ffff45e80ec in rustc_codegen_llvm::back::write::write_output_file::hc90a2827bb923d12 ()
   from /home/adrian/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-5e50657ff02a973e.so
#7  0x00007ffff45f6691 in _$LT$rustc_codegen_llvm..LlvmCodegenBackend$u20$as$u20$rustc_codegen_ssa..traits..write..WriteBackendMethods$GT$::codegen::ha8c39af6b7c16298 ()
   from /home/adrian/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-5e50657ff02a973e.so
#8  0x00007ffff45d390e in rustc_codegen_ssa::back::write::execute_work_item::h231de6df064a4aa2 ()
   from /home/adrian/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-5e50657ff02a973e.so
#9  0x00007ffff44be1de in std::sys_common::backtrace::__rust_begin_short_backtrace::hdfebaabb7e3ed33a ()
   from /home/adrian/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-5e50657ff02a973e.so
#10 0x00007ffff44f4685 in core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hca1d744f80cd1e97 ()
   from /home/adrian/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-5e50657ff02a973e.so
#11 0x00007ffff3a2869a in _$LT$alloc..boxed..Box$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$A$GT$$GT$::call_once::h13d34828db364579 ()
    at /rustc/6bb3dbfc6c6d8992d08431f320ba296a0c2f7498/src/liballoc/boxed.rs:1076
#12 _$LT$alloc..boxed..Box$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$A$GT$$GT$::call_once::hd51b619e0f884abf () at /rustc/6bb3dbfc6c6d8992d08431f320ba296a0c2f7498/src/liballoc/boxed.rs:1076
#13 std::sys::unix::thread::Thread::new::thread_start::h02c6e34c2c73f344 () at src/libstd/sys/unix/thread.rs:87
#14 0x00007ffff378f6ba in start_thread (arg=0x7fffe0f7f700) at pthread_create.c:333
#15 0x00007ffff30b941d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

[AdrianCX]

With debug symbols and asserts:

Failure:

rustc: /home/adrian/work/forks/rust/src/llvm-project/llvm/include/llvm/ADT/ilist_iterator.h:139: llvm::ilist_iterator<OptionsT, IsReverse, IsConst>::reference llvm::ilist_iterator<OptionsT, IsReverse, IsConst>::operator*() const [with OptionsT = llvm::ilist_detail::node_options<llvm::MachineInstr, true, true, void>; bool IsReverse = false; bool IsConst = false; llvm::ilist_iterator<OptionsT, IsReverse, IsConst>::reference = llvm::MachineInstr&]: Assertion `!NodePtr->isKnownSentinel()' failed.

#0  0x00007fffebb01428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007fffebb0302a in __GI_abort () at abort.c:89
#2  0x00007fffebaf9bd7 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x7ffff316d4ad "!NodePtr->isKnownSentinel()", 
    file=file@entry=0x7ffff316d458 "/home/adrian/work/forks/rust/src/llvm-project/llvm/include/llvm/ADT/ilist_iterator.h", line=line@entry=139, 
    function=function@entry=0x7ffff316dbe0 <llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineInstr, true, true, void>, false, false>::operator*() const::__PRETTY_FUNCTION__> "llvm::ilist_iterator<OptionsT, IsReverse, IsConst>::reference llvm::ilist_iterator<OptionsT, IsReverse, IsConst>::operator*() const [with OptionsT = llvm::ilist_detail::node_options<llvm::MachineInstr"...) at assert.c:92
#3  0x00007fffebaf9c82 in __GI___assert_fail (assertion=0x7ffff316d4ad "!NodePtr->isKnownSentinel()", 
    file=0x7ffff316d458 "/home/adrian/work/forks/rust/src/llvm-project/llvm/include/llvm/ADT/ilist_iterator.h", line=139, 
    function=0x7ffff316dbe0 <llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineInstr, true, true, void>, false, false>::operator*() const::__PRETTY_FUNCTION__> "llvm::ilist_iterator<OptionsT, IsReverse, IsConst>::reference llvm::ilist_iterator<OptionsT, IsReverse, IsConst>::operator*() const [with OptionsT = llvm::ilist_detail::node_options<llvm::MachineInstr"...) at assert.c:101
#4  0x00007fffed1b68f8 in llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineInstr, true, true, void>, false, false>::operator* (this=0x7fffd9ff3810)
    at /home/adrian/work/forks/rust/src/llvm-project/llvm/include/llvm/ADT/ilist_iterator.h:139
#5  0x00007fffed1b5ea8 in llvm::MachineInstrBundleIterator<llvm::MachineInstr, false>::operator* (this=0x7fffd9ff3810)
    at /home/adrian/work/forks/rust/src/llvm-project/llvm/include/llvm/CodeGen/MachineInstrBundleIterator.h:178
#6  0x00007fffed49b509 in llvm::MachineBasicBlock::back (this=0x7fff983a4a70) at /home/adrian/work/forks/rust/src/llvm-project/llvm/include/llvm/CodeGen/MachineBasicBlock.h:196
#7  0x00007fffed4f78dd in (anonymous namespace)::X86LoadValueInjectionRetHardeningPass::runOnMachineFunction (this=0x7fff98137dd0, MF=...)
    at /home/adrian/work/forks/rust/src/llvm-project/llvm/lib/Target/X86/X86LoadValueInjectionRetHardening.cpp:102
#8  0x00007fffef133f86 in llvm::MachineFunctionPass::runOnFunction (this=0x7fff98137dd0, F=...) at /home/adrian/work/forks/rust/src/llvm-project/llvm/lib/CodeGen/MachineFunctionPass.cpp:73
#9  0x00007ffff01fbe73 in llvm::FPPassManager::runOnFunction (this=0x7fff980d62f0, F=...) at /home/adrian/work/forks/rust/src/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1481
#10 0x00007ffff01fc10d in llvm::FPPassManager::runOnModule (this=0x7fff980d62f0, M=...) at /home/adrian/work/forks/rust/src/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1517
#11 0x00007ffff01fc541 in (anonymous namespace)::MPPassManager::runOnModule (this=0x7fff980d7d80, M=...) at /home/adrian/work/forks/rust/src/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1582
#12 0x00007ffff01fccfd in llvm::legacy::PassManagerImpl::run (this=0x7fff982b4740, M=...) at /home/adrian/work/forks/rust/src/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1694
#13 0x00007ffff01fcf0d in llvm::legacy::PassManager::run (this=0x7fff980e7970, M=...) at /home/adrian/work/forks/rust/src/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1725
#14 0x00007fffed15efb5 in LLVMRustWriteOutputFile () from /home/adrian/work/forks/rust/build/x86_64-unknown-linux-gnu/stage2/bin/../lib/librustc_driver-ffe092153b2873b6.so
#15 0x00007fffed0dfa7c in rustc_codegen_llvm::back::write::write_output_file::h8c79908e73d667e7 (handler=0x7fffd9ff43c0, target=0x25e16, pm=0x25e39, m=0x6, output=0x7fff98240520, file_type=ObjectFile)
    at src/librustc_codegen_llvm/back/write.rs:55
#16 0x00007fffed0ee021 in rustc_codegen_llvm::back::write::codegen::_$u7b$$u7b$closure$u7d$$u7d$::hb505fb2137b3a870 (cpm=0x7fff980e7970) at src/librustc_codegen_llvm/back/write.rs:770
#17 rustc_codegen_llvm::back::write::codegen::with_codegen::hb4c33854bc53e6a1 (tm=0x7fff98000a10, llmod=0x7fff98005540, no_builtins=<optimized out>, f=...) at src/librustc_codegen_llvm/back/write.rs:658
#18 rustc_codegen_llvm::back::write::codegen::h109860b82c79452f (cgcx=0x7fffd9ff4668, diag_handler=0x7fffd9ff43c0, module=..., config=<optimized out>) at src/librustc_codegen_llvm/back/write.rs:769
#19 _$LT$rustc_codegen_llvm..LlvmCodegenBackend$u20$as$u20$rustc_codegen_ssa..traits..write..WriteBackendMethods$GT$::codegen::h10a75730f6c85806 (cgcx=0x7fffd9ff4668, diag_handler=0x7fffd9ff43c0, module=..., 
    config=0x7fffe6e25b10) at src/librustc_codegen_llvm/lib.rs:166
#20 0x00007fffed0c670e in rustc_codegen_ssa::back::write::execute_lto_work_item::h8bd7b3be71408c4a (cgcx=0x7fffd9ff4668, module=..., module_config=0x7fffe6e25b10)
    at /home/adrian/work/forks/rust/src/librustc_codegen_ssa/back/write.rs:878
#21 rustc_codegen_ssa::back::write::execute_work_item::h8ee3a1f38352d869 (cgcx=0x7fffd9ff4668, work_item=...) at /home/adrian/work/forks/rust/src/librustc_codegen_ssa/back/write.rs:725
#22 0x00007fffecfb061e in rustc_codegen_ssa::back::write::spawn_work::_$u7b$$u7b$closure$u7d$$u7d$::hb1c431603bd9bc1f () at /home/adrian/work/forks/rust/src/librustc_codegen_ssa/back/write.rs:1553
#23 std::sys_common::backtrace::__rust_begin_short_backtrace::hbc3693d63a24ac0f (f=...) at /home/adrian/work/forks/rust/src/libstd/sys_common/backtrace.rs:130
#24 0x00007fffecfe6615 in std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h701989b5ab992fe1 () at /home/adrian/work/forks/rust/src/libstd/thread/mod.rs:475
#25 _$LT$std..panic..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h985d6a144895dab2 (self=..., _args=<optimized out>)
    at /home/adrian/work/forks/rust/src/libstd/panic.rs:318
#26 std::panicking::try::do_call::h40c250001af7d622 (data=<optimized out>) at /home/adrian/work/forks/rust/src/libstd/panicking.rs:296
#27 std::panicking::try::h7169173aba098c2e (f=...) at /home/adrian/work/forks/rust/src/libstd/panicking.rs:273
#28 std::panic::catch_unwind::h0a7f081fdd5c74e0 (f=...) at /home/adrian/work/forks/rust/src/libstd/panic.rs:394
#29 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::hc9fcb206e095e365 () at /home/adrian/work/forks/rust/src/libstd/thread/mod.rs:474
#30 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h28ae7dd985eda5f1 () at /home/adrian/work/forks/rust/src/libcore/ops/function.rs:232
#31 0x00007fffebefd458 in _$LT$alloc..boxed..Box$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$A$GT$$GT$::call_once::h179d78afe4596b81 (self=..., args=<optimized out>)
    at /home/adrian/work/forks/rust/src/liballoc/boxed.rs:1076
#32 _$LT$alloc..boxed..Box$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$A$GT$$GT$::call_once::h5a273b8025c93696 (self=0x7fffd40040b0, args=<optimized out>)
    at /home/adrian/work/forks/rust/src/liballoc/boxed.rs:1076
#33 std::sys::unix::thread::Thread::new::thread_start::h2623899efc8402a9 (main=0x7fffd40040b0) at src/libstd/sys/unix/thread.rs:87
#34 0x00007fffeb4986ba in start_thread (arg=0x7fffd9ff5700) at pthread_create.c:333
#35 0x00007fffebbd341d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

[AdrianCX]

Crashes on line below because MBB is empty and you can't use back on that (nothing to get reference to).

Line 102: https://github.com/rust-lang/llvm-project/blob/rustc/10.0-2020-05-05/llvm/lib/Target/X86/X86LoadValueInjectionRetHardening.cpp

    MachineInstr &MI = MBB.back();

(gdb) p MBB.empty()
$22 = true

Easiest fix is to add an empty() check and raise a PR for that. Will make the change and kick off another build (takes a couple of hours) and will discuss raising a PR on Monday. (hopefully nothing else crashes)

[AdrianCX]

this is already fixed upstream in LLVM - https://github.com/llvm/llvm-project/blob/master/llvm/lib/Target/X86/X86LoadValueInjectionRetHardening.cpp

so we need to request rust-lang to update/cherry pick

llvm/llvm-project@0505181#diff-f97bb92de5266622718828396fed70cd