rewriter: add `IA2_{CON,DE}STRUCTOR` attribute macros, emit `ia2_type_registry_*` calls, and enable `type_confusion` tests #524

kkysen · 2025-03-09T09:38:36Z

Note that this PR includes #521 (git cherry-picked) since the branches are orthogonal (#521 points directly to main), so once #521 is merged, those commits can be rebased out of this PR.

tools/rewriter/GenCallAsm.cpp

fw-immunant

Code looks OK (one comment inline), but tests seem to be failing; fix them before merging.

tools/rewriter/GenCallAsm.cpp

fw-immunant · 2025-06-05T01:02:07Z

I diagnosed and (as far as I can tell locally) fixed the compartment violation segfault here. It stemmed from the default panic handler using TLS to access the panic counter. While I was here, I slimmed down the Rust so it has a minimal impact on the call-gate shims; we don't want to bring in libstd if we can avoid it.

fw-immunant · 2025-06-05T01:05:02Z

CI is failing with the tracer because donna has 0xfffffff0 in /sys/kernel/debug/x86/init_pkru. When we reboot or set it back to 0x55555554 CI here should pass.

tests/type_confusion/CMakeLists.txt

kkysen · 2025-06-10T02:44:55Z

runtime/type-registry/src/lib.rs

-use std::sync::LazyLock;
-use std::sync::RwLock;
+#![no_std]
+/* use the C allocator; don't use libstd */


Why do we need to avoid std? It doesn't seem smart to instead write our own Mutex; there's a lot we can do incorrectly.

If we're going to write our own Mutex, we should use no_std lock_api to write it.

We want to be careful about what we're including in the TCB and libstd does a lot on its own, for example randomization of hash maps and panic infrastructure that allocates and accesses thead locals. We want to start from zero and control everything, because we can't assume to be operating in a normal process context.

kkysen · 2025-06-10T02:45:58Z

runtime/type-registry/src/lib.rs

@@ -44,7 +162,9 @@ impl Debug for TypeId {
    }
 }

-static GLOBAL_TYPE_REGISTRY: LazyLock<TypeRegistry> = LazyLock::new(TypeRegistry::default);
+static GLOBAL_TYPE_REGISTRY: TypeRegistry = TypeRegistry {
+    map: Mutex::new(BTreeMap::new()),


We can build a const-compatible HashMap now (HashMap::with_hasher(BuildHasherDefault::new())).

The de-LazyLock PR is merged; the remaining reason to use BTreeMap instead is that HashMap requires std.

Ah, I didn't realize that (rust-lang/rust#27242). That's annoying. We could just use hashbrown, though, which is what std uses, and it can be used with no_std + alloc.

runtime/type-registry/src/lib.rs

kkysen · 2025-06-10T10:01:12Z

runtime/type-registry/Cargo.toml

@@ -15,6 +15,8 @@ crate-type = ["staticlib"]

 [profile.dev]
 panic = "abort"
+lto = true


Why do we need LTO for debug builds?

I hadn't compared sizes in debug builds. It turns out it doesn't help us much there (in fact it somehow increases .so size by ~0.1M?), and I'm not sure why:

debug+lto: $ du -sh runtime/type-registry/debug/libtype_registry.a 6,2M runtime/type-registry/debug/libtype_registry.a $ du -sh tests/type_confusion/libtype_confusion_call_gates.so 1,8M tests/type_confusion/libtype_confusion_call_gates.so debug sans lto: $ du -sh runtime/type-registry/debug/libtype_registry.a 7,8M runtime/type-registry/debug/libtype_registry.a $ du -sh tests/type_confusion/libtype_confusion_call_gates.so 1,7M tests/type_confusion/libtype_confusion_call_gates.so release+lto: $ du -sh runtime/type-registry/release/libtype_registry.a 3,1M runtime/type-registry/release/libtype_registry.a $ du -sh tests/type_confusion/libtype_confusion_call_gates.so 68K tests/type_confusion/libtype_confusion_call_gates.so release sans lto: $ du -sh runtime/type-registry/release/libtype_registry.a 6,4M runtime/type-registry/release/libtype_registry.a $ du -sh tests/type_confusion/libtype_confusion_call_gates.so 1,2M tests/type_confusion/libtype_confusion_call_gates.so

The critical number here is that we only take 68kb with release+LTO, which is reasonable to be including in our call gates. If there's a major drawback to using LTO in debug we can disable it.

LTO is just not normally used for debug builds at all. It's link time optimization when there's no optimization happening in the first place. I'm fine with enabling it for release builds, though; that makes a lot of sense.

… APIs to match the format

…n` tests `WITHOUT_SANDBOX` is added in `CMakeLists.txt`, as the tracer bug that causes tests to hang indefinitely is fixed in #488, but #488 is now moved out of this PR stack, as it needs more work.

…y and add clearer docs

the latter is a no-op in some build configurations

this takes _call_gates.so from MB to double-digit kB

HashMap and RwLock require libstd, but with BTreeMap and a simple Mutex of our own, we can avoid it finally, we define our own panic handler because even in panic=abort mode, the libstd one writes to a thread-local for the panic count and does more work than we want

kkysen requested a review from fw-immunant March 9, 2025 09:38

kkysen force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch 2 times, most recently from 0e6f7bc to 7369994 Compare March 11, 2025 00:02

kkysen force-pushed the kkysen/IA2_PRE_POST_CONDITION-minor-fixes branch from ee114a4 to d2fbef6 Compare March 12, 2025 19:37

kkysen force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch from 7369994 to 8791b04 Compare May 7, 2025 00:53

kkysen force-pushed the kkysen/IA2_PRE_POST_CONDITION-minor-fixes branch 2 times, most recently from 9009d7d to c691d75 Compare May 7, 2025 08:07

kkysen force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch from 8791b04 to 37710ee Compare May 7, 2025 08:07

kkysen force-pushed the kkysen/IA2_PRE_POST_CONDITION-minor-fixes branch from c691d75 to 1d9bbbb Compare May 7, 2025 20:47

kkysen force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch from 37710ee to 69f8e0a Compare May 7, 2025 20:48

kkysen force-pushed the kkysen/IA2_PRE_POST_CONDITION-minor-fixes branch from 1d9bbbb to 7a56926 Compare May 9, 2025 08:48

kkysen force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch from 69f8e0a to afac829 Compare May 9, 2025 08:54

kkysen force-pushed the kkysen/IA2_PRE_POST_CONDITION-minor-fixes branch from 7a56926 to 589c938 Compare May 10, 2025 08:21

Base automatically changed from kkysen/IA2_PRE_POST_CONDITION-minor-fixes to main May 10, 2025 08:35

kkysen force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch from afac829 to 57b933d Compare May 10, 2025 10:11

fw-immunant reviewed May 12, 2025

View reviewed changes

tools/rewriter/GenCallAsm.cpp Outdated Show resolved Hide resolved

kkysen force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch 2 times, most recently from 9f4155a to ba28321 Compare May 13, 2025 17:55

kkysen changed the base branch from main to kkysen/Context May 13, 2025 17:56

kkysen force-pushed the kkysen/Context branch from 781fb15 to 7caa8a8 Compare May 21, 2025 03:40

kkysen force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch from ba28321 to a0da8ae Compare May 21, 2025 03:40

Base automatically changed from kkysen/Context to main May 21, 2025 05:46

kkysen force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch 2 times, most recently from d207a80 to 4bf1d75 Compare May 22, 2025 11:14

fw-immunant approved these changes May 23, 2025

View reviewed changes

tools/rewriter/GenCallAsm.cpp Outdated Show resolved Hide resolved

kkysen force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch from 4bf1d75 to ac42bf8 Compare May 27, 2025 14:08

kkysen commented May 27, 2025

View reviewed changes

tools/rewriter/GenCallAsm.cpp Show resolved Hide resolved

kkysen force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch from b6f1f13 to aa6cec2 Compare June 2, 2025 18:26

fw-immunant force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch from d4f7180 to 2067064 Compare June 6, 2025 15:20

kkysen commented Jun 9, 2025

View reviewed changes

tests/type_confusion/CMakeLists.txt Outdated Show resolved Hide resolved

kkysen commented Jun 10, 2025

View reviewed changes

runtime/type-registry/src/lib.rs Outdated Show resolved Hide resolved

kkysen commented Jun 10, 2025

View reviewed changes

fw-immunant force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch from 2067064 to 994870c Compare June 10, 2025 16:36

kkysen and others added 11 commits June 10, 2025 12:37

rewriter: add Structors pass to find {con,de}structors

0aae26b

runtime: add IA2_{CON,DE}STRUCTOR attribute macros

e8f21e0

tests/type_confusion: add IA2_{CON,DE}STRUCTOR macros and update fn…

458acd4

… APIs to match the format

rewriter: refactor out emit_(revert_)align_stack

da9d784

rewriter: emit ia2_type_registry_* calls and turn on `type_confusio…

334b1a9

…n` tests `WITHOUT_SANDBOX` is added in `CMakeLists.txt`, as the tracer bug that causes tests to hang indefinitely is fixed in #488, but #488 is now moved out of this PR stack, as it needs more work.

rewriter: rename emit_(revert_)align_stack{ => _8_to_16} for clarit…

ca76445

…y and add clearer docs

rewriter: refactor out emit_type_registry_checks

bc08b53

test_runner: make cr_assert unconditional

ccebcef

test_runner: use cr_assert, not assert

5ba0712

the latter is a no-op in some build configurations

type-registry: use lto

d1a3d40

this takes _call_gates.so from MB to double-digit kB

type-registry: add libc_alloc dependency

d97d25f

fw-immunant force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch from 994870c to 55e0440 Compare June 10, 2025 16:39

fw-immunant added 2 commits June 10, 2025 12:45

type-registry: avoid allocating in panic handler

7e7aaac

fw-immunant force-pushed the kkysen/IA2_CON_DE_STRUCTOR branch from 55e0440 to 7e7aaac Compare June 10, 2025 16:47

This was referenced Jun 13, 2025

test_runner: make cr_assert unconditional and use instead of assert #596

Closed

type-registry: use LTO for release builds only #597

Merged

SIGABRT and some other signals in tests cause a ~1 s delay in the test runner #601

Closed

fw-immunant merged commit d93149b into main Jun 17, 2025
35 checks passed

fw-immunant mentioned this pull request Jun 17, 2025

type-registry: use panic_abort! instead of panic! #598

Closed

kkysen deleted the kkysen/IA2_CON_DE_STRUCTOR branch June 17, 2025 17:57

kkysen mentioned this pull request Jun 19, 2025

#524 broke cargo test for type-registry. #603

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

rewriter: add `IA2_{CON,DE}STRUCTOR` attribute macros, emit `ia2_type_registry_*` calls, and enable `type_confusion` tests #524

rewriter: add `IA2_{CON,DE}STRUCTOR` attribute macros, emit `ia2_type_registry_*` calls, and enable `type_confusion` tests #524

Uh oh!

kkysen commented Mar 9, 2025

Uh oh!

Uh oh!

fw-immunant left a comment

Uh oh!

Uh oh!

Uh oh!

fw-immunant commented Jun 5, 2025

Uh oh!

fw-immunant commented Jun 5, 2025

Uh oh!

Uh oh!

kkysen Jun 10, 2025

Uh oh!

kkysen Jun 10, 2025

Uh oh!

fw-immunant Jun 10, 2025

Uh oh!

kkysen Jun 10, 2025

Uh oh!

fw-immunant Jun 10, 2025

Uh oh!

kkysen Jun 10, 2025

Uh oh!

Uh oh!

kkysen Jun 10, 2025

Uh oh!

fw-immunant Jun 10, 2025

Uh oh!

kkysen Jun 10, 2025

Uh oh!

Uh oh!

Uh oh!

rewriter: add IA2_{CON,DE}STRUCTOR attribute macros, emit ia2_type_registry_* calls, and enable type_confusion tests #524

rewriter: add IA2_{CON,DE}STRUCTOR attribute macros, emit ia2_type_registry_* calls, and enable type_confusion tests #524

Uh oh!

Conversation

kkysen commented Mar 9, 2025

Uh oh!

Uh oh!

fw-immunant left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

fw-immunant commented Jun 5, 2025

Uh oh!

fw-immunant commented Jun 5, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rewriter: add `IA2_{CON,DE}STRUCTOR` attribute macros, emit `ia2_type_registry_*` calls, and enable `type_confusion` tests #524

rewriter: add `IA2_{CON,DE}STRUCTOR` attribute macros, emit `ia2_type_registry_*` calls, and enable `type_confusion` tests #524