using serde_derive without precompiled binary

[decathorpe]

I'm working on packaging serde for Fedora Linux, and I noticed that recent versions of serde_derive ship a precompiled binary now. This is problematic for us, since we cannot, under no circumstances (with only very few exceptions, for firmware or the like), redistribute precompiled binaries.

Right now the fallback I am trying to apply for the short-term is to patch serde_derive/lib.rs to unconditionally include lib_from_source.rs (we don't really care about compilation speed for our non-interactive builds).

I'm wondering, how is the x86_64-unknown-linux-gnu binary actually produced? The workspace layout in this project looks very differently from when I last looked in here ... Would it be possible for us to re-create the binary ourselves so we can actually ship it? Or would it be possible to adapt the serde_derive crate to fall back to the non-precompiled code path if the binary file is missing?

[dtolnay]

the fallback is to patch serde_derive/lib.rs to unconditionally include lib_from_source.rs

This is fine. If it's any simpler, you can even rename lib_from_source.rs to lib.rs.

how is the x86_64-unknown-linux-gnu binary actually produced? Would it be possible for us to re-create the binary ourselves so we can actually ship it?

By https://github.com/serde-rs/serde/blob/v1.0.177/precompiled/build.sh. Yes.

would it be possible to adapt the serde_derive crate to fall back to the non-precompiled code path if the binary file is missing?

No, not when serde_derive is being built by Cargo. The dependency graph (whether or not serde_derive depends on syn, quote, proc-macro2) must be resolved before anything compiles.

[cuviper]

It could be a Cargo feature to force a rebuild, enabling the dependencies as well.

[decathorpe]

Thanks for confirming - I'll apply a downstream patch for now.

I'm looking forward to the compile time savings that would be achievable by changes like this in the future 💪🏼

[quentinmit]

Trying to get this to work with cargo2nix, which has a similar problem, and I found that renaming/patching lib_from_source.rs is insufficient because the Cargo.toml also excludes the dependencies on linux. It's also particularly hard to patch in extra dependencies, so for the moment I'm stuck with pinning the previous version.

Please provide a feature that can be used to disable precompiled binaries so it can be turned off in a cargo-compatible way.

[decathorpe]

@quentinmit these are my current patches for Fedora, maybe this could work to provide a similar workaround for cargo2nix?

• https://src.fedoraproject.org/rpms/rust-serde_derive/blob/rawhide/f/serde_derive-fix-metadata.diff
• https://src.fedoraproject.org/rpms/rust-serde_derive/blob/rawhide/f/0001-Drop-usage-of-precompiled-binary-that-s-not-usable-f.patch

[pmatouse]

@dtolnay please consider moving the precompiled serde_derive version to a different crate and default serde_derive to building from source so that users that want the benefit of precompiled binary can opt-in to use it. or vice-versa. or any other solution that allows building from source without having to patch serde_derive.

having a binary shipped as part of the crate, while I understand the build time speed benefits, is for security reasons not a viable solution for some library users.

thank you for your consideration.

[nightkr]

From Nix's perspective the issue is that the dependency's manifest (project) folder is not available when building dependees, only its build output.

In theory this could be worked around by include_bytes!()-ing it instead, but that would require it to be extracted before it could be executed. Personally I'd rather just see a feature to ignore the precompiled binary and revert to the old behaviour.

[ericmcbride]

This broke our Bazel build system as well, pinning to 1.0.171 resolved it