build-rs crate API design

[kornelski]

The #12432 issue adopted the build-rs crate, but I haven't seen discussion on how its API should look like.

The API of the already-released versions happens to be a very direct mapping of functions to the existing printed cargo:… directives. This has an advantage of being immediately familiar to Rust/Cargo users, but because the API keeps using global state and emitting Cargo directives as a side effect, it remains difficult to coordinate interaction with Cargo across build-time dependencies.

• Build-time dependencies have a dilemma whether they should emit cargo: directives automatically to have a convenient robust API, or not, and let the caller decide what to emit. cc and pkg-config crates have an option to disable printing of cargo: directives, but this is all-or-nothing, and it's not easy for callers to customize and filter the directives.

• Build-time dependencies can't emit cargo:error from Result-based APIs, because the caller of the function could recover from the error. However, this prevents libraries from emitting well-formatted Cargo errors when their callers use library.call().unwrap().

• Build-time dependencies can't emit rerun-if-changed by default, because presence of any such directive disables Cargo's standard heuristics. This needs to be configured on case-by-case basis for each build dependency, because there's no standard way for build scripts to specify whether they want the standard heuristics or not.

• The global stateless functions don't have ability to be customized in the future, unless the configuration would be global as well. However, global configuration doesn't compose well, so build-time dependencies would struggle to use it without causing issues for other parts of the build script.

• Cargo doesn't have first-class support for handling build scripts failing via a panic. Cargo just gets usual rustc output as an opaque text, which isn't formatted like a native Cargo error (with color output, etc.), and includes a panic stack trace that is irrelevant for most build scripts, especially sys crates.

Some of these issues could be solved by build dependencies themselves offering richer APIs, but I think there's an opportunity for the build-rs crate to improve Cargo's interface and offer a standard way of composing build dependencies.

[epage]

At least for now, I do not see us trying to offer this as a stable enough API for build script libraries to expose in their API.

For myself, I see this overall as low level, offering users the building blocks for a build script in a more type safe manner without providing opinions on how to do things. The most high level I've considered adding is env variable tracking like build_env

[kornelski]

I think the API could be future-proofed with a very small change: move the global stateless functions to be methods on an object.

let mut b = build_rs::Build::new();
b.rerun_if_changed(path);
b.rustc_link_lib(libfoo);

For now this would be a zero-sized object, but eventually it could become configurable to e.g. discard or buffer the directives instead of printing them immediately, and that could be passed to other dependencies as the build environment to use.

[epage]

For myself, I think its important that we offer a low level API that maps to the bare interface that Cargo provides. I have been wondering about having a type that the API is built around but more so what we print to and read from can we swapped out for testing purposes. I think any other higher level constructs would be out of scope for that type.

[kornelski]

It's reasonable to stick to what Cargo already does as a starting point, but I don't understand why you seem to be excluding possibility of expanding this interface in the future.

The current Cargo build script features are by necessity limited to what a bare env+stdout could do, but having an official Rust-native interface opens up many possibilities for improving build dependencies and error reporting.

Thinking big, if fn main() can optionally support impl Termination, it could be made to support Cargo's API too:

fn main(build: &mut build_rs::Build) {
}

which would make build scripts feel like a proper first-class Rust feature.

[epage]

Note that I didn't say that build-rs (or another crate) should never support anything more. I said specifically that we should have an API that allows people to do low level operations that forms the basis of anything else on top. Maybe one way to think of what I am suggesting is that we start with is a type-safe, rust-analyzer integrated version of Build Scripts and Environment variables Cargo sets for build scripts.

One crate can offer APIs that fill different roles (high level or low level). I do think its too early to be designing such a thing as we're still figuring out what the low level API should be! We need to iterate and get this API right, figure out what MSRV expectations are especially post-MSRV-aware resolver, what people's build-time expectations are, etc. Any of those could tank this effort.

We then need to look at and maybe help with experimenting with what to build on top. What patterns exist? What policy works generally? If you have an idea for this, please, go and create a crate and let us see what we can learn from the effort.