Remap target dir

[alepez]

Problem

I'm using ZFS filesystem, which has an instant snapshot feature very useful for disaster recovery. I use this feature to automatically make snapshots of my projects. The snapshot is applied to the whole ZFS dataset and there is no way to exclude certain directories.

The problem with Rust projects is that they may have many files in the target directory (which is usually inside the project directory). File under target directory change often (e.g. when re-building) and can be effortlessly recreated. So they don't fit well with ZFS automatic snapshotting system.

Similar problem exists if someone wants to always use a different disk or partition for target directories (which are frequently changed, effortlessly recoverable, unlike the source code).

I could override CARGO_TARGET_DIR environment variable, but I would need to remember to do it for every every project.

I could create an alias of cargo which automatically overrides CARGO_TARGET_DIR, but it would also override whatever is set as target directory in the config.toml, which can cause unpredictable issues.

Proposed Solution

I propose to add the environmental variable CARGO_TARGET_DIR_REMAP which let the user remap where target dir should be created.

Example: crate is in /home/user/projects/foo and default target directory is in /home/user/projects/foo/target. Setting CARGO_TARGET_DIR_REMAP=/home/user/projects=/tmp/cargo-build should change the real target directory so it becomes /tmp/cargo-build/foo/target

The syntax is inspired by rustc command line argument --remap-path-prefix

Notes

I've already experimented with a possible implementation. This is a simple working patch.

diff --git a/src/cargo/util/config/mod.rs b/src/cargo/util/config/mod.rs
index fddd47acb..5bc0c94ba 100644
--- a/src/cargo/util/config/mod.rs
+++ b/src/cargo/util/config/mod.rs
@@ -518,6 +518,13 @@ impl Config {
             }
 
             Ok(Some(Filesystem::new(path)))
+        } else if let Some(remap) = self.env.get("CARGO_TARGET_DIR_REMAP") {
+            remap_target_dir(remap, self.cwd.clone())
+                .map(|dir| Some(Filesystem::new(dir)))
+                .map_err(|err| anyhow!(
+                    "the target directory remap is not valid in the \
+                    `CARGO_TARGET_DIR_REMAP` environment variable. reason: {}", err)
+                )
         } else {
             Ok(None)
         }
@@ -2459,3 +2466,45 @@ macro_rules! drop_eprint {
         $crate::__shell_print!($config, err, false, $($arg)*)
     );
 }
+
+fn remap_target_dir(remap: &str, mut path: PathBuf) -> CargoResult<PathBuf> {
+    let (from, to) = remap.split_once('=')
+        .ok_or_else(|| anyhow!("must be a value of the form FROM=TO"))?;
+
+    // Change the path only if it starts with the `from` prefix
+    if path.starts_with(from) {
+        let relative = path.strip_prefix(from)?;
+        path = PathBuf::from(to).join(relative);
+    }
+
+    Ok(path)
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_remap_target_dir_start() {
+        let original_path = PathBuf::from("/foo/bar/baz");
+        let remap = String::from("/foo/bar=/new/target/prefix");
+        let remapped = remap_target_dir(&remap, original_path);
+        assert_eq!(PathBuf::from("/new/target/prefix/baz"), remapped.unwrap());
+    }
+
+    #[test]
+    fn test_remap_target_dir_different_start_unchanged() {
+        let original_path = PathBuf::from("/baz/foo/bar/baz");
+        let remap = String::from("/foo/bar=/new/target/prefix");
+        let remapped = remap_target_dir(&remap, original_path);
+        assert_eq!(PathBuf::from("/baz/foo/bar/baz"), remapped.unwrap());
+    }
+
+    #[test]
+    fn test_remap_target_dir_invalid_syntax() {
+        let original_path = PathBuf::from("/baz/foo/bar/baz");
+        let remap = String::from("/new/target/prefix");
+        let remapped = remap_target_dir(&remap, original_path);
+        assert!(remapped.is_err());
+    }
+}
\ No newline at end of file

[epage]

More use cases:

• I work at a place that does continual, whole-system, unconfigurable backup and making it easy to move the target dir out of the backup would help to avoid wasted processing from the backup
• I used to work at a place that checked all run programs against a central server and I had to authorize new programs through slack and then run a command to sync that down. This is incompatible with build scripts and cargo run. We had some exception directories and I had to regularly change my target dir to one of these to even be able to do anything.

I appreciate that this proposal is looking to precedence for a design.

Since --remap-path-prefix is a repeatable option, we would need to specify what a separator character is for the env variable (most likely the same used in PATH). I'm assuming we'd also want a config field for this.

[alepez]

Being a repeatable option can be useful. I don't know which is the best delimiter in this case. Using : as in PATH is a good idea, but maybe we already have a feature in the Rust ecosystem which is using a different delimiter?

The config field makes sense. In that case I would use the same logic already implemented for the target option.

target_dir_remap = "a=b"
target_dir_remap = ["a=b"]
target_dir_remap = ["a=b", "c=d"]

[alepez]

I'm not sure what should be the correct interaction with CARGO_TARGET_DIR env var and target_dir config.

CARGO_TARGET_DIR overrides CARGO_TARGET_DIR_REMAP

Example:

cwd=/a/b
CARGO_TARGET_DIR=c
CARGO_TARGET_DIR_REMAP=/a=/x

Result target dir is /a/b/c

CARGO_TARGET_DIR works together with CARGO_TARGET_DIR_REMAP

Example:

cwd=/a/b
CARGO_TARGET_DIR=c
CARGO_TARGET_DIR_REMAP=/a=/x

Result target dir is /x/b/c

[alepez]

And how the env var and the config field interact? One could override the other, or they could be merged.

[lorenz]

I'd personally like to see Cargo adopt an approach similar to what Bazel does, i.e. keep its cache under XDG_CACHE_DIR/cargo-whatever/... and make target a symlink to a cache directory under there. This has multiple advantages:

• It doesn't break most workflows as accesses through target are still possible. Only tools explicitly resolving the symlink might behave differently.
• It can work without configuration and still have significant benefits, i.e. most backup utilities avoid XDG_CACHE_DIR out of the box.
• It allows users to centrally customize where to put all their cache data (by modifying XDG_CACHE_DIR).

WDYT?

[epage]

I'd personally like to see Cargo adopt an approach similar to what Bazel does, i.e. keep its cache under XDG_CACHE_DIR/cargo-whatever/... and make target a symlink to a cache directory under there. This has multiple advantages:

How do we maintain a unique location within XDG_CACHE_DIR for each package directory?

It allows users to centrally customize where to put all their cache data (by modifying XDG_CACHE_DIR).

• It seems like this would help the ZFS case.
• Its "meh" for the use case I mentioned as i wouldn't be wanting to put all caches into the trusted directory
• This is disruptive to the ecosystem as people and tools that are used to operate directly on targetwill need to keep a symlink in mind. Whether we would want to consider this a major or a minor breaking change, I'm unsure.

[lorenz]

How do we maintain a unique location within XDG_CACHE_DIR for each package directory?

I see two approaches:

1. Generate a random string, create a symlink target -> $XDG_CACHE_DIR/cargo/$RANDOM_STRING, then just read the symlink back to figure out what was chosen. This is resilient to moving the source directory around and having multiple views (mount namespaces, symlinks, ...) into it. But cargo would need to check if the used symlink target is actually pointing to $XDG_CACHE_DIR/cargo/... and recreate it if it is not.
2. Hash the path to the Cargo.toml file and use that, with a similar path construction. This means the cache path can just be computed from the path of the Cargo.toml file.

I personally prefer the first option as it allows for the most flexibility at moderate complexity.

Its "meh" for the use case I mentioned as i wouldn't be wanting to put all caches into the trusted directory

True, in that specific case it also depends on how the executable checking is done and if a symlink would count. I don't know enough about a workflow would look like with such a system in place to make any recommendations here.

This is disruptive to the ecosystem as people and tools that are used to operate directly on targetwill need to keep a symlink in mind. Whether we would want to consider this a major or a minor breaking change, I'm unsure.

Most tools should not notice the difference. They would need to explicitly call readlink or similar to be affected. Cargo is responsible for creating target and thus the symlink and everything under it would just work normally.

I've done tests by creating the symlink by hand and everything I tested (cargo build, run, rust-analyzer) at it worked OOTB.

[epage]

True, in that specific case it also depends on how the executable checking is done and if a symlink would count. I don't know enough about a workflow would look like with such a system in place to make any recommendations here.

In my case, the symlink does not count

Most tools should not notice the difference. They would need to explicitly call readlink or similar to be affected. Cargo is responsible for creating target and thus the symlink and everything under it would just work normally.

For myself, I am used to running rm -rf */target. That would no longer do what I expected. We'd need more due-diligence than "probably fine".

[lorenz]

For myself, I am used to running rm -rf */target. That would no longer do what I expected. We'd need more due-diligence than "probably fine".

This would still (mostly) do what you intend (if we go with option 1 for the directory mapping), it would just leave a dangling cache directory.

[epage]

This would still (mostly) do what you intend (if we go with option 1 for the directory mapping), it would just leave a dangling cache directory.

No, I generally do it to recover space from all of my projects when I upgrade cargo. If someone knows about the move, this makes it even easier. The problem is how do we help adjust workflows.

[alepez]

@lorenz proposal would help in my case. But what about users which do not have freedesktop XDG? Or CI pipelines? The behavior would be inconsistent between different operating systems.

And it would be the default behavior, possibly breaking some build systems.

With my proposal, you just need to set

CARGO_TARGET_DIR_REMAP=/=$XDG_CACHE_DIR/cargo

and everything is put inside the xdg cache dir, with the full path of Cargo.toml appended to it (though i like the idea of having an hash of it). And you need to explicitly opt-in for this feature, so it won't break any existing system.

[sunshowers]

I ran into the same issue with btrfs and wrote a wrapper as a quick hack: https://github.com/sunshowers/targo

[epage]

@sunshowers looks like targo takes an approach similar to what @lorenz was proposing, always use a central target root.

Mind explaining your design and trade offs? I'm assuming its too early to get a feel for benefits and pitfalls.