Description
This is intended to be somewhat compatible with other repositories wishing to do the same, but for now it is reasonable to hardcode assumptions that can be made for rust-lang/rust.
I believe these are reasonable steps to making this work:
Adjust compiletest to have a --bless-into $DIR flag which acts like --bless but writes new files at a different root (e.g., if editing src/test/ui/issue-3433.stderr, we'd instead save to $DIR/src/test/ui/issue-3433.stderr). It should not change the status reported by tests, merely write new files (i.e., it is safe to always pass this on CI).
We will want to do the same for rustfmt blessing; it is not entirely clear what the best way to do this is though. We wouldn't want to write all files to the new directory, just those that have changed when rustfmt is applied.
Once both of these are done, we will want to fetch this directory from CI somehow. I suspect we want this to be in the PR CI job so that everyone can use it (vs. waiting for bors to fail, which seems suboptimal). Eventually we'll want to add to the bors builders as well, so that all bless-able failures can be resolved via this bot.
This is somewhat hard -- we can't readily grant PR CI keys to write into an S3 bucket or anything like that. I suspect that a good way to do this might be GitHub Action's artifacts. AFAICT, we get a unique location guaranteed by GitHub via these artifacts so they should be safe to use.
Once all of that is done, we will already have a manual way to bless without building Rust: download the zip file from GHA, and "merge" in the new files locally, and push up a commit. This step can be automated via triagebot as well without too much trouble (though identifying the workflow ID is IIRC annoying).