Package compatibility guarantees

[wlandau]

Current approach

The R-multiverse Staging process is a Debian-like approach to ensuring package stability. We freeze base R and CRAN dependencies for a month, then we freeze healthy packages for another month, and then we take the Production snapshot.

Potential issues

Although promising, this approach alone does not absolutely guarantee that packages are compatible with the exact versions of their dependencies in Staging. Checks may use outdated versions of dependencies from the same universe. In particular, if a dependency updates, then the checks on the reverse dependency do not currently rerun.

Action

Our current model may turn out to work well in practice, and it may be reasonable to wait and monitor several snapshot cycles before taking further action. But if we find incompatible package versions in our Production snapshots, I would like to place hard CRAN-like guarantees on package compatibility.

(NB: to guide intuition, https://r-multiverse.org/status/staging.html now reports when a package was last rebuilt/rechecked/published in the Staging universe. Likewise for https://r-multiverse.org/status/community.html and https://r-multiverse.org/status/production.html.)

Idea

We might not need it, but I think I have an idea for implementing these CRAN-like guarantees. R-universe could recheck packages when their dependencies update, and R-multiverse could flag checks that used outdated dependencies.

For R-universe

1. Timestamp of the latest update: in the API, post a timestamp to mark when a package last published a new RemoteSHA/version. This is just like the current _published field, except that it only updates when the RemoteSHA updates.
2. Timestamp of the latest check start: in the API, post another timestamp to mark when the latest round of checks started.
3. Rerun package checks when a dependency in the same universe updates: Rerun a package's checks whenever a strong dependency updates r-universe-org/help#369. This would include strong (Depends:, Imports:, or LinkingTo:) dependencies anywhere upstream, not just the ones directly mentioned in the DESCRIPTION of the given package to recheck.

For R-multiverse

We should not trust a check that started before the package's dependencies were last updated. To enforce this skepticism, we would compare the timestamp of the last check start (2) of the current package to the maximum timestamp of the latest update (1) of the dependencies. If the starting timestamp is greater, we trust the check. Otherwise, we flag the check as outdated.

issues_dependencies() already creates a DAG of strong dependencies, so this computation would already be straightforward in multiverse.internals.

[llrs]

I like this idea and I think it is one of the strong quality features of CRAN. Note that when a package fails it might need to be "updated" to note up to which packages (or R version) it is compatible. Even with the release cycle I'm not sure this is ensured, from exploring the Bioconductor checks, I think there is no guarantee they will work well with updates on repositories.