Release process tweaks

* Verify release commit is in the main branch

Typically the tag is pushed first, CI runs, clearing the way for 'main' to be pushed. However we must not release until this is done.

* Improve CI status check

Execute logic within jq; look for any passing workflow and fail if there are none.

* Execute package.json check within jq

No need for Bash, jq can handle this.

* Perform package.json check first

It's the most important check, are we publishing what we think we're publishing.

* Update instructions

Author: novemberborn

.github/workflows/release.yml

@@ -31,30 +31,40 @@ jobs:
           ref: ${{ github.event_name == 'workflow_dispatch' && inputs.tag || github.ref }}
           fetch-depth: 0
 
-      - name: Check CI status
-        if: ${{ !inputs.skip_ci_check }}
+      - name: Verify tag matches package.json version
         run: |
-          # Check if CI has completed successfully for this commit
-          RESULT=$(gh run list --commit ${{ github.sha }} --status success --json conclusion,workflowName | jq '.[]|select(.workflowName == "Install and test AVA")')
-          if [ -z "$RESULT" ]; then
-            echo "CI has not completed successfully for this commit"
-            exit 1
-          fi
-          echo "All CI checks have passed!"
+          jq --raw-output --exit-status --arg tag "$RELEASE_TAG" '
+            if (.version == ($tag | ltrimstr("v"))) then
+              "Package version (\(.version)) matches tag version (\($tag | ltrimstr("v")))"
+            else
+              "Package version (\(.version)) does not match tag version (\($tag | ltrimstr("v")))" | halt_error(1)
+            end' package.json
         env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          RELEASE_TAG: ${{ github.event_name == 'workflow_dispatch' && inputs.tag || github.ref_name }}
 
-      - name: Verify tag matches package.json version
+      - name: Verify commit is in main branch
         run: |
-          PACKAGE_VERSION=$(jq -r '.version' package.json)
-          TAG_VERSION=${RELEASE_TAG#v}
-          if [ "$PACKAGE_VERSION" != "$TAG_VERSION" ]; then
-            echo "Package version ($PACKAGE_VERSION) does not match tag version ($TAG_VERSION)"
+          # Check if the tagged commit is included in the main branch
+          if git merge-base --is-ancestor ${{ github.sha }} origin/main; then
+            echo "Tagged commit is properly included in main branch"
+          else
+            echo "Tagged commit is not included in the main branch"
+            echo "Please push the commit to main before releasing"
             exit 1
           fi
-        env:
-          RELEASE_TAG: ${{ github.event_name == 'workflow_dispatch' && inputs.tag || github.ref_name }}
 
+      - name: Check CI status
+        if: ${{ !inputs.skip_ci_check }}
+        run: |
+          # Check if CI has completed successfully for this commit
+          gh run list --commit ${{ github.sha }} --status success --json workflowName | jq --raw-output --exit-status '
+            if any(.[]; .workflowName == "Install and test AVA") then
+              "All CI checks have passed!"
+            else
+              "CI has not completed successfully for this commit" | halt_error(1)
+            end'
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
       - name: Setup Node.js
         uses: actions/setup-node@v4

maintaining.md

@@ -6,51 +6,67 @@
 
 ## Testing
 
-* `npm test`: Lint the code and run the entire test suite with coverage.
-* `npx test-ava`: Run self-hosted tests from `test/`. Wraps a stable version of AVA.
-* `npx tap`: Run legacy tests from `test-tap/`.
-
-Note that in CI we only run linting with the Node.js version set in the `package.json` file under the `"volta"` key.
+- `npm test`: Lint the code and run the entire test suite with coverage.
+- `npx tap test-tap/fork.js --bail`: Run a specific test file and bail on the first failure (useful when hunting bugs).
+- `npx test-ava test/{file}.js`: Run self-hosted tests.
 
 ## CI
 
-We test across Linux, macOS and Windows, across all supported Node.js versions. The occasional failure in a specific environment is to be expected. If jobs fail, review carefully.
-
-TypeScript jobs should all pass.
+- Tests sometimes fail on Windows. Review the errors carefully.
+- At least one Windows job must pass.
+- All other jobs must pass.
 
 ## Updating dependencies
 
-* Make sure new dependency versions are compatible with our supported Node.js versions.
-* TypeScript dependency changes require CI changes to ensure backwards compatibility, see below.
-* Open a PR with the updates and only merge when CI passes (see the previous section).
+- Make sure new dependency versions are compatible with our supported Node.js versions.
+- Leave the TypeScript dependency as it is, to avoid accidental breakage.
+- Open a PR with the updates and only merge when CI passes (see the previous section).
 
 ## Updating TypeScript
 
 TypeScript itself does not follow SemVer. Consequently we may have to make changes to the type definition that, technically, are breaking changes for users with an older TypeScript version. That's OK, but we should be aware.
 
-When updating the TypeScript dependency, *also* add it to the CI workflow. This enables us to do typechecking with previous TypeScript versions and avoid unintentional breakage. For instance we won't accidentally rely on newer TypeScript features.
+Only update the TypeScript dependency when truly necessary. This helps avoid accidental breakage. For instance we won't accidentally rely on newer TypeScript features.
 
 Speaking of, using newer TypeScript features could be considered a breaking change. This needs to be assessed on a case-by-case basis.
 
 ## Pull requests
 
-* New features should come with tests and documentation.
-* Ensure the [contributing guidelines](.github/CONTRIBUTING.md) are followed.
-* Usually we squash commits when merging. Rebases may sometimes be appropriate.
+- New features should come with tests and documentation.
+- Ensure the [contributing guidelines](.github/CONTRIBUTING.md) are followed.
+- Squash commits when merging.
 
 ## Experiments
 
-* Implement breaking changes as an experiment first, requiring opt-in.
-* Ship new features early by treating them as an experiment, requiring opt-in.
+- Implement breaking changes as an experiment first, requiring opt-in.
+- Ship new features early by treating them as an experiment, requiring opt-in.
 
 ## Release process
 
-* Use `npm version` with the correct increment and push the resulting tag and `main` branch.
-* CI will run against the tag. Wait for this to complete.
-* Approve the Release workflow within GitHub. The workflow includes npm provenance for enhanced security and supply chain transparency.
+1. In the `main` branch, use `npm version` with the correct increment.
+1. Push the resulting tag (`git push --tags`).
+1. Wait for minimal CI checks to pass and push the `main` branch.
+1. Wait for full CI run to complete on the tag.
+1. The *Release* workflow will automatically run and publish to npm with provenance. It will also create a draft GitHub release.
+1. Review and publish the [draft GitHub release](https://github.com/avajs/ava/releases).
+
+The *Release* workflow includes several safety checks:
+
+- Validates the tag version matches `package.json`
+- Verifies the tagged commit is included in the `main` branch
+- Confirms CI has passed for the commit
+
+### Manual Release
+
+If CI fails for the tag and you're confident this is not due to a fault in the release, you can manually trigger the *Release* workflow:
+
+1. Go to the [*Release* workflow](https://github.com/avajs/ava/actions/workflows/release.yml)
+1. Click "Run workflow"
+1. Enter the release tag (e.g., `v1.2.3`)
+1. Optionally check "Skip CI status check"
 
 ### Setup Requirements
 
-For the automated workflows to work, the following secrets must be configured in the repository:
+For the Release workflow to work, the `NPM_TOKEN` must be configured in the [`npm` environment](https://github.com/avajs/ava/settings/environments/7070437878/edit#environment-secrets).
 
-- `NPM_TOKEN`: An npm automation token with publish permissions to the AVA package, within the `npm` environment
+This must be a granular automation token which can publish the AVA package. It must be set to expire after no more than 90 days.