monorepo-hash

A CLI tool to generate hashes for the workspaces of your monorepo

📝 Features

🏃 Fast : Runs in huge monorepos in no time, processes workspaces in parallel
🎯 Accurate : Generates hashes based on every tracked file
↔️ Complete : Supports transitive workspace dependencies
👌 No config : Drop-in and instantly usable
💻 Cross-platform : Works on Windows, Linux and macOS
#️⃣ Deterministic : Same input, same output
📦 Lightweight : No bloat, just the essentials

🤔 Why

When you're working with monorepos, there's often a lot of workspaces (packages) that end up being created.
And as your project grows, so does the number of workspaces (and so does your build times...).
If you ever worked with stuff like Next.js, you know what I'm talking about. And since every workspace requires another, you need everything to be built to test your changes.

Although there are tools that allow your scripts to run only when files have changed (ex turbo), the complete CI step cannot benefit from this. For example with turbo again, they allow you to prune just the right workspaces and dependencies when building in a Docker, but this requires copying the entire monorepo into the container so we can't benefit from Docker's layers caching.
If only there could be a way to determine if a workspace hasn't changed to not rebuild it for nothing...

Well lucky you, monorepo-hash is here to help with that !

Note

monorepo-hash was created when I was doing my internship at Nexelec.
I really put a lot of energy in this script so I decided to release monorepo-hash as a standalone CLI tool to help anyone struggling with this problem !

🔰 Usage

Installation

You can install monorepo-hash globally, but it's best to add it as a dev dependency at the root of your monorepo :

pnpm add -D monorepo-hash

Tip

Make sure that the packages field in your pnpm-workspace.yaml file is set up correctly, as monorepo-hash will use it to find your workspaces. Globs are supported.
monorepo-hash will also use the workspace: field in your package.json files to detect transitive dependencies.
Finally, it will generate .hash files that you would need to keep in your VCS in order for it to be efficient (ex : to be reused in your CI).

Get help

pnpm monorepo-hash --help

Tip

Short versions of all arguments are also available.

Generate hashes for your entire monorepo

pnpm monorepo-hash --generate

Generate hashes for specific workspaces

Specify them in quotes, separated by commas, no spaces, and with no leading or trailing slashes.
The target name is the path to the workspace relative to the root of your monorepo, and uses forward slashes no matter your platform.

pnpm monorepo-hash --generate --target="packages/example,services/ui"

Compare hashes

pnpm monorepo-hash --compare

Compare hashes for specific workspaces

Same rules apply.

pnpm monorepo-hash --compare --target="packages/example"

Run in silent mode

This will suppress all output except for errors. This can be useful for example in CI where only the exit code matters.

pnpm monorepo-hash --compare --silent

Run in debug mode

The debug mode will :

in generate mode, output .debug-hash files which will contain the hashes of each individual file in the workspace as a JSON object
in compare mode, read those .debug-hash files and tell you exactly which files have changed in each workspace, and what their hashes are
This can be useful to check why the hashes appear to be different, or to debug issues with the hashes generation.

pnpm monorepo-hash --generate --debug
# later on...
pnpm monorepo-hash --compare --debug

Don't forget to delete these files afterwards !

Exit codes

0 : No changes detected (or you wanted to get help)
1 : Changes detected in the hashes
2 : Error with the arguments (either --generate or --compare is missing, or both were provided)
3 : Unknown arguments provided
4 : No workspaces found, either the pnpm-workspace.yaml file is missing or the packages field is not set up correctly
5 : An unexpected error occurred, please open an issue with the logs

🧪 Examples

Outputs

Tested in the small monorepo, with the following directory structure :

.
├── database
├── packages
│   ├── cli-tools
│   └── linter
├── services
│   ├── backend
│   └── frontend
└── pnpm-workspace.yaml

Hash generation

$ pnpm monorepo-hash --generate
ℹ️  Generating hashes for all workspaces...

✅ Computed all hashes (5)

✅ services\backend (f4cc294c3165f90990c03a4285796f98555b35bcf845981710a92ce66c7166e3) written to .hash
✅ packages\linter (c3f5ddaafb7382c35fa1b8045955c56a9a8b03754ed2cf3021acc335093d7e0d) written to .hash
✅ packages\cli-tools (b85b9d51a1536c94094fb652e7e577e36ac154072d3de8e42f3b3eb81e669054) written to .hash
✅ services\frontend (d5be1221077403acfd888a60ed5e11c66a1394d6ae044fe026ca199093b81f9b) written to .hash
✅ database (adb587b8758d1a9066a1a479ce4da12765b5cf128f5538312a017af233f85adb) written to .hash

Hash comparison - no changes

$ pnpm monorepo-hash --compare
ℹ️  Comparing hashes for all workspaces...

✅ Computed all hashes (5)

✅ Unchanged (5) :
• database
• packages\linter
• packages\cli-tools
• services\backend
• services\frontend

Hash comparison - changes detected

$ pnpm monorepo-hash --compare
ℹ️  Comparing hashes for all workspaces...

✅ Computed all hashes (5)

⚠️  Changed (5) :
• database
        old : adb587b8758d1a9066a1a479ce4da12765b5cf128f5538312a017af233f85adb
        new : c1ef17109f6a0b830d2c071934df47454b87389100cf573d918afca3f0958a6e
        🚧 changed dependency(s) :
                • packages\linter
• packages\linter
        old : c3f5ddaafb7382c35fa1b8045955c56a9a8b03754ed2cf3021acc335093d7e0d
        new : b12603b6d38b7bbf1c1ecea354e7e7bf5f7dc4ea42665e4935343d6f7c8b6ec9
• packages\cli-tools
        old : b85b9d51a1536c94094fb652e7e577e36ac154072d3de8e42f3b3eb81e669054
        new : f161ac1745e45054fa6123969f7cf08a818c1096b109d82b553d27551031d987
        🚧 changed dependency(s) :
                • packages\linter
• services\backend
        old : f4cc294c3165f90990c03a4285796f98555b35bcf845981710a92ce66c7166e3
        new : d27dc39124bef0de217d630ad891f79227780beda86893dca9ed5ff82fb06827
        🚧 changed dependency(s) :
                • packages\linter
                • database
                • packages\cli-tools
• services\frontend
        old : d5be1221077403acfd888a60ed5e11c66a1394d6ae044fe026ca199093b81f9b
        new : 36ffd97f62ab83a109f28549d3f23754e5e6d10a357f83ac76f63c6fec093efc
        🚧 changed dependency(s) :
                • packages\linter

Hash comparison - missing hashes

$ pnpm monorepo-hash --compare
ℹ️  Comparing hashes for all workspaces...

✅ Computed all hashes (5)

✅ Unchanged (4) :
• packages\linter
• packages\cli-tools
• services\backend
• services\frontend

❓ Missing .hash files (1) :
• database (would be f1d150816fef2890b2a0121f6267863a0f0efab59f5008f266d07a6045f60774)

Hash generation - specific workspaces

$ pnpm monorepo-hash --generate --target="packages/cli-tools,services/frontend"
ℹ️  Generating hashes for specified targets... (packages\cli-tools, services\frontend)

✅ Computed all hashes (3)

✅ packages\cli-tools (f161ac1745e45054fa6123969f7cf08a818c1096b109d82b553d27551031d987) written to .hash
✅ services\frontend (36ffd97f62ab83a109f28549d3f23754e5e6d10a357f83ac76f63c6fec093efc) written to .hash

Hash comparison - specific workspaces - no changes

$ pnpm monorepo-hash --compare --target="packages/cli-tools,services/frontend"
ℹ️  Comparing hashes for specified targets... (packages\cli-tools, services\frontend)

✅ Computed all hashes (3)

✅ Unchanged (2) :
• packages\cli-tools
• services\frontend

Hash comparison - specific workspaces - changes detected

$ pnpm monorepo-hash --compare --target="services/backend"
ℹ️  Comparing hashes for specified targets... (services\backend)

✅ Computed all hashes (4)

⚠️  Changed (1) :
• services\backend
        old : f4cc294c3165f90990c03a4285796f98555b35bcf845981710a92ce66c7166e3
        new : 8d61651bdb6f3219625bf910019ddfec0d32257f025f4c8f7f1018115287ae11
        🚧 changed dependency(s) :
                • packages\cli-tools

Usage in CI

This was the main reason I created this tool, and whether it's in GitHub Actions or locally through act, it can help you to reduce drastically CI times.
Here's an example workflow that uses monorepo-hash to only build the workspaces that have changed :

# The boring stuff

jobs:
  build-and-test:
    runs-on: ubuntu-22.04
    defaults:
      run:
        shell: bash
    env:
      IMAGE_TAG: "demo-${{ github.sha }}"
    strategy:
      fail-fast: false
      matrix:
        node-version: [22]

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Setup pnpm
        uses: pnpm/action-setup@v4

      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
          cache: "pnpm"

      - name: Install dependencies
        run: pnpm i --frozen-lockfile

      - name: Restore .hash cache
        uses: actions/cache@v4
        with:
          path: |
            **/.hash
          key: hash-files-${{ runner.os }}-pnpm-${{ hashFiles('**/pnpm-lock.yaml') }}
          restore-keys: |
            hash-files-${{ runner.os }}-pnpm-

      - name: Check if workspace-name is unchanged
        id: check-workspace-name
        run: |
          # These 2 lines are useful only if you use act, as a way to ensure the images are built if not present
          # WORKSPACENAME_DOCKER_EXISTS=$(docker images -q username/workspace-name:${{ env.IMAGE_TAG }} | wc -l)
          # echo "WORKSPACENAME_DOCKER_EXISTS=$WORKSPACENAME_DOCKER_EXISTS" >> ${GITHUB_OUTPUT}
          set +e
          pnpm monorepo-hash --compare --target="services/workspace-name"
          EXIT_CODE=$?
          echo "WORKSPACENAME_HASH_EXIT_CODE=$EXIT_CODE" >> ${GITHUB_OUTPUT}

      # Do this as much as needed for your workspaces

      - name: Build the workspace-name Docker image
        if: steps.check-workspace-name.outputs.WORKSPACENAME_HASH_EXIT_CODE != '0'
        # act version :
        # if: (steps.check-workspace-name.outputs.WORKSPACENAME_HASH_EXIT_CODE != '0' || steps.check-workspace-name.outputs.WORKSPACENAME_DOCKER_EXISTS == '0')
        uses: docker/build-push-action@v6
        with:
          context: .
          file: services/workspace-name/Dockerfile
          tags: username/workspace-name:${{ env.IMAGE_TAG }}
          load: true

      # Build things and test them

      - name: Save .hash cache
        uses: actions/cache@v4
        with:
          path: |
            **/.hash
          key: hash-files-${{ runner.os }}-pnpm-${{ hashFiles('**/pnpm-lock.yaml') }}
          restore-keys: |
            hash-files-${{ runner.os }}-pnpm-

Here we use the actions cache to store the .hash files, so that we can reuse them in the next runs.
This is especially useful because when you generate hashes, the action will pick them up from the latest commit and not the latest run.
For the very first run, you might need to create a workflow which will only checkout and save the .hash files in a cache for future runs.

🚧 Limitations

Only works with PNPM for now
If you really need support for Yarn or NPM, feel free to open an issue or even submit a pull request !
Bases the transitive dependency detection on the workspace: field in the package.json files
If you use another Version Control System than git, we can't ignore your files correctly for the hashes generation
Your EOL (End of Line) should be consistent across your monorepo's files and the different environments it's being used in. Since Docker containers and GitHub Actions runners are based on Linux, it's recommended to use LF as EOL.
I recommend to set this up in your IDE and formatter config.

🚀 Benchmarks

These benchmarks have been realised on a Windows 11 laptop with an AMD Ryzen 5 5500U CPU clocked at 2.10 GHz, 16 Gb of DDR3 RAM and an old SSD (needless to say, not a very performant machine).
They have been reproduced multiple time with a warm cache (node already run once) and with all applications closed.

Important

These benchmarks have been realised with the version 1.0.0.
Important performance improvements have been made since then, new benchmarks will be added soon.

Small monorepo, 5k LoC : 150 ms (5 workspaces of 100 files each, files composed of 1 line of text)
Medium monorepo, 505k LoC : 2.66 s (5 workspaces of 100 folders each, with each folder containing 100 files, files composed of 10 lines of text)
Large monorepo, 505m LoC : 47.7 s (5 workspaces of 100 folders each, with each folder containing 10 files and 10 folders, and each of these folders containing 100 files, files composed of 100 lines of text)

In order to not clunk up Git, these demo repos are compressed.

🛠️ Contributing

Here's a quick guide for contributing to monorepo-hash :

Fork the repository (and star it 😉)
Clone your fork

git clone https://github.com/USERNAME/monorepo-hash.git
cd monorepo-hash
pnpm i

Do your changes
Test your changes
Feel free to add tests to the tests directory.

pnpm run test

Commit your changes
Open a pull request

👀 Who uses `monorepo-hash` ?

Nexelec

If you use monorepo-hash in your project(s), whether you're an individual or a company, please let me know by opening an issue or a pull request, and I'll add you to this list !

💸 Donate

I'm a young developer from France, and as I write this I'm actively seeking for a job.
If you want to support me, here's how you can do it :

Star this repository
Follow me on GitHub
Donate :

📜 License

monorepo-hash is licensed under the MIT License

Name		Name	Last commit message	Last commit date
Latest commit History 47 Commits
.github		.github
images		images
src		src
tests		tests
.gitignore		.gitignore
.oxlintrc.json		.oxlintrc.json
LICENSE		LICENSE
README.md		README.md
eslint.config.ts		eslint.config.ts
eslint.stylistic.config.ts		eslint.stylistic.config.ts
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
tsconfig.json		tsconfig.json
tsdown.config.ts		tsdown.config.ts
vitest.config.ts		vitest.config.ts

Uh oh!

License

EDM115/monorepo-hash

Folders and files

Latest commit

History

Repository files navigation

monorepo-hash

📝 Features

🤔 Why

🔰 Usage

Installation

Get help

Generate hashes for your entire monorepo

Generate hashes for specific workspaces

Compare hashes

Compare hashes for specific workspaces

Run in silent mode

Run in debug mode

Exit codes

🧪 Examples

Outputs

Hash generation

Hash comparison - no changes

Hash comparison - changes detected

Hash comparison - missing hashes

Hash generation - specific workspaces

Hash comparison - specific workspaces - no changes

Hash comparison - specific workspaces - changes detected

Usage in CI

🚧 Limitations

🚀 Benchmarks

🛠️ Contributing

👀 Who uses monorepo-hash ?

💸 Donate

📜 License

About

Topics

Resources

License

Code of conduct

Uh oh!

Stars

Watchers

Forks

Releases 4

Sponsor this project

Uh oh!

Contributors 2

Uh oh!

Languages

👀 Who uses `monorepo-hash` ?