Add option to uv export to obtain version in workspace packages

[ivan94fi]

Summary

Description

We have a monorepo project, which is composed of several python packages. We use uv as project manager and we use a private pypi server where our wheels are published. When creating the docker image for our product we install the packages from the private pypi server, so we need exact versions, and we would like to avoid having to mount the sources, to be sure to have the published version in the final docker image.

Request

We would like to obtain package versions also for workspace members whe using uv export --frozen, so that later we can just install the generated requirements file and get packages from the pypi server.

Current situation:

Command:
uv export --frozen --no-hashes

Output:

-e ./packages/company.package1
-e ./packages/company.package2
-e ./packages/company.package3
-e ./packages/company.package4
accelerate==1.1.0
annotated-types==0.7.0
anyio==4.6.2.post1
certifi==2024.8.30
charset-normalizer==3.4.0
click==8.1.7
.
.
.

Example

Here we use an hypothetical --emit-workspace-with-versions flag, to emit the workspace members as "remote" dependencies.

Command:
uv export --frozen --emit-workspace-with-versions --no-hashes

Output:

company.package1=1.3.2
company.package2==0.3.2
company.package3==1.0.0
company.package4==3.2.2
accelerate==1.1.0
annotated-types==0.7.0
anyio==4.6.2.post1
certifi==2024.8.30
charset-normalizer==3.4.0
click==8.1.7
.
.
.

The versions in the workspace's members here are the ones taken from the pyproject.toml file of each member at the time the command is run.

[ivan94fi]

Any updates on this?

Thank you

[tpanum]

We have a similar use case, and I believe this sort of setup is very common among organizations.

I am curious why it seems like all workspace/monorepo examples completely ignore this use case?

Can you provide some insight on this, @konstin?

[zanieb]

I think this would be reasonable to support, though I'm not sure what it would look like. Perhaps --no-sources should do this already?

[tpanum]

I think this would be reasonable to support, though I'm not sure what it would look like. Perhaps --no-sources should do this already?

From my experience it doesn't, or if it does, I would need to know how to apply it correctly.

I have made a small minimalist example of the problem here. I have tried very combinations of uv build and uv export and I can't seem to get anything remotely useful.

[zanieb]

Yeah --no-sources does not today, that was a comment on changing that behavior instead of adding a new toggle. Sorry for the confusion.

[zanieb]

I don't quite follow why uv build is being used, could you talk about that a bit?

[tpanum]

I don't quite follow why uv build is being used, could you talk about that a bit?

Use case

I think a very common use case for many companies is to have a set of packages that can be interlinked. As an example, having some sort of core-package which is a dependency in many downstream internal packages. Each package (both core and downstream packages) are intended to be published to some internal package index.

Having multiple interconnected packages in a single repository works fantastic locally with uv (both workspace and non-workspace settings, thanks to uv.sources). However, when you have the scenario of wanting to publish your set of changed packages, uv (from my experience) ignores all version information about the packages. To reference my example:

We have a package named pkg_b which depends on pkg_a (in the same repository -- see my repository as an example). Now I locally change some code in pkg_a, and locally verify pkg_b is working (e.g. due to uv.sources creating directory links and having --editable).

Now I need to publish my changes, which mean the following (for simplicity assume we just bump the versions manually in respective pyproject.toml files):

• A new version of pkg_a needs to be published with the code changes we made.
• A new version of pkg_b needs to be published, that requires a new minimum version of pkg_a.

Having this sort of workflow easily accessible I think is a core feature, because hardly anyone needs to develop packages that (i) does not need sharing, or (ii) loves extremely loose coupling. I have tried various solutions, but I have yet to find some way of doing this with uv, yielding the workspace feature not very useful to me.

Your original question

The uv build serves as a pre-step of checking which sort of package (and version requirements) is being built and what version requirements are set in the wheel. I believe I came across some other thread that mentioned versions in the built packages follows PEP440 and thus only incorporate version information specified in the pyproject.toml.

I completely honor the desire to be PEP-compliant, but I believe it creates this paradoxical scenario where your local environment is tightly coupled, and your remote packages are (undesirable) loosely coupled. I think having some sort of feature to enable the ability to mimic the tight coupling of the local environment to remote packages is highly desirable.

What I have tried

Ideally I hope the feature and workflow I am describing can already be handled using uv. However, I have not been able to establish a feasible solution. I tried looking into uv export (and its various flags) but I couldn't get it to yield any version information for other packages in the same repository.

[tpanum]

I think this issue is similar or related to these:

• Add uv.workspace.dependencies to share dependency versions across workspace members #8949
• Build a wheel using pinned/resolved dependencies from uv.lock? #8729: @konstin provides an example in a comment of how to use uv export, but in my experience it does not yield any package version information in the generated requirements.txt (using 0.6.2)
• Building and publishing uv workspace members with pinned version constraints #9811

[konstin]

Thank you for the example repo!

We have a package named pkg_b which depends on pkg_a (in the same repository -- see my repository as an example). Now I locally change some code in pkg_a, and locally verify pkg_b is working (e.g. due to uv.sources creating directory links and having --editable).

Now I need to publish my changes, which mean the following (for simplicity assume we just bump the versions manually in respective pyproject.toml files):

* A new version of `pkg_a` needs to be published with the code changes we made.

* A new version of `pkg_b` needs to be published, that requires a new minimum version of `pkg_a`.

Having this sort of workflow easily accessible I think is a core feature, because hardly anyone needs to develop packages that (i) does not need sharing, or (ii) loves extremely loose coupling. I have tried various solutions, but I have yet to find some way of doing this with uv, yielding the workspace feature not very useful to me.

I'm trying to figure out the exact workflow that needs to be exposed. Would let's say we had this hypothetical workflow, would that do what you need?

uv build --package pkg_a 
uv publish --package pkg_a
uv upgrade --package pkg_b pkg_a --lower-bound # Upgrade the lower bound for pkg_a, but only in pkg_b's pyproject.toml
uv build --package pkg_b 
uv publish --package pkg_b

I'm trying to figure out whether uv export needs to be involved here, or if this is a case of the upgrade workflow that bumps lower bounds.

[ivan94fi]

Hi,

We internally wrote a script that does exactly that: bump all the workspace packages in the right order and publish to our pypi server.

However export is needed at deployment time, where we need to install from a published wheel, not from sources (in the dockerfile of our product we do not want to clone the repo, we just want to use the published package at the right version from our pypi server). So we need the exact dependencies of our internal packages pinned when exporting, as it already happens for external packages.

Right now I am build a requirements.txt with a script by merging uv export with the versions of our packages manually parsed from uv.lock file.

[tpanum]

Thank you for the example repo!
...
I'm trying to figure out the exact workflow that needs to be exposed. Would let's say we had this hypothetical workflow, would that do what you need?

uv build --package pkg_a 
uv publish --package pkg_a
uv upgrade --package pkg_b pkg_a --lower-bound # Upgrade the lower bound for pkg_a, but only in pkg_b's pyproject.toml
uv build --package pkg_b 
uv publish --package pkg_b

I'm trying to figure out whether uv export needs to be involved here, or if this is a case of the upgrade workflow that bumps lower bounds.

I think your series of commands are precisely what is desired, however I think the problem can be thought of more conceptually.

Workspaces

From my understanding workspaces in uv is inspired by Cargo. I am unfortunately unfamiliar with Cargo and how workspaces in Cargo work, so I am unsure about best practices and the precise design choices behind them. However, when talking uv workspaces, I think they are currently very limited by the fact that the coupling between packages in a typical development environment is very difficult to replicate in a package-based production environment. In essence, this is what I think this issue here tries to describe at its core.

I think workspaces are highly desirable in a setting where you have a set of interconnected packages where you want certainty and robustness about how they interact (which is also reflected by having a shared lockfile). As mentioned previously, you have high certainty about this in a development environment, but as soon you shift to distributing the packages (in current version of uv) you loose all kinds of certainty and you need to develop custom scripts to achieve some sort of certainty (as mentioned by @ivan94fi).

Given that workspaces are already a non-PEP initiative, I think it would be highly desirable if uv sought out to replicate this certainty at publish-time. So basically, the problem is:

• Given I want to publish all packages in my workspaces, how can I ensure that the certainty of package states in my development environment are identical to my package-based production environment?

To put it a bit boldly, I think workspaces are almost irrelevant without this feature, I think the amount of issues related to this topic also indicate that it is highly desirable by the community. I think there are multiple ways of solving this problem, and which one is the most ideal in a uv setting, I am unsure of.

Back to the question

Beyond the precise series of commands, I think it would be desirable if something along those lines could be achieved automatically. Basically:

• Detect which packages to update, and in which order
• Detect packages needs to update their dependencies