Skip to content

Stop including 'wheel', setuptools 70.1 has native bdist_wheel support #2868

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 28, 2025

Conversation

stefanor
Copy link
Contributor

@stefanor stefanor commented Apr 10, 2025

setuptools has native bdist_wheel support these days, so I don't think it's necessary to bundle the wheel library any more. Or am I missing something?

Possibly we could include a noop --no-wheel argument (but I'm not sure where that would be best defined).

Thanks for contributing, make sure you address all the checklists (for details on how see development documentation)

  • ran the linter to address style issues (tox -e fix)
  • wrote descriptive pull request text
  • ensured there are test(s) validating the fix
  • added news fragment in docs/changelog folder
  • updated/extended the documentation

@stefanor stefanor force-pushed the no-wheel branch 5 times, most recently from bce9ebb to d0cb03a Compare April 10, 2025 22:49
@stefanor
Copy link
Contributor Author

The two blockers are resolved. Note that this PR now bumps the bundled pip (except for Python 3.8, which I assume we'll remove shortly).

Copy link
Contributor

@gaborbernat gaborbernat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR still failing.

@gaborbernat gaborbernat enabled auto-merge April 28, 2025 13:39
@gaborbernat gaborbernat merged commit 3fa94b7 into pypa:main Apr 28, 2025
42 checks passed
@stefanor stefanor deleted the no-wheel branch April 28, 2025 16:34
stefanor added a commit to stefanor/cffi that referenced this pull request Apr 29, 2025
…these tests

Previously we relied on pip to build the packages in non-PEP517 mode,
which implied no build isolation.

The latest `virtualenv` (with pypa/virtualenv#2868) won't include
`wheel` in the virtualenv, which will mean that pip uses PEP-517 mode,
which is isolated by default.
@henryiii
Copy link

henryiii commented May 5, 2025

By the way, I'm now getting dozens of errors like this (I have warnings as errors turned on in tests):

DeprecationWarning: The --no-wheel option is deprecated. It has no effect for Python >= 3.8 as wheel is no longer bundled in virtualenv.

Ahh, I'm actually adding --no-wheel, that's why! Nevermind. Is there an ETA when this will be removed? Just trying to decide if I should conditionally remove it or ignore the error for now. I test the lower bound of dependencies.

Edit: nevermind for that too, checking the version looks pretty easy, so I'll just do that.

@gaborbernat
Copy link
Contributor

April next year.

@hroncok
Copy link
Contributor

hroncok commented May 5, 2025

May I suggest an alternative approach? What if it was never removed?

Keeping a dummy no-action option is not a maintenance burden.

Removing its usage from countless places on the other end seems not worth it.

henryiii added a commit to scikit-build/scikit-build-core that referenced this pull request May 6, 2025
See pypa/virtualenv#2868. It looks easy to check
the virtualenv version, so let's just do that.

Signed-off-by: Henry Schreiner <[email protected]>
@JuroOravec
Copy link

JuroOravec commented May 6, 2025

Hi, I'm writing here because this issue was linked to the change in changelog for "v20.31.0 (2025-05-05)".

I have a suspicion this broke the airspeed-velocity library, because there they used the wheel=bundle CLI argument. See airspeed-velocity/asv#1484

How to fix this? I skimmed through the changelog, but not 100% sure what to do to make the fix backwards compatible:

  • Should I replace wheel=bundle with VIRTUALENV_WHEEL=bundle?
  • Or should I remove wheel=bundle when on Python >=3.12?

@hroncok
Copy link
Contributor

hroncok commented May 6, 2025

I think you should remove wheel=bundle on Python 3.8+. What is your motivation to use it? The general idea is that it should not be needed with the updated setuptools and pip.

(However I relize that it's a distruption, perhaps it should warn but not fail?)

@allanlewis
Copy link

(However I relize that it's a distruption, perhaps it should warn but not fail?)

This is breaking our builds using Poetry 1.8.5, which I don't expect the Poetry team to update for this breaking change. We'll update to Poetry 2.x at some point but I'd rather not be forced to by this change.

@JuroOravec
Copy link

I managed to fix it by omitting the wheel=bundle for Py3.12+. But @hroncok after seeing your comment I will update it to use py3.8 as cutoff.

The problematic code is here: https://github.com/airspeed-velocity/asv/blob/01f7cfa3da86f6d98ef68984ecc663ac6ff7dfaa/asv/plugins/virtualenv.py#L136

And here is how I fixed it:

util.check_call([
    sys.executable,
    "-mvirtualenv",
    # Omit `--wheel=bundle` from py3.9+
    # See https://github.com/airspeed-velocity/asv/issues/1484
    *(["--wheel=bundle"] if sys.version_info < (3, 9) else []),
    "--setuptools=bundle",
    "-p",
    self._executable,
    self._path], env=env)

As for why that command is called with those arguments, I don't know. I'm maintainer at django-components and went to report / complain at airspeed-velocity, which brought me here 😄

@allanlewis
Copy link

I think this is breaking any Poetry project when at least one dependency needs a wheel to be built from source: see python-poetry/poetry#10378 (comment)
I expect the same to be the case with other package managers that use virtualenv under-the-hood.

@hroncok
Copy link
Contributor

hroncok commented May 6, 2025

I think a solution for the poetry problem (and similar) might be:

  • On Python 3.8, --wheel=bundle works as before.
  • On newer Pythons, --wheel=bundle does nothing and DeprecateWarns.

@PidgeyBE
Copy link

PidgeyBE commented May 6, 2025

It's indeed a breaking change for any project (poetry and others) having the --wheel argument still around in older versions.
I think removing the --wheel argument (even if it's a noop) justifies a major bump 😬

@hroncok
Copy link
Contributor

hroncok commented May 6, 2025

I think a solution for the poetry problem (and similar) might be: On Python 3.8, --wheel=bundle works as before. On newer Pythons, --wheel=bundle does nothing and DeprecateWarns.

I was looking into this, but fundamentally at:

if hasattr(options, "wheel"):
# Python 3.8
self.wheel_version = options.wheel
self.no_wheel = options.no_wheel
elif options.no_wheel:
warn(
"The --no-wheel option is deprecated. "
"It has no effect for Python >= 3.8 as wheel is no longer "
"bundled in virtualenv.",
DeprecationWarning,
stacklevel=1,
)

We would have all the options, and I am not entirely sure how to check interpreter.version_info from this method.

Gandalf-the-Grey pushed a commit to openhive-network/hive that referenced this pull request May 6, 2025
@gaborbernat
Copy link
Contributor

Yeah, we should probably make it a no op with a warning on newer Python interpreters. cc @stefanor

@hroncok
Copy link
Contributor

hroncok commented May 6, 2025

This is my draft PR (which doesn't work, as I am stuck). #2884

@stefanor
Copy link
Contributor Author

stefanor commented May 6, 2025

On newer Pythons, --wheel=bundle does nothing and DeprecateWarns.

This also risks upsetting users. --wheel=bundle is already an override that means "I know you don't want to install wheel any more, but I'm forcing you to". We're just interpreting this as "you wanted wheel so setuptools could turn source packages into wheels, we know you don't really need it".

But, I suppose the VIRTUALENV_WHEEL=bundle environment variable will be silently ignored, so we could do the same for the command line argument...

What got us into this pickle was wheel no longer being bundle-able, so we had to remove it immediately without a deprecation period.

@stefanor
Copy link
Contributor Author

stefanor commented May 6, 2025

I think removing the --wheel argument (even if it's a noop) justifies a major bump 😬

Note that there have been no major bumps since the rewrite. I guess because behavioural changes have been keyed to Python versions.

@hroncok
Copy link
Contributor

hroncok commented May 6, 2025

What got us into this pickle was wheel no longer being bundle-able, so we had to remove it immediately without a deprecation period.

Well, the other option was to keep the last known good version of wheel. (The new one was yanked BTW, so we might just as well do that for now.)

self.app_data = options.app_data
self.periodic_update = not options.no_periodic_update

if options.no_wheel:
warn(
Copy link
Contributor

@hroncok hroncok May 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stefanor I would also like to point out that as an user, I did not see this warning when I tested the option.

@stefanor
Copy link
Contributor Author

stefanor commented May 6, 2025

Well, the other option was to keep the last known good version of wheel

Not an option in the distribution contexts, where we don't include the bundled wheels, but rather wheels built from our own wheel source package. That's what brought me here.

@stefanor
Copy link
Contributor Author

stefanor commented May 6, 2025

Well, the other option was to keep the last known good version of wheel

Oh, also, the current setuptools is not compatible with the old wheel version from before the packaging de-vendor, IIRC. Even worse, it's all tied up in a single enormous refactoring commit :( (in wheel)

@black-snow
Copy link

black-snow commented May 7, 2025

What's the conclusion here? Can I just wait for my poetry 1.x pipelines to suddenly work again or do I need to take action?

/edit: nvm, fixed by upgrading to poetry 2 - removing (?) a CLI arg should've been a breaking change

@gaborbernat
Copy link
Contributor

https://github.com/pypa/virtualenv/releases/tag/20.31.2 thanks @hroncok for the fix 🤟

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants