Skip to content

Adapt to the PEP 600 perennial manywheel standard #246

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mattip opened this issue Jun 8, 2020 · 3 comments
Closed

Adapt to the PEP 600 perennial manywheel standard #246

mattip opened this issue Jun 8, 2020 · 3 comments

Comments

@mattip
Copy link

mattip commented Jun 8, 2020

How best to support the PEP 600 manywheel_X_Y standard? The PEP allows creating a wheel by only specifying the glibc version X.Y.

  • Should there be some generic "support everything" approach? If everything, what does the policy.json file look like?
  • Or do we choose a target, and add only that glibc version and build a corresponding policy.json section? If this is the chosen approach, what targets should we add first? It seems from this list Ubuntu 16.04 with glibc 2.23 is the "next step" from the CentOS7 glibc 2.17 used in manylinux2014.
Motivation My motivation is to move beyond manylinux2014 for non-x86 architectures (arm64, powerpc64le, s390x) that have free CI available. NumPy replaces some of the buggy manylinux2014 glibc trigonometric functions, which were fixed in later glibc versions, but the replacement versions do not really support non-x86 architectures. Rather than fix the functions, I would rather get numpy packages for a more modern platform.
@takluyver
Copy link
Member

AIUI, for a given glibc. e.g. 2.23, we ideally want to consider all the distros which have >= 2.23, and find what versions of other common libraries are reliably available in all of them. So if we took glibc 2.23, we'd start by looking at libraries in Ubuntu 16.04 (glibc 2.23), Slackware 14.2 (glibc 2.23) and Debian 9 (glibc 2.24).

Obviously the more that can be automated, the more distros we can practically check.

Then, once the set of common library versions are known, we'll need to prepare a build environment, such as a Docker container. Ideally we'd stick to one distro for that. So far, that's been CentOS, but CentOS 8 (according to that list) jumps to 2.28, which is newer than Ubuntu 16.04 and 18.04. So we'd need to either make a big compatibility jump, or compile older libraries on CentOS for the build environment, or have a build environment based on a different distro.

@mayeut
Copy link
Member

mayeut commented Sep 27, 2020

I didn't had time to dig into this before. I had a little time to work on this during the week-end but too little to get very far ahead.
As @takluyver said, the more can be automated, the better.

In order to ease those checks, I created https://github.com/mayeut/pep600_compliance a while ago.
PR #262 shall help me get pep600_compliance to output an updated policy for any manylinux_x_y we want (well if it's possible at all as reported by the tool).

@mayeut
Copy link
Member

mayeut commented Dec 6, 2020

Release 3.3.0 adds policy for manylinux_2_24
If other policies are required, please open another issue as I'd rather not add all policies discovered by https://github.com/mayeut/pep600_compliance until #152 is dealt with (I'm updating pep600_compliance at a slow pace - as time allows - in order to tackle this issue).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants