Skip to content

session.conda_install fails when specifying min/max version with newer conda #951

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jefsayshi opened this issue Apr 17, 2025 · 6 comments · Fixed by #957
Open

session.conda_install fails when specifying min/max version with newer conda #951

jefsayshi opened this issue Apr 17, 2025 · 6 comments · Fixed by #957
Labels

Comments

@jefsayshi
Copy link

jefsayshi commented Apr 17, 2025

Current Behavior

nox > Running session flake8
nox > Creating conda env in .nox/flake8 with python=3.9
nox > conda install --yes --prefix /home/jeff/workspace/nox_test/.nox/flake8 '"flake8>=4.0.1"'
nox > Command conda install --yes --prefix /home/jeff/workspace/nox_test/.nox/flake8 '"flake8>=4.0.1"' failed with exit code 1:

InvalidMatchSpec: Invalid spec '"flake8>=4.0.1"': Invalid version '4.0.1"': invalid character(s)

nox > Session flake8 failed.

Expected Behavior

I expect the package to be installed when specifying a min or max version

Steps To Reproduce

noxfile.py:

import nox

@nox.session(python='3.9', venv_backend='conda')
def flake8(session):
    session.conda_install('flake8>=4.0.1')
    session.install('--no-deps', '-e', '.')
    session.run('flake8')
  1. Install a new miniconda version (https://repo.anaconda.com/miniconda/Miniconda3-py312_25.1.1-2-Linux-x86_64.sh)
  2. conda create -n nox python=3.11 nox
  3. conda activate nox -y
  4. nox

Environment

- OS: Ubuntu 24.04.2
- Python: 3.12.9
- Nox: 2025.2.9
- Conda: 25.1.1

Anything else?

Temporary workaround:
nox.sessions._dblquote_pkg_install_args = lambda args: args

You appear to routing this through subprocess.Popen which states:

Unlike some other popen functions, this library will not implicitly choose to call a system shell. This means that all characters, including shell metacharacters, can safely be passed to child processes.

I believe that this quoting can be removed completely: https://github.com/wntrblm/nox/blob/main/nox/sessions.py#L726-L727

Fails with:

  • conda 25.1.1
  • conda 24.11.1

Works with:

  • conda 24.9.2
@jefsayshi jefsayshi added the bug label Apr 17, 2025
@henryiii
Copy link
Collaborator

This is needed on Windows, but not on other systems (shell=True requires it, and you pretty much have to pass that on Windows).

@jefsayshi
Copy link
Author

@henryiii Even if my resolution is not correct the issue still exists and should be reopened

However, I am not seeing where shell=True is added for windows: https://github.com/wntrblm/nox/blob/main/nox/popen.py#L98

Also the shell escaping issue you describe would exist no matter what executable is called, so why is pip treated different than conda?

@henryiii henryiii reopened this May 1, 2025
@jefsayshi
Copy link
Author

I just saw that you did patch the issue for platforms other than windows. Thanks!

@henryiii
Copy link
Collaborator

henryiii commented May 1, 2025

#957 fixed this on UNIX, and left the behavior alone on Windows (yes, so this should remain open till fixed on Windows). That's out in 2025.05.01, which was just released.

I don't understand why it's happening, I don't see shell=True anywhere (we had to use that cibuildwheel due to bugs when it's False), but it clearly fails in CI on Windows if I do requests<99 (Using < is better than > since it claims the file 99 doesn't exist instead of writing out to file 99). With the quoting, then conda gets the quotes. Mamba also doesn't like them.

Better investigation will require me actually to be on Windows, which I'm not going to do for a while. I'd really like to at least verify that subprocess.run(["pip", "install", "requests<99"]) does work on Windows (outside of nox), it seems like it should according to the docs, but I don't see how we are doing anything different with conda.

@jefsayshi
Copy link
Author

jefsayshi commented May 2, 2025

Okay, I dug into the windows side and there is a difference depending on whether the batch file or executable is being invoked. The batch file + Popen does not appear to properly pass on the quoting/escaping.

Here is what I tried:

import subprocess
from pathlib import Path

conda_bat = Path.cwd() / 'miniconda3' / 'condabin' / 'conda.bat'
conda_exe = Path.cwd() / 'miniconda3' / 'Scripts' / 'conda.exe'

# Returns "The system cannot find the file specified"
subprocess.Popen([conda_bat, 'install', '--dry-run', 'requests<99'])
subprocess.Popen([conda_bat, 'install', '--dry-run', 'requests^<99'])

# Returns "InvalidMatchSpec"
subprocess.Popen([conda_bat, 'install', '--dry-run', '"requests<99"'])

# Works
subprocess.Popen([conda_exe, 'install', '--dry-run', 'requests<99'])

Edit: if you want to use the bat file this escaping seems to work

subprocess.Popen([conda_bat, 'install', '--dry-run', 'requests^^^<99'])

Edit2: This seems really hacky but seems to force Popen to properly quote them and works for both the batch file and executable

subprocess.Popen([conda_bat, 'install', '--dry-run', 'requests <99'])
subprocess.Popen([conda_exe, 'install', '--dry-run', 'requests <99'])

@jefsayshi
Copy link
Author

I don't know if it will go anywhere, but I filed this behavior as a bug in cpython python/cpython#133545

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging a pull request may close this issue.

2 participants