Wrong environment selected when installed with pipx

[mdczaplicki]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: python:3.7.13-slim-buster
• Poetry version: 1.2.0
• Link of a Gist with the contents of your pyproject.toml file: n/a

Issue

When I've installed poetry using pipx in my Docker image and then set poetry not to create virtual environment - poetry installed all dependencies in venv created by pipx for poetry instead in system's python.

Any workaround for this?

[neersighted]

This is working as intended right now -- when you ask Poetry to not create a virtual environment, it is just going to install with the default Python it is aware of (aka sys.executable/the Python it is running under). This is more of a feature request for an explicit support for installation to the a system environment (which would be roughly what is suggested at #6398 (comment)).

[dimbleby]

You sure about that? it seems in tension with all those other issues complaining that poetry installations do collide with the system environment.

Experiment suggests that poetry installed by pipx behaves differently than poetry installed by the installer in this respect, which seems strange: poetry-via-the-installer installs to /usr/local/lib/python3.7/site-packages/ and not to /root/.local/share/pypoetry/venv/

[dimbleby]

Ah, there's a special case in the installer that other installation methods don't get to do

poetry/install-poetry.py

Lines 311 to 313 in 1242fac

	# We add a special file so that Poetry can detect
	# its own virtual environment
	target.joinpath("poetry_env").touch()

so that explains the difference

[neersighted]

Yeah, barring our special marker I am not sure there's much we can do about this.

[neersighted]

(And I think "don't create a virtual environment" is overloaded and instead "install to this Python interpreter's site-packages" is a better way to solve this)

[mdczaplicki]

So do you guys know a workaround to get it install in system's python? :P

What if I would set venvs path and prompt to point to system's directory? Would it cheat poetry to think that venv is already present and install in it?

[dimbleby]

For one: just use a virtual environment. Your life will be so much simpler, I promise.

For two: please read the thread! We already established that a poetry installed by the poetry installer has the behaviour that you think you want.

But for three: just use a virtual environment...

[mdczaplicki]

1. What simplicity comes from having a virtual environment inside virtual environment? We're talking about having python's venv inside docker container which is already an isolated venv
2. Yes, I've read, however I'm not in favor of using a script that can change anytime and install something that I don't want. I prefer to stick to pip/pipx installations
3. Docker container is already venv

[mdczaplicki]

I've managed to trick poetry into knowing that he's installed inside it's virtual environment.

Like poetry does it when it's installed by the script - I've created a file poetry_env inside it's venv.

touch /root/.local/pipx/venvs/poetry/poetry_env

alongside with:

poetry config virtualenvs.create false

it did the job and poetry installed dependencies system-wide.

[finswimmer]

Docker container is already venv

No, no, no. It's repeated a lot, but is not getting true ;)

Python's venv isolate dependencies from the global available python packages. As soon as you install two or more Python projects globally their dependencies influence each other. This is the case inside and outside docker. So if you need a reproducible python environment always use venv.

[mdczaplicki]

I disagree to this, but it's opinion-based, so let's not get into this.

I've raised a bug that there is inconsistency between native installer & pipx/pip installation.
I've also submitted a workaround to that. I'm good now.

[neersighted]

#6398 is relevant here again. It's really not a matter of opinion -- a container is not the same as a virtual environment and does not provide the same isolation as a virtual environment. While it does provide isolation from your container host, software inside the container can still run into classic Python packaging issues when packages from different sources/projects are mixed.

I'm not sure we'll be able to do anything about the pipx case as generally detecting and seeing through a virtual environment is fraught, but this issue is good for those who might have a similar question and want to work around it. I'm going to close this as currently intended, but this behavior could always be revisited in a rework of environment management.

[mdczaplicki]

What other sources/projects do you mean? If I expect to serve only my project from within the container - Is this case still worth creating Python's virtual environment inside docker container?

[neersighted]

Sources include your base image/Linux disto as well as Poetry itself -- frequently we see Poetry installed into the same environment (system or otherwise) as the project, and Poetry will frequently break itself as a result. Likewise, bad package versions from the distro can end up confusing Poetry as they are installed without normal Python tooling.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.