Skip to content

Disable assertion rewriting external modules #13421

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
231cb63
13403: Disable assertion rewriting for external modules
Tusenka May 11, 2025
bdc096c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 13, 2025
0701fcb
13403: Disable assertion rewriting for external modules - move root p…
Tusenka May 13, 2025
0d0551e
13403: Disable assertion rewriting for external modules
Tusenka May 11, 2025
3ec4da8
13403: Disable assertion rewriting for external modules - refactor
Tusenka May 15, 2025
f03f24e
13403: Disable assertion rewriting for external modules - refactor
Tusenka May 16, 2025
b6db388
13403: Disable assertion rewriting for external modules - refactor
Tusenka May 16, 2025
1346500
13403: Disable assertion rewriting for external modules - add tests
Tusenka May 18, 2025
023dfdb
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 16, 2025
1fee11d
13403: Disable assertion rewriting for external modules - add test fo…
Tusenka May 19, 2025
ba41963
13403: Disable assertion rewriting for external modules - add test fo…
Tusenka May 22, 2025
253497e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 22, 2025
67e7a69
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 16, 2025
ee836ad
13403: Disable assertion rewriting for external modules - add test fo…
Tusenka May 24, 2025
e7a98bb
13403: Disable assertion rewriting for external modules - eliminate o…
Tusenka May 25, 2025
c947f9a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 27, 2025
abd69ac
13403: Disable assertion rewriting for external modules - fix ruff
Tusenka Jun 1, 2025
fd59d87
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 1, 2025
3da821d
Update testing/test_assertrewrite.py
Tusenka Jun 1, 2025
48cfbc3
13403: Disable assertion rewriting for external modules - add tests
Tusenka Jun 3, 2025
eceae2e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 12, 2025
8c95bcd
13403: Disable assertion rewriting for external modules - add tests
Tusenka Jun 12, 2025
1d5a612
13403: Disable assertion rewriting for external modules - add tests
Tusenka Jun 12, 2025
c1f3772
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 13, 2025
edcf484
13403: Disable assertion rewriting for external modules - refactor As…
Tusenka Jun 13, 2025
5116841
13403: Disable assertion rewriting for external modules - refactor As…
Tusenka Jun 15, 2025
6a67526
13403: Disable assertion rewriting for external modules - refactor As…
Tusenka Jun 16, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog/13403.bugfix.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Disable assertion for modules outside current working dir(cwd)
1 change: 1 addition & 0 deletions src/_pytest/assertion/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ class AssertionState:
def __init__(self, config: Config, mode) -> None:
self.mode = mode
self.trace = config.trace.root.get("assertion")
self.invocation_path = str(config.invocation_params.dir)
self.hook: rewrite.AssertionRewritingHook | None = None


Expand Down
4 changes: 3 additions & 1 deletion src/_pytest/assertion/rewrite.py
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,9 @@ def _should_rewrite(self, name: str, fn: str, state: AssertionState) -> bool:
# rewritten if they match the naming convention for test files
fn_path = PurePath(fn)
for pat in self.fnpats:
if fnmatch_ex(pat, fn_path):
if fnmatch_ex(pat, fn_path) and fn_path.is_relative_to(
state.invocation_path
):
state.trace(f"matched test file {fn!r}")
return True

Expand Down
4 changes: 4 additions & 0 deletions src/_pytest/pytester.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@

from _pytest import timing
from _pytest._code import Source
from _pytest.assertion.rewrite import assertstate_key
from _pytest.capture import _get_multicapture
from _pytest.compat import NOTSET
from _pytest.compat import NotSetType
Expand Down Expand Up @@ -749,6 +750,9 @@ def chdir(self) -> None:
This is done automatically upon instantiation.
"""
self._monkeypatch.chdir(self.path)
self._monkeypatch.setattr(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do i gather correct that this bit is to alter the assertion rewtiter state

we should alter only that state, not the full pytest config of the surrounding pytest - this looks like a assertion rewriter shortcoming that needs a followup after this

but we certainly need to alter the assertion hook in sys metapath instead of a pytest config

Copy link
Author

@Tusenka Tusenka Jun 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pytester doesn't know anything about the AssertionState yet. In order to solve that I need to think in someway how to do this better

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should go with a temporary solution and add a followup issue

self._request.config.stash[assertstate_key], "invocation_path", self.path
)

def _makefile(
self,
Expand Down
2 changes: 2 additions & 0 deletions testing/example_scripts/rewrite/pytest.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[pytest]
python_files = *.py
11 changes: 11 additions & 0 deletions testing/example_scripts/rewrite/some_plugin/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
from __future__ import annotations

import pytest


@pytest.fixture
def special_asserter():
def special_assert(x, y):
assert {"x": x} == {"x": y}

return special_assert
Empty file.
9 changes: 9 additions & 0 deletions testing/example_scripts/rewrite/src/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
"""Stub file for testing"""

from __future__ import annotations


def func(x: int, y: int) -> int:
"""Stub function"""
assert (x) > 0
return 0 if x == y else 1 if x > y else -1
Empty file.
16 changes: 16 additions & 0 deletions testing/example_scripts/rewrite/tests/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
from __future__ import annotations

from _pytest.fixtures import fixture


pytest_plugins = ["pytester", "some_plugin"]


@fixture
def b():
return 1


@fixture
def a():
return 2
13 changes: 13 additions & 0 deletions testing/example_scripts/rewrite/tests/test_main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
from __future__ import annotations

from typing import Callable

from testing.example_scripts.rewrite.src.main import func


def test_plugin(a: int, b: int, special_asserter: Callable[[int, int], bool]):
special_asserter(a, b)


def test_func(a: int, b: int, special_asserter: Callable[[int, int], bool]):
assert {"res": func(a, b)} == {"res": 0}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
def some_check(a: int):
assert abs(a)<2
return a in set(0, 1, -1)
59 changes: 59 additions & 0 deletions testing/test_assertrewrite.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,14 @@
from _pytest.assertion.rewrite import _get_maxsize_for_saferepr
from _pytest.assertion.rewrite import _saferepr
from _pytest.assertion.rewrite import AssertionRewritingHook
from _pytest.assertion.rewrite import assertstate_key
from _pytest.assertion.rewrite import get_cache_dir
from _pytest.assertion.rewrite import PYC_TAIL
from _pytest.assertion.rewrite import PYTEST_TAG
from _pytest.assertion.rewrite import rewrite_asserts
from _pytest.config import Config
from _pytest.config import ExitCode
from _pytest.monkeypatch import MonkeyPatch
from _pytest.pathlib import make_numbered_dir
from _pytest.pytester import Pytester
import pytest
Expand Down Expand Up @@ -370,6 +372,7 @@ def test_rewrites_plugin_as_a_package(self, pytester: Pytester) -> None:
pytester.makeconftest('pytest_plugins = ["plugin"]')
pytester.makepyfile("def test(special_asserter): special_asserter(1, 2)\n")
result = pytester.runpytest()

result.stdout.fnmatch_lines(["*assert 1 == 2*"])

def test_honors_pep_235(self, pytester: Pytester, monkeypatch) -> None:
Expand Down Expand Up @@ -1294,6 +1297,36 @@ def test_meta_path():
)
assert pytester.runpytest().ret == 0

def test_invocation_dir(self, pytester: Pytester, monkeypatch: MonkeyPatch) -> None:
"""Test get invocation param from AssertionState"""
from _pytest.assertion import AssertionState

config = pytester.parseconfig()
state = AssertionState(config, "rewrite")

assert state.invocation_path == str(config.invocation_params.dir)

new_rootpath = pytester.path / "test"
if not os.path.exists(new_rootpath):
os.mkdir(new_rootpath)
monkeypatch.setattr(
config,
"invocation_params",
Config.InvocationParams(
args=(),
plugins=(),
dir=new_rootpath,
),
)
state = AssertionState(config, "rewrite")
assert state.invocation_path == str(new_rootpath)

@pytest.mark.skipif(
sys.platform.startswith("win32"), reason="cannot remove cwd on Windows"
)
@pytest.mark.skipif(
sys.platform.startswith("sunos5"), reason="cannot remove cwd on Solaris"
)
Comment on lines +1324 to +1329
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these skipif marks still necessary?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is sunos still a supported python platform

As far as I'm aware sun is down

def test_write_pyc(self, pytester: Pytester, tmp_path) -> None:
from _pytest.assertion import AssertionState
from _pytest.assertion.rewrite import _write_pyc
Expand Down Expand Up @@ -1971,6 +2004,32 @@ def test_simple_failure():
assert hook.find_spec("file") is not None
assert self.find_spec_calls == ["file"]

def test_assert_rewrites_only_invocation_path(
self, pytester: Pytester, hook: AssertionRewritingHook, monkeypatch
) -> None:
"""Do not rewrite assertions in tests outside `AssertState.rootpath` (#13403)."""
pytester.makepyfile(
Copy link
Member

@nicoddemus nicoddemus Jun 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to avoid mocking here, instead using a real scenario to ensure Python files outside of the rootpath do not have their assertions rewritten.

Please create a simple but real-world project using pytester, something like:

myproject/
   /venv
     /lib
        /external_lib.py
     pyvenv.cfg  

   /src
      main.py
   /tests
      conftest.py
      test_main.py
   pytest.ini
outside.py

pytest.ini should have these contents:

[pytest]
python_files = *.py

Then execute pytest.runpytest(), which will execute from the root of the above tree.

If I understand the objective of the issue correctly:

  • myproject/venv/lib/external_lib.py: should not have assertions rewritten -- the file is inside a virtual environment and is not part of a pytest plugin. The pyvenv.cfg file is how pytest detects virtual environments.
  • myproject/src/main.py: rewritten -- inside rootpath and matches python_files.
  • myproject/tests/conftest.py: rewritten -- conftest.py files are always rewritten.
  • myproject/tests/test_main.py: rewritten -- test files are always rewritten.
  • outside.py: should not have assertions rewritten -- outside the rootpath.

I think the scenario above should cover what needs to be tested.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to include an external plugin here too, to test the same scenario as in test_assert_rewrite_correct_for_plugins.

Copy link
Author

@Tusenka Tusenka Jun 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried to create the structure along with a some conftest plugin. Even witout changes it doen't rewrite conftest plugins assertions
image

Example:
https://github.com/Tusenka/pytest/tree/disable_assertion_rewriting_external_modules_draft/testing/example_scripts/rewrite
image

However it sees only top-level plugins:
['anyio', '_hypothesis_ftz_detector', '_hypothesis_globals', '_hypothesis_pytestplugin', 'hypothesis', 'pytest_twisted', 'typeguard']
Logs for rewriting

Does it an expected behavior? How does pytest provide plugins which are needed to be rewritten at the import stage? May be I do something wrong

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should rewrite the conftest files

If it doesn't then we have a bug we missed that needs a additional regression test

**{
"file.py": """\
def test_simple_failure():
assert 1 + 1 == 3
"""
}
)

with mock.patch.object(hook, "fnpats", ["*.py"]):
assert hook.find_spec("file") is not None

invocation_path = f"{os.getcwd()}/tests"
monkeypatch.setattr(
pytester._request.config.stash[assertstate_key],
"invocation_path",
invocation_path,
)

with mock.patch.object(hook, "fnpats", ["*.py"]):
assert hook.find_spec("file") is None

@pytest.mark.skipif(
sys.platform.startswith("win32"), reason="cannot remove cwd on Windows"
)
Expand Down
Loading