Skip to content

[self-tests] test time degradation #13482

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
stanislavlevin opened this issue Jun 3, 2025 · 3 comments · May be fixed by #13513
Open

[self-tests] test time degradation #13482

stanislavlevin opened this issue Jun 3, 2025 · 3 comments · May be fixed by #13513
Labels
type: regression indicates a problem that was introduced in a release which was working previously type: selftests a problem in the tests of pytest

Comments

@stanislavlevin
Copy link

With Pytest 8.4.0 its tests suite takes significantly more time (compared to Pytest 8.3.5).

pytest's GHA:
8.3.x https://github.com/pytest-dev/pytest/actions/runs/15395204364/job/43314570031

platform linux -- Python 3.12.10, pytest-8.3.6.dev28+g1764fd049, pluggy-1.6.0
===== 3640 passed, 114 skipped, 11 xfailed, 1 xpassed in 335.83s (0:05:35) =====

8.4.x https://github.com/pytest-dev/pytest/actions/runs/15400542935/job/43331856904

platform linux -- Python 3.12.10, pytest-8.4.1.dev3+g7ddd7468e, pluggy-1.6.0
==== 3782 passed, 117 skipped, 11 xfailed, 1 xpassed in 1227.11s (0:20:27) =====

I can see this locally too.
@RonnyPfannschmidt RonnyPfannschmidt added type: regression indicates a problem that was introduced in a release which was working previously type: selftests a problem in the tests of pytest labels Jun 3, 2025
@bluetech
Copy link
Member

bluetech commented Jun 3, 2025

Bisected to 4b32e7c Enhancements to unraisable plugin (#12958). The major chunk of the extra time is gc_collect_harder but not all of it.

cc @graingert

@nicoddemus
Copy link
Member

nicoddemus commented Jun 14, 2025 •
edited
Loading

@graingert any chance you can take a look at this? The CI times are really high, with some jobs taking 30+ minutes...

Image

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Jun 14, 2025
@nicoddemus
pytester: patch GC_COLLECT_ITERATIONS to speed up the test suite
7728370 
Because `pytester.runpytest()` executes the full session cycle (including `pytest_unconfigure`), it was calling `gc.collect()` in a loop multiple times—even for small, fast tests. This significantly increased the total test suite runtime.

To optimize performance, pytester now patches `GC_COLLECT_ITERATIONS` to skip `gc.collect()` entirely, matching the behavior before pytest-dev#12958.

Locally the test suite runtime improved dramatically, dropping from 425s to 160s.

Fixes pytest-dev#13482.
@nicoddemus nicoddemus linked a pull request Jun 14, 2025 that will close this issue
Open
@nicoddemus
Copy link
Member

nicoddemus commented Jun 14, 2025 •
edited
Loading

Proposed a solution in #13513.

Image

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Jun 14, 2025
@nicoddemus
pytester: patch GC_COLLECT_ITERATIONS to speed up the test suite
bb542d4 
Because `pytester.runpytest()` executes the full session cycle (including `pytest_unconfigure`), it was calling `gc.collect()` in a loop multiple times—even for small, fast tests. This significantly increased the total test suite runtime.

To optimize performance, pytester now patches `GC_COLLECT_ITERATIONS` to skip `gc.collect()` entirely, matching the behavior before pytest-dev#12958.

Locally the test suite runtime improved dramatically, dropping from 425s to 160s.

Fixes pytest-dev#13482.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type: regression indicates a problem that was introduced in a release which was working previously type: selftests a problem in the tests of pytest
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

pytester: patch GC_COLLECT_ITERATIONS to speed up the test suite nicoddemus/pytest
4 participants
@RonnyPfannschmidt @nicoddemus @bluetech @stanislavlevin