Skip to content

chore: Update test dependencies in Python projects #18581

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 18 commits into
base: edge
Choose a base branch
from

Conversation

SyntaxColoring
Copy link
Contributor

@SyntaxColoring SyntaxColoring commented Jun 10, 2025

Overview

This does updates and light cleanup on most Python projects' test dependencies.

There's no particular urgency to this. Just in case this breaks something, I think we should hold off until v8.5.0 has been released and the dust has settled on mergebacks, and then merge this into edge.

Test Plan and Hands on Testing

Just make sure CI keeps passing on everything.

Changelog

Notable changes:

  • Upgrading Decoy to v2.2.0 resolves a lot of spurious warnings that we've been filtering out.
  • Upgrading Pytest to v8 has a lot of little things, including better tracebacks.
    • v8 caught some erroneous @pytest.marks that were applied to fixtures, which doesn't do anything.
    • v8 caught some async def tests that were not being run (!) because our asyncio plugin was not configured. They are running now.
    • system-server and robot-server are held back to Pytest v7 because Tavern still needs v7, unfortunately.
  • Upgrading Hypothesis has a lot of little things, including better defaults for CI, and additional cleverness for generating test data.
  • pytest-lazy-fixture is incompatible with Pytest v8 and is apparently just generally unmaintained, so I've replaced it with pytest-lazy-fixtures (with an s).

Review requests

None in particular.

Risk assessment

Low. Changes are limited to dev dependencies.

@SyntaxColoring SyntaxColoring added the DO NOT MERGE Indicates a PR should not be merged, even if there's a shiny green merge button available label Jun 10, 2025
pipenv upgrade --dev 'pytest>=8.4.0,<9'
Also:
- uninstall unused mock and types-mock deps.
- replace pytest-lazy-fixture with pytest-lazy-fixtures.
pipenv upgrade --dev 'decoy>=2.2.0,<3' 'pytest-asyncio>=0.23.0,<2' 'pytest>=8,<9' 'hypothesis>=6.96.1,<7'
Resolved in Decoy 2.2.0.
pytest-lazy-fixtures was erroring here because of something having to do with old_settings being scope=session, whereas all of the things it was built on (v1_config, v2_config, etc.) were scope=function.

I don't fully understand this, but moving everything to scope=function fixes it and seems strictly safer (more isolated), so let's just do that.
Marks have apparently never had any effect when applied to fixtures.
Also delete graphviz, which has not been used here since PR #14544.
pytest 7 was silently skipping these, and pytest 8 made them an error.
I don't know for how long they were being skipped.
Fortunately, they all still pass when we reenable them using asyncio_mode=auto, which follows the pytest-aiohttp docs.
This was added deliberately to match robot-server, but I'm not sure why? Neither server-utils nor any of its dependencies use it.
@SyntaxColoring SyntaxColoring force-pushed the python_test_dep_updates branch from beae8e3 to d5338b1 Compare June 10, 2025 19:13
@SyntaxColoring SyntaxColoring marked this pull request as ready for review June 10, 2025 19:17
@SyntaxColoring SyntaxColoring requested review from a team as code owners June 10, 2025 19:17
Copy link

codecov bot commented Jun 10, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 22.57%. Comparing base (60f6651) to head (d5338b1).
Report is 30 commits behind head on edge.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             edge   #18581      +/-   ##
==========================================
- Coverage   23.77%   22.57%   -1.21%     
==========================================
  Files        3233     3233              
  Lines      279281   279670     +389     
  Branches    27012    27025      +13     
==========================================
- Hits        66413    63144    -3269     
- Misses     212845   216504    +3659     
+ Partials       23       22       -1     
Flag Coverage Δ
app 0.33% <ø> (-2.00%) ⬇️
step-generation 4.38% <ø> (-0.06%) ⬇️
update-server 63.58% <ø> (+13.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 122 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DO NOT MERGE Indicates a PR should not be merged, even if there's a shiny green merge button available
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant