Skip to content

[red-knot] Allow ellipsis default params in stub functions #17243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 7, 2025

Conversation

MatthewMckee4
Copy link
Contributor

Summary

Fixes #17234

Test Plan

Add tests to functions/paremeters.md

Copy link
Contributor

github-actions bot commented Apr 6, 2025

mypy_primer results

Changes were detected when running on open source projects
typeshed-stats (https://github.com/AlexWaygood/typeshed-stats)
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/typeshed-stats/scripts/runtests.py:17:5: Default value of type `EllipsisType` is not assignable to annotated parameter type `Literal[False]`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/typeshed-stats/scripts/runtests.py:26:5: Default value of type `EllipsisType` is not assignable to annotated parameter type `None`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/typeshed-stats/scripts/runtests.py:34:5: Default value of type `EllipsisType` is not assignable to annotated parameter type `Literal[False]`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/typeshed-stats/scripts/runtests.py:35:5: Default value of type `EllipsisType` is not assignable to annotated parameter type `None`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/typeshed-stats/scripts/runtests.py:42:5: Default value of type `EllipsisType` is not assignable to annotated parameter type `bool`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/typeshed-stats/scripts/runtests.py:43:5: Default value of type `EllipsisType` is not assignable to annotated parameter type `Path | None`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/typeshed-stats/scripts/runtests.py:44:5: Default value of type `EllipsisType` is not assignable to annotated parameter type `bool`
- Found 33 diagnostics
+ Found 26 diagnostics

werkzeug (https://github.com/pallets/werkzeug)
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/werkzeug/src/werkzeug/wrappers/request.py:387:9: Default value of type `ellipsis` is not assignable to annotated parameter type `Literal[True]`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/werkzeug/src/werkzeug/wrappers/request.py:568:15: Default value of type `ellipsis` is not assignable to annotated parameter type `bool`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/werkzeug/src/werkzeug/wrappers/request.py:568:34: Default value of type `ellipsis` is not assignable to annotated parameter type `Literal[False]`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/werkzeug/src/werkzeug/wrappers/request.py:568:66: Default value of type `ellipsis` is not assignable to annotated parameter type `bool`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/werkzeug/src/werkzeug/wrappers/request.py:573:15: Default value of type `ellipsis` is not assignable to annotated parameter type `bool`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/werkzeug/src/werkzeug/wrappers/request.py:573:34: Default value of type `ellipsis` is not assignable to annotated parameter type `bool`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/werkzeug/src/werkzeug/wrappers/request.py:573:54: Default value of type `ellipsis` is not assignable to annotated parameter type `bool`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/werkzeug/src/werkzeug/datastructures/accept.py:163:55: Default value of type `ellipsis` is not assignable to annotated parameter type `str`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/werkzeug/src/werkzeug/datastructures/accept.py:289:55: Default value of type `ellipsis` is not assignable to annotated parameter type `str`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/werkzeug/src/werkzeug/wrappers/response.py:596:24: Default value of type `ellipsis` is not assignable to annotated parameter type `bool`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/werkzeug/src/werkzeug/wrappers/response.py:596:43: Default value of type `ellipsis` is not assignable to annotated parameter type `Literal[False]`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/werkzeug/src/werkzeug/wrappers/response.py:599:24: Default value of type `ellipsis` is not assignable to annotated parameter type `bool`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/werkzeug/src/werkzeug/wrappers/response.py:599:43: Default value of type `ellipsis` is not assignable to annotated parameter type `bool`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/werkzeug/src/werkzeug/datastructures/headers.py:286:19: Default value of type `ellipsis` is not assignable to annotated parameter type `int | None`
- Found 697 diagnostics
+ Found 683 diagnostics

scrapy (https://github.com/scrapy/scrapy)
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/scrapy/scrapy/utils/spider.py:70:5: Default value of type `ellipsis` is not assignable to annotated parameter type `bool`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/scrapy/scrapy/utils/spider.py:71:5: Default value of type `ellipsis` is not assignable to annotated parameter type `bool`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/scrapy/scrapy/utils/spider.py:80:5: Default value of type `ellipsis` is not assignable to annotated parameter type `bool`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/scrapy/scrapy/utils/spider.py:81:5: Default value of type `ellipsis` is not assignable to annotated parameter type `bool`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/scrapy/scrapy/utils/spider.py:90:5: Default value of type `ellipsis` is not assignable to annotated parameter type `bool`
- error[lint:invalid-parameter-default] /tmp/mypy_primer/projects/scrapy/scrapy/utils/spider.py:91:5: Default value of type `ellipsis` is not assignable to annotated parameter type `bool`
- Found 1502 diagnostics
+ Found 1496 diagnostics

@AlexWaygood AlexWaygood added ty Multi-file analysis & type inference bug Something isn't working labels Apr 6, 2025
@MatthewMckee4 MatthewMckee4 marked this pull request as ready for review April 6, 2025 21:26
@MatthewMckee4 MatthewMckee4 changed the title Allow ellipsis default params in stub functions [red-knot] Allow ellipsis default params in stub functions Apr 6, 2025
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, that primer diff looks fantastic! And I confirmed that this gets rid of all invalid-parameter-default diagnostics when running red-knot on typeshed-stats

})
}

fn in_function_overload_or_abstract(&self) -> bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: maybe call this in_overloaded_or_abstract_function?

Suggested change
fn in_function_overload_or_abstract(&self) -> bool {
fn in_overloaded_or_abstract_function(&self) -> bool {

Copy link
Contributor

@carljm carljm Apr 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about this naming change, but I'm also not sure about the semantics of the function in the first place. I think being "in an overloaded function" suggests that it would include the main implementation (which is not decorated with @overload), but that shouldn't be included here. Have to go at the moment but would like to look at this a bit more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll test to see if this surprises an error of an ellipsis in the main overload

Copy link
Contributor Author

@MatthewMckee4 MatthewMckee4 Apr 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it doesn't, but i also see thats not really what you were referring to

Copy link
Contributor

@carljm carljm Apr 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok sorry for the cryptic note here, I was on a plane yesterday and the flight attendant was literally asking me to put away my computer as I was submitting that comment 😆

There were three things I wanted to look at here:

  1. Do we handle generic methods correctly (not just methods of generic classes, which you already added a test for)? The answer is yes, we do, but I'll push a couple tests for that as well.

  2. I wanted to clarify my understanding of how this same code works for both default values and for return statements inside the function, since default values are inferred in the outer scope, not the function body scope. But what I realized looking at it more closely is that function parameter definitions are handled when inferring the function body scope, so we are actually in the function body scope for all call sites of these functions. I think I might add doc comments to these methods clarifying their usage, but otherwise I think this looks good.

  3. The question of how to name this function. I think the name (and doc comment) should be clear that it returns true for overloads, not for overloaded functions, since the latter could be read to mean the main implementation function. Thus I wouldn't favor the name change suggested above, and I'm not coming up with anything I like significantly better than the current name.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3. The question of how to name this function. I think the name (and doc comment) should be clear that it returns true for overloads, not for overloaded functions, since the latter could be read to mean the main implementation function. Thus I wouldn't favor the name change suggested above, and I'm not coming up with anything I like significantly better than the current name.

I do understand what you're saying here... in_function_decorated_with_abstractmethod_or_overload? But it's obviously awfully verbose.

in_stublike_function?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with the very verbose name if you like it better, we won't have tons of call sites. I think we could also go with just in_function_overload_or_abstractmethod if you like that better? Tbh I don't know what alternatives to suggest since I'm not clear on precisely what it is you dislike about the current name :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like the current name because it doesn't make grammatical sense -- you can have a "function overload", sure, but you can't have a "function abstract" (it would be "abstract function")

in_overload_or_abstractmethod or similar would work fine for me!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I think the current name is perfectly grammatical, it just needs to be read with different grouping :) It's "in (function overload) or (abstractmethod)", not "in function (overload or abstractmethod)"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry, I think we were talking at cross purposes here? I didn't realise the name had already changed. The original name of this function in the first version of this PR was in_function_overload_or_abstract(), and that's the version of this PR that my original comment here was attached to. The name of the function in the merged version of the PR is in_function_overload_or_abstractmethod(), which, yes, I agree is perfectly grammatical and much better!

Copy link
Contributor

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

})
}

fn in_function_overload_or_abstract(&self) -> bool {
Copy link
Contributor

@carljm carljm Apr 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok sorry for the cryptic note here, I was on a plane yesterday and the flight attendant was literally asking me to put away my computer as I was submitting that comment 😆

There were three things I wanted to look at here:

  1. Do we handle generic methods correctly (not just methods of generic classes, which you already added a test for)? The answer is yes, we do, but I'll push a couple tests for that as well.

  2. I wanted to clarify my understanding of how this same code works for both default values and for return statements inside the function, since default values are inferred in the outer scope, not the function body scope. But what I realized looking at it more closely is that function parameter definitions are handled when inferring the function body scope, so we are actually in the function body scope for all call sites of these functions. I think I might add doc comments to these methods clarifying their usage, but otherwise I think this looks good.

  3. The question of how to name this function. I think the name (and doc comment) should be clear that it returns true for overloads, not for overloaded functions, since the latter could be read to mean the main implementation function. Thus I wouldn't favor the name change suggested above, and I'm not coming up with anything I like significantly better than the current name.

@carljm carljm enabled auto-merge (squash) April 7, 2025 17:33
@carljm carljm merged commit 4a4a376 into astral-sh:main Apr 7, 2025
20 checks passed
@MatthewMckee4 MatthewMckee4 deleted the ellipsis-in-stub-method branch April 7, 2025 18:04
dcreager added a commit that referenced this pull request Apr 7, 2025
* main: (42 commits)
  [playground] New default program (#17277)
  [red-knot] Add `--python-platform` CLI option (#17284)
  [red-knot] Allow ellipsis default params in stub functions (#17243)
  [red-knot] Fix stale syntax errors in playground (#17280)
  Update Rust crate clap to v4.5.35 (#17273)
  Fix RUF100 to detect unused file-level noqa directives with specific codes (#17042) (#17061)
  [ci] Fix pattern for code changes (#17275)
  [`airflow`] Update oudated `AIR301`, `AIR302` rules (#17123)
  [docs] fix formatting of "See Style Guide" link (#17272)
  [red-knot] Support stub packages (#17204)
  ruff_annotate_snippets: address unused code warnings
  [red-knot] Add a couple more tests for `*` imports (#17270)
  [red-knot] Add 'Format document' to playground (#17217)
  Update actions/setup-node action to v4.3.0 (#17259)
  Update actions/upload-artifact action to v4.6.2 (#17261)
  Update actions/download-artifact action to v4.2.1 (#17258)
  Update actions/setup-python action to v5.5.0 (#17260)
  Update actions/cache action to v4.2.3 (#17256)
  Update Swatinem/rust-cache action to v2.7.8 (#17255)
  Update actions/checkout action to v4.2.2 (#17257)
  ...
dcreager added a commit that referenced this pull request Apr 8, 2025
* main: (222 commits)
  [playground] New default program (#17277)
  [red-knot] Add `--python-platform` CLI option (#17284)
  [red-knot] Allow ellipsis default params in stub functions (#17243)
  [red-knot] Fix stale syntax errors in playground (#17280)
  Update Rust crate clap to v4.5.35 (#17273)
  Fix RUF100 to detect unused file-level noqa directives with specific codes (#17042) (#17061)
  [ci] Fix pattern for code changes (#17275)
  [`airflow`] Update oudated `AIR301`, `AIR302` rules (#17123)
  [docs] fix formatting of "See Style Guide" link (#17272)
  [red-knot] Support stub packages (#17204)
  ruff_annotate_snippets: address unused code warnings
  [red-knot] Add a couple more tests for `*` imports (#17270)
  [red-knot] Add 'Format document' to playground (#17217)
  Update actions/setup-node action to v4.3.0 (#17259)
  Update actions/upload-artifact action to v4.6.2 (#17261)
  Update actions/download-artifact action to v4.2.1 (#17258)
  Update actions/setup-python action to v5.5.0 (#17260)
  Update actions/cache action to v4.2.3 (#17256)
  Update Swatinem/rust-cache action to v2.7.8 (#17255)
  Update actions/checkout action to v4.2.2 (#17257)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working ty Multi-file analysis & type inference
Projects
None yet
3 participants