Formatter can delete comments between f-string expr and format spec

[MeGaGiGaGon]

Summary

As the title says, any comments between the f-string expr and format spec can get deleted by the formatter if it gets collapsed to one line and there is a # in the format spec playground

f"{1
# test
:#}"

Formats to

f"{1:#}"

Version

playground

[LaBatata101]

@MichaReiser how should the formatted output look like?

I was prototyping a fix for this issue and got this output:

f"{
    1:# comment
    #
}"

[MeGaGiGaGon]

@LaBatata101 Unfortunately that's invalid code on 3.13.5/3.14, see #18632
It also changes the meaning of the code since in format specs #s are not comments (running on 3.12 to avoid the error)

>>> f"{1:  # comment? nope
... }"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Invalid format specifier '  # comment? nope' for object of type 'int'

[MichaReiser]

@MichaReiser how should the formatted output look like?

I don't know. There's no one-fits-all solution to this. It requires finding heurisitcs that keep the comments close to where they were, keep them in source order and, ideally, preseves if the comment was an own-line or end-of-line comment.

I think in this case, a reasonable formatting would be

f"{1:#}"  # test

But we need to be careful to only apply it if there's no debug expression

[dhruvmanila]

After playing around with the original code, I found that if the comment is an end-of-line comment for the expression, and if there's a format spec present, the comment gets moved after the format spec which makes it not a comment anymore.

f"{1 # comment
:.2f
}"

The above gets formatted to:

f"{
    1:.2f  # comment
}"

The core issue might be the same although I'm not sure.

[MichaReiser]

I think I need to fix this to fix the format spec issue because we drop the comment in

x = f"{x  !s
         :>{0
         # comment 21
         }}"