☂️ Formatter: Support formatting of embedded code

[MichaReiser]

Support formatting Python code embedded in other languages like:

• Markdown
• reStructuredText
• HTML
• ...

The goal of this issue is not that we implement support for all these languages but to build up the infrastructure to run ruff (at least the formatter) on files that contain embedded python code and format it. Ideally, the infrastructure would, in the future, allow us to support arbitrary nesting:

• Format SQL in Python
• Format Markdown in Python
• ...

Prettier and JetBrains code formatter do an excellent job at this.

Related:

• Using ruff to format code examples in docstrings #7146
• Apply ruff to markdown code blocks #3792

[dhruvmanila]

The goal of this issue is not that we implement support for all these languages but to build up the infrastructure to run ruff (at least the formatter) on files that contain embedded python code and format it.

I think we should keep the linter in mind while designing the infrastructure as, and I'm not 100% sure but it's mainly my intuition from working on the Notebook support, it's more than likely than we get the formatter support for free. Free in the sense that it'll require considerably less effort on the formatter side once the infrastructure is in place.

[MichaReiser]

I agree, but I wanted to keep this issue scoped. The linter and formatter likely have similar requirements and need similar infrastructure, but with slight nuances.

[ddelange]

potential duplicate of #3792

[henryiii]

https://github.com/adamchainz/blacken-docs does this with black and markdown, ReST, and LaTeX. You can see the block types supported there. (python, pycon, etc). Even better, maybe the block types could be configurable, say setting md-blocks = ["python", "ipython", "{code-cell} ipython3"] would allow you to run on python, ipython, and executable Python (see https://jupyterbook.org/en/stable/file-types/myst-notebooks.html).