Ingore notebook syntax

[azachar]

Hello,
I am trying to apply changes also on some notebooks from databricks stored as .py files, from time to time, there is %pip install or %magic command, could you please tell me how to ignore it easily, do you support such a case?

Thank you!
Best regards,
Andrej

[thatch]

The best way to do this would be alter the grammar. Some untested hints, I think you'll want to treat these like comments:

1. https://github.com/jreese/fissix/blob/main/fissix/pgen2/tokenize.py#L69 add a new regex similar to Comment that is for Directive
2. Add Directive to PsuedoExtras https://github.com/jreese/fissix/blob/main/fissix/pgen2/tokenize.py#L130
3. Generate some kind of token (calling them COMMENT saves you some effort) https://github.com/jreese/fissix/blob/main/fissix/pgen2/tokenize.py#L537

Note these are re.compile with default args, so if you want to preserve their special behavior only at the beginning of the line, that's more complicated.