You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Simplify location tracking in the lexer.
This commit changes the location tracking in lexer.rs to be almost
completely implicit, except for line tracking. The offset can be
computed from the start of the &str and the current position of the
innermost iterator. The final location information is only created in
Lexer::next, at the start and the end of the currently build token.
To simplify the control flow for each char consumed, inner iterators
are changed to not use Peekable but instead do copies of inner iterators
as needed.
0 commit comments