Replace ocamllex with sedlex for unicode support

[ufl-taeber]

Pyre fails to parse files with θ (and most likely other non-ASCII characters supported by CPython).

import math
from typing import Tuple


def cartesian(r: int, θ: int) -> Tuple[float, float]:
    rad = math.radians(θ)
    return r*math.cos(rad), r*math.sin(rad)


r = 12
θ = 195

x,y = cartesian(r, θ)
print(f"({r}, {θ}°) = ({x:.2f}, {y:.2f})")

$ python3 src/example.py
(12, 195°) = (-11.59, -3.11)

$ pyre --source-directory src check
Setting up a `.pyre_configuration` with `pyre init` may reduce overhead.
 ƛ Could not parse file at example.py:5:22-5:22
 ƛ Could not parse 1 file due to syntax errors!
 ƛ No type errors found

I'm using Python 3.7.7 installed via homebrew on macOS Catalina 10.15.4.

$ pyre --version
No watchman binary found. 
To enable pyre incremental, you can install watchman: https://facebook.github.io/watchman/docs/install.html
Defaulting to non-incremental check.
Binary version: 8a73fbdd6f7e74fa832ecbf5a9e2ebe20be46cd5
Client version: internal-dev

[mrkmndz]

The background here is that our lexer library is not unicode compatible :/. Therefore supporting this would require reimplementing our lexing into a new library, which would be a significant amount of work.

We do plan on doing this at some point, but it will likely be a while.

[EricLiclair]

Does this mean changing line 1 here ?

Or will it require much more than just this?

[aryx]

Actually you can hack ocamllex to recognize some unicode characters.
See https://github.com/returntocorp/pfff/pull/502/files for examples.