WIP: Test lowest versions of all required and optional dependencies #3639
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,8 +1,8 @@ | ||
| # Test PyGMT with GMT legacy versions and old Python dependencies on Linux/macOS/Windows | ||
| # | ||
| # This workflow runs regular PyGMT tests with GMT legacy versions and old versions of | ||
| # all optional and required Python dependencies. Due to minor baseline image changes | ||
| # between GMT versions, the workflow runs the tests but doesn't do image comparisons. | ||
| # | ||
| # It is scheduled to run every Tuesday on the main branch. | ||
| # | ||
|
|
@@ -12,7 +12,7 @@ on: | |
| # push: | ||
| # branches: [ main ] | ||
| # Uncomment the 'pull_request' line below to trigger the workflow in PR | ||
| pull_request: | ||
| # types: [ready_for_review] | ||
| # paths: | ||
| # - 'pygmt/**' | ||
|
|
@@ -59,24 +59,9 @@ jobs: | |
| environment-name: pygmt | ||
| create-args: >- | ||
| python=3.11 | ||
| pyproj | ||
| gmt=${{ matrix.gmt_version }} | ||
| ghostscript<10 | ||
|
|
||
| # Download cached remote files (artifacts) from GitHub | ||
| - name: Download remote data from GitHub | ||
|
|
@@ -93,10 +78,21 @@ jobs: | |
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
|
|
||
| # Install uv package manager | ||
| - name: Install uv | ||
| uses: astral-sh/[email protected] | ||
| with: | ||
| activate-environment: true | ||
| python-version: 3.11 | ||
|
|
||
| # Install the package that we want to test | ||
| - name: Install the package | ||
| run: | | ||
| uv run --with pip==23 --resolution lowest-direct --all-extras --dev make install | ||
| uv pip list | ||
|
|
||
| # Run the tests but skip images | ||
| - name: Run tests | ||
| run: uv run --with pytest==8,pytest-mpl==0.17,pytest-doctestplus==1.2 --resolution lowest-direct --all-extras --dev make test_no_images PYTEST_EXTRA="-r P" | ||
| env: | ||
| GMT_LIBRARY_PATH: $CONDA_PREFIX/lib | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -46,9 +46,9 @@ dynamic = ["version"] | |
| all = [ | ||
| "contextily>=1.5", | ||
| "geopandas>=1.0", | ||
| "IPython>=8", # 'ipython' is not the correct module name. | ||
| "pyarrow>=16", | ||
|
There was a problem hiding this comment. Pinning to ________ test_to_numpy_pyarrow_array_pyarrow_dtypes_string[string_view] ________
> ???
E KeyError: 'string_view'
pyarrow/types.pxi:5025: KeyError
During handling of the above exception, another exception occurred:
dtype = 'string_view'
@pytest.mark.skipif(not _HAS_PYARROW, reason="pyarrow is not installed")
@pytest.mark.parametrize(
"dtype",
[
None,
"string",
"utf8", # alias for string
"large_string",
"large_utf8", # alias for large_string
"string_view",
],
)
def test_to_numpy_pyarrow_array_pyarrow_dtypes_string(dtype):
"""
Test the _to_numpy function with PyArrow arrays of PyArrow string types.
"""
> array = pa.array(["abc", "defg", "12345"], type=dtype)
../pygmt/tests/test_clib_to_numpy.py:333:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pyarrow/array.pxi:230: in pyarrow.lib.array
???
pyarrow/types.pxi:5040: in pyarrow.lib.ensure_type
???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E ValueError: No type alias for string_view
pyarrow/types.pxi:5027: ValueErrorCode was added in #3608: pygmt/pygmt/tests/test_clib_to_numpy.py Lines 326 to 336 in 97185e8 The There was a problem hiding this comment. PyArrow v16.0 was released in April, 2024. Instead of pinning pyarrow>=16.0, we can just skip There was a problem hiding this comment. Ok, I just tested older versions from pyarrow v12 to v15, and think we can pin to _________________________________________________________________ test_to_numpy_pyarrow_array_pyarrow_dtypes_date[date64[ms]] __________________________________________________________________
dtype = 'date64[ms]', expected_dtype = 'datetime64[ms]'
@pytest.mark.skipif(not _HAS_PYARROW, reason="pyarrow is not installed")
@pytest.mark.parametrize(
("dtype", "expected_dtype"),
[
pytest.param("date32[day]", "datetime64[D]", id="date32[day]"),
pytest.param("date64[ms]", "datetime64[ms]", id="date64[ms]"),
],
)
def test_to_numpy_pyarrow_array_pyarrow_dtypes_date(dtype, expected_dtype):
"""
Test the _to_numpy function with PyArrow arrays of PyArrow date types.
date32[day] and date64[ms] are stored as 32-bit and 64-bit integers, respectively,
representing the number of days and milliseconds since the UNIX epoch (1970-01-01).
Here we explicitly check the dtype and date unit of the result.
"""
data = [
date(2024, 1, 1),
datetime(2024, 1, 2),
datetime(2024, 1, 3),
]
array = pa.array(data, type=dtype)
result = _to_numpy(array)
_check_result(result, np.datetime64)
> assert result.dtype == expected_dtype # Explicitly check the date unit.
E AssertionError: assert dtype('<M8[D]') == 'datetime64[ms]'
E + where dtype('<M8[D]') = array(['2024-01-01', '2024-01-02', '2024-01-03'], dtype='datetime64[D]').dtype
../pygmt/tests/test_clib_to_numpy.py:364: AssertionErrorThe main change appears to be in apache/arrow#33321, when pyarrow supported preserving the datetime64 temporal resolution. Maybe we can keep the |
||
| "rioxarray>=0.14", | ||
|
There was a problem hiding this comment. Getting this test failure on windows-2022 at https://github.com/GenericMappingTools/pygmt/actions/runs/15524973929/job/43703740279?pr=3639#step:7:1095: _______________ [doctest] pygmt.datasets.tile_map.load_tile_map _______________
118 >>> raster.sizes
119 Frozen({'band': 3, 'y': 256, 'x': 512})
120 >>> raster.coords # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE
121 Coordinates:
122 * band (band) uint8... 1 2 3
123 * y (y) float64... -7.081e-10 -7.858e+04 ... -1.996e+07 -2.004e+07
124 * x (x) float64... -2.004e+07 -1.996e+07 ... 1.996e+07 2.004e+07
125 spatial_ref int... 0
126 >>> # CRS is set only if rioxarray is available
127 >>> if hasattr(raster, "rio"):
Expected:
'EPSG:3857'
Got:
'PROJCS["WGS 84 / Pseudo-Mercator",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]],PROJECTION["Mercator_1SP"],PARAMETER["central_meridian",0],PARAMETER["scale_factor",1],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1],AXIS["Easting",EAST],AXIS["Northing",NORTH],EXTENSION["PROJ4","+proj=merc +a=6378137 +b=6378137 +lat_ts=0 +lon_0=0 +x_0=0 +y_0=0 +k=1 +units=m +nadgrids=@null +wktext +no_defs"],AUTHORITY["EPSG","3857"]]'
D:\a\pygmt\pygmt\pygmt\datasets\tile_map.py:127: DocTestFailureSimilar situation to the one reported at #3575. We could perhaps pin to a rioxarray version that requires Suggestions on what to do? There was a problem hiding this comment. The actual error messages are Some related discussions are in corteva/rioxarray#447. I'm unsure if pin rioxarray>=0.19.0 can solve the issue, but please give it a try first. There was a problem hiding this comment. Hmm you're right, got the same There was a problem hiding this comment. The dependency tree GMT->GDAL->PROJ installs PROJ 9.4.0, while uv/pypi installs pyproj 3.7.1 which contains PROJ 9.5.1. Perhaps we can install pyproj via micromamba instead? There was a problem hiding this comment.
Tried at commit 4540a5d, same error 😢 There was a problem hiding this comment. What happens if setting env xref: https://pyproj4.github.io/pyproj/dev/installation.html#setup-pyproj There was a problem hiding this comment.
|
||
| ] | ||
|
|
||
| [project.entry-points."xarray.backends"] | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using
--resolution lowest-directfor now, because--resolution lowestgrabs many other transitive dependencies that may be hard to install (e.g. missing wheels for newer Python versions, so requires compilation from source). We could switch to--resolution lowestonce the Python ecosystem does lower bound pins a bit more thoroughly (which may take years).