[BUG - Fixed] Nan values in the raster giving issue while viewing

[shubham-scisar]

I am trying to view raster having nan values using cog/viewer , when trying to visualize,I get the following error :

titiler_1  |   File "/usr/local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 375, in run_asgi
titiler_1  |     result = await app(self.scope, self.receive, self.send)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
titiler_1  |     return await self.app(scope, receive, send)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/fastapi/applications.py", line 199, in __call__
titiler_1  |     await super().__call__(scope, receive, send)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/starlette/applications.py", line 112, in __call__
titiler_1  |     await self.middleware_stack(scope, receive, send)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
titiler_1  |     raise exc from None
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
titiler_1  |     await self.app(scope, receive, _send)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/starlette/middleware/base.py", line 25, in __call__
titiler_1  |     response = await self.dispatch_func(request, self.call_next)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/titiler/application/middleware.py", line 38, in dispatch
titiler_1  |     response = await call_next(request)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/starlette/middleware/base.py", line 45, in call_next
titiler_1  |     task.result()
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/starlette/middleware/base.py", line 38, in coro
titiler_1  |     await self.app(scope, receive, send)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/brotli_asgi/__init__.py", line 72, in __call__
titiler_1  |     await responder(scope, receive, send)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/brotli_asgi/__init__.py", line 111, in __call__
titiler_1  |     await self.app(scope, receive, self.send_with_brotli)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/starlette/middleware/cors.py", line 78, in __call__
titiler_1  |     await self.app(scope, receive, send)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
titiler_1  |     raise exc from None
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
titiler_1  |     await self.app(scope, receive, sender)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 580, in __call__
titiler_1  |     await route.handle(scope, receive, send)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 241, in handle
titiler_1  |     await self.app(scope, receive, send)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 52, in app
titiler_1  |     response = await func(request)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 216, in app
titiler_1  |     raw_response = await run_endpoint_function(
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 151, in run_endpoint_function
titiler_1  |     return await run_in_threadpool(dependant.call, **values)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/starlette/concurrency.py", line 40, in run_in_threadpool
titiler_1  |     return await loop.run_in_executor(None, func, *args)
titiler_1  |   File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
titiler_1  |     result = self.fn(*self.args, **self.kwargs)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/titiler/core/factory.py", line 271, in metadata
titiler_1  |     return src_dst.metadata(
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/rio_tiler/io/base.py", line 134, in metadata
titiler_1  |     stats = self.stats(pmin, pmax, **kwargs)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/rio_tiler/io/cogeo.py", line 248, in stats
titiler_1  |     stats = reader.stats(
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/rio_tiler/reader.py", line 405, in stats
titiler_1  |     return {
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/rio_tiler/reader.py", line 406, in <dictcomp>
titiler_1  |     f"{indexes[b]}": raster_stats(data[b], percentiles=percentiles, **hist_options)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/rio_tiler/utils.py", line 78, in _stats
titiler_1  |     sample, edges = numpy.histogram(arr[~arr.mask], **kwargs)
titiler_1  |   File "<__array_function__ internals>", line 5, in histogram
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/numpy/lib/histograms.py", line 793, in histogram
titiler_1  |     bin_edges, uniform_bins = _get_bin_edges(a, bins, range, weights)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/numpy/lib/histograms.py", line 426, in _get_bin_edges
titiler_1  |     first_edge, last_edge = _get_outer_edges(a, range)
titiler_1  |   File "/usr/local/lib/python3.8/site-packages/numpy/lib/histograms.py", line 323, in _get_outer_edges
titiler_1  |     raise ValueError(
titiler_1  | ValueError: autodetected range of [nan, nan] is not finite

Any help on this ?

[vincentsarago]

@shubhamsharma1609 do your raster has internal nodata value?

To be honest this is more a rio-tiler issue in https://github.com/cogeotiff/rio-tiler/blob/master/rio_tiler/utils.py#L51-L88

could you open an issue in rio-tiler saying that we shouldn't calculate statistics of Nan values 🙏

[shubham-scisar]

@vincentsarago
Output :

bounds=BoundingBox(left=66.75882949703501, bottom=6.6509180118181455, right=98.64961531807052, top=37.16081813018124) center=(82.70422240755276, 21.905868070999695, 3) minzoom=3 maxzoom=9 band_metadata=[('1', {})] band_descriptions=[('1', '')] dtype='float32' nodata_type='Nodata' colorinterp=['gray'] scale=None offset=None colormap=None statistics={'1': ImageStatistics(percentiles=[-46.90216255187988, 245.09640502929688], min=-176.258056640625, max=701.0830688476562, std=74.65402966167625, histogram=[[5.0, 2097.0, 4986.0, 538.0, 873.0, 70.0, 5.0, 1.0, 0.0, 1.0], [-176.258056640625, -88.52394409179688, -0.7898315429687557, 86.94428100585935, 174.6783935546875, 262.4125061035156, 350.1466186523437, 437.8807312011719, 525.61484375, 613.348956298828, 701.0830688476562]], valid_percent=13.673469387755102)} nodata_value=nan count=1 width=13490 height=12906 overviews=[2, 4, 8, 16, 32] driver='GTiff' check-1

[vincentsarago]

I don't understand why this can't be encoded in JSON 🤷

would you be able to share the file?

[shubham-scisar]

@vincentsarago Pfa the tif file.
clipped.zip

[vincentsarago]

Excellent I found the bug! I'll fix it in rio-tiler and publish a new version before the end of the day 🙏

[vincentsarago]

FYI: I couldn't see at first why NaN wasn't supported in the response. This is all because starlette disable NaN in response https://github.com/encode/starlette/blob/42592d68e5d7fd044b79d7846eabe7f33961527c/starlette/responses.py#L160

In [1]: import json

In [2]: json.dumps({"a": float('nan')})
Out[2]: '{"a": NaN}'

In [3]: json.dumps({"a": float('nan')}, allow_nan=False)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-0efe7d1813c4> in <module>
----> 1 json.dumps({"a": float('nan')}, allow_nan=False)

/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
    232     if cls is None:
    233         cls = JSONEncoder
--> 234     return cls(
    235         skipkeys=skipkeys, ensure_ascii=ensure_ascii,
    236         check_circular=check_circular, allow_nan=allow_nan, indent=indent,

/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/json/encoder.py in encode(self, o)
    197         # exceptions aren't as detailed.  The list call should be roughly
    198         # equivalent to the PySequence_Fast that ''.join() would do.
--> 199         chunks = self.iterencode(o, _one_shot=True)
    200         if not isinstance(chunks, (list, tuple)):
    201             chunks = list(chunks)

/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/json/encoder.py in iterencode(self, o, _one_shot)
    255                 self.key_separator, self.item_separator, self.sort_keys,
    256                 self.skipkeys, _one_shot)
--> 257         return _iterencode(o, 0)
    258 
    259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,

ValueError: Out of range float values are not JSON compliant

This is ok and seems to be the safest choice (returning NaN in the web client might not be accepted).

[vincentsarago]

@shubhamsharma1609 thanks for raising the issue. turns out this was a known issue in FastAPI but without really an easy fix. I've opened #374 with hope we can merge/publish tomorrow

[shubham-scisar]

Hi @vincentsarago !!!
Thanks for your fast actions.Kindly commit the changes.

[vincentsarago]

your welcome @shubhamsharma1609

I just pushed https://github.com/developmentseed/titiler/releases/tag/0.3.10, it should be on pypi in couple seconds ;-)