`save_cog_with_dask`: Cannot convert fill_value 999999 to dtype uint8

remi-braun · November 22, 2024, 3:19pm

Hi there ,

I am currently trying to write COGs via Dask with odc.geo.cog.save_cog_with_dask.
Most of the time, it seems fine, but with the same code, deserialisation issue happen sometimes:

  File "C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\numpy\ma\core.py", line 489, in _check_fill_value
    fill_value = np.asarray(fill_value, dtype=ndtype)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: Python integer 999999 out of bounds for uint8

My code is:

cog.save_cog_with_dask(
    xds.copy(data=xds.fillna(nodata).astype(dtype)).rio.set_nodata(nodata),
    str(path),
).compute()

I triple checked, everything is fine in the nodata area, for both rioxarray and odc accessors and encoding.0 in float shouldn’t be an issue as they are safely cast into uint8 data.

Full stack trace

2024-11-22 09:17:34,307 - distributed.protocol.core - CRITICAL - Failed to deserialize
Traceback (most recent call last):
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\numpy\ma\core.py”, line 489, in _check_fill_value
fill_value = np.asarray(fill_value, dtype=ndtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: Python integer 999999 out of bounds for uint8
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\protocol\core.py”, line 175, in loads
return msgpack.loads(
^^^^^^^^^^^^^^
File “msgpack\_unpacker.pyx”, line 194, in msgpack._cmsgpack.unpackb
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\protocol\core.py”, line 159, in _decode_default
return merge_and_deserialize(
^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\contextlib.py”, line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\protocol\serialize.py”, line 525, in merge_and_deserialize
return deserialize(header, merged_frames, deserializers=deserializers)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\protocol\serialize.py”, line 452, in deserialize
return loads(header, frames)
^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\protocol\serialize.py”, line 66, in dask_loads
return loads(header[“sub-header”], frames)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\protocol\numpy.py”, line 219, in deserialize_numpy_maskedarray
return np.ma.masked_array(data, mask=mask, fill_value=fill_value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\numpy\ma\core.py”, line 2968, in new
_data._fill_value = _check_fill_value(fill_value, _data.dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\numpy\ma\core.py”, line 495, in _check_fill_value
raise TypeError(err_msg % (fill_value, ndtype)) from e
TypeError: Cannot convert fill_value 999999 to dtype uint8
2024-11-22 09:17:34,312 - distributed.worker - ERROR - Cannot convert fill_value 999999 to dtype uint8
Traceback (most recent call last):
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\numpy\ma\core.py”, line 489, in _check_fill_value
fill_value = np.asarray(fill_value, dtype=ndtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: Python integer 999999 out of bounds for uint8
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\worker.py”, line 2075, in gather_dep
response = await get_data_from_worker(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\worker.py”, line 2881, in get_data_from_worker
response = await send_recv(
^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\core.py”, line 1018, in send_recv
response = await comm.read(deserializers=deserializers)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\comm\tcp.py”, line 247, in read
msg = await from_frames(
^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\comm\utils.py”, line 78, in from_frames
res = _from_frames()
^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\comm\utils.py”, line 61, in _from_frames
return protocol.loads(
^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\protocol\core.py”, line 175, in loads
return msgpack.loads(
^^^^^^^^^^^^^^
File “msgpack\_unpacker.pyx”, line 194, in msgpack._cmsgpack.unpackb
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\protocol\core.py”, line 159, in _decode_default
return merge_and_deserialize(
^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\contextlib.py”, line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\protocol\serialize.py”, line 525, in merge_and_deserialize
return deserialize(header, merged_frames, deserializers=deserializers)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\protocol\serialize.py”, line 452, in deserialize
return loads(header, frames)
^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\protocol\serialize.py”, line 66, in dask_loads
return loads(header[“sub-header”], frames)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\protocol\numpy.py”, line 219, in deserialize_numpy_maskedarray
return np.ma.masked_array(data, mask=mask, fill_value=fill_value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\numpy\ma\core.py”, line 2968, in new
_data._fill_value = _check_fill_value(fill_value, _data.dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\numpy\ma\core.py”, line 495, in _check_fill_value
raise TypeError(err_msg % (fill_value, ndtype)) from e
TypeError: Cannot convert fill_value 999999 to dtype uint8
2024-11-22 09:17:34,367 - distributed.core - ERROR - Exception while handling op get_data
Traceback (most recent call last):
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\core.py”, line 834, in _handle_comm
result = await result
^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\core.py”, line 981, in wrapper
return await func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\worker.py”, line 1798, in get_data
assert response == “OK”, response
^^^^^^^^^^^^^^^^
AssertionError: {‘op’: ‘get_data’, ‘keys’: {(‘from_sequence-9bfcd20b0840146be3c99f402b3807dc’, 83)}, ‘who’: ‘tcp://127.0.0.1:57260’, ‘reply’: True}
2024-11-22 09:17:34,372 - distributed.worker - ERROR - {‘op’: ‘get_data’, ‘keys’: {(‘from_sequence-9bfcd20b0840146be3c99f402b3807dc’, 83)}, ‘who’: ‘tcp://127.0.0.1:57260’, ‘reply’: True}
Traceback (most recent call last):
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\worker.py”, line 2075, in gather_dep
response = await get_data_from_worker(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\worker.py”, line 2881, in get_data_from_worker
response = await send_recv(
^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\core.py”, line 1043, in send_recv
raise exc.with_traceback(tb)
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\core.py”, line 834, in _handle_comm
result = await result
^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\core.py”, line 981, in wrapper
return await func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^
File “C:\Users\rbraun\Anaconda3\envs\eoprocesses\Lib\site-packages\distributed\worker.py”, line 1798, in get_data
assert response == “OK”, response
^^^^^^^^^^^^^^^^^
AssertionError: {‘op’: ‘get_data’, ‘keys’: {(‘from_sequence-9bfcd20b0840146be3c99f402b3807dc’, 83)}, ‘who’: ‘tcp://127.0.0.1:57260’, ‘reply’: True}

Does somebody ever encountered something like that?

Sorry it’s difficult to adress a minimum workable example as the rasters I’m using are very heavy (and commercial data).

Conda list

conda list

packages in environment at C:\Users\rbraun\Anaconda3\envs\eoprocesses:

NB: It’s a repost from `save_cog_with_dask`: Cannot convert fill_value 999999 to dtype uint8 - Dask Forum
I don’t think (or know) if it’s a bug, sio I rather start the discussion here before creating an issue there.

chuckwondo · November 23, 2024, 12:21pm

It appears that you are attempting to create an array of type uint8, which can only hold integer values in the range [0, 255], but attempting to insert the value 999999, which is far outside of that range. You either need to choose a value within that range, or use a dtype with a range that includes your large value.

remi-braun · November 23, 2024, 2:43pm

This is where I don’t understand, where does this value 999999 comes from?
You can see on the screenshot that all the nodata values are set to 0.

Topic		Replies	Views
Dtype is ignored if _FillValue in encoding is provided for xr.to_zarr()? Data	0	394	August 9, 2022
Parallel COG / TIFF Storage Data	0	428	August 14, 2023
Pandas dtypes: now free from nanosecond limitation News & Announcements	4	551	January 30, 2023
ValueError: cannot reshape array of size 1 into shape (13968,) Data	1	754	October 16, 2023
Xarray trouble decoding NetCDF with compressed integers Data	3	58	November 5, 2024

`save_cog_with_dask`: Cannot convert fill_value 999999 to dtype uint8

packages in environment at C:\Users\rbraun\Anaconda3\envs\eoprocesses:

Name Version Build Channel

Related topics