Skip to content

FIX: Persist DataArray attrs in zarr append mode#11263

Closed
krokosik wants to merge 4 commits intopydata:mainfrom
krokosik:fix/dataarray-zarr-append-attrs
Closed

FIX: Persist DataArray attrs in zarr append mode#11263
krokosik wants to merge 4 commits intopydata:mainfrom
krokosik:fix/dataarray-zarr-append-attrs

Conversation

@krokosik
Copy link
Copy Markdown
Contributor

@krokosik krokosik commented Mar 26, 2026

This PR fixes an inconsistency in zarr append behavior between Dataset and DataArray paths.

  • DataArray.to_zarr(..., mode="a"/"a-") previously updated data but did not persist attrs for pre-existing arrays.
  • The fix writes encoded attrs for existing arrays in append mode (while keeping r+ semantics unchanged).
  • Added regression tests that:
    • verify DataArray attrs persist in append mode, and
    • verify Dataset/DataArray attr behavior is consistent.
      I also added a warning for region writes that update attrs in append mode.
      Rationale:
  • region writes are often used for parallel/distributed chunk updates (thanks @max-sixty for mentioning that in Update Variable metadata when using 'a' mode in Zarr #8159)
  • attrs are global metadata updates and can race across concurrent writers even if data regions are disjoint
  • warning is non-breaking and nudges users toward safer patterns (serialize metadata updates, keep region writes data-only, or use r+ for strict in-place data updates)

Reproducer

import xarray as xr

# Dataset attrs update in append mode
ds_path = "tmp/ds"
xr.Dataset().assign_attrs(a=1).to_zarr(ds_path, mode="w")
xr.open_dataset(ds_path, engine="zarr").assign_attrs(b=2).to_zarr(ds_path, mode="a")

# DataArray attrs update in append mode
da_path = "tmp/da"
xr.DataArray([0], dims=["x"]).assign_attrs(a=1).to_zarr(da_path, mode="w")
xr.open_dataarray(da_path, engine="zarr").assign_attrs(b=2).to_zarr(da_path, mode="a")

assert xr.open_dataset(ds_path, engine="zarr").attrs == xr.open_dataarray(da_path, engine="zarr").attrs

Before this change, the DataArray append path did not persist updated attrs for existing arrays.

@github-actions github-actions Bot added topic-backends topic-zarr Related to zarr storage library io labels Mar 26, 2026
@krokosik
Copy link
Copy Markdown
Contributor Author

Just read through issues and stumbled upon discussions in #8755 and #8343. My understanding is that it is intentional after all

@krokosik krokosik closed this Mar 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

io topic-backends topic-zarr Related to zarr storage library

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Updating DataArray attributes in a Zarr archive does not work as for Dataset

1 participant