This comment by @flying-sheep made me realise we are missing a potentially large perf optimisation for zarrs-python. We should reuse partial decoders for batches of chunk_descriptions that refer to the same chunk in retrieve_chunks_and_apply_index.
https://github.com/ilan-gold/zarrs-python/blob/53af191c7ca5215c31221520da1ee1c634852b78/src/lib.rs#L312-L329
The ChunkRepresentation can be reused for each batch too.
Relevant: https://book.zarrs.dev/arrays/array_read.html#reading-multiple-chunk-subsets
This comment by @flying-sheep made me realise we are missing a potentially large perf optimisation for
zarrs-python. We should reuse partial decoders for batches ofchunk_descriptionsthat refer to the same chunk inretrieve_chunks_and_apply_index.https://github.com/ilan-gold/zarrs-python/blob/53af191c7ca5215c31221520da1ee1c634852b78/src/lib.rs#L312-L329
The
ChunkRepresentationcan be reused for each batch too.Relevant: https://book.zarrs.dev/arrays/array_read.html#reading-multiple-chunk-subsets