Skip to content

Conversation

@boydcpeters
Copy link

Summary

Hi all, this PR fixes an issue in the Python ndstorage writer where the ordering of axes in the NDTiff index file could be inconsistent across frames. This caused downstream reading with tifffile to produce incorrect array shapes (cgohlke/tifffile#311).

Importantly, this change brings the Python writer in line with Micro-Manager Java NDTiff writer, which seems to always serializes axes alphabetically to produce a consistent index key. (I am pretty sure it is here:

String indexKey = IndexEntryData.serializeAxes(axessss);
, but I am not really familiar with the codebase so feel free to correct me.)

Problem

In the original implementation, index keys were derived from axes dictionaries by:

  1. First converting these axes dictionaries to a frozenset (used as intermediate keys)
    index_data_entry = self.current_writer.write_image(frozenset(coordinates.items()), image, metadata)
  2. When writing the index, converting the frozenset back to dictionaries:
    axes = {axis_name: position for axis_name, position in self.axes_key}
  3. And then serializing these dictionaries with json.dumps():
    axes_key_bytes = json.dumps(axes).encode('utf-8')

However:

  • frozenset is unordered, so the order of axes is not guaranteed.
  • As a result, the same image coordinates could produce different serialized index keys for different frames.

Solution

This PR modifies NDTiffIndexEntry.as_byte_buffer() to use serialize_axes(axes) for generating the index key. This ensures:

  • Axes are always sorted in a canonical, consistent order.
  • Python writer now produces the same index keys as the Java writer.
  • Downstream readers like tifffile can rely on consistent axes order, preventing shape mismatches.

@boydcpeters boydcpeters changed the title fix: ensure consistent NDTiff axes ordering for indices in index file with Python reader fix: ensure consistent NDTiff axes ordering for indices in index file with Python writer Oct 1, 2025
@henrypinkard
Copy link
Collaborator

I'd suggest merging this (assuming it passes test). However, my write access to this repository has been removed. Perhaps @nicost or @marktsuchida could merge it?

@marktsuchida
Copy link
Member

Thanks, @henrypinkard! That would be the effect of you retiring from the org, but we can add you as a collaborator on specific repos. I've sent an invite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants