Skip to content

feat: support header_type UInt32#83

Merged
ranocha merged 4 commits into
JuliaVTK:mainfrom
henry2004y:main
Mar 6, 2026
Merged

feat: support header_type UInt32#83
ranocha merged 4 commits into
JuliaVTK:mainfrom
henry2004y:main

Conversation

@henry2004y
Copy link
Copy Markdown
Contributor

This PR addresses two main issues related to parsing VTK files, improving compatibility with files generated by third-party tools like ParaView/PyVista (#78):

  1. Support for UInt32 Header Types: Previously, the dataset header type was hardcoded to UInt64. This PR updates the VTKFile parser to dynamically read the header_type XML attribute and properly support both UInt32 and UInt64 headers across the package.

  2. Support for Multi-Block Zlib Compressed Data: Fixes a ZlibError: incorrect header check (code: -3) encountered when reading appended or binary data that has been compressed into multiple blocks (a common behavior in ParaView/PyVista output). The previous implementation assumed a hardcoded compression header size of 4 integers, which is only valid for single-block compression (such as the default used by WriteVTK.jl). The parsing logic in get_data has been updated to reliably read num_blocks from the start of the header, dynamically calculate the correct header length (3 + num_blocks integers), and accurately define the byte range for the Zlib decompression step.

Changes Made

  • src/ReadVTK.jl:
    • Added a header_type::DataType field to the VTKFile struct.
    • Updated the VTKFile constructor to parse the header_type (validating it as "UInt32" or "UInt64").
    • Updated get_data for FormatBinary and FormatAppended arrays to calculate offsets dynamically using header_len = 3 + num_blocks instead of a hardcoded length of 4.
    • Verified that ZLib decompressor gracefully handles the multi-block payload natively once the byte range is correctly identified.
  • Documentation: Updated README.md and docs/src/index.md to remove the caveat stating that header_type must be UInt64.

Note that I have not added new test for this feature since I don't know how to add a proper minimal test. Feel free to come up with a suitable test in the dataset.

@ranocha
Copy link
Copy Markdown
Member

ranocha commented Mar 4, 2026

Thanks a lot for your contribution!

Note that I have not added new test for this feature since I don't know how to add a proper minimal test. Feel free to come up with a suitable test in the dataset.

We use the repository https://github.com/JuliaVTK/ReadVTK_examples to store example files. Please

  • create some example files locally that require the new features you added and test whether ReadVTK.jl can read them correctly with your changes
  • create a PR to https://github.com/JuliaVTK/ReadVTK_examples with your example files
  • update the commit hash in
    TEST_EXAMPLES_COMMIT = "1f961a7ad38e76a8d552b8392d8cd661ab9f79b4"
    in this PR
  • add tests in this PR where you read the example files you added

Comment thread test/rectilinear.jl Outdated
@henry2004y
Copy link
Copy Markdown
Contributor Author

After merging JuliaVTK/ReadVTK_examples#3, I will push the new test here.

@ranocha
Copy link
Copy Markdown
Member

ranocha commented Mar 5, 2026

I merged your PR adding the new example file.

Comment thread test/structuredgrid.jl
Copy link
Copy Markdown
Member

@ranocha ranocha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot! I will give @sloede a chance to review this PR as well.

Copy link
Copy Markdown
Member

@sloede sloede left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice PR, thanks a lot for this feature! I have only two minor remarks, after which this PR is ready to merge.

Comment thread src/ReadVTK.jl
# If data is stored compressed, the first four integers of type `header_type` are the header and
# must be discarded
first = 1 + 4 * sizeof(header_type(data_array.vtk_file))
# If data is stored compressed, the header consists of `3 + num_blocks` integers of type
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a reference where this information can be found that you can put here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really. It is found via experiments and I do not really guarantee it is universal.

Comment thread src/ReadVTK.jl
end

uncompressed = transcode(ZlibDecompressor, raw[first:last])
uncompressed = transcode(ZlibDecompressor, raw[first_data:last_data])
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work even if the data is compressed in multiple blocks? That is, it is not necessary to decompress block by block?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I have tried it on one sample data with 19 blocks, and it also works. The sample test file I added has only 1 block though.

@ranocha
Copy link
Copy Markdown
Member

ranocha commented Mar 6, 2026

Thanks a lot! I will merge this now and release the new version. It would be great if you could create a second PR (to the examples repository and afterward to this repository) with another file having multiple blocks to ensure that this feature continues to work in the future.

@ranocha ranocha merged commit 7789af4 into JuliaVTK:main Mar 6, 2026
11 checks passed
@ranocha ranocha mentioned this pull request Mar 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants