Skip to content

feat: support header_type UInt32 #83

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ranocha merged 4 commits into from
Mar 6, 2026
Merged

feat: support header_type UInt32 #83

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1e41b71
feat: support header_type UInt32
henry2004y Mar 3, 2026
12a5eac
formatting
henry2004y Mar 3, 2026
9bf419f
Revert syntax changes
henry2004y Mar 4, 2026
9499266
Add test for header type UInt32
henry2004y Mar 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,6 @@ Further example VTK files can be found in the
* Probably reading from VTK files that were *not* created by [WriteVTK.jl](https://github.com/JuliaVTK/WriteVTK.jl) will fail, specifically since
* compressed data is assumed to be stored as a single block
* appended data is assumed to be stored as `raw`
* `header_type` is hardcoded to `UInt64`
* Extracting primitives from `PolyData` files other than vortices, lines, and/or polygons
* Likely anything else that is not specifically mentioned under *What works*

Expand Down
1 change: 0 additions & 1 deletion docs/src/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,6 @@ Further example VTK files can be found in the
* Probably reading from VTK files that were *not* created by [WriteVTK.jl](https://github.com/JuliaVTK/WriteVTK.jl) will fail, specifically since
* compressed data is assumed to be stored as a single block
* appended data is assumed to be stored as `raw`
* `header_type` is hardcoded to `UInt64`
* Extracting primitives from `PolyData` files other than vortices, lines, and/or polygons
* Likely anything else that is not specifically mentioned under *What works*
## Development
Expand Down
51 changes: 30 additions & 21 deletions src/ReadVTK.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,13 +47,14 @@ mutable struct VTKFile
appended_data::Vector{UInt8}
n_points::Int
n_cells::Int
header_type::DataType

# Create inner constructor to add finalizer (this requires VTKFile to be a *mutable* struct)
function VTKFile(filename, xml_file, file_type, version, byte_order, compressor,
appended_data, n_points, n_cells)
appended_data, n_points, n_cells, header_type)
# Create new object
vtk_file = new(filename, xml_file, file_type, version, byte_order, compressor,
appended_data, n_points, n_cells)
appended_data, n_points, n_cells, header_type)

# Create finalizer that releases memory for VTK file
f(vtk_file) = free(vtk_file.xml_file)
Expand All @@ -63,8 +64,8 @@ mutable struct VTKFile
end
end

# Header type is hardcoded
header_type(::VTKFile) = UInt64
# Header type
header_type(vtk_file::VTKFile) = vtk_file.header_type

# Return true if data is compressed (= XML attribute `compressor` is non-empty in VTK file)
is_compressed(vtk_file::VTKFile) = !isempty(vtk_file.compressor)
Expand Down Expand Up @@ -133,7 +134,8 @@ function VTKFile(filename)
(byte_order == "BigEndian" && !is_little_endian))

# Ensure matching header type
@assert header_type == "UInt64"
@assert header_type in ("UInt32", "UInt64")
header_type_parsed = string_to_data_type(header_type)

# Ensure supported compression type
@assert in(compressor, ("", "vtkZLibDataCompressor"))
Expand Down Expand Up @@ -162,7 +164,7 @@ function VTKFile(filename)

# Create and return VTKFile
return VTKFile(filename, xml_file, file_type, version, byte_order, compressor,
appended_data, n_points, n_cells)
appended_data, n_points, n_cells, header_type_parsed)
end

# Show basic information on REPL
Expand Down Expand Up @@ -766,17 +768,21 @@ function get_data(data_array::VTKDataArray{T, N, <:FormatBinary}) where {T, N}
raw = base64decode(split(strip(content(data_array.data_array)), "\n")[1])

if is_compressed(data_array)
# If data is stored compressed, the first four integers of type `header_type` are the header and
# must be discarded
first = 1 + 4 * sizeof(header_type(data_array.vtk_file))
# If data is stored compressed, the header consists of `3 + num_blocks` integers of type
Copy link
Copy Markdown
Member

@sloede sloede Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a reference where this information can be found that you can put here?

Copy link
Copy Markdown
Contributor Author

@henry2004y henry2004y Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really. It is found via experiments and I do not really guarantee it is universal.

# `header_type`. It must be discarded.
HeaderType = header_type(data_array.vtk_file)
header_num_blocks = reinterpret(HeaderType, raw[1:sizeof(HeaderType)])[1]
header_len = 3 + header_num_blocks
first = 1 + header_len * sizeof(HeaderType)
last = length(raw)

# Pass data through ZLib decompressor
uncompressed = transcode(ZlibDecompressor, raw[first:last])
else
# If data is stored uncompressed, the first integer of type `header_type` is the header and must
# be discarded
first = 1 + sizeof(header_type(data_array.vtk_file))
HeaderType = header_type(data_array.vtk_file)
first = 1 + sizeof(HeaderType)
last = length(raw)
uncompressed = view(raw, first:last)
end
Expand All @@ -799,24 +805,28 @@ function get_data(data_array::VTKDataArray{T, N, <:FormatAppended}) where {T, N}
HeaderType = header_type(data_array.vtk_file)

if is_compressed(data_array)
# If data is stored compressed, the first four integers of type `header_type` are the header and
# the fourth value contains the number of bytes to read
# If data is stored compressed, the header consists of `3 + num_blocks` integers of type `header_type`.
# The sum of the subsequent blocks contains the total number of compressed bytes to read.
first = data_array.offset + 1
last = data_array.offset + 4 * sizeof(HeaderType)
header = Int.(reinterpret(HeaderType, raw[first:last]))
n_bytes = header[4]
header_num_blocks = reinterpret(HeaderType,
raw[first:(first + sizeof(HeaderType) - 1)])[1]
header_len = 3 + header_num_blocks

first = data_array.offset + 4 * sizeof(HeaderType) + 1
last = first + n_bytes - 1
last_header = data_array.offset + header_len * sizeof(HeaderType)
header = Int.(reinterpret(HeaderType, raw[first:last_header]))
n_bytes = sum(@view header[4:end])

first_data = last_header + 1
last_data = first_data + n_bytes - 1

# Pass data through ZLib decompressor
if last > length(raw)
if last_data > length(raw)
@show data_array data_array.vtk_file.xml_file
@show first, last, size(raw)
@show first, last_data, size(raw)
error("mistake in get_data")
end

uncompressed = transcode(ZlibDecompressor, raw[first:last])
uncompressed = transcode(ZlibDecompressor, raw[first_data:last_data])
Copy link
Copy Markdown
Member

@sloede sloede Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work even if the data is compressed in multiple blocks? That is, it is not necessary to decompress block by block?

Copy link
Copy Markdown
Contributor Author

@henry2004y henry2004y Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I have tried it on one sample data with 19 blocks, and it also works. The sample test file I added has only 1 block though.

else
# If data is stored uncompressed, the first integer of type `header_type` is the header and
# contains the number of bytes to read
Expand Down Expand Up @@ -981,7 +991,6 @@ function get_extents(xml_file, min_extent = [0; 0; 0])
end



"""
get_points(vtk_file::VTKFile)

Expand Down
6 changes: 3 additions & 3 deletions test/rectilinear.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ for compress in [true, false]
y = zeros(FloatType, Nj)
z = zeros(FloatType, Nk)

[x[i] = i * i / Ni / Ni for i in 1:Ni]
[y[j] = sqrt(j / Nj) for j in 1:Nj]
[z[k] = k / Nk for k in 1:Nk]
[x[i] in i * i / Ni / Ni for i in 1:Ni]
[y[j] in sqrt(j / Nj) for j in 1:Nj]
[z[k] in k / Nk for k in 1:Nk]
Comment thread
henry2004y marked this conversation as resolved.
Outdated

# Create some scalar and vector data.
p = zeros(FloatType, Ni, Nj, Nk)
Expand Down
Loading