v2.0.3

What's Changed

• fix: skip empty paths in set_virtual_refs_arr by @TomNicholas in #2080
• fix: Dont write non-persisted config to v2 repo on migration by @mpiannucci in #2081
• Release/2.0.3 by @mpiannucci in #2082

Full Changelog: v2.0.2...v2.0.3

v2.0.2

What's Changed

• merge changelog from 1.x into main changelog by @ianhi in #2058
• Pass storage settings to virtual chunk fetchers by @TomNicholas in #2061
• Bump the js-dependencies group in /icechunk-js with 2 updates by @dependabot[bot] in #2022
• Bump @oxc-node/core from 0.0.35 to 0.1.0 in /icechunk-js in the js-dependencies group by @dependabot[bot] in #2068
• Bump the rust-dependencies group with 2 updates by @dependabot[bot] in #2067
• Add Arraylake comparison page to docs by @TomNicholas in #2066
• Add set_virtual_refs_arr with numpy arrays by @TomNicholas in #2049
• Update to_icechunk mode docstring to match xarray by @TomNicholas in #2072
• Show effective default values in config reprs by @TomNicholas in #2062
• Fix SVG ancestry graph text invisible on dark backgrounds by @TomNicholas in #2070
• Add async from_store and make walk public on GroupNode by @ianhi in #2074
• Release v2.0.2 by @ianhi in #2075

Full Changelog: v2.0.1...v2.0.2

v2.0.1

This minor patch release fixes a few unintended regressions in Icechunk 2.

Migrate your existing repos using our migration guide and review the breaking changes from Icechunk 1.

See the complete change log to Icechunk 2 here:

https://github.com/earth-mover/icechunk/blob/v2.0.0/Changelog.python.md

What's Changed

• chore(ci): shard hypothesis tests across 3 parallel runners by @samantha-earthmover in #2046
• Fix icechunk.store module renamed to icechunk.zarr by @TomNicholas in #2047
• Fix sdist build: symlink README from root by @dcherian in #1937
• Update docs banner for Icechunk 2.0 by @TomNicholas in #2052
• Optimize set_virtual_refs to do a single node lookup by @TomNicholas in #2045
• Set MSRV to 1.91.0; bump version to 2.0.1 by @dcherian in #2054

Full Changelog: v2.0.0...v2.0.1

v2.0.0

Icechunk 2 is here!

This release represents the next evolution of the Icechunk library and on-disk format,
shaped by hundreds of commits from dozens of contributors.

Icechunk 2 brings new features, stronger consistency guarantees, and higher performance across several
workloads, all while maintaining format compatibility. Icechunk 2 can read and write your existing
Icechunk 1 repos, and when you're ready to upgrade the format, a single function call migrates
your repos quickly and safely.

Migrate your existing repos using our migration guide and review the breaking changes from Icechunk 1.

See the complete change log here:

https://github.com/earth-mover/icechunk/blob/v2.0.0/Changelog.python.md

New Contributors

• @ebarylko made their first contribution in #1343
• @evaporei made their first contribution in #1426
• @d0rianb made their first contribution in #1489
• @aladinor made their first contribution in #1491
• @samantha-earthmover made their first contribution in #1646
• @pfackeldey made their first contribution in #1650
• @tylanderson made their first contribution in #1645
• @jayendra13 made their first contribution in #1490

Full Changelog: v1.1.11...v2.0.0

v2.0.0-alpha.7

What's Changed

• Make V2 branch expiration behaviour match V1 by @dcherian in #1932
• third-wheel instructions by @ianhi in #1946
• feat: Allow changing repo status without creating an Update by @paraseba in #1950
• chore: Change RustFS port to avoid conflicts by @paraseba in #1951
• Fix amend and moved nodes in transaction log by @li-em in #1896
• PyRepr trait for structured, recursive Python reprs by @TomNicholas in #1944
• Python reprs for every class by @TomNicholas in #1948
• More categorization in ic2-changes by @dcherian in #1971
• First pass at a V2 spec by @paraseba in #1914
• Docs: add sample datasets to nav, and 3rd-party examples by @TomNicholas in #1938
• Docs/spec flatbuffers inline by @ianhi in #1968
• Temporarily disable Tigris integration tests. by @dcherian in #1981
• Compare GC/expiry behaviour between V1 & V2 by @dcherian in #1979
• Spec structure improvements by @TomNicholas in #1984
• Add structured Storage repr and format_version to Repository by @TomNicholas in #1983
• fix: repr test failure by @li-em in #1993
• Strip x-amz-checksum-crc32 header from R2's 304 responses. by @dcherian in #1991
• Raise error in manifest_files field is inconsistent with actual manifests listed in arrays by @li-em in #1989
• Verify all parent nodes exist when moving a node by @li-em in #1974
• Filter out all checksum headers on R2 (not just CRC32). by @dcherian in #1999
• Retry on "streaming error" too by @dcherian in #2000
• expand transaction log section of spec by @ianhi in #1987
• chore: prepare alpha.7 by @li-em in #2006
• Document flush by @TomNicholas in #2008

Full Changelog: v2.0.0-alpha.6...v2.0.0-alpha.7

v2.0.0-alpha.6

What's Changed

• ci: pin all GitHub Actions to commit SHAs by @samantha-earthmover in #1908
• Use manifest_files_v2 field in the flatbuffers by @paraseba in #1909
• Move all dependencies to workspace, use workspace=true in members by @li-em in #1912
• Introduce a mechanism for unconditional repo info write by @paraseba in #1915
• Bump Rust icechunk to alpha.5 by @paraseba in #1916
• Bump all rust crates by @paraseba in #1918
• Do eager tracking of moves by @ianhi in #1910
• Stateful test for parallel sessions by @dcherian in #1018
• Check for some important conflicts on session merge by @paraseba in #1815
• Bump the rust-dependencies group with 4 updates by @dependabot[bot] in #1923
• Hypothesis Test Improve + new inspection functions by @ianhi in #1920
• Update build targets: drop python 3.11; support free-threading by @ianhi in #1905
• Fix stubtest: mark dunder method params as positional-only by @dcherian in #1936
• added support for str, repr and repr_html by @jayendra13 in #1490
• Add node ID and type to transaction log, clarify move ordering by @li-em in #1919
• Bump to 2.0.0-alpha.6 by @dcherian in #1941
• Bump remaining crates to 2.0.0-alpha.6 by @dcherian in #1942
• Test format readability with Python libraries by @dcherian in #1939

New Contributors

• @jayendra13 made their first contribution in #1490

Full Changelog: v2.0.0-alpha.4...v2.0.0-alpha.6

v2.0.0-alpha.4

What's Changed

• Add updated_chunks_counts to transaction log interface by @paraseba in #1805
• Add TimeoutSettings to storage and wire through to S3 client by @dcherian in #1794
• Bump quinn-proto from 0.11.13 to 0.11.14 by @dependabot[bot] in #1807
• Change enums to snake case, default rewrite_manifest to use new commit by @mpiannucci in #1812
• Upgrade all dependencies by @paraseba in #1813
• Honor can_write in Storage by @li-em in #1771
• Fix compat config by @mpiannucci in #1809
• Delete PR template for now by @paraseba in #1816
• Handle unknown future spec versions better by @mpiannucci in #1817
• Pin xarray child dep for now to unblock ci by @mpiannucci in #1837
• Move all integration tests to a single binary by @paraseba in #1829
• Prepare workflows for merge queue by @paraseba in #1839
• Add user agent to storage backends by @mpiannucci in #1818
• Fix a few FIXMEs by @paraseba in #1821
• Fix bug in list_prefix implementation by @paraseba in #1825
• Various minor improvements, todos and fixmes by @paraseba in #1827
• Add asset manager benchmarks by @dcherian in #1845
• More instrumentation in asset manager by @paraseba in #1847
• Bump the rust-dependencies group with 3 updates by @dependabot[bot] in #1848
• Undeprecate accessing storage from asset manager by @paraseba in #1851
• Feat: add LatencyStorage + Fix: Flaky stateful test failures by @ianhi in #1846
• Check repository status before doing any operations by @li-em in #1797
• Small cleanup edits by @dcherian in #1855
• Support rectilinear chunk grids by @dcherian in #1806
• Instrument fetch_config by @paraseba in #1854
• Delete the v1 main branch ref first after migration to kill v1 usage quickly by @mpiannucci in #1859
• Add example to calculate size of very large repo info object by @paraseba in #1860
• Fix bug in ops log after migration by @paraseba in #1861
• Benchmark virtual chunks set/get with prefixes by @dcherian in #1858
• Copy object race condition by @paraseba in #1863
• Concurrency permutation testing with shuttle \m/ by @dcherian in #1840
• Accumulate 1k updates per repo info file, instead of 100 by @paraseba in #1864
• Regenerate test data by @paraseba in #1868
• Process nodes in parallel during the flush process by @paraseba in #1856
• Create a SpecVersion enum for Python, and listing supported spec versions by @li-em in #1874
• Implement fork in Rust by @dcherian in #1876
• Concurrently fetch tx logs in rebase by @dcherian in #1869
• Consolidate benchmarks into a single binary by @dcherian in #1865
• add hypothesis profiles by @ianhi in #1870
• Bump quinn-proto from 0.11.13 to 0.11.14 by @dependabot[bot] in #1879
• update version policy: major version tracks on-disk format by @ianhi in #1769
• Start documenting IC2 changes by @paraseba in #1878
• Release v2.0.0-alpha.4 by @ianhi in #1882
• split hypothesis tests in CI by @ianhi in #1881
• Handle empty chunks in reindex by @ianhi in #1877
• Update dependencies by @paraseba in #1883
• More restrictive set of lints by @paraseba in #1884
• Add version to shuttle dependency by @paraseba in #1886
• Comment out shuttle usage by @paraseba in #1887
• Add upstream CI workflow to test zarrs against latest icechunk main by @jhamman in #1888
• Update flake lock, rustc 1.94 by @paraseba in #1892
• Split into crates by @paraseba in #1889
• Update rustc to 1.94.0 by @paraseba in #1893
• Fix flaky GC stateful test step (yet again) by @dcherian in #1894
• Fix rust publish GHA by @paraseba in #1895
• Comment out tests on crates release by @paraseba in #1897
• Version bump icechunk-macros by @paraseba in #1898
• Add property tests using generated zarr trees by @ianhi in #1826
• Fix stateful repo ops typing by @li-em in #1900
• feat: Azure refreshable credentials by @maxrjones in #1902
• Use stable rust toolchain on maturin hosts by @mpiannucci in #1906
• fix zip misalignment in tree strategy by @ianhi in #1907

Full Changelog: v2.0.0-alpha.3...v2.0.0-alpha.4

v1.1.21

What's Changed

• Backport list_prefix bug fix by @paraseba in #1842
• release 1.1.21 by @ianhi in #1843
• Backport version policy updates to support/v1.x by @ianhi in #1844

Full Changelog: v1.1.20...v1.1.21

v2.0.0-alpha.3

What's Changed

• better contributing docs build instructions by @ianhi in #1347
• add serialization and deserialization property tests for types in config file by @ebarylko in #1343
• New method Repository.manifest_files to get the snap manifests by @paraseba in #1353
• [DONT MERGE]: Icechunk 2.0 by @paraseba in #1154
• Bump the rust-dependencies group across 1 directory with 3 updates by @dependabot[bot] in #1354
• Satisfy clippy by @TomNicholas in #1361
• Support gs:// prefix in virtual chunk containers by @TomNicholas in #1363
• Add some missing steps for first-time rust contributors by @TomNicholas in #1362
• Ian/v1 ci main by @ianhi in #1359
• Fix typo by @paraseba in #1370
• Expose RepositoryConfig.merge() method in python by @TomNicholas in #1372
• add version policy by @ianhi in #1346
• Ignore mypy complaint about dask.__version__ by @TomNicholas in #1376
• Allow support branch in release workflow by @TomNicholas in #1380
• fix version generation for actual release by @ianhi in #1381
• Improve release guide by @TomNicholas in #1378
• New HTTP Storage instance by @paraseba in #1379
• Add note about conda-forge to release guide by @TomNicholas in #1383
• Bump the rust-dependencies group with 3 updates by @dependabot[bot] in #1387
• Reduce GHA log volume by @paraseba in #1388
• Introduce redirect_storage by @paraseba in #1386
• builds: fix RTD explicitly add patchelf + update macos runners by @ianhi in #1390
• Remove out-of date GCS retry warning by @dcherian in #1393
• fix errors in bug report by @ianhi in #1401
• use rustls-tls instead of native-tls -> openssl by @ianhi in #1402
• Speed up ops_log by storing up to 100 changes per repo file by @paraseba in #1395
• Fix Repository.total_chunks_storage for V1 repos. by @paraseba in #1407
• Upgrade to Rust 1.91 by @paraseba in #1410
• Allow creation of V1 repos by @paraseba in #1414
• We can commit to IC1 repos now by @paraseba in #1415
• More write operations on V1 repos by @paraseba in #1417
• Session.flush on V1 repos by @paraseba in #1418
• GC and expiration for V1 repos by @paraseba in #1422
• docs: fix command for editable mode by @evaporei in #1426
• simplify upstream dependecies to avoid resolution errors by @ianhi in #1429
• fix typing errors by @ianhi in #1433
• A bunch of testing for IC1 repos by @paraseba in #1423
• Bump the rust-dependencies group with 5 updates by @dependabot[bot] in #1437
• Bump actions/checkout from 5 to 6 in the actions group by @dependabot[bot] in #1436
• Update FAQ with spec stability and implementations info by @TomNicholas in #1430
• repository: add list_ prefix to manifest_files() method by @evaporei in #1428
• Update dependencies by @paraseba in #1451
• Port requester pays from support branch by @paraseba in #1454
• mypy tests by @evaporei in #1443
• Get/Set/Update repository level metadata by @paraseba in #1457
• DOC: update overview page with more concrete examples by @ianhi in #1391
• [docs] Update zarr.config.get to zarr.config.set in performance.md by @norlandrhagen in #1459
• Specify Python version for mypy check in upstream by @ianhi in #1461
• Document proxy support by @TomNicholas in #1452
• Clarifications to version policy by @TomNicholas in #1470
• Port: Fix fetching of virtual refs with spaces and other non-url char… by @paraseba in #1472
• Port: More fixes for virtual chunk url escapes (#1474) by @paraseba in #1477
• Fix spec version for new snapshots on V1 repos. by @paraseba in #1482
• Fix icechunk dependency URL in bug report template by @TomNicholas in #1473
• Bump the rust-dependencies group with 3 updates by @dependabot[bot] in #1486
• Made get_client and get_path_str public (#1487) by @d0rianb in #1489
• Bump the actions group with 3 updates by @dependabot[bot] in #1485
• Add configurable max_arrays_to_scan to ManifestPreloadConfig (#1464) by @aladinor in #1491
• Fix anonymous parameter not passed to S3Options in storage functions by @aladinor in #1493
• Make spec_version a property instead of a function by @mpiannucci in #1495
• Add basic http/https docs by @mpiannucci in #1496
• Support creating and pickling conflict errors by @mpiannucci in #1497
• Bump the rust-dependencies group with 4 updates by @dependabot[bot] in #1500
• Bump scientific-python/upload-nightly-action from 0.6.2 to 0.6.3 in the actions group by @dependabot[bot] in #1499
• Minor bug in merge conflict report by @paraseba in #1504
• Upgrade object_store dependency to get proper retries on 429 by @paraseba in #1509
• Allow reindex_array on V1 repos by @dcherian in #1514
• Bump the rust-dependencies group with 5 updates by @dependabot[bot] in #1528
• Extend storage stats calculation to include virtual and inline chunks by @TomNicholas in #1483
• Update libraries by @paraseba in #1535
• Port: Bump up railguards for flatbuffer file size. (#1530) by @paraseba in #1536
• Have default contributing instructions do less intensive rebuilds by @ianhi in #1539
• add repo spec version to zarr stateful tests by @ianhi in #1540
• Add amend to stateful_repo_ops by @ianhi in #1537
• Bump the rust-dependencies group with 4 updates by @dependabot[bot] in #1543
• Clarify supported file formats for virtual references by @TomNicholas in #1546
• Add missing check for GcsCredentials.Anonymous by @li-em in #1556
• Identify chunks as virtual/native by @li-em in #1547
• Add roundtrip serialization property tests for types in change sets file by @ebarylko in #1525
• Add flatbuffer schematics by @dcherian in #1570
• Retry on 408 by @dcherian in #1572
• Add warning for large number of chunks in a single commit by @li-em in #1568
• Remove unused Cargo lock file by @li-em in #1575
• Bump the rust-dependencies group across 1 directory with 8 updates by @dependabot[bot] in #1573
• Smaller property test by @paraseba in #1582
• Add snapshot id to NewCommitUpdate repo change by @paraseba in #1581
• Manifests inform the nodes they contain by @paraseba in #1584
• Allow to create manifests with known id by @paraseba in #1585
• Obtain the repo info object use...

v1.1.20

What's Changed

• throw when opening v2 repo from v1 icechunk by @mpiannucci in #1749
• Backports for IC1 by @ianhi in #1780
• Release v1.1.20 by @ianhi in #1804

Full Changelog: v1.1.19...v1.1.20