1.3.0-rc0

What's Changed

• fix(s3stream): fix deadlock issue by @ShadowySpirits in #1808
• fix(e2e): adaptation to pr: #1794 by @llzcx in #1809
• feat(shell): new consumer in pref command should start consuming from the beginning by @ShadowySpirits in #1812
• fix(metadata): fix sparse index serialization capacity by @SCNieh in #1814
• chore(logger): move trim log to broker by @superhx in #1819
• chore(workflows): e2e support release runner by @superhx in #1825
• chore(config): change quorum default timeout to 5s by @superhx in #1823
• fix(s3strea/wal): handle IOException during flushing WAL header by @Chillax-0v0 in #1827
• chore(workflows): add e2e base image release by @superhx in #1830
• fix(test): fix rolling_update_test e2e by @superhx in #1835
• feat(s3stream): add AsyncLRUCache metric by @lifepuzzlefun in #1829
• fix(s3stream): fix storage failure handle deadlock by @superhx in #1838
• fix(s3stream/wal): fail all IO operations once the WAL is failed by @Chillax-0v0 in #1840
• feat(metadata): limit the size of sparse index cache by @SCNieh in #1832
• feat(issues1842): cleanup metastream kv after deleting topic by @superhx in #1843
• fix(s3stream): error callback by @superhx in #1845
• fix(revert): error callback by @superhx in #1851
• feat(metadata): avoid serialize empty index by @SCNieh in #1852
• feat(metadata): limit sparse index cache update interval by @SCNieh in #1847
• feat(s3stream): support call #close(force=true) after #close(force=false) by @superhx in #1854
• feat(metadata): prune invalid sparse index with image on startup by @SCNieh in #1856
• feat(s3stream): add network rate limiter for s3 wal to export metrics by @ShadowySpirits in #1859
• feat(s3stream): change the checksum algorithm to crc32c by @ShadowySpirits in #1860
• feat(s3stream): support change checksum algorithm by @ShadowySpirits in #1863
• feat(e2e): use cloud vms for e2e tests; move wal path by @mooc9988 in #1862
• fix(e2e): always archive result artifacts by @mooc9988 in #1866
• fix(storage): regard all produce requests as duplicate whose sequence… by @Chillax-0v0 in #1865
• revert: enable LocalStreamRangeIndexCacheTest#testEvict by @Chillax-0v0 in #1869
• feat(wal): failoverTest optimization by @CLFutureX in #1871
• fix(metadata): catch exception to prevent unnecessary error log by @SCNieh in #1872
• perf(s3stream): check the logger level before trace and debug by @Chillax-0v0 in #1875
• perf(s3stream/wal): reuse the ByteBuf for record headers by @Chillax-0v0 in #1877
• perf(s3stream/wal): increase the size of the buffer pool as CPU cores by @Chillax-0v0 in #1882
• chore(tools/perf): increase the message sending rate during the warmup to accelerate JVM warmup by @Chillax-0v0 in #1883
• test(metadata/image): thread sync issue by @Pey-crypto in #1879
• fix(test): fix consumer mode e2e by increase timeout by @superhx in #1885
• feat(issue1462): add user case. by @vintagewang in #1889
• fix(e2e): fix bug by @llzcx in #1891
• chore(logger): change e2e streams log level to INFO by @superhx in #1892
• perf(s3stream): optimize the critical section in s3 wal by @ShadowySpirits in #1880
• test(e2e): version upgrade test by @llzcx in #1822
• fix(s3stream/wal): fix memory leak by @Chillax-0v0 in #1896
• fix(e2e): fix data map format by @mooc9988 in #1898
• fix(metadata): fix sparse index leak on compaction split by @SCNieh in #1894
• fix(metadata): fix evict sparse index cache potential endless loop by @SCNieh in #1900
• feat(metadata): add metrics for sparse index cache monitor by @SCNieh in #1902
• fix(metadata): prevent concurrent uploading of local sparse index cache by @SCNieh in #1904
• fix(metadata): fix wrong stream number in serialization by @SCNieh in #1906
• feat(s3stream): add docs for checksumAlgorithm configuration by @ShadowySpirits in #1908
• fix(metadata): fix interrupted batch upload by @SCNieh in #1913
• perf(config): increase the default index interval from 4KiB to 1MiB by @Chillax-0v0 in #1915
• chore: add nightly release for automq by @daniel-y in #1916
• chore: polish the nightly release by @daniel-y in #1917
• chore: make the nightly release as a pre release by @daniel-y in #1918
• feat(s3stream): unify object not exist exception by @superhx in #1920
• fix(e2e): fix bug by @llzcx in #1922
• feat(s3stream): limit the inflight fast retry request count by @ShadowySpirits in #1919
• feat(eventLoop): the visibility of the shutdown flag for the event loop and optimizations for graceful shutdown by @CLFutureX in #1796
• chore(test): set runner to e2e by @superhx in #1929
• chore(telemetry): set the endpoint of otlp receiver manually by @Chillax-0v0 in #1927
• feat(metadata): expire node sparse index cache after write by @SCNieh in #1925
• feat(auto_balancer): limit the max number of nodes to reassign partit… by @SCNieh in #1933
• feat(metadata): blocking wait for uploading local index cache on stre… by @SCNieh in #1931
• feat(readme): add github trending logo by @vintagewang in #1936
• feat(s3stream): add checksum for s3 wal by @ShadowySpirits in #1939
• feat(s3stream): ensure compaction could be executed (#1935) by @superhx in #1937
• docs(s3stream/version): statement of features supported by S3Stream V1 by @Chillax-0v0 in #1943
• fix(s3stream): resolve unintended buffer reuse by @ShadowySpirits in #1945
• feat(s3stream): make s3client configuration can be override by @warr99 in #1947
• fix(s3stream): fix buffer leak by @ShadowySpirits in #1948
• fix(bufAlloc): adjust the default size by @CLFutureX in #1955
• perf(core): disable DeleteProduce tryComplete trace log by @lifepuzzlefun in #1951
• fix(s3stream): change reject handler to prevent task lost by @SCNieh in #1956
• feat(s3stream): change append callback thread number by @lifepuzzlefun in #1950
• feat(s3stream): support limit write bandwidth for WALBlockDeviceChannel by @lifepuzzlefun in #1941
• feat(readme): update aws marketplace url(#1936) by @vintagewang in #1958
• fix(issues1960): allow UNSET attributes for version 1.1 by @superhx in #1962
• build(dependencies): update Jackson to 2.17.1 by @warr99 in #1963
• feat(s3stream): add tail read throttle strategy by @SCNieh in #1965
• feat(wal): unOverCapacityException will lead to memory leaks by @CLFutureX in #1952
• perf(core): optimize FetchSession.update high memory allocate rate by @lifepuzzlefun in #1966
• perf(s3stream): optimize ByteBufAlloc.USAGE_STATS high cpu usage by @lifepuzzlefun in #1959
• fix(s3stream/wal): record metrics on WAL full by @Chillax-0v0 in #1968
• feat(s3stream): optimize recovery read throttle priority by @SCNieh in #1967
• Update README.md with new videos by @KaimingWan in #1969
• fix(log): use the same view to calculate trim offsets by @Chillax-0v0 in https://github.com...

1.2.1

What's Changed

• feat(s3stream): add read timeout by @superhx in #2059
• fix(scripts): start controller node in daemon mode by @Chillax-0v0 in #2064
• fix(scripts): fix typo by @Chillax-0v0 in #2067
• fix(scripts): fix "advertised.listeners" when deploying cluster by @Chillax-0v0 in #2069
• fix(scripts): add single quotes around the env values by @Chillax-0v0 in #2071
• fix(config): update example config "listeners" for brokers by @Chillax-0v0 in #2073
• perf(kraft): decrease the index interval bytes of KRaft Log from 1MB to 4KB by @Chillax-0v0 in #2079
• feat(version): bump version by @superhx in #2083
• chore: remove log and add topic check method (#2086) by @Gezi-lzq in #2088
• feat(version): bump version to 1.2.1-rc1 by @superhx in #2090
• fix(core): fix time unit by @SCNieh in #2091
• fix(sasl): fix sasl configs by @Chillax-0v0 in #2096
• chore: bump version to 1.2.1-rc2 by @Chillax-0v0 in #2099
• fix(auto_balancer): fix mistakenly reused reference by @SCNieh in #2103
• fix(issue2108): avoid blocking at the end of a compaction iteration when there are un-uploaded data by @SCNieh in #2110
• fix(s3stream): wait force upload complete before return by @superhx in #2114
• fix(compaction): prevent double release on compaction shutdown by @SCNieh in #2117
• fix(issue2139): prevent read object info from closed ObjectReader by @SCNieh in #2142
• fix(issue2139): add computeIfAbsent atomic operation to AsyncLRUCache by @SCNieh in #2146
• fix(issue2140): remove override equals and hashCode method for Object… by @SCNieh in #2149
• feat(version): bump version to 1.2.1 by @superhx in #2154

Full Changelog: 1.2.0...1.2.1

nightly

Automated nightly release, will be updated every day at 20:00 UTC

1.2.1-rc2

What's Changed

• fix(core): fix time unit by @SCNieh in #2091
• fix(sasl): fix sasl configs by @Chillax-0v0 in #2096
• chore: bump version to 1.2.1-rc2 by @Chillax-0v0 in #2099

Full Changelog: 1.2.1-rc1...1.2.1-rc2

1.2.1-rc1

What's Changed

• chore: remove log and add topic check method (#2086) by @Gezi-lzq in #2088
• feat(version): bump version to 1.2.1-rc1 by @superhx in #2090

Full Changelog: 1.2.1-rc0...1.2.1-rc1

1.2.1-rc0

What's Changed

• feat(s3stream): add read timeout by @superhx in #2059
• fix(scripts): start controller node in daemon mode by @Chillax-0v0 in #2064
• fix(scripts): fix typo by @Chillax-0v0 in #2067
• fix(scripts): fix "advertised.listeners" when deploying cluster by @Chillax-0v0 in #2069
• fix(scripts): add single quotes around the env values by @Chillax-0v0 in #2071
• fix(config): update example config "listeners" for brokers by @Chillax-0v0 in #2073
• perf(kraft): decrease the index interval bytes of KRaft Log from 1MB to 4KB by @Chillax-0v0 in #2079
• feat(version): bump version by @superhx in #2083

Full Changelog: 1.2.0...1.2.1-rc0

1.1.3

What's Changed

• fix(issues1999): fix Processor.channelContexts memory leak by @superhx in #2001
• fix(issue2004): fix AutoBalancerMetricsReporter cannot process with t… by @SCNieh in #2007
• fix(core): release all records before delayed fetch by @ShadowySpirits in #2010
• feat(metrics): switch to stable jvm semconv by @SCNieh in #2011
• perf(DelayedFetch): only try to fast read on complete a delayed fetch by @Chillax-0v0 in #2012
• feat(version): bump version to 1.1.3-rc0 by @superhx in #2015
• feat(core): revert rack aware assignment on broker fence by @SCNieh in #2016
• fix(core): rsp immediately for catch-up read even if rst is not enough by @SCNieh in #2020
• fix(core): remove offset metrics when expired by @SCNieh in #2021
• feat(version): bump version to 1.1.3-rc1 by @superhx in #2024
• feat(version): bump version to 1.1.3 by @superhx in #2063

Full Changelog: 1.1.2...1.1.3

1.2.0

What's Changed

• docs(metrics): refine telemetry README and remove unused components by @SCNieh in #1333
• fix(issues1334): fix stream reader endless looping by @superhx in #1335
• fix(perf): increase request timeout of the admin client by @Chillax-0v0 in #1336
• feat(core): forward stream apis by @ShadowySpirits in #1338
• chore: set code owners for the whole repo by @Chillax-0v0 in #1340
• feat(core): add command config by @ShadowySpirits in #1342
• fix(auto_balancer): fix broker status change by @SCNieh in #1339
• feat(metadata): add missing tags for the old stream by @superhx in #1346
• fix(s3stream): refine network rate limiter for better precision by @SCNieh in #1344
• fix(s3stream): fix variable typo by @yx9o in #1351
• docs(telemetry): simplify grafana dashboard templates by @SCNieh in #1354
• feat(shell): remove automq cli by @ShadowySpirits in #1355
• feat(core): log the block operation in controller event loop by @ShadowySpirits in #1352
• fix(s3stream): temporarily disable percentile metrics for performance issue by @SCNieh in #1353
• perf(s3stream/allocator): increase trunk size of the netty allocator by @Chillax-0v0 in #1364
• fix(issues1357): fix missing deletes when load from snapshot by @superhx in #1358
• chore(log): fix log by @superhx in #1365
• fix(s3stream): fix mismatched placeholders by @yx9o in #1369
• feat(shell): add basic command for automq cli by @ShadowySpirits in #1370
• fix(issues1367): stream reader prevent adding invalid blocks by @superhx in #1372
• chore(s3stream): add toString methods to fix logs by @Chillax-0v0 in #1381
• fix(cache): fix readhead fail when user read fast than readahead by @superhx in #1382
• fix(core): add tolerance delay when append with non-zero seq number with empty producer snapshot (#1377) by @SCNieh in #1384
• refactor(s3stream): remove DefaultS3Client's dependency on KafkaConfig by @Chillax-0v0 in #1388
• fix(issues604): fix response size not match by @superhx in #1392
• feat(tools/perf): adjust send rate during warmup and catchup by @Chillax-0v0 in #1395
• feat(s3stream): add new object storage abstract by @superhx in #1396
• fix(action): fix release action (#1398) by @superhx in #1399
• feat(core): optimize the block operation log in controller event loop by @ShadowySpirits in #1403
• fix(auto_balancer): remove unnecessary map copy by @SCNieh in #1404
• feat(core): make controller builder extensible by @SCNieh in #1406
• perf(s3stream): reduce lock granularity in StreamMetadataManager by @lifepuzzlefun in #1411
• fix(TimeIndex): wait for the last append to finish before fetching the stream by @Chillax-0v0 in #1407
• feat(auto_balancer): opt cluster model interface by @SCNieh in #1414
• chore(EventQueue): increase EVENT_PROCESS_TIME_THRESHOLD_MICROSECOND to 5ms by @Chillax-0v0 in #1412
• fix(cache): fix cache reuse for re-created topic by @superhx in #1417
• perf(metadata): use Timeline for objects image by @superhx in #1419
• feat(s3stream): add aws s3 rangeRead implementation for object storage by @warr99 in #1416
• fix(metadata): fix metadata access outdated image by @superhx in #1420
• fix(core): fix unexpected truncation of file cache by @ShadowySpirits in #1421
• feat(core): report s3 object number only on active controller by @SCNieh in #1424
• perf(FileCache): avoid evicting too many blocks at once by @Chillax-0v0 in #1425
• feat: merge from apache kafka 3.8 0971924 by @superhx in #1427
• fix(docs): fix contributing guide by @jitokim in #1428
• chore(examples): remove hardcoded value in TransactionProducer by @jitokim in #1426
• fix(config): fix default config for auto balancer listener name by @SCNieh in #1429
• fix(tools/perf): discard invalid latency values by @Chillax-0v0 in #1430
• test(s3stream): add objectStorage rangeRead test by @warr99 in #1423
• feat(s3stream): add aws s3 write implementation for object storage by @warr99 in #1434
• fix(auto_balancer): fix metrics compatibility for version V0 by @SCNieh in #1433
• fix(auto_balancer): match all endpoints when listener name is not spe… by @SCNieh in #1435
• feat(s3stream): composite object reader&writer by @superhx in #1432
• feat(s3stream): add s3 multipart implementation for object storage by @warr99 in #1438
• feat(s3stream): composite object add object attributes by @superhx in #1437
• fix(s3stream/wal): check the block size before start by @Chillax-0v0 in #1439
• feat(tools): fix StorageTool by @ShadowySpirits in #1442
• feat(s3stream): compact to composite object by @superhx in #1443
• feat(s3stream): add composite object deletion by @superhx in #1444
• feat(s3stream): add s3 delete implementation for object storage by @warr99 in #1441
• perf(s3stream): Avoid non-reuse TimerUtil object creation on write path by @lifepuzzlefun in #1447
• feat(s3stream): object storage writer by @warr99 in #1445
• feat(s3stream): add stream object compaction v1 trigger by @superhx in #1448
• feat(issue1134): prevent trivial scheduling by @SCNieh in #1449
• feat(auto_balancer): move internal topic partition only when necessary by @SCNieh in #1450
• feat(s3stream): adapt to new object storage abstraction by @superhx in #1451
• feat(core): implement broker and cluster level quota by @ShadowySpirits in #1440
• fix: stream e2e by @superhx in #1453
• feat: add composite object test by @superhx in #1454
• feat(s3stream): integrate composite object by @superhx in #1456
• fix(test): fix e2e by @superhx in #1457
• test(s3stream): add test case to verify end-of-object read behavior by @warr99 in #1455
• feat(auto_balancer): refactor cluster load snapshot for better extens… by @SCNieh in #1459
• chore: update README.md by @wensongz in #1460
• refactor(license): organize license files together by @KaimingWan in #1458
• chore: add a readme doc for s3stream by @daniel-y in #1461
• feat(issue1462): Optimize the Readme. #1462 by @vintagewang in #1463
• feat(s3stream): implement a prototype for the S3 Write-Ahead Log by @ShadowySpirits in #1465
• feat(s3stream): remove dependency on stream range offset by @SCNieh in #1466
• feat(metadata): optimize streamsetobject to support huge cluster by @superhx in #1464
• fix(test): fix connect e2e task timeout by @superhx in #1467
• fix(test): skip unsupported test by @superhx in #1468
• fix(s3stream): ensure wal header shutdown type UNGRACEFULLY when startup by @lifepuzzlefun in #1469
• chore(all): adapt to java21 by @superhx in #1471
• feat(s3stream): implement AcknowledgmentService by @ShadowySpirits in #1470
• feat(core): add extension point for TelemetryManager by @SCNieh in #1472
• fix(auto_balancer): fix broker status by @SCNieh in #1480
• fix(auto_balancer): remove partition from cluster model when it has n… ...

1.2.0-rc1

What's Changed

• feat(core): revert rack aware assignment on broker fence by @SCNieh in #2017
• fix(core): rsp immediately for catch-up read even if rst is not enough by @SCNieh in #2019
• fix(core): remove offset metrics when expired by @SCNieh in #2022
• feat(version): bump version to 1.2.0-rc1 by @superhx in #2025

Full Changelog: 1.2.0-rc0...1.2.0-rc1

1.1.3-rc1

What's Changed

• feat(core): revert rack aware assignment on broker fence by @SCNieh in #2016
• fix(core): rsp immediately for catch-up read even if rst is not enough by @SCNieh in #2020
• fix(core): remove offset metrics when expired by @SCNieh in #2021
• feat(version): bump version to 1.1.3-rc1 by @superhx in #2024

Full Changelog: 1.1.3-rc0...1.1.3-rc1