Skip to content

Support offset value v4#851

Merged
bai merged 1 commit into
linkedin:masterfrom
justin-chen:offset-value-v4
Sep 30, 2025
Merged

Support offset value v4#851
bai merged 1 commit into
linkedin:masterfrom
justin-chen:offset-value-v4

Conversation

@justin-chen

Copy link
Copy Markdown
Contributor

Similar to: #472
OffsetCommitValue V4: apache/kafka#19683

For compatibility with Kafka 4.1.0, add decodeOffsetValueV4. Fixes decoding errors such as:

{
  "level": "warn",
  "ts": 1759190908.5072575,
  "msg": "failed to decode",
  "type": "module",
  "coordinator": "consumer",
  "class": "kafka",
  "name": "kafka-staging-us-east1-consumers",
  "offset_topic": "__consumer_offsets",
  "offset_partition": 31,
  "offset_offset": 120351075,
  "message_type": "offset",
  "group": "kafka-client-go-test-staging",
  "topic": "kafka_client_go_test",
  "partition": 1,
  "offset": 4025888141,
  "timestamp": 0,
  "reason": "metadata"
}

By decoding offset value metadata using compact string, we determine string length by reading with uint8 rather than int16, which should be the default:

COMPACT_STRING | Represents a sequence of characters. First the length N + 1 is given as an UNSIGNED_VARINT . Then N bytes follow which are the UTF-8 encoding of the character sequence.

@justin-chen

Copy link
Copy Markdown
Contributor Author

Addressed lint errors by adding constants for the errors.

@bai

bai commented Sep 30, 2025

Copy link
Copy Markdown
Collaborator

You rock! Thanks for your contribution!

@bai bai merged commit f928239 into linkedin:master Sep 30, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants