Skip to content

Conversation

@vbhavsar
Copy link

@vbhavsar vbhavsar commented May 12, 2025

Changes

  • Introduce a new rack_id parameter on AIOKafkaConsumer and store it on the instance.
  • Propagate rack_id (alongside new session metadata fields session_id, session_epoch and forgotten_topics_data) into the internal Fetcher.
  • Revise fetch-request version logic:
    • For kafka 2.4.0+ clients, bump to FetchRequest V11 and include the consumer's rack_id in the payload
    • Fall back to the existing protocal branches (V7 and V4) for older API versions

Fixes #1016

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes
  • Add a new news fragment into the CHANGES folder
    • name it <issue_id>.<type> (e.g. 588.bugfix)
    • if you don't have an issue_id change it to the pr id after creating the PR
    • ensure type is one of the following:
      • .feature: Signifying a new feature.
      • .bugfix: Signifying a bug fix.
      • .doc: Signifying a documentation improvement.
      • .removal: Signifying a deprecation or removal of public API.
      • .misc: A ticket has been closed, but it is not of interest to users.
    • Make sure to use full sentences with correct case and punctuation, for example: Fix issue with non-ascii contents in doctest text files.

@vbhavsar
Copy link
Author

image

Wireshark capture of FetchRequest V11 packet. It includes the new rack_id field serialized.

Comment on lines +402 to +405
session_id=0,
session_epoch=-1,
forgotten_topics_data=None,
rack_id="",
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vbhavsar vbhavsar changed the title Vishal/support client.rack 2 Add support for rack id in the consumer May 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add client.rack configuration option to support consumer fetching from closest replica (KIP-392)

2 participants