Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix region aware placement policy disk weight dose not update. #4522

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

dragonls
Copy link

Motivation

We are using region aware policy in our Pulsar cluster and enable the disk weight base placement feature.

ensemblePlacementPolicy=org.apache.bookkeeper.client.RegionAwareEnsemblePlacementPolicy
bookkeeperDiskWeightBasedPlacementEnabled=false

There are different CVMs with different free disk space in a region, but we found that the traffic still seems to be balanced and is not distributed according to the weight of the free disk space.

The root cause of this case is that the disk weight information actually used to allocate bookies in RegionAwareEnsemblePlacementPolicy has not been updated at all.

The RegionAwareEnsemblePlacementPolicy actually choose bookies for ensemble by RackawareEnsemblePlacementPolicy of each region in perRegionPlacement, see org.apache.bookkeeper.client.RegionAwareEnsemblePlacementPolicy#newEnsemble. RegionAwareEnsemblePlacementPolicy.updateBookieInfo dose not update the bookieInfoMap in RackawareEnsemblePlacementPolicy of each region in perRegionPlacement, which results that the disk weight information actually used to allocate bookies in RegionAwareEnsemblePlacementPolicy has not been updated at all.

Changes

  1. Overwrites the updateBookieInfo in RegionAwareEnsemblePlacementPolicy, update the bookie info in RackawareEnsemblePlacementPolicy of each region in perRegionPlacement.
  2. Add test cases org.apache.bookkeeper.client.TestRegionAwareEnsemblePlacementPolicy#testRegionsWithDifferentDiskWeight.

@dragonls dragonls force-pushed the fix_region_aware_placement_policy_disk_weight_does_not_update branch from da24cb8 to f4901c7 Compare November 11, 2024 09:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant