Per Partition Automatic Failover: Adds Hub Region Processing Only While Routing Requests Failed with 404/1002. #5447
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
aavasthy
requested review from
FabianMeiswinkel,
Pilchie,
adityasa,
khdang,
kirankumarkolli,
kirillg,
neildsh and
sboshra
as code owners
aavasthy
changed the title
| && subStatusCode == SubStatusCodes.ReadSessionNotAvailable) | ||
| { | ||
| { | ||
| this.addHubRegionProcessingOnlyHeader = true; |
There was a problem hiding this comment.
The flag for addHubRegionProcessingOnlyHeader is set for all the instances of 404/1002(Read session not found) and consequently we would set the header for every 404/1002 retry, is this expected?
| } | ||
|
|
||
| if (statusCode == HttpStatusCode.NotFound | ||
| && subStatusCode == SubStatusCodes.ReadSessionNotAvailable) |
There was a problem hiding this comment.
also double check: does this change also targeted for MM as well? For MM, writes can happen in any region, also enable this for MM might cause regression
There was a problem hiding this comment.
confirmed this with backend team and this change is not intended to be used in multi-master.
| if (this.addHubRegionProcessingOnlyHeader) | ||
| { | ||
| request.Headers[HubRegionHeader] = bool.TrueString; | ||
| this.addHubRegionProcessingOnlyHeader = false; // reset after applying |
There was a problem hiding this comment.
what would be the errors returned if SDK try to read from non-hub region?
There was a problem hiding this comment.
Also falling back to new hub for that partition.
Pull Request Template
Description
During partition-level failover and failback under session consistency, a timing gap can cause read requests to fail with 404/1002 errors. When a partition temporarily fails over to a secondary region and later begins failing back to the primary region, the SDK’s read circuit breaker (PPCB) may start routing reads back to the primary region before it has fully caught up with the writes from the failover region. As a result, reads using session tokens from the previous write region may fail because the primary region does not yet have the corresponding session state. Since the SDK currently does not perform cross-regional retries for 404/1002 responses, these reads continue to fail until the primary region is fully synchronized. The goal is to leverage the new backend header x-ms-cosmos-hub-region-processing-only to detect such conditions and route retry requests to the correct write (hub) region, ensuring successful session-consistent reads during the failback window.
Type of change
Please delete options that are not relevant.
Closing issues
To automatically close an issue: closes #5440