KAFKA-18575: Transaction Version 2 doesn't correctly handle race condition with completing and new transaction #18604
+43
−2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @@ -1052,7 +1057,7 @@ class UnifiedLog(@volatile var logStartOffset: Long, | |||
| } | |||
|
|
|||
| private def batchMissingRequiredVerification(batch: MutableRecordBatch, requestVerificationGuard: VerificationGuard): Boolean = { | |||
| producerStateManager.producerStateManagerConfig().transactionVerificationEnabled() && !batch.isControlBatch && | |||
| producerStateManager.producerStateManagerConfig().transactionVerificationEnabled() && | |||
There was a problem hiding this comment.
What's the reason to move the check for control batch out of this function?
There was a problem hiding this comment.
I moved the check to outside the if condition (short-circuiting the stament before hasOngoingTransaction otherwise we can't write commit markers)
I can put this back, but it will just be a duplicate check.
There was a problem hiding this comment.
No, this is good. I'm just thinking if having this function makes the code better or worse -- it's really called exactly from one place and looks like the order of checks is important and can lead to subtle bugs.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There is a subtle race condition with transactions V2 if a transaction is still completing when checking if we need to add a partition, but it completes when the request reaches the coordinator.
One approach was to remove the verification for TV2 and just check the epoch on write, but a simpler one is to simply return concurrent transactions from the partition leader (before attempting to add the partition). I've done this and added a test for this behavior.
Locally, I reproduced the race but adding a 1 second sleep when handling the WriteTxnMarkersRequest and a 2 second delay before adding the partition to the AddPartitionsToTxnManager. Without this change, the race happened on every second transaction as the first one completed. With this change, the error went away.
As a followup, we may want to clean up some of the code and comments with respect to verification as the code is used by both TV0 + verification and TV2. But that doesn't need to complete for 4.0. This does :)