fix: single statement lock-and-fetch unresolved task from first batch…#811
Conversation
| } | ||
|
|
||
| private List<Execution> lockAndFetchSingleStatement(Instant now, int limit) { | ||
| return jdbcRunner.inTransaction( |
There was a problem hiding this comment.
There is some tradeoff to add a transaction instead of one query. I think atomicity is better here — the second query will be executed only in rare unresolved task cases, and the transaction impact will be just the second (for batch) roundtrip.
OptimumCode
left a comment
There was a problem hiding this comment.
@strelchm Thanks for making the PR! We are currently experiencing a problem with false-positive dead executions because the lock-and-fetch strategy does not unpick unresolved tasks (when they were grabbed by an old worker that does not yet have the task definition). And this PR is just what we need!
| partitioningBy( | ||
| execution -> taskResolver.isUnresolved(execution.getTaskName()))); | ||
|
|
||
| List<Execution> unresolvedExecutions = allCandidatesByIsUnresolved.get(true); |
There was a problem hiding this comment.
nitpick: We could use Boolean.TRUE to avoid unnecessary boxing here (and Boolean.FALSE below)
There was a problem hiding this comment.
Good point, thanks! Fixed.
… dead state is fixed
d721c99 to
df851ee
Compare
Added unpicking of unresolved tasks in single‑statement
LOCK_AND_FETCHmodeWhen a task type becomes unresolved (e.g., not registered in the current scheduler instance due to a rolling update where new versions don't have the task yet), the compensation batch query unpicks those tasks so they can be processed by other scheduler instances that have the task type available (this behavior is already built-in for
FETCHand genericLOCK_AND_FETCHmodes out of the box).Implementation Details
Compensation query in single statement
LOCK_AND_FETCHmode:SELECT FOR UPDATE SKIP LOCKEDthat picks a task. The second query unpicks unresolved tasks if any exist. If there are no unresolved executions, the second query is skipped and only the transaction commit is performed.Unpick operation (
unpickPickedBatchrepository method):picked = falsewith version increment).If any of the unpicked tasks is not found or has a mismatched version, the method throws an exception.
Tests Added
Several additional tests are added to
TaskResolverTestfor better coverage.Fixes
#804
Reminders
mvn spotless:applycc @kagkarlsson