Problem
The current scheduler uses a * * * * * cron job (UpdateCharacter) that fires every minute and distributes character batch jobs across the cron interval window. This has two problems:
- Fixed-interval polling — the cron fires regardless of how many jobs the queue is still processing. Under load this causes overlapping batches and queue flooding.
- No staggering — all characters seeded in the same cron tick are dispatched at once, creating a burst. CCP recommends staggered requests, not periodic bursts.
CCP Recommendation
Use staggered scheduling for periodic requests when possible. Ideally not */5 cronjobs but rather: 5 minutes after the last job finished.
Target Behaviour
Instead of a wall-clock cron driving all character updates, each character self-reschedules its own next update after the current batch finishes:
App boot / hourly catchup → UpdateCharacter (bootstrapper only)
→ dispatches chars with no BatchUpdate OR stale finished_at
→ stagger initial dispatches with random delay
CharacterBatchJob::finally() (scheduled dispatches only)
→ marks finished_at
→ dispatches CharacterBatchJob::dispatch($id, $queue, reschedule: true)->delay(REFRESH_DELAY_MINUTES)
↑ self-rescheduling — no cron needed for steady-state
This keeps a character-complete guarantee: all endpoints for one character are fully processed before the next cycle starts. Under load, characters naturally spread out rather than bunching up.
Implementation Steps
1. Add $reschedule flag to CharacterBatchJob
Add bool $reschedule = false as a constructor parameter. Only the scheduled dispatcher (UpdateCharacter::updateNextIncrementOfCharacters) passes true. All manual dispatches leave it as false.
Rationale: A manual dispatch (new token login, recruiter "refresh now", scope change) should run once and stop — it must not inject itself into the automatic rotation.
| Caller |
$reschedule |
UpdateCharacter::updateNextIncrementOfCharacters() |
true |
UpdateCharacter::updateSingleCharacter() |
false (default) |
ReactOnFreshRefreshToken |
false (default) |
UpdatingRefreshTokenListener |
false (default) |
| Recruiter "refresh now" (web package) |
false (default) |
2. Self-reschedule in CharacterBatchJob::finally()
Add REFRESH_DELAY_MINUTES = 5 (ESI minimum cache TTL). In the finally() callback:
if ($this->reschedule) {
CharacterBatchJob::dispatch($update->batchable_id, $this->queue, reschedule: true)
->delay(now()->addMinutes(self::REFRESH_DELAY_MINUTES));
}
BatchUpdate::where('batch_id', $batch->id)->update(['finished_at' => now()]);
3. Store queue on BatchUpdate
Add a queue column (string, default 'default') to the batch_updates table so finally() knows which queue to re-dispatch on.
4. UpdateCharacter becomes a bootstrap/catchup job
Change dispatch logic: only dispatch characters that:
- Have no
BatchUpdate record at all (never started), OR
finished_at < now() - 2 × REFRESH_DELAY_MINUTES AND not currently pending (recovery for crashed workers)
Stagger initial dispatches: ->delay(random_int(0, $i) * 2) seconds.
Change schedule seeder default: * * * * * → */30 * * * * (catchup / recovery every 30 min).
5. Simplify shouldDiscardUpdate
With self-scheduling, a job arrives only when it's supposed to. Simplify the guard: discard only if is_pending. Remove the isSameHour heuristic — it was a workaround for the fixed-interval cron approach.
Out of Scope
Character/corporation priority configuration is tracked in #676.
Problem
The current scheduler uses a
* * * * *cron job (UpdateCharacter) that fires every minute and distributes character batch jobs across the cron interval window. This has two problems:CCP Recommendation
Target Behaviour
Instead of a wall-clock cron driving all character updates, each character self-reschedules its own next update after the current batch finishes:
This keeps a character-complete guarantee: all endpoints for one character are fully processed before the next cycle starts. Under load, characters naturally spread out rather than bunching up.
Implementation Steps
1. Add
$rescheduleflag toCharacterBatchJobAdd
bool $reschedule = falseas a constructor parameter. Only the scheduled dispatcher (UpdateCharacter::updateNextIncrementOfCharacters) passestrue. All manual dispatches leave it asfalse.Rationale: A manual dispatch (new token login, recruiter "refresh now", scope change) should run once and stop — it must not inject itself into the automatic rotation.
$rescheduleUpdateCharacter::updateNextIncrementOfCharacters()trueUpdateCharacter::updateSingleCharacter()false(default)ReactOnFreshRefreshTokenfalse(default)UpdatingRefreshTokenListenerfalse(default)false(default)2. Self-reschedule in
CharacterBatchJob::finally()Add
REFRESH_DELAY_MINUTES = 5(ESI minimum cache TTL). In thefinally()callback:3. Store queue on
BatchUpdateAdd a
queuecolumn (string, default'default') to thebatch_updatestable sofinally()knows which queue to re-dispatch on.4.
UpdateCharacterbecomes a bootstrap/catchup jobChange dispatch logic: only dispatch characters that:
BatchUpdaterecord at all (never started), ORfinished_at < now() - 2 × REFRESH_DELAY_MINUTESAND not currently pending (recovery for crashed workers)Stagger initial dispatches:
->delay(random_int(0, $i) * 2)seconds.Change schedule seeder default:
* * * * *→*/30 * * * *(catchup / recovery every 30 min).5. Simplify
shouldDiscardUpdateWith self-scheduling, a job arrives only when it's supposed to. Simplify the guard: discard only if
is_pending. Remove theisSameHourheuristic — it was a workaround for the fixed-interval cron approach.Out of Scope
Character/corporation priority configuration is tracked in #676.