Skip to content

feat: reactive character scheduling (CCP best practice) #675

@herpaderpaldent

Description

@herpaderpaldent

Problem

The current scheduler uses a * * * * * cron job (UpdateCharacter) that fires every minute and distributes character batch jobs across the cron interval window. This has two problems:

  1. Fixed-interval polling — the cron fires regardless of how many jobs the queue is still processing. Under load this causes overlapping batches and queue flooding.
  2. No staggering — all characters seeded in the same cron tick are dispatched at once, creating a burst. CCP recommends staggered requests, not periodic bursts.

CCP Recommendation

Use staggered scheduling for periodic requests when possible. Ideally not */5 cronjobs but rather: 5 minutes after the last job finished.

Target Behaviour

Instead of a wall-clock cron driving all character updates, each character self-reschedules its own next update after the current batch finishes:

App boot / hourly catchup → UpdateCharacter (bootstrapper only)
    → dispatches chars with no BatchUpdate OR stale finished_at
    → stagger initial dispatches with random delay

CharacterBatchJob::finally() (scheduled dispatches only)
    → marks finished_at
    → dispatches CharacterBatchJob::dispatch($id, $queue, reschedule: true)->delay(REFRESH_DELAY_MINUTES)
        ↑ self-rescheduling — no cron needed for steady-state

This keeps a character-complete guarantee: all endpoints for one character are fully processed before the next cycle starts. Under load, characters naturally spread out rather than bunching up.

Implementation Steps

1. Add $reschedule flag to CharacterBatchJob

Add bool $reschedule = false as a constructor parameter. Only the scheduled dispatcher (UpdateCharacter::updateNextIncrementOfCharacters) passes true. All manual dispatches leave it as false.

Rationale: A manual dispatch (new token login, recruiter "refresh now", scope change) should run once and stop — it must not inject itself into the automatic rotation.

Caller $reschedule
UpdateCharacter::updateNextIncrementOfCharacters() true
UpdateCharacter::updateSingleCharacter() false (default)
ReactOnFreshRefreshToken false (default)
UpdatingRefreshTokenListener false (default)
Recruiter "refresh now" (web package) false (default)

2. Self-reschedule in CharacterBatchJob::finally()

Add REFRESH_DELAY_MINUTES = 5 (ESI minimum cache TTL). In the finally() callback:

if ($this->reschedule) {
    CharacterBatchJob::dispatch($update->batchable_id, $this->queue, reschedule: true)
        ->delay(now()->addMinutes(self::REFRESH_DELAY_MINUTES));
}
BatchUpdate::where('batch_id', $batch->id)->update(['finished_at' => now()]);

3. Store queue on BatchUpdate

Add a queue column (string, default 'default') to the batch_updates table so finally() knows which queue to re-dispatch on.

4. UpdateCharacter becomes a bootstrap/catchup job

Change dispatch logic: only dispatch characters that:

  • Have no BatchUpdate record at all (never started), OR
  • finished_at < now() - 2 × REFRESH_DELAY_MINUTES AND not currently pending (recovery for crashed workers)

Stagger initial dispatches: ->delay(random_int(0, $i) * 2) seconds.

Change schedule seeder default: * * * * **/30 * * * * (catchup / recovery every 30 min).

5. Simplify shouldDiscardUpdate

With self-scheduling, a job arrives only when it's supposed to. Simplify the guard: discard only if is_pending. Remove the isSameHour heuristic — it was a workaround for the fixed-interval cron approach.

Out of Scope

Character/corporation priority configuration is tracked in #676.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions