Skip to content

feat(api): separate sandbox state crons - #5010

Open
idagelic wants to merge 1 commit into
mainfrom
states-cron
Open

feat(api): separate sandbox state crons#5010
idagelic wants to merge 1 commit into
mainfrom
states-cron

Conversation

@idagelic

@idagelic idagelic commented Jun 11, 2026

Copy link
Copy Markdown
Member

Description

Separates the pending_build cron from the rest for more resilience, also moves the pulling_snapshot state to be processed along with restoring

Documentation

  • This change requires a documentation update
  • I have made corresponding changes to the documentation

Summary by cubic

Separated sandbox state syncing into dedicated crons to reduce contention and improve reliability. PENDING_BUILD now has its own cron, and PULLING_SNAPSHOT is processed with restoring.

  • Refactors
    • New cron for PENDING_BUILD runs every 10s with a global lock and concurrency limit of 10 (batch size 100).
    • RESTORING cron now includes PULLING_SNAPSHOT, runs every 10s, and processes up to 100 items.
    • General pending sync now excludes RESTORING, PULLING_SNAPSHOT, and PENDING_BUILD states.

Written for commit d289248. Summary will update on new commits.

Review in cubic

Signed-off-by: Ivan Dagelic <dagelic.ivan@gmail.com>
Copilot AI review requested due to automatic review settings June 11, 2026 07:28
@nx-cloud

nx-cloud Bot commented Jun 11, 2026

Copy link
Copy Markdown

View your CI Pipeline Execution ↗ for commit d289248

Command Status Duration Result
nx e2e:cleanup daytona-e2e ✅ Succeeded <1s View ↗
nx run-many --target=test:e2e --all --nxBail=true ✅ Succeeded 6m 30s View ↗
nx e2e daytona-e2e ✅ Succeeded 59s View ↗
nx run-many --target=build --projects=api,runne... ✅ Succeeded 5s View ↗

💡 Verify your cache is correct by running tasks in a sandbox. Read docs ↗


☁️ Nx Cloud last updated this comment at 2026-06-11 07:40:25 UTC

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Re-trigger cubic

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors sandbox state syncing in SandboxManager by splitting out PENDING_BUILD into its own cron and moving PULLING_SNAPSHOT handling into the restoring-state cron, aiming to reduce contention and improve resilience of state processing.

Changes:

  • Exclude RESTORING, PULLING_SNAPSHOT, and PENDING_BUILD from the general per-runner pending sync cron.
  • Expand the restoring-state cron to also process PULLING_SNAPSHOT, adjust cadence, and increase batch size.
  • Introduce a dedicated sync-pending-build-states cron with a global lock and concurrency limiting.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 898 to 902
const sandboxes = await this.sandboxRepository.find({
where: {
runnerId,
state: SandboxState.RESTORING,
state: In([SandboxState.RESTORING, SandboxState.PULLING_SNAPSHOT]),
pending: true,
Comment on lines 904 to 906
select: ['id'],
take: 50,
take: 100,
})
Comment on lines +940 to +943
const lockKey = 'sync-pending-build-states'
if (!(await this.redisLockProvider.lock(lockKey, 1 * 60))) {
return
}
@vedranjukic
vedranjukic force-pushed the main branch 2 times, most recently from 2c15a7f to b40f732 Compare June 23, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants