add simple queue #543
add simple queue #543
Conversation
WalkthroughThis change refactors the queuing mechanism for evaluating release targets throughout the codebase. Bulk queue operations using Changes
Sequence Diagram(s)sequenceDiagram
participant Caller as Any Module
participant Events as queueEvaluateReleaseTarget
participant Queue as EvaluateReleaseTarget Queue
Caller->>Events: queueEvaluateReleaseTarget(releaseTarget)
Events->>Queue: get waiting jobs
Events->>Events: Check for duplicate job
alt Duplicate exists
Events-->>Caller: Return (do not enqueue)
else No duplicate
Events->>Queue: Add job with releaseTarget data
Events-->>Caller: Return Job
end
Poem
📜 Recent review detailsConfiguration used: .coderabbit.yaml 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms (4)
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
packages/events/src/index.ts (1)
41-69: New deduplication queue function added with potential improvement areas.The new
queueEvaluateReleaseTargetfunction provides a helpful mechanism to prevent duplicate jobs in the queue by checking for existing jobs with matching properties before adding a new one.While the implementation is generally solid, there are a few considerations:
- The deduplication only checks "waiting" jobs, but not "active" or "delayed" jobs, which could still lead to duplicates in certain scenarios.
- There's no indication to the caller when a duplicate is found (like a boolean return value or logging).
- Fetching all waiting jobs with
getWaiting()might be inefficient with large queues.Consider these improvements:
export const queueEvaluateReleaseTarget = async ( value: ChannelMap[Channel.EvaluateReleaseTarget], ) => { const q = getQueue(Channel.EvaluateReleaseTarget); - const exists = - (await q.getWaiting()).filter((t) => + // Check both waiting and active jobs for duplicates + const waitingJobs = await q.getWaiting(); + const activeJobs = await q.getActive(); + const allJobs = [...waitingJobs, ...activeJobs]; + + const exists = allJobs.some((t) => _.isEqual( _.pick(value, [ "environmentId", "resourceId", "deploymentId", "skipDuplicateCheck", ]), _.pick(t.data, [ "environmentId", "resourceId", "deploymentId", "skipDuplicateCheck", ]), ), - ).length > 0; + ); - if (exists) return; + if (exists) { + logger.debug('Skipped duplicate job', { + channel: Channel.EvaluateReleaseTarget, + job: `${value.environmentId}-${value.resourceId}-${value.deploymentId}` + }); + return false; + } - return q.add( + await q.add( `${value.environmentId}-${value.resourceId}-${value.deploymentId}`, value, ); + return true; };
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
apps/event-worker/src/utils/dispatch-evaluate-jobs.ts(1 hunks)apps/event-worker/src/workers/evaluate-release-target.ts(2 hunks)apps/jobs/src/policy-checker/index.ts(0 hunks)packages/api/src/router/deployment-version-checks/approvals.ts(1 hunks)packages/api/src/router/redeploy.ts(2 hunks)packages/events/src/index.ts(2 hunks)
💤 Files with no reviewable changes (1)
- apps/jobs/src/policy-checker/index.ts
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.{ts,tsx}`: **Note on Error Handling:** Avoid strict enforcement of try/catch blocks. Code may use early returns, Promise chains (.then().catch()), or other patterns for error...
**/*.{ts,tsx}: Note on Error Handling:
Avoid strict enforcement of try/catch blocks. Code may use early returns, Promise chains (.then().catch()), or other patterns for error handling. These are acceptable as long as they maintain clarity and predictability.
packages/api/src/router/redeploy.tspackages/api/src/router/deployment-version-checks/approvals.tsapps/event-worker/src/workers/evaluate-release-target.tspackages/events/src/index.tsapps/event-worker/src/utils/dispatch-evaluate-jobs.ts
🧬 Code Graph Analysis (5)
packages/api/src/router/redeploy.ts (2)
packages/events/src/index.ts (1)
queueEvaluateReleaseTarget(41-69)packages/db/src/schema/release.ts (1)
releaseTarget(20-42)
packages/api/src/router/deployment-version-checks/approvals.ts (1)
packages/events/src/index.ts (1)
queueEvaluateReleaseTarget(41-69)
apps/event-worker/src/workers/evaluate-release-target.ts (2)
packages/events/src/index.ts (1)
queueEvaluateReleaseTarget(41-69)packages/db/src/schema/job.ts (1)
job(75-107)
packages/events/src/index.ts (1)
packages/events/src/types.ts (1)
ChannelMap(47-79)
apps/event-worker/src/utils/dispatch-evaluate-jobs.ts (2)
packages/rule-engine/src/types.ts (1)
ReleaseTargetIdentifier(74-78)packages/events/src/index.ts (1)
queueEvaluateReleaseTarget(41-69)
⏰ Context from checks skipped due to timeout of 90000ms (4)
- GitHub Check: build (linux/amd64)
- GitHub Check: Typecheck
- GitHub Check: Lint
- GitHub Check: build (linux/amd64)
🔇 Additional comments (9)
packages/api/src/router/deployment-version-checks/approvals.ts (1)
115-116: Change looks good: sequential queue evaluation with duplicate prevention.The updated code now uses the new
queueEvaluateReleaseTargetfunction to process each release target individually. This approach provides built-in duplicate job detection and prevention, which improves system efficiency by avoiding redundant evaluations.packages/api/src/router/redeploy.ts (2)
9-13: Import structure updated correctly to include new function.The import statement has been properly expanded to include the new
queueEvaluateReleaseTargetfunction alongside existing imports. The naming and organization are consistent.
65-65: Good refactor: now using centralized queue function with duplicate prevention.Replaced direct queue manipulation with the new
queueEvaluateReleaseTargetfunction. This change improves code maintainability and adds automatic deduplication of evaluation jobs.apps/event-worker/src/workers/evaluate-release-target.ts (2)
8-13: Import structure updated correctly to include new function.The import statement has been properly updated to include the
queueEvaluateReleaseTargetfunction. The organization with named imports is clean and maintainable.
185-185: Improved error handling with consistent queue approach.The error handling now uses
queueEvaluateReleaseTargetto retry jobs when encountering row lock or uncommitted target errors. This is more consistent with the rest of the codebase and benefits from the built-in deduplication checks.apps/event-worker/src/utils/dispatch-evaluate-jobs.ts (2)
3-7: Import structure updated correctly to include new function.The import statement has been properly updated to include the
queueEvaluateReleaseTargetfunction. The named imports format is clean and consistent.
10-12: Replaced bulk queue with sequential processing for better reliability.Changed from bulk queue operations to sequential processing with the new
queueEvaluateReleaseTargetfunction. This approach provides:
- Built-in deduplication of jobs
- More explicit control flow with awaited operations
- Consistent error handling
Note that for very large batches, sequential processing could be slower than bulk operations, but the deduplication benefit likely outweighs this consideration in most cases.
packages/events/src/index.ts (2)
4-4: New lodash dependency added.The addition of lodash is appropriate for the deep equality comparison needed in the new queue function.
10-10: Import of Channel enum from types module.This import is necessary for the new function to access the Channel enum value.
Summary by CodeRabbit
New Features
Refactor
Chores