Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(dask): introduce labels and node selector to Dask resources (#628) #628

Merged
merged 1 commit into from
Jan 29, 2025

Conversation

Alputer
Copy link
Member

@Alputer Alputer commented Jan 29, 2025

This commit introduces support for adding labels and a node selector to Dask resources, allowing finer control over resource scheduling in K8s. Following changes are introduced:

  • reana-run-dask-owner-uuid and reana-run-dask-workflow-uuid label for Dask scheduler and worker pods in addition to cluster and autoscaler custom K8s resources.

  • runtime_jobs node selector for Dask scheduler and worker pods if node_label_runtimejobs is defined in values.yaml.

Closes #623

@Alputer Alputer requested a review from tiborsimko January 29, 2025 11:15
@Alputer Alputer self-assigned this Jan 29, 2025
Copy link

codecov bot commented Jan 29, 2025

Codecov Report

Attention: Patch coverage is 77.77778% with 2 lines in your changes missing coverage. Please review.

Project coverage is 73.65%. Comparing base (a71ad83) to head (3a8e37e).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
reana_workflow_controller/dask.py 77.77% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #628      +/-   ##
==========================================
- Coverage   73.71%   73.65%   -0.07%     
==========================================
  Files          17       17              
  Lines        1948     1951       +3     
==========================================
+ Hits         1436     1437       +1     
- Misses        512      514       +2     
Files with missing lines Coverage Δ
reana_workflow_controller/dask.py 59.55% <77.77%> (-0.36%) ⬇️

…nahub#628)

This commit introduces support for adding labels and a node selector to
Dask resources, allowing finer control over resource scheduling in K8s.
Following changes are introduced:
- `reana-run-dask-owner-uuid` and `reana-run-dask-workflow-uuid` label
for Dask scheduler and worker pods in addition to cluster and autoscaler
custom K8s resources.
- runtime_jobs node selector for Dask scheduler and worker pods if
`node_label_runtimejobs` is defined in `values.yaml`.

Closes reanahub#623
Copy link
Member

@tiborsimko tiborsimko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works nicely 👍

@tiborsimko tiborsimko merged commit 3a8e37e into reanahub:master Jan 29, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

dask: spawn jobs only to selected cluster nodes
2 participants