Skip to content

Support splits in SYPD and ASYPD per job#196

Open
LuiggiTenorioK wants to merge 6 commits into
mainfrom
170-splits-sypd
Open

Support splits in SYPD and ASYPD per job#196
LuiggiTenorioK wants to merge 6 commits into
mainfrom
170-splits-sypd

Conversation

@LuiggiTenorioK
Copy link
Copy Markdown
Member

@LuiggiTenorioK LuiggiTenorioK commented May 12, 2025

Closes: #170

This PR adds this information to the jobs in the /v3/tree/{expid}, /v3/graph/{expid}, /v3/pklinfo/{expid}, and /v3/pkltreeinfo/{expid}:

  • The split number of the job
  • The number of splits in the job group
  • Job SYPD
  • Job ASYPD (in case it was supported before)

Then, the SYPD and ASYPD formula is updated to divide the numerator by the number of splits in the job group.

Unfortunately, since Autosubmit didn't support saving the split information in the historical DB, the job and experiment endpoints aren't updated and might have different SYPD/ASYPD values.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 12, 2025

Codecov Report

❌ Patch coverage is 84.68468% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.86%. Comparing base (e4e3d7a) to head (39fce32).

Files with missing lines Patch % Lines
autosubmit_api/history/experiment_history.py 81.63% 9 Missing ⚠️
autosubmit_api/repositories/job_data.py 57.14% 6 Missing ⚠️
autosubmit_api/components/jobs/job_factory.py 84.61% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #196      +/-   ##
==========================================
- Coverage   80.89%   80.86%   -0.03%     
==========================================
  Files         109      109              
  Lines        9514     9545      +31     
==========================================
+ Hits         7696     7719      +23     
- Misses       1818     1826       +8     
Flag Coverage Δ
fast-tests 80.86% <84.68%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@LuiggiTenorioK LuiggiTenorioK self-assigned this May 12, 2025
@LuiggiTenorioK LuiggiTenorioK marked this pull request as ready for review May 13, 2025 07:27
@LuiggiTenorioK LuiggiTenorioK requested a review from Copilot May 13, 2025 07:27
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR updates various endpoints and job processing functions to include and utilize split-related information for SYPD and ASYPD calculations.

  • Introduces split and splits attributes to jobs in pkl files and the database model.
  • Updates performance calculations in utility functions and node representations in tree and graph components.
  • Propagates the new attributes in job creation within the job factory.

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
autosubmit_api/persistance/pkl_reader.py Adds logic to extract split-related attributes from pkl data.
autosubmit_api/performance/utils.py Updates SYPD/ASYPD calculations to account for splits and fixes a shebang typo.
autosubmit_api/experiment/common_requests.py Passes the job.splits parameter during pkl data processing.
autosubmit_api/database/models.py Extends the job model with split and splits attributes.
autosubmit_api/components/representations/tree/tree.py Incorporates split/splits in node data and performance calculations.
autosubmit_api/components/representations/graph/graph.py Adjusts performance function calls and node data to include split-related fields.
autosubmit_api/components/jobs/job_factory.py Maps split attributes from the pkl model to runtime job objects.
Comments suppressed due to low confidence (1)

autosubmit_api/performance/utils.py:1

  • There is a typo in the shebang; please change 'pytthon' to 'python'.
#!/usr/bin/env pytthon

@kinow
Copy link
Copy Markdown
Member

kinow commented Dec 9, 2025

@LuiggiTenorioK is this the PR you mentioned in the last meeting for me to have a look/review?

@LuiggiTenorioK
Copy link
Copy Markdown
Member Author

@LuiggiTenorioK is this the PR you mentioned in the last meeting for me to have a look/review?

Hi @kinow. No, this is partially blocked until Autosubmit 4.2.0.

The one that is pending for review is BSC-ES/autosubmit-gui#376 with #239 to enable the set status command in the BSC GUI.

@LuiggiTenorioK LuiggiTenorioK moved this from Review to In Progress in Autosubmit project Apr 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

SYPD and ASYPD incorrect values for SPLITs

4 participants