SYPD and ASYPD incorrect values for SPLITs

[mcastril]

I have found that the SPLITs report SYPD and ASYPD as if each SPLIT was responsible for the same workflow as its CHUNK

For example, in a workflow with SPLITs inside a CHUNK with MONTH as a unit and 1 as length (monthly chunks), when these SPLITs take 24 seconds the given SYPD is 300. This corresponds to (3600*24)/(12*24), where 3600*24 is the number of seconds in one day, and 12*24 is the time that a job running for 24 seconds and computing one month would take to compute one year.

I think that we could agree in general that by using SPLITs the workload for a CHUNK is equally distributed across its SPLITS, and therefore the SYs of a SPLIT should be SY_CHUNK/N_SPLITS. For example, in the workflow above, if the CHUNK computes one month of 30 days and we have 30 SPLITs, each SPLITs SY should be ((3600/12)/30), which roughly corresponds to one day.

Then, all the other metrics like SYPD, ASPYD and CHPSY should be automatically adjusted.

@LuiggiTenorioK @kinow @dbeltrankyl @franra9

[mcastril]

To give more information, I think the problem is that the GUI shows (and counts) the Start Date and End Date of the CHUNK instead of the SPLIT

This is making the operators have to manually calculate the ASYPD of the OPA and APP jobs in DE_340. Could it be fixed in the next release?

CC @ainagaya

[LuiggiTenorioK]

In general, the SYPD takes all the run times of the SIM jobs without knowing if it is a CHUNK or a SPLIT. If the SPLIT job is properly recognized to belong to the SIM section, this run time will be taken into the denominator sum in the formula. Otherwise, what could be miscalculated is the sum of the SY (which only depends on the chunk unit and size) used in the numerator of the formula. So, taking into account the right jobs in that SY sum could fix this. This case should be the same with the other metrics.

Can anyone bring a real example with the expected metric so we can validate that this could be a solution? @ainagaya @franra9

Additionally, @dbeltrankyl do we have information that a job is a SPLIT in the DB? Or is it something that is only visible in the pkl in recent versions?

[dbeltrankyl]

Additionally, @dbeltrankyl do we have information that a job is a SPLIT in the DB? Or is it something that is only visible in the pkl in recent versions?

Splits were in the code before I started to work on AS, but it seems that they never were added in the db (at least not in the job_data one)

So it has to be added.

The only way of knowing the split number is to parse the job name.

[dbeltrankyl]

You can also obtain it by loading the pkl

[mcastril]

Otherwise, what could be miscalculated is the sum of the SY (which only depends on the chunk unit and size) used in the numerator of the formula. So, taking into account the right jobs in that SY sum could fix this. This case should be the same with the other metrics.

Totally, I am sure the problem is in the SY and not in the duration, if I understood well

The SY are calculated with the CHUNK start and end date. That's why I was pointing out to the start and end date of the SPLIT.

Following what I said in the description:

I think that we could agree in general that by using SPLITs the workload for a CHUNK is equally distributed across its SPLITS, and therefore the SYs of a SPLIT should be SY_CHUNK/N_SPLITS. For example, in the workflow above, if the CHUNK computes one month of 30 days and we have 30 SPLITs, each SPLITs SY should be ((3600/12)/30), which roughly corresponds to one day.

I think that's the way to go. Dividing the SY (determined by the CHUNK dates) and dividing it by the SPLIT number, considering that each SPLIT computes and even part of the workload.

[franra9]

Can anyone bring a real example with the expected metric so we can validate that this could be a solution? @ainagaya @franra9

Hi @LuiggiTenorioK , you may find this experiment useful: t05t https://climatedt-wf.csc.fi/experiment/t05t/graph

In it you have splits that correspond each one to a day and the chunk is of 1 month.

best,
F

[LuiggiTenorioK]

Hi @LuiggiTenorioK , you may find this experiment useful: t05t https://climatedt-wf.csc.fi/experiment/t05t/graph

In it you have splits that correspond each one to a day and the chunk is of 1 month.

In this experiment, the ASYPD is calculated based on the 2 SIM jobs it has. I'm not sure if just fixing the total SY will work in this case.

How should the correct ASYPD be calculated in t05t? Should it take into account the APP_ENERGY_ONSHORE and/or OPA_ENERGY_ONSHORE sections' run times instead of the SIM ones?

[mcastril]

For this issue, it's enough to fix the SYPD/ASYPD for the individual SPLITs, when one checks the details in the tree view or graph view

For the general performance metrics, we agreed to select only the SIM, since this is usually the name of the simulation step (the generation of the data).

The rest of the jobs (data consumers and processing) should be considered in the PSYPD and WSYPD. For those metrics in #188 to work well we should also make sure that SPLITs are not perturbing them, but this could be discussed in the other issue, I think

[LuiggiTenorioK]

I'm implementing a patch for this issue in #196 using the pkl information of the latest pkl format, which also gives the number of split jobs, which lets us fix the SY of the SYPD/ASYPD straightforward.

Unfortunately, Autosubmit doesn't save the splits' historical data in the job_data table, so I cannot update other GUI features that involve knowing the history. This means that when someone searches for the "Job history" in the job panel of the GUI, they might find a different value in the SYPD/ASYPD calculation. Same with the "load previous run" feature in the Tree View.

We'll need to update that in Autosubmit in the future to allow this feature to be fully consistent 😞

[mcastril]

Did we know the Autosubmit version used in each run?

Because, if that's the case, you could always divide the value in bugged versions by the number of SPLITs, and show it in italic font to highlight that it was fixed a posteriori

[LuiggiTenorioK]

Did we know the Autosubmit version used in each run?

Because, if that's the case, you could always divide the value in bugged versions by the number of SPLITs, and show it in italic font to highlight that it was fixed a posterior

Unfortunately, no. We don't have the version of Autosubmit in which each run was run.