Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

job metrics: exclude skip / dummy mode tasks #626

Open
oliver-sanders opened this issue Aug 30, 2024 · 1 comment
Open

job metrics: exclude skip / dummy mode tasks #626

oliver-sanders opened this issue Aug 30, 2024 · 1 comment
Labels
data workflows team Work pertaining to the analysis/gantt/etc views
Milestone

Comments

@oliver-sanders
Copy link
Member

Cylc can run tasks in a few different modes:

  • Live (i.e. normal)
  • Simulation (don't actually run the task, just make it look like it's running)
  • Dummy (run a job, but remove all the script)
  • Skip - new at 8.4.0 (similar to simulation mode, but the task completes instantly)

The job metrics used by the analysis views are only interested in "Live" tasks, all of the other modes are various forms of simulation.

A common use case for "skip" mode might be to skip a cycle of tasks that are no longer needed in order to allow the workflow to catch up. These skipped tasks will cause a blip in the analysis view as they succeed instantly.

Ideally we would omit these tasks from the analysis statistics to avoid them messing up the averages.

You should be able to tell which mode a task was run in by looking at the "platform" field in the DB. For more details ask @wxtim.

Note, make sure any solution is back-compatible to Cylc 8.3.0 databases.

@oliver-sanders oliver-sanders added the data workflows team Work pertaining to the analysis/gantt/etc views label Aug 30, 2024
@oliver-sanders oliver-sanders added this to the pending milestone Aug 30, 2024
@wxtim
Copy link
Member

wxtim commented Sep 2, 2024

Dummy and simulation modes are back to being set for the whole workflow. You can't change mode of a running task to dummy or sim in an otherwise live workflow. This means that we only need to weed out skip mode tasks. If a user runs the workflow in sim or dummy mode then all the data in the job metrics view is meaningless. It might be worth popping up some sort of warning if the workflow is being run in dummy or sim mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data workflows team Work pertaining to the analysis/gantt/etc views
Projects
None yet
Development

No branches or pull requests

2 participants