play: support multiple workflows

[ColemanTom]

Problem

I can pause multiple workflows by doing cylc pause *pattern*. But, when I want to resume them, I have to do for w in *pattern*; do cylc play $w; done (or similar).

Proposed Solution

Should cylc play work on multiple workflows?

[oliver-sanders]

At present cylc play does not support multiple workflows.

It's a slightly tricky thing to implement due to some of the options cylc play supports. The biggest issue is with --no-detach, when this option is used, runs in the terminal session rather than detaching into a new fork. As a result, if we supported multiple workflows, --no-detach would cause them to run in serial rather than parallel.

[hjoliver]

It's a valid point though. Intuitively, if you pause a bunch of workflows with a single command you should be able to un-pause them with a single command too.

--no-detach has no effect on un-pausing a running workflow.

For start-up, I suppose we could prompt for confirmation if --no-detach is used with multiple workflow IDs. Or make it an error (although we might actually be able to dream up valid use cases for running multiple small workflows serially).

It's a slightly tricky thing to implement due to some of the options cylc play supports.

Aside from --no-detach I presume you're concerned about the fact that options like --stop-cycle-point are unlikely to be valid for multiple workflows? Or is it more than that?

If that's the only issue, I think it's reasonable to assume that all options apply to all workflows on the command line, otherwise don't "play" them en masse.

[ColemanTom]

Intuitively, if you pause a bunch of workflows with a single command you should be able to un-pause them with a single command too.

Yes, this is what I was expecting, as you say, intuitively. It's not a major problem or anything like that of course, low priority to consider for me, etc

[oliver-sanders]

For start-up, I suppose we could prompt for confirmation if --no-detach is used with multiple workflow IDs. Or make it an error

I think it would be easier to raise an error. I don't think we should encourage using the cylc play command as a pipeline manager!

Aside from --no-detach I presume you're concerned about the fact that options like --stop-cycle-point are unlikely to be valid for multiple workflows? Or is it more than that?

Would need to go through the option properly, --upgrade is another option that will require some thinking.

There shouldn't be any fatal blockers, just needs work, it isn't a one-liner.

[oliver-sanders]

Another technical barrier to cross (related to #6297).

If restarting (as opposed to resuming) a workflow, the restart code must be called synchronously with no asyncio event loops running. This is a bit awkward as the code will be async up to this point (e.g. because workflow scanning is async) and effectively means that the schedulers could only be started in series (though that isn't necessarily a bad way to get this functionality started).

The easy way out is probably just to call the restart part in a new process where it is detached from the async logic of the parent process.