Group re-run proposal

docs/proposal-group-rerun

@@ -0,0 +1,98 @@
+# PROPOSAL: Task Group Rerun
+
+A proposal to allow re-running a selected group of tasks without worrying
+about off-flow prerequistes.
+
+## Background
+
+It has been reported that some users find it easier to understand how to
+rerun a sub-graph of tasks (typically identified by a family name) in
+Cylc 7, than in Cylc 8.
+
+### Cylc 7 group rerun
+
+C7_a: Cylc 7 (group members and upstream parents still in task pool):
+1. identify the group to rerun (e.g. by family name)
+2. reset all group members to waiting
+
+C7_b: Cylc 7 (if any group members or parents have left the pool):
+1. identify the group to rerun (e.g. by family name)
+2. insert all group members as waiting
+3. insert all off-flow parents as waiting
+4. reset all the parents to succeeded
+
+Dependency matching will then cause the sub-graph to run correctly. 
+
+### Cylc 8 group rerun
+
+C8_a: Cylc 8 (general, new flow):
+1. identify the initial task(s) of the sub-graph
+2. trigger them with --flow=new
+3. set any off-flow prerequisites
+
+In this case, you might need to think about 
+4. (consider post-sub-graph flow-stop if needed)
+
+C8_b: Cylc 8 (general, re-flow):
+1. identify the group to rerun (e.g. by family name)
+2. `cylc remove` all group members to erase the previous flow
+3. trigger the initial task(s) of the group (/sub-graph)
+4. set any off-flow prerequisites
+
+### Comparison
+
+C7_a is the simplest. It relies on the group and its parents still being in the task
+pool, but that is often the case when dealing with same-cycle problems. In that case,
+users can get away without understanding the task pool. 
+
+C7_b is probably the least intuitive of all - it requires some understanding
+of the "task pool", the strange concept of task "insertion", and the graph - to
+identify off-flow parent tasks.
+
+C8_a is conceptually clean, and general, but it requires an understanding of the graph structure
+to identify off-flow prerequisites, and it might result in unwanted downstream
+activity.
+
+C8_b is like C8_a, but trades off unwanted downstream activity for `cylc remove`.
+
+## Cylc 8 group "rerun?" proposal
+
+This proposal should make re-running any Cylc 8 sub-graph as easy as the
+(non-general) C7_a case.
+
+C8_c: Cylc 8 (general, proposal):
+1. identify the group to rerun (e.g. by family name)
+2. `cylc trigger --group=FAMILY [--flow=new]` (e.g.)
+
+### Implementation:
+
+For each group member, automatically:
+   - remove the previous flow (if not --flow=new)
+   - set all off-flow (out-group) prerequisites
+   - unset all in-flow (in-group) prerequisites (existing n=0 only)
+   - "set all" any parentless in-group tasks (i.e., promote to task pool)
+
+### CLI
+
+```
+cylc trigger --group=GROUP
+```
+
+This triggers a group "as a group", i.e. respecting internal dependencies
+(as opposed to triggering all group members at once).
+
+Even though the implementation just "sets prerequisites" I think the trigger
+command is the more appropriate home for this functionality than `cylc set`
+(which currently sets prerequisites or outputs on individual tasks - it
+doesn't handle multiple tasks at once). 
+
+### Outputs
+
+In principle we could also examine group member outputs and remove or set any
+out-group outputs, to avoid unwanted downstream activity.
+
+However, we would often want the flow to continue if the rerun is successful;
+and using `cylc remove` by default ensures that we won't re-run anything 
+downstream that already succeeded in the flow. 
+
+So I thinking managing off-flow outputs is not necessary.