[RFC] task_group_dynamic_dependencies

[vossmjp]

Description

A proposal to extend task_group:

1. Extend semantics and useful lifetime of task_handle. We propose task_handle to represent tasks for the purpose of adding dependencies. The useful lifetime and semantics of task_handle will need to be extended to include tasks that have been submitted, are currently executing, or have been completed.
2. Add functions to set task dependencies. In the current task_group, tasks can only be waited for as a group and there is no direct way to add any before-after relationships between individual tasks. We will discuss options for spelling.
3. Add a function to move successors from a currently executing task to a new task. This functionality is necessary for recursively generated task graphs. This case represents a situation where it is safe to modify dependencies for an already submitted task.

Type of change

• bug fix - change that fixes an issue
• new feature - change that adds functionality
• tests - change in tests
• infrastructure - change in infrastructure and CI
• documentation - documentation update

Tests

• added - required for new features and some bug fixes
• not needed

Documentation

• updated in # - add PR number
• needs to be updated
• not needed

Breaks backward compatibility

• Yes
• No
• Unknown

Notify the following users

List users with @ to send notifications

Other information

[pavelkumbrasev]

I think this proposal is lacking the final definition of class task_handle including all the new methods.

[aleksei-fedotov]

Since this method is related to the currently executing task, what about including this API into tbb::this_task:: namespace? By analogy with tbb::this_task_arena:: namespace.

[akukanov]

I would use tbb::task namespace, since it is already used for suspend (which applies to the currently running task) and resume functions.
Actually no, I think it should not be in the namespace task or this_task or just tbb, but rather it should be a static function or a member function in task_group. The reason is that, since task_group::defer is the only way to create a new non-empty task_handle, the method to transfer successors cannot be used in arbitrary tasks, only in task_group tasks.

[vossmjp]

Will add to task_group

[aleksei-fedotov]

These are interesting questions.

While reading this RFC, I kind of rushed proposing additional syntax sugar that besides being more user-friendly, since it seems to represent popular use cases, can save some CPU cycles. So I am posting them here for a discussion.

1. Would it be useful to have a method that simultaneously joins (or even fuses?) the instantiation of a task and transferring of the current task successors?
   Something like:

    template <typename Func>
    task_handle transfer_successors_to(Func&& f);

For the recursive decomposition scenario, instantiating a new task within an executing task and transferring successors to that new task seems to be the main model of writing such algorithms. Although, it seems to be not saving much (only one call to the library and assign to a couple of pointers?), there is always(?) going to be such a sequence in the code. Otherwise, how else an execution of already submitted tasks can be postponed?

Shall we also consider a question of having that API instead or in addition to the one proposed?

2. As for add_predecessor(s) and add_successor(s) I have a couple of thoughts.
   a. It seems again that it might be useful to merge instantiation of the new task handles and adding them as successors/predecessors:

       template <typename Func>
       void add_predecessor(Func&& f);
   
       template <typename Func>
       void add_successor(Func&& f);

   b. Also, I think having an API that would allow adding more than one predecessor/successor at once can be useful, since usually a number of successors/predecessors are instantiated. I only think that we don't need to limit ourselves to only two parameters as it was proposed optionally, but allow passing of an arbitrary size of task handles or even user lambdas. Of course, a pattern of having a single task producer might be viewed as a limiting one, but there actually might be the cases where tasks cannot be made available to the scheduler (i.e. spawned) until all of them are gathered together from different producers, which essentially represents a barrier in the execution pipeline. Not to mention that the spawning of a bunch of tasks all together are done faster than regular one by one spawnings. Spawning of a bunch of tasks at once was implemented in the old TBB, as far as I remember.
   So here I suggest to have something like:

       template <typename... Func>
       void add_predecessors(Func&& ...f);
   
       template <typename... Func>
       void add_successor(Func&& ...f);

   However, this also seems to ask for having the task_group::run() method to accept an arbitrary size of task handles and/or functors. So, perhaps, it is more related to another RFC/extension...

[vossmjp]

I think we should start with minimal API then extend with syntactic sugar based on use cases. I suspect this will start as an experimental feature to allow some feedback on API.

Even so, I'm also open to including these additional APIs in the initial implementation, if others think they're likely needed.

[akukanov]

I would prefer not to add methods to task_handle but use "external" functions, perhaps in the task_group class. This would be more consistent with the current approach (defer, run, run_and_wait) as well as with transfer_successors_to.

[vossmjp]

What would the signature of such a function look like? The nice part of it being in task_handle is that h2.add_predecessor(h1) is easy to understand -- h1 becomes a predecessor of h2. As part of task_group, the order becomes less clear: tg.add_predecessor(h1,h2) might be read as adding two predecessors to task_group tg.

[akukanov]

What would the signature of such a function look like?

Similar to make_edge, but perhaps with a different name: connect, set_order, set_dependency, etc., with the left-to-right ordering semantics.

Currently, a task_handle is what its name says - a handle, an object that might represent a created task or be empty. It cannot even be copied, only moved.

The proposal suggests that a task_handle should represent a task at any state, so should perhaps internally track the task state somehow - that's OK. We can even think of a method to query the task state; that would also be OK.

But making the handle also tracking task dependencies, and so serving as a part of the scheduling system, rather than just an object to be scheduled, does not sound right to me. Conceptually, I see scheduling and dependency management as functions of a task group.

[vossmjp]

Last year, we discuss naming quite a bit. One of the reasons to offer make_edge as an option was to be consistent with flow graph terminology and therefore the expected left-to-right ordering. Some people that worked with tasks before thought of parent-child relationships or dependency relations as a depends on b. And so something like task_group::set_dependency(a, b) could easily be read two ways. The benefit of task_handle::add_predecessor is its readibility and a task_group::make_edge(a, b) might benefit from exposure to flow::make_edge. I don't think we should consider it now, but we could also consider make_edge between a task_handle and a flow graph continue_node.

[akukanov]

I support this proposal in principle, and would approve it up to but not including the proposed API changes.

The API changes seem both insufficient (see the comment about the run function semantics) and arguable (when it comes to the way to set dependencies, see the discussion in comments above). I would not approve it even for an experimental API, at least until alternative API semantics are considered and compared.

[akukanov]

Note that this is technically done by the run functions accepting a task_handle as an rvalue. The task group can then move-construct or move-assign from that handle, which will make it empty.

In order to "extend useful lifetime of a task handle`, the run functions should therefore treat the handle argument differently. This part is not covered by the proposal.

[vossmjp]

Yes, needs to be added.

[akukanov]

Do not forget also about task_arena::enqueue. What would be its behavior?

[vossmjp]

Yes, run and enqueue functions are not yet well covered.

[akukanov]

I can suggest a few more potentially interesting cases:

• a wavefront, where tasks have multiple predecessors and multiple successors;
• a two-stage parallel scan (reduce-then-scan), with left-to-right propagation of the accumulated prefix sum;
• non-trivial divide-and-conquer patterns, such as Matteo Frigo's algorithm for the N-Body problem (see e.g. https://dspace.mit.edu/bitstream/handle/1721.1/122680/6-172-fall-2010/contents/projects/MIT6_172F10_proj4_1.pdf).

These seem to be interesting examples of "dynamic task graphs that are not trees", as stated in the introduction. I do not suggest that we must support these patterns, but it is interesting to see if we could, and what would be needed for that.

[vossmjp]

Yes, more example would be good. In particular, we need to see if the very limited ways to add and modified predecessors is sufficient. And how lifetimes of task handles will be managed.

[kboyarinov]

I think we need to add information about possible limitations (or lack of limitations) for the task states represented by task_handle to which we are transferring the successors. Does it sufficient to allow only the handles representing created tasks, or other 3 states are also allowed (i.e. transferring successors to the task_handle that was already submitted to task_group::run).

[kboyarinov]

I have noticed one thing that is not related to the dependencies between tasks, but relates into covering the migration from the old tasking API to another, so I decided to add it as a comment here.

Let's consider the recursive Fibonacci example rewritten using the API proposed in this RFC (the splitting stage):

long* left = new long(0);
long* right = new long(0);

tbb::task_handle fib_left = tg.defer([&tg, num, left] {
    recursive_fib(tg, num - 2, *left);
});
tbb::task_handle fib_right = tg.defer([&tg, num, right] {
    recursive_fib(tg, num - 1, *right);
});
tbb::task_handle fib_sum = tg.defer([&result, left, right] {
    result = *left + *right;
    delete left;
    delete right;
});

|
The main difference between merge sort and this example is some data that is required for executing the task (left and right - the placeholders for partial results of Fibonacci calculations on leaft).
Since the lifetime of this data should be preserved until the sum task is executed, it cannot be placed on stack of current function and needs to be allocated dynamically.
Back to old TBB, this data was placed inside of the corresponding task that provides the required lifetime guarantees.
The question is do we need to extend the task_handle API somehow to allow putting the additional data to the task.