Skip to content

Parallelism

Stephen M. Coakley edited this page Aug 10, 2020 · 1 revision

One of Riptide's goals is to take common use-cases that are usually awkwardly done with external tools and integrate them into the language instead if there is a clear benefit. One such example is running parallel jobs. Tools like xargs and GNU parallel are nice, but can be awkward to use, especially if you want the job to be a code block, or have odd quoting issues, etc.

We should be able to replace most uses of xargs or GNU parallel by making builtins support a parallel option by default. For example, each-line could look like this:

each-line [line] {
    curl -XDELETE (basename $line)
} < paths.txt

You could use xargs to run this loop in parallel, but that could get complicated because of the use of a subshell. Instead, each-line could support a simple flag to enable parallel operation:

each-line --parallel -j8 [line] {
    curl -XDELETE (basename $line)
} < paths.txt

This helps lead users toward the pit of success, because refactoring a serial loop into a parallel one simply requires setting some options instead of requiring them to rewrite the loop completely to conform with what some external tool expects.

Clone this wiki locally