Skip to content

Commit 55bb8a8

Browse files
committedApr 23, 2025
Add rationale page to docs
1 parent a378f3d commit 55bb8a8

9 files changed

+183
-8
lines changed
 

‎deps.edn

+1-1
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
:exec-fn codox.main/generate-docs
3030
:exec-args {:source-paths ["src/main/clojure"]
3131
:namespaces [clojure.core.async clojure.core.async.flow clojure.core.async.flow.spi]
32-
:doc-files ["doc/flow.md"]
32+
:doc-files ["doc/rationale.md" "doc/flow.md"]
3333
:output-path "docs"
3434
:html {:namespace-list :flat}}}
3535
}}

‎doc/flow.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
# core.async.flow
2+
23
## Rationale
34

4-
The [rationale](https://clojure.org/news/2013/06/28/clojure-clore-async-channels) for **core.async** says "There comes a time in all good programs when components or subsystems must stop communicating directly with one another." And core.async provides fundamental tools (channels) for doing that.
5+
The [rationale](rationale.html) for **core.async** says "There comes a time in all good programs when components or subsystems must stop communicating directly with one another." And core.async provides fundamental tools (channels) for doing that.
56

67
But using core.async well, e.g. keeping your I/O out of your computational logic, requires discipline and architectural savvy, and to do so consistently throughout an application or organization, conventions. Given channels, many architectural decisions remain regarding thread execution, backpressure, error handling etc. And often the topology of your network of communicating processes *emerges* out of the flow of control of your program as various pieces of code create threads and wire channels together, interleaved with computation, making it difficult to see the topology or administer it in one place.
78

‎doc/rationale.md

+107
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# Rationale
2+
3+
*June 28, 2013*
4+
5+
[core.async](https://github.com/clojure/core.async) is a new contrib library for Clojure that adds support for asynchronous programming using channels.
6+
7+
There comes a time in all good programs when components or subsystems must stop communicating directly with one another. This is often achieved via the introduction of queues between the producers of data and the consumers/processors of that data. This architectural indirection ensures that important decisions can be made with some degree of independence, and leads to systems that are easier to understand, manage, monitor and change, and make better use of computational resources, etc.
8+
9+
On the JVM, the [java.util.concurrent](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/package-summary.html) package provides some good concurrent blocking queues, and they are a viable and popular choice for Clojure programs. However, in order to use the queues one must dedicate one or more actual threads to their consumption. Per-thread stack allocation and task-switching overheads limit the number of threads that can be used in practice. Another limitation of j.u.c. queues is there is no way to block waiting on a set of alternatives.
10+
11+
On JavaScript engines, there are no threads and no queues.
12+
13+
Thread overheads or lack of threads often cause people to move to systems based upon events/callbacks, in the pursuit of greater efficiency (often under the misnomer 'scalability', which doesn't apply since you can't scale a single machine). Events complect communication and flow of control. While there are various mechanisms to make events/callbacks cleaner (FRP, Rx/Observables) they don't change their fundamental nature, which is that upon an event an arbitrary amount of other code is run, possibly on the same thread, leading to admonitions such as "don't do too much work in your handler", and phrases like "callback hell".
14+
15+
The objectives of core.async are:
16+
17+
* To provide facilities for independent threads of activity, communicating via queue-like _channels_
18+
* To support both real threads and shared use of thread pools (in any combination), as well as ClojureScript on JS engines
19+
* To build upon the work done on CSP and its derivatives
20+
21+
It is our hope that async channels will greatly simplify efficient server-side Clojure programs, and offer simpler and more robust techniques for front-end programming in ClojureScript.
22+
23+
## History
24+
25+
The roots of this style go back at least as far as [Hoare's Communicating Sequential Processes (CSP)](https://en.wikipedia.org/wiki/Communicating_sequential_processes), followed by realizations and extensions in e.g. [occam](https://en.wikipedia.org/wiki/Occam_programming_language), [Java CSP](https://www.cs.kent.ac.uk/projects/ofa/jcsp/) and the [Go programming language](https://golang.org/).
26+
27+
In modern incarnations, the notion of a channel becomes first class, and in doing so provides us the indirection and independence we seek.
28+
29+
A key characteristic of channels is that they are blocking. In the most primitive form, an unbuffered channel acts as a rendezvous, any reader will await a writer and vice-versa. Buffering can be introduced, but unbounded buffering is discouraged, as bounded buffering with blocking can be an important tool coordinating pacing and back pressure, ensuring a system doesn't take on more work than it can achieve.
30+
31+
## Details
32+
33+
### Just a library
34+
35+
*core.async* is a library. It doesn't modify Clojure. It is designed to support Clojure 1.5+.
36+
37+
### Creating channels
38+
39+
You can create a channel with the [chan](https://clojure.github.io/core.async/clojure.core.async.html#var-chan) function. This will return a channel that supports multiple writers and readers. By default, the channel is unbuffered, but you can supply a number to indicate a buffer size, or supply a buffer object created via [buffer](https://clojure.github.io/core.async/clojure.core.async.html#var-buffer), [dropping-buffer](https://clojure.github.io/core.async/clojure.core.async.html#var-dropping-buffer) or [sliding-buffer](https://clojure.github.io/core.async/clojure.core.async.html#var-sliding-buffer).
40+
41+
The fundamental operations on channels are putting and taking values. Both of those operations potentially block, but the nature of the blocking depends on the nature of the thread of control in which the operation is performed. core.async supports two kinds of threads of control - ordinary threads and IOC (inversion of control) 'threads'. Ordinary threads can be created in any manner, but IOC threads are created via [go blocks](https://clojure.github.io/core.async/clojure.core.async.html#var-go). Because JS does not have threads, only `go` blocks and IOC threads are supported in ClojureScript.
42+
43+
### go blocks and IOC 'threads'
44+
45+
`go` is a macro that takes its body and examines it for any channel operations. It will turn the body into a state machine. Upon reaching any blocking operation, the state machine will be 'parked' and the actual thread of control will be released. This approach is similar to that used in [C# async](https://msdn.microsoft.com/en-us/library/vstudio/hh191443.aspx). When the blocking operation completes, the code will be resumed (on a thread-pool thread, or the sole thread in a JS VM). In this way the inversion of control that normally leaks into the program itself with event/callback systems is encapsulated by the mechanism, and you are left with straightforward sequential code. It will also provide the illusion of threads, and more important, separable sequential subsystems, to ClojureScript.
46+
47+
The primary channel operations within go blocks are [>!](https://clojure.github.io/core.async/clojure.core.async.html#var-.3E.21) (_put_) and [<!](https://clojure.github.io/core.async/clojure.core.async.html#var-.3C.21) (_take_). The go block itself immediately returns a channel, on which it will eventually put the value of the last expression of the body (if non-nil), and then close.
48+
49+
### Channel on ordinary threads
50+
51+
There are analogous operations for use on ordinary threads - [>!!](https://clojure.github.io/core.async/clojure.core.async.html#var-.3E.21.21) (_put blocking_) and [<!!](https://clojure.github.io/core.async/clojure.core.async.html#var-.3C.21.21) (_take blocking_), which will block the thread on which they are called, until complete. While you can use these operations on threads created with e.g. future, there is also a macro, [thread](https://clojure.github.io/core.async/clojure.core.async.html#var-thread), analogous to `go`, that will launch a first-class thread and similarly return a channel, and should be preferred over `future` for channel work.
52+
53+
### Mixing modes
54+
55+
You can put on a channel from either flavor of `>!`/`>!!` and similarly take with either of `<!`/`<!!` in any combination, i.e. the channel is oblivious to the nature of the threads which use it.
56+
57+
### alt
58+
59+
It is often desirable to be able to wait for any one (and only one) of a set of channel operations to complete. This powerful facility is made available through the [alts!](https://clojure.github.io/core.async/clojure.core.async.html#var-alts.21) function (for use in `go` blocks), and the analogous [alts!!](https://clojure.github.io/core.async/clojure.core.async.html#var-alts.21.21) (_alts blocking_). If more than one operation is available to complete, one can be chosen at random or by priority (i.e. in the order they are supplied). There are corresponding [alt!](https://clojure.github.io/core.async/clojure.core.async.html#var-alt.21) and [alt!!](https://clojure.github.io/core.async/clojure.core.async.html#var-alt.21.21) macros that combine the choice with conditional evaluation of expressions.
60+
61+
### Timeouts
62+
63+
Timeouts are just channels that automatically close after a period of time. You can create one with the [timeout](https://clojure.github.io/core.async/clojure.core.async.html#var-timeout) function, then just include the timeout in an `alt` variant. A nice aspect of this is that timeouts can be shared between threads of control, e.g. in order to have a set of activities share a bound.
64+
65+
### The value of values
66+
67+
As with STM, the pervasive use of persistent data structures offers particular benefits for CSP-style channels. In particular, it is always safe and efficient to put a Clojure data structure on a channel, without fear of its subsequent use by either the producer or consumer.
68+
69+
### Contrasting Go language channels
70+
71+
core.async has obvious similarities to Go channels. Some differences with Go are:
72+
73+
* All of the operations are expressions (not statements)
74+
* This is a library, not syntax
75+
* `alts!` is a function (and supports a runtime-variable number of operations)
76+
* Priority is supported in `alt`
77+
78+
Finally, Clojure is hosted, i.e. we are bringing these facilities to existing platforms, not needing a custom runtime. The flip-side is we don't have the underpinnings we would with a custom runtime. Reaching existing platforms remains a core Clojure value proposition.
79+
80+
### Whither actors?
81+
82+
I remain unenthusiastic about actors. They still couple the producer with the consumer. Yes, one can emulate or implement certain kinds of queues with actors (and, notably, people often do), but since any actor mechanism already incorporates a queue, it seems evident that queues are more primitive. It should be noted that Clojure's mechanisms for concurrent use of state remain viable, and channels are oriented towards the flow aspects of a system.
83+
84+
### Deadlocks
85+
86+
Note that, unlike other Clojure concurrency constructs, channels, like all communications, are subject to deadlocks, the simplest being waiting for a message that will never arrive, which must be dealt with manually via timeouts etc. CSP proper is amenable to certain kinds of automated correctness analysis. No work has been done on that front for core.async as yet.
87+
88+
Also note that async channels are not intended for fine-grained computational parallelism, though you might see examples in that vein.
89+
90+
## Future directions
91+
92+
Networks channels and distribution are interesting areas for attention. We will also being doing performance tuning and refining the APIs.
93+
94+
### Team
95+
96+
I'd like to thank the original team that helped bring core.async to life:
97+
98+
* Timothy Baldridge
99+
* Ghadi Shayban
100+
* Alex Miller
101+
* Alex Redington
102+
* Sam Umbach
103+
104+
I hope that these async channels will help you build simpler and more robust programs.
105+
106+
Rich
107+

‎docs/clojure.core.async.flow.html

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
<!DOCTYPE html PUBLIC ""
22
"">
3-
<html><head><meta charset="UTF-8" /><title>clojure.core.async.flow documentation</title><link rel="stylesheet" type="text/css" href="css/default.css" /><link rel="stylesheet" type="text/css" href="css/highlight.css" /><script type="text/javascript" src="js/highlight.min.js"></script><script type="text/javascript" src="js/jquery.min.js"></script><script type="text/javascript" src="js/page_effects.js"></script><script>hljs.initHighlightingOnLoad();</script></head><body><div id="header"><h2>Generated by <a href="https://github.com/weavejester/codox">Codox</a></h2><h1><a href="index.html"><span class="project-title"><span class="project-name"></span> <span class="project-version"></span></span></a></h1></div><div class="sidebar primary"><h3 class="no-link"><span class="inner">Project</span></h3><ul class="index-link"><li class="depth-1 "><a href="index.html"><div class="inner">Index</div></a></li></ul><h3 class="no-link"><span class="inner">Topics</span></h3><ul><li class="depth-1 "><a href="flow.html"><div class="inner"><span>core.async.flow</span></div></a></li></ul><h3 class="no-link"><span class="inner">Namespaces</span></h3><ul><li class="depth-1 "><a href="clojure.core.async.html"><div class="inner"><span>clojure.core.async</span></div></a></li><li class="depth-1 current"><a href="clojure.core.async.flow.html"><div class="inner"><span>clojure.core.async.flow</span></div></a></li><li class="depth-1 "><a href="clojure.core.async.flow.spi.html"><div class="inner"><span>clojure.core.async.flow.spi</span></div></a></li></ul></div><div class="sidebar secondary"><h3><a href="#top"><span class="inner">Public Vars</span></a></h3><ul><li class="depth-1"><a href="clojure.core.async.flow.html#var-create-flow"><div class="inner"><span>create-flow</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-futurize"><div class="inner"><span>futurize</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-inject"><div class="inner"><span>inject</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-lift*-.3Estep"><div class="inner"><span>lift*-&gt;step</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-lift1-.3Estep"><div class="inner"><span>lift1-&gt;step</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-map-.3Estep"><div class="inner"><span>map-&gt;step</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-pause"><div class="inner"><span>pause</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-pause-proc"><div class="inner"><span>pause-proc</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-ping"><div class="inner"><span>ping</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-ping-proc"><div class="inner"><span>ping-proc</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-process"><div class="inner"><span>process</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-resume"><div class="inner"><span>resume</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-resume-proc"><div class="inner"><span>resume-proc</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-start"><div class="inner"><span>start</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-stop"><div class="inner"><span>stop</span></div></a></li></ul></div><div class="namespace-docs" id="content"><h1 class="anchor" id="top">clojure.core.async.flow</h1><div class="doc"><pre class="plaintext">
3+
<html><head><meta charset="UTF-8" /><title>clojure.core.async.flow documentation</title><link rel="stylesheet" type="text/css" href="css/default.css" /><link rel="stylesheet" type="text/css" href="css/highlight.css" /><script type="text/javascript" src="js/highlight.min.js"></script><script type="text/javascript" src="js/jquery.min.js"></script><script type="text/javascript" src="js/page_effects.js"></script><script>hljs.initHighlightingOnLoad();</script></head><body><div id="header"><h2>Generated by <a href="https://github.com/weavejester/codox">Codox</a></h2><h1><a href="index.html"><span class="project-title"><span class="project-name"></span> <span class="project-version"></span></span></a></h1></div><div class="sidebar primary"><h3 class="no-link"><span class="inner">Project</span></h3><ul class="index-link"><li class="depth-1 "><a href="index.html"><div class="inner">Index</div></a></li></ul><h3 class="no-link"><span class="inner">Topics</span></h3><ul><li class="depth-1 "><a href="rationale.html"><div class="inner"><span>Rationale</span></div></a></li><li class="depth-1 "><a href="flow.html"><div class="inner"><span>core.async.flow</span></div></a></li></ul><h3 class="no-link"><span class="inner">Namespaces</span></h3><ul><li class="depth-1 "><a href="clojure.core.async.html"><div class="inner"><span>clojure.core.async</span></div></a></li><li class="depth-1 current"><a href="clojure.core.async.flow.html"><div class="inner"><span>clojure.core.async.flow</span></div></a></li><li class="depth-1 "><a href="clojure.core.async.flow.spi.html"><div class="inner"><span>clojure.core.async.flow.spi</span></div></a></li></ul></div><div class="sidebar secondary"><h3><a href="#top"><span class="inner">Public Vars</span></a></h3><ul><li class="depth-1"><a href="clojure.core.async.flow.html#var-create-flow"><div class="inner"><span>create-flow</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-futurize"><div class="inner"><span>futurize</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-inject"><div class="inner"><span>inject</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-lift*-.3Estep"><div class="inner"><span>lift*-&gt;step</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-lift1-.3Estep"><div class="inner"><span>lift1-&gt;step</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-map-.3Estep"><div class="inner"><span>map-&gt;step</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-pause"><div class="inner"><span>pause</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-pause-proc"><div class="inner"><span>pause-proc</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-ping"><div class="inner"><span>ping</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-ping-proc"><div class="inner"><span>ping-proc</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-process"><div class="inner"><span>process</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-resume"><div class="inner"><span>resume</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-resume-proc"><div class="inner"><span>resume-proc</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-start"><div class="inner"><span>start</span></div></a></li><li class="depth-1"><a href="clojure.core.async.flow.html#var-stop"><div class="inner"><span>stop</span></div></a></li></ul></div><div class="namespace-docs" id="content"><h1 class="anchor" id="top">clojure.core.async.flow</h1><div class="doc"><pre class="plaintext">
44
Note - Alpha, work-in-progress, names and other details are in flux
55

66
A library for building concurrent, event driven data processing

0 commit comments

Comments
 (0)