Skip to content

Commit ad01b47

Browse files
Lt-Flashrazvancrainea
authored andcommitted
core: name both kernel changes behind the EPOLLEXCLUSIVE gate
The comments and docs said the woken-but-not-running problem that got EPOLLEXCLUSIVE disabled in 3596efd was fixed in Linux 6.0. The main fix is 412895f03cbf in 5.7, which removes a woken waiter from the epoll wait queue when the wake-up succeeds; 6.0 (a16ceb139610, ep_autoremove_wake_function) removes it even when the wake-up fails. Say so, and that the 6.0 gate stays because 5.7-5.19 were not tested. Comments and documentation only.
1 parent 4c2b4a6 commit ad01b47

3 files changed

Lines changed: 12 additions & 7 deletions

File tree

‎docs/manual/Script-CoreParameters.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1275,7 +1275,7 @@ How TCP main hands a received message to the TCP workers. The default value is `
12751275

12761276
* `shared` - all TCP workers wait on one shared dispatch socket. Every dispatched message wakes every idle worker (one of them wins the read, the others return empty-handed), and the kernel wakes the last-forked worker first, so the per-worker load forms a descending ladder from the last worker to the first. This was the only behaviour before this parameter existed.
12771277
* `per_worker` - one dispatch socket per TCP worker; TCP main hands each message to the next idle active worker in round-robin order (or to the next active one when all are busy). One wakeup per message, even load across the workers.
1278-
* `exclusive` - the shared socket, registered with `EPOLLEXCLUSIVE` so that only one waiting worker is woken per message. Allowed with the `epoll` reactor on Linux 6.0 and newer only; on older kernels a woken but not yet running worker keeps absorbing the following messages, so bursts serialize on it - OpenSIPS falls back to `shared` with a warning there.
1278+
* `exclusive` - the shared socket, registered with `EPOLLEXCLUSIVE` so that only one waiting worker is woken per message. Allowed with the `epoll` reactor on Linux 6.0 and newer only, and OpenSIPS falls back to `shared` with a warning elsewhere. Before Linux 5.7 a woken but not yet running worker keeps absorbing the following messages, so bursts serialize on it; 5.7 removes a woken worker from the epoll wait queue at wake-up and 6.0 completes that, and kernels between the two were not tested.
12791279
* `auto` - `per_worker`, on every platform (it only needs `AF_UNIX` datagram socket pairs, which the `shared` mode already uses).
12801280

12811281
Example of usage:

‎globals.h‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,8 @@ extern int udp_workers_no;
5656
* shared - one socket, every worker polls it (each packet wakes all idle
5757
* workers, the last-forked one is woken first)
5858
* exclusive - same socket, EPOLLEXCLUSIVE (one wakeup per packet; needs a
59-
* 6.0+ kernel, otherwise bursts serialize on one worker)
59+
* 6.0+ kernel: before 5.7 bursts serialize on one worker,
60+
* 5.7-5.19 are untested, see io_watch_add())
6061
* reuseport - one SO_REUSEPORT socket per worker, a cBPF program spreads
6162
* the packets uniformly at random (needs a 4.5+ kernel)
6263
* auto - reuseport when the kernel supports it, else shared */

‎io_wait.h‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -550,11 +550,15 @@ inline static int io_watch_add( io_wait_h* h, // lgtm [cpp/use-of-goto]
550550
again1:
551551
#ifdef EPOLLEXCLUSIVE
552552
/* opt-in only (IO_WATCH_EXCLUSIVE); the unconditional use
553-
* was disabled in 3596efdc00 because, with level-triggered
554-
* epoll on kernels before 6.0, a woken but not yet running
555-
* process keeps absorbing the following events (fixed by
556-
* ep_autoremove_wake_function) - callers must gate on the
557-
* kernel version, see os_kernel_at_least() */
553+
* was disabled in 3596efdc00 because a woken but not yet
554+
* running process kept absorbing the following events: it
555+
* stayed on its epoll's wait queue until it ran, so
556+
* ep_poll_callback() still saw a waiter and ended the
557+
* exclusive wake-up there. Linux 5.7 (412895f03cbf) removes
558+
* the waiter when the wake-up succeeds, 6.0 (a16ceb139610,
559+
* ep_autoremove_wake_function) even when it fails. 5.4
560+
* serializes bursts, 6.12 does not, 5.7-5.19 are untested,
561+
* so callers gate on 6.0, see os_kernel_at_least() */
558562
if (excl && (flags & IO_WATCH_READ))
559563
ep_event.events|=EPOLLEXCLUSIVE;
560564
#endif

0 commit comments

Comments
 (0)