Skip to content

Commit f8241df

Browse files
committed
add read me
1 parent 21b3f13 commit f8241df

1 file changed

Lines changed: 52 additions & 0 deletions

File tree

README.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# IPC queue
2+
3+
This is a study on how to do create a queue via IPC (inter-process communication).
4+
5+
Please take a look at the examples of [producer](./producer.c) and [consumer](./consumer.c)
6+
7+
## Design
8+
9+
There are two main components that can be used independently:
10+
11+
**SharedMem**
12+
13+
exposes an interface to share memory among processes,
14+
it uses [MMAP](https://en.wikipedia.org/wiki/Mmap) under the hood, and pre-allocates space that will be used.
15+
16+
---
17+
18+
**SharedQueue**
19+
20+
It uses the shared memory to create a ring buffer in the available space.
21+
Each message has a sequence number, that is prepended to the front on the message like a header, so we can keep track when reading.
22+
23+
The messages for this implementation have a fixed size, this way we can have O(1) access to memory by calculating the memory offset.
24+
This has some downsides if your domain has variable-length messages, but this could be easily implemented by modifying the queue and adding the message size into the header.
25+
26+
Due to each process being responsible for managing its own sequence number we end up having a lock-free, single-producer, multi-consumer model.
27+
This is heavily inspired by [LMAX's Disruptor](https://lmax-exchange.github.io/disruptor/).
28+
29+
Be aware that due to the ring buffer reuse of space some messages can be lost if the consumer is lagging (not ingesting messages fast enough).
30+
This has to be measured for your use case and can be solved by increasing the buffer size of the shared memory.
31+
32+
---
33+
34+
Warning: those components can act as writer/reader (or producer/consumer) but never both simultaneously.
35+
36+
## How to run
37+
38+
Just run `make` and then start the `./producer` and `./consumer` in different terminals.
39+
40+
## Improvements
41+
42+
I'm not a C expert, and I'm sure this code can be improved a lot
43+
44+
This is just for demonstration purposes; thus, there are lots of cases that are not covered, some examples:
45+
* trying to write to a read-only resources and so on
46+
* try to change initiate more than once (writer becomes a reader)
47+
48+
Feel free to contribute.
49+
50+
---
51+
52+
Shout-out to @apuxbt for sharing some snippets that inspired me

0 commit comments

Comments
 (0)