dbus-broker consumes more memory than dbus-daemon

dbus-broker seems to offer significantly better performance than dbus-daemon. However, this improved performance comes at the cost of higher memory usage compared to dbus-daemon. In my comparison tests with dbus-broker and dbus-daemon, I observed that dbus-broker consumes approximately 8MB more memory. This may not be a problem at all on a desktop, and may not be noticeable at all, but it can be a burden on the system on an embedded system. Is this an issue only I have, or have you experienced similar issues?

Could you please let me know if there is any configuration or other way to reduce or control the memory usage of dbus-broker?

Any ideas or advice would be appreciated.

[dvdhrm]

Hi! Thanks for the report! Can you elaborate on your tests and numbers a bit? Like how big is the absolute memory usage in your tests? What kind of memory did you measure (resident, virtual, etc.)? What did the test environment look like (full distro? how many connections? ...)?

Also, what kind of numbers did you expect? Is this memory usage actually a problem? How much of the overall memory consumption is this? What would the ideal consumption look like to you?

Generally, there should not be much of a difference between dbus-daemon and dbus-broker. The only real knob to work with is the size of the staging message buffers. dbus-daemon might release them on every transaction, but dbus-broker retains them and (I think) reserves a page per connection. This can certainly be tweaked, but I would like to have some actual numbers to judge.

Thank you for your reply and let me share some more details.

I believe memory usage can be viewed from two perspectives: the overall system and the internal usage of D-Bus(dbus-broker and dbus-daemon). When I initially mentioned that dbus-broker uses 8MB more memory than dbus-daemon, I was referring to the overall system memory consumption. When comparing the internal consumption of dbus-broker and dbus-daemon, it seems that dbus-broker uses about 4.5MB more memory than dbus-daemon (This can vary depending on the situation and let's just say 5MB for simplicity). Discussing the overall system consumption results can be more complicated, so for now let's just focus specifically on the internal memory consumption results of dbus-broker and dbus-daemon.

The test is carried out below conditions:

1. Poky(Yocto) is being used as a distro.
2. PSS(proportional share size) is the main evaluation indicator, so the memory consumption is measured by PSS from dbus-damon and dbus-broker.

The test is carried out below order:

1. Before starting the test, evaluate the current memory consumption.
2. Run the test which D-Bus calls 1000 times to write 100 KB of data each.
3. Evaluate the the memory consumption again after 10 seconds from the start running the test, while the test is still ongoing.
4. Evaluate again immediately after test completion.
5. Run the above tests on both dbus-daemon and dbus-broker and compare the results.

Through testing, dbus-broker consumes around 12MB and dbus-daemon consumes around 7.5MB(about 4.5MB difference). The dbus-broker figure is the sum of dbus-broker and dbus-broker-launch.
You can see more details through below picture. "D-Bus current memory" shows the PSS memory consumption inside dbus-broker and dbus-daemon.

I was hoping that dbus-broker would have less memory consumption, or at most 1-2 MB more. However, I think that the 5MB increase in memory usage due to IPC could be burdensome for embedded systems.

Could you please explain where in the code I can find and adjust the staging message buffers?

If you need any further information, please let me know.

[dvdhrm]

Thanks for the details! The strategy sounds plausible to me, but I would definitely say that full system benchmarks are very valid plans as well. Also, you might wanna try adding USS numbers to your graphs, since they usually correlate more with actual runtime allocations. RSS/PSS can still differ between systems, depending on which (unrelated) software you run. For instance, if you run many other applications that use libdbus, numbers of dbus-daemon will look much better. But this will not (greatly) affect the deltas you posted, so that should be fine.

I think the buffer you want to experiment with is defined in src/dbus/queue.h as struct IQueue.buffer. Its size is determined by IQUEUE_RECV_MAX (same file). This size must be at least 16 to ensure message headers can be read, but I think you want at least something like 128 or 256 to not tank performance, but I might be wrong. By default it is set to 2048. You can adjust QUEUE_RECV_MAX to see how it affects your benchmarks.

As per your suggestion, I reduced IQUEUE_RECV_MAX to 128 and conducted tests. The results show that memory consumption decreased by approximately 600KB to 1MB. However, this improvement was not still enough to meet the requirements.

Secondly, as per your suggestion, I added USS and included a new graph below. The graph shows a significant difference in private dirty memory between dbus-daemon and dbus-broker, which may indicate significant memory allocations and changes in the stack or heap.

If you have any additional suggestions or ideas for further testing, I would appreciate your advice.

[dvdhrm]

Hi, can you elaborate at which point of the bemchmark your graph is drawn? While it is running, or when done?

The test is carried out below order:

1. Before starting the test, evaluate the current memory consumption. The test involving message transmission via D-Bus has not started yet.
2. Run the test, which D-Bus calls 1000 times to write 100 KB of data each over SSH .
3. Evaluate the the memory consumption again after 10 seconds from the start running the test, while the test is still ongoing. The message is currently being transmitted via D-Bus.
4. Evaluate again immediately after test completion. Since the test is complete, there is no message transmission via D-Bus.
5. Run the above tests on both dbus-daemon and dbus-broker and compare the results.

In the following, the dbus-broker displayed represents the combined memory consumption of both dbus-broker and dbus-broker-launch.
[Test result from No.1]
The following is a table and graph showing the results measured at point 1.

[Test result from No.3]
The following is a table and graph showing the results measured at point 3.

To summarize the test results, it can be observed that even before the actual test begins, dbus-broker already consumes more memory than dbus-daemon. When the test starts, there is a slight increase in memory consumption, but no significant change. It seems likely that the allocated buffer is already large enough.

If you have any additional suggestions or ideas for further testing, I would appreciate your advice.

As can be seen in the image above, the biggest factor contributing to the difference in memory usage between dbus-daemon and dbus-broker is the private dirty memory. Additionally, there is no significant change in memory usage before(test result from No.1), during(test result from No.3) the test. It seems that dbus-broker has already secured a large buffer size before the test starts.

So, I investigated the private dirty area as follows:

The difference in binary size below is not big .
dbus-daemon : 862KB
dbus-broker : 1.1MB
This is explained as there being no significant difference between the text and data section. I think what remains in the private area is the BSS, heap, stack and mmap areas. Although reducing their size are very difficult, I have tried the following. However, there was no improvement.
IQUEUE_RECV_MAX → 128(Your recommendation)
IQUEUE_LINE_MAX: 2K → 128
LOG_SIZE_MAX: 2M → 1K
mmap in log.c: log->map_size: 1000(hard-coded)

Is there anything else worth trying?

[dvdhrm]

Can you measure this directly on dbus-broker, rather than the combination of the launcher+broker?

I have separated and displayed the memory usage for both dbus-broker and dbus-broker-launch individually below. I hope this is helpful.

Thank you for your help.

[CoelacanthusHex]

Hi. Maybe a bit later. pmap is a good tool to analyze process memory usage. With pmap -p -X $PID, you can see some details about memory usage, although it can't know the internal details of the heap and mmap.

[aktau]

I just switched my NixOS setup from dbus-daemon to dbus-broker/dbus-broker-launch.

For dbus-daemon, I had the following (3 processes: 1 systemwide bus and 2 users):

message+  8492 /nix/store/fvgf0xw1xksz2xhz9qlwvxkkxfv8j1z0-dbus-1.14.10/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
kodi      4180 /nix/store/fvgf0xw1xksz2xhz9qlwvxkkxfv8j1z0-dbus-1.14.10/bin/dbus-daemon --session --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
aktau     4392 /nix/store/fvgf0xw1xksz2xhz9qlwvxkkxfv8j1z0-dbus-1.14.10/bin/dbus-daemon --session --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only

So about 8MiB for the systemwide daemon and 4MiB per user daemon. 16MiB total.

message+  4800 /nix/store/6kbfpk830lb7s8h26l87r6i5kwfl68x3-dbus-broker-37/bin/dbus-broker-launch --scope system --audit
  message+  3360 dbus-broker --log 11 --controller 10 --machine-id 2fc5204c4c03482f8844a9c832f262e1 --max-bytes 536870912 --max-fds 4096 --max-matches 16384 --audit
kodi      4300 /nix/store/6kbfpk830lb7s8h26l87r6i5kwfl68x3-dbus-broker-37/bin/dbus-broker-launch --scope user
  kodi      2604 dbus-broker --log 11 --controller 10 --machine-id 2fc5204c4c03482f8844a9c832f262e1 --max-bytes 100000000000000 --max-fds 25000000000000 --max-matches 5000000000
aktau     4328 /nix/store/6kbfpk830lb7s8h26l87r6i5kwfl68x3-dbus-broker-37/bin/dbus-broker-launch --scope user
  aktau     2572 dbus-broker --log 11 --controller 10 --machine-id 2fc5204c4c03482f8844a9c832f262e1 --max-bytes 100000000000000 --max-fds 25000000000000 --max-matches 5000000000

While the invidividual processes are either the same or slightly smaller than dbus-daemon, since there is more of then, we reach: 21965 KiB = 21 MiB