Skip to content

Commit ae3b768

Browse files
luatorluator
authoredJul 22, 2024··
Merge pull request #132 from open-dynamic-robot-initiative/fkloss/doc_rt
doc: Update RT kernel pages
2 parents 934eff7 + 6bf7e15 commit ae3b768

File tree

3 files changed

+171
-88
lines changed

3 files changed

+171
-88
lines changed
 

‎doc/about_architecture.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.. _architecture::
1+
.. _architecture:
22

33
About the Software Architecture
44
===============================

‎doc/preempt_rt_kernel.rst

+40-67
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,16 @@
11
:orphan:
22

3-
***********************
4-
PREEMPT_RT Linux Kernel
5-
***********************
3+
*********************************************
4+
Build and Install the PREEMPT_RT Linux Kernel
5+
*********************************************
66

77
To get good real-time performance on Linux, we recommend using a Linux kernel
88
with the `PREEMPT_RT patch`_. Unfortunately, this requires one to build the kernel
99
from source (at least on Ubuntu). We provide some help for this in the
1010
following.
1111

12-
Note that this page only covers installation of the kernel, for further
13-
real-time-related configuration of the system see :doc:`real_time`.
14-
15-
Unfortunately, Nvidia does not support the PREEMPT_RT patch, so you may not be
16-
able to use Nvidia drivers (and thus also no CUDA) when using this kernel.
12+
Note that Nvidia drivers do officially not support the PREEMPT_RT patch. It may still
13+
work (see :ref:`nvidia_preempt_rt` below) but you are at your own risk.
1714

1815
An alternative may be the "lowlatency" kernel. It is easier to install and
1916
works with Nvidia drivers but has weaker real-time capabilities (see
@@ -24,86 +21,62 @@ Install PREEMPT_RT Kernel
2421
=========================
2522

2623
To install the patched kernel on Ubuntu, you may use our script
27-
install_rt_preempt_kernel.sh_. It is tested for Ubuntu 18.04 but may also work
28-
with other recent versions.
29-
30-
Before running the script, you may want to modify it a bit, though:
24+
``install_rt_preempt_kernel.sh``. We provide adapted versions for different Ubuntu
25+
versions, which can be found `here <install_rt_preempt_kernel.sh_>`__.
3126

32-
- The ``VERSION_`` variables at the top refer to the kernel version that will be
33-
installed. Usually you can leave the default values. See
34-
preempt_rt_versions_ for available versions in case you want to change it.
27+
The ``VERSION_`` variables at the top of the script refer to the kernel version that
28+
will be installed. Usually you can leave the default values, but you can change it here
29+
if you want a different version. See preempt_rt_versions_ for available versions.
3530

3631
Then simply execute the script in a terminal. Internally, sudo is used in some
3732
steps, so the user executing it needs to have sudo-permission.
3833

39-
In the beginning (after downloading some things) you will be asked to manually
40-
adjust some configuration settings. Before entering the menu, the script prints
41-
the following instructions::
34+
.. note::
4235

43-
Please apply the following configurations in the next step:
36+
In the beginning (after downloading some things) you will be asked to manually
37+
adjust some configuration settings. Before entering the menu, the script prints
38+
instructions like the following::
4439

45-
General setup [Enter]
46-
Local version - append to kernel release: [Enter] Add '-preempt-rt'
40+
Please apply the following configurations in the next step:
4741

48-
General setup [Enter]
49-
Preemption Model (Voluntary Kernel Preemption (Desktop)) [Enter]
50-
Fully Preemptible Kernel (RT) [Enter] #Select
42+
[...]
5143

52-
However, depending on the kernel version the "Preemption Model" setting may be
53-
found in the "Processor type and features" menu instead.
44+
General setup [Enter]
45+
Preemption Model (Voluntary Kernel Preemption (Desktop)) [Enter]
46+
Fully Preemptible Kernel (RT) [Enter] #Select
47+
48+
However, depending on the kernel version the "Preemption Model" setting may be
49+
found in the "Processor type and features" menu instead.
5450

5551
The script will automatically download, build and install the kernel. This will
5652
take a while.
5753

54+
When finished, go back to :doc:`real_time` and follow the further steps to configure
55+
your system for real-time usage.
5856

59-
.. _boot_rt_kernel:
60-
61-
Boot with the PREEMPT_RT Kernel
62-
===============================
63-
64-
Once the PREEMPT_RT kernel is installed, you need to reboot and select the
65-
"preempt-rt" kernel in the GRUB menu (go to "Advanced options for Ubuntu", it
66-
should be listed there).
67-
68-
When the system is running, you can check which kernel is running with
69-
``uname -a``. It should print something containing "PREEMPT_RT".
70-
71-
72-
Select PREEMPT_RT Kernel by Default
73-
-----------------------------------
74-
75-
You can configure GRUB to automatically select this kernel by setting
76-
``GRUB_DEFAULT`` in ``/etc/default/grub``. For this, first the identifier of
77-
the kernel needs to be determined. Open a terminal and run
78-
79-
.. code-block:: bash
80-
81-
cat /boot/grub/grub.cfg | grep -w -e menuentry -e submenu
82-
83-
It should print something like this::
84-
85-
menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-1a26991b-b045-48dd-bb12-064a2725b80b' {
86-
submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1a26991b-b045-48dd-bb12-064a2725b80b' {
87-
menuentry 'Ubuntu, with Linux 5.4.93-rt51-preempt-rt' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.4.93-rt51-preempt-rt-advanced-1a26991b-b045-48dd-bb12-064a2725b80b' {
88-
menuentry 'Ubuntu, with Linux 5.4.93-rt51-preempt-rt (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.4.93-rt51-preempt-rt-recovery-1a26991b-b045-48dd-bb12-064a2725b80b' {
89-
menuentry 'Ubuntu, with Linux 5.4.0-65-generic' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.4.0-65-generic-advanced-1a26991b-b045-48dd-bb12-064a2725b80b' {
90-
menuentry 'Ubuntu, with Linux 5.4.0-65-generic (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.4.0-65-generic-recovery-1a26991b-b045-48dd-bb12-064a2725b80b' {
9157

58+
.. _nvidia_preempt_rt:
9259

93-
For ``GRUB_DEFAULT`` the path full submenu/menuentry is needed, using the id of
94-
each step (the last part in the line, "gnulinux-..."), separated by ">". In
95-
this specific case, the setting for starting the rt-kernel would be::
60+
Using Nvidia Drivers with the PREEMPT_RT Kernel
61+
===============================================
9662

97-
GRUB_DEFAULT = "gnulinux-advanced-1a26991b-b045-48dd-bb12-064a2725b80b>gnulinux-5.4.93-rt51-preempt-rt-advanced-471e9718-013f-4cbb-91a7-d22635173b70"
63+
Officially, Nvidia drivers do not support the PREEMPT_RT kernel. However, at least with
64+
more recent versions, it seems to work in practice. However, you need set the
65+
environment variable ``IGNORE_PREEMPT_RT_PRESENCE=1`` when installing it with apt.
66+
Complete steps:
9867

99-
After saving the changes in ``/etc/default/grub`` you need to run the following
100-
command for the changes to become active::
68+
1. First uninstall any Nvidia drivers
69+
2. Install the PREEMPT_RT kernel (see above)
70+
3. Install the drivers with the following command (adjust driver version to the desired
71+
one)::
10172

102-
sudo update-grub
73+
sudo IGNORE_PREEMPT_RT_PRESENCE=1 apt install nvidia-driver-530
10374

104-
Then reboot and verify that the correct kernel is used.
75+
Please note that this variable also needs to be set when upgrading packages, so you may
76+
want to set it in a global place like `/etc/environment` and disable unattended upgrades
77+
for the driver.
10578

10679

10780
.. _PREEMPT_RT patch: https://wiki.linuxfoundation.org/realtime/start
108-
.. _install_rt_preempt_kernel.sh: https://github.com/machines-in-motion/ubuntu_installation_scripts/blob/master/rt-preempt/ubuntu18.04/install_rt_preempt_kernel.sh
81+
.. _install_rt_preempt_kernel.sh: https://github.com/machines-in-motion/ubuntu_installation_scripts/tree/master/rt-preempt
10982
.. _preempt_rt_versions: https://wiki.linuxfoundation.org/realtime/preempt_rt_versions

‎doc/real_time.rst

+130-20
Original file line numberDiff line numberDiff line change
@@ -6,36 +6,144 @@ Since a constant update rate of the :cpp:class:`~robot_interfaces::RobotBackend`
66
is important for stable control of the robot, a real-time capable operating
77
system should be used.
88

9-
We are using Ubuntu with a PREEMPT_RT kernel patch. Below are instructions on
10-
how to install the patched kernel and how to configure the system to use it.
9+
For this we are using a standard Linux distribution (Ubuntu in our case) but install a
10+
kernel with better real-time capabilities. There are two options: the "low-latency"
11+
kernel which is easy to install but doesn't give full real-time guarantees, and the
12+
preempt_rt kernel, which has better real-time performance but is more tedious to
13+
install.
1114

15+
Below are instructions for both of them and on how to configure the system to use them.
1216

13-
Install PREEMPT_RT Kernel
14-
=========================
1517

16-
See :doc:`preempt_rt_kernel`.
18+
.. _lowlatency_kernel:
1719

20+
Installation: Low-latency vs PREEMPT_RT Kernel
21+
==============================================
1822

19-
.. _lowlatency_kernel:
23+
We recommend one of the following Linux kernel variants:
24+
25+
**low-latency:**
26+
27+
- Fewer latency guarantees as with PREEMPT_RT but often good enough. Latency can
28+
increase if the system load is high, though. It can be useful to assign separate CPU
29+
cores for the real-time critical processes.
30+
- Can be used with Nvidia drivers.
31+
- Easy to install. On Ubuntu it's just::
32+
33+
sudo apt install linux-lowlatency
34+
35+
36+
**PREEMPT_RT patch:**
37+
38+
- Better real-time guarantees.
39+
- Officially not supported by Nvidia drivers (but may still kind of work, see
40+
:ref:`nvidia_preempt_rt`).
41+
- Difficult to install as it has to be built from source. See :doc:`preempt_rt_kernel`.
42+
43+
44+
Historically we used the PREEMPT_RT kernel (and still do on many of our robots) but
45+
lately also used the low-latency kernel in some projects without problems (you have to
46+
be a bit more careful to not overload the CPU, though).
47+
Since the low-latency Kernel is much easier to install, you may want to try with that
48+
one first and only move to the PREEMPT_RT version if you notice timing-related issues.
49+
50+
The further system configuration described in the following is the same in both cases.
51+
52+
53+
.. _boot_rt_kernel:
54+
55+
Boot with the real-time Kernel
56+
==============================
57+
58+
Once the real-time kernel is installed, you need to reboot and select the corresponding
59+
kernel ("preempt-rt" or "low-latency") kernel in the GRUB menu (go to "Advanced options
60+
for Ubuntu", it should be listed there).
61+
62+
When the system is running, you can check which kernel is running with
63+
``uname -a``. It should print something containing "PREEMPT_RT"/"lowlatency".
64+
65+
66+
Select real-time Kernel by Default
67+
----------------------------------
68+
69+
You can configure GRUB to automatically select a specific kernel when booting the
70+
computer. We provide a script to semi-automatically update the configuration (option
71+
1), but you may also edit it manually (option 2).
72+
73+
Option 1: Semi-automatic
74+
~~~~~~~~~~~~~~~~~~~~~~~~
75+
76+
We provide a script grub_select_default_kernel.py_ for this. Download it and execute
77+
with
2078

21-
Alternative: The Low Latency Kernel
22-
===================================
79+
.. code-block:: bash
2380
24-
A potential alternative to the PREEMPT_RT patch is using the "lowlatency"
25-
kernel. It is easier to install and does not conflict with Nvidia drivers. On
26-
the other hand, the real-time capabilities it provides are weaker than those of
27-
the PREEMPT_RT kernel. Whether it is sufficient for you depends on your
28-
application (e.g. how much computational load you put on the system while the
29-
robot is running). We still recommend using the PREEMPT_RT kernel but you may
30-
test with the lowlatency kernel first.
81+
sudo python3 grub_select_default_kernel.py
3182
32-
To install on Ubuntu 20.04 (on other versions of Ubuntu adjust the version at
33-
the end)::
83+
It will automatically parse the GRUB configuration and list the available boot options.
84+
Select the desired kernel and press enter. It will then show a diff of the change it
85+
intends to apply to ``/etc/default/grub``. **Review the diff carefully and only confirm
86+
if it looks good!** It should only modify the line starting with ``GRUB_DEFAULT``, i.e.
87+
like this:
3488

35-
sudo apt install linux-lowlatency-hwe-20.04
89+
.. code-block:: diff
90+
91+
--- current
92+
+++ new
93+
@@ -3,7 +3,7 @@
94+
# For full documentation of the options in this file, see:
95+
# info -f grub -n 'Simple configuration'
96+
97+
-GRUB_DEFAULT=0
98+
+GRUB_DEFAULT="gnulinux-advanced-cc71b1fb-b530-4694-a839-aecf600f1f49>gnulinux-5.15.0-112-generic-advanced-cc71b1fb-b530-4694-a839-aecf600f1f49"
99+
GRUB_TIMEOUT_STYLE=hidden
100+
GRUB_TIMEOUT=0
101+
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
102+
103+
If all is good and you confirmed, you still need to run
104+
105+
.. code-block:: bash
106+
107+
sudo update-grub
108+
109+
to apply the change. Then reboot and verify that the correct kernel is used.
110+
111+
112+
Option 2: Manually
113+
~~~~~~~~~~~~~~~~~~
114+
115+
If you don't want to use the script mentioned above, you may also edit the GRUB
116+
configuration manually.
117+
118+
For this, first the identifier of the kernel needs to be determined. Open a terminal and run
119+
120+
.. code-block:: bash
121+
122+
cat /boot/grub/grub.cfg | grep -w -e menuentry -e submenu
123+
124+
It should print something like this::
125+
126+
menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-1a26991b-b045-48dd-bb12-064a2725b80b' {
127+
submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1a26991b-b045-48dd-bb12-064a2725b80b' {
128+
menuentry 'Ubuntu, with Linux 5.4.93-rt51-preempt-rt' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.4.93-rt51-preempt-rt-advanced-1a26991b-b045-48dd-bb12-064a2725b80b' {
129+
menuentry 'Ubuntu, with Linux 5.4.93-rt51-preempt-rt (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.4.93-rt51-preempt-rt-recovery-1a26991b-b045-48dd-bb12-064a2725b80b' {
130+
menuentry 'Ubuntu, with Linux 5.4.0-65-generic' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.4.0-65-generic-advanced-1a26991b-b045-48dd-bb12-064a2725b80b' {
131+
menuentry 'Ubuntu, with Linux 5.4.0-65-generic (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.4.0-65-generic-recovery-1a26991b-b045-48dd-bb12-064a2725b80b' {
132+
133+
134+
For ``GRUB_DEFAULT`` the path full submenu/menuentry is needed, using the id of
135+
each step (the last part in the line, "gnulinux-..."), separated by ">". In
136+
this specific case, the setting for starting the rt-kernel would be::
137+
138+
GRUB_DEFAULT = "gnulinux-advanced-1a26991b-b045-48dd-bb12-064a2725b80b>gnulinux-5.4.93-rt51-preempt-rt-advanced-471e9718-013f-4cbb-91a7-d22635173b70"
139+
140+
After saving the changes in ``/etc/default/grub`` you need to run the following
141+
command for the changes to become active::
142+
143+
sudo update-grub
144+
145+
Then reboot and verify that the correct kernel is used.
36146

37-
Configuration to use this kernel by default works the same as for the PREEMPT_RT
38-
kernel, see :ref:`boot_rt_kernel`.
39147

40148

41149
System Configuration
@@ -120,3 +228,5 @@ following command::
120228
121229
Warning this thread is not going to be real time.
122230
231+
232+
.. _grub_select_default_kernel.py: https://github.com/machines-in-motion/ubuntu_installation_scripts/blob/master/rt-preempt/grub_select_default_kernel.py

0 commit comments

Comments
 (0)
Please sign in to comment.