build system: use thread-safe stdio

[maribu]

Contribution description

build system: introduce bug_% feature category

This introduces a new feature category to declare which bugs are present
in a given build that we cannot fix in RIOT, but need to work around.
These bugs may be silicon bugs, software bugs in ROM, software bugs in
binary blobs needed for a platform, or bugs in the toolchain.

These features behave the same way as arch_% features: Every provided
bug is always used, so that we can inspect FEATURES_USED to check
which bugs need to be worked around.

build system: use thread-safe stdio

This makes use of the new bug modeling to declare all platforms that
can use newlib and have no reentrancy hooks as affected by the
non-thread-safe stdio bug. (Which is every platform but ESP* and AVR,
the former because the reentrancy hooks are provided, the latter because
we do not and never will support newlib on them.)

Building on that, the mpaland-printf package is used when newlib is used
and the bug is present. This way we can rely on the stdio being
thread-safe on every platform and not causing random crashes at
run time.

sys/newlib: drop workaround for stdio

This drops a workaround that initialized newlib's reentrancy structure
on boot to reduce the chances of crashes when using the non-thread-safe
(unless reentrancy hooks are provided) stdio implementation of newlib.

Now that the newlib stdio implementation is only ever used if it is
thread-safe, we no longer need a workaround that reduces the chance
of crashes on concurrent use of stdio.

Testing procedure

Now the package mpaland-printf should be pulled in for all non-ESP boards when newlib is used. E.g.

for board in hifive1b same54-xpro native32 native64 arduino-mega2560 esp32-mh-et-live-minikit esp8266-esp-12x z1 nucleo-l011k4; do
    printf '\n%s\n===================\n' $board
    echo "Features:"
    make BOARD=$board --no-print-directory -C examples/basic/hello-world info-features-used | grep -E 'newlib|libc'
    echo "Packages:"
    make BOARD=$board --no-print-directory -C examples/basic/hello-world info-packages
done

hifive1b
===================
Features:
bug_newlib_broken_stdio
newlib
Packages:
mpaland-printf

same54-xpro
===================
Features:
bug_newlib_broken_stdio
newlib
Packages:
cmsis
mpaland-printf

native32
===================
Features:
Packages:

native64
===================
Features:
Packages:

arduino-mega2560
===================
Features:
Packages:

esp32-mh-et-live-minikit
===================
Features:
newlib
Packages:
esp32_sdk

esp8266-esp-12x
===================
Features:
newlib
Packages:
esp8266_sdk

z1
===================
Features:
bug_newlib_broken_stdio
Packages:
mpaland-printf

nucleo-l011k4
===================
Features:
bug_newlib_broken_stdio
picolibc
Packages:
cmsis

With this, the test in tests/sys/snprintf now should also pass for each and every board, except for ESP32 boards. The reason is that, since ESP* provides the reentrancy hooks, newlib's stdio is thread-safe on those. So we do not pull in the alternative printf. For ESP8266 boards newlib is configured to use the non-nano stdio, which passes the test. For ESP32 newlib-nano is used, which does not implement all format specifiers and, therefore, fails the test.

Issues/PRs references

Depends on and includes:

• pkg/mpaland-printf: fix overriding printf with printf_float #21439

[riot-ci]

Murdock results

✔️ PASSED

bb76083 sys/newlib: drop workaround for stdio

Success	Failures	Total	Runtime
10319	0	10320	12m:25s

Artifacts

• Documentation preview

[crasbe]

We should probably give this a full build without fast fail, otherwise you'll be chasing your tail for days 😅

[maribu]

We should probably give this a full build without fast fail, otherwise you'll be chasing your tail for days 😅

No, actually the memory requirements will go down with this. Unless... both printf variants get linked in into the same app.

See #21439 for a fix of that.

[Enoch247]

in one of your commit messages and in the description of this PR you have every misspelled.

This way we can rely on the stdio being thread-safe on evyer platform and not causing random crashes at run time.

[maribu]

For MSP430 both .text and .data slightly increase with this. This is not expected and needs to be investigated first.

[maribu]

Ah, 64 bit math is expensive on MSP430. Disabling support for long long and .data stays the same and .text goes down when switching to mpaland-printf.

I was hoping that mpaland-printf would be smaller for all platforms while supporting all features. But maybe support for printing long long should be a module to not cause regressions for MSP430. It could still be a default module on 32 bit and 64 bit platforms, were support for long long is super cheap.

[crasbe]

Btw: you can use the built-in function to compare Build Sizes: https://doc.riot-os.org/advanced-build-system-tricks.html#comparing-build-sizes

Someone (👀👀) recently fixed it :D

[crasbe]

I didn't want to install the RISC-V and ESP32 toolchains as well, so this is just a smaller set.

cbuec@W11nMate:~/RIOTstuff/riot-mpaland/RIOT/examples/basic/hello-world$ OLDBIN=master NEWBIN=mpaland make info-buildsizes-diff
using BOARD="native64" as "native" on a 64-bit system
text    data    bss     dec     BOARD/BINDIRBASE

22      0       0       22      arduino-mega2560
5532    118     904     6554    master
5554    118     904     6576    mpaland

24      0       0       24      native64
27566   1080    59128   87774   master
27590   1080    59128   87798   mpaland

20      0       0       20      nucleo-l011k4
5996    24      1772    7792    master
6016    24      1772    7812    mpaland

-3188   -108    -8      -3304   same54-xpro
12632   116     2308    15056   master
9444    8       2300    11752   mpaland

[maribu]

I think there is missing a RIOT_CI_BUILD=1,as the longer version (which includes the branch name unless it is master) is probably the cause for the size diff on AVR. (avrlibc's stdio is thread safe naturally and better optimized for 8 bit platforms than mpaland-printf, so sticking with the native printf there makes sense and is done here.) Same with the nucleo-l011k, which only supports picolibc. And on picolibc the native printf is also excellent.

[crasbe]

Yes, with RIOT_CI_BUILD=1 the sizes are identical for the other boards.

[maribu]

#21445 now makes 64 bit printing support optional. It makes that support opt-in for MSP430 (due it being expensive there), and opt-out for 32-bit systems (due to it being cheap there).