How to build the Toolchains with specific CFLAGS and CXXFLAGS?

[Krotti83]

I have some issues with the built binaries, especially GDB. In the default build I get an illegal instruction exception (SIGILL) when trying to use the simulator from GDB. Seems the binary from the simulator use some x86 AVX instructions which are not available on my very old system.

For non-RISC-V toolchains I pass the options with CFLAGS and CXXFLAGS to the configure script to the build system, where I define -march and -mtune. I have tried to pass the option in the configure script from this repository too, but these flags will be ignored.

[cmuellner]

Thanks for the report!

I would not expect a normal host GCC default build to emit AVX instructions. On x86_64, GCC normally defaults to a baseline such as -march=x86-64 -mtune=generic, so a SIGILL from AVX would usually point to one of these instead:

• the build inherited host flags such as -march=native, -mavx, or a distro/profile default like -march=x86-64-v3
• the illegal instruction is in a linked host library rather than in GDB/sim itself
• the compiler used to build the toolchain has a non-generic default
• stale build artifacts were reused from an earlier build with different flags

Could you check where the SIGILL happens?

gdb --args /path/to/riscv64-unknown-elf-gdb
run
x/8i $pc
bt
info sharedlibrary

If $pc is inside a shared library, then rebuilding GDB with different CFLAGS may not be sufficient; the corresponding host library would also need to be built for your older CPU.

Regarding flags: passing CFLAGS / CXXFLAGS to this repository's top-level ./configure is not the right mechanism, because those values are not generally propagated into each subproject configure invocation. For GDB, you can pass extra configure-time assignments through GDB_TARGET_FLAGS_EXTRA, for example:

make clean
make GDB_TARGET_FLAGS_EXTRA='CFLAGS="-O2 -march=x86-64 -mtune=generic" CXXFLAGS="-O2 -march=x86-64 -mtune=generic"'

For a Linux toolchain target, use the same make variable with make linux.

If you have already built once, please remove the stale GDB build directory/stamp or run make clean before retrying, the previous build may be reused.

[Krotti83]

Hi @cmuellner !

Thanks for the quick answer!

Could you check where the SIGILL happens?

I cannot figure that out currently, because the built gdb doesn't crash by itself. Only the simulator seems to be crashing. So I don't really know how to debug this behavior currently:

$ gdb `which riscv64-unknown-linux-gnu-gdb`
GNU gdb (Debian 16.3-1) 16.3
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /cross/gnu/riscv/bin/riscv64-unknown-linux-gnu-gdb...
(gdb) r -batch -ex "target sim" -ex "load" -ex "b _start" -ex "run" test.elf
Starting program: /cross/gnu/riscv/bin/riscv64-unknown-linux-gnu-gdb -batch -ex "target sim" -ex "load" -ex "b _start" -ex "run" test.elf
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[Detaching after vfork from child process 794005]
[New Thread 0x7ffff69ff6c0 (LWP 794006)]
[New Thread 0x7fffeffff6c0 (LWP 794007)]
[New Thread 0x7ffff61fe6c0 (LWP 794008)]
[New Thread 0x7ffff59fd6c0 (LWP 794009)]
Connected to the simulator.
Loading section .text, size 0x8 lma 100b0
Start address 100b0
Transfer rate: 64 bits in <1 sec.
Breakpoint 1 at 0x100b2

Program terminated with signal SIGILL, Illegal instruction.
The program no longer exists.
[Thread 0x7ffff59fd6c0 (LWP 794009) exited]
[Thread 0x7ffff61fe6c0 (LWP 794008) exited]
[Thread 0x7ffff69ff6c0 (LWP 794006) exited]
[Thread 0x7ffff7dd4240 (LWP 794002) exited]
[Thread 0x7fffeffff6c0 (LWP 794007) exited]
[New process 794002]
[Inferior 1 (process 794002) exited normally]
(gdb)

The simulator from GDB crashes when simple to try to run a simple code snippet like this:

.text

.globl _start
.func _start
.type _start,%function
_start:
    nop
1:
    nop
    j 1b

.endfunc

.end

Regarding flags: passing CFLAGS / CXXFLAGS to this repository's top-level ./configure is not the right mechanism, because > those values are not generally propagated into each subproject configure invocation. For GDB, you can pass extra >configure-time assignments through GDB_TARGET_FLAGS_EXTRA, for example:

Thanks will try it with these flags.

[cmuellner]

Thanks, this clarifies the situation.

test.elf is a RISC-V ELF file. Since this is a RISC-V cross toolchain, the binary is not meant to run on the x86 host CPU directly.

Your command uses GDB's built-in simulator: -ex "target sim" and the output confirms that path (Connected to the simulator). So the SIGILL here is not GDB itself crashing on an x86 AVX instruction. It is the RISC-V target program being run through GDB's internal simulator.

For a riscv64-unknown-linux-gnu toolchain, please use QEMU for execution. For example:

qemu-riscv64 -L "$(riscv64-unknown-linux-gnu-gcc -print-sysroot)" ./test.elf

If you need to debug it, start QEMU with its GDB stub (this opens port 1234 for GDB to connect):

qemu-riscv64 -L "$(riscv64-unknown-linux-gnu-gcc -print-sysroot)" -g 1234 ./test.elf

and attach with the cross GDB to port 1234 from another terminal:

  riscv64-unknown-linux-gnu-gdb ./test.elf
  (gdb) target remote :1234

GDB's internal target sim exists for some targets, including RISC-V in some builds, but it is not a replacement for QEMU/Linux execution. It is mainly useful for very limited bare-metal-style testing. For Linux userspace binaries, QEMU user-mode is the expected execution environment.

[Krotti83]

Your command uses GDB's built-in simulator: -ex "target sim" and the output confirms that path (Connected to the >simulator). So the SIGILL here is not GDB itself crashing on an x86 AVX instruction. It is the RISC-V target program being run >through GDB's internal simulator.

Okay, thanks for the information. I have ever thought GDB's internal simulator simulates for the specific architecture. RISC-V in this case. Yes, primarily I use an QEMU instance for this. Sorry, my mistake.

BTW:
I have seen in Makefile.in there are only GDB_TARGET_FLAGS_EXTRA, BINUTILS_TARGET_FLAGS_EXTRA and for each specific C library (glibc, musl, ...). But there is currently no GCC_TARGET_FLAGS_EXTRA. Is there a way without modifying the Makefile.in to add these build flags? Meanwhile I added GCC_TARGET_FLAGS_EXTRA in Makefile.in myself and currently start a new build for my old CPU.

[cmuellner]

No need to patch Makefile.in for this. The existing hook for GCC is GCC_EXTRA_CONFIGURE_FLAGS. For host compiler flags, you can pass configure variable assignments through it, e.g.:

make clean
make linux GCC_EXTRA_CONFIGURE_FLAGS='CFLAGS="-O2 -march=x86-64 -mtune=generic" CXXFLAGS="-O2 -march=x86-64 -mtune=generic"'

The name is intentional: it is not called GCC_TARGET_FLAGS_EXTRA because for GCC “target flags” would be easy to confuse with flags for code built for the RISC-V target. GCC_EXTRA_CONFIGURE_FLAGS means extra arguments passed to GCC's configure script, which can include regular configure options and configure variable assignments such as CFLAGS=... and CXXFLAGS=....