section "priv_stacks_noinit" LMA address seems to be configured as loaded from ROM, which could be removed to save flash space

[pasuzw]

Hi There

linker file:

• lib\Nordic_nRF_Connect_SDK\zephyr\include\zephyr\linker\kobject-priv-stacks.ld

section "priv_stacks_noinit" is managed in above ld file.
At the end of that file, it is defined like this:

_} GROUP_DATA_LINK_IN(RAMABLE_REGION, ROMABLE_REGION)_

And in my environment: CONFIG_XIP=y, so the following applied:

#elif defined(CONFIG_XIP) || defined(Z_VM_KERNEL)
#define GROUP_DATA_LINK_IN(vregion, lregion) > vregion AT > lregion

Therefore, the privileged stack would exist in both ROM and RAM, and copied from ROM to RAM at power on.

And I checked the commit message and it doesn't show the necessity of loading it from ROM.

Is there a specific reason why it must be loaded from ROM ?
At this moment, if a user wants to increase CONFIG_PRIVILEGED_STACK_SIZE, the ROM size would be increased as well due to above reason.

[cookesan]

Your macro expansion is right, but I would not assume it is actually consuming initialized flash bytes just from the AT > ROMABLE_REGION text.

That section is declared as:

SECTION_DATA_PROLOGUE(priv_stacks_noinit,(NOLOAD),)
...
} GROUP_DATA_LINK_IN(RAMABLE_REGION, ROMABLE_REGION)

Under XIP, GROUP_DATA_LINK_IN does expand to > RAMABLE_REGION AT > ROMABLE_REGION. However the output section is also (NOLOAD), so the linker should treat it as non-loaded/no initialized contents. The newer CMake linker-script version also classifies .priv_stacks_noinit as NOINIT_REGION, NOINPUT, NOINIT, which points in the same direction.

What is confusing is that the legacy .ld file uses the data-group macro while common-noinit.ld uses GROUP_NOLOAD_LINK_IN, which under XIP maps no-load sections to > RAMABLE_REGION AT > RAMABLE_REGION. So a cleanup to use the no-load group may be reasonable for clarity/consistency, but I would verify the actual image impact first with the generated map/objdump -h/ROM report. If the section is NOBITS/not in a load segment, changing the macro may not save flash even though it makes the script less misleading.

[pasuzw]

Thanks for pointing out for the (NOLOAD) option, which made me realize that my zephyr version 3.3.9 doesn't have that fix. And I tried it but it didn't work, please refer to my reply to dcpleung.

[dcpleung]

Could you use tools like readelf (e.g. readelf -S zephyr.elf) to see whether the privileged stack area is PROGBITS or NOBITS? The Off (offset) column on the output will also tell you if the section is occupying any ROM space. You can look at the .noinit section as reference.

One thing though, even if the section is not occupying space in the ELF file, it is possible that your ELF to flashing binary file conversion is adding that into the binary file.

[dcpleung]

The offset column might be misleading. I tried building west build -p -b mps2/an385 tests/kernel/mem_protect/mem_protect/ and I got this:

  [36] bss               NOBITS          20001b80 019b80 00181b 00  WA  0   0 32
  [37] noinit            NOBITS          20003800 019b80 005200 00  WA  0   0 2048
  [38] priv_stacks_noinit NOBITS          20008a00 019b80 002200 00  WA  0   0 64

However, when I looked at the ELF file, the offset points to the middle of the string. So that offset does not seem to be usable.

Are you sure it is actually occupying ROM space? I tried changing GROUP_DATA_LINK_IN to GROUP_NOLOAD_LINK_IN. Although the load address line is no longer in the map file, the readelf symbol lists were the same for both ELFs. And the sizes of both ELF files were the same.

[pasuzw]

Sum up first:
I feel with NOLOAD and GROUP_DATA_LINK_IN, 'priv_stacks_noinit' is not occupying the ROM space or say not flashed into the microcontroller ROM. However, because of GROUP_DATA_LINK_IN defining a LMA address in rom, the linker script(ld.bfd.exe) will count 'priv_stacks_noinit' in for my appflash, and then check if my appflash is full.

Thus I propose to use GROUP_NOLOAD_LINK_IN in the lastest zephyr rtos.

Here is today's update:
1. I tried to use Nordic SDK v3.2.3 which uses Zephyr 4.2.99 in my environment, which means I have

SECTION_DATA_PROLOGUE(priv_stacks_noinit,(NOLOAD),)
...
} GROUP_DATA_LINK_IN(RAMABLE_REGION, ROMABLE_REGION)

but it didn't fix my problem.

-- if I increase CONFIG_PRIVILEGED_STACK_SIZE to 4k, I still get the following error of section priv_stacks_noinit' will not fit in region appflash', which is reported by ld.bfd.exe from zephyr-sdk.

My understanding about this: Even I use latest zephyr rtos, it cannot fix the issue on my side.

2. I made a table to compare the appflash usage and rom report in my earlier environment using zephyr 3.3.99.

My understanding is:

• the section option "NOLOAD" will affect the final rom report, which means as long as we add "NOLOAD", it won't occupy microcontroller ROM after flash. Otherwise, it will occupy.

• during linking, the appflash size check or calculation is not dependent on "NOLOAD", but depends on LMA address.

Note 1: The appflash usage meaning the used flash size calculated during linking stage.

Note 2: The ROM_REPORT is the total used rom size from: west build -t rom_report.

3. I have performed objdump on the elf file in various configs:

• Config1: secition option is empty and use GROUP_DATA_LINK_IN:

My understanding: _priv_stacks_noinit_ has a rom LMA address because there is no option NOLOAD. It is close to "datas" section.

---

• Config2: secition option is NOLOAD and use GROUP_DATA_LINK_IN:

My takeaway1: - _priv_stacks_noinit_ no longer has LOAD and DATA and CONTENTS labels due to section option NOLOAD. Now it is close to noinit, except that its LMA address is still a valid ROM address. My takeaway2: - Notice that the 31 datas section's LMA address(0x79ed5) is after _priv_stacks_noinit_ (= 0x78555+0x1980). I think the ld.bfd.exe is calculating the appflash size based on the rom location counter which depends on LMA address. It doesn't care about NOLOAD option at that point. My takeaway3: - This is the same config with the lastest zephyr rtos. It is important.

---

• Config3: secition option is NOLOAD and use GROUP_NOLOAD_LINK_IN:

My understanding: - This is ideal , LMA address is same with VMA. - My problem is gone with this config after testing.

---

• Config4: secition option is empty and use GROUP_NOLOAD_LINK_IN:

My understanding: a weild config, VMA = LMA = ram address. but with LOAD, DATA lables, shall not use this.

[dcpleung]

We can always change it to use GROUP_NOLOAD_LINK_IN(). But I think a bug report also needed to filed against appflash. The section is marked as NOBITS which should never have included in the calculation.

[pasuzw]

I will probably make a patch in my project for now. In a long term, hopefully, it could be changed in the Zephyr as there is no benefit to use a LMA address in ROM for priv_stack_noinit.

Since my appflash overflow is reported by c:/buildtools/nordic_nrf_connect_sdk_toolchain/v3.2.3/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd.exe: , I am not sure about where the bug report you mentioned should go to. Isn't this the generic behavior of gnu ld.exe ?

P.S There is a similar one raised in GNU Binutils bug report system shown in the following link, but it is not really processed further.

• https://sourceware.org/bugzilla/show_bug.cgi?id=16127

[dcpleung]

You can submit a patch on main since you discovered this issue. If you are using a specific version of Zephyr (dictated by the nrf connect sdk), we can try backporting it.