build_release_files: read the linker memory table when size.json is missing - #11314
Conversation
dhalbert
left a comment
dhalbert
left a comment
There was a problem hiding this comment.
Should we consider just fixing the ports that fail to generate firmware.size.json, or alternatively always looking in make_output for the size info?
Seven ports never write firmware.size.json, so build_release_files.py cannot tell whether the en_US build left room and builds all 17 languages for every board of those ports on every pull request. They cannot use this script as it stands: it reads the region size out of a linker script, which needs the size to be a literal there. raspberrypi writes LENGTH = firmware_size, a symbol the board can override; broadcom, cxd56 and silabs link a script from a submodule or an SDK; zephyr-cp has no script of its own at all. The linker map has the same numbers with the symbols already resolved, and every one of these ports either writes a map already or is one flag away from it. So read the Memory Configuration table when the file has one, and keep the linker script path for the ports that pass one. The region name is FLASH_FIRMWARE or FLASH, or --region for anything else. Wired into raspberrypi, renode, mimxrt10xx, broadcom, cxd56 and silabs, which only needed -Wl,-Map added. A missing map or an unknown region name prints what was found and writes nothing, so the language skip falls back to today's behaviour instead of failing the build. zephyr-cp still needs doing; its Makefile has none of the toolchain variables.
Its makefile has none of the toolchain variables the other ports get from circuitpy_mkenv.mk, so there is no size(1) to pipe in. There does not need to be: what the image occupies in flash is the size of the binary, which is how ports/espressif has always measured it. Pass --image and the region still comes from the map. Measured on rpi_pico: 502256 bytes used of a 1040128 byte region, against the 502000 and 1040128 Zephyr prints itself -- the difference is the 256 byte second stage, which the .bin includes and the FLASH region does not.
Neither has a flash firmware region to measure against. cxd56 links the image into RAM through the Spresense SDK script, so its map has one region and it is called ram; broadcom boots kernel.img off an SD card and its regions are RAM and READONLY. Nothing there constrains the firmware, so there is no headroom to compute and the step only printed what it could not find.
fc7ff89 to
48e5ad1
Compare
atmel-samd, analog, litex, nordic and stm were still passing their linker script, which left the script answering the same question two ways. Their maps carry the same numbers -- the script computed them, ld resolved them -- so point those five at the map as well and the linker script parser, the K and M suffix handling and the eval that needed them all come out. What a port supplies is now one rule: the map for the region, size(1) or --image for what is used.
|
Had to dig deeper on this 🙂. Unfortunately I could not unify everything on the So I looked how to generate firmware.size.json instead. The current way does not work for raspberrypi, whose linker script says But the linker map is the third way to get the numbers. ld resolves the region lengths there, and it is a file rather than build output. I verified it works on the ports that were missing it. Then I used the same route for the ports that already worked, so it is uniform. The only exception is espressif (again - no ld region). It keeps its own script, but it writes the same firmware.size.json as everything else, so there is still just one interface. |
|
Results from this PR CI run:
|
dhalbert
left a comment
dhalbert
left a comment
There was a problem hiding this comment.
Thank you so much. This is tremendous work!
2 participants
On pull requests the languages other than en_US are skipped when the en_US build leaves 10 KB of headroom. That reads firmware.size.json, which raspberrypi, mimxrt10xx, broadcom, silabs, cxd56, renode and zephyr-cp never write, so every PR builds all 17 languages for every board of those ports; a Pico 2 with 190 KB free included. Their link step prints the ld memory usage table (FLASH_FIRMWARE: 1378984 B 1532 KB 87.90%), so when size.json is missing the used and region sizes are taken from that line instead.