Large filesizes with C++ projects?

[tobybear]

I have a question about filesizes when creating C++ code.

I have a C++ project that uses some parts of the STL library quite a lot, particularly std::string, std::vectorstd::string and std::map<std::string, std::string>.
Now the good thing is when I compile this using cosmoc++, it creates a binary that runs fine on the systems I can test with (Windows, MacOS, Ubuntu).

The resulting filesize of the binary however is quite big with 15 MB. Even when adding the -mtiny flag, it only comes down to 14 MB.
Only when manually adding these command line parameters I can trim it further down:
-Os (optimize for size)
-s (strip debug symbols)
-fdata-sections -ffunction-sections -Wl,--gc-sections (remove uncalled function code)
This will result in an APE binary that is "only" 1.8 MB and still seems to run the same.

However, compiling the project natively with g++ for each platform results in a binary of 90k (Windows) to 200k (Linux, MacOS).
So I don't see much advantage in compiling the project binary itself as an APE, but I am rather thinking of using APE as a simple shell to call the platform specific binaries that I'll attach with the zip feature piggy-back style, to have a smaller filesize but still a single user-facing binary.

Or did I overlook something to properly make C++ code created cosmoc++ smaller?
I assume the big filesize comes from the C++ template code being directly linked in, while the native g++ builds dynamically call the C++ runtime of the OS (i.e. /lib/x86_64-linux-gnu/libstdc++.so.6 on Linux and msvcrt.dll on Windows).
I tried the CTL approach as well, but it did not result in much smaller files, required lots of changes in existing code and crashed in certain places with a segfault.

Any idea what I could do to make it smaller in size?

[jart]

Use C.

You nailed most of the things you can do to make C++ binaries smaller. Beyond that you can analyze what's being linked into your program. I have this command ~/bin/bloat that I use a lot on the .dbg files:

#!/bin/sh
nm -C --size "$@" |
  sort -r |
  grep ' [bBtTRr] ' |
  exec less

That'll hopefully give you a better idea of what sort of stuff the linker is pulling into the binary.

[tobybear]

Thanks for that command. It won't work on APE binaries (naturally), and also not on stripped builds, but for unstripped builds (like the intermediate elf from cosmo or a direct gcc compile) it shows me what I expected - no external dependencies apart from my own code and the STL string/map/vector implementations.
It is just a bit confusing to me that the cosmo APE build is so much larger in size, but I assume it is mostly because the same code is contained several times for different target platforms.

One thing I noticed though when comparing the cosmoc++ elf with the one generated by g++ directly:
Only on the cosmo binary, the third largest section from nm displayed is this:
0000000000008208 b __deadlock
It is not found in the build made with g++.
Anyone knows what this is?

[jart]

That b means it's BSS memory so it doesn't contribute to binary size. It's used for the posix mutex deadlock detector.

[tobybear]

Ok, thanks!
So here is an example output of the 8 largest sections of one of my binaries, using your bloat script.
Comparisons are between cosmoc++ v14.1.0 (run on out.exe.aarch64.elf) and g++ 13.3.0, compiled on the same Ubuntu 13.3 system/x64. I added the first column to give decimal size in kb.

As you can see, the same 8 (internal) functions are present (albeit in different order), but the section size is considerably larger on cosmoc++. I just want to know if there is anything sensible to do about this from my end or if I should just leave it as it is :)

cosmoc++:

40 9a80 T WockyInterpreterBase::run(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, bool)
36 8dd8 T Tiffy::parse(WockyClient&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::vector<wockyResult, std::__1::allocator<wockyResult> >&, bool)
33 80a8 T WockyInterpreterBase::evaluate(WockyClient&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&)
22 56d0 T WockyInterpreterBase::match(WockyClient&, wockyResult&, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)
19 498c T WockyInterpreterBase::matcher(WockyClient&, wockyResult&, std::__1::vector<wockyResult, std::__1::allocator<wockyResult> >&)
17 42ec T WockyInterpreterBase::getObjectDescription(WockyClient&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int, bool, std::__1::set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >*)
17 40c4 T WockyInterpreterBase::loadWorld(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)
13 3334 T main

g++:

25 60ec T Tiffy::parse(WockyClient&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<wockyResult, std::allocator<wockyResult> >&, bool)
22 5596 T WockyInterpreterBase::evaluate(WockyClient&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)
19 4c46 T WockyInterpreterBase::run(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, bool)
13 3297 T WockyInterpreterBase::loadWorld(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)
11 2aba T WockyInterpreterBase::match(WockyClient&, wockyResult&, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)
 9 2410 T WockyInterpreterBase::getObjectDescription(WockyClient&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, bool, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >*)
 9 2380 T WockyInterpreterBase::matcher(WockyClient&, wockyResult&, std::vector<wockyResult, std::allocator<wockyResult> >&)
 5 15eb T main

[tobybear]

As you wish, attached is the full output of bloat from a cosmoc++ dbg file vs. the same compiled with g++
bloat_cosmoc++.txt
bloat_g++.txt

And the corresponding command line calls for the compilers:
g++ main.cpp -o tiffy.out -Os -std=c++11

cosmoc++ -fdata-sections -ffunction-sections main.cpp -o tiffy.exe -mtiny -Os -s -std=c++11 -Wno-literal-suffix -Wl,--gc-sections

[jart]

It's mostly your code that's contributing to the binary size. For example, your WockyInterpreterBase::run() function alone is 43kb. Since cosmoc++ creates a binary that includes both an AMD64 and ARM64 build of your program, it means that function is actually 86kb. So that one function alone is 1/20th of your binary size. Read https://justine.lol/sizetricks/

[tobybear]

Yeah, that is what I suspected as well. I'll see if I can cut down on the function code of my own sources (it is mostly code that is 30 years old 😂 ).
Thanks for your help and insights and links!

[tobybear]

Just a quick note of my intermediate findings:
I was able to reduce the file size in tiny bits by replacing some of my internal functions, but the biggest reduction came when I added -fno-exceptions to the cosmoc++ call.
Since I don't use any exceptions myself in the code and don't catch any, I thought why not give it a try. And that parameter alone reduced the size by about 300kb :)

[jart]

Oh yeah, that DWARF exception handling part of the binary can be brutal. Even C code needs to be compiled to include that bloat. I'll give some thought to bundling a cosmo.a file for folks who link with -mtiny -fno-exceptions.

[tobybear]

Using further g++ compiler flags for disabling unwanted features I have now reduced the file size fom 1.8 MB to 1.3 MB, which is quite nice. :)

Two more questions:

• Is there a way to disable the error message strings that are part of the strerror function? As soon as I include cstring (or string.h), these get included, but I only need memcpy and memset from that header. In the binary there is always a big blob with these error messages.
• I replaced all instances of printf and sprintf in my code with write(1, ...), but that did nothing to the filesize. Looking at the bloat script output, I see sections for __vfprintf_flbuf, __kvprintf, vfprintf, vsnprintf, __kprintf, __ksnprintf, __kvsnprintf. Any chance to see which call pulls these in or if they can be disabled?

Thanks!

[jart]

5ddb5c2 made size improvements to strerror() that'll be rolled out into cosmocc soon. You need to hunt down why it's being pulled into the linkage. Cosmopolitan headers don't link things. But stuff can be linked in sneaky ways. To figure it out you need to pass --cref -Map=prog.map to the linker. If you somehow manage to not depend on __fmt() then you can probably trim another ~70kb off the file size. You can use double_conversion directly for floating point number coding, which is smaller and faster.

[tobybear]

Further findings (not sure if helpful or interesting to you):

• the message strings for strerror() get pulled in as soon as I use C++'s "new" - using "malloc" does not, but that is hardly feasible with the large C++ code base for the project I am working on at the moment.

• "_fmt" also gets pulled in as soon as I use new(), or any class that is using "new" internally like std::string.

• I did not succeed in adding "--cref -Map=prog.map" to the linker. I tried LDFLAGS as well as putting it directly in the gcc linker (and also the apelink) command in the cosmocc script ("command --cref not found"). Any insights?

[tobybear]

And further down the rabbit hole - who knew code golfing could be so much fun? :)
So after some larger efforts that only shaved off some bytes here and there, the biggest breakthrough for me was simply to overload new and delete with my own functions that just map to malloc/free - that alone reduced the size of the APE by several 100 kB!
I then replaced the std::vector, map, set and string classes with those from a minimal STL subset I had implemented years ago for game development on PS1 and Gamecube games, and now I am at around 600 kB - not bad for a file that started with 15 MB in size, and without sacrificing any functionality.

My next plan is to move some string tables / const char* definitions and static strings from my code to an eternal file that I'll include with the zip mechanism. This should then at least prevent them from appearing in the binary twice, as it is at the moment.