Evaluate using Profile-Guided Optimization (PGO) and Post-Link Optimization (PLO) for Ustore

[zamazan4ik]

Hi!

Recently I did many Profile-Guided Optimization (PGO) benchmarks on multiple projects (including many databases like PostgreSQL, ClickHouse, Redis, MongoDB, etc.) - the results are available here. So that's why I think it's worth trying to apply PGO to Ustore to improve the database performance further.

I can suggest the following things to do:

• Evaluate PGO's results on Ustore.
• If PGO helps to achieve better performance - add a note to Ustore's documentation about that (probably somewhere in the README file). In this case, users and maintainers will be aware of another optimization opportunity for Ustore.
• Provide PGO integration into the build scripts. It can help users and maintainers easily apply PGO for their own workloads.
• Optimize prebuilt binaries with PGO.

Here are some examples of how PGO is already integrated into other projects' build scripts:

• Rustc: a CI script for the multi-stage build
• GCC:
  • Official docs, section "Building with profile feedback" (even AutoFDO build is supported)
  • A part in a "wonderful" configure script
• Clang: Docs
• Python:
  • CPython: README
  • Pyston: README
• Go: Bash script
• V8: Bazel flag
• ChakraCore: Scripts
• Chromium: Script
• Firefox: Docs
  • Thunderbird has PGO support too
• PHP - Makefile command and old Centminmod scripts
• MySQL: CMake script
• YugabyteDB: GitHub commit
• FoundationDB: Script
• Zstd: Makefile
• Foot: Scripts
• Windows Terminal: GitHub PR
• Pydantic-core: GitHub PR
• file.d: GitHub PR
• OceanBase: CMake flag
• ISPC: CMake scipts
• NodeJS: Configure script
• Android Open Source Project (AOSP):
  • Official documentation
  • Committed PGO profiles: repository
• DMD: Custom build rule
• LDC: GitHub action
• tsv-utils: Makefile
• Erlang OTP: Makefile
• Clingo (PGO enabled only in Spack): Package recipe
• SWI-Prolog:
  • Script
  • CMake module
• hck: Justfile

After PGO, I can suggest evaluating PLO with LLVM BOLT as an additional optimization step after PGO.

Also, I can suggest evaluating these optimization techniques for other projects like https://github.com/unum-cloud/usearch and https://github.com/unum-cloud/ucall (and maybe something else). If they are the libraries - having information that their performance can be optimized with PGO would be helpful for the users since in this case, they can consider enabling PGO for their projects when they use these libraries.