Skip to content

robotpy-build meson rewrite #252

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 71 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
5790fc2
Remove deprecated generate option
virtuald Dec 14, 2024
58e6211
Class name in YAML must have namespace now
virtuald Dec 14, 2024
793ade2
All classes must be specified in .yml file now
virtuald Dec 26, 2024
7e12914
Remove pydantic and switch to dataclasses/validobj
virtuald Mar 2, 2025
e7445df
Rename most of the project to semiwrap
virtuald Mar 2, 2025
410cb10
Temporary commit
virtuald Feb 21, 2025
4c98fa9
Add pyi to makeplan
virtuald Feb 26, 2025
7b98fce
fix src layout
virtuald Mar 2, 2025
4054eb4
Update references
virtuald Mar 2, 2025
e2014ab
bikeshed
virtuald Mar 5, 2025
cb5e6d5
Update tests
virtuald Mar 24, 2025
42c3da9
Many fixes
virtuald Mar 24, 2025
2b341f5
Test programs build, most tests pass, incremental build works
virtuald Mar 25, 2025
6c60276
Refactor plan
virtuald Mar 26, 2025
1e4c98c
Tests pass
virtuald Mar 26, 2025
82ae55c
Rename test repo
virtuald Mar 26, 2025
66222e8
Fix bug
virtuald Mar 26, 2025
3d5d8df
Bump cxxheaderparser
virtuald Mar 26, 2025
3f17c36
Remove setuptools cruft
virtuald Mar 26, 2025
6fdf573
All tests pass
virtuald Mar 27, 2025
cef1da8
Fix hatch and CI
virtuald Mar 27, 2025
0a240cd
CI
virtuald Mar 27, 2025
9f47c40
Hatchling is a dependency
virtuald Mar 27, 2025
53f7fc4
Fix python 3.8
virtuald Mar 27, 2025
6df14d7
Add pkgconf
virtuald Mar 27, 2025
890c659
Changed my mind
virtuald Mar 27, 2025
a33174b
Fix cross
virtuald Mar 27, 2025
1015bf8
More pyproject
virtuald Mar 27, 2025
29f3707
require
virtuald Mar 27, 2025
aac222f
Fix < 3.12
virtuald Mar 27, 2025
e8570c5
Remove distutils maybe
virtuald Mar 27, 2025
894b378
Add ninja dependency
virtuald Mar 27, 2025
d2bae7e
encoding
virtuald Mar 27, 2025
6d1c232
Fix windows support
virtuald Mar 27, 2025
0a49359
Black
virtuald Mar 27, 2025
9727b34
Nobody cares about x86
virtuald Mar 27, 2025
5faa9d7
Setup MSVC
virtuald Mar 27, 2025
6695e53
Don't use bash
virtuald Mar 27, 2025
96a249b
This is gonna work
virtuald Mar 27, 2025
057e662
Really
virtuald Mar 27, 2025
a111728
Let's see
virtuald Mar 27, 2025
ffc8f9a
Maybe this is better
virtuald Mar 27, 2025
e68c07b
Try to get the wheel
virtuald Mar 28, 2025
22c36db
Resolve DOS paths
virtuald Mar 28, 2025
52cf4ce
Skip pyi
virtuald Mar 28, 2025
3cd43aa
Fix pyi bug
virtuald Mar 28, 2025
39f5a47
Remove debug
virtuald Mar 28, 2025
da7adab
Won't work yet, but it'll get farther
virtuald Mar 28, 2025
fcb68c5
Update
virtuald Mar 28, 2025
3fa7fcf
Update cross compilation
virtuald Mar 30, 2025
51e4824
Appease them
virtuald Mar 30, 2025
7327ca6
Add rasbpian too
virtuald Mar 30, 2025
4a95f3e
Fix pyi and some other things too
virtuald Apr 9, 2025
85dcfd3
More
virtuald Apr 9, 2025
4c89f77
Fix libinit
virtuald Apr 9, 2025
f8b3124
Remove debug
virtuald Apr 9, 2025
0996470
Less annoying gitignore
virtuald Apr 9, 2025
2775fe1
Better error messages
virtuald Apr 10, 2025
500288a
Rename autogen_module_init.hpp
virtuald Apr 10, 2025
69a4159
Add internal dependency test
virtuald Apr 10, 2025
69ddca9
refactor extension module to use a dependency object instead
virtuald Apr 10, 2025
cb9b380
Add better error
virtuald Apr 18, 2025
9b05c59
Add test for resolving caster in a namespace
virtuald Apr 18, 2025
f9faea7
Pretty print error messages
virtuald Apr 19, 2025
2378f0a
toposort extension modules so that ordering doesn't matter
virtuald Apr 19, 2025
c9a6c6e
Old python compat
virtuald Apr 20, 2025
c207e24
Fix old python
virtuald Apr 20, 2025
517b6a2
Fix python 3.8/3.9 compat
virtuald Apr 21, 2025
08ee135
Remove relative path
virtuald Apr 21, 2025
4d4fd35
Setup ccache maybe
virtuald Apr 21, 2025
4e1ec38
Add C++ macro query variable
virtuald Apr 21, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 36 additions & 40 deletions .github/workflows/dist.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,14 @@ jobs:
with:
submodules: true

- uses: actions/setup-python@v4
with:
python-version: 3.8
- name: Sphinx
run: |
pip --disable-pip-version-check install -e .
pip --disable-pip-version-check install -r docs/requirements.txt
cd docs && make clean html SPHINXOPTS="-W --keep-going"
# - uses: actions/setup-python@v5
# with:
# python-version: 3.8
# - name: Sphinx
# run: |
# pip --disable-pip-version-check install -e .
# pip --disable-pip-version-check install -r docs/requirements.txt
# cd docs && make clean html SPHINXOPTS="-W --keep-going"

build:
runs-on: ubuntu-22.04
Expand All @@ -67,7 +67,7 @@ jobs:
submodules: recursive
fetch-depth: 0

- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: 3.8

Expand Down Expand Up @@ -97,12 +97,12 @@ jobs:
- '3.11'
- '3.12'
- '3.13'
architecture: [x86, x64]
exclude:
- os: macos-13
architecture: x86
- os: ubuntu-22.04
architecture: x86
architecture: [x64]
# exclude:
# - os: macos-13
# architecture: x86
# - os: ubuntu-22.04
# architecture: x86
include:
- os: macos-14
python_version: 3.9
Expand All @@ -116,7 +116,7 @@ jobs:
with:
submodules: true

- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python_version }}
architecture: ${{ matrix.architecture }}
Expand All @@ -133,25 +133,32 @@ jobs:
key: ${{ matrix.os }}-${{ matrix.architecture }}-${{ matrix.python_version }}
variant: ccache

- name: Setup MSVC
uses: bus1/cabuild/action/msdevshell@e22aba57d6e74891d059d66501b6b5aed8123c4d # v1
with:
architecture: x64
if: runner.os == 'Windows'

- name: Configure ccache for MSVC
shell: bash
# https://github.com/mesonbuild/meson/issues/10423
run: |
echo "CC=ccache.exe cl" >> $GITHUB_ENV
echo "CXX=ccache.exe cl" >> $GITHUB_ENV
if: runner.os == 'Windows'

- name: Install
shell: bash
working-directory: dist
run: python -m pip --disable-pip-version-check install *.whl

- name: Install test dependencies
shell: bash
working-directory: tests
run: python -m pip --disable-pip-version-check install -r requirements.txt

- name: Test wheel
shell: bash
env:
RPYBUILD_PARALLEL: 1
RPYBUILD_STRIP_LIBPYTHON: 1
RPYBUILD_CC_LAUNCHER: ccache
working-directory: tests
run: |
python run_tests.py
run: python run_tests.py

cross-build:
runs-on: ubuntu-latest
Expand All @@ -160,9 +167,8 @@ jobs:
max-parallel: ${{ fromJSON(needs.setup_concurrency.outputs.max-parallel).v }}
matrix:
container:
- wpilib/roborio-cross-ubuntu:2024-22.04-py312
- wpilib/raspbian-cross-ubuntu:bullseye-22.04-py312
- wpilib/aarch64-cross-ubuntu:bullseye-22.04-py312
- wpilib/roborio-cross-ubuntu:2025-22.04-py313
- wpilib/raspbian-cross-ubuntu:2025-bookworm-22.04-py313

container:
image: "${{ matrix.container }}"
Expand All @@ -184,33 +190,23 @@ jobs:
name: dist
path: dist

- name: Install setuptools
run: |
/build/venv/bin/build-pip --disable-pip-version-check install setuptools
/build/venv/bin/cross-pip --disable-pip-version-check install setuptools
- name: Setup cross environment
uses: robotpy/build-actions/setup-cross-meson@semiwrap

- name: Install
working-directory: dist
run: |
# install to both cross and build in case things get mixed up
/build/venv/bin/build-pip --disable-pip-version-check install *.whl
/build/venv/bin/cross-pip --disable-pip-version-check install *.whl

- name: Install test dependencies
shell: bash
working-directory: tests
run: |
# install to both cross and build in case things get mixed up
/build/venv/bin/build-pip --disable-pip-version-check install -r requirements.txt
/build/venv/bin/cross-pip --disable-pip-version-check install -r requirements.txt

- name: Build cross wheel
env:
RPYBUILD_PARALLEL: 1
RPYBUILD_STRIP_LIBPYTHON: 1
RPYBUILD_CC_LAUNCHER: ccache
working-directory: tests/cpp
run: /build/venv/bin/cross-python run_install.py wheel
run: /build/venv/bin/cross-python run_install.py --config-settings=setup-args=--cross-file=cross.txt

publish:
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ __pycache__
/dist
/build

/robotpy_build/version.py
/src/semiwrap/version.py
4 changes: 2 additions & 2 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[submodule "robotpy_build/pybind11"]
path = robotpy_build/pybind11
[submodule "src/semiwrap/pybind11"]
path = src/semiwrap/pybind11
url = https://github.com/pybind/pybind11.git
2 changes: 0 additions & 2 deletions MANIFEST.in

This file was deleted.

12 changes: 7 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
robotpy-build
=============
semiwrap
========

This is a build tool intended to be generally useful for any python project
that has binary dependencies. It is especially designed to meet the needs
Expand All @@ -14,13 +14,15 @@ Requires Python 3.8+
Documentation
-------------

Documentation can be found at https://robotpy-build.readthedocs.io/
Documentation can be found at https://semiwrap.readthedocs.io/

Author
------

Dustin Spicuzza is the primary author of robotpy-build, but it is the
Dustin Spicuzza is the primary author of semiwrap.

Semiwrap is a direct decendant of the robotpy-build project, and is
culmination of many years of experimentation with automated wrapper
generation by members of the RobotPy community.

robotpy-build is available under the BSD 3-clause license.
semiwrap is available under the BSD 3-clause license.
4 changes: 2 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

os.environ["GENERATING_DOCUMENTATION"] = "True"

__version__ = pkg_resources.get_distribution("robotpy-build").version
__version__ = pkg_resources.get_distribution("semiwrap").version


# -- RTD configuration ------------------------------------------------
Expand All @@ -25,7 +25,7 @@

# -- Project information -----------------------------------------------------

project = "robotpy-build"
project = "semiwrap"
copyright = "2020, RobotPy Development Team"
author = "RobotPy Development Team"

Expand Down
Empty file removed examples/demo/gen/.gitkeep
Empty file.
23 changes: 23 additions & 0 deletions examples/demo/meson.build
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
project('demo', ['cpp'],
default_options: ['warning_level=1', 'cpp_std=c++20',
'b_colorout=auto', 'optimization=2', 'b_pie=true'])

# Include autogenerated wrapcfg/meson.build
subdir('wrapcfg')

# Add additional source files to predefined variable in wrapcfg/meson.build
demo_sources += files(
'swdemo/src/demo.cpp',
'swdemo/src/main.cpp',
)

# You can add extra compilation arguments by adding a dependency to predefined
# variable
demo_deps += [
declare_dependency(include_directories: ['swdemo/include'])
]

# Include autogenerated wrapcfg/modules/meson.build
# - Builds the extension modules
# - Generates the pyi file for the extension modules
subdir('wrapcfg/modules')
75 changes: 36 additions & 39 deletions examples/demo/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,41 +1,38 @@

# This section tells pip to install robotpy-build before starting a build
# This section tells pip to install semiwrap before starting a build
[build-system]
requires = ["robotpy-build"]

# Tells robotpy-build where to place autogenerated metadata
[tool.robotpy-build]
base_package = "rpydemo"

# This section configures the 'rpydemo' python package. Multiple
# sections are possible to build multiple packages
[tool.robotpy-build.wrappers."rpydemo"]
name = "rpydemo"

# C++ source files to compile, path is relative to the root of the project
sources = [
"rpydemo/src/demo.cpp",
"rpydemo/src/main.cpp"
]

# This is a directory that can be used to customize the autogenerated
# C++ code
# -> autogenerate those files via `robotpy-build create-gen`
generation_data = "gen"

# This tells robotpy-build to parse include/demo.h and autogenerate pybind11
# wrappers for the contents of the header.
# -> autogenerate this via `robotpy-build scan-headers`
[tool.robotpy-build.wrappers."rpydemo".autogen_headers]
demo = "demo.h"


# Standard python package metadata
[tool.robotpy-build.metadata]
name = "robotpy-build-demo"
description = "robotpy-build demo program"
author = "RobotPy Development Team"
author_email = "[email protected]"
url = "https://github.com/robotpy/robotpy-build"
license = "BSD-3-Clause"
install_requires = []
build-backend = "hatchling.build"
requires = ["semiwrap", "hatch-meson", "hatchling"]

[project]
name = "swdemo"
description = "Demo program"
version = "0.0.1"

#
# hatch-semiwrap configuration
# .. this generates meson.build to perform autogen
#

[tool.hatch.build.hooks.semiwrap]
# autogen_build_path = "autogen"

#
# hatch-meson configuration
# .. this executes meson to build python extension modules
#

[tool.hatch.build.hooks.meson]


#
# semiwrap code generation configuration
#

[tool.semiwrap]

[tool.semiwrap.extension_modules."swdemo._demo"]
name = "demo"

[tool.semiwrap.extension_modules."swdemo._demo".headers]
demo = "include/demo.h"
3 changes: 0 additions & 3 deletions examples/demo/rpydemo/src/main.cpp

This file was deleted.

4 changes: 0 additions & 4 deletions examples/demo/setup.py

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# autogenerated by 'robotpy-build create-imports rpydemo rpydemo._rpydemo'
from ._rpydemo import DemoClass, add2
from ._demo import DemoClass, add2

__all__ = ["DemoClass", "add2"]
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
/** Adds 2 to the first parameter and returns it */
int add2(int x);

namespace demo {

/**
Doxygen documentation is automatically added to your python objects
when the bindings are autogenerated.
Expand All @@ -20,3 +22,5 @@ class DemoClass {
private:
int m_x = 0;
};

} // namespace demo
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,14 @@ int add2(int x) {
return x + 2;
}

namespace demo {

void DemoClass::setX(int x) {
m_x = x;
}

int DemoClass::getX() const {
return m_x;
}
}

} // namespace demo
5 changes: 5 additions & 0 deletions examples/demo/swdemo/src/main.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#include <semiwrap_init.swdemo._demo.hpp>

SEMIWRAP_PYBIND11_MODULE(m) {
initWrapper(m);
}
1 change: 1 addition & 0 deletions examples/demo/wrapcfg/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/meson.build
9 changes: 9 additions & 0 deletions examples/demo/wrapcfg/demo.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---

functions:
add2:
classes:
demo::DemoClass:
methods:
setX:
getX:
1 change: 1 addition & 0 deletions examples/demo/wrapcfg/modules/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/meson.build
1 change: 1 addition & 0 deletions examples/demo/wrapcfg/trampolines/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/meson.build
Loading
Loading