Skip to content

FXC-4741 move core modules to common submodule #3138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
marcorudolphflex wants to merge 29 commits into
base: yaugenst-flex/pydantic-v2
Choose a base branch
from
Draft

FXC-4741 move core modules to common submodule #3138

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
61fe5ad
refactor: pydantic-v2 and mypy
mahlau-flex Jan 5, 2026
ec50bca
refactor(tidy3d): FXC-4683-move-config-to-new-common-submodule
marcorudolphflex Jan 7, 2026
9a75430
split core
marcorudolphflex Jan 7, 2026
66ceca3
split core
marcorudolphflex Jan 7, 2026
473e13a
split compat
marcorudolphflex Jan 7, 2026
8785fce
prepare base split
marcorudolphflex Jan 7, 2026
3e1b631
prepare base split
marcorudolphflex Jan 7, 2026
6f6d3fa
prepare base split
marcorudolphflex Jan 7, 2026
d29f2bf
moved base
marcorudolphflex Jan 7, 2026
18da330
prepare geom
marcorudolphflex Jan 7, 2026
c693690
split subset geom/data
marcorudolphflex Jan 7, 2026
76ec60e
update from rebase
marcorudolphflex Jan 8, 2026
2197087
update from rebase
marcorudolphflex Jan 8, 2026
b65f6ef
unmove dataset and split data array
marcorudolphflex Jan 9, 2026
a2c6a50
prepare geometry split
marcorudolphflex Jan 9, 2026
63061d3
prepare geometry split
marcorudolphflex Jan 9, 2026
0e66365
add base to type imports
marcorudolphflex Jan 9, 2026
ff16213
moved viz and geom
marcorudolphflex Jan 9, 2026
5bcef60
split packaging and utils
marcorudolphflex Jan 9, 2026
69d8c4f
split validators, resplit data arrays and dataset
marcorudolphflex Jan 9, 2026
692f27a
changes after rebase
marcorudolphflex Jan 13, 2026
57286a6
local cache move
marcorudolphflex Jan 13, 2026
2e78a15
make cache protocol, re-move mode solver helper from common, fix tests
marcorudolphflex Jan 13, 2026
524db7c
import fix
marcorudolphflex Jan 13, 2026
e6d3754
prepare source move
marcorudolphflex Jan 13, 2026
c5ec032
move source
marcorudolphflex Jan 13, 2026
cd98800
working medium refactor
marcorudolphflex Jan 21, 2026
cdb1154
rebase changes
marcorudolphflex Jan 22, 2026
85f7dc4
fix tests
marcorudolphflex Jan 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
96 changes: 90 additions & 6 deletions .github/workflows/tidy3d-python-client-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,9 @@ on:
permissions:
contents: read

env:
VERIFICATIONS_PY_VERSION: '3.11'

jobs:
determine-test-scope:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -223,6 +226,39 @@ jobs:
echo "version_match_tests=$version_match_tests"
echo "extras_integration_tests=$extras_integration_tests"
echo "test_type=$test_type"


move-type-imports:
name: move-type-imports
needs: determine-test-scope
if: needs.determine-test-scope.outputs.code_quality_tests == 'true'
runs-on: ubuntu-latest
steps:
- name: checkout-branch
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
fetch-depth: 0
persist-credentials: false

- name: setup-python
uses: actions/setup-python@v5
with:
python-version: ${{ env.VERIFICATIONS_PY_VERSION }}

- name: install-dependencies
run: |
set -euo pipefail
python -m venv .venv
source .venv/bin/activate
pip install libcst

- name: verify-type-import-guards
run: |
set -euo pipefail
source .venv/bin/activate
python scripts/move_type_imports.py --mode check_on_change

lint:
needs: determine-test-scope
Expand Down Expand Up @@ -256,10 +292,10 @@ jobs:
submodules: false
persist-credentials: false

- name: set-python-3.10
- name: set-python
uses: actions/setup-python@v4
with:
python-version: '3.10'
python-version: ${{ env.VERIFICATIONS_PY_VERSION }}

- name: Install mypy
run: |
Expand All @@ -270,6 +306,27 @@ jobs:
run: |
mypy --config-file=pyproject.toml

ensure-common-imports:
name: ensure-common-imports
needs: determine-test-scope
if: needs.determine-test-scope.outputs.code_quality_tests == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1
submodules: false
persist-credentials: false

- name: set-python-3.10
uses: actions/setup-python@v4
with:
python-version: '3.10'

- name: Run tidy3d._common import check
run: |
python scripts/ensure_imports_from_common.py

zizmor:
name: Run zizmor 🌈
runs-on: ubuntu-latest
Expand All @@ -287,7 +344,7 @@ jobs:
uses: astral-sh/setup-uv@b75a909f75acd358c2196fb9a5f1299a9a8868a4 # v6.7.0

- name: Run zizmor 🌈
run: uvx zizmor .github/workflows/*.y* --format=sarif . > results.sarif
run: uvx zizmor==1.19.0 .github/workflows/*.y* --format=sarif . > results.sarif
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Expand All @@ -298,7 +355,7 @@ jobs:
category: zizmor

- name: run zizmor directly # this gets a success or fail result
run: uvx zizmor .github/workflows/*.y*
run: uvx zizmor==1.19.0 .github/workflows/*.y*
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Expand Down Expand Up @@ -370,6 +427,8 @@ jobs:
runs-on: ubuntu-latest
if: needs.determine-test-scope.outputs.code_quality_tests == 'true'
name: lint-commit-messages
# Soft-fail on PRs (early feedback), hard-fail in merge queue (enforced)
continue-on-error: ${{ github.event_name == 'pull_request' }}
steps:
- name: Check out source code
uses: actions/checkout@v4
Expand All @@ -391,7 +450,18 @@ jobs:
node --version
npm --version
npx commitlint --version


- name: Check commit messages (pull_request)
if: github.event_name == 'pull_request'
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
npx commitlint --from $BASE_SHA --to $HEAD_SHA --verbose || {
echo "::warning::Commit messages don't follow conventional commits format. Fix before merge queue."
exit 1
}

- name: Check commit messages (merge_group)
if: github.event_name == 'merge_group'
env:
Expand Down Expand Up @@ -429,7 +499,7 @@ jobs:

- name: install-depedencies
run: |
uv venv $GITHUB_WORKSPACE/.venv -p 3.11
uv venv $GITHUB_WORKSPACE/.venv -p "$VERIFICATIONS_PY_VERSION"
source $GITHUB_WORKSPACE/.venv/bin/activate
uv pip install -e "$GITHUB_WORKSPACE"

Expand Down Expand Up @@ -981,8 +1051,10 @@ jobs:
- determine-test-scope
- local-tests
- remote-tests
- move-type-imports
- lint
- mypy
- ensure-common-imports
- verify-schema-change
- lint-commit-messages
- lint-branch-name
Expand All @@ -993,6 +1065,12 @@ jobs:
- extras-integration-tests
runs-on: ubuntu-latest
steps:
- name: move-type-imports
if: ${{ needs.determine-test-scope.outputs.code_quality_tests == 'true' && needs.move-type-imports.result != 'success' && needs.move-type-imports.result != 'skipped' }}
run: |
echo "❌ Found imports used only for typing that are not guarded by if TYPE_CHECKING."
exit 1

- name: check-linting-result
if: ${{ needs.determine-test-scope.outputs.code_quality_tests == 'true' && needs.lint.result != 'success' && needs.lint.result != 'skipped' }}
run: |
Expand All @@ -1004,6 +1082,12 @@ jobs:
run: |
echo "❌ Mypy type checking failed."
exit 1

- name: check-common-imports-result
if: ${{ needs.determine-test-scope.outputs.code_quality_tests == 'true' && needs.ensure-common-imports.result != 'success' && needs.ensure-common-imports.result != 'skipped' }}
run: |
echo "❌ tidy3d._common import check failed."
exit 1

- name: check-schema-change-verification
if: ${{ needs.determine-test-scope.outputs.code_quality_tests == 'true' && needs.verify-schema-change.result != 'success' && needs.verify-schema-change.result != 'skipped' }}
Expand Down
4 changes: 3 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Batch JSON files
batch*.json
*.vtu
simulation.json

# Byte-compiled / optimized / DLL files
*$py.class
Expand Down Expand Up @@ -135,5 +136,6 @@ htmlcov/
.idea
.vscode

# cProfile output
# profile outputs
*.prof
pytest_profile_stats.txt
6 changes: 0 additions & 6 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +0,0 @@
[submodule "docs/notebooks"]
path = docs/notebooks
url = [email protected]:flexcompute/tidy3d-notebooks.git
[submodule "docs/faq"]
path = docs/faq
url = https://github.com/flexcompute/tidy3d-faq
10 changes: 9 additions & 1 deletion .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,14 @@ default_install_hook_types:
- pre-commit
- commit-msg
repos:
- repo: local
hooks:
- id: move-type-imports
name: move type-only imports under TYPE_CHECKING
entry: poetry run python scripts/move_type_imports.py --mode fix --only-changed
language: system
pass_filenames: false
stages: [pre-commit]
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: "v0.11.11"
hooks:
Expand All @@ -20,7 +28,7 @@ repos:
entry: bash -c 'commitlint --edit || exit 0'
- repo: https://github.com/zizmorcore/zizmor-pre-commit
# Zizmor version.
rev: v1.15.2
rev: v1.19.0
hooks:
- id: zizmor
stages: [pre-commit]
Expand Down
1 change: 1 addition & 0 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,5 +45,6 @@
- Follow Conventional Commits per `.commitlintrc.json`.
- Branch names must use an allowed prefix (`chore`, `hotfix`, `daily-chore`) or include a Jira key to satisfy CI.
- PRs should link issues, summarize behavior changes, list the `poetry run …` checks you executed, and call out docs/schema updates.
- Add a changelog entry under `## [Unreleased]` in `CHANGELOG.md` for user-facing changes (new features, bug fixes, breaking changes).

_Reminder: update this AGENTS.md whenever workflow, tooling, or review expectations change so agents stay in sync with the repo._
39 changes: 37 additions & 2 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,47 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]
## [2.10.2] - 2026-01-21

### Added
- Added `warn_once` option to logging configuration (`td.config.logging.warn_once`) that causes each unique warning message to be shown only once per process, reducing noise from repeated validation warnings.

### Changed
- Unified inside/outside permittivity handling for all geometries when computing shape gradients.
- Enabled PEC gradients for dielectric structures embedded in PEC through use of `background_medium` field in `Structure` which includes shapes with combinations of dielectric-dielectric and dielectric-PEC boundaries.
- Improved `ComponentModeler` task monitoring.

### Fixed
- Fixed interpolation handling for permittivity and conductivity gradients in `CustomPoleResidue`.
- Fixed docstrings with missing Notes sections causing spurious parameters in Sphinx documentation.

## [2.10.1] - 2026-01-14

### Added
- Added priority attribute to `TopologyDesignRegion` to enable manual control of overlapping structures.
- `to_mat_file()` method is now available on `ModeSimulationData` and `HeatChargeSimulationData` for exporting results to MATLAB `.mat` files.
- Added autograd support for diagonal `AnisotropicMedium` and `CustomAnisotropicMedium` with diagonal permittivity tensor.
- Added support of numpy 2.4
- Added validation to `DCVoltageSource` that warns when duplicate voltage values are detected in the `voltage` array, including treating `0` and `-0` as the same value.

### Changed
- For `HeatChargeSimulation` objects, the `plot` function now adds the simulation boundary conditions.

### Fixed
- Fixed `AutoImpedanceSpec` validation to check path intersections against all conductors, not just filtered ones, as well as the mode plane bounds.
- Fixed `WavePort` validation so invalid `mode_spec` errors are no longer masked by a `KeyError`.
- Fixed adjoint gradients being treated as zero due to scale-dependent `np.allclose(..., atol=1e-8)` checks, which could skip adjoint simulations and return zero gradients.
- Fixed adjoint setup crashing when traced monitor outputs produce no adjoint sources, returning no adjoint simulations instead.
- Fixed autograd `interpn` compatibility with newer SciPy versions by avoiding dtype coercion during interpolation setup.
- Fixed interpolation handling for permittivity and conductivity gradients in CustomMedium.
- Restored original batch-load logging by suppressing per-task “Loading simulation…” messages.
- Fixed output range of `tidy3d.plugins.invdes.FilterAndProject` to be between 0 and 1.
- Cropped adjoint monitor sizes in 2D simulations to planar geometry intersection.
- Fixed `Batch.download()` silently succeeding when background downloads fail (e.g., gzip extraction errors).
- Handling of zero values when using `sim_data.plot_field` with `scale=dB`.
- Fixed `intersections_plane` method in `PolySlab`, which sometimes missed vertices for planes coincident with `PolySlab` side faces.
- Fixed `http_interceptor` crash on non-dict JSON responses
- Fixed gradient regression for `Box` geometries where outside permittivity was not being correctly sampled.

## [2.10.0] - 2025-12-18

Expand Down Expand Up @@ -1907,7 +1940,9 @@ which fields are to be projected is now determined automatically based on the me
- Job and Batch classes for better simulation handling (eventually to fully replace webapi functions).
- A large number of small improvements and bug fixes.

[Unreleased]: https://github.com/flexcompute/tidy3d/compare/v2.10.0...develop
[Unreleased]: https://github.com/flexcompute/tidy3d/compare/v2.10.2...develop
[2.10.2]: https://github.com/flexcompute/tidy3d/compare/v2.10.1...v2.10.2
[2.10.1]: https://github.com/flexcompute/tidy3d/compare/v2.10.0...v2.10.1
[2.10.0]: https://github.com/flexcompute/tidy3d/compare/v2.9.3...v2.10.0
[2.9.3]: https://github.com/flexcompute/tidy3d/compare/v2.9.2...v2.9.3
[2.9.2]: https://github.com/flexcompute/tidy3d/compare/v2.9.1...v2.9.2
Expand Down
2 changes: 1 addition & 1 deletion docs/api/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ API |:computer:|
mediums
material_library
boundary_conditions
discretization
discretization/index
sources
monitors
output_data
Expand Down
5 changes: 5 additions & 0 deletions docs/api/plugins/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
Plugins
=======

.. warning::

|:warning:| The 'adjoint' plugin (legacy JAX-based adjoint plugin)
was deprecated in Tidy3D 'v2.7.0' and is disabled as of 'v2.9.0'.

.. toctree::
:maxdepth: 1

Expand Down
8 changes: 6 additions & 2 deletions docs/development/usage.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,12 @@ There are a range of handy development functions that you might want to use to s
* - Running ``pytest`` commands inside the ``poetry`` environment.
- Make sure you have already installed ``tidy3d`` in ``poetry`` and you are in the root directory.
- ``poetry run pytest``
* - Analyze slow ``pytest`` runs with durations / cProfile / debug subset helpers.
- Use ``--debug`` to run only the first N collected tests or ``--profile`` to capture call stacks.
- ``python scripts/profile_pytest.py [options]``
* - Track ``pytest`` RAM usage over time and per test.
- Defaults to pytest's configured parallelism; use ``--single-process`` to disable xdist.
- ``poetry run python scripts/pytest_ram_profile.py [options]``
* - Run ``coverage`` testing from the ``poetry`` environment.
-
- ``poetry run coverage run -m pytest``
Expand All @@ -83,5 +89,3 @@ There are a range of handy development functions that you might want to use to s
-
- ``poetry run tidy3d develop replace-in-files``



Loading