Skip to content

Commit 46014b0

Browse files
ssmmnn11ssmmnn11
authored
Merge branch 'main' into feat/swiglu_mlp
2 parents 59a5c7c + 266f5a6 commit 46014b0

30 files changed

Lines changed: 939 additions & 68 deletions

‎.github/workflows/inactivity-bot.yml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
name: Mark inactive issues and PRs
2+
3+
on:
4+
schedule:
5+
- cron: "0 23 * * *" # every day at 23pm on default(main) branch
6+
workflow_dispatch: # Allows manual trigger
7+
8+
jobs:
9+
stale:
10+
runs-on: ubuntu-latest
11+
steps:
12+
- uses: actions/stale@v9
13+
with:
14+
repo-token: $
15+
16+
# Issue settings
17+
days-before-issue-stale: 90 # (~3 months)
18+
days-before-issue-close: 10
19+
stale-issue-label: inactive
20+
stale-issue-message: >
21+
This issue has been marked as inactive due to inactivity.
22+
It will be closed if no further activity occurs in the next 10 days.
23+
exempt-issue-labels: 'good first issue,help wanted,roadmap,known-issue'
24+
25+
# PR settings
26+
days-before-pr-stale: 90 # (~3 months)
27+
days-before-pr-close: 10
28+
stale-pr-label: inactive
29+
stale-pr-message: >
30+
This PR has been marked as inactive due to inactivity.
31+
It will be closed if no further activity occurs in the next 10 days.
32+
exempt-pr-labels: 'autorelease: pending'
33+
34+
remove-stale-when-updated: true

‎.github/workflows/integration-tests-hpc.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ jobs:
7575
source .venv/bin/activate
7676
pip install --upgrade pip
7777
pip install -e ./training[all,tests,profile] -e ./models[all,tests] -e ./graphs[all,tests]
78-
srun python -m pytest -v training/tests/integration --slow --multigpu -k "test_benchmark_training_cycle"
78+
srun python -m pytest -v training/tests/integration/test_benchmark.py --slow --multigpu
7979
deactivate
8080
rm -rf $REPO_NAME
8181
sbatch_options: |

‎.gitignore‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -128,7 +128,6 @@ _dev/
128128
_api/
129129
./outputs
130130
*tmp_data/
131-
*uv.lock
132131

133132
# Project specific
134133
?

‎.release-please-manifest.json‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"training": "0.9.1",
3-
"graphs": "0.8.5",
4-
"models": "0.12.1"
2+
"training": "0.10.0",
3+
"graphs": "0.9.0",
4+
"models": "0.13.0"
55
}

‎graphs/CHANGELOG.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88
Please add your functional changes to the appropriate section in the PR.
99
Keep it human-readable, your future self will thank you!
1010

11+
## [0.9.0](https://github.com/ecmwf/anemoi-core/compare/graphs-0.8.5...graphs-0.9.0) (2026-03-12)
12+
13+
14+
### ⚠ BREAKING CHANGES
15+
16+
* **training:** expands graph config for multiple datasets ([#895](https://github.com/ecmwf/anemoi-core/issues/895))
17+
* **training:** remove `grid_indices` inside Dataset ([#848](https://github.com/ecmwf/anemoi-core/issues/848))
18+
19+
### Features
20+
21+
* **graphs:** Add custom area weightings ([#926](https://github.com/ecmwf/anemoi-core/issues/926)) ([5c43509](https://github.com/ecmwf/anemoi-core/commit/5c435092dee7f09735d8e958c298b620e7313c33))
22+
23+
24+
### Bug Fixes
25+
26+
* **training:** Call correct open_datasets ([#965](https://github.com/ecmwf/anemoi-core/issues/965)) ([470b725](https://github.com/ecmwf/anemoi-core/commit/470b72588f178ddac913c6efd503ffd065dbf60d))
27+
* **training:** Expands graph config for multiple datasets ([#895](https://github.com/ecmwf/anemoi-core/issues/895)) ([33d7f4e](https://github.com/ecmwf/anemoi-core/commit/33d7f4e11fff8da22243fbb924036bab07f17c65))
28+
29+
30+
### Code Refactoring
31+
32+
* **training:** Remove `grid_indices` inside Dataset ([#848](https://github.com/ecmwf/anemoi-core/issues/848)) ([dcd69ac](https://github.com/ecmwf/anemoi-core/commit/dcd69ac415894131876c2f19af643896f1605fe6))
33+
1134
## [0.8.5](https://github.com/ecmwf/anemoi-core/compare/graphs-0.8.4...graphs-0.8.5) (2026-02-09)
1235

1336

‎models/CHANGELOG.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88
Please add your functional changes to the appropriate section in the PR.
99
Keep it human-readable, your future self will thank you!
1010

11+
## [0.13.0](https://github.com/ecmwf/anemoi-core/compare/models-0.12.1...models-0.13.0) (2026-03-12)
12+
13+
14+
### ⚠ BREAKING CHANGES
15+
16+
* **training:** expands graph config for multiple datasets ([#895](https://github.com/ecmwf/anemoi-core/issues/895))
17+
* **models:** move functionality to base model class ([#922](https://github.com/ecmwf/anemoi-core/issues/922))
18+
19+
### Features
20+
21+
* Consolidate and expand spectral losses ([#788](https://github.com/ecmwf/anemoi-core/issues/788)) ([9d70464](https://github.com/ecmwf/anemoi-core/commit/9d7046462e8b9b270296f34fdb10ea8e9cf08e06))
22+
* **models,training:** Deprecate dynamic imputers ([#942](https://github.com/ecmwf/anemoi-core/issues/942)) ([fbdfa0c](https://github.com/ecmwf/anemoi-core/commit/fbdfa0c7c586697d9523380f3e539c4b35ab9f11))
23+
* **models:** Move functionality to base model class ([#922](https://github.com/ecmwf/anemoi-core/issues/922)) ([b735ec3](https://github.com/ecmwf/anemoi-core/commit/b735ec3c58d7f3e09b2ad8138855b37a05bee297))
24+
* Remove single out time interpolator ([#915](https://github.com/ecmwf/anemoi-core/issues/915)) ([80713fb](https://github.com/ecmwf/anemoi-core/commit/80713fb6d04b76ecfd44c6e5e450a546ac2a0d1c))
25+
* **training:** Support flash attention v4 ([#959](https://github.com/ecmwf/anemoi-core/issues/959)) ([b4a3a3e](https://github.com/ecmwf/anemoi-core/commit/b4a3a3e2f8ab89a61a99e2f1a84801a273c8a765))
26+
27+
28+
### Bug Fixes
29+
30+
* `distributed.all_gather()` support for uneven tensors by explicit padding ([#927](https://github.com/ecmwf/anemoi-core/issues/927)) ([b6b5df9](https://github.com/ecmwf/anemoi-core/commit/b6b5df91b3636609f4e2c6cf33dd5f116e0f37fb))
31+
* Diffusion sampling ([#924](https://github.com/ecmwf/anemoi-core/issues/924)) ([9bd9cae](https://github.com/ecmwf/anemoi-core/commit/9bd9caec1b52ee75e9989e058f2c5d67dba00b68))
32+
* **models:** Fix in-out sharding for all models ([#929](https://github.com/ecmwf/anemoi-core/issues/929)) ([2c24de5](https://github.com/ecmwf/anemoi-core/commit/2c24de571dbd65de86a381f5af5329085f33583e))
33+
* **models:** Fix migration script, remove unused arg ([#893](https://github.com/ecmwf/anemoi-core/issues/893)) ([531546e](https://github.com/ecmwf/anemoi-core/commit/531546e8a28b6560f6f1398878407ff35e74af53))
34+
* **models:** Keep_batch_sharded false logic ([#912](https://github.com/ecmwf/anemoi-core/issues/912)) ([88b5066](https://github.com/ecmwf/anemoi-core/commit/88b5066b11ad0fe062df1095cf3d2681de8710a0))
35+
* **models:** Use flex attention block mask for sdpa windowed attention ([#940](https://github.com/ecmwf/anemoi-core/issues/940)) ([2bf1842](https://github.com/ecmwf/anemoi-core/commit/2bf1842d7fc24c0eb1bb4e1d29093542f93d17f5))
36+
* **training:** Expands graph config for multiple datasets ([#895](https://github.com/ecmwf/anemoi-core/issues/895)) ([33d7f4e](https://github.com/ecmwf/anemoi-core/commit/33d7f4e11fff8da22243fbb924036bab07f17c65))
37+
* **training:** Raise triton tolerence ([#870](https://github.com/ecmwf/anemoi-core/issues/870)) ([81e913b](https://github.com/ecmwf/anemoi-core/commit/81e913be973e9f84d787f1cbce6809898b848dbd))
38+
1139
## [0.12.1](https://github.com/ecmwf/anemoi-core/compare/models-0.12.0...models-0.12.1) (2026-02-09)
1240

1341

‎models/src/anemoi/models/migrations/scripts/1773048851_fuse_multiple_perdataset_graphs.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
metadata = MigrationMetadata(
1515
versions={
1616
"migration": "1.0.0",
17-
"anemoi-models": "%NEXT_ANEMOI_MODELS_VERSION%",
17+
"anemoi-models": "0.13.0",
1818
},
1919
)
2020
# <-- END DO NOT CHANGE

‎models/src/anemoi/models/models/base.py‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ def _calculate_shapes_and_indices(self, data_indices: dict) -> None:
102102
self._internal_output_idx = {}
103103
self._decoding_forcing_input_idx = {}
104104
self.input_dim = {}
105-
self.input_dim_latent = {}
105+
self.input_dim_latent = self._calculate_input_dim_latent()
106106
self.target_dim = {}
107107
self.output_dim = {}
108108

@@ -121,15 +121,16 @@ def _calculate_shapes_and_indices(self, data_indices: dict) -> None:
121121
self.num_output_channels[dataset_name] = len(dataset_indices.model.output)
122122

123123
self.input_dim[dataset_name] = self._calculate_input_dim(dataset_name)
124-
self.input_dim_latent[dataset_name] = self._calculate_input_dim_latent(dataset_name)
125124
self.target_dim[dataset_name] = self._calculate_target_dim(dataset_name)
126125
self.output_dim[dataset_name] = self._calculate_output_dim(dataset_name)
127126

128127
def _calculate_input_dim(self, dataset_name: str) -> int:
129128
return self.n_step_input * self.num_input_channels[dataset_name] + self.node_attributes.attr_ndims[dataset_name]
130129

131-
def _calculate_input_dim_latent(self, dataset_name: str) -> int:
132-
return self.node_attributes.attr_ndims[dataset_name]
130+
def _calculate_input_dim_latent(self) -> int:
131+
"""Calculate the latent input dimension."""
132+
nodes_name = self._graph_name_hidden if isinstance(self._graph_name_hidden, str) else self._graph_name_hidden[0]
133+
return self.node_attributes.attr_ndims[nodes_name]
133134

134135
def _assert_hidden_nodes_name(self, hidden_nodes_name: str) -> None:
135136
if isinstance(hidden_nodes_name, str):

‎models/src/anemoi/models/models/encoder_processor_decoder.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ def _build_networks(self, model_config: DotDict) -> None:
4949
model_config.model.encoder,
5050
_recursive_=False, # Avoids instantiation of layer_kernels here
5151
in_channels_src=self.input_dim[dataset_name],
52-
in_channels_dst=self.input_dim_latent[dataset_name],
52+
in_channels_dst=self.input_dim_latent,
5353
hidden_dim=self.num_channels,
5454
edge_dim=self.encoder_graph_provider[dataset_name].edge_dim,
5555
)

‎models/src/anemoi/models/models/hierarchical.py‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,6 @@
2626
class AnemoiModelEncProcDecHierarchical(AnemoiModelEncProcDec):
2727
"""Message passing hierarchical graph neural network."""
2828

29-
def _calculate_input_dim_latent(self, dataset_name: str) -> int:
30-
return self.node_attributes.attr_ndims[self._graph_name_hidden[0]]
31-
3229
def _build_networks(self, model_config):
3330
"""Builds the model components."""
3431
# note that this is called by the super class init
@@ -52,7 +49,7 @@ def _build_networks(self, model_config):
5249
model_config.model.encoder,
5350
_recursive_=False, # Avoids instantiation of layer_kernels here
5451
in_channels_src=self.input_dim[dataset_name],
55-
in_channels_dst=self.input_dim_latent[dataset_name],
52+
in_channels_dst=self.input_dim_latent,
5653
hidden_dim=self.hidden_dims[self._graph_name_hidden[0]],
5754
edge_dim=self.encoder_graph_provider[dataset_name].edge_dim,
5855
)

0 commit comments

Comments
 (0)