Skip to content

Commit e938874

Browse files
authored
Merge pull request #247 from LSSTDESC/jscora/railv2-bug-fixes
railv2-bug-fixes
2 parents 5c4f576 + 0a30fe0 commit e938874

19 files changed

Lines changed: 275 additions & 470 deletions
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# This workflow will install Python dependencies, the supported rail packages, and run the create_interactive_structure.py script
2+
# If it fails with exit code = 2, that means that there have been updates to any of the stages
3+
# this usually means that you just need to create a new branch, run create_interactive_structure.py and do a PR with rail_base to update the .pyi
4+
# stub files
5+
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
6+
7+
name: Running create_interactive_structure script
8+
9+
on:
10+
push:
11+
branches: [ main ]
12+
pull_request:
13+
branches: [ main ]
14+
15+
jobs:
16+
build:
17+
18+
runs-on: ubuntu-latest
19+
strategy:
20+
matrix:
21+
python-version: ['3.10', '3.11', '3.12']
22+
steps:
23+
- uses: actions/checkout@v3
24+
with:
25+
fetch-tags: true
26+
fetch-depth: 0
27+
- name: Set up Python ${{ matrix.python-version }}
28+
uses: actions/setup-python@v4
29+
with:
30+
python-version: ${{ matrix.python-version }}
31+
- name: System deps (MPI + build)
32+
run: |
33+
sudo apt-get update
34+
sudo apt-get install -y --no-install-recommends \
35+
build-essential libbz2-dev \
36+
openmpi-bin libopenmpi-dev
37+
- name: Install dependencies
38+
run: |
39+
# sudo apt-get update
40+
# python -m pip install --upgrade pip
41+
python -m pip install --upgrade pip setuptools wheel
42+
python -m pip install "numpy<2" cython
43+
python -m pip install --upgrade --no-binary=mpi4py mpi4py
44+
python -m pip install --no-build-isolation somoclu
45+
python -m pip install --upgrade pip
46+
pip install wheel numpy
47+
pip install .
48+
pip install .[dev]
49+
pip install .[algos]
50+
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
51+
52+
# runs the create interactive structure test to see if the interactive structure needs to be changed
53+
# returns a 1 if something went wrong with the script, or a 2 if the files need to be changed
54+
- name: Run create_interactive_structure.py
55+
run: |
56+
python create_interactive_structure.py
57+
- name: Send status to Slack app (RAIL CI Reporter)
58+
if: ${{ failure() && github.event_name != 'workflow_dispatch' }}
59+
id: slack
60+
uses: slackapi/slack-github-action@v1.24.0
61+
with:
62+
# For posting a rich message using Block Kit
63+
payload: |
64+
{
65+
"blocks": [
66+
{
67+
"type": "header",
68+
"text": {
69+
"type": "plain_text",
70+
"text": "${{ github.repository }} create_interactive_structure workflow"
71+
}
72+
},
73+
{
74+
"type": "section",
75+
"text": {
76+
"type": "mrkdwn",
77+
"text": "GitHub Action build result: *${{ job.status }}* :${{ job.status }}:"
78+
}
79+
},
80+
{
81+
"type": "divider"
82+
},
83+
{
84+
"type": "section",
85+
"text": {
86+
"type": "mrkdwn",
87+
"text": "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
88+
}
89+
}
90+
]
91+
}
92+
env:
93+
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
94+
SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK

.github/workflows/smoke-test.yml

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,3 +34,42 @@ jobs:
3434
- name: Run unit tests with pytest
3535
run: |
3636
python -m pytest tests
37+
38+
- name: Send status to Slack app (RAIL CI Reporter)
39+
if: ${{ failure() && github.event_name != 'workflow_dispatch' }}
40+
id: slack
41+
uses: slackapi/slack-github-action@v1.24.0
42+
with:
43+
# For posting a rich message using Block Kit
44+
payload: |
45+
{
46+
"blocks": [
47+
{
48+
"type": "header",
49+
"text": {
50+
"type": "plain_text",
51+
"text": "${{ github.repository }}"
52+
}
53+
},
54+
{
55+
"type": "section",
56+
"text": {
57+
"type": "mrkdwn",
58+
"text": "GitHub Action build result: *${{ job.status }}* :${{ job.status }}:"
59+
}
60+
},
61+
{
62+
"type": "divider"
63+
},
64+
{
65+
"type": "section",
66+
"text": {
67+
"type": "mrkdwn",
68+
"text": "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
69+
}
70+
}
71+
]
72+
}
73+
env:
74+
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
75+
SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK

.github/workflows/testing-and-coverage.yml

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44
name: Unit test and code coverage
55

66
on:
7+
schedule:
8+
- cron: 45 7 * * 1
79
push:
810
branches: [ main ]
911
pull_request:
@@ -12,12 +14,15 @@ on:
1214
jobs:
1315
build:
1416

15-
runs-on: ubuntu-latest
17+
runs-on: ubuntu-latest
1618
strategy:
1719
matrix:
1820
python-version: ['3.10', '3.11', '3.12']
1921
steps:
2022
- uses: actions/checkout@v3
23+
with:
24+
fetch-tags: true
25+
fetch-depth: 0
2126
- name: Set up Python ${{ matrix.python-version }}
2227
uses: actions/setup-python@v4
2328
with:
@@ -26,6 +31,7 @@ jobs:
2631
run: |
2732
sudo apt-get update
2833
python -m pip install --upgrade pip
34+
pip install wheel numpy
2935
pip install .
3036
pip install .[dev]
3137
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

create_interactive_structure.py

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -153,9 +153,14 @@ def write_modules() -> None:
153153

154154
def write_stubs() -> None:
155155
for module in interactive_modules:
156-
module = "rail.interactive." + module
157-
importlib.import_module(module)
158-
_initialize_interactive_module(module, write_stubs=True)
156+
stub_directory = InteractiveModule(
157+
subfolder=module.replace(".", "/")
158+
).path.parent
159+
full_module = "rail.interactive." + module
160+
importlib.import_module(full_module)
161+
_initialize_interactive_module(
162+
full_module, write_stubs=True, stub_directory=stub_directory
163+
)
159164

160165

161166
def store_pyi(delete: bool) -> dict[Path, str | None]:

pyproject.toml

Lines changed: 29 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,25 @@ dev = [
4141
"isort", # as above
4242
]
4343

44+
algos = [
45+
"pz-rail-astro-tools",
46+
"pz-rail-bpz",
47+
"pz-rail-cmnn",
48+
"pz-rail-dnf",
49+
"pz-rail-dsps",
50+
"pz-rail-flexzboost",
51+
"pz-rail-fsps",
52+
"pz-rail-gpz-v1",
53+
"pz-rail-pzflow",
54+
"pz-rail-sklearn",
55+
"pz-rail-som",
56+
"pz-rail-yaw",
57+
"pz-rail-lephare",
58+
"pz-rail-tpz",
59+
"pz-rail-delight",
60+
"pz-rail-pipelines",
61+
]
62+
4463
[project.scripts]
4564
rail = "rail.cli.rail.commands:cli"
4665

@@ -56,7 +75,7 @@ write_to = "src/rail/core/_version.py"
5675

5776
[tool.pytest.ini_options]
5877
testpaths = ["tests"]
59-
addopts = ["--cov=rail", "--cov-report=html"]
78+
addopts = ["--cov=rail", "--cov-report=html"]
6079

6180
[tool.pylint]
6281
disable = [
@@ -82,16 +101,16 @@ max-branches = 25
82101
max-public-methods = 50
83102

84103
[tool.mypy]
85-
disallow_untyped_defs = true
104+
disallow_untyped_defs = true
86105
disallow_incomplete_defs = true
87-
ignore_missing_imports = true
88-
local_partial_types = true
89-
no_implicit_reexport = true
90-
show_error_codes = true
91-
strict_equality = true
92-
warn_redundant_casts = true
93-
warn_unreachable = true
94-
warn_unused_ignores = true
106+
ignore_missing_imports = true
107+
local_partial_types = true
108+
no_implicit_reexport = true
109+
show_error_codes = true
110+
strict_equality = true
111+
warn_redundant_casts = true
112+
warn_unreachable = true
113+
warn_unused_ignores = true
95114

96115

97116
[tool.coverage.run]

src/rail/evaluation/metrics/cdeloss.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ def evaluate(self) -> stat_and_pval:
2929
"""
3030

3131
# Calculate first term E[\int f*(z | X)^2 dz]
32-
term1 = np.mean(np.trapz(self._pdfs**2, x=self._xvals))
32+
term1 = np.mean(np.trapezoid(self._pdfs**2, x=self._xvals))
3333
# z bin closest to ztrue
3434
nns = [np.argmin(np.abs(self._xvals - z)) for z in self._ztrue]
3535
# Calculate second term E[f*(Z | X)]

src/rail/evaluation/metrics/tomography.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,7 @@ def run(self) -> None:
9191
p_j = kde_j(eval_grid)
9292

9393
# Compute overlap as the integral of the minimum of both distributions
94-
overlap = np.trapz(np.minimum(p_i, p_j), eval_grid)
94+
overlap = np.trapezoid(np.minimum(p_i, p_j), eval_grid)
9595
overlap_matrix[i, j] = overlap
9696
overlap_matrix[j, i] = overlap # Symmetric matrix
9797

src/rail/interactive/creation/degraders/grid_selection.pyi

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -97,12 +97,10 @@ def grid_selection(**kwargs) -> Any:
9797
Default: 100.0
9898
ratio_file : str, optional
9999
path to ratio file
100-
Default: rail_astro_tools/src/rail/examples_data/creation_data/data/hsc_ratios_a
101-
nd_specz.hdf5
100+
Default: rail/examples_data/creation_data/data/hsc_ratios_and_specz.hdf5
102101
settings_file : str, optional
103102
path to pickled parameters file
104-
Default:
105-
rail_astro_tools/src/rail/examples_data/creation_data/data/HSC_grid_settings.pkl
103+
Default: rail/examples_data/creation_data/data/HSC_grid_settings.pkl
106104
random_seed : int, optional
107105
random seed for reproducibility
108106
Default: 12345

src/rail/interactive/creation/degraders/observing_condition_degrader.pyi

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -82,12 +82,12 @@ def obs_condition(**kwargs) -> Any:
8282
Default: 128
8383
mask : str, optional
8484
mask for the input maps in HEALPIX format.
85-
Default: rail_astro_tools/src/rail/examples_data/creation_data/data/survey_condi
86-
tions/DC2-mask-neg-nside-128.fits
85+
Default: rail/examples_data/creation_data/data/survey_conditions/DC2-mask-neg-
86+
nside-128.fits
8787
weight : str, optional
8888
weight for assigning pixels to galaxies in HEALPIX format.
89-
Default: rail_astro_tools/src/rail/examples_data/creation_data/data/survey_condi
90-
tions/DC2-dr6-galcounts-i20-i25.3-nside-128.fits
89+
Default: rail/examples_data/creation_data/data/survey_conditions/DC2-dr6-
90+
galcounts-i20-i25.3-nside-128.fits
9191
tot_nVis_flag : bool, optional
9292
flag indicating whether nVisYr is the total or average per year if supplied.
9393
Default: True

0 commit comments

Comments
 (0)