Skip to content

Commit 40fe7e0

Browse files
JoanneBogartJoanneBogart
authored
Merge pull request #4 from LSSTDESC/u/jrbogart/CI_resource_issue
U/jrbogart/ci resource issue Address CI resource problem. Add documentation.
2 parents 18677d7 + 0acb131 commit 40fe7e0

8 files changed

Lines changed: 217 additions & 36 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 23 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ jobs:
2121
strategy:
2222
matrix:
2323
os: [ ubuntu-latest ]
24-
py: [ "3.12" ]
24+
py: [ "3.13" ]
2525
CC: [ gcc ]
2626
CXX: [ g++ ]
2727

@@ -60,7 +60,7 @@ jobs:
6060
strategy:
6161
matrix:
6262
os: [ ubuntu-latest ]
63-
py: [ "3.12" ]
63+
py: [ "3.13" ]
6464
CC: [ gcc ]
6565
CXX: [ g++ ]
6666

@@ -71,6 +71,27 @@ jobs:
7171

7272
steps:
7373
- uses: actions/checkout@v4
74+
75+
- name: Set up Python ${{ matrix.py }}
76+
uses: actions/setup-python@v5
77+
with:
78+
python-version: ${{ matrix.py }}
79+
80+
- name: Verify pytho and pip versions
81+
run: |
82+
python --version
83+
pip --version
84+
85+
- name: Audit disk usage and remove unneeded tooling
86+
run: |
87+
df -h
88+
sudo rm -rf /usr/lib/jvm
89+
sudo rm -rf /usr/share/dotnet
90+
sudo rm -rf /usr/local/.ghcup
91+
sudo rm -rf /usr/local/lib/android
92+
sudo rm -rf /usr/local/chromium
93+
df -h
94+
7495
- name: Setup conda
7596
uses: conda-incubator/setup-miniconda@v3
7697
with:

‎.github/workflows/documentation.yml‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
name: Docs
2+
on: [push, pull_request, workflow_dispatch]
3+
jobs:
4+
docs:
5+
runs-on: ubuntu-latest
6+
steps:
7+
- uses: actions/checkout@v3
8+
- uses: actions/setup-python@v4
9+
with:
10+
python-version: "3.13"
11+
- name: Install dependencies
12+
run: |
13+
python -m pip install --upgrade pip
14+
python -m pip install .
15+
pip install sphinx sphinx_rtd_theme sphinx_toolbox sphinxcontrib-autoprogram sphinxcontrib.datatemplates
16+
- name: Sphinx build
17+
run: |
18+
sphinx-build docs _build
19+
- name: Deploy
20+
uses: peaceiris/actions-gh-pages@v3
21+
# Uncomment for debugging
22+
# if: ${{ github.ref == 'refs/heads/u/jrbogart/CI_resource_issue' }}
23+
# with:
24+
# publish_branch: gh-pages
25+
# github_token: ${{ secrets.GITHUB_TOKEN }}
26+
# publish_dir: _build/
27+
# force_orphan: true
28+
if: ${{ github.event_name == 'workflow_dispatch' && github.ref == 'refs/heads/main' }}
29+
with:
30+
publish_branch: gh-pages
31+
github_token: ${{ secrets.GITHUB_TOKEN }}
32+
publish_dir: _build/
33+
force_orphan: true

‎docs/Makefile‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Minimal makefile for Sphinx documentation
2+
#
3+
4+
# You can set these variables from the command line, and also
5+
# from the environment for the first two.
6+
SPHINXOPTS ?=
7+
SPHINXBUILD ?= sphinx-build
8+
SOURCEDIR = .
9+
BUILDDIR = _build
10+
11+
# Put it first so that "make" without argument is like "make help".
12+
help:
13+
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
14+
15+
.PHONY: help Makefile
16+
17+
# Catch-all target: route all unknown targets to Sphinx using the new
18+
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
19+
%: Makefile
20+
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

‎docs/conf.py‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Configuration file for the Sphinx documentation builder.
2+
#
3+
# This file only contains a selection of the most common options. For a full
4+
# list see the documentation:
5+
# https://www.sphinx-doc.org/en/master/usage/configuration.html
6+
7+
# -- Path setup --------------------------------------------------------------
8+
9+
10+
# Add top-level directory to path so python modules can be referred to
11+
# using the normal relative path
12+
import os
13+
import sys
14+
import importlib.util
15+
sys.path.insert(0, os.path.abspath('..'))
16+
17+
18+
# -- Project information -----------------------------------------------------
19+
20+
project = 'skyCatalogs_creator'
21+
copyright = '2025-2026, LSSTDESC'
22+
author = 'LSSTDESC'
23+
24+
# The full version, including alpha/beta/rc tags
25+
# release = '1.0.1'
26+
# Use load_skycatalogs_creator_version() to determine release dynamically
27+
28+
def load_skycatalogs_creator_version():
29+
"""Extract version of skyCatalogs_creator without importing the whole module"""
30+
31+
spec = importlib.util.spec_from_file_location(
32+
"skycatalogs_creator_version",
33+
os.path.join(os.path.dirname(__file__), "..", "skycatalogs_creator",
34+
"_version.py"),
35+
)
36+
module = importlib.util.module_from_spec(spec)
37+
spec.loader.exec_module(module)
38+
return module
39+
40+
41+
# -- General configuration ---------------------------------------------------
42+
43+
pygments_style = 'sphinx'
44+
# Add any Sphinx extension module names here, as strings. They can be
45+
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
46+
# ones.
47+
extensions = [
48+
"sphinx_rtd_theme",
49+
"sphinx.ext.autodoc",
50+
"sphinx.ext.napoleon"
51+
]
52+
# "sphinxcontrib.autoprogram" # for CLI; probably don't need
53+
# ]
54+
55+
# Add any paths that contain templates here, relative to this directory.
56+
templates_path = ['_templates']
57+
58+
# List of patterns, relative to source directory, that match files and
59+
# directories to ignore when looking for source files.
60+
# This pattern also affects html_static_path and html_extra_path.
61+
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
62+
63+
skycatalogs_creator_version = load_skycatalogs_creator_version()
64+
release = skycatalogs_creator_version.__version__
65+
66+
# -- Options for HTML output -------------------------------------------------
67+
68+
# The theme to use for HTML and HTML Help pages. See the documentation for
69+
# a list of builtin themes.
70+
#
71+
# Normally want this theme, but may need to comment out if module not available
72+
html_theme = 'sphinx_rtd_theme'
73+
74+
# Add any paths that contain custom static files (such as style sheets) here,
75+
# relative to this directory. They are copied after the builtin static files,
76+
# so a file named "default.css" will overwrite the builtin "default.css".
77+
html_static_path = ['_static']
78+
79+
# The sphinx_rtd_theme does not properly handle wrapping long lines in
80+
# table cells when rendering to HTML due to a CSS issue (see
81+
# https://github.com/readthedocs/sphinx_rtd_theme/issues/1505). Until
82+
# the issue is fixed upstream in sphinx_rtd_theme, we can simply
83+
# override the CSS here.
84+
rst_prolog = """
85+
.. raw:: html
86+
87+
<style>
88+
.wy-table-responsive table td,.wy-table-responsive table th{white-space:normal}
89+
</style>
90+
"""

‎docs/installation.rst‎

Lines changed: 30 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ Installing LSST science pipelines
2222
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2323
There are several methods of installation. Only the simplest (using a prebuilt cvmfs version) is described here. For other methods, see the imSim installation instructions.
2424

25-
If you are working at the USDF (Rubin Project computing) or at NERSC (DESC computing), perhaps the easiest way to setup and use *skyCatalogs* is to rely on the prebuilt versions of the pipelines contained in the cvmfs distribution which is installed there. This solution is also appropriate for personal laptops and university/lab based computing systems if you are able to install the *cvmfs* system.
25+
If you are working at the USDF (Rubin Project computing) or at NERSC (DESC computing), perhaps the easiest way to setup and use *skyCatalogs_creator* is to rely on the prebuilt versions of the pipelines contained in the cvmfs distribution which is installed there. This solution is also appropriate for personal laptops and university/lab based computing systems if you are able to install the *cvmfs* system.
2626

2727
The `CernVM file system <https://cvmfs.readthedocs.io/>`_ (cvmfs) is a distributed read-only file system developed at CERN for reliable low-maintenance world-wide software distribution. LSST-France distributes weekly builds of the Rubin science pipelines for both Linux and MacOS. Details and installation instructions can be found at `sw.lsst.eu <https://sw.lsst.eu/index.html>`_ . The distribution includes conda and skyCatalogs dependencies from conda-forge along with the science pipelines.
2828

@@ -35,7 +35,7 @@ First you need to setup the science pipelines. This involves sourcing a setup f
3535

3636
.. note::
3737

38-
Version ``w_2025_28`` or later of the science pipelines is recommended. This will guarantee other dependencies of skyCatalogs, such as GalSim, are new enough.
38+
Version ``w_2025_49`` or later of the science pipelines is recommended. This will guarantee other dependencies of skyCatalogs, such as GalSim, are new enough.
3939

4040
Also note: the cvmfs distribution is a read-only distribution. This means you cannot add packages to the included conda environment and packages you install via *pip* will be installed in the user area. If you need a *conda* environment you will need to use a different installation method.
4141

@@ -57,13 +57,19 @@ your installation directory SKYCATALOGS_HOME as described in the section :ref:`p
5757
5858
git clone https://github.com/LSSTDESC/skyCatalogs
5959
60-
at this point if you would only like to use *skyCatalogs* you can ``pip install skyCatalog/`` however we instead suggest using the *eups* tool to simply setup the package for use without installing it. This will allow you to edit the package in place, use multiple versions, change branches etc. You should definitely do this if you plan to do any *skyCatalogs* development.
60+
at this point if you would only like to use *skyCatalogs* you can ``pip install skyCatalog/`` however we instead suggest using the *eups* tool to simply setup the package for use without installing it. This will allow you to edit the package in place, use multiple versions, change branches etc. You should definitely do this if you plan to do any *skyCatalogs* or *skyCatalogs_creator* development.
6161

62-
If you do not intend to do any development you may choose instead to clone the most recent release tag. It should be at least v2.3.0.
62+
If you do not intend to do any development you may choose instead to clone or pip install the most recent release tag. It should be at least v2.4.0.
6363

6464
.. code-block:: sh
6565
66-
git clone https://github.com/LSSTDESC/skyCatalogs.git
66+
git clone https://github.com/LSSTDESC/skyCatalogs.git --branch v2.4.0
67+
68+
or
69+
70+
.. code-block:: sh
71+
72+
pip install skyCatalogs
6773
6874
.. _trilegal
6975
@@ -81,14 +87,12 @@ In order to create trilegal catalogs you need to install the pystellibs and astr
8187
pip install --no-build-isolation --no-deps astro-datalab
8288
8389
84-
Install skyCatalogs
85-
~~~~~~~~~~~~~~~~~~~
86-
87-
All you need to do is pip install:
90+
Install skyCatalogs_creator
91+
~~~~~~~~~~~~~~~~~~~~~~~~~~~
8892

8993
.. code-block:: sh
9094
91-
pip install skyCatalogs
95+
git clone https://github.com/LSSTDESC/skyCatalogs_creator.git
9296
9397
.. _install-data-files:
9498

@@ -102,7 +106,7 @@ Go to your `SKYCATALOGS_HOME` directory and download some needed data files (you
102106
curl https://s3df.slac.stanford.edu/groups/rubin/static/sim-data/rubin_sim_data/throughputs_2023_09_07.tgz | tar -C rubin_sim_data -xz
103107
curl https://s3df.slac.stanford.edu/groups/rubin/static/sim-data/sed_library/seds_170124.tar.gz | tar -C rubin_sim_data/sims_sed_library -xz
104108
105-
109+
106110
.. _per-session:
107111

108112
Per-session setup
@@ -122,7 +126,7 @@ define a ``SKYCATALOGS_HOME`` directory where other needed files (see e.g. secti
122126
# For data files
123127
export RUBIN_SIM_DATA_DIR=$SKYCATALOGS_HOME/rubin_sim_data
124128
export SIMS_SED_LIBRARY_DIR=$SKYCATALOGS_HOME/rubin_sim_data/sims_sed_library
125-
129+
126130
127131
If you're creating trilegal catalogs you also need to make pystellibs and
128132
the Astro Datalab software accessible.
@@ -132,23 +136,22 @@ You may need to do something like this:
132136
133137
export PYTHONPATH=${SKYCATALOGS_HOME}/pystellibs/src:${PYTHONPATH}
134138
135-
Using skyCatalogs
136-
-----------------
137-
138-
You should now be able to import the code you need from the skyCatalogs package, e.g.
139-
140-
.. code-block:: python
139+
Using skyCatalogs_creator
140+
-------------------------
141141

142-
from skycatalogs.skyCatalogs import open_catalog
143-
from skycatalogs.utils.shapes import Disk
142+
For object types handled by this package, the ouput catalog files are of two
143+
types: main files and flux files. For the most part main files contain
144+
quantities directly read from an upstream source, including everything
145+
needed to compute fluxes. The flux files are created in a separate step,
146+
using the information stored in the main files.
144147

145-
skycatalog_root = "path_to/skycatalog_files" # folder containing catalog
146-
config_file = "some_folder/skyCatalog.yaml"
148+
To create main files, use the script `create_main.py` in subdirectory
149+
`skycatalogs_creator/scripts`. For flux files use
150+
`create_flux.py`. To see what options are available type
147151

148-
cat = open_catalog(config_file, skycatalog_root=skycatalog_root)
152+
.. code-block:: sh
149153
150-
# define disk at ra, dec = 45.0, -9.0 of radius 100 arcseconds
151-
disk = disk(45.0, -9.0, 100.0)
154+
python skycatalogs/creator/scripts/create_main.py --help
155+
python skycatalogs/creator/scripts/create_flux.py --help
152156
153-
# get galaxies and stars in the region
154-
objects = cat.get_objects_by_region(disk, obj_type_set={'galaxy', 'star'})
157+
See also the page "Creating New Catalogs" in this site.

‎docs/usage_create.rst‎

Lines changed: 19 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -24,10 +24,17 @@ for most invocations and even fewer are required. The options determine
2424
One particularly handy option, ``--options-file``, allows you to specify
2525
everything else as key-value pairs in a yaml file.
2626

27+
.. note::
28+
For create_main `object_type` is a positional argument. If you use an
29+
options file, it must be specified in both places.
30+
2731
Options table
2832
+++++++++++++
29-
Here is the complete list of options as they appear in an options file.
30-
On the command line, prepend ``--`` and change all underscores to hyphens.
33+
Here is the complete list of options as they may appear in an options file as
34+
of January, 2026.
35+
36+
On the command line, prepend ``--`` (except `object_type` since it's a
37+
positional parameter) and change all underscores to hyphens.
3138

3239
===================== ========= ============ ===============================
3340
Name Datatype Default Description
@@ -55,7 +62,8 @@ knots_magnitude_cut float 27.0 Omit knots component from
5562
log_level string "INFO" Log level
5663
no_knots boolean False Omit knot component
5764
options_file string None Path to file where other
58-
options are set
65+
options are set. Valid on
66+
command line only.
5967
pixels int list [9556] healpix pixels for which
6068
catalog will be created
6169
skip_done boolean False do not overwrite existing files
@@ -74,9 +82,14 @@ The script ``create_flux.py`` and its options
7482
plus a couple new ones. Only ``object_type`` is required but several
7583
others are specified for most invocations.
7684

85+
.. note::
86+
For create_flux `object_type` is a keyword argument. If you use an
87+
options file, it must be specified in both places.
88+
7789
Options table
7890
+++++++++++++
79-
Here is the complete list of options as they appear in an options file.
91+
Here is the complete list of options as of January, 2026 as they would appear
92+
in an options file.
8093
On the command line, prepend ``--`` and change all underscores to hyphens.
8194

8295
===================== ========= ============ ===============================
@@ -98,7 +111,8 @@ include_roman_flux boolean False If True calculate & store Roman
98111
as well as Rubin fluxes.
99112
log_level string "INFO" Log level
100113
options_file string None Path to file where other
101-
options are set
114+
options are set. Valid only
115+
on command line.
102116
pixels int list [9556] healpix pixels for which
103117
catalog will be created
104118
skip_done boolean False do not overwrite existing files

‎etc/conda_requirements.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# conda install --file etc/conda_requirements should install all required dependencies of imSim including a conda based version of the Rubin pipelines.
22

3-
stackvana>=0.2025.28
3+
stackvana>=0.2025.49
44
gitpython
55
dustmaps
66
dust_extinction

‎skycatalogs_creator/_version.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
__version__ = "1.0.0"
1+
__version__ = "1.0.1"

0 commit comments

Comments
 (0)