From 54f9fdd767d64d7fc134aab42c5382f0ac87af4e Mon Sep 17 00:00:00 2001 From: "T. E. Pickering" Date: Mon, 12 Sep 2022 15:20:04 -0700 Subject: [PATCH 01/11] enable nitpicky mode for sphinx and clean up the various messes to get the docs to build cleanly --- docs/conf.py | 7 ++++-- docs/extinction.rst | 6 ++--- docs/index.rst | 6 ++--- docs/process/index.rst | 2 +- docs/specphot_standards.rst | 46 +++++++++++++++++----------------- specreduce/calibration_data.py | 32 +++++++++++------------ specreduce/extract.py | 32 +++++++++++------------ specreduce/tracing.py | 6 ++--- 8 files changed, 69 insertions(+), 68 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index 8c3855c7..cad6d72e 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -169,8 +169,11 @@ # -- Turn on nitpicky mode for sphinx (to warn about references not found) ---- # -# nitpicky = True -# nitpick_ignore = [] +nitpicky = True +nitpick_ignore = [ + ("py:class", "specutils.spectra.spectrum1d.Spectrum1D"), + ("py:obj", "specutils.Spectrum1D") +] # # Some warnings are impossible to suppress, and you can list specific references # that should be ignored in a nitpick-exceptions file which should be inside diff --git a/docs/extinction.rst b/docs/extinction.rst index daed27e1..204ba85a 100644 --- a/docs/extinction.rst +++ b/docs/extinction.rst @@ -11,7 +11,7 @@ and infrared. Supported Optical Extinction Models ----------------------------------- -`specreduce` offers support for average optical extinction models for a set of observatories: +``specreduce`` offers support for average optical extinction models for a set of observatories: .. csv-table:: :header: "Model Name", "Observatory", "Lat", "Lon", "Elevation (m)", "Ref" @@ -55,7 +55,7 @@ https://www.aanda.org/articles/aa/pdf/2011/03/aa15537-10.pdf. available at https://www.apo.nmsu.edu/arc35m/Instruments/DIS/ (https://www.apo.nmsu.edu/arc35m/Instruments/DIS/images/apoextinct.dat). In each case, the extinction is given in magnitudes per airmass and the wavelengths are in Angstroms. Here is an example that -uses the `AtmosphericExtinction` class to load each model and plots the extinction in magnitudes as well as fractional transmission +uses the ``AtmosphericExtinction`` class to load each model and plots the extinction in magnitudes as well as fractional transmission as a function of wavelength: .. plot:: @@ -76,7 +76,7 @@ as a function of wavelength: plt.tight_layout() fig.show() -A convenience class, `AtmosphericTransmission`, is provided for loading data files containing atmospheric transmission versus wavelength. +A convenience class, ``AtmosphericTransmission``, is provided for loading data files containing atmospheric transmission versus wavelength. The common use case for this would be loading the output of telluric models. By default it loads a telluric model for an airmass of 1 and 1 mm of precipitable water. Some resources for generating more realistic model atmospheric transmission spectra include https://mwvgroup.github.io/pwv_kpno/1.0.0/documentation/html/index.html and http://www.eso.org/sci/software/pipelines/skytools/molecfit. diff --git a/docs/index.rst b/docs/index.rst index 46ad89d2..caa6dcc4 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -2,12 +2,12 @@ Specreduce documentation ######################## -The `specreduce` package aims to provide a data reduction toolkit for optical +The ``specreduce`` package aims to provide a data reduction toolkit for optical and infrared spectroscopy, on which applications such as pipeline processes for specific instruments can be built. The scope of its functionality is limited to basic spectroscopic reduction, with basic *image* processing steps (such as -bias subtraction) instead covered by `ccdproc` and other packages, data -analysis by `specutils` and visualization by `specviz` or `cubeviz`. A few +bias subtraction) instead covered by ``ccdproc`` and other packages, data +analysis by ``specutils`` and visualization by ``specviz`` or ``cubeviz``. A few examples will nevertheless be provided of its usage in conjunction with these complementary packages. diff --git a/docs/process/index.rst b/docs/process/index.rst index 1dc33d91..ab86d176 100644 --- a/docs/process/index.rst +++ b/docs/process/index.rst @@ -6,7 +6,7 @@ Spectroscopic data reduction outline Here are some examples of complete DR processes, to help guide development (including basic image reduction steps, provided by other packages). These are -not a summary of functionality actually implemented in `specreduce` today. +not a summary of functionality actually implemented in ``specreduce`` today. .. _specreduce-nir-mos: diff --git a/docs/specphot_standards.rst b/docs/specphot_standards.rst index 5211987f..586f82de 100644 --- a/docs/specphot_standards.rst +++ b/docs/specphot_standards.rst @@ -7,7 +7,7 @@ Introduction ------------ Instrument sensitivity as a function of wavelength is calibrated using observations of -spectrophotometric standard stars. `~specreduce` offers some convenience functions for accessing +spectrophotometric standard stars. ``specreduce`` offers some convenience functions for accessing some databases of commonly used standard stars and loading the data into `~specutils.Spectrum1D` instances. @@ -18,7 +18,7 @@ Probably the most well-curated database of spectrophotometric calibration data i `CALSPEC `_ database at `MAST `_ (Ref.: `Bohlin, Gordon, & Tremblay 2014 `_). It also has the advantage of including data that extends well into both the UV and the IR. The `~specreduce.calibration_data.load_MAST_calspec` -function provides a way to easily load CALSPEC data either directly from `MAST` (specifically, https://archive.stsci.edu/hlsps/reference-atlases/cdbs/calspec/) +function provides a way to easily load CALSPEC data either directly from ``MAST`` (specifically, https://archive.stsci.edu/hlsps/reference-atlases/cdbs/calspec/) or from a previously downloaded local file. Here is an example of how to use it and of a CALSPEC standard that has both UV and IR coverage: .. plot:: @@ -38,25 +38,25 @@ or from a previously downloaded local file. Here is an example of how to use it fig.show() The `specreduce_data `_ package provides several datasets of spectrophotometric standard spectra. -The bulk of them are inherited from IRAF's `onedstds` datasets, but some more recently curated datasets from `ESO +The bulk of them are inherited from IRAF's ``onedstds`` datasets, but some more recently curated datasets from `ESO `_, the `Nearby Supernova Factory `_, and `Gemini `_ are included as well. The `~specreduce.calibration_data.load_onedstds` function is provided to load these data into `~specutils.Spectrum1D` -instances. If `specreduce_data` is not installed, the data will be downloaded from the GitHub +instances. If ``specreduce_data`` is not installed, the data will be downloaded from the GitHub `repository `_. The available database names and their descriptions are listed here. Please refer to the `specreduce-data repository `_ for details on the specific data files that are available: -- `bstdscal`: Directory of the brighter KPNO IRS standards (i.e. those with HR numbers) at 29 bandpasses, +- ``bstdscal``: Directory of the brighter KPNO IRS standards (i.e. those with HR numbers) at 29 bandpasses, data from various sources transformed to the Hayes and Latham system, unpublished. -- `ctiocal`: Directory containing fluxes for the southern tertiary standards as published by +- ``ctiocal``: Directory containing fluxes for the southern tertiary standards as published by `Baldwin & Stone, 1984, MNRAS, 206, 241 `_ and `Stone and Baldwin, 1983, MNRAS, 204, 347 `_. -- `ctionewcal`: Directory containing fluxes at 50A steps in the blue range 3300-7550A for the +- ``ctionewcal``: Directory containing fluxes at 50A steps in the blue range 3300-7550A for the tertiary standards of Baldwin and Stone derived from the revised calibration of `Hamuy et al., 1992, PASP, 104, 533 `_. This @@ -67,32 +67,32 @@ specific data files that are available: overlap. The separate red and blue fluxes may be selected by following the star name with "red" or "blue"; i.e. CD 32 blue. -- `iidscal`: Directory of the KPNO IIDS standards at 29 bandpasses, +- ``iidscal``: Directory of the KPNO IIDS standards at 29 bandpasses, data from various sources transformed to the Hayes and Latham system, unpublished. -- `irscal`: Directory of the KPNO IRS standards at 78 bandpasses, +- ``irscal``: Directory of the KPNO IRS standards at 78 bandpasses, data from various sources transformed to the Hayes and Latham system, unpublished (note that in this directory the - brighter standards have no values - the `bstdscal` directory + brighter standards have no values - the ``bstdscal`` directory must be used for these standards). -- `oke1990`: Directory of spectrophotometric standards observed for use with the HST, Table +- ``oke1990``: Directory of spectrophotometric standards observed for use with the HST, Table VII, `Oke 1990, AJ, 99, 1621 `_ (no correction was applied). An arbitrary 1A bandpass is specified for these smoothed and interpolated flux "points". -- `redcal`: Directory of standard stars with flux data beyond 8370A. +- ``redcal``: Directory of standard stars with flux data beyond 8370A. These stars are from the IRS or the IIDS directory but have data extending as far out into the red as the literature permits. Data from various sources. -- `spechayescal`: The KPNO spectrophotometric standards at the Hayes flux +- ``spechayescal``: The KPNO spectrophotometric standards at the Hayes flux points, Table IV, Spectrophotometric Standards, `Massey et al., 1988, ApJ 328, p. 315 `_. -- `spec16cal`: Directory containing fluxes at 16A steps in the blue +- ``spec16cal``: Directory containing fluxes at 16A steps in the blue range 3300-7550A for the secondary standards, published in `Hamuy et al., 1992, PASP, 104, 533 `_. This @@ -104,24 +104,24 @@ specific data files that are available: fluxes in the overlap. The separate red and blue fluxes may be selected by following the star name with "red" or "blue"; i.e. HR 1544 blue. -- `spec50cal`: The KPNO spectrophotometric standards at 50 A intervals. The data +- ``spec50cal``: The KPNO spectrophotometric standards at 50 A intervals. The data are from (1) Table V, Spectrophotometric Standards, `Massey et al., 1988, ApJ 328, p. 315 `_ and (2) Table 3, The Kitt Peak Spectrophotometric Standards: Extension to 1 micron, `Massey and Gronwall, 1990, ApJ 358, p. 344 `_. -- `snfactory`: Preferred standard stars from the LBL Nearby Supernova Factory project: +- ``snfactory``: Preferred standard stars from the LBL Nearby Supernova Factory project: https://ui.adsabs.harvard.edu/abs/2002SPIE.4836...61A/abstract Data compiled from https://snfactory.lbl.gov/snf/snf-specstars.html. See notes there for details and references. -- `eso`: Directories of spectrophotometric standards copied from +- ``eso``: Directories of spectrophotometric standards copied from ftp://ftp.eso.org/pub/stecf/standards/. See https://www.eso.org/sci/observing/tools/standards/spectra/stanlis.html for links, notes, and details. -- `gemini`: Directory of spectrophotometric standards used by Gemini. +- ``gemini``: Directory of spectrophotometric standards used by Gemini. Originally copied from https://github.com/GeminiDRSoftware/DRAGONS/tree/master/geminidr/gemini/lookups/spectrophotometric_standards. @@ -133,7 +133,7 @@ Many commonly used standard stars have spectra in multiple datasets, but the qua The `~specreduce.calibration_data.load_MAST_calspec` and `~specreduce.calibration_data.load_onedstds` functions can be useful tools for exploring and comparing spectra from the various databases. An example is shown here for `LTT 9491 `_ which has -spectra available from `MAST`, `ESO`, and the Nearby Supernova factory: +spectra available from ``MAST``, ``ESO``, and the Nearby Supernova factory: .. plot:: :include-source: @@ -156,8 +156,8 @@ spectra available from `MAST`, `ESO`, and the Nearby Supernova factory: ax.legend() fig.show() -The `MAST` data have the best UV coverage, but that's not useful from the ground and they only extend to 0.9 microns in the red in this case. -The other data extend to 1.0 microns, but both spectra show systematics due to telluric absorption. The `SNFactory` -data extend well past the atmospheric cutoff with no correction applied for atmospheric transmission. The `ESO` data, on the -other hand, are not corrected for the telluric features in the near-IR while the `SNFactory` data are. Regions affected by +The ``MAST`` data have the best UV coverage, but that's not useful from the ground and they only extend to 0.9 microns in the red in this case. +The other data extend to 1.0 microns, but both spectra show systematics due to telluric absorption. The ``SNFactory`` +data extend well past the atmospheric cutoff with no correction applied for atmospheric transmission. The ``ESO`` data, on the +other hand, are not corrected for the telluric features in the near-IR while the ``SNFactory`` data are. Regions affected by such telluric systematics should be masked out before these spectra are used for calibration purposes. diff --git a/specreduce/calibration_data.py b/specreduce/calibration_data.py index f23fdff9..c3c87f6a 100644 --- a/specreduce/calibration_data.py +++ b/specreduce/calibration_data.py @@ -23,8 +23,8 @@ ] """ -Make specreduce_data optional. If it's available, great and we can access its data via -pkg_resources. If not, we'll fall back to downloading and optionally caching it using +Make ``specreduce_data`` optional. If it's available, great and we can access its data via +``pkg_resources``. If not, we'll fall back to downloading and optionally caching it using `~astropy.utils.data`. """ LOCAL_DATA = True @@ -66,14 +66,14 @@ def get_reference_file_path(path=None, cache=False, show_progress=False): """ - Basic function to take a path to a file and load it via pkg_resources if the specreduce_data - package is available and load it via ghithub raw user content if not. + Basic function to take a path to a file and load it via ``pkg_resources`` if the ``specreduce_data`` + package is available and load it via github raw user content if not. Parameters ---------- path : str or None (default: None) Filename of reference file relative to the reference_data directory within - specreduce_data package. + ``specreduce_data`` package. cache : bool (default: False) Set whether file is cached if file is downloaded. @@ -125,12 +125,12 @@ def get_reference_file_path(path=None, cache=False, show_progress=False): def load_MAST_calspec(filename, remote=True, cache=True, show_progress=False): """ - Load a standard star spectrum from the `calspec` database at MAST. These spectra are provided in + Load a standard star spectrum from the ``calspec`` database at MAST. These spectra are provided in FITS format and are described in detail at: https://www.stsci.edu/hst/instrumentation/reference-data-for-calibration-and-tools/astronomical-catalogs/calspec # noqa - If `remote` is True, the spectrum will be downloaded from MAST. Set `remote` to False to load + If ``remote`` is True, the spectrum will be downloaded from MAST. Set ``remote`` to False to load a local file. Parameters @@ -139,7 +139,7 @@ def load_MAST_calspec(filename, remote=True, cache=True, show_progress=False): FITS filename of the standard star spectrum, e.g. g191b2b_005.fits. remote : bool (default = True) - If True, download the spectrum from MAST. If False, check if `filename` exists and load + If True, download the spectrum from MAST. If False, check if ``filename`` exists and load it. cache : bool (default = True) Toggle whether downloaded data is cached or not. @@ -148,10 +148,9 @@ def load_MAST_calspec(filename, remote=True, cache=True, show_progress=False): Returns ------- - spectrum : None or `~specutils.Spectrum1D` + spectrum : `~specutils.Spectrum1D` or None If the spectrum can be loaded, return it as a `~specutils.Spectrum1D`. - Otherwise return None. The spectral_axis units are Å (`~astropy.units.angstrom`) and - the flux units are milli-Janskys (`~astropy.units.mJy`). + Otherwise return None. The spectral_axis units are Å and the flux units are milli-Janskys. """ if remote: url = f"https://archive.stsci.edu/hlsps/reference-atlases/cdbs/calspec/{filename}" @@ -190,7 +189,7 @@ def load_MAST_calspec(filename, remote=True, cache=True, show_progress=False): def load_onedstds(dataset="snfactory", specfile="EG131.dat", cache=True, show_progress=False): """ This is a convenience function for loading a standard star spectrum from the 'onedstds' - dataset in the `~specreduce_data` package. If that package is installed, `~pkg_resources` + dataset in the ``specreduce_data`` package. If that package is installed, ``pkg_resources`` will be used to locate the data files locally. Otherwise they will be downloaded from the repository on github. @@ -212,8 +211,7 @@ def load_onedstds(dataset="snfactory", specfile="EG131.dat", cache=True, show_pr ------- spectrum : None or `~specutils.Spectrum1D` If the spectrum can be loaded, return it as a `~specutils.Spectrum1D`. - Otherwise return None. The spectral_axis units are Å (`~astropy.units.angstrom`) and - the flux units are milli-Janskys (`~astropy.units.mJy`). + Otherwise return None. The spectral_axis units are Å and the flux units are milli-Janskys. """ if dataset not in SPECPHOT_DATASETS: msg = (f"Specfied dataset, {dataset}, not in list of supported datasets of " @@ -248,12 +246,12 @@ class AtmosphericExtinction(Spectrum1D): Spectrum container for atmospheric extinction in magnitudes as a function of wavelength. If extinction and spectral_axis are provided, this will use them to build a custom model. If they are not, the 'model' parameter will be used to lookup and load a pre-defined - atmospheric extinction model from the `specreduce_data` package. + atmospheric extinction model from the ``specreduce_data`` package. Parameters ---------- model : str - Name of atmospheric extinction model provided by `specreduce_data`. Valid + Name of atmospheric extinction model provided by ``specreduce_data``. Valid options are: kpno - Kitt Peak National Observatory (default) @@ -270,7 +268,7 @@ class AtmosphericExtinction(Spectrum1D): to build custom atmospheric extinction model. If no units are provided, assumed to be given in magnitudes. - spectral_axis : `~astropy.units.Quantity` or `~specutils.SpectralCoord` or None + spectral_axis : `~astropy.units.Quantity` or `~astropy.coordinates.SpectralCoord` or None Optional Dispersion information with the same shape as the last (or only) dimension of flux, or one greater than the last dimension of flux if specifying bin edges. Used along with flux to build custom atmospheric diff --git a/specreduce/extract.py b/specreduce/extract.py index 62d28bd5..813c3e2f 100644 --- a/specreduce/extract.py +++ b/specreduce/extract.py @@ -183,7 +183,7 @@ class HorneExtract(SpecreduceOperation): image : `~astropy.nddata.NDData` or array-like, required The input 2D spectrum from which to extract a source. An NDData object must specify uncertainty and a mask. An array - requires use of the `variance`, `mask`, & `unit` arguments. + requires use of the ``variance``, ``mask``, & ``unit`` arguments. trace_object : `~specreduce.tracing.Trace`, required The associated 1D trace object created for the 2D image. @@ -199,22 +199,22 @@ class HorneExtract(SpecreduceOperation): [default: models.Polynomial1D(2)] variance : `~numpy.ndarray`, optional - (Only used if `image` is not an NDData object.) + (Only used if ``image`` is not an NDData object.) The associated variances for each pixel in the image. Must - have the same dimensions as `image`. If all zeros, the variance + have the same dimensions as ``image``. If all zeros, the variance will be ignored and treated as all ones. If any zeros, those elements will be excluded via masking. If any negative values, an error will be raised. [default: None] mask : `~numpy.ndarray`, optional - (Only used if `image` is not an NDData object.) + (Only used if ``image`` is not an NDData object.) Whether to mask each pixel in the image. Must have the same - dimensions as `image`. If blank, all non-NaN pixels are + dimensions as ``image``. If blank, all non-NaN pixels are unmasked. [default: None] - unit : `~astropy.units.core.Unit` or str, optional - (Only used if `image` is not an NDData object.) - The associated unit for the data in `image`. If blank, + unit : `~astropy.units.Unit` or str, optional + (Only used if ``image`` is not an NDData object.) + The associated unit for the data in ``image``. If blank, fluxes are interpreted as unitless. [default: None] """ @@ -246,7 +246,7 @@ def __call__(self, image=None, trace_object=None, image : `~astropy.nddata.NDData` or array-like, required The input 2D spectrum from which to extract a source. An NDData object must specify uncertainty and a mask. An array - requires use of the `variance`, `mask`, & `unit` arguments. + requires use of the ``variance``, ``mask``, & ``unit`` arguments. trace_object : `~specreduce.tracing.Trace`, required The associated 1D trace object created for the 2D image. @@ -261,22 +261,22 @@ def __call__(self, image=None, trace_object=None, A model for the image's background flux. variance : `~numpy.ndarray`, optional - (Only used if `image` is not an NDData object.) + (Only used if ``image`` is not an NDData object.) The associated variances for each pixel in the image. Must - have the same dimensions as `image`. If all zeros, the variance + have the same dimensions as ``image``. If all zeros, the variance will be ignored and treated as all ones. If any zeros, those elements will be excluded via masking. If any negative values, an error will be raised. mask : `~numpy.ndarray`, optional - (Only used if `image` is not an NDData object.) + (Only used if ``image`` is not an NDData object.) Whether to mask each pixel in the image. Must have the same - dimensions as `image`. If blank, all non-NaN pixels are + dimensions as ``image``. If blank, all non-NaN pixels are unmasked. - unit : `~astropy.units.core.Unit` or str, optional - (Only used if `image` is not an NDData object.) - The associated unit for the data in `image`. If blank, + unit : `~astropy.units.Unit` or str, optional + (Only used if ``image`` is not an NDData object.) + The associated unit for the data in ``image``. If blank, fluxes are interpreted as unitless. diff --git a/specreduce/tracing.py b/specreduce/tracing.py index 98b3bc34..0014711f 100644 --- a/specreduce/tracing.py +++ b/specreduce/tracing.py @@ -148,7 +148,7 @@ class KosmosTrace(Trace): trace along every pixel in the dispersion direction. (The original version of this algorithm is sourced from James - Davenport's `kosmos` repository.) + Davenport's ``kosmos`` repository.) Example: :: @@ -172,12 +172,12 @@ class KosmosTrace(Trace): finder. Good for tracing a fainter source if multiple traces are present. [default: None] window : int, optional - Fit the trace to a region with size `window * 2` around the + Fit the trace to a region with size ``window * 2`` around the guess position. Useful for tracing faint sources if multiple traces are present, but potentially bad if the trace is substantially bent or warped. [default: None] peak_method : string, optional - One of 'gaussian' (default), 'centroid', or 'max'. + One of ``gaussian`` (default), ``centroid``, or ``max``. gaussian: fits a gaussian to the window within each bin and adopts the central value. centroid: takes the centroid of the window within in bin. smooth_max: takes the position with the From e79a42dfbbd54cc0792a27e6c9662ef7506c6743 Mon Sep 17 00:00:00 2001 From: "T. E. Pickering" Date: Mon, 12 Sep 2022 15:28:54 -0700 Subject: [PATCH 02/11] fix line too long by 4 chars to make codestyle test happy --- specreduce/calibration_data.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specreduce/calibration_data.py b/specreduce/calibration_data.py index c3c87f6a..983e3bbf 100644 --- a/specreduce/calibration_data.py +++ b/specreduce/calibration_data.py @@ -66,8 +66,8 @@ def get_reference_file_path(path=None, cache=False, show_progress=False): """ - Basic function to take a path to a file and load it via ``pkg_resources`` if the ``specreduce_data`` - package is available and load it via github raw user content if not. + Basic function to take a path to a file and load it via ``pkg_resources`` if + the ``specreduce_data`` package is available and load it via github raw user content if not. Parameters ---------- From cb27de216211acd72084e2b9d868eaf5caa51d16 Mon Sep 17 00:00:00 2001 From: "T. E. Pickering" Date: Mon, 12 Sep 2022 21:31:04 -0700 Subject: [PATCH 03/11] Update docs/specphot_standards.rst Co-authored-by: P. L. Lim <2090236+pllim@users.noreply.github.com> --- docs/specphot_standards.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/specphot_standards.rst b/docs/specphot_standards.rst index 586f82de..30c933bd 100644 --- a/docs/specphot_standards.rst +++ b/docs/specphot_standards.rst @@ -18,7 +18,7 @@ Probably the most well-curated database of spectrophotometric calibration data i `CALSPEC `_ database at `MAST `_ (Ref.: `Bohlin, Gordon, & Tremblay 2014 `_). It also has the advantage of including data that extends well into both the UV and the IR. The `~specreduce.calibration_data.load_MAST_calspec` -function provides a way to easily load CALSPEC data either directly from ``MAST`` (specifically, https://archive.stsci.edu/hlsps/reference-atlases/cdbs/calspec/) +function provides a way to easily load CALSPEC data either directly from MAST (specifically, https://archive.stsci.edu/hlsps/reference-atlases/cdbs/calspec/) or from a previously downloaded local file. Here is an example of how to use it and of a CALSPEC standard that has both UV and IR coverage: .. plot:: From e1e0f290a90f8ca0f059a4f07741cef8c9eb2c36 Mon Sep 17 00:00:00 2001 From: "T. E. Pickering" Date: Mon, 12 Sep 2022 21:31:20 -0700 Subject: [PATCH 04/11] Update docs/specphot_standards.rst Co-authored-by: P. L. Lim <2090236+pllim@users.noreply.github.com> --- docs/specphot_standards.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/specphot_standards.rst b/docs/specphot_standards.rst index 30c933bd..3e8b3ee1 100644 --- a/docs/specphot_standards.rst +++ b/docs/specphot_standards.rst @@ -133,7 +133,7 @@ Many commonly used standard stars have spectra in multiple datasets, but the qua The `~specreduce.calibration_data.load_MAST_calspec` and `~specreduce.calibration_data.load_onedstds` functions can be useful tools for exploring and comparing spectra from the various databases. An example is shown here for `LTT 9491 `_ which has -spectra available from ``MAST``, ``ESO``, and the Nearby Supernova factory: +spectra available from MAST, ESO, and the Nearby Supernova factory: .. plot:: :include-source: From 1c2a440bfd3b8bd707d711c2ee3d29bc2d244849 Mon Sep 17 00:00:00 2001 From: "T. E. Pickering" Date: Mon, 12 Sep 2022 21:31:29 -0700 Subject: [PATCH 05/11] Update docs/specphot_standards.rst Co-authored-by: P. L. Lim <2090236+pllim@users.noreply.github.com> --- docs/specphot_standards.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/specphot_standards.rst b/docs/specphot_standards.rst index 3e8b3ee1..25ccc075 100644 --- a/docs/specphot_standards.rst +++ b/docs/specphot_standards.rst @@ -156,7 +156,7 @@ spectra available from MAST, ESO, and the Nearby Supernova factory: ax.legend() fig.show() -The ``MAST`` data have the best UV coverage, but that's not useful from the ground and they only extend to 0.9 microns in the red in this case. +The MAST data have the best UV coverage, but that's not useful from the ground and they only extend to 0.9 microns in the red in this case. The other data extend to 1.0 microns, but both spectra show systematics due to telluric absorption. The ``SNFactory`` data extend well past the atmospheric cutoff with no correction applied for atmospheric transmission. The ``ESO`` data, on the other hand, are not corrected for the telluric features in the near-IR while the ``SNFactory`` data are. Regions affected by From f3e50cceb874a70f2579c8b2bc1d02112790fb44 Mon Sep 17 00:00:00 2001 From: "T. E. Pickering" Date: Mon, 12 Sep 2022 21:33:36 -0700 Subject: [PATCH 06/11] Update specreduce/calibration_data.py Co-authored-by: P. L. Lim <2090236+pllim@users.noreply.github.com> --- specreduce/calibration_data.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specreduce/calibration_data.py b/specreduce/calibration_data.py index 983e3bbf..0bb3a95a 100644 --- a/specreduce/calibration_data.py +++ b/specreduce/calibration_data.py @@ -67,7 +67,7 @@ def get_reference_file_path(path=None, cache=False, show_progress=False): """ Basic function to take a path to a file and load it via ``pkg_resources`` if - the ``specreduce_data`` package is available and load it via github raw user content if not. + the ``specreduce_data`` package is available and load it via GitHub raw user content if not. Parameters ---------- From 80a434df8362750449262b6184e835e97382ef0c Mon Sep 17 00:00:00 2001 From: "T. E. Pickering" Date: Tue, 13 Sep 2022 11:32:08 -0700 Subject: [PATCH 07/11] manually fix sphinx mapping to point to documentation it wasn't finding automatically --- docs/conf.py | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index cad6d72e..906387b8 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -170,10 +170,13 @@ # -- Turn on nitpicky mode for sphinx (to warn about references not found) ---- # nitpicky = True -nitpick_ignore = [ - ("py:class", "specutils.spectra.spectrum1d.Spectrum1D"), - ("py:obj", "specutils.Spectrum1D") -] +intersphinx_mapping.update( + { + 'astropy': ('https://docs.astropy.org/en/latest/', None), + 'ccdproc': ('https://ccdproc.readthedocs.io/en/latest/', None), + 'specutils': ('https://specutils.readthedocs.io/en/latest/', None) + } +) # # Some warnings are impossible to suppress, and you can list specific references # that should be ignored in a nitpick-exceptions file which should be inside From 88817ff2f38c16f7d476a2113189e1e8360776a7 Mon Sep 17 00:00:00 2001 From: "T. E. Pickering" Date: Tue, 13 Sep 2022 12:34:24 -0700 Subject: [PATCH 08/11] various documentation fixes and fleshing out of links --- docs/extinction.rst | 10 ++--- docs/index.rst | 6 +-- docs/specphot_standards.rst | 71 ++++++++++++++++++++-------------- specreduce/calibration_data.py | 3 +- 4 files changed, 51 insertions(+), 39 deletions(-) diff --git a/docs/extinction.rst b/docs/extinction.rst index 204ba85a..90536a83 100644 --- a/docs/extinction.rst +++ b/docs/extinction.rst @@ -55,8 +55,8 @@ https://www.aanda.org/articles/aa/pdf/2011/03/aa15537-10.pdf. available at https://www.apo.nmsu.edu/arc35m/Instruments/DIS/ (https://www.apo.nmsu.edu/arc35m/Instruments/DIS/images/apoextinct.dat). In each case, the extinction is given in magnitudes per airmass and the wavelengths are in Angstroms. Here is an example that -uses the ``AtmosphericExtinction`` class to load each model and plots the extinction in magnitudes as well as fractional transmission -as a function of wavelength: +uses the `~specreduce.calibration_data.AtmosphericExtinction` class to load each model and plots the extinction in magnitudes as well +as fractional transmission as a function of wavelength: .. plot:: :include-source: @@ -76,9 +76,9 @@ as a function of wavelength: plt.tight_layout() fig.show() -A convenience class, ``AtmosphericTransmission``, is provided for loading data files containing atmospheric transmission versus wavelength. -The common use case for this would be loading the output of telluric models. By default it loads a telluric model for an airmass of 1 and -1 mm of precipitable water. Some resources for generating more realistic model atmospheric transmission spectra include +A convenience class, `~specreduce.calibration_data.AtmosphericTransmission`, is provided for loading data files containing atmospheric transmission +versus wavelength. The common use case for this would be loading the output of telluric models. By default it loads a telluric model for an +airmass of 1 and 1 mm of precipitable water. Some resources for generating more realistic model atmospheric transmission spectra include https://mwvgroup.github.io/pwv_kpno/1.0.0/documentation/html/index.html and http://www.eso.org/sci/software/pipelines/skytools/molecfit. .. plot:: diff --git a/docs/index.rst b/docs/index.rst index caa6dcc4..4139117c 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -2,12 +2,12 @@ Specreduce documentation ######################## -The ``specreduce`` package aims to provide a data reduction toolkit for optical +The `specreduce `_ package aims to provide a data reduction toolkit for optical and infrared spectroscopy, on which applications such as pipeline processes for specific instruments can be built. The scope of its functionality is limited to basic spectroscopic reduction, with basic *image* processing steps (such as -bias subtraction) instead covered by ``ccdproc`` and other packages, data -analysis by ``specutils`` and visualization by ``specviz`` or ``cubeviz``. A few +bias subtraction) instead covered by `ccdproc `_ and other packages, data +analysis by `specutils `_ and visualization by `matplotlib `_. A few examples will nevertheless be provided of its usage in conjunction with these complementary packages. diff --git a/docs/specphot_standards.rst b/docs/specphot_standards.rst index 25ccc075..564f8615 100644 --- a/docs/specphot_standards.rst +++ b/docs/specphot_standards.rst @@ -7,8 +7,8 @@ Introduction ------------ Instrument sensitivity as a function of wavelength is calibrated using observations of -spectrophotometric standard stars. ``specreduce`` offers some convenience functions for accessing -some databases of commonly used standard stars and loading the data into `~specutils.Spectrum1D` +spectrophotometric standard stars. `specreduce `_ offers some +convenience functions for accessing some databases of commonly used standard stars and loading the data into `~specutils.Spectrum1D` instances. Supported Databases @@ -18,8 +18,9 @@ Probably the most well-curated database of spectrophotometric calibration data i `CALSPEC `_ database at `MAST `_ (Ref.: `Bohlin, Gordon, & Tremblay 2014 `_). It also has the advantage of including data that extends well into both the UV and the IR. The `~specreduce.calibration_data.load_MAST_calspec` -function provides a way to easily load CALSPEC data either directly from MAST (specifically, https://archive.stsci.edu/hlsps/reference-atlases/cdbs/calspec/) -or from a previously downloaded local file. Here is an example of how to use it and of a CALSPEC standard that has both UV and IR coverage: +function provides a way to easily load CALSPEC data either directly from `MAST `_ +(specifically, https://archive.stsci.edu/hlsps/reference-atlases/cdbs/calspec/) or from a previously downloaded local file. +Here is an example of how to use it and of a CALSPEC standard that has both UV and IR coverage: .. plot:: :include-source: @@ -38,25 +39,27 @@ or from a previously downloaded local file. Here is an example of how to use it fig.show() The `specreduce_data `_ package provides several datasets of spectrophotometric standard spectra. -The bulk of them are inherited from IRAF's ``onedstds`` datasets, but some more recently curated datasets from `ESO -`_, the +The bulk of them are inherited from IRAF's `onedstds `_ datasets, but +some more recently curated datasets from `ESO `_, the `Nearby Supernova Factory `_, and `Gemini `_ are included as well. The `~specreduce.calibration_data.load_onedstds` function is provided to load these data into `~specutils.Spectrum1D` -instances. If ``specreduce_data`` is not installed, the data will be downloaded from the GitHub +instances. If `specreduce_data `_ is not installed, the data will be downloaded from the GitHub `repository `_. The available database names and their descriptions are listed here. Please refer to the `specreduce-data repository `_ for details on the specific data files that are available: -- ``bstdscal``: Directory of the brighter KPNO IRS standards (i.e. those with HR numbers) at 29 bandpasses, - data from various sources transformed to the Hayes and Latham system, unpublished. +- `bstdscal `_: Directory of the brighter KPNO IRS + standards (i.e. those with HR numbers) at 29 bandpasses, data from various sources transformed to the Hayes and Latham system, unpublished. -- ``ctiocal``: Directory containing fluxes for the southern tertiary standards as published by +- `ctiocal `_: + Directory containing fluxes for the southern tertiary standards as published by `Baldwin & Stone, 1984, MNRAS, 206, 241 `_ and `Stone and Baldwin, 1983, MNRAS, 204, 347 `_. -- ``ctionewcal``: Directory containing fluxes at 50A steps in the blue range 3300-7550A for the +- `ctionewcal `_: + Directory containing fluxes at 50A steps in the blue range 3300-7550A for the tertiary standards of Baldwin and Stone derived from the revised calibration of `Hamuy et al., 1992, PASP, 104, 533 `_. This @@ -67,32 +70,38 @@ specific data files that are available: overlap. The separate red and blue fluxes may be selected by following the star name with "red" or "blue"; i.e. CD 32 blue. -- ``iidscal``: Directory of the KPNO IIDS standards at 29 bandpasses, +- `iidscal `_: + Directory of the KPNO IIDS standards at 29 bandpasses, data from various sources transformed to the Hayes and Latham system, unpublished. -- ``irscal``: Directory of the KPNO IRS standards at 78 bandpasses, +- `irscal `_: + Directory of the KPNO IRS standards at 78 bandpasses, data from various sources transformed to the Hayes and Latham system, unpublished (note that in this directory the - brighter standards have no values - the ``bstdscal`` directory - must be used for these standards). + brighter standards have no values - the `bstdscal `_ + directory must be used for these standards). -- ``oke1990``: Directory of spectrophotometric standards observed for use with the HST, Table +- `oke1990 `_: + Directory of spectrophotometric standards observed for use with the HST, Table VII, `Oke 1990, AJ, 99, 1621 `_ (no correction was applied). An arbitrary 1A bandpass is specified for these smoothed and interpolated flux "points". -- ``redcal``: Directory of standard stars with flux data beyond 8370A. +- `redcal `_: + Directory of standard stars with flux data beyond 8370A. These stars are from the IRS or the IIDS directory but have data extending as far out into the red as the literature permits. Data from various sources. -- ``spechayescal``: The KPNO spectrophotometric standards at the Hayes flux +- `spechayescal `_: + The KPNO spectrophotometric standards at the Hayes flux points, Table IV, Spectrophotometric Standards, `Massey et al., 1988, ApJ 328, p. 315 `_. -- ``spec16cal``: Directory containing fluxes at 16A steps in the blue +- `spec16cal `_: + Directory containing fluxes at 16A steps in the blue range 3300-7550A for the secondary standards, published in `Hamuy et al., 1992, PASP, 104, 533 `_. This @@ -104,25 +113,28 @@ specific data files that are available: fluxes in the overlap. The separate red and blue fluxes may be selected by following the star name with "red" or "blue"; i.e. HR 1544 blue. -- ``spec50cal``: The KPNO spectrophotometric standards at 50 A intervals. The data +- `spec50cal `_: + The KPNO spectrophotometric standards at 50 A intervals. The data are from (1) Table V, Spectrophotometric Standards, `Massey et al., 1988, ApJ 328, p. 315 `_ and (2) Table 3, The Kitt Peak Spectrophotometric Standards: Extension to 1 micron, `Massey and Gronwall, 1990, ApJ 358, p. 344 `_. -- ``snfactory``: Preferred standard stars from the LBL Nearby Supernova Factory project: +- `snfactory `_: + Preferred standard stars from the LBL Nearby Supernova Factory project: https://ui.adsabs.harvard.edu/abs/2002SPIE.4836...61A/abstract Data compiled from https://snfactory.lbl.gov/snf/snf-specstars.html. See notes there for details and references. -- ``eso``: Directories of spectrophotometric standards copied from +- `eso`_: + Directories of spectrophotometric standards copied from ftp://ftp.eso.org/pub/stecf/standards/. See https://www.eso.org/sci/observing/tools/standards/spectra/stanlis.html for links, notes, and details. -- ``gemini``: Directory of spectrophotometric standards used by Gemini. - Originally copied from +- `gemini `_: + Directory of spectrophotometric standards used by Gemini. Originally copied from https://github.com/GeminiDRSoftware/DRAGONS/tree/master/geminidr/gemini/lookups/spectrophotometric_standards. @@ -156,8 +168,9 @@ spectra available from MAST, ESO, and the Nearby Supernova factory: ax.legend() fig.show() -The MAST data have the best UV coverage, but that's not useful from the ground and they only extend to 0.9 microns in the red in this case. -The other data extend to 1.0 microns, but both spectra show systematics due to telluric absorption. The ``SNFactory`` -data extend well past the atmospheric cutoff with no correction applied for atmospheric transmission. The ``ESO`` data, on the -other hand, are not corrected for the telluric features in the near-IR while the ``SNFactory`` data are. Regions affected by -such telluric systematics should be masked out before these spectra are used for calibration purposes. +The `MAST`_ data have the best UV coverage, but that's not useful from the ground and +they only extend to 0.9 microns in the red in this case. The other data extend to 1.0 microns, but both spectra show +systematics due to telluric absorption. The `SNFactory`_ data extend well past the atmospheric +cutoff with no correction applied for atmospheric transmission. The `ESO`_ data, on the other hand, +are not corrected for the telluric features in the near-IR while the `SNFactory`_ data are. +Regions affected by such telluric systematics should be masked out before these spectra are used for calibration purposes. diff --git a/specreduce/calibration_data.py b/specreduce/calibration_data.py index 0bb3a95a..730ed86f 100644 --- a/specreduce/calibration_data.py +++ b/specreduce/calibration_data.py @@ -23,8 +23,7 @@ ] """ -Make ``specreduce_data`` optional. If it's available, great and we can access its data via -``pkg_resources``. If not, we'll fall back to downloading and optionally caching it using +If specreduce_data is not available, we'll fall back to downloading and optionally caching it using `~astropy.utils.data`. """ LOCAL_DATA = True From ebfe870b6d0a05b1865c0839e5dd9323bd84f65e Mon Sep 17 00:00:00 2001 From: "T. E. Pickering" Date: Tue, 13 Sep 2022 14:10:15 -0700 Subject: [PATCH 09/11] switch default doc mapping to stable versions --- docs/conf.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index 906387b8..7a668af0 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -172,9 +172,9 @@ nitpicky = True intersphinx_mapping.update( { - 'astropy': ('https://docs.astropy.org/en/latest/', None), - 'ccdproc': ('https://ccdproc.readthedocs.io/en/latest/', None), - 'specutils': ('https://specutils.readthedocs.io/en/latest/', None) + 'astropy': ('https://docs.astropy.org/en/stable/', None), + 'ccdproc': ('https://ccdproc.readthedocs.io/en/stable/', None), + 'specutils': ('https://specutils.readthedocs.io/en/stable/', None) } ) # From 856dfa396a840139d0a3468831e6c71aadae1c16 Mon Sep 17 00:00:00 2001 From: "T. E. Pickering" Date: Tue, 13 Sep 2022 14:18:46 -0700 Subject: [PATCH 10/11] minor typos --- docs/specphot_standards.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/specphot_standards.rst b/docs/specphot_standards.rst index 564f8615..0444903a 100644 --- a/docs/specphot_standards.rst +++ b/docs/specphot_standards.rst @@ -51,7 +51,7 @@ database names and their descriptions are listed here. Please refer to the `spec specific data files that are available: - `bstdscal `_: Directory of the brighter KPNO IRS - standards (i.e. those with HR numbers) at 29 bandpasses, data from various sources transformed to the Hayes and Latham system, unpublished. + standards (i.e., those with HR numbers) at 29 bandpasses, data from various sources transformed to the Hayes and Latham system, unpublished. - `ctiocal `_: Directory containing fluxes for the southern tertiary standards as published by @@ -66,7 +66,7 @@ specific data files that are available: directory also contains the fluxes of the tertiaries in the red (6050-10000A) at 50A steps as will be published in PASP (Hamuy et al 1994). The combined fluxes are obtained by gray shifting the blue fluxes to match the red fluxes in the - overlap region of 6500A-7500A and averaging the red and blue fluxes in the + overlap region of 6500-7500A and averaging the red and blue fluxes in the overlap. The separate red and blue fluxes may be selected by following the star name with "red" or "blue"; i.e. CD 32 blue. @@ -109,7 +109,7 @@ specific data files that are available: at 16A steps as will be published in PASP (`Hamuy et al 1994 `_). The combined fluxes are obtained by gray shifting the blue fluxes to match the red - fluxes in the overlap region of 6500A-7500A and averaging the blue and red + fluxes in the overlap region of 6500-7500A and averaging the blue and red fluxes in the overlap. The separate red and blue fluxes may be selected by following the star name with "red" or "blue"; i.e. HR 1544 blue. From 76f75ad96b1742dd52e5eda3ddd4c637a78bdd40 Mon Sep 17 00:00:00 2001 From: "T. E. Pickering" Date: Tue, 13 Sep 2022 17:29:30 -0700 Subject: [PATCH 11/11] fix reference and convert to real angstrom symbols --- docs/process/index.rst | 3 ++- docs/specphot_standards.rst | 24 ++++++++++++------------ 2 files changed, 14 insertions(+), 13 deletions(-) diff --git a/docs/process/index.rst b/docs/process/index.rst index ab86d176..ed659d19 100644 --- a/docs/process/index.rst +++ b/docs/process/index.rst @@ -6,7 +6,8 @@ Spectroscopic data reduction outline Here are some examples of complete DR processes, to help guide development (including basic image reduction steps, provided by other packages). These are -not a summary of functionality actually implemented in ``specreduce`` today. +not a summary of functionality actually implemented in +`specreduce `_ today. .. _specreduce-nir-mos: diff --git a/docs/specphot_standards.rst b/docs/specphot_standards.rst index 0444903a..5ad9be22 100644 --- a/docs/specphot_standards.rst +++ b/docs/specphot_standards.rst @@ -59,14 +59,14 @@ specific data files that are available: and `Stone and Baldwin, 1983, MNRAS, 204, 347 `_. - `ctionewcal `_: - Directory containing fluxes at 50A steps in the blue range 3300-7550A for the + Directory containing fluxes at 50 Å steps in the blue range 3300-7550 Å for the tertiary standards of Baldwin and Stone derived from the revised calibration of `Hamuy et al., 1992, PASP, 104, 533 `_. This - directory also contains the fluxes of the tertiaries in the red (6050-10000A) at - 50A steps as will be published in PASP (Hamuy et al 1994). The combined fluxes + directory also contains the fluxes of the tertiaries in the red (6050-10000 Å) at + 50 Å steps as will be published in PASP (Hamuy et al 1994). The combined fluxes are obtained by gray shifting the blue fluxes to match the red fluxes in the - overlap region of 6500-7500A and averaging the red and blue fluxes in the + overlap region of 6500-7500 Å and averaging the red and blue fluxes in the overlap. The separate red and blue fluxes may be selected by following the star name with "red" or "blue"; i.e. CD 32 blue. @@ -86,11 +86,11 @@ specific data files that are available: Directory of spectrophotometric standards observed for use with the HST, Table VII, `Oke 1990, AJ, 99, 1621 `_ (no - correction was applied). An arbitrary 1A bandpass is specified for these + correction was applied). An arbitrary 1 Å bandpass is specified for these smoothed and interpolated flux "points". - `redcal `_: - Directory of standard stars with flux data beyond 8370A. + Directory of standard stars with flux data beyond 8370 Å. These stars are from the IRS or the IIDS directory but have data extending as far out into the red as the literature permits. Data from various sources. @@ -101,20 +101,20 @@ specific data files that are available: et al., 1988, ApJ 328, p. 315 `_. - `spec16cal `_: - Directory containing fluxes at 16A steps in the blue - range 3300-7550A for the secondary standards, published in `Hamuy et al., 1992, + Directory containing fluxes at 16 Å steps in the blue + range 3300-7550 Å for the secondary standards, published in `Hamuy et al., 1992, PASP, 104, 533 `_. This - directory also contains the fluxes of the secondaries in the red (6020-10300A) - at 16A steps as will be published in PASP (`Hamuy et al 1994 + directory also contains the fluxes of the secondaries in the red (6020-10300 Å) + at 16 Å steps as will be published in PASP (`Hamuy et al 1994 `_). The combined fluxes are obtained by gray shifting the blue fluxes to match the red - fluxes in the overlap region of 6500-7500A and averaging the blue and red + fluxes in the overlap region of 6500-7500 Å and averaging the blue and red fluxes in the overlap. The separate red and blue fluxes may be selected by following the star name with "red" or "blue"; i.e. HR 1544 blue. - `spec50cal `_: - The KPNO spectrophotometric standards at 50 A intervals. The data + The KPNO spectrophotometric standards at 50 Å intervals. The data are from (1) Table V, Spectrophotometric Standards, `Massey et al., 1988, ApJ 328, p. 315 `_ and (2) Table 3, The Kitt Peak Spectrophotometric Standards: Extension to 1