diff --git a/notebooks/MIRI/MIRI_LRS_spectral_extraction/data.zip b/notebooks/MIRI/MIRI_LRS_spectral_extraction/data.zip new file mode 100644 index 000000000..53c78674e Binary files /dev/null and b/notebooks/MIRI/MIRI_LRS_spectral_extraction/data.zip differ diff --git a/notebooks/MIRI/MIRI_LRS_spectral_extraction/miri_lrs_advanced_extraction_part1.ipynb b/notebooks/MIRI/MIRI_LRS_spectral_extraction/miri_lrs_advanced_extraction_part1.ipynb deleted file mode 100644 index d8093b8d5..000000000 --- a/notebooks/MIRI/MIRI_LRS_spectral_extraction/miri_lrs_advanced_extraction_part1.ipynb +++ /dev/null @@ -1,815 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "id": "20ce4694", - "metadata": {}, - "source": [ - "# MIRI LRS Slit Spectroscopy: Spectral Extraction using the JWST Pipeline\n", - "\n", - "July 2023\n", - "\n", - "**Use case:** Spectral extraction of slit spectra with the JWST calibration pipeline.
\n", - "**Data:** Publicly available science data
\n", - "**Tools:** jwst, matplotlib, astropy.
\n", - "**Cross-intrument:** NIRSpec, MIRI.
\n", - "**Documentation:** This notebook is part of a STScI's larger [post-pipeline Data Analysis Tools Ecosystem](https://jwst-docs.stsci.edu/jwst-post-pipeline-data-analysis) and can be [downloaded](https://github.com/spacetelescope/dat_pyinthesky/tree/main/jdat_notebooks/MRS_Mstar_analysis) directly from the [JDAT Notebook Github directory](https://github.com/spacetelescope/jdat_notebooks).
\n", - "\n", - "\n", - "### Introduction: Spectral extraction in the JWST calibration pipeline\n", - "\n", - "The JWST calibration pipeline performs spectrac extraction for all spectroscopic data using basic default assumptions that are tuned to produce accurately calibrated spectra for the majority of science cases. This default method is a simple fixed-width boxcar extraction, where the spectrum is summed over a number of pixels along the cross-dispersion axis, over the valid wavelength range. An aperture correction is applied at each pixel along the spectrum to account for flux lost from the finite-width aperture. \n", - "\n", - "The ``extract_1d`` step uses the following inputs for its algorithm:\n", - "- the spectral extraction reference file: this is a json-formatted file, available as a reference file from the [JWST CRDS system](https://jwst-crds.stsci.edu)\n", - "- the bounding box: the ``assign_wcs`` step attaches a bounding box definition to the data, which defines the region over which a valid calibration is available. We will demonstrate below how to visualize this region. \n", - "\n", - "However the ``extract_1d`` step has the capability to perform more complex spectral extractions, requiring some manual editing of parameters and re-running of the pipeline step. \n", - "\n", - "\n", - "### Aims\n", - "\n", - "This notebook will demonstrate how to re-run the spectral extraction step with different settings to illustrate the capabilities of the JWST calibration pipeline. \n", - "\n", - "\n", - "### Assumptions\n", - "\n", - "We will demonstrate the spectral extraction methods on resampled, calibrated spectral images. The basic demo and two examples run on Level 3 data, in which the nod exposures have been combined into a single spectral image. Two examples will use the Level 2b data - one of the nodded exposures. \n", - "\n", - "\n", - "### Test data\n", - "\n", - "The data used in this notebook is an observation of the Type Ia supernova SN2021aefx, observed by Jha et al in PID 2072 (Obs 1). These data were taken with zero exclusive access period, and published in [Kwok et al 2023](https://ui.adsabs.harvard.edu/abs/2023ApJ...944L...3K/abstract). You can retrieve the data from [this Box folder](https://stsci.box.com/s/i2xi18jziu1iawpkom0z2r94kvf9n9kb), and we recommend you place the files in the ``data/`` folder of this repository, or change the directory settings in the notebook prior to running. \n", - "\n", - "You can of course use your own data instead of the demo data. \n", - "\n", - "\n", - "### JWST pipeline version and CRDS context\n", - "\n", - "This notebook was written using the calibration pipeline version 1.10.2. We set the CRDS context explicitly to 1089 to match the current latest version in MAST. If you use different pipeline versions or CRDS context, please read the relevant release notes ([here for pipeline](https://github.com/spacetelescope/jwst), [here for CRDS](https://jwst-crds.stsci.edu)) for possibly relevant changes.\n", - "\n", - "### Contents\n", - "\n", - "1. [The Level 3 data products](#l3data)\n", - "2. [The spectral extraction reference file](#x1dref)\n", - "3. [Example 1: Changing the aperture width](#ex1)\n", - "4. [Example 2: Changing the aperture location](#ex2)\n", - "5. [Example 3: Extraction with background subtraction](#ex3)\n", - "6. [Example 4: Tapered column extraction](#ex4)\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "id": "14ff9543", - "metadata": {}, - "source": [ - "## Import Packages" - ] - }, - { - "cell_type": "markdown", - "id": "e698ce3a-fdaf-4d3b-9109-b980794e94aa", - "metadata": {}, - "source": [ - "- `astropy.io` fits for accessing FITS files\n", - "- `os` for managing system paths\n", - "- `matplotlib` for plotting data\n", - "- `urllib` for downloading data\n", - "- `tarfile` for unpacking data\n", - "- `numpy` for basic array manipulation\n", - "- `jwst` for running JWST pipeline and handling data products\n", - "- `json` for working with json files\n", - "- `crds` for working with JWST reference files" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "3a130dd2", - "metadata": {}, - "outputs": [], - "source": [ - "# Set CRDS variables first\n", - "import os\n", - "\n", - "os.environ['CRDS_CONTEXT'] = 'jwst_1089.pmap'\n", - "os.environ['CRDS_PATH'] = os.environ['HOME']+'/crds_cache'\n", - "os.environ['CRDS_SERVER_URL'] = 'https://jwst-crds.stsci.edu'\n", - "print(f'CRDS cache location: {os.environ[\"CRDS_PATH\"]}')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "08ddf5f7", - "metadata": {}, - "outputs": [], - "source": [ - "%matplotlib inline\n", - "import urllib.request\n", - "import tarfile\n", - "\n", - "import numpy as np\n", - "import matplotlib.pyplot as plt\n", - "\n", - "import astropy.io.fits as fits\n", - "import astropy.units as u\n", - "from astropy.modeling import models, fitting\n", - "\n", - "import jwst\n", - "from jwst import datamodels\n", - "from jwst.extract_1d import Extract1dStep\n", - "\n", - "from matplotlib.patches import Rectangle\n", - "\n", - "import json\n", - "import crds\n", - "\n", - "print(f'Using JWST calibration pipeline version {jwst.__version__}')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "305103d5", - "metadata": {}, - "outputs": [], - "source": [ - "data_tar_url = 'https://data.science.stsci.edu/redirect/JWST/jwst-data_analysis_tools/MIRI_LRS_notebook/data.tar.gz'\n", - "\n", - "# Download and unpack data if needed\n", - "if not os.path.exists(\"data.tar.gz\"):\n", - " print(\"Downloading Data\")\n", - " urllib.request.urlretrieve(data_tar_url, 'data.tar.gz')\n", - "if not os.path.exists(\"data/\"):\n", - " print(\"Unpacking Data\")\n", - " with tarfile.open('./data.tar.gz', \"r:gz\") as tar:\n", - " tar.extractall(filter='data')" - ] - }, - { - "cell_type": "markdown", - "id": "611086f4", - "metadata": {}, - "source": [ - "## 1. The Level 3 Data Products \n", - "\n", - "\n", - "Let's start by plotting the main default Level 3 output products:\n", - "* the ``s2d`` file: this is the 2D image built from the co-added resampled individual nod exposures. \n", - "* the ``x1d`` file: this is the 1-D extracted spectrum, extracted from the Level 3 ``s2d`` file. \n", - "\n", - "The ``s2d`` image shows a bright central trace, flanked by two negative traces. These result from the combination of the nod exposures, each of which also contains a positive and negative trace due to being mutually subtracted for background subtraction. \n", - "\n", - "We restrict the short-wavelength end of the x-axis to 5 micron, as our calibration is very poor below this wavelength. The Level 3 spectrum is extracted from the resampled, dither-combined, calibrated exposure. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a8012bfa", - "metadata": {}, - "outputs": [], - "source": [ - "l3_s2d_file = 'data/jw02072-o001_t010_miri_p750l_s2d_1089.fits'\n", - "l3_s2d = datamodels.open(l3_s2d_file)\n", - "fig, ax = plt.subplots(figsize=[2, 8])\n", - "im2d = ax.imshow(l3_s2d.data, origin='lower', aspect='auto', cmap='gist_gray')\n", - "ax.set_xlabel('column')\n", - "ax.set_ylabel('row')\n", - "ax.set_title('SN2021aefx - Level 3 resampled 2D spectral image')\n", - "fig.colorbar(im2d)\n", - "fig.show()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c51f421b", - "metadata": {}, - "outputs": [], - "source": [ - "l3_file = 'data/jw02072-o001_t010_miri_p750l_x1d_1089.fits'\n", - "l3_spec = datamodels.open(l3_file)\n", - "\n", - "fig2, ax2 = plt.subplots(figsize=[12, 4])\n", - "ax2.plot(l3_spec.spec[0].spec_table['WAVELENGTH'], l3_spec.spec[0].spec_table['FLUX'])\n", - "ax2.set_xlabel('wavelength (um)')\n", - "ax2.set_ylabel('flux (Jy)')\n", - "ax2.set_title('SN2021aefx - Level 3 spectrum in MAST (pmap 1089)')\n", - "ax2.set_xlim(5., 14.)\n", - "fig2.show()" - ] - }, - { - "cell_type": "markdown", - "id": "3ae110b4", - "metadata": {}, - "source": [ - "## The spectral extraction reference file \n", - "\n", - "The reference file that tells the ``extract_1d`` algorithm what parameters to use is a text file using the `json` format that is available in [CRDS](https://jwst-crds.stsci.edu). The second reference file used in the extraction is the aperture correction; this corrects for the flux lost as a function of wavelength for the extraction aperture size used. You can use the datamodel attributes of the ``x1d`` file to check which extraction reference file was called by the algorithm. \n", - "\n", - "We show below how to examine the file programmatically to see what aperture was used to produce the default Level 3 spectrum shown above. **Note: this json file can easily be opened and edited with a simple text editor**. \n", - "\n", - "Full documentation of the ``extract_1d`` reference file is available [here](https://jwst-pipeline.readthedocs.io/en/latest/jwst/extract_1d/reference_files.html). We recommend you read this page and any links therein carefully to understand how the parameters in the file are applied to the data. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f0574ee0-84a0-4fa8-ae54-d7b6ca34a7a7", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "print(f'Spectral extraction reference file used: {l3_spec.meta.ref_file.extract1d.name}')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "95f20a0a-0f24-4d4b-8480-2c37574ad6e8", - "metadata": { - "scrolled": true, - "tags": [ - "scroll-output" - ] - }, - "outputs": [], - "source": [ - "file_path = 'data/jw02072-o001_t010_miri_p750l_x1d_1089.fits'\n", - "with fits.open(file_path) as hdul:\n", - " header = hdul[0].header\n", - " json_ref_default = crds.getreferences(header)['extract1d']\n", - "\n", - " with open(json_ref_default) as json_ref:\n", - " x1dref_default = json.load(json_ref)\n", - " print('Settings for SLIT data: {}'.format(x1dref_default['apertures'][0]))\n", - " print(' ')\n", - " print('Settings for SLITLESS data: {}'.format(x1dref_default['apertures'][1]))" - ] - }, - { - "cell_type": "markdown", - "id": "ecc11d6b", - "metadata": {}, - "source": [ - "Let's look at what's in this file. \n", - "\n", - "* **id**: identification label, in this case specifying the exposure type the parameters will be applied to.\n", - "* **region_type**: optional, if included must be set to 'target'\n", - "* **disp_axis**: defines the direction of dispersion (1 for x-axis, 2 for y-axis). **For MIRI LRS this should always be set to 2**. \n", - "* **xstart** (int): first pixel in the horizontal direction (x-axis; 0-indexed) \n", - "* **xstop** (int): last pixel in the horizontal direction (x-axis; 0-indexed; limit is **inclusive**)\n", - "* **bkg_order**: \n", - "* **use_source_posn** (True/False): if True, this will use the target coordinates to locate the target in the field, and offset the extraction aperture to this location. **We recommend this is set to False**. \n", - "* **bkg_order**: the polynomial order to be used for background fitting. if the accompanying parameter **bkg_coeff** is not provided, no background fitting will be performed. **For MIRI LRS slit data, default background subtraction is achieved in the Spec2Pipeline, by mutually subtracting nod expsosures**.\n", - "\n", - "As for MIRI LRS the dispersion is in the vertical direction (i.e. `disp_axis` = 2), the extraction aperture width is specified with the coordinates `xstart` and `xstop`. If no coordinates `ystart` and `ystop` are provided, the spectrum will be extracted over the full height of the ``s2d`` cutout region. We can illustrate the default extraction parameters on the Level 3 ``s2d`` file. \n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "703f59cd", - "metadata": {}, - "outputs": [], - "source": [ - "xstart = x1dref_default['apertures'][0]['xstart']\n", - "xstop = x1dref_default['apertures'][0]['xstop']\n", - "ap_height = np.shape(l3_s2d.data)[0]\n", - "ap_width = xstop - xstart + 1\n", - "x1d_rect = Rectangle(xy=(xstart, 0), width=ap_width, height=ap_height, angle=0., edgecolor='red',\n", - " facecolor='None', ls='-', lw=1.5)\n", - "\n", - "fig, ax = plt.subplots(figsize=[2, 8])\n", - "im2d = ax.imshow(l3_s2d.data, origin='lower', aspect='auto', cmap='gist_gray')\n", - "ax.add_patch(x1d_rect)\n", - "ax.set_xlabel('column')\n", - "ax.set_ylabel('row')\n", - "ax.set_title('SN2021aefx - Level 3 resampled 2D spectral image')\n", - "fig.colorbar(im2d)\n", - "fig.show()" - ] - }, - { - "cell_type": "markdown", - "id": "1fd784d9", - "metadata": {}, - "source": [ - "## Example 1: Changing the extraction width \n", - "\n", - "In this example, we demonstrate how to change the extraction width from the default. Instead of 8 pixels, we'll extract 12, keeping the aperture centred on the trace. \n", - "\n", - "We will modify the values in the json files in python in this notebook, but the file can also simply be edited in a text editor. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "06dc8eb5", - "metadata": {}, - "outputs": [], - "source": [ - "xstart2 = xstart - 2\n", - "xstop2 = xstop + 2\n", - "print('New xstart, xstop values = {0},{1}'.format(xstart2, xstop2))\n", - "\n", - "with open(json_ref_default) as json_ref:\n", - " x1dref_default = json.load(json_ref)\n", - " x1dref_ex1 = x1dref_default.copy()\n", - " x1dref_ex1['apertures'][0]['xstart'] = xstart2\n", - " x1dref_ex1['apertures'][0]['xstop'] = xstop2\n", - "\n", - "with open('x1d_reffile_example1.json', 'w') as jsrefout:\n", - " json.dump(x1dref_ex1, jsrefout, indent=4)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "5bc85413", - "metadata": {}, - "outputs": [], - "source": [ - "ap_width2 = xstop2 - xstart2 + 1\n", - "x1d_rect1 = Rectangle(xy=(xstart, 0), width=ap_width, height=ap_height, angle=0., edgecolor='red',\n", - " facecolor='None', ls='-', lw=1, label='8-px aperture (default)')\n", - "\n", - "x1d_rect2 = Rectangle(xy=(xstart2, 0), width=ap_width2, height=ap_height, angle=0., edgecolor='cyan',\n", - " facecolor='None', ls='-', lw=1, label='12-px aperture')\n", - "\n", - "fig4, ax4 = plt.subplots(figsize=[2, 8])\n", - "im2d = ax4.imshow(l3_s2d.data, origin='lower', aspect='auto', cmap='gist_gray')\n", - "# ax4.add_collection(aps_collection)\n", - "ax4.add_patch(x1d_rect1)\n", - "ax4.add_patch(x1d_rect2)\n", - "\n", - "ax4.set_xlabel('column')\n", - "ax4.set_ylabel('row')\n", - "ax4.set_title('Example 1: Default vs modified extraction aperture')\n", - "ax4.legend(loc=3)\n", - "fig.colorbar(im2d)\n", - "fig.show()" - ] - }, - { - "cell_type": "markdown", - "id": "87efcb9f", - "metadata": {}, - "source": [ - "Next we run the spectral extraction step, using this modified reference file. Note: when a step is run individually the file name suffix is different from when we run the Spec3Pipeline in its entirety. The extracted spectrum will now have ``extract1dstep.fits`` in the filename. The custom parameters we pass to the step call:\n", - "\n", - "* ``output_file``: we provide a custom output filename for this example (including an output filename renders the ``save_results`` parameter obsolete)\n", - "* ``override_extract1d``: here we provide the name of the custom reference file we created above\n", - "\n", - "We will plot the output against the default extracted product. We expect the spectra to be almost identical; differences can be apparent at the longer wavelengths as our path loss correction is less well calibrated in this low SNR region. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7304f758", - "metadata": { - "scrolled": true, - "tags": [ - "scroll-output" - ] - }, - "outputs": [], - "source": [ - "sp3_ex1 = Extract1dStep.call(l3_s2d, output_dir='data/', \n", - " output_file='lrs_slit_extract_example1', override_extract1d='x1d_reffile_example1.json')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "91199fd1", - "metadata": {}, - "outputs": [], - "source": [ - "print(sp3_ex1)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "91ebfc64", - "metadata": {}, - "outputs": [], - "source": [ - "fig5, ax5 = plt.subplots(figsize=[12, 4])\n", - "ax5.plot(l3_spec.spec[0].spec_table['WAVELENGTH'], l3_spec.spec[0].spec_table['FLUX'], label='8-px aperture')\n", - "ax5.plot(sp3_ex1.spec[0].spec_table['WAVELENGTH'], sp3_ex1.spec[0].spec_table['FLUX'], label='12-px aperture')\n", - "ax5.set_xlabel('wavelength (um)')\n", - "ax5.set_ylabel('flux (Jy)')\n", - "ax5.set_title('Example 1: Difference aperture sizes')\n", - "ax5.set_xlim(5., 14.)\n", - "ax5.legend()\n", - "fig5.show()" - ] - }, - { - "cell_type": "markdown", - "id": "f28a4f8e", - "metadata": {}, - "source": [ - "## Example 2: Changing aperture location\n", - "\n", - "In this example we will demonstrate spectral extraction at a different location in the slit. A good use case for this is to extract a spectrum from one of the nodded exposures, prior to combination of the nods in the Spec3Pipeline. We will take the ``s2d`` output from the Spec2Pipeline, and extract the spectrum. In the nod 1 exposure we see the spectrum peak is located in column 13 (0-indexed), and we extract a default 8-px fixed-width aperture. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "8a8cf793", - "metadata": {}, - "outputs": [], - "source": [ - "l2_s2d_file = 'data/jw02072001001_06101_00001_mirimage_s2d.fits'\n", - "l2_s2d = datamodels.open(l2_s2d_file)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "55c81453", - "metadata": {}, - "outputs": [], - "source": [ - "xstart3 = 9\n", - "xstop3 = 17\n", - "\n", - "with open(json_ref_default) as json_ref:\n", - " x1dref_default = json.load(json_ref)\n", - " x1dref_ex2 = x1dref_default.copy()\n", - " x1dref_ex2['apertures'][0]['xstart'] = xstart3\n", - " x1dref_ex2['apertures'][0]['xstop'] = xstop3\n", - "\n", - "with open('x1d_reffile_example2.json', 'w') as jsrefout:\n", - " json.dump(x1dref_ex2, jsrefout, indent=4)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "fe340506", - "metadata": {}, - "outputs": [], - "source": [ - "ap_width3 = xstop3 - xstart3 + 1\n", - "x1d_rect3 = Rectangle(xy=(xstart3, 0), width=ap_width3, height=ap_height, angle=0., edgecolor='red',\n", - " facecolor='None', ls='-', lw=1, label='8-px aperture at nod 1 location')\n", - "\n", - "fig6, ax6 = plt.subplots(figsize=[2, 8])\n", - "im2d = ax6.imshow(l2_s2d.data, origin='lower', aspect='auto', cmap='gist_gray')\n", - "ax6.add_patch(x1d_rect3)\n", - "ax6.set_xlabel('column')\n", - "ax6.set_ylabel('row')\n", - "ax6.set_title('Example 2: Different aperture location')\n", - "ax6.legend(loc=3)\n", - "fig6.colorbar(im2d)\n", - "fig6.show()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "3b0b287b", - "metadata": { - "scrolled": true, - "tags": [ - "scroll-output" - ] - }, - "outputs": [], - "source": [ - "sp2_ex2 = Extract1dStep.call(l2_s2d_file, output_dir='data/', output_file='lrs_slit_extract_example2',\n", - " override_extract1d='x1d_reffile_example2.json')" - ] - }, - { - "cell_type": "markdown", - "id": "32eaa24e", - "metadata": {}, - "source": [ - "Let's again plot the output against the default extracted product. We expect this 1-nod spectrum to be noisier but not significantly different from the combined product. The spectrum may have more bad pixels that manifest as spikes or dips in the spectrum. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "ce8eccfb", - "metadata": {}, - "outputs": [], - "source": [ - "fig7, ax7 = plt.subplots(figsize=[12, 4])\n", - "ax7.plot(l3_spec.spec[0].spec_table['WAVELENGTH'], l3_spec.spec[0].spec_table['FLUX'], label='default location (nods combined)')\n", - "ax7.plot(sp2_ex2.spec[0].spec_table['WAVELENGTH'], sp2_ex2.spec[0].spec_table['FLUX'], label='nod 1 location (single nod)')\n", - "ax7.set_xlabel('wavelength (um)')\n", - "ax7.set_ylabel('flux (Jy)')\n", - "ax7.set_title('Example 2: Different aperture locations')\n", - "ax7.set_xlim(5., 14.)\n", - "ax7.legend()\n", - "fig7.show()" - ] - }, - { - "cell_type": "markdown", - "id": "39492ab3", - "metadata": {}, - "source": [ - "## Example 3: Extraction with background subtraction\n", - "\n", - "For LRS slit observations, the default background subtraction strategy is performed in the ``background`` step in the Spec2Pipeline; the 2 nodded exposures are mutually subtracted, resulting in each returning a 2D spectral image with a positive and a negative trace, and the background subtracted. \n", - "\n", - "For non-standard cases or slitless LRS data it is however possible to subtract a background as part of the spectral extraction in ``extract_1d``. In the ``extract_1d`` reference file we can pass specific parameters for the background:\n", - "* bkg_coeff (list or list of floats): the regions to be used as background. **This is the main parameter required for background subtraction**\n", - "* bkg_fit (string): the type or method of the background computation. (e.g. None, 'poly', 'mean' or 'median')\n", - "* bkg_order (int): the order of polynomial to fit to background regions. if bkg_fit is not set to 'poly', this parameter will be ignored. \n", - "* smoothing_length (odd int; optional): the width of the boxcar filter that will be used to smooth the background signal in the dispersion direction. This can provide a better quality in case of noisy data. \n", - "\n", - "The 'poly' option for the ``bkg_fit`` parameter will take the value of all pixels in the background region on a given row, and fit a polynomial of order ``bkg_order`` to them. This option can be useful in cases where a gradient is present in the background. \n", - "\n", - "The data we're using here already has the background subtracted so we expect the impact of this to be minimal, but we provide a demonstration using the nod 1, level 2b spectral image. In this example we will calculate the background from 2 4-column windows, setting the ``bkg_fit`` to 'median'. \n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b4ea99ea", - "metadata": {}, - "outputs": [], - "source": [ - "rows = [140, 200, 325]\n", - "fig8, ax8 = plt.subplots(figsize=[8, 4])\n", - "ncols = np.shape(l2_s2d.data)[1]\n", - "pltx = np.arange(ncols)\n", - "for rr in rows:\n", - " label = 'row {}'.format(rr)\n", - " ax8.plot(pltx, l2_s2d.data[rr, :], label=label)\n", - "ax8.axvline(x=1, ymin=0, ymax=1, ls='--', lw=1., color='coral', label='background regions')\n", - "ax8.axvline(x=5, ymin=0, ymax=1, ls='--', lw=1., color='coral')\n", - "ax8.axvline(x=39, ymin=0, ymax=1, ls='--', lw=1., color='coral')\n", - "ax8.axvline(x=43, ymin=0, ymax=1, ls='--', lw=1., color='coral')\n", - "ax8.legend()\n", - "fig8.show()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "1238d6c9", - "metadata": {}, - "outputs": [], - "source": [ - "with open(json_ref_default) as json_ref:\n", - " x1dref_default = json.load(json_ref)\n", - " x1dref_ex3 = x1dref_default.copy()\n", - " x1dref_ex3['apertures'][0]['xstart'] = xstart3\n", - " x1dref_ex3['apertures'][0]['xstop'] = xstop3\n", - " x1dref_ex3['apertures'][0]['bkg_coeff'] = [[0.5], [4.5], [38.5], [43.5]]\n", - " x1dref_ex3['apertures'][0]['bkg_fit'] = 'median'\n", - "\n", - "with open('x1d_reffile_example3.json', 'w') as jsrefout:\n", - " json.dump(x1dref_ex3, jsrefout, indent=4)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "ac0d2746", - "metadata": { - "scrolled": true, - "tags": [ - "scroll-output" - ] - }, - "outputs": [], - "source": [ - "sp2_ex3 = Extract1dStep.call(l2_s2d_file, output_dir='data/', output_file='lrs_slit_extract_example3',\n", - " override_extract1d='x1d_reffile_example3.json')" - ] - }, - { - "cell_type": "markdown", - "id": "45b69d9b", - "metadata": {}, - "source": [ - "When the ``extract_1d`` step performs a background subtraction, the background spectrum is part of the output product, so you can check what was subtracted. In the plot below we can see that, as expected, the background for this particular exposure is near-zero (apart from the noisy long-wavelength end), as the subtraction was already performed. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7210c9ac", - "metadata": {}, - "outputs": [], - "source": [ - "fig9, ax9 = plt.subplots(nrows=2, ncols=1, figsize=[12, 4])\n", - "# ax9.plot(l3_spec.spec[0].spec_table['WAVELENGTH'], l3_spec.spec[0].spec_table['FLUX'], label='default location (nods combined)')\n", - "ax9[0].plot(sp2_ex2.spec[0].spec_table['WAVELENGTH'], sp2_ex2.spec[0].spec_table['FLUX'], label='nod 1 spectrum - no bkg sub')\n", - "ax9[0].plot(sp2_ex3.spec[0].spec_table['WAVELENGTH'], sp2_ex3.spec[0].spec_table['FLUX'], label='nod 1 spectrum - with bkg sub')\n", - "ax9[1].plot(sp2_ex3.spec[0].spec_table['WAVELENGTH'], sp2_ex3.spec[0].spec_table['BACKGROUND'], label='background')\n", - "ax9[1].set_xlabel('wavelength (um)')\n", - "ax9[0].set_ylabel('flux (Jy)')\n", - "ax9[0].set_title('Example 3: Extraction with background subtraction')\n", - "ax9[0].set_xlim(5., 14.)\n", - "ax9[1].set_xlim(5., 14.)\n", - "ax9[0].legend()\n", - "ax9[1].legend()\n", - "fig9.show()" - ] - }, - { - "cell_type": "markdown", - "id": "e01786eb", - "metadata": {}, - "source": [ - "## Example 4: Tapered column extraction\n", - "\n", - "In this example we will use the JWST calibration pipeline to perform a spectral extraction in a tapered column aperture. The way to specify this in the extraction reference file is to use the ``src_soeff`` parameter instead of the simpler ``xstart``, ``xstop`` settings. The ``src_coeff`` parameter can take polynomial coefficients rather than fixed pixel values. In this example, we will define a tapered column aperture corresponding to 3 * the FWHM of the spatial profile. \n", - "\n", - "Polynomial definitions for the extraction aperture can be specified as a function of pixels or wavelength, which is defined in the ``independent_var`` parameter. \n", - "\n", - "We will use pre-measured FWHM values as a function of **wavelength** to fit a straight line to the FWHM($\\lambda$) profile, and set the extraction parameters according to this fit. The FWHM can of course also be measured directly from the data as well. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9e3c2433", - "metadata": {}, - "outputs": [], - "source": [ - "def calc_xap_fit():\n", - " # these are values measured from commissioning data. FWHM is in arcsec.\n", - " lam = [5.0, 7.5, 10.0, 12.0]\n", - " fwhm = [0.29, 0.3, 0.36, 0.42]\n", - "\n", - " # convert from arcsec to pixel using MIRI pixel scaling of 0.11 arcsec/px\n", - " fwhm_px = fwhm / (0.11*u.arcsec/u.pixel)\n", - "\n", - " # we want to extract 3 * fwhm, which means 1.5 * fwhm on either side of the trace\n", - " xap_pix = 1.5 * fwhm_px\n", - "\n", - " # now we want to fit a line to these points\n", - " line_init = models.Linear1D()\n", - " fit = fitting.LinearLSQFitter()\n", - "\n", - " fitted_line = fit(line_init, lam, xap_pix.value)\n", - " print(fitted_line)\n", - "\n", - " fig, ax = plt.subplots(figsize=[8, 4])\n", - " xplt = np.linspace(4.0, 14., num=50)\n", - " ax.plot(lam, xap_pix.value, 'rx', label='1.5 * FWHM(px)')\n", - " ax.plot(xplt, fitted_line(xplt), 'b-', label='best-fit line')\n", - " ax.set_xlabel('wavelength')\n", - " ax.set_ylabel('px')\n", - " ax.legend()\n", - "\n", - " return fitted_line" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "e21fcec5", - "metadata": {}, - "outputs": [], - "source": [ - "poly_pos = calc_xap_fit()\n", - "print(poly_pos.slope, poly_pos.intercept)" - ] - }, - { - "cell_type": "markdown", - "id": "f7dc8785", - "metadata": {}, - "source": [ - "The above polynomial defines the relationship between wavelength and the number of pixels to extract. To ensure that the extractio location is centred on the location of the spectrum, we add to the intercept value the central location of the trace, which is at column 30.5. \n", - "\n", - "In the next cell, we provide these parameters to the ``src_coeff`` parameter in the extraction reference file. **Note: The ``src_coeff`` parameter takes precedence over the ``xstart`` and ``xstop`` parameters if all 3 are present; for clarity we remove the latter from our reference file.**" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a9c9d403", - "metadata": {}, - "outputs": [], - "source": [ - "trace_cen = 30.5\n", - "\n", - "with open(json_ref_default) as json_ref:\n", - " x1dref_default = json.load(json_ref)\n", - " x1dref_ex4 = x1dref_default.copy()\n", - " x1dref_ex4['apertures'][0]['xstart'] = None\n", - " x1dref_ex4['apertures'][0]['xstop'] = None\n", - " x1dref_ex4['apertures'][0]['independent_var'] = 'wavelength'\n", - " x1dref_ex4['apertures'][0]['src_coeff'] = [[-1*poly_pos.intercept.value + trace_cen, -1*poly_pos.slope.value], [poly_pos.intercept.value + trace_cen, poly_pos.slope.value]]\n", - "\n", - "with open('x1d_reffile_example4.json', 'w') as jsrefout:\n", - " json.dump(x1dref_ex4, jsrefout, indent=4)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "71789e83", - "metadata": { - "scrolled": true, - "tags": [ - "scroll-output" - ] - }, - "outputs": [], - "source": [ - "sp3_ex4 = Extract1dStep.call(l3_s2d, output_dir='data/', \n", - " output_file='lrs_slit_extract_example4', override_extract1d='x1d_reffile_example4.json')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9d1bc74c", - "metadata": {}, - "outputs": [], - "source": [ - "fig10, ax10 = plt.subplots(figsize=[12, 4])\n", - "ax10.plot(l3_spec.spec[0].spec_table['WAVELENGTH'], l3_spec.spec[0].spec_table['FLUX'], label='default fixed-width aperture')\n", - "ax10.plot(sp3_ex4.spec[0].spec_table['WAVELENGTH'], sp3_ex4.spec[0].spec_table['FLUX'], label='tapered column aperture')\n", - "ax10.set_xlabel('wavelength (um)')\n", - "ax10.set_ylabel('flux (Jy)')\n", - "ax10.set_title('Example 4: Tapered column vs. fixed-width extraction aperture')\n", - "ax10.set_xlim(5., 14.)\n", - "ax10.legend()\n", - "fig10.show()" - ] - }, - { - "cell_type": "markdown", - "id": "378f8393", - "metadata": {}, - "source": [ - "The output spectrum also contains a reference to the number of pixels used for the extraction as a function of wavelength. Let's visualize that too. \n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "78ca0c68", - "metadata": {}, - "outputs": [], - "source": [ - "fig11, ax11 = plt.subplots(figsize=[12, 4])\n", - "ax11.plot(l3_spec.spec[0].spec_table['WAVELENGTH'], l3_spec.spec[0].spec_table['NPIXELS'], label='default fixed-width aperture')\n", - "ax11.plot(sp3_ex4.spec[0].spec_table['WAVELENGTH'], sp3_ex4.spec[0].spec_table['NPIXELS'], label='tapered column aperture')\n", - "ax11.set_xlabel('wavelength (um)')\n", - "ax11.set_ylabel('number of pixels')\n", - "ax11.set_title('Example 4: Number of pixels extracted')\n", - "ax11.set_xlim(5., 14.)\n", - "ax11.legend()\n", - "fig11.show()" - ] - }, - { - "cell_type": "markdown", - "id": "f4501ee7", - "metadata": {}, - "source": [ - "## Summary\n", - "\n", - "We hope this notebook was useful in helping you understand the capabilities of the JWST calibration for spectral extraction. The above examples are not an exhaustive list of all the possibilities: different methods of source and background extraction can be combined for more complex extraction operations. \n", - "\n", - "**If you have any questions, comments, or requests for further demos of these capabilities, please contact the [JWST Helpdesk](http://jwsthelp.stsci.edu/).**" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.17" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/notebooks/MIRI/MIRI_LRS_spectral_extraction/miri_lrs_slit_advanced_extraction.ipynb b/notebooks/MIRI/MIRI_LRS_spectral_extraction/miri_lrs_slit_advanced_extraction.ipynb new file mode 100644 index 000000000..4c7e3d413 --- /dev/null +++ b/notebooks/MIRI/MIRI_LRS_spectral_extraction/miri_lrs_slit_advanced_extraction.ipynb @@ -0,0 +1,1493 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "20ce4694", + "metadata": {}, + "source": [ + "# MIRI LRS Slit Spectroscopy: Spectral Extraction using the JWST Pipeline\n", + "\n", + "**Authors:** Sarah Kendrew, Ian Wong; MIRI branch
\n", + "**Last Updated:** December 17, 2025
\n", + "**Pipeline Version:** 1.20.2 (Build 12.1)\n", + "\n", + "**Purpose:**
\n", + "The aim of this notebook is to demonstrate the custom spectral extraction capabilities of the JWST calibration pipeline. The JWST calibration pipeline performs spectral extraction for all spectroscopic data using a set of basic default assumptions that are tuned to produce accurately calibrated spectra for the majority of science cases. In the case of MIRI LRS data obtained in the slit, the default method is a simple 8-pixel-wide boxcar extraction, where the spectrum is summed along the cross-dispersion axis over the valid wavelength range. An aperture correction is applied at each wavelength element along the spectrum to account for flux lost from the finite-width aperture. \n", + "\n", + "The ``extract_1d`` step uses the following inputs for its algorithm:\n", + "- the spectral extraction reference file: this is a json-formatted file, available as a reference file from the [JWST Calibration Reference Data System (CRDS)](https://jwst-crds.stsci.edu)\n", + "- the bounding box: the ``assign_wcs`` step attaches a bounding box definition to the data, which defines the region over which a valid calibration is available.\n", + "\n", + "Beyond simple boxcar extraction, the ``extract_1d`` step has the capability to perform more complex spectral extractions, which require the user to manually edit some of the parameters contained in the spectral extraction reference file and rerun the pipeline step. Of particular note is the ability to perform optimal spectral extraction, which can be activated using the newly added ``extraction_type`` parameter.\n", + "\n", + "**Data:**
\n", + "The data used in this notebook is an observation of the Type Ia supernova SN2021aefx, observed by Jha et al. in Program #2072 (Observation 1). These data were taken with zero exclusive access period and published in [Kwok et al 2023](https://ui.adsabs.harvard.edu/abs/2023ApJ...944L...3K/abstract). The relevant data files are located in the compressed ``data.zip`` file and will be automatically unpacked and placed in a subdirectory called ``data/``. The user can also manually specify a particular set of input files of their choice below.\n", + "\n", + "We will demonstrate most of the spectral extraction methods on resampled, calibrated spectral images (``s2d`` files). The basic demo and many examples run on Level 3 data, in which the pair of nodded exposures has been combined into a single spectral image. Several of the other examples use a Level 2b data file, corresponding to just the first of the two nodded exposures. For optimal spectral extraction (Example 5), the pipeline requires unrectified, calibrated spectral images (``cal`` files).
\n", + "\n", + "\n", + "**JWST pipeline version and CRDS context:**
\n", + "This notebook was written for the above-specified pipeline version and associated build context for this version of the JWST Calibration Pipeline. Information about this and other contexts can be found in the JWST CRDS [server](https://jwst-crds.stsci.edu/). If you use different pipeline versions, please refer to the table [here](https://jwst-crds.stsci.edu/display_build_contexts/) to determine what context to use. To learn more about the differences for the pipeline, read the relevant [documentation](https://jwst-docs.stsci.edu/jwst-science-calibration-pipeline/jwst-operations-pipeline-build-information#references).\n", + "\n", + "Please note that pipeline software development is a continuous process, so results in some cases may be slightly different if a subsequent version is used. **For optimal results, users are strongly encouraged to reprocess their data using the most recent pipeline version and [associated CRDS context](https://jwst-crds.stsci.edu/display_build_contexts/), taking advantage of bug fixes and algorithm improvements.** Any [known issues](https://jwst-docs.stsci.edu/known-issues-with-jwst-data#gsc.tab=0) for this build are noted in the notebook.\n", + "\n", + "\n", + "**Documentation:**
\n", + "This notebook is part of a STScI's larger [post-pipeline Data Analysis Tools Ecosystem](https://jwst-docs.stsci.edu/jwst-post-pipeline-data-analysis) and can be [downloaded](https://github.com/spacetelescope/dat_pyinthesky/tree/main/jdat_notebooks/MRS_Mstar_analysis) directly from the [JDAT Notebook Github directory](https://github.com/spacetelescope/jdat_notebooks).\n", + "\n", + "**Recent updates:**
\n", + "July 2023: First public version of notebook created
\n", + "December 17, 2025: Formatting cleanup and updates. Added optimal spectral extraction and updated background subtraction examples." + ] + }, + { + "cell_type": "markdown", + "id": "2c0d47a4-98c2-40bc-820e-5a8dd468fc00", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "id": "00de505e-97ad-46d2-81ec-16dfc87713e8", + "metadata": {}, + "source": [ + "## Table of Contents\n", + "\n", + "0. [Package imports](#Package-imports)\n", + "1. [The Level 2 & 3 data products](#The-Level-2-&-3-data-products)\n", + "2. [The spectral extraction reference file](#The-spectral-extraction-reference-file)\n", + "3. [Example 1: Changing the extraction aperture width](#Example-1:-Changing-the-extraction-aperture-width)\n", + "4. [Example 2: Changing the extraction aperture location](#Example-2:-Changing-the-extraction-aperture-location)\n", + "5. [Example 3: Extraction with background subtraction](#Example-3:-Extraction-with-background-subtraction)\n", + "6. [Example 4: Tapered column extraction](#Example-4:-Tapered-column-extraction)\n", + "7. [Example 5: Optimal spectral extraction](#Example-5:-Optimal-spectral-extraction)" + ] + }, + { + "cell_type": "markdown", + "id": "da297e1c-ffed-4f74-9f39-282a0e363d8b", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "id": "14ff9543", + "metadata": {}, + "source": [ + "## Package imports" + ] + }, + { + "cell_type": "markdown", + "id": "e698ce3a-fdaf-4d3b-9109-b980794e94aa", + "metadata": {}, + "source": [ + "It is assumed that the user has installed a number of relevant dependencies, which are listed below. If this is not the case, the missing dependencies can be installed using ``pip``. We recommend that the user create a dedicated conda environment for running the pipeline and installing dependencies.\n", + "\n", + "- `astropy.io.fits` for accessing FITS files\n", + "- `os` for managing system paths\n", + "- `matplotlib` for plotting data\n", + "- `urllib` for downloading data\n", + "- `zipfile` for unpacking data\n", + "- `numpy` for basic array manipulation\n", + "- `jwst` for running JWST pipeline and handling data products\n", + "- `json` for working with json files\n", + "- `crds` for working with JWST reference files" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3a130dd2", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "\n", + "# ------------------------Set CRDS context and paths----------------------\n", + "# Each version of the calibration pipeline is associated with a specific CRDS\n", + "# context file. The pipeline will select the appropriate context file behind\n", + "# the scenes while running. However, if you wish to override the default context\n", + "# file and run the pipeline with a different context, you can set that using\n", + "# the CRDS_CONTEXT environment variable. Here we show how this is done,\n", + "# although we leave the line commented out in order to use the default context.\n", + "\n", + "# If you wish to specify a different context, uncomment the line below.\n", + "#os.environ['CRDS_CONTEXT'] = 'jwst_1322.pmap' # CRDS context for 1.17.1\n", + "\n", + "# Check whether the local CRDS cache directory has been set.\n", + "# If not, set it to the user home directory.\n", + "if (os.getenv('CRDS_PATH') is None):\n", + " os.environ['CRDS_PATH'] = os.path.join(os.path.expanduser('~'), 'crds_cache')\n", + " \n", + "# Check whether the CRDS server URL has been set. If not, set it.\n", + "if (os.getenv('CRDS_SERVER_URL') is None):\n", + " os.environ['CRDS_SERVER_URL'] = 'https://jwst-crds.stsci.edu'\n", + "\n", + "# Print out CRDS path and context that will be used\n", + "print('CRDS local filepath:', os.environ['CRDS_PATH'])\n", + "print('CRDS file server:', os.environ['CRDS_SERVER_URL'])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "08ddf5f7", + "metadata": {}, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "import urllib\n", + "import zipfile\n", + "\n", + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "from matplotlib.patches import Rectangle\n", + "import matplotlib.colors as colors\n", + "\n", + "import astropy.io.fits as fits\n", + "import astropy.units as u\n", + "from astropy.modeling import models, fitting\n", + "\n", + "# JWST calibration pipeline and extract_1d step\n", + "import jwst\n", + "from jwst import datamodels\n", + "from jwst.extract_1d import Extract1dStep\n", + "\n", + "import json\n", + "import crds\n", + "\n", + "print(f'Using JWST calibration pipeline version {jwst.__version__}')" + ] + }, + { + "cell_type": "markdown", + "id": "7685cdbc-6b0d-4621-ba71-760d1ce19978", + "metadata": {}, + "source": [ + "Now we download the data that will be used by this notebook." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "305103d5", + "metadata": {}, + "outputs": [], + "source": [ + "data_zip_file = 'data.zip'\n", + "\n", + "# Unpack data, if needed\n", + "if not os.path.exists('data/'):\n", + " print('Unpacking Data')\n", + " with zipfile.ZipFile(data_zip_file, 'r') as zip_ref:\n", + " zip_ref.extractall()" + ] + }, + { + "cell_type": "markdown", + "id": "e18dfef5-1592-4675-aa5c-54ab858e0da1", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "id": "611086f4", + "metadata": {}, + "source": [ + "## The Level 2 & 3 data products\n", + "\n", + "The ``data/`` subdirectory contains both Level 2 and Level 3 data products. There are two relevant Level 3 data files:\n", + "* the ``s2d`` file: this is the flux-calibrated 2D image built from the co-added resampled individual nod exposures. \n", + "* the ``x1d`` file: this is the 1-D extracted spectrum, which is extracted from the Level 3 ``s2d`` file by the calibration pipeline using default settings.\n", + "\n", + "The Level 3 ``s2d`` image shows a bright central trace, flanked by two negative traces. These result from the combination of the nod exposures, each of which contains a positive and negative trace due to the mutual subtraction of the two nodded exposures. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a8332ad6-5f5a-4d03-bd7b-7120b8c0d6b5", + "metadata": {}, + "outputs": [], + "source": [ + "l3_s2d_file = 'data/jw02072-o001_t001_miri_p750l_s2d.fits'\n", + "l3_x1d_file = 'data/jw02072-o001_t001_miri_p750l_x1d.fits'" + ] + }, + { + "cell_type": "markdown", + "id": "234e2ca1-8663-41d2-90f8-96a4c5dc813e", + "metadata": {}, + "source": [ + "We now plot the Level 3 spectral image and extracted spectrum. The flux unit in the spectral image is MJy/steradian. When plotting the spectrum, the short-wavelength end of the x-axis is restricted to 5 microns, as the flux calibration is very poor below this wavelength. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a8012bfa", + "metadata": {}, + "outputs": [], + "source": [ + "l3_s2d = datamodels.open(l3_s2d_file)\n", + "\n", + "fig, ax = plt.subplots(figsize=[2, 8])\n", + "im2d = ax.imshow(l3_s2d.data, origin='lower', aspect='auto', cmap='gist_gray')\n", + "ax.set_xlabel('column')\n", + "ax.set_ylabel('row')\n", + "ax.set_title('SN2021aefx - Level 3 resampled 2D spectral image')\n", + "fig.colorbar(im2d)\n", + "fig.show()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c51f421b", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "l3_spec = datamodels.open(l3_x1d_file)\n", + "\n", + "fig2, ax2 = plt.subplots(figsize=[12, 4])\n", + "ax2.plot(l3_spec.spec[0].spec_table['WAVELENGTH'], l3_spec.spec[0].spec_table['FLUX'])\n", + "ax2.set_xlabel('wavelength (μm)')\n", + "ax2.set_ylabel('flux (Jy)')\n", + "ax2.set_title('SN2021aefx - Level 3 spectrum from default extraction')\n", + "ax2.set_xlim(5., 14.)\n", + "fig2.show()" + ] + }, + { + "cell_type": "markdown", + "id": "c8eb6433-9f64-4cac-841c-e4d1da9f119c", + "metadata": {}, + "source": [ + "The data subdirectory also contains two different versions of the Level 2 ``s2d`` file for the first nodded exposure. Both of them are resampled and rectified spectral images produced by the second stage of the calibration pipeline. The default file ``l2_s2d_file`` follows the standard pipeline procedure of pairwise nod subtraction and shows a single pair of positive and negative traces. The ``l2_s2d_file_nobsub`` file was custom generated with the background subtraction step skipped and therefore only contains a positive trace. For a general overview of custom pipeline processing, refer to the [MIRI LRS Slit Mode Pipeline Notebook](https://github.com/spacetelescope/jwst-pipeline-notebooks/blob/main/notebooks/MIRI/LRS-slit/JWPipeNB-MIRI-LRS-slit.ipynb). The choice to skip nod subtraction may be motivated by the presence of additional sources within the slit or an extended target. The ``extract_1d`` step provides flexible options for customized background subtraction for instances where simple mutual subtraction of the nodded exposures is not appropriate." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "663e3cf4-d0d5-4d53-bd2e-d754b33a4cad", + "metadata": {}, + "outputs": [], + "source": [ + "l2_s2d_file = 'data/jw02072001001_06101_00001_mirimage_s2d.fits'\n", + "l2_s2d_file_nobsub = 'data/jw02072001001_06101_00001_mirimage_s2d_nobsub.fits'" + ] + }, + { + "cell_type": "markdown", + "id": "8c01f3f7-6b6c-413b-909c-fdef04d64e6c", + "metadata": {}, + "source": [ + "We now plot the Level 2 spectral image generated without pairwise nod subtraction. The background level rises dramatically toward the bottom of the spectral image (i.e., longer wavelengths) and shows a nonuniform distribution along the horizontal direction." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3632c827-e3e9-485a-806c-8c6d2d4eb7ca", + "metadata": {}, + "outputs": [], + "source": [ + "l2_s2d_nobsub = datamodels.open(l2_s2d_file_nobsub)\n", + "\n", + "fig, ax = plt.subplots(figsize=[2, 8])\n", + "im2d = ax.imshow(l2_s2d_nobsub.data, origin='lower', aspect='auto', cmap='gist_gray')\n", + "ax.set_xlabel('column')\n", + "ax.set_ylabel('row')\n", + "ax.set_title('SN2021aefx - Level 2 2D spectral image (no bkg sub)')\n", + "fig.colorbar(im2d)\n", + "fig.show()" + ] + }, + { + "cell_type": "markdown", + "id": "0e8749f0-ccb8-4ae4-8817-91ff4a24b94a", + "metadata": {}, + "source": [ + "Lastly, there are a pair of Level 2 ``cal`` files, which are the respective unrectified spectral images that correspond to the Level 2 ``s2d`` files described above. These images contain the full detector readout and will be used for the optimal spectral extraction example (Example 5)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "815827a8-9f48-430c-aa55-557a70c6e70c", + "metadata": {}, + "outputs": [], + "source": [ + "l2_cal_file = 'data/jw02072001001_06101_00001_mirimage_cal.fits'\n", + "l2_cal_file_nobsub = 'data/jw02072001001_06101_00001_mirimage_cal_nobsub.fits'" + ] + }, + { + "cell_type": "markdown", + "id": "0fb0ef23-bcca-47c0-b7e4-f84d74125b70", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "id": "3ae110b4", + "metadata": {}, + "source": [ + "## The spectral extraction reference file\n", + "\n", + "The reference file that handles the parameter settings read in by the ``extract_1d`` step is a plain-text file in the `json` format that is available in [CRDS](https://jwst-crds.stsci.edu). The second reference file used in the extraction is the aperture correction; this file contains vectors that correct for the flux lost as a function of wavelength based on the specific extraction aperture size used. You can use the datamodel attributes of the ``x1d`` file to check which extraction reference file was called by the algorithm. \n", + "\n", + "We demonstrate below how to examine the file programmatically to see what aperture size was used to produce the default Level 3 spectrum shown above. **Note: this json file can easily be opened and edited with a simple text editor**. \n", + "\n", + "A full description of the ``extract_1d`` reference file and the parameter definitions is available [here](https://jwst-pipeline.readthedocs.io/en/latest/jwst/extract_1d/reference_files.html)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f0574ee0-84a0-4fa8-ae54-d7b6ca34a7a7", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "print(f'Spectral extraction reference file used: {l3_spec.meta.ref_file.extract1d.name}')" + ] + }, + { + "cell_type": "markdown", + "id": "f205c00a-cc4b-4eca-9427-6594f7db4e22", + "metadata": {}, + "source": [ + "The same ``extract_1d`` reference file is used by the pipeline for both LRS slit and slitless observations, and each mode has a separate set of default parameters. We can examine those values as follows." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "95f20a0a-0f24-4d4b-8480-2c37574ad6e8", + "metadata": { + "scrolled": true, + "tags": [ + "scroll-output" + ] + }, + "outputs": [], + "source": [ + "file_path = l3_x1d_file\n", + "with fits.open(file_path) as hdul:\n", + " header = hdul[0].header\n", + " json_ref_default = crds.getreferences(header)['extract1d']\n", + "\n", + " with open(json_ref_default) as json_ref:\n", + " x1dref_default = json.load(json_ref)\n", + " print('Settings for SLIT data: {}'.format(x1dref_default['apertures'][0]))\n", + " print(' ')\n", + " print('Settings for SLITLESS data: {}'.format(x1dref_default['apertures'][1]))" + ] + }, + { + "cell_type": "markdown", + "id": "ecc11d6b", + "metadata": {}, + "source": [ + "Here is a brief breakdown of the parameters that can be included in a MIRI LRS reference file:\n", + "\n", + "* ``id``: identification label that specifies the exposure type (slit vs. slitless) the parameters will be applied to.\n", + "* ``region_type``: optional; if included, it must be set to 'target'.\n", + "* ``disp_axis``: defines the direction of the dispersion (1 for x-axis, 2 for y-axis). **For MIRI LRS this should always be set to 2.**\n", + "* ``xstart`` (int or float): first pixel of the extraction aperture in the horizontal direction (0-indexed).\n", + "* ``xstop`` (int or float): last pixel of the extraction aperture in the horizontal direction (0-indexed; the limit is **inclusive**).\n", + "* ``ystart`` (int or float): first pixel of the extraction aperture in the vertical direction (0-indexed).\n", + "* ``ystop`` (int or float): last pixel of the extraction aperture in the vertical direction (0-indexed; the limit is **inclusive**).\n", + "* ``extract_width`` (int or float): width of extraction aperture in the cross-dispersion direction. This parameter takes precedence over the ``xstart`` and ``xstop`` settings. If both are given, the ``xstart`` and ``xstop`` positions are used to determine the location of the spectral trace, but the spectral extraction utilizes an aperture of width equal to ``extract_width``.\n", + "* ``bkg_fit``: the type of background fit or computation used; can be 'median', 'mean', or 'poly' (polynomial fitting).\n", + "* ``bkg_order``: if ``bkg_fit`` is set to 'poly', this is the polynomial order to be used for background fitting; if the accompanying parameter ``bkg_coeff`` is not provided, no background fitting will be performed. **For MIRI LRS slit data, default background subtraction is achieved in the Spec2Pipeline by mutually subtracting the nodded expsosures, so the basic use case does not require additional background subtraction from the spectral images.**\n", + "* ``bkg_coeff``: polynomial coefficients for defining background extraction region limits (list of lists of float); single-entry lists are interpreted as vertical boundaries.\n", + "* ``src_coeff``: polynomial coefficients for the source extraction region limits (list of lists of float); this setting takes precedence over the start/stop values if both are provided.\n", + "* ``smoothing_length`` (odd int): width of the boxcar kernel used for smoothing the background regions along the dispersion direction.\n", + "* ``independent_var``: variable against which the ``bkg_coeff`` and ``src_coeff`` values are defined; can be 'wavelength' or 'pixel'.\n", + "* ``use_source_posn`` (True/False/None): if set to True, the pipeline will use the target coordinates to locate the target along the slit and move the center of the extraction aperture to this location; if set to None, the aperture shift will be carried out for point sources only.\n", + "\n", + "Note that in addition to the ``extract_1d`` reference file, the spectral extraction parameter settings can also be adjusted at the pipeline level. For example, the current default pipeline workflow sets ``use_source_posn`` = True for Stage 2 processing of all LRS slit data, due to the nodded exposures; however, ``use_source_posn`` is set to False for Stage 3 processing, where the combined resampled spectrum is typically at a nominal position.\n", + "\n", + "For MIRI LRS, the dispersion is in the vertical direction (i.e. `disp_axis` = 2), and so the extraction aperture width is specified with the coordinates `xstart` and `xstop`. If `ystart` and `ystop` are not provided, the spectrum will be extracted over the full height of the ``s2d`` cutout region. We can illustrate the default extraction region in the Level 3 ``s2d`` file. \n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "703f59cd", + "metadata": {}, + "outputs": [], + "source": [ + "xstart = x1dref_default['apertures'][0]['xstart']\n", + "xstop = x1dref_default['apertures'][0]['xstop']\n", + "ap_height = np.shape(l3_s2d.data)[0] # no ystart/ystop provided, so the aperture extends the full height\n", + "ap_width = xstop - xstart + 1\n", + "x1d_rect = Rectangle(xy=(xstart, 0), width=ap_width, height=ap_height, angle=0., edgecolor='red',\n", + " facecolor='None', ls='-', lw=1.5)\n", + "\n", + "fig, ax = plt.subplots(figsize=[2, 8])\n", + "im2d = ax.imshow(l3_s2d.data, origin='lower', aspect='auto', cmap='gist_gray')\n", + "ax.add_patch(x1d_rect)\n", + "ax.set_xlabel('column')\n", + "ax.set_ylabel('row')\n", + "ax.set_title('SN2021aefx - Level 3 resampled 2D spectral image')\n", + "fig.colorbar(im2d)\n", + "fig.show()" + ] + }, + { + "cell_type": "markdown", + "id": "7bc74010-0d59-4d0d-a08a-e9997350b8b7", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "id": "1fd784d9", + "metadata": {}, + "source": [ + "## Example 1: Changing the extraction aperture width\n", + "\n", + "In this example, we demonstrate how to change the extraction width from the default 8 pixels to 12 pixels while keeping the aperture centred on the trace. \n", + "\n", + "Here, we modify the values programmatically, having previously read in the default settings from the spectral extraction reference file. In practice, the values can also be simply edited in the reference file using a text editor. **Note: because the default ``extract_1d`` parameter file does not have ``use_source_posn`` set, the ``extract_1d`` step assumes that it is set to None and will activate the source finder for point sources. To explicitly fix the extraction region programmatically, we must set ``use_source_posn`` = False.**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "06dc8eb5", + "metadata": {}, + "outputs": [], + "source": [ + "xstart2 = xstart - 2\n", + "xstop2 = xstop + 2\n", + "print('New xstart, xstop values = {0}, {1}'.format(xstart2, xstop2))\n", + "\n", + "# Replace default values\n", + "with open(json_ref_default) as json_ref:\n", + " x1dref_default = json.load(json_ref)\n", + " x1dref_ex1 = x1dref_default.copy()\n", + " x1dref_ex1['apertures'][0]['use_source_posn'] = False\n", + " x1dref_ex1['apertures'][0]['xstart'] = xstart2\n", + " x1dref_ex1['apertures'][0]['xstop'] = xstop2\n", + "\n", + "# Save new reference file\n", + "with open('x1d_reffile_example1.json', 'w') as jsrefout:\n", + " json.dump(x1dref_ex1, jsrefout, indent=4)" + ] + }, + { + "cell_type": "markdown", + "id": "bb41d374-ebb4-4e84-bfb4-286859da8d52", + "metadata": {}, + "source": [ + "We now overplot the two apertures (default vs. custom) on the Level 3 spectral image." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5bc85413", + "metadata": {}, + "outputs": [], + "source": [ + "ap_width2 = xstop2 - xstart2 + 1\n", + "x1d_rect1 = Rectangle(xy=(xstart, 0), width=ap_width, height=ap_height, angle=0., edgecolor='red',\n", + " facecolor='None', ls='-', lw=1, label='8-px aperture (default)')\n", + "\n", + "x1d_rect2 = Rectangle(xy=(xstart2, 0), width=ap_width2, height=ap_height, angle=0., edgecolor='cyan',\n", + " facecolor='None', ls='-', lw=1, label='12-px aperture')\n", + "\n", + "fig4, ax4 = plt.subplots(figsize=[2, 8])\n", + "im2d = ax4.imshow(l3_s2d.data, origin='lower', aspect='auto', cmap='gist_gray')\n", + "# ax4.add_collection(aps_collection)\n", + "ax4.add_patch(x1d_rect1)\n", + "ax4.add_patch(x1d_rect2)\n", + "\n", + "ax4.set_xlabel('column')\n", + "ax4.set_ylabel('row')\n", + "ax4.set_title('Example 1: Default vs modified extraction aperture')\n", + "ax4.legend(loc='lower center')\n", + "fig4.colorbar(im2d)\n", + "fig4.show()" + ] + }, + { + "cell_type": "markdown", + "id": "87efcb9f", + "metadata": {}, + "source": [ + "Next, we run the spectral extraction step using this modified reference file. **Note: when a step is run individually, the file name suffix is different from when the Spec3Pipeline is run in its entirety.** The extracted spectrum will now have ``extract1dstep.fits`` in the filename. \n", + "\n", + "To use the new modified reference file and customize the output filename for specificity, we pass two custom parameters to the step call:\n", + "\n", + "* ``output_file``: the custom output filename for this example (including an output filename renders the ``save_results`` parameter obsolete)\n", + "* ``override_extract1d``: (relative or absolute) path to the modified reference file we created above\n", + "\n", + "Here, we also demonstrate the usage of the ``extraction_type`` parameter. By default, a boxcar extraction is performed, which can be explicitly set using ``extraction_type = 'box'``. This parameter is necessary when performing optimal spectral extraction (Example 5)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7304f758", + "metadata": { + "scrolled": true, + "tags": [ + "scroll-output" + ] + }, + "outputs": [], + "source": [ + "sp3_ex1 = Extract1dStep.call(l3_s2d, output_dir='data/', \n", + " output_file='lrs_slit_extract_example1', override_extract1d='x1d_reffile_example1.json',\n", + " extraction_type = 'box')" + ] + }, + { + "cell_type": "markdown", + "id": "e2f69145-dae3-441b-a4d1-00ee51cec2c0", + "metadata": {}, + "source": [ + "We now plot the spectrum from the modified extraction against the default spectrum. Recall that the spectral extraction includes an aperture correction that accounts for the flux loss outside of the aperture, and so the spectra should be largely identical in flux level throughout. Minor differences can be apparent at longer wavelengths as the aperture correction is less well-calibrated in this low SNR region. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "91ebfc64", + "metadata": {}, + "outputs": [], + "source": [ + "fig5, ax5 = plt.subplots(figsize=[12, 4])\n", + "ax5.plot(l3_spec.spec[0].spec_table['WAVELENGTH'], l3_spec.spec[0].spec_table['FLUX'], label='8-px aperture')\n", + "ax5.plot(sp3_ex1.spec[0].spec_table['WAVELENGTH'], sp3_ex1.spec[0].spec_table['FLUX'], label='12-px aperture')\n", + "ax5.set_xlabel('wavelength (μm)')\n", + "ax5.set_ylabel('flux (Jy)')\n", + "ax5.set_title('Example 1: Difference aperture sizes')\n", + "ax5.set_xlim(5., 14.)\n", + "ax5.legend()\n", + "fig5.show()" + ] + }, + { + "cell_type": "markdown", + "id": "c93a0a30-ef2b-4df2-a1e8-1491aa926e69", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "id": "f28a4f8e", + "metadata": {}, + "source": [ + "## Example 2: Changing the extraction aperture location\n", + "\n", + "In this example, we demonstrate how to adjust the location of the spectral extraction region. A good use case for this is extracting the spectrum from one of the Level 2 nodded exposures. We will take the ``s2d`` output from the Spec2Pipeline and extract the spectrum. \n", + "\n", + "For the first nodded exposure, the spectral trace is centered at column 13 (0-indexed). There are two ways we can adjust the aperture location. First, we can manually adjust the ``xstart`` and ``xstop`` values to account for the actual location of the spectral trace." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8a8cf793", + "metadata": {}, + "outputs": [], + "source": [ + "l2_s2d = datamodels.open(l2_s2d_file)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "55c81453", + "metadata": {}, + "outputs": [], + "source": [ + "xstart3 = 9\n", + "xstop3 = 16\n", + "\n", + "with open(json_ref_default) as json_ref:\n", + " x1dref_default = json.load(json_ref)\n", + " x1dref_ex2 = x1dref_default.copy()\n", + " x1dref_ex2['apertures'][0]['use_source_posn'] = False\n", + " x1dref_ex2['apertures'][0]['xstart'] = xstart3\n", + " x1dref_ex2['apertures'][0]['xstop'] = xstop3\n", + "\n", + "with open('x1d_reffile_example2a.json', 'w') as jsrefout:\n", + " json.dump(x1dref_ex2, jsrefout, indent=4)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fe340506", + "metadata": {}, + "outputs": [], + "source": [ + "ap_width3 = xstop3 - xstart3 + 1\n", + "x1d_rect3 = Rectangle(xy=(xstart3, 0), width=ap_width3, height=ap_height, angle=0., edgecolor='red',\n", + " facecolor='None', ls='-', lw=1, label='8-px aperture at nod 1 location')\n", + "\n", + "fig6, ax6 = plt.subplots(figsize=[2, 8])\n", + "im2d = ax6.imshow(l2_s2d.data, origin='lower', aspect='auto', cmap='gist_gray')\n", + "ax6.add_patch(x1d_rect3)\n", + "ax6.set_xlabel('column')\n", + "ax6.set_ylabel('row')\n", + "ax6.set_title('Example 2a: Different aperture location')\n", + "ax6.legend(loc=3)\n", + "fig6.colorbar(im2d)\n", + "fig6.show()" + ] + }, + { + "cell_type": "markdown", + "id": "6694ba0f-be1d-4a2a-9348-aed0863d0f4d", + "metadata": {}, + "source": [ + "As before, we pass the modified reference file to the ``extract_1d`` step call." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3b0b287b", + "metadata": { + "scrolled": true, + "tags": [ + "scroll-output" + ] + }, + "outputs": [], + "source": [ + "sp2_ex2a = Extract1dStep.call(l2_s2d_file, output_dir='data/', output_file='lrs_slit_extract_example2a',\n", + " override_extract1d='x1d_reffile_example2a.json', extraction_type = 'box')" + ] + }, + { + "cell_type": "markdown", + "id": "32eaa24e", + "metadata": {}, + "source": [ + "Let's again plot the output against the default extracted product. We expect this single-nod spectrum to be noisier but not significantly different from the combined product. The spectrum may have more bad pixels that manifest as spikes or dips in the spectrum. Deviations at long wavelengths stem from imperfections in the photometric calibration in that region; the standard stars used for calibration have low SNR at the longest wavelengths. Moreover, the flux calibration of the MIRI LRS instrument is currently based off of Level 3 nod-combined spectra and therefore does not readily account for possible spatial variations in the sensitivity across the detector." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ce8eccfb", + "metadata": {}, + "outputs": [], + "source": [ + "fig7, ax7 = plt.subplots(figsize=[12, 4])\n", + "ax7.plot(l3_spec.spec[0].spec_table['WAVELENGTH'], l3_spec.spec[0].spec_table['FLUX'], label='default location (nods combined)')\n", + "ax7.plot(sp2_ex2a.spec[0].spec_table['WAVELENGTH'], sp2_ex2a.spec[0].spec_table['FLUX'], label='nod 1 location (single nod)')\n", + "ax7.set_xlabel('wavelength (μm)')\n", + "ax7.set_ylabel('flux (Jy)')\n", + "ax7.set_title('Example 2a: Different aperture locations')\n", + "ax7.set_xlim(5., 14.)\n", + "ax7.legend()\n", + "fig7.show()" + ] + }, + { + "cell_type": "markdown", + "id": "8bceb778-decf-47a4-9a7d-da9496c42969", + "metadata": {}, + "source": [ + "Alternatively, we can set ``use_source_posn`` to True and let the pipeline determine the location of the aperture based on the pointing information stored in the file header. As long as the target acquisition executed nominally, the header information will reliably account for the offset between the nod positions, allowing the ``extract_1d`` step to shift the aperture appropriately. \n", + "\n", + "Because we care primarily about the width of the extraction region (8 pixels in this case), we can set the ``extract_width`` parameter to 8, which will automatically establish an 8-pixel-wide boxcar aperture centered on the precise pixel location of the source spectral trace determined by the ``extract_1d`` routine." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e3e1d69d-f1e2-47f6-8c0d-5ce280c2a677", + "metadata": {}, + "outputs": [], + "source": [ + "with open(json_ref_default) as json_ref:\n", + " x1dref_default = json.load(json_ref)\n", + " x1dref_ex2 = x1dref_default.copy()\n", + " x1dref_ex2['apertures'][0]['use_source_posn'] = True\n", + " x1dref_ex2['apertures'][0]['extract_width'] = 8\n", + "\n", + "with open('x1d_reffile_example2b.json', 'w') as jsrefout:\n", + " json.dump(x1dref_ex2, jsrefout, indent=4)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "af8819cd-07c7-48ff-b33e-3a91ad054957", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "sp2_ex2b = Extract1dStep.call(l2_s2d_file, output_dir='data/', output_file='lrs_slit_extract_example2b',\n", + " override_extract1d='x1d_reffile_example2b.json', extraction_type = 'box')" + ] + }, + { + "cell_type": "markdown", + "id": "50e225ca-4274-4bc4-97c9-05d858157231", + "metadata": {}, + "source": [ + "Let's retrieve the aperture boundary locations determined by the spectral extraction. Note that the boundary information stored in the metadata is (usually) 1-indexed, unlike the 0-indexed convention used in the ``extract_1d`` reference file." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "be26c2c0-1a22-4f87-9c3b-6d3caa316778", + "metadata": {}, + "outputs": [], + "source": [ + "xstart4, xstop4 = sp2_ex2b.spec[0].extraction_xstart, sp2_ex2b.spec[0].extraction_xstop\n", + "print('xstart, xstop values (1-indexed) = {0:.3f}, {1:.3f}'.format(xstart4, xstop4))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3cd62fb6-447c-4b3b-bd54-89d5b827129b", + "metadata": {}, + "outputs": [], + "source": [ + "fig7, ax7 = plt.subplots(figsize=[12, 4])\n", + "ax7.plot(l3_spec.spec[0].spec_table['WAVELENGTH'], l3_spec.spec[0].spec_table['FLUX'], label='default location (nods combined)')\n", + "ax7.plot(sp2_ex2b.spec[0].spec_table['WAVELENGTH'], sp2_ex2b.spec[0].spec_table['FLUX'], label='nod 1 location (single nod, use_source_posn=True)')\n", + "ax7.set_xlabel('wavelength (μm)')\n", + "ax7.set_ylabel('flux (Jy)')\n", + "ax7.set_title('Example 2b: Different aperture locations')\n", + "ax7.set_xlim(5., 14.)\n", + "ax7.legend()\n", + "fig7.show()" + ] + }, + { + "cell_type": "markdown", + "id": "10bb9a7f-4187-42b3-bc26-ee796ff6afcc", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "id": "39492ab3", + "metadata": {}, + "source": [ + "## Example 3: Extraction with background subtraction\n", + "\n", + "For LRS slit observations, the default background subtraction strategy is performed in the ``background`` step in Spec2Pipeline; the 2 nodded exposures are mutually subtracted, resulting in a pair of ``s2d`` files each containing a background-subtracted 2D spectral image with a positive and a negative trace. This approach is optimized for point sources, and the current flux calibration of the JWST pipeline is benchmarked to yield the correct flux level when using this method.\n", + "\n", + "For non-standard cases, however, it may be desirable to skip the mutual nodded exposure subtraction step and instead subtract the background directly from the spectral image as part of the spectral extraction in ``extract_1d``. For example, LRS observations of extended sources or scenes with multiple sources within the slit (e.g., binary stars) are not well-suited for the default background subtraction step and require extra care to isolate the signal from the desired target. \n", + "\n", + "In the ``extract_1d`` reference file, we can pass specific parameters for the background subtraction. The most important consideration is the background region from which the background flux level will be modeled. Care should be taken to ensure that the background region avoids astrophysical signal from sources within the slit, while simultaneously being extensive enough to provide a reliable representation of the background level. \n", + "\n", + "For this example, we will use the Level 2 ``s2d`` spectral image produced without nod subtraction: ``l2_s2d_file_nobsub``. Let us examine the flux distribution in the cross-dispersion direction for a select number of rows.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b4ea99ea", + "metadata": {}, + "outputs": [], + "source": [ + "l2_s2d_nobsub = datamodels.open(l2_s2d_file_nobsub)\n", + "\n", + "rows = [140, 200, 325]\n", + "fig8, ax8 = plt.subplots(figsize=[8, 4])\n", + "ncols = np.shape(l2_s2d_nobsub.data)[1]\n", + "pltx = np.arange(ncols)\n", + "for rr in rows:\n", + " label = 'row {}'.format(rr)\n", + " ax8.plot(pltx, l2_s2d_nobsub.data[rr, :], label=label)\n", + "ax8.set_xlabel('column')\n", + "ax8.set_ylabel('flux density (MJy/sr)')\n", + "ax8.legend()\n", + "fig8.show()" + ] + }, + { + "cell_type": "markdown", + "id": "3dd6c51c-e596-4f34-a9ca-67d12170cbec", + "metadata": {}, + "source": [ + "From the plot, it is evident that the background level is not uniform in the cross-dispersion direction across the spectral image, particularly at long wavelengths (i.e., lower row numbers), where it rises from left to right. To estimate the background level in the region of the spectral trace, let's establish two background regions on either side of the source. The location of peak flux is located around columns 13-14, so we will take columns 1-6 and 21-26 (inclusive) as the background regions. \n", + "\n", + "Let's visualize the background regions." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a0344b30-c200-491b-8b48-7a39d310be39", + "metadata": {}, + "outputs": [], + "source": [ + "fig8, ax8 = plt.subplots(figsize=[8, 4])\n", + "ncols = np.shape(l2_s2d_nobsub.data)[1]\n", + "pltx = np.arange(ncols)\n", + "for rr in rows:\n", + " label = 'row {}'.format(rr)\n", + " ax8.plot(pltx, l2_s2d_nobsub.data[rr, :], label=label)\n", + "ax8.axvline(x=1, ymin=0, ymax=1, ls='--', lw=1., color='coral', label='background regions')\n", + "ax8.axvline(x=6, ymin=0, ymax=1, ls='--', lw=1., color='coral')\n", + "ax8.axvline(x=21, ymin=0, ymax=1, ls='--', lw=1., color='coral')\n", + "ax8.axvline(x=26, ymin=0, ymax=1, ls='--', lw=1., color='coral')\n", + "ax8.set_xlabel('column')\n", + "ax8.set_ylabel('flux density (MJy/sr)')\n", + "ax8.legend()\n", + "fig8.show()" + ] + }, + { + "cell_type": "markdown", + "id": "a1df45c8-8884-4db8-aca7-43185db90869", + "metadata": {}, + "source": [ + "Having chosen the desired background region, we use the ``bkg_coeff`` parameter to pass our selection to the ``extract_1d`` step. It is important to note that the pixel coordinate convention in ``bkg_coeff`` is somewhat different than for ``xstart`` and ``xstop``. For typical use cases, ``xstart`` and ``xstop`` are integers, which correspond to the whole pixels on the detector, i.e., the aperture boundary extends from the left edge of the ``xstart`` pixel to the right edge of the ``xstop`` pixel. In contrast, ``bkg_coeff`` assumes exact pixel coordinates, where a whole number pixel value cooresponds to the geometric center of that pixel on the detector. Therefore, in order to include the full pixels within the two column ranges defined above, we must use the pixel boundary coordinates 0.5, 6.5, 20.5, and 26.5. \n", + "\n", + "The ``bkg_coeff`` parameter consists of a list of lists, where each list in the collection is a set of coefficients for a polynomial boundary along the vertical axis. Here, because we are using simple vertical lines as boundaries, each list consists of just a single number, corresponding to a pixel location. \n", + "\n", + "The ``bkg_fit`` parameter instructs the pipeline on the method of background modeling per row. The possible choices are 'mean', 'median', and 'poly'. Given that the background flux distribution is quasilinear, we will choose 'poly'. If 'poly' is chosen, the user must also define the order of the polynomial via the ``bkg_order`` parameter; in this case, we set it to 1, for a linear fit." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1238d6c9", + "metadata": {}, + "outputs": [], + "source": [ + "with open(json_ref_default) as json_ref:\n", + " x1dref_default = json.load(json_ref)\n", + " x1dref_ex3 = x1dref_default.copy()\n", + " x1dref_ex3['apertures'][0]['use_source_posn'] = False\n", + " x1dref_ex3['apertures'][0]['xstart'] = xstart3\n", + " x1dref_ex3['apertures'][0]['xstop'] = xstop3\n", + " x1dref_ex3['apertures'][0]['bkg_coeff'] = [[0.5], [6.5], [20.5], [26.5]]\n", + " x1dref_ex3['apertures'][0]['bkg_fit'] = 'poly'\n", + " x1dref_ex3['apertures'][0]['bkg_order'] = 1\n", + "\n", + "with open('x1d_reffile_example3a.json', 'w') as jsrefout:\n", + " json.dump(x1dref_ex3, jsrefout, indent=4)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ac0d2746", + "metadata": { + "scrolled": true, + "tags": [ + "scroll-output" + ] + }, + "outputs": [], + "source": [ + "sp2_ex3a = Extract1dStep.call(l2_s2d_file_nobsub, output_dir='data/', output_file='lrs_slit_extract_example3a',\n", + " override_extract1d='x1d_reffile_example3a.json', extraction_type = 'box')" + ] + }, + { + "cell_type": "markdown", + "id": "45b69d9b", + "metadata": {}, + "source": [ + "When the ``extract_1d`` step performs a background subtraction, the background spectrum is part of the output product, so we can check what was subtracted. In the plot below, we compare the spectrum extracted using in-scene background subtraction of the Level 2 spectral image without pipeline nod subtraction to the one we previously derived from the nod-subtracted image (with no additional background removal)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7210c9ac", + "metadata": {}, + "outputs": [], + "source": [ + "fig9, ax9 = plt.subplots(nrows=2, ncols=1, figsize=[12, 4])\n", + "ax9[0].plot(sp2_ex2a.spec[0].spec_table['WAVELENGTH'], sp2_ex2a.spec[0].spec_table['FLUX'], label='nod 1 spectrum - pipeline nod subtraction')\n", + "ax9[0].plot(sp2_ex3a.spec[0].spec_table['WAVELENGTH'], sp2_ex3a.spec[0].spec_table['FLUX'], label='nod 1 spectrum - with in-scene bkg sub')\n", + "ax9[1].plot(sp2_ex3a.spec[0].spec_table['WAVELENGTH'], sp2_ex3a.spec[0].spec_table['BACKGROUND'])\n", + "ax9[1].set_xlabel('wavelength (μm)')\n", + "ax9[0].set_ylabel('flux (Jy)')\n", + "ax9[1].set_ylabel('background (MJy/sr)')\n", + "ax9[0].set_title('Example 3a: Extraction with background subtraction')\n", + "ax9[0].set_xlim(5., 14.)\n", + "ax9[1].set_xlim(5., 14.)\n", + "ax9[0].legend()\n", + "fig9.show()" + ] + }, + { + "cell_type": "markdown", + "id": "5d360c6c-e4d1-45fa-9121-01cb9af6deff", + "metadata": {}, + "source": [ + "The two spectra are largely comparable. It is important to recall that the current photometric calibration of the LRS slit is based on nod-subtracted Level 3 products. At the longest wavelengths, where the PSF of the spectral trace broadens considerably, there is nonnegligible overlap between the two nodded spectral profiles. In other words, when carrying out the nod subtraction, some fraction of the target flux in Nod 2 is removed from the Nod 1 spectrum, and vice versa. The aperture correction accounts for this during standard pipeline processing. However, when producing custom spectra from ``s2d`` files without nod subtraction, there will be some bias in the overall flux level of the extracted spectra due to the discrepant processing workflows. Users that require accurate absolute flux measurements should therefore recalibrate the flux levels by extracting the spectra of standard stars using an analogous methodology and comparing them to stellar model spectra.\n", + "\n", + "From the figure, we can see that the background level rises steeply at long wavelengths, consistent with our earlier inspection of the spectral image. We also note some significant jitter at long wavelengths, likely due to bad pixels and generally poorer SNR. It may be helpful to smooth the background model to lessen the impact of these pixel-to-pixel variations while preserving the overall trend with wavelength. This can be accomplished by using the ``smoothing_length`` parameter, which provides the width of the boxcar kernel used for smoothing the background regions along the dispersion direction; the parameter must be an odd integer." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "95c6e2c4-40c5-464f-858e-d4a0c5d2f8df", + "metadata": {}, + "outputs": [], + "source": [ + "with open(json_ref_default) as json_ref:\n", + " x1dref_default = json.load(json_ref)\n", + " x1dref_ex3 = x1dref_default.copy()\n", + " x1dref_ex3['apertures'][0]['use_source_posn'] = False\n", + " x1dref_ex3['apertures'][0]['xstart'] = xstart3\n", + " x1dref_ex3['apertures'][0]['xstop'] = xstop3\n", + " x1dref_ex3['apertures'][0]['bkg_coeff'] = [[0.5], [6.5], [20.5], [26.5]]\n", + " x1dref_ex3['apertures'][0]['bkg_fit'] = 'poly'\n", + " x1dref_ex3['apertures'][0]['bkg_order'] = 1\n", + " x1dref_ex3['apertures'][0]['smoothing_length'] = 21\n", + "\n", + "with open('x1d_reffile_example3b.json', 'w') as jsrefout:\n", + " json.dump(x1dref_ex3, jsrefout, indent=4)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d72b29e1-ba51-4755-9ce2-39dbd0e0dff9", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "sp2_ex3b = Extract1dStep.call(l2_s2d_file_nobsub, output_dir='data/', output_file='lrs_slit_extract_example3b',\n", + " override_extract1d='x1d_reffile_example3b.json', extraction_type = 'box')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3c22a625-1ce0-4705-80b9-b13a5aac8c82", + "metadata": {}, + "outputs": [], + "source": [ + "fig9, ax9 = plt.subplots(nrows=2, ncols=1, figsize=[12, 4])\n", + "ax9[0].plot(sp2_ex2a.spec[0].spec_table['WAVELENGTH'], sp2_ex2a.spec[0].spec_table['FLUX'], label='nod 1 spectrum - pipeline nod subtraction')\n", + "ax9[0].plot(sp2_ex3b.spec[0].spec_table['WAVELENGTH'], sp2_ex3b.spec[0].spec_table['FLUX'], label='nod 1 spectrum - with in-scene bkg sub (smoothed)')\n", + "ax9[1].plot(sp2_ex3b.spec[0].spec_table['WAVELENGTH'], sp2_ex3b.spec[0].spec_table['BACKGROUND'])\n", + "ax9[1].set_xlabel('wavelength (μm)')\n", + "ax9[0].set_ylabel('flux (Jy)')\n", + "ax9[1].set_ylabel('background (MJy/sr)')\n", + "ax9[0].set_title('Example 3b: Extraction with smoothed background subtraction')\n", + "ax9[0].set_xlim(5., 14.)\n", + "ax9[1].set_xlim(5., 14.)\n", + "ax9[0].legend()\n", + "fig9.show()" + ] + }, + { + "cell_type": "markdown", + "id": "7a833b74-68ff-4d33-abf8-b2d00a493c5f", + "metadata": {}, + "source": [ + "As expected, the modeled background array and background-corrected spectrum are considerably smoother. The residual outliers in the extracted spectrum are likely caused by bad pixels within the extraction aperture, which cannot be directly addressed through the ``extract_1d`` step and must be dealt with separately." + ] + }, + { + "cell_type": "markdown", + "id": "678eeb21-4ab5-4c5c-ae1d-70e19d85cbe5", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "id": "e01786eb", + "metadata": {}, + "source": [ + "## Example 4: Tapered column extraction\n", + "\n", + "The ``extract_1d`` reference file supports variable apertures that change as a function of pixel position or wavelength. In this example, we will use the JWST calibration pipeline to perform a spectral extraction with a tapered column aperture that grows wider as a function of wavelength, mirroring the broadening of the spectral trace PSF. Specifically, we want to set the aperture width to 3 times the FWHM of the spatial profile. The way to specify this in the reference file is to use the ``src_coeff`` parameter instead of the simpler ``xstart`` and ``xstop`` settings. The ``src_coeff`` parameter can take polynomial coefficients rather than fixed pixel values.\n", + "\n", + "Polynomial definitions for the extraction aperture can be specified as a function of either pixels or wavelength. The particular choice must be denoted using the ``independent_var`` parameter. We will use pre-measured FWHM values as a function of **wavelength** and fit a straight line to the FWHM($\\lambda$) profile. Of course, the FWHM can also be measured directly from the data. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9e3c2433", + "metadata": {}, + "outputs": [], + "source": [ + "def calc_xap_fit():\n", + " # FWHM values measured from commissioning data. FWHM is in arcsec.\n", + " lam = [5.0, 7.5, 10.0, 12.0]\n", + " fwhm = [0.29, 0.3, 0.36, 0.42]\n", + "\n", + " # Convert FWHM from arcsec to pixel units using the MIRI pixel scale of 0.11 arcsec/px\n", + " fwhm_px = fwhm / (0.11*u.arcsec/u.pixel)\n", + "\n", + " # Define a aperture with width of 3 * FWHM width, i.e., 1.5 * FWHM on either side of the trace\n", + " xap_pix = 1.5 * fwhm_px\n", + "\n", + " # Fit a line to these points\n", + " line_init = models.Linear1D()\n", + " fit = fitting.LinearLSQFitter()\n", + " fitted_line = fit(line_init, lam, xap_pix.value)\n", + "\n", + " # Plot\n", + " fig, ax = plt.subplots(figsize=[8, 4])\n", + " xplt = np.linspace(4.0, 14., num=50)\n", + " ax.plot(lam, xap_pix.value, 'rx', label='1.5 * FWHM')\n", + " ax.plot(xplt, fitted_line(xplt), 'b-', label='best-fit line')\n", + " ax.set_xlabel('wavelength (μm)')\n", + " ax.set_ylabel('aperture half-width (px)')\n", + " ax.legend()\n", + " fig.show()\n", + "\n", + " return fitted_line" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e21fcec5", + "metadata": {}, + "outputs": [], + "source": [ + "poly_pos = calc_xap_fit()\n", + "print('slope = {0:.5f}, intercept = {1:.5f}'.format(poly_pos.slope.value, poly_pos.intercept.value))" + ] + }, + { + "cell_type": "markdown", + "id": "f7dc8785", + "metadata": {}, + "source": [ + "The above polynomial defines the relationship between wavelength and the location of the left/right boundary of the extract aperture. To ensure that the extraction location is centred on the location of the spectrum, we utilize the automatic aperture centering functionality and set ``use_source_posn`` to True.\n", + "\n", + "In the next cell, we provide the polynomial coefficients to the ``src_coeff`` parameter in the extraction reference file. **Note: the ``src_coeff`` parameter takes precedence over the ``xstart`` and ``xstop`` parameters if all three are present; for clarity we remove the latter from our reference file.**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a9c9d403", + "metadata": {}, + "outputs": [], + "source": [ + "with open(json_ref_default) as json_ref:\n", + " x1dref_default = json.load(json_ref)\n", + " x1dref_ex4 = x1dref_default.copy()\n", + " x1dref_ex4['apertures'][0]['xstart'] = None\n", + " x1dref_ex4['apertures'][0]['xstop'] = None\n", + " x1dref_ex4['apertures'][0]['use_source_posn'] = True\n", + " x1dref_ex4['apertures'][0]['independent_var'] = 'wavelength'\n", + " x1dref_ex4['apertures'][0]['src_coeff'] = [[-1*poly_pos.intercept.value, -1*poly_pos.slope.value], [poly_pos.intercept.value, poly_pos.slope.value]]\n", + "\n", + "with open('x1d_reffile_example4.json', 'w') as jsrefout:\n", + " json.dump(x1dref_ex4, jsrefout, indent=4)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "71789e83", + "metadata": { + "scrolled": true, + "tags": [ + "scroll-output" + ] + }, + "outputs": [], + "source": [ + "sp3_ex4 = Extract1dStep.call(l3_s2d, output_dir='data/', \n", + " output_file='lrs_slit_extract_example4', override_extract1d='x1d_reffile_example4.json',\n", + " extraction_type = 'box')" + ] + }, + { + "cell_type": "markdown", + "id": "a31aae65-7e5e-4b94-9de1-f03f3881a4bf", + "metadata": {}, + "source": [ + "Plotting the default Level 3 spectrum with our tapered column extraction result, we see good agreement between the two spectra. In the case of a variable aperture, the aperture correction at each wavelength is computed for the precise width of the aperture. The aperture correction reference file only stores correction vectors for integer-width apertures, so minor discrepancies may arise due to the inherent imperfection of the interpolation process when calculating the aperture correction factor for fractional pixel width apertures." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9d1bc74c", + "metadata": {}, + "outputs": [], + "source": [ + "fig10, ax10 = plt.subplots(figsize=[12, 4])\n", + "ax10.plot(l3_spec.spec[0].spec_table['WAVELENGTH'], l3_spec.spec[0].spec_table['FLUX'], label='default fixed-width aperture')\n", + "ax10.plot(sp3_ex4.spec[0].spec_table['WAVELENGTH'], sp3_ex4.spec[0].spec_table['FLUX'], label='tapered column aperture')\n", + "ax10.set_xlabel('wavelength (μm)')\n", + "ax10.set_ylabel('flux (Jy)')\n", + "ax10.set_title('Example 4: Tapered column vs. fixed-width extraction aperture')\n", + "ax10.set_xlim(5., 14.)\n", + "ax10.legend()\n", + "fig10.show()" + ] + }, + { + "cell_type": "markdown", + "id": "378f8393", + "metadata": {}, + "source": [ + "The output spectrum object contains information about the number of pixels used for the extraction as a function of wavelength, which we visualize below." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "78ca0c68", + "metadata": {}, + "outputs": [], + "source": [ + "fig11, ax11 = plt.subplots(figsize=[12, 4])\n", + "ax11.plot(l3_spec.spec[0].spec_table['WAVELENGTH'], l3_spec.spec[0].spec_table['NPIXELS'], label='default fixed-width aperture')\n", + "ax11.plot(sp3_ex4.spec[0].spec_table['WAVELENGTH'], sp3_ex4.spec[0].spec_table['NPIXELS'], label='tapered column aperture')\n", + "ax11.set_xlabel('wavelength (μm)')\n", + "ax11.set_ylabel('aperture width (px)')\n", + "ax11.set_title('Example 4: Aperture width')\n", + "ax11.set_xlim(5., 14.)\n", + "ax11.set_ylim(6., 13.)\n", + "ax11.legend()\n", + "fig11.show()" + ] + }, + { + "cell_type": "markdown", + "id": "d65b451c-cb56-4d95-9769-96e62ac7c584", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "id": "14398ca1-2945-4de0-b617-92950e49e21a", + "metadata": {}, + "source": [ + "## Example 5: Optimal spectral extraction" + ] + }, + { + "cell_type": "markdown", + "id": "b975063d-ad4c-4aba-b02c-1985309751a9", + "metadata": {}, + "source": [ + "As of Version 1.18.0 of the JWST calibration pipeline, the ``extract_1d`` step allows for optimal spectral extraction. A finely sampled empirical PSF model of the LRS slit spectral trace is available in CRDS and is based on high SNR observations of relatively bright point sources. Together with a spectral trace information provided in the file header, the extraction step generates a normalized PSF model for each row, which is then fit to the measured pixel values to yield the calculated flux. Because the empirical PSF in principle encapsulates the full dispersed flux of a point source at each wavelength, no aperture correction is needed.\n", + "\n", + "There are several advantages of optimal spectral extraction. First, the input spectral image need not be rectified, as the algorithm leverages the spectral trace profile stored in the file header to align the PSF model in each row; this allows the user to avoid any interpolation artifacts that stem from the rectification process required to generate ``s2d`` files. Second, optimal spectral extraction is far more robust to missing data (e.g., bad pixels), cosmic rays, and point-to-point scatter and generally produces higher SNR spectra.\n", + "\n", + "Currently, the optimal spectral extraction option in ``extract_1d`` only allows for unrectified spectral images of the full detector (i.e., ``cal`` files).\n", + "\n", + "Optimal spectral extraction typically does not utilize the aperture and background subtraction information contained within the ``extract_1d`` reference file. Instead, control over the extraction process is handled directly via parameter settings that must be passed directly to the ``extract_1d`` call. Let's examine the relevant parameter settings:\n", + "\n", + "* ``extraction_type``: this is the crucial toggle that instructs the ``extract_1d`` algorithm to carry out optimal spectral extraction; 'box' is the default setting, which we implicitly used for all of the other examples in this notebook.\n", + "* ``use_source_posn`` (True/False/None): setting this to True is recommended, which allows the pipeline to carry out a first-order adjustment of the PSF model to align it with the expected location of the target along the slit; this setting has the same function as the analogous one contained in the ``extract_1d`` reference file.\n", + "* ``model_nod_pair`` (True/False): must be set to True if the ``s2d`` file contains a negative trace to instruct the algorithm to model all positive and negative traces using the same empirical PSF model.\n", + "* ``optimize_psf_location`` (True/False): setting this to True is strongly recommended, given inherent uncertainties in target location and pointing accuracy. When this option is set to True, the location of the positive and negative traces (if used) are optimized with respect to the residuals of the scene modeled by the PSF at that location. This option is particularly essential when ``model_nod_pair`` is set to True, because the negative nod location is less reliably estimated than the positive trace.\n", + "* ``subtract_background`` (True/False): set to True if mutual nod subtraction was not carried out to allow the algorithm simultaneously model the background.\n", + "* ``save_scene_model`` and ``save_residual_image`` (True/False): if True, these settings generate plots of the best-fit 2D composite scene model containing the scaled PSF(s) and background used to extract the spectrum and the corresponding residuals.\n", + "\n", + "We now proceed with the extraction. When ``extraction_type`` is set to 'optimal', the various parameter settings in the ``extract_1d`` reference file are ignored, as they do not factor into the optimal spectral extraction process. As a consequence, we must also stipulate ``save_results`` = True, given that we are not passing a custom reference file to the step call." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "15ce62bc-bf8a-489f-9d61-1aae7469f673", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "sp2_ex5a = Extract1dStep.call(l2_cal_file, output_dir='data/', \n", + " output_file='lrs_slit_extract_example5a', \n", + " extraction_type='optimal',\n", + " use_source_posn=True,\n", + " model_nod_pair=True,\n", + " optimize_psf_location=True,\n", + " save_scene_model=True,\n", + " save_residual_image=True,\n", + " save_results=True)" + ] + }, + { + "cell_type": "markdown", + "id": "4beccd44-ac99-4857-ae9b-234e04208903", + "metadata": {}, + "source": [ + "Let's take a look at the various outputs produced by this run. In addition to the spectrum, there are two diagnostics image files — a ``*scene_model.fits`` file and a ``*residual.fits`` file. These are the best-fit scene (containing both the scaled positive and negative PSFs) and the associated residuals from the fit.\n", + "\n", + "We plot these products below alongside the input ``cal`` spectral image. Recall that ``cal`` files contain the detector readout, whereas the LRS slit spectral region occupies only a small portion of the full array, and so we plot just the relevant subregion for inspection." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5bfc8bb5-f2e0-45cc-9adc-d3d880d640bf", + "metadata": {}, + "outputs": [], + "source": [ + "l2_cal = datamodels.open(l2_cal_file)\n", + "scene = datamodels.open('data/lrs_slit_extract_example5a_scene_model.fits')\n", + "resid = datamodels.open('data/lrs_slit_extract_example5a_residual.fits')\n", + "\n", + "fig12, ax12 = plt.subplots(nrows=1, ncols=3, figsize=[10, 8])\n", + "im2d = ax12[0].imshow(l2_cal.data[0:400,300:350], origin='lower', aspect='equal', cmap='viridis')\n", + "fig12.colorbar(im2d)\n", + "sce2d = ax12[1].imshow(scene.data[0:400,300:350], origin='lower', aspect='equal', cmap='viridis')\n", + "fig12.colorbar(sce2d)\n", + "res2d = ax12[2].imshow(resid.data[0:400,300:350], origin='lower', aspect='equal', cmap='viridis')\n", + "fig12.colorbar(res2d)\n", + "for i in range(3):\n", + " ax12[i].set_xlabel('column')\n", + " ax12[i].set_ylabel('row')\n", + "ax12[0].set_title('image')\n", + "ax12[1].set_title('scene model')\n", + "ax12[2].set_title('residuals')\n", + "fig12.show()" + ] + }, + { + "cell_type": "markdown", + "id": "c95b4d85-056b-40a0-8139-77360fd71483", + "metadata": {}, + "source": [ + "From the residual image, we find that the optimal spectral extraction does a decent job at modeling both the positive and negative spectral traces, though there is some discernible structure. These residual features are primarily due to an imperfect spectral trace profile in the header WCS solution, resulting in systematic deviations between the assumed trace shape and the true profile. Future updates to the ``specwcs`` reference file will improve the trace model.\n", + "\n", + "We now compare the spectrum we obtained through optimal extraction to the boxcar spectrum previously derived in Example 2b." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "eb47d382-5de3-43ea-8163-da918b5fcb5b", + "metadata": {}, + "outputs": [], + "source": [ + "fig13, ax13 = plt.subplots(figsize=[12, 4])\n", + "ax13.plot(sp2_ex2b.spec[0].spec_table['WAVELENGTH'], sp2_ex2b.spec[0].spec_table['FLUX'], label='boxcar extraction (single nod)')\n", + "ax13.plot(sp2_ex5a.spec[0].spec_table['WAVELENGTH'], sp2_ex5a.spec[0].spec_table['FLUX'], label='optimal extraction (single nod)')\n", + "ax13.set_xlabel('wavelength (μm)')\n", + "ax13.set_ylabel('flux (Jy)')\n", + "ax13.set_title('Example 5a: Optimal spectral extraction')\n", + "ax13.set_xlim(5., 14.)\n", + "ax13.legend()\n", + "fig13.show()" + ] + }, + { + "cell_type": "markdown", + "id": "5907f96d-da6f-4fb1-9bfd-6b3fb27487aa", + "metadata": {}, + "source": [ + "From the plot, it is evident that optimal spectral extraction yields the greatest improvement in SNR and the most significant reduction in spectral scatter at the longest wavelengths, where the instrument throughput is low and there is a high level of point-to-point variations in flux level across the detector. The systematic deviations and enhanced scatter of the optimal extraction spectrum near the flux maximum at 12 microns are attributable to poor PSF model fitting due to the imperfect assumed spectral trace profile." + ] + }, + { + "cell_type": "markdown", + "id": "840ae796-74b1-40c4-b343-a230dd8d236b", + "metadata": {}, + "source": [ + "Optimal spectral extraction can also be performed with simultaneous background modeling, which is useful for cases where mutual nod subtraction has not been carried out. Here, we repeat this exercise with the ``l2_cal_file_nobsub`` file and switch ``model_nod_pair`` to False. To activate background subtraction for optimal spectral extraction, we set ``subtract_background`` as True. \n", + "\n", + "By default, the current implementation of simultaneous background fitting assumes that the background flux level in each row is constant. However, we know that this is not true in actual LRS slit images, as discussed previously. To enable more robust background modeling as part of the optimal spectral extraction process, we require a number of parameter settings to be passed to the step through the ``extract_1d`` reference file. These are identical to the settings used for background subtraction in boxcar extractions (see Example 3).\n", + "\n", + "**Note: currently, the simultaneous background modeling is carried out row by row, without the option to smooth the background flux along the dispersion axis.** " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "34b77419-42ab-4adf-a6e3-126a981f2413", + "metadata": {}, + "outputs": [], + "source": [ + "with open(json_ref_default) as json_ref:\n", + " x1dref_default = json.load(json_ref)\n", + " x1dref_ex5 = x1dref_default.copy()\n", + " x1dref_ex5['apertures'][0]['bkg_fit'] = 'poly'\n", + " x1dref_ex5['apertures'][0]['bkg_order'] = 1\n", + "\n", + "with open('x1d_reffile_example5b.json', 'w') as jsrefout:\n", + " json.dump(x1dref_ex5, jsrefout, indent=4)" + ] + }, + { + "cell_type": "markdown", + "id": "8954ad6b-3a48-450c-9e62-20eaba750adb", + "metadata": {}, + "source": [ + "We can now run ``extract_1d`` with both the customized reference file and the step parameters for optimal spectral extraction." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7eb9e919-1cb5-4f02-8f5e-148d9284f054", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "sp2_ex5b = Extract1dStep.call(l2_cal_file_nobsub, output_dir='data/', \n", + " output_file='lrs_slit_extract_example5b', override_extract1d='x1d_reffile_example5b.json',\n", + " extraction_type='optimal',\n", + " use_source_posn=True,\n", + " model_nod_pair=False,\n", + " optimize_psf_location=True,\n", + " subtract_background=True,\n", + " save_scene_model=True,\n", + " save_residual_image=True,\n", + " save_results=True)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e8767cff-515c-4e0d-8f91-4dde34a3ed46", + "metadata": {}, + "outputs": [], + "source": [ + "l2_cal = datamodels.open(l2_cal_file_nobsub)\n", + "scene = datamodels.open('data/lrs_slit_extract_example5b_scene_model.fits')\n", + "resid = datamodels.open('data/lrs_slit_extract_example5b_residual.fits')\n", + "\n", + "fig14, ax14 = plt.subplots(nrows=1, ncols=3, figsize=[10, 8])\n", + "im2d = ax14[0].imshow(l2_cal.data[0:400,300:350], origin='lower', aspect='equal', cmap='viridis')\n", + "fig14.colorbar(im2d)\n", + "sce2d = ax14[1].imshow(scene.data[0:400,300:350], origin='lower', aspect='equal', cmap='viridis')\n", + "fig14.colorbar(sce2d)\n", + "res2d = ax14[2].imshow(resid.data[0:400,300:350], origin='lower', aspect='equal', cmap='viridis')\n", + "fig14.colorbar(res2d)\n", + "for i in range(3):\n", + " ax14[i].set_xlabel('column')\n", + " ax14[i].set_ylabel('row')\n", + "ax14[0].set_title('image')\n", + "ax14[1].set_title('scene model')\n", + "ax14[2].set_title('residuals')\n", + "fig14.show()" + ] + }, + { + "cell_type": "markdown", + "id": "53cdc350-8822-4e76-9d94-fcf56c5cd56e", + "metadata": {}, + "source": [ + "As illustrated in the plotted scene model, the fitted background largely reproduces the nonuniform distribution across the detector, with the flux rising toward the lower right corner of the LRS slit region. The resultant residual image is mostly absent of significant gradients, demonstrating the efficacy of this approach.\n", + "\n", + "Let's compare the result of this run with the spectrum from Example 3b, i.e., the boxcar extraction that used a similar linear background modeling approach but with background smoothing in the dispersion direction." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "08b49768-c06c-495c-a6d2-786dd24d06a6", + "metadata": {}, + "outputs": [], + "source": [ + "fig15, ax15 = plt.subplots(figsize=[12, 4])\n", + "ax15.plot(sp2_ex3b.spec[0].spec_table['WAVELENGTH'], sp2_ex3b.spec[0].spec_table['FLUX'], label='boxcar extraction (single nod)')\n", + "ax15.plot(sp2_ex5b.spec[0].spec_table['WAVELENGTH'], sp2_ex5b.spec[0].spec_table['FLUX'], label='optimal extraction (single nod)')\n", + "ax15.set_xlabel('wavelength (μm)')\n", + "ax15.set_ylabel('flux (Jy)')\n", + "ax15.set_title('Example 5b: Optimal spectral extraction with background subtraction')\n", + "ax15.set_xlim(5., 14.)\n", + "ax15.legend()\n", + "fig15.show()" + ] + }, + { + "cell_type": "markdown", + "id": "06dedfaf-d5cf-4aab-a2e6-70c87e3eec0b", + "metadata": {}, + "source": [ + "The optimal spectral extraction result is a significant improvement over the boxcar extraction spectrum, even with background smoothing applied to the latter. Moreover, the simultaneous background modeling and subtraction partially accounts for the previously observed mismatch in flux level near 12 microns when carrying out optimal spectral extraction on the nod-subtracted image. However, the fundamental issue of the misaligned trace profile persists and will be addressed in future updates.\n", + "\n", + "We note that ``subtract_background`` can be activated even if mutual nod subtraction has been performed. In that case, the background modeling can account for any residual background arising from differences in the background flux distribution between the two nodded exposures." + ] + }, + { + "cell_type": "markdown", + "id": "a7e717ac-0f57-4fda-aef9-663d6a94b827", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "id": "f4501ee7", + "metadata": {}, + "source": [ + "## Summary\n", + "\n", + "We hope this notebook is useful in demonstrating the advanced capabilities of the JWST calibration pipeline for spectral extraction of MIRI LRS slit datasets. The techniques described here are largely transferable to the analysis of other types of spectroscopic data from JWST, including MIRI LRS slitless and NIRSpec fixed slit observations. The above examples are not an exhaustive list of all possible use cases, and different methods for source and background extraction can be combined for even more complex extraction operations. \n", + "\n", + "**If you have any questions, comments, or requests for further demonstrations of these capabilities, please contact the [JWST Helpdesk](http://jwsthelp.stsci.edu/).**" + ] + }, + { + "cell_type": "markdown", + "id": "b4c3c7ed-44fa-4e53-abf0-4b285064746d", + "metadata": {}, + "source": [ + "
" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.14.3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/notebooks/MIRI/MIRI_LRS_spectral_extraction/requirements.txt b/notebooks/MIRI/MIRI_LRS_spectral_extraction/requirements.txt index 2ac92d612..184396d11 100644 --- a/notebooks/MIRI/MIRI_LRS_spectral_extraction/requirements.txt +++ b/notebooks/MIRI/MIRI_LRS_spectral_extraction/requirements.txt @@ -1,4 +1,2 @@ -jdaviz >= 3.6.0 astropy >= 5.3.1 -jwst >= 1.11.3 -specreduce >= 1.3.0 \ No newline at end of file +jwst >= 1.11.3 \ No newline at end of file diff --git a/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example1.json b/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example1.json new file mode 100644 index 000000000..5fa730cde --- /dev/null +++ b/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example1.json @@ -0,0 +1,45 @@ +{ + "reftype": "EXTRACT1D", + "instrument": "MIRI", + "telescope": "JWST", + "exp_type": "MIR_LRS-FIXEDSLIT|MIR_LRS-SLITLESS", + "pedigree": "INFLIGHT 2022-06-01 2024-12-31", + "description": "MIRI LRS extraction params for in-flight data", + "author": "I.Wong", + "history": "2025-May-29", + "useafter": "2022-01-01T00:00:00", + "apertures": [ + { + "id": "MIR_LRS-FIXEDSLIT", + "region_type": "target", + "bkg_order": 0, + "dispaxis": 2, + "xstart": 25, + "xstop": 36, + "use_source_posn": false + }, + { + "id": "MIR_LRS-SLITLESS", + "region_type": "target", + "bkg_fit": "median", + "dispaxis": 2, + "xstart": 30, + "xstop": 41, + "bkg_coeff": [ + [ + 9.5 + ], + [ + 17.5 + ], + [ + 53.5 + ], + [ + 61.5 + ] + ], + "use_source_posn": false + } + ] +} \ No newline at end of file diff --git a/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example2a.json b/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example2a.json new file mode 100644 index 000000000..f324248f5 --- /dev/null +++ b/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example2a.json @@ -0,0 +1,45 @@ +{ + "reftype": "EXTRACT1D", + "instrument": "MIRI", + "telescope": "JWST", + "exp_type": "MIR_LRS-FIXEDSLIT|MIR_LRS-SLITLESS", + "pedigree": "INFLIGHT 2022-06-01 2024-12-31", + "description": "MIRI LRS extraction params for in-flight data", + "author": "I.Wong", + "history": "2025-May-29", + "useafter": "2022-01-01T00:00:00", + "apertures": [ + { + "id": "MIR_LRS-FIXEDSLIT", + "region_type": "target", + "bkg_order": 0, + "dispaxis": 2, + "xstart": 9, + "xstop": 16, + "use_source_posn": false + }, + { + "id": "MIR_LRS-SLITLESS", + "region_type": "target", + "bkg_fit": "median", + "dispaxis": 2, + "xstart": 30, + "xstop": 41, + "bkg_coeff": [ + [ + 9.5 + ], + [ + 17.5 + ], + [ + 53.5 + ], + [ + 61.5 + ] + ], + "use_source_posn": false + } + ] +} \ No newline at end of file diff --git a/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example2b.json b/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example2b.json new file mode 100644 index 000000000..160683677 --- /dev/null +++ b/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example2b.json @@ -0,0 +1,46 @@ +{ + "reftype": "EXTRACT1D", + "instrument": "MIRI", + "telescope": "JWST", + "exp_type": "MIR_LRS-FIXEDSLIT|MIR_LRS-SLITLESS", + "pedigree": "INFLIGHT 2022-06-01 2024-12-31", + "description": "MIRI LRS extraction params for in-flight data", + "author": "I.Wong", + "history": "2025-May-29", + "useafter": "2022-01-01T00:00:00", + "apertures": [ + { + "id": "MIR_LRS-FIXEDSLIT", + "region_type": "target", + "bkg_order": 0, + "dispaxis": 2, + "xstart": 27, + "xstop": 34, + "use_source_posn": true, + "extract_width": 8 + }, + { + "id": "MIR_LRS-SLITLESS", + "region_type": "target", + "bkg_fit": "median", + "dispaxis": 2, + "xstart": 30, + "xstop": 41, + "bkg_coeff": [ + [ + 9.5 + ], + [ + 17.5 + ], + [ + 53.5 + ], + [ + 61.5 + ] + ], + "use_source_posn": false + } + ] +} \ No newline at end of file diff --git a/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example3a.json b/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example3a.json new file mode 100644 index 000000000..d22d9d77e --- /dev/null +++ b/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example3a.json @@ -0,0 +1,60 @@ +{ + "reftype": "EXTRACT1D", + "instrument": "MIRI", + "telescope": "JWST", + "exp_type": "MIR_LRS-FIXEDSLIT|MIR_LRS-SLITLESS", + "pedigree": "INFLIGHT 2022-06-01 2024-12-31", + "description": "MIRI LRS extraction params for in-flight data", + "author": "I.Wong", + "history": "2025-May-29", + "useafter": "2022-01-01T00:00:00", + "apertures": [ + { + "id": "MIR_LRS-FIXEDSLIT", + "region_type": "target", + "bkg_order": 1, + "dispaxis": 2, + "xstart": 9, + "xstop": 16, + "use_source_posn": false, + "bkg_coeff": [ + [ + 0.5 + ], + [ + 6.5 + ], + [ + 20.5 + ], + [ + 26.5 + ] + ], + "bkg_fit": "poly" + }, + { + "id": "MIR_LRS-SLITLESS", + "region_type": "target", + "bkg_fit": "median", + "dispaxis": 2, + "xstart": 30, + "xstop": 41, + "bkg_coeff": [ + [ + 9.5 + ], + [ + 17.5 + ], + [ + 53.5 + ], + [ + 61.5 + ] + ], + "use_source_posn": false + } + ] +} \ No newline at end of file diff --git a/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example3b.json b/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example3b.json new file mode 100644 index 000000000..4cddc3892 --- /dev/null +++ b/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example3b.json @@ -0,0 +1,61 @@ +{ + "reftype": "EXTRACT1D", + "instrument": "MIRI", + "telescope": "JWST", + "exp_type": "MIR_LRS-FIXEDSLIT|MIR_LRS-SLITLESS", + "pedigree": "INFLIGHT 2022-06-01 2024-12-31", + "description": "MIRI LRS extraction params for in-flight data", + "author": "I.Wong", + "history": "2025-May-29", + "useafter": "2022-01-01T00:00:00", + "apertures": [ + { + "id": "MIR_LRS-FIXEDSLIT", + "region_type": "target", + "bkg_order": 1, + "dispaxis": 2, + "xstart": 9, + "xstop": 16, + "use_source_posn": false, + "bkg_coeff": [ + [ + 0.5 + ], + [ + 6.5 + ], + [ + 20.5 + ], + [ + 26.5 + ] + ], + "bkg_fit": "poly", + "smoothing_length": 21 + }, + { + "id": "MIR_LRS-SLITLESS", + "region_type": "target", + "bkg_fit": "median", + "dispaxis": 2, + "xstart": 30, + "xstop": 41, + "bkg_coeff": [ + [ + 9.5 + ], + [ + 17.5 + ], + [ + 53.5 + ], + [ + 61.5 + ] + ], + "use_source_posn": false + } + ] +} \ No newline at end of file diff --git a/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example4.json b/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example4.json new file mode 100644 index 000000000..e31b25ddb --- /dev/null +++ b/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example4.json @@ -0,0 +1,56 @@ +{ + "reftype": "EXTRACT1D", + "instrument": "MIRI", + "telescope": "JWST", + "exp_type": "MIR_LRS-FIXEDSLIT|MIR_LRS-SLITLESS", + "pedigree": "INFLIGHT 2022-06-01 2024-12-31", + "description": "MIRI LRS extraction params for in-flight data", + "author": "I.Wong", + "history": "2025-May-29", + "useafter": "2022-01-01T00:00:00", + "apertures": [ + { + "id": "MIR_LRS-FIXEDSLIT", + "region_type": "target", + "bkg_order": 0, + "dispaxis": 2, + "xstart": null, + "xstop": null, + "use_source_posn": true, + "independent_var": "wavelength", + "src_coeff": [ + [ + -2.4456187153704048, + -0.25795198029961014 + ], + [ + 2.4456187153704048, + 0.25795198029961014 + ] + ] + }, + { + "id": "MIR_LRS-SLITLESS", + "region_type": "target", + "bkg_fit": "median", + "dispaxis": 2, + "xstart": 30, + "xstop": 41, + "bkg_coeff": [ + [ + 9.5 + ], + [ + 17.5 + ], + [ + 53.5 + ], + [ + 61.5 + ] + ], + "use_source_posn": false + } + ] +} \ No newline at end of file diff --git a/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example5b.json b/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example5b.json new file mode 100644 index 000000000..eb90b65dd --- /dev/null +++ b/notebooks/MIRI/MIRI_LRS_spectral_extraction/x1d_reffile_example5b.json @@ -0,0 +1,45 @@ +{ + "reftype": "EXTRACT1D", + "instrument": "MIRI", + "telescope": "JWST", + "exp_type": "MIR_LRS-FIXEDSLIT|MIR_LRS-SLITLESS", + "pedigree": "INFLIGHT 2022-06-01 2024-12-31", + "description": "MIRI LRS extraction params for in-flight data", + "author": "I.Wong", + "history": "2025-May-29", + "useafter": "2022-01-01T00:00:00", + "apertures": [ + { + "id": "MIR_LRS-FIXEDSLIT", + "region_type": "target", + "bkg_order": 1, + "dispaxis": 2, + "xstart": 27, + "xstop": 34, + "bkg_fit": "poly" + }, + { + "id": "MIR_LRS-SLITLESS", + "region_type": "target", + "bkg_fit": "median", + "dispaxis": 2, + "xstart": 30, + "xstop": 41, + "bkg_coeff": [ + [ + 9.5 + ], + [ + 17.5 + ], + [ + 53.5 + ], + [ + 61.5 + ] + ], + "use_source_posn": false + } + ] +} \ No newline at end of file