Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fci_l1c_nc reader drops channel attribute radiance_unit_conversion_coefficient when more than one file is loaded #3067

Open
evrose54 opened this issue Feb 19, 2025 · 0 comments

Comments

@evrose54
Copy link

Describe the bug
When creating a Scene object using the fci_l1c_nc reader, I've noticed that the calibration="radiance" channel attribute labeled radiance_unit_conversion_coefficient is missing from scn[channel].attrs when the Scene object is instantiated with more than one file. @ameraner was kind enough to point me in the right direction to locate this bug.

When a Scene object is instantiated with only one file, the channel attribute radiance_unit_conversion_coefficient is present in scn[channel].attrs. It'd be nice to have these values for full disk imagery, so I don't have to use radiance measured in wavenumber, and instead can use radiance measured in wavelength.

To Reproduce

"""Simple python script depicting fci_l1c_reader bug.

Demonstrates that attribute 'radiance_unit_conversion_coefficient' is missing from
a loaded radiance channel when that Scene was instantiated with more than one file.
"""

from glob import glob

from satpy import Scene

# One chunk of the full disk
single_path = "/home/erose/geoips/geoips_packages/test_data/test_data_geocolor/data/mtg/20240924.1500/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20240924151005_IDPFI_OPE_20240924150755_20240924150835_N__C_0091_0034.nc"
# Directory which contains all BODY chunks as well as the TRAIL chunk.
# 41 files in total.
all_paths = sorted(
    glob(
        "/home/erose/geoips/geoips_packages/test_data/test_data_geocolor/data/mtg/20240924.1500/*.nc"
    )
)

single_file_scn = Scene(filenames=[single_path], reader="fci_l1c_nc")
multi_file_scn = Scene(filenames=all_paths, reader="fci_l1c_nc")

single_file_scn.load(["vis_04"], calibration="radiance")
print(single_file_scn["vis_04"].attrs["radiance_unit_conversion_coefficient"])

# KeyError will occur here
multi_file_scn.load(["vis_04"], calibration="radiance")
print(multi_file_scn["vis_04"].attrs["radiance_unit_conversion_coefficient"])

Expected behavior
Per @ameraner and my own tests, I'd expect loading in all chunks of full disk imagery would behave the same way as loading in one chunk of the imagery. Something like this:

scn = Scene(filenames=['/tcenas/fbf/mtg/RollingBuffer/OPE/idpfi/FCI-L1/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20250219075028_IDPFI_OPE_20250219074755_20250219074835_N__O_0047_0034.nc'], reader='fci_l1c_nc')
scn.load(['ir_105'], calibration='radiance')
scn['ir_105'].attrs['radiance_unit_conversion_coefficient']
Out[9]: 
<xarray.DataArray 'radiance_unit_conversion_coefficient' ()> Size: 4B
array(0.08997274, dtype=float32)
Attributes:
    _FillValue:  9.96921e+36
    long_name:   Conversion coefficients to convert radiance units from mW.m^...

Actual results
Instead, when loading in all 41 chunks (or 40, if not including the TRAIL file), I get this error:

from glob import glob

from satpy import Scene
from satpy.utils import debug_on

debug_on()

# Directory which contains all BODY chunks as well as the TRAIL chunk.
# 41 files in total.
all_paths = sorted(
    glob(
        "/home/erose/geoips/geoips_packages/test_data/test_data_geocolor/data/mtg/20240924.1500/*.nc"
    )
)

multi_file_scn = Scene(filenames=all_paths, reader="fci_l1c_nc")

# Key error will occur here
multi_file_scn.load(["vis_04"], calibration="radiance")
print(multi_file_scn["vis_04"].attrs["radiance_unit_conversion_coefficient"])

"""
Results in:
Traceback (most recent call last):
  File "/home/erose/geoips/geoips_packages/geoips/geoips/plugins/modules/readers/bug.py", line 32, in <module>
    print(multi_file_scn["vis_04"].attrs["radiance_unit_conversion_coefficient"])
KeyError: 'radiance_unit_conversion_coefficient'
"""

satpy debug output

[DEBUG: 2025-02-19 17:16:33 : satpy.readers.yaml_reader] Reading ('/home/erose/anaconda3/envs/geoips/lib/python3.10/site-packages/satpy/etc/readers/fci_l1c_nc.yaml',)
[DEBUG: 2025-02-19 17:16:33 : satpy.readers.yaml_reader] Assigning to fci_l1c_nc: ['/home/erose/geoips/geoips_packages/test_data/test_data_geocolor/data/mtg/20240924.1500/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20240924150623_IDPFI_OPE_20240924150317_20240924150406_N__C_0091_0017.nc', 
...
[DEBUG: 2025-02-19 17:16:33 : satpy.readers.fci_l1c_nc] Start: 2024-09-24 15:00:00
[DEBUG: 2025-02-19 17:16:33 : satpy.readers.fci_l1c_nc] End: 2024-09-24 15:10:00
[DEBUG: 2025-02-19 17:16:33 : satpy.readers.fci_l1c_nc] Reading: /home/erose/geoips/geoips_packages/test_data/test_data_geocolor/data/mtg/20240924.1500/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20240924150904_IDPFI_OPE_20240924150633_20240924150711_N__C_0091_0028.nc
[DEBUG: 2025-02-19 17:16:33 : satpy.readers.fci_l1c_nc] Start: 2024-09-24 15:00:00
[DEBUG: 2025-02-19 17:16:33 : satpy.readers.fci_l1c_nc] End: 2024-09-24 15:10:00
[DEBUG: 2025-02-19 17:16:33 : satpy.readers.fci_l1c_nc] Reading: /home/erose/geoips/geoips_packages/test_data/test_data_geocolor/data/mtg/20240924.1500/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20240924150817_IDPFI_OPE_20240924150545_20240924150614_N__C_0091_0025.nc
...
[DEBUG: 2025-02-19 17:16:36 : satpy.composites.config_loader] Looking for composites config file fci.yaml
[DEBUG: 2025-02-19 17:16:36 : satpy.composites.config_loader] Looking for composites config file visir.yaml
[DEBUG: 2025-02-19 17:16:36 : pyorbital.tlefile] Path to the Pyorbital configuration (where e.g. platforms.txt is found): /home/erose/anaconda3/envs/geoips/lib/python3.10/site-packages/pyorbital/etc
[DEBUG: 2025-02-19 17:16:36 : satpy.readers.fci_l1c_nc] Reading vis_04 from /home/erose/geoips/geoips_packages/test_data/test_data_geocolor/data/mtg/20240924.1500/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20240924150410_IDPFI_OPE_20240924150007_20240924150023_N__C_0091_0001.nc
...
[DEBUG: 2025-02-19 17:16:37 : satpy.readers.fci_l1c_nc] Channel vis_04 resolution: 11136
[DEBUG: 2025-02-19 17:16:37 : satpy.readers.fci_l1c_nc] Row/Cols: 278 / 11136
[DEBUG: 2025-02-19 17:16:37 : satpy.readers.fci_l1c_nc] Calculated area extent: (-5567999.998550753, -5289999.998623111, 5567999.998550748, -5567999.998550753)
[DEBUG: 2025-02-19 17:16:37 : satpy.readers.fci_l1c_nc] Channel vis_04 resolution: 11136
[DEBUG: 2025-02-19 17:16:37 : satpy.readers.fci_l1c_nc] Row/Cols: 278 / 11136
[DEBUG: 2025-02-19 17:16:37 : satpy.readers.fci_l1c_nc] Calculated area extent: (-5567999.998550753, -5011999.9986954685, 5567999.998550748, -5289999.998623111)
...
[DEBUG: 2025-02-19 17:16:37 : satpy.readers.yaml_reader] Requested orientation for Dataset vis_04 is 'native' (default). No flipping is applied.
[DEBUG: 2025-02-19 17:16:37 : satpy.readers.fci_l1c_nc] Reading vis_04_pixel_quality from /home/erose/geoips/geoips_packages/test_data/test_data_geocolor/data/mtg/20240924.1500/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20240924150410_IDPFI_OPE_20240924150007_20240924150023_N__C_0091_0001.nc
[DEBUG: 2025-02-19 17:16:37 : satpy.readers.fci_l1c_nc] Reading vis_04_pixel_quality from /home/erose/geoips/geoips_packages/test_data/test_data_geocolor/data/mtg/20240924.1500/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20240924150420_IDPFI_OPE_20240924150007_20240924150036_N__C_0091_0002.nc
[DEBUG: 2025-02-19 17:16:37 : satpy.readers.fci_l1c_nc] Reading vis_04_pixel_quality from 
...
[DEBUG: 2025-02-19 17:16:37 : satpy.readers.yaml_reader] Requested orientation for Dataset None is 'native' (default). No flipping is applied.
Traceback (most recent call last):
  File "/home/erose/geoips/geoips_packages/geoips/geoips/plugins/modules/readers/bug.py", line 32, in <module>
    print(multi_file_scn["vis_04"].attrs["radiance_unit_conversion_coefficient"])
KeyError: 'radiance_unit_conversion_coefficient'

Environment Info:

  • OS: Linux
  • Satpy Version: 0.54.0
  • PyResample Version: 1.31.0
  • Readers and writers dependencies (when relevant): fci_l1c_nc satpy reader.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant