RC for JASMES Snow Cover (GHR, M5C, SVC)

[wakuphas]

---

1. source_id['source_id'][key]['source_name'] = 'JASMES Snow Cover'
2. source_id['source_id'][key]['release_year'] = '2024'
3. source_id['source_id'][key]['source_description'] = 'Global daily surface snow area fraction derived from merged satellite observations using a consistent objective snow cover mapping algorithm applied to multiple generations of optical sensors since 1978.'
4. source_id['source_id'][key]['source_version_number'] = '1'
5. source_id['source_id'][key]['institution_id'] = 'JAXA'
6. source_id['source_id'][key]['region'] = 'global'
7. source_id['source_id'][key]['source_type'] = 'satellite_merged'
8. CMIP variable_ids: snc

---

[gleckler1]

@wakuphas The "source_name" needs to be an acronym or abbreviated string without spaces (that is already or) you want to be used along with the "source_version_number" as the common identifier used for the dataset. For example, "JAXA-SNOW" would be a suitable "source_name", but maybe you would prefer something else? With this example, when we combine it with the "source_version_number", the offical "source_id" would be "JAXA-SNOW-1. Please let us know if you are satisfied with this suggestion or if you would prefer another name.

[Darwyn72]

@wakuphas and @gleckler1 I think it would be better to use 'JASMES' as the source_name - in the original proposal to obs4MIPs it had the acronym of 'JASMES 1.9 so maybe JASMES-1-0 if it's just version 1.0 of the data?

[wakuphas]

Dear @gleckler1 and @Darwyn72 ,

Thank you very much for the helpful discussion.

Looking ahead, JASMES is a framework that includes multiple geophysical products beyond snow cover. In order to allow independent versioning and updates at the product level, I would like to propose defining separate source_name entries for each JASMES product.

For the snow cover dataset, this would mean using:

source_name: JASMES-SNOWCOVER
source_id: JASMES-SNOWCOVER-1-0

This approach would allow future updates to the snow cover product without affecting other JASMES products (in the future) that may be registered separately.

Please let me know if this understanding is correct, or if you would recommend any adjustments.

[Darwyn72]

@wakuphas I think this is a good approach - I'd recommend amending the original RC issue above

[gleckler1]

@wakuphas Your proposed source_id is now included in obs4MIPs. Please don't hesitate to ask if you have any questions about next steps.

[wakuphas]

Thank you. I will proceed to the next steps.

[wakuphas]

Dear @gleckler1 and @Darwyn72,

I would like to request a pre-review of our obs4MIPs CMORization setup
for the JASMES Snow Cover product before producing the full dataset.

This dataset provides a global daily surface snow area fraction (snc)
over land only, derived from merged satellite observations since 1978.

Key characteristics:

• Variable: snc (surface_snow_area_fraction)
• Realm: land (sea ice excluded)
• Grid: global regular lat/lon, 0.05°
• Frequency: daily
• Time coverage: 197812–202512
• Source: JASMES Snow Cover (multi-sensor)

Sensor usage (described in metadata as input_sensor_provenance):

• GHR: AVHRR onboard NOAA series satellites (1978–2000)
• M5C: MODIS onboard Terra and Aqua (2001–2018)
• SVC: VIIRS onboard Suomi NPP and JPSS-1, and SGLI onboard GCOM-C (2019–2025)

We have prepared:

• The draft obs4MIPs JSON file
  obs4MIPs_JASMES_SNOWCOVER.json
• The CMORization Python code
  CMOR_JASMES_SNOWCOVER.py
• A small set of sample CMORized NetCDF files
  (one representative day for each sensor period: 3 files)
  https://app.box.com/s/fjzcsqm1wqqfvy405vklm0vgy0njnurs

We would appreciate your feedback on:

• The variable definition and unit choice (%)
• The realm and exclusion of sea ice
• Provenance and metadata description
• Any other issues to address before full-scale production

Thank you very much for your guidance.

[gleckler1]

@wakuphas Excellent, thank you. You should have received an invite from GitHub to become a collaborator on the obs4MIPs-cmor-tables repo. Once you have accepted the invitation, you should be able to push a branch to this repository. To do that, can you create a branch named something like 605_WAK_JASMESsnow, and put obs4MIPs_JASMES_SNOWCOVER.json and CMOR_JASMES_SNOWCOVER.py in a new directory obs4MIPs-cmor-tables/inputs/JAXA/SNOWCOVER? You should then be able to push this branch. Doing this will help us to more easily collaborate on this and any other products you may prepare, and you will become identified as an official collaborator.

[gleckler1]

@wakuphas I forgot to mention that the starting point for the above comment is for you to clone this repo, but presumably you have already done that to prepare your codes. If you have difficulties with the above I could create the branch with the .json and .py that you have provided, although this won't facilitate our working together (if needed) to finalize your codes.

[wakuphas]

Thank you for your guidance.
I created a new branch named "605_WAK_JASMESsnow", and put obs4MIPs_JASMES_SNOWCOVER.json and CMOR_JASMES_SNOWCOVER.py in a new directory obs4MIPs-cmor-tables/inputs/JAXA/JASMES-SNOWCOVER-1-0/.

https://github.com/PCMDI/obs4MIPs-cmor-tables/tree/605_WAK_JASMESsnow/inputs/JAXA/JASMES-SNOWCOVER-1-0

I would appreciate your feedback.

[gleckler1]

@wakuphas @Darwyn72 Many thanks for your efforts preparing the branch. There are a few minor things to consider, but I have already merged your branch with master as a placeholder. You can still work with the same branch but do merge with master before making changes. Here are a few things to consider...

The sample data you have prepared appears to have runoff errors in a few longitudes, latitudes and their associated bounds. For example,

Latitudes: -10.45, -10.4, -10.35, -10.3, -10.25, -10.2, -10.15, -10.1, -10.05, -10,
-9.95, -9.90000000000001, -9.85000000000001, -9.80000000000001, -9.75,

This is the first snow cover dataset in obs4MIPs, and although we are planning to migrate coordination with CMIP7 when it is ready, we are still linked to CMIP6. In CMIP6, "snc" is located in a LImon table. Tomorrow I will try to add a LImon table to obs4MIPs-cmor-tables and notify you when it is there so that you can try it.

It looks like "processing_code_location" did not work. I suspect this may be because you first need to commit your code and then push it. Maybe you can try that before your next test.

Otherwise, I think you have done a fine job dealing with the many details of preparing the data.

[wakuphas]

Dear @gleckler1,

Thank you for your sggestion.
Now I have revised floating-point precision issues.
However, during cmor.write(), CMOR reports the following error:

! Error: The attribute "realm" could not be validated.
! The current input value is "landIce land", which is not valid.
!
! Valid values must match those found in the "realm" section
! of your Controlled Vocabulary (CV) file "../../../Tables/obs4MIPs_CV.json".

What I have checked so far:

• In obs4MIPs_CV.json, the allowed realm values are:
  ['aerosol', 'atmos', 'atmosChem', 'land', 'landIce', 'ocean', 'ocnBgchem', 'seaIce']
• However, CMOR reports the invalid value as "landIce land"
• This suggests that the value may be coming from the table definition (modeling_realm) rather than the dataset JSON.

As an additional check, I found that if I locally modify the obs4MIPs_Aday.json table and change the snc entry from:
"modeling_realm": "landIce land"
to:
"modeling_realm": "land"
the CMOR run completes successfully.

I appreciate your advice on this issue.

[gleckler1]

@wakuphas Nice investigation! Using a modified modeling_realm should work ok. I have updated the obs4MIPs_Aday.json file on the master branch.

[wakuphas]

@gleckler1 Thank you for the update.

I have pushed the latest CMORization setup to my branch (minor changes):

• inputs/JAXA/JASMES-SNOWCOVER-1-0/CMOR_JASMES_SNOWCOVER.py
• inputs/JAXA/JASMES-SNOWCOVER-1-0/obs4MIPs_JASMES_SNOWCOVER.json

In addition, I created a small set of sample CMORized NetCDF files (3 files; one representative day for each sensor period) for quick inspection:
https://app.box.com/s/ilp7xh3217bpkm1b3ippc3yqeav3lmnm

Could you please take a look and let me know if everything looks OK (metadata/attributes, coordinates, variable naming, etc.)?
Once I get your confirmation, I will proceed to CMORize the full dataset.

[gleckler1]

@wakuphas @Darwyn72 Your updates and sample data look good. There are no subdirectories, but the output_path_template in your input.json looks fine so I presume the directory structure is also.
One thought... your files, including 1day/file, are very small, ~3Mb. While ok, you might consider including storing 1yr/file, which is more typical of the way CMIP daily output and obs4MIPs daily date are provided.
Do you need assistance publishing this data to obs4MIPs on ESGF?