Updating the RDASApp ctests (adding new obs or updating test reference files)

RDASApp uses a method for merging individual observation-space yamls into one "super yaml" for its ctests. These super yamls are created from individual templates when RDASApp is built. If you have validated a new observation type and wish to add it to the RDASApp ctests, you will need to make a few changes in addition to adding the yaml for the new observation type. Please see the detailed list below for additional developments that need to be made during your PR.

NOTE: if you are just updating the observation-space yamls (i.e., not adding a new observation type), please skip to step #4.

1. Stage the new observation data under RDAS_DATA

This step can be done by contacting someone on the fix file management team (Guoqing Ge, Samuel Degelia, or Donald Lippi). You can also include the path to this data in your PR so that we can update the staged data on each machine. Please follow a similar naming convention to other observations: ioda_${instrument}.nc.

2. Add the new observation-space yaml file under rrfs-test/validated_yamls/templates/obtype_config

Please just include the lines of the yaml corresponding to the new observation you are adding (i.e., start the yaml with the line - obs space:). Also note that additional sections need to be included in the yaml so that it can be used for both Var-based DA or EnKF. These include the distribution and obs localizations headers. Please see the example yamls already included in the rrfs-test/validated_yamls/templates/obtype_config directory.

Additionally, please do not include the distribution name directly in the yaml. Instead, please just use the following example below which allows the @DISTRIBUTION@ string to be replaced when creating the super yaml.

         distribution:
           name: "@DISTRIBUTION@"
           halo size: 300e3

Finally, we note that the name of the observation should follow a standardized form of obsfile: "data/obs/ioda_${obtype}.nc" where ${obtype} is the name of the instrument (e.g., msonet, abi_g16, etc.). Please see the other yamls for examples. Also please include the double quotes around the file name. These settings are important for ensuring that any file name replacements are executed correctly (such as for the GETKF solver).

3. Add the new observation type to the merging script in rrfs-test/validated_yamls/gen_yaml_ctest.sh

This script controls the observation-space yamls that are merged together for the ctests when RDASApp is built. Please append your new yaml under the obtype_configs list at the top of this shell script.

4. Run the ctests and update the test reference data

Normally, the super yamls are only generated when RDASApp is built. However, the super yamls can also be regenerated using tools included in RDASApp/ush (note: NOT within rrfs-test). Once you have added your yaml to gen_yaml_ctest.sh, you can update the input files for the ctests through:

cd ush
./update_ctest_inputs.sh

This script will regenerate the super yamls and copy them into the ctest run directories. It will also link any new observations from Step 1 into the ctest run directories. Next, you can run the ctests from the same directory using:

./run_rrfs_tests.sh ${ACCOUNT} # account is the SLURM_ACCOUNT you wish to use (e.g., fv3-cam, wrfruc, etc.)

Since you just updated the test input files, we expect the ctests to fail (unless the changes are minor). Please refer to build/rrfs-test/Testing/Temporary/LastTest.log to confirm that the failures are for expected reasons (such as a ReferenceMismatchError). You will next need to update the test reference files given the changes to the ctest. You can use the following script from the same ush/ directory to do this:

./update_ctest_refs.sh

This script will copy the test output files from the ctest run directories into rrfs-test/testoutput.

Finally, please run the ctests again to ensure that they pass. If the tests pass, then please commit the updates to the test reference files in your PR (the super yamls in rrfs-test/testinput do not need to be committed).