Skip to content

Add QC filters for radar reflectivity to align with GSI #522

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ShunLiu-NOAA merged 2 commits into from
Jan 14, 2026
Merged

Add QC filters for radar reflectivity to align with GSI #522

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
149c26b
update yaml with new filters
Jan 13, 2026
ae63d3a
update test reference
Jan 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 62 additions & 3 deletions parm/jcb-rdas/observations/atmosphere/mrms_refl.yaml.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,28 @@
action:
name: {{iuse_mrms_refl | default("passivate", true)}} # accept, reject, passivate

# Compute height AGL for height check filter
- filter: Variable Assignment
udescriptor: "calc_height_agl"
assignments:
- name: DerivedMetaData/height_agl
type: float
function:
name: ObsFunction/Arithmetic
options:
variables: [MetaData/height, GeoVaLs/height_above_mean_sea_level_at_surface]
coefs: [1.0, -1.0]

# Reject obs above 10 km AGL
- filter: Domain Check
udescriptor: height_check
filter variables:
- name: equivalentReflectivityFactor
where:
- variable: DerivedMetaData/height_agl
maxvalue: 10000.0
defer to post: true

# Initial error assignment
- filter: Perform Action
udescriptor: "intial_error_assignment"
Expand All @@ -51,13 +73,50 @@
name: assign error
error parameter: 5.0

# Gross Error Check
# Inflate obs error in precipitation
- filter: Perform Action
udescriptor: inflate_error_precip
apply at iterations: 0,1
filter variables:
- name: equivalentReflectivityFactor
where:
- variable: ObsValue/equivalentReflectivityFactor
minvalue: 5
action:
name: inflate error
inflation factor: 2.0

# First gross error check to inflate errors if ratio > 4
- filter: Background Check
udescriptor: gross_error_check_inflt
apply at iterations: 0,1
filter variables:
- name: equivalentReflectivityFactor
where:
- variable: ObsValue/equivalentReflectivityFactor
minvalue: 5
threshold: 4.0
action:
name: assign error
error function:
name: ObsFunction/Arithmetic
options:
variables: [ObsValue/equivalentReflectivityFactor, HofX/equivalentReflectivityFactor]
coefficients: [1.0, -1.0]
total coefficient: 0.25
absolute value: [true, true]
defer to post: true

# Second gross error check to reject anything with ratio > 8
- filter: Background Check
udescriptor: "gross_error_check"
udescriptor: gross_error_check_reject
apply at iterations: 0,1
filter variables:
- name: equivalentReflectivityFactor
threshold: 12.0
where:
- variable: ObsValue/equivalentReflectivityFactor
minvalue: 5
threshold: 8.0
action:
name: reject
defer to post: true
80 changes: 40 additions & 40 deletions rrfs-test/testoutput/rrfs-fv3jedi-3denvar_refl.ref
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,74 +1,74 @@
CostJo : Nonlinear Jo(refl10cm) = 2.9273997371874127e+05, nobs = 667006, Jo/n = 4.3888656731534836e-01, err = 5.0000000000000000e+00
CostFunction: Nonlinear J = 2.9273997371874127e+05
DRPCGMinimizer: reduction in residual norm = 6.3047084144204199e-03
CostJo : Nonlinear Jo(refl10cm) = 1.5717569957151997e+05, nobs = 637916, Jo/n = 2.4638933585537903e-01, err = 5.3721387247444952e+00
CostFunction: Nonlinear J = 1.5717569957151997e+05
DRPCGMinimizer: reduction in residual norm = 3.9716065974210657e-03
CostFunction::addIncrement: Analysis:
----------------------------------------------------------------------------------------------------
State print | number of fields = 29 | cube sphere face size: C420
eastward_wind | Min:-2.7829661342422241e+01 Max:+5.2821489801137972e+01 RMS:+1.3287579459489436e+01
northward_wind | Min:-3.9898378166296553e+01 Max:+4.1428335860306696e+01 RMS:+6.5163310783190420e+00
air_temperature | Min:+1.9402692974317924e+02 Max:+3.1449734497070312e+02 RMS:+2.5824326001428659e+02
eastward_wind | Min:-2.9110587622043379e+01 Max:+5.2821489681222467e+01 RMS:+1.3296416832611769e+01
northward_wind | Min:-3.9898376882288126e+01 Max:+3.8638981467820635e+01 RMS:+6.4777939430078781e+00
air_temperature | Min:+1.9402678104644190e+02 Max:+3.1449734497070312e+02 RMS:+2.5824095019401915e+02
air_pressure_thickness | Min:+1.4039907836914062e+02 Max:+3.3267851562500000e+03 RMS:+1.7747128794076129e+03
water_vapor_mixing_ratio_wrt_moist_air | Min:+0.0000000000000000e+00 Max:+2.9584785756597946e-02 RMS:+5.5262049273146381e-03
cloud_liquid_ice | Min:+0.0000000000000000e+00 Max:+4.0369947823390873e-04 RMS:+4.5535494026725958e-06
cloud_liquid_water | Min:+0.0000000000000000e+00 Max:+1.6822066691003948e-03 RMS:+1.9629626175813760e-05
ozone_mass_mixing_ratio | Min:+0.0000000000000000e+00 Max:+1.6040661648730747e-05 RMS:+4.4486157793992573e-06
geopotential_height_times_gravity_at_surface | Min:+0.0000000000000000e+00 Max:+3.5301851562500000e+04 RMS:+7.8466628379223748e+03
water_vapor_mixing_ratio_wrt_moist_air | Min:+0.0000000000000000e+00 Max:+2.4484865780424268e-02 RMS:+5.5033830259319351e-03
cloud_liquid_ice | Min:+0.0000000000000000e+00 Max:+3.5684457302802004e-04 RMS:+3.2766098674811528e-06
cloud_liquid_water | Min:+0.0000000000000000e+00 Max:+1.5652595830563708e-03 RMS:+1.8242558879799437e-05
ozone_mass_mixing_ratio | Min:+0.0000000000000000e+00 Max:+1.6040661648730747e-05 RMS:+4.4486256747499933e-06
geopotential_height_times_gravity_at_surface | Min:+0.0000000000000000e+00 Max:+3.5301851562500000e+04 RMS:+7.8466628379223739e+03
f10m | Min:+1.0004938840866089e+00 Max:+1.0948654413223267e+00 RMS:+1.0188698647586811e+00
slmsk | Min:+0.0000000000000000e+00 Max:+2.0000000000000000e+00 RMS:+7.8767192390790086e-01
sheleg | Min:+0.0000000000000000e+00 Max:+1.6966506958007812e+01 RMS:+2.0734254733790997e-01
skin_temperature_at_surface | Min:+2.7119995117187500e+02 Max:+3.2296466064453125e+02 RMS:+2.9475363971841762e+02
skin_temperature_at_surface | Min:+2.7119995117187500e+02 Max:+3.2296466064453125e+02 RMS:+2.9475363971841756e+02
vtype | Min:+0.0000000000000000e+00 Max:+2.0000000000000000e+01 RMS:+7.1648734770793263e+00
stype | Min:+0.0000000000000000e+00 Max:+1.6000000000000000e+01 RMS:+5.0688774416549585e+00
vfrac | Min:+0.0000000000000000e+00 Max:+9.9000000953674316e-01 RMS:+4.4064937959870992e-01
soilt | Min:+2.7023922729492188e+02 Max:+3.2296466064453125e+02 RMS:+2.9475315924680575e+02
soilm | Min:+1.4970073476433754e-02 Max:+1.0000000000000000e+00 RMS:+6.3592927195821791e-01
eastward_wind_at_surface | Min:-1.2974018096923828e+01 Max:+1.2164360046386719e+01 RMS:+3.6171520010516645e+00
northward_wind_at_surface | Min:-1.3966897010803223e+01 Max:+1.2346966743469238e+01 RMS:+3.7014659136589296e+00
air_pressure_at_surface | Min:+6.6027499740600586e+04 Max:+1.0240637915590193e+05 RMS:+9.6261161888063216e+04
rain_water | Min:+0.0000000000000000e+00 Max:+1.7971144187022932e-03 RMS:+1.5920228759900871e-05
snow_water | Min:+0.0000000000000000e+00 Max:+4.3326309534649212e-03 RMS:+4.9592763863228261e-05
graupel | Min:+0.0000000000000000e+00 Max:+7.8382273660709259e-04 RMS:+3.4994877371972981e-06
northward_wind_at_surface | Min:-1.3966897010803223e+01 Max:+1.2346966743469238e+01 RMS:+3.7014659136589292e+00
air_pressure_at_surface | Min:+6.6027499740600586e+04 Max:+1.0240638052820925e+05 RMS:+9.6261324127937667e+04
rain_water | Min:+0.0000000000000000e+00 Max:+1.7955460166075763e-03 RMS:+1.5657599584847733e-05
snow_water | Min:+0.0000000000000000e+00 Max:+3.6321065836000424e-03 RMS:+4.5167899821397335e-05
graupel | Min:+0.0000000000000000e+00 Max:+6.3211027326170229e-04 RMS:+3.2846438800460970e-06
cloud_droplet_number_concentration | Min:+0.0000000000000000e+00 Max:+1.5055759360000000e+09 RMS:+2.0745188331041779e+07
cloud_ice_number_concentration | Min:+0.0000000000000000e+00 Max:+5.0166070000000000e+06 RMS:+2.8482317076120682e+04
rain_number_concentration | Min:+0.0000000000000000e+00 Max:+8.6749025000000000e+05 RMS:+2.1760084842806405e+03
upward_air_velocity | Min:-1.5786052468604346e+00 Max:+4.4702885756559114e+00 RMS:+6.0871089893464819e-02
equivalent_reflectivity_factor | Min:-3.5493583193441395e+01 Max:+8.5611056808349289e+01 RMS:+3.4509237200077680e+00
upward_air_velocity | Min:-1.9158170026131696e+00 Max:+3.0514554006384120e+00 RMS:+5.8370851896598054e-02
equivalent_reflectivity_factor | Min:-3.3405663019368923e+01 Max:+6.5427631659966877e+01 RMS:+3.2426146569047556e+00
----------------------------------------------------------------------------------------------------
CostJo : Nonlinear Jo(refl10cm) = 134772.2688587668, nobs = 667006, Jo/n = 0.2020555570096322, err = 5
CostFunction: Nonlinear J = 134772.2688587668
DRPCGMinimizer: reduction in residual norm = 0.03102133137879767
CostJo : Nonlinear Jo(refl10cm) = 43468.34948658482, nobs = 637916, Jo/n = 0.06814118079274516, err = 6.654007668620585
CostFunction: Nonlinear J = 43468.34948658482
DRPCGMinimizer: reduction in residual norm = 0.01633878088943338
CostFunction::addIncrement: Analysis:
----------------------------------------------------------------------------------------------------
State print | number of fields = 29 | cube sphere face size: C420
eastward_wind | Min:-2.7904081785863674e+01 Max:+5.2821489451257989e+01 RMS:+1.3287916035900247e+01
northward_wind | Min:-3.9898378192810306e+01 Max:+4.1523339793126397e+01 RMS:+6.5160333931028660e+00
air_temperature | Min:+1.9402702879440091e+02 Max:+3.1449734497070312e+02 RMS:+2.5824325770165780e+02
eastward_wind | Min:-2.8211561608417494e+01 Max:+5.2821489436449653e+01 RMS:+1.3305047892120744e+01
northward_wind | Min:-3.9898376541384323e+01 Max:+3.8606808497898051e+01 RMS:+6.4546357955858680e+00
air_temperature | Min:+1.9402681606924833e+02 Max:+3.1449734497070312e+02 RMS:+2.5824087870881203e+02
air_pressure_thickness | Min:+1.4039907836914062e+02 Max:+3.3267851562500000e+03 RMS:+1.7747128794076129e+03
water_vapor_mixing_ratio_wrt_moist_air | Min:+0.0000000000000000e+00 Max:+2.9690678008349302e-02 RMS:+5.5259886838785544e-03
cloud_liquid_ice | Min:+0.0000000000000000e+00 Max:+4.0580096676382242e-04 RMS:+4.5557317656810206e-06
cloud_liquid_water | Min:+0.0000000000000000e+00 Max:+1.6690937107325722e-03 RMS:+1.9611795658726312e-05
ozone_mass_mixing_ratio | Min:+0.0000000000000000e+00 Max:+1.6040661648730747e-05 RMS:+4.4486157945571414e-06
geopotential_height_times_gravity_at_surface | Min:+0.0000000000000000e+00 Max:+3.5301851562500000e+04 RMS:+7.8466628379223748e+03
water_vapor_mixing_ratio_wrt_moist_air | Min:+0.0000000000000000e+00 Max:+2.4484865792909531e-02 RMS:+5.4884439277352922e-03
cloud_liquid_ice | Min:+0.0000000000000000e+00 Max:+3.5538980246518703e-04 RMS:+2.6393564073903285e-06
cloud_liquid_water | Min:+0.0000000000000000e+00 Max:+1.4715781642883181e-03 RMS:+1.7773528815269722e-05
ozone_mass_mixing_ratio | Min:+4.6755048878328607e-09 Max:+1.6040661648730747e-05 RMS:+4.4486302871500302e-06
geopotential_height_times_gravity_at_surface | Min:+0.0000000000000000e+00 Max:+3.5301851562500000e+04 RMS:+7.8466628379223739e+03
f10m | Min:+1.0004938840866089e+00 Max:+1.0948654413223267e+00 RMS:+1.0188698647586811e+00
slmsk | Min:+0.0000000000000000e+00 Max:+2.0000000000000000e+00 RMS:+7.8767192390790086e-01
sheleg | Min:+0.0000000000000000e+00 Max:+1.6966506958007812e+01 RMS:+2.0734254733790997e-01
skin_temperature_at_surface | Min:+2.7119995117187500e+02 Max:+3.2296466064453125e+02 RMS:+2.9475363971841762e+02
skin_temperature_at_surface | Min:+2.7119995117187500e+02 Max:+3.2296466064453125e+02 RMS:+2.9475363971841756e+02
vtype | Min:+0.0000000000000000e+00 Max:+2.0000000000000000e+01 RMS:+7.1648734770793263e+00
stype | Min:+0.0000000000000000e+00 Max:+1.6000000000000000e+01 RMS:+5.0688774416549585e+00
vfrac | Min:+0.0000000000000000e+00 Max:+9.9000000953674316e-01 RMS:+4.4064937959870992e-01
soilt | Min:+2.7023922729492188e+02 Max:+3.2296466064453125e+02 RMS:+2.9475315924680575e+02
soilm | Min:+1.4970073476433754e-02 Max:+1.0000000000000000e+00 RMS:+6.3592927195821791e-01
eastward_wind_at_surface | Min:-1.2974018096923828e+01 Max:+1.2164360046386719e+01 RMS:+3.6171520010516645e+00
northward_wind_at_surface | Min:-1.3966897010803223e+01 Max:+1.2346966743469238e+01 RMS:+3.7014659136589296e+00
air_pressure_at_surface | Min:+6.6027499740600586e+04 Max:+1.0240638389294178e+05 RMS:+9.6261177205228218e+04
rain_water | Min:+0.0000000000000000e+00 Max:+1.7973675668299745e-03 RMS:+1.5917643184537145e-05
snow_water | Min:+0.0000000000000000e+00 Max:+4.2809528540932174e-03 RMS:+4.9531004691612784e-05
graupel | Min:+0.0000000000000000e+00 Max:+7.6621973153764779e-04 RMS:+3.4934837961576292e-06
northward_wind_at_surface | Min:-1.3966897010803223e+01 Max:+1.2346966743469238e+01 RMS:+3.7014659136589292e+00
air_pressure_at_surface | Min:+6.6027499740600586e+04 Max:+1.0240638393522671e+05 RMS:+9.6261458572998337e+04
rain_water | Min:+0.0000000000000000e+00 Max:+1.7951889540823874e-03 RMS:+1.5454287468617475e-05
snow_water | Min:+0.0000000000000000e+00 Max:+3.5704335337893062e-03 RMS:+4.2056141065637780e-05
graupel | Min:+0.0000000000000000e+00 Max:+6.3210312860550999e-04 RMS:+3.0982976357908518e-06
cloud_droplet_number_concentration | Min:+0.0000000000000000e+00 Max:+1.5055759360000000e+09 RMS:+2.0745188331041779e+07
cloud_ice_number_concentration | Min:+0.0000000000000000e+00 Max:+5.0166070000000000e+06 RMS:+2.8482317076120682e+04
rain_number_concentration | Min:+0.0000000000000000e+00 Max:+8.6749025000000000e+05 RMS:+2.1760084842806405e+03
upward_air_velocity | Min:-1.5745316985875273e+00 Max:+4.4207389044743106e+00 RMS:+6.0848807854868021e-02
equivalent_reflectivity_factor | Min:-3.5099878581398492e+01 Max:+8.5621465029616374e+01 RMS:+3.4493469711820945e+00
upward_air_velocity | Min:-2.0991565619989516e+00 Max:+2.1992036736105116e+00 RMS:+5.7146473602936988e-02
equivalent_reflectivity_factor | Min:-3.3477985224035983e+01 Max:+5.3437831664278846e+01 RMS:+3.0663976794785244e+00
----------------------------------------------------------------------------------------------------
CostJo : Nonlinear Jo(refl10cm) = 134632.1515037495, nobs = 667006, Jo/n = 0.201845487902282, err = 5
CostFunction: Nonlinear J = 134632.1515037495
CostJo : Nonlinear Jo(refl10cm) = 43213.5821258342, nobs = 637916, Jo/n = 0.06774180632847303, err = 6.654007668620585
CostFunction: Nonlinear J = 43213.5821258342