Data table timing

[patricktokeeffe]

I recall there being consensus for:

• 10hz retention of flux sensor data
• 30-min output for calculated fluxes and averages of meteorological & soil data
• 15-min output for averages of select meteorology to report to LTAR common observatory
• 15-min output for soil data (averages?)
• 1-min output for select meteorology (averages? or maybe grab sample depending on sensor acquisition frequency?)

Current programs have:

• 10hz retention of flux sensor data ✔️
• 30-min output for calculated fluxes and averages of meteorological & soil data ✔️
• 15-min output for averages of select meteorology to report to LTAR common observatory ✔️
• 15-min output for averages of
  • vertical soil T/VWC profile
  • soil heat flux plate(s)
  • soil T/VWC probe for energy closure
  • soil averaging thermocouple
• 1-min output for averages of select meteorology to report to LTAR common observatory
• 1-min output for averages of
  • soil heat flux plate(s)
  • soil T/VWC probe for energy closure
  • soil averaging thermocouple

Comments in source code indicate the 1-min outputs were originally on a 5-min basis.

[patricktokeeffe]

Questions to resolve:

Any reason to revert from 1-min back to 5-min data outputs?

• is 1-min DAQ timing of soil probes (5TM/TDR-310) too fast? seems no
  • Cook West accumulated 3 skipped scans in ~70 days runtime
  • Cook East accumulated 10 skipped scans in ~6 mos runtime
  • Boyd North no skipped scans (just 2 soil probes, not 6)
• is 1-min data published or shared with anyone?
• is 1-min data being used internally?
• calculating wind-related (WS, WD, stdev of WD) each minute is expensive
  • preferable on 5-min basis?
  • not needed at all? since the component data is available at 10Hz in the timeseries file (windvector uses Uy/Ux sans coordinate system rotation)

Should 1-min data be grab sample instead of averages?

• significant reduction in processing requirements
• soil sensor data is acquired on 1-min basis anyway (i.e. averaging one data point) moved back into 5-sec slow sequence
• radiation type sensors probably benefit from averaging vs. grab sampling, but pressure? no; water vapor? doubtful; air temp? maybe

[patricktokeeffe]

Consensus among team members was 5-minutes is probably the best compromise. That's good: many 5-min statistics are already available from the stationarity test tables.

LTAR CORE reporting is no longer necessary so which columns to include is completely open-ended. Instead of picking individual columns to keep, copy the entirety of Flux and Flux_Notes, then remove all of the flux results and QC columns.