Organize data by variable rather than month?

[bpbond]

In the Theme 2 meeting today @bsulman said that it might be more intuitive/useful for his workflows to have data organized by variable rather than month.

Something to consider for version 2 perhaps

[bsulman]

I think this would be really helpful for data users, thanks!

[bpbond]

This approach would be consistent with e.g. CMOR used by earth system models.

[bpbond]

Current files in the GWI_2024 folder:

GWI_W_20240101-20240131_L1_v1-2.csv
GWI_W_20240201-20240228_L1_v1-2.csv
GWI_W_20240301-20240331_L1_v1-2.csv
GWI_W_20240401-20240430_L1_v1-2.csv
GWI_W_20240501-20240531_L1_v1-2.csv
GWI_W_20240601-20240630_L1_v1-2.csv
GWI_W_20240701-20240731_L1_v1-2.csv
GWI_W_20240801-20240831_L1_v1-2.csv
GWI_W_20240901-20240930_L1_v1-2.csv
GWI_W_20241001-20241031_L1_v1-2.csv
GWI_W_20241101-20241130_L1_v1-2.csv
GWI_W_20241201-20241231_L1_v1-2.csv

An organize-by-variable approach would look something like this:

GWI_W_battery_voltage_20240101-20241231_L1_v1-2.csv
GWI_W_gw_act_cond_20240101-20241231_L1_v1-2.csv
GWI_W_gw_battery_20240101-20241231_L1_v1-2.csv
GWI_W_solar_voltage_20240101-20241231_L1_v1-2.csv

(etc.)

Right away can see one issue: if we're using "_" as a filename delimiter, then the research_name labels can't use that character.

[bpbond]

@stephpenn1

What do you think about polling the COMPASS-FME team on this, and/or discussing at one of the Theme meetings? I very much see @bsulman 's point, but this change will break every single existing script, so we'd want to be sure that it's worth the cost.

[stephpenn1]

I think that's a great idea, I could mock up a survey? And yes we could ask people to fill out at the theme meetings

[bpbond]

That sounds like a good step. Thank you!