Add ability to specify separate directories for profile paths/years

[dostuffthatmatters]

Instead of storing all GGG2020 profiles (map/mod/vmr) in one place, we should make the paths configurable.

For example:

{
    "map_path_pattern": "/a/directory/$(YYYY)/$(YYYY)$(MM)$(DD)$(HH)_$(LAT)$(LON).map",
    "mod_path_pattern": "/b/directory/$(YYYY)/$(YYYY)$(MM)$(DD)$(HH)_$(LAT)$(LON).mod",
    "vmr_path_pattern": "/c/directory/$(YYYY)/$(YYYY)$(MM)$(DD)$(HH)_$(LAT)$(LON).vmr"
}

The downloader can put its files on these paths and the retrieval can use the map-file path for the retrieval. This way, people could reuse their own profile storing logic.

[dostuffthatmatters]

@cfleur maybe you are interested in doing this :D you mentioned that you have a setup that requires this. No urgency from our side.

[cfleur]

Hi, thanks for remembering. There is some limited utility in separating them and in that case the file names would also need to be configurable. At the moment we are not using the .mod or .vrm in this system. However, what would be more interesting for our system is more location-awareness. For example, if we could specify the .map path names for each location (in locations.json) and also download separately the locations into those folders. We are also wondering how you deal with custom profiles for nearby sites (i.e. locations that fall into the same grid square according to the rounding)? Do they all use the same profiles in your context?

[cfleur]

@dostuffthatmatters ^

[dostuffthatmatters]

Hi @cfleur,

sorry for the late reply - I was busy with other work.

Nearby sites: In sensors.json, you can use the atmospheric_profile_location_id parameter to change the location from which it is using the atmospheric profile. The priors only differ slightly from one grid cell to the next though.

Separate paths for locations: We could include a routine that writes these profiles also to location specific paths. However, since the pipeline uses the general paths to determine, which profiles it has already downloaded, we need to keep one general path.