Skip to content

Select multiple STASH codes in PP/FF before cube creation #6398

@pp-mo

Description

@pp-mo

Very much the same as #6228 :
We would like to be able to load from PP/FF, specifying multiple STASH codes, which will filter out only the wanted fields prior to any costly cube conversion.
Because often we want to load from large dump files, containing lots and lots of data we are not interested in.

We already support a (virtually undocumented!) kind of "shortcut" whereby a single AttributeConstraint with the name "STASH" gets converted to a filter on the PP fields.
It happens here, here and here.

NOTE:
Loading of course accepts a list of input constraints, and individual constraints can also be combined using "&".
But unfortunately constraints also frequently use functions and lambdas to implement inclusive OR-type or value-range checking, which renders them opaque to inspection, so that a general inspection to deduce which STASHes are accepted is not possible.
( E.G.s
AttributeConstraint('m01s02i301') & Constraint(model_level=3) is currently not understood as possible to accelerate,
and anything with e.g. Constraint(forecast_time=lambda t: t < 10) would be an actual impossibility to decode
)

Proposal:

We can at least extend the current support to a list of AttributeConstraints, if all have the name "STASH",
and that will probably be good enough ??

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    No status

    Status

    📚 Backlog

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions