Skip to content

Update CDF metadata with feedback from Andriy for ISTP and SPASE compliance #155

@alastairtree

Description

@alastairtree

Thanks for sending the file and sorry for delayed response. Here are the comments I have (they will probably similarly apply to the other datasets you are creating). Also, I attached .skt file (metadata extracted from the CDF file) and .gif with plotted variables ("vectors" variable only since other variables, including "magnitude", are defined with VAR_TYPE="support_data", see comments on this below).

ISTP Variables and Attributes:

(1) For "epoch" (time) variables, we ask not to compress them. We do recommend compression for other variables.

(2) For "vectors" variable, but similar for other variables, VALIDMIN and VALIDMAX are currently set as min/max values of that variable's data type (CDF_DOUBLE in case of "vectors"). The VALIDMIN/VALIDMAX should instead be set to min/max values possible over the time of whole mission (or, at least, to the range the instrument is capable of measuring). They are used to filtering outliers not marked by FILLVAL value. And for FILLVAL (also for this variable and others), most importantly, the value has to be outside of VALIDMIN/VALIDMAX range (a value cannot be both valid and fill value), and then we also have recommended FILLVAL values depending on variable data type if you prefer (https://spdf.gsfc.nasa.gov/istp_guide/vattributes.html#FILLVAL). Also, UNITS should be "nT".

Also, for "vectors" variable, you don't need DEPEND_1 attribute since it plotted as 1-D time_series (and labels are provided in metadata variable "direction_label" pointed to by LABL_PTR_1), and so "direction" variable is also not needed. You can see example of vector magnetic field variable definition here https://spdf.gsfc.nasa.gov/istp_guide/variables.html#data_eg2 .

And for the metadata variable "direction_label", it should hold as its value the array of strings that will be displayed on as y-axis labels for three components in the time series plot, so should be like ["Bx DSRF", "By DSRF", "Bz DSRF"] instead of current ["0", "1", "2"], otherwise the labels look as in the attached file. FORMAT value will also need to be changed to "A7" (maximum length of those strings). Here is an example of definition https://spdf.gsfc.nasa.gov/istp_guide/variables.html#Metadata_eg1

(3) For "quality_flags" variable, since it is critical for users to see the quality flag value, VAR_TYPE should be set to "data" instead of "support_data" so that the variable is listed/plotted in CDAWeb and is also returned by different APIs. "support_data" are designed only for variables like time or dimensions, attached to "data" variable via "DEPEND_i", and also other variables that need to be saved in the file but not important for users (like possibly "range" variable in your file). Also, for users to immediately understand the meaning of the flag value, you should probably also add "0: Good data, 1: Bad data" in the end of CATDESC, so that CATDESC = "Data quality flag. 0: Good data, 1: Bad data". CATDESC is what is shown in the list of variables in CDAWeb. Also, since in "quality_flags" you store integers, an integer data type for that variable would be more appropriate.

And the variable data type is especially important for "quality_bitmask", where you actually describe bitwise flags. Currently this variable is of CDF_DOUBLE data type and should be integer instead. If you change variable data type, please remember that VALIDMIN, VALIDMAX, FILLVAL data types have to match variable data type, and "FORMAT" should also correspond to the variable data type and cover VALIDMIN/VALIDMAX range. Also, you need to decide if "quality_bitmask" is important for user, and if so, VAR_TYPE should be changed to "data" for that variable to be visible in the list of variables in CDAWeb and returned by APIs. Also, it would be helpful to clarify in VAR_NOTES (and possibly even in CATDESC if you decide to make that variable VAR_TYPE="data") how the two quality flag variables are related, and if user only need to check "quality_flags" or both before using the data.

(4) For "range" variable, since the value is integer, an integer data type would be more appropriate. And if you make the change, again, please remember that VALIDMIN, VALIDMAX, FILLVAL attribute data types have to match variable data type.

(5) For "magnitude" variable, VAR_TYPE should be "data" so that it is listed/plotted.

SPASE metadata:

(6) For the variable SPASE metadata (in DICT_KEY ISTP attribute) for "vectors" in DSRF, since we need to use CoordinateSystemName value from the controlled list (https://spase-group.org/data/model/spase-latest/spase-latest_xsd.htm#CoordinateSystemName), if at all possible, I would suggest "SC" (since its definition is general and assumes both spinning and despun cases). There is also better "SR2" which is explicitly despun, but I am not sure how the other axes in IMAP despun frame are defined, so it may be different from "SR2" description, but "SC" should still work. And you have the actual description in CATDESC, so that you are not losing information about DSRF.

(7) For "magnitude" variable, DICT_KEY should be "SPASE>Field>FieldQuantity:Magnetic,Qualifier:Magnitude"

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions