XIOS

XIOS = XML I/O Server

Developed at IPSL: http://forge.ipsl.jussieu.fr/ioserver/wiki

Hello World example

<xios>
    <context id="hello_word" >

        <!-- Define a 1D axis and horizontal 2D-domain associated with the model variables -->
        <axis_definition>
            <axis id="vertical_axis" n_glo="70" />
        </axis_definition>

        <domain_definition>
            <domain id="horizontal_domain" ni_glo="144" nj_glo="90" />
        </domain_definition>

        <!-- Define grid associated with the associated axis and domains -->
        <grid_definition>
            <grid id="grid_3d">
            <domain domain_ref="horizontal_domain">
            <axis axis_ref="vertical_axis">
        </grid_definition>

        <!-- Define fields (variables) associated with the grid -->
        <field_definition >
            <field id="a_field" operation="average" grid_ref="grid_3d" />
        </field_definition>

        <!-- Associate fields with files -->
        <file_definition type="one_file" output_freq="1d" enabled=".TRUE.">
            <file id="output" name="hello_world" output_freq="1d">
                <field field_ref="a_field" />
            </file>
        </file_definition>
    </context>
</xios>

XML: Extensible Markup Language

• Set of rules to define a documents in a format
• Both human-readable and machine readable

Tag: a markup construct that’s begin by "<" and ends by ">"

• Start-tag: <field>
• End-tag: </field>

Element: construct delimited by a start-tag and an end-tag

• May be empty: <field></field>
• Can be written with empty-tag syntax:
• May have child elements

<field_group>
<field />
<field />
</field_group>

• May have content: text between start-tag and end-tag element <field> content </field>
• Only use in XIOS to define arithmetic operations

Attributes

• Construct composed of a pair : name="value" defined into a start-tag or an empty tag element
• Example: Element field has 3 attributes : id, name and unit

<field id="temp" name="temperature" unit="K" />

Comments: delimited by starting tag comment <!-- and ending tag comment -->

<field> <!-- this is a comment, not a child nor a content --> </field>

XML document must be well-formed

• XML document must contains only one root element
• All start-tag element must have the matching end-tag element (case sensitive) and reciprocally
• All element must be correctly nested

XML master file must be iodef.xml

• Parsed first at XIOS initialization
• Root element name is free
• Root element can only contain type elements.

Main element families: represent objects type stored into XIOS database

• context: isolate and confine models definition, avoiding interference between them
• field: define incoming field from model
• axis: define 1D-axis
• domain: define 2D-domain
• grid: define multi-dimensional grid, composed of axis and domains
• file: define input or output file
• variable : define parameters for models or for XIOS parameterization

These families have 3 flavors (except for context elements)

• Simple elements: e.g.<field />
• Group elements: e.g. <field_group />
  • Can contain children simple element
  • Can contain children nested group of the same type
• Definition elements: e.g. <field_definition>
  • Unique root element type
  • Act as a group element, i.e. can contain other groups or simple elements

Each element may have several attributes

• For example: <file id="out" name="output" output_freq="1d" />
• Attributes give information for the related element
• Some attributes are mandatory, so error is generated without assigned value (small part)
• Some other are optional but have a default value
• Some other are completely optional

Attributes values are ASCII string and, depending of the attribute, can represent :

• A character string: e.g. name="temperature"
• An integer or floating value: e.g. output_level="3" add_offset="273.15"
• A boolean: true/false: e.g. enabled="true"
  • Fortran notations .TRUE./.FALSE. are allowed but obsolete
• A date or duration: e.g. start_date="2000-01-01 12:00:00"
  • See format later
• A bound array (inf,sup)[values]: e.g. value="(0,11) [1 2 3 4 5 6 7 8 9 10 11 12]"

XML instructions can be split into several parts

Example:

• File iodef.xml:

<context src="./nemo_def.xml" />

• File nemo_def.xml:

<context id="nemo" >
<field_definition ...>
...
</context>

Calendar

Each context must define an associate calendar

• One calendar by context
• Define a calendar type
  • Date and duration operation are defined with respect to the calendar
• Define starting date of the model
• Define time step of the model

Calendar type

• Gregorian : standard Gregorian calendar
• D360 : fixed 360 days calendar
• NoLeap : fixed 365 days calendar
• AllLeap : fixed 366 days calendar
• Julian : Julian calendar
• user_defined : months and days can be defined by user (planetology and paleoclimate)

Date and Duration

• A lot of XML attributes are of date or duration type
• Operation between date and duration are strongly dependent of the chosen calendar
  • For example, date + 1 month is equal to date + 30 day only for April, June, September & November

Duration units

• Year : y
• Month : mo
• Day : d
• Hour : h
• Minute : mi
• Second : s
• Time step : ts (related to time step context definition)

Duration format

• Value of unit may be integer or floating (not recommended), mixed unit may be used in a duration definition
  • e.g. "1mo2d1.5h30s"
  • e.g. "5ts"

Date format

• year-month-day hour:min:sec
  • e.g. "2015-12-15 10:15:00"
• Partial definition are allowed taking taking into account rightmost part
  • "2015-12" equivalent to "2015-12-01 00:00:00"
• Date can be also defined with an offset
  • Useful for defining a calendar based on standard units (seconds for example)
  • e.g. "+3600s"
  • e.g. "2012-15 +3600s" equivalent to "2012-15-01 01:00:00"

Specific attribute calendar

• type : define the calendar type, e.g. "Gregorian", "D360", etc.
• (date) time_origin : define the simulation starting date (fixed at "0000-01-01" by default)
• (date) start_date : define the starting date of the run (fixed at "0000-01-01" by default)
• (duration) timestep : define the time step of the model (mandatory)
• Example:

<context id="nemo">
    <calendar type="Gregorian" time_origin="2000-01-01" start_date="2015-12" timestep="1h"/>
    ...
</context>

User-defined calendar: idealised / exoplanet studies

• Defining day_length in seconds (default 86400)
• Defining month_lengths: number of days for each of 12 months
• Or if you don't want to specify months: year_lenght in seconds: months would be equally distributed

<calendar type="user_defined" day_length="86400"
month_lengths="(1,12) [31 28 31 30 31 30 31 31 30 31 30 31]" />

• Possibility to define leap year
  • Attributes: leap_year_month, leap_year_drift, leap_year_drift_offset

Grid definition

• Field geometry is provided by the underlying mesh description
• Describe field dimensionality (1D, 2D, 3D or more)
• Describe field topology (rectilinear, curvilinear, unstructured)
• Describe field data distribution for parallelism

The grid element:

• Can describe element of any dimension : 0D (scalar), 1D, 2D, 3D, or more.
• Defined by composition of 1D-axis and 2D-horizontal domain
  • axis and domain elements
  • Can be masked and subset
• Empty grid is representing a scalar

<grid_definition>
    <grid id="grid_3d"> <!-- grid 3D of dimension 100x50x20 -->
        <axis n_glo="100"/>
        <axis n_glo="50"/>
        <axis n_glo="20">
    </grid>
    
    <grid id="grid_4d"> <!-- grid 4D of dimension 100x100x50x20 -->
        <domain ni_glo="100" nj_glo="100" type="rectilinear"/>
        <axis n_glo="50"/>
        <axis n_glo="20">
    </grid >
</grid_definition>

Fields

• The <field \> element: represents input/output data flux
• Fields geometry and parallel distribution is defined by the underlying grid description
  • (string) grid_ref attribute: id of the grid
• For more flexibility fields can refer to a domain
  • (string) domain_ref attributes: a virtual 2D grid composed of the referred domain
• Or a domain and an axis to create a virtual 3D grid
  • domain_ref and axis_ref

<grid id="grid_3d">
<domain id="domain_2d" />
<axis id="axis_1d" />
</grid>
...
<field id="temp" grid_ref="grid_3d"/>

is equivalent to

<axis id="axis_1d" />
<domain id="domain_2d" />
...
<field id="temp" domain_ref="domain_2d" axis_ref="axis_1d/>

Output to files

• Will appear as a child file element
• Construct files by including list of field required
• A field can appear, in more than one file
  • using the reference attribute : field_ref

<field_definition>
    <field id="temp" grid_ref="grid_3d"/>
    <field id="precip" grid_ref="domain_2d"/>
    <field id="pressure" domain_ref="grid_3d"/>
</field_definition>

<file_definition>
    <file name="daily_output" freq_output="1d">
    <field field_ref="temp" />
    <field field_ref="pressure" />
</file>

<file name="monthly_output" freq_output="1mo">
    <field field_ref="temp" />
    <field field_ref="precip" />
</file>
</file_definition>

Field attributes related to file output

• Field description :
  • (string) name : name of the field in the file. If not specified, "id" will be used in place
  • (string) long_name : set "long_name" netCDF attribute conforming to CF conventions
  • (string) standard_name : set "standard_name" netCDF attribute
  • (string) unit : set "unit" netCDF attribute
  • (double) valid_min/valid_max : fix min & max netCDF attribute
• Enable/disable field output
  • (boolean) enabled : if false, field will not be output (default=true)
  • (integer) level : fix the level of output of the field (default=0) with respect to "level_output" file attribute if (level>level_output) the field will not be output
• Setting missing values : fix masked point value in output file
  • (double) default_value : if not set, masked point value will be undefined
• Setting precision / compression
  • (integer) prec : define the output precision of the field : 8 -> double, 4 -> single, 2 -> 2 byte integer
  • (double) add_offset, scale_factor : output will be (field+add_offset)/scale_factor, useful to combine with prec=2
  • (integer) compression_level (0-9) : fix the gzip compression level provided by netCDF4/HDF5: due to HDF5 limitation, doesn’t work for parallel writing. If not set data is not compressed.
  • (boolean) indexed_output : if set to true, only not masked value are output.

Field time operations

• At each time step , data field are exposed from model (xios_send_field)
• Data are extracted according to the grid definition
• Time integration can be performed on incoming data
• The time integration period is fixed by file output frequency (output_freq attribute)
• (string) operation attribute : time operation applied on incoming flux
  • once : data are used one time (first time)
  • instant : instant data values will be used
  • maximum : retains maximum data values over the integration period
  • minimum : retains minimum data values
  • average : make a time average over the period
  • cumulate : cumulate date over the period
• Example : each day, output the time average and instant values of "temp" field

<file name="output" freq_output="1d">
    <field field_ref="temp" name="temp_average" operation="average"/>
    <field field_ref="temp" name="temp_instant" operation="instant"/>
</file>

Time sampling

• Some field are not computed every time step
• (duration) freq_op attribute: frequency with which the field will be extracted from model
• (duration) freq_offset attribute: time offset before extracting the field
• Strongly advised to set freq_op and freq_offset as a multiple of time step
• Example : for daily averaging, get "temp" value every 10 time step. The first value extracted will be at 2nd time step.

<file name="output" freq_output="1d">
    <field field_ref="temp" operation="average" freq_op="10ts" freq_offset="1ts"/>
</file>

Undefined values and time operation

• Undefined values must not participate in time integration operation
• Set default_value attribute as the undefined value (missing value)
• (boolean) detect_missing_value : for the current time step, all field value equal to default_value (undefined value) will not be taking into account to perform the time integration (average, minimum, maximum, cumulate)
• Very resource-expensive since each value of the mesh must be tested independently

Examples of the output on a lat-lon grid

First, we define a lat-lon domain (e.g. in a file called domain_def_latlon.xml):

<!-- Output lat-lon domain definitions  -->
<domain_definition>
  <domain id="latlon1_domain" ni_glo="144" nj_glo="90" type="rectilinear" nvertex="1">
    <generate_rectilinear_domain />
    <interpolate_domain order="1" />
  </domain>
  <domain id="latlon2_domain" ni_glo="144" nj_glo="90" type="rectilinear" nvertex="1">
    <generate_rectilinear_domain />
    <interpolate_domain order="2" />
  </domain>
</domain_definition>

These are set with 144/90 points in the lat/lon directions. The interpolation order can be specified as order 1 or 2 (order of the conservative interpolation). According to the ESM website, "the second-order conservative calculation also includes the gradient across the source cell, so in general it gives a smoother, more accurate representation of the source field. This is particularly true when going from a coarse to finer grid." Also, iris-esmf-regrid uses an order-1 conservative interpolation by default.

Next, we connect the horizontal domain with a vertical axis (defined in a separate XML file) in a file called e.g. grid_def_latlon.xml:

!-- Output lat-lon grid definitions -->
<grid_definition>
  <grid id="ml_latlon1">
    <domain domain_ref="latlon1_domain" />
    <axis axis_ref="vert_axis_full_levels" />
  </grid>
  <grid id="hl_latlon1">
    <domain domain_ref="latlon1_domain" />
    <axis axis_ref="vert_axis_half_levels" />
  </grid>
</grid_definition>

Next, we import these definitions to the main iodef file:

    <!-- Include diagnostic domain definitions -->
    <domain_definition src="./domain_def_latlon.xml"/>

    <!-- Include grid definitions -->
    <grid_definition src="./grid_def_latlon.xml"/>

And then we can request diagnostics using these grid definitions:

  <file id="lfric_latlon_diags" name="lfric_latlon_diags" output_freq="1d" convention="CF" enabled=".TRUE.">
    <field_group operation="instant" freq_op="1d" grid_ref="fl_latlon1" >
      <field field_ref="theta" />
      <field field_ref="u_in_w3"/>
      <field field_ref="v_in_w3"/>
    </field_group>
    <field_group operation="instant" freq_op="1d" domain_ref="latlon1_domain" >
      <field field_ref="surface__grid_surface_temperature" />
    </field_group>
  </file>

Example of the vertical axis definition for an output on pressure levels:

<axis id="pressure_levels1" name="pressure" unit="Pa" positive="down" n_glo="17" value="(0,16)[100000. 92500. 85000. 70000. 60000. 50000. 40000. 30000. 25000. 20000. 15000. 10000. 7000. 5000. 3000. 2000. 1000.]">
    <interpolate_axis type="polynomial" order="1" coordinate="pressure_in_wth" />
</axis>