Improve with a API to list IDSs with occurrences

[deepakmaroo]

Requirement: My automation script is responsible to validate IDSs and occurrences of given simulations (URI) with physically available database files.

Use case:
The first step in working with a given simulation is to identify the output IDSs. This is crucial because, even in automated workflows, the initial requirement is to prepare and validate the IDS list before performing operations such as reading, manipulating, or writing data.

Having a utility to generate the IDS list is especially useful for users who do not have direct access to the physical database but need to know the available IDSs for a given URI. This functionality simplifies the process of understanding the data structure and ensures that subsequent operations are performed on valid and existing IDSs.

Workaround: For now IDS list is derived from ids_names (list of IDSs in data_dictionary) by reading ids_properties.

Proposal: Instated of iterating all imas.ids_names list and capturing expectation of unavailable ids's, can we provide a API which returns the list of IDSs along with occurrences.
Internal to API, it can open master file and parsed the idss, why because it is generated by code and keeping accurate information of the simulations.

[deepakmaroo]

import h5py

Specify the path to the HDF5 file

file_path = "/path/to/simulation/master.h5"

Open the HDF5 file in read mode

with h5py.File(file_path, "r") as hdf:
# List all groups and datasets in the file
print("Keys in the HDF5 file:")
for key in hdf.keys():
    print(key)

Access a specific group or dataset

if "group_name" in hdf:
    group = hdf["group_name"]
    print("\nContents of 'group_name':")
    for sub_key in group.keys():
        print(sub_key)

# Access a specific dataset and read its data
if "dataset_name" in hdf:
    dataset = hdf["dataset_name"]
    print("\nData in 'dataset_name':")
    print(dataset[:])  # Read the entire dataset as a NumPy array

Sample output

core_profiles
core_sources
core_transport
dataset_description
edge_profiles
edge_sources
edge_transport
equilibrium
equilibrium_1
summary

[maarten-ic]

Hi @deepakmaroo,

You're right, this could be a useful addition to the API!
For reference, the recommended workaround at this moment is to do the following:

filled_idss_and_occurrences = {}
with imas.DBEntry(URI, "r") as entry:
    for ids_name in entry.factory.ids_names():
        occurrences = entry.list_all_occurrences(ids_name)
        if occurrences:
            filled_idss_and_occurrences[ids_name] = occurrences

print(filled_idss_and_occurrences)