IO

io

Reader

read(input_files, input_format, requested_fields, interp_grid=True, **kwargs)[source]

Read simulation data from input files and return an xarray Dataset.

Parameters:

  • input_files (Path) – Path to the input file(s) containing simulation data.

  • input_format (str) – Format of the input data. Supported formats are: sgs, um, monc

  • requested_fields (list[str]) – List of variable names to extract from the input data.

  • kwargs – Additional keyword arguments depending on the input format. The um format, requires resolution (float) specifying horizontal grid spacing. It also accepts field_names_dict dict[str,str] prescribing a field-names lookup table

Return type:

Dataset

Returns:

xarray Dataset containing the requested fields and metadata, including the horizontal resolution stored in attrs["h_resolution"].

Note

  • For monc format, resolution is inferred from metadata and assumed

    isotropic in x and y

  • For um format, resolution must be explicitly provided via kwargs.

  • For sgs format, if h_resolution is not a dataset attribute

    it is guessed by the spacing in “x” and “y” coordinates

UM

base_fields_dict = {'THETA_AFTER_TIMESTEP': 'theta', 'U_COMPNT_OF_WIND_AFTER_TIMESTEP': 'u', 'V_COMPNT_OF_WIND_AFTER_TIMESTEP': 'v', 'W_COMPNT_OF_WIND_AFTER_TIMESTEP': 'w'}
Water_dict = {'CLD_ICE_MIXING_RATIO__mcf__AFTER_TS': 'q_i', 'CLD_LIQ_MIXING_RATIO__mcl__AFTER_TS': 'q_l', 'GRAUPEL_MIXING_RATIO__mg__AFTER_TS': 'q_g', 'LARGE_SCALE_RAINFALL_RATE____KG_M2_S': 'rain', 'SPECIFIC_HUMIDITY_AFTER_TIMESTEP': 'q_v'}
Smagorinsky_dict = {'CS_THETA': 'cs_theta', 'GRADIENT_RICHARDSON_NUMBER': 'Richardson', 'MIXING_LENGTH_RNEUTML': 'csDelta', 'SHEAR_AT_SCALE_DELTA': 's', 'SMAG__S__SHEAR_TERM_': 's_smag', 'SMAG__VISC_H': 'smag_visc_h', 'SMAG__VISC_M': 'smag_visc_m', 'TURBULENT_KINETIC_ENERGY': 'tke'}
dynamic_SGS_dict = {'CS_SQUARED_AT_2_DELTA': 'cs2d', 'CS_SQUARED_AT_4_DELTA': 'cs4d', 'CS_THETA_AT_SCALE_2DELTA': 'cs_theta_2d', 'CS_THETA_AT_SCALE_4DELTA': 'cs_theta_4d'}
dynamic_anisotropic_SGS_dict = {'CS_1': 'cs_1', 'CS_2': 'cs_2', 'CS_3': 'cs_3', 'CS_THETA_1': 'cs_theta_1', 'CS_THETA_2': 'cs_theta_2', 'CS_THETA_3': 'cs_theta_3', 'RHOKH_DIFF_COEFF___LOCAL_SCHEME': 'smag_visc_h_vert', 'RHOKM_DIFF_COEFF___LOCAL_SCHEME': 'smag_visc_m_vert'}
dynamic_SGS_diag_dict = {'D11_TENSOR_COMPONENT': 'diag11', 'D12_TENSOR_COMPONENT': 'diag12', 'D13_TENSOR_COMPONENT': 'diag13', 'D22_TENSOR_COMPONENT': 'diag22', 'D23_TENSOR_COMPONENT': 'diag23', 'D33_TENSOR_COMPONENT': 'diag33', 'FjFj_CONT_VECTORS': 'ff', 'HjTj_CONT_VECTORS': 'ht', 'Lagrangian_averaged_FjFj_vector': 'FF', 'Lagrangian_averaged_HjTj_vector': 'HT', 'Lagrangian_averaged_LijMij_tensors': 'LM', 'Lagrangian_averaged_MijMij_tensors': 'MM', 'Lagrangian_averaged_NijNij_tensors': 'NN', 'Lagrangian_averaged_QijNij_tensors': 'QN', 'Lagrangian_averaged_RjFj_vector': 'RF', 'Lagrangian_averaged_TjTj_vector': 'TT', 'LijMij_CONT_TENSORS': 'lm', 'MijMij_CONT_TENSORS': 'mm', 'NijNij_CONT_TENSORS': 'nn', 'QijNij_CONT_TENSORS': 'qn', 'Richardson': 'Richardson_diag', 'RjFj_CONT_VECTORS': 'rf', 'SHEAR_AT_SCALE_2DELTA': 's2d', 'SHEAR_AT_SCALE_4DELTA': 's4d', 'Tdecorr_heat': 'Tdecorr_heat', 'Tdecorr_momentum': 'Tdecorr_momentum', 'TjTj_CONT_VECTORS': 'tt'}
read_stash_files(fname_pattern, chunks='auto')[source]

combine a list of output Stash files

Parameters:

fname_pattern (Path | str) – filename(s) to read. Will be interpreted as a glob pattern.

Return type:

Dataset

Returns:

xarray.Dataset with all available variables

rename_variables(ds)[source]
rename STASH variables:

UM STASH varaibles adopt their long_name with special characters replaced by ‘_’. The stash code is retained as an attribute for back-searches. Spacial coordinates/dimesions are renamed to z_{theta|rho} and {x|y}_{face|centre}. Time coordinate becomes t and t_0.

Parameters:

ds (Dataset) – input dataset

Return type:

Dataset

Returns:

dataset with renamed variables

unify_coords(ds, res)[source]

unify coordinate names

implement correct x-spacing using res, assume res is given in the correct units rename coordinates with reference to a logically-cartesian grid

Parameters:

  • ds (Dataset) – input dataset

  • res (float) – resolution of dataset – to create correct x-y grid for idealised runs (the existing one is in lat-lon coords)

Return type:

Dataset

Returns:

dataset with renamed variables

data_ingest_UM(fname_pattern, res, requested_fields=['u', 'v', 'w', 'theta'], field_names_dict=None)[source]

read and pre-process UM data using sgs_tools naming convention. Any unknown fields will retain their original names.

Parameters:

  • fname_pattern (Path | str) – UM NetCDF diagnostic file(s) to read. will be interpreted as a glob pattern. (should belong to the same simulation)

  • res (float) – horizontal resolution (will use to overwrite horizontal coordinates). NB works for ideal simulations

  • requested_fields (list[str]) – list of fields to retain in ds, if falsy will retain all.

  • field_names_dict (dict[str, str] | None) – a look-up table used to rename fields in the input dataset will use default_field_names_dict if None

Return type:

Dataset

data_ingest_UM_on_single_grid(fname_pattern, res, requested_fields=['u', 'v', 'w', 'theta'], field_names_dict=None)[source]

read, pre-process UM data and interpolate to a cell-centred grid Any unknown fields will retain their original names.

Parameters:

  • fname_pattern (Path | str) – UM NetCDF diagnostic file(s) to read. will be interpreted as a glob pattern. (should belong to the same simulation)

  • res (float) – horizontal resolution (will use to overwrite horizontal coordinates). NB works for ideal simulations

  • requested_fields (list[str]) – list of fields to retain in ds, if falsy will retain all.

  • field_names_dict (dict[str, str] | None) – a look-up table used to rename fields in the input dataset. will use default_field_names_dict if None

Return type:

Dataset

MONC

data_ingest_MONC(fname_pattern, requested_fields=['u', 'v', 'w', 'theta'], chunks='auto')[source]

read and pre-process MONC data using sgs_tools naming convention. Any unknown fields will retain their original names.

Parameters:

  • fname_pattern (Path | str) – MONC NetCDF diagnostic file to read. can be a glob pattern. (should belong to the same simulation)

  • requested_fields (list[str]) – list of fields to retain in ds, if falsy will retain all.

  • chunks (Any) – chunking of datasets “auto” or a dictionary of {coordinate: chunks}.

Returns:

metadata dictionary, xarray Dataset of fields.

data_ingest_MONC_on_single_grid(fname_pattern, requested_fields=['u', 'v', 'w', 'theta'], chunks='auto')[source]
read pre-process MONC data and interpolate to a cell-centred grid

Any unknown fields will retain their original names.

Parameters:

  • fname_pattern (Path | str) – MONC NetCDF diagnostic file(s) to read. will be interpreted as a glob pattern. (should belong to the same simulation)

  • requested_fields (list[str]) – list of fields to retain in ds, if falsy will retain all.

  • chunks (Any) – chunking of datasets “auto” or a dictionary of {coordinate: chunks}.

Return type:

tuple[dict[str, str], Dataset]

SGS (own format)

data_ingest_SGS(fname_pattern, requested_fields=['u', 'v', 'w', 'theta'], chunks='auto')[source]

read and pre-process local-convention (sgs_tools) NetCDF data using sgs_tools naming convention. Will not rename any fields, assume they are in local convention.

Parameters:

  • fname_pattern (Path | str) – NetCDF diagnostic file to read. can be a glob pattern. (should belong to the same simulation)

  • requested_fields (list[str]) – list of fields to retain in ds, if falsy will retain all.

  • chunks (Any) – chunking for data

Writer

class NetCDFWriter(overwrite=False, verbose=False)[source]

A class to write xarray datasets to NetCDF files.

Variables:

overwrite – overwrite existing files if set to True. If False, raises an OSError if the file already exists.

check_filename(filename)[source]

Check if the file exists.

Parameters:

filename (Path) – Path to the NetCDF file.

Return type:

bool

write(array, filename)[source]

Write an xarray Dataset to a NetCDF file.

Parameters:

  • array (Dataset) – xarray Dataset to write.

  • filename (Path) – Path to the output NetCDF file.

Return type:

None