clisops package

CLISOPS - Climate simulation operations.

clisops.showwarning(message, *args, **kwargs)[source]: Inject warnings from warnings.warn into loguru.

Subpackages

Submodules

clisops.version module

clisops.config module

clisops.config._chain_config_types(conf, keys)[source]

clisops.config._expand_mappings(mappings, modifier, items)[source]: Expands mappings by replacing modifier with list of items in each case.

clisops.config._gather_config_files(package: str | PathLike[str] | Path | None = None)[source]

clisops.config._get_mappers(conf)[source]

clisops.config._load_config(package=None)[source]

clisops.config._modify_fixed_path_mappings(config, name)[source]: Expands the contents of fixed_path_mappings based on other fixed path modifiers`. Returns None - changes contents in place.

clisops.config._post_process(config)[source]: Post-processes the contents of the config file to modify sections based on certain rules. Returns None.

clisops.config._to_boolean(i)[source]

clisops.config._to_dict(i)[source]

clisops.config._to_float(i)[source]

clisops.config._to_int(i)[source]

clisops.config._to_list(i)[source]

clisops.config.get_config(package=None)[source]

clisops.config.reload_config(package=None)[source]

Reloads the configuration from the config file.

Used for forcibly reloading the configuration from the config file, particularly useful for pytesting mock imports.

clisops.exceptions module

exception clisops.exceptions.InconsistencyError[source]

Bases: Exception

Raised when there is some inconsistency which prevents files being scanned.

exception clisops.exceptions.InvalidCollection(message='Some or all of the requested collection are not in the list of available data.')[source]: Bases: Exception

exception clisops.exceptions.InvalidParameterValue[source]: Bases: Exception

exception clisops.exceptions.InvalidProject[source]

Bases: Exception

Raised when the project is unknown to roocs.ini

exception clisops.exceptions.MissingParameterValue[source]: Bases: Exception

clisops.project_utils module

class clisops.project_utils.DatasetMapper(dset, project=None, force=False)[source]

Bases: object

Class to map to data path, dataset ID and files from any dataset input.

Dset must be a string and can be input as:

A dataset ID: e.g. “cmip5.output1.INM.inmcm4.rcp45.mon.ocean.Omon.r1i1p1.latest.zostoga”.
A file path: e.g. “/badc/cmip5/data/cmip5/output1/MOHC/HadGEM2-ES/rcp85/mon/atmos/Amon/r1i1p1/latest/tas/tas_Amon_HadGEM2-ES_rcp85_r1i1p1_200512-203011.nc”.
A path to a group of files: e.g. “/badc/cmip5/data/cmip5/output1/MOHC/HadGEM2-ES/rcp85/mon/atmos/Amon/r1i1p1/latest/tas/*.nc”.
A directory e.g. “/badc/cmip5/data/cmip5/output1/MOHC/HadGEM2-ES/rcp85/mon/atmos/Amon/r1i1p1/latest/tas”.
An instance of the FileMapper class (that represents a set of files within a single directory).

When force=True, if the project can not be identified, any attempt to use the base_dir of a project to resolve the data path will be ignored. Any of data_path, ds_id and files that can be set, will be set.

SUPPORTED_EXTENSIONS = ('.nc', '.gz')

_deduce_project(dset)[source]

static _get_base_dirs_dict()[source]

static _is_ds_id(dset)[source]

_parse(force)[source]

property base_dir: The base directory of the input dataset.

property data_path: Dataset input converted to a data path.

property ds_id: Dataset input converted to a ds id.

property files: The files found from the input dataset.

property project: The project of the dataset input.

property raw: Raw dataset input.

clisops.project_utils.datapath_to_dsid(datapath)[source]

Switches from dataset path to ds id.

Parameters:: datapath – dataset path.
Returns:: dataset id of input dataset path.

clisops.project_utils.derive_ds_id(dset)[source]

Derives the dataset id of the provided dset.

Parameters:: dset – dset input of type described by DatasetMapper.
Returns:: ds id of input dataset.

clisops.project_utils.derive_dset(dset)[source]

Derives the dataset path of the provided dset.

Parameters:: dset – dset input of type described by DatasetMapper.
Returns:: dataset path of input dataset.

clisops.project_utils.dset_to_filepaths(dset, force=False)[source]

Gets filepaths deduced from input dset.

Parameters:

dset – dset input of type described by DatasetMapper.
force – When True and if the project of the input dset cannot be identified, DatasetMapper will attempt to find the files anyway. Default is False.

Returns:

File paths deduced from input dataset.

clisops.project_utils.dsid_to_datapath(dsid)[source]

Switches from ds id to dataset path.

Parameters:: dsid – dataset id.
Returns:: dataset path of input dataset id.

clisops.project_utils.get_data_node_dirs_dict()[source]: Get a dictionary of the data node roots used for retrieving original files.

clisops.project_utils.get_facet(facet_name, facets, project)[source]: Get facet from project config

clisops.project_utils.get_project_base_dir(project)[source]: Get the base directory of a project from the config.

clisops.project_utils.get_project_from_data_node_root(url)[source]: Identify the project from data node root by identifying the data node root in the input url.

clisops.project_utils.get_project_from_ds(ds)[source]

Gets the project from an xarray Dataset/DataArray.

Parameters:: ds – xarray Dataset/DataArray.
Returns:: The project derived from the input dataset.

clisops.project_utils.get_project_name(dset)[source]

Gets the project from an input dset.

Parameters:: dset – dset input of type described by DatasetMapper.
Returns:: The project derived from the input dataset.

clisops.project_utils.get_projects()[source]: Gets all the projects available in the config.

clisops.project_utils.map_facet(facet, project)[source]: Return mapped facet value from config or facet name if not found.

clisops.project_utils.switch_dset(dset)[source]

Switches between dataset path and ds id.

Parameters:: dset – either dataset path or dataset ID.
Returns:: either dataset path or dataset ID - switched from the input.

clisops.project_utils.url_to_file_path(url)[source]: Convert input url of an original file to a file path