Note
This page was generated from examples/get_available_simulation.ipynb.
Interactive online version: 
Get available CMIP6 and CORDEX simulations¶
Downclim allows you to easily access CMIP6 and CORDEX simulations.
This notebook shows how to get a list of available simulations for a given set of variables and regions. This is a prior step to actually download the data.
[ ]:
from __future__ import annotations
from downclim.dataset.cmip6 import CMIP6Context
from downclim.dataset.cordex import CORDEXContext
from downclim.dataset.utils import Frequency, save_simulations_list
CMIP6Context and CORDEXContext are the main classes to interact with CMIP6 and CORDEX simulations. They are containers of all the information needed to proceed to your request.
CMIP6 simulations¶
To request CMIP6 simulations available on Google Cloud File System (GCFS), you must use the CMIP6Context class.
You can use all classical fields for filtering CMIP6 simulations. Also, even though we use GCFS, and thus cannot guarantee full accordance with ESGF server, you can also check and filter CMIP6 data, e.g. here: https://esgf-node.ipsl.upmc.fr/search/cmip6-ipsl/.
Here are the main fields you can use :
[3]:
cmip6_context = CMIP6Context(
project = ["ScenarioMIP", "CMIP"],
institute = ["NOAA-GFDL", "CMCC"],
experiment = ["ssp126", "historical"],
ensemble = "r1i1p1f1",
frequency = Frequency.MONTHLY,
variable = ["tas", "pr"],
grid_label = "gn",
)
cmip6_context.model_dump()
[3]:
{'project': ['ScenarioMIP', 'CMIP'],
'institute': ['NOAA-GFDL', 'CMCC'],
'source': None,
'experiment': ['ssp126', 'historical'],
'ensemble': ['r1i1p1f1'],
'frequency': <Frequency.MONTHLY: 'monthly'>,
'variable': ['tas', 'pr'],
'grid_label': 'gn'}
You can have more information about the fields you can provide to your CMIP6Context object :
[4]:
help(CMIP6Context)
Help on class CMIP6Context in module downclim.dataset.cmip6:
class CMIP6Context(pydantic.main.BaseModel)
| CMIP6Context(
| *,
| project: list[str] | None = ['ScenarioMIP', 'CMIP'],
| institute: list[str] | None = None,
| source: list[str] | None = None,
| experiment: list[str] | None = ['ssp245', 'historical'],
| ensemble: list[str] | None = ['r1i1p1f1'],
| frequency: downclim.dataset.utils.Frequency = <Frequency.MONTHLY: 'monthly'>,
| variable: list[str] | None = ['tas', 'pr'],
| grid_label: str | None = None
| ) -> None
|
| Context about the query on the CMIP6 dataset.
|
| Entries of the dictionary can be either `str` or `list` of `str` if multiple values are provided. These following keys are available. None are mandatory:
|
| - activity_id: str, e.g "ScenarioMIP", "CMIP"
| - institution_id: str, e.g "IPSL", "NCAR"
| - source_id: str, e.g "IPSL-CM6A-LR", "CMCC-CM2-HR4"
| - experiment_id: str, e.g "ssp126", "historical"
| - member_id: str, e.g "r1i1p1f1"
| - table_id: str, e.g "Amon", "day"
| - variable_id: str, e.g "tas", "pr"
| - grid_label: str, e.g "gn", "gr"
|
| Method resolution order:
| CMIP6Context
| pydantic.main.BaseModel
| builtins.object
|
| Methods defined here:
|
| list_available_simulations(
| self,
| cmip6_catalog_url: 'str' = 'https://storage.googleapis.com/cmip6/cmip6-zarr-consolidated-stores.csv'
| ) -> 'pd.DataFrame'
| List all available CMIP6 simulations available on Google Cloud Storage for a given set of context.
|
| Parameters
| ----------
| cmip6_catalog_url: str (default: DataProduct.CMIP6.url)
| URL to the CMIP6 catalog on the Google Cloud File System.
|
| Returns:
| -------
| pd.DataFrame: DataFrame containing information about the available datasets matching
|
| ----------------------------------------------------------------------
| Class methods defined here:
|
| to_list(v: 'Any') -> 'list[Any]'
|
| validate_experiment_id(v: 'str | Iterable[str] | None') -> 'list[str]'
|
| validate_list(v: 'Any') -> 'list[Any]'
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| __weakref__
| list of weak references to the object
|
| ----------------------------------------------------------------------
| Data and other attributes defined here:
|
| Config = <class 'downclim.dataset.cmip6.CMIP6Context.Config'>
| Pydantic configuration for the DownClimContext class.
|
|
| __abstractmethods__ = frozenset()
|
| __annotations__ = {'ensemble': 'list[str] | None', 'experiment': 'list...
|
| __class_vars__ = set()
|
| __private_attributes__ = {}
|
| __pydantic_complete__ = True
|
| __pydantic_computed_fields__ = {}
|
| __pydantic_core_schema__ = {'cls': <class 'downclim.dataset.cmip6.CMIP...
|
| __pydantic_custom_init__ = False
|
| __pydantic_decorators__ = DecoratorInfos(validators={}, field_validato...
|
| __pydantic_fields__ = {'ensemble': FieldInfo(annotation=Union[list[str...
|
| __pydantic_generic_metadata__ = {'args': (), 'origin': None, 'paramete...
|
| __pydantic_parent_namespace__ = None
|
| __pydantic_post_init__ = None
|
| __pydantic_serializer__ = SchemaSerializer(serializer=Model(
| Model...
|
| __pydantic_validator__ = SchemaValidator(title="CMIP6Context", validat...
|
| __signature__ = <Signature (*, project: list[str] | None = ['Sce...', ...
|
| model_config = {'extra': 'forbid'}
|
| ----------------------------------------------------------------------
| Methods inherited from pydantic.main.BaseModel:
|
| __copy__(self) -> 'Self'
| Returns a shallow copy of the model.
|
| __deepcopy__(self, memo: 'dict[int, Any] | None' = None) -> 'Self'
| Returns a deep copy of the model.
|
| __delattr__(self, item: 'str') -> 'Any'
| Implement delattr(self, name).
|
| __eq__(self, other: 'Any') -> 'bool'
| Return self==value.
|
| __getattr__(self, item: 'str') -> 'Any'
|
| __getstate__(self) -> 'dict[Any, Any]'
| Helper for pickle.
|
| __init__(self, /, **data: 'Any') -> 'None'
| Create a new model by parsing and validating input data from keyword arguments.
|
| Raises [`ValidationError`][pydantic_core.ValidationError] if the input data cannot be
| validated to form a valid model.
|
| `self` is explicitly positional-only to allow `self` as a field name.
|
| __iter__(self) -> 'TupleGenerator'
| So `dict(model)` works.
|
| __pretty__(self, fmt: 'typing.Callable[[Any], Any]', **kwargs: 'Any') -> 'typing.Generator[Any, None, None]' from pydantic._internal._repr.Representation
| Used by devtools (https://python-devtools.helpmanual.io/) to pretty print objects.
|
| __replace__(self, **changes: 'Any') -> 'Self'
| # Because we make use of `@dataclass_transform()`, `__replace__` is already synthesized by
| # type checkers, so we define the implementation in this `if not TYPE_CHECKING:` block:
|
| __repr__(self) -> 'str'
| Return repr(self).
|
| __repr_args__(self) -> '_repr.ReprArgs'
|
| __repr_name__(self) -> 'str' from pydantic._internal._repr.Representation
| Name of the instance's class, used in __repr__.
|
| __repr_recursion__(self, object: 'Any') -> 'str' from pydantic._internal._repr.Representation
| Returns the string representation of a recursive object.
|
| __repr_str__(self, join_str: 'str') -> 'str' from pydantic._internal._repr.Representation
|
| __rich_repr__(self) -> 'RichReprResult' from pydantic._internal._repr.Representation
| Used by Rich (https://rich.readthedocs.io/en/stable/pretty.html) to pretty print objects.
|
| __setattr__(self, name: 'str', value: 'Any') -> 'None'
| Implement setattr(self, name, value).
|
| __setstate__(self, state: 'dict[Any, Any]') -> 'None'
|
| __str__(self) -> 'str'
| Return str(self).
|
| copy(
| self,
| *,
| include: 'AbstractSetIntStr | MappingIntStrAny | None' = None,
| exclude: 'AbstractSetIntStr | MappingIntStrAny | None' = None,
| update: 'Dict[str, Any] | None' = None,
| deep: 'bool' = False
| ) -> 'Self'
| Returns a copy of the model.
|
| !!! warning "Deprecated"
| This method is now deprecated; use `model_copy` instead.
|
| If you need `include` or `exclude`, use:
|
| ```python {test="skip" lint="skip"}
| data = self.model_dump(include=include, exclude=exclude, round_trip=True)
| data = {**data, **(update or {})}
| copied = self.model_validate(data)
| ```
|
| Args:
| include: Optional set or mapping specifying which fields to include in the copied model.
| exclude: Optional set or mapping specifying which fields to exclude in the copied model.
| update: Optional dictionary of field-value pairs to override field values in the copied model.
| deep: If True, the values of fields that are Pydantic models will be deep-copied.
|
| Returns:
| A copy of the model with included, excluded and updated fields as specified.
|
| dict(
| self,
| *,
| include: 'IncEx | None' = None,
| exclude: 'IncEx | None' = None,
| by_alias: 'bool' = False,
| exclude_unset: 'bool' = False,
| exclude_defaults: 'bool' = False,
| exclude_none: 'bool' = False
| ) -> 'Dict[str, Any]'
|
| json(
| self,
| *,
| include: 'IncEx | None' = None,
| exclude: 'IncEx | None' = None,
| by_alias: 'bool' = False,
| exclude_unset: 'bool' = False,
| exclude_defaults: 'bool' = False,
| exclude_none: 'bool' = False,
| encoder: 'Callable[[Any], Any] | None' = PydanticUndefined,
| models_as_dict: 'bool' = PydanticUndefined,
| **dumps_kwargs: 'Any'
| ) -> 'str'
|
| model_copy(
| self,
| *,
| update: 'Mapping[str, Any] | None' = None,
| deep: 'bool' = False
| ) -> 'Self'
| Usage docs: https://docs.pydantic.dev/2.10/concepts/serialization/#model_copy
|
| Returns a copy of the model.
|
| Args:
| update: Values to change/add in the new model. Note: the data is not validated
| before creating the new model. You should trust this data.
| deep: Set to `True` to make a deep copy of the model.
|
| Returns:
| New model instance.
|
| model_dump(
| self,
| *,
| mode: "Literal['json', 'python'] | str" = 'python',
| include: 'IncEx | None' = None,
| exclude: 'IncEx | None' = None,
| context: 'Any | None' = None,
| by_alias: 'bool' = False,
| exclude_unset: 'bool' = False,
| exclude_defaults: 'bool' = False,
| exclude_none: 'bool' = False,
| round_trip: 'bool' = False,
| warnings: "bool | Literal['none', 'warn', 'error']" = True,
| serialize_as_any: 'bool' = False
| ) -> 'dict[str, Any]'
| Usage docs: https://docs.pydantic.dev/2.10/concepts/serialization/#modelmodel_dump
|
| Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
|
| Args:
| mode: The mode in which `to_python` should run.
| If mode is 'json', the output will only contain JSON serializable types.
| If mode is 'python', the output may contain non-JSON-serializable Python objects.
| include: A set of fields to include in the output.
| exclude: A set of fields to exclude from the output.
| context: Additional context to pass to the serializer.
| by_alias: Whether to use the field's alias in the dictionary key if defined.
| exclude_unset: Whether to exclude fields that have not been explicitly set.
| exclude_defaults: Whether to exclude fields that are set to their default value.
| exclude_none: Whether to exclude fields that have a value of `None`.
| round_trip: If True, dumped values should be valid as input for non-idempotent types such as Json[T].
| warnings: How to handle serialization errors. False/"none" ignores them, True/"warn" logs errors,
| "error" raises a [`PydanticSerializationError`][pydantic_core.PydanticSerializationError].
| serialize_as_any: Whether to serialize fields with duck-typing serialization behavior.
|
| Returns:
| A dictionary representation of the model.
|
| model_dump_json(
| self,
| *,
| indent: 'int | None' = None,
| include: 'IncEx | None' = None,
| exclude: 'IncEx | None' = None,
| context: 'Any | None' = None,
| by_alias: 'bool' = False,
| exclude_unset: 'bool' = False,
| exclude_defaults: 'bool' = False,
| exclude_none: 'bool' = False,
| round_trip: 'bool' = False,
| warnings: "bool | Literal['none', 'warn', 'error']" = True,
| serialize_as_any: 'bool' = False
| ) -> 'str'
| Usage docs: https://docs.pydantic.dev/2.10/concepts/serialization/#modelmodel_dump_json
|
| Generates a JSON representation of the model using Pydantic's `to_json` method.
|
| Args:
| indent: Indentation to use in the JSON output. If None is passed, the output will be compact.
| include: Field(s) to include in the JSON output.
| exclude: Field(s) to exclude from the JSON output.
| context: Additional context to pass to the serializer.
| by_alias: Whether to serialize using field aliases.
| exclude_unset: Whether to exclude fields that have not been explicitly set.
| exclude_defaults: Whether to exclude fields that are set to their default value.
| exclude_none: Whether to exclude fields that have a value of `None`.
| round_trip: If True, dumped values should be valid as input for non-idempotent types such as Json[T].
| warnings: How to handle serialization errors. False/"none" ignores them, True/"warn" logs errors,
| "error" raises a [`PydanticSerializationError`][pydantic_core.PydanticSerializationError].
| serialize_as_any: Whether to serialize fields with duck-typing serialization behavior.
|
| Returns:
| A JSON string representation of the model.
|
| model_post_init(self, _BaseModel__context: 'Any') -> 'None'
| Override this method to perform additional initialization after `__init__` and `model_construct`.
| This is useful if you want to do some validation that requires the entire model to be initialized.
|
| ----------------------------------------------------------------------
| Class methods inherited from pydantic.main.BaseModel:
|
| __class_getitem__(typevar_values: 'type[Any] | tuple[type[Any], ...]') -> 'type[BaseModel] | _forward_ref.PydanticRecursiveRef'
|
| __get_pydantic_core_schema__(
| source: 'type[BaseModel]',
| handler: 'GetCoreSchemaHandler',
| /
| ) -> 'CoreSchema'
| Hook into generating the model's CoreSchema.
|
| Args:
| source: The class we are generating a schema for.
| This will generally be the same as the `cls` argument if this is a classmethod.
| handler: A callable that calls into Pydantic's internal CoreSchema generation logic.
|
| Returns:
| A `pydantic-core` `CoreSchema`.
|
| __get_pydantic_json_schema__(
| core_schema: 'CoreSchema',
| handler: 'GetJsonSchemaHandler',
| /
| ) -> 'JsonSchemaValue'
| Hook into generating the model's JSON schema.
|
| Args:
| core_schema: A `pydantic-core` CoreSchema.
| You can ignore this argument and call the handler with a new CoreSchema,
| wrap this CoreSchema (`{'type': 'nullable', 'schema': current_schema}`),
| or just call the handler with the original schema.
| handler: Call into Pydantic's internal JSON schema generation.
| This will raise a `pydantic.errors.PydanticInvalidForJsonSchema` if JSON schema
| generation fails.
| Since this gets called by `BaseModel.model_json_schema` you can override the
| `schema_generator` argument to that function to change JSON schema generation globally
| for a type.
|
| Returns:
| A JSON schema, as a Python object.
|
| __pydantic_init_subclass__(**kwargs: 'Any') -> 'None'
| This is intended to behave just like `__init_subclass__`, but is called by `ModelMetaclass`
| only after the class is actually fully initialized. In particular, attributes like `model_fields` will
| be present when this is called.
|
| This is necessary because `__init_subclass__` will always be called by `type.__new__`,
| and it would require a prohibitively large refactor to the `ModelMetaclass` to ensure that
| `type.__new__` was called in such a manner that the class would already be sufficiently initialized.
|
| This will receive the same `kwargs` that would be passed to the standard `__init_subclass__`, namely,
| any kwargs passed to the class definition that aren't used internally by pydantic.
|
| Args:
| **kwargs: Any keyword arguments passed to the class definition that aren't used internally
| by pydantic.
|
| construct(_fields_set: 'set[str] | None' = None, **values: 'Any') -> 'Self'
|
| from_orm(obj: 'Any') -> 'Self'
|
| model_construct(_fields_set: 'set[str] | None' = None, **values: 'Any') -> 'Self'
| Creates a new instance of the `Model` class with validated data.
|
| Creates a new model setting `__dict__` and `__pydantic_fields_set__` from trusted or pre-validated data.
| Default values are respected, but no other validation is performed.
|
| !!! note
| `model_construct()` generally respects the `model_config.extra` setting on the provided model.
| That is, if `model_config.extra == 'allow'`, then all extra passed values are added to the model instance's `__dict__`
| and `__pydantic_extra__` fields. If `model_config.extra == 'ignore'` (the default), then all extra passed values are ignored.
| Because no validation is performed with a call to `model_construct()`, having `model_config.extra == 'forbid'` does not result in
| an error if extra values are passed, but they will be ignored.
|
| Args:
| _fields_set: A set of field names that were originally explicitly set during instantiation. If provided,
| this is directly used for the [`model_fields_set`][pydantic.BaseModel.model_fields_set] attribute.
| Otherwise, the field names from the `values` argument will be used.
| values: Trusted or pre-validated data dictionary.
|
| Returns:
| A new instance of the `Model` class with validated data.
|
| model_json_schema(
| by_alias: 'bool' = True,
| ref_template: 'str' = '#/$defs/{model}',
| schema_generator: 'type[GenerateJsonSchema]' = <class 'pydantic.json_schema.GenerateJsonSchema'>,
| mode: 'JsonSchemaMode' = 'validation'
| ) -> 'dict[str, Any]'
| Generates a JSON schema for a model class.
|
| Args:
| by_alias: Whether to use attribute aliases or not.
| ref_template: The reference template.
| schema_generator: To override the logic used to generate the JSON schema, as a subclass of
| `GenerateJsonSchema` with your desired modifications
| mode: The mode in which to generate the schema.
|
| Returns:
| The JSON schema for the given model class.
|
| model_parametrized_name(params: 'tuple[type[Any], ...]') -> 'str'
| Compute the class name for parametrizations of generic classes.
|
| This method can be overridden to achieve a custom naming scheme for generic BaseModels.
|
| Args:
| params: Tuple of types of the class. Given a generic class
| `Model` with 2 type variables and a concrete model `Model[str, int]`,
| the value `(str, int)` would be passed to `params`.
|
| Returns:
| String representing the new class where `params` are passed to `cls` as type variables.
|
| Raises:
| TypeError: Raised when trying to generate concrete names for non-generic models.
|
| model_rebuild(
| *,
| force: 'bool' = False,
| raise_errors: 'bool' = True,
| _parent_namespace_depth: 'int' = 2,
| _types_namespace: 'MappingNamespace | None' = None
| ) -> 'bool | None'
| Try to rebuild the pydantic-core schema for the model.
|
| This may be necessary when one of the annotations is a ForwardRef which could not be resolved during
| the initial attempt to build the schema, and automatic rebuilding fails.
|
| Args:
| force: Whether to force the rebuilding of the model schema, defaults to `False`.
| raise_errors: Whether to raise errors, defaults to `True`.
| _parent_namespace_depth: The depth level of the parent namespace, defaults to 2.
| _types_namespace: The types namespace, defaults to `None`.
|
| Returns:
| Returns `None` if the schema is already "complete" and rebuilding was not required.
| If rebuilding _was_ required, returns `True` if rebuilding was successful, otherwise `False`.
|
| model_validate(
| obj: 'Any',
| *,
| strict: 'bool | None' = None,
| from_attributes: 'bool | None' = None,
| context: 'Any | None' = None
| ) -> 'Self'
| Validate a pydantic model instance.
|
| Args:
| obj: The object to validate.
| strict: Whether to enforce types strictly.
| from_attributes: Whether to extract data from object attributes.
| context: Additional context to pass to the validator.
|
| Raises:
| ValidationError: If the object could not be validated.
|
| Returns:
| The validated model instance.
|
| model_validate_json(
| json_data: 'str | bytes | bytearray',
| *,
| strict: 'bool | None' = None,
| context: 'Any | None' = None
| ) -> 'Self'
| Usage docs: https://docs.pydantic.dev/2.10/concepts/json/#json-parsing
|
| Validate the given JSON data against the Pydantic model.
|
| Args:
| json_data: The JSON data to validate.
| strict: Whether to enforce types strictly.
| context: Extra variables to pass to the validator.
|
| Returns:
| The validated Pydantic model.
|
| Raises:
| ValidationError: If `json_data` is not a JSON string or the object could not be validated.
|
| model_validate_strings(
| obj: 'Any',
| *,
| strict: 'bool | None' = None,
| context: 'Any | None' = None
| ) -> 'Self'
| Validate the given object with string data against the Pydantic model.
|
| Args:
| obj: The object containing string data to validate.
| strict: Whether to enforce types strictly.
| context: Extra variables to pass to the validator.
|
| Returns:
| The validated Pydantic model.
|
| parse_file(
| path: 'str | Path',
| *,
| content_type: 'str | None' = None,
| encoding: 'str' = 'utf8',
| proto: 'DeprecatedParseProtocol | None' = None,
| allow_pickle: 'bool' = False
| ) -> 'Self'
|
| parse_obj(obj: 'Any') -> 'Self'
|
| parse_raw(
| b: 'str | bytes',
| *,
| content_type: 'str | None' = None,
| encoding: 'str' = 'utf8',
| proto: 'DeprecatedParseProtocol | None' = None,
| allow_pickle: 'bool' = False
| ) -> 'Self'
|
| schema(by_alias: 'bool' = True, ref_template: 'str' = '#/$defs/{model}') -> 'Dict[str, Any]'
|
| schema_json(
| *,
| by_alias: 'bool' = True,
| ref_template: 'str' = '#/$defs/{model}',
| **dumps_kwargs: 'Any'
| ) -> 'str'
|
| update_forward_refs(**localns: 'Any') -> 'None'
|
| validate(value: 'Any') -> 'Self'
|
| ----------------------------------------------------------------------
| Readonly properties inherited from pydantic.main.BaseModel:
|
| __fields_set__
|
| model_computed_fields
| Get metadata about the computed fields defined on the model.
|
| Deprecation warning: you should be getting this information from the model class, not from an instance.
| In V3, this property will be removed from the `BaseModel` class.
|
| Returns:
| A mapping of computed field names to [`ComputedFieldInfo`][pydantic.fields.ComputedFieldInfo] objects.
|
| model_extra
| Get extra fields set during validation.
|
| Returns:
| A dictionary of extra fields, or `None` if `config.extra` is not set to `"allow"`.
|
| model_fields
| Get metadata about the fields defined on the model.
|
| Deprecation warning: you should be getting this information from the model class, not from an instance.
| In V3, this property will be removed from the `BaseModel` class.
|
| Returns:
| A mapping of field names to [`FieldInfo`][pydantic.fields.FieldInfo] objects.
|
| model_fields_set
| Returns the set of fields that have been explicitly set on this model instance.
|
| Returns:
| A set of strings representing the fields that have been set,
| i.e. that were not filled from defaults.
|
| ----------------------------------------------------------------------
| Data descriptors inherited from pydantic.main.BaseModel:
|
| __dict__
| dictionary for instance variables
|
| __pydantic_extra__
|
| __pydantic_fields_set__
|
| __pydantic_private__
|
| ----------------------------------------------------------------------
| Data and other attributes inherited from pydantic.main.BaseModel:
|
| __hash__ = None
|
| __pydantic_root_model__ = False
Note You can also use a standard Python dictionary using the same fields as keys. However the CMIP6Context class provides a more user-friendly way to interact with the data, including automatic checking of the fields.
Intermediate solution consist in using a dictionary to provide the fields, and then convert it to a CMIP6Context object:
my_dict = {
'experiment_id': 'historical',
'variable_id': 'tas',
'table_id': 'Amon',
...
}
cmip6_context = CMIP6Context(**my_dict)
Once your context is correctly defined, you can check which simulations are available according to the filters you set in the context.
[5]:
cmip6_simulations = cmip6_context.list_available_simulations()
cmip6_simulations
[5]:
| project | institute | source | experiment | ensemble | table | variable | grid_label | datanode | dcpp_init_year | version | domain | product | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | CMIP | CMCC | CMCC-CM2-SR5 | historical | r1i1p1f1 | Amon | tas | gn | gs://cmip6/CMIP6/CMIP/CMCC/CMCC-CM2-SR5/histor... | NaN | 20200616 | GLOBAL | output |
| 1 | CMIP | CMCC | CMCC-CM2-SR5 | historical | r1i1p1f1 | Amon | pr | gn | gs://cmip6/CMIP6/CMIP/CMCC/CMCC-CM2-SR5/histor... | NaN | 20200616 | GLOBAL | output |
| 2 | ScenarioMIP | CMCC | CMCC-CM2-SR5 | ssp126 | r1i1p1f1 | Amon | tas | gn | gs://cmip6/CMIP6/ScenarioMIP/CMCC/CMCC-CM2-SR5... | NaN | 20200717 | GLOBAL | output |
| 3 | ScenarioMIP | CMCC | CMCC-CM2-SR5 | ssp126 | r1i1p1f1 | Amon | pr | gn | gs://cmip6/CMIP6/ScenarioMIP/CMCC/CMCC-CM2-SR5... | NaN | 20200717 | GLOBAL | output |
| 4 | CMIP | CMCC | CMCC-ESM2 | historical | r1i1p1f1 | Amon | tas | gn | gs://cmip6/CMIP6/CMIP/CMCC/CMCC-ESM2/historica... | NaN | 20210114 | GLOBAL | output |
| 5 | CMIP | CMCC | CMCC-ESM2 | historical | r1i1p1f1 | Amon | pr | gn | gs://cmip6/CMIP6/CMIP/CMCC/CMCC-ESM2/historica... | NaN | 20210114 | GLOBAL | output |
| 6 | ScenarioMIP | CMCC | CMCC-ESM2 | ssp126 | r1i1p1f1 | Amon | pr | gn | gs://cmip6/CMIP6/ScenarioMIP/CMCC/CMCC-ESM2/ss... | NaN | 20210126 | GLOBAL | output |
| 7 | ScenarioMIP | CMCC | CMCC-ESM2 | ssp126 | r1i1p1f1 | Amon | tas | gn | gs://cmip6/CMIP6/ScenarioMIP/CMCC/CMCC-ESM2/ss... | NaN | 20210126 | GLOBAL | output |
CORDEX simulations¶
Similarly, you can use the CORDEXContext class to request CORDEX simulations available on ESGF nodes.
To know more about CORDEX domains, you can check the CORDEX domains website to see to which domain your area of interest belongs.
To help you with the search fields, you can have a look on the ESGF search page, e.g. : https://esgf-node.ipsl.upmc.fr/search/cordex-ipsl/ .
[6]:
cordex_context = CORDEXContext(
domain = "AUS-44",
experiment = ["rcp26", "historical"],
ensemble = "r1i1p1",
frequency = Frequency.MONTHLY,
variable = ["tas", "tasmin", "tasmax", "pr"],
)
cordex_context.model_dump()
[6]:
{'project': ['CORDEX'],
'product': ['output'],
'domain': ['AUS-44'],
'institute': None,
'driving_model': None,
'experiment': ['rcp26', 'historical'],
'experiment_family': None,
'ensemble': ['r1i1p1'],
'rcm_name': None,
'rcm_version': None,
'frequency': <Frequency.MONTHLY: 'monthly'>,
'variable': ['tas', 'tasmin', 'tasmax', 'pr'],
'variable_long_name': None}
[6]:
cordex_context.model_dump()
[6]:
{'project': 'CORDEX',
'product': 'output',
'domain': 'AUS-44',
'institute': None,
'driving_model': None,
'experiment': ['rcp26', 'historical'],
'experiment_family': None,
'ensemble': 'r1i1p1',
'rcm_model': None,
'downscaling_realisation': None,
'time_frequency': 'mon',
'variable': ['tas', 'tasmin', 'tasmax', 'pr'],
'variable_long_name': None}
[11]:
cordex_simulations = cordex_context.list_available_simulations(esgf_credential="../../config/esgf_credential.yaml")
cordex_simulations
---------------------------------------------------------------------------
ConnectionRefusedError Traceback (most recent call last)
Cell In[11], line 1
----> 1 cordex_simulations = cordex_context.list_available_simulations(esgf_credential="../../config/esgf_credential.yaml")
2 cordex_simulations
File ~/Documents/Modeles/Codes/DownClim/src/downclim/dataset/cordex.py:224, in CORDEXContext.list_available_simulations(self, esgf_credential, server)
221 context = self.model_dump()
223 # esgf connection
--> 224 conn = connect_to_esgf(esgf_credential, server)
225 # list CORDEX datasets matching context
226 cordex_simulations = inspect_cordex(context=context, connector=conn)
File ~/Documents/Modeles/Codes/DownClim/src/downclim/dataset/connectors.py:84, in connect_to_esgf(esgf_credential, server)
82 creds = yaml.safe_load(stream)
83 lm = LogonManager()
---> 84 lm.logon_with_openid(
85 openid=creds["openid"],
86 password=creds["password"],
87 interactive=False,
88 bootstrap=True,
89 )
91 return SearchConnection(server, distrib=True)
File ~/miniconda3/envs/downclim/lib/python3.13/site-packages/pyesgf/logon.py:147, in LogonManager.logon_with_openid(self, openid, password, bootstrap, update_trustroots, interactive)
136 """
137 Obtains ESGF credentials by detecting the MyProxy parameters from
138 the users OpenID. Some ESGF compatible OpenIDs do not contain enough
(...)
144 ``interactive``, ``bootstrap`` and ``update_trustroots``.
145 """
146 username, myproxy = self._get_logon_details(openid)
--> 147 return self.logon(username, password, myproxy,
148 bootstrap=bootstrap,
149 update_trustroots=update_trustroots,
150 interactive=interactive)
File ~/miniconda3/envs/downclim/lib/python3.13/site-packages/pyesgf/logon.py:184, in LogonManager.logon(self, username, password, hostname, bootstrap, update_trustroots, interactive)
180 raise OpenidResolutionError('Full logon details not available')
182 c = MyProxyClient(hostname=hostname, caCertDir=self.esgf_certs_dir)
--> 184 creds = c.logon(username, password,
185 bootstrap=bootstrap,
186 updateTrustRoots=update_trustroots)
187 with open(self.esgf_credentials, 'wb') as fh:
188 for cred in creds:
File ~/miniconda3/envs/downclim/lib/python3.13/site-packages/myproxy/client/__init__.py:1453, in MyProxyClient.logon(self, username, passphrase, credname, lifetime, keyPair, certReq, nBitsForKey, bootstrap, updateTrustRoots, authnGetTrustRootsCall, sslCertFile, sslKeyFile, sslKeyFilePassphrase)
1450 else:
1451 getTrustRootsKw = {}
-> 1453 self.getTrustRoots(writeToCACertDir=True,
1454 bootstrap=bootstrap,
1455 **getTrustRootsKw)
1457 lifetime = lifetime or self.proxyCertLifetime
1459 # Basic sanity check on username to avoid overhead on server
File ~/miniconda3/envs/downclim/lib/python3.13/site-packages/myproxy/client/__init__.py:1609, in MyProxyClient.getTrustRoots(self, username, passphrase, writeToCACertDir, bootstrap)
1607 # Set-up SSL connection
1608 conn = self._initConnection(verifyPeerWithTrustRoots=(not bootstrap))
-> 1609 conn.connect((self.hostname, self.port))
1611 # send globus compatibility stuff
1612 conn.write(self.__class__.GLOBUS_INIT_MSG)
File ~/miniconda3/envs/downclim/lib/python3.13/site-packages/OpenSSL/SSL.py:2390, in Connection.connect(self, addr)
2381 """
2382 Call the :meth:`connect` method of the underlying socket and set up SSL
2383 on the socket, using the :class:`Context` object supplied to this
(...)
2387 :return: What the socket's connect method returns
2388 """
2389 _lib.SSL_set_connect_state(self._ssl)
-> 2390 return self._socket.connect(addr)
ConnectionRefusedError: [Errno 61] Connection refused
This request returns no result.
Note Actually, there are CORDEX simulations matching all the requirements except the “experiment” one : there is no set with both ‘rcp26’ and ‘historical’ experiments.
Save your requested simulations¶
You now have two DataFrames containing the information about the simulations you requested.
Of course, now you can manually remove any simulations you are not interested in.
Finally, you can save them to a csv file to keep track of your request. This will be useful to process downloading and downscaling using Downclim later on.
[ ]:
save_simulations_list(
cordex_simulations = cordex_simulations,
cmip6_simulations = cmip6_simulations,
output_file = "simulations_list.csv"
)
[ ]: