{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "source": [
    "# Datasets accessible with `DownClim`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`DownClim` is designed to be a simple and easy-to-use tool for downloading climate datasets. \n",
    "\n",
    "In this page, we provide a brief description of all the datasets that are accessible with `DownClim`, as well as a short example of how to download them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First define the areas of interest (AOI) we want to work on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import annotations\n",
    "\n",
    "import ee\n",
    "\n",
    "from downclim.aoi import get_aoi\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "aoi1 = get_aoi(\"Vanuatu\")     # AOI by name, from GADM\n",
    "aoi2 = get_aoi((10, 10, 20, 20, \"box\"))   # AOI by coordinates, associated with a name\n",
    "aois = [aoi1, aoi2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset on Google Earth Engine"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`DownClim` uses [Google Earth Engine (GEE)](https://earthengine.google.com/) to access and download some climate datasets.\n",
    "\n",
    "In particular, we use it to download [CHIRPS](#chirps-dataset) and [GSHTD](#gshtd-dataset) datasets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```{warning}\n",
    "Products hosted on Earth Engine now require authentication with a google account AND to have an Earth Engine project. Under certain conditions (among which academic projects), the use of the service remains free.\n",
    "\n",
    "Check instructions [here](https://developers.google.com/earth-engine)\n",
    "```\n",
    "\n",
    "You first need (only the first time on your machine) to authenticate and follow the instructions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then you need to have an Earth Engine project. You can create one and manage your projects [here](https://console.cloud.google.com/earth-engine/welcome). Having a project is free for non profit use. Full information can be found [here](https://developers.google.com/earth-engine/guides/transition_to_cloud_projects?hl=fr).\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "your_ee_project_id = \"downclim\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<p>To authorize access needed by Earth Engine, open the following\n",
       "        URL in a web browser and follow the instructions:</p>\n",
       "        <p><a href=https://code.earthengine.google.com/client-auth?scopes=https%3A//www.googleapis.com/auth/earthengine%20https%3A//www.googleapis.com/auth/cloud-platform%20https%3A//www.googleapis.com/auth/devstorage.full_control&request_id=5St2v-5M1LKkqQ_djZQ3yeFKwmx_IGFtnxsRXWXuZUw&tc=v7YgH0VaCEcRGiXG4dxbrgKz3P14nSAxiof0gD6Nhnk&cc=wGQZ7XWqkCr1OT2D-1W5dQrHd2zLyIqZXJ3mnpMEXuc>https://code.earthengine.google.com/client-auth?scopes=https%3A//www.googleapis.com/auth/earthengine%20https%3A//www.googleapis.com/auth/cloud-platform%20https%3A//www.googleapis.com/auth/devstorage.full_control&request_id=5St2v-5M1LKkqQ_djZQ3yeFKwmx_IGFtnxsRXWXuZUw&tc=v7YgH0VaCEcRGiXG4dxbrgKz3P14nSAxiof0gD6Nhnk&cc=wGQZ7XWqkCr1OT2D-1W5dQrHd2zLyIqZXJ3mnpMEXuc</a></p>\n",
       "        <p>The authorization workflow will generate a code, which you should paste in the box below.</p>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Successfully saved authorization token.\n"
     ]
    }
   ],
   "source": [
    "# Authenticate to Earth Engine\n",
    "# ee.Authenticate() if necessary, only need to do this once on your machine\n",
    "ee.Initialize(opt_url=\"https://earthengine-highvolume.googleapis.com\", project=your_ee_project_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Historical datasets based on observations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CHELSA2 dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[CHELSA2 dataset](https://chelsa-climate.org/) is a very high-resolution (30 arc sec, ~1km) downscaled climate dataset for the entire globe. Currently it is updated in version `v2.1`, and data is available from 1979 to 2020. Reference product used for downscaling is ERA5 reanalysis data.\n",
    "\n",
    "If you use monthly mean variables (the only option available so far), available variables are:\n",
    "\n",
    "| Variable | Description |\n",
    "| --- | --- |\n",
    "| `clt` | Cloud cover |\n",
    "| `cmi` | Climatic moisture index (difference between precipitation amount and potential evapotranspiration) |\n",
    "| `hurs` | Near surface relative humidity |\n",
    "| `pet` | Potential evapotranspiration |\n",
    "| `pr` | Precipitation |\n",
    "| `rsds` | Downward shortwave radiation flux |\n",
    "| `sfcWind` | Near surface wind speed |\n",
    "| `tas` | near-surface mean air temperature |\n",
    "| `tasmin` | near-surface minimum air temperature |\n",
    "| `tasmax` | near-surface maximum air temperature |\n",
    "| `vpd` | Vapour pressure deficit |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's download the `CHELSA2` dataset for the areas of interest defined above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading CHELSA data...\n",
      "CHELSA data for year '1980' and variable 'pr' already downloaded. Not downloading,\n",
      "              but the behaviour of the function is not affected.\n",
      "              If this is not the desired behavior, please remove the file(s) from the temporary folder\n",
      "              ./results/tmp/chelsa and rerun the function.CHELSA data for year '1981' and variable 'pr' already downloaded. Not downloading,\n",
      "              but the behaviour of the function is not affected.\n",
      "              If this is not the desired behavior, please remove the file(s) from the temporary folder\n",
      "              ./results/tmp/chelsa and rerun the function.\n",
      "\n",
      "CHELSA data for year '1980' and variable 'tas' already downloaded. Not downloading,\n",
      "              but the behaviour of the function is not affected.\n",
      "              If this is not the desired behavior, please remove the file(s) from the temporary folder\n",
      "              ./results/tmp/chelsa and rerun the function.\n",
      "CHELSA data for year '1981' and variable 'tas' already downloaded. Not downloading,\n",
      "              but the behaviour of the function is not affected.\n",
      "              If this is not the desired behavior, please remove the file(s) from the temporary folder\n",
      "              ./results/tmp/chelsa and rerun the function.\n",
      "CHELSA data for year '1980' and variable 'tasmin' already downloaded. Not downloading,\n",
      "              but the behaviour of the function is not affected.\n",
      "              If this is not the desired behavior, please remove the file(s) from the temporary folder\n",
      "              ./results/tmp/chelsa and rerun the function.CHELSA data for year '1981' and variable 'tasmin' already downloaded. Not downloading,\n",
      "              but the behaviour of the function is not affected.\n",
      "              If this is not the desired behavior, please remove the file(s) from the temporary folder\n",
      "              ./results/tmp/chelsa and rerun the function.\n",
      "\n",
      "CHELSA data for year '1980' and variable 'tasmax' already downloaded. Not downloading,\n",
      "              but the behaviour of the function is not affected.\n",
      "              If this is not the desired behavior, please remove the file(s) from the temporary folder\n",
      "              ./results/tmp/chelsa and rerun the function.CHELSA data for year '1981' and variable 'tasmax' already downloaded. Not downloading,\n",
      "              but the behaviour of the function is not affected.\n",
      "              If this is not the desired behavior, please remove the file(s) from the temporary folder\n",
      "              ./results/tmp/chelsa and rerun the function.\n",
      "\n",
      "Merging files by aoi...\n",
      "File for area Vanuatu and period (1980, 1981) already exists. Not downloading.\n",
      "                Please make sure this is the expected behaviour\n",
      "File for area box and period (1980, 1981) already exists. Not downloading.\n",
      "                Please make sure this is the expected behaviour\n"
     ]
    }
   ],
   "source": [
    "from downclim.dataset.chelsa2 import get_chelsa2\n",
    "\n",
    "get_chelsa2(\n",
    "    aoi=aois,\n",
    "    variable=[\"pr\", \"tas\", \"tasmin\", \"tasmax\"],\n",
    "    period=(1980, 1981),\n",
    "    keep_tmp_dir=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CHIRPS dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[CHIRPS dataset](https://www.chc.ucsb.edu/data/chirps) is a high-resolution (0.05°, ~5km) quasi-global (50°S - 50°N) gridded rainfall dataset. Data is available from 1981 to near-present. Data is available at daily time scale, and is retrieved from the [Earth Engine Data Catalog](https://developers.google.com/earth-engine/datasets/catalog/UCSB-CHG_CHIRPS_DAILY?hl=fr)\n",
    "\n",
    "Only one variable is available for this dataset:\n",
    "| Variable | Description |\n",
    "| --- | --- |\n",
    "| `pr` | Precipitation |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's download the `CHIRPS` dataset for the areas of interest defined above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Already connected to Earth Engine with project 'downclim'.\n",
      "Downloading CHIRPS data...\n",
      "Getting CHIRPS data for period : \"(1981, 1982)\" and area of interest : \"Vanuatu\"\n",
      "Getting CHIRPS data for period : \"(1981, 1982)\" and area of interest : \"box\"\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: earthengine-highvolume.googleapis.com. Connection pool size: 10\n"
     ]
    }
   ],
   "source": [
    "from downclim.dataset.chirps import get_chirps\n",
    "\n",
    "get_chirps(\n",
    "    aoi=aois,\n",
    "    period=(1981, 1982),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GSHTD dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The \"Global Seamless High-resolution Temperature Dataset\" (citation) is a high-resolution (30 arc sec, ~1km) temperature dataset for the entire globe. It is based on the MODIS land surface temperature product and is available from 2001 to 2020. It uses the Estimation of Temperature Difference (ETD) method and employs a machine learning algorithm to estimate monthly values of temperature mean, min and max.\n",
    "\n",
    "It is available on the [Google Earth Engne data catalog](https://gee-community-catalog.org/projects/gshtd/).\n",
    "\n",
    "Available variables are:\n",
    "\n",
    "| Variable | Description |\n",
    "| --- | --- |\n",
    "| `tas` | near-surface mean air temperature |\n",
    "| `tasmin` | near-surface minimum air temperature |\n",
    "| `tasmax` | near-surface maximum air temperature |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Already connected to Earth Engine with project 'downclim'.\n",
      "Downloading GSHTD data...\n",
      "Getting GSHTD data for period : \"(2001, 2002)\" and variable : \"tas\" on area of interest : \"Vanuatu\"\n",
      "Getting GSHTD data for period : \"(2001, 2002)\" and variable : \"tasmin\" on area of interest : \"Vanuatu\"\n",
      "Getting GSHTD data for period : \"(2001, 2002)\" and variable : \"tasmax\" on area of interest : \"Vanuatu\"\n",
      "Getting GSHTD data for period : \"(2001, 2002)\" and variable : \"tas\" on area of interest : \"box\"\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: earthengine-highvolume.googleapis.com. Connection pool size: 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Getting GSHTD data for period : \"(2001, 2002)\" and variable : \"tasmin\" on area of interest : \"box\"\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: earthengine-highvolume.googleapis.com. Connection pool size: 10\n",
      "WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: earthengine-highvolume.googleapis.com. Connection pool size: 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Getting GSHTD data for period : \"(2001, 2002)\" and variable : \"tasmax\" on area of interest : \"box\"\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: earthengine-highvolume.googleapis.com. Connection pool size: 10\n",
      "WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: earthengine-highvolume.googleapis.com. Connection pool size: 10\n"
     ]
    }
   ],
   "source": [
    "from downclim.dataset.gshtd import get_gshtd\n",
    "\n",
    "# Warning : starts from 2001\n",
    "get_gshtd(\n",
    "    aoi=aois,\n",
    "    variable=[\"tas\", \"tasmin\", \"tasmax\"],\n",
    "    period=(2001, 2002),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simulations datasets for historical and future periods"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`DownClim` provides easy access to both `CMIP6` and `CORDEX` simulations datasets. Please refer to the [\"Get available simulations\"](./get_available_simulations.html) page to see how to efficiently retrieve simulations datasets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CMIP6 dataset\n",
    "\n",
    "[CMIP6](https://pcmdi.llnl.gov/CMIP6/) stands for \"Coupled Model Intercomparison Project Phase 6\". It is a collection of climate model simulations from the World Climate Research Programme (WCRP). \n",
    "\n",
    "Access to this dataset is provided by [Google Cloud](https://console.cloud.google.com/marketplace/details/noaa-public/cmip6). This database may not be complete and up-to-date, as it is an ongoing project to transfer all the data to the cloud. However, cloud access is particularly efficient."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CORDEX dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```{note}\n",
    "To access this dataset, you need to create an account on the ESGF portal.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[CORDEX](https://cordex.org/) stands for \"Coordinated Regional Climate Downscaling Experiment\". As CMIP6, it is a WCRP framework to evaluate regional climate models simulations for different regions of the world. Regional climate models, applied over limited areas and driven by global climate models, are used to provide information at higher resolution than the global models. This is particularly useful for impact and adaptation assessments.\n",
    "\n",
    "You can find a description of the different regions [here](https://cordex.org/domains/cordex-domain-description/).\n",
    "\n",
    "Access to this dataset is provided by the Earth System Grid Federation (ESGF) infrastructure, and can be accessed by [one of its portals](https://cordex.org/data-access/cordex-cmip5-data/cordex-cmip5-esgf/). It is free of access but credentials are required."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "downclim",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}