{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# An extensive walk through `DownClim`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```{note}\n",
    "Although all the described methods and functions describe below are fully functional, this is not the recommended way to use the library, as you don't fully take advantage of the workflow and abstractions provided. Take a look at other notebooks from the `examples` section to see how to use `DownClim` in a more efficient way.\n",
    "\n",
    "However, maybe at some point you'll find yourself your own way to use `DownClim` !\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Definition of the workflow\n",
    "\n",
    "In this intentiionaly simplified example, we will use the `DownClim` library to:\n",
    "- download `CHELSA` data for the baseline period 1980-1981\n",
    "- download `CHIRPS` and `GSHTD` data for the evaluation period 2005-2006\n",
    "- download `CORDEX` and `CMIP6` data for the baseline, evaluation and projection periods"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Authentication\n",
    "\n",
    "First, we need to authenticate to Earth Engine to retrieve data from `GSHTD`, `CHIRPS` and `CMIP6`.\n",
    "\n",
    "Although we also need to authenticate to `ESGF` for `CORDEX` data, login information can be provided in a separate file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<p>To authorize access needed by Earth Engine, open the following\n",
       "        URL in a web browser and follow the instructions:</p>\n",
       "        <p><a href=https://code.earthengine.google.com/client-auth?scopes=https%3A//www.googleapis.com/auth/earthengine%20https%3A//www.googleapis.com/auth/cloud-platform%20https%3A//www.googleapis.com/auth/devstorage.full_control&request_id=Pw9PT7NEt_8jlpWQXMHNgKkJxBtOdkSco7XiH4Yuimo&tc=GO1I884-SY-NFOvo1p5wgYpCPmj6bWwyoKT6UDNGesQ&cc=oOMTgYkSAhbgwRzQ__vPQSkzrNXS2md-PZ5sO-O2xSc>https://code.earthengine.google.com/client-auth?scopes=https%3A//www.googleapis.com/auth/earthengine%20https%3A//www.googleapis.com/auth/cloud-platform%20https%3A//www.googleapis.com/auth/devstorage.full_control&request_id=Pw9PT7NEt_8jlpWQXMHNgKkJxBtOdkSco7XiH4Yuimo&tc=GO1I884-SY-NFOvo1p5wgYpCPmj6bWwyoKT6UDNGesQ&cc=oOMTgYkSAhbgwRzQ__vPQSkzrNXS2md-PZ5sO-O2xSc</a></p>\n",
       "        <p>The authorization workflow will generate a code, which you should paste in the box below.</p>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Successfully saved authorization token.\n"
     ]
    }
   ],
   "source": [
    "from __future__ import annotations\n",
    "\n",
    "import ee\n",
    "\n",
    "ee.Authenticate()\n",
    "ee.Initialize(opt_url=\"https://earthengine-highvolume.googleapis.com\", project=\"downclim\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Areas of interest\n",
    "\n",
    "We first need to define the areas of interest. This will define the boundaries for downloading the data and the area for which we will be predicting the downscaling.\n",
    "\n",
    "There are multiple ways to define the areas of interest (cf. api link to `get_aoi`). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from downclim.aoi import get_aoi\n",
    "\n",
    "aoi1 = get_aoi(\"Vanuatu\")\n",
    "aoi2 = get_aoi((10, 10, 20, 20, \"box\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download CHELSA data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading CHELSA data...\n",
      "Getting year \"1980\" for variables \"pr\" and areas of interest : \"['Vanuatu', 'box']\"\n",
      "Getting year \"1981\" for variables \"pr\" and areas of interest : \"['Vanuatu', 'box']\"\n",
      "Concatenating data for area of interest : Vanuatu\n",
      "Concatenating data for area of interest : Vanuatu\n",
      "saving file ./results/tmp/chelsa/chelsa_Vanuatu_pr_1981.nc\n",
      "Concatenating data for area of interest : box\n",
      "saving file ./results/tmp/chelsa/chelsa_Vanuatu_pr_1980.nc\n",
      "Concatenating data for area of interest : box\n",
      "saving file ./results/tmp/chelsa/chelsa_box_pr_1981.nc\n",
      "saving file ./results/tmp/chelsa/chelsa_box_pr_1980.nc\n",
      "Getting year \"1980\" for variables \"tas\" and areas of interest : \"['Vanuatu', 'box']\"Getting year \"1981\" for variables \"tas\" and areas of interest : \"['Vanuatu', 'box']\"\n",
      "\n",
      "Concatenating data for area of interest : Vanuatu\n",
      "Concatenating data for area of interest : Vanuatu\n",
      "saving file ./results/tmp/chelsa/chelsa_Vanuatu_tas_1981.nc\n",
      "Concatenating data for area of interest : box\n",
      "saving file ./results/tmp/chelsa/chelsa_Vanuatu_tas_1980.nc\n",
      "Concatenating data for area of interest : box\n",
      "saving file ./results/tmp/chelsa/chelsa_box_tas_1981.nc\n",
      "saving file ./results/tmp/chelsa/chelsa_box_tas_1980.nc\n",
      "Getting year \"1981\" for variables \"tasmin\" and areas of interest : \"['Vanuatu', 'box']\"Getting year \"1980\" for variables \"tasmin\" and areas of interest : \"['Vanuatu', 'box']\"\n",
      "\n",
      "Concatenating data for area of interest : Vanuatu\n",
      "Concatenating data for area of interest : Vanuatu\n",
      "saving file ./results/tmp/chelsa/chelsa_Vanuatu_tasmin_1981.nc\n",
      "Concatenating data for area of interest : box\n",
      "saving file ./results/tmp/chelsa/chelsa_Vanuatu_tasmin_1980.nc\n",
      "Concatenating data for area of interest : box\n",
      "saving file ./results/tmp/chelsa/chelsa_box_tasmin_1980.nc\n",
      "saving file ./results/tmp/chelsa/chelsa_box_tasmin_1981.nc\n",
      "Getting year \"1980\" for variables \"tasmax\" and areas of interest : \"['Vanuatu', 'box']\"\n",
      "Getting year \"1981\" for variables \"tasmax\" and areas of interest : \"['Vanuatu', 'box']\"\n",
      "Concatenating data for area of interest : Vanuatu\n",
      "Concatenating data for area of interest : Vanuatu\n",
      "saving file ./results/tmp/chelsa/chelsa_Vanuatu_tasmax_1981.nc\n",
      "Concatenating data for area of interest : box\n",
      "saving file ./results/tmp/chelsa/chelsa_Vanuatu_tasmax_1980.nc\n",
      "Concatenating data for area of interest : box\n",
      "saving file ./results/tmp/chelsa/chelsa_box_tasmax_1980.nc\n",
      "saving file ./results/tmp/chelsa/chelsa_box_tasmax_1981.nc\n",
      "Merging files by aoi...\n",
      "Merging files for area Vanuatu...\n",
      "Merging files for area box...\n"
     ]
    }
   ],
   "source": [
    "from downclim.dataset.chelsa2 import get_chelsa2\n",
    "\n",
    "get_chelsa2(\n",
    "    aoi=[aoi1, aoi2],\n",
    "    variable=[\"pr\", \"tas\", \"tasmin\", \"tasmax\"],\n",
    "    period=(1980, 1981),\n",
    "    keep_tmp_dir=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Downalod CHIRPS and GSHTD data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Already connected to Earth Engine with project 'downclim'.\n",
      "Downloading CHIRPS data...\n",
      "Getting CHIRPS data for period : \"(2006, 2007)\" and area of interest : \"Vanuatu\"\n",
      "Getting CHIRPS data for period : \"(2006, 2007)\" and area of interest : \"box\"\n",
      "Already connected to Earth Engine with project 'downclim'.\n",
      "Downloading GSHTD data...\n",
      "Getting GSHTD data for period : \"(2006, 2007)\" and variable : \"tas\" on area of interest : \"Vanuatu\"\n",
      "Getting GSHTD data for period : \"(2006, 2007)\" and variable : \"tasmin\" on area of interest : \"Vanuatu\"\n",
      "Getting GSHTD data for period : \"(2006, 2007)\" and variable : \"tasmax\" on area of interest : \"Vanuatu\"\n",
      "Getting GSHTD data for period : \"(2006, 2007)\" and variable : \"tas\" on area of interest : \"box\"\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: earthengine-highvolume.googleapis.com. Connection pool size: 10\n",
      "WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: earthengine-highvolume.googleapis.com. Connection pool size: 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Getting GSHTD data for period : \"(2006, 2007)\" and variable : \"tasmin\" on area of interest : \"box\"\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: earthengine-highvolume.googleapis.com. Connection pool size: 10\n",
      "WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: earthengine-highvolume.googleapis.com. Connection pool size: 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Getting GSHTD data for period : \"(2006, 2007)\" and variable : \"tasmax\" on area of interest : \"box\"\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: earthengine-highvolume.googleapis.com. Connection pool size: 10\n",
      "WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: earthengine-highvolume.googleapis.com. Connection pool size: 10\n"
     ]
    }
   ],
   "source": [
    "from downclim.dataset.chirps import get_chirps\n",
    "from downclim.dataset.gshtd import get_gshtd\n",
    "\n",
    "get_chirps(\n",
    "    aoi=[aoi1, aoi2],\n",
    "    period=(2006, 2007),\n",
    "    project = \"downclim\", # project name for Earth Engine\n",
    ")\n",
    "\n",
    "get_gshtd(\n",
    "    aoi=[aoi1, aoi2],\n",
    "    variable=[\"tas\", \"tasmin\", \"tasmax\"],\n",
    "    period=(2006, 2007),\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download climate simulations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download `CORDEX` data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "ename": "FileNotFoundError",
     "evalue": "[Errno 2] No such file or directory: 'config/esgf_credential.yaml'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mFileNotFoundError\u001b[0m                         Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[7], line 19\u001b[0m\n\u001b[1;32m      8\u001b[0m cordex_context \u001b[38;5;241m=\u001b[39m CORDEXContext(\n\u001b[1;32m      9\u001b[0m     domain\u001b[38;5;241m=\u001b[39m[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mAUS-22\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mAFR-44\u001b[39m\u001b[38;5;124m\"\u001b[39m],\n\u001b[1;32m     10\u001b[0m     experiment\u001b[38;5;241m=\u001b[39m[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mhistorical\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrcp26\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrcp85\u001b[39m\u001b[38;5;124m\"\u001b[39m],\n\u001b[1;32m     11\u001b[0m     frequency\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmon\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m     12\u001b[0m     variable\u001b[38;5;241m=\u001b[39m[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mpr\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtas\u001b[39m\u001b[38;5;124m\"\u001b[39m],\n\u001b[1;32m     13\u001b[0m )\n\u001b[1;32m     15\u001b[0m cordex_simulations \u001b[38;5;241m=\u001b[39m list_available_cordex_simulations(\n\u001b[1;32m     16\u001b[0m     cordex_context, esgf_credential\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m../../config/esgf_credential.yaml\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m     17\u001b[0m )\n\u001b[0;32m---> 19\u001b[0m cordex_simulations \u001b[38;5;241m=\u001b[39m \u001b[43mget_download_scripts\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcordex_simulations\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mesgf_credential\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mconfig/esgf_credential.yaml\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[1;32m     20\u001b[0m cordex_simulations\u001b[38;5;241m.\u001b[39mto_csv(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mresults/cordex/cordex_simulations.csv\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n",
      "File \u001b[0;32m~/Documents/Modeles/Codes/DownClim/src/downclim/dataset/cordex.py:279\u001b[0m, in \u001b[0;36mget_download_scripts\u001b[0;34m(simulations, esgf_credential)\u001b[0m\n\u001b[1;32m    268\u001b[0m \u001b[38;5;250m\u001b[39m\u001b[38;5;124;03m\"\"\"Get the esgf download scripts for the simulations described in the DataFrame.\u001b[39;00m\n\u001b[1;32m    269\u001b[0m \n\u001b[1;32m    270\u001b[0m \u001b[38;5;124;03mArgs:\u001b[39;00m\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    275\u001b[0m \u001b[38;5;124;03m   pd.DataFrame: same DataFrame as the input with the download scripts added.\u001b[39;00m\n\u001b[1;32m    276\u001b[0m \u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m    278\u001b[0m \u001b[38;5;66;03m# connect\u001b[39;00m\n\u001b[0;32m--> 279\u001b[0m connector \u001b[38;5;241m=\u001b[39m \u001b[43mconnect_to_esgf\u001b[49m\u001b[43m(\u001b[49m\u001b[43mesgf_credential\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mserver\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mDataProduct\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mCORDEX\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43murl\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    281\u001b[0m facets \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m, \u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;241m.\u001b[39mjoin(simulations_columns)\n\u001b[1;32m    282\u001b[0m cordex_scripts \u001b[38;5;241m=\u001b[39m []\n",
      "File \u001b[0;32m~/Documents/Modeles/Codes/DownClim/src/downclim/dataset/connectors.py:79\u001b[0m, in \u001b[0;36mconnect_to_esgf\u001b[0;34m(esgf_credential, server)\u001b[0m\n\u001b[1;32m     65\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21mconnect_to_esgf\u001b[39m(esgf_credential: \u001b[38;5;28mstr\u001b[39m, server: \u001b[38;5;28mstr\u001b[39m) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m pyesgf\u001b[38;5;241m.\u001b[39mSearchConnection:\n\u001b[1;32m     66\u001b[0m \u001b[38;5;250m    \u001b[39m\u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m     67\u001b[0m \u001b[38;5;124;03m    Connector to ESGF server.\u001b[39;00m\n\u001b[1;32m     68\u001b[0m \n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m     77\u001b[0m \n\u001b[1;32m     78\u001b[0m \u001b[38;5;124;03m    \"\"\"\u001b[39;00m\n\u001b[0;32m---> 79\u001b[0m     \u001b[38;5;28;01mwith\u001b[39;00m \u001b[43mPath\u001b[49m\u001b[43m(\u001b[49m\u001b[43mesgf_credential\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mopen\u001b[49m\u001b[43m(\u001b[49m\u001b[43mencoding\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mutf-8\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m \u001b[38;5;28;01mas\u001b[39;00m stream:\n\u001b[1;32m     80\u001b[0m         creds \u001b[38;5;241m=\u001b[39m yaml\u001b[38;5;241m.\u001b[39msafe_load(stream)\n\u001b[1;32m     81\u001b[0m     lm \u001b[38;5;241m=\u001b[39m LogonManager()\n",
      "File \u001b[0;32m~/miniconda3/envs/downclim/lib/python3.13/pathlib/_local.py:537\u001b[0m, in \u001b[0;36mPath.open\u001b[0;34m(self, mode, buffering, encoding, errors, newline)\u001b[0m\n\u001b[1;32m    535\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mb\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;129;01min\u001b[39;00m mode:\n\u001b[1;32m    536\u001b[0m     encoding \u001b[38;5;241m=\u001b[39m io\u001b[38;5;241m.\u001b[39mtext_encoding(encoding)\n\u001b[0;32m--> 537\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mio\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mopen\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmode\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mbuffering\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mencoding\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43merrors\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mnewline\u001b[49m\u001b[43m)\u001b[49m\n",
      "\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: 'config/esgf_credential.yaml'"
     ]
    }
   ],
   "source": [
    "from downclim.dataset.cordex import (\n",
    "    CORDEXContext,\n",
    "    get_cordex_from_list,\n",
    "    get_download_scripts,\n",
    "    list_available_cordex_simulations,\n",
    ")\n",
    "\n",
    "# Define the research context for CORDEX data\n",
    "cordex_context = CORDEXContext(\n",
    "    domain=[\"AUS-22\", \"AFR-44\"],\n",
    "    experiment=[\"historical\", \"rcp26\", \"rcp85\"],\n",
    "    frequency=\"mon\",\n",
    "    variable=[\"pr\", \"tas\"],\n",
    ")\n",
    "\n",
    "# Use the previously defined context to list available simulations\n",
    "# ! This step requires ESGF credentials\n",
    "cordex_simulations = list_available_cordex_simulations(\n",
    "    cordex_context, esgf_credential=\"../../config/esgf_credential.yaml\"\n",
    ")\n",
    "\n",
    "# Retrieve download scripts for the available simulations\n",
    "cordex_simulations = get_download_scripts(cordex_simulations, esgf_credential=\"config/esgf_credential.yaml\")\n",
    "\n",
    "# Save the list of simulations to a CSV file. This can be useful if you want to perform hand-selection.\n",
    "cordex_simulations.to_csv(\"results/cordex/cordex_simulations.csv\")\n",
    "\n",
    "get_cordex_from_list(\n",
    "    aoi=[aoi1, aoi2],\n",
    "    cordex_simulations=cordex_simulations,\n",
    "    historical_period=(1980, 1981),\n",
    "    evaluation_period=(2006, 2007),\n",
    "    projection_period=(2071, 2072),\n",
    "    output_dir=\"./results/cordex\",\n",
    "    tmp_dir = \"./results/tmp/cordex\",\n",
    "    keep_tmp_dir = True,\n",
    "    esgf_credential=\"config/esgf_credential.yaml\"\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download `CMIP6` data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "ename": "OSError",
     "evalue": "Cannot save file into a non-existent directory: 'results/cmip6'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mOSError\u001b[0m                                   Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[10], line 17\u001b[0m\n\u001b[1;32m      7\u001b[0m cmip6_context \u001b[38;5;241m=\u001b[39m CMIP6Context(\n\u001b[1;32m      8\u001b[0m     project\u001b[38;5;241m=\u001b[39m[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mScenarioMIP\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mCMIP\u001b[39m\u001b[38;5;124m\"\u001b[39m],\n\u001b[1;32m      9\u001b[0m     institute\u001b[38;5;241m=\u001b[39m[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mNOAA-GFDL\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mCMCC\u001b[39m\u001b[38;5;124m\"\u001b[39m],\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m     14\u001b[0m     grid_label\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mgn\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m     15\u001b[0m )\n\u001b[1;32m     16\u001b[0m cmip6_simulations \u001b[38;5;241m=\u001b[39m list_available_cmip6_simulations(cmip6_context)\n\u001b[0;32m---> 17\u001b[0m \u001b[43mcmip6_simulations\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mto_csv\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mresults/cmip6/cmip6_simulations.csv\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[1;32m     19\u001b[0m get_cmip6_from_list(\n\u001b[1;32m     20\u001b[0m     aoi\u001b[38;5;241m=\u001b[39maois,\n\u001b[1;32m     21\u001b[0m     cmip6_simulations\u001b[38;5;241m=\u001b[39mcmip6_simulations,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m     25\u001b[0m     output_dir\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m./results/cmip6\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m     26\u001b[0m )\n",
      "File \u001b[0;32m~/miniconda3/envs/downclim/lib/python3.13/site-packages/pandas/util/_decorators.py:333\u001b[0m, in \u001b[0;36mdeprecate_nonkeyword_arguments.<locals>.decorate.<locals>.wrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    327\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(args) \u001b[38;5;241m>\u001b[39m num_allow_args:\n\u001b[1;32m    328\u001b[0m     warnings\u001b[38;5;241m.\u001b[39mwarn(\n\u001b[1;32m    329\u001b[0m         msg\u001b[38;5;241m.\u001b[39mformat(arguments\u001b[38;5;241m=\u001b[39m_format_argument_list(allow_args)),\n\u001b[1;32m    330\u001b[0m         \u001b[38;5;167;01mFutureWarning\u001b[39;00m,\n\u001b[1;32m    331\u001b[0m         stacklevel\u001b[38;5;241m=\u001b[39mfind_stack_level(),\n\u001b[1;32m    332\u001b[0m     )\n\u001b[0;32m--> 333\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/miniconda3/envs/downclim/lib/python3.13/site-packages/pandas/core/generic.py:3967\u001b[0m, in \u001b[0;36mNDFrame.to_csv\u001b[0;34m(self, path_or_buf, sep, na_rep, float_format, columns, header, index, index_label, mode, encoding, compression, quoting, quotechar, lineterminator, chunksize, date_format, doublequote, escapechar, decimal, errors, storage_options)\u001b[0m\n\u001b[1;32m   3956\u001b[0m df \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(\u001b[38;5;28mself\u001b[39m, ABCDataFrame) \u001b[38;5;28;01melse\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mto_frame()\n\u001b[1;32m   3958\u001b[0m formatter \u001b[38;5;241m=\u001b[39m DataFrameFormatter(\n\u001b[1;32m   3959\u001b[0m     frame\u001b[38;5;241m=\u001b[39mdf,\n\u001b[1;32m   3960\u001b[0m     header\u001b[38;5;241m=\u001b[39mheader,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m   3964\u001b[0m     decimal\u001b[38;5;241m=\u001b[39mdecimal,\n\u001b[1;32m   3965\u001b[0m )\n\u001b[0;32m-> 3967\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mDataFrameRenderer\u001b[49m\u001b[43m(\u001b[49m\u001b[43mformatter\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mto_csv\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m   3968\u001b[0m \u001b[43m    \u001b[49m\u001b[43mpath_or_buf\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3969\u001b[0m \u001b[43m    \u001b[49m\u001b[43mlineterminator\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mlineterminator\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3970\u001b[0m \u001b[43m    \u001b[49m\u001b[43msep\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43msep\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3971\u001b[0m \u001b[43m    \u001b[49m\u001b[43mencoding\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mencoding\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3972\u001b[0m \u001b[43m    \u001b[49m\u001b[43merrors\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43merrors\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3973\u001b[0m \u001b[43m    \u001b[49m\u001b[43mcompression\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcompression\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3974\u001b[0m \u001b[43m    \u001b[49m\u001b[43mquoting\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mquoting\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3975\u001b[0m \u001b[43m    \u001b[49m\u001b[43mcolumns\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcolumns\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3976\u001b[0m \u001b[43m    \u001b[49m\u001b[43mindex_label\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mindex_label\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3977\u001b[0m \u001b[43m    \u001b[49m\u001b[43mmode\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmode\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3978\u001b[0m \u001b[43m    \u001b[49m\u001b[43mchunksize\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mchunksize\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3979\u001b[0m \u001b[43m    \u001b[49m\u001b[43mquotechar\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mquotechar\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3980\u001b[0m \u001b[43m    \u001b[49m\u001b[43mdate_format\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdate_format\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3981\u001b[0m \u001b[43m    \u001b[49m\u001b[43mdoublequote\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdoublequote\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3982\u001b[0m \u001b[43m    \u001b[49m\u001b[43mescapechar\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mescapechar\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3983\u001b[0m \u001b[43m    \u001b[49m\u001b[43mstorage_options\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstorage_options\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   3984\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/miniconda3/envs/downclim/lib/python3.13/site-packages/pandas/io/formats/format.py:1014\u001b[0m, in \u001b[0;36mDataFrameRenderer.to_csv\u001b[0;34m(self, path_or_buf, encoding, sep, columns, index_label, mode, compression, quoting, quotechar, lineterminator, chunksize, date_format, doublequote, escapechar, errors, storage_options)\u001b[0m\n\u001b[1;32m    993\u001b[0m     created_buffer \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mFalse\u001b[39;00m\n\u001b[1;32m    995\u001b[0m csv_formatter \u001b[38;5;241m=\u001b[39m CSVFormatter(\n\u001b[1;32m    996\u001b[0m     path_or_buf\u001b[38;5;241m=\u001b[39mpath_or_buf,\n\u001b[1;32m    997\u001b[0m     lineterminator\u001b[38;5;241m=\u001b[39mlineterminator,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m   1012\u001b[0m     formatter\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mfmt,\n\u001b[1;32m   1013\u001b[0m )\n\u001b[0;32m-> 1014\u001b[0m \u001b[43mcsv_formatter\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msave\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1016\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m created_buffer:\n\u001b[1;32m   1017\u001b[0m     \u001b[38;5;28;01massert\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(path_or_buf, StringIO)\n",
      "File \u001b[0;32m~/miniconda3/envs/downclim/lib/python3.13/site-packages/pandas/io/formats/csvs.py:251\u001b[0m, in \u001b[0;36mCSVFormatter.save\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    247\u001b[0m \u001b[38;5;250m\u001b[39m\u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m    248\u001b[0m \u001b[38;5;124;03mCreate the writer & save.\u001b[39;00m\n\u001b[1;32m    249\u001b[0m \u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m    250\u001b[0m \u001b[38;5;66;03m# apply compression and byte/text conversion\u001b[39;00m\n\u001b[0;32m--> 251\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m \u001b[43mget_handle\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    252\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfilepath_or_buffer\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    253\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmode\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    254\u001b[0m \u001b[43m    \u001b[49m\u001b[43mencoding\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mencoding\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    255\u001b[0m \u001b[43m    \u001b[49m\u001b[43merrors\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43merrors\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    256\u001b[0m \u001b[43m    \u001b[49m\u001b[43mcompression\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcompression\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    257\u001b[0m \u001b[43m    \u001b[49m\u001b[43mstorage_options\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mstorage_options\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    258\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m \u001b[38;5;28;01mas\u001b[39;00m handles:\n\u001b[1;32m    259\u001b[0m     \u001b[38;5;66;03m# Note: self.encoding is irrelevant here\u001b[39;00m\n\u001b[1;32m    260\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mwriter \u001b[38;5;241m=\u001b[39m csvlib\u001b[38;5;241m.\u001b[39mwriter(\n\u001b[1;32m    261\u001b[0m         handles\u001b[38;5;241m.\u001b[39mhandle,\n\u001b[1;32m    262\u001b[0m         lineterminator\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mlineterminator,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    267\u001b[0m         quotechar\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mquotechar,\n\u001b[1;32m    268\u001b[0m     )\n\u001b[1;32m    270\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_save()\n",
      "File \u001b[0;32m~/miniconda3/envs/downclim/lib/python3.13/site-packages/pandas/io/common.py:749\u001b[0m, in \u001b[0;36mget_handle\u001b[0;34m(path_or_buf, mode, encoding, compression, memory_map, is_text, errors, storage_options)\u001b[0m\n\u001b[1;32m    747\u001b[0m \u001b[38;5;66;03m# Only for write methods\u001b[39;00m\n\u001b[1;32m    748\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mr\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;129;01min\u001b[39;00m mode \u001b[38;5;129;01mand\u001b[39;00m is_path:\n\u001b[0;32m--> 749\u001b[0m     \u001b[43mcheck_parent_directory\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mstr\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mhandle\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    751\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m compression:\n\u001b[1;32m    752\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m compression \u001b[38;5;241m!=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mzstd\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[1;32m    753\u001b[0m         \u001b[38;5;66;03m# compression libraries do not like an explicit text-mode\u001b[39;00m\n",
      "File \u001b[0;32m~/miniconda3/envs/downclim/lib/python3.13/site-packages/pandas/io/common.py:616\u001b[0m, in \u001b[0;36mcheck_parent_directory\u001b[0;34m(path)\u001b[0m\n\u001b[1;32m    614\u001b[0m parent \u001b[38;5;241m=\u001b[39m Path(path)\u001b[38;5;241m.\u001b[39mparent\n\u001b[1;32m    615\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m parent\u001b[38;5;241m.\u001b[39mis_dir():\n\u001b[0;32m--> 616\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mOSError\u001b[39;00m(\u001b[38;5;124mrf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mCannot save file into a non-existent directory: \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mparent\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n",
      "\u001b[0;31mOSError\u001b[0m: Cannot save file into a non-existent directory: 'results/cmip6'"
     ]
    }
   ],
   "source": [
    "from downclim.dataset.cmip6 import (\n",
    "                                    CMIP6Context,\n",
    "                                    get_cmip6,\n",
    "                                    list_available_cmip6_simulations,\n",
    ")\n",
    "\n",
    "cmip6_context = CMIP6Context(\n",
    "    project=[\"ScenarioMIP\", \"CMIP\"],\n",
    "    institute=[\"NOAA-GFDL\", \"CMCC\"],\n",
    "    experiment=[\"ssp126\", \"historical\"],\n",
    "    ensemble=\"r1i1p1f1\",\n",
    "    frequency=\"mon\",\n",
    "    variable=[\"tas\", \"pr\"],\n",
    "    grid_label=\"gn\",\n",
    ")\n",
    "cmip6_simulations = list_available_cmip6_simulations(cmip6_context)\n",
    "cmip6_simulations.to_csv(\"results/cmip6/cmip6_simulations.csv\")\n",
    "\n",
    "get_cmip6(\n",
    "    aoi=[aoi1, aoi2],\n",
    "    cmip6_simulations=cmip6_simulations,\n",
    "    historical_period=(1980, 1981),\n",
    "    evaluation_period=(2006, 2007),\n",
    "    projection_period=(2071, 2072),\n",
    "    output_dir=\"./results/cmip6\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Downscaling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "downclim",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}