{
"cells": [
{
"cell_type": "markdown",
"id": "0",
"metadata": {},
"source": [
"# `clisops` regridding functionalities - powered by `xesmf`\n",
"\n",
"The regridding functionalities of clisops consist of the regridding operator/function `regrid` in `clisops.ops`, allowing one-line remapping of `xarray.Datasets` or `xarray.DataArrays`, while orchestrating the use of classes and functions in `clisops.core`:\n",
"- the `Grid` and `Weights` classes, to check and pre-process input as well as output grids and to generate the remapping weights\n",
"- a `regrid` function, performing the remapping by applying the generated weights on the input data\n",
"\n",
"For the weight generation and the regridding, the [xESMF](https://github.com/pangeo-data/xESMF) `Regridder` class is used, which itself allows an easy application of many of the remapping functionalities of [ESMF](https://earthsystemmodeling.org/)/[ESMPy](https://github.com/esmf-org/esmf/blob/develop/src/addon/ESMPy/README.md)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1",
"metadata": {},
"outputs": [],
"source": [
"# Imports\n",
"\n",
"%matplotlib inline\n",
"# Set required environment variable for ESMPy\n",
"import os\n",
"from pathlib import Path\n",
"\n",
"import cartopy.crs as ccrs\n",
"import cf_xarray as cfxr\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"import xarray as xr\n",
"from matplotlib.collections import PolyCollection\n",
"\n",
"os.environ[\"ESMFMKFILE\"] = str(Path(os.__file__).parent.parent / \"esmf.mk\")\n",
"import xesmf as xe\n",
"from roocs_grids import grid_annotations\n",
"\n",
"import clisops as cl # atm. the regrid-main-martin branch of clisops\n",
"import clisops.core as clore\n",
"import clisops.ops as clops\n",
"\n",
"print(f\"Using xarray in version {xr.__version__}\")\n",
"print(f\"Using cf_xarray in version {cfxr.__version__}\")\n",
"print(f\"Using xESMF in version {xe.__version__}\")\n",
"print(f\"Using clisops in version {cl.__version__}\")\n",
"\n",
"xr.set_options(display_style=\"html\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2",
"metadata": {},
"outputs": [],
"source": [
"# Initialize the testing data\n",
"import clisops.utils.testing as clite\n",
"\n",
"Stratus = clite.stratus(\n",
" repo=clite.ESGF_TEST_DATA_REPO_URL, branch=clite.ESGF_TEST_DATA_VERSION, cache_dir=clite.ESGF_TEST_DATA_CACHE_DIR\n",
")\n",
"\n",
"mini_esgf_data = (\n",
" clite.get_esgf_file_paths(Stratus.path) or clite.get_esgf_glob_paths(Stratus.path) or clite.get_kerchunk_datasets()\n",
")"
]
},
{
"cell_type": "markdown",
"id": "3",
"metadata": {},
"source": [
"## `clisops.ops.regrid`\n",
"\n",
"One-line remapping with `clisops.ops.regrid`:\n",
"\n",
"```python\n",
"\n",
"def regrid(\n",
" ds: Union[xr.Dataset, str, Path],\n",
" *,\n",
" method: Optional[str] = \"nearest_s2d\",\n",
" adaptive_masking_threshold: Optional[Union[int, float]] = 0.5,\n",
" grid: Optional[Union[xr.Dataset, xr.DataArray, int, float, tuple, str]] = \"adaptive\",\n",
" output_dir: Optional[Union[str, Path]] = None,\n",
" output_type: Optional[str] = \"netcdf\",\n",
" split_method: Optional[str] = \"time:auto\",\n",
" file_namer: Optional[str] = \"standard\",\n",
" keep_attrs: Optional[Union[bool, str]] = True,\n",
") -> List[Union[xr.Dataset, str]]:\n",
" pass\n",
"```\n",
"\n",
"The different options for the `method`, `grid` and `adaptive_masking_threshold` parameters are described in below sections:\n",
"\n",
"* [clisops.core.Grid](#clisops.core.Grid)\n",
"* [clisops.core.Weights](#clisops.core.Weights)\n",
"* [clisops.core.regrid](#clisops.core.regrid)\n"
]
},
{
"cell_type": "markdown",
"id": "4",
"metadata": {},
"source": [
"### Remap a global `xarray.Dataset` to a global 2.5 degree grid using the bilinear method\n",
"\n",
"#### Load the dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5",
"metadata": {},
"outputs": [],
"source": [
"ds_vert = xr.open_dataset(mini_esgf_data[\"CMIP6_ATM_VERT_ONE_TIMESTEP\"])\n",
"ds_vert"
]
},
{
"cell_type": "markdown",
"id": "6",
"metadata": {},
"source": [
"#### Take a look at the grid"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7",
"metadata": {},
"outputs": [],
"source": [
"# Create 2D coordinate variables\n",
"lon, lat = np.meshgrid(ds_vert[\"lon\"].data, ds_vert[\"lat\"].data)\n",
"\n",
"# Plot\n",
"plt.figure(figsize=(8, 5))\n",
"plt.scatter(lon[::3, ::3], lat[::3, ::3], s=0.5)\n",
"plt.xlabel(\"lon\")\n",
"plt.ylabel(\"lat\")"
]
},
{
"cell_type": "markdown",
"id": "8",
"metadata": {},
"source": [
"#### Remap to global 2.5 degree grid with the bilinear method"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9",
"metadata": {},
"outputs": [],
"source": [
"ds_remap = clops.regrid(ds_vert, method=\"bilinear\", grid=\"2pt5deg\", output_type=\"xarray\")[0]\n",
"ds_remap"
]
},
{
"cell_type": "markdown",
"id": "10",
"metadata": {},
"source": [
"#### Plot the remapped data next to the source data"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "11",
"metadata": {},
"outputs": [],
"source": [
"fig, axes = plt.subplots(ncols=2, figsize=(18, 4), subplot_kw={\"projection\": ccrs.PlateCarree()})\n",
"for ax in axes:\n",
" ax.coastlines()\n",
"\n",
"# Source data\n",
"ds_vert.o3.isel(time=0, lev=0).plot.pcolormesh(ax=axes[0], x=\"lon\", y=\"lat\", shading=\"auto\")\n",
"axes[0].title.set_text(\"Source - MPI-ESM1-2-LR ECHAM6 (T63L47, ~1.9° resolution)\")\n",
"# Remapped data\n",
"ds_remap.o3.isel(time=0, lev=0).plot.pcolormesh(ax=axes[1], x=\"lon\", y=\"lat\", shading=\"auto\")\n",
"axes[1].title.set_text(\"Target - regular lat-lon (2.5° resolution)\")"
]
},
{
"cell_type": "markdown",
"id": "12",
"metadata": {},
"source": [
"### Remap regional `xarray.Dataset` to a regional grid of adaptive resolution using the bilinear method\n",
"Adaptive resolution means, that the regular lat-lon target grid will have approximately the same resolution as the source grid."
]
},
{
"cell_type": "markdown",
"id": "13",
"metadata": {},
"source": [
"#### Load the dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "14",
"metadata": {},
"outputs": [],
"source": [
"ds_cordex = xr.open_dataset(mini_esgf_data[\"CORDEX_TAS_ONE_TIMESTEP\"])\n",
"ds_cordex"
]
},
{
"cell_type": "markdown",
"id": "15",
"metadata": {},
"source": [
"#### Take a look at the grid"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "16",
"metadata": {},
"outputs": [],
"source": [
"plt.figure(figsize=(8, 5))\n",
"plt.scatter(ds_cordex[\"lon\"][::4, ::4], ds_cordex[\"lat\"][::4, ::4], s=0.1)\n",
"plt.xlabel(\"lon\")\n",
"plt.ylabel(\"lat\")"
]
},
{
"cell_type": "markdown",
"id": "17",
"metadata": {},
"source": [
"#### Remap to regional regular lat-lon grid of adaptive resolution with the bilinear method"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "18",
"metadata": {},
"outputs": [],
"source": [
"ds_remap = clops.regrid(ds_cordex, method=\"bilinear\", grid=\"adaptive\", output_type=\"xarray\")[0]\n",
"ds_remap"
]
},
{
"cell_type": "markdown",
"id": "19",
"metadata": {},
"source": [
"#### Plot the remapped data next to the source data"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "20",
"metadata": {},
"outputs": [],
"source": [
"fig, axes = plt.subplots(ncols=2, figsize=(18, 4), subplot_kw={\"projection\": ccrs.PlateCarree()})\n",
"for ax in axes:\n",
" ax.coastlines()\n",
"\n",
"# Source data\n",
"ds_cordex.tas.isel(time=0).plot.pcolormesh(ax=axes[0], x=\"lon\", y=\"lat\", shading=\"auto\", cmap=\"RdBu_r\")\n",
"axes[0].title.set_text(\"Source - GERICS-REMO2015 (EUR22, ~0.22° resolution)\")\n",
"# Remapped data\n",
"ds_remap.tas.isel(time=0).plot.pcolormesh(ax=axes[1], x=\"lon\", y=\"lat\", shading=\"auto\", cmap=\"RdBu_r\")\n",
"axes[1].title.set_text(\"Target - regional regular lat-lon (adaptive resolution)\")"
]
},
{
"cell_type": "markdown",
"id": "21",
"metadata": {},
"source": [
"### Remap unstructured `xarray.Dataset` to a global grid of adaptive resolution using the nearest neighbour method\n",
"\n",
"For unstructured grids, at least for the moment, only the nearest neighbour remapping method is supported.\n",
"\n",
"#### Load the dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "22",
"metadata": {},
"outputs": [],
"source": [
"ds_icono = xr.open_dataset(mini_esgf_data[\"CMIP6_UNSTR_VERT_ICON_O\"])\n",
"ds_icono"
]
},
{
"cell_type": "markdown",
"id": "23",
"metadata": {},
"source": [
"#### Take a look at the grid"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "24",
"metadata": {},
"outputs": [],
"source": [
"plt.figure(figsize=(16, 9))\n",
"plt.scatter(ds_icono[\"longitude\"][::2], ds_icono[\"latitude\"][::2], s=0.05)\n",
"plt.xlabel(\"lon\")\n",
"plt.ylabel(\"lat\")"
]
},
{
"cell_type": "markdown",
"id": "25",
"metadata": {},
"source": [
"#### Remap to global grid of adaptive resolution with the nearest neighbour method"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "26",
"metadata": {},
"outputs": [],
"source": [
"ds_remap = clops.regrid(ds_icono, method=\"nearest_s2d\", grid=\"adaptive\", output_type=\"xarray\")[0]\n",
"ds_remap"
]
},
{
"cell_type": "markdown",
"id": "27",
"metadata": {},
"source": [
"#### Plot source data and remapped data\n",
"\n",
"Below we use matplotlib to plot the data on the unstructured ICON-Grid, which requires some processing of the cell bounds. Using [psyplot](https://psyplot.github.io/) to plot the unstructured data is a simpler alternative. `psyplot` / `psymaps` is however not included in the `clisops` dependencies as it comes with complex dependencies itself.\n",
"\n",
"```python\n",
"# Source data - psyplot\n",
"ds_icono_path = ds_icono.encoding[\"source\"]\n",
"maps = psy.plot.mapplot(\n",
" ds_icono_path,\n",
" cmap=\"RdBu_r\",\n",
" title=\"Source - ICON-ESM-LR ICON-O (Ruby-0, 40km resolution)\",\n",
" time=[0],\n",
" lev=[0]\n",
")\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "28",
"metadata": {},
"outputs": [],
"source": [
"# Source data\n",
"\n",
"# Function to fix cell_bounds crossing the meridian\n",
"# to avoid horizontal lines in the plot\n",
"def fix_meridian_crossing(lon_bnds, lat_bnds):\n",
" \"\"\"Shift longitudes from -180,180 to 0,360.\"\"\"\n",
" fixed_lon = []\n",
" fixed_lat = []\n",
" for lon, lat in zip(lon_bnds, lat_bnds, strict=False):\n",
" lon = np.array(lon)\n",
" lat = np.array(lat)\n",
" lon_range = lon.max() - lon.min()\n",
" if lon_range > 180: # crosses meridian\n",
" # Shift longitudes < 180 by +360\n",
" lon_corrected = np.where(lon < 180, lon + 360, lon)\n",
" else:\n",
" lon_corrected = lon\n",
" fixed_lon.append(lon_corrected)\n",
" fixed_lat.append(lat)\n",
" return np.array(fixed_lon), np.array(fixed_lat)\n",
"\n",
"\n",
"fixed_lon, fixed_lat = fix_meridian_crossing(ds_icono.longitude_bnds.values, ds_icono.latitude_bnds.values)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "29",
"metadata": {},
"outputs": [],
"source": [
"# Source data\n",
"fig, ax = plt.subplots(subplot_kw={\"projection\": ccrs.PlateCarree()}, figsize=(10, 6))\n",
"\n",
"# Plot as collection of polygons\n",
"pc = PolyCollection(\n",
" [list(zip(*lonlat, strict=False)) for lonlat in zip(fixed_lon, fixed_lat, strict=False)],\n",
" array=ds_icono[\"thetao\"].isel(time=0, lev=0).values.squeeze(),\n",
" cmap=\"RdBu_r\",\n",
" edgecolor=\"face\",\n",
" transform=ccrs.PlateCarree(),\n",
")\n",
"ax.add_collection(pc)\n",
"\n",
"# Adjust colorbar, title, labels and ticks\n",
"xticks = range(-180, 181, 60)\n",
"yticks = range(-90, 91, 30)\n",
"ax.set_xticks(xticks, crs=ccrs.PlateCarree())\n",
"ax.set_yticks(yticks, crs=ccrs.PlateCarree())\n",
"ax.set_xlabel(\"Longitude\", fontsize=12)\n",
"ax.set_ylabel(\"Latitude\", fontsize=12)\n",
"ax.autoscale()\n",
"ax.coastlines(resolution=\"10m\")\n",
"fig.colorbar(pc, ax=ax, label=\"thetao\", shrink=0.6)\n",
"fig.suptitle(\"Source - ICON-ESM-LR ICON-O (Ruby-0, 40km resolution)\", fontsize=14, x=0.45, y=0.8)\n",
"\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "30",
"metadata": {},
"outputs": [],
"source": [
"# Remapped data\n",
"plt.figure(figsize=(9, 4))\n",
"ax = plt.axes(projection=ccrs.PlateCarree())\n",
"ds_remap.thetao.isel(time=0, lev=0).plot.pcolormesh(\n",
" ax=ax, x=\"lon\", y=\"lat\", shading=\"auto\", cmap=\"RdBu_r\", vmin=-1, vmax=40\n",
")\n",
"ax.title.set_text(\"Target - regular lat-lon (adaptive resolution)\")\n",
"ax.coastlines()"
]
},
{
"cell_type": "markdown",
"id": "31",
"metadata": {},
"source": [
"\n",
"\n",
"## `clisops.core.Grid`"
]
},
{
"cell_type": "markdown",
"id": "32",
"metadata": {},
"source": [
"### Create a grid object from an `xarray.Dataset`\n",
"\n",
"#### Load the dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "33",
"metadata": {},
"outputs": [],
"source": [
"dso = xr.open_dataset(mini_esgf_data[\"CMIP6_TOS_ONE_TIME_STEP\"])\n",
"dso"
]
},
{
"cell_type": "markdown",
"id": "34",
"metadata": {},
"source": [
"#### Create the Grid object"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "35",
"metadata": {},
"outputs": [],
"source": [
"grido = clore.Grid(ds=dso)\n",
"grido"
]
},
{
"cell_type": "markdown",
"id": "36",
"metadata": {},
"source": [
"The `xarray.Dataset` is attached to the `clisops.core.Grid` object. Auxiliary coordinates and data variables have been (re)set appropriately."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "37",
"metadata": {},
"outputs": [],
"source": [
"grido.ds"
]
},
{
"cell_type": "markdown",
"id": "38",
"metadata": {},
"source": [
"#### Plot the data"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "39",
"metadata": {},
"outputs": [],
"source": [
"plt.figure(figsize=(9, 4))\n",
"ax = plt.axes(projection=ccrs.PlateCarree())\n",
"grido.ds.tos.isel(time=0).plot.pcolormesh(\n",
" ax=ax, x=grido.lon, y=grido.lat, shading=\"auto\", cmap=\"RdBu_r\", vmin=-1, vmax=40\n",
")\n",
"ax.coastlines()"
]
},
{
"cell_type": "markdown",
"id": "40",
"metadata": {},
"source": [
"### Create a grid object from an `xarray.DataArray`\n",
"\n",
"Note that `xarray.DataArray` objects do not support the bounds of coordinate variables to be defined.\n",
"\n",
"#### Extract tos `DataArray`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "41",
"metadata": {},
"outputs": [],
"source": [
"dao = dso.tos\n",
"dao"
]
},
{
"cell_type": "markdown",
"id": "42",
"metadata": {},
"source": [
"#### Create Grid object for MPIOM tos dataarray:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "43",
"metadata": {},
"outputs": [],
"source": [
"grido_tos = clore.Grid(ds=dao)\n",
"grido_tos"
]
},
{
"cell_type": "markdown",
"id": "44",
"metadata": {},
"source": [
"### Create a grid object using a `grid_instructor`\n",
"\n",
"* global grid: `grid_instructor = (lon_step, lat_step)` or `grid_instructor = step`\n",
"* regional grid:`grid_instructor = (lon_start, lon_end, lon_step, lat_start, lat_end, lat_step)` or `grid_instructor = (start, end, step)`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "45",
"metadata": {},
"outputs": [],
"source": [
"grid_1deg = clore.Grid(grid_instructor=1)\n",
"grid_1deg"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "46",
"metadata": {},
"outputs": [],
"source": [
"grid_1degx2deg_regional = clore.Grid(grid_instructor=(0.0, 90.0, 1.0, 35.0, 50.0, 2.0))\n",
"grid_1degx2deg_regional"
]
},
{
"cell_type": "markdown",
"id": "47",
"metadata": {},
"source": [
"### Create a grid object using a `grid_id`\n",
"\n",
"Makes use of the predefined grids of `roocs_grids`, which is a collection of grids used for example for the [IPCC Atlas](https://github.com/IPCC-WG1/Atlas/tree/main/reference-grids) and for [CMIP6 Regridding Weights generation](https://docs.google.com/document/d/1BfVVsKAk9MAsOYstwFSWI2ZBt5mrO_Nmcu7rLGDuL08/edit)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "48",
"metadata": {},
"outputs": [],
"source": [
"for key, gridinfo in grid_annotations.items():\n",
" print(f\"- {key:20} {gridinfo}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "49",
"metadata": {},
"outputs": [],
"source": [
"grid_era5 = clore.Grid(grid_id=\"0pt25deg_era5\")\n",
"grid_era5"
]
},
{
"cell_type": "markdown",
"id": "50",
"metadata": {},
"source": [
"### `clisops.core.Grid` objects can be compared to one another\n",
"\n",
"Optional verbose output gives information on where the grids differ: lat, lon, lat_bnds, lon_bnds, mask?\n",
"\n",
"#### Compare the tos dataset to the tos dataarray"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "51",
"metadata": {},
"outputs": [],
"source": [
"comp = grido.compare_grid(grido_tos, verbose=True)\n",
"print(\"Grids are equal?\", comp)"
]
},
{
"cell_type": "markdown",
"id": "52",
"metadata": {},
"source": [
"#### Compare both 0.25° ERA5 Grids"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "53",
"metadata": {},
"outputs": [],
"source": [
"# Create the Grid object\n",
"grid_era5_lsm = clore.Grid(grid_id=\"0pt25deg_era5_lsm\", compute_bounds=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "54",
"metadata": {},
"outputs": [],
"source": [
"# Compare\n",
"comp = grid_era5.compare_grid(grid_era5_lsm, verbose=True)\n",
"print(\"Grids are equal?\", comp)"
]
},
{
"cell_type": "markdown",
"id": "55",
"metadata": {},
"source": [
"### Strip `clisops.core.Grid` objects of all `data_vars` and `coords` unrelated to the horizontal grid"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "56",
"metadata": {},
"outputs": [],
"source": [
"grid_era5_lsm.ds"
]
},
{
"cell_type": "markdown",
"id": "57",
"metadata": {},
"source": [
"The parameter `keep_attrs` can be set, the default is `False`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "58",
"metadata": {},
"outputs": [],
"source": [
"grid_era5_lsm._drop_vars(keep_attrs=False)\n",
"grid_era5_lsm.ds"
]
},
{
"cell_type": "markdown",
"id": "59",
"metadata": {},
"source": [
"### Transfer coordinate variables between `clisops.core.Grid` objects that are unrelated to the horizontal grid\n",
"\n",
"The parameter `keep_attrs` can be set, the default is `True`. All settings for `keep_attrs` are described later in section [clisops.core.regrid](#clisops.core.regrid).\n",
"\n",
"#### Load the dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "60",
"metadata": {},
"outputs": [],
"source": [
"ds_vert = xr.open_dataset(mini_esgf_data[\"CMIP6_ATM_VERT_ONE_TIMESTEP\"])\n",
"ds_vert"
]
},
{
"cell_type": "markdown",
"id": "61",
"metadata": {},
"source": [
"#### Create grid object"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "62",
"metadata": {},
"outputs": [],
"source": [
"grid_vert = clore.Grid(ds_vert)\n",
"grid_vert"
]
},
{
"cell_type": "markdown",
"id": "63",
"metadata": {},
"source": [
"#### Transfer the coordinates to the ERA5 grid object"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "64",
"metadata": {},
"outputs": [],
"source": [
"grid_era5_lsm._transfer_coords(grid_vert, keep_attrs=True)\n",
"grid_era5_lsm.ds"
]
},
{
"cell_type": "markdown",
"id": "65",
"metadata": {},
"source": [
"\n",
"\n",
"## `clisops.core.Weights`\n",
"\n",
"Create regridding weights to regrid between two grids. Supported are the following of [xESMF's remapping methods](https://pangeo-xesmf.readthedocs.io/en/latest/notebooks/Compare_algorithms.html):\n",
"* `nearest_s2d`\n",
"* `bilinear`\n",
"* `conservative`\n",
"* `patch`"
]
},
{
"cell_type": "markdown",
"id": "66",
"metadata": {},
"source": [
"### Create 2-degree target grid"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "67",
"metadata": {},
"outputs": [],
"source": [
"grid_2deg = clore.Grid(grid_id=\"2deg_lsm\", compute_bounds=True)\n",
"grid_2deg"
]
},
{
"cell_type": "markdown",
"id": "68",
"metadata": {},
"source": [
"### Create conservative remapping weights using the `clisops.core.Weights` class\n",
"`grid_in` and `grid_out` are `Grid` objects"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "69",
"metadata": {},
"outputs": [],
"source": [
"%time weights = clore.Weights(grid_in = grido, grid_out = grid_2deg, method=\"conservative\")"
]
},
{
"cell_type": "markdown",
"id": "70",
"metadata": {},
"source": [
"### Local weights cache\n",
"\n",
"Weights are cached on disk and do not have to be created more than once. The default cache directory is platform-dependent and set via the package `platformdirs`. For Linux it is `'/home/my_user/.local/share/clisops/weights_dir'` and can optionally be adjusted:\n",
"\n",
"- permanently by modifying the parameter `grid_weights: local_weights_dir` in the `roocs.ini` configuration file that can be found in the clisops installation directory\n",
"- or temporarily via:\n",
"```python\n",
"from clisops import core as clore\n",
"clore.weights_cache_init(\"/dir/for/weights/cache\")\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "71",
"metadata": {},
"outputs": [],
"source": [
"from clisops.core.regrid import CONFIG\n",
"\n",
"print(CONFIG[\"clisops:grid_weights\"][\"local_weights_dir\"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "72",
"metadata": {},
"outputs": [],
"source": [
"!ls -sh {CONFIG[\"clisops:grid_weights\"][\"local_weights_dir\"]}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "73",
"metadata": {},
"outputs": [],
"source": [
"!cat {CONFIG[\"clisops:grid_weights\"][\"local_weights_dir\"]}/weights_*_conservative.json"
]
},
{
"cell_type": "markdown",
"id": "74",
"metadata": {},
"source": [
"Now the weights will be read directly from the cache"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "75",
"metadata": {},
"outputs": [],
"source": [
"%time weights = clore.Weights(grid_in = grido, grid_out = grid_2deg, method=\"conservative\")"
]
},
{
"cell_type": "markdown",
"id": "76",
"metadata": {},
"source": [
"The weights cache can be flushed, which removes all weight and grid files as well as the json files holding the metadata. To see what would be removed, one can use the `dryrun=True` parameter. To re-initialize the weights cache in a different directory, one can use the `weights_dir_init=\"/new/dir/for/weights/cache\"` parameter. Even when re-initializing the weights cache under a new path, using `clore.weights_cache_flush`, no directory is getting removed, only above listed files. When `dryrun` is not set, the files that are getting deleted can be displayed with `verbose=True`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "77",
"metadata": {},
"outputs": [],
"source": [
"clore.weights_cache_flush(dryrun=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "78",
"metadata": {},
"outputs": [],
"source": [
"clore.weights_cache_flush(verbose=True)"
]
},
{
"cell_type": "markdown",
"id": "79",
"metadata": {},
"source": [
"\n",
"\n",
"## `clisops.core.regrid`\n",
"\n",
"This function allows to perform the eventual regridding and provides a resulting `xarray.Dataset`\n",
"\n",
"```python\n",
"def regrid(\n",
" grid_in: Grid,\n",
" grid_out: Grid,\n",
" weights: Weights,\n",
" adaptive_masking_threshold: Optional[float] = 0.5,\n",
" keep_attrs: Optional[bool] = True,\n",
"):\n",
" pass\n",
"```\n",
"\n",
"* `grid_in` and `grid_out` are `Grid` objects, `weights` is a `Weights` object.\n",
"* `adaptive_masking_threshold` (AMT) A value within the [0., 1.] interval that defines the maximum `RATIO` of missing_values amongst the total number of data values contributing to the calculation of the target grid cell value. For a fraction [0., AMT[ of the contributing source data missing, the target grid cell will be set to missing_value, else, it will be re-normalized by the factor `1./(1.-RATIO)`. Thus, if AMT is set to 1, all source grid cells that contribute to a target grid cell must be missing in order for the target grid cell to be defined as missing itself. Values greater than 1 or less than 0 will cause adaptive masking to be turned off. This adaptive masking technique allows to reuse generated weights for differently masked data (e.g. land-sea masks or orographic masks that vary with depth / height).\n",
"* `keep_attrs` can have the following settings:\n",
" * `True` : The resulting `xarray.Dataset` will have all attributes of `grid_in.ds.attrs`, despite attributes that have to be added and altered due to the new grid.\n",
" * `False` : The resulting `xarray.Dataset` will have no attributes despite attributes generated by the regridding process.\n",
" * `\"target\"` : The resulting `xarray.Dataset` will have all attributes of `grid_out.ds.attrs`, despite attributes generated by the regridding process. Not recommended.\n",
"\n",
"### In the following an example showing the function application and the effect of the adaptive masking."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "80",
"metadata": {},
"outputs": [],
"source": [
"ds_out_amt0 = clore.regrid(grido, grid_2deg, weights, adaptive_masking_threshold=-1) # noqa: F821"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "81",
"metadata": {},
"outputs": [],
"source": [
"ds_out_amt1 = clore.regrid(grido, grid_2deg, weights, adaptive_masking_threshold=0.5) # noqa: F821"
]
},
{
"cell_type": "markdown",
"id": "82",
"metadata": {},
"source": [
"#### Plot the resulting data"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "83",
"metadata": {},
"outputs": [],
"source": [
"# Create panel plot of regridded data (global)\n",
"fig, axes = plt.subplots(\n",
" ncols=2,\n",
" nrows=1,\n",
" figsize=(18, 5), # global\n",
" subplot_kw={\"projection\": ccrs.PlateCarree()},\n",
")\n",
"\n",
"ds_out_amt0[\"tos\"].isel(time=0).plot.pcolormesh(ax=axes[0], vmin=0, vmax=30, cmap=\"plasma\")\n",
"axes[0].title.set_text(\"Target (2° regular lat-lon) - No adaptive masking\")\n",
"\n",
"ds_out_amt1[\"tos\"].isel(time=0).plot.pcolormesh(ax=axes[1], vmin=0, vmax=30, cmap=\"plasma\")\n",
"axes[1].title.set_text(\"Target (2° regular lat-lon) - Adaptive masking\")\n",
"\n",
"for axis in axes.flatten():\n",
" axis.coastlines()\n",
" axis.set_xlabel(\"lon\")\n",
" axis.set_ylabel(\"lat\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "84",
"metadata": {},
"outputs": [],
"source": [
"# Create a panel plot of regridded data (Japan)\n",
"fig, axes = plt.subplots(\n",
" ncols=3,\n",
" nrows=1,\n",
" figsize=(18, 4), # Japan\n",
" subplot_kw={\"projection\": ccrs.PlateCarree()},\n",
")\n",
"\n",
"grido.ds.tos.isel(time=0).plot.pcolormesh(\n",
" ax=axes[0], x=grido.lon, y=grido.lat, vmin=0, vmax=30, cmap=\"plasma\", shading=\"auto\"\n",
")\n",
"axes[0].title.set_text(\"Source - MPI-ESM1-2-HR MPIOM (TP04, ~0.4° resolution)\")\n",
"\n",
"ds_out_amt0[\"tos\"].isel(time=0).plot.pcolormesh(ax=axes[1], vmin=0, vmax=30, cmap=\"plasma\")\n",
"axes[1].title.set_text(\"Target - No adaptive masking\")\n",
"\n",
"ds_out_amt1[\"tos\"].isel(time=0).plot.pcolormesh(ax=axes[2], vmin=0, vmax=30, cmap=\"plasma\")\n",
"axes[2].title.set_text(\"Target - Adaptive masking\")\n",
"\n",
"for axis in axes.flatten():\n",
" axis.coastlines()\n",
" axis.set_xlabel(\"lon\")\n",
" axis.set_ylabel(\"lat\")\n",
" axis.set_xlim([125, 150])\n",
" axis.set_ylim([25, 50])"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.11"
}
},
"nbformat": 4,
"nbformat_minor": 5
}