{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "e06a27bf",
   "metadata": {},
   "source": [
    "# CQL2 Filtering\n",
    "\n",
    "This notebook demonstrates the use of pystac-client to use [CQL2 filtering](https://github.com/radiantearth/stac-api-spec/tree/master/fragments/filter). The server needs to support this and advertise conformance as the `https://api.stacspec.org/v1.0.0-rc.1/item-search#filter` class in the `conformsTo` attribute of the root API.\n",
    "\n",
    "**This should be considered an experimental feature. This notebook uses the Microsoft Planetary Computer API, as it is currently the only public CQL2 implementation.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b65de617",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pystac_client import Client\n",
    "\n",
    "# set pystac_client logger to DEBUG to see API calls\n",
    "import logging\n",
    "logging.basicConfig()\n",
    "logger = logging.getLogger('pystac_client')\n",
    "logger.setLevel(logging.INFO)\n",
    "\n",
    "# function for creating GeoDataFrame from Items\n",
    "from copy import deepcopy\n",
    "import geopandas as gpd\n",
    "import pandas as pd\n",
    "from shapely.geometry import shape\n",
    "\n",
    "# convert a list of STAC Items into a GeoDataFrame\n",
    "def items_to_geodataframe(items):\n",
    "    _items = []\n",
    "    for i in items:\n",
    "        _i = deepcopy(i)\n",
    "        _i['geometry'] = shape(_i['geometry'])\n",
    "        _items.append(_i)\n",
    "    gdf = gpd.GeoDataFrame(pd.json_normalize(_items))\n",
    "    for field in ['properties.datetime', 'properties.created', 'properties.updated']:\n",
    "        if field in gdf:\n",
    "            gdf[field] = pd.to_datetime(gdf[field])\n",
    "    gdf.set_index('properties.datetime', inplace=True)\n",
    "    return gdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "98942e75",
   "metadata": {},
   "outputs": [],
   "source": [
    "# STAC API root URL\n",
    "URL = 'https://planetarycomputer.microsoft.com/api/stac/v1'\n",
    "\n",
    "# custom headers\n",
    "headers = []\n",
    "\n",
    "cat = Client.open(URL, headers=headers)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e16077c",
   "metadata": {},
   "source": [
    "## Initial Search Parameters\n",
    "\n",
    "Here we perform a search with the `Client.search` function, providing a geometry (`intersects`) a datetime range (`datetime`), and filtering by Item properties (`filter`) using CQL2-JSON."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d8af6334",
   "metadata": {},
   "outputs": [],
   "source": [
    "# AOI around Delfzijl, in the north of The Netherlands\n",
    "geom = {\n",
    "    \"type\": \"Polygon\",\n",
    "    \"coordinates\": [\n",
    "      [\n",
    "        [\n",
    "          6.42425537109375,\n",
    "          53.174765470134616\n",
    "        ],\n",
    "        [\n",
    "          7.344360351562499,\n",
    "          53.174765470134616\n",
    "        ],\n",
    "        [\n",
    "          7.344360351562499,\n",
    "          53.67393435835391\n",
    "        ],\n",
    "        [\n",
    "          6.42425537109375,\n",
    "          53.67393435835391\n",
    "        ],\n",
    "        [\n",
    "          6.42425537109375,\n",
    "          53.174765470134616\n",
    "        ]\n",
    "      ]\n",
    "    ]\n",
    "}\n",
    "\n",
    "params = {\n",
    "    \"max_items\": 100,\n",
    "    \"collections\": \"landsat-8-c2-l2\",\n",
    "    \"intersects\": geom,\n",
    "    \"datetime\": \"2018-01-01/2020-12-31\",\n",
    "}\n",
    "\n",
    "import hvplot.pandas\n",
    "import json\n",
    "\n",
    "# reusable search function\n",
    "def search_fetch_plot(params, filt):\n",
    "    # limit sets the # of items per page so we can see multiple pages getting fetched\n",
    "    params['filter'] = filt\n",
    "    search = cat.search(**params)\n",
    "    items = list(search.items_as_dicts()) # safe b/c we set max_items = 100\n",
    "    # DataFrame\n",
    "    items_df = pd.DataFrame(items_to_geodataframe(items))\n",
    "    print(f\"{len(items_df.index)} items found\")\n",
    "    field = 'properties.eo:cloud_cover'\n",
    "    return items_df.hvplot(y=field, label=json.dumps(filt), frame_height=500, frame_width=800)    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44d3bc04",
   "metadata": {},
   "source": [
    "## CQL2 Filters\n",
    "\n",
    "Below are examples of several different CQL2 filters on the `eo:cloud_cover` property. Up to 100 Items are fetched and the eo:cloud_cover values plotted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dfc0e759",
   "metadata": {},
   "outputs": [],
   "source": [
    "filt = {\n",
    "    \"op\": \"lte\",\n",
    "    \"args\": [{\"property\": \"eo:cloud_cover\"}, 10]\n",
    "}\n",
    "\n",
    "search_fetch_plot(params, filt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c2f9ca1",
   "metadata": {},
   "outputs": [],
   "source": [
    "filt = {\n",
    "    \"op\": \"gte\",\n",
    "    \"args\" : [{\"property\": \"eo:cloud_cover\"}, 80]\n",
    "}\n",
    "\n",
    "search_fetch_plot(params, filt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "109f673c",
   "metadata": {},
   "outputs": [],
   "source": [
    "filt = { \n",
    "    \"op\": \"and\",\n",
    "    \"args\": [\n",
    "      { \n",
    "          \"op\":\"lte\", \n",
    "          \"args\": [{\"property\": \"eo:cloud_cover\"}, 60]\n",
    "      },\n",
    "      { \n",
    "          \"op\": \"gte\", \n",
    "          \"args\": [{\"property\": \"eo:cloud_cover\"}, 40]\n",
    "      }\n",
    "    ]\n",
    "}\n",
    "\n",
    "search_fetch_plot(params, filt)"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "6b6313dbab648ff537330b996f33bf845c0da10ea77ae70864d6ca8e2699c7ea"
  },
  "kernelspec": {
   "display_name": "Python 3.9.11 ('.venv': venv)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}