Merge branch 'develop'

Author: sbrunato

CHANGES.rst

@@ -1,6 +1,57 @@
 Release history
 ===============
 
+v3.1.0b1 (2025-01-13)
+---------------------
+
+Core features and fixes
+^^^^^^^^^^^^^^^^^^^^^^^
+
+* Updated `queryables <https://eodag.readthedocs.io/en/latest/notebooks/api_user_guide/5_queryables.html>`_ mechanism
+  and ecmwf-like plugins (:pull:`1397`)(:pull:`1427`)(:pull:`1462`)
+* Order and download polling times update (:pull:`1440`)
+* Do not retry downloading skipped products during download_all (:pull:`1465`)
+* Renamed record files that were using previous mechanism (:pull:`1396`, thanks `@gasparakos\
+  <https://github.com/gasparakos>`_)
+* New ``to_lower()`` and ``to_upper()`` `parameters mapping\
+  <https://eodag.readthedocs.io/en/latest/params_mapping.html#formatters>`_ methods (:pull:`1410`, thanks
+  `@jgaucher-cs <https://github.com/jgaucher-cs>`_)
+
+Providers and product types updates
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* ``geodes`` updated ``id`` (:pull:`1441`) and ``tileIdentifier`` parameters (:pull:`1457`), and metadata mapping fix
+  (:pull:`1468`)
+* New MTG product types for ``eumetsat_ds`` (:pull:`1455`)
+* ``FIRE_HISTORICAL`` on ``wekeo_ecmwf`` (:pull:`1392`)
+* Various product types metadata-mapping and default values updates: for ``cop_ads`` and ``wekeo_ecmwf`` (:pull:`1389`),
+  GLOFAS and EFAS product types (:pull:`1467`), ``EEA_DAILY_VI`` on ``wekeo_main`` (:pull:`1464`)
+
+Plugins new features and fixes
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* Removed default :class:`~eodag.plugins.download.http.HTTPDownload` zip extension (:pull:`1400`)
+* Order and poll without downloading (:pull:`1437`)
+* :class:`~eodag.plugins.authentication.token.TokenAuth` distinct headers for token retrieve and authentication
+  (:pull:`1451`, thanks `@jgaucher-cs <https://github.com/jgaucher-cs>`_)
+* Compare only offset-aware datetimes on openid authentication plugins (:pull:`1418`)
+* Fixed ``creodias_s3`` search and download when no asset is available (:pull:`1425`)
+
+Server mode
+^^^^^^^^^^^
+
+* Dedicated liveness endpoint added (:pull:`1353`)
+* Processing level parsing fix in external STAC collections (:pull:`1429`)
+
+Miscellaneous
+^^^^^^^^^^^^^
+* **[docs]** `Queryables <https://eodag.readthedocs.io/en/latest/notebooks/api_user_guide/5_queryables.html>`_
+  documentation in a dedicated section (:pull:`1447`)
+* Various minor fixes and improvements (:pull:`1390`)(:pull:`1403`)(:pull:`1411`)(:pull:`1415`)(:pull:`1419`)
+  (:pull:`1428`)(:pull:`1430`)(:pull:`1434`)(:pull:`1445`)(:pull:`1448`)(:pull:`1458`)(:pull:`1466`)
+* External product types reference updates (:pull:`1387`)(:pull:`1391`)(:pull:`1401`)(:pull:`1404`)(:pull:`1406`)
+  (:pull:`1408`)(:pull:`1416`)(:pull:`1424`)(:pull:`1453`)(:pull:`1459`)
+
 v3.0.1 (2024-11-06)
 -------------------
 
@@ -233,10 +284,10 @@ Core features and fixes
 * Configurable download timeout (:pull:`1124`)
 * `Search by id <https://eodag.readthedocs.io/en/stable/notebooks/api_user_guide/4_search.html#id-and-provider>`_ now
   uses :meth:`~eodag.api.core.EODataAccessGateway.search_all` and
-  `crunch <https://eodag.readthedocs.io/en/stable/notebooks/api_user_guide/6_crunch.html#Filter-by-property>`_
+  `crunch <https://eodag.readthedocs.io/en/stable/notebooks/api_user_guide/7_crunch.html#Filter-by-property>`_
   (:pull:`1099`).
 * Free text search available for all fields when `guessing a produc type
-  <https://eodag.readthedocs.io/en/stable/notebooks/api_user_guide/6_crunch.html#Filter-by-property>`_ (:pull:`1070`),
+  <https://eodag.readthedocs.io/en/stable/notebooks/api_user_guide/7_crunch.html#Filter-by-property>`_ (:pull:`1070`),
   mission dates filtering support (:pull:`1222`)
 * Configurable requests ``ssl_verify`` (:pull:`1045`)
 * Download record hash independent from provider (:pull:`1023`)

CONTRIBUTING.rst

@@ -109,7 +109,7 @@ The notebooks listed below are **always executed** by `nbsphinx`:
 
 * `<notebooks/api_user_guide/2_providers_products_available.ipynb>`_
 * `<notebooks/api_user_guide/3_configuration.ipynb>`_
-* `<notebooks/api_user_guide/6_crunch.ipynb>`_
+* `<notebooks/api_user_guide/7_crunch.ipynb>`_
 
 For the other notebooks, their **cell output as long as their widget state** need so be saved.
 If not, the outputs and the widgets (e.g. progress bar) won't be displayed in the online documentation.

README.rst

@@ -187,7 +187,7 @@ An eodag instance can be exposed through a STAC compliant REST api from the comm
 
 .. code-block:: bash
 
-    docker run -p 5000:5000 --rm csspace/eodag-server:3.0.1
+    docker run -p 5000:5000 --rm csspace/eodag-server:3.1.0b1
 
 You can also browse over your STAC API server using `STAC Browser <https://github.com/radiantearth/stac-browser>`_.
 Simply run:

charts/eodag-server/Chart.yaml

@@ -1,5 +1,5 @@
 apiVersion: v2
-appVersion: 3.0.1
+appVersion: 3.1.0b1
 dependencies:
 - name: common
   repository:  oci://registry-1.docker.io/bitnamicharts
@@ -15,4 +15,4 @@ name: eodag-server
 sources:
 - https://github.com/CS-SI/eodag
 type: application
-version: 3.0.1
+version: 3.1.0b1

charts/eodag-server/templates/deployment.yaml

@@ -111,7 +111,7 @@ spec:
           {{- else if .Values.livenessProbe.enabled }}
           livenessProbe:
             httpGet:
-              path: /
+              path: /_mgmt/ping/
               port: {{ .Values.containerPorts.http }}
             initialDelaySeconds: {{ .Values.livenessProbe.initialDelaySeconds }}
             periodSeconds: {{ .Values.livenessProbe.periodSeconds }}

docker/stac-server.dockerfile

@@ -24,7 +24,8 @@ ENV \
 
 # update system
 RUN apt-get update \
-    && apt-get upgrade -y
+    && apt-get upgrade -y \
+    && echo "Build time: $(date)" > /dev/null
 
 # reconfigure timezone
 RUN echo $TZ > /etc/timezone && \

docs/_static/custom.css

@@ -91,3 +91,8 @@ dl.attribute dt {
 dl.attribute dd {
     grid-column-start: 2;
 }
+
+.nboutput table {
+    margin-left: unset;
+    margin-right: unset;
+}

docs/_static/params_mapping_extra.csv

@@ -1,111 +1,37 @@
 parameter,cop_ads,cop_cds,cop_dataspace,cop_ewds,cop_marine,creodias,dedt_lumi,earth_search,earth_search_cog,earth_search_gcs,ecmwf,eumetsat_ds,geodes,onda,peps,planetary_computer,sara,theia,usgs_satapi_aws
 _date,metadata only,metadata only,,metadata only,,,,,,,,,,,,,,,
 acquisitionInformation,,,,,,,,,,,,metadata only,,,,,,,
-activity,,,,,,,:green:`queryable metadata`,,,,,,,,,,,,
-aerosol_type,:green:`queryable metadata`,,,,,,,,,,,,,,,,,,
-altitude,:green:`queryable metadata`,,,,,,,,,,,,,,,,,,
-anoffset,,,,,,,:green:`queryable metadata`,,,,,,,,,,,,
-api_product_type,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,,,,,,,,,,
 assets,,,,,,,,metadata only,metadata only,metadata only,,metadata only,metadata only,,,metadata only,,,metadata only
 awsProductId,,,,,,,,,,,,,,,,,,,metadata only
-band,:green:`queryable metadata`,,,,,,,,,,,,,,,,,,
-cdr_type,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-class,,,,,,,:green:`queryable metadata`,,,,,,,,,,,,
 collection,,,:green:`queryable metadata`,,,:green:`queryable metadata`,,,,,,,,,,,,,
-dataset,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,,,:green:`queryable metadata`,,,,,,,,,,,,
-dataset_type,,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,,,,,,,,,,
-day,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,,,,,,,,,,
 defaultGeometry,metadata only,metadata only,,metadata only,metadata only,,,,,,,metadata only,,,,,,,
-domain,,,,,,,:green:`queryable metadata`,,,,,,,,,,,,
 downloadLink,metadata only,metadata only,metadata only,metadata only,,metadata only,metadata only,metadata only,metadata only,metadata only,,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only
-download_format,,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,,,,,,,,,,
-ensemble_member,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-experiment,,:green:`queryable metadata`,,,,,:green:`queryable metadata`,,,,,,,,,,,,
-expver,,,,,,,:green:`queryable metadata`,,,,,,,,,,,,
+ecmwf:system_version,,,,:green:`queryable metadata`,,,,,,,,,,,,,,,
+ecmwf:version,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
 extraInformation,,,,,,,,,,,,metadata only,,,,,,,
-filter,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-forcing_type,:green:`queryable metadata`,,,,,,,,,,,,,,,,,,
-format,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,,,,,,,,,,
-gcm,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-generation,,,,,,,:green:`queryable metadata`,,,,,,,,,,,,
 geometry,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,metadata only,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`
-grid,,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,,,,,,,,,,
 gridSquare,,,,,,,,:green:`queryable metadata`,,,,,,,,,,,
-hdate,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-hday,,,,:green:`queryable metadata`,,,,,,,,,,,,,,,
-hmonth,,,,:green:`queryable metadata`,,,,,,,,,,,,,,,
-horizontal_resolution,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-hydrological_model,,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,,,,,,,,,,
-hyear,,,,:green:`queryable metadata`,,,,,,,,,,,,,,,
 id,metadata only,metadata only,:green:`queryable metadata`,metadata only,,:green:`queryable metadata`,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,metadata only,:green:`queryable metadata`
-input_observations,:green:`queryable metadata`,,,,,,,,,,,,,,,,,,
-latitude,:green:`queryable metadata`,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
 latitudeBand,,,,,,,,:green:`queryable metadata`,,,,,,,,,,,
-leadtime_hour,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,,,,,,,,,,
-leadtime_month,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-level,:green:`queryable metadata`,,,,,,,,,,,,,,,,,,
-levelist,,,,,,,:green:`queryable metadata`,,,,,,,,,,,,
-levtype,:green:`queryable metadata`,,,,,,:green:`queryable metadata`,,,,,,,,,,,,
 links,,,,,,,,,,,,,,,metadata only,,,,
-location,:green:`queryable metadata`,,,,,,,,,,,,,,,,,,
-longitude,:green:`queryable metadata`,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-model,:green:`queryable metadata`,,,,,,:green:`queryable metadata`,,,,,,,,,,,,
-model_level,:green:`queryable metadata`,,,,,,,,,,,,,,,,,,
-model_levels,,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,,,,,,,,,,
 modifiedAfter,,,:green:`queryable metadata`,,,,,,,,,,,,,,,,
 modifiedBefore,,,:green:`queryable metadata`,,,,,,,,,,,,,,,,
-month,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,,,,,,,,,,
-nominal_day,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
 orderLink,metadata only,metadata only,,metadata only,,,metadata only,,,,,,,metadata only,,,,,
-origin,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-originating_centre,,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,,,,,,,,,,
-param,:green:`queryable metadata`,:green:`queryable metadata`,,,,,:green:`queryable metadata`,,,,,,,,,,,,
-period,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
 polarizationChannels,,,:green:`queryable metadata`,,,:green:`queryable metadata`,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,,,:green:`queryable metadata`,metadata only,,:green:`queryable metadata`,,metadata only,:green:`queryable metadata`
 polarizationMode,,,,,,,,,metadata only,,,,,,:green:`queryable metadata`,,:green:`queryable metadata`,metadata only,
-pressure_level,:green:`queryable metadata`,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-processing_level,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-processing_type,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-product,:green:`queryable metadata`,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
 productIdentifier,,,metadata only,,,metadata only,,,,,,,,,,,,,
 productInformation,,,,,,,,,,,,metadata only,,,,,,,
-product_version,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
 providerProductType,,,,,,,,,,,,,:green:`queryable metadata`,,,,,,
 publishedAfter,,,:green:`queryable metadata`,,,,,,,,,,,,,,,,
 publishedBefore,,,:green:`queryable metadata`,,,,,,,,,,,,,,,,
 qs,metadata only,metadata only,,metadata only,,,metadata only,,,,,,,,,,,,
-quantity,:green:`queryable metadata`,,,,,,,,,,,,,,,,,,
 quicklook,,,metadata only,,,metadata only,,metadata only,metadata only,metadata only,,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only
-rcm,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-realization,,,,,,,:green:`queryable metadata`,,,,,,,,,,,,
-region,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
 relativeOrbitNumber,,,:green:`queryable metadata`,,,,,,,,,,,,,,,,
-satellite,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-sensor,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-sensor_and_algorithm,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
 services,,,,,,,,,,,,,,,metadata only,,,,
 size,,,,,,,,,,,,:green:`queryable metadata`,,,,,,,
-sky_type,:green:`queryable metadata`,,,,,,,,,,,,,,,,,,
-soil_level,,,,:green:`queryable metadata`,,,,,,,,,,,,,,,
-source,:green:`queryable metadata`,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-statistic,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-step,:green:`queryable metadata`,:green:`queryable metadata`,,,,,:green:`queryable metadata`,,,,,,,,,,,,
 storageStatus,metadata only,metadata only,metadata only,metadata only,,metadata only,metadata only,metadata only,metadata only,metadata only,,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only,metadata only
-stream,,,,,,,:green:`queryable metadata`,,,,,,,,,,,,
-system,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-system_version,,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,,,,,,,,,,
-target,:green:`queryable metadata`,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-temporal_aggregation,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
 thumbnail,,,metadata only,,,metadata only,,metadata only,metadata only,metadata only,,,metadata only,,metadata only,metadata only,metadata only,metadata only,metadata only
-tileIdentifier,,,:green:`queryable metadata`,,,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,
-time,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,,,:green:`queryable metadata`,,,,,,,,,,,,
-time_aggregation,:green:`queryable metadata`,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-time_reference,:green:`queryable metadata`,,,,,,,,,,,,,,,,,,
-time_step,:green:`queryable metadata`,,,,,,,,,,,,,,,,,,
-type,:green:`queryable metadata`,:green:`queryable metadata`,,,,,:green:`queryable metadata`,,,,,metadata only,,,,,,,
+tileIdentifier,,,:green:`queryable metadata`,,,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,
+type,,,,,,,,,,,,metadata only,,,,,,,
 uid,,,metadata only,,,metadata only,,,,,,metadata only,metadata only,metadata only,metadata only,,metadata only,metadata only,
 utmZone,,,,,,,,:green:`queryable metadata`,,,,,,,,,,,
-variable,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,,,,,,,,,,
-variable_type,,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-version,:green:`queryable metadata`,:green:`queryable metadata`,,,,,,,,,,,,,,,,,
-year,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,,,,,,,,,,,,,,,

docs/_static/params_mapping_offline_infos.json

@@ -37,7 +37,7 @@ productQualityStatus,,,,,,,,,,,,,,:green:`queryable metadata`,metadata only,,met
 ":abbr:`productType ([OpenSearch Parameters for Collection Search] A string identifying the entry type (e.g. ER02_SAR_IM__0P, MER_RR__1P, SM_SLC__1S, GES_DISC_AIRH3STD_V005) (String ))`",metadata only,metadata only,:green:`queryable metadata`,metadata only,,:green:`queryable metadata`,metadata only,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`
 productVersion,,,,,,,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,,,:green:`queryable metadata`,:green:`queryable metadata`,metadata only,:green:`queryable metadata`,metadata only,metadata only,:green:`queryable metadata`
 :abbr:`publicationDate ([Additional INSPIRE obligated OpenSearch Parameters for Collection Search] The date when the resource was issued (Date time))`,,,metadata only,,,metadata only,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,,:green:`queryable metadata`,:green:`queryable metadata`,,metadata only,:green:`queryable metadata`,metadata only,metadata only,:green:`queryable metadata`
-resolution,,,:green:`queryable metadata`,,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,,,:green:`queryable metadata`,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,metadata only,:green:`queryable metadata`
+resolution,,,:green:`queryable metadata`,,,:green:`queryable metadata`,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,,,:green:`queryable metadata`,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,metadata only,:green:`queryable metadata`
 sensorMode,,,:green:`queryable metadata`,,,:green:`queryable metadata`,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,,,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,:green:`queryable metadata`,metadata only,:green:`queryable metadata`
 ":abbr:`sensorType ([OpenSearch Parameters for Collection Search] A string identifying the sensor type. Suggested values are: OPTICAL, RADAR, ALTIMETRIC, ATMOSPHERIC, LIMB (String))`",,,,,,,,,,,,,,:green:`queryable metadata`,,,,,
 snowCover,,,,,,,,,,,,,,,:green:`queryable metadata`,,:green:`queryable metadata`,metadata only,