fix readme

Author: Grzegorz Pustulka

Dockerfile

@@ -0,0 +1,33 @@
+FROM python:3.12-slim
+
+WORKDIR /app
+
+ENV APP_HOST=0.0.0.0 \
+    APP_PORT=8080 \
+    ENVIRONMENT=production \
+    BACKEND=opensearch \
+    STAC_FASTAPI_TITLE=stac-fastapi-opensearch \
+    STAC_FASTAPI_DESCRIPTION="A STAC FastAPI with an OpenSearch backend" \
+    STAC_FASTAPI_VERSION=2.1 \
+    RELOAD=false \
+    WEB_CONCURRENCY=1 \
+    STAC_FASTAPI_RATE_LIMIT=null \
+    ES_HTTP_COMPRESS=true \
+    ENABLE_DIRECT_RESPONSE=true \
+    ES_HOST=pgstac03.intra.cloudferro.com \
+    ES_PORT=9200 \
+    ES_USER=pgstac \
+    ES_PASS=19rs98j2jhrd34h9hr99j \
+    ES_USE_SSL=false \
+    ES_VERIFY_CERTS=false \
+    OPENSEARCH_VERSION=2.19.1
+
+COPY . .
+
+RUN pip install -e ./stac_fastapi/opensearch
+RUN pip install -e ./stac_fastapi/core
+RUN pip install -e ./stac_fastapi/sfeos_helpers
+
+EXPOSE 8080
+
+CMD ["python", "-m", "stac_fastapi.opensearch.app"]

README.md

@@ -841,320 +841,4 @@ The system uses a precise naming convention:
   - Helps prevent API abuse and maintains system stability
   - Ensures fair resource allocation among all clients
   
-- **Examples**: Implementation examples are available in the [examples/rate_limit](examples/rate_limit) directory.
-
-sed index selection using collection IDs. Requires indexes in format: STAC_ITEMS_INDEX_PREFIX_collection-id_start_year-start_month-start_day-end_year-end_month-end_day, e.g. items_sentinel-2-l2a_2025-06-06-2025-09-22.                                                                                                  | `false`                                              | Optional |
-| `DATETIME_INDEX_MAX_SIZE_GB` | Maximum size limit in GB for datetime-based indexes. When an index exceeds this size, a new time-partitioned index will be created. Note: This value should account for ~25% overhead due to OS/ES caching of data structures and metadata. Only applies when`ENABLE_DATETIME_INDEX_FILTERING` is enabled.                                                                               | `25`                                                 | Optional |
-
-> [!NOTE]
-> The variables `ES_HOST`, `ES_PORT`, `ES_USE_SSL`, `ES_VERIFY_CERTS` and `ES_TIMEOUT` apply to both Elasticsearch and OpenSearch backends, so there is no need to rename the key names to `OS_` even if you're using OpenSearch.
-
-## Interacting with the API
-
-- **Creating a Collection**:
-  ```shell
-  curl -X "POST" "http://localhost:8080/collections" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-    "id": "my_collection"
-  }'
-  ```
-
-- **Adding an Item to a Collection**:
-  ```shell
-  curl -X "POST" "http://localhost:8080/collections/my_collection/items" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d @item.json
-  ```
-
-- **Searching for Items**:
-  ```shell
-  curl -X "GET" "http://localhost:8080/search" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-    "collections": ["my_collection"],
-    "limit": 10
-  }'
-  ```
-
-- **Filtering by Bbox**:
-  ```shell
-  curl -X "GET" "http://localhost:8080/search" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-    "collections": ["my_collection"],
-    "bbox": [-180, -90, 180, 90]
-  }'
-  ```
-
-- **Filtering by Datetime**:
-  ```shell
-  curl -X "GET" "http://localhost:8080/search" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-    "collections": ["my_collection"],
-    "datetime": "2020-01-01T00:00:00Z/2020-12-31T23:59:59Z"
-  }'
-  ```
-
-## Configure the API
-
-- **API Title and Description**: By default set to `stac-fastapi-<backend>`. Customize these by setting:
-  - `STAC_FASTAPI_TITLE`: Changes the API title in the documentation
-  - `STAC_FASTAPI_DESCRIPTION`: Changes the API description in the documentation
-
-- **Database Indices**: By default, the API reads from and writes to:
-  - `collections` index for collections
-  - `items_<collection name>` indices for items
-  - Customize with `STAC_COLLECTIONS_INDEX` and `STAC_ITEMS_INDEX_PREFIX` environment variables
-
-- **Root Path Configuration**: The application root path is the base URL by default.
-  - For AWS Lambda with Gateway API: Set `STAC_FASTAPI_ROOT_PATH` to match the Gateway API stage name (e.g., `/v1`)
-
-
-## Collection Pagination
-
-- **Overview**: The collections route supports pagination through optional query parameters.
-- **Parameters**:
-  - `limit`: Controls the number of collections returned per page
-  - `token`: Used to retrieve subsequent pages of results
-- **Response Structure**: The `links` field in the response contains a `next` link with the token for the next page of results.
-- **Example Usage**:
-  ```shell
-  curl -X "GET" "http://localhost:8080/collections?limit=1&token=example_token"
-  ```
-
-## Ingesting Sample Data CLI Tool
-
-- **Overview**: The `data_loader.py` script provides a convenient way to load STAC items into the database.
-
-- **Usage**:
-  ```shell
-  python3 data_loader.py --base-url http://localhost:8080
-  ```
-
-- **Options**:
-  ```
-  --base-url TEXT       Base URL of the STAC API  [required]
-  --collection-id TEXT  ID of the collection to which items are added
-  --use-bulk            Use bulk insert method for items
-  --data-dir PATH       Directory containing collection.json and feature
-                        collection file
-  --help                Show this message and exit.
-  ```
-
-- **Example Workflows**:
-  - **Loading Sample Data**: 
-    ```shell
-    python3 data_loader.py --base-url http://localhost:8080
-    ```
-  - **Loading Data to a Specific Collection**:
-    ```shell
-    python3 data_loader.py --base-url http://localhost:8080 --collection-id my-collection
-    ```
-  - **Using Bulk Insert for Performance**:
-    ```shell
-    python3 data_loader.py --base-url http://localhost:8080 --use-bulk
-    ```
-
-## Elasticsearch Mappings
-
-- **Overview**: Mappings apply to search index, not source data. They define how documents and their fields are stored and indexed.
-- **Implementation**: 
-  - Mappings are stored in index templates that are created on application startup
-  - These templates are automatically applied when creating new Collection and Item indices
-  - The `sfeos_helpers` package contains shared mapping definitions used by both Elasticsearch and OpenSearch backends
-- **Customization**: Custom mappings can be defined by extending the base mapping templates.
-
-## Managing Elasticsearch Indices
-
-### Snapshots
-
-- **Overview**: Snapshots provide a way to backup and restore your indices.
-
-- **Creating a Snapshot Repository**:
-  ```shell
-  curl -X "PUT" "http://localhost:9200/_snapshot/my_fs_backup" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-               "type": "fs",
-               "settings": {
-                   "location": "/usr/share/elasticsearch/snapshots/my_fs_backup"
-               }
-  }'
-  ```
-  - This creates a snapshot repository that stores files in the elasticsearch/snapshots directory in this git repo clone
-  - The elasticsearch.yml and compose files create a mapping from that directory to /usr/share/elasticsearch/snapshots within the Elasticsearch container and grant permissions for using it
-
-- **Creating a Snapshot**:
-  ```shell
-  curl -X "PUT" "http://localhost:9200/_snapshot/my_fs_backup/my_snapshot_2?wait_for_completion=true" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-    "metadata": {
-      "taken_because": "dump of all items",
-      "taken_by": "pvarner"
-    },
-    "include_global_state": false,
-    "ignore_unavailable": false,
-    "indices": "items_my-collection"
-  }'
-  ```
-  - This creates a snapshot named my_snapshot_2 and waits for the action to be completed before returning
-  - This can also be done asynchronously by omitting the wait_for_completion parameter, and queried for status later
-  - The indices parameter determines which indices are snapshotted, and can include wildcards
-
-- **Viewing Snapshots**:
-  ```shell
-  # View a specific snapshot
-  curl http://localhost:9200/_snapshot/my_fs_backup/my_snapshot_2
-  
-  # View all snapshots
-  curl http://localhost:9200/_snapshot/my_fs_backup/_all
-  ```
-  - These commands allow you to check the status and details of your snapshots
-
-- **Restoring a Snapshot**:
-  ```shell
-  curl -X "POST" "http://localhost:9200/_snapshot/my_fs_backup/my_snapshot_2/_restore?wait_for_completion=true" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-    "include_aliases": false,
-    "include_global_state": false,
-    "ignore_unavailable": true,
-    "rename_replacement": "items_$1-copy",
-    "indices": "items_*",
-    "rename_pattern": "items_(.+)"
-  }'
-  ```
-  - This specific command will restore any indices that match items_* and rename them so that the new index name will be suffixed with -copy
-  - The rename_pattern and rename_replacement parameters allow you to restore indices under new names
-
-- **Updating Collection References**:
-  ```shell
-  curl -X "POST" "http://localhost:9200/items_my-collection-copy/_update_by_query" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-      "query": {
-          "match_all": {}
-  },
-    "script": {
-      "lang": "painless",
-      "params": {
-        "collection": "my-collection-copy"
-      },
-      "source": "ctx._source.collection = params.collection"
-    }
-  }'
-  ```
-  - After restoring, the item documents have been restored in the new index (e.g., my-collection-copy), but the value of the collection field in those documents is still the original value of my-collection
-  - This command updates these values to match the new collection name using Elasticsearch's Update By Query feature
-
-- **Creating a New Collection**:
-  ```shell
-  curl -X "POST" "http://localhost:8080/collections" \
-       -H 'Content-Type: application/json' \
-       -d $'{
-    "id": "my-collection-copy"
-  }'
-  ```
-  - The final step is to create a new collection through the API with the new name for each of the restored indices
-  - This gives you a copy of the collection that has a resource URI (/collections/my-collection-copy) and can be correctly queried by collection name
-
-### Reindexing
-
-- **Overview**: Reindexing allows you to copy documents from one index to another, optionally transforming them in the process.
-
-- **Use Cases**:
-  - Apply changes to documents
-  - Correct dynamically generated mappings
-  - Transform data (e.g., lowercase identifiers)
-  - The index templates will make sure that manually created indices will also have the correct mappings and settings
-
-- **Example: Reindexing with Transformation**:
-  ```shell
-  curl -X "POST" "http://localhost:9200/_reindex" \
-    -H 'Content-Type: application/json' \
-    -d $'{
-      "source": {
-        "index": "items_my-collection-lower_my-collection-hex-000001"
-      }, 
-      "dest": {
-        "index": "items_my-collection-lower_my-collection-hex-000002"
-      },
-      "script": {
-        "source": "ctx._source.id = ctx._source.id.toLowerCase()",
-        "lang": "painless"
-      }
-    }'
-  ```
-  - In this example, we make a copy of an existing Item index but change the Item identifier to be lowercase
-  - The script parameter allows you to transform documents during the reindexing process
-
-- **Updating Aliases**:
-  ```shell
-  curl -X "POST" "http://localhost:9200/_aliases" \
-    -H 'Content-Type: application/json' \
-    -d $'{
-      "actions": [
-        {
-          "remove": {
-            "index": "*",
-            "alias": "items_my-collection"
-          }
-        },
-        {
-          "add": {
-            "index": "items_my-collection-lower_my-collection-hex-000002",
-            "alias": "items_my-collection"
-          }
-        }
-      ]
-    }'
-  ```
-  - If you are happy with the data in the newly created index, you can move the alias items_my-collection to the new index
-  - This makes the modified Items with lowercase identifiers visible to users accessing my-collection in the STAC API
-  - Using aliases allows you to switch between different index versions without changing the API endpoint
-
-## Auth
-
-- **Overview**: Authentication is an optional feature that can be enabled through Route Dependencies.
-- **Implementation Options**:
-  - Basic authentication
-  - OAuth2 with Keycloak
-  - Custom route dependencies
-- **Configuration**: Authentication can be configured using the `STAC_FASTAPI_ROUTE_DEPENDENCIES` environment variable.
-- **Examples and Documentation**: Detailed examples and implementation guides can be found in the [examples/auth](examples/auth) directory.
-
-## Aggregation
-
-- **Supported Aggregations**:
-  - Spatial aggregations of points and geometries
-  - Frequency distribution aggregation of any property including dates
-  - Temporal distribution of datetime values
-
-- **Endpoint Locations**:
-  - Root Catalog level: `/aggregations`
-  - Collection level: `/<collection_id>/aggregations`
-
-- **Implementation Details**: The `sfeos_helpers.aggregation` package provides specialized functionality for both Elasticsearch and OpenSearch backends.
-
-- **Documentation**: Detailed information about supported aggregations can be found in [the aggregation docs](./docs/src/aggregation.md).
-
-
-## Rate Limiting
-
-- **Overview**: Rate limiting is an optional security feature that controls API request frequency on a remote address basis.
-
-- **Configuration**: Enabled by setting the `STAC_FASTAPI_RATE_LIMIT` environment variable:
-  ```
-  STAC_FASTAPI_RATE_LIMIT=500/minute
-  ```
-
-- **Functionality**: 
-  - Limits each client to a specified number of requests per time period (e.g., 500 requests per minute)
-  - Helps prevent API abuse and maintains system stability
-  - Ensures fair resource allocation among all clients
-  
-- **Examples**: Implementation examples are available in the [examples/rate_limit](examples/rate_limit) directory.
-
+- **Examples**: Implementation examples are available in the [examples/rate_limit](examples/rate_limit) directory.