Skip to content

Conversation

@cportele
Copy link
Collaborator

@cportele cportele commented Nov 12, 2025

Pull request checklist

  • Added/updated unit tests
  • Updated documentation
  • All checks are passing

Changes introduced by this PR

Closes #1508
Closes #1345

Depends on ldproxy/xtraplatform-spatial#416.

A new feature provider option is introduced to enable skipping unnecessary/unused pipeline steps:

queryProcessing: 
  skipUnusedPipelineSteps: true

Tests with the simple strassenbaukataster data from #1345 and a feature query with a large bounding box that matches 224995 features, varying page sizes, the native CRS as the target CRS, and GeoJSON and FlatGeobuf as output formats:

curl "http://localhost:7080/strassenbaukataster/collections/strassenbaukataster/items?bbox=9.7513%2C53.4112%2C10.2635%2C53.7450&crs=http%3A%2F%2Fwww.opengis.net%2Fdef%2Fcrs%2FEPSG%2F0%2F25832&f=$format&limit=$limit" -o /dev/null

Page size Format Time without skipping pipeline steps (ms) Time with skipping pipeline steps (ms)
1000 GeoJSON 281 209
1000 FlatGeobuf 536 234
10000 GeoJSON 1179 543
10000 FlatGeobuf 1366 393
100000 GeoJSON 8837 3842
100000 FlatGeobuf 11005 2791
224995 GeoJSON 19217 9013
224995 FlatGeobuf 24623 6526

@cportele cportele requested a review from azahnen as a code owner November 12, 2025 12:28
@azahnen
Copy link
Collaborator

azahnen commented Nov 12, 2025

Deployment of preview was successful: https://ldproxy-ldproxy-pr-1509.surge.sh

@cportele cportele marked this pull request as draft November 12, 2025 13:44
@cportele
Copy link
Collaborator Author

@azahnen - Please wait with a review. Currently the changes do not improve performance for queries for generating vector tiles. I will refactor the changes to also support vector tile generation.

move to xtraplatform-spatial
@cportele cportele marked this pull request as ready for review November 12, 2025 19:08
@cportele
Copy link
Collaborator Author

@azahnen - This PR is ready for review. Latest changes:

  • code moved to xtraplatform-features to support also feature queries for tiles;
  • removed the DATE_FORMAT transformation that was added for a property with value type DATE as the SQL response is already in format yyyy-MM-dd;
  • split the MAPPING step into two options (mapping of schema and values and value transformations only).

I tested it with a number of API configurations and typically the GeoJSON and FlatGeobuf formats (comparison that the result is the same, excluding timestamps, with and without the option); FlatGeobuf only for the simpler data schemas.

We could also ignore the option for single feature queries or small page sizes as for those cases the overhead to determine the necessary steps is probably similar to potential the savings.

@azahnen azahnen merged commit c430007 into master Dec 15, 2025
4 checks passed
@azahnen azahnen deleted the pipeline-opt branch December 15, 2025 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

improve feature query performance: skip unnecessary pipeline steps ogc api features: general performance issue

3 participants