-
Notifications
You must be signed in to change notification settings - Fork 6
adding a lambda handler and client for database access #206
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
aaarendt
wants to merge
35
commits into
master
Choose a base branch
from
lambda
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Supports deployment as an AWS Lambda function using Docker containers. This enables serverless execution with significantly reduced package size (~200MB vs several GB) while maintaining core database functionality. This is a lightweight deployment that excludes GDAL/rasterio/geopandas for smaller containers, for now. Secure database credential management achieved through AWS Secrets Manager.
…dding error handling for geometric operations
…ame is no longer hard coded
…thod for PostGIS support
…ependency and improving PostGIS integration
…integration support
…ocal database connection
…and memory settings
…irements documentation
…edentialing approach
…h for storing credentials as environment variables
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #206 +/- ##
===========================================
- Coverage 75.88% 59.68% -16.20%
===========================================
Files 21 23 +2
Lines 651 1146 +495
===========================================
+ Hits 494 684 +190
- Misses 157 462 +305 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
jomey
approved these changes
Dec 22, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
This PR has scripts to deploy a Python handler to AWS Lambda that leverages native functionality of the existing api. The geospatial heavy operations require large libraries that were too cumbersome for Lambda, so many changes are implemented here to simplify and avoid use of geopandas. This includes relying on PostGIS functionality at the db backend, and relying on the client side to query geometries and convert back to geospatial objects, thereby avoiding needing geopandas in the Lambda handler itself.
Performance issues were discovered along the way since this is the first time doing operations on the 29 GB layers table. Specifically,
all_instrumentsquery should useEXISTSinstead ofDISTINCTwith join which vastly improves performance.Throughout I updated the handling of credentials following #204 .
This code was primarily developed using AI-assistance.