Skip to content

Conversation

@paytoncain
Copy link

This pull request is intended to at least partially address #11 by removing the need for asynchronous data requests from the DCDB's point store and allowing this repository to directly reach out to the CSB REST API.

Changes included in this PR include:

  1. A new interface named 'Cache' which is intended to act as a provider of data to the 'Workflow' interface
  2. An implementation of the 'Cache' interface which queries the CSB REST API. Query results from the API are parsed into a request to the NODD S3 bucket, where csv files are downloaded and stored in a cache directory
  3. An additional 'apply_workflow' method which runs the 'Cache' interface's update method before running the original 'apply_workflow' method in this project

Additional improvements / use cases this cache could be extended for:

  1. Downloading geojson and metadata from CSB REST API instead of csv from NODD bucket. (will require an adaptor that reads from 2 files at once. This could also be an update to the dcdb's CSV loader if geojson is not deemed necessary)

@paytoncain paytoncain marked this pull request as ready for review March 28, 2025 01:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants