Skip to content

Update hand-fim image#1115

Merged
shawncrawley merged 24 commits intoNOAA-OWP:tifrom
groutr:fim-reorg
Apr 3, 2025
Merged

Update hand-fim image#1115
shawncrawley merged 24 commits intoNOAA-OWP:tifrom
groutr:fim-reorg

Conversation

@groutr
Copy link
Copy Markdown
Contributor

@groutr groutr commented Mar 20, 2025

Summary of changes:

  1. Reorganize the directory layout for easier development. Uses deploy/code directories to simplify image building. Anything in the deploy/ directory gets uploaded to codebuild and anything in code/ directory gets included in final image.
  2. Remove the retry logic from dataset opening/reading. After considering this code for a bit, I don't think we actually execute it in any case short of an S3 outage.
  3. Cache S3 file reads locally using fsspec. This uses the new opener interface in rasterio 1.4. Some of the files are opened twice and this should avoid redownloads by caching locally to a temporary directory.
  4. Remove the s3_file import and checks file existence directly on S3 via fsspec.
  5. Create a new terraform module for hand-fim image.
  6. Edit code for better clarity.

Supersedes #1108

Copy link
Copy Markdown
Collaborator

@shawncrawley shawncrawley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@shawncrawley shawncrawley merged commit 01cc422 into NOAA-OWP:ti Apr 3, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants