#47 Reduced runtime by 50% by reducing number of downloads when normalizing HREFs

[jathavaan]

This pull request introduces several improvements to the STAC catalog handling and release pipeline, focusing on more precise catalog saving for releases, enhanced control over file download behavior, and interface updates to support these changes. The most significant updates are grouped below.

STAC Catalog Saving and Release Handling:

• The save_catalog method in both the IStacService interface and its implementation now requires a release parameter, ensuring that catalog saving is release-specific and that only the appropriate release subtree is saved. [1] [2] [3] [4]
• The save_catalog implementation now clones the catalog, isolates the relevant release, normalizes hrefs, and updates the catalog structure before saving, improving the integrity and organization of release catalogs.

File Download Control in STAC IO Service:

• Added a skip_file_download property (with getter and setter) to the IStacIOService interface and its StacIOService implementation, allowing the pipeline to temporarily disable file downloads during catalog save operations for efficiency. [1] [2] [3]
• The save_catalog function in the pipeline now sets skip_file_download to True before saving and resets it to False afterward, preventing unnecessary file downloads during the operation.

Interface and Dependency Injection Updates:

• Updated dependency injection and function signatures throughout the pipeline to accommodate the new release parameter and the IStacIOService dependency, ensuring the correct services and parameters are used. [1] [2] [3]

Miscellaneous:

• Added an (optional, commented out) import and usage of cProfile in main.py for profiling support during development. [1] [2]

[Copilot]

Pull request overview

This PR aims to reduce runtime during STAC catalog normalization/saving by avoiding unnecessary blob downloads and by saving release-specific catalog content.

Changes:

• Updated IStacService.save_catalog(...) (and call sites) to require a release argument and save release-specific output.
• Introduced skip_file_download on IStacIOService/StacIOService and toggled it during catalog saving.
• Added (currently active) cProfile import in main.py with commented profiling invocation.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
src/presentation/entrypoints/release_pipeline.py	Passes release to catalog save and toggles skip_file_download during save.
src/infra/infrastructure/services/stac_service.py	Reworks catalog save logic to normalize/save a release subtree.
src/infra/infrastructure/services/stac_io_service.py	Adds skip_file_download flag plumbing and minor refactor around upload checks.
src/application/contracts/stac_service_interface.py	Updates service contract to require release in save_catalog.
src/application/contracts/stac_io_service_interface.py	Adds skip_file_download to the STAC IO interface.
main.py	Adds cProfile import and leaves a commented profiling line.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.