Skip to content

feat(lib-storage): S3TransferManager #7238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 37 commits into
base: main
Choose a base branch
from

Conversation

lukachad
Copy link

Description

Implemented an S3TransferManager in the lib-storage package. Still in development, currently defined all types for the Transfer Manager, implemented multipart download, and event listeners.

Testing

Three files were created two test the S3TransferManager: S3TransferManager.spec.ts (Unit tests), S3TransferManager.browser.spec.ts (DOM unit tests), and S3TransferManager.e2e.spec.ts (e2e tests).


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

lukachad and others added 30 commits July 21, 2025 15:27
feat(lib-storage): added doc comments for interfaces and types

feat(lib-storage): address PR review feedback for transfer manager types

feat(lib-storage): addressed minor review feedback

feat(lib-storage): added example code file

feat(lib-storage): changed "handler" to "listener" and changed type file names

feat(lib-storage): created TransferManager class and constructor with defaults

feat(lib-storage): beginning implementation for download(), created TM index file

feat(lib-storage): range multipart download

feat(lib-storage): download() improvements post pair-programming

feat(lib-storage): transfermanager download() iteration

feat(lib-storage): transfermanager interation post pair programming

feat(lib-storage): joinstream iteration, web stream not fully functional

feat(lib-storage): both cases of range download handled

feat(lib-storage): range download working

chore: acquire lock on streams

feat(lib-storage): bug fixes and test env setup

feat(lib-storage): implemented dispatchEvent(), need to add dispatches for complete and fail

feat(lib-storage): requests array, eventListeners revision, needs more testing

feat(lib-storage): addEventListener & removeEventListener implemented, needs support for options

feat(lib-storage): added support for adding event listeners at request level

feat(lib-storage): added ETag verification for subsequent GetObjectRequests

feat(lib-storage): totalSize changes, and added validateExpectedRanges()

feat(lib-storage): addEventListener once parameter, type fixes

feat(lib-storage): s3TM SEP test cases and added TODOs

feat(lib-storage): validateExpectedRanges fixes and unit tests

feat(lib-storage): added check in validateExpectedRanges if final part doesnt download total object

feat(lib-storage): s3TM constructor and add event listener tests

feat(lib-storage): addEventListener tests and error checking adjustment and dispatchEvent tests

feat(lib-storage): added test cases
Browser streams support & concurrency
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants