Skip to content

feat: add s3 backend#22

Merged
alecthomas merged 9 commits intomainfrom
johnm/add-s3-backend
Jan 14, 2026
Merged

feat: add s3 backend#22
alecthomas merged 9 commits intomainfrom
johnm/add-s3-backend

Conversation

@js-murph
Copy link
Copy Markdown
Contributor

@js-murph js-murph commented Jan 13, 2026

What

Adds a new cache backend that supports s3 for cache storage using the minio client.

Why

s3 will work as a durable storage layer capability of storing larger amounts of data than the currently implemented cache backends.

We're using the Minio s3 client due to it being lighter than the AWS SDK.

Tests

We've added integration tests via a minio test container.

Performed the following steps for manual validation against an AWS s3 bucket:

  1. Create a test s3 bucket
aws s3 mb --region us-west-2 s3://sftpc-test-bucket 
  1. Created a sftpc.hcl config
host "https://w3.org" {}

s3 {
  bucket = "sftpc-test-bucket"
  region = "us-west-2"
}
  1. Start the proxy
go build ./cmd/sfptcd && ./sfptcd --config sfptc.hcl
  1. Validate the behaviour
curl http://127.0.0.1:8080/w3.org/TR/html5/
# ...snip...
aws s3 ls s3://sftpc-test-bucket/ --recursive
2026-01-14 09:39:31     152586 2e/2ecdce061e0b582a42737e8eba56a67c6bdcb157c440c9e500a870e7c427a16d

@js-murph js-murph self-assigned this Jan 13, 2026
@js-murph js-murph requested a review from alecthomas as a code owner January 13, 2026 00:58
@js-murph js-murph added the enhancement New feature or request label Jan 13, 2026
Comment thread internal/cache/s3.go Outdated
Comment thread internal/cache/s3.go Outdated
Comment thread internal/cache/s3.go Outdated
Comment thread internal/cache/s3.go Outdated
Comment thread internal/cache/s3.go Outdated
Comment thread internal/cache/s3.go Outdated
Comment thread internal/cache/s3.go Outdated
Comment thread internal/cache/s3.go Outdated
Comment thread go.sum
Comment thread internal/cache/s3.go Outdated
@js-murph js-murph changed the title [WIP] feat: add s3 backend feat: add s3 backend Jan 13, 2026
@alecthomas alecthomas merged commit bb6fff3 into main Jan 14, 2026
4 checks passed
@alecthomas alecthomas deleted the johnm/add-s3-backend branch January 14, 2026 04:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants