Skip to content

Conversation

@oyiz-michael
Copy link
Contributor

@oyiz-michael oyiz-michael commented Aug 6, 2025

Issue number: #7124

closes #7124

Summary

This PR adds comprehensive File parameter support for handling file uploads in multipart/form-data requests within the AWS Lambda Powertools Python Event Handler with OpenAPI validation.

Changes

  • Added File class in aws_lambda_powertools/event_handler/openapi/params.py

    • New parameter type specifically for file uploads
    • Inherits from Form with format: binary in OpenAPI schema
    • Supports validation constraints (max_length, etc.)
  • Enhanced multipart parsing in aws_lambda_powertools/event_handler/middlewares/openapi_validation.py

    • Added _parse_multipart_data method for parsing multipart/form-data
    • WebKit boundary support for Safari/Chrome compatibility
    • Base64 decoding support for Lambda event handling
    • Distinguishes between file fields and form fields
  • Comprehensive test suite with 13 test scenarios covering:

    • Basic file uploads and multiple file handling
    • File + form data combinations
    • WebKit boundary parsing and base64 encoded content
    • Validation constraints and error handling
    • Optional file parameters
  • Complete usage example in examples/event_handler_rest/src/file_parameter_example.py

User experience

Before: Users could not handle file uploads in multipart/form-data requests with OpenAPI validation. They had to manually parse request bodies or disable validation entirely.

After: Users can now use type-annotated File parameters that automatically:

  • Parse multipart/form-data file uploads
  • Generate proper OpenAPI schema with format: binary
  • Apply validation constraints
  • Work seamlessly with existing form parameters
from typing import Annotated
from aws_lambda_powertools.event_handler import APIGatewayRestResolver
from aws_lambda_powertools.event_handler.openapi.params import File, Form

app = APIGatewayRestResolver(enable_validation=True)

@app.post("/upload")
def upload_file(
    file: Annotated[bytes, File(description="File to upload", max_length=1000000)],
    title: Annotated[str, Form(description="File title")]
):
    return {"file_size": len(file), "title": title, "status": "uploaded"}

Checklist

If your change doesn't seem to apply, please leave them unchecked.

Is this a breaking change? RFC issue number: N/A

This is not a breaking change - it's a new feature addition that doesn't modify existing functionality.

Checklist:

  • Migration process documented
  • Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…ploads

- Add public File parameter class extending _File
- Support multipart/form-data parsing with WebKit boundary compatibility
- OpenAPI schema generation with format: binary for file uploads
- Enhanced dependant logic to handle File + Form parameter combinations
- Clean implementation based on upstream develop branch

Changes:
- params.py: Add File(_File) public class with proper documentation
- dependant.py: Add File parameter support in body field info logic
- openapi_validation.py: Add multipart parsing with boundary detection
- test_file_form_validation.py: Basic test coverage for File parameters

This provides customers with File parameter support using the same
pattern as Query, Path, Header parameters with Annotated types.
- Add File parameter class in openapi/params.py with binary format schema
- Implement comprehensive multipart/form-data parsing in openapi_validation.py
  * Support for WebKit and standard boundary formats
  * Base64-encoded request handling for AWS Lambda
  * Mixed file and form data parsing
- Update dependant.py to handle File parameters in body field resolution
- Add comprehensive test suite (13 tests) covering:
  * Basic file upload parsing and validation
  * WebKit boundary format support
  * Base64-encoded multipart data
  * Multiple file uploads
  * File size constraints validation
  * Optional file parameters
  * Error handling for invalid boundaries and missing files
- Add file_parameter_example.py demonstrating various File parameter use cases
- Clean up unnecessary imports and pragma comments

Resolves file upload functionality with full OpenAPI schema generation and validation support.
@oyiz-michael oyiz-michael requested a review from a team as a code owner August 6, 2025 22:12
@pull-request-size pull-request-size bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Aug 6, 2025
- Break down _parse_multipart_data method into smaller helper methods
- Reduce cognitive complexity from 43 to under 15 per SonarCloud requirement
- Improve code readability and maintainability
- All existing tests continue to pass

Helper methods created:
- _decode_request_body: Handle base64 decoding
- _extract_boundary_bytes: Extract multipart boundary
- _parse_multipart_sections: Parse sections into data dict
- _parse_multipart_section: Handle individual section parsing
- _split_section_headers_and_content: Split headers/content
- _decode_form_field_content: Decode form field as string

Addresses SonarCloud cognitive complexity violation while maintaining
all existing functionality for File parameter multipart parsing.
@oyiz-michael oyiz-michael changed the title Feature/file parameter clean feat: add File parameter support for multipart/form-data uploads Aug 6, 2025
@github-actions github-actions bot added the feature New feature or functionality label Aug 6, 2025
@oyiz-michael oyiz-michael changed the title feat: add File parameter support for multipart/form-data uploads feat(event_handler): add File parameter support for multipart/form-data uploads in OpenAPI utility Aug 6, 2025
- Add missing __future__ annotations imports
- Remove unused pytest imports from test files
- Remove unused json import from example
- Fix line length violations in test files
- All File parameter tests continue to pass (13/13)

Addresses ruff linting violations:
- FA102: Missing future annotations for PEP 604 unions
- F401: Unused imports
- E501: Line too long violations
@leandrodamascena
Copy link
Contributor

Hi @oyiz-michael, I see you are working on this PR and please let me know when you need a first round of review or any help.

@leandrodamascena leandrodamascena linked an issue Aug 7, 2025 that may be closed by this pull request
2 tasks
- Replace bytes | None with Union[bytes, None] for broader compatibility
- Replace str | None with Union[str, None] in examples
- Add noqa: UP007 comments to suppress linter preference for newer syntax
- Ensures compatibility with Python environments that don't support PEP 604 unions
- Fixes test failure: 'Unable to evaluate type annotation bytes | None'

All File parameter tests continue to pass (13/13) across Python versions.
@codecov
Copy link

codecov bot commented Aug 7, 2025

Codecov Report

❌ Patch coverage is 96.96970% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.53%. Comparing base (60043ca) to head (35edb78).
⚠️ Report is 3 commits behind head on develop.

Files with missing lines Patch % Lines
...ls/event_handler/middlewares/openapi_validation.py 95.86% 3 Missing and 2 partials ⚠️
..._lambda_powertools/event_handler/openapi/params.py 96.36% 1 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##           develop    #7132    +/-   ##
=========================================
  Coverage    96.52%   96.53%            
=========================================
  Files          275      275            
  Lines        13117    13377   +260     
  Branches       986     1036    +50     
=========================================
+ Hits         12661    12913   +252     
- Misses         353      356     +3     
- Partials       103      108     +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@oyiz-michael
Copy link
Contributor Author

Hi @oyiz-michael, I see you are working on this PR and please let me know when you need a first round of review or any help.

@leandrodamascena fixing some failing test and should be ready for a review and feed back

@pull-request-size pull-request-size bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 7, 2025
@leandrodamascena
Copy link
Contributor

Hi @oyiz-michael, a quick tip: run make pr in your local environment and then you can catch errors before committing and pushing the files.

@leandrodamascena
Copy link
Contributor

Hi @oyiz-michael! After merging the PRs I mentioned in the previous comment, I see we have some merge conflicts here. Can you fix them before we move forward?

Thanks

oyiz-michael and others added 14 commits October 14, 2025 21:42
Merge conflicts resolved after upstream PRs aws-powertools#7227 and aws-powertools#7253:

Core Changes:
- Updated File class rename from File -> _File with public File alias
- Fixed Union import for _resolve_field_type function
- Removed unused _File import in dependant.py
- Maintained all UploadFile and validation functionality

Testing:
- All 25 comprehensive tests passing
- 96.36%+ codecov coverage maintained
- Code quality checks passing (make format && make pr)

Compatibility:
- Preserves public File API for backward compatibility
- Maintains UploadFile OpenAPI schema generation
- All validation middleware features intact
- Changed back from '_File with alias' to direct 'class File(Form)'
- All 25 tests passing with 96.36%+ coverage
- Code quality checks passing (make format && make pr)
Add type: ignore[misc] comment to suppress mypy error about inheriting
from final FieldInfo class in Pydantic. This matches the upstream pattern
used for similar classes and allows the code to pass mypy type checking
in Python 3.10+.

- Fix mypy error: Cannot inherit from final class 'FieldInfo'
- Matches upstream _File class pattern with type ignore
- All tests passing (25/25)
- All quality checks passing (format, lint, mypy)
- Added UploadFile-to-bytes conversion in _normalize_field_value()
- Handles type annotations including Annotated[bytes, File()]
- Fixes 24 failing tests in test_file_parameter.py
- All tests now return 200 OK instead of 422 validation errors

Resolves multipart form data parsing issue where UploadFile instances
weren't being converted to bytes before Pydantic validation.
yizzy and others added 4 commits October 26, 2025 15:39
…to ensure_upload_file_schema_references; tidy openapi package
- Created dedicated _convert_uploadfile_to_bytes() function for type conversion
- Removed conversion logic from _normalize_field_value() to preserve its original intent
- Updated both call sites to chain normalize → convert → validate
- Added explanatory comment for __all__ in params.py to document public API intent

Addresses reviewer feedback about separation of concerns between normalization
(structural changes) and conversion (type transformations/IO operations).
The test was calling _normalize_field_value expecting UploadFile→bytes conversion,
but we moved that logic to _convert_uploadfile_to_bytes as part of the separation
of concerns refactoring.
@oyiz-michael oyiz-michael requested a review from tonnico November 2, 2025 18:30
leandrodamascena and others added 3 commits November 3, 2025 14:46
The openapi directory needs an __init__.py to be recognized as a Python package
by the documentation build system (mkdocstrings). Added a descriptive docstring
to document the module's purpose without exporting any symbols.
@tonnico
Copy link
Contributor

tonnico commented Nov 4, 2025

Would it make sense to rely on python-multipart here instead of implementing custom parsing logic?
That library is already used by Starlette and FastAPI, handles boundary quirks and large uploads reliably, and would likely reduce maintenance and edge-case handling on our side. Using it as an optional dependency might be a cleaner long-term approach.

@sonarqubecloud
Copy link

sonarqubecloud bot commented Nov 4, 2025

@powertools-for-aws-oss-automation

Not all issues are linked correctly.

Please link each issue to the PR either manually or using a closing keyword in the format fixes #<issue-number> format.

If mentioning more than one issue, separate them with commas: i.e. fixes #<issue-number-1>, closes #<issue-number-2>.

@leandrodamascena
Copy link
Contributor

I'm reviewing this PR this week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge event_handlers feature New feature or functionality on-hold This item is on-hold and will be revisited in the future size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. tests

Projects

None yet

3 participants