Skip to content

Conversation

@Ahmedsam199
Copy link

New Features

  • Direct Upload Checkbox: Introduced a checkbox in dfp_external_storage to enable direct uploads, enhancing user control over the upload process.
  • Direct Upload Button: Implemented a direct upload button in the file uploader, providing a more intuitive and faster upload experience.
  • Multiple Buckets Validation: Added validation in DFPExternalStorage to prevent configuration with multiple buckets, ensuring data integrity and preventing potential conflicts.
  • Direct Upload Validation: Implemented a validation mechanism in dfp_external_storage to prevent multiple direct uploads while enabled, avoiding resource contention.
  • Get Presigned URL: Implemented API endpoint to generate presigned URLs, enhancing security and flexibility in accessing files.
  • Create File API: Added a new API endpoint for creating files, enabling programmatic file management.
  • Enabled Direct Upload Doc Fetching: Enhanced the API to fetch only enabled direct upload documents, optimizing performance.
  • Minio Presigned URL Connection: Exposed get_presigned_url connection for MinioConnection, allowing external access to files via presigned URLs.

Bug Fixes

  • File Population Removal: Removed file population for new records in the file uploader, resolving issues related to data integrity.
  • Child Table Field Trigger: Fixed a bug where the field in the child table would not trigger when a file was uploaded using the main Frappe on_upload function.
  • Server Timestamp Issue: Removed the parent document refresh in api.py to prevent server timestamp issues during file uploads.
  • S3 Key Length: Extended the dfp_external_storage_s3_key length to 255 characters to accommodate longer file paths, preventing truncation issues.

Documentation

  • Direct Upload Documentation: Added documentation for direct uploads to the README file, providing comprehensive instructions for users.

Refactor

  • Storage Name Dependency Removal: Removed the storage_name dependency in api.py, improving code maintainability.

Testing

  • Create File API Test: Added a test case for the create file API, ensuring its reliability and functionality.
  • Presigned URL Tests: Added tests for presigned URLs, verifying their proper generation and usage.
  • Multiple Disabled Direct Upload Docs Test: Added a test case for multiple disabled direct upload documents, validating the system's behavior in such scenarios.

@developmentforpeople
Copy link
Owner

Hi @Ahmedsam199 thank you very much for your PR :) !!! I am reviewing it locally. I will ask some questions here related to some lines :) Thanks thanks thanks for your time and effort doing this new great functionality and creating a PR! :)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ahmedsam199 for all "file_uploader" folder, do you think it is possible to reuse funcionality from Frappe and extend it here with custom props and methods?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couldn't figure it out honestly



@frappe.whitelist()
def generate_presigned_url(file_name, file_path="Record"):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not using the MinioConnection->presigned_get_object() within dfp_external_storage.py?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i just want to keep the api separate from your logic

}

@frappe.whitelist()
def create_file_record(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whitelisted method create_file_record is open to anyone without system user access to upload files to your S3, is that right? I am wrong?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's because if you want to use the direct upload separated from frappe UI

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ahmedsam199 I installed locally your PR to do some checks.

At this moment I localized one security issue that should be resolved before merging the PR.

  1. create_file_record could be used by a low permissions user to create almost any document in Frappe, because kwargs can override even the doctype parameter and document is created with ignore_permissions=True:
Screenshot from 2025-12-25 16-57-10
  1. We should be able to find a way for overriding class FileUploader to add our functionality without duplicate so many files from frappe. If Frappe team makes improvements to those files, we will be always affected. If not easy we could create only a bundle.js but importing all files that are the same, for example. Tell me if you want I think about it :)

  2. Using always te same: spaces or tabs, not mixed:

image

@Ahmedsam199 thanks again for your time, tell me if I can help you with something ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants