Skip to content

add force_update param on upload#397

Open
gschoeni wants to merge 3 commits intomainfrom
gs/force-update-on-upload
Open

add force_update param on upload#397
gschoeni wants to merge 3 commits intomainfrom
gs/force-update-on-upload

Conversation

@gschoeni
Copy link
Copy Markdown
Collaborator

We needed a way to bump a file's latest commit / timestamp, without bumping the files content. So I introduced this force_update parameter. Open to ideas on parameter naming...but this is the usage with our import and upload APIs:

Import from multi-part:

curl -v -X POST "http://localhost:3001/api/repos/Subaru/playground/workspaces/upload/main/uploads"
-H "Authorization: Bearer $API_KEY"
-F "commit_message=testing from cli"
-F "force_update=true"
-F "file[]=@/path/to/file.png"

and import from url:

curl -X POST \
"${BASE_API_URL}/repos/${NAMESPACE}/playground/import/main/uploads" \
-H "Authorization: Bearer ${TOKEN}"
-H "Content-Type: application/json" \
-d '{
"download_url": "https://example.com/image.jpg",
"name": "Greg",
"email": "greg@example.com",
"force_update": true
}'

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 27, 2026

📝 Walkthrough

Walkthrough

A force_update boolean parameter is introduced across the upload, import, and staging pipeline to allow re-uploading identical content and forcing new commits when no effective changes are detected. The flag is threaded through client APIs, core staging logic, server controllers, and staging behavior to override unmodified-file detection.

Changes

Cohort / File(s) Summary
API Client: Import & Upload
crates/lib/src/api/client/import.rs, crates/lib/src/api/client/versions.rs, crates/lib/src/api/client/workspaces/files.rs
Added force_update parameter to import_url(), parallel_large_file_upload(), and created new add_with_opts() with force_update; threading flag through upload/batching control flow to skip unmodified-file skipping when enabled.
Core Layer: Staging & File Operations
crates/lib/src/core/v_latest/add.rs, crates/lib/src/core/v_latest/workspaces/files.rs
Added force_update parameter to stage_file_with_hash() and created new add_with_opts(); when flag is true, using FileTime::now() and overriding unmodified files to StagedEntryStatus::Modified.
Core Layer: Push & Commit Writing
crates/lib/src/core/v_latest/push.rs, crates/lib/src/repositories/commits/commit_writer.rs
Updated multipart upload call to pass force_update; modified directory-hash aggregation to include vnode.id bytes in hasher.
Repository Layer: Workspace Files & Commits
crates/lib/src/repositories/workspaces/files.rs
Added add_with_opts() wrapper and updated import() signature to accept and forward force_update parameter.
Server Controllers: Import & Uploads
crates/server/src/controllers/import.rs, crates/server/src/controllers/versions/chunks.rs, crates/server/src/controllers/workspaces/files.rs
Extended request parsing to extract force_update from JSON/multipart; introduced StagingQueryParams for query-based flag; updated save_parts() to return force_update and multipart parsing to extract it.
Data Model
crates/lib/src/view/versions.rs
Added force_update: bool field with #[serde(default)] to CompleteVersionUploadRequest.
Integration Tests
crates/lib/src/api/client/import.rs, crates/lib/src/api/client/workspaces/commits.rs
Added test for import-with-force-update; added three tests validating re-upload behavior with/without force-update across workspaces.

Sequence Diagram

sequenceDiagram
    participant Client
    participant ServerCtrl as Server<br/>Controller
    participant CoreStage as Core<br/>Staging
    participant Storage

    Client->>ServerCtrl: POST /import or upload<br/>(with force_update flag)
    ServerCtrl->>ServerCtrl: Parse force_update<br/>from request
    ServerCtrl->>CoreStage: add_version_file(...,<br/>force_update)
    alt force_update = true
        CoreStage->>CoreStage: Override Unmodified→Modified<br/>Use FileTime::now()
        CoreStage->>Storage: Store file with new mtime
    else force_update = false
        CoreStage->>CoreStage: Skip if unmodified
        CoreStage->>Storage: Only stage if modified
    end
    CoreStage->>ServerCtrl: Return result
    ServerCtrl->>Client: Commit response
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • Oxen-AI/Oxen#604: Modifies the same client/server/core upload and multipart-completion codepaths affected by this force_update feature.
  • Oxen-AI/Oxen#645: Modifies workspace add implementation in crates/lib/src/api/client/workspaces/files.rs, overlapping with the new add_with_opts() delegation pattern introduced here.
  • Fix the usage of get_version_path in add_version_files #375: Modifies add_version_files() in core/v_latest/workspaces/files.rs for error-handling changes, intersecting with the force_update parameter threading.

Suggested reviewers

  • jcelliott
  • CleanCut
  • malcolmgreaves

Poem

🐰 A flag to force the upload flow,
No more "unmodified" no-no,
Staging files with brand new time,
Every version now sublime! ✨
Force it true, the commits bloom—
Clearing out the upload room!

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description check ✅ Passed The description clearly explains the purpose of the force_update parameter, provides concrete usage examples for both multipart and URL-based imports, and relates directly to the changes in the PR.
Docstring Coverage ✅ Passed Docstring coverage is 85.37% which is sufficient. The required threshold is 80.00%.
Title check ✅ Passed The title 'add force_update param on upload' is specific and directly summarizes the main change of adding a force_update parameter to upload-related endpoints, matching the changeset's primary objective.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch gs/force-update-on-upload

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
crates/lib/src/api/client/workspaces/files.rs (1)

207-225: Consider exposing force_update in add_files for consistency.

The add_files function hardcodes force_update: false. While this may be intentional for the current use case (bulk importing existing file structures), consider whether this function should also support the force_update parameter for API consistency.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/lib/src/api/client/workspaces/files.rs` around lines 207 - 225, The
add_files function currently calls upload_multiple_files with force_update
hardcoded to false; add a force_update parameter to add_files' signature (e.g.,
force_update: bool) and pass it through to upload_multiple_files instead of the
literal false, updating call sites of add_files to supply the appropriate value
(or default to false where callers aren’t changed) and updating any docs/tests
accordingly so API behavior is consistent with other file operations that accept
force_update.
crates/lib/src/api/client/import.rs (1)

233-234: Consider test stability with external URL dependency.

The test relies on an external URL (https://hub.oxen.ai/...). While this is likely a stable Oxen-hosted resource, external dependencies can cause test flakiness. Consider whether this test should be marked as an integration test or if there's a way to use a mock/local resource.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/lib/src/api/client/import.rs` around lines 233 - 234, The test
hard-codes an external URL into the download_url variable which can make the
test flaky; change the test in crates::lib::src::api::client::import.rs to avoid
the external dependency by either (a) using a local test fixture/bundled asset
path instead of "download_url", (b) mocking HTTP responses with a test HTTP
server (e.g., httptest/wiremock) and point download_url at the mock server, or
(c) mark the test as an integration/ignored test (#[ignore] or a dedicated
integration test cfg) if it must hit the real Oxen endpoint; update the test
setup and assertions accordingly so they reference the download_url symbol you
modify.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/server/src/controllers/import.rs`:
- Around line 183-186: The current extraction of force_update silently defaults
to false for non-boolean values; update the import handler where force_update is
computed (the let force_update = body.get("force_update")... expression) to
validate types: if body contains the "force_update" key but
body.get("force_update").as_bool() is None, return a 400 Bad Request response
indicating an invalid type; otherwise set force_update to the boolean value or
false when the key is absent. Ensure the new branch returns the appropriate
error early from the same function handling the request.

---

Nitpick comments:
In `@crates/lib/src/api/client/import.rs`:
- Around line 233-234: The test hard-codes an external URL into the download_url
variable which can make the test flaky; change the test in
crates::lib::src::api::client::import.rs to avoid the external dependency by
either (a) using a local test fixture/bundled asset path instead of
"download_url", (b) mocking HTTP responses with a test HTTP server (e.g.,
httptest/wiremock) and point download_url at the mock server, or (c) mark the
test as an integration/ignored test (#[ignore] or a dedicated integration test
cfg) if it must hit the real Oxen endpoint; update the test setup and assertions
accordingly so they reference the download_url symbol you modify.

In `@crates/lib/src/api/client/workspaces/files.rs`:
- Around line 207-225: The add_files function currently calls
upload_multiple_files with force_update hardcoded to false; add a force_update
parameter to add_files' signature (e.g., force_update: bool) and pass it through
to upload_multiple_files instead of the literal false, updating call sites of
add_files to supply the appropriate value (or default to false where callers
aren’t changed) and updating any docs/tests accordingly so API behavior is
consistent with other file operations that accept force_update.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c2a0eb18-da7a-4c09-bed9-65f21eff1ff3

📥 Commits

Reviewing files that changed from the base of the PR and between 08354c1 and 2007a5a.

📒 Files selected for processing (13)
  • crates/lib/src/api/client/import.rs
  • crates/lib/src/api/client/versions.rs
  • crates/lib/src/api/client/workspaces/commits.rs
  • crates/lib/src/api/client/workspaces/files.rs
  • crates/lib/src/core/v_latest/add.rs
  • crates/lib/src/core/v_latest/push.rs
  • crates/lib/src/core/v_latest/workspaces/files.rs
  • crates/lib/src/repositories/commits/commit_writer.rs
  • crates/lib/src/repositories/workspaces/files.rs
  • crates/lib/src/view/versions.rs
  • crates/server/src/controllers/import.rs
  • crates/server/src/controllers/versions/chunks.rs
  • crates/server/src/controllers/workspaces/files.rs

Comment on lines +183 to +186
let force_update = body
.get("force_update")
.and_then(|v| v.as_bool())
.unwrap_or(false);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Reject invalid force_update types instead of silently disabling the feature.

Line 183 currently falls back to false for non-boolean values, so malformed payloads can silently ignore a requested force update. Return 400 when the key is present but not boolean.

💡 Suggested fix
-    let force_update = body
-        .get("force_update")
-        .and_then(|v| v.as_bool())
-        .unwrap_or(false);
+    let force_update = match body.get("force_update") {
+        None => false,
+        Some(v) => v.as_bool().ok_or_else(|| {
+            OxenHttpError::BadRequest("`force_update` must be a boolean".into())
+        })?,
+    };
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
let force_update = body
.get("force_update")
.and_then(|v| v.as_bool())
.unwrap_or(false);
let force_update = match body.get("force_update") {
None => false,
Some(v) => v.as_bool().ok_or_else(|| {
OxenHttpError::BadRequest("`force_update` must be a boolean".into())
})?,
};
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/server/src/controllers/import.rs` around lines 183 - 186, The current
extraction of force_update silently defaults to false for non-boolean values;
update the import handler where force_update is computed (the let force_update =
body.get("force_update")... expression) to validate types: if body contains the
"force_update" key but body.get("force_update").as_bool() is None, return a 400
Bad Request response indicating an invalid type; otherwise set force_update to
the boolean value or false when the key is absent. Ensure the new branch returns
the appropriate error early from the same function handling the request.

@gschoeni gschoeni changed the title Gs/force update on upload add force_update param on upload Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant