Skip to content

Fix Azure File SAS signed-start/expiry window (#1533)#1534

Open
bingran-you wants to merge 2 commits intodevfrom
bry/sas-token-window-fix-1533
Open

Fix Azure File SAS signed-start/expiry window (#1533)#1534
bingran-you wants to merge 2 commits intodevfrom
bry/sas-token-window-fix-1533

Conversation

@bingran-you
Copy link
Copy Markdown
Contributor

Summary

Fix for #1533: production worker SAS tokens for Azure File shares were being generated with signed-start time at/after signed-expiry, producing immediate 403 AuthenticationFailed during azcopy download.

Changes to generate_share_sas() in DoWhiz_service/run_task_module/src/run_task/codex.rs:

  • Explicitly pass --start back-dated 15 minutes (clock-skew buffer between this VM and the Azure File service).
  • Extend --expiry from 1 h → 6 h to cover long azcopy runs on large workspaces (observed Elapsed Time (Minutes): 62.2958 on prod).

Evidence

Observed on dowhizprod1 dw_worker (see #1533 for table). All observed SAS windows were ≤0 seconds wide.

Test plan

  • cargo build -p run_task_module --release
  • cargo test -p run_task_module — existing fake-az test fixtures accept any args, so no test updates needed.
  • Deploy to staging VM, run a real codex execution (./DoWhiz_service/scripts/run_employee.sh boiled_egg 9001 --skip-hook --skip-ngrok), verify no has to be after signed start time in worker logs.
  • Deploy to prod, watch pm2 logs dw_worker for 30 min — expect zero recurrences.

Passing only --expiry to az storage share generate-sas has been observed
to produce tokens whose signed-start lands at or after signed-expiry,
yielding 403 AuthenticationFailed "Signed expiry time has to be after
signed start time" during azcopy download on production.

Explicitly set --start back-dated 15 minutes (clock-skew buffer) and
extend --expiry from 1h to 6h so long azcopy runs on large workspaces
complete inside the window.

Fixes #1533
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 22, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
dowhiz Ready Ready Preview, Comment Apr 22, 2026 5:16am

@bingran-you
Copy link
Copy Markdown
Contributor Author

I don't see any human review comments or requested reviewers on this PR right now, so there was nothing specific to address. I still validated the branch locally:

  • AUTO-RUN-01: PASS — cargo test -p run_task_module
  • LIVE / MANUAL verification: SKIP — not executed from this workspace

This reply was drafted by breeze, an autonomous agent running on behalf of the account owner.

@bingran-you bingran-you added breeze:done Breeze finished handling this item and removed breeze:wip Breeze is actively working on this item labels Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breeze:done Breeze finished handling this item

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant