PSv2 cleanup: use is_complete() and dispatch_mode in job progress handler

[mihow]

Summary

• Replace hardcoded stage == "results" check in _update_job_progress with job.progress.is_complete(), which verifies all stages are done rather than assuming a specific stage name signals completion
• Replace feature_flags.async_pipeline_workers check in _cleanup_job_if_needed with job.dispatch_mode == JobDispatchMode.ASYNC_API, which is immutable for the job's lifetime and more correct than re-reading a mutable flag

Both issues were identified during the dispatch_mode PR review. The existing abstractions (JobProgress.is_complete() and Job.dispatch_mode) already handle these cases correctly.

Test plan

• All 32 job-related tests pass (python manage.py test -k job)
• Cleanup integration tests updated to set dispatch_mode=ASYNC_API and complete all stages
• No remaining hardcoded stage == "results" checks in ami/

Summary by CodeRabbit

• Bug Fixes

  • Improved job completion detection to be more reliable across different processing workflows.

• Chores

  • Refactored resource cleanup logic to align with job dispatch configuration.

[netlify]

✅ Deploy Preview for antenna-ssec canceled.

Name	Link
🔨 Latest commit	6fde58e
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-ssec/deploys/698a754385a16a0008997bfd

[netlify]

✅ Deploy Preview for antenna-preview ready!

Name	Link
🔨 Latest commit	6fde58e
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-preview/deploys/698a7543c6de8c0008bb55ae
😎 Deploy Preview	https://deploy-preview-1125--antenna-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
Lighthouse	1 paths audited Performance: 41 (🔴 down 25 from production) Accessibility: 80 (no change from production) Best Practices: 100 (no change from production) SEO: 92 (no change from production) PWA: 80 (no change from production) View the detailed breakdown and full score reports

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[coderabbitai]

📝 Walkthrough

Walkthrough

This change refactors job completion and cleanup logic from stage/progress-based conditions to generalized completeness checks, and replaces feature-flag-based routing with dispatch-mode-based cleanup eligibility determination.

Changes

Cohort / File(s)	Summary
Job completion and cleanup logic ami/jobs/tasks.py	Refactored progress update trigger from stage == "results" and progress >= 1.0 check to job.progress.is_complete(). Updated cleanup eligibility from ML job feature flag check to job.dispatch_mode == JobDispatchMode.ASYNC_API condition. Job is now re-fetched outside transaction before cleanup execution.
Cleanup test updates ami/ml/orchestration/tests/test_cleanup.py	Added JobDispatchMode.ASYNC_API enum import and usage in Job creation. Test now advances through all three stages (collect, process, results) instead of single stage update to exercise completion logic.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

• fix: Properly handle async job state with celery tasks  #1114: Introduces JobProgress.is_complete() for preventing premature job SUCCESS in Celery task handlers, aligning with completion check refactoring in this PR.
• PSv2: Implement queue clean-up upon job completion #1113: Original implementation of _cleanup_job_if_needed() with feature-flag-based routing; this PR refactors that same cleanup logic to use dispatch mode-based eligibility.

Poem

🐰 A rabbit hops through cleanup paths so clear,
Dispatch modes guide each job without fear,
Progress checks unified, no stages to decode,
Simpler logic makes the burrow's road! ✨

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description includes Summary and Test plan sections, but lacks List of Changes, Related Issues, Detailed Description, How to Test, Screenshots, Deployment Notes, and Checklist sections from the template.	Add missing template sections: List of Changes (modified files), Related Issues (PR #1125), Detailed Description (rationale and side effects), How to Test (test commands), Deployment Notes, and Checklist items.
Docstring Coverage	⚠️ Warning	Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main changes: refactoring to use is_complete() and dispatch_mode in the job progress handler.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch fix/job-status-cleanup

---

No actionable comments were generated in the recent review. 🎉

🧹 Recent nitpick comments

ami/ml/orchestration/tests/test_cleanup.py (1)

20-21: Stale docstring: still references async_pipeline_workers.

The setUp docstring says "Set up test fixtures with async_pipeline_workers enabled" but the cleanup eligibility is now driven by dispatch_mode, not the feature flag. Consider updating for consistency.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Refactors async job progress completion and cleanup logic to rely on existing abstractions (JobProgress.is_complete() and Job.dispatch_mode) rather than hardcoded stage names and mutable feature flags.

Changes:

• Update _update_job_progress to treat job completion based on job.progress.is_complete() instead of stage == "results".
• Update _cleanup_job_if_needed to gate NATS/Redis cleanup on job.dispatch_mode == ASYNC_API rather than project.feature_flags.async_pipeline_workers.
• Adjust cleanup integration test setup to use dispatch_mode=ASYNC_API and complete all stages before asserting cleanup.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
ami/jobs/tasks.py	Switches completion detection and cleanup gating to job-level abstractions (is_complete, dispatch_mode).
ami/ml/orchestration/tests/test_cleanup.py	Updates integration test to align with new completion semantics and dispatch mode gating.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.