ML Intern backlog prioritization report - 2026-05-04

[lewtun]

ML Intern Backlog Prioritization

Generated: 2026-05-04T13:18:56.926357+00:00
Model: anthropic/claude-opus-4-6

Sources: github_issue=24, github_pr=50, hf_discussion=14

Summary

Critical security PRs (#96, #85, #83) and agent-quality fix (#88) are reviewed and ready to merge. Hosted Space has multiple reliability regressions (session zombies, Pro detection, onboarding CSS, blank pages) blocking paying users. Provider expansion (local models, Bedrock, Azure, Gemini) is the top feature theme with 6+ PRs. Cost guardrails and session durability are strategic priorities.

Can Be Closed

No high-confidence resolved-in-main candidates found.

Highest Impact Next

1. Merge 3 security PRs: CVE patch, sandbox auth, SSRF fix (impact 5/5, effort 1/5, confidence 0.95)

   • Recommendation: Merge PR fix: upgrade authlib to 1.6.9 (CVE-2026-27962) #96 (authlib CVE), fix(sandbox): require bearer-token auth on sandbox API (fixes #78) #85 (sandbox bearer auth), fix: validate URL origin in fetch_hf_docs to prevent SSRF #83 (SSRF origin validation) — all reviewed APPROVE/safe-to-merge, effort 1 each
   • Rationale: Unauthenticated sandbox RCE + HF_TOKEN exfil, SSRF token leak, and CRITICAL CVE — all exploitable today with reviewed fixes waiting
   • Next action: Rebase fix(sandbox): require bearer-token auth on sandbox API (fixes #78) #85 to resolve dirty state; approve and merge all three in one batch
   • Sources: github_pr#96, github_pr#85, github_pr#83

2. Merge thinking_blocks fix (PR fix: thread thinking_blocks through history so extended thinking survives tool turns #88) (impact 5/5, effort 1/5, confidence 0.95)

   • Recommendation: Merge PR fix: thread thinking_blocks through history so extended thinking survives tool turns #88 — reviewed 'SAFE TO MERGE, LOW RISK', 51 lines, 1 file. Fixes silent quality degradation on every Anthropic session.
   • Rationale: Extended thinking silently drops after first tool call on all Anthropic models — degrades agent quality for the entire session
   • Next action: Confirm rebase is clean and merge
   • Sources: github_pr#88

3. Fix onboarding CSS overflow blocking Start button (impact 5/5, effort 1/5, confidence 0.95)

   • Recommendation: Change onboarding container overflow:hidden to overflow-y:auto — root cause fully documented with screenshots and confirmed fix
   • Rationale: Completely blocks first-time users on 1366x768 screens; users must open DevTools to use the product
   • Next action: Push 1-line CSS fix to main
   • Sources: hf_discussion#19

4. Fix MCP tool stall (hub_repo_details hangs 5+ min) (impact 5/5, effort 2/5, confidence 0.9)

   • Recommendation: Adopt PR fix: add MCP tool timeout + running state to prevent silent stall after hub_repo_details (#127) #190's 30s timeout + running-state emission into a maintainer branch since PR was closed but issue Agent appears to stall for minutes after hub_repo_details with no progress logs #127 persists
   • Rationale: Maintainer-filed bug; agent hangs silently forcing Ctrl+C, losing all accumulated state
   • Next action: Cherry-pick timeout logic from PR fix: add MCP tool timeout + running state to prevent silent stall after hub_repo_details (#127) #190 into maintainer fix; add integration test
   • Sources: github_issue#127, github_pr#190

5. Fix session zombie state and quota cleanup (impact 5/5, effort 3/5, confidence 0.85)

   • Recommendation: Fix session lifecycle so stopped sessions free slots; merge PR Fix session creation race condition and add capacity tests #77 (race condition) after pending review completes
   • Rationale: Users permanently locked out after hitting session limit; all remaining sessions stuck in 'catching up' — completely blocks further use
   • Next action: Audit session cleanup on stop/delete; review PR Fix session creation race condition and add capacity tests #77 race condition fix; deploy together
   • Sources: hf_discussion#17, github_pr#77, hf_discussion#20

6. Fix Pro plan detection and entitlement checks (impact 4/5, effort 2/5, confidence 0.85)

   • Recommendation: Merge community PR fix from discussion Improve job approval UX with tabs and job URL #21; verify entitlement logic covers all subscription states
   • Rationale: Paying Pro users can't access paid features — 3 separate users confirmed; directly harms revenue trust
   • Next action: Review and merge community PR from discussion Improve job approval UX with tabs and job URL #21; add test for subscription edge cases
   • Sources: hf_discussion#18, hf_discussion#25

7. Merge Bedrock region prefix fix (PR fix(research): inherit main model's Bedrock region prefix in sub-agent #185) (impact 4/5, effort 1/5, confidence 0.9)

   • Recommendation: Cherry-pick or merge PR fix(research): inherit main model's Bedrock region prefix in sub-agent #185 — 17-line targeted fix, verified with live Bedrock calls, blocks all non-US AWS users
   • Rationale: Hard-coded 'us.' prefix completely breaks research sub-agent for non-US Bedrock users
   • Next action: Approve and merge; CI failure confirmed unrelated to fix
   • Sources: github_issue#184, github_pr#185

8. Add LICENSE file to unblock adoption (impact 4/5, effort 1/5, confidence 0.9)

   • Recommendation: Add Apache 2.0 LICENSE file — 5 thumbs-up, enterprise users explicitly blocked from testing
   • Rationale: No license = legal blocker for enterprises and external contributors; 5 reactions + multiple +1 comments
   • Next action: Add LICENSE file and README badge; use PR License and Citation #178 as reference
   • Sources: github_issue#41, github_pr#178

Features

1. Local model support (Ollama/vLLM/OpenAI-compat) (impact 5/5, effort 3/5, confidence 0.75)

   • Recommendation: Review PR Improved local model support #166 (improved local model support with feature gate); use Add local model support via Ollama #44 Ollama work as reference. Highest-demand feature.
   • Next action: Maintainer review of PR Improved local model support #166 security model; decide on ENABLE_LOCAL_MODELS gate
   • Sources: github_issue#94, github_pr#166, github_pr#44, github_pr#68

2. Provider adapter refactor + Bedrock/Azure/Gemini (impact 4/5, effort 3/5, confidence 0.75)

   • Recommendation: Merge PR feat: Refactor existing LLM resolution into provider adapters #55 (adapter refactor, reviewed SAFE) first, then feat: support Azure OpenAI via azure/ prefix #80 (Azure, LGTM) and Add Amazon Bedrock support for Claude models #66 (Bedrock). Gemini PRs feat: Add Gemini API support and GEAP/Vertex AI support #131/feat(gemini): add Gemini model support via GEMINI_API_KEY #95 need conflict resolution.
   • Next action: Rebase and merge PR feat: Refactor existing LLM resolution into provider adapters #55; queue feat: support Azure OpenAI via azure/ prefix #80 and Add Amazon Bedrock support for Claude models #66 immediately after
   • Sources: github_pr#55, github_pr#66, github_pr#80, github_pr#182, github_pr#131, github_pr#95

3. Image, file, and dataset attachments for web+CLI (impact 4/5, effort 4/5, confidence 0.6)

   • Recommendation: Run automated review on PR Add image, file, and dataset attachments for web and CLI #186; it addresses both give access to laptop files to upload as dataset or to train #157 and upload images or files #158 with CLI+web support
   • Next action: Trigger PR review; resolve dirty merge state
   • Sources: github_issue#157, github_issue#158, github_pr#186

4. OpenRouter / custom OpenAI-compatible base URL (impact 4/5, effort 1/5, confidence 0.8)

   • Recommendation: Implement OPENAI_BASE_URL passthrough — minimal change, follows ecosystem convention, unlocks 300+ models
   • Next action: Accept a small PR respecting OPENAI_BASE_URL env var
   • Sources: github_issue#197, github_pr#188

5. Cost guardrails: prompt caching, iteration cap, research concurrency (impact 5/5, effort 4/5, confidence 0.85)

   • Recommendation: Sprint on P0 items: add cache_control markers, lower max_iterations 300→40, cap concurrent research subagents, default cheaper research model
   • Next action: Internal sprint; start with max_iterations cap and prompt caching
   • Sources: github_issue#61

6. Expand pre-flight checks for hf_jobs approval (impact 4/5, effort 2/5, confidence 0.7)

   • Recommendation: Add 4 static checks (timeout, hub_model_id, flash-attn, trackio) + credit pre-check to prevent wasted GPU spend
   • Next action: Accept PR for reliability_checks.py expansion; add test coverage
   • Sources: github_issue#203, github_issue#125

7. Notify user when agent needs input/approval (impact 3/5, effort 1/5, confidence 0.7)

   • Recommendation: Review PR feat(cli): optional notify when blocked on approval #106 (--notify-on-block CLI flag) after CI fix ci: switch review workflow to pull_request_target for fork PRs #107 lands; simpler and safer than PR feat: add notifications when intern needs user input #71
   • Next action: Merge PR ci: switch review workflow to pull_request_target for fork PRs #107 first, then review feat(cli): optional notify when blocked on approval #106
   • Sources: github_issue#65, github_issue#103, github_pr#106

8. Evaluation and benchmarking CLI (impact 4/5, effort 4/5, confidence 0.5)

   • Recommendation: Request author fix Claude-flagged correctness issues in PR Add local evaluation and benchmarking CLI for issue 84 #98 before re-review
   • Next action: Comment on PR Add local evaluation and benchmarking CLI for issue 84 #98 with specific fix requests; set deadline
   • Sources: github_issue#84, github_pr#98

9. Background sessions on Mongo control plane (impact 5/5, effort 5/5, confidence 0.7)

   • Recommendation: Fix P0 blockers (_enforce_gated_model_quota crash, reconnect 404) before merge; PR has 83 tests and thorough design
   • Next action: Author to fix remaining P0s; re-trigger automated review
   • Sources: github_pr#206

10. Claude Code project-mode / plugin support (impact 4/5, effort 4/5, confidence 0.5)

• Recommendation: Strategic decision needed: is Claude Code a supported frontend? If yes, review PR feat: add Claude Code project-mode support #113 first; defer feat: ship as a Claude Code plugin #114 until feat: add Claude Code project-mode support #113 lands
• Next action: Maintainer decision on Claude Code scope
• Sources: github_pr#113, github_pr#114, github_issue#74

11. Opt-in LangFuse observability callback (impact 3/5, effort 2/5, confidence 0.65)

• Recommendation: Adopt env-gated design from PR feat(observability): opt-in LangFuse callback for litellm calls #198 (mandatory LANGFUSE_HOST); implement as litellm callback
• Next action: Reopen or reimplement based on PR feat(observability): opt-in LangFuse callback for litellm calls #198 approach
• Sources: github_issue#196, github_pr#198

12. Frontend UX: example prompts, sidebar, copy/regenerate (impact 3/5, effort 3/5, confidence 0.8)

• Recommendation: Fix isProcessing stuck-true bug and accessibility issues in PR frontend: UI polish, example prompts, sidebar actions, assistant copy/regenerate #38, then merge — frontend-only, good screenshots
• Next action: Author to fix regenerate error handling; rebase and merge
• Sources: github_pr#38, hf_discussion#28

Fixes

1. CVE-2026-27962 authlib upgrade (impact 5/5, effort 1/5, confidence 0.9)

   • Recommendation: Fix title mismatch (says 1.6.9, is 1.7.0), vet joserfc dep, merge urgently
   • Sources: github_pr#96

2. Sandbox API unauthenticated RCE (impact 5/5, effort 1/5, confidence 0.95)

   • Recommendation: Rebase and merge — reviewed APPROVE, adds bearer-token auth to all sandbox endpoints
   • Sources: github_pr#85

3. SSRF in fetch_hf_docs leaks HF token (impact 5/5, effort 1/5, confidence 0.95)

   • Recommendation: Merge — reviewed safe-to-merge, origin validation with good test coverage
   • Sources: github_pr#83

4. Extended thinking drops after first tool call (impact 5/5, effort 1/5, confidence 0.95)

   • Recommendation: Merge — 51 lines, 1 file, 'SAFE TO MERGE, LOW RISK'
   • Sources: github_pr#88

5. Start session button hidden by CSS overflow (impact 5/5, effort 1/5, confidence 0.95)

   • Recommendation: 1-line CSS fix: overflow:hidden → overflow-y:auto on onboarding container
   • Sources: hf_discussion#19

6. Concurrent plan overwrites + CORS rejection on Spaces (impact 4/5, effort 1/5, confidence 0.8)

   • Recommendation: Cherry-pick per-session plan isolation and CORS SPACE_HOST fix from closed PR fix: async plan isolation, blocking API call, and CORS for production #205
   • Sources: github_pr#205

7. Read tool crashes on string offset/limit (impact 4/5, effort 1/5, confidence 0.9)

   • Recommendation: Merge after also fixing bash_handler timeout (same bug); reviewed 'ready to merge'
   • Sources: github_pr#110

8. HF Pro subscription not detected after upgrade (impact 4/5, effort 2/5, confidence 0.85)

   • Recommendation: Review community PR fix from discussion Improve job approval UX with tabs and job URL #21; test all subscription states
   • Sources: hf_discussion#18, hf_discussion#25

9. Stopped sessions don't free quota; catch-up stuck (impact 5/5, effort 3/5, confidence 0.85)

   • Recommendation: Fix session cleanup lifecycle; merge PR Fix session creation race condition and add capacity tests #77 race condition fix after review
   • Sources: hf_discussion#17, hf_discussion#20, github_pr#77

10. Chat messages spontaneously disappear mid-session (impact 4/5, effort 3/5, confidence 0.65)

• Recommendation: Audit chat state persistence for race conditions or storage limits
• Sources: hf_discussion#26, hf_discussion#29

11. Dotenv ignored when shell has stale env var (impact 3/5, effort 1/5, confidence 0.8)

• Recommendation: Confirm maintainer intent (ambiguous close comment); merge if acceptable — reviewed LGTM
• Sources: github_pr#120

12. 401 Client Error on sandbox creation (impact 4/5, effort 3/5, confidence 0.55)

• Recommendation: Investigate regression; merge PR Log exception for HF whoami username resolution failure #219 for better logging; add sandbox-creation alerting
• Sources: github_issue#214, hf_discussion#33

Other / Watchlist

1. Add LICENSE file (Apache 2.0) (impact 4/5, effort 1/5)

   • Recommendation: Add immediately — blocks enterprise adoption; use PR License and Citation #178 as reference
   • Sources: github_issue#41, github_pr#178

2. Add CONTRIBUTING.md (impact 2/5, effort 1/5)

   • Recommendation: Merge PR docs: add CONTRIBUTING.md #130 — docs-only, 156 additions, no code changes
   • Sources: github_pr#130

3. Update README model example + env var annotations (impact 2/5, effort 1/5)

   • Recommendation: Quick-merge both — trivial doc fixes, zero risk
   • Sources: github_pr#220, github_pr#109

4. Fix CI review workflow for fork PRs (impact 3/5, effort 1/5)

   • Recommendation: Rebase and merge — blocks automated review for all external contributors
   • Sources: github_pr#107

5. Restructure system prompt tone (impact 3/5, effort 2/5)

   • Recommendation: Re-add 3 dropped behavioral rules flagged in review before merging
   • Sources: github_pr#33

6. Use huggingface_hub.get_token() as fallback (impact 3/5, effort 1/5)

   • Recommendation: Review and merge — fixes real onboarding friction (issue Use logged in hf token using huggingface_hub.get_token() instead of requiring env variable #23)
   • Sources: github_pr#34

Clusters

• Security vulnerabilities

  • Summary: CVE in authlib, unauthenticated sandbox RCE, and SSRF token leak — all have reviewed PRs ready to merge
  • Sources: github_pr#96, github_pr#85, github_pr#83

• Session reliability & state persistence

  • Summary: Sessions get stuck as zombies, catch-up broken, messages disappear, race conditions on creation — users lose work and get locked out
  • Sources: hf_discussion#17, hf_discussion#20, hf_discussion#26, hf_discussion#29, github_pr#77, github_pr#205

• Pro/payment detection failures

  • Summary: Pro subscribers can't access paid features; 402 errors silently swallowed on training jobs
  • Sources: hf_discussion#18, hf_discussion#25, github_issue#125

• Provider expansion (local, Azure, Bedrock, Gemini)

  • Summary: Users locked to Anthropic/OpenAI APIs; demand for local models, Bedrock, Azure, Gemini, OpenRouter across 10+ PRs and issues
  • Sources: github_issue#94, github_pr#166, github_pr#44, github_pr#55, github_pr#60, github_pr#66, github_pr#80, github_pr#131, github_pr#95, github_pr#182, github_issue#197, github_pr#188, github_pr#68

• File & data uploads

  • Summary: Users can't provide local files, images, or datasets to the agent — blocks real ML workflows
  • Sources: github_issue#157, github_issue#158, github_pr#186, github_issue#187

• Notifications when agent needs input

  • Summary: Users multitasking miss approval prompts; two PRs compete — feat(cli): optional notify when blocked on approval #106 is simpler and safer
  • Sources: github_issue#65, github_issue#103, github_pr#71, github_pr#106

• Agent stalls & cost runaway

  • Summary: Agent hangs silently on MCP calls; research subagents run to 100k+ tokens with no cost cap or caching
  • Sources: github_issue#127, github_pr#190, github_issue#61

• Bedrock region & sub-agent routing

  • Summary: Hard-coded 'us.' prefix breaks research sub-agent for all non-US AWS regions; 17-line fix ready
  • Sources: github_issue#184, github_pr#185

• Sandbox creation & auth errors

  • Summary: 401/429 errors on sandbox creation; model default regression blocked all users; logging gaps hide root cause
  • Sources: github_issue#214, hf_discussion#33, hf_discussion#31, github_pr#219

• Claude Code integration

  • Summary: Claude Code project-mode and plugin support to use ml-intern tools within Claude Max subscriptions
  • Sources: github_pr#113, github_pr#114, github_issue#74

• Documentation & i18n

  • Summary: Missing LICENSE, no CONTRIBUTING.md, outdated README, translation requests
  • Sources: github_issue#41, github_pr#178, github_pr#130, github_pr#220, github_pr#109, github_pr#149, github_issue#147

• Observability & evaluation

  • Summary: No LLM trace export, no tool-level telemetry, no benchmark evaluation pipeline
  • Sources: github_issue#196, github_pr#198, github_issue#155, github_issue#84, github_pr#98