Skip to content

feat(monitor): backend-driven SWE-bench evaluation with thread-native traces #93

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
shuxueshuxue wants to merge 6 commits into from
Closed

feat(monitor): backend-driven SWE-bench evaluation with thread-native traces #93

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
0dcbc51
feat(monitor): bind trace to thread/run and simplify session page
Feb 25, 2026
1d797c2
refactor(monitor): run swebench evals directly and remove session com…
Feb 25, 2026
4c94b3d
fix(monitor): fallback thread trace to checkpoints for swebench runs
Feb 25, 2026
7166349
monitor: derive eval final status from score errors
Feb 26, 2026
f171784
monitor: enforce sandboxed eval runner and live progress stats
Feb 26, 2026
191ab98
feat(monitor): governance baseline for eval semantics and publishable…
Feb 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion backend/web/models/requests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ class CreateThreadRequest(BaseModel):
sandbox: str = "local"
cwd: str | None = None
model: str | None = None
agent: str | None = None
mode: str = "normal"


class RunRequest(BaseModel):
Expand Down
3 changes: 2 additions & 1 deletion backend/web/models/thread_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,5 @@ class ThreadConfig(BaseModel):
model: str | None = None
queue_mode: str = "steer" # Deprecated: kept for SQLite schema compat, not read/written
observation_provider: str | None = None # "langfuse" | "langsmith" | None
agent: str | None = None # Member name for this thread
thread_mode: str = "normal" # "normal" | "evaluation"
keep_full_trace: bool = False
Loading