Skip to content

Add agentic workflow enhancements: timeout, timing, truncation, enriched info#2

Open
samtalki wants to merge 5 commits intomainfrom
feature/agentic-workflow-enhancements
Open

Add agentic workflow enhancements: timeout, timing, truncation, enriched info#2
samtalki wants to merge 5 commits intomainfrom
feature/agentic-workflow-enhancements

Conversation

@samtalki
Copy link
Owner

Summary

  • Eval timeout: optional timeout parameter kills the worker on hung/infinite code (e.g., eval(code="while true end", timeout=5))
  • Execution timing: every eval result now includes elapsed time ([45.2ms] or [1.23s])
  • Output truncation: max_output parameter (default 50K chars) prevents context window overflow, preserving head (60%) + tail (40%)
  • Enriched info: info tool now shows variable types and sizes (e.g., x::Int64, df::DataFrame (1000, 5)) instead of just names
  • Configurable stackframes: max_stackframes parameter on eval controls error trace depth (default 5)

Zero new dependencies, no new MCP tools — all features extend existing eval and info tools. Plugin docs updated.

Test plan

  • All 103 tests pass (julia --project=. -e "using Pkg; Pkg.test()")
  • New tests for: timeout (sleep+kill+respawn), timing (format_elapsed, result contains timing), truncation (head+tail, marker), enriched info (types/sizes), configurable stackframes
  • Manual: eval(code="1+1") shows timing, eval(code="sleep(5)", timeout=1) times out, info shows typed variables

🤖 Generated with Claude Code

… and configurable stackframes

Five agentic workflow enhancements to the eval and info tools:

1. Eval timeout with worker kill: optional `timeout` parameter races eval
   against a timer; on timeout, kills the worker and returns TimeoutError
2. Execution timing: every eval returns elapsed time (e.g., [45.2ms])
3. Output truncation: `max_output` parameter (default 50K chars) prevents
   context window overflow with head+tail preservation
4. Enriched info: variables now include type and size (e.g., x::Int64,
   df::DataFrame (1000, 5)) — no extra eval calls needed
5. Configurable stacktrace depth: `max_stackframes` parameter on eval

Zero new dependencies, no new MCP tools. All 103 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@samtalki
Copy link
Owner Author

Code review

Found 7 issues:

  1. truncate_output uses byte-based string indexing on Unicode strings, will crash with StringIndexError on multi-byte UTF-8 characters (box-drawing chars, Greek letters, math symbols common in Julia output). length(text) returns character count but text[1:n] indexes by bytes. Fix: use first(text, n) and last(text, n).

return text[1:head_chars] *
"\n\n... [$(removed) characters truncated] ...\n\n" *
text[end-tail_chars+1:end]
end

  1. Channel race condition in timeout implementation leaks async tasks. Channel{Any}(1) with two @async producers means the losing task's put! creates an unconsumed item. After timeout kills the worker, the eval @async task continues running and attempts fetch(future) on a dead worker, then put!s the error to an abandoned channel. This also departs from the documented expression-based IPC pattern (CLAUDE.md says "Uses remotecall_fetch(Core.eval, worker_id, Main, expr) instead of closures to avoid serialization issues").

AgentREPL.jl/src/worker.jl

Lines 134 to 153 in 4b5d7f0

# With timeout: race remotecall against a timer
result_channel = Channel{Any}(1)
future = remotecall(Core.eval, worker_id, Main, eval_expr)
# Race: eval completion vs timeout
@async begin
try
result = fetch(future)
put!(result_channel, (:ok, result))
catch e
put!(result_channel, (:error, e))
end
end
@async begin
sleep(timeout)
put!(result_channel, (:timeout, nothing))
end
tag, payload = take!(result_channel)

  1. get_worker_info() size logic shows misleading length=1 for all scalar variables. For scalars, size(42) returns (), length(()) == 0 is not > 1, so it falls through to "length=$(length(val))" which gives "length=1" for every scalar. Fix: check !isempty(s) before producing size output.

AgentREPL.jl/src/worker.jl

Lines 191 to 197 in 4b5d7f0

size_str = try
if applicable(size, val) && !(val isa AbstractString)
s = size(val)
length(s) > 1 ? string(s) : "length=$(length(val))"
elseif applicable(length, val)
"length=$(length(val))"
else

  1. mode_tool is still registered in server.jl but removed from plugin README and SKILL.md documentation, creating an inconsistency between server and docs. (CLAUDE.md says "Seven tools registered via ModelContextProtocol.jl" and lists mode as tool #7)

log_viewer_tool = create_log_viewer_tool()
mode_tool = create_mode_tool()
# Create and start the server
server = mcp_server(
name = "julia-repl",
version = "0.5.0",
description = "Persistent Julia REPL for AI agents - eliminates TTFX",
tools = [eval_tool, reset_tool, info_tool, pkg_tool, activate_tool, log_viewer_tool, mode_tool]
)

  1. CLAUDE.md Key Functions section is now stale: capture_eval_on_worker(code) signature changed to capture_eval_on_worker(code; timeout=nothing) returning a 4-tuple (value_str, output, error_str, elapsed) instead of a 3-tuple. New functions truncate_output and format_elapsed in formatting.jl are also not listed.

- **`capture_eval_on_worker(code)`** - Evaluates code on the worker with output capture (`worker.jl`)

  1. .mcp.json passes "JULIA_REPL_PROJECT": "${JULIA_REPL_PROJECT}" which may resolve to an empty string when unset, causing server.jl to error with "Cannot activate project: directory '' not found" since empty string passes the !== nothing check but fails isdir(""). (CLAUDE.md documents this as a variable users set manually before launching)

"JULIA_REPL_PROJECT": "${JULIA_REPL_PROJECT}"

  1. New timeout tests directly access AgentREPL.WORKER.worker_id for assertions, but the docstring on WORKER in src/types.jl says "Access via ensure_worker!() rather than directly."

# Worker should have been killed
@test AgentREPL.WORKER.worker_id === nothing
end
@testset "Next eval after timeout spawns fresh worker" begin
# After the timeout above killed the worker, a new eval should work
value_str, output, error_str, _ = AgentREPL.capture_eval_on_worker("42")
@test error_str === nothing
@test value_str == "42"
@test AgentREPL.WORKER.worker_id !== nothing
end

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

samtalki and others added 4 commits February 26, 2026 22:03
…e_tool removal, docs

- Fix Unicode crash in truncate_output by using first()/last() instead of byte indexing
- Fix channel race condition in timeout by increasing Channel capacity to 2
- Fix misleading length=1 for scalars in info tool by guarding on !isempty(size())
- Remove deprecated mode_tool registration from server
- Fix JULIA_REPL_PROJECT empty string crash in entry point
- Update CLAUDE.md: key functions, tool count, and WORKER docstring

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update marketplace.json versions from 0.3.0 to 0.5.0 to match
plugin.json and Project.toml. Fix HIGHLIGHT_CONFIG being cached at
precompile time by moving env var reads into __init__(), ensuring
JULIA_REPL_OUTPUT_FORMAT and JULIA_REPL_HIGHLIGHT are read at runtime.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mirror the styling from logging.jl into format_result: bold green
julia> prompt, dim timing, bold red errors with dim stacktraces,
dim → prefix when both stdout and return value exist, and styled
truncation markers. All conditional on highlighting enabled + ANSI format.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Close the result channel after take! so the losing async task
(timer or eval) gets an InvalidStateException on put! and exits
promptly instead of leaking.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant