Skip to content

Conversation

@andrei-rusu
Copy link
Collaborator

@andrei-rusu andrei-rusu commented Oct 15, 2025

UiPath Agents Gym Sample

A comprehensive reference implementation of the Agents Gym loop and agents compatible with the SDK.

What's Included

Two Agents:

  • Calculator Agent - Uses add and multiply tools to calculate an expression. Escalates for missing numbers.
  • Loan Agent - Loan request processing with mocked escalations.

Dual-Mode Architecture:

  • CLI Mode - Supports execution via uipath run, uipath dev and uipath eval
  • Evaluation Mode - Custom runtime for batch execution of evaluations

Developer Experience:

  • Convenience Scripts - run.sh, eval.sh, export_evals.sh for common operations
  • Debugging - The custom runtime facilitates seamless debugging for evaluators

Key Technical Patterns

  • Separate Graph Builders - build_cli_graph() for runtime input, build_evaluation_graph() for custom runtime batch execution of evaluations
  • Evaluation Export - Convert Python evaluators/datapoints to UiPath eval JSON format

Quick Start

cd samples/gym-sample
uv sync                              # Install package
uv run uipath auth                   # Authenticate
./run.sh calculator                  # Run with `uipath run` CLI using a test input
uipath dev                           # Run with `uipath dev` CLI to inspect the traces and other execution metadata
./export_evals.sh                    # Export datapoints and evaluators to JSON
./eval.sh calculator                 # Run with `uipath eval` CLI to execute + evaluate the datapoints
python -m gym_sample.run --agent_name $AGENT_NAME   # Run the custom evaluations runtime

@andrei-rusu andrei-rusu changed the title feat(samples): Add UiPath gym sample with custom runtime but compatible with the SDK CLI commands feat(samples): Add UiPath gym sample with custom runtime Oct 15, 2025
@andrei-rusu andrei-rusu changed the title feat(samples): Add UiPath gym sample with custom runtime feat(samples): Add Agent Gym sample and custom runtime Oct 15, 2025
@andrei-rusu andrei-rusu force-pushed the dev/andreiru/agents-gym-sample branch from e3869c9 to 86cd255 Compare October 15, 2025 14:31
@andrei-rusu andrei-rusu force-pushed the dev/andreiru/agents-gym-sample branch from e224f5c to 5021ee8 Compare October 17, 2025 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants