Skip to content

fix: harden scenario/optimizer tools for AI agent reliability#3

Merged
nialloulton merged 1 commit intomainfrom
fix/mcp-scenario-optimizer-hardening
Apr 6, 2026
Merged

fix: harden scenario/optimizer tools for AI agent reliability#3
nialloulton merged 1 commit intomainfrom
fix/mcp-scenario-optimizer-hardening

Conversation

@nialloulton
Copy link
Copy Markdown
Contributor

@nialloulton nialloulton commented Apr 6, 2026

Summary

  • Default rebuild_model=True in run_scenario — the model graph is never available after S3 load, so prediction always requires rebuilding
  • Improve run_optimizer docstring: document channel naming requirements, bounds format (percentages), CPM discovery from template
  • Improve get_scenario_template docstring: emphasize it as a required pre-step for both optimizer and scenario tools
  • Clarify run_scenario polling requirement (must call get_scenario_results after)

Context

An AI agent test revealed that the core upload-fit-results workflow works end-to-end, but optimizer and scenario tools fail due to:

  1. Missing model graph (scenario runs with rebuild_model=False, but PyMC model is not loaded from S3)
  2. Insufficient guidance in tool docstrings for the AI to construct valid optimizer payloads

Test plan

  • Verify run_scenario now sends rebuild_model: true by default
  • Verify run_optimizer docstring guides AI to use correct channel names from template
  • End-to-end test: upload -> fit -> scenario -> optimizer via MCP

Made with Cursor


Note

Medium Risk
Behavior changes by defaulting run_scenario to rebuild the model graph, which can impact performance and could alter scenario execution in existing clients. Other changes are docstring-only but affect how agents/clients construct optimizer/scenario payloads.

Overview
Hardens MCP scenario/optimizer tools for agent reliability. run_scenario now defaults rebuild_model=True and only includes the rebuild_model flag in the API payload when enabled.

Documentation for run_optimizer and get_scenario_template is expanded to make get_scenario_template the required first step and to spell out exact channel-key matching requirements, bounds-as-percentages format, period array length constraints, and CPM discovery; run_scenario docs now explicitly require polling get_scenario_results and clarify expected scenario_data/spend_metadata shapes.

Reviewed by Cursor Bugbot for commit 380fde4. Bugbot is set up for automated code reviews on this repo. Configure here.

@nialloulton nialloulton merged commit 2d46591 into main Apr 6, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants