add Agent high-level primitive (system prompt + tools + memory + loop policy)

[cchinchilla-dev]

Description

AgentLoom can compose LLM calls, tool steps, routers, and subworkflows into a DAG, but it has no first-class concept of an Agent: a unit that owns (system prompt + tool registry subset + memory + decision loop policy) and can be invoked as a single step. Building an agent today requires:

• A Conversation in state (add conversation history primitive with token-budget trimming and multi-agent threads #119).
• Tool calling on llm_call (add native tool/function calling with streaming and parallel-call support #116).
• A loop step (add bounded retry loops with feedback #38, open) wrapping the LLM call to iterate until no tool call.
• Manual wiring of system prompt, tool subset, max iterations, exit conditions per workflow.

This is mechanically possible but ergonomically painful, and it leaks the agent's internals across the YAML. It also makes Agents non-portable: there is no way to publish "the customer-support agent" as a reusable artifact and import it into another workflow.

The PhD's Simulator depends on multi-agent orchestration (single-agent-vs-user, cooperation, negotiation, A2A — see agenttest-planteamiento.md). Each "agent" in those scenarios is a coherent unit that needs to be defined once and reused across many simulation runs with different counterparties. Without an Agent primitive, the Simulator becomes a YAML factory.

Proposal

Introduce an Agent definition as a high-level construct that compiles down to existing primitives (llm_call with tools + conversation + bounded loop), and a step type that invokes it.

1. Agent definition (YAML):

# agents/customer_support.yaml
name: customer_support
description: "Resolves billing and account questions."
model: claude-sonnet-4-5
system_prompt: |
  You are a customer support agent for ACME Corp.
  Use tools to look up account info before answering.
tools:
  - lookup_account
  - search_knowledge_base
  - create_ticket
memory:
  type: conversation
  token_budget: 8000
  trim_policy: summarize_oldest
loop_policy:
  max_iterations: 10
  exit_condition: "no_tool_call"   # or expression
  on_max_iterations: error          # or "warn_and_return"
output_schema:
  type: pydantic
  model: examples.schemas.SupportReply
constraints:
  max_cost_usd: 0.50
  max_latency_ms: 30000
  forbidden_patterns: ["I'm just an AI", "As a language model"]

2. Agent step type:

- id: handle_customer
  type: agent
  agent: agents/customer_support.yaml      # path or inline
  input: state.user_message                 # user turn appended to conversation
  conversation: state.support_chat          # optional; auto-creates if absent
  output: state.support_chat                # the agent's reply (and updated conversation)

The step:

1. Loads the agent definition.
2. Loads or creates the Conversation.
3. Runs the internal tool-call loop (per add native tool/function calling with streaming and parallel-call support #116) with the agent's tools subset and bounded iterations.
4. Validates output against output_schema (per add structured output (JSON schema, response_format) across providers #117) if specified.
5. Enforces constraints — fails the step if violated.
6. Persists the updated conversation.

3. Tool-registry scoping:

The agent's tools: [lookup_account, ...] declares which tools it may call. The engine creates a scoped ToolRegistry view that only exposes those names — even if the parent registry has more tools, the agent can't reach them. Defense-in-depth against prompt injection making the agent call an unintended tool.

4. Programmatic API:

from agentloom import Agent, ProviderGateway

agent = Agent.from_yaml("agents/customer_support.yaml", gateway=gateway, tool_registry=registry)
result = await agent.invoke(
    user_message="My bill seems wrong",
    conversation=existing_conversation,  # optional
)
# result.reply: SupportReply  (typed per output_schema)
# result.conversation: Conversation  (with new turns)
# result.cost_usd: float
# result.iterations: int

5. Agent introspection:

agentloom agent inspect agents/customer_support.yaml
# Shows: tools available, max iterations, expected cost range,
#        output schema, constraints

6. Composition:

Agents can be nested via subworkflow: an agent's tool can itself be another agent, enabling supervisor / worker patterns:

tools:
  - name: delegate_to_specialist
    type: agent
    agent: agents/specialist.yaml

(Implementation: tool registry accepts Agent instances as tools whose parameters_schema is derived from the agent's input schema.)

7. Observability:

• Span agent:<name> wrapping the step, with attributes: agent.name, agent.iterations, agent.tool_calls_count, agent.tokens_total, agent.cost_usd, agent.output_valid.
• Per-iteration child span linking to the LLM call span.
• Counter agentloom_agent_invocations_total{name, status}.
• Histogram agentloom_agent_iterations{name}.

Scope

• src/agentloom/core/models.py — AgentDefinition, LoopPolicy, MemoryConfig, Constraints.
• src/agentloom/core/agent.py — Agent class with invoke() method.
• src/agentloom/steps/agent.py — new step type agent.
• src/agentloom/cli/agent.py — agentloom agent inspect command.
• src/agentloom/observability/observer.py — agent-level hooks.
• examples/ — at least one full agent definition + workflow that uses it.

Regression tests

• test_agent_loads_from_yaml
• test_agent_invoke_runs_tool_call_loop
• test_agent_max_iterations_enforced
• test_agent_tool_registry_scoping_blocks_other_tools
• test_agent_output_schema_validates_reply
• test_agent_constraint_max_cost_aborts_when_exceeded
• test_agent_forbidden_patterns_marks_step_failed
• test_agent_step_in_workflow_persists_conversation
• test_nested_agent_as_tool

Notes

• Depends on: add native tool/function calling with streaming and parallel-call support #116 (tool calling), add conversation history primitive with token-budget trimming and multi-agent threads #119 (conversation), add structured output (JSON schema, response_format) across providers #117 (structured output), add bounded retry loops with feedback #38 (loop step) for the looping foundation.
• Without these prereqs the Agent step has no underlying mechanics — list as blocker dependencies.
• Backward compat: nothing existing breaks; this is purely additive.
• Agents become reusable across workflows and across repos: a published agent definition + its tool dependencies is the canonical way to share LLM behavior.
• This is the cleanest answer to the Simulator's needs in agenttest-planteamiento.md: each simulated participant is an Agent; the Simulator orchestrates Agent-to-Agent message passing.