Updating agent benchmarking to latest

.env.example

@@ -3,4 +3,4 @@ MODELS_RUN_URL=https://models.aixplain.com/api/v1/execute
 PIPELINE_API_KEY=<API_KEY>
 MODEL_API_KEY=<API_KEY>
 LOG_LEVEL=DEBUG
-TEAM_API_KEY=<YOUR_API_KEY>
+TEAM_API_KEY=<YOUR_API_KEY>

.github/workflows/docs.yaml

@@ -31,7 +31,7 @@ jobs:
       - name: Run pydoc-markdown
         run: |
           pydoc-markdown pydoc-markdown.yml
-          
+
       - name: Create Pull Request if docs changed
         uses: peter-evans/create-pull-request@v7
         with:

.github/workflows/pre-commit.yaml

@@ -0,0 +1,32 @@
+name: Pre-commit Checks
+
+on:
+  push:
+    branches:
+      - '**'
+
+jobs:
+  pre-commit:
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.9"
+          cache: 'pip'
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install ".[test]"
+          pip install pre-commit coverage
+
+      - name: Run pre-commit hooks
+        env:
+          TEAM_API_KEY: ${{ secrets.TEAM_API_KEY }}
+        run: pre-commit run --all-files

.gitignore

@@ -30,7 +30,10 @@ share/python-wheels/
 *.egg
 MANIFEST
 notebooks/
-
+scripts/
+.claude/
+.cursor/
+.cursorrules
 # PyInstaller
 #  Usually these files are written by a python script from a template
 #  before PyInstaller builds the exe, so as to inject date/other infos into it.

.pre-commit-config.yaml

@@ -3,7 +3,9 @@ repos:
     rev: v5.0.0 # Use the latest version
     hooks:
       - id: trailing-whitespace
+        exclude: ^docs/
       - id: end-of-file-fixer
+        exclude: ^docs/
       - id: check-merge-conflict
       - id: check-added-large-files
 
@@ -12,7 +14,9 @@ repos:
     hooks:
       - id: ruff
         args: [--fix]
+        files: ^aixplain/v2/
       - id: ruff-format
+        files: ^aixplain/v2/
 
   - repo: local
     hooks:

README.md

@@ -1,37 +1,75 @@
-# aiXplain Agents SDK
-
-**Build and deploy autonomous AI agents on production-grade infrastructure, instantly.**
-
----
-
-## aiXplain agents
-
-aiXplain Agents SDK gives developers Python and REST APIs to build, run, and deploy autonomous multi-step agents on [AgenticOS](https://docs.aixplain.com/getting-started/agenticos). Agents include built-in memory for short- and long-term context (opt-in), and adapt at runtime by planning steps, selecting tools and models, running code, and refining outputs until tasks are complete.
-
-aiXplain agents include micro-agents for runtime policy enforcement and access control, plus proprietary meta-agents like Evolver for self-improvement.
+<p align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="docs/assets/aixplain-logo-light.png">
+    <source media="(prefers-color-scheme: light)" srcset="docs/assets/aixplain-logo-dark.png">
+    <img src="docs/assets/aixplain-logo-dark.png" alt="aiXplain" width="520">
+  </picture>
+</p>
+
+<h1 align="center">aiXplain SDK</h1>
+
+<p align="center">
+  <a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-2ea44f?style=flat-square" alt="License"></a>
+  <a href="https://studio.aixplain.com/browse"><img src="https://img.shields.io/badge/Marketplace-900%2B%20models%20%26%20tools-0b74de?style=flat-square" alt="Marketplace size"></a>
+  <a href="https://console.aixplain.com/settings/keys"><img src="https://img.shields.io/badge/%F0%9F%94%91%20PAYG%20API%20key-Console-0b74de?style=flat-square" alt="PAYG API key"></a>
+  <a href="https://discord.gg/aixplain"><img src="https://img.shields.io/badge/Discord-Join-5865F2?style=flat-square&logo=discord&logoColor=white" alt="Discord"></a>
+</p>
+
+**Build, deploy, and govern autonomous AI agents for your business operations.**
+
+aiXplain SDK provides Python and REST APIs for agents that plan, use tools, call models and data, run code, and adapt at runtime. It also works natively with MCP-compatible coding agents and IDEs.
+
+> **Become an agentic-first organization**
+>
+> Designed for business operations: autonomous, governed, MCP-compatible, and built for context management. Your interactive AI assistant is [a click away](https://auth.aixplain.com/).
+>
+> _We operate our business with aiXplain agents, using them across product, business development, and marketing._
+
+## Why aiXplain
+
+- **Autonomous runtime loop** — plan, call tools and models, reflect, and continue without fixed flowcharts.
+- **Multi-agent execution** — delegate work to specialized subagents at runtime.
+- **Governance by default** — runtime access and policy enforcement on every run.
+- **Production observability** — inspect step-level traces, tool calls, and outcomes for debugging.
+- **Model and tool portability** — swap assets without rewriting application glue code.
+- **MCP-native access** — connect MCP clients to [900+ aiXplain-hosted assets](#mcp-servers) with one PAYG API key.
+- **Flexible deployment** — run the same agent definition serverless or private.
 
-With one API key, access 900+ vendor-agnostic models, tools, and integrations in the aiXplain Marketplace with consolidated billing, and swap assets without rewriting pipelines.
+| | aiXplain SDK | Other agent frameworks |
+|---|---|---|
+| Governance | Runtime access and policy enforcement built in | Usually custom code or external guardrails |
+| Models and tools | 900+ models and tools with one API key | Provider-by-provider setup |
+| Deployment | Cloud (instant) or on-prem | Usually self-assembled runtime and infra |
+| Observability | Built-in traces and dashboards | Varies by framework |
+| Coding-agent workflows | Works natively with MCP-compatible coding agents and IDEs | Usually not a first-class workflow target |
 
-### Why aiXplain for developers
+## AgenticOS
 
-- **Autonomy** — agents plan and adapt at runtime instead of following fixed workflows.
-- **Delegation** — route complex work to specialized subagents during execution.
-- **Policy enforcement** — apply runtime guardrails with Inspector and Bodyguard on every run.
-- **Observability** — inspect step-level traces, tool calls, and outcomes for debugging.
-- **Portability** — swap models and tools without rewriting application logic.
-- **Flexible deployment** — run the same agent definition serverless or private.
+AgenticOS is the portable runtime platform behind aiXplain agents. AgentEngine orchestrates planning, execution, and delegation for autonomous agents. AssetServing connects agents to models, tools, and data through a governed runtime layer. Observability captures traces, metrics, and monitoring for every production run across Cloud (instant) and on-prem deployments.
 
 <div align="center">
-  <img src="docs/assets/aixplain-workflow-teamagent.png" alt="aiXplain team-agent runtime flow" title="aiXplain"/>
+  <img src="docs/assets/aixplain-agentic-os-architecture.svg" alt="aiXplain AgenticOS architecture" title="aiXplain"/>
 </div>
 
-## AgenticOS
+---
 
-AgenticOS is the runtime behind aiXplain Agents. It orchestrates multi-step execution, routes model and tool calls with fallback policies, enforces governance at runtime, records step-level traces, and supports both serverless and private deployment.
+## MCP Server Marketplace
 
-<div align="center">
-  <img src="docs/assets/aixplain-agentic-os-architecture.png" alt="aiXplain AgenticOS architecture" title="aiXplain"/>
-</div>
+[aiXplain Marketplace](https://studio.aixplain.com/browse) now also exposes MCP servers for **900+ models and tools**, allowing external clients to access selected **tool, integration, and model assets**, for example **Opus 4.6, Kimi, Qwen, Airtable, and Slack**, through **aiXplain-hosted MCP endpoints** with a single API key 🔑.
+
+Read the full MCP setup guide in the [MCP servers docs](https://docs.aixplain.com/api-reference/mcp-servers).
+
+```json
+{
+  "ms1": {
+    "url": "https://models-mcp.aixplain.com/mcp/<AIXPLAIN_ASSET_ID>",
+    "headers": {
+      "Authorization": "Bearer <AIXPLAIN_APIKEY>",
+      "Accept": "application/json, text/event-stream"
+    }
+  }
+}
+```
 
 ---
 
@@ -49,54 +87,98 @@ Get your API key from your [aiXplain account](https://console.aixplain.com/setti
 ### Create and run your first agent (v2)
 
 ```python
+from uuid import uuid4
 from aixplain import Aixplain
 
 aix = Aixplain(api_key="<AIXPLAIN_API_KEY>")
 
 search_tool = aix.Tool.get("tavily/tavily-web-search/tavily")
+search_tool.allowed_actions = ["search"]
 
 agent = aix.Agent(
-    name="Research agent",
+    name=f"Research agent {uuid4().hex[:8]}",
     description="Answers questions with concise web-grounded findings.",
     instructions="Use the search tool when needed and cite key findings.",
     tools=[search_tool],
 )
 agent.save()
 
-result = agent.run(query="Summarize the latest AgenticOS updates.")
+result = agent.run(
+    query="Who is the CEO of OpenAI? Answer in one sentence.",
+)
 print(result.data.output)
 ```
 
 ### Build a multi-agent team (v2)
 
 ```python
+from uuid import uuid4
 from aixplain import Aixplain
+from aixplain.v2 import EditorConfig, EvaluatorConfig, EvaluatorType, Inspector, InspectorAction, InspectorActionConfig, InspectorSeverity, InspectorTarget
 
 aix = Aixplain(api_key="<AIXPLAIN_API_KEY>")
 search_tool = aix.Tool.get("tavily/tavily-web-search/tavily")
-
-planner = aix.Agent(
-    name="Planner",
-    instructions="Break requests into clear subtasks."
+search_tool.allowed_actions = ["search"]
+
+def never_edit(text: str) -> bool:
+    return False
+
+def passthrough(text: str) -> str:
+    return text
+
+noop_inspector = Inspector(
+    name=f"noop-output-inspector-{uuid4().hex[:8]}",
+    severity=InspectorSeverity.LOW,
+    targets=[InspectorTarget.OUTPUT],
+    action=InspectorActionConfig(type=InspectorAction.EDIT),
+    evaluator=EvaluatorConfig(
+        type=EvaluatorType.FUNCTION,
+        function=never_edit,
+    ),
+    editor=EditorConfig(
+        type=EvaluatorType.FUNCTION,
+        function=passthrough,
+    ),
 )
 
 researcher = aix.Agent(
-    name="Researcher",
+    name=f"Researcher {uuid4().hex[:8]}",
     instructions="Find and summarize reliable sources.",
     tools=[search_tool],
 )
 
 team_agent = aix.Agent(
-    name="Research team",
-    instructions="Delegate work to subagents, then return one final answer.",
-    subagents=[planner, researcher],
+    name=f"Research team {uuid4().hex[:8]}",
+    instructions="Research the topic and return exactly 5 concise bullet points.",
+    subagents=[researcher],
+    inspectors=[noop_inspector],
 )
-team_agent.save()
+team_agent.save(save_subcomponents=True)
 
-response = team_agent.run(query="Compare top open-source agent frameworks in 5 bullets.")
+response = team_agent.run(
+    query="Compare OpenAI and Anthropic in exactly 5 concise bullet points.",
+)
 print(response.data.output)
 ```
 
+<div align="center">
+  <img src="docs/assets/aixplain-workflow-teamagent.png" alt="aiXplain team-agent runtime flow" title="aiXplain"/>
+</div>
+
+Execution order:
+
+```text
+Human prompt: "Compare OpenAI and Anthropic in exactly 5 concise bullet points."
+
+Team agent
+├── Planner: breaks the goal into research and synthesis steps
+├── Orchestrator: routes work to the right subagent
+├── Researcher subagent
+│   └── Tavily search tool: finds and summarizes reliable sources
+├── Inspector: checks the final output through a simple runtime policy
+├── Orchestrator: decides whether another pass is needed
+└── Responder: returns one final answer
+```
 </details>
 
 <details>
@@ -130,13 +212,14 @@ You can still access legacy docs at [docs.aixplain.com/1.0](https://docs.aixplai
 
 aiXplain applies runtime governance and enterprise controls by default:
 
+- **We do not train on your data** — your data is not used to train foundation models.
 - **No data retained by default** — agent memory is opt-in (short-term and long-term).
 - **SOC 2 Type II certified** — enterprise security and compliance posture.
 - **Runtime policy enforcement** — Inspector and Bodyguard govern every agent execution.
-- **Sovereign deployment options** — serverless or private (on-prem, VPC, and air-gapped).
+- **Portable deployment options** — Cloud (instant) or on-prem (including VPC and air-gapped environments).
 - **Encryption** — TLS 1.2+ in transit and encrypted storage at rest.
 
-Learn more at [aiXplain Security](https://aixplain.com/security/) and [Sovereignty](https://aixplain.com/sovereignty/).
+Learn more at aiXplain [Security](https://aixplain.com/security/) and aiXplain [pricing](https://aixplain.com/pricing/).
 
 ---
 
@@ -148,7 +231,7 @@ Start free, then scale with usage-based pricing.
 - **Subscription plans** — reduce effective consumption-based rates.
 - **Custom enterprise pricing** — available for advanced scale and deployment needs.
 
-Learn more at [aiXplain Pricing](https://aixplain.com/pricing/).
+Learn more at aiXplain [pricing](https://aixplain.com/pricing/).
 
 ---
 

aixplain/utils/convert_datatype_utils.py

@@ -48,16 +48,16 @@ def normalize_expected_output(obj):
             if hasattr(obj, "model_json_schema")
             else obj.schema()
         )
-        return json.dumps(schema)  
+        return json.dumps(schema)
 
     if isinstance(obj, BaseModel):
         return (
             obj.model_dump_json() if hasattr(obj, "model_dump_json") else obj.json()
-        )  
+        )
 
     if isinstance(obj, (dict, str)) or obj is None:
         return (
             obj if isinstance(obj, str) else json.dumps(obj) if obj is not None else obj
         )
 
-    return json.dumps(obj) 
+    return json.dumps(obj)

aixplain/utils/file_utils.py

@@ -94,6 +94,22 @@ def download_data(url_link: str, local_filename: Optional[str] = None) -> str:
     return local_filename
 
 
+def _build_s3_link_from_presigned_url(presigned_url: Text, path: Text) -> Text:
+    """Build an S3 URI from a presigned upload URL."""
+    bucket_match = re.findall(r"https://(.*?).s3.amazonaws.com", presigned_url)
+    if bucket_match:
+        return f"s3://{bucket_match[0]}/{path}"
+
+    parsed_url = urlparse(presigned_url)
+    host_parts = parsed_url.netloc.split(".")
+    if host_parts and host_parts[0] and not host_parts[0].startswith("s3"):
+        return f"s3://{host_parts[0]}/{path}"
+
+    path_parts = parsed_url.path.lstrip("/").split("/", 1)
+    bucket_name = path_parts[0] if path_parts and path_parts[0] else "aixplain-uploads"
+    return f"s3://{bucket_name}/{path}"
+
+
 def upload_data(
     file_name: Union[Text, Path],
     tags: Optional[List[Text]] = None,
@@ -187,9 +203,7 @@ def upload_data(
             else:
                 raise Exception("File Uploading Error: Failure on Uploading to S3.")
         if return_download_link is False:
-            bucket_name = re.findall(r"https://(.*?).s3.amazonaws.com", presigned_url)[0]
-            s3_link = f"s3://{bucket_name}/{path}"
-            return s3_link
+            return _build_s3_link_from_presigned_url(presigned_url, path)
         return download_link
     except Exception:
         if nattempts > 0:

aixplain/v1/factories/agent_factory/__init__.py

@@ -137,8 +137,8 @@ def create(
         if llm is None and llm_id is not None:
             llm = get_llm_instance(llm_id, api_key=api_key, use_cache=True)
         elif llm is None:
-            # Use default GPT-4o if no LLM specified
-            llm = get_llm_instance("6895d6d1d50c89537c1cf237", api_key=api_key, use_cache=True)
+            # Use the default GPT-5.4 model if no LLM specified
+            llm = get_llm_instance("69b7e5f1b2fe44704ab0e7d0", api_key=api_key, use_cache=True)
 
         if output_format == OutputFormat.JSON:
             assert expected_output is not None and (