traceAI

Open-source (OSS) observability toolkit for AI workflows - production-grade, standardized, and framework-agnostic.

Table of Contents

• Introduction
• Features
• Supported Frameworks
• Quickstart
• Contributing
• Resources
• Connect With Us
• License

Introduction

traceAI is an open-source (OSS) project for standardized tracing of AI applications and frameworks.

It provides a set of conventions and plugins that complement OpenTelemetry, enabling instrumentation and monitoring of code executions across models, frameworks, and vendors. All traces and spans are mapped to a consistent set of standardized attributes, making observability simple and uniform.

The project currently consists of multiple Python packages (e.g., traceAI-openai, traceAI-langchain, traceAI-anthropic) that implement these conventions for popular AI SDKs and frameworks. Each package integrates seamlessly with OpenTelemetry, establishing a consistent observability pipeline across projects.

While traceAI is natively supported by Future AGI, its packages can be used with any OpenTelemetry-compatible backend. These packages help developers instrument AI workflows, monitor performance, and maintain observability across different frameworks and vendors with minimal effort.

Features

Feature	Description
OpenTelemetry Integration	Native OpenTelemetry support enables standardized, end-to-end tracing
Unified Trace Format	Consistent schema for evaluations and executions, independent of framework or model type
Framework-Agnostic	Works out of the box with LangChain, LlamaIndex, DSPy, Haystack, and custom pipelines
Standardized Metrics	Captures latency, accuracy, reliability, and cost metrics for structured benchmarking
Multi-Modal Tracing	Supports text, audio, and image models with shared context identifiers
Interoperable Design	Connects with observability platforms and visualization dashboards
Lightweight Implementation	Minimal dependencies for easy embedding in existing stacks
Extensible Schema	Supports custom span attributes and metadata for organization-specific workflows
Extensible Plugins	Easily add custom instrumentation for unsupported frameworks and adapters
Open & Developer-First	Fully open-source and community-driven, maintained with native support from Future AGI

Supported Frameworks

traceAI provides drop-in instrumentation for popular AI frameworks - enabling standardized tracing across models, frameworks, and vendors.

Major LLM Vendors

Standardized instrumentation for major large language model vendors - enabling consistent tracing across models.

Framework	Instrumentation Package	Python (PyPI)
OpenAI	traceAI-openai
Anthropic	traceAI-anthropic
MistralAI	traceAI-mistralai
Groq	traceAI-groq
AWS Bedrock	traceAI-bedrock

Google AI Services

Instrumentation for Google AI services, providing unified trace observability.

Framework	Instrumentation Package	Python (PyPI)
VertexAI	traceAI-vertexai
Google GenAI	traceAI-google-genai
Google ADK	traceAI-google-adk

Agent & RAG Frameworks

Trace instrumentation for agents and retrieval-augmented generation frameworks - supporting multi-modal pipelines.

Framework	Instrumentation Package	Python (PyPI)
LlamaIndex	traceAI-llamaindex
LangChain	traceAI-langchain
Autogen	traceAI-autogen
CrewAI	traceAI-crewai
Haystack	traceAI-haystack
DSPy	traceAI-dspy
Guardrails	traceAI-guardrails
OpenAI Agents	traceAI-openai-agents
SmolAgents	traceAI-smolagents

Utilities & Plugins

Supplementary instrumentation and utility packages for workflow integration and plugin support.

Framework	Instrumentation Package	Python (PyPI)
LiteLLM	traceAI-litellm
Instructor	traceAI-instructor
PortKey	traceAI-portkey
MCP	traceAI-mcp

🟦 JavaScript / TypeScript Packages (npm)

Standardized instrumentation for AI frameworks - built for Node.js and TypeScript developers.

⚙️ More traceAI integrations for JS/TS frameworks are actively being developed - stay tuned.

Framework	Instrumentation Package	npm
OpenAI	@traceai/openai
Anthropic	@traceai/anthropic
AWS Bedrock	@traceai/bedrock
LangChain	@traceai/langchain
LlamaIndex	@traceai/llamaindex
MCP	@traceai/mcp

Quickstart

OpenAI Implementation

1. Install traceAI OpenAI

pip install traceAI-openai

2. Set Environment Variables

Set up your environment variables to authenticate with Future AGI and OpenAI. This must be done before running any instrumented code.

import os

os.environ["FI_API_KEY"] = FI_API_KEY
os.environ["FI_SECRET_KEY"] = FI_SECRET_KEY
os.environ["OPENAI_API_KEY"] = OPENAI_API_KEY

3. Register Tracer Provider

Register a trace provider to establish the observability pipeline for your application. All instrumented calls will report traces via this provider.

from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType

trace_provider = register(
    project_type=ProjectType.OBSERVE,
    project_name="openai_app"
)

4. Configure OpenAI Instrumentation

Enable traceAI instrumentation for OpenAI models (text, image, audio). This wraps the OpenAI client so that all API calls are automatically traced.

from traceai_openai import OpenAIInstrumentor

OpenAIInstrumentor().instrument(tracer_provider=trace_provider)

5. Create OpenAI Components

Run your OpenAI client as usual. This example demonstrates that instrumentation is active and traces will be generated for the request.

import openai

openai.api_key = os.environ["OPENAI_API_KEY"]

response = openai.ChatCompletion.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Can you tell me a joke?"}
    ]
)

print(response.choices[0].message['content'].strip())

Contributing

We welcome contributions from the community! Follow these steps to contribute:

1. Fork the repository

Create your own copy of the repository on GitHub by clicking the Fork button.

2. Sync with Upstream (Optional but Recommended)

Before starting work, ensure your local copy is up-to-date with the main repository:

git fetch upstream
git checkout main
git merge upstream/main

3. Create a branch

Create a new branch for your feature or bug fix:

git checkout -b feature/your-feature-name

4. Make Changes and Commit

Make your changes locally. Stage and commit them with a descriptive message:

git add .
git commit -m "Add feature/bug description"

5. Push Your Branch

Push your branch to your fork on GitHub:

git push origin feature/your-feature-name

6. Open a Pull Request

Go to your fork on GitHub and open a Pull Request against the upstream repository’s main branch. Include a clear description of your changes, reference any related issues, and follow the project’s contribution guidelines.

Resources

• Official Website: https://www.futureagi.com
• Documentation: https://docs.futureagi.com
• Integrations: https://docs.futureagi.com/future-agi/integrations/overview
• Cookbooks: How-To Implement Observability

⚡ Tip: Explore the Integrations for ready-to-use recipes for instrumenting AI pipelines across frameworks and modalities.

Connect With Us

Stay updated and connect with the Future AGI community:

• LinkedIn: https://www.linkedin.com/company/futureagi
• Twitter/X: https://x.com/FutureAGI_
• Reddit: https://www.reddit.com/user/Future_AGI/submitted/
• Substack: https://substack.com/@futureagi

Developed and maintained by Future AGI.

License

Licensed under the Apache License, Version 2.0.

---

⭐ If you find traceAI useful, please consider giving it a star on GitHub!