Pipelex Roadmap

[lchoquel]

Last updated: 2025-11-24
This document outlines the direction and priorities for this project.
It is a living document and will be updated as plans evolve.

💡 Vision & design principles

Pipelex is a declarative language and Python runtime for AI workflows. It lets you define repeatable, multi-step AI transformations as simple, readable files made of modular “pipes” with structured inputs and outputs, so your workflows read like documentation instead of plumbing code.

We believe AI workflows should be:

• Declarative & conceptual – you describe what you want and the concepts involved, in natural language where needed, not low-level APIs.
• Repeatable, portable & open – workflows are just files that run anywhere (CLI, Python, FastAPI, MCP, n8n, …) without lock-in.
• Understandable by humans & agents – syntax carries semantic context so LLMs can generate/update workflows, while people can review them like specs.
• Modular & extensible – with adapters for plugins and dependency injections where it matters.
• Transparent & observable – logs, traces, and metadata make it easy to see what happened, and how to fix or improve, if needed.
• Not an agent framework – agents explore and improvise; Pipelex is where you write down the stable workflow so anyone (or any agent) can run it again.

🎯 Main tracks

• Easy to use
  → it just works, with minimal friction
• Robust workflow executions
  → if something fails, you get first-class traces to analyze and you can resume without restarting
• Fully leverage the conceptual aspects
  → validate concepts and assess operations automatically at each pipe input / output
• Connectivity
  Pipelex can already be used via Python, API, MCP, n8n and soon via all automation platforms.
  → Should we design an API to support Pipelex connectors?
  → Or should Pipelex become an MCP Client (in addition to being an MCP Server)?
• Community
  → Enable pipes to call other pipes across the Web and fulfill our vision of the Know-How Graph

🔥 Burning issues

These are not really bugs, rather missing or unfinished features that have become like traps that we are all falling into.

• Implicit Concept be gone

  Currently, you can reference a concept in a pipe even if that concept is not defined → we implicitly create a text concept for it.

  • Problem: if it’s defined in a different bundle, you probably wanted to use it, right? In many cases, it will replace your carefully defined structured concept with a non-structured text concept, without you knowing, and that leads to bugs and confusion.
  • Solution: we must get rid of implicit concepts, plain and simple. All concepts you use must be defined, period.

• Static Code generation for concept Structures

  It’s pretty convenient to define a concept structure inside a bundle (.plx), and that’s also what our Pipe Builder does for you. What we do at runtime is programmatically generate python code for that structure (as a pydantic BaseModel) so the python objects can be instantiated.

  • Problem: unless you declare your structures using pydantic BaseModels yourself, your python code knows nothing about the generated concept structures. So you don’t have a class to import for the output type of your pipes…
  • Solution: generate the code statically into your codebase so that your python code can reference it.
  • Bonus: at the same time, we will be able to generate code for other languages such as typescript, go, or a plain json schema.

⏱️ Near-term roadmap

🕺 Features

This section is the most likely to change as we learn from users.

• Visual Flowchart generator
  We know. Although being language-first is a core principle of Pipelex, it can be hard to grasp a workflow just by skimming through the pipes. We must present a view of a workflow as a flowchart showing all its possible paths. We must also offer a view of an actual pipeline execution, showing the paths that were effectively taken and enabling you to inspect the intermediate inputs and outputs.

• PipeLLM

  • PDF inputs to PipeLLM
    Our current APIs for LLM calls support prompting with text and images.
    Need: several LLM providers have added support for PDF input, so we must support that too in PipeLLM.
    Issue: this overlaps with the core feature of PipeExtract which is to extract stuff from documents. But this feature is urgent so we’ll just do it like that and solve the overlap issue later on.
  • Test/Confirm support of other inference providers: Scaleway, Cerebras, SambaNova…
  • Integrate OpenAI responses API
  • Support setting reasoning effort

• Workflow reliability & tooling

  • Temporal.io
  • Delegate all content persistence to StorageProvider: keep only references in WorkingMemory, for efficiency and scalability
  • Practical support for long jobs (several hours): we need a practical solution for notifying the client when we’re done or if a problem occurred

• Language improvements

  • Support defining a local model deck (aliases, presets and waterfalls) in a bundle.plx

• Add capabilities

  • HuggingFace: various model capabilities via huggingface_hub.InferenceClient
  • Implement GoogleImgGenWorker to generate images with Gemini models such as Nano Banana
  • Web search / scrape: Linkup, Gemini web search, Perplexity, Tavily, Parallel.ai, Firecrawl…
  • File search: ZeroEntropy, Gemini File search
  • Document extraction: Numind, markitdown, Docling, DeepSeek-OCR, ERNIE-4.5-VL, Nvidia Nemotron-Nano-VL/Parse v1.1…
  • Misc: document anonymization with Hexagone AI

• Add modalities

  • Voice: pyannoteAl, Whisper, ElevenLabs…
  • Video: via Gemini, Veo, Sora and other models dealing with video input/output

• Pipe Builder

  • Interpret and validate the user’s prompt
  • Run recursively to split tasks into smaller tasks to arbitrary depth

• IDE Extensions

  • Enforce bundle blueprint schema
  • Run our validation tools from the extension’s Language Server
  • Pinpoint language errors to the precise line & column
  • Reformat and complete code
  • Extension for IntelliJ in addition to the VS Code Extension

• Integrate with automation

  • make.com custom App
  • Zapier integration

👌 Examples and Howtos

We gather example workflows in our Cookbook is not a term reserved, but this section is about examples of deeper customization of Pipelex:

• Integrating an observability tool
• Integrating evals
• Integrating custom AI models: we have an example for custom LLM but we need to present it better and we need other kinds of models

🛠️ Refactoring & Chores

It’s hard to tell exactly when we’ll do these, but they must be done.

• Complete our test coverage, especially e2e tests for the CLIs
• Remove code related to streaming, it’s useless dead code in our context
• Rewrite PipeLLM to be elementary, revamp ou trick for structuring with preliminary_text
• Rethink the ContentGeneratorProtocol, this prehistoric artefact makes no more sense

👐 Commercial services

These services are essentially infrastructure and they will remain closed source. But they are totally optional and they come with free tiers and free credits:

• Robust AI model routing to all models with a single API key: current service isn't robust enough, and lacking models
• Hosted API for running workflows on-demand: coming soon
• Remote MCP Server: coming soon

🙌 What to contribute to?

If you’d like to help, please do! 🙏🏻

The area where we need the most help is adding capabilities and integrations. In fact, there are so many possibilities, and our small team must focus on the Burning issues, the Visual Flowchart Generator, adding Temporal, and the various Refactorings: we’re already halfway through these.

To give some good options, we will add good-first-issue and help-wanted tags to issues. In any case, if you want to work on such topics, please give us a heads-up by adding a comment here or on the issue itself or, even better, contacting us on Discord. It’s important so that we have a chance to discuss details and, most of all, avoid double work.

🗣️ Feedback & discussion

We treat this roadmap as a conversation with the community.

• 💡 Do you have any feedback or suggestion to prioritize?

  Please comment in this thread.

• 🐞 Found a bug or a missing feature?

  Open an Issue or submit an Idea.