Problem
When running multi-step pipelines (especially overnight), there's no visibility into cost until after the run completes. The CLI runtime returns costUsd per step but the pipeline doesn't aggregate or enforce limits.
For the IMC crew (8 steps, Opus model), a single run costs ~$4-6. During iterative development (run crew → fix paper → run crew), costs add up quickly with no guardrails.
Suggested fix
-
Cost aggregation in trace: The pipeline trace already captures per-step cost. Add a totalCostUsd to the trace summary.
-
Budget limit: Add a maxBudgetUsd option to pipeline.run():
await crew.run(input, {
runtime,
maxBudgetUsd: 10.0 // abort if cumulative cost exceeds $10
});
-
Cost estimation in analyzer: The analyze command already estimates token counts. Extend it to estimate cost per model (using known per-token pricing).
-
Real-time cost logging: Log cumulative cost after each step completes:
[step 3/8] methodology-expert: $0.82 (cumulative: $2.41)
Labels
enhancement, observability
Problem
When running multi-step pipelines (especially overnight), there's no visibility into cost until after the run completes. The CLI runtime returns
costUsdper step but the pipeline doesn't aggregate or enforce limits.For the IMC crew (8 steps, Opus model), a single run costs ~$4-6. During iterative development (run crew → fix paper → run crew), costs add up quickly with no guardrails.
Suggested fix
Cost aggregation in trace: The pipeline trace already captures per-step cost. Add a
totalCostUsdto the trace summary.Budget limit: Add a
maxBudgetUsdoption topipeline.run():Cost estimation in analyzer: The
analyzecommand already estimates token counts. Extend it to estimate cost per model (using known per-token pricing).Real-time cost logging: Log cumulative cost after each step completes:
Labels
enhancement, observability