Skip to content

[Radar] Track upstream session-log fidelity gaps for cost and tool failures #97

@luoyuctl

Description

@luoyuctl

Background

External agent ecosystems keep surfacing a shared reliability gap: local session logs are useful, but some tools either omit cost details, undercount usage, or report tool failures without enough error context. agenttrace should track this as a radar item before deciding whether parser warnings, docs notes, or fixture work are needed.

Evidence

User value

Users reading local reports need to know when a metric is derived from complete local evidence versus when the upstream session log may be missing cost, failure, or final-output details.

Adoption rationale

Clear confidence boundaries improve Developer experience and Reliability value. They help users trust agenttrace reports while avoiding false precision when an upstream tool did not persist enough evidence.

Suggested scope

  • Keep this as radar until at least one minimal public fixture or reproducible local sample is available.
  • Decide whether agenttrace should add parser-level confidence notes for known upstream log gaps.
  • Decide whether docs should mention source-specific limitations for cost and tool-failure attribution.
  • If fixture evidence becomes available, split concrete parser or product issues by source tool.

Non-goals

  • Do not infer private billing data that is not present in local logs.
  • Do not upload or request user transcripts.
  • Do not change parser behavior without fixture-backed evidence.
  • Do not treat unrelated model-routing or hosted observability products as direct requirements.

Acceptance criteria

  • Maintainer decides whether this remains radar, becomes docs guidance, or splits into parser/product issues.
  • Any follow-up issue names the source tool and the specific missing or unreliable field.
  • Follow-up acceptance criteria require local fixture evidence or a reproducible public sample.
  • agenttrace public copy avoids overclaiming exact cost attribution where the upstream log is known to be incomplete.

Suggested lane

lane/radar, priority/P2, status/needs-human

Risk

Medium. Overreacting could add noisy warnings; ignoring the signal could make reports look more precise than the underlying logs support.

Source

source/radar: Tavily scan of public GitHub and ecosystem signals on 2026-05-04.

Metadata

Metadata

Assignees

No one assigned

    Labels

    lane/radarResearch and routing from ecosystem radarpriority/P2Useful follow-up worksource/radarCreated or updated by ecosystem radarstatus/needs-humanNeeds maintainer/product decision

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions