Skip to content

[FEATURE] Include cache usage metrics in span #776

@CodingTomo

Description

@CodingTomo

Problem Statement

Hi,
A recent pull request (#531) added metrics to track the number of tokens written to and read from the cache. However, these new metrics are not being exported within the span at the end of the agent invocation.

Proposed Solution

In tracer.py, add gen_ai.usage.cache_read_input_tokens and gen_ai.usage.cache_write_input_tokens to the attributes dictionary.

attributes: Dict[str, AttributeValue] = {
            "gen_ai.usage.prompt_tokens": usage["inputTokens"],
            "gen_ai.usage.input_tokens": usage["inputTokens"],
            "gen_ai.usage.completion_tokens": usage["outputTokens"],
            "gen_ai.usage.cache_read_input_tokens": usage["cacheReadInputTokens"],
            "gen_ai.usage.cache_write_input_tokens": usage["cacheWriteInputTokens"],
            "gen_ai.usage.output_tokens": usage["outputTokens"],
            "gen_ai.usage.total_tokens": usage["totalTokens"]
}

Use Case

To monitor cache usage and correctly compute the costs.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions