generated from amazon-archives/__template_Apache-2.0
-
Notifications
You must be signed in to change notification settings - Fork 428
Closed
Labels
area-otelOpen-telemetry relatedOpen-telemetry relatedenhancementNew feature or requestNew feature or requestready for contributionPull requests welcomePull requests welcome
Description
Problem Statement
Hi,
A recent pull request (#531) added metrics to track the number of tokens written to and read from the cache. However, these new metrics are not being exported within the span at the end of the agent invocation.
Proposed Solution
In tracer.py
, add gen_ai.usage.cache_read_input_tokens
and gen_ai.usage.cache_write_input_tokens
to the attributes
dictionary.
attributes: Dict[str, AttributeValue] = {
"gen_ai.usage.prompt_tokens": usage["inputTokens"],
"gen_ai.usage.input_tokens": usage["inputTokens"],
"gen_ai.usage.completion_tokens": usage["outputTokens"],
"gen_ai.usage.cache_read_input_tokens": usage["cacheReadInputTokens"],
"gen_ai.usage.cache_write_input_tokens": usage["cacheWriteInputTokens"],
"gen_ai.usage.output_tokens": usage["outputTokens"],
"gen_ai.usage.total_tokens": usage["totalTokens"]
}
Use Case
To monitor cache usage and correctly compute the costs.
alesanfra and labeveryday
Metadata
Metadata
Assignees
Labels
area-otelOpen-telemetry relatedOpen-telemetry relatedenhancementNew feature or requestNew feature or requestready for contributionPull requests welcomePull requests welcome