Conversation
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
2 issues found across 15 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="apps/backend/src/services/transcribe.service.ts">
<violation number="1" location="apps/backend/src/services/transcribe.service.ts:85">
P2: Transcription cost calculation silently falls back to zero for missing duration or unknown model pricing, causing underreported voice usage costs.</violation>
</file>
<file name="apps/backend/src/queries/usage.queries.ts">
<violation number="1" location="apps/backend/src/queries/usage.queries.ts:70">
P2: New action-cost feature uses `llm_inference` provider filtering, but provider options are still sourced only from `chat_message`, so some valid providers may be impossible to select in the UI.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| durationInSeconds: number | undefined, | ||
| ): number { | ||
| if (!durationInSeconds) { | ||
| return 0; |
There was a problem hiding this comment.
P2: Transcription cost calculation silently falls back to zero for missing duration or unknown model pricing, causing underreported voice usage costs.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/backend/src/services/transcribe.service.ts, line 85:
<comment>Transcription cost calculation silently falls back to zero for missing duration or unknown model pricing, causing underreported voice usage costs.</comment>
<file context>
@@ -63,6 +76,19 @@ export async function getAvailableModels(projectId: string) {
+ durationInSeconds: number | undefined,
+): number {
+ if (!durationInSeconds) {
+ return 0;
+ }
+ const modelDef = TRANSCRIBE_PROVIDERS[provider].models.find((m) => m.id === modelId);
</file context>
| inputCacheReadCost: Number(row.inputCacheReadCost ?? 0), | ||
| inputCacheWriteCost: Number(row.inputCacheWriteCost ?? 0), | ||
| outputCost: Number(row.outputCost ?? 0), | ||
| const actionCostsByDate = await getActionCostsByDate(projectId, filter); |
There was a problem hiding this comment.
P2: New action-cost feature uses llm_inference provider filtering, but provider options are still sourced only from chat_message, so some valid providers may be impossible to select in the UI.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/backend/src/queries/usage.queries.ts, line 70:
<comment>New action-cost feature uses `llm_inference` provider filtering, but provider options are still sourced only from `chat_message`, so some valid providers may be impossible to select in the UI.</comment>
<file context>
@@ -71,30 +67,44 @@ export const getMessagesUsage = async (projectId: string, filter: UsageFilter):
- inputCacheReadCost: Number(row.inputCacheReadCost ?? 0),
- inputCacheWriteCost: Number(row.inputCacheWriteCost ?? 0),
- outputCost: Number(row.outputCost ?? 0),
+ const actionCostsByDate = await getActionCostsByDate(projectId, filter);
+
+ const messagesByDate = new Map(rows.map((row) => [row.date, row]));
</file context>
There was a problem hiding this comment.
Pull request overview
Implements issue #316 by extending LLM inference tracking and surfacing cost breakdowns by action type (chat/test/memory/voice) in the admin Usage & Costs panel, plus a small Stories preview navigation improvement.
Changes:
- Add
estimated_costtollm_inference(SQLite + Postgres) and record additional inference types (chat,test,voice) across agent/test/transcribe flows. - Extend usage aggregation to compute per-action daily/hourly/monthly costs from
llm_inferenceand merge them with existing message usage data (including action-only dates). - Add an “Actions” chart view on the frontend usage page and add a back button on the story preview page.
Reviewed changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| package-lock.json | Lockfile updates reflecting dependency metadata changes and added optional packages. |
| apps/frontend/src/routes/_sidebar-layout.stories.preview.$chatId.$storyId.tsx | Adds a back button in the story preview header linking back to the Stories list. |
| apps/frontend/src/routes/_sidebar-layout.settings.usage.tsx | Adds “Actions” chart view rendering cost series by action type. |
| apps/frontend/src/components/settings/usage-filters.tsx | Extends chart view selector with the new actions option. |
| apps/backend/src/utils/date.ts | Ensures fillMissingDates initializes new action-cost fields to 0. |
| apps/backend/src/types/usage.ts | Extends UsageRecord with chatCost, testCost, memoryCost, voiceCost. |
| apps/backend/src/types/llm.ts | Expands LLM_INFERENCE_TYPES to include chat, test, voice. |
| apps/backend/src/trpc/transcribe.routes.ts | Passes ctx.user.id into transcription to attribute usage to a user. |
| apps/backend/src/services/transcribe.service.ts | Tracks voice transcription in llm_inference using duration-based estimatedCost. |
| apps/backend/src/services/test-agent.service.ts | Adds LLM inference tracking for test runs/verifications and threads userId through. |
| apps/backend/src/services/agent.ts | Tracks chat streaming calls and title_generation in llm_inference. |
| apps/backend/src/routes/test.ts | Threads authenticated userId into test agent calls for attribution. |
| apps/backend/src/queries/usage.queries.ts | Adds getActionCostsByDate, merges action costs into usage output, and fixes action-only date dropping. |
| apps/backend/src/db/sqlite-schema.ts | Adds estimated_cost column to SQLite llm_inference. |
| apps/backend/src/db/pg-schema.ts | Adds estimated_cost column to Postgres llm_inference. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| updatedAt: Date.now(), | ||
| messages: [userMessage], | ||
| userId: 'test', | ||
| projectId, | ||
| }; |
There was a problem hiding this comment.
runTest now accepts a userId (and uses it for the inference record), but the temporary chat passed into this.create(...) still hardcodes userId: 'test'. This means the agent run will load memory/context for a non-existent "test" user and can diverge from the usage attribution stored in llm_inference. Consider setting tempChat.userId to the passed userId (or, if intentionally anonymous, keep it consistent by also attributing the inference record to the same user).
🚀 Preview Deployment
Preview will be automatically removed when this PR is closed. |
|
Hey, it seems the migrations files are missing. |
|
Hey is it possible to add a column to LLM inference table to capture token usage by file reads and tool use etc (e.g. split a chat message by its message parts)... I think as an admin, the total cost and token usage by file reads in particular would be useful information as it could guide me in making my context more efficient. |
Summary
Implements #316 — surfaces LLM cost breakdown by action type (chat, test, memory, voice) in the admin Usage & Costs panel.
Backend
estimated_costcolumn tollm_inferencetable (both SQLite and PostgreSQL) for voice transcription costs that use per-minute pricing instead of per-tokenllm_inferenceviascheduleSaveLlmInferenceRecord:chat— streaming responses inAgentManager.stream()title_generation— chat title generation inAgentManager._generateTitle()test— test runs and verifications inTestAgentServicevoice— audio transcription intranscribeAudio()with duration-based costgetActionCostsByDate()query aggregates costs fromllm_inferencegrouped by action type, merged with existing message usage data. Chat cost includeschat,compaction, andtitle_generationtypes. Voice cost uses the pre-computedestimated_costcolumn instead of token math.Frontend
UX sanding
/storiesTest plan
npm run db:pushto apply schema changesllm_inferencerecords are created withtype = 'chat'nao testand verifytype = 'test'records appeartype = 'voice'records withestimated_cost/storiesnpm run lintpasses