-
Notifications
You must be signed in to change notification settings - Fork 153
feat: add Fireworks AI plugin for usage tracking and billing metrics #359
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -31,6 +31,7 @@ OpenUsage lives in your menu bar and shows you how much of your AI coding subscr | |||||||||
| - [**Copilot**](docs/providers/copilot.md) / premium, chat, completions | ||||||||||
| - [**Cursor**](docs/providers/cursor.md) / credits, total usage, auto usage, API usage, on-demand, CLI auth | ||||||||||
| - [**Factory / Droid**](docs/providers/factory.md) / standard, premium tokens | ||||||||||
| - [**Fireworks AI**](docs/providers/fireworks-ai.md) / serverless usage, prompt/generated TPS, monthly spend | ||||||||||
|
||||||||||
| - [**Fireworks AI**](docs/providers/fireworks-ai.md) / serverless usage, prompt/generated TPS, monthly spend | |
| - [**Fireworks AI**](docs/providers/fireworks-ai.md) / serverless usage, prompt/generated tokens, monthly spend |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2: README documents Fireworks metrics as TPS, but the plugin and provider docs expose prompt/generated token totals, creating a user-facing contract mismatch.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At README.md, line 34:
<comment>README documents Fireworks metrics as TPS, but the plugin and provider docs expose prompt/generated token totals, creating a user-facing contract mismatch.</comment>
<file context>
@@ -31,6 +31,7 @@ OpenUsage lives in your menu bar and shows you how much of your AI coding subscr
- [**Copilot**](docs/providers/copilot.md) / premium, chat, completions
- [**Cursor**](docs/providers/cursor.md) / credits, total usage, auto usage, API usage, on-demand, CLI auth
- [**Factory / Droid**](docs/providers/factory.md) / standard, premium tokens
+- [**Fireworks AI**](docs/providers/fireworks-ai.md) / serverless usage, prompt/generated TPS, monthly spend
- [**Gemini**](docs/providers/gemini.md) / pro, flash, workspace/free/paid tier
- [**JetBrains AI Assistant**](docs/providers/jetbrains-ai-assistant.md) / quota, remaining
</file context>
| - [**Fireworks AI**](docs/providers/fireworks-ai.md) / serverless usage, prompt/generated TPS, monthly spend | |
| - [**Fireworks AI**](docs/providers/fireworks-ai.md) / serverless usage, prompt/generated tokens, monthly spend |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| # Fireworks AI | ||
|
|
||
| > Based on Fireworks AI's documented control-plane API and quota model. | ||
|
|
||
| ## Overview | ||
|
|
||
| - **Product:** [Fireworks AI](https://fireworks.ai/) | ||
| - **Auth:** Fireworks API key | ||
| - **Primary API:** `https://api.fireworks.ai/v1` | ||
| - **Key setup:** macOS Keychain first, `FIREWORKS_API_KEY` fallback | ||
|
|
||
| OpenUsage uses Fireworks' official account + quota endpoints plus Fireworks' official billing-export path when available. It does not depend on a browser login. | ||
|
|
||
| ## Plugin Metrics | ||
|
|
||
| | Metric | Source | Scope | Format | Notes | | ||
| | --- | --- | --- | --- | --- | | ||
| | Serverless usage | billing export or aggregate token counter | overview | text | Main cumulative usage line, shown as a compact token total like `104.85M tokens` | | ||
| | Prompt tokens | billing export or token counter | overview | text | Prompt/input token total for the selected rolling window | | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. docs/providers/fireworks-ai.md:19 — The docs say Prompt/Generated tokens come from “billing export or token counter”, but the implementation only fills these from billing export (quota fallback is only used for an aggregate total). This may mislead users when Severity: medium 🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage. |
||
| | Generated tokens | billing export or token counter | overview | text | Generated/output token total for the selected rolling window | | ||
| | Month spend | `monthly-spend-usd.usage` | overview | text | Current calendar-month billable spend | | ||
| | Budget | `monthly-spend-usd.value` / `maxValue` | detail | text | Configured monthly budget plus the tier cap | | ||
| | Status | account `state` / `suspendState` | detail | badge | Only shown when the account is not in a healthy state | | ||
|
Comment on lines
+16
to
+23
|
||
|
|
||
| The plan label is inferred from Fireworks' documented monthly budget cap tiers: | ||
|
|
||
| - Tier 1: `$50` | ||
| - Tier 2: `$500` | ||
| - Tier 3: `$5,000` | ||
| - Tier 4: `$50,000` | ||
|
|
||
| ## API Calls | ||
|
|
||
| ### 1) List accounts | ||
|
|
||
| ```http | ||
| GET https://api.fireworks.ai/v1/accounts?pageSize=200 | ||
| Authorization: Bearer <API_KEY> | ||
| Accept: application/json | ||
| ``` | ||
|
|
||
| This returns the accounts attached to the key. OpenUsage picks the first healthy account (`READY` + `UNSUSPENDED`) and falls back to the first returned account if none are healthy. | ||
|
|
||
| ### 2) List quotas | ||
|
|
||
| ```http | ||
| GET https://api.fireworks.ai/v1/accounts/{account_id}/quotas?pageSize=200 | ||
| Authorization: Bearer <API_KEY> | ||
| Accept: application/json | ||
| ``` | ||
|
|
||
| OpenUsage currently reads: | ||
|
|
||
| - `monthly-spend-usd` | ||
| - aggregate token counters when the quota payload exposes them | ||
| - any other live quota only when it is a truthful user-facing limit worth showing | ||
|
|
||
| ### 3) Export billing metrics | ||
|
|
||
| Fireworks documents `firectl billing export-metrics` as the official way to export billable usage, and the command accepts `--api-key` and `--account-id`. OpenUsage uses that path when available to compute rolling token totals without relying on a browser session. The working export format is date-only `YYYY-MM-DD` windows. | ||
|
|
||
| Observed/expected quota fields: | ||
|
|
||
| - `name` | ||
| - `value` | ||
| - `maxValue` | ||
| - `usage` | ||
| - `currentUsage` | ||
| - `updateTime` | ||
|
|
||
| ## Credential Setup | ||
|
|
||
| ### Recommended: macOS Keychain | ||
|
|
||
| OpenUsage looks for this service name first: | ||
|
|
||
| ```text | ||
| OpenUsage Fireworks AI API Key | ||
| ``` | ||
|
|
||
| Add/update it with: | ||
|
|
||
| ```bash | ||
| security add-generic-password -U -a "$(id -un)" -s "OpenUsage Fireworks AI API Key" -w "<your-fireworks-api-key>" | ||
| ``` | ||
|
|
||
| ### Fallback: environment variable | ||
|
|
||
| ```bash | ||
| export FIREWORKS_API_KEY="<your-fireworks-api-key>" | ||
| ``` | ||
|
|
||
| Restart OpenUsage after changing shell env. The host caches env values for the app session. | ||
|
|
||
| ## Notes | ||
|
|
||
| - Fireworks' official docs document aggregate usage export through `firectl billing export-metrics`, but not a public documented browser-free history endpoint for the dashboard chart itself. OpenUsage therefore prefers the official billing-export path for cumulative token totals, falls back to live account token counters when present, and otherwise falls back to spend/budget only. | ||
| - Live validation on this account showed that Fireworks exposes monthly spend and request-rate quotas, but not prompt/generated token-per-second quotas. OpenUsage therefore does not ship speculative prompt/generated rate bars for Fireworks. | ||
| - Fireworks docs describe monthly spend limits and tier caps. OpenUsage labels the live spend number as `Month spend` to avoid implying a settled invoice total or a rolling 30-day window. | ||
| - If no key is configured, OpenUsage shows a direct setup hint instead of a generic failure. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
README.md:34 — The provider blurb says
prompt/generated TPS, but the plugin surfaces prompt/generated token totals (and doesn’t expose per-second throughput), so this looks user-facing inaccurate.Severity: low
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.