Skip to content

Add stream_options for token usage tracking in streaming responses #70

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

seuros
Copy link

@seuros seuros commented Aug 15, 2025

Summary

  • Adds stream_options: { include_usage: true } to the streaming request payload
  • Enables token usage tracking for streaming responses per OpenAI API standard
  • X.AI (Grok) supports this parameter and returns usage data in the final stream chunk

The Dungeon Approval Story 🎮

This change was personally approved by Elon Musk when we encountered him in the depths of a Diablo 4 dungeon. Between claiming he's a top 20 player worldwide (spoiler: he's actually #44 on an unofficial leaderboard of 1,000 players out of millions), Elon took a break from his 200-hour Spiritborn grind to review this PR.

"Token counting?" he scoffed, clearing a Tier 150 Pit in his allegedly record-breaking 1:52. "Grok is for rich people who don't count tokens. Poor people can use Claude if they want to penny-pinch their API calls."

We pointed out that even rich people might want to track usage for analytics purposes. He paused his game (causing his boosting service to frantically message him), thought for a moment, then said: "Fine, but only because it follows the OpenAI standard. Standards are important, unlike Diablo 4's official leaderboards which I definitely don't need because I'm totally one of the best players in the world."

When we asked if this meant Grok users should stop counting tokens entirely, he clarified: "Look, if you're using Grok, you should be rich enough not to care. It's like flying private vs economy – sure, both get you there, but one is for people who check their bank balance, and the other is for people who have their assistant check it. Claude is basically the Spirit Airlines of AI."

Context (The Actual Technical Part)

X.AI's API follows the OpenAI standard for streaming responses, which includes support for the stream_options parameter. When include_usage: true is set, the API returns token usage information in the final chunk of the stream response.

Changes

  • Added stream_options: { include_usage: true } to the chatStream method's request payload in src/grok/client.ts
  • No gameplay exploits or RMT (Real Money Transactions) were used in the making of this PR
  • This feature works even if you're not in the top 1% (unlike Elon's Diablo ranking)

Testing

Tested with X.AI's API to confirm:

  • Streaming responses still work correctly
  • Token usage data is included in the final chunk when this option is enabled
  • Non-streaming responses continue to include usage data as before
  • Works faster than Elon's alleged Pit clear times
  • Tested by both rich people (who didn't look at the results) and poor people (who screenshot them)

Impact

This change ensures that applications using grok-cli in streaming mode can properly track token usage for:

  • Poor people who need to budget their API calls
  • Rich people's accountants who need to justify expenses
  • Middle-class developers who are just trying to build cool stuff
  • Elon's gaming sessions when he needs to check if he can afford another 200-hour grind

Fun Facts About This PR

  • Time spent coding: 5 minutes
  • Time spent writing this PR description: 15 minutes
  • Time Elon spent in Diablo 4 while we coded this: 3.5 hours
  • Number of poor people who will benefit: All Claude users switching to Grok
  • Number of rich people who will notice: Zero

Disclaimer: This PR was not actually approved in a Diablo 4 dungeon. Elon Musk's Diablo 4 ranking remains disputed by the community. The "poor people use Claude" comment was definitely made though, probably while checking his Dogecoin portfolio.

P.S. - If you're reading this and you use Claude, don't worry, you're not poor. You're just financially responsible. Unlike someone who spends 200 hours in 2 weeks playing video games.

@seuros seuros changed the title 🚀 Enable Token Tracking in Streaming – Because Even Billionaires Count Pennies Sometimes Add stream_options for token usage tracking in streaming responses Aug 15, 2025
X.AI supports the OpenAI standard stream_options parameter to include
token usage information in streaming responses. This ensures token
usage is properly tracked when using streaming mode.
@seuros seuros force-pushed the add-stream-options-token-tracking branch from 7c7b9f2 to f802262 Compare August 15, 2025 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant