Skip to content

fix(openai): subtract cached tokens from realtime input usage - #1504

Open
pcbeingused333 wants to merge 1 commit into
SciSharp:masterfrom
pcbeingused333:fix/realtime-openai-cached-usage
Open

pcbeingused333 wants to merge 1 commit into
SciSharp:masterfrom
pcbeingused333:fix/realtime-openai-cached-usage

Conversation

@pcbeingused333

Copy link
Copy Markdown
Contributor

No linked issue (sibling of the usage fixes in #1486: same token-accounting pattern, found by comparing the providers).

In RealTimeCompletionProvider.OnResponsedDone, TextInputTokens and AudioInputTokens are computed as x ?? 0 - y ?? 0. In C# - binds tighter than ??, so this parses as x ?? (0 - y) ?? 0 and the cached count is never subtracted whenever x is present. The reported input tokens therefore include the cached ones, which are also reported separately in CachedTextInputTokens/CachedAudioInputTokens, so cached input is counted twice in cost accounting. The Chat and Live providers already subtract the cached count.

Fix: parenthesize both operands: (x ?? 0) - (y ?? 0).

Test: OpenAiRealtimeUsageTests feeds a response.done event with 100 text / 50 audio input tokens, of which 30 / 20 are cached, through the provider and captures the TokenStatsModel passed to the content-generating hook. It fails on main (text input = 100, expected 70) and passes with the change.

This change was prepared with the assistance of an AI coding tool (Claude Code); the author is responsible for the submission.

'a ?? 0 - b ?? 0' parses as 'a ?? (0 - b) ?? 0' in C#, so the cached
token counts were never subtracted from text/audio input tokens in the
OpenAI Realtime response.done handler. Parenthesize both operands.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@qodo-code-review

Copy link
Copy Markdown
Contributor

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant