mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-09-20 17:07:59 +00:00
feat(gateway): pass through exact upstream token usage in streams
Streaming responses now carry the upstream's real token usage instead of gateway estimates: - UnifiedChunk gains an optional Usage field; adapters (opencode, openai) extract usage from upstream stream chunks (including the final chunk with empty choices) and pass it through. - standardSSEChunk preserves usage for passthrough adapters. - Gateway emits the exact usage in the final stream chunk when available, falling back to estimates only when the upstream provided none. Non-streaming usage was already fixed to emit OpenAI-standard keys.
This commit is contained in:
@ -138,6 +138,11 @@ type UnifiedChunk struct {
|
||||
Done bool `json:"done"`
|
||||
ToolCalls json.RawMessage `json:"tool_calls,omitempty"`
|
||||
ReasoningContent string `json:"reasoning_content,omitempty"`
|
||||
// Usage carries the upstream token usage when the stream chunk provides
|
||||
// it (OpenAI-style streams attach usage to some chunks; the final one
|
||||
// often has empty choices). Gateway uses it to emit exact usage in the
|
||||
// final stream chunk instead of estimates.
|
||||
Usage *TokenUsage `json:"usage,omitempty"`
|
||||
}
|
||||
|
||||
// Meta passed to Lua build_headers hook
|
||||
|
||||
Reference in New Issue
Block a user