mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-09-20 08:57:57 +00:00
feat: surface zero cache hits — distinguish 'missed' from 'not reported'
Live testing across the zen pool showed models report prompt_tokens_details.cached_tokens even when the hit count is 0 (e.g. nemotron-3-ultra-free returns cached_tokens:0, audio_tokens:0, cache_write_tokens:0). The previous >0 guard dropped those objects, so a cache-enabled upstream looked identical to one without cache support. - types: PromptTokensDetails.CachedTokens always emitted (drop inner omitempty) so clients see cached_tokens:0 explicitly; dsh reads it as a 0% hit instead of 'no data' - adapters (9): forward prompt_tokens_details whenever the upstream provides it (presence check instead of >0) - Req: add cache_reported flag set when usage carried cache accounting; WebUI shows an amber 0% tag for reported-but-missed rows and keeps the em-dash only for sources that never report cache data
This commit is contained in:
@ -35,6 +35,10 @@ type Req struct {
|
||||
// Both 0 = upstream gave no cache data.
|
||||
CacheHit int64 `json:"cache_hit_tokens,omitempty"`
|
||||
CacheMiss int64 `json:"cache_miss_tokens,omitempty"`
|
||||
// CacheReported marks that the upstream usage reported cache
|
||||
// accounting at all (even when the hit count is 0). The WebUI shows
|
||||
// "0%" instead of "—" for such rows.
|
||||
CacheReported bool `json:"cache_reported,omitempty"`
|
||||
OK bool `json:"ok"`
|
||||
// Status http status code
|
||||
Status int `json:"status"`
|
||||
|
||||
Reference in New Issue
Block a user