mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-09-20 08:57:57 +00:00
feat: record cache hit/miss per request in audit trail and WebUI
- Req: add CacheHit and CacheMiss fields (carrying upstream cache accounting from either prompt_tokens_details.cached_tokens or legacy prompt_cache_hit_tokens) - recordChatUsage (non-streaming): copy cache fields from resp.TokenUsage - pumpStream (streaming): write lastUsage cache fields back onto rec at stream end, so streaming requests carry cache data too - CSV export: add first_byte_ms, cache_hit_tokens, cache_miss_tokens columns alongside the existing latency/prompt/completion - WebUI request-records table: add a Cache column showing hit% per row (green/amber tag with tooltip hit/miss breakdown; em-dash when the upstream reported no cache data)
This commit is contained in:
@ -30,6 +30,11 @@ type Req struct {
|
||||
// to the first SSE chunk sent to the client); for non-streaming it
|
||||
// equals LatMs. 0 when unmeasured (legacy records).
|
||||
FirstByteMs int64 `json:"first_byte_ms,omitempty"`
|
||||
// CacheHit / CacheMiss carry the upstream prompt-cache accounting
|
||||
// (DeepSeek-style hit/miss tokens) when the upstream reports it.
|
||||
// Both 0 = upstream gave no cache data.
|
||||
CacheHit int64 `json:"cache_hit_tokens,omitempty"`
|
||||
CacheMiss int64 `json:"cache_miss_tokens,omitempty"`
|
||||
OK bool `json:"ok"`
|
||||
// Status http status code
|
||||
Status int `json:"status"`
|
||||
|
||||
Reference in New Issue
Block a user