feat: record cache hit/miss per request in audit trail and WebUI

- Req: add CacheHit and CacheMiss fields (carrying upstream cache
  accounting from either prompt_tokens_details.cached_tokens or legacy
  prompt_cache_hit_tokens)
- recordChatUsage (non-streaming): copy cache fields from resp.TokenUsage
- pumpStream (streaming): write lastUsage cache fields back onto rec at
  stream end, so streaming requests carry cache data too
- CSV export: add first_byte_ms, cache_hit_tokens, cache_miss_tokens
  columns alongside the existing latency/prompt/completion
- WebUI request-records table: add a Cache column showing hit% per row
  (green/amber tag with tooltip hit/miss breakdown; em-dash when the
  upstream reported no cache data)
This commit is contained in:
dev
2026-08-25 09:36:21 +08:00
parent 18c2385c61
commit 24609289e8
4 changed files with 44 additions and 3 deletions

View File

@ -30,6 +30,11 @@ type Req struct {
// to the first SSE chunk sent to the client); for non-streaming it
// equals LatMs. 0 when unmeasured (legacy records).
FirstByteMs int64 `json:"first_byte_ms,omitempty"`
// CacheHit / CacheMiss carry the upstream prompt-cache accounting
// (DeepSeek-style hit/miss tokens) when the upstream reports it.
// Both 0 = upstream gave no cache data.
CacheHit int64 `json:"cache_hit_tokens,omitempty"`
CacheMiss int64 `json:"cache_miss_tokens,omitempty"`
OK bool `json:"ok"`
// Status http status code
Status int `json:"status"`