mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-09-20 17:07:59 +00:00
feat: add per-source TTFB and tokens/s metrics to status page
- Req: add FirstByteMs field (ms to first byte, tracked for streaming) - Stat: add FirstByteSum for aggregation - SourceAverages(): new method computing per-source avg TTFB and tokens/s from the in-memory ring (300s window) - SourceStatus: add AvgFirstByteMs and AvgTokPerS fields - pumpStream: record FirstByteMs after first SSE chunk sent to client - singleChat/singleChatAuto: set FirstByteMs = LatMs (non-streaming) - handleStatusAPI: populate the new SourceStatus fields from SourceAverages() - WebUI source table: two new columns showing TTFB (s) and Tokens/s
This commit is contained in:
@ -217,6 +217,12 @@ type SourceStatus struct {
|
||||
// probe was rate-limited by the upstream).
|
||||
RecentOK int64 `json:"recent_ok,omitempty"`
|
||||
RecentErr int64 `json:"recent_err,omitempty"`
|
||||
// AvgFirstByteMs is the mean time-to-first-byte (ms) for recently
|
||||
// successful requests served by this source (0 when unmeasured).
|
||||
AvgFirstByteMs int64 `json:"avg_first_byte_ms,omitempty"`
|
||||
// AvgTokPerS is the aggregate completion throughput (tokens/s) for
|
||||
// recently successful requests served by this source (0 when no data).
|
||||
AvgTokPerS int64 `json:"avg_tok_per_s,omitempty"`
|
||||
}
|
||||
|
||||
// ProbeAll runs a live reachability check for every provider (in parallel).
|
||||
|
||||
Reference in New Issue
Block a user