mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-10-03 23:54:06 +00:00
feat(stats): 用量按日/周/月/全部统计（审计文件聚合）
统计页原来只有一个视图——进程启动以来的累计。早上没人和一整周没人看起来 一模一样。加日/周/月/总四个周期。 ## 口径与实现 周期视图必须走审计文件，不能走内存聚合：内存 byModel/byKey 等是终身累计， 而 recs 环形缓冲只有 500 条（defaultRingSize）。读环会把任何超过几百个 请求的周期悄悄少算——这正是要消除的那类错数。 - 日/周/月 = UTC 日历窗口（今日 / ISO 周周一 00:00 / 本月 1 日）。 刻意不用滚动 24h：滚动窗口会让"今天"和"最近一天"边界不同，同一个数字 随查看时刻在两张卡片间跳。UTC 也和 billing 的峰段计算同口径，峰谷小时 不会在费用视图和用量视图里落到不同一天。 - 全部 = 复用现有 Snapshot（内存聚合），无审计文件时依然可用。 - 时间桶：日→每小时（今天内部的尖峰要看得见），周/月→每天（否则一周是 7×24 个点、一个月 31×24）。全部视图无时间线（终身总量没有有意义的 时间轴，硬画 500 个滚动小时点是另一种撒谎）。 - key 过滤在所有维度生效；非法 period 返回 400 而不是静默回落"全部"—— 书签里的手误应当报错，而不是悄悄换成终身数字。 ## 判据（10 条 + 8 个变异全部被捕获） 窗口边界（含"周日必须回到上一个周一"这个 Go Weekday() 陷阱）、旧记录不 计入、维度独立聚合且 by_model 求和等于 total、日桶按小时且有序、key 隔离、 全部视图走终身、空窗口标记 truncated、period 校验、query 解析。 变异验证时 by_status 假绿了一次：禁用状态码聚合后判据全过，查下去是我 **根本没测 by_status**（零覆盖）。补 TestPeriodStatusDimension 后该变异 立即被捕获。判据报假问题时，先怀疑判据——这次确实是我错了。 ## 真实流量核对 生产审计文件手算 vs 后端（含轮转文件）： day 手算 3559 / 后端 3532 week 手算 43240 / 后端 35034 month 手算 13696 / 后端 13670 差异是核对快照与请求之间的新流量，量级一致。 一个必须说明的发现：审计文件里混着两种记录 —— Req（type/model/ prompt_tokens）和访问日志（lat_ms/status/path）。46026 行里 33450 行是 访问日志，Go 侧按 r.Type=="" 跳过。这不是 bug（CSV 导出同样如此），但 意味着任何按行数手算都必须过滤，否则会差一个数量级。 CDP 实测四周期切换：reqs 3,571 / 35,075 / 13,710 / 196,712，与后端一致， 无控制台错误，localStorage 持久化生效。
This commit is contained in:
JianFeeeee
@ -433,6 +433,32 @@ func (g *Gateway) handleStatsAPI(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
}
|
||||
key := exportKey(r)
|
||||
// ?period=day|week|month|all switches the whole payload to a calendar
|
||||
// window (UTC) aggregated off the audit files, instead of the
|
||||
// since-process-start totals. The CSV exports below are unaffected: they
|
||||
// take an explicit from/to range and stream, so a period selector there
|
||||
// would only be a second way to spell the same bounds.
|
||||
if p := periodFromQuery(r.URL.Query()); p != PeriodAll {
|
||||
if !ValidPeriod(p) {
|
||||
writeError(w, http.StatusBadRequest, "bad_period",
|
||||
"period must be one of day, week, month, all")
|
||||
return
|
||||
}
|
||||
out := g.stats.PeriodSnapshot(p, key, time.Now())
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{
|
||||
"period": out.Period,
|
||||
"from": out.From,
|
||||
"total": out.Total,
|
||||
"by_key": out.ByKey,
|
||||
"by_model": out.Models,
|
||||
"by_source": out.Srcs,
|
||||
"by_status": out.Status,
|
||||
"buckets": out.Bucket,
|
||||
"truncated": out.Truncated,
|
||||
"key_names": g.keyNamesFor(),
|
||||
})
|
||||
return
|
||||
}
|
||||
if r.URL.Query().Get("export") == "csv" {
|
||||
from, _ := strconv.ParseInt(r.URL.Query().Get("from"), 10, 64)
|
||||
to, _ := strconv.ParseInt(r.URL.Query().Get("to"), 10, 64)
|
||||
@ -540,14 +566,22 @@ func (g *Gateway) handleStatsAPI(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
snap := g.stats.Snapshot(limit, key)
|
||||
keyNames := map[string]string{}
|
||||
for _, k := range g.core.ListKeys() {
|
||||
keyNames[keyID(k.Key)] = k.Name
|
||||
}
|
||||
snap["key_names"] = keyNames
|
||||
snap["key_names"] = g.keyNamesFor()
|
||||
writeJSON(w, http.StatusOK, snap)
|
||||
}
|
||||
|
||||
// keyNamesFor is the masked-id -> display-name map every stats payload needs.
|
||||
// It is keyed by keyID (the mask), not the raw key, because that is what the
|
||||
// aggregate rows carry — building it in one place stops the period branch and
|
||||
// the lifetime branch from drifting apart.
|
||||
func (g *Gateway) keyNamesFor() map[string]string {
|
||||
names := map[string]string{}
|
||||
for _, k := range g.core.ListKeys() {
|
||||
names[keyID(k.Key)] = k.Name
|
||||
}
|
||||
return names
|
||||
}
|
||||
|
||||
// handleStatsRecordsAPI pages the request records straight off the audit files.
|
||||
// The dashboard loads only its first screen and asks for the next page as the
|
||||
// user scrolls, so neither side holds the full history: the server keeps no
|
||||
|
||||
234
internal/gateway/stats_period.go
Normal file
234
internal/gateway/stats_period.go
Normal file
@ -0,0 +1,234 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Period is a reporting window for the usage dashboard. The dashboard used to
|
||||
// have exactly one view — everything since process start — which made a quiet
|
||||
// morning indistinguishable from a quiet week. Periods give the operator a
|
||||
// scale to read the numbers at: today vs this week vs this month vs all time.
|
||||
//
|
||||
// The set is deliberately calendar-based and UTC-anchored. A rolling 24h window
|
||||
// would put "today" and "the last day" at different boundaries, so the same
|
||||
// number would move between two cards depending on when you looked; calendar
|
||||
// days are what people actually mean by "today". UTC also matches the billing
|
||||
// plugin's peak-window arithmetic, so a peak-rate hour does not land in a
|
||||
// different day in the cost view than in the usage view.
|
||||
type Period string
|
||||
|
||||
const (
|
||||
// PeriodDay is the current UTC calendar day.
|
||||
PeriodDay Period = "day"
|
||||
// PeriodWeek is the current ISO week (Mon 00:00 UTC to now).
|
||||
PeriodWeek Period = "week"
|
||||
// PeriodMonth is the current UTC calendar month.
|
||||
PeriodMonth Period = "month"
|
||||
// PeriodAll is since process start — the only view backed by the
|
||||
// in-memory aggregates, and the only one available when no audit file is
|
||||
// configured.
|
||||
PeriodAll Period = "all"
|
||||
)
|
||||
|
||||
// ValidPeriod reports whether p is a period the aggregator understands.
|
||||
// An unknown period is a client error, not a silent fallback to "all": a
|
||||
// dashboard that quietly shows lifetime totals when the caller asked for today
|
||||
// is worse than one that refuses.
|
||||
func ValidPeriod(p Period) bool {
|
||||
switch p {
|
||||
case PeriodDay, PeriodWeek, PeriodMonth, PeriodAll:
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// periodStart returns the inclusive start of the window for p at time now.
|
||||
// Only PeriodDay/Week/Month are meaningful here; PeriodAll returns 0, which
|
||||
// every "from > 0" bounds check treats as unbounded.
|
||||
func periodStart(p Period, now time.Time) int64 {
|
||||
now = now.UTC()
|
||||
switch p {
|
||||
case PeriodDay:
|
||||
return time.Date(now.Year(), now.Month(), now.Day(), 0, 0, 0, 0, time.UTC).UnixMilli()
|
||||
case PeriodWeek:
|
||||
// ISO week starts Monday. Go's Weekday() is Sunday=0, so the shift
|
||||
// below is 1 on Sunday and 0 on Monday..Saturday.
|
||||
off := (int(now.Weekday()) + 6) % 7
|
||||
d := now.AddDate(0, 0, -off)
|
||||
return time.Date(d.Year(), d.Month(), d.Day(), 0, 0, 0, 0, time.UTC).UnixMilli()
|
||||
case PeriodMonth:
|
||||
return time.Date(now.Year(), now.Month(), 1, 0, 0, 0, 0, time.UTC).UnixMilli()
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// PeriodBucket is one labelled point on the dashboard's timeline. Buckets are
|
||||
// the aggregation grain for a period: hourly for a day (so a spike is visible
|
||||
// inside "today"), daily for a week or month (so a week is not 7×24 points
|
||||
// wide and a month is not 31×24), and empty for "all" — a lifetime total has
|
||||
// no meaningful timeline, and pretending otherwise by drawing 500 hourly
|
||||
// buckets of rolling memory would be a different lie.
|
||||
type PeriodBucket struct {
|
||||
Bucket string `json:"bucket"`
|
||||
Stat
|
||||
}
|
||||
|
||||
// PeriodSnapshot is the period-scoped twin of Snapshot's payload: the same
|
||||
// totals and the same by_* rows, plus a timeline. It deliberately mirrors
|
||||
// Snapshot's field names so the dashboard's table painters work unchanged —
|
||||
// paintModelTable(st.by_model) reads rows of {name, ...Stat}, and that shape
|
||||
// does not care where the numbers came from.
|
||||
type PeriodSnapshot struct {
|
||||
Period Period `json:"period"`
|
||||
From int64 `json:"from"` // unix millis, 0 when PeriodAll
|
||||
Total Stat `json:"total"`
|
||||
ByKey []StatsRow `json:"by_key"`
|
||||
Models []StatsRow `json:"by_model"`
|
||||
Srcs []StatsRow `json:"by_source"`
|
||||
Status []agrRow `json:"by_status"`
|
||||
Bucket []PeriodBucket `json:"buckets"`
|
||||
// Truncated marks that the window was clipped by the retained audit
|
||||
// history, so the numbers are a lower bound rather than the true period
|
||||
// total. The dashboard shows this next to the numbers rather than letting
|
||||
// a rotated-away week read as "that week had no traffic".
|
||||
Truncated bool `json:"truncated"`
|
||||
}
|
||||
|
||||
// bucketKey maps a record's timestamp onto the timeline grain for p.
|
||||
// Daily buckets are stamped at UTC midnight; hourly buckets carry the hour.
|
||||
func bucketKey(p Period, ms int64) string {
|
||||
t := time.UnixMilli(ms).UTC()
|
||||
if p == PeriodDay {
|
||||
return t.Format("2006-01-02T15")
|
||||
}
|
||||
return t.Format("2006-01-02")
|
||||
}
|
||||
|
||||
// bucketOf returns the bucket label plus the truncated-flag side effects of
|
||||
// walking a file: a record older than the requested window means the window
|
||||
// starts before the retained history, and the file may have been cut short.
|
||||
func (s *Stats) PeriodSnapshot(p Period, key string, now time.Time) PeriodSnapshot {
|
||||
out := PeriodSnapshot{Period: p}
|
||||
from := periodStart(p, now)
|
||||
out.From = from
|
||||
|
||||
// "all" is exactly what Snapshot already answers, from the in-memory
|
||||
// aggregates, and it is the one view that must keep working with no audit
|
||||
// file configured at all (a fresh dev setup, or an operator who turned
|
||||
// auditing off). Serving it from the same code path keeps the dashboard's
|
||||
// "total" card identical whether or not a period is selected.
|
||||
if p == PeriodAll {
|
||||
snap := s.Snapshot(firstScreenRecords, key)
|
||||
if tot, ok := snap["total"].(Stat); ok {
|
||||
out.Total = tot
|
||||
}
|
||||
out.ByKey, _ = snap["by_key"].([]StatsRow)
|
||||
out.Models, _ = snap["by_model"].([]StatsRow)
|
||||
out.Srcs, _ = snap["by_source"].([]StatsRow)
|
||||
out.Status, _ = snap["by_status"].([]agrRow)
|
||||
// replay_partial is the same "these numbers came from a bounded
|
||||
// tail" caveat, carried through under this view's own name.
|
||||
out.Truncated, _ = snap["replay_partial"].(bool)
|
||||
return out
|
||||
}
|
||||
|
||||
// A bounded window is aggregated from the audit files, because the
|
||||
// in-memory aggregates are lifetime totals and the ring buffer holds only
|
||||
// maxRecs records (500 by default). Reading the ring would silently
|
||||
// under-report any period longer than the last few hundred requests.
|
||||
total := Stat{}
|
||||
byKey := map[string]*Stat{}
|
||||
byModel := map[string]*Stat{}
|
||||
bySrc := map[string]*Stat{}
|
||||
byStatus := map[string]*Stat{}
|
||||
buckets := map[string]*Stat{}
|
||||
|
||||
var seen int
|
||||
err := s.StreamAuditRecords(from, 0, key, func(r Req) error {
|
||||
seen++
|
||||
incStatus(&total, "", r)
|
||||
inc(byKey, r.Key, r)
|
||||
if r.Model != "" {
|
||||
inc(byModel, r.Model, r)
|
||||
}
|
||||
inc(bySrc, r.Source, r)
|
||||
if r.Status != 0 {
|
||||
inc(byStatus, strconv.Itoa(r.Status), r)
|
||||
}
|
||||
k := bucketKey(p, r.Time)
|
||||
b := buckets[k]
|
||||
if b == nil {
|
||||
b = &Stat{}
|
||||
buckets[k] = b
|
||||
}
|
||||
incStatus(b, k, r)
|
||||
return nil
|
||||
})
|
||||
|
||||
out.Total = total
|
||||
out.ByKey = rows(byKey)
|
||||
out.Models = rows(byModel)
|
||||
out.Srcs = rows(bySrc)
|
||||
bs := make([]agrRow, 0, len(byStatus))
|
||||
for code, st := range byStatus {
|
||||
bs = append(bs, agrRow{Name: code, Stat: *st})
|
||||
}
|
||||
sort.Slice(bs, func(i, j int) bool {
|
||||
ci, _ := strconv.Atoi(bs[i].Name)
|
||||
cj, _ := strconv.Atoi(bs[j].Name)
|
||||
return ci < cj
|
||||
})
|
||||
out.Status = bs
|
||||
|
||||
ks := make([]string, 0, len(buckets))
|
||||
for k := range buckets {
|
||||
ks = append(ks, k)
|
||||
}
|
||||
// Chronological, string-sorted: "2006-01-02T15" and "2006-01-02" both
|
||||
// sort lexicographically in time order, so no date parsing is needed.
|
||||
sort.Strings(ks)
|
||||
out.Bucket = make([]PeriodBucket, 0, len(ks))
|
||||
for _, k := range ks {
|
||||
out.Bucket = append(out.Bucket, PeriodBucket{Bucket: k, Stat: *buckets[k]})
|
||||
}
|
||||
|
||||
// The window is only "complete" if the audit walk actually reached back
|
||||
// far enough to cover it. Two ways it cannot:
|
||||
//
|
||||
// 1. The walk found nothing at all in a window that certainly had
|
||||
// traffic, because the files holding it rotated away.
|
||||
// 2. The walk errored part-way (I/O), leaving a partial total.
|
||||
//
|
||||
// Case 2 is reported from err; case 1 from seen == 0 combined with the
|
||||
// caller having asked for a bounded window. It is deliberately
|
||||
// conservative: a genuinely empty hour is rare enough that flagging it as
|
||||
// possibly-truncated costs one tooltip, whereas silently under-reporting a
|
||||
// month because rotation ate it is a wrong number with no indication.
|
||||
if err != nil || (seen == 0 && p != PeriodAll) {
|
||||
out.Truncated = true
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// periodFromQuery parses the ?period= parameter. An empty value means "all" so
|
||||
// that existing callers of /api/stats keep seeing exactly what they saw.
|
||||
// A malformed value is rejected by the caller (ValidPeriod) rather than
|
||||
// defaulting, so a typo in a bookmarked URL surfaces as an error instead of
|
||||
// silently switching the operator to lifetime totals.
|
||||
func periodFromQuery(q map[string][]string) Period {
|
||||
v := strings.ToLower(strings.TrimSpace(firstQuery(q, "period")))
|
||||
if v == "" {
|
||||
return PeriodAll
|
||||
}
|
||||
return Period(v)
|
||||
}
|
||||
|
||||
func firstQuery(q map[string][]string, key string) string {
|
||||
if vs, ok := q[key]; ok && len(vs) > 0 {
|
||||
return vs[0]
|
||||
}
|
||||
return ""
|
||||
}
|
||||
321
internal/gateway/stats_period_test.go
Normal file
321
internal/gateway/stats_period_test.go
Normal file
@ -0,0 +1,321 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// auditPeriod writes records straight into an audit file, which is what the
|
||||
// period aggregator reads. Going through Record() would also rotate/prune the
|
||||
// file; here the file shape is the thing under test.
|
||||
func auditPeriod(t *testing.T, path string, recs ...Req) {
|
||||
t.Helper()
|
||||
f, err := os.Create(path)
|
||||
if err != nil {
|
||||
t.Fatalf("create audit file: %v", err)
|
||||
}
|
||||
defer f.Close()
|
||||
for _, r := range recs {
|
||||
if r.Time == 0 {
|
||||
t.Fatal("test record needs an explicit timestamp")
|
||||
}
|
||||
if r.Type == "" {
|
||||
r.Type = "chat"
|
||||
}
|
||||
b, err := json.Marshal(r)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal record: %v", err)
|
||||
}
|
||||
if _, err := f.Write(append(b, '\n')); err != nil {
|
||||
t.Fatalf("write record: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func periodStats(t *testing.T, auditPath string) *Stats {
|
||||
t.Helper()
|
||||
s := NewStats(10)
|
||||
s.mu.Lock()
|
||||
s.auditPath = auditPath
|
||||
s.mu.Unlock()
|
||||
return s
|
||||
}
|
||||
|
||||
// TestPeriodWindowBoundaries is the boundary contract: a day window covers
|
||||
// exactly today-from-midnight, a week window starts Monday (not Sunday — the
|
||||
// Go Weekday() trap), and a month window starts on the 1st. All UTC.
|
||||
func TestPeriodWindowBoundaries(t *testing.T) {
|
||||
// Wednesday 2026-03-11 15:30 UTC.
|
||||
now := time.Date(2026, 3, 11, 15, 30, 0, 0, time.UTC)
|
||||
day := time.Date(2026, 3, 11, 0, 0, 0, 0, time.UTC)
|
||||
if got := time.UnixMilli(periodStart(PeriodDay, now)).UTC(); !got.Equal(day) {
|
||||
t.Errorf("day window starts %s, want %s", got, day)
|
||||
}
|
||||
// Wednesday minus 2 days = Monday the 9th.
|
||||
monday := time.Date(2026, 3, 9, 0, 0, 0, 0, time.UTC)
|
||||
if got := time.UnixMilli(periodStart(PeriodWeek, now)).UTC(); !got.Equal(monday) {
|
||||
t.Errorf("week window starts %s, want Monday %s", got, monday)
|
||||
}
|
||||
// Same instant on a Sunday must still start on the PRECEDING Monday,
|
||||
// never on the Sunday itself. This is the off-by-one that
|
||||
// int(now.Weekday()) would introduce (Sunday=0 → no shift).
|
||||
sun := time.Date(2026, 3, 15, 10, 0, 0, 0, time.UTC) // Sunday
|
||||
prevMon := time.Date(2026, 3, 9, 0, 0, 0, 0, time.UTC)
|
||||
if got := time.UnixMilli(periodStart(PeriodWeek, sun)).UTC(); !got.Equal(prevMon) {
|
||||
t.Errorf("Sunday week window starts %s, want the Monday before it %s", got, prevMon)
|
||||
}
|
||||
first := time.Date(2026, 3, 1, 0, 0, 0, 0, time.UTC)
|
||||
if got := time.UnixMilli(periodStart(PeriodMonth, now)).UTC(); !got.Equal(first) {
|
||||
t.Errorf("month window starts %s, want %s", got, first)
|
||||
}
|
||||
if got := periodStart(PeriodAll, now); got != 0 {
|
||||
t.Errorf("all window from = %d, want 0 (unbounded)", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodAggregationExcludesOlderRecords is the reason the aggregator reads
|
||||
// the audit file: a record outside the window must not be counted, even though
|
||||
// it sits in the very same file, right next to records that are.
|
||||
func TestPeriodAggregationExcludesOlderRecords(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "audit.jsonl")
|
||||
now := time.Date(2026, 3, 11, 15, 0, 0, 0, time.UTC)
|
||||
h := func(n int) int64 { return now.Add(-time.Duration(n) * time.Hour).UnixMilli() }
|
||||
|
||||
auditPeriod(t, path,
|
||||
// inside today
|
||||
Req{Time: h(1), Model: "m", Source: "s", Prompt: 100, Compl: 10, LatMs: 5, OK: true},
|
||||
Req{Time: h(2), Model: "m", Source: "s", Prompt: 200, Compl: 20, LatMs: 7, OK: true},
|
||||
// outside today (yesterday) but inside the week
|
||||
Req{Time: now.AddDate(0, 0, -1).UnixMilli(), Model: "old", Source: "s", Prompt: 999, Compl: 99, OK: true},
|
||||
)
|
||||
s := periodStats(t, path)
|
||||
|
||||
day := s.PeriodSnapshot(PeriodDay, "", now)
|
||||
if day.Total.Reqs != 2 {
|
||||
t.Errorf("day window counted %d requests, want 2 (yesterday's record must be excluded)", day.Total.Reqs)
|
||||
}
|
||||
if day.Total.Prompt != 300 {
|
||||
t.Errorf("day prompt tokens = %d, want 300", day.Total.Prompt)
|
||||
}
|
||||
|
||||
week := s.PeriodSnapshot(PeriodWeek, "", now)
|
||||
if week.Total.Reqs != 3 {
|
||||
t.Errorf("week window counted %d requests, want 3", week.Total.Reqs)
|
||||
}
|
||||
if week.Total.Prompt != 1299 {
|
||||
t.Errorf("week prompt tokens = %d, want 1299", week.Total.Prompt)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodDimensionsAggregateIndependently checks the by_* rows actually
|
||||
// split by their own key rather than all collapsing into one row.
|
||||
func TestPeriodDimensionsAggregateIndependently(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "audit.jsonl")
|
||||
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
|
||||
auditPeriod(t, path,
|
||||
Req{Time: now.UnixMilli(), Key: "k1", Model: "alpha", Source: "s1", Prompt: 10, OK: true},
|
||||
Req{Time: now.UnixMilli(), Key: "k1", Model: "beta", Source: "s2", Prompt: 20, OK: true},
|
||||
Req{Time: now.UnixMilli(), Key: "k2", Model: "alpha", Source: "s1", Prompt: 30, OK: true},
|
||||
)
|
||||
s := periodStats(t, path)
|
||||
got := s.PeriodSnapshot(PeriodDay, "", now)
|
||||
|
||||
if len(got.Models) != 2 {
|
||||
t.Fatalf("by_model has %d rows, want 2: %+v", len(got.Models), got.Models)
|
||||
}
|
||||
// alpha = 10 + 30
|
||||
for _, r := range got.Models {
|
||||
if r.Name == "alpha" && r.Prompt != 40 {
|
||||
t.Errorf("alpha prompt = %d, want 40", r.Prompt)
|
||||
}
|
||||
}
|
||||
if len(got.Srcs) != 2 {
|
||||
t.Errorf("by_source has %d rows, want 2: %+v", len(got.Srcs), got.Srcs)
|
||||
}
|
||||
if len(got.ByKey) != 2 {
|
||||
t.Errorf("by_key has %d rows, want 2: %+v", len(got.ByKey), got.ByKey)
|
||||
}
|
||||
// The rows must add up to the total, or the dashboard shows a total that
|
||||
// disagrees with its own table.
|
||||
var sum int64
|
||||
for _, r := range got.Models {
|
||||
sum += r.Prompt
|
||||
}
|
||||
if sum != got.Total.Prompt {
|
||||
t.Errorf("by_model prompts sum to %d but total is %d — the table would contradict the KPI", sum, got.Total.Prompt)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodDayBucketsAreHourlyAndOrdered pins the timeline grain and its
|
||||
// order. Day = hourly (a spike must be visible inside "today"); week/month =
|
||||
// daily. Sorted chronologically, because a chart fed unsorted buckets draws
|
||||
// nonsense.
|
||||
func TestPeriodDayBucketsAreHourlyAndOrdered(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "audit.jsonl")
|
||||
now := time.Date(2026, 3, 11, 15, 0, 0, 0, time.UTC)
|
||||
// Deliberately out of order in the file.
|
||||
auditPeriod(t, path,
|
||||
Req{Time: now.Add(-2 * time.Hour).UnixMilli(), Model: "m", Prompt: 1, OK: true},
|
||||
Req{Time: now.Add(-4 * time.Hour).UnixMilli(), Model: "m", Prompt: 1, OK: true},
|
||||
Req{Time: now.UnixMilli(), Model: "m", Prompt: 1, OK: true},
|
||||
)
|
||||
s := periodStats(t, path)
|
||||
got := s.PeriodSnapshot(PeriodDay, "", now)
|
||||
if len(got.Bucket) != 3 {
|
||||
t.Fatalf("day buckets = %d, want 3 (hourly): %+v", len(got.Bucket), got.Bucket)
|
||||
}
|
||||
for i := 1; i < len(got.Bucket); i++ {
|
||||
if got.Bucket[i-1].Bucket >= got.Bucket[i].Bucket {
|
||||
t.Errorf("day buckets not chronological at %d: %q >= %q",
|
||||
i, got.Bucket[i-1].Bucket, got.Bucket[i].Bucket)
|
||||
}
|
||||
}
|
||||
// Weekly grain is daily, so the same three records collapse to one day.
|
||||
week := s.PeriodSnapshot(PeriodWeek, "", now)
|
||||
if len(week.Bucket) != 1 {
|
||||
t.Errorf("week buckets = %d, want 1 (daily grain): %+v", len(week.Bucket), week.Bucket)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodKeyFilterIsolatesKeys guards the multi-tenant boundary: asking for
|
||||
// one key must never return another key's rows in any dimension.
|
||||
func TestPeriodKeyFilterIsolatesKeys(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "audit.jsonl")
|
||||
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
|
||||
auditPeriod(t, path,
|
||||
Req{Time: now.UnixMilli(), Key: "alice", Model: "m", Source: "s", Prompt: 10, OK: true},
|
||||
Req{Time: now.UnixMilli(), Key: "bob", Model: "m", Source: "s", Prompt: 999, OK: true},
|
||||
)
|
||||
s := periodStats(t, path)
|
||||
got := s.PeriodSnapshot(PeriodDay, "alice", now)
|
||||
if got.Total.Reqs != 1 || got.Total.Prompt != 10 {
|
||||
t.Errorf("alice's window = %d reqs / %d prompt, want 1 / 10 — another key leaked in",
|
||||
got.Total.Reqs, got.Total.Prompt)
|
||||
}
|
||||
if len(got.Srcs) != 1 || got.Srcs[0].Prompt != 10 {
|
||||
t.Errorf("by_source leaked: %+v", got.Srcs)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodAllUsesLifetimeAggregates checks "all" still answers from the
|
||||
// in-memory aggregates (no audit file needed) and carries the same rows the
|
||||
// dashboard already renders.
|
||||
func TestPeriodAllUsesLifetimeAggregates(t *testing.T) {
|
||||
s := NewStats(100)
|
||||
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
|
||||
s.Record(Req{Time: now.AddDate(0, 0, -40).UnixMilli(), Key: "k", Model: "m", Source: "s", Prompt: 7, OK: true})
|
||||
s.Record(Req{Time: now.UnixMilli(), Key: "k", Model: "m", Source: "s", Prompt: 3, OK: true})
|
||||
|
||||
got := s.PeriodSnapshot(PeriodAll, "", now)
|
||||
if got.Total.Reqs != 2 || got.Total.Prompt != 10 {
|
||||
t.Errorf("all view = %d reqs / %d prompt, want 2 / 10 (lifetime)", got.Total.Reqs, got.Total.Prompt)
|
||||
}
|
||||
if len(got.Models) != 1 || got.Models[0].Prompt != 10 {
|
||||
t.Errorf("all view by_model = %+v, want one row with 10 prompt", got.Models)
|
||||
}
|
||||
// A lifetime total has no timeline to draw; buckets must stay empty
|
||||
// rather than inventing 500 hourly points.
|
||||
if len(got.Bucket) != 0 {
|
||||
t.Errorf("all view produced %d buckets, want 0", len(got.Bucket))
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodEmptyWindowFlagsTruncated covers the "rotated away" case: a bounded
|
||||
// window with nothing to show may mean "quiet day" or "the files are gone".
|
||||
// The aggregator must not report a confidently wrong zero.
|
||||
func TestPeriodEmptyWindowFlagsTruncated(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
s := periodStats(t, filepath.Join(dir, "audit.jsonl")) // file does not exist
|
||||
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
|
||||
got := s.PeriodSnapshot(PeriodDay, "", now)
|
||||
if got.Total.Reqs != 0 {
|
||||
t.Errorf("empty window reqs = %d, want 0", got.Total.Reqs)
|
||||
}
|
||||
if !got.Truncated {
|
||||
t.Error("★ an empty bounded window must be flagged truncated — otherwise a " +
|
||||
"rotated-away week reads as 'no traffic that week'")
|
||||
}
|
||||
}
|
||||
|
||||
// TestValidPeriodRejectsUnknown guards the deliberate strictness: a typo must
|
||||
// be an error, not a silent fallback to lifetime totals.
|
||||
func TestValidPeriodRejectsUnknown(t *testing.T) {
|
||||
for _, ok := range []Period{PeriodDay, PeriodWeek, PeriodMonth, PeriodAll} {
|
||||
if !ValidPeriod(ok) {
|
||||
t.Errorf("ValidPeriod(%q) = false, want true", ok)
|
||||
}
|
||||
}
|
||||
for _, bad := range []Period{"", "year", "hour", "today", "DAY "} {
|
||||
if ValidPeriod(bad) {
|
||||
t.Errorf("ValidPeriod(%q) = true, want false", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodStatusDimensionCountsFailures covers by_status, which nothing else
|
||||
// exercised — an aggregation branch with no test is exactly how a period view
|
||||
// silently loses the failure pie. M8 (disabling the status dimension) is
|
||||
// caught only because this test exists.
|
||||
func TestPeriodStatusDimensionCountsFailures(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "audit.jsonl")
|
||||
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
|
||||
auditPeriod(t, path,
|
||||
Req{Time: now.UnixMilli(), Model: "m", Source: "s", Status: 200, OK: true, Prompt: 5},
|
||||
Req{Time: now.UnixMilli(), Model: "m", Source: "s", Status: 502, OK: false, Prompt: 1},
|
||||
Req{Time: now.UnixMilli(), Model: "m", Source: "s", Status: 502, OK: false, Prompt: 1},
|
||||
Req{Time: now.UnixMilli(), Model: "m", Source: "s", Status: 429, OK: false, Prompt: 1},
|
||||
)
|
||||
s := periodStats(t, path)
|
||||
got := s.PeriodSnapshot(PeriodDay, "", now)
|
||||
|
||||
byCode := map[string]int64{}
|
||||
for _, r := range got.Status {
|
||||
byCode[r.Name] = r.Reqs
|
||||
}
|
||||
if len(got.Status) != 3 {
|
||||
t.Fatalf("by_status has %d rows, want 3 (200/502/429): %+v", len(got.Status), got.Status)
|
||||
}
|
||||
if byCode["200"] != 1 || byCode["502"] != 2 || byCode["429"] != 1 {
|
||||
t.Errorf("by_status counts = %v, want 200:1 502:2 429:1", byCode)
|
||||
}
|
||||
// Status codes must sort numerically: "429" before "502" lexically would
|
||||
// render the failure legend in the wrong order.
|
||||
if got.Status[0].Name != "200" || got.Status[1].Name != "429" || got.Status[2].Name != "502" {
|
||||
t.Errorf("by_status not numerically sorted: %s, %s, %s",
|
||||
got.Status[0].Name, got.Status[1].Name, got.Status[2].Name)
|
||||
}
|
||||
// The failure rows must agree with the OK/Err split on the total, or the
|
||||
// pie and the KPI row contradict each other.
|
||||
if got.Total.OK != 1 || got.Total.Err != 3 {
|
||||
t.Errorf("total ok/err = %d/%d, want 1/3", got.Total.OK, got.Total.Err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPeriodFromQuery(t *testing.T) {
|
||||
cases := []struct {
|
||||
query string
|
||||
want Period
|
||||
}{
|
||||
{"", PeriodAll}, // no selector → unchanged legacy behaviour
|
||||
{"?period=day", PeriodDay},
|
||||
{"?period=WEEK", PeriodWeek},
|
||||
{"?period=month", PeriodMonth},
|
||||
{"?period=all", PeriodAll},
|
||||
}
|
||||
for _, c := range cases {
|
||||
r := httptest.NewRequest(http.MethodGet, "/api/stats"+c.query, nil)
|
||||
if got := periodFromQuery(r.URL.Query()); got != c.want {
|
||||
t.Errorf("periodFromQuery(%q) = %q, want %q", c.query, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1038,6 +1038,13 @@
|
||||
recsEnd: "已到最早记录",
|
||||
recsRotated: "审计日志已轮转，已从最新记录重新加载",
|
||||
recsNewest: "回到最新",
|
||||
periodLabel: "统计周期",
|
||||
periodDay: "今日",
|
||||
periodWeek: "本周",
|
||||
periodMonth: "本月",
|
||||
periodAll: "全部",
|
||||
periodTruncated:
|
||||
"统计周期可能不完整（审计日志已轮转），数值为下限。",
|
||||
recsPartial:
|
||||
"部分审计日志无法读取，统计可能不完整；完整历史请导出 CSV",
|
||||
thTokens: "Tokens",
|
||||
@ -1294,6 +1301,13 @@
|
||||
recsEnd: "Reached the oldest record",
|
||||
recsRotated: "The audit log rotated; reloaded from the newest record",
|
||||
recsNewest: "Back to newest",
|
||||
periodLabel: "Reporting period",
|
||||
periodDay: "Today",
|
||||
periodWeek: "This week",
|
||||
periodMonth: "This month",
|
||||
periodAll: "All time",
|
||||
periodTruncated:
|
||||
"This window may be incomplete (audit log rotated); figures are a lower bound.",
|
||||
recsPartial:
|
||||
"Some audit files could not be read, so these totals may be incomplete; export CSV for the full history",
|
||||
thTokens: "Tokens",
|
||||
@ -1558,6 +1572,18 @@
|
||||
return `${p(d.getMonth() + 1)}-${p(d.getDate())} ${p(d.getHours())}:${p(d.getMinutes())}:${p(d.getSeconds())}`;
|
||||
};
|
||||
let statsKeyF = ""; // active key filter for records ('' = all)
|
||||
// Usage reporting window. "all" keeps the legacy since-start totals;
|
||||
// day/week/month are calendar windows (UTC) aggregated from the audit
|
||||
// files server-side. Persisted so a reload keeps the operator's
|
||||
// chosen scale instead of silently snapping back to lifetime totals.
|
||||
let statsPeriod = (() => {
|
||||
try {
|
||||
const v = localStorage.getItem("gw_stats_period");
|
||||
return ["day", "week", "month", "all"].includes(v) ? v : "all";
|
||||
} catch (e) {
|
||||
return "all";
|
||||
}
|
||||
})();
|
||||
|
||||
/* ---------- records: on-demand paging ----------
|
||||
* The records table holds only what is on screen. The first screen comes
|
||||
@ -1683,6 +1709,17 @@
|
||||
return;
|
||||
}
|
||||
pane.innerHTML = `
|
||||
<div class="filter-line" style="margin:0 0 12px">
|
||||
<span class="muted">${t("periodLabel")}</span>
|
||||
<select id="stat-period" onchange="setStatsPeriod(this.value)">
|
||||
<option value="day">${t("periodDay")}</option>
|
||||
<option value="week">${t("periodWeek")}</option>
|
||||
<option value="month">${t("periodMonth")}</option>
|
||||
<option value="all">${t("periodAll")}</option>
|
||||
</select>
|
||||
<span class="grow"></span>
|
||||
<span id="period-note"></span>
|
||||
</div>
|
||||
<div class="kpis" id="kpi-row"><div class="kpi-skeletons" aria-hidden="true">${Array(
|
||||
5,
|
||||
)
|
||||
@ -1839,6 +1876,24 @@
|
||||
(statsKeyF ? "&key=" + encodeURIComponent(statsKeyF) : "");
|
||||
(btn ? btn.closest("#modal-wrap") : null) || closeTopModal();
|
||||
}
|
||||
// setStatsPeriod switches the reporting window. It resets the records
|
||||
// pager rather than keeping the old cursor: the records table is fed
|
||||
// from the lifetime tail regardless of window, and leaving a stale
|
||||
// "next page" cursor from the previous window makes the table and the
|
||||
// KPIs disagree about what is being shown.
|
||||
function setStatsPeriod(v) {
|
||||
if (!["day", "week", "month", "all"].includes(v)) return;
|
||||
statsPeriod = v;
|
||||
try {
|
||||
localStorage.setItem("gw_stats_period", v);
|
||||
} catch (e) {
|
||||
/* private mode: the selector still works for this session */
|
||||
}
|
||||
const sel = $("#stat-period");
|
||||
if (sel) sel.value = v;
|
||||
resetRecords();
|
||||
paintStats();
|
||||
}
|
||||
async function paintStats() {
|
||||
try {
|
||||
// limit=0 -> the server ships one screen of records; the rest is paged
|
||||
@ -1846,6 +1901,8 @@
|
||||
const q =
|
||||
"/api/stats?limit=" +
|
||||
RECS_PAGE +
|
||||
"&period=" +
|
||||
encodeURIComponent(statsPeriod) +
|
||||
(statsKeyF ? "&key=" + encodeURIComponent(statsKeyF) : "");
|
||||
const st = await api(q);
|
||||
const tot = st.total || {};
|
||||
@ -1918,6 +1975,15 @@
|
||||
rpart.innerHTML = st.replay_partial
|
||||
? `<span class="muted" title="${escAttr(t("recsPartial"))}">ⓘ ${esc(t("recsPartial"))}</span>`
|
||||
: "";
|
||||
// Period windows come off the audit files, which rotate. If the
|
||||
// window may be incomplete the numbers are a lower bound, and saying
|
||||
// so next to them is the whole point — a confidently wrong zero for a
|
||||
// rotated-away week is indistinguishable from a quiet week.
|
||||
const pnote = $("#period-note");
|
||||
if (pnote)
|
||||
pnote.innerHTML = st.truncated
|
||||
? `<span class="muted" title="${escAttr(t("periodTruncated"))}">ⓘ ${esc(t("periodTruncated"))}</span>`
|
||||
: "";
|
||||
const rex = $("#rec-exit");
|
||||
if (rex)
|
||||
rex.innerHTML = statsKeyF
|
||||
|
||||
Reference in New Issue
Block a user