mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-10-03 23:54:06 +00:00
fix(billing): 缓存命中统计缺失 + Billing 页空白 + 侧栏图标
三个问题都来自生产实测，不是代码审阅。
## 1. 缓存命中被计费却不被统计
网关确实从上游 usage 提取了 prompt_cache_hit_tokens（审计里能看到
cache_hit_tokens: 270104 / cache_reported: true，占 prompt 的 99.9%），
costFor() 也用它给缓存段定价了 —— 但**没有任何 bucket 记录它**。
结果：一个 99.88% 命中率的网关，报表显示 prompt_tokens 却看不出其中
多少是缓存读，也无从按源/模型/key 看命中率。
每个 bucket 现在多三个字段：
cache_hit_tokens 命中数（按上游上报）
cache_fresh_tokens 未命中的 prompt
cache_reported_reqs 上游确实上报了缓存数的请求数
第三个字段是刻意的：**「零命中」与「上游根本不上报」在命中总量里完全一样**，
而它们在「缓存折扣有没有生效」这个问题上含义相反。没有它就无法区分，
只能猜。
chat.go 的 payload 之前**没有** cache_reported（审计有、插件没有），
所以任何插件侧的缓存统计都只能猜 —— 已补上。
旧 state 文件的 bucket 没有这些字段：Lua 里 nil + number 会抛错，而钩子抛错
会让**该请求完全不记账**（一个统计缺口会变成静默缺口）。add() 里做了回填。
UI 增加 fresh/cache/cache% 三列 + Cache hit rate KPI；未上报的显示 n/r 而不是 0%。
## 2. Billing 页空白：render() 引用了未定义的 s
`render(st)` 里两处 KPI 写成 `s.degraded_reqs`，ReferenceError 让整个渲染
中断，所有表格停在初始的空 innerHTML。症状是「页面加载了但什么都没有」，
而 /api/plugins/billing/state 返回 200 且有真实数据 —— 载荷完全正确，
DOM 是空的。
更糟的是 refresh() 里的 `catch (e) { /* never break the page */ }` 把错误
**静默吞掉**了：网络面板一切正常，页面什么都没有。现在 catch 会
console.error（仍然不抛，装饰性组件不该拖垮宿主页，但必须留痕）。
## 3. 侧栏图标
billing 声明 icon = "💰"，而原生 tab 全是内联 SVG（stroke: currentColor）。
emoji 尺寸不对、不跟随主题。
WebUI 增加 pluginIconHTML：插件图标可以是文本，也可以是内联 SVG。
**SVG 走严格白名单**（tag + 属性都是 allowlist，不是 denylist）——
插件是在运维者浏览器里跑的第三方代码，不能"信任插件"；但也不能直接拒绝
SVG，因为那是唯一能和原生 tab 视觉一致的方式。
用真实 Chromium 验证 12 个用例，全部挡住，包括 foreignObject 里嵌 HTML
命名空间 <img onerror> 这个经典绕过（整体丢弃，所以 img/onerror 也没了）。
★ node 里没有 DOMParser/jsdom，所以没法在单测里跑这个过滤器 —— 用正则近似
会得到一个"测试通过但浏览器里失效"的过滤器，这比没有测试更糟。
顺带修了过滤器的两个真缺陷：输出里嵌套了空 `<svg></svg>`，且 viewBox
是从包装元素读的（永远是 null）而不是插件自己的，所以任何自定义 viewBox
的图标都会丢失。
## 判据（新增 7 项，全部变异验证）
写「注入脚本能否正常执行」这个守卫时我错了四次：
1. 静态扫「已声明的名字」→ 把 HTML 字符串里的 CSS 类名（class/div/td）
全报成未定义
2. 用 CSS 选择器解析器查样式表 → 报样式表本身坏了
3. 只挂 process 的 uncaughtException → 脚本在 IIFE 里异步跑，错误是
unhandledRejection，判据对原 bug 全绿
4. 只查「有没有抛错」→ render() 开头是 `if (!st) return`，传错字段是
**静默 no-op**：不抛、不打日志、不报错，只是页面空白
最终判据是：在 node 里用 DOM stub 真跑一遍，同时要求「无异常」且
「至少写进一个容器」，并监听 console.error。变异验证：还原 s → 红；
render 收到 undefined 字段 → 红。
表头/行列数一致性也有守卫：row() 加了缓存列而表头没加时，表格会整体错位
（cache% 落到 completion 列下）—— 渲染正常、有数据、但要仔细看才发现。
## 生产验证
重启后价目表与累计账完整保留（1.17 亿 prompt tokens）。
新请求缓存统计生效：cache_hit 947,436 / cache_fresh 888，
cache_reported_reqs 7 / 395（其余来自旧 state，正是该字段存在的意义）。
真实浏览器：表格 3 行、KPI 7 项、表头 name/cost/reqs/prompt/fresh/cache/cache%/completion、
SVG 图标 currentColor 渲染、控制台无 billing 错误。391 个测试全绿。
## 另发现一个无关 bug（未修）
首页 stats 图表抛 IndexSizeError: arc 半径为负（-2），在 ui/index.html 的
paintStats 附近。属状态页图表，不在本次范围。
This commit is contained in:
JianFeeeee
@ -98,8 +98,25 @@ plugin.cache_discount = 0.1
|
||||
-- Sorted top-N lists are maintained incrementally rather than re-sorted on
|
||||
-- every request: this hook runs once per request on the hot path, so it does
|
||||
-- map updates only. The sort happens when state is READ.
|
||||
-- CACHE ACCOUNTING (added after production showed the gap):
|
||||
-- the gateway extracts prompt_cache_hit_tokens from upstream usage and puts it
|
||||
-- in the request_end payload, and costFor() already used it to price the cache
|
||||
-- leg — but no bucket recorded it. So a gateway where 99.88% of prompt tokens
|
||||
-- were cache reads showed a prompt_tokens number with no indication of that,
|
||||
-- and there was no way to see cache hit rate per source/model/key at all.
|
||||
--
|
||||
-- cache_hit_tokens hits, as reported by upstream
|
||||
-- cache_fresh_tokens prompt tokens that were NOT cache reads
|
||||
-- cache_reported_reqs requests where upstream gave a cache number at all.
|
||||
-- Kept separate from a zero: "upstream does not report cache usage" and
|
||||
-- "upstream reported zero hits" look identical in a hit total, and they mean
|
||||
-- opposite things when you are trying to work out whether a cache discount is
|
||||
-- doing anything.
|
||||
local function emptyBucket()
|
||||
return { cost = 0, requests = 0, prompt_tokens = 0, completion_tokens = 0, failures = 0 }
|
||||
return {
|
||||
cost = 0, requests = 0, prompt_tokens = 0, completion_tokens = 0, failures = 0,
|
||||
cache_hit_tokens = 0, cache_fresh_tokens = 0, cache_reported_reqs = 0,
|
||||
}
|
||||
end
|
||||
|
||||
plugin.state = {
|
||||
@ -120,12 +137,21 @@ local function bucket(tbl, k)
|
||||
return b
|
||||
end
|
||||
|
||||
local function add(b, cost, prompt, completion, ok)
|
||||
local function add(b, cost, prompt, completion, ok, cacheHit, cacheReported)
|
||||
b.cost = b.cost + cost
|
||||
b.requests = b.requests + 1
|
||||
b.prompt_tokens = b.prompt_tokens + prompt
|
||||
b.completion_tokens = b.completion_tokens + completion
|
||||
if not ok then b.failures = b.failures + 1 end
|
||||
-- Backfill guards a bucket that predates these fields (a state file written
|
||||
-- by an older build, or one restored from disk): nil + number is an error in
|
||||
-- Lua, and a hook that throws stops accounting for that request entirely.
|
||||
if b.cache_hit_tokens == nil then b.cache_hit_tokens = 0 end
|
||||
if b.cache_fresh_tokens == nil then b.cache_fresh_tokens = 0 end
|
||||
if b.cache_reported_reqs == nil then b.cache_reported_reqs = 0 end
|
||||
b.cache_hit_tokens = b.cache_hit_tokens + (cacheHit or 0)
|
||||
b.cache_fresh_tokens = b.cache_fresh_tokens + ((prompt or 0) - (cacheHit or 0))
|
||||
if cacheReported then b.cache_reported_reqs = b.cache_reported_reqs + 1 end
|
||||
end
|
||||
|
||||
-- ---------- pricing ----------
|
||||
@ -393,15 +419,25 @@ function plugin.on_request_end(payload)
|
||||
-- once, whereas the walk may contain several skipped tiers.
|
||||
if payload.degraded then s.degraded_reqs = s.degraded_reqs + 1 end
|
||||
|
||||
add(s.total, cost, prompt, completion, ok)
|
||||
local cacheHit = tonumber(payload.cache_hit_tokens) or 0
|
||||
if cacheHit < 0 then cacheHit = 0 end
|
||||
if cacheHit > prompt then cacheHit = prompt end
|
||||
-- cache_reported is the gateway's own signal that UPSTREAM gave a cache
|
||||
-- number. Without it a source that never reports cache usage is
|
||||
-- indistinguishable from one that always reports zero hits.
|
||||
local cacheReported = payload.cache_reported and true or false
|
||||
local C = cacheHit
|
||||
local R = cacheReported
|
||||
|
||||
add(s.total, cost, prompt, completion, ok, C, R)
|
||||
if payload.source ~= nil and payload.source ~= "" then
|
||||
add(bucket(s.by_source, payload.source), cost, prompt, completion, ok)
|
||||
add(bucket(s.by_source, payload.source), cost, prompt, completion, ok, C, R)
|
||||
end
|
||||
if payload.model ~= nil and payload.model ~= "" then
|
||||
add(bucket(s.by_model, payload.model), cost, prompt, completion, ok)
|
||||
add(bucket(s.by_model, payload.model), cost, prompt, completion, ok, C, R)
|
||||
end
|
||||
if payload.key ~= nil and payload.key ~= "" then
|
||||
add(bucket(s.by_key, payload.key), cost, prompt, completion, ok)
|
||||
add(bucket(s.by_key, payload.key), cost, prompt, completion, ok, C, R)
|
||||
end
|
||||
|
||||
-- Daily rollup, so the dashboard can draw a trend without the browser
|
||||
@ -410,7 +446,7 @@ function plugin.on_request_end(payload)
|
||||
local ts = payload.time
|
||||
if ts ~= nil and ts > 0 then
|
||||
if ts > 1000000000000 then ts = ts / 1000 end -- kernel sends unix MILLIseconds
|
||||
add(bucket(s.by_day, dayKey(ts)), cost, prompt, completion, ok)
|
||||
add(bucket(s.by_day, dayKey(ts)), cost, prompt, completion, ok, C, R)
|
||||
end
|
||||
return nil -- last stage: nobody downstream would read a return value
|
||||
end
|
||||
@ -423,7 +459,7 @@ plugin.ui = {
|
||||
page = {
|
||||
page_id = "billing",
|
||||
title = "Billing",
|
||||
icon = "💰",
|
||||
icon = [==[<svg viewBox="0 0 24 24"><circle cx="12" cy="12" r="9"/><path d="M14.5 9.5a3 3 0 0 0-2.5-1.3c-1.4 0-2.4.7-2.4 1.8 0 2.6 5.2 1.4 5.2 4 0 1.1-1 1.8-2.5 1.8-1.1 0-2.1-.4-2.7-1.2"/><path d="M12 6.4v11.2"/></svg>]==],
|
||||
order = 40,
|
||||
mount = [==[
|
||||
<div id="billing-root" style="padding:16px">
|
||||
@ -464,9 +500,25 @@ plugin.ui = {
|
||||
return { "&": "&", "<": "<", ">": ">", '"': """ }[c];
|
||||
});
|
||||
}
|
||||
// Cache hit rate, with the reporting caveat made visible.
|
||||
//
|
||||
// A bucket whose upstream never reports cache usage would render as "0%" from
|
||||
// a 0/0 and read as "the cache is not working", when the truth is "this
|
||||
// provider does not tell us". "n/r" keeps those apart.
|
||||
function cacheRate(b) {
|
||||
var prompt = b.prompt_tokens || 0;
|
||||
var hit = b.cache_hit_tokens || 0;
|
||||
if (!prompt) return "\u2014";
|
||||
if (!b.cache_reported_reqs) return "n/r";
|
||||
return ((hit / prompt) * 100).toFixed(1) + "%";
|
||||
}
|
||||
function row(name, b, cur) {
|
||||
var fresh = (b.cache_fresh_tokens === undefined) ? (b.prompt_tokens || 0) : b.cache_fresh_tokens;
|
||||
return "<tr><td><b>" + esc(name) + "</b></td><td>" + money(b.cost, cur) +
|
||||
"</td><td>" + (b.requests || 0) + "</td><td>" + (b.prompt_tokens || 0) +
|
||||
"</td><td>" + fresh +
|
||||
"</td><td>" + (b.cache_hit_tokens || 0) +
|
||||
"</td><td>" + esc(cacheRate(b)) +
|
||||
"</td><td>" + (b.completion_tokens || 0) + "</td></tr>";
|
||||
}
|
||||
function tableFor(el, obj, cur, empty) {
|
||||
@ -475,7 +527,8 @@ plugin.ui = {
|
||||
keys.sort(function (a, b) { return (obj[b].cost || 0) - (obj[a].cost || 0); });
|
||||
var h = "<table style='width:100%;border-collapse:collapse;font-size:13px'>" +
|
||||
"<tr style='text-align:left;opacity:.65'><th>name</th><th>cost</th><th>reqs</th>" +
|
||||
"<th>prompt</th><th>completion</th></tr>";
|
||||
"<th>prompt</th><th>fresh</th><th>cache</th><th>cache%</th>" +
|
||||
"<th>completion</th></tr>";
|
||||
for (var i = 0; i < keys.length; i++) {
|
||||
var k = keys[i];
|
||||
h += "<tr style='border-top:1px solid rgba(120,90,150,.14)'>" + row(k, obj[k], cur) + "</tr>";
|
||||
@ -489,8 +542,8 @@ plugin.ui = {
|
||||
document.getElementById("billing-kpis").innerHTML = [
|
||||
["Total", money(t.cost, cur)],
|
||||
["Requests", t.requests || 0],
|
||||
["Degraded", s.degraded_reqs || 0],
|
||||
["Unpriced", s.unpriced_reqs || 0],
|
||||
["Degraded", st.degraded_reqs || 0],
|
||||
["Unpriced", st.unpriced_reqs || 0],
|
||||
["Prompt tokens", t.prompt_tokens || 0],
|
||||
["Completion tokens", t.completion_tokens || 0],
|
||||
["Failures", t.failures || 0]
|
||||
@ -510,7 +563,14 @@ plugin.ui = {
|
||||
if (!r.ok) return;
|
||||
var j = await r.json();
|
||||
render(j.state);
|
||||
} catch (e) { /* the pane is optional decoration; never break the page */ }
|
||||
} catch (e) {
|
||||
// Swallowing this is what made the production bug invisible: render() threw
|
||||
// a ReferenceError on an undefined `s`, the catch ate it, every table kept
|
||||
// its empty placeholder, and the page looked fine in the network tab while
|
||||
// showing nothing. Still must not THROW (the pane is decoration and must
|
||||
// never break the host page) — but it must leave a trace.
|
||||
if (window.console && console.error) console.error("[billing] render failed", e);
|
||||
}
|
||||
}
|
||||
window.__billingRefresh = refresh;
|
||||
refresh();
|
||||
@ -560,7 +620,12 @@ plugin.ui = {
|
||||
}
|
||||
document.getElementById("billing-status-sub").textContent =
|
||||
(st.total.requests || 0) + " requests" + (parts.length ? " · top: " + parts.join(" · ") : "");
|
||||
} catch (e) { /* decoration only */ }
|
||||
} catch (e) {
|
||||
// Same reasoning as the Billing page: decoration must never break the
|
||||
// host page, but a silent catch turns a broken widget into "the plugin
|
||||
// just doesn't show anything" with no way to tell why.
|
||||
if (window.console && console.error) console.error("[billing] status tile refresh failed", e);
|
||||
}
|
||||
}
|
||||
if (window.pluginAPI && pluginAPI.onTabShown) pluginAPI.onTabShown(tick);
|
||||
tick();
|
||||
|
||||
Reference in New Issue
Block a user