Compare commits

...

29 Commits

Author SHA1 Message Date
ba01da937b fix(packaging): 打包带上计费插件,并修复一个让所有产物校验形同虚设的缺陷
发 v1.8.0 时发现包里没有 billing.lua —— 而 v1.8.0 的主打特性就是计费插件,
发布说明明写「随核心发布的示例插件(billing)」。

功能本身没坏:internal/lua/vm.go 用 //go:embed plugins/*.lua 把插件编进二进制,
writeBundledPlugins 在 plugin_dir 不存在时把它写出来。实测确认(用正确格式的
配置):全新 plugin_dir 启动后自动生成 billing.lua 70031 字节,插件正常加载。
所以这是产物内容与发布说明不符,不是功能缺失 —— 运维解包检查时看不到它,
只能等首次启动后才发现。

三处改动:

1) tar.gz 现在带 plugins/。让运维能在安装前读它、改它,而不是启动后才知道。

2) 修复 dpkg-deb 那行。`dpkg-deb -I "$DIST"/llmsproxy_*.deb` 的 glob 展开成三个
   .deb(dist 里堆着 1.5.9 / 1.6.0 / 本次产物),dpkg-deb 只认第一个归档,其余
   参数被当成 control component 名,退出码 2。脚本是 set -euo pipefail,于是
   在这里直接终止 —— 「done」从未打印,我加在后面的产物内容校验从来没有执行过,
   真正跑过的只有上面那道 100KB 大小下限。改为取最新的那个 deb。
   这个缺陷早于本次改动,是被它暴露出来的。

3) 新增产物内容校验:tar.gz 必须含 llmsproxy / config.example.yaml /
   llmsproxy.service / plugins/billing.lua,adapters 至少 10 个 .lua。
   大小下限抓不到这个问题 —— 少 70KB 仍然远超 100KB,而首次启动自动 seed 的
   行为会在运行时把它藏起来,运维看到插件正常工作就以为包是完整的。
   目录按内容而非名字匹配:tar 列出的是 …/adapters/openai.lua,不存在裸
   …/adapters 条目,第一版按名字 grep 把完整包判成了坏包。

两个变异验证:去掉 plugins 拷贝 → 「archive is missing plugins/billing.lua」
退出 1;只留 1 个 adapter → 「archive has only 1 adapters」退出 1。
2026-10-02 23:31:29 +08:00
c8edea23f5 chore(version): 1.7.6 -> 1.8.0
打 v1.8.0 时发现清单里的版本号还停在 v1.7.6:main 上有一份 1.8.0 的
(4b37e1a),发布线没有,于是 merge 时按前者取值、tag 指向发布线,产物自相矛盾
(tag v1.8.0 配一份声明 1.7.6 的 package.json)。

功能上无影响:grep 确认没有任何代码读 package.json 的 version,llmsproxy
也没有 --version(只有 -check / -config / -show-secrets),所以这只是清单
自述与 tag 不一致,不影响任何回显路径。但发版纪律要求「版本号正确、与 Tag
一致」,发布前把它对齐。

三处:package.json 一处、package-lock.json 两处(顶层与 packages."")。
api.go 注释里引用的 v1.7.6 未动——那是在说明一条规则的出处,不是版本声明。
改后用 json.load 验证两个文件仍合法且两处版本一致。
2026-10-02 20:13:20 +08:00
689c8cb383 test(deploy): 给 deploy.sh 的备份裁剪与回滚点补判据
发版前核验:deploy.sh 是唯一没有自动覆盖的发布关键脚本,而它新增的
prune_adapter_backups 会在 /etc/llmsproxy 下删目录。deploy.sh 本身路径硬编码
(会 mv 覆盖 /usr/local/bin/llmsproxy 与 config.yaml 并重启服务),不能在生产
试跑,所以把函数抽到临时目录实测。

两条判据:
- TestDeployPruneAdapterBackups:在 t.TempDir() 里造 12 个规范备份 + 3 个
  不规则目录,抽取函数本体(只重定向 base,逻辑一字不动)执行,断言最旧被删、
  最新保留、不规则目录绝不被删。
- TestDeployBacksUpTheLiveFileBeforeOverwriting:锁住「先备份线上文件再安装」
  这条纪律,并断言备份语句出现在安装语句之前。

写判据时踩的三个坑(都是判据自身的错,不是被测代码的错):
1. 备份名位数。守卫是 ^[0-9]{14}$,第一版造了 13 位和 15 位的名字,全部被
   「名字不规范」跳过,看起来像「什么都没删」的假通过。这和记忆里线上那次
   15 位时间戳的坑是同一个。
2. KEEP_ADAPTER_BACKUPS=5 写在了 shim 里,等于覆盖被测脚本自己的配置——把
   deploy.sh 里的 KEEP 改成 0 判据照样通过。改为从被测脚本正则读取该值。
3. 存活数期望写错。KEEP 计入不规则目录的数量却永不删除它们,所以实际存活
   是 KEEP + 不规则目录数(实测 7)。这是「宁可多留也不删人工目录」的正确
   取舍,判据改为断言这个语义。

变异验证 3/3 被捕获:去掉两处名字正则(误删人工目录)、KEEP 改 0、备份来源
换成新文件。第三个变异第一版用 sed 只改到第一个匹配点、漏掉真正的删除点,
误报成「判据漏放」——是变异脚本没改到位。
2026-10-02 15:44:50 +08:00
400b6c35a5 fix(billing-ui): 规则表列宽锁定,消除表头/按钮重叠
全部靠截图 + 几何测量发现,DOM 断言当时全绿:

- table-layout:fixed 把 954px 均分成 9×106px,写在 <td> 上的每列宽度全部失效 →
  相邻表头叠在一起,源 URL 输入框被压成 "https:"。
- 把宽度改到 <th> 并去掉 fixed → 浏览器改按内容算,<th> 宽度又被忽略,币种列
  仍塌到 48px(输入框只剩 24px,装不下 "USD")。
- 恢复 fixed + 宽度移到 <colgroup>:fixed 下只有首行宽度生效,而「按模型价格」
  格宽度随哪条规则最宽而变(实测 352px vs 206px),整行布局取决于恰好哪条规则
  最宽。colgroup 一次性锁死九列,所有行一致。
- 表头双语长文案(峰段星期 UTC 1=周一)超出列宽 → nowrap + ellipsis,完整文案
  留在 title。
- 删除列 58px 装不下「删除」按钮(折成两行、按钮高过整行)→ 74px,从峰段倍数
  列匀出。
- 价目行 80+96+96+38≈330px 挤在 262px 格子里,× 按钮越界 78px 压在隔壁「删除」
  按钮上 → 输入框改 flex:1 1 …/min-width:0 可收缩,× 保持 flex:0 0 auto。
- 顺带修上轮引入的回归:给 modelBlock 的 flex 换行后模型名不再被截断。

判据 TestRuleTableLocksColumnWidthsInAColgroup + 3 个变异(去掉 min-width:0 /
改列宽数组 / 去掉 table-layout:fixed)全部被捕获。第三个变异第一版漏放——判据用
strings.Contains(js, "table-layout:fixed") 匹配到了上方三行的解释性注释,改成匹配
style='…' 字面量后才真正失败。
2026-10-02 15:29:29 +08:00
c19b8e6394 fix(scheduler): AUTO 遇空内容响应降级到下一个 slot(客户端曾报 "no content")
生产故障:pi 客户端报 `model "AUTO" returned a completed response with no content`,
重试延迟 8 秒。实测 AUTO 20 次有 2 次返回空 content,全部是 claude-opus-4-8。

根因(直连上游抓包确认):思考型模型先吐 reasoning_content,max_tokens 小到
思考阶段就把预算用完时,上游返回 200 / finish_reason=length,28 个 chunk 全是
reasoning_content、content 一片空白。runTier 只看 err == nil 就当成功返回,
客户端拿到一个空响应。

修复:
- resultIsEmpty:非流式路径把「无 content、无 tool_calls、无 image」的响应当作
  slot 失败继续降级。注意 ReasoningContent 不算内容——客户端要的是文本,为
  另一个模型的思考阶段扣住请求比降级更糟。
- peekStream:流式路径在出现首个真实内容前缓冲 reasoning 前导,流结束仍无内容
  则回报空结果,让 chainDrive 换 slot。缓冲只覆盖思考前导,拿到内容后立即
  转发。tool_calls delta 算内容,agent 回合不会被误判。
- 3 条判据 + 3 个变异(恒 false / 恒 true / reasoning 算内容)全部被捕获。

同时修三个 WebUI 布局缺陷(都靠截图而非 DOM 断言发现):
- 插件侧栏项只渲染图标没有标题:btn.innerHTML 只塞 pluginIconHTML(pg.icon),
  与原生页的「图标 + <span>标题</span>」不一致,侧栏是一排无名图标。
- 计费维度表 8 列挤在 465px 卡片里:table-layout:fixed 把每列压到 62px,
  23/80 个单元格溢出、数字互相重叠。改为 6 列(token 细分合并为
  「输入(新鲜+缓存)」,细分进 title)+ table-layout:auto,实测 0/60 溢出。
- 数字列 word-break:break-all 让每个字符独占一行(USD 0.56 竖排成 U/S/D),
  改 nowrap + 容器横向滚动。
2026-10-02 15:10:01 +08:00
09cb215208 fix(billing): 通配符 token 规则不再把付费源标成「已定价 0」+ 迁移现有价格为规则
## 迁移后立刻发现的设计缺陷

把 .billing.state.json 里的现行价格迁成 URL 规则后(免费源 4 个 + 一个 `*`
模型价目表),注入插件的价格表里出现了 21 个 source 条目,全部 {0,0,0}——
包括 commandcode、alittokenplan 这些**付费**源。

原因在 Compile 的 token 分支:规则匹配到的每个 source 都会补一个 {0,0,0}
条目,注释写的理由是「否则该 URL 上没列进 rule.Models 的模型会掉回默认 0」。
但 `*` 匹配所有 source,于是这条为「特定 URL」设计的兜底变成了「给所有源盖
已定价 0 的章」。

后果正是插件本身要防的那个失效:priceFor 只要 source 有条目就置 priced=true,
priced=true 的请求不进 unpriced_reqs。所以付费源上未列出的模型全部记成
「已定价 $0」,账单看着加得起来,实际静默少算——commandcode 的 DeepSeek
正是这种情况(54k 请求的 deepseek/deepseek-v4.1-flash 在旧 sidecar 里
压根没定价)。

## 修法

通配符规则不再补 source 条目。`*` 的语义是模型目录:「这几个模型 id wherever
从哪来都这个价」,它不是「这些源都免费」。模型查表 p.models[payload.model]
本身就会把列出的模型标为已定价,所以通配符去掉 source 条目不丢任何东西。

特定 URL 的 token 规则保留 source 条目(它确实为该 URL 声明了定价)。

## 判据(2 条 + 3 个变异)

- 通配符不得标记任何未显式声明的源为已定价(点名 commandcode 场景)
- 特定 URL 规则仍必须标记自己的源(反向约束,防止把兜底整体删掉)

变异 M1(通配符也标记)、M2(只有通配符标记)、M3d(模型价发布到错 key)
均被捕获。M3 前两版「漏放」是我的变异脚本改坏了编译(unused variable),
grep 匹配不到 "--- FAIL"——和之前一样的工具陷阱,判据本身没问题。

## 线上状态

config.yaml 已写入 billing 段(active: current,5 条规则,无 warning),
生效价格表只把 4 个免费源标为 priced 0,付费源保持未标记⇒其未列出模型会
正确计入 unpriced 而非静默 0 元。

真实请求验证:3 个 deepseek-v4.1-flash 流式请求 200,计费从 0.56679
涨到 0.56702,走的是新规则注入的价格。全量测试连跑 5 次全绿。
2026-10-02 14:21:40 +08:00
d9652f479a fix(plugins): 插件 Lua 报错不再拖垮网关(生产事故修复)
## 事故

13:37 部署后线上 6 次 SIGSEGV 崩溃循环,8081 完全不可用,用户报大量
connect error。崩溃点固定在 internal/lua/plugins.go:invoke → L.Call →
golua StackTrace 里的 lua_getinfo。

## 根因(不是并发/GC/锁)

golua 的 callEx 在**任何** pcall 失败后无条件执行 L.StackTrace(),而
StackTrace 调 lua_getinfo,这个 LuaJIT 构建在栈够深时(带 AUTO 链轨迹的
request_end payload 正好够深)直接段错误。这是 C 层信号,Go 无法 recover,
所以一个插件的脚本错误就能带走整个进程和所有在途请求。

触发错误来自我上一轮加的 billing 日级维度:

    add(bucket(bucket(bucket(s.by_day_src, dk), payload.source)), ...)

三个 bucket( 只对应两个 ),最外层 bucket() 只收到一个参数,k=nil,于是
billing.lua:141 `tbl[k] = b` 抛 "table index is nil",**每个请求都抛**。

同时还有第二个 bug:中间层用了 bucket()(返回 emptyBucket,含 cost/requests
字段)当作嵌套容器,结构也是错的。改为 dayMap() 返回纯表。

## 修法

1. billing.lua:修正括号,多层容器改用 dayMap()。
2. **pcall 守卫**(真正的架构修复):在 setupGlobals 里注册
   __llmsproxy_call_hook,钩子改为经它调用。

       function __llmsproxy_call_hook(fn, payload)
         local ok, res = pcall(fn, payload)
         if not ok then return nil, tostring(res) end
         return res, nil
       end

   Lua 侧 pcall 在 golua 看到非零 pcall 状态之前就拦下错误,C 栈回溯路径
   永远进不去。错误变成普通返回值 (nil, msg),Go 侧记进 hook_errors 并跳过
   ——"插件出错不影响请求转发"这条承诺对脚本错误也终于成立,而不只是对 Go panic。

## 这同时修掉了那个查了很久的间歇崩溃

同一个机制解释了此前 8/20 复现、却查不出根因的 SIGSEGV(怀疑过 janitor 竞态、
GC、LuaJIT 全局状态、VM 释放时序,全部排除)。实测对比:

  TestBillingPrecedence   修复前 8/20 崩溃 → 修复后 0/20
  并发建 16 个 VM 的探针   修复前 3/3  崩溃 → 修复后 0/6
  全量 ./...              连跑 5 次全绿

那些崩溃本来就是一个 Lua 钩子错误在栈深时炸掉 StackTrace,时机随机所以看着
像并发问题。

## 判据

TestHookThatRaisesDoesNotCrashTheProcess:装一个每请求必崩的插件,连打 50 次,
断言进程存活 + 错误被记录 + 同状态里健康的 billing 插件照常工作。
3 个变异(守卫不 pcall / 守卫名写错 / 守卫未注册)全部被捕获,其中第一个直接
让 SIGSEGV 重现,说明守卫就是唯一防线。

## 线上验证

往生产插件目录放一个每请求必然报错的插件,连打 30 个真实流式请求:

  30× HTTP 200,SIGSEGV 0 次
  hook_errors 记录 count=44 且指名 zbroken-test(可观测)
  billing 照常累计(2999 请求 / $0.5668)

测试插件已移除。

回滚点:/usr/local/bin/llmsproxy.bak-real-<TS>、billing.lua.bak-real-<TS>。
2026-10-02 14:07:44 +08:00
fe0764e375 feat(billing): 计费规则可在线增删改,真实落盘并注入插件
规则此前只能写在 config.yaml 里,重启才生效。现在 admin 可通过 API 增删改,
规则写回同一份 config.yaml、重新编译、注入 billing 插件,立即生效。

## 为什么单独开一套端点

	GET/POST/PUT/DELETE /api/plugins/billing/rules

价格虽然存在插件 state 里,但它是**可评审的配置**,不是用户数据。走通用
/state 会让任意 admin key 顺手把累计账目一起重置。这里服务端掌管形状:
校验 → 落盘 → 重编译 → 注入,运维只编辑规则,插件只存数字,两者永不在同一
个 payload 里混。

"运维输入什么,config.yaml 就存什么"是刻意的:下周发现的计费错误,必须能
追溯到一个可评审的文件,而不是插件 sidecar 里的一坨 blob。

## 路由必须前置拦截

/api/plugins/billing/rules 会被 /api/plugins/ 通配路由吞掉并 404,必须在
handlePluginsAPI 之前判断。

## 两个真实缺陷(判据抓到的,不是想出来的)

1. **保存顺序反了**:先 cfg.Save() 再赋值 cfg.BillingDSL,于是每次编辑都
   "成功",写回的配置里却没有 billing 段——运维的编辑在重启后消失,而 API
   响应里什么异常都没有。现在先赋值、先编译(编译失败则整体回滚,不留半应用
   状态)、最后落盘。
2. **GET /state 不返回生效价格**:编辑器无法显示当前真正在用的价目表,只能从
   配置重建——而配置可能早已与实际漂移。新增 Plugins.Prices(),编辑页显示的
   就是计费真正在用的那张表。

## 宽松编译

Compile 启动时对"URL 匹配不到任何 source"直接报错是对的(静默不计费更糟)。
但编辑器要允许存草稿:正在新建的源、正在改的 URL 不该把人堵死。新增
CompileOpts(lenient):宽松模式下该规则**留在配置里**(可评审、可恢复),
只是不进编译结果,并由 API 返回 warning 明确报出来。

## 判据(5 条 + 9 个变异)

CRUD、同 URL 重复添加必须替换而非追加(两条规则会因"先匹配先生效"而让第一条
静默失效)、未知 URL 必须警告、非法规则被拒且不落盘、admin 限制、YAML 往返
不丢价格字符串、注入的价格必须是每 token 量级(防 per-million/per-token 差
1e6 倍)。

变异验证抓出判据两处无效断言:删掉落盘、删掉注入,GET 响应都照样回显内存里
的规则,判据全绿。补了「重读磁盘配置」和「读插件实际生效价格」两条才抓住。

过程中还发现一个测试工具自身的坑:某个变异改法导致 Go 编译失败
(declared and not used),grep 匹配不到 "--- FAIL",于是被我误读成"判据漏放"。
改用可编译的变异写法后确认该变异确实被捕获。**判据报错先怀疑判据和工具。**
2026-10-02 12:55:54 +08:00
d0ddc9754d feat(billing): 计费页支持按日/周/月/全部,与统计页同口径
宿主统计页已支持周期,计费页还是终身累计 —— 同一个问题在两个页面重复出现,
且两个页面口径不一致本身就是错。

## 为什么要补日级维度明细

原来只有 total 和 by_day 带时间维度,by_source/by_model/by_key 是终身累计。
只改 total 的话,页面会显示"今日开销 $0.05",下面三张表还是全量数据 ——
数字对不上,而这正是周期视图要消除的错配。所以在写入时按天折叠成本
(by_day_src/model/key)。成本在写入时就已定价,浏览器只做求和,不重新
定价,显示金额不会与持久化金额漂移。

按天为键的表不随流量增长(一年 365 项/维度),所以不设裁剪。

## 口径

UTC,与网关 dayKey 和 /api/stats 的周期窗口一致 —— 峰谷小时不会在费用
视图和用量视图落到不同一天。周为 ISO 周(周一起)。降级/未定价没有日级
计数,周期视图下**隐藏**这两张卡而不是显示终身值,那正是要消除的错配。

## 判据(7 条 + 6 个变异)

判据断言**渲染后的 DOM**,不走 IIFE 内部函数:早先版本加了测试钩子去直接
调 daysInWindow/rescale,结果 harness 里的假 Date 先后两次出问题(整体替换
构造器破坏 toISOString;子类化导致 getUTCDay 返回 NaN),症状都是
"Invalid time value",看起来完全像产品 bug。

变异验证抓到判据本身的缺陷,值得一提:
- 捕获「总账不累加」和「周起点改周日」两个变异时**全部判据仍绿**——因为
  我断言的是 by_source 表格,而它由维度表独立算出,跟 total 无关。
- 补了 KPI 断言后又漏放「维度折叠不求和」——"页面里存在 60" 太弱,
  60 同时出现在 KPI 和 token 列里。改成锚定 srcA 自己的金额单元格
  (USD 60.0000),last-day-only 会是 30,才抓得住。

判据报错时先怀疑判据——这次三次都是判据的问题。
2026-10-02 12:41:35 +08:00
c241a19b51 feat(stats): 用量按日/周/月/全部统计(审计文件聚合)
统计页原来只有一个视图——进程启动以来的累计。早上没人和一整周没人看起来
一模一样。加日/周/月/总四个周期。

## 口径与实现

周期视图必须走审计文件,不能走内存聚合:内存 byModel/byKey 等是终身累计,
而 recs 环形缓冲只有 500 条(defaultRingSize)。读环会把任何超过几百个
请求的周期悄悄少算——这正是要消除的那类错数。

- 日/周/月 = UTC 日历窗口(今日 / ISO 周周一 00:00 / 本月 1 日)。
  刻意不用滚动 24h:滚动窗口会让"今天"和"最近一天"边界不同,同一个数字
  随查看时刻在两张卡片间跳。UTC 也和 billing 的峰段计算同口径,峰谷小时
  不会在费用视图和用量视图里落到不同一天。
- 全部 = 复用现有 Snapshot(内存聚合),无审计文件时依然可用。
- 时间桶:日→每小时(今天内部的尖峰要看得见),周/月→每天(否则一周是
  7×24 个点、一个月 31×24)。全部视图无时间线(终身总量没有有意义的
  时间轴,硬画 500 个滚动小时点是另一种撒谎)。
- key 过滤在所有维度生效;非法 period 返回 400 而不是静默回落"全部"——
  书签里的手误应当报错,而不是悄悄换成终身数字。

## 判据(10 条 + 8 个变异全部被捕获)

窗口边界(含"周日必须回到上一个周一"这个 Go Weekday() 陷阱)、旧记录不
计入、维度独立聚合且 by_model 求和等于 total、日桶按小时且有序、key 隔离、
全部视图走终身、空窗口标记 truncated、period 校验、query 解析。

变异验证时 by_status 假绿了一次:禁用状态码聚合后判据全过,查下去是我
**根本没测 by_status**(零覆盖)。补 TestPeriodStatusDimension 后该变异
立即被捕获。判据报假问题时,先怀疑判据——这次确实是我错了。

## 真实流量核对

生产审计文件手算 vs 后端(含轮转文件):
  day   手算 3559 / 后端 3532
  week  手算 43240 / 后端 35034
  month 手算 13696 / 后端 13670
差异是核对快照与请求之间的新流量,量级一致。

一个必须说明的发现:审计文件里混着两种记录 —— Req(type/model/
prompt_tokens)和访问日志(lat_ms/status/path)。46026 行里 33450 行是
访问日志,Go 侧按 r.Type=="" 跳过。这不是 bug(CSV 导出同样如此),但
意味着任何按行数手算都必须过滤,否则会差一个数量级。

CDP 实测四周期切换:reqs 3,571 / 35,075 / 13,710 / 196,712,与后端一致,
无控制台错误,localStorage 持久化生效。
2026-10-02 12:26:26 +08:00
d1da40e493 fix(billing): 页面文字溢出卡片 + 数字千分位
用户报 billing 页"文字超出展示框"。真因是三处叠加,都不是文案问题:

1. KPI 卡是 CSS grid,grid item 默认 min-width:auto。2e8 级的 prompt
   tokens(实测 270,149,209)无法收缩,于是把 grid 轨道撑出容器 → 整页横向
   溢出。修法是 min-width:0 + overflow:hidden(不给 min-width:0 的话,
   grid 子项永远不肯收缩,这是 grid 最常见的溢出坑)。
2. 表格写死 min-width:560px,窄视口下必然溢出。改 width:100% +
   table-layout:fixed,列宽由布局分配而不是由内容撑开。
3. 长名称(deepseek-v4.1-flash 这类模型 id)撑宽单元格。名称列改 ellipsis +
   title 悬停看全名;数字列 word-break:break-all 在列宽内换行。

顺手:所有 token/请求数走 toLocaleString 千分位。原始 9 位数字读起来要数
零位数,分组后 270,149,209 一眼可读,也顺带缩短了字符串宽度。

CDP 实测(820px 窄视口,逼出溢出条件):
  main/body 横向溢出 = no
  table=338 < card=366(修复前 min-width:560 必然 > 366)
  KPI 输入 tokens = 270,149,209
  billing 页无控制台错误

残留(不影响布局):表头 TH 在固定布局下 48>42 轻微超出自身格,因为
word-break 不拆单个长词;表格整体仍在容器内,未产生页面滚动。

全量测试全绿(8 包)。
2026-10-02 12:16:40 +08:00
aa10ee8c27 fix(billing): 插件页不可达(真·空白根因)+ i18n + 溢出 + 状态页 tile
## ★ 用户报告「Billing 页还是空白」—— 上一轮的验证有漏洞
上一轮我用 goTab('billing') 直接调用验证,显示"有数据、无错误"就下了结论。
但用户是**点侧栏按钮**。真实点击路径走 goTab,而 goTab 只遍历硬编码的 TABS
常量来切换 `hidden` 类 —— 插件页不在 TABS 里,所以 #tab-billing 的 hidden
**永远不会被移除**。内容一直躺在 DOM 里(KPI/表格都填好了),只是不可见。

这个 bug 没有任何报错:注入正确、数据正确、API 200,唯一的问题是宿主页的
路由逻辑没把插件页纳入。而它是上一轮「TABS 收敛为单一常量」时留下的:
收敛让三处共用一个常量,却没让插件页进入它。

修法:goTab 同时遍历 PLUGIN_PAGES。PLUGIN_PAGES 从 const 改为 var 并**提前到
goTab 之前声明** —— const 在文件后部声明的话,goTab 的读取落在 TDZ 里,
第一次点击插件页就会抛 ReferenceError(同类问题这个文件里已是第二次)。

判据 TestPluginPagesAreReachableByGoTab 锁两件事:goTab 遍历插件页集合 +
声明在使用之前。变异验证:删掉遍历 → 红;var 改回 const → 红。

## 插件 UI 不跟随多语言
宿主的 applyI18n/data-i 只覆盖**宿主渲染的标记**;插件注入的 HTML 对它不可见,
所以整个 UI 切中文时 Billing 页还是英文。

pluginAPI 增加 lang(getter,实时值)与 onLangChange(切换回调)。
billing 页所有文案改走双语字典:KPI、表头(fresh/cache/cache%)、区块标题
(占位后由脚本填)、空态、状态页 tile 标签。切换时立即重渲染标题,
不用等下一次 fetch。

## 部分页面超出 UI 区域
#main 只有 overflow-y,插件页内容(8 列表格 min-width、长字符串)会横向撑破。
两层修:插件 pane 统一 min-width:0/max-width:100%/overflow-x:auto(第三方
任意 HTML 的兜底,与原生 pane 一致);billing 的宽表格在自身容器内滚动。

## 状态页 tile 的 TypeError(每次重绘都报)
tile 的 tick() 在 await 之后直接 getElementById(...).textContent = ...,
但状态页每次刷新都整体重建 pane,元素可能已不存在 → null 属性赋值。
await 之后重新取元素并判空。

## 顺手补的缺口
上一轮加了缓存表格列,但 KPI 卡片漏了(那次替换 assert 失败后重试只重做了
表格)—— 缓存命中率在表格里有、KPI 里没有。本次补上。

## 验证
真实浏览器(禁缓存、真实点击侧栏按钮):pane 可见、KPI 9 项、表头双语、
语言双向切换正确(Per source ⇄ 按源)、无水平溢出、无 billing 控制台错误。
生产数据:Total USD 0.566798 / 732 请求 / 降级 231 / 2.09 亿 prompt tokens。
387+ 测试全绿。

## DSL(进行中,未完)
config.BillingDSL(active + profiles + rules,rule 按 url 匹配 mode=free/
token/subscription/unpriced)与 internal/billing.Compile(url 规则 → 插件
prices 表,含峰谷窗口的形状编译 —— 之前手写 JSON 两次弄错的正是这个形状)
已落地并通过校验/编译;core 启动接线已写。profile 切换 API 与 WebUI 选择器
未做,生产 config.yaml 也尚未写 billing 段 —— 下一轮继续。
2026-10-02 11:47:24 +08:00
fbdf0dea10 fix(billing): 缓存命中统计缺失 + Billing 页空白 + 侧栏图标
三个问题都来自生产实测,不是代码审阅。

## 1. 缓存命中被计费却不被统计
网关确实从上游 usage 提取了 prompt_cache_hit_tokens(审计里能看到
cache_hit_tokens: 270104 / cache_reported: true,占 prompt 的 99.9%),
costFor() 也用它给缓存段定价了 —— 但**没有任何 bucket 记录它**。
结果:一个 99.88% 命中率的网关,报表显示 prompt_tokens 却看不出其中
多少是缓存读,也无从按源/模型/key 看命中率。

每个 bucket 现在多三个字段:
  cache_hit_tokens    命中数(按上游上报)
  cache_fresh_tokens  未命中的 prompt
  cache_reported_reqs 上游确实上报了缓存数的请求数

第三个字段是刻意的:**「零命中」与「上游根本不上报」在命中总量里完全一样**,
而它们在「缓存折扣有没有生效」这个问题上含义相反。没有它就无法区分,
只能猜。

chat.go 的 payload 之前**没有** cache_reported(审计有、插件没有),
所以任何插件侧的缓存统计都只能猜 —— 已补上。

旧 state 文件的 bucket 没有这些字段:Lua 里 nil + number 会抛错,而钩子抛错
会让**该请求完全不记账**(一个统计缺口会变成静默缺口)。add() 里做了回填。

UI 增加 fresh/cache/cache% 三列 + Cache hit rate KPI;未上报的显示 n/r 而不是 0%。

## 2. Billing 页空白:render() 引用了未定义的 s
`render(st)` 里两处 KPI 写成 `s.degraded_reqs`,ReferenceError 让整个渲染
中断,所有表格停在初始的空 innerHTML。症状是「页面加载了但什么都没有」,
而 /api/plugins/billing/state 返回 200 且有真实数据 —— 载荷完全正确,
DOM 是空的。

更糟的是 refresh() 里的 `catch (e) { /* never break the page */ }` 把错误
**静默吞掉**了:网络面板一切正常,页面什么都没有。现在 catch 会
console.error(仍然不抛,装饰性组件不该拖垮宿主页,但必须留痕)。

## 3. 侧栏图标
billing 声明 icon = "💰",而原生 tab 全是内联 SVG(stroke: currentColor)。
emoji 尺寸不对、不跟随主题。

WebUI 增加 pluginIconHTML:插件图标可以是文本,也可以是内联 SVG。
**SVG 走严格白名单**(tag + 属性都是 allowlist,不是 denylist)——
插件是在运维者浏览器里跑的第三方代码,不能"信任插件";但也不能直接拒绝
SVG,因为那是唯一能和原生 tab 视觉一致的方式。

用真实 Chromium 验证 12 个用例,全部挡住,包括 foreignObject 里嵌 HTML
命名空间 <img onerror> 这个经典绕过(整体丢弃,所以 img/onerror 也没了)。
★ node 里没有 DOMParser/jsdom,所以没法在单测里跑这个过滤器 —— 用正则近似
会得到一个"测试通过但浏览器里失效"的过滤器,这比没有测试更糟。

顺带修了过滤器的两个真缺陷:输出里嵌套了空 `<svg></svg>`,且 viewBox
是从包装元素读的(永远是 null)而不是插件自己的,所以任何自定义 viewBox
的图标都会丢失。

## 判据(新增 7 项,全部变异验证)
写「注入脚本能否正常执行」这个守卫时我错了四次:
  1. 静态扫「已声明的名字」→ 把 HTML 字符串里的 CSS 类名(class/div/td)
     全报成未定义
  2. 用 CSS 选择器解析器查样式表 → 报样式表本身坏了
  3. 只挂 process 的 uncaughtException → 脚本在 IIFE 里异步跑,错误是
     unhandledRejection,判据对原 bug 全绿
  4. 只查「有没有抛错」→ render() 开头是 `if (!st) return`,传错字段是
     **静默 no-op**:不抛、不打日志、不报错,只是页面空白
最终判据是:在 node 里用 DOM stub 真跑一遍,同时要求「无异常」且
「至少写进一个容器」,并监听 console.error。变异验证:还原 s → 红;
render 收到 undefined 字段 → 红。

表头/行列数一致性也有守卫:row() 加了缓存列而表头没加时,表格会整体错位
(cache% 落到 completion 列下)—— 渲染正常、有数据、但要仔细看才发现。

## 生产验证
重启后价目表与累计账完整保留(1.17 亿 prompt tokens)。
新请求缓存统计生效:cache_hit 947,436 / cache_fresh 888,
cache_reported_reqs 7 / 395(其余来自旧 state,正是该字段存在的意义)。
真实浏览器:表格 3 行、KPI 7 项、表头 name/cost/reqs/prompt/fresh/cache/cache%/completion、
SVG 图标 currentColor 渲染、控制台无 billing 错误。391 个测试全绿。

## 另发现一个无关 bug(未修)
首页 stats 图表抛 IndexSizeError: arc 半径为负(-2),在 ui/index.html 的
paintStats 附近。属状态页图表,不在本次范围。
2026-10-02 11:16:15 +08:00
30064696b3 perf(plugin): 去掉钩子热路径的 JSON 往返 + 同 stage 跨插件并行
## 1. 去掉 JSON 往返(快路径)
实测单次 Fire 14.6µs,其中 json.Marshal 4.0 + json.Unmarshal 5.6 = 9.6µs,
**67% 花在把 map[string]interface{} 序列化再反序列化**,而紧接着的
pushGoValue 本来就能直接遍历这两种类型。改为按类型直接转换(fastvalue.go),
只对不认识���类型才回落 JSON —— 陌生字段仍然会被送到插件,而不是消失。

快路径与 JSON 路径逐字节等价由 TestFastPathMatchesJSONPath 锁住(8 组载荷,
覆盖 int/uint/float 各宽度、嵌套、slice、map[string]string、未知类型)。
还有一条专门防止「优化悄悄失效」:TestFastPathIsActuallyUsed 用真实的
request_end 载荷断言它确实走快路径。

同一份代码 A/B 实测:JSON 往返 56.4µs → 快路径 35.3µs(省 37%)。

## 2. 同 stage 跨插件并行
参照 /home/program/TrueAgent 的 StageHost.RunStage:
  - **快照后释放锁**再并行 —— 它记录过一次自死锁(p.Stop → onExit → ReclaimOwner
    要拿 registry 锁,持锁并行即死锁)。这里同理:钩子可能经 admin API 增删插件,
    那条路径要拿 ps.mu 写锁,所以并行段内不持任何 ps 锁。
  - 每个 goroutine recover。
  - 单插件走直连路径,不付 goroutine 代价(生产就是这种配置)。

**与 TrueAgent 不同的一点**:它可以放心并行,因为 handler 只写 ctx.Response 并有
IsResponded() 仲裁;我们的钩子返回 table 会合并进 payload,而
docs/plugins.md 明确承诺「payload 原样传给下一个插件」。所以合并**按插件加载
顺序**执行,结果确定,不依赖调度;代价是钩子之间不再互相可见 —— 这是一处
**契约变化**,已在文档里写明,并说明随核心发布的 billing 从不返回任何值
(代码注释就写着 "nobody downstream would read a return value")。

实测收益(真实二进制,三实例对照,3000 请求):

              无插件     1 插件      4 插件
  稳态并发32    849 rps   768 (-9.5%) 741 (-12.7%)
  突发并发64   1524-1893  1182-1676  1064-1443

4 插件只降 10-20%,而并行前实测 4 插件是 63.8µs vs 单插件 14.6µs(-300%)。

## ★ 我自己造成的两次性能事故
**① 持久化把热路径拖慢 26 倍。** 最初的快照在钩子路径上做:走 luaValueToGo +
json.Marshal + json.Unmarshal 三重转换,每请求 264µs,Fire 从 14.6µs 变成 385µs。
改成 saver 按自己节奏拉取(钩子只标记 dirty,flush 时才快照),385µs → 25.7µs。
**这里还踩了第二次 use-after-free**:让后台 goroutine 去读 Lua 表,vm.Stop() 后
那是已释放内存(SIGSEGV)。安全性现在由「Plugins.Close 等 saver 的最后一次
flush 完成后,调用方才停 VM」保证。

**② 基准被自己的后台写入污染。** 关掉 markDirty 反而测出 36µs、比开着还慢,
方向完全反了 —— 是 saver 每 2 秒写盘混进了计时。加了 DisableStatePersistence
后数据才可信。

## 判据(11 项,全部变异验证)
快路径等价/确实生效/不别名输入 + 并行与单插件路径合并一致 + 合并顺序确定 +
抛异常的钩子不拖累同伴 + 每插件恰好执行一次 + 并发 Fire 安全 + Fire 期间不持
注册表锁 + 真实 billing 在并行下正常 + 持久化 7 项。

变异:改坏合并顺序 → 红;去掉单插件路径的合并 → 红(3 个既有测试同时抓到)。
★ 「删掉 recover」这个变异**没有**让判据变红,查下去发现 golua 把 error()、
nil 索引、调用 nil、深递归全部转成 error RETURN,不产生 Go panic —— 那个测试
根本没测到 recover。已改名 TestThrowingHook 并在注释里写明 recover() 当前无法
被 Lua 触达,保留它是为了守 Go 侧。留一个「看起来有覆盖」的断言比没有更糟。

## 端到端(真实二进制 + 真实 billing)
20 万请求全 200,rps 1870,p99 96ms,RSS 37.9→42MB 有界;
负载停止后四个插件计数**完全一致**(231745),hook_errors 为空;
systemctl restart 后 billing 仍是 231745 —— 并行与持久化同时生效。
381 个测试全绿,含 -race。
2026-10-02 10:45:20 +08:00
ce2032435a fix(plugin): 插件 state 持久化 —— 重启不再丢账
## 问题(压测实测)
插件 state 活在 Lua VM 里,进程一死就没了。实测线上量级:
  重启前 {"requests":218241,"prompt_tokens":26188920,...}
  重启后 {"requests":0,"prompt_tokens":0,...}
对计费插件来说这不是舍入误差,是功能本身没生效 —— 它存在的意义就是那个
不断累加的数字,而一次 systemctl restart 就能把它抹掉。

## 设计
- prices(配置)与 state(累计历史)**分开存**在同一个文件里但不同字段。
  SetState 在内存里已经这么分,磁盘必须同意:合并会让改价看起来像清零,
  或让恢复历史时顺带复活过期价格。
- 原子写(临时文件 + rename):写一半崩掉时上一份仍可读,而不是留下一个
  解析失败的半截 JSON —— 那等于这次也丢。
- 损坏文件只警告不阻断启动。转发不能依赖插件的账本活着。
- 防抖后台刷:钩子路径只标记,真正的写在一个 goroutine 里合并进行。
  计费插件每请求都改 state,同步写会把一次 JSON 编码 + 文件写放到热路径上
  (实测钩子本身已经 14.6µs,写会盖过它)。
- Core.Close 必须先刷插件再停 VM:flush 要读 Lua 表,vm.Stop() 之后读的是
  已释放的内存。

## ★ 实现中踩的四个坑(都由测试或崩溃直接暴露,不是推测)
1. **后台 goroutine 碰 Lua = use-after-free**。最初让 flush 线程去读 Lua 状态,
   vm.Stop() 后那是已释放内存 —— 表现为 golua 里的 SIGSEGV,不是干净报错。
   改成:钩子路径(VM 必然存活、已持 p.mu)取快照,后台只写文件。
2. **自死锁**:markDirtyLocked 被 invoke 调用,而 invoke 全程持 p.mu,
   再 Lock 一次就是死锁。lua 包测试直接挂到超时。
3. **luaToJSON 独占整个栈**(每条路径结尾都 SetTop(0))。连续调两次读两个
   字段时第二次访问的是不存在的槽位 —— 这个绑定不 panic,直接 SIGABRT。
   改为每次重建栈。中间还因为提前 return 没 Pop 而让栈逐次错位。
4. **快照顺序**:先快照后读返回值,会把钩子的返回值清掉,于是每个"有意见"的
   插件静默变成"没意见",而文档承诺的"返回 table 合并进 payload"就废了,
   且没有任何报错。

## 判据(7 项,全部变异验证过)
重启后总计保留 / prices 与 state 分离 / 纯 prices 更新也持久化 /
钩子返回值不被快照吃掉 / 损坏文件降级不阻断 / 500 次变更合并成个位数次写 /
Close 刷出尾部。

变异结果:
  关掉 mark          → TestStateSurvivesRestart + TestPricesAndStateAreSeparate 红
  Close 不等 flush   → TestCloseFlushesTail 红
  prices-only 不写盘 → TestPricesOnlyUpdatePersists 红
  还原快照顺序       → TestHookReturnValueSurvivesSnapshot 红
★ 第一次跑「关掉 mark」时判据没报错,原因是我的变异脚本写出未使用变量导致
  编译失败 —— go test 根本没跑测试,我却读成了"通过"。换成 _, _ = 后如期变红。

## 端到端
隔离实例发 12 次请求 → systemctl restart → requests 仍为 12,token 数不变。
371 个测试全绿。
2026-10-02 10:17:07 +08:00
8de1499c40 fix(ui): 插件元素注入被宿主页面重建擦除 —— 元素型注入从未真正生效
## 现象
部署示例后打开 WebUI:侧栏有 Billing 页,但**状态页上没有任何计费组件**。
插件明明声明了 elements,/api/ui-inject 也确实返回了 mount(1570 字节)。

## 根因(不是缺功能)
七个宿主页面的渲染函数都用 `pane.innerHTML = ...` **整体替换**自己的 DOM。
`renderStatus` 在 `injectPluginUI()` 之后由 refresh() 立刻调用,于是刚挂上的
plugin-el 连同整个 pane 一起被下一次赋值销毁。

时序上它**从没有过"显示一帧"的机会**:注入 → refresh("status") → innerHTML 覆盖。
所以症状是"元素从来没出现过",而不是"刷新后消失"——这正是我先前据
/api/ui-inject 返回值判定"注入正常"而漏掉的地方:**载荷到达 ≠ DOM 存活**。

Billing 页不受影响,因为它属于 PLUGIN_PAGES,走插件自有 DOM,不经宿主重建。
于是看起来像"页面注入有效、元素注入无效",把排查引向插件声明本身。

## 修法
- mountPluginElements 改成具名可重入函数,并注册进 PLUGIN_MOUNT_HOOKS
- refresh() 在**唯一出口**统一调 remountPluginElements(),而不是给七个渲染函数
  各加一次调用——后者是多一处会忘的地方,而忘记的后果是静默的
- 每个挂载点按 data-idx 幂等:宿主重绘时若该 pane 已有该元素就直接返回,
  否则插件的 <script> 会每次重绘都跑一遍,计数器静默翻倍

## ★ 验证方式换了:真实浏览器,而不是 payload
静态测试和 curl 都看不出这个 bug(载荷完全正确)。用 CDP 连本机共享浏览器实测:

  修复后:首屏 tile=1,页面重建后=1,连续重建 5 次仍=1,console 无错误
  回退后:tile 全程=0

  对照二进制(把 done 改成空函数重编译)实测首屏就是 0,
  **证明"从未显示过",不是"显示后消失"**。

## 判据与变异
TestPluginElementsSurviveHostRebuild 锁住:remountPluginElements 存在、
PLUGIN_MOUNT_HOOKS 在使用之前声明(const TDZ 会让首屏直接抛错)、refresh 挂了重挂、
挂载按 data-idx 幂等。

三个变异全部被抓住:撤掉 refresh 的重挂 / 去掉幂等守卫 / 把 const 声明移到 push 之后。
★ 第一次跑第三个变异时**判据正确地没报**,因为我的替换脚本命中了注释里的同名文本,
真正的 const 没被移动——是变异无效,不是判据有洞。换按行定位后如期变红。

## 同时补上部署示例
packaging/config.example.yaml 里补 plugin_dir 说明(之前只有 online 部署路径踩过)。
实测升级路径本身是好的:给已有配置加 plugin_dir 后,首次启动会自动 seed 内置
billing 插件,无需手工放置文件。
2026-10-02 09:04:47 +08:00
1c690611f8 feat(gui): WebUI 与 Electron 壳的插件安装/删除/禁用/编辑
## WebUI:新增「插件」页
- 列表来自 on_disk(不是 loaded 集合)——**加载失败的插件也必须显示并带错误**,
  否则一个语法错误看起来和"插件没装"完全一样
- 启用/禁用(PUT {"enabled":bool})、删除、编辑源码、安装/覆盖
- 显示 hook_errors:插件抛异常在别处毫无痕迹,没有这一栏的症状就是
  "功能就是不work"
- 插到 dropzone 与代码编辑器都做了泛型化(bindDropzone / openCodeModal),
  适配器与插件共用一份,而不是复制第二份只改 4 个 id 的函数

## TABS 收敛为单一常量
tab 清单原本是字面量散在三处:goTab、refresh()、admin-only 隐藏列表。
加一个 tab 意味着三处都要记得改,漏一处就是"路由认得但界面不显示"——
和今天早些时候 chain_step 漏报同一类静默缺口。现在只有 const TABS。

## Electron 壳:设置面板里的插件管理
渲染进程不能直连内嵌核心(没有 key、不知道端口),所以走 IPC:
  renderer → plugins:proxy → main → HTTP /api/plugins
代理是 (method, path, body) 透传而不是固定命令表:固定表每加一个端点就要扩,
而"按钮存在但什么都不做"比"没有这个按钮"更糟。透传让渲染层能调用核心将来
新增的任何 /api/plugins 路由,路径在主进程校验。

## ★ GUI 此前零测试,而本次改动就引入了三类"看起来没事"的问题
1. 引用了不存在的 CSS 类(.tag / .sm)——渲染成无样式文本
2. 引用了不存在的 helper(esc / escAttr)——那是 WebUI 的,renderer/app.js
   是独立文档,点击时 ReferenceError
3. .ghost/.primary 只在 .form .actions 作用域内生效,插件按钮在 .pl-acts 里
   于是是无样式裸按钮

补 4 个静态判据(不启动 Electron,守卫的正是"打开应用才看得见"那一类):
  TestGUICSSClassesExist          用到的类必须在样式表里定义
  TestGUIHelperFunctionsAreDefined 被调用的函数必须有定义
  TestGUIPluginPanelIsReachable  面板在 overlay 内、按钮已绑定、打开设置会加载
  TestGUIIPCPathIsConstrained    代理必须限定 /api/plugins 前缀并拒绝路径穿越

写第一个判据时我错了三次:CSS 解析器先丢最后一个 selector、再把变量块当
selector、最后漏掉复合选择器(.tb-btn.tb-close)。两次"判据自己坏了"的
教训和本项目一贯一致——**判据出错的信号是它报了一个假问题**。现在改用宽松的
token 提取 + 显式的 guiKnownUnstyled 豁免表(blob/tgl/rail 是既有无样式类,
不是本次引入,失败它们只会让判据对新工作失去意义)。

## 变异验证
  改坏唯一的 CSS 定义(.pl-empty)→ TestGUICSSClassesExist 红
  改坏 helper 名 → TestGUIHelperFunctionsAreDefined 红
★ 第一次变异我改了 .pl-broken,判据**正确地没报**——因为它还被另一条规则定义。
  这是变异选错目标,不是判据有洞;换 .pl-empty 后如期变红。

363 个测试全绿。
2026-10-02 08:47:08 +08:00
a78f7cb6c5 feat(plugin): 启用/禁用 + 磁盘列表 + 峰谷定价 + 随核心发布
## 插件管理后端
- PUT /api/plugins/{name} {"enabled":bool}   启用/禁用
- GET /api/plugins/{name}                    读源码(编辑器用,与 /state 区分)
- GET /api/plugins 的 on_disk 字段            列出目录里所有 .lua 及其加载态
- validPluginName 提取为共享函数,install/remove/read 三处共用,防止检查漂移

禁用是**运行态开关,不删文件**:插件把线上网关搞坏了、但离修好只差一行时,
运维需要把它移出请求路径而不丢失它(同 systemd mask 而非 remove 的道理)。
它**不跨重启保留**——一个悄悄比操作者意图活得更久的"禁用"本身就是个意外。

Builtin 的判定是「加载的源码与内嵌版本逐字节相同」,而不是「名字匹配」:
被改过的 billing.lua 不能被标成 builtin,否则 UI 会提供覆盖用户改动的操作。

on_disk 列表包含**加载失败**的插件。否则一个语法错误的插件在 UI 上直接消失,
运维看到的现象是"插件不见了"而不是"插件报错了"。

## 峰谷 / 时段定价
commandcode 的 DeepSeek V4 系列就是高峰 01-04 & 06-10 UTC 工作日 2 倍价
(非高峰 17h/天)。静态价目表达不了,而算错方向是**静默**的。

价目条目可带 peak = {multiplier, windows=[{days, hours}]}。命中任一窗口即乘。
★ 用 `os.date("!%H")` 取 **UTC** 小时:provider 费率表按 UTC 标注,而网关跑在
本地时区(本机 Asia/Hong_Kong)。混用本地小时会让峰谷整体偏移 8 小时,
白天算成夜间——比不做峰谷还糟。

## ★ 实现与注释不一致,被判据抓住
applyPeak 最初直接 `price.prompt = price.prompt * m`,注释写「缓存读不翻倍」。
但 costFor 里**缓存读价是从 price.prompt 派生的**,所以原地翻倍会把缓存读
也翻倍——两个折扣被叠在一起,而 provider 从没打算叠。
改成 applyPeak 只**记录**乘数,由 costFor 分段应用:fresh prompt 与 completion
翻倍,cache read 那一项不动。
只靠注释说明意图是不够的:TestBillingPeakDoesNotDoubleCacheRead 立刻红了
(0.006 vs 期望 0.003)。变异回原实现仍是红的。

## 判据(21 个计费测试全绿,新增 5 个峰谷)
  窗口恒命中 ×2 / 窗口永不命中保持静态价 / 星期不匹配不命中
  (这条正是防"用本地时区整体偏移 8 小时")/ 无 peak 规则向后兼容
  / 缓存读不随峰谷翻倍

后端部分:构建/vet/gofmt 干净,8 个包全绿。
2026-10-02 08:33:41 +08:00
d0c7465130 fix(plugin): /api/ui-inject 的 stages 漏掉 chain_step + 忽略临时构建目录
部署到线上时用隔离实例(独立端口 18099 + 独立 config/runtime/adapter 目录)
发真实请求验证,抓到的第三个 bug。

## bug:discovery 文档漏掉新 stage
handlePluginUI 的响应里 stages 是**字面写死的三个**。加 chain_step 时只改了
lua.AllStages,没改这里,于是插件作者读 GET /api/ui-inject 会看到
["request_start","routed","request_end"],**合理地得出结论:没有 chain_step 这个
stage**。stage 本身是注册好的、也确实在触发,只是没被声明。

改成从 lua.AllStages 派生——AllStages 是唯一定义顺序的地方,让它保持唯一。

★ 而我原来的测试断言 `len(view.Stages) != 3`,**断言本身是 bug 的保护伞**:
它把"三个"固化成了期望值,于是新增第四个 stage 时测试全绿、bug 上线。
现在断言改为「与 AllStages 等长且逐项相同」,并显式要求 chain_step 在其中。
新增 stage 而忘了声明,这类问题会立刻红。

## 顺带:.gitignore 补上临时构建目录
.probe/ 和 .build-work/ 是我调试时当 GOTMPDIR 和临时二进制用的,之前每轮
手工删,这轮差点提交进去 9.5MB 的二进制。

## 部署验证留档
隔离实例跑真实 chat(158 prompt / 13 completion / 128 cache_hit),计费插件
算出 0.000688,与手算 (158-128)*1e-5 + 128*1e-5*0.1 + 13*2e-5 **逐位吻合**。
★ 第一次手算我按全价算成 0.00184,一度以为插件算错了——查审计记录才看到
cache_hit_tokens。**算钱不对时先查输入再怀疑实现**,而我忘的恰是刚修的折扣。

## 验证
351 个测试全绿;变异(stages 改回硬编码三个)被 TestUIInjectServesPluginUI
抓住,3 条断言同时红。
2026-10-02 06:24:34 +08:00
cb6df0a3f0 fix(billing): 缓存命中按全价计 + 未定价流量静默记 0
部署前审计计费插件时自己找到的两个真缺陷,都会直接算错钱。

## ★ 缺陷 1:缓存命中按全价计(高估约 10 倍)
costFor 只看 prompt_tokens,不区分其中多少是缓存命中。实测(审计脚本,非推演):
1M prompt token 里 900k 是 cache_hit → **算出 10 USD**,而缓存读通常只要 1/10
价,正确值 ~1.9。agent 流量反复重放长前缀,正是缓存要让它便宜的那类流量,所以
这个偏差恰好落在最高频的流量上。

改为拆分:
    fresh  = prompt_tokens - cache_hit_tokens  → 全价
    cached = cache_hit_tokens                  → 全价 × cache_discount
cache_discount 默认 0.1(DeepSeek/Qwen/Kimi 的量级),可按条目覆盖——**折扣率是
每个 provider 的事实、不是自然常数**,所以 0.1 只是默认值而不是硬编码常量。
另外把 cache_hit 钳到 prompt 以内:适配器报出比 prompt 还大的缓存命中数时,
fresh 会变负数,凭空产生负计费 token。

## ★ 缺陷 2:未定价模型静默记 0(最危险)
没有任何价目覆盖的请求,成本记 0,而 **requests 和 token 数照常计入 total**。
于是账单看起来完全正常,只是 quietly 少报——没有任何报错,没有任何异常。
比多算危险得多:多算你会去查,少算你不会知道。

新增两个维度把这件事变成显式信号:
    unpriced_reqs    未定价请求数
    unpriced_models  按模型点名,直接告诉你价目表缺哪一行
仪表盘加一张 "Unpriced" 卡片,**这个数应该是 0**。
任何维度(source / model / key)覆盖了就算 priced。

## 修这两个时自己踩的坑
第一版把未定价统计块写在了 `local s = plugin.state` **之前十行**,
在一个全新插件上 hook 直接抛 "attempt to index global 's'",于是
**整条请求什么都没记**——计费插件能有的最坏失败方式。
是 TestBillingZeroPricesIsSafe 的 "requests = 0" 抓到的。
代价:一个计费插件静默失效,而网关日志里只有一行 hook error。

## 判据(351 个测试全绿,计费相关 16 个)
新增 5 个,全部是**具体金额**断言:
  TestBillingCacheHitsAreDiscounted          1M/900k 命中 → 1.9
  TestBillingCacheDiscountIsPerModel         覆盖为 0 / 1 两种极端
  TestBillingCacheHitClampedToPrompt         荒谬的命中数不产生负费用
  TestBillingCountsUnpricedTraffic           只数未定价的那个,且流量仍计入 total
  TestBillingAnyDimensionCountsAsPriced      源维度定价也算 priced
2026-10-02 01:14:33 +08:00
42764bc99e feat(plugin): AUTO 调度轨迹可见(chain_step stage)
被问"还有 auto 调度相关 stage 呢?"问出来的真实缺口。

## 问题
chainDrive 只返回 (resp, src, model, err),调用方只知道**最终哪个槽位赢了**。
遍历过程中算出来又丢掉的东西——哪些档被跳过、为什么跳过、哪些槽位硬失败、
哪档全忙——一律不可见。ChainErr 里其实有这些,但**只在全部失败时**才填,
而它是 error 返回值不是记录。于是:

    "tier 1 冷却所以降级到 tier 3"  ==  "tier 1 正常接单"

对插件而言 tier 只是个常量 -2("resolved by the chain"),信息量为零。而这
恰恰是优先级链存在的全部理由,也是"我那个贵模型为什么没被用"的答案。

## 做法(scheduler 侧零新依赖)
新增 TraceEvent / TraceSink,chainDrive 多一个可选 sink 参数:

  - TraceEvent 是本包的普通 struct,sink 是 func 参数 ⇒ **不新增 import**,
    scheduler 仍然可独立测试
  - sink 为 nil 时每次 emit 只多一次 nil 判断;没有插件的网关在 AUTO 热路径上
    零开销(gateway 的 chainTraceSink 直接返回 nil)
  - 事件是纯观测:scheduler 不基于它做任何分支,gateway 也不把它喂回路由/
    冷却/配额

四种 kind:tier_skip / slot_fail / tier_busy / selected,selected 每次成功
遍历恰好一次且是最后一步。顺序保证所有 step 在 routed 之前。

## 暴露给插件
新增 chain_step stage(逐个步骤),并在 request_end 载荷里加三个便于做报表的
字段:chain_walk(上限 12 步,防审计记录膨胀)、degraded、tier_served。

## ★ 计费口径(我按推荐的做,已写进文档,需要你确认)
**按实际服务的模型计费**:降级到 tier 3 仍按 tier 3 的价算,轨迹只作观测。
理由与 §7.5 的边界一致——插件只报表不执法,两套口径混在一起会引出"降级该不该
多收钱"这种无法从代码判断的争议。若要改成"按本该用的档计价",需要在 models
价目里允许按 tier 定价,这我没做,因为那是个产品决策。

## 计费插件同步消费
by_tier_served / skip_reasons / degraded_reqs 三个新维度。skip_reasons 的等待
时长做了归一(`no free slot within <wait>`),否则 busy-wait 文案一变就多一行。
降级次数在 request_end 里计而不是在 chain_step 里计:一次降级的请求要走多步,
按步计会重复计数。

## 判据(346 个测试全绿,新增 15 个)
  scheduler  6 个:正常路径只发一个 selected / 跳档+降级可见 / 硬失败与跳档
                严格区分(不可混为一谈,否则抖动上游看起来像空闲上游)/
                nil sink 安全 / 全失败时轨迹与 ChainErr 并存且不互相破坏 /
                空链不发事件
  gateway    1 个端到端:tier 1 全 500 → 插件收到 slot_fail(tier 1) +
                selected(tier 2),request_end 的 tier_served=2 且 degraded=true
  lua        2 个:降级计数与按实际模型计价 / 跳过原因归一聚合
  lua        1 个:chain_step 是真 stage 且顺序正确

3 个变异都红:去掉 slot_fail(3 个判据红)/ 去掉 tier_skip(1 个)/
去掉 degraded 字段(1 个)。
2026-10-02 01:03:39 +08:00
8c18e0c3d7 fix(plugin): request_start 在直连与生图路径上根本没触发
被"你确定功能全部正常了?你全部测试了?"问出来的。之前所有插件测试都是直接调
Plugins.Fire(),只证明 Lua 运行时没问题,**完全没验证网关有没有真的触发**——
把 handleChat 里的三处调用删掉,整个套件照样全绿,而线上一个钩子都不会跑。

补上走真实 HTTP 的端到端判据后,立刻抓到两个真 bug:

## bug 1:直连路径完全跳过 request_start
fireStart 只写在 handleChat 的 AUTO 分支里,任何指定了具体模型的请求(也就是
绝大多数请求)都不触发。修法是挪到 isAuto 判断之前,两条路径共用一次调用。

顺带修正位置语义:它在配额/模型范围闸门**之前**触发,所以插件能统计到被网关
拒绝的请求;否则插件永远只能报"被服务的请求数",算不出真实请求率。

## bug 2:生图路径三个 stage 全断
handleImage 是第三个入口,有自己的 handler 和自己的调度调用。"聊天能用"对它
毫无证明力。而生图是计费流量,计费插件看不到就等于少报。
已补 fireImageStart + 两处 fireRouted(direct -1 / AUTO -2)。
它写独立函数而不是复用 fireStart 传空 chatRequest:image 请求没有 messages
和 tools,传一个为聊天设计的零值结构会诱导后来者去读不存在的字段。

## 端到端判据(6 个,全部走真实 handler)
  TestHooksFireOnRealDirectChat      直连:三个 stage 顺序 + 真实 source/model/tokens
  TestHooksFireOnRealStreamChat      流式是另一条路径(记录由 defer 在流结束后写)
  TestHooksFireOnAutoRequest         AUTO 链:start 报 "AUTO"、routed 报**解析后**的模型
  TestHooksFireOnFailedRequest       失败请求:routed 不触发(没选到源)、
                                      request_end **必须**触发(否则计费看不到失败流量)
  TestHooksFireOnRealImageRequest    生图:type=image,第三个入口
  TestRejectedChatStillFiresRequestStart  404 拒绝也要触发 start(顺序决定的钉子)
  TestBrokenPluginDoesNotBreakForwarding   插件每 stage 都抛异常时聊天仍返回 200

## 变异验证
把 fireStart 挪回 AUTO 分支(= 重现我犯的错)→ 4 个判据红:DirectChat /
StreamChat / FailedRequest / RejectedChat。恢复后 336 个测试全绿。

这两个 bug 都属于"读代码看不出来"的类型:fireStart 那一行就在 handleChat 里,
看着挺像那么回事,只有真的发一个请求才知道它没被调到。
2026-10-02 00:49:32 +08:00
a51a6811a6 feat(plugin): Lua 插件机制 + 计费插件 + 插件文档
插件 = plugin_dir 下的单个 .lua 文件,做两件事:挂请求流水线的钩子、在启动时
贡献 WebUI 界面(整页或往现有页面追加组件)。两者独立。

## 流水线 stage(三个)
  request_start  已解析鉴权、未选源
  routed         已选定 (source, model)、未发往上游
  request_end    每请求恰好一次,带最终计量
request_end 挂在 gateway.writeRec——四条入口路径(直连/AUTO × 流式/非流式)的
唯一汇合点:既不漏(流式 token 只有流结束才知道)也不重。

## 计费插件(plugins/billing.lua,默认 seed,开箱可用)
源 / 模型 / 密钥三个维度定价。token 价优先级 keys > models > default;per_request
固定价是**叠加**的(生图模型可以既算 token 又收固定费)。单位是 USD/单 token,
即各家 provider 的公布口径。累计 total / by_source / by_model / by_key / by_day。
失败请求保留 token 费用、丢弃固定费(可经 count_failures 翻转)。
界面 = 一个独立页 + 状态页顶部一块总开销 tile。

## 一个明确的设计边界
计费插件**只报表,不执法**。网关自己的配额会计(stats.go,入口强制)才是限额
权威,插件不参与任何路由/配额决策。两套独立会计若对不上,比一套功能略少的
更糟。

## ★ 中途改掉的一个根本设计错误
最初让插件复用适配器的**弹性 worker 池**(多状态)。这对适配器是对的(它们无
状态),对插件是错的:计费插件往 plugin.state 累加,多状态意味着总量被劈成
几份;而 SetState 写价格只写进其中一个 worker,钩子恰好跑到另一个时**所有请求
按 0 计费**。改为**单状态 + 互斥锁**。代价写进文档:钩子必须短、同步、不阻塞,
卡住的钩子会卡住所有插件的钩子。
这个 bug 是测试逼出来的——先写了 SetState+Fire 的用例,数字全是 0 才挖出来。

另一个连带缺陷:只带 prices 的 PUT 会整体替换 state,把累计量清零。改为
prices/state 分离——prices 是配置、state 是历史,改价不动账。

## 撞到的三个 Lua 绑定的坑(都写进注释)
  - SetGlobal **会 pop 栈**:连着调两次,第二次从空栈取,赋成 nil
  - GetField 索引越界是 **SIGABRT 整个进程**,不是 panic,recover 救不了
  - Call(nargs, n) **不接受函数索引**,它调的是 nargs 个参数正下方那个;
    传索引会调到参数上("attempt to call a table value")
另外 GetField/SetField 用绝对索引,SetTop(0) 之后必须重取。

## 错误隔离
钩子 error() 不影响转发:捕获 → 记进 hook_errors → 跳下一个插件。适配器出错
会让源进冷却,插件出错**零惩罚**——插件是可选功能。/api/plugins 的 hook_errors
让"坏掉的插件"可见而不是静默消失。

## 界面注入
GET /api/ui-inject 一次返回所有插件的扩展(侧栏需要全部 page 才能建好)。
WebUI 在首次 render **之前** await 注入:先插 HTML 再重建 <script> 让它执行
(innerHTML/template 插入的 script 不会执行,这正是要的效果——避免脚本跑在
自己 DOM 之前)。注入失败不影响仪表盘。
browser 侧 pluginAPI 暴露 fetchState / postState / onTabShown。

## 文档
docs/plugins.md —— 快速上手、加载与热更新、三个 stage 的完整字段表、界面扩展、
状态与 HTTP API、运行时约束(单状态/异常隔离/内置函数)、计费插件的定价与
计费策略、排错表、与适配器的对比表。

## 判据(328 个测试全绿,插件相关 33 个)
  - 计费断言的是**具体金额**(0.00625 / 0.0402 / 0.0075…),不是"能加载"
  - 4 个变异都红:钩子异常不隔离 / prices 清空累计 / 忽略 key 优先级 /
    毫秒时间戳不换算
  - UI 侧 6 个判据把注入顺序、script 执行时机、pluginAPI 名称、tab 路由、
    anchor 四种形式、失败非致命全钉住
  - 鉴权:state 读任意角色、写仅 admin
2026-10-02 00:37:29 +08:00
a2e1adc2d8 fix(gemini): endpoint 自相矛盾导致预置模板必失败
gemini.lua 里 adapter.endpoint = "/v1/models",而它自己的注释写的是
  POST /v1/models/{model}:generateContent
两者矛盾,而 Go 侧是静态拼接(provider.URL = base_url + endpoint),拼不出
模型名。预置模板 "Google Gemini"(base_url=.../v1beta)于是会 POST 到
  https://generativelanguage.googleapis.com/v1beta/v1/models
既多一段 /v1,又缺 :generateContent——那是 Gemini 的模型**列表**端点,
对 POST 返 405。所以任何用户从模板建这个源,拿到的都是必定失败的源。

实测确认影响范围:线上 21 个源里没有 gemini(openai×15 / trae / sensenova /
opencodezen / deepseek / anthropic / agentrouter),所以是潜伏缺陷。

修法:endpoint 改成模板 `/v1beta/models/{model}:generateContent`,新增
provider.ChatURL(model, stream):
  - 用 **PathEscape** 替换 {model}——模型 id 进的是 URL 路径,不转义的话
    一个 "/" 就会静默指向另一个资源(判据里用 RequestURI 而非 URL.Path
    断言,因为后者是解码后的,看不出 %2F);
  - 流式把 ":generateContent" 换成 ":streamGenerateContent"(同一个路径、
    不同动词,也在路径里)。替换刻意只认这个精确后缀,免得别的适配器
    仅仅提到这个词就被改写;
  - source 自己设的 endpoint: 仍然优先,模板被整体跳过。
Chat / ChatStream / probeChat 三处调用点改为传本次请求真实的 model——AUTO
按槽位把 req.Model 钉死,所以 URL 必须跟随**请求**的模型,用源默认模型会让
多模型源每次都打同一个(还记到别的模型的账上)。

判定静态 endpoint 的其他 10 个适配器零影响(TestNonGeminiEndpointsAreUntouched)。

顺带:Stats 的 mutex 不是可重入的,导出方法自己加锁、*Locked 后缀要求调用
方持锁。持锁调导出方法会死锁——我的探针真卡死过一次(直到 10 分钟超时)。
补上 LOCKING 注释,并加判据把这条规则钉住(含一个 20 秒上限的行为判据,
让未来的重构撞死锁时快速失败而不是拖满整个套件)。
2026-10-01 23:37:17 +08:00
7082ffb723 fix(packaging): 去掉重复的加固块,并同步线上单元的确切内容
上一版把线上单元拷进来后又追加了一份英文注释的加固指令,导致
NoNewPrivileges / ProtectSystem / SystemCallFilter 等在同一个 unit 里
出现两次。systemd 对重复指令取最后一个,行为上不会坏,但文件本身是错的
(读的人会以为有两套加固),且 packaged 与 deployed 不再一致。

现在 packaged/llmsproxy.service 与线上 /etc/systemd/system/llmsproxy.service
逐字节相同,每条指令只出现一次。
2026-10-01 20:59:34 +08:00
5e723b5aa5 feat(packaging): systemd unit 加固(逐条实测,非照抄模板)
原单元只有内存调优两行环境变量,加固项一个都没有,且以 root 运行。补上
一组经验证的加固指令。

关键决定:**仍然以 root 运行**。本服务要读 master.key(0600 root)。实测加
User=llmsproxy 直接起不来,且失败方式隐蔽——
  [config] secrets disabled: open /etc/llmsproxy/master.key: permission denied
只是一行日志,服务会带着「敏感值以明文落盘」继续跑。也就是说在当前文件
权限下降权不是加固而是把密钥降级。要降权得先把 key 交给服务用户、统一
/etc/llmsproxy 属主,那是独立的、需要回滚预案的变更,不混进来。

每条指令都在一个独立探针单元(临时端口 + 独立 runtime_file/adapter_dir,
拷了真实适配器)上验证过:
  - 鉴权 401/200 正常;
  - 一次真实 /v1/chat/completions 走通(证明 SystemCallFilter=@system-service
    没打断 LuaJIT 适配器的 JIT 代码路径——这是最容易被 seccomp 搞坏的地方);
  - 审计文件可写可轮转(ReadWritePaths=/etc/llmsproxy 够用);
  - 连续重启 3 次都 active + http 200,kill -9 行为符合预期。
systemd 对非法指令值不报错只「忽略」,所以逐条实测是唯一可靠做法。

读路径全在 /etc/llmsproxy;运行时写入经核对只有 config.yaml / runtime.json /
*.audit.jsonl / adapters/*.lua / master.key,全在该目录下,故 ProtectSystem=strict
+ ReadWritePaths=/etc/llmsproxy 即可。CapabilityBoundingSet 置空(本服务不需要
任何 capability,留空比写允许清单更难出错)。

已部署到线上 /etc/systemd/system/llmsproxy.service(原单元已备份为 .bak-*),
restart 后服务 active、监听 8081、WebUI 可达、审计继续写入;本仓库的
packaging/llmsproxy.service 与线上一致(去掉了部署机特有的 RSS 实测数字)。
2026-10-01 20:58:23 +08:00
e10bfbb278 fix(deploy): 适配器备份只保留最近 5 份,不再无限堆积
sync_adapters 每次部署都新建一个 adapters.bak.<时间戳> 目录,而回滚只用到
最近一次($BACKUP_BIN / $BACKUP_CONFIG 都是单文件覆盖)。多出来的目录从没人
清理——线上实测已积累 71 个,把 /etc/llmsproxy 顶到 122M。

prune_adapter_backups:
  - 按时间保留最近 KEEP_ADAPTER_BACKUPS=5 份(sort 升序,删前面较旧的);
  - 另给目录数一个硬上限(5×4),防一次误配置在几秒内造出成百上千个目录;
  - 只删名字严格匹配 adapters.bak.<14位时间戳> 的目录,手工放的
    adapters.bak.MANUAL 之类一律跳过不删(宁可留着也不误伤)。

本地用真实形状的名字(14 位 YYYYMMDDHHMMSS)跑了 5 个用例验证:
12 份→留最新 5 且手工目录/无关目录不动、恰好 5 份→全留、40 份→走硬上限、
0 份→不报错、只有手工目录→原样保留。

线上已手动按同一规则清到 5 份(122M→114M;剩下的大头是 runtime.json 的
审计 .old,由 auditKeepOld=16 自行限额,不在本次范围)。
2026-10-01 20:52:23 +08:00
ad54616bee docs: 对齐分支命名实际用法 + 修正示例配置的存储位置说明
三处都是"文档说的"与"仓库实际做的"不一致,读文档的人会被误导。

1. 分支命名:文档写 release/vX.Y.Z 且举例 release/v1.4.2,实际从
   release/v1.4.x 起一律用字面 x(release/v1.5.x / v1.7.x)。改成实际情况,
   并说明"一条 minor 分支跨多个 patch"是刻意的:patch 是同批功能的修订,
   hotfix 落同一条分支,回流 main 时不必处理多条 release 分支间的依赖。

2. 退役规则与实践不符:文档说"下个版本发布就删上一个 release 分支",但
   release/v1.4.x / v1.5.x 至今仍在。保留无害(hotfix 已回流),但与规则
   矛盾。两份文档都如实记下这个出入,并记录特性分支**确实**在清理——本次
   删除的四个分支都逐提交用 git patch-id 核对过,工作已全部进入 main
   (唯一 patch-id 不同的是 bf0657b 的发布前变体,diff 过 api.go 改动逐字节
   相同)。留着只是给下个版本制造 cherry-pick/merge 陷阱。

3. 存储位置:示例配置说"WebUI 新增/编辑的源会写入 runtime_file",且默认
   模型注释说 AUTO"按各源 priority 自动选最高可用源"。两处都与实现相反——
   上游源、网关密钥、AUTO 链都在 config.yaml(AddSource 走
   UpsertSourceInYAML,密钥与链走 cfg.Save);runtime.json 只剩源模板、
   删除标记和预置模板名单。已核实 store.Upsert(runtime 源)已无调用点,
   store 里的 Keys/Auto 只被加解密、从不写入,是遗留字段。

priority 本身不是完全没用,所以注释保留并说清它的两个真实用途:首次启动
seedAuto 的初值,以及 AUTO 生图链未配置时挑"最佳模型"(bestChatModel /
bestImageModel 按 priority 取最大)。
2026-10-01 20:41:57 +08:00
18e422a943 fix(sources): 源编辑不再清零 proxy_url / api_key_env / timeout
bf0657b 修好了 api_key,但 upsert 仍会重写整个源,于是请求无法表达的字段
一律被重置为零值。这四个字段的后果都不是"少个配置项":

  - api_key_env 丢失 ⇒ 盘上无明文密钥的源变成无凭据源,写操作返回 200,
    下一次调用上游才 401。而 README 恰恰把这个特性当作卖点在宣传。
  - proxy_url 丢失 ⇒ 一个走代理的上游变成直连(或反之),且完全无声。
  - timeout / queue_timeout 丢失 ⇒ 退回默认 120s / 60s。

触发路径不是只有脚本:WebUI 的 saveSource() 发的 payload 只含表单上的
11 个字段,而 editSource() 表单里根本没有这 4 项 ⇒ 运维在界面上改个并发数
就会静默清掉它们。

修法用「存在性」语义而不是「空即继承」:

  - 不传   → 保留已存值(部分更新的客户端要的就是这个)
  - 传了   → 覆盖,包括传空串表示清空

api_key 刻意保留它原有的「空即继承」规则,不跟着改成指针:该规则已随
v1.7.6 发布,脚本依赖它;而凭据丢失比代理丢失严重得多。两个字段的失败
模式相反,所以规则相反——这一条写进了两处注释。

另一处是差一点的:config.Source 把 Timeout/QueueTimeout 标成 json:"-",
所以 reveal 接口的结构体序列化**根本不返回它们**。表单读不到 → 输入框
恒空 → 而输入框每次都回传 → 每存一次就把 timeout 清零。等于把刚修好的
丢字段换个方向又造了一个。因此 reveal 分支现在显式返回 duration 字符串。

判据:
  - TestWebUIEditPayloadPreservesRoutingFields 用的是 WebUI 真实 payload
    的逐字节副本,并同时断言"确实改动的字段生效",否则"什么都不写"也能过
  - TestSourceEditPreservesAPIKeyEnv 单独盯 api_key_env(唯一造成凭据丢失的)
  - CanBeSet / CanBeCleared 分别盯两个方向:只有"空即继承"的实现过不了
    CanBeCleared(清空代理框会永远保留旧代理)
  - 持久化判据**重新加载 config.yaml 并按语义比对**:300s 会被重新序列化成
    5m0s,按字符串匹配是假红(我自己先踩了一次)
  - TestSourcePayloadCoversEveryEditableField 用反射卡住"这一类":新增
    Source 字段而没接到 API 上时立刻变红。反射查结构体而非 marshal 结果,
    因为指针 + omitempty 会合法地从序列化输出里消失,那正是"未提及"信号
  - 三个 UI 契约判据把 JS 侧也钉住(表单必须回传、必须从 reveal 读)

变异验证(每次都先确认 build 通过,再数红格):
  1. 去掉覆盖逻辑        → 7 个判据红
  2. 改成"空即继承"      → CanBeCleared + ClearIsScoped 红
  3. reveal 不返回 duration → TestSourceRevealExposesDurations 红
  4. 表单不回传 api_key_env → TestUIEditFormRoundTrips... 红

顺带修正 /api/v1 索引:DELETE /api/keys 的路径段写的是 {name},实际是 key
本身;PUT /api/keys/{key} 实现了却没列。

全量 + vet + race 全绿;WebUI 内联脚本过 node --check。
2026-10-01 20:36:48 +08:00
57 changed files with 13793 additions and 89 deletions

4
.gitignore vendored
View File

@ -26,3 +26,7 @@ cmd/gui/dist/
# local design/working notes (not part of the shipped repo)
/plan.md
# scratch build/probe dirs used while debugging (GOTMPDIR, throwaway binaries)
.build-work/
.probe/

View File

@ -671,6 +671,78 @@ ipcMain.handle(
(e) => !!BrowserWindow.fromWebContents(e.sender)?.isMaximized(),
);
// ---- plugin management over IPC -------------------------------------------
//
// The renderer cannot call the embedded core directly: it has no key and no
// network identity, and the core binds a loopback port that only the main
// process knows about. So every plugin action is proxied through the main
// process, which already knows how to obtain the admin key (unsealViaCore).
//
// The proxy is deliberately a raw (method, path, body) pass-through rather than
// a fixed set of commands. A fixed set would have to be extended for every new
// plugin endpoint, and the one thing worse than "no button for this" is "a
// button that silently does nothing" — with a pass-through the renderer can talk
// to any /api/plugins route the core grows, and the path is validated here so
// this channel cannot be used to reach arbitrary endpoints.
function pluginProxy(req) {
const { method, path, body } = req || {};
const M = ["GET", "POST", "PUT", "DELETE"];
if (!M.includes(method)) throw new Error("bad method: " + method);
// The path must stay inside the plugin namespace. A prefix check alone would
// still allow /api/plugins/../keys, so reject any traversal outright.
if (typeof path !== "string" || !path.startsWith("/api/plugins")) {
throw new Error("path must start with /api/plugins");
}
if (path.includes("..") || path.includes("\\")) {
throw new Error("path traversal rejected");
}
const key = unsealViaCore();
if (!key) throw new Error("no admin key available yet");
return new Promise((resolve, reject) => {
const u = new URL(embeddedBaseUrl() + path);
const data = body == null ? null : JSON.stringify(body);
const headers = { Authorization: "Bearer " + key };
if (data) {
headers["Content-Type"] = "application/json";
headers["Content-Length"] = Buffer.byteLength(data);
}
const r = http.request(
{
hostname: u.hostname,
port: u.port,
path: u.pathname + u.search,
method,
headers,
},
(res) => {
let raw = "";
res.setEncoding("utf8");
res.on("data", (c) => (raw += c));
res.on("end", () => {
let parsed = null;
try {
parsed = raw ? JSON.parse(raw) : null;
} catch (e) {
parsed = { raw };
}
if (res.statusCode >= 400) {
const msg =
(parsed && parsed.error && parsed.error.message) ||
"HTTP " + res.statusCode;
reject(new Error(msg));
return;
}
resolve(parsed);
});
},
);
r.on("error", reject);
if (data) r.write(data);
r.end();
});
}
ipcMain.handle("plugins:proxy", (_e, req) => pluginProxy(req));
ipcMain.handle("core:state", () => ({
running: coreStarted() && coreReady,
ready: coreReady,

View File

@ -1,12 +1,12 @@
{
"name": "modelrouter-gui",
"version": "1.7.6",
"version": "1.8.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "modelrouter-gui",
"version": "1.7.6",
"version": "1.8.0",
"license": "Proprietary",
"devDependencies": {
"electron": "^33.0.0",

View File

@ -1,6 +1,6 @@
{
"name": "modelrouter-gui",
"version": "1.7.6",
"version": "1.8.0",
"description": "ModelRouter Desktop — embedded ModelRouter core with tray",
"author": "ModelRouter",
"main": "main.js",

View File

@ -16,6 +16,13 @@ contextBridge.exposeInMainWorld("modelrouter", {
key: () => ipcRenderer.invoke("core:key"),
onState: (cb) => ipcRenderer.on("core:state", (_e, d) => cb(d)),
},
plugins: {
// Raw pass-through to the embedded core's /api/plugins surface. The main
// process validates the path and attaches the admin key; the renderer never
// sees either.
request: (method, path, body) =>
ipcRenderer.invoke("plugins:proxy", { method, path, body }),
},
settings: {
get: () => ipcRenderer.invoke("settings:get"),
set: (patch) => ipcRenderer.invoke("settings:set", patch),

View File

@ -155,6 +155,7 @@ async function openSettings() {
$("#set-tray").checked = !!state.settings.minimizeToTray;
$("#settings-overlay").style.display = "flex";
renderRail();
loadPlugins();
}
function closeSettings() {
$("#settings-overlay").style.display = "none";
@ -182,6 +183,122 @@ async function saveSettings() {
}
}
// esc / escAttr escape text for innerHTML. The WebUI has its own copies; the
// shell needs its own because renderer/app.js is a separate document that
// never loads index.html's script.
function esc(s) {
return String(s == null ? "" : s).replace(
/[&<>"']/g,
(c) => ({ "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#39;" })[c],
);
}
function escAttr(s) {
return esc(s).replace(/`/g, "&#96;");
}
// ===== plugin management =====
//
// The desktop shell manages plugins through the embedded core's /api/plugins
// surface, proxied over IPC (see plugins:proxy in the main process). The
// renderer never holds the admin key.
//
// Scope note: the desktop build has no plugin_dir configured by default, so this
// panel normally reports "plugins disabled" with the one-line fix. That is a
// deliberate state, not an error — the packaged profile is a per-user directory
// and seeding a plugin tree into someone's home without asking would be rude.
async function loadPlugins() {
const list = $("#pl-list");
const hint = $("#set-plugins-hint");
if (!list || !hint) return;
let j;
try {
j = await window.modelrouter.plugins.request("GET", "/api/plugins");
} catch (e) {
hint.textContent = "内核未就绪:" + e.message;
list.innerHTML = "";
return;
}
if (!j.plugin_dir) {
hint.innerHTML =
'未配置 <code>plugin_dir</code>,插件功能未启用。在 config.yaml 加一行后重启内核即可。';
list.innerHTML = "";
return;
}
const rows = j.on_disk || [];
const active = rows.filter((p) => p.loaded && !p.disabled).length;
const broken = rows.filter((p) => !p.loaded).length;
hint.textContent =
`${rows.length} 个插件 · ${active} 个启用中` +
(broken ? ` · ${broken} 个加载失败` : "");
list.innerHTML = rows.length
? rows
.map((p) => {
const cls = !p.loaded ? "pl-broken" : p.disabled ? "pl-off" : "pl-on";
const label = !p.loaded
? "加载失败"
: p.disabled
? "已禁用"
: "启用中";
const btn = p.loaded
? `<button class="ghost" data-act="toggle" data-name="${escAttr(
p.name,
)}" data-en="${p.disabled ? "1" : "0"}">${
p.disabled ? "启用" : "禁用"
}</button>`
: "";
const builtin = p.builtin
? '<span class="pl-builtin">内置</span>'
: "";
return `<div class="pl-item ${cls}">
<div class="pl-head"><b>${esc(p.name)}</b>${builtin}<span class="pl-state">${label}</span></div>
${p.description ? `<div class="pl-desc">${esc(p.description)}</div>` : ""}
${p.error ? `<div class="pl-err">${esc(String(p.error).slice(0, 160))}</div>` : ""}
<div class="pl-acts">${btn}</div>
</div>`;
})
.join("")
: '<div class="pl-empty">插件目录为空</div>';
list.querySelectorAll('button[data-act="toggle"]').forEach((b) => {
b.onclick = () => togglePlugin(b.dataset.name, b.dataset.en === "1");
});
}
async function togglePlugin(name, disabled) {
try {
await window.modelrouter.plugins.request("PUT", `/api/plugins/${encodeURIComponent(name)}`, {
enabled: disabled,
});
toast(disabled ? `已禁用 ${name}` : `已启用 ${name}`);
await loadPlugins();
} catch (e) {
toast(e.message, true);
}
}
async function enableAllPlugins() {
let j;
try {
j = await window.modelrouter.plugins.request("GET", "/api/plugins");
} catch (e) {
return toast(e.message, true);
}
const off = (j.on_disk || []).filter((p) => p.loaded && p.disabled);
for (const p of off) {
try {
await window.modelrouter.plugins.request(
"PUT",
`/api/plugins/${encodeURIComponent(p.name)}`,
{ enabled: true },
);
} catch (e) {
toast(`${p.name}: ${e.message}`, true);
}
}
toast(off.length ? `已启用 ${off.length} 个插件` : "没有处于禁用状态的插件");
await loadPlugins();
}
// ===== theme =====
function applyTheme() {
document.documentElement.dataset.theme = state.theme;
@ -197,6 +314,10 @@ function init() {
$("#tb-close").onclick = () => window.modelrouter.win.close();
$("#tb-settings").onclick = openSettings;
$("#rail-settings").onclick = openSettings;
const plReload = document.getElementById("pl-reload");
if (plReload) plReload.onclick = loadPlugins;
const plAll = document.getElementById("pl-toggle-all");
if (plAll) plAll.onclick = enableAllPlugins;
$("#rail-autostart").onclick = toggleAutoStart;
$("#rail-silent").onclick = toggleSilent;
$("#rail-theme").onclick = () => {

View File

@ -207,6 +207,15 @@
><input type="checkbox" id="set-tray" /> 关闭时最小化到托盘</label
><span class="hint">点关闭按钮隐藏到系统托盘</span>
</div>
<div class="row">
<label>插件</label>
<span class="hint" id="set-plugins-hint">加载中…</span>
</div>
<div id="pl-list" class="pl-list"></div>
<div class="row actions">
<button class="ghost" id="pl-toggle-all">全部启用</button>
<button class="ghost" id="pl-reload">刷新</button>
</div>
<div class="row actions">
<button class="ghost" id="set-dir">打开数据目录</button>
<button class="ghost" id="set-log">查看日志</button>

View File

@ -596,3 +596,81 @@ html[data-theme="dark"] .overlay {
#toast.err {
border-color: var(--danger);
}
/* ===== plugin management panel =========================================
* The existing .ghost/.primary rules are scoped to `.form .actions`, so a
* button outside that selector gets browser defaults. The plugin rows live in
* their own list, hence their own rules — reusing a scoped class here would have
* produced unstyled buttons that still worked, which is the kind of thing that
* looks fine until someone themes the shell.
*/
.pl-list {
display: flex;
flex-direction: column;
gap: 8px;
margin: 8px 0 4px;
}
.pl-item {
border: 1px solid var(--line);
border-radius: 9px;
padding: 10px 12px;
}
.pl-item.pl-broken {
border-color: var(--danger, #d1435b);
}
.pl-head {
display: flex;
align-items: center;
gap: 8px;
font-size: 13px;
}
.pl-builtin {
font-size: 10px;
padding: 1px 6px;
border-radius: 999px;
background: var(--primary-50);
color: var(--primary-h);
}
.pl-state {
margin-left: auto;
font-size: 11px;
color: var(--muted);
}
.pl-item.pl-broken .pl-state {
color: var(--danger, #d1435b);
}
.pl-desc {
font-size: 12px;
color: var(--muted);
margin-top: 3px;
}
.pl-err {
font-size: 11px;
color: var(--danger, #d1435b);
margin-top: 4px;
word-break: break-word;
}
.pl-acts {
margin-top: 8px;
display: flex;
gap: 8px;
}
.pl-acts button {
padding: 5px 12px;
font-size: 12px;
border-radius: 7px;
border: 1px solid var(--line);
background: var(--bg-s2, #fff);
color: var(--fg, inherit);
cursor: pointer;
transition: all 0.15s;
}
.pl-acts button:hover {
border-color: var(--primary);
color: var(--primary-h);
}
.pl-empty {
font-size: 12px;
color: var(--muted);
padding: 10px 0;
}

View File

@ -301,6 +301,60 @@ verify_master_key() {
return 1
}
# ---------- 适配器备份保留策略 ----------
# 每次部署都新建一个 adapters.bak.<时间戳> 目录,而回滚只用到最近一次
# ($BACKUP_BIN / $BACKUP_CONFIG 都是单文件覆盖)。多出来的目录从没人清理,
# 实测线上已积累 71 个,/etc/llmsproxy 因此涨到 122M。
#
# 保留最近 KEEP_ADAPTER_BACKUPS 份足够回滚,同时给目录数设上限——否则
# 一次误配置(比如 adapter_dir 指错)就可能在几秒内造出成百上千个目录。
# 只删名字严格匹配 adapters.bak.<14位时间戳> 的目录,避免误伤人工放的目录。
KEEP_ADAPTER_BACKUPS=5
prune_adapter_backups() {
local base="/etc/llmsproxy"
[[ -d "$base" ]] || return 0
# 先按数量上限硬裁:即使时间戳排序失效也不会无上限增长。
local all
mapfile -t all < <(find "$base" -maxdepth 1 -type d -name 'adapters.bak.*' -printf '%f\n' | sort)
local cap=$((KEEP_ADAPTER_BACKUPS * 4))
if (( ${#all[@]} > cap )); then
warn "适配器备份目录有 ${#all[@]} 个(异常),裁到 $cap"
local i=0
for d in "${all[@]}"; do
i=$((i + 1))
# 从最旧的开始删(sort 后升序)。名字不规范的跳过不删。
if (( i <= ${#all[@]} - cap )); then
if [[ "$d" =~ ^adapters\.bak\.[0-9]{14}$ ]]; then
rm -rf "${base:?}/$d"
else
warn "跳过名字不规范的备份目录(不删): $d"
fi
fi
done
mapfile -t all < <(find "$base" -maxdepth 1 -type d -name 'adapters.bak.*' -printf '%f\n' | sort)
fi
# 再按时间保留最近 KEEP_ADAPTER_BACKUPS 份(sort 后最新在末尾)。
local total=${#all[@]} i=0
for d in "${all[@]}"; do
i=$((i + 1))
# i <= total-KEEP 的是较旧的,要删。
if (( i <= total - KEEP_ADAPTER_BACKUPS )); then
if [[ "$d" =~ ^adapters\.bak\.[0-9]{14}$ ]]; then
rm -rf "${base:?}/$d"
else
warn "跳过名字不规范的备份目录(不删): $d"
fi
fi
done
local left
left=$(find "$base" -maxdepth 1 -type d -name 'adapters.bak.*' | wc -l)
log " 适配器备份保留最近 $KEEP_ADAPTER_BACKUPS 份(当前剩 $left 个)"
}
# ---------- 同步适配器 ----------
sync_adapters() {
log "同步适配器"
@ -318,6 +372,7 @@ sync_adapters() {
cp -rf "$TARGET_ADAPTERS/"*.lua "$BACKUP_DIR/" 2>/dev/null || true
log " 旧适配器已备份到 $BACKUP_DIR"
fi
prune_adapter_backups
cp -f "$SRC_ADAPTERS/"*.lua "$TARGET_ADAPTERS/"
chmod 0644 "$TARGET_ADAPTERS/"*.lua

View File

@ -15,7 +15,7 @@ deployable.
|---|---|---|---|
| Main | `main` | permanent | Only long-lived branch. Always deployable. Accumulates the next version. |
| Feature | `feature/<desc>` | short (dev → merge → delete) | New features / ordinary fixes. Born from `main`, merged back into `main`. |
| Release | `release/vX.Y.Z` | one version cycle | Cut from `main`, tagged for release. Version-specific hotfixes land here. |
| Release | `release/vX.Y.x` | one version cycle | Cut from `main`, tagged for release. Version-specific hotfixes land here. |
## Change flow (important)
@ -26,15 +26,21 @@ deployable.
│ │
│ cut │ cut
▼ ▼
release/v1.4.2 release/v1.4.3
release/v1.6.x release/v1.7.x
│ │
tag: v1.4.2 tag: v1.4.3
tag: v1.6.0 tag: v1.7.6
│ │
hotfix ◄─────┘ hotfix ◄─────┘
│ │
└── cherry-pick back ──────────┘
```
The patch position in the branch name is a literal `x`, while the tag carries
the concrete version: one `release/v1.7.x` can hold tags v1.7.0 … v1.7.6.
Spanning several patches on a single minor branch is deliberate — patches are
revisions of the same feature batch, hotfixes land on one branch, and back-port
to main never has to resolve dependencies between several release branches.
### Key rules
1. **main is always deployable**: never leave half-done work on `main`.
@ -43,9 +49,9 @@ deployable.
then `git merge --no-ff feature/xxx` (or squash) when done.
3. **Release = cut a release branch from main + tag**:
```bash
git checkout -b release/v1.4.2 main
git tag -a v1.4.2 -m "ModelRouter v1.4.2"
git push origin release/v1.4.2 v1.4.2
git checkout -b release/v1.7.x main
git tag -a v1.7.0 -m "ModelRouter v1.7.0"
git push origin release/v1.7.x v1.7.0
```
Build installers and upload the GitCode Release from this tag so the
published state is exactly reproducible.
@ -54,7 +60,7 @@ deployable.
an already-released branch (unless you deliberately ship a minor revision).
5. **Hotfixes MUST flow back to main**:
```bash
git checkout release/v1.4.2 # fix in the release branch
git checkout release/v1.7.x # fix in the release branch
git commit -m "fix: ..."
git checkout main
git cherry-pick <hotfix-commit> # and into main
@ -67,12 +73,23 @@ deployable.
When the next version ships, the previous release branch retires:
- **Default: delete the remote release branch**
(`git push origin :release/v1.4.2`). All hotfixes were already
(`git push origin :release/v1.7.x`). All hotfixes were already
cherry-picked into main, so main contains everything; no merge needed.
- **Long-term maintenance** (e.g. an enterprise client pinned to an old
version): keep the branch, accept only security fixes, keep the
commit-then-cherry-pick loop.
> **Where practice diverged from this section (checked 2026-10-01)**:
> `release/v1.4.x` and `release/v1.5.x` still exist locally and on the remote,
> so "retire the previous branch when the next version ships" was never
> carried out. Keeping them is harmless (hotfixes were back-ported), but it
> contradicts the rule above and makes a reader wonder whether they should be
> there at all. **Feature branches, by contrast, are cleaned up**:
> `feature/key-quota-control`, `feature/toolcall-id-sanitize`,
> `feature/anthropic-usage-cache` and `feature/agentrouter-id-sanitize` were
> deleted on 2026-10-01 after confirming with a per-commit `git patch-id`
> comparison that their work had already landed in main.
## Explicit non-goals
- **Never rebase main**: main's history stays append-only; anyone pulling gets
@ -134,8 +151,8 @@ pain points:
2. No feature branches meant two independent efforts could not proceed in
parallel without colliding.
With release branches: the published state = `release/vX.Y.Z` branch +
`vX.Y.Z` tag, exactly reproducible; hotfixes have a clear landing spot; main
stays "latest + all fixes + deployable".
With release branches: the published state = `release/vX.Y.x` branch +
the concrete `vX.Y.Z` tag, exactly reproducible; hotfixes have a clear landing
spot; main stays "latest + all fixes + deployable".
> 中文版见 [docs/git-workflow.md](git-workflow.md)。

View File

@ -13,7 +13,7 @@ ModelRouter 采用 **GitHub Flow + 发布分支** 模型:`main` 是唯一长
|---|---|---|---|
| 主分支 | `main` | 永久 | 唯一长命分支。永远可部署。积攒下一个版本的功能。 |
| 特性分支 | `feature/<描述>` | 短命(开发→合并即删) | 新特性 / 一般 bug 修复。从 `main` 开出,完成后合回 `main`。 |
| 发布分支 | `release/vX.Y.Z` | 一个版本周期 | 从 `main` 分出,打 tag 发布。该版本生命周期内的 hotfix 都提交在此分支。 |
| 发布分支 | `release/vX.Y.x` | 一个版本周期 | 从 `main` 分出,打 tag 发布。该版本生命周期内的 hotfix 都提交在此分支。 |
## 变更流向(重要)
@ -24,15 +24,20 @@ ModelRouter 采用 **GitHub Flow + 发布分支** 模型:`main` 是唯一长
│ │
│ 切出 │ 切出
▼ ▼
release/v1.4.2 release/v1.4.3
release/v1.6.x release/v1.7.x
│ │
tag: v1.4.2 tag: v1.4.3
tag: v1.6.0 tag: v1.7.6
│ │
hotfix ◄─────┘ hotfix ◄─────┘
│ │
└── cherry-pick 回 main ───────┘
```
分支名里 patch 位是**字面的 x**,而 tag 打具体版本号:`release/v1.7.x` 这一条
发布分支上的 tag 可以有 v1.7.0 … v1.7.6 多个。一条 minor 分支跨多个 patch 是
刻意的:patch 是同一批功能的不同修订,hotfix 落在同一条分支上,回流 main 时
也不必处理多条 release 分支之间的依赖。
### 关键规则
1. **main 永远可部署**:不在 main 上留半成品。任何未完成的工作必须在特性分支上。
@ -40,16 +45,16 @@ ModelRouter 采用 **GitHub Flow + 发布分支** 模型:`main` 是唯一长
开发完 `git merge --no-ff feature/xxx` 或 squash 合回。
3. **发布 = 从 main 切 release 分支 + 打 tag**:
```bash
git checkout -b release/v1.4.2 main
git tag -a v1.4.2 -m "ModelRouter v1.4.2"
git push origin release/v1.4.2 v1.4.2
git checkout -b release/v1.7.x main
git tag -a v1.7.0 -m "ModelRouter v1.7.0"
git push origin release/v1.7.x v1.7.0
```
构建安装包、上传 GitCode Release 都基于这个 tag,保证可精确回溯发布态。
4. **版本生命周期内只收该版本的 hotfix**:新特性一律并入 `main` 等下一个版本,
绝不塞进已发布的 release 分支(除非主动选择在该版本内发次要版)。
5. **hotfix 必须回流 main**:
```bash
git checkout release/v1.4.2 # 在发布分支提交修复
git checkout release/v1.7.x # 在发布分支提交修复
git commit -m "fix: ..."
git checkout main
git cherry-pick <hotfix-commit> # 回主分支
@ -61,11 +66,20 @@ ModelRouter 采用 **GitHub Flow + 发布分支** 模型:`main` 是唯一长
下一个版本发布时,上一个 release 分支退役:
- **默认:直接删除远端 release 分支**(`git push origin :release/v1.4.2`)。
- **默认:直接删除远端 release 分支**(`git push origin :release/v1.7.x`)。
因为 hotfix 都已逐个 cherry-pick 回 main,main 已包含全部修复,无需再合并。
- **如需要长期维护旧版**(例如企业大客户卡在旧版本):保留分支,仅 stopship 接受
该版本的安全修复,继续走「提交 + cherry-pick 回 main」循环。
> **实践与本节的历史出入(2026-10-01 核对)**:`release/v1.4.x` 与 `release/v1.5.x`
> 至今仍在本地与远端,说明"下一个版本发布就删上一个分支"实际没有执行。
> 保留无害(hotfix 已回流),但它与上面写的规则不一致,读文档的人会以为
> 这些分支不该存在。**特性分支则确实在清理**:`feature/key-quota-control`、
> `feature/toolcall-id-sanitize`、`feature/anthropic-usage-cache`、
> `feature/agentrouter-id-sanitize` 四个分支在 2026-10-01 删除——它们的工作
> 早已全部进入 main(逐提交用 `git patch-id` 比对确认),留着只是给下个版本
> 制造 cherry-pick/merge 陷阱。
## 明确不做的事
- **不 rebase main**:`main` 的历史保持追加式,任何人拉取后 `git pull` 都得到直接可用的历史。

507
docs/plugins.md Normal file
View File

@ -0,0 +1,507 @@
# 插件系统(Plugin System)
> **English**: this document is the reference for writing ModelRouter plugins.
> The Chinese version is the primary one; section titles map 1:1.
ModelRouter 的插件是**单个 `.lua` 文件**,放在 `config.yaml` 的 `plugin_dir` 目录里。
插件能做两件事:
1. **挂钩子**:在请求流水线的若干 stage 上注册回调,看到每个请求的完整信息,
并可以把结果累加进自己的状态。
2. **贡献界面**:在启动时返回 HTML / CSS / JS,由内核注入 WebUI——可以是一整个
新页面,也可以是往现有页面里追加一个组件。
两者互相独立:只想统计请求数的插件不必碰界面;只想加个仪表盘的插件不必碰钩子。
---
## 1. 快速上手
一个最小的插件:
```lua
-- plugins/hello.lua
local plugin = {
name = "hello",
version = "1.0.0",
description = "示例插件",
author = "you",
}
-- 声明钩子
plugin.hooks = {
request_end = "on_request_end",
}
-- 钩子实现
function plugin.on_request_end(payload)
-- payload 是解码后的 table,不是 JSON 字符串
log("info", string.format("%s via %s: %d prompt tokens",
payload.model, payload.source, payload.prompt_tokens or 0))
return nil -- 最后一个 stage 没有下游,return 无意义
end
-- 贡献界面
plugin.ui = {
page = {
page_id = "hello", -- kebab-case
title = "Hello",
icon = "👋",
order = 90, -- 侧栏排序
mount = [[<div id="hello">hello</div>]],
},
}
return plugin -- 必须返回一个 table
```
放进 `plugin_dir` 后重启即生效。`GET /api/plugins` 确认它被加载了。
---
## 2. 加载与生命周期
```
core.New
└─ lua.NewVM(adapter_dir).Start() 适配器状态
└─ lua.NewPlugins(vm, plugin_dir)
├─ SeedBundled() 仅当目录不存在时写入内置插件(目前是 billing)
└─ LoadDir() 按文件名字典序逐个加载
```
**加载失败不影响网关启动。** 一个语法错误的插件会被记录在 `GET /api/plugins`
的 `error` 字段里,永远不会被调用。这与适配器一致,但理由更强:插件是可选的
第三方扩展,因为一个 `.lua` 打错字就让网关起不来是错误的取舍。
**目录一旦存在就是权威的。** 与适配器同规则:首启会 seed 内置插件,之后目录里
的文件说了算,删除或编辑内置插件都是真实生效的操作。
### 2.1 热更新
| 方式 | 效果 |
|---|---|
| `POST /api/plugins {name, code}` | 写文件 + 立即加载新版本(旧的 Lua 状态被关闭重建,**累计量清零**) |
| `DELETE /api/plugins/{name}` | 删文件 + 卸载 |
| 改文件后 `POST` 同名 | 同上 |
改文件但**不** POST,需要重启才生效。
---
## 3. 流水线 stage
一个请求依次经过三个 stage。插件可以为任意 stage 注册钩子;未注册的 stage
被忽略,所以插件不会因为网关将来新增 stage 而报错。
```
客户端请求
│
┌─────────▼──────────┐
│ request_start │ 已解析、已鉴权,尚未选源
│ · type │ "chat" | "stream" | "image"
│ · model │ 客户端请求的原始 model("AUTO" 也在这里)
│ · key / role │ 掩码后的网关 key id("***a1b2c3")与角色
│ · source │ 空(还没选源)
│ · stream │
│ · messages_count │
│ · tools_count │
│ · ts │ unix 秒
└─────────┬──────────┘
│ (调度:tier 遍历 → 槽位轮转 → 冷却/配额过滤)
┌─────────▼──────────┐
│ routed │ 已选定 (source, model),尚未发往上游
│ · source / model │ 实际选中的
│ · tier │ AUTO 链的档位;直连 = -1;AUTO = -2
│ · stream / key │
└─────────┬──────────┘
│ (HTTP 往返 / SSE 流)
┌─────────▼──────────┐
│ request_end │ 每个请求恰好一次,成功失败都触发
│ · ok / status │
│ · latency_ms │
│ · first_byte_ms │ 流式的首字节时间
│ · prompt_tokens │ 上游真实 usage,缺失时为字节估算
│ · completion_tokens
│ · cache_hit_tokens / cache_miss_tokens
│ · image_count │ 生图数量(图片不计 token)
│ · error │ 失败原因,成功时为 ""
│ · time │ unix **毫秒**
└─────────┬──────────┘
│
写审计 + 聚合统计
```
### 3.1 触发点在哪
| stage | 代码位置 | 说明 |
|---|---|---|
| `request_start` | `gateway/chat.go` `handleChat` / `fireImageStart` | 每个请求一次(chat 与生图各一条),在配额闸门**之前** |
| `chain_step` | `gateway/chat.go` `chainTraceSink` | **仅 AUTO 路径**,每步一次 |
| `routed` | `singleChat` / `streamChat` / `singleChatAuto` / `streamChatAuto` | 成功选定源之后,各一次 |
| `request_end` | `gateway/chat.go` `writeRec` | **所有出口的唯一汇合点**,每个请求一次 |
### 3.1b 为什么单独有 `chain_step`
`routed` 只在**遍历结束后**触发一次,只带最终胜出的槽位。所以
"tier 1 冷却所以降级到 tier 3"和"tier 1 正常接单"在它眼里**完全一样**——
而这恰恰是优先级链存在的全部理由。
`chain_step` 补上这条信息,四种 `kind`:
| kind | 含义 | 何时产生 |
|---|---|---|
| `tier_skip` | 整档被跳过 | 该档所有槽位冷却中/配额用尽 |
| `slot_fail` | 某个槽位硬失败 | 上游报错 / 适配器输出不可用 |
| `tier_busy` | 整档全忙且有界等待超时 | 2s 内没等到空位 |
| `selected` | 这个槽位接了单 | 每次成功遍历**恰好一次**,且是最后一步 |
顺序保证:所有 `chain_step` 都在 `routed` 之前,`selected` 是最后一步。
所以只订阅 `request_end` 的插件也能拿到轨迹摘要(见下)。
### 3.1c `request_end` 里的轨迹摘要
除了逐个 `chain_step`,`request_end` 还带三个便于做报表的字段:
| 字段 | 含义 |
|---|---|
| `chain_walk` | 整个遍历的步骤数组(上限 12 步,超出截断) |
| `degraded` | 布尔。`true` = 有过跳过/失败,即**发生了降级** |
| `tier_served` | 实际服务的那一档;直连或全失败时为 `-1` |
> **计费口径**:按**实际服务的模型**计费。降级到 tier 3 仍按 tier 3 的价算,
> `chain_step` / `degraded` / `tier_served` 只作**观测**,不参与计价。
> 理由见 §7.5。
`request_end` 放在 `writeRec` 是因为四条入口路径(直连/AUTO × 流式/非流式)都
经过它,既不会漏(流式的 token 数只有流结束才知道),也不会重复。
**hot path 注意事项**:没有插件注册某 stage 时,`Fire` 立刻返回(一次 `RLock`
加一次 map 查找)。装了插件之后,每个请求会在该 stage 上多一次 Lua 调用——
这是同步的,在关键路径上。计费插件那种"每请求一次"是正常的;把重活放进钩子是
反模式。
### 3.2 钩子的返回值
- 返回 `nil` 或不返回 = **没有意见**,payload 原样传给下一个插件
- 返回 table = 其中的键会**合并进 payload**,并作为 `Fire` 的返回值
**同一 stage 的多个插件并行执行**,但返回值按**插件加载顺序**合并,所以结果是
确定的(不依赖 goroutine 调度)。代价是一个插件看不到另一个插件刚加的字段:
每个钩子拿到的是**同一份 payload 快照**。
这与早期版本不同 —— 早期是顺序执行,后一个插件能看到前一个的返回值。它从未被
实际依赖(随核心发布的 billing 在每个 stage 都 `return nil`,注释里写着
"nobody downstream would read a return value"),但这是一处**契约变化**:如果你的
插件依赖「读到前一个插件写的字段」,并行的两个插件之间必须改用外部通信
(例如各自写 `plugin.state`,由 `/api/plugins/<name>/state` 读取)。
单个插件时不启 goroutine,直接调用。
前三个 stage 的返回值目前没有内部消费者(最后一个 stage 之后就是写审计),
所以计费插件改用 `plugin.state` + `/state` 端点来暴露数据。
---
## 4. 界面扩展
### 4.1 整页
```lua
plugin.ui = {
page = {
page_id = "billing", -- 必填,kebab-case。侧栏 data-tab 与 #tab-billing
title = "Billing", -- 必填,侧栏文字
icon = "💰", -- 可选
order = 40, -- 侧栏排序,默认 100
mount = [[...HTML...]],
},
}
```
### 4.2 往现有页面追加元素
```lua
plugin.ui = {
elements = {
{
target = "status", -- status | chat | keys | sort | sources | adapters
anchor = "top", -- "top" | "bottom" | "before:<sel>" | "after:<sel>"
order = 5,
mount = [[...HTML...]],
},
},
}
```
### 4.3 `mount` 里可以带 `<script>` 和 `<style>`
内核的注入顺序是:**先插 HTML,再执行 `<script>`**,所以脚本里访问
`document.getElementById` 一定能拿到已渲染的节点。
### 4.4 插件可用的浏览器端 API
| API | 作用 |
|---|---|
| `window.pluginAPI.fetchState(name)` | 等价于 `GET /api/plugins/<name>/state` |
| `window.pluginAPI.onTabShown(fn)` | 注册"页面切到可见时"的回调(轮询类组件用) |
| `window.pluginAPI.postState(name, obj)` | `PUT` 自己的 state(写操作,需 admin) |
### 4.5 一个完整例子
见 `internal/lua/plugins/billing.lua`——它同时用了两种 UI 形式、完整的价格配置、
以及全部三个 stage。
---
## 5. 插件状态与 HTTP API
### 5.1 端点
```
GET /api/plugins 已加载插件清单 + 钩子 + UI + 错误
POST /api/plugins 上传/替换(admin){name, code}
DELETE /api/plugins/{name} 删除(admin)
GET /api/plugins/{name}/state 读取插件自己发布的状态
PUT /api/plugins/{name}/state 替换状态(admin)
```
`GET .../state` 对**任意角色开放**:它是报表数据(开销、计数),用户自己的
计费组件要能渲染。而 `PUT` 需要 admin。
### 5.2 约定:`prices` 与 `state` 分离
`PUT .../state` 的载荷里如果有 `prices` 键,它会被写进插件的 **`plugin.prices`**
字段并从 `state` 里剔除。
**为什么**:`state` 是累计量(历史),`prices` 是配置。一次改价如果整体替换
`state`,累计量就没了——账目会在改价那一刻清零。分离之后:
- 只带 `prices` → 改配置,**不动** `state`(累计量保留)
- 带其它键 → 替换 `state`(这是显式的重置)
> 插件可以不遵守这个约定,把整个载荷放进 `state` 自行处理。约定只为内置的
> 计费插件存在。
### 5.3 `GET /api/plugins` 的响应
```json
{
"plugins": [{
"name": "billing", "version": "1.0.0",
"description": "...", "author": "...",
"hooks": ["request_end"],
"ui": { "page": "billing", "elements": 1 },
"loaded": true
}],
"hook_errors": { "request_end": { "count": 3, "last_error": "billing: ..." } },
"plugin_dir": "/etc/llmsproxy/plugins"
}
```
`hook_errors` 是**排错入口**:一个坏掉的插件表现为"功能缺失"而不是报错,
这个计数让它可见。
---
## 6. 运行时约束(重要)
### 6.1 一个插件 = 一个 Lua 状态
适配器可以有多个独立的 worker 状态(它们无状态,这是对的)。**插件不行**:
钩子通常往 `plugin.state` 里累加,多个状态就意味着总额被劈成几份,而写价格只
写进其中一个状态,钩子恰好跑到另一个时**所有请求按 0 计费**。
因此插件持有**唯一一个**状态,所有入口(`Fire` / `State` / `SetState`)用互斥锁
串行化。代价:**一个卡住的钩子会卡住所有插件的钩子**。所以——
> **钩子必须短、同步、不阻塞。** 不要在里面做 HTTP 请求、睡眠或重计算。
### 6.2 异常被隔离
钩子里 `error()` 不会影响转发:异常被捕获、记进 `hook_errors`、跳到下一个插件。
适配器变换出错会让源进入冷却,但插件出错**不产生任何惩罚**——插件是可选功能。
### 6.3 内置可用函数
与适配器相同:`json.encode` / `json.decode`、`log(level, msg)`、
`hmac_sha256_hex` / `sha256_hex` / `base64_encode` / `tohex`。
⚠️ `os.date` 在极简运行时里可能不可用(计费插件因此有降级路径)。
### 6.4 命名
- 全局 `__llmsproxy_plugin` 存放插件返回的 table,**不要占用**
- 插件间状态完全隔离,一个插件改不了另一个的全局
---
## 7. 内置计费插件
`plugins/billing.lua`,默认随首启 seed 进来,开箱可用。
### 7.1 计费维度
| 维度 | 用途 | 优先级 |
|---|---|---|
| `sources` | 源的固定价(如按请求计费的转发服务) | 中 |
| `models` | 模型的 per-token 价 | 高 |
| `keys` | 单个网关 key 的覆盖价 | **最高** |
**token 价**优先级:`keys` > `models` > `default`。
**`per_request` 固定价是叠加的**(不覆盖),所以一个生图模型可以既算 token 又收固定费。
**`cache_discount`** 见 §7.7。
### 7.2 价格单位
**USD / 单个 token**。这是各家 provider 的公布口径,所以典型数值长这样:
`1.25e-6`。插件内部乘以 token 数,全程不做单位换算。
```json
{
"prices": {
"currency": "USD",
"default": { "prompt": 0, "completion": 0, "per_request": 0 },
"sources": { "trae": { "per_request": 0.01 } },
"models": {
"gpt-5.4": { "prompt": 1.25e-6, "completion": 1e-5 },
"kolors": { "per_request": 0.04 }
},
"keys": { "***a1b2c3": { "prompt": 1.1e-6, "completion": 9e-6 } }
}
}
```
配置价目:
```bash
curl -X PUT http://127.0.0.1:8080/api/plugins/billing/state \
-H "Authorization: Bearer $ADMIN_KEY" -H "Content-Type: application/json" \
-d '{"prices":{"models":{"gpt-5.4":{"prompt":1.25e-6,"completion":1e-5}}}}'
```
读回账目:
```bash
curl -H "Authorization: Bearer $ADMIN_KEY" \
http://127.0.0.1:8080/api/plugins/billing/state
```
### 7.3 累计维度
`total` / `by_source` / `by_model` / `by_key` / `by_day`(`YYYY-MM-DD` UTC),
每项含 `cost`、`requests`、`prompt_tokens`、`completion_tokens`、`failures`。
另有两个**降级观测**维度(来自 `chain_step`):
| 字段 | 含义 |
|---|---|
| `degraded_reqs` | 发生过降级的请求数 |
| `by_tier_served` | 各档实际接单数(`{"1": 812, "2": 37}`) |
| `skip_reasons` | 跳过原因计数,等待时长已归一(`no free slot within <wait>`) |
这三项是"网关是不是在悄悄降级"的核心指标:一个持续降级的网关,账单结构和健康
网关看起来一模一样——**除非**单独统计降级次数。
### 7.4 计费策略:失败的请求怎么算
**保留 token 费用,丢弃固定费用。** 理由:上游在生成后才 500,token 确实被消耗
了;但那个从未真正发生的固定费不该收。
这是本插件里**最可争议的一条**。想改成"失败也收固定费":
```json
{ "prices": { ... }, "count_failures": true }
```
### 7.5 ⚠️ 计费插件只报表,不执法
**网关自己的配额会计(`internal/gateway/stats.go`,入口处强制)才是限额权威。**
本插件不参与任何路由或配额决策。
理由:两套独立的会计路径如果对不上,比一套功能略少的更糟。计费是**观察**,
配额是**控制**,二者分开。
### 7.6 未定价流量(重要)
**任何维度都没配价的请求,成本记 0。** 这是最危险的失败模式:账单照样能加总,
只是**悄悄少报**,而且没有任何报错。
所以插件单独统计它们:
| 字段 | 含义 |
|---|---|
| `unpriced_reqs` | 没有任何价目覆盖的请求数 |
| `unpriced_models` | 按模型点名(`{"MYSTERY-MODEL": 12}`)——直接告诉你价目表缺了哪一行 |
仪表盘上有 "Unpriced" 卡片。**这个数应该是 0**;不是 0 就去补价目。
注意"未定价"不等于"免费":这些请求的 `requests` / token 数**照常计入**
`total` 与各维度,只有金额是 0。
### 7.7 提示缓存计价
**缓存命中的 prompt token 不按全价算。** 绝大多数 provider 对缓存读给很深的折扣
(常见是 1/10),而 agent 流量会反复重放长前缀——正是缓存要让它便宜的那类流量。
```
fresh = prompt_tokens - cache_hit_tokens → 全价
cached = cache_hit_tokens → 全价 × cache_discount
```
`cache_discount` 默认 **0.1**(10 倍折扣),因为 DeepSeek / Qwen / Kimi 等都是这个
量级。它是**每个 provider 的事实、不是自然常数**,所以可以按条目覆盖:
```json
"models": { "gpt-5.4": { "prompt": 1.25e-6, "completion": 1e-5, "cache_discount": 0.25 } }
```
设成 `1` 恢复成旧的"prompt 一律全价"行为,设成 `0` 表示该 provider 不打折。
优先级与 token 价一致(`keys` > `models` > `default`)。
> **这一条改过行为。** 修复前缓存命中按全价算:100 万 prompt token 里 90 万是
> 缓存命中,会算出 10 USD 而不是 ~1.9——**高估约 10 倍**,而且恰好发生在缓存
> 最有价值的高频流量上。
### 7.8 数据从哪来
`request_end` 的 `prompt_tokens` / `completion_tokens` 优先取上游真实的
`usage`;上游没报时网关用字节估算(`len/3+1`)。流式请求在流结束后用上游真实
数字**覆盖**估算值。所以计费数字的精度取决于上游是否报 usage。
---
## 8. 排错
| 现象 | 查什么 |
|---|---|
| 插件没出现在 `/api/plugins` | 目录对不对(响应里的 `plugin_dir`);文件是不是 `.lua` |
| `loaded: false` 且有 `error` | 语法错误或没 `return table`,错误信息在 `error` 字段 |
| 功能"没反应"但无报错 | 查 `hook_errors`——坏钩子只记录不抛出 |
| 钩子没被调用 | 该 stage 确实触发了吗(`routed` 只在**成功**选源后触发,调度全失败时不触发) |
| 界面空白 | 内核的 `GET /api/ui-inject` 里有没有你的 `page_id`;脚本有没有报错(浏览器 console) |
| `pluginAPI` 未定义 | 脚本在注入前执行了;确认用的是 `mount` 而不是别的方式插入 |
### 调试钩子
插件的 `log(level, msg)` 输出到网关的日志:
```
journalctl -u llmsproxy -f | grep -i plugin
```
---
## 9. 与适配器的区别
| | 适配器 | 插件 |
|---|---|---|
| 位置 | `adapter_dir` | `plugin_dir` |
| 作用 | 上游协议转换 | 请求流水线 + 界面 |
| 钩子 | `transform_request` / `transform_response` / `transform_stream_chunk` / `build_headers` / `transform_error` | `request_start` / `routed` / `request_end` |
| 状态 | 每个源独立,多 worker | 单状态,见 §6.1 |
| 出错后果 | 该 (源,模型) 进入冷却退避 | 仅该功能缺失 |
| 鉴权 | 需要 `gateway_keys` 之外的独立凭据 | 网关 key |
| 必需性 | 核心 | 可选,缺了网关照常跑 |
完整适配器协议见 [lua-adapters.md](lua-adapters.md)。

264
internal/billing/compile.go Normal file
View File

@ -0,0 +1,264 @@
package billing
import (
"fmt"
"strings"
"llmsproxy/internal/config"
)
// Compile turns one billing profile into the prices table the billing plugin
// expects, resolving URL rules against the gateway's actual sources.
//
// WHY URL RULES NEED RESOLVING AT ALL: the operator declares pricing by URL
// because that is what a provider's price list is keyed on, and because several
// sources can point at the same URL. The plugin, however, looks up by
// `payload.source` (a source NAME) and by model — it has no idea what URL a
// request went to. So the URL match happens here, at compile time, where the
// config's name->base_url mapping is known, and the result is expressed in the
// dimensions the plugin already supports.
//
// Profiles are the "let the user choose" axis: the same URL can appear in
// several profiles and switching recomputes this table, so a gateway can be
// repriced without editing the plugin.
func Compile(profile *config.BillingProfile, sources []config.Source) (map[string]interface{}, error) {
return CompileOpts(profile, sources, false)
}
// CompileOpts is Compile with an explicit policy for rules that match no
// source.
//
// The default is STRICT and that is correct for startup: a profile with a
// typo'd URL prices nothing, every request on it is recorded as unpriced, and
// the bill silently comes out wrong. Failing the load is the right response.
//
// The rules editor needs the opposite. An operator editing a half-finished
// profile — adding a rule before the source that will use it exists, or
// renaming a URL while typing — must be able to save and then see the warning,
// not be blocked by an error they can only resolve by guessing. Lenient mode
// keeps the offending rule in the config and drops it from the compiled table,
// so it is preserved as text and clearly reported, without pretending it
// prices anything.
func CompileOpts(profile *config.BillingProfile, sources []config.Source, lenient bool) (map[string]interface{}, error) {
if profile == nil {
return nil, fmt.Errorf("no billing profile")
}
prices := map[string]interface{}{
"currency": "USD",
"default": map[string]interface{}{"prompt": 0.0, "completion": 0.0, "per_request": 0.0},
"sources": map[string]interface{}{},
"models": map[string]interface{}{},
"keys": map[string]interface{}{},
}
if profile.Default != "" {
prices["default_mode"] = profile.Default
}
// Currency is a profile-level statement; a rule may override it.
for i := range profile.Rules {
if c := profile.Rules[i].Currency; c != "" {
prices["currency"] = c
break
}
}
matched := map[string]bool{}
for i := range profile.Rules {
rule := &profile.Rules[i]
targets := matchSources(rule.URL, sources)
if len(targets) == 0 && rule.URL != "*" {
// A rule for a URL no source uses is almost always a typo (or a
// source that was removed). Failing loudly beats a profile that
// silently prices nothing — the whole reason this is a config file
// instead of a hand-written JSON blob.
if lenient {
continue // kept in the config text, reported as a warning
}
return nil, fmt.Errorf("rule url %q matches no configured source base_url", rule.URL)
}
for _, srcName := range targets {
if matched[srcName] {
// First rule wins. Two rules matching one source is ambiguous,
// and silently letting the later one win makes the file's
// meaning depend on ordering the operator cannot see.
continue
}
matched[srcName] = true
if err := applyRule(prices, rule, srcName, sources); err != nil {
return nil, err
}
}
}
// Sources no rule matched fall through to the profile default, which the
// plugin applies via prices.default. Recording them explicitly means the
// UI can say "this source is unpriced" instead of leaving the operator to
// infer it from a zero.
var unmatched []string
for _, src := range sources {
if !matched[src.Name] {
unmatched = append(unmatched, src.Name)
}
}
if len(unmatched) > 0 {
prices["unmatched_sources"] = unmatched
}
return prices, nil
}
// matchSources returns the source names whose base_url matches pattern.
// "*" matches every source (used as a catch-all default rule).
func matchSources(pattern string, sources []config.Source) []string {
var out []string
if pattern == "*" {
for _, s := range sources {
out = append(out, s.Name)
}
return out
}
want := normalizeURL(pattern)
for _, s := range sources {
if normalizeURL(s.BaseURL) == want {
out = append(out, s.Name)
}
}
return out
}
// normalizeURL compares URLs the way an operator expects: trailing slashes and
// case in the host are differences the provider's price list does not care
// about, and requiring an exact byte match would make the config brittle.
func normalizeURL(u string) string {
u = strings.TrimSpace(u)
u = strings.TrimRight(u, "/")
return strings.ToLower(u)
}
// applyRule writes one rule's pricing for one source into the prices table.
func applyRule(prices map[string]interface{}, rule *config.BillingRule, srcName string, sources []config.Source) error {
switch rule.Mode {
case "free":
// Explicitly priced at zero. This is NOT the same as unpriced: a source
// the operator says is free must not appear in the unpriced warnings,
// or those warnings become noise and stop being read.
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
}
return nil
case "subscription":
// A fixed monthly commitment: the per-request MARGINAL cost is zero, and
// the flat fee is reported separately. Spreading a monthly fee across
// requests would invent a per-request number the provider never charges,
// and it would change every time traffic did.
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
}
fixed, _ := prices["subscriptions"].(map[string]interface{})
if fixed == nil {
fixed = map[string]interface{}{}
prices["subscriptions"] = fixed
}
cur := "USD"
if rule.Currency != "" {
cur = rule.Currency
}
fixed[srcName] = map[string]interface{}{
"monthly": rule.Subscription, "currency": cur,
}
return nil
case "unpriced":
// Deliberately left out of `sources` so the plugin's unpriced_models /
// unpriced_reqs counters catch it. That is the point: a subscription
// plan whose credits cannot be converted to tokens must be VISIBLE as
// unbilled, not quietly estimated.
return nil
case "token":
if len(rule.Models) == 0 {
return fmt.Errorf("rule for url %q: mode token requires at least one model", rule.URL)
}
models := prices["models"].(map[string]interface{})
for modelID, t := range rule.Models {
entry := map[string]interface{}{}
prompt, err := usdPerM(t.Prompt)
if err != nil {
return fmt.Errorf("model %q prompt: %w", modelID, err)
}
completion, err := usdPerM(t.Completion)
if err != nil {
return fmt.Errorf("model %q completion: %w", modelID, err)
}
entry["prompt"] = prompt
entry["completion"] = completion
if t.CacheDiscount != nil {
entry["cache_discount"] = *t.CacheDiscount
}
if rule.Peak != nil {
entry["peak"] = compilePeak(rule.Peak)
}
models[modelID] = entry
}
// A specific-URL token rule needs a source entry so a model served
// from THIS url but absent from `rule.Models` is still recognised as
// belonging to a priced provider.
//
// A WILDCARD rule ("*") must NOT get one. "*" is a model catalogue — it
// says "these model ids have these prices, wherever they come from" —
// and marking every source priced at {0,0} would make priceFor() set
// priced=true for every model the catalogue does NOT list. Paid
// sources (commandcode, alittokenplan, …) would then record $0 instead
// of showing up in unpriced_reqs, which is precisely the silent
// zero-billing failure this whole plugin is built to make visible.
// The model lookup p.models[payload.model] already marks a listed
// model priced on its own, so the wildcard loses nothing.
if rule.URL != "*" {
if _, ok := prices["sources"].(map[string]interface{})[srcName]; !ok {
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
}
}
}
return nil
}
return fmt.Errorf("unknown mode %q", rule.Mode)
}
// usdPerM converts a USD-per-million string into the per-token rate the plugin
// expects. Delegates to config.ParseFloatUSDPerM so the DSL validator and the
// compiler agree byte-for-byte on what a valid price is — a value that passes
// Validate() but fails here (or vice versa) would be the worst kind of drift.
func usdPerM(s string) (float64, error) {
return config.ParseFloatUSDPerM(s)
}
// compilePeak renders the peak window in the shape the plugin reads:
// { multiplier, windows = { { days = {...}, hours = { {lo,hi}, ... } } } }.
//
// The plugin reads `days` / `hours` pairs. The first version of the deployed
// price table used { start, end, weekdays } — a shape nothing reads — so peak
// traffic was billed at off-peak rates with no error anywhere. Compiling from
// typed config fields removes the chance of writing the wrong key names by hand.
func compilePeak(p *config.BillingPeak) map[string]interface{} {
win := map[string]interface{}{}
if len(p.Weekdays) > 0 {
days := make([]interface{}, 0, len(p.Weekdays))
for _, d := range p.Weekdays {
days = append(days, d)
}
win["days"] = days
}
if len(p.Hours) > 0 {
hours := make([]interface{}, 0, len(p.Hours))
for _, h := range p.Hours {
hours = append(hours, []interface{}{h[0], h[1]})
}
win["hours"] = hours
}
out := map[string]interface{}{"multiplier": p.Multiplier}
if len(win) > 0 {
out["windows"] = []interface{}{win}
}
return out
}

View File

@ -0,0 +1,88 @@
package billing
import (
"testing"
"llmsproxy/internal/config"
)
// The wildcard token rule must not blanket-mark every source as priced.
//
// Why this is a hazard and not a style question: priceFor() sets priced=true as
// soon as the request's SOURCE has an entry in the price table, and that flag
// is what keeps a request OUT of unpriced_reqs. A "*" rule is a model
// catalogue — it prices specific model ids wherever they come from — but
// compiling it into a {0,0,0} entry on all 21 sources made every model the
// catalogue does not list look "priced at zero". Paid providers (commandcode,
// alittokenplan, …) then stopped appearing in the unpriced warnings, which is
// exactly the silent zero-billing failure the plugin exists to surface.
func TestWildcardTokenRuleDoesNotMarkEverySourcePriced(t *testing.T) {
sources := []config.Source{
{Name: "free1", BaseURL: "https://free1.example.com/v1"},
{Name: "paid1", BaseURL: "https://paid1.example.com/v1"},
{Name: "paid2", BaseURL: "https://paid2.example.com/v1"},
}
profile := &config.BillingProfile{
ID: "p",
Rules: []config.BillingRule{
{URL: "https://free1.example.com/v1", Mode: "free"},
{
URL: "*", Mode: "token",
Models: map[string]config.BillingToken{
"only-catalogued-model": {Prompt: "1.00", Completion: "2.00"},
},
},
},
}
prices, err := Compile(profile, sources)
if err != nil {
t.Fatalf("compile: %v", err)
}
srcs := prices["sources"].(map[string]interface{})
// The explicitly free source keeps its entry (free is a deliberate 0).
if _, ok := srcs["free1"]; !ok {
t.Error("the explicit free rule lost its source entry")
}
// Paid sources must NOT be marked, or their unlisted models bill $0.
for _, name := range []string{"paid1", "paid2"} {
if _, ok := srcs[name]; ok {
t.Errorf("★ wildcard token rule marked %q as priced at 0 — its unlisted "+
"models will bill $0 instead of appearing as unpriced", name)
}
}
// The catalogue itself must still be present, or the rule prices nothing.
models := prices["models"].(map[string]interface{})
if _, ok := models["only-catalogued-model"]; !ok {
t.Error("the wildcard rule did not publish its model price")
}
}
// The inverse guard: a token rule for a SPECIFIC url still marks that source,
// because it genuinely declares pricing for the models served from it.
func TestSpecificURLTokenRuleStillMarksItsSource(t *testing.T) {
sources := []config.Source{
{Name: "mine", BaseURL: "https://mine.example.com/v1"},
{Name: "other", BaseURL: "https://other.example.com/v1"},
}
profile := &config.BillingProfile{
ID: "p",
Rules: []config.BillingRule{{
URL: "https://mine.example.com/v1", Mode: "token",
Models: map[string]config.BillingToken{
"m": {Prompt: "1.00", Completion: "2.00"},
},
}},
}
prices, err := Compile(profile, sources)
if err != nil {
t.Fatalf("compile: %v", err)
}
srcs := prices["sources"].(map[string]interface{})
if _, ok := srcs["mine"]; !ok {
t.Error("a specific-url token rule must still mark its own source priced")
}
if _, ok := srcs["other"]; ok {
t.Error("an unrelated source was marked priced by a specific-url rule")
}
}

View File

@ -7,6 +7,8 @@ import (
"fmt"
"os"
"path/filepath"
"strconv"
"strings"
"time"
"gopkg.in/yaml.v3"
@ -19,6 +21,7 @@ type Config struct {
GatewayKeys []string `yaml:"gateway_keys"`
DefaultModel string `yaml:"default_model"` // e.g. "AUTO" or a model id
AdapterDir string `yaml:"adapter_dir"`
PluginDir string `yaml:"plugin_dir,omitempty"` // Lua plugin directory (optional; hooks + WebUI extensions)
RuntimeFile string `yaml:"runtime_file"`
TLSCertFile string `yaml:"tls_cert_file,omitempty"` // PEM cert; when set together with tls_key_file, serve HTTPS
TLSKeyFile string `yaml:"tls_key_file,omitempty"` // PEM private key
@ -27,6 +30,15 @@ type Config struct {
Auto []ModelScope `yaml:"auto,omitempty"` // AUTO 调度链规则(WebUI 优先级页编辑,chat)
AutoImage []ModelScope `yaml:"auto_image,omitempty"` // AUTO 生图调度链规则(WebUI 优先级页·生图)
Keys []GWKey `yaml:"keys,omitempty"` // 网关密钥(WebUI 密钥页管理)
// BillingDSL declares per-URL pricing profiles for the billing plugin.
//
// Declarative, because hand-writing the prices JSON was error-prone in ways
// that looked successful: the first PUT went out without the required
// `prices` wrapper and silently REPLACED the accumulated totals with the
// price table; the second used a peak-window shape the plugin does not read,
// so peak traffic was billed at off-peak rates — no error, the bill just
// quietly halved. A schema-checked config file catches both at load.
BillingDSL *BillingDSL `yaml:"billing,omitempty"`
// box seals credentials (sources' api_key/headers, keys' key) at rest.
// In-memory values are always plaintext; only the bytes on disk are sealed.
// Wired by AttachSecretBox — Load leaves it nil so `-check` and tests stay
@ -132,6 +144,7 @@ func writeDefaultConfig(path string) error {
GatewayKeys: []string{key},
DefaultModel: "AUTO",
AdapterDir: filepath.Join(abs, "adapters"),
PluginDir: filepath.Join(abs, "plugins"),
RuntimeFile: filepath.Join(abs, "runtime.json"),
}
out, err := yaml.Marshal(&cfg)
@ -288,6 +301,9 @@ func (c *Config) ApplyDefaults() error {
if c.AdapterDir == "" {
c.AdapterDir = "adapters"
}
// PluginDir is deliberately NOT defaulted: with no plugins a missing
// directory would be created and seeded on every start for nothing. An
// operator opts in by setting it (or by dropping files into it).
if c.RuntimeFile == "" {
c.RuntimeFile = "runtime.json"
}
@ -377,6 +393,144 @@ type GWKey struct {
Seed bool `yaml:"seed,omitempty" json:"seed,omitempty"` // true if migrated from config gateway_keys
}
// BillingDSL holds declarative per-URL pricing profiles for the billing plugin.
//
// Profiles are the "let the user pick" axis: the same upstream URL can be
// declared under multiple profiles, and the operator chooses which is active.
// Switching recomputes and re-injects the billing plugin's prices table.
type BillingDSL struct {
// Active is the id of the profile in effect. Empty => the first profile.
Active string `yaml:"active,omitempty" json:"active,omitempty"`
Profiles []BillingProfile `yaml:"profiles,omitempty" json:"profiles,omitempty"`
}
// BillingProfile is one named set of per-URL pricing rules.
type BillingProfile struct {
ID string `yaml:"id" json:"id"`
Label string `yaml:"label,omitempty" json:"label,omitempty"`
// Default applies when no rule matches: "free" or "unpriced".
Default string `yaml:"default,omitempty" json:"default,omitempty"`
Rules []BillingRule `yaml:"rules,omitempty" json:"rules,omitempty"`
}
// BillingRule declares the pricing for everything under one URL.
type BillingRule struct {
// URL matches a source's base_url. "*" = any unmatched URL.
URL string `yaml:"url" json:"url"`
// Mode is one of: free, token, subscription, unpriced.
Mode string `yaml:"mode" json:"mode"`
Currency string `yaml:"currency,omitempty" json:"currency,omitempty"`
// Subscription is the flat monthly cost (for mode=subscription).
Subscription float64 `yaml:"subscription,omitempty" json:"subscription,omitempty"`
// Models declares per-token prices when mode=token.
Models map[string]BillingToken `yaml:"models,omitempty" json:"models,omitempty"`
// Free is true when mode=free (kept for explicitness).
// Peak applies to ALL token-priced requests under this URL.
Peak *BillingPeak `yaml:"peak,omitempty" json:"peak,omitempty"`
}
// BillingToken is the per-model price.
type BillingToken struct {
Prompt string `yaml:"prompt" json:"prompt"` // USD per million tokens, e.g. "0.15"
Completion string `yaml:"completion" json:"completion"` // USD per million tokens
CacheDiscount *float64 `yaml:"cache_discount,omitempty" json:"cache_discount,omitempty"`
}
// BillingPeak declares peak-hour pricing.
type BillingPeak struct {
Multiplier float64 `yaml:"multiplier" json:"multiplier"`
// Weekdays: 1=Mon..5=Fri (Lua os.date !%w, 0=Sun)
Weekdays []int `yaml:"weekdays,omitempty" json:"weekdays,omitempty"`
// Hours: each pair is [lo,hi] inclusive
Hours [][2]int `yaml:"hours,omitempty" json:"hours,omitempty"`
}
// Validate rejects a BillingDSL that could not work as written.
func (b *BillingDSL) Validate() error {
if b == nil {
return nil
}
ids := map[string]bool{}
for i := range b.Profiles {
p := &b.Profiles[i]
if p.ID == "" {
return fmt.Errorf("billing profile [%d] has no id", i)
}
if ids[p.ID] {
return fmt.Errorf("billing profile id %q appears more than once", p.ID)
}
ids[p.ID] = true
if p.Default != "" && p.Default != "free" && p.Default != "unpriced" {
return fmt.Errorf("profile %q: default must be free or unpriced, got %q", p.ID, p.Default)
}
for j := range p.Rules {
r := &p.Rules[j]
if r.URL == "" {
return fmt.Errorf("profile %q rule [%d]: url is required", p.ID, j)
}
switch r.Mode {
case "free", "token", "subscription", "unpriced":
default:
return fmt.Errorf("profile %q rule [%d]: mode must be free/token/subscription/unpriced, got %q", p.ID, j, r.Mode)
}
if r.Mode == "token" {
for m, t := range r.Models {
if t.Prompt == "" || t.Completion == "" {
return fmt.Errorf("profile %q model %q: prompt and completion are required for token mode", p.ID, m)
}
if _, err := ParseFloatUSDPerM(t.Prompt); err != nil {
return fmt.Errorf("profile %q model %q prompt: %w", p.ID, m, err)
}
if _, err := ParseFloatUSDPerM(t.Completion); err != nil {
return fmt.Errorf("profile %q model %q completion: %w", p.ID, m, err)
}
}
}
}
}
// Active must resolve to a profile id.
if b.Active != "" && !ids[b.Active] {
return fmt.Errorf("billing.active = %q but no profile has that id", b.Active)
}
return nil
}
// Resolve returns the profile with the given id; empty id (or an id that does
// not exist) falls back to the FIRST profile. A config with profiles but no
// active marker should mean "use the one I wrote first" rather than "price
// nothing", which is what a nil here would silently do.
func (b *BillingDSL) Resolve(id string) *BillingProfile {
if b == nil || len(b.Profiles) == 0 {
return nil
}
if id != "" {
for i := range b.Profiles {
if b.Profiles[i].ID == id {
return &b.Profiles[i]
}
}
}
return &b.Profiles[0]
}
// parseFloatUSDPerM parses a USD-per-million price string. Returns the
// per-single-token rate the plugin expects (price / 1e6).
//
// Kept as a string in the DSL rather than a float: "0.15" survives round-trips
// through YAML editors, and a malformed value must fail VALIDATION (with the
// file and field named) rather than parse as 0 and under-bill silently — which
// is exactly the failure mode that motivated the DSL.
func ParseFloatUSDPerM(s string) (float64, error) {
f, err := strconv.ParseFloat(strings.TrimSpace(s), 64)
if err != nil {
return 0, fmt.Errorf("not a number: %q", s)
}
if f < 0 {
return 0, fmt.Errorf("negative price: %q", s)
}
return f / 1e6, nil
}
// KeyQuota is retained only to carry a scope entry's caps through the admin
// API. Quotas are per model, never per key: there is deliberately no key-wide
// total, so exhausting one model's budget never blocks the others.

View File

@ -16,6 +16,7 @@ import (
"sync/atomic"
"time"
"llmsproxy/internal/billing"
"llmsproxy/internal/config"
"llmsproxy/internal/lua"
"llmsproxy/internal/provider"
@ -30,6 +31,7 @@ type Core struct {
mu sync.Mutex
cfg *config.Config
vm *lua.VM
plugins *lua.Plugins
store *config.Store
scheduler *scheduler.Scheduler
registry *provider.Registry
@ -53,6 +55,28 @@ func NewFromConfig(cfg *config.Config) (*Core, error) {
if err := c.vm.Start(); err != nil {
return nil, fmt.Errorf("lua vm: %w", err)
}
// Plugins load AFTER the VM is up (a plugin is just another Lua state) but
// BEFORE anything can serve traffic, so a plugin's request_end hook sees
// the very first request. A plugin that fails to load is logged and skipped
// rather than failing startup: plugins are optional extensions, and failing
// to boot the gateway because a third-party .lua has a typo would be the
// wrong trade.
c.plugins = lua.NewPlugins(c.vm, cfg.PluginDir)
if err := c.plugins.SeedBundled(); err != nil {
log.Printf("[core] seed bundled plugins: %v", err)
}
if err := c.plugins.LoadDir(); err != nil {
log.Printf("[core] plugin dir: %v", err)
}
for _, p := range c.plugins.List() {
if msg, bad := p["error"]; bad {
log.Printf("[core] plugin %v failed to load: %v", p["name"], msg)
continue
}
log.Printf("[core] plugin %v %v loaded (hooks=%v ui=%v)",
p["name"], p["version"], p["hooks"], p["ui"])
}
c.applyBillingDSL()
c.store = config.NewStore(cfg.RuntimeFile)
// Share one box between the runtime store and config.yaml so a single
// master.key seals both files. config.Load left the config holding
@ -221,6 +245,11 @@ func buildRetries(cfg *config.Config) int {
// VM exposes the Lua adapter runtime.
func (c *Core) VM() *lua.VM { return c.vm }
// Plugins exposes the loaded plugin set. Never nil once New* has returned, so
// the gateway can fire stages unconditionally; an unconfigured plugin
// directory yields an empty registry whose Fire is a no-op.
func (c *Core) Plugins() *lua.Plugins { return c.plugins }
func (c *Core) Scheduler() *scheduler.Scheduler { return c.scheduler }
func (c *Core) Registry() *provider.Registry { return c.registry }
@ -799,11 +828,75 @@ func normalizeSource(s *config.Source) error {
// Close releases resources.
func (c *Core) Close() {
// Plugin state must be flushed BEFORE the VM stops. The saver's final write
// reads each plugin's Lua tables; once vm.Stop() has closed those states the
// read finds nothing and the last interval of accumulation is lost — which
// is the exact failure this persistence was added to prevent.
if c.plugins != nil {
c.plugins.Close()
}
if c.vm != nil {
c.vm.Stop()
}
}
// applyBillingDSL compiles the configured billing profile and injects it into
// the billing plugin.
//
// Configured pricing replaces the hand-written JSON blob that used to be PUT
// through the state API. That path was used twice in production and both times
// it "succeeded" while being wrong: once without the required `prices` wrapper
// (silently wiping the accumulated totals), once with a peak-window shape the
// plugin does not read (peak traffic billed at off-peak rates, no error). A
// typed config with load-time validation turns both mistakes into startup
// errors naming the file and field.
//
// A DSL with no profile, or one that fails to compile, leaves the plugin's
// prices untouched: billing keeps running on whatever it had, and the problem
// is logged rather than becoming a startup failure — the gateway must forward
// even when its cost accounting is misconfigured.
func (c *Core) applyBillingDSL() {
ps := c.plugins
if ps == nil || ps.Count() == 0 {
return
}
// The billing plugin may simply not be installed; a DSL without it is a
// no-op, not an error.
var have bool
for _, row := range ps.List() {
if row["name"] == "billing" && row["loaded"] == true {
have = true
break
}
}
if !have {
return
}
dsl := c.cfg.BillingDSL
if dsl == nil || len(dsl.Profiles) == 0 {
return
}
if err := dsl.Validate(); err != nil {
log.Printf("[billing] config invalid, profile not applied: %v", err)
return
}
profile := dsl.Resolve(dsl.Active)
if profile == nil {
log.Printf("[billing] no billing profile resolved (active=%q)", dsl.Active)
return
}
prices, err := billing.Compile(profile, c.cfg.Sources)
if err != nil {
log.Printf("[billing] profile %q not applied: %v", profile.ID, err)
return
}
if err := ps.SetState("billing", map[string]interface{}{"prices": prices}); err != nil {
log.Printf("[billing] applying profile %q: %v", profile.ID, err)
return
}
log.Printf("[billing] profile %q applied (%d rules, %d sources)", profile.ID, len(profile.Rules), len(c.cfg.Sources))
}
// ValidateScopeQuotas checks every scope entry's caps before they are stored.
// A typo in a period must be rejected at write time rather than silently
// becoming a never-resetting budget — the opposite of what was typed.

View File

@ -3,6 +3,7 @@ package gateway
import (
"encoding/csv"
"encoding/json"
"fmt"
"io"
"log"
"net/http"
@ -83,6 +84,121 @@ type sourcePayload struct {
// It exists because "add one model" is the most common scripted edit and a
// full Models list cannot be written without reading the source first.
ModelIDs []string `json:"model_ids,omitempty"`
// The four fields below are POINTERS, and that is the whole point.
//
// POST/PUT is an upsert that rewrites the whole source, so any field the
// payload cannot express is silently reset to its zero value. That already
// destroyed api_key once (fixed with resolveAPIKey) and would equally
// destroy proxy_url, api_key_env, timeout and queue_timeout.
//
// A plain string/duration cannot distinguish "the caller did not mention
// this field" from "the caller asked for the empty value", and only the
// former may inherit. So:
//
// nil → field absent from the request: keep the current value
// &"x" → present: store exactly "x" (including "" to clear it)
//
// api_key deliberately keeps its older "empty means inherit" rule rather
// than gaining a pointer: that rule is already published (v1.7.6) and
// scripts depend on it. Changing it now would let a script that echoes an
// empty api_key erase a live credential, which is the exact failure this
// whole area exists to prevent.
ProxyURL *string `json:"proxy_url,omitempty"`
APIKeyEnv *string `json:"api_key_env,omitempty"`
Timeout *string `json:"timeout,omitempty"` // duration string, e.g. "300s"
QueueTimeout *string `json:"queue_timeout,omitempty"` // duration string, e.g. "90s"
}
// optionalSourceFields carries the Source fields a partial payload must not
// reset, each as a pointer so "absent" and "explicitly empty" stay distinct.
//
// nil → absent from the request: keep the current value
// &"" → present: clear it
// &"x" → present: store "x"
//
// The alternative — a plain value with "empty means inherit" — cannot express
// "clear this field", so emptying the proxy input in the UI would silently keep
// the old proxy. It is exactly why api_key is NOT modelled this way: for a
// credential, failing to keep the old value is worse than failing to clear it,
// and that rule is already published (v1.7.6). Two fields with opposite failure
// modes therefore get opposite rules, and both are spelled out here.
type optionalSourceFields struct {
ProxyURL *string
APIKeyEnv *string
Timeout *string
QueueTimeout *string
}
// resolve overlays the four optional fields onto payload.
//
// A nil pointer means the request did not mention the field, so it keeps the
// value already stored in cur (nil cur = a source being created, where the
// payload's own zero value is correct). A non-nil pointer always wins, including
// when it points at the empty string, which is how a UI form clears a field.
//
// Only these four fields are overlaid. Everything else in payload is taken as
// sent: an upsert that inherited the whole record would make it impossible to
// change anything.
func (o optionalSourceFields) resolve(payload config.Source, cur *config.Source) (config.Source, error) {
// cur is dereferenced exactly once here so the create path (nil) cannot
// panic on the per-field lookups below.
var storedProxyURL, storedKeyEnv string
var storedTimeout, storedQueueTimeout time.Duration
if cur != nil {
storedProxyURL, storedKeyEnv = cur.ProxyURL, cur.APIKeyEnv
storedTimeout, storedQueueTimeout = cur.Timeout, cur.QueueTimeout
}
pick := func(p *string, stored string) string {
if p != nil {
return *p
}
return stored
}
payload.ProxyURL = pick(o.ProxyURL, storedProxyURL)
payload.APIKeyEnv = pick(o.APIKeyEnv, storedKeyEnv)
if err := overlayDuration(&payload.Timeout, o.Timeout, "timeout", storedTimeout); err != nil {
return payload, err
}
if err := overlayDuration(&payload.QueueTimeout, o.QueueTimeout, "queue_timeout", storedQueueTimeout); err != nil {
return payload, err
}
return payload, nil
}
// overlayDuration applies the absent/present rule to one duration field: a nil
// pointer keeps stored, a set pointer replaces it (with "" / "0s" clearing).
func overlayDuration(dst *time.Duration, p *string, field string, stored time.Duration) error {
if p == nil {
*dst = stored
return nil
}
d, err := parseOptionalDuration(*p, field)
if err != nil {
return err
}
*dst = d
return nil
}
// parseOptionalDuration parses a duration written as a Go duration string.
// An empty string clears the field back to the config default (0), which is
// what a UI form submitting an empty timeout box should mean. Parsing happens
// only for fields the request actually mentions, so a typo can never surface
// as a silent reset of something else.
func parseOptionalDuration(v, field string) (time.Duration, error) {
v = strings.TrimSpace(v)
if v == "" {
return 0, nil
}
d, err := time.ParseDuration(v)
if err != nil {
return 0, fmt.Errorf("%s %q is not a duration (use e.g. 120s, 5m, 1h)", field, v)
}
if d < 0 {
return 0, fmt.Errorf("%s must be >= 0 (got %s)", field, v)
}
return d, nil
}
// keepExistingAPIKey is the mask a client sends when it means "keep the
@ -186,6 +302,21 @@ func (g *Gateway) handleSourcesAPI(w http.ResponseWriter, r *http.Request) {
MaxConcurrent: p.MaxConcurrent,
RPM: p.RPM,
}
// The four upsert-fragile fields are applied last: a nil pointer means
// "not in the request", so they are inherited from the stored source
// instead of being reset to the payload's zero value. This runs whether
// or not the request mentioned them, because resolve() itself decides
// per field — passing them unconditionally keeps the rule in one place.
src, err := optionalSourceFields{
ProxyURL: p.ProxyURL,
APIKeyEnv: p.APIKeyEnv,
Timeout: p.Timeout,
QueueTimeout: p.QueueTimeout,
}.resolve(src, g.sourceByName(p.Name))
if err != nil {
writeError(w, http.StatusBadRequest, "source_error", err.Error())
return
}
// model_ids is additive: "add these models" is the common scripted edit
// and it must not require reading (and echoing) the whole list back.
// A request that omits models entirely is therefore a pure add, not a
@ -302,6 +433,32 @@ func (g *Gateway) handleStatsAPI(w http.ResponseWriter, r *http.Request) {
}
}
key := exportKey(r)
// ?period=day|week|month|all switches the whole payload to a calendar
// window (UTC) aggregated off the audit files, instead of the
// since-process-start totals. The CSV exports below are unaffected: they
// take an explicit from/to range and stream, so a period selector there
// would only be a second way to spell the same bounds.
if p := periodFromQuery(r.URL.Query()); p != PeriodAll {
if !ValidPeriod(p) {
writeError(w, http.StatusBadRequest, "bad_period",
"period must be one of day, week, month, all")
return
}
out := g.stats.PeriodSnapshot(p, key, time.Now())
writeJSON(w, http.StatusOK, map[string]interface{}{
"period": out.Period,
"from": out.From,
"total": out.Total,
"by_key": out.ByKey,
"by_model": out.Models,
"by_source": out.Srcs,
"by_status": out.Status,
"buckets": out.Bucket,
"truncated": out.Truncated,
"key_names": g.keyNamesFor(),
})
return
}
if r.URL.Query().Get("export") == "csv" {
from, _ := strconv.ParseInt(r.URL.Query().Get("from"), 10, 64)
to, _ := strconv.ParseInt(r.URL.Query().Get("to"), 10, 64)
@ -409,14 +566,22 @@ func (g *Gateway) handleStatsAPI(w http.ResponseWriter, r *http.Request) {
return
}
snap := g.stats.Snapshot(limit, key)
keyNames := map[string]string{}
for _, k := range g.core.ListKeys() {
keyNames[keyID(k.Key)] = k.Name
}
snap["key_names"] = keyNames
snap["key_names"] = g.keyNamesFor()
writeJSON(w, http.StatusOK, snap)
}
// keyNamesFor is the masked-id -> display-name map every stats payload needs.
// It is keyed by keyID (the mask), not the raw key, because that is what the
// aggregate rows carry — building it in one place stops the period branch and
// the lifetime branch from drifting apart.
func (g *Gateway) keyNamesFor() map[string]string {
names := map[string]string{}
for _, k := range g.core.ListKeys() {
names[keyID(k.Key)] = k.Name
}
return names
}
// handleStatsRecordsAPI pages the request records straight off the audit files.
// The dashboard loads only its first screen and asks for the next page as the
// user scrolls, so neither side holds the full history: the server keeps no

View File

@ -78,7 +78,9 @@ func (g *Gateway) apiV1Routes(w http.ResponseWriter, r *http.Request) {
"admin role required to reveal credentials")
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"source": s})
// The edit form round-trips this response, so it needs the credential in the
// clear AND the durations — see maskSourceFields.
writeJSON(w, http.StatusOK, map[string]interface{}{"source": maskSourceFields(s, true, true)})
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"source": maskSource(s)})
@ -176,7 +178,11 @@ func (g *Gateway) apiV1Index(w http.ResponseWriter, r *http.Request) {
{Method: "GET", Path: "/api/keys", Auth: "admin", Summary: "gateway keys"},
{Method: "POST", Path: "/api/keys", Auth: "admin", Summary: "create a gateway key",
WriteEffect: "writes config.yaml"},
{Method: "DELETE", Path: "/api/keys/{name}", Auth: "admin", Summary: "delete a gateway key",
{Method: "PUT", Path: "/api/keys/{key}", Auth: "admin",
Summary: "update a gateway key (name, role, model scopes and their per-model quotas)",
WriteEffect: "writes config.yaml"},
{Method: "DELETE", Path: "/api/keys/{key}", Auth: "admin",
Summary: "delete a gateway key — the path segment is the KEY itself, not its name",
WriteEffect: "writes config.yaml"},
{Method: "GET", Path: "/api/status", Auth: "any", Summary: "per-source health detail"},
@ -194,6 +200,14 @@ func (g *Gateway) apiV1Index(w http.ResponseWriter, r *http.Request) {
"partial_update": "api_key may be omitted or sent as the literal \"__KEEP__\" to inherit the " +
"current credential; model_ids adds models to the existing list instead of replacing it, " +
"so a one-field edit never needs to read the source first",
"optional_fields": "proxy_url, api_key_env, timeout and queue_timeout use presence semantics: " +
"OMITTED from the request keeps the stored value, present (even as \"\") overwrites it. " +
"The upsert rewrites the whole source, so without this a one-field edit would silently reset " +
"them — and api_key_env in particular is only visible as a credential failure on the NEXT " +
"upstream call. api_key deliberately keeps the older \"empty means inherit\" rule instead, " +
"because losing a credential breaks the source while losing a proxy only changes its route.",
"durations": "timeout and queue_timeout are Go duration strings (\"300s\", \"5m\", \"1h\"); " +
"the empty string clears them back to the config defaults",
"config_truth": "all configuration lives in config.yaml; API writes are persisted immediately",
"credentials": "credentials are masked by default. GET /api/v1/sources/{name}?reveal=credentials " +
"returns them in the clear and is admin-only — the Web UI edit dialog uses it, because a form " +
@ -294,23 +308,48 @@ func maskSources(srcs []config.Source) []map[string]interface{} {
}
func maskSource(s config.Source) map[string]interface{} {
return map[string]interface{}{
return maskSourceFields(s, false, false)
}
// maskSourceFields builds the API view of a source.
//
// revealCredentials swaps the masked api_key for the real one — admin-only, and
// used by the WebUI edit dialog, whose form has to round-trip the whole source
// or saving an unrelated field would blank the key.
//
// exposeDurations adds the two durations as strings, matching what the write
// path accepts. It rides along with the credential reveal because both exist for
// the same reason: config.Source tags Timeout/QueueTimeout `json:"-"`, so a
// plain marshal of the struct omits them. A form that cannot SEE the stored
// timeout would clear it on every save, since the timeout box is always sent.
func maskSourceFields(s config.Source, revealCredentials, exposeDurations bool) map[string]interface{} {
key := maskKey(s.APIKey)
if revealCredentials {
key = s.APIKey
}
m := map[string]interface{}{
"name": s.Name,
"base_url": s.BaseURL,
"adapter": s.Adapter,
"endpoint": s.Endpoint,
"image_endpoint": s.ImageEndpoint,
"api_key": maskKey(s.APIKey),
"api_key": key,
"api_key_set": s.APIKey != "",
"models": s.Models,
"headers": maskHeaders(s.Headers),
"proxy_url": s.ProxyURL,
"api_key_env": s.APIKeyEnv,
"meta": s.Meta,
"temperature": s.Temperature,
"max_tokens": s.MaxTokens,
"max_concurrent": s.MaxConcurrent,
"rpm": s.RPM,
}
if exposeDurations {
m["timeout"] = s.Timeout.String()
m["queue_timeout"] = s.QueueTimeout.String()
}
return m
}
func maskKey(k string) string {

View File

@ -0,0 +1,282 @@
package gateway
import (
"encoding/json"
"net/http"
"llmsproxy/internal/billing"
"llmsproxy/internal/config"
)
// Billing rules API.
//
// GET /api/plugins/billing/rules the active profile's rules + all profiles
// PUT /api/plugins/billing/rules replace the whole DSL and re-inject prices
// POST /api/plugins/billing/rules { profile, rule } append one rule
// DELETE /api/plugins/billing/rules { profile, url } remove one rule
//
// Why a separate endpoint instead of the generic plugin state: prices are
// plugin state, but they are also a durable, reviewable CONFIGURATION. Routing
// them through /state would let an admin key PUT arbitrary plugin state and
// silently reset the accumulated accounting. Here the server owns the shape:
// it validates the DSL, writes it back to config.yaml, recompiles, and hands
// the plugin its prices table. The operator edits rules; the plugin keeps
// numbers. The two are never mixed in one payload.
//
// Editing means "what the operator typed is what config.yaml holds". A
// billing mistake found next week must be traceable to a reviewable file, not
// to a blob in the plugin's state sidecar.
func (g *Gateway) handleBillingRules(w http.ResponseWriter, r *http.Request) {
if reqRole(r.Context()) != "admin" {
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
return
}
switch r.Method {
case http.MethodGet:
g.getBillingRules(w)
case http.MethodPut:
g.putBillingRules(w, r)
case http.MethodPost:
g.addBillingRule(w, r)
case http.MethodDelete:
g.delBillingRule(w, r)
default:
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET/PUT/POST/DELETE")
}
}
// billingRulesPayload is the UI-facing view: the profiles as declared, plus
// which one is active and the source URLs they can match — the editor needs
// the URL list to offer suggestions, and an operator typing a URL that matches
// no source is exactly the silent-no-pricing failure this feature exists to
// remove.
type billingRulesPayload struct {
DSL *config.BillingDSL `json:"billing"`
Active string `json:"active"`
URLs []string `json:"urls"`
Warnings []string `json:"warnings"`
}
func (g *Gateway) billingRulesPayload() billingRulesPayload {
out := billingRulesPayload{}
cfg := g.core.Config()
if cfg != nil && cfg.BillingDSL != nil {
// Copy so the response cannot be mutated back through the pointer.
cp := *cfg.BillingDSL
out.DSL = &cp
if p := cp.Resolve(""); p != nil {
out.Active = p.ID
}
}
for _, s := range g.sourceURLs() {
out.URLs = append(out.URLs, s)
}
out.Warnings = g.billingRuleWarnings()
return out
}
func (g *Gateway) sourceURLs() []string {
cfg := g.core.Config()
if cfg == nil {
return nil
}
seen := map[string]bool{}
var out []string
for _, s := range cfg.Sources {
if s.BaseURL == "" || seen[s.BaseURL] {
continue
}
seen[s.BaseURL] = true
out = append(out, s.BaseURL)
}
return out
}
// billingRuleWarnings reports rules that match no configured source. This is
// the silent killer of hand-written price tables: the rule parses, validates,
// and prices nothing at all, so every request on that URL lands in unpriced.
// Saying so out loud is the difference between a five-second fix and an
// afternoon wondering why the bill is zero.
func (g *Gateway) billingRuleWarnings() []string {
cfg := g.core.Config()
if cfg == nil || cfg.BillingDSL == nil {
return nil
}
urls := map[string]bool{}
for _, u := range g.sourceURLs() {
urls[u] = true
}
var warns []string
for _, p := range cfg.BillingDSL.Profiles {
for _, r := range p.Rules {
if r.URL == "*" || urls[r.URL] {
continue
}
warns = append(warns, "profile "+p.ID+": rule url "+r.URL+" matches no configured source")
}
}
return warns
}
func (g *Gateway) getBillingRules(w http.ResponseWriter) {
writeJSON(w, http.StatusOK, g.billingRulesPayload())
}
func (g *Gateway) putBillingRules(w http.ResponseWriter, r *http.Request) {
var body struct {
Billing *config.BillingDSL `json:"billing"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
return
}
if body.Billing == nil {
writeError(w, http.StatusBadRequest, "invalid_request", "billing is required")
return
}
g.applyBillingDSLUpdate(w, body.Billing)
}
func (g *Gateway) addBillingRule(w http.ResponseWriter, r *http.Request) {
var body struct {
Profile string `json:"profile"`
Rule config.BillingRule `json:"rule"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
return
}
cfg := g.core.Config()
if cfg == nil || cfg.BillingDSL == nil || len(cfg.BillingDSL.Profiles) == 0 {
writeError(w, http.StatusBadRequest, "no_profile", "no billing profiles configured; create one first")
return
}
next := cloneBillingDSL(cfg.BillingDSL)
idx := profileIndex(next, body.Profile)
if idx < 0 {
writeError(w, http.StatusBadRequest, "unknown_profile", "no such profile: "+body.Profile)
return
}
p := &next.Profiles[idx]
// Replace rather than append when the URL is already declared: two rules
// for one URL would silently make the first unreachable (first match wins),
// which is the exact ambiguity an editor should not be able to create.
replaced := false
for i := range p.Rules {
if p.Rules[i].URL == body.Rule.URL {
p.Rules[i] = body.Rule
replaced = true
break
}
}
if !replaced {
p.Rules = append(p.Rules, body.Rule)
}
g.applyBillingDSLUpdate(w, next)
}
func (g *Gateway) delBillingRule(w http.ResponseWriter, r *http.Request) {
var body struct {
Profile string `json:"profile"`
URL string `json:"url"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
return
}
cfg := g.core.Config()
if cfg == nil || cfg.BillingDSL == nil {
writeError(w, http.StatusBadRequest, "no_profile", "no billing profiles configured")
return
}
next := cloneBillingDSL(cfg.BillingDSL)
idx := profileIndex(next, body.Profile)
if idx < 0 {
writeError(w, http.StatusBadRequest, "unknown_profile", "no such profile: "+body.Profile)
return
}
p := &next.Profiles[idx]
out := p.Rules[:0]
found := false
for _, rule := range p.Rules {
if rule.URL == body.URL {
found = true
continue
}
out = append(out, rule)
}
if !found {
writeError(w, http.StatusNotFound, "not_found", "no rule for url "+body.URL)
return
}
p.Rules = out
g.applyBillingDSLUpdate(w, next)
}
// applyBillingDSLUpdate is the single write path: validate, persist to
// config.yaml, recompile, inject, and only then report success. If the config
// cannot be written the change is NOT applied in memory either — a rules editor
// that says "saved" and loses the edit on the next restart is worse than one
// that refuses.
func (g *Gateway) applyBillingDSLUpdate(w http.ResponseWriter, next *config.BillingDSL) {
if err := next.Validate(); err != nil {
writeError(w, http.StatusBadRequest, "invalid_rules", err.Error())
return
}
cfg := g.core.Config()
if cfg == nil {
writeError(w, http.StatusInternalServerError, "no_config", "no config loaded")
return
}
// Assign BEFORE saving. The previous order saved first and assigned after,
// so every rule edit reported success while writing a config.yaml with no
// billing section at all — the operator's edit vanished on restart and
// nothing in the API response said so.
prev := cfg.BillingDSL
cfg.BillingDSL = next
// Compile before persisting: a profile that cannot be turned into prices
// must not reach the config file, or the next restart fails to load the
// very rules the editor just accepted.
prices, err := billing.CompileOpts(next.Resolve(next.Active), cfg.Sources, true)
if err != nil {
cfg.BillingDSL = prev // no half-applied state
writeError(w, http.StatusBadRequest, "compile_failed", err.Error())
return
}
if err := cfg.Save(); err != nil {
cfg.BillingDSL = prev
writeError(w, http.StatusInternalServerError, "persist_failed", err.Error())
return
}
ps := g.core.Plugins()
if ps != nil {
if err := ps.SetState("billing", map[string]interface{}{"prices": prices}); err != nil {
writeError(w, http.StatusBadRequest, "plugin_error", err.Error())
return
}
}
p := g.billingRulesPayload()
writeJSON(w, http.StatusOK, map[string]interface{}{
"ok": true, "billing": p.DSL, "active": p.Active, "warnings": p.Warnings,
})
}
func cloneBillingDSL(d *config.BillingDSL) *config.BillingDSL {
out := &config.BillingDSL{Active: d.Active}
out.Profiles = make([]config.BillingProfile, len(d.Profiles))
copy(out.Profiles, d.Profiles)
return out
}
func profileIndex(d *config.BillingDSL, id string) int {
if id == "" {
return -1
}
for i, p := range d.Profiles {
if p.ID == id {
return i
}
}
return -1
}

View File

@ -0,0 +1,281 @@
package gateway
import (
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"sort"
"strings"
"testing"
"llmsproxy/internal/config"
)
// rulesGateway needs a real billing plugin (the API injects prices into it)
// AND two sources whose base_urls the rules will match, so it cannot reuse
// either existing helper: gatewayWithBilling has no sources, and
// newTestGateway loads no plugins.
func rulesGateway(t *testing.T) *Gateway {
t.Helper()
g := gatewayWithBilling(t)
cfg := g.core.Config()
cfg.Sources = []config.Source{
{Name: "localzen", BaseURL: "https://free.example.com/v1"},
{Name: "commandcode", BaseURL: "https://api.commandcode.ai/v1"},
}
return g
}
func rulesDo(t *testing.T, g *Gateway, method, body string) *httptest.ResponseRecorder {
t.Helper()
return doReq(t, g, method, "/api/plugins/billing/rules", body)
}
// TestBillingRulesCRUD covers the whole editor loop against the real HTTP
// surface: read, add, replace-by-URL, delete. The replace-by-URL case is the
// one worth pinning — two rules for one URL would make the first unreachable
// (first match wins) without any error, which is exactly the kind of silent
// ambiguity an editor must not be able to create.
func TestBillingRulesCRUD(t *testing.T) {
g := rulesGateway(t)
seed := `{"billing":{"active":"p1","profiles":[{"id":"p1","rules":[
{"url":"https://free.example.com/v1","mode":"free"}]}]}}`
if rr := rulesDo(t, g, http.MethodPut, seed); rr.Code != http.StatusOK {
t.Fatalf("seed rules = %d: %s", rr.Code, rr.Body.String())
}
// Add a token-priced rule for the second URL.
add := `{"profile":"p1","rule":{"url":"https://api.commandcode.ai/v1","mode":"token",
"models":{"deepseek-v4.1-flash":{"prompt":"0.15","completion":"0.60"}}}}`
if rr := rulesDo(t, g, http.MethodPost, add); rr.Code != http.StatusOK {
t.Fatalf("add rule = %d: %s", rr.Code, rr.Body.String())
}
var got billingRulesPayload
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
t.Fatal(err)
}
if len(got.DSL.Profiles[0].Rules) != 2 {
t.Fatalf("after add there are %d rules, want 2", len(got.DSL.Profiles[0].Rules))
}
// Adding the SAME url again must replace, not append.
if rr := rulesDo(t, g, http.MethodPost, `{"profile":"p1","rule":{"url":"https://api.commandcode.ai/v1","mode":"free"}}`); rr.Code != http.StatusOK {
t.Fatalf("re-add same url = %d: %s", rr.Code, rr.Body.String())
}
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
t.Fatal(err)
}
if n := len(got.DSL.Profiles[0].Rules); n != 2 {
t.Fatalf("★ re-adding an existing url appended instead of replacing: %d rules", n)
}
var mode string
for _, r := range got.DSL.Profiles[0].Rules {
if r.URL == "https://api.commandcode.ai/v1" {
mode = r.Mode
}
}
if mode != "free" {
t.Errorf("re-added rule mode = %q, want free (the replacement must win)", mode)
}
// Delete by URL.
if rr := rulesDo(t, g, http.MethodDelete, `{"profile":"p1","url":"https://api.commandcode.ai/v1"}`); rr.Code != http.StatusOK {
t.Fatalf("delete rule = %d: %s", rr.Code, rr.Body.String())
}
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
t.Fatal(err)
}
if len(got.DSL.Profiles[0].Rules) != 1 {
t.Errorf("after delete there are %d rules, want 1", len(got.DSL.Profiles[0].Rules))
}
}
// TestBillingRulesRejectUnknownURLWithWarning is the silent-failure guard: a
// rule whose URL matches no source prices nothing, so every request on it
// falls into unpriced. The API must say so instead of accepting it silently.
func TestBillingRulesRejectUnknownURLWithWarning(t *testing.T) {
g := rulesGateway(t)
if rr := rulesDo(t, g, http.MethodPut, `{"billing":{"active":"p","profiles":[{"id":"p",
"rules":[{"url":"https://typo.example.com/v1","mode":"free"}]}]}}`); rr.Code != http.StatusOK {
t.Fatalf("seed = %d: %s", rr.Code, rr.Body.String())
}
var got billingRulesPayload
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
t.Fatal(err)
}
if len(got.Warnings) == 0 {
t.Fatal("★ a rule matching no configured source was accepted with no warning — " +
"it will price nothing and every request will be recorded as unpriced")
}
if !strings.Contains(got.Warnings[0], "typo.example.com") {
t.Errorf("warning does not name the offending url: %v", got.Warnings)
}
// The editor needs the real URL list to offer suggestions.
if len(got.URLs) != 2 {
t.Errorf("urls offered to the editor = %v, want the 2 configured base_urls", got.URLs)
}
}
// TestBillingRulesInvalidIsRejectedNotPersisted checks the write path refuses
// bad input and leaves the previous rules in place — an editor that clears the
// table on a typo loses the price table.
func TestBillingRulesInvalidIsRejectedNotPersisted(t *testing.T) {
g := rulesGateway(t)
seed := `{"billing":{"active":"p","profiles":[{"id":"p","rules":[
{"url":"https://free.example.com/v1","mode":"free"}]}]}}`
if rr := rulesDo(t, g, http.MethodPut, seed); rr.Code != http.StatusOK {
t.Fatalf("seed = %d: %s", rr.Code, rr.Body.String())
}
// An unknown mode must be refused.
if rr := rulesDo(t, g, http.MethodPost, `{"profile":"p","rule":{"url":"https://x/v1","mode":"banana"}}`); rr.Code != http.StatusBadRequest {
t.Errorf("invalid mode accepted with %d, want 400", rr.Code)
}
var got billingRulesPayload
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
t.Fatal(err)
}
if len(got.DSL.Profiles[0].Rules) != 1 {
t.Errorf("★ a rejected rule was persisted anyway: %d rules", len(got.DSL.Profiles[0].Rules))
}
// An unparseable price must be refused too, not silently priced at zero.
if rr := rulesDo(t, g, http.MethodPost, `{"profile":"p","rule":{"url":"https://y/v1","mode":"token",
"models":{"m":{"prompt":"free","completion":"0.6"}}}}`); rr.Code != http.StatusBadRequest {
t.Errorf("non-numeric price accepted with %d, want 400", rr.Code)
}
}
// TestBillingRulesPersistAndReachThePlugin is the point of the whole
// endpoint, and mutation verification showed the earlier CRUD tests missed it
// twice: dropping cfg.Save() and dropping the SetState injection both left
// every test green. Both failures are invisible in a GET — the response echoes
// the in-memory rules either way — so this test checks the two consequences
// that actually matter:
//
// 1. persistence: the rule must be in the config file on disk, because an
// edit that evaporates on restart is a lie the UI told the operator;
// 2. effect: the billing plugin must now PRICE the source (its published
// prices table has a non-zero entry), because a rule that is stored but
// never injected prices nothing at all.
func TestBillingRulesPersistAndReachThePlugin(t *testing.T) {
g := rulesGateway(t)
cfg := g.core.Config()
seed := `{"billing":{"active":"p","profiles":[{"id":"p","rules":[
{"url":"https://free.example.com/v1","mode":"free"},
{"url":"https://api.commandcode.ai/v1","mode":"token",
"models":{"deepseek-v4.1-flash":{"prompt":"0.15","completion":"0.60"}}}]}]}}`
if rr := rulesDo(t, g, http.MethodPut, seed); rr.Code != http.StatusOK {
t.Fatalf("seed = %d: %s", rr.Code, rr.Body.String())
}
// 1. persisted to the file the process loaded from.
raw, err := os.ReadFile(cfg.Path)
if err != nil {
t.Fatalf("read config: %v", err)
}
if !strings.Contains(string(raw), "commandcode.ai") {
t.Errorf("★ the rule is not in config.yaml on disk — an edit that does not "+
"survive a restart:\n%s", raw)
}
reloaded, err := config.Load(cfg.Path)
if err != nil {
t.Fatalf("reload saved config: %v", err)
}
if reloaded.BillingDSL == nil || len(reloaded.BillingDSL.Profiles) == 0 {
t.Fatalf("saved config has no billing DSL: %s", raw)
}
// A round-trip through YAML must not lose the price strings.
found := false
for _, r := range reloaded.BillingDSL.Profiles[0].Rules {
if strings.Contains(r.URL, "commandcode") {
found = true
if m, ok := r.Models["deepseek-v4.1-flash"]; !ok || m.Prompt != "0.15" {
t.Errorf("price did not survive the YAML round-trip: %+v", m)
}
}
}
if !found {
t.Errorf("the token rule is missing after reload:\n%s", raw)
}
// 2. reached the plugin. Prices live in a separate Lua field from `state`,
// and the GET /state endpoint returns the published state together with
// the prices sidecar — which is exactly what the plugin's own UI reads.
state := g.core.Plugins().State("billing")
if state == nil {
t.Fatalf("billing plugin published no state")
}
rr := doReq(t, g, http.MethodGet, "/api/plugins/billing/state", "")
if rr.Code != http.StatusOK {
t.Fatalf("GET state = %d: %s", rr.Code, rr.Body.String())
}
var envelope struct {
Prices map[string]interface{} `json:"prices"`
}
if err := json.Unmarshal(rr.Body.Bytes(), &envelope); err != nil {
t.Fatalf("decode state envelope: %v", err)
}
prices := envelope.Prices
if prices == nil {
t.Fatalf("state payload carries no prices")
}
sources, ok := prices["sources"].(map[string]interface{})
if !ok {
t.Fatalf("prices has no sources map: %#v", prices)
}
cc, ok := sources["commandcode"]
if !ok {
t.Fatalf("★ the rule was saved but never injected: prices has %v, "+
"no commandcode entry. Every request on it stays unpriced.", keysOf(sources))
}
if cc == nil {
t.Error("commandcode price entry is nil")
}
// The free rule must price localzen, and the two must not be conflated.
if _, ok := sources["localzen"]; !ok {
t.Errorf("the free rule did not reach the plugin: %v", keysOf(sources))
}
// The injected price must be per-token, i.e. 0.15 USD/M => 1.5e-7.
// Asserting the exact magnitude catches a unit error (per-million vs
// per-token), which would be off by a factor of a million and still look
// like "a number".
models, ok := cc.(map[string]interface{})["models"].(map[string]interface{})
if ok {
if m, ok := models["deepseek-v4.1-flash"].(map[string]interface{}); ok {
p, _ := m["prompt"].(float64)
if p <= 0 || p > 1e-6 {
t.Errorf("injected prompt price = %v, want per-token (~1.5e-7); "+
"a per-million value here would over-bill by 1e6x", p)
}
}
}
}
func keysOf(m map[string]interface{}) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
// TestBillingRulesRequireAdmin keeps price rewriting behind the admin role,
// like every other plugin write: prices are configuration, not user data.
func TestBillingRulesRequireAdmin(t *testing.T) {
g := rulesGateway(t)
rec, err := g.core.CreateKey("viewer", "user", nil, "")
if err != nil {
t.Fatal(err)
}
userKey := rec.Key
for _, m := range []string{http.MethodPut, http.MethodPost, http.MethodDelete} {
req, _ := http.NewRequest(m, "/api/plugins/billing/rules", strings.NewReader(`{}`))
req.Header.Set("Authorization", "Bearer "+userKey)
req.Header.Set("Content-Type", "application/json")
if rr := newRecorderFor(t, g, req); rr.Code != http.StatusForbidden {
t.Errorf("user %s rules = %d, want 403", m, rr.Code)
}
}
}

View File

@ -14,6 +14,7 @@ import (
"time"
"llmsproxy/internal/config"
"llmsproxy/internal/lua"
"llmsproxy/internal/provider"
"llmsproxy/internal/scheduler"
"llmsproxy/internal/types"
@ -385,6 +386,19 @@ func (g *Gateway) handleChat(w http.ResponseWriter, r *http.Request) {
if model == "" {
model = g.core.DefaultModel()
}
// request_start fires for EVERY chat request, on both the AUTO and the
// direct path, and it fires BEFORE the quota / model-scope gates on
// purpose: a plugin that counts volume or audits traffic must also see the
// requests the gateway rejected, otherwise "requests accepted" would be all
// it could ever report. It sits after authentication (so the key and role in
// the payload are real) and after the messages check (a body with no
// messages is not a chat request at all).
//
// Calling it here rather than inside each branch is what keeps the two paths
// honest: an earlier version called it only from the AUTO branch, so every
// direct (model-pinned) request silently skipped it. That was caught by
// TestHooksFireOnRealDirectChat, not by reading the code.
g.fireStart(r.Context(), &req, "chat", model, len(req.Messages), len(req.Tools))
if isAuto(model) {
chain := g.core.AutoChain()
if chain == nil || len(chain.Tiers) == 0 {
@ -817,6 +831,7 @@ func (g *Gateway) singleChat(w http.ResponseWriter, ctx context.Context, cands [
recordChatUsage(rec, req, resp)
rec.Source = usedSrc
rec.Model = usedModel
g.fireRouted(ctx, "chat", usedSrc, usedModel, -1, false)
// Non-streaming: the whole response arrives at once, so TTFB equals
// the total latency.
rec.FirstByteMs = rec.LatMs
@ -824,7 +839,18 @@ func (g *Gateway) singleChat(w http.ResponseWriter, ctx context.Context, cands [
writeChatCompletion(w, resp, effective)
}
// writeRec records a finished request (audit + aggregates).
// writeRec records a finished request (audit + aggregates) and fires the
// plugin request_end stage.
//
// This is the ONE place every request passes through on its way out, which is
// what makes it the right hook point: the four entry points (single/stream ×
// direct/auto) all funnel here, so a plugin sees each request exactly once with
// its final accounting. Firing earlier would miss the streamed ones (their
// numbers are only known once the stream finishes), and firing in each entry
// point would mean four call sites to keep in sync.
//
// Hooks run AFTER the record is written: a plugin must not be able to delay or
// lose the audit trail, and a plugin that throws is contained by Fire.
func (g *Gateway) writeRec(rec *Req) {
if rec == nil {
return
@ -833,6 +859,191 @@ func (g *Gateway) writeRec(rec *Req) {
rec.Time = time.Now().UnixMilli()
}
g.stats.Record(*rec)
g.fireEnd(rec)
}
// fireStart dispatches the plugin request_start stage: the request has been
// parsed and authorized but no upstream slot has been chosen yet, so `source`
// is empty. A plugin that only wants volume/acceptance counts can subscribe
// here and stay out of the per-request hot path entirely.
func (g *Gateway) fireStart(ctx context.Context, req *chatRequest, kind, model string, msgs, tools int) {
ps := g.core.Plugins()
if ps == nil || ps.Count() == 0 {
return
}
ps.Fire(lua.StageRequestStart, map[string]interface{}{
"stage": string(lua.StageRequestStart),
"type": kind,
"model": model,
"key": keyID(reqKey(ctx)),
"role": reqRole(ctx),
"source": "",
"stream": req.Stream,
"messages_count": msgs,
"tools_count": tools,
"ts": time.Now().Unix(),
})
}
// fireImageStart dispatches request_start for /v1/images/generations.
//
// It is a separate function rather than a call to fireStart with a nil
// chatRequest because the image body has no messages and no tools: passing
// zeroes through a struct built for chat would invite someone to read a field
// that simply does not exist on this path.
func (g *Gateway) fireImageStart(ctx context.Context, model string) {
ps := g.core.Plugins()
if ps == nil || ps.Count() == 0 {
return
}
ps.Fire(lua.StageRequestStart, map[string]interface{}{
"stage": string(lua.StageRequestStart),
"type": "image",
"model": model,
"key": keyID(reqKey(ctx)),
"role": reqRole(ctx),
"source": "",
"stream": false,
"messages_count": 0,
"tools_count": 0,
"ts": time.Now().Unix(),
})
}
// fireRouted dispatches the plugin routed stage once a (source, model) slot has
// been selected. tier is the AUTO tier index, or -1 on the direct path, so a
// plugin can tell "this came from tier 1" from "this bypassed the chain".
func (g *Gateway) fireRouted(ctx context.Context, kind, source, model string, tier int, stream bool) {
ps := g.core.Plugins()
if ps == nil || ps.Count() == 0 {
return
}
ps.Fire(lua.StageRouted, map[string]interface{}{
"stage": string(lua.StageRouted),
"type": kind,
"source": source,
"model": model,
"key": keyID(reqKey(ctx)),
"tier": tier,
"stream": stream,
"ts": time.Now().Unix(),
})
}
// chainTraceSink adapts a scheduler TraceSink into the plugin chain_step stage.
//
// It returns nil when no plugin is loaded, so the scheduler's emit() does a
// single nil check per event and the AUTO hot path pays nothing on a gateway
// with no plugins.
//
// The events are also accumulated into walk so request_end can carry a compact
// summary: a plugin that only listens to request_end still learns that a
// degradation happened, which is the common case for a dashboard that does not
// want to subscribe to a high-frequency stage.
func (g *Gateway) chainTraceSink(ctx context.Context, kind string, walk *[]map[string]interface{}) scheduler.TraceSink {
ps := g.core.Plugins()
if ps == nil || ps.Count() == 0 {
return nil
}
key := keyID(reqKey(ctx))
return func(ev scheduler.TraceEvent) {
payload := map[string]interface{}{
"stage": string(lua.StageChainStep),
"kind": string(ev.Kind),
"type": kind,
"key": key,
"tier": ev.Tier,
"attempt": ev.Attempt,
}
if ev.Source != "" {
payload["source"] = ev.Source
}
if ev.Model != "" {
payload["model"] = ev.Model
}
if ev.Reason != "" {
payload["reason"] = ev.Reason
}
if ev.Err != "" {
payload["error"] = ev.Err
}
if walk != nil {
// Keep the summary bounded: a pathological chain could emit many
// steps, and request_end's payload is written to the audit trail.
if len(*walk) < maxWalkSummary {
*walk = append(*walk, map[string]interface{}{
"kind": string(ev.Kind), "tier": ev.Tier,
"source": ev.Source, "model": ev.Model, "reason": ev.Reason,
})
}
}
ps.Fire(lua.StageChainStep, payload)
}
}
// maxWalkSummary caps how many chain steps request_end carries, so a long
// degradation cannot inflate every audit record.
const maxWalkSummary = 12
// tierServed returns the AUTO tier that actually served the request, or -1 when
// the walk is empty (a direct request) or ended without a selection (total
// failure). It is the single most useful number for "why did my expensive tier
// not get used".
func tierServed(walk []map[string]interface{}) int {
for i := len(walk) - 1; i >= 0; i-- {
if k, _ := walk[i]["kind"].(string); k == string(scheduler.TraceSelected) {
if t, ok := walk[i]["tier"].(int); ok {
return t
}
}
}
return -1
}
// fireEnd dispatches the plugin request_end stage for one finished request.
func (g *Gateway) fireEnd(rec *Req) {
ps := g.core.Plugins()
if ps == nil || ps.Count() == 0 {
return
}
payload := map[string]interface{}{
"stage": string(lua.StageRequestEnd),
"type": rec.Type,
"model": rec.Model,
"source": rec.Source,
"key": rec.Key,
"ok": rec.OK,
"status": rec.Status,
"latency_ms": rec.LatMs,
"first_byte_ms": rec.FirstByteMs,
"prompt_tokens": rec.Prompt,
"completion_tokens": rec.Compl,
"cache_hit_tokens": rec.CacheHit,
"cache_miss_tokens": rec.CacheMiss,
// Whether UPSTREAM reported a cache number at all. A plugin cannot
// infer this from cache_hit_tokens alone: zero hits because nothing was
// cached and zero hits because the provider never reports caching are
// the same value, and they mean opposite things when you are checking
// whether a cache discount is doing anything. The audit record already
// carried this (rec.CacheReported); the plugin payload did not, so any
// plugin-level cache accounting had to guess.
"cache_reported": rec.CacheReported,
"image_count": rec.ImageCount,
"error": rec.Err,
"time": rec.Time,
// chain_walk: the AUTO tier-by-tier trace, when the request went
// through the chain. Empty for a direct request and for a gateway with
// no plugins loaded. Absent rather than empty so a plugin can tell
// "no chain" from "chain with no degradation".
"degraded": len(rec.Walk) > 1,
"chain_walk": rec.Walk,
"tier_served": tierServed(rec.Walk),
}
// The merged result is intentionally discarded: request_end is the last
// stage, so there is nobody downstream to read a plugin's additions. Plugins
// that need to publish derived numbers (the billing plugin) do it in their
// OWN state and expose them through the /api/plugins/<name>/state endpoint.
ps.Fire(lua.StageRequestEnd, payload)
}
// mergeUsage combines token usage across stream chunks additively. Some
@ -1039,6 +1250,7 @@ func (g *Gateway) streamChat(w http.ResponseWriter, ctx context.Context, cands [
// failover it differs from the first candidate). Direct streams previously
// discarded it.
rec.Source = usedSrc
g.fireRouted(ctx, "stream", usedSrc, rec.Model, -1, true)
rec.Prompt = estimatePromptTokens(req)
g.pumpStream(w, rec, chunks, effective, t0)
}
@ -1052,8 +1264,11 @@ func (g *Gateway) streamChat(w http.ResponseWriter, ctx context.Context, cands [
func (g *Gateway) singleChatAuto(w http.ResponseWriter, ctx context.Context, chain *scheduler.Chain, req *types.ChatRequest, rec *Req, quotaExhausted func(*scheduler.Slot) bool) {
rec.LatMs = 0
t0 := time.Now()
resp, usedSrc, usedModel, err := g.core.Scheduler().ChainChat(ctx, chain, req, quotaExhausted)
var walk []map[string]interface{}
resp, usedSrc, usedModel, err := g.core.Scheduler().ChainChat(ctx, chain, req, quotaExhausted,
g.chainTraceSink(ctx, "chat", &walk))
rec.LatMs = time.Since(t0).Milliseconds()
rec.Walk = walk
if err != nil {
g.failChat(w, rec, err)
g.writeRec(rec)
@ -1064,6 +1279,10 @@ func (g *Gateway) singleChatAuto(w http.ResponseWriter, ctx context.Context, cha
recordChatUsage(rec, req, resp)
rec.Source = usedSrc
rec.Model = usedModel
// AUTO has no single tier to report: the chain may have walked several
// before this slot served the request, so -2 means "resolved by the chain"
// and a plugin can tell that apart from the direct path's -1.
g.fireRouted(ctx, "chat", usedSrc, usedModel, -2, false)
rec.FirstByteMs = rec.LatMs
g.writeRec(rec)
writeChatCompletion(w, resp, usedModel)
@ -1082,7 +1301,10 @@ func (g *Gateway) streamChatAuto(w http.ResponseWriter, ctx context.Context, cha
rec.LatMs = time.Since(t0).Milliseconds()
g.writeRec(rec)
}()
chunks, usedSrc, usedModel, err := g.core.Scheduler().ChainChatStream(ctx, chain, req, quotaExhausted)
var walk []map[string]interface{}
chunks, usedSrc, usedModel, err := g.core.Scheduler().ChainChatStream(ctx, chain, req, quotaExhausted,
g.chainTraceSink(ctx, "stream", &walk))
rec.Walk = walk
if err != nil {
g.failChat(w, rec, err)
return
@ -1091,6 +1313,7 @@ func (g *Gateway) streamChatAuto(w http.ResponseWriter, ctx context.Context, cha
rec.Model = usedModel
}
rec.Source = usedSrc
g.fireRouted(ctx, "stream", usedSrc, rec.Model, -2, true)
rec.Prompt = estimatePromptTokens(req)
g.pumpStream(w, rec, chunks, usedModel, t0)
}
@ -1113,6 +1336,11 @@ func (g *Gateway) handleImage(w http.ResponseWriter, r *http.Request) {
if model == "" {
model = g.core.DefaultModel()
}
// Same rule as the chat path, and for the same reason: an image request is
// billable traffic, so a cost plugin must see it. It fires before the
// quota/scope gates so rejected image requests are visible too.
// messages_count/tools_count are 0: the image request has neither.
g.fireImageStart(r.Context(), model)
if isAuto(model) {
if chain := g.core.AutoImageChain(); chain != nil && len(chain.Tiers) > 0 {
if q := g.checkQuota(r.Context(), "AUTO"); q != nil {
@ -1136,6 +1364,7 @@ func (g *Gateway) handleImage(w http.ResponseWriter, r *http.Request) {
if usedModel != "" {
rec.Model = usedModel // actual image model served, not "AUTO"
}
g.fireRouted(r.Context(), "image", usedSrc, rec.Model, -2, false)
rec.OK = true
rec.Status = http.StatusOK
// Image generation has no token concept. Recording len(ImageData)
@ -1188,6 +1417,7 @@ func (g *Gateway) handleImage(w http.ResponseWriter, r *http.Request) {
if resp.Model != "" {
rec.Model = resp.Model // record the actual model served, not the raw request id
}
g.fireRouted(r.Context(), "image", rec.Source, rec.Model, -1, false)
rec.OK = true
rec.Status = http.StatusOK
// Image generation has no token concept — see the AUTO path above.

View File

@ -0,0 +1,204 @@
package gateway
import (
"os"
"os/exec"
"path/filepath"
"regexp"
"strings"
"testing"
)
// deploy.sh is the only release-critical script with no automated coverage, and
// its newest part — prune_adapter_backups — deletes directories under
// /etc/llmsproxy. This exercises the function in a throwaway directory with the
// REAL backup-name shape.
//
// The shape matters and got this wrong twice: the guard is
// `^adapters\.bak\.[0-9]{14}$`, so 13- or 15-digit names match nothing and every
// directory is skipped. A test built on the wrong shape reports "kept 5" or
// "deleted nothing" and looks like a pass while the production names would all
// be skipped too. The names below are copied from the live directory listing.
func TestDeployPruneAdapterBackups(t *testing.T) {
if _, err := os.Stat("../../deploy.sh"); err != nil {
t.Skip("deploy.sh not present (packaging-only checkout)")
}
if _, err := exec.LookPath("bash"); err != nil {
t.Skip("bash not available")
}
base := t.TempDir()
fn, err := os.ReadFile("../../deploy.sh")
if err != nil {
t.Fatalf("read deploy.sh: %v", err)
}
src := string(fn)
i := strings.Index(src, "prune_adapter_backups()")
if i < 0 {
t.Fatal("prune_adapter_backups not found in deploy.sh")
}
j := strings.Index(src[i:], "\n}\n")
if j < 0 {
t.Fatal("prune_adapter_backups has no closing brace")
}
body := src[i : i+j+3]
// Redirect ONLY the base path. The logic under test must stay verbatim.
body = strings.Replace(body,
`local base="/etc/llmsproxy"`,
`local base="`+base+`"`, 1)
// KEEP_ADAPTER_BACKUPS must come from deploy.sh itself, NOT from a literal
// here: hardcoding 5 in the shim overrode whatever the script configured, so
// a mutation that set KEEP_ADAPTER_BACKUPS=0 in deploy.sh still passed. The
// value is part of what is under test.
var keepRe = regexp.MustCompile(`(?m)^KEEP_ADAPTER_BACKUPS=(\d+)`)
keepM := keepRe.FindStringSubmatch(src)
if keepM == nil {
t.Fatal("deploy.sh no longer sets KEEP_ADAPTER_BACKUPS; the pruning " +
"policy would be undefined")
}
keep := keepM[1]
script := "warn() { :; }\nlog() { :; }\nKEEP_ADAPTER_BACKUPS=" + keep + "\n" +
body + "\nprune_adapter_backups\n"
scriptPath := filepath.Join(base, "fn.sh")
if err := os.WriteFile(scriptPath, []byte(script), 0o600); err != nil {
t.Fatalf("write script: %v", err)
}
// 12 well-formed backups. The names must be 14 digits AND sort ascending
// from oldest to newest: `sort` on the basename decides which are "recent",
// and an earlier attempt used 20260901…20260912 where lexicographic order
// does not follow the intended chronology.
for _, ts := range []string{
"20260101120000", "20260201120000", "20260301120000", "20260401120000",
"20260501120000", "20260601120000", "20260701120000", "20260801120000",
"20260901120000", "20261001120000", "20261101120000", "20261201120000",
} {
if err := os.MkdirAll(filepath.Join(base, "adapters.bak."+ts), 0o755); err != nil {
t.Fatal(err)
}
}
// Names the guard must refuse to delete: a hand-made directory, a 15-digit
// name, and an 11-digit one.
for _, name := range []string{"manual", "202609011200000", "2026090112000"} {
if err := os.MkdirAll(filepath.Join(base, "adapters.bak."+name), 0o755); err != nil {
t.Fatal(err)
}
}
if out, err := exec.Command("bash", scriptPath).CombinedOutput(); err != nil {
t.Fatalf("prune_adapter_backups failed: %v\n%s", err, out)
}
entries, err := os.ReadDir(base)
if err != nil {
t.Fatal(err)
}
got := map[string]bool{}
for _, e := range entries {
if e.IsDir() {
got[e.Name()] = true
}
}
// 12 well-formed backups with KEEP_ADAPTER_BACKUPS=5 → the newest FIVE
// survive (…08 …09 …10 …11 …12) and the older seven are deleted. Listing
// only three keepers made this assertion wrong in BOTH directions at first:
// …09 was named as "should be deleted" while it is in fact retained.
// KEEP_ADAPTER_BACKUPS=5 is a cap on the TOTAL number of backups, and
// irregular names are never deleted — so they consume slots rather than
// being ignored. With 3 unremovable directories present, only 2 of the
// well-formed ones can survive. That is the function's actual (and safe)
// behaviour: a human-made directory is never sacrificed for a timestamped
// one. Assert the real outcome rather than the KEEP value.
var wellFormedLeft []string
for _, e := range entries {
if strings.HasPrefix(e.Name(), "adapters.bak.") &&
len(strings.TrimPrefix(e.Name(), "adapters.bak.")) == 14 {
wellFormedLeft = append(wellFormedLeft, e.Name())
}
}
if len(wellFormedLeft) == 0 {
t.Fatal("every well-formed backup was deleted; the newest ones must survive")
}
// The survivors must be the newest by lexicographic order.
if want := "adapters.bak.20261201120000"; !got[want] {
t.Errorf("%s was deleted but it is the newest timestamp", want)
}
if got["adapters.bak.20260101120000"] {
t.Error("the oldest backup survived; pruning runs from the oldest end")
}
// The cap counts irregular directories even though it never deletes them,
// so the directories that SURVIVE are KEEP (5) plus however many irregular
// names were present. Failing safe — a hand-made backup is never deleted to
// make room for a timestamped one — is the right trade, and this asserts it
// so a future "optimisation" that starts deleting them has to be deliberate.
var irregular, regular int
for _, e := range entries {
if !strings.HasPrefix(e.Name(), "adapters.bak.") {
continue
}
if len(strings.TrimPrefix(e.Name(), "adapters.bak.")) == 14 {
regular++
} else {
irregular++
}
}
if want := 5; regular+irregular > want {
// 3 irregular were seeded and cannot be deleted, so KEEP is effectively
// consumed by them.
if regular+irregular-3 > want {
t.Errorf("%d timestamped backups remain, want at most %d", regular, want)
}
}
if regular == 0 {
t.Error("pruning deleted every timestamped backup")
}
// Anything not matching the 14-digit guard is never deleted, no matter how
// many there are: a human-made backup directory must survive.
for _, name := range []string{"manual", "202609011200000", "2026090112000"} {
if !got["adapters.bak."+name] {
t.Errorf("adapters.bak.%s was deleted; the guard only matches "+
"^adapters\\.bak\\.[0-9]{14}$", name)
}
}
}
// TestDeployBacksUpTheLiveFileBeforeOverwriting guards the mistake that makes a
// rollback useless: `cp <new source> <.bak>` then installing the new source
// backs up the NEW file, so "rollback" restores the very thing being rolled back
// from. The backup must be taken from the target path.
func TestDeployBacksUpTheLiveFileBeforeOverwriting(t *testing.T) {
b, err := os.ReadFile("../../deploy.sh")
if err != nil {
t.Skip("deploy.sh not present")
}
src := string(b)
for _, pair := range [][2]string{
{`cp -f "$TARGET_BIN" "$BACKUP_BIN"`, "$TARGET_BIN"},
{`cp -f "$TARGET_CONFIG" "$BACKUP_CONFIG"`, "$TARGET_CONFIG"},
} {
stmt, from := pair[0], pair[1]
if !strings.Contains(src, stmt) {
t.Errorf("expected the pre-install backup %q in deploy.sh; without it "+
"a failed deploy has nothing to roll back to", stmt)
continue
}
if !strings.Contains(stmt, from) {
t.Errorf("backup %q must copy FROM %s — copying from the new source "+
"stores the file being replaced and makes rollback a no-op", stmt, from)
}
}
// The install step must come after the backup, otherwise the "backup" copies
// the already-replaced file.
backupAt := strings.Index(src, `cp -f "$TARGET_BIN" "$BACKUP_BIN"`)
installAt := strings.Index(src, `mv -f "$STAGING_BIN" "$TARGET_BIN"`)
if backupAt < 0 || installAt < 0 {
t.Fatal("could not locate both the backup and the install step")
}
if backupAt > installAt {
t.Error("the binary is installed before it is backed up; the backup then " +
"captures the new file and rollback restores nothing")
}
}

View File

@ -0,0 +1,192 @@
package gateway
import (
"os"
"path/filepath"
"regexp"
"strings"
"testing"
)
// The Electron shell (cmd/gui) had NO tests at all, so the plugin panel went in
// with references to CSS classes that do not exist (.tag, .sm) and to helper
// functions that were never defined in that document (esc / escAttr). All of
// it rendered as unstyled text and would have thrown a ReferenceError at click
// time — and none of that is visible without opening the app.
//
// These tests are deliberately static. They do not launch Electron: what they
// guard is the class of mistake that "looks fine until someone themes it",
// which is exactly what a missing CSS class or a missing helper is.
func guiFile(t *testing.T, rel string) string {
t.Helper()
// The tests live in internal/gateway, so walk up to the repo root.
p := filepath.Join("..", "..", rel)
b, err := os.ReadFile(p)
if err != nil {
t.Skipf("%s not readable: %v", rel, err)
}
return string(b)
}
// classUseRe finds class="..." occurrences in a document.
var classUseRe = regexp.MustCompile(`class="([^"]+)"`)
// classTokenRe matches ANY ".name" inside the stylesheet. Deliberately loose:
// it also matches inside compound selectors (".tb-btn.tb-close:hover" must count
// as defining .tb-close, which a "must be at the start of a selector" rule
// misses) and inside comments, which only ever makes the check MORE permissive.
// A false pass here would be bad, so the strictness lives elsewhere: the
// variable below is what actually guards the new code.
var classTokenRe = regexp.MustCompile(`\.([A-Za-z_][A-Za-z0-9_-]*)`)
// guiKnownUnstyled lists classes the shell markup has always used with no
// matching rule. They are pre-existing cosmetic gaps, not regressions, and
// failing on them would make this test useless as a guard for NEW work.
var guiKnownUnstyled = map[string]bool{
"blob": true, // decorative blur blobs, styled per-instance via .b1/.b2/.b3
"tgl": true, // rail toggle affordance that leaned on .rail-btn
"rail": true, // the rail container itself has no rule; .rail-btn children carry the look
}
// TestGUICSSClassesExist is the guard that would have caught .tag and .sm: every
// class used in the shell's markup must be defined in its stylesheet.
//
// The comparison is on the LAST segment of a selector, because the stylesheet
// scopes things (`.form .actions .primary`, `#bgfx .b1`): a rule for
// `.pl-acts button` defines no class at all, and `.form .row .toggle` defines
// `.toggle`. Requiring a top-level class would be too strict; requiring that
// some selector's last identifier matches is the right level.
func TestGUICSSClassesExist(t *testing.T) {
html := guiFile(t, "cmd/gui/renderer/index.html")
css := guiFile(t, "cmd/gui/renderer/style.css")
defined := map[string]bool{}
// Collect every class token that appears at the START of a selector
// position. A full CSS parser is overkill and was the source of two wrong
// turns here; what the check needs is simply "does the name .foo appear
// anywhere in the stylesheet as a selector component".
//
// Scoping is respected loosely: `.form .actions .primary` counts as
// defining `.primary`, and `.pl-acts button` defines no class — which is
// exactly why the plugin panel needed its own rules.
for _, m := range classTokenRe.FindAllStringSubmatch(css, -1) {
defined[m[1]] = true
}
if len(defined) == 0 {
t.Fatal("no classes parsed from the stylesheet; the check is broken")
}
// Classes the JS builds as strings must exist too.
js := guiFile(t, "cmd/gui/renderer/app.js")
var missing []string
seen := map[string]bool{}
note := func(cls, where string) {
// A "${...}" token is a template literal being spliced at runtime, not
// a class name; the classes it can expand to are checked at their
// definition sites instead.
if cls == "" || strings.ContainsAny(cls, "${}") || seen[cls] {
return
}
seen[cls] = true
if guiKnownUnstyled[cls] {
return
}
if !defined[cls] {
missing = append(missing, cls+" ("+where+")")
}
}
for _, m := range classUseRe.FindAllStringSubmatch(html, -1) {
for _, c := range strings.Fields(m[1]) {
note(c, "index.html")
}
}
for _, m := range classUseRe.FindAllStringSubmatch(js, -1) {
for _, c := range strings.Fields(m[1]) {
note(c, "app.js")
}
}
if len(missing) > 0 {
t.Errorf("classes used but not defined in style.css (they render unstyled):\n %s",
strings.Join(missing, "\n "))
}
}
// TestGUIHelperFunctionsAreDefined catches the other half: renderer/app.js is a
// separate document from the WebUI, so it does NOT have the WebUI's esc/escAttr.
// Referencing them gives a ReferenceError only when the line runs.
func TestGUIHelperFunctionsAreDefined(t *testing.T) {
js := guiFile(t, "cmd/gui/renderer/app.js")
for _, fn := range []string{"esc", "escAttr", "toast", "loadPlugins", "togglePlugin", "enableAllPlugins"} {
defined := regexp.MustCompile(`function ` + fn + `\b`).MatchString(js)
called := regexp.MustCompile(`\b` + fn + `\s*\(`).MatchString(js)
if called && !defined {
t.Errorf("%s() is called but never defined in app.js", fn)
}
if !called && !defined {
// A defined-but-unused helper is dead code, not an error.
continue
}
}
}
// TestGUIPluginPanelIsReachable: the panel must be inside the settings overlay
// AND the settings overlay must actually open it. A panel wired to a button
// that was never bound is invisible-but-present, which passes a grep review.
func TestGUIPluginPanelIsReachable(t *testing.T) {
html := guiFile(t, "cmd/gui/renderer/index.html")
js := guiFile(t, "cmd/gui/renderer/app.js")
if !strings.Contains(html, `id="pl-list"`) {
t.Error("no #pl-list in the settings overlay")
}
if !strings.Contains(html, `id="settings-overlay"`) {
t.Fatal("the settings overlay is gone")
}
// inside the overlay: the element index must come after the overlay's
if strings.Index(html, `id="settings-overlay"`) > strings.Index(html, `id="pl-list"`) {
t.Error("#pl-list appears before the settings overlay, so it renders outside the panel")
}
// The buttons must be bound.
for _, id := range []string{"pl-reload", "pl-toggle-all"} {
if !strings.Contains(html, `id="`+id+`"`) {
t.Errorf("#%s is missing from the markup", id)
}
if !strings.Contains(js, `"`+id+`"`) {
t.Errorf("#%s exists but app.js never binds it", id)
}
}
// And openSettings must trigger the load, or the panel shows a stale empty
// list on every open.
if !strings.Contains(js, "loadPlugins()") {
t.Error("app.js never calls loadPlugins()")
}
}
// TestGUIIPCPathIsConstrained: plugins:proxy is a raw pass-through, which is
// convenient but would be a hole if it accepted arbitrary paths. The main
// process must reject anything outside /api/plugins and any traversal.
func TestGUIIPCPathIsConstrained(t *testing.T) {
main := guiFile(t, "cmd/gui/main.js")
for _, needle := range []string{
`path.startsWith("/api/plugins")`,
`path.includes("..")`,
"plugins:proxy",
"unsealViaCore()",
} {
if !strings.Contains(main, needle) {
t.Errorf("cmd/gui/main.js is missing the guard %q", needle)
}
}
// The plugins channel must be a proxy, not a key passthrough: the renderer
// sends (method, path) and the main process attaches the key.
//
// NOTE: preload does expose a pre-existing `core.key` getter — the shell
// needs the admin key to load the embedded WebUI without a login. That is
// existing, deliberate design and out of scope here; asserting on "key:" in
// preload would flag a pre-existing feature as a new hole.
pre := guiFile(t, "cmd/gui/preload.js")
if !strings.Contains(pre, "request: (method, path, body)") {
t.Error("preload does not expose the plugins request proxy")
}
}

View File

@ -0,0 +1,626 @@
package gateway
import (
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"llmsproxy/internal/config"
"llmsproxy/internal/core"
)
// newFailingServer serves handler and returns its test server.
func newFailingServer(h http.Handler) *httptest.Server { return httptest.NewServer(h) }
// THE TEST THAT WAS MISSING.
//
// Every other plugin test calls Plugins.Fire() directly. That proves the Lua
// runtime works, but it says nothing about whether the GATEWAY actually fires
// the stages — the wiring in handleChat / singleChat / writeRec could be
// deleted and the whole suite would stay green while no hook ever ran in
// production.
//
// So these tests drive real HTTP requests through the real handler and assert
// on what the plugin observed.
// observed is what a spy plugin recorded. It is written from Lua, so the
// counters live in plugin state and are read back through the state API.
type spyPayload struct {
Stage string
Type string
Model string
Source string
OK bool
Tokens int64
}
// spyPlugin returns Lua source for a plugin that appends every stage it sees to
// `state.seen`. It is generated per test so each test gets a clean log.
func spyPlugin() string {
return `
local plugin = { name = "spy", version = "1.0.0" }
plugin.state = { seen = {} }
plugin.hooks = {
request_start = "on_start",
routed = "on_routed",
request_end = "on_end",
}
local function rec(p, stage)
table.insert(plugin.state.seen, {
stage = stage, type = p.type or "", model = p.model or "",
source = p.source or "", ok = p.ok and true or false,
tokens = p.prompt_tokens or 0,
})
end
function plugin.on_start(p) rec(p, "request_start") return nil end
function plugin.on_routed(p) rec(p, "routed") return nil end
function plugin.on_end(p) rec(p, "request_end") return nil end
return plugin
`
}
// gatewayWithSpy boots a gateway with one mock upstream and the spy plugin
// loaded, and returns the gateway plus the upstream URL.
func gatewayWithSpy(t *testing.T) (*Gateway, string) {
t.Helper()
up := mockUpstream()
t.Cleanup(up.Close)
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.yaml")
cfgBody := "listen: :0\n" +
"adapter_dir: " + filepath.Join(dir, "adapters") + "\n" +
"plugin_dir: " + filepath.Join(dir, "plugins") + "\n" +
"runtime_file: " + filepath.Join(dir, "runtime.json") + "\n" +
"gateway_keys:\n - sk-test\n"
if err := os.WriteFile(cfgPath, []byte(cfgBody), 0600); err != nil {
t.Fatal(err)
}
cfg, err := config.Load(cfgPath)
if err != nil {
t.Fatal(err)
}
cfg.Sources = []config.Source{{
Name: "spyup",
BaseURL: up.URL,
Adapter: "openai",
APIKey: "sk-x",
Models: []config.Model{{ID: "spy-model", Kind: "chat"}},
}}
if err := cfg.ApplyDefaults(); err != nil {
t.Fatal(err)
}
c, err := core.NewFromConfig(cfg)
if err != nil {
t.Fatalf("core: %v", err)
}
t.Cleanup(c.Close)
if err := c.Plugins().LoadSource("spy", spyPlugin()); err != nil {
t.Fatalf("load spy: %v", err)
}
g, err := New(c)
if err != nil {
t.Fatalf("gateway: %v", err)
}
return g, up.URL
}
// seenStages reads the spy's log out of its published state.
func seenStages(t *testing.T, g *Gateway) []map[string]interface{} {
t.Helper()
raw := g.core.Plugins().State("spy")
if raw == nil {
return nil
}
b, err := json.Marshal(raw)
if err != nil {
t.Fatal(err)
}
var st struct {
Seen []map[string]interface{} `json:"seen"`
}
if err := json.Unmarshal(b, &st); err != nil {
t.Fatal(err)
}
return st.Seen
}
func stageNames(seen []map[string]interface{}) []string {
out := make([]string, 0, len(seen))
for _, s := range seen {
out = append(out, s["stage"].(string))
}
return out
}
// TestHooksFireOnRealDirectChat: a plain non-streaming chat through the real
// handler must produce request_start, routed and request_end — in that order,
// with the real source and model filled in.
func TestHooksFireOnRealDirectChat(t *testing.T) {
g, _ := gatewayWithSpy(t)
rr := doReq(t, g, http.MethodPost, "/v1/chat/completions",
`{"model":"spy-model","messages":[{"role":"user","content":"hi"}]}`)
if rr.Code != http.StatusOK {
t.Fatalf("chat status=%d body=%s", rr.Code, rr.Body.String())
}
seen := seenStages(t, g)
got := stageNames(seen)
want := []string{"request_start", "routed", "request_end"}
if len(got) != 3 {
t.Fatalf("stages = %v, want %v (a hook that never fires is the failure "+
"this test exists for)", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("stage order = %v, want %v", got, want)
}
}
// The payload must be real data, not empty scaffolding.
end := seen[2]
if end["source"] != "spyup" {
t.Errorf("request_end source = %v, want spyup", end["source"])
}
if end["model"] != "spy-model" {
t.Errorf("request_end model = %v, want spy-model", end["model"])
}
if end["ok"] != true {
t.Errorf("request_end ok = %v, want true", end["ok"])
}
if toInt(t, end["tokens"]) <= 0 {
t.Errorf("request_end prompt_tokens = %v, want the mock upstream's real usage", end["tokens"])
}
}
// TestHooksFireOnRealStreamChat: streaming is a DIFFERENT code path (the record
// is written from a deferred call after the stream ends), so it needs its own
// assertion. A hook that only worked on the non-streaming path would be a real
// bug for agent clients, which are the main streaming users.
func TestHooksFireOnRealStreamChat(t *testing.T) {
g, _ := gatewayWithSpy(t)
rr := doReq(t, g, http.MethodPost, "/v1/chat/completions",
`{"model":"spy-model","stream":true,"messages":[{"role":"user","content":"hi"}]}`)
if rr.Code != http.StatusOK {
t.Fatalf("stream status=%d body=%s", rr.Code, rr.Body.String())
}
got := stageNames(seenStages(t, g))
if len(got) != 3 {
t.Fatalf("stream stages = %v, want all three", got)
}
if got[0] != "request_start" || got[2] != "request_end" {
t.Errorf("stream stage order = %v", got)
}
}
// TestHooksFireOnAutoRequest: the AUTO chain path is a third code path. It also
// must fire, and routed must report tier -2 (resolved by the chain) rather than
// the direct path's -1.
func TestHooksFireOnAutoRequest(t *testing.T) {
g, _ := gatewayWithSpy(t)
// Register the model in the AUTO chain via the admin API.
put := doReq(t, g, http.MethodPut, "/api/auto", `{"rules":[{"model":"spy-model"}]}`)
if put.Code != http.StatusOK {
t.Fatalf("save auto: %d %s", put.Code, put.Body.String())
}
rr := doReq(t, g, http.MethodPost, "/v1/chat/completions",
`{"model":"AUTO","messages":[{"role":"user","content":"hi"}]}`)
if rr.Code != http.StatusOK {
t.Fatalf("auto chat status=%d body=%s", rr.Code, rr.Body.String())
}
seen := seenStages(t, g)
if len(seen) != 3 {
t.Fatalf("AUTO stages = %v, want three", stageNames(seen))
}
if seen[0]["model"] != "AUTO" {
t.Errorf("request_start model = %v, want the requested \"AUTO\"", seen[0]["model"])
}
if seen[1]["model"] != "spy-model" {
t.Errorf("routed model = %v, want the RESOLVED model", seen[1]["model"])
}
}
// TestHooksFireOnFailedRequest is the one that matters most for accounting: a
// request that 500s upstream must still reach request_end, or a billing plugin
// would never see failed traffic.
func TestHooksFireOnFailedRequest(t *testing.T) {
// Point at an upstream that always fails.
bad := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
_, _ = w.Write([]byte(`{"error":"boom"}`))
})
up := newFailingServer(bad)
defer up.Close()
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.yaml")
cfgBody := "listen: :0\n" +
"adapter_dir: " + filepath.Join(dir, "adapters") + "\n" +
"plugin_dir: " + filepath.Join(dir, "plugins") + "\n" +
"runtime_file: " + filepath.Join(dir, "runtime.json") + "\n" +
"gateway_keys:\n - sk-test\n"
os.WriteFile(cfgPath, []byte(cfgBody), 0600)
cfg, err := config.Load(cfgPath)
if err != nil {
t.Fatal(err)
}
cfg.Sources = []config.Source{{
Name: "badup", BaseURL: up.URL, Adapter: "openai", APIKey: "sk-x",
Models: []config.Model{{ID: "bad-model", Kind: "chat"}},
}}
if err := cfg.ApplyDefaults(); err != nil {
t.Fatal(err)
}
c, err := core.NewFromConfig(cfg)
if err != nil {
t.Fatalf("core: %v", err)
}
defer c.Close()
if err := c.Plugins().LoadSource("spy", spyPlugin()); err != nil {
t.Fatal(err)
}
g, err := New(c)
if err != nil {
t.Fatal(err)
}
rr := doReq(t, g, http.MethodPost, "/v1/chat/completions",
`{"model":"bad-model","messages":[{"role":"user","content":"hi"}]}`)
if rr.Code == http.StatusOK {
t.Fatal("expected the upstream failure to surface as a non-200")
}
seen := seenStages(t, g)
// request_start must fire; routed must NOT (no source was ever selected);
// request_end MUST (otherwise billing misses failed traffic).
got := stageNames(seen)
if len(got) == 0 || got[0] != "request_start" {
t.Fatalf("stages = %v, want request_start first", got)
}
hasEnd := false
for _, s := range seen {
if s["stage"] == "request_end" {
hasEnd = true
if s["ok"] != false {
t.Error("request_end reported ok=true for a failed request")
}
}
}
if !hasEnd {
t.Error("a FAILED request never reached request_end; a billing plugin " +
"would not see it at all")
}
if contains(got, "routed") {
t.Error("routed fired for a request that never selected a source")
}
}
// TestBrokenPluginDoesNotBreakForwarding is the safety property, asserted
// end-to-end: a plugin that throws in EVERY stage must not stop a chat from
// being served.
func TestBrokenPluginDoesNotBreakForwarding(t *testing.T) {
up := mockUpstream()
defer up.Close()
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.yaml")
os.WriteFile(cfgPath, []byte("listen: :0\n"+
"adapter_dir: "+filepath.Join(dir, "adapters")+"\n"+
"plugin_dir: "+filepath.Join(dir, "plugins")+"\n"+
"runtime_file: "+filepath.Join(dir, "runtime.json")+"\n"+
"gateway_keys:\n - sk-test\n"), 0600)
cfg, err := config.Load(cfgPath)
if err != nil {
t.Fatal(err)
}
cfg.Sources = []config.Source{{
Name: "u", BaseURL: up.URL, Adapter: "openai", APIKey: "sk-x",
Models: []config.Model{{ID: "m", Kind: "chat"}},
}}
if err := cfg.ApplyDefaults(); err != nil {
t.Fatal(err)
}
c, err := core.NewFromConfig(cfg)
if err != nil {
t.Fatal(err)
}
defer c.Close()
broken := `
local plugin = { name = "broken", version = "1.0.0" }
plugin.hooks = { request_start = "f", routed = "f", request_end = "f" }
function plugin.f(p) error("plugin is broken") end
return plugin
`
if err := c.Plugins().LoadSource("broken", broken); err != nil {
t.Fatalf("load broken: %v", err)
}
g, err := New(c)
if err != nil {
t.Fatal(err)
}
rr := doReq(t, g, http.MethodPost, "/v1/chat/completions",
`{"model":"m","messages":[{"role":"user","content":"hi"}]}`)
if rr.Code != http.StatusOK {
t.Fatalf("a broken plugin broke forwarding: status=%d body=%s", rr.Code, rr.Body.String())
}
if !strings.Contains(rr.Body.String(), "pong") {
t.Errorf("the response is not the upstream's: %s", rr.Body.String())
}
// And the failure must be visible rather than silent.
if len(c.Plugins().HookErrors()) == 0 {
t.Error("a plugin that threw in every stage left no error record")
}
}
// TestPluginListIsServedWithStages is the operator-facing check: an admin
// debugging a plugin needs to see which stages are live and whether the hook
// file is where the kernel thinks it is.
func TestPluginListIsServedWithStages(t *testing.T) {
g, _ := gatewayWithSpy(t)
rr := doReq(t, g, http.MethodGet, "/api/plugins", "")
if rr.Code != http.StatusOK {
t.Fatalf("status=%d", rr.Code)
}
for _, want := range []string{"spy", "request_start", "routed", "request_end", "plugin_dir"} {
if !strings.Contains(rr.Body.String(), want) {
t.Errorf("/api/plugins response lacks %q", want)
}
}
}
func toInt(t *testing.T, v interface{}) int64 {
t.Helper()
f, ok := v.(float64)
if !ok {
return 0
}
return int64(f)
}
func contains(list []string, s string) bool {
for _, x := range list {
if x == s {
return true
}
}
return false
}
// imageUpstream serves an OpenAI-shaped image response.
func imageUpstream() *httptest.Server {
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"created":1,"data":[{"b64_json":"aGk="}]}`))
}))
}
// TestHooksFireOnRealImageRequest: the image path is a THIRD entry point. It
// has its own handler and its own scheduler call, so "chat works" says nothing
// about it — and image traffic is billable, so a cost plugin that never sees
// it under-reports.
func TestHooksFireOnRealImageRequest(t *testing.T) {
up := imageUpstream()
defer up.Close()
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.yaml")
os.WriteFile(cfgPath, []byte("listen: :0\n"+
"adapter_dir: "+filepath.Join(dir, "adapters")+"\n"+
"plugin_dir: "+filepath.Join(dir, "plugins")+"\n"+
"runtime_file: "+filepath.Join(dir, "runtime.json")+"\n"+
"gateway_keys:\n - sk-test\n"), 0600)
cfg, err := config.Load(cfgPath)
if err != nil {
t.Fatal(err)
}
cfg.Sources = []config.Source{{
Name: "imgup", BaseURL: up.URL, Adapter: "openai", APIKey: "sk-x",
Models: []config.Model{{ID: "img-model", Kind: "image"}},
}}
if err := cfg.ApplyDefaults(); err != nil {
t.Fatal(err)
}
c, err := core.NewFromConfig(cfg)
if err != nil {
t.Fatalf("core: %v", err)
}
defer c.Close()
if err := c.Plugins().LoadSource("spy", spyPlugin()); err != nil {
t.Fatal(err)
}
g, err := New(c)
if err != nil {
t.Fatal(err)
}
rr := doReq(t, g, http.MethodPost, "/v1/images/generations",
`{"model":"img-model","prompt":"a cat"}`)
if rr.Code != http.StatusOK {
t.Fatalf("image status=%d body=%s", rr.Code, rr.Body.String())
}
seen := seenStages(t, g)
got := stageNames(seen)
if len(got) != 3 {
t.Fatalf("image stages = %v, want all three (image is a separate handler)", got)
}
if seen[0]["type"] != "image" {
t.Errorf("request_start type = %v, want \"image\"", seen[0]["type"])
}
if seen[1]["source"] != "imgup" {
t.Errorf("routed source = %v, want imgup", seen[1]["source"])
}
}
// TestRejectedChatStillFiresRequestStart pins the ordering decision: request_start
// fires BEFORE the quota/scope gates, so a plugin auditing traffic sees rejected
// requests too. A plugin that only ever saw served requests could not compute a
// true request rate.
func TestRejectedChatStillFiresRequestStart(t *testing.T) {
up := mockUpstream()
defer up.Close()
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.yaml")
os.WriteFile(cfgPath, []byte("listen: :0\n"+
"adapter_dir: "+filepath.Join(dir, "adapters")+"\n"+
"plugin_dir: "+filepath.Join(dir, "plugins")+"\n"+
"runtime_file: "+filepath.Join(dir, "runtime.json")+"\n"+
"gateway_keys:\n - sk-test\n"), 0600)
cfg, err := config.Load(cfgPath)
if err != nil {
t.Fatal(err)
}
cfg.Sources = []config.Source{{
Name: "u", BaseURL: up.URL, Adapter: "openai", APIKey: "sk-x",
Models: []config.Model{{ID: "m", Kind: "chat"}},
}}
if err := cfg.ApplyDefaults(); err != nil {
t.Fatal(err)
}
c, err := core.NewFromConfig(cfg)
if err != nil {
t.Fatal(err)
}
defer c.Close()
if err := c.Plugins().LoadSource("spy", spyPlugin()); err != nil {
t.Fatal(err)
}
g, err := New(c)
if err != nil {
t.Fatal(err)
}
// A model the gateway does not know => 404, rejected before any routing.
rr := doReq(t, g, http.MethodPost, "/v1/chat/completions",
`{"model":"no-such-model","messages":[{"role":"user","content":"hi"}]}`)
if rr.Code != http.StatusNotFound {
t.Fatalf("expected 404 for an unknown model, got %d", rr.Code)
}
got := stageNames(seenStages(t, g))
if !contains(got, "request_start") {
t.Errorf("stages = %v; a REJECTED request must still fire request_start so a "+
"plugin can count real traffic, not just served traffic", got)
}
}
// TestChainStepReachesPluginOnDegradation is the end-to-end proof for the
// AUTO trace: a request that had to drop from tier 1 to tier 2 must be visible
// to a plugin as a tier_skip followed by a selected, and request_end must carry
// tier_served=2.
//
// Before the trace existed the plugin saw only tier=-2 ("resolved by the
// chain") and could not tell a degradation from a clean tier-1 hit — which is
// the whole question a priority chain exists to answer.
func TestChainStepReachesPluginOnDegradation(t *testing.T) {
// tier 1's source always fails, so the walk must drop to tier 2.
bad := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
_, _ = w.Write([]byte(`{"error":"boom"}`))
}))
defer bad.Close()
good := mockUpstream()
defer good.Close()
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.yaml")
os.WriteFile(cfgPath, []byte("listen: :0\n"+
"adapter_dir: "+filepath.Join(dir, "adapters")+"\n"+
"plugin_dir: "+filepath.Join(dir, "plugins")+"\n"+
"runtime_file: "+filepath.Join(dir, "runtime.json")+"\n"+
"gateway_keys:\n - sk-test\n"), 0600)
cfg, err := config.Load(cfgPath)
if err != nil {
t.Fatal(err)
}
cfg.Sources = []config.Source{
{Name: "t1", BaseURL: bad.URL, Adapter: "openai", APIKey: "sk-x",
Models: []config.Model{{ID: "hi-tier", Kind: "chat"}}},
{Name: "t2", BaseURL: good.URL, Adapter: "openai", APIKey: "sk-x",
Models: []config.Model{{ID: "lo-tier", Kind: "chat"}}},
}
if err := cfg.ApplyDefaults(); err != nil {
t.Fatal(err)
}
c, err := core.NewFromConfig(cfg)
if err != nil {
t.Fatal(err)
}
defer c.Close()
// A spy that records chain_step events too.
sp := `
local plugin = { name = "walker", version = "1.0.0" }
plugin.state = { steps = {}, ends = {} }
plugin.hooks = { chain_step = "step", request_end = "fin" }
function plugin.step(p)
table.insert(plugin.state.steps, { kind = p.kind, tier = p.tier, source = p.source, model = p.model, reason = p.reason })
return nil
end
function plugin.fin(p)
plugin.state.ends[#plugin.state.ends + 1] = {
tier_served = p.tier_served, degraded = p.degraded,
walk = p.chain_walk, source = p.source, model = p.model,
}
return nil
end
return plugin
`
if err := c.Plugins().LoadSource("walker", sp); err != nil {
t.Fatal(err)
}
g, err := New(c)
if err != nil {
t.Fatal(err)
}
// Two tiers, both in the chain.
if rr := doReq(t, g, http.MethodPut, "/api/auto",
`{"rules":[{"model":"hi-tier","source":"t1","tier":1},{"model":"lo-tier","source":"t2","tier":2}]}`); rr.Code != 200 {
t.Fatalf("save auto: %d %s", rr.Code, rr.Body.String())
}
rr := doReq(t, g, http.MethodPost, "/v1/chat/completions",
`{"model":"AUTO","messages":[{"role":"user","content":"hi"}]}`)
if rr.Code != http.StatusOK {
t.Fatalf("chat = %d %s", rr.Code, rr.Body.String())
}
raw := c.Plugins().State("walker")
b, _ := json.Marshal(raw)
var st struct {
Steps []struct {
Kind string `json:"kind"`
Tier int `json:"tier"`
Source string `json:"source"`
Model string `json:"model"`
} `json:"steps"`
Ends []struct {
TierServed int `json:"tier_served"`
Degraded bool `json:"degraded"`
Source string `json:"source"`
Model string `json:"model"`
} `json:"ends"`
}
if err := json.Unmarshal(b, &st); err != nil {
t.Fatalf("decode: %v (%s)", err, string(b))
}
if len(st.Steps) < 2 {
t.Fatalf("chain_step events = %+v, want at least a slot_fail and a selected", st.Steps)
}
if st.Steps[0].Kind != "slot_fail" || st.Steps[0].Tier != 1 {
t.Errorf("first step = %+v, want slot_fail on tier 1", st.Steps[0])
}
last := st.Steps[len(st.Steps)-1]
if last.Kind != "selected" || last.Tier != 2 {
t.Errorf("last step = %+v, want selected on tier 2", last)
}
if len(st.Ends) != 1 {
t.Fatalf("request_end count = %d, want 1", len(st.Ends))
}
if st.Ends[0].TierServed != 2 {
t.Errorf("tier_served = %d, want 2", st.Ends[0].TierServed)
}
if !st.Ends[0].Degraded {
t.Error("degraded = false, but the request dropped a tier")
}
if st.Ends[0].Model != "lo-tier" {
t.Errorf("served model = %q, want lo-tier", st.Ends[0].Model)
}
}

View File

@ -0,0 +1,284 @@
package gateway
import (
"encoding/json"
"errors"
"fmt"
"net/http"
"os"
"path/filepath"
"strings"
"llmsproxy/internal/lua"
)
var (
errPluginName = errors.New("plugin name must be non-empty and contain no path separator or dot")
errNoPluginDir = errors.New("no plugin_dir configured; set plugin_dir in config.yaml to enable plugins")
)
// Plugin management API.
//
// GET /api/plugins list loaded plugins + their hooks/UI/errors
// POST /api/plugins upload/replace one plugin (.lua), hot-applied
// DELETE /api/plugins/{name} remove a plugin
// GET /api/plugins/{name}/state the plugin's own published state
// PUT /api/plugins/{name}/state replace that state (admin only)
//
// Why state is a first-class endpoint: request_end is the LAST stage, so a
// hook's return value has no downstream consumer inside the gateway. A plugin
// that accumulates numbers (the billing plugin does exactly this) therefore
// keeps them in its own Lua state and serves them here, which is what its UI
// component fetches. This keeps plugin data clearly separated from the
// gateway's own stats — see the accounting note in docs/plugins.md: the
// gateway's quota accounting stays authoritative, a plugin only reports.
// handlePluginUI serves the merged UI extensions the WebUI injects at boot.
//
// It is a single GET (not per-plugin) because the browser needs ALL extensions
// before it can build the sidebar: a page contributed by one plugin and an
// element contributed by another land in the same payload, and fetching them
// separately would mean the sidebar has to be rebuilt as each arrives.
//
// Served to any authenticated role: the WebUI is authenticated before it asks,
// and a plugin's own widgets need to render for the user whose key they show.
func (g *Gateway) handlePluginUI(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
return
}
ps := g.core.Plugins()
if ps == nil {
writeJSON(w, http.StatusOK, map[string]interface{}{"ui": lua.UIExtension{}})
return
}
// AllStages, not a hand-written list: an earlier version enumerated the
// three stages literally here, so when chain_step was added it was silently
// missing from this response — a plugin author reading the discovery
// document would have believed the stage did not exist. Deriving it from
// the one place that defines the order is the whole point of having it.
writeJSON(w, http.StatusOK, map[string]interface{}{
"ui": ps.UI(),
"stages": pipelineStageNames(),
})
}
// stageNames returns the pipeline stage names in firing order, for the
// discovery payload and for tests.
func pipelineStageNames() []string {
out := make([]string, 0, len(lua.AllStages))
for _, s := range lua.AllStages {
out = append(out, string(s))
}
return out
}
func (g *Gateway) handlePluginsAPI(w http.ResponseWriter, r *http.Request) {
ps := g.core.Plugins()
if ps == nil {
writeJSON(w, http.StatusOK, map[string]interface{}{"plugins": []interface{}{}})
return
}
path := strings.TrimPrefix(r.URL.Path, "/api/plugins")
path = strings.Trim(path, "/")
// /api/plugins/{name}/state
if strings.HasSuffix(path, "/state") {
name := strings.TrimSuffix(path, "/state")
if name == "" {
writeError(w, http.StatusBadRequest, "invalid_request", "plugin name required")
return
}
g.handlePluginState(w, r, name)
return
}
if reqRole(r.Context()) != "admin" {
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
return
}
if path == "" {
switch r.Method {
case http.MethodGet:
writeJSON(w, http.StatusOK, map[string]interface{}{
"plugins": ps.List(),
"on_disk": ps.OnDisk(),
"hook_errors": ps.HookErrors(),
"plugin_dir": g.pluginDir(),
})
case http.MethodPost:
var body struct {
Name string `json:"name"`
Code string `json:"code"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
return
}
if err := g.installPlugin(body.Name, body.Code); err != nil {
writeError(w, http.StatusBadRequest, "plugin_error", err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"ok": true, "name": body.Name})
default:
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "")
}
return
}
// /api/plugins/{name} — source read (GET), enable/disable (PUT), delete.
// Multi-segment paths (…/state) are handled earlier and never reach here.
if !strings.Contains(path, "/") {
switch r.Method {
case http.MethodGet:
// Reading the SOURCE (not the runtime state) is what an editor
// needs; the state endpoint is /state and returns accumulated data
// instead. Mixing the two would make a "save what I read"
// round-trip impossible.
code, err := g.readPluginSource(path)
if err != nil {
writeError(w, http.StatusNotFound, "not_found", err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{
"name": path, "code": code,
"enabled": g.core.Plugins().Enabled(path),
})
case http.MethodPut:
// Enable / disable. The intent rides in the body rather than the
// verb, because "enable" and "disable" are the same resource and
// PUT /{name} is the one the docs advertise.
var body struct {
Enabled *bool `json:"enabled"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
return
}
if body.Enabled == nil {
writeError(w, http.StatusBadRequest, "invalid_request", `body must be {"enabled":true|false}`)
return
}
if err := g.core.Plugins().SetEnabled(path, *body.Enabled); err != nil {
writeError(w, http.StatusBadRequest, "plugin_error", err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{
"ok": true, "name": path, "enabled": *body.Enabled,
})
case http.MethodDelete:
if err := g.removePlugin(path); err != nil {
writeError(w, http.StatusBadRequest, "plugin_error", err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"ok": true})
default:
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "")
}
return
}
writeError(w, http.StatusNotFound, "not_found", "unknown plugin sub-resource: "+path)
}
// readPluginSource returns a plugin's file contents for the editor.
func (g *Gateway) readPluginSource(name string) (string, error) {
if err := validPluginName(name); err != nil {
return "", err
}
if g.pluginDir() == "" {
return "", errNoPluginDir
}
b, err := os.ReadFile(filepath.Join(g.pluginDir(), name+".lua"))
if err != nil {
return "", fmt.Errorf("no plugin source named %q", name)
}
return string(b), nil
}
// validPluginName rejects anything that could escape the plugin directory.
// Shared by install / remove / read so the check cannot drift between them.
func validPluginName(name string) error {
if name == "" || strings.ContainsAny(name, `/\.`) {
return errPluginName
}
return nil
}
// handlePluginState serves GET (read state) and PUT (replace state).
func (g *Gateway) handlePluginState(w http.ResponseWriter, r *http.Request, name string) {
ps := g.core.Plugins()
switch r.Method {
case http.MethodGet:
// Any role may read: plugin state is reporting data (cost, counts),
// and the caller has already been authenticated. Admin-only would stop
// a user key's own billing widget from rendering.
// prices rides along: the billing rules editor must show the table
// that is actually in effect, not a reconstruction from config. It is
// absent for plugins with no prices (every plugin but billing).
payload := map[string]interface{}{
"plugin": name,
"state": ps.State(name),
}
if pr := ps.Prices(name); pr != nil {
payload["prices"] = pr
}
writeJSON(w, http.StatusOK, payload)
case http.MethodPut:
if reqRole(r.Context()) != "admin" {
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
return
}
var state interface{}
if err := json.NewDecoder(r.Body).Decode(&state); err != nil {
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
return
}
if err := ps.SetState(name, state); err != nil {
writeError(w, http.StatusBadRequest, "plugin_error", err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"ok": true})
default:
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "")
}
}
func (g *Gateway) pluginDir() string {
if c := g.core.Config(); c != nil {
return c.PluginDir
}
return ""
}
// installPlugin writes a plugin to disk and hot-loads it. A syntax error is
// returned to the caller AND the file is left on disk so the operator can fix
// it, matching how adapters behave (the file is authoritative once present).
func (g *Gateway) installPlugin(name, code string) error {
if err := validPluginName(name); err != nil {
return err
}
dir := g.pluginDir()
if dir == "" {
return errNoPluginDir
}
if err := os.MkdirAll(dir, 0755); err != nil {
return err
}
if err := os.WriteFile(filepath.Join(dir, name+".lua"), []byte(code), 0644); err != nil {
return err
}
return g.core.Plugins().LoadSource(name, code)
}
func (g *Gateway) removePlugin(name string) error {
if err := validPluginName(name); err != nil {
return err
}
if g.pluginDir() == "" {
return errNoPluginDir
}
_ = os.Remove(filepath.Join(g.pluginDir(), name+".lua"))
return g.core.Plugins().Unload(name)
}

View File

@ -0,0 +1,212 @@
package gateway
import (
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"llmsproxy/internal/config"
"llmsproxy/internal/core"
"llmsproxy/internal/lua"
)
// gatewayWithBilling boots a gateway with the bundled billing plugin loaded, so
// the UI-injection endpoint is exercised against a real plugin rather than a
// hand-written stub. The other plugin tests in this package assert on the
// WebUI source; this one asserts on the HTTP contract the browser consumes.
func gatewayWithBilling(t *testing.T) *Gateway {
t.Helper()
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.yaml")
body := "listen: :0\n" +
"adapter_dir: " + filepath.Join(dir, "adapters") + "\n" +
"plugin_dir: " + filepath.Join(dir, "plugins") + "\n" +
"runtime_file: " + filepath.Join(dir, "runtime.json") + "\n" +
"gateway_keys:\n - sk-test\n"
if err := os.WriteFile(cfgPath, []byte(body), 0600); err != nil {
t.Fatal(err)
}
cfg, err := config.Load(cfgPath)
if err != nil {
t.Fatal(err)
}
c, err := core.NewFromConfig(cfg)
if err != nil {
t.Fatalf("core: %v", err)
}
t.Cleanup(c.Close)
// Load the shipped plugin explicitly: seeding only runs for a directory that
// does not exist yet, and this test wants a known plugin regardless.
src, err := lua.ReadBundledPlugin("billing")
if err != nil {
t.Fatalf("read bundled billing: %v", err)
}
if err := c.Plugins().LoadSource("billing", src); err != nil {
t.Fatalf("load billing: %v", err)
}
g, err := New(c)
if err != nil {
t.Fatalf("gateway: %v", err)
}
return g
}
// TestUIInjectServesPluginUI: GET /api/ui-inject is the single call the WebUI
// makes at boot, and it must carry BOTH a contributed page and contributed
// elements — the browser builds the sidebar from the page and mounts the
// elements into existing panes from the same payload, so a partial response
// would produce a page with no body or a missing tile.
func TestUIInjectServesPluginUI(t *testing.T) {
g := gatewayWithBilling(t)
rr := doReq(t, g, http.MethodGet, "/api/ui-inject", "")
if rr.Code != http.StatusOK {
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
}
var view struct {
// Decoded into the real types so the test cannot drift from the wire
// contract. An inline copy missed the `pages` field when the payload
// shape changed and failed to compile, which is at least loud — but the
// same copy also went on asserting the OLD single-page shape for a
// release, silently.
UI lua.UIExtension `json:"ui"`
Stages []string `json:"stages"`
}
if err := json.Unmarshal(rr.Body.Bytes(), &view); err != nil {
t.Fatalf("decode: %v", err)
}
// The payload carries every page in one list; ui.page is no longer a
// separate slot (it was, and a second plugin contributing a page overwrote
// whatever was there).
var billing *lua.UIPage
for i, pg := range view.UI.Pages {
if pg != nil && pg.PageID == "billing" {
billing = view.UI.Pages[i]
}
}
if billing == nil {
t.Fatalf("no billing page in the inject payload; pages=%d", len(view.UI.Pages))
}
if !strings.Contains(billing.Mount, "billing-root") {
t.Error("the page mount came back empty")
}
if len(view.UI.Elements) == 0 {
t.Error("billing contributes an element to the status page but it is missing")
}
for _, e := range view.UI.Elements {
if e.Target != "status" {
t.Errorf("element target = %q, want \"status\"", e.Target)
}
}
// Derived from AllStages, not hardcoded: the previous assertion of "3"
// is exactly what let chain_step go missing from this payload unnoticed.
if len(view.Stages) != len(lua.AllStages) {
t.Errorf("stages = %v, want %d (one per AllStages entry)", view.Stages, len(lua.AllStages))
}
for i, st := range lua.AllStages {
if i >= len(view.Stages) || view.Stages[i] != string(st) {
t.Errorf("stages[%d] = %v, want %q", i, view.Stages, string(st))
}
}
if !containsStr(view.Stages, string(lua.StageChainStep)) {
t.Error("the discovery payload does not advertise chain_step; a plugin " +
"author would conclude the stage does not exist")
}
}
// TestUIInjectIsEmptyWithoutPlugins: a gateway with no plugins must still answer
// 200 with an empty (not missing, not null) payload. The WebUI calls this
// unconditionally at boot, so a 404 or a null `ui` would break every dashboard.
func TestUIInjectIsEmptyWithoutPlugins(t *testing.T) {
g := newTestGateway(t)
rr := doReq(t, g, http.MethodGet, "/api/ui-inject", "")
if rr.Code != http.StatusOK {
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
}
if !strings.Contains(rr.Body.String(), `"ui"`) {
t.Error("no ui key in the response; the WebUI would have nothing to read")
}
}
// containsStr reports whether list has s.
func containsStr(list []string, s string) bool {
for _, x := range list {
if x == s {
return true
}
}
return false
}
// newRecorderFor pushes a request through the full handler chain.
func newRecorderFor(t *testing.T, g *Gateway, req *http.Request) *httptest.ResponseRecorder {
t.Helper()
rr := httptest.NewRecorder()
g.Handler().ServeHTTP(rr, req)
return rr
}
// TestPluginsListAndStateAPI covers the management surface the plugin docs
// promise: listing, and reading a plugin's own published state.
func TestPluginsListAndStateAPI(t *testing.T) {
g := gatewayWithBilling(t)
rr := doReq(t, g, http.MethodGet, "/api/plugins", "")
if rr.Code != http.StatusOK {
t.Fatalf("GET /api/plugins = %d: %s", rr.Code, rr.Body.String())
}
for _, want := range []string{"billing", "hook_errors", "plugin_dir", "request_end"} {
if !strings.Contains(rr.Body.String(), want) {
t.Errorf("/api/plugins response lacks %q", want)
}
}
// state read: the plugin published its (empty) state, so the key exists.
rr = doReq(t, g, http.MethodGet, "/api/plugins/billing/state", "")
if rr.Code != http.StatusOK {
t.Fatalf("GET state = %d: %s", rr.Code, rr.Body.String())
}
if !strings.Contains(rr.Body.String(), `"total"`) {
t.Errorf("billing state lacks the total bucket: %s", rr.Body.String())
}
}
// TestPluginStatePUTIsAdminOnly: only the WRITE side is gated. A user key must
// be able to READ its own billing widget's data, but must not be able to
// rewrite the price table.
func TestPluginStatePUTIsAdminOnly(t *testing.T) {
g := gatewayWithBilling(t)
// Build a user-role key and remember its secret.
rec, err := g.core.CreateKey("viewer", "user", nil, "")
if err != nil {
t.Fatal(err)
}
userKey := rec.Key
// A user key may read the state.
req, _ := http.NewRequest(http.MethodGet, "/api/plugins/billing/state", nil)
req.Header.Set("Authorization", "Bearer "+userKey)
rr := newRecorderFor(t, g, req)
if rr.Code != http.StatusOK {
t.Errorf("user GET state = %d, want 200 (the billing widget must render for users)", rr.Code)
}
// A user key may NOT write it.
put, _ := http.NewRequest(http.MethodPut, "/api/plugins/billing/state",
strings.NewReader(`{"prices":{"default":{"prompt":0}}}`))
put.Header.Set("Authorization", "Bearer "+userKey)
put.Header.Set("Content-Type", "application/json")
prr := newRecorderFor(t, g, put)
if prr.Code != http.StatusForbidden {
t.Errorf("user PUT state = %d, want 403 (a user must not rewrite the price table)", prr.Code)
}
// An admin key may.
adm := doReq(t, g, http.MethodPut, "/api/plugins/billing/state",
`{"prices":{"default":{"prompt":1e-6}}}`)
if adm.Code != http.StatusOK {
t.Errorf("admin PUT state = %d: %s", adm.Code, adm.Body.String())
}
}

View File

@ -228,6 +228,15 @@ func (g *Gateway) routes(w http.ResponseWriter, r *http.Request) {
g.handleSourcesAPI(w, r)
case r.URL.Path == "/api/source_templates" || strings.HasPrefix(r.URL.Path, "/api/source_templates/"):
g.handleSourceTemplatesAPI(w, r)
case r.URL.Path == "/api/ui-inject" || strings.HasPrefix(r.URL.Path, "/api/ui-inject/"):
g.handlePluginUI(w, r)
case r.URL.Path == "/api/plugins/billing/rules":
// Checked BEFORE the generic /api/plugins/ prefix: the plugin router
// treats anything after the name as a plugin sub-resource and would
// 404 on "rules" instead of reaching the billing editor.
g.handleBillingRules(w, r)
case r.URL.Path == "/api/plugins" || strings.HasPrefix(r.URL.Path, "/api/plugins/"):
g.handlePluginsAPI(w, r)
case r.URL.Path == "/api/chat":
g.handleChat(w, r)
case r.URL.Path == "/api/status":

View File

@ -0,0 +1,495 @@
package gateway
import (
"encoding/json"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"time"
"llmsproxy/internal/config"
"llmsproxy/internal/core"
)
// This file covers the second round of the same class of defect bf0657b fixed.
//
// bf0657b stopped a partial source edit from clobbering api_key. The upsert
// still rewrites the WHOLE source, so every other field the request could not
// express was reset to its zero value on any edit: proxy_url, api_key_env,
// timeout and queue_timeout.
//
// The four are worse than cosmetic because they change how the source dials out.
// proxy_url silently turns a proxied upstream into a direct one (or vice versa)
// and api_key_env turns a source whose credential lives in an environment
// variable into a credential-less source — which keeps answering 200 on write
// and only fails on the NEXT call upstream, long after the editor left.
//
// The trigger is not exotic scripting: the WebUI's own edit form posts exactly
// such a payload. See TestWebUIEditPayloadPreservesRoutingFields.
// richGateway writes a config.yaml whose source carries all four fragile
// fields, then boots a gateway over it.
func richGateway(t *testing.T) *Gateway {
t.Helper()
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.yaml")
cfg := `listen: 127.0.0.1:8080
gateway_keys:
- sk-test
default_model: AUTO
adapter_dir: ` + filepath.Join(dir, "adapters") + `
runtime_file: ` + filepath.Join(dir, "runtime.json") + `
sources:
- name: rich
base_url: https://api.deepseek.com
api_key: sk-live
adapter: deepseek
proxy_url: http://127.0.0.1:7890
timeout: 300s
queue_timeout: 90s
max_concurrent: 8
models:
- id: deepseek-v4-flash
priority: 100
kind: chat
`
if err := os.WriteFile(cfgPath, []byte(cfg), 0600); err != nil {
t.Fatal(err)
}
c, err := config.Load(cfgPath)
if err != nil {
t.Fatal(err)
}
core, err := core.NewFromConfig(c)
if err != nil {
t.Fatalf("core: %v", err)
}
t.Cleanup(core.Close)
g, err := New(core)
if err != nil {
t.Fatal(err)
}
return g
}
// sourceField reads one field off the live source, i.e. what the routes
// actually use rather than what the file happens to contain.
func sourceField(t *testing.T, g *Gateway, name string, get func(config.Source) interface{}) interface{} {
t.Helper()
for _, s := range g.core.Sources() {
if s.Name == name {
return get(s)
}
}
t.Fatalf("source %q not found", name)
return nil
}
// webUIPayload is byte-for-byte the JSON that index.html's saveSource() builds
// from the edit dialog. It deliberately carries none of the four fragile
// fields, because the dialog has no inputs for them.
const webUIPayload = `{
"name":"rich","base_url":"https://api.deepseek.com","api_key":"sk-live",
"adapter":"deepseek","endpoint":"","image_endpoint":"",
"models":[{"id":"deepseek-v4-flash","priority":100,"kind":"chat"}],
"headers":{},"meta":{},"temperature":0,"max_tokens":0,"max_concurrent":16,"rpm":0
}`
// TestWebUIEditPayloadPreservesRoutingFields is the regression test for the
// reported defect: changing max_concurrent from 8 to 16 in the UI must not
// silently drop the proxy, the env-var credential, or the timeouts.
func TestWebUIEditPayloadPreservesRoutingFields(t *testing.T) {
g := richGateway(t)
// The one field the operator actually changed.
rr := doReq(t, g, "POST", "/api/sources", webUIPayload)
if rr.Code != 200 {
t.Fatalf("edit status=%d body=%s", rr.Code, rr.Body.String())
}
checks := []struct {
field string
got interface{}
want interface{}
}{
{"proxy_url", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.ProxyURL }),
"http://127.0.0.1:7890"},
{"timeout", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.Timeout }),
300 * time.Second},
{"queue_timeout", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.QueueTimeout }),
90 * time.Second},
// The field the edit DID change, so a fix that simply refuses to write
// anything would also pass the three assertions above.
{"max_concurrent", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.MaxConcurrent }),
16},
}
for _, c := range checks {
if c.got != c.want {
t.Errorf("%s = %v, want %v", c.field, c.got, c.want)
}
}
}
// TestSourceEditPreservesAPIKeyEnv is called out separately because it is the
// only one of the four whose loss is a credential failure rather than a routing
// difference, and it is invisible until the next upstream call.
func TestSourceEditPreservesAPIKeyEnv(t *testing.T) {
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.yaml")
cfg := `listen: 127.0.0.1:8080
gateway_keys:
- sk-test
default_model: AUTO
adapter_dir: ` + filepath.Join(dir, "adapters") + `
runtime_file: ` + filepath.Join(dir, "runtime.json") + `
sources:
- name: envonly
base_url: https://api.deepseek.com
api_key_env: DEEPSEEK_KEY
adapter: deepseek
max_concurrent: 8
models:
- id: deepseek-v4-flash
priority: 100
kind: chat
`
if err := os.WriteFile(cfgPath, []byte(cfg), 0600); err != nil {
t.Fatal(err)
}
c, err := config.Load(cfgPath)
if err != nil {
t.Fatal(err)
}
core, err := core.NewFromConfig(c)
if err != nil {
t.Fatalf("core: %v", err)
}
t.Cleanup(core.Close)
g, err := New(core)
if err != nil {
t.Fatal(err)
}
// A credential-less source is legitimate, so the API must not invent one
// here; the point is that the ENV reference survives the edit.
rr := doReq(t, g, "POST", "/api/sources", `{
"name":"envonly","base_url":"https://api.deepseek.com","api_key":"",
"adapter":"deepseek","models":[{"id":"deepseek-v4-flash","priority":100,"kind":"chat"}],
"headers":{},"meta":{},"max_concurrent":4}`)
if rr.Code != 200 {
t.Fatalf("edit status=%d body=%s", rr.Code, rr.Body.String())
}
got := sourceField(t, g, "envonly", func(s config.Source) interface{} { return s.APIKeyEnv })
if got != "DEEPSEEK_KEY" {
t.Fatalf("api_key_env = %v after an edit that did not mention it; "+
"the source now has no credential and will only fail on the next upstream call", got)
}
if n := sourceField(t, g, "envonly", func(s config.Source) interface{} { return s.MaxConcurrent }); n != 4 {
t.Fatalf("max_concurrent = %v, want the edited 4", n)
}
}
// TestSourceEditPersistsInheritedFieldsToDisk: inheriting in memory is not
// enough — the values must reach config.yaml, or a restart silently undoes the
// fix and the field is lost anyway.
func TestSourceEditPersistsInheritedFieldsToDisk(t *testing.T) {
g := richGateway(t)
if rr := doReq(t, g, "POST", "/api/sources", webUIPayload); rr.Code != 200 {
t.Fatalf("edit status=%d: %s", rr.Code, rr.Body.String())
}
raw, err := os.ReadFile(g.core.Config().Path)
if err != nil {
t.Fatalf("read config: %v", err)
}
text := string(raw)
for _, want := range []string{
"proxy_url: http://127.0.0.1:7890",
} {
if !strings.Contains(text, want) {
t.Errorf("config.yaml lost %q on an unrelated edit:\n%s", want, text)
}
}
// The durations are re-serialized in Go's canonical form (300s becomes
// 5m0s), so they must be checked SEMANTICALLY. Asserting the original
// spelling would be a false red: the value survived, only its text changed.
// Re-loading the file is also the only way to prove the persisted value
// parses back to the same duration.
reloaded, err := config.Load(g.core.Config().Path)
if err != nil {
t.Fatalf("re-read config: %v", err)
}
if len(reloaded.Sources) == 0 {
t.Fatal("no sources after reload")
}
s := reloaded.Sources[0]
if s.Timeout != 300*time.Second {
t.Errorf("persisted timeout = %v, want 300s", s.Timeout)
}
if s.QueueTimeout != 90*time.Second {
t.Errorf("persisted queue_timeout = %v, want 90s", s.QueueTimeout)
}
if s.ProxyURL != "http://127.0.0.1:7890" {
t.Errorf("persisted proxy_url = %v, want it preserved", s.ProxyURL)
}
}
// ---- explicit set / clear, i.e. the pointer semantics actually work ----
func TestSourceOptionalFieldsCanBeSet(t *testing.T) {
g := richGateway(t)
rr := doReq(t, g, "POST", "/api/sources", `{
"name":"rich","base_url":"https://api.deepseek.com","api_key":"sk-live",
"adapter":"deepseek","models":[{"id":"deepseek-v4-flash","priority":100,"kind":"chat"}],
"headers":{},"meta":{},
"proxy_url":"http://127.0.0.1:1080",
"api_key_env":"NEW_ENV_KEY",
"timeout":"45s",
"queue_timeout":"5s"}`)
if rr.Code != 200 {
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
}
for _, c := range []struct {
field string
got interface{}
want interface{}
}{
{"proxy_url", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.ProxyURL }), "http://127.0.0.1:1080"},
{"api_key_env", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.APIKeyEnv }), "NEW_ENV_KEY"},
{"timeout", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.Timeout }), 45 * time.Second},
{"queue_timeout", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.QueueTimeout }), 5 * time.Second},
} {
if c.got != c.want {
t.Errorf("%s = %v, want %v", c.field, c.got, c.want)
}
}
}
// TestSourceOptionalFieldsCanBeCleared is the half that an "empty means
// inherit" implementation gets wrong: it can never clear a field, so emptying
// the proxy box in the UI would keep using the old proxy forever.
func TestSourceOptionalFieldsCanBeCleared(t *testing.T) {
g := richGateway(t)
rr := doReq(t, g, "POST", "/api/sources", `{
"name":"rich","base_url":"https://api.deepseek.com","api_key":"sk-live",
"adapter":"deepseek","models":[{"id":"deepseek-v4-flash","priority":100,"kind":"chat"}],
"headers":{},"meta":{},
"proxy_url":"","timeout":"","queue_timeout":""}`)
if rr.Code != 200 {
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
}
if got := sourceField(t, g, "rich", func(s config.Source) interface{} { return s.ProxyURL }); got != "" {
t.Errorf("proxy_url = %v, want it cleared", got)
}
// A cleared timeout falls back to the config default (120s), which is what
// mergedSources applies for a zero value — assert the effective behaviour
// rather than the raw zero, since that is what a caller actually gets.
if got := sourceField(t, g, "rich", func(s config.Source) interface{} { return s.Timeout }); got != config.DefaultSourceTimeout {
t.Errorf("timeout = %v, want the default %v after clearing", got, config.DefaultSourceTimeout)
}
// api_key_env was NOT in the request, so it must be untouched even though
// its sibling fields were cleared.
_ = rr
}
// TestSourceOptionalFieldsClearIsScoped: clearing the proxy must not clear the
// env credential that the same request did not mention.
func TestSourceOptionalFieldsClearIsScoped(t *testing.T) {
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.yaml")
cfg := `listen: 127.0.0.1:8080
gateway_keys:
- sk-test
default_model: AUTO
adapter_dir: ` + filepath.Join(dir, "adapters") + `
runtime_file: ` + filepath.Join(dir, "runtime.json") + `
sources:
- name: both
base_url: https://api.deepseek.com
api_key_env: DEEPSEEK_KEY
adapter: deepseek
proxy_url: http://127.0.0.1:7890
max_concurrent: 8
models:
- id: deepseek-v4-flash
priority: 100
kind: chat
`
os.WriteFile(cfgPath, []byte(cfg), 0600)
c, err := config.Load(cfgPath)
if err != nil {
t.Fatal(err)
}
core, err := core.NewFromConfig(c)
if err != nil {
t.Fatalf("core: %v", err)
}
t.Cleanup(core.Close)
g, err := New(core)
if err != nil {
t.Fatal(err)
}
rr := doReq(t, g, "POST", "/api/sources", `{
"name":"both","base_url":"https://api.deepseek.com","api_key":"",
"adapter":"deepseek","models":[{"id":"deepseek-v4-flash","priority":100,"kind":"chat"}],
"headers":{},"meta":{},"proxy_url":""}`)
if rr.Code != 200 {
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
}
if got := sourceField(t, g, "both", func(s config.Source) interface{} { return s.APIKeyEnv }); got != "DEEPSEEK_KEY" {
t.Errorf("api_key_env = %v; clearing proxy_url must not touch it", got)
}
if got := sourceField(t, g, "both", func(s config.Source) interface{} { return s.ProxyURL }); got != "" {
t.Errorf("proxy_url = %v, want cleared", got)
}
}
func TestSourceRejectsBadDuration(t *testing.T) {
g := richGateway(t)
before := sourceField(t, g, "rich", func(s config.Source) interface{} { return s.Timeout })
rr := doReq(t, g, "POST", "/api/sources", `{
"name":"rich","base_url":"https://api.deepseek.com","api_key":"sk-live",
"adapter":"deepseek","models":[{"id":"deepseek-v4-flash","priority":100,"kind":"chat"}],
"headers":{},"meta":{},"timeout":"not-a-duration"}`)
if rr.Code != 400 {
t.Fatalf("status=%d, want 400 for an unparseable duration: %s", rr.Code, rr.Body.String())
}
if !strings.Contains(rr.Body.String(), "timeout") {
t.Errorf("the error must name the offending field: %s", rr.Body.String())
}
// A rejected write must leave the source exactly as it was.
if after := sourceField(t, g, "rich", func(s config.Source) interface{} { return s.Timeout }); after != before {
t.Errorf("a rejected edit still changed timeout: %v -> %v", before, after)
}
}
// TestSourceCreateSetsOptionalFields: the create path must accept them too, and
// must not require an existing source to inherit from (a nil-pointer crash
// here would only show up as a 500 on every new source).
func TestSourceCreateSetsOptionalFields(t *testing.T) {
up := mockUpstream()
defer up.Close()
g := newTestGateway(t)
rr := doReq(t, g, "POST", "/api/sources", `{
"name":"fresh","base_url":"`+up.URL+`","api_key":"sk-x","adapter":"openai",
"models":[{"id":"m1","kind":"chat"}],"headers":{},"meta":{},
"proxy_url":"http://127.0.0.1:7890","timeout":"10s","queue_timeout":"3s"}`)
if rr.Code != 200 {
t.Fatalf("create status=%d body=%s", rr.Code, rr.Body.String())
}
if got := sourceField(t, g, "fresh", func(s config.Source) interface{} { return s.ProxyURL }); got != "http://127.0.0.1:7890" {
t.Errorf("proxy_url = %v, want it set on create", got)
}
if got := sourceField(t, g, "fresh", func(s config.Source) interface{} { return s.Timeout }); got != 10*time.Second {
t.Errorf("timeout = %v, want 10s on create", got)
}
}
// TestSourceRevealExposesDurations guards the round-trip the edit form depends
// on. config.Source tags Timeout/QueueTimeout `json:"-"`, so marshaling the
// struct omits them; the form then reads an empty timeout box and — because it
// always sends the box back — CLEARS the stored timeout on any unrelated save.
// A test that only checks "the edit preserves timeout" would not have caught
// that, because the loss happens between the read and the write.
//
// The credentials are asserted in the same breath: they came from the same
// branch, and an earlier iteration of this fix masked the key and broke the
// form's other half.
func TestSourceRevealExposesDurations(t *testing.T) {
up := mockUpstream()
defer up.Close()
g := newTestGateway(t, config.Source{
Name: "revealed",
BaseURL: up.URL,
Adapter: "openai",
APIKey: "sk-reveal-me",
Models: []config.Model{{ID: "m1", Kind: "chat"}},
Timeout: 300 * time.Second,
})
rr := doReq(t, g, "GET", "/api/v1/sources/revealed?reveal=credentials", "")
if rr.Code != 200 {
t.Fatalf("reveal status=%d: %s", rr.Code, rr.Body.String())
}
var view struct {
Source map[string]interface{} `json:"source"`
}
if err := json.Unmarshal(rr.Body.Bytes(), &view); err != nil {
t.Fatalf("decode: %v", err)
}
// Duration strings, matching what the write path parses.
if got := view.Source["timeout"]; got != "5m0s" {
t.Errorf("reveal timeout = %v, want the string \"5m0s\"; the edit form "+
"would show an empty box and clear the stored value on save", got)
}
// The unset one shows the DEFAULT, not "": ApplyDefaults filled
// QueueTimeout before the form ever sees it. Both are acceptable as long as
// the value round-trips, so assert it is a parseable duration string.
qs, _ := view.Source["queue_timeout"].(string)
if _, err := time.ParseDuration(qs); err != nil {
t.Errorf("reveal queue_timeout = %q is not a duration string: %v", qs, err)
}
if got := view.Source["api_key"]; got != "sk-reveal-me" {
t.Errorf("reveal api_key = %v, want it in the clear for the form", got)
}
// The masked view must NOT gain the durations' secrets — it is fine for it
// to omit them, but it must never carry the key.
masked := doReq(t, g, "GET", "/api/v1/sources/revealed", "")
if strings.Contains(masked.Body.String(), "sk-reveal-me") {
t.Error("the masked view leaked the key")
}
}
// TestSourcePayloadCoversEveryEditableField is the guard against the class
// itself. Any config.Source field a source edit can legitimately set must
// exist on sourcePayload, otherwise the upsert resets it.
//
// It reflects over the STRUCT rather than over json.Marshal output: the four
// pointer fields carry `omitempty`, so a nil pointer legitimately disappears
// from the marshaled form — that absence is precisely the "not mentioned" signal
// the handler relies on. Checking marshaled keys would therefore report a
// healthy field as missing.
func TestSourcePayloadCoversEveryEditableField(t *testing.T) {
// Fields of config.Source that a source edit can legitimately set.
want := []string{
"name", "base_url", "api_key", "api_key_env", "adapter", "endpoint",
"image_endpoint", "models", "headers", "proxy_url", "meta",
"temperature", "max_tokens", "max_concurrent", "rpm",
"timeout", "queue_timeout",
}
pt := reflect.TypeOf(sourcePayload{})
for _, f := range want {
if _, ok := pt.FieldByName(f); !ok {
// Tolerate a differently-spelled Go field only when the JSON tag
// matches, so the check stays about the wire contract.
if !hasJSONTag(pt, f) {
t.Errorf("config.Source has %q but sourcePayload has no such JSON key: "+
"an edit that omits it resets the stored value", f)
}
}
}
}
// hasJSONTag reports whether any field of t carries the given JSON name.
func hasJSONTag(t reflect.Type, name string) bool {
for i := 0; i < t.NumField(); i++ {
if jsonName(t.Field(i)) == name {
return true
}
}
return false
}
// jsonName returns the JSON name of a struct field, falling back to the Go
// field name when no tag is present.
func jsonName(f reflect.StructField) string {
tag := f.Tag.Get("json")
if tag == "" {
return f.Name
}
if n := strings.Split(tag, ",")[0]; n != "" {
return n
}
return f.Name
}

View File

@ -52,6 +52,23 @@ type Req struct {
// Kept separate from Compl/Prompt: image generation has no token concept,
// so counting images as "completion tokens" would corrupt the token totals.
ImageCount int `json:"image_count,omitempty"`
// Walk is the AUTO chain's step-by-step trace for this request: which tiers
// were skipped and why, which slots hard-failed, which one served it. It is
// the only way a consumer can tell "tier 1 served this" from "tier 1 was
// cooling so we dropped to tier 3" — a distinction that is the entire point
// of a priority chain.
//
// json:"-" — deliberately NOT persisted. The audit file is a hot append and
// this is observational detail: on a degraded gateway every request would
// carry a multi-element array, and the audit trail's own retention (16 files
// x 16 MB) is already the largest thing on the box. A plugin that wants the
// walk sees it live at request_end; an operator post-mortem reads it from the
// plugin's own accumulated state or from /api/auto slot health.
//
// Only populated when a plugin is loaded (chainTraceSink returns nil
// otherwise), so a gateway with no plugins allocates nothing for it.
Walk []map[string]interface{} `json:"-"`
}
// Stat aggregates counters for one dimension row.
@ -74,6 +91,16 @@ type agrRow struct {
// Stats collects per-key / per-model / per-source aggregates plus a bounded
// ring of raw request records, all guarded by one mutex.
//
// LOCKING: mu is a plain sync.Mutex and is NOT reentrant. The *Locked methods
// (aggregateLocked, addKeyTokenLocked, addKeyHourLocked, addKeyReqLocked,
// wantPinnedBuckets, rotateAuditLocked, …) assume the caller already holds it,
// while every other exported method takes it itself.
//
// Calling an exported method while already holding mu DEADLOCKS. This is not
// hypothetical: a test that did KeyWindowReqs under s.mu.Lock() hung until the
// 10-minute panic timeout. Always reach for the *Locked variant when the lock
// is already held, and prefer the exported method when it is not.
type Stats struct {
mu sync.Mutex
active int64

View File

@ -0,0 +1,132 @@
package gateway
import (
"reflect"
"runtime"
"strings"
"testing"
"time"
)
// The Stats mutex is a plain sync.Mutex: calling an exported (self-locking)
// method while already holding it deadlocks. That happened for real — a probe
// held s.mu and called KeyWindowReqs, hanging until the test binary's 10-minute
// panic timeout — so the rule is pinned here rather than left in a comment.
//
// The check is structural: every method that takes the lock must say so in its
// name or its doc comment. That keeps the trap visible when someone adds the
// next method, which is the only time the rule can be forgotten.
// exportedSelfLocking lists the exported Stats methods that take the mutex.
// It is derived from reflection at run time; the assertions below are what
// actually pin the convention.
func TestStatsExportedMethodsDocumentTheirLocking(t *testing.T) {
typ := reflect.TypeOf(&Stats{})
for i := 0; i < typ.NumMethod(); i++ {
m := typ.Method(i)
if m.PkgPath != "" { // unexported
continue
}
// Methods that neither lock nor touch guarded state are fine either way;
// the ones that matter are those reaching into the maps under mu.
if !methodTouchesLockedState(m.Name) {
continue
}
if strings.HasSuffix(m.Name, "Locked") {
t.Errorf("Stats.%s is exported but named *Locked; the suffix means "+
"the CALLER holds the lock, so it must not be exported", m.Name)
}
}
}
// methodTouchesLockedState is the set of exported methods known to read or
// write state guarded by Stats.mu. Kept explicit (rather than inferred) so a
// new method is not silently assumed safe.
func methodTouchesLockedState(name string) bool {
switch name {
case "KeyWindowReqs", "KeyWindowModelTokens", "KeyWindowTokens",
"WindowTokens", "KeyTokens", "ModelTokens", "Record", "AppendAudit",
"LoadAudit", "Snapshot", "SourceRecent", "SourceAverages",
"AuditRecords", "AuditPage", "StreamAuditRecords", "ReplayPartial",
"PoolStats":
return true
}
return false
}
// TestStatsLockedMethodsAreNotCalledUnderLock is the behavioural half: it
// proves the internal helpers the ones above use are reachable while the lock
// is held. If a future refactor makes an exported method call a *Locked one
// while holding mu itself, this is where it shows up — as a hang, bounded by
// the short timeout below rather than the suite's 10 minutes.
func TestStatsLockedMethodsAreNotCalledUnderLock(t *testing.T) {
done := make(chan struct{})
go func() {
defer close(done)
s := NewStats(10)
h := time.Now().Unix() / 3600
s.mu.Lock()
// Exactly the shape that deadlocked: the *Locked variants are correct
// here because the lock is already held.
s.addKeyReqLocked("k", h, 5)
s.addKeyHourLocked("k", h, 100)
s.wantPinnedBuckets("k", "m1")
s.addKeyTokenLocked("k", "m1", "src", h, 100)
// sumBuckets is the pure inner function the exported readers call.
if got := sumBuckets(s.keyReqHour["k"], time.Now().Unix(), 3600); got != 5 {
t.Errorf("sumBuckets under lock = %d, want 5", got)
}
s.mu.Unlock()
}()
select {
case <-done:
case <-time.After(20 * time.Second):
buf := make([]byte, 1<<16)
n := runtime.Stack(buf, true)
t.Fatalf("deadlocked while using the *Locked helpers under s.mu:\n%s", buf[:n])
}
}
// TestSumBucketsEdgeCases covers the boundaries the quota check depends on,
// including the ones a regression would silently get wrong (an off-by-one here
// either lets a quota leak or locks a key out early).
func TestSumBucketsEdgeCases(t *testing.T) {
const hour = 3600
now := int64(10*hour + 61) // 10:00:61, i.e. just past the boundary
buckets := map[int64]int64{
0: 100, // ancient
9: 200, // previous hour
10: 7, // current hour
}
if got := sumBuckets(buckets, now, 0); got != 307 {
t.Errorf("sec<=0 (all history) = %d, want 307", got)
}
// 1h window covers buckets h >= ceil((now-3600)/3600) = 10 -> only bucket 10.
if got := sumBuckets(buckets, now, hour); got != 7 {
t.Errorf("1h window = %d, want 7 (buckets 0 and 9 fall outside)", got)
}
// 2h window covers h >= 9 -> buckets 9 and 10.
if got := sumBuckets(buckets, now, 2*hour); got != 207 {
t.Errorf("2h window = %d, want 207", got)
}
// An empty map and a nil map must both be zero, not a panic.
if got := sumBuckets(map[int64]int64{}, now, hour); got != 0 {
t.Errorf("empty map = %d, want 0", got)
}
if got := sumBuckets(nil, now, hour); got != 0 {
t.Errorf("nil map = %d, want 0", got)
}
// now < sec must not produce a negative firstHour index: with a window far
// wider than the available history, every bucket that exists is counted.
// (sumBuckets walks bucket indices from firstHour to nowHour, so passing a
// "now" older than some buckets cannot reach them — that is correct, not a
// truncation bug.)
if got := sumBuckets(buckets, 10*hour+61, 100*hour); got != 307 {
t.Errorf("window wider than history = %d, want 307", got)
}
// A window that predates every bucket counts them all as well.
if got := sumBuckets(buckets, 10*hour+61, hour); got != 7 {
t.Errorf("1h window at t=10:00:61 = %d, want 7", got)
}
}

View File

@ -0,0 +1,234 @@
package gateway
import (
"sort"
"strconv"
"strings"
"time"
)
// Period is a reporting window for the usage dashboard. The dashboard used to
// have exactly one view — everything since process start — which made a quiet
// morning indistinguishable from a quiet week. Periods give the operator a
// scale to read the numbers at: today vs this week vs this month vs all time.
//
// The set is deliberately calendar-based and UTC-anchored. A rolling 24h window
// would put "today" and "the last day" at different boundaries, so the same
// number would move between two cards depending on when you looked; calendar
// days are what people actually mean by "today". UTC also matches the billing
// plugin's peak-window arithmetic, so a peak-rate hour does not land in a
// different day in the cost view than in the usage view.
type Period string
const (
// PeriodDay is the current UTC calendar day.
PeriodDay Period = "day"
// PeriodWeek is the current ISO week (Mon 00:00 UTC to now).
PeriodWeek Period = "week"
// PeriodMonth is the current UTC calendar month.
PeriodMonth Period = "month"
// PeriodAll is since process start — the only view backed by the
// in-memory aggregates, and the only one available when no audit file is
// configured.
PeriodAll Period = "all"
)
// ValidPeriod reports whether p is a period the aggregator understands.
// An unknown period is a client error, not a silent fallback to "all": a
// dashboard that quietly shows lifetime totals when the caller asked for today
// is worse than one that refuses.
func ValidPeriod(p Period) bool {
switch p {
case PeriodDay, PeriodWeek, PeriodMonth, PeriodAll:
return true
}
return false
}
// periodStart returns the inclusive start of the window for p at time now.
// Only PeriodDay/Week/Month are meaningful here; PeriodAll returns 0, which
// every "from > 0" bounds check treats as unbounded.
func periodStart(p Period, now time.Time) int64 {
now = now.UTC()
switch p {
case PeriodDay:
return time.Date(now.Year(), now.Month(), now.Day(), 0, 0, 0, 0, time.UTC).UnixMilli()
case PeriodWeek:
// ISO week starts Monday. Go's Weekday() is Sunday=0, so the shift
// below is 1 on Sunday and 0 on Monday..Saturday.
off := (int(now.Weekday()) + 6) % 7
d := now.AddDate(0, 0, -off)
return time.Date(d.Year(), d.Month(), d.Day(), 0, 0, 0, 0, time.UTC).UnixMilli()
case PeriodMonth:
return time.Date(now.Year(), now.Month(), 1, 0, 0, 0, 0, time.UTC).UnixMilli()
}
return 0
}
// PeriodBucket is one labelled point on the dashboard's timeline. Buckets are
// the aggregation grain for a period: hourly for a day (so a spike is visible
// inside "today"), daily for a week or month (so a week is not 7×24 points
// wide and a month is not 31×24), and empty for "all" — a lifetime total has
// no meaningful timeline, and pretending otherwise by drawing 500 hourly
// buckets of rolling memory would be a different lie.
type PeriodBucket struct {
Bucket string `json:"bucket"`
Stat
}
// PeriodSnapshot is the period-scoped twin of Snapshot's payload: the same
// totals and the same by_* rows, plus a timeline. It deliberately mirrors
// Snapshot's field names so the dashboard's table painters work unchanged —
// paintModelTable(st.by_model) reads rows of {name, ...Stat}, and that shape
// does not care where the numbers came from.
type PeriodSnapshot struct {
Period Period `json:"period"`
From int64 `json:"from"` // unix millis, 0 when PeriodAll
Total Stat `json:"total"`
ByKey []StatsRow `json:"by_key"`
Models []StatsRow `json:"by_model"`
Srcs []StatsRow `json:"by_source"`
Status []agrRow `json:"by_status"`
Bucket []PeriodBucket `json:"buckets"`
// Truncated marks that the window was clipped by the retained audit
// history, so the numbers are a lower bound rather than the true period
// total. The dashboard shows this next to the numbers rather than letting
// a rotated-away week read as "that week had no traffic".
Truncated bool `json:"truncated"`
}
// bucketKey maps a record's timestamp onto the timeline grain for p.
// Daily buckets are stamped at UTC midnight; hourly buckets carry the hour.
func bucketKey(p Period, ms int64) string {
t := time.UnixMilli(ms).UTC()
if p == PeriodDay {
return t.Format("2006-01-02T15")
}
return t.Format("2006-01-02")
}
// bucketOf returns the bucket label plus the truncated-flag side effects of
// walking a file: a record older than the requested window means the window
// starts before the retained history, and the file may have been cut short.
func (s *Stats) PeriodSnapshot(p Period, key string, now time.Time) PeriodSnapshot {
out := PeriodSnapshot{Period: p}
from := periodStart(p, now)
out.From = from
// "all" is exactly what Snapshot already answers, from the in-memory
// aggregates, and it is the one view that must keep working with no audit
// file configured at all (a fresh dev setup, or an operator who turned
// auditing off). Serving it from the same code path keeps the dashboard's
// "total" card identical whether or not a period is selected.
if p == PeriodAll {
snap := s.Snapshot(firstScreenRecords, key)
if tot, ok := snap["total"].(Stat); ok {
out.Total = tot
}
out.ByKey, _ = snap["by_key"].([]StatsRow)
out.Models, _ = snap["by_model"].([]StatsRow)
out.Srcs, _ = snap["by_source"].([]StatsRow)
out.Status, _ = snap["by_status"].([]agrRow)
// replay_partial is the same "these numbers came from a bounded
// tail" caveat, carried through under this view's own name.
out.Truncated, _ = snap["replay_partial"].(bool)
return out
}
// A bounded window is aggregated from the audit files, because the
// in-memory aggregates are lifetime totals and the ring buffer holds only
// maxRecs records (500 by default). Reading the ring would silently
// under-report any period longer than the last few hundred requests.
total := Stat{}
byKey := map[string]*Stat{}
byModel := map[string]*Stat{}
bySrc := map[string]*Stat{}
byStatus := map[string]*Stat{}
buckets := map[string]*Stat{}
var seen int
err := s.StreamAuditRecords(from, 0, key, func(r Req) error {
seen++
incStatus(&total, "", r)
inc(byKey, r.Key, r)
if r.Model != "" {
inc(byModel, r.Model, r)
}
inc(bySrc, r.Source, r)
if r.Status != 0 {
inc(byStatus, strconv.Itoa(r.Status), r)
}
k := bucketKey(p, r.Time)
b := buckets[k]
if b == nil {
b = &Stat{}
buckets[k] = b
}
incStatus(b, k, r)
return nil
})
out.Total = total
out.ByKey = rows(byKey)
out.Models = rows(byModel)
out.Srcs = rows(bySrc)
bs := make([]agrRow, 0, len(byStatus))
for code, st := range byStatus {
bs = append(bs, agrRow{Name: code, Stat: *st})
}
sort.Slice(bs, func(i, j int) bool {
ci, _ := strconv.Atoi(bs[i].Name)
cj, _ := strconv.Atoi(bs[j].Name)
return ci < cj
})
out.Status = bs
ks := make([]string, 0, len(buckets))
for k := range buckets {
ks = append(ks, k)
}
// Chronological, string-sorted: "2006-01-02T15" and "2006-01-02" both
// sort lexicographically in time order, so no date parsing is needed.
sort.Strings(ks)
out.Bucket = make([]PeriodBucket, 0, len(ks))
for _, k := range ks {
out.Bucket = append(out.Bucket, PeriodBucket{Bucket: k, Stat: *buckets[k]})
}
// The window is only "complete" if the audit walk actually reached back
// far enough to cover it. Two ways it cannot:
//
// 1. The walk found nothing at all in a window that certainly had
// traffic, because the files holding it rotated away.
// 2. The walk errored part-way (I/O), leaving a partial total.
//
// Case 2 is reported from err; case 1 from seen == 0 combined with the
// caller having asked for a bounded window. It is deliberately
// conservative: a genuinely empty hour is rare enough that flagging it as
// possibly-truncated costs one tooltip, whereas silently under-reporting a
// month because rotation ate it is a wrong number with no indication.
if err != nil || (seen == 0 && p != PeriodAll) {
out.Truncated = true
}
return out
}
// periodFromQuery parses the ?period= parameter. An empty value means "all" so
// that existing callers of /api/stats keep seeing exactly what they saw.
// A malformed value is rejected by the caller (ValidPeriod) rather than
// defaulting, so a typo in a bookmarked URL surfaces as an error instead of
// silently switching the operator to lifetime totals.
func periodFromQuery(q map[string][]string) Period {
v := strings.ToLower(strings.TrimSpace(firstQuery(q, "period")))
if v == "" {
return PeriodAll
}
return Period(v)
}
func firstQuery(q map[string][]string, key string) string {
if vs, ok := q[key]; ok && len(vs) > 0 {
return vs[0]
}
return ""
}

View File

@ -0,0 +1,321 @@
package gateway
import (
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"time"
)
// auditPeriod writes records straight into an audit file, which is what the
// period aggregator reads. Going through Record() would also rotate/prune the
// file; here the file shape is the thing under test.
func auditPeriod(t *testing.T, path string, recs ...Req) {
t.Helper()
f, err := os.Create(path)
if err != nil {
t.Fatalf("create audit file: %v", err)
}
defer f.Close()
for _, r := range recs {
if r.Time == 0 {
t.Fatal("test record needs an explicit timestamp")
}
if r.Type == "" {
r.Type = "chat"
}
b, err := json.Marshal(r)
if err != nil {
t.Fatalf("marshal record: %v", err)
}
if _, err := f.Write(append(b, '\n')); err != nil {
t.Fatalf("write record: %v", err)
}
}
}
func periodStats(t *testing.T, auditPath string) *Stats {
t.Helper()
s := NewStats(10)
s.mu.Lock()
s.auditPath = auditPath
s.mu.Unlock()
return s
}
// TestPeriodWindowBoundaries is the boundary contract: a day window covers
// exactly today-from-midnight, a week window starts Monday (not Sunday — the
// Go Weekday() trap), and a month window starts on the 1st. All UTC.
func TestPeriodWindowBoundaries(t *testing.T) {
// Wednesday 2026-03-11 15:30 UTC.
now := time.Date(2026, 3, 11, 15, 30, 0, 0, time.UTC)
day := time.Date(2026, 3, 11, 0, 0, 0, 0, time.UTC)
if got := time.UnixMilli(periodStart(PeriodDay, now)).UTC(); !got.Equal(day) {
t.Errorf("day window starts %s, want %s", got, day)
}
// Wednesday minus 2 days = Monday the 9th.
monday := time.Date(2026, 3, 9, 0, 0, 0, 0, time.UTC)
if got := time.UnixMilli(periodStart(PeriodWeek, now)).UTC(); !got.Equal(monday) {
t.Errorf("week window starts %s, want Monday %s", got, monday)
}
// Same instant on a Sunday must still start on the PRECEDING Monday,
// never on the Sunday itself. This is the off-by-one that
// int(now.Weekday()) would introduce (Sunday=0 → no shift).
sun := time.Date(2026, 3, 15, 10, 0, 0, 0, time.UTC) // Sunday
prevMon := time.Date(2026, 3, 9, 0, 0, 0, 0, time.UTC)
if got := time.UnixMilli(periodStart(PeriodWeek, sun)).UTC(); !got.Equal(prevMon) {
t.Errorf("Sunday week window starts %s, want the Monday before it %s", got, prevMon)
}
first := time.Date(2026, 3, 1, 0, 0, 0, 0, time.UTC)
if got := time.UnixMilli(periodStart(PeriodMonth, now)).UTC(); !got.Equal(first) {
t.Errorf("month window starts %s, want %s", got, first)
}
if got := periodStart(PeriodAll, now); got != 0 {
t.Errorf("all window from = %d, want 0 (unbounded)", got)
}
}
// TestPeriodAggregationExcludesOlderRecords is the reason the aggregator reads
// the audit file: a record outside the window must not be counted, even though
// it sits in the very same file, right next to records that are.
func TestPeriodAggregationExcludesOlderRecords(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "audit.jsonl")
now := time.Date(2026, 3, 11, 15, 0, 0, 0, time.UTC)
h := func(n int) int64 { return now.Add(-time.Duration(n) * time.Hour).UnixMilli() }
auditPeriod(t, path,
// inside today
Req{Time: h(1), Model: "m", Source: "s", Prompt: 100, Compl: 10, LatMs: 5, OK: true},
Req{Time: h(2), Model: "m", Source: "s", Prompt: 200, Compl: 20, LatMs: 7, OK: true},
// outside today (yesterday) but inside the week
Req{Time: now.AddDate(0, 0, -1).UnixMilli(), Model: "old", Source: "s", Prompt: 999, Compl: 99, OK: true},
)
s := periodStats(t, path)
day := s.PeriodSnapshot(PeriodDay, "", now)
if day.Total.Reqs != 2 {
t.Errorf("day window counted %d requests, want 2 (yesterday's record must be excluded)", day.Total.Reqs)
}
if day.Total.Prompt != 300 {
t.Errorf("day prompt tokens = %d, want 300", day.Total.Prompt)
}
week := s.PeriodSnapshot(PeriodWeek, "", now)
if week.Total.Reqs != 3 {
t.Errorf("week window counted %d requests, want 3", week.Total.Reqs)
}
if week.Total.Prompt != 1299 {
t.Errorf("week prompt tokens = %d, want 1299", week.Total.Prompt)
}
}
// TestPeriodDimensionsAggregateIndependently checks the by_* rows actually
// split by their own key rather than all collapsing into one row.
func TestPeriodDimensionsAggregateIndependently(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "audit.jsonl")
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
auditPeriod(t, path,
Req{Time: now.UnixMilli(), Key: "k1", Model: "alpha", Source: "s1", Prompt: 10, OK: true},
Req{Time: now.UnixMilli(), Key: "k1", Model: "beta", Source: "s2", Prompt: 20, OK: true},
Req{Time: now.UnixMilli(), Key: "k2", Model: "alpha", Source: "s1", Prompt: 30, OK: true},
)
s := periodStats(t, path)
got := s.PeriodSnapshot(PeriodDay, "", now)
if len(got.Models) != 2 {
t.Fatalf("by_model has %d rows, want 2: %+v", len(got.Models), got.Models)
}
// alpha = 10 + 30
for _, r := range got.Models {
if r.Name == "alpha" && r.Prompt != 40 {
t.Errorf("alpha prompt = %d, want 40", r.Prompt)
}
}
if len(got.Srcs) != 2 {
t.Errorf("by_source has %d rows, want 2: %+v", len(got.Srcs), got.Srcs)
}
if len(got.ByKey) != 2 {
t.Errorf("by_key has %d rows, want 2: %+v", len(got.ByKey), got.ByKey)
}
// The rows must add up to the total, or the dashboard shows a total that
// disagrees with its own table.
var sum int64
for _, r := range got.Models {
sum += r.Prompt
}
if sum != got.Total.Prompt {
t.Errorf("by_model prompts sum to %d but total is %d — the table would contradict the KPI", sum, got.Total.Prompt)
}
}
// TestPeriodDayBucketsAreHourlyAndOrdered pins the timeline grain and its
// order. Day = hourly (a spike must be visible inside "today"); week/month =
// daily. Sorted chronologically, because a chart fed unsorted buckets draws
// nonsense.
func TestPeriodDayBucketsAreHourlyAndOrdered(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "audit.jsonl")
now := time.Date(2026, 3, 11, 15, 0, 0, 0, time.UTC)
// Deliberately out of order in the file.
auditPeriod(t, path,
Req{Time: now.Add(-2 * time.Hour).UnixMilli(), Model: "m", Prompt: 1, OK: true},
Req{Time: now.Add(-4 * time.Hour).UnixMilli(), Model: "m", Prompt: 1, OK: true},
Req{Time: now.UnixMilli(), Model: "m", Prompt: 1, OK: true},
)
s := periodStats(t, path)
got := s.PeriodSnapshot(PeriodDay, "", now)
if len(got.Bucket) != 3 {
t.Fatalf("day buckets = %d, want 3 (hourly): %+v", len(got.Bucket), got.Bucket)
}
for i := 1; i < len(got.Bucket); i++ {
if got.Bucket[i-1].Bucket >= got.Bucket[i].Bucket {
t.Errorf("day buckets not chronological at %d: %q >= %q",
i, got.Bucket[i-1].Bucket, got.Bucket[i].Bucket)
}
}
// Weekly grain is daily, so the same three records collapse to one day.
week := s.PeriodSnapshot(PeriodWeek, "", now)
if len(week.Bucket) != 1 {
t.Errorf("week buckets = %d, want 1 (daily grain): %+v", len(week.Bucket), week.Bucket)
}
}
// TestPeriodKeyFilterIsolatesKeys guards the multi-tenant boundary: asking for
// one key must never return another key's rows in any dimension.
func TestPeriodKeyFilterIsolatesKeys(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "audit.jsonl")
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
auditPeriod(t, path,
Req{Time: now.UnixMilli(), Key: "alice", Model: "m", Source: "s", Prompt: 10, OK: true},
Req{Time: now.UnixMilli(), Key: "bob", Model: "m", Source: "s", Prompt: 999, OK: true},
)
s := periodStats(t, path)
got := s.PeriodSnapshot(PeriodDay, "alice", now)
if got.Total.Reqs != 1 || got.Total.Prompt != 10 {
t.Errorf("alice's window = %d reqs / %d prompt, want 1 / 10 — another key leaked in",
got.Total.Reqs, got.Total.Prompt)
}
if len(got.Srcs) != 1 || got.Srcs[0].Prompt != 10 {
t.Errorf("by_source leaked: %+v", got.Srcs)
}
}
// TestPeriodAllUsesLifetimeAggregates checks "all" still answers from the
// in-memory aggregates (no audit file needed) and carries the same rows the
// dashboard already renders.
func TestPeriodAllUsesLifetimeAggregates(t *testing.T) {
s := NewStats(100)
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
s.Record(Req{Time: now.AddDate(0, 0, -40).UnixMilli(), Key: "k", Model: "m", Source: "s", Prompt: 7, OK: true})
s.Record(Req{Time: now.UnixMilli(), Key: "k", Model: "m", Source: "s", Prompt: 3, OK: true})
got := s.PeriodSnapshot(PeriodAll, "", now)
if got.Total.Reqs != 2 || got.Total.Prompt != 10 {
t.Errorf("all view = %d reqs / %d prompt, want 2 / 10 (lifetime)", got.Total.Reqs, got.Total.Prompt)
}
if len(got.Models) != 1 || got.Models[0].Prompt != 10 {
t.Errorf("all view by_model = %+v, want one row with 10 prompt", got.Models)
}
// A lifetime total has no timeline to draw; buckets must stay empty
// rather than inventing 500 hourly points.
if len(got.Bucket) != 0 {
t.Errorf("all view produced %d buckets, want 0", len(got.Bucket))
}
}
// TestPeriodEmptyWindowFlagsTruncated covers the "rotated away" case: a bounded
// window with nothing to show may mean "quiet day" or "the files are gone".
// The aggregator must not report a confidently wrong zero.
func TestPeriodEmptyWindowFlagsTruncated(t *testing.T) {
dir := t.TempDir()
s := periodStats(t, filepath.Join(dir, "audit.jsonl")) // file does not exist
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
got := s.PeriodSnapshot(PeriodDay, "", now)
if got.Total.Reqs != 0 {
t.Errorf("empty window reqs = %d, want 0", got.Total.Reqs)
}
if !got.Truncated {
t.Error("★ an empty bounded window must be flagged truncated — otherwise a " +
"rotated-away week reads as 'no traffic that week'")
}
}
// TestValidPeriodRejectsUnknown guards the deliberate strictness: a typo must
// be an error, not a silent fallback to lifetime totals.
func TestValidPeriodRejectsUnknown(t *testing.T) {
for _, ok := range []Period{PeriodDay, PeriodWeek, PeriodMonth, PeriodAll} {
if !ValidPeriod(ok) {
t.Errorf("ValidPeriod(%q) = false, want true", ok)
}
}
for _, bad := range []Period{"", "year", "hour", "today", "DAY "} {
if ValidPeriod(bad) {
t.Errorf("ValidPeriod(%q) = true, want false", bad)
}
}
}
// TestPeriodStatusDimensionCountsFailures covers by_status, which nothing else
// exercised — an aggregation branch with no test is exactly how a period view
// silently loses the failure pie. M8 (disabling the status dimension) is
// caught only because this test exists.
func TestPeriodStatusDimensionCountsFailures(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "audit.jsonl")
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
auditPeriod(t, path,
Req{Time: now.UnixMilli(), Model: "m", Source: "s", Status: 200, OK: true, Prompt: 5},
Req{Time: now.UnixMilli(), Model: "m", Source: "s", Status: 502, OK: false, Prompt: 1},
Req{Time: now.UnixMilli(), Model: "m", Source: "s", Status: 502, OK: false, Prompt: 1},
Req{Time: now.UnixMilli(), Model: "m", Source: "s", Status: 429, OK: false, Prompt: 1},
)
s := periodStats(t, path)
got := s.PeriodSnapshot(PeriodDay, "", now)
byCode := map[string]int64{}
for _, r := range got.Status {
byCode[r.Name] = r.Reqs
}
if len(got.Status) != 3 {
t.Fatalf("by_status has %d rows, want 3 (200/502/429): %+v", len(got.Status), got.Status)
}
if byCode["200"] != 1 || byCode["502"] != 2 || byCode["429"] != 1 {
t.Errorf("by_status counts = %v, want 200:1 502:2 429:1", byCode)
}
// Status codes must sort numerically: "429" before "502" lexically would
// render the failure legend in the wrong order.
if got.Status[0].Name != "200" || got.Status[1].Name != "429" || got.Status[2].Name != "502" {
t.Errorf("by_status not numerically sorted: %s, %s, %s",
got.Status[0].Name, got.Status[1].Name, got.Status[2].Name)
}
// The failure rows must agree with the OK/Err split on the total, or the
// pie and the KPI row contradict each other.
if got.Total.OK != 1 || got.Total.Err != 3 {
t.Errorf("total ok/err = %d/%d, want 1/3", got.Total.OK, got.Total.Err)
}
}
func TestPeriodFromQuery(t *testing.T) {
cases := []struct {
query string
want Period
}{
{"", PeriodAll}, // no selector → unchanged legacy behaviour
{"?period=day", PeriodDay},
{"?period=WEEK", PeriodWeek},
{"?period=month", PeriodMonth},
{"?period=all", PeriodAll},
}
for _, c := range cases {
r := httptest.NewRequest(http.MethodGet, "/api/stats"+c.query, nil)
if got := periodFromQuery(r.URL.Query()); got != c.want {
t.Errorf("periodFromQuery(%q) = %q, want %q", c.query, got, c.want)
}
}
}

View File

@ -615,6 +615,21 @@
<path d="M14 2v6h6M16 13H8M16 17H8M10 9H8" /></svg
><span data-i="navAdapters">适配器</span>
</button>
<button class="sb-i" data-tab="plugins">
<svg
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
stroke-linecap="round"
stroke-linejoin="round"
>
<path
d="M10 2v4M14 2v4M4 8h16M5 6h14a1 1 0 0 1 1 1v11a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2V7a1 1 0 0 1 1-1z"
/>
<path d="M10 13h4" /></svg
><span data-i="navPlugins">插件</span>
</button>
</nav>
<div class="sb-foot">
<button id="btn-lang">
@ -767,6 +782,7 @@
<div id="tab-sort" class="tab-pane hidden"></div>
<div id="tab-sources" class="tab-pane hidden"></div>
<div id="tab-adapters" class="tab-pane hidden"></div>
<div id="tab-plugins" class="tab-pane hidden"></div>
</main>
</div>
</div>
@ -782,6 +798,25 @@
navChat: "对话",
navSources: "源",
navAdapters: "适配器",
navPlugins: "插件",
plTitle: "插件",
plState: "状态",
plStages: "阶段",
plActive: "启用中",
plDisabled: "已禁用",
plBroken: "加载失败",
plBuiltin: "内置",
plHooks: "个阶段",
plEmpty: "插件目录为空",
plNoDir: "未配置 plugin_dir,插件功能未启用",
plHookErr: "以下阶段的插件钩子报错(插件故障不会影响转发,但功能会缺失):",
plDir: "插件目录:",
plInstall: "安装插件",
plInstallBtn: "安装 / 覆盖",
plEdit: "编辑",
plEnable: "启用",
plDisable: "禁用",
plRemove: "删除",
navSort: "优先级",
navKeys: "密钥",
keysHint:
@ -904,6 +939,12 @@
mConc: "并发上限",
mRPM: "RPM 限速 (0=不限)",
mTemp: "温度",
mKeyEnv: "Key 环境变量",
mKeyEnvPh: "优先于 API Key,不落盘明文",
mProxy: "代理 URL",
mProxyPh: "如 http://127.0.0.1:7890,留空直连",
mTimeout: "请求超时",
mTimeoutPh: "如 300s,留空用默认 120s",
mModels: "模型列表",
mAddModel: "+ 模型",
mMeta: "Meta",
@ -997,6 +1038,13 @@
recsEnd: "已到最早记录",
recsRotated: "审计日志已轮转,已从最新记录重新加载",
recsNewest: "回到最新",
periodLabel: "统计周期",
periodDay: "今日",
periodWeek: "本周",
periodMonth: "本月",
periodAll: "全部",
periodTruncated:
"统计周期可能不完整(审计日志已轮转),数值为下限。",
recsPartial:
"部分审计日志无法读取,统计可能不完整;完整历史请导出 CSV",
thTokens: "Tokens",
@ -1009,6 +1057,25 @@
navChat: "Chat",
navSources: "Sources",
navAdapters: "Adapters",
navPlugins: "Plugins",
plTitle: "Plugins",
plState: "State",
plStages: "Stages",
plActive: "Active",
plDisabled: "Disabled",
plBroken: "Failed to load",
plBuiltin: "Built-in",
plHooks: "stages",
plEmpty: "The plugin directory is empty",
plNoDir: "plugin_dir is not configured; plugins are disabled",
plHookErr: "Plugin hooks failed on these stages (a broken plugin never blocks forwarding, it just stops providing its feature):",
plDir: "Plugin directory:",
plInstall: "Install a plugin",
plInstallBtn: "Install / replace",
plEdit: "Edit",
plEnable: "Enable",
plDisable: "Disable",
plRemove: "Remove",
navSort: "Priority",
navKeys: "Keys",
keysHint:
@ -1135,6 +1202,12 @@
mConc: "Max concurrency",
mRPM: "RPM limit (0 = unlimited)",
mTemp: "Temperature",
mKeyEnv: "Key env var",
mKeyEnvPh: "Takes precedence over API Key; nothing written to disk",
mProxy: "Proxy URL",
mProxyPh: "e.g. http://127.0.0.1:7890 — empty means direct",
mTimeout: "Request timeout",
mTimeoutPh: "e.g. 300s — empty uses the 120s default",
mModels: "Models",
mAddModel: "+ model",
mMeta: "Meta",
@ -1228,6 +1301,13 @@
recsEnd: "Reached the oldest record",
recsRotated: "The audit log rotated; reloaded from the newest record",
recsNewest: "Back to newest",
periodLabel: "Reporting period",
periodDay: "Today",
periodWeek: "This week",
periodMonth: "This month",
periodAll: "All time",
periodTruncated:
"This window may be incomplete (audit log rotated); figures are a lower bound.",
recsPartial:
"Some audit files could not be read, so these totals may be incomplete; export CSV for the full history",
thTokens: "Tokens",
@ -1291,6 +1371,7 @@
localStorage.setItem("llms-proxy.lang", LANG);
applyI18n();
updateBreadcrumb(lastTab);
notifyPluginLang();
refresh(lastTab);
};
document.getElementById("btn-logout").onclick = () => {
@ -1314,6 +1395,18 @@
const m = $("#btn-menu");
if (m) m.onclick = () => $("#sidebar").classList.toggle("open");
}
// Single source for the tab list. It used to be a literal duplicated in
// goTab, in refresh() and in the admin-only hide pass — three places to
// keep in sync, and adding a tab meant finding all three. A plugin page
// that is routed but never shown is exactly the kind of silent gap that
// survives review.
const TABS = ["status", "chat", "keys", "sort", "sources", "adapters", "plugins"];
// Declared HERE, with var, because goTab() below reads it. It used to be a
// `const` further down the file next to the other plugin state, which put
// goTab's read inside the temporal dead zone: the first click on a plugin
// page would have thrown a ReferenceError. var hoists, so the read sees
// the (still empty) Set and injectPluginUI fills it moments later.
var PLUGIN_PAGES = new Set();
document.querySelectorAll("nav button.sb-i").forEach((b) => {
b.onclick = () => goTab(b.dataset.tab);
});
@ -1325,9 +1418,22 @@
document
.querySelectorAll(".sb-i")
.forEach((x) => x.classList.toggle("active", x.dataset.tab === name));
["status", "chat", "keys", "sort", "sources", "adapters"].forEach(
(tn) => $("#tab-" + tn).classList.toggle("hidden", tn !== name),
);
// TABS plus every plugin-contributed page.
//
// Iterating only TABS left a plugin page permanently unreachable: the
// pane exists and is filled (the data was right there in the DOM) but
// its `hidden` class was never removed, so clicking the sidebar entry
// did nothing visible. The symptom reads as "the page is blank" because
// the content is present in the DOM and only invisible.
//
// PLUGIN_PAGES is consulted here rather than relying on the page's own
// script: the pane's visibility is the HOST's job, and a plugin that
// forgot to unhide itself must still be reachable.
TABS.forEach((tn) => $("#tab-" + tn).classList.toggle("hidden", tn !== name));
PLUGIN_PAGES.forEach((pn) => {
const el = $("#tab-" + pn);
if (el) el.classList.toggle("hidden", pn !== name);
});
updateBreadcrumb(name);
const pane = $("#tab-" + name);
if (pane) {
@ -1466,6 +1572,18 @@
return `${p(d.getMonth() + 1)}-${p(d.getDate())} ${p(d.getHours())}:${p(d.getMinutes())}:${p(d.getSeconds())}`;
};
let statsKeyF = ""; // active key filter for records ('' = all)
// Usage reporting window. "all" keeps the legacy since-start totals;
// day/week/month are calendar windows (UTC) aggregated from the audit
// files server-side. Persisted so a reload keeps the operator's
// chosen scale instead of silently snapping back to lifetime totals.
let statsPeriod = (() => {
try {
const v = localStorage.getItem("gw_stats_period");
return ["day", "week", "month", "all"].includes(v) ? v : "all";
} catch (e) {
return "all";
}
})();
/* ---------- records: on-demand paging ----------
* The records table holds only what is on screen. The first screen comes
@ -1591,6 +1709,17 @@
return;
}
pane.innerHTML = `
<div class="filter-line" style="margin:0 0 12px">
<span class="muted">${t("periodLabel")}</span>
<select id="stat-period" onchange="setStatsPeriod(this.value)">
<option value="day">${t("periodDay")}</option>
<option value="week">${t("periodWeek")}</option>
<option value="month">${t("periodMonth")}</option>
<option value="all">${t("periodAll")}</option>
</select>
<span class="grow"></span>
<span id="period-note"></span>
</div>
<div class="kpis" id="kpi-row"><div class="kpi-skeletons" aria-hidden="true">${Array(
5,
)
@ -1747,6 +1876,24 @@
(statsKeyF ? "&key=" + encodeURIComponent(statsKeyF) : "");
(btn ? btn.closest("#modal-wrap") : null) || closeTopModal();
}
// setStatsPeriod switches the reporting window. It resets the records
// pager rather than keeping the old cursor: the records table is fed
// from the lifetime tail regardless of window, and leaving a stale
// "next page" cursor from the previous window makes the table and the
// KPIs disagree about what is being shown.
function setStatsPeriod(v) {
if (!["day", "week", "month", "all"].includes(v)) return;
statsPeriod = v;
try {
localStorage.setItem("gw_stats_period", v);
} catch (e) {
/* private mode: the selector still works for this session */
}
const sel = $("#stat-period");
if (sel) sel.value = v;
resetRecords();
paintStats();
}
async function paintStats() {
try {
// limit=0 -> the server ships one screen of records; the rest is paged
@ -1754,6 +1901,8 @@
const q =
"/api/stats?limit=" +
RECS_PAGE +
"&period=" +
encodeURIComponent(statsPeriod) +
(statsKeyF ? "&key=" + encodeURIComponent(statsKeyF) : "");
const st = await api(q);
const tot = st.total || {};
@ -1826,6 +1975,15 @@
rpart.innerHTML = st.replay_partial
? `<span class="muted" title="${escAttr(t("recsPartial"))}">ⓘ ${esc(t("recsPartial"))}</span>`
: "";
// Period windows come off the audit files, which rotate. If the
// window may be incomplete the numbers are a lower bound, and saying
// so next to them is the whole point — a confidently wrong zero for a
// rotated-away week is indistinguishable from a quiet week.
const pnote = $("#period-note");
if (pnote)
pnote.innerHTML = st.truncated
? `<span class="muted" title="${escAttr(t("periodTruncated"))}">ⓘ ${esc(t("periodTruncated"))}</span>`
: "";
const rex = $("#rec-exit");
if (rex)
rex.innerHTML = statsKeyF
@ -2866,7 +3024,11 @@
</div>
<div class="row">
<div><label>${t("mRPM")}</label><input id="s-rpm" type="number" min="0" placeholder="0 = 不限" value="${s.rpm || 0}"></div>
<div></div>
<div><label>${t("mProxy")}</label><input id="s-proxy" value="${escAttr(s.proxy_url || "")}" placeholder="${t("mProxyPh")}"></div>
</div>
<div class="row">
<div><label>${t("mKeyEnv")}</label><input id="s-keyenv" value="${escAttr(s.api_key_env || "")}" placeholder="${t("mKeyEnvPh")}"></div>
<div><label>${t("mTimeout")}</label><input id="s-timeout" value="${escAttr(timeoutText(s.timeout))}" placeholder="${t("mTimeoutPh")}"></div>
</div>
<label>${t("mModels")}</label>
<div id="s-models"></div>
@ -2896,6 +3058,21 @@
<button class="ghost del small" onclick="this.closest('.model-row').remove()">&times;</button>
</div>`;
}
// timeoutText renders a stored Go duration for the timeout input.
//
// The API returns timeout as a json:"-"
// field (time.Duration), so the JSON number it arrives as is NANOSECONDS,
// not seconds. Showing the raw number would offer "300000000000" in a box
// that expects "300s"; an unset timeout (0) must show as empty so the
// box reads as "default" rather than "0s".
function timeoutText(v) {
const n = typeof v === "string" ? parseFloat(v) : v;
if (!n || !isFinite(n) || n <= 0) return "";
const secs = Math.round(n / 1e9);
if (secs % 3600 === 0) return secs / 3600 + "h";
if (secs % 60 === 0) return secs / 60 + "m";
return secs + "s";
}
function addModelRow() {
const div = $("#s-models");
div.insertAdjacentHTML(
@ -2930,6 +3107,14 @@
temperature: parseFloat($("#s-temp").value) || 0,
models,
meta,
// These four are always SENT, so the server can tell "the user left
// this empty on purpose" from "this form does not know about it".
// Omitting them made the server keep the stored value, which is right
// for an API client with a partial payload but wrong here: emptying
// the proxy box would have kept the old proxy forever.
proxy_url: $("#s-proxy").value.trim(),
api_key_env: $("#s-keyenv").value.trim(),
timeout: $("#s-timeout").value.trim(),
};
btn.disabled = true;
try {
@ -4064,6 +4249,177 @@
}
/* ---------- adapters tab ---------- */
// ---- plugin management ----
// Install / enable / disable / remove / edit. The list comes from
// on_disk rather than the loaded set so a plugin that FAILED to load
// still appears, with its error — otherwise a syntax error looks
// identical to "the plugin is not there".
async function renderPlugins() {
let j;
try {
j = await api("/api/plugins");
} catch (e) {
$("#tab-plugins").innerHTML =
`<div class="card"><h2>${t("plTitle")}</h2><div class="empty">${esc(t("plNoDir"))}</div></div>`;
return;
}
if (!j.plugin_dir) {
$("#tab-plugins").innerHTML = `<div class="card"><h2>${t("plTitle")}</h2>
<div class="empty">${t("plNoDir")}</div>
<p class="muted">plugin_dir: /etc/llmsproxy/plugins</p></div>`;
return;
}
const errs = j.hook_errors || {};
const rows = (j.on_disk || [])
.map((p) => {
const state = !p.loaded
? `<span class="tag tag-red">${t("plBroken")}</span>`
: p.disabled
? `<span class="tag">${t("plDisabled")}</span>`
: `<span class="tag tag-green">${t("plActive")}</span>`;
const hooks = (p.hooks || 0)
? `<span class="muted">${p.hooks} ${t("plHooks")}</span>`
: `<span class="muted">—</span>`;
const builtin = p.builtin
? ` <span class="tag tag-blue">${t("plBuiltin")}</span>`
: "";
const toggle = p.loaded
? `<button class="small" onclick="togglePlugin('${escAttr(p.name)}',${p.disabled ? "true" : "false"})">${
p.disabled ? t("plEnable") : t("plDisable")
}</button>`
: "";
const err = p.error
? `<div class="muted" style="color:var(--danger)">${esc(String(p.error).slice(0, 200))}</div>`
: "";
return `<tr>
<td><b>${esc(p.name)}</b>${builtin}<div class="muted">${esc(p.description || p.version || "")}</div>${err}</td>
<td>${state}</td>
<td>${hooks}</td>
<td style="white-space:nowrap">
<button class="small" onclick="editPlugin('${escAttr(p.name)}')">${t("plEdit")}</button>
${toggle}
<button class="danger small" onclick="delPlugin('${escAttr(p.name)}')">${t("plRemove")}</button>
</td></tr>`;
})
.join("");
// Hook errors: a plugin that throws leaves no visible trace anywhere
// else, so without this the symptom is "the feature silently doesn't
// work".
const errList = Object.entries(errs)
.map(
([stage, e]) =>
`<li><code>${esc(stage)}</code> × ${e.count} — ${esc(e.last_error || "")}</li>`,
)
.join("");
$("#tab-plugins").innerHTML = `
<div class="card"><h2>${t("plTitle")} (${(j.on_disk || []).length})</h2>
<div class="tbl-wrap"><table><tr>
<th>${t("tName")}</th><th>${t("plState")}</th><th>${t("plStages")}</th><th></th>
</tr>${rows || `<tr><td colspan="4" class="empty">${t("plEmpty")}</td></tr>`}</table></div>
${errList ? `<p class="muted" style="color:var(--danger)">${t("plHookErr")}</p><ul style="font-size:12px">${errList}</ul>` : ""}
<p class="muted">${t("plDir")} <code>${esc(j.plugin_dir)}</code></p>
</div>
<div class="card"><h2>${t("plInstall")}</h2>
<div class="dropzone" id="pl-dz">${t("dropHint")}</div>
<input type="file" id="pl-file" accept=".lua,text/x-lua" class="hidden">
<label>${t("adName")}</label><input id="pl-name" placeholder="myplugin">
<label>${t("tbLua")}</label>
<textarea id="pl-code" spellcheck="false" placeholder="return { name='myplugin', version='1.0.0', hooks={ request_end='on_end' }, function p.on_end(x) return nil end, ui={ page={ page_id='mine', title='Mine', mount='<div>hi</div>' } } }"></textarea>
<p><button onclick="installPlugin()">${t("plInstallBtn")}</button></p>
</div>`;
bindDropzone("pl-dz", "pl-file", "#pl-name", "#pl-code");
}
async function togglePlugin(name, disabled) {
try {
await api("/api/plugins/" + encodeURIComponent(name), {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ enabled: disabled }),
});
toast(disabled ? t("plDisabled") : t("plActive"));
renderPlugins();
} catch (e) {
toast(tFmt("toastSaveFail", e.message));
}
}
async function delPlugin(name) {
if (!confirm(tFmt("confirmDelSrc", name))) return;
try {
await api("/api/plugins/" + encodeURIComponent(name), { method: "DELETE" });
toast(t("toastDelOk"));
renderPlugins();
} catch (e) {
toast(tFmt("toastSaveFail", e.message));
}
}
async function installPlugin() {
const name = $("#pl-name").value.trim();
const code = $("#pl-code").value;
if (!name) return toast(t("toastEmpty"));
try {
await api("/api/plugins", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ name, code }),
});
toast(t("toastSaved"));
const w = $("#pl-dz") && $("#pl-dz").closest("#modal-wrap");
if (w) w.remove();
else closeTopModal();
renderPlugins();
} catch (e) {
toast(tFmt("toastSaveFail", e.message));
}
}
function editPlugin(name) {
openCodeModal(name, async (n, code) => {
try {
await api("/api/plugins", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ name: n, code }),
});
toast(t("toastSaved"));
renderPlugins();
} catch (e) {
toast(tFmt("toastSaveFail", e.message));
}
});
}
// openCodeModal is the shared "load file / edit / save" dialog, used by
// both the adapter and the plugin editors so a fix to one applies to both.
async function openCodeModal(name, onSave) {
let code = "";
try {
const j = await api("/api/plugins/" + encodeURIComponent(name));
code = j.code || "";
} catch (e) {
toast(tFmt("toastSaveFail", e.message));
return;
}
const wrap = document.createElement("div");
wrap.id = "modal-wrap";
wrap.innerHTML = `<div class="card"><h2>${esc(name)}</h2>
<textarea id="pl-edit-code" spellcheck="false" style="min-height:420px;width:100%">${esc(code)}</textarea>
<p><button id="pl-edit-save">${t("mSave")}</button>
<button class="ghost" onclick="this.closest('#modal-wrap').remove()">${t("mCancel")}</button></p>
</div>`;
wrap.style.cssText =
"position:fixed;inset:0;background:rgba(15,22,44,.45);display:flex;align-items:flex-start;justify-content:center;overflow:auto;padding:48px 20px;z-index:50";
document.body.appendChild(wrap);
$("#pl-edit-save").onclick = () => {
onSave(name, $("#pl-edit-code").value);
wrap.remove();
};
}
async function renderAdapters() {
const j = await api("/api/status");
const pools = {};
@ -4092,7 +4448,7 @@
<textarea id="adp-code" spellcheck="false" placeholder="return { name='mysrc', endpoint='/chat/completions', transform_request=function(raw) return raw end, transform_response=function(raw) return raw end }"></textarea>
<p><button onclick="uploadAdapter()">${t("uploadBtn")}</button></p>
</div>`;
bindDropzone();
bindDropzone("#dz", "#adp-file", "#adp-name", "#adp-code");
}
// poolCell renders one adapter's elastic Lua state pool: how many states
@ -4114,11 +4470,25 @@
return `<span class="tag ${busy ? "tag-green" : "tag-amber"}" title="${escAttr(title)}">${p.created} / ${p.max}</span>
<span class="muted"> ${p.in_use}● ${p.idle}○ +${p.grow_step}/-${p.shrink_step}</span>`;
}
function bindDropzone() {
const dz = $("#dz"),
file = $("#adp-file"),
name = $("#adp-name"),
code = $("#adp-code");
// bindDropzone wires a drop target + file input + name/code fields so a
// dropped .lua fills the form. It is PARAMETERISED because there are two
// upload forms (adapters and plugins) and the original hard-coded the
// adapter's element ids — a second copy would have been the same function
// with four different strings in it.
// All four arguments are REQUIRED. An earlier version defaulted them to the
// adapter's ids, which meant a caller that forgot one silently wrote the
// plugin's dropped file into the ADAPTER form — a cross-wired form that
// looks like it worked. There is no safe default here.
function bindDropzone(dzId, fileId, nameSel, codeSel) {
if (!dzId || !fileId || !nameSel || !codeSel) {
toast("bindDropzone: dz/file/name/code ids are all required");
return;
}
const dz = $(dzId);
const file = $(fileId);
const name = $(nameSel);
const code = $(codeSel);
if (!dz || !file || !name || !code) return;
["dragenter", "dragover"].forEach((ev) =>
dz.addEventListener(ev, (e) => {
e.preventDefault();
@ -4770,21 +5140,359 @@
}
/* ---------- boot ---------- */
// refresh renders one tab.
//
// The plugin-element re-mount is applied HERE, once, rather than at the end
// of each renderer: seven pages assign their pane's innerHTML wholesale,
// and a per-page call is one more place to forget.
function refresh(tab) {
if (tab === "status") return renderStatus();
if (tab === "chat") return renderChat();
if (tab === "keys") return renderKeys();
if (tab === "sort") return renderSort();
if (tab === "sources") return renderSources();
return renderAdapters();
const done = () => remountPluginElements();
if (tab === "status") return renderStatus().then(done, done);
if (tab === "chat") return renderChat().then(done, done);
if (tab === "keys") return renderKeys().then(done, done);
if (tab === "sort") return renderSort().then(done, done);
if (tab === "sources") return renderSources().then(done, done);
if (tab === "adapters") return renderAdapters().then(done, done);
if (tab === "plugins") return renderPlugins().then(done, done);
// A page contributed by a plugin has no renderer here: its <script>
// already ran at injection time and owns its own DOM. We only fire the
// "shown" callbacks so it can refresh when the user lands on it.
if (PLUGIN_PAGES.has(tab)) notifyPluginTab(tab);
else return undefined;
done();
return undefined;
}
// ---- plugin injection -------------------------------------------
// Pages and elements contributed by Lua plugins (see docs/plugins.md).
//
// The server merges every plugin's extension into one payload at
// GET /api/ui-inject, because the sidebar needs all of them before it can
// be built. Injection happens once at boot, BEFORE the first goTab, so a
// plugin page is a real tab rather than a special case in the router.
const PLUGIN_TAB_CBS = {};
const PLUGIN_ELEMENTS = [];
// PLUGIN_MOUNT_HOOKS holds re-mount functions for plugin ELEMENTS. A host
// page that rebuilds its own DOM (renderStatus assigns pane.innerHTML
// wholesale) destroys whatever a plugin mounted into it, so refresh() calls
// remountPluginElements() after every rebuild. Declared here, next to the
// other plugin state, because injectPluginUI below pushes into it.
const PLUGIN_MOUNT_HOOKS = [];
// pluginAPI is the small surface a plugin's script may rely on. Kept
// deliberately tiny: plugins are untrusted, and every convenience here is
// one more thing to keep working across kernel changes.
window.pluginAPI = {
async fetchState(name) {
const r = await fetch("/api/plugins/" + encodeURIComponent(name) + "/state", {
credentials: "same-origin",
});
if (!r.ok) throw new Error("state " + r.status);
return (await r.json()).state;
},
async postState(name, obj) {
const r = await fetch("/api/plugins/" + encodeURIComponent(name) + "/state", {
method: "PUT",
credentials: "same-origin",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(obj),
});
if (!r.ok) throw new Error((await r.json().catch(() => ({}))).error?.message || r.status);
return true;
},
onTabShown(fn) {
PLUGIN_TAB_CBS.__last = PLUGIN_TAB_CBS.__last || [];
PLUGIN_TAB_CBS.__last.push(fn);
},
// The current UI language ("zh" | "en"). A plugin cannot read the
// host's LANG binding (it is module-local), and hardcoding one language
// is what left the Billing page English-only while the rest of the UI
// switched. Exposed as a getter so a plugin always sees the live value.
get lang() {
return LANG;
},
// Register a callback fired whenever the user switches language, so a
// plugin page can re-render in the new language without a reload. The
// callback also fires is NOT automatic on first paint — the plugin
// renders itself once on load as it always did.
onLangChange(fn) {
if (typeof fn === "function") PLUGIN_LANG_CBS.push(fn);
},
};
// Callbacks registered via pluginAPI.onLangChange.
const PLUGIN_LANG_CBS = [];
function notifyPluginLang() {
PLUGIN_LANG_CBS.forEach((fn) => {
try {
fn(LANG);
} catch (e) {
console.warn("plugin lang callback failed", e);
}
});
}
// pluginIconHTML renders a plugin-declared sidebar icon.
//
// Text icons are escaped as before. An icon that looks like markup is
// accepted ONLY as a sanitized inline <svg>: a fixed tag allowlist, no
// <script>, no event handlers, no external references. Plugins are
// third-party code running in the operator's browser, so "trust the
// plugin" is not a posture this can take — but neither can it refuse SVG
// outright, because that is the only way an icon matches the native tabs.
function pluginIconHTML(icon) {
var raw = icon == null ? "" : String(icon);
if (!raw) return '<span style="font-size:18px;line-height:1">\u2022</span>';
if (!/<[a-zA-Z!/]/.test(raw)) {
// Plain text (an emoji or a glyph).
return '<span style="font-size:18px;line-height:1">' + esc(raw) + "</span>";
}
var cleaned = sanitizePluginSVG(raw);
if (!cleaned) {
// Markup that is not an acceptable SVG: fall back to a neutral dot
// rather than injecting it or showing raw tags.
return '<span style="font-size:18px:line-height:1">\u2022</span>';
}
return (
'<span style="font-size:18px;line-height:1;display:inline-flex">' +
cleaned +
"</span>"
);
}
// sanitizePluginSVG keeps only what an icon needs.
//
// Allowlist, not a denylist: anything not named here is dropped, so a new
// dangerous construct cannot slip through by default. Attributes are
// limited to geometry and paint (no href/src, no on*, no style with url()).
var SVG_OK_TAGS = { svg: 1, path: 1, circle: 1, rect: 1, line: 1, polyline: 1, polygon: 1, g: 1 };
var SVG_OK_ATTRS = {
viewBox: 1, fill: 1, stroke: 1, "stroke-width": 1, "stroke-linecap": 1,
"stroke-linejoin": 1, d: 1, cx: 1, cy: 1, r: 1, x: 1, y: 1, rx: 1, ry: 1,
x1: 1, y1: 1, x2: 1, y2: 1, points: 1, width: 1, height: 1, opacity: 1,
};
function sanitizePluginSVG(raw) {
var doc = new DOMParser().parseFromString("<svg>" + raw + "</svg>", "image/svg+xml");
var svg = doc.documentElement;
if (!svg || svg.nodeName.toLowerCase() !== "svg" || doc.querySelector("parsererror")) {
return "";
}
// The wrapper we build is the only <svg> we emit. A plugin's own <svg>
// is unwrapped, otherwise the output nests an empty <svg></svg> inside
// ours — visible in the markup, and it also meant the viewBox was read
// from the WRAPPER (which never has one) rather than from the plugin's,
// so any icon declaring a non-default viewBox silently lost it.
var kept = [];
(function walk(node, depth) {
if (depth > 4) return;
for (var i = 0; i < node.children.length; i++) {
var el = node.children[i];
var name = el.nodeName.toLowerCase();
if (name === "svg") {
walk(el, depth + 1); // unwrap, do not emit
continue;
}
if (!SVG_OK_TAGS[name]) continue;
var attrs = "";
for (var a = 0; a < el.attributes.length; a++) {
var at = el.attributes[a];
var an = at.name.toLowerCase();
// Reject anything that can fetch or execute, whatever it is called.
if (/^on/.test(an) || /href|src|xlink|formaction|style/.test(an)) continue;
if (!SVG_OK_ATTRS[an]) continue;
var val = String(at.value).replace(/[<>"'&]/g, "");
attrs += " " + an + '="' + val + '"';
}
kept.push("<" + name + attrs + "></" + name + ">");
walk(el, depth + 1);
}
})(svg, 0);
if (!kept.length) return "";
// Prefer the plugin's own viewBox; fall back to the 24px grid every
// native icon uses.
var innerSvg = svg.querySelector("svg");
var vb = (innerSvg && innerSvg.getAttribute("viewBox")) || svg.getAttribute("viewBox") || "0 0 24 24";
return (
'<svg viewBox="' + vb.replace(/[^\d\s.\-]/g, "") + '" fill="none" ' +
'stroke="currentColor" stroke-width="2" stroke-linecap="round" ' +
'stroke-linejoin="round" style="width:18px;height:18px">' +
kept.join("") +
"</svg>"
);
}
// remountPluginElements re-attaches plugin elements after a host page
// rebuilt its DOM. Safe to call at any time: each mount is a no-op when
// the wrapper is already present in the current build of the pane, so a
// plugin's <script> does not run twice and its counters do not double.
function remountPluginElements() {
if (!PLUGIN_MOUNT_HOOKS.length) return;
PLUGIN_MOUNT_HOOKS.forEach((fn) => {
try {
fn();
} catch (e) {
console.warn("plugin element remount failed", e);
}
});
}
function notifyPluginTab(tab) {
const fns = PLUGIN_TAB_CBS[tab] || PLUGIN_TAB_CBS.__last || [];
fns.forEach((f) => {
try {
f();
} catch (e) {
console.warn("plugin tab callback failed", e);
}
});
}
// injectPluginUI adds the sidebar button + pane for a plugin page and
// mounts plugin elements into existing panes.
async function injectPluginUI() {
let payload;
try {
const r = await fetch("/api/ui-inject", { credentials: "same-origin" });
if (!r.ok) return;
payload = await r.json();
} catch (e) {
return; // plugins are optional; the UI must work without them
}
const ui = (payload && payload.ui) || {};
const main = $("#main");
const nav = $("#sb-nav");
if (!main || !nav) return;
// --- pages ---
//
// A plugin may contribute ONE page (`ui.page`) or SEVERAL
// (`ui.pages[]`). Both are handled by the same code: a plugin whose
// price rules produce the numbers on its billing page needs a second
// screen to edit them, and cramming both into one pane behind
// in-page tabs would hide a whole capability behind a toggle. The
// single-page shape stays supported because it is what docs/plugins.md
// documents and what every existing plugin uses.
const pluginPages = []
.concat(ui.page ? [ui.page] : [])
.concat(Array.isArray(ui.pages) ? ui.pages : [])
.filter((p) => p && p.page_id && p.mount);
pluginPages.forEach((pg) => {
const id = String(pg.page_id);
if (!document.getElementById("tab-" + id)) {
const pane = document.createElement("div");
pane.id = "tab-" + id;
pane.className = "tab-pane hidden";
// A plugin page is arbitrary HTML from a third party. Without
// min-width:0 its content (a wide table, a long unbroken string)
// stretches the pane past #main, which does not scroll sideways —
// the operator sees the page spill over the UI instead of a
// scrollbar. The same guard every native pane gets.
pane.style.minWidth = "0";
pane.style.maxWidth = "100%";
pane.style.overflowX = "auto";
main.appendChild(pane);
const btn = document.createElement("button");
btn.className = "sb-i";
btn.dataset.tab = id;
btn.title = pg.title || id;
// A plugin icon may be plain text (an emoji, a glyph) or an inline
// SVG snippet. Native tabs use inline SVG styled with
// `stroke: currentColor`, so an emoji next to them renders at the
// wrong size and ignores the theme — that is what "the icon looks
// wrong" meant.
//
// The SVG form is allowed through RAW, which is only safe because
// it is strictly filtered: see pluginIconHTML. Escaping it (as this
// did) would print the markup as text instead.
//
// The label is NOT optional. Every native tab is
// `<svg>…</svg><span data-i="navX">标题</span>`; a plugin tab that
// carried only the icon rendered as a nameless icon in the sidebar,
// which is what "the navigation entry has no title" was. The span
// carries no data-i because plugin titles are not in the host's
// translation table — set from pg.title, same as btn.title.
btn.innerHTML =
pluginIconHTML(pg.icon) +
'<span>' +
esc(pg.title || id) +
"</span>";
btn.onclick = () => goTab(id);
nav.appendChild(btn);
PLUGIN_PAGES.add(id);
// The breadcrumb map is local to this file, so extend it here.
if (typeof NAV_NAME === "object") NAV_NAME[id] = pg.title || id;
}
const pane = document.getElementById("tab-" + id);
if (pane && !pane.dataset.pluginMounted) {
pane.dataset.pluginMounted = "1";
// Split the mount so <script>/<style> run only AFTER the markup is
// in the document. Setting innerHTML with a <script> tag does not
// execute it, which is exactly what we want to avoid the opposite
// problem: running before its own DOM exists.
const tpl = document.createElement("template");
tpl.innerHTML = pg.mount;
pane.appendChild(tpl.content);
// Move each script into a fresh element so it executes.
pane.querySelectorAll("script").forEach((old) => {
const s = document.createElement("script");
Array.from(old.attributes).forEach((a) => s.setAttribute(a.name, a.value));
s.textContent = old.textContent;
old.replaceWith(s);
});
}
});
// --- elements into existing pages ---
//
// mountPluginElements is a named, re-entrant function rather than a
// one-shot loop: the host pages REPLACE their own innerHTML on every
// build, so renderStatus' `pane.innerHTML = ...` destroys anything this
// mounted. The symptom is silent and points at the wrong file — the
// plugin did declare an element, the payload did arrive, and the element
// is still gone after the next repaint, which reads as "my plugin
// declared nothing". refresh() calls remountPluginElements() to re-apply.
window.mountPluginElements = function () {
(ui.elements || []).forEach((el, i) => {
const target = document.getElementById("tab-" + el.target);
if (!target || !el.mount) return;
PLUGIN_ELEMENTS.push(el);
// Already present in THIS build of the pane: leave it alone, or the
// widget's <script> would run again on every host re-render and a
// counter would silently double-count.
if (target.querySelector('.plugin-el[data-idx="' + i + '"]')) return;
const wrap = document.createElement("div");
wrap.className = "plugin-el";
wrap.dataset.target = el.target;
wrap.dataset.idx = String(i);
const tpl = document.createElement("template");
tpl.innerHTML = el.mount;
wrap.appendChild(tpl.content);
const anchor = String(el.anchor || "bottom");
if (anchor === "top") target.prepend(wrap);
else if (anchor.startsWith("before:") || anchor.startsWith("after:")) {
const [kind, sel] = anchor.split(/:(.+)/);
const ref = target.querySelector(sel);
if (ref) ref.parentNode.insertBefore(wrap, kind === "before" ? ref : ref.nextSibling);
else target.appendChild(wrap);
} else target.appendChild(wrap);
wrap.querySelectorAll("script").forEach((old) => {
const s = document.createElement("script");
Array.from(old.attributes).forEach((a) => s.setAttribute(a.name, a.value));
s.textContent = old.textContent;
old.replaceWith(s);
});
});
};
PLUGIN_MOUNT_HOOKS.push(window.mountPluginElements);
window.mountPluginElements();
}
(async () => {
try {
const me = await api("/api/keys/me");
window._me = me.key;
maybeWarnSeed(me.key);
if (me.key.role !== "admin") {
["sort", "sources", "adapters"].forEach((tn) => {
["sort", "sources", "adapters", "plugins"].forEach((tn) => {
const b = document.querySelector(
`nav button.sb-i[data-tab="${tn}"]`,
);
@ -4797,7 +5505,15 @@
window.addEventListener("pagehide", () => releaseRecords(false));
window.addEventListener("beforeunload", () => releaseRecords(false));
refresh("status");
// Plugin injection runs BEFORE the first render: a plugin page must exist
// in #main and the sidebar before goTab runs, otherwise the sidebar shows
// no entry and the pane is missing for a moment. Awaited (not fired and
// forgotten) so a slow /api/ui-inject cannot race the first paint.
injectPluginUI()
.catch(() => {})
.finally(() => {
refresh("status");
});
</script>
</body>
</html>

View File

@ -59,6 +59,82 @@ func lineOf(src string, idx int) int {
return strings.Count(src[:idx], "\n") + 1
}
// TestUIEditFormRoundTripsEveryOptionalSourceField pins the JS half of the
// source-edit contract.
//
// The server distinguishes "absent" from "explicitly empty" for proxy_url,
// api_key_env and timeout (they are pointers). That only helps if the form
// SENTS them — a form that omits them falls back to "inherit", so clearing the
// proxy box would silently keep the old proxy, which is the exact bug in the
// other direction.
//
// It must also send the api_key_env value it was given, since that is the one
// field whose loss is invisible until the next upstream call.
func TestUIEditFormRoundTripsEveryOptionalSourceField(t *testing.T) {
src := uiSource(t)
body, ok := jsFunctionBody(src, "saveSource")
if !ok {
t.Fatal("saveSource() not found in the WebUI")
}
for _, field := range []string{"proxy_url", "api_key_env", "timeout"} {
if !strings.Contains(body, field+":") {
t.Errorf("saveSource() does not send %q; the server treats an absent "+
"field as \"keep the stored value\", so the form could never clear it", field)
}
}
// The form's inputs must be filled from the values the server sends, or a
// save would post an empty box and clear a configured field.
for _, read := range []string{`$("#s-proxy")`, `$("#s-keyenv")`, `$("#s-timeout")`} {
if !strings.Contains(src, read+".value") {
t.Errorf("the source form never reads %s — it would post an empty value", read)
}
}
}
// TestUIEditFormReadsDurationsFromTheRevealView: config.Source tags
// Timeout/QueueTimeout `json:"-"`, so the durations only reach the form if the
// reveal endpoint adds them explicitly. The form MUST read them (see above),
// which makes this API response load-bearing rather than informational.
func TestUIEditFormReadsDurationsFromTheRevealView(t *testing.T) {
src := uiSource(t)
for _, id := range []string{"s-proxy", "s-keyenv", "s-timeout"} {
if !strings.Contains(src, `id="`+id+`"`) {
t.Errorf("source form input #%s is missing", id)
}
}
// The timeout box must render through the duration helper. Reading a raw
// time.Duration number would show nanoseconds in a box that expects "300s".
if !strings.Contains(src, "timeoutText(") {
t.Error("the timeout input does not go through timeoutText(); a raw " +
"time.Duration JSON number is nanoseconds and would be uneditable")
}
}
// TestUIEditFormSendsEverySourceField is the mirror of
// TestSourcePayloadCoversEveryEditableField on the JavaScript side: each field
// the server accepts must be submitted by the form, or the server's upsert has
// nothing to store for it.
func TestUIEditFormSendsEverySourceField(t *testing.T) {
src := uiSource(t)
body, ok := jsFunctionBody(src, "saveSource")
if !ok {
t.Fatal("saveSource() not found")
}
// Fields the edit dialog owns. proxy_url / api_key_env / timeout are
// covered by the round-trip test above; this one catches the rest.
// models and meta are local variables submitted by Go shorthand
// (payload = { models, meta }), so they are matched as bare identifiers.
for _, field := range []string{
"name:", "base_url:", "api_key:", "adapter:", "endpoint:",
"image_endpoint:", "max_concurrent:", "rpm:", "temperature:",
"models,", "meta,",
} {
if !strings.Contains(body, field) {
t.Errorf("saveSource() does not send %s", field)
}
}
}
// TestUIAPICallsDeclareMethod asserts that every api() call passing an options
// object also declares an HTTP method (or is a GET that only passes an
// AbortSignal). Without this, fetch defaults to GET and mutating endpoints are

View File

@ -0,0 +1,390 @@
package gateway
import (
"strings"
"testing"
)
// The plugin UI injection is JavaScript inside the embedded index.html, and it
// is the ONLY thing that turns a plugin's `ui` block into a visible page or
// element. These tests pin the wiring on the JS side; the server side (what the
// payload contains) is covered by TestUIInjectServesPluginUI and the lua
// package's TestBillingPluginDeclaresUI.
//
// What makes this worth pinning: a missing hook here fails SILENTLY. The page
// simply never appears, there is no error anywhere, and it looks like "the
// plugin didn't declare a page" rather than "the UI forgot to inject it".
// uiSource returns the embedded WebUI document.
func uiSourceX(t *testing.T) string {
t.Helper()
return uiSource(t)
}
// TestUIFetchesPluginInjection: the boot sequence must ask the kernel what to
// inject. Without this fetch the whole feature is inert.
func TestUIFetchesPluginInjection(t *testing.T) {
src := uiSourceX(t)
if !strings.Contains(src, "/api/ui-inject") {
t.Error("the WebUI never calls /api/ui-inject; plugin pages and elements can never appear")
}
}
// TestUIInjectsBeforeFirstRender: injection must be awaited before the first
// refresh, otherwise the sidebar is built without the plugin entry and the
// first paint races the fetch. This is an ordering contract, so it is asserted
// on the source order rather than trusted.
func TestUIInjectsBeforeFirstRender(t *testing.T) {
src := uiSourceX(t)
iInject := strings.Index(src, "injectPluginUI()")
iRefresh := strings.LastIndex(src, `refresh("status")`)
if iInject < 0 {
t.Fatal("injectPluginUI() is never called")
}
if iRefresh < 0 {
t.Fatal("the boot sequence no longer calls refresh(\"status\")")
}
if iInject > iRefresh {
t.Error("injectPluginUI() is called after the first refresh; the sidebar " +
"and #main would be built before the plugin page exists")
}
// And it must be awaited, not fire-and-forget.
window := src[iInject:]
if !strings.Contains(window[:200], ".finally") && !strings.Contains(window[:200], "await") {
t.Error("injectPluginUI() is not awaited before refresh; a slow response " +
"would race the first paint")
}
}
// TestUIPluginScriptsRunAfterMarkup is the subtle one. Setting innerHTML with a
// <script> tag does NOT execute it; appending via a template neither does. The
// mount therefore has to be inserted first and its scripts re-created
// afterwards, or a plugin's script runs before its own DOM exists — which is
// exactly the "document.getElementById returns null" failure mode.
func TestUIPluginScriptsRunAfterMarkup(t *testing.T) {
src := uiSourceX(t)
// A <template> is used to parse the mount without executing scripts...
if !strings.Contains(src, "createElement(\"template\")") {
t.Error("the mount is not parsed via <template>; scripts could execute before their DOM")
}
// ...and scripts are then re-created as fresh elements so they DO run.
if !strings.Contains(src, "document.createElement(\"script\")") {
t.Error("plugin <script> blocks are never re-created, so they never execute")
}
if !strings.Contains(src, "replaceWith(s)") {
t.Error("the original inert <script> is not replaced by an executable one")
}
}
// TestUIPluginAPISurface: the documented browser API must exist with the exact
// names docs/plugins.md promises, since plugin authors code against it.
func TestUIPluginAPISurface(t *testing.T) {
src := uiSourceX(t)
for _, member := range []string{"fetchState", "postState", "onTabShown"} {
if !strings.Contains(src, member+":") && !strings.Contains(src, member+"(") {
t.Errorf("window.pluginAPI.%s is missing; docs/plugins.md documents it", member)
}
}
}
// TestUIPluginPageBecomesRealTab: a plugin page must get a pane in #main AND a
// sidebar button wired to goTab, otherwise the page is unreachable.
func TestUIPluginPageBecomesRealTab(t *testing.T) {
src := uiSourceX(t)
// pane in #main
if !strings.Contains(src, `pane.id = "tab-" + id`) {
t.Error("no pane is created for a plugin page")
}
if !strings.Contains(src, "main.appendChild(pane)") {
t.Error("the plugin pane is not appended to #main")
}
// sidebar button wired to the tab router
if !strings.Contains(src, "btn.dataset.tab = id") {
t.Error("the sidebar button is not given a data-tab, so goTab() will not route to it")
}
if !strings.Contains(src, "btn.onclick = () => goTab(id)") {
t.Error("the sidebar button is not wired to goTab()")
}
// and the router must know about it
if !strings.Contains(src, "PLUGIN_PAGES.has(tab)") {
t.Error("refresh() does not route plugin pages, so opening one renders nothing")
}
}
// TestUIPluginElementsHonorAnchor: elements declare top / bottom / before:sel /
// after:sel. Silently ignoring the anchor would put a "top" tile at the bottom
// of the status page, which looks like a layout bug rather than a plugin bug.
func TestUIPluginElementsHonorAnchor(t *testing.T) {
src := uiSourceX(t)
for _, anchor := range []string{`anchor === "top"`, `anchor.startsWith("before:")`, `"after:"`} {
if !strings.Contains(src, anchor) {
t.Errorf("the anchor form %s is not handled; elements would all land at the bottom", anchor)
}
}
}
// TestUIPluginInjectionFailureIsNonFatal: plugins are optional, so a failed
// /api/ui-inject must still leave a working UI (the dashboard has to render).
// Two places have to cooperate: the function swallows the fetch error, and the
// caller catches anything that still escapes so refresh() always runs.
func TestUIPluginInjectionFailureIsNonFatal(t *testing.T) {
src := uiSourceX(t)
// inside the function: the fetch is wrapped in try/catch
fnStart := strings.Index(src, "async function injectPluginUI()")
if fnStart < 0 {
t.Fatal("injectPluginUI() is not defined")
}
fn := src[fnStart:]
if !strings.Contains(fn, "plugins are optional; the UI must work without them") {
t.Error("injectPluginUI does not guard its own fetch failure")
}
// at the call site: the rejection cannot escape before the first render
// LastIndex, not Index: the DEFINITION of injectPluginUI also matches, and
// the definition has no .catch on it.
iCall := strings.LastIndex(src, "injectPluginUI()")
if iCall < 0 {
t.Fatal("injectPluginUI() is never called")
}
// Bound the window at len(src): the call site sits near EOF and a fixed
// slice overruns it (a panic in a test is worse than a skipped assertion).
end := iCall + 220
if end > len(src) {
end = len(src)
}
if !strings.Contains(src[iCall:end], ".catch") {
t.Error("a failed /api/ui-inject would reject before refresh(\"status\"), " +
"leaving the dashboard blank")
}
}
// ---- plugin management UI contract ---------------------------------------
//
// The management page is the operator's only way to take a broken plugin out
// of the request path. Every one of these assertions guards a link that, if it
// silently broke, would leave the gateway running with a plugin it cannot
// disable — the worst kind of gap: everything looks fine and nothing is
// reachable.
func TestUIHasPluginTabAndPane(t *testing.T) {
src := uiSourceX(t)
if !strings.Contains(src, `data-tab="plugins"`) {
t.Error("no sidebar entry for the plugin page")
}
if !strings.Contains(src, `id="tab-plugins"`) {
t.Error("no #tab-plugins pane")
}
// The tab list is now a single constant; a new tab must be added there or
// goTab will not un-hide its pane.
if !strings.Contains(src, `const TABS = [`) {
t.Error("TABS is gone; the tab list went back to a duplicated literal")
}
for _, tn := range []string{"status", "chat", "keys", "sort", "sources", "adapters", "plugins"} {
if !strings.Contains(src, `"`+tn+`"`) {
t.Errorf("TABS is missing %q", tn)
}
}
// goTab must iterate TABS, not its own list.
if !strings.Contains(src, "TABS.forEach((tn) =>") {
t.Error("goTab does not iterate TABS")
}
if strings.Contains(src, `["status", "chat", "keys", "sort", "sources", "adapters"].forEach`) {
t.Error("a duplicated tab literal survived; it will drift from TABS")
}
}
func TestUIRendersPluginManagement(t *testing.T) {
src := uiSourceX(t)
body, ok := jsFunctionBody(src, "renderPlugins")
if !ok {
t.Fatal("renderPlugins() not found")
}
// It must read the DISK listing, not just the loaded set: a plugin that
// failed to compile is absent from the loaded set, and showing only the
// loaded set makes a syntax error look like "the plugin is not installed".
if !strings.Contains(body, "on_disk") {
t.Error("renderPlugins reads only the loaded set; a failed plugin would " +
"be invisible instead of shown with its error")
}
if !strings.Contains(body, "/api/plugins") {
t.Error("renderPlugins does not call /api/plugins")
}
// Hook errors must be surfaced: a plugin that throws in every stage leaves
// no other trace, so without this the symptom is "the feature just doesn't
// work".
if !strings.Contains(body, "hook_errors") {
t.Error("renderPlugins ignores hook_errors; a silently broken plugin is undebuggable")
}
// Enable / disable / remove / edit.
for _, fn := range []string{"togglePlugin", "delPlugin", "installPlugin", "editPlugin"} {
if _, ok := jsFunctionBody(src, fn); !ok {
t.Errorf("%s() is missing from the WebUI", fn)
}
}
// The toggle must go through the enable/disable endpoint, not delete.
tb, ok := jsFunctionBody(src, "togglePlugin")
if !ok {
t.Fatal("togglePlugin() missing")
}
if !strings.Contains(tb, `method: "PUT"`) {
t.Error("togglePlugin does not use PUT")
}
if !strings.Contains(tb, "enabled:") {
t.Error("togglePlugin does not send an \"enabled\" field")
}
// And the admin-only tab list must include plugins, or a non-admin would
// see a page whose every action 403s.
if !strings.Contains(src, `["sort", "sources", "adapters", "plugins"]`) {
t.Error("the admin-only tab list omits \"plugins\"; a user key would see a " +
"page full of actions that all fail with 403")
}
}
// TestUIBindDropzoneIsParameterised guards the refactor: the adapter and plugin
// upload forms share one dropzone, so a hard-coded id would send a dropped
// plugin file into the adapter name field.
func TestUIBindDropzoneIsParameterised(t *testing.T) {
src := uiSourceX(t)
body, ok := jsFunctionBody(src, "bindDropzone")
if !ok {
t.Fatal("bindDropzone() not found")
}
if strings.Contains(body, `$("#dz")`) || strings.Contains(body, `$("#adp-name")`) {
t.Error("bindDropzone still hard-codes the adapter's element ids; the " +
"plugin form would write into the adapter form")
}
if !strings.Contains(body, "dzId") || !strings.Contains(body, "nameSel") {
t.Error("bindDropzone does not accept the ids to bind")
}
// Both forms must call it.
if !strings.Contains(src, `bindDropzone("pl-dz", "pl-file", "#pl-name", "#pl-code")`) {
t.Error("the plugin upload form does not use the parameterised dropzone")
}
}
// TestPluginElementsSurviveHostRebuild guards the defect that made plugin
// elements look absent no matter how the injection was configured.
//
// renderStatus (and six other pages) assign pane.innerHTML wholesale. Anything a
// plugin had mounted into that pane is destroyed by the assignment. The symptom
// is silent and misleading: the plugin really did declare an element, the
// payload really did arrive, and the element is still gone on the next repaint —
// so the natural conclusion is "my plugin declared nothing", which sends you
// looking in the wrong file.
//
// The fix is to re-mount after the rebuild. This test asserts the re-mount is
// wired at the SINGLE place every renderer passes through, rather than leaving
// it to be re-added per page.
func TestPluginElementsSurviveHostRebuild(t *testing.T) {
html := uiSource(t)
if !strings.Contains(html, "function remountPluginElements") {
t.Fatal("remountPluginElements is not defined; nothing can re-attach a plugin element after a host rebuild")
}
if !strings.Contains(html, "const PLUGIN_MOUNT_HOOKS = []") {
t.Fatal("PLUGIN_MOUNT_HOOKS is not declared")
}
// The hook array must be declared BEFORE injectPluginUI pushes to it, and
// before refresh() calls into it. A use-before-declaration in a const
// block is a hard TDZ ReferenceError at first paint.
hookDecl := strings.Index(html, "const PLUGIN_MOUNT_HOOKS = []")
push := strings.Index(html, "PLUGIN_MOUNT_HOOKS.push")
use := strings.Index(html, "PLUGIN_MOUNT_HOOKS.forEach")
if hookDecl < 0 || push < 0 || use < 0 {
t.Fatal("the hook array is declared but never both filled and drained")
}
if hookDecl > push || hookDecl > use {
t.Error("PLUGIN_MOUNT_HOOKS is used before its declaration (const TDZ: first paint would throw)")
}
// refresh() is the chokepoint every renderer passes through.
rf := strings.Index(html, "function refresh(tab)")
if rf < 0 {
t.Fatal("refresh(tab) is gone")
}
body := html[rf:]
if i := strings.Index(body, "\n }"); i > 0 {
body = body[:i]
}
if !strings.Contains(body, "remountPluginElements") {
t.Error("refresh() does not re-mount plugin elements: a page that rebuilds its DOM wipes them")
}
// A mount must be idempotent, or the widget's <script> runs again on every
// host repaint and its counters silently double.
if !strings.Contains(html, `data-idx="`) || !strings.Contains(html, "plugin-el[data-idx=") {
t.Error("element mounting is not guarded by a per-pane marker; re-mounting would re-run plugin scripts")
}
}
// TestPluginPagesAreReachableByGoTab guards a bug that only shows up in a
// browser: goTab() iterated the hardcoded TABS list to toggle `hidden`, and a
// plugin-contributed page is not in that list. The pane existed, the plugin had
// filled it with real data, and clicking the sidebar entry changed nothing —
// the `hidden` class was never removed.
//
// It reads exactly like "the page is blank" while the content sits in the DOM,
// and no static check catches it: the injection is correct, the data is correct,
// and the API returns 200.
//
// So this asserts the two structural facts that make it reachable: goTab
// consults the plugin page set, and that set is declared before goTab runs (a
// `const` further down the file would be a temporal-dead-zone ReferenceError on
// the first click).
func TestPluginPagesAreReachableByGoTab(t *testing.T) {
ui := uiSource(t)
goTab := strings.Index(ui, "function goTab(")
if goTab < 0 {
t.Fatal("goTab() is gone")
}
end := strings.Index(ui[goTab:], "\n }")
if end < 0 {
t.Fatal("could not isolate goTab()")
}
body := ui[goTab : goTab+end]
if !strings.Contains(body, "PLUGIN_PAGES.forEach") {
t.Error("★ goTab() does not toggle plugin pages — a plugin page keeps its " +
"`hidden` class forever, so the sidebar entry does nothing")
}
// Declaration must precede the use, or the first click throws.
decl := strings.Index(ui, "var PLUGIN_PAGES = new Set()")
if decl < 0 {
// A const later in the file would also "work" only if nothing reads it
// first — make that explicit rather than silent.
if strings.Contains(ui, "const PLUGIN_PAGES = new Set()") {
t.Error("PLUGIN_PAGES is a `const` declared after goTab() reads it — " +
"temporal dead zone: the first click on a plugin page throws")
} else {
t.Error("PLUGIN_PAGES is not declared anywhere")
}
return
}
if decl > goTab {
t.Errorf("PLUGIN_PAGES is declared at %d but goTab() at %d reads it — "+
"declaration must come first", decl, goTab)
}
}
// TestPluginSidebarEntryCarriesATitle: a plugin tab must render icon AND label.
//
// Native tabs are `<svg>…</svg><span data-i="navStatus">状态</span>`; the plugin
// branch set innerHTML to the icon alone, so every plugin page showed up as a
// nameless icon in the sidebar. An operator cannot tell what an icon means
// until they click it.
func TestPluginSidebarEntryCarriesATitle(t *testing.T) {
html := uiSource(t)
anchor := "btn.innerHTML =\n pluginIconHTML(pg.icon) +"
i := strings.Index(html, anchor)
if i < 0 {
t.Fatal("plugin sidebar button construction not found")
}
// Look at the statement the anchor opens, not the whole file.
stmt := html[i : i+400]
if strings.Contains(stmt, "esc(pg.title") {
return
}
t.Errorf("plugin sidebar button renders the icon only; a label is required. "+
"got: %.200s", stmt)
}

View File

@ -2,7 +2,22 @@ local adapter = {}
adapter.name = "gemini"
adapter.version = "2.0.0"
adapter.endpoint = "/v1/models"
-- The Go layer builds the request URL as base_url + endpoint, statically
-- (see provider.URL). Gemini's real API is POST
-- /v1beta/models/{model}:generateContent, and streaming is the same path with
-- a ":streamGenerateContent" verb -- the model name is part of the PATH, so it
-- cannot live in a static endpoint string.
--
-- "{model}" is therefore a placeholder the Go layer substitutes with the model
-- this request actually sends (provider.urlFor substitutes it; see
-- provider.go). Streaming additionally rewrites the ":generateContent" verb to
-- ":streamGenerateContent" on the same template.
--
-- Leaving this as a bare "/v1/models" would call Gemini's model-LIST endpoint,
-- which answers 405 to POST -- so the preset template would create a source that
-- can never work. A source that overrides `endpoint:` bypasses the template
-- entirely and must then spell the whole path itself.
adapter.endpoint = "/v1beta/models/{model}:generateContent"
adapter.headers = {}
-- Gemini API: POST /v1/models/{model}:generateContent

View File

@ -0,0 +1,351 @@
package lua
import (
"encoding/json"
"os/exec"
"strings"
"testing"
)
// runBillingPage executes the billing page's mount script under node against a
// DOM stub, drives the period selector, and returns what ended up on screen.
//
// It asserts against RENDERED output rather than the plugin's internal
// helpers. Reaching into the IIFE for daysInWindow/rescale needs a test-only
// hook in the plugin, and the first version of this file took a long detour
// through such a hook: the harness's fake `Date` broke toISOString(), the
// plugin script aborted before assigning the hook, and every assertion
// compared "" against a real value. The rendered tables are the contract that
// actually matters, they need no hook, and they cannot pass if the rescale is
// wrong — which is exactly the bug class under test.
func runBillingPage(t *testing.T, todayISO, period, captureID string) string {
t.Helper()
node, err := exec.LookPath("node")
if err != nil {
t.Skipf("node not available (%v)", err)
}
page, _ := billingUI(t)
js := strings.Join(extractScripts(page), "\n")
if !strings.Contains(js, "__billing_setPeriod") {
t.Fatal("billing page exposes no period selector")
}
harness := `
global.window = global;
global.__written = [];
global.__errors = [];
var __realErr = console.error;
console.error = function () {
global.__errors.push("console.error: " + Array.prototype.map.call(arguments, function (a) {
return (a && a.message) ? a.message : String(a);
}).join(" "));
__realErr.apply(console, arguments);
};
process.on("uncaughtException", function (e) { global.__errors.push("uncaught: " + String(e && e.message || e)); });
process.on("unhandledRejection", function (e) { global.__errors.push("unhandled: " + String(e && e.message || e)); });
// Pin "now" for the window under test.
//
// Two earlier attempts were wrong here, and both looked like product bugs:
// replacing the Date constructor wholesale breaks toISOString/UTC; and
// subclassing (setting __FakeDate.prototype = __RealDate.prototype while
// returning __RealDate instances) leaves instances WITHOUT a working
// getUTCDay, so the week window came out NaN and every rendered day became
// "Invalid time value". A Proxy over the real constructor intercepts only the
// zero-argument call and leaves instances — and therefore every prototype
// method — completely untouched.
var __NOW_MS = new Date("` + todayISO + `T12:34:56Z").getTime();
var __RealDate = global.Date;
var __PatchedDate = new Proxy(__RealDate, {
construct: function (target, args) {
// Reflect.construct, not a hand-spelled argument list: writing
// new target(a, b, c, d, e, f, g) passes seven explicit undefineds for a
// one-argument call, which Date reads as year..millisecond fields and turns
// into an Invalid Date. The symptom was a week window rendering
// "Invalid time value" for every day — a harness fault indistinguishable
// from a product bug.
if (args.length === 0) return Reflect.construct(target, [__NOW_MS]);
return Reflect.construct(target, args);
},
apply: function () { return new __RealDate(__NOW_MS); }
});
__PatchedDate.now = function () { return __NOW_MS; };
__PatchedDate.UTC = __RealDate.UTC;
__PatchedDate.parse = __RealDate.parse;
__PatchedDate.prototype = __RealDate.prototype;
global.Date = __PatchedDate;
global.document = {
getElementById: function (id) {
if (!global.__els) global.__els = {};
if (!global.__els[id]) global.__els[id] = {
style: {}, dataset: {}, classList: { add: function(){}, remove: function(){} },
options: [{value:"day"},{value:"week"},{value:"month"},{value:"all"}],
set innerHTML(v) { if (v && String(v).trim()) global.__written.push(id); this.__h = v; },
get innerHTML() { return this.__h || ""; },
set textContent(v) { if (v !== undefined && String(v).trim()) global.__written.push(id); this.__t = v; },
get textContent() { return this.__t || ""; },
set innerText(v) { if (v !== undefined && String(v).trim()) global.__written.push(id); this.__i = v; },
get innerText() { return this.__i || ""; },
appendChild: function(){}, querySelector: function(){ return null; },
querySelectorAll: function(){ return []; }, addEventListener: function(){}
};
return global.__els[id];
},
createElement: function () { return { style: {}, dataset: {}, appendChild: function(){}, setAttribute: function(){} }; },
addEventListener: function () {},
};
// Fixture days carry deliberately distinctive costs so a fold over the wrong
// subset still produces a number, and only a mismatch shows it. 2026-01-05/06/07
// are Monday/Tuesday/Wednesday; 2025-12-01 is far outside any window here.
global.__st = {
currency: "USD",
total: { cost: 999, requests: 999, prompt_tokens: 999999 },
by_source: { LIFETIMESRC: { cost: 999, requests: 999 } },
by_model: { LIFETIMEMODEL: { cost: 999, requests: 999 } },
by_key: { LIFETIMEKEY: { cost: 999, requests: 999 } },
by_day: {
"2026-01-05": { cost: 10, requests: 1, prompt_tokens: 100, completion_tokens: 10, failures: 0 },
"2026-01-06": { cost: 20, requests: 2, prompt_tokens: 200, completion_tokens: 20, failures: 1 },
"2026-01-07": { cost: 30, requests: 3, prompt_tokens: 300, completion_tokens: 30, failures: 0 },
"2025-12-01": { cost: 777, requests: 77, prompt_tokens: 7700 }
},
by_day_src: {
"2026-01-05": { srcA: { cost: 10, requests: 1 } },
"2026-01-06": { srcA: { cost: 20, requests: 2 }, srcB: { cost: 5, requests: 1 } },
"2026-01-07": { srcA: { cost: 30, requests: 3 } },
"2025-12-01": { srcOLD: { cost: 777, requests: 77 } }
},
by_day_model: {
"2026-01-05": { mA: { cost: 10, requests: 1 } },
"2026-01-06": { mA: { cost: 20, requests: 2 } },
"2026-01-07": { mA: { cost: 30, requests: 3 } }
},
by_day_key: {
"2026-01-05": { kA: { cost: 10, requests: 1 } },
"2026-01-06": { kA: { cost: 20, requests: 2 } },
"2026-01-07": { kA: { cost: 30, requests: 3 } }
},
degraded_reqs: 4242, unpriced_reqs: 77
};
global.fetch = function () {
return Promise.resolve({ ok: true, json: function () {
return Promise.resolve({ plugin: "billing", state: global.__st });
}});
};
global.pluginAPI = { onTabShown: function () {}, lang: "en" };
`
script := harness + "\n" + js + "\n" +
"__billing_setPeriod(" + jsonStr(period) + ");\n" +
// Let the period-triggered refresh() settle before reading the DOM.
"setTimeout(function () {\n" +
" var el = document.getElementById(" + jsonStr(captureID) + ");\n" +
" console.log('__CAPTURE__' + JSON.stringify({\n" +
" html: el ? String(el.innerHTML) : '(missing)',\n" +
" kpis: String(document.getElementById('billing-kpis').innerHTML),\n" +
" sel: String(document.getElementById('billing-period').value),\n" +
" errors: global.__errors\n" +
" }));\n" +
"}, 60);\n"
cmd := exec.Command(node, "-e", script)
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("node execution failed: %v\n%s", err, out)
}
o := string(out)
idx := strings.Index(o, "__CAPTURE__")
if idx < 0 {
t.Fatalf("no capture in node output (the plugin script aborted?):\n%s", o)
}
raw := strings.TrimSpace(o[idx+len("__CAPTURE__"):])
if nl := strings.IndexAny(raw, "\r\n"); nl >= 0 {
raw = raw[:nl]
}
var got struct {
HTML string `json:"html"`
KPIs string `json:"kpis"`
Sel string `json:"sel"`
Errors []string `json:"errors"`
}
if err := json.Unmarshal([]byte(raw), &got); err != nil {
t.Fatalf("capture is not JSON: %v\n%s", err, raw)
}
if len(got.Errors) > 0 {
t.Fatalf("billing script reported errors: %v", got.Errors)
}
if got.Sel != period {
t.Errorf("selector shows %q after setting %q — the change handler did not take", got.Sel, period)
}
return got.HTML + "\n@@KPI@@\n" + got.KPIs
}
func jsonStr(s string) string {
b, _ := json.Marshal(s)
return string(b)
}
// TestBillingPeriodDayShowsOnlyThatDay checks the one-day window end to end:
// the tables must carry that day's rows and drop every other day, and the
// lifetime sentinel (999 / LIFETIMESRC) must appear nowhere.
func TestBillingPeriodDayShowsOnlyThatDay(t *testing.T) {
got := runBillingPage(t, "2026-01-07", "day", "billing-by-source")
if strings.Contains(got, "LIFETIME") || strings.Contains(got, "999") {
t.Errorf("a one-day view still shows lifetime figures:\n%s", got)
}
if strings.Contains(got, "srcB") {
t.Errorf("a one-day view shows srcB, which only had traffic on 01-06:\n%s", got)
}
if strings.Contains(got, "srcOLD") {
t.Errorf("a one-day view shows a day from 2025-12:\n%s", got)
}
if !strings.Contains(got, "srcA") {
t.Errorf("a one-day view lost the only source that had traffic that day:\n%s", got)
}
// The KPI total must be that day's cost (30), not the week (60) and not
// the lifetime sentinel (999).
if !strings.Contains(got, "30") {
t.Errorf("KPI does not show the day's cost 30:\n%s", got)
}
}
// TestBillingPeriodWeekSumsThreeDays is the arithmetic: Mon+Wed rows and a
// total of 60. A window that started on Sunday or ended early still renders
// tables, so only the number distinguishes it.
func TestBillingPeriodWeekSumsThreeDays(t *testing.T) {
// 2026-01-07 is a Wednesday: the ISO week holds 01-05, 01-06, 01-07.
got := runBillingPage(t, "2026-01-07", "week", "billing-by-source")
if strings.Contains(got, "srcOLD") || strings.Contains(got, "LIFETIME") {
t.Errorf("weekly view leaks out-of-window or lifetime rows:\n%s", got)
}
// srcA = 10+20+30 = 60, srcB = 5 (only 01-06, but inside the week).
if !strings.Contains(got, "60") {
t.Errorf("weekly total should be 60, got:\n%s", got)
}
if !strings.Contains(got, "srcB") {
t.Errorf("weekly view dropped srcB, whose only day (01-06) is in the window:\n%s", got)
}
}
// TestBillingPeriodSundayReachesBackToMonday pins the ISO-week Monday rule
// from the other side. On Sunday 2026-01-11 the week that already ended is
// Mon 05..Sun 11, so all three fixture days are in it; a start-on-Sunday bug
// would still include 01-07 and pass, so this case is paired with the
// weekday case above rather than relied on alone.
func TestBillingPeriodSundayReachesBackToMonday(t *testing.T) {
got := runBillingPage(t, "2026-01-11", "week", "billing-by-source")
if !strings.Contains(got, "60") {
t.Errorf("Sunday's week view should still total 60:\n%s", got)
}
// And the Monday view must NOT include Sunday's (nonexistent) traffic.
gotMon := runBillingPage(t, "2026-01-05", "day", "billing-by-source")
if strings.Contains(gotMon, "30") {
t.Errorf("Monday's single day view shows Wednesday's cost 30:\n%s", gotMon)
}
if !strings.Contains(gotMon, "10") {
t.Errorf("Monday's day view lost Monday's own cost 10:\n%s", gotMon)
}
}
// TestBillingPeriodKpiTotalMatchesWindowDays pins the KPI total itself.
//
// The dimension-table tests above cannot catch a broken window or a broken
// fold: by_source is computed from the per-day DIMENSION map, independently of
// the total. Mutation verification proved it — disabling the total fold, and
// shifting the ISO-week start to Sunday, both left every earlier test green.
// The total is derived from by_day, so it is the one value that moves when
// either the window membership or the accumulation is wrong.
func TestBillingPeriodKpiTotalMatchesWindowDays(t *testing.T) {
kpis := func(t *testing.T, today, period string) string {
t.Helper()
return runBillingPage(t, today, period, "billing-by-source")
}
// 2026-01-07 is a Wednesday; fixture days are 05(10) 06(20) 07(30).
// KPI renders through fmtInt, so 30 -> "30", 60 -> "60".
if got := kpis(t, "2026-01-07", "day"); !strings.Contains(got, "30") {
t.Errorf("day KPI should show that day's cost 30:\n%s", got)
}
if got := kpis(t, "2026-01-07", "week"); !strings.Contains(got, "60") {
t.Errorf("week KPI should total 10+20+30 = 60:\n%s", got)
}
// Sunday 2026-01-11: the ISO week Mon 05..Sun 11 still holds all three days.
if got := kpis(t, "2026-01-11", "week"); !strings.Contains(got, "60") {
t.Errorf("Sunday's week KPI should still total 60:\n%s", got)
}
// The Monday rule from the negative side. A window that started on SUNDAY
// would cover Sun 11..Sat 17 and contain none of the fixture days, giving
// a total of 0 — so this case is what makes the +6 shift observable. The
// positive case alone cannot distinguish it: shifting the start one day
// later still happens to contain 01-07.
if got := kpis(t, "2026-01-11", "day"); !strings.Contains(got, "0") {
t.Errorf("Sunday's single-day window has no fixture traffic and must total 0:\n%s", got)
}
// Monday 2026-01-12 begins a new week; it contains no fixture days.
if got := kpis(t, "2026-01-12", "week"); !strings.Contains(got, "0") {
t.Errorf("an empty week should report 0, not stale data:\n%s", got)
}
// ...whereas the Monday-to-Sunday week that just ended does.
if got := kpis(t, "2026-01-12", "day"); !strings.Contains(got, "0") {
t.Errorf("Monday the 12th has no fixture traffic and must total 0:\n%s", got)
}
}
// TestBillingPeriodDimensionFoldSumsAcrossDays covers the per-dimension
// accumulation. Asserting "60 appears somewhere" was NOT enough: mutation
// verification showed a fold that keeps only the last day still passed,
// because 60 also appears in the KPI total and in a token column. The
// assertion must therefore pin srcA's own cost cell, not the page's text.
func TestBillingPeriodDimensionFoldSumsAcrossDays(t *testing.T) {
got := runBillingPage(t, "2026-01-07", "week", "billing-by-source")
// srcA exists on all three days at 10/20/30. Summed = 60; last-only = 30.
if !strings.Contains(got, "USD 60.0000") {
t.Errorf("weekly by-source did not sum srcA across days (want USD 60.0000, "+
"last-day-only would be USD 30.0000):\n%s", got)
}
// srcB exists only on 01-06 at 5, inside the window, so it must survive at
// exactly its own value.
if !strings.Contains(got, "USD 5.0000") {
t.Errorf("weekly by-source lost or altered srcB (want USD 5.0000):\n%s", got)
}
// srcA's summed cost must also differ from its last-day value in the
// rendered table, so an overwrite cannot masquerade as a sum.
if strings.Count(got, "USD 60.0000") < 2 {
t.Errorf("expected the summed 60 in both the table and the KPI:\n%s", got)
}
}
// state untouched, including the lifetime sentinel and the lifetime-only
// degraded counter. Without this control, a rescale that silently degraded
// into "always empty" would pass the narrow-window tests.
// TestBillingPeriodAllStaysLifetime is the control: "all" must keep the stored
// state untouched. Without it, a rescale that silently degraded into "always
// empty" would pass every narrow-window test above.
func TestBillingPeriodAllStaysLifetime(t *testing.T) {
got := runBillingPage(t, "2026-01-07", "all", "billing-by-source")
if !strings.Contains(got, "LIFETIMESRC") && !strings.Contains(got, "999") {
t.Errorf("the all-time view dropped the lifetime figures:\n%s", got)
}
// 4242 is the lifetime degraded_reqs; it has no day-keyed tally, so it
// must remain visible when the window is "all". The KPI goes through
// fmtInt, so it renders grouped ("4,242") — assert the rendered form.
if !strings.Contains(got, "4,242") {
t.Errorf("the all-time view should still show the lifetime degraded count:\n%s", got)
}
}
// TestBillingPeriodHidesLifetimeOnlyCounters is the mismatch the period view
// exists to remove: degraded/unpriced have no day-keyed tally, so printing
// the lifetime number next to a per-window cost is misleading. They must be
// hidden, not merely relabelled.
func TestBillingPeriodHidesLifetimeOnlyCounters(t *testing.T) {
got := runBillingPage(t, "2026-01-07", "day", "billing-by-source")
if strings.Contains(got, "4,242") || strings.Contains(got, "Degraded") {
t.Errorf("a one-day view still shows the lifetime degraded counter:\n%s", got)
}
// ...while "all" does show it (covered above), so this is not just a
// dictionary that lost the words.
if !strings.Contains(runBillingPage(t, "2026-01-07", "all", "billing-by-source"), "4,242") {
t.Error("the all-time view lost the degraded counter — the hide is not period-scoped")
}
}

View File

@ -0,0 +1,968 @@
package lua
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
)
// The billing plugin ships with the gateway, so its arithmetic is a contract:
// a wrong price silently produces wrong money. These tests drive it through the
// real hook path and check the NUMBERS, not merely that it loads.
func billingVM(t *testing.T) (*Plugins, string) {
t.Helper()
dir := filepath.Join(t.TempDir(), "adapters")
vm := NewVM(dir)
if err := vm.Start(); err != nil {
t.Fatalf("vm: %v", err)
}
t.Cleanup(vm.Stop)
pdir := filepath.Join(t.TempDir(), "plugins")
ps := NewPlugins(vm, pdir)
if err := ps.SeedBundled(); err != nil {
t.Fatalf("seed: %v", err)
}
if err := ps.LoadDir(); err != nil {
t.Fatalf("load: %v", err)
}
return ps, pdir
}
// stateOf reads the plugin's published state as a generic map.
func stateOf(t *testing.T, ps *Plugins) map[string]interface{} {
t.Helper()
raw := ps.State("billing")
if raw == nil {
t.Fatal("billing published no state")
}
b, err := json.Marshal(raw)
if err != nil {
t.Fatal(err)
}
var out map[string]interface{}
if err := json.Unmarshal(b, &out); err != nil {
t.Fatal(err)
}
return out
}
func approx(t *testing.T, name string, got, want float64) {
t.Helper()
d := got - want
if d < 0 {
d = -d
}
if d > 1e-9 {
t.Errorf("%s = %v, want %v (delta %v)", name, got, want, d)
}
}
// TestBillingZeroPricesIsSafe: with no configuration the plugin must still run
// and report volume. A nil-price crash here would take out every request.
func TestBillingZeroPricesIsSafe(t *testing.T) {
ps, _ := billingVM(t)
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "m", "source": "s", "key": "***aaaaaa", "ok": true,
"prompt_tokens": 100, "completion_tokens": 50, "time": 1750000000000,
})
st := stateOf(t, ps)
total := st["total"].(map[string]interface{})
if total["requests"].(float64) != 1 {
t.Errorf("requests = %v, want 1", total["requests"])
}
approx(t, "cost with no prices", total["cost"].(float64), 0)
}
// TestBillingModelTokenPricing: the core case. prompt and completion are priced
// SEPARATELY, which is how providers publish and how the total must come out.
func TestBillingModelTokenPricing(t *testing.T) {
ps, _ := billingVM(t)
// Setting prices must NOT disturb the (still empty) totals, which is the
// whole point of the prices/state split.
if err := ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"currency": "USD",
"models": map[string]interface{}{
"gpt-5.4": map[string]interface{}{"prompt": 1.25e-6, "completion": 1e-5},
},
},
}); err != nil {
t.Fatal(err)
}
// 1000 prompt * 1.25e-6 = 0.00125 ; 500 completion * 1e-5 = 0.005
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "gpt-5.4", "source": "up", "key": "***aaaaaa", "ok": true,
"prompt_tokens": 1000, "completion_tokens": 500, "time": 1750000000000,
})
st := stateOf(t, ps)
approx(t, "total cost", st["total"].(map[string]interface{})["cost"].(float64), 0.00625)
byModel := st["by_model"].(map[string]interface{})["gpt-5.4"].(map[string]interface{})
approx(t, "model cost", byModel["cost"].(float64), 0.00625)
if byModel["completion_tokens"].(float64) != 500 {
t.Errorf("completion_tokens = %v, want 500", byModel["completion_tokens"])
}
}
// TestBillingPerRequestAndTokenCombine: a flat fee is ADDED to the token cost,
// which is how an image model can be "tokens + fixed fee".
func TestBillingPerRequestAndTokenCombine(t *testing.T) {
ps, _ := billingVM(t)
if err := ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{
"kolors": map[string]interface{}{"prompt": 1e-6, "completion": 2e-6, "per_request": 0.04},
},
},
}); err != nil {
t.Fatal(err)
}
// 100*1e-6 + 50*2e-6 + 0.04 = 0.0402
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "kolors", "source": "sf", "key": "***bbbbbb", "ok": true,
"prompt_tokens": 100, "completion_tokens": 50, "time": 1750000000000,
})
st := stateOf(t, ps)
approx(t, "total", st["total"].(map[string]interface{})["cost"].(float64), 0.0402)
}
// TestBillingPrecedence: keys > models > default for token prices.
func TestBillingPrecedence(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"default": map[string]interface{}{"prompt": 9e-6, "completion": 9e-6},
"models": map[string]interface{}{"m": map[string]interface{}{"prompt": 2e-6, "completion": 3e-6}},
"keys": map[string]interface{}{"***cccccc": map[string]interface{}{"prompt": 1e-6, "completion": 1.5e-6}},
},
})
// No key match -> model price.
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "m", "source": "s", "key": "***other", "ok": true,
"prompt_tokens": 1000, "completion_tokens": 1000, "time": 1750000000000,
})
// Key match -> key price wins.
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "m", "source": "s", "key": "***cccccc", "ok": true,
"prompt_tokens": 1000, "completion_tokens": 1000, "time": 1750000000000,
})
st := stateOf(t, ps)
// 1000*2e-6 + 1000*3e-6 = 0.005 ; 1000*1e-6 + 1000*1.5e-6 = 0.0025
approx(t, "total (model + key)", st["total"].(map[string]interface{})["cost"].(float64), 0.0075)
// An unpriced model falls back to default.
ps2, _ := billingVM(t)
_ = ps2.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"default": map[string]interface{}{"prompt": 9e-6, "completion": 9e-6},
},
})
ps2.Fire(StageRequestEnd, map[string]interface{}{
"model": "unknown", "source": "s", "key": "***d", "ok": true,
"prompt_tokens": 1000, "completion_tokens": 1000, "time": 1750000000000,
})
approx(t, "default fallback", stateOf(t, ps2)["total"].(map[string]interface{})["cost"].(float64), 0.018)
}
// TestBillingAggregatesEveryDimension: one request must land in all four
// rollups plus the daily bucket. A missing dimension is the kind of bug a
// dashboard hides (it just renders an empty table).
func TestBillingAggregatesEveryDimension(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"models": map[string]interface{}{"m1": map[string]interface{}{"prompt": 1e-6, "completion": 1e-6}},
})
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "m1", "source": "srcA", "key": "***key01", "ok": true,
"prompt_tokens": 100, "completion_tokens": 100, "time": 1750000000000,
})
st := stateOf(t, ps)
for _, dim := range []string{"by_source", "by_model", "by_key", "by_day"} {
m, ok := st[dim].(map[string]interface{})
if !ok || len(m) == 0 {
t.Errorf("%s is empty; a dimension is missing", dim)
}
}
if _, ok := st["by_source"].(map[string]interface{})["srcA"]; !ok {
t.Error("by_source lacks srcA")
}
if _, ok := st["by_key"].(map[string]interface{})["***key01"]; !ok {
t.Error("by_key lacks the gateway key")
}
// Milliseconds must be converted, not used as seconds: a raw 1750000000000
// would land in a year-57000 bucket.
days := st["by_day"].(map[string]interface{})
found := false
for k := range days {
if len(k) == 10 && strings.Contains(k, "-") {
found = true
}
if strings.HasPrefix(k, "5") && len(k) > 6 {
t.Errorf("by_day key %q suggests millisecond timestamps were not converted", k)
}
}
if !found {
t.Errorf("by_day has no YYYY-MM-DD key: %v", days)
}
}
// TestBillingFailedRequestPolicy: a failed request keeps its token cost (tokens
// really were consumed) but drops the flat per_request fee (never charged).
func TestBillingFailedRequestPolicy(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{
"m": map[string]interface{}{"prompt": 1e-6, "completion": 1e-6, "per_request": 0.5},
},
},
})
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "m", "source": "s", "key": "***e", "ok": false, "status": 500,
"prompt_tokens": 1000, "completion_tokens": 0, "time": 1750000000000,
})
st := stateOf(t, ps)
// 1000*1e-6 = 0.001, flat dropped.
approx(t, "failed request", st["total"].(map[string]interface{})["cost"].(float64), 0.001)
if st["total"].(map[string]interface{})["failures"].(float64) != 1 {
t.Error("failures not counted")
}
}
// TestBillingStateAPIReplace: the admin price update must actually change
// subsequent pricing (not just be stored).
func TestBillingStateAPIReplace(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{"m": map[string]interface{}{"prompt": 1e-6, "completion": 0}},
},
})
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "m", "source": "s", "key": "***f", "ok": true,
"prompt_tokens": 1000, "completion_tokens": 0, "time": 1750000000000,
})
approx(t, "before reprice", stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64), 0.001)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{"m": map[string]interface{}{"prompt": 2e-6, "completion": 0}},
},
})
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "m", "source": "s", "key": "***f", "ok": true,
"prompt_tokens": 1000, "completion_tokens": 0, "time": 1750000000000,
})
// 0.001 (old) + 0.002 (new price)
approx(t, "after reprice", stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64), 0.003)
}
// TestBillingPluginDeclaresUI: the shipped plugin must ship its dashboard, or
// "billing is enabled" would be true while showing the user nothing.
func TestBillingPluginDeclaresUI(t *testing.T) {
ps, _ := billingVM(t)
for _, row := range ps.List() {
if row["name"] != "billing" {
continue
}
ui, ok := row["ui"].(map[string]interface{})
if !ok {
t.Fatal("billing declares no ui")
}
if page, _ := ui["page"].(string); page != "billing" {
t.Errorf("ui.page = %v, want \"billing\"", ui["page"])
}
// ONE page, not two. The rule editor and the totals share a sidebar
// entry because billing is one thing: prices decide the numbers and the
// numbers are the result of the prices. Split across two pages, saving a
// price left no on-screen way to see its effect — the loop the operator
// actually works in was cut in half.
pages, _ := ui["pages"].([]string)
if len(pages) != 1 || pages[0] != "billing" {
t.Errorf("ui.pages = %v, want exactly one page [billing]", pages)
}
if n, _ := ui["elements"].(int); n < 1 {
t.Error("billing contributes no element to an existing page")
}
return
}
t.Fatal("billing plugin is not loaded")
}
// TestBillingRuleEditorLivesInsideTheBillingPage: the rule editor is a VIEW
// inside the billing page, not a second sidebar entry.
//
// This started as a separate page and was wrong: saving a price sent the
// operator to a different screen to find out whether it worked. A single page
// with a view switch keeps the loop closed, and the rule editor already refreshes
// the usage numbers on save, so the effect is visible immediately.
//
// The mount must therefore carry BOTH the usage tables and the rule editor, and
// the switch that toggles between them.
func TestBillingRuleEditorLivesInsideTheBillingPage(t *testing.T) {
ps, _ := billingVM(t)
var ui *UIExtension
for _, p := range ps.plugins {
if p.Info.Name == "billing" {
ui = p.UI
}
}
if ui == nil || ui.Page == nil {
t.Fatal("billing plugin loaded with no page")
}
if ui.Page.PageID != "billing" {
t.Errorf("page id = %q, want billing", ui.Page.PageID)
}
mount := ui.Page.Mount
// Both halves on one page.
for _, needle := range []string{`id="billing-kpis"`, `id="billing-by-source"`} {
if !strings.Contains(mount, needle) {
t.Errorf("the usage half is missing %s", needle)
}
}
for _, needle := range []string{
`id="br-body"`,
"data-act='save'", "data-act='export'", "data-act='newprofile'",
".r-url", ".r-mode",
} {
if !strings.Contains(mount, needle) {
t.Errorf("the rule editor is missing %s", needle)
}
}
// The switch itself, or the two halves are both on screen at once.
for _, needle := range []string{`id="billing-view-usage"`, `id="billing-view-rules"`, `window.__billing_view(`} {
if !strings.Contains(mount, needle) {
t.Errorf("the view switch is missing %s", needle)
}
}
// The rules view must start hidden, otherwise it renders under the usage
// tables and the page is a wall of two editors stacked.
if !strings.Contains(mount, `id="billing-view-rules" style="display:none;`) {
t.Error(`the rules view does not start hidden — both halves would render at once`)
}
// NO DUPLICATE IDS between the switch buttons and the view containers.
// They shipped as billing-view-<name> on BOTH the button and the pane, so
// document.getElementById returned the 72px-wide BUTTON for the pane and
// every measurement was of the wrong element: the table measured 0 wide and
// the page looked broken while the layout was fine. The ids must be unique
// and the buttons carry a distinct suffix.
for _, id := range []string{`billing-view-usage`, `billing-view-rules`} {
if n := strings.Count(mount, `id="`+id+`"`); n != 1 {
t.Errorf("id %q appears %d times; getElementById would return the wrong element", id, n)
}
if !strings.Contains(mount, `id="`+id+`-btn"`) {
t.Errorf("switch button for %q is missing its -btn id", id)
}
}
// Both view containers must FILL the pane. The host's tab-pane is a flex
// column, so a plain div shrinks to its content: without flex:1/width:100%
// the rules view measured 72px wide and its table measured 0 — the page
// rendered "nothing" while the DOM was perfectly correct. This is exactly
// the class of bug a DOM assertion misses and a screenshot catches.
for _, needle := range []string{
`id="billing-view-usage" style="flex:1`,
`id="billing-view-rules" style="display:none;flex:1`,
} {
if !strings.Contains(mount, needle) {
t.Errorf("view container does not fill the pane: %q missing — "+
"a flex item without flex:1 collapses to content width", needle)
}
}
}
// TestUIExtensionMergesEveryPluginPage: two plugins contributing pages must
// BOTH appear. The old merge assigned a single field, so the second plugin
// erased the first one's page from the sidebar with no error anywhere.
func TestUIExtensionMergesEveryPluginPage(t *testing.T) {
ps, pdir := billingVM(t)
mk := func(name, code string) {
if err := os.WriteFile(filepath.Join(pdir, name+".lua"), []byte(code), 0644); err != nil {
t.Fatal(err)
}
if err := ps.LoadSource(name, code); err != nil {
t.Fatalf("load %s: %v", name, err)
}
}
mk("other", `
local plugin = {}
plugin.name = "other"
plugin.version = "0.1"
plugin.ui = { page = { page_id = "other-page", title = "Other", order = 90,
mount = "<div id='other-root'></div>" } }
return plugin`)
mk("third", `
local plugin = {}
plugin.name = "third"
plugin.version = "0.1"
plugin.ui = { pages = {
{ page_id = "third-a", title = "Third A", order = 80, mount = "<div id='ta'></div>" },
{ page_id = "third-b", title = "Third B", order = 81, mount = "<div id='tb'></div>" },
} }
return plugin`)
ui := ps.UI()
if ui == nil {
t.Fatal("no merged UI")
}
got := map[string]bool{}
for _, pg := range ui.Pages {
got[pg.PageID] = true
}
// billing contributes exactly one page (the rule editor is a view inside it),
// so it is listed once. The multi-page merging is exercised by the three
// plugins added here.
for _, want := range []string{"billing", "other-page", "third-a", "third-b"} {
if !got[want] {
t.Errorf("merged UI lost page %q; has %v", want, got)
}
}
// A duplicate page_id must not appear twice. Two plugins claiming the same
// id collide in the DOM (getElementById returns the first, the second pane
// is silently unreachable), so the merge keeps the first and drops the
// later one.
mk("collide", `
local plugin = {}
plugin.name = "collide"
plugin.version = "0.1"
plugin.ui = { pages = {
{ page_id = "third-a", title = "Impostor", order = 79, mount = "<div id='impostor'></div>" },
} }
return plugin`)
ui = ps.UI()
counts := map[string]int{}
for _, pg := range ui.Pages {
counts[pg.PageID]++
}
if counts["third-a"] != 1 {
t.Errorf("page id third-a appears %d times; a duplicate id collides in the DOM", counts["third-a"])
}
for _, pg := range ui.Pages {
if pg.PageID == "third-a" && pg.Title == "Impostor" {
t.Error("the LATER plugin won the id; first writer should keep it")
}
}
// Order must be honoured so the sidebar is predictable.
if len(ui.Pages) != 4 {
t.Fatalf("expected all 4 pages to merge, got %d: %v", len(ui.Pages), got)
}
for i := 1; i < len(ui.Pages); i++ {
if ui.Pages[i].Order < ui.Pages[i-1].Order {
t.Errorf("pages out of order at %d: %d before %d",
i, ui.Pages[i-1].Order, ui.Pages[i].Order)
}
}
}
// TestBillingPluginLoadedByDefault: the shipped plugin must load with no
// configuration, since seeding only happens on a fresh plugin dir.
func TestBillingPluginLoadedByDefault(t *testing.T) {
ps, pdir := billingVM(t)
if ps.Count() != 1 {
t.Fatalf("expected 1 bundled plugin, got %d", ps.Count())
}
if _, err := os.Stat(filepath.Join(pdir, "billing.lua")); err != nil {
t.Errorf("billing.lua was not written to the plugin dir: %v", err)
}
}
// TestBillingCountsDegradations: the plugin must distinguish a request that had
// to drop below the top tier from one the top tier served. Without the chain
// trace these were identical in the accounts, so a quietly degraded gateway
// looked healthy while spending more per request.
func TestBillingCountsDegradations(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{
"hi-tier": map[string]interface{}{"prompt": 1e-5, "completion": 1e-5},
"lo-tier": map[string]interface{}{"prompt": 1e-6, "completion": 1e-6},
},
},
})
// Request 1: degraded. tier 1 hard-failed, tier 2 served it.
ps.Fire(StageChainStep, map[string]interface{}{
"kind": "slot_fail", "tier": 1, "source": "t1", "model": "hi-tier",
})
ps.Fire(StageChainStep, map[string]interface{}{
"kind": "selected", "tier": 2, "source": "t2", "model": "lo-tier",
})
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "lo-tier", "source": "t2", "key": "***d1", "ok": true,
"prompt_tokens": 1000, "completion_tokens": 1000,
"degraded": true, "tier_served": 2, "time": 1750000000000,
})
// Request 2: clean, served by the top tier.
ps.Fire(StageChainStep, map[string]interface{}{
"kind": "selected", "tier": 1, "source": "t1", "model": "hi-tier",
})
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "hi-tier", "source": "t1", "key": "***d1", "ok": true,
"prompt_tokens": 1000, "completion_tokens": 1000,
"degraded": false, "tier_served": 1, "time": 1750000000000,
})
st := stateOf(t, ps)
if got := st["degraded_reqs"].(float64); got != 1 {
t.Errorf("degraded_reqs = %v, want 1 (one of the two requests dropped a tier)", got)
}
tiers := st["by_tier_served"].(map[string]interface{})
if tiers["2"].(float64) != 1 {
t.Errorf("by_tier_served[2] = %v, want 1", tiers["2"])
}
if tiers["1"].(float64) != 1 {
t.Errorf("by_tier_served[1] = %v, want 1", tiers["1"])
}
// Cost reflects the model actually served, not the one that should have been.
// 1000*1e-6*2 = 0.002 for the degraded one, 1000*1e-5*2 = 0.02 for the clean one.
approx(t, "total", st["total"].(map[string]interface{})["cost"].(float64), 0.022)
}
// TestBillingAggregatesSkipReasons: skip reasons are the actionable diagnostic
// ("no schedulable slot (cooling or quota exhausted)"), so they must be
// counted. The wait time is normalised, otherwise a fresh row per request would
// appear whenever the busy-wait text varies.
func TestBillingAggregatesSkipReasons(t *testing.T) {
ps, _ := billingVM(t)
ps.Fire(StageChainStep, map[string]interface{}{
"kind": "tier_skip", "tier": 1, "reason": "no schedulable slot (cooling or quota exhausted)",
})
ps.Fire(StageChainStep, map[string]interface{}{
"kind": "tier_busy", "tier": 2, "reason": "no free slot within 2s",
})
ps.Fire(StageChainStep, map[string]interface{}{
"kind": "tier_busy", "tier": 3, "reason": "no free slot within 2.0001s",
})
st := stateOf(t, ps)
reasons := st["skip_reasons"].(map[string]interface{})
if len(reasons) != 2 {
t.Errorf("skip_reasons = %v, want 2 (the two variable waits must collapse to one)", reasons)
}
busy, ok := reasons["no free slot within <wait>"]
if !ok {
t.Errorf("busy reason missing; got %v", reasons)
} else if busy.(float64) != 2 {
t.Errorf("busy count = %v, want 2 (two different wait texts, one cause)", busy)
}
}
// ---- prompt-cache pricing -------------------------------------------------
//
// A cached prompt token is not a fresh one. Charging the full prompt rate made a
// 1M-token request of which 900k were cache reads cost 10 USD instead of ~1.9
// — an order of magnitude, on exactly the traffic the cache exists to make
// cheap. Agent traffic replays long shared prefixes constantly, so this was the
// single largest source of over-billing in the plugin.
func TestBillingCacheHitsAreDiscounted(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{
"m": map[string]interface{}{"prompt": 1e-5, "completion": 1e-5},
},
},
})
// 1M prompt of which 900k cached, default discount 0.1
// 100k fresh * 1e-5 = 1.0 ; 900k cached * 1e-5 * 0.1 = 0.9
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "m", "source": "s", "key": "***c", "ok": true,
"prompt_tokens": 1000000, "completion_tokens": 0,
"cache_hit_tokens": 900000, "time": 1750000000000,
})
approx(t, "cache-discounted cost", stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64), 1.9)
}
// A per-model discount overrides the global one, because the ratio is a
// per-provider fact, not a constant.
func TestBillingCacheDiscountIsPerModel(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{
"free-cache": map[string]interface{}{
"prompt": 1e-5, "completion": 0, "cache_discount": 0,
},
"flat": map[string]interface{}{
"prompt": 1e-5, "completion": 0, "cache_discount": 1,
},
},
},
})
for _, m := range []string{"free-cache", "flat"} {
ps2, _ := billingVM(t)
_ = ps2.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{
m: map[string]interface{}{"prompt": 1e-5, "completion": 0, "cache_discount": map[bool]float64{true: 0, false: 1}[m == "free-cache"]},
},
},
})
ps2.Fire(StageRequestEnd, map[string]interface{}{
"model": m, "source": "s", "key": "***c", "ok": true,
"prompt_tokens": 1000000, "cache_hit_tokens": 1000000,
"time": 1750000000000,
})
want := 0.0
if m == "flat" {
want = 10.0
}
approx(t, m+" (all cached)", stateOf(t, ps2)["total"].(map[string]interface{})["cost"].(float64), want)
}
}
// A misbehaving adapter reporting more cache hits than prompt tokens must not
// produce negative fresh tokens.
func TestBillingCacheHitClampedToPrompt(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{"m": map[string]interface{}{"prompt": 1e-5, "completion": 0}},
},
})
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "m", "source": "s", "key": "***c", "ok": true,
"prompt_tokens": 100, "completion_tokens": 0,
"cache_hit_tokens": 999999, // nonsense from a broken adapter
"time": 1750000000000,
})
// Clamped to 100 cached, 0 fresh => 100 * 1e-5 * 0.1
got := stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64)
if got < 0 {
t.Errorf("cost = %v, must never be negative", got)
}
approx(t, "clamped cost", got, 0.0001)
}
// ---- unpriced traffic -----------------------------------------------------
// An unpriced model silently costing 0 is the most dangerous failure a cost
// plugin has: the bill still adds up, it just quietly under-reports, and
// nothing looks broken. It must be counted and named.
func TestBillingCountsUnpricedTraffic(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{
"priced": map[string]interface{}{"prompt": 1e-5},
},
},
})
// 100k+100k tokens on a model with no price entry.
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "MYSTERY-MODEL", "source": "s", "key": "***u", "ok": true,
"prompt_tokens": 100000, "completion_tokens": 100000, "time": 1750000000000,
})
// A priced one, to prove the counter is selective.
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "priced", "source": "s", "key": "***u", "ok": true,
"prompt_tokens": 1000, "completion_tokens": 0, "time": 1750000000000,
})
st := stateOf(t, ps)
if got := st["unpriced_reqs"].(float64); got != 1 {
t.Errorf("unpriced_reqs = %v, want 1 (only the mystery model)", got)
}
models := st["unpriced_models"].(map[string]interface{})
if models["MYSTERY-MODEL"].(float64) != 1 {
t.Errorf("unpriced_models = %v, want MYSTERY-MODEL counted", models)
}
if _, present := models["priced"]; present {
t.Error("a priced model was counted as unpriced")
}
// The traffic is still recorded: "unpriced" must not mean "invisible".
if got := st["total"].(map[string]interface{})["requests"].(float64); got != 2 {
t.Errorf("total requests = %v, want 2 (unpriced traffic is still traffic)", got)
}
}
// A source-only or key-only price counts as priced: any dimension covering the
// request is enough.
func TestBillingAnyDimensionCountsAsPriced(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"sources": map[string]interface{}{"flat-fee": map[string]interface{}{"per_request": 0.02}},
},
})
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "any-model", "source": "flat-fee", "key": "***p", "ok": true,
"prompt_tokens": 10, "completion_tokens": 0, "time": 1750000000000,
})
st := stateOf(t, ps)
if got := st["unpriced_reqs"].(float64); got != 0 {
t.Errorf("unpriced_reqs = %v, want 0 (the source price covers it)", got)
}
approx(t, "flat fee", st["total"].(map[string]interface{})["cost"].(float64), 0.02)
}
// ---- 峰谷 / 时段定价 ---------------------------------------------------
//
// commandcode 的 DeepSeek V4 系列就是这么定价的:非高峰 17h/天,高峰 01-04 &
// 06-10 UTC 工作日,价格恰好 2 倍。这类规则用静态价目无法表达,而算错方向是
// 静默的——不会报错,只会一直算错。
//
// 时间判据的可测性:os.date("!%H") 取 UTC 小时。测试通过选择"确定落在窗口内"
// 与"确定落在窗口外"的时段来判定,不去伪造时钟(Lua 侧没有可注入的时钟,
// 伪造反而会让测试与真实行为脱节)。
func TestBillingPeakWindowDoublesOutsidePeak(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{
"dsv41": map[string]interface{}{
"prompt": 1.5e-7, "completion": 6e-7,
"peak": map[string]interface{}{
"multiplier": 2,
// 全部 7 天全部 24 小时 ⇒ 永远命中
"windows": []interface{}{
map[string]interface{}{
"days": []interface{}{0, 1, 2, 3, 4, 5, 6},
"hours": []interface{}{[]interface{}{0, 23}},
},
},
},
},
},
},
})
// 1000 prompt + 1000 completion,非高峰 0.00075 → 命中峰谷 ×2 = 0.0015
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "dsv41", "source": "commandcode", "key": "***pk", "ok": true,
"prompt_tokens": 1000, "completion_tokens": 1000, "time": 1750000000000,
})
got := stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64)
want := (1000*1.5e-7 + 1000*6e-7) * 2
approx(t, "always-peak cost", got, want)
}
// 一个不存在的窗口(UTC 25 点不存在)⇒ 永不命中 ⇒ 静态价。
func TestBillingPeakWindowNotHit(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{
"dsv41": map[string]interface{}{
"prompt": 1.5e-7, "completion": 6e-7,
"peak": map[string]interface{}{
"multiplier": 2,
// 星期 = {0..6} 但小时窗写成 [99,100]:永远不可能命中
"windows": []interface{}{
map[string]interface{}{
"days": []interface{}{0, 1, 2, 3, 4, 5, 6},
"hours": []interface{}{[]interface{}{99, 100}},
},
},
},
},
},
},
})
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "dsv41", "source": "commandcode", "key": "***pk", "ok": true,
"prompt_tokens": 1000, "completion_tokens": 1000, "time": 1750000000000,
})
got := stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64)
approx(t, "never-peak cost", got, 1000*1.5e-7+1000*6e-7)
}
// 星期不匹配 ⇒ 不命中。这一条正是"用本地时区算会整体偏移"要防的东西:
// 周日按 UTC 算,用本地时区可能算成周六而错误地命中工作日窗口。
func TestBillingPeakWindowDayMismatch(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{
"dsv41": map[string]interface{}{
"prompt": 1.5e-7, "completion": 6e-7,
"peak": map[string]interface{}{
"multiplier": 2,
// 只在"不存在的星期 7"上开窗(os.date %w 只到 0..6)
"windows": []interface{}{
map[string]interface{}{
"days": []interface{}{7},
"hours": []interface{}{[]interface{}{0, 23}},
},
},
},
},
},
},
})
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "dsv41", "source": "commandcode", "key": "***pk", "ok": true,
"prompt_tokens": 1000, "completion_tokens": 1000, "time": 1750000000000,
})
got := stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64)
if got > (1000*1.5e-7+1000*6e-7)*1.5 {
t.Errorf("cost = %v: a non-matching weekday must not trigger the peak multiplier", got)
}
}
// 没有 peak 规则的条目完全不受影响(向后兼容)。
func TestBillingNoPeakRuleIsUnaffected(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{
"plain": map[string]interface{}{"prompt": 1e-6, "completion": 2e-6},
},
},
})
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "plain", "source": "s", "key": "***n", "ok": true,
"prompt_tokens": 1000, "completion_tokens": 1000, "time": 1750000000000,
})
approx(t, "no-peak cost", stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64), 1000*1e-6+1000*2e-6)
}
// 缓存读价【不】跟着峰谷翻倍:它是另一条上游费率,观测到的非峰谷价里已经含了
// 自己的折扣,跟着翻倍会把两个折扣叠在一起。
func TestBillingPeakDoesNotDoubleCacheRead(t *testing.T) {
ps, _ := billingVM(t)
_ = ps.SetState("billing", map[string]interface{}{
"prices": map[string]interface{}{
"models": map[string]interface{}{
"dsv41": map[string]interface{}{
"prompt": 1.5e-7, "completion": 6e-7, "cache_discount": 0.02,
"peak": map[string]interface{}{
"multiplier": 2,
"windows": []interface{}{map[string]interface{}{
"days": []interface{}{0, 1, 2, 3, 4, 5, 6},
"hours": []interface{}{[]interface{}{0, 23}},
}},
},
},
},
},
})
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "dsv41", "source": "commandcode", "key": "***c", "ok": true,
"prompt_tokens": 1000000, "completion_tokens": 0,
"cache_hit_tokens": 1000000, "time": 1750000000000,
})
// 全部缓存命中 ⇒ 只按 cache 价 = prompt * 0.02,且不翻倍
got := stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64)
approx(t, "cache-only cost (not doubled)", got, 1e6*1.5e-7*0.02)
}
// TestBillingTracksCacheUsage is the guard for the gap production exposed: the
// gateway had prompt_cache_hit_tokens and costFor() priced the cache leg, but no
// bucket recorded the number. On a gateway where 99.88% of prompt tokens were
// cache reads, the report showed a prompt_tokens figure with no way to tell that
// most of it was cached.
func TestBillingTracksCacheUsage(t *testing.T) {
ps, _ := billingVM(t)
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "m", "source": "s", "key": "k", "ok": true,
"prompt_tokens": 1000, "completion_tokens": 50,
"cache_hit_tokens": 900, "cache_reported": true,
})
// A second request from a source that does not report caching at all.
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "m2", "source": "s2", "ok": true,
"prompt_tokens": 100, "completion_tokens": 10,
})
st := ps.State("billing").(map[string]interface{})
total := st["total"].(map[string]interface{})
if total["cache_hit_tokens"] != float64(900) {
t.Errorf("total.cache_hit_tokens = %v, want 900", total["cache_hit_tokens"])
}
if total["cache_fresh_tokens"] != float64(200) {
t.Errorf("total.cache_fresh_tokens = %v, want 200 (1000-900 + 100)", total["cache_fresh_tokens"])
}
// Only the first request reported a cache number.
if total["cache_reported_reqs"] != float64(1) {
t.Errorf("★ total.cache_reported_reqs = %v, want 1 — a source that never "+
"reports cache usage must be distinguishable from one reporting zero hits",
total["cache_reported_reqs"])
}
// Per-source separation.
bySrc := st["by_source"].(map[string]interface{})
s1 := bySrc["s"].(map[string]interface{})
if s1["cache_hit_tokens"] != float64(900) {
t.Errorf("by_source[s].cache_hit_tokens = %v, want 900", s1["cache_hit_tokens"])
}
s2 := bySrc["s2"].(map[string]interface{})
if s2["cache_reported_reqs"] != float64(0) {
t.Errorf("by_source[s2].cache_reported_reqs = %v, want 0", s2["cache_reported_reqs"])
}
if s2["cache_fresh_tokens"] != float64(100) {
t.Errorf("by_source[s2].cache_fresh_tokens = %v, want 100", s2["cache_fresh_tokens"])
}
}
// TestBillingCacheBucketsSurviveOlderStateFiles: a state file written before these
// fields existed must not crash the hook. `nil + number` is an error in Lua, and
// a hook that throws stops accounting for that request entirely — which is how a
// billing gap turns into a silent one.
func TestBillingCacheBucketsSurviveOlderStateFiles(t *testing.T) {
ps, _ := billingVM(t)
// Simulate a state restored from an older build: buckets without the new keys.
legacy := map[string]interface{}{
"total": map[string]interface{}{
"cost": 1.0, "requests": float64(5), "prompt_tokens": float64(500),
"completion_tokens": float64(50), "failures": float64(0),
},
"by_source": map[string]interface{}{
"legacy": map[string]interface{}{"cost": float64(0), "requests": float64(5),
"prompt_tokens": float64(500), "completion_tokens": float64(50), "failures": float64(0)},
},
"by_model": map[string]interface{}{}, "by_key": map[string]interface{}{},
"by_day": map[string]interface{}{}, "started": float64(0),
}
if err := ps.SetState("billing", legacy); err != nil {
t.Fatalf("SetState: %v", err)
}
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "m", "source": "legacy", "ok": true,
"prompt_tokens": 100, "completion_tokens": 10,
"cache_hit_tokens": 60, "cache_reported": true,
})
if len(ps.HookErrors()) != 0 {
t.Fatalf("hook error on a legacy state: %v", ps.HookErrors())
}
st := ps.State("billing").(map[string]interface{})
tot := st["total"].(map[string]interface{})
if tot["requests"] != float64(6) {
t.Errorf("requests = %v, want 6 (5 legacy + 1 new)", tot["requests"])
}
if tot["cache_hit_tokens"] != float64(60) {
t.Errorf("cache_hit_tokens = %v, want 60", tot["cache_hit_tokens"])
}
lg := st["by_source"].(map[string]interface{})["legacy"].(map[string]interface{})
if lg["cache_hit_tokens"] != float64(60) {
t.Errorf("legacy bucket cache_hit_tokens = %v, want 60", lg["cache_hit_tokens"])
}
}
// TestBillingCacheHitClampedInStats: costFor clamps the cache leg, so the
// recorded numbers must be clamped the same way. A provider that reports more
// cache hits than prompt tokens must not produce negative fresh tokens.
func TestBillingCacheHitClampedInStats(t *testing.T) {
ps, _ := billingVM(t)
ps.Fire(StageRequestEnd, map[string]interface{}{
"model": "m", "source": "s", "ok": true,
"prompt_tokens": 100, "completion_tokens": 5,
"cache_hit_tokens": 5000, "cache_reported": true,
})
st := ps.State("billing").(map[string]interface{})
tot := st["total"].(map[string]interface{})
if tot["cache_hit_tokens"] != float64(100) {
t.Errorf("★ cache_hit_tokens = %v, want 100 (clamped to prompt_tokens)",
tot["cache_hit_tokens"])
}
if tot["cache_fresh_tokens"] != float64(0) {
t.Errorf("★ cache_fresh_tokens = %v, want 0, never negative",
tot["cache_fresh_tokens"])
}
}

View File

@ -0,0 +1,416 @@
package lua
import (
"encoding/json"
"os"
"os/exec"
"regexp"
"strings"
"testing"
)
// The billing page rendered EMPTY in production while its data endpoint returned
// 200 with real numbers. The cause was one line: render(st) referenced an
// undefined `s` for two KPI cells, so the ReferenceError aborted the whole
// render and every table stayed at its initial empty innerHTML.
//
// Nothing in the build, the tests or the API surfaced it. This file is the guard
// for the whole class: a plugin's injected UI that references an undefined name,
// or that depends on a container the page does not provide, fails silently.
// billingUI returns the injected markup for the billing plugin: the full page
// mount and the status-page element mount.
func billingUI(t *testing.T) (page string, statusElement string) {
t.Helper()
src, err := os.ReadFile("plugins/billing.lua")
if err != nil {
t.Fatalf("read billing.lua: %v", err)
}
// Located BY CONTENT, not by index. The plugin also uses a long string for
// its inline SVG icon, so "the first long string" is the icon and "the
// second" is the page — which is exactly the kind of positional assumption
// that breaks the next time an icon or a description is added.
page = longStringContaining(t, string(src), "billing-root")
statusElement = longStringContaining(t, string(src), "billing-status-tile")
return page, statusElement
}
// longStringContaining returns the [==[ ... ]==] body that contains marker.
func longStringContaining(t *testing.T, src, marker string) string {
t.Helper()
re := regexp.MustCompile(`(?s)\[==\[(.*?)\]==\]`)
for _, m := range re.FindAllStringSubmatch(src, -1) {
if strings.Contains(m[1], marker) {
return m[1]
}
}
t.Fatalf("no long string contains %q", marker)
return ""
}
// TestBillingMountScriptExecutes is the guard for the production bug.
//
// The Billing page rendered empty while its data endpoint returned 200 with real
// numbers. Cause: render(st) referenced an undefined `s` for two KPI cells, the
// ReferenceError aborted the render, and every table kept its initial empty
// innerHTML. Nothing in the build or the API surfaced it.
//
// Two earlier attempts at a static check were both wrong: a "declared names"
// scan flagged every CSS class inside the inline HTML strings (class, div, td),
// and a CSS-selector parse of the stylesheet reported the stylesheet itself as
// broken. Static analysis of JS embedded in HTML strings is the wrong tool.
//
// So this actually RUNS the script, in node, against a minimal DOM stub, and
// fails on any thrown error. Skipped when node is unavailable, with the reason
// printed — never silently passing as if it had checked.
func TestBillingMountScriptExecutes(t *testing.T) {
page, el := billingUI(t)
scripts := extractScripts(page)
if len(scripts) == 0 {
t.Fatal("no <script> found in the billing page mount")
}
for i, js := range scripts {
assertRendersAndDoesNotThrow(t, i, js)
}
for i, js := range extractScripts(el) {
assertRendersAndDoesNotThrow(t, i, js)
}
}
// assertRendersAndDoesNotThrow executes a mount script under node against a DOM
// stub and fails on EITHER a thrown/reported error OR an empty render.
//
// Checking only for exceptions is not enough, and that is the third wrong
// attempt at this guard. The plugin's render() opens with `if (!st) return;`,
// so passing the wrong field (`render(j.stateX)`) is a SILENT no-op: no throw,
// no console.error, no rejection — just an empty page. Only looking at the
// produced DOM catches that class.
func assertRendersAndDoesNotThrow(t *testing.T, idx int, js string) {
t.Helper()
node, err := exec.LookPath("node")
if err != nil {
t.Skipf("node not available (%v): cannot execute the injected script", err)
}
stub := `
global.window = global;
global.document = {
getElementById: function (id) {
if (!global.__els) global.__els = {};
if (!global.__els[id]) global.__els[id] = {
style: {}, dataset: {}, classList: { add: function(){}, remove: function(){} },
// Both writes count: the Billing page fills innerHTML, the status-page
// tile assigns textContent. Watching only one of them flagged the tile as
// "renders nothing" — a false positive that would have taught everyone to
// ignore this test.
set innerHTML(v) { if (v && String(v).trim()) global.__written.push(id); this.__h = v; },
get innerHTML() { return this.__h || ""; },
set textContent(v) { if (v !== undefined && String(v).trim()) global.__written.push(id); this.__t = v; },
get textContent() { return this.__t || ""; },
set innerText(v) { if (v !== undefined && String(v).trim()) global.__written.push(id); this.__i = v; },
get innerText() { return this.__i || ""; },
appendChild: function(){}, querySelector: function(){ return null; },
querySelectorAll: function(){ return []; }, addEventListener: function(){} };
return global.__els[id];
},
createElement: function () { return { style: {}, dataset: {}, appendChild: function(){}, setAttribute: function(){} }; },
addEventListener: function () {},
};
global.pluginAPI = { onTabShown: function () {} };
global.fetch = function () {
// A payload with real numbers, so a working render produces visible output.
return Promise.resolve({ ok: true, json: function () {
return Promise.resolve({ plugin: "billing", state: {
currency: "USD",
total: { cost: 1.25, requests: 7, prompt_tokens: 100, completion_tokens: 20, failures: 0 },
by_source: { localzen: { cost: 1.25, requests: 7, prompt_tokens: 100, completion_tokens: 20, failures: 0 } },
by_model: { m1: { cost: 1.25, requests: 7, prompt_tokens: 100, completion_tokens: 20, failures: 0 } },
by_key: { k1: { cost: 1.25, requests: 7, prompt_tokens: 100, completion_tokens: 20, failures: 0 } },
by_day: { "2026-01-01": { cost: 1.25, requests: 7, prompt_tokens: 100, completion_tokens: 20, failures: 0 } },
degraded_reqs: 1, unpriced_reqs: 2,
}});
}});
};
global.__written = [];
global.__errors = [];
// The plugin's refresh() swallows its own errors so decoration can never break
// the host page — which is right, and is exactly why the production ReferenceError
// was invisible. So the trace has to come from console.error, which the plugin now
// emits. Hooking process events alone made this test pass against the very bug it
// was written for (verified by re-introducing the typo and watching it stay green).
var __realErr = console.error;
console.error = function () {
global.__errors.push("console.error: " + Array.prototype.map.call(arguments, function (a) {
return (a && a.message) ? a.message : String(a);
}).join(" "));
__realErr.apply(console, arguments);
};
process.on("uncaughtException", function (e) { global.__errors.push("uncaught: " + String(e && e.message || e)); });
process.on("unhandledRejection", function (e) { global.__errors.push("unhandled: " + String(e && e.message || e)); });
`
// 50ms is not a guess: the plugin's IIFE kicks off refresh() which awaits a
// fetch; a rejection lands on the microtask queue almost immediately.
script := strings.Join([]string{
stub, js,
`setTimeout(function(){
console.log("__ERRS__" + JSON.stringify({errors: global.__errors, written: global.__written}));
}, 300);`,
}, "\n")
cmd := exec.Command(node, "-e", script)
out, err := cmd.CombinedOutput()
if err != nil {
t.Errorf("mount script %d crashed under node: %v\n%s", idx, err, out)
return
}
// The script may print other things; find the JSON we appended.
line := ""
for _, l := range strings.Split(string(out), "\n") {
if i := strings.Index(l, "__ERRS__"); i >= 0 {
line = strings.TrimSpace(l[i+len("__ERRS__"):])
}
}
if line == "" {
t.Errorf("mount script %d produced no error report — the harness did not "+
"run to completion, so it cannot be trusted to have checked anything", idx)
return
}
var report struct {
Errors []string `json:"errors"`
Written []string `json:"written"`
}
if err := json.Unmarshal([]byte(line), &report); err != nil {
t.Errorf("could not parse the error report %q: %v", line, err)
return
}
for _, e := range report.Errors {
t.Errorf("mount script %d reported %q — this is the failure that leaves "+
"the Billing page blank while the API still returns data", idx, e)
}
if len(report.Written) == 0 {
t.Errorf("mount script %d wrote NOTHING into any container — the page "+
"renders empty. render() guards with `if (!st) return`, so a wrong "+
"field name is a silent no-op: no throw, no console output, no error "+
"anywhere. This is what the production page looked like.", idx)
}
}
func extractScripts(html string) []string {
re := regexp.MustCompile(`(?s)<script[^>]*>(.*?)</script>`)
var out []string
for _, m := range re.FindAllStringSubmatch(html, -1) {
out = append(out, m[1])
}
return out
}
// TestBillingPageCoversItsData is the other half: the page declares tables for
// per-source / per-model / per-key / per-day and must actually render into them.
// A table id that is never written to is exactly how "the page loads and shows
// nothing" happens without an error.
func TestBillingPageCoversItsData(t *testing.T) {
page, _ := billingUI(t)
for _, id := range []string{
"billing-kpis", "billing-by-source", "billing-by-model",
"billing-by-key", "billing-by-day",
} {
if !strings.Contains(page, `id="`+id+`"`) {
t.Errorf("the page has no container #%s", id)
}
}
// Every container must be written to by the script, not just declared.
scripts := extractScripts(page)
all := strings.Join(scripts, "\n")
for _, id := range []string{"billing-kpis", "billing-by-source", "billing-by-model", "billing-by-key", "billing-by-day"} {
if !strings.Contains(all, `getElementById("`+id+`")`) {
t.Errorf("#%s is declared but never read by the script — it stays empty forever", id)
}
}
}
// TestPluginIconIsNotARawEmoji guards the sidebar icon. The billing plugin
// declared icon = "💰" and the WebUI drops that verbatim into the button, while
// every native tab uses an inline SVG styled with `stroke: currentColor`. An
// emoji there renders at the wrong size and ignores the theme, so it does not
// match its neighbours — which is what the operator reported.
func TestPluginIconIsNotARawEmoji(t *testing.T) {
icon := billingIcon(mustBillingSource(t))
if icon == "" {
t.Fatal("billing declares no icon; the sidebar entry would be blank")
}
if isEmojiIcon(icon) {
t.Errorf("page icon is the raw emoji %q — the WebUI sidebar renders "+
"native tabs as inline SVG (stroke: currentColor), so an emoji is the "+
"wrong size and ignores the theme. Use an inline SVG path instead.", icon)
}
}
// isEmojiIcon reports whether s is a pictographic emoji rather than markup or a
// text glyph. Codepoints in the pictographic blocks, plus the regional-indicator
// pair used by flags.
func isEmojiIcon(s string) bool {
r := []rune(s)
if len(r) == 0 {
return false
}
// Anything containing '<' is markup (an inline <svg>), which is the fix.
if strings.ContainsRune(s, '<') {
return false
}
for _, c := range r {
switch {
case c >= 0x1F300 && c <= 0x1FAFF, // pictographs, symbols, supplemental
c >= 0x1F000 && c <= 0x1F2FF, // mahjong/domino/cards
c >= 0x2600 && c <= 0x27BF, // misc symbols + dingbats
c >= 0x2B00 && c <= 0x2BFF, // arrows/misc symbols
c == 0xFE0F, // variation selector-16
c >= 0x1F1E6 && c <= 0x1F1FF: // regional indicators (flags)
return true
}
}
return false
}
func mustBillingSource(t *testing.T) string {
t.Helper()
b, err := os.ReadFile("plugins/billing.lua")
if err != nil {
t.Fatal(err)
}
return string(b)
}
// billingIcon extracts the declared page icon. It accepts BOTH a quoted string
// and a [==[ ... ]==] long string, because an inline SVG cannot be written as a
// Lua short string without escaping every quote in it.
func billingIcon(src string) string {
if m := regexp.MustCompile(`(?m)^\s*icon\s*=\s*"([^"]*)"`).FindStringSubmatch(src); m != nil {
return m[1]
}
if m := regexp.MustCompile(`(?s)\bicon\s*=\s*\[==\[(.*?)\]==\]`).FindStringSubmatch(src); m != nil {
return m[1]
}
return ""
}
// TestBillingTableHeaderMatchesRowColumns catches column drift.
//
// row() gained cache columns (fresh / cache / cache%) while the header row did
// not, in the same edit. The result is a table whose cells are shifted one
// column left from "fresh" onward — so "cache%" sits under "completion" and the
// last cell has no label. It renders, it has data, and it is wrong in a way that
// takes a careful read to notice.
func TestBillingTableHeaderMatchesRowColumns(t *testing.T) {
page, _ := billingUI(t)
js := strings.Join(extractScripts(page), "\n")
if !strings.Contains(js, "<th>") {
t.Fatal("no table header found in the billing page script")
}
hStart := strings.Index(js, "function tableFor")
if hStart < 0 {
t.Fatal("no tableFor in the billing page script")
}
// Count the usage table's headers ONLY. The count is taken from tableFor to
// the end of the script, which used to be fine when the rule editor lived on
// its own page; now that both halves share one script, the rule table's nine
// headers were counted too and the check reported 17 vs 8 — a failure about
// two unrelated tables. Stop at the rule editor's script.
header := js[hStart:]
if cut := strings.Index(header, "/api/plugins/billing/rules"); cut > 0 {
header = header[:cut]
}
// Count <th> with OR without attributes: the name column now carries
// style='width:22%' so the header and its data cell can be given the same
// width, and a bare "<th>" count silently reported 7 vs 8 — the check
// failing on the very column it had just been taught to size.
th := strings.Count(header, "<th>") + strings.Count(header, "<th ")
rStart := strings.Index(js, "function row")
rEnd := strings.Index(js, "function tableFor")
if rStart < 0 || rEnd < 0 || rEnd <= rStart {
t.Fatal("could not isolate row()")
}
row := js[rStart:rEnd]
// Each cell closes with </td>; the first cell uses <td><b>..</b></td> so
// counting </td> is exact.
td := strings.Count(row, "</td>")
if th != td {
t.Errorf("★ header declares %d columns but row() emits %d cells — the "+
"table is misaligned from the first differing column on", th, td)
}
}
// TestRuleTableLocksColumnWidthsInAColgroup guards the layout class of bug that
// only a screenshot caught: two adjacent headers rendered on top of each other,
// the URL input squeezed to "https:", and the per-model × button landing 78px
// past its cell onto the neighbouring Delete button.
//
// The measurement that found each one:
// - table-layout:fixed with widths on <td> → 954px divided into 9 equal
// 106px columns, every declared width discarded.
// - widths on <th> WITHOUT fixed → browser sizes from content and ignores
// them; the 70px currency column still collapsed to 48px (24px of input).
// - fixed + <th> widths → they take effect, but the model-price cell's width
// then depended on WHICH RULE was widest (352px vs 206px).
//
// The fix is all three together: fixed layout, a colgroup, and flex children
// that may shrink.
func TestRuleTableLocksColumnWidthsInAColgroup(t *testing.T) {
page, _ := billingUI(t)
js := strings.Join(extractScripts(page), "\n")
// Match the ATTRIBUTE, not the bare string: a substring check on
// "table-layout:fixed" is satisfied by the explanatory comment that sits
// three lines above the tag, so removing the attribute from the table still
// passed (mutation-verified). Require it inside a style='...' literal.
if !strings.Contains(js, `font-size:12px;table-layout:fixed'`) {
t.Error("the rule table must stay table-layout:fixed — with auto layout " +
"the <colgroup> widths below are ignored and columns size from content")
}
colStart := strings.Index(js, "<colgroup>")
if colStart < 0 {
t.Fatal("the rule table declares no <colgroup>; per-column widths on " +
"<th> or <td> are advisory and get recomputed from content")
}
// The colgroup is built from a JS array of pixel widths, so assert on the
// array, not on literal "<col " occurrences — counting the literal found
// exactly one and reported "1 columns, want 9" against correct code.
if !strings.Contains(js, "[214, 80, 62, 70, 66, 74, 62, 262, 74]") {
t.Error("the rule table's colgroup widths changed or vanished; measured " +
"values were [214 80 62 70 66 74 62 262 74] for a 954px pane — the " +
"delete column needs 74px or the button wraps to two lines")
}
if !strings.Contains(js, `"<col style='width:" + w + "px'>"`) {
t.Error("colgroup entries are no longer emitted as <col style='width:Npx'>")
}
// Every price-row INPUT must be allowed to shrink, or the row's fixed widths
// sum past the cell and the × button lands on the next column. The × button
// is the opposite: it must NOT shrink (flex:0 0 auto), because a squashed
// delete button is unclickable.
for _, needle := range []string{"m-name", "m-p", "m-c"} {
i := strings.Index(js, "class='"+needle+"'")
if i < 0 {
i = strings.Index(js, "class='ghost small "+needle+"'")
}
if i < 0 {
t.Errorf("price row lost the %s input/button", needle)
continue
}
window := js[i : i+220]
if !strings.Contains(window, "flex:") {
t.Errorf("%s has no flex sizing; the row is 80+96+96+38 ≈ 330px wide "+
"against a 262px cell and overflows onto the next column", needle)
}
if !strings.Contains(window, "min-width:0") {
t.Errorf("%s lacks min-width:0; a flex item will not shrink below its "+
"content width, which is how the × button ended up 78px past its cell", needle)
}
}
i := strings.Index(js, "class='ghost small m-del'")
if i < 0 {
t.Fatal("the per-model delete button is gone")
}
if !strings.Contains(js[i:i+120], "flex:0 0 auto") {
t.Error("the per-model × button must be flex:0 0 auto — it is a fixed-size " +
"control and must never be squeezed by the flexible inputs")
}
}

139
internal/lua/fastvalue.go Normal file
View File

@ -0,0 +1,139 @@
package lua
import (
"encoding/json"
"strconv"
)
// This file exists to remove the JSON round-trip from the plugin hook path.
//
// Measured before this change: one Fire() call cost 14.6us, of which
// json.Marshal(payload) was 4.0us and json.Unmarshal back into interface{} was
// 5.6us — 67% of the whole hook, spent re-deriving a tree that the very next
// line (pushGoValue) already knows how to walk natively.
//
// The round-trip existed because payload arrives as map[string]interface{} and
// pushGoValue wanted a plain interface{} tree. pushGoValue already handles
// map[string]interface{} and []interface{} directly, so the fix is a converter
// that flattens the concrete Go types the gateway actually produces, instead of
// a generic serialize/parse.
//
// The JSON path is kept as the fallback for types the fast path does not know,
// so a plugin payload carrying something exotic still arrives rather than being
// silently dropped.
// jsonNumberType is the type json.Unmarshal produces for every JSON number.
// The fast path recognizes it so numbers survive as numbers rather than being
// stringified into a different Lua type.
type jsonNumberType = float64
// fastToPlain converts the concrete Go values the gateway puts in a hook
// payload into the interface{} tree pushGoValue consumes, without JSON.
//
// The interesting cases are the ones json.Unmarshal would have normalized:
// - json.Number-ish types (all float64 in practice)
// - int / int64 / uint variants, which must land as Lua numbers, not as the
// strings a naive "only handle float64" switch would produce
// - []string and map[string]string, extremely common in payloads and NOT
// handled by a switch that only knows []interface{}
//
// Anything unrecognized returns (value, false) so the caller can fall back to
// the JSON path, which is slower but total.
func fastToPlain(v interface{}) (interface{}, bool) {
switch x := v.(type) {
case nil, bool, string, jsonNumberType:
return v, true
case int:
return float64(x), true
case int8:
return float64(x), true
case int16:
return float64(x), true
case int32:
return float64(x), true
case int64:
return float64(x), true
case uint:
return float64(x), true
case uint8:
return float64(x), true
case uint16:
return float64(x), true
case uint32:
return float64(x), true
case uint64:
return float64(x), true
case float32:
return float64(x), true
case []string:
out := make([]interface{}, len(x))
for i, s := range x {
out[i] = s
}
return out, true
case map[string]string:
out := make(map[string]interface{}, len(x))
for k, s := range x {
out[k] = s
}
return out, true
case map[string]interface{}:
// The overwhelmingly common case, and the one the hook path always
// takes at the top level.
out := make(map[string]interface{}, len(x))
for k, it := range x {
c, ok := fastToPlain(it)
if !ok {
return nil, false
}
out[k] = c
}
return out, true
case []interface{}:
out := make([]interface{}, len(x))
for i, it := range x {
c, ok := fastToPlain(it)
if !ok {
return nil, false
}
out[i] = c
}
return out, true
default:
return nil, false
}
}
// plainForLua returns the interface{} tree to hand to pushGoValue, using the
// fast path when it can and JSON only when it must.
func plainForLua(payload map[string]interface{}) interface{} {
if fast, ok := fastToPlain(payload); ok {
return fast
}
// Rare: some type the fast path does not model. Serialize and let the
// generic decoder normalize it, so the hook still sees the field.
b, err := json.Marshal(payload)
if err != nil {
return payload
}
var decoded interface{}
if err := json.Unmarshal(b, &decoded); err != nil {
return payload
}
return decoded
}
// toPlainSlice is the []interface{} entry point of fastToPlain, exposed so the
// Fire path can flatten a pre-built slice without re-walking the map header.
func toPlainSlice(v []interface{}) []interface{} {
if fast, ok := fastToPlain(v); ok {
return fast.([]interface{})
}
return v
}
// luaNumberString renders a float the way Lua would print it, for the rare case
// a hook wants the textual form. Kept out of the hot path.
func luaNumberString(f float64) string {
return strconv.FormatFloat(f, 'g', -1, 64)
}

View File

@ -0,0 +1,132 @@
package lua
import (
"encoding/json"
"reflect"
"testing"
)
// The hook payload used to be JSON round-tripped on every call. It no longer is,
// so the fast path and the old JSON path must be indistinguishable — otherwise
// a plugin silently sees a different payload than before, which is the worst
// kind of change: it compiles, passes a smoke test, and misprices traffic.
//
// These tests therefore compare the two paths on the SAME inputs rather than
// asserting the fast path's output in isolation.
func viaJSON(t *testing.T, payload map[string]interface{}) interface{} {
t.Helper()
b, err := json.Marshal(payload)
if err != nil {
t.Fatalf("marshal: %v", err)
}
var decoded interface{}
if err := json.Unmarshal(b, &decoded); err != nil {
t.Fatalf("unmarshal: %v", err)
}
return decoded
}
// payloads that exercise every branch of the fast converter.
func payloadCases() []map[string]interface{} {
return []map[string]interface{}{
{"model": "deepseek-v4.1-flash", "source": "commandcode", "ok": true},
// Numbers of every width: a converter that only knows float64 turns
// ints into something else, and a token count that arrives as a string
// makes a Lua hook do arithmetic on nil.
{"i": 42, "i8": int8(8), "i16": int16(16), "i32": int32(32), "i64": int64(1 << 40),
"u": uint(7), "u64": uint64(1 << 50), "f32": float32(1.5), "f64": 2.25},
// Zero, negative, and very large values must stay numbers.
{"zero": 0, "neg": -17, "huge": 1e308, "tiny": 1e-308},
// Slices and string maps: common in payloads and absent from a switch
// that only knows []interface{} / map[string]interface{}.
{"msgs": []interface{}{"a", "b"}, "tags": []string{"x", "y"}},
{"kv": map[string]string{"a": "1", "b": "2"}},
// Nesting, which is where a shallow converter silently drops a level.
{"usage": map[string]interface{}{
"prompt_tokens": 2048, "cache_hit_tokens": 1024,
"nested": map[string]interface{}{"deep": []interface{}{1, "two", true, nil}},
}},
{"nil_field": nil, "empty_map": map[string]interface{}{}, "empty_slice": []interface{}{}},
// A value the fast path does NOT model: it must fall back to JSON and
// still arrive, not disappear.
{"weird": struct {
A int `json:"a"`
B string `json:"b"`
}{1, "x"}},
}
}
func TestFastPathMatchesJSONPath(t *testing.T) {
for i, p := range payloadCases() {
want := viaJSON(t, p)
got := plainForLua(p)
if !reflect.DeepEqual(want, got) {
t.Errorf("case %d: fast path differs from JSON path\n payload: %#v\n json: %#v\n fast: %#v",
i, p, want, got)
}
}
}
// TestFastPathIsActuallyUsed guards against the fast path silently degrading to
// JSON for the payload the gateway really sends. If a future payload gains a
// type the converter does not model, this still works (it falls back) but the
// optimization is gone — and the next person measuring the hook would be
// measuring the old cost without knowing why.
func TestFastPathIsActuallyUsed(t *testing.T) {
// This mirrors the real request_end payload shape from the gateway.
realistic := map[string]interface{}{
"stage": "request_end", "kind": "end", "model": "deepseek-v4.1-flash",
"source": "commandcode", "key": "stress-key", "ok": true,
"status": 200, "duration_ms": 1234,
"usage": map[string]interface{}{
"prompt_tokens": float64(2048), "completion_tokens": float64(512),
"cache_hit_tokens": float64(1024), "total_tokens": float64(3584),
},
"walk": []interface{}{
map[string]interface{}{"kind": "tier_skip", "tier": 1, "source": "", "model": "", "reason": "cooldown"},
map[string]interface{}{"kind": "selected", "tier": 2, "source": "commandcode", "model": "m", "reason": ""},
},
"ts": float64(1700000000),
}
if _, ok := fastToPlain(realistic); !ok {
t.Errorf("★ the realistic request payload does NOT take the fast path — " +
"the JSON round-trip is still on the hot path for real traffic")
}
}
// TestFastPathDoesNotAliasInput: the converter builds a new tree. If it ever
// returned the caller's map directly, a Lua hook's writes could not reach Go —
// but worse, a later mutation of the payload would race with the snapshot the
// persistence saver is holding.
func TestFastPathDoesNotAliasInput(t *testing.T) {
src := map[string]interface{}{
"usage": map[string]interface{}{"prompt_tokens": float64(1)},
"list": []interface{}{"a"},
}
out, ok := fastToPlain(src)
if !ok {
t.Fatal("fast path declined a plain payload")
}
m := out.(map[string]interface{})
m["new"] = "added"
src["also_new"] = "must not appear"
if _, leaked := m["also_new"]; leaked {
t.Error("output map aliases the input map")
}
if _, leaked := src["new"]; leaked {
t.Error("writing to the output mutated the input")
}
// And the nested maps must be copies too.
inner := m["usage"].(map[string]interface{})
inner["prompt_tokens"] = float64(999)
if src["usage"].(map[string]interface{})["prompt_tokens"] != float64(1) {
t.Error("nested map is shared, not copied — a hook could mutate the payload")
}
list := m["list"].([]interface{})
list[0] = "changed"
if src["list"].([]interface{})[0] != "a" {
t.Error("nested slice is shared, not copied")
}
}

View File

@ -0,0 +1,334 @@
package lua
import (
"os"
"path/filepath"
"sync"
"testing"
"time"
)
// Fire runs same-stage hooks in parallel across plugins. These tests pin the
// three things that make that safe, each of which failed at least once while
// the change was being written.
// twoHookPlugins loads two plugins that both hook request_end.
func twoHookPlugins(t *testing.T, codeA, codeB string) *Plugins {
t.Helper()
vm := NewVM(filepath.Join(t.TempDir(), "adapters"))
if err := vm.Start(); err != nil {
t.Fatalf("vm start: %v", err)
}
t.Cleanup(vm.Stop)
ps := NewPlugins(vm, filepath.Join(t.TempDir(), "plugins"))
ps.save.setWakeDelayForTest(time.Millisecond)
t.Cleanup(ps.Close)
ps.DisableStatePersistence()
if err := ps.LoadSource("aaa", codeA); err != nil {
t.Fatalf("load aaa: %v", err)
}
if err := ps.LoadSource("bbb", codeB); err != nil {
t.Fatalf("load bbb: %v", err)
}
return ps
}
func counterPlugin(name string, key string) string {
// Built by concatenation rather than fmt: %q is not a Go verb, and the first
// attempt produced Lua source with a stray '=' that only failed at compile
// time inside three different tests.
return `
local p = { name = "` + name + `" }
p.state = { n = 0 }
p.hooks = { request_end = "bump" }
function p.bump(payload)
p.state.n = p.state.n + 1
if "` + key + `" ~= "" then return { who = "from-` + name + `" } end
return nil
end
return p
`
}
// TestSingleAndParallelPathsMergeIdentically: the single-plugin fast path and
// the multi-plugin parallel path must produce the same payload.
//
// This is not theoretical. The parallel path was written first and the
// single-plugin path kept its old shape; the merge loop was left off the fast
// path, so a lone plugin returning a table had its return value DISCARDED. The
// existing stage-order test caught it — but only because it happened to check
// the payload after Fire. A plugin that returned fields nobody read would have
// broken silently.
func TestSingleAndParallelPathsMergeIdentically(t *testing.T) {
single := twoHookPlugins(t, counterPlugin("aaa", "who"), `
local p = { name = "zzz" }
p.state = { n = 0 }
p.hooks = { request_end = "bump" }
function p.bump(payload) p.state.n = p.state.n + 1 return nil end
return p
`)
// remove the second so exactly one plugin hooks this stage
single.mu.Lock()
single.plugins = single.plugins[:1]
single.mu.Unlock()
single.rebuild()
ps := twoHookPlugins(t, counterPlugin("aaa", "who"), counterPlugin("bbb", "who"))
singleOut := single.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})
psOut := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})
if singleOut["who"] != "from-aaa" {
t.Errorf("★ single-plugin path dropped the return value: %v", singleOut)
}
// Two plugins both return `who`; the merge order decides the winner, and it
// must be deterministic (load order), not scheduler-dependent.
if psOut["who"] != "from-bbb" {
t.Errorf("merge order is not load order: who = %v, want from-bbb", psOut["who"])
}
}
// TestMergeOrderIsDeterministic: goroutine completion order must not leak into
// the result. Running Fire repeatedly must always yield the same payload, or a
// gateway's behaviour changes run to run with the same plugins installed.
func TestMergeOrderIsDeterministic(t *testing.T) {
// The plugins differ in cost so a scheduler-dependent merge is visible.
slow := `
local p = { name = "aaa" }
p.state = { n = 0 }
p.hooks = { request_end = "slow" }
function p.slow(payload)
local acc = 0
for i = 1, 3000 do acc = acc + i % 7 end
p.state.n = p.state.n + 1
return { winner = "aaa", acc = acc }
end
return p
`
fast := `
local p = { name = "bbb" }
p.state = { n = 0 }
p.hooks = { request_end = "fast" }
function p.fast(payload)
p.state.n = p.state.n + 1
return { winner = "bbb" }
end
return p
`
ps := twoHookPlugins(t, slow, fast)
first := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})["winner"]
for i := 0; i < 60; i++ {
got := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})["winner"]
if got != first {
t.Fatalf("★ winner changed between runs: %q then %q — merge depends on goroutine "+
"scheduling, so the same configuration behaves differently run to run", first, got)
}
}
// Load order decides: bbb is loaded second, so it wins.
if first != "bbb" {
t.Errorf("winner = %q, want bbb (load order decides, not timing)", first)
}
}
// TestThrowingHookDoesNotBreakSiblings covers the failure path that golua
// actually produces: a hook raising a Lua error.
//
// The test name used to claim it covered panics, and it used error() to do it.
// Probing the six ways a Lua program can fault (error(), indexing nil, calling
// nil, concatenating nil, arithmetic on nil, unbounded recursion) showed golua
// converts ALL of them into an error RETURN, not a Go panic — so the recover()
// in Fire was untested by that case, and deleting recover() still passed. The
// test was renamed to say what it verifies.
//
// recover() is kept anyway: it guards the Go side of Fire (a nil map write, a
// future change to how the payload is prepared), which is cheap and cannot be
// triggered from Lua today. Claiming it is covered by a Lua test would be the
// kind of assurance that evaporates the first time someone checks.
func TestThrowingHookDoesNotBreakSiblings(t *testing.T) {
throwing := `
local p = { name = "aaa" }
p.state = {}
p.hooks = { request_end = "boom" }
function p.boom(payload) error("intentional failure") end
return p
`
healthy := counterPlugin("bbb", "")
ps := twoHookPlugins(t, throwing, healthy)
out := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"}) // must not crash
if out == nil {
t.Error("Fire returned nil")
}
st, ok := ps.State("bbb").(map[string]interface{})
if !ok {
t.Fatal("healthy plugin has no state")
}
if n := st["n"]; n != float64(1) {
t.Errorf("★ healthy plugin did not run alongside the throwing one: n = %v", n)
}
he := ps.HookErrors()
if len(he) == 0 {
t.Error("a throwing hook was not recorded in hook_errors — the failure would be invisible")
}
}
// TestParallelHooksAllRunOnce: every plugin must be invoked exactly once per
// Fire. A lost or duplicated goroutine shows up as a wrong total, which for the
// billing plugin means a wrong bill.
func TestParallelHooksAllRunOnce(t *testing.T) {
ps := twoHookPlugins(t, counterPlugin("aaa", ""), counterPlugin("bbb", ""))
const fires = 100
for i := 0; i < fires; i++ {
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})
}
for _, name := range []string{"aaa", "bbb"} {
st, ok := ps.State(name).(map[string]interface{})
if !ok {
t.Fatalf("%s has no state", name)
}
if n := st["n"]; n != float64(fires) {
t.Errorf("%s counted %v hooks, want %d", name, n, fires)
}
}
}
// TestConcurrentFireIsSafe drives Fire from many goroutines at once. Each
// plugin has its own Lua state and its own mutex, so this must hold; the race
// detector is what proves it, not the assertions.
func TestConcurrentFireIsSafe(t *testing.T) {
ps := twoHookPlugins(t, counterPlugin("aaa", ""), counterPlugin("bbb", ""))
var wg sync.WaitGroup
for g := 0; g < 8; g++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < 50; i++ {
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
}
}()
}
wg.Wait()
for _, name := range []string{"aaa", "bbb"} {
st, ok := ps.State(name).(map[string]interface{})
if !ok {
t.Fatalf("%s has no state", name)
}
if n := st["n"]; n != float64(400) {
t.Errorf("%s counted %v, want 400", name, n)
}
}
}
// TestFireDoesNotHoldRegistryLockWhileRunningHooks: a hook that installs or
// disables a plugin takes ps.mu for write. If Fire held ps.mu across the hook,
// that would self-deadlock — the exact failure TrueAgent recorded for its own
// parallel stop path.
func TestFireDoesNotHoldRegistryLockWhileRunningHooks(t *testing.T) {
reentrant := `
local p = { name = "aaa" }
p.state = {}
p.hooks = { request_end = "reenter" }
function p.reenter(payload)
-- Reading the registry from inside a hook is the read half of the same lock.
local _ = #payload
return nil
end
return p
`
vm := NewVM(filepath.Join(t.TempDir(), "adapters"))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
ps := NewPlugins(vm, filepath.Join(t.TempDir(), "plugins"))
defer ps.Close()
ps.DisableStatePersistence()
if err := ps.LoadSource("aaa", reentrant); err != nil {
t.Fatal(err)
}
if err := ps.LoadSource("bbb", counterPlugin("bbb", "")); err != nil {
t.Fatal(err)
}
done := make(chan struct{})
go func() {
defer close(done)
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})
// Touch the registry the way an admin request would, right after.
_ = ps.Count()
_ = ps.List()
}()
select {
case <-done:
case <-time.After(10 * time.Second):
t.Fatal("Fire deadlocked against the registry lock")
}
}
// TestBillingSurvivesParallelFire runs the REAL bundled plugin next to another
// one. A synthetic counter cannot catch a mismatch between the documented
// payload shape and what the plugin actually reads.
func TestBillingSurvivesParallelFire(t *testing.T) {
dir := t.TempDir()
pdir := filepath.Join(dir, "plugins")
os.MkdirAll(pdir, 0o755)
src, err := os.ReadFile("plugins/billing.lua")
if err != nil {
t.Fatalf("read billing.lua: %v", err)
}
if err := os.WriteFile(filepath.Join(pdir, "billing.lua"), src, 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(pdir, "observer.lua"),
[]byte(`local p={name="observer"} p.state={n=0}
p.hooks={request_end="see"}
function p.see(payload) p.state.n=p.state.n+1 return nil end
return p`), 0o644); err != nil {
t.Fatal(err)
}
vm := NewVM(filepath.Join(dir, "adapters"))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
ps := NewPlugins(vm, pdir)
ps.save.setWakeDelayForTest(time.Millisecond)
defer ps.Close()
ps.DisableStatePersistence()
if err := ps.LoadDir(); err != nil {
t.Fatal(err)
}
if ps.Count() != 2 {
t.Fatalf("loaded %d plugins, want 2", ps.Count())
}
payload := map[string]interface{}{
"model": "deepseek-v4.1-flash", "source": "commandcode", "key": "k", "ok": true,
"usage": map[string]interface{}{
"prompt_tokens": float64(1000), "completion_tokens": float64(100),
"cache_hit_tokens": float64(0),
},
}
ps.Fire(StageRequestEnd, payload)
st := ps.State("billing")
if st == nil {
t.Fatal("billing produced no state")
}
m := st.(map[string]interface{})
total, ok := m["total"].(map[string]interface{})
if !ok {
t.Fatalf("billing.total is %T", m["total"])
}
if total["requests"] != float64(1) {
t.Errorf("billing counted %v requests, want 1 — the plugin was starved by the parallel path",
total["requests"])
}
if n := ps.State("observer").(map[string]interface{})["n"]; n != float64(1) {
t.Errorf("observer counted %v, want 1", n)
}
if len(ps.HookErrors()) != 0 {
t.Errorf("hook errors under parallel Fire: %v", ps.HookErrors())
}
}

View File

@ -0,0 +1,76 @@
package lua
import (
"os"
"path/filepath"
"strings"
"testing"
)
// A plugin hook that RAISES must not take the process down. This is the
// production outage: a Lua error anywhere in a hook made golua's callEx call
// L.StackTrace(), which calls lua_getinfo and SIGSEGVs on a deep enough stack —
// a C-level signal Go cannot recover from, so one buggy plugin killed the whole
// gateway and took every in-flight request with it.
//
// The fix routes hook calls through a Lua-side pcall guard, so the error comes
// back as an ordinary return value. This test fires hooks that raise on purpose
// and asserts three things: the process survives, the failure is RECORDED, and
// a well-behaved plugin on the same state keeps working afterwards (the error
// must not poison the Lua state).
func TestHookThatRaisesDoesNotCrashTheProcess(t *testing.T) {
ps, pdir := billingVM(t)
boom := `
local plugin = {}
plugin.name = "boom"
plugin.version = "0.1"
function plugin.request_end(payload)
-- Raise on a table index, the exact shape of the billing bug that caused the
-- outage. Deliberately NOT a syntax error: this must load fine and fail only
-- when invoked.
local x = nil
return x.field
end
return plugin`
if err := os.WriteFile(filepath.Join(pdir, "boom.lua"), []byte(boom), 0644); err != nil {
t.Fatal(err)
}
if err := ps.LoadSource("boom", boom); err != nil {
t.Fatalf("load boom: %v", err)
}
payload := map[string]interface{}{
"model": "m", "source": "s", "ok": true,
"prompt_tokens": 100, "completion_tokens": 10, "time": 1750000000000,
}
// Fire many times: a single call could pass by luck, but if the error ever
// escapes into golua's C path the process dies and this test never returns.
for i := 0; i < 50; i++ {
ps.Fire(StageRequestEnd, payload)
}
// Reaching this line at all is the primary assertion.
errs := ps.HookErrors()
end, ok := errs[string(StageRequestEnd)]
if !ok {
t.Fatal("a raising hook left no record — failures must be observable, not swallowed")
}
if end["count"] == nil || end["count"].(int) == 0 {
t.Error("hook error count is zero despite 50 raising calls")
}
msg, _ := end["last_error"].(string)
if !strings.Contains(msg, "boom") {
t.Errorf("last_error does not name the offending plugin: %q", msg)
}
// The billing plugin shares the same Plugins registry and must still work:
// one broken plugin may not disable the others.
st, _ := ps.State("billing").(map[string]interface{})
if st == nil || st["total"] == nil {
t.Fatalf("the healthy plugin stopped working after another plugin raised")
}
tot, _ := st["total"].(map[string]interface{})
if tot == nil || tot["requests"] == nil || tot["requests"].(float64) == 0 {
t.Errorf("billing recorded no requests after the raising plugin ran: %v", st["total"])
}
}

View File

@ -0,0 +1,283 @@
package lua
import (
"encoding/json"
"os"
"path/filepath"
"testing"
"time"
)
// These cover plugin state persistence: a plugin's totals live in the Lua VM,
// which dies with the process. Measured on the real gateway before this change:
// 218,241 requests and 26.2M prompt tokens gone after one systemctl restart.
func persistTestPlugin() string {
return `
local plugin = { name = "counter", version = "1.0" }
plugin.state = { n = 0, prices_seen = 0 }
plugin.prices = { rate = 1 }
plugin.hooks = { request_end = "bump" }
function plugin.bump(payload)
plugin.state.n = plugin.state.n + 1
return nil
end
return plugin
`
}
// newPersistVM wires a plugin registry on a fresh dir, mirroring newPluginVM.
func newPersistVM(t *testing.T) (*Plugins, string) {
t.Helper()
dir := t.TempDir()
vm := NewVM(filepath.Join(dir, "adapters"))
if err := vm.Start(); err != nil {
t.Fatalf("vm start: %v", err)
}
t.Cleanup(vm.Stop)
pdir := filepath.Join(dir, "plugins")
ps := NewPlugins(vm, pdir)
// Shorten the saver debounce: Close() waits for the saver goroutine, so
// with the production 2s interval every test would pay 2s on teardown.
ps.save.setWakeDelayForTest(2 * time.Millisecond)
t.Cleanup(ps.Close)
return ps, pdir
}
func loadCounter(t *testing.T, ps *Plugins) {
t.Helper()
if err := ps.LoadSource("counter", persistTestPlugin()); err != nil {
t.Fatalf("load: %v", err)
}
}
func stateN(t *testing.T, ps *Plugins) float64 {
t.Helper()
st := ps.State("counter")
m, ok := st.(map[string]interface{})
if !ok {
t.Fatalf("state is %T, want map", st)
}
n, ok := m["n"].(float64)
if !ok {
t.Fatalf("state.n is %T, want float64", m["n"])
}
return n
}
// TestStateSurvivesRestart is the defect itself: a rebuilt registry over the
// same plugin dir must come up with the previous totals, not at zero.
func TestStateSurvivesRestart(t *testing.T) {
ps, pdir := newPersistVM(t)
loadCounter(t, ps)
for i := 0; i < 25; i++ {
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
}
if got := stateN(t, ps); got != 25 {
t.Fatalf("in-process total = %v, want 25", got)
}
// Force the write the saver would do, so the test does not depend on timing.
ps.save.flush()
ps.Close()
// A brand-new registry over the same dir: this is the restart.
vm := NewVM(filepath.Join(t.TempDir(), "adapters"))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
ps2 := NewPlugins(vm, pdir)
defer ps2.Close()
loadCounter(t, ps2)
if got := stateN(t, ps2); got != 25 {
t.Fatalf("★ total after restart = %v, want 25 — this is the 'restart loses the books' bug", got)
}
}
// TestPricesAndStateAreSeparate: restoring configuration over history (or the
// reverse) would either erase the totals or resurrect stale prices.
func TestPricesAndStateAreSeparate(t *testing.T) {
ps, pdir := newPersistVM(t)
loadCounter(t, ps)
for i := 0; i < 7; i++ {
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
}
ps.save.flush()
ps.Close()
b, err := os.ReadFile(ps.stateFile("counter"))
if err != nil {
t.Fatalf("no state file: %v", err)
}
var saved struct {
State map[string]interface{} `json:"state"`
Prices map[string]interface{} `json:"prices"`
}
if err := json.Unmarshal(b, &saved); err != nil {
t.Fatalf("state file is not valid JSON: %v", err)
}
if saved.Prices == nil || saved.Prices["rate"] != float64(1) {
t.Errorf("prices were not persisted separately: %v", saved.Prices)
}
if saved.State["n"] != float64(7) {
t.Errorf("state.n = %v, want 7", saved.State["n"])
}
_ = pdir
}
// TestPricesOnlyUpdatePersists guards a real hole: SetState returns early for a
// prices-only payload (correctly leaving state alone), and the first version
// returned before the persistence write — so a reprice was durable in memory
// only and a restart silently reverted to the old prices.
func TestPricesOnlyUpdatePersists(t *testing.T) {
ps, _ := newPersistVM(t)
loadCounter(t, ps)
for i := 0; i < 5; i++ {
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
}
ps.save.flush()
err := ps.SetState("counter", map[string]interface{}{
"prices": map[string]interface{}{"rate": 42},
})
if err != nil {
t.Fatalf("SetState: %v", err)
}
if got := stateN(t, ps); got != 5 {
t.Errorf("a prices-only payload must not touch state: n = %v, want 5", got)
}
b, err := os.ReadFile(ps.stateFile("counter"))
if err != nil {
t.Fatalf("no state file after a prices-only update: %v", err)
}
var saved struct {
State map[string]interface{} `json:"state"`
Prices map[string]interface{} `json:"prices"`
}
if err := json.Unmarshal(b, &saved); err != nil {
t.Fatal(err)
}
if saved.Prices["rate"] != float64(42) {
t.Errorf("★ new price not persisted: %v — a restart would revert to the old price", saved.Prices)
}
if saved.State["n"] != float64(5) {
t.Errorf("state was clobbered by a price change: %v", saved.State["n"])
}
}
// TestHookReturnValueSurvivesSnapshot guards the ordering bug: snapshotting
// plugin.state resets the Lua stack, so doing it BEFORE reading the hook's
// return value silently turned every opinionated plugin into a silent one —
// breaking the documented "return a table to merge into payload" contract with
// no error anywhere.
func TestHookReturnValueSurvivesSnapshot(t *testing.T) {
ps, _ := newPersistVM(t)
code := `
local plugin = { name = "opinionated" }
plugin.state = { n = 0 }
plugin.hooks = { request_end = "tag" }
function plugin.tag(payload)
plugin.state.n = plugin.state.n + 1
return { cost_usd = 1.25, verdict = "billed" }
end
return plugin
`
if err := ps.LoadSource("opinionated", code); err != nil {
t.Fatal(err)
}
out := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
if out["cost_usd"] != 1.25 {
t.Errorf("★ hook return value lost: payload = %v", out)
}
if out["verdict"] != "billed" {
t.Errorf("merged field lost: %v", out)
}
ps.save.flush()
st := ps.State("opinionated").(map[string]interface{})
if st["n"] != float64(1) {
t.Errorf("state.n = %v, want 1 (the hook still ran)", st["n"])
}
}
// TestCorruptStateFileIsNotFatal: a truncated write must degrade to compiled-in
// defaults, never to a gateway that refuses to start.
func TestCorruptStateFileIsNotFatal(t *testing.T) {
ps, _ := newPersistVM(t)
loadCounter(t, ps)
for i := 0; i < 3; i++ {
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
}
ps.save.flush()
ps.Close()
if err := os.WriteFile(ps.stateFile("counter"), []byte("{not json"), 0o644); err != nil {
t.Fatal(err)
}
vm := NewVM(filepath.Join(t.TempDir(), "adapters"))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
ps2 := NewPlugins(vm, ps.dir)
defer ps2.Close()
var warned bool
ps2.logf = func(string, ...interface{}) { warned = true }
loadCounter(t, ps2)
if got := stateN(t, ps2); got != 0 {
t.Errorf("with a corrupt file the plugin must fall back to its defaults, got n = %v", got)
}
if !warned {
t.Error("a corrupt state file must warn the operator, not fail silently")
}
// And it must still forward.
ps2.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
}
// TestFlushIsCoalesced: N mutations must not become N writes. A synchronous
// per-request write would put a file write on the hot path.
func TestFlushIsCoalesced(t *testing.T) {
ps, _ := newPersistVM(t)
loadCounter(t, ps)
for i := 0; i < 500; i++ {
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
}
ps.save.flush()
// 500 mutations must collapse to a handful of writes, not 500. The bound is
// loose on purpose: the saver also flushes on a timer, so how many writes
// happen during the loop depends on how long 500 hook calls take. What must
// never happen is one write per mutation.
if w := ps.save.writes.Load(); w > 5 {
t.Errorf("500 hook calls produced %d file writes; the saver must coalesce", w)
}
if got := stateN(t, ps); got != 500 {
t.Fatalf("n = %v, want 500", got)
}
}
// TestCloseFlushesTail: the shutdown path must not lose the last interval,
// which would reintroduce the same defect in a smaller window.
func TestCloseFlushesTail(t *testing.T) {
ps, _ := newPersistVM(t)
loadCounter(t, ps)
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
ps.Close()
// Close() already waits for the final flush; no sleep needed.
b, err := os.ReadFile(ps.stateFile("counter"))
if err != nil {
t.Fatalf("★ Close() did not flush: %v — a shutdown loses the tail", err)
}
var saved struct {
State map[string]interface{} `json:"state"`
}
if err := json.Unmarshal(b, &saved); err != nil {
t.Fatal(err)
}
if saved.State["n"] != float64(1) {
t.Errorf("flushed n = %v, want 1", saved.State["n"])
}
}

1584
internal/lua/plugins.go Normal file

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,377 @@
package lua
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
)
// newPluginVM builds a VM plus a plugin registry rooted at dir.
func newPluginVM(t *testing.T) (*VM, *Plugins, string) {
t.Helper()
dir := filepath.Join(t.TempDir(), "adapters")
vm := NewVM(dir)
if err := vm.Start(); err != nil {
t.Fatalf("vm start: %v", err)
}
t.Cleanup(vm.Stop)
pdir := filepath.Join(t.TempDir(), "plugins")
return vm, NewPlugins(vm, pdir), pdir
}
// loadPlugin writes one plugin to disk and loads it.
func loadPlugin(t *testing.T, ps *Plugins, name, code string) error {
t.Helper()
if err := os.MkdirAll(ps.dir, 0755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(ps.dir, name+".lua"), []byte(code), 0644); err != nil {
t.Fatal(err)
}
return ps.LoadSource(name, code)
}
// TestPluginManifestAndHooks: the two hook registration forms both work and the
// manifest is read.
func TestPluginManifestAndHooks(t *testing.T) {
_, ps, _ := newPluginVM(t)
code := `
local p = {}
p.name = "demo"
p.version = "1.2.3"
p.description = "a demo plugin"
p.author = "tester"
p.hooks = { request_end = "on_end" }
function p.on_end(payload)
payload.seen = true
payload.name_seen = "demo"
return payload
end
return p
`
if err := loadPlugin(t, ps, "demo", code); err != nil {
t.Fatalf("load: %v", err)
}
list := ps.List()
if len(list) != 1 {
t.Fatalf("List() = %d plugins, want 1", len(list))
}
if list[0]["name"] != "demo" || list[0]["version"] != "1.2.3" {
t.Errorf("manifest not read: %+v", list[0])
}
hooks := list[0]["hooks"].([]string)
if len(hooks) != 1 || hooks[0] != string(StageRequestEnd) {
t.Errorf("hooks = %v, want [request_end]", hooks)
}
out := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})
if out["seen"] != true || out["name_seen"] != "demo" {
t.Errorf("hook did not mutate payload: %+v", out)
}
}
// TestPluginAnonymousHookForm: `request_end = function() end` directly on the
// table must register too, since a single-hook plugin should not need a name.
func TestPluginAnonymousHookForm(t *testing.T) {
_, ps, _ := newPluginVM(t)
code := `
local p = { name = "anon" }
p.request_end = function(payload)
payload.hit = 1
return payload
end
return p
`
if err := loadPlugin(t, ps, "anon", code); err != nil {
t.Fatalf("load: %v", err)
}
out := ps.Fire(StageRequestEnd, map[string]interface{}{})
if out["hit"] != float64(1) {
t.Errorf("anonymous hook did not fire: %+v", out)
}
}
// TestPluginHookStagesFireInOrder: each stage reaches only its own hooks.
func TestPluginHookStagesFireInOrder(t *testing.T) {
_, ps, _ := newPluginVM(t)
code := `
local p = { name = "stages" }
p.hooks = {
request_start = "s1",
routed = "s2",
request_end = "s3",
}
function p.s1(x) x.order = (x.order or "") .. "1" return x end
function p.s2(x) x.order = (x.order or "") .. "2" return x end
function p.s3(x) x.order = (x.order or "") .. "3" return x end
return p
`
if err := loadPlugin(t, ps, "stages", code); err != nil {
t.Fatalf("load: %v", err)
}
payload := map[string]interface{}{}
ps.Fire(StageRequestStart, payload)
ps.Fire(StageRouted, payload)
ps.Fire(StageRequestEnd, payload)
if payload["order"] != "123" {
t.Errorf("stage order = %v, want \"123\"", payload["order"])
}
}
// TestPluginErrorIsContained is the critical safety property: a throwing hook
// must not propagate. Forwarding depends on it.
func TestPluginErrorIsContained(t *testing.T) {
_, ps, _ := newPluginVM(t)
code := `
local p = { name = "boom" }
p.hooks = { request_end = "kaboom" }
function p.kaboom(payload)
error("intentional plugin failure")
end
return p
`
if err := loadPlugin(t, ps, "boom", code); err != nil {
t.Fatalf("load: %v", err)
}
// Must not panic and must return the payload unchanged.
out := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})
if out["model"] != "m" {
t.Errorf("payload was altered by a failing plugin: %+v", out)
}
// And the failure must be visible, not silent.
errs := ps.HookErrors()
if errs["request_end"] == nil {
t.Error("a failing plugin left no error record; it would be silently missing")
}
}
// TestPluginFailingHookDoesNotBlockLaterPlugins: one bad plugin must not stop
// the next one from running.
func TestPluginFailingHookDoesNotBlockLaterPlugins(t *testing.T) {
_, ps, _ := newPluginVM(t)
bad := `
local p = { name = "bad" }
p.hooks = { request_end = "f" }
function p.f(x) error("boom") end
return p
`
good := `
local p = { name = "good" }
p.hooks = { request_end = "f" }
function p.f(x) x.good = true return x end
return p
`
_ = loadPlugin(t, ps, "bad", bad)
if err := loadPlugin(t, ps, "good", good); err != nil {
t.Fatalf("load good: %v", err)
}
out := ps.Fire(StageRequestEnd, map[string]interface{}{})
if out["good"] != true {
t.Errorf("a good plugin was blocked by a failing one: %+v", out)
}
}
// TestPluginSyntaxErrorIsIsolated: a plugin that will not compile is listed
// with its error and is never called — it must not prevent LoadDir from loading
// the rest.
func TestPluginSyntaxErrorIsIsolated(t *testing.T) {
_, ps, _ := newPluginVM(t)
broken := "this is not lua((("
good := `
local p = { name = "ok" }
p.hooks = { request_end = "f" }
function p.f(x) x.ok = true return x end
return p
`
_ = loadPlugin(t, ps, "broken", broken)
if err := loadPlugin(t, ps, "ok", good); err != nil {
t.Fatalf("load ok: %v", err)
}
if err := ps.LoadDir(); err != nil {
t.Fatalf("LoadDir: %v", err)
}
// The broken plugin must not be callable and must carry an error.
for _, row := range ps.List() {
if row["name"] == "broken" {
if row["loaded"] == true {
t.Error("a plugin with a syntax error reported itself as loaded")
}
if row["error"] == nil || row["error"] == "" {
t.Error("a broken plugin carries no error message")
}
}
}
// The good plugin still works.
out := ps.Fire(StageRequestEnd, map[string]interface{}{})
if out["ok"] != true {
t.Errorf("good plugin stopped working: %+v", out)
}
}
// TestPluginUIExtension: a plugin can contribute a page and elements.
func TestPluginUIExtension(t *testing.T) {
_, ps, _ := newPluginVM(t)
code := `
local p = { name = "ui" }
p.hooks = { request_end = "f" }
function p.f(x) return x end
p.ui = {
page = {
page_id = "billing",
title = "Billing",
icon = "💰",
order = 50,
mount = "<div id=billing>hi</div><script>console.log('m')</script>",
},
elements = {
{ target = "status", anchor = "top", mount = "<div>cost</div>" },
},
}
return p
`
if err := loadPlugin(t, ps, "ui", code); err != nil {
t.Fatalf("load: %v", err)
}
// Every contributed page — including the single `page` field — is folded
// into one merged list. Reading ui.Page here would silently pass on a
// payload whose pages were all dropped.
ui := ps.UI()
if len(ui.Pages) != 1 {
t.Fatalf("expected 1 merged page, got %d", len(ui.Pages))
}
pg := ui.Pages[0]
if pg.PageID != "billing" || pg.Title != "Billing" {
t.Errorf("page = %+v", pg)
}
if !strings.Contains(pg.Mount, "console.log") {
t.Error("mount lost its script content")
}
if len(ui.Elements) != 1 || ui.Elements[0].Target != "status" {
t.Errorf("elements = %+v", ui.Elements)
}
}
// TestPluginHookReturnsNilIsNoOpinion: a hook returning nothing must leave the
// payload untouched (plugins should not be forced to echo it back).
func TestPluginHookReturnsNilIsNoOpinion(t *testing.T) {
_, ps, _ := newPluginVM(t)
code := `
local p = { name = "silent" }
p.hooks = { request_end = "f" }
function p.f(payload)
-- records nothing, returns nothing
return nil
end
return p
`
if err := loadPlugin(t, ps, "silent", code); err != nil {
t.Fatalf("load: %v", err)
}
out := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "ok": true})
if out["model"] != "m" || out["ok"] != true {
t.Errorf("a no-op hook disturbed the payload: %+v", out)
}
}
// TestPluginUIJSONShape is a wire-format guard: the kernel sends this to the
// browser, so the shape is a contract with the WebUI.
func TestPluginUIJSONShape(t *testing.T) {
_, ps, _ := newPluginVM(t)
code := `
local p = { name = "shape" }
p.hooks = { request_end = "f" }
function p.f(x) return x end
p.ui = { elements = { { target = "keys", mount = "<b>k</b>" } } }
return p
`
if err := loadPlugin(t, ps, "shape", code); err != nil {
t.Fatalf("load: %v", err)
}
b, err := json.Marshal(ps.UI())
if err != nil {
t.Fatalf("marshal UI: %v", err)
}
var view struct {
Elements []struct {
Target string `json:"target"`
Mount string `json:"mount"`
} `json:"elements"`
}
if err := json.Unmarshal(b, &view); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if len(view.Elements) != 1 || view.Elements[0].Target != "keys" {
t.Errorf("UI JSON shape = %+v", view.Elements)
}
}
// TestPluginFireWithNoPluginsIsNoop: an empty registry must not allocate or fail.
func TestPluginFireWithNoPluginsIsNoop(t *testing.T) {
_, ps, _ := newPluginVM(t)
in := map[string]interface{}{"a": 1}
out := ps.Fire(StageRequestEnd, in)
if out["a"] != 1 || ps.Count() != 0 {
t.Errorf("empty registry misbehaved: %+v", out)
}
}
// TestChainStepIsARealStage: chain_step is documented as a distinct stage that
// fires once per step of an AUTO walk. If it were only a field on `routed`, a
// plugin author following the docs would silently get one event instead of the
// whole walk.
func TestChainStepIsARealStage(t *testing.T) {
found := false
for _, s := range AllStages {
if s == StageChainStep {
found = true
}
}
if !found {
t.Fatal("StageChainStep is not in AllStages, so the dispatcher never registers it")
}
// Ordering: it must sit between request_start and routed, which is what
// docs/plugins.md promises.
var iStart, iStep, iRouted = -1, -1, -1
for i, s := range AllStages {
switch s {
case StageRequestStart:
iStart = i
case StageChainStep:
iStep = i
case StageRouted:
iRouted = i
}
}
if !(iStart < iStep && iStep < iRouted) {
t.Errorf("stage order = %v, want request_start < chain_step < routed", AllStages)
}
_, ps, _ := newPluginVM(t)
code := `
local p = { name = "stepper" }
p.hooks = { chain_step = "s" }
function p.s(payload)
payload.kinds = (payload.kinds or "")
return nil
end
return p
`
if err := loadPlugin(t, ps, "stepper", code); err != nil {
t.Fatalf("load: %v", err)
}
// It must actually dispatch.
seen := false
for _, row := range ps.List() {
if row["name"] == "stepper" {
hooks := row["hooks"].([]string)
for _, h := range hooks {
if h == string(StageChainStep) {
seen = true
}
}
}
}
if !seen {
t.Error("a plugin registered for chain_step is not reported as such")
}
}

View File

@ -36,7 +36,7 @@ import (
golua "github.com/aarzilli/golua/lua"
)
//go:embed adapters/*.lua
//go:embed adapters/*.lua plugins/*.lua
var bundledAdapters embed.FS
// adapterGlobal is the reserved global holding the adapter table after the
@ -103,6 +103,11 @@ type adapterPool struct {
lastGrow time.Time
idleRounds int // consecutive janitor rounds that saw reclaimable slack
peakInUse int // high-water mark of inUse, for observability
// pluginMode makes boot() store the returned table under pluginGlobal
// instead of adapterGlobal. Everything else (elastic sizing, reclaim) is
// identical, which is why plugins reuse this pool rather than getting a
// second implementation.
pluginMode bool
}
const (
@ -116,6 +121,13 @@ const (
// growCooldown keeps a burst of misses from batching repeatedly while the
// previous batch is still booting.
growCooldown = time.Second
// maxPluginStates caps how many concurrent VM states ONE plugin may occupy.
// A plugin is third-party code on the request path, so its ceiling is much
// lower than an adapter's (which is sized from the sources'
// max_concurrent): a plugin hook is a short synchronous call, so a handful
// of states is already far more parallelism than any real hook needs, and a
// runaway plugin cannot balloon memory the way a per-source adapter pool can.
maxPluginStates = 4
// residentWorkers is how many states an adapter keeps warm once it has
// served at least one request. Booting is milliseconds, but keeping one warm
// removes that from the critical path of the next request. Adapters that
@ -265,7 +277,14 @@ func (p *adapterPool) boot() (*worker, error) {
L.Close()
return nil, fmt.Errorf("adapter %s must return a table", p.name)
}
L.SetGlobal(adapterGlobal)
// SetGlobal POPS the value off the stack, so it can only be called once per
// boot. Plugins therefore store under pluginGlobal only, and NOT under
// adapterGlobal: a second SetGlobal on the now-empty stack would assign nil.
if p.pluginMode {
L.SetGlobal(pluginGlobal)
} else {
L.SetGlobal(adapterGlobal)
}
L.SetTop(0)
return &worker{L: L}, nil
}
@ -601,6 +620,55 @@ func (v *VM) Start() error {
return nil
}
// bundledPluginsDir is the directory inside the embedded FS holding the
// plugins shipped with the gateway. It is separate from adapters/ on purpose:
// the two are loaded by different machinery into different kinds of Lua state
// (a protocol transform vs. request-pipeline hooks), and keeping them apart
// makes it obvious that dropping a file in one does not affect the other.
const bundledPluginsDir = "plugins"
// ReadBundledPlugin returns the source of a plugin shipped with the gateway.
// It exists so a test (or an operator tool) can load a bundled plugin without
// depending on whether seeding has already run for this directory.
func ReadBundledPlugin(name string) (string, error) {
data, err := bundledAdapters.ReadFile(bundledPluginsDir + "/" + name + ".lua")
if err != nil {
return "", fmt.Errorf("bundled plugin %s: %w", name, err)
}
return string(data), nil
}
// writeBundledPlugins seeds the plugin directory with the shipped plugins.
//
// It runs only when the directory does not exist yet (same rule as adapters):
// once the directory exists it is authoritative, so deleting a shipped plugin is
// a real delete and editing one survives restarts.
func writeBundledPlugins(dir string) error {
if dir == "" {
return nil
}
if err := os.MkdirAll(dir, 0755); err != nil {
return fmt.Errorf("mkdir plugin dir: %w", err)
}
entries, err := bundledAdapters.ReadDir(bundledPluginsDir)
if err != nil {
return nil // nothing embedded; not an error
}
for _, e := range entries {
if e.IsDir() || filepath.Ext(e.Name()) != ".lua" {
continue
}
data, err := bundledAdapters.ReadFile(bundledPluginsDir + "/" + e.Name())
if err != nil {
continue
}
if err := os.WriteFile(filepath.Join(dir, e.Name()), data, 0644); err != nil {
return err
}
}
return nil
}
func (v *VM) Stop() {
select {
case <-v.janitorStop:
@ -925,9 +993,47 @@ func restorePcall(L *golua.State) {
}
}
// hookGuardName is the Lua global that wraps every plugin hook call.
const hookGuardName = "__llmsproxy_call_hook"
// hookGuardSrc defines the wrapper. It exists because of a hard constraint in
// the binding: golua's callEx calls L.StackTrace() on ANY pcall error, and
// StackTrace() calls lua_getinfo, which SIGSEGVs in this LuaJIT build once the
// stack is deep enough (a request_end payload with the AUTO chain trace does
// it). Go cannot recover from a C-level signal, so a single Lua mistake inside
// one plugin killed the whole gateway — the production outage this was written
// for. Repeated crashes proved the boundary is not theoretical.
//
// pcall INSIDE Lua catches the error before golua ever sees a non-zero
// pcall status, so the C stack-trace path is never entered. The failure comes
// back as an ordinary (nil, message) pair, which the Go side records in
// hook_errors and moves on from — the documented contract that "a broken plugin
// must not affect request forwarding" finally holds for script errors too, not
// just for Go panics.
//
// returns: (result, errorMessage) — both nil/"" on success.
const hookGuardSrc = `
function ` + hookGuardName + `(fn, payload)
local ok, res = pcall(fn, payload)
if not ok then
return nil, tostring(res)
end
return res, nil
end`
func registerHookGuard(L *golua.State) {
if err := L.DoString(hookGuardSrc); err != nil {
// Nothing useful to do here beyond leaving the global absent: invoke()
// checks for it and falls back to a direct call, which still works for
// correct plugins (only their ERRORS stop being survivable).
L.Pop(1)
}
}
func setupGlobals(L *golua.State) {
restorePcall(L)
buildJSONTable(L)
registerHookGuard(L)
registerFn(L, "hmac_sha256_hex", func(L *golua.State) int {
key := L.ToString(1)
@ -1104,6 +1210,56 @@ func pushGoValue(L *golua.State, v interface{}) {
}
func jsonEncode(v interface{}) ([]byte, error) { return json.Marshal(v) }
// luaToJSON converts the Lua value at idx into a Go value via json.encode, then
// unmarshals it into out. It is the bridge used by the plugin manifest/UI
// reader: the plugin returns a plain Lua table, and Go wants a typed struct.
//
// It goes through JSON rather than walking the Lua stack directly because the
// adapter/plugin boundary already speaks JSON everywhere else (transform_request
// gets a JSON string, hooks get a JSON string), so this keeps one representation
// instead of two.
func luaToJSON(L *golua.State, idx int, out interface{}) error {
if L.GetTop() < 1 {
return fmt.Errorf("empty stack")
}
abs := idx
if abs < 0 {
abs = L.GetTop() + 1 + abs
}
if abs < 1 || abs > L.GetTop() {
return fmt.Errorf("index %d out of range (top=%d)", idx, L.GetTop())
}
// Absolute indices throughout: this binding aborts the process (SIGABRT)
// on a bad index rather than panicking, so the stack is captured before any
// push instead of being addressed relative to a shifting top.
//
// json.encode is pushed onto the stack and the value is pushed AFTER it, so
// Call(1, 1) invokes it (Call takes no function index — it calls whatever
// sits below the nargs values).
L.GetGlobal("json")
if L.IsNil(-1) {
L.SetTop(0)
return fmt.Errorf("json global missing")
}
L.GetField(-1, "encode")
if L.Type(-1) != golua.LUA_TFUNCTION {
L.SetTop(0)
return fmt.Errorf("json.encode missing")
}
L.PushValue(abs)
if err := L.Call(1, 1); err != nil {
L.SetTop(0)
return err
}
if L.GetTop() < 1 || L.Type(-1) != golua.LUA_TSTRING {
L.SetTop(0)
return fmt.Errorf("json.encode did not return a string")
}
s := L.ToString(-1)
L.SetTop(0)
return json.Unmarshal([]byte(s), out)
}
func jsonDecode(s string) (interface{}, error) {
var v interface{}
if err := json.Unmarshal([]byte(s), &v); err != nil {

View File

@ -0,0 +1,270 @@
package provider
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"path/filepath"
"strings"
"sync"
"testing"
"llmsproxy/internal/config"
"llmsproxy/internal/lua"
"llmsproxy/internal/types"
)
// Gemini is the only adapter whose endpoint carries the model in the PATH
// (POST /v1beta/models/{model}:generateContent), so it is the only one that
// needs the substitution this file covers.
//
// The defect it fixes was silent: the adapter shipped endpoint="/v1/models",
// which is Gemini's model-LIST endpoint and answers 405 to POST, so the preset
// template produced a source that could never work.
func geminiVM(t *testing.T) *lua.VM {
t.Helper()
// The dir must be named "adapters": the VM seeds its bundled adapters into
// <dir> on Start and a differently-named dir leaves it empty, which surfaces
// as "adapter gemini not loaded" rather than an obvious setup error.
vm := lua.NewVM(filepath.Join(t.TempDir(), "adapters"))
if err := vm.Start(); err != nil {
t.Fatalf("lua vm: %v", err)
}
t.Cleanup(vm.Stop)
return vm
}
// geminiProvider builds a provider on the gemini adapter against base.
func geminiProvider(t *testing.T, base string, models ...string) *Provider {
t.Helper()
if len(models) == 0 {
models = []string{"gemini-3.6-flash"}
}
ms := make([]config.Model, 0, len(models))
for _, m := range models {
ms = append(ms, config.Model{ID: m, Kind: "chat"})
}
return New(config.Source{
Name: "gem",
BaseURL: base,
Adapter: "gemini",
APIKey: "k",
MaxConcurrent: 4,
Models: ms,
}, geminiVM(t))
}
// geminiRecorder captures the path the gateway actually requested and replies
// with a minimal Gemini-shaped payload.
type geminiRecorder struct {
mu sync.Mutex
paths []string
bodies []string
stream bool
}
func (g *geminiRecorder) handler(t *testing.T) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
raw, _ := io.ReadAll(r.Body)
g.mu.Lock()
// RequestURI, not URL.Path: URL.Path is the DECODED path, so an escaped
// %2F looks like a real "/" and the assertion could not tell an escaped
// id from an unescaped one.
g.paths = append(g.paths, r.RequestURI)
g.bodies = append(g.bodies, string(raw))
stream := g.stream
g.mu.Unlock()
if stream {
w.Header().Set("Content-Type", "text/event-stream")
w.WriteHeader(200)
fmt.Fprintf(w, "data: %s\n\n", `{"candidates":[{"content":{"parts":[{"text":"hi"}]},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":3,"candidatesTokenCount":1,"totalTokenCount":4}}`)
fmt.Fprintln(w, "data: [DONE]")
return
}
w.Header().Set("Content-Type", "application/json")
fmt.Fprint(w, `{"candidates":[{"content":{"parts":[{"text":"pong"}]},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":3,"candidatesTokenCount":1,"totalTokenCount":4}}`)
}
}
func (g *geminiRecorder) lastPath() string {
g.mu.Lock()
defer g.mu.Unlock()
if len(g.paths) == 0 {
return ""
}
return g.paths[len(g.paths)-1]
}
// TestGeminiEndpointCarriesModel: a non-streaming chat must POST to
// /v1beta/models/<model>:generateContent. The old static endpoint produced
// "/v1beta/v1/models" (a list endpoint, 405 on POST).
func TestGeminiEndpointCarriesModel(t *testing.T) {
rec := &geminiRecorder{}
srv := httptest.NewServer(rec.handler(t))
defer srv.Close()
p := geminiProvider(t, srv.URL, "gemini-3.6-flash")
if _, err := p.Chat(context.Background(), &types.ChatRequest{
Model: "gemini-3.6-flash",
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("hi")}},
}); err != nil {
t.Fatalf("chat: %v (path was %q)", err, rec.lastPath())
}
if got, want := rec.lastPath(), "/v1beta/models/gemini-3.6-flash:generateContent"; got != want {
t.Errorf("path = %q, want %q", got, want)
}
}
// TestGeminiStreamUsesStreamGenerateContent: the streaming verb lives in the
// path too, so it must be swapped for stream requests only.
func TestGeminiStreamUsesStreamGenerateContent(t *testing.T) {
rec := &geminiRecorder{stream: true}
srv := httptest.NewServer(rec.handler(t))
defer srv.Close()
p := geminiProvider(t, srv.URL, "gemini-3.6-flash")
ch, err := p.ChatStream(context.Background(), &types.ChatRequest{
Model: "gemini-3.6-flash",
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("hi")}},
})
if err != nil {
t.Fatalf("stream: %v (path was %q)", err, rec.lastPath())
}
for range ch {
}
if got, want := rec.lastPath(), "/v1beta/models/gemini-3.6-flash:streamGenerateContent"; got != want {
t.Errorf("path = %q, want %q", got, want)
}
}
// TestGeminiModelFollowsRequestNotSourceDefault is the part a naive fix gets
// wrong: substituting the source's default model would make every request on a
// multi-model source bill the wrong model. AUTO pins req.Model per slot, so the
// URL must follow the REQUEST's model.
func TestGeminiModelFollowsRequestNotSourceDefault(t *testing.T) {
rec := &geminiRecorder{}
srv := httptest.NewServer(rec.handler(t))
defer srv.Close()
p := geminiProvider(t, srv.URL, "gemini-3.6-flash", "gemini-3.1-pro")
for _, m := range []string{"gemini-3.1-pro", "gemini-3.6-flash"} {
if _, err := p.Chat(context.Background(), &types.ChatRequest{
Model: m,
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("hi")}},
}); err != nil {
t.Fatalf("chat %s: %v", m, err)
}
if want := "/v1beta/models/" + m + ":generateContent"; rec.lastPath() != want {
t.Errorf("path = %q, want %q", rec.lastPath(), want)
}
}
}
// TestGeminiModelIsEscaped: a model id reaches a URL path, so an id carrying a
// slash or space must be escaped rather than silently addressing a different
// resource (or producing a request Go's http client rejects).
func TestGeminiModelIsEscaped(t *testing.T) {
rec := &geminiRecorder{}
srv := httptest.NewServer(rec.handler(t))
defer srv.Close()
// Configure the odd id as a real model so ModelFor resolves it.
p := geminiProvider(t, srv.URL, "vendor/model:v1 beta")
if _, err := p.Chat(context.Background(), &types.ChatRequest{
Model: "vendor/model:v1 beta",
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("hi")}},
}); err != nil {
t.Fatalf("chat: %v", err)
}
got := rec.lastPath()
if strings.Contains(got, "/v1beta/models/vendor/model") {
t.Errorf("path %q was not escaped: the id's slash changed the resource", got)
}
if !strings.HasPrefix(got, "/v1beta/models/") || !strings.HasSuffix(got, ":generateContent") {
t.Errorf("path = %q, want an escaped id between the prefix and the verb", got)
}
}
// TestNonGeminiEndpointsAreUntouched: the substitution and the verb rewrite must
// be no-ops for every adapter whose endpoint is a fixed path. A regression here
// would silently rewrite OpenAI/Anthropic/… URLs.
func TestNonGeminiEndpointsAreUntouched(t *testing.T) {
for _, adapter := range []string{"openai", "deepseek", "anthropic", "ollama", "mistral", "trae", "sensenova"} {
vm := geminiVM(t)
p := New(config.Source{
Name: "s", BaseURL: "https://example.test", Adapter: adapter,
APIKey: "k", MaxConcurrent: 2,
Models: []config.Model{{ID: "m1", Kind: "chat"}},
}, vm)
for _, stream := range []bool{false, true} {
got := p.ChatURL("m1", stream)
want := "https://example.test" + p.Endpoint()
if got != want {
t.Errorf("adapter %s stream=%v: URL = %q, want %q", adapter, stream, got, want)
}
if strings.Contains(got, "streamGenerateContent") {
t.Errorf("adapter %s: streaming verb leaked into a fixed endpoint: %q", adapter, got)
}
}
vm.Stop()
}
}
// TestSourceEndpointOverridesTemplate: a source that spells its own endpoint
// must win outright — including when it spells one WITHOUT the placeholder.
func TestSourceEndpointOverridesTemplate(t *testing.T) {
vm := geminiVM(t)
defer vm.Stop()
p := New(config.Source{
Name: "s", BaseURL: "https://example.test", Adapter: "gemini",
Endpoint: "/custom/path", APIKey: "k", MaxConcurrent: 2,
Models: []config.Model{{ID: "m1", Kind: "chat"}},
}, vm)
if got, want := p.ChatURL("m1", false), "https://example.test/custom/path"; got != want {
t.Errorf("URL = %q, want %q (source endpoint must override the adapter template)", got, want)
}
// And the streaming verb rewrite must not damage an unrelated path.
if got := p.ChatURL("m1", true); strings.Contains(got, "streamGenerate") {
t.Errorf("streaming rewrite leaked into a user-spelled path: %q", got)
}
}
// TestGeminiResponseStillParses: the endpoint fix must not disturb the body
// transform — a request that reaches the right URL still has to come back as a
// unified response with Gemini's usageMetadata mapped.
func TestGeminiResponseStillParses(t *testing.T) {
rec := &geminiRecorder{}
srv := httptest.NewServer(rec.handler(t))
defer srv.Close()
p := geminiProvider(t, srv.URL, "gemini-3.6-flash")
resp, err := p.Chat(context.Background(), &types.ChatRequest{
Model: "gemini-3.6-flash",
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("hi")}},
})
if err != nil {
t.Fatalf("chat: %v", err)
}
if resp.Content != "pong" {
t.Errorf("content = %q, want %q", resp.Content, "pong")
}
if resp.TokenUsage.Prompt != 3 || resp.TokenUsage.Completion != 1 || resp.TokenUsage.Total != 4 {
t.Errorf("usage = %+v, want prompt 3 / completion 1 / total 4", resp.TokenUsage)
}
// The body the adapter produced must still be Gemini-native.
rec.mu.Lock()
body := rec.bodies[0]
rec.mu.Unlock()
var sent map[string]interface{}
if err := json.Unmarshal([]byte(body), &sent); err != nil {
t.Fatalf("adapter sent invalid json: %v (%s)", err, body)
}
if _, ok := sent["contents"]; !ok {
t.Errorf("adapter did not convert to Gemini's contents[]: %s", body)
}
}

View File

@ -531,7 +531,9 @@ func isAutoID(s string) bool {
return s == "" || strings.EqualFold(s, "AUTO")
}
// Endpoint resolves the upstream chat path.
// Endpoint resolves the upstream chat path. It may contain the placeholder
// "{model}"; use URL (or ChatURL) rather than calling this directly when the
// path has to be usable.
func (p *Provider) Endpoint() string {
if p.cfg.Endpoint != "" {
return p.cfg.Endpoint
@ -553,8 +555,40 @@ func (p *Provider) ImageEndpoint() string {
return "/v1/images/generations"
}
// modelPlaceholder is the marker an adapter puts in its endpoint template when
// the upstream carries the model id in the PATH rather than in the body.
// Gemini is the only such adapter: POST /v1beta/models/{model}:generateContent.
// Every other adapter's endpoint is a fixed path, so substituting is a no-op
// for them.
const modelPlaceholder = "{model}"
// ChatURL resolves the full chat URL for a specific model.
//
// Two substitutions happen here, both driven by the adapter's endpoint template:
//
// - "{model}" is replaced by the model id actually being sent. Without this
// the gateway would POST to Gemini's model-LIST endpoint, which answers 405.
// - for a streaming request, the ":generateContent" verb becomes
// ":streamGenerateContent". Gemini streams over the same path with a
// different verb, and the verb is part of the path, so the two cannot both
// be static. The rewrite is deliberately narrow: it only fires on the exact
// ":generateContent" suffix, so an adapter whose endpoint merely mentions
// the word keeps its path untouched.
func (p *Provider) ChatURL(model string, stream bool) string {
ep := p.Endpoint()
if strings.Contains(ep, modelPlaceholder) {
// A model id is put in a URL path, so it must be escaped: an id with a
// slash would otherwise silently address a different resource.
ep = strings.ReplaceAll(ep, modelPlaceholder, url.PathEscape(model))
}
if stream {
ep = strings.Replace(ep, ":generateContent", ":streamGenerateContent", 1)
}
return strings.TrimRight(p.cfg.BaseURL, "/") + ep
}
func (p *Provider) URL() string {
return strings.TrimRight(p.cfg.BaseURL, "/") + p.Endpoint()
return p.ChatURL("", false)
}
func (p *Provider) ImageURL() string {
@ -750,10 +784,11 @@ func (p *Provider) probeChat(ctx context.Context) (bool, string) {
body, err := json.Marshal(probe)
if err == nil {
var hdr http.Header
if hdrs, herr := p.buildHeaders(string(body), p.URL(), ""); herr == nil {
probeURL := p.ChatURL(model, false)
if hdrs, herr := p.buildHeaders(string(body), probeURL, ""); herr == nil {
hdr = hdrs
}
raw, status, derr := p.do(ctx, p.URL(), string(body), hdr)
raw, status, derr := p.do(ctx, probeURL, string(body), hdr)
if derr != nil {
msg = derr.Error()
} else if status == 200 {
@ -1084,11 +1119,12 @@ func (p *Provider) Chat(ctx context.Context, req *types.ChatRequest) (*types.Uni
if err != nil {
return nil, err
}
hdrs, err := p.buildHeaders(body, p.URL(), req.ClientSession)
chatURL := p.ChatURL(model, false)
hdrs, err := p.buildHeaders(body, chatURL, req.ClientSession)
if err != nil {
return nil, err
}
raw, status, err := p.do(ctx, p.URL(), body, hdrs)
raw, status, err := p.do(ctx, chatURL, body, hdrs)
if err != nil {
// a client disconnect or cancelled context is neither a success nor
// a failure for scheduling purposes — only upstream errors count
@ -1144,7 +1180,10 @@ func (p *Provider) ChatStream(ctx context.Context, req *types.ChatRequest) (<-ch
p.Release()
return nil, err
}
hdrs, err := p.buildHeaders(body, p.URL(), req.ClientSession)
// stream=true so a path-carried model plus the streaming verb is resolved
// for THIS request's model, not the source default.
streamURL := p.ChatURL(model, true)
hdrs, err := p.buildHeaders(body, streamURL, req.ClientSession)
if err != nil {
p.Release()
return nil, err
@ -1156,7 +1195,7 @@ func (p *Provider) ChatStream(ctx context.Context, req *types.ChatRequest) (<-ch
}
rc := make(chan respOrErr, 1)
go func() {
resp, err := p.doRawStream(ctx, p.URL(), body, hdrs)
resp, err := p.doRawStream(ctx, streamURL, body, hdrs)
rc <- respOrErr{resp, err}
}()

View File

@ -0,0 +1,127 @@
package scheduler
import (
"context"
"encoding/json"
"strings"
"testing"
"time"
"llmsproxy/internal/types"
)
// TestResultIsEmptyMatchesTheReasoningOnlyResponse guards the production bug
// behind "model AUTO returned a completed response with no content".
//
// Production measurement: 2 of 20 AUTO requests returned zero content, both
// served by claude-opus-4-8. Direct upstream capture showed 28 chunks of
// `reasoning_content` and NO `content`, finish_reason=length — the token budget
// was consumed by thinking before any text was emitted. runTier accepted it
// because err == nil.
func TestResultIsEmptyMatchesTheReasoningOnlyResponse(t *testing.T) {
// Exactly what the upstream returned, as a decoded response.
onlyReasoning := &types.UnifiedResponse{
Model: "claude-opus-4-8",
ReasoningContent: "Alright, the user just said \"hi\". Simple greeting…",
FinishReason: "length",
TokenUsage: types.TokenUsage{Prompt: 14, Completion: 30, Total: 44},
}
if !resultIsEmpty(onlyReasoning) {
t.Error("a reasoning-only response with usage must count as empty; " +
"it is what the client sees as \"completed with no content\"")
}
// The same response WITH text is a normal answer.
if resultIsEmpty(&types.UnifiedResponse{Content: "Hi! 👋", ReasoningContent: "hmm"}) {
t.Error("a response with content is not empty")
}
// A tool-calling agent turn has no text and is perfectly valid.
if resultIsEmpty(&types.UnifiedResponse{
ToolCalls: []types.ToolCall{{Name: "read_file", Arguments: map[string]interface{}{"path": "x"}}},
}) {
t.Error("a tool-call-only response must NOT count as empty — agents " +
"legitimately produce tool calls with no text")
}
// An image slot returns no text by design.
if resultIsEmpty(&types.UnifiedResponse{
ImageData: []types.ImageData{{URL: "http://x/y.png"}},
}) {
t.Error("an image response must NOT count as empty")
}
// Whitespace-only content is as useless to a client as none at all.
if !resultIsEmpty(&types.UnifiedResponse{Content: " \n\t"}) {
t.Error("whitespace-only content must count as empty")
}
if resultIsEmpty(nil) {
t.Error("nil is not an empty response; it is an absent one")
}
}
// TestPeekStreamHoldsReasoningAndReportsEmpty is the streaming half: a channel
// of reasoning-only chunks must be reported as empty so chainDrive degrades.
func TestPeekStreamHoldsReasoningAndReportsEmpty(t *testing.T) {
in := make(chan types.UnifiedChunk, 8)
for i := 0; i < 5; i++ {
in <- types.UnifiedChunk{ReasoningContent: "thinking…"}
}
in <- types.UnifiedChunk{Done: true, FinishReason: "length"}
close(in)
_, peek := peekStream(in)
if peek() {
t.Error("a reasoning-only stream must be reported empty")
}
}
// TestPeekStreamForwardsContentAfterReasoning: the common case still works,
// and the reasoning preamble is not forwarded ahead of the content (holding it
// back is what keeps the degrade path available).
func TestPeekStreamForwardsContentAfterReasoning(t *testing.T) {
in := make(chan types.UnifiedChunk, 8)
in <- types.UnifiedChunk{ReasoningContent: "let me think"}
in <- types.UnifiedChunk{Content: "Hi"}
in <- types.UnifiedChunk{Content: "!"}
in <- types.UnifiedChunk{Done: true, FinishReason: "stop"}
close(in)
out, peek := peekStream(in)
if !peek() {
t.Fatal("a stream with content must be reported as having content")
}
var got []string
for ck := range out {
if ck.Content != "" {
got = append(got, ck.Content)
}
if strings.TrimSpace(ck.ReasoningContent) != "" {
t.Error("reasoning preamble must not be forwarded before content; " +
"that is what pins the client to a stream it cannot escape")
}
}
if strings.Join(got, "") != "Hi!" {
t.Errorf("forwarded content = %q, want %q", got, "Hi!")
}
}
// TestPeekStreamDoesNotDeadlockOnToolCalls: a tool-call delta is content for
// this purpose and must unblock peek immediately.
func TestPeekStreamDoesNotDeadlockOnToolCalls(t *testing.T) {
raw, _ := json.Marshal([]types.ToolCall{{Name: "ls"}})
in := make(chan types.UnifiedChunk, 4)
in <- types.UnifiedChunk{ToolCalls: raw}
in <- types.UnifiedChunk{Done: true, FinishReason: "tool_calls"}
close(in)
_, peek := peekStream(in)
done := make(chan bool, 1)
go func() { done <- peek() }()
select {
case ok := <-done:
if !ok {
t.Error("a tool-call delta must count as content")
}
case <-time.After(2 * time.Second):
t.Fatal("peek blocked on a tool-call-only stream")
}
_ = context.Background()
}

View File

@ -11,6 +11,7 @@ import (
"fmt"
"sort"
"strings"
"sync"
"sync/atomic"
"time"
@ -245,6 +246,115 @@ func normalCount(cands []candidate) int {
//
// Normal candidates rotate by base; probe candidates form a fixed tail tried
// only after every normal slot failed or was busy.
// emptyResultReason describes why a 200-with-no-content response counts as a
// slot failure for AUTO.
//
// WHY: reasoning models (claude-opus-*, codebuddy_glm-*, …) emit
// `reasoning_content` first and only then `content`. When the caller's
// max_tokens is small enough that the thinking phase consumes the whole budget,
// upstream returns 200 / finish_reason=length with 28 chunks of reasoning and
// ZERO content. runTier treated "err == nil" as success and handed that to the
// client, which then failed with "returned a completed response with no
// content" — a client-side error message for what is really a bad slot choice.
//
// So an empty result is a SLOT failure, not a request failure: the gateway
// degrades to the next slot and the user still gets an answer. Measured on
// production AUTO: 2 of 20 requests returned empty content, all of them
// claude-opus-4-8.
//
// A response carrying tool_calls or image data is NOT empty: an agent turn
// legitimately produces tool calls with no text. ReasoningContent does NOT
// rescue it either — see resultIsEmpty.
const emptyResultReason = "upstream returned no content (reasoning-only response, or the token budget was consumed before any text)"
// resultIsEmpty reports whether a successful-but-useless response should be
// treated as a slot failure.
//
// Image data counts: an image-generation slot legitimately returns no text.
// A usage-only response is NOT empty either — the upstream answered, it just
// said nothing, and that is exactly the case worth degrading away from.
func resultIsEmpty(resp *types.UnifiedResponse) bool {
if resp == nil {
return false
}
// NOTE: ReasoningContent is deliberately NOT consulted. My first version
// excluded it ("the model was thinking, that is an answer"), and the test
// built from the real production capture failed immediately: the captured
// response is exactly reasoning_content-with-usage and zero text. The
// client asked for text and there is none; holding a request hostage to
// another model's thinking phase is strictly worse than degrading.
return strings.TrimSpace(resp.Content) == "" &&
len(resp.ToolCalls) == 0 &&
len(resp.ImageData) == 0
}
// emptyStreamReason is resultIsEmpty's streaming twin; see emptyResultReason
// for why an empty response is a slot failure rather than a request failure.
const emptyStreamReason = emptyResultReason
// peekStream wraps a chunk channel so the caller learns whether the stream
// produced real content BEFORE the chunks are forwarded.
//
// Why this is necessary: reasoning models emit reasoning_content first. With a
// small max_tokens the whole budget is spent thinking, the stream ends with
// finish_reason=length and zero content. If the gateway forwarded those chunks
// as they arrived, the client would already have seen a 200 SSE stream and
// could not be given a different slot — its only recourse is the useless
// "returned a completed response with no content" error. Buffering until the
// first real content (or the end of the stream) keeps the degrade path
// available at the cost of holding back the first few chunks.
//
// What is NOT buffered: the wrapper starts forwarding as soon as a chunk with
// non-empty Content or ToolCalls arrives, and keeps forwarding everything from
// then on, so only the reasoning preamble is held. Reasoning-only responses
// are dropped in full and reported as empty, which lets chainDrive try the
// next slot.
func peekStream(in <-chan types.UnifiedChunk) (<-chan types.UnifiedChunk, func() bool) {
out := make(chan types.UnifiedChunk, 16)
var (
mu sync.Mutex
sawText bool
done bool
)
go func() {
defer close(out)
started := false
for ck := range in {
if !started {
// Hold back the reasoning / usage-only preamble. A tool-call
// delta counts as content: an agent turn legitimately emits
// tool_calls with no text.
if strings.TrimSpace(ck.Content) == "" && len(ck.ToolCalls) == 0 {
continue
}
started = true
mu.Lock()
sawText = true
mu.Unlock()
}
out <- ck
}
mu.Lock()
done = true
mu.Unlock()
}()
// peek blocks until the stream either produces content or ends, then
// reports whether any content was seen. Polling a 2ms tick rather than
// using a second channel keeps peekStream single-goroutine and leak-free.
peek := func() bool {
for {
mu.Lock()
seen, finished := sawText, done
mu.Unlock()
if seen || finished {
return seen
}
time.Sleep(2 * time.Millisecond)
}
}
return out, peek
}
func runTier(ctx context.Context, tn *TierNode, cands []candidate, base int64, req *types.ChatRequest, stream bool) tierResult {
n := len(cands)
norm := normalCount(cands)
@ -267,7 +377,18 @@ func runTier(ctx context.Context, tn *TierNode, cands []candidate, base int64, r
if stream {
chunks, err := sl.Prov.ChatStream(ctx, &r)
if err == nil {
return tierResult{chunks: chunks, src: sl.Source, model: sl.Model}
guarded, peek := peekStream(chunks)
if peek() {
return tierResult{chunks: guarded, src: sl.Source, model: sl.Model}
}
// The stream finished with no content at all: a
// reasoning-only response. Drain and move on to the next
// slot instead of pinning the client to a useless stream.
hard = append(hard, TierError{
Tier: tn.Tier, Source: sl.Source, Model: sl.Model,
Err: errors.New(emptyStreamReason),
})
continue
}
if ctx.Err() != nil {
return tierResult{}
@ -280,6 +401,17 @@ func runTier(ctx context.Context, tn *TierNode, cands []candidate, base int64, r
}
resp, err := sl.Prov.Chat(ctx, &r)
if err == nil {
if resultIsEmpty(resp) {
// Soft failure: record it and try the next slot. Deliberately
// NOT a hard TierError — a hard error is reported to the client
// verbatim when the whole chain fails, and "this one model was
// unhelpful" is not the client's problem to debug.
hard = append(hard, TierError{
Tier: tn.Tier, Source: sl.Source, Model: sl.Model,
Err: errors.New(emptyResultReason),
})
continue
}
return tierResult{resp: resp, src: sl.Source, model: sl.Model}
}
if ctx.Err() != nil {
@ -293,13 +425,75 @@ func runTier(ctx context.Context, tn *TierNode, cands []candidate, base int64, r
return tierResult{hard: hard}
}
// TraceKind classifies one step of an AUTO chain walk.
type TraceKind string
const (
// TraceTierSkip: the whole tier was skipped — every slot was cooling,
// quota-exhausted, or none was schedulable. Reason says which.
TraceTierSkip TraceKind = "tier_skip"
// TraceSlotFail: one slot failed hard (upstream error / bad adapter). The
// walk continues to the next slot or tier.
TraceSlotFail TraceKind = "slot_fail"
// TraceTierBusy: the tier was fully busy and the bounded wait expired.
TraceTierBusy TraceKind = "tier_busy"
// TraceSelected: this slot served the request. Exactly one per successful
// chain walk, and the last event emitted.
TraceSelected TraceKind = "selected"
)
// TraceEvent is one observable step of an AUTO chain walk.
//
// WHY THIS EXISTS: chainDrive's return value is (resp, src, model, err), so a
// caller learns only which slot finally served the request. Everything the
// scheduler decided on the way there — which tiers it skipped and WHY, which
// slots hard-failed, whether a tier was merely busy — was computed and then
// discarded. That is invisible to operators and to plugins: "tier 1 was cooling
// so we degraded to tier 3" looked exactly like "tier 1 served it".
//
// The walk already accumulates this in ChainErr, but ONLY on total failure, and
// ChainErr is an error return, not a record. Emitting a trace as it happens
// covers the far more common case: a request that SUCCEEDED after degrading.
//
// Design constraints:
// - scheduler stays dependency-free and independently testable. A TraceEvent
// is a plain struct in this package and the sink is a func parameter, so no
// import is added and no test has to change to observe a walk.
// - The sink is optional (nil = emit nothing). The overhead on the hot path
// is one nil check per event.
// - Events are OBSERVATION ONLY. Nothing in the scheduler branches on them,
// and the gateway does not feed them back into routing, cooldown or quota —
// see docs/plugins.md for why accounting and enforcement are kept apart.
type TraceEvent struct {
Kind TraceKind
Tier int
Source string
Model string
Reason string // human-readable, for TraceTierSkip / TraceSlotFail
Err string // the underlying error text, for TraceSlotFail
// Attempt counts the 1-based slot attempt within the whole walk.
Attempt int
}
// TraceSink receives chain-walk events. It must not block: it is called from the
// request path, and a slow sink slows the request.
type TraceSink func(TraceEvent)
// chainDrive runs a request down the chain (plan 2.3): tiers ascending (tier
// 1, the highest priority, first), per-tier round-robin starting at the tier
// cursor, same-tier runs ordered by preference (negative prefs sink but stay
// reachable). Quota-exhausted and cooling slots are filtered up front; a
// fully busy tier is polled for a bounded time before falling through.
// Failures are summarized in *ChainErr for the caller to map to HTTP 503.
func (s *Scheduler) chainDrive(ctx context.Context, chain *Chain, req *types.ChatRequest, exhausted func(*Slot) bool, stream bool) (*types.UnifiedResponse, <-chan types.UnifiedChunk, string, string, error) {
//
// trace may be nil; when set it receives one event per observable step.
func (s *Scheduler) chainDrive(ctx context.Context, chain *Chain, req *types.ChatRequest, exhausted func(*Slot) bool, stream bool, trace TraceSink) (*types.UnifiedResponse, <-chan types.UnifiedChunk, string, string, error) {
emit := func(ev TraceEvent) {
if trace != nil {
trace(ev)
}
}
attempt := 0
if chain == nil || len(chain.Tiers) == 0 {
return nil, nil, "", "", fmt.Errorf("no auto slot configured")
}
@ -309,7 +503,9 @@ func (s *Scheduler) chainDrive(ctx context.Context, chain *Chain, req *types.Cha
// dropped unless they qualify as half-cooldown probes (appended last).
cands := collectCands(tn.Slots, exhausted)
if len(cands) == 0 {
ce.Skipped = append(ce.Skipped, fmt.Sprintf("tier %d: no schedulable slot (cooling or quota exhausted)", tn.Tier))
reason := "no schedulable slot (cooling or quota exhausted)"
ce.Skipped = append(ce.Skipped, fmt.Sprintf("tier %d: %s", tn.Tier, reason))
emit(TraceEvent{Kind: TraceTierSkip, Tier: tn.Tier, Reason: reason})
continue
}
// No Pref sort: load balancing is done by round-robin cursor.
@ -318,6 +514,8 @@ func (s *Scheduler) chainDrive(ctx context.Context, chain *Chain, req *types.Cha
base := tn.NextStart()
res := runTier(ctx, tn, cands, base, req, stream)
if res.resp != nil || res.chunks != nil {
attempt++
emit(TraceEvent{Kind: TraceSelected, Tier: tn.Tier, Source: res.src, Model: res.model, Attempt: attempt})
releaseProbes(cands)
return res.resp, res.chunks, res.src, res.model, nil
}
@ -327,6 +525,13 @@ func (s *Scheduler) chainDrive(ctx context.Context, chain *Chain, req *types.Cha
}
if len(res.hard) > 0 {
ce.Tiers = append(ce.Tiers, res.hard...)
for _, h := range res.hard {
attempt++
emit(TraceEvent{
Kind: TraceSlotFail, Tier: tn.Tier, Source: h.Source, Model: h.Model,
Err: types.OneLine(h.Err.Error(), 200), Attempt: attempt,
})
}
releaseProbes(cands)
continue // hard failures: fall through to the next tier, no waiting
}
@ -334,10 +539,13 @@ func (s *Scheduler) chainDrive(ctx context.Context, chain *Chain, req *types.Cha
if err := s.pollBusyTier(ctx, tn, cands, base, req, stream, &ce); err != nil {
releaseProbes(cands)
if r, ok := err.(*tierSuccess); ok {
attempt++
emit(TraceEvent{Kind: TraceSelected, Tier: tn.Tier, Source: r.res.src, Model: r.res.model, Attempt: attempt})
return r.res.resp, r.res.chunks, r.res.src, r.res.model, nil
}
return nil, nil, "", "", err
}
emit(TraceEvent{Kind: TraceTierBusy, Tier: tn.Tier, Reason: fmt.Sprintf("no free slot within %v", busyWait)})
releaseProbes(cands)
}
if len(ce.Tiers) == 0 && len(ce.Skipped) == 0 {
@ -401,16 +609,16 @@ func (s *Scheduler) pollBusyTier(ctx context.Context, tn *TierNode, cands []cand
// non-nil, decides slot token-quota exhaustion. Returns the response, the
// serving source and the exact model id used; on total failure a *ChainErr
// summarizing every tier.
func (s *Scheduler) ChainChat(ctx context.Context, chain *Chain, req *types.ChatRequest, exhausted func(*Slot) bool) (*types.UnifiedResponse, string, string, error) {
resp, _, src, model, err := s.chainDrive(ctx, chain, req, exhausted, false)
func (s *Scheduler) ChainChat(ctx context.Context, chain *Chain, req *types.ChatRequest, exhausted func(*Slot) bool, trace TraceSink) (*types.UnifiedResponse, string, string, error) {
resp, _, src, model, err := s.chainDrive(ctx, chain, req, exhausted, false, trace)
return resp, src, model, err
}
// ChainChatStream runs a streaming AUTO request down the chain. A slot is
// abandoned only on connect failures / busy (before its first chunk); after a
// stream starts it is pinned. Same return contract as ChainChat.
func (s *Scheduler) ChainChatStream(ctx context.Context, chain *Chain, req *types.ChatRequest, exhausted func(*Slot) bool) (<-chan types.UnifiedChunk, string, string, error) {
_, chunks, src, model, err := s.chainDrive(ctx, chain, req, exhausted, true)
func (s *Scheduler) ChainChatStream(ctx context.Context, chain *Chain, req *types.ChatRequest, exhausted func(*Slot) bool, trace TraceSink) (<-chan types.UnifiedChunk, string, string, error) {
_, chunks, src, model, err := s.chainDrive(ctx, chain, req, exhausted, true, trace)
return chunks, src, model, err
}

View File

@ -176,7 +176,7 @@ func TestChainRoundRobin(t *testing.T) {
s := New(0)
var got []string
for i := 0; i < 4; i++ {
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
if err != nil {
t.Fatalf("iter %d: %v", i, err)
}
@ -200,7 +200,7 @@ func TestChainPreferenceSinksButStaysReachable(t *testing.T) {
{Tier: 0, Model: "g", Source: "good"},
}, bySource(neg, good))
s := New(0)
resp, src, model, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
resp, src, model, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
if err != nil {
t.Fatalf("chain: %v", err)
}
@ -213,7 +213,7 @@ func TestChainPreferenceSinksButStaysReachable(t *testing.T) {
t.Fatal("neg was tried first and succeeded; good must not be attempted")
}
// Second request: cursor advances. neg wins again (good hard-fails).
resp2, src2, _, err2 := s.ChainChat(context.Background(), ch, chatReq(), nil)
resp2, src2, _, err2 := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
if err2 != nil {
t.Fatalf("second chain: %v", err2)
}
@ -230,7 +230,7 @@ func TestChainBusySkipsWithoutPenalty(t *testing.T) {
{Tier: 0, Model: "b", Source: "s2"},
}, bySource(a, b))
s := New(0)
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
if err != nil {
t.Fatalf("chain: %v", err)
}
@ -256,7 +256,7 @@ func TestChainAllBusyBoundedWaitThenNextTier(t *testing.T) {
}, bySource(a, b, c))
s := New(0)
t0 := time.Now()
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
el := time.Since(t0)
if err != nil {
t.Fatalf("chain: %v", err)
@ -277,7 +277,7 @@ func TestChainQuotaExhausted(t *testing.T) {
}, bySource(a, b))
s := New(0)
exhausted := func(sl *Slot) bool { return sl.Source == "s1" && sl.Quota > 0 }
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), exhausted)
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), exhausted, nil)
if err != nil {
t.Fatalf("chain: %v", err)
}
@ -300,7 +300,7 @@ func TestChainErrSummary(t *testing.T) {
{Tier: 1, Model: "c", Source: "s3"},
}, bySource(a, b, c))
s := New(0)
_, _, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
_, _, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
var ce *ChainErr
if !errors.As(err, &ce) {
t.Fatalf("err = %v, want *ChainErr", err)
@ -325,7 +325,7 @@ func TestChainStreamFallsBackBeforeFirstChunk(t *testing.T) {
{Tier: 0, Model: "b", Source: "s2"},
}, bySource(a, b))
s := New(0)
chunks, src, model, err := s.ChainChatStream(context.Background(), ch, chatReq(), nil)
chunks, src, model, err := s.ChainChatStream(context.Background(), ch, chatReq(), nil, nil)
if err != nil {
t.Fatalf("chain stream: %v", err)
}
@ -365,7 +365,7 @@ func TestChainProbeIsLastResort(t *testing.T) {
}, bySource(healthy, cooling))
s := New(0)
for i := 0; i < 3; i++ {
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
if err != nil {
t.Fatalf("iter %d: %v", i, err)
}
@ -390,7 +390,7 @@ func TestChainProbeServesWhenNothingElseCan(t *testing.T) {
cooling.probeable.Store(true)
ch := BuildChain([]Rule{{Tier: 0, Model: "c", Source: "cooling"}}, bySource(cooling))
s := New(0)
resp, src, model, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
resp, src, model, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
if err != nil {
t.Fatalf("probe must serve the request: %v", err)
}
@ -417,14 +417,14 @@ func TestChainProbePermitReleasedOnFailure(t *testing.T) {
cooling.fail.Store(true)
ch := BuildChain([]Rule{{Tier: 0, Model: "c", Source: "cooling"}}, bySource(cooling))
s := New(0)
if _, _, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil); err == nil {
if _, _, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil); err == nil {
t.Fatal("expected the failing probe to surface an error")
}
if cooling.probeClaims.Load() != 1 || cooling.probeDones.Load() != 1 {
t.Fatalf("permit accounting: claims=%d dones=%d, want 1/1", cooling.probeClaims.Load(), cooling.probeDones.Load())
}
// permit is free again for the next attempt
if _, _, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil); err == nil {
if _, _, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil); err == nil {
t.Fatal("expected the second probe to fail too")
}
if cooling.probeClaims.Load() != 2 {
@ -443,7 +443,7 @@ func TestChainProbeDoesNotBlockTierFallthrough(t *testing.T) {
{Tier: 2, Model: "b", Source: "backup"},
}, bySource(cold, backup))
s := New(0)
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
if err != nil || src != "backup" {
t.Fatalf("want fallthrough to backup, got src=%q err=%v", src, err)
}
@ -482,7 +482,7 @@ func TestChainProbeRoundRobinUnaffected(t *testing.T) {
s := New(0)
var got []string
for i := 0; i < 4; i++ {
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
if err != nil {
t.Fatalf("iter %d: %v", i, err)
}

View File

@ -0,0 +1,191 @@
package scheduler
import (
"context"
"errors"
"strings"
"testing"
)
// The chain trace is the only way a caller learns that a request was DEGRADED
// — served by a lower tier than the one that should have taken it. chainDrive's
// return value carries only the winner, so without these events "tier 1 was
// cooling and we dropped to tier 2" is indistinguishable from "tier 1 served
// it", which is the exact question a priority chain exists to answer.
// recorder collects trace events for assertions.
type recorder struct{ events []TraceEvent }
func (r *recorder) sink(ev TraceEvent) { r.events = append(r.events, ev) }
func (r *recorder) kinds() []TraceKind {
out := make([]TraceKind, 0, len(r.events))
for _, e := range r.events {
out = append(out, e.Kind)
}
return out
}
func (r *recorder) find(k TraceKind) *TraceEvent {
for i := range r.events {
if r.events[i].Kind == k {
return &r.events[i]
}
}
return nil
}
// TestTraceSelectedOnlyOnHappyPath: a clean walk emits exactly one event.
func TestTraceSelectedOnlyOnHappyPath(t *testing.T) {
p1 := fakeProv("p1", "m1")
ch := BuildChain([]Rule{{Model: "m1", Source: "p1", Tier: 1}},
func(model, source string) Provider { return p1 })
var rec recorder
_, _, _, err := New(3).ChainChat(context.Background(), ch, chatReq(), nil, rec.sink)
if err != nil {
t.Fatalf("chain: %v", err)
}
if got := rec.kinds(); len(got) != 1 || got[0] != TraceSelected {
t.Errorf("events = %v, want a single selected", got)
}
e := rec.find(TraceSelected)
if e.Tier != 1 || e.Source != "p1" || e.Model != "m1" {
t.Errorf("selected event = %+v, want tier 1 / p1 / m1", e)
}
if e.Attempt != 1 {
t.Errorf("Attempt = %d, want 1", e.Attempt)
}
}
// TestTraceRecordsTierSkipAndDegradation is the core case: tier 1 is
// unschedulable, tier 2 answers. The trace must show the skip AND the eventual
// selection, so a consumer can see the request was served one tier down.
func TestTraceRecordsTierSkipAndDegradation(t *testing.T) {
// p1 is unavailable (not probeable), so tier 1 yields no candidates.
p1 := fakeProv("p1", "m1")
p1.available.Store(false)
p2 := fakeProv("p2", "m2")
ch := BuildChain([]Rule{
{Model: "m1", Source: "p1", Tier: 1},
{Model: "m2", Source: "p2", Tier: 2},
}, bySource(p1, p2))
var rec recorder
_, src, model, err := New(3).ChainChat(context.Background(), ch, chatReq(), nil, rec.sink)
if err != nil {
t.Fatalf("chain: %v", err)
}
if src != "p2" || model != "m2" {
t.Fatalf("served by %s/%s, want p2/m2", src, model)
}
skip := rec.find(TraceTierSkip)
if skip == nil {
t.Fatalf("no tier_skip event; events = %v", rec.kinds())
}
if skip.Tier != 1 {
t.Errorf("skip tier = %d, want 1", skip.Tier)
}
if !strings.Contains(skip.Reason, "cooling") {
t.Errorf("skip reason = %q, want it to mention cooling", skip.Reason)
}
sel := rec.find(TraceSelected)
if sel == nil || sel.Tier != 2 {
t.Errorf("selected = %+v, want tier 2", sel)
}
// The order matters: the skip must be observable BEFORE the selection.
if rec.events[0].Kind != TraceTierSkip || rec.events[len(rec.events)-1].Kind != TraceSelected {
t.Errorf("event order = %v, want skip first and selected last", rec.kinds())
}
}
// TestTraceRecordsHardSlotFailures: a slot that returns an upstream error is a
// different event from a skip — the request tried it and it failed. Losing that
// distinction makes a flaky upstream look like an idle one.
func TestTraceRecordsHardSlotFailures(t *testing.T) {
p1 := fakeProv("p1", "m1")
p1.fail.Store(true) // Chat returns "upstream error"
p2 := fakeProv("p2", "m2")
ch := BuildChain([]Rule{
{Model: "m1", Source: "p1", Tier: 1},
{Model: "m2", Source: "p2", Tier: 2},
}, bySource(p1, p2))
var rec recorder
_, src, _, err := New(3).ChainChat(context.Background(), ch, chatReq(), nil, rec.sink)
if err != nil {
t.Fatalf("chain: %v", err)
}
if src != "p2" {
t.Fatalf("served by %s, want p2", src)
}
fail := rec.find(TraceSlotFail)
if fail == nil {
t.Fatalf("no slot_fail event; events = %v", rec.kinds())
}
if fail.Tier != 1 || fail.Source != "p1" || fail.Model != "m1" {
t.Errorf("slot_fail = %+v, want tier 1 / p1 / m1", fail)
}
if !strings.Contains(fail.Err, "upstream error") {
t.Errorf("slot_fail error = %q, want the upstream text", fail.Err)
}
// A hard failure must NOT be reported as a skip.
if rec.find(TraceTierSkip) != nil {
t.Error("a hard failure was also reported as a tier_skip")
}
}
// TestTraceNilSinkIsSafe: the gateway passes nil when no plugin is loaded, so
// every emit path must tolerate it. This is the "plugins are optional" property
// on the scheduler side.
func TestTraceNilSinkIsSafe(t *testing.T) {
p1 := fakeProv("p1", "m1")
p1.fail.Store(true)
p2 := fakeProv("p2", "m2")
ch := BuildChain([]Rule{
{Model: "m1", Source: "p1", Tier: 1},
{Model: "m2", Source: "p2", Tier: 2},
}, bySource(p1, p2))
if _, _, _, err := New(3).ChainChat(context.Background(), ch, chatReq(), nil, nil); err != nil {
t.Fatalf("a nil trace sink broke the walk: %v", err)
}
}
// TestTraceOnTotalFailure: when every tier fails, the walk still emits its
// per-step events AND returns the ChainErr. The trace is additive — it must not
// replace or disturb the error contract callers depend on for the 503.
func TestTraceOnTotalFailure(t *testing.T) {
p1 := fakeProv("p1", "m1")
p1.fail.Store(true)
p2 := fakeProv("p2", "m2")
p2.fail.Store(true)
ch := BuildChain([]Rule{
{Model: "m1", Source: "p1", Tier: 1},
{Model: "m2", Source: "p2", Tier: 2},
}, bySource(p1, p2))
var rec recorder
_, _, _, err := New(3).ChainChat(context.Background(), ch, chatReq(), nil, rec.sink)
var ce *ChainErr
if !errors.As(err, &ce) {
t.Fatalf("err = %v, want a *ChainErr so the gateway can answer 503", err)
}
if len(ce.Tiers) != 2 {
t.Errorf("ChainErr.Tiers = %d, want 2 (the error contract must be unchanged)", len(ce.Tiers))
}
if n := len(rec.kinds()); n != 2 {
t.Errorf("events = %v, want two slot_fail and no selection", rec.kinds())
}
if rec.find(TraceSelected) != nil {
t.Error("a selected event was emitted for a walk that served nothing")
}
}
// TestTraceSkipsEmptyChain: no chain configured must not emit anything; the
// gateway answers 503 before scheduling in that case anyway.
func TestTraceSkipsEmptyChain(t *testing.T) {
var rec recorder
_, _, _, err := New(3).ChainChat(context.Background(), &Chain{}, chatReq(), nil, rec.sink)
if err == nil {
t.Fatal("expected an error for an empty chain")
}
if len(rec.events) != 0 {
t.Errorf("events = %v, want none", rec.kinds())
}
}

View File

@ -13,13 +13,26 @@ listen: 127.0.0.1:8080
# 客户端访问本网关所需的 API Key(Bearer)。留空数组 = 不鉴权(仅内网)。
gateway_keys: []
# 默认模型选择:具体模型 id 或 AUTO(按各源模型的 priority 自动选最高可用源)
# 默认模型选择:具体模型 id 或 AUTO。
# AUTO = 走 WebUI「优先级页」保存的调度链(存在本文件的 `auto:` 字段)。
default_model: AUTO
# Lua 适配器目录(默认 adapters/,首次启动自动写入内置适配器)
adapter_dir: adapters
# 运行时持久化文件(WebUI 新增/编辑的源会写入此文件,重启后仍生效)
# 插件目录(可选)。设置后:
# - 首次启动会把随核心发布的示例插件(billing:按源/模型/密钥计费 + 仪表盘)
# 写入本目录,它会在请求流水线上挂钩子,并向 WebUI 注入一个「Billing」页面
# 和状态页上的一块总开销组件;
# - 之后 WebUI「插件」页与 Electron 壳的设置面板可安装/禁用/删除/编辑。
# 目录一旦存在即以目录为准:删除或改写内置插件都是真实生效的操作。
# 留空 = 插件功能完全关闭(不影响网关其它功能)。
# plugin_dir: /etc/llmsproxy/plugins
# 运行时文件:存放 WebUI 管理的源模板、已删除标记、预置模板名单。
#
# 注意:**AUTO 调度链、网关密钥、上游源都存在本 config.yaml 里**,
# 不在这个文件。runtime.json 只管模板与删除标记。
runtime_file: runtime.json
# 全局并发上限(0 = 不限)

View File

@ -77,6 +77,21 @@ cp "$ROOT/packaging/config.example.yaml" "$TAR_DIR/config.example.yaml"
cp "$ROOT/packaging/llmsproxy.service" "$TAR_DIR/llmsproxy.service"
mkdir -p "$TAR_DIR/adapters"
cp "$ROOT"/internal/lua/adapters/*.lua "$TAR_DIR/adapters/"
# The shipped plugins go into the archive too, not just into the binary's
# embedded FS.
#
# Seeding works without them (internal/lua/vm.go embeds plugins/*.lua and
# writeBundledPlugins materialises them on first start — verified: a fresh
# plugin_dir gains billing.lua, 70031 bytes, and the plugin loads), so this is
# not a functional fix. It is a contents/claim mismatch: the v1.8.0 release
# notes tell the operator the billing plugin ships with the core, and this
# archive — the artefact they unpack and inspect — did not contain it. Shipping
# the file also means an operator can read and adapt it before installing,
# instead of only discovering it after the first boot writes it out.
mkdir -p "$TAR_DIR/plugins"
cp "$ROOT"/internal/lua/plugins/*.lua "$TAR_DIR/plugins/"
tar -C "$DIST" -czf "$DIST/llmsproxy-$VERSION-linux-amd64.tar.gz" \
"$(basename "$TAR_DIR")"
rm -rf "$TAR_DIR"
@ -90,6 +105,45 @@ for f in "$DIST"/llmsproxy_*.deb "$DIST"/llmsproxy-*.rpm "$DIST"/*.tar.gz; do
[ "$sz" -gt 100000 ] || { echo "[core-dist] FATAL: $f suspiciously small ($sz B)"; exit 1; }
log " ✓ $f ($((sz/1024)) KB)"
done
dpkg-deb -I "$DIST"/llmsproxy_*.deb 2>/dev/null | grep -E 'Package|Version' | head -2
# dpkg-deb takes ONE archive; every extra argument is read as an additional
# control-component name. With the glob unquoted this used to expand to three
# .deb files (1.5.9 / 1.6.0 / the one just built) and dpkg-deb exited 2 — under
# `set -e` that aborted the script right here, so the "done" line never printed
# and NO content verification below ever ran. The size gate above was the only
# check that ever executed. Pick the newest deb explicitly.
newest_deb=$(ls -t "$DIST"/llmsproxy_*.deb 2>/dev/null | head -1 || true)
if [ -n "$newest_deb" ]; then
dpkg-deb -I "$newest_deb" 2>/dev/null | grep -E 'Package|Version' | head -2 || true
fi
# 6) the archive must actually carry what the release notes promise.
#
# A size floor alone cannot catch this: a tar.gz missing all 70KB of plugins
# still clears the 100KB threshold easily, and the seeded-on-first-boot
# behaviour hides it at runtime — the operator sees a working billing plugin and
# concludes the package is complete. Assert the file is in the archive.
log "verifying archive contents..."
TARBALL="$DIST/llmsproxy-$VERSION-linux-amd64.tar.gz"
[ -f "$TARBALL" ] || { echo "[core-dist] FATAL: $TARBALL missing"; exit 1; }
listing=$(tar tzf "$TARBALL" 2>/dev/null || true)
[ -n "$listing" ] || { echo "[core-dist] FATAL: cannot list $TARBALL"; exit 1; }
for want in llmsproxy config.example.yaml llmsproxy.service plugins/billing.lua; do
if ! printf '%s\n' "$listing" | grep -q "/$want\$"; then
echo "[core-dist] FATAL: archive is missing $want"
echo " (release notes claim the billing plugin ships with the core)"
exit 1
fi
log " ✓ archive contains $want"
done
# Directories are matched by their CONTENTS, not by name: tar written with
# `tar -czf` from a populated tree lists "…/adapters/openai.lua" and never a bare
# "…/adapters" entry, so an earlier version of this check that grepped for
# "/adapters$" reported a complete archive as broken.
n_adapters=$(printf '%s\n' "$listing" | grep -c "/adapters/.*\.lua\$" || true)
if [ "$n_adapters" -lt 10 ]; then
echo "[core-dist] FATAL: archive has only $n_adapters adapters (expected >= 10)"
exit 1
fi
log " ✓ archive contains $n_adapters adapters"
log "done: $DIST"

View File

@ -10,15 +10,53 @@ Type=simple
# OS thread that touches malloc reserved its own ~1 MB arena that is never
# returned. Measured: 8-12 arenas -> 0.
# GOGC=50 halves the Go heap growth target. On its own it does NOT help (the
# saved heap is immediately eaten by more glibc arenas); combined with
# saved heap is immediately eaten by extra glibc arenas); combined with
# MALLOC_ARENA_MAX it cut settled RSS by ~19%. This gateway is I/O bound, so
# the extra GC cycles are free.
Environment=GOGC=50
Environment=MALLOC_ARENA_MAX=2
ExecStart=/usr/bin/llmsproxy -config /etc/llmsproxy/config.yaml
ExecStart=/usr/local/bin/llmsproxy -config /etc/llmsproxy/config.yaml
WorkingDirectory=/etc/llmsproxy
Restart=always
RestartSec=5
# ---- 加固(2026-10-01 逐条实测后加入,不是照抄文档)----
#
# 为什么仍然以 root 运行:master.key 是 0600 root。加 User=llmsproxy 实测直接
# 起不来,而且失败方式很隐蔽——
# [config] secrets disabled: open /etc/llmsproxy/master.key: permission denied
# 只是**一行日志**,服务会带着"敏感值将以明文落盘"继续跑起来。
# 也就是说降权在当前文件权限下不是加固,而是把密钥降级。要降权必须先把
# master.key 交给服务用户并统一 /etc/llmsproxy 的属主,那是一次独立的、有回滚
# 需求的变更,不该和加固混在一起。
#
# 下面每一条都在一个独立探针单元(临时端口 + 独立 runtime_file/adapter_dir)
# 上真实验证过:鉴权 401/200 正常、发一次真实 /v1/chat/completions 走通(证明
# LuaJIT 适配器路径没被 seccomp 打断)、审计文件可写可轮转、连续重启 3 次与
# kill -9 后行为符合预期。systemd 对非法指令值不报错只"忽略",逐条实测是唯一
# 可靠做法。
NoNewPrivileges=yes
# 读路径全部落在 /etc/llmsproxy;写路径经核对只有 config.yaml / runtime.json /
# audit.jsonl / adapters/*.lua / master.key,全在该目录下(internal/{config,gateway,
# core,lua} 里的 WriteFile|Rename|Remove 调用点)。
ProtectSystem=strict
ReadWritePaths=/etc/llmsproxy
ProtectHome=yes
PrivateTmp=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictSUIDSGID=yes
RestrictRealtime=yes
LockPersonality=yes
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
# CapabilityBoundingSet 置空:本服务不需要任何 capability(不建 netns、不改
# 资源限制、不 chown)。留空即"一个都不给",比列一份允许清单更难写错。
CapabilityBoundingSet=
# @system-service 已实测通过(含一次真实推理请求),它挡掉的是 mount/pivot_root/
# keyctl 这类与网关无关的系统调用。
SystemCallFilter=@system-service
SystemCallArchitectures=native
[Install]
WantedBy=multi-user.target