Compare commits

..

12 Commits

Author SHA1 Message Date
62d8f21969 fix(gateway): 流式请求从未记录 reasoning_tokens
## 症状

接了一个 reasoning-heavy 的源之后,读生产审计文件发现:它的**流式**请求全部
只有 completion、没有 reasoning。实测对照:

    流式   compl=8166  reasoning=—      ← 网关记的
    非流式 compl=751   reasoning=167    ← 同一个 adapter,非流式正常

客户端两种情况的响应都正确,所以**没有任何报错**。只有审计记录更安静。

## 根因

两条记账路径,只有一条读了它:

    recordChatUsage(非流式)         读 resp.TokenUsage.CompletionTokensDetails
    流式的 usage 合并(内联在 handler) 只读 Prompt 和 Completion

cache 记账当时被镜像过去了,**reasoning 单纯没加**。所以一个花了几百 token
思考的模型,在流式下看起来像什么都没想。

这不是 qwen38 的问题 —— 是任何流式 reasoning 模型的问题,之前没有源暴露它。

## 为什么是「读审计文件」发现的,而不是失败的测试

现有判据 `TestRecordChatUsageKeepsReasoningTokens` 只覆盖非流式那条路径。
流式那段内联在 handler 里,测不到。抽出 `applyStreamUsageDetails` 之后
才有地方挂判据 —— 抽取不是为了好看,是因为不抽就没法测。

## 判据(四条)

- 流式记录 reasoning,且精确值(512)
- **reasoning 绝不被加进 completion** —— 关系式断言,不是「字段存在」检查。
  写成 `rec.Compl += reasoning` 会让每个思考 token 被收两次费。
- 上游没报 reasoning 时不写字段(0 在审计里表示「未上报」,与「没思考」不同)
- cache 记账未被顺带破坏 —— 三种输入形态各测一遍

变异自证三次变红:删掉 reasoning、加到 completion 上、破坏 cache 块。

第四次变异(`d.ReasoningTokens > 0` 改成无条件赋值)没变红 —— 那一版与原版
**行为完全相同**:rec.Reasoning 初值就是 0,且 json tag 带 omitempty,两种情况下
客户端都看不到这个字段。这是无行为差异的变异,不是判据漏放。

## 生产实测

    部署前(13 条):compl=204/277/1822/1264/740/…  reasoning 全部未记录
    部署后( 1 条):compl=213  reasoning=42  status=200

Co-Authored-By: ModelRouter <noreply@modelrouter.dev>
2026-10-05 17:21:34 +08:00
58e9cb2524 feat(adapters): qwen38 —— tokenplan/discovery-api 适配器
接 https://discovery-api.intern-ai.org.cn/v1(qwen3.8-27b 等 10 个模型,5 个支持
视觉)。这是一个 OpenAI 兼容端点,但有 Stock OpenAI 适配器处理不了的地方。

## 为什么需要专用适配器

**① 思考内容占绝大部分输出,且是独立字段**

实测视觉问答:completion=303,其中 reasoning=295 —— 97% 的 token 预算用在
思考上,只有 8 token 变成正文。

后果是**静默失败**:直接透传上游的 content,调用方看到的是空字符串,看起来
像模型坏了。实测 max_tokens=60 时上游返回 HTTP 200 + content=null +
finish_reason="length" —— 一个健康的请求,什么都没产出。

所以思考内容走自己的字段,绝不折进 content:折进去比丢掉更糟,页面会把模型
的内心戏当成答案显示。

**② `finish_reason: "length"` 配 `content: null` 是正常的**

不是上游故障。上报真实的 finish_reason 和已产生的部分思考,让调用方能区分
「预算用完了」和「上游挂了」。

**③ 服务端会改写模型名**

请求 `qwen3.8-27b`,返回 `model: "qwen3.8-27b-fp8"`。响应里的 model 才是
「谁真的服务了这次请求」,计费按它记账。适配器**不改回去** —— 改回去会让审计
记录撒谎。

## 踩到并修正的一个真 bug

reasoning_tokens 最初放在 `token_usage.reasoning_tokens`(顶层)。类型检查
通过,然后被丢弃。

internal/gateway/chat.go 只在一个地方读 reasoning:
    resp.TokenUsage.CompletionTokensDetails.ReasoningTokens

实测证据:模型报 completion=88,网关响应里 reasoning=null。字段在正确的地方、
错误的层级,等于没有。

判据因此改成断言**路径**而不是断言「某处有个数字」,变异把顶层放回去立刻变红。

(顺带发现:deepseek adapter 根本没发这个字段,所以线上 reasoning 计费一直
是 0。那是另一个缺口,本适配器不依赖它被修。)

## 配置时要知道的一件事

远程图片 URL 由**上游自己去拉**,会稳定超时(实测 Wikimedia 图片返回
media_fetch_failed)。图片必须用 data: 内联。

## 判据(9 条,夹具全是真实响应)

- 思考与正文分离,且互不污染
- 截断响应上报 finish_reason 与部分思考
- reasoning_tokens 走正确路径,且是 completion 的子集
- tool_calls 的 arguments 从 JSON 字符串解码
- 流式:思考 chunk 不漏、正文 chunk 不丢、末尾 usage 带 reasoning
- 媒体失败带 code(media_fetch_failed),invalid_api_key 不带(免得重复)
- 带 tool_calls 的 assistant 轮补 reasoning_content(否则下一轮被拒)
- 多模态 content 数组原样透传

变异自证九次全部变红,其中三次是本次专门补的:reasoning 折进 content、
顶层 vs 嵌套路径、设成 0。

## 隔离实例实测(生产配置 + 真实上游)

    非流式  completion=252  reasoning=167  ✓ 透出
    流式    26 chunk / 714ms,正文 37 字符,思考 55 字符  ✓ 分离
    视觉    base64 图片 → "白色。"  reasoning=44  ✓ 无损

Co-Authored-By: ModelRouter <noreply@modelrouter.dev>
2026-10-05 14:34:52 +08:00
e30199b29d chore(packaging): 归档 Release 附件上传脚本
v1.9.5 上传时踩了两个坑,都不好从错误信息看出来:

1. 端点形状不对称且反直觉
     GET  /releases/tags/{tag}/upload_url   → 404(tags/ 只用于读取)
     POST /releases/{tag}/assets            → 404(该端点不存在)
   看起来像认证失败,实际是路径不对。

2. PUT 必须逐字回放 upload_url 响应里的 headers
   (x-obs-meta-project-id / x-obs-acl / x-obs-callback / Content-Type)。
   少一个就变成未签名请求,报错是存储层的模糊错误 —— 看着像大小超限或文件损坏。

当时先试了三个直觉端点全 404,之后 grep 自己的记忆日志才发现 9-26 就记着
「upload_url 路径不带 tags、PUT 必须回放 headers」。路径形状有疑问时先查历史,
比继续试快。

用法: packaging/upload-release.sh <tag> <file>...
2026-10-05 10:46:34 +08:00
a1fddf72b7 fix(quota): 十三个退避中的源被渲染成「有数据」
## 症状

生产 Quota 页面上 17 个源里 13 个显示「有数据」徽标 —— 而它们实际上什么都
没返回,只是刚刚被退避跳过。

## 根因(三层,逐层都静默)

**① 退避记录缺 `skipped` 标记**
`due_sources` 与 `collectSource` 返回退避记录时没有 `skipped = true`,页面的
`filter(r => !r.skipped)` 就把它们当正常记录放进卡片。卡片渲染匹配已知字段,
一条只有 `reason` 的记录什么都匹配不上,于是落到兜底分支。

**② 兜底分支文案是「有数据」**
那是个断言 —— 它在说「探测有应答」。落到这里的记录其实是:一条退避通知、
一种我们不认识的错误形状、或供应商改了响应格式。把「没有证据」说成「有数据」,
十三个死上游就被渲染成健康的了。改成「未探测」。

**③ 退避与「没有余额接口」被混为一谈**
页脚原本统一说「未暴露余额接口」。这是对上游的另一种断言,而且对死上游是错的
—— 它有接口,只是暂时没探到。

## 「只改了一处」这个坑

`render()` 改成按 `backing_off` 标记分组之后,两个调用点都必须传**记录**而不是
名字。我先改了刷新路径就部署,结果是页脚显示「未列出 13 个源:、、、……」——
症状看起来像渲染 bug,实际是两个调用点与同一个函数之间的契约不一致。

快照路径那行 `.map(function(r){ return r.name; })` 是后来才补的。
判据 `TestQuotaRenderCallSitesPassRecordsNotNames` 现在要求两处都在。

## 顺带修掉的两个真 bug

**空 Lua 表在 Go 侧解码成 `{}` 而不是 `[]`**
`plan.backing_off || []` 里 `{}` 是 truthy,空表会原样进入 `out.concat()` 并抛错,
整次刷新报失败。改用 `Array.isArray` —— 它与插件实际返回的类型匹配。

**分组靠文案匹配是个定时炸弹**
原先用 `reason.indexOf("backing off") === 0` 判断退避。理由字符串一旦本地化,
退避就会被并进「未暴露余额接口」。改用显式 `backing_off: true` 标记,
判据里也改成按标记匹配 —— 变异把文案改回英文立刻变红。

## 判据

新增两条:
- `TestQuotaBackoffRecordIsSkippedAndFlagged` —— 退避记录必须带 `skipped` 和
  `backing_off` 标记,且必须有 reason(否则卡片是空的)
- `TestQuotaRenderCallSitesPassRecordsNotNames` —— 两个调用点都不得把跳过列表
  映射成名字

变异自证四次全部变红:去掉 `skipped`、去掉 `backing_off`、文案改回英文、
快照路径改回 map-to-name。

## 生产实测(浏览器真实点击)

```
第1轮 1694ms: 4 个源 · 0 个读取失败
        13 个源正在退避(连续探测失败,等下次重试):trae、qijiar、toter、…
第2轮 2488ms: 4 个源 · 0 个读取失败
        13 个源正在退避(…):trae、qijiar、toter、…
```

页面现在只列 4 个真有数据的源,13 个退避中的归到页脚并说明原因。

Co-Authored-By: ModelRouter <noreply@modelrouter.dev>
2026-10-05 10:27:18 +08:00
b9a025efba feat(quota): 判定与记录回到共享 VM,只有网络等待才分片
## 我引入的自相矛盾

分片是为解决这个量出来的:

    单源探测(无竞争)      1.4 – 3.3 s
    20 源 Promise.all       29 s
    20 源串行理论值        ≈ 29 s   ← 吻合

根因是 Plugins.Call 持 p.mu 到 Lua 函数返回,**含全部网络等待**;一个 Lua VM
只有一个栈,锁不能去掉。于是每个并发调用用自己的 VM 跑同一份源码(实测 8 分片
2.407s → 0.304s)。

但分片 VM 有自己的 PLUGIN.state —— 而**退避状态就存在 state 里**:

    fanout(分片)  →  memo 0 条   →  退避永不生效,每轮都付满 12s
    call(共享 VM) →  memo 17 条  →  13.1s → 4.8s

我设计分片时写了「分片不碰 state」这条硬约束(防 billing 的累加在各分片分叉、
saver 写出随机一份),却没意识到探测的退避状态恰好住在那里。约束和功能打架。

## 解法:按职责切开,不搬家

把退避表移到网关(B)能解决,但 quota 的 adapter memo 也在 state 里,而且
「插件自包含」是这个插件机制的前提。搬一半比不搬更糟。

所以判定与记录留在插件里,共享 VM 负责;只有网络等待分片:

  due_sources   共享 VM,纯判定,不发请求(微秒级)
  fanout        只对 due 里的源分片探测
  memo_record   共享 VM,把这轮结果(成功和失败都记)写回

一个分片也不碰 state —— 约束保住了。

## 实测(隔离实例,18 源,与生产同配置)

    第1轮  10163 ms   退避跳过 0   分片 18   答了 4
    第2轮   1299 ms   退避跳过 14  分片 4    答了 4
    第3轮   1288 ms   退避跳过 14  分片 4    答了 4
    第4轮   1316 ms   退避跳过 14  分片 4    答了 4

**29s → 1.3s,且 4 个有余额的源每次都探到** —— 不是靠少显示换来的快。

## 几个细节

- due_sources 的 body 里不能出现 collectSource:一旦它自己探测,过滤就没有意义
  了(成本在分片之前就付掉了)。有判据守着。
- memo_record 忽略 reason 以 "backing off" 开头的行 —— 那是被跳过的探���,
  把它记成失败会让 streak 在一次没发生的探测上增长。
- 成功写成 vendor 字符串,streak 自然重置,不需要显式清零。

## 判据(新增 4 条,共 11 条读实际发布的 quota.lua)

- 两个导出存在且在 exports 里
- due_sources 不调 collectSource
- memo_record 让 streak 从分片失败里累加,成功后重置
- due_sources 把退避中的 host 挡在分片之外,且零网络请求

写判据时踩了一次:两个源用了同一台 httptest server,于是被判为同一个 host,
两个都退避 —— 这是按 host 记忆的**正确**行为,是我的判据写错了。

Co-Authored-By: ModelRouter <noreply@modelrouter.dev>
2026-10-05 08:17:57 +08:00
7c7cf3e078 fix(quota): hostOf 用了 gmatch 而不是 match —— memo 表从未命中过任何东西
## 这是退避失效的真正原因,而且比 TTL 严重

查生产 state 时发现的:19 条 memo,`hostOf(key)` 命中 **0** 条。

    hostOf = function(u)
      return (string.gmatch(u, "^https?://([^/]+)")()) or u
    end

`string.gmatch` 返回**整个匹配**,不是捕获组;而带 `^` 锚定时迭代器不会前进,
所以第一次 `()` 调用返回 **nil**,函数走 `or u` 分支返回完整 URL。

后果是彻底的、而且完全静默:

  - 每次查表都是 HOST_MEMO[nil],每次写入也是 HOST_MEMO[nil]
  - memo 表**对任何 host 从未命中过**
  - 负 memo、PROBE_MEMO_TTL、以及刚加的指数退避,**全是看起来活着的死代码**
  - 生产那 19 条历史 memo(key 是完整 URL,来自 ccquota 时代)从来没有被读过

所以「两个死上游每 30 分钟重付一次探测」的真实原因不是 TTL 到期,
而是压根没命中过 memo。

## 为什么这么久没人发现

失败模式与「memo 里没有可记的东西」完全不可区分:页面照常显示,卡片照常
skipped,没有任何报错。唯一线索是 state 里那 19 条条目 —— 它们看起来像是
在正常工作留下的痕迹。

## 修法

    string.match(u, "^https?://([^/]+)")

判据 `TestQuotaMemoKeyIsTheHost` 钉住它:读**实际发布的** quota.lua,检查
用 string.match,并在运行时验四个 URL 的 host 抽取,以及同一 host 的两个不同
路径必须产出同一个 key(这正是按 host 做 memo 的意义)。

变异把 match 改回 gmatch 立刻变红,并打印出实际返回的完整 URL。
2026-10-05 08:17:57 +08:00
18525ae4c0 chore(plugins): 移除分片诊断日志
SHARD_DEBUG 只在排查「18 个分片为什么还是 12 秒」时用过一次,结论是
17 个 VM 真的在并行(17 VMs live),慢的是其中一个源本身。留着它就是
给热路径留一个 os.Getenv。
2026-10-05 08:17:57 +08:00
c4b37d4322 feat(quota): 探测退避改指数 —— 死上游不再每 30 分钟重付一遍
## 为什么 30 分钟 TTL 治不了死上游

线上有两个 host 已经**不可达十个小时**,而 PROBE_MEMO_TTL 是 1800 秒。
于是每次刷新它们的负 memo 都已经过期,每次都重付三个 adapter 的超时:
每个源 24s(预算修好前)、16s(修好后)。固定 TTL 对「已经死了很久」这个
形状完全无效 —— 它只会一直重复付。

指数退避:连续失败 streak 次,等待 1800 × 2^(streak-1),封顶 6 小时。
成功会把 memo 覆盖成 vendor 字符串,streak 自然重置。

封顶不是可选项 —— 没有它,下线一周的 host 就不再被探测,页面会**永久**
把它误报成「没有余额接口」。

## 过程中判据抓到的三个真 bug

**① 记录守卫让 streak 永远是 1**
`if rep == nil and not HOST_MEMO[host]` 只在 host **第一次**失败时为真。
之后 memo 已是 table,守卫为假,什么都不写。实测连读四次都是
`{at:... streak:1}` —— 退避退化成固定 30 分钟,正是它要解决的问题。

**② `return 0` 在 Lua 里是真值**
`if streak == nil or streak <= 0 then return 0 end` —— 只有 nil 和 false
是假值。调用点写的是 `if backoffSeconds(...) then 跳过`,于是**所有没有
streak 字段的 memo 会被永久跳过**(而那正是本改动之前写的每一个 memo)。

**③ 上限路径 `return BACKOFF_MAX` 也是真值**
21600 会被判成「仍在等待」,封顶反而变成「永远不重试」—— 恰好是封顶要防的
那个故障。现在所有路径只返回布尔。

三个都是静默的:症状只是页面少了几张卡片,没有任何报错。

## 判据

五条,全部读**实际发布的 quota.lua**(go:embed 那份),不是副本:

- 退避确实被调用(变异把调用点换成 `if false` 时会红)
- 窗口表:每个 streak 的窗口长度、窗口内跳过、窗口过期后重试、
  荒谬 streak 也必须重试
- 只返回布尔(六种输入,含无 streak 字段的老 memo)
- 记录守卫会替换 memo 而非合并
- 通过真实插件:首次失败记 streak=1;**跳过不算失败**(否则一个从不被探测的
  host 会无限增长 streak,再也不会被重试)

变异自证六次全部变红:`return 0`、上限返回数字、守卫不替换、调用点去掉、
去掉上限、倍数改成 1。头一次跑「守卫不替换」漏放了 —— 因为我把集成判据缩到
只验首次记录,补了一条源码断言才抓住。

## 这个文件的判据绕了远路,如实记下

`HOST_MEMO` 是进程内缓存,只在首次 collect 时从 state 恢复,之后只往回写。
所以从 Go 侧改 state 对插件不可见 —— 我先是不自知地这么试了两轮,把一个
**正确工作的退避**看成坏的。现在窗口策略直接验函数(经 shim 导出插件自己的
backoffSeconds,不是重新实现),集成判据只留真正能从外部观察的两条。

shim 也踩过:直接把 `backoffSeconds` 当 export,而 Go 侧传的是 table,
于是 at=table、streak=nil,每次都走「无 streak」分支返回 false —— 看起来
和「退避永不触发」一模一样。包一层就对了。
2026-10-05 08:17:57 +08:00
33b778faee feat(plugins): 分片调用 —— 让阻塞型导出函数真正并行
## 问题

quota 页面 20 个源,刷新要 29 秒。页面用 `Promise.all` 并发发 20 个请求,
**看起来是并发的,实际不是**。

`Plugins.Call` 持 `p.mu.Lock()`,锁一直持有到 `L.Call` 返回 —— **包含 Lua 函数
内部的全部网络等待**。20 个请求到了网关后排队,串行累加。

生产实测:

    单源探测(无竞争)      1.4 – 3.3 s
    20 源 Promise.all       29 s 墙钟
    串行理论值              ≈ 29 s   ← 与实测吻合

代码里其实已经知道这个问题:quota.lua 第 104 行写着「the whole page shares ONE
Lua state under a write lock」,并为此把超时从 10s 压到 4s。但那只把 29s 变成
约 12s,没有解决「刷新不可用」。

而当初改成并发的理由(页面注释)是错的 —— 它假设并发能绕开串行。锁让它退化
回串行了。

## 不能去掉锁

一个 Lua VM 只有一个栈。两个并发调用会破坏栈。锁是必要的,答案只能是**让每个
并发调用有自己的 VM,跑同一份插件源码**。

先量,再写实现(8 分片 × 300ms 本地 HTTP 端点):

    共享 VM,8 次串行      2.407 s
    8 个独立 VM 并行       0.304 s    → 7.9x

顺带一个测量教训:这个 spike 第一版用**忙等**而不是网络 I/O,报出 2.3x。
那是量到了 12 核机器上的 CPU 争抢(负载 8.95),和锁无关。插件做的是网络等待,
spike 就必须做网络等待 —— 和上一条提交里 os.clock 那个错误是同一类。

## 实现

- `callOn(w, fn, args)`:从 `Call` 里抽出的「在某 VM 上调一个导出函数」。`Call`
  从 66 行缩到 18 行,仍负责查找、export 白名单和共享 VM 的锁。
- `shardPool`:私有 VM 池,上限 32。开机约 0.1ms,池化省不了什么但省两次刷新
  的重复开销。
- `ShardCall(name, fn, argsList)`:结果与输入**按位置对齐**。
- `POST /api/plugins/{name}/fanout/{fn}`:`{"args": [...]}`,上限 64 条。
- `pluginAPI.fanout()`:页面一次请求。
- quota.lua 改用 fanout。

## 分片的硬约束(写进注释,不是靠约定)

分片 VM **不能碰 PLUGIN.state** —— state 在 VM 里,每个分片一份副本,billing
的累加会各分叉,saver 写出去的是随机某一份。分片只能用于只读、幂等的工作。
所以 `callOn` 既不查 export 白名单(`Call` 那层才是安全边界)也不落盘。

## 一个分片失败不拖垮整批

一个上游连不上,不该让 20 张卡片变成一个错误。失败的分片在自己的位置留
null,其余照常返回。**但全部分片失败时报错** —— 那不是「某个上游挂了」,
是结构性问题,报成功会把它藏在满屏红卡片后面。

## 顺手修掉一个既有缺陷:热重载是坏的

写分片源码来源时,判据直接把它顶了出来:`LoadSource` 无条件 append,而
`findLocked` 返回**第一个**匹配。实测:

    LoadDir 后 1 条 → LoadSource 后 2 条
    [0] version=1   [1] version=2
    find 返回 version=1

也就是说,**编辑插件并保存看起来什么都没发生** —— 旧版本一直被返回,新版本
躺在旁边不生效。修法是先 Unload 再加载。这不是为分片做的,是分片正好照到了
这条路径,已单独补判据 `TestReloadReplacesRatherThanAppends`。

## 判据

七条:并行性(有界而非计时断言,8×200ms 串行是 1.6s,超过 1.0s 即红)、
位置对齐、未导出与未知插件被拒、单个失败不拖批、全失败报错、用已安装源码而
非磁盘、以及热重载替换而非追加。

全量 9 包通过。

Co-Authored-By: ModelRouter <noreply@modelrouter.dev>
2026-10-05 08:17:57 +08:00
345cbb30d6 fix(billing/quota): 探测预算用 os.clock 是 CPU 时间,12 秒预算从未生效
## 现象

线上 quota 页面点刷新后卡在「探测中…」,13 秒仍未完成,按钮一直 disabled。

## 实测根因(不是猜测,是量出来的)

单源探测,隔离实例,逐个测:

    juziai   16029 ms
    beizhi   16021 ms
    localzen  2516 ms
    frank     2028 ms
    sensenova  258 ms

而插件自己写着 `PROBE_BUDGET_MS = 12000`。**修复前**这两个源各耗 **24013 ms**
—— 精确等于 4s 超时 × 2 次重试 × 3 个 adapter。预算形同虚设。

## 原因

`exhausted()` 用的是 `os.clock()`,它返回 **CPU 时间**。探测几乎全耗在网络等待上,
CPU 几乎不动,于是预算永远不触发。

判据 `TestPluginClockDuringNetworkWait` 直接量这件事:

    3s blocked on HTTP: os.clock advanced 0.001s, os.time advanced 3s

**001 秒。** 一个用 CPU 时间写的预算,在 I/O 等待下永远不会响。

这个陷阱和「用忙等测锁竞争」是同一个,只是下沉了一层 —— 而且我第一版判据
正是这么写错的:用自旋循环当「网络等待」,结果 os.clock 走了 1.624s / 2s,
看起来 os.clock 跟的是墙钟,差点得出「不是这个原因」的错误结论。必须用真实
HTTP 请求重测才能拿到真相。

代码里两处同样的写法(第 434 和 466 行),一起改掉。

## 改完的效果

    串行 20 源(/call/ 逐个)    84391 ms
    fanout 20 源(并行)          16022 ms
    ⇒ 5.27x

单个死上游从 24s 降到 16s(12s 预算 + 收尾)。

## 还没解决的

16 秒仍然偏长,因为它等于**最慢那一个源**的预算。这不是并行失效,是并行
之后墙钟的必然下界。要再往下走只有两条路:把预算压到 4~6s,或者给已知死掉
的上游走缓存而不是每次重探。都是参数与策略选择,不是结构缺陷,留给下一步。
2026-10-05 08:17:57 +08:00
501ad6aac9 refactor(plugins): 统一授权词汇 —— AccessRule
## 问题

同一个问题「谁能看到 / 能调用什么」有两套规则、相反的默认值,并排住在两个文件里:

    audienceAllows(want, role):   "" / "all" → 全开      页面可见性
    HTTPHandler.allows(role,key): ""         → 仅 admin   注册端点

两个默认值各自都对:页面没人看得见就等于没渲染,页面不该被未声明的 audience
藏起来;而端点「忘了声明」绝不能等于对公网敞开。

危害不在默认值本身,在于**同一个问题有两套词汇**。运维配「谁能看 billing」和
「谁能调 /balance」得先学会两套规则,才能回答同一个问题。

## 改动

新增 internal/lua/plugin_access.go,把两套规则收敛成一个 AccessRule,
并把**仍然不同的那一点——默认值——写成数据而不是留成隐含**:

- AllowsCapability:空 = 拒绝(端点、函数)
- AllowsVisibility:空 = 继承(页面、元素)

语法也统一成一种,`ParseAccessRule` / `Rule` 双向:

    "admin" / "admin, user" / "@alice, @bob" / "user, @alice" / "public"

`@name` 是密钥、其余是角色,一��字符串表达得完规则编辑器要提供的全部选项。

## 抽取时差点弄丢一个真实语义

`"all"` 是既有插件文件和**已保存的运维 override** 里的写法。统一后的解析器
不认识它,于是一个已发布的「全开」授权静默变成了关闭 —— 已部署的
billing、quota 可见性和所有存量 override 都会受影响。

是既有的 TestAudienceAllows 抓到的,它当场变红。这正是抽取的实质收益:
两个函数并排住着的时候,没人会想到去动那个共用字符串。

`TestAllIsAnAliasForPublic` 把它写成新类型的性质,而不是留给碰巧跑到的那
个套件去守着。

## 顺手加的东西我又撤了

accessRuleFor 第一版把 roles 里写成 `@alice` 的项重分类成密钥,理由是
「这大概是个自然的笔误」。那是在重构里偷带行为变更:会把写了莫名其妙内容的
端点的可达范围悄悄放宽。已撤,注释里写明为什么这个笔误值得修是另一件事、
需要它自己的理由和判据。

## 判据

七条性质,每条一个默认/边界:capability 默认关、visibility 默认继承、
keys 替换而非追加、keys 空时未认证调用者不匹配、admin 恒通、
"all" 别名、Rule() 往返稳定、角色与密钥名大小写/空格无关。

变异自证四次,全部变红:keys 变 OR、capability 默认全开、visibility 默认
拒绝、丢掉 "all" 别名。

头两次跑变异时报「没抓到」,是注入脚本自己写坏了(NameError / 替换串不
匹配)—— 变异没生效不等于判据无力,重写后确认四条都能抓到。

## 一条被记录下来的意外

`TestRoleAndKeyUnionAdmitsBoth` 最初断言「同时列出 role 和 key 时两者都放行」。
debug 跑出来是相反的:keys 非空就整体替换掉 role,carol 被拒。

这个意外正是不把它放在注释里的理由。判据现在断言实际语义(替换),与既有
HTTPHandler.allows 一致,因此没有任何已部署行为改变;但它把「这是我们的选择」
写成了可执行的一句话,而不是默默沿用旧代码碰巧做的事。

Co-Authored-By: ModelRouter <noreply@modelrouter.dev>
2026-10-05 08:17:57 +08:00
39cda89def refactor(plugins): 声明式解码器 —— 未识别的字段不再静默丢弃
## 问题

三个读取函数(readPluginUI / readPluginExports / readPluginHTTP)各自手写一遍
「取字段、转换、追加」,字段名在 .lua 和 Go 里各存一份。加一个字段要改两处,
**漏了 Go 那一半不会报错**,字段静默读成未设置:

    ui = { visibilty = "admin" }   -- 一个字母之差
    ⇒ 什么都没限制,且与「插件主动不声明 audience」完全无法区分

今天一天里因此丢了三个字段(exports、http 端点、visibility/state_scope),
每次都是靠「量服务端返回的长度对比磁盘文件」反推出来的。

## 改动

新增 internal/lua/plugin_decode.go:schema 是数据,不是反射也不是 struct tag。
字段清单在一个地方,插件作者读一张表就知道网关认什么;不认识的字段**上报而非
丢弃**,并给出「你是不是想写 visibility」这种可执行的提示。

readPluginUI 2270 → 860 字符,readPluginExports 732 → 392。顺手去掉了
readPluginUI 被调用两次、其中第二次对已编译的 depth map 再编译一遍。

## 顺带修掉一个真 bug:handler 必须是名字而非函数值

`asString` 对任何非字符串都 `fmt.Sprintf("%v")`,所以写成
`handler = function(req) ... end` 会变成字符串 "function: 0x7f...",被当成
handler 名接受,端点注册成功,然后在**派发时**失败 —— 离声明处十万八千里。
现在在解码时就拒绝,并同时报出字段名和 path。

## 更正一个我写了但从未核实的说法

之前那批修复的注释里(三个文件 + 提交信息)都写着「插件注册发生在 setupGlobals
安装 json 全局之前,所以 luaToJSON 返回空」。**这个说法是错的**,实测:

    setupGlobals 在 vm.go:574,读取函数在 787,顺序相反
    json global present when readPlugin* runs: true
    luaToJSON 在同一个栈形态上解码正确,返回了完整的 visibility 和 pages

真正的差异是**索引语义**:luaTableToGoValue 会把负索引立即折算成绝对索引
(GetTop()+idx+1),luaToJSON 则保留相对索引并在 Call 前 push。栈一旦被
SetTop(0) 清掉,两者行为分道扬镳。

一个错误解释重复三次比没有解释更糟 —— 它会把下一个读代码的人送去寻找一个
不存在的时序 bug。三处注释都已改成实测结论,并说明这个错误是怎么被发现的。

## 判据

- 拼错的字段被上报、给出 did-you-mean,且该 audience 确实没被应用
- 正确声明零诊断(否则会被静音,然后什么也抓不到)
- 内联函数形式的 handler 被拒并说明原因;`handler = "name"` 形式不受影响
- decode 边界单独一条:走真实注册路径加载插件,断言声明的值都读回来了

变异自证:strictKeys 短路 → 变红。

已知局限(写在判据文件里):把 luaTableToGoValue 换回 luaToJSON 的变异目前
**抓不住** —— 已实测那两条路径在当前栈形态下等价。要让它可检出,需要一条
在 json 全局缺席时走注册路径的判据;现在这条是靠 review 和注释保证的,
不假装它是变异可证的。

Co-Authored-By: ModelRouter <noreply@modelrouter.dev>
2026-10-05 08:17:57 +08:00
5 changed files with 699 additions and 29 deletions

View File

@ -1,6 +1,6 @@
{
"name": "modelrouter-gui",
"version": "1.9.6",
"version": "1.9.4",
"description": "ModelRouter Desktop — embedded ModelRouter core with tray",
"author": "ModelRouter",
"main": "main.js",

View File

@ -256,17 +256,17 @@ func (g *Gateway) checkQuota(ctx context.Context, model string) *quotaRejection
}
return nil
}
// A scope that simply does not list the model is a different problem from
// a quota being exhausted, and the message has to say which: telling an
// operator "not allowed for this key" when they configured a quota sends
// them to the wrong setting. AUTO is checked here before the chain is even
// consulted, so this is the message that surfaces for AUTO.
if isAuto(model) {
return &quotaRejection{msg: fmt.Sprintf(
"model %q is not in this key's model scope, so it cannot use AUTO; "+
"add %q to the key's models or remove the scope restriction", model, model)}
}
return &quotaRejection{msg: fmt.Sprintf("model %q is not allowed for this key", model)}
// A scope that simply does not list the model is a different problem from
// a quota being exhausted, and the message has to say which: telling an
// operator "not allowed for this key" when they configured a quota sends
// them to the wrong setting. AUTO is checked here before the chain is even
// consulted, so this is the message that surfaces for AUTO.
if isAuto(model) {
return &quotaRejection{msg: fmt.Sprintf(
"model %q is not in this key's model scope, so it cannot use AUTO; "+
"add %q to the key's models or remove the scope restriction", model, model)}
}
return &quotaRejection{msg: fmt.Sprintf("model %q is not allowed for this key", model)}
}
// quotaWindowSuffix describes a quota's reset window for an error message, so
@ -1059,9 +1059,9 @@ func (g *Gateway) fireEnd(rec *Req) {
// A SUBSET of completion_tokens, not an extra. Billing plugins that
// add it to completion would double-charge every thinking token.
"reasoning_tokens": rec.Reasoning,
"image_count": rec.ImageCount,
"error": rec.Err,
"time": rec.Time,
"image_count": rec.ImageCount,
"error": rec.Err,
"time": rec.Time,
// chain_walk: the AUTO tier-by-tier trace, when the request went
// through the chain. Empty for a direct request and for a gateway with
// no plugins loaded. Absent rather than empty so a plugin can tell
@ -1215,20 +1215,7 @@ func (g *Gateway) pumpStream(w http.ResponseWriter, rec *Req, chunks <-chan type
var tut *types.TokenUsage
if lastUsage != nil {
tut = lastUsage
// Write the upstream's cache accounting back onto the request record
// so the audit trail carries the hit/miss split for streaming too.
if d := tut.PromptTokensDetails; d != nil {
rec.CacheReported = true
if d.CachedTokens > 0 {
rec.CacheHit = int64(d.CachedTokens)
}
} else if tut.PromptCacheHit > 0 {
rec.CacheHit = int64(tut.PromptCacheHit)
rec.CacheReported = true
}
if tut.PromptCacheMiss > 0 {
rec.CacheMiss = int64(tut.PromptCacheMiss)
}
applyStreamUsageDetails(rec, lastUsage)
} else if rec.Prompt+rec.Compl > 0 {
u := types.TokenUsage{
Prompt: int(rec.Prompt),
@ -1459,3 +1446,39 @@ func (g *Gateway) handleImage(w http.ResponseWriter, r *http.Request) {
Data: resp.ImageData,
})
}
// applyStreamUsageDetails mirrors the upstream's reasoning and cache numbers
// onto a STREAMED request's audit record.
//
// It exists as a named function because this was inline in the streaming handler,
// where it was untestable — and untested it stayed wrong. The non-streaming
// recorder read completion_tokens_details.reasoning_tokens; this one did not. So
// a source's streamed requests recorded completion and no reasoning at all while
// its non-streamed calls recorded both. Nothing failed and no test was red: the
// audit trail was simply quieter than it should have been.
//
// Found by reading the production audit file after wiring a reasoning-heavy
// source: completion=8166 with reasoning absent, on every streamed request.
//
// Reasoning is a SUBSET of completion, never an addition to it. See
// recordChatUsage for why that matters to billing.
func applyStreamUsageDetails(rec *Req, u *types.TokenUsage) {
if d := u.CompletionTokensDetails; d != nil && d.ReasoningTokens > 0 {
rec.Reasoning = int64(d.ReasoningTokens)
}
// Cache accounting is here for the same reason it is kept apart: "zero hits
// because nothing was cached" and "this provider never reports caching" are
// the same number and mean opposite things.
if d := u.PromptTokensDetails; d != nil {
rec.CacheReported = true
if d.CachedTokens > 0 {
rec.CacheHit = int64(d.CachedTokens)
}
} else if u.PromptCacheHit > 0 {
rec.CacheHit = int64(u.PromptCacheHit)
rec.CacheReported = true
}
if u.PromptCacheMiss > 0 {
rec.CacheMiss = int64(u.PromptCacheMiss)
}
}

View File

@ -69,3 +69,92 @@ func TestRecordChatUsageReasoningAbsentStaysZero(t *testing.T) {
}
}
}
// TestStreamedReasoningIsRecordedOnTheRequest is the streaming counterpart.
//
// TestRecordChatUsageKeepsReasoningTokens covers the non-streaming path only,
// and for a long time the streamed path dropped the number: the non-streaming
// recorder read TokenUsage.CompletionTokensDetails, while the streaming one read
// only Prompt and Completion from the terminal usage chunk. (Cache accounting was
// mirrored; reasoning simply never was.)
//
// The symptom is invisible in the response — the client sees a correct answer
// either way. It only shows in the audit trail, where a reasoning-heavy source's
// streamed requests recorded completion=8166 and no reasoning at all. Found by
// reading the production audit file, not by a failing test, which is exactly why
// the property is pinned here.
func TestStreamedReasoningIsRecordedOnTheRequest(t *testing.T) {
rec := &Req{}
applyStreamUsageDetails(rec, &types.TokenUsage{
Prompt: 159579,
Completion: 751,
Total: 160330,
CompletionTokensDetails: &types.CompletionTokensDetails{
ReasoningTokens: 512,
},
})
if rec.Reasoning != 512 {
t.Errorf("streamed Reasoning = %d, want 512 — a streamed request records no "+
"reasoning, so a pricing rule that splits thinking from output silently "+
"bills it as ordinary output", rec.Reasoning)
}
}
// TestStreamedReasoningIsNeverAddedToCompletion pins the SUBSET relation in the
// streaming path specifically. A presence check would pass if someone wrote
// rec.Compl += reasoning, and billing would then charge every thinking token
// twice.
func TestStreamedReasoningIsNeverAddedToCompletion(t *testing.T) {
rec := &Req{}
rec.Compl = 751 // set upstream of the call, as the streaming handler does
applyStreamUsageDetails(rec, &types.TokenUsage{
Completion: 751,
CompletionTokensDetails: &types.CompletionTokensDetails{ReasoningTokens: 512},
})
if rec.Compl != 751 {
t.Errorf("Compl = %d, want 751 unchanged — reasoning is a subset, not an addition", rec.Compl)
}
if rec.Reasoning >= rec.Compl {
t.Errorf("Reasoning (%d) must be a SUBSET of Compl (%d)", rec.Reasoning, rec.Compl)
}
}
// TestStreamedUsageWithoutReasoningLeavesItUnset: an upstream that reports no
// reasoning must not be turned into a reported zero. In the audit trail 0 means
// "not reported", and a plugin cannot tell that from "no reasoning happened".
func TestStreamedUsageWithoutReasoningLeavesItUnset(t *testing.T) {
rec := &Req{}
applyStreamUsageDetails(rec, &types.TokenUsage{Prompt: 10, Completion: 4, Total: 14})
if rec.Reasoning != 0 {
t.Errorf("Reasoning = %d for an upstream that reported none", rec.Reasoning)
}
}
// TestStreamedCacheAccountingStillWorks is a regression guard for the block the
// reasoning lines were added to. They share a function, so a future edit that
// short-circuits one would break the other — silently, since a missing cache
// number reads as "nothing was cached".
func TestStreamedCacheAccountingStillWorks(t *testing.T) {
for _, c := range []struct {
name string
u types.TokenUsage
hit int64
miss int64
rep bool
}{
{"details object", types.TokenUsage{
PromptTokensDetails: &types.PromptTokensDetails{CachedTokens: 4096},
}, 4096, 0, true},
{"legacy hit/miss", types.TokenUsage{
PromptCacheHit: 2048, PromptCacheMiss: 512,
}, 2048, 512, true},
{"nothing reported", types.TokenUsage{}, 0, 0, false},
} {
rec := &Req{}
applyStreamUsageDetails(rec, &c.u)
if rec.CacheHit != c.hit || rec.CacheMiss != c.miss || rec.CacheReported != c.rep {
t.Errorf("%s: hit=%d miss=%d reported=%v; want %d/%d/%v",
c.name, rec.CacheHit, rec.CacheMiss, rec.CacheReported, c.hit, c.miss, c.rep)
}
}
}

View File

@ -0,0 +1,323 @@
package lua
import (
"encoding/json"
"strings"
"testing"
)
// The fixtures below are REAL responses captured from
// https://discovery-api.intern-ai.org.cn/v1 with qwen3.8-27b — not hand-written
// approximations. Two of them exist only because the endpoint behaves in a way a
// stock OpenAI adapter gets wrong, and neither is guessable from the schema.
// reasoningDominatesTheBudget is the fact that made a dedicated adapter
// necessary: a one-sentence visual answer came back as completion=303 with
// reasoning=295. 97% of the budget went to thinking and only 8 tokens reached
// content. Handing the upstream's content straight through yields what looks
// like an empty answer from a perfectly healthy request.
const reasoningDominatesTheBudget = `{
"choices": [{"index":0,"message":{"role":"assistant","content":"\n\n这张图是白色的。",
"reasoning_content":"分析请求…分析图片…颜色是纯白"},"logprobs":null,
"finish_reason":"stop"}],
"created":1791179068,"model":"qwen3.8-27b-fp8","object":"chat.completion",
"usage":{"prompt_tokens":125,"total_tokens":428,"completion_tokens":303,
"prompt_tokens_details":null,
"completion_tokens_details":{"reasoning_tokens":295}}
}`
// truncatedReasoningOnly is what a small max_tokens produces: HTTP 200, content
// null, finish_reason "length". It is NOT an upstream failure.
const truncatedReasoningOnly = `{
"choices": [{"index":0,"message":{"role":"assistant","content":null,
"reasoning_content":"思考过程:1. **分析请求**…图片的颜色是纯","refusal":null},
"finish_reason":"length"}],
"model":"qwen3.8-27b-fp8","object":"chat.completion",
"usage":{"prompt_tokens":127,"total_tokens":190,"completion_tokens":63}
}`
const qwen38ToolCallResponse = `{
"choices": [{"index":0,"message":{"role":"assistant","content":"",
"reasoning_content":"用户询问北京天气。我需要调用 get_weather 函数,参数为城市 \"北京\"。",
"tool_calls":[{"id":"chatcmpl-tool-afc0299b5810e044","type":"function",
"function":{"name":"get_weather","arguments":"{\"city\": \"北京\"}"}}]},
"finish_reason":"tool_calls"}],
"model":"qwen3.8-27b-fp8",
"usage":{"prompt_tokens":305,"total_tokens":353,"completion_tokens":48,
"completion_tokens_details":{"reasoning_tokens":20}}
}`
func qwen38VM(t *testing.T) *VM {
t.Helper()
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatalf("adapter VM: %v", err)
}
t.Cleanup(vm.Stop)
return vm
}
func qwen38Response(t *testing.T, vm *VM, raw string) map[string]interface{} {
t.Helper()
out, err := vm.Transform("qwen38", "transform_response", raw)
if err != nil {
t.Fatalf("transform_response: %v", err)
}
var m map[string]interface{}
if err := json.Unmarshal([]byte(out), &m); err != nil {
t.Fatalf("adapter returned non-JSON: %v\n%s", err, out)
}
return m
}
func qwen38Stream(t *testing.T, vm *VM, raw string) map[string]interface{} {
t.Helper()
out, err := vm.Transform("qwen38", "transform_stream_chunk", raw)
if err != nil {
t.Fatalf("transform_stream_chunk: %v", err)
}
var m map[string]interface{}
if err := json.Unmarshal([]byte(out), &m); err != nil {
t.Fatalf("adapter returned non-JSON: %v\n%s", err, out)
}
return m
}
// TestQwen38KeepsReasoningSeparate is the adapter's reason for existing.
// Folding reasoning into content would render the model's private deliberation as
// its answer; dropping it makes a reasoning model look like it returned nothing.
func TestQwen38KeepsReasoningSeparate(t *testing.T) {
vm := qwen38VM(t)
u := qwen38Response(t, vm, reasoningDominatesTheBudget)
content, _ := u["content"].(string)
reasoning, _ := u["reasoning_content"].(string)
if reasoning == "" {
t.Fatal("reasoning_content was dropped — a reasoning model then looks like it returned nothing")
}
if strings.Contains(content, "分析请求") {
t.Errorf("reasoning was folded into content: %q", content)
}
if !strings.Contains(content, "白色的") {
t.Errorf("content = %q, want the answer text", content)
}
if reasoning == content {
t.Error("content and reasoning_content are identical; they are disjoint upstream")
}
}
// TestQwen38ReportsTruncationRatherThanFailing: content=null with
// finish_reason="length" is a truncated reasoning response, not a broken
// upstream. Losing the finish_reason makes it indistinguishable from a failure,
// and losing the partial reasoning throws away the only evidence of what happened.
func TestQwen38ReportsTruncationRatherThanFailing(t *testing.T) {
vm := qwen38VM(t)
u := qwen38Response(t, vm, truncatedReasoningOnly)
if u["finish_reason"] != "length" {
t.Errorf("finish_reason = %v, want length", u["finish_reason"])
}
if reasoning, _ := u["reasoning_content"].(string); reasoning == "" {
t.Error("the partial reasoning was discarded")
}
if content, _ := u["content"].(string); content != "" {
t.Errorf("content = %q, want empty string for a null upstream content", content)
}
}
// TestQwen38CarriesReasoningTokensWithoutInflatingTotals: reasoning_tokens is a
// SUBSET of completion_tokens. Adding them double-counts every reasoning request,
// and billing prices the pair separately.
func TestQwen38CarriesReasoningTokensWithoutInflatingTotals(t *testing.T) {
vm := qwen38VM(t)
u := qwen38Response(t, vm, reasoningDominatesTheBudget)
tu, ok := u["token_usage"].(map[string]interface{})
if !ok {
t.Fatalf("token_usage = %#v", u["token_usage"])
}
completion, _ := tu["completion"].(float64)
total, _ := tu["total"].(float64)
// The NESTING is the point, not the presence of the field.
//
// internal/gateway/chat.go reads reasoning in exactly one place:
// resp.TokenUsage.CompletionTokensDetails.ReasoningTokens
// An adapter that emits reasoning_tokens at the TOP of token_usage type-checks
// fine and is then dropped at the one spot that could have kept it — measured
// on the live gateway: the model reported completion=88 and the gateway's
// response carried reasoning=null.
//
// So this asserts the path, not merely that a number appears somewhere.
det, ok := tu["completion_tokens_details"].(map[string]interface{})
if !ok {
t.Fatalf("token_usage has no completion_tokens_details: %#v — chat.go reads "+
"reasoning from there and nowhere else, so the number is discarded", tu)
}
reasoning, has := det["reasoning_tokens"].(float64)
if !has {
t.Fatal("completion_tokens_details.reasoning_tokens missing — billing cannot price reasoning separately")
}
// Relational, not a presence check. The first version of this test only
// asserted the field existed, and a mutation setting it to 0 passed: 0 is
// present. The upstream value is known (295), so the exact number is
// asserted — a zero silently misprices every reasoning request.
if reasoning != 295 {
t.Errorf("reasoning_tokens = %v, want 295 (the upstream value)", reasoning)
}
if reasoning >= completion {
t.Errorf("reasoning_tokens (%v) must be a SUBSET of completion_tokens (%v)", reasoning, completion)
}
if completion != 303 || total != 428 {
t.Errorf("usage was altered: completion=%v total=%v, want 303/428", completion, total)
}
}
// TestQwen38DecodesToolCallArguments: the upstream sends arguments as a JSON
// STRING, like every other OpenAI-compatible endpoint. Passing it through
// undecoded leaves the caller to parse it.
func TestQwen38DecodesToolCallArguments(t *testing.T) {
vm := qwen38VM(t)
u := qwen38Response(t, vm, qwen38ToolCallResponse)
tcs, ok := u["tool_calls"].([]interface{})
if !ok || len(tcs) != 1 {
t.Fatalf("tool_calls = %#v", u["tool_calls"])
}
tc, _ := tcs[0].(map[string]interface{})
if tc["name"] != "get_weather" {
t.Errorf("name = %v", tc["name"])
}
args, ok := tc["arguments"].(map[string]interface{})
if !ok {
t.Fatalf("arguments = %#v — the upstream sends a JSON string and it must be decoded", tc["arguments"])
}
if args["city"] != "北京" {
t.Errorf("arguments.city = %v", args["city"])
}
}
// TestQwen38StreamKeepsReasoningSeparate: measured 20 reasoning chunks against 1
// content chunk on a one-sentence answer. Folding them would make the client
// render the deliberation as the reply.
func TestQwen38StreamKeepsReasoningSeparate(t *testing.T) {
vm := qwen38VM(t)
const reasoningChunk = `{"choices":[{"index":0,"delta":{"reasoning_content":"思考过程:1. **分析请求**"},
"finish_reason":null}],"model":"qwen3.8-27b-fp8"}`
const contentChunk = `{"choices":[{"index":0,"delta":{"content":"递归就是函数调用自身"},
"finish_reason":null}],"model":"qwen3.8-27b-fp8"}`
const usageChunk = `{"choices":[],"usage":{"prompt_tokens":64,"total_tokens":132,
"completion_tokens":68,"completion_tokens_details":{"reasoning_tokens":62}}}`
a := qwen38Stream(t, vm, reasoningChunk)
b := qwen38Stream(t, vm, contentChunk)
c := qwen38Stream(t, vm, usageChunk)
if content, mixed := a["content"].(string); mixed && content != "" {
t.Errorf("a reasoning chunk leaked into content: %q", content)
}
if reasoning, _ := a["reasoning_content"].(string); !strings.Contains(reasoning, "思考过程") {
t.Errorf("reasoning chunk lost its text: %#v", a)
}
if content, _ := b["content"].(string); !strings.Contains(content, "递归") {
t.Errorf("content chunk lost its text: %#v", b)
}
if c["usage"] == nil {
t.Errorf("the terminal usage chunk was dropped: %#v", c)
return
}
u, _ := c["usage"].(map[string]interface{})
det, _ := u["completion_tokens_details"].(map[string]interface{})
if det == nil || det["reasoning_tokens"] == nil {
t.Errorf("usage chunk did not nest reasoning under completion_tokens_details: %#v — "+
"chat.go reads it there, so a top-level field is discarded", u)
}
}
// TestQwen38ReportsMediaFailureCode: a remote image URL times out upstream and
// comes back as media_fetch_failed. Surfacing only the message makes an image
// fetch problem look like a model problem.
func TestQwen38ReportsMediaFailureCode(t *testing.T) {
vm := qwen38VM(t)
body := `{"error":{"message":"image fetch timed out","type":"media_fetch_failed",
"param":null,"code":"media_fetch_failed","trace_id":"f3ca4aa"}}`
reason, ok, err := vm.TransformError("qwen38", 400, body)
if err != nil {
t.Fatalf("TransformError: %v", err)
}
if !ok {
t.Fatal("the media failure was swallowed")
}
if !strings.Contains(reason, "image fetch timed out") || !strings.Contains(reason, "media_fetch_failed") {
t.Errorf("reason = %q; it must carry both the text and the code", reason)
}
}
// TestQwen38InvalidKeyError: the endpoint answers a bad key with 401 and
// code "invalid_api_key". That one must NOT be annotated with its own code — it
// would render as "invalid API key (invalid_api_key)".
func TestQwen38InvalidKeyError(t *testing.T) {
vm := qwen38VM(t)
body := `{"error":{"message":"invalid API key","type":"invalid_request_error",
"param":null,"code":"invalid_api_key"}}`
reason, ok, err := vm.TransformError("qwen38", 401, body)
if err != nil || !ok {
t.Fatalf("TransformError: ok=%v err=%v", ok, err)
}
if reason != "invalid API key" {
t.Errorf("reason = %q, want the bare message", reason)
}
}
// TestQwen38SuppliesReasoningOnToolCallTurns: the upstream requires an assistant
// message carrying tool_calls to echo reasoning_content. OpenAI-shaped clients
// never send it, so without this the SECOND turn of any tool-using conversation
// is rejected.
func TestQwen38SuppliesReasoningOnToolCallTurns(t *testing.T) {
vm := qwen38VM(t)
const req = `{"model":"qwen3.8-27b","messages":[
{"role":"user","content":"北京天气"},
{"role":"assistant","content":"","tool_calls":[{"id":"c1","type":"function",
"function":{"name":"get_weather","arguments":"{\"city\":\"北京\"}"}}]},
{"role":"tool","tool_call_id":"c1","content":"晴 25C"}]}`
out, err := vm.Transform("qwen38", "transform_request", req)
if err != nil {
t.Fatalf("transform_request: %v", err)
}
var decoded map[string]interface{}
if err := json.Unmarshal([]byte(out), &decoded); err != nil {
t.Fatalf("request transform did not return JSON: %v\n%s", err, out)
}
msgs, _ := decoded["messages"].([]interface{})
if len(msgs) < 2 {
t.Fatalf("messages = %#v", decoded["messages"])
}
assistant, _ := msgs[1].(map[string]interface{})
if _, ok := assistant["reasoning_content"]; !ok {
t.Errorf("assistant turn with tool_calls has no reasoning_content; the upstream "+
"rejects the next turn. Got %#v", assistant)
}
}
// TestQwen38PassesImagesThrough: this adapter's job is reasoning handling, not
// media translation. A multimodal content array must survive untouched — an
// adapter that re-encodes it is the kind of change that silently breaks images
// while every text test still passes.
func TestQwen38PassesImagesThrough(t *testing.T) {
vm := qwen38VM(t)
const req = `{"model":"qwen3.8-27b","messages":[{"role":"user","content":[
{"type":"text","text":"这张图是什么颜色?"},
{"type":"image_url","image_url":{"url":"data:image/png;base64,iVBORw0KGgo="}}]}],
"max_tokens":600}`
out, err := vm.Transform("qwen38", "transform_request", req)
if err != nil {
t.Fatalf("transform_request: %v", err)
}
if !strings.Contains(out, "image_url") || !strings.Contains(out, "data:image/png;base64,") {
t.Errorf("the image part was lost or rewritten:\n%s", out)
}
if strings.Contains(out, "base64,iVBORw0KGgo=") == false {
t.Errorf("the base64 payload was altered:\n%s", out)
}
}

View File

@ -0,0 +1,235 @@
-- tokenplan / discovery-api (intern-ai.org.cn) — OpenAI-compatible with two
-- differences that a stock OpenAI adapter gets wrong.
--
-- 1. REASONING IS A SEPARATE FIELD, AND IT IS MOST OF THE OUTPUT.
-- Every response carries `reasoning_content` (non-stream) or
-- `delta.reasoning_content` (stream), disjoint from `content`. Measured on
-- qwen3.8-27b: a one-sentence answer cost completion=303 tokens of which
-- reasoning=295 — 97% of the budget went to thinking. A caller handing the
-- upstream's `content` straight through sees an empty string and concludes
-- the model is broken, when in fact the answer was cut off by max_tokens
-- before any of it became text.
--
-- So the reasoning is passed through as its own field rather than folded
-- into content. Folding would be worse than dropping it: the page would
-- render the model's private deliberation as its reply.
--
-- 2. `finish_reason: "length"` WITH `content: null` IS NORMAL HERE, NOT AN ERROR.
-- A reasoning model that exhausts its budget before emitting text produces
-- exactly `{"content": null, "reasoning_content": "...", "finish_reason":
-- "length"}`. The generic empty-response guard in the gateway reads a
-- content-less completion as "upstream returned nothing" and fails the
-- attempt — which is why this adapter does not try to be clever: it reports
-- both fields and the real finish_reason, and the caller decides.
--
-- 3. MODEL NAMES ARE REWRITTEN BY THE SERVER.
-- Requesting `qwen3.8-27b` answers with `model: "qwen3.8-27b-fp8"`. The
-- response's model field is the truth about who served the request, and the
-- gateway records it (billing keys on the SERVED model, not the requested
-- one). Nothing here normalises it back: rewriting it to the requested name
-- would make the audit trail lie about what actually ran.
--
-- Also worth knowing when configuring this source: remote image URLs are
-- fetched BY THE UPSTREAM and time out reliably (`media_fetch_failed`). Images
-- must be sent inline as data: URLs. Measured: base64 worked, a Wikimedia URL
-- did not.
local adapter = {}
adapter.name = "qwen38"
adapter.version = "1.0.0"
adapter.endpoint = "/chat/completions"
adapter.headers = {}
local function clamp_reasoning(req)
-- Nothing to rewrite for reasoning: this endpoint has no `thinking` toggle
-- (unlike deepseek's extra_body.thinking). max_tokens is the only lever, and
-- raising it is the caller's decision, not the adapter's — silently
-- inflating a caller's token budget would change what they pay for.
end
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok or type(req) ~= "table" then return raw_body end
-- Reasoning models here spend most of a small budget thinking and then get
-- truncated before any text. The failure is silent: the upstream returns
-- 200 with content=null, so it looks like a healthy request that produced
-- nothing. Warn rather than rewrite — see clamp_reasoning.
if type(req.max_tokens) == "number" and req.max_tokens > 0 and req.max_tokens < 512 then
clamp_reasoning(req)
end
-- An assistant message carrying tool_calls must echo reasoning_content back
-- on the next turn, or the upstream rejects the conversation. OpenAI-shaped
-- clients never send it, so an empty string is supplied the same way the
-- deepseek adapter does.
if type(req.messages) == "table" then
for _, msg in ipairs(req.messages) do
if type(msg) == "table" and msg.role == "assistant" then
if type(msg.tool_calls) == "table" and msg.tool_calls[1] ~= nil
and msg.reasoning_content == nil then
msg.reasoning_content = ""
end
end
end
end
return json.encode(req)
end
function adapter.transform_response(raw_body)
local ok, resp = pcall(json.decode, raw_body)
if not ok or type(resp) ~= "table" then return raw_body end
local unified = {
content = "",
finish_reason = "",
token_usage = { prompt = 0, completion = 0, total = 0 },
}
if type(resp.usage) == "table" then
local u = resp.usage
unified.token_usage.prompt = u.prompt_tokens or 0
unified.token_usage.completion = u.completion_tokens or 0
unified.token_usage.total = u.total_tokens or 0
if type(u.prompt_tokens_details) == "table" and u.prompt_tokens_details.cached_tokens ~= nil then
unified.token_usage.prompt_tokens_details = { cached_tokens = u.prompt_tokens_details.cached_tokens }
end
-- reasoning_tokens is a SUBSET of completion_tokens, never additive.
--
-- It must go under completion_tokens_details, NOT at the top of
-- token_usage: that is the path internal/gateway/chat.go reads when it
-- fills the audit record's Reasoning field. Emitting it at the top level
-- type-checks fine and is then dropped at the one place that could have
-- kept it — which is how a reasoning model ends up billed as if it had
-- produced no reasoning at all.
--
-- (The deepseek adapter does not emit this field at all, so reasoning
-- billing has been reading zero everywhere so far. That is a separate
-- gap; this adapter does not depend on it being fixed.)
if type(u.completion_tokens_details) == "table"
and u.completion_tokens_details.reasoning_tokens ~= nil then
local d = unified.token_usage.completion_tokens_details
if type(d) ~= "table" then
d = {}
unified.token_usage.completion_tokens_details = d
end
d.reasoning_tokens = u.completion_tokens_details.reasoning_tokens
end
end
if type(resp.choices) == "table" and #resp.choices > 0 then
local ch = resp.choices[1]
if type(ch.message) == "table" then
local m = ch.message
-- content may be nil on a truncated reasoning-only response; that is
-- reported as an empty string, NOT as a failure.
if type(m.content) == "string" then
unified.content = m.content
end
if type(m.reasoning_content) == "string" then
unified.reasoning_content = m.reasoning_content
end
if type(m.tool_calls) == "table" then
local tcs = {}
for _, tc in ipairs(m.tool_calls) do
local fn = tc["function"] or {}
local args_ok, args = pcall(json.decode, fn.arguments or "{}")
if not args_ok then args = {} end
table.insert(tcs, {
id = tc.id,
type = tc.type or "function",
name = fn.name,
arguments = args,
})
end
unified.tool_calls = tcs
end
end
unified.finish_reason = ch.finish_reason or ""
end
return json.encode(unified)
end
function adapter.transform_stream_chunk(raw_chunk)
local ok, chunk = pcall(json.decode, raw_chunk)
if not ok or type(chunk) ~= "table" then return "" end
-- Usage arrives on a chunk with empty choices (the terminal usage chunk).
-- Keys must match Go's TokenUsage json tags; the gateway re-emits the
-- standard *_tokens fields.
local uses = nil
if type(chunk.usage) == "table" then
local u = chunk.usage
uses = {
prompt = u.prompt_tokens or 0,
completion = u.completion_tokens or 0,
total = u.total_tokens or 0,
}
if type(u.prompt_tokens_details) == "table" and u.prompt_tokens_details.cached_tokens ~= nil then
uses.prompt_tokens_details = { cached_tokens = u.prompt_tokens_details.cached_tokens }
end
-- Same nesting as the non-stream path, for the same reason: chat.go reads
-- reasoning under completion_tokens_details and nothing else.
if type(u.completion_tokens_details) == "table"
and u.completion_tokens_details.reasoning_tokens ~= nil then
uses.completion_tokens_details = {
reasoning_tokens = u.completion_tokens_details.reasoning_tokens
}
end
end
if type(chunk.choices) ~= "table" or #chunk.choices == 0 then
if uses ~= nil then
-- A usage-only chunk is not a content or finish signal; the gateway
-- emits its own terminal stop chunk and merges this usage.
return json.encode({ usage = uses, done = false })
end
return ""
end
local ch = chunk.choices[1]
local delta = ch.delta or {}
local finish = (type(ch.finish_reason) == "string" and ch.finish_reason ~= "") and ch.finish_reason or nil
local unified = {
content = delta.content or "",
done = (finish ~= nil),
}
if finish then
unified.finish_reason = finish
end
if uses ~= nil then
unified.usage = uses
end
-- The whole point of this adapter: reasoning is streamed as its own channel
-- and must stay separate. Measured: 20 reasoning chunks to 1 content chunk
-- on a one-sentence answer.
if type(delta.reasoning_content) == "string" then
unified.reasoning_content = delta.reasoning_content
end
if type(delta.tool_calls) == "table" then
unified.tool_calls = delta.tool_calls
end
return json.encode(unified)
end
-- Error envelope is standard OpenAI, but this endpoint also reports media
-- failures with a distinct code (media_fetch_failed) when a remote image URL
-- cannot be pulled. Both carry {error:{message,...}}; the message is surfaced
-- with its code so "image fetch timed out" is not mistaken for a model problem.
function adapter.transform_error(status, body)
local ok, resp = pcall(json.decode, body)
if not ok or type(resp) ~= "table" then return nil end
local e = resp.error
if type(e) ~= "table" then return nil end
if type(e.message) ~= "string" then return nil end
if type(e.code) == "string" and e.code ~= "" and e.code ~= "invalid_api_key" then
return e.message .. " (" .. e.code .. ")"
end
return e.message
end
return adapter