Compare commits

...

24 Commits
main ... v1.6.0

Author SHA1 Message Date
d81bfb1460 chore(version): 1.6.0
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-26 14:21:51 +08:00
a8cff57e24 feat(deploy): 部署前校验 master.key 可解封配置 + llmsproxy -show-secrets
密钥校验(deploy.sh)
- 新增 verify_master_key,在 build/替换任何文件之前执行。失败则二进制与
  配置分毫未动、服务不受影响(已负向验证:缺钥匙、错钥匙两种情况都挡住)
- 走真实的 -show-secrets 解密路径,而不是只检查钥匙文件格式——格式合法
  但内容不匹配(重新生成、恢复了错的备份、换机器)同样会被拒
- 钥匙来源与 config 包一致:LLMS_PROXY_MASTER_KEY 优先,否则
  dirname(runtime_file)/master.key
- 配置里没有密文时跳过并提示(首次加密场景)

-show-secrets
- llmsproxy -show-secrets -config <path>:把凭据打到 stdout 后退出
- 不启动任何东西、不写任何文件(已验证 mtime 不变)
- 加密往返无损:封存前后输出逐字节一致

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-26 14:21:44 +08:00
926b9f6565 feat: WebUI 改为依赖 /api/v1,UI 与 agent 共用一套 API 契约
- sources / sort / keys 三个页面的数据源从 /api/sources 切到 /api/v1/sources
  (写操作仍走 /api/sources:v1 是只读门面,不做变更)
- 编辑弹窗改用 /api/v1/sources/{name}?reveal=credentials(admin-only)取明文 key。
  这是必须的:表单要整体回传源,若不回填 key,改个端口就会把 key 清空。
- 遮蔽视图仍是默认,只有显式 reveal 才返回明文

端到端验证(真浏览器 + 临时实例,非仅 API 测试):
- sources/sort/keys 三页实际发出 GET /api/v1/sources,0 console error
- editSource('demo') → reveal=credentials,#s-key 与 #s-url 正确回填
- 写入往返:改 base_url /v1→/v2 后重开,key 仍在(未被清空)
- 落盘 api_key 明文残留 0、密文 1

测试:+1(reveal 必须 admin,否则任意 user key 可读全部凭据)
变异验证:reveal 去掉 admin 校验 → 403 断言变红

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-26 14:08:44 +08:00
ad28a924a5 feat: 密钥静态加密 + /api/v1 agent 管理 API
密钥加密(写侧封存 / 读侧解封)
- config.yaml 的 sources[].api_key、sources[].headers、keys[].key 落盘即
  AES-256-GCM 密文(enc:v1: 前缀),master.key 复用 runtime store 那把
- 内存里永远是明文:鉴权比对、API 返回新建 key、WebUI 编辑回填都不受影响
- 启动时一次性封存现存明文(幂等,已封存则不写盘);-check 不写文件
- UpsertSourceInYAML 增加 box 参数,新加的源不再以明文落盘
- 解密失败改为硬错误:原先 MustDecrypt 返回密文会被下次 Save 二次封存
  (实测:源 key 18→20、静默损坏),现在启动即失败且配置分毫不动

/api/v1:面向 agent 的管理 API(WebUI 零影响)
- GET /api/v1            机器可读索引,列出每个端点的方法/权限/用途
- GET /api/v1/overview   一次调用看全貌:源 + AUTO 链 + 密钥数 + 健康度
- GET /api/v1/health     仅健康快照
- GET /api/v1/models     按源分组的可路由模型清单
- GET /api/v1/sources[/{name}]  凭据遮蔽后的源
- GET /api/v1/auto       调度链与实时槽位状态
- GET /api/v1/keys       admin only,密钥元数据,绝不回显密钥本身
- 沿用同一套网关 key 鉴权;读端点任意角色,写仍需 admin

测试:15 个新用例(含负向:泄密、越权、写操作必须被拒)
变异验证:maskKey 不遮蔽→红、去掉 admin 校验→红

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-26 14:01:47 +08:00
432129243e chore(version): 1.5.9 2026-09-11 18:19:47 +08:00
c744ee151e feat(opencode): 透传 completion_tokens_details.reasoning_tokens 与上游 cost
回答「opencodego 的用量与费用透传呢」时逐字段核对上游产出,发现 usage 漏了
一项、费用整项丢失。

## 上游实际发什么(实测 opencode.ai/zen/go/v1)

  {
    "choices": [...],
    "usage": { "prompt_tokens": 37, "completion_tokens": 40, "total_tokens": 77,
               "prompt_cache_hit_tokens": 0, "prompt_cache_miss_tokens": 37,
               "prompt_tokens_details": {"cached_tokens": 0},
               "completion_tokens_details": {"reasoning_tokens": 40} },
    "cost": "0"
  }

cost 在**顶层**且是**字符串**。流式时还会单独发一帧:
{"choices":[],"cost":"0"}

## 此前丢了两样

1. completion_tokens_details.reasoning_tokens —— 输出里有多少是思考 token。
   没有它,客户端无法判断 completion_tokens 里多少是可见回答、多少是思考,
   而两者都按输出计费。
2. cost —— 唯一的费用信号,网关整个丢弃。Go 订阅是包月制恒为 "0",
   但 Zen 按量付费模型(以及未来的其它源)有信息量。

顺带修掉一处流式/非流式不一致:命中缓存时上游同时给
prompt_tokens_details.cached_tokens 和独立的 hit/miss,流式路径写成了 elseif,
只留 details,与非流式产出不同(只认独立字段的老客户端会看不到缓存)。

## 实现

- types.TokenUsage += CompletionTokensDetails;UnifiedResponse / UnifiedChunk += Cost
- opencodego/opencodezen 适配器映射两个字段;空 choices 帧改成 usage 与 cost
  都可带(早退只带 usage 会把同帧的 cost 丢干净 —— 新测试先抓到的就是这个)
- Gateway ChatCompletion / ChatChunk += cost,随终帧发(对齐上游的
  {"choices":[],"cost":"0"} 形态)
- Go 兜底 standardSSEChunk 同步支持(openai 系适配器不再漏 reasoning_tokens;
  纯 cost 帧不再被整体丢弃),新增 rawCostString 兼容字符串/数字两种形态

费用只做**搬运**:不解析、不换算、不汇总 —— 它是上游事实,且只有部分上游提供。

## 验证

经网关实测 gozen:deepseek-v4.1-flash,流式与非流式产出逐字段一致:
  prompt_tokens_details.cached_tokens=6784
  prompt_cache_hit_tokens=6784 / miss=148
  completion_tokens_details.reasoning_tokens=16
  cost="0"

测试:TestOpenCodeCostAndReasoningPassthrough(含「无数据不得凭空造字段」反例)、
TestOpenCodeStreamCacheFieldsMatchNonStream、TestTokenUsageMarshalsCompletionTokensDetails。
2026-09-11 18:19:40 +08:00
55f5d7a0f4 chore(version): 1.5.8 2026-09-11 16:54:36 +08:00
c9c09b2ba2 fix(opencode): 采纳客户端真实会话 id + 超窗消息不再被限流措辞封杀
两处都源于同一次排查:pi 到底有没有带会话标识、超窗为什么触发不了压缩。

## 1) 客户端会话 id:pi 一直在发,只是被配置关掉了

之前结论是「通用客户端不发会话 id」——只对了一半。pi 有会话 id,且能发:
pi-ai 的 createClient 在 compat.sendSessionAffinityHeaders 为真时,会把
平台会话 id(uuidv7,整个会话恒定)放到 x-session-affinity /
x-client-request-id / session_id 上。该开关默认 false,而 llmsproxy 的
provider 配置里没开,所以此前一直收不到。

现在网关按优先级采纳:x-session-affinity → x-session-id → session_id →
body 的 prompt_cache_key,并把值经 types.ChatRequest.ClientSession 传到
适配器 meta.client_session。适配器的会号种子优先级变为:
客户端会话 id > 首条 user 消息指纹 > 按源固定。

刻意不采纳 x-client-request-id:名字含 request,部分客户端每请求都换,
拿它当会话会让上游前缀缓存永不命中(pi 总会同时发 x-session-affinity,够用)。

实测:抓 127.0.0.1:8081 的真实 pi 请求,配置打开后收到
x-session-affinity = session_id = x-client-request-id = <子会话 uuid>。
上游缓存确为会话级隔离(同前缀、不同会号:A 冷→命中,B 首次仍为 0),
两个不同 header 值互不命中,反证网关确实采纳了客户端会话 id。

## 2) 超窗消息必须「干净」,否则被同链的限流措辞反向封杀

pi 的 isContextOverflow 先查 NON_OVERFLOW_PATTERNS(/rate limit/、
/too many requests/、Bedrock 前缀),命中就直接判为「非超窗」——**即使
消息里已经有 context_length_exceeded**,pi 也不会压缩重试。

而 AUTO 链的失败消息天生是多 tier 原因的拼接,超窗 tier(gozen 400
maximum context length)常与配额/限流 tier(429 token plan exhausted、
cooling、no free slot)同时出现。此前把 tier 明细原样拼在归一化标记后面,
等于让一条限流 tier 的措辞反过来封杀超窗识别。

现在超窗走独立的干净消息:
  context_length_exceeded: context window is full; reduce the length of
  the messages (gozen/deepseek-v4.1-flash)
只留超窗措辞 + 超窗源名,不带任何其它 tier 的文本。

测试:TestOverflowMessageSurvivesRateLimitedSiblingTier 用 pi 的完整判定
顺序(先 NON_OVERFLOW 后 OVERFLOW)断言同链限流 tier 不再封杀超窗识别;
TestClientSessionFromRequestHeaders / TestClientRequestIDIsNotUsedAsSession /
TestOpenCodePrefersClientSessionID 覆盖会话采纳与优先级。
2026-09-11 16:54:31 +08:00
091c9c961f chore(version): 1.5.7 2026-09-11 16:04:21 +08:00
39b48e556e feat(opencode): per-conversation session via first-user-message fingerprint
Follow-on to the session-stability fix. "Per source" already made the
prefix cache hit, but it puts every conversation into one upstream session.

Using the client's own session id is not possible: capturing real agent
traffic (tcpdump on 127.0.0.1:8081) shows generic OpenAI clients send NO
session identifier at all — no user / session_id / conversation_id /
metadata in the body, and no session header (only X-Stainless-* plus
User-Agent: pi). The x-opencode-session the Go endpoint asks for is an
OpenCode native-client concept that a generic client cannot forward.

Since history is replayed every turn, the FIRST user message is invariant
for the life of a conversation, so it is used as the conversation
fingerprint. The session becomes stable within a conversation and distinct
across conversations; requests with no user message fall back to per-source
stability.

Measured through the gateway (same 5.7k-token prompt): 2nd call
cached_tokens=5504, and an unrelated conversation gets its own session.

Test: TestOpenCodeSessionIsStableForCache covers same-conversation
stability, cross-conversation separation, per-request request ids and the
sessionless fallback.
2026-09-11 16:04:17 +08:00
7df413453b chore(version): 1.5.6 2026-09-11 15:42:26 +08:00
791d198f47 fix(opencode): make x-opencode-session stable so the upstream prefix cache can hit
The opencode adapters derived x-opencode-session from meta.timestamp, i.e. a
brand new session on every request. The upstream prefix cache is
session-scoped, so no request could ever hit it, and the cache fields the
endpoint does report (prompt_tokens_details.cached_tokens,
prompt_cache_hit_tokens/prompt_cache_miss_tokens) always came back 0/absent.

Measured against the live endpoint, same 6032-token prompt:

  fixed session id   -> 2nd call: hit 5888, miss 144
  rotating session id -> every call: hit 0, miss 6032

Fix: derive the session from the source name (stable), matching how
x-opencode-project is already derived. x-opencode-request stays unique per
request — it is only a request identifier, not part of the cache key.
Applied to both opencodego and opencodezen.

Through the gateway the same prompt now reports, on the 2nd call:
  details={'cached_tokens': 5888} hit=5888 miss=144      (non-streaming)
  prompt_tokens_details={'cached_tokens': 5888}          (streaming)

Test: TestOpenCodeSessionIsStableForCache asserts the session is stable
across requests for one source while the request id differs.
2026-09-11 15:42:22 +08:00
1f7d17feac chore(version): 1.5.5 2026-09-11 15:28:17 +08:00
d1a72cd23a fix(opencodego): inject empty reasoning_content on tool-calling turns
v1.5.4 stopped stripping reasoning_content, which fixes clients that send
it — but most agent clients (pi included) never store or replay their
reasoning, keeping only the tool call. OpenCode Go validates the field on
any assistant turn that carries tool_calls and rejects the whole request:

  400 invalid_request_error: The `reasoning_content` in the thinking mode
  must be passed back to the API.

Verified against the live endpoint that an EMPTY string satisfies the
check, so the adapter now fills in "" when a tool-calling assistant turn
has no reasoning_content. Nothing is fabricated: the reasoning shown to
the client is still exactly what the upstream returned for that turn.

Measured: with a tool_call + tool_result history and no reasoning_content,
all 25 configured Go models returned 400 before and all 25 answer
correctly now.

Test: TestOpenCodeGoVsZenReasoning also pins that a plain assistant turn
(no tool calls) must NOT gain the field.
2026-09-11 15:28:14 +08:00
19c4fc3a13 chore(version): 1.5.4 2026-09-11 15:14:14 +08:00
71e9040a45 feat(adapters): split opencode into opencodezen and opencodego
Zen (https://opencode.ai/zen/v1) and Go (https://opencode.ai/zen/go/v1)
are different services with different requirements, and one shared adapter
could not satisfy both.

The decisive difference is reasoning_content:

  * OpenCode Go runs thinking models and REQUIRES the assistant turn's
    reasoning_content to be echoed back. The shared adapter stripped it
    (msg.reasoning_content = nil), so every replay of a thinking turn
    failed with:
      400 invalid_request_error: The `reasoning_content` in the thinking
      mode must be passed back to the API.
    Reproduced directly: the same request with reasoning_content -> 200,
    without -> 400. That is why the Go tier never worked in an agent loop.

  * The Zen free pool must not receive it, so it keeps stripping.

Both adapters keep the earlier fixes they share (never drop an assistant
turn carrying tool_calls; send stream_options only when streaming; role
whitelist; multimodal strip) and the opencode client fingerprint headers —
the Go endpoint additionally REQUIRES x-opencode-session, which the
adapter already sends.

config: localzen -> opencodezen, gozen -> opencodego.
Verified: all 25 gozen models answer correctly through the gateway with a
thinking + tool_call + tool_result history (was 0/25 before), streaming
included; the Zen free models still pass.

Test: TestOpenCodeGoVsZenReasoning pins the Go-keeps / Zen-strips split.
2026-09-11 15:00:45 +08:00
0528941e24 fix(opencode): only send stream_options with stream:true
OpenCode Go (and other strict OpenAI-compatible upstreams) reject a
non-streaming request that carries stream_options with
"stream_options should be set along with stream". The adapter attached it
unconditionally, so every non-stream call through the opencode adapter
failed on those upstreams.

Verified against OpenCode Go: 25/25 configured models now pass a real
completion through the gateway (they previously 400'd).

Test: TestOpenCodeStreamOptionsOnlyWhenStreaming (absent when
non-streaming, include_usage present when streaming).
2026-09-11 13:13:05 +08:00
12af22fda6 chore(version): 1.5.3 2026-09-10 21:57:58 +08:00
9114468753 fix: empty-array content becomes invalid {} on every pass-through adapter; gemini/ollama drop tool calls
Three related forwarding defects found by auditing every adapter with a
tool-calling replay (assistant turn with content:[] + tool_calls).

1) content:[] -> content:{} (all 12 openai-adapter sources, plus
   deepseek/trae/sensenova/agentrouter/github/groq/kimicode/mistral)

   Lua adapters json.decode the request and re-encode it, and an empty Lua
   table is indistinguishable from an empty JSON array — the encoder emits
   {} for both. Agent clients serialise a tool-calling assistant turn with
   no text as content:[], so every pass-through adapter rewrote it to
   content:{} — not valid OpenAI (content is string|array|null). Verified
   against a live upstream: content:[] produced "400 invalid arguments"
   while content:"" was accepted.

   Fixed once at the decode boundary (types.ChatMessage.UnmarshalJSON):
   empty-array content normalises to "" and an empty tool_calls array is
   dropped, so every adapter — including future ones — sees a valid shape.

2) gemini dropped tool_calls and never emitted functionCall /
   functionResponse; the tool role also stayed as an invalid role inside
   contents and system was not moved to systemInstruction.

3) ollama copied only role/content, dropping tool_calls and the call
   attribution entirely (it needs tool_name, not tool_call_id).

Test: TestAdaptersPreserveToolCalls asserts, for every adapter, that the
call id (or function name where the wire format has no id), the function
name, the tool result and the trailing user turn all survive, plus a
negative control for plain text.
2026-09-10 21:57:54 +08:00
611e975456 fix(gateway): 上下文超窗错误归一化,客户端才能压缩重试
问题:上游返回上下文超窗时,客户端(pi)既不压缩也不重试,只看到一条
普通上游错误。链路两处叠加:

1. 措辞不在客户端识别列表里。pi 靠 @earendil-works/pi-ai 的
   OVERFLOW_PATTERNS(25 条正则)判断超窗,而 justworker 返回的是
   「请精简对话历史…(Context window is full…)」——与那 25 条一条都不匹配
   (最接近的 /context window exceeds limit/i 也不命中,因为措辞是 "is full"
   而非 "exceeds limit")。
2. AUTO 链路的每条 tier 错误按 80 字节 OneLine 截断,而 "Context window
   is full" 这类短诊断词常出现在尾部,正好被按字节切掉(直连路径是 160
   才保住)。

修法:
- 新增 overflowMarkers 识别超窗措辞(含中文写法),命中时把客户端可见
  错误归一化为 context_length_exceeded 前缀——它命中 pi 的
  /context[_ ]length[_ ]exceeded/i,超窗因此可被发现并触发压缩重试。
- AUTO 链路每条 tier 错误宽度 80→160,短诊断词不再被截断。

归一化只加前缀,原始诊断信息保留,便于定位是哪一层超窗。

测试:internal/gateway/overflow_err_test.go(5 例,含「标记必须命中 pi
正则」、非超窗不得误标、80 vs 160 宽度的回归对比)。
2026-09-10 20:38:34 +08:00
6733534282 chore(version): 1.5.2 2026-09-10 19:45:31 +08:00
499f0cac2f fix(opencode): never drop an assistant turn that carries tool_calls
An agent client (pi) serialises an assistant turn whose content is only
[thinking, toolCall] as content:[] with tool_calls. The multimodal-strip
pass treated an empty content array as 'nothing left, drop the whole
message' and discarded the tool_calls with it.

The next message is that call's tool result, so it arrived orphaned: the
model saw a result for a call it had never made and re-issued the same
call on every turn — an endless repeated-tool-call loop. Reproduced
against a capture sink: content:[] lost tool_calls, while content:"" and
content:null kept them.

Only messages with neither usable content NOR a tool call now get
dropped. Content that collapses to empty but still has tool_calls or a
tool_call_id is emitted as "" instead.

Test: TestOpenCodeKeepsToolCallWithEmptyContent (plus a negative control
that an image-only message without tool calls is still dropped).
2026-09-10 19:32:47 +08:00
7fb8f96b82 fix(gateway): AUTO scope grants all models — restrict to routing mode only
A key with scope=[AUTO] could previously:
1. request ANY concrete model id directly (hasScopeModel/checkModelScope
   treated AUTO as a wildcard)
2. see the full 56-model list on /v1/models (intersectModels considered
   AUTO as grant-everything)

AUTO now only authorizes the AUTO routing mode. Direct requests to a
specific model require an explicit scope entry.

Also carries agentrouter.lua WAF fingerprint headers (Origin/Referer/
X-Requested-With) already staged on this branch.

Tests: TestHasScopeModelWithSourcePrefix updated; full suite green.
2026-09-10 12:51:40 +08:00
6a34cd6f86 chore(version): 1.4.2 -> 1.5.0 2026-09-06 10:17:43 +08:00
28 changed files with 3448 additions and 87 deletions

View File

@ -1,12 +1,12 @@
{
"name": "modelrouter-gui",
"version": "1.4.2",
"version": "1.6.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "modelrouter-gui",
"version": "1.4.2",
"version": "1.6.0",
"license": "Proprietary",
"devDependencies": {
"electron": "^33.0.0",

View File

@ -1,6 +1,6 @@
{
"name": "modelrouter-gui",
"version": "1.4.2",
"version": "1.6.0",
"description": "ModelRouter Desktop — embedded ModelRouter core with tray",
"author": "ModelRouter",
"main": "main.js",

View File

@ -24,6 +24,7 @@ import (
func main() {
cfgPath := flag.String("config", "config.yaml", "path to gateway config file")
checkOnly := flag.Bool("check", false, "validate the config file and exit (0 = valid, 1 = invalid); nothing is started and no file is written")
showSecrets := flag.Bool("show-secrets", false, "print the config's credentials in the clear and exit; nothing is started and no file is written")
flag.Parse()
// -check is the deploy-time preflight: parse and validate the config
@ -41,6 +42,20 @@ func main() {
return
}
// -show-secrets is the operator escape hatch for a config whose
// credentials are sealed at rest: it prints them to stdout and exits
// without starting anything or writing anything. Nothing else in the
// program prints a credential, and this path never logs to a file.
if *showSecrets {
if _, err := os.Stat(*cfgPath); err != nil {
log.Fatalf("[llmsproxy] show-secrets: %v", err)
}
if err := config.PrintSecrets(*cfgPath); err != nil {
log.Fatalf("[llmsproxy] show-secrets: %v", err)
}
return
}
created, err := config.EnsureDefault(*cfgPath)
if err != nil {
log.Fatalf("[llmsproxy] config: %v", err)

View File

@ -244,6 +244,63 @@ verify_config() {
return 1
}
# ---------- 密钥可解封校验 ----------
# config.yaml 里的凭据以 enc:v1: 密文落盘,解密钥匙是 master.key(或
# LLMS_PROXY_MASTER_KEY)。一旦钥匙丢失/错位,配置在**启动时**才会报错,
# 那时服务已经处于「二进制已替换 + 配置已替换」的半死状态,只能靠回滚。
# 所以在替换任何东西之前先验:能不能用现有的钥匙解开现有的密文。
#
# 两种情形要分开:
# 1. 配置里**有**密文 ⇒ 钥匙必须能解开(否则拒绝部署)
# 2. 配置里**没有**密文 ⇒ 全新安装/首次加密,无需钥匙(但会顺手记一条,
# 因为它意味着这次部署会把明文写成密文,而钥匙必须同时就位)
verify_master_key() {
log "密钥可解封校验"
[[ -f "$TARGET_CONFIG" ]] || return 0 # 配置都不在,交给 verify_config 报错
local sealed
sealed=$(grep -c 'enc:v1:' "$TARGET_CONFIG" 2>/dev/null || echo 0)
if [[ "$sealed" -eq 0 ]]; then
warn "配置内无密文(尚未加密或全为明文)——首次加密需要 master.key 可写"
return 0
fi
# 钥匙来源与 config 包一致:env 优先,否则 runtime_file 同目录的 master.key
local runtime_file key_file
runtime_file=$(awk '/^runtime_file:/{print $2; exit}' "$TARGET_CONFIG")
[[ -n "$runtime_file" ]] || runtime_file="$(dirname "$TARGET_CONFIG")/runtime.json"
key_file="$(dirname "$runtime_file")/master.key"
if [[ -n "${LLMS_PROXY_MASTER_KEY:-}" ]]; then
log " 钥匙来源: LLMS_PROXY_MASTER_KEY(环境变量)"
# 走真实解密路径验证,而不是只检查格式
if out=$("$TARGET_BIN" -show-secrets -config "$TARGET_CONFIG" 2>&1); then
log "✓ 凭据可解封($(echo "$out" | grep -cE '^(source|key) ') 项)"
return 0
fi
err "凭据无法用 LLMS_PROXY_MASTER_KEY 解封,未替换任何文件:"
printf '%s\n' "$out" >&2
return 1
fi
if [[ ! -f "$key_file" ]]; then
err "配置含 $sealed 处密文,但钥匙文件不存在: $key_file"
err " 没有它,部署后服务将无法启动。恢复 master.key 或设置 LLMS_PROXY_MASTER_KEY 后重试。"
return 1
fi
log " 钥匙文件: $key_file ($(stat -c '%A' "$key_file" 2>/dev/null || echo '?'),$(wc -c < "$key_file") 字节)"
if out=$("$TARGET_BIN" -show-secrets -config "$TARGET_CONFIG" 2>&1); then
log "✓ 凭据可解封($(echo "$out" | grep -cE '^(source|key) ') 项)"
return 0
fi
err "凭据无法解密(钥匙与配置不匹配),未替换任何文件:"
printf '%s\n' "$out" | head -5 >&2
err " 密钥来源: $key_file"
err " 常见原因:master.key 被重新生成、恢复了不匹配的备份,或换了机器。"
return 1
}
# ---------- 同步适配器 ----------
sync_adapters() {
log "同步适配器"
@ -393,6 +450,10 @@ main() {
parse_args "$@"
log "═══ llmsproxy 原子部署开始 ═══"
precheck
# 密钥校验放在 build/替换之前:它的失败后果最重(配置里的密文永久解不开),
# 必须在动任何文件之前就挡住。此阶段用的是**当前线上二进制**,它已经
# 支持 -show-secrets(首次引入该子命令的版本除外,此时跳过并提示)。
verify_master_key
build
sync_adapters
backup_old_state

View File

@ -27,6 +27,11 @@ type Config struct {
Auto []ModelScope `yaml:"auto,omitempty"` // AUTO 调度链规则(WebUI 优先级页编辑,chat)
AutoImage []ModelScope `yaml:"auto_image,omitempty"` // AUTO 生图调度链规则(WebUI 优先级页·生图)
Keys []GWKey `yaml:"keys,omitempty"` // 网关密钥(WebUI 密钥页管理)
// box seals credentials (sources' api_key/headers, keys' key) at rest.
// In-memory values are always plaintext; only the bytes on disk are sealed.
// Wired by AttachSecretBox — Load leaves it nil so `-check` and tests stay
// filesystem-free.
box *SecretBox
}
// Defaults applied to any source (YAML or runtime) that leaves a field unset.
@ -192,10 +197,15 @@ func RemoveSourceFromYAML(path, name string) error {
// UpsertSourceInYAML adds or updates a source entry in the YAML config file.
// Uses yaml.Node to preserve the rest of the file's comments and formatting.
func UpsertSourceInYAML(path, name string, src Source) error {
// When box is non-nil the source's credentials are sealed before writing, so a
// newly added source never lands in the file as plaintext.
func UpsertSourceInYAML(path, name string, src Source, box *SecretBox) error {
if path == "" {
return fmt.Errorf("config path is empty")
}
if err := sealSource(&src, box); err != nil {
return err
}
data, err := os.ReadFile(path)
if err != nil {
return err
@ -253,11 +263,17 @@ func UpsertSourceInYAML(path, name string, src Source) error {
// Save writes the current config back to the YAML file (preserving comments
// via yaml.Node round-trip when possible, or full marshaling as fallback).
// Credentials are sealed on the way out and the in-memory copy is restored to
// plaintext afterwards, so callers keep working with usable values.
func (c *Config) Save() error {
if c.Path == "" {
return fmt.Errorf("config path is empty")
}
if err := c.sealInPlace(c.box); err != nil {
return err
}
out, err := yaml.Marshal(c)
c.unsealAfterWrite(c.box)
if err != nil {
return fmt.Errorf("marshal config: %w", err)
}

View File

@ -0,0 +1,272 @@
package config
// Secret handling for config.yaml.
//
// config.yaml is 0644 world-readable by design (ops need to inspect it), so any
// credential in it must not sit there in plaintext. Sources' api_key / headers
// and gateway keys are therefore sealed at rest with the same SecretBox used by
// the runtime store, and unsealed in memory at load time.
//
// The invariant that makes this safe: **in-memory values are always plaintext**,
// and the enc:v1: prefix is what marks a file value as sealed. Read paths that
// predate this (core.resolveSourceKey) already unseal, so only the write side and
// the load-time normalize step are new.
import (
"fmt"
"log"
"os"
"strings"
)
// sealSource seals one source's credentials in place (used by the YAML upsert
// path, which is a package function and therefore has no Config to borrow a box
// from).
func sealSource(s *Source, box *SecretBox) error {
if box == nil {
return nil
}
if s.APIKey != "" && !strings.HasPrefix(s.APIKey, encPrefix) {
v, err := box.Encrypt(s.APIKey)
if err != nil {
return err
}
s.APIKey = v
}
for k, v := range s.Headers {
if v == "" || strings.HasPrefix(v, encPrefix) {
continue
}
e, err := box.Encrypt(v)
if err != nil {
return err
}
s.Headers[k] = e
}
return nil
}
// normalizeSecrets unseals every credential in the freshly parsed config so the
// rest of the program only ever sees plaintext. A value without the enc:v1:
// prefix is left untouched, which keeps hand-written plaintext configs working
// (and is what a pre-encryption config file looks like).
//
// A value that carries the prefix but fails to decrypt is a hard error, not
// something to paper over: returning the ciphertext (MustDecrypt's behavior)
// would let the next Save re-seal it and turn one bad value into permanent,
// compounding corruption. Losing the master key must be loud.
func (c *Config) normalizeSecrets(box *SecretBox) error {
if box == nil {
return nil
}
for i := range c.Sources {
s := &c.Sources[i]
if strings.HasPrefix(s.APIKey, encPrefix) {
v, err := box.Decrypt(s.APIKey)
if err != nil {
return fmt.Errorf("source %q api_key: %w", s.Name, err)
}
s.APIKey = v
}
for k, v := range s.Headers {
if strings.HasPrefix(v, encPrefix) {
d, err := box.Decrypt(v)
if err != nil {
return fmt.Errorf("source %q header %q: %w", s.Name, k, err)
}
s.Headers[k] = d
}
}
}
for i := range c.Keys {
if strings.HasPrefix(c.Keys[i].Key, encPrefix) {
v, err := box.Decrypt(c.Keys[i].Key)
if err != nil {
return fmt.Errorf("gateway key %q: %w", c.Keys[i].Name, err)
}
c.Keys[i].Key = v
}
}
return nil
}
// sealInPlace replaces plaintext credentials with ciphertext for writing. It is
// deliberately a separate step from Marshal: callers that need the plaintext
// (auth comparisons, log output, returning a key to the operator who just
// created it) must not be handed a sealed config by accident.
func (c *Config) sealInPlace(box *SecretBox) error {
if box == nil {
return nil
}
for i := range c.Sources {
s := &c.Sources[i]
if s.APIKey != "" && !strings.HasPrefix(s.APIKey, encPrefix) {
v, err := box.Encrypt(s.APIKey)
if err != nil {
return err
}
s.APIKey = v
}
if len(s.Headers) > 0 {
sealed := make(map[string]string, len(s.Headers))
for k, v := range s.Headers {
if v == "" || strings.HasPrefix(v, encPrefix) {
sealed[k] = v
continue
}
e, err := box.Encrypt(v)
if err != nil {
return err
}
sealed[k] = e
}
s.Headers = sealed
}
}
for i := range c.Keys {
k := &c.Keys[i]
if k.Key != "" && !strings.HasPrefix(k.Key, encPrefix) {
v, err := box.Encrypt(k.Key)
if err != nil {
return err
}
k.Key = v
}
}
return nil
}
// unsealAfterWrite restores plaintext after a sealed marshal so the live process
// keeps working on plaintext values (mirrors Store.persistLocked's dance).
func (c *Config) unsealAfterWrite(box *SecretBox) {
// Save just encrypted every value it can see, so a failure here is
// impossible; ignore the error rather than panic in a write path.
_ = c.normalizeSecrets(box)
}
// hasPlaintextSecrets reports whether any credential in the config is still in
// the clear. Used to decide whether a startup migration write is needed, and to
// warn (without leaking values) when no master key is available.
func (c *Config) hasPlaintextSecrets() bool {
for _, s := range c.Sources {
if s.APIKey != "" && !strings.HasPrefix(s.APIKey, encPrefix) {
return true
}
for _, v := range s.Headers {
if v != "" && !strings.HasPrefix(v, encPrefix) {
return true
}
}
}
for _, k := range c.Keys {
if k.Key != "" && !strings.HasPrefix(k.Key, encPrefix) {
return true
}
}
return false
}
// countPlaintextSecrets returns how many credentials are still in the clear, for
// an operator-facing migration log line that must not print the values.
func (c *Config) countPlaintextSecrets() int {
n := 0
for _, s := range c.Sources {
if s.APIKey != "" && !strings.HasPrefix(s.APIKey, encPrefix) {
n++
}
for _, v := range s.Headers {
if v != "" && !strings.HasPrefix(v, encPrefix) {
n++
}
}
}
for _, k := range c.Keys {
if k.Key != "" && !strings.HasPrefix(k.Key, encPrefix) {
n++
}
}
return n
}
// NormalizeSecretsForRun unseals the loaded config and then seals it back on
// disk if anything was still in the clear. Order matters: Load() read the file
// with ciphertext still in place, so the unseal has to happen before the
// registry (and any Save the startup path performs) sees the values.
func (c *Config) NormalizeSecretsForRun(box *SecretBox) error {
c.AttachSecretBox(box)
if err := c.normalizeSecrets(box); err != nil {
return err
}
return c.migratePlaintextSecrets()
}
// AttachSecretBox wires the encryption box into the config so Save can seal
// credentials. Kept as an explicit call (rather than a constructor argument) so
// config.Load stays usable in contexts that have no filesystem secrets (tests,
// `-check`).
func (c *Config) AttachSecretBox(box *SecretBox) { c.box = box }
// SecretBox returns the wired encryption box, or nil when none is attached.
func (c *Config) SecretBox() *SecretBox { return c.box }
// migratePlaintextSecrets seals any credential still in the clear and writes the
// file once. It is idempotent: a config that is already sealed (or has no
// secrets) is left alone and nothing is written.
func (c *Config) migratePlaintextSecrets() error {
if c.box == nil || c.Path == "" {
return nil
}
if !c.hasPlaintextSecrets() {
return nil
}
n := c.countPlaintextSecrets()
if err := c.Save(); err != nil {
return err
}
log.Printf("[config] sealed %d plaintext credential(s) in %s", n, c.Path)
return nil
}
// PrintSecrets writes the config's credentials to stdout in the clear and
// returns an error only when the file cannot be read or the master key does not
// match. It is the operator counterpart to sealing-at-rest: with keys stored as
// ciphertext, "which key is this source using" must still be answerable.
//
// It deliberately takes a path rather than a *Config so the caller cannot
// accidentally hand it a config that has already been unsealed in memory, and it
// never writes anything.
func PrintSecrets(path string) error {
cfg, err := Load(path)
if err != nil {
return err
}
box, err := NewSecretBox(cfg.RuntimeFile)
if err != nil {
return fmt.Errorf("%w (is master.key present and intact?)", err)
}
if err := cfg.normalizeSecrets(box); err != nil {
return err
}
w := os.Stdout
fmt.Fprintf(w, "# %s — %d source(s), %d gateway key(s)\n", path, len(cfg.Sources), len(cfg.Keys))
for _, s := range cfg.Sources {
fmt.Fprintf(w, "source %-16s api_key=%s\n", s.Name, orNone(s.APIKey))
for k, v := range s.Headers {
if v != "" {
fmt.Fprintf(w, "source %-16s header[%s]=%s\n", s.Name, k, v)
}
}
}
for _, k := range cfg.Keys {
fmt.Fprintf(w, "key %-16s role=%-5s %s\n", k.Name, k.Role, k.Key)
}
fmt.Fprintf(w, "gateway_keys (legacy): %s\n", strings.Join(cfg.GatewayKeys, " "))
return nil
}
func orNone(s string) string {
if s == "" {
return "(unset)"
}
return s
}

View File

@ -0,0 +1,283 @@
package config
import (
"os"
"path/filepath"
"strings"
"testing"
)
// writeMasterKey plants a deterministic master.key next to the given runtime
// file so tests exercise the real box instead of the env-var shortcut.
func writeMasterKey(t *testing.T, runtimeFile string) *SecretBox {
t.Helper()
if err := os.WriteFile(filepath.Join(filepath.Dir(runtimeFile), "master.key"),
[]byte(strings.Repeat("ab", 32)), 0600); err != nil {
t.Fatal(err)
}
box, err := NewSecretBox(runtimeFile)
if err != nil {
t.Fatalf("NewSecretBox: %v", err)
}
return box
}
func TestSaveSealsCredentialsAndLoadUnseals(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "config.yaml")
runtime := filepath.Join(dir, "runtime.json")
box := writeMasterKey(t, runtime)
cfg := &Config{
Path: path,
Listen: "127.0.0.1:0",
Sources: []Source{{
Name: "up",
BaseURL: "http://up/v1",
APIKey: "sk-plaintext-secret",
Adapter: "openai",
Headers: map[string]string{"X-Extra": "header-secret"},
Models: []Model{{ID: "m", Kind: "chat"}},
}},
Keys: []GWKey{{Key: "sk-gw-plain", Role: "admin", Name: "admin"}},
}
cfg.AttachSecretBox(box)
if err := cfg.Save(); err != nil {
t.Fatalf("Save: %v", err)
}
raw, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
text := string(raw)
if strings.Contains(text, "sk-plaintext-secret") {
t.Error("source api_key written in plaintext")
}
if strings.Contains(text, "header-secret") {
t.Error("source header written in plaintext")
}
if strings.Contains(text, "sk-gw-plain") {
t.Error("gateway key written in plaintext")
}
if n := strings.Count(text, encPrefix); n != 3 {
t.Errorf("expected 3 sealed values, found %d", n)
}
// The live in-memory config must still hold plaintext after Save.
if cfg.Sources[0].APIKey != "sk-plaintext-secret" {
t.Errorf("in-memory api_key mutated by Save: %q", cfg.Sources[0].APIKey)
}
if cfg.Keys[0].Key != "sk-gw-plain" {
t.Errorf("in-memory gateway key mutated by Save: %q", cfg.Keys[0].Key)
}
// A fresh load must hand back plaintext again.
loaded, err := Load(path)
if err != nil {
t.Fatalf("Load: %v", err)
}
loaded.AttachSecretBox(box)
loaded.normalizeSecrets(box)
if loaded.Sources[0].APIKey != "sk-plaintext-secret" {
t.Errorf("loaded api_key = %q, want plaintext", loaded.Sources[0].APIKey)
}
if loaded.Sources[0].Headers["X-Extra"] != "header-secret" {
t.Errorf("loaded header = %q, want plaintext", loaded.Sources[0].Headers["X-Extra"])
}
if loaded.Keys[0].Key != "sk-gw-plain" {
t.Errorf("loaded gateway key = %q, want plaintext", loaded.Keys[0].Key)
}
}
func TestPlaintextConfigStillLoads(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "config.yaml")
// A hand-written, pre-encryption config: no enc:v1: anywhere.
body := `listen: 127.0.0.1:0
sources:
- name: legacy
base_url: http://legacy/v1
api_key: sk-written-by-hand
adapter: openai
models:
- id: m
kind: chat
`
if err := os.WriteFile(path, []byte(body), 0644); err != nil {
t.Fatal(err)
}
cfg, err := Load(path)
if err != nil {
t.Fatalf("Load: %v", err)
}
box := writeMasterKey(t, filepath.Join(dir, "runtime.json"))
cfg.AttachSecretBox(box)
cfg.normalizeSecrets(box)
if cfg.Sources[0].APIKey != "sk-written-by-hand" {
t.Errorf("plaintext config value changed: %q", cfg.Sources[0].APIKey)
}
}
func TestMigratePlaintextSecretsIsIdempotent(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "config.yaml")
runtime := filepath.Join(dir, "runtime.json")
box := writeMasterKey(t, runtime)
cfg := &Config{
Path: path,
Listen: "127.0.0.1:0",
Sources: []Source{{Name: "up", BaseURL: "http://up/v1", APIKey: "sk-clear", Adapter: "openai", Models: []Model{{ID: "m"}}}},
}
cfg.AttachSecretBox(box)
if err := cfg.migratePlaintextSecrets(); err != nil {
t.Fatalf("first migrate: %v", err)
}
first, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(first), encPrefix) {
t.Fatal("migration did not seal the value")
}
// In-memory must be plaintext so the running process keeps working.
if cfg.Sources[0].APIKey != "sk-clear" {
t.Errorf("in-memory api_key = %q, want plaintext", cfg.Sources[0].APIKey)
}
// Second run: already sealed => no write.
before, _ := os.Stat(path)
if err := cfg.migratePlaintextSecrets(); err != nil {
t.Fatalf("second migrate: %v", err)
}
after, _ := os.Stat(path)
if before.ModTime() != after.ModTime() {
t.Error("second migrate rewrote an already-sealed config")
}
}
func TestSealedValueDoesNotDoubleEncrypt(t *testing.T) {
dir := t.TempDir()
box := writeMasterKey(t, filepath.Join(dir, "runtime.json"))
cfg := &Config{Path: filepath.Join(dir, "config.yaml"), Sources: []Source{{Name: "a", BaseURL: "http://a"}}}
cfg.AttachSecretBox(box)
once, err := box.Encrypt("sk-x")
if err != nil {
t.Fatal(err)
}
cfg.Sources[0].APIKey = once
if err := cfg.Save(); err != nil {
t.Fatal(err)
}
got, err := box.Decrypt(cfg.Sources[0].APIKey)
if err != nil {
t.Fatalf("value became undecryptable: %v", err)
}
if got != "sk-x" {
t.Errorf("round trip = %q, want sk-x", got)
}
}
func TestUpsertSourceInYAMLSealsCredentials(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "config.yaml")
box := writeMasterKey(t, filepath.Join(dir, "runtime.json"))
if err := os.WriteFile(path, []byte("listen: 127.0.0.1:0\nsources: []\n"), 0644); err != nil {
t.Fatal(err)
}
src := Source{Name: "new", BaseURL: "http://new/v1", APIKey: "sk-fresh", Adapter: "openai", Models: []Model{{ID: "m"}}}
if err := UpsertSourceInYAML(path, src.Name, src, box); err != nil {
t.Fatalf("UpsertSourceInYAML: %v", err)
}
raw, _ := os.ReadFile(path)
if strings.Contains(string(raw), "sk-fresh") {
t.Error("newly added source stored its api_key in plaintext")
}
if !strings.Contains(string(raw), encPrefix) {
t.Error("newly added source was not sealed")
}
}
func TestCountPlaintextSecrets(t *testing.T) {
cfg := &Config{
Sources: []Source{
{Name: "a", APIKey: encPrefix + "abc", Headers: map[string]string{"H": encPrefix + "x"}},
{Name: "b", APIKey: "sk-in-clear", Headers: map[string]string{"H2": "clear-too"}},
},
Keys: []GWKey{{Key: "sk-gw-clear"}},
}
if got := cfg.countPlaintextSecrets(); got != 3 {
t.Errorf("countPlaintextSecrets = %d, want 3", got)
}
if !cfg.hasPlaintextSecrets() {
t.Error("hasPlaintextSecrets = false, want true")
}
cfg.Sources[1].APIKey = encPrefix + "y"
cfg.Sources[1].Headers["H2"] = encPrefix + "z"
cfg.Keys[0].Key = encPrefix + "k"
if cfg.hasPlaintextSecrets() {
t.Error("hasPlaintextSecrets = true after sealing everything")
}
}
// TestNormalizeSecretsFailsLoudlyOnWrongMasterKey is the negative case that
// guards the migration: with a wrong key, a sealed value must NOT be handed
// back as ciphertext for a later re-seal (that compounds corruption and turns
// one lost key file into permanently unusable config).
func TestNormalizeSecretsFailsLoudlyOnWrongMasterKey(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "config.yaml")
runtime := filepath.Join(dir, "runtime.json")
good := writeMasterKey(t, runtime)
cfg := &Config{
Path: path,
Listen: "127.0.0.1:0",
Sources: []Source{{Name: "up", BaseURL: "http://up/v1", APIKey: "sk-secret", Adapter: "openai", Models: []Model{{ID: "m"}}}},
}
cfg.AttachSecretBox(good)
if err := cfg.Save(); err != nil {
t.Fatal(err)
}
sealedOnce, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
// A different master key must be rejected, loudly. NewSecretBox derives the
// key path from the RUNTIME file's directory, so the wrong key has to live
// in a different directory — otherwise it would read the good master.key.
otherDir := t.TempDir()
if err := os.WriteFile(filepath.Join(otherDir, "master.key"),
[]byte(strings.Repeat("cd", 32)), 0600); err != nil {
t.Fatal(err)
}
badBox, err := NewSecretBox(filepath.Join(otherDir, "runtime.json"))
if err != nil {
t.Fatal(err)
}
loaded, err := Load(path)
if err != nil {
t.Fatal(err)
}
loaded.AttachSecretBox(badBox)
if err := loaded.NormalizeSecretsForRun(badBox); err == nil {
t.Fatal("expected an error with the wrong master key, got nil")
}
// Crucially: the file must be untouched — no second seal on top.
after, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if string(after) != string(sealedOnce) {
t.Error("config file was rewritten despite the decrypt failure")
}
if n := strings.Count(string(after), encPrefix); n != strings.Count(string(sealedOnce), encPrefix) {
t.Errorf("sealed count changed %d -> %d (double encryption)", strings.Count(string(sealedOnce), encPrefix), n)
}
}

View File

@ -7,6 +7,7 @@ import (
"crypto/rand"
"encoding/hex"
"fmt"
"log"
"os"
"path/filepath"
"sort"
@ -53,6 +54,10 @@ func NewFromConfig(cfg *config.Config) (*Core, error) {
return nil, fmt.Errorf("lua vm: %w", err)
}
c.store = config.NewStore(cfg.RuntimeFile)
// Share one box between the runtime store and config.yaml so a single
// master.key seals both files. config.Load left the config holding
// ciphertext (if it was sealed); unseal it now that the box exists.
cfg.AttachSecretBox(c.store.SecretBox())
if err := c.store.Load(); err != nil {
return nil, fmt.Errorf("runtime store: %w", err)
}
@ -74,6 +79,12 @@ func NewFromConfig(cfg *config.Config) (*Core, error) {
if err := c.seedPresetTemplates(); err != nil {
return nil, fmt.Errorf("seed preset templates: %w", err)
}
// Seal any credential still in the clear in config.yaml. Idempotent: an
// already-sealed config is not rewritten, so a normal restart writes
// nothing. This is the only place that rewrites the file on startup.
if err := cfg.NormalizeSecretsForRun(c.store.SecretBox()); err != nil {
return nil, fmt.Errorf("normalize secrets: %w", err)
}
return c, nil
}
@ -439,6 +450,11 @@ func (c *Core) mergedSources() []config.Source {
}
// resolveSourceKey applies api_key_env and decrypts enc:v1: ciphertext.
// Config values are already unsealed at startup (Config.NormalizeSecretsForRun),
// so the decrypt branch is the belt-and-braces path for a source that arrived
// already sealed through another route. A failure there leaves the ciphertext in
// place, which makes the upstream reject the key loudly rather than sending an
// empty Authorization header that might look like a config-less source.
func (c *Core) resolveSourceKey(s config.Source) config.Source {
if s.APIKeyEnv != "" {
if v := os.Getenv(s.APIKeyEnv); v != "" {
@ -447,7 +463,11 @@ func (c *Core) resolveSourceKey(s config.Source) config.Source {
return s
}
if box := c.store.SecretBox(); box != nil && strings.HasPrefix(s.APIKey, "enc:v1:") {
s.APIKey = box.MustDecrypt(s.APIKey)
if v, err := box.Decrypt(s.APIKey); err == nil {
s.APIKey = v
} else {
log.Printf("[core] source %s: api_key decrypt failed: %v", s.Name, err)
}
}
return s
}
@ -655,7 +675,7 @@ func (c *Core) AddSource(src config.Source) error {
if err := normalizeSource(&src); err != nil {
return err
}
if err := config.UpsertSourceInYAML(c.cfg.Path, src.Name, src); err != nil {
if err := config.UpsertSourceInYAML(c.cfg.Path, src.Name, src, c.cfg.SecretBox()); err != nil {
return err
}
// Update in-memory Sources so mergedSources() finds the entry.

415
internal/gateway/apiv1.go Normal file
View File

@ -0,0 +1,415 @@
package gateway
// Agent-facing management API.
//
// The Web UI has always driven the gateway through /api/*, so the management
// surface already exists. What it lacked was anything an agent could *rely* on:
// no way to discover the contract, no single call that answers "what is the
// current state", and per-endpoint response shapes that make scripting brittle.
//
// This file adds a versioned, self-describing facade under /api/v1 without
// touching the existing endpoints, so the Web UI keeps working unchanged while
// agents get a stable contract:
//
// GET /api/v1 — machine-readable index of every endpoint
// GET /api/v1/overview — one call: sources + auto chain + keys + health
// GET /api/v1/sources — sources, credentials masked
// GET /api/v1/sources/{name} — one source
// GET /api/v1/auto — scheduling chain + live slot states
// GET /api/v1/keys — key metadata (never the secret)
// GET /api/v1/models — every model id the gateway can route
// GET /api/v1/health — per-source health, no auth needed beyond the key
//
// Auth is the same gateway key as everywhere else (Authorization: Bearer, or
// ?api_key=, or the gw_key cookie). Read endpoints accept any role; writes
// still require admin, enforced by the same middleware the UI goes through.
import (
"errors"
"net/http"
"net/url"
"sort"
"strings"
"llmsproxy/internal/config"
)
// apiV1Routes dispatches /api/v1/*. Kept separate from routes() so the original
// switch stays readable and the versioned surface can grow on its own.
func (g *Gateway) apiV1Routes(w http.ResponseWriter, r *http.Request) {
path := strings.TrimPrefix(r.URL.Path, "/api/v1")
path = strings.Trim(path, "/")
switch {
case path == "":
g.apiV1Index(w, r)
case path == "overview":
g.apiV1Overview(w, r)
case path == "health":
g.apiV1Health(w, r)
case path == "models":
g.apiV1Models(w, r)
case path == "sources":
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"sources": maskSources(g.core.Sources())})
case strings.HasPrefix(path, "sources/"):
name, err := decodePathSegment(strings.TrimPrefix(path, "sources/"))
if err != nil {
writeError(w, http.StatusBadRequest, "invalid_request", err.Error())
return
}
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed",
"use PUT/DELETE on /api/sources/{name} to modify a source")
return
}
for _, s := range g.core.Sources() {
if s.Name == name {
// ?reveal=credentials is admin-only and is what the Web UI's
// edit dialog uses: a form that round-trips a source must be
// able to show the current key, otherwise saving an unrelated
// field would blank it. Everything else stays masked.
if r.URL.Query().Get("reveal") == "credentials" {
if reqRole(r.Context()) != "admin" {
writeError(w, http.StatusForbidden, "forbidden",
"admin role required to reveal credentials")
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"source": s})
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"source": maskSource(s)})
return
}
}
writeError(w, http.StatusNotFound, "not_found", "no such source: "+name)
case path == "auto":
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{
"rules": g.core.AutoRules(),
"image_rules": g.core.AutoImageRules(),
"states": g.core.AutoSlotStates(),
})
case path == "keys":
if reqRole(r.Context()) != "admin" {
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
return
}
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed",
"use POST/PUT/DELETE on /api/keys to manage keys")
return
}
keys := g.core.ListKeys()
out := make([]map[string]interface{}, 0, len(keys))
for _, k := range keys {
out = append(out, map[string]interface{}{
"name": k.Name,
"role": k.Role,
"models": k.Models,
"note": k.Note,
"created_at": k.CreatedAt,
"seed": k.Seed,
// The secret itself is never echoed. An operator that needs it
// already has it from creation time or from config.yaml.
"key_prefix": maskKey(k.Key),
})
}
writeJSON(w, http.StatusOK, map[string]interface{}{"keys": out})
default:
g.apiV1Index(w, r)
}
}
// apiV1Index is the discovery document: it names every endpoint with its
// method, auth requirement and purpose, so an agent does not have to scrape the
// HTML to find out what it can call.
func (g *Gateway) apiV1Index(w http.ResponseWriter, r *http.Request) {
type ep struct {
Method string `json:"method"`
Path string `json:"path"`
Auth string `json:"auth"`
Summary string `json:"summary"`
WriteEffect string `json:"writes,omitempty"`
}
index := map[string]interface{}{
"api_version": "v1",
"description": "llmsproxy management API. Authenticate with a gateway key: " +
"'Authorization: Bearer <key>', '?api_key=<key>', or the gw_key cookie. " +
"Read endpoints accept any role; mutations require an admin key.",
"discovery": []ep{
{Method: "GET", Path: "/api/v1", Auth: "any", Summary: "this document"},
{Method: "GET", Path: "/api/v1/overview", Auth: "any",
Summary: "current state in one call: sources, auto chain, key count, source health"},
{Method: "GET", Path: "/api/v1/health", Auth: "any",
Summary: "per-source health snapshot only"},
{Method: "GET", Path: "/api/v1/models", Auth: "any",
Summary: "every model id the gateway can route, grouped by owning source"},
{Method: "GET", Path: "/api/v1/sources", Auth: "any",
Summary: "all sources with credentials masked"},
{Method: "GET", Path: "/api/v1/sources/{name}", Auth: "any", Summary: "one source"},
{Method: "GET", Path: "/api/v1/auto", Auth: "any",
Summary: "AUTO scheduling chain and live per-slot state"},
{Method: "GET", Path: "/api/v1/keys", Auth: "admin",
Summary: "gateway key metadata; secrets are never returned"},
{Method: "GET", Path: "/api/sources", Auth: "admin", Summary: "raw source list (includes api_key)"},
{Method: "POST", Path: "/api/sources", Auth: "admin", Summary: "add or replace a source",
WriteEffect: "writes config.yaml (api_key sealed at rest)"},
{Method: "PUT", Path: "/api/sources/{name}", Auth: "admin", Summary: "update one source",
WriteEffect: "writes config.yaml"},
{Method: "DELETE", Path: "/api/sources/{name}", Auth: "admin", Summary: "delete a source",
WriteEffect: "writes config.yaml"},
{Method: "GET", Path: "/api/auto", Auth: "any", Summary: "scheduling chain (readable by any role)"},
{Method: "PUT", Path: "/api/auto", Auth: "admin", Summary: "replace the scheduling chain",
WriteEffect: "writes config.yaml (the AUTO chain lives here)"},
{Method: "GET", Path: "/api/keys", Auth: "admin", Summary: "gateway keys"},
{Method: "POST", Path: "/api/keys", Auth: "admin", Summary: "create a gateway key",
WriteEffect: "writes config.yaml"},
{Method: "DELETE", Path: "/api/keys/{name}", Auth: "admin", Summary: "delete a gateway key",
WriteEffect: "writes config.yaml"},
{Method: "GET", Path: "/api/status", Auth: "any", Summary: "per-source health detail"},
{Method: "GET", Path: "/api/stats", Auth: "any", Summary: "usage aggregates"},
{Method: "GET", Path: "/api/stats/records", Auth: "any", Summary: "paged request records"},
{Method: "GET", Path: "/api/adapters", Auth: "admin", Summary: "installed Lua adapters"},
{Method: "GET", Path: "/api/source_templates", Auth: "admin", Summary: "source templates"},
{Method: "POST", Path: "/v1/chat/completions", Auth: "any", Summary: "OpenAI-compatible inference"},
},
"conventions": map[string]interface{}{
"errors": "{ \"error\": { \"type\": <code>, \"message\": <text> } }",
"path_escape": "URL-encode source and key names; {name} is a single path segment",
"idempotency": "POST /api/sources and PUT /api/sources/{name} both upsert by name",
"config_truth": "all configuration lives in config.yaml; API writes are persisted immediately",
"credentials": "credentials are masked by default. GET /api/v1/sources/{name}?reveal=credentials " +
"returns them in the clear and is admin-only — the Web UI edit dialog uses it, because a form " +
"that round-trips a source must show the current key or saving another field would blank it.",
},
}
writeJSON(w, http.StatusOK, index)
}
// apiV1Overview is the "what is the current state" call an agent makes first.
// One round trip instead of five.
func (g *Gateway) apiV1Overview(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
return
}
sources := g.core.Sources()
names := make([]string, 0, len(sources))
modelCount := 0
for _, s := range sources {
names = append(names, s.Name)
modelCount += len(s.Models)
}
keys := g.core.ListKeys()
adminCount, userCount := 0, 0
for _, k := range keys {
if k.Role == "admin" {
adminCount++
} else {
userCount++
}
}
writeJSON(w, http.StatusOK, map[string]interface{}{
"api_version": "v1",
"summary": map[string]interface{}{
"source_count": len(sources),
"model_count": modelCount,
"gateway_key_count": len(keys),
"admin_key_count": adminCount,
"user_key_count": userCount,
"auto_slots": len(g.core.AutoRules()),
},
"sources": maskSources(sources),
"auto": g.core.AutoRules(),
"auto_image": g.core.AutoImageRules(),
"health": g.sourceHealthBrief(),
"caller": map[string]interface{}{
"role": reqRole(r.Context()),
"key": maskKey(reqKey(r.Context())),
},
})
}
func (g *Gateway) apiV1Health(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"sources": g.sourceHealthBrief()})
}
// apiV1Models lists routable model ids per source, which is what an agent needs
// to build a valid request — /v1/models flattens them and hides the owner.
func (g *Gateway) apiV1Models(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
return
}
bySource := map[string][]string{}
var all []string
for _, s := range g.core.Sources() {
for _, m := range s.Models {
bySource[s.Name] = append(bySource[s.Name], m.ID)
all = append(all, m.ID)
}
}
sort.Strings(all)
all = dedupeStrings(all)
writeJSON(w, http.StatusOK, map[string]interface{}{
"count": len(all),
"models": all,
"by_source": bySource,
"note": "request a model as \"<source>:<model>\" to pin one source, or the bare id to let the gateway choose",
})
}
// ---- helpers ----
// maskSources returns sources with credentials replaced by a presence marker.
// The UI still uses /api/sources (which returns real values, because its edit
// form round-trips them); this is the safe view for programmatic callers.
func maskSources(srcs []config.Source) []map[string]interface{} {
out := make([]map[string]interface{}, 0, len(srcs))
for _, s := range srcs {
out = append(out, maskSource(s))
}
return out
}
func maskSource(s config.Source) map[string]interface{} {
return map[string]interface{}{
"name": s.Name,
"base_url": s.BaseURL,
"adapter": s.Adapter,
"endpoint": s.Endpoint,
"image_endpoint": s.ImageEndpoint,
"api_key": maskKey(s.APIKey),
"api_key_set": s.APIKey != "",
"models": s.Models,
"headers": maskHeaders(s.Headers),
"proxy_url": s.ProxyURL,
"meta": s.Meta,
"temperature": s.Temperature,
"max_tokens": s.MaxTokens,
"max_concurrent": s.MaxConcurrent,
"rpm": s.RPM,
}
}
func maskKey(k string) string {
if k == "" {
return ""
}
if strings.HasPrefix(k, "enc:v1:") {
return "(sealed)"
}
if len(k) <= 10 {
return k[:2] + "…"
}
return k[:6] + "…" + k[len(k)-4:]
}
func maskHeaders(h map[string]string) map[string]string {
if h == nil {
return nil
}
out := make(map[string]string, len(h))
for k, v := range h {
out[k] = maskKey(v)
}
return out
}
// sourceHealthBrief reuses the same registry status the UI's /api/status shows,
// so the agent view and the UI view cannot drift apart. Recent-traffic counters
// come from the same stats window, because a source that is actually serving
// traffic must never look down just because a probe was rate-limited.
func (g *Gateway) sourceHealthBrief() []map[string]interface{} {
sts := g.core.Registry().Status()
recent := g.stats.SourceRecent(300)
avgs := g.stats.SourceAverages(300)
for i := range sts {
if v, ok := recent[sts[i].Name]; ok {
sts[i].RecentOK = v[0]
sts[i].RecentErr = v[1]
}
if a, ok := avgs[sts[i].Name]; ok {
sts[i].AvgFirstByteMs = a.AvgFirstByteMs
sts[i].AvgTokPerS = a.AvgTokPerS
}
}
out := make([]map[string]interface{}, 0, len(sts))
for _, s := range sts {
row := map[string]interface{}{
"name": s.Name,
"adapter": s.Adapter,
"healthy": s.Healthy,
"available": s.Available,
"live_available": s.LiveAvailable,
"model_count": len(s.Models),
}
if s.LastError != "" {
row["last_error"] = s.LastError
}
if s.LastChecked > 0 {
row["last_checked"] = s.LastChecked
}
if s.RecentOK > 0 || s.RecentErr > 0 {
row["recent_ok"] = s.RecentOK
row["recent_err"] = s.RecentErr
}
if s.AvgFirstByteMs > 0 {
row["avg_first_byte_ms"] = s.AvgFirstByteMs
}
if s.AvgTokPerS > 0 {
row["avg_tok_per_s"] = s.AvgTokPerS
}
if s.FailCount > 0 {
row["fail_count"] = s.FailCount
}
if s.Permanent {
row["permanent"] = true
}
out = append(out, row)
}
return out
}
// decodePathSegment URL-decodes one path segment and rejects an empty result,
// so a name containing a slash cannot be silently mis-resolved.
func decodePathSegment(seg string) (string, error) {
s, err := url.PathUnescape(seg)
if err != nil {
return "", err
}
if s == "" {
return "", errEmptySegment
}
return s, nil
}
func dedupeStrings(in []string) []string {
out := in[:0]
var last string
for i, s := range in {
if i == 0 || s != last {
out = append(out, s)
}
last = s
}
return out
}
// errEmptySegment marks a path like /api/v1/sources/ with no name after it.
var errEmptySegment = errors.New("empty path segment")

View File

@ -0,0 +1,299 @@
package gateway
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"llmsproxy/internal/config"
)
// v1Gateway builds a gateway with one source and two keys, and returns the
// secrets so a test can assert they never appear in a response.
func v1Gateway(t *testing.T) (*Gateway, string, string) {
t.Helper()
g := newTestGateway(t, config.Source{
Name: "up",
BaseURL: "http://up.example/v1",
APIKey: "sk-up-secret",
Adapter: "openai",
Models: []config.Model{{ID: "m1", Kind: "chat"}},
})
// The shared helper only seeds one admin key; add a user key through the
// core so role-gated endpoints have something to reject.
if _, err := g.core.CreateKey("agent", "user", nil, "test agent key"); err != nil {
t.Fatalf("CreateKey: %v", err)
}
rec := doReq(t, g, http.MethodGet, "/api/keys", "")
var doc struct {
Keys []config.GWKey `json:"keys"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &doc); err != nil {
t.Fatalf("decode keys: %v", err)
}
userKey := ""
for _, k := range doc.Keys {
if k.Role == "user" {
userKey = k.Key
}
}
if userKey == "" {
t.Fatal("no user key was created")
}
return g, "sk-test", userKey
}
func decodeJSON(t *testing.T, body string, v interface{}) {
t.Helper()
if err := json.Unmarshal([]byte(body), v); err != nil {
t.Fatalf("decode %s: %v", body, err)
}
}
func TestAPIV1IndexIsDiscoverable(t *testing.T) {
g, _, _ := v1Gateway(t)
rec := doReq(t, g, http.MethodGet, "/api/v1", "")
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200: %s", rec.Code, rec.Body.String())
}
var doc struct {
APIVersion string `json:"api_version"`
Description string `json:"description"`
Discovery []struct {
Method string `json:"method"`
Path string `json:"path"`
Auth string `json:"auth"`
Summary string `json:"summary"`
} `json:"discovery"`
Conventions map[string]interface{} `json:"conventions"`
}
decodeJSON(t, rec.Body.String(), &doc)
if doc.APIVersion != "v1" {
t.Errorf("api_version = %q", doc.APIVersion)
}
if doc.Description == "" {
t.Error("index should carry a usage description")
}
if len(doc.Discovery) == 0 {
t.Fatal("discovery list is empty")
}
want := map[string]bool{
"/api/v1/overview": false, "/api/v1/sources": false,
"/api/v1/auto": false, "/api/v1/models": false,
"/api/v1/health": false, "/api/v1/keys": false,
"/api/sources": false, "/api/auto": false, "/v1/chat/completions": false,
}
for _, e := range doc.Discovery {
if _, ok := want[e.Path]; ok {
want[e.Path] = true
}
if e.Method == "" || e.Path == "" || e.Auth == "" || e.Summary == "" {
t.Errorf("incomplete discovery entry: %+v", e)
}
}
for p, found := range want {
if !found {
t.Errorf("discovery is missing %s", p)
}
}
if doc.Conventions["errors"] == nil {
t.Error("conventions should document the error shape")
}
}
func TestAPIV1OverviewSummarisesState(t *testing.T) {
g, _, _ := v1Gateway(t)
rec := doReq(t, g, http.MethodGet, "/api/v1/overview", "")
if rec.Code != http.StatusOK {
t.Fatalf("status = %d: %s", rec.Code, rec.Body.String())
}
var doc struct {
Summary struct {
SourceCount int `json:"source_count"`
ModelCount int `json:"model_count"`
GatewayKeyCount int `json:"gateway_key_count"`
} `json:"summary"`
Sources []map[string]interface{} `json:"sources"`
Auto []map[string]interface{} `json:"auto"`
Health []map[string]interface{} `json:"health"`
Caller struct {
Role string `json:"role"`
} `json:"caller"`
}
decodeJSON(t, rec.Body.String(), &doc)
if doc.Summary.SourceCount != 1 {
t.Errorf("source_count = %d, want 1", doc.Summary.SourceCount)
}
if doc.Summary.ModelCount != 1 {
t.Errorf("model_count = %d, want 1", doc.Summary.ModelCount)
}
if doc.Summary.GatewayKeyCount < 2 {
t.Errorf("gateway_key_count = %d, want >= 2", doc.Summary.GatewayKeyCount)
}
if len(doc.Sources) != 1 || doc.Sources[0]["name"] != "up" {
t.Errorf("sources = %+v", doc.Sources)
}
if doc.Caller.Role != "admin" {
t.Errorf("caller.role = %q, want admin", doc.Caller.Role)
}
}
func TestAPIV1NeverEchoesSecrets(t *testing.T) {
g, admin, user := v1Gateway(t)
for _, path := range []string{"/api/v1", "/api/v1/overview", "/api/v1/sources", "/api/v1/sources/up", "/api/v1/keys", "/api/v1/models", "/api/v1/health", "/api/v1/auto"} {
rec := doReq(t, g, http.MethodGet, path, "")
if rec.Code != http.StatusOK {
t.Fatalf("%s status = %d: %s", path, rec.Code, rec.Body.String())
}
if strings.Contains(rec.Body.String(), "sk-up-secret") {
t.Errorf("%s leaked the source api_key", path)
}
if admin != "" && strings.Contains(rec.Body.String(), admin) {
t.Errorf("%s leaked the admin gateway key", path)
}
if strings.Contains(rec.Body.String(), user) {
t.Errorf("%s leaked the user gateway key", path)
}
}
// Masking must still be useful: it says a key is configured.
rec := doReq(t, g, http.MethodGet, "/api/v1/sources/up", "")
if !strings.Contains(rec.Body.String(), `"api_key_set":true`) {
t.Error("masked source should report that a key is configured")
}
}
func TestAPIV1KeysRequiresAdminAndMasks(t *testing.T) {
g, _, user := v1Gateway(t)
// Authenticate as the user key by swapping the shared helper's header.
req := newAuthedRequest(t, http.MethodGet, "/api/v1/keys", user)
rec := serveViaHandler(t, g, req)
if rec.Code != http.StatusForbidden {
t.Errorf("user role on /api/v1/keys = %d, want 403: %s", rec.Code, rec.Body.String())
}
}
func TestAPIV1SourcesNotFound(t *testing.T) {
g, _, _ := v1Gateway(t)
rec := doReq(t, g, http.MethodGet, "/api/v1/sources/nope", "")
if rec.Code != http.StatusNotFound {
t.Errorf("status = %d, want 404: %s", rec.Code, rec.Body.String())
}
}
func TestAPIV1RejectsWrites(t *testing.T) {
g, _, _ := v1Gateway(t)
for _, m := range []string{http.MethodPost, http.MethodPut, http.MethodDelete, http.MethodPatch} {
rec := doReq(t, g, m, "/api/v1/sources", `{}`)
if rec.Code == http.StatusOK {
t.Errorf("%s /api/v1/sources returned 200; the v1 read facade must not mutate", m)
}
}
// And the source must be untouched.
rec := doReq(t, g, http.MethodGet, "/api/v1/sources", "")
if !strings.Contains(rec.Body.String(), `"name":"up"`) {
t.Error("the source disappeared after a rejected write")
}
}
func TestAPIV1ModelsGroupsBySource(t *testing.T) {
g, _, _ := v1Gateway(t)
rec := doReq(t, g, http.MethodGet, "/api/v1/models", "")
if rec.Code != http.StatusOK {
t.Fatalf("status = %d", rec.Code)
}
var doc struct {
Count int `json:"count"`
Models []string `json:"models"`
BySource map[string][]string `json:"by_source"`
}
decodeJSON(t, rec.Body.String(), &doc)
if doc.Count != 1 || doc.Models[0] != "m1" {
t.Errorf("models = %+v", doc.Models)
}
if len(doc.BySource["up"]) != 1 || doc.BySource["up"][0] != "m1" {
t.Errorf("by_source = %+v", doc.BySource)
}
}
func TestAPIV1RequiresAuthentication(t *testing.T) {
g, _, _ := v1Gateway(t)
for _, path := range []string{"/api/v1", "/api/v1/overview", "/api/v1/sources", "/api/v1/health", "/api/v1/models"} {
req := newRequest(http.MethodGet, path, "")
rec := serveViaHandler(t, g, req)
if rec.Code != http.StatusUnauthorized {
t.Errorf("%s without a key = %d, want 401", path, rec.Code)
}
}
}
func TestMaskKeyHidesTheSecret(t *testing.T) {
cases := map[string]string{
"": "",
"ab": "ab…",
"sk-gw-abcdefghijkl": "sk-gw-…ijkl",
"enc:v1:AAAA": "(sealed)",
}
for in, want := range cases {
if got := maskKey(in); got != want {
t.Errorf("maskKey(%q) = %q, want %q", in, got, want)
}
}
// The masked form must not reveal the middle of the secret.
full := maskKey("sk-gw-1234567890abcdef")
if strings.Contains(full, "4567890abcdef") {
t.Errorf("maskKey leaked the middle: %q", full)
}
}
// newRequest builds an unauthenticated request.
func newRequest(method, path, body string) *http.Request {
req, _ := http.NewRequest(method, path, strings.NewReader(body))
if body != "" {
req.Header.Set("Content-Type", "application/json")
}
return req
}
// newAuthedRequest builds a request carrying the given bearer key.
func newAuthedRequest(t *testing.T, method, path, key string) *http.Request {
t.Helper()
req := newRequest(method, path, "")
req.Header.Set("Authorization", "Bearer "+key)
return req
}
// serveViaHandler pushes a request through the full handler chain (auth included).
func serveViaHandler(t *testing.T, g *Gateway, req *http.Request) *httptest.ResponseRecorder {
t.Helper()
rec := httptest.NewRecorder()
g.Handler().ServeHTTP(rec, req)
return rec
}
// TestAPIV1RevealRequiresAdmin: the only way to read a key in the clear is the
// explicit admin-only reveal. Without the role check this endpoint would hand
// every source credential to any valid (even user-scoped) key.
func TestAPIV1RevealRequiresAdmin(t *testing.T) {
g, _, user := v1Gateway(t)
rec := serveViaHandler(t, g, newAuthedRequest(t, http.MethodGet, "/api/v1/sources/up?reveal=credentials", user))
if rec.Code != http.StatusForbidden {
t.Errorf("user reveal = %d, want 403: %s", rec.Code, rec.Body.String())
}
if strings.Contains(rec.Body.String(), "sk-up-secret") {
t.Error("a forbidden reveal still leaked the key")
}
rec = doReq(t, g, http.MethodGet, "/api/v1/sources/up?reveal=credentials", "")
if rec.Code != http.StatusOK {
t.Fatalf("admin reveal = %d: %s", rec.Code, rec.Body.String())
}
if !strings.Contains(rec.Body.String(), "sk-up-secret") {
t.Error("admin reveal should return the key in the clear (the edit form needs it)")
}
// The masked default view must still hide it on the same path.
rec = doReq(t, g, http.MethodGet, "/api/v1/sources/up", "")
if strings.Contains(rec.Body.String(), "sk-up-secret") {
t.Error("the default view leaked the key")
}
}

View File

@ -7,6 +7,7 @@ import (
"fmt"
"log"
"net/http"
"regexp"
"strings"
"sync/atomic"
"time"
@ -28,6 +29,9 @@ type chatRequest struct {
ToolChoice interface{} `json:"tool_choice,omitempty"`
DisableThinking bool `json:"disable_thinking"`
ExtraBody map[string]interface{} `json:"extra_body,omitempty"`
// PromptCacheKey 是 OpenAI 原生的缓存键;部分客户端用它携带会话标识。
PromptCacheKey string `json:"prompt_cache_key,omitempty"`
}
// ChatCompletion is the non-streaming OpenAI response object.
@ -38,6 +42,10 @@ type ChatCompletion struct {
Model string `json:"model"`
Choices []ChatChoice `json:"choices"`
Usage *types.TokenUsage `json:"usage,omitempty"`
// Cost is the upstream-reported charge for this request, passed through
// verbatim (OpenCode reports it as a decimal string, always "0" on the
// flat-rate Go subscription). Absent when the upstream reports nothing.
Cost string `json:"cost,omitempty"`
}
type ChatChoice struct {
@ -62,6 +70,10 @@ type ChatChunk struct {
// Usage is sent in the final chunk of a stream (empty choices) so
// OpenAI-compatible clients can read token usage.
Usage *types.TokenUsage `json:"usage,omitempty"`
// Cost is the upstream-reported charge for this request, passed through
// verbatim (OpenCode reports it as a decimal string). Emitted on the
// terminal chunk, mirroring OpenCode's own {"choices":[],"cost":"0"}.
Cost string `json:"cost,omitempty"`
}
type ChunkChoice struct {
@ -115,13 +127,15 @@ func (g *Gateway) resolveCands(ctx context.Context, req *chatRequest) ([]*provid
}
// filterCandsByModels keeps only providers exposing at least one model of the
// scope (used for user keys with a restricted model scope). An "AUTO" scope
// entry means the key is allowed to use any model. Scope entries with a
// Source pinned to a specific upstream narrow the candidates to that source
// scope (used for user keys with a restricted model scope). Scope entries with
// a Source pinned to a specific upstream narrow the candidates to that source
// for the matching model.
//
// An "AUTO" scope entry only allows the AUTO routing mode; it does NOT grant
// access to specific models.
func filterCandsByModels(cands []*provider.Provider, allow []config.ModelScope) []*provider.Provider {
for _, m := range allow {
if m.Model == "" || strings.EqualFold(m.Model, "AUTO") {
if m.Model == "" {
return cands
}
}
@ -152,20 +166,18 @@ func filterCandsByModels(cands []*provider.Provider, allow []config.ModelScope)
return out
}
// intersectModels restricts a model list to the scope (preserving order). An
// "AUTO" scope entry grants every model.
// intersectModels restricts a model list to the scope (preserving order).
// An "AUTO" scope entry only allows the AUTO routing mode; it does NOT grant
// every model.
func intersectModels(models []string, allow []config.ModelScope) []string {
allowed := make(map[string]bool, len(allow))
any := false
for _, m := range allow {
if m.Model == "" || strings.EqualFold(m.Model, "AUTO") {
any = true
break
if m.Model == "" {
return models
}
if !strings.EqualFold(m.Model, "AUTO") {
allowed[m.Model] = true
}
allowed[m.Model] = true
}
if any {
return models
}
out := make([]string, 0, len(models))
seen := map[string]bool{}
@ -179,24 +191,16 @@ func intersectModels(models []string, allow []config.ModelScope) []string {
}
// checkModelScope validates the effective model against the key's model scope
// and token quota. Returns an error message when rejected. A scope entry with
// model "AUTO" grants all models; its quota caps the key's total tokens.
// and token quota. Returns an error message when rejected.
//
// A scope entry with model "AUTO" only allows requests where the effective
// model is AUTO (the routing mode). It does NOT grant access to specific model
// ids — that requires an explicit scope entry for the model.
func (g *Gateway) checkModelScope(ctx context.Context, model string) string {
allow := g.allowedModels(ctx)
if allow == nil {
return ""
}
for _, sc := range allow {
if sc.Model != "" && strings.EqualFold(sc.Model, "AUTO") {
if sc.TokenQuota > 0 {
used := g.scopeTokens(ctx, sc)
if used >= sc.TokenQuota {
return fmt.Sprintf("token quota exceeded (%d/%d)", used, sc.TokenQuota)
}
}
return ""
}
}
for _, sc := range allow {
if sc.Model != model {
continue
@ -229,12 +233,15 @@ func (g *Gateway) scopeTokens(ctx context.Context, sc config.ModelScope) int64 {
// source that actually serves the bare model (via Registry.EffectiveModel), so
// model ids that themselves contain separators (e.g. "deepseek-v4-flash-free")
// are never corrupted (P10-2).
//
// A scope entry with model "AUTO" only matches the literal AUTO routing mode;
// it does NOT grant access to specific model ids.
func (g *Gateway) hasScopeModel(list []config.ModelScope, s string) bool {
if r := g.core.Registry(); r != nil {
s = r.EffectiveModel(s)
}
for _, x := range list {
if x.Model == s || (x.Model != "" && strings.EqualFold(x.Model, "AUTO")) {
if x.Model == s {
return true
}
}
@ -287,6 +294,8 @@ func (g *Gateway) handleChat(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
return
}
// 客户端自带的会话标识(若有)比推导出来的更准,见 clientSessionFromRequest。
clientSession := clientSessionFromRequest(r, req.PromptCacheKey)
if len(req.Messages) == 0 {
writeError(w, http.StatusBadRequest, "invalid_request", "messages is required")
return
@ -317,6 +326,7 @@ func (g *Gateway) handleChat(w http.ResponseWriter, r *http.Request) {
ToolChoice: req.ToolChoice,
DisableThinking: req.DisableThinking,
ExtraBody: req.ExtraBody,
ClientSession: clientSession,
}
rec := &Req{
Key: keyID(reqKey(ctx)),
@ -372,6 +382,7 @@ func (g *Gateway) handleChat(w http.ResponseWriter, r *http.Request) {
ToolChoice: req.ToolChoice,
DisableThinking: req.DisableThinking,
ExtraBody: req.ExtraBody,
ClientSession: clientSession,
}
rec := &Req{
Key: keyID(reqKey(ctx)),
@ -517,21 +528,115 @@ func upstreamErrStatus(err error) int {
return http.StatusBadGateway
}
// overflowMarkers 匹配上游「上下文超窗」类措辞。
//
// 上游写法五花八门,且**不在 pi 客户端的识别列表里**。pi 靠
// @earendil-works/pi-ai 的 OVERFLOW_PATTERNS 判断超窗并据此触发压缩重试,
// 而 justworker 返回的是「请精简对话历史…(Context window is full…)」——
// 与那 25 条正则一条都不匹配,于是 pi 既不压缩也不重试,只把它当成一条
// 普通上游错误。这里把可识别的超窗措辞归一化成 pi 一定认得的标记。
var overflowMarkers = []*regexp.Regexp{
regexp.MustCompile(`(?i)context[ _-]?window is full`),
regexp.MustCompile(`(?i)context[_ ]length[_ ]exceeded`),
regexp.MustCompile(`(?i)exceeds? the context window`),
regexp.MustCompile(`(?i)maximum context length`),
regexp.MustCompile(`(?i)reduce the length of the messages`),
regexp.MustCompile(`(?i)too many tokens`),
regexp.MustCompile(`(?i)token limit exceeded`),
regexp.MustCompile(`请精简对话历史`),
regexp.MustCompile(`上下文(长度)?超(出|限)`),
regexp.MustCompile(`对话历史过长`),
}
// overflowCanonical 命中 pi 的 /context[_ ]length[_ ]exceeded/i。
//
// 注意:这个前缀只是必要条件,不是充分条件——pi 还会先用
// NON_OVERFLOW_PATTERNS 排除整条消息,见 overflowClientMessage。
const overflowCanonical = "context_length_exceeded"
// looksLikeOverflow 判断错误文本是否属于上下文超窗。
func looksLikeOverflow(s string) bool {
for _, re := range overflowMarkers {
if re.MatchString(s) {
return true
}
}
return false
}
// clientUpstreamErr collapses upstream failure details into a short message
// for the client: per-tier bodies (WAF HTML pages, quota payloads, ...) stay
// in rec.Err / the stats API and the server log instead of the response.
// Per-tier one-line reasons are kept (quota/cooling skips carry no error body
// and are the actionable part); each is capped so HTML dumps can't leak.
//
// 超窗会被归一化成「干净」的 overflowCanonical 消息,见 overflowClientMessage。
func clientUpstreamErr(err error) string {
log.Printf("[gateway] upstream failure surfaced to client: %v", err)
if looksLikeOverflow(err.Error()) {
return overflowClientMessage(err)
}
return upstreamErrSummary(err)
}
// overflowHint 是给客户端的超窗说明,刻意只用 pi 认得的措辞。
const overflowHint = "context window is full; reduce the length of the messages"
// clientSessionFromRequest 取客户端自带的会话标识,拿不到时返回 ""。
//
// 通用 OpenAI 客户端默认不带会话 id,但 pi 支持:只要 provider 的 compat 里打开
// sendSessionAffinityHeaders,pi 就会把**平台真实的会话 id**(uuidv7,整个会话
// 恒定)放到 x-session-affinity / x-client-request-id 上(sessionAffinityFormat
// 为 openrouter 时是 x-session-id,为 openai 时是 session_id)。
// 有了它就不必再从首条 user 消息推导会话指纹(那只在客户端不发会话 id 时才作为
// 退路,且历史压缩后会漂移)。prompt_cache_key 是 OpenAI 原生的缓存键,
// 客户端若带也一并当会话用。
//
// 刻意不采纳 x-client-request-id:名字含 request,部分客户端每请求都换,
// 拿它当会话会让上游缓存永不命中。pi 总会同时发 x-session-affinity,够用。
func clientSessionFromRequest(r *http.Request, bodyKey string) string {
for _, h := range []string{"X-Session-Affinity", "X-Session-Id", "Session-Id"} {
if v := strings.TrimSpace(r.Header.Get(h)); v != "" {
return v
}
}
return strings.TrimSpace(bodyKey)
}
// overflowClientMessage 为超窗失败生成「干净」的客户端消息。
//
// 为什么不能沿用 upstreamErrSummary:pi 的 isContextOverflow 先查
// NON_OVERFLOW_PATTERNS(/rate limit/、/too many requests/、Bedrock 前缀),
// 一旦命中就直接判为「非超窗」——**哪怕消息里已经有 context_length_exceeded**,
// pi 也不会压缩重试。而 AUTO 链的失败消息天生是多 tier 原因的拼接:
// 超窗 tier(gozen 400 maximum context length)常与配额/限流 tier
// (429 token plan exhausted、cooling、no free slot)同时出现。
// 把明细原样带出去,等于让一条限流 tier 的措辞反过来封杀超窗识别。
// 所以这里只保留「超窗」措辞 + 是哪个源超的窗,其余一律不带。
func overflowClientMessage(err error) string {
var ce *scheduler.ChainErr
if errors.As(err, &ce) {
for _, t := range ce.Tiers {
if looksLikeOverflow(t.Err.Error()) {
return fmt.Sprintf("%s: %s (%s/%s)", overflowCanonical, overflowHint, t.Source, t.Model)
}
}
}
return fmt.Sprintf("%s: %s", overflowCanonical, overflowHint)
}
// upstreamErrSummary 把上游失败压成一行短消息。
func upstreamErrSummary(err error) string {
var ce *scheduler.ChainErr
if errors.As(err, &ce) {
parts := make([]string, 0, len(ce.Tiers)+len(ce.Skipped))
for _, t := range ce.Tiers {
parts = append(parts, types.OneLine(fmt.Sprintf("%s/%s: %v", t.Source, t.Model, t.Err), 80))
// 160 而不是 80:短诊断词("Context window is full")常落在尾部,
// 80 字节按字节截断正好会把它切掉,超窗就再也认不出来。
parts = append(parts, types.OneLine(fmt.Sprintf("%s/%s: %v", t.Source, t.Model, t.Err), 160))
}
for _, sk := range ce.Skipped {
parts = append(parts, types.OneLine(sk, 80))
parts = append(parts, types.OneLine(sk, 160))
}
msg := strings.Join(parts, "; ")
if len(msg) > 300 {
@ -606,6 +711,7 @@ func writeChatCompletion(w http.ResponseWriter, resp *types.UnifiedResponse, mod
Created: time.Now().Unix(),
Model: modelName,
Choices: []ChatChoice{{Index: 0, Message: msg, FinishReason: resp.FinishReason}},
Cost: resp.Cost,
}
if resp.TokenUsage.Total > 0 || resp.TokenUsage.Prompt > 0 || resp.TokenUsage.Completion > 0 {
out.Usage = &resp.TokenUsage
@ -725,11 +831,15 @@ func (g *Gateway) pumpStream(w http.ResponseWriter, rec *Req, chunks <-chan type
// source's status-page latency average.
rec.FirstByteMs = time.Since(t0).Milliseconds()
var lastUsage *types.TokenUsage
var lastCost string
lastFinish := ""
for ck := range chunks {
if ck.Usage != nil {
lastUsage = mergeUsage(lastUsage, ck.Usage)
}
if ck.Cost != "" {
lastCost = ck.Cost
}
chunk := ChatChunk{
ID: id, Object: "chat.completion.chunk", Created: created, Model: modelName,
}
@ -796,6 +906,14 @@ func (g *Gateway) pumpStream(w http.ResponseWriter, rec *Req, chunks <-chan type
ID: id, Object: "chat.completion.chunk", Created: created, Model: modelName,
Choices: []ChunkChoice{},
Usage: tut,
Cost: lastCost,
})
} else if lastCost != "" {
// Cost arrived without any usage (OpenCode sends it on its own frame).
send(ChatChunk{
ID: id, Object: "chat.completion.chunk", Created: created, Model: modelName,
Choices: []ChunkChoice{},
Cost: lastCost,
})
}
fmt.Fprintf(w, "data: [DONE]\n\n")

View File

@ -0,0 +1,47 @@
package gateway
import (
"net/http"
"testing"
)
// 客户端自带的会话标识必须被采纳:它比「首条 user 消息指纹」更准
// (历史压缩后指纹会漂移),也是唯一能天然按会话隔离的来源。
func TestClientSessionFromRequestHeaders(t *testing.T) {
cases := []struct {
name string
headers map[string]string
bodyKey string
want string
}{
{"pi 默认格式 x-session-affinity", map[string]string{"X-Session-Affinity": "ses-1"}, "", "ses-1"},
{"openrouter 格式 x-session-id", map[string]string{"X-Session-Id": "ses-2"}, "", "ses-2"},
{"openai 格式 session_id", map[string]string{"Session-Id": "ses-3"}, "", "ses-3"},
{"body prompt_cache_key 兜底", nil, "cache-4", "cache-4"},
{"头优先于 body", map[string]string{"X-Session-Affinity": "ses-5"}, "cache-x", "ses-5"},
{"空白不算", map[string]string{"X-Session-Affinity": " "}, "", ""},
{"都没有则为空(适配器自行推导)", nil, "", ""},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
r, _ := http.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
for k, v := range tc.headers {
r.Header.Set(k, v)
}
if got := clientSessionFromRequest(r, tc.bodyKey); got != tc.want {
t.Errorf("clientSessionFromRequest = %q, want %q", got, tc.want)
}
})
}
}
// 刻意不采纳 x-client-request-id:名字含 request,部分客户端每请求都换,
// 拿它当会话会让上游前缀缓存永不命中。
func TestClientRequestIDIsNotUsedAsSession(t *testing.T) {
r, _ := http.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
r.Header.Set("X-Client-Request-Id", "per-request-uuid")
if got := clientSessionFromRequest(r, ""); got != "" {
t.Errorf("x-client-request-id 不应被当作会话(否则缓存永不命中),got %q", got)
}
}

View File

@ -670,8 +670,11 @@ func TestHasScopeModelWithSourcePrefix(t *testing.T) {
if g.hasScopeModel(scope, "deepseek-v4-flash-free-extra") {
t.Error("hasScopeModel returned true for unrelated model")
}
if !g.hasScopeModel([]config.ModelScope{{Model: "AUTO"}}, "zen:anything") {
t.Error("AUTO scope should allow any prefixed model")
if g.hasScopeModel([]config.ModelScope{{Model: "AUTO"}}, "zen:deepseek-v4-flash-free") {
t.Error("AUTO scope must NOT grant specific model ids even with prefix")
}
if g.hasScopeModel([]config.ModelScope{{Model: "AUTO"}}, "deepseek-v4-pro") {
t.Error("AUTO scope must NOT grant specific model ids")
}
}

View File

@ -0,0 +1,167 @@
package gateway
import (
"errors"
"regexp"
"strings"
"testing"
"llmsproxy/internal/scheduler"
"llmsproxy/internal/types"
)
// piOverflowRe 是 @earendil-works/pi-ai 的 OVERFLOW_PATTERNS 中对应的一条。
// 归一化标记必须命中它,否则改了等于没改:客户端仍不会压缩重试。
var piOverflowRe = regexp.MustCompile(`context[_ ]length[_ ]exceeded`)
func TestOverflowCanonicalMatchesPiPattern(t *testing.T) {
if !piOverflowRe.MatchString(overflowCanonical) {
t.Fatalf("归一化标记 %q 不匹配 pi 的超窗正则,客户端仍不会压缩重试", overflowCanonical)
}
}
// justworker 的真实措辞:与 pi 现有 25 条正则一条都不匹配,
// 所以必须在网关侧归一化,否则超窗对客户端完全不可见。
func TestJustworkerOverflowIsNormalized(t *testing.T) {
raw := "请精简对话历史…(Context window is full, please reduce the conversation history)…"
// 前提校验:原始措辞确实不被 pi 认出(否则本用例失去意义)。
if piOverflowRe.MatchString(raw) {
t.Fatalf("前提不成立:原始措辞本就匹配 pi 正则: %q", raw)
}
if !looksLikeOverflow(raw) {
t.Fatalf("网关应识别出这是超窗: %q", raw)
}
err := &scheduler.ChainErr{Tiers: []scheduler.TierError{{
Tier: 1, Source: "justworker", Model: "claude-opus-5", Err: errors.New(raw),
}}}
got := clientUpstreamErr(err)
if !piOverflowRe.MatchString(got) {
t.Fatalf("归一化后仍不匹配 pi 正则,客户端不会压缩重试: %q", got)
}
}
// 直连路径(非 AUTO 链)也要归一化——它走的是同一个入口。
func TestDirectPathOverflowIsNormalized(t *testing.T) {
got := clientUpstreamErr(errors.New("upstream 400: Context window is full"))
if !piOverflowRe.MatchString(got) {
t.Fatalf("直连超窗未被归一化: %q", got)
}
}
// 非超窗错误不能被加上超窗标记,否则会把普通失败误报成需要压缩。
func TestNonOverflowNotNormalized(t *testing.T) {
err := &scheduler.ChainErr{Tiers: []scheduler.TierError{{
Tier: 1, Source: "x", Model: "y", Err: errors.New("502 Bad Gateway: upstream connect error"),
}}}
got := clientUpstreamErr(err)
if strings.Contains(got, overflowCanonical) {
t.Fatalf("普通上游失败被误标为超窗: %q", got)
}
if !strings.Contains(got, "502 Bad Gateway") {
t.Fatalf("普通失败的诊断信息应保留: %q", got)
}
}
// 回归:短诊断词常落在错误尾部,80 字节截断会正好把它切掉,
// 超窗就永远认不出来。宽度放宽后必须保住。
func TestOverflowPhraseSurvivesTruncationWidth(t *testing.T) {
prefix := strings.Repeat("x", 100)
// OneLine 的长度限制作用于整条 "source/model: err",不是只算 err 本身。
full := "justworker/claude-opus-5: " + prefix + " Context window is full"
// 前提:80 字节下短语确实会被切掉(这正是修复前的行为)。
if got := types.OneLine(full, 80); strings.Contains(got, "Context window is full") {
t.Fatalf("前提不成立:80 字节下短语居然还在: %q", got)
}
// 160 字节下必须保住,否则超窗识别会被截断悄悄破坏。
if got := types.OneLine(full, 160); !strings.Contains(got, "Context window is full") {
t.Fatalf("160 字节应保住尾部短诊断词: %q", got)
}
err := &scheduler.ChainErr{Tiers: []scheduler.TierError{{
Tier: 1, Source: "justworker", Model: "claude-opus-5",
Err: errors.New(prefix + " Context window is full"),
}}}
got := clientUpstreamErr(err)
if !strings.Contains(strings.ToLower(got), "context window is full") {
t.Fatalf("宽度 160 应保住尾部短诊断词,实际: %q", got)
}
if !piOverflowRe.MatchString(got) {
t.Fatalf("归一化标记丢失: %q", got)
}
}
// piNonOverflowRe 是 @earendil-works/pi-ai 的 NON_OVERFLOW_PATTERNS。
// pi 的 isContextOverflow 先查这一组,命中则直接判为「非超窗」——
// 即使 OVERFLOW_PATTERNS 也命中,也不会压缩重试。
var piNonOverflowRe = []*regexp.Regexp{
regexp.MustCompile(`^(Throttling error|Service unavailable):`),
regexp.MustCompile(`(?i)rate limit`),
regexp.MustCompile(`(?i)too many requests`),
}
// piWouldCompact 复刻 isContextOverflow 的 Case 1 判定顺序:
// 先排除 NON_OVERFLOW,再匹配 OVERFLOW。
func piWouldCompact(errorMessage string) bool {
for _, p := range piNonOverflowRe {
if p.MatchString(errorMessage) {
return false
}
}
return piOverflowRe.MatchString(errorMessage)
}
// issue 回归:AUTO 链失败消息天然把「超窗 tier」和「配额/限流 tier」拼在一起,
// 超窗标记后面只要出现 rate limit / too many requests 的字样,
// pi 就会把整条消息判为「非超窗」而拒绝压缩重试。
func TestOverflowMessageSurvivesRateLimitedSiblingTier(t *testing.T) {
cases := []struct {
name string
sibling string
}{
{"rate limit 措辞", "api error 429: rate limit exceeded"},
{"too many requests 措辞", "429 Too Many Requests"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
err := &scheduler.ChainErr{Tiers: []scheduler.TierError{
{Tier: 1, Source: "sensenova", Model: "glm-5.2", Err: errors.New(tc.sibling)},
{Tier: 3, Source: "gozen", Model: "deepseek-v4.1-flash",
Err: errors.New("api error 400: This model's maximum context length is 1048576 tokens")},
}}
got := clientUpstreamErr(err)
if !piWouldCompact(got) {
t.Fatalf("同链限流 tier 封杀了超窗识别,pi 不会压缩重试: %q", got)
}
// 干净:不得把限流 tier 的措辞带出去。
for _, p := range piNonOverflowRe {
if p.MatchString(got) {
t.Fatalf("超窗消息仍带限流措辞 %v: %q", p, got)
}
}
// 但仍要能看出是哪个源超的窗。
if !strings.Contains(got, "gozen/deepseek-v4.1-flash") {
t.Errorf("超窗源信息丢失: %q", got)
}
})
}
}
// 直连超窗同样要干净可用。
func TestDirectOverflowMessageIsClean(t *testing.T) {
raw := "api error 400: Error from provider (Console Go): This model's maximum context length is 1048576 tokens. " +
"However, you requested 1440050 tokens. Please reduce the length of the messages or completion."
got := clientUpstreamErr(errors.New(raw))
if !piWouldCompact(got) {
t.Fatalf("直连超窗未被 pi 识别: %q", got)
}
if !strings.Contains(got, overflowCanonical) {
t.Fatalf("缺少归一化标记: %q", got)
}
}

View File

@ -222,6 +222,8 @@ func (g *Gateway) routes(w http.ResponseWriter, r *http.Request) {
g.handleModels(w, r)
case r.URL.Path == "/api/adapters" || strings.HasPrefix(r.URL.Path, "/api/adapters/"):
g.handleAdaptersAPI(w, r)
case r.URL.Path == "/api/v1" || strings.HasPrefix(r.URL.Path, "/api/v1/"):
g.apiV1Routes(w, r)
case r.URL.Path == "/api/sources" || strings.HasPrefix(r.URL.Path, "/api/sources/"):
g.handleSourcesAPI(w, r)
case r.URL.Path == "/api/source_templates" || strings.HasPrefix(r.URL.Path, "/api/source_templates/"):

View File

@ -2772,7 +2772,10 @@
/* ---------- sources tab ---------- */
async function renderSources() {
const j = await api("/api/sources");
// Read through /api/v1: the UI is a client of the same public API an
// agent uses, so there is one contract rather than a private path.
// The list view needs no credentials, so the masked view is enough.
const j = await api("/api/v1/sources");
const rows = j.sources
.map(
(
@ -2794,13 +2797,23 @@
function editSource(name) {
const wrap = document.createElement("div");
wrap.id = "modal-wrap";
Promise.all([api("/api/sources"), api("/api/status")])
.then(([src, st]) => {
const s = src.sources.find((x) => x.name === name) || {
name: name,
models: [{ id: "", priority: 0, kind: "chat" }],
};
const cur = s.adapter || "openai";
// The edit dialog round-trips the whole source, so it must load the
// real api_key — otherwise saving an unrelated field would blank
// it. That is what ?reveal=credentials is for (admin-only).
const srcPath = name
? "/api/v1/sources/" + encodeURIComponent(name) + "?reveal=credentials"
: "/api/v1/sources";
Promise.all([api(srcPath), api("/api/status")])
.then(([src, st]) => {
// Editing: the single-source shape. Creating: an empty form.
const s = name
? src.source
: { name: "", models: [{ id: "", priority: 0, kind: "chat" }] };
if (!s) {
toast(t("toastSave"));
return;
}
const cur = s.adapter || "openai";
const apps = ["", ...(st.adapters || []).map((a) => a.name)];
if (cur && !apps.includes(cur)) apps.push(cur);
const adSel =
@ -3215,7 +3228,7 @@
const sortLanes = { chat: [], image: [] };
let sortStateMap = new Map();
async function renderSort() {
const j = await api("/api/sources");
const j = await api("/api/v1/sources");
let autoR = [];
let autoImg = [];
try {
@ -4155,7 +4168,7 @@
async function loadModelPairs() {
let srcs = [];
try {
srcs = (await api("/api/sources")).sources || [];
srcs = (await api("/api/v1/sources")).sources || [];
} catch (e) {}
const pairs = [];
(srcs || []).forEach((s) =>

View File

@ -5,10 +5,12 @@ adapter.version = "1.0.0"
adapter.endpoint = "/chat/completions"
adapter.headers = {}
-- AgentRouter 的 WAF 会按客户端指纹白名单校验,只放行官方客户端。
-- 这里通过 build_headers 注入官方客户端 User-Agent 以通过校验。
-- 默认 QwenCode 指纹(实测可通);可在 source.meta.user_agent 覆盖成 Claude Code 指纹。
-- AgentRouter 的阿里云 WAF 按三重维度校验:出口 IP(必须海外)、
-- TLS 指纹(JS/Go HTTP 层与 curl 不同)、浏览器指纹 header 集。
-- 纯 UA 已不够:WAF 拦 12 种 agent CLI UA 中的 10 种(全 405 HTML),
-- 只有同时带 Origin: qwen.ai + Referer + X-Requested-With 才放行,
-- 且同 IP 短时间连续 3+ 次请求会被速率封禁,靠 adapter 侧低频调用 +
-- llmsproxy 的 prefFailStep 冷却自然限制,来源无法控制速率。
local default_ua = "QwenCode/0.2.0 (linux; x64)"
-- AgentRouter fronts Claude models (claude-opus-4-8), and Claude upstreams
@ -65,6 +67,13 @@ function adapter.build_headers(meta)
["Content-Type"] = "application/json",
["Authorization"] = "Bearer " .. meta.api_key,
["User-Agent"] = ua,
-- QwenCode client fingerprint: WAF rejects bare UA without these.
["Accept"] = "application/json, text/plain, */*",
["Accept-Language"] = "zh-CN,zh;q=0.9",
["Origin"] = "https://qwen.ai",
["Referer"] = "https://qwen.ai/",
["X-Requested-With"] = "XMLHttpRequest",
["Connection"] = "keep-alive",
}
end

View File

@ -7,6 +7,10 @@ adapter.headers = {}
-- Gemini API: POST /v1/models/{model}:generateContent
-- Auth: API key in query param ?key=XXX or Authorization: Bearer XXX
--
-- 请求方向必须把 OpenAI 的工具调用翻译成 Gemini 的 functionCall /
-- functionResponse,否则 agent 回放的历史里助手那一轮的调用会凭空消失,
-- 紧随其后的工具结果就成了「无来源」的孤立结果,模型只能反复重发同一个调用。
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
@ -14,16 +18,23 @@ function adapter.transform_request(raw_body)
-- 将 OpenAI 风格 content(字符串或 [{type:*}] 数组)拆成 Gemini parts
local function to_parts(content)
if type(content) == "string" then
if content == "" then return {} end
return { { text = content } }
end
local parts = {}
for _, p in ipairs(content or {}) do
if p.type == "text" then
table.insert(parts, { text = p.text })
elseif p.type == "image_url" and type(p.image_url) == "table" and p.image_url.url then
local mt, b64 = string.match(p.image_url.url, "^data:([^,]+);base64,(.+)$")
if b64 then
table.insert(parts, { inline_data = { mime_type = mt or "image/png", data = b64 } })
if type(p) == "string" then
if p ~= "" then table.insert(parts, { text = p }) end
elseif type(p) == "table" then
if p.type == "text" then
if p.text ~= nil and p.text ~= "" then
table.insert(parts, { text = p.text })
end
elseif p.type == "image_url" and type(p.image_url) == "table" and p.image_url.url then
local mt, b64 = string.match(p.image_url.url, "^data:([^,]+);base64,(.+)$")
if b64 then
table.insert(parts, { inline_data = { mime_type = mt or "image/png", data = b64 } })
end
end
end
end
@ -31,11 +42,59 @@ function adapter.transform_request(raw_body)
end
local contents = {}
local system = ""
local call_names = {} -- tool_call_id -> 函数名(functionResponse 只认名字)
for _, m in ipairs(req.messages or {}) do
table.insert(contents, {
role = (m.role == "assistant") and "model" or m.role,
parts = to_parts(m.content)
})
local role = m.role or "user"
if role == "system" then
if type(m.content) == "string" then
system = system .. m.content .. "\n"
end
elseif role == "assistant" then
local parts = to_parts(m.content)
if type(m.tool_calls) == "table" then
for _, tc in ipairs(m.tool_calls) do
if type(tc) == "table" then
local fn = tc["function"] or {}
local args = fn.arguments
if type(args) == "string" and args ~= "" then
local aok, decoded = pcall(json.decode, args)
args = aok and decoded or {}
elseif type(args) ~= "table" then
args = {}
end
if tc.id ~= nil then call_names[tc.id] = fn.name or "" end
table.insert(parts, {
functionCall = { name = fn.name or "", args = args }
})
end
end
end
-- Gemini 不接受空的 parts 数组;无可发送内容的轮次直接跳过
if #parts > 0 then
table.insert(contents, { role = "model", parts = parts })
end
elseif role == "tool" then
local name = call_names[m.tool_call_id] or m.name or ""
local text = type(m.content) == "string" and m.content or ""
table.insert(contents, {
role = "user",
parts = { { functionResponse = {
name = name,
response = { content = text },
} } }
})
else
local parts = to_parts(m.content)
if #parts > 0 then
table.insert(contents, { role = "user", parts = parts })
end
end
end
local gemini_req = {
@ -46,6 +105,30 @@ function adapter.transform_request(raw_body)
}
}
if system ~= "" then
gemini_req.systemInstruction = { parts = { { text = system } } }
end
-- tools -> functionDeclarations
if type(req.tools) == "table" then
local decls = {}
for _, t in ipairs(req.tools) do
if type(t) == "table" and type(t["function"]) == "table" then
local fn = t["function"]
local params = fn.parameters
if type(params) ~= "table" then params = { type = "object", properties = {} } end
table.insert(decls, {
name = fn.name or "",
description = fn.description or "",
parameters = params,
})
end
end
if #decls > 0 then
gemini_req.tools = { { functionDeclarations = decls } }
end
end
if req.stream then
gemini_req.stream = true
end

View File

@ -6,6 +6,10 @@ adapter.endpoint = "/api/chat"
adapter.headers = {}
-- Ollama API 格式:{ model, messages, stream, options:{temperature,num_predict} }
-- 工具调用必须一并翻译:Ollama 的 assistant 消息用 tool_calls(arguments 是对象
-- 而非 JSON 字符串),工具结果用 tool 角色 + tool_name。此前这里只复制了
-- role/content,助手那轮的调用和 tool 消息的归属全部丢失,模型看到无来源的
-- 工具结果就只能反复重发同一个调用。
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
@ -22,6 +26,7 @@ function adapter.transform_request(raw_body)
-- 转换 messages 格式(Ollama messages 支持 images base64 数组)
if req.messages then
local msgs = {}
local call_names = {} -- tool_call_id -> 函数名
for _, m in ipairs(req.messages) do
local text, images
if type(m.content) == "string" then
@ -30,22 +35,61 @@ function adapter.transform_request(raw_body)
text = ""
images = {}
for _, p in ipairs(m.content or {}) do
if p.type == "text" then
text = text .. (p.text or "")
elseif p.type == "image_url" and type(p.image_url) == "table" and p.image_url.url then
local b64 = string.match(p.image_url.url, "^data:[^,]+;base64,(.+)$")
if b64 then table.insert(images, b64) end
if type(p) == "table" then
if p.type == "text" then
text = text .. (p.text or "")
elseif p.type == "image_url" and type(p.image_url) == "table" and p.image_url.url then
local b64 = string.match(p.image_url.url, "^data:[^,]+;base64,(.+)$")
if b64 then table.insert(images, b64) end
end
end
end
if #images == 0 then images = nil end
end
local msg = { role = m.role, content = text }
if images then msg.images = images end
-- 助手轮的工具调用:arguments 转成对象
if type(m.tool_calls) == "table" and #m.tool_calls > 0 then
local tcs = {}
for _, tc in ipairs(m.tool_calls) do
if type(tc) == "table" then
local fn = tc["function"] or {}
local args = fn.arguments
if type(args) == "string" and args ~= "" then
local aok, decoded = pcall(json.decode, args)
args = aok and decoded or {}
elseif type(args) ~= "table" then
args = {}
end
if tc.id ~= nil then call_names[tc.id] = fn.name or "" end
table.insert(tcs, {
["function"] = { name = fn.name or "", arguments = args }
})
end
end
if #tcs > 0 then msg.tool_calls = tcs end
end
-- 工具结果:Ollama 用 tool_name 标识归属(不认 tool_call_id)
if m.role == "tool" then
local name = call_names[m.tool_call_id] or m.name or ""
if m.tool_call_id ~= nil then
msg.tool_call_id = m.tool_call_id
end
if name ~= "" then msg.tool_name = name end
end
table.insert(msgs, msg)
end
ollama_req.messages = msgs
end
-- tools 透传(Ollama 的 shape 与 OpenAI 一致)
if type(req.tools) == "table" and #req.tools > 0 then
ollama_req.tools = req.tools
end
return json.encode(ollama_req)
end

View File

@ -34,8 +34,10 @@ end
-- OpenAI /chat/completions format (pass-through, strip provider-specific fields)
-- zen 上游 schema 只接受 text content part(无视觉/音频能力):多模态 part
-- (image_url / input_audio / file 等)一律剥离;因此失去全部 content 的
-- 消息整条丢弃,避免上游 "unknown variant `image_url`, expected `text`"。
-- (image_url / input_audio / file 等)一律剥离。剥离后 content 变空的消息
-- 若不再携带 tool_calls / tool_call_id 才整条丢弃(避免上游
-- "unknown variant `image_url`, expected `text`");带工具调用的必须保留,
-- 否则会把紧随其后的 tool 结果变成孤儿,模型会反复重发同一个调用。
-- zen 上游角色白名单只有 system / user / assistant / tool / latest_reminder:
-- OpenAI 的 developer(及 function 等)不在其中,直接透传会触发上游
-- "unknown variant `developer`, expected one of ..." 错误;统一归一化为 system。
@ -52,8 +54,14 @@ function adapter.transform_request(raw_body)
if not ok then return raw_body end
req.disable_thinking = nil
req.extra_body = nil
if type(req.stream_options) ~= "table" then req.stream_options = {} end
req.stream_options.include_usage = true
-- stream_options is only valid alongside stream:true. Sending it on a
-- non-streaming request is rejected by stricter upstreams (OpenCode Go:
-- "stream_options should be set along with stream"), which failed every
-- non-stream call. Only attach it when the request actually streams.
if req.stream then
if type(req.stream_options) ~= "table" then req.stream_options = {} end
req.stream_options.include_usage = true
end
if req.messages then
local kept = {}
for _, msg in ipairs(req.messages) do
@ -72,7 +80,24 @@ function adapter.transform_request(raw_body)
end
end
if #parts == 0 then
drop = true
-- Content collapsed to nothing after stripping unsupported
-- parts. A message that still carries a tool call must
-- NEVER be dropped: the very next message is its tool
-- result, and dropping the call orphans that result. The
-- model then sees a result for a call it never made and
-- re-issues the same tool call on every turn (observed as
-- an infinite "repeated tool call" loop).
--
-- This is the common shape for agent clients: an assistant
-- turn whose content is only [thinking, toolCall] serialises
-- to content:[] with tool_calls — exactly the case that used
-- to vanish here. Emit "" instead, which zen accepts.
if (type(msg.tool_calls) == "table" and #msg.tool_calls > 0)
or msg.tool_call_id ~= nil then
msg.content = ""
else
drop = true
end
else
msg.content = parts
end

View File

@ -0,0 +1,354 @@
local adapter = {}
adapter.name = "opencodego"
adapter.version = "1.0.0"
adapter.endpoint = "/chat/completions"
-- OpenCode Go(包月订阅端点,https://opencode.ai/zen/go/v1)
--
-- 与 OpenCode Zen(opencodezen.lua,https://opencode.ai/zen/v1)是两个不同的
-- 服务,行为要求并不相同,因此各有专用适配器:
--
-- * 本适配器(Go)服务包月订阅池。上游**强制要求**
-- `x-opencode-session` 头,缺失直接 400 "MissingSessionID"。
-- * Go 的 thinking 模型(deepseek-v4.1-flash 等)**必须回传** assistant 轮的
-- `reasoning_content`,否则 400:
-- The `reasoning_content` in the thinking mode must be passed back to the API.
-- 这是本适配器与 Zen 适配器最关键的差异 —— Zen 会剥掉它,Go 必须原样保留。
-- * Go 上游对协议更严格:非流式请求带 `stream_options` 会被拒
-- ("stream_options should be set along with stream")。
--
-- 同样需要 opencode 客户端指纹(UA + x-opencode-*)才能被正确识别与路由。
adapter.headers = {
["User-Agent"] = "opencode/1.18.21 ai-sdk/provider-utils/4.0.23 runtime/bun/1.3.14",
}
-- 每请求生成身份头。沙箱无 os/math,用 meta.timestamp + 请求体哈希派生:
-- 同秒内重复请求 id 相同可接受(zen 只校验存在性,不校验格式)。
local function rand_id(prefix, seed)
return prefix .. string.sub(sha256_hex(seed), 1, 24)
end
-- 会话指纹:取历史里**第一条 user 消息**。
--
-- 为什么不直接用客户端的会话 id:实测抓包(tcpdump 抓 127.0.0.1:8081 的真实
-- agent 请求)确认通用 OpenAI 客户端**根本不发**任何会话标识 —— body 里没有
-- user / session_id / conversation_id / metadata,请求头也只有 X-Stainless-*
-- (OpenAI JS SDK)与 User-Agent。opencode 原生客户端那套 x-opencode-session
-- 是它自己的概念,通用客户端无从转发。
--
-- 退而求其次但要够用:历史被逐轮重放,**第一条 user 消息在整个会话里恒定**,
-- 用它做指纹即可得到"每会话一个 session",而不是"每源一个 session"。
-- 拿不到时(无 user 消息)返回空串,调用方回退到按源稳定。
local function conversation_fingerprint(body)
if type(body) ~= "string" or body == "" then return "" end
local ok, req = pcall(json.decode, body)
if not ok or type(req) ~= "table" then return "" end
for _, m in ipairs(req.messages or {}) do
if type(m) == "table" and m.role == "user" then
local c = m.content
if type(c) == "string" then
if c ~= "" then return c end
elseif type(c) == "table" then
for _, part in ipairs(c) do
if type(part) == "table" and part.type == "text" and part.text then
return part.text
end
end
end
return ""
end
end
return ""
end
-- session_seed 决定上游会话号(x-opencode-session 的种子)。
--
-- 优先级:
-- 1) 客户端自带的会话标识(meta.client_session)——pi 等在开启
-- compat.sendSessionAffinityHeaders 后会发 x-session-affinity,
-- 这是平台真实的会话 id,整个会话恒定且天然按会话隔离。
-- 2) 退路:历史里**第一条 user 消息**做会话指纹。
-- 3) 再退:只按源固定(连 user 消息都没有时)。
--
-- 为什么需要 2/3:通用 OpenAI 客户端默认**根本不发**会话标识 —— 实测抓包
-- (tcpdump 抓 127.0.0.1:8081 真实 agent 请求)确认 body 里没有
-- user / session_id / conversation_id / metadata,请求头也只有
-- X-Stainless-*(OpenAI JS SDK)与 User-Agent。opencode 原生客户端那套
-- x-opencode-session 是它自己的概念,通用客户端无从转发。
--
-- 为什么必须稳定:上游前缀缓存是**会话级**的。会话号每请求一变,
-- 缓存永不命中(实测:固定会号第 2 次命中 5888,每请求换会号则恒为 0)。
local function session_seed(meta)
local src = (meta.source and meta.source.name) or ""
local base = "session|llmsproxy|" .. src
local cs = meta.client_session
if type(cs) == "string" and cs ~= "" then
return base .. "|client|" .. cs
end
return base .. "|" .. conversation_fingerprint(meta.body)
end
function adapter.build_headers(meta)
local ts = tostring(meta.timestamp or "")
local src = (meta.source and meta.source.name) or ""
return {
["User-Agent"] = adapter.headers["User-Agent"],
["x-opencode-client"] = "cli",
-- project 固定:同网关实例共享一个工作区身份
["x-opencode-project"] = string.sub(sha256_hex("llmsproxy|" .. src), 1, 32),
-- session 必须**按源稳定**,不能每请求换:上游的前缀缓存在同一 session
-- 内才复用。实测(同一段 6032 token 提示词):
-- 固定 session -> 第 2 次命中 5888/6032,cached_tokens=5888
-- 每请求换 session -> 永远 0 命中
-- 原先用 meta.timestamp 派生,等于每请求都是新会话,缓存永远无效,
-- 上游也无法做会话亲和路由。
["x-opencode-session"] = rand_id("ses_", session_seed(meta)),
-- request id 仍每请求唯一(它只是请求标识,不参与缓存键)
["x-opencode-request"] = rand_id("msg_", "request|" .. ts .. "|" .. tostring(meta.body or "")),
}
end
-- OpenAI /chat/completions format (pass-through, strip provider-specific fields)
-- zen 上游 schema 只接受 text content part(无视觉/音频能力):多模态 part
-- (image_url / input_audio / file 等)一律剥离。剥离后 content 变空的消息
-- 若不再携带 tool_calls / tool_call_id 才整条丢弃(避免上游
-- "unknown variant `image_url`, expected `text`");带工具调用的必须保留,
-- 否则会把紧随其后的 tool 结果变成孤儿,模型会反复重发同一个调用。
-- zen 上游角色白名单只有 system / user / assistant / tool / latest_reminder:
-- OpenAI 的 developer(及 function 等)不在其中,直接透传会触发上游
-- "unknown variant `developer`, expected one of ..." 错误;统一归一化为 system。
local ROLE_WHITELIST = {
system = true,
user = true,
assistant = true,
tool = true,
latest_reminder = true,
}
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
req.disable_thinking = nil
req.extra_body = nil
-- stream_options is only valid alongside stream:true; sending it on a
-- non-streaming request is rejected by strict upstreams.
if req.stream then
if type(req.stream_options) ~= "table" then req.stream_options = {} end
req.stream_options.include_usage = true
end
if req.messages then
local kept = {}
for _, msg in ipairs(req.messages) do
if type(msg.role) == "string" and not ROLE_WHITELIST[msg.role] then
msg.role = "system"
end
-- Go 的 thinking 模式**必须**回传 reasoning_content:剥掉它会让
-- 上游直接 400("The `reasoning_content` in the thinking mode must
-- be passed back to the API"),agent 的每一轮都会失败。
-- 注意:与 opencodezen.lua 的行为**相反**,不要在这里剥。
--
-- 但绝大多数客户端(pi 等)根本不保存也不回传 reasoning,只保留
-- 工具调用本身。上游只在“带 tool_calls 的助手轮”上校验这个字段,
-- 实测**空串即可通过校验**,所以缺省时补空串:既满足上游的
-- 一致性要求,又不伪造任何推理内容(用户看到的 reasoning 仍然是
-- 上游本轮真实返回的)。
if msg.role == "assistant"
and type(msg.tool_calls) == "table" and #msg.tool_calls > 0
and msg.reasoning_content == nil then
msg.reasoning_content = ""
end
local drop = false
if type(msg.content) == "table" then
local parts = {}
for _, part in ipairs(msg.content) do
if type(part) == "table" and part.type ~= nil and part.type ~= "text" then
-- multimodal part not supported by zen
else
table.insert(parts, part)
end
end
if #parts == 0 then
-- Content collapsed to nothing after stripping unsupported
-- parts. A message that still carries a tool call must
-- NEVER be dropped: the very next message is its tool
-- result, and dropping the call orphans that result. The
-- model then sees a result for a call it never made and
-- re-issues the same tool call on every turn (observed as
-- an infinite "repeated tool call" loop).
if (type(msg.tool_calls) == "table" and #msg.tool_calls > 0)
or msg.tool_call_id ~= nil then
msg.content = ""
else
drop = true
end
else
msg.content = parts
end
end
if not drop then
table.insert(kept, msg)
end
end
req.messages = kept
end
return json.encode(req)
end
function adapter.transform_response(raw_body)
local ok, resp = pcall(json.decode, raw_body)
if not ok or resp == nil then return raw_body end
local unified = {
content = "",
finish_reason = "",
token_usage = { prompt = 0, completion = 0, total = 0 }
}
if type(resp.usage) == "table" then
unified.token_usage.prompt = resp.usage.prompt_tokens or 0
unified.token_usage.completion = resp.usage.completion_tokens or 0
unified.token_usage.total = resp.usage.total_tokens or 0
local hit = 0
if type(resp.usage.prompt_tokens_details) == "table" and resp.usage.prompt_tokens_details.cached_tokens ~= nil then
hit = resp.usage.prompt_tokens_details.cached_tokens
unified.token_usage.prompt_tokens_details = { cached_tokens = hit }
end
if (resp.usage.prompt_cache_hit_tokens or 0) > 0 then
unified.token_usage.prompt_cache_hit_tokens = resp.usage.prompt_cache_hit_tokens
unified.token_usage.prompt_cache_miss_tokens = resp.usage.prompt_cache_miss_tokens or 0
if hit == 0 then
unified.token_usage.prompt_tokens_details = { cached_tokens = resp.usage.prompt_cache_hit_tokens }
end
end
-- 上游报告输出里有多少是思考 token。不给客户端的话,无法判断
-- completion_tokens 里多少是「可见回答」、多少是「思考」(都按输出计费)。
local ctd = resp.usage.completion_tokens_details
if type(ctd) == "table" and ctd.reasoning_tokens ~= nil then
unified.token_usage.completion_tokens_details = { reasoning_tokens = ctd.reasoning_tokens }
end
end
-- 本次调用的费用,上游放在**顶层**且是**字符串**(如 "0"、"0.0012")。
-- Go 订阅是包月制,恒为 "0";只有 Zen 按量付费模型才有信息量。
-- 原样透传:网关不解析、不换算、不汇总 —— 它只是上游事实的搬运者。
if resp.cost ~= nil then
unified.cost = tostring(resp.cost)
end
if type(resp.choices) == "table" and #resp.choices > 0 then
local ch = resp.choices[1]
if type(ch.message) == "table" then
unified.content = ch.message.content or ""
local reasoning = ch.message.reasoning_content or ch.message.reasoning
if reasoning then
unified.reasoning_content = reasoning
end
if type(ch.message.tool_calls) == "table" then
local tcs = {}
for _, tc in ipairs(ch.message.tool_calls) do
local args_ok, args = pcall(json.decode, tc["function"].arguments)
if not args_ok then args = {} end
table.insert(tcs, {
id = tc.id,
type = tc.type or "function",
name = tc["function"].name,
arguments = args
})
end
unified.tool_calls = tcs
end
end
unified.finish_reason = ch.finish_reason or ""
end
return json.encode(unified)
end
function adapter.transform_stream_chunk(raw_chunk)
local ok, chunk = pcall(json.decode, raw_chunk)
if not ok then return "" end
-- OpenAI-style streams may attach usage to a chunk with empty choices
-- (the final usage chunk). Preserve it; the gateway emits it as the
-- terminal usage chunk. Note: keys must match Go's TokenUsage json tags
-- (prompt/completion/total); the gateway re-emits standard *_tokens.
local uses = nil
if type(chunk.usage) == "table" then
uses = {
prompt = chunk.usage.prompt_tokens or chunk.usage.prompt or 0,
completion = chunk.usage.completion_tokens or chunk.usage.completion or 0,
total = chunk.usage.total_tokens or chunk.usage.total or 0,
}
-- 独立字段与 prompt_tokens_details **并存**透传,不要写成 elseif:
-- 上游命中缓存时两者都发,非流式路径也是两个都带。写成 elseif 会让
-- 流式丢掉 prompt_cache_hit_tokens/miss,与同一源的**非流式**产出不一致
-- (dsh 优先读 details,但只认独立字段的老客户端会看不到缓存)。
if type(chunk.usage.prompt_tokens_details) == "table" and chunk.usage.prompt_tokens_details.cached_tokens ~= nil then
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_tokens_details.cached_tokens }
end
if (chunk.usage.prompt_cache_hit_tokens or 0) > 0 then
uses.prompt_cache_hit_tokens = chunk.usage.prompt_cache_hit_tokens
uses.prompt_cache_miss_tokens = chunk.usage.prompt_cache_miss_tokens or 0
if uses.prompt_tokens_details == nil then
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_cache_hit_tokens }
end
end
local ctd = chunk.usage.completion_tokens_details
if type(ctd) == "table" and ctd.reasoning_tokens ~= nil then
uses.completion_tokens_details = { reasoning_tokens = ctd.reasoning_tokens }
end
end
if not chunk.choices or #chunk.choices == 0 then
-- 空 choices 的帧既不是内容也不是结束信号。上游发两种:
-- {"choices":[],"usage":{...}} —— 终帧用量
-- {"choices":[],"cost":"0"} —— 单独的费用帧
-- 两者可能落在同一帧上,必须都转出去;早退只带 usage 会把费用丢干净。
local out = { done = false }
if uses ~= nil then out.usage = uses end
if chunk.cost ~= nil then out.cost = tostring(chunk.cost) end
if uses == nil and chunk.cost == nil then return "" end
return json.encode(out)
end
local delta = chunk.choices[1].delta or {}
local fr = chunk.choices[1].finish_reason
local finish = (type(fr) == "string" and fr ~= "") and fr or nil
local unified = {
content = delta.content or "",
done = (finish ~= nil)
}
if finish then
unified.finish_reason = finish
end
if uses ~= nil then
unified.usage = uses
end
-- zen 用 reasoning 字段承载推理文本(OpenAI 惯例是 reasoning_content)
local reasoning = delta.reasoning_content or delta.reasoning
if reasoning then
unified.reasoning_content = reasoning
end
if delta.tool_calls then
-- pass raw streaming fragments through; OpenAI clients accumulate index+id+name+arguments
unified.tool_calls = delta.tool_calls
end
return json.encode(unified)
end
-- 错误收敛(可选钩子):zen 错误信封固定为 {error={type,message}};
-- 免费池限流 FreeUsageLimitError 单独标注。返回 nil 走通用兜底。
function adapter.transform_error(status, body)
local ok, resp = pcall(json.decode, body)
if not ok or type(resp) ~= "table" then return nil end
local e = resp.error
if type(e) ~= "table" then return nil end
if e.type == "FreeUsageLimitError" then
return "zen free pool quota exhausted"
end
return e.message
end
return adapter

View File

@ -0,0 +1,342 @@
local adapter = {}
adapter.name = "opencodezen"
adapter.version = "1.0.0"
adapter.endpoint = "/chat/completions"
-- OpenCode Zen(按量付费端点,https://opencode.ai/zen/v1)
--
-- 与 OpenCode Go(opencodego.lua,https://opencode.ai/zen/go/v1)是两个不同的
-- 服务,行为要求并不相同,因此各有专用适配器:
--
-- * 本适配器(Zen)服务免费池 / 按量付费池。免费模型必须带 opencode 客户端
-- 指纹(UA + x-opencode-*),否则上游拒绝:"free tier can only be used in
-- OpenCode"。Zen 的免费池以非 thinking 模型为主,历史上回传
-- reasoning_content 会引出上游报错,故这里剥掉(Go 侧相反,见 opencodego.lua)。
-- * Go 订阅读者请看 opencodego.lua:那边**必须**回传 reasoning_content,
-- 否则 thinking 模式直接 400。
--
-- 缺 x-opencode-client/session/request/project 身份头会被判为匿名客户端,
-- 部分模型在带 tools 时上游直接失败(流式返回单 chunk
-- "finish_reason":"network_error" 空 content,非流式 503
-- "Endpoint is unavailable",表现为"空回复")。
adapter.headers = {
["User-Agent"] = "opencode/1.18.21 ai-sdk/provider-utils/4.0.23 runtime/bun/1.3.14",
}
-- 每请求生成身份头。沙箱无 os/math,用 meta.timestamp + 请求体哈希派生:
-- 同秒内重复请求 id 相同可接受(zen 只校验存在性,不校验格式)。
local function rand_id(prefix, seed)
return prefix .. string.sub(sha256_hex(seed), 1, 24)
end
-- 会话指纹:取历史里**第一条 user 消息**。
--
-- 为什么不直接用客户端的会话 id:实测抓包(tcpdump 抓 127.0.0.1:8081 的真实
-- agent 请求)确认通用 OpenAI 客户端**根本不发**任何会话标识 —— body 里没有
-- user / session_id / conversation_id / metadata,请求头也只有 X-Stainless-*
-- (OpenAI JS SDK)与 User-Agent。opencode 原生客户端那套 x-opencode-session
-- 是它自己的概念,通用客户端无从转发。
--
-- 退而求其次但要够用:历史被逐轮重放,**第一条 user 消息在整个会话里恒定**,
-- 用它做指纹即可得到"每会话一个 session",而不是"每源一个 session"。
-- 拿不到时(无 user 消息)返回空串,调用方回退到按源稳定。
local function conversation_fingerprint(body)
if type(body) ~= "string" or body == "" then return "" end
local ok, req = pcall(json.decode, body)
if not ok or type(req) ~= "table" then return "" end
for _, m in ipairs(req.messages or {}) do
if type(m) == "table" and m.role == "user" then
local c = m.content
if type(c) == "string" then
if c ~= "" then return c end
elseif type(c) == "table" then
for _, part in ipairs(c) do
if type(part) == "table" and part.type == "text" and part.text then
return part.text
end
end
end
return ""
end
end
return ""
end
-- session_seed 决定上游会话号(x-opencode-session 的种子)。
--
-- 优先级:
-- 1) 客户端自带的会话标识(meta.client_session)——pi 等在开启
-- compat.sendSessionAffinityHeaders 后会发 x-session-affinity,
-- 这是平台真实的会话 id,整个会话恒定且天然按会话隔离。
-- 2) 退路:历史里**第一条 user 消息**做会话指纹。
-- 3) 再退:只按源固定(连 user 消息都没有时)。
--
-- 为什么需要 2/3:通用 OpenAI 客户端默认**根本不发**会话标识 —— 实测抓包
-- (tcpdump 抓 127.0.0.1:8081 真实 agent 请求)确认 body 里没有
-- user / session_id / conversation_id / metadata,请求头也只有
-- X-Stainless-*(OpenAI JS SDK)与 User-Agent。opencode 原生客户端那套
-- x-opencode-session 是它自己的概念,通用客户端无从转发。
--
-- 为什么必须稳定:上游前缀缓存是**会话级**的。会话号每请求一变,
-- 缓存永不命中(实测:固定会号第 2 次命中 5888,每请求换会号则恒为 0)。
local function session_seed(meta)
local src = (meta.source and meta.source.name) or ""
local base = "session|llmsproxy|" .. src
local cs = meta.client_session
if type(cs) == "string" and cs ~= "" then
return base .. "|client|" .. cs
end
return base .. "|" .. conversation_fingerprint(meta.body)
end
function adapter.build_headers(meta)
local ts = tostring(meta.timestamp or "")
local src = (meta.source and meta.source.name) or ""
return {
["User-Agent"] = adapter.headers["User-Agent"],
["x-opencode-client"] = "cli",
-- project 固定:同网关实例共享一个工作区身份
["x-opencode-project"] = string.sub(sha256_hex("llmsproxy|" .. src), 1, 32),
-- session 必须**按源稳定**,不能每请求换:上游的前缀缓存在同一 session
-- 内才复用。实测(同一段 6032 token 提示词):
-- 固定 session -> 第 2 次命中 5888/6032,cached_tokens=5888
-- 每请求换 session -> 永远 0 命中
-- 原先用 meta.timestamp 派生,等于每请求都是新会话,缓存永远无效,
-- 上游也无法做会话亲和路由。
["x-opencode-session"] = rand_id("ses_", session_seed(meta)),
-- request id 仍每请求唯一(它只是请求标识,不参与缓存键)
["x-opencode-request"] = rand_id("msg_", "request|" .. ts .. "|" .. tostring(meta.body or "")),
}
end
-- OpenAI /chat/completions format (pass-through, strip provider-specific fields)
-- zen 上游 schema 只接受 text content part(无视觉/音频能力):多模态 part
-- (image_url / input_audio / file 等)一律剥离。剥离后 content 变空的消息
-- 若不再携带 tool_calls / tool_call_id 才整条丢弃(避免上游
-- "unknown variant `image_url`, expected `text`");带工具调用的必须保留,
-- 否则会把紧随其后的 tool 结果变成孤儿,模型会反复重发同一个调用。
-- zen 上游角色白名单只有 system / user / assistant / tool / latest_reminder:
-- OpenAI 的 developer(及 function 等)不在其中,直接透传会触发上游
-- "unknown variant `developer`, expected one of ..." 错误;统一归一化为 system。
local ROLE_WHITELIST = {
system = true,
user = true,
assistant = true,
tool = true,
latest_reminder = true,
}
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
req.disable_thinking = nil
req.extra_body = nil
-- stream_options is only valid alongside stream:true; sending it on a
-- non-streaming request is rejected by strict upstreams.
if req.stream then
if type(req.stream_options) ~= "table" then req.stream_options = {} end
req.stream_options.include_usage = true
end
if req.messages then
local kept = {}
for _, msg in ipairs(req.messages) do
if type(msg.role) == "string" and not ROLE_WHITELIST[msg.role] then
msg.role = "system"
end
-- Zen 免费池不接受回传 reasoning_content(Go 侧相反)
msg.reasoning_content = nil
local drop = false
if type(msg.content) == "table" then
local parts = {}
for _, part in ipairs(msg.content) do
if type(part) == "table" and part.type ~= nil and part.type ~= "text" then
-- multimodal part not supported by zen
else
table.insert(parts, part)
end
end
if #parts == 0 then
-- Content collapsed to nothing after stripping unsupported
-- parts. A message that still carries a tool call must
-- NEVER be dropped: the very next message is its tool
-- result, and dropping the call orphans that result. The
-- model then sees a result for a call it never made and
-- re-issues the same tool call on every turn (observed as
-- an infinite "repeated tool call" loop).
if (type(msg.tool_calls) == "table" and #msg.tool_calls > 0)
or msg.tool_call_id ~= nil then
msg.content = ""
else
drop = true
end
else
msg.content = parts
end
end
if not drop then
table.insert(kept, msg)
end
end
req.messages = kept
end
return json.encode(req)
end
function adapter.transform_response(raw_body)
local ok, resp = pcall(json.decode, raw_body)
if not ok or resp == nil then return raw_body end
local unified = {
content = "",
finish_reason = "",
token_usage = { prompt = 0, completion = 0, total = 0 }
}
if type(resp.usage) == "table" then
unified.token_usage.prompt = resp.usage.prompt_tokens or 0
unified.token_usage.completion = resp.usage.completion_tokens or 0
unified.token_usage.total = resp.usage.total_tokens or 0
local hit = 0
if type(resp.usage.prompt_tokens_details) == "table" and resp.usage.prompt_tokens_details.cached_tokens ~= nil then
hit = resp.usage.prompt_tokens_details.cached_tokens
unified.token_usage.prompt_tokens_details = { cached_tokens = hit }
end
if (resp.usage.prompt_cache_hit_tokens or 0) > 0 then
unified.token_usage.prompt_cache_hit_tokens = resp.usage.prompt_cache_hit_tokens
unified.token_usage.prompt_cache_miss_tokens = resp.usage.prompt_cache_miss_tokens or 0
if hit == 0 then
unified.token_usage.prompt_tokens_details = { cached_tokens = resp.usage.prompt_cache_hit_tokens }
end
end
-- 上游报告输出里有多少是思考 token。不给客户端的话,无法判断
-- completion_tokens 里多少是「可见回答」、多少是「思考」(都按输出计费)。
local ctd = resp.usage.completion_tokens_details
if type(ctd) == "table" and ctd.reasoning_tokens ~= nil then
unified.token_usage.completion_tokens_details = { reasoning_tokens = ctd.reasoning_tokens }
end
end
-- 本次调用的费用,上游放在**顶层**且是**字符串**(如 "0"、"0.0012")。
-- Go 订阅是包月制,恒为 "0";只有 Zen 按量付费模型才有信息量。
-- 原样透传:网关不解析、不换算、不汇总 —— 它只是上游事实的搬运者。
if resp.cost ~= nil then
unified.cost = tostring(resp.cost)
end
if type(resp.choices) == "table" and #resp.choices > 0 then
local ch = resp.choices[1]
if type(ch.message) == "table" then
unified.content = ch.message.content or ""
local reasoning = ch.message.reasoning_content or ch.message.reasoning
if reasoning then
unified.reasoning_content = reasoning
end
if type(ch.message.tool_calls) == "table" then
local tcs = {}
for _, tc in ipairs(ch.message.tool_calls) do
local args_ok, args = pcall(json.decode, tc["function"].arguments)
if not args_ok then args = {} end
table.insert(tcs, {
id = tc.id,
type = tc.type or "function",
name = tc["function"].name,
arguments = args
})
end
unified.tool_calls = tcs
end
end
unified.finish_reason = ch.finish_reason or ""
end
return json.encode(unified)
end
function adapter.transform_stream_chunk(raw_chunk)
local ok, chunk = pcall(json.decode, raw_chunk)
if not ok then return "" end
-- OpenAI-style streams may attach usage to a chunk with empty choices
-- (the final usage chunk). Preserve it; the gateway emits it as the
-- terminal usage chunk. Note: keys must match Go's TokenUsage json tags
-- (prompt/completion/total); the gateway re-emits standard *_tokens.
local uses = nil
if type(chunk.usage) == "table" then
uses = {
prompt = chunk.usage.prompt_tokens or chunk.usage.prompt or 0,
completion = chunk.usage.completion_tokens or chunk.usage.completion or 0,
total = chunk.usage.total_tokens or chunk.usage.total or 0,
}
-- 独立字段与 prompt_tokens_details **并存**透传,不要写成 elseif:
-- 上游命中缓存时两者都发,非流式路径也是两个都带。写成 elseif 会让
-- 流式丢掉 prompt_cache_hit_tokens/miss,与同一源的**非流式**产出不一致
-- (dsh 优先读 details,但只认独立字段的老客户端会看不到缓存)。
if type(chunk.usage.prompt_tokens_details) == "table" and chunk.usage.prompt_tokens_details.cached_tokens ~= nil then
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_tokens_details.cached_tokens }
end
if (chunk.usage.prompt_cache_hit_tokens or 0) > 0 then
uses.prompt_cache_hit_tokens = chunk.usage.prompt_cache_hit_tokens
uses.prompt_cache_miss_tokens = chunk.usage.prompt_cache_miss_tokens or 0
if uses.prompt_tokens_details == nil then
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_cache_hit_tokens }
end
end
local ctd = chunk.usage.completion_tokens_details
if type(ctd) == "table" and ctd.reasoning_tokens ~= nil then
uses.completion_tokens_details = { reasoning_tokens = ctd.reasoning_tokens }
end
end
if not chunk.choices or #chunk.choices == 0 then
-- 空 choices 的帧既不是内容也不是结束信号。上游发两种:
-- {"choices":[],"usage":{...}} —— 终帧用量
-- {"choices":[],"cost":"0"} —— 单独的费用帧
-- 两者可能落在同一帧上,必须都转出去;早退只带 usage 会把费用丢干净。
local out = { done = false }
if uses ~= nil then out.usage = uses end
if chunk.cost ~= nil then out.cost = tostring(chunk.cost) end
if uses == nil and chunk.cost == nil then return "" end
return json.encode(out)
end
local delta = chunk.choices[1].delta or {}
local fr = chunk.choices[1].finish_reason
local finish = (type(fr) == "string" and fr ~= "") and fr or nil
local unified = {
content = delta.content or "",
done = (finish ~= nil)
}
if finish then
unified.finish_reason = finish
end
if uses ~= nil then
unified.usage = uses
end
-- zen 用 reasoning 字段承载推理文本(OpenAI 惯例是 reasoning_content)
local reasoning = delta.reasoning_content or delta.reasoning
if reasoning then
unified.reasoning_content = reasoning
end
if delta.tool_calls then
-- pass raw streaming fragments through; OpenAI clients accumulate index+id+name+arguments
unified.tool_calls = delta.tool_calls
end
return json.encode(unified)
end
-- 错误收敛(可选钩子):zen 错误信封固定为 {error={type,message}};
-- 免费池限流 FreeUsageLimitError 单独标注。返回 nil 走通用兜底。
function adapter.transform_error(status, body)
local ok, resp = pcall(json.decode, body)
if not ok or type(resp) ~= "table" then return nil end
local e = resp.error
if type(e) ~= "table" then return nil end
if e.type == "FreeUsageLimitError" then
return "zen free pool quota exhausted"
end
return e.message
end
return adapter

View File

@ -0,0 +1,353 @@
package lua
import (
"encoding/json"
"strings"
"testing"
)
// TestAdaptersPreserveToolCalls is the fleet-wide guard for the
// "orphaned tool result" class of bug: when an agent client replays a turn
// whose assistant message carries tool_calls, EVERY adapter must forward both
// the call and the attribution of its result. Dropping the call (or the
// tool_call_id / tool_name that ties the result to it) makes the model see a
// result for a call it never made and re-issue the same call forever.
//
// The input deliberately uses content:[] — the shape most agent clients emit
// for a tool-calling assistant turn with no text.
func TestAdaptersPreserveToolCalls(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
body := `{"model":"m","messages":[
{"role":"user","content":"read /tmp/x"},
{"role":"assistant","content":[],"tool_calls":[{"id":"call_abc","type":"function","function":{"name":"read","arguments":"{\"path\":\"/tmp/x\"}"}}]},
{"role":"tool","tool_call_id":"call_abc","content":"hello world"},
{"role":"user","content":"summarize"}
]}`
const (
callID = "call_abc"
fnName = "read"
toolOut = "hello world"
lastUser = "summarize"
)
// Adapters that identify a tool result by the call id.
byID := []string{"openai", "deepseek", "github", "groq", "kimicode", "mistral",
"sensenova", "trae", "agentrouter", "opencode", "anthropic"}
for _, a := range byID {
out, err := vm.Transform(a, "transform_request", body)
if err != nil {
t.Errorf("%s: transform_request: %v", a, err)
continue
}
if !strings.Contains(out, callID) {
t.Errorf("%s dropped the tool call id %q: %s", a, callID, out)
}
if !strings.Contains(out, fnName) {
t.Errorf("%s dropped the function name %q: %s", a, fnName, out)
}
if !strings.Contains(out, toolOut) {
t.Errorf("%s dropped the tool result: %s", a, out)
}
if !strings.Contains(out, lastUser) {
t.Errorf("%s dropped the final user turn: %s", a, out)
}
}
// Ollama identifies the result with tool_name (its API has no tool_call_id
// on assistant turns) and takes arguments as an object, not a JSON string.
out, err := vm.Transform("ollama", "transform_request", body)
if err != nil {
t.Fatalf("ollama: %v", err)
}
for _, want := range []string{`"tool_calls"`, fnName, `"tool_name"`, toolOut} {
if !strings.Contains(out, want) {
t.Errorf("ollama missing %s: %s", want, out)
}
}
// Gemini identifies the result by function NAME and has no call id in its
// wire format; both directions must be present as functionCall /
// functionResponse parts, and the system role must move to
// systemInstruction rather than sitting in contents.
gBody := `{"model":"m","messages":[
{"role":"system","content":"be brief"},
{"role":"user","content":"read /tmp/x"},
{"role":"assistant","content":[],"tool_calls":[{"id":"call_abc","type":"function","function":{"name":"read","arguments":"{\"path\":\"/tmp/x\"}"}}]},
{"role":"tool","tool_call_id":"call_abc","content":"hello world"}
]}`
gout, err := vm.Transform("gemini", "transform_request", gBody)
if err != nil {
t.Fatalf("gemini: %v", err)
}
for _, want := range []string{`"functionCall"`, `"functionResponse"`, fnName, toolOut, `"systemInstruction"`} {
if !strings.Contains(gout, want) {
t.Errorf("gemini missing %s: %s", want, gout)
}
}
// No message may keep the OpenAI-only roles inside contents.
var greq struct {
Contents []struct {
Role string `json:"role"`
} `json:"contents"`
}
if err := json.Unmarshal([]byte(gout), &greq); err != nil {
t.Fatalf("gemini unmarshal: %v (%s)", err, gout)
}
for _, c := range greq.Contents {
if c.Role != "user" && c.Role != "model" {
t.Errorf("gemini contents carry an invalid role %q: %s", c.Role, gout)
}
}
// Negative control: a text-only turn must still be forwarded verbatim.
plain := `{"model":"m","messages":[{"role":"user","content":"just text"}]}`
for _, a := range append(byID, "ollama", "gemini") {
pout, err := vm.Transform(a, "transform_request", plain)
if err != nil {
t.Errorf("%s plain: %v", a, err)
continue
}
if !strings.Contains(pout, "just text") {
t.Errorf("%s dropped plain text: %s", a, pout)
}
}
}
// TestOpenCodeStreamOptionsOnlyWhenStreaming pins the fix for
// "stream_options should be set along with stream": OpenCode Go (and other
// strict OpenAI-compatible upstreams) reject a non-streaming request that
// carries stream_options, which broke every non-stream call through the
// opencode adapter.
func TestOpenCodeStreamOptionsOnlyWhenStreaming(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
// non-streaming: stream_options must be absent entirely
out, err := vm.Transform("opencode", "transform_request",
`{"model":"m","messages":[{"role":"user","content":"hi"}]}`)
if err != nil {
t.Fatal(err)
}
if strings.Contains(out, "stream_options") {
t.Fatalf("non-streaming request must not carry stream_options: %s", out)
}
// streaming: stream_options.include_usage must be set
sout, err := vm.Transform("opencode", "transform_request",
`{"model":"m","stream":true,"messages":[{"role":"user","content":"hi"}]}`)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(sout, "stream_options") || !strings.Contains(sout, "include_usage") {
t.Fatalf("streaming request must carry stream_options.include_usage: %s", sout)
}
}
// TestOpenCodeGoVsZenReasoning pins the one behavioural difference that made a
// shared adapter wrong: OpenCode Go's thinking mode REQUIRES the assistant
// turn's reasoning_content to be echoed back ("The `reasoning_content` in the
// thinking mode must be passed back to the API"), while the Zen free pool must
// not receive it. Hence two purpose-built adapters.
func TestOpenCodeGoVsZenReasoning(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
body := `{"model":"m","messages":[
{"role":"user","content":"hi"},
{"role":"assistant","content":"","reasoning_content":"I should read the file.","tool_calls":[{"id":"call_1","type":"function","function":{"name":"read","arguments":"{\"path\":\"/x\"}"}}]},
{"role":"tool","tool_call_id":"call_1","content":"r"}
]}`
// Go: reasoning_content must survive, and the tool call must stay paired.
gout, err := vm.Transform("opencodego", "transform_request", body)
if err != nil {
t.Fatalf("opencodego: %v", err)
}
if !strings.Contains(gout, "I should read the file.") {
t.Errorf("opencodego must pass reasoning_content back (thinking mode requires it): %s", gout)
}
if !strings.Contains(gout, "call_1") || !strings.Contains(gout, "tool_call_id") {
t.Errorf("opencodego must keep the tool call paired with its result: %s", gout)
}
// Zen: reasoning_content is stripped.
zout, err := vm.Transform("opencodezen", "transform_request", body)
if err != nil {
t.Fatalf("opencodezen: %v", err)
}
if strings.Contains(zout, "I should read the file.") {
t.Errorf("opencodezen must strip reasoning_content: %s", zout)
}
if !strings.Contains(zout, "call_1") {
t.Errorf("opencodezen must still keep the tool call: %s", zout)
}
// An assistant turn that carries tool_calls but no reasoning_content must
// get an empty one injected: Go validates that field on tool-calling turns
// and 400s when it is absent, while most clients never send it.
// Verified against the live endpoint that an empty string passes.
bare := `{"model":"m","messages":[
{"role":"user","content":"go"},
{"role":"assistant","content":"","tool_calls":[{"id":"c1","type":"function","function":{"name":"read","arguments":"{}"}}]},
{"role":"tool","tool_call_id":"c1","content":"r"}
]}`
bout, err := vm.Transform("opencodego", "transform_request", bare)
if err != nil {
t.Fatalf("opencodego bare: %v", err)
}
if !strings.Contains(bout, `"reasoning_content":""`) {
t.Errorf("opencodego must inject an empty reasoning_content on a tool-calling turn: %s", bout)
}
// A plain assistant turn (no tool calls) must NOT gain the field.
plainAsst := `{"model":"m","messages":[{"role":"user","content":"hi"},{"role":"assistant","content":"hello"}]}`
pout, err := vm.Transform("opencodego", "transform_request", plainAsst)
if err != nil {
t.Fatal(err)
}
if strings.Contains(pout, "reasoning_content") {
t.Errorf("opencodego must not inject reasoning_content on a plain assistant turn: %s", pout)
}
// Both must still omit stream_options on non-streaming requests.
for _, a := range []string{"opencodego", "opencodezen"} {
out, err := vm.Transform(a, "transform_request", `{"model":"m","messages":[{"role":"user","content":"hi"}]}`)
if err != nil {
t.Fatalf("%s: %v", a, err)
}
if strings.Contains(out, "stream_options") {
t.Errorf("%s must not send stream_options when not streaming: %s", a, out)
}
}
}
// TestOpenCodeSessionIsStableForCache pins the session-derivation contract that
// makes the upstream prefix cache usable.
//
// Measured against the live OpenCode Go endpoint: the prefix cache is
// SESSION-scoped. Replaying the same 6032-token prompt hits 5888 cached tokens
// when the session id is fixed, and 0 when it changes per request. Deriving the
// session from meta.timestamp (the previous behaviour) meant every request was a
// new session, so the cache could never hit.
//
// The session must be stable WITHIN one conversation and differ BETWEEN
// conversations. Since generic OpenAI clients send no session id at all
// (verified by capturing real agent traffic: no user / session_id /
// conversation_id / metadata in the body, no session header), the conversation
// is fingerprinted by its FIRST user message — history is replayed every turn,
// so that message is invariant for the life of the conversation.
func TestOpenCodeSessionIsStableForCache(t *testing.T) {
for _, name := range []string{"opencodego", "opencodezen"} {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
meta := func(ts int, body string) map[string]interface{} {
return map[string]interface{}{
"timestamp": ts,
"body": body,
"source": map[string]interface{}{"name": "somesource", "meta": nil},
}
}
// Same conversation, later turn: history grew, first user msg unchanged.
h1, err := vm.BuildHeaders(name, meta(1000,
`{"messages":[{"role":"user","content":"fix the bug"},{"role":"assistant","content":"looking"}]}`))
if err != nil {
t.Fatalf("%s: %v", name, err)
}
h2, err := vm.BuildHeaders(name, meta(2000,
`{"messages":[{"role":"user","content":"fix the bug"},{"role":"assistant","content":"looking"},{"role":"user","content":"and tests"}]}`))
if err != nil {
t.Fatalf("%s: %v", name, err)
}
// Different conversation.
h3, err := vm.BuildHeaders(name, meta(3000,
`{"messages":[{"role":"user","content":"an unrelated task"}]}`))
if err != nil {
t.Fatalf("%s: %v", name, err)
}
if h1["x-opencode-session"] == "" {
t.Fatalf("%s: session header must be set", name)
}
if h1["x-opencode-session"] != h2["x-opencode-session"] {
t.Errorf("%s: x-opencode-session must be STABLE across turns of one conversation "+
"(%q vs %q) — the prefix cache is session-scoped", name, h1["x-opencode-session"], h2["x-opencode-session"])
}
if h1["x-opencode-session"] == h3["x-opencode-session"] {
t.Errorf("%s: a different conversation must get a different session", name)
}
if h1["x-opencode-request"] == h2["x-opencode-request"] {
t.Errorf("%s: x-opencode-request must differ per request", name)
}
// No user message at all falls back to per-source stability.
h4, _ := vm.BuildHeaders(name, meta(4000, `{"messages":[{"role":"system","content":"s"}]}`))
h5, _ := vm.BuildHeaders(name, meta(5000, `{"messages":[{"role":"system","content":"s"}]}`))
if h4["x-opencode-session"] != h5["x-opencode-session"] {
t.Errorf("%s: sessionless requests must still be stable per source", name)
}
vm.Stop()
}
}
// 客户端自带的会话标识(pi 的 x-session-affinity)必须优先于会话指纹:
// 它是平台真实会话 id,历史压缩后也不会漂移,且天然按会话隔离。
func TestOpenCodePrefersClientSessionID(t *testing.T) {
for _, name := range []string{"opencodego", "opencodezen"} {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
meta := func(body, cs string) map[string]interface{} {
return map[string]interface{}{
"timestamp": 1000,
"body": body,
"client_session": cs,
"source": map[string]interface{}{"name": "somesource", "meta": nil},
}
}
// 同一条会话:历史在长,但客户端会话 id 不变。
a, err := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"fix the bug"}]}`, "cs-aaa"))
if err != nil {
t.Fatal(err)
}
b, err := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"a totally different opener"}]}`, "cs-aaa"))
if err != nil {
t.Fatal(err)
}
if a["x-opencode-session"] != b["x-opencode-session"] {
t.Errorf("%s: 客户端会话 id 相同就必须同会话(内容无关): %q vs %q",
name, a["x-opencode-session"], b["x-opencode-session"])
}
// 不同会话。
c, _ := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"fix the bug"}]}`, "cs-bbb"))
if a["x-opencode-session"] == c["x-opencode-session"] {
t.Errorf("%s: 不同客户端会话必须不同会号", name)
}
// 客户端没给会话 id 时,回落到指纹(同内容 -> 同会号)。
d, _ := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"fix the bug"}]}`, ""))
e, _ := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"fix the bug"}]}`, ""))
if d["x-opencode-session"] != e["x-opencode-session"] {
t.Errorf("%s: 无客户端会号时应回落到稳定指纹", name)
}
if d["x-opencode-session"] == a["x-opencode-session"] {
t.Errorf("%s: 客户端会号与指纹两条路必须产出不同会号", name)
}
vm.Stop()
}
}

View File

@ -0,0 +1,177 @@
package lua
import (
"encoding/json"
"testing"
)
// OpenCode Zen/Go 在 usage 里报告输出有多少是思考 token,并在**顶层**用一个
// **字符串**给出本次费用。前者决定「completion_tokens 里多少是可见回答」,
// 后者是唯一的费用信号。两者都曾被适配器丢掉。
func TestOpenCodeCostAndReasoningPassthrough(t *testing.T) {
for _, name := range []string{"opencodego", "opencodezen"} {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
// ---- 非流式 ----
resp := `{"id":"x","object":"chat.completion","model":"deepseek-v4.1-flash",
"choices":[{"index":0,"message":{"role":"assistant","content":"hi","reasoning_content":"think"},
"finish_reason":"stop"}],
"usage":{"prompt_tokens":37,"completion_tokens":40,"total_tokens":77,
"prompt_cache_hit_tokens":0,"prompt_cache_miss_tokens":37,
"prompt_tokens_details":{"cached_tokens":0},
"completion_tokens_details":{"reasoning_tokens":40}},
"cost":"0"}`
out, err := vm.Transform(name, "transform_response", resp)
if err != nil {
t.Fatalf("%s transform_response: %v", name, err)
}
var r struct {
TokenUsage struct {
Completion int `json:"completion"`
Details *struct {
ReasoningTokens int `json:"reasoning_tokens"`
} `json:"completion_tokens_details"`
} `json:"token_usage"`
Cost string `json:"cost"`
}
if err := json.Unmarshal([]byte(out), &r); err != nil {
t.Fatalf("%s unmarshal: %v (%s)", name, err, out)
}
if r.Cost != "0" {
t.Errorf("%s: cost 未透传(上游给的是字符串 \"0\"): %q", name, r.Cost)
}
if r.TokenUsage.Details == nil || r.TokenUsage.Details.ReasoningTokens != 40 {
t.Errorf("%s: completion_tokens_details.reasoning_tokens 未透传: %s", name, out)
}
// 前提校验:reasoning_tokens 确实 ≤ completion_tokens(全为思考)。
if r.TokenUsage.Details != nil && r.TokenUsage.Details.ReasoningTokens > r.TokenUsage.Completion {
t.Errorf("%s: reasoning 不应超过 completion", name)
}
// ---- 流式:非零费用也要透传 ----
chunk := `{"id":"x","object":"chat.completion.chunk","choices":[],
"usage":{"prompt_tokens":10,"completion_tokens":5,"total_tokens":15,
"completion_tokens_details":{"reasoning_tokens":4}},"cost":"0.0012"}`
co, err := vm.Transform(name, "transform_stream_chunk", chunk)
if err != nil {
t.Fatalf("%s transform_stream_chunk: %v", name, err)
}
var c struct {
Cost string `json:"cost"`
Usage *struct {
Details *struct {
ReasoningTokens int `json:"reasoning_tokens"`
} `json:"completion_tokens_details"`
} `json:"usage"`
}
if err := json.Unmarshal([]byte(co), &c); err != nil {
t.Fatalf("%s unmarshal chunk: %v (%s)", name, err, co)
}
if c.Usage == nil || c.Usage.Details == nil || c.Usage.Details.ReasoningTokens != 4 {
t.Errorf("%s: 流式 reasoning_tokens 未透传: %s", name, co)
}
if c.Cost != "0.0012" {
t.Errorf("%s: 流式 cost 未透传: %s", name, co)
}
// ---- 独立的 cost 帧(上游就这么发:choices 为空、只有 cost)----
only := `{"choices":[],"cost":"0"}`
oo, err := vm.Transform(name, "transform_stream_chunk", only)
if err != nil {
t.Fatalf("%s cost-only chunk: %v", name, err)
}
if oo == "" {
t.Fatalf("%s: 纯 cost 帧被整个丢弃(客户端将看不到费用)", name)
}
var oc struct {
Cost string `json:"cost"`
}
if err := json.Unmarshal([]byte(oo), &oc); err != nil {
t.Fatalf("%s unmarshal cost chunk: %v (%s)", name, err, oo)
}
if oc.Cost != "0" {
t.Errorf("%s: 纯 cost 帧内容丢失: %s", name, oo)
}
// ---- 回归:没有 cost / 没有 details 时不得凭空造字段 ----
bare := `{"choices":[{"index":0,"delta":{"content":"hi"},"finish_reason":"stop"}],
"usage":{"prompt_tokens":3,"completion_tokens":1,"total_tokens":4}}`
bo, _ := vm.Transform(name, "transform_response", bare)
if got := bo; len(got) > 0 && containsAll(got, `"cost"`) {
t.Errorf("%s: 上游没给 cost 却透传了该字段: %s", name, got)
}
vm.Stop()
}
}
func containsAll(s string, subs ...string) bool {
for _, sub := range subs {
found := false
for i := 0; i+len(sub) <= len(s); i++ {
if s[i:i+len(sub)] == sub {
found = true
break
}
}
if !found {
return false
}
}
return true
}
// 流式与非流式对同一上游切片的产出必须一致:命中缓存时上游同时给
// prompt_tokens_details.cached_tokens 和独立的 hit/miss 字段,两边都要透传。
func TestOpenCodeStreamCacheFieldsMatchNonStream(t *testing.T) {
for _, name := range []string{"opencodego", "opencodezen"} {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
type cache struct {
Details *struct {
CachedTokens int `json:"cached_tokens"`
} `json:"prompt_tokens_details"`
Hit int `json:"prompt_cache_hit_tokens"`
Miss int `json:"prompt_cache_miss_tokens"`
}
usage := `"usage":{"prompt_tokens":5732,"completion_tokens":20,"total_tokens":5752,
"prompt_cache_hit_tokens":5504,"prompt_cache_miss_tokens":228,
"prompt_tokens_details":{"cached_tokens":5504}}`
// 非流式
nout, _ := vm.Transform(name, "transform_response",
`{"choices":[{"index":0,"message":{"role":"assistant","content":"hi"},"finish_reason":"stop"}],`+usage+`}`)
var nr struct {
U cache `json:"token_usage"`
}
if err := json.Unmarshal([]byte(nout), &nr); err != nil {
t.Fatalf("%s: %v", name, err)
}
// 流式
sout, _ := vm.Transform(name, "transform_stream_chunk",
`{"choices":[],`+usage+`}`)
var sr struct {
U cache `json:"usage"`
}
if err := json.Unmarshal([]byte(sout), &sr); err != nil {
t.Fatalf("%s chunk: %v (%s)", name, err, sout)
}
if nr.U.Hit != 5504 || nr.U.Miss != 228 {
t.Errorf("%s: 非流式 hit/miss 异常: %+v", name, nr.U)
}
if sr.U.Hit != nr.U.Hit || sr.U.Miss != nr.U.Miss {
t.Errorf("%s: 流式 hit/miss 与非流式不一致 (stream %d/%d vs non-stream %d/%d): %s",
name, sr.U.Hit, sr.U.Miss, nr.U.Hit, nr.U.Miss, sout)
}
if sr.U.Details == nil || sr.U.Details.CachedTokens != 5504 {
t.Errorf("%s: 流式 cached_tokens 丢失: %s", name, sout)
}
vm.Stop()
}
}

View File

@ -221,6 +221,70 @@ func TestOpenCodeStripsMultiModalParts(t *testing.T) {
}
}
// TestOpenCodeKeepsToolCallWithEmptyContent pins the fix for an infinite
// "repeated tool call" loop: an assistant turn whose content serialises to an
// empty ARRAY while carrying tool_calls must survive the multimodal strip.
// Dropping it orphans the following tool result, so the model never sees the
// call it already made and re-issues it on every turn.
func TestOpenCodeKeepsToolCallWithEmptyContent(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
body := `{"model":"x","messages":[
{"role":"user","content":"read /tmp/x"},
{"role":"assistant","content":[],"tool_calls":[{"id":"call_abc","type":"function","function":{"name":"read","arguments":"{\"path\":\"/tmp/x\"}"}}]},
{"role":"tool","tool_call_id":"call_abc","content":"hello"},
{"role":"user","content":"now summarize"}
]}`
out, err := vm.Transform("opencode", "transform_request", body)
if err != nil {
t.Fatalf("transform_request: %v", err)
}
var req struct {
Messages []struct {
Role string `json:"role"`
Content json.RawMessage `json:"content"`
ToolCalls []struct {
ID string `json:"id"`
} `json:"tool_calls"`
ToolCallID string `json:"tool_call_id"`
} `json:"messages"`
}
if err := json.Unmarshal([]byte(out), &req); err != nil {
t.Fatalf("unmarshal: %v (%s)", err, out)
}
if len(req.Messages) != 4 {
t.Fatalf("expected 4 messages (tool call must not be dropped), got %d: %s", len(req.Messages), out)
}
asst := req.Messages[1]
if asst.Role != "assistant" || len(asst.ToolCalls) != 1 || asst.ToolCalls[0].ID != "call_abc" {
t.Fatalf("assistant tool_calls lost: %s", out)
}
if string(asst.Content) != `""` {
t.Fatalf("empty content must be rewritten to \"\", got %s", asst.Content)
}
if req.Messages[2].Role != "tool" || req.Messages[2].ToolCallID != "call_abc" {
t.Fatalf("tool result must stay paired with its call: %s", out)
}
// Negative control: a message with neither text nor tool calls is still dropped.
only := `{"model":"x","messages":[
{"role":"user","content":[{"type":"image_url","image_url":{"url":"https://ex.com/a.png"}}]},
{"role":"user","content":"keep me"}
]}`
out2, err := vm.Transform("opencode", "transform_request", only)
if err != nil {
t.Fatal(err)
}
if got := strings.Count(out2, `"role"`); got != 1 {
t.Fatalf("image-only message without tool calls must still be dropped, got %d: %s", got, out2)
}
}
func TestBuildHeadersCustomHook(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {

View File

@ -707,7 +707,7 @@ func (p *Provider) probeModels(ctx context.Context) (bool, string) {
if err != nil {
return false, ""
}
if hdrs, herr := p.buildHeaders("{}", u); herr == nil {
if hdrs, herr := p.buildHeaders("{}", u, ""); herr == nil {
req.Header = hdrs
}
resp, err := p.client.Do(req)
@ -750,7 +750,7 @@ func (p *Provider) probeChat(ctx context.Context) (bool, string) {
body, err := json.Marshal(probe)
if err == nil {
var hdr http.Header
if hdrs, herr := p.buildHeaders(string(body), p.URL()); herr == nil {
if hdrs, herr := p.buildHeaders(string(body), p.URL(), ""); herr == nil {
hdr = hdrs
}
raw, status, derr := p.do(ctx, p.URL(), string(body), hdr)
@ -1031,13 +1031,15 @@ func (p *Provider) Throttle(ctx context.Context) error {
// ---- request construction ----
func (p *Provider) buildHeaders(body, url string) (http.Header, error) {
func (p *Provider) buildHeaders(body, url, clientSession string) (http.Header, error) {
meta := map[string]interface{}{
"url": url,
"method": http.MethodPost,
"body": body,
"api_key": p.cfg.APIKey,
"timestamp": types.Now(),
// client_session 是客户端自带的会话标识(请求头),为空时适配器自行推导。
"client_session": clientSession,
"source": map[string]interface{}{
"name": p.cfg.Name,
"meta": p.cfg.Meta,
@ -1082,7 +1084,7 @@ func (p *Provider) Chat(ctx context.Context, req *types.ChatRequest) (*types.Uni
if err != nil {
return nil, err
}
hdrs, err := p.buildHeaders(body, p.URL())
hdrs, err := p.buildHeaders(body, p.URL(), req.ClientSession)
if err != nil {
return nil, err
}
@ -1142,7 +1144,7 @@ func (p *Provider) ChatStream(ctx context.Context, req *types.ChatRequest) (<-ch
p.Release()
return nil, err
}
hdrs, err := p.buildHeaders(body, p.URL())
hdrs, err := p.buildHeaders(body, p.URL(), req.ClientSession)
if err != nil {
p.Release()
return nil, err
@ -1411,7 +1413,7 @@ func (p *Provider) Image(ctx context.Context, req *types.ImageGenRequest) (*type
// fall back to passthrough adapter (openai-style)
transformed = string(b)
}
hdrs, err := p.buildHeaders(transformed, p.ImageURL())
hdrs, err := p.buildHeaders(transformed, p.ImageURL(), "")
if err != nil {
return nil, err
}
@ -1516,12 +1518,24 @@ func standardSSEChunk(data string) string {
PromptTokensDetails *struct {
CachedTokens int `json:"cached_tokens"`
} `json:"prompt_tokens_details"`
CompletionTokensDetails *struct {
ReasoningTokens int `json:"reasoning_tokens"`
} `json:"completion_tokens_details"`
} `json:"usage"`
// Cost is a top-level decimal string on OpenCode Zen/Go responses.
Cost json.RawMessage `json:"cost"`
}
if err := json.Unmarshal([]byte(data), &raw); err != nil {
return ""
}
if len(raw.Choices) == 0 && raw.UpstreamUsage.Total == 0 && raw.UpstreamUsage.TotalTokens == 0 {
// Cost-only chunk: OpenCode sends the charge on its own frame
// {"choices":[],"cost":"0"} with no usage at all. Dropping it here
// would lose the only cost signal the client can see.
if c := rawCostString(raw.Cost); c != "" {
out, _ := json.Marshal(types.UnifiedChunk{Cost: c})
return string(out)
}
return ""
}
var usage *types.TokenUsage
@ -1539,6 +1553,11 @@ func standardSSEChunk(data string) string {
CachedTokens: pu.PromptTokensDetails.CachedTokens,
}
}
if pu.CompletionTokensDetails != nil {
usage.CompletionTokensDetails = &types.CompletionTokensDetails{
ReasoningTokens: pu.CompletionTokensDetails.ReasoningTokens,
}
}
}
finish := ""
done := false
@ -1563,6 +1582,24 @@ func standardSSEChunk(data string) string {
return string(out)
}
// rawCostString normalizes an upstream cost field into a plain string. The
// value is a decimal **string** on OpenCode Zen/Go ("0", "0.0012"), but other
// upstreams have been seen to send a bare number, so both are accepted.
// Returns "" when absent, null, or empty.
func rawCostString(raw json.RawMessage) string {
s := strings.TrimSpace(string(raw))
if s == "" || s == "null" {
return ""
}
if s[0] == '"' {
var unq string
if json.Unmarshal(raw, &unq) == nil {
return strings.TrimSpace(unq)
}
}
return s
}
// pickFirst returns a if non-zero, else b (for usage keys that may appear in
// either standard *_tokens or legacy short form).
func pickFirst(a, b int) int {

View File

@ -3,6 +3,7 @@
package types
import (
"bytes"
"encoding/json"
"errors"
"strings"
@ -28,6 +29,10 @@ type ChatRequest struct {
ToolChoice interface{} `json:"tool_choice,omitempty"`
DisableThinking bool `json:"disable_thinking"`
ExtraBody map[string]interface{} `json:"-"`
// ClientSession 是客户端自带的会话标识(请求头 x-session-affinity 等)。
// 不参与序列化:它只用于让适配器生成稳定的上游会话,不应泄漏给上游 body。
ClientSession string `json:"-"`
}
func (r *ChatRequest) MarshalJSON() ([]byte, error) {
@ -61,6 +66,39 @@ type ChatMessage struct {
func StringContent(s string) json.RawMessage { b, _ := json.Marshal(s); return b }
// isEmptyJSONArray reports whether raw is the literal empty JSON array [].
func isEmptyJSONArray(raw json.RawMessage) bool {
t := bytes.TrimSpace(raw)
return len(t) == 2 && t[0] == '[' && t[1] == ']'
}
// UnmarshalJSON decodes a chat message, normalising an empty-array content
// ("content":[]) to an empty string and dropping an empty tool_calls array.
//
// Why this exists: Lua adapters json.decode the request and re-encode it, and
// an empty Lua table is indistinguishable from an empty JSON array — the
// encoder emits {} for both. Agent clients serialise an assistant turn that
// carries tool_calls and no text as content:[], so every pass-through adapter
// turned it into content:{} — a shape that is not valid OpenAI (content is
// string | array of parts | null) and that real upstreams reject with
// "400 invalid arguments". Normalising at the decode boundary fixes every
// adapter at once, including ones added later.
func (m *ChatMessage) UnmarshalJSON(b []byte) error {
type alias ChatMessage
var a alias
if err := json.Unmarshal(b, &a); err != nil {
return err
}
*m = ChatMessage(a)
if isEmptyJSONArray(m.Content) {
m.Content = StringContent("")
}
if isEmptyJSONArray(m.ToolCalls) {
m.ToolCalls = nil
}
return nil
}
type ToolCall struct {
ID string `json:"id"`
Type string `json:"type"`
@ -79,6 +117,11 @@ type UnifiedResponse struct {
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
// ImageData used by image-generation adapters.
ImageData []ImageData `json:"image_data,omitempty"`
// Cost is the upstream-reported charge for this request, passed through
// verbatim (OpenCode Zen/Go put a decimal **string** at the response top
// level). Deliberately not parsed or summed by the gateway: it is an
// upstream fact, and only some upstreams report it at all.
Cost string `json:"cost,omitempty"`
}
type TokenUsage struct {
@ -95,6 +138,16 @@ type TokenUsage struct {
// prompt_tokens_details.cached_tokens is absent.
PromptCacheHit int `json:"prompt_cache_hit_tokens,omitempty"`
PromptCacheMiss int `json:"prompt_cache_miss_tokens,omitempty"`
// CompletionTokensDetails mirrors the OpenAI v2 usage.completion_tokens_details
// object. OpenCode Zen/Go report how much of the completion was thinking
// tokens here; without it clients cannot tell how much of the billed output
// was reasoning rather than visible answer.
CompletionTokensDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
}
// CompletionTokensDetails is the OpenAI v2 completion_tokens_details object.
type CompletionTokensDetails struct {
ReasoningTokens int `json:"reasoning_tokens"`
}
// PromptTokensDetails is the OpenAI v2 prompt_tokens_details object. Only
@ -119,15 +172,16 @@ func (t TokenUsage) MarshalJSON() ([]byte, error) {
pdetails = &PromptTokensDetails{CachedTokens: t.PromptCacheHit}
}
return json.Marshal(struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
Prompt int `json:"prompt"`
Completion int `json:"completion"`
Total int `json:"total"`
PromptTokensDetails *PromptTokensDetails `json:"prompt_tokens_details,omitempty"`
PromptCacheHitTokens int `json:"prompt_cache_hit_tokens,omitempty"`
PromptCacheMissTokens int `json:"prompt_cache_miss_tokens,omitempty"`
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
Prompt int `json:"prompt"`
Completion int `json:"completion"`
Total int `json:"total"`
PromptTokensDetails *PromptTokensDetails `json:"prompt_tokens_details,omitempty"`
PromptCacheHitTokens int `json:"prompt_cache_hit_tokens,omitempty"`
PromptCacheMissTokens int `json:"prompt_cache_miss_tokens,omitempty"`
CompletionDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
}{
PromptTokens: t.Prompt,
CompletionTokens: t.Completion,
@ -138,6 +192,7 @@ func (t TokenUsage) MarshalJSON() ([]byte, error) {
PromptTokensDetails: pdetails,
PromptCacheHitTokens: t.PromptCacheHit,
PromptCacheMissTokens: t.PromptCacheMiss,
CompletionDetails: t.CompletionTokensDetails,
})
}
@ -181,6 +236,9 @@ type UnifiedChunk struct {
// often has empty choices). Gateway uses it to emit exact usage in the
// final stream chunk instead of estimates.
Usage *TokenUsage `json:"usage,omitempty"`
// Cost is upstream-reported charge for this request (decimal string),
// passed through verbatim; OpenCode sends it on its own stream chunk.
Cost string `json:"cost,omitempty"`
}
// Meta passed to Lua build_headers hook

View File

@ -2,6 +2,7 @@ package types
import (
"encoding/json"
"strings"
"testing"
)
@ -68,3 +69,86 @@ func TestTokenUsageMarshalCacheFields(t *testing.T) {
t.Logf("DeepSeek JSON: %s", string(b))
t.Logf("OpenAI JSON: %s", string(b2))
}
// TestChatMessageNormalizesEmptyArrayContent pins the fix for the malformed
// content:{} that pass-through adapters produced for an assistant turn carrying
// tool_calls with no text. Lua adapters cannot tell an empty JSON array from an
// empty object, so the decode boundary normalises it for every adapter at once.
func TestChatMessageNormalizesEmptyArrayContent(t *testing.T) {
var msg ChatMessage
raw := `{"role":"assistant","content":[],"tool_calls":[{"id":"call_1","type":"function","function":{"name":"read","arguments":"{}"}}]}`
if err := json.Unmarshal([]byte(raw), &msg); err != nil {
t.Fatal(err)
}
if string(msg.Content) != `""` {
t.Fatalf("empty-array content must normalise to \"\", got %s", msg.Content)
}
if len(msg.ToolCalls) == 0 {
t.Fatal("tool_calls must survive normalisation")
}
// A non-empty content array must be left byte-identical (multimodal path).
var mm ChatMessage
multi := `{"role":"user","content":[{"type":"text","text":"hi"}]}`
if err := json.Unmarshal([]byte(multi), &mm); err != nil {
t.Fatal(err)
}
if string(mm.Content) != `[{"type":"text","text":"hi"}]` {
t.Fatalf("multimodal content must pass through untouched, got %s", mm.Content)
}
// A plain string content is untouched.
var sm ChatMessage
if err := json.Unmarshal([]byte(`{"role":"user","content":"plain"}`), &sm); err != nil {
t.Fatal(err)
}
if string(sm.Content) != `"plain"` {
t.Fatalf("string content must pass through untouched, got %s", sm.Content)
}
// An omitted content stays omitted (omitempty semantics preserved).
var om ChatMessage
if err := json.Unmarshal([]byte(`{"role":"assistant","tool_calls":[]}`), &om); err != nil {
t.Fatal(err)
}
if om.Content != nil {
t.Fatalf("absent content must stay absent, got %s", om.Content)
}
if om.ToolCalls != nil {
t.Fatalf("empty tool_calls array must be dropped, got %s", om.ToolCalls)
}
}
// 上游报告输出里有多少是思考 token;缺少这个字段,客户端无法判断
// completion_tokens 里多少是可见回答、多少是思考(两者都按输出计费)。
func TestTokenUsageMarshalsCompletionTokensDetails(t *testing.T) {
u := TokenUsage{
Prompt: 37, Completion: 40, Total: 77,
CompletionTokensDetails: &CompletionTokensDetails{ReasoningTokens: 40},
}
b, err := json.Marshal(u)
if err != nil {
t.Fatal(err)
}
var got map[string]interface{}
if err := json.Unmarshal(b, &got); err != nil {
t.Fatal(err)
}
ctd, ok := got["completion_tokens_details"].(map[string]interface{})
if !ok {
t.Fatalf("completion_tokens_details 未输出: %s", b)
}
if n, _ := ctd["reasoning_tokens"].(float64); int(n) != 40 {
t.Errorf("reasoning_tokens = %v, want 40 (%s)", ctd["reasoning_tokens"], b)
}
// 标准 OpenAI 字段同时存在,老客户端不受影响。
if _, ok := got["completion_tokens"]; !ok {
t.Errorf("标准 completion_tokens 丢失: %s", b)
}
// 未设置时不得凭空造字段。
b2, _ := json.Marshal(TokenUsage{Prompt: 1, Completion: 2, Total: 3})
if strings.Contains(string(b2), "completion_tokens_details") {
t.Errorf("无数据时不应输出 completion_tokens_details: %s", b2)
}
}