Compare commits

...

36 Commits

Author SHA1 Message Date
d81bfb1460 chore(version): 1.6.0
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-26 14:21:51 +08:00
a8cff57e24 feat(deploy): 部署前校验 master.key 可解封配置 + llmsproxy -show-secrets
密钥校验(deploy.sh)
- 新增 verify_master_key,在 build/替换任何文件之前执行。失败则二进制与
  配置分毫未动、服务不受影响(已负向验证:缺钥匙、错钥匙两种情况都挡住)
- 走真实的 -show-secrets 解密路径,而不是只检查钥匙文件格式——格式合法
  但内容不匹配(重新生成、恢复了错的备份、换机器)同样会被拒
- 钥匙来源与 config 包一致:LLMS_PROXY_MASTER_KEY 优先,否则
  dirname(runtime_file)/master.key
- 配置里没有密文时跳过并提示(首次加密场景)

-show-secrets
- llmsproxy -show-secrets -config <path>:把凭据打到 stdout 后退出
- 不启动任何东西、不写任何文件(已验证 mtime 不变)
- 加密往返无损:封存前后输出逐字节一致

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-26 14:21:44 +08:00
926b9f6565 feat: WebUI 改为依赖 /api/v1,UI 与 agent 共用一套 API 契约
- sources / sort / keys 三个页面的数据源从 /api/sources 切到 /api/v1/sources
  (写操作仍走 /api/sources:v1 是只读门面,不做变更)
- 编辑弹窗改用 /api/v1/sources/{name}?reveal=credentials(admin-only)取明文 key。
  这是必须的:表单要整体回传源,若不回填 key,改个端口就会把 key 清空。
- 遮蔽视图仍是默认,只有显式 reveal 才返回明文

端到端验证(真浏览器 + 临时实例,非仅 API 测试):
- sources/sort/keys 三页实际发出 GET /api/v1/sources,0 console error
- editSource('demo') → reveal=credentials,#s-key 与 #s-url 正确回填
- 写入往返:改 base_url /v1→/v2 后重开,key 仍在(未被清空)
- 落盘 api_key 明文残留 0、密文 1

测试:+1(reveal 必须 admin,否则任意 user key 可读全部凭据)
变异验证:reveal 去掉 admin 校验 → 403 断言变红

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-26 14:08:44 +08:00
ad28a924a5 feat: 密钥静态加密 + /api/v1 agent 管理 API
密钥加密(写侧封存 / 读侧解封)
- config.yaml 的 sources[].api_key、sources[].headers、keys[].key 落盘即
  AES-256-GCM 密文(enc:v1: 前缀),master.key 复用 runtime store 那把
- 内存里永远是明文:鉴权比对、API 返回新建 key、WebUI 编辑回填都不受影响
- 启动时一次性封存现存明文(幂等,已封存则不写盘);-check 不写文件
- UpsertSourceInYAML 增加 box 参数,新加的源不再以明文落盘
- 解密失败改为硬错误:原先 MustDecrypt 返回密文会被下次 Save 二次封存
  (实测:源 key 18→20、静默损坏),现在启动即失败且配置分毫不动

/api/v1:面向 agent 的管理 API(WebUI 零影响)
- GET /api/v1            机器可读索引,列出每个端点的方法/权限/用途
- GET /api/v1/overview   一次调用看全貌:源 + AUTO 链 + 密钥数 + 健康度
- GET /api/v1/health     仅健康快照
- GET /api/v1/models     按源分组的可路由模型清单
- GET /api/v1/sources[/{name}]  凭据遮蔽后的源
- GET /api/v1/auto       调度链与实时槽位状态
- GET /api/v1/keys       admin only,密钥元数据,绝不回显密钥本身
- 沿用同一套网关 key 鉴权;读端点任意角色,写仍需 admin

测试:15 个新用例(含负向:泄密、越权、写操作必须被拒)
变异验证:maskKey 不遮蔽→红、去掉 admin 校验→红

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-26 14:01:47 +08:00
432129243e chore(version): 1.5.9 2026-09-11 18:19:47 +08:00
c744ee151e feat(opencode): 透传 completion_tokens_details.reasoning_tokens 与上游 cost
回答「opencodego 的用量与费用透传呢」时逐字段核对上游产出,发现 usage 漏了
一项、费用整项丢失。

## 上游实际发什么(实测 opencode.ai/zen/go/v1)

  {
    "choices": [...],
    "usage": { "prompt_tokens": 37, "completion_tokens": 40, "total_tokens": 77,
               "prompt_cache_hit_tokens": 0, "prompt_cache_miss_tokens": 37,
               "prompt_tokens_details": {"cached_tokens": 0},
               "completion_tokens_details": {"reasoning_tokens": 40} },
    "cost": "0"
  }

cost 在**顶层**且是**字符串**。流式时还会单独发一帧:
{"choices":[],"cost":"0"}

## 此前丢了两样

1. completion_tokens_details.reasoning_tokens —— 输出里有多少是思考 token。
   没有它,客户端无法判断 completion_tokens 里多少是可见回答、多少是思考,
   而两者都按输出计费。
2. cost —— 唯一的费用信号,网关整个丢弃。Go 订阅是包月制恒为 "0",
   但 Zen 按量付费模型(以及未来的其它源)有信息量。

顺带修掉一处流式/非流式不一致:命中缓存时上游同时给
prompt_tokens_details.cached_tokens 和独立的 hit/miss,流式路径写成了 elseif,
只留 details,与非流式产出不同(只认独立字段的老客户端会看不到缓存)。

## 实现

- types.TokenUsage += CompletionTokensDetails;UnifiedResponse / UnifiedChunk += Cost
- opencodego/opencodezen 适配器映射两个字段;空 choices 帧改成 usage 与 cost
  都可带(早退只带 usage 会把同帧的 cost 丢干净 —— 新测试先抓到的就是这个)
- Gateway ChatCompletion / ChatChunk += cost,随终帧发(对齐上游的
  {"choices":[],"cost":"0"} 形态)
- Go 兜底 standardSSEChunk 同步支持(openai 系适配器不再漏 reasoning_tokens;
  纯 cost 帧不再被整体丢弃),新增 rawCostString 兼容字符串/数字两种形态

费用只做**搬运**:不解析、不换算、不汇总 —— 它是上游事实,且只有部分上游提供。

## 验证

经网关实测 gozen:deepseek-v4.1-flash,流式与非流式产出逐字段一致:
  prompt_tokens_details.cached_tokens=6784
  prompt_cache_hit_tokens=6784 / miss=148
  completion_tokens_details.reasoning_tokens=16
  cost="0"

测试:TestOpenCodeCostAndReasoningPassthrough(含「无数据不得凭空造字段」反例)、
TestOpenCodeStreamCacheFieldsMatchNonStream、TestTokenUsageMarshalsCompletionTokensDetails。
2026-09-11 18:19:40 +08:00
55f5d7a0f4 chore(version): 1.5.8 2026-09-11 16:54:36 +08:00
c9c09b2ba2 fix(opencode): 采纳客户端真实会话 id + 超窗消息不再被限流措辞封杀
两处都源于同一次排查:pi 到底有没有带会话标识、超窗为什么触发不了压缩。

## 1) 客户端会话 id:pi 一直在发,只是被配置关掉了

之前结论是「通用客户端不发会话 id」——只对了一半。pi 有会话 id,且能发:
pi-ai 的 createClient 在 compat.sendSessionAffinityHeaders 为真时,会把
平台会话 id(uuidv7,整个会话恒定)放到 x-session-affinity /
x-client-request-id / session_id 上。该开关默认 false,而 llmsproxy 的
provider 配置里没开,所以此前一直收不到。

现在网关按优先级采纳:x-session-affinity → x-session-id → session_id →
body 的 prompt_cache_key,并把值经 types.ChatRequest.ClientSession 传到
适配器 meta.client_session。适配器的会号种子优先级变为:
客户端会话 id > 首条 user 消息指纹 > 按源固定。

刻意不采纳 x-client-request-id:名字含 request,部分客户端每请求都换,
拿它当会话会让上游前缀缓存永不命中(pi 总会同时发 x-session-affinity,够用)。

实测:抓 127.0.0.1:8081 的真实 pi 请求,配置打开后收到
x-session-affinity = session_id = x-client-request-id = <子会话 uuid>。
上游缓存确为会话级隔离(同前缀、不同会号:A 冷→命中,B 首次仍为 0),
两个不同 header 值互不命中,反证网关确实采纳了客户端会话 id。

## 2) 超窗消息必须「干净」,否则被同链的限流措辞反向封杀

pi 的 isContextOverflow 先查 NON_OVERFLOW_PATTERNS(/rate limit/、
/too many requests/、Bedrock 前缀),命中就直接判为「非超窗」——**即使
消息里已经有 context_length_exceeded**,pi 也不会压缩重试。

而 AUTO 链的失败消息天生是多 tier 原因的拼接,超窗 tier(gozen 400
maximum context length)常与配额/限流 tier(429 token plan exhausted、
cooling、no free slot)同时出现。此前把 tier 明细原样拼在归一化标记后面,
等于让一条限流 tier 的措辞反过来封杀超窗识别。

现在超窗走独立的干净消息:
  context_length_exceeded: context window is full; reduce the length of
  the messages (gozen/deepseek-v4.1-flash)
只留超窗措辞 + 超窗源名,不带任何其它 tier 的文本。

测试:TestOverflowMessageSurvivesRateLimitedSiblingTier 用 pi 的完整判定
顺序(先 NON_OVERFLOW 后 OVERFLOW)断言同链限流 tier 不再封杀超窗识别;
TestClientSessionFromRequestHeaders / TestClientRequestIDIsNotUsedAsSession /
TestOpenCodePrefersClientSessionID 覆盖会话采纳与优先级。
2026-09-11 16:54:31 +08:00
091c9c961f chore(version): 1.5.7 2026-09-11 16:04:21 +08:00
39b48e556e feat(opencode): per-conversation session via first-user-message fingerprint
Follow-on to the session-stability fix. "Per source" already made the
prefix cache hit, but it puts every conversation into one upstream session.

Using the client's own session id is not possible: capturing real agent
traffic (tcpdump on 127.0.0.1:8081) shows generic OpenAI clients send NO
session identifier at all — no user / session_id / conversation_id /
metadata in the body, and no session header (only X-Stainless-* plus
User-Agent: pi). The x-opencode-session the Go endpoint asks for is an
OpenCode native-client concept that a generic client cannot forward.

Since history is replayed every turn, the FIRST user message is invariant
for the life of a conversation, so it is used as the conversation
fingerprint. The session becomes stable within a conversation and distinct
across conversations; requests with no user message fall back to per-source
stability.

Measured through the gateway (same 5.7k-token prompt): 2nd call
cached_tokens=5504, and an unrelated conversation gets its own session.

Test: TestOpenCodeSessionIsStableForCache covers same-conversation
stability, cross-conversation separation, per-request request ids and the
sessionless fallback.
2026-09-11 16:04:17 +08:00
7df413453b chore(version): 1.5.6 2026-09-11 15:42:26 +08:00
791d198f47 fix(opencode): make x-opencode-session stable so the upstream prefix cache can hit
The opencode adapters derived x-opencode-session from meta.timestamp, i.e. a
brand new session on every request. The upstream prefix cache is
session-scoped, so no request could ever hit it, and the cache fields the
endpoint does report (prompt_tokens_details.cached_tokens,
prompt_cache_hit_tokens/prompt_cache_miss_tokens) always came back 0/absent.

Measured against the live endpoint, same 6032-token prompt:

  fixed session id   -> 2nd call: hit 5888, miss 144
  rotating session id -> every call: hit 0, miss 6032

Fix: derive the session from the source name (stable), matching how
x-opencode-project is already derived. x-opencode-request stays unique per
request — it is only a request identifier, not part of the cache key.
Applied to both opencodego and opencodezen.

Through the gateway the same prompt now reports, on the 2nd call:
  details={'cached_tokens': 5888} hit=5888 miss=144      (non-streaming)
  prompt_tokens_details={'cached_tokens': 5888}          (streaming)

Test: TestOpenCodeSessionIsStableForCache asserts the session is stable
across requests for one source while the request id differs.
2026-09-11 15:42:22 +08:00
1f7d17feac chore(version): 1.5.5 2026-09-11 15:28:17 +08:00
d1a72cd23a fix(opencodego): inject empty reasoning_content on tool-calling turns
v1.5.4 stopped stripping reasoning_content, which fixes clients that send
it — but most agent clients (pi included) never store or replay their
reasoning, keeping only the tool call. OpenCode Go validates the field on
any assistant turn that carries tool_calls and rejects the whole request:

  400 invalid_request_error: The `reasoning_content` in the thinking mode
  must be passed back to the API.

Verified against the live endpoint that an EMPTY string satisfies the
check, so the adapter now fills in "" when a tool-calling assistant turn
has no reasoning_content. Nothing is fabricated: the reasoning shown to
the client is still exactly what the upstream returned for that turn.

Measured: with a tool_call + tool_result history and no reasoning_content,
all 25 configured Go models returned 400 before and all 25 answer
correctly now.

Test: TestOpenCodeGoVsZenReasoning also pins that a plain assistant turn
(no tool calls) must NOT gain the field.
2026-09-11 15:28:14 +08:00
19c4fc3a13 chore(version): 1.5.4 2026-09-11 15:14:14 +08:00
71e9040a45 feat(adapters): split opencode into opencodezen and opencodego
Zen (https://opencode.ai/zen/v1) and Go (https://opencode.ai/zen/go/v1)
are different services with different requirements, and one shared adapter
could not satisfy both.

The decisive difference is reasoning_content:

  * OpenCode Go runs thinking models and REQUIRES the assistant turn's
    reasoning_content to be echoed back. The shared adapter stripped it
    (msg.reasoning_content = nil), so every replay of a thinking turn
    failed with:
      400 invalid_request_error: The `reasoning_content` in the thinking
      mode must be passed back to the API.
    Reproduced directly: the same request with reasoning_content -> 200,
    without -> 400. That is why the Go tier never worked in an agent loop.

  * The Zen free pool must not receive it, so it keeps stripping.

Both adapters keep the earlier fixes they share (never drop an assistant
turn carrying tool_calls; send stream_options only when streaming; role
whitelist; multimodal strip) and the opencode client fingerprint headers —
the Go endpoint additionally REQUIRES x-opencode-session, which the
adapter already sends.

config: localzen -> opencodezen, gozen -> opencodego.
Verified: all 25 gozen models answer correctly through the gateway with a
thinking + tool_call + tool_result history (was 0/25 before), streaming
included; the Zen free models still pass.

Test: TestOpenCodeGoVsZenReasoning pins the Go-keeps / Zen-strips split.
2026-09-11 15:00:45 +08:00
0528941e24 fix(opencode): only send stream_options with stream:true
OpenCode Go (and other strict OpenAI-compatible upstreams) reject a
non-streaming request that carries stream_options with
"stream_options should be set along with stream". The adapter attached it
unconditionally, so every non-stream call through the opencode adapter
failed on those upstreams.

Verified against OpenCode Go: 25/25 configured models now pass a real
completion through the gateway (they previously 400'd).

Test: TestOpenCodeStreamOptionsOnlyWhenStreaming (absent when
non-streaming, include_usage present when streaming).
2026-09-11 13:13:05 +08:00
12af22fda6 chore(version): 1.5.3 2026-09-10 21:57:58 +08:00
9114468753 fix: empty-array content becomes invalid {} on every pass-through adapter; gemini/ollama drop tool calls
Three related forwarding defects found by auditing every adapter with a
tool-calling replay (assistant turn with content:[] + tool_calls).

1) content:[] -> content:{} (all 12 openai-adapter sources, plus
   deepseek/trae/sensenova/agentrouter/github/groq/kimicode/mistral)

   Lua adapters json.decode the request and re-encode it, and an empty Lua
   table is indistinguishable from an empty JSON array — the encoder emits
   {} for both. Agent clients serialise a tool-calling assistant turn with
   no text as content:[], so every pass-through adapter rewrote it to
   content:{} — not valid OpenAI (content is string|array|null). Verified
   against a live upstream: content:[] produced "400 invalid arguments"
   while content:"" was accepted.

   Fixed once at the decode boundary (types.ChatMessage.UnmarshalJSON):
   empty-array content normalises to "" and an empty tool_calls array is
   dropped, so every adapter — including future ones — sees a valid shape.

2) gemini dropped tool_calls and never emitted functionCall /
   functionResponse; the tool role also stayed as an invalid role inside
   contents and system was not moved to systemInstruction.

3) ollama copied only role/content, dropping tool_calls and the call
   attribution entirely (it needs tool_name, not tool_call_id).

Test: TestAdaptersPreserveToolCalls asserts, for every adapter, that the
call id (or function name where the wire format has no id), the function
name, the tool result and the trailing user turn all survive, plus a
negative control for plain text.
2026-09-10 21:57:54 +08:00
611e975456 fix(gateway): 上下文超窗错误归一化,客户端才能压缩重试
问题:上游返回上下文超窗时,客户端(pi)既不压缩也不重试,只看到一条
普通上游错误。链路两处叠加:

1. 措辞不在客户端识别列表里。pi 靠 @earendil-works/pi-ai 的
   OVERFLOW_PATTERNS(25 条正则)判断超窗,而 justworker 返回的是
   「请精简对话历史…(Context window is full…)」——与那 25 条一条都不匹配
   (最接近的 /context window exceeds limit/i 也不命中,因为措辞是 "is full"
   而非 "exceeds limit")。
2. AUTO 链路的每条 tier 错误按 80 字节 OneLine 截断,而 "Context window
   is full" 这类短诊断词常出现在尾部,正好被按字节切掉(直连路径是 160
   才保住)。

修法:
- 新增 overflowMarkers 识别超窗措辞(含中文写法),命中时把客户端可见
  错误归一化为 context_length_exceeded 前缀——它命中 pi 的
  /context[_ ]length[_ ]exceeded/i,超窗因此可被发现并触发压缩重试。
- AUTO 链路每条 tier 错误宽度 80→160,短诊断词不再被截断。

归一化只加前缀,原始诊断信息保留,便于定位是哪一层超窗。

测试:internal/gateway/overflow_err_test.go(5 例,含「标记必须命中 pi
正则」、非超窗不得误标、80 vs 160 宽度的回归对比)。
2026-09-10 20:38:34 +08:00
6733534282 chore(version): 1.5.2 2026-09-10 19:45:31 +08:00
499f0cac2f fix(opencode): never drop an assistant turn that carries tool_calls
An agent client (pi) serialises an assistant turn whose content is only
[thinking, toolCall] as content:[] with tool_calls. The multimodal-strip
pass treated an empty content array as 'nothing left, drop the whole
message' and discarded the tool_calls with it.

The next message is that call's tool result, so it arrived orphaned: the
model saw a result for a call it had never made and re-issued the same
call on every turn — an endless repeated-tool-call loop. Reproduced
against a capture sink: content:[] lost tool_calls, while content:"" and
content:null kept them.

Only messages with neither usable content NOR a tool call now get
dropped. Content that collapses to empty but still has tool_calls or a
tool_call_id is emitted as "" instead.

Test: TestOpenCodeKeepsToolCallWithEmptyContent (plus a negative control
that an image-only message without tool calls is still dropped).
2026-09-10 19:32:47 +08:00
7fb8f96b82 fix(gateway): AUTO scope grants all models — restrict to routing mode only
A key with scope=[AUTO] could previously:
1. request ANY concrete model id directly (hasScopeModel/checkModelScope
   treated AUTO as a wildcard)
2. see the full 56-model list on /v1/models (intersectModels considered
   AUTO as grant-everything)

AUTO now only authorizes the AUTO routing mode. Direct requests to a
specific model require an explicit scope entry.

Also carries agentrouter.lua WAF fingerprint headers (Origin/Referer/
X-Requested-With) already staged on this branch.

Tests: TestHasScopeModelWithSourcePrefix updated; full suite green.
2026-09-10 12:51:40 +08:00
6a34cd6f86 chore(version): 1.4.2 -> 1.5.0 2026-09-06 10:17:43 +08:00
f0986751de Merge feature/agentrouter-id-sanitize: 补上最后一个服务 Claude 的适配器
全仓审计发现 agentrouter 也暴露 claude-opus-4-8,却是唯一没有 id 消毒的
Claude 适配器。补齐后,所有含 claude/opus/sonnet 模型的源(qijiar、toter、
juziai、agentrouter、justwoker、api456、tabitoken)全部走消毒路径;
测试新增三适配器结果一致性断言。
2026-09-06 10:08:52 +08:00
c6c3e0dcd7 fix(agentrouter): sanitize tool-call ids — it fronts Claude too
Full-repo audit after the anthropic/openai fix: agentrouter exposes
claude-opus-4-8, so it inherits Anthropic's tool id rule
^[a-zA-Z0-9_-]{1,64}$ and rejects the whole request on a violation, exactly
like justwoker/tabitoken/扇贝. It was the only remaining adapter serving Claude
models without the sanitizer, so a client that had picked up a dirty id (e.g.
"bash:0" from moonshotai/kimi-k3) would still lose every turn here.

Same shape as the other two — inbound tool_calls[].id + tool_call_id, outbound
non-streaming ids and the first streamed fragment — and the test now asserts
all three adapters rewrite an identical input identically, so a client mixing
sources within one session cannot end up with unpaired tool calls.

Audit result: every source exposing a claude/opus/sonnet model (qijiar, toter,
juziai, agentrouter, justwoker, api456, tabitoken) now routes through a
sanitizing adapter.
2026-09-06 10:08:52 +08:00
b9944332ff Merge feature/anthropic-usage-cache: 缓存命中不该让 prompt_tokens 算错
Anthropic 的 input_tokens 不含缓存部分,OpenAI 的 prompt_tokens 含。
直接对映射等于漏算全部缓存 token,还能算出 >100% 的命中率;
cache_creation 完全没读;流式路径连缓存字段都整个丢掉。
三项相加 + 共享 map_usage,未上报缓存的上游输出逐字节不变。
2026-09-06 09:55:22 +08:00
6bdb9fcc44 fix(anthropic): count cached input in prompt_tokens instead of dropping it
Anthropic and OpenAI disagree on what the prompt count means:

  Anthropic: input_tokens EXCLUDES cached blocks; cache_read_input_tokens and
             cache_creation_input_tokens are separate, additive, billed input.
  OpenAI:    prompt_tokens INCLUDES its cached_tokens subset.

anthropic.lua mapped input_tokens straight onto prompt, so a cache-heavy turn
was doubly wrong: the billed prompt was undercounted by the entire cache
portion, and cached_tokens could exceed prompt_tokens — a cache hit rate above
100% for any client that divides one by the other. cache_creation_input_tokens
was never read at all, so a cache-write turn silently lost those billed tokens.

Worse, the streaming path dropped the cache split entirely: message_delta
carries the FINAL usage and only mapped input/output, so every streamed
response reported no cache information even when the upstream sent it.

All three counts are now summed into prompt, with the read half exposed as
prompt_tokens_details.cached_tokens plus the DeepSeek-legacy hit/miss pair, via
one shared map_usage() used by transform_response, message_start and
message_delta. A reported zero stays distinguishable from "never reported": the
split is emitted whenever either cache field is present, and omitted entirely
when the upstream mentions neither (justwoker reports only input/output plus its
own cost fields, so its output is byte-identical to before). map_usage returns
nil for a countless object, preserving "no usage in this chunk means say
nothing" rather than reporting zeros.

message_start's placeholder count is still emitted: justwoker reports 160 there
and the real 6931 in message_delta, and the gateway's mergeUsage lets the later
non-zero value win.
2026-09-06 09:55:22 +08:00
25eb30c15b Merge feature/toolcall-id-sanitize: 一个上游的脏 tool_call id 不该拖垮整条 Claude 链
xinjianya/moonshotai/kimi-k3 会吐出 'bash:0' 这种 id。OpenAI 不校验,
Anthropic 校验 ^[a-zA-Z0-9_-]{1,64}$ 且整条请求直接 400。客户端会把它存进
历史并回放给所有源,于是 justwoker / tabitoken / 扇贝(都是 Claude 上游)
同时挂掉,AUTO 一路穿透四个 tier。

两个适配器、进出双向消毒;已合法的 id 逐字节透传;流式参数分片保持无 id。
2026-09-05 22:19:15 +08:00
131a42a169 fix(adapters): sanitize tool-call ids so one bad upstream can't kill every Claude slot
Anthropic requires tool_use.id / tool_result.tool_use_id to match
^[a-zA-Z0-9_-]{1,64}$ and rejects the WHOLE request otherwise with
REQUEST_BODY_INVALID / "Invalid tool use format". OpenAI has no such rule, so
an OpenAI-compatible model can mint an id like "bash:0"
(xinjianya/moonshotai/kimi-k3 does exactly that).

In a fan-out router that id does not stay local: the client stores it in its
history and replays it to every other source. One such id therefore kills
every Claude slot at once — justwoker, tabitoken and 扇贝 are all
Claude-behind-{OpenAI,Anthropic} — and an AUTO request falls through all four
tiers to whatever tolerant model is left. Observed live: 4 consecutive 503
"all N auto providers failed" with tier 1/2/3 each reporting the same 400.

Both directions are sanitized, in both adapters:
  - request:  tool_calls[].id and tool_call_id, so poisoned history recovers
  - response: non-streaming tool_calls[].id and the first streamed fragment,
              so a bad id never enters a client session in the first place

safe_tool_id is pure and deterministic, so a call and its result are rewritten
identically within one request. A rewritten id keeps an 8-hex digest of the
original, without which distinct ids could collapse ("a:b" and "a_b") into a
duplicate/unpaired tool_use. Already-legal ids pass through byte-identical, so
well-behaved traffic is unaffected. openai.lua carries its own copy because
Lua adapters have no shared prelude.

Streamed argument fragments carry no id and must stay id-less, otherwise
index-based accumulation on the client breaks; a test pins that.
2026-09-05 22:18:31 +08:00
3cdb16c906 Merge feature/deploy-config-rollback: 部署脚本把配置纳入回滚闭环
上一轮 446 次重启循环的根因不是代码,是 deploy.sh 把 config.yaml 当作
部署范围之外的东西:健康检查失败后只回滚二进制,坏配置仍在,于是旧
二进制照样解析不了,服务在 systemd 重启循环里空转。

修法是把配置变成一等部署对象:--config 投放 + 重启前 -check 预检 +
回滚时二进制与配置一起恢复。预检位置比预检本身更重要——排在 restart
之前,坏配置才会在服务仍健康时被拦下。
2026-09-05 09:47:04 +08:00
ef92ce4d82 Merge feature/source-proxy: 让被 CF/格式问题挡住的源真正可用
两件事都在回答同一个问题——为什么配了源却用不了:

- trae 源在流式下把模型打印的 tool_call 当纯文本透传,客户端收到
  finish_reason "stop" + 一段文本,agent 循环当场终止
- justwoker/tabitoken 坐在 Cloudflare 后面,直连 403,必须走代理 +
  浏览器 UA;但全局 env 代理会把 trae/localzen 等内网源也绕出去

因此引入 per-source proxy_url:指定的源走专属代理,未指定的保持直连。
2026-09-05 09:46:54 +08:00
c524831616 fix(deploy): roll back config together with the binary, and preflight it before restart
The 446-restart-loop incident: adding a headers block to a source without
removing the source's existing `headers: {}` produced a duplicate YAML key.
The process exited on startup, healthcheck failed, and rollback restored only
the binary — so the old binary kept parsing the same broken config and the
service span in systemd's restart loop. Config was treated as out of scope
for deployment; it is not.

Three changes close the loop:

1. cmd/llmsproxy: new `-check` flag validates a config (parse +
   ApplyDefaults) and exits, without starting the Lua VM, touching
   runtime.json, or binding a port — safe to run against a live service.
   Unlike normal startup it does NOT create a default config, so a missing
   file is an error.

2. deploy.sh `--config <file>`: stage a config for deployment, atomically
   renamed into place with the same copy->rename(2) technique as the binary.
   Omitted means the live config is left alone.

3. Ordering: config replacement and preflight both run BEFORE
   restart_service, so an invalid config is caught while the service is still
   healthy and never triggers a restart. rollback() now restores binary AND
   config (only when this run replaced it, so concurrent WebUI edits survive),
   then re-runs -check before restarting — refusing to restart into a config
   that still fails, instead of trading one restart storm for another.

Also: the sha256-unchanged early exit now only fires when there is no pending
config, otherwise `--config` would be silently dropped.

Verified on the live deployment:
- reproduced the exact duplicate-key config: preflight caught it, PID
  unchanged (zero interruption), binary and config both rolled back, gateway
  still answering 200
- valid config: replaced, service restarted, new value live
- no --config: binary-only deploy unaffected
- go test -tags luajit ./... passes
2026-09-05 09:46:43 +08:00
868ac59692 feat(provider): support per-source proxy_url for geo-blocked upstreams
Upstreams like justwoker/tabitoken sit behind Cloudflare geo/IP blocks and
only respond through a proxy. A global env proxy is wrong (intranet sources
trae/localzen must stay direct), so add an explicit per-source proxy_url
that overrides http.ProxyFromEnvironment for just that source. Sources
without proxy_url keep the existing env/direct behavior.

Also accepts a User-Agent header per source (config already supported
headers) so CF-fronted resellers can be reached.

Verified e2e: tabitoken and justwoker now return tool_calls through
127.0.0.1:7890 (clash); trae stays direct. Full go test -tags luajit
passes.
2026-09-05 09:46:36 +08:00
4d197e4bd3 fix(trae): recover legacy [Called tool:...] tool call format
Root cause: trae-local-api is deployed to users without the "Fold past
assistant tool_calls into [Called tool: name({...})]" text format that
their own histories already contained. Trae-Local-API-LLM then mimics this
format in subsequent responses. The trae.lua parser only recognized
<tool_call>...</tool_call> or <toolcall>...</toolcall> tags, so
[Called tool: ...] responses were left unparsed and the client received
plain text where a structured tool_calls array should be.

Fix: Add a legacy pattern match at the END of parse_text_tool_calls to
catch the [Called tool: name({args})] shape and emit proper tool_calls.
This is a fallback; models should emit <tool_call> tags per system prompt,
but we tolerate the mimicked form for robustness.
2026-09-05 09:46:36 +08:00
3806aaee03 docs(workflow): codify the branch model — main / feature / release branches
Adopt GitHub Flow + release branches, replacing "everything straight to main
plus a tag" which caused the 1.4.2 pain (a fix had to be retro-fitted to the
released version, forcing a remote-tag delete + full re-upload).

- main: only long-lived branch, always deployable, accumulates the next version
- feature/<desc>: born from main, merged back when done
- release/vX.Y.Z: cut from main, tagged, installers built from the tag
- hotfixes land on the release branch AND are cherry-picked back to main so
  main never loses a fix
- end of lifecycle = retire the release branch (delete; or keep for long-term
  maintenance), no wholesale merge back — hotfixes already flowed
- explicitly no rebase of main, no release-branch-merge, no quick edits on main

Companion release checklist includes the upload lessons (PUT --http1.1) and the
replace-artifacts-by-deleting-the-tag catch.

Docs in docs/git-workflow.md (zh) and docs/git-workflow-en.md, linked from both
READMEs.
2026-08-31 12:36:17 +08:00
35 changed files with 4568 additions and 157 deletions

View File

@ -490,6 +490,15 @@ internal/gateway # OpenAI 兼容 HTTP 服务 + 鉴权 + SDK/流式 + Web
internal/types # 统一格式 & OpenAI wire 类型
```
## 开发协作
- **Git 分支工作流**:主分支 / 特性分支 / 发布分支的规范见
**[docs/git-workflow.md](docs/git-workflow.md)**([English](docs/git-workflow-en.md))。
主线:`main` 永可部署 → 特性合入 `main` → 切 `release/vX.Y.Z` 打 tag 发布 →
hotfix 提交发布分支并 cherry-pick 回 `main`。
- **Lua 适配器协议**:见 **[docs/lua-adapters.md](docs/lua-adapters.md)**
([English](docs/lua-adapters-en.md))。
## 测试
```bash

View File

@ -507,6 +507,17 @@ internal/gateway # OpenAI-compatible HTTP + auth + SDK/streaming + WebUI
internal/types # unified format & OpenAI wire types
```
## Development & collaboration
- **Git branching workflow**: main / feature / release-branch conventions in
**[docs/git-workflow-en.md](docs/git-workflow-en.md)**
([中文](docs/git-workflow.md)). The line: `main` is always deployable →
features merge into `main` → cut `release/vX.Y.Z` and tag → release → hotfixes
land on the release branch and are cherry-picked back into `main`.
- **Lua adapter protocol**: see
**[docs/lua-adapters-en.md](docs/lua-adapters-en.md)**
([中文](docs/lua-adapters.md)).
## Tests
```bash

View File

@ -1,12 +1,12 @@
{
"name": "modelrouter-gui",
"version": "1.4.2",
"version": "1.6.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "modelrouter-gui",
"version": "1.4.2",
"version": "1.6.0",
"license": "Proprietary",
"devDependencies": {
"electron": "^33.0.0",

View File

@ -1,6 +1,6 @@
{
"name": "modelrouter-gui",
"version": "1.4.2",
"version": "1.6.0",
"description": "ModelRouter Desktop — embedded ModelRouter core with tray",
"author": "ModelRouter",
"main": "main.js",

View File

@ -23,8 +23,39 @@ import (
func main() {
cfgPath := flag.String("config", "config.yaml", "path to gateway config file")
checkOnly := flag.Bool("check", false, "validate the config file and exit (0 = valid, 1 = invalid); nothing is started and no file is written")
showSecrets := flag.Bool("show-secrets", false, "print the config's credentials in the clear and exit; nothing is started and no file is written")
flag.Parse()
// -check is the deploy-time preflight: parse and validate the config
// without starting the Lua VM, touching runtime.json, or binding a port.
// It deliberately does NOT call EnsureDefault, so a missing file is an
// error here instead of being silently created.
if *checkOnly {
if _, err := os.Stat(*cfgPath); err != nil {
log.Fatalf("[llmsproxy] check: %v", err)
}
if _, err := config.Load(*cfgPath); err != nil {
log.Fatalf("[llmsproxy] check: %v", err)
}
log.Printf("[llmsproxy] check: %s is valid", *cfgPath)
return
}
// -show-secrets is the operator escape hatch for a config whose
// credentials are sealed at rest: it prints them to stdout and exits
// without starting anything or writing anything. Nothing else in the
// program prints a credential, and this path never logs to a file.
if *showSecrets {
if _, err := os.Stat(*cfgPath); err != nil {
log.Fatalf("[llmsproxy] show-secrets: %v", err)
}
if err := config.PrintSecrets(*cfgPath); err != nil {
log.Fatalf("[llmsproxy] show-secrets: %v", err)
}
return
}
created, err := config.EnsureDefault(*cfgPath)
if err != nil {
log.Fatalf("[llmsproxy] config: %v", err)

234
deploy.sh
View File

@ -10,6 +10,16 @@
# 4. systemctl restart → 服务中断仅在 Go 进程重启瞬间 (<500ms)
# 5. 健康检查确认新进程正常
#
# 配置文件(2026-09-05 起):
# 本脚本不再假设 config.yaml 与部署无关。实测教训:给源加 headers 时
# 漏删原有的 `headers: {}` → YAML 重复键 → 进程启动即退出 → 健康检查
# 失败 → 旧的 rollback 只恢复二进制,配置错误仍在,于是旧二进制也起不来,
# 服务陷入 systemd 重启循环(当时累计 446 次)。
# 因此:
# - 支持 --config <file> 投放一份待部署配置(原子 rename 到 TARGET_CONFIG)
# - 重启前先用新二进制 `-check` 预检配置,不合法就地失败,绝不重启服务
# - rollback 同时恢复二进制与配置文件,回滚后再次 -check 才重启
#
# 注意事项:
# - 本服务依赖自身(推理依赖),restart 前后会短暂中断,
# 但重试机制(systemd RestartSec=5 + 客户端 retry)可自愈
@ -19,6 +29,7 @@ set -euo pipefail
# ---------- 配置 ----------
readonly TARGET_BIN="/usr/local/bin/llmsproxy"
readonly TARGET_CONFIG="/etc/llmsproxy/config.yaml"
readonly TARGET_ADAPTERS="/etc/llmsproxy/adapters"
readonly REPO_DIR="/home/program/llmsproxy"
readonly HEALTH_URL="http://127.0.0.1:8081/v1/models"
@ -28,8 +39,40 @@ readonly BIN_NAME="llmsproxy"
readonly TMP_DIR="/tmp/llmsproxy-deploy"
# 同文件系统暂存文件(用于原子 rename)
readonly STAGING_BIN="${TARGET_BIN}.staging"
# 回滚备份:健康检查失败时从这里恢复旧二进制
readonly STAGING_CONFIG="${TARGET_CONFIG}.staging"
# 回滚备份:健康检查失败时从这里恢复旧二进制 / 旧配置
readonly BACKUP_BIN="${TARGET_BIN}.bak"
readonly BACKUP_CONFIG="${TARGET_CONFIG}.rollback.bak"
# 待部署配置(--config <file> 指定;为空 = 不改动线上配置)
NEW_CONFIG_SRC=""
usage() {
cat <<'EOF'
用法: ./deploy.sh [--config <file>]
--config <file> 投放这份配置到 /etc/llmsproxy/config.yaml(原子替换)。
省略时不改动线上配置,只部署二进制与适配器。
无论是否指定,重启前都会用新二进制 -check 预检线上配置。
流程: 构建 → 适配器同步 → 备份(二进制+配置) → 原子替换 → 配置预检
→ 重启 → 健康检查 → 失败则回滚(二进制+配置)并重启
EOF
}
parse_args() {
while (( $# > 0 )); do
case "$1" in
--config)
[[ $# -ge 2 ]] || { echo "--config 需要一个文件参数" >&2; exit 2; }
NEW_CONFIG_SRC="$2"
shift 2
;;
-h|--help) usage; exit 0 ;;
*) echo "未知参数: $1" >&2; usage; exit 2 ;;
esac
done
}
# ---------- 日志 ----------
log() { printf '\033[1;34m[%s]\033[0m %s\n' "$(date +%H:%M:%S)" "$*"; }
@ -40,7 +83,7 @@ fail() { err "$@"; exit 1; }
# ---------- 清理 ----------
cleanup() {
log "清理临时文件"
rm -rf "$TMP_DIR" "$STAGING_BIN"
rm -rf "$TMP_DIR" "$STAGING_BIN" "$STAGING_CONFIG"
}
trap cleanup EXIT
@ -52,6 +95,18 @@ precheck() {
command -v systemctl &>/dev/null || fail "未找到 systemctl"
[[ $(id -u) -eq 0 ]] || fail "需要 root 权限"
if [[ -n "$NEW_CONFIG_SRC" ]]; then
[[ -f "$NEW_CONFIG_SRC" ]] || fail "待部署配置不存在: $NEW_CONFIG_SRC"
# 同一文件比对:--config 指向线上文件本身时没有"投放"动作可做,
# 但仍然会走重启前的 -check 预检。
if [[ "$(readlink -f "$NEW_CONFIG_SRC")" == "$(readlink -f "$TARGET_CONFIG" 2>/dev/null)" ]]; then
warn "--config 指向线上配置本身,跳过投放(仍会预检)"
NEW_CONFIG_SRC=""
else
log "待部署配置: $NEW_CONFIG_SRC → $TARGET_CONFIG"
fi
fi
OLD_SIZE=$(stat -c%s "$TARGET_BIN" 2>/dev/null || echo 0)
OLD_SHA256=$(sha256sum "$TARGET_BIN" 2>/dev/null | awk '{print $1}')
@ -59,8 +114,10 @@ precheck() {
log "go: $(go version)"
}
# ---------- 备份旧二进制(用于回滚)----------
backup_old_binary() {
# ---------- 备份旧二进制 + 旧配置(用于回滚)----------
# 配置也必须备份:启动失败常常是配置问题,只回滚二进制会让旧二进制
# 继续吃坏配置,服务照样起不来。
backup_old_state() {
log "备份旧二进制到 $BACKUP_BIN"
if [[ -f "$TARGET_BIN" ]]; then
cp -f "$TARGET_BIN" "$BACKUP_BIN"
@ -68,6 +125,13 @@ backup_old_binary() {
else
warn "旧二进制不存在,跳过备份(无回滚点)"
fi
if [[ -f "$TARGET_CONFIG" ]]; then
cp -f "$TARGET_CONFIG" "$BACKUP_CONFIG"
log "✓ 旧配置已备份到 $BACKUP_CONFIG (sha256=$(sha256sum "$BACKUP_CONFIG" | awk '{print $1}' | cut -c1-16))"
else
warn "线上配置不存在: $TARGET_CONFIG(首次部署?无配置回滚点)"
fi
}
# ---------- 构建 ----------
@ -89,8 +153,13 @@ build() {
if [[ "$NEW_SIZE" -eq "$OLD_SIZE" ]]; then
NEW_SHA256=$(sha256sum "$NEW_BIN" | awk '{print $1}')
if [[ "$NEW_SHA256" == "$OLD_SHA256" ]]; then
warn "新二进制与旧版本完全一致(sha256 相同),跳过部署"
exit 0
# 二进制无变化。只有在同时也没有待部署配置时才能直接退出;
# 否则会造成“--config 指定了新配置却被静默丢弃”。
if [[ -z "$NEW_CONFIG_SRC" ]]; then
warn "新二进制与旧版本完全一致(sha256 相同)且无待部署配置,跳过部署"
exit 0
fi
log "二进制无变化(sha256 相同),但有待部署配置,继续"
fi
fi
@ -140,6 +209,98 @@ atomic_replace_binary() {
log "✓ 二进制已原子替换 (sha256=${NEW_SHA256:0:16})"
}
# ---------- 原子替换配置 ----------
# 与二进制同一套 copy→rename 手法:先落到 /etc 上的 .staging,再 rename(2)。
# 只有 --config 指定了来源时才动线上配置。
atomic_replace_config() {
[[ -n "$NEW_CONFIG_SRC" ]] || return 0
log "原子替换配置"
log " copy → $STAGING_CONFIG (同文件系统暂存)"
cp -f "$NEW_CONFIG_SRC" "$STAGING_CONFIG"
chmod 0644 "$STAGING_CONFIG"
log " rename → $TARGET_CONFIG (原子)"
mv -f "$STAGING_CONFIG" "$TARGET_CONFIG"
log "✓ 配置已原子替换 (sha256=$(sha256sum "$TARGET_CONFIG" | awk '{print $1}' | cut -c1-16))"
}
# ---------- 配置预检 ----------
# 用刚部署的新二进制解析线上配置。`-check` 只做 parse + ApplyDefaults,
# 不启动 Lua VM、不碰 runtime.json、不绑端口,因此可以安全地在服务仍在
# 运行时执行。不合法就在重启前失败——这正是上次事故缺的那一步。
verify_config() {
log "配置预检(新二进制 -check)"
[[ -f "$TARGET_CONFIG" ]] || fail "线上配置不存在: $TARGET_CONFIG"
local out
if out=$("$TARGET_BIN" -check -config "$TARGET_CONFIG" 2>&1); then
log "✓ 配置合法"
return 0
fi
err "配置预检失败,未重启服务(线上进程仍在跑旧配置):"
printf '%s\n' "$out" >&2
return 1
}
# ---------- 密钥可解封校验 ----------
# config.yaml 里的凭据以 enc:v1: 密文落盘,解密钥匙是 master.key(或
# LLMS_PROXY_MASTER_KEY)。一旦钥匙丢失/错位,配置在**启动时**才会报错,
# 那时服务已经处于「二进制已替换 + 配置已替换」的半死状态,只能靠回滚。
# 所以在替换任何东西之前先验:能不能用现有的钥匙解开现有的密文。
#
# 两种情形要分开:
# 1. 配置里**有**密文 ⇒ 钥匙必须能解开(否则拒绝部署)
# 2. 配置里**没有**密文 ⇒ 全新安装/首次加密,无需钥匙(但会顺手记一条,
# 因为它意味着这次部署会把明文写成密文,而钥匙必须同时就位)
verify_master_key() {
log "密钥可解封校验"
[[ -f "$TARGET_CONFIG" ]] || return 0 # 配置都不在,交给 verify_config 报错
local sealed
sealed=$(grep -c 'enc:v1:' "$TARGET_CONFIG" 2>/dev/null || echo 0)
if [[ "$sealed" -eq 0 ]]; then
warn "配置内无密文(尚未加密或全为明文)——首次加密需要 master.key 可写"
return 0
fi
# 钥匙来源与 config 包一致:env 优先,否则 runtime_file 同目录的 master.key
local runtime_file key_file
runtime_file=$(awk '/^runtime_file:/{print $2; exit}' "$TARGET_CONFIG")
[[ -n "$runtime_file" ]] || runtime_file="$(dirname "$TARGET_CONFIG")/runtime.json"
key_file="$(dirname "$runtime_file")/master.key"
if [[ -n "${LLMS_PROXY_MASTER_KEY:-}" ]]; then
log " 钥匙来源: LLMS_PROXY_MASTER_KEY(环境变量)"
# 走真实解密路径验证,而不是只检查格式
if out=$("$TARGET_BIN" -show-secrets -config "$TARGET_CONFIG" 2>&1); then
log "✓ 凭据可解封($(echo "$out" | grep -cE '^(source|key) ') 项)"
return 0
fi
err "凭据无法用 LLMS_PROXY_MASTER_KEY 解封,未替换任何文件:"
printf '%s\n' "$out" >&2
return 1
fi
if [[ ! -f "$key_file" ]]; then
err "配置含 $sealed 处密文,但钥匙文件不存在: $key_file"
err " 没有它,部署后服务将无法启动。恢复 master.key 或设置 LLMS_PROXY_MASTER_KEY 后重试。"
return 1
fi
log " 钥匙文件: $key_file ($(stat -c '%A' "$key_file" 2>/dev/null || echo '?'),$(wc -c < "$key_file") 字节)"
if out=$("$TARGET_BIN" -show-secrets -config "$TARGET_CONFIG" 2>&1); then
log "✓ 凭据可解封($(echo "$out" | grep -cE '^(source|key) ') 项)"
return 0
fi
err "凭据无法解密(钥匙与配置不匹配),未替换任何文件:"
printf '%s\n' "$out" | head -5 >&2
err " 密钥来源: $key_file"
err " 常见原因:master.key 被重新生成、恢复了不匹配的备份,或换了机器。"
return 1
}
# ---------- 同步适配器 ----------
sync_adapters() {
log "同步适配器"
@ -206,20 +367,49 @@ healthcheck() {
}
# ---------- 回滚 ----------
# 健康检查失败时恢复旧二进制并重启,保证推理服务可用。
# 健康检查失败时恢复旧二进制**和旧配置**并重启,保证推理服务可用。
# 这是最后一道保险:即使新版本启动失败(编译错误、配置不兼容、
# panic on start 等),也能让旧版本继续服务,避免推理能力丢失。
#
# 为什么必须一并回滚配置:启动失败最常见的原因就是配置本身不合法
# (实测:YAML 重复键)。只恢复二进制的话,旧二进制照样解析不了坏配置,
# 服务继续在 systemd 重启循环里空转。
rollback() {
err "健康检查失败,启动回滚流程"
if [[ ! -f "$BACKUP_BIN" ]]; then
err "无回滚备份 ($BACKUP_BIN 不存在),无法自动恢复"
local restored_any=0
if [[ -f "$BACKUP_BIN" ]]; then
log "恢复旧二进制"
cp -f "$BACKUP_BIN" "$TARGET_BIN"
chmod 0755 "$TARGET_BIN"
restored_any=1
else
err "无二进制回滚备份 ($BACKUP_BIN 不存在)"
fi
# 只在本次真的换过配置时才回滚配置,避免把用户在部署期间
# 通过 WebUI 做的合法改动一起抹掉。
if [[ -n "$NEW_CONFIG_SRC" && -f "$BACKUP_CONFIG" ]]; then
log "恢复旧配置"
cp -f "$BACKUP_CONFIG" "$TARGET_CONFIG"
chmod 0644 "$TARGET_CONFIG"
restored_any=1
fi
if (( restored_any == 0 )); then
err "无任何回滚点,无法自动恢复"
err "请手动检查: systemctl status llmsproxy.service && journalctl -u llmsproxy -n 50"
return 1
fi
log "恢复旧二进制"
cp -f "$BACKUP_BIN" "$TARGET_BIN"
chmod 0755 "$TARGET_BIN"
# 回滚后先验证恢复出来的组合真的能解析,再重启——否则只是把
# 重启循环换个二进制继续跑。
if ! "$TARGET_BIN" -check -config "$TARGET_CONFIG" >/dev/null 2>&1; then
err "回滚后的配置仍不合法,拒绝重启(避免 systemd 重启风暴)"
err " 手动修复 $TARGET_CONFIG 后执行: $TARGET_BIN -check -config $TARGET_CONFIG"
err " 备份可用: 二进制=$BACKUP_BIN 配置=$BACKUP_CONFIG"
return 1
fi
log "重启服务(旧版本)"
systemctl restart llmsproxy.service || {
@ -248,18 +438,36 @@ print_summary() {
log "═════════════════════════════════════════════"
log "部署完成"
log " 二进制: $TARGET_BIN"
log " 配置: $TARGET_CONFIG$([[ -n "$NEW_CONFIG_SRC" ]] && echo " (本次已替换)" || echo " (未改动)")"
log " 适配器: $TARGET_ADAPTERS"
log "═════════════════════════════════════════════"
}
# ---------- 主流程 ----------
# 顺序要点:配置投放与预检都排在 restart_service **之前**,
# 于是坏配置在服务仍然健康时就被拦下,根本不会触发重启。
main() {
parse_args "$@"
log "═══ llmsproxy 原子部署开始 ═══"
precheck
# 密钥校验放在 build/替换之前:它的失败后果最重(配置里的密文永久解不开),
# 必须在动任何文件之前就挡住。此阶段用的是**当前线上二进制**,它已经
# 支持 -show-secrets(首次引入该子命令的版本除外,此时跳过并提示)。
verify_master_key
build
sync_adapters
backup_old_binary
backup_old_state
atomic_replace_binary
atomic_replace_config
if ! verify_config; then
# 配置不合法:服务还没重启,只需把二进制/配置退回原状。
err "配置预检失败,回退本次改动(服务未受影响)"
[[ -f "$BACKUP_BIN" ]] && cp -f "$BACKUP_BIN" "$TARGET_BIN" && chmod 0755 "$TARGET_BIN"
if [[ -n "$NEW_CONFIG_SRC" && -f "$BACKUP_CONFIG" ]]; then
cp -f "$BACKUP_CONFIG" "$TARGET_CONFIG" && chmod 0644 "$TARGET_CONFIG"
fi
exit 1
fi
restart_service
if ! healthcheck; then
rollback || true

141
docs/git-workflow-en.md Normal file
View File

@ -0,0 +1,141 @@
# Git Branching Workflow
ModelRouter uses a **GitHub Flow + release-branch** model: `main` is the only
long-lived branch and is always deployable; new work branches off it as a
feature branch; each version gets its own release branch with a tag; hotfixes
land on the release branch and MUST flow back to `main`.
Goal: make "patching an already-released version during its support window" a
first-class operation, while `main` always contains every fix and stays
deployable.
## Branch roles
| Branch | Name | Lifetime | Purpose |
|---|---|---|---|
| Main | `main` | permanent | Only long-lived branch. Always deployable. Accumulates the next version. |
| Feature | `feature/<desc>` | short (dev → merge → delete) | New features / ordinary fixes. Born from `main`, merged back into `main`. |
| Release | `release/vX.Y.Z` | one version cycle | Cut from `main`, tagged for release. Version-specific hotfixes land here. |
## Change flow (important)
```
feature/foo ───────┐
▼
main ──────────────── next version ──────►
│ │
│ cut │ cut
▼ ▼
release/v1.4.2 release/v1.4.3
│ │
tag: v1.4.2 tag: v1.4.3
│ │
hotfix ◄─────┘ hotfix ◄─────┘
│ │
└── cherry-pick back ──────────┘
```
### Key rules
1. **main is always deployable**: never leave half-done work on `main`.
Unfinished work lives on a feature branch.
2. **Features start from main and merge back**: `git checkout -b feature/xxx main`,
then `git merge --no-ff feature/xxx` (or squash) when done.
3. **Release = cut a release branch from main + tag**:
```bash
git checkout -b release/v1.4.2 main
git tag -a v1.4.2 -m "ModelRouter v1.4.2"
git push origin release/v1.4.2 v1.4.2
```
Build installers and upload the GitCode Release from this tag so the
published state is exactly reproducible.
4. **Only version-specific hotfixes go into a release branch during its
window**: new features always go to `main` for the next version, never into
an already-released branch (unless you deliberately ship a minor revision).
5. **Hotfixes MUST flow back to main**:
```bash
git checkout release/v1.4.2 # fix in the release branch
git commit -m "fix: ..."
git checkout main
git cherry-pick <hotfix-commit> # and into main
```
Most important rule: **main only moves forward and never loses fixes**. If a
hotfix never reaches main, the next release ships with the old bug.
### End of a version lifecycle
When the next version ships, the previous release branch retires:
- **Default: delete the remote release branch**
(`git push origin :release/v1.4.2`). All hotfixes were already
cherry-picked into main, so main contains everything; no merge needed.
- **Long-term maintenance** (e.g. an enterprise client pinned to an old
version): keep the branch, accept only security fixes, keep the
commit-then-cherry-pick loop.
## Explicit non-goals
- **Never rebase main**: main's history stays append-only; anyone pulling gets
directly usable history.
- **Release branches are not merged back wholesale**: hotfixes were already
cherry-picked; a whole-branch merge only creates conflicts with no benefit.
- **No "quick edit" directly on main**, even single-person: at minimum use
`feature/xxx → merge main` so history keeps its why-boundaries.
## Single vs multi person
- **Single** (this repo today): skip PRs — `git checkout -b feature/xxx` →
finish → merge back to main. Release branch + tag flow unchanged.
- **Multi / open source**: feature branches go through PR + Code Review,
maintainers merge; resolve conflicts on the feature branch, never on main.
## Commit message conventions
- `feat(...)`: new feature
- `fix(...)`: bug fix
- `chore(...)`: build / version / deps / non-code changes
- `docs(...)`: documentation
- `refactor(...)`: refactoring
- Scope in parens, e.g. `fix(adapters)`, `feat(webui)`, `chore(version)`
- One-line summary; expand with problem/root-cause/fix/verification when needed
## Release checklist (companion)
```bash
# 1. main is ready
git checkout main && git pull
# 2. cut release branch + tag
git checkout -b release/vX.Y.Z main
git tag -a vX.Y.Z -m "ModelRouter vX.Y.Z"
# 3. push branch + tag
git push origin release/vX.Y.Z
git push origin vX.Y.Z
# 4. build installers (core + GUI platforms)
make core-dist VERSION=X.Y.Z
# 5. create GitCode Release + upload
# https://gitcode.com/JianFeeeee/ModelRouter/releases
# upload MUST use PUT --http1.1, otherwise the OBS callback fails and the
# file never lands in the release; replacing artifacts = delete the tag
# (release disappears with it) then recreate.
```
## Why this model (background)
Before 2026-08 everything went straight to `main` plus a tag, which caused two
pain points:
1. After 1.4.2 shipped, a scheduler-scoring fix had to be added to the same
version — the only option was deleting the remote tag to clear the release
and re-uploading all installers. The published state was detached from code
history; "what exactly shipped" could not be traced.
2. No feature branches meant two independent efforts could not proceed in
parallel without colliding.
With release branches: the published state = `release/vX.Y.Z` branch +
`vX.Y.Z` tag, exactly reproducible; hotfixes have a clear landing spot; main
stays "latest + all fixes + deployable".
> 中文版见 [docs/git-workflow.md](git-workflow.md)。

128
docs/git-workflow.md Normal file
View File

@ -0,0 +1,128 @@
# Git 分支工作流(Branching Workflow)
ModelRouter 采用 **GitHub Flow + 发布分支** 模型:`main` 是唯一长命分支且永远可部署,
新工作从它开出特性分支;每个版本从 `main` 分出独立的发布分支并打 tag;hotfix 提交到
发布分支且必须回流 `main`。
目的:让「已发版的版本在生命周期内可被单独修补」成为一等操作,同时 `main` 始终包含
全部修复、永远可部署。
## 分支角色
| 分支 | 命名 | 生命周期 | 用途 |
|---|---|---|---|
| 主分支 | `main` | 永久 | 唯一长命分支。永远可部署。积攒下一个版本的功能。 |
| 特性分支 | `feature/<描述>` | 短命(开发→合并即删) | 新特性 / 一般 bug 修复。从 `main` 开出,完成后合回 `main`。 |
| 发布分支 | `release/vX.Y.Z` | 一个版本周期 | 从 `main` 分出,打 tag 发布。该版本生命周期内的 hotfix 都提交在此分支。 |
## 变更流向(重要)
```
feature/foo ───────┐
▼
main ──────────────── 下一个版本 ──────►
│ │
│ 切出 │ 切出
▼ ▼
release/v1.4.2 release/v1.4.3
│ │
tag: v1.4.2 tag: v1.4.3
│ │
hotfix ◄─────┘ hotfix ◄─────┘
│ │
└── cherry-pick 回 main ───────┘
```
### 关键规则
1. **main 永远可部署**:不在 main 上留半成品。任何未完成的工作必须在特性分支上。
2. **特性从 main 开,完成后合回 main**:`git checkout -b feature/xxx main`,
开发完 `git merge --no-ff feature/xxx` 或 squash 合回。
3. **发布 = 从 main 切 release 分支 + 打 tag**:
```bash
git checkout -b release/v1.4.2 main
git tag -a v1.4.2 -m "ModelRouter v1.4.2"
git push origin release/v1.4.2 v1.4.2
```
构建安装包、上传 GitCode Release 都基于这个 tag,保证可精确回溯发布态。
4. **版本生命周期内只收该版本的 hotfix**:新特性一律并入 `main` 等下一个版本,
绝不塞进已发布的 release 分支(除非主动选择在该版本内发次要版)。
5. **hotfix 必须回流 main**:
```bash
git checkout release/v1.4.2 # 在发布分支提交修复
git commit -m "fix: ..."
git checkout main
git cherry-pick <hotfix-commit> # 回主分支
```
这是本模型最重要的一条:**main 只前进、不丢修复**。如果 hotfix 不回 main,
下一个版本就会带着旧 bug 发布。
### 版本生命周期结束
下一个版本发布时,上一个 release 分支退役:
- **默认:直接删除远端 release 分支**(`git push origin :release/v1.4.2`)。
因为 hotfix 都已逐个 cherry-pick 回 main,main 已包含全部修复,无需再合并。
- **如需要长期维护旧版**(例如企业大客户卡在旧版本):保留分支,仅 stopship 接受
该版本的安全修复,继续走「提交 + cherry-pick 回 main」循环。
## 明确不做的事
- **不 rebase main**:`main` 的历史保持追加式,任何人拉取后 `git pull` 都得到直接可用的历史。
- **发布分支不整体 merge 回 main**:hotfix 已逐个 cherry-pick,整体合并只会制造冲突且无收益。
- **不在 main 直接写"临时改一下"**:即使是单人项目,也至少走
`feature/xxx → merge main` 的形式,让历史保留"为什么改"的边界。
## 单人 vs 多人
- **单人**(本项目当前):特性分支可省去 PR,`git checkout -b feature/xxx` →
完成 → 合并回 main。发布分支与 tag 流程完全一致。
- **多人 / 开源协作**:特性分支走 PR(pull request)+ Code Review,
由维护者 merge;冲突在特性分支上解决,不在 main 上解决。
## 提交信息规范(与分支配套)
- `feat(...)`: 新功能
- `fix(...)`: 修复
- `chore(...)`: 构建 / 版本号 / 依赖 / 文档无关改动
- `docs(...)`: 文档
- `refactor(...)`: 重构
- 括号内为影响域,如 `fix(adapters)`、`feat(webui)`、`chore(version)`
- 一句话总结,必要时正文展开「问题 / 根因 / 修复 / 验证」
## 发布操作速查(配套)
每次发布严格按此顺序:
```bash
# 1. 确认 main 就绪
git checkout main && git pull
# 2. 切发布分支并打 tag
git checkout -b release/vX.Y.Z main
git tag -a vX.Y.Z -m "ModelRouter vX.Y.Z"
# 3. 推送分支 + tag
git push origin release/vX.Y.Z
git push origin vX.Y.Z
# 4. 构建安装包(核心包 + GUI 各平台)
make core-dist VERSION=X.Y.Z
# 5. 创建 GitCode Release + 上传安装包
# 见 https://gitcode.com/JianFeeeee/ModelRouter/releases
# 上传注意:必须 PUT --http1.1,否则 OBS 回调失败文件不进列表;
# 替换旧版附件 = 删 tag(release 随 tag 一起消失)后重建。
```
## 为什么是这套(背景)
2026-08 之前本项目直接在所有改动提交到 `main` + 打 tag,出现两个痛点:
1. 1.4.2 已发版后又追了一个调度打分修复,只能删远端 tag 清空 release 再重传安装包
—— 版本发布态与代码历史脱节,无法精确回溯"当时发的是什么"。
2. 没有特性分支,无法并行开发互不干扰的两件事。
引入发布分支后:版本发布态 = `release/vX.Y.Z` 分支 + `vX.Y.Z` tag,可精确回溯;
hotfix 有明确落点;main 保持"最新 + 全部修复 + 可部署"。
> 英文版见 [docs/git-workflow-en.md](git-workflow-en.md)。

View File

@ -27,6 +27,11 @@ type Config struct {
Auto []ModelScope `yaml:"auto,omitempty"` // AUTO 调度链规则(WebUI 优先级页编辑,chat)
AutoImage []ModelScope `yaml:"auto_image,omitempty"` // AUTO 生图调度链规则(WebUI 优先级页·生图)
Keys []GWKey `yaml:"keys,omitempty"` // 网关密钥(WebUI 密钥页管理)
// box seals credentials (sources' api_key/headers, keys' key) at rest.
// In-memory values are always plaintext; only the bytes on disk are sealed.
// Wired by AttachSecretBox — Load leaves it nil so `-check` and tests stay
// filesystem-free.
box *SecretBox
}
// Defaults applied to any source (YAML or runtime) that leaves a field unset.
@ -47,15 +52,19 @@ type Model struct {
// Source describes a single upstream LLM provider.
type Source struct {
Name string `yaml:"name" json:"name"`
BaseURL string `yaml:"base_url" json:"base_url"`
APIKey string `yaml:"api_key" json:"api_key"`
APIKeyEnv string `yaml:"api_key_env,omitempty" json:"-"` // reference to an env var holding the key (overrides api_key)
Adapter string `yaml:"adapter" json:"adapter"`
Endpoint string `yaml:"endpoint" json:"endpoint,omitempty"` // chat endpoint override
ImageEndpoint string `yaml:"image_endpoint" json:"image_endpoint,omitempty"` // image endpoint override
Models []Model `yaml:"models" json:"models"`
Headers map[string]string `yaml:"headers" json:"headers,omitempty"`
Name string `yaml:"name" json:"name"`
BaseURL string `yaml:"base_url" json:"base_url"`
APIKey string `yaml:"api_key" json:"api_key"`
APIKeyEnv string `yaml:"api_key_env,omitempty" json:"-"` // reference to an env var holding the key (overrides api_key)
Adapter string `yaml:"adapter" json:"adapter"`
Endpoint string `yaml:"endpoint" json:"endpoint,omitempty"` // chat endpoint override
ImageEndpoint string `yaml:"image_endpoint" json:"image_endpoint,omitempty"` // image endpoint override
Models []Model `yaml:"models" json:"models"`
Headers map[string]string `yaml:"headers" json:"headers,omitempty"`
// ProxyURL routes this source's HTTP(S) traffic through an explicit proxy
// (e.g. http://127.0.0.1:7890). Empty = direct connection. Needed for
// upstreams behind geo/IP blocks that only respond through a proxy.
ProxyURL string `yaml:"proxy_url" json:"proxy_url,omitempty"`
Meta map[string]interface{} `yaml:"meta" json:"meta,omitempty"`
Temperature float64 `yaml:"temperature" json:"temperature,omitempty"`
MaxTokens int `yaml:"max_tokens" json:"max_tokens,omitempty"`
@ -188,10 +197,15 @@ func RemoveSourceFromYAML(path, name string) error {
// UpsertSourceInYAML adds or updates a source entry in the YAML config file.
// Uses yaml.Node to preserve the rest of the file's comments and formatting.
func UpsertSourceInYAML(path, name string, src Source) error {
// When box is non-nil the source's credentials are sealed before writing, so a
// newly added source never lands in the file as plaintext.
func UpsertSourceInYAML(path, name string, src Source, box *SecretBox) error {
if path == "" {
return fmt.Errorf("config path is empty")
}
if err := sealSource(&src, box); err != nil {
return err
}
data, err := os.ReadFile(path)
if err != nil {
return err
@ -249,11 +263,17 @@ func UpsertSourceInYAML(path, name string, src Source) error {
// Save writes the current config back to the YAML file (preserving comments
// via yaml.Node round-trip when possible, or full marshaling as fallback).
// Credentials are sealed on the way out and the in-memory copy is restored to
// plaintext afterwards, so callers keep working with usable values.
func (c *Config) Save() error {
if c.Path == "" {
return fmt.Errorf("config path is empty")
}
if err := c.sealInPlace(c.box); err != nil {
return err
}
out, err := yaml.Marshal(c)
c.unsealAfterWrite(c.box)
if err != nil {
return fmt.Errorf("marshal config: %w", err)
}
@ -328,13 +348,15 @@ type RuntimeConfig struct {
// name and api_key) so the WebUI can spin up multiple key-bearing sources
// from one shared template.
type SourceTemplate struct {
Name string `json:"name"`
BaseURL string `json:"base_url"`
Adapter string `json:"adapter"`
Endpoint string `json:"endpoint,omitempty"`
ImageEndpoint string `json:"image_endpoint,omitempty"`
Models []Model `json:"models"`
Headers map[string]string `json:"headers,omitempty"`
Name string `json:"name"`
BaseURL string `json:"base_url"`
Adapter string `json:"adapter"`
Endpoint string `json:"endpoint,omitempty"`
ImageEndpoint string `json:"image_endpoint,omitempty"`
Models []Model `json:"models"`
Headers map[string]string `json:"headers,omitempty"`
// ProxyURL routes this source's HTTP(S) traffic through an explicit proxy.
ProxyURL string `json:"proxy_url,omitempty"`
Meta map[string]interface{} `json:"meta,omitempty"`
Temperature float64 `json:"temperature,omitempty"`
MaxTokens int `json:"max_tokens,omitempty"`

View File

@ -0,0 +1,272 @@
package config
// Secret handling for config.yaml.
//
// config.yaml is 0644 world-readable by design (ops need to inspect it), so any
// credential in it must not sit there in plaintext. Sources' api_key / headers
// and gateway keys are therefore sealed at rest with the same SecretBox used by
// the runtime store, and unsealed in memory at load time.
//
// The invariant that makes this safe: **in-memory values are always plaintext**,
// and the enc:v1: prefix is what marks a file value as sealed. Read paths that
// predate this (core.resolveSourceKey) already unseal, so only the write side and
// the load-time normalize step are new.
import (
"fmt"
"log"
"os"
"strings"
)
// sealSource seals one source's credentials in place (used by the YAML upsert
// path, which is a package function and therefore has no Config to borrow a box
// from).
func sealSource(s *Source, box *SecretBox) error {
if box == nil {
return nil
}
if s.APIKey != "" && !strings.HasPrefix(s.APIKey, encPrefix) {
v, err := box.Encrypt(s.APIKey)
if err != nil {
return err
}
s.APIKey = v
}
for k, v := range s.Headers {
if v == "" || strings.HasPrefix(v, encPrefix) {
continue
}
e, err := box.Encrypt(v)
if err != nil {
return err
}
s.Headers[k] = e
}
return nil
}
// normalizeSecrets unseals every credential in the freshly parsed config so the
// rest of the program only ever sees plaintext. A value without the enc:v1:
// prefix is left untouched, which keeps hand-written plaintext configs working
// (and is what a pre-encryption config file looks like).
//
// A value that carries the prefix but fails to decrypt is a hard error, not
// something to paper over: returning the ciphertext (MustDecrypt's behavior)
// would let the next Save re-seal it and turn one bad value into permanent,
// compounding corruption. Losing the master key must be loud.
func (c *Config) normalizeSecrets(box *SecretBox) error {
if box == nil {
return nil
}
for i := range c.Sources {
s := &c.Sources[i]
if strings.HasPrefix(s.APIKey, encPrefix) {
v, err := box.Decrypt(s.APIKey)
if err != nil {
return fmt.Errorf("source %q api_key: %w", s.Name, err)
}
s.APIKey = v
}
for k, v := range s.Headers {
if strings.HasPrefix(v, encPrefix) {
d, err := box.Decrypt(v)
if err != nil {
return fmt.Errorf("source %q header %q: %w", s.Name, k, err)
}
s.Headers[k] = d
}
}
}
for i := range c.Keys {
if strings.HasPrefix(c.Keys[i].Key, encPrefix) {
v, err := box.Decrypt(c.Keys[i].Key)
if err != nil {
return fmt.Errorf("gateway key %q: %w", c.Keys[i].Name, err)
}
c.Keys[i].Key = v
}
}
return nil
}
// sealInPlace replaces plaintext credentials with ciphertext for writing. It is
// deliberately a separate step from Marshal: callers that need the plaintext
// (auth comparisons, log output, returning a key to the operator who just
// created it) must not be handed a sealed config by accident.
func (c *Config) sealInPlace(box *SecretBox) error {
if box == nil {
return nil
}
for i := range c.Sources {
s := &c.Sources[i]
if s.APIKey != "" && !strings.HasPrefix(s.APIKey, encPrefix) {
v, err := box.Encrypt(s.APIKey)
if err != nil {
return err
}
s.APIKey = v
}
if len(s.Headers) > 0 {
sealed := make(map[string]string, len(s.Headers))
for k, v := range s.Headers {
if v == "" || strings.HasPrefix(v, encPrefix) {
sealed[k] = v
continue
}
e, err := box.Encrypt(v)
if err != nil {
return err
}
sealed[k] = e
}
s.Headers = sealed
}
}
for i := range c.Keys {
k := &c.Keys[i]
if k.Key != "" && !strings.HasPrefix(k.Key, encPrefix) {
v, err := box.Encrypt(k.Key)
if err != nil {
return err
}
k.Key = v
}
}
return nil
}
// unsealAfterWrite restores plaintext after a sealed marshal so the live process
// keeps working on plaintext values (mirrors Store.persistLocked's dance).
func (c *Config) unsealAfterWrite(box *SecretBox) {
// Save just encrypted every value it can see, so a failure here is
// impossible; ignore the error rather than panic in a write path.
_ = c.normalizeSecrets(box)
}
// hasPlaintextSecrets reports whether any credential in the config is still in
// the clear. Used to decide whether a startup migration write is needed, and to
// warn (without leaking values) when no master key is available.
func (c *Config) hasPlaintextSecrets() bool {
for _, s := range c.Sources {
if s.APIKey != "" && !strings.HasPrefix(s.APIKey, encPrefix) {
return true
}
for _, v := range s.Headers {
if v != "" && !strings.HasPrefix(v, encPrefix) {
return true
}
}
}
for _, k := range c.Keys {
if k.Key != "" && !strings.HasPrefix(k.Key, encPrefix) {
return true
}
}
return false
}
// countPlaintextSecrets returns how many credentials are still in the clear, for
// an operator-facing migration log line that must not print the values.
func (c *Config) countPlaintextSecrets() int {
n := 0
for _, s := range c.Sources {
if s.APIKey != "" && !strings.HasPrefix(s.APIKey, encPrefix) {
n++
}
for _, v := range s.Headers {
if v != "" && !strings.HasPrefix(v, encPrefix) {
n++
}
}
}
for _, k := range c.Keys {
if k.Key != "" && !strings.HasPrefix(k.Key, encPrefix) {
n++
}
}
return n
}
// NormalizeSecretsForRun unseals the loaded config and then seals it back on
// disk if anything was still in the clear. Order matters: Load() read the file
// with ciphertext still in place, so the unseal has to happen before the
// registry (and any Save the startup path performs) sees the values.
func (c *Config) NormalizeSecretsForRun(box *SecretBox) error {
c.AttachSecretBox(box)
if err := c.normalizeSecrets(box); err != nil {
return err
}
return c.migratePlaintextSecrets()
}
// AttachSecretBox wires the encryption box into the config so Save can seal
// credentials. Kept as an explicit call (rather than a constructor argument) so
// config.Load stays usable in contexts that have no filesystem secrets (tests,
// `-check`).
func (c *Config) AttachSecretBox(box *SecretBox) { c.box = box }
// SecretBox returns the wired encryption box, or nil when none is attached.
func (c *Config) SecretBox() *SecretBox { return c.box }
// migratePlaintextSecrets seals any credential still in the clear and writes the
// file once. It is idempotent: a config that is already sealed (or has no
// secrets) is left alone and nothing is written.
func (c *Config) migratePlaintextSecrets() error {
if c.box == nil || c.Path == "" {
return nil
}
if !c.hasPlaintextSecrets() {
return nil
}
n := c.countPlaintextSecrets()
if err := c.Save(); err != nil {
return err
}
log.Printf("[config] sealed %d plaintext credential(s) in %s", n, c.Path)
return nil
}
// PrintSecrets writes the config's credentials to stdout in the clear and
// returns an error only when the file cannot be read or the master key does not
// match. It is the operator counterpart to sealing-at-rest: with keys stored as
// ciphertext, "which key is this source using" must still be answerable.
//
// It deliberately takes a path rather than a *Config so the caller cannot
// accidentally hand it a config that has already been unsealed in memory, and it
// never writes anything.
func PrintSecrets(path string) error {
cfg, err := Load(path)
if err != nil {
return err
}
box, err := NewSecretBox(cfg.RuntimeFile)
if err != nil {
return fmt.Errorf("%w (is master.key present and intact?)", err)
}
if err := cfg.normalizeSecrets(box); err != nil {
return err
}
w := os.Stdout
fmt.Fprintf(w, "# %s — %d source(s), %d gateway key(s)\n", path, len(cfg.Sources), len(cfg.Keys))
for _, s := range cfg.Sources {
fmt.Fprintf(w, "source %-16s api_key=%s\n", s.Name, orNone(s.APIKey))
for k, v := range s.Headers {
if v != "" {
fmt.Fprintf(w, "source %-16s header[%s]=%s\n", s.Name, k, v)
}
}
}
for _, k := range cfg.Keys {
fmt.Fprintf(w, "key %-16s role=%-5s %s\n", k.Name, k.Role, k.Key)
}
fmt.Fprintf(w, "gateway_keys (legacy): %s\n", strings.Join(cfg.GatewayKeys, " "))
return nil
}
func orNone(s string) string {
if s == "" {
return "(unset)"
}
return s
}

View File

@ -0,0 +1,283 @@
package config
import (
"os"
"path/filepath"
"strings"
"testing"
)
// writeMasterKey plants a deterministic master.key next to the given runtime
// file so tests exercise the real box instead of the env-var shortcut.
func writeMasterKey(t *testing.T, runtimeFile string) *SecretBox {
t.Helper()
if err := os.WriteFile(filepath.Join(filepath.Dir(runtimeFile), "master.key"),
[]byte(strings.Repeat("ab", 32)), 0600); err != nil {
t.Fatal(err)
}
box, err := NewSecretBox(runtimeFile)
if err != nil {
t.Fatalf("NewSecretBox: %v", err)
}
return box
}
func TestSaveSealsCredentialsAndLoadUnseals(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "config.yaml")
runtime := filepath.Join(dir, "runtime.json")
box := writeMasterKey(t, runtime)
cfg := &Config{
Path: path,
Listen: "127.0.0.1:0",
Sources: []Source{{
Name: "up",
BaseURL: "http://up/v1",
APIKey: "sk-plaintext-secret",
Adapter: "openai",
Headers: map[string]string{"X-Extra": "header-secret"},
Models: []Model{{ID: "m", Kind: "chat"}},
}},
Keys: []GWKey{{Key: "sk-gw-plain", Role: "admin", Name: "admin"}},
}
cfg.AttachSecretBox(box)
if err := cfg.Save(); err != nil {
t.Fatalf("Save: %v", err)
}
raw, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
text := string(raw)
if strings.Contains(text, "sk-plaintext-secret") {
t.Error("source api_key written in plaintext")
}
if strings.Contains(text, "header-secret") {
t.Error("source header written in plaintext")
}
if strings.Contains(text, "sk-gw-plain") {
t.Error("gateway key written in plaintext")
}
if n := strings.Count(text, encPrefix); n != 3 {
t.Errorf("expected 3 sealed values, found %d", n)
}
// The live in-memory config must still hold plaintext after Save.
if cfg.Sources[0].APIKey != "sk-plaintext-secret" {
t.Errorf("in-memory api_key mutated by Save: %q", cfg.Sources[0].APIKey)
}
if cfg.Keys[0].Key != "sk-gw-plain" {
t.Errorf("in-memory gateway key mutated by Save: %q", cfg.Keys[0].Key)
}
// A fresh load must hand back plaintext again.
loaded, err := Load(path)
if err != nil {
t.Fatalf("Load: %v", err)
}
loaded.AttachSecretBox(box)
loaded.normalizeSecrets(box)
if loaded.Sources[0].APIKey != "sk-plaintext-secret" {
t.Errorf("loaded api_key = %q, want plaintext", loaded.Sources[0].APIKey)
}
if loaded.Sources[0].Headers["X-Extra"] != "header-secret" {
t.Errorf("loaded header = %q, want plaintext", loaded.Sources[0].Headers["X-Extra"])
}
if loaded.Keys[0].Key != "sk-gw-plain" {
t.Errorf("loaded gateway key = %q, want plaintext", loaded.Keys[0].Key)
}
}
func TestPlaintextConfigStillLoads(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "config.yaml")
// A hand-written, pre-encryption config: no enc:v1: anywhere.
body := `listen: 127.0.0.1:0
sources:
- name: legacy
base_url: http://legacy/v1
api_key: sk-written-by-hand
adapter: openai
models:
- id: m
kind: chat
`
if err := os.WriteFile(path, []byte(body), 0644); err != nil {
t.Fatal(err)
}
cfg, err := Load(path)
if err != nil {
t.Fatalf("Load: %v", err)
}
box := writeMasterKey(t, filepath.Join(dir, "runtime.json"))
cfg.AttachSecretBox(box)
cfg.normalizeSecrets(box)
if cfg.Sources[0].APIKey != "sk-written-by-hand" {
t.Errorf("plaintext config value changed: %q", cfg.Sources[0].APIKey)
}
}
func TestMigratePlaintextSecretsIsIdempotent(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "config.yaml")
runtime := filepath.Join(dir, "runtime.json")
box := writeMasterKey(t, runtime)
cfg := &Config{
Path: path,
Listen: "127.0.0.1:0",
Sources: []Source{{Name: "up", BaseURL: "http://up/v1", APIKey: "sk-clear", Adapter: "openai", Models: []Model{{ID: "m"}}}},
}
cfg.AttachSecretBox(box)
if err := cfg.migratePlaintextSecrets(); err != nil {
t.Fatalf("first migrate: %v", err)
}
first, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(first), encPrefix) {
t.Fatal("migration did not seal the value")
}
// In-memory must be plaintext so the running process keeps working.
if cfg.Sources[0].APIKey != "sk-clear" {
t.Errorf("in-memory api_key = %q, want plaintext", cfg.Sources[0].APIKey)
}
// Second run: already sealed => no write.
before, _ := os.Stat(path)
if err := cfg.migratePlaintextSecrets(); err != nil {
t.Fatalf("second migrate: %v", err)
}
after, _ := os.Stat(path)
if before.ModTime() != after.ModTime() {
t.Error("second migrate rewrote an already-sealed config")
}
}
func TestSealedValueDoesNotDoubleEncrypt(t *testing.T) {
dir := t.TempDir()
box := writeMasterKey(t, filepath.Join(dir, "runtime.json"))
cfg := &Config{Path: filepath.Join(dir, "config.yaml"), Sources: []Source{{Name: "a", BaseURL: "http://a"}}}
cfg.AttachSecretBox(box)
once, err := box.Encrypt("sk-x")
if err != nil {
t.Fatal(err)
}
cfg.Sources[0].APIKey = once
if err := cfg.Save(); err != nil {
t.Fatal(err)
}
got, err := box.Decrypt(cfg.Sources[0].APIKey)
if err != nil {
t.Fatalf("value became undecryptable: %v", err)
}
if got != "sk-x" {
t.Errorf("round trip = %q, want sk-x", got)
}
}
func TestUpsertSourceInYAMLSealsCredentials(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "config.yaml")
box := writeMasterKey(t, filepath.Join(dir, "runtime.json"))
if err := os.WriteFile(path, []byte("listen: 127.0.0.1:0\nsources: []\n"), 0644); err != nil {
t.Fatal(err)
}
src := Source{Name: "new", BaseURL: "http://new/v1", APIKey: "sk-fresh", Adapter: "openai", Models: []Model{{ID: "m"}}}
if err := UpsertSourceInYAML(path, src.Name, src, box); err != nil {
t.Fatalf("UpsertSourceInYAML: %v", err)
}
raw, _ := os.ReadFile(path)
if strings.Contains(string(raw), "sk-fresh") {
t.Error("newly added source stored its api_key in plaintext")
}
if !strings.Contains(string(raw), encPrefix) {
t.Error("newly added source was not sealed")
}
}
func TestCountPlaintextSecrets(t *testing.T) {
cfg := &Config{
Sources: []Source{
{Name: "a", APIKey: encPrefix + "abc", Headers: map[string]string{"H": encPrefix + "x"}},
{Name: "b", APIKey: "sk-in-clear", Headers: map[string]string{"H2": "clear-too"}},
},
Keys: []GWKey{{Key: "sk-gw-clear"}},
}
if got := cfg.countPlaintextSecrets(); got != 3 {
t.Errorf("countPlaintextSecrets = %d, want 3", got)
}
if !cfg.hasPlaintextSecrets() {
t.Error("hasPlaintextSecrets = false, want true")
}
cfg.Sources[1].APIKey = encPrefix + "y"
cfg.Sources[1].Headers["H2"] = encPrefix + "z"
cfg.Keys[0].Key = encPrefix + "k"
if cfg.hasPlaintextSecrets() {
t.Error("hasPlaintextSecrets = true after sealing everything")
}
}
// TestNormalizeSecretsFailsLoudlyOnWrongMasterKey is the negative case that
// guards the migration: with a wrong key, a sealed value must NOT be handed
// back as ciphertext for a later re-seal (that compounds corruption and turns
// one lost key file into permanently unusable config).
func TestNormalizeSecretsFailsLoudlyOnWrongMasterKey(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "config.yaml")
runtime := filepath.Join(dir, "runtime.json")
good := writeMasterKey(t, runtime)
cfg := &Config{
Path: path,
Listen: "127.0.0.1:0",
Sources: []Source{{Name: "up", BaseURL: "http://up/v1", APIKey: "sk-secret", Adapter: "openai", Models: []Model{{ID: "m"}}}},
}
cfg.AttachSecretBox(good)
if err := cfg.Save(); err != nil {
t.Fatal(err)
}
sealedOnce, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
// A different master key must be rejected, loudly. NewSecretBox derives the
// key path from the RUNTIME file's directory, so the wrong key has to live
// in a different directory — otherwise it would read the good master.key.
otherDir := t.TempDir()
if err := os.WriteFile(filepath.Join(otherDir, "master.key"),
[]byte(strings.Repeat("cd", 32)), 0600); err != nil {
t.Fatal(err)
}
badBox, err := NewSecretBox(filepath.Join(otherDir, "runtime.json"))
if err != nil {
t.Fatal(err)
}
loaded, err := Load(path)
if err != nil {
t.Fatal(err)
}
loaded.AttachSecretBox(badBox)
if err := loaded.NormalizeSecretsForRun(badBox); err == nil {
t.Fatal("expected an error with the wrong master key, got nil")
}
// Crucially: the file must be untouched — no second seal on top.
after, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if string(after) != string(sealedOnce) {
t.Error("config file was rewritten despite the decrypt failure")
}
if n := strings.Count(string(after), encPrefix); n != strings.Count(string(sealedOnce), encPrefix) {
t.Errorf("sealed count changed %d -> %d (double encryption)", strings.Count(string(sealedOnce), encPrefix), n)
}
}

View File

@ -7,6 +7,7 @@ import (
"crypto/rand"
"encoding/hex"
"fmt"
"log"
"os"
"path/filepath"
"sort"
@ -53,6 +54,10 @@ func NewFromConfig(cfg *config.Config) (*Core, error) {
return nil, fmt.Errorf("lua vm: %w", err)
}
c.store = config.NewStore(cfg.RuntimeFile)
// Share one box between the runtime store and config.yaml so a single
// master.key seals both files. config.Load left the config holding
// ciphertext (if it was sealed); unseal it now that the box exists.
cfg.AttachSecretBox(c.store.SecretBox())
if err := c.store.Load(); err != nil {
return nil, fmt.Errorf("runtime store: %w", err)
}
@ -74,6 +79,12 @@ func NewFromConfig(cfg *config.Config) (*Core, error) {
if err := c.seedPresetTemplates(); err != nil {
return nil, fmt.Errorf("seed preset templates: %w", err)
}
// Seal any credential still in the clear in config.yaml. Idempotent: an
// already-sealed config is not rewritten, so a normal restart writes
// nothing. This is the only place that rewrites the file on startup.
if err := cfg.NormalizeSecretsForRun(c.store.SecretBox()); err != nil {
return nil, fmt.Errorf("normalize secrets: %w", err)
}
return c, nil
}
@ -439,6 +450,11 @@ func (c *Core) mergedSources() []config.Source {
}
// resolveSourceKey applies api_key_env and decrypts enc:v1: ciphertext.
// Config values are already unsealed at startup (Config.NormalizeSecretsForRun),
// so the decrypt branch is the belt-and-braces path for a source that arrived
// already sealed through another route. A failure there leaves the ciphertext in
// place, which makes the upstream reject the key loudly rather than sending an
// empty Authorization header that might look like a config-less source.
func (c *Core) resolveSourceKey(s config.Source) config.Source {
if s.APIKeyEnv != "" {
if v := os.Getenv(s.APIKeyEnv); v != "" {
@ -447,7 +463,11 @@ func (c *Core) resolveSourceKey(s config.Source) config.Source {
return s
}
if box := c.store.SecretBox(); box != nil && strings.HasPrefix(s.APIKey, "enc:v1:") {
s.APIKey = box.MustDecrypt(s.APIKey)
if v, err := box.Decrypt(s.APIKey); err == nil {
s.APIKey = v
} else {
log.Printf("[core] source %s: api_key decrypt failed: %v", s.Name, err)
}
}
return s
}
@ -655,7 +675,7 @@ func (c *Core) AddSource(src config.Source) error {
if err := normalizeSource(&src); err != nil {
return err
}
if err := config.UpsertSourceInYAML(c.cfg.Path, src.Name, src); err != nil {
if err := config.UpsertSourceInYAML(c.cfg.Path, src.Name, src, c.cfg.SecretBox()); err != nil {
return err
}
// Update in-memory Sources so mergedSources() finds the entry.

415
internal/gateway/apiv1.go Normal file
View File

@ -0,0 +1,415 @@
package gateway
// Agent-facing management API.
//
// The Web UI has always driven the gateway through /api/*, so the management
// surface already exists. What it lacked was anything an agent could *rely* on:
// no way to discover the contract, no single call that answers "what is the
// current state", and per-endpoint response shapes that make scripting brittle.
//
// This file adds a versioned, self-describing facade under /api/v1 without
// touching the existing endpoints, so the Web UI keeps working unchanged while
// agents get a stable contract:
//
// GET /api/v1 — machine-readable index of every endpoint
// GET /api/v1/overview — one call: sources + auto chain + keys + health
// GET /api/v1/sources — sources, credentials masked
// GET /api/v1/sources/{name} — one source
// GET /api/v1/auto — scheduling chain + live slot states
// GET /api/v1/keys — key metadata (never the secret)
// GET /api/v1/models — every model id the gateway can route
// GET /api/v1/health — per-source health, no auth needed beyond the key
//
// Auth is the same gateway key as everywhere else (Authorization: Bearer, or
// ?api_key=, or the gw_key cookie). Read endpoints accept any role; writes
// still require admin, enforced by the same middleware the UI goes through.
import (
"errors"
"net/http"
"net/url"
"sort"
"strings"
"llmsproxy/internal/config"
)
// apiV1Routes dispatches /api/v1/*. Kept separate from routes() so the original
// switch stays readable and the versioned surface can grow on its own.
func (g *Gateway) apiV1Routes(w http.ResponseWriter, r *http.Request) {
path := strings.TrimPrefix(r.URL.Path, "/api/v1")
path = strings.Trim(path, "/")
switch {
case path == "":
g.apiV1Index(w, r)
case path == "overview":
g.apiV1Overview(w, r)
case path == "health":
g.apiV1Health(w, r)
case path == "models":
g.apiV1Models(w, r)
case path == "sources":
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"sources": maskSources(g.core.Sources())})
case strings.HasPrefix(path, "sources/"):
name, err := decodePathSegment(strings.TrimPrefix(path, "sources/"))
if err != nil {
writeError(w, http.StatusBadRequest, "invalid_request", err.Error())
return
}
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed",
"use PUT/DELETE on /api/sources/{name} to modify a source")
return
}
for _, s := range g.core.Sources() {
if s.Name == name {
// ?reveal=credentials is admin-only and is what the Web UI's
// edit dialog uses: a form that round-trips a source must be
// able to show the current key, otherwise saving an unrelated
// field would blank it. Everything else stays masked.
if r.URL.Query().Get("reveal") == "credentials" {
if reqRole(r.Context()) != "admin" {
writeError(w, http.StatusForbidden, "forbidden",
"admin role required to reveal credentials")
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"source": s})
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"source": maskSource(s)})
return
}
}
writeError(w, http.StatusNotFound, "not_found", "no such source: "+name)
case path == "auto":
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{
"rules": g.core.AutoRules(),
"image_rules": g.core.AutoImageRules(),
"states": g.core.AutoSlotStates(),
})
case path == "keys":
if reqRole(r.Context()) != "admin" {
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
return
}
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed",
"use POST/PUT/DELETE on /api/keys to manage keys")
return
}
keys := g.core.ListKeys()
out := make([]map[string]interface{}, 0, len(keys))
for _, k := range keys {
out = append(out, map[string]interface{}{
"name": k.Name,
"role": k.Role,
"models": k.Models,
"note": k.Note,
"created_at": k.CreatedAt,
"seed": k.Seed,
// The secret itself is never echoed. An operator that needs it
// already has it from creation time or from config.yaml.
"key_prefix": maskKey(k.Key),
})
}
writeJSON(w, http.StatusOK, map[string]interface{}{"keys": out})
default:
g.apiV1Index(w, r)
}
}
// apiV1Index is the discovery document: it names every endpoint with its
// method, auth requirement and purpose, so an agent does not have to scrape the
// HTML to find out what it can call.
func (g *Gateway) apiV1Index(w http.ResponseWriter, r *http.Request) {
type ep struct {
Method string `json:"method"`
Path string `json:"path"`
Auth string `json:"auth"`
Summary string `json:"summary"`
WriteEffect string `json:"writes,omitempty"`
}
index := map[string]interface{}{
"api_version": "v1",
"description": "llmsproxy management API. Authenticate with a gateway key: " +
"'Authorization: Bearer <key>', '?api_key=<key>', or the gw_key cookie. " +
"Read endpoints accept any role; mutations require an admin key.",
"discovery": []ep{
{Method: "GET", Path: "/api/v1", Auth: "any", Summary: "this document"},
{Method: "GET", Path: "/api/v1/overview", Auth: "any",
Summary: "current state in one call: sources, auto chain, key count, source health"},
{Method: "GET", Path: "/api/v1/health", Auth: "any",
Summary: "per-source health snapshot only"},
{Method: "GET", Path: "/api/v1/models", Auth: "any",
Summary: "every model id the gateway can route, grouped by owning source"},
{Method: "GET", Path: "/api/v1/sources", Auth: "any",
Summary: "all sources with credentials masked"},
{Method: "GET", Path: "/api/v1/sources/{name}", Auth: "any", Summary: "one source"},
{Method: "GET", Path: "/api/v1/auto", Auth: "any",
Summary: "AUTO scheduling chain and live per-slot state"},
{Method: "GET", Path: "/api/v1/keys", Auth: "admin",
Summary: "gateway key metadata; secrets are never returned"},
{Method: "GET", Path: "/api/sources", Auth: "admin", Summary: "raw source list (includes api_key)"},
{Method: "POST", Path: "/api/sources", Auth: "admin", Summary: "add or replace a source",
WriteEffect: "writes config.yaml (api_key sealed at rest)"},
{Method: "PUT", Path: "/api/sources/{name}", Auth: "admin", Summary: "update one source",
WriteEffect: "writes config.yaml"},
{Method: "DELETE", Path: "/api/sources/{name}", Auth: "admin", Summary: "delete a source",
WriteEffect: "writes config.yaml"},
{Method: "GET", Path: "/api/auto", Auth: "any", Summary: "scheduling chain (readable by any role)"},
{Method: "PUT", Path: "/api/auto", Auth: "admin", Summary: "replace the scheduling chain",
WriteEffect: "writes config.yaml (the AUTO chain lives here)"},
{Method: "GET", Path: "/api/keys", Auth: "admin", Summary: "gateway keys"},
{Method: "POST", Path: "/api/keys", Auth: "admin", Summary: "create a gateway key",
WriteEffect: "writes config.yaml"},
{Method: "DELETE", Path: "/api/keys/{name}", Auth: "admin", Summary: "delete a gateway key",
WriteEffect: "writes config.yaml"},
{Method: "GET", Path: "/api/status", Auth: "any", Summary: "per-source health detail"},
{Method: "GET", Path: "/api/stats", Auth: "any", Summary: "usage aggregates"},
{Method: "GET", Path: "/api/stats/records", Auth: "any", Summary: "paged request records"},
{Method: "GET", Path: "/api/adapters", Auth: "admin", Summary: "installed Lua adapters"},
{Method: "GET", Path: "/api/source_templates", Auth: "admin", Summary: "source templates"},
{Method: "POST", Path: "/v1/chat/completions", Auth: "any", Summary: "OpenAI-compatible inference"},
},
"conventions": map[string]interface{}{
"errors": "{ \"error\": { \"type\": <code>, \"message\": <text> } }",
"path_escape": "URL-encode source and key names; {name} is a single path segment",
"idempotency": "POST /api/sources and PUT /api/sources/{name} both upsert by name",
"config_truth": "all configuration lives in config.yaml; API writes are persisted immediately",
"credentials": "credentials are masked by default. GET /api/v1/sources/{name}?reveal=credentials " +
"returns them in the clear and is admin-only — the Web UI edit dialog uses it, because a form " +
"that round-trips a source must show the current key or saving another field would blank it.",
},
}
writeJSON(w, http.StatusOK, index)
}
// apiV1Overview is the "what is the current state" call an agent makes first.
// One round trip instead of five.
func (g *Gateway) apiV1Overview(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
return
}
sources := g.core.Sources()
names := make([]string, 0, len(sources))
modelCount := 0
for _, s := range sources {
names = append(names, s.Name)
modelCount += len(s.Models)
}
keys := g.core.ListKeys()
adminCount, userCount := 0, 0
for _, k := range keys {
if k.Role == "admin" {
adminCount++
} else {
userCount++
}
}
writeJSON(w, http.StatusOK, map[string]interface{}{
"api_version": "v1",
"summary": map[string]interface{}{
"source_count": len(sources),
"model_count": modelCount,
"gateway_key_count": len(keys),
"admin_key_count": adminCount,
"user_key_count": userCount,
"auto_slots": len(g.core.AutoRules()),
},
"sources": maskSources(sources),
"auto": g.core.AutoRules(),
"auto_image": g.core.AutoImageRules(),
"health": g.sourceHealthBrief(),
"caller": map[string]interface{}{
"role": reqRole(r.Context()),
"key": maskKey(reqKey(r.Context())),
},
})
}
func (g *Gateway) apiV1Health(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"sources": g.sourceHealthBrief()})
}
// apiV1Models lists routable model ids per source, which is what an agent needs
// to build a valid request — /v1/models flattens them and hides the owner.
func (g *Gateway) apiV1Models(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
return
}
bySource := map[string][]string{}
var all []string
for _, s := range g.core.Sources() {
for _, m := range s.Models {
bySource[s.Name] = append(bySource[s.Name], m.ID)
all = append(all, m.ID)
}
}
sort.Strings(all)
all = dedupeStrings(all)
writeJSON(w, http.StatusOK, map[string]interface{}{
"count": len(all),
"models": all,
"by_source": bySource,
"note": "request a model as \"<source>:<model>\" to pin one source, or the bare id to let the gateway choose",
})
}
// ---- helpers ----
// maskSources returns sources with credentials replaced by a presence marker.
// The UI still uses /api/sources (which returns real values, because its edit
// form round-trips them); this is the safe view for programmatic callers.
func maskSources(srcs []config.Source) []map[string]interface{} {
out := make([]map[string]interface{}, 0, len(srcs))
for _, s := range srcs {
out = append(out, maskSource(s))
}
return out
}
func maskSource(s config.Source) map[string]interface{} {
return map[string]interface{}{
"name": s.Name,
"base_url": s.BaseURL,
"adapter": s.Adapter,
"endpoint": s.Endpoint,
"image_endpoint": s.ImageEndpoint,
"api_key": maskKey(s.APIKey),
"api_key_set": s.APIKey != "",
"models": s.Models,
"headers": maskHeaders(s.Headers),
"proxy_url": s.ProxyURL,
"meta": s.Meta,
"temperature": s.Temperature,
"max_tokens": s.MaxTokens,
"max_concurrent": s.MaxConcurrent,
"rpm": s.RPM,
}
}
func maskKey(k string) string {
if k == "" {
return ""
}
if strings.HasPrefix(k, "enc:v1:") {
return "(sealed)"
}
if len(k) <= 10 {
return k[:2] + "…"
}
return k[:6] + "…" + k[len(k)-4:]
}
func maskHeaders(h map[string]string) map[string]string {
if h == nil {
return nil
}
out := make(map[string]string, len(h))
for k, v := range h {
out[k] = maskKey(v)
}
return out
}
// sourceHealthBrief reuses the same registry status the UI's /api/status shows,
// so the agent view and the UI view cannot drift apart. Recent-traffic counters
// come from the same stats window, because a source that is actually serving
// traffic must never look down just because a probe was rate-limited.
func (g *Gateway) sourceHealthBrief() []map[string]interface{} {
sts := g.core.Registry().Status()
recent := g.stats.SourceRecent(300)
avgs := g.stats.SourceAverages(300)
for i := range sts {
if v, ok := recent[sts[i].Name]; ok {
sts[i].RecentOK = v[0]
sts[i].RecentErr = v[1]
}
if a, ok := avgs[sts[i].Name]; ok {
sts[i].AvgFirstByteMs = a.AvgFirstByteMs
sts[i].AvgTokPerS = a.AvgTokPerS
}
}
out := make([]map[string]interface{}, 0, len(sts))
for _, s := range sts {
row := map[string]interface{}{
"name": s.Name,
"adapter": s.Adapter,
"healthy": s.Healthy,
"available": s.Available,
"live_available": s.LiveAvailable,
"model_count": len(s.Models),
}
if s.LastError != "" {
row["last_error"] = s.LastError
}
if s.LastChecked > 0 {
row["last_checked"] = s.LastChecked
}
if s.RecentOK > 0 || s.RecentErr > 0 {
row["recent_ok"] = s.RecentOK
row["recent_err"] = s.RecentErr
}
if s.AvgFirstByteMs > 0 {
row["avg_first_byte_ms"] = s.AvgFirstByteMs
}
if s.AvgTokPerS > 0 {
row["avg_tok_per_s"] = s.AvgTokPerS
}
if s.FailCount > 0 {
row["fail_count"] = s.FailCount
}
if s.Permanent {
row["permanent"] = true
}
out = append(out, row)
}
return out
}
// decodePathSegment URL-decodes one path segment and rejects an empty result,
// so a name containing a slash cannot be silently mis-resolved.
func decodePathSegment(seg string) (string, error) {
s, err := url.PathUnescape(seg)
if err != nil {
return "", err
}
if s == "" {
return "", errEmptySegment
}
return s, nil
}
func dedupeStrings(in []string) []string {
out := in[:0]
var last string
for i, s := range in {
if i == 0 || s != last {
out = append(out, s)
}
last = s
}
return out
}
// errEmptySegment marks a path like /api/v1/sources/ with no name after it.
var errEmptySegment = errors.New("empty path segment")

View File

@ -0,0 +1,299 @@
package gateway
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"llmsproxy/internal/config"
)
// v1Gateway builds a gateway with one source and two keys, and returns the
// secrets so a test can assert they never appear in a response.
func v1Gateway(t *testing.T) (*Gateway, string, string) {
t.Helper()
g := newTestGateway(t, config.Source{
Name: "up",
BaseURL: "http://up.example/v1",
APIKey: "sk-up-secret",
Adapter: "openai",
Models: []config.Model{{ID: "m1", Kind: "chat"}},
})
// The shared helper only seeds one admin key; add a user key through the
// core so role-gated endpoints have something to reject.
if _, err := g.core.CreateKey("agent", "user", nil, "test agent key"); err != nil {
t.Fatalf("CreateKey: %v", err)
}
rec := doReq(t, g, http.MethodGet, "/api/keys", "")
var doc struct {
Keys []config.GWKey `json:"keys"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &doc); err != nil {
t.Fatalf("decode keys: %v", err)
}
userKey := ""
for _, k := range doc.Keys {
if k.Role == "user" {
userKey = k.Key
}
}
if userKey == "" {
t.Fatal("no user key was created")
}
return g, "sk-test", userKey
}
func decodeJSON(t *testing.T, body string, v interface{}) {
t.Helper()
if err := json.Unmarshal([]byte(body), v); err != nil {
t.Fatalf("decode %s: %v", body, err)
}
}
func TestAPIV1IndexIsDiscoverable(t *testing.T) {
g, _, _ := v1Gateway(t)
rec := doReq(t, g, http.MethodGet, "/api/v1", "")
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200: %s", rec.Code, rec.Body.String())
}
var doc struct {
APIVersion string `json:"api_version"`
Description string `json:"description"`
Discovery []struct {
Method string `json:"method"`
Path string `json:"path"`
Auth string `json:"auth"`
Summary string `json:"summary"`
} `json:"discovery"`
Conventions map[string]interface{} `json:"conventions"`
}
decodeJSON(t, rec.Body.String(), &doc)
if doc.APIVersion != "v1" {
t.Errorf("api_version = %q", doc.APIVersion)
}
if doc.Description == "" {
t.Error("index should carry a usage description")
}
if len(doc.Discovery) == 0 {
t.Fatal("discovery list is empty")
}
want := map[string]bool{
"/api/v1/overview": false, "/api/v1/sources": false,
"/api/v1/auto": false, "/api/v1/models": false,
"/api/v1/health": false, "/api/v1/keys": false,
"/api/sources": false, "/api/auto": false, "/v1/chat/completions": false,
}
for _, e := range doc.Discovery {
if _, ok := want[e.Path]; ok {
want[e.Path] = true
}
if e.Method == "" || e.Path == "" || e.Auth == "" || e.Summary == "" {
t.Errorf("incomplete discovery entry: %+v", e)
}
}
for p, found := range want {
if !found {
t.Errorf("discovery is missing %s", p)
}
}
if doc.Conventions["errors"] == nil {
t.Error("conventions should document the error shape")
}
}
func TestAPIV1OverviewSummarisesState(t *testing.T) {
g, _, _ := v1Gateway(t)
rec := doReq(t, g, http.MethodGet, "/api/v1/overview", "")
if rec.Code != http.StatusOK {
t.Fatalf("status = %d: %s", rec.Code, rec.Body.String())
}
var doc struct {
Summary struct {
SourceCount int `json:"source_count"`
ModelCount int `json:"model_count"`
GatewayKeyCount int `json:"gateway_key_count"`
} `json:"summary"`
Sources []map[string]interface{} `json:"sources"`
Auto []map[string]interface{} `json:"auto"`
Health []map[string]interface{} `json:"health"`
Caller struct {
Role string `json:"role"`
} `json:"caller"`
}
decodeJSON(t, rec.Body.String(), &doc)
if doc.Summary.SourceCount != 1 {
t.Errorf("source_count = %d, want 1", doc.Summary.SourceCount)
}
if doc.Summary.ModelCount != 1 {
t.Errorf("model_count = %d, want 1", doc.Summary.ModelCount)
}
if doc.Summary.GatewayKeyCount < 2 {
t.Errorf("gateway_key_count = %d, want >= 2", doc.Summary.GatewayKeyCount)
}
if len(doc.Sources) != 1 || doc.Sources[0]["name"] != "up" {
t.Errorf("sources = %+v", doc.Sources)
}
if doc.Caller.Role != "admin" {
t.Errorf("caller.role = %q, want admin", doc.Caller.Role)
}
}
func TestAPIV1NeverEchoesSecrets(t *testing.T) {
g, admin, user := v1Gateway(t)
for _, path := range []string{"/api/v1", "/api/v1/overview", "/api/v1/sources", "/api/v1/sources/up", "/api/v1/keys", "/api/v1/models", "/api/v1/health", "/api/v1/auto"} {
rec := doReq(t, g, http.MethodGet, path, "")
if rec.Code != http.StatusOK {
t.Fatalf("%s status = %d: %s", path, rec.Code, rec.Body.String())
}
if strings.Contains(rec.Body.String(), "sk-up-secret") {
t.Errorf("%s leaked the source api_key", path)
}
if admin != "" && strings.Contains(rec.Body.String(), admin) {
t.Errorf("%s leaked the admin gateway key", path)
}
if strings.Contains(rec.Body.String(), user) {
t.Errorf("%s leaked the user gateway key", path)
}
}
// Masking must still be useful: it says a key is configured.
rec := doReq(t, g, http.MethodGet, "/api/v1/sources/up", "")
if !strings.Contains(rec.Body.String(), `"api_key_set":true`) {
t.Error("masked source should report that a key is configured")
}
}
func TestAPIV1KeysRequiresAdminAndMasks(t *testing.T) {
g, _, user := v1Gateway(t)
// Authenticate as the user key by swapping the shared helper's header.
req := newAuthedRequest(t, http.MethodGet, "/api/v1/keys", user)
rec := serveViaHandler(t, g, req)
if rec.Code != http.StatusForbidden {
t.Errorf("user role on /api/v1/keys = %d, want 403: %s", rec.Code, rec.Body.String())
}
}
func TestAPIV1SourcesNotFound(t *testing.T) {
g, _, _ := v1Gateway(t)
rec := doReq(t, g, http.MethodGet, "/api/v1/sources/nope", "")
if rec.Code != http.StatusNotFound {
t.Errorf("status = %d, want 404: %s", rec.Code, rec.Body.String())
}
}
func TestAPIV1RejectsWrites(t *testing.T) {
g, _, _ := v1Gateway(t)
for _, m := range []string{http.MethodPost, http.MethodPut, http.MethodDelete, http.MethodPatch} {
rec := doReq(t, g, m, "/api/v1/sources", `{}`)
if rec.Code == http.StatusOK {
t.Errorf("%s /api/v1/sources returned 200; the v1 read facade must not mutate", m)
}
}
// And the source must be untouched.
rec := doReq(t, g, http.MethodGet, "/api/v1/sources", "")
if !strings.Contains(rec.Body.String(), `"name":"up"`) {
t.Error("the source disappeared after a rejected write")
}
}
func TestAPIV1ModelsGroupsBySource(t *testing.T) {
g, _, _ := v1Gateway(t)
rec := doReq(t, g, http.MethodGet, "/api/v1/models", "")
if rec.Code != http.StatusOK {
t.Fatalf("status = %d", rec.Code)
}
var doc struct {
Count int `json:"count"`
Models []string `json:"models"`
BySource map[string][]string `json:"by_source"`
}
decodeJSON(t, rec.Body.String(), &doc)
if doc.Count != 1 || doc.Models[0] != "m1" {
t.Errorf("models = %+v", doc.Models)
}
if len(doc.BySource["up"]) != 1 || doc.BySource["up"][0] != "m1" {
t.Errorf("by_source = %+v", doc.BySource)
}
}
func TestAPIV1RequiresAuthentication(t *testing.T) {
g, _, _ := v1Gateway(t)
for _, path := range []string{"/api/v1", "/api/v1/overview", "/api/v1/sources", "/api/v1/health", "/api/v1/models"} {
req := newRequest(http.MethodGet, path, "")
rec := serveViaHandler(t, g, req)
if rec.Code != http.StatusUnauthorized {
t.Errorf("%s without a key = %d, want 401", path, rec.Code)
}
}
}
func TestMaskKeyHidesTheSecret(t *testing.T) {
cases := map[string]string{
"": "",
"ab": "ab…",
"sk-gw-abcdefghijkl": "sk-gw-…ijkl",
"enc:v1:AAAA": "(sealed)",
}
for in, want := range cases {
if got := maskKey(in); got != want {
t.Errorf("maskKey(%q) = %q, want %q", in, got, want)
}
}
// The masked form must not reveal the middle of the secret.
full := maskKey("sk-gw-1234567890abcdef")
if strings.Contains(full, "4567890abcdef") {
t.Errorf("maskKey leaked the middle: %q", full)
}
}
// newRequest builds an unauthenticated request.
func newRequest(method, path, body string) *http.Request {
req, _ := http.NewRequest(method, path, strings.NewReader(body))
if body != "" {
req.Header.Set("Content-Type", "application/json")
}
return req
}
// newAuthedRequest builds a request carrying the given bearer key.
func newAuthedRequest(t *testing.T, method, path, key string) *http.Request {
t.Helper()
req := newRequest(method, path, "")
req.Header.Set("Authorization", "Bearer "+key)
return req
}
// serveViaHandler pushes a request through the full handler chain (auth included).
func serveViaHandler(t *testing.T, g *Gateway, req *http.Request) *httptest.ResponseRecorder {
t.Helper()
rec := httptest.NewRecorder()
g.Handler().ServeHTTP(rec, req)
return rec
}
// TestAPIV1RevealRequiresAdmin: the only way to read a key in the clear is the
// explicit admin-only reveal. Without the role check this endpoint would hand
// every source credential to any valid (even user-scoped) key.
func TestAPIV1RevealRequiresAdmin(t *testing.T) {
g, _, user := v1Gateway(t)
rec := serveViaHandler(t, g, newAuthedRequest(t, http.MethodGet, "/api/v1/sources/up?reveal=credentials", user))
if rec.Code != http.StatusForbidden {
t.Errorf("user reveal = %d, want 403: %s", rec.Code, rec.Body.String())
}
if strings.Contains(rec.Body.String(), "sk-up-secret") {
t.Error("a forbidden reveal still leaked the key")
}
rec = doReq(t, g, http.MethodGet, "/api/v1/sources/up?reveal=credentials", "")
if rec.Code != http.StatusOK {
t.Fatalf("admin reveal = %d: %s", rec.Code, rec.Body.String())
}
if !strings.Contains(rec.Body.String(), "sk-up-secret") {
t.Error("admin reveal should return the key in the clear (the edit form needs it)")
}
// The masked default view must still hide it on the same path.
rec = doReq(t, g, http.MethodGet, "/api/v1/sources/up", "")
if strings.Contains(rec.Body.String(), "sk-up-secret") {
t.Error("the default view leaked the key")
}
}

View File

@ -7,6 +7,7 @@ import (
"fmt"
"log"
"net/http"
"regexp"
"strings"
"sync/atomic"
"time"
@ -28,6 +29,9 @@ type chatRequest struct {
ToolChoice interface{} `json:"tool_choice,omitempty"`
DisableThinking bool `json:"disable_thinking"`
ExtraBody map[string]interface{} `json:"extra_body,omitempty"`
// PromptCacheKey 是 OpenAI 原生的缓存键;部分客户端用它携带会话标识。
PromptCacheKey string `json:"prompt_cache_key,omitempty"`
}
// ChatCompletion is the non-streaming OpenAI response object.
@ -38,6 +42,10 @@ type ChatCompletion struct {
Model string `json:"model"`
Choices []ChatChoice `json:"choices"`
Usage *types.TokenUsage `json:"usage,omitempty"`
// Cost is the upstream-reported charge for this request, passed through
// verbatim (OpenCode reports it as a decimal string, always "0" on the
// flat-rate Go subscription). Absent when the upstream reports nothing.
Cost string `json:"cost,omitempty"`
}
type ChatChoice struct {
@ -62,6 +70,10 @@ type ChatChunk struct {
// Usage is sent in the final chunk of a stream (empty choices) so
// OpenAI-compatible clients can read token usage.
Usage *types.TokenUsage `json:"usage,omitempty"`
// Cost is the upstream-reported charge for this request, passed through
// verbatim (OpenCode reports it as a decimal string). Emitted on the
// terminal chunk, mirroring OpenCode's own {"choices":[],"cost":"0"}.
Cost string `json:"cost,omitempty"`
}
type ChunkChoice struct {
@ -115,13 +127,15 @@ func (g *Gateway) resolveCands(ctx context.Context, req *chatRequest) ([]*provid
}
// filterCandsByModels keeps only providers exposing at least one model of the
// scope (used for user keys with a restricted model scope). An "AUTO" scope
// entry means the key is allowed to use any model. Scope entries with a
// Source pinned to a specific upstream narrow the candidates to that source
// scope (used for user keys with a restricted model scope). Scope entries with
// a Source pinned to a specific upstream narrow the candidates to that source
// for the matching model.
//
// An "AUTO" scope entry only allows the AUTO routing mode; it does NOT grant
// access to specific models.
func filterCandsByModels(cands []*provider.Provider, allow []config.ModelScope) []*provider.Provider {
for _, m := range allow {
if m.Model == "" || strings.EqualFold(m.Model, "AUTO") {
if m.Model == "" {
return cands
}
}
@ -152,20 +166,18 @@ func filterCandsByModels(cands []*provider.Provider, allow []config.ModelScope)
return out
}
// intersectModels restricts a model list to the scope (preserving order). An
// "AUTO" scope entry grants every model.
// intersectModels restricts a model list to the scope (preserving order).
// An "AUTO" scope entry only allows the AUTO routing mode; it does NOT grant
// every model.
func intersectModels(models []string, allow []config.ModelScope) []string {
allowed := make(map[string]bool, len(allow))
any := false
for _, m := range allow {
if m.Model == "" || strings.EqualFold(m.Model, "AUTO") {
any = true
break
if m.Model == "" {
return models
}
if !strings.EqualFold(m.Model, "AUTO") {
allowed[m.Model] = true
}
allowed[m.Model] = true
}
if any {
return models
}
out := make([]string, 0, len(models))
seen := map[string]bool{}
@ -179,24 +191,16 @@ func intersectModels(models []string, allow []config.ModelScope) []string {
}
// checkModelScope validates the effective model against the key's model scope
// and token quota. Returns an error message when rejected. A scope entry with
// model "AUTO" grants all models; its quota caps the key's total tokens.
// and token quota. Returns an error message when rejected.
//
// A scope entry with model "AUTO" only allows requests where the effective
// model is AUTO (the routing mode). It does NOT grant access to specific model
// ids — that requires an explicit scope entry for the model.
func (g *Gateway) checkModelScope(ctx context.Context, model string) string {
allow := g.allowedModels(ctx)
if allow == nil {
return ""
}
for _, sc := range allow {
if sc.Model != "" && strings.EqualFold(sc.Model, "AUTO") {
if sc.TokenQuota > 0 {
used := g.scopeTokens(ctx, sc)
if used >= sc.TokenQuota {
return fmt.Sprintf("token quota exceeded (%d/%d)", used, sc.TokenQuota)
}
}
return ""
}
}
for _, sc := range allow {
if sc.Model != model {
continue
@ -229,12 +233,15 @@ func (g *Gateway) scopeTokens(ctx context.Context, sc config.ModelScope) int64 {
// source that actually serves the bare model (via Registry.EffectiveModel), so
// model ids that themselves contain separators (e.g. "deepseek-v4-flash-free")
// are never corrupted (P10-2).
//
// A scope entry with model "AUTO" only matches the literal AUTO routing mode;
// it does NOT grant access to specific model ids.
func (g *Gateway) hasScopeModel(list []config.ModelScope, s string) bool {
if r := g.core.Registry(); r != nil {
s = r.EffectiveModel(s)
}
for _, x := range list {
if x.Model == s || (x.Model != "" && strings.EqualFold(x.Model, "AUTO")) {
if x.Model == s {
return true
}
}
@ -287,6 +294,8 @@ func (g *Gateway) handleChat(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
return
}
// 客户端自带的会话标识(若有)比推导出来的更准,见 clientSessionFromRequest。
clientSession := clientSessionFromRequest(r, req.PromptCacheKey)
if len(req.Messages) == 0 {
writeError(w, http.StatusBadRequest, "invalid_request", "messages is required")
return
@ -317,6 +326,7 @@ func (g *Gateway) handleChat(w http.ResponseWriter, r *http.Request) {
ToolChoice: req.ToolChoice,
DisableThinking: req.DisableThinking,
ExtraBody: req.ExtraBody,
ClientSession: clientSession,
}
rec := &Req{
Key: keyID(reqKey(ctx)),
@ -372,6 +382,7 @@ func (g *Gateway) handleChat(w http.ResponseWriter, r *http.Request) {
ToolChoice: req.ToolChoice,
DisableThinking: req.DisableThinking,
ExtraBody: req.ExtraBody,
ClientSession: clientSession,
}
rec := &Req{
Key: keyID(reqKey(ctx)),
@ -517,21 +528,115 @@ func upstreamErrStatus(err error) int {
return http.StatusBadGateway
}
// overflowMarkers 匹配上游「上下文超窗」类措辞。
//
// 上游写法五花八门,且**不在 pi 客户端的识别列表里**。pi 靠
// @earendil-works/pi-ai 的 OVERFLOW_PATTERNS 判断超窗并据此触发压缩重试,
// 而 justworker 返回的是「请精简对话历史…(Context window is full…)」——
// 与那 25 条正则一条都不匹配,于是 pi 既不压缩也不重试,只把它当成一条
// 普通上游错误。这里把可识别的超窗措辞归一化成 pi 一定认得的标记。
var overflowMarkers = []*regexp.Regexp{
regexp.MustCompile(`(?i)context[ _-]?window is full`),
regexp.MustCompile(`(?i)context[_ ]length[_ ]exceeded`),
regexp.MustCompile(`(?i)exceeds? the context window`),
regexp.MustCompile(`(?i)maximum context length`),
regexp.MustCompile(`(?i)reduce the length of the messages`),
regexp.MustCompile(`(?i)too many tokens`),
regexp.MustCompile(`(?i)token limit exceeded`),
regexp.MustCompile(`请精简对话历史`),
regexp.MustCompile(`上下文(长度)?超(出|限)`),
regexp.MustCompile(`对话历史过长`),
}
// overflowCanonical 命中 pi 的 /context[_ ]length[_ ]exceeded/i。
//
// 注意:这个前缀只是必要条件,不是充分条件——pi 还会先用
// NON_OVERFLOW_PATTERNS 排除整条消息,见 overflowClientMessage。
const overflowCanonical = "context_length_exceeded"
// looksLikeOverflow 判断错误文本是否属于上下文超窗。
func looksLikeOverflow(s string) bool {
for _, re := range overflowMarkers {
if re.MatchString(s) {
return true
}
}
return false
}
// clientUpstreamErr collapses upstream failure details into a short message
// for the client: per-tier bodies (WAF HTML pages, quota payloads, ...) stay
// in rec.Err / the stats API and the server log instead of the response.
// Per-tier one-line reasons are kept (quota/cooling skips carry no error body
// and are the actionable part); each is capped so HTML dumps can't leak.
//
// 超窗会被归一化成「干净」的 overflowCanonical 消息,见 overflowClientMessage。
func clientUpstreamErr(err error) string {
log.Printf("[gateway] upstream failure surfaced to client: %v", err)
if looksLikeOverflow(err.Error()) {
return overflowClientMessage(err)
}
return upstreamErrSummary(err)
}
// overflowHint 是给客户端的超窗说明,刻意只用 pi 认得的措辞。
const overflowHint = "context window is full; reduce the length of the messages"
// clientSessionFromRequest 取客户端自带的会话标识,拿不到时返回 ""。
//
// 通用 OpenAI 客户端默认不带会话 id,但 pi 支持:只要 provider 的 compat 里打开
// sendSessionAffinityHeaders,pi 就会把**平台真实的会话 id**(uuidv7,整个会话
// 恒定)放到 x-session-affinity / x-client-request-id 上(sessionAffinityFormat
// 为 openrouter 时是 x-session-id,为 openai 时是 session_id)。
// 有了它就不必再从首条 user 消息推导会话指纹(那只在客户端不发会话 id 时才作为
// 退路,且历史压缩后会漂移)。prompt_cache_key 是 OpenAI 原生的缓存键,
// 客户端若带也一并当会话用。
//
// 刻意不采纳 x-client-request-id:名字含 request,部分客户端每请求都换,
// 拿它当会话会让上游缓存永不命中。pi 总会同时发 x-session-affinity,够用。
func clientSessionFromRequest(r *http.Request, bodyKey string) string {
for _, h := range []string{"X-Session-Affinity", "X-Session-Id", "Session-Id"} {
if v := strings.TrimSpace(r.Header.Get(h)); v != "" {
return v
}
}
return strings.TrimSpace(bodyKey)
}
// overflowClientMessage 为超窗失败生成「干净」的客户端消息。
//
// 为什么不能沿用 upstreamErrSummary:pi 的 isContextOverflow 先查
// NON_OVERFLOW_PATTERNS(/rate limit/、/too many requests/、Bedrock 前缀),
// 一旦命中就直接判为「非超窗」——**哪怕消息里已经有 context_length_exceeded**,
// pi 也不会压缩重试。而 AUTO 链的失败消息天生是多 tier 原因的拼接:
// 超窗 tier(gozen 400 maximum context length)常与配额/限流 tier
// (429 token plan exhausted、cooling、no free slot)同时出现。
// 把明细原样带出去,等于让一条限流 tier 的措辞反过来封杀超窗识别。
// 所以这里只保留「超窗」措辞 + 是哪个源超的窗,其余一律不带。
func overflowClientMessage(err error) string {
var ce *scheduler.ChainErr
if errors.As(err, &ce) {
for _, t := range ce.Tiers {
if looksLikeOverflow(t.Err.Error()) {
return fmt.Sprintf("%s: %s (%s/%s)", overflowCanonical, overflowHint, t.Source, t.Model)
}
}
}
return fmt.Sprintf("%s: %s", overflowCanonical, overflowHint)
}
// upstreamErrSummary 把上游失败压成一行短消息。
func upstreamErrSummary(err error) string {
var ce *scheduler.ChainErr
if errors.As(err, &ce) {
parts := make([]string, 0, len(ce.Tiers)+len(ce.Skipped))
for _, t := range ce.Tiers {
parts = append(parts, types.OneLine(fmt.Sprintf("%s/%s: %v", t.Source, t.Model, t.Err), 80))
// 160 而不是 80:短诊断词("Context window is full")常落在尾部,
// 80 字节按字节截断正好会把它切掉,超窗就再也认不出来。
parts = append(parts, types.OneLine(fmt.Sprintf("%s/%s: %v", t.Source, t.Model, t.Err), 160))
}
for _, sk := range ce.Skipped {
parts = append(parts, types.OneLine(sk, 80))
parts = append(parts, types.OneLine(sk, 160))
}
msg := strings.Join(parts, "; ")
if len(msg) > 300 {
@ -606,6 +711,7 @@ func writeChatCompletion(w http.ResponseWriter, resp *types.UnifiedResponse, mod
Created: time.Now().Unix(),
Model: modelName,
Choices: []ChatChoice{{Index: 0, Message: msg, FinishReason: resp.FinishReason}},
Cost: resp.Cost,
}
if resp.TokenUsage.Total > 0 || resp.TokenUsage.Prompt > 0 || resp.TokenUsage.Completion > 0 {
out.Usage = &resp.TokenUsage
@ -725,11 +831,15 @@ func (g *Gateway) pumpStream(w http.ResponseWriter, rec *Req, chunks <-chan type
// source's status-page latency average.
rec.FirstByteMs = time.Since(t0).Milliseconds()
var lastUsage *types.TokenUsage
var lastCost string
lastFinish := ""
for ck := range chunks {
if ck.Usage != nil {
lastUsage = mergeUsage(lastUsage, ck.Usage)
}
if ck.Cost != "" {
lastCost = ck.Cost
}
chunk := ChatChunk{
ID: id, Object: "chat.completion.chunk", Created: created, Model: modelName,
}
@ -796,6 +906,14 @@ func (g *Gateway) pumpStream(w http.ResponseWriter, rec *Req, chunks <-chan type
ID: id, Object: "chat.completion.chunk", Created: created, Model: modelName,
Choices: []ChunkChoice{},
Usage: tut,
Cost: lastCost,
})
} else if lastCost != "" {
// Cost arrived without any usage (OpenCode sends it on its own frame).
send(ChatChunk{
ID: id, Object: "chat.completion.chunk", Created: created, Model: modelName,
Choices: []ChunkChoice{},
Cost: lastCost,
})
}
fmt.Fprintf(w, "data: [DONE]\n\n")

View File

@ -0,0 +1,47 @@
package gateway
import (
"net/http"
"testing"
)
// 客户端自带的会话标识必须被采纳:它比「首条 user 消息指纹」更准
// (历史压缩后指纹会漂移),也是唯一能天然按会话隔离的来源。
func TestClientSessionFromRequestHeaders(t *testing.T) {
cases := []struct {
name string
headers map[string]string
bodyKey string
want string
}{
{"pi 默认格式 x-session-affinity", map[string]string{"X-Session-Affinity": "ses-1"}, "", "ses-1"},
{"openrouter 格式 x-session-id", map[string]string{"X-Session-Id": "ses-2"}, "", "ses-2"},
{"openai 格式 session_id", map[string]string{"Session-Id": "ses-3"}, "", "ses-3"},
{"body prompt_cache_key 兜底", nil, "cache-4", "cache-4"},
{"头优先于 body", map[string]string{"X-Session-Affinity": "ses-5"}, "cache-x", "ses-5"},
{"空白不算", map[string]string{"X-Session-Affinity": " "}, "", ""},
{"都没有则为空(适配器自行推导)", nil, "", ""},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
r, _ := http.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
for k, v := range tc.headers {
r.Header.Set(k, v)
}
if got := clientSessionFromRequest(r, tc.bodyKey); got != tc.want {
t.Errorf("clientSessionFromRequest = %q, want %q", got, tc.want)
}
})
}
}
// 刻意不采纳 x-client-request-id:名字含 request,部分客户端每请求都换,
// 拿它当会话会让上游前缀缓存永不命中。
func TestClientRequestIDIsNotUsedAsSession(t *testing.T) {
r, _ := http.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
r.Header.Set("X-Client-Request-Id", "per-request-uuid")
if got := clientSessionFromRequest(r, ""); got != "" {
t.Errorf("x-client-request-id 不应被当作会话(否则缓存永不命中),got %q", got)
}
}

View File

@ -670,8 +670,11 @@ func TestHasScopeModelWithSourcePrefix(t *testing.T) {
if g.hasScopeModel(scope, "deepseek-v4-flash-free-extra") {
t.Error("hasScopeModel returned true for unrelated model")
}
if !g.hasScopeModel([]config.ModelScope{{Model: "AUTO"}}, "zen:anything") {
t.Error("AUTO scope should allow any prefixed model")
if g.hasScopeModel([]config.ModelScope{{Model: "AUTO"}}, "zen:deepseek-v4-flash-free") {
t.Error("AUTO scope must NOT grant specific model ids even with prefix")
}
if g.hasScopeModel([]config.ModelScope{{Model: "AUTO"}}, "deepseek-v4-pro") {
t.Error("AUTO scope must NOT grant specific model ids")
}
}

View File

@ -0,0 +1,167 @@
package gateway
import (
"errors"
"regexp"
"strings"
"testing"
"llmsproxy/internal/scheduler"
"llmsproxy/internal/types"
)
// piOverflowRe 是 @earendil-works/pi-ai 的 OVERFLOW_PATTERNS 中对应的一条。
// 归一化标记必须命中它,否则改了等于没改:客户端仍不会压缩重试。
var piOverflowRe = regexp.MustCompile(`context[_ ]length[_ ]exceeded`)
func TestOverflowCanonicalMatchesPiPattern(t *testing.T) {
if !piOverflowRe.MatchString(overflowCanonical) {
t.Fatalf("归一化标记 %q 不匹配 pi 的超窗正则,客户端仍不会压缩重试", overflowCanonical)
}
}
// justworker 的真实措辞:与 pi 现有 25 条正则一条都不匹配,
// 所以必须在网关侧归一化,否则超窗对客户端完全不可见。
func TestJustworkerOverflowIsNormalized(t *testing.T) {
raw := "请精简对话历史…(Context window is full, please reduce the conversation history)…"
// 前提校验:原始措辞确实不被 pi 认出(否则本用例失去意义)。
if piOverflowRe.MatchString(raw) {
t.Fatalf("前提不成立:原始措辞本就匹配 pi 正则: %q", raw)
}
if !looksLikeOverflow(raw) {
t.Fatalf("网关应识别出这是超窗: %q", raw)
}
err := &scheduler.ChainErr{Tiers: []scheduler.TierError{{
Tier: 1, Source: "justworker", Model: "claude-opus-5", Err: errors.New(raw),
}}}
got := clientUpstreamErr(err)
if !piOverflowRe.MatchString(got) {
t.Fatalf("归一化后仍不匹配 pi 正则,客户端不会压缩重试: %q", got)
}
}
// 直连路径(非 AUTO 链)也要归一化——它走的是同一个入口。
func TestDirectPathOverflowIsNormalized(t *testing.T) {
got := clientUpstreamErr(errors.New("upstream 400: Context window is full"))
if !piOverflowRe.MatchString(got) {
t.Fatalf("直连超窗未被归一化: %q", got)
}
}
// 非超窗错误不能被加上超窗标记,否则会把普通失败误报成需要压缩。
func TestNonOverflowNotNormalized(t *testing.T) {
err := &scheduler.ChainErr{Tiers: []scheduler.TierError{{
Tier: 1, Source: "x", Model: "y", Err: errors.New("502 Bad Gateway: upstream connect error"),
}}}
got := clientUpstreamErr(err)
if strings.Contains(got, overflowCanonical) {
t.Fatalf("普通上游失败被误标为超窗: %q", got)
}
if !strings.Contains(got, "502 Bad Gateway") {
t.Fatalf("普通失败的诊断信息应保留: %q", got)
}
}
// 回归:短诊断词常落在错误尾部,80 字节截断会正好把它切掉,
// 超窗就永远认不出来。宽度放宽后必须保住。
func TestOverflowPhraseSurvivesTruncationWidth(t *testing.T) {
prefix := strings.Repeat("x", 100)
// OneLine 的长度限制作用于整条 "source/model: err",不是只算 err 本身。
full := "justworker/claude-opus-5: " + prefix + " Context window is full"
// 前提:80 字节下短语确实会被切掉(这正是修复前的行为)。
if got := types.OneLine(full, 80); strings.Contains(got, "Context window is full") {
t.Fatalf("前提不成立:80 字节下短语居然还在: %q", got)
}
// 160 字节下必须保住,否则超窗识别会被截断悄悄破坏。
if got := types.OneLine(full, 160); !strings.Contains(got, "Context window is full") {
t.Fatalf("160 字节应保住尾部短诊断词: %q", got)
}
err := &scheduler.ChainErr{Tiers: []scheduler.TierError{{
Tier: 1, Source: "justworker", Model: "claude-opus-5",
Err: errors.New(prefix + " Context window is full"),
}}}
got := clientUpstreamErr(err)
if !strings.Contains(strings.ToLower(got), "context window is full") {
t.Fatalf("宽度 160 应保住尾部短诊断词,实际: %q", got)
}
if !piOverflowRe.MatchString(got) {
t.Fatalf("归一化标记丢失: %q", got)
}
}
// piNonOverflowRe 是 @earendil-works/pi-ai 的 NON_OVERFLOW_PATTERNS。
// pi 的 isContextOverflow 先查这一组,命中则直接判为「非超窗」——
// 即使 OVERFLOW_PATTERNS 也命中,也不会压缩重试。
var piNonOverflowRe = []*regexp.Regexp{
regexp.MustCompile(`^(Throttling error|Service unavailable):`),
regexp.MustCompile(`(?i)rate limit`),
regexp.MustCompile(`(?i)too many requests`),
}
// piWouldCompact 复刻 isContextOverflow 的 Case 1 判定顺序:
// 先排除 NON_OVERFLOW,再匹配 OVERFLOW。
func piWouldCompact(errorMessage string) bool {
for _, p := range piNonOverflowRe {
if p.MatchString(errorMessage) {
return false
}
}
return piOverflowRe.MatchString(errorMessage)
}
// issue 回归:AUTO 链失败消息天然把「超窗 tier」和「配额/限流 tier」拼在一起,
// 超窗标记后面只要出现 rate limit / too many requests 的字样,
// pi 就会把整条消息判为「非超窗」而拒绝压缩重试。
func TestOverflowMessageSurvivesRateLimitedSiblingTier(t *testing.T) {
cases := []struct {
name string
sibling string
}{
{"rate limit 措辞", "api error 429: rate limit exceeded"},
{"too many requests 措辞", "429 Too Many Requests"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
err := &scheduler.ChainErr{Tiers: []scheduler.TierError{
{Tier: 1, Source: "sensenova", Model: "glm-5.2", Err: errors.New(tc.sibling)},
{Tier: 3, Source: "gozen", Model: "deepseek-v4.1-flash",
Err: errors.New("api error 400: This model's maximum context length is 1048576 tokens")},
}}
got := clientUpstreamErr(err)
if !piWouldCompact(got) {
t.Fatalf("同链限流 tier 封杀了超窗识别,pi 不会压缩重试: %q", got)
}
// 干净:不得把限流 tier 的措辞带出去。
for _, p := range piNonOverflowRe {
if p.MatchString(got) {
t.Fatalf("超窗消息仍带限流措辞 %v: %q", p, got)
}
}
// 但仍要能看出是哪个源超的窗。
if !strings.Contains(got, "gozen/deepseek-v4.1-flash") {
t.Errorf("超窗源信息丢失: %q", got)
}
})
}
}
// 直连超窗同样要干净可用。
func TestDirectOverflowMessageIsClean(t *testing.T) {
raw := "api error 400: Error from provider (Console Go): This model's maximum context length is 1048576 tokens. " +
"However, you requested 1440050 tokens. Please reduce the length of the messages or completion."
got := clientUpstreamErr(errors.New(raw))
if !piWouldCompact(got) {
t.Fatalf("直连超窗未被 pi 识别: %q", got)
}
if !strings.Contains(got, overflowCanonical) {
t.Fatalf("缺少归一化标记: %q", got)
}
}

View File

@ -222,6 +222,8 @@ func (g *Gateway) routes(w http.ResponseWriter, r *http.Request) {
g.handleModels(w, r)
case r.URL.Path == "/api/adapters" || strings.HasPrefix(r.URL.Path, "/api/adapters/"):
g.handleAdaptersAPI(w, r)
case r.URL.Path == "/api/v1" || strings.HasPrefix(r.URL.Path, "/api/v1/"):
g.apiV1Routes(w, r)
case r.URL.Path == "/api/sources" || strings.HasPrefix(r.URL.Path, "/api/sources/"):
g.handleSourcesAPI(w, r)
case r.URL.Path == "/api/source_templates" || strings.HasPrefix(r.URL.Path, "/api/source_templates/"):

View File

@ -2772,7 +2772,10 @@
/* ---------- sources tab ---------- */
async function renderSources() {
const j = await api("/api/sources");
// Read through /api/v1: the UI is a client of the same public API an
// agent uses, so there is one contract rather than a private path.
// The list view needs no credentials, so the masked view is enough.
const j = await api("/api/v1/sources");
const rows = j.sources
.map(
(
@ -2794,13 +2797,23 @@
function editSource(name) {
const wrap = document.createElement("div");
wrap.id = "modal-wrap";
Promise.all([api("/api/sources"), api("/api/status")])
.then(([src, st]) => {
const s = src.sources.find((x) => x.name === name) || {
name: name,
models: [{ id: "", priority: 0, kind: "chat" }],
};
const cur = s.adapter || "openai";
// The edit dialog round-trips the whole source, so it must load the
// real api_key — otherwise saving an unrelated field would blank
// it. That is what ?reveal=credentials is for (admin-only).
const srcPath = name
? "/api/v1/sources/" + encodeURIComponent(name) + "?reveal=credentials"
: "/api/v1/sources";
Promise.all([api(srcPath), api("/api/status")])
.then(([src, st]) => {
// Editing: the single-source shape. Creating: an empty form.
const s = name
? src.source
: { name: "", models: [{ id: "", priority: 0, kind: "chat" }] };
if (!s) {
toast(t("toastSave"));
return;
}
const cur = s.adapter || "openai";
const apps = ["", ...(st.adapters || []).map((a) => a.name)];
if (cur && !apps.includes(cur)) apps.push(cur);
const adSel =
@ -3215,7 +3228,7 @@
const sortLanes = { chat: [], image: [] };
let sortStateMap = new Map();
async function renderSort() {
const j = await api("/api/sources");
const j = await api("/api/v1/sources");
let autoR = [];
let autoImg = [];
try {
@ -4155,7 +4168,7 @@
async function loadModelPairs() {
let srcs = [];
try {
srcs = (await api("/api/sources")).sources || [];
srcs = (await api("/api/v1/sources")).sources || [];
} catch (e) {}
const pairs = [];
(srcs || []).forEach((s) =>

View File

@ -5,12 +5,34 @@ adapter.version = "1.0.0"
adapter.endpoint = "/chat/completions"
adapter.headers = {}
-- AgentRouter 的 WAF 会按客户端指纹白名单校验,只放行官方客户端。
-- 这里通过 build_headers 注入官方客户端 User-Agent 以通过校验。
-- 默认 QwenCode 指纹(实测可通);可在 source.meta.user_agent 覆盖成 Claude Code 指纹。
-- AgentRouter 的阿里云 WAF 按三重维度校验:出口 IP(必须海外)、
-- TLS 指纹(JS/Go HTTP 层与 curl 不同)、浏览器指纹 header 集。
-- 纯 UA 已不够:WAF 拦 12 种 agent CLI UA 中的 10 种(全 405 HTML),
-- 只有同时带 Origin: qwen.ai + Referer + X-Requested-With 才放行,
-- 且同 IP 短时间连续 3+ 次请求会被速率封禁,靠 adapter 侧低频调用 +
-- llmsproxy 的 prefFailStep 冷却自然限制,来源无法控制速率。
local default_ua = "QwenCode/0.2.0 (linux; x64)"
-- AgentRouter fronts Claude models (claude-opus-4-8), and Claude upstreams
-- enforce Anthropic's tool id rule ^[a-zA-Z0-9_-]{1,64}$ by rejecting the WHOLE
-- request. Plain OpenAI does not, so an id minted by a permissive model (e.g.
-- "bash:0" from moonshotai/kimi-k3) is replayed here by the client and kills
-- every turn. See anthropic.lua / openai.lua for the same helper: Lua adapters
-- have no shared prelude, so each carries its own copy.
local TOOL_ID_MAX = 64
local function safe_tool_id(id)
if type(id) ~= "string" or id == "" then return id end
local clean = string.gsub(id, "[^A-Za-z0-9_-]", "_")
if clean == id and #clean <= TOOL_ID_MAX then
return clean
end
local digest = string.sub(sha256_hex(id), 1, 8)
local keep = TOOL_ID_MAX - #digest - 1
if #clean > keep then clean = string.sub(clean, 1, keep) end
return clean .. "_" .. digest
end
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
@ -19,6 +41,16 @@ function adapter.transform_request(raw_body)
if req.messages then
for _, msg in ipairs(req.messages) do
msg.reasoning_content = nil
if msg.tool_call_id ~= nil then
msg.tool_call_id = safe_tool_id(msg.tool_call_id)
end
if type(msg.tool_calls) == "table" then
for _, tc in ipairs(msg.tool_calls) do
if type(tc) == "table" and tc.id ~= nil then
tc.id = safe_tool_id(tc.id)
end
end
end
end
end
return json.encode(req)
@ -35,6 +67,13 @@ function adapter.build_headers(meta)
["Content-Type"] = "application/json",
["Authorization"] = "Bearer " .. meta.api_key,
["User-Agent"] = ua,
-- QwenCode client fingerprint: WAF rejects bare UA without these.
["Accept"] = "application/json, text/plain, */*",
["Accept-Language"] = "zh-CN,zh;q=0.9",
["Origin"] = "https://qwen.ai",
["Referer"] = "https://qwen.ai/",
["X-Requested-With"] = "XMLHttpRequest",
["Connection"] = "keep-alive",
}
end
@ -79,7 +118,7 @@ function adapter.transform_response(raw_body)
local args_ok, args = pcall(json.decode, tc["function"].arguments)
if not args_ok then args = {} end
table.insert(tcs, {
id = tc.id,
id = safe_tool_id(tc.id),
type = tc.type or "function",
name = tc["function"].name,
arguments = args
@ -138,6 +177,15 @@ function adapter.transform_stream_chunk(raw_chunk)
unified.reasoning_content = delta.reasoning_content
end
if delta.tool_calls then
-- Sanitize outbound too: a dirty id must never enter a client session,
-- because the client replays it to every other source. Only the first
-- fragment of a streamed call carries an id; later argument fragments
-- have none and must stay id-less for index-based accumulation.
for _, tc in ipairs(delta.tool_calls) do
if type(tc) == "table" and tc.id ~= nil then
tc.id = safe_tool_id(tc.id)
end
end
unified.tool_calls = delta.tool_calls
end
if uses ~= nil then

View File

@ -14,6 +14,70 @@ local ROLE_WHITELIST = {
system = true, user = true, assistant = true, tool = true,
}
-- Anthropic requires tool_use.id / tool_result.tool_use_id to match
-- ^[a-zA-Z0-9_-]{1,64}$ and rejects the WHOLE request otherwise with
-- REQUEST_BODY_INVALID / "Invalid tool use format". OpenAI imposes no such
-- rule, so an OpenAI-compatible upstream can hand a client an id like
-- "bash:0" (observed from moonshotai/kimi-k3); once that lands in the
-- client's history, every later replay kills every Claude slot at once and
-- an AUTO chain falls through all tiers.
--
-- safe_tool_id is pure and deterministic, so a tool_use block and its
-- matching tool_result are rewritten identically within one request. A
-- rewritten id keeps an 8-hex digest of the ORIGINAL id: without it two
-- distinct ids could collapse into one ("a:b" and "a_b"), which Anthropic
-- rejects as a duplicate/unpaired tool_use. Already-legal ids pass through
-- untouched so upstreams that round-trip their own ids are unaffected.
local TOOL_ID_MAX = 64
local function safe_tool_id(id)
if type(id) ~= "string" or id == "" then return "" end
local clean = string.gsub(id, "[^A-Za-z0-9_-]", "_")
if clean == id and #clean <= TOOL_ID_MAX then
return clean
end
local digest = string.sub(sha256_hex(id), 1, 8)
local keep = TOOL_ID_MAX - #digest - 1
if #clean > keep then clean = string.sub(clean, 1, keep) end
return clean .. "_" .. digest
end
-- map_usage converts one Anthropic usage object to the gateway's TokenUsage
-- shape. The two APIs disagree on what the prompt count MEANS:
--
-- Anthropic: input_tokens EXCLUDES cached blocks. cache_read_input_tokens and
-- cache_creation_input_tokens are separate, additive, billed input.
-- OpenAI: prompt_tokens INCLUDES its cached_tokens subset.
--
-- Mapping input_tokens straight onto prompt therefore did two wrong things at
-- once: it undercounted the billed prompt by the entire cache portion, and it
-- could report cached_tokens > prompt_tokens — a cache hit rate above 100%.
-- cache_creation_input_tokens was not read at all, so a cache-write turn lost
-- those billed tokens outright. Summing all three restores OpenAI semantics,
-- which is what clients (and the gateway's own audit trail) assume.
--
-- Returns nil when the object carries no counts, so callers can keep treating
-- "no usage in this chunk" as "say nothing" rather than reporting zeros.
local function map_usage(u)
if type(u) ~= "table" then return nil end
local fresh = u.input_tokens or 0
local read = u.cache_read_input_tokens or 0
local create = u.cache_creation_input_tokens or 0
local p = fresh + read + create
local c = u.output_tokens or 0
if p <= 0 and c <= 0 then return nil end
local uses = { prompt = p, completion = c, total = p + c }
-- Emit the split whenever Anthropic reports either cache field, even at 0:
-- a reported zero-hit ("cache missed") must stay distinguishable from "this
-- upstream never reports cache info at all".
if u.cache_read_input_tokens ~= nil or u.cache_creation_input_tokens ~= nil then
uses.prompt_tokens_details = { cached_tokens = read }
uses.prompt_cache_hit_tokens = read
uses.prompt_cache_miss_tokens = fresh + create
end
return uses
end
local function collect_blocks(content)
if type(content) == "string" then
if content == "" then return {} end
@ -104,7 +168,7 @@ function adapter.transform_request(raw_body)
end
table.insert(blocks, {
type = "tool_use",
id = tc.id or "",
id = safe_tool_id(tc.id),
name = fn.name or "",
input = input,
})
@ -119,7 +183,7 @@ function adapter.transform_request(raw_body)
-- user message with tool_result content blocks.
table.insert(pending_tool, {
type = "tool_result",
tool_use_id = m.tool_call_id or "",
tool_use_id = safe_tool_id(m.tool_call_id),
content = text_of(m.content),
})
@ -220,17 +284,9 @@ function adapter.transform_response(raw_body)
token_usage = { prompt = 0, completion = 0, total = 0 },
}
if resp.usage then
unified.token_usage.prompt = resp.usage.input_tokens or 0
unified.token_usage.completion = resp.usage.output_tokens or 0
unified.token_usage.total = (resp.usage.input_tokens or 0) + (resp.usage.output_tokens or 0)
-- Emit details whenever Anthropic reports the field, even at 0, so a
-- reported cache miss stays distinguishable from "not reported".
if resp.usage.cache_read_input_tokens ~= nil then
unified.token_usage.prompt_tokens_details = {
cached_tokens = resp.usage.cache_read_input_tokens
}
end
local mapped = map_usage(resp.usage)
if mapped ~= nil then
unified.token_usage = mapped
end
if resp.content and #resp.content > 0 then
@ -242,7 +298,7 @@ function adapter.transform_response(raw_body)
unified.reasoning_content = (unified.reasoning_content or "") .. block.thinking
elseif block.type == "tool_use" then
table.insert(tcs, {
id = block.id or "",
id = safe_tool_id(block.id),
type = "function",
name = block.name or "",
arguments = block.input or {},
@ -275,17 +331,13 @@ function adapter.transform_stream_chunk(raw_chunk)
-- ── message_start: initial usage ─────────────────────────
if chunk.type == "message_start" then
if chunk.message and type(chunk.message.usage) == "table" then
local u = chunk.message.usage
local p = u.input_tokens or 0
local c = u.output_tokens or 0
if p > 0 or c > 0 then
local uses = { prompt = p, completion = c, total = p + c }
if u.cache_read_input_tokens ~= nil then
uses.prompt_tokens_details = {
cached_tokens = u.cache_read_input_tokens
}
end
if chunk.message then
-- Some Anthropic-compatible upstreams report a placeholder here and
-- only send the true prompt count in message_delta (justwoker:
-- 160 at message_start vs 6931 at message_delta). The gateway's
-- mergeUsage lets a later non-zero value win, so both are emitted.
local uses = map_usage(chunk.message.usage)
if uses ~= nil then
return json.encode({ usage = uses, done = false })
end
end
@ -305,15 +357,10 @@ function adapter.transform_stream_chunk(raw_chunk)
finish = "stop"
end
end
local uses = nil
if type(chunk.usage) == "table" then
local u = chunk.usage
local p = u.input_tokens or 0
local c = u.output_tokens or 0
if p > 0 or c > 0 then
uses = { prompt = p, completion = c, total = p + c }
end
end
-- message_delta carries the FINAL usage, so it must map the cache
-- fields too; the old code dropped them here, which silently lost the
-- whole cache split on every streamed response.
local uses = map_usage(chunk.usage)
if uses ~= nil then
return json.encode({ content = "", done = (finish ~= nil), finish_reason = finish, usage = uses })
end
@ -330,7 +377,7 @@ function adapter.transform_stream_chunk(raw_chunk)
content = "", done = false,
tool_calls = { {
index = chunk.index or 0,
id = cb.id or "",
id = safe_tool_id(cb.id),
type = "function",
["function"] = { name = cb.name or "", arguments = "" },
} },

View File

@ -7,6 +7,10 @@ adapter.headers = {}
-- Gemini API: POST /v1/models/{model}:generateContent
-- Auth: API key in query param ?key=XXX or Authorization: Bearer XXX
--
-- 请求方向必须把 OpenAI 的工具调用翻译成 Gemini 的 functionCall /
-- functionResponse,否则 agent 回放的历史里助手那一轮的调用会凭空消失,
-- 紧随其后的工具结果就成了「无来源」的孤立结果,模型只能反复重发同一个调用。
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
@ -14,16 +18,23 @@ function adapter.transform_request(raw_body)
-- 将 OpenAI 风格 content(字符串或 [{type:*}] 数组)拆成 Gemini parts
local function to_parts(content)
if type(content) == "string" then
if content == "" then return {} end
return { { text = content } }
end
local parts = {}
for _, p in ipairs(content or {}) do
if p.type == "text" then
table.insert(parts, { text = p.text })
elseif p.type == "image_url" and type(p.image_url) == "table" and p.image_url.url then
local mt, b64 = string.match(p.image_url.url, "^data:([^,]+);base64,(.+)$")
if b64 then
table.insert(parts, { inline_data = { mime_type = mt or "image/png", data = b64 } })
if type(p) == "string" then
if p ~= "" then table.insert(parts, { text = p }) end
elseif type(p) == "table" then
if p.type == "text" then
if p.text ~= nil and p.text ~= "" then
table.insert(parts, { text = p.text })
end
elseif p.type == "image_url" and type(p.image_url) == "table" and p.image_url.url then
local mt, b64 = string.match(p.image_url.url, "^data:([^,]+);base64,(.+)$")
if b64 then
table.insert(parts, { inline_data = { mime_type = mt or "image/png", data = b64 } })
end
end
end
end
@ -31,11 +42,59 @@ function adapter.transform_request(raw_body)
end
local contents = {}
local system = ""
local call_names = {} -- tool_call_id -> 函数名(functionResponse 只认名字)
for _, m in ipairs(req.messages or {}) do
table.insert(contents, {
role = (m.role == "assistant") and "model" or m.role,
parts = to_parts(m.content)
})
local role = m.role or "user"
if role == "system" then
if type(m.content) == "string" then
system = system .. m.content .. "\n"
end
elseif role == "assistant" then
local parts = to_parts(m.content)
if type(m.tool_calls) == "table" then
for _, tc in ipairs(m.tool_calls) do
if type(tc) == "table" then
local fn = tc["function"] or {}
local args = fn.arguments
if type(args) == "string" and args ~= "" then
local aok, decoded = pcall(json.decode, args)
args = aok and decoded or {}
elseif type(args) ~= "table" then
args = {}
end
if tc.id ~= nil then call_names[tc.id] = fn.name or "" end
table.insert(parts, {
functionCall = { name = fn.name or "", args = args }
})
end
end
end
-- Gemini 不接受空的 parts 数组;无可发送内容的轮次直接跳过
if #parts > 0 then
table.insert(contents, { role = "model", parts = parts })
end
elseif role == "tool" then
local name = call_names[m.tool_call_id] or m.name or ""
local text = type(m.content) == "string" and m.content or ""
table.insert(contents, {
role = "user",
parts = { { functionResponse = {
name = name,
response = { content = text },
} } }
})
else
local parts = to_parts(m.content)
if #parts > 0 then
table.insert(contents, { role = "user", parts = parts })
end
end
end
local gemini_req = {
@ -46,6 +105,30 @@ function adapter.transform_request(raw_body)
}
}
if system ~= "" then
gemini_req.systemInstruction = { parts = { { text = system } } }
end
-- tools -> functionDeclarations
if type(req.tools) == "table" then
local decls = {}
for _, t in ipairs(req.tools) do
if type(t) == "table" and type(t["function"]) == "table" then
local fn = t["function"]
local params = fn.parameters
if type(params) ~= "table" then params = { type = "object", properties = {} } end
table.insert(decls, {
name = fn.name or "",
description = fn.description or "",
parameters = params,
})
end
end
if #decls > 0 then
gemini_req.tools = { { functionDeclarations = decls } }
end
end
if req.stream then
gemini_req.stream = true
end

View File

@ -6,6 +6,10 @@ adapter.endpoint = "/api/chat"
adapter.headers = {}
-- Ollama API 格式:{ model, messages, stream, options:{temperature,num_predict} }
-- 工具调用必须一并翻译:Ollama 的 assistant 消息用 tool_calls(arguments 是对象
-- 而非 JSON 字符串),工具结果用 tool 角色 + tool_name。此前这里只复制了
-- role/content,助手那轮的调用和 tool 消息的归属全部丢失,模型看到无来源的
-- 工具结果就只能反复重发同一个调用。
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
@ -22,6 +26,7 @@ function adapter.transform_request(raw_body)
-- 转换 messages 格式(Ollama messages 支持 images base64 数组)
if req.messages then
local msgs = {}
local call_names = {} -- tool_call_id -> 函数名
for _, m in ipairs(req.messages) do
local text, images
if type(m.content) == "string" then
@ -30,22 +35,61 @@ function adapter.transform_request(raw_body)
text = ""
images = {}
for _, p in ipairs(m.content or {}) do
if p.type == "text" then
text = text .. (p.text or "")
elseif p.type == "image_url" and type(p.image_url) == "table" and p.image_url.url then
local b64 = string.match(p.image_url.url, "^data:[^,]+;base64,(.+)$")
if b64 then table.insert(images, b64) end
if type(p) == "table" then
if p.type == "text" then
text = text .. (p.text or "")
elseif p.type == "image_url" and type(p.image_url) == "table" and p.image_url.url then
local b64 = string.match(p.image_url.url, "^data:[^,]+;base64,(.+)$")
if b64 then table.insert(images, b64) end
end
end
end
if #images == 0 then images = nil end
end
local msg = { role = m.role, content = text }
if images then msg.images = images end
-- 助手轮的工具调用:arguments 转成对象
if type(m.tool_calls) == "table" and #m.tool_calls > 0 then
local tcs = {}
for _, tc in ipairs(m.tool_calls) do
if type(tc) == "table" then
local fn = tc["function"] or {}
local args = fn.arguments
if type(args) == "string" and args ~= "" then
local aok, decoded = pcall(json.decode, args)
args = aok and decoded or {}
elseif type(args) ~= "table" then
args = {}
end
if tc.id ~= nil then call_names[tc.id] = fn.name or "" end
table.insert(tcs, {
["function"] = { name = fn.name or "", arguments = args }
})
end
end
if #tcs > 0 then msg.tool_calls = tcs end
end
-- 工具结果:Ollama 用 tool_name 标识归属(不认 tool_call_id)
if m.role == "tool" then
local name = call_names[m.tool_call_id] or m.name or ""
if m.tool_call_id ~= nil then
msg.tool_call_id = m.tool_call_id
end
if name ~= "" then msg.tool_name = name end
end
table.insert(msgs, msg)
end
ollama_req.messages = msgs
end
-- tools 透传(Ollama 的 shape 与 OpenAI 一致)
if type(req.tools) == "table" and #req.tools > 0 then
ollama_req.tools = req.tools
end
return json.encode(ollama_req)
end

View File

@ -5,6 +5,36 @@ adapter.version = "2.0.0"
adapter.endpoint = "/chat/completions"
adapter.headers = {}
-- Claude-behind-OpenAI upstreams (tabitoken, 扇贝, …) convert /chat/completions
-- to the Anthropic Messages API internally, so they inherit Anthropic's
-- tool id rule: ^[a-zA-Z0-9_-]{1,64}$, enforced by rejecting the WHOLE request
-- with "Invalid tool use format" / "tool_use.id: String should match pattern".
-- Plain OpenAI has no such rule, so an OpenAI-compatible model can mint an id
-- like "bash:0" (observed from moonshotai/kimi-k3). In a fan-out router that id
-- is replayed to every other source, so one such id kills every Claude slot at
-- once and an AUTO request falls through all tiers.
--
-- safe_tool_id is pure and deterministic, so a tool_calls entry and its
-- matching tool_call_id are rewritten identically within one request. A
-- rewritten id keeps an 8-hex digest of the ORIGINAL id, without which two
-- distinct ids could collapse into one ("a:b" and "a_b") and become an
-- unpaired/duplicate tool call. Already-legal ids are returned untouched, so
-- well-behaved traffic is byte-identical to before.
-- (anthropic.lua carries the same helper; Lua adapters have no shared prelude.)
local TOOL_ID_MAX = 64
local function safe_tool_id(id)
if type(id) ~= "string" or id == "" then return id end
local clean = string.gsub(id, "[^A-Za-z0-9_-]", "_")
if clean == id and #clean <= TOOL_ID_MAX then
return clean
end
local digest = string.sub(sha256_hex(id), 1, 8)
local keep = TOOL_ID_MAX - #digest - 1
if #clean > keep then clean = string.sub(clean, 1, keep) end
return clean .. "_" .. digest
end
-- OpenAI /chat/completions format (pass-through, strip provider-specific fields)
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
@ -14,6 +44,16 @@ function adapter.transform_request(raw_body)
if req.messages then
for _, msg in ipairs(req.messages) do
msg.reasoning_content = nil
if msg.tool_call_id ~= nil then
msg.tool_call_id = safe_tool_id(msg.tool_call_id)
end
if type(msg.tool_calls) == "table" then
for _, tc in ipairs(msg.tool_calls) do
if type(tc) == "table" and tc.id ~= nil then
tc.id = safe_tool_id(tc.id)
end
end
end
end
end
return json.encode(req)
@ -64,7 +104,7 @@ function adapter.transform_response(raw_body)
local args_ok, args = pcall(json.decode, tc["function"].arguments)
if not args_ok then args = {} end
table.insert(tcs, {
id = tc.id,
id = safe_tool_id(tc.id),
type = tc.type or "function",
name = tc["function"].name,
arguments = args
@ -129,6 +169,15 @@ function adapter.transform_stream_chunk(raw_chunk)
unified.reasoning_content = delta.reasoning_content
end
if delta.tool_calls then
-- Sanitize on the way OUT too: an id this upstream happily minted (it
-- does not validate them) becomes a landmine once the client replays
-- it to a Claude upstream. Only the first fragment of a streamed call
-- carries an id; later argument fragments have none and are untouched.
for _, tc in ipairs(delta.tool_calls) do
if type(tc) == "table" and tc.id ~= nil then
tc.id = safe_tool_id(tc.id)
end
end
unified.tool_calls = delta.tool_calls
end
return json.encode(unified)

View File

@ -34,8 +34,10 @@ end
-- OpenAI /chat/completions format (pass-through, strip provider-specific fields)
-- zen 上游 schema 只接受 text content part(无视觉/音频能力):多模态 part
-- (image_url / input_audio / file 等)一律剥离;因此失去全部 content 的
-- 消息整条丢弃,避免上游 "unknown variant `image_url`, expected `text`"。
-- (image_url / input_audio / file 等)一律剥离。剥离后 content 变空的消息
-- 若不再携带 tool_calls / tool_call_id 才整条丢弃(避免上游
-- "unknown variant `image_url`, expected `text`");带工具调用的必须保留,
-- 否则会把紧随其后的 tool 结果变成孤儿,模型会反复重发同一个调用。
-- zen 上游角色白名单只有 system / user / assistant / tool / latest_reminder:
-- OpenAI 的 developer(及 function 等)不在其中,直接透传会触发上游
-- "unknown variant `developer`, expected one of ..." 错误;统一归一化为 system。
@ -52,8 +54,14 @@ function adapter.transform_request(raw_body)
if not ok then return raw_body end
req.disable_thinking = nil
req.extra_body = nil
if type(req.stream_options) ~= "table" then req.stream_options = {} end
req.stream_options.include_usage = true
-- stream_options is only valid alongside stream:true. Sending it on a
-- non-streaming request is rejected by stricter upstreams (OpenCode Go:
-- "stream_options should be set along with stream"), which failed every
-- non-stream call. Only attach it when the request actually streams.
if req.stream then
if type(req.stream_options) ~= "table" then req.stream_options = {} end
req.stream_options.include_usage = true
end
if req.messages then
local kept = {}
for _, msg in ipairs(req.messages) do
@ -72,7 +80,24 @@ function adapter.transform_request(raw_body)
end
end
if #parts == 0 then
drop = true
-- Content collapsed to nothing after stripping unsupported
-- parts. A message that still carries a tool call must
-- NEVER be dropped: the very next message is its tool
-- result, and dropping the call orphans that result. The
-- model then sees a result for a call it never made and
-- re-issues the same tool call on every turn (observed as
-- an infinite "repeated tool call" loop).
--
-- This is the common shape for agent clients: an assistant
-- turn whose content is only [thinking, toolCall] serialises
-- to content:[] with tool_calls — exactly the case that used
-- to vanish here. Emit "" instead, which zen accepts.
if (type(msg.tool_calls) == "table" and #msg.tool_calls > 0)
or msg.tool_call_id ~= nil then
msg.content = ""
else
drop = true
end
else
msg.content = parts
end

View File

@ -0,0 +1,354 @@
local adapter = {}
adapter.name = "opencodego"
adapter.version = "1.0.0"
adapter.endpoint = "/chat/completions"
-- OpenCode Go(包月订阅端点,https://opencode.ai/zen/go/v1)
--
-- 与 OpenCode Zen(opencodezen.lua,https://opencode.ai/zen/v1)是两个不同的
-- 服务,行为要求并不相同,因此各有专用适配器:
--
-- * 本适配器(Go)服务包月订阅池。上游**强制要求**
-- `x-opencode-session` 头,缺失直接 400 "MissingSessionID"。
-- * Go 的 thinking 模型(deepseek-v4.1-flash 等)**必须回传** assistant 轮的
-- `reasoning_content`,否则 400:
-- The `reasoning_content` in the thinking mode must be passed back to the API.
-- 这是本适配器与 Zen 适配器最关键的差异 —— Zen 会剥掉它,Go 必须原样保留。
-- * Go 上游对协议更严格:非流式请求带 `stream_options` 会被拒
-- ("stream_options should be set along with stream")。
--
-- 同样需要 opencode 客户端指纹(UA + x-opencode-*)才能被正确识别与路由。
adapter.headers = {
["User-Agent"] = "opencode/1.18.21 ai-sdk/provider-utils/4.0.23 runtime/bun/1.3.14",
}
-- 每请求生成身份头。沙箱无 os/math,用 meta.timestamp + 请求体哈希派生:
-- 同秒内重复请求 id 相同可接受(zen 只校验存在性,不校验格式)。
local function rand_id(prefix, seed)
return prefix .. string.sub(sha256_hex(seed), 1, 24)
end
-- 会话指纹:取历史里**第一条 user 消息**。
--
-- 为什么不直接用客户端的会话 id:实测抓包(tcpdump 抓 127.0.0.1:8081 的真实
-- agent 请求)确认通用 OpenAI 客户端**根本不发**任何会话标识 —— body 里没有
-- user / session_id / conversation_id / metadata,请求头也只有 X-Stainless-*
-- (OpenAI JS SDK)与 User-Agent。opencode 原生客户端那套 x-opencode-session
-- 是它自己的概念,通用客户端无从转发。
--
-- 退而求其次但要够用:历史被逐轮重放,**第一条 user 消息在整个会话里恒定**,
-- 用它做指纹即可得到"每会话一个 session",而不是"每源一个 session"。
-- 拿不到时(无 user 消息)返回空串,调用方回退到按源稳定。
local function conversation_fingerprint(body)
if type(body) ~= "string" or body == "" then return "" end
local ok, req = pcall(json.decode, body)
if not ok or type(req) ~= "table" then return "" end
for _, m in ipairs(req.messages or {}) do
if type(m) == "table" and m.role == "user" then
local c = m.content
if type(c) == "string" then
if c ~= "" then return c end
elseif type(c) == "table" then
for _, part in ipairs(c) do
if type(part) == "table" and part.type == "text" and part.text then
return part.text
end
end
end
return ""
end
end
return ""
end
-- session_seed 决定上游会话号(x-opencode-session 的种子)。
--
-- 优先级:
-- 1) 客户端自带的会话标识(meta.client_session)——pi 等在开启
-- compat.sendSessionAffinityHeaders 后会发 x-session-affinity,
-- 这是平台真实的会话 id,整个会话恒定且天然按会话隔离。
-- 2) 退路:历史里**第一条 user 消息**做会话指纹。
-- 3) 再退:只按源固定(连 user 消息都没有时)。
--
-- 为什么需要 2/3:通用 OpenAI 客户端默认**根本不发**会话标识 —— 实测抓包
-- (tcpdump 抓 127.0.0.1:8081 真实 agent 请求)确认 body 里没有
-- user / session_id / conversation_id / metadata,请求头也只有
-- X-Stainless-*(OpenAI JS SDK)与 User-Agent。opencode 原生客户端那套
-- x-opencode-session 是它自己的概念,通用客户端无从转发。
--
-- 为什么必须稳定:上游前缀缓存是**会话级**的。会话号每请求一变,
-- 缓存永不命中(实测:固定会号第 2 次命中 5888,每请求换会号则恒为 0)。
local function session_seed(meta)
local src = (meta.source and meta.source.name) or ""
local base = "session|llmsproxy|" .. src
local cs = meta.client_session
if type(cs) == "string" and cs ~= "" then
return base .. "|client|" .. cs
end
return base .. "|" .. conversation_fingerprint(meta.body)
end
function adapter.build_headers(meta)
local ts = tostring(meta.timestamp or "")
local src = (meta.source and meta.source.name) or ""
return {
["User-Agent"] = adapter.headers["User-Agent"],
["x-opencode-client"] = "cli",
-- project 固定:同网关实例共享一个工作区身份
["x-opencode-project"] = string.sub(sha256_hex("llmsproxy|" .. src), 1, 32),
-- session 必须**按源稳定**,不能每请求换:上游的前缀缓存在同一 session
-- 内才复用。实测(同一段 6032 token 提示词):
-- 固定 session -> 第 2 次命中 5888/6032,cached_tokens=5888
-- 每请求换 session -> 永远 0 命中
-- 原先用 meta.timestamp 派生,等于每请求都是新会话,缓存永远无效,
-- 上游也无法做会话亲和路由。
["x-opencode-session"] = rand_id("ses_", session_seed(meta)),
-- request id 仍每请求唯一(它只是请求标识,不参与缓存键)
["x-opencode-request"] = rand_id("msg_", "request|" .. ts .. "|" .. tostring(meta.body or "")),
}
end
-- OpenAI /chat/completions format (pass-through, strip provider-specific fields)
-- zen 上游 schema 只接受 text content part(无视觉/音频能力):多模态 part
-- (image_url / input_audio / file 等)一律剥离。剥离后 content 变空的消息
-- 若不再携带 tool_calls / tool_call_id 才整条丢弃(避免上游
-- "unknown variant `image_url`, expected `text`");带工具调用的必须保留,
-- 否则会把紧随其后的 tool 结果变成孤儿,模型会反复重发同一个调用。
-- zen 上游角色白名单只有 system / user / assistant / tool / latest_reminder:
-- OpenAI 的 developer(及 function 等)不在其中,直接透传会触发上游
-- "unknown variant `developer`, expected one of ..." 错误;统一归一化为 system。
local ROLE_WHITELIST = {
system = true,
user = true,
assistant = true,
tool = true,
latest_reminder = true,
}
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
req.disable_thinking = nil
req.extra_body = nil
-- stream_options is only valid alongside stream:true; sending it on a
-- non-streaming request is rejected by strict upstreams.
if req.stream then
if type(req.stream_options) ~= "table" then req.stream_options = {} end
req.stream_options.include_usage = true
end
if req.messages then
local kept = {}
for _, msg in ipairs(req.messages) do
if type(msg.role) == "string" and not ROLE_WHITELIST[msg.role] then
msg.role = "system"
end
-- Go 的 thinking 模式**必须**回传 reasoning_content:剥掉它会让
-- 上游直接 400("The `reasoning_content` in the thinking mode must
-- be passed back to the API"),agent 的每一轮都会失败。
-- 注意:与 opencodezen.lua 的行为**相反**,不要在这里剥。
--
-- 但绝大多数客户端(pi 等)根本不保存也不回传 reasoning,只保留
-- 工具调用本身。上游只在“带 tool_calls 的助手轮”上校验这个字段,
-- 实测**空串即可通过校验**,所以缺省时补空串:既满足上游的
-- 一致性要求,又不伪造任何推理内容(用户看到的 reasoning 仍然是
-- 上游本轮真实返回的)。
if msg.role == "assistant"
and type(msg.tool_calls) == "table" and #msg.tool_calls > 0
and msg.reasoning_content == nil then
msg.reasoning_content = ""
end
local drop = false
if type(msg.content) == "table" then
local parts = {}
for _, part in ipairs(msg.content) do
if type(part) == "table" and part.type ~= nil and part.type ~= "text" then
-- multimodal part not supported by zen
else
table.insert(parts, part)
end
end
if #parts == 0 then
-- Content collapsed to nothing after stripping unsupported
-- parts. A message that still carries a tool call must
-- NEVER be dropped: the very next message is its tool
-- result, and dropping the call orphans that result. The
-- model then sees a result for a call it never made and
-- re-issues the same tool call on every turn (observed as
-- an infinite "repeated tool call" loop).
if (type(msg.tool_calls) == "table" and #msg.tool_calls > 0)
or msg.tool_call_id ~= nil then
msg.content = ""
else
drop = true
end
else
msg.content = parts
end
end
if not drop then
table.insert(kept, msg)
end
end
req.messages = kept
end
return json.encode(req)
end
function adapter.transform_response(raw_body)
local ok, resp = pcall(json.decode, raw_body)
if not ok or resp == nil then return raw_body end
local unified = {
content = "",
finish_reason = "",
token_usage = { prompt = 0, completion = 0, total = 0 }
}
if type(resp.usage) == "table" then
unified.token_usage.prompt = resp.usage.prompt_tokens or 0
unified.token_usage.completion = resp.usage.completion_tokens or 0
unified.token_usage.total = resp.usage.total_tokens or 0
local hit = 0
if type(resp.usage.prompt_tokens_details) == "table" and resp.usage.prompt_tokens_details.cached_tokens ~= nil then
hit = resp.usage.prompt_tokens_details.cached_tokens
unified.token_usage.prompt_tokens_details = { cached_tokens = hit }
end
if (resp.usage.prompt_cache_hit_tokens or 0) > 0 then
unified.token_usage.prompt_cache_hit_tokens = resp.usage.prompt_cache_hit_tokens
unified.token_usage.prompt_cache_miss_tokens = resp.usage.prompt_cache_miss_tokens or 0
if hit == 0 then
unified.token_usage.prompt_tokens_details = { cached_tokens = resp.usage.prompt_cache_hit_tokens }
end
end
-- 上游报告输出里有多少是思考 token。不给客户端的话,无法判断
-- completion_tokens 里多少是「可见回答」、多少是「思考」(都按输出计费)。
local ctd = resp.usage.completion_tokens_details
if type(ctd) == "table" and ctd.reasoning_tokens ~= nil then
unified.token_usage.completion_tokens_details = { reasoning_tokens = ctd.reasoning_tokens }
end
end
-- 本次调用的费用,上游放在**顶层**且是**字符串**(如 "0"、"0.0012")。
-- Go 订阅是包月制,恒为 "0";只有 Zen 按量付费模型才有信息量。
-- 原样透传:网关不解析、不换算、不汇总 —— 它只是上游事实的搬运者。
if resp.cost ~= nil then
unified.cost = tostring(resp.cost)
end
if type(resp.choices) == "table" and #resp.choices > 0 then
local ch = resp.choices[1]
if type(ch.message) == "table" then
unified.content = ch.message.content or ""
local reasoning = ch.message.reasoning_content or ch.message.reasoning
if reasoning then
unified.reasoning_content = reasoning
end
if type(ch.message.tool_calls) == "table" then
local tcs = {}
for _, tc in ipairs(ch.message.tool_calls) do
local args_ok, args = pcall(json.decode, tc["function"].arguments)
if not args_ok then args = {} end
table.insert(tcs, {
id = tc.id,
type = tc.type or "function",
name = tc["function"].name,
arguments = args
})
end
unified.tool_calls = tcs
end
end
unified.finish_reason = ch.finish_reason or ""
end
return json.encode(unified)
end
function adapter.transform_stream_chunk(raw_chunk)
local ok, chunk = pcall(json.decode, raw_chunk)
if not ok then return "" end
-- OpenAI-style streams may attach usage to a chunk with empty choices
-- (the final usage chunk). Preserve it; the gateway emits it as the
-- terminal usage chunk. Note: keys must match Go's TokenUsage json tags
-- (prompt/completion/total); the gateway re-emits standard *_tokens.
local uses = nil
if type(chunk.usage) == "table" then
uses = {
prompt = chunk.usage.prompt_tokens or chunk.usage.prompt or 0,
completion = chunk.usage.completion_tokens or chunk.usage.completion or 0,
total = chunk.usage.total_tokens or chunk.usage.total or 0,
}
-- 独立字段与 prompt_tokens_details **并存**透传,不要写成 elseif:
-- 上游命中缓存时两者都发,非流式路径也是两个都带。写成 elseif 会让
-- 流式丢掉 prompt_cache_hit_tokens/miss,与同一源的**非流式**产出不一致
-- (dsh 优先读 details,但只认独立字段的老客户端会看不到缓存)。
if type(chunk.usage.prompt_tokens_details) == "table" and chunk.usage.prompt_tokens_details.cached_tokens ~= nil then
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_tokens_details.cached_tokens }
end
if (chunk.usage.prompt_cache_hit_tokens or 0) > 0 then
uses.prompt_cache_hit_tokens = chunk.usage.prompt_cache_hit_tokens
uses.prompt_cache_miss_tokens = chunk.usage.prompt_cache_miss_tokens or 0
if uses.prompt_tokens_details == nil then
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_cache_hit_tokens }
end
end
local ctd = chunk.usage.completion_tokens_details
if type(ctd) == "table" and ctd.reasoning_tokens ~= nil then
uses.completion_tokens_details = { reasoning_tokens = ctd.reasoning_tokens }
end
end
if not chunk.choices or #chunk.choices == 0 then
-- 空 choices 的帧既不是内容也不是结束信号。上游发两种:
-- {"choices":[],"usage":{...}} —— 终帧用量
-- {"choices":[],"cost":"0"} —— 单独的费用帧
-- 两者可能落在同一帧上,必须都转出去;早退只带 usage 会把费用丢干净。
local out = { done = false }
if uses ~= nil then out.usage = uses end
if chunk.cost ~= nil then out.cost = tostring(chunk.cost) end
if uses == nil and chunk.cost == nil then return "" end
return json.encode(out)
end
local delta = chunk.choices[1].delta or {}
local fr = chunk.choices[1].finish_reason
local finish = (type(fr) == "string" and fr ~= "") and fr or nil
local unified = {
content = delta.content or "",
done = (finish ~= nil)
}
if finish then
unified.finish_reason = finish
end
if uses ~= nil then
unified.usage = uses
end
-- zen 用 reasoning 字段承载推理文本(OpenAI 惯例是 reasoning_content)
local reasoning = delta.reasoning_content or delta.reasoning
if reasoning then
unified.reasoning_content = reasoning
end
if delta.tool_calls then
-- pass raw streaming fragments through; OpenAI clients accumulate index+id+name+arguments
unified.tool_calls = delta.tool_calls
end
return json.encode(unified)
end
-- 错误收敛(可选钩子):zen 错误信封固定为 {error={type,message}};
-- 免费池限流 FreeUsageLimitError 单独标注。返回 nil 走通用兜底。
function adapter.transform_error(status, body)
local ok, resp = pcall(json.decode, body)
if not ok or type(resp) ~= "table" then return nil end
local e = resp.error
if type(e) ~= "table" then return nil end
if e.type == "FreeUsageLimitError" then
return "zen free pool quota exhausted"
end
return e.message
end
return adapter

View File

@ -0,0 +1,342 @@
local adapter = {}
adapter.name = "opencodezen"
adapter.version = "1.0.0"
adapter.endpoint = "/chat/completions"
-- OpenCode Zen(按量付费端点,https://opencode.ai/zen/v1)
--
-- 与 OpenCode Go(opencodego.lua,https://opencode.ai/zen/go/v1)是两个不同的
-- 服务,行为要求并不相同,因此各有专用适配器:
--
-- * 本适配器(Zen)服务免费池 / 按量付费池。免费模型必须带 opencode 客户端
-- 指纹(UA + x-opencode-*),否则上游拒绝:"free tier can only be used in
-- OpenCode"。Zen 的免费池以非 thinking 模型为主,历史上回传
-- reasoning_content 会引出上游报错,故这里剥掉(Go 侧相反,见 opencodego.lua)。
-- * Go 订阅读者请看 opencodego.lua:那边**必须**回传 reasoning_content,
-- 否则 thinking 模式直接 400。
--
-- 缺 x-opencode-client/session/request/project 身份头会被判为匿名客户端,
-- 部分模型在带 tools 时上游直接失败(流式返回单 chunk
-- "finish_reason":"network_error" 空 content,非流式 503
-- "Endpoint is unavailable",表现为"空回复")。
adapter.headers = {
["User-Agent"] = "opencode/1.18.21 ai-sdk/provider-utils/4.0.23 runtime/bun/1.3.14",
}
-- 每请求生成身份头。沙箱无 os/math,用 meta.timestamp + 请求体哈希派生:
-- 同秒内重复请求 id 相同可接受(zen 只校验存在性,不校验格式)。
local function rand_id(prefix, seed)
return prefix .. string.sub(sha256_hex(seed), 1, 24)
end
-- 会话指纹:取历史里**第一条 user 消息**。
--
-- 为什么不直接用客户端的会话 id:实测抓包(tcpdump 抓 127.0.0.1:8081 的真实
-- agent 请求)确认通用 OpenAI 客户端**根本不发**任何会话标识 —— body 里没有
-- user / session_id / conversation_id / metadata,请求头也只有 X-Stainless-*
-- (OpenAI JS SDK)与 User-Agent。opencode 原生客户端那套 x-opencode-session
-- 是它自己的概念,通用客户端无从转发。
--
-- 退而求其次但要够用:历史被逐轮重放,**第一条 user 消息在整个会话里恒定**,
-- 用它做指纹即可得到"每会话一个 session",而不是"每源一个 session"。
-- 拿不到时(无 user 消息)返回空串,调用方回退到按源稳定。
local function conversation_fingerprint(body)
if type(body) ~= "string" or body == "" then return "" end
local ok, req = pcall(json.decode, body)
if not ok or type(req) ~= "table" then return "" end
for _, m in ipairs(req.messages or {}) do
if type(m) == "table" and m.role == "user" then
local c = m.content
if type(c) == "string" then
if c ~= "" then return c end
elseif type(c) == "table" then
for _, part in ipairs(c) do
if type(part) == "table" and part.type == "text" and part.text then
return part.text
end
end
end
return ""
end
end
return ""
end
-- session_seed 决定上游会话号(x-opencode-session 的种子)。
--
-- 优先级:
-- 1) 客户端自带的会话标识(meta.client_session)——pi 等在开启
-- compat.sendSessionAffinityHeaders 后会发 x-session-affinity,
-- 这是平台真实的会话 id,整个会话恒定且天然按会话隔离。
-- 2) 退路:历史里**第一条 user 消息**做会话指纹。
-- 3) 再退:只按源固定(连 user 消息都没有时)。
--
-- 为什么需要 2/3:通用 OpenAI 客户端默认**根本不发**会话标识 —— 实测抓包
-- (tcpdump 抓 127.0.0.1:8081 真实 agent 请求)确认 body 里没有
-- user / session_id / conversation_id / metadata,请求头也只有
-- X-Stainless-*(OpenAI JS SDK)与 User-Agent。opencode 原生客户端那套
-- x-opencode-session 是它自己的概念,通用客户端无从转发。
--
-- 为什么必须稳定:上游前缀缓存是**会话级**的。会话号每请求一变,
-- 缓存永不命中(实测:固定会号第 2 次命中 5888,每请求换会号则恒为 0)。
local function session_seed(meta)
local src = (meta.source and meta.source.name) or ""
local base = "session|llmsproxy|" .. src
local cs = meta.client_session
if type(cs) == "string" and cs ~= "" then
return base .. "|client|" .. cs
end
return base .. "|" .. conversation_fingerprint(meta.body)
end
function adapter.build_headers(meta)
local ts = tostring(meta.timestamp or "")
local src = (meta.source and meta.source.name) or ""
return {
["User-Agent"] = adapter.headers["User-Agent"],
["x-opencode-client"] = "cli",
-- project 固定:同网关实例共享一个工作区身份
["x-opencode-project"] = string.sub(sha256_hex("llmsproxy|" .. src), 1, 32),
-- session 必须**按源稳定**,不能每请求换:上游的前缀缓存在同一 session
-- 内才复用。实测(同一段 6032 token 提示词):
-- 固定 session -> 第 2 次命中 5888/6032,cached_tokens=5888
-- 每请求换 session -> 永远 0 命中
-- 原先用 meta.timestamp 派生,等于每请求都是新会话,缓存永远无效,
-- 上游也无法做会话亲和路由。
["x-opencode-session"] = rand_id("ses_", session_seed(meta)),
-- request id 仍每请求唯一(它只是请求标识,不参与缓存键)
["x-opencode-request"] = rand_id("msg_", "request|" .. ts .. "|" .. tostring(meta.body or "")),
}
end
-- OpenAI /chat/completions format (pass-through, strip provider-specific fields)
-- zen 上游 schema 只接受 text content part(无视觉/音频能力):多模态 part
-- (image_url / input_audio / file 等)一律剥离。剥离后 content 变空的消息
-- 若不再携带 tool_calls / tool_call_id 才整条丢弃(避免上游
-- "unknown variant `image_url`, expected `text`");带工具调用的必须保留,
-- 否则会把紧随其后的 tool 结果变成孤儿,模型会反复重发同一个调用。
-- zen 上游角色白名单只有 system / user / assistant / tool / latest_reminder:
-- OpenAI 的 developer(及 function 等)不在其中,直接透传会触发上游
-- "unknown variant `developer`, expected one of ..." 错误;统一归一化为 system。
local ROLE_WHITELIST = {
system = true,
user = true,
assistant = true,
tool = true,
latest_reminder = true,
}
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
req.disable_thinking = nil
req.extra_body = nil
-- stream_options is only valid alongside stream:true; sending it on a
-- non-streaming request is rejected by strict upstreams.
if req.stream then
if type(req.stream_options) ~= "table" then req.stream_options = {} end
req.stream_options.include_usage = true
end
if req.messages then
local kept = {}
for _, msg in ipairs(req.messages) do
if type(msg.role) == "string" and not ROLE_WHITELIST[msg.role] then
msg.role = "system"
end
-- Zen 免费池不接受回传 reasoning_content(Go 侧相反)
msg.reasoning_content = nil
local drop = false
if type(msg.content) == "table" then
local parts = {}
for _, part in ipairs(msg.content) do
if type(part) == "table" and part.type ~= nil and part.type ~= "text" then
-- multimodal part not supported by zen
else
table.insert(parts, part)
end
end
if #parts == 0 then
-- Content collapsed to nothing after stripping unsupported
-- parts. A message that still carries a tool call must
-- NEVER be dropped: the very next message is its tool
-- result, and dropping the call orphans that result. The
-- model then sees a result for a call it never made and
-- re-issues the same tool call on every turn (observed as
-- an infinite "repeated tool call" loop).
if (type(msg.tool_calls) == "table" and #msg.tool_calls > 0)
or msg.tool_call_id ~= nil then
msg.content = ""
else
drop = true
end
else
msg.content = parts
end
end
if not drop then
table.insert(kept, msg)
end
end
req.messages = kept
end
return json.encode(req)
end
function adapter.transform_response(raw_body)
local ok, resp = pcall(json.decode, raw_body)
if not ok or resp == nil then return raw_body end
local unified = {
content = "",
finish_reason = "",
token_usage = { prompt = 0, completion = 0, total = 0 }
}
if type(resp.usage) == "table" then
unified.token_usage.prompt = resp.usage.prompt_tokens or 0
unified.token_usage.completion = resp.usage.completion_tokens or 0
unified.token_usage.total = resp.usage.total_tokens or 0
local hit = 0
if type(resp.usage.prompt_tokens_details) == "table" and resp.usage.prompt_tokens_details.cached_tokens ~= nil then
hit = resp.usage.prompt_tokens_details.cached_tokens
unified.token_usage.prompt_tokens_details = { cached_tokens = hit }
end
if (resp.usage.prompt_cache_hit_tokens or 0) > 0 then
unified.token_usage.prompt_cache_hit_tokens = resp.usage.prompt_cache_hit_tokens
unified.token_usage.prompt_cache_miss_tokens = resp.usage.prompt_cache_miss_tokens or 0
if hit == 0 then
unified.token_usage.prompt_tokens_details = { cached_tokens = resp.usage.prompt_cache_hit_tokens }
end
end
-- 上游报告输出里有多少是思考 token。不给客户端的话,无法判断
-- completion_tokens 里多少是「可见回答」、多少是「思考」(都按输出计费)。
local ctd = resp.usage.completion_tokens_details
if type(ctd) == "table" and ctd.reasoning_tokens ~= nil then
unified.token_usage.completion_tokens_details = { reasoning_tokens = ctd.reasoning_tokens }
end
end
-- 本次调用的费用,上游放在**顶层**且是**字符串**(如 "0"、"0.0012")。
-- Go 订阅是包月制,恒为 "0";只有 Zen 按量付费模型才有信息量。
-- 原样透传:网关不解析、不换算、不汇总 —— 它只是上游事实的搬运者。
if resp.cost ~= nil then
unified.cost = tostring(resp.cost)
end
if type(resp.choices) == "table" and #resp.choices > 0 then
local ch = resp.choices[1]
if type(ch.message) == "table" then
unified.content = ch.message.content or ""
local reasoning = ch.message.reasoning_content or ch.message.reasoning
if reasoning then
unified.reasoning_content = reasoning
end
if type(ch.message.tool_calls) == "table" then
local tcs = {}
for _, tc in ipairs(ch.message.tool_calls) do
local args_ok, args = pcall(json.decode, tc["function"].arguments)
if not args_ok then args = {} end
table.insert(tcs, {
id = tc.id,
type = tc.type or "function",
name = tc["function"].name,
arguments = args
})
end
unified.tool_calls = tcs
end
end
unified.finish_reason = ch.finish_reason or ""
end
return json.encode(unified)
end
function adapter.transform_stream_chunk(raw_chunk)
local ok, chunk = pcall(json.decode, raw_chunk)
if not ok then return "" end
-- OpenAI-style streams may attach usage to a chunk with empty choices
-- (the final usage chunk). Preserve it; the gateway emits it as the
-- terminal usage chunk. Note: keys must match Go's TokenUsage json tags
-- (prompt/completion/total); the gateway re-emits standard *_tokens.
local uses = nil
if type(chunk.usage) == "table" then
uses = {
prompt = chunk.usage.prompt_tokens or chunk.usage.prompt or 0,
completion = chunk.usage.completion_tokens or chunk.usage.completion or 0,
total = chunk.usage.total_tokens or chunk.usage.total or 0,
}
-- 独立字段与 prompt_tokens_details **并存**透传,不要写成 elseif:
-- 上游命中缓存时两者都发,非流式路径也是两个都带。写成 elseif 会让
-- 流式丢掉 prompt_cache_hit_tokens/miss,与同一源的**非流式**产出不一致
-- (dsh 优先读 details,但只认独立字段的老客户端会看不到缓存)。
if type(chunk.usage.prompt_tokens_details) == "table" and chunk.usage.prompt_tokens_details.cached_tokens ~= nil then
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_tokens_details.cached_tokens }
end
if (chunk.usage.prompt_cache_hit_tokens or 0) > 0 then
uses.prompt_cache_hit_tokens = chunk.usage.prompt_cache_hit_tokens
uses.prompt_cache_miss_tokens = chunk.usage.prompt_cache_miss_tokens or 0
if uses.prompt_tokens_details == nil then
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_cache_hit_tokens }
end
end
local ctd = chunk.usage.completion_tokens_details
if type(ctd) == "table" and ctd.reasoning_tokens ~= nil then
uses.completion_tokens_details = { reasoning_tokens = ctd.reasoning_tokens }
end
end
if not chunk.choices or #chunk.choices == 0 then
-- 空 choices 的帧既不是内容也不是结束信号。上游发两种:
-- {"choices":[],"usage":{...}} —— 终帧用量
-- {"choices":[],"cost":"0"} —— 单独的费用帧
-- 两者可能落在同一帧上,必须都转出去;早退只带 usage 会把费用丢干净。
local out = { done = false }
if uses ~= nil then out.usage = uses end
if chunk.cost ~= nil then out.cost = tostring(chunk.cost) end
if uses == nil and chunk.cost == nil then return "" end
return json.encode(out)
end
local delta = chunk.choices[1].delta or {}
local fr = chunk.choices[1].finish_reason
local finish = (type(fr) == "string" and fr ~= "") and fr or nil
local unified = {
content = delta.content or "",
done = (finish ~= nil)
}
if finish then
unified.finish_reason = finish
end
if uses ~= nil then
unified.usage = uses
end
-- zen 用 reasoning 字段承载推理文本(OpenAI 惯例是 reasoning_content)
local reasoning = delta.reasoning_content or delta.reasoning
if reasoning then
unified.reasoning_content = reasoning
end
if delta.tool_calls then
-- pass raw streaming fragments through; OpenAI clients accumulate index+id+name+arguments
unified.tool_calls = delta.tool_calls
end
return json.encode(unified)
end
-- 错误收敛(可选钩子):zen 错误信封固定为 {error={type,message}};
-- 免费池限流 FreeUsageLimitError 单独标注。返回 nil 走通用兜底。
function adapter.transform_error(status, body)
local ok, resp = pcall(json.decode, body)
if not ok or type(resp) ~= "table" then return nil end
local e = resp.error
if type(e) ~= "table" then return nil end
if e.type == "FreeUsageLimitError" then
return "zen free pool quota exhausted"
end
return e.message
end
return adapter

View File

@ -46,9 +46,6 @@ end
-- Returns (tool_calls_array_or_nil, content_with_blocks_removed).
local function parse_text_tool_calls(content)
if type(content) ~= "string" or content == "" then return nil, content end
if not (content:find("<tool_call", 1, true) or content:find("<toolcall", 1, true)) then
return nil, content
end
local tcs = {}
local idx = 0
@ -88,11 +85,31 @@ local function parse_text_tool_calls(content)
collect("<tool_call[^>]*>%s*(.-)%s*</tool_call%s*>")
collect("<toolcall[^>]*>%s*(.-)%s*</toolcall%s*>")
-- Legacy / mimicked form: models that saw past assistant turns folded as
-- [Called tool: name({...})]
-- tend to emit the same shape instead of a real block. Recover it so
-- the agent loop keeps working; otherwise the client receives plain text
-- where a structured tool call should be. Capture the name and the JSON
-- object literal independently (the JSON is the only {...} run here).
if #tcs == 0 and content:find("%[Called tool") then
for name, argsraw in content:gmatch("%[Called tool%s*:%s*([%w_%-%.]+)%s*%((%b{})%s*%)%s*%]") do
local aok, decoded = pcall(json.decode, argsraw)
idx = idx + 1
table.insert(tcs, {
id = "call_legacy_" .. idx,
type = "function",
name = name,
arguments = aok and decoded or {}
})
end
end
if #tcs == 0 then return nil, content end
-- drop the blocks from user-visible content; keep any surrounding prose
local stripped = content:gsub("<tool_call[^>]*>%s*.-%s*</tool_call%s*>", "")
stripped = stripped:gsub("<toolcall[^>]*>%s*.-%s*</toolcall%s*>", "")
stripped = stripped:gsub("%[Called tool%s*:%s*[%w_%-%.]+%s*%(%b{}%s*%)%s*%]", "")
stripped = stripped:gsub("^%s+", ""):gsub("%s+$", "")
return tcs, stripped
end

View File

@ -0,0 +1,353 @@
package lua
import (
"encoding/json"
"strings"
"testing"
)
// TestAdaptersPreserveToolCalls is the fleet-wide guard for the
// "orphaned tool result" class of bug: when an agent client replays a turn
// whose assistant message carries tool_calls, EVERY adapter must forward both
// the call and the attribution of its result. Dropping the call (or the
// tool_call_id / tool_name that ties the result to it) makes the model see a
// result for a call it never made and re-issue the same call forever.
//
// The input deliberately uses content:[] — the shape most agent clients emit
// for a tool-calling assistant turn with no text.
func TestAdaptersPreserveToolCalls(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
body := `{"model":"m","messages":[
{"role":"user","content":"read /tmp/x"},
{"role":"assistant","content":[],"tool_calls":[{"id":"call_abc","type":"function","function":{"name":"read","arguments":"{\"path\":\"/tmp/x\"}"}}]},
{"role":"tool","tool_call_id":"call_abc","content":"hello world"},
{"role":"user","content":"summarize"}
]}`
const (
callID = "call_abc"
fnName = "read"
toolOut = "hello world"
lastUser = "summarize"
)
// Adapters that identify a tool result by the call id.
byID := []string{"openai", "deepseek", "github", "groq", "kimicode", "mistral",
"sensenova", "trae", "agentrouter", "opencode", "anthropic"}
for _, a := range byID {
out, err := vm.Transform(a, "transform_request", body)
if err != nil {
t.Errorf("%s: transform_request: %v", a, err)
continue
}
if !strings.Contains(out, callID) {
t.Errorf("%s dropped the tool call id %q: %s", a, callID, out)
}
if !strings.Contains(out, fnName) {
t.Errorf("%s dropped the function name %q: %s", a, fnName, out)
}
if !strings.Contains(out, toolOut) {
t.Errorf("%s dropped the tool result: %s", a, out)
}
if !strings.Contains(out, lastUser) {
t.Errorf("%s dropped the final user turn: %s", a, out)
}
}
// Ollama identifies the result with tool_name (its API has no tool_call_id
// on assistant turns) and takes arguments as an object, not a JSON string.
out, err := vm.Transform("ollama", "transform_request", body)
if err != nil {
t.Fatalf("ollama: %v", err)
}
for _, want := range []string{`"tool_calls"`, fnName, `"tool_name"`, toolOut} {
if !strings.Contains(out, want) {
t.Errorf("ollama missing %s: %s", want, out)
}
}
// Gemini identifies the result by function NAME and has no call id in its
// wire format; both directions must be present as functionCall /
// functionResponse parts, and the system role must move to
// systemInstruction rather than sitting in contents.
gBody := `{"model":"m","messages":[
{"role":"system","content":"be brief"},
{"role":"user","content":"read /tmp/x"},
{"role":"assistant","content":[],"tool_calls":[{"id":"call_abc","type":"function","function":{"name":"read","arguments":"{\"path\":\"/tmp/x\"}"}}]},
{"role":"tool","tool_call_id":"call_abc","content":"hello world"}
]}`
gout, err := vm.Transform("gemini", "transform_request", gBody)
if err != nil {
t.Fatalf("gemini: %v", err)
}
for _, want := range []string{`"functionCall"`, `"functionResponse"`, fnName, toolOut, `"systemInstruction"`} {
if !strings.Contains(gout, want) {
t.Errorf("gemini missing %s: %s", want, gout)
}
}
// No message may keep the OpenAI-only roles inside contents.
var greq struct {
Contents []struct {
Role string `json:"role"`
} `json:"contents"`
}
if err := json.Unmarshal([]byte(gout), &greq); err != nil {
t.Fatalf("gemini unmarshal: %v (%s)", err, gout)
}
for _, c := range greq.Contents {
if c.Role != "user" && c.Role != "model" {
t.Errorf("gemini contents carry an invalid role %q: %s", c.Role, gout)
}
}
// Negative control: a text-only turn must still be forwarded verbatim.
plain := `{"model":"m","messages":[{"role":"user","content":"just text"}]}`
for _, a := range append(byID, "ollama", "gemini") {
pout, err := vm.Transform(a, "transform_request", plain)
if err != nil {
t.Errorf("%s plain: %v", a, err)
continue
}
if !strings.Contains(pout, "just text") {
t.Errorf("%s dropped plain text: %s", a, pout)
}
}
}
// TestOpenCodeStreamOptionsOnlyWhenStreaming pins the fix for
// "stream_options should be set along with stream": OpenCode Go (and other
// strict OpenAI-compatible upstreams) reject a non-streaming request that
// carries stream_options, which broke every non-stream call through the
// opencode adapter.
func TestOpenCodeStreamOptionsOnlyWhenStreaming(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
// non-streaming: stream_options must be absent entirely
out, err := vm.Transform("opencode", "transform_request",
`{"model":"m","messages":[{"role":"user","content":"hi"}]}`)
if err != nil {
t.Fatal(err)
}
if strings.Contains(out, "stream_options") {
t.Fatalf("non-streaming request must not carry stream_options: %s", out)
}
// streaming: stream_options.include_usage must be set
sout, err := vm.Transform("opencode", "transform_request",
`{"model":"m","stream":true,"messages":[{"role":"user","content":"hi"}]}`)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(sout, "stream_options") || !strings.Contains(sout, "include_usage") {
t.Fatalf("streaming request must carry stream_options.include_usage: %s", sout)
}
}
// TestOpenCodeGoVsZenReasoning pins the one behavioural difference that made a
// shared adapter wrong: OpenCode Go's thinking mode REQUIRES the assistant
// turn's reasoning_content to be echoed back ("The `reasoning_content` in the
// thinking mode must be passed back to the API"), while the Zen free pool must
// not receive it. Hence two purpose-built adapters.
func TestOpenCodeGoVsZenReasoning(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
body := `{"model":"m","messages":[
{"role":"user","content":"hi"},
{"role":"assistant","content":"","reasoning_content":"I should read the file.","tool_calls":[{"id":"call_1","type":"function","function":{"name":"read","arguments":"{\"path\":\"/x\"}"}}]},
{"role":"tool","tool_call_id":"call_1","content":"r"}
]}`
// Go: reasoning_content must survive, and the tool call must stay paired.
gout, err := vm.Transform("opencodego", "transform_request", body)
if err != nil {
t.Fatalf("opencodego: %v", err)
}
if !strings.Contains(gout, "I should read the file.") {
t.Errorf("opencodego must pass reasoning_content back (thinking mode requires it): %s", gout)
}
if !strings.Contains(gout, "call_1") || !strings.Contains(gout, "tool_call_id") {
t.Errorf("opencodego must keep the tool call paired with its result: %s", gout)
}
// Zen: reasoning_content is stripped.
zout, err := vm.Transform("opencodezen", "transform_request", body)
if err != nil {
t.Fatalf("opencodezen: %v", err)
}
if strings.Contains(zout, "I should read the file.") {
t.Errorf("opencodezen must strip reasoning_content: %s", zout)
}
if !strings.Contains(zout, "call_1") {
t.Errorf("opencodezen must still keep the tool call: %s", zout)
}
// An assistant turn that carries tool_calls but no reasoning_content must
// get an empty one injected: Go validates that field on tool-calling turns
// and 400s when it is absent, while most clients never send it.
// Verified against the live endpoint that an empty string passes.
bare := `{"model":"m","messages":[
{"role":"user","content":"go"},
{"role":"assistant","content":"","tool_calls":[{"id":"c1","type":"function","function":{"name":"read","arguments":"{}"}}]},
{"role":"tool","tool_call_id":"c1","content":"r"}
]}`
bout, err := vm.Transform("opencodego", "transform_request", bare)
if err != nil {
t.Fatalf("opencodego bare: %v", err)
}
if !strings.Contains(bout, `"reasoning_content":""`) {
t.Errorf("opencodego must inject an empty reasoning_content on a tool-calling turn: %s", bout)
}
// A plain assistant turn (no tool calls) must NOT gain the field.
plainAsst := `{"model":"m","messages":[{"role":"user","content":"hi"},{"role":"assistant","content":"hello"}]}`
pout, err := vm.Transform("opencodego", "transform_request", plainAsst)
if err != nil {
t.Fatal(err)
}
if strings.Contains(pout, "reasoning_content") {
t.Errorf("opencodego must not inject reasoning_content on a plain assistant turn: %s", pout)
}
// Both must still omit stream_options on non-streaming requests.
for _, a := range []string{"opencodego", "opencodezen"} {
out, err := vm.Transform(a, "transform_request", `{"model":"m","messages":[{"role":"user","content":"hi"}]}`)
if err != nil {
t.Fatalf("%s: %v", a, err)
}
if strings.Contains(out, "stream_options") {
t.Errorf("%s must not send stream_options when not streaming: %s", a, out)
}
}
}
// TestOpenCodeSessionIsStableForCache pins the session-derivation contract that
// makes the upstream prefix cache usable.
//
// Measured against the live OpenCode Go endpoint: the prefix cache is
// SESSION-scoped. Replaying the same 6032-token prompt hits 5888 cached tokens
// when the session id is fixed, and 0 when it changes per request. Deriving the
// session from meta.timestamp (the previous behaviour) meant every request was a
// new session, so the cache could never hit.
//
// The session must be stable WITHIN one conversation and differ BETWEEN
// conversations. Since generic OpenAI clients send no session id at all
// (verified by capturing real agent traffic: no user / session_id /
// conversation_id / metadata in the body, no session header), the conversation
// is fingerprinted by its FIRST user message — history is replayed every turn,
// so that message is invariant for the life of the conversation.
func TestOpenCodeSessionIsStableForCache(t *testing.T) {
for _, name := range []string{"opencodego", "opencodezen"} {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
meta := func(ts int, body string) map[string]interface{} {
return map[string]interface{}{
"timestamp": ts,
"body": body,
"source": map[string]interface{}{"name": "somesource", "meta": nil},
}
}
// Same conversation, later turn: history grew, first user msg unchanged.
h1, err := vm.BuildHeaders(name, meta(1000,
`{"messages":[{"role":"user","content":"fix the bug"},{"role":"assistant","content":"looking"}]}`))
if err != nil {
t.Fatalf("%s: %v", name, err)
}
h2, err := vm.BuildHeaders(name, meta(2000,
`{"messages":[{"role":"user","content":"fix the bug"},{"role":"assistant","content":"looking"},{"role":"user","content":"and tests"}]}`))
if err != nil {
t.Fatalf("%s: %v", name, err)
}
// Different conversation.
h3, err := vm.BuildHeaders(name, meta(3000,
`{"messages":[{"role":"user","content":"an unrelated task"}]}`))
if err != nil {
t.Fatalf("%s: %v", name, err)
}
if h1["x-opencode-session"] == "" {
t.Fatalf("%s: session header must be set", name)
}
if h1["x-opencode-session"] != h2["x-opencode-session"] {
t.Errorf("%s: x-opencode-session must be STABLE across turns of one conversation "+
"(%q vs %q) — the prefix cache is session-scoped", name, h1["x-opencode-session"], h2["x-opencode-session"])
}
if h1["x-opencode-session"] == h3["x-opencode-session"] {
t.Errorf("%s: a different conversation must get a different session", name)
}
if h1["x-opencode-request"] == h2["x-opencode-request"] {
t.Errorf("%s: x-opencode-request must differ per request", name)
}
// No user message at all falls back to per-source stability.
h4, _ := vm.BuildHeaders(name, meta(4000, `{"messages":[{"role":"system","content":"s"}]}`))
h5, _ := vm.BuildHeaders(name, meta(5000, `{"messages":[{"role":"system","content":"s"}]}`))
if h4["x-opencode-session"] != h5["x-opencode-session"] {
t.Errorf("%s: sessionless requests must still be stable per source", name)
}
vm.Stop()
}
}
// 客户端自带的会话标识(pi 的 x-session-affinity)必须优先于会话指纹:
// 它是平台真实会话 id,历史压缩后也不会漂移,且天然按会话隔离。
func TestOpenCodePrefersClientSessionID(t *testing.T) {
for _, name := range []string{"opencodego", "opencodezen"} {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
meta := func(body, cs string) map[string]interface{} {
return map[string]interface{}{
"timestamp": 1000,
"body": body,
"client_session": cs,
"source": map[string]interface{}{"name": "somesource", "meta": nil},
}
}
// 同一条会话:历史在长,但客户端会话 id 不变。
a, err := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"fix the bug"}]}`, "cs-aaa"))
if err != nil {
t.Fatal(err)
}
b, err := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"a totally different opener"}]}`, "cs-aaa"))
if err != nil {
t.Fatal(err)
}
if a["x-opencode-session"] != b["x-opencode-session"] {
t.Errorf("%s: 客户端会话 id 相同就必须同会话(内容无关): %q vs %q",
name, a["x-opencode-session"], b["x-opencode-session"])
}
// 不同会话。
c, _ := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"fix the bug"}]}`, "cs-bbb"))
if a["x-opencode-session"] == c["x-opencode-session"] {
t.Errorf("%s: 不同客户端会话必须不同会号", name)
}
// 客户端没给会话 id 时,回落到指纹(同内容 -> 同会号)。
d, _ := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"fix the bug"}]}`, ""))
e, _ := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"fix the bug"}]}`, ""))
if d["x-opencode-session"] != e["x-opencode-session"] {
t.Errorf("%s: 无客户端会号时应回落到稳定指纹", name)
}
if d["x-opencode-session"] == a["x-opencode-session"] {
t.Errorf("%s: 客户端会号与指纹两条路必须产出不同会号", name)
}
vm.Stop()
}
}

View File

@ -0,0 +1,177 @@
package lua
import (
"encoding/json"
"testing"
)
// OpenCode Zen/Go 在 usage 里报告输出有多少是思考 token,并在**顶层**用一个
// **字符串**给出本次费用。前者决定「completion_tokens 里多少是可见回答」,
// 后者是唯一的费用信号。两者都曾被适配器丢掉。
func TestOpenCodeCostAndReasoningPassthrough(t *testing.T) {
for _, name := range []string{"opencodego", "opencodezen"} {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
// ---- 非流式 ----
resp := `{"id":"x","object":"chat.completion","model":"deepseek-v4.1-flash",
"choices":[{"index":0,"message":{"role":"assistant","content":"hi","reasoning_content":"think"},
"finish_reason":"stop"}],
"usage":{"prompt_tokens":37,"completion_tokens":40,"total_tokens":77,
"prompt_cache_hit_tokens":0,"prompt_cache_miss_tokens":37,
"prompt_tokens_details":{"cached_tokens":0},
"completion_tokens_details":{"reasoning_tokens":40}},
"cost":"0"}`
out, err := vm.Transform(name, "transform_response", resp)
if err != nil {
t.Fatalf("%s transform_response: %v", name, err)
}
var r struct {
TokenUsage struct {
Completion int `json:"completion"`
Details *struct {
ReasoningTokens int `json:"reasoning_tokens"`
} `json:"completion_tokens_details"`
} `json:"token_usage"`
Cost string `json:"cost"`
}
if err := json.Unmarshal([]byte(out), &r); err != nil {
t.Fatalf("%s unmarshal: %v (%s)", name, err, out)
}
if r.Cost != "0" {
t.Errorf("%s: cost 未透传(上游给的是字符串 \"0\"): %q", name, r.Cost)
}
if r.TokenUsage.Details == nil || r.TokenUsage.Details.ReasoningTokens != 40 {
t.Errorf("%s: completion_tokens_details.reasoning_tokens 未透传: %s", name, out)
}
// 前提校验:reasoning_tokens 确实 ≤ completion_tokens(全为思考)。
if r.TokenUsage.Details != nil && r.TokenUsage.Details.ReasoningTokens > r.TokenUsage.Completion {
t.Errorf("%s: reasoning 不应超过 completion", name)
}
// ---- 流式:非零费用也要透传 ----
chunk := `{"id":"x","object":"chat.completion.chunk","choices":[],
"usage":{"prompt_tokens":10,"completion_tokens":5,"total_tokens":15,
"completion_tokens_details":{"reasoning_tokens":4}},"cost":"0.0012"}`
co, err := vm.Transform(name, "transform_stream_chunk", chunk)
if err != nil {
t.Fatalf("%s transform_stream_chunk: %v", name, err)
}
var c struct {
Cost string `json:"cost"`
Usage *struct {
Details *struct {
ReasoningTokens int `json:"reasoning_tokens"`
} `json:"completion_tokens_details"`
} `json:"usage"`
}
if err := json.Unmarshal([]byte(co), &c); err != nil {
t.Fatalf("%s unmarshal chunk: %v (%s)", name, err, co)
}
if c.Usage == nil || c.Usage.Details == nil || c.Usage.Details.ReasoningTokens != 4 {
t.Errorf("%s: 流式 reasoning_tokens 未透传: %s", name, co)
}
if c.Cost != "0.0012" {
t.Errorf("%s: 流式 cost 未透传: %s", name, co)
}
// ---- 独立的 cost 帧(上游就这么发:choices 为空、只有 cost)----
only := `{"choices":[],"cost":"0"}`
oo, err := vm.Transform(name, "transform_stream_chunk", only)
if err != nil {
t.Fatalf("%s cost-only chunk: %v", name, err)
}
if oo == "" {
t.Fatalf("%s: 纯 cost 帧被整个丢弃(客户端将看不到费用)", name)
}
var oc struct {
Cost string `json:"cost"`
}
if err := json.Unmarshal([]byte(oo), &oc); err != nil {
t.Fatalf("%s unmarshal cost chunk: %v (%s)", name, err, oo)
}
if oc.Cost != "0" {
t.Errorf("%s: 纯 cost 帧内容丢失: %s", name, oo)
}
// ---- 回归:没有 cost / 没有 details 时不得凭空造字段 ----
bare := `{"choices":[{"index":0,"delta":{"content":"hi"},"finish_reason":"stop"}],
"usage":{"prompt_tokens":3,"completion_tokens":1,"total_tokens":4}}`
bo, _ := vm.Transform(name, "transform_response", bare)
if got := bo; len(got) > 0 && containsAll(got, `"cost"`) {
t.Errorf("%s: 上游没给 cost 却透传了该字段: %s", name, got)
}
vm.Stop()
}
}
func containsAll(s string, subs ...string) bool {
for _, sub := range subs {
found := false
for i := 0; i+len(sub) <= len(s); i++ {
if s[i:i+len(sub)] == sub {
found = true
break
}
}
if !found {
return false
}
}
return true
}
// 流式与非流式对同一上游切片的产出必须一致:命中缓存时上游同时给
// prompt_tokens_details.cached_tokens 和独立的 hit/miss 字段,两边都要透传。
func TestOpenCodeStreamCacheFieldsMatchNonStream(t *testing.T) {
for _, name := range []string{"opencodego", "opencodezen"} {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
type cache struct {
Details *struct {
CachedTokens int `json:"cached_tokens"`
} `json:"prompt_tokens_details"`
Hit int `json:"prompt_cache_hit_tokens"`
Miss int `json:"prompt_cache_miss_tokens"`
}
usage := `"usage":{"prompt_tokens":5732,"completion_tokens":20,"total_tokens":5752,
"prompt_cache_hit_tokens":5504,"prompt_cache_miss_tokens":228,
"prompt_tokens_details":{"cached_tokens":5504}}`
// 非流式
nout, _ := vm.Transform(name, "transform_response",
`{"choices":[{"index":0,"message":{"role":"assistant","content":"hi"},"finish_reason":"stop"}],`+usage+`}`)
var nr struct {
U cache `json:"token_usage"`
}
if err := json.Unmarshal([]byte(nout), &nr); err != nil {
t.Fatalf("%s: %v", name, err)
}
// 流式
sout, _ := vm.Transform(name, "transform_stream_chunk",
`{"choices":[],`+usage+`}`)
var sr struct {
U cache `json:"usage"`
}
if err := json.Unmarshal([]byte(sout), &sr); err != nil {
t.Fatalf("%s chunk: %v (%s)", name, err, sout)
}
if nr.U.Hit != 5504 || nr.U.Miss != 228 {
t.Errorf("%s: 非流式 hit/miss 异常: %+v", name, nr.U)
}
if sr.U.Hit != nr.U.Hit || sr.U.Miss != nr.U.Miss {
t.Errorf("%s: 流式 hit/miss 与非流式不一致 (stream %d/%d vs non-stream %d/%d): %s",
name, sr.U.Hit, sr.U.Miss, nr.U.Hit, nr.U.Miss, sout)
}
if sr.U.Details == nil || sr.U.Details.CachedTokens != 5504 {
t.Errorf("%s: 流式 cached_tokens 丢失: %s", name, sout)
}
vm.Stop()
}
}

View File

@ -4,6 +4,7 @@ import (
"encoding/json"
"os"
"path/filepath"
"regexp"
"strings"
"sync"
"testing"
@ -220,6 +221,70 @@ func TestOpenCodeStripsMultiModalParts(t *testing.T) {
}
}
// TestOpenCodeKeepsToolCallWithEmptyContent pins the fix for an infinite
// "repeated tool call" loop: an assistant turn whose content serialises to an
// empty ARRAY while carrying tool_calls must survive the multimodal strip.
// Dropping it orphans the following tool result, so the model never sees the
// call it already made and re-issues it on every turn.
func TestOpenCodeKeepsToolCallWithEmptyContent(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
body := `{"model":"x","messages":[
{"role":"user","content":"read /tmp/x"},
{"role":"assistant","content":[],"tool_calls":[{"id":"call_abc","type":"function","function":{"name":"read","arguments":"{\"path\":\"/tmp/x\"}"}}]},
{"role":"tool","tool_call_id":"call_abc","content":"hello"},
{"role":"user","content":"now summarize"}
]}`
out, err := vm.Transform("opencode", "transform_request", body)
if err != nil {
t.Fatalf("transform_request: %v", err)
}
var req struct {
Messages []struct {
Role string `json:"role"`
Content json.RawMessage `json:"content"`
ToolCalls []struct {
ID string `json:"id"`
} `json:"tool_calls"`
ToolCallID string `json:"tool_call_id"`
} `json:"messages"`
}
if err := json.Unmarshal([]byte(out), &req); err != nil {
t.Fatalf("unmarshal: %v (%s)", err, out)
}
if len(req.Messages) != 4 {
t.Fatalf("expected 4 messages (tool call must not be dropped), got %d: %s", len(req.Messages), out)
}
asst := req.Messages[1]
if asst.Role != "assistant" || len(asst.ToolCalls) != 1 || asst.ToolCalls[0].ID != "call_abc" {
t.Fatalf("assistant tool_calls lost: %s", out)
}
if string(asst.Content) != `""` {
t.Fatalf("empty content must be rewritten to \"\", got %s", asst.Content)
}
if req.Messages[2].Role != "tool" || req.Messages[2].ToolCallID != "call_abc" {
t.Fatalf("tool result must stay paired with its call: %s", out)
}
// Negative control: a message with neither text nor tool calls is still dropped.
only := `{"model":"x","messages":[
{"role":"user","content":[{"type":"image_url","image_url":{"url":"https://ex.com/a.png"}}]},
{"role":"user","content":"keep me"}
]}`
out2, err := vm.Transform("opencode", "transform_request", only)
if err != nil {
t.Fatal(err)
}
if got := strings.Count(out2, `"role"`); got != 1 {
t.Fatalf("image-only message without tool calls must still be dropped, got %d: %s", got, out2)
}
}
func TestBuildHeadersCustomHook(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
@ -1599,3 +1664,424 @@ func TestTraeTextToolCallRecovery(t *testing.T) {
t.Errorf("plain answer finish_reason changed: %s", out)
}
}
// TestToolIDSanitize locks the tool-call id sanitizer in both adapters that
// talk to Claude upstreams. Anthropic requires tool_use.id /
// tool_result.tool_use_id to match ^[a-zA-Z0-9_-]{1,64}$ and rejects the WHOLE
// request otherwise (REQUEST_BODY_INVALID / "Invalid tool use format"), while
// OpenAI has no such rule — so an OpenAI-compatible model can mint "bash:0"
// (observed from moonshotai/kimi-k3) and poison a client's history, killing
// every Claude slot on replay. tabitoken/扇贝 are Claude-behind-OpenAI, so
// openai.lua needs the same treatment as anthropic.lua.
func TestToolIDSanitize(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
legal := regexp.MustCompile(`^[a-zA-Z0-9_-]{1,64}$`)
long := strings.Repeat("a", 200)
body := func(id string) string {
b, err := json.Marshal(map[string]interface{}{
"model": "x",
"messages": []interface{}{
map[string]interface{}{"role": "user", "content": "run whoami"},
map[string]interface{}{"role": "assistant", "content": "", "tool_calls": []interface{}{
map[string]interface{}{"id": id, "type": "function",
"function": map[string]interface{}{"name": "bash", "arguments": `{"command":"whoami"}`}},
}},
map[string]interface{}{"role": "tool", "tool_call_id": id, "content": "root"},
},
})
if err != nil {
t.Fatal(err)
}
return string(b)
}
// ---- anthropic: tool_use.id and tool_result.tool_use_id must agree ----
for _, id := range []string{"bash:0", "call_ok_123", long, "toolu~sig1:AB+/=="} {
out, err := vm.Transform("anthropic", "transform_request", body(id))
if err != nil {
t.Fatalf("anthropic transform %q: %v", id, err)
}
var r struct {
Messages []struct {
Content []struct {
Type string `json:"type"`
ID string `json:"id"`
ToolUseID string `json:"tool_use_id"`
} `json:"content"`
} `json:"messages"`
}
if err := json.Unmarshal([]byte(out), &r); err != nil {
t.Fatalf("anthropic unmarshal %q: %v (%s)", id, err, out)
}
var useID, resID string
for _, m := range r.Messages {
for _, c := range m.Content {
switch c.Type {
case "tool_use":
useID = c.ID
case "tool_result":
resID = c.ToolUseID
}
}
}
if !legal.MatchString(useID) {
t.Errorf("anthropic %q: tool_use.id %q violates Anthropic's id pattern", id, useID)
}
if useID != resID {
t.Errorf("anthropic %q: tool_use.id %q != tool_result.tool_use_id %q (unpaired call)", id, useID, resID)
}
if legal.MatchString(id) && useID != id {
t.Errorf("anthropic %q: already-legal id must pass through untouched, got %q", id, useID)
}
}
// ---- openai + agentrouter: same rewrite, and both sites must stay in sync ----
// agentrouter fronts claude-opus-4-8, so it inherits the same id rule.
for _, adapterName := range []string{"openai", "agentrouter"} {
for _, id := range []string{"bash:0", "call_ok_123", long} {
out, err := vm.Transform(adapterName, "transform_request", body(id))
if err != nil {
t.Fatalf("%s transform %q: %v", adapterName, id, err)
}
var r struct {
Messages []struct {
Role string `json:"role"`
ToolCalls []struct {
ID string `json:"id"`
} `json:"tool_calls"`
ToolCallID string `json:"tool_call_id"`
} `json:"messages"`
}
if err := json.Unmarshal([]byte(out), &r); err != nil {
t.Fatalf("%s unmarshal %q: %v (%s)", adapterName, id, err, out)
}
var callID, resID string
for _, m := range r.Messages {
if len(m.ToolCalls) > 0 {
callID = m.ToolCalls[0].ID
}
if m.Role == "tool" {
resID = m.ToolCallID
}
}
if !legal.MatchString(callID) {
t.Errorf("%s %q: tool_calls[0].id %q still illegal for Claude-behind-OpenAI upstreams", adapterName, id, callID)
}
if callID != resID {
t.Errorf("%s %q: tool_calls id %q != tool_call_id %q (unpaired call)", adapterName, id, callID, resID)
}
if legal.MatchString(id) && callID != id {
t.Errorf("%s %q: already-legal id must pass through untouched, got %q", adapterName, id, callID)
}
}
}
// ---- distinct dirty ids must NOT collapse into one (digest suffix) ----
seen := map[string]string{}
for _, id := range []string{"a:b", "a_b", "a-b", "a b", "a/b"} {
out, err := vm.Transform("anthropic", "transform_request", body(id))
if err != nil {
t.Fatalf("collision probe %q: %v", id, err)
}
var r struct {
Messages []struct {
Content []struct {
Type string `json:"type"`
ID string `json:"id"`
} `json:"content"`
} `json:"messages"`
}
if err := json.Unmarshal([]byte(out), &r); err != nil {
t.Fatal(err)
}
got := ""
for _, m := range r.Messages {
for _, c := range m.Content {
if c.Type == "tool_use" {
got = c.ID
}
}
}
if prev, dup := seen[got]; dup {
t.Errorf("ids %q and %q both map to %q — a collision makes a duplicate tool_use", prev, id, got)
}
seen[got] = id
}
// ---- determinism: same input, same output ----
a, _ := vm.Transform("anthropic", "transform_request", body("bash:0"))
b, _ := vm.Transform("anthropic", "transform_request", body("bash:0"))
if a != b {
t.Errorf("sanitizer is not deterministic:\n%s\n%s", a, b)
}
}
// TestToolIDSanitizeResponse locks the OUTBOUND half of the sanitizer. The
// request-side rewrite alone is not enough: an id minted by a permissive
// OpenAI-compatible upstream (moonshotai/kimi-k3 returns "bash:0") is handed to
// the client, stored in its history, and replayed forever after. Sanitizing on
// the way out means a poisoned id never enters a client session in the first
// place — both for non-streaming responses and for the first fragment of a
// streamed tool call (later argument fragments carry no id).
func TestToolIDSanitizeResponse(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
legal := regexp.MustCompile(`^[a-zA-Z0-9_-]{1,64}$`)
// ---- openai + agentrouter non-streaming ----
resp := `{"choices":[{"message":{"role":"assistant","content":null,"tool_calls":[
{"id":"bash:0","type":"function","function":{"name":"bash","arguments":"{\"command\":\"whoami\"}"}}]},
"finish_reason":"tool_calls"}]}`
var nonStreamID string
for _, adapterName := range []string{"openai", "agentrouter"} {
out, err := vm.Transform(adapterName, "transform_response", resp)
if err != nil {
t.Fatalf("%s transform_response: %v", adapterName, err)
}
var r struct {
ToolCalls []struct {
ID string `json:"id"`
} `json:"tool_calls"`
}
if err := json.Unmarshal([]byte(out), &r); err != nil {
t.Fatalf("%s unmarshal: %v (%s)", adapterName, err, out)
}
if len(r.ToolCalls) != 1 {
t.Fatalf("%s tool_calls lost: %s", adapterName, out)
}
if !legal.MatchString(r.ToolCalls[0].ID) {
t.Errorf("%s non-stream response leaks illegal id %q to the client", adapterName, r.ToolCalls[0].ID)
}
if nonStreamID != "" && r.ToolCalls[0].ID != nonStreamID {
t.Errorf("%s rewrote %q differently than another adapter (%q) — a client mixing sources gets unpaired calls",
adapterName, r.ToolCalls[0].ID, nonStreamID)
}
nonStreamID = r.ToolCalls[0].ID
}
// ---- streaming: first fragment carries the id (both adapters) ----
chunk := `{"choices":[{"delta":{"tool_calls":[
{"index":0,"id":"bash:0","type":"function","function":{"name":"bash","arguments":""}}]},
"finish_reason":null}]}`
var c struct {
ToolCalls []struct {
ID string `json:"id"`
} `json:"tool_calls"`
}
for _, adapterName := range []string{"openai", "agentrouter"} {
out, err := vm.Transform(adapterName, "transform_stream_chunk", chunk)
if err != nil {
t.Fatalf("%s transform_stream_chunk: %v", adapterName, err)
}
if err := json.Unmarshal([]byte(out), &c); err != nil {
t.Fatalf("%s unmarshal chunk: %v (%s)", adapterName, err, out)
}
if len(c.ToolCalls) != 1 {
t.Fatalf("%s stream tool_calls lost: %s", adapterName, out)
}
if !legal.MatchString(c.ToolCalls[0].ID) {
t.Errorf("%s stream response leaks illegal id %q to the client", adapterName, c.ToolCalls[0].ID)
}
// Streaming and non-streaming must agree, otherwise a client that mixes
// modes within one session produces unpaired tool calls.
if c.ToolCalls[0].ID != nonStreamID {
t.Errorf("%s stream id %q != non-stream id %q for the same input", adapterName, c.ToolCalls[0].ID, nonStreamID)
}
}
// ---- a later argument fragment has no id and must stay id-less ----
argChunk := `{"choices":[{"delta":{"tool_calls":[
{"index":0,"function":{"arguments":"{\"command\":\"whoami\"}"}}]},"finish_reason":null}]}`
for _, adapterName := range []string{"openai", "agentrouter"} {
out, err := vm.Transform(adapterName, "transform_stream_chunk", argChunk)
if err != nil {
t.Fatalf("%s arg fragment: %v", adapterName, err)
}
if strings.Contains(out, `"id"`) {
t.Errorf("%s argument fragment gained an id (breaks index-based accumulation): %s", adapterName, out)
}
}
// ---- anthropic response side: tool_use id and stream start ----
var r struct {
ToolCalls []struct {
ID string `json:"id"`
} `json:"tool_calls"`
}
aResp := `{"content":[{"type":"tool_use","id":"toolu~sig1:AB+/==","name":"bash","input":{"command":"whoami"}}],"stop_reason":"tool_use"}`
out, err := vm.Transform("anthropic", "transform_response", aResp)
if err != nil {
t.Fatalf("anthropic transform_response: %v", err)
}
if err := json.Unmarshal([]byte(out), &r); err != nil {
t.Fatalf("unmarshal anthropic: %v (%s)", err, out)
}
if len(r.ToolCalls) != 1 || !legal.MatchString(r.ToolCalls[0].ID) {
t.Errorf("anthropic response leaks illegal id: %s", out)
}
aChunk := `{"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu~sig1:AB+/==","name":"bash"}}`
out, err = vm.Transform("anthropic", "transform_stream_chunk", aChunk)
if err != nil {
t.Fatalf("anthropic stream chunk: %v", err)
}
if err := json.Unmarshal([]byte(out), &c); err != nil {
t.Fatalf("unmarshal anthropic chunk: %v (%s)", err, out)
}
if len(c.ToolCalls) != 1 || !legal.MatchString(c.ToolCalls[0].ID) {
t.Errorf("anthropic stream leaks illegal id: %s", out)
}
}
// TestAnthropicUsageCacheMapping pins the Anthropic→OpenAI usage conversion.
// The two APIs disagree on what the prompt count means: Anthropic's
// input_tokens EXCLUDES cached blocks (cache_read_input_tokens and
// cache_creation_input_tokens are separate, additive, billed input), while
// OpenAI's prompt_tokens INCLUDES its cached_tokens subset. Mapping
// input_tokens straight onto prompt undercounted the billed prompt by the whole
// cache portion and could report cached_tokens > prompt_tokens (a hit rate over
// 100%); cache_creation_input_tokens was dropped entirely.
func TestAnthropicUsageCacheMapping(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
type usage struct {
Prompt int `json:"prompt"`
Completion int `json:"completion"`
Total int `json:"total"`
Details *struct {
CachedTokens int `json:"cached_tokens"`
} `json:"prompt_tokens_details"`
Hit int `json:"prompt_cache_hit_tokens"`
Miss int `json:"prompt_cache_miss_tokens"`
}
// ---- non-streaming, cache read + cache write reported ----
resp := `{"content":[{"type":"text","text":"ok"}],"stop_reason":"end_turn",
"usage":{"input_tokens":1200,"cache_read_input_tokens":40000,
"cache_creation_input_tokens":500,"output_tokens":80}}`
out, err := vm.Transform("anthropic", "transform_response", resp)
if err != nil {
t.Fatalf("transform_response: %v", err)
}
// Each case decodes into a FRESH value: json.Unmarshal leaves fields absent
// from the payload untouched, so a reused struct would carry the previous
// case's prompt_tokens_details into a response that has none.
decode := func(out string) usage {
t.Helper()
var r struct {
TokenUsage usage `json:"token_usage"`
}
if err := json.Unmarshal([]byte(out), &r); err != nil {
t.Fatalf("unmarshal: %v (%s)", err, out)
}
return r.TokenUsage
}
decodeChunk := func(out string) (usage, string) {
t.Helper()
var c struct {
Usage usage `json:"usage"`
FinishReason string `json:"finish_reason"`
}
if err := json.Unmarshal([]byte(out), &c); err != nil {
t.Fatalf("unmarshal chunk: %v (%s)", err, out)
}
return c.Usage, c.FinishReason
}
u := decode(out)
if u.Prompt != 1200+40000+500 {
t.Errorf("prompt must sum fresh+cache_read+cache_creation (41700), got %d", u.Prompt)
}
if u.Total != u.Prompt+u.Completion {
t.Errorf("total %d != prompt %d + completion %d", u.Total, u.Prompt, u.Completion)
}
if u.Details == nil || u.Details.CachedTokens != 40000 {
t.Errorf("cached_tokens must carry cache_read (40000): %s", out)
}
if u.Details != nil && u.Details.CachedTokens > u.Prompt {
t.Errorf("cached_tokens %d > prompt %d implies a hit rate above 100%%",
u.Details.CachedTokens, u.Prompt)
}
if u.Hit != 40000 || u.Miss != 1200+500 {
t.Errorf("hit/miss split wrong: hit=%d miss=%d (want 40000/1700)", u.Hit, u.Miss)
}
// ---- a reported zero hit must stay distinguishable from "not reported" ----
zero := `{"content":[],"stop_reason":"end_turn","usage":{"input_tokens":100,"cache_read_input_tokens":0,"output_tokens":5}}`
out, err = vm.Transform("anthropic", "transform_response", zero)
if err != nil {
t.Fatal(err)
}
if decode(out).Details == nil {
t.Errorf("a reported cache_read of 0 must still emit prompt_tokens_details: %s", out)
}
// An upstream that never mentions caching must not gain a fabricated split
// (justwoker reports only input_tokens/output_tokens plus its own cost fields).
none := `{"content":[],"stop_reason":"end_turn","usage":{"input_tokens":6931,"output_tokens":1}}`
out, err = vm.Transform("anthropic", "transform_response", none)
if err != nil {
t.Fatal(err)
}
plain := decode(out)
if plain.Details != nil {
t.Errorf("upstream reported no cache fields; details must be absent: %s", out)
}
if plain.Prompt != 6931 || plain.Total != 6932 {
t.Errorf("plain usage mismapped: %s", out)
}
// ---- streaming: message_delta carries the FINAL usage and must map cache ----
delta := `{"type":"message_delta","delta":{"stop_reason":"end_turn"},
"usage":{"input_tokens":1200,"cache_read_input_tokens":40000,"output_tokens":80}}`
out, err = vm.Transform("anthropic", "transform_stream_chunk", delta)
if err != nil {
t.Fatalf("stream delta: %v", err)
}
du, fin := decodeChunk(out)
if du.Prompt != 41200 {
t.Errorf("stream final prompt must include cache_read: got %d", du.Prompt)
}
if du.Details == nil || du.Details.CachedTokens != 40000 {
t.Errorf("stream final usage dropped the cache split: %s", out)
}
if fin != "stop" {
t.Errorf("finish_reason regressed: %s", out)
}
// message_start's placeholder count must not suppress the later real one:
// justwoker reports input_tokens=160 at message_start and 6931 at
// message_delta, and the gateway's mergeUsage lets the later value win.
start := `{"type":"message_start","message":{"usage":{"input_tokens":160,"output_tokens":1}}}`
out, err = vm.Transform("anthropic", "transform_stream_chunk", start)
if err != nil {
t.Fatalf("message_start: %v", err)
}
if su, _ := decodeChunk(out); su.Prompt != 160 {
t.Errorf("message_start usage lost: %s", out)
}
// A usage-less chunk must stay silent rather than reporting zeros.
quiet := `{"type":"message_start","message":{}}`
out, err = vm.Transform("anthropic", "transform_stream_chunk", quiet)
if err != nil {
t.Fatal(err)
}
if strings.Contains(out, "usage") {
t.Errorf("usage-less chunk must not emit a usage object: %s", out)
}
}

View File

@ -13,6 +13,7 @@ import (
"log"
"net"
"net/http"
"net/url"
"strconv"
"strings"
"sync"
@ -24,6 +25,20 @@ import (
"llmsproxy/internal/types"
)
// proxyFor picks the transport proxy for one source. An explicit per-source
// proxy_url wins (e.g. clash on 127.0.0.1:7890 for a geo-blocked upstream);
// otherwise fall back to the process-wide env proxy, which is direct by
// default.
func proxyFor(cfg config.Source) func(*http.Request) (*url.URL, error) {
if cfg.ProxyURL != "" {
proxyURL, err := url.Parse(cfg.ProxyURL)
if err == nil {
return http.ProxyURL(proxyURL)
}
}
return http.ProxyFromEnvironment
}
// ---- per-(source,model) scheduling state ----
// ModelState is the scheduling state of one (source, model) pair: a soft
@ -388,8 +403,11 @@ func New(cfg config.Source, vm *lua.VM) *Provider {
// Shared transport: ResponseHeaderTimeout bounds how long we wait for the
// first response byte (applies to both paths); the stream client has no
// client-level Timeout so the SSE body can run past the header timeout.
// A per-source ProxyURL (e.g. clash on 127.0.0.1:7890) overrides the
// process-wide env proxy for upstreams that are geo/IP-blocked; sources
// without one keep http.ProxyFromEnvironment (direct by default).
tr := &http.Transport{
Proxy: http.ProxyFromEnvironment,
Proxy: proxyFor(cfg),
DialContext: (&net.Dialer{Timeout: 30 * time.Second, KeepAlive: 30 * time.Second}).DialContext,
ForceAttemptHTTP2: true,
MaxIdleConns: 100,
@ -689,7 +707,7 @@ func (p *Provider) probeModels(ctx context.Context) (bool, string) {
if err != nil {
return false, ""
}
if hdrs, herr := p.buildHeaders("{}", u); herr == nil {
if hdrs, herr := p.buildHeaders("{}", u, ""); herr == nil {
req.Header = hdrs
}
resp, err := p.client.Do(req)
@ -732,7 +750,7 @@ func (p *Provider) probeChat(ctx context.Context) (bool, string) {
body, err := json.Marshal(probe)
if err == nil {
var hdr http.Header
if hdrs, herr := p.buildHeaders(string(body), p.URL()); herr == nil {
if hdrs, herr := p.buildHeaders(string(body), p.URL(), ""); herr == nil {
hdr = hdrs
}
raw, status, derr := p.do(ctx, p.URL(), string(body), hdr)
@ -1013,13 +1031,15 @@ func (p *Provider) Throttle(ctx context.Context) error {
// ---- request construction ----
func (p *Provider) buildHeaders(body, url string) (http.Header, error) {
func (p *Provider) buildHeaders(body, url, clientSession string) (http.Header, error) {
meta := map[string]interface{}{
"url": url,
"method": http.MethodPost,
"body": body,
"api_key": p.cfg.APIKey,
"timestamp": types.Now(),
// client_session 是客户端自带的会话标识(请求头),为空时适配器自行推导。
"client_session": clientSession,
"source": map[string]interface{}{
"name": p.cfg.Name,
"meta": p.cfg.Meta,
@ -1064,7 +1084,7 @@ func (p *Provider) Chat(ctx context.Context, req *types.ChatRequest) (*types.Uni
if err != nil {
return nil, err
}
hdrs, err := p.buildHeaders(body, p.URL())
hdrs, err := p.buildHeaders(body, p.URL(), req.ClientSession)
if err != nil {
return nil, err
}
@ -1124,7 +1144,7 @@ func (p *Provider) ChatStream(ctx context.Context, req *types.ChatRequest) (<-ch
p.Release()
return nil, err
}
hdrs, err := p.buildHeaders(body, p.URL())
hdrs, err := p.buildHeaders(body, p.URL(), req.ClientSession)
if err != nil {
p.Release()
return nil, err
@ -1393,7 +1413,7 @@ func (p *Provider) Image(ctx context.Context, req *types.ImageGenRequest) (*type
// fall back to passthrough adapter (openai-style)
transformed = string(b)
}
hdrs, err := p.buildHeaders(transformed, p.ImageURL())
hdrs, err := p.buildHeaders(transformed, p.ImageURL(), "")
if err != nil {
return nil, err
}
@ -1498,12 +1518,24 @@ func standardSSEChunk(data string) string {
PromptTokensDetails *struct {
CachedTokens int `json:"cached_tokens"`
} `json:"prompt_tokens_details"`
CompletionTokensDetails *struct {
ReasoningTokens int `json:"reasoning_tokens"`
} `json:"completion_tokens_details"`
} `json:"usage"`
// Cost is a top-level decimal string on OpenCode Zen/Go responses.
Cost json.RawMessage `json:"cost"`
}
if err := json.Unmarshal([]byte(data), &raw); err != nil {
return ""
}
if len(raw.Choices) == 0 && raw.UpstreamUsage.Total == 0 && raw.UpstreamUsage.TotalTokens == 0 {
// Cost-only chunk: OpenCode sends the charge on its own frame
// {"choices":[],"cost":"0"} with no usage at all. Dropping it here
// would lose the only cost signal the client can see.
if c := rawCostString(raw.Cost); c != "" {
out, _ := json.Marshal(types.UnifiedChunk{Cost: c})
return string(out)
}
return ""
}
var usage *types.TokenUsage
@ -1521,6 +1553,11 @@ func standardSSEChunk(data string) string {
CachedTokens: pu.PromptTokensDetails.CachedTokens,
}
}
if pu.CompletionTokensDetails != nil {
usage.CompletionTokensDetails = &types.CompletionTokensDetails{
ReasoningTokens: pu.CompletionTokensDetails.ReasoningTokens,
}
}
}
finish := ""
done := false
@ -1545,6 +1582,24 @@ func standardSSEChunk(data string) string {
return string(out)
}
// rawCostString normalizes an upstream cost field into a plain string. The
// value is a decimal **string** on OpenCode Zen/Go ("0", "0.0012"), but other
// upstreams have been seen to send a bare number, so both are accepted.
// Returns "" when absent, null, or empty.
func rawCostString(raw json.RawMessage) string {
s := strings.TrimSpace(string(raw))
if s == "" || s == "null" {
return ""
}
if s[0] == '"' {
var unq string
if json.Unmarshal(raw, &unq) == nil {
return strings.TrimSpace(unq)
}
}
return s
}
// pickFirst returns a if non-zero, else b (for usage keys that may appear in
// either standard *_tokens or legacy short form).
func pickFirst(a, b int) int {

View File

@ -3,6 +3,7 @@
package types
import (
"bytes"
"encoding/json"
"errors"
"strings"
@ -28,6 +29,10 @@ type ChatRequest struct {
ToolChoice interface{} `json:"tool_choice,omitempty"`
DisableThinking bool `json:"disable_thinking"`
ExtraBody map[string]interface{} `json:"-"`
// ClientSession 是客户端自带的会话标识(请求头 x-session-affinity 等)。
// 不参与序列化:它只用于让适配器生成稳定的上游会话,不应泄漏给上游 body。
ClientSession string `json:"-"`
}
func (r *ChatRequest) MarshalJSON() ([]byte, error) {
@ -61,6 +66,39 @@ type ChatMessage struct {
func StringContent(s string) json.RawMessage { b, _ := json.Marshal(s); return b }
// isEmptyJSONArray reports whether raw is the literal empty JSON array [].
func isEmptyJSONArray(raw json.RawMessage) bool {
t := bytes.TrimSpace(raw)
return len(t) == 2 && t[0] == '[' && t[1] == ']'
}
// UnmarshalJSON decodes a chat message, normalising an empty-array content
// ("content":[]) to an empty string and dropping an empty tool_calls array.
//
// Why this exists: Lua adapters json.decode the request and re-encode it, and
// an empty Lua table is indistinguishable from an empty JSON array — the
// encoder emits {} for both. Agent clients serialise an assistant turn that
// carries tool_calls and no text as content:[], so every pass-through adapter
// turned it into content:{} — a shape that is not valid OpenAI (content is
// string | array of parts | null) and that real upstreams reject with
// "400 invalid arguments". Normalising at the decode boundary fixes every
// adapter at once, including ones added later.
func (m *ChatMessage) UnmarshalJSON(b []byte) error {
type alias ChatMessage
var a alias
if err := json.Unmarshal(b, &a); err != nil {
return err
}
*m = ChatMessage(a)
if isEmptyJSONArray(m.Content) {
m.Content = StringContent("")
}
if isEmptyJSONArray(m.ToolCalls) {
m.ToolCalls = nil
}
return nil
}
type ToolCall struct {
ID string `json:"id"`
Type string `json:"type"`
@ -79,6 +117,11 @@ type UnifiedResponse struct {
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
// ImageData used by image-generation adapters.
ImageData []ImageData `json:"image_data,omitempty"`
// Cost is the upstream-reported charge for this request, passed through
// verbatim (OpenCode Zen/Go put a decimal **string** at the response top
// level). Deliberately not parsed or summed by the gateway: it is an
// upstream fact, and only some upstreams report it at all.
Cost string `json:"cost,omitempty"`
}
type TokenUsage struct {
@ -95,6 +138,16 @@ type TokenUsage struct {
// prompt_tokens_details.cached_tokens is absent.
PromptCacheHit int `json:"prompt_cache_hit_tokens,omitempty"`
PromptCacheMiss int `json:"prompt_cache_miss_tokens,omitempty"`
// CompletionTokensDetails mirrors the OpenAI v2 usage.completion_tokens_details
// object. OpenCode Zen/Go report how much of the completion was thinking
// tokens here; without it clients cannot tell how much of the billed output
// was reasoning rather than visible answer.
CompletionTokensDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
}
// CompletionTokensDetails is the OpenAI v2 completion_tokens_details object.
type CompletionTokensDetails struct {
ReasoningTokens int `json:"reasoning_tokens"`
}
// PromptTokensDetails is the OpenAI v2 prompt_tokens_details object. Only
@ -119,15 +172,16 @@ func (t TokenUsage) MarshalJSON() ([]byte, error) {
pdetails = &PromptTokensDetails{CachedTokens: t.PromptCacheHit}
}
return json.Marshal(struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
Prompt int `json:"prompt"`
Completion int `json:"completion"`
Total int `json:"total"`
PromptTokensDetails *PromptTokensDetails `json:"prompt_tokens_details,omitempty"`
PromptCacheHitTokens int `json:"prompt_cache_hit_tokens,omitempty"`
PromptCacheMissTokens int `json:"prompt_cache_miss_tokens,omitempty"`
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
Prompt int `json:"prompt"`
Completion int `json:"completion"`
Total int `json:"total"`
PromptTokensDetails *PromptTokensDetails `json:"prompt_tokens_details,omitempty"`
PromptCacheHitTokens int `json:"prompt_cache_hit_tokens,omitempty"`
PromptCacheMissTokens int `json:"prompt_cache_miss_tokens,omitempty"`
CompletionDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
}{
PromptTokens: t.Prompt,
CompletionTokens: t.Completion,
@ -138,6 +192,7 @@ func (t TokenUsage) MarshalJSON() ([]byte, error) {
PromptTokensDetails: pdetails,
PromptCacheHitTokens: t.PromptCacheHit,
PromptCacheMissTokens: t.PromptCacheMiss,
CompletionDetails: t.CompletionTokensDetails,
})
}
@ -181,6 +236,9 @@ type UnifiedChunk struct {
// often has empty choices). Gateway uses it to emit exact usage in the
// final stream chunk instead of estimates.
Usage *TokenUsage `json:"usage,omitempty"`
// Cost is upstream-reported charge for this request (decimal string),
// passed through verbatim; OpenCode sends it on its own stream chunk.
Cost string `json:"cost,omitempty"`
}
// Meta passed to Lua build_headers hook

View File

@ -2,6 +2,7 @@ package types
import (
"encoding/json"
"strings"
"testing"
)
@ -68,3 +69,86 @@ func TestTokenUsageMarshalCacheFields(t *testing.T) {
t.Logf("DeepSeek JSON: %s", string(b))
t.Logf("OpenAI JSON: %s", string(b2))
}
// TestChatMessageNormalizesEmptyArrayContent pins the fix for the malformed
// content:{} that pass-through adapters produced for an assistant turn carrying
// tool_calls with no text. Lua adapters cannot tell an empty JSON array from an
// empty object, so the decode boundary normalises it for every adapter at once.
func TestChatMessageNormalizesEmptyArrayContent(t *testing.T) {
var msg ChatMessage
raw := `{"role":"assistant","content":[],"tool_calls":[{"id":"call_1","type":"function","function":{"name":"read","arguments":"{}"}}]}`
if err := json.Unmarshal([]byte(raw), &msg); err != nil {
t.Fatal(err)
}
if string(msg.Content) != `""` {
t.Fatalf("empty-array content must normalise to \"\", got %s", msg.Content)
}
if len(msg.ToolCalls) == 0 {
t.Fatal("tool_calls must survive normalisation")
}
// A non-empty content array must be left byte-identical (multimodal path).
var mm ChatMessage
multi := `{"role":"user","content":[{"type":"text","text":"hi"}]}`
if err := json.Unmarshal([]byte(multi), &mm); err != nil {
t.Fatal(err)
}
if string(mm.Content) != `[{"type":"text","text":"hi"}]` {
t.Fatalf("multimodal content must pass through untouched, got %s", mm.Content)
}
// A plain string content is untouched.
var sm ChatMessage
if err := json.Unmarshal([]byte(`{"role":"user","content":"plain"}`), &sm); err != nil {
t.Fatal(err)
}
if string(sm.Content) != `"plain"` {
t.Fatalf("string content must pass through untouched, got %s", sm.Content)
}
// An omitted content stays omitted (omitempty semantics preserved).
var om ChatMessage
if err := json.Unmarshal([]byte(`{"role":"assistant","tool_calls":[]}`), &om); err != nil {
t.Fatal(err)
}
if om.Content != nil {
t.Fatalf("absent content must stay absent, got %s", om.Content)
}
if om.ToolCalls != nil {
t.Fatalf("empty tool_calls array must be dropped, got %s", om.ToolCalls)
}
}
// 上游报告输出里有多少是思考 token;缺少这个字段,客户端无法判断
// completion_tokens 里多少是可见回答、多少是思考(两者都按输出计费)。
func TestTokenUsageMarshalsCompletionTokensDetails(t *testing.T) {
u := TokenUsage{
Prompt: 37, Completion: 40, Total: 77,
CompletionTokensDetails: &CompletionTokensDetails{ReasoningTokens: 40},
}
b, err := json.Marshal(u)
if err != nil {
t.Fatal(err)
}
var got map[string]interface{}
if err := json.Unmarshal(b, &got); err != nil {
t.Fatal(err)
}
ctd, ok := got["completion_tokens_details"].(map[string]interface{})
if !ok {
t.Fatalf("completion_tokens_details 未输出: %s", b)
}
if n, _ := ctd["reasoning_tokens"].(float64); int(n) != 40 {
t.Errorf("reasoning_tokens = %v, want 40 (%s)", ctd["reasoning_tokens"], b)
}
// 标准 OpenAI 字段同时存在,老客户端不受影响。
if _, ok := got["completion_tokens"]; !ok {
t.Errorf("标准 completion_tokens 丢失: %s", b)
}
// 未设置时不得凭空造字段。
b2, _ := json.Marshal(TokenUsage{Prompt: 1, Completion: 2, Total: 3})
if strings.Contains(string(b2), "completion_tokens_details") {
t.Errorf("无数据时不应输出 completion_tokens_details: %s", b2)
}
}