Compare commits

...

22 Commits

Author SHA1 Message Date
19c4fc3a13 chore(version): 1.5.4 2026-09-11 15:14:14 +08:00
71e9040a45 feat(adapters): split opencode into opencodezen and opencodego
Zen (https://opencode.ai/zen/v1) and Go (https://opencode.ai/zen/go/v1)
are different services with different requirements, and one shared adapter
could not satisfy both.

The decisive difference is reasoning_content:

  * OpenCode Go runs thinking models and REQUIRES the assistant turn's
    reasoning_content to be echoed back. The shared adapter stripped it
    (msg.reasoning_content = nil), so every replay of a thinking turn
    failed with:
      400 invalid_request_error: The `reasoning_content` in the thinking
      mode must be passed back to the API.
    Reproduced directly: the same request with reasoning_content -> 200,
    without -> 400. That is why the Go tier never worked in an agent loop.

  * The Zen free pool must not receive it, so it keeps stripping.

Both adapters keep the earlier fixes they share (never drop an assistant
turn carrying tool_calls; send stream_options only when streaming; role
whitelist; multimodal strip) and the opencode client fingerprint headers —
the Go endpoint additionally REQUIRES x-opencode-session, which the
adapter already sends.

config: localzen -> opencodezen, gozen -> opencodego.
Verified: all 25 gozen models answer correctly through the gateway with a
thinking + tool_call + tool_result history (was 0/25 before), streaming
included; the Zen free models still pass.

Test: TestOpenCodeGoVsZenReasoning pins the Go-keeps / Zen-strips split.
2026-09-11 15:00:45 +08:00
0528941e24 fix(opencode): only send stream_options with stream:true
OpenCode Go (and other strict OpenAI-compatible upstreams) reject a
non-streaming request that carries stream_options with
"stream_options should be set along with stream". The adapter attached it
unconditionally, so every non-stream call through the opencode adapter
failed on those upstreams.

Verified against OpenCode Go: 25/25 configured models now pass a real
completion through the gateway (they previously 400'd).

Test: TestOpenCodeStreamOptionsOnlyWhenStreaming (absent when
non-streaming, include_usage present when streaming).
2026-09-11 13:13:05 +08:00
12af22fda6 chore(version): 1.5.3 2026-09-10 21:57:58 +08:00
9114468753 fix: empty-array content becomes invalid {} on every pass-through adapter; gemini/ollama drop tool calls
Three related forwarding defects found by auditing every adapter with a
tool-calling replay (assistant turn with content:[] + tool_calls).

1) content:[] -> content:{} (all 12 openai-adapter sources, plus
   deepseek/trae/sensenova/agentrouter/github/groq/kimicode/mistral)

   Lua adapters json.decode the request and re-encode it, and an empty Lua
   table is indistinguishable from an empty JSON array — the encoder emits
   {} for both. Agent clients serialise a tool-calling assistant turn with
   no text as content:[], so every pass-through adapter rewrote it to
   content:{} — not valid OpenAI (content is string|array|null). Verified
   against a live upstream: content:[] produced "400 invalid arguments"
   while content:"" was accepted.

   Fixed once at the decode boundary (types.ChatMessage.UnmarshalJSON):
   empty-array content normalises to "" and an empty tool_calls array is
   dropped, so every adapter — including future ones — sees a valid shape.

2) gemini dropped tool_calls and never emitted functionCall /
   functionResponse; the tool role also stayed as an invalid role inside
   contents and system was not moved to systemInstruction.

3) ollama copied only role/content, dropping tool_calls and the call
   attribution entirely (it needs tool_name, not tool_call_id).

Test: TestAdaptersPreserveToolCalls asserts, for every adapter, that the
call id (or function name where the wire format has no id), the function
name, the tool result and the trailing user turn all survive, plus a
negative control for plain text.
2026-09-10 21:57:54 +08:00
611e975456 fix(gateway): 上下文超窗错误归一化,客户端才能压缩重试
问题:上游返回上下文超窗时,客户端(pi)既不压缩也不重试,只看到一条
普通上游错误。链路两处叠加:

1. 措辞不在客户端识别列表里。pi 靠 @earendil-works/pi-ai 的
   OVERFLOW_PATTERNS(25 条正则)判断超窗,而 justworker 返回的是
   「请精简对话历史…(Context window is full…)」——与那 25 条一条都不匹配
   (最接近的 /context window exceeds limit/i 也不命中,因为措辞是 "is full"
   而非 "exceeds limit")。
2. AUTO 链路的每条 tier 错误按 80 字节 OneLine 截断,而 "Context window
   is full" 这类短诊断词常出现在尾部,正好被按字节切掉(直连路径是 160
   才保住)。

修法:
- 新增 overflowMarkers 识别超窗措辞(含中文写法),命中时把客户端可见
  错误归一化为 context_length_exceeded 前缀——它命中 pi 的
  /context[_ ]length[_ ]exceeded/i,超窗因此可被发现并触发压缩重试。
- AUTO 链路每条 tier 错误宽度 80→160,短诊断词不再被截断。

归一化只加前缀,原始诊断信息保留,便于定位是哪一层超窗。

测试:internal/gateway/overflow_err_test.go(5 例,含「标记必须命中 pi
正则」、非超窗不得误标、80 vs 160 宽度的回归对比)。
2026-09-10 20:38:34 +08:00
6733534282 chore(version): 1.5.2 2026-09-10 19:45:31 +08:00
499f0cac2f fix(opencode): never drop an assistant turn that carries tool_calls
An agent client (pi) serialises an assistant turn whose content is only
[thinking, toolCall] as content:[] with tool_calls. The multimodal-strip
pass treated an empty content array as 'nothing left, drop the whole
message' and discarded the tool_calls with it.

The next message is that call's tool result, so it arrived orphaned: the
model saw a result for a call it had never made and re-issued the same
call on every turn — an endless repeated-tool-call loop. Reproduced
against a capture sink: content:[] lost tool_calls, while content:"" and
content:null kept them.

Only messages with neither usable content NOR a tool call now get
dropped. Content that collapses to empty but still has tool_calls or a
tool_call_id is emitted as "" instead.

Test: TestOpenCodeKeepsToolCallWithEmptyContent (plus a negative control
that an image-only message without tool calls is still dropped).
2026-09-10 19:32:47 +08:00
7fb8f96b82 fix(gateway): AUTO scope grants all models — restrict to routing mode only
A key with scope=[AUTO] could previously:
1. request ANY concrete model id directly (hasScopeModel/checkModelScope
   treated AUTO as a wildcard)
2. see the full 56-model list on /v1/models (intersectModels considered
   AUTO as grant-everything)

AUTO now only authorizes the AUTO routing mode. Direct requests to a
specific model require an explicit scope entry.

Also carries agentrouter.lua WAF fingerprint headers (Origin/Referer/
X-Requested-With) already staged on this branch.

Tests: TestHasScopeModelWithSourcePrefix updated; full suite green.
2026-09-10 12:51:40 +08:00
6a34cd6f86 chore(version): 1.4.2 -> 1.5.0 2026-09-06 10:17:43 +08:00
f0986751de Merge feature/agentrouter-id-sanitize: 补上最后一个服务 Claude 的适配器
全仓审计发现 agentrouter 也暴露 claude-opus-4-8,却是唯一没有 id 消毒的
Claude 适配器。补齐后,所有含 claude/opus/sonnet 模型的源(qijiar、toter、
juziai、agentrouter、justwoker、api456、tabitoken)全部走消毒路径;
测试新增三适配器结果一致性断言。
2026-09-06 10:08:52 +08:00
c6c3e0dcd7 fix(agentrouter): sanitize tool-call ids — it fronts Claude too
Full-repo audit after the anthropic/openai fix: agentrouter exposes
claude-opus-4-8, so it inherits Anthropic's tool id rule
^[a-zA-Z0-9_-]{1,64}$ and rejects the whole request on a violation, exactly
like justwoker/tabitoken/扇贝. It was the only remaining adapter serving Claude
models without the sanitizer, so a client that had picked up a dirty id (e.g.
"bash:0" from moonshotai/kimi-k3) would still lose every turn here.

Same shape as the other two — inbound tool_calls[].id + tool_call_id, outbound
non-streaming ids and the first streamed fragment — and the test now asserts
all three adapters rewrite an identical input identically, so a client mixing
sources within one session cannot end up with unpaired tool calls.

Audit result: every source exposing a claude/opus/sonnet model (qijiar, toter,
juziai, agentrouter, justwoker, api456, tabitoken) now routes through a
sanitizing adapter.
2026-09-06 10:08:52 +08:00
b9944332ff Merge feature/anthropic-usage-cache: 缓存命中不该让 prompt_tokens 算错
Anthropic 的 input_tokens 不含缓存部分,OpenAI 的 prompt_tokens 含。
直接对映射等于漏算全部缓存 token,还能算出 >100% 的命中率;
cache_creation 完全没读;流式路径连缓存字段都整个丢掉。
三项相加 + 共享 map_usage,未上报缓存的上游输出逐字节不变。
2026-09-06 09:55:22 +08:00
6bdb9fcc44 fix(anthropic): count cached input in prompt_tokens instead of dropping it
Anthropic and OpenAI disagree on what the prompt count means:

  Anthropic: input_tokens EXCLUDES cached blocks; cache_read_input_tokens and
             cache_creation_input_tokens are separate, additive, billed input.
  OpenAI:    prompt_tokens INCLUDES its cached_tokens subset.

anthropic.lua mapped input_tokens straight onto prompt, so a cache-heavy turn
was doubly wrong: the billed prompt was undercounted by the entire cache
portion, and cached_tokens could exceed prompt_tokens — a cache hit rate above
100% for any client that divides one by the other. cache_creation_input_tokens
was never read at all, so a cache-write turn silently lost those billed tokens.

Worse, the streaming path dropped the cache split entirely: message_delta
carries the FINAL usage and only mapped input/output, so every streamed
response reported no cache information even when the upstream sent it.

All three counts are now summed into prompt, with the read half exposed as
prompt_tokens_details.cached_tokens plus the DeepSeek-legacy hit/miss pair, via
one shared map_usage() used by transform_response, message_start and
message_delta. A reported zero stays distinguishable from "never reported": the
split is emitted whenever either cache field is present, and omitted entirely
when the upstream mentions neither (justwoker reports only input/output plus its
own cost fields, so its output is byte-identical to before). map_usage returns
nil for a countless object, preserving "no usage in this chunk means say
nothing" rather than reporting zeros.

message_start's placeholder count is still emitted: justwoker reports 160 there
and the real 6931 in message_delta, and the gateway's mergeUsage lets the later
non-zero value win.
2026-09-06 09:55:22 +08:00
25eb30c15b Merge feature/toolcall-id-sanitize: 一个上游的脏 tool_call id 不该拖垮整条 Claude 链
xinjianya/moonshotai/kimi-k3 会吐出 'bash:0' 这种 id。OpenAI 不校验,
Anthropic 校验 ^[a-zA-Z0-9_-]{1,64}$ 且整条请求直接 400。客户端会把它存进
历史并回放给所有源,于是 justwoker / tabitoken / 扇贝(都是 Claude 上游)
同时挂掉,AUTO 一路穿透四个 tier。

两个适配器、进出双向消毒;已合法的 id 逐字节透传;流式参数分片保持无 id。
2026-09-05 22:19:15 +08:00
131a42a169 fix(adapters): sanitize tool-call ids so one bad upstream can't kill every Claude slot
Anthropic requires tool_use.id / tool_result.tool_use_id to match
^[a-zA-Z0-9_-]{1,64}$ and rejects the WHOLE request otherwise with
REQUEST_BODY_INVALID / "Invalid tool use format". OpenAI has no such rule, so
an OpenAI-compatible model can mint an id like "bash:0"
(xinjianya/moonshotai/kimi-k3 does exactly that).

In a fan-out router that id does not stay local: the client stores it in its
history and replays it to every other source. One such id therefore kills
every Claude slot at once — justwoker, tabitoken and 扇贝 are all
Claude-behind-{OpenAI,Anthropic} — and an AUTO request falls through all four
tiers to whatever tolerant model is left. Observed live: 4 consecutive 503
"all N auto providers failed" with tier 1/2/3 each reporting the same 400.

Both directions are sanitized, in both adapters:
  - request:  tool_calls[].id and tool_call_id, so poisoned history recovers
  - response: non-streaming tool_calls[].id and the first streamed fragment,
              so a bad id never enters a client session in the first place

safe_tool_id is pure and deterministic, so a call and its result are rewritten
identically within one request. A rewritten id keeps an 8-hex digest of the
original, without which distinct ids could collapse ("a:b" and "a_b") into a
duplicate/unpaired tool_use. Already-legal ids pass through byte-identical, so
well-behaved traffic is unaffected. openai.lua carries its own copy because
Lua adapters have no shared prelude.

Streamed argument fragments carry no id and must stay id-less, otherwise
index-based accumulation on the client breaks; a test pins that.
2026-09-05 22:18:31 +08:00
3cdb16c906 Merge feature/deploy-config-rollback: 部署脚本把配置纳入回滚闭环
上一轮 446 次重启循环的根因不是代码,是 deploy.sh 把 config.yaml 当作
部署范围之外的东西:健康检查失败后只回滚二进制,坏配置仍在,于是旧
二进制照样解析不了,服务在 systemd 重启循环里空转。

修法是把配置变成一等部署对象:--config 投放 + 重启前 -check 预检 +
回滚时二进制与配置一起恢复。预检位置比预检本身更重要——排在 restart
之前,坏配置才会在服务仍健康时被拦下。
2026-09-05 09:47:04 +08:00
ef92ce4d82 Merge feature/source-proxy: 让被 CF/格式问题挡住的源真正可用
两件事都在回答同一个问题——为什么配了源却用不了:

- trae 源在流式下把模型打印的 tool_call 当纯文本透传,客户端收到
  finish_reason "stop" + 一段文本,agent 循环当场终止
- justwoker/tabitoken 坐在 Cloudflare 后面,直连 403,必须走代理 +
  浏览器 UA;但全局 env 代理会把 trae/localzen 等内网源也绕出去

因此引入 per-source proxy_url:指定的源走专属代理,未指定的保持直连。
2026-09-05 09:46:54 +08:00
c524831616 fix(deploy): roll back config together with the binary, and preflight it before restart
The 446-restart-loop incident: adding a headers block to a source without
removing the source's existing `headers: {}` produced a duplicate YAML key.
The process exited on startup, healthcheck failed, and rollback restored only
the binary — so the old binary kept parsing the same broken config and the
service span in systemd's restart loop. Config was treated as out of scope
for deployment; it is not.

Three changes close the loop:

1. cmd/llmsproxy: new `-check` flag validates a config (parse +
   ApplyDefaults) and exits, without starting the Lua VM, touching
   runtime.json, or binding a port — safe to run against a live service.
   Unlike normal startup it does NOT create a default config, so a missing
   file is an error.

2. deploy.sh `--config <file>`: stage a config for deployment, atomically
   renamed into place with the same copy->rename(2) technique as the binary.
   Omitted means the live config is left alone.

3. Ordering: config replacement and preflight both run BEFORE
   restart_service, so an invalid config is caught while the service is still
   healthy and never triggers a restart. rollback() now restores binary AND
   config (only when this run replaced it, so concurrent WebUI edits survive),
   then re-runs -check before restarting — refusing to restart into a config
   that still fails, instead of trading one restart storm for another.

Also: the sha256-unchanged early exit now only fires when there is no pending
config, otherwise `--config` would be silently dropped.

Verified on the live deployment:
- reproduced the exact duplicate-key config: preflight caught it, PID
  unchanged (zero interruption), binary and config both rolled back, gateway
  still answering 200
- valid config: replaced, service restarted, new value live
- no --config: binary-only deploy unaffected
- go test -tags luajit ./... passes
2026-09-05 09:46:43 +08:00
868ac59692 feat(provider): support per-source proxy_url for geo-blocked upstreams
Upstreams like justwoker/tabitoken sit behind Cloudflare geo/IP blocks and
only respond through a proxy. A global env proxy is wrong (intranet sources
trae/localzen must stay direct), so add an explicit per-source proxy_url
that overrides http.ProxyFromEnvironment for just that source. Sources
without proxy_url keep the existing env/direct behavior.

Also accepts a User-Agent header per source (config already supported
headers) so CF-fronted resellers can be reached.

Verified e2e: tabitoken and justwoker now return tool_calls through
127.0.0.1:7890 (clash); trae stays direct. Full go test -tags luajit
passes.
2026-09-05 09:46:36 +08:00
4d197e4bd3 fix(trae): recover legacy [Called tool:...] tool call format
Root cause: trae-local-api is deployed to users without the "Fold past
assistant tool_calls into [Called tool: name({...})]" text format that
their own histories already contained. Trae-Local-API-LLM then mimics this
format in subsequent responses. The trae.lua parser only recognized
<tool_call>...</tool_call> or <toolcall>...</toolcall> tags, so
[Called tool: ...] responses were left unparsed and the client received
plain text where a structured tool_calls array should be.

Fix: Add a legacy pattern match at the END of parse_text_tool_calls to
catch the [Called tool: name({args})] shape and emit proper tool_calls.
This is a fallback; models should emit <tool_call> tags per system prompt,
but we tolerate the mimicked form for robustness.
2026-09-05 09:46:36 +08:00
3806aaee03 docs(workflow): codify the branch model — main / feature / release branches
Adopt GitHub Flow + release branches, replacing "everything straight to main
plus a tag" which caused the 1.4.2 pain (a fix had to be retro-fitted to the
released version, forcing a remote-tag delete + full re-upload).

- main: only long-lived branch, always deployable, accumulates the next version
- feature/<desc>: born from main, merged back when done
- release/vX.Y.Z: cut from main, tagged, installers built from the tag
- hotfixes land on the release branch AND are cherry-picked back to main so
  main never loses a fix
- end of lifecycle = retire the release branch (delete; or keep for long-term
  maintenance), no wholesale merge back — hotfixes already flowed
- explicitly no rebase of main, no release-branch-merge, no quick edits on main

Companion release checklist includes the upload lessons (PUT --http1.1) and the
replace-artifacts-by-deleting-the-tag catch.

Docs in docs/git-workflow.md (zh) and docs/git-workflow-en.md, linked from both
READMEs.
2026-08-31 12:36:17 +08:00
26 changed files with 2330 additions and 129 deletions

View File

@ -490,6 +490,15 @@ internal/gateway # OpenAI 兼容 HTTP 服务 + 鉴权 + SDK/流式 + Web
internal/types # 统一格式 & OpenAI wire 类型
```
## 开发协作
- **Git 分支工作流**:主分支 / 特性分支 / 发布分支的规范见
**[docs/git-workflow.md](docs/git-workflow.md)**([English](docs/git-workflow-en.md))。
主线:`main` 永可部署 → 特性合入 `main` → 切 `release/vX.Y.Z` 打 tag 发布 →
hotfix 提交发布分支并 cherry-pick 回 `main`。
- **Lua 适配器协议**:见 **[docs/lua-adapters.md](docs/lua-adapters.md)**
([English](docs/lua-adapters-en.md))。
## 测试
```bash

View File

@ -507,6 +507,17 @@ internal/gateway # OpenAI-compatible HTTP + auth + SDK/streaming + WebUI
internal/types # unified format & OpenAI wire types
```
## Development & collaboration
- **Git branching workflow**: main / feature / release-branch conventions in
**[docs/git-workflow-en.md](docs/git-workflow-en.md)**
([中文](docs/git-workflow.md)). The line: `main` is always deployable →
features merge into `main` → cut `release/vX.Y.Z` and tag → release → hotfixes
land on the release branch and are cherry-picked back into `main`.
- **Lua adapter protocol**: see
**[docs/lua-adapters-en.md](docs/lua-adapters-en.md)**
([中文](docs/lua-adapters.md)).
## Tests
```bash

View File

@ -1,12 +1,12 @@
{
"name": "modelrouter-gui",
"version": "1.4.2",
"version": "1.5.4",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "modelrouter-gui",
"version": "1.4.2",
"version": "1.5.4",
"license": "Proprietary",
"devDependencies": {
"electron": "^33.0.0",

View File

@ -1,6 +1,6 @@
{
"name": "modelrouter-gui",
"version": "1.4.2",
"version": "1.5.4",
"description": "ModelRouter Desktop — embedded ModelRouter core with tray",
"author": "ModelRouter",
"main": "main.js",

View File

@ -23,8 +23,24 @@ import (
func main() {
cfgPath := flag.String("config", "config.yaml", "path to gateway config file")
checkOnly := flag.Bool("check", false, "validate the config file and exit (0 = valid, 1 = invalid); nothing is started and no file is written")
flag.Parse()
// -check is the deploy-time preflight: parse and validate the config
// without starting the Lua VM, touching runtime.json, or binding a port.
// It deliberately does NOT call EnsureDefault, so a missing file is an
// error here instead of being silently created.
if *checkOnly {
if _, err := os.Stat(*cfgPath); err != nil {
log.Fatalf("[llmsproxy] check: %v", err)
}
if _, err := config.Load(*cfgPath); err != nil {
log.Fatalf("[llmsproxy] check: %v", err)
}
log.Printf("[llmsproxy] check: %s is valid", *cfgPath)
return
}
created, err := config.EnsureDefault(*cfgPath)
if err != nil {
log.Fatalf("[llmsproxy] config: %v", err)

173
deploy.sh
View File

@ -10,6 +10,16 @@
# 4. systemctl restart → 服务中断仅在 Go 进程重启瞬间 (<500ms)
# 5. 健康检查确认新进程正常
#
# 配置文件(2026-09-05 起):
# 本脚本不再假设 config.yaml 与部署无关。实测教训:给源加 headers 时
# 漏删原有的 `headers: {}` → YAML 重复键 → 进程启动即退出 → 健康检查
# 失败 → 旧的 rollback 只恢复二进制,配置错误仍在,于是旧二进制也起不来,
# 服务陷入 systemd 重启循环(当时累计 446 次)。
# 因此:
# - 支持 --config <file> 投放一份待部署配置(原子 rename 到 TARGET_CONFIG)
# - 重启前先用新二进制 `-check` 预检配置,不合法就地失败,绝不重启服务
# - rollback 同时恢复二进制与配置文件,回滚后再次 -check 才重启
#
# 注意事项:
# - 本服务依赖自身(推理依赖),restart 前后会短暂中断,
# 但重试机制(systemd RestartSec=5 + 客户端 retry)可自愈
@ -19,6 +29,7 @@ set -euo pipefail
# ---------- 配置 ----------
readonly TARGET_BIN="/usr/local/bin/llmsproxy"
readonly TARGET_CONFIG="/etc/llmsproxy/config.yaml"
readonly TARGET_ADAPTERS="/etc/llmsproxy/adapters"
readonly REPO_DIR="/home/program/llmsproxy"
readonly HEALTH_URL="http://127.0.0.1:8081/v1/models"
@ -28,8 +39,40 @@ readonly BIN_NAME="llmsproxy"
readonly TMP_DIR="/tmp/llmsproxy-deploy"
# 同文件系统暂存文件(用于原子 rename)
readonly STAGING_BIN="${TARGET_BIN}.staging"
# 回滚备份:健康检查失败时从这里恢复旧二进制
readonly STAGING_CONFIG="${TARGET_CONFIG}.staging"
# 回滚备份:健康检查失败时从这里恢复旧二进制 / 旧配置
readonly BACKUP_BIN="${TARGET_BIN}.bak"
readonly BACKUP_CONFIG="${TARGET_CONFIG}.rollback.bak"
# 待部署配置(--config <file> 指定;为空 = 不改动线上配置)
NEW_CONFIG_SRC=""
usage() {
cat <<'EOF'
用法: ./deploy.sh [--config <file>]
--config <file> 投放这份配置到 /etc/llmsproxy/config.yaml(原子替换)。
省略时不改动线上配置,只部署二进制与适配器。
无论是否指定,重启前都会用新二进制 -check 预检线上配置。
流程: 构建 → 适配器同步 → 备份(二进制+配置) → 原子替换 → 配置预检
→ 重启 → 健康检查 → 失败则回滚(二进制+配置)并重启
EOF
}
parse_args() {
while (( $# > 0 )); do
case "$1" in
--config)
[[ $# -ge 2 ]] || { echo "--config 需要一个文件参数" >&2; exit 2; }
NEW_CONFIG_SRC="$2"
shift 2
;;
-h|--help) usage; exit 0 ;;
*) echo "未知参数: $1" >&2; usage; exit 2 ;;
esac
done
}
# ---------- 日志 ----------
log() { printf '\033[1;34m[%s]\033[0m %s\n' "$(date +%H:%M:%S)" "$*"; }
@ -40,7 +83,7 @@ fail() { err "$@"; exit 1; }
# ---------- 清理 ----------
cleanup() {
log "清理临时文件"
rm -rf "$TMP_DIR" "$STAGING_BIN"
rm -rf "$TMP_DIR" "$STAGING_BIN" "$STAGING_CONFIG"
}
trap cleanup EXIT
@ -52,6 +95,18 @@ precheck() {
command -v systemctl &>/dev/null || fail "未找到 systemctl"
[[ $(id -u) -eq 0 ]] || fail "需要 root 权限"
if [[ -n "$NEW_CONFIG_SRC" ]]; then
[[ -f "$NEW_CONFIG_SRC" ]] || fail "待部署配置不存在: $NEW_CONFIG_SRC"
# 同一文件比对:--config 指向线上文件本身时没有"投放"动作可做,
# 但仍然会走重启前的 -check 预检。
if [[ "$(readlink -f "$NEW_CONFIG_SRC")" == "$(readlink -f "$TARGET_CONFIG" 2>/dev/null)" ]]; then
warn "--config 指向线上配置本身,跳过投放(仍会预检)"
NEW_CONFIG_SRC=""
else
log "待部署配置: $NEW_CONFIG_SRC → $TARGET_CONFIG"
fi
fi
OLD_SIZE=$(stat -c%s "$TARGET_BIN" 2>/dev/null || echo 0)
OLD_SHA256=$(sha256sum "$TARGET_BIN" 2>/dev/null | awk '{print $1}')
@ -59,8 +114,10 @@ precheck() {
log "go: $(go version)"
}
# ---------- 备份旧二进制(用于回滚)----------
backup_old_binary() {
# ---------- 备份旧二进制 + 旧配置(用于回滚)----------
# 配置也必须备份:启动失败常常是配置问题,只回滚二进制会让旧二进制
# 继续吃坏配置,服务照样起不来。
backup_old_state() {
log "备份旧二进制到 $BACKUP_BIN"
if [[ -f "$TARGET_BIN" ]]; then
cp -f "$TARGET_BIN" "$BACKUP_BIN"
@ -68,6 +125,13 @@ backup_old_binary() {
else
warn "旧二进制不存在,跳过备份(无回滚点)"
fi
if [[ -f "$TARGET_CONFIG" ]]; then
cp -f "$TARGET_CONFIG" "$BACKUP_CONFIG"
log "✓ 旧配置已备份到 $BACKUP_CONFIG (sha256=$(sha256sum "$BACKUP_CONFIG" | awk '{print $1}' | cut -c1-16))"
else
warn "线上配置不存在: $TARGET_CONFIG(首次部署?无配置回滚点)"
fi
}
# ---------- 构建 ----------
@ -89,8 +153,13 @@ build() {
if [[ "$NEW_SIZE" -eq "$OLD_SIZE" ]]; then
NEW_SHA256=$(sha256sum "$NEW_BIN" | awk '{print $1}')
if [[ "$NEW_SHA256" == "$OLD_SHA256" ]]; then
warn "新二进制与旧版本完全一致(sha256 相同),跳过部署"
exit 0
# 二进制无变化。只有在同时也没有待部署配置时才能直接退出;
# 否则会造成“--config 指定了新配置却被静默丢弃”。
if [[ -z "$NEW_CONFIG_SRC" ]]; then
warn "新二进制与旧版本完全一致(sha256 相同)且无待部署配置,跳过部署"
exit 0
fi
log "二进制无变化(sha256 相同),但有待部署配置,继续"
fi
fi
@ -140,6 +209,41 @@ atomic_replace_binary() {
log "✓ 二进制已原子替换 (sha256=${NEW_SHA256:0:16})"
}
# ---------- 原子替换配置 ----------
# 与二进制同一套 copy→rename 手法:先落到 /etc 上的 .staging,再 rename(2)。
# 只有 --config 指定了来源时才动线上配置。
atomic_replace_config() {
[[ -n "$NEW_CONFIG_SRC" ]] || return 0
log "原子替换配置"
log " copy → $STAGING_CONFIG (同文件系统暂存)"
cp -f "$NEW_CONFIG_SRC" "$STAGING_CONFIG"
chmod 0644 "$STAGING_CONFIG"
log " rename → $TARGET_CONFIG (原子)"
mv -f "$STAGING_CONFIG" "$TARGET_CONFIG"
log "✓ 配置已原子替换 (sha256=$(sha256sum "$TARGET_CONFIG" | awk '{print $1}' | cut -c1-16))"
}
# ---------- 配置预检 ----------
# 用刚部署的新二进制解析线上配置。`-check` 只做 parse + ApplyDefaults,
# 不启动 Lua VM、不碰 runtime.json、不绑端口,因此可以安全地在服务仍在
# 运行时执行。不合法就在重启前失败——这正是上次事故缺的那一步。
verify_config() {
log "配置预检(新二进制 -check)"
[[ -f "$TARGET_CONFIG" ]] || fail "线上配置不存在: $TARGET_CONFIG"
local out
if out=$("$TARGET_BIN" -check -config "$TARGET_CONFIG" 2>&1); then
log "✓ 配置合法"
return 0
fi
err "配置预检失败,未重启服务(线上进程仍在跑旧配置):"
printf '%s\n' "$out" >&2
return 1
}
# ---------- 同步适配器 ----------
sync_adapters() {
log "同步适配器"
@ -206,20 +310,49 @@ healthcheck() {
}
# ---------- 回滚 ----------
# 健康检查失败时恢复旧二进制并重启,保证推理服务可用。
# 健康检查失败时恢复旧二进制**和旧配置**并重启,保证推理服务可用。
# 这是最后一道保险:即使新版本启动失败(编译错误、配置不兼容、
# panic on start 等),也能让旧版本继续服务,避免推理能力丢失。
#
# 为什么必须一并回滚配置:启动失败最常见的原因就是配置本身不合法
# (实测:YAML 重复键)。只恢复二进制的话,旧二进制照样解析不了坏配置,
# 服务继续在 systemd 重启循环里空转。
rollback() {
err "健康检查失败,启动回滚流程"
if [[ ! -f "$BACKUP_BIN" ]]; then
err "无回滚备份 ($BACKUP_BIN 不存在),无法自动恢复"
local restored_any=0
if [[ -f "$BACKUP_BIN" ]]; then
log "恢复旧二进制"
cp -f "$BACKUP_BIN" "$TARGET_BIN"
chmod 0755 "$TARGET_BIN"
restored_any=1
else
err "无二进制回滚备份 ($BACKUP_BIN 不存在)"
fi
# 只在本次真的换过配置时才回滚配置,避免把用户在部署期间
# 通过 WebUI 做的合法改动一起抹掉。
if [[ -n "$NEW_CONFIG_SRC" && -f "$BACKUP_CONFIG" ]]; then
log "恢复旧配置"
cp -f "$BACKUP_CONFIG" "$TARGET_CONFIG"
chmod 0644 "$TARGET_CONFIG"
restored_any=1
fi
if (( restored_any == 0 )); then
err "无任何回滚点,无法自动恢复"
err "请手动检查: systemctl status llmsproxy.service && journalctl -u llmsproxy -n 50"
return 1
fi
log "恢复旧二进制"
cp -f "$BACKUP_BIN" "$TARGET_BIN"
chmod 0755 "$TARGET_BIN"
# 回滚后先验证恢复出来的组合真的能解析,再重启——否则只是把
# 重启循环换个二进制继续跑。
if ! "$TARGET_BIN" -check -config "$TARGET_CONFIG" >/dev/null 2>&1; then
err "回滚后的配置仍不合法,拒绝重启(避免 systemd 重启风暴)"
err " 手动修复 $TARGET_CONFIG 后执行: $TARGET_BIN -check -config $TARGET_CONFIG"
err " 备份可用: 二进制=$BACKUP_BIN 配置=$BACKUP_CONFIG"
return 1
fi
log "重启服务(旧版本)"
systemctl restart llmsproxy.service || {
@ -248,18 +381,32 @@ print_summary() {
log "═════════════════════════════════════════════"
log "部署完成"
log " 二进制: $TARGET_BIN"
log " 配置: $TARGET_CONFIG$([[ -n "$NEW_CONFIG_SRC" ]] && echo " (本次已替换)" || echo " (未改动)")"
log " 适配器: $TARGET_ADAPTERS"
log "═════════════════════════════════════════════"
}
# ---------- 主流程 ----------
# 顺序要点:配置投放与预检都排在 restart_service **之前**,
# 于是坏配置在服务仍然健康时就被拦下,根本不会触发重启。
main() {
parse_args "$@"
log "═══ llmsproxy 原子部署开始 ═══"
precheck
build
sync_adapters
backup_old_binary
backup_old_state
atomic_replace_binary
atomic_replace_config
if ! verify_config; then
# 配置不合法:服务还没重启,只需把二进制/配置退回原状。
err "配置预检失败,回退本次改动(服务未受影响)"
[[ -f "$BACKUP_BIN" ]] && cp -f "$BACKUP_BIN" "$TARGET_BIN" && chmod 0755 "$TARGET_BIN"
if [[ -n "$NEW_CONFIG_SRC" && -f "$BACKUP_CONFIG" ]]; then
cp -f "$BACKUP_CONFIG" "$TARGET_CONFIG" && chmod 0644 "$TARGET_CONFIG"
fi
exit 1
fi
restart_service
if ! healthcheck; then
rollback || true

141
docs/git-workflow-en.md Normal file
View File

@ -0,0 +1,141 @@
# Git Branching Workflow
ModelRouter uses a **GitHub Flow + release-branch** model: `main` is the only
long-lived branch and is always deployable; new work branches off it as a
feature branch; each version gets its own release branch with a tag; hotfixes
land on the release branch and MUST flow back to `main`.
Goal: make "patching an already-released version during its support window" a
first-class operation, while `main` always contains every fix and stays
deployable.
## Branch roles
| Branch | Name | Lifetime | Purpose |
|---|---|---|---|
| Main | `main` | permanent | Only long-lived branch. Always deployable. Accumulates the next version. |
| Feature | `feature/<desc>` | short (dev → merge → delete) | New features / ordinary fixes. Born from `main`, merged back into `main`. |
| Release | `release/vX.Y.Z` | one version cycle | Cut from `main`, tagged for release. Version-specific hotfixes land here. |
## Change flow (important)
```
feature/foo ───────┐
▼
main ──────────────── next version ──────►
│ │
│ cut │ cut
▼ ▼
release/v1.4.2 release/v1.4.3
│ │
tag: v1.4.2 tag: v1.4.3
│ │
hotfix ◄─────┘ hotfix ◄─────┘
│ │
└── cherry-pick back ──────────┘
```
### Key rules
1. **main is always deployable**: never leave half-done work on `main`.
Unfinished work lives on a feature branch.
2. **Features start from main and merge back**: `git checkout -b feature/xxx main`,
then `git merge --no-ff feature/xxx` (or squash) when done.
3. **Release = cut a release branch from main + tag**:
```bash
git checkout -b release/v1.4.2 main
git tag -a v1.4.2 -m "ModelRouter v1.4.2"
git push origin release/v1.4.2 v1.4.2
```
Build installers and upload the GitCode Release from this tag so the
published state is exactly reproducible.
4. **Only version-specific hotfixes go into a release branch during its
window**: new features always go to `main` for the next version, never into
an already-released branch (unless you deliberately ship a minor revision).
5. **Hotfixes MUST flow back to main**:
```bash
git checkout release/v1.4.2 # fix in the release branch
git commit -m "fix: ..."
git checkout main
git cherry-pick <hotfix-commit> # and into main
```
Most important rule: **main only moves forward and never loses fixes**. If a
hotfix never reaches main, the next release ships with the old bug.
### End of a version lifecycle
When the next version ships, the previous release branch retires:
- **Default: delete the remote release branch**
(`git push origin :release/v1.4.2`). All hotfixes were already
cherry-picked into main, so main contains everything; no merge needed.
- **Long-term maintenance** (e.g. an enterprise client pinned to an old
version): keep the branch, accept only security fixes, keep the
commit-then-cherry-pick loop.
## Explicit non-goals
- **Never rebase main**: main's history stays append-only; anyone pulling gets
directly usable history.
- **Release branches are not merged back wholesale**: hotfixes were already
cherry-picked; a whole-branch merge only creates conflicts with no benefit.
- **No "quick edit" directly on main**, even single-person: at minimum use
`feature/xxx → merge main` so history keeps its why-boundaries.
## Single vs multi person
- **Single** (this repo today): skip PRs — `git checkout -b feature/xxx` →
finish → merge back to main. Release branch + tag flow unchanged.
- **Multi / open source**: feature branches go through PR + Code Review,
maintainers merge; resolve conflicts on the feature branch, never on main.
## Commit message conventions
- `feat(...)`: new feature
- `fix(...)`: bug fix
- `chore(...)`: build / version / deps / non-code changes
- `docs(...)`: documentation
- `refactor(...)`: refactoring
- Scope in parens, e.g. `fix(adapters)`, `feat(webui)`, `chore(version)`
- One-line summary; expand with problem/root-cause/fix/verification when needed
## Release checklist (companion)
```bash
# 1. main is ready
git checkout main && git pull
# 2. cut release branch + tag
git checkout -b release/vX.Y.Z main
git tag -a vX.Y.Z -m "ModelRouter vX.Y.Z"
# 3. push branch + tag
git push origin release/vX.Y.Z
git push origin vX.Y.Z
# 4. build installers (core + GUI platforms)
make core-dist VERSION=X.Y.Z
# 5. create GitCode Release + upload
# https://gitcode.com/JianFeeeee/ModelRouter/releases
# upload MUST use PUT --http1.1, otherwise the OBS callback fails and the
# file never lands in the release; replacing artifacts = delete the tag
# (release disappears with it) then recreate.
```
## Why this model (background)
Before 2026-08 everything went straight to `main` plus a tag, which caused two
pain points:
1. After 1.4.2 shipped, a scheduler-scoring fix had to be added to the same
version — the only option was deleting the remote tag to clear the release
and re-uploading all installers. The published state was detached from code
history; "what exactly shipped" could not be traced.
2. No feature branches meant two independent efforts could not proceed in
parallel without colliding.
With release branches: the published state = `release/vX.Y.Z` branch +
`vX.Y.Z` tag, exactly reproducible; hotfixes have a clear landing spot; main
stays "latest + all fixes + deployable".
> 中文版见 [docs/git-workflow.md](git-workflow.md)。

128
docs/git-workflow.md Normal file
View File

@ -0,0 +1,128 @@
# Git 分支工作流(Branching Workflow)
ModelRouter 采用 **GitHub Flow + 发布分支** 模型:`main` 是唯一长命分支且永远可部署,
新工作从它开出特性分支;每个版本从 `main` 分出独立的发布分支并打 tag;hotfix 提交到
发布分支且必须回流 `main`。
目的:让「已发版的版本在生命周期内可被单独修补」成为一等操作,同时 `main` 始终包含
全部修复、永远可部署。
## 分支角色
| 分支 | 命名 | 生命周期 | 用途 |
|---|---|---|---|
| 主分支 | `main` | 永久 | 唯一长命分支。永远可部署。积攒下一个版本的功能。 |
| 特性分支 | `feature/<描述>` | 短命(开发→合并即删) | 新特性 / 一般 bug 修复。从 `main` 开出,完成后合回 `main`。 |
| 发布分支 | `release/vX.Y.Z` | 一个版本周期 | 从 `main` 分出,打 tag 发布。该版本生命周期内的 hotfix 都提交在此分支。 |
## 变更流向(重要)
```
feature/foo ───────┐
▼
main ──────────────── 下一个版本 ──────►
│ │
│ 切出 │ 切出
▼ ▼
release/v1.4.2 release/v1.4.3
│ │
tag: v1.4.2 tag: v1.4.3
│ │
hotfix ◄─────┘ hotfix ◄─────┘
│ │
└── cherry-pick 回 main ───────┘
```
### 关键规则
1. **main 永远可部署**:不在 main 上留半成品。任何未完成的工作必须在特性分支上。
2. **特性从 main 开,完成后合回 main**:`git checkout -b feature/xxx main`,
开发完 `git merge --no-ff feature/xxx` 或 squash 合回。
3. **发布 = 从 main 切 release 分支 + 打 tag**:
```bash
git checkout -b release/v1.4.2 main
git tag -a v1.4.2 -m "ModelRouter v1.4.2"
git push origin release/v1.4.2 v1.4.2
```
构建安装包、上传 GitCode Release 都基于这个 tag,保证可精确回溯发布态。
4. **版本生命周期内只收该版本的 hotfix**:新特性一律并入 `main` 等下一个版本,
绝不塞进已发布的 release 分支(除非主动选择在该版本内发次要版)。
5. **hotfix 必须回流 main**:
```bash
git checkout release/v1.4.2 # 在发布分支提交修复
git commit -m "fix: ..."
git checkout main
git cherry-pick <hotfix-commit> # 回主分支
```
这是本模型最重要的一条:**main 只前进、不丢修复**。如果 hotfix 不回 main,
下一个版本就会带着旧 bug 发布。
### 版本生命周期结束
下一个版本发布时,上一个 release 分支退役:
- **默认:直接删除远端 release 分支**(`git push origin :release/v1.4.2`)。
因为 hotfix 都已逐个 cherry-pick 回 main,main 已包含全部修复,无需再合并。
- **如需要长期维护旧版**(例如企业大客户卡在旧版本):保留分支,仅 stopship 接受
该版本的安全修复,继续走「提交 + cherry-pick 回 main」循环。
## 明确不做的事
- **不 rebase main**:`main` 的历史保持追加式,任何人拉取后 `git pull` 都得到直接可用的历史。
- **发布分支不整体 merge 回 main**:hotfix 已逐个 cherry-pick,整体合并只会制造冲突且无收益。
- **不在 main 直接写"临时改一下"**:即使是单人项目,也至少走
`feature/xxx → merge main` 的形式,让历史保留"为什么改"的边界。
## 单人 vs 多人
- **单人**(本项目当前):特性分支可省去 PR,`git checkout -b feature/xxx` →
完成 → 合并回 main。发布分支与 tag 流程完全一致。
- **多人 / 开源协作**:特性分支走 PR(pull request)+ Code Review,
由维护者 merge;冲突在特性分支上解决,不在 main 上解决。
## 提交信息规范(与分支配套)
- `feat(...)`: 新功能
- `fix(...)`: 修复
- `chore(...)`: 构建 / 版本号 / 依赖 / 文档无关改动
- `docs(...)`: 文档
- `refactor(...)`: 重构
- 括号内为影响域,如 `fix(adapters)`、`feat(webui)`、`chore(version)`
- 一句话总结,必要时正文展开「问题 / 根因 / 修复 / 验证」
## 发布操作速查(配套)
每次发布严格按此顺序:
```bash
# 1. 确认 main 就绪
git checkout main && git pull
# 2. 切发布分支并打 tag
git checkout -b release/vX.Y.Z main
git tag -a vX.Y.Z -m "ModelRouter vX.Y.Z"
# 3. 推送分支 + tag
git push origin release/vX.Y.Z
git push origin vX.Y.Z
# 4. 构建安装包(核心包 + GUI 各平台)
make core-dist VERSION=X.Y.Z
# 5. 创建 GitCode Release + 上传安装包
# 见 https://gitcode.com/JianFeeeee/ModelRouter/releases
# 上传注意:必须 PUT --http1.1,否则 OBS 回调失败文件不进列表;
# 替换旧版附件 = 删 tag(release 随 tag 一起消失)后重建。
```
## 为什么是这套(背景)
2026-08 之前本项目直接在所有改动提交到 `main` + 打 tag,出现两个痛点:
1. 1.4.2 已发版后又追了一个调度打分修复,只能删远端 tag 清空 release 再重传安装包
—— 版本发布态与代码历史脱节,无法精确回溯"当时发的是什么"。
2. 没有特性分支,无法并行开发互不干扰的两件事。
引入发布分支后:版本发布态 = `release/vX.Y.Z` 分支 + `vX.Y.Z` tag,可精确回溯;
hotfix 有明确落点;main 保持"最新 + 全部修复 + 可部署"。
> 英文版见 [docs/git-workflow-en.md](git-workflow-en.md)。

View File

@ -47,15 +47,19 @@ type Model struct {
// Source describes a single upstream LLM provider.
type Source struct {
Name string `yaml:"name" json:"name"`
BaseURL string `yaml:"base_url" json:"base_url"`
APIKey string `yaml:"api_key" json:"api_key"`
APIKeyEnv string `yaml:"api_key_env,omitempty" json:"-"` // reference to an env var holding the key (overrides api_key)
Adapter string `yaml:"adapter" json:"adapter"`
Endpoint string `yaml:"endpoint" json:"endpoint,omitempty"` // chat endpoint override
ImageEndpoint string `yaml:"image_endpoint" json:"image_endpoint,omitempty"` // image endpoint override
Models []Model `yaml:"models" json:"models"`
Headers map[string]string `yaml:"headers" json:"headers,omitempty"`
Name string `yaml:"name" json:"name"`
BaseURL string `yaml:"base_url" json:"base_url"`
APIKey string `yaml:"api_key" json:"api_key"`
APIKeyEnv string `yaml:"api_key_env,omitempty" json:"-"` // reference to an env var holding the key (overrides api_key)
Adapter string `yaml:"adapter" json:"adapter"`
Endpoint string `yaml:"endpoint" json:"endpoint,omitempty"` // chat endpoint override
ImageEndpoint string `yaml:"image_endpoint" json:"image_endpoint,omitempty"` // image endpoint override
Models []Model `yaml:"models" json:"models"`
Headers map[string]string `yaml:"headers" json:"headers,omitempty"`
// ProxyURL routes this source's HTTP(S) traffic through an explicit proxy
// (e.g. http://127.0.0.1:7890). Empty = direct connection. Needed for
// upstreams behind geo/IP blocks that only respond through a proxy.
ProxyURL string `yaml:"proxy_url" json:"proxy_url,omitempty"`
Meta map[string]interface{} `yaml:"meta" json:"meta,omitempty"`
Temperature float64 `yaml:"temperature" json:"temperature,omitempty"`
MaxTokens int `yaml:"max_tokens" json:"max_tokens,omitempty"`
@ -328,13 +332,15 @@ type RuntimeConfig struct {
// name and api_key) so the WebUI can spin up multiple key-bearing sources
// from one shared template.
type SourceTemplate struct {
Name string `json:"name"`
BaseURL string `json:"base_url"`
Adapter string `json:"adapter"`
Endpoint string `json:"endpoint,omitempty"`
ImageEndpoint string `json:"image_endpoint,omitempty"`
Models []Model `json:"models"`
Headers map[string]string `json:"headers,omitempty"`
Name string `json:"name"`
BaseURL string `json:"base_url"`
Adapter string `json:"adapter"`
Endpoint string `json:"endpoint,omitempty"`
ImageEndpoint string `json:"image_endpoint,omitempty"`
Models []Model `json:"models"`
Headers map[string]string `json:"headers,omitempty"`
// ProxyURL routes this source's HTTP(S) traffic through an explicit proxy.
ProxyURL string `json:"proxy_url,omitempty"`
Meta map[string]interface{} `json:"meta,omitempty"`
Temperature float64 `json:"temperature,omitempty"`
MaxTokens int `json:"max_tokens,omitempty"`

View File

@ -7,6 +7,7 @@ import (
"fmt"
"log"
"net/http"
"regexp"
"strings"
"sync/atomic"
"time"
@ -115,13 +116,15 @@ func (g *Gateway) resolveCands(ctx context.Context, req *chatRequest) ([]*provid
}
// filterCandsByModels keeps only providers exposing at least one model of the
// scope (used for user keys with a restricted model scope). An "AUTO" scope
// entry means the key is allowed to use any model. Scope entries with a
// Source pinned to a specific upstream narrow the candidates to that source
// scope (used for user keys with a restricted model scope). Scope entries with
// a Source pinned to a specific upstream narrow the candidates to that source
// for the matching model.
//
// An "AUTO" scope entry only allows the AUTO routing mode; it does NOT grant
// access to specific models.
func filterCandsByModels(cands []*provider.Provider, allow []config.ModelScope) []*provider.Provider {
for _, m := range allow {
if m.Model == "" || strings.EqualFold(m.Model, "AUTO") {
if m.Model == "" {
return cands
}
}
@ -152,20 +155,18 @@ func filterCandsByModels(cands []*provider.Provider, allow []config.ModelScope)
return out
}
// intersectModels restricts a model list to the scope (preserving order). An
// "AUTO" scope entry grants every model.
// intersectModels restricts a model list to the scope (preserving order).
// An "AUTO" scope entry only allows the AUTO routing mode; it does NOT grant
// every model.
func intersectModels(models []string, allow []config.ModelScope) []string {
allowed := make(map[string]bool, len(allow))
any := false
for _, m := range allow {
if m.Model == "" || strings.EqualFold(m.Model, "AUTO") {
any = true
break
if m.Model == "" {
return models
}
if !strings.EqualFold(m.Model, "AUTO") {
allowed[m.Model] = true
}
allowed[m.Model] = true
}
if any {
return models
}
out := make([]string, 0, len(models))
seen := map[string]bool{}
@ -179,24 +180,16 @@ func intersectModels(models []string, allow []config.ModelScope) []string {
}
// checkModelScope validates the effective model against the key's model scope
// and token quota. Returns an error message when rejected. A scope entry with
// model "AUTO" grants all models; its quota caps the key's total tokens.
// and token quota. Returns an error message when rejected.
//
// A scope entry with model "AUTO" only allows requests where the effective
// model is AUTO (the routing mode). It does NOT grant access to specific model
// ids — that requires an explicit scope entry for the model.
func (g *Gateway) checkModelScope(ctx context.Context, model string) string {
allow := g.allowedModels(ctx)
if allow == nil {
return ""
}
for _, sc := range allow {
if sc.Model != "" && strings.EqualFold(sc.Model, "AUTO") {
if sc.TokenQuota > 0 {
used := g.scopeTokens(ctx, sc)
if used >= sc.TokenQuota {
return fmt.Sprintf("token quota exceeded (%d/%d)", used, sc.TokenQuota)
}
}
return ""
}
}
for _, sc := range allow {
if sc.Model != model {
continue
@ -229,12 +222,15 @@ func (g *Gateway) scopeTokens(ctx context.Context, sc config.ModelScope) int64 {
// source that actually serves the bare model (via Registry.EffectiveModel), so
// model ids that themselves contain separators (e.g. "deepseek-v4-flash-free")
// are never corrupted (P10-2).
//
// A scope entry with model "AUTO" only matches the literal AUTO routing mode;
// it does NOT grant access to specific model ids.
func (g *Gateway) hasScopeModel(list []config.ModelScope, s string) bool {
if r := g.core.Registry(); r != nil {
s = r.EffectiveModel(s)
}
for _, x := range list {
if x.Model == s || (x.Model != "" && strings.EqualFold(x.Model, "AUTO")) {
if x.Model == s {
return true
}
}
@ -517,21 +513,67 @@ func upstreamErrStatus(err error) int {
return http.StatusBadGateway
}
// overflowMarkers 匹配上游「上下文超窗」类措辞。
//
// 上游写法五花八门,且**不在 pi 客户端的识别列表里**。pi 靠
// @earendil-works/pi-ai 的 OVERFLOW_PATTERNS 判断超窗并据此触发压缩重试,
// 而 justworker 返回的是「请精简对话历史…(Context window is full…)」——
// 与那 25 条正则一条都不匹配,于是 pi 既不压缩也不重试,只把它当成一条
// 普通上游错误。这里把可识别的超窗措辞归一化成 pi 一定认得的标记。
var overflowMarkers = []*regexp.Regexp{
regexp.MustCompile(`(?i)context[ _-]?window is full`),
regexp.MustCompile(`(?i)context[_ ]length[_ ]exceeded`),
regexp.MustCompile(`(?i)exceeds? the context window`),
regexp.MustCompile(`(?i)maximum context length`),
regexp.MustCompile(`(?i)reduce the length of the messages`),
regexp.MustCompile(`(?i)too many tokens`),
regexp.MustCompile(`(?i)token limit exceeded`),
regexp.MustCompile(`请精简对话历史`),
regexp.MustCompile(`上下文(长度)?超(出|限)`),
regexp.MustCompile(`对话历史过长`),
}
// overflowCanonical 命中 pi 的 /context[_ ]length[_ ]exceeded/i。
const overflowCanonical = "context_length_exceeded"
// looksLikeOverflow 判断错误文本是否属于上下文超窗。
func looksLikeOverflow(s string) bool {
for _, re := range overflowMarkers {
if re.MatchString(s) {
return true
}
}
return false
}
// clientUpstreamErr collapses upstream failure details into a short message
// for the client: per-tier bodies (WAF HTML pages, quota payloads, ...) stay
// in rec.Err / the stats API and the server log instead of the response.
// Per-tier one-line reasons are kept (quota/cooling skips carry no error body
// and are the actionable part); each is capped so HTML dumps can't leak.
//
// 超窗会被归一化成 overflowCanonical 前缀,见 overflowMarkers 的说明。
func clientUpstreamErr(err error) string {
log.Printf("[gateway] upstream failure surfaced to client: %v", err)
msg := upstreamErrSummary(err)
if looksLikeOverflow(err.Error()) || looksLikeOverflow(msg) {
return fmt.Sprintf("%s: %s", overflowCanonical, msg)
}
return msg
}
// upstreamErrSummary 把上游失败压成一行短消息。
func upstreamErrSummary(err error) string {
var ce *scheduler.ChainErr
if errors.As(err, &ce) {
parts := make([]string, 0, len(ce.Tiers)+len(ce.Skipped))
for _, t := range ce.Tiers {
parts = append(parts, types.OneLine(fmt.Sprintf("%s/%s: %v", t.Source, t.Model, t.Err), 80))
// 160 而不是 80:短诊断词("Context window is full")常落在尾部,
// 80 字节按字节截断正好会把它切掉,超窗就再也认不出来。
parts = append(parts, types.OneLine(fmt.Sprintf("%s/%s: %v", t.Source, t.Model, t.Err), 160))
}
for _, sk := range ce.Skipped {
parts = append(parts, types.OneLine(sk, 80))
parts = append(parts, types.OneLine(sk, 160))
}
msg := strings.Join(parts, "; ")
if len(msg) > 300 {

View File

@ -670,8 +670,11 @@ func TestHasScopeModelWithSourcePrefix(t *testing.T) {
if g.hasScopeModel(scope, "deepseek-v4-flash-free-extra") {
t.Error("hasScopeModel returned true for unrelated model")
}
if !g.hasScopeModel([]config.ModelScope{{Model: "AUTO"}}, "zen:anything") {
t.Error("AUTO scope should allow any prefixed model")
if g.hasScopeModel([]config.ModelScope{{Model: "AUTO"}}, "zen:deepseek-v4-flash-free") {
t.Error("AUTO scope must NOT grant specific model ids even with prefix")
}
if g.hasScopeModel([]config.ModelScope{{Model: "AUTO"}}, "deepseek-v4-pro") {
t.Error("AUTO scope must NOT grant specific model ids")
}
}

View File

@ -0,0 +1,96 @@
package gateway
import (
"errors"
"regexp"
"strings"
"testing"
"llmsproxy/internal/scheduler"
"llmsproxy/internal/types"
)
// piOverflowRe 是 @earendil-works/pi-ai 的 OVERFLOW_PATTERNS 中对应的一条。
// 归一化标记必须命中它,否则改了等于没改:客户端仍不会压缩重试。
var piOverflowRe = regexp.MustCompile(`context[_ ]length[_ ]exceeded`)
func TestOverflowCanonicalMatchesPiPattern(t *testing.T) {
if !piOverflowRe.MatchString(overflowCanonical) {
t.Fatalf("归一化标记 %q 不匹配 pi 的超窗正则,客户端仍不会压缩重试", overflowCanonical)
}
}
// justworker 的真实措辞:与 pi 现有 25 条正则一条都不匹配,
// 所以必须在网关侧归一化,否则超窗对客户端完全不可见。
func TestJustworkerOverflowIsNormalized(t *testing.T) {
raw := "请精简对话历史…(Context window is full, please reduce the conversation history)…"
// 前提校验:原始措辞确实不被 pi 认出(否则本用例失去意义)。
if piOverflowRe.MatchString(raw) {
t.Fatalf("前提不成立:原始措辞本就匹配 pi 正则: %q", raw)
}
if !looksLikeOverflow(raw) {
t.Fatalf("网关应识别出这是超窗: %q", raw)
}
err := &scheduler.ChainErr{Tiers: []scheduler.TierError{{
Tier: 1, Source: "justworker", Model: "claude-opus-5", Err: errors.New(raw),
}}}
got := clientUpstreamErr(err)
if !piOverflowRe.MatchString(got) {
t.Fatalf("归一化后仍不匹配 pi 正则,客户端不会压缩重试: %q", got)
}
}
// 直连路径(非 AUTO 链)也要归一化——它走的是同一个入口。
func TestDirectPathOverflowIsNormalized(t *testing.T) {
got := clientUpstreamErr(errors.New("upstream 400: Context window is full"))
if !piOverflowRe.MatchString(got) {
t.Fatalf("直连超窗未被归一化: %q", got)
}
}
// 非超窗错误不能被加上超窗标记,否则会把普通失败误报成需要压缩。
func TestNonOverflowNotNormalized(t *testing.T) {
err := &scheduler.ChainErr{Tiers: []scheduler.TierError{{
Tier: 1, Source: "x", Model: "y", Err: errors.New("502 Bad Gateway: upstream connect error"),
}}}
got := clientUpstreamErr(err)
if strings.Contains(got, overflowCanonical) {
t.Fatalf("普通上游失败被误标为超窗: %q", got)
}
if !strings.Contains(got, "502 Bad Gateway") {
t.Fatalf("普通失败的诊断信息应保留: %q", got)
}
}
// 回归:短诊断词常落在错误尾部,80 字节截断会正好把它切掉,
// 超窗就永远认不出来。宽度放宽后必须保住。
func TestOverflowPhraseSurvivesTruncationWidth(t *testing.T) {
prefix := strings.Repeat("x", 100)
// OneLine 的长度限制作用于整条 "source/model: err",不是只算 err 本身。
full := "justworker/claude-opus-5: " + prefix + " Context window is full"
// 前提:80 字节下短语确实会被切掉(这正是修复前的行为)。
if got := types.OneLine(full, 80); strings.Contains(got, "Context window is full") {
t.Fatalf("前提不成立:80 字节下短语居然还在: %q", got)
}
// 160 字节下必须保住,否则超窗识别会被截断悄悄破坏。
if got := types.OneLine(full, 160); !strings.Contains(got, "Context window is full") {
t.Fatalf("160 字节应保住尾部短诊断词: %q", got)
}
err := &scheduler.ChainErr{Tiers: []scheduler.TierError{{
Tier: 1, Source: "justworker", Model: "claude-opus-5",
Err: errors.New(prefix + " Context window is full"),
}}}
got := clientUpstreamErr(err)
if !strings.Contains(strings.ToLower(got), "context window is full") {
t.Fatalf("宽度 160 应保住尾部短诊断词,实际: %q", got)
}
if !piOverflowRe.MatchString(got) {
t.Fatalf("归一化标记丢失: %q", got)
}
}

View File

@ -5,12 +5,34 @@ adapter.version = "1.0.0"
adapter.endpoint = "/chat/completions"
adapter.headers = {}
-- AgentRouter 的 WAF 会按客户端指纹白名单校验,只放行官方客户端。
-- 这里通过 build_headers 注入官方客户端 User-Agent 以通过校验。
-- 默认 QwenCode 指纹(实测可通);可在 source.meta.user_agent 覆盖成 Claude Code 指纹。
-- AgentRouter 的阿里云 WAF 按三重维度校验:出口 IP(必须海外)、
-- TLS 指纹(JS/Go HTTP 层与 curl 不同)、浏览器指纹 header 集。
-- 纯 UA 已不够:WAF 拦 12 种 agent CLI UA 中的 10 种(全 405 HTML),
-- 只有同时带 Origin: qwen.ai + Referer + X-Requested-With 才放行,
-- 且同 IP 短时间连续 3+ 次请求会被速率封禁,靠 adapter 侧低频调用 +
-- llmsproxy 的 prefFailStep 冷却自然限制,来源无法控制速率。
local default_ua = "QwenCode/0.2.0 (linux; x64)"
-- AgentRouter fronts Claude models (claude-opus-4-8), and Claude upstreams
-- enforce Anthropic's tool id rule ^[a-zA-Z0-9_-]{1,64}$ by rejecting the WHOLE
-- request. Plain OpenAI does not, so an id minted by a permissive model (e.g.
-- "bash:0" from moonshotai/kimi-k3) is replayed here by the client and kills
-- every turn. See anthropic.lua / openai.lua for the same helper: Lua adapters
-- have no shared prelude, so each carries its own copy.
local TOOL_ID_MAX = 64
local function safe_tool_id(id)
if type(id) ~= "string" or id == "" then return id end
local clean = string.gsub(id, "[^A-Za-z0-9_-]", "_")
if clean == id and #clean <= TOOL_ID_MAX then
return clean
end
local digest = string.sub(sha256_hex(id), 1, 8)
local keep = TOOL_ID_MAX - #digest - 1
if #clean > keep then clean = string.sub(clean, 1, keep) end
return clean .. "_" .. digest
end
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
@ -19,6 +41,16 @@ function adapter.transform_request(raw_body)
if req.messages then
for _, msg in ipairs(req.messages) do
msg.reasoning_content = nil
if msg.tool_call_id ~= nil then
msg.tool_call_id = safe_tool_id(msg.tool_call_id)
end
if type(msg.tool_calls) == "table" then
for _, tc in ipairs(msg.tool_calls) do
if type(tc) == "table" and tc.id ~= nil then
tc.id = safe_tool_id(tc.id)
end
end
end
end
end
return json.encode(req)
@ -35,6 +67,13 @@ function adapter.build_headers(meta)
["Content-Type"] = "application/json",
["Authorization"] = "Bearer " .. meta.api_key,
["User-Agent"] = ua,
-- QwenCode client fingerprint: WAF rejects bare UA without these.
["Accept"] = "application/json, text/plain, */*",
["Accept-Language"] = "zh-CN,zh;q=0.9",
["Origin"] = "https://qwen.ai",
["Referer"] = "https://qwen.ai/",
["X-Requested-With"] = "XMLHttpRequest",
["Connection"] = "keep-alive",
}
end
@ -79,7 +118,7 @@ function adapter.transform_response(raw_body)
local args_ok, args = pcall(json.decode, tc["function"].arguments)
if not args_ok then args = {} end
table.insert(tcs, {
id = tc.id,
id = safe_tool_id(tc.id),
type = tc.type or "function",
name = tc["function"].name,
arguments = args
@ -138,6 +177,15 @@ function adapter.transform_stream_chunk(raw_chunk)
unified.reasoning_content = delta.reasoning_content
end
if delta.tool_calls then
-- Sanitize outbound too: a dirty id must never enter a client session,
-- because the client replays it to every other source. Only the first
-- fragment of a streamed call carries an id; later argument fragments
-- have none and must stay id-less for index-based accumulation.
for _, tc in ipairs(delta.tool_calls) do
if type(tc) == "table" and tc.id ~= nil then
tc.id = safe_tool_id(tc.id)
end
end
unified.tool_calls = delta.tool_calls
end
if uses ~= nil then

View File

@ -14,6 +14,70 @@ local ROLE_WHITELIST = {
system = true, user = true, assistant = true, tool = true,
}
-- Anthropic requires tool_use.id / tool_result.tool_use_id to match
-- ^[a-zA-Z0-9_-]{1,64}$ and rejects the WHOLE request otherwise with
-- REQUEST_BODY_INVALID / "Invalid tool use format". OpenAI imposes no such
-- rule, so an OpenAI-compatible upstream can hand a client an id like
-- "bash:0" (observed from moonshotai/kimi-k3); once that lands in the
-- client's history, every later replay kills every Claude slot at once and
-- an AUTO chain falls through all tiers.
--
-- safe_tool_id is pure and deterministic, so a tool_use block and its
-- matching tool_result are rewritten identically within one request. A
-- rewritten id keeps an 8-hex digest of the ORIGINAL id: without it two
-- distinct ids could collapse into one ("a:b" and "a_b"), which Anthropic
-- rejects as a duplicate/unpaired tool_use. Already-legal ids pass through
-- untouched so upstreams that round-trip their own ids are unaffected.
local TOOL_ID_MAX = 64
local function safe_tool_id(id)
if type(id) ~= "string" or id == "" then return "" end
local clean = string.gsub(id, "[^A-Za-z0-9_-]", "_")
if clean == id and #clean <= TOOL_ID_MAX then
return clean
end
local digest = string.sub(sha256_hex(id), 1, 8)
local keep = TOOL_ID_MAX - #digest - 1
if #clean > keep then clean = string.sub(clean, 1, keep) end
return clean .. "_" .. digest
end
-- map_usage converts one Anthropic usage object to the gateway's TokenUsage
-- shape. The two APIs disagree on what the prompt count MEANS:
--
-- Anthropic: input_tokens EXCLUDES cached blocks. cache_read_input_tokens and
-- cache_creation_input_tokens are separate, additive, billed input.
-- OpenAI: prompt_tokens INCLUDES its cached_tokens subset.
--
-- Mapping input_tokens straight onto prompt therefore did two wrong things at
-- once: it undercounted the billed prompt by the entire cache portion, and it
-- could report cached_tokens > prompt_tokens — a cache hit rate above 100%.
-- cache_creation_input_tokens was not read at all, so a cache-write turn lost
-- those billed tokens outright. Summing all three restores OpenAI semantics,
-- which is what clients (and the gateway's own audit trail) assume.
--
-- Returns nil when the object carries no counts, so callers can keep treating
-- "no usage in this chunk" as "say nothing" rather than reporting zeros.
local function map_usage(u)
if type(u) ~= "table" then return nil end
local fresh = u.input_tokens or 0
local read = u.cache_read_input_tokens or 0
local create = u.cache_creation_input_tokens or 0
local p = fresh + read + create
local c = u.output_tokens or 0
if p <= 0 and c <= 0 then return nil end
local uses = { prompt = p, completion = c, total = p + c }
-- Emit the split whenever Anthropic reports either cache field, even at 0:
-- a reported zero-hit ("cache missed") must stay distinguishable from "this
-- upstream never reports cache info at all".
if u.cache_read_input_tokens ~= nil or u.cache_creation_input_tokens ~= nil then
uses.prompt_tokens_details = { cached_tokens = read }
uses.prompt_cache_hit_tokens = read
uses.prompt_cache_miss_tokens = fresh + create
end
return uses
end
local function collect_blocks(content)
if type(content) == "string" then
if content == "" then return {} end
@ -104,7 +168,7 @@ function adapter.transform_request(raw_body)
end
table.insert(blocks, {
type = "tool_use",
id = tc.id or "",
id = safe_tool_id(tc.id),
name = fn.name or "",
input = input,
})
@ -119,7 +183,7 @@ function adapter.transform_request(raw_body)
-- user message with tool_result content blocks.
table.insert(pending_tool, {
type = "tool_result",
tool_use_id = m.tool_call_id or "",
tool_use_id = safe_tool_id(m.tool_call_id),
content = text_of(m.content),
})
@ -220,17 +284,9 @@ function adapter.transform_response(raw_body)
token_usage = { prompt = 0, completion = 0, total = 0 },
}
if resp.usage then
unified.token_usage.prompt = resp.usage.input_tokens or 0
unified.token_usage.completion = resp.usage.output_tokens or 0
unified.token_usage.total = (resp.usage.input_tokens or 0) + (resp.usage.output_tokens or 0)
-- Emit details whenever Anthropic reports the field, even at 0, so a
-- reported cache miss stays distinguishable from "not reported".
if resp.usage.cache_read_input_tokens ~= nil then
unified.token_usage.prompt_tokens_details = {
cached_tokens = resp.usage.cache_read_input_tokens
}
end
local mapped = map_usage(resp.usage)
if mapped ~= nil then
unified.token_usage = mapped
end
if resp.content and #resp.content > 0 then
@ -242,7 +298,7 @@ function adapter.transform_response(raw_body)
unified.reasoning_content = (unified.reasoning_content or "") .. block.thinking
elseif block.type == "tool_use" then
table.insert(tcs, {
id = block.id or "",
id = safe_tool_id(block.id),
type = "function",
name = block.name or "",
arguments = block.input or {},
@ -275,17 +331,13 @@ function adapter.transform_stream_chunk(raw_chunk)
-- ── message_start: initial usage ─────────────────────────
if chunk.type == "message_start" then
if chunk.message and type(chunk.message.usage) == "table" then
local u = chunk.message.usage
local p = u.input_tokens or 0
local c = u.output_tokens or 0
if p > 0 or c > 0 then
local uses = { prompt = p, completion = c, total = p + c }
if u.cache_read_input_tokens ~= nil then
uses.prompt_tokens_details = {
cached_tokens = u.cache_read_input_tokens
}
end
if chunk.message then
-- Some Anthropic-compatible upstreams report a placeholder here and
-- only send the true prompt count in message_delta (justwoker:
-- 160 at message_start vs 6931 at message_delta). The gateway's
-- mergeUsage lets a later non-zero value win, so both are emitted.
local uses = map_usage(chunk.message.usage)
if uses ~= nil then
return json.encode({ usage = uses, done = false })
end
end
@ -305,15 +357,10 @@ function adapter.transform_stream_chunk(raw_chunk)
finish = "stop"
end
end
local uses = nil
if type(chunk.usage) == "table" then
local u = chunk.usage
local p = u.input_tokens or 0
local c = u.output_tokens or 0
if p > 0 or c > 0 then
uses = { prompt = p, completion = c, total = p + c }
end
end
-- message_delta carries the FINAL usage, so it must map the cache
-- fields too; the old code dropped them here, which silently lost the
-- whole cache split on every streamed response.
local uses = map_usage(chunk.usage)
if uses ~= nil then
return json.encode({ content = "", done = (finish ~= nil), finish_reason = finish, usage = uses })
end
@ -330,7 +377,7 @@ function adapter.transform_stream_chunk(raw_chunk)
content = "", done = false,
tool_calls = { {
index = chunk.index or 0,
id = cb.id or "",
id = safe_tool_id(cb.id),
type = "function",
["function"] = { name = cb.name or "", arguments = "" },
} },

View File

@ -7,6 +7,10 @@ adapter.headers = {}
-- Gemini API: POST /v1/models/{model}:generateContent
-- Auth: API key in query param ?key=XXX or Authorization: Bearer XXX
--
-- 请求方向必须把 OpenAI 的工具调用翻译成 Gemini 的 functionCall /
-- functionResponse,否则 agent 回放的历史里助手那一轮的调用会凭空消失,
-- 紧随其后的工具结果就成了「无来源」的孤立结果,模型只能反复重发同一个调用。
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
@ -14,16 +18,23 @@ function adapter.transform_request(raw_body)
-- 将 OpenAI 风格 content(字符串或 [{type:*}] 数组)拆成 Gemini parts
local function to_parts(content)
if type(content) == "string" then
if content == "" then return {} end
return { { text = content } }
end
local parts = {}
for _, p in ipairs(content or {}) do
if p.type == "text" then
table.insert(parts, { text = p.text })
elseif p.type == "image_url" and type(p.image_url) == "table" and p.image_url.url then
local mt, b64 = string.match(p.image_url.url, "^data:([^,]+);base64,(.+)$")
if b64 then
table.insert(parts, { inline_data = { mime_type = mt or "image/png", data = b64 } })
if type(p) == "string" then
if p ~= "" then table.insert(parts, { text = p }) end
elseif type(p) == "table" then
if p.type == "text" then
if p.text ~= nil and p.text ~= "" then
table.insert(parts, { text = p.text })
end
elseif p.type == "image_url" and type(p.image_url) == "table" and p.image_url.url then
local mt, b64 = string.match(p.image_url.url, "^data:([^,]+);base64,(.+)$")
if b64 then
table.insert(parts, { inline_data = { mime_type = mt or "image/png", data = b64 } })
end
end
end
end
@ -31,11 +42,59 @@ function adapter.transform_request(raw_body)
end
local contents = {}
local system = ""
local call_names = {} -- tool_call_id -> 函数名(functionResponse 只认名字)
for _, m in ipairs(req.messages or {}) do
table.insert(contents, {
role = (m.role == "assistant") and "model" or m.role,
parts = to_parts(m.content)
})
local role = m.role or "user"
if role == "system" then
if type(m.content) == "string" then
system = system .. m.content .. "\n"
end
elseif role == "assistant" then
local parts = to_parts(m.content)
if type(m.tool_calls) == "table" then
for _, tc in ipairs(m.tool_calls) do
if type(tc) == "table" then
local fn = tc["function"] or {}
local args = fn.arguments
if type(args) == "string" and args ~= "" then
local aok, decoded = pcall(json.decode, args)
args = aok and decoded or {}
elseif type(args) ~= "table" then
args = {}
end
if tc.id ~= nil then call_names[tc.id] = fn.name or "" end
table.insert(parts, {
functionCall = { name = fn.name or "", args = args }
})
end
end
end
-- Gemini 不接受空的 parts 数组;无可发送内容的轮次直接跳过
if #parts > 0 then
table.insert(contents, { role = "model", parts = parts })
end
elseif role == "tool" then
local name = call_names[m.tool_call_id] or m.name or ""
local text = type(m.content) == "string" and m.content or ""
table.insert(contents, {
role = "user",
parts = { { functionResponse = {
name = name,
response = { content = text },
} } }
})
else
local parts = to_parts(m.content)
if #parts > 0 then
table.insert(contents, { role = "user", parts = parts })
end
end
end
local gemini_req = {
@ -46,6 +105,30 @@ function adapter.transform_request(raw_body)
}
}
if system ~= "" then
gemini_req.systemInstruction = { parts = { { text = system } } }
end
-- tools -> functionDeclarations
if type(req.tools) == "table" then
local decls = {}
for _, t in ipairs(req.tools) do
if type(t) == "table" and type(t["function"]) == "table" then
local fn = t["function"]
local params = fn.parameters
if type(params) ~= "table" then params = { type = "object", properties = {} } end
table.insert(decls, {
name = fn.name or "",
description = fn.description or "",
parameters = params,
})
end
end
if #decls > 0 then
gemini_req.tools = { { functionDeclarations = decls } }
end
end
if req.stream then
gemini_req.stream = true
end

View File

@ -6,6 +6,10 @@ adapter.endpoint = "/api/chat"
adapter.headers = {}
-- Ollama API 格式:{ model, messages, stream, options:{temperature,num_predict} }
-- 工具调用必须一并翻译:Ollama 的 assistant 消息用 tool_calls(arguments 是对象
-- 而非 JSON 字符串),工具结果用 tool 角色 + tool_name。此前这里只复制了
-- role/content,助手那轮的调用和 tool 消息的归属全部丢失,模型看到无来源的
-- 工具结果就只能反复重发同一个调用。
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
@ -22,6 +26,7 @@ function adapter.transform_request(raw_body)
-- 转换 messages 格式(Ollama messages 支持 images base64 数组)
if req.messages then
local msgs = {}
local call_names = {} -- tool_call_id -> 函数名
for _, m in ipairs(req.messages) do
local text, images
if type(m.content) == "string" then
@ -30,22 +35,61 @@ function adapter.transform_request(raw_body)
text = ""
images = {}
for _, p in ipairs(m.content or {}) do
if p.type == "text" then
text = text .. (p.text or "")
elseif p.type == "image_url" and type(p.image_url) == "table" and p.image_url.url then
local b64 = string.match(p.image_url.url, "^data:[^,]+;base64,(.+)$")
if b64 then table.insert(images, b64) end
if type(p) == "table" then
if p.type == "text" then
text = text .. (p.text or "")
elseif p.type == "image_url" and type(p.image_url) == "table" and p.image_url.url then
local b64 = string.match(p.image_url.url, "^data:[^,]+;base64,(.+)$")
if b64 then table.insert(images, b64) end
end
end
end
if #images == 0 then images = nil end
end
local msg = { role = m.role, content = text }
if images then msg.images = images end
-- 助手轮的工具调用:arguments 转成对象
if type(m.tool_calls) == "table" and #m.tool_calls > 0 then
local tcs = {}
for _, tc in ipairs(m.tool_calls) do
if type(tc) == "table" then
local fn = tc["function"] or {}
local args = fn.arguments
if type(args) == "string" and args ~= "" then
local aok, decoded = pcall(json.decode, args)
args = aok and decoded or {}
elseif type(args) ~= "table" then
args = {}
end
if tc.id ~= nil then call_names[tc.id] = fn.name or "" end
table.insert(tcs, {
["function"] = { name = fn.name or "", arguments = args }
})
end
end
if #tcs > 0 then msg.tool_calls = tcs end
end
-- 工具结果:Ollama 用 tool_name 标识归属(不认 tool_call_id)
if m.role == "tool" then
local name = call_names[m.tool_call_id] or m.name or ""
if m.tool_call_id ~= nil then
msg.tool_call_id = m.tool_call_id
end
if name ~= "" then msg.tool_name = name end
end
table.insert(msgs, msg)
end
ollama_req.messages = msgs
end
-- tools 透传(Ollama 的 shape 与 OpenAI 一致)
if type(req.tools) == "table" and #req.tools > 0 then
ollama_req.tools = req.tools
end
return json.encode(ollama_req)
end

View File

@ -5,6 +5,36 @@ adapter.version = "2.0.0"
adapter.endpoint = "/chat/completions"
adapter.headers = {}
-- Claude-behind-OpenAI upstreams (tabitoken, 扇贝, …) convert /chat/completions
-- to the Anthropic Messages API internally, so they inherit Anthropic's
-- tool id rule: ^[a-zA-Z0-9_-]{1,64}$, enforced by rejecting the WHOLE request
-- with "Invalid tool use format" / "tool_use.id: String should match pattern".
-- Plain OpenAI has no such rule, so an OpenAI-compatible model can mint an id
-- like "bash:0" (observed from moonshotai/kimi-k3). In a fan-out router that id
-- is replayed to every other source, so one such id kills every Claude slot at
-- once and an AUTO request falls through all tiers.
--
-- safe_tool_id is pure and deterministic, so a tool_calls entry and its
-- matching tool_call_id are rewritten identically within one request. A
-- rewritten id keeps an 8-hex digest of the ORIGINAL id, without which two
-- distinct ids could collapse into one ("a:b" and "a_b") and become an
-- unpaired/duplicate tool call. Already-legal ids are returned untouched, so
-- well-behaved traffic is byte-identical to before.
-- (anthropic.lua carries the same helper; Lua adapters have no shared prelude.)
local TOOL_ID_MAX = 64
local function safe_tool_id(id)
if type(id) ~= "string" or id == "" then return id end
local clean = string.gsub(id, "[^A-Za-z0-9_-]", "_")
if clean == id and #clean <= TOOL_ID_MAX then
return clean
end
local digest = string.sub(sha256_hex(id), 1, 8)
local keep = TOOL_ID_MAX - #digest - 1
if #clean > keep then clean = string.sub(clean, 1, keep) end
return clean .. "_" .. digest
end
-- OpenAI /chat/completions format (pass-through, strip provider-specific fields)
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
@ -14,6 +44,16 @@ function adapter.transform_request(raw_body)
if req.messages then
for _, msg in ipairs(req.messages) do
msg.reasoning_content = nil
if msg.tool_call_id ~= nil then
msg.tool_call_id = safe_tool_id(msg.tool_call_id)
end
if type(msg.tool_calls) == "table" then
for _, tc in ipairs(msg.tool_calls) do
if type(tc) == "table" and tc.id ~= nil then
tc.id = safe_tool_id(tc.id)
end
end
end
end
end
return json.encode(req)
@ -64,7 +104,7 @@ function adapter.transform_response(raw_body)
local args_ok, args = pcall(json.decode, tc["function"].arguments)
if not args_ok then args = {} end
table.insert(tcs, {
id = tc.id,
id = safe_tool_id(tc.id),
type = tc.type or "function",
name = tc["function"].name,
arguments = args
@ -129,6 +169,15 @@ function adapter.transform_stream_chunk(raw_chunk)
unified.reasoning_content = delta.reasoning_content
end
if delta.tool_calls then
-- Sanitize on the way OUT too: an id this upstream happily minted (it
-- does not validate them) becomes a landmine once the client replays
-- it to a Claude upstream. Only the first fragment of a streamed call
-- carries an id; later argument fragments have none and are untouched.
for _, tc in ipairs(delta.tool_calls) do
if type(tc) == "table" and tc.id ~= nil then
tc.id = safe_tool_id(tc.id)
end
end
unified.tool_calls = delta.tool_calls
end
return json.encode(unified)

View File

@ -34,8 +34,10 @@ end
-- OpenAI /chat/completions format (pass-through, strip provider-specific fields)
-- zen 上游 schema 只接受 text content part(无视觉/音频能力):多模态 part
-- (image_url / input_audio / file 等)一律剥离;因此失去全部 content 的
-- 消息整条丢弃,避免上游 "unknown variant `image_url`, expected `text`"。
-- (image_url / input_audio / file 等)一律剥离。剥离后 content 变空的消息
-- 若不再携带 tool_calls / tool_call_id 才整条丢弃(避免上游
-- "unknown variant `image_url`, expected `text`");带工具调用的必须保留,
-- 否则会把紧随其后的 tool 结果变成孤儿,模型会反复重发同一个调用。
-- zen 上游角色白名单只有 system / user / assistant / tool / latest_reminder:
-- OpenAI 的 developer(及 function 等)不在其中,直接透传会触发上游
-- "unknown variant `developer`, expected one of ..." 错误;统一归一化为 system。
@ -52,8 +54,14 @@ function adapter.transform_request(raw_body)
if not ok then return raw_body end
req.disable_thinking = nil
req.extra_body = nil
if type(req.stream_options) ~= "table" then req.stream_options = {} end
req.stream_options.include_usage = true
-- stream_options is only valid alongside stream:true. Sending it on a
-- non-streaming request is rejected by stricter upstreams (OpenCode Go:
-- "stream_options should be set along with stream"), which failed every
-- non-stream call. Only attach it when the request actually streams.
if req.stream then
if type(req.stream_options) ~= "table" then req.stream_options = {} end
req.stream_options.include_usage = true
end
if req.messages then
local kept = {}
for _, msg in ipairs(req.messages) do
@ -72,7 +80,24 @@ function adapter.transform_request(raw_body)
end
end
if #parts == 0 then
drop = true
-- Content collapsed to nothing after stripping unsupported
-- parts. A message that still carries a tool call must
-- NEVER be dropped: the very next message is its tool
-- result, and dropping the call orphans that result. The
-- model then sees a result for a call it never made and
-- re-issues the same tool call on every turn (observed as
-- an infinite "repeated tool call" loop).
--
-- This is the common shape for agent clients: an assistant
-- turn whose content is only [thinking, toolCall] serialises
-- to content:[] with tool_calls — exactly the case that used
-- to vanish here. Emit "" instead, which zen accepts.
if (type(msg.tool_calls) == "table" and #msg.tool_calls > 0)
or msg.tool_call_id ~= nil then
msg.content = ""
else
drop = true
end
else
msg.content = parts
end

View File

@ -0,0 +1,248 @@
local adapter = {}
adapter.name = "opencodego"
adapter.version = "1.0.0"
adapter.endpoint = "/chat/completions"
-- OpenCode Go(包月订阅端点,https://opencode.ai/zen/go/v1)
--
-- 与 OpenCode Zen(opencodezen.lua,https://opencode.ai/zen/v1)是两个不同的
-- 服务,行为要求并不相同,因此各有专用适配器:
--
-- * 本适配器(Go)服务包月订阅池。上游**强制要求**
-- `x-opencode-session` 头,缺失直接 400 "MissingSessionID"。
-- * Go 的 thinking 模型(deepseek-v4.1-flash 等)**必须回传** assistant 轮的
-- `reasoning_content`,否则 400:
-- The `reasoning_content` in the thinking mode must be passed back to the API.
-- 这是本适配器与 Zen 适配器最关键的差异 —— Zen 会剥掉它,Go 必须原样保留。
-- * Go 上游对协议更严格:非流式请求带 `stream_options` 会被拒
-- ("stream_options should be set along with stream")。
--
-- 同样需要 opencode 客户端指纹(UA + x-opencode-*)才能被正确识别与路由。
adapter.headers = {
["User-Agent"] = "opencode/1.18.21 ai-sdk/provider-utils/4.0.23 runtime/bun/1.3.14",
}
-- 每请求生成身份头。沙箱无 os/math,用 meta.timestamp + 请求体哈希派生:
-- 同秒内重复请求 id 相同可接受(zen 只校验存在性,不校验格式)。
local function rand_id(prefix, seed)
return prefix .. string.sub(sha256_hex(seed), 1, 24)
end
function adapter.build_headers(meta)
local ts = tostring(meta.timestamp or "")
return {
["User-Agent"] = adapter.headers["User-Agent"],
["x-opencode-client"] = "cli",
-- project 固定:同网关实例共享一个工作区身份
["x-opencode-project"] = string.sub(sha256_hex("llmsproxy|" .. (meta.source and meta.source.name or "")), 1, 32),
["x-opencode-session"] = rand_id("ses_", "session|" .. ts),
["x-opencode-request"] = rand_id("msg_", "request|" .. ts .. "|" .. tostring(meta.body or "")),
}
end
-- OpenAI /chat/completions format (pass-through, strip provider-specific fields)
-- zen 上游 schema 只接受 text content part(无视觉/音频能力):多模态 part
-- (image_url / input_audio / file 等)一律剥离。剥离后 content 变空的消息
-- 若不再携带 tool_calls / tool_call_id 才整条丢弃(避免上游
-- "unknown variant `image_url`, expected `text`");带工具调用的必须保留,
-- 否则会把紧随其后的 tool 结果变成孤儿,模型会反复重发同一个调用。
-- zen 上游角色白名单只有 system / user / assistant / tool / latest_reminder:
-- OpenAI 的 developer(及 function 等)不在其中,直接透传会触发上游
-- "unknown variant `developer`, expected one of ..." 错误;统一归一化为 system。
local ROLE_WHITELIST = {
system = true,
user = true,
assistant = true,
tool = true,
latest_reminder = true,
}
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
req.disable_thinking = nil
req.extra_body = nil
-- stream_options is only valid alongside stream:true; sending it on a
-- non-streaming request is rejected by strict upstreams.
if req.stream then
if type(req.stream_options) ~= "table" then req.stream_options = {} end
req.stream_options.include_usage = true
end
if req.messages then
local kept = {}
for _, msg in ipairs(req.messages) do
if type(msg.role) == "string" and not ROLE_WHITELIST[msg.role] then
msg.role = "system"
end
-- Go 的 thinking 模式**必须**回传 reasoning_content:剥掉它会让
-- 上游直接 400("The `reasoning_content` in the thinking mode must
-- be passed back to the API"),agent 的每一轮都会失败。
-- 注意:与 opencodezen.lua 的行为**相反**,不要在这里剥。
local drop = false
if type(msg.content) == "table" then
local parts = {}
for _, part in ipairs(msg.content) do
if type(part) == "table" and part.type ~= nil and part.type ~= "text" then
-- multimodal part not supported by zen
else
table.insert(parts, part)
end
end
if #parts == 0 then
-- Content collapsed to nothing after stripping unsupported
-- parts. A message that still carries a tool call must
-- NEVER be dropped: the very next message is its tool
-- result, and dropping the call orphans that result. The
-- model then sees a result for a call it never made and
-- re-issues the same tool call on every turn (observed as
-- an infinite "repeated tool call" loop).
if (type(msg.tool_calls) == "table" and #msg.tool_calls > 0)
or msg.tool_call_id ~= nil then
msg.content = ""
else
drop = true
end
else
msg.content = parts
end
end
if not drop then
table.insert(kept, msg)
end
end
req.messages = kept
end
return json.encode(req)
end
function adapter.transform_response(raw_body)
local ok, resp = pcall(json.decode, raw_body)
if not ok or resp == nil then return raw_body end
local unified = {
content = "",
finish_reason = "",
token_usage = { prompt = 0, completion = 0, total = 0 }
}
if type(resp.usage) == "table" then
unified.token_usage.prompt = resp.usage.prompt_tokens or 0
unified.token_usage.completion = resp.usage.completion_tokens or 0
unified.token_usage.total = resp.usage.total_tokens or 0
local hit = 0
if type(resp.usage.prompt_tokens_details) == "table" and resp.usage.prompt_tokens_details.cached_tokens ~= nil then
hit = resp.usage.prompt_tokens_details.cached_tokens
unified.token_usage.prompt_tokens_details = { cached_tokens = hit }
end
if (resp.usage.prompt_cache_hit_tokens or 0) > 0 then
unified.token_usage.prompt_cache_hit_tokens = resp.usage.prompt_cache_hit_tokens
unified.token_usage.prompt_cache_miss_tokens = resp.usage.prompt_cache_miss_tokens or 0
if hit == 0 then
unified.token_usage.prompt_tokens_details = { cached_tokens = resp.usage.prompt_cache_hit_tokens }
end
end
end
if type(resp.choices) == "table" and #resp.choices > 0 then
local ch = resp.choices[1]
if type(ch.message) == "table" then
unified.content = ch.message.content or ""
local reasoning = ch.message.reasoning_content or ch.message.reasoning
if reasoning then
unified.reasoning_content = reasoning
end
if type(ch.message.tool_calls) == "table" then
local tcs = {}
for _, tc in ipairs(ch.message.tool_calls) do
local args_ok, args = pcall(json.decode, tc["function"].arguments)
if not args_ok then args = {} end
table.insert(tcs, {
id = tc.id,
type = tc.type or "function",
name = tc["function"].name,
arguments = args
})
end
unified.tool_calls = tcs
end
end
unified.finish_reason = ch.finish_reason or ""
end
return json.encode(unified)
end
function adapter.transform_stream_chunk(raw_chunk)
local ok, chunk = pcall(json.decode, raw_chunk)
if not ok then return "" end
-- OpenAI-style streams may attach usage to a chunk with empty choices
-- (the final usage chunk). Preserve it; the gateway emits it as the
-- terminal usage chunk. Note: keys must match Go's TokenUsage json tags
-- (prompt/completion/total); the gateway re-emits standard *_tokens.
local uses = nil
if type(chunk.usage) == "table" then
uses = {
prompt = chunk.usage.prompt_tokens or chunk.usage.prompt or 0,
completion = chunk.usage.completion_tokens or chunk.usage.completion or 0,
total = chunk.usage.total_tokens or chunk.usage.total or 0,
}
if type(chunk.usage.prompt_tokens_details) == "table" and chunk.usage.prompt_tokens_details.cached_tokens ~= nil then
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_tokens_details.cached_tokens }
elseif (chunk.usage.prompt_cache_hit_tokens or 0) > 0 then
uses.prompt_cache_hit_tokens = chunk.usage.prompt_cache_hit_tokens
uses.prompt_cache_miss_tokens = chunk.usage.prompt_cache_miss_tokens or 0
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_cache_hit_tokens }
end
end
if not chunk.choices or #chunk.choices == 0 then
if uses ~= nil then
-- usage-only chunk is not a content/finish signal; the gateway
-- emits its own terminal stop chunk and merges this usage.
return json.encode({ usage = uses, done = false })
end
return ""
end
local delta = chunk.choices[1].delta or {}
local fr = chunk.choices[1].finish_reason
local finish = (type(fr) == "string" and fr ~= "") and fr or nil
local unified = {
content = delta.content or "",
done = (finish ~= nil)
}
if finish then
unified.finish_reason = finish
end
if uses ~= nil then
unified.usage = uses
end
-- zen 用 reasoning 字段承载推理文本(OpenAI 惯例是 reasoning_content)
local reasoning = delta.reasoning_content or delta.reasoning
if reasoning then
unified.reasoning_content = reasoning
end
if delta.tool_calls then
-- pass raw streaming fragments through; OpenAI clients accumulate index+id+name+arguments
unified.tool_calls = delta.tool_calls
end
return json.encode(unified)
end
-- 错误收敛(可选钩子):zen 错误信封固定为 {error={type,message}};
-- 免费池限流 FreeUsageLimitError 单独标注。返回 nil 走通用兜底。
function adapter.transform_error(status, body)
local ok, resp = pcall(json.decode, body)
if not ok or type(resp) ~= "table" then return nil end
local e = resp.error
if type(e) ~= "table" then return nil end
if e.type == "FreeUsageLimitError" then
return "zen free pool quota exhausted"
end
return e.message
end
return adapter

View File

@ -0,0 +1,247 @@
local adapter = {}
adapter.name = "opencodezen"
adapter.version = "1.0.0"
adapter.endpoint = "/chat/completions"
-- OpenCode Zen(按量付费端点,https://opencode.ai/zen/v1)
--
-- 与 OpenCode Go(opencodego.lua,https://opencode.ai/zen/go/v1)是两个不同的
-- 服务,行为要求并不相同,因此各有专用适配器:
--
-- * 本适配器(Zen)服务免费池 / 按量付费池。免费模型必须带 opencode 客户端
-- 指纹(UA + x-opencode-*),否则上游拒绝:"free tier can only be used in
-- OpenCode"。Zen 的免费池以非 thinking 模型为主,历史上回传
-- reasoning_content 会引出上游报错,故这里剥掉(Go 侧相反,见 opencodego.lua)。
-- * Go 订阅读者请看 opencodego.lua:那边**必须**回传 reasoning_content,
-- 否则 thinking 模式直接 400。
--
-- 缺 x-opencode-client/session/request/project 身份头会被判为匿名客户端,
-- 部分模型在带 tools 时上游直接失败(流式返回单 chunk
-- "finish_reason":"network_error" 空 content,非流式 503
-- "Endpoint is unavailable",表现为"空回复")。
adapter.headers = {
["User-Agent"] = "opencode/1.18.21 ai-sdk/provider-utils/4.0.23 runtime/bun/1.3.14",
}
-- 每请求生成身份头。沙箱无 os/math,用 meta.timestamp + 请求体哈希派生:
-- 同秒内重复请求 id 相同可接受(zen 只校验存在性,不校验格式)。
local function rand_id(prefix, seed)
return prefix .. string.sub(sha256_hex(seed), 1, 24)
end
function adapter.build_headers(meta)
local ts = tostring(meta.timestamp or "")
return {
["User-Agent"] = adapter.headers["User-Agent"],
["x-opencode-client"] = "cli",
-- project 固定:同网关实例共享一个工作区身份
["x-opencode-project"] = string.sub(sha256_hex("llmsproxy|" .. (meta.source and meta.source.name or "")), 1, 32),
["x-opencode-session"] = rand_id("ses_", "session|" .. ts),
["x-opencode-request"] = rand_id("msg_", "request|" .. ts .. "|" .. tostring(meta.body or "")),
}
end
-- OpenAI /chat/completions format (pass-through, strip provider-specific fields)
-- zen 上游 schema 只接受 text content part(无视觉/音频能力):多模态 part
-- (image_url / input_audio / file 等)一律剥离。剥离后 content 变空的消息
-- 若不再携带 tool_calls / tool_call_id 才整条丢弃(避免上游
-- "unknown variant `image_url`, expected `text`");带工具调用的必须保留,
-- 否则会把紧随其后的 tool 结果变成孤儿,模型会反复重发同一个调用。
-- zen 上游角色白名单只有 system / user / assistant / tool / latest_reminder:
-- OpenAI 的 developer(及 function 等)不在其中,直接透传会触发上游
-- "unknown variant `developer`, expected one of ..." 错误;统一归一化为 system。
local ROLE_WHITELIST = {
system = true,
user = true,
assistant = true,
tool = true,
latest_reminder = true,
}
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
req.disable_thinking = nil
req.extra_body = nil
-- stream_options is only valid alongside stream:true; sending it on a
-- non-streaming request is rejected by strict upstreams.
if req.stream then
if type(req.stream_options) ~= "table" then req.stream_options = {} end
req.stream_options.include_usage = true
end
if req.messages then
local kept = {}
for _, msg in ipairs(req.messages) do
if type(msg.role) == "string" and not ROLE_WHITELIST[msg.role] then
msg.role = "system"
end
-- Zen 免费池不接受回传 reasoning_content(Go 侧相反)
msg.reasoning_content = nil
local drop = false
if type(msg.content) == "table" then
local parts = {}
for _, part in ipairs(msg.content) do
if type(part) == "table" and part.type ~= nil and part.type ~= "text" then
-- multimodal part not supported by zen
else
table.insert(parts, part)
end
end
if #parts == 0 then
-- Content collapsed to nothing after stripping unsupported
-- parts. A message that still carries a tool call must
-- NEVER be dropped: the very next message is its tool
-- result, and dropping the call orphans that result. The
-- model then sees a result for a call it never made and
-- re-issues the same tool call on every turn (observed as
-- an infinite "repeated tool call" loop).
if (type(msg.tool_calls) == "table" and #msg.tool_calls > 0)
or msg.tool_call_id ~= nil then
msg.content = ""
else
drop = true
end
else
msg.content = parts
end
end
if not drop then
table.insert(kept, msg)
end
end
req.messages = kept
end
return json.encode(req)
end
function adapter.transform_response(raw_body)
local ok, resp = pcall(json.decode, raw_body)
if not ok or resp == nil then return raw_body end
local unified = {
content = "",
finish_reason = "",
token_usage = { prompt = 0, completion = 0, total = 0 }
}
if type(resp.usage) == "table" then
unified.token_usage.prompt = resp.usage.prompt_tokens or 0
unified.token_usage.completion = resp.usage.completion_tokens or 0
unified.token_usage.total = resp.usage.total_tokens or 0
local hit = 0
if type(resp.usage.prompt_tokens_details) == "table" and resp.usage.prompt_tokens_details.cached_tokens ~= nil then
hit = resp.usage.prompt_tokens_details.cached_tokens
unified.token_usage.prompt_tokens_details = { cached_tokens = hit }
end
if (resp.usage.prompt_cache_hit_tokens or 0) > 0 then
unified.token_usage.prompt_cache_hit_tokens = resp.usage.prompt_cache_hit_tokens
unified.token_usage.prompt_cache_miss_tokens = resp.usage.prompt_cache_miss_tokens or 0
if hit == 0 then
unified.token_usage.prompt_tokens_details = { cached_tokens = resp.usage.prompt_cache_hit_tokens }
end
end
end
if type(resp.choices) == "table" and #resp.choices > 0 then
local ch = resp.choices[1]
if type(ch.message) == "table" then
unified.content = ch.message.content or ""
local reasoning = ch.message.reasoning_content or ch.message.reasoning
if reasoning then
unified.reasoning_content = reasoning
end
if type(ch.message.tool_calls) == "table" then
local tcs = {}
for _, tc in ipairs(ch.message.tool_calls) do
local args_ok, args = pcall(json.decode, tc["function"].arguments)
if not args_ok then args = {} end
table.insert(tcs, {
id = tc.id,
type = tc.type or "function",
name = tc["function"].name,
arguments = args
})
end
unified.tool_calls = tcs
end
end
unified.finish_reason = ch.finish_reason or ""
end
return json.encode(unified)
end
function adapter.transform_stream_chunk(raw_chunk)
local ok, chunk = pcall(json.decode, raw_chunk)
if not ok then return "" end
-- OpenAI-style streams may attach usage to a chunk with empty choices
-- (the final usage chunk). Preserve it; the gateway emits it as the
-- terminal usage chunk. Note: keys must match Go's TokenUsage json tags
-- (prompt/completion/total); the gateway re-emits standard *_tokens.
local uses = nil
if type(chunk.usage) == "table" then
uses = {
prompt = chunk.usage.prompt_tokens or chunk.usage.prompt or 0,
completion = chunk.usage.completion_tokens or chunk.usage.completion or 0,
total = chunk.usage.total_tokens or chunk.usage.total or 0,
}
if type(chunk.usage.prompt_tokens_details) == "table" and chunk.usage.prompt_tokens_details.cached_tokens ~= nil then
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_tokens_details.cached_tokens }
elseif (chunk.usage.prompt_cache_hit_tokens or 0) > 0 then
uses.prompt_cache_hit_tokens = chunk.usage.prompt_cache_hit_tokens
uses.prompt_cache_miss_tokens = chunk.usage.prompt_cache_miss_tokens or 0
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_cache_hit_tokens }
end
end
if not chunk.choices or #chunk.choices == 0 then
if uses ~= nil then
-- usage-only chunk is not a content/finish signal; the gateway
-- emits its own terminal stop chunk and merges this usage.
return json.encode({ usage = uses, done = false })
end
return ""
end
local delta = chunk.choices[1].delta or {}
local fr = chunk.choices[1].finish_reason
local finish = (type(fr) == "string" and fr ~= "") and fr or nil
local unified = {
content = delta.content or "",
done = (finish ~= nil)
}
if finish then
unified.finish_reason = finish
end
if uses ~= nil then
unified.usage = uses
end
-- zen 用 reasoning 字段承载推理文本(OpenAI 惯例是 reasoning_content)
local reasoning = delta.reasoning_content or delta.reasoning
if reasoning then
unified.reasoning_content = reasoning
end
if delta.tool_calls then
-- pass raw streaming fragments through; OpenAI clients accumulate index+id+name+arguments
unified.tool_calls = delta.tool_calls
end
return json.encode(unified)
end
-- 错误收敛(可选钩子):zen 错误信封固定为 {error={type,message}};
-- 免费池限流 FreeUsageLimitError 单独标注。返回 nil 走通用兜底。
function adapter.transform_error(status, body)
local ok, resp = pcall(json.decode, body)
if not ok or type(resp) ~= "table" then return nil end
local e = resp.error
if type(e) ~= "table" then return nil end
if e.type == "FreeUsageLimitError" then
return "zen free pool quota exhausted"
end
return e.message
end
return adapter

View File

@ -46,9 +46,6 @@ end
-- Returns (tool_calls_array_or_nil, content_with_blocks_removed).
local function parse_text_tool_calls(content)
if type(content) ~= "string" or content == "" then return nil, content end
if not (content:find("<tool_call", 1, true) or content:find("<toolcall", 1, true)) then
return nil, content
end
local tcs = {}
local idx = 0
@ -88,11 +85,31 @@ local function parse_text_tool_calls(content)
collect("<tool_call[^>]*>%s*(.-)%s*</tool_call%s*>")
collect("<toolcall[^>]*>%s*(.-)%s*</toolcall%s*>")
-- Legacy / mimicked form: models that saw past assistant turns folded as
-- [Called tool: name({...})]
-- tend to emit the same shape instead of a real block. Recover it so
-- the agent loop keeps working; otherwise the client receives plain text
-- where a structured tool call should be. Capture the name and the JSON
-- object literal independently (the JSON is the only {...} run here).
if #tcs == 0 and content:find("%[Called tool") then
for name, argsraw in content:gmatch("%[Called tool%s*:%s*([%w_%-%.]+)%s*%((%b{})%s*%)%s*%]") do
local aok, decoded = pcall(json.decode, argsraw)
idx = idx + 1
table.insert(tcs, {
id = "call_legacy_" .. idx,
type = "function",
name = name,
arguments = aok and decoded or {}
})
end
end
if #tcs == 0 then return nil, content end
-- drop the blocks from user-visible content; keep any surrounding prose
local stripped = content:gsub("<tool_call[^>]*>%s*.-%s*</tool_call%s*>", "")
stripped = stripped:gsub("<toolcall[^>]*>%s*.-%s*</toolcall%s*>", "")
stripped = stripped:gsub("%[Called tool%s*:%s*[%w_%-%.]+%s*%(%b{}%s*%)%s*%]", "")
stripped = stripped:gsub("^%s+", ""):gsub("%s+$", "")
return tcs, stripped
end

View File

@ -0,0 +1,207 @@
package lua
import (
"encoding/json"
"strings"
"testing"
)
// TestAdaptersPreserveToolCalls is the fleet-wide guard for the
// "orphaned tool result" class of bug: when an agent client replays a turn
// whose assistant message carries tool_calls, EVERY adapter must forward both
// the call and the attribution of its result. Dropping the call (or the
// tool_call_id / tool_name that ties the result to it) makes the model see a
// result for a call it never made and re-issue the same call forever.
//
// The input deliberately uses content:[] — the shape most agent clients emit
// for a tool-calling assistant turn with no text.
func TestAdaptersPreserveToolCalls(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
body := `{"model":"m","messages":[
{"role":"user","content":"read /tmp/x"},
{"role":"assistant","content":[],"tool_calls":[{"id":"call_abc","type":"function","function":{"name":"read","arguments":"{\"path\":\"/tmp/x\"}"}}]},
{"role":"tool","tool_call_id":"call_abc","content":"hello world"},
{"role":"user","content":"summarize"}
]}`
const (
callID = "call_abc"
fnName = "read"
toolOut = "hello world"
lastUser = "summarize"
)
// Adapters that identify a tool result by the call id.
byID := []string{"openai", "deepseek", "github", "groq", "kimicode", "mistral",
"sensenova", "trae", "agentrouter", "opencode", "anthropic"}
for _, a := range byID {
out, err := vm.Transform(a, "transform_request", body)
if err != nil {
t.Errorf("%s: transform_request: %v", a, err)
continue
}
if !strings.Contains(out, callID) {
t.Errorf("%s dropped the tool call id %q: %s", a, callID, out)
}
if !strings.Contains(out, fnName) {
t.Errorf("%s dropped the function name %q: %s", a, fnName, out)
}
if !strings.Contains(out, toolOut) {
t.Errorf("%s dropped the tool result: %s", a, out)
}
if !strings.Contains(out, lastUser) {
t.Errorf("%s dropped the final user turn: %s", a, out)
}
}
// Ollama identifies the result with tool_name (its API has no tool_call_id
// on assistant turns) and takes arguments as an object, not a JSON string.
out, err := vm.Transform("ollama", "transform_request", body)
if err != nil {
t.Fatalf("ollama: %v", err)
}
for _, want := range []string{`"tool_calls"`, fnName, `"tool_name"`, toolOut} {
if !strings.Contains(out, want) {
t.Errorf("ollama missing %s: %s", want, out)
}
}
// Gemini identifies the result by function NAME and has no call id in its
// wire format; both directions must be present as functionCall /
// functionResponse parts, and the system role must move to
// systemInstruction rather than sitting in contents.
gBody := `{"model":"m","messages":[
{"role":"system","content":"be brief"},
{"role":"user","content":"read /tmp/x"},
{"role":"assistant","content":[],"tool_calls":[{"id":"call_abc","type":"function","function":{"name":"read","arguments":"{\"path\":\"/tmp/x\"}"}}]},
{"role":"tool","tool_call_id":"call_abc","content":"hello world"}
]}`
gout, err := vm.Transform("gemini", "transform_request", gBody)
if err != nil {
t.Fatalf("gemini: %v", err)
}
for _, want := range []string{`"functionCall"`, `"functionResponse"`, fnName, toolOut, `"systemInstruction"`} {
if !strings.Contains(gout, want) {
t.Errorf("gemini missing %s: %s", want, gout)
}
}
// No message may keep the OpenAI-only roles inside contents.
var greq struct {
Contents []struct {
Role string `json:"role"`
} `json:"contents"`
}
if err := json.Unmarshal([]byte(gout), &greq); err != nil {
t.Fatalf("gemini unmarshal: %v (%s)", err, gout)
}
for _, c := range greq.Contents {
if c.Role != "user" && c.Role != "model" {
t.Errorf("gemini contents carry an invalid role %q: %s", c.Role, gout)
}
}
// Negative control: a text-only turn must still be forwarded verbatim.
plain := `{"model":"m","messages":[{"role":"user","content":"just text"}]}`
for _, a := range append(byID, "ollama", "gemini") {
pout, err := vm.Transform(a, "transform_request", plain)
if err != nil {
t.Errorf("%s plain: %v", a, err)
continue
}
if !strings.Contains(pout, "just text") {
t.Errorf("%s dropped plain text: %s", a, pout)
}
}
}
// TestOpenCodeStreamOptionsOnlyWhenStreaming pins the fix for
// "stream_options should be set along with stream": OpenCode Go (and other
// strict OpenAI-compatible upstreams) reject a non-streaming request that
// carries stream_options, which broke every non-stream call through the
// opencode adapter.
func TestOpenCodeStreamOptionsOnlyWhenStreaming(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
// non-streaming: stream_options must be absent entirely
out, err := vm.Transform("opencode", "transform_request",
`{"model":"m","messages":[{"role":"user","content":"hi"}]}`)
if err != nil {
t.Fatal(err)
}
if strings.Contains(out, "stream_options") {
t.Fatalf("non-streaming request must not carry stream_options: %s", out)
}
// streaming: stream_options.include_usage must be set
sout, err := vm.Transform("opencode", "transform_request",
`{"model":"m","stream":true,"messages":[{"role":"user","content":"hi"}]}`)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(sout, "stream_options") || !strings.Contains(sout, "include_usage") {
t.Fatalf("streaming request must carry stream_options.include_usage: %s", sout)
}
}
// TestOpenCodeGoVsZenReasoning pins the one behavioural difference that made a
// shared adapter wrong: OpenCode Go's thinking mode REQUIRES the assistant
// turn's reasoning_content to be echoed back ("The `reasoning_content` in the
// thinking mode must be passed back to the API"), while the Zen free pool must
// not receive it. Hence two purpose-built adapters.
func TestOpenCodeGoVsZenReasoning(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
body := `{"model":"m","messages":[
{"role":"user","content":"hi"},
{"role":"assistant","content":"","reasoning_content":"I should read the file.","tool_calls":[{"id":"call_1","type":"function","function":{"name":"read","arguments":"{\"path\":\"/x\"}"}}]},
{"role":"tool","tool_call_id":"call_1","content":"r"}
]}`
// Go: reasoning_content must survive, and the tool call must stay paired.
gout, err := vm.Transform("opencodego", "transform_request", body)
if err != nil {
t.Fatalf("opencodego: %v", err)
}
if !strings.Contains(gout, "I should read the file.") {
t.Errorf("opencodego must pass reasoning_content back (thinking mode requires it): %s", gout)
}
if !strings.Contains(gout, "call_1") || !strings.Contains(gout, "tool_call_id") {
t.Errorf("opencodego must keep the tool call paired with its result: %s", gout)
}
// Zen: reasoning_content is stripped.
zout, err := vm.Transform("opencodezen", "transform_request", body)
if err != nil {
t.Fatalf("opencodezen: %v", err)
}
if strings.Contains(zout, "I should read the file.") {
t.Errorf("opencodezen must strip reasoning_content: %s", zout)
}
if !strings.Contains(zout, "call_1") {
t.Errorf("opencodezen must still keep the tool call: %s", zout)
}
// Both must still omit stream_options on non-streaming requests.
for _, a := range []string{"opencodego", "opencodezen"} {
out, err := vm.Transform(a, "transform_request", `{"model":"m","messages":[{"role":"user","content":"hi"}]}`)
if err != nil {
t.Fatalf("%s: %v", a, err)
}
if strings.Contains(out, "stream_options") {
t.Errorf("%s must not send stream_options when not streaming: %s", a, out)
}
}
}

View File

@ -4,6 +4,7 @@ import (
"encoding/json"
"os"
"path/filepath"
"regexp"
"strings"
"sync"
"testing"
@ -220,6 +221,70 @@ func TestOpenCodeStripsMultiModalParts(t *testing.T) {
}
}
// TestOpenCodeKeepsToolCallWithEmptyContent pins the fix for an infinite
// "repeated tool call" loop: an assistant turn whose content serialises to an
// empty ARRAY while carrying tool_calls must survive the multimodal strip.
// Dropping it orphans the following tool result, so the model never sees the
// call it already made and re-issues it on every turn.
func TestOpenCodeKeepsToolCallWithEmptyContent(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
body := `{"model":"x","messages":[
{"role":"user","content":"read /tmp/x"},
{"role":"assistant","content":[],"tool_calls":[{"id":"call_abc","type":"function","function":{"name":"read","arguments":"{\"path\":\"/tmp/x\"}"}}]},
{"role":"tool","tool_call_id":"call_abc","content":"hello"},
{"role":"user","content":"now summarize"}
]}`
out, err := vm.Transform("opencode", "transform_request", body)
if err != nil {
t.Fatalf("transform_request: %v", err)
}
var req struct {
Messages []struct {
Role string `json:"role"`
Content json.RawMessage `json:"content"`
ToolCalls []struct {
ID string `json:"id"`
} `json:"tool_calls"`
ToolCallID string `json:"tool_call_id"`
} `json:"messages"`
}
if err := json.Unmarshal([]byte(out), &req); err != nil {
t.Fatalf("unmarshal: %v (%s)", err, out)
}
if len(req.Messages) != 4 {
t.Fatalf("expected 4 messages (tool call must not be dropped), got %d: %s", len(req.Messages), out)
}
asst := req.Messages[1]
if asst.Role != "assistant" || len(asst.ToolCalls) != 1 || asst.ToolCalls[0].ID != "call_abc" {
t.Fatalf("assistant tool_calls lost: %s", out)
}
if string(asst.Content) != `""` {
t.Fatalf("empty content must be rewritten to \"\", got %s", asst.Content)
}
if req.Messages[2].Role != "tool" || req.Messages[2].ToolCallID != "call_abc" {
t.Fatalf("tool result must stay paired with its call: %s", out)
}
// Negative control: a message with neither text nor tool calls is still dropped.
only := `{"model":"x","messages":[
{"role":"user","content":[{"type":"image_url","image_url":{"url":"https://ex.com/a.png"}}]},
{"role":"user","content":"keep me"}
]}`
out2, err := vm.Transform("opencode", "transform_request", only)
if err != nil {
t.Fatal(err)
}
if got := strings.Count(out2, `"role"`); got != 1 {
t.Fatalf("image-only message without tool calls must still be dropped, got %d: %s", got, out2)
}
}
func TestBuildHeadersCustomHook(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
@ -1599,3 +1664,424 @@ func TestTraeTextToolCallRecovery(t *testing.T) {
t.Errorf("plain answer finish_reason changed: %s", out)
}
}
// TestToolIDSanitize locks the tool-call id sanitizer in both adapters that
// talk to Claude upstreams. Anthropic requires tool_use.id /
// tool_result.tool_use_id to match ^[a-zA-Z0-9_-]{1,64}$ and rejects the WHOLE
// request otherwise (REQUEST_BODY_INVALID / "Invalid tool use format"), while
// OpenAI has no such rule — so an OpenAI-compatible model can mint "bash:0"
// (observed from moonshotai/kimi-k3) and poison a client's history, killing
// every Claude slot on replay. tabitoken/扇贝 are Claude-behind-OpenAI, so
// openai.lua needs the same treatment as anthropic.lua.
func TestToolIDSanitize(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
legal := regexp.MustCompile(`^[a-zA-Z0-9_-]{1,64}$`)
long := strings.Repeat("a", 200)
body := func(id string) string {
b, err := json.Marshal(map[string]interface{}{
"model": "x",
"messages": []interface{}{
map[string]interface{}{"role": "user", "content": "run whoami"},
map[string]interface{}{"role": "assistant", "content": "", "tool_calls": []interface{}{
map[string]interface{}{"id": id, "type": "function",
"function": map[string]interface{}{"name": "bash", "arguments": `{"command":"whoami"}`}},
}},
map[string]interface{}{"role": "tool", "tool_call_id": id, "content": "root"},
},
})
if err != nil {
t.Fatal(err)
}
return string(b)
}
// ---- anthropic: tool_use.id and tool_result.tool_use_id must agree ----
for _, id := range []string{"bash:0", "call_ok_123", long, "toolu~sig1:AB+/=="} {
out, err := vm.Transform("anthropic", "transform_request", body(id))
if err != nil {
t.Fatalf("anthropic transform %q: %v", id, err)
}
var r struct {
Messages []struct {
Content []struct {
Type string `json:"type"`
ID string `json:"id"`
ToolUseID string `json:"tool_use_id"`
} `json:"content"`
} `json:"messages"`
}
if err := json.Unmarshal([]byte(out), &r); err != nil {
t.Fatalf("anthropic unmarshal %q: %v (%s)", id, err, out)
}
var useID, resID string
for _, m := range r.Messages {
for _, c := range m.Content {
switch c.Type {
case "tool_use":
useID = c.ID
case "tool_result":
resID = c.ToolUseID
}
}
}
if !legal.MatchString(useID) {
t.Errorf("anthropic %q: tool_use.id %q violates Anthropic's id pattern", id, useID)
}
if useID != resID {
t.Errorf("anthropic %q: tool_use.id %q != tool_result.tool_use_id %q (unpaired call)", id, useID, resID)
}
if legal.MatchString(id) && useID != id {
t.Errorf("anthropic %q: already-legal id must pass through untouched, got %q", id, useID)
}
}
// ---- openai + agentrouter: same rewrite, and both sites must stay in sync ----
// agentrouter fronts claude-opus-4-8, so it inherits the same id rule.
for _, adapterName := range []string{"openai", "agentrouter"} {
for _, id := range []string{"bash:0", "call_ok_123", long} {
out, err := vm.Transform(adapterName, "transform_request", body(id))
if err != nil {
t.Fatalf("%s transform %q: %v", adapterName, id, err)
}
var r struct {
Messages []struct {
Role string `json:"role"`
ToolCalls []struct {
ID string `json:"id"`
} `json:"tool_calls"`
ToolCallID string `json:"tool_call_id"`
} `json:"messages"`
}
if err := json.Unmarshal([]byte(out), &r); err != nil {
t.Fatalf("%s unmarshal %q: %v (%s)", adapterName, id, err, out)
}
var callID, resID string
for _, m := range r.Messages {
if len(m.ToolCalls) > 0 {
callID = m.ToolCalls[0].ID
}
if m.Role == "tool" {
resID = m.ToolCallID
}
}
if !legal.MatchString(callID) {
t.Errorf("%s %q: tool_calls[0].id %q still illegal for Claude-behind-OpenAI upstreams", adapterName, id, callID)
}
if callID != resID {
t.Errorf("%s %q: tool_calls id %q != tool_call_id %q (unpaired call)", adapterName, id, callID, resID)
}
if legal.MatchString(id) && callID != id {
t.Errorf("%s %q: already-legal id must pass through untouched, got %q", adapterName, id, callID)
}
}
}
// ---- distinct dirty ids must NOT collapse into one (digest suffix) ----
seen := map[string]string{}
for _, id := range []string{"a:b", "a_b", "a-b", "a b", "a/b"} {
out, err := vm.Transform("anthropic", "transform_request", body(id))
if err != nil {
t.Fatalf("collision probe %q: %v", id, err)
}
var r struct {
Messages []struct {
Content []struct {
Type string `json:"type"`
ID string `json:"id"`
} `json:"content"`
} `json:"messages"`
}
if err := json.Unmarshal([]byte(out), &r); err != nil {
t.Fatal(err)
}
got := ""
for _, m := range r.Messages {
for _, c := range m.Content {
if c.Type == "tool_use" {
got = c.ID
}
}
}
if prev, dup := seen[got]; dup {
t.Errorf("ids %q and %q both map to %q — a collision makes a duplicate tool_use", prev, id, got)
}
seen[got] = id
}
// ---- determinism: same input, same output ----
a, _ := vm.Transform("anthropic", "transform_request", body("bash:0"))
b, _ := vm.Transform("anthropic", "transform_request", body("bash:0"))
if a != b {
t.Errorf("sanitizer is not deterministic:\n%s\n%s", a, b)
}
}
// TestToolIDSanitizeResponse locks the OUTBOUND half of the sanitizer. The
// request-side rewrite alone is not enough: an id minted by a permissive
// OpenAI-compatible upstream (moonshotai/kimi-k3 returns "bash:0") is handed to
// the client, stored in its history, and replayed forever after. Sanitizing on
// the way out means a poisoned id never enters a client session in the first
// place — both for non-streaming responses and for the first fragment of a
// streamed tool call (later argument fragments carry no id).
func TestToolIDSanitizeResponse(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
legal := regexp.MustCompile(`^[a-zA-Z0-9_-]{1,64}$`)
// ---- openai + agentrouter non-streaming ----
resp := `{"choices":[{"message":{"role":"assistant","content":null,"tool_calls":[
{"id":"bash:0","type":"function","function":{"name":"bash","arguments":"{\"command\":\"whoami\"}"}}]},
"finish_reason":"tool_calls"}]}`
var nonStreamID string
for _, adapterName := range []string{"openai", "agentrouter"} {
out, err := vm.Transform(adapterName, "transform_response", resp)
if err != nil {
t.Fatalf("%s transform_response: %v", adapterName, err)
}
var r struct {
ToolCalls []struct {
ID string `json:"id"`
} `json:"tool_calls"`
}
if err := json.Unmarshal([]byte(out), &r); err != nil {
t.Fatalf("%s unmarshal: %v (%s)", adapterName, err, out)
}
if len(r.ToolCalls) != 1 {
t.Fatalf("%s tool_calls lost: %s", adapterName, out)
}
if !legal.MatchString(r.ToolCalls[0].ID) {
t.Errorf("%s non-stream response leaks illegal id %q to the client", adapterName, r.ToolCalls[0].ID)
}
if nonStreamID != "" && r.ToolCalls[0].ID != nonStreamID {
t.Errorf("%s rewrote %q differently than another adapter (%q) — a client mixing sources gets unpaired calls",
adapterName, r.ToolCalls[0].ID, nonStreamID)
}
nonStreamID = r.ToolCalls[0].ID
}
// ---- streaming: first fragment carries the id (both adapters) ----
chunk := `{"choices":[{"delta":{"tool_calls":[
{"index":0,"id":"bash:0","type":"function","function":{"name":"bash","arguments":""}}]},
"finish_reason":null}]}`
var c struct {
ToolCalls []struct {
ID string `json:"id"`
} `json:"tool_calls"`
}
for _, adapterName := range []string{"openai", "agentrouter"} {
out, err := vm.Transform(adapterName, "transform_stream_chunk", chunk)
if err != nil {
t.Fatalf("%s transform_stream_chunk: %v", adapterName, err)
}
if err := json.Unmarshal([]byte(out), &c); err != nil {
t.Fatalf("%s unmarshal chunk: %v (%s)", adapterName, err, out)
}
if len(c.ToolCalls) != 1 {
t.Fatalf("%s stream tool_calls lost: %s", adapterName, out)
}
if !legal.MatchString(c.ToolCalls[0].ID) {
t.Errorf("%s stream response leaks illegal id %q to the client", adapterName, c.ToolCalls[0].ID)
}
// Streaming and non-streaming must agree, otherwise a client that mixes
// modes within one session produces unpaired tool calls.
if c.ToolCalls[0].ID != nonStreamID {
t.Errorf("%s stream id %q != non-stream id %q for the same input", adapterName, c.ToolCalls[0].ID, nonStreamID)
}
}
// ---- a later argument fragment has no id and must stay id-less ----
argChunk := `{"choices":[{"delta":{"tool_calls":[
{"index":0,"function":{"arguments":"{\"command\":\"whoami\"}"}}]},"finish_reason":null}]}`
for _, adapterName := range []string{"openai", "agentrouter"} {
out, err := vm.Transform(adapterName, "transform_stream_chunk", argChunk)
if err != nil {
t.Fatalf("%s arg fragment: %v", adapterName, err)
}
if strings.Contains(out, `"id"`) {
t.Errorf("%s argument fragment gained an id (breaks index-based accumulation): %s", adapterName, out)
}
}
// ---- anthropic response side: tool_use id and stream start ----
var r struct {
ToolCalls []struct {
ID string `json:"id"`
} `json:"tool_calls"`
}
aResp := `{"content":[{"type":"tool_use","id":"toolu~sig1:AB+/==","name":"bash","input":{"command":"whoami"}}],"stop_reason":"tool_use"}`
out, err := vm.Transform("anthropic", "transform_response", aResp)
if err != nil {
t.Fatalf("anthropic transform_response: %v", err)
}
if err := json.Unmarshal([]byte(out), &r); err != nil {
t.Fatalf("unmarshal anthropic: %v (%s)", err, out)
}
if len(r.ToolCalls) != 1 || !legal.MatchString(r.ToolCalls[0].ID) {
t.Errorf("anthropic response leaks illegal id: %s", out)
}
aChunk := `{"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu~sig1:AB+/==","name":"bash"}}`
out, err = vm.Transform("anthropic", "transform_stream_chunk", aChunk)
if err != nil {
t.Fatalf("anthropic stream chunk: %v", err)
}
if err := json.Unmarshal([]byte(out), &c); err != nil {
t.Fatalf("unmarshal anthropic chunk: %v (%s)", err, out)
}
if len(c.ToolCalls) != 1 || !legal.MatchString(c.ToolCalls[0].ID) {
t.Errorf("anthropic stream leaks illegal id: %s", out)
}
}
// TestAnthropicUsageCacheMapping pins the Anthropic→OpenAI usage conversion.
// The two APIs disagree on what the prompt count means: Anthropic's
// input_tokens EXCLUDES cached blocks (cache_read_input_tokens and
// cache_creation_input_tokens are separate, additive, billed input), while
// OpenAI's prompt_tokens INCLUDES its cached_tokens subset. Mapping
// input_tokens straight onto prompt undercounted the billed prompt by the whole
// cache portion and could report cached_tokens > prompt_tokens (a hit rate over
// 100%); cache_creation_input_tokens was dropped entirely.
func TestAnthropicUsageCacheMapping(t *testing.T) {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
defer vm.Stop()
type usage struct {
Prompt int `json:"prompt"`
Completion int `json:"completion"`
Total int `json:"total"`
Details *struct {
CachedTokens int `json:"cached_tokens"`
} `json:"prompt_tokens_details"`
Hit int `json:"prompt_cache_hit_tokens"`
Miss int `json:"prompt_cache_miss_tokens"`
}
// ---- non-streaming, cache read + cache write reported ----
resp := `{"content":[{"type":"text","text":"ok"}],"stop_reason":"end_turn",
"usage":{"input_tokens":1200,"cache_read_input_tokens":40000,
"cache_creation_input_tokens":500,"output_tokens":80}}`
out, err := vm.Transform("anthropic", "transform_response", resp)
if err != nil {
t.Fatalf("transform_response: %v", err)
}
// Each case decodes into a FRESH value: json.Unmarshal leaves fields absent
// from the payload untouched, so a reused struct would carry the previous
// case's prompt_tokens_details into a response that has none.
decode := func(out string) usage {
t.Helper()
var r struct {
TokenUsage usage `json:"token_usage"`
}
if err := json.Unmarshal([]byte(out), &r); err != nil {
t.Fatalf("unmarshal: %v (%s)", err, out)
}
return r.TokenUsage
}
decodeChunk := func(out string) (usage, string) {
t.Helper()
var c struct {
Usage usage `json:"usage"`
FinishReason string `json:"finish_reason"`
}
if err := json.Unmarshal([]byte(out), &c); err != nil {
t.Fatalf("unmarshal chunk: %v (%s)", err, out)
}
return c.Usage, c.FinishReason
}
u := decode(out)
if u.Prompt != 1200+40000+500 {
t.Errorf("prompt must sum fresh+cache_read+cache_creation (41700), got %d", u.Prompt)
}
if u.Total != u.Prompt+u.Completion {
t.Errorf("total %d != prompt %d + completion %d", u.Total, u.Prompt, u.Completion)
}
if u.Details == nil || u.Details.CachedTokens != 40000 {
t.Errorf("cached_tokens must carry cache_read (40000): %s", out)
}
if u.Details != nil && u.Details.CachedTokens > u.Prompt {
t.Errorf("cached_tokens %d > prompt %d implies a hit rate above 100%%",
u.Details.CachedTokens, u.Prompt)
}
if u.Hit != 40000 || u.Miss != 1200+500 {
t.Errorf("hit/miss split wrong: hit=%d miss=%d (want 40000/1700)", u.Hit, u.Miss)
}
// ---- a reported zero hit must stay distinguishable from "not reported" ----
zero := `{"content":[],"stop_reason":"end_turn","usage":{"input_tokens":100,"cache_read_input_tokens":0,"output_tokens":5}}`
out, err = vm.Transform("anthropic", "transform_response", zero)
if err != nil {
t.Fatal(err)
}
if decode(out).Details == nil {
t.Errorf("a reported cache_read of 0 must still emit prompt_tokens_details: %s", out)
}
// An upstream that never mentions caching must not gain a fabricated split
// (justwoker reports only input_tokens/output_tokens plus its own cost fields).
none := `{"content":[],"stop_reason":"end_turn","usage":{"input_tokens":6931,"output_tokens":1}}`
out, err = vm.Transform("anthropic", "transform_response", none)
if err != nil {
t.Fatal(err)
}
plain := decode(out)
if plain.Details != nil {
t.Errorf("upstream reported no cache fields; details must be absent: %s", out)
}
if plain.Prompt != 6931 || plain.Total != 6932 {
t.Errorf("plain usage mismapped: %s", out)
}
// ---- streaming: message_delta carries the FINAL usage and must map cache ----
delta := `{"type":"message_delta","delta":{"stop_reason":"end_turn"},
"usage":{"input_tokens":1200,"cache_read_input_tokens":40000,"output_tokens":80}}`
out, err = vm.Transform("anthropic", "transform_stream_chunk", delta)
if err != nil {
t.Fatalf("stream delta: %v", err)
}
du, fin := decodeChunk(out)
if du.Prompt != 41200 {
t.Errorf("stream final prompt must include cache_read: got %d", du.Prompt)
}
if du.Details == nil || du.Details.CachedTokens != 40000 {
t.Errorf("stream final usage dropped the cache split: %s", out)
}
if fin != "stop" {
t.Errorf("finish_reason regressed: %s", out)
}
// message_start's placeholder count must not suppress the later real one:
// justwoker reports input_tokens=160 at message_start and 6931 at
// message_delta, and the gateway's mergeUsage lets the later value win.
start := `{"type":"message_start","message":{"usage":{"input_tokens":160,"output_tokens":1}}}`
out, err = vm.Transform("anthropic", "transform_stream_chunk", start)
if err != nil {
t.Fatalf("message_start: %v", err)
}
if su, _ := decodeChunk(out); su.Prompt != 160 {
t.Errorf("message_start usage lost: %s", out)
}
// A usage-less chunk must stay silent rather than reporting zeros.
quiet := `{"type":"message_start","message":{}}`
out, err = vm.Transform("anthropic", "transform_stream_chunk", quiet)
if err != nil {
t.Fatal(err)
}
if strings.Contains(out, "usage") {
t.Errorf("usage-less chunk must not emit a usage object: %s", out)
}
}

View File

@ -13,6 +13,7 @@ import (
"log"
"net"
"net/http"
"net/url"
"strconv"
"strings"
"sync"
@ -24,6 +25,20 @@ import (
"llmsproxy/internal/types"
)
// proxyFor picks the transport proxy for one source. An explicit per-source
// proxy_url wins (e.g. clash on 127.0.0.1:7890 for a geo-blocked upstream);
// otherwise fall back to the process-wide env proxy, which is direct by
// default.
func proxyFor(cfg config.Source) func(*http.Request) (*url.URL, error) {
if cfg.ProxyURL != "" {
proxyURL, err := url.Parse(cfg.ProxyURL)
if err == nil {
return http.ProxyURL(proxyURL)
}
}
return http.ProxyFromEnvironment
}
// ---- per-(source,model) scheduling state ----
// ModelState is the scheduling state of one (source, model) pair: a soft
@ -388,8 +403,11 @@ func New(cfg config.Source, vm *lua.VM) *Provider {
// Shared transport: ResponseHeaderTimeout bounds how long we wait for the
// first response byte (applies to both paths); the stream client has no
// client-level Timeout so the SSE body can run past the header timeout.
// A per-source ProxyURL (e.g. clash on 127.0.0.1:7890) overrides the
// process-wide env proxy for upstreams that are geo/IP-blocked; sources
// without one keep http.ProxyFromEnvironment (direct by default).
tr := &http.Transport{
Proxy: http.ProxyFromEnvironment,
Proxy: proxyFor(cfg),
DialContext: (&net.Dialer{Timeout: 30 * time.Second, KeepAlive: 30 * time.Second}).DialContext,
ForceAttemptHTTP2: true,
MaxIdleConns: 100,

View File

@ -3,6 +3,7 @@
package types
import (
"bytes"
"encoding/json"
"errors"
"strings"
@ -61,6 +62,39 @@ type ChatMessage struct {
func StringContent(s string) json.RawMessage { b, _ := json.Marshal(s); return b }
// isEmptyJSONArray reports whether raw is the literal empty JSON array [].
func isEmptyJSONArray(raw json.RawMessage) bool {
t := bytes.TrimSpace(raw)
return len(t) == 2 && t[0] == '[' && t[1] == ']'
}
// UnmarshalJSON decodes a chat message, normalising an empty-array content
// ("content":[]) to an empty string and dropping an empty tool_calls array.
//
// Why this exists: Lua adapters json.decode the request and re-encode it, and
// an empty Lua table is indistinguishable from an empty JSON array — the
// encoder emits {} for both. Agent clients serialise an assistant turn that
// carries tool_calls and no text as content:[], so every pass-through adapter
// turned it into content:{} — a shape that is not valid OpenAI (content is
// string | array of parts | null) and that real upstreams reject with
// "400 invalid arguments". Normalising at the decode boundary fixes every
// adapter at once, including ones added later.
func (m *ChatMessage) UnmarshalJSON(b []byte) error {
type alias ChatMessage
var a alias
if err := json.Unmarshal(b, &a); err != nil {
return err
}
*m = ChatMessage(a)
if isEmptyJSONArray(m.Content) {
m.Content = StringContent("")
}
if isEmptyJSONArray(m.ToolCalls) {
m.ToolCalls = nil
}
return nil
}
type ToolCall struct {
ID string `json:"id"`
Type string `json:"type"`

View File

@ -68,3 +68,52 @@ func TestTokenUsageMarshalCacheFields(t *testing.T) {
t.Logf("DeepSeek JSON: %s", string(b))
t.Logf("OpenAI JSON: %s", string(b2))
}
// TestChatMessageNormalizesEmptyArrayContent pins the fix for the malformed
// content:{} that pass-through adapters produced for an assistant turn carrying
// tool_calls with no text. Lua adapters cannot tell an empty JSON array from an
// empty object, so the decode boundary normalises it for every adapter at once.
func TestChatMessageNormalizesEmptyArrayContent(t *testing.T) {
var msg ChatMessage
raw := `{"role":"assistant","content":[],"tool_calls":[{"id":"call_1","type":"function","function":{"name":"read","arguments":"{}"}}]}`
if err := json.Unmarshal([]byte(raw), &msg); err != nil {
t.Fatal(err)
}
if string(msg.Content) != `""` {
t.Fatalf("empty-array content must normalise to \"\", got %s", msg.Content)
}
if len(msg.ToolCalls) == 0 {
t.Fatal("tool_calls must survive normalisation")
}
// A non-empty content array must be left byte-identical (multimodal path).
var mm ChatMessage
multi := `{"role":"user","content":[{"type":"text","text":"hi"}]}`
if err := json.Unmarshal([]byte(multi), &mm); err != nil {
t.Fatal(err)
}
if string(mm.Content) != `[{"type":"text","text":"hi"}]` {
t.Fatalf("multimodal content must pass through untouched, got %s", mm.Content)
}
// A plain string content is untouched.
var sm ChatMessage
if err := json.Unmarshal([]byte(`{"role":"user","content":"plain"}`), &sm); err != nil {
t.Fatal(err)
}
if string(sm.Content) != `"plain"` {
t.Fatalf("string content must pass through untouched, got %s", sm.Content)
}
// An omitted content stays omitted (omitempty semantics preserved).
var om ChatMessage
if err := json.Unmarshal([]byte(`{"role":"assistant","tool_calls":[]}`), &om); err != nil {
t.Fatal(err)
}
if om.Content != nil {
t.Fatalf("absent content must stay absent, got %s", om.Content)
}
if om.ToolCalls != nil {
t.Fatalf("empty tool_calls array must be dropped, got %s", om.ToolCalls)
}
}