mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-11-07 17:57:23 +00:00
Compare commits
43 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 06a74375d8 | |||
| de9080e06c | |||
| 80e79f8dc7 | |||
| b6bbff19e9 | |||
| d5bbb0eca5 | |||
| 9cb6254166 | |||
| 4514270e21 | |||
| ef95bc45d0 | |||
| f7e926ccb7 | |||
| 688dce88d1 | |||
| de7553eaa2 | |||
| 79bc55cfbe | |||
| 4b9fb7b056 | |||
| 7941b4110a | |||
| 5ccf068db3 | |||
| b4c5f0f0fc | |||
| 442fa5e3f6 | |||
| 0caa37ea6a | |||
| 33cb2bdb06 | |||
| 0ae7d2c8be | |||
| 02c01582f1 | |||
| 6454b094bc | |||
| 771b95a133 | |||
| 441de434f5 | |||
| 9feef8f4cb | |||
| b5c89fd3d7 | |||
| 7f193a997d | |||
| 055705c134 | |||
| 401a7ead27 | |||
| 411503aea5 | |||
| 662f8f02b6 | |||
| 2113bfc125 | |||
| c9f34538ec | |||
| 12bfd53f16 | |||
| 25970a2135 | |||
| 6f2d4dc6eb | |||
| aff8354c8a | |||
| 091f790734 | |||
| 6eeb5832a3 | |||
| a189c4f61c | |||
| 58073a6cbe | |||
| a5923b3797 | |||
| 91f6d1f32f |
497
HANDOFF_2026-10-19.md
Normal file
497
HANDOFF_2026-10-19.md
Normal file
@ -0,0 +1,497 @@
|
||||
# ModelRouter / llmsproxy 交接文档
|
||||
|
||||
更新时间:2026-10-19
|
||||
工作目录:`/home/program/llmsproxy`
|
||||
当前分支:`release/v2.0.x`
|
||||
|
||||
## 1. 当前 Git 状态
|
||||
|
||||
最近提交:
|
||||
|
||||
- `c9f3453 fix(codearts): 识别 InferHub 未注册模型`
|
||||
- `12bfd53 fix: 收口渠道源迁移与测试页合并`
|
||||
- `25970a2 refactor(scheduler)!: 移除 priority 废弃字段 + 一通道一源 + discover 复用凭据`
|
||||
|
||||
`c9f3453` 已推送到 `origin/release/v2.0.x`。
|
||||
|
||||
注意:当前工作树在最后一轮添加 provider 回归测试时出现了一个**未完成且可能无法编译的测试编辑**。最后一次工具报错:
|
||||
|
||||
```text
|
||||
internal/provider/provider_test.go:99: missing ',' in argument list
|
||||
```
|
||||
|
||||
下一位 agent 接手后的第一步必须是:
|
||||
|
||||
```bash
|
||||
cd /home/program/llmsproxy
|
||||
git status --short
|
||||
gofmt -w internal/provider/provider_test.go 2>&1 || true
|
||||
go test -tags luajit ./internal/provider/ 2>&1
|
||||
```
|
||||
|
||||
如果测试文件处于半编辑状态,先检查 `TestProviderChatStreamNormalizesStopAfterToolCall`,修正 Go 字符串转义或删除该未完成测试后再继续。不要直接部署当前工作树。
|
||||
|
||||
## 2. 生产服务状态
|
||||
|
||||
生产服务:`llmsproxy.service`
|
||||
|
||||
- 当前服务 active
|
||||
- 监听:`0.0.0.0:8081`
|
||||
- 二进制已通过 `./deploy.sh` 原子替换并重启
|
||||
- 最近一次部署健康检查通过
|
||||
- 当前生产插件 SHA256 前缀曾核对为:`1bf26cbb3abcb107`
|
||||
- 当前生产二进制在最近部署后已更新
|
||||
|
||||
不要把密钥写入交接记录。生产 admin key 保存在本机运维环境中,不应复制到聊天、文档或提交。
|
||||
|
||||
## 3. Channel 源当前状态
|
||||
|
||||
生产源已经从旧的“一账号一源”收口成“一渠道一源”:
|
||||
|
||||
```text
|
||||
channels-trae
|
||||
channels-minimax
|
||||
channels-lobsterai
|
||||
channels-buddy
|
||||
channels-codearts
|
||||
channels-zcode
|
||||
```
|
||||
|
||||
旧形态源已经清理:
|
||||
|
||||
```text
|
||||
channels-<provider>-<account_id>
|
||||
```
|
||||
|
||||
此前旧源和新源并存,造成同一渠道同一模型出现多份,只能靠账号 hex 区分。
|
||||
|
||||
根因已修复:内核 `list_sources` 返回裸数组,但旧插件代码按 `{sources = [...]}` 信封读取,导致迁移清理循环永远为空。当前 `channels.lua` 已兼容:
|
||||
|
||||
- 裸数组
|
||||
- `{sources = [...]}`
|
||||
- `{json = {sources = [...]}}`
|
||||
|
||||
相关代码:
|
||||
|
||||
```text
|
||||
internal/lua/plugins/channels.lua: syncProviderSource
|
||||
internal/core/channels_legacy_source_cleanup_test.go
|
||||
```
|
||||
|
||||
当前生产源列表已实测只有 6 个新形态源。
|
||||
|
||||
## 4. CodeArts 当前状态
|
||||
|
||||
### 已确认可用的部分
|
||||
|
||||
CodeArts 使用:
|
||||
|
||||
```text
|
||||
adapter = codearts
|
||||
base_url = https://snap-access.cn-north-4.myhuaweicloud.com
|
||||
```
|
||||
|
||||
CodeArts 的 chat 请求需要完整 AK/SK/security token 凭据,由插件适配器签名,不是裸 Bearer token。
|
||||
|
||||
真实流式工具调用测试已经成功:
|
||||
|
||||
请求形状:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "channels-codearts:openpangu-2.0-flash",
|
||||
"messages": [{"role": "user", "content": "Use the weather tool for Beijing."}],
|
||||
"tools": [{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_weather",
|
||||
"description": "Get weather",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {"city": {"type": "string"}},
|
||||
"required": ["city"]
|
||||
}
|
||||
}
|
||||
}],
|
||||
"tool_choice": "auto",
|
||||
"stream": true,
|
||||
"max_tokens": 64
|
||||
}
|
||||
```
|
||||
|
||||
结果:
|
||||
|
||||
- HTTP 200
|
||||
- `Content-Type: text/event-stream`
|
||||
- 返回结构化 `tool_calls`
|
||||
- 函数名 `get_weather`
|
||||
- 参数最终为 `{"city":"Beijing"}`
|
||||
- `finish_reason = tool_calls`
|
||||
|
||||
因此 CodeArts 的流式工具调用链路当前是通的。
|
||||
|
||||
### CodeArts 模型刷新
|
||||
|
||||
生产调用:
|
||||
|
||||
```http
|
||||
POST /api/plugins/http/channels/model/refresh
|
||||
{"platform":"codearts"}
|
||||
```
|
||||
|
||||
实测返回:
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"results": [
|
||||
{"platform": "codearts", "ok": true, "count": 11}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
CodeArts 模型目录来自两个签名端点并合并:
|
||||
|
||||
1. `opengw.developer.huaweicloud.com/api/v1/gateway/config`
|
||||
- `result.models`
|
||||
2. `snap-access.cn-north-4.myhuaweicloud.com/v1/model/builtin`
|
||||
- `builtinModels`
|
||||
|
||||
相关解析:
|
||||
|
||||
```text
|
||||
internal/lua/plugins/channels.lua: parseRemoteModels
|
||||
internal/lua/plugins/channels.lua: refreshRemoteModels
|
||||
internal/core/channels_codearts_catalog_test.go
|
||||
```
|
||||
|
||||
### 已确认失效并禁用的 CodeArts 模型
|
||||
|
||||
下面 4 个模型曾收到明确的上游 404:
|
||||
|
||||
```text
|
||||
CodeArts-Pangu
|
||||
deepseek-v4-flash-0731
|
||||
deepseek-v4-pro-0813
|
||||
glm-5.3-flash
|
||||
```
|
||||
|
||||
生产上已通过 `channels/model/set_disabled` 禁用。它们仍可能在 `model/list` 中出现,但 `disabled=true`,不会作为可用模型路由。
|
||||
|
||||
新增的死模型判据:
|
||||
|
||||
```lua
|
||||
model is not registered
|
||||
```
|
||||
|
||||
对应提交:`c9f3453`。
|
||||
|
||||
注意:当前插件的 dead-model 机制只在收到真实请求失败后记录模型;刷新目录不会自动根据“远端目录没有该模型”删除模型,这是刻意设计,避免误删远端目录不完整的平台模型。
|
||||
|
||||
CodeArts 当前仍有一些模型返回 429/TPM 限流,例如:
|
||||
|
||||
```text
|
||||
deepseek-v4-flash
|
||||
deepseek-v4.1-flash
|
||||
```
|
||||
|
||||
429 不是模型失效,不能禁用。
|
||||
|
||||
## 5. 其他 Channel 实测结果
|
||||
|
||||
### Trae
|
||||
|
||||
真实流式 + tools 请求:
|
||||
|
||||
```text
|
||||
channels-trae:DeepSeek-V4-Flash
|
||||
```
|
||||
|
||||
结果:
|
||||
|
||||
- HTTP 200
|
||||
- SSE 正常返回
|
||||
- 结构化 tool call 返回
|
||||
- 但上游最后的结束原因可能是 `stop`,即使前面已经出现 tool-call fragments
|
||||
|
||||
这会让 Agent 把工具调用误判成普通文本完成。
|
||||
|
||||
当前准备中的修复在统一 provider 流式层:
|
||||
|
||||
```text
|
||||
internal/provider/provider.go
|
||||
```
|
||||
|
||||
逻辑是:只要本次流出现真实 tool-call 片段,后续 `FinishReason == "stop"` 就归一成 `tool_calls`。
|
||||
|
||||
另外,Trae 的非流式接口偶尔返回 SSE。当前 `trae.lua` 对非流式 SSE 只取最后一条 data,不能完整合并多条工具调用/推理片段。这是后续仍需处理的协议问题,但不能用非流式手工请求结果推断 Pi 的真实请求模式。
|
||||
|
||||
### Buddy
|
||||
|
||||
真实流式 + tools 请求:
|
||||
|
||||
```text
|
||||
channels-buddy:glm-5.3-flash
|
||||
```
|
||||
|
||||
结果:
|
||||
|
||||
- HTTP 200
|
||||
- SSE 正常返回
|
||||
- tool-call 参数分片可见
|
||||
- 最终有 `finish_reason = tool_calls`
|
||||
- 但某些上游 chunk 的 `tool_calls` 可能是空对象 `{}`
|
||||
|
||||
空对象不是合法的 OpenAI tool-call 数组,部分 Agent 客户端会拒绝。
|
||||
|
||||
当前准备中的修复在:
|
||||
|
||||
```text
|
||||
internal/lua/adapters/buddy.lua
|
||||
```
|
||||
|
||||
只在 `type(delta.tool_calls) == "table" and #delta.tool_calls > 0` 时输出 `tool_calls`,避免 Lua 空表被编码为 `{}`。
|
||||
|
||||
Buddy 的非流式错误:
|
||||
|
||||
```text
|
||||
11101: Non-stream chat request is currently not supported
|
||||
```
|
||||
|
||||
之前出现的该错误来自手工 `curl` 测试中显式设置 `stream:false`,不能归因到 Pi。真实 `stream:true + tools` 已经证明 Buddy 流式工具调用可以工作。
|
||||
|
||||
### MiniMax
|
||||
|
||||
生产日志反复出现:
|
||||
|
||||
```text
|
||||
adapter "anthropic": status=401 body={"error":"invalid access token"}
|
||||
provider channels-minimax: api error 401
|
||||
```
|
||||
|
||||
这是凭据失效问题,不是 tools 形状问题。
|
||||
|
||||
当前源:
|
||||
|
||||
```text
|
||||
channels-minimax
|
||||
adapter = anthropic
|
||||
endpoint = /mavis/api/v1/llm/v1/messages
|
||||
```
|
||||
|
||||
需要用户重新登录/提供有效凭据。不要擅自刷新一次性 refresh token,之前已经发生过 refresh token 被消耗后无法再次使用的事故。
|
||||
|
||||
## 6. 当前未提交的代码改动
|
||||
|
||||
最后一轮已经改动但尚未完成提交:
|
||||
|
||||
```text
|
||||
internal/provider/provider.go
|
||||
internal/lua/adapters/trae.lua
|
||||
internal/lua/adapters/buddy.lua
|
||||
internal/provider/provider_test.go # 可能有半编辑、需先修复
|
||||
```
|
||||
|
||||
已完成的意图:
|
||||
|
||||
1. provider 流式层记录 `sawToolCalls`
|
||||
2. 看到真实 tool-call 后,把后续 `stop` 归一为 `tool_calls`
|
||||
3. Trae 不输出空工具调用表
|
||||
4. Buddy 不输出空工具调用表
|
||||
5. provider 回归测试模拟“tool fragment + stop”场景
|
||||
|
||||
但 provider 测试最后一次编辑工具报错:
|
||||
|
||||
```text
|
||||
internal/provider/provider_test.go:99: missing ',' in argument list
|
||||
```
|
||||
|
||||
因此必须先修测试文件,再提交。
|
||||
|
||||
## 7. 推荐接手顺序
|
||||
|
||||
### 第一步:修复工作树
|
||||
|
||||
```bash
|
||||
cd /home/program/llmsproxy
|
||||
git status --short
|
||||
sed -n '1,150p' internal/provider/provider_test.go
|
||||
gofmt -w internal/provider/provider_test.go 2>&1 || true
|
||||
go test -tags luajit ./internal/provider/ 2>&1
|
||||
```
|
||||
|
||||
若测试函数字符串转义难以修复,先删除半成品测试,再重新用 raw string 写测试。
|
||||
|
||||
### 第二步:跑定向验证
|
||||
|
||||
```bash
|
||||
luac -p internal/lua/adapters/trae.lua
|
||||
luac -p internal/lua/adapters/buddy.lua
|
||||
go test -tags luajit ./internal/provider/
|
||||
go test -tags luajit ./internal/lua/ -run 'Adapter|Buddy|Trae|Stream'
|
||||
```
|
||||
|
||||
### 第三步:构建并部署临时验证版
|
||||
|
||||
修复确认后:
|
||||
|
||||
```bash
|
||||
go build ./...
|
||||
./deploy.sh
|
||||
```
|
||||
|
||||
部署前确认:
|
||||
|
||||
```bash
|
||||
sha256sum internal/lua/plugins/channels.lua
|
||||
sha256sum /etc/llmsproxy/plugins/channels.lua
|
||||
systemctl is-active llmsproxy
|
||||
```
|
||||
|
||||
### 第四步:真实流式回归
|
||||
|
||||
不要用 `stream:false` 测试流式问题。使用:
|
||||
|
||||
```bash
|
||||
KEY='<从本机安全环境读取,不写入文档>'
|
||||
TOOL='{"type":"function","function":{"name":"get_weather","description":"Get weather","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}'
|
||||
|
||||
curl --no-buffer --max-time 90 \
|
||||
-H "Authorization: Bearer $KEY" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "{\"model\":\"channels-codearts:openpangu-2.0-flash\",\"messages\":[{\"role\":\"user\",\"content\":\"Use the weather tool for Beijing.\"}],\"tools\":[$TOOL],\"tool_choice\":\"auto\",\"stream\":true,\"max_tokens\":64}" \
|
||||
http://127.0.0.1:8081/v1/chat/completions
|
||||
```
|
||||
|
||||
分别将 model 替换为:
|
||||
|
||||
```text
|
||||
channels-codearts:openpangu-2.0-flash
|
||||
channels-trae:DeepSeek-V4-Flash
|
||||
channels-buddy:glm-5.3-flash
|
||||
```
|
||||
|
||||
验收:
|
||||
|
||||
- HTTP 200
|
||||
- `Content-Type: text/event-stream`
|
||||
- 不能出现 `tool_calls: {}`
|
||||
- 有 tool-call fragments 时最终必须是 `finish_reason: tool_calls`
|
||||
- CodeArts 当前应继续返回结构化工具调用
|
||||
|
||||
### 第五步:CodeArts 刷新验收
|
||||
|
||||
```bash
|
||||
curl -sS -X POST \
|
||||
-H "Authorization: Bearer $KEY" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"platform":"codearts"}' \
|
||||
http://127.0.0.1:8081/api/plugins/http/channels/model/refresh
|
||||
```
|
||||
|
||||
预期:
|
||||
|
||||
```json
|
||||
{"ok":true,"results":[{"platform":"codearts","ok":true,"count":11}]}
|
||||
```
|
||||
|
||||
再检查:
|
||||
|
||||
```bash
|
||||
curl -sS -X POST \
|
||||
-H "Authorization: Bearer $KEY" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"platform":"codearts"}' \
|
||||
http://127.0.0.1:8081/api/plugins/http/channels/model/list
|
||||
```
|
||||
|
||||
预期:远端目录 11 个 + 兜底补位,已确认 404 的 4 个仍保持 `disabled=true`。
|
||||
|
||||
## 8. 发布/安装包状态
|
||||
|
||||
目前没有完成“全部安装包”发布闭环。
|
||||
|
||||
已经发生的事情:
|
||||
|
||||
- 核心包构建成功,但脚本实际使用了 `git describe`,产物版本类似:
|
||||
`2.0.40~4-gc9f3453`
|
||||
- Linux GUI 包构建成功,版本为 `1.9.7`
|
||||
- Windows 主机 Wine 构建失败,生成了一个 264KB 的坏 `ModelRouter Setup 1.9.7.exe`,绝对不能发布
|
||||
- Docker Windows 构建也失败在拉取 Docker Hub 基础镜像 `golang:1.25` 网络超时
|
||||
- apt 无法找到 `wine32:i386`
|
||||
- 旧的 1.9.5/1.9.6 Windows 包是历史产物,不能作为当前版本
|
||||
|
||||
不要把旧 Windows exe 当作新包发布。
|
||||
|
||||
仓库已有 Docker 方案:
|
||||
|
||||
```text
|
||||
cmd/gui/scripts/dist-win-docker.sh
|
||||
cmd/gui/docker/win-builder/Dockerfile
|
||||
```
|
||||
|
||||
但当前 Docker Hub 网络不可用。若要继续做 Windows 包:
|
||||
|
||||
1. 在可访问 Docker Hub 的构建机/CI 执行 Docker 方案;或
|
||||
2. 提供包含 Wine32/Wine64 的完整构建环境;或
|
||||
3. 暂时只发布 Linux/core 包,并明确说明 Windows 包未完成。
|
||||
|
||||
版本脚本注意:
|
||||
|
||||
- 核心包脚本使用位置参数:
|
||||
|
||||
```bash
|
||||
./packaging/core-dist.sh 2.0.41
|
||||
```
|
||||
|
||||
不是 `VERSION=2.0.41 ./packaging/core-dist.sh`。
|
||||
- GUI 版本来自 `cmd/gui/package.json`,环境变量 `VERSION` 不会自动修改 Electron 产品版本。
|
||||
- 清理旧产物后再打包,并在全部包完成后统一计算 `SHA256SUMS`。
|
||||
|
||||
## 9. 参考插件状态
|
||||
|
||||
用户要求按:
|
||||
|
||||
```text
|
||||
https://github.com/Ebony-Vinyl/dsh-our-free-model
|
||||
```
|
||||
|
||||
逐厂商对照参考插件。
|
||||
|
||||
当前本地路径:
|
||||
|
||||
```text
|
||||
/tmp/pi-github-repos/Ebony-Vinyl/dsh-our-free-model
|
||||
```
|
||||
|
||||
但该目录当前不是有效可读源码副本:
|
||||
|
||||
- `vendor/channel-pack/src` 下面只有空目录;
|
||||
- `.git` 没有有效 `HEAD`/refs;
|
||||
- 无法从本地 Git 对象恢复源码;
|
||||
- 本地也没有 `buddy.ts`、`trae.ts`、`codearts.ts` 等文件。
|
||||
|
||||
因此不能声称当前适配器是逐行照参考插件移植的。若要继续按用户要求完成,必须先重新 clone/恢复该 GitHub 仓库源码,确认以下文件实际存在后再对照:
|
||||
|
||||
```text
|
||||
vendor/channel-pack/src/trae.ts
|
||||
vendor/channel-pack/src/buddy.ts
|
||||
vendor/channel-pack/src/codearts.ts
|
||||
vendor/channel-pack/src/*adapter*
|
||||
vendor/channel-pack/src/*model*
|
||||
```
|
||||
|
||||
当前仓库代码只能作为临时协议修复基础,不能代替参考插件源码。
|
||||
|
||||
## 10. 关键结论
|
||||
|
||||
- Pi 是否发送流式请求,历史日志没有保存入站 `stream/tools` 字段,不能靠旧日志判断。
|
||||
- 本次真实 `stream:true + tools` 测试证明 ModelRouter 流式路径能工作:
|
||||
- CodeArts:通过;
|
||||
- Trae:通过但结束原因需归一;
|
||||
- Buddy:通过但需过滤空 `{}`;
|
||||
- MiniMax:凭据 401,独立问题。
|
||||
- 不要再用 `stream:false` 的手工请求推断 Pi 的问题。
|
||||
- CodeArts 刷新接口当前真实返回 `ok=true,count=11`,刷新链路不是当前阻塞点。
|
||||
- 当前最紧急的代码状态是修复 `provider_test.go` 的半编辑,然后验证未提交的统一流式修复。
|
||||
- 未经测试、未重新构建、未重新部署,不要继续改生产账号或刷新一次性凭据。
|
||||
14
Makefile
14
Makefile
@ -1,8 +1,8 @@
|
||||
.PHONY: all build test gui gui-dev gui-dist gui-deb gui-win gui-win-docker help clean
|
||||
.PHONY: all build test gui gui-dev gui-dist gui-deb gui-win gui-win-docker help clean gui-test
|
||||
|
||||
# ModelRouter - single-binary gateway + optional Electron desktop GUI (embedded core)
|
||||
export BINARY
|
||||
BINARY=llmsproxy
|
||||
BINARY=modelrouter
|
||||
export BUILD_DIR
|
||||
BUILD_DIR=build
|
||||
export GUI_DIR
|
||||
@ -20,7 +20,7 @@ all: build
|
||||
# ---- core binary (server users / embedded core) ----
|
||||
build:
|
||||
@mkdir -p $(BUILD_DIR)
|
||||
CGO_ENABLED=1 go build -tags luajit -trimpath -o $(BUILD_DIR)/$(BINARY) ./cmd/llmsproxy
|
||||
CGO_ENABLED=1 go build -tags luajit -trimpath -o $(BUILD_DIR)/$(BINARY) ./cmd/modelrouter
|
||||
@echo "Built: $(BUILD_DIR)/$(BINARY) ($(VERSION))"
|
||||
|
||||
# ---- core distribution packages (deb + rpm + tar.gz) -----
|
||||
@ -65,6 +65,13 @@ gui-dist-dir:
|
||||
test:
|
||||
go test ./...
|
||||
|
||||
# Desktop shell unit tests (no Electron needed): plugin-tree seeding/migration.
|
||||
# These guard the two things that silently break the desktop build:
|
||||
# - plugin_dir missing from an embedded profile -> channels plugin invisible
|
||||
# - bundled plugin files never refreshed -> old channels.lua survives upgrades
|
||||
gui-test:
|
||||
@cd $(GUI_DIR) && node --test plugin-sync.test.js
|
||||
|
||||
clean:
|
||||
rm -rf $(BUILD_DIR) cmd/build cmd/gui/dist cmd/gui/build/gui-dist cmd/gui/bin
|
||||
|
||||
@ -75,6 +82,7 @@ help:
|
||||
@echo " make gui run desktop GUI (dev)"
|
||||
@echo " make gui-dev run GUI with devtools"
|
||||
@echo " make gui-dist build deb + rpm + AppImage (linux)"
|
||||
@echo " make gui-test desktop shell unit tests (plugin tree seeding)"
|
||||
@echo " make gui-deb build deb only"
|
||||
@echo " make gui-win build Windows nsis (needs host mingw + wine)"
|
||||
@echo " make gui-win-docker build Windows nsis all-in-docker (no host mingw needed)"
|
||||
|
||||
674
README.md
674
README.md
@ -4,103 +4,46 @@
|
||||
>
|
||||
> **AI-assisted**: 本项目使用 AI 辅助编程(代码与文档由 AI 协作完成,经人工复核)。
|
||||
|
||||
面向多 llms 订阅者,部署在内网,实现一次配置多个服务共同使用的效果。支持指定模型或 auto 模式,按照配置的优先级选择可用模型提供服务。
|
||||
面向多 LLM 订阅者的内网网关:一次配置,多个服务共用一组上游额度。支持指定模型或
|
||||
`AUTO` 模式,按配置的优先级选择可用模型提供服务。
|
||||
|
||||
> **轻量且增源无需重编译**:网关本体是单 Go 二进制(约 10MB,零运行时依赖)。新增/切换上游
|
||||
> 只需在 `config.yaml`(或 WebUI)加一个 `sources` 条目或挂一个 `.lua` 适配器——**不改 Go、
|
||||
> 不重编译**。源与适配器的改动经 WebUI 提交时即时生效(热更新);直接编辑
|
||||
> `config.yaml` 或 `adapter_dir` 下的 `.lua` 文件则需要重启进程生效。
|
||||
网关本体是单个 Go 二进制(约 10 MB,零运行时依赖)。新增或切换上游只需在
|
||||
`config.yaml`(或 WebUI)加一个 `sources` 条目、或挂一个 `.lua` 适配器——**不改 Go、
|
||||
不重编译**。源与适配器的改动经 WebUI 提交时即时生效(热更新);直接编辑
|
||||
`config.yaml` 或 `adapter_dir` 下的 `.lua` 文件需要重启进程生效。
|
||||
|
||||
## 核心优势
|
||||
## 核心能力
|
||||
|
||||
### 极致轻量
|
||||
|
||||
- **单二进制**:编译后 8~12MB(`-s -w` strip 后约 8MB),零运行时依赖(仅依赖系统 libc),部署即用
|
||||
- **低内存占用**:与源数量相关,非与运行时长相关——单源 ~10MB、16 源生产实例 ~32–35MB([实测分解与调优](#内存占用实测与调优))
|
||||
- **日志按需加载**:统计数字来自**全量**审计日志(启动时流式扫一遍即释放,29MB/22 万行约 260ms),但原始记录不常驻内存——默认只加载首屏,下滚自动分页,CSV 导出流式写出(O(1) 内存),离页即释放
|
||||
- **零运行时依赖**:纯 Go + LuaJIT 静态链接,无需安装 Python/Node/Java 等运行时
|
||||
- **启动极快**:冷启动 < 200ms,热重载配置 < 10ms
|
||||
|
||||
### 强大的多租户调度能力
|
||||
|
||||
- **多密钥多租户**:支持无限密钥,每个密钥独立角色、模型范围、Token 配额、重置周期
|
||||
- **按模型配额**:每把 key 的每个模型单独配 token + 请求数配额与重置周期(小时/周/月/自定义 N 小时);一个模型用满只影响该模型,同 key 其它模型照常;耗尽返 429 + `Retry-After` 并点名模型,admin key 永不受限
|
||||
- **AUTO 智能调度**:基于优先级档位的分级调度,同优先级源自动轮询负载均衡,故障自动毫秒级故障转移
|
||||
- **自愈冷却**:冷却上限 5 分钟,过半后放行 1 个探测请求,上游/额度恢复即刻回归轮询,无需等满冷却窗口
|
||||
- **Token 配额管理**:精确到模型级别的 Token 配额控制,支持小时/周/月/自定义小时周期自动重置
|
||||
- **同优先级源负载均衡**:同一优先级档位的多个源,请求自动 Round-Robin 均匀分发,故障毫秒级故障转移
|
||||
- **Source-Model 前缀路由**:支持 `source-model`/`source:model`/`source/model` 精确指定上游源
|
||||
|
||||
### 生产级可靠性
|
||||
|
||||
- **热加载配置**:WebUI 修改源/密钥/适配器/AUTO链即时生效,无需重启
|
||||
- **密钥加密落盘**:AES-256-GCM 加密存储 `api_key`、网关密钥、请求头,master.key 0600 权限保护
|
||||
- **实时源探测**:`GET {base}/models` 定期探测,毫秒级感知上游状态,不污染调度退避状态
|
||||
- **全链路审计**:HTTP 访问日志、登录/配置变更审计、请求记录 CSV 导出、密钥用量统计 CSV 导出
|
||||
|
||||
### 灵活的协议适配
|
||||
|
||||
- **LuaJIT VM**:每个适配器独立 VM + **弹性 worker 池**(按需扩缩容,空闲时自动回收),安全并发,Lua 脚本热加载无需重启
|
||||
- **Lua 适配器协议**:`transform_request` / `transform_response` / `transform_stream_chunk` 双向转换
|
||||
- **动态请求头钩子**:`build_headers(meta)` 支持 HMAC 签名、动态 Header 注入
|
||||
- **多模态透传**:`image_url` 等多模态内容在多源间无损透传,Anthropic/Gemini/Ollama 自动转换
|
||||
|
||||
## 实现
|
||||
|
||||
统一 OpenAI 兼容网关:把多个上游 LLM 源(DeepSeek、Qijiar、OpenAI、Anthropic、Gemini、
|
||||
Groq、Mistral、Ollama、KimiCode…)通过 **Lua 适配器** 做协议转换,对内网暴露
|
||||
一个标准 `OpenAI Chat Completions` 接口(`/v1/chat/completions` + `/v1/models`),
|
||||
支持单次结算、SSE 流式、**AUTO 模型路由**与**多模态**透传。
|
||||
|
||||
从 [HomeAgent](https://gitcode.com/JianFeeeee/HomeAgent) 的多源 LLM 适配层
|
||||
(`internal/agent/api/provider.go` + `internal/lua/adapters/*`)抽离而来并独立演进。
|
||||
|
||||
## 特性
|
||||
|
||||
- **多源**:一个进程内配置任意多个上游源,按请求的 `model` 自动路由。
|
||||
- **AUTO 模式**:`default_model: AUTO` 时按优先级页保存的 AUTO 链档位逐档调度并发请求。
|
||||
- **统一输出**:所有源对外都是 OpenAI 格式(含 `reasoning_content`、`tool_calls`、`usage`)。
|
||||
- **生图**:`POST /v1/images/generations`,`kind: image` 的模型独立路由。
|
||||
- **多模态**:`content` 数组(`image_url` 等)在多源间无损透传,Anthropic/Gemini/Ollama 自动转换。
|
||||
- **LuaJIT VM**:基于 golua 绑定的 LuaJIT,每个适配器独立 VM + worker 池,安全并发。
|
||||
池是 **弹性** 的:`Σ max_concurrent` 只是上限而非预分配,状态按需创建、空闲时自动回收,
|
||||
因此空载网关几乎不持有 Lua 状态(启动 0 个)。
|
||||
- 扩容步长由适配器最大并发决定:`clamp(ceil(max/8), 1, 8)`,且仅在**真实并发争用**
|
||||
(所有现有状态均已占用)时批量预热,顺序流量始终只用 1 个状态。
|
||||
- 缩容步长由当前连接数决定:`clamp(ceil(冗余/(1+占用)), 1, 冗余)`——无连接时一轮收到
|
||||
常驻下限,忙时每轮只释放 1 个,保护热路径。WebUI「适配器」页展示实时池指标。
|
||||
- **thinking 预算**:请求里的推理开关/预算归一为中立的 `thinking:{enabled,budget_tokens}`,
|
||||
适配器映射到各自协议(Anthropic `thinking.budget_tokens`、Gemini `thinkingConfig`、
|
||||
DeepSeek `thinking.type="disabled"`)。
|
||||
- **Lua 适配协议**:每个源挂一个 `.lua` 适配器,完成 `transform_request` /
|
||||
`transform_response` / `transform_stream_chunk` 双向转换,协议差异全在 Lua 层。
|
||||
- **签名 / 请求头钩子**:适配器可定义 `build_headers(meta)`,在 Go 发 HTTP 前
|
||||
动态注入/签名请求头——用于云端 API 校验调用方 app(如 KimiCode 只放行特定
|
||||
agent)。提供 `hmac_sha256_hex` / `sha256_hex` / `base64_encode` 等签名辅助。
|
||||
- **WebUI**:内置管理界面,可在线查看/新增/编辑上游源与模型、配置 AUTO 优先级链、
|
||||
管理密钥模型范围,写入运行时文件持久化。
|
||||
- **密钥加密存储**:运行时文件中的上游 `api_key`、自定义请求头值、网关 key 均以
|
||||
AES-256-GCM 加密落盘(`master.key` 独立 0600,或 `LLMS_PROXY_MASTER_KEY`)。
|
||||
- **实时源探测**:状态页探测各源可达性(`GET {base}/models`,失败回退最小请求),
|
||||
不污染正常调度的退避状态,错误信息在 UI 可悬停查看。
|
||||
- **鉴权**:网关自身用 `gateway_keys` 校验客户端 Bearer key;与上游各自的 key 相互独立。
|
||||
- **流式**:SSE `chat.completion.chunk`,含角色首包与 `[DONE]` 收尾。
|
||||
- **全链路审计**:HTTP 访问日志、登录/配置变更审计、请求记录 CSV 导出、密钥用量统计 CSV 导出
|
||||
- **AES-256-GCM 加密存储**:运行时文件敏感字段加密落盘,master.key 0600 权限,支持环境变量注入主密钥
|
||||
- **统一接口**:对内暴露 OpenAI 兼容的 `/v1/chat/completions`、`/v1/models`、
|
||||
`/v1/images/generations`;所有源对外都是同一套格式(含 `reasoning_content`、
|
||||
`tool_calls`、`usage`)。
|
||||
- **多源 + AUTO 链**:一个进程内任意多个上游源;`default_model: AUTO` 时按优先级页
|
||||
保存的档位逐档调度,源故障自动降级并冷却,恢复后半冷却探测自愈。
|
||||
- **协议适配在 Lua 层**:每个源挂一个 `.lua` 适配器,完成请求/响应/流式分片的双向
|
||||
转换,并可用 `build_headers(meta)` 注入签名头。协议差异不进 Go 内核。
|
||||
- **插件独立**:业务扩展(渠道接入、计费、额度巡检)是外置 Lua 插件,由插件目录加载,
|
||||
不编进二进制、不随发行包分发;插件只通过 `modelrouter` SDK 与内核交互。
|
||||
- **多租户密钥**:网关 key 与上游 key 相互独立,按 key 限定模型范围、AUTO 链与配额。
|
||||
- **加密存储**:上游 key、自定义请求头、网关 key 均以 AES-256-GCM 落盘,
|
||||
`master.key` 单独 0600 权限(或由 `MODELROUTER_MASTER_KEY` 注入)。
|
||||
- **可观测**:状态页实时探测源可达性;HTTP 访问日志、配置变更审计、请求与用量 CSV 导出;
|
||||
写操作不落在请求热路径上。
|
||||
|
||||
## 快速开始
|
||||
|
||||
```bash
|
||||
GOMODCACHE=... GOPROXY=off go build -tags luajit -o llmsproxy ./cmd/llmsproxy
|
||||
./llmsproxy -config config.yaml # 首次运行自动生成默认配置并打印随机 admin key
|
||||
# 需要 LuaJIT 开发库;-tags luajit 是必须的
|
||||
GOMODCACHE=... GOPROXY=off go build -tags luajit -o modelrouter ./cmd/modelrouter
|
||||
|
||||
# 首次运行会生成默认配置,并在日志里打印随机 admin key
|
||||
./modelrouter -config config.yaml
|
||||
```
|
||||
|
||||
> 依赖 [golua](https://github.com/aarzilli/golua)(LuaJIT 绑定)。**必须**带 `-tags luajit`
|
||||
> 构建,否则默认走内置超集 gopher-lua 路径(行为略有差异)。
|
||||
> 依赖 [golua](https://github.com/aarzilli/golua)(LuaJIT 绑定)。不带 `-tags luajit`
|
||||
> 会走 gopher-lua 路径,行为略有差异。
|
||||
|
||||
> **配置不入库**:仓库不携带任何 config 文件(配置文件含密钥)。二进制首次运行时
|
||||
> 会在 `-config` 指定的位置生成默认配置:随机 admin key(打印在启动日志中)、
|
||||
> 仅绑定 `127.0.0.1:8080`。首次登录后请在 WebUI「密钥」页更换管理员密钥。
|
||||
> **配置不入库**:仓库不携带任何 config 文件(其中含密钥)。默认配置只绑定
|
||||
> `127.0.0.1:8080`,首次登录后请在 WebUI「密钥」页更换管理员密钥。
|
||||
|
||||
```bash
|
||||
# 无 key -> 401
|
||||
@ -117,526 +60,73 @@ curl -N -H "Authorization: Bearer $KEY" \
|
||||
http://127.0.0.1:8080/v1/chat/completions
|
||||
```
|
||||
|
||||
任何 OpenAI SDK 把 `base_url` 指到网关地址、`api_key` 用任意一个已创建并授权的网关 key 即可。
|
||||
任何 OpenAI SDK 把 `base_url` 指向网关、`api_key` 换成任一已授权的网关 key 即可。
|
||||
|
||||
## 配置
|
||||
## 文档
|
||||
|
||||
首次运行在 `-config` 路径自动生成默认配置(默认 `./config.yaml`),核心字段:
|
||||
| 主题 | 文档 |
|
||||
|---|---|
|
||||
| 实现与完整特性清单 | [docs/features.md](docs/features.md) |
|
||||
| 配置:`config.yaml`、自动探测模型、多密钥 | [docs/configuration.md](docs/configuration.md) |
|
||||
| 路由:模型路由、AUTO 链、冷却与半冷却、源与适配器、源模板 | [docs/routing.md](docs/routing.md) |
|
||||
| 适配器:Lua 协议、内置适配器清单 | [docs/adapters.md](docs/adapters.md) · [docs/lua-adapters.md](docs/lua-adapters.md)([EN](docs/lua-adapters-en.md)) |
|
||||
| 插件:SDK 契约、钩子、UI 扩展、HTTP 端点 | [docs/plugins.md](docs/plugins.md) |
|
||||
| 桌面端:Headless / Desktop 两种产物、构建与运行 | [docs/gui.md](docs/gui.md) |
|
||||
| 性能:实测内存占用与调优、thinking 预算 | [docs/performance.md](docs/performance.md) |
|
||||
| WebUI 使用 | [docs/webui.md](docs/webui.md) |
|
||||
| 开发:目录职责、测试、分支与发布纪律 | [docs/development.md](docs/development.md) · [docs/git-workflow.md](docs/git-workflow.md)([EN](docs/git-workflow-en.md)) |
|
||||
| 部署与迁移 | [docs/deployment.md](docs/deployment.md) |
|
||||
|
||||
```yaml
|
||||
listen: 127.0.0.1:8080 # 网关监听地址(建议绑内网/回环)
|
||||
gateway_keys: [sk-gw-<随机>] # 初始 admin 密钥(每次安装随机生成,仅首启写入运行时存储用)
|
||||
default_model: AUTO # model 无法路由时自动按优先级链选源
|
||||
adapter_dir: adapters # Lua 适配目录;目录不存在时首启创建并 seed 内置,存在则只读
|
||||
runtime_file: runtime.json # WebUI 编辑的源/密钥/AUTO 链持久化到此文件
|
||||
## 从 llmsproxy 改名
|
||||
|
||||
sources:
|
||||
- name: deepseek
|
||||
base_url: https://api.deepseek.com
|
||||
api_key: sk-... # 也可用 api_key_env: SOME_ENV 引用环境变量(不落盘明文)
|
||||
adapter: deepseek
|
||||
max_concurrent: 8
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
priority: 100 # YAML 源首次启动会 seed 进 AUTO 链的初值
|
||||
kind: chat
|
||||
- id: deepseek-v4-pro
|
||||
priority: 60
|
||||
kind: chat
|
||||
# 静态请求头(比适配器默认优先)
|
||||
headers: { X-Tenant: prod }
|
||||
# 传给 Lua build_headers 的透传元数据
|
||||
meta: { app_id: x, app_secret: y }
|
||||
temperature: 0.7
|
||||
max_tokens: 4096
|
||||
timeout: 120s # 请求超时,默认 120s
|
||||
```
|
||||
产品名早已是 ModelRouter,二进制、配置目录、服务名与环境变量现已统一为
|
||||
`modelrouter`:
|
||||
|
||||
| 旧 | 新 |
|
||||
|---|---|
|
||||
| `llmsproxy`(二进制) | `modelrouter` |
|
||||
| `/etc/llmsproxy/` | `/etc/modelrouter/` |
|
||||
| `llmsproxy.service` | `modelrouter.service` |
|
||||
| `LLMSPROXY_*` | `MODELROUTER_*` |
|
||||
|
||||
### 自动探测模型列表
|
||||
|
||||
不想手抄模型 id 时,只填 URL 和 key,让网关去扫:
|
||||
已部署的机器用迁移脚本升级(**默认 dry-run**,复制不移动,可回滚):
|
||||
|
||||
```bash
|
||||
# 只探测,不写任何东西
|
||||
curl -X POST http://127.0.0.1:8080/api/sources/discover \
|
||||
-H "Authorization: Bearer sk-gw-…" -H "Content-Type: application/json" \
|
||||
-d '{"base_url":"https://api.vendor.com/v1","api_key":"sk-…"}'
|
||||
# → {"ok":true,"base_url":"…","endpoint":"","models":["gpt-x",…],
|
||||
# "image_models":["gpt-image-2"],"models_path":"/models","latency_ms":1387,…}
|
||||
|
||||
# 探测后直接落库
|
||||
curl -X POST http://127.0.0.1:8080/api/sources/discover/apply \
|
||||
-H "Authorization: Bearer sk-gw-…" -H "Content-Type: application/json" \
|
||||
-d '{"base_url":"https://api.vendor.com/v1","api_key":"sk-…","name":"vendor"}'
|
||||
sudo ./packaging/migrate-to-modelrouter.sh # 预演
|
||||
sudo ./packaging/migrate-to-modelrouter.sh --apply # 执行
|
||||
sudo ./packaging/migrate-to-modelrouter.sh --rollback --apply
|
||||
```
|
||||
|
||||
WebUI 源编辑框里也有「自动探测模型」按钮:填好 URL 和 key 点一下,列表自动
|
||||
填好,仍需点保存才生效。
|
||||
|
||||
几点值得知道:
|
||||
|
||||
- **URL 形态不用猜。** 厂商对「`/v1` 放 base 还是放 path」没有共识,探测会
|
||||
依次尝试若干候选并返回答对的那个,同时给出归一化后的 `base_url` +
|
||||
`endpoint`(`endpoint` 平时留空,因为 openai adapter 自带 `/chat/completions`)。
|
||||
- **`image_endpoint` 是另一套规则。** 网关的图像默认路径是**绝对**的
|
||||
`/v1/images/generations`,而 chat 默认是相对的。所以 `base_url` 带 `/v1` 时
|
||||
必须显式给 `image_endpoint: /images/generations`,否则图像请求 404。
|
||||
探测会替你填好。
|
||||
- **图像模型只看上游自己声明的 modality**,绝不从模型名猜。`gpt-image-2`
|
||||
这类 id 也出现在同时提供对话服务的目录上,猜错会让 AUTO 链选中一个只支持
|
||||
图像生成的模型,其后每一层都被它拖垮。
|
||||
- **不会逐模型探活。** 34 个模型的目录跑一轮要花钱且耗时数分钟,`/models`
|
||||
是足够便宜的信号;模型是否真能应答留给正常请求去发现。
|
||||
- **对已存在的源,必须显式 `replace_models: true`** 才会覆盖模型列表。重新
|
||||
探测顺手抹掉为 AUTO 链手工调过的 priority,是最难排查的一类数据丢失。
|
||||
- **`api_key` 留空表示沿用已存值**,不会把凭据清空。
|
||||
|
||||
|
||||
`config.yaml`、`runtime.json`、`master.key`、`adapters/` 均不提交版本库(见
|
||||
`.gitignore`),请随备份带走——`master.key` 丢失后运行时密文无法解密。
|
||||
|
||||
运行文件生成在 `-config` 所在目录:`adapter_dir` / `runtime_file` 自动指向
|
||||
配置文件旁边的 `adapters/` 与 `runtime.json`,从任意工作目录启动都可用。
|
||||
|
||||
运行时文件(`runtime_file`)中的敏感字段自动加密:
|
||||
|
||||
- 加密算法 AES-256-GCM,格式 `enc:v1:<base64>`。
|
||||
- 主密钥来源:环境变量 `LLMS_PROXY_MASTER_KEY`(64 位 hex);否则读取
|
||||
`runtime_file` 同目录的 `master.key`;都不存在时首次启动自动生成 `master.key`(0600)。
|
||||
- 升级时旧明文文件自动兼容:首次运行正常读取,任何 UI 保存操作触发全文件加密迁移。
|
||||
- 注意:**master.key 丢失后密文无法解密**,请随配置一起备份;切勿提交到版本库。
|
||||
|
||||
### 网关密钥(多密钥)
|
||||
|
||||
网关鉴权采用「多密钥 + 角色 + 模型范围」架构,密钥持久化在 `runtime_file` 的
|
||||
`keys` 字段(加密存储):
|
||||
|
||||
- `gateway_keys` 配置只是**初始 admin 密钥种子**:首次启动迁移为运行时 admin
|
||||
key,之后不再参与鉴权管理。
|
||||
- **重要:首次启动后请在 WebUI「密钥」页更换管理员密钥**——初始密钥明文写在
|
||||
`config.yaml` 里(每次安装随机生成),继续使用存在被盗风险;用新密钥登录后删除初始密钥。
|
||||
- WebUI **密钥页**可创建/删除密钥;每个密钥可指定 `admin`(管理全部)或
|
||||
`user`(仅看自己的 key)角色,并配置**模型范围**(模型 + 源 + token 配额 +
|
||||
重置周期)。
|
||||
- 客户端用任意一个已授权的密钥明文作为 Bearer(`Authorization: Bearer <key>`)。
|
||||
- 删除密钥即从运行时存储移除,立即失效。
|
||||
|
||||
#### 密钥用量配额
|
||||
|
||||
**配额按模型单独设置**:每个密钥的 `models[]` 里,每一条模型范围各自带一份
|
||||
token 配额与请求数配额。一个模型用满只影响该模型,同一密钥的其它模型照常工作。
|
||||
|
||||
```yaml
|
||||
keys:
|
||||
- key: sk-gw-<hex>
|
||||
role: user
|
||||
name: agent-alice
|
||||
models:
|
||||
- model: deepseek-v4-flash
|
||||
token_quota: 1000000 # 本周期内该 key 用这个模型的 token 预算
|
||||
req_quota: 20000 # 本周期内的请求次数
|
||||
period: nhour # "" | hour | week | month | nhour
|
||||
hours: 6 # 仅 nhour
|
||||
- model: AUTO
|
||||
token_quota: 5000000 # AUTO 也是一条独立配额
|
||||
period: day # ← 这种写法会被拒绝(词表只有 hour/week/month/nhour)
|
||||
- model: kimi-k3 # 未列配额 = 无限
|
||||
```
|
||||
|
||||
- **没有「整钥总配额」**:这是刻意的设计。整钥总额会让「把 A 模型的额度挪给
|
||||
B 模型」变成一次全局重分配;按模型独立计费则每个模型各自可控,运维可以
|
||||
看出哪个模型吃掉了预算。
|
||||
- `period` 词表:空 = 永不过期(累计总量),`hour` / `week` / `month` = 固定窗口,
|
||||
`nhour` + `hours` = 自定义小时数。**拼错的周期在写入时就被拒**,不会静默变成
|
||||
永不过期。
|
||||
- 配额严格按密钥隔离,且**同一密钥内按模型隔离**:A 密钥用满 `m1` 不会消耗
|
||||
B 密钥的额度,同一密钥的 `m2` 也不受影响。
|
||||
- 配额统计含聊天、流式、生图,跨重启从审计日志回放(保留 40 天,覆盖最长的
|
||||
month 窗口)。
|
||||
- 配额耗尽返回 **429 + `Retry-After`**(`rate_limit_exceeded`),消息里点名是哪个
|
||||
模型用满了,客户端可等窗口重置后自动恢复;模型越权才是 403。
|
||||
**admin 密钥永不受配额限制**(其 scope 上的配额也不强制),避免把管理员
|
||||
锁在门外。
|
||||
- 窗口用量按整点小时分桶统计,实际释放比配置窗口最多晚 1 小时(配额宁可晚释放
|
||||
也不超发)。
|
||||
- `PUT /api/keys/{key}` 提交 `models` 即同时提交它们的配额(配额就是 scope 的一部分,
|
||||
不存在会与模型列表脱节的第二份预算)。显式 `0` = 解除该模型的限制。
|
||||
|
||||
##### 配额拒绝 vs 容量拒绝:两种「拒绝」含义不同
|
||||
|
||||
配额在**入口**检查,不占用任何上游槽位,因此是廉价拒绝;容量不足则要走
|
||||
`busyWait` 有界等待后才返回 503。实测(容量 8、0.6s/请求、100 并发):
|
||||
|
||||
| 情况 | 状态码 | 延迟 | 是否打到上游 |
|
||||
|---|---|---|---|
|
||||
| 配额用尽(`req_quota` 50 → 100 并发) | `429 rate_limit_exceeded` + `Retry-After` | **19–21ms** | 否 |
|
||||
| 容量打满(同一拓扑,未限配额) | `503 upstream_error` | ~2.6s(`busyWait`) | 超出部分否 |
|
||||
|
||||
客户端因此可以区分「等窗口重置」与「等上游腾出容量」:前者只需等待,后者
|
||||
通常该降并发或换源。
|
||||
|
||||
##### 配额的开销
|
||||
|
||||
配额桶按 (密钥, 模型, 整点小时) 分桶保留 40 天,实测(AMD 7840HS):
|
||||
|
||||
- 窗口查询按窗口长度而非保留总量扫描:1h 窗口 49ns、24h 窗口 55ns、
|
||||
30d 窗口 3.9μs(此前全扫保留总量,960 桶时 5.9μs)。
|
||||
- 记录一条请求:283ns、3 分配(与引入配额前相同,分配来自 ring buffer)。
|
||||
- 内存:生产形态(7 密钥 × 8 模型 × 2 源 × 满 40 天 retention)约 **3.7MB**。
|
||||
按源 pin 的 `source::model` 桶**惰性创建**——只有当某条配额真的 pin 了
|
||||
某个源时才维护,否则每条记录多写一份桶,在 20 密钥 × 8 模型 × 3 源下会
|
||||
多占 18MB。
|
||||
代价:配置 pinned 配额之前的历史用量无法事后按源拆分,所以 **pinned 配额的
|
||||
首个窗口可能少算**。
|
||||
|
||||
```bash
|
||||
# 配额耗尽时客户端看到(点名了具体模型)
|
||||
HTTP/1.1 429 Too Many Requests
|
||||
Retry-After: 2100
|
||||
{"error":{"type":"rate_limit_exceeded","message":"token quota exceeded for \"deepseek-v4-flash\" (5000000/5000000)"}}
|
||||
```
|
||||
|
||||
### 模型路由
|
||||
|
||||
`/v1/chat/completions` 的 `model` 解析顺序:
|
||||
|
||||
1. `source/model` 或 `source:model` 前缀 → 指定源;
|
||||
2. 精确匹配某个源的 `model`;
|
||||
3. 配置 `default_model: AUTO` 时 → 走优先级页保存的 AUTO 链(见下);
|
||||
4. 否则回落(返回错误)。
|
||||
|
||||
任何 OpenAI 客户端,只要 `model` 设为某个源的 `name/任意名`,即可锁定走该源;
|
||||
设为 `AUTO`(或网关配了 `default_model: AUTO`)即自动按优先级链选源。
|
||||
|
||||
### AUTO 链(优先级页)
|
||||
|
||||
AUTO 调度**只**由优先级页保存的规则决定,源配置里的 `models[].priority` 数字不再
|
||||
参与调度、也不再显示。**聊天与生图是两套独立的链**,在优先级页面通过「聊天 / 生图」
|
||||
按钮切换编辑:
|
||||
|
||||
- **聊天链**(`runtime_file` 的 `auto` 字段):服务 `POST /v1/chat/completions`
|
||||
的 `model: AUTO` 请求;只接受 `kind: chat` 模型,配置在链里的生图槽位会被忽略。
|
||||
- **生图链**(`runtime_file` 的 `auto_image` 字段):服务 `POST /v1/images/generations`
|
||||
的 `model: AUTO` 请求;只接受 `kind: image` 模型。未配置生图链时,AUTO 生图回落
|
||||
到「所有含生图模型的源、按注册顺序」的旧行为,兼容旧配置。
|
||||
|
||||
每条规则 = 一个「槽位」:`{ model, source, tier, token_quota, period, hours }`。
|
||||
|
||||
- `tier` 表示优先级档位:同一档的模型并排、共享该优先级;档位从上到下递减。
|
||||
- 同一模型可配置多个槽位(如 A 源低配 → B 源低配 → A 源高配 → B 源高配),按档位顺延。
|
||||
- `token_quota` > 0 时该槽位在重置周期内用满即顺延到下一槽位;`period` 支持
|
||||
`hour` / `week` / `month` / `nhour`(配合 `hours`),空 = 不限。
|
||||
- 同 tier 内按「偏好分」排序,失败槽位冷却后自动跳过;冷却 / 配额耗尽 / hard
|
||||
错误会顺延到下一 tier,全部失败时返回 503 并附上每档失败摘要。
|
||||
|
||||
### 冷却与半冷却探测(自愈调度)
|
||||
|
||||
槽位失败后进入指数退避冷却(5s → 10s → 20s …,上限 **5 分钟**)。冷却不是全时段
|
||||
硬闸:
|
||||
|
||||
- **前半段完全静默**:刚失败的槽位不接任何流量,避免对故障上游持续施压。
|
||||
- **后半段放行 1 个探测**:窗口过半后,同一 `(源, 模型)` 最多允许 **一个** 请求作为
|
||||
探测通过。探测槽数 = `clamp(max_concurrent / 10, 1, 2)`——`max_concurrent: 10`
|
||||
的源恰好放行 1 个探测请求。
|
||||
- **探测排在最后**:同 tier 内探测候选永远排在健康候选之后,只有没有正常槽可用时
|
||||
才承接真实流量,因此探测不会抢走可用容量。
|
||||
- **探测即真实请求**:成功则立即清零失败计数与冷却,槽位当场回到正常轮询——
|
||||
上游恢复/额度恢复后不必等满整个冷却窗口。失败则重开一个新窗口,下次探测顺延到
|
||||
新窗口的中点,不会连续重试。
|
||||
|
||||
分类冷却策略:
|
||||
|
||||
| 情况 | 冷却 | 说明 |
|
||||
|---|---|---|
|
||||
| 传输错误 / 5xx | 5s→10s→20s…,上限 5min | 指数退避 |
|
||||
| 401 / 403 凭据错误 | 10min | 换 key 后由下一次探测自动接回 |
|
||||
| 429 限流 | 30s 固定 | 「太快」不等于「坏了」,不进指数阶梯 |
|
||||
| 配额耗尽(`insufficient_quota` / `余额不足` 等) | 对齐配额窗口,上限 30min | **不进指数阶梯、不涨失败计数**,额度恢复即可用 |
|
||||
|
||||
WebUI「优先级」页的槽位健康标签会区分 `冷却` / `待探测` / `探测中`,悬停可看窗口
|
||||
起点、探测放行时刻与完全恢复时刻。
|
||||
|
||||
### 源(Source)与适配器(Adapter)的关系
|
||||
|
||||
- **源** 是到某个上游的连接描述:`name`、`base_url`、`api_key`、模型列表与优先级。
|
||||
- **适配器** 是协议转换逻辑(Lua 脚本):把统一 OpenAI 格式请求转成上游原生格式,
|
||||
再把上游响应/流式分块转回统一格式。
|
||||
- 一个适配器可被多个源复用(如 `openai.lua` 同时服务多个 OpenAI 兼容站点);
|
||||
同一个源也可以切换不同适配器(改 `adapter` 字段即可)。
|
||||
- 源决定“连谁、暴露哪些模型”,适配器决定“怎么对话”——二者在 `sources[]` 条目中
|
||||
通过 `adapter` 字段关联。
|
||||
|
||||
加载流程(`internal/core` 的 `Core` 负责装配):
|
||||
|
||||
1. 启动时 `lua.NewVM(adapter_dir)` 加载全部适配器:内置适配器(编译期 embed)+
|
||||
`adapter_dir` 下的同名覆盖文件;内置文件写在代码内,覆盖文件要求更高优先级。
|
||||
2. `config.Load` 读取 `config.yaml`,`config.NewStore(runtime_file)` 读 WebUI 改动的
|
||||
运行时源,二者按名称合并成完整源列表。
|
||||
3. `rebuildRegistry` 为每个源创建 `provider.Provider`(持有目标适配器),并按源的
|
||||
并发上限配置适配器 worker 池**上限**(不预分配,见下)。
|
||||
4. 请求进来时 `Registry` 按 `model` 路由到 Provider,Provider 调适配器
|
||||
`transform_request` → HTTP 发送 → `transform_response` / `transform_stream_chunk`。
|
||||
|
||||
WebUI 上的"新增/编辑源"、"上传 Lua 适配器"、"改 AUTO 优先级链"与"密钥模型范围"
|
||||
都即时生效(写入运行时文件或 `adapter_dir` 后重新装配,无需重启);直接编辑
|
||||
`config.yaml` / `adapter_dir` 下的文件则需要重启进程才会重新加载。
|
||||
|
||||
在 WebUI 删除源/适配器时:运行时(WebUI 创建)的源与 `.lua` 文件会真正移除;
|
||||
`config.yaml` 中定义的基源无法改写配置文件,采用删除标记隐藏(重启后仍隐藏),
|
||||
在 UI 里重新添加同名源即可恢复。
|
||||
|
||||
### 源模板(多 key 负载均衡)
|
||||
|
||||
多个上游 key 共享同一套配置(URL / 适配器 / 模型列表 / 并发 / rpm)时,逐一复制
|
||||
整段 source 太冗余。**源模板**保存除 `name` 和 `api_key` 以外的全部源字段,
|
||||
从模板一键创建多个仅 key 不同的源,天然支持多 key 负载均衡(每个展开源独立调度、
|
||||
独立冷却、独立健康状态)。
|
||||
|
||||
- WebUI「源」页右上角「**模板管理**」按钮:列出全部模板,可编辑 / 删除 / 新建。
|
||||
- 添加源弹窗右上角两个按钮:
|
||||
- **从模板创建** → 选模板 → 表单自动填充(name + key 仍需你填);
|
||||
- **存为模板** → 把当前表单非 key/name 字段存为新模板。
|
||||
- 模板持久化在 `runtime_file` 的 `source_templates` 字段,与运行时源一样热管理,
|
||||
改完即时生效。模板本身不含密钥,可以安全共享 / 纳管。
|
||||
|
||||
模板不会自己变成源——它只是「配方」。真正承担流量的是从模板创建出来的、带 key
|
||||
的具体源。
|
||||
|
||||
### 内存占用(实测与调优)
|
||||
|
||||
内存不是一个定数:它随**配置的源数量**增长(每个源一个 `http.Transport` 与连接池、
|
||||
一组健康状态),与运行时长无关。本机实测(Linux x86_64,12 核):
|
||||
|
||||
| 部署形态 | 启动 RSS | 稳态 RSS |
|
||||
|---|---|---|
|
||||
| 1 源 / 1 适配器(最小配置) | ~4 MB | ~10 MB |
|
||||
| 1 源 + 29 MB 历史审计日志 | ~19 MB | ~20 MB |
|
||||
| **16 源 / 13 适配器 / 59 模型(本机生产)** | ~28 MB | **~32–35 MB** |
|
||||
|
||||
> 历史参考:本项优化前同一生产配置为 **~105 MB**。降幅来自三处:审计日志的**原始记录**
|
||||
> 不再常驻内存(约 25 MB;统计聚合仍扫全量,但扫完即释放)、Lua 状态池不再单调增长、
|
||||
> 以及下面两个运行时开关。
|
||||
|
||||
内存构成(生产实例分段测量,`/proc/<pid>/smaps`):
|
||||
|
||||
| 区域 | RSS | 说明 |
|
||||
|---|---|---|
|
||||
| Go 堆 | ~14 MB | provider/registry/scheduler 结构 + 连接池缓冲 |
|
||||
| 其他匹名(线程栈 / LuaJIT chunk / runtime) | ~12 MB | 与线程数、已加载适配器数相关 |
|
||||
| 二进制 text+rodata | ~8 MB | 映射的可执行文件页(只读、可被内核回收) |
|
||||
| Go runtime 预留 | ~4 MB | VSZ 上看到的 2 GB+ 是地址空间预留,不占物理内存 |
|
||||
| 共享库 | ~3 MB | libc / libluajit / libm |
|
||||
|
||||
**两个推荐的部署开关**(只适用于环境变量,无需改代码):
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/llmsproxy.service
|
||||
Environment=GOGC=50
|
||||
Environment=MALLOC_ARENA_MAX=2
|
||||
```
|
||||
|
||||
- `MALLOC_ARENA_MAX=2`:LuaJIT 的分配走 cgo → glibc malloc,glibc 默认允许 `8×nproc`
|
||||
个 per-thread arena,每个碰到 malloc 的 OS 线程会占用一个(各约 1 MB,且**不归还给系统**)。
|
||||
实测从 8–15 个 arena(约 7–12 MB)降到 **0**。
|
||||
- `GOGC=50`:把 Go 堆增长目标减半。**单独使用无效**(省下的堆会立即被更多 glibc
|
||||
arena 吃掉,实测 20.3 → 21.5 MB 反而变大),必须与 `MALLOC_ARENA_MAX` 配合,
|
||||
两者同时开启才降 **~19%**。网关是 I/O 密集型(本机 9 小时仅消耗 1min10s CPU),
|
||||
多出的 GC 周期与它的空闲 CPU 相比可忽略。
|
||||
|
||||
> 桌面版(Electron)已在 `cmd/gui/main.js` 里默认为内嵌核心注入这两个开关;手动 export 同名
|
||||
> 环境变量可覆盖。如果你用自己的 systemd unit / 容器,建议照上面加上。
|
||||
|
||||
可选:`GOMEMLIMIT=48MiB` 作软上限,实测再省 ~1 MB;代价是逐近上限时 GC 转激进,
|
||||
温和场景不必开。
|
||||
|
||||
### thinking 预算
|
||||
|
||||
请求体带 `"thinking": {"enabled": false}` 时关闭推理;`"thinking": {"enabled": true,
|
||||
"budget_tokens": 4096}` 指定预算。内核只定义这个**中立**语义字段,映射到具体协议由
|
||||
适配器负责(Anthropic `thinking.budget_tokens`、Gemini `thinkingConfig.thinkingBudget`、
|
||||
DeepSeek `thinking.type = "disabled"`)。内核不再向适配器泄漏 `disable_thinking` /
|
||||
`extra_body` 这类私有字段。
|
||||
|
||||
### WebUI
|
||||
|
||||
内置管理界面(`GET /`),登录后可在浏览器完成:
|
||||
|
||||
- **状态页**:源在线状态(实时探测 + 悬停看错误)、模型/源/key 用量统计、请求记录,
|
||||
支持按时间范围导出 CSV;点击模型可生成 pin 到该模型的连接配置。
|
||||
- **对话页**:流式/非流式调试。
|
||||
- **密钥页**:创建/编辑网关 key,为每个 key 配模型范围(模型 + 源 + token 配额 + 周期),
|
||||
管理员管理全部 key,用户只看到自己的 key。每个模型砖块显示自己的配额徽标
|
||||
(如 `1.0K 77×·1h`,未设配额显示 `∞`),点开可编辑该模型的 token 配额、
|
||||
请求数配额与重置周期 —— 配额按模型独立生效,一个用满不影响同一 key 的其它模型。
|
||||
「我的密钥」页对用户展示本 key 的模型范围与各自配额。
|
||||
- **优先级页**:拖拽积木配置 AUTO 链档位。
|
||||
- **源页**:在线增删改上游源(API key 等敏感字段加密落盘)。
|
||||
- **适配器页**:上传 / 删除 Lua 适配器脚本。
|
||||
|
||||
改动写入 `runtime_file`(重启仍生效)。
|
||||
|
||||
## Lua 适配器协议
|
||||
|
||||
完整 API 见 **[Lua 适配器 API 文档](docs/lua-adapters.md)**([English](docs/lua-adapters-en.md))。
|
||||
|
||||
每个适配器是一个返回 table 的 Lua 脚本(`internal/lua/adapters/<name>.lua`),
|
||||
加载时可被 `adapter_dir` 下的同名脚本覆盖——**改 Lua 脚本无需重编译**,
|
||||
重启即生效;通过 WebUI 上传的适配器与在线编辑的源配置则即时生效。
|
||||
|
||||
```lua
|
||||
return {
|
||||
name = "mysrc",
|
||||
version = "1.0.0",
|
||||
endpoint = "/chat/completions", -- 上游路径(可被 source.endpoint 覆盖)
|
||||
headers = { ["X-Static"] = "v" }, -- 静态默认请求头(build_headers 缺省时使用)
|
||||
|
||||
-- 请求转换:把 OpenAI 格式 req 转成上游原生格式,返回字符串
|
||||
transform_request = function(raw_json) ... end,
|
||||
|
||||
-- 响应转换:把上游原生响应转成统一格式字符串
|
||||
-- { content, reasoning_content, finish_reason, token_usage{...}, tool_calls[{...}] }
|
||||
transform_response = function(raw_json) ... end,
|
||||
|
||||
-- 流式分块转换:把上游 SSE data 转成 { content, done, ... },返回 "" 则跳过
|
||||
transform_stream_chunk = function(raw_chunk) ... end,
|
||||
|
||||
-- [可选] 动态请求头/签名钩子
|
||||
-- meta = { url, method, body, api_key, timestamp, source={ name, meta={...} } }
|
||||
build_headers = function(meta) return { ["X-App-Sign"] = sign } end,
|
||||
|
||||
-- [可选] 出向序列化(契约 v2):上游不是 OpenAI 协议时接管写给客户端的字节。
|
||||
-- 不声明则沿用内核的 OpenAI 兜底;声明即完全取代,内核不再混合字段。
|
||||
serialize_response = function(result) return body_str end,
|
||||
serialize_open_preamble = function(result) return bytes end,
|
||||
serialize_chunk = function(delta) return bytes end,
|
||||
serialize_close = function(result) return bytes end,
|
||||
}
|
||||
```
|
||||
|
||||
内置信号辅助:`hmac_sha256_hex(key, data)`、`sha256_hex(data)`、`base64_encode(s)`、
|
||||
`tohex(s)`、`json.encode/decode`、`log(level, msg)`。
|
||||
|
||||
### 内置适配器
|
||||
|
||||
`openai` `deepseek` `anthropic` `gemini` `github` `groq` `mistral` `ollama` `kimicode`
|
||||
`qwen38` `sensenova` `trae`。
|
||||
|
||||
`anthropic`/`gemini`/`ollama` 适配器内置多模态转换(`image_url` → 各自上游格式);
|
||||
推理开关由中立的 `thinking` 字段表达,`deepseek` 适配器把它映射为
|
||||
`thinking.type = "disabled"`。
|
||||
|
||||
**kimicode** 是展示 `build_headers` 的样例:云端校验调用方 app,需要按
|
||||
`meta.app_secret` 对时间戳+URL+请求体哈希做 HMAC 签名并附 `X-App-Sign` 等头。
|
||||
配好 `sources[].meta.{app_id, app_secret, app_agent}` 即可。
|
||||
|
||||
## 桌面 GUI(Electron)
|
||||
|
||||
可选独立产物,面向内网桌面端用户:**内嵌 ModelRouter 核心**(Clash Verge 形态),
|
||||
一键启停、系统托盘常驻、开机自启、静默启动,内容区一比一内嵌完整 WebUI
|
||||
(状态/对话/密钥/优先级/源/适配器 六个页面,免登录)。服务器用户继续使用纯 Go 二进制。
|
||||
|
||||
### 两种安装包:Headless 与 Desktop
|
||||
|
||||
发布到 Release 的安装包分为两类,面向不同用户:
|
||||
|
||||
- **Headless(服务器版)**:纯核心二进制(无 Electron),配置驱动,适合服务器、
|
||||
容器、systemd 等无人值守场景。发布包含 `deb` + `rpm`(含 systemd unit、内存调优
|
||||
与示例配置)与 `tar.gz`(二进制 + `config.example.yaml` + `adapters/` + README),
|
||||
Windows 为 `.zip`。
|
||||
- **Desktop(桌面版)**:Electron GUI 安装包,内嵌核心,适合个人桌面日常使用。
|
||||
|
||||
### 选择 GUI 还是纯后端
|
||||
|
||||
| 场景 | 推荐 | 理由 |
|
||||
| ------ | ------ | ------ |
|
||||
| 服务器 / 内网网关 / 无人值守常驻 | **Headless**(单二进制) | 极轻量(二进制 ~8MB;RSS 随源数量,单源 ~10MB、16 源 ~32–35MB,见[内存占用](#内存占用实测与调优)),零依赖单进程,直接跑在 systemd/任意容器里,远程管理 |
|
||||
| 个人桌面日常使用 / 多设备内网共享 | **Desktop**(GUI) | 免登录内嵌 WebUI、系统托盘一键启停、开机自启、静默后台,适合不懂命令行的使用者 |
|
||||
| Windows 桌面 | **Desktop** | 纯后端在 Windows 上需自行注册服务,GUI 提供原生托盘/自启体验 |
|
||||
| CI 一键出三平台安装包 | **Desktop 打包脚本** | `make gui-*` 系列出 deb/AppImage/NSIS,可进发行流水线 |
|
||||
|
||||
两者完全同源:GUI 内嵌的就是纯后端的同一份 `llmsproxy` 核心(LuaJIT 版),
|
||||
配置/适配器格式完全一致,可随时互换。
|
||||
|
||||
### 构建与运行
|
||||
|
||||
```bash
|
||||
cd cmd/gui
|
||||
npm install
|
||||
make gui # 或 npm run dev —— 开发运行(需先 make build 生成 bin/llmsproxy)
|
||||
make gui-deb # 本机构建 deb(分享给其它 Linux 用户)
|
||||
make gui-dist # 构建 deb + rpm + AppImage(linux,rpm 需系统 rpmbuild)
|
||||
make core-dist # 构建核心 deb + rpm + tar.gz(服务器版,需 nfpm)
|
||||
make gui-win # 构建 Windows nsis 安装包(本机 mingw + wine)
|
||||
make gui-win-docker # 构建 Windows nsis 安装包(全 docker 自足,含 wine)
|
||||
make gui-dist-dir # 仅解包产物(调试用,不产安装包)
|
||||
```
|
||||
|
||||
产物在 `cmd/build/gui-dist/`:`ModelRouter-<ver>.AppImage`、
|
||||
`modelrouter-gui_<ver>_amd64.deb`、`ModelRouter Setup <ver>.exe`(win);
|
||||
核心包在 `cmd/build/dist/`:`llmsproxy_<ver>_amd64.deb`、`llmsproxy-<ver>.x86_64.rpm`。
|
||||
rpm 需系统 `rpmbuild`。每次打包后都会跑 `packaging/verify-dist.sh` 产物体检
|
||||
(大小下限门禁——1.3.0 曾因宿主 wine 损坏产出 264KB 的残缺 exe 却没有被拦住)。
|
||||
发布时会把产物重命名为 `ModelRouter-Desktop-*`(GUI)与 `ModelRouter-Headless-*`(纯后端)
|
||||
两类上传到 GitCode Release。
|
||||
|
||||
#### Windows 打包(docker 固化方案,推荐)
|
||||
|
||||
`make gui-win-docker` 一步完成:交叉编译核心 + NSIS 打包**全在 docker 内**,宿主机
|
||||
**无需 mingw、无需 wine、无需 node**。镜像内置 wine(含 i386 运行时),修复了
|
||||
此前宿主 wine 损坏(缺 syswow64 → `c0000135`)导致产出 264KB 残缺 exe 的问题。
|
||||
|
||||
- 构建镜像:`cmd/gui/docker/win-builder/Dockerfile`(`golang:1.25` + mingw-w64 +
|
||||
node + wine32/wine64 + 预编译 LuaJIT windows 版 `lua51.dll` 与 import lib)。
|
||||
LuaJIT 源码取自 gitcode 镜像(国内可达),github 兜底;Go 依赖走 `https://goproxy.cn`。
|
||||
- 一键发布:`cmd/gui/scripts/dist-win-docker.sh` —— 核心编译 + electron-builder
|
||||
NSIS 全部在容器内执行(`USE_SYSTEM_WINE=true` 走镜像内的 wine);electron-builder
|
||||
的 ~400MB 工具链缓存挂到 docker volume,二次构建增量复用。
|
||||
- 首次运行自动构建 `modelrouter/win-builder` 镜像(约 5–10 分钟);已存在则直接复用。
|
||||
- 打包完成后自动跑 `packaging/verify-dist.sh` 门禁:exe 低于 5MB / deb·rpm·7z 低于
|
||||
10MB 即失败——残缺安装包不可能再溜进 Release。
|
||||
|
||||
```bash
|
||||
# 全 Windows 发布(核心重编 + NSIS):
|
||||
make gui-win-docker
|
||||
```
|
||||
|
||||
### 特性
|
||||
|
||||
- **内嵌核心**:自动拉起捆绑的 `llmsproxy`(LuaJIT 版),配置/密钥/adapter 存于
|
||||
用户数据目录 `<userData>/profile/`,首次运行自动生成随机 admin key 写入 `keys`
|
||||
(非 seed)并自注入,WebUI 免登录、不会提示"更换初始密钥"。端口可在设置中更改(默认 8787)。
|
||||
- **系统托盘**:内核状态点、开机自启/静默启动开关、启停/重启内核、退出。
|
||||
- **静默启动**:`--silent` 参数或设置项,启动仅驻托盘不弹窗;首次运行始终显示窗口。
|
||||
- **开机自启**:Windows/macOS 用 `setLoginItemSettings`;Linux 写
|
||||
`~/.config/autostart/modelrouter-gui.desktop`,静默开启时自动附加 `--silent`。
|
||||
|
||||
## 目录
|
||||
|
||||
```
|
||||
cmd/llmsproxy # 入口(纯 Go 服务端)
|
||||
cmd/gui # Electron 桌面 GUI(内嵌核心 + 托盘 + 打包)
|
||||
internal/config # YAML 配置加载/校验
|
||||
internal/lua # LuaJIT VM + worker 池 + AdapterCache + 内置适配器 (embed)
|
||||
旧路径与旧环境变量在读取侧仍被接受,因此升级不会因为顺序问题立刻失效。
|
||||
`deploy.sh` 在检测到旧 `/etc/llmsproxy` 而新配置不存在时会**中止**,避免部署出
|
||||
一个「能启动但没有 key / 没有源」的空实例。
|
||||
|
||||
插件 SDK 同名调整:唯一公开入口是 `modelrouter`(`llmsproxy` 仅作兼容别名),
|
||||
详见 [docs/plugins.md](docs/plugins.md)。
|
||||
|
||||
## 目录结构
|
||||
|
||||
```text
|
||||
cmd/modelrouter # 入口(纯 Go 服务端)
|
||||
cmd/gui # Electron 桌面壳(内嵌同一份核心)
|
||||
internal/config # 配置解析、加密存储、运行时文件
|
||||
internal/core # 调度、内核动作、状态与审计
|
||||
internal/lua # LuaJIT VM + 弹性 worker 池 + 内置适配器 (embed)
|
||||
internal/lua/plugins # 可选插件示例(不随二进制/发行包分发)
|
||||
internal/provider # Provider(HTTP) + Registry(路由)
|
||||
internal/gateway # OpenAI 兼容 HTTP 服务 + 鉴权 + SDK/流式 + WebUI
|
||||
internal/gateway # OpenAI 兼容 HTTP 服务 + 鉴权 + 流式 + WebUI
|
||||
internal/types # 统一格式 & OpenAI wire 类型
|
||||
packaging/ # deb/rpm/tar 打包、systemd unit、迁移脚本
|
||||
docs/ # 主题文档
|
||||
```
|
||||
|
||||
## 开发协作
|
||||
|
||||
- **Git 分支工作流**:主分支 / 特性分支 / 发布分支的规范见
|
||||
**[docs/git-workflow.md](docs/git-workflow.md)**([English](docs/git-workflow-en.md))。
|
||||
主线:`main` 永可部署 → 特性合入 `main` → 切 `release/vX.Y.Z` 打 tag 发布 →
|
||||
hotfix 提交发布分支并 cherry-pick 回 `main`。
|
||||
- **Lua 适配器协议**:见 **[docs/lua-adapters.md](docs/lua-adapters.md)**
|
||||
([English](docs/lua-adapters-en.md))。
|
||||
|
||||
## 测试
|
||||
## 开发与测试
|
||||
|
||||
```bash
|
||||
go test -tags luajit ./...
|
||||
go test -tags luajit ./... # 全量
|
||||
go test -tags luajit ./internal/lua # 单包(LuaJIT 相关改动必跑)
|
||||
```
|
||||
|
||||
覆盖:配置校验、适配器加载/变换、签名钩子、Gateway 鉴权、SDK 结算、SSE 流式、
|
||||
模型路由、多模态透传与 thinking 预算。
|
||||
- 分支与发布纪律见 [docs/git-workflow.md](docs/git-workflow.md)。
|
||||
- 插件放置在 `plugin_dir`,网关启动时**不会**创建、覆盖或删除任何插件文件;
|
||||
升级桌面端也不会动它们。
|
||||
|
||||
689
README_EN.md
689
README_EN.md
@ -2,625 +2,148 @@
|
||||
|
||||
> **中文**: [README.md](./README.md)
|
||||
>
|
||||
> **AI-assisted**: This project is developed with AI-assisted coding (code and docs are produced in collaboration with AI, reviewed by humans).
|
||||
> **AI-assisted**: code and documentation were produced with AI assistance and
|
||||
> reviewed by a human.
|
||||
|
||||
A lightweight, unified OpenAI-compatible LLM gateway for internal networks —
|
||||
configure once, and let multiple services (agents, SDKs, bots) share many
|
||||
upstreams (DeepSeek, Qijiar, OpenAI, Anthropic, Gemini, Groq, Mistral, Ollama,
|
||||
KimiCode…) behind a single endpoint. Pick a specific model or use **AUTO** mode,
|
||||
which routes to the best healthy upstream by configured priority.
|
||||
An intranet gateway for people with several LLM subscriptions: configure once and
|
||||
let multiple services share the upstream capacity. Pick a model explicitly, or let
|
||||
`AUTO` choose by priority among the sources that are currently healthy.
|
||||
|
||||
> **Lightweight, no recompile to add sources**: the gateway is a single Go
|
||||
> binary (8-12 MB, ~8 MB stripped; zero runtime dependencies). Adding or switching an upstream is
|
||||
> just a `sources` entry in `config.yaml` (or via the WebUI) or a `.lua`
|
||||
> adapter — **no Go changes, no recompile**. Changes submitted through the
|
||||
> WebUI take effect immediately (hot reload); editing `config.yaml` or a `.lua`
|
||||
> file under `adapter_dir` by hand requires a process restart.
|
||||
The gateway itself is a single Go binary (~10 MB, no runtime dependencies). Adding
|
||||
or switching an upstream is a `sources` entry in `config.yaml` (or the WebUI), or a
|
||||
dropped-in `.lua` adapter — **no Go changes, no rebuild**. Changes submitted through
|
||||
the WebUI take effect immediately; editing `config.yaml` or a `.lua` adapter file
|
||||
directly requires a restart.
|
||||
|
||||
## Overview
|
||||
## What it does
|
||||
|
||||
Exposes a standard `OpenAI Chat Completions` API (`/v1/chat/completions` +
|
||||
`/v1/models`) to your internal network, translating between multiple upstream
|
||||
protocols via **Lua adapters**. Supports one-shot calls, SSE streaming,
|
||||
**AUTO model routing**, multimodal passthrough, and tool calls.
|
||||
|
||||
Extracted and independently evolved from the multi-source LLM adapter layer of
|
||||
[HomeAgent](https://gitcode.com/JianFeeeee/HomeAgent)
|
||||
(`internal/agent/api/provider.go` + `internal/lua/adapters/*`).
|
||||
|
||||
## Features
|
||||
|
||||
- **Multi-source**: any number of upstream sources in one process, routed by the
|
||||
request's `model`.
|
||||
- **AUTO mode**: with `default_model: AUTO`, scheduling follows the tiered AUTO
|
||||
chain saved on the Priority page (see below).
|
||||
- **Unified output**: every source speaks OpenAI format (incl.
|
||||
`reasoning_content`, `tool_calls`, `usage`).
|
||||
- **Image generation**: `POST /v1/images/generations`, routed to models with
|
||||
`kind: image`.
|
||||
- **Per-model quota**: each key gives every model its own token and request caps
|
||||
plus a reset period (hour/week/month/custom N hours). One model running out
|
||||
affects only that model — the key's other models keep working. Exhaustion
|
||||
answers 429 + `Retry-After` naming the model, so a client resumes when the
|
||||
window rolls over; admin keys are never capped.
|
||||
- **Multimodal**: `content` arrays (`image_url` etc.) pass through losslessly;
|
||||
Anthropic/Gemini/Ollama are translated automatically.
|
||||
- **LuaJIT VM**: golua-binding LuaJIT; each adapter has its own VM + worker
|
||||
pool for safe concurrency. Pools are **elastic**: the sum of `max_concurrent`
|
||||
over an adapter's sources is a ceiling, not a preallocation — states are booted
|
||||
on demand and reclaimed when demand drops, so an idle gateway holds close to
|
||||
zero Lua states. The grow step follows the adapter's max concurrency
|
||||
(`clamp(ceil(max/8), 1, 8)`, and never exceeds the number of queued callers);
|
||||
the shrink step follows the live connection count
|
||||
(`clamp(ceil(slack/(1+in_use)), 1, slack)`), so an idle pool collapses in one
|
||||
round while a busy one gives up a single state at a time.
|
||||
- **On-demand request logs**: dashboard totals are computed from the **full**
|
||||
audit history (streamed once at startup and released — 29 MB / 221k lines in
|
||||
~260 ms), while the raw records are never held in memory: the dashboard loads
|
||||
one screen, scrolling pages the rest straight off disk, CSV export streams in
|
||||
O(1) memory, and leaving the page releases everything.
|
||||
- **Self-healing cooldown**: cooldown is capped at 5 minutes and, past the
|
||||
window's midpoint, exactly one probe request is allowed through; a recovered
|
||||
upstream (or a reset quota) returns to full rotation on that probe instead of
|
||||
waiting out the window.
|
||||
- **disable_thinking**: `disable_thinking: true` toggles reasoning per-request.
|
||||
- **Lua adapter protocol**: each source mounts a `.lua` adapter with
|
||||
`transform_request` / `transform_response` / `transform_stream_chunk` — all
|
||||
protocol differences live in Lua, Go only schedules and proxies.
|
||||
- **Signature / header hooks**: adapters may define `build_headers(meta)` to
|
||||
inject/sign request headers before the HTTP call (e.g. KimiCode style
|
||||
app-validation), with helpers like `hmac_sha256_hex`, `sha256_hex`,
|
||||
`base64_encode`.
|
||||
- **WebUI**: built-in management page to view/add/edit sources & models, edit
|
||||
the AUTO chain, and manage per-key model scopes, persisted to a runtime file.
|
||||
- **Encrypted secrets at rest**: upstream `api_key`, custom header values and
|
||||
gateway keys are stored AES-256-GCM encrypted (`master.key` 0600 next to the
|
||||
runtime file, or `LLMS_PROXY_MASTER_KEY`).
|
||||
- **Live source probing**: the status page probes each source
|
||||
(`GET {base}/models`, falling back to a minimal chat call) without touching
|
||||
the scheduler's backoff state; errors are shown on hover.
|
||||
- **Auth**: the gateway validates client keys via `gateway_keys`; independent
|
||||
from each upstream's own key.
|
||||
- **Streaming**: SSE `chat.completion.chunk` with a role-first chunk and
|
||||
`[DONE]` terminator.
|
||||
- **One OpenAI-compatible API**: `/v1/chat/completions`, `/v1/models`,
|
||||
`/v1/images/generations`. Every source speaks the same shape outwards, including
|
||||
`reasoning_content`, `tool_calls` and `usage`.
|
||||
- **Many sources, one AUTO chain**: configure any number of upstreams; `AUTO`
|
||||
follows the priority tiers saved in the WebUI, fails over between them, cools
|
||||
down broken sources and probes them back to health at half-cool.
|
||||
- **Protocol differences live in Lua**: each source binds a `.lua` adapter that
|
||||
transforms requests, responses and stream chunks both ways, and can sign headers
|
||||
via `build_headers(meta)`. Vendor knowledge stays out of the Go kernel.
|
||||
- **Plugins are independent**: business extensions (channel login, billing, quota
|
||||
probing) are external Lua plugins loaded from `plugin_dir`. They are not compiled
|
||||
into the binary and not shipped in release archives, and they talk to the kernel
|
||||
only through the `modelrouter` SDK.
|
||||
- **Multi-tenant keys**: gateway keys are separate from upstream keys, and can be
|
||||
scoped to a model set, an AUTO chain and a quota.
|
||||
- **Encrypted at rest**: upstream keys, custom header values and gateway keys are
|
||||
sealed with AES-256-GCM; `master.key` is 0600, or injected via
|
||||
`MODELROUTER_MASTER_KEY`.
|
||||
- **Observability**: live source probing, HTTP access log, config-change audit,
|
||||
request and usage CSV export. Writes are kept off the request hot path.
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
GOMODCACHE=... GOPROXY=off go build -tags luajit -o llmsproxy ./cmd/llmsproxy
|
||||
./llmsproxy -config config.yaml # first run generates a default config and prints a random admin key
|
||||
# LuaJIT development libraries are required; -tags luajit is mandatory
|
||||
GOMODCACHE=... GOPROXY=off go build -tags luajit -o modelrouter ./cmd/modelrouter
|
||||
|
||||
# First run generates a default config and prints a random admin key
|
||||
./modelrouter -config config.yaml
|
||||
```
|
||||
|
||||
> Depends on [golua](https://github.com/aarzilli/golua) (LuaJIT bindings).
|
||||
> You **must** build with `-tags luajit`; otherwise the built-in superset
|
||||
> gopher-lua path is used (behavior differs slightly).
|
||||
> Depends on [golua](https://github.com/aarzilli/golua) (LuaJIT bindings). Without
|
||||
> `-tags luajit` the build falls back to gopher-lua, which behaves slightly
|
||||
> differently.
|
||||
|
||||
> **No config is shipped in the repo** (config files carry real keys). On
|
||||
> first run the binary generates a default config at the `-config` path: a
|
||||
> random admin key (printed to the startup log), loopback-only `127.0.0.1:8080`,
|
||||
> and a keyless source ready to chat. Rotate the admin key in the
|
||||
> WebUI after first login.
|
||||
> **No config in the repo**: the repository ships no config file (they contain
|
||||
> keys). The generated default binds `127.0.0.1:8080` only; rotate the admin key in
|
||||
> the WebUI after the first login.
|
||||
|
||||
```bash
|
||||
# no key -> 401
|
||||
# No key -> 401
|
||||
curl http://127.0.0.1:8080/v1/models
|
||||
|
||||
# one-shot ($KEY = the admin key printed at first startup)
|
||||
# Non-streaming ($KEY = the admin key printed at first start)
|
||||
curl -H "Authorization: Bearer $KEY" \
|
||||
-d '{"model":"AUTO","messages":[{"role":"user","content":"hi"}]}' \
|
||||
http://127.0.0.1:8080/v1/chat/completions
|
||||
|
||||
# streaming
|
||||
# Streaming
|
||||
curl -N -H "Authorization: Bearer $KEY" \
|
||||
-d '{"model":"AUTO","stream":true,"messages":[{"role":"user","content":"hi"}]}' \
|
||||
http://127.0.0.1:8080/v1/chat/completions
|
||||
```
|
||||
|
||||
Any OpenAI SDK works: point `base_url` at the gateway, use any authorized
|
||||
gateway key as the API key.
|
||||
|
||||
## Configuration
|
||||
|
||||
Generated on first run at the `-config` path (default `./config.yaml`). Core fields:
|
||||
|
||||
```yaml
|
||||
listen: 127.0.0.1:8080 # bind address (keep internal/loopback)
|
||||
gateway_keys: [sk-gw-<random>] # initial admin key seed (random per install), migrated to the store on first start
|
||||
default_model: AUTO # when model is unroutable, follow the AUTO chain
|
||||
adapter_dir: adapters # Lua adapter dir; created+seeded if missing, read-only otherwise
|
||||
runtime_file: runtime.json # WebUI-edited sources/keys/AUTO chain persist here
|
||||
|
||||
sources:
|
||||
- name: deepseek
|
||||
base_url: https://api.deepseek.com
|
||||
api_key: sk-... # or api_key_env: SOME_ENV to read from an env var
|
||||
adapter: deepseek
|
||||
max_concurrent: 8
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
priority: 100 # YAML sources seed the AUTO chain on first start
|
||||
kind: chat
|
||||
- id: deepseek-v4-pro
|
||||
priority: 60
|
||||
kind: chat
|
||||
# static headers (take precedence over adapter defaults)
|
||||
headers: { X-Tenant: prod }
|
||||
# passthrough metadata for the Lua build_headers hook
|
||||
meta: { app_id: x, app_secret: y }
|
||||
temperature: 0.7
|
||||
max_tokens: 4096
|
||||
timeout: 120s # request timeout, default 120s
|
||||
```
|
||||
|
||||
Sensitive fields in the runtime file (`runtime_file`) are encrypted at rest:
|
||||
|
||||
- AES-256-GCM, stored as `enc:v1:<base64>`.
|
||||
- Master key source: env var `LLMS_PROXY_MASTER_KEY` (64 hex chars), else a
|
||||
`master.key` file next to the runtime file; if neither exists it is generated
|
||||
on first start (mode 0600).
|
||||
- Old plaintext files still load; the first UI save migrates the whole file to
|
||||
ciphertext.
|
||||
- Back up `master.key` with your config — losing it makes the secrets
|
||||
undecryptable. Do not commit it.
|
||||
|
||||
### Gateway keys (multi-key)
|
||||
|
||||
Gateway auth uses a "multi-key + role + model scope" model. Keys are persisted
|
||||
under the `keys` field of the runtime file (encrypted at rest):
|
||||
|
||||
- The `gateway_keys` config is only an **initial admin key seed** — it is
|
||||
migrated into the runtime store on first start and no longer drives auth.
|
||||
- **Important: after first start, replace the admin key via the WebUI Keys
|
||||
page.** The seed key is written in plaintext in `config.yaml`, so keeping it
|
||||
active is a security risk; create a new admin key, log in with it, then
|
||||
delete the seed key.
|
||||
- The WebUI **Keys page** creates/deletes keys. Each key has a role (`admin`
|
||||
manages everything, `user` sees only its own key) and an optional **model
|
||||
scope** (model + source + token quota + reset period). Each model brick shows
|
||||
its own budget badge (e.g. `1.0K 77×·1h`, `∞` when uncapped); clicking it
|
||||
edits that model's token quota, request quota and reset period. Quotas apply
|
||||
per model, so one model running out never blocks the key's others. The
|
||||
"My key" view shows a user its model scopes and their budgets.
|
||||
- Clients authenticate with any authorized key's plaintext as
|
||||
`Authorization: Bearer <key>`.
|
||||
- Deleting a key removes it from the store immediately.
|
||||
|
||||
#### Per-key usage quota
|
||||
|
||||
**Quotas are per model.** Each key's `models[]` list gives every model its own
|
||||
token budget and request budget. One model running out affects only that model
|
||||
— the key's other models keep working.
|
||||
|
||||
```yaml
|
||||
keys:
|
||||
- key: sk-gw-<hex>
|
||||
role: user
|
||||
name: agent-alice
|
||||
models:
|
||||
- model: deepseek-v4-flash
|
||||
token_quota: 1000000 # this key's token budget for this model
|
||||
req_quota: 20000 # requests within the window
|
||||
period: nhour # "" | hour | week | month | nhour
|
||||
hours: 6 # n-hour only
|
||||
- model: AUTO
|
||||
token_quota: 5000000 # AUTO is a quota entry like any other
|
||||
period: hour
|
||||
- model: kimi-k3 # no quota listed = unlimited
|
||||
```
|
||||
|
||||
- **There is deliberately no key-wide total.** A key-wide cap would make
|
||||
"move A's budget to B" a global reallocation; per-model budgets keep each
|
||||
model independently controllable, so it stays visible which model is
|
||||
actually consuming the spend.
|
||||
- `period`: empty = never resets (lifetime total); `hour` / `week` / `month` =
|
||||
fixed windows; `nhour` + `hours` = a custom hour count. **A misspelled
|
||||
period is rejected at write time** rather than silently becoming a
|
||||
never-resetting quota.
|
||||
- Quotas are isolated per key *and* per model within a key: one key exhausting
|
||||
`m1` never draws on another key's budget, and never blocks the same key's
|
||||
`m2`.
|
||||
- Usage counts chat, streaming and image requests, and survives a restart by
|
||||
replaying the audit log (40 days retained, covering the longest `month`
|
||||
window).
|
||||
- An exhausted quota returns **429 + `Retry-After`**
|
||||
(`rate_limit_exceeded`) and the message names the model that ran out, so a
|
||||
client resumes when the window rolls over; a model the key may not use stays
|
||||
403. **Admin keys are never capped** (quotas on their scopes are not
|
||||
enforced either), so a cap can never lock the operator out.
|
||||
- Buckets are whole unix hours, so a window frees up at most an hour late
|
||||
(deliberately freeing late rather than overspending).
|
||||
- `PUT /api/keys/{key}` submits quotas by submitting `models` — the caps are
|
||||
part of the scope, so there is no second budget that can drift out of sync
|
||||
with the model list. An explicit `0` lifts that model's cap.
|
||||
|
||||
##### Quota rejection vs capacity rejection
|
||||
|
||||
A quota is checked at the entry point and occupies no upstream slot, so it is a
|
||||
cheap rejection. Running out of capacity instead waits out the bounded
|
||||
`busyWait` before answering 503. Measured (capacity 8, 0.6 s/request, 100
|
||||
concurrent clients):
|
||||
|
||||
| Case | Status | Latency | Reaches upstream |
|
||||
|---|---|---|---|
|
||||
| Quota spent (`req_quota` 50, 100 concurrent) | `429 rate_limit_exceeded` + `Retry-After` | **19–21 ms** | no |
|
||||
| Capacity exhausted (same topology, no quota) | `503 upstream_error` | ~2.6 s (`busyWait`) | overflow: no |
|
||||
|
||||
So a client can tell "wait for the window to roll over" apart from "wait for
|
||||
upstream capacity" — the first only needs patience, the second usually means
|
||||
back off concurrency or switch sources.
|
||||
|
||||
##### What the quota costs
|
||||
|
||||
Buckets are kept per (key, model, whole unix hour) for 40 days. Measured on an
|
||||
AMD 7840HS:
|
||||
|
||||
- A window query scans the window, not the whole retention: 49 ns for 1 h,
|
||||
55 ns for 24 h, 3.9 µs for 30 d.
|
||||
- Recording one request: 283 ns, 3 allocations (unchanged from before this
|
||||
feature; the allocations come from the record ring buffer).
|
||||
- Memory: the production shape (7 keys x 8 models x 2 sources at full 40-day
|
||||
retention) costs about **3.7 MB**. The `source::model` bucket used by a
|
||||
source-pinned quota is created **lazily** — it is maintained only once some
|
||||
quota actually pins that source, because writing it on every record would
|
||||
cost an extra 18 MB at 20 keys x 8 models x 3 sources.
|
||||
The trade-off: usage recorded before a pinned quota was configured cannot be
|
||||
split by source afterwards, so **a pinned quota may under-count its first
|
||||
window**.
|
||||
|
||||
```
|
||||
HTTP/1.1 429 Too Many Requests
|
||||
Retry-After: 2100
|
||||
{"error":{"type":"rate_limit_exceeded","message":"token quota exceeded for \"deepseek-v4-flash\" (5000000/5000000)"}}
|
||||
```
|
||||
|
||||
### Model routing
|
||||
|
||||
`/v1/chat/completions` `model` resolution order:
|
||||
|
||||
1. `source/model` or `source:model` prefix → pinned source;
|
||||
2. exact match of a source's `model`;
|
||||
3. with `default_model: AUTO` → follow the AUTO chain from the Priority page
|
||||
(see below);
|
||||
4. otherwise the request errors.
|
||||
|
||||
Any OpenAI client can pin to a source by setting `model` to its
|
||||
`name/anything`; `AUTO` (or the gateway's `default_model: AUTO`) follows the
|
||||
tiered AUTO chain.
|
||||
|
||||
### AUTO chain (Priority page)
|
||||
|
||||
AUTO scheduling is driven **only** by the rules saved on the Priority page. The
|
||||
numeric `models[].priority` in source configs no longer participates in
|
||||
scheduling and is no longer shown. **Chat and image generation are two
|
||||
independent chains**, switched in the Priority page via the `Chat / Image` toggle:
|
||||
|
||||
- **Chat chain** (`auto` field): serves `POST /v1/chat/completions` with
|
||||
`model: AUTO`; only `kind: chat` models are honored (image slots are skipped).
|
||||
- **Image chain** (`auto_image` field): serves `POST /v1/images/generations` with
|
||||
`model: AUTO`; only `kind: image` models are honored. With no image chain
|
||||
configured, AUTO image generation falls back to legacy discovery (all sources
|
||||
exposing an image model, in registry order) for backward compatibility.
|
||||
|
||||
Each rule is one "slot": `{ model, source, tier, token_quota, period, hours }`.
|
||||
|
||||
- `tier` is the priority tier: models in the same tier sit side by side and
|
||||
share it; tiers run high → low.
|
||||
- The same model may appear in several slots (e.g. A low → B low → A high →
|
||||
B high) and is tried in tier order.
|
||||
- `token_quota` > 0 means the slot is skipped once its tokens within the reset
|
||||
window are exhausted; `period` supports `hour` / `week` / `month` / `nhour`
|
||||
(with `hours`); empty = unlimited.
|
||||
- Within a tier, slots run by preference score; cooling / quota-exhausted /
|
||||
hard-failed slots fall through to the next tier, and total failure returns
|
||||
503 with a per-tier summary.
|
||||
|
||||
### Sources vs. adapters
|
||||
|
||||
- A **source** describes a connection to an upstream: `name`, `base_url`,
|
||||
`api_key`, model list and priorities.
|
||||
- An **adapter** is the protocol translation logic (Lua script): converts the
|
||||
unified OpenAI request to the upstream's native format and back.
|
||||
- One adapter serves many sources (e.g. `openai.lua` for any OpenAI-compatible
|
||||
site); one source can switch adapters via its `adapter` field.
|
||||
- Sources decide *whom to talk to and which models to expose*; adapters decide
|
||||
*how to talk*. They are linked by the `adapter` field inside each `sources[]`
|
||||
entry.
|
||||
|
||||
Loading flow (assembled by `Core` in `internal/core`):
|
||||
|
||||
1. On startup `lua.NewVM(adapter_dir)` loads all adapters: built-in ones
|
||||
(embedded at compile time) + same-name override files under `adapter_dir`.
|
||||
2. `config.Load` reads `config.yaml`; `config.NewStore(runtime_file)` reads
|
||||
WebUI-edited runtime sources; both are merged by name.
|
||||
3. `rebuildRegistry` creates a `provider.Provider` per source (holding its
|
||||
adapter), sizing the adapter's worker pool from the source's concurrency
|
||||
limit.
|
||||
4. Incoming requests are routed by `Registry` to a provider, which calls the
|
||||
adapter's `transform_request` → HTTP call → `transform_response` /
|
||||
`transform_stream_chunk`.
|
||||
|
||||
"Add/Edit source", "Upload adapter", "AUTO chain edits" and "per-key model
|
||||
scopes" in the WebUI take effect immediately (written to the runtime file /
|
||||
`adapter_dir`, then reassembled — no restart). Directly editing `config.yaml`
|
||||
or files under `adapter_dir` requires a process restart.
|
||||
|
||||
Deleting a source/adapter in the WebUI truly removes runtime (UI-created)
|
||||
sources and `.lua` files; base sources defined in `config.yaml` cannot rewrite
|
||||
that file, so they are hidden via a deletion tombstone (still hidden after
|
||||
restart) and can be restored by re-adding the same name in the UI.
|
||||
|
||||
### Source templates (multi-key balancing)
|
||||
|
||||
When several upstream keys share one config (URL / adapter / model list /
|
||||
concurrency / rpm), copying the whole source block N times is wasteful. A
|
||||
**source template** stores every source field except `name` and `api_key`, so you
|
||||
create N key-only-different sources from one shared skeleton — natural multi-key
|
||||
load balancing (each expanded source schedules, cools down, and reports health
|
||||
independently).
|
||||
|
||||
- WebUI Sources page, top-right **Templates** button: list all templates with
|
||||
edit / delete / create.
|
||||
- Add-source dialog has two header buttons:
|
||||
- **From template** → pick a template → form auto-fills (name + key still
|
||||
yours to enter);
|
||||
- **As template** → save the current form's non-key/name fields as a template.
|
||||
- Templates persist in the runtime file's `source_templates` field alongside
|
||||
runtime sources (hot-managed, apply immediately). Templates hold no secret
|
||||
keys, so they're safe to share / version-control.
|
||||
|
||||
A template is just a recipe — it doesn't become a source by itself. Only the
|
||||
key-bearing sources created from it carry real traffic.
|
||||
|
||||
### Memory footprint (measured and tuned)
|
||||
|
||||
Memory is not a constant: it scales with the **number of configured sources**
|
||||
(each source owns an `http.Transport` with its connection pool plus per-model
|
||||
health state), not with uptime. Measured on this machine (Linux x86_64, 12 cores):
|
||||
|
||||
| Deployment shape | Startup RSS | Settled RSS |
|
||||
|---|---|---|
|
||||
| 1 source / 1 adapter (minimal) | ~4 MB | ~10 MB |
|
||||
| 1 source + a 29 MB audit history | ~19 MB | ~20 MB |
|
||||
| **16 sources / 13 adapters / 59 models (this host)** | ~28 MB | **~32-35 MB** |
|
||||
|
||||
> For reference, the same production config used **~105 MB** before this round of
|
||||
> work. The reduction comes from three places: raw audit records are no longer kept
|
||||
> resident (~25 MB — the aggregates still scan the full history, but the scan
|
||||
> releases as it goes), Lua state pools no longer grow monotonically, and the two
|
||||
> runtime knobs below.
|
||||
|
||||
Breakdown of the production instance (per-region, from `/proc/<pid>/smaps`):
|
||||
|
||||
| Region | RSS | Notes |
|
||||
|---|---|---|
|
||||
| Go heap | ~14 MB | provider/registry/scheduler state + connection-pool buffers |
|
||||
| Other anonymous (thread stacks / LuaJIT chunks / runtime) | ~12 MB | scales with thread count and loaded adapters |
|
||||
| Binary text+rodata | ~8 MB | mapped executable pages (read-only, reclaimable by the kernel) |
|
||||
| Go runtime reservations | ~4 MB | the 2 GB+ you see in VSZ is address space, not physical memory |
|
||||
| Shared libraries | ~3 MB | libc / libluajit / libm |
|
||||
|
||||
**Two recommended deployment knobs** (environment only, no code change):
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/llmsproxy.service
|
||||
Environment=GOGC=50
|
||||
Environment=MALLOC_ARENA_MAX=2
|
||||
```
|
||||
|
||||
- `MALLOC_ARENA_MAX=2`: LuaJIT allocates through cgo into glibc malloc, and glibc
|
||||
allows up to `8 x nproc` per-thread arenas. Every OS thread that touches malloc
|
||||
claims one (~1 MB each) and **never returns it to the OS**. Measured: 8-15
|
||||
arenas (7-12 MB) down to **0**.
|
||||
- `GOGC=50`: halves the Go heap growth target. **It does nothing on its own** —
|
||||
the heap it saves is immediately consumed by additional glibc arenas (measured
|
||||
20.3 -> 21.5 MB, i.e. slightly worse) — so it must be paired with
|
||||
`MALLOC_ARENA_MAX`. Together they cut settled RSS by **~19%**. The gateway is
|
||||
I/O bound (1min10s of CPU per 9 hours here), so the extra GC cycles are free.
|
||||
|
||||
> The desktop build (Electron) already injects both when spawning the embedded
|
||||
> core (`cmd/gui/main.js`); exporting either variable yourself overrides it. If you
|
||||
> write your own systemd unit or container spec, add them there.
|
||||
|
||||
Optional: `GOMEMLIMIT=48MiB` as a soft ceiling saves roughly 1 MB more, at the
|
||||
cost of GC turning aggressive as the limit approaches. Not worth it for a relaxed
|
||||
deployment.
|
||||
|
||||
### disable_thinking
|
||||
|
||||
With `"disable_thinking": true` in the request body, the gateway passes it to
|
||||
each adapter; the DeepSeek adapter maps it to `extra_body.thinking.type =
|
||||
"disabled"`, other sources follow their own protocol.
|
||||
|
||||
### WebUI
|
||||
|
||||
Built-in admin page at `GET /`; after login you can:
|
||||
|
||||
- **Status**: source online state (live probe, error on hover), per
|
||||
model/source/key usage stats, request records filtered by key, and CSV export
|
||||
over a time range; click a model to pin the connection snippet to it.
|
||||
- **Chat**: streaming / non-streaming debug.
|
||||
- **Keys**: create/edit gateway keys, set per-key model scopes (model + source +
|
||||
token quota + reset period); admins manage all keys, users see only their own.
|
||||
- **Priority**: drag blocks to build the tiered AUTO chain.
|
||||
- **Sources**: add/edit/delete upstream sources online (secrets stored
|
||||
encrypted).
|
||||
- **Adapters**: upload / delete Lua adapter scripts.
|
||||
|
||||
Changes persist to `runtime_file` (survive restarts).
|
||||
|
||||
## Lua adapter protocol
|
||||
|
||||
Full API: **[Lua adapter docs](docs/lua-adapters.md)**
|
||||
([中文](docs/lua-adapters-en.md)).
|
||||
|
||||
Each adapter is a Lua script returning a table
|
||||
(`internal/lua/adapters/<name>.lua`), overridable by a same-name file under
|
||||
`adapter_dir` — **no recompile needed**, restart to take effect; uploads via
|
||||
the WebUI take effect immediately.
|
||||
|
||||
```lua
|
||||
return {
|
||||
name = "mysrc",
|
||||
version = "1.0.0",
|
||||
endpoint = "/chat/completions", -- upstream path (source.endpoint overrides)
|
||||
headers = { ["X-Static"] = "v" }, -- static default headers (fallback)
|
||||
|
||||
-- request: convert unified OpenAI req -> upstream native format, return string
|
||||
transform_request = function(raw_json) ... end,
|
||||
|
||||
-- response: convert upstream response to unified format string
|
||||
-- { content, reasoning_content, finish_reason, token_usage{...}, tool_calls[{...}] }
|
||||
transform_response = function(raw_json) ... end,
|
||||
|
||||
-- streaming chunk: convert upstream SSE data to { content, done, ... }; "" skips
|
||||
transform_stream_chunk = function(raw_chunk) ... end,
|
||||
|
||||
-- [optional] dynamic headers / signature hook
|
||||
-- meta = { url, method, body, api_key, timestamp, source={ name, meta={...} } }
|
||||
build_headers = function(meta) return { ["X-App-Sign"] = sign } end,
|
||||
}
|
||||
```
|
||||
|
||||
Shared helpers: `hmac_sha256_hex(key, data)`, `sha256_hex(data)`,
|
||||
`base64_encode(s)`, `tohex(s)`, `json.encode/decode`, `log(level, msg)`.
|
||||
|
||||
### Built-in adapters
|
||||
|
||||
`openai` `deepseek` `anthropic` `gemini` `github` `groq` `mistral` `ollama`
|
||||
`kimicode` `qwen38` `sensenova` `trae`.
|
||||
|
||||
`anthropic`/`gemini`/`ollama` include multimodal conversion
|
||||
(`image_url` → their native format); with `disable_thinking` the `deepseek`
|
||||
adapter sets `extra_body.thinking.type` to `disabled`.
|
||||
|
||||
**kimicode** demonstrates `build_headers`: the cloud validates the calling
|
||||
app, so you HMAC-sign timestamp+URL+body with `meta.app_secret` and add
|
||||
`X-App-Sign`-style headers. Configure `sources[].meta.{app_id, app_secret,
|
||||
app_agent}`.
|
||||
|
||||
## Desktop GUI (Electron)
|
||||
|
||||
Optional standalone product for desktop users on your internal network: an
|
||||
**embedded ModelRouter core** (Clash Verge-style): one-click start/stop, system
|
||||
tray, autostart, silent launch, and a pixel-for-pixel embedded full WebUI
|
||||
(Status/Chat/Keys/Priority/Sources/Adapters — no login needed). Server users
|
||||
keep using the plain Go binary.
|
||||
|
||||
### Two installer kinds: Headless and Desktop
|
||||
|
||||
Release installers ship in two flavors for different audiences:
|
||||
|
||||
- **Headless (server)**: the plain core binary (no Electron), config-driven,
|
||||
for servers / containers / systemd / unattended runs. Ships as `deb` + `rpm`
|
||||
(systemd unit with memory tuning + example config) and `tar.gz` (binary +
|
||||
`config.example.yaml` + `adapters/` + README), Windows as `.zip`.
|
||||
- **Desktop (desktop app)**: the Electron GUI installer with an embedded core,
|
||||
for personal daily desktop use.
|
||||
|
||||
### GUI vs. plain backend
|
||||
|
||||
| Scenario | Pick | Why |
|
||||
| ---- | ---- | ---- |
|
||||
| Server / intranet gateway / unattended long-running | **Headless** (single binary) | ~8 MB binary; RSS scales with the number of sources (~10 MB for one, ~32-35 MB for 16 — see [Memory footprint](#memory-footprint-measured-and-tuned)), zero-dep single process — drop it into systemd or any container, remote admin |
|
||||
| Personal desktop daily use / multi-device intranet sharing | **Desktop** (GUI) | no-login embedded WebUI, tray one-click, autostart, silent background — for non-CLI users |
|
||||
| Windows desktop | **Desktop** | plain backend needs manual service registration; GUI ships native tray/autostart |
|
||||
| CI one-shot 3-platform installers | **Desktop packaging scripts** | `make gui-*` emits deb / AppImage / NSIS, drops straight into a release pipeline |
|
||||
|
||||
Both share the exact same `llmsproxy` core (LuaJIT build) — configs and
|
||||
adapters are fully compatible, freely interchangeable.
|
||||
|
||||
### Build & run
|
||||
Any OpenAI SDK works: point `base_url` at the gateway and use an authorized gateway
|
||||
key as `api_key`.
|
||||
|
||||
## Documentation
|
||||
|
||||
The detailed documentation is currently written in Chinese.
|
||||
|
||||
| Topic | Document |
|
||||
|---|---|
|
||||
| Implementation and the full feature list | [docs/features.md](docs/features.md) |
|
||||
| Configuration: `config.yaml`, model discovery, multiple keys | [docs/configuration.md](docs/configuration.md) |
|
||||
| Routing: model routing, AUTO chain, cooldown, sources vs adapters, source templates | [docs/routing.md](docs/routing.md) |
|
||||
| Adapters: Lua protocol and built-in adapters | [docs/adapters.md](docs/adapters.md) · [docs/lua-adapters-en.md](docs/lua-adapters-en.md) (English) |
|
||||
| Plugins: SDK contract, hooks, UI extensions, HTTP endpoints | [docs/plugins.md](docs/plugins.md) |
|
||||
| Desktop app: Headless / Desktop artifacts, build and run | [docs/gui.md](docs/gui.md) |
|
||||
| Performance: measured memory use, tuning, thinking budget | [docs/performance.md](docs/performance.md) |
|
||||
| WebUI usage | [docs/webui.md](docs/webui.md) |
|
||||
| Development: layout, tests, branch and release discipline | [docs/development.md](docs/development.md) · [docs/git-workflow-en.md](docs/git-workflow-en.md) (English) |
|
||||
| Deployment and migration | [docs/deployment.md](docs/deployment.md) |
|
||||
|
||||
## Renamed from llmsproxy
|
||||
|
||||
The product name has been ModelRouter for a while; the binary, config directory,
|
||||
service name and environment variables are now `modelrouter` as well:
|
||||
|
||||
| Before | After |
|
||||
|---|---|
|
||||
| `llmsproxy` (binary) | `modelrouter` |
|
||||
| `/etc/llmsproxy/` | `/etc/modelrouter/` |
|
||||
| `llmsproxy.service` | `modelrouter.service` |
|
||||
| `LLMSPROXY_*` | `MODELROUTER_*` |
|
||||
|
||||
Existing installs migrate with a dry-run-by-default script (copies, never moves,
|
||||
and can roll back):
|
||||
|
||||
```bash
|
||||
cd cmd/gui
|
||||
npm install
|
||||
make gui # or npm run dev — dev run (needs make build first for bin/llmsproxy)
|
||||
make gui-deb # local deb (share with other Linux users)
|
||||
make gui-dist # deb + rpm + AppImage (linux; rpm needs system rpmbuild)
|
||||
make core-dist # core deb + rpm + tar.gz (server edition; needs nfpm)
|
||||
make gui-win # Windows NSIS (host mingw + wine)
|
||||
make gui-win-docker # Windows NSIS, fully dockerized (wine in image; no host mingw/node/wine)
|
||||
sudo ./packaging/migrate-to-modelrouter.sh # preview
|
||||
sudo ./packaging/migrate-to-modelrouter.sh --apply # apply
|
||||
sudo ./packaging/migrate-to-modelrouter.sh --rollback --apply
|
||||
```
|
||||
|
||||
Artifacts land in `cmd/build/gui-dist/`: `ModelRouter-<ver>.AppImage`,
|
||||
`modelrouter-gui_<ver>_amd64.deb`, `ModelRouter Setup <ver>.exe` (win); core
|
||||
packages land in `cmd/build/dist/`: `llmsproxy_<ver>_amd64.deb`,
|
||||
`llmsproxy-<ver>.x86_64.rpm`. Both target families run
|
||||
`packaging/verify-dist.sh` afterwards — a size floor that rejects degenerate
|
||||
installers (1.3.0 shipped a 264 KB NSIS stub because the host wine was broken
|
||||
and nothing caught it). At release time artifacts are renamed into the
|
||||
`ModelRouter-Desktop-*` (GUI) and `ModelRouter-Headless-*` (plain backend)
|
||||
families before upload to GitCode Releases.
|
||||
Old paths and old environment variable names are still accepted on the read side,
|
||||
so an upgrade cannot fail merely because of ordering. `deploy.sh` refuses to run
|
||||
when it finds the old `/etc/llmsproxy` without a new config, because that would
|
||||
deploy a gateway with no key and no sources.
|
||||
|
||||
#### Windows packaging (dockerized, recommended)
|
||||
|
||||
`make gui-win-docker` does cross-compile + NSIS **entirely in docker**; the host
|
||||
needs **no mingw, no wine, no node**. Wine (with the i386 runtime) lives inside
|
||||
the image, fixing the broken-host-wine failure that produced the 264 KB stub
|
||||
installer.
|
||||
|
||||
- Image: `cmd/gui/docker/win-builder/Dockerfile` (`golang:1.25` + mingw-w64 +
|
||||
node + wine32/wine64 + prebuilt LuaJIT for Windows: `lua51.dll` + import lib).
|
||||
LuaJIT source comes from the gitcode mirror (reachable in CN) with github
|
||||
fallback; Go modules go through `https://goproxy.cn`.
|
||||
- One-shot release: `cmd/gui/scripts/dist-win-docker.sh` — core build + NSIS
|
||||
packaging run inside the container (`USE_SYSTEM_WINE=true` uses the image's
|
||||
wine); electron-builder's ~400 MB toolchain cache lives in a docker volume so
|
||||
later builds reuse it.
|
||||
- The `modelrouter/win-builder` image is auto-built on first run (~5-10 min)
|
||||
and reused afterwards.
|
||||
- After packaging, `packaging/verify-dist.sh` runs as a gate: an exe under
|
||||
5 MB (or a deb/rpm/7z under 10 MB) fails the build — a broken installer can
|
||||
never reach a Release again.
|
||||
|
||||
```bash
|
||||
# Full Windows release (rebuild core + NSIS):
|
||||
make gui-win-docker
|
||||
```
|
||||
|
||||
### Features
|
||||
|
||||
- **Embedded core**: auto-spawns the bundled `llmsproxy` (LuaJIT build);
|
||||
config/keys/adapters live in `<userData>/profile/`. First run generates a
|
||||
random admin key written into `keys` (not seed) and self-injects it — WebUI
|
||||
needs no login and never nags about changing the initial key. Port changeable
|
||||
in settings (default 8787).
|
||||
- **System tray**: core status dot, autostart/silent toggles, start/stop/restart
|
||||
core, quit.
|
||||
- **Silent launch**: `--silent` flag or setting — starts to the tray only;
|
||||
first run always shows the window.
|
||||
- **Autostart**: `setLoginItemSettings` on Windows/macOS; writes
|
||||
`~/.config/autostart/modelrouter-gui.desktop` on Linux, appending `--silent`
|
||||
when silent mode is on.
|
||||
The plugin SDK follows the same naming: the only public entry point is
|
||||
`modelrouter` (`llmsproxy` remains a compatibility alias). See
|
||||
[docs/plugins.md](docs/plugins.md).
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
cmd/llmsproxy # entry point
|
||||
internal/config # YAML config loading/validation
|
||||
internal/lua # LuaJIT VM + worker pool + AdapterCache + built-ins (embed)
|
||||
```text
|
||||
cmd/modelrouter # entry point (pure Go server)
|
||||
cmd/gui # Electron desktop shell (embeds the same core)
|
||||
internal/config # config parsing, sealed storage, runtime file
|
||||
internal/core # scheduling, kernel actions, state and audit
|
||||
internal/lua # LuaJIT VM + elastic worker pool + built-in adapters (embed)
|
||||
internal/lua/plugins # optional plugin examples (not shipped in binary/archives)
|
||||
internal/provider # Provider(HTTP) + Registry(routing)
|
||||
internal/gateway # OpenAI-compatible HTTP + auth + SDK/streaming + WebUI
|
||||
internal/types # unified format & OpenAI wire types
|
||||
internal/gateway # OpenAI-compatible HTTP service + auth + streaming + WebUI
|
||||
internal/types # unified format and OpenAI wire types
|
||||
packaging/ # deb/rpm/tar packaging, systemd unit, migration script
|
||||
docs/ # topic documentation
|
||||
```
|
||||
|
||||
## Development & collaboration
|
||||
|
||||
- **Git branching workflow**: main / feature / release-branch conventions in
|
||||
**[docs/git-workflow-en.md](docs/git-workflow-en.md)**
|
||||
([中文](docs/git-workflow.md)). The line: `main` is always deployable →
|
||||
features merge into `main` → cut `release/vX.Y.Z` and tag → release → hotfixes
|
||||
land on the release branch and are cherry-picked back into `main`.
|
||||
- **Lua adapter protocol**: see
|
||||
**[docs/lua-adapters-en.md](docs/lua-adapters-en.md)**
|
||||
([中文](docs/lua-adapters.md)).
|
||||
|
||||
## Tests
|
||||
## Development and tests
|
||||
|
||||
```bash
|
||||
go test -tags luajit ./...
|
||||
go test -tags luajit ./... # everything
|
||||
go test -tags luajit ./internal/lua # one package (required for LuaJIT changes)
|
||||
```
|
||||
|
||||
Covers: config validation, adapter load/transform, signature hooks, gateway
|
||||
auth, SDK round trip, SSE streaming, model routing, multimodal passthrough and
|
||||
disable_thinking.
|
||||
- Branch and release discipline: [docs/git-workflow-en.md](docs/git-workflow-en.md).
|
||||
- Plugins live in `plugin_dir`. The gateway never creates, overwrites or deletes
|
||||
plugin files, and a desktop upgrade never touches them either.
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
// ModelRouter Desktop GUI — main process
|
||||
// Clash-Verge-style: bundles and manages an embedded llmsproxy core instance
|
||||
// Desktop shell: bundles and manages an embedded modelrouter core instance
|
||||
// in a per-user profile dir (userData), auto-starts it, keeps a tray icon,
|
||||
// supports launch-at-login. Server-side users keep using the pure Go binary.
|
||||
const {
|
||||
@ -18,6 +18,7 @@ const http = require("http");
|
||||
const os = require("os");
|
||||
const crypto = require("crypto");
|
||||
const { spawn, execFileSync } = require("child_process");
|
||||
const pluginSync = require("./plugin-sync");
|
||||
|
||||
const LOG_FILE = path.join(app.getPath("userData"), "gui.log");
|
||||
function log(msg) {
|
||||
@ -40,13 +41,36 @@ console.error = function () {
|
||||
// Per-user profile dir: <userData>/profile
|
||||
const PROFILE_DIR = path.join(app.getPath("userData"), "profile");
|
||||
const CONFIG_FILE = path.join(PROFILE_DIR, "config.yaml");
|
||||
// Plugin tree for the embedded core. The core deliberately does NOT default
|
||||
// plugin_dir (an operator opts in), so the desktop build must set it — without
|
||||
// it the whole plugin system stays off and the channels plugin (multi-platform
|
||||
// account login + credential vault + auto check-in) is invisible in the WebUI.
|
||||
// Seeding is the core's job (SeedBundled fills MISSING files only), so a user's
|
||||
// edited copy always wins.
|
||||
const PLUGINS_DIR = path.join(PROFILE_DIR, "plugins");
|
||||
// Packaged: extraResource at process.resourcesPath/bin (outside asar, spawnable).
|
||||
// Dev: cmd/gui/bin, populated by prepare-core.js.
|
||||
const CORE_EXE = path.join(
|
||||
// The embedded core is named after the product. Installs produced before the
|
||||
// llmsproxy -> modelrouter rename still carry the old file name inside their
|
||||
// profile, so resolution falls back to it instead of declaring the core missing.
|
||||
const CORE_BIN_DIR = path.join(
|
||||
app.isPackaged ? process.resourcesPath : __dirname,
|
||||
"bin",
|
||||
);
|
||||
const CORE_EXE = path.join(
|
||||
CORE_BIN_DIR,
|
||||
process.platform === "win32" ? "modelrouter.exe" : "modelrouter",
|
||||
);
|
||||
const LEGACY_CORE_EXE = path.join(
|
||||
CORE_BIN_DIR,
|
||||
process.platform === "win32" ? "llmsproxy.exe" : "llmsproxy",
|
||||
);
|
||||
|
||||
function resolveCoreExe() {
|
||||
if (fs.existsSync(CORE_EXE)) return CORE_EXE;
|
||||
if (fs.existsSync(LEGACY_CORE_EXE)) return LEGACY_CORE_EXE;
|
||||
return CORE_EXE;
|
||||
}
|
||||
// Packaged window icon: must be a REAL file on disk (extraResource), not an
|
||||
// asar path — Electron/GTK pulls BrowserWindow icons from the filesystem and
|
||||
// crashes (SIGSEGV) when given an asar virtual path on Linux.
|
||||
@ -155,7 +179,7 @@ function unsealViaCore() {
|
||||
let key = "";
|
||||
try {
|
||||
const out = execFileSync(
|
||||
CORE_EXE,
|
||||
resolveCoreExe(),
|
||||
["-config", CONFIG_FILE, "-show-secrets"],
|
||||
{
|
||||
encoding: "utf-8",
|
||||
@ -207,7 +231,7 @@ function coreStarted() {
|
||||
function writeDefaultEmbeddedConfig() {
|
||||
// Generate a self-contained embedded profile with a fresh random admin key.
|
||||
// No upstream sources are preconfigured — the desktop user adds them via
|
||||
// the WebUI's sources page (like Clash Verge's empty-by-default profile).
|
||||
// the WebUI's sources page (empty by default, like the WebUI profile).
|
||||
// If a config already exists we do NOT overwrite it (keep user's edits).
|
||||
if (fs.existsSync(CONFIG_FILE)) return;
|
||||
const key = "sk-gw-" + crypto.randomBytes(16).toString("hex");
|
||||
@ -216,6 +240,9 @@ function writeDefaultEmbeddedConfig() {
|
||||
"listen: 127.0.0.1:" + (settings.port || DEFAULT_PORT),
|
||||
"default_model: AUTO",
|
||||
"adapter_dir: " + JSON.stringify(path.join(PROFILE_DIR, "adapters")),
|
||||
// Plugin tree: channels/billing/quota/usage ship inside the core binary and
|
||||
// are seeded into this dir on first start. Omitting it disables plugins.
|
||||
"plugin_dir: " + JSON.stringify(PLUGINS_DIR),
|
||||
"runtime_file: " + JSON.stringify(path.join(PROFILE_DIR, "runtime.json")),
|
||||
"",
|
||||
// Admin key as a plain `keys` entry (no seed flag) so the web UI never
|
||||
@ -261,15 +288,18 @@ function startCore() {
|
||||
console.log("core already running");
|
||||
return;
|
||||
}
|
||||
if (!fs.existsSync(CORE_EXE)) {
|
||||
console.error("embedded core binary missing:", CORE_EXE);
|
||||
const coreExe = resolveCoreExe();
|
||||
if (!fs.existsSync(coreExe)) {
|
||||
console.error("embedded core binary missing:", coreExe);
|
||||
return;
|
||||
}
|
||||
// ensure profile dir + config + local adapters exist
|
||||
fs.mkdirSync(PROFILE_DIR, { recursive: true });
|
||||
if (!fs.existsSync(CONFIG_FILE)) writeDefaultEmbeddedConfig();
|
||||
pluginSync.ensurePluginDirConfigured(CONFIG_FILE, PLUGINS_DIR, log);
|
||||
fs.mkdirSync(PLUGINS_DIR, { recursive: true });
|
||||
|
||||
coreProc = spawn(CORE_EXE, ["-config", CONFIG_FILE], {
|
||||
coreProc = spawn(coreExe, ["-config", CONFIG_FILE], {
|
||||
cwd: PROFILE_DIR,
|
||||
env: Object.assign({}, process.env, {
|
||||
LLMS_PROXY_GUI: "1",
|
||||
@ -622,12 +652,57 @@ function createWindow() {
|
||||
},
|
||||
});
|
||||
mainWindow.loadFile(path.join(__dirname, "renderer", "index.html"));
|
||||
// External links ALWAYS go to the user's default browser, never an in-app
|
||||
// window. Three paths have to be covered, and missing any one of them is why
|
||||
// "click the login link and nothing happens / the app navigates away" shows up:
|
||||
// 1. window.open / target=_blank -> setWindowOpenHandler
|
||||
// 2. top-level navigation (same-tab click) -> will-navigate
|
||||
// 3. navigation inside the WebUI <iframe> -> will-frame-navigate
|
||||
// The embedded core's own origin (and file:// for the shell) must keep
|
||||
// working, otherwise the WebUI and plugin pages stop loading.
|
||||
const isInternal = (url) =>
|
||||
!url ||
|
||||
url === "about:blank" ||
|
||||
url.startsWith("file://") ||
|
||||
url.startsWith(embeddedBaseUrl()) ||
|
||||
url.startsWith("devtools://");
|
||||
const openExternally = (url) => {
|
||||
if (url && (url.startsWith("http://") || url.startsWith("https://"))) {
|
||||
log("[gui] open in default browser: " + url);
|
||||
shell.openExternal(url);
|
||||
}
|
||||
};
|
||||
mainWindow.webContents.setWindowOpenHandler(({ url }) => {
|
||||
if (url && url.startsWith("http")) shell.openExternal(url);
|
||||
if (!isInternal(url)) openExternally(url);
|
||||
return { action: "deny" };
|
||||
});
|
||||
mainWindow.webContents.on("will-navigate", (e, url) => {
|
||||
if (isInternal(url)) return;
|
||||
e.preventDefault();
|
||||
openExternally(url);
|
||||
});
|
||||
// iframe navigations (the WebUI content area and the plugin pages inside it).
|
||||
// The channels plugin renders its server-side login browser (noVNC) in an
|
||||
// iframe on a LOOPBACK port, so loopback MUST stay in-app — blocking it would
|
||||
// break VNC login. Everything non-loopback is external: open the default
|
||||
// browser instead of silently replacing the gateway UI with a third-party page.
|
||||
const isLoopback = (url) => {
|
||||
try {
|
||||
const h = new URL(url).hostname;
|
||||
return h === "127.0.0.1" || h === "localhost" || h === "[::1]";
|
||||
} catch (e) {
|
||||
return false;
|
||||
}
|
||||
};
|
||||
if (mainWindow.webContents.on) {
|
||||
mainWindow.webContents.on("will-frame-navigate", (e) => {
|
||||
if (isInternal(e.url) || isLoopback(e.url)) return;
|
||||
e.preventDefault();
|
||||
openExternally(e.url);
|
||||
});
|
||||
}
|
||||
// Silent start: `--silent` CLI flag or settings.silentStart keeps the app
|
||||
// in the tray without showing the window (Clash-Verge style). First run
|
||||
// in the tray without showing the window (desktop-shell behaviour). First run
|
||||
// always shows the window so the user sees the dashboard.
|
||||
mainWindow.once("ready-to-show", () => {
|
||||
const firstRun = !fs.existsSync(
|
||||
@ -751,8 +826,8 @@ ipcMain.handle("core:state", () => ({
|
||||
port: settings.port || DEFAULT_PORT,
|
||||
profileDir: PROFILE_DIR,
|
||||
configFile: CONFIG_FILE,
|
||||
coreExe: CORE_EXE,
|
||||
coreExists: fs.existsSync(CORE_EXE),
|
||||
coreExe: resolveCoreExe(),
|
||||
coreExists: fs.existsSync(resolveCoreExe()),
|
||||
}));
|
||||
ipcMain.handle("core:start", () => {
|
||||
app.isQuitting = false;
|
||||
@ -824,7 +899,7 @@ if (gotLock) {
|
||||
createWindow();
|
||||
createTray();
|
||||
|
||||
// start embedded core if not already running (another instance of llmsproxy)
|
||||
// start embedded core if not already running (another instance of modelrouter)
|
||||
setTimeout(async () => {
|
||||
const p = await probeEmbedded();
|
||||
if (!p.ok) {
|
||||
|
||||
14
cmd/gui/package-lock.json
generated
14
cmd/gui/package-lock.json
generated
@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "modelrouter-gui",
|
||||
"version": "1.9.3",
|
||||
"version": "3.0.1",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "modelrouter-gui",
|
||||
"version": "1.9.3",
|
||||
"version": "3.0.1",
|
||||
"license": "Proprietary",
|
||||
"devDependencies": {
|
||||
"electron": "^33.0.0",
|
||||
@ -1245,7 +1245,7 @@
|
||||
}
|
||||
},
|
||||
"node_modules/chownr": {
|
||||
"version": "3.0.0",
|
||||
"version": "3.0.1",
|
||||
"resolved": "https://registry.npmjs.org/chownr/-/chownr-3.0.0.tgz",
|
||||
"integrity": "sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==",
|
||||
"dev": true,
|
||||
@ -2288,7 +2288,7 @@
|
||||
}
|
||||
},
|
||||
"node_modules/global-agent": {
|
||||
"version": "3.0.0",
|
||||
"version": "3.0.1",
|
||||
"resolved": "https://registry.npmjs.org/global-agent/-/global-agent-3.0.0.tgz",
|
||||
"integrity": "sha512-PT6XReJ+D07JvGoxQMkT6qji/jVNfX/h364XHZOWeRzy64sSFr+xJ5OX7LI3b4MPQzdL4H8Y8M0xzPpsVMwA8Q==",
|
||||
"dev": true,
|
||||
@ -2532,7 +2532,7 @@
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/is-fullwidth-code-point": {
|
||||
"version": "3.0.0",
|
||||
"version": "3.0.1",
|
||||
"resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz",
|
||||
"integrity": "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg==",
|
||||
"dev": true,
|
||||
@ -2715,7 +2715,7 @@
|
||||
}
|
||||
},
|
||||
"node_modules/matcher": {
|
||||
"version": "3.0.0",
|
||||
"version": "3.0.1",
|
||||
"resolved": "https://registry.npmjs.org/matcher/-/matcher-3.0.0.tgz",
|
||||
"integrity": "sha512-OkeDaAZ/bQCxeFAozM55PKcKU0yJMPGifLwV4Qgjitu+5MoAfSQN4lsLJeXZ1b8w0x+/Emda6MZgXS1jvsapng==",
|
||||
"dev": true,
|
||||
@ -3492,7 +3492,7 @@
|
||||
}
|
||||
},
|
||||
"node_modules/shebang-regex": {
|
||||
"version": "3.0.0",
|
||||
"version": "3.0.1",
|
||||
"resolved": "https://registry.npmjs.org/shebang-regex/-/shebang-regex-3.0.0.tgz",
|
||||
"integrity": "sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A==",
|
||||
"dev": true,
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "modelrouter-gui",
|
||||
"version": "1.9.7",
|
||||
"version": "3.0.1",
|
||||
"description": "ModelRouter Desktop — embedded ModelRouter core with tray",
|
||||
"author": "ModelRouter",
|
||||
"main": "main.js",
|
||||
@ -39,8 +39,8 @@
|
||||
},
|
||||
"extraResources": [
|
||||
{
|
||||
"from": "bin/llmsproxy",
|
||||
"to": "bin/llmsproxy"
|
||||
"from": "bin/modelrouter",
|
||||
"to": "bin/modelrouter"
|
||||
},
|
||||
{
|
||||
"from": "build/icon.png",
|
||||
@ -66,8 +66,8 @@
|
||||
"icon": "build/icon.ico",
|
||||
"extraResources": [
|
||||
{
|
||||
"from": "bin/llmsproxy.exe",
|
||||
"to": "bin/llmsproxy.exe"
|
||||
"from": "bin/modelrouter.exe",
|
||||
"to": "bin/modelrouter.exe"
|
||||
},
|
||||
{
|
||||
"from": "bin/lua51.dll",
|
||||
@ -81,6 +81,7 @@
|
||||
},
|
||||
"files": [
|
||||
"main.js",
|
||||
"plugin-sync.js",
|
||||
"preload.js",
|
||||
"renderer/**/*",
|
||||
"build/**/*",
|
||||
|
||||
33
cmd/gui/plugin-sync.js
Normal file
33
cmd/gui/plugin-sync.js
Normal file
@ -0,0 +1,33 @@
|
||||
// Plugin-tree maintenance for the desktop profile (no Electron imports, so this
|
||||
// is unit-testable with plain node).
|
||||
//
|
||||
// The desktop may configure an operator-selected plugin directory, but it never
|
||||
// owns the files in that directory. In particular, upgrades must not seed,
|
||||
// replace, rename, or delete plugins. Plugin installation is an explicit
|
||||
// operator action (or an external package manager action).
|
||||
const fs = require("fs");
|
||||
|
||||
// ensurePluginDirConfigured back-fills `plugin_dir` into an existing config.
|
||||
// Appending a top-level YAML key is safe for the shapes this file can have
|
||||
// (flat mapping + a keys list). Returns "added" | "present" | "absent" | "error".
|
||||
function ensurePluginDirConfigured(configFile, pluginsDir, log) {
|
||||
try {
|
||||
if (!fs.existsSync(configFile)) return "absent";
|
||||
const cur = fs.readFileSync(configFile, "utf-8");
|
||||
if (/^\s*plugin_dir\s*:/m.test(cur)) return "present";
|
||||
fs.writeFileSync(
|
||||
configFile,
|
||||
cur.replace(/\s*$/, "") +
|
||||
"\n# added by ModelRouter Desktop: enables the operator plugin directory\n" +
|
||||
"plugin_dir: " + JSON.stringify(pluginsDir) + "\n",
|
||||
"utf-8",
|
||||
);
|
||||
if (log) log("[gui] plugin_dir back-filled into " + configFile);
|
||||
return "added";
|
||||
} catch (e) {
|
||||
if (log) log("[gui] ensurePluginDirConfigured failed: " + e.message);
|
||||
return "error";
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { ensurePluginDirConfigured };
|
||||
48
cmd/gui/plugin-sync.test.js
Normal file
48
cmd/gui/plugin-sync.test.js
Normal file
@ -0,0 +1,48 @@
|
||||
// Unit tests for desktop plugin-directory configuration (plain node).
|
||||
const test = require("node:test");
|
||||
const assert = require("node:assert");
|
||||
const fs = require("fs");
|
||||
const os = require("os");
|
||||
const path = require("path");
|
||||
const { ensurePluginDirConfigured } = require("./plugin-sync");
|
||||
|
||||
const tmp = () => fs.mkdtempSync(path.join(os.tmpdir(), "plugin-sync-"));
|
||||
|
||||
test("back-fills plugin_dir into an existing profile without it", () => {
|
||||
const dir = tmp();
|
||||
const cfg = path.join(dir, "config.yaml");
|
||||
fs.writeFileSync(cfg, "listen: 127.0.0.1:8787\ndefault_model: AUTO\nkeys:\n - key: k\n role: admin\n");
|
||||
const plugins = path.join(dir, "plugins");
|
||||
assert.equal(ensurePluginDirConfigured(cfg, plugins, null), "added");
|
||||
const out = fs.readFileSync(cfg, "utf-8");
|
||||
assert.match(out, /^plugin_dir: "/m);
|
||||
assert.match(out, /^listen: 127.0.0.1:8787$/m);
|
||||
});
|
||||
|
||||
test("leaves a profile that already sets plugin_dir alone", () => {
|
||||
const dir = tmp();
|
||||
const cfg = path.join(dir, "config.yaml");
|
||||
const orig = 'plugin_dir: "/custom/plugins"\nlisten: ":1"\n';
|
||||
fs.writeFileSync(cfg, orig);
|
||||
assert.equal(ensurePluginDirConfigured(cfg, "/whatever", null), "present");
|
||||
assert.equal(fs.readFileSync(cfg, "utf-8"), orig);
|
||||
});
|
||||
|
||||
test("missing config is not created by the back-fill", () => {
|
||||
const dir = tmp();
|
||||
const cfg = path.join(dir, "config.yaml");
|
||||
assert.equal(ensurePluginDirConfigured(cfg, path.join(dir, "p"), null), "absent");
|
||||
assert.equal(fs.existsSync(cfg), false);
|
||||
});
|
||||
|
||||
test("desktop upgrade never modifies operator plugin files", () => {
|
||||
const dir = tmp();
|
||||
const plugins = path.join(dir, "plugins");
|
||||
fs.mkdirSync(plugins, { recursive: true });
|
||||
fs.writeFileSync(path.join(plugins, "channels.lua"), "OPERATOR FILE");
|
||||
fs.writeFileSync(path.join(plugins, "my-own.lua"), "MINE");
|
||||
// There is deliberately no version stamp and no upgrade synchronizer.
|
||||
assert.equal(fs.readFileSync(path.join(plugins, "channels.lua"), "utf-8"), "OPERATOR FILE");
|
||||
assert.equal(fs.readFileSync(path.join(plugins, "my-own.lua"), "utf-8"), "MINE");
|
||||
assert.equal(fs.readdirSync(dir).includes(".plugin-seed-version"), false);
|
||||
});
|
||||
@ -1,5 +1,5 @@
|
||||
// ModelRouter Desktop GUI — renderer
|
||||
// Clash-Verge-style: the content area is a full-screen iframe loading the
|
||||
// Desktop shell: the content area is a full-screen iframe loading the
|
||||
// real WebUI 1:1 (status / chat / keys / priority / sources / adapters).
|
||||
// The shell only adds the titlebar, settings overlay and tray integration.
|
||||
const state = {
|
||||
|
||||
@ -83,7 +83,7 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- side rail: autostart / silent toggles (Clash-Verge style) -->
|
||||
<!-- side rail: autostart / silent toggles (desktop-shell style) -->
|
||||
<aside class="rail" id="rail">
|
||||
<div class="rail-gap"></div>
|
||||
<button
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
/* =====================================================================
|
||||
ModelRouter Desktop GUI — Clash-Verge-style shell
|
||||
ModelRouter Desktop GUI — desktop-shell-style shell
|
||||
Content area = full-screen iframe embedding the real WebUI 1:1.
|
||||
Palette mirrors the WebUI (sakura/frost/glassmorphism).
|
||||
===================================================================== */
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
# One-shot Windows release build for the GUI — ENTIRELY inside docker.
|
||||
#
|
||||
# 1) cross-compile the embedded core in docker (llmsproxy.exe + lua51.dll)
|
||||
# 1) cross-compile the embedded core in docker (modelrouter.exe + lua51.dll)
|
||||
# 2) electron-builder NSIS packaging ALSO inside docker (wine for the
|
||||
# uninstaller step lives in the image — see Dockerfile). This is the fix
|
||||
# for the 1.3.0/1.4.0 broken installers: the host had amd64-only wine
|
||||
@ -37,13 +37,15 @@ docker run --rm \
|
||||
-v "$ROOT":/workspace \
|
||||
-v "$CACHE_VOL":/root/.cache \
|
||||
-e GOPROXY="$PROXY" \
|
||||
-e ELECTRON_MIRROR="https://npmmirror.com/mirrors/electron/" \
|
||||
-e ELECTRON_BUILDER_BINARIES_MIRROR="https://npmmirror.com/mirrors/electron-builder-binaries/" \
|
||||
-w /workspace \
|
||||
"$IMAGE" bash -c '
|
||||
set -euo pipefail
|
||||
cp /opt/luajit2/src/lua51.dll /workspace/cmd/gui/bin/lua51.dll
|
||||
GOOS=windows GOARCH=amd64 CGO_ENABLED=1 CC=x86_64-w64-mingw32-gcc \
|
||||
go build -buildvcs=false -tags luajit -trimpath \
|
||||
-o /workspace/cmd/gui/bin/llmsproxy.exe ./cmd/llmsproxy
|
||||
go build -buildvcs=false -tags luajit -trimpath -ldflags "-s -w" \
|
||||
-o /workspace/cmd/gui/bin/modelrouter.exe ./cmd/modelrouter
|
||||
echo " -> core built"
|
||||
# wine needs one prefix initialization so its wow64 runtime (syswow64/ntdll)
|
||||
# exists — the exact thing that was missing on the broken host. Use a shared
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env node
|
||||
// Build the embedded llmsproxy core(s) for the GUI package.
|
||||
// Build the embedded modelrouter core(s) for the GUI package.
|
||||
// node prepare-core.js -> linux (native, luajit) + win (cross, best effort)
|
||||
// node prepare-core.js linux -> linux only
|
||||
// node prepare-core.js win -> win only
|
||||
@ -12,21 +12,24 @@ const binDir = path.join(guiDir, "bin");
|
||||
const projectRoot = path.join(guiDir, "..", "..");
|
||||
|
||||
function buildLinux() {
|
||||
const out = path.join(binDir, "llmsproxy");
|
||||
console.log("[prepare-core] building llmsproxy (linux/luajit)...");
|
||||
const out = path.join(binDir, "modelrouter");
|
||||
console.log("[prepare-core] building modelrouter (linux/luajit)...");
|
||||
fs.mkdirSync(binDir, { recursive: true });
|
||||
execFileSync(
|
||||
"go",
|
||||
["build", "-tags", "luajit", "-trimpath", "-o", out, "./cmd/llmsproxy"],
|
||||
// -s -w: same stripping as packaging/core-dist.sh. Without it the bundled
|
||||
// core carries debug info (~14.4 MB vs ~10.3 MB) and every installer pays
|
||||
// for it, for a binary the user never debugs.
|
||||
["build", "-tags", "luajit", "-trimpath", "-ldflags", "-s -w", "-o", out, "./cmd/modelrouter"],
|
||||
{ cwd: projectRoot, stdio: "inherit" },
|
||||
);
|
||||
return out;
|
||||
}
|
||||
|
||||
function buildWin() {
|
||||
const out = path.join(binDir, "llmsproxy.exe");
|
||||
const out = path.join(binDir, "modelrouter.exe");
|
||||
console.log(
|
||||
"[prepare-core] cross-building llmsproxy.exe (windows/amd64, luajit)...",
|
||||
"[prepare-core] cross-building modelrouter.exe (windows/amd64, luajit)...",
|
||||
);
|
||||
fs.mkdirSync(binDir, { recursive: true });
|
||||
const cc = process.env.MINGW_CC || "x86_64-w64-mingw32-gcc";
|
||||
@ -43,7 +46,7 @@ function buildWin() {
|
||||
}
|
||||
execFileSync(
|
||||
"go",
|
||||
["build", "-tags", "luajit", "-trimpath", "-o", out, "./cmd/llmsproxy"],
|
||||
["build", "-tags", "luajit", "-trimpath", "-ldflags", "-s -w", "-o", out, "./cmd/modelrouter"],
|
||||
{
|
||||
cwd: projectRoot,
|
||||
stdio: "inherit",
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
// Command llmsproxy is a standalone gateway that exposes multiple upstream LLM
|
||||
// Command modelrouter is a standalone gateway that exposes multiple upstream LLM
|
||||
// sources behind a unified OpenAI-compatible HTTP API. Protocol differences are
|
||||
// handled by Lua adapters (per source), optionally signing outgoing requests.
|
||||
// It supports explicit model selection or AUTO routing by priority, image
|
||||
@ -16,9 +16,9 @@ import (
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/core"
|
||||
"llmsproxy/internal/gateway"
|
||||
"modelrouter/internal/config"
|
||||
"modelrouter/internal/core"
|
||||
"modelrouter/internal/gateway"
|
||||
)
|
||||
|
||||
func main() {
|
||||
@ -33,12 +33,12 @@ func main() {
|
||||
// error here instead of being silently created.
|
||||
if *checkOnly {
|
||||
if _, err := os.Stat(*cfgPath); err != nil {
|
||||
log.Fatalf("[llmsproxy] check: %v", err)
|
||||
log.Fatalf("[modelrouter] check: %v", err)
|
||||
}
|
||||
if _, err := config.Load(*cfgPath); err != nil {
|
||||
log.Fatalf("[llmsproxy] check: %v", err)
|
||||
log.Fatalf("[modelrouter] check: %v", err)
|
||||
}
|
||||
log.Printf("[llmsproxy] check: %s is valid", *cfgPath)
|
||||
log.Printf("[modelrouter] check: %s is valid", *cfgPath)
|
||||
return
|
||||
}
|
||||
|
||||
@ -48,34 +48,34 @@ func main() {
|
||||
// program prints a credential, and this path never logs to a file.
|
||||
if *showSecrets {
|
||||
if _, err := os.Stat(*cfgPath); err != nil {
|
||||
log.Fatalf("[llmsproxy] show-secrets: %v", err)
|
||||
log.Fatalf("[modelrouter] show-secrets: %v", err)
|
||||
}
|
||||
if err := config.PrintSecrets(*cfgPath); err != nil {
|
||||
log.Fatalf("[llmsproxy] show-secrets: %v", err)
|
||||
log.Fatalf("[modelrouter] show-secrets: %v", err)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
created, err := config.EnsureDefault(*cfgPath)
|
||||
if err != nil {
|
||||
log.Fatalf("[llmsproxy] config: %v", err)
|
||||
log.Fatalf("[modelrouter] config: %v", err)
|
||||
}
|
||||
|
||||
c, err := core.New(*cfgPath)
|
||||
if err != nil {
|
||||
log.Fatalf("[llmsproxy] core: %v", err)
|
||||
log.Fatalf("[modelrouter] core: %v", err)
|
||||
}
|
||||
defer c.Close()
|
||||
|
||||
if created {
|
||||
for _, k := range c.GatewayKeys() {
|
||||
log.Printf("[llmsproxy] generated default config at %s — admin key: %s (rotate it in the WebUI after first login)", *cfgPath, k)
|
||||
log.Printf("[modelrouter] generated default config at %s — admin key: %s (rotate it in the WebUI after first login)", *cfgPath, k)
|
||||
}
|
||||
}
|
||||
|
||||
gw, err := gateway.New(c)
|
||||
if err != nil {
|
||||
log.Fatalf("[llmsproxy] gateway: %v", err)
|
||||
log.Fatalf("[modelrouter] gateway: %v", err)
|
||||
}
|
||||
|
||||
// Ops hygiene: surface the two most common footguns instead of silently
|
||||
@ -88,16 +88,16 @@ func main() {
|
||||
// (e.g. after rotating the starter key away) it reported "every request is
|
||||
// rejected" while seven working keys, one of them admin, were in service.
|
||||
if keys := c.ListKeys(); len(keys) == 0 {
|
||||
log.Printf("[llmsproxy] WARNING: no gateway key configured — every request will be rejected. Add one in the WebUI or set gateway_keys")
|
||||
log.Printf("[modelrouter] WARNING: no gateway key configured — every request will be rejected. Add one in the WebUI or set gateway_keys")
|
||||
} else {
|
||||
for _, k := range keys {
|
||||
if k.Key == "sk-gw-local-0001" || k.Key == "sk-local-0001" {
|
||||
log.Printf("[llmsproxy] WARNING: gateway key %q looks like the starter/example key — rotate it before exposing the gateway", k.Key)
|
||||
log.Printf("[modelrouter] WARNING: gateway key %q looks like the starter/example key — rotate it before exposing the gateway", k.Key)
|
||||
}
|
||||
}
|
||||
}
|
||||
if l := c.Listen(); strings.HasPrefix(l, "0.0.0.0:") || strings.HasPrefix(l, "::") {
|
||||
log.Printf("[llmsproxy] WARNING: listen=%s binds ALL interfaces — bind an internal address in production", l)
|
||||
log.Printf("[modelrouter] WARNING: listen=%s binds ALL interfaces — bind an internal address in production", l)
|
||||
}
|
||||
|
||||
srv := &http.Server{
|
||||
@ -108,22 +108,22 @@ func main() {
|
||||
go func() {
|
||||
cert, key := c.TLS()
|
||||
if cert != "" && key != "" {
|
||||
log.Printf("[llmsproxy] listening HTTPS on %s (cert=%q) (default_model=%s, models=%v, adapters=%d)",
|
||||
log.Printf("[modelrouter] listening HTTPS on %s (cert=%q) (default_model=%s, models=%v, adapters=%d)",
|
||||
c.Listen(), cert, c.DefaultModel(), c.Registry().ModelList(), len(c.ListAdapters()))
|
||||
if err := srv.ListenAndServeTLS(cert, key); err != nil && err != http.ErrServerClosed {
|
||||
log.Fatalf("[llmsproxy] server: %v", err)
|
||||
log.Fatalf("[modelrouter] server: %v", err)
|
||||
}
|
||||
return
|
||||
}
|
||||
log.Printf("[llmsproxy] listening HTTP on %s (default_model=%s, models=%v, adapters=%d)",
|
||||
log.Printf("[modelrouter] listening HTTP on %s (default_model=%s, models=%v, adapters=%d)",
|
||||
c.Listen(), c.DefaultModel(), c.Registry().ModelList(), len(c.ListAdapters()))
|
||||
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
|
||||
log.Fatalf("[llmsproxy] server: %v", err)
|
||||
log.Fatalf("[modelrouter] server: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
stop := make(chan os.Signal, 1)
|
||||
signal.Notify(stop, syscall.SIGINT, syscall.SIGTERM)
|
||||
<-stop
|
||||
log.Printf("[llmsproxy] shutting down")
|
||||
log.Printf("[modelrouter] shutting down")
|
||||
}
|
||||
90
deploy.sh
90
deploy.sh
@ -1,10 +1,10 @@
|
||||
#!/usr/bin/env bash
|
||||
# ============================================================
|
||||
# llmsproxy 原子化滚动部署脚本
|
||||
# modelrouter 原子化滚动部署脚本
|
||||
#
|
||||
# 策略:
|
||||
# 1. /tmp (tmpfs) 构建,/usr/local/bin (ext4) 部署
|
||||
# 2. 构建后先 copy 到目标文件系统(.llmsproxy.tmp),
|
||||
# 2. 构建后先 copy 到目标文件系统(.modelrouter.tmp),
|
||||
# 再 rename(2) 做原子替换(同一文件系统内 mv 是原子的)
|
||||
# 3. runtime.json 不替换(保留运行时密钥、审计状态)
|
||||
# 4. systemctl restart → 服务中断仅在 Go 进程重启瞬间 (<500ms)
|
||||
@ -28,20 +28,21 @@
|
||||
set -euo pipefail
|
||||
|
||||
# ---------- 配置 ----------
|
||||
readonly TARGET_BIN="/usr/local/bin/llmsproxy"
|
||||
readonly TARGET_CONFIG="/etc/llmsproxy/config.yaml"
|
||||
readonly TARGET_ADAPTERS="/etc/llmsproxy/adapters"
|
||||
readonly TARGET_PLUGINS="/etc/llmsproxy/plugins"
|
||||
readonly REPO_DIR="/home/program/llmsproxy"
|
||||
readonly TARGET_BIN="/usr/local/bin/modelrouter"
|
||||
readonly TARGET_CONFIG="/etc/modelrouter/config.yaml"
|
||||
readonly TARGET_ADAPTERS="/etc/modelrouter/adapters"
|
||||
readonly TARGET_PLUGINS="/etc/modelrouter/plugins"
|
||||
# 仓库所在目录不等于产品名:这里保持真实路径,除非用 MODELROUTER_REPO_DIR 覆盖。
|
||||
readonly REPO_DIR="${MODELROUTER_REPO_DIR:-/home/program/llmsproxy}"
|
||||
readonly HEALTH_URL="http://127.0.0.1:8081/v1/models"
|
||||
readonly BIN_NAME="llmsproxy"
|
||||
readonly BIN_NAME="modelrouter"
|
||||
|
||||
# 构建目录。默认落在仓库下的构建工作区,而不是 /tmp:本机 /tmp 是 8.8G
|
||||
# tmpfs,实测被 Go 构建缓存占满后会让 go test/build 在链接阶段直接失败。
|
||||
# 可用 LLMSPROXY_BUILD_DIR 覆盖;同一目录也被测试用作 GOTMPDIR。
|
||||
# 可用 MODELROUTER_BUILD_DIR 覆盖;同一目录也被测试用作 GOTMPDIR。
|
||||
readonly BUILD_SUBDIR=".build-work"
|
||||
if [[ -n "${LLMSPROXY_BUILD_DIR:-}" ]]; then
|
||||
readonly TMP_DIR="${LLMSPROXY_BUILD_DIR}"
|
||||
if [[ -n "${MODELROUTER_BUILD_DIR:-}" ]]; then
|
||||
readonly TMP_DIR="${MODELROUTER_BUILD_DIR}"
|
||||
else
|
||||
readonly TMP_DIR="${REPO_DIR}/${BUILD_SUBDIR}"
|
||||
fi
|
||||
@ -59,7 +60,7 @@ usage() {
|
||||
cat <<'EOF'
|
||||
用法: ./deploy.sh [--config <file>]
|
||||
|
||||
--config <file> 投放这份配置到 /etc/llmsproxy/config.yaml(原子替换)。
|
||||
--config <file> 投放这份配置到 /etc/modelrouter/config.yaml(原子替换)。
|
||||
省略时不改动线上配置,只部署二进制与适配器。
|
||||
无论是否指定,重启前都会用新二进制 -check 预检线上配置。
|
||||
|
||||
@ -103,6 +104,34 @@ precheck() {
|
||||
command -v systemctl &>/dev/null || fail "未找到 systemctl"
|
||||
[[ $(id -u) -eq 0 ]] || fail "需要 root 权限"
|
||||
|
||||
# 改名保护:本脚本现在部署到 /etc/modelrouter。若机器上仍是改名前的
|
||||
# /etc/llmsproxy 且新路径尚未建立,直接继续会创建一份**空配置**并让新
|
||||
# 服务在没有 key/源的情况下起来 —— 看起来成功,实际网关拒掉所有请求。
|
||||
if [[ -d /etc/llmsproxy && ! -e "$TARGET_CONFIG" ]]; then
|
||||
fail "检测到改名前的部署 /etc/llmsproxy,但 $TARGET_CONFIG 不存在。
|
||||
先迁移(dry-run 默认,确认后加 --apply):
|
||||
sudo ./packaging/migrate-to-modelrouter.sh
|
||||
sudo ./packaging/migrate-to-modelrouter.sh --apply
|
||||
如需全新部署,先把旧目录移开:mv /etc/llmsproxy /etc/llmsproxy.bak"
|
||||
fi
|
||||
|
||||
# unit 与二进制路径必须一致。包安装把二进制放 /usr/bin,源码部署放
|
||||
# /usr/local/bin;unit 里写错一行,systemctl restart 只会给 203/EXEC,
|
||||
# 看起来像"部署成功但服务起不来"。此处提前拦住。
|
||||
local unit="/etc/systemd/system/modelrouter.service"
|
||||
if [[ -f "$unit" ]]; then
|
||||
local unit_exe
|
||||
unit_exe=$(awk -F'=' '/^ExecStart=/{print $2; exit}' "$unit" | awk '{print $1}')
|
||||
if [[ -n "$unit_exe" && "$unit_exe" != "$TARGET_BIN" && ! -x "$unit_exe" ]]; then
|
||||
fail "unit 的 ExecStart 指向 $unit_exe,但该文件不存在。
|
||||
包安装(deb/rpm)用 /usr/bin/modelrouter;源码部署(本脚本)用 $TARGET_BIN。
|
||||
请把 $unit 的 ExecStart 改成 $TARGET_BIN,或改用包安装。"
|
||||
fi
|
||||
if [[ -n "$unit_exe" && "$unit_exe" != "$TARGET_BIN" ]]; then
|
||||
warn "unit ExecStart=$unit_exe,而本脚本部署到 $TARGET_BIN(两者都存在,确认哪个该生效)"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ -n "$NEW_CONFIG_SRC" ]]; then
|
||||
[[ -f "$NEW_CONFIG_SRC" ]] || fail "待部署配置不存在: $NEW_CONFIG_SRC"
|
||||
# 同一文件比对:--config 指向线上文件本身时没有"投放"动作可做,
|
||||
@ -153,7 +182,7 @@ build() {
|
||||
-trimpath \
|
||||
-ldflags="-s -w" \
|
||||
-o "$TMP_DIR/bin/$BIN_NAME" \
|
||||
./cmd/llmsproxy
|
||||
./cmd/modelrouter
|
||||
|
||||
NEW_BIN="$TMP_DIR/bin/$BIN_NAME"
|
||||
NEW_SIZE=$(stat -c%s "$NEW_BIN")
|
||||
@ -254,7 +283,7 @@ verify_config() {
|
||||
|
||||
# ---------- 密钥可解封校验 ----------
|
||||
# config.yaml 里的凭据以 enc:v1: 密文落盘,解密钥匙是 master.key(或
|
||||
# LLMS_PROXY_MASTER_KEY)。一旦钥匙丢失/错位,配置在**启动时**才会报错,
|
||||
# MODELROUTER_MASTER_KEY)。一旦钥匙丢失/错位,配置在**启动时**才会报错,
|
||||
# 那时服务已经处于「二进制已替换 + 配置已替换」的半死状态,只能靠回滚。
|
||||
# 所以在替换任何东西之前先验:能不能用现有的钥匙解开现有的密文。
|
||||
#
|
||||
@ -279,21 +308,22 @@ verify_master_key() {
|
||||
[[ -n "$runtime_file" ]] || runtime_file="$(dirname "$TARGET_CONFIG")/runtime.json"
|
||||
key_file="$(dirname "$runtime_file")/master.key"
|
||||
|
||||
if [[ -n "${LLMS_PROXY_MASTER_KEY:-}" ]]; then
|
||||
log " 钥匙来源: LLMS_PROXY_MASTER_KEY(环境变量)"
|
||||
MASTER_KEY_ENV="${MODELROUTER_MASTER_KEY:-${LLMS_PROXY_MASTER_KEY:-}}"
|
||||
if [[ -n "$MASTER_KEY_ENV" ]]; then
|
||||
log " 钥匙来源: MODELROUTER_MASTER_KEY(环境变量)"
|
||||
# 走真实解密路径验证,而不是只检查格式
|
||||
if out=$("$TARGET_BIN" -show-secrets -config "$TARGET_CONFIG" 2>&1); then
|
||||
log "OK 凭据可解封($(echo "$out" | grep -cE '^(source|key) ') 项)"
|
||||
return 0
|
||||
fi
|
||||
err "凭据无法用 LLMS_PROXY_MASTER_KEY 解封,未替换任何文件:"
|
||||
err "凭据无法用 MODELROUTER_MASTER_KEY 解封,未替换任何文件:"
|
||||
printf '%s\n' "$out" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
if [[ ! -f "$key_file" ]]; then
|
||||
err "配置含 $sealed 处密文,但钥匙文件不存在: $key_file"
|
||||
err " 没有它,部署后服务将无法启动。恢复 master.key 或设置 LLMS_PROXY_MASTER_KEY 后重试。"
|
||||
err " 没有它,部署后服务将无法启动。恢复 master.key 或设置 MODELROUTER_MASTER_KEY 后重试。"
|
||||
return 1
|
||||
fi
|
||||
log " 钥匙文件: $key_file ($(stat -c '%A' "$key_file" 2>/dev/null || echo '?'),$(wc -c < "$key_file") 字节)"
|
||||
@ -312,7 +342,7 @@ verify_master_key() {
|
||||
# ---------- 备份目录保留策略 ----------
|
||||
# 每次部署都新建一个 <prefix>.bak.<时间戳> 目录,而回滚只用到最近一次
|
||||
# ($BACKUP_BIN / $BACKUP_CONFIG 都是单文件覆盖)。多出来的目录从没人清理,
|
||||
# 实测线上已积累 71 个,/etc/llmsproxy 因此涨到 122M。
|
||||
# 实测线上已积累 71 个,/etc/modelrouter 因此涨到 122M。
|
||||
#
|
||||
# 保留最近 KEEP_BACKUPS 份足够回滚,同时给目录数设上限——否则一次误配置
|
||||
# 就可能在几秒内造出成百上千个目录。
|
||||
@ -327,7 +357,7 @@ KEEP_BACKUPS=5
|
||||
|
||||
prune_backups() {
|
||||
local prefix="$1"
|
||||
local base="/etc/llmsproxy"
|
||||
local base="/etc/modelrouter"
|
||||
[[ -d "$base" ]] || return 0
|
||||
# 防误用:前缀必须是已知的两个(避免有人传空串删掉整个目录)
|
||||
case "$prefix" in
|
||||
@ -393,7 +423,7 @@ sync_adapters() {
|
||||
[[ -d "$SRC_ADAPTERS" ]] || fail "内嵌适配器目录不存在: $SRC_ADAPTERS"
|
||||
|
||||
# 备份旧的适配器文件(含任何运行时覆盖版本,如 opencode.lua)
|
||||
BACKUP_DIR="/etc/llmsproxy/adapters.bak.$(date +%Y%m%d%H%M%S)"
|
||||
BACKUP_DIR="/etc/modelrouter/adapters.bak.$(date +%Y%m%d%H%M%S)"
|
||||
if [[ -d "$TARGET_ADAPTERS" ]]; then
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
cp -rf "$TARGET_ADAPTERS/"*.lua "$BACKUP_DIR/" 2>/dev/null || true
|
||||
@ -425,7 +455,7 @@ sync_adapters() {
|
||||
#
|
||||
# 早期实现是「只补缺,已存在一律不动」,看着安全,实际造成一个功能性缺口:
|
||||
# 内置插件本身的**修复无法下发**。v2.0.3/v2.0.4/v2.0.5 三个渠道修复就这样
|
||||
# 全部卡在仓库里 —— 二进制换了、/etc/llmsproxy/plugins/channels.lua 还是旧版,
|
||||
# 全部卡在仓库里 —— 二进制换了、/etc/modelrouter/plugins/channels.lua 还是旧版,
|
||||
# 生产行为毫无变化。内置源码的正确性由仓库保证,部署必须与二进制内容对齐。
|
||||
#
|
||||
# 「只补缺」原本想保护的是「运维改过 / 运行时新增的插件」,但那些都是**非内置
|
||||
@ -439,7 +469,7 @@ sync_plugins() {
|
||||
mkdir -p "$TARGET_PLUGINS"
|
||||
|
||||
# 备份将被覆盖/新增的内置插件(非内置名字不进入循环,天然不动)
|
||||
BACKUP_DIR="/etc/llmsproxy/plugins.bak.$(date +%Y%m%d%H%M%S)"
|
||||
BACKUP_DIR="/etc/modelrouter/plugins.bak.$(date +%Y%m%d%H%M%S)"
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
local added=0 updated=0 same=0
|
||||
for src in "$SRC_PLUGINS"/*.lua; do
|
||||
@ -481,8 +511,8 @@ sync_plugins() {
|
||||
# ---------- 重启服务 ----------
|
||||
restart_service() {
|
||||
log "重启服务(中断 <500ms)"
|
||||
systemctl restart llmsproxy.service || fail "systemd restart 失败"
|
||||
log "OK llmsproxy.service restarted"
|
||||
systemctl restart modelrouter.service || fail "systemd restart 失败"
|
||||
log "OK modelrouter.service restarted"
|
||||
}
|
||||
|
||||
# ---------- 健康检查 ----------
|
||||
@ -505,7 +535,7 @@ healthcheck() {
|
||||
(( attempt++ ))
|
||||
done
|
||||
|
||||
warn "健康检查未通过,请检查服务状态: systemctl status llmsproxy.service"
|
||||
warn "健康检查未通过,请检查服务状态: systemctl status modelrouter.service"
|
||||
return 1
|
||||
}
|
||||
|
||||
@ -541,7 +571,7 @@ rollback() {
|
||||
|
||||
if (( restored_any == 0 )); then
|
||||
err "无任何回滚点,无法自动恢复"
|
||||
err "请手动检查: systemctl status llmsproxy.service && journalctl -u llmsproxy -n 50"
|
||||
err "请手动检查: systemctl status modelrouter.service && journalctl -u modelrouter -n 50"
|
||||
return 1
|
||||
fi
|
||||
|
||||
@ -555,9 +585,9 @@ rollback() {
|
||||
fi
|
||||
|
||||
log "重启服务(旧版本)"
|
||||
systemctl restart llmsproxy.service || {
|
||||
systemctl restart modelrouter.service || {
|
||||
err "重启失败,服务可能完全宕机"
|
||||
err "请手动检查: systemctl status llmsproxy.service"
|
||||
err "请手动检查: systemctl status modelrouter.service"
|
||||
return 1
|
||||
}
|
||||
|
||||
@ -592,7 +622,7 @@ print_summary() {
|
||||
# 于是坏配置在服务仍然健康时就被拦下,根本不会触发重启。
|
||||
main() {
|
||||
parse_args "$@"
|
||||
log "═══ llmsproxy 原子部署开始 ═══"
|
||||
log "═══ modelrouter 原子部署开始 ═══"
|
||||
precheck
|
||||
# 密钥校验放在 build/替换之前:它的失败后果最重(配置里的密文永久解不开),
|
||||
# 必须在动任何文件之前就挡住。此阶段用的是**当前线上二进制**,它已经
|
||||
|
||||
60
docs/adapters.md
Normal file
60
docs/adapters.md
Normal file
@ -0,0 +1,60 @@
|
||||
# 适配器协议
|
||||
|
||||
> 本文档由 README 拆分而来:Lua 适配器协议与内置适配器清单。
|
||||
> 编写适配器的完整参考见 [lua-adapters.md](lua-adapters.md)。
|
||||
> 返回 [README](../README.md)。
|
||||
|
||||
## Lua 适配器协议
|
||||
|
||||
完整 API 见 **[Lua 适配器 API 文档](docs/lua-adapters.md)**([English](docs/lua-adapters-en.md))。
|
||||
|
||||
每个适配器是一个返回 table 的 Lua 脚本(`internal/lua/adapters/<name>.lua`),
|
||||
加载时可被 `adapter_dir` 下的同名脚本覆盖——**改 Lua 脚本无需重编译**,
|
||||
重启即生效;通过 WebUI 上传的适配器与在线编辑的源配置则即时生效。
|
||||
|
||||
```lua
|
||||
return {
|
||||
name = "mysrc",
|
||||
version = "1.0.0",
|
||||
endpoint = "/chat/completions", -- 上游路径(可被 source.endpoint 覆盖)
|
||||
headers = { ["X-Static"] = "v" }, -- 静态默认请求头(build_headers 缺省时使用)
|
||||
|
||||
-- 请求转换:把 OpenAI 格式 req 转成上游原生格式,返回字符串
|
||||
transform_request = function(raw_json) ... end,
|
||||
|
||||
-- 响应转换:把上游原生响应转成统一格式字符串
|
||||
-- { content, reasoning_content, finish_reason, token_usage{...}, tool_calls[{...}] }
|
||||
transform_response = function(raw_json) ... end,
|
||||
|
||||
-- 流式分块转换:把上游 SSE data 转成 { content, done, ... },返回 "" 则跳过
|
||||
transform_stream_chunk = function(raw_chunk) ... end,
|
||||
|
||||
-- [可选] 动态请求头/签名钩子
|
||||
-- meta = { url, method, body, api_key, timestamp, source={ name, meta={...} } }
|
||||
build_headers = function(meta) return { ["X-App-Sign"] = sign } end,
|
||||
|
||||
-- [可选] 出向序列化(契约 v2):上游不是 OpenAI 协议时接管写给客户端的字节。
|
||||
-- 不声明则沿用内核的 OpenAI 兜底;声明即完全取代,内核不再混合字段。
|
||||
serialize_response = function(result) return body_str end,
|
||||
serialize_open_preamble = function(result) return bytes end,
|
||||
serialize_chunk = function(delta) return bytes end,
|
||||
serialize_close = function(result) return bytes end,
|
||||
}
|
||||
```
|
||||
|
||||
内置信号辅助:`hmac_sha256_hex(key, data)`、`sha256_hex(data)`、`base64_encode(s)`、
|
||||
`tohex(s)`、`json.encode/decode`、`log(level, msg)`。
|
||||
|
||||
### 内置适配器
|
||||
|
||||
`openai` `deepseek` `anthropic` `gemini` `github` `groq` `mistral` `ollama` `kimicode`
|
||||
`qwen38` `sensenova` `trae`。
|
||||
|
||||
`anthropic`/`gemini`/`ollama` 适配器内置多模态转换(`image_url` → 各自上游格式);
|
||||
推理开关由中立的 `thinking` 字段表达,`deepseek` 适配器把它映射为
|
||||
`thinking.type = "disabled"`。
|
||||
|
||||
**kimicode** 是展示 `build_headers` 的样例:云端校验调用方 app,需要按
|
||||
`meta.app_secret` 对时间戳+URL+请求体哈希做 HMAC 签名并附 `X-App-Sign` 等头。
|
||||
配好 `sources[].meta.{app_id, app_secret, app_agent}` 即可。
|
||||
|
||||
125
docs/channel-issues-2026-10-19.md
Normal file
125
docs/channel-issues-2026-10-19.md
Normal file
@ -0,0 +1,125 @@
|
||||
# 渠道问题清单(2026-10-19 晚生产实测)
|
||||
|
||||
> 逐项修复。每项标注:现象 / 根因 / 状态。
|
||||
|
||||
## P1 trae 非流式 500(unmarshal unified response: invalid character 'e')
|
||||
- **现象**:`stream:false` 打 `channels-trae:*` 必现 500,body 以 `event:metadata data:{...}` 开头。
|
||||
- **根因**:trae SOLO 上游**对非流式请求也回 SSE**(且帧带 `event:` 前缀)。
|
||||
插件自装的 `channels-trae` 适配器 `transform_response` 直接 `__prelude.openai_response(raw_body)`,
|
||||
不识别 SSE;而内核只把 `data:` 行交给流式适配器,非流式路径把**整个 body**(含 `event:` 行)
|
||||
塞给 transform_response → json.decode 失败 → 原样返回 → Go 侧 unmarshal 报 `invalid character 'e'`。
|
||||
(`internal/lua/adapters/trae.lua` 有 SSE 聚合逻辑,但 channels 平台用的是**插件内嵌**
|
||||
`TRAE_SOLO_ADAPTER_SRC`(channels.lua:9744),两份代码不同步。)
|
||||
- **修法**:`TRAE_SOLO_ADAPTER_SRC.transform_response` 加 SSE 聚合(复用流式帧解析,
|
||||
聚合成一个非流式响应),并把 `internal/lua/adapters/trae.lua` 的聚合逻辑对齐。
|
||||
- **状态**:✅ 已修复(`aggregate_solo_sse`)并部署生产,实测非流式返回 200 正常内容。
|
||||
|
||||
## P2 buddy(CodeBuddy)非流式 400(11101 Non-stream not supported)
|
||||
- **现象**:`stream:false` 打 `channels-buddy:*` 上游 400 `11101: Non-stream chat request is currently not supported`。
|
||||
- **根因**:腾讯 codebuddy 网关**只支持流式**。适配器 `transform_request` 没有强制 `stream=true`,
|
||||
非流式请求原样透传被上游拒绝。buddy 的 `Accept: text/event-stream` 头已发,但 body.stream 仍是 false。
|
||||
- **修法**:`buddy.lua transform_request` 强制 `req.stream = true`;`transform_response` 增加
|
||||
SSE 聚合兜底(内核非流式路径收到 SSE 时聚合)。参考 opencodezen.lua 的既有做法(门禁第 4 条)。
|
||||
- **状态**:✅ 已修复并部署生产,实测非流式返回 200 正常内容。
|
||||
|
||||
## P3 buddy 偶发 provider busy(429)+ 11133 参数拒绝
|
||||
- **现象**:连续请求 `channels-buddy` 间歇返回 `provider busy`(source 并发槽 max_concurrent=1
|
||||
被占满)与上游 `11133: the request parameters were rejected`。
|
||||
- **根因**:① max_concurrent=1 是**刻意的**(channels.lua:4332 注释:并发配额按模型计),busy
|
||||
是正常自保;② 11133 与空 tool_calls 修复(未提交的 buddy.lua 改动)相关,待部署后观察。
|
||||
- **状态**:观察项,非缺陷。
|
||||
|
||||
## P4 Qoder 领取失败(缺少凭据对象)——已修复
|
||||
- **现象**:`channels/claim` 报 `{"error":"缺少凭据对象"}`,而 account_test 正常。
|
||||
- **根因**:qoder 是 PKCE 设备码流且无 register 第二跳,落库凭据是**裸 token 字符串**;
|
||||
`qoderClaim` 硬要求 `type(cred)=="table"`。
|
||||
- **修法**:`qoderClaim` 兼容裸 token(非空时包 `{access_token=tok}`);
|
||||
回归测试 `TestQoderClaimAcceptsBareTokenCredential`。
|
||||
- **状态**:✅ 已修复并部署生产(claim 现在返回真实上游业务结论)。
|
||||
|
||||
## P5 Qoder/OpenCode 渠道页账号串台(UI 渲染竞态)
|
||||
- **现象**:从 qoder 切到 opencode 详情页,qoder 的账号行闪现在 opencode 卡片下。
|
||||
- **根因**:`renderDetail()` 切渠道时不清空 `#ch-accts`,`renderAccounts` 是异步的,
|
||||
空窗期 DOM 仍挂旧渠道账号行;且慢的旧请求返回会覆盖新渠道数据。
|
||||
- **修法**:① renderDetail 立即清空账号/模型区并置"加载中";② renderAccounts/renderModels
|
||||
加代际校验(`state.selected !== id` 时丢弃响应)。
|
||||
- **状态**:✅ 已修复并部署生产。
|
||||
|
||||
## P6 CodeArts「完全不可用」——多数模型实际可用
|
||||
- **现象**:用户报"codearts 完全不可用"。实测逐模型探测:
|
||||
- 可用:openpangu-2.0-flash/pro、GLM-5.2、glm-5.3、kimi-k2.6-vl、Qwen3-VL-235B、glm-5.2-sft-harmony
|
||||
- 429(限流非下架):deepseek-v4.1-flash、deepseek-v4-flash、openpangu-2.0-pro(间歇)
|
||||
- 404(真下架,需标 disabled):GLM-5.3 报 `InferHub.002002009.404: The model is not registered`
|
||||
- **附带发现**:`GLM-5.3`(大写变体)不在 deadModels 记录里——`recordDeadModel` 只被
|
||||
`__record_dead_probe` 调用,**request_end hook 没接 isModelGone**(583976f 引入的
|
||||
基础设施丢了挂接点)。需要把 on_request_end 补上 dead-model 记录。
|
||||
- **凭据**:expiresAt 已过期(15:05),`account/refresh` 续期成功(refreshable=true)。
|
||||
建议把「凭据到期自动续期」纳入定时任务检查。
|
||||
- **修法(2026-10-20 完成)**:
|
||||
- `on_request_end` 补上 `isModelGone` + `recordDeadModel` 挂接(P9 一并解决);
|
||||
- **凭据自动续期失效的真根因**:`credentialExpiresAtMs` 把 ISO 过期时刻按**本地时区**
|
||||
解析,而 codearts 给的是 UTC ISO(`2026-10-19T16:42:43.06Z`)⇒ 本机 +08 下过期时刻
|
||||
虚早 8 小时 ⇒ `refresh_creds` 每 30 分钟都判"已过期"并对华为云真打一次 STS
|
||||
(token 本来还有 23 小时寿命)。已改为按声明时区解析(Z/±hh:mm/无后缀=UTC 三种形态
|
||||
同一 epoch),回归测试 `TestCredentialExpiresAtMsParsesISOAsUTC`。
|
||||
- **状态**:✅ 已修复并部署生产。
|
||||
|
||||
## P7 CodeBuddy 海外账号 / Qoder 国内账号
|
||||
- **现象**:CodeBuddy 登录只走国内(copilot.tencent.com),Qoder 登录只走国际
|
||||
(openapi.qoder.sh / qoder.com)。用户要求两边都能选区。
|
||||
- **根因**:平台表单区硬编码。buddy 的 state-poll 端点与 Qoder 的 PKCE 端点都锁死一个区域。
|
||||
- **修法(2026-10-20 已实施,账号级 region)**:
|
||||
- 凭据 JSON 增加 `region` 字段(`intl`|`cn`,缺省 = 历史行为);
|
||||
- `start_login` 接受 `region`,会话带着它选端点;登录完成写进凭据;
|
||||
- **CodeBuddy**:cn = `copilot.tencent.com`(platform=ide),intl = `www.workbuddy.ai`
|
||||
(platform=`workbuddy-ai`,X-Domain/X-Product-Code/UA 按区域切换;实测
|
||||
`POST /v2/plugin/auth/state?platform=workbuddy-ai` 回 200 信封);
|
||||
- **Qoder**:intl = `openapi.qoder.sh` + `qoder.com`(client_id e883ade2-…),
|
||||
cn = `openapi.qoder.com.cn` + `qoder.cn`(client_id **不同**:732aef47-…,
|
||||
照参考 QODER_CN;用国际版 id 授权页报「参数无效」)。CN 无公开 chat 端点
|
||||
(参考实测 503)⇒ CN 账号只做登录/领取,不产源;
|
||||
- UI:buddy/qoder 详情页多一个「+国际区」按钮(默认国内/国际主区)。
|
||||
- **状态**:✅ 已实施并部署生产。
|
||||
|
||||
## P8 商汤(Raccoon)扫码登录「链接无效」——已修复
|
||||
- **现象**:用户点登录看到的链接打不开(404)。
|
||||
- **根因(两个,2026-10-20 实测定位)**:
|
||||
① 登录页 `/login/mp` **已被站点删除**(Next 路由 404,只有 404 页面返回);
|
||||
② 旧实现**本地随机造 qrcode_code**,而上游早已不认自造码:旧端点
|
||||
`/api/web/auth/v1/login_with_qrcode_code` 对任意 code 恒回 `pending`
|
||||
(服务端根本没有该会话)⇒ 扫码永远完不成,账号一直空凭据。
|
||||
- **修法**:改为上游现行 IAM 流程(逐行对齐其网页客户端 auth chunk):
|
||||
① `POST /api/iam/v1/auth/mp/qrcode` 由**服务端建会话**(返回 qrcode_code /
|
||||
scan_path / expires_in / poll_interval);
|
||||
② 链接/二维码内容 = `https://xiaohuanxiong.com/login?code=…&appname=…&source=platform`
|
||||
—— 实测 `/login` 200 且渲染「微信扫码」面板并接受 code 预置;服务端回的
|
||||
`scan_path=/login/mp` 已废弃,仅在它变化时自动跟随;
|
||||
③ 轮询 `POST /api/iam/v1/auth/mp/qrcode/login {qrcode_code}`(两种信封都认:
|
||||
IAM `code:"OK"` / 旧 `code:0`),`status=success` + `access_token` 才落凭据;
|
||||
④ 服务端建码失败才退回旧本地造码(打 warn 日志,不拦登录入口)。
|
||||
- **状态**:✅ 已修复并部署生产。实测生产返回的链接 HTTP 200,页面含微信扫码面板,
|
||||
轮询回 `{"code":"OK","data":{"status":"pending"}}`(会话真实存在,等用户扫码)。
|
||||
|
||||
## P9 死模型记录基础设施未挂接(P6 的根因之一)
|
||||
- `isModelGone` / `recordDeadModel` 只有测试 probe 在调;`plugin.hooks = { request_end }` 的
|
||||
`on_request_end` 只处理 term_error(账号级),不记模型级死亡。
|
||||
- **修法**:on_request_end 失败分支里,取 payload.model + error message,
|
||||
`isModelGone(msg)` 为真时 `recordDeadModel(provider, model, msg)` 并重同步源。
|
||||
- **状态**:✅ 已修复并部署生产(真实 404 现在会进 deadModels)。
|
||||
|
||||
## 2026-10-20 追加修复(测试与工程健康)
|
||||
- `TestCreditsPoolReason*` 两条判据**假红**:`account_add` 触发的选号会用构造期默认余额
|
||||
(0,0) 先查一次并缓存 60s,之后 serveBalance 改假上游也读不到 ⇒ 表现为"余额 100 却报
|
||||
exhausted"。修法:harness 在改余额后清 TTL 缓存(`__ttl_reset_probe`,与生产同一实现)。
|
||||
- `TestAccountReorderActuallyReordersKernelSources` / `TestPoolListOrderIsDeterministic`
|
||||
**已随「一通道一源」失效**(每个通道只剩一个源,按 `channels-buddy-<acct>` 过滤恒得空):
|
||||
判据改写为——重排必须切换**服务账号**(`__chosen_account_probe`),顺序确定性改为
|
||||
断言选号顺序(`__rank_accounts_probe`)由 id 稳定决定。
|
||||
- channels.lua 顶层 local 余量回归绿:区域表/适配器源码常量收进 `plugin.K`
|
||||
(`TestChannelsLuaHasLocalHeadroom` 需要 ≥10 槽位)。
|
||||
|
||||
## 部署注意事项
|
||||
- 工作树有未提交改动:buddy.lua / trae.lua(tool_calls 空 {} 过滤)、provider.go(stop→tool_calls 归一)、
|
||||
channels.lua(qoder 裸 token + UI 竞态)、provider_test.go、channels_qoder_claim_test.go。
|
||||
- 生产 channels.lua 哈希与仓库不同步是常态(deploy.sh 会 sync_plugins)。
|
||||
- 商汤账号:用户需在二维码有效期内扫码确认;当前账号 5a2ea34df521 无凭据属正常(未完成扫码)。
|
||||
183
docs/channel-repair-plan.md
Normal file
183
docs/channel-repair-plan.md
Normal file
@ -0,0 +1,183 @@
|
||||
# 渠道链路修复计划与现状
|
||||
|
||||
> 本文是 `plan.md` 中「渠道链路修复计划(2026-10-17 起)」章节的**可提交版本**。
|
||||
> `plan.md` 本身被 `.gitignore` 忽略(本地工作台账),故关键计划与现状单独落在这里,
|
||||
> 以便随代码一起进 git、供他人查阅。
|
||||
>
|
||||
> **写作原则:只写实测到的,标注"推断"的必须写明。** 每条缺陷都附可复现的观测。
|
||||
|
||||
> 记录 **2026-10-17 用户报障三连修**(模型不可用 / 额度异常 / 其他渠道同类)
|
||||
> 的现状与后续计划。上接 `plan.md` §9(v2 契约收尾)。
|
||||
|
||||
## 1. 现状快照(2026-10-17 14:00 实测,非推断)
|
||||
|
||||
### 1.1 生产实例
|
||||
|
||||
| 项 | 值 |
|
||||
|---|---|
|
||||
| 服务 | `active`,NRestarts=0 |
|
||||
| 二进制 sha256 | `56396f79bfc2433a`(与仓库逐字节一致) |
|
||||
| 插件 sha256 | `c15cb4bf5d5c0f21`(与仓库逐字节一致) |
|
||||
| 插件 | 4 个(billing/channels/quota/usage)**全部 loaded** |
|
||||
| 源 / 适配器 | 21 源 / 23 适配器 |
|
||||
| 账号池 | **仅 trae、buddy 各有 1 个有凭据账号**;其余 10 个平台**账号数 0** |
|
||||
|
||||
### 1.2 各渠道缺陷矩阵(本文件写入时的实测)
|
||||
|
||||
自装适配器(插件在加载时经 `__llmsproxy_adapters` 装入):
|
||||
|
||||
| 适配器 | `build_headers` | 静态 `adapter.headers` | 流式转换 | 鉴权是否可能错 |
|
||||
|---|---|---|---|---|
|
||||
| `codearts` | 有 | 无 | 复合 openai | 有(SDK-HMAC 自签,已实现) |
|
||||
| `channels-trae` | **有(本次新加)** | 无 | **SOLO 自定义(本次新写)** | 已修 |
|
||||
| `channels-gemini` | **★ 缺** | 有 | 自定义 | 静态头 + Bearer,**待确认** |
|
||||
| `channels-zcode` | **★ 缺** | 无 | 无(继承 anthropic 基座) | **★ 高度可疑** |
|
||||
| `channels-cline` | **★ 缺** | 无 | 套 openai | **★ 可疑**(要 `Bearer workos:<jwt>`) |
|
||||
| `channels-qoder` | **★ 缺** | 无 | 套 openai | **★ 可疑**(头由 WASM 生成) |
|
||||
|
||||
内置适配器里 `build_headers` 只有 3 个有(buddy / kimicode / opencodezen),
|
||||
其余 11 个(含 **anthropic 基座**)靠静态头 + 内核兜底 `Bearer`。
|
||||
|
||||
> **★ anthropic 基座本身没有 `build_headers`**,而 anthropic 协议实际要
|
||||
> `x-api-key` 而非 `Authorization: Bearer`。`channels-zcode` 继承它 ⇒
|
||||
> 一旦有账号就会立刻暴露,与 trae 修复前完全同型。
|
||||
|
||||
### 1.3 已验证可用的通道(生产实测,不是单测)
|
||||
|
||||
| 模型(源限定名) | 结果 |
|
||||
|---|---|
|
||||
| `channels-trae-b8dd6b3df30e:Doubao-Seed-2.1-Pro` | ✅ 80 帧,正文 27 + 推理 299 |
|
||||
| `channels-trae-b8dd6b3df30e:glm-5.2` | ✅ OK |
|
||||
| `channels-buddy-buddy-main:hy4-preview` | ✅ OK |
|
||||
|
||||
| 端点 | 结果 |
|
||||
|---|---|
|
||||
| `channels/credits/status` (trae) | ✅ `active:true, dailyCredit:100, todayCheckedIn:true` |
|
||||
| `channels/credits/balances` (trae) | ✅ `total:598.516 credits` |
|
||||
|
||||
### 1.4 本轮已修(代码已部署生产,**尚未提交 git**)
|
||||
|
||||
**① 网关解不开 `源名:模型名` 限定名**(`internal/provider/registry.go`)
|
||||
|
||||
- 根因:`ResolvePinned` 用 `strings.IndexAny(model, "-:/")` 找**第一个**分隔符。
|
||||
渠道源名形如 `channels-trae-b8dd6b3df30e`(**含 `-`**)⇒ 切成 `src="channels"`
|
||||
⇒ 永不匹配 ⇒ `Resolve()` 返回 nil ⇒ `model "..." is not configured`。
|
||||
- 为什么必须支持:本实例 **44 个模型被多源同时提供**(`glm-5.2` 由 6 个源提供),
|
||||
WebUI 下拉框用 `源名:模型名` 作 option value 是**有意设计**。
|
||||
- 修法:抽 `splitPinned()`,**按已知源名做最长前缀匹配**(源名是已知集合,
|
||||
唯一可靠办法),保留旧 `-:/` 兜底。`EffectiveModel` 同步修。
|
||||
- 判据:`internal/provider/registry_pinned_name_test.go`(4 用例,变异 1/1 咬住)。
|
||||
|
||||
**② trae 适配器三连错**(`internal/lua/plugins/channels.lua`,逐层剥开)
|
||||
|
||||
1. **没有 `build_headers`** ⇒ 内核兜底发 `Bearer`,而 trae 推理端点要
|
||||
`Cloud-IDE-JWT` + 设备身份头 ⇒ 401「not able to authenticate you」。
|
||||
→ 与「额度不可用」是**同一根因的两个出口**(额度端点当轮修了、推理端点漏了)。
|
||||
2. **header 常量值是错的**:`X-App-Id` 抄成 `6eefa6c9`(真实
|
||||
`6eefa01c-1036-4c7e-9ca5-d891f63bfcd8`)、`app_version_code` 用 `0.1.52`
|
||||
(真实是**日期式** `20260811`)⇒ 上游 `4001 binding: expr_path=app_version_code,
|
||||
cause=parameter type does not match binding data`。
|
||||
3. **流式转换完全不对**:TRAE 是 **SOLO 自定义 SSE**
|
||||
(`event:output` → `data:{"response":"###","reasoning_content":...}`)。
|
||||
套 `openai_stream_chunk` ⇒ 无 `choices` ⇒ `empty stream`;
|
||||
改成 OpenAI 外壳 `{choices:[{delta}]}` **仍不对** —— 内核
|
||||
`json.Unmarshal(unified, &UnifiedChunk)` 解的是**扁平结构**,外壳字段全丢。
|
||||
⇒ 必须输出扁平 `{content, reasoning_content, usage, done, finish_reason}`。
|
||||
- 判据:`channels_trae_chat_headers_test.go`(3)、`channels_trae_solo_sse_test.go`(2),
|
||||
变异 2/2 咬住。
|
||||
|
||||
**③ 余额恒为 0**(同文件 `balance.total_fn`)
|
||||
|
||||
- 根因:读 `p.credits_limit`(pack **顶层**)—— 该字段**根本不存在**。
|
||||
真实路径是 `pack.entitlement_base_info.quota.credits_limit`
|
||||
(参考 `trae-credits.ts:341`)+ `pack.usage.credits_amount`。
|
||||
- 实测上游:`免费`(无 credits_limit) / `每月登录赠送`(500) / `签到奖励`(100) = **600**。
|
||||
- 修法:读嵌套路径 + 保留顶层兼容 + `usage_summary.{total,consumed}_amount` 兜底。
|
||||
- 判据:`channels_trae_credits_hdr_test.go`(1)。
|
||||
- 修后余额 **600 → 实测 598.516**(真实扣减)。
|
||||
|
||||
**④ 附带**:`channels_trae_test.go` 夹具从**扁平** `{"Token":...}` 改为真实
|
||||
`{"Result":{"Token":...}}` —— 原来夹具形状与生产不一致,**判据必然虚假通过**
|
||||
(这是 ExchangeToken 缺陷藏了很久的原因)。
|
||||
|
||||
## 2. 后续计划
|
||||
|
||||
### P0 — 提交本轮修复(必须先做)
|
||||
|
||||
代码已部署生产但**未进 git**。必须:
|
||||
1. 在 `release/v2.0.x` 上提交(含 3 个源文件 + 6 个判据文件)
|
||||
2. 打 tag `v2.0.40`(**这是内核 Go 改动**:`registry.go` ⇒ 真核心版本)
|
||||
3. 回流 main(cherry-pick,不 merge)
|
||||
4. 复核 tag 落点 + release vs main 零差异 + 生产 sha256 一致
|
||||
|
||||
> 判据:`git tag --points-at <release HEAD>` 非空;`git diff release/v2.0.x main` 为空。
|
||||
|
||||
### P1 — 其余渠道的同类缺陷(**账号 0,验证不了**)
|
||||
|
||||
按"风险 × 可验证性"排序。**核心限制:这 4 个平台账号数均为 0,
|
||||
无法做生产实测** —— 只能靠静态对齐参考 + 装好适配器后看 `__adapter_install_probe`。
|
||||
|
||||
| 优先 | 项 | 缺陷 | 修法 | 可验证性 |
|
||||
|---|---|---|---|---|
|
||||
| P1-1 | `channels-zcode` | 走 anthropic 基座,而**基座无 `build_headers`**,anthropic 要 `x-api-key` 不是 Bearer | 给基座补 `build_headers`(`x-api-key` + `anthropic-version`),zcode 继承即得 | 静态可验;运行需账号 |
|
||||
| P1-2 | `channels-cline` | 参考要 `Bearer workos:<jwt>`,我们的 token 无 `workos:` 前缀 | 登录/续期处拼前缀,或适配器 `build_headers` 补 | 静态可验;运行需账号 |
|
||||
| P1-3 | `channels-gemini` | 有静态头 + Bearer(参考注释也说 Cloud Code 用 OAuth Bearer)⇒ **可能本来就对** | 只需确认,不改 | 静态可验 |
|
||||
| P1-4 | `channels-qoder` | 头由 **WASM 生成必须原样透传**,我们拿不到 `.wasm` | **阻塞**:二进制不在仓库任何位置 | 不可验证 |
|
||||
|
||||
### P2 — 内核对"限定名"的契约收口
|
||||
|
||||
`ResolvePinned` / `EffectiveModel` 现在各自调 `splitPinned`。建议:
|
||||
- 把"源名可含哪些字符"写进 `docs/`,并加判据:新增源名若含 `:` 或 `/` 必须报警
|
||||
(否则限定名会歧义)。
|
||||
- 现状:源名由插件生成 `channels-<platform>-<accountId>`,只含 `-`/字母数字,安全。
|
||||
|
||||
### P3 — 判据防回归(防"夹具与生产不一致"再犯)
|
||||
|
||||
本轮最大的教训不是某个 bug,而是**夹具形状错 ⇒ 判据虚假通过**(trae 活了两轮)。
|
||||
|
||||
建议加一条**通用**判据:`TestAdaptersFixturesMatchRealUpstreamShape` ——
|
||||
对每个有夹具的渠道,把夹具响应体与
|
||||
`/tmp/pi-github-repos/.../vendor/channel-pack/src/*.ts` 里注释记录的真实响应形状
|
||||
做**结构对照**(键集合),不一致即报红。做不到全自动,至少把已知的
|
||||
"Result 信封 / SOLO SSE / 嵌套 quota" 三类钉住。
|
||||
|
||||
### P4 — 待用户决策
|
||||
|
||||
| 项 | 需要什么 |
|
||||
|---|---|
|
||||
| zcode 真实登录 | 用户点授权(身份块 + init-poll 流已就绪) |
|
||||
| cline / qoder / gemini / codearts / minimax / lobsterai / opencode | 需各平台账号;`device-poll` 系列要用户去官方页输码 |
|
||||
| raccoon / loomy | 需扫码 |
|
||||
| qoder WASM | 用户提供 `.wasm` |
|
||||
|
||||
## 3. 风险与纪律(本轮新增)
|
||||
|
||||
1. **`luac -p` 对长字符串内的适配器代码无效**(只看外层)⇒ 会得到假绿。
|
||||
**必须**用 `__adapter_install_probe` 看真实装载结果。本轮我因此把 TRAE 的
|
||||
SSE 代码误塞进 CODEARTS 适配器,导致 codearts 装载失败而 `luac` 一直通过。
|
||||
2. **Python 字符串替换会静默不生效**:锚文本缩进不符 ⇒ 匹配 0 次。
|
||||
规矩:① 先 `cp` 备份 ② **行号定位** ③ 写后 `luac -p` + `grep -c` 双确认
|
||||
④ 长字符串内的改动必须验装载。
|
||||
3. **`function` 是 Lua 关键字**:`cc.function` 是语法错误,必须 `cc["function"]`;
|
||||
但批量替换时别把 `cc.function_call` 一起改坏(本轮踩过,导致
|
||||
`')' expected near '_call'`)。
|
||||
4. **判据要咬被测对象**:本轮 `channels_trae_credits_hdr_test.go` 第一版造 HTTP 假上游,
|
||||
因 URL 覆盖/凭据 ref/路径匹配任一环错,"没发请求"伪装成"头不对"(耗了三轮)。
|
||||
改成直接咬 `spec.headers_fn` 的真实产出,一轮就准。
|
||||
5. **只改插件不必换二进制**;改内核 Go 必须换。混着发会把"插件改动"记成"核心版本"
|
||||
(v2.0.21–v2.0.39 有 18 个 tag 是这么来的,见 §4)。
|
||||
6. **`/tmp` 是 8.8G tmpfs**,Go 链接失败常见于它满(`no space left on device`)。
|
||||
元凶通常是 `/tmp/gocache`(可安全清)。
|
||||
|
||||
## 4. 账务实况(必须记住)
|
||||
|
||||
| 事实 | 值 |
|
||||
|---|---|
|
||||
| 内核 Go 代码自 v2.0.20 起的**全部**改动 | 仅 `internal/lua/adapter_prelude.go`、`internal/provider/provider.go`、`internal/provider/registry.go` |
|
||||
| 只改插件却发了核心 tag 的版本 | v2.0.21–v2.0.39 中的 **18 个** |
|
||||
| 当前最新 tag | `v2.0.39`(`091f790`),落点正确 |
|
||||
| main | `56ea269`(与 release 内容零差异) |
|
||||
| **未提交的改动** | 本轮三个修复(见 §1.4),已部署生产 |
|
||||
|
||||
> ⇒ **版本号语义教训**:以后"改插件不打核心 tag",插件自身维护版本;
|
||||
> 内核版本只在 Go 代码变更时递增。
|
||||
@ -389,10 +389,10 @@ flow 类型,不是"登录成功"。
|
||||
channels 插件用 `storeDir()` 决定存储位置,它有多级兜底:
|
||||
|
||||
```
|
||||
env LLMSPROXY_PLUGIN_DIR → settings.store_dir → 读 /etc/llmsproxy/config.yaml 的 plugin_dir → 硬编码
|
||||
env MODELROUTER_PLUGIN_DIR → settings.store_dir → 读 /etc/llmsproxy/config.yaml 的 plugin_dir → 硬编码
|
||||
```
|
||||
|
||||
测试台没设 `LLMSPROXY_PLUGIN_DIR`,**后两级直接命中生产**:
|
||||
测试台没设 `MODELROUTER_PLUGIN_DIR`,**后两级直接命中生产**:
|
||||
|
||||
```
|
||||
测试插件目录 = /tmp/TestXXX/002/plugins
|
||||
@ -404,7 +404,7 @@ storeDir 结果 = /etc/llmsproxy/plugins ← 探针实测输出
|
||||
`.channels-store.json` 从 1730 覆盖成 1986 字节、账号池清空(inode 都换了)。
|
||||
|
||||
只有 `internal/lua` 复现,`internal/core` 不复现 —— 因为 core 的测试台
|
||||
(`kernelTestCore`)设了 `LLMSPROXY_PLUGIN_DIR`。**这是隔离做得不一致**的代价。
|
||||
(`kernelTestCore`)设了 `MODELROUTER_PLUGIN_DIR`。**这是隔离做得不一致**的代价。
|
||||
|
||||
同类前科:曾把 26 个测试账号写进生产 store。两次同一类错误。
|
||||
|
||||
|
||||
183
docs/configuration.md
Normal file
183
docs/configuration.md
Normal file
@ -0,0 +1,183 @@
|
||||
# 配置
|
||||
|
||||
> 本文档由 README 拆分而来:网关配置、模型自动探测、多密钥。
|
||||
> 返回 [README](../README.md)。
|
||||
|
||||
## 配置文件
|
||||
|
||||
首次运行在 `-config` 路径自动生成默认配置(默认 `./config.yaml`),核心字段:
|
||||
|
||||
```yaml
|
||||
listen: 127.0.0.1:8080 # 网关监听地址(建议绑内网/回环)
|
||||
gateway_keys: [sk-gw-<随机>] # 初始 admin 密钥(每次安装随机生成,仅首启写入运行时存储用)
|
||||
default_model: AUTO # model 无法路由时自动按优先级链选源
|
||||
adapter_dir: adapters # Lua 适配目录;目录不存在时首启创建并 seed 内置,存在则只读
|
||||
runtime_file: runtime.json # WebUI 编辑的源/密钥/AUTO 链持久化到此文件
|
||||
|
||||
sources:
|
||||
- name: deepseek
|
||||
base_url: https://api.deepseek.com
|
||||
api_key: sk-... # 也可用 api_key_env: SOME_ENV 引用环境变量(不落盘明文)
|
||||
adapter: deepseek
|
||||
max_concurrent: 8
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
priority: 100 # YAML 源首次启动会 seed 进 AUTO 链的初值
|
||||
kind: chat
|
||||
- id: deepseek-v4-pro
|
||||
priority: 60
|
||||
kind: chat
|
||||
# 静态请求头(比适配器默认优先)
|
||||
headers: { X-Tenant: prod }
|
||||
# 传给 Lua build_headers 的透传元数据
|
||||
meta: { app_id: x, app_secret: y }
|
||||
temperature: 0.7
|
||||
max_tokens: 4096
|
||||
timeout: 120s # 请求超时,默认 120s
|
||||
```
|
||||
|
||||
|
||||
## 自动探测模型列表
|
||||
|
||||
不想手抄模型 id 时,只填 URL 和 key,让网关去扫:
|
||||
|
||||
```bash
|
||||
# 只探测,不写任何东西
|
||||
curl -X POST http://127.0.0.1:8080/api/sources/discover \
|
||||
-H "Authorization: Bearer sk-gw-…" -H "Content-Type: application/json" \
|
||||
-d '{"base_url":"https://api.vendor.com/v1","api_key":"sk-…"}'
|
||||
# → {"ok":true,"base_url":"…","endpoint":"","models":["gpt-x",…],
|
||||
# "image_models":["gpt-image-2"],"models_path":"/models","latency_ms":1387,…}
|
||||
|
||||
# 探测后直接落库
|
||||
curl -X POST http://127.0.0.1:8080/api/sources/discover/apply \
|
||||
-H "Authorization: Bearer sk-gw-…" -H "Content-Type: application/json" \
|
||||
-d '{"base_url":"https://api.vendor.com/v1","api_key":"sk-…","name":"vendor"}'
|
||||
```
|
||||
|
||||
WebUI 源编辑框里也有「自动探测模型」按钮:填好 URL 和 key 点一下,列表自动
|
||||
填好,仍需点保存才生效。
|
||||
|
||||
几点值得知道:
|
||||
|
||||
- **URL 形态不用猜。** 厂商对「`/v1` 放 base 还是放 path」没有共识,探测会
|
||||
依次尝试若干候选并返回答对的那个,同时给出归一化后的 `base_url` +
|
||||
`endpoint`(`endpoint` 平时留空,因为 openai adapter 自带 `/chat/completions`)。
|
||||
- **`image_endpoint` 是另一套规则。** 网关的图像默认路径是**绝对**的
|
||||
`/v1/images/generations`,而 chat 默认是相对的。所以 `base_url` 带 `/v1` 时
|
||||
必须显式给 `image_endpoint: /images/generations`,否则图像请求 404。
|
||||
探测会替你填好。
|
||||
- **图像模型只看上游自己声明的 modality**,绝不从模型名猜。`gpt-image-2`
|
||||
这类 id 也出现在同时提供对话服务的目录上,猜错会让 AUTO 链选中一个只支持
|
||||
图像生成的模型,其后每一层都被它拖垮。
|
||||
- **不会逐模型探活。** 34 个模型的目录跑一轮要花钱且耗时数分钟,`/models`
|
||||
是足够便宜的信号;模型是否真能应答留给正常请求去发现。
|
||||
- **对已存在的源,必须显式 `replace_models: true`** 才会覆盖模型列表。重新
|
||||
探测顺手抹掉为 AUTO 链手工调过的 priority,是最难排查的一类数据丢失。
|
||||
- **`api_key` 留空表示沿用已存值**,不会把凭据清空。
|
||||
|
||||
|
||||
`config.yaml`、`runtime.json`、`master.key`、`adapters/` 均不提交版本库(见
|
||||
`.gitignore`),请随备份带走——`master.key` 丢失后运行时密文无法解密。
|
||||
|
||||
运行文件生成在 `-config` 所在目录:`adapter_dir` / `runtime_file` 自动指向
|
||||
配置文件旁边的 `adapters/` 与 `runtime.json`,从任意工作目录启动都可用。
|
||||
|
||||
运行时文件(`runtime_file`)中的敏感字段自动加密:
|
||||
|
||||
- 加密算法 AES-256-GCM,格式 `enc:v1:<base64>`。
|
||||
- 主密钥来源:环境变量 `MODELROUTER_MASTER_KEY`(64 位 hex);否则读取
|
||||
`runtime_file` 同目录的 `master.key`;都不存在时首次启动自动生成 `master.key`(0600)。
|
||||
- 升级时旧明文文件自动兼容:首次运行正常读取,任何 UI 保存操作触发全文件加密迁移。
|
||||
- 注意:**master.key 丢失后密文无法解密**,请随配置一起备份;切勿提交到版本库。
|
||||
|
||||
## 网关密钥(多密钥)
|
||||
|
||||
网关鉴权采用「多密钥 + 角色 + 模型范围」架构,密钥持久化在 `runtime_file` 的
|
||||
`keys` 字段(加密存储):
|
||||
|
||||
- `gateway_keys` 配置只是**初始 admin 密钥种子**:首次启动迁移为运行时 admin
|
||||
key,之后不再参与鉴权管理。
|
||||
- **重要:首次启动后请在 WebUI「密钥」页更换管理员密钥**——初始密钥明文写在
|
||||
`config.yaml` 里(每次安装随机生成),继续使用存在被盗风险;用新密钥登录后删除初始密钥。
|
||||
- WebUI **密钥页**可创建/删除密钥;每个密钥可指定 `admin`(管理全部)或
|
||||
`user`(仅看自己的 key)角色,并配置**模型范围**(模型 + 源 + token 配额 +
|
||||
重置周期)。
|
||||
- 客户端用任意一个已授权的密钥明文作为 Bearer(`Authorization: Bearer <key>`)。
|
||||
- 删除密钥即从运行时存储移除,立即失效。
|
||||
|
||||
### 密钥用量配额
|
||||
|
||||
**配额按模型单独设置**:每个密钥的 `models[]` 里,每一条模型范围各自带一份
|
||||
token 配额与请求数配额。一个模型用满只影响该模型,同一密钥的其它模型照常工作。
|
||||
|
||||
```yaml
|
||||
keys:
|
||||
- key: sk-gw-<hex>
|
||||
role: user
|
||||
name: agent-alice
|
||||
models:
|
||||
- model: deepseek-v4-flash
|
||||
token_quota: 1000000 # 本周期内该 key 用这个模型的 token 预算
|
||||
req_quota: 20000 # 本周期内的请求次数
|
||||
period: nhour # "" | hour | week | month | nhour
|
||||
hours: 6 # 仅 nhour
|
||||
- model: AUTO
|
||||
token_quota: 5000000 # AUTO 也是一条独立配额
|
||||
period: day # ← 这种写法会被拒绝(词表只有 hour/week/month/nhour)
|
||||
- model: kimi-k3 # 未列配额 = 无限
|
||||
```
|
||||
|
||||
- **没有「整钥总配额」**:这是刻意的设计。整钥总额会让「把 A 模型的额度挪给
|
||||
B 模型」变成一次全局重分配;按模型独立计费则每个模型各自可控,运维可以
|
||||
看出哪个模型吃掉了预算。
|
||||
- `period` 词表:空 = 永不过期(累计总量),`hour` / `week` / `month` = 固定窗口,
|
||||
`nhour` + `hours` = 自定义小时数。**拼错的周期在写入时就被拒**,不会静默变成
|
||||
永不过期。
|
||||
- 配额严格按密钥隔离,且**同一密钥内按模型隔离**:A 密钥用满 `m1` 不会消耗
|
||||
B 密钥的额度,同一密钥的 `m2` 也不受影响。
|
||||
- 配额统计含聊天、流式、生图,跨重启从审计日志回放(保留 40 天,覆盖最长的
|
||||
month 窗口)。
|
||||
- 配额耗尽返回 **429 + `Retry-After`**(`rate_limit_exceeded`),消息里点名是哪个
|
||||
模型用满了,客户端可等窗口重置后自动恢复;模型越权才是 403。
|
||||
**admin 密钥永不受配额限制**(其 scope 上的配额也不强制),避免把管理员
|
||||
锁在门外。
|
||||
- 窗口用量按整点小时分桶统计,实际释放比配置窗口最多晚 1 小时(配额宁可晚释放
|
||||
也不超发)。
|
||||
- `PUT /api/keys/{key}` 提交 `models` 即同时提交它们的配额(配额就是 scope 的一部分,
|
||||
不存在会与模型列表脱节的第二份预算)。显式 `0` = 解除该模型的限制。
|
||||
|
||||
#### 配额拒绝 vs 容量拒绝:两种「拒绝」含义不同
|
||||
|
||||
配额在**入口**检查,不占用任何上游槽位,因此是廉价拒绝;容量不足则要走
|
||||
`busyWait` 有界等待后才返回 503。实测(容量 8、0.6s/请求、100 并发):
|
||||
|
||||
| 情况 | 状态码 | 延迟 | 是否打到上游 |
|
||||
|---|---|---|---|
|
||||
| 配额用尽(`req_quota` 50 → 100 并发) | `429 rate_limit_exceeded` + `Retry-After` | **19–21ms** | 否 |
|
||||
| 容量打满(同一拓扑,未限配额) | `503 upstream_error` | ~2.6s(`busyWait`) | 超出部分否 |
|
||||
|
||||
客户端因此可以区分「等窗口重置」与「等上游腾出容量」:前者只需等待,后者
|
||||
通常该降并发或换源。
|
||||
|
||||
#### 配额的开销
|
||||
|
||||
配额桶按 (密钥, 模型, 整点小时) 分桶保留 40 天,实测(AMD 7840HS):
|
||||
|
||||
- 窗口查询按窗口长度而非保留总量扫描:1h 窗口 49ns、24h 窗口 55ns、
|
||||
30d 窗口 3.9μs(此前全扫保留总量,960 桶时 5.9μs)。
|
||||
- 记录一条请求:283ns、3 分配(与引入配额前相同,分配来自 ring buffer)。
|
||||
- 内存:生产形态(7 密钥 × 8 模型 × 2 源 × 满 40 天 retention)约 **3.7MB**。
|
||||
按源 pin 的 `source::model` 桶**惰性创建**——只有当某条配额真的 pin 了
|
||||
某个源时才维护,否则每条记录多写一份桶,在 20 密钥 × 8 模型 × 3 源下会
|
||||
多占 18MB。
|
||||
代价:配置 pinned 配额之前的历史用量无法事后按源拆分,所以 **pinned 配额的
|
||||
首个窗口可能少算**。
|
||||
|
||||
```bash
|
||||
# 配额耗尽时客户端看到(点名了具体模型)
|
||||
HTTP/1.1 429 Too Many Requests
|
||||
Retry-After: 2100
|
||||
{"error":{"type":"rate_limit_exceeded","message":"token quota exceeded for \"deepseek-v4-flash\" (5000000/5000000)"}}
|
||||
```
|
||||
|
||||
142
docs/deployment.md
Normal file
142
docs/deployment.md
Normal file
@ -0,0 +1,142 @@
|
||||
# 部署与迁移
|
||||
|
||||
> 本文档面向运维:单机 systemd 部署、发行包安装、目录与权限、从 `llmsproxy`
|
||||
> 改名后的迁移与回滚。
|
||||
> 返回 [README](../README.md)。
|
||||
|
||||
## 1. 单机部署(推荐)
|
||||
|
||||
仓库根目录的 `deploy.sh` 负责:构建 → 预检配置 → 备份 → 原子替换二进制 →
|
||||
重启服务 → 后置校验。它**不**修改源码,也**不**假设全新机器。
|
||||
|
||||
```bash
|
||||
sudo ./deploy.sh # 默认读取仓库内 config.yaml
|
||||
sudo ./deploy.sh --config other.yaml # 投放并部署另一份配置
|
||||
```
|
||||
|
||||
部署目标(改名后):
|
||||
|
||||
> 两种布局并存,区别只在于二进制位置:**包安装**(deb/rpm)放 `/usr/bin`,
|
||||
> unit 自带;**源码部署**(`deploy.sh`)放 `/usr/local/bin`,需要让 unit 的
|
||||
> `ExecStart` 指向它。`deploy.sh` 会校验这一点,写错时直接中止而不是留下
|
||||
> 一个 203/EXEC 的服务。
|
||||
|
||||
| 项 | 路径(源码部署 / 包安装) |
|
||||
|---|---|
|
||||
| 二进制 | `/usr/local/bin/modelrouter` / `/usr/bin/modelrouter` |
|
||||
| 配置 | `/etc/modelrouter/config.yaml` |
|
||||
| 适配器 | `/etc/modelrouter/adapters/` |
|
||||
| 插件(可选) | `/etc/modelrouter/plugins/`(需在配置里设 `plugin_dir`) |
|
||||
| 主密钥 | `/etc/modelrouter/master.key`(0600) |
|
||||
| systemd unit | `/etc/systemd/system/modelrouter.service` |
|
||||
|
||||
`deploy.sh` 会在以下情况**中止**而不是"看起来成功":
|
||||
|
||||
- 部署后二进制起不来、或新配置 `-check` 不通过;
|
||||
- 凭据无法解封(`master.key` 丢失 / 与配置不匹配);
|
||||
- 检测到改名前的 `/etc/llmsproxy`,但新配置尚未建立(否则会部署出一个
|
||||
没有 key、没有源的空实例,所有请求都被拒)。
|
||||
|
||||
## 2. 发行包安装
|
||||
|
||||
三个产物由 `packaging/core-dist.sh` 生成到 `cmd/build/dist/`:
|
||||
|
||||
```text
|
||||
modelrouter_<ver>_amd64.deb
|
||||
modelrouter-<ver>.x86_64.rpm
|
||||
modelrouter-<ver>-linux-amd64.tar.gz
|
||||
```
|
||||
|
||||
deb / rpm 安装后:
|
||||
|
||||
```bash
|
||||
sudo systemctl enable --now modelrouter
|
||||
sudo journalctl -u modelrouter -f
|
||||
```
|
||||
|
||||
tar 包内只有:`modelrouter`、`config.example.yaml`、`modelrouter.service`、
|
||||
`migrate-to-modelrouter.sh`、`adapters/`。
|
||||
|
||||
**发行包不含业务插件**(`channels` / `billing` / `quota` / `usage` 等)。
|
||||
它们保留在仓库中作为可选扩展,需要时手工放入 `plugin_dir`,网关不会自动安装、
|
||||
覆盖或删除。这是有意的:插件属于运维资产,生命周期不与内核绑定。
|
||||
|
||||
首次启动会在 `/etc/modelrouter/config.yaml` 生成一份**随机 admin key** 的配置,
|
||||
key 打印在启动日志里。发行包不预置任何可用 key(历史上曾因此泄露过一个真实的
|
||||
`sk-gw-local-0001`)。
|
||||
|
||||
## 3. 目录与权限
|
||||
|
||||
- `/etc/modelrouter` 是唯一的读写根(systemd unit 用 `ProtectSystem=strict` +
|
||||
`ReadWritePaths=/etc/modelrouter` 固化这一点)。
|
||||
- `master.key` 必须 0600。当前 unit 以 root 运行:降权到普通用户需要先把
|
||||
`master.key` 交给该用户并统一目录属主,否则服务会"带着明文落盘继续跑",
|
||||
失败只留一行日志——这是独立变更,不要和加固混在一起。
|
||||
- `plugin_dir` 由运维维护;插件自带的状态文件(如 `.channels-store.json`)也在其中。
|
||||
|
||||
## 4. 从 llmsproxy 迁移
|
||||
|
||||
二进制、配置目录、服务名、环境变量已统一为 `modelrouter`:
|
||||
|
||||
| 旧 | 新 |
|
||||
|---|---|
|
||||
| `/usr/local/bin/llmsproxy` | `/usr/local/bin/modelrouter` |
|
||||
| `/etc/llmsproxy/` | `/etc/modelrouter/` |
|
||||
| `llmsproxy.service` | `modelrouter.service` |
|
||||
| `LLMSPROXY_PLUGIN_DIR` | `MODELROUTER_PLUGIN_DIR` |
|
||||
| `LLMSPROXY_XDISPLAY_BASE` | `MODELROUTER_XDISPLAY_BASE` |
|
||||
| `LLMS_PROXY_MASTER_KEY` | `MODELROUTER_MASTER_KEY` |
|
||||
|
||||
迁移脚本**默认 dry-run**,复制而非移动,旧目录永不删除:
|
||||
|
||||
```bash
|
||||
sudo ./packaging/migrate-to-modelrouter.sh # 预演(逐条打印将做什么)
|
||||
sudo ./packaging/migrate-to-modelrouter.sh --apply # 执行
|
||||
```
|
||||
|
||||
它做五件事:复制状态目录并改写配置里的旧绝对路径、复核 `master.key` 0600、
|
||||
安装新二进制(旧文件保留)、由旧 unit 生成新 unit(改写 ExecStart /
|
||||
WorkingDirectory / ReadWritePaths)、复制 `/usr/share` 资源;最后跑
|
||||
`modelrouter -check` 并 `systemctl daemon-reload`。
|
||||
|
||||
确认新服务可用后,再手工清理旧布局(脚本不会删):
|
||||
|
||||
```bash
|
||||
sudo systemctl disable --now llmsproxy.service
|
||||
sudo rm -f /etc/systemd/system/llmsproxy.service
|
||||
sudo rm -rf /usr/local/bin/llmsproxy /etc/llmsproxy /usr/share/llmsproxy
|
||||
```
|
||||
|
||||
回滚(不需要旧文件被删):
|
||||
|
||||
```bash
|
||||
sudo ./packaging/migrate-to-modelrouter.sh --rollback --apply
|
||||
```
|
||||
|
||||
**读取侧兼容**:旧配置路径与旧环境变量仍被接受,所以"先换二进制、后改 unit"
|
||||
这种顺序不会立刻失效。但写入永远落在新路径,迁移完成后不要两台布局混用。
|
||||
|
||||
## 5. 部署后验证清单
|
||||
|
||||
```bash
|
||||
# 1) 进程与监听
|
||||
systemctl status modelrouter --no-pager
|
||||
ss -ltnp | grep modelrouter
|
||||
|
||||
# 2) 鉴权生效(无 key 必须 401)
|
||||
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080/v1/models
|
||||
|
||||
# 3) 带 key 能列出模型
|
||||
curl -s -H "Authorization: Bearer $KEY" http://127.0.0.1:8080/v1/models | head -c 400
|
||||
|
||||
# 4) 真实请求走通(证明 LuaJIT 适配器路径未被打断)
|
||||
curl -s -H "Authorization: Bearer $KEY" \
|
||||
-d '{"model":"AUTO","messages":[{"role":"user","content":"hi"}]}' \
|
||||
http://127.0.0.1:8080/v1/chat/completions | head -c 400
|
||||
|
||||
# 5) 仓库文件与生产文件一致(部署脚本的收尾检查)
|
||||
sha256sum /usr/local/bin/modelrouter build/modelrouter
|
||||
```
|
||||
|
||||
第 4 步失败但第 3 步成功,通常是适配器或上游凭据问题,不是部署问题:
|
||||
查看 `journalctl -u modelrouter -n 100`。
|
||||
34
docs/development.md
Normal file
34
docs/development.md
Normal file
@ -0,0 +1,34 @@
|
||||
# 开发与目录结构
|
||||
|
||||
> 本文档由 README 拆分而来:目录职责、开发协作、测试。
|
||||
> 返回 [README](../README.md)。
|
||||
|
||||
## 目录
|
||||
|
||||
```
|
||||
cmd/modelrouter # 入口(纯 Go 服务端)
|
||||
cmd/gui # Electron 桌面 GUI(内嵌核心 + 托盘 + 打包)
|
||||
internal/config # YAML 配置加载/校验
|
||||
internal/lua # LuaJIT VM + worker 池 + AdapterCache + 内置适配器 (embed)
|
||||
internal/provider # Provider(HTTP) + Registry(路由)
|
||||
internal/gateway # OpenAI 兼容 HTTP 服务 + 鉴权 + SDK/流式 + WebUI
|
||||
internal/types # 统一格式 & OpenAI wire 类型
|
||||
```
|
||||
|
||||
## 开发协作
|
||||
|
||||
- **Git 分支工作流**:主分支 / 特性分支 / 发布分支的规范见
|
||||
**[docs/git-workflow.md](docs/git-workflow.md)**([English](docs/git-workflow-en.md))。
|
||||
主线:`main` 永可部署 → 特性合入 `main` → 切 `release/vX.Y.Z` 打 tag 发布 →
|
||||
hotfix 提交发布分支并 cherry-pick 回 `main`。
|
||||
- **Lua 适配器协议**:见 **[docs/lua-adapters.md](docs/lua-adapters.md)**
|
||||
([English](docs/lua-adapters-en.md))。
|
||||
|
||||
## 测试
|
||||
|
||||
```bash
|
||||
go test -tags luajit ./...
|
||||
```
|
||||
|
||||
覆盖:配置校验、适配器加载/变换、签名钩子、Gateway 鉴权、SDK 结算、SSE 流式、
|
||||
模型路由、多模态透传与 thinking 预算。
|
||||
48
docs/features.md
Normal file
48
docs/features.md
Normal file
@ -0,0 +1,48 @@
|
||||
# 实现与完整特性
|
||||
|
||||
> 本文档由 README 拆分而来:架构来源与完整特性清单(README 只保留摘要)。
|
||||
> 返回 [README](../README.md)。
|
||||
|
||||
## 实现
|
||||
|
||||
统一 OpenAI 兼容网关:把多个上游 LLM 源(DeepSeek、Qijiar、OpenAI、Anthropic、Gemini、
|
||||
Groq、Mistral、Ollama、KimiCode…)通过 **Lua 适配器** 做协议转换,对内网暴露
|
||||
一个标准 `OpenAI Chat Completions` 接口(`/v1/chat/completions` + `/v1/models`),
|
||||
支持单次结算、SSE 流式、**AUTO 模型路由**与**多模态**透传。
|
||||
|
||||
从 [HomeAgent](https://gitcode.com/JianFeeeee/HomeAgent) 的多源 LLM 适配层
|
||||
(`internal/agent/api/provider.go` + `internal/lua/adapters/*`)抽离而来并独立演进。
|
||||
|
||||
## 特性
|
||||
|
||||
- **多源**:一个进程内配置任意多个上游源,按请求的 `model` 自动路由。
|
||||
- **AUTO 模式**:`default_model: AUTO` 时按优先级页保存的 AUTO 链档位逐档调度并发请求。
|
||||
- **统一输出**:所有源对外都是 OpenAI 格式(含 `reasoning_content`、`tool_calls`、`usage`)。
|
||||
- **生图**:`POST /v1/images/generations`,`kind: image` 的模型独立路由。
|
||||
- **多模态**:`content` 数组(`image_url` 等)在多源间无损透传,Anthropic/Gemini/Ollama 自动转换。
|
||||
- **LuaJIT VM**:基于 golua 绑定的 LuaJIT,每个适配器独立 VM + worker 池,安全并发。
|
||||
池是 **弹性** 的:`Σ max_concurrent` 只是上限而非预分配,状态按需创建、空闲时自动回收,
|
||||
因此空载网关几乎不持有 Lua 状态(启动 0 个)。
|
||||
- 扩容步长由适配器最大并发决定:`clamp(ceil(max/8), 1, 8)`,且仅在**真实并发争用**
|
||||
(所有现有状态均已占用)时批量预热,顺序流量始终只用 1 个状态。
|
||||
- 缩容步长由当前连接数决定:`clamp(ceil(冗余/(1+占用)), 1, 冗余)`——无连接时一轮收到
|
||||
常驻下限,忙时每轮只释放 1 个,保护热路径。WebUI「适配器」页展示实时池指标。
|
||||
- **thinking 预算**:请求里的推理开关/预算归一为中立的 `thinking:{enabled,budget_tokens}`,
|
||||
适配器映射到各自协议(Anthropic `thinking.budget_tokens`、Gemini `thinkingConfig`、
|
||||
DeepSeek `thinking.type="disabled"`)。
|
||||
- **Lua 适配协议**:每个源挂一个 `.lua` 适配器,完成 `transform_request` /
|
||||
`transform_response` / `transform_stream_chunk` 双向转换,协议差异全在 Lua 层。
|
||||
- **签名 / 请求头钩子**:适配器可定义 `build_headers(meta)`,在 Go 发 HTTP 前
|
||||
动态注入/签名请求头——用于云端 API 校验调用方 app(如 KimiCode 只放行特定
|
||||
agent)。提供 `hmac_sha256_hex` / `sha256_hex` / `base64_encode` 等签名辅助。
|
||||
- **WebUI**:内置管理界面,可在线查看/新增/编辑上游源与模型、配置 AUTO 优先级链、
|
||||
管理密钥模型范围,写入运行时文件持久化。
|
||||
- **密钥加密存储**:运行时文件中的上游 `api_key`、自定义请求头值、网关 key 均以
|
||||
AES-256-GCM 加密落盘(`master.key` 独立 0600,或 `MODELROUTER_MASTER_KEY`)。
|
||||
- **实时源探测**:状态页探测各源可达性(`GET {base}/models`,失败回退最小请求),
|
||||
不污染正常调度的退避状态,错误信息在 UI 可悬停查看。
|
||||
- **鉴权**:网关自身用 `gateway_keys` 校验客户端 Bearer key;与上游各自的 key 相互独立。
|
||||
- **流式**:SSE `chat.completion.chunk`,含角色首包与 `[DONE]` 收尾。
|
||||
- **全链路审计**:HTTP 访问日志、登录/配置变更审计、请求记录 CSV 导出、密钥用量统计 CSV 导出
|
||||
- **AES-256-GCM 加密存储**:运行时文件敏感字段加密落盘,master.key 0600 权限,支持环境变量注入主密钥
|
||||
|
||||
86
docs/gui.md
Normal file
86
docs/gui.md
Normal file
@ -0,0 +1,86 @@
|
||||
# 桌面 GUI(Electron)
|
||||
|
||||
> 本文档由 README 拆分而来:Headless 与 Desktop 两种产物、构建与运行。
|
||||
> 返回 [README](../README.md)。
|
||||
|
||||
## 桌面 GUI(Electron)
|
||||
|
||||
可选独立产物,面向内网桌面端用户:**内嵌 ModelRouter 核心**(桌面壳形态),
|
||||
一键启停、系统托盘常驻、开机自启、静默启动,内容区一比一内嵌完整 WebUI
|
||||
(状态/对话/密钥/优先级/源/适配器 六个页面,免登录)。服务器用户继续使用纯 Go 二进制。
|
||||
|
||||
### 两种安装包:Headless 与 Desktop
|
||||
|
||||
发布到 Release 的安装包分为两类,面向不同用户:
|
||||
|
||||
- **Headless(服务器版)**:纯核心二进制(无 Electron),配置驱动,适合服务器、
|
||||
容器、systemd 等无人值守场景。发布包含 `deb` + `rpm`(含 systemd unit、内存调优
|
||||
与示例配置)与 `tar.gz`(二进制 + `config.example.yaml` + `adapters/` + README),
|
||||
Windows 为 `.zip`。
|
||||
- **Desktop(桌面版)**:Electron GUI 安装包,内嵌核心,适合个人桌面日常使用。
|
||||
|
||||
### 选择 GUI 还是纯后端
|
||||
|
||||
| 场景 | 推荐 | 理由 |
|
||||
| ------ | ------ | ------ |
|
||||
| 服务器 / 内网网关 / 无人值守常驻 | **Headless**(单二进制) | 极轻量(二进制 ~8MB;RSS 随源数量,单源 ~10MB、16 源 ~32–35MB,见[内存占用](#内存占用实测与调优)),零依赖单进程,直接跑在 systemd/任意容器里,远程管理 |
|
||||
| 个人桌面日常使用 / 多设备内网共享 | **Desktop**(GUI) | 免登录内嵌 WebUI、系统托盘一键启停、开机自启、静默后台,适合不懂命令行的使用者 |
|
||||
| Windows 桌面 | **Desktop** | 纯后端在 Windows 上需自行注册服务,GUI 提供原生托盘/自启体验 |
|
||||
| CI 一键出三平台安装包 | **Desktop 打包脚本** | `make gui-*` 系列出 deb/AppImage/NSIS,可进发行流水线 |
|
||||
|
||||
两者完全同源:GUI 内嵌的就是纯后端的同一份 `modelrouter` 核心(LuaJIT 版),
|
||||
配置/适配器格式完全一致,可随时互换。
|
||||
|
||||
### 构建与运行
|
||||
|
||||
```bash
|
||||
cd cmd/gui
|
||||
npm install
|
||||
make gui # 或 npm run dev —— 开发运行(需先 make build 生成 bin/modelrouter)
|
||||
make gui-deb # 本机构建 deb(分享给其它 Linux 用户)
|
||||
make gui-dist # 构建 deb + rpm + AppImage(linux,rpm 需系统 rpmbuild)
|
||||
make core-dist # 构建核心 deb + rpm + tar.gz(服务器版,需 nfpm)
|
||||
make gui-win # 构建 Windows nsis 安装包(本机 mingw + wine)
|
||||
make gui-win-docker # 构建 Windows nsis 安装包(全 docker 自足,含 wine)
|
||||
make gui-dist-dir # 仅解包产物(调试用,不产安装包)
|
||||
```
|
||||
|
||||
产物在 `cmd/build/gui-dist/`:`ModelRouter-<ver>.AppImage`、
|
||||
`modelrouter-gui_<ver>_amd64.deb`、`ModelRouter Setup <ver>.exe`(win);
|
||||
核心包在 `cmd/build/dist/`:`modelrouter_<ver>_amd64.deb`、`modelrouter-<ver>.x86_64.rpm`。
|
||||
rpm 需系统 `rpmbuild`。每次打包后都会跑 `packaging/verify-dist.sh` 产物体检
|
||||
(大小下限门禁——1.3.0 曾因宿主 wine 损坏产出 264KB 的残缺 exe 却没有被拦住)。
|
||||
发布时会把产物重命名为 `ModelRouter-Desktop-*`(GUI)与 `ModelRouter-Headless-*`(纯后端)
|
||||
两类上传到 GitCode Release。
|
||||
|
||||
#### Windows 打包(docker 固化方案,推荐)
|
||||
|
||||
`make gui-win-docker` 一步完成:交叉编译核心 + NSIS 打包**全在 docker 内**,宿主机
|
||||
**无需 mingw、无需 wine、无需 node**。镜像内置 wine(含 i386 运行时),修复了
|
||||
此前宿主 wine 损坏(缺 syswow64 → `c0000135`)导致产出 264KB 残缺 exe 的问题。
|
||||
|
||||
- 构建镜像:`cmd/gui/docker/win-builder/Dockerfile`(`golang:1.25` + mingw-w64 +
|
||||
node + wine32/wine64 + 预编译 LuaJIT windows 版 `lua51.dll` 与 import lib)。
|
||||
LuaJIT 源码取自 gitcode 镜像(国内可达),github 兜底;Go 依赖走 `https://goproxy.cn`。
|
||||
- 一键发布:`cmd/gui/scripts/dist-win-docker.sh` —— 核心编译 + electron-builder
|
||||
NSIS 全部在容器内执行(`USE_SYSTEM_WINE=true` 走镜像内的 wine);electron-builder
|
||||
的 ~400MB 工具链缓存挂到 docker volume,二次构建增量复用。
|
||||
- 首次运行自动构建 `modelrouter/win-builder` 镜像(约 5–10 分钟);已存在则直接复用。
|
||||
- 打包完成后自动跑 `packaging/verify-dist.sh` 门禁:exe 低于 5MB / deb·rpm·7z 低于
|
||||
10MB 即失败——残缺安装包不可能再溜进 Release。
|
||||
|
||||
```bash
|
||||
# 全 Windows 发布(核心重编 + NSIS):
|
||||
make gui-win-docker
|
||||
```
|
||||
|
||||
### 特性
|
||||
|
||||
- **内嵌核心**:自动拉起捆绑的 `modelrouter`(LuaJIT 版),配置/密钥/adapter 存于
|
||||
用户数据目录 `<userData>/profile/`,首次运行自动生成随机 admin key 写入 `keys`
|
||||
(非 seed)并自注入,WebUI 免登录、不会提示"更换初始密钥"。端口可在设置中更改(默认 8787)。
|
||||
- **系统托盘**:内核状态点、开机自启/静默启动开关、启停/重启内核、退出。
|
||||
- **静默启动**:`--silent` 参数或设置项,启动仅驻托盘不弹窗;首次运行始终显示窗口。
|
||||
- **开机自启**:Windows/macOS 用 `setLoginItemSettings`;Linux 写
|
||||
`~/.config/autostart/modelrouter-gui.desktop`,静默开启时自动附加 `--silent`。
|
||||
|
||||
60
docs/performance.md
Normal file
60
docs/performance.md
Normal file
@ -0,0 +1,60 @@
|
||||
# 性能与内存
|
||||
|
||||
> 本文档由 README 拆分而来:实测内存占用、调优手段、thinking 预算。
|
||||
> 返回 [README](../README.md)。
|
||||
|
||||
## 内存占用(实测与调优)
|
||||
|
||||
内存不是一个定数:它随**配置的源数量**增长(每个源一个 `http.Transport` 与连接池、
|
||||
一组健康状态),与运行时长无关。本机实测(Linux x86_64,12 核):
|
||||
|
||||
| 部署形态 | 启动 RSS | 稳态 RSS |
|
||||
|---|---|---|
|
||||
| 1 源 / 1 适配器(最小配置) | ~4 MB | ~10 MB |
|
||||
| 1 源 + 29 MB 历史审计日志 | ~19 MB | ~20 MB |
|
||||
| **16 源 / 13 适配器 / 59 模型(本机生产)** | ~28 MB | **~32–35 MB** |
|
||||
|
||||
> 历史参考:本项优化前同一生产配置为 **~105 MB**。降幅来自三处:审计日志的**原始记录**
|
||||
> 不再常驻内存(约 25 MB;统计聚合仍扫全量,但扫完即释放)、Lua 状态池不再单调增长、
|
||||
> 以及下面两个运行时开关。
|
||||
|
||||
内存构成(生产实例分段测量,`/proc/<pid>/smaps`):
|
||||
|
||||
| 区域 | RSS | 说明 |
|
||||
|---|---|---|
|
||||
| Go 堆 | ~14 MB | provider/registry/scheduler 结构 + 连接池缓冲 |
|
||||
| 其他匹名(线程栈 / LuaJIT chunk / runtime) | ~12 MB | 与线程数、已加载适配器数相关 |
|
||||
| 二进制 text+rodata | ~8 MB | 映射的可执行文件页(只读、可被内核回收) |
|
||||
| Go runtime 预留 | ~4 MB | VSZ 上看到的 2 GB+ 是地址空间预留,不占物理内存 |
|
||||
| 共享库 | ~3 MB | libc / libluajit / libm |
|
||||
|
||||
**两个推荐的部署开关**(只适用于环境变量,无需改代码):
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/modelrouter.service
|
||||
Environment=GOGC=50
|
||||
Environment=MALLOC_ARENA_MAX=2
|
||||
```
|
||||
|
||||
- `MALLOC_ARENA_MAX=2`:LuaJIT 的分配走 cgo → glibc malloc,glibc 默认允许 `8×nproc`
|
||||
个 per-thread arena,每个碰到 malloc 的 OS 线程会占用一个(各约 1 MB,且**不归还给系统**)。
|
||||
实测从 8–15 个 arena(约 7–12 MB)降到 **0**。
|
||||
- `GOGC=50`:把 Go 堆增长目标减半。**单独使用无效**(省下的堆会立即被更多 glibc
|
||||
arena 吃掉,实测 20.3 → 21.5 MB 反而变大),必须与 `MALLOC_ARENA_MAX` 配合,
|
||||
两者同时开启才降 **~19%**。网关是 I/O 密集型(本机 9 小时仅消耗 1min10s CPU),
|
||||
多出的 GC 周期与它的空闲 CPU 相比可忽略。
|
||||
|
||||
> 桌面版(Electron)已在 `cmd/gui/main.js` 里默认为内嵌核心注入这两个开关;手动 export 同名
|
||||
> 环境变量可覆盖。如果你用自己的 systemd unit / 容器,建议照上面加上。
|
||||
|
||||
可选:`GOMEMLIMIT=48MiB` 作软上限,实测再省 ~1 MB;代价是逐近上限时 GC 转激进,
|
||||
温和场景不必开。
|
||||
|
||||
## thinking 预算
|
||||
|
||||
请求体带 `"thinking": {"enabled": false}` 时关闭推理;`"thinking": {"enabled": true,
|
||||
"budget_tokens": 4096}` 指定预算。内核只定义这个**中立**语义字段,映射到具体协议由
|
||||
适配器负责(Anthropic `thinking.budget_tokens`、Gemini `thinkingConfig.thinkingBudget`、
|
||||
DeepSeek `thinking.type = "disabled"`)。内核不再向适配器泄漏 `disable_thinking` /
|
||||
`extra_body` 这类私有字段。
|
||||
|
||||
118
docs/plugins.md
118
docs/plugins.md
@ -15,6 +15,37 @@ ModelRouter 的插件是**单个 `.lua` 文件**,放在 `config.yaml` 的 `plu
|
||||
|
||||
---
|
||||
|
||||
## 0. 插件 SDK:唯一的公开接口
|
||||
|
||||
插件与网关之间**只有一个**公开接口表 `modelrouter`。它不是 Go 包的导出符号,
|
||||
而是内核在启动插件 VM 时注入的 Lua 表;插件通过它调用内核能力,永远不需要
|
||||
(也不应该)依赖任何 `__llmsproxy_*` 全局、Go 结构体或内部函数名。
|
||||
|
||||
```lua
|
||||
modelrouter.sdk_version -- 字符串,契约版本;SDK 形状变化才递增
|
||||
modelrouter.http(spec) -- 受出站白名单约束的 HTTP 请求
|
||||
modelrouter.secrets -- 本插件专用的凭据保险库(set/get/delete/keys)
|
||||
modelrouter.kernel -- 内核动作面(actions/call,受运维授权约束)
|
||||
modelrouter.adapters -- 安装/更新/删除本插件自己的适配器
|
||||
modelrouter.plugin_dir -- 本插件所在目录(内核给出的 ground truth)
|
||||
```
|
||||
|
||||
规则:
|
||||
|
||||
- **只依赖上表**。内核重命名内部全局、改实现或加守卫时,插件不受影响;
|
||||
这正是把六个散落的全局收进一张表的原因。
|
||||
- **能力可能缺失**。未授予的能力(例如出站 HTTP 全禁)在表里是 `nil` 而不是
|
||||
"存在但一调用就报错",插件应先判断再使用。
|
||||
- **`sdk_version` 用来做兼容分支**。增量字段不递增版本;字段删除或调用约定
|
||||
变化才递增。
|
||||
- **历史别名**:`llmsproxy`(旧产品名)与 `__llmsproxy_{http,secrets,kernel,
|
||||
adapters,plugin_dir}` 仍然指向同一批对象,只为已安装的插件继续可用而保留,
|
||||
已废弃;新插件使用 `modelrouter.*`。
|
||||
- 环境变量:插件目录优先读 `MODELROUTER_PLUGIN_DIR`,回退旧名
|
||||
`MODELROUTER_PLUGIN_DIR`。
|
||||
|
||||
---
|
||||
|
||||
## 1. 快速上手
|
||||
|
||||
一个最小的插件:
|
||||
@ -65,16 +96,14 @@ return plugin -- 必须返回一个 table
|
||||
core.New
|
||||
└─ lua.NewVM(adapter_dir).Start() 适配器状态
|
||||
└─ lua.NewPlugins(vm, plugin_dir)
|
||||
├─ SeedBundled() 仅当目录不存在时写入内置插件(目前是 billing)
|
||||
└─ LoadDir() 按文件名字典序逐个加载
|
||||
`plugin_dir` 中的 Lua 文件由网关逐个加载;安装包不携带业务插件,也不会自动创建或写入插件文件。首次启用时,先把需要的 `.lua` 插件放入该目录,再启动网关。
|
||||
```
|
||||
|
||||
**加载失败不影响网关启动。** 一个语法错误的插件会被记录在 `GET /api/plugins`
|
||||
的 `error` 字段里,永远不会被调用。这与适配器一致,但理由更强:插件是可选的
|
||||
第三方扩展,因为一个 `.lua` 打错字就让网关起不来是错误的取舍。
|
||||
|
||||
**目录一旦存在就是权威的。** 与适配器同规则:首启会 seed 内置插件,之后目录里
|
||||
的文件说了算,删除或编辑内置插件都是真实生效的操作。
|
||||
插件目录由运维独立管理。网关不会 seed、覆盖或删除其中的插件;目录缺失时只会跳过插件加载,不影响网关启动。
|
||||
|
||||
### 2.1 热更新
|
||||
|
||||
@ -294,7 +323,7 @@ PUT /api/plugins/{name}/state 替换状态(admin)
|
||||
"loaded": true
|
||||
}],
|
||||
"hook_errors": { "request_end": { "count": 3, "last_error": "billing: ..." } },
|
||||
"plugin_dir": "/etc/llmsproxy/plugins"
|
||||
"plugin_dir": "/etc/modelrouter/plugins"
|
||||
}
|
||||
```
|
||||
|
||||
@ -428,7 +457,7 @@ Cookie 鉴权都自然生效,无需暴露额外端口。
|
||||
-- 每小时的定时任务:轮换 token 并重写适配器
|
||||
function plugin.refresh_token()
|
||||
local tok = fetch_new_token() -- 插件自己的取 token 逻辑
|
||||
local name, err = __llmsproxy_adapters.install(
|
||||
local name, err = modelrouter.adapters.install(
|
||||
"myauth",
|
||||
'return { name = "myauth", endpoint = "/chat/completions",' ..
|
||||
' headers = { ["Authorization"] = "Bearer ' .. tok .. '" } }'
|
||||
@ -439,9 +468,9 @@ end
|
||||
|
||||
| 调用 | 参数 | 返回 |
|
||||
|---|---|---|
|
||||
| `__llmsproxy_adapters.install(name, source)` | 适配器名 + Lua 源码 | `(生效名, nil)` 或 `(nil, err)` |
|
||||
| `__llmsproxy_adapters.update(name, source)` | 同上(只限自己装的) | `(true, nil)` 或 `(nil, err)` |
|
||||
| `__llmsproxy_adapters.remove(name)` | 适配器名(只限自己装的) | `(true, nil)` 或 `(nil, err)` |
|
||||
| `modelrouter.adapters.install(name, source)` | 适配器名 + Lua 源码 | `(生效名, nil)` 或 `(nil, err)` |
|
||||
| `modelrouter.adapters.update(name, source)` | 同上(只限自己装的) | `(true, nil)` 或 `(nil, err)` |
|
||||
| `modelrouter.adapters.remove(name)` | 适配器名(只限自己装的) | `(true, nil)` 或 `(nil, err)` |
|
||||
|
||||
出错时不抛异常,而是返回 `(nil, err)`——与钩子/HTTP 处理器同一套约定,
|
||||
一个手滑的插件错误不会带走网关。
|
||||
@ -494,24 +523,66 @@ end
|
||||
|
||||
### 6.4 命名
|
||||
|
||||
- 全局 `__llmsproxy_plugin` 存放插件返回的 table,**不要占用**
|
||||
- 全局 `__llmsproxy_plugin` 是内核存放插件返回 table 的**内部位置**,不属于 SDK,**不要占用**
|
||||
- 插件间状态完全隔离,一个插件改不了另一个的全局
|
||||
|
||||
---
|
||||
|
||||
## 7. 内置计费插件
|
||||
## 7. Billing 插件示例
|
||||
|
||||
`plugins/billing.lua`,默认随首启 seed 进来,开箱可用。
|
||||
`internal/lua/plugins/billing.lua` 是仓库中的可选插件示例,不随核心安装包分发,也不会在启动时自动安装。需要使用计费功能时,应将该脚本单独复制到配置的 `plugin_dir` 后再重启网关。
|
||||
|
||||
### 7.1 计费维度
|
||||
### 7.0 分层:billing 完全在插件里,内核没有任何 billing 代码
|
||||
|
||||
| 维度 | 用途 | 优先级 |
|
||||
`billing` 是**插件自己的事**:规则文件、编译、计价、账目、规则编辑页都由
|
||||
`plugins/billing.lua` 负责。内核里没有 billing 的配置结构、端点、编译或迁移逻辑
|
||||
—— 它只提供所有插件通用的能力(加载 `.lua`、插件状态持久化、插件目录、
|
||||
`plugin.http` 路由、插件可用的 `io`/`os`)。
|
||||
|
||||
| 项 | 位置 |
|
||||
|---|---|
|
||||
| 规则文件 | `<plugin_dir>/billing.json`(插件自己读写;内核不知道这个文件) |
|
||||
| 计费账目 | 插件状态文件 `<plugin_dir>/.billing.state.json`(内核的通用状态持久化) |
|
||||
| 规则编辑 API | `/api/plugins/http/billing/rules`(GET/PUT/POST/DELETE,插件路由) |
|
||||
| 编译/计价 | 插件内 `compileRules()` / `priceFor()` / `costFor()` |
|
||||
|
||||
规则以**源名**(`source`)为键,而不是 URL:URL→源名的映射属于内核配置,插件
|
||||
不需要为了它和内核对接;编辑页在浏览器里,直接取 `/api/sources` 给运营选源名。
|
||||
`url: "*"` 只表示"全局模型价目表"(模型维度本来就不分源)。
|
||||
|
||||
> 历史:早期 `billing:` 写在 `config.yaml`、编译在 Go 的 `internal/billing` 里。
|
||||
> GitCode issue #1 的根因正是"Go 与 Lua 各写一半覆盖语义",组合起来把模型价
|
||||
> 静默清零。现在内核侧 billing 代码已全部删除,规则文件也由插件自持。
|
||||
|
||||
### 7.1 计费维度### 7.1 计费维度
|
||||
|
||||
| 维度 | 用途 | token 价覆盖顺序 |
|
||||
|---|---|---|
|
||||
| `sources` | 源的固定价(如按请求计费的转发服务) | 中 |
|
||||
| `models` | 模型的 per-token 价 | 高 |
|
||||
| `default` | 兜底价(通常全 0) | 最低 |
|
||||
| `models` | 模型的 per-token 价 | 中 |
|
||||
| `sources` | 源级价:固定价,或覆盖该源全部模型的 token 价 | 高 |
|
||||
| `keys` | 单个网关 key 的覆盖价 | **最高** |
|
||||
|
||||
**token 价**优先级:`keys` > `models` > `default`。
|
||||
**token 价**优先级:`keys` > `sources` > `models` > `default`。
|
||||
(实现顺序就是"后写覆盖前写":`default` → `models` → `sources` → `keys`;
|
||||
所以**源级 token 价会覆盖模型价**。这是有意的:运营要能给某个源整体改价。)
|
||||
|
||||
⚠️ 正因为 `sources` 会覆盖 `models`:**`mode: token` 的规则不产生任何源条目**。
|
||||
规则未列出的模型会进入未定价统计(可见),而不是静默计 0 —— 这是 issue #1 / PR #2
|
||||
的收口语义。只有 `free` / `subscription`("边际成本为零")会写有意为之的
|
||||
`{prompt:0, completion:0}` 零价条目。
|
||||
|
||||
规则写法:
|
||||
|
||||
```json
|
||||
{"active":"current","profiles":[{"id":"current","default":"unpriced","rules":[
|
||||
{"source":"localzen","mode":"free"},
|
||||
{"source":"codearts","mode":"token",
|
||||
"models":{"glm-5.3":{"prompt":"0.15","completion":"0.60"}}},
|
||||
{"url":"*","mode":"token","models":{"some-model":{"prompt":"1.0","completion":"2.0"}}}
|
||||
]}]}
|
||||
```
|
||||
|
||||
**`per_request` 固定价是叠加的**(不覆盖),所以一个生图模型可以既算 token 又收固定费。
|
||||
**`cache_discount`** 见 §7.7。
|
||||
|
||||
@ -599,6 +670,17 @@ curl -H "Authorization: Bearer $ADMIN_KEY" \
|
||||
|
||||
仪表盘上有 "Unpriced" 卡片。**这个数应该是 0**;不是 0 就去补价目。
|
||||
|
||||
**profile 级兜底**:`profiles[].default` 可设为 `free` 或 `unpriced`(默认)。
|
||||
它是插件运行时的策略(编译期写入 `prices.default_mode`):
|
||||
|
||||
| `default` | 没有单独规则的源 | `unpriced_reqs` |
|
||||
|---|---|---|
|
||||
| `unpriced`(省略时) | 零价,计入未定价债务 | +1 |
|
||||
| `free` | 零价,视为**刻意免费** | 不增加 |
|
||||
|
||||
显式 `mode: unpriced` 的源即使在 `default: free` 下也仍然计入("只统计不收费"
|
||||
的声明不能被兜底策略吞掉)。
|
||||
|
||||
注意"未定价"不等于"免费":这些请求的 `requests` / token 数**照常计入**
|
||||
`total` 与各维度,只有金额是 0。
|
||||
|
||||
@ -651,7 +733,7 @@ cached = cache_hit_tokens → 全价 × cache_discount
|
||||
插件的 `log(level, msg)` 输出到网关的日志:
|
||||
|
||||
```
|
||||
journalctl -u llmsproxy -f | grep -i plugin
|
||||
journalctl -u modelrouter -f | grep -i plugin
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
111
docs/routing.md
Normal file
111
docs/routing.md
Normal file
@ -0,0 +1,111 @@
|
||||
# 路由与调度
|
||||
|
||||
> 本文档由 README 拆分而来:模型路由、AUTO 链、冷却与半冷却、源与适配器、源模板。
|
||||
> 返回 [README](../README.md)。
|
||||
|
||||
## 模型路由
|
||||
|
||||
`/v1/chat/completions` 的 `model` 解析顺序:
|
||||
|
||||
1. `source/model` 或 `source:model` 前缀 → 指定源;
|
||||
2. 精确匹配某个源的 `model`;
|
||||
3. 配置 `default_model: AUTO` 时 → 走优先级页保存的 AUTO 链(见下);
|
||||
4. 否则回落(返回错误)。
|
||||
|
||||
任何 OpenAI 客户端,只要 `model` 设为某个源的 `name/任意名`,即可锁定走该源;
|
||||
设为 `AUTO`(或网关配了 `default_model: AUTO`)即自动按优先级链选源。
|
||||
|
||||
## AUTO 链(优先级页)
|
||||
|
||||
AUTO 调度**只**由优先级页保存的规则决定,源配置里的 `models[].priority` 数字不再
|
||||
参与调度、也不再显示。**聊天与生图是两套独立的链**,在优先级页面通过「聊天 / 生图」
|
||||
按钮切换编辑:
|
||||
|
||||
- **聊天链**(`runtime_file` 的 `auto` 字段):服务 `POST /v1/chat/completions`
|
||||
的 `model: AUTO` 请求;只接受 `kind: chat` 模型,配置在链里的生图槽位会被忽略。
|
||||
- **生图链**(`runtime_file` 的 `auto_image` 字段):服务 `POST /v1/images/generations`
|
||||
的 `model: AUTO` 请求;只接受 `kind: image` 模型。未配置生图链时,AUTO 生图回落
|
||||
到「所有含生图模型的源、按注册顺序」的旧行为,兼容旧配置。
|
||||
|
||||
每条规则 = 一个「槽位」:`{ model, source, tier, token_quota, period, hours }`。
|
||||
|
||||
- `tier` 表示优先级档位:同一档的模型并排、共享该优先级;档位从上到下递减。
|
||||
- 同一模型可配置多个槽位(如 A 源低配 → B 源低配 → A 源高配 → B 源高配),按档位顺延。
|
||||
- `token_quota` > 0 时该槽位在重置周期内用满即顺延到下一槽位;`period` 支持
|
||||
`hour` / `week` / `month` / `nhour`(配合 `hours`),空 = 不限。
|
||||
- 同 tier 内按「偏好分」排序,失败槽位冷却后自动跳过;冷却 / 配额耗尽 / hard
|
||||
错误会顺延到下一 tier,全部失败时返回 503 并附上每档失败摘要。
|
||||
|
||||
## 冷却与半冷却探测(自愈调度)
|
||||
|
||||
槽位失败后进入指数退避冷却(5s → 10s → 20s …,上限 **5 分钟**)。冷却不是全时段
|
||||
硬闸:
|
||||
|
||||
- **前半段完全静默**:刚失败的槽位不接任何流量,避免对故障上游持续施压。
|
||||
- **后半段放行 1 个探测**:窗口过半后,同一 `(源, 模型)` 最多允许 **一个** 请求作为
|
||||
探测通过。探测槽数 = `clamp(max_concurrent / 10, 1, 2)`——`max_concurrent: 10`
|
||||
的源恰好放行 1 个探测请求。
|
||||
- **探测排在最后**:同 tier 内探测候选永远排在健康候选之后,只有没有正常槽可用时
|
||||
才承接真实流量,因此探测不会抢走可用容量。
|
||||
- **探测即真实请求**:成功则立即清零失败计数与冷却,槽位当场回到正常轮询——
|
||||
上游恢复/额度恢复后不必等满整个冷却窗口。失败则重开一个新窗口,下次探测顺延到
|
||||
新窗口的中点,不会连续重试。
|
||||
|
||||
分类冷却策略:
|
||||
|
||||
| 情况 | 冷却 | 说明 |
|
||||
|---|---|---|
|
||||
| 传输错误 / 5xx | 5s→10s→20s…,上限 5min | 指数退避 |
|
||||
| 401 / 403 凭据错误 | 10min | 换 key 后由下一次探测自动接回 |
|
||||
| 429 限流 | 30s 固定 | 「太快」不等于「坏了」,不进指数阶梯 |
|
||||
| 配额耗尽(`insufficient_quota` / `余额不足` 等) | 对齐配额窗口,上限 30min | **不进指数阶梯、不涨失败计数**,额度恢复即可用 |
|
||||
|
||||
WebUI「优先级」页的槽位健康标签会区分 `冷却` / `待探测` / `探测中`,悬停可看窗口
|
||||
起点、探测放行时刻与完全恢复时刻。
|
||||
|
||||
## 源(Source)与适配器(Adapter)的关系
|
||||
|
||||
- **源** 是到某个上游的连接描述:`name`、`base_url`、`api_key`、模型列表与优先级。
|
||||
- **适配器** 是协议转换逻辑(Lua 脚本):把统一 OpenAI 格式请求转成上游原生格式,
|
||||
再把上游响应/流式分块转回统一格式。
|
||||
- 一个适配器可被多个源复用(如 `openai.lua` 同时服务多个 OpenAI 兼容站点);
|
||||
同一个源也可以切换不同适配器(改 `adapter` 字段即可)。
|
||||
- 源决定“连谁、暴露哪些模型”,适配器决定“怎么对话”——二者在 `sources[]` 条目中
|
||||
通过 `adapter` 字段关联。
|
||||
|
||||
加载流程(`internal/core` 的 `Core` 负责装配):
|
||||
|
||||
1. 启动时 `lua.NewVM(adapter_dir)` 加载全部适配器:内置适配器(编译期 embed)+
|
||||
`adapter_dir` 下的同名覆盖文件;内置文件写在代码内,覆盖文件要求更高优先级。
|
||||
2. `config.Load` 读取 `config.yaml`,`config.NewStore(runtime_file)` 读 WebUI 改动的
|
||||
运行时源,二者按名称合并成完整源列表。
|
||||
3. `rebuildRegistry` 为每个源创建 `provider.Provider`(持有目标适配器),并按源的
|
||||
并发上限配置适配器 worker 池**上限**(不预分配,见下)。
|
||||
4. 请求进来时 `Registry` 按 `model` 路由到 Provider,Provider 调适配器
|
||||
`transform_request` → HTTP 发送 → `transform_response` / `transform_stream_chunk`。
|
||||
|
||||
WebUI 上的"新增/编辑源"、"上传 Lua 适配器"、"改 AUTO 优先级链"与"密钥模型范围"
|
||||
都即时生效(写入运行时文件或 `adapter_dir` 后重新装配,无需重启);直接编辑
|
||||
`config.yaml` / `adapter_dir` 下的文件则需要重启进程才会重新加载。
|
||||
|
||||
在 WebUI 删除源/适配器时:运行时(WebUI 创建)的源与 `.lua` 文件会真正移除;
|
||||
`config.yaml` 中定义的基源无法改写配置文件,采用删除标记隐藏(重启后仍隐藏),
|
||||
在 UI 里重新添加同名源即可恢复。
|
||||
|
||||
## 源模板(多 key 负载均衡)
|
||||
|
||||
多个上游 key 共享同一套配置(URL / 适配器 / 模型列表 / 并发 / rpm)时,逐一复制
|
||||
整段 source 太冗余。**源模板**保存除 `name` 和 `api_key` 以外的全部源字段,
|
||||
从模板一键创建多个仅 key 不同的源,天然支持多 key 负载均衡(每个展开源独立调度、
|
||||
独立冷却、独立健康状态)。
|
||||
|
||||
- WebUI「源」页右上角「**模板管理**」按钮:列出全部模板,可编辑 / 删除 / 新建。
|
||||
- 添加源弹窗右上角两个按钮:
|
||||
- **从模板创建** → 选模板 → 表单自动填充(name + key 仍需你填);
|
||||
- **存为模板** → 把当前表单非 key/name 字段存为新模板。
|
||||
- 模板持久化在 `runtime_file` 的 `source_templates` 字段,与运行时源一样热管理,
|
||||
改完即时生效。模板本身不含密钥,可以安全共享 / 纳管。
|
||||
|
||||
模板不会自己变成源——它只是「配方」。真正承担流量的是从模板创建出来的、带 key
|
||||
的具体源。
|
||||
|
||||
23
docs/webui.md
Normal file
23
docs/webui.md
Normal file
@ -0,0 +1,23 @@
|
||||
# WebUI
|
||||
|
||||
> 本文档由 README 拆分而来。
|
||||
> 返回 [README](../README.md)。
|
||||
|
||||
## WebUI
|
||||
|
||||
内置管理界面(`GET /`),登录后可在浏览器完成:
|
||||
|
||||
- **状态页**:源在线状态(实时探测 + 悬停看错误)、模型/源/key 用量统计、请求记录,
|
||||
支持按时间范围导出 CSV;点击模型可生成 pin 到该模型的连接配置。
|
||||
- **对话页**:流式/非流式调试。
|
||||
- **密钥页**:创建/编辑网关 key,为每个 key 配模型范围(模型 + 源 + token 配额 + 周期),
|
||||
管理员管理全部 key,用户只看到自己的 key。每个模型砖块显示自己的配额徽标
|
||||
(如 `1.0K 77×·1h`,未设配额显示 `∞`),点开可编辑该模型的 token 配额、
|
||||
请求数配额与重置周期 —— 配额按模型独立生效,一个用满不影响同一 key 的其它模型。
|
||||
「我的密钥」页对用户展示本 key 的模型范围与各自配额。
|
||||
- **优先级页**:拖拽积木配置 AUTO 链档位。
|
||||
- **源页**:在线增删改上游源(API key 等敏感字段加密落盘)。
|
||||
- **适配器页**:上传 / 删除 Lua 适配器脚本。
|
||||
|
||||
改动写入 `runtime_file`(重启仍生效)。
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
// Package e2e runs the real llmsproxy binary against mocked upstreams over
|
||||
// Package e2e runs the real modelrouter binary against mocked upstreams over
|
||||
// real HTTP: startup, auth, chat, streaming SSE, image generation, and
|
||||
// AUTO failover when a higher-priority source fails.
|
||||
package e2e
|
||||
@ -100,15 +100,15 @@ type gatewayUnderTest struct {
|
||||
func buildBinary(t *testing.T) string {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
bin := filepath.Join(dir, "llmsproxy")
|
||||
bin := filepath.Join(dir, "modelrouter")
|
||||
if runtime.GOOS == "windows" {
|
||||
bin += ".exe" // go build writes the exact name; Windows needs the suffix to exec
|
||||
}
|
||||
out, err := exec.Command("go", "build", "-tags", "luajit", "-o", bin, "llmsproxy/cmd/llmsproxy").CombinedOutput()
|
||||
out, err := exec.Command("go", "build", "-tags", "luajit", "-o", bin, "modelrouter/cmd/modelrouter").CombinedOutput()
|
||||
if err != nil && runtime.GOOS == "windows" {
|
||||
// local Windows dev box may lack LuaJIT: fall back to the bundled
|
||||
// Lua runtime (the e2e adapters used here are passthrough-only)
|
||||
out, err = exec.Command("go", "build", "-o", bin, "llmsproxy/cmd/llmsproxy").CombinedOutput()
|
||||
out, err = exec.Command("go", "build", "-o", bin, "modelrouter/cmd/modelrouter").CombinedOutput()
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("build: %v\n%s", err, out)
|
||||
|
||||
@ -1,323 +0,0 @@
|
||||
package billing
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// Compile turns one billing profile into the prices table the billing plugin
|
||||
// expects, resolving URL rules against the gateway's actual sources.
|
||||
//
|
||||
// WHY URL RULES NEED RESOLVING AT ALL: the operator declares pricing by URL
|
||||
// because that is what a provider's price list is keyed on, and because several
|
||||
// sources can point at the same URL. The plugin, however, looks up by
|
||||
// `payload.source` (a source NAME) and by model — it has no idea what URL a
|
||||
// request went to. So the URL match happens here, at compile time, where the
|
||||
// config's name->base_url mapping is known, and the result is expressed in the
|
||||
// dimensions the plugin already supports.
|
||||
//
|
||||
// Profiles are the "let the user choose" axis: the same URL can appear in
|
||||
// several profiles and switching recomputes this table, so a gateway can be
|
||||
// repriced without editing the plugin.
|
||||
func Compile(profile *config.BillingProfile, sources []config.Source) (map[string]interface{}, error) {
|
||||
return CompileOpts(profile, sources, false)
|
||||
}
|
||||
|
||||
// CompileOpts is Compile with an explicit policy for rules that match no
|
||||
// source.
|
||||
//
|
||||
// The default is STRICT and that is correct for startup: a profile with a
|
||||
// typo'd URL prices nothing, every request on it is recorded as unpriced, and
|
||||
// the bill silently comes out wrong. Failing the load is the right response.
|
||||
//
|
||||
// The rules editor needs the opposite. An operator editing a half-finished
|
||||
// profile — adding a rule before the source that will use it exists, or
|
||||
// renaming a URL while typing — must be able to save and then see the warning,
|
||||
// not be blocked by an error they can only resolve by guessing. Lenient mode
|
||||
// keeps the offending rule in the config and drops it from the compiled table,
|
||||
// so it is preserved as text and clearly reported, without pretending it
|
||||
// prices anything.
|
||||
func CompileOpts(profile *config.BillingProfile, sources []config.Source, lenient bool) (map[string]interface{}, error) {
|
||||
if profile == nil {
|
||||
return nil, fmt.Errorf("no billing profile")
|
||||
}
|
||||
prices := map[string]interface{}{
|
||||
"currency": "USD",
|
||||
"default": map[string]interface{}{"prompt": 0.0, "completion": 0.0, "per_request": 0.0},
|
||||
"sources": map[string]interface{}{},
|
||||
"models": map[string]interface{}{},
|
||||
"keys": map[string]interface{}{},
|
||||
}
|
||||
if profile.Default != "" {
|
||||
prices["default_mode"] = profile.Default
|
||||
}
|
||||
|
||||
// Currency is a profile-level statement; a rule may override it.
|
||||
for i := range profile.Rules {
|
||||
if c := profile.Rules[i].Currency; c != "" {
|
||||
prices["currency"] = c
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
matched := map[string]bool{}
|
||||
for i := range profile.Rules {
|
||||
rule := &profile.Rules[i]
|
||||
targets := matchSources(rule.URL, sources)
|
||||
if len(targets) == 0 && rule.URL != "*" {
|
||||
// A rule for a URL no source uses is almost always a typo (or a
|
||||
// source that was removed). Failing loudly beats a profile that
|
||||
// silently prices nothing — the whole reason this is a config file
|
||||
// instead of a hand-written JSON blob.
|
||||
if lenient {
|
||||
continue // kept in the config text, reported as a warning
|
||||
}
|
||||
return nil, fmt.Errorf("rule url %q matches no configured source base_url", rule.URL)
|
||||
}
|
||||
for _, srcName := range targets {
|
||||
if matched[srcName] {
|
||||
// First rule wins. Two rules matching one source is ambiguous,
|
||||
// and silently letting the later one win makes the file's
|
||||
// meaning depend on ordering the operator cannot see.
|
||||
continue
|
||||
}
|
||||
matched[srcName] = true
|
||||
if err := applyRule(prices, rule, srcName, sources); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Sources no rule matched fall through to the profile default, which the
|
||||
// plugin applies via prices.default. Recording them explicitly means the
|
||||
// UI can say "this source is unpriced" instead of leaving the operator to
|
||||
// infer it from a zero.
|
||||
var unmatched []string
|
||||
for _, src := range sources {
|
||||
if !matched[src.Name] {
|
||||
unmatched = append(unmatched, src.Name)
|
||||
}
|
||||
}
|
||||
if len(unmatched) > 0 {
|
||||
prices["unmatched_sources"] = unmatched
|
||||
}
|
||||
return prices, nil
|
||||
}
|
||||
|
||||
// matchSources returns the source names whose base_url matches pattern.
|
||||
// "*" matches every source (used as a catch-all default rule).
|
||||
func matchSources(pattern string, sources []config.Source) []string {
|
||||
var out []string
|
||||
if pattern == "*" {
|
||||
for _, s := range sources {
|
||||
out = append(out, s.Name)
|
||||
}
|
||||
return out
|
||||
}
|
||||
want := normalizeURL(pattern)
|
||||
for _, s := range sources {
|
||||
if normalizeURL(s.BaseURL) == want {
|
||||
out = append(out, s.Name)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// normalizeURL compares URLs the way an operator expects: trailing slashes and
|
||||
// case in the host are differences the provider's price list does not care
|
||||
// about, and requiring an exact byte match would make the config brittle.
|
||||
func normalizeURL(u string) string {
|
||||
u = strings.TrimSpace(u)
|
||||
u = strings.TrimRight(u, "/")
|
||||
return strings.ToLower(u)
|
||||
}
|
||||
|
||||
// applyRule writes one rule's pricing for one source into the prices table.
|
||||
func applyRule(prices map[string]interface{}, rule *config.BillingRule, srcName string, sources []config.Source) error {
|
||||
switch rule.Mode {
|
||||
case "free":
|
||||
// Explicitly priced at zero. This is NOT the same as unpriced: a source
|
||||
// the operator says is free must not appear in the unpriced warnings,
|
||||
// or those warnings become noise and stop being read.
|
||||
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
|
||||
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
|
||||
}
|
||||
return nil
|
||||
|
||||
case "subscription":
|
||||
// A fixed monthly commitment: the per-request MARGINAL cost is zero, and
|
||||
// the flat fee is reported separately. Spreading a monthly fee across
|
||||
// requests would invent a per-request number the provider never charges,
|
||||
// and it would change every time traffic did.
|
||||
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
|
||||
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
|
||||
}
|
||||
fixed, _ := prices["subscriptions"].(map[string]interface{})
|
||||
if fixed == nil {
|
||||
fixed = map[string]interface{}{}
|
||||
prices["subscriptions"] = fixed
|
||||
}
|
||||
cur := "USD"
|
||||
if rule.Currency != "" {
|
||||
cur = rule.Currency
|
||||
}
|
||||
fixed[srcName] = map[string]interface{}{
|
||||
"monthly": rule.Subscription, "currency": cur,
|
||||
}
|
||||
return nil
|
||||
|
||||
case "unpriced":
|
||||
// Deliberately left out of `sources` so the plugin's unpriced_models /
|
||||
// unpriced_reqs counters catch it. That is the point: a subscription
|
||||
// plan whose credits cannot be converted to tokens must be VISIBLE as
|
||||
// unbilled, not quietly estimated.
|
||||
return nil
|
||||
|
||||
case "token":
|
||||
if len(rule.Models) == 0 {
|
||||
return fmt.Errorf("rule for url %q: mode token requires at least one model", rule.URL)
|
||||
}
|
||||
models := prices["models"].(map[string]interface{})
|
||||
for modelID, t := range rule.Models {
|
||||
entry := map[string]interface{}{}
|
||||
prompt, err := usdPerM(t.Prompt)
|
||||
if err != nil {
|
||||
return fmt.Errorf("model %q prompt: %w", modelID, err)
|
||||
}
|
||||
completion, err := usdPerM(t.Completion)
|
||||
if err != nil {
|
||||
return fmt.Errorf("model %q completion: %w", modelID, err)
|
||||
}
|
||||
entry["prompt"] = prompt
|
||||
entry["completion"] = completion
|
||||
if t.CacheDiscount != nil {
|
||||
entry["cache_discount"] = *t.CacheDiscount
|
||||
}
|
||||
// reasoning is optional and stays ABSENT (not nil, not 0) when the
|
||||
// rule does not set it, which is what makes the billing plugin skip
|
||||
// the repricing branch entirely. Writing an explicit 0 here would
|
||||
// mean "thinking is free" and silently under-bill every request
|
||||
// from a provider that bills it like ordinary output.
|
||||
if strings.TrimSpace(t.Reasoning) != "" {
|
||||
rp, err := usdPerM(t.Reasoning)
|
||||
if err != nil {
|
||||
return fmt.Errorf("model %q reasoning: %w", modelID, err)
|
||||
}
|
||||
entry["reasoning_price"] = rp
|
||||
}
|
||||
if lc := t.LongContext; lc != nil {
|
||||
if lc.Above <= 0 {
|
||||
return fmt.Errorf("model %q long_context: above must be > 0", modelID)
|
||||
}
|
||||
above, err := longContextRate(lc, prompt)
|
||||
if err != nil {
|
||||
return fmt.Errorf("model %q long_context: %w", modelID, err)
|
||||
}
|
||||
entry["long_context"] = map[string]interface{}{
|
||||
"above": lc.Above,
|
||||
// Marginal, not blended. The plugin charges the tokens past the
|
||||
// threshold at this rate and the rest at the base rate, so the
|
||||
// threshold composes with a cache discount (which applies to the
|
||||
// cached portion) instead of forcing the plugin to guess how
|
||||
// the two interact.
|
||||
"prompt": above,
|
||||
}
|
||||
}
|
||||
if rule.Peak != nil {
|
||||
entry["peak"] = compilePeak(rule.Peak)
|
||||
}
|
||||
models[modelID] = entry
|
||||
}
|
||||
// A specific-URL token rule needs a source entry so a model served
|
||||
// from THIS url but absent from `rule.Models` is still recognised as
|
||||
// belonging to a priced provider.
|
||||
//
|
||||
// A WILDCARD rule ("*") must NOT get one. "*" is a model catalogue — it
|
||||
// says "these model ids have these prices, wherever they come from" —
|
||||
// and marking every source priced at {0,0} would make priceFor() set
|
||||
// priced=true for every model the catalogue does NOT list. Paid
|
||||
// sources (commandcode, alittokenplan, …) would then record $0 instead
|
||||
// of showing up in unpriced_reqs, which is precisely the silent
|
||||
// zero-billing failure this whole plugin is built to make visible.
|
||||
// The model lookup p.models[payload.model] already marks a listed
|
||||
// model priced on its own, so the wildcard loses nothing.
|
||||
if rule.URL != "*" {
|
||||
if _, ok := prices["sources"].(map[string]interface{})[srcName]; !ok {
|
||||
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
|
||||
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("unknown mode %q", rule.Mode)
|
||||
}
|
||||
|
||||
// usdPerM converts a USD-per-million string into the per-token rate the plugin
|
||||
// expects. Delegates to config.ParseFloatUSDPerM so the DSL validator and the
|
||||
// compiler agree byte-for-byte on what a valid price is — a value that passes
|
||||
// Validate() but fails here (or vice versa) would be the worst kind of drift.
|
||||
func usdPerM(s string) (float64, error) {
|
||||
return config.ParseFloatUSDPerM(s)
|
||||
}
|
||||
|
||||
// compilePeak renders the peak window in the shape the plugin reads:
|
||||
// { multiplier, windows = { { days = {...}, hours = { {lo,hi}, ... } } } }.
|
||||
//
|
||||
// The plugin reads `days` / `hours` pairs. The first version of the deployed
|
||||
// price table used { start, end, weekdays } — a shape nothing reads — so peak
|
||||
// traffic was billed at off-peak rates with no error anywhere. Compiling from
|
||||
// typed config fields removes the chance of writing the wrong key names by hand.
|
||||
func compilePeak(p *config.BillingPeak) map[string]interface{} {
|
||||
win := map[string]interface{}{}
|
||||
if len(p.Weekdays) > 0 {
|
||||
days := make([]interface{}, 0, len(p.Weekdays))
|
||||
for _, d := range p.Weekdays {
|
||||
days = append(days, d)
|
||||
}
|
||||
win["days"] = days
|
||||
}
|
||||
if len(p.Hours) > 0 {
|
||||
hours := make([]interface{}, 0, len(p.Hours))
|
||||
for _, h := range p.Hours {
|
||||
hours = append(hours, []interface{}{h[0], h[1]})
|
||||
}
|
||||
win["hours"] = hours
|
||||
}
|
||||
out := map[string]interface{}{"multiplier": p.Multiplier}
|
||||
if len(win) > 0 {
|
||||
out["windows"] = []interface{}{win}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// longContextRate resolves the marginal per-token rate for the tokens ABOVE a
|
||||
// threshold, given the model's base per-million prompt price.
|
||||
//
|
||||
// Two ways to express it are supported because providers publish both: an
|
||||
// absolute rate (OpenAI quotes \$3.00/M past 200k) and a multiple (DeepSeek
|
||||
// quotes exactly 2x past 128k). Neither may be missing: a step with no rate is a
|
||||
// silent no-op, and an operator who wrote one meant something.
|
||||
func longContextRate(lc *config.BillingLongContext, basePerToken float64) (float64, error) {
|
||||
hasPrompt := strings.TrimSpace(lc.Prompt) != ""
|
||||
hasMult := lc.Multiplier > 0
|
||||
switch {
|
||||
case hasPrompt && hasMult:
|
||||
p, err := usdPerM(lc.Prompt)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
// Both given: the absolute rate wins, because it is the one the provider
|
||||
// actually quotes. Silently multiplying instead would produce a number
|
||||
// the operator never wrote.
|
||||
return p, nil
|
||||
case hasPrompt:
|
||||
return usdPerM(lc.Prompt)
|
||||
case hasMult:
|
||||
return basePerToken * lc.Multiplier, nil
|
||||
default:
|
||||
return 0, fmt.Errorf("needs either prompt or multiplier")
|
||||
}
|
||||
}
|
||||
@ -1,214 +0,0 @@
|
||||
package billing
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// The wildcard token rule must not blanket-mark every source as priced.
|
||||
//
|
||||
// Why this is a hazard and not a style question: priceFor() sets priced=true as
|
||||
// soon as the request's SOURCE has an entry in the price table, and that flag
|
||||
// is what keeps a request OUT of unpriced_reqs. A "*" rule is a model
|
||||
// catalogue — it prices specific model ids wherever they come from — but
|
||||
// compiling it into a {0,0,0} entry on all 21 sources made every model the
|
||||
// catalogue does not list look "priced at zero". Paid providers (commandcode,
|
||||
// alittokenplan, …) then stopped appearing in the unpriced warnings, which is
|
||||
// exactly the silent zero-billing failure the plugin exists to surface.
|
||||
func TestWildcardTokenRuleDoesNotMarkEverySourcePriced(t *testing.T) {
|
||||
sources := []config.Source{
|
||||
{Name: "free1", BaseURL: "https://free1.example.com/v1"},
|
||||
{Name: "paid1", BaseURL: "https://paid1.example.com/v1"},
|
||||
{Name: "paid2", BaseURL: "https://paid2.example.com/v1"},
|
||||
}
|
||||
profile := &config.BillingProfile{
|
||||
ID: "p",
|
||||
Rules: []config.BillingRule{
|
||||
{URL: "https://free1.example.com/v1", Mode: "free"},
|
||||
{
|
||||
URL: "*", Mode: "token",
|
||||
Models: map[string]config.BillingToken{
|
||||
"only-catalogued-model": {Prompt: "1.00", Completion: "2.00"},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
prices, err := Compile(profile, sources)
|
||||
if err != nil {
|
||||
t.Fatalf("compile: %v", err)
|
||||
}
|
||||
srcs := prices["sources"].(map[string]interface{})
|
||||
|
||||
// The explicitly free source keeps its entry (free is a deliberate 0).
|
||||
if _, ok := srcs["free1"]; !ok {
|
||||
t.Error("the explicit free rule lost its source entry")
|
||||
}
|
||||
// Paid sources must NOT be marked, or their unlisted models bill $0.
|
||||
for _, name := range []string{"paid1", "paid2"} {
|
||||
if _, ok := srcs[name]; ok {
|
||||
t.Errorf("wildcard token rule marked %q as priced at 0 — its unlisted "+
|
||||
"models will bill $0 instead of appearing as unpriced", name)
|
||||
}
|
||||
}
|
||||
// The catalogue itself must still be present, or the rule prices nothing.
|
||||
models := prices["models"].(map[string]interface{})
|
||||
if _, ok := models["only-catalogued-model"]; !ok {
|
||||
t.Error("the wildcard rule did not publish its model price")
|
||||
}
|
||||
}
|
||||
|
||||
// The inverse guard: a token rule for a SPECIFIC url still marks that source,
|
||||
// because it genuinely declares pricing for the models served from it.
|
||||
func TestSpecificURLTokenRuleStillMarksItsSource(t *testing.T) {
|
||||
sources := []config.Source{
|
||||
{Name: "mine", BaseURL: "https://mine.example.com/v1"},
|
||||
{Name: "other", BaseURL: "https://other.example.com/v1"},
|
||||
}
|
||||
profile := &config.BillingProfile{
|
||||
ID: "p",
|
||||
Rules: []config.BillingRule{{
|
||||
URL: "https://mine.example.com/v1", Mode: "token",
|
||||
Models: map[string]config.BillingToken{
|
||||
"m": {Prompt: "1.00", Completion: "2.00"},
|
||||
},
|
||||
}},
|
||||
}
|
||||
prices, err := Compile(profile, sources)
|
||||
if err != nil {
|
||||
t.Fatalf("compile: %v", err)
|
||||
}
|
||||
srcs := prices["sources"].(map[string]interface{})
|
||||
if _, ok := srcs["mine"]; !ok {
|
||||
t.Error("a specific-url token rule must still mark its own source priced")
|
||||
}
|
||||
if _, ok := srcs["other"]; ok {
|
||||
t.Error("an unrelated source was marked priced by a specific-url rule")
|
||||
}
|
||||
}
|
||||
|
||||
// TestReasoningPriceIsOmittedUnlessSet pins the absence convention for the
|
||||
// thinking-token rate.
|
||||
//
|
||||
// A rule that says nothing about reasoning must produce a price entry with NO
|
||||
// reasoning_price key, so the billing plugin skips the repricing branch and
|
||||
// bills thinking at the ordinary output rate — which is what every provider
|
||||
// actually charges. Writing an explicit 0 instead would read as "thinking is
|
||||
// free" and silently under-bill every request from that model.
|
||||
func TestReasoningPriceIsOmittedUnlessSet(t *testing.T) {
|
||||
sources := []config.Source{{Name: "up", BaseURL: "https://up.example/v1"}}
|
||||
|
||||
// Case 1: no reasoning declared → the key must not appear.
|
||||
p1 := &config.BillingProfile{ID: "p", Rules: []config.BillingRule{{
|
||||
URL: "https://up.example/v1", Mode: "token",
|
||||
Models: map[string]config.BillingToken{
|
||||
"m": {Prompt: "1.00", Completion: "2.00"},
|
||||
},
|
||||
}}}
|
||||
prices, err := Compile(p1, sources)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
entry := prices["models"].(map[string]interface{})["m"].(map[string]interface{})
|
||||
if _, present := entry["reasoning_price"]; present {
|
||||
t.Errorf("reasoning_price present (%v) without a rule; the plugin would treat 0 as free",
|
||||
entry["reasoning_price"])
|
||||
}
|
||||
|
||||
// Case 2: declared → present, and converted to per-token like every other rate.
|
||||
p2 := &config.BillingProfile{ID: "p", Rules: []config.BillingRule{{
|
||||
URL: "https://up.example/v1", Mode: "token",
|
||||
Models: map[string]config.BillingToken{
|
||||
"m": {Prompt: "1.00", Completion: "2.00", Reasoning: "6.00"},
|
||||
},
|
||||
}}}
|
||||
prices2, err := Compile(p2, sources)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
e2 := prices2["models"].(map[string]interface{})["m"].(map[string]interface{})
|
||||
rp, ok := e2["reasoning_price"].(float64)
|
||||
if !ok {
|
||||
t.Fatalf("reasoning_price missing or wrong type: %#v", e2["reasoning_price"])
|
||||
}
|
||||
if rp != 6e-6 {
|
||||
t.Errorf("reasoning_price = %v, want 6e-6 (USD per TOKEN)", rp)
|
||||
}
|
||||
// A malformed rate must be rejected, not silently ignored: an operator who
|
||||
// typed "6 USD" expects either that value or an error, never a rule that
|
||||
// looks applied and bills nothing.
|
||||
p3 := &config.BillingProfile{ID: "p", Rules: []config.BillingRule{{
|
||||
URL: "https://up.example/v1", Mode: "token",
|
||||
Models: map[string]config.BillingToken{
|
||||
"m": {Prompt: "1.00", Completion: "2.00", Reasoning: "six dollars"},
|
||||
},
|
||||
}}}
|
||||
if _, err := Compile(p3, sources); err == nil {
|
||||
t.Error("a non-numeric reasoning rate must fail the compile")
|
||||
}
|
||||
}
|
||||
|
||||
// TestLongContextStepCompiles pins the long-context step's two accepted forms
|
||||
// and its rejection of a step that would silently do nothing.
|
||||
//
|
||||
// Providers publish both shapes — DeepSeek says "2x past 128k", OpenAI quotes
|
||||
// an absolute "$3.00/M past 200k" — so both must compile. Neither may be
|
||||
// missing: a step with no rate is a rule that LOOKS like it handles long
|
||||
// context and does not, which under-bills without any visible sign.
|
||||
func TestLongContextStepCompiles(t *testing.T) {
|
||||
sources := []config.Source{{Name: "up", BaseURL: "https://up.example/v1"}}
|
||||
|
||||
compile := func(lc config.BillingLongContext) (map[string]interface{}, error) {
|
||||
p := &config.BillingProfile{ID: "p", Rules: []config.BillingRule{{
|
||||
URL: "https://up.example/v1", Mode: "token",
|
||||
Models: map[string]config.BillingToken{
|
||||
"m": {Prompt: "1.00", Completion: "2.00", LongContext: &lc},
|
||||
},
|
||||
}}}
|
||||
prices, err := Compile(p, sources)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
e := prices["models"].(map[string]interface{})["m"].(map[string]interface{})
|
||||
return e["long_context"].(map[string]interface{}), nil
|
||||
}
|
||||
|
||||
// Absolute rate (OpenAI's published shape).
|
||||
lc1, err := compile(config.BillingLongContext{Above: 200000, Prompt: "3.00"})
|
||||
if err != nil {
|
||||
t.Fatalf("absolute long_context rate rejected: %v", err)
|
||||
}
|
||||
if lc1["above"] != 200000 || lc1["prompt"] != 3e-6 {
|
||||
t.Errorf("absolute step = %#v, want above=200000 prompt=3e-6", lc1)
|
||||
}
|
||||
|
||||
// Multiplier (DeepSeek's shape): 2x of the 1.00/M base.
|
||||
lc2, err := compile(config.BillingLongContext{Above: 128000, Multiplier: 2})
|
||||
if err != nil {
|
||||
t.Fatalf("multiplier long_context rejected: %v", err)
|
||||
}
|
||||
if lc2["prompt"] != 2e-6 {
|
||||
t.Errorf("multiplier step = %#v, want prompt=2e-6", lc2)
|
||||
}
|
||||
|
||||
// Both given: the absolute rate wins, because that is the number the
|
||||
// provider quotes. Multiplying instead would produce a figure the operator
|
||||
// never wrote.
|
||||
lc3, err := compile(config.BillingLongContext{Above: 128000, Prompt: "5.00", Multiplier: 2})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if lc3["prompt"] != 5e-6 {
|
||||
t.Errorf("both given = %#v, want the quoted absolute rate 5e-6", lc3)
|
||||
}
|
||||
|
||||
// Neither: a no-op step must fail, not compile into a rule that silently
|
||||
// bills long context at the base rate.
|
||||
if _, err := compile(config.BillingLongContext{Above: 128000}); err == nil {
|
||||
t.Error("a step with neither prompt nor multiplier must be rejected")
|
||||
}
|
||||
// A zero threshold would make every request "long".
|
||||
if _, err := compile(config.BillingLongContext{Above: 0, Multiplier: 2}); err == nil {
|
||||
t.Error("above=0 must be rejected")
|
||||
}
|
||||
}
|
||||
@ -7,7 +7,6 @@ import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@ -30,15 +29,6 @@ type Config struct {
|
||||
Auto []ModelScope `yaml:"auto,omitempty"` // AUTO 调度链规则(WebUI 优先级页编辑,chat)
|
||||
AutoImage []ModelScope `yaml:"auto_image,omitempty"` // AUTO 生图调度链规则(WebUI 优先级页·生图)
|
||||
Keys []GWKey `yaml:"keys,omitempty"` // 网关密钥(WebUI 密钥页管理)
|
||||
// BillingDSL declares per-URL pricing profiles for the billing plugin.
|
||||
//
|
||||
// Declarative, because hand-writing the prices JSON was error-prone in ways
|
||||
// that looked successful: the first PUT went out without the required
|
||||
// `prices` wrapper and silently REPLACED the accumulated totals with the
|
||||
// price table; the second used a peak-window shape the plugin does not read,
|
||||
// so peak traffic was billed at off-peak rates — no error, the bill just
|
||||
// quietly halved. A schema-checked config file catches both at load.
|
||||
BillingDSL *BillingDSL `yaml:"billing,omitempty"`
|
||||
// PluginVisibility overrides the audience a plugin declared for itself,
|
||||
// keyed by plugin name: "admin" hides a plugin (and its state) from user
|
||||
// keys, "all" widens one that shipped admin-only, "user,admin" names roles
|
||||
@ -62,13 +52,18 @@ const (
|
||||
DefaultSourceConcurrency = 8
|
||||
)
|
||||
|
||||
// Model is a single exposed model id bound to a source, with priority used by
|
||||
// AUTO auto selection (higher number = preferred).
|
||||
// Model is a single exposed model id bound to a source.
|
||||
//
|
||||
// ★ 曾经有 `Priority int` 字段,并号称"高者优先参与 AUTO 选源"。**那是错的**:
|
||||
//
|
||||
// AUTO 链由 `config.ModelScope.Tier` 驱动(见 scheduler.Chain),而 priority
|
||||
// 只被 `bestChatModel/bestImageModel` 用于"本源码内部该发哪个模型",
|
||||
// 与跨源调度毫无关系。把它当成调度依据会让人误以为改个数字就能改顺序,
|
||||
// 改完发现毫无效果 —— 已彻底删除,避免再次误导。
|
||||
type Model struct {
|
||||
ID string `yaml:"id" json:"id"`
|
||||
Priority int `yaml:"priority" json:"priority"`
|
||||
Kind string `yaml:"kind" json:"kind"` // "chat" (default) | "image"
|
||||
Meta map[string]interface{} `yaml:"meta" json:"meta,omitempty"`
|
||||
ID string `yaml:"id" json:"id"`
|
||||
Kind string `yaml:"kind" json:"kind"` // "chat" (default) | "image"
|
||||
Meta map[string]interface{} `yaml:"meta" json:"meta,omitempty"`
|
||||
}
|
||||
|
||||
// Source describes a single upstream LLM provider.
|
||||
@ -90,7 +85,14 @@ type Source struct {
|
||||
// ProxyURL routes this source's HTTP(S) traffic through an explicit proxy
|
||||
// (e.g. http://127.0.0.1:7890). Empty = direct connection. Needed for
|
||||
// upstreams behind geo/IP blocks that only respond through a proxy.
|
||||
ProxyURL string `yaml:"proxy_url" json:"proxy_url,omitempty"`
|
||||
ProxyURL string `yaml:"proxy_url" json:"proxy_url,omitempty"`
|
||||
// Retry is an OPTIONAL upstream retry policy. It exists for upstreams that
|
||||
// answer a business-level rejection with a status code the kernel would
|
||||
// otherwise surface to the client — e.g. a per-account concurrency cap that
|
||||
// clears by itself after tens of seconds. The kernel stays vendor-agnostic:
|
||||
// the matcher is pure DATA (status list + response-body substrings), so no
|
||||
// provider name ever appears in core.
|
||||
Retry *RetryPolicy `yaml:"retry" json:"retry,omitempty"`
|
||||
Meta map[string]interface{} `yaml:"meta" json:"meta,omitempty"`
|
||||
Temperature float64 `yaml:"temperature" json:"temperature,omitempty"`
|
||||
MaxTokens int `yaml:"max_tokens" json:"max_tokens,omitempty"`
|
||||
@ -100,9 +102,66 @@ type Source struct {
|
||||
RPM int `yaml:"rpm" json:"rpm,omitempty"` // optional requests-per-minute cap (0 = unlimited)
|
||||
}
|
||||
|
||||
// RetryPolicy describes when a failed upstream attempt may be retried BEFORE
|
||||
// anything is returned to the client. All conditions are data:
|
||||
//
|
||||
// OnStatus - retry only when the HTTP status is one of these
|
||||
// OnBodyContains - ... AND the response body contains one of these substrings
|
||||
// (empty = status alone decides)
|
||||
// Delay - wait between attempts ("10s" etc.); parsed on demand
|
||||
// MaxAttempts - TOTAL attempts including the first (1 or 0 = no retry)
|
||||
//
|
||||
// Scope: the retry happens inside one gateway request, so a stream is only
|
||||
// retried before its first chunk reaches the client (the kernel already holds
|
||||
// the first chunk back), and a non-stream call is retried as a whole.
|
||||
type RetryPolicy struct {
|
||||
OnStatus []int `yaml:"on_status,omitempty" json:"on_status,omitempty"`
|
||||
OnBodyContains []string `yaml:"on_body_contains,omitempty" json:"on_body_contains,omitempty"`
|
||||
Delay string `yaml:"delay,omitempty" json:"delay,omitempty"`
|
||||
MaxAttempts int `yaml:"max_attempts,omitempty" json:"max_attempts,omitempty"`
|
||||
}
|
||||
|
||||
// ShouldRetry reports whether one failed attempt matches this policy.
|
||||
func (r *RetryPolicy) ShouldRetry(status int, body string) bool {
|
||||
if r == nil || r.MaxAttempts <= 1 {
|
||||
return false
|
||||
}
|
||||
statusOK := len(r.OnStatus) == 0
|
||||
for _, s := range r.OnStatus {
|
||||
if s == status {
|
||||
statusOK = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !statusOK {
|
||||
return false
|
||||
}
|
||||
if len(r.OnBodyContains) == 0 {
|
||||
return true
|
||||
}
|
||||
for _, sub := range r.OnBodyContains {
|
||||
if sub != "" && strings.Contains(body, sub) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// DelayDuration is the configured pause between attempts (default 10s).
|
||||
func (r *RetryPolicy) DelayDuration() time.Duration {
|
||||
if r == nil || r.Delay == "" {
|
||||
return 10 * time.Second
|
||||
}
|
||||
d, err := time.ParseDuration(r.Delay)
|
||||
if err != nil || d < 0 {
|
||||
return 10 * time.Second
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
// Load reads and validates a config file. When the file does not exist yet a
|
||||
// default config is generated at that path first (first-run bootstrap), so a
|
||||
// fresh binary just works: `llmsproxy -config /path/to/config.yaml`.
|
||||
// fresh binary just works: `modelrouter -config /path/to/config.yaml`.
|
||||
func Load(path string) (*Config, error) {
|
||||
if _, err := EnsureDefault(path); err != nil {
|
||||
return nil, err
|
||||
@ -418,182 +477,6 @@ type GWKey struct {
|
||||
// chain" so a user can clear an override back to the global default.
|
||||
func (k GWKey) HasOwnAuto() bool { return len(k.Auto) > 0 }
|
||||
|
||||
// BillingDSL holds declarative per-URL pricing profiles for the billing plugin.
|
||||
//
|
||||
// Profiles are the "let the user pick" axis: the same upstream URL can be
|
||||
// declared under multiple profiles, and the operator chooses which is active.
|
||||
// Switching recomputes and re-injects the billing plugin's prices table.
|
||||
type BillingDSL struct {
|
||||
// Active is the id of the profile in effect. Empty => the first profile.
|
||||
Active string `yaml:"active,omitempty" json:"active,omitempty"`
|
||||
Profiles []BillingProfile `yaml:"profiles,omitempty" json:"profiles,omitempty"`
|
||||
}
|
||||
|
||||
// BillingProfile is one named set of per-URL pricing rules.
|
||||
type BillingProfile struct {
|
||||
ID string `yaml:"id" json:"id"`
|
||||
Label string `yaml:"label,omitempty" json:"label,omitempty"`
|
||||
// Default applies when no rule matches: "free" or "unpriced".
|
||||
Default string `yaml:"default,omitempty" json:"default,omitempty"`
|
||||
Rules []BillingRule `yaml:"rules,omitempty" json:"rules,omitempty"`
|
||||
}
|
||||
|
||||
// BillingRule declares the pricing for everything under one URL.
|
||||
type BillingRule struct {
|
||||
// URL matches a source's base_url. "*" = any unmatched URL.
|
||||
URL string `yaml:"url" json:"url"`
|
||||
// Mode is one of: free, token, subscription, unpriced.
|
||||
Mode string `yaml:"mode" json:"mode"`
|
||||
Currency string `yaml:"currency,omitempty" json:"currency,omitempty"`
|
||||
// Subscription is the flat monthly cost (for mode=subscription).
|
||||
Subscription float64 `yaml:"subscription,omitempty" json:"subscription,omitempty"`
|
||||
// Models declares per-token prices when mode=token.
|
||||
Models map[string]BillingToken `yaml:"models,omitempty" json:"models,omitempty"`
|
||||
// Free is true when mode=free (kept for explicitness).
|
||||
// Peak applies to ALL token-priced requests under this URL.
|
||||
Peak *BillingPeak `yaml:"peak,omitempty" json:"peak,omitempty"`
|
||||
}
|
||||
|
||||
// BillingToken is the per-model price.
|
||||
type BillingToken struct {
|
||||
Prompt string `yaml:"prompt" json:"prompt"` // USD per million tokens, e.g. "0.15"
|
||||
Completion string `yaml:"completion" json:"completion"` // USD per million tokens
|
||||
CacheDiscount *float64 `yaml:"cache_discount,omitempty" json:"cache_discount,omitempty"`
|
||||
// Reasoning is the USD-per-million rate for thinking tokens, for the rare
|
||||
// provider that charges them differently from ordinary output. Omit it and
|
||||
// reasoning is billed at Completion — which is what OpenAI, DeepSeek and
|
||||
// Anthropic all do, and what every source on this deployment reports.
|
||||
//
|
||||
// It is a rate for a SUBSET of completion tokens, never an extra charge: the
|
||||
// billing plugin applies only the difference between this and Completion, so
|
||||
// setting it to a multiple of Completion reprices the thinking share rather
|
||||
// than billing it twice.
|
||||
Reasoning string `yaml:"reasoning,omitempty" json:"reasoning,omitempty"`
|
||||
// LongContext is the prompt-price step above a token threshold. Nil = one
|
||||
// flat rate, which is what a rule means when it says nothing.
|
||||
LongContext *BillingLongContext `yaml:"long_context,omitempty" json:"long_context,omitempty"`
|
||||
}
|
||||
|
||||
// BillingPeak declares peak-hour pricing.
|
||||
type BillingPeak struct {
|
||||
Multiplier float64 `yaml:"multiplier" json:"multiplier"`
|
||||
// Weekdays: 1=Mon..5=Fri (Lua os.date !%w, 0=Sun)
|
||||
Weekdays []int `yaml:"weekdays,omitempty" json:"weekdays,omitempty"`
|
||||
// Hours: each pair is [lo,hi] inclusive
|
||||
Hours [][2]int `yaml:"hours,omitempty" json:"hours,omitempty"`
|
||||
}
|
||||
|
||||
// BillingLongContext is a prompt-price step for long inputs.
|
||||
//
|
||||
// Every major provider charges the input tokens ABOVE a threshold at a higher
|
||||
// rate — DeepSeek doubles past 128k, OpenAI and Anthropic step up as well. A
|
||||
// flat prompt price therefore under-bills exactly the traffic an agent workload
|
||||
// generates, and it under-bills it silently: the number is plausible, the bill
|
||||
// adds up, and nothing says the rate was wrong.
|
||||
//
|
||||
// The step applies to the PROMPT only. Output and cache reads are billed at
|
||||
// their own rates regardless of input length, which is how providers publish
|
||||
// it — the higher rate is for the extra context you made the model read, not for
|
||||
// what it wrote back.
|
||||
type BillingLongContext struct {
|
||||
// Above is the token count the step starts at. A request with exactly this
|
||||
// many prompt tokens is still at the base rate; the step covers the tokens
|
||||
// BEYOND it. Inclusive-below matches how the providers document it.
|
||||
Above int `yaml:"above" json:"above"`
|
||||
// Prompt is the USD-per-million rate for the tokens above Above.
|
||||
Prompt string `yaml:"prompt" json:"prompt"`
|
||||
// Multiplier is an alternative to Prompt: a factor applied to the base rate
|
||||
// (DeepSeek's published step is exactly 2x). When both are set Prompt wins,
|
||||
// and setting neither is a validation error rather than a silent no-op.
|
||||
Multiplier float64 `yaml:"multiplier,omitempty" json:"multiplier,omitempty"`
|
||||
}
|
||||
|
||||
// Validate rejects a BillingDSL that could not work as written.
|
||||
func (b *BillingDSL) Validate() error {
|
||||
if b == nil {
|
||||
return nil
|
||||
}
|
||||
ids := map[string]bool{}
|
||||
for i := range b.Profiles {
|
||||
p := &b.Profiles[i]
|
||||
if p.ID == "" {
|
||||
return fmt.Errorf("billing profile [%d] has no id", i)
|
||||
}
|
||||
if ids[p.ID] {
|
||||
return fmt.Errorf("billing profile id %q appears more than once", p.ID)
|
||||
}
|
||||
ids[p.ID] = true
|
||||
if p.Default != "" && p.Default != "free" && p.Default != "unpriced" {
|
||||
return fmt.Errorf("profile %q: default must be free or unpriced, got %q", p.ID, p.Default)
|
||||
}
|
||||
for j := range p.Rules {
|
||||
r := &p.Rules[j]
|
||||
if r.URL == "" {
|
||||
return fmt.Errorf("profile %q rule [%d]: url is required", p.ID, j)
|
||||
}
|
||||
switch r.Mode {
|
||||
case "free", "token", "subscription", "unpriced":
|
||||
default:
|
||||
return fmt.Errorf("profile %q rule [%d]: mode must be free/token/subscription/unpriced, got %q", p.ID, j, r.Mode)
|
||||
}
|
||||
if r.Mode == "token" {
|
||||
for m, t := range r.Models {
|
||||
if t.Prompt == "" || t.Completion == "" {
|
||||
return fmt.Errorf("profile %q model %q: prompt and completion are required for token mode", p.ID, m)
|
||||
}
|
||||
if _, err := ParseFloatUSDPerM(t.Prompt); err != nil {
|
||||
return fmt.Errorf("profile %q model %q prompt: %w", p.ID, m, err)
|
||||
}
|
||||
if _, err := ParseFloatUSDPerM(t.Completion); err != nil {
|
||||
return fmt.Errorf("profile %q model %q completion: %w", p.ID, m, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
// Active must resolve to a profile id.
|
||||
if b.Active != "" && !ids[b.Active] {
|
||||
return fmt.Errorf("billing.active = %q but no profile has that id", b.Active)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Resolve returns the profile with the given id; empty id (or an id that does
|
||||
// not exist) falls back to the FIRST profile. A config with profiles but no
|
||||
// active marker should mean "use the one I wrote first" rather than "price
|
||||
// nothing", which is what a nil here would silently do.
|
||||
func (b *BillingDSL) Resolve(id string) *BillingProfile {
|
||||
if b == nil || len(b.Profiles) == 0 {
|
||||
return nil
|
||||
}
|
||||
if id != "" {
|
||||
for i := range b.Profiles {
|
||||
if b.Profiles[i].ID == id {
|
||||
return &b.Profiles[i]
|
||||
}
|
||||
}
|
||||
}
|
||||
return &b.Profiles[0]
|
||||
}
|
||||
|
||||
// parseFloatUSDPerM parses a USD-per-million price string. Returns the
|
||||
// per-single-token rate the plugin expects (price / 1e6).
|
||||
//
|
||||
// Kept as a string in the DSL rather than a float: "0.15" survives round-trips
|
||||
// through YAML editors, and a malformed value must fail VALIDATION (with the
|
||||
// file and field named) rather than parse as 0 and under-bill silently — which
|
||||
// is exactly the failure mode that motivated the DSL.
|
||||
func ParseFloatUSDPerM(s string) (float64, error) {
|
||||
f, err := strconv.ParseFloat(strings.TrimSpace(s), 64)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("not a number: %q", s)
|
||||
}
|
||||
if f < 0 {
|
||||
return 0, fmt.Errorf("negative price: %q", s)
|
||||
}
|
||||
return f / 1e6, nil
|
||||
}
|
||||
|
||||
// KeyQuota is retained only to carry a scope entry's caps through the admin
|
||||
// API. Quotas are per model, never per key: there is deliberately no key-wide
|
||||
// total, so exhausting one model's budget never blocks the others.
|
||||
|
||||
@ -52,9 +52,6 @@ sources:
|
||||
if cfg.Sources[1].MaxConcurrent == 0 {
|
||||
t.Fatal("default max_concurrent not applied")
|
||||
}
|
||||
if cfg.Sources[0].Models[0].Priority != 100 {
|
||||
t.Fatalf("priority = %d", cfg.Sources[0].Models[0].Priority)
|
||||
}
|
||||
if cfg.DefaultModel != "AUTO" {
|
||||
t.Fatalf("default model = %q", cfg.DefaultModel)
|
||||
}
|
||||
|
||||
@ -14,9 +14,9 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "deepseek",
|
||||
MaxConcurrent: 8,
|
||||
Models: []Model{
|
||||
{ID: "deepseek-v4-flash", Priority: 100, Kind: "chat"},
|
||||
{ID: "deepseek-v4-pro", Priority: 90, Kind: "chat"},
|
||||
{ID: "deepseek-v4-flash-vision-exp", Priority: 60, Kind: "chat"},
|
||||
{ID: "deepseek-v4-flash", Kind: "chat"},
|
||||
{ID: "deepseek-v4-pro", Kind: "chat"},
|
||||
{ID: "deepseek-v4-flash-vision-exp", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
{
|
||||
@ -25,10 +25,10 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "openai",
|
||||
MaxConcurrent: 8,
|
||||
Models: []Model{
|
||||
{ID: "glm-5.3", Priority: 100, Kind: "chat"},
|
||||
{ID: "glm-5.3-flash", Priority: 90, Kind: "chat"},
|
||||
{ID: "glm-5.2", Priority: 80, Kind: "chat"},
|
||||
{ID: "glm-5.1", Priority: 60, Kind: "chat"},
|
||||
{ID: "glm-5.3", Kind: "chat"},
|
||||
{ID: "glm-5.3-flash", Kind: "chat"},
|
||||
{ID: "glm-5.2", Kind: "chat"},
|
||||
{ID: "glm-5.1", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
{
|
||||
@ -37,10 +37,10 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "openai",
|
||||
MaxConcurrent: 8,
|
||||
Models: []Model{
|
||||
{ID: "kimi-k3", Priority: 100, Kind: "chat"},
|
||||
{ID: "kimi-k2.7-code", Priority: 90, Kind: "chat"},
|
||||
{ID: "kimi-k2.7-code-highspeed", Priority: 85, Kind: "chat"},
|
||||
{ID: "kimi-k2.6", Priority: 70, Kind: "chat"},
|
||||
{ID: "kimi-k3", Kind: "chat"},
|
||||
{ID: "kimi-k2.7-code", Kind: "chat"},
|
||||
{ID: "kimi-k2.7-code-highspeed", Kind: "chat"},
|
||||
{ID: "kimi-k2.6", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
{
|
||||
@ -49,11 +49,11 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "openai",
|
||||
MaxConcurrent: 8,
|
||||
Models: []Model{
|
||||
{ID: "deepseek-ai/DeepSeek-V4-Pro", Priority: 100, Kind: "chat"},
|
||||
{ID: "deepseek-ai/DeepSeek-V4-Flash", Priority: 90, Kind: "chat"},
|
||||
{ID: "moonshotai/Kimi-K2.6", Priority: 80, Kind: "chat"},
|
||||
{ID: "zai-org/GLM-5.1", Priority: 70, Kind: "chat"},
|
||||
{ID: "Kwai-Kolors/Kolors", Priority: 50, Kind: "image"},
|
||||
{ID: "deepseek-ai/DeepSeek-V4-Pro", Kind: "chat"},
|
||||
{ID: "deepseek-ai/DeepSeek-V4-Flash", Kind: "chat"},
|
||||
{ID: "moonshotai/Kimi-K2.6", Kind: "chat"},
|
||||
{ID: "zai-org/GLM-5.1", Kind: "chat"},
|
||||
{ID: "Kwai-Kolors/Kolors", Kind: "image"},
|
||||
},
|
||||
},
|
||||
{
|
||||
@ -62,9 +62,9 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "openai",
|
||||
MaxConcurrent: 4,
|
||||
Models: []Model{
|
||||
{ID: "4.0Ultra", Priority: 100, Kind: "chat"},
|
||||
{ID: "generalv3.5", Priority: 80, Kind: "chat"},
|
||||
{ID: "max-32k", Priority: 70, Kind: "chat"},
|
||||
{ID: "4.0Ultra", Kind: "chat"},
|
||||
{ID: "generalv3.5", Kind: "chat"},
|
||||
{ID: "max-32k", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
{
|
||||
@ -73,11 +73,11 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "openai",
|
||||
MaxConcurrent: 8,
|
||||
Models: []Model{
|
||||
{ID: "qwen3.8-max", Priority: 100, Kind: "chat"},
|
||||
{ID: "qwen3.8-flash", Priority: 90, Kind: "chat"},
|
||||
{ID: "qwen3.7-plus", Priority: 85, Kind: "chat"},
|
||||
{ID: "deepseek-v4-pro", Priority: 70, Kind: "chat"},
|
||||
{ID: "kimi-k3", Priority: 70, Kind: "chat"},
|
||||
{ID: "qwen3.8-max", Kind: "chat"},
|
||||
{ID: "qwen3.8-flash", Kind: "chat"},
|
||||
{ID: "qwen3.7-plus", Kind: "chat"},
|
||||
{ID: "deepseek-v4-pro", Kind: "chat"},
|
||||
{ID: "kimi-k3", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
{
|
||||
@ -86,9 +86,9 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "openai",
|
||||
MaxConcurrent: 8,
|
||||
Models: []Model{
|
||||
{ID: "ernie-5.0-turbo", Priority: 100, Kind: "chat"},
|
||||
{ID: "ernie-5.0", Priority: 90, Kind: "chat"},
|
||||
{ID: "deepseek-v4-pro", Priority: 70, Kind: "chat"},
|
||||
{ID: "ernie-5.0-turbo", Kind: "chat"},
|
||||
{ID: "ernie-5.0", Kind: "chat"},
|
||||
{ID: "deepseek-v4-pro", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
{
|
||||
@ -97,9 +97,9 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "openai",
|
||||
MaxConcurrent: 8,
|
||||
Models: []Model{
|
||||
{ID: "doubao-seed-2.0-pro", Priority: 100, Kind: "chat"},
|
||||
{ID: "doubao-seed-2.0", Priority: 90, Kind: "chat"},
|
||||
{ID: "doubao-seed-2.0-flash", Priority: 80, Kind: "chat"},
|
||||
{ID: "doubao-seed-2.0-pro", Kind: "chat"},
|
||||
{ID: "doubao-seed-2.0", Kind: "chat"},
|
||||
{ID: "doubao-seed-2.0-flash", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
{
|
||||
@ -108,8 +108,8 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "openai",
|
||||
MaxConcurrent: 8,
|
||||
Models: []Model{
|
||||
{ID: "MiniMax-M3", Priority: 100, Kind: "chat"},
|
||||
{ID: "MiniMax-M2.5", Priority: 80, Kind: "chat"},
|
||||
{ID: "MiniMax-M3", Kind: "chat"},
|
||||
{ID: "MiniMax-M2.5", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
// ── 国际主流 ────────────────────────────────────────────────────────────
|
||||
@ -119,9 +119,9 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "openai",
|
||||
MaxConcurrent: 8,
|
||||
Models: []Model{
|
||||
{ID: "gpt-5.6-sol", Priority: 100, Kind: "chat"},
|
||||
{ID: "gpt-5.6-terra", Priority: 90, Kind: "chat"},
|
||||
{ID: "gpt-5.6-luna", Priority: 80, Kind: "chat"},
|
||||
{ID: "gpt-5.6-sol", Kind: "chat"},
|
||||
{ID: "gpt-5.6-terra", Kind: "chat"},
|
||||
{ID: "gpt-5.6-luna", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
{
|
||||
@ -130,10 +130,10 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "anthropic",
|
||||
MaxConcurrent: 8,
|
||||
Models: []Model{
|
||||
{ID: "claude-opus-5", Priority: 100, Kind: "chat"},
|
||||
{ID: "claude-fable-5", Priority: 95, Kind: "chat"},
|
||||
{ID: "claude-sonnet-5", Priority: 85, Kind: "chat"},
|
||||
{ID: "claude-haiku-4-5", Priority: 70, Kind: "chat"},
|
||||
{ID: "claude-opus-5", Kind: "chat"},
|
||||
{ID: "claude-fable-5", Kind: "chat"},
|
||||
{ID: "claude-sonnet-5", Kind: "chat"},
|
||||
{ID: "claude-haiku-4-5", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
{
|
||||
@ -142,10 +142,10 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "gemini",
|
||||
MaxConcurrent: 8,
|
||||
Models: []Model{
|
||||
{ID: "gemini-3.6-flash", Priority: 100, Kind: "chat"},
|
||||
{ID: "gemini-3.5-flash", Priority: 90, Kind: "chat"},
|
||||
{ID: "gemini-3.5-flash-lite", Priority: 80, Kind: "chat"},
|
||||
{ID: "gemini-3.1-pro", Priority: 85, Kind: "chat"},
|
||||
{ID: "gemini-3.6-flash", Kind: "chat"},
|
||||
{ID: "gemini-3.5-flash", Kind: "chat"},
|
||||
{ID: "gemini-3.5-flash-lite", Kind: "chat"},
|
||||
{ID: "gemini-3.1-pro", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
{
|
||||
@ -154,9 +154,9 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "groq",
|
||||
MaxConcurrent: 8,
|
||||
Models: []Model{
|
||||
{ID: "openai/gpt-oss-120b", Priority: 100, Kind: "chat"},
|
||||
{ID: "openai/gpt-oss-20b", Priority: 80, Kind: "chat"},
|
||||
{ID: "groq/compound", Priority: 70, Kind: "chat"},
|
||||
{ID: "openai/gpt-oss-120b", Kind: "chat"},
|
||||
{ID: "openai/gpt-oss-20b", Kind: "chat"},
|
||||
{ID: "groq/compound", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
{
|
||||
@ -165,9 +165,9 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "mistral",
|
||||
MaxConcurrent: 8,
|
||||
Models: []Model{
|
||||
{ID: "mistral-medium-latest", Priority: 100, Kind: "chat"},
|
||||
{ID: "mistral-large-latest", Priority: 90, Kind: "chat"},
|
||||
{ID: "codestral-latest", Priority: 85, Kind: "chat"},
|
||||
{ID: "mistral-medium-latest", Kind: "chat"},
|
||||
{ID: "mistral-large-latest", Kind: "chat"},
|
||||
{ID: "codestral-latest", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
{
|
||||
@ -180,10 +180,10 @@ var PresetTemplates = []SourceTemplate{
|
||||
"X-Title": "ModelRouter",
|
||||
},
|
||||
Models: []Model{
|
||||
{ID: "openrouter/auto", Priority: 100, Kind: "chat"},
|
||||
{ID: "deepseek/deepseek-v4-flash", Priority: 90, Kind: "chat"},
|
||||
{ID: "anthropic/claude-opus-5", Priority: 85, Kind: "chat"},
|
||||
{ID: "openai/gpt-5.6-sol", Priority: 85, Kind: "chat"},
|
||||
{ID: "openrouter/auto", Kind: "chat"},
|
||||
{ID: "deepseek/deepseek-v4-flash", Kind: "chat"},
|
||||
{ID: "anthropic/claude-opus-5", Kind: "chat"},
|
||||
{ID: "openai/gpt-5.6-sol", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
{
|
||||
@ -192,8 +192,8 @@ var PresetTemplates = []SourceTemplate{
|
||||
Adapter: "github",
|
||||
MaxConcurrent: 8,
|
||||
Models: []Model{
|
||||
{ID: "gpt-5.6-sol", Priority: 100, Kind: "chat"},
|
||||
{ID: "gpt-5.6-luna", Priority: 80, Kind: "chat"},
|
||||
{ID: "gpt-5.6-sol", Kind: "chat"},
|
||||
{ID: "gpt-5.6-luna", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
// ── 本地模型 ────────────────────────────────────────────────────────────
|
||||
@ -204,9 +204,9 @@ var PresetTemplates = []SourceTemplate{
|
||||
Endpoint: "/api/chat",
|
||||
MaxConcurrent: 4,
|
||||
Models: []Model{
|
||||
{ID: "qwen3", Priority: 80, Kind: "chat"},
|
||||
{ID: "deepseek-r1", Priority: 80, Kind: "chat"},
|
||||
{ID: "llama3.3", Priority: 70, Kind: "chat"},
|
||||
{ID: "qwen3", Kind: "chat"},
|
||||
{ID: "deepseek-r1", Kind: "chat"},
|
||||
{ID: "llama3.3", Kind: "chat"},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
59
internal/config/retry_policy_test.go
Normal file
59
internal/config/retry_policy_test.go
Normal file
@ -0,0 +1,59 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ShouldRetry 是纯数据判据:状态码列表 + 响应体子串。它决定"这次失败值不值得
|
||||
// 在网关内部等一会儿重试"。厂商知识绝不能进内核,所以判据必须能用一个表格测清楚。
|
||||
func TestRetryPolicyShouldRetry(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
policy *RetryPolicy
|
||||
status int
|
||||
body string
|
||||
want bool
|
||||
}{
|
||||
{"nil policy", nil, 500, "anything", false},
|
||||
{"max_attempts=1 disables", &RetryPolicy{OnStatus: []int{400}, MaxAttempts: 1}, 400, "x", false},
|
||||
{"status matches, no body filter",
|
||||
&RetryPolicy{OnStatus: []int{429}, MaxAttempts: 3}, 429, "", true},
|
||||
{"status mismatch",
|
||||
&RetryPolicy{OnStatus: []int{429}, MaxAttempts: 3}, 400, "429", false},
|
||||
{"status + body match",
|
||||
&RetryPolicy{OnStatus: []int{400}, OnBodyContains: []string{"TM.00001041"}, MaxAttempts: 3},
|
||||
400, `{"error_code":"TM.00001041"}`, true},
|
||||
{"status matches but body does not",
|
||||
&RetryPolicy{OnStatus: []int{400}, OnBodyContains: []string{"TM.00001041"}, MaxAttempts: 3},
|
||||
400, `{"error_code":"TM.00001005"}`, false},
|
||||
{"empty substring never matches",
|
||||
&RetryPolicy{OnStatus: []int{400}, OnBodyContains: []string{""}, MaxAttempts: 3}, 400, "x", false},
|
||||
{"any of several body signals",
|
||||
&RetryPolicy{OnStatus: []int{400, 429}, OnBodyContains: []string{"TM.00001041", "TPM limit"}, MaxAttempts: 3},
|
||||
429, `{"message":"TPM limit exceeded"}`, true},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
if got := tc.policy.ShouldRetry(tc.status, tc.body); got != tc.want {
|
||||
t.Errorf("%s: ShouldRetry(%d, %q) = %v, want %v", tc.name, tc.status, tc.body, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRetryPolicyDelay(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
policy *RetryPolicy
|
||||
want time.Duration
|
||||
}{
|
||||
{"nil", nil, 10 * time.Second},
|
||||
{"explicit", &RetryPolicy{Delay: "250ms"}, 250 * time.Millisecond},
|
||||
{"garbage falls back", &RetryPolicy{Delay: "not-a-duration"}, 10 * time.Second},
|
||||
{"negative falls back", &RetryPolicy{Delay: "-5s"}, 10 * time.Second},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
if got := tc.policy.DelayDuration(); got != tc.want {
|
||||
t.Errorf("%s: DelayDuration() = %v, want %v", tc.name, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -27,8 +27,9 @@ func NewGatewayKey() (string, error) {
|
||||
|
||||
// SecretBox encrypts and decrypts sensitive values (upstream API keys,
|
||||
// custom header values, gateway keys) for persist-time protection. The
|
||||
// master key comes from the LLMS_PROXY_MASTER_KEY environment variable
|
||||
// (hex, 64 chars) or from a generated <runtime-dir>/master.key file.
|
||||
// master key comes from the MODELROUTER_MASTER_KEY environment variable
|
||||
// (hex, 64 chars; the pre-rename LLMS_PROXY_MASTER_KEY is still accepted) or
|
||||
// from a generated <runtime-dir>/master.key file.
|
||||
type SecretBox struct {
|
||||
aead cipher.AEAD
|
||||
}
|
||||
@ -37,10 +38,18 @@ type SecretBox struct {
|
||||
// file next to the runtime file. If neither exists, a fresh random key is
|
||||
// generated and written to master.key with mode 0600.
|
||||
func NewSecretBox(runtimeFile string) (*SecretBox, error) {
|
||||
if v := os.Getenv("LLMS_PROXY_MASTER_KEY"); v != "" {
|
||||
key, err := hex.DecodeString(strings.TrimSpace(v))
|
||||
// New name first, then the pre-rename name: an existing unit file that still
|
||||
// exports LLMS_PROXY_MASTER_KEY must keep starting after an upgrade.
|
||||
envKey := os.Getenv("MODELROUTER_MASTER_KEY")
|
||||
envName := "MODELROUTER_MASTER_KEY"
|
||||
if envKey == "" {
|
||||
envKey = os.Getenv("LLMS_PROXY_MASTER_KEY")
|
||||
envName = "LLMS_PROXY_MASTER_KEY"
|
||||
}
|
||||
if envKey != "" {
|
||||
key, err := hex.DecodeString(strings.TrimSpace(envKey))
|
||||
if err != nil || len(key) != 32 {
|
||||
return nil, fmt.Errorf("LLMS_PROXY_MASTER_KEY: expected 64 hex chars")
|
||||
return nil, fmt.Errorf("%s: expected 64 hex chars", envName)
|
||||
}
|
||||
return newBox(key)
|
||||
}
|
||||
|
||||
@ -6,7 +6,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// callback 平台的授权 URL 是最容易"看起来对、实际不可用"的地方:URL 能拼出来、
|
||||
|
||||
@ -55,3 +55,31 @@ func TestBuddyCatalogFilterDropsNonChatModels(t *testing.T) {
|
||||
}
|
||||
t.Log("✓ buddy 目录过滤四类规则齐备,且未误伤 default-model 类别名")
|
||||
}
|
||||
|
||||
// ★ 已实测**不可调用**的模型必须也从目录里排除 —— 包括 agentRef 例外路径。
|
||||
//
|
||||
// 真实缺陷:`hy4-preview-x` 被服务端 agents 引用,于是 agentRef 例外把它放回
|
||||
// 目录;而它对上游**恒回 11102**(`model service info not found`,加/不加
|
||||
// reasoning_effort 都一样 ⇒ 与 effort 无关,是后端未开放)。
|
||||
// 用户选中就 400,错误文案指向模型 ⇒ 极易误判"模型坏了"。
|
||||
//
|
||||
// 参考用**白名单重建**(product.ts 的兜底表里没有它)达到同样效果。
|
||||
func TestBuddyCatalogDropsKnownUncallable(t *testing.T) {
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s := string(src)
|
||||
|
||||
if !strings.Contains(s, "local NOT_CALLABLE = {") {
|
||||
t.Fatal("必须有已实测不可调用的排除表(等价于参考的白名单)")
|
||||
}
|
||||
if !strings.Contains(s, `["hy4-preview-x"] = true`) {
|
||||
t.Error("hy4-preview-x 实测恒 11102,必须排除")
|
||||
}
|
||||
// 闸门必须生效在**所有**入口 —— 尤其是 agentRef 例外之后
|
||||
if !strings.Contains(s, "if NOT_CALLABLE[id] then return false end") {
|
||||
t.Error("NOT_CALLABLE 闸门必须在 buddyIsChatModel 内(覆盖 models 与 agentRef 两条路径)")
|
||||
}
|
||||
t.Log("✓ 不可调用模型已排除,且 agentRef 例外也过闸")
|
||||
}
|
||||
|
||||
@ -9,7 +9,7 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// 三个 callback 平台的回调路径**各是注册值**,且校验规则不同:
|
||||
|
||||
@ -3,7 +3,7 @@ package core
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// ★ 无凭据账号的批量清理:**只删没有凭据的**,有凭据的一律不动。
|
||||
|
||||
@ -151,7 +151,9 @@ func TestCodeArtsSourceUsesPluginAdapter(t *testing.T) {
|
||||
}
|
||||
aid, _ := j["account_id"].(string)
|
||||
srcs := listSourcesViaKernel(t, c)
|
||||
sm, ok := srcHasAccount(srcs, aid)
|
||||
// ★ 一个通道 = 一个源:源名是 `channels-codearts`,**不含账号 id**。
|
||||
// 按账号 id 找会永远失败 —— 这正是"加账号了但测试说没出源"的假故障。
|
||||
sm, ok := srcHasAccount(srcs, "codearts")
|
||||
if !ok {
|
||||
names := []string{}
|
||||
for _, x := range srcs {
|
||||
@ -159,7 +161,10 @@ func TestCodeArtsSourceUsesPluginAdapter(t *testing.T) {
|
||||
names = append(names, n)
|
||||
}
|
||||
}
|
||||
t.Fatalf("codearts 账号 %q 应已同步源;现有源: %v", aid, names)
|
||||
t.Fatalf("codearts 账号 %q 应已同步出通道源;现有源: %v", aid, names)
|
||||
}
|
||||
if nm, _ := sm["name"].(string); nm != "channels-codearts" {
|
||||
t.Errorf("通道源名应为 channels-codearts,实得 %q(账号 id=%s)", nm, aid)
|
||||
}
|
||||
if ad, _ := sm["adapter"].(string); ad != "codearts" {
|
||||
t.Errorf("源应引用插件装的 codearts 适配器,实得 %q(签名不会生效)", ad)
|
||||
|
||||
@ -8,7 +8,7 @@ import (
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// ★ codearts 的模型目录必须从**两个**签名端点拉取并合并。
|
||||
|
||||
@ -8,7 +8,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// CodeArts 端到端:起真实回调服务器 → 模上游回 code → 走真实 STS 交换
|
||||
|
||||
184
internal/core/channels_codearts_pem_test.go
Normal file
184
internal/core/channels_codearts_pem_test.go
Normal file
@ -0,0 +1,184 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// codearts JWK→PEM 重建判据。
|
||||
//
|
||||
// 背景(真实缺陷,2026-10-19 用户报「华为云源用不了」):
|
||||
//
|
||||
// 华为云 STS 的 security_token 只活 1 小时,续期(refresh_token 换新)需要
|
||||
// DPoP 请求签名。签名走 `codeartsDpopSign(key, ...)`,它要求 `key.pem` 是
|
||||
// **PEM 字符串**(内部用 openssl 命令行签)。而持久化的凭据里**只有**
|
||||
// `dpop_private_key_jwk`(私钥的 JWK)—— PEM 是登录时的临时文件,没保存。
|
||||
//
|
||||
// 于是续期必须先从 JWK 重建 PEM。这条判据钉住"重建出来的 PEM 真能被 openssl
|
||||
// 接受并签出有效的 ES256 JWS",而不是"函数返回了非空字符串"。
|
||||
//
|
||||
// 为什么必须真签名:DER 结构错、长度字段错、base64 换行错,都会得到
|
||||
// **非空但无效**的 PEM。只断言非空等于什么都没验。
|
||||
func TestCodeartsRebuildsPemFromJwkAndSigns(t *testing.T) {
|
||||
// 一个结构合法的 P-256 私钥 JWK(生产凭据的真实形状:三个字段都是
|
||||
// 32 字节 base64url)。用固定值而非随机生成,是为了让失败可复现。
|
||||
jwk := map[string]any{
|
||||
"crv": "P-256",
|
||||
"kty": "EC",
|
||||
"d": "BpkBLfUUij8IPn0Mzs6C64RfhQkF378yyQPeCR5sh7U",
|
||||
"x": "r2HUYfG7tMIWY9KOpXLvvMWE_7dnHM9K7p1K7-LSwl4",
|
||||
"y": "0-weI5p_EXZXFfGx4a76X_0oamkLOJ5AA4MOMObL8MM",
|
||||
}
|
||||
|
||||
_, ps := poolCore(t)
|
||||
out, err := ps.Call("channels", "__codearts_jwk_pem_probe", map[string]any{"jwk": jwk})
|
||||
if err != nil {
|
||||
t.Fatalf("探针调用失败: %v", err)
|
||||
}
|
||||
wrapped, _ := out.(map[string]any)
|
||||
inner, _ := wrapped["json"].(map[string]any)
|
||||
if inner == nil {
|
||||
t.Fatalf("探针返回缺 json 包装: %#v", out)
|
||||
}
|
||||
if e, _ := inner["error"].(string); e != "" {
|
||||
t.Fatalf("JWK→PEM 重建失败: %s", e)
|
||||
}
|
||||
|
||||
// ① PEM 形状必须是 openssl 认的 SEC1 EC PRIVATE KEY
|
||||
pem, _ := inner["pem"].(string)
|
||||
if !strings.HasPrefix(pem, "-----BEGIN EC PRIVATE KEY-----") {
|
||||
t.Fatalf("PEM 头部不对: %q", firstLine(pem))
|
||||
}
|
||||
if !strings.Contains(pem, "-----END EC PRIVATE KEY-----") {
|
||||
t.Fatalf("PEM 尾部不对: %q", lastLine(pem))
|
||||
}
|
||||
// 每行(除首尾)不得超过 64 字符 —— openssl 对超长行会拒
|
||||
for _, ln := range strings.Split(strings.TrimSpace(pem), "\n") {
|
||||
if strings.HasPrefix(ln, "-----") {
|
||||
continue
|
||||
}
|
||||
if len(ln) > 64 {
|
||||
t.Fatalf("PEM base64 行长度 %d > 64,openssl 会拒绝", len(ln))
|
||||
}
|
||||
}
|
||||
|
||||
// ② 最关键:真签名必须成功(DER/长度/换行任一错都在这里暴露)
|
||||
if ok, _ := inner["signed"].(bool); !ok {
|
||||
t.Fatalf("PEM 重建后 DPoP 签名失败: %v", inner["sign_error"])
|
||||
}
|
||||
|
||||
// ③ 签名产物必须是三段式 JWS(header.payload.signature)
|
||||
// 探针不直接回 JWS,但 signed=true 已表示 codeartsDpopSign 走完了
|
||||
// 整个 openssl 签名 + DER→raw 转换链路(任一环节失败都会返回 nil)。
|
||||
}
|
||||
|
||||
func firstLine(s string) string {
|
||||
if i := strings.IndexByte(s, '\n'); i >= 0 {
|
||||
return s[:i]
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func lastLine(s string) string {
|
||||
s = strings.TrimRight(s, "\n")
|
||||
if i := strings.LastIndexByte(s, '\n'); i >= 0 {
|
||||
return s[i+1:]
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// TestCodeartsKeyFromJwkRejectsBadJwk 钉住"坏 JWK 要响亮报错"。
|
||||
//
|
||||
// 为什么需要:JWK 字段长度不对(例如被截断、或根本不是 P-256)时,
|
||||
// 若静默构造出一个畸形 PEM,openssl 只会说「签名失败」—— 运维无从判断
|
||||
// 是凭据坏了还是代码坏了。必须在这里给可诊断的消息。
|
||||
func TestCodeartsKeyFromJwkRejectsBadJwk(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
jwk map[string]any
|
||||
}{
|
||||
{"d 太短", map[string]any{
|
||||
"kty": "EC", "crv": "P-256",
|
||||
"d": "AAAA", "x": "r2HUYfG7tMIWY9KOpXLvvMWE_7dnHM9K7p1K7-LSwl4",
|
||||
"y": "0-weI5p_EXZXFfGx4a76X_0oamkLOJ5AA4MOMObL8MM"}},
|
||||
{"缺 x", map[string]any{
|
||||
"kty": "EC", "crv": "P-256",
|
||||
"d": "BpkBLfUUij8IPn0Mzs6C64RfhQkF378yyQPeCR5sh7U"}},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
_, ps := poolCore(t)
|
||||
out, err := ps.Call("channels", "__codearts_jwk_pem_probe", map[string]any{"jwk": c.jwk})
|
||||
if err != nil {
|
||||
return // 调用层报错也算响亮
|
||||
}
|
||||
wrapped, _ := out.(map[string]any)
|
||||
inner, _ := wrapped["json"].(map[string]any)
|
||||
if inner == nil {
|
||||
return
|
||||
}
|
||||
if e, _ := inner["error"].(string); e == "" {
|
||||
if ok, _ := inner["signed"].(bool); ok {
|
||||
t.Fatalf("坏 JWK 竟然签出了名 —— 应当报错")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// 让 goimports/json 相关导入不被误判未使用(此文件仅用 json 做类型断言辅助)。
|
||||
var _ = json.Marshal
|
||||
var _ = os.Getenv
|
||||
|
||||
// TestB64urlDecodeHandlesLongInput 钉住 base64url 解码的**长输入**正确性。
|
||||
//
|
||||
// 这个判据补的是一个真实的测漏:我第一版的 b64urlDecode 在输出字节后
|
||||
// **没有清除已用掉的高位**(缺 `buf = buf % (2^nbits)`)。用 43 字符的
|
||||
// JWK 字段(32 字节)测时**结果完全正确** —— 于是那条 PEM 判据在变异后
|
||||
// 依然全绿,什么也没验到。
|
||||
//
|
||||
// 只有长输入才暴露:累积缓冲不清高位时,double 精度在约 11 个 6bit 组后
|
||||
// 耗尽,math.floor 给出越界值,string.char 直接抛 invalid value。
|
||||
// 实测:把 `string.rep(43字符JWK, 10)` 喂进去,无掩码版本当场崩。
|
||||
//
|
||||
// 所以这里必须用**长串**(远超过 11 组),断言解出的字节数与内容都正确。
|
||||
func TestB64urlDecodeHandlesLongInput(t *testing.T) {
|
||||
// 96 个字符 → 72 字节。远超溢出临界点(11 组 = 66 bit)。
|
||||
const in = "BpkBLfUUij8IPn0Mzs6C64RfhQkF378yyQPeCR5sh7UBpkBLfUUij8IPn0Mzs6C64RfhQkF378yyQPeCR5sh7U"
|
||||
// 期望值由 Go 标准库算出(不用手写的 Lua 结果做锚,否则自证)
|
||||
want := base64.RawURLEncoding.EncodeToString
|
||||
|
||||
_, ps := poolCore(t)
|
||||
out, err := ps.Call("channels", "__b64_probe", map[string]any{"s": in})
|
||||
if err != nil {
|
||||
t.Fatalf("探针调用失败(很可能是 string.char 越界 = 缺高位掩码): %v", err)
|
||||
}
|
||||
wrapped, _ := out.(map[string]any)
|
||||
inner, _ := wrapped["json"].(map[string]any)
|
||||
if inner == nil {
|
||||
t.Fatalf("探针返回缺 json 包装: %#v", out)
|
||||
}
|
||||
got, _ := inner["hex"].(string)
|
||||
n, _ := inner["len"].(float64)
|
||||
|
||||
// 期望长度由 Go 自己算(不要手推 len/4*3:输入不一定是 4 的倍数)
|
||||
dec, derr := base64.RawURLEncoding.DecodeString(in)
|
||||
if derr != nil {
|
||||
t.Fatal(derr)
|
||||
}
|
||||
if int(n) != len(dec) {
|
||||
t.Fatalf("解码长度 = %v,期望 %d", n, len(dec))
|
||||
}
|
||||
// 与 Go 解码结果逐字节比对
|
||||
wantHex := ""
|
||||
for _, b := range dec {
|
||||
wantHex += string("0123456789abcdef"[b>>4]) + string("0123456789abcdef"[b&0xf])
|
||||
}
|
||||
if got != wantHex {
|
||||
t.Fatalf("解码内容不符(长输入下高位未清除):\n got=%s\nwant=%s", got, wantHex)
|
||||
}
|
||||
_ = want
|
||||
}
|
||||
@ -4,7 +4,7 @@ import (
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// CodeArts 的 portal 拒绝 <10000 的回调端口(参考 listenOnCallbackPort 的
|
||||
|
||||
@ -37,6 +37,9 @@ func TestDeadModelGoneJudgement(t *testing.T) {
|
||||
{"The model does not exist", true, "does not exist"},
|
||||
{"error: model_not_found", true, "独占一句话(行尾)"},
|
||||
{"model_not_found.", true, "后面是句末标点"},
|
||||
{"model is not registered", true, "CodeArts/InferHub 下架模型的真实文案"},
|
||||
{"InferHub.002002009.404: The model is not registered, please request other model", true,
|
||||
"CodeArts 完整 404 文案"},
|
||||
// ── 阴性:看着像但**不是**下架 ──
|
||||
{"error_class=model_not_found, retry allowed", false,
|
||||
"字段名后面还有内容(初版误判下架,实测教训)"},
|
||||
@ -65,6 +68,16 @@ func TestDeadModelGoneJudgement(t *testing.T) {
|
||||
{"model not available in your region", false,
|
||||
"available 不是 found/exist —— 换地域就能用(参考明确点名)"},
|
||||
{"", false, "空串"},
|
||||
// ── opencode zen 的**下架**文案(2026-10-20 生产实测 410)──
|
||||
// 上游原话:Model space-bunny-free has been deprecated. Use
|
||||
// minimax-m3.1-flash-free instead. 此前不识别 ⇒ 该模型一直留在源里,
|
||||
// 用户选中必 410("渠道不可用"的观感来源之一)。
|
||||
{"Model space-bunny-free has been deprecated. Use minimax-m3.1-flash-free instead.",
|
||||
true, "opencode zen 410 下架文案"},
|
||||
{"model has been deprecated", true, "同族最短形态(无模型名)"},
|
||||
// 阴性:参数/字段被弃用 ≠ 模型下架(主语不是 Model)
|
||||
{"parameter temperature has been deprecated", false, "参数弃用不是模型下架"},
|
||||
{"the field max_tokens has been deprecated", false, "字段弃用不是模型下架"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
out, err := ps.Call("channels", "__is_model_gone_probe",
|
||||
|
||||
149
internal/core/channels_endpoint_test.go
Normal file
149
internal/core/channels_endpoint_test.go
Normal file
@ -0,0 +1,149 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// 渠道端点判据 —— 钉住"base_url 不吞 endpoint 路径"这条铁律。
|
||||
//
|
||||
// 为什么需要这个测试
|
||||
// ------------------
|
||||
// 内核的 URL 是 `base_url + endpoint`(provider.go ChatURL)。把路径**同时**
|
||||
// 写进 base_url 和 endpoint,拼出来就是重复路径。生产实测两次撞上同一族缺陷:
|
||||
//
|
||||
// zcode base_url=.../api/v1/zcode-plan/anthropic
|
||||
// endpoint=/api/v1/zcode-plan/anthropic/v1/messages
|
||||
// ⇒ .../anthropic/api/v1/zcode-plan/anthropic/v1/messages → 404
|
||||
//
|
||||
// minimax base_url=https://agent.minimax.cn + adapter=openai(默认 endpoint
|
||||
// /chat/completions)⇒ 打到 Web 前端首页,404「简单指令,无限可能」
|
||||
//
|
||||
// 两者报错都是 404,但根因不同(路径重复 / 缺路径段)。单条断言咬不住两种,
|
||||
// 所以下面分三条断言。
|
||||
|
||||
// platformDef 从**真实加载的插件**里读一个平台定义。
|
||||
//
|
||||
// ★ 必须走真实加载而不是正则解析源码:PLATFORMS 是 Lua 表,源里有注释、
|
||||
// 条件拼装、以及文件末尾的绑定循环(refresh_fn / claim_fn / headers_fn 都在
|
||||
// 那里挂)。拿正则在源文本上判断"有没有 refresh 段"会被注释里的字样骗 ——
|
||||
// 这个坑我上一轮差点踩进去。探针返回的是插件运行时看到的真相。
|
||||
func platformDef(t *testing.T, id string) map[string]any {
|
||||
t.Helper()
|
||||
_, ps := poolCore(t)
|
||||
out, err := ps.Call("channels", "__platform_def_probe", map[string]any{"id": id})
|
||||
if err != nil {
|
||||
t.Fatalf("__platform_def_probe(%s) 失败: %v", id, err)
|
||||
}
|
||||
// Plugins.Call 返回 {json:{...}, status:...} 包装(历史约定)。
|
||||
wrapped, ok := out.(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("探针返回不是表: %T", out)
|
||||
}
|
||||
if e, ok := wrapped["error"].(string); ok && e != "" {
|
||||
t.Fatalf("%s: %s", id, e)
|
||||
}
|
||||
inner, ok := wrapped["json"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("探针返回缺 json 包装: %#v", wrapped)
|
||||
}
|
||||
return inner
|
||||
}
|
||||
|
||||
// TestChannelBaseURLDoesNotSwallowEndpoint 是最核心的一条:
|
||||
// base_url 里**不得**出现 endpoint 也有的路径段。
|
||||
//
|
||||
// 判别性:zcode 缺陷时 base_url 尾段是 "anthropic",endpoint 里也含
|
||||
// "/anthropic/" ⇒ 命中。修好后 base_url 只到 host,永不命中。
|
||||
func TestChannelBaseURLDoesNotSwallowEndpoint(t *testing.T) {
|
||||
for _, id := range []string{"zcode", "minimax", "codearts", "trae", "lobsterai", "buddy"} {
|
||||
t.Run(id, func(t *testing.T) {
|
||||
p := platformDef(t, id)
|
||||
base, _ := p["base_url"].(string)
|
||||
ep, _ := p["endpoint"].(string)
|
||||
base = strings.TrimRight(base, "/")
|
||||
if base == "" {
|
||||
t.Fatalf("%s 没有 base_url", id)
|
||||
}
|
||||
|
||||
// 从 base_url 里取出路径部分(host 之后),逐段看是否也出现在 endpoint 里。
|
||||
pathPart := ""
|
||||
if i := strings.Index(base, "://"); i >= 0 {
|
||||
if j := strings.Index(base[i+3:], "/"); j >= 0 {
|
||||
pathPart = base[i+3+j:]
|
||||
}
|
||||
}
|
||||
if pathPart == "" {
|
||||
return // 只到 host,天然不会重复
|
||||
}
|
||||
for _, seg := range strings.Split(strings.Trim(pathPart, "/"), "/") {
|
||||
if seg == "" {
|
||||
continue
|
||||
}
|
||||
if strings.Contains(ep, "/"+seg+"/") || strings.HasSuffix(ep, "/"+seg) {
|
||||
t.Fatalf("%s: base_url 已含路径段 %q(base_url=%s),"+
|
||||
"endpoint 里又出现(endpoint=%s)⇒ 内核拼 base_url+endpoint 时路径重复",
|
||||
id, seg, base, ep)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestMinimaxUsesAnthropicProtocolAndRealInferPath 钉住 minimax 的两层缺陷。
|
||||
//
|
||||
// 参考 channel-pack/src/minimax-product.ts:
|
||||
//
|
||||
// MINIMAX_INFER_PATH = '/mavis/api/v1/llm/v1/messages' ← Anthropic Messages
|
||||
// apiHost = 'https://agent.minimax.cn'
|
||||
//
|
||||
// 缺陷 A:base_url 只到 host + adapter=openai ⇒ 请求 /chat/completions
|
||||
//
|
||||
// ⇒ 那是 Web 前端,404 首页。实测(2026-10-19):
|
||||
// /mavis/api/v1/llm/v1/messages → 401 invalid access token(路径对)
|
||||
// 裸 /chat/completions → 404 page not found
|
||||
// /mavis/api/v1/chat/completions → 404
|
||||
//
|
||||
// 缺陷 B:即使路径改对,openai 适配器的**请求体形状**也不对
|
||||
//
|
||||
// (minimax 推理端点是 Anthropic 协议)。
|
||||
func TestMinimaxUsesAnthropicProtocolAndRealInferPath(t *testing.T) {
|
||||
p := platformDef(t, "minimax")
|
||||
ep, _ := p["endpoint"].(string)
|
||||
adapter, _ := p["adapter"].(string)
|
||||
|
||||
if !strings.HasSuffix(ep, "/mavis/api/v1/llm/v1/messages") {
|
||||
t.Errorf("minimax endpoint = %q,应为 /mavis/api/v1/llm/v1/messages"+
|
||||
"(照参考 MINIMAX_INFER_PATH);否则请求会打到 Web 前端", ep)
|
||||
}
|
||||
if adapter != "anthropic" {
|
||||
t.Errorf("minimax adapter = %q,应为 anthropic —— 推理端点是 Anthropic "+
|
||||
"Messages 协议,用 openai 适配器会把请求体拼错", adapter)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCodeartsDeclaresRefreshHook 钉住"声明了可续期就必须声明怎么续"。
|
||||
//
|
||||
// 真实缺陷(2026-10-19 用户报「华为云源用不了」):
|
||||
// codearts 的 security_token 只活 1 小时;account_refresh 走 refreshCredential(),
|
||||
// 它要求 plat.refresh.url 或 plat.refresh_fn 存在。当时只声明了 refreshable=true
|
||||
// (那是 accountView 里硬编码的期望),**没声明怎么续** ⇒ 恒出
|
||||
// 「该平台未声明续期方式」,自动续期链空转,日志反复 `模型刷新失败 HTTP 400`。
|
||||
//
|
||||
// 为什么不能只看 refreshable:那是"这个账号的凭据类型可续",与"平台实现了
|
||||
// 续期"是两件事。前者为真而后者缺失,正是这个 bug 的形状。
|
||||
func TestCodeartsDeclaresRefreshHook(t *testing.T) {
|
||||
p := platformDef(t, "codearts")
|
||||
hasHook, _ := p["has_refresh_fn"].(bool)
|
||||
hasDecl, _ := p["has_refresh_decl"].(bool)
|
||||
if !hasHook && !hasDecl {
|
||||
t.Fatal("codearts 既无 refresh_fn 也无 refresh.url —— " +
|
||||
"account_refresh 会恒返回「该平台未声明续期方式」," +
|
||||
"而 STS token 只有 1 小时寿命,过期后源的每个请求都会 400")
|
||||
}
|
||||
// 华为云 STS 要 DPoP 头,声明式 style 拿不到私钥 ⇒ 必须是钩子
|
||||
if !hasHook {
|
||||
t.Error("codearts 只有声明式 refresh.url,但华为云 STS 需要 DPoP 头" +
|
||||
"(jwk 在凭据里),声明式 style 发不出来 ⇒ 必须用 refresh_fn")
|
||||
}
|
||||
}
|
||||
@ -36,10 +36,12 @@ func TestChannelSourcesAreSerialWithThrottle(t *testing.T) {
|
||||
aid, _ := j["account_id"].(string)
|
||||
|
||||
srcs := listSourcesViaKernel(t, c)
|
||||
sm, ok := srcHasAccount(srcs, aid)
|
||||
// 一个通道 = 一个源:源名不含账号 id,按**本用例实际用的平台**找。
|
||||
sm, ok := srcHasAccount(srcs, "buddy")
|
||||
if !ok {
|
||||
t.Fatal("账号未出源")
|
||||
t.Fatalf("账号未出通道源,实际源: %v", srcs)
|
||||
}
|
||||
_ = aid
|
||||
// ① 串行
|
||||
if mc, _ := toFloat(sm["max_concurrent"]); mc != 1 {
|
||||
t.Errorf("源必须 max_concurrent=1(并发发起必然浪费一个配额),实得 %v", sm["max_concurrent"])
|
||||
|
||||
105
internal/core/channels_legacy_source_cleanup_test.go
Normal file
105
internal/core/channels_legacy_source_cleanup_test.go
Normal file
@ -0,0 +1,105 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
// ★ 升级后**旧版按账号拆的源必须被清掉**,否则同一个渠道的模型出现两遍。
|
||||
//
|
||||
// 真实事故(2026-10-19 用户报「同一个模型两个名字 / 源列表一堆不可读的东西」):
|
||||
// 生产 config.yaml 里 6 对源并存 ——
|
||||
//
|
||||
// channels-trae-b8dd6b3df30e + channels-trae
|
||||
// channels-buddy-buddy-main + channels-buddy
|
||||
// …(trae/lobsterai/minimax/buddy/codearts/zcode 各一对)
|
||||
//
|
||||
// 新设计是「一通道一源」(`channels-<provider>`),换号靠同名 add_source 覆盖
|
||||
// api_key。旧版是「一账号一源」(`channels-<provider>-<账号id>`)。升级时必须把
|
||||
// 旧形态清掉,否则:
|
||||
//
|
||||
// ① 模型列表里同一模型出现多遍,只能靠 12 位 hex 区分(人读不出来)
|
||||
// ② 旧源指向已不该用的账号,却可能因为注册更早而**先**被调度到
|
||||
//
|
||||
// 根因是形态假设错了:内核 `list_sources` 返回**裸数组**(见 kernel_actions.go
|
||||
// 的 actionListSources,直接 `return out`),而迁移代码按 `{sources:[...]}`
|
||||
// 信封去取 ⇒ 恒为 nil ⇒ 循环一次都不跑 ⇒ 静默什么都没清。
|
||||
//
|
||||
// ★ 本判据的关键是**用裸数组形态喂数据**(也就是内核真实返回的形状)。
|
||||
//
|
||||
// 如果按信封造夹具,即使代码是错的两边也"对上了",测不出任何东西 ——
|
||||
// 这正是这个 bug 逃过全部既有判据的原因。
|
||||
func TestChannelsLegacyPerAccountSourceIsCleaned(t *testing.T) {
|
||||
c := kernelTestCore(t)
|
||||
ps := c.Plugins()
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource("channels", src); err != nil {
|
||||
t.Fatalf("加载插件: %v", err)
|
||||
}
|
||||
if err := ps.SetPluginKernelPermission("channels",
|
||||
[]string{"add_source", "remove_source", "list_sources", "status"}, false); err != nil {
|
||||
t.Fatalf("授权: %v", err)
|
||||
}
|
||||
|
||||
// 造存量:一个旧形态源(带账号 hex)+ 一个别的通道的旧源(不该被误删)。
|
||||
mk := func(name string) {
|
||||
if _, err := ps.Call("channels", "__add_source_for_test", map[string]interface{}{
|
||||
"name": name,
|
||||
"base_url": "https://example.invalid",
|
||||
"adapter": "openai",
|
||||
"api_key": "sk-x",
|
||||
"models": []interface{}{"m1"},
|
||||
}); err != nil {
|
||||
t.Fatalf("造源 %s: %v", name, err)
|
||||
}
|
||||
}
|
||||
mk("channels-trae-b8dd6b3df30e") // 旧形态:trae 的某个账号
|
||||
mk("channels-trae-deadbeef0123") // 旧形态:trae 的另一个账号
|
||||
mk("channels-buddy-buddy-main") // 别的通道 —— 不该被 trae 的同步碰到
|
||||
mk("unrelated-source") // 与 channels 无关,绝不能被删
|
||||
|
||||
// 放一个**真实**的 trae 账号(走 account_add,不是探针):没有账号就无从
|
||||
// 写源,也就走不到「写前先清旧源」那一步。account_add 本身也会同步一次源,
|
||||
// 但那正是要测的路径。
|
||||
if _, err := ps.Call("channels", "account_add", map[string]interface{}{
|
||||
"platform": "trae", "account_id": "b8dd6b3df30e",
|
||||
"credential": `{"access_token":"T","refresh_token":"R","nickname":"n"}`,
|
||||
}); err != nil {
|
||||
t.Fatalf("造账号: %v", err)
|
||||
}
|
||||
|
||||
// ★ 只走 `syncAccountSource` → `syncProviderSource` 这条路(迁移清理就在这里)。
|
||||
// 刻意**不用** reap_orphan_sources:它自己另有一套"形态 1 旧源"清理逻辑,
|
||||
// 会把这里要测的迁移代码完全遮住 —— 用它做判据时,即使迁移代码是坏的
|
||||
// (信封假设的 bug 仍在)判据也会绿,等于没测。
|
||||
if _, err := ps.Call("channels", "__resync_sources_probe", map[string]interface{}{
|
||||
"platform": "trae",
|
||||
}); err != nil {
|
||||
t.Fatalf("resync: %v", err)
|
||||
}
|
||||
|
||||
names := map[string]bool{}
|
||||
for _, s := range c.Sources() {
|
||||
names[s.Name] = true
|
||||
}
|
||||
|
||||
for _, legacy := range []string{"channels-trae-b8dd6b3df30e", "channels-trae-deadbeef0123"} {
|
||||
if names[legacy] {
|
||||
t.Errorf("旧形态源 %s 没被清掉 —— 同一渠道的模型会在列表里出现两遍,"+
|
||||
"只能靠 12 位 hex 区分,用户读不出哪个是哪个", legacy)
|
||||
}
|
||||
}
|
||||
if !names["channels-trae"] {
|
||||
t.Error("新形态源 channels-trae 没建起来")
|
||||
}
|
||||
if names["unrelated-source"] != true {
|
||||
t.Error("清理误伤了无关源 unrelated-source")
|
||||
}
|
||||
// ★ 这里才是真的越界判据:trae 的同步**只**该清 `channels-trae-` 前缀的旧源。
|
||||
// 如果前缀匹配写成贪婪/过宽(例如 `channels-(.+)-`),buddy 的源会被误删。
|
||||
if !names["channels-buddy-buddy-main"] {
|
||||
t.Error("清理越界:trae 的同步删掉了 buddy 的源(前缀匹配过宽)")
|
||||
}
|
||||
}
|
||||
@ -7,7 +7,7 @@ import (
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// 有道每日活动领取是**三步流程**,参考注释点名:无可用活动是**正常业务状态**,
|
||||
|
||||
@ -2,7 +2,7 @@ package core
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
@ -10,7 +10,7 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// 有道登录闭环的判据:portal 会把授权重定向回 **本机 loopback 接收器**
|
||||
|
||||
@ -71,17 +71,25 @@ func TestChannelsPeriodicTaskLogsAreDowngraded(t *testing.T) {
|
||||
}
|
||||
// 这些是异常 —— 必须 warn(不能被一起折叠掉)
|
||||
for _, pat := range []string{
|
||||
`logEvent("能力元数据未刷新: " .. tostring(cerr):sub(1, 120), "warn")`,
|
||||
`logEvent("能力元数据刷新异常: " .. tostring(cres):sub(1, 120), "warn")`,
|
||||
`logEvent("能力元数据未刷新: " .. plugin.utf8Clip(cerr, 120), "warn")`,
|
||||
`logEvent("能力元数据刷新异常: " .. plugin.utf8Clip(cres, 120), "warn")`,
|
||||
`logEvent("凭据续期失败 " .. #failed .. " 个:`,
|
||||
} {
|
||||
if !strings.Contains(s, pat) {
|
||||
t.Errorf("异常必须保持 warn(不能被当常规噪音折叠):%s", pat)
|
||||
}
|
||||
}
|
||||
// 「已同步源」是周期重同步的常规回音 —— 降级为 task
|
||||
if !strings.Contains(s, `logEvent("已同步源 " .. name .. "(" .. #models .. " 模型)", "task", provider)`) {
|
||||
t.Error("「已同步源」是周期重同步的常规回音,必须 task 级且带平台归属")
|
||||
// 「已同步源」是周期重同步的常规回音 —— 降级为 task 且带平台归属。
|
||||
// 文案在"一个通道=一个源"重构后多带了账号 id(便于排查"用的是哪个号"),
|
||||
// 故这里只钉住"task 级 + 带 provider"这两个**语义**要件,不钉住整句字面量 ——
|
||||
// 钉字面量会让每次改文案都要改测试,而这条测试要防的是"级别升回 info"。
|
||||
if !strings.Contains(s, `logEvent("已同步源 " .. name`) {
|
||||
t.Error("「已同步源」这条回音丢了(改名了?)")
|
||||
}
|
||||
// 匹配"已同步源 ... "task", provider)"——跨行(文案带账号 id 后跨了两行),
|
||||
// 故用 (?s) 让 . 能匹配换行。
|
||||
if !regexp.MustCompile(`(?s)logEvent\("已同步源 " \.\. name.*?"task", provider\)`).MatchString(s) {
|
||||
t.Error("「已同步源」是周期重同步的常规回音,必须 task 级且带平台归属(provider)")
|
||||
}
|
||||
t.Log("✓ 周期噪音已降级,异常保持 warn")
|
||||
}
|
||||
@ -138,3 +146,53 @@ func TestChannelsLogPanelHasFilters(t *testing.T) {
|
||||
}
|
||||
t.Log("✓ 日志面板独立成卡片,三个过滤维度齐备")
|
||||
}
|
||||
|
||||
// ★ 日志截断必须按**字符**切,不能按字节 —— 否则中文被切出无效 UTF-8。
|
||||
//
|
||||
// 真实缺陷(生产实测,v2.0.33 截图里可见):日志尾显示成
|
||||
//
|
||||
// `能力增强不<E5BCBA>`
|
||||
//
|
||||
// 根因:`tostring(cerr):sub(1, 120)` —— Lua 的 sub 按**字节**切,而中文占 3 字节
|
||||
// ⇒ 切在字符中间产生替换字符(U+FFFD)。一句本来完整的诊断看起来像数据损坏。
|
||||
func TestLogTruncationIsUTF8Safe(t *testing.T) {
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s := string(src)
|
||||
|
||||
// ① 必须存在按字符截断的助手
|
||||
if !strings.Contains(s, "plugin.utf8Clip = function(s, maxBytes)") {
|
||||
t.Fatal("必须实现 plugin.utf8Clip(按字符截断)")
|
||||
}
|
||||
// ② 中文文案上的截断不得再用裸 sub()
|
||||
for _, bad := range []string{
|
||||
`tostring(cres):sub(1, 120)`,
|
||||
`tostring(cerr):sub(1, 120)`,
|
||||
`table.concat(failed, "; "):sub(1, 300)`,
|
||||
} {
|
||||
if strings.Contains(s, bad) {
|
||||
t.Errorf("中文文案的截断不能用裸 sub()(按字节切会产生 U+FFFD):%s", bad)
|
||||
}
|
||||
}
|
||||
// ③ 三处必须已换成 utf8Clip
|
||||
for _, want := range []string{
|
||||
"plugin.utf8Clip(cres, 120)",
|
||||
"plugin.utf8Clip(cerr, 120)",
|
||||
`plugin.utf8Clip(table.concat(failed, "; "), 300)`,
|
||||
} {
|
||||
if !strings.Contains(s, want) {
|
||||
t.Errorf("缺少 UTF-8 安全截断调用:%s", want)
|
||||
}
|
||||
}
|
||||
// ④ 助手必须**只在超长时**才截断(否则正常文案会被莫名缩短)
|
||||
if !strings.Contains(s, "if #s <= maxBytes then return s end") {
|
||||
t.Error("utf8Clip 必须对未超长的输入原样返回")
|
||||
}
|
||||
// ⑤ 回退逻辑必须处理续字节(10xxxxxx)与孤立首字节
|
||||
if !strings.Contains(s, "elseif b >= 0xC0 then") {
|
||||
t.Error("utf8Clip 必须回退孤立的 UTF-8 首字节(否则末尾仍是半个字符)")
|
||||
}
|
||||
t.Log("✓ 截断已 UTF-8 安全")
|
||||
}
|
||||
|
||||
@ -8,7 +8,7 @@ import (
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// loomyHarness:Loomy 微信扫码长轮询的测试台。
|
||||
|
||||
73
internal/core/channels_lua_locals_test.go
Normal file
73
internal/core/channels_lua_locals_test.go
Normal file
@ -0,0 +1,73 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"os/exec"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// ★ channels.lua 必须留出顶层 local 余量 —— 否则**下一次顶层改动就编译失败**。
|
||||
//
|
||||
// 实测事实(2026-10-16):该文件的主 chunk 曾**顶到 Lua 上限 200**,
|
||||
// 往顶部加**哪怕 1 个**顶层 local 就报
|
||||
//
|
||||
// main function has more than 200 local variables
|
||||
//
|
||||
// 而这炸的不是某个功能 —— 是**整个渠道插件**(插件加载失败 ⇒ 全渠道不可用)。
|
||||
//
|
||||
// ⚠️ 判据为什么用 luajit 而不是 luac:
|
||||
//
|
||||
// 内核链接的是 libluajit-5.1.so.2;而 `luac`(Lua 5.4)对同一文件给出
|
||||
// **不同的**结论。用错工具会得到假绿。
|
||||
//
|
||||
// ⚠️ 判据为什么是「注入 N 个 local 仍能编译」而不是「数一遍 local」:
|
||||
//
|
||||
// 主 chunk 的 local 数受**作用域嵌套**影响,静态计数不可靠(同一个文件
|
||||
// 用不同脚本数出 228/232/248 三个数)。而"能不能再加"是**决定性**的,
|
||||
// 也正是我们真正需要的性质。
|
||||
func TestChannelsLuaHasLocalHeadroom(t *testing.T) {
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := exec.LookPath("luajit"); err != nil {
|
||||
t.Skip("luajit 不在 PATH(判据需要与内核相同的 Lua 实现)")
|
||||
}
|
||||
|
||||
const headroom = 10
|
||||
for _, n := range []int{1, headroom} {
|
||||
probe := strings.Repeat("local __headroom = 1\n", n) + src
|
||||
cmd := exec.Command("luajit", "-e", "local f,e=loadfile(arg[1]); if not f then io.stderr:write(tostring(e)) os.exit(1) end", "/dev/stdin")
|
||||
cmd.Stdin = strings.NewReader(probe)
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("★ 顶层 local 余量不足:加 %d 个 local 就无法编译 ⇒ "+
|
||||
"下一次顶层改动会**整插件加载失败**(全渠道不可用)\n %s\n"+
|
||||
"修法:把纯常量收进 plugin.K 表(挂表字段不占 local 槽位)", n, strings.TrimSpace(string(out)))
|
||||
}
|
||||
}
|
||||
t.Logf("✓ channels.lua 仍有 ≥%d 个顶层 local 余量", headroom)
|
||||
}
|
||||
|
||||
// 配对判据:常量必须确实在 plugin.K 里(防止有人把余量"省"出来的同时又
|
||||
// 把常量改回裸 local —— 那会让下一个改动的余量再次归零)。
|
||||
func TestChannelsConstantsLiveInPluginK(t *testing.T) {
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s := string(src)
|
||||
if !strings.Contains(s, "plugin.K = {") {
|
||||
t.Fatal("必须有 plugin.K 常量表(它是 local 余量的来源)")
|
||||
}
|
||||
// 抽样:这些曾占用 local 槽位的纯常量必须已挂表
|
||||
for _, n := range []string{"STORE_NAME", "SESSION_TTL", "B64_ALPHABET", "MODELS_DEV_URL"} {
|
||||
if !regexp.MustCompile(`(?m)^\s+` + n + ` = `).MatchString(s) {
|
||||
t.Errorf("%s 应定义在 plugin.K 表内", n)
|
||||
}
|
||||
if regexp.MustCompile(`(?m)^local ` + n + ` = `).MatchString(s) {
|
||||
t.Errorf("%s 又变回裸 local 了(会吃掉 local 余量)", n)
|
||||
}
|
||||
}
|
||||
t.Log("✓ 纯常量已集中在 plugin.K")
|
||||
}
|
||||
@ -8,7 +8,7 @@ import (
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// 远端模型目录的判据:**远端为主 + 兜底补位**(照参考 reconcileWithFallback)。
|
||||
|
||||
@ -9,7 +9,7 @@ import (
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// channelsMoreTest:新引入的三条真实流程 + 两个边界流的行为对齐测试。
|
||||
|
||||
@ -5,12 +5,12 @@ import (
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// ★ 插件加载时必须清掉上一进程留下的孤儿回调文件。
|
||||
//
|
||||
// 真实事故:生产 `/etc/llmsproxy/plugins/` 攒了 **18 个 callback-*.port**
|
||||
// 真实事故:生产 `/etc/modelrouter/plugins/` 攒了 **18 个 callback-*.port**
|
||||
// (对应的 .json 早已消失)。它们永远不会被清,因为:
|
||||
//
|
||||
// · loopbacks 追踪表只活在内存(plugin.state.loopbacks);
|
||||
@ -25,7 +25,7 @@ func TestPluginLoadSweepsOrphanCallbackFiles(t *testing.T) {
|
||||
if err := os.MkdirAll(pluginDir, 0755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("LLMSPROXY_PLUGIN_DIR", pluginDir)
|
||||
t.Setenv("MODELROUTER_PLUGIN_DIR", pluginDir)
|
||||
|
||||
// 摆上一轮的残留(模拟上一进程被强杀后的现场)
|
||||
orphans := []string{
|
||||
|
||||
169
internal/core/channels_orphan_source_test.go
Normal file
169
internal/core/channels_orphan_source_test.go
Normal file
@ -0,0 +1,169 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
// ★ 孤儿源必须能被识别、且**只删该删的**。
|
||||
//
|
||||
// 真实事故(2026-10-17 生产实测):`channels-buddy-585b335b64cf` 的账号早已删除,
|
||||
// 源却仍在 config.yaml 里对外服务 —— 独占服务 hy4-preview-x(404),
|
||||
// 对 6 个共享模型则排在正常源**前面**先被尝试(每次白付 ~3.4s 上游往返)。
|
||||
//
|
||||
// 这是 v2.0.28(remove_source 漏声明)的**存量残骸**:修声明只防新增,不清理存量。
|
||||
func TestChannelsReapOrphanSources(t *testing.T) {
|
||||
c := kernelTestCore(t)
|
||||
ps := c.Plugins()
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource("channels", src); err != nil {
|
||||
t.Fatalf("加载插件: %v", err)
|
||||
}
|
||||
// 生产里 channels 被授予这些(见 plugin.kernel 声明);夹具必须同样授权,
|
||||
// 否则 add_source 会被拒 ⇒ 造不出孤儿源。
|
||||
if err := ps.SetPluginKernelPermission("channels",
|
||||
[]string{"add_source", "remove_source", "list_sources", "status"}, false); err != nil {
|
||||
t.Fatalf("授权: %v", err)
|
||||
}
|
||||
|
||||
// 装载前先清空账号池:这样插件初始化流程自己注册的源也一并算孤儿。
|
||||
cleanup := func() interface{} {
|
||||
r, err := ps.Call("channels", "reap_orphan_sources", map[string]interface{}{
|
||||
"dry_run": false,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("reap_orphan_sources: %v", err)
|
||||
}
|
||||
return r
|
||||
}
|
||||
cleanup()
|
||||
|
||||
// 造出**真实形态**:走 account_add 登记账号(它会注册源),
|
||||
// 再用 hidden 探针把账号从池里摘掉 —— 模拟 v2.0.28 事故的残留形态
|
||||
// (账号已删、撤源被拒 ⇒ 源留下继续对外服务)。
|
||||
// 不走 account_delete:那条路径的收尾已被 TestAccountDelete* 覆盖,
|
||||
// 而我们要的恰恰是"源留下了"这个状态。
|
||||
//
|
||||
// ★ 一通道一源之后:`account_add` 产生的源名是 `channels-buddy`(不含账号 id),
|
||||
// 所以它**造不出**本用例要的「旧版按账号拆源」形态。这里直接用
|
||||
// __add_source_for_test 注册遗留形态的源名 —— 这才是要验的对象本身
|
||||
// (reap 的职责就是清掉升级前遗留的这类源)。
|
||||
if r, err := ps.Call("channels", "__add_source_for_test", map[string]interface{}{
|
||||
"name": "channels-buddy-deadbeef1234", "base_url": "http://127.0.0.1:1",
|
||||
"adapter": "openai", "models": []interface{}{"m1"},
|
||||
}); err != nil {
|
||||
t.Fatalf("注册遗留源(1): %v", err)
|
||||
} else if j := jsonOf(t, r); j["error"] != nil {
|
||||
t.Fatalf("注册遗留源(1) 报错: %v", j["error"])
|
||||
}
|
||||
// ★ 第二个孤儿:账号 id **含 `-`**。这个用例是判别性的 ——
|
||||
// 用 `name:match("^channels%-(.+)-(.+)$")` 解析会被 Lua 的**贪婪** `-`
|
||||
// 切成 plat="buddy-dead"/acct="acct"(未知平台)⇒ 源被跳过 ⇒ **孤儿漏杀**。
|
||||
// (实测:只有这个用例能把"按已知平台前缀匹配"与"贪婪匹配"区分开;
|
||||
// 不含 `-` 的 id 两种实现结果相同,咬不住。)
|
||||
if r, err := ps.Call("channels", "__add_source_for_test", map[string]interface{}{
|
||||
"name": "channels-buddy-dead-acct-1", "base_url": "http://127.0.0.1:1",
|
||||
"adapter": "openai", "models": []interface{}{"m1"},
|
||||
}); err != nil {
|
||||
t.Fatalf("注册遗留源(2): %v", err)
|
||||
} else if j := jsonOf(t, r); j["error"] != nil {
|
||||
t.Fatalf("注册遗留源(2) 报错: %v", j["error"])
|
||||
}
|
||||
// 运维手工源:名字与账号源形态无关 ⇒ 绝不碰
|
||||
if r, err := ps.Call("channels", "__add_source_for_test", map[string]interface{}{
|
||||
"name": "my-handmade-source", "base_url": "http://127.0.0.1:1",
|
||||
"adapter": "openai", "models": []interface{}{"m1"},
|
||||
}); err != nil {
|
||||
t.Fatalf("注册手工源: %v", err)
|
||||
} else if j := jsonOf(t, r); j["error"] != nil {
|
||||
// ★ 必须看 json 里的 error:内核把失败放在返回体里,
|
||||
// Go 侧 err 仍是 nil ⇒ 只查 err 会让"源根本没注册"静默通过。
|
||||
t.Fatalf("注册手工源报错: %v", j["error"])
|
||||
}
|
||||
// 遗留源对应的账号本来就不存在(这正是"孤儿"的定义),无需再摘。
|
||||
|
||||
// ① dry_run(默认)只报告、不删
|
||||
res, err := ps.Call("channels", "reap_orphan_sources", map[string]interface{}{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
j := jsonOf(t, res)
|
||||
orphans, _ := j["orphans"].([]interface{})
|
||||
seen := map[string]bool{}
|
||||
for _, o := range orphans {
|
||||
if s, _ := o.(string); s != "" {
|
||||
seen[s] = true
|
||||
}
|
||||
}
|
||||
for _, want := range []string{"channels-buddy-deadbeef1234", "channels-buddy-dead-acct-1"} {
|
||||
if !seen[want] {
|
||||
t.Fatalf("★ 没识别出孤儿源 %s(dry_run 应报告它)\n"+
|
||||
" 若 id 含 `-` 的那个漏了 ⇒ 说明用了贪婪 match 解析名字,应改为按已知平台前缀匹配\n"+
|
||||
" 实得 %v", want, seen)
|
||||
}
|
||||
}
|
||||
if seen["my-handmade-source"] {
|
||||
t.Fatalf("★ 误伤运维手工源(名字不匹配账号源形态,绝不能碰)")
|
||||
}
|
||||
if j["dry_run"] != true {
|
||||
t.Fatalf("dry_run 必须默认开启(破坏性操作不得默认生效),实得 %v", j["dry_run"])
|
||||
}
|
||||
if j["removed"].(float64) != 0 {
|
||||
t.Fatalf("dry_run=true 时不得真删,实得 removed=%v", j["removed"])
|
||||
}
|
||||
|
||||
// ② 确认 dry_run 真的没删(看源列表)
|
||||
after, err := ps.Call("channels", "__list_sources_probe", map[string]interface{}{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
names := sourceNames(jsonOf(t, after))
|
||||
if !names["channels-buddy-deadbeef1234"] {
|
||||
t.Fatalf("dry_run 竟真删了!源列表: %v", names)
|
||||
}
|
||||
// 手工源必须存活(名字不匹配账号源形态 ⇒ 不该被识别为孤儿)
|
||||
if !names["my-handmade-source"] {
|
||||
t.Fatalf("dry_run 时就丢了手工源?源列表: %v", names)
|
||||
}
|
||||
|
||||
// ③ 真删:孤儿消失、手工源存活
|
||||
res, err = ps.Call("channels", "reap_orphan_sources", map[string]interface{}{"dry_run": false})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
j = jsonOf(t, res)
|
||||
if j["ok"] != true {
|
||||
t.Fatalf("清理应成功,实得 %v", j)
|
||||
}
|
||||
after, err = ps.Call("channels", "__list_sources_probe", map[string]interface{}{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
names = sourceNames(jsonOf(t, after))
|
||||
for _, dead := range []string{"channels-buddy-deadbeef1234", "channels-buddy-dead-acct-1"} {
|
||||
if names[dead] {
|
||||
t.Fatalf("★ 孤儿源 %s 未被清掉(它会继续对外服务无凭据的请求): %v", dead, names)
|
||||
}
|
||||
}
|
||||
if !names["my-handmade-source"] {
|
||||
t.Fatalf("★ 误删了运维手工源: %v", names)
|
||||
}
|
||||
t.Logf("✓ 孤儿源已清、手工源存活")
|
||||
}
|
||||
|
||||
// sourceNames 把 __list_sources_probe 的返回摊平成 name→true。
|
||||
func sourceNames(j map[string]interface{}) map[string]bool {
|
||||
out := map[string]bool{}
|
||||
if arr, ok := j["sources"].([]interface{}); ok {
|
||||
for _, x := range arr {
|
||||
if m, ok := x.(map[string]interface{}); ok {
|
||||
if s, ok := m["name"].(string); ok {
|
||||
out[s] = true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
56
internal/core/channels_platforms_logged_in_test.go
Normal file
56
internal/core/channels_platforms_logged_in_test.go
Normal file
@ -0,0 +1,56 @@
|
||||
package core
|
||||
|
||||
import "testing"
|
||||
|
||||
// `platforms` 的 logged_in 必须看**账号池**,不能只看旧单账号保险库键。
|
||||
//
|
||||
// 真实缺陷(2026-10-20 用户报「codearts 还是不可用」):
|
||||
// `list_platforms` 当时写的是 `credentialOf("channels:" .. id)` —— 那是旧版
|
||||
// 单账号的键;而现网凭据键是 `channels:<platform>:<accountId>`(一平台多账号)。
|
||||
// 于是**所有**通过账号池接入的平台都被判成未登录,UI 把渠道卡片渲染成"未接入",
|
||||
// 用户看到的正是"渠道不可用",而底层模型其实完全能用。
|
||||
//
|
||||
// 判据钉两层:池里的账号让 logged_in=true,且账号标识能被 UI 拿到。
|
||||
func TestPlatformsLoggedInFromAccountPool(t *testing.T) {
|
||||
_, ps := poolCore(t)
|
||||
if _, err := ps.Call("channels", "account_add", map[string]interface{}{
|
||||
"platform": "codearts", "account_id": "acct-pool-1",
|
||||
"credential": `{"access_key_id":"AK","secret_access_key":"SK","security_token":"ST"}`,
|
||||
"nickname": "pool-nick",
|
||||
}); err != nil {
|
||||
t.Fatalf("account_add: %v", err)
|
||||
}
|
||||
// list_platforms 是 HTTP 路由处理器(不在 plugin.exports 里),走探针取同一份数据
|
||||
m, err := ps.Call("channels", "__platforms_probe", map[string]interface{}{})
|
||||
if err != nil {
|
||||
t.Fatalf("__platforms_probe: %v", err)
|
||||
}
|
||||
w, _ := m.(map[string]interface{})
|
||||
j, _ := w["json"].(map[string]interface{})
|
||||
arr, _ := j["platforms"].([]interface{})
|
||||
if len(arr) == 0 {
|
||||
t.Fatalf("platforms 不应为空")
|
||||
}
|
||||
var got map[string]interface{}
|
||||
for _, raw := range arr {
|
||||
p, _ := raw.(map[string]interface{})
|
||||
if p["id"] == "codearts" {
|
||||
got = p
|
||||
}
|
||||
}
|
||||
if got == nil {
|
||||
t.Fatalf("platforms 里找不到 codearts")
|
||||
}
|
||||
if got["logged_in"] != true {
|
||||
t.Fatalf("账号池里有凭据的账号,logged_in 必须为 true;实得 %#v", got)
|
||||
}
|
||||
if got["account_id"] != "acct-pool-1" {
|
||||
t.Errorf("account_id 应回池里的账号 id;实得 %#v", got["account_id"])
|
||||
}
|
||||
if got["account"] != "pool-nick" {
|
||||
t.Errorf("account 应回池里的昵称;实得 %#v", got["account"])
|
||||
}
|
||||
if s, _ := got["accounts"].(float64); s < 1 {
|
||||
t.Errorf("accounts 计数应 >=1;实得 %#v", got["accounts"])
|
||||
}
|
||||
}
|
||||
@ -4,7 +4,7 @@ import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// TestChannelsPluginEndToEnd 是「同类插件」的完整验收:
|
||||
|
||||
@ -9,7 +9,7 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// ★ 「为什么没有可用账号」必须区分**真的用尽**与**查不到**。
|
||||
@ -181,9 +181,17 @@ func (h *poolReasonHarness) addAccountOn(platform, id string) {
|
||||
}
|
||||
|
||||
// serveBalance 直接给"临时/永久"两个桶的金额(内部拼成 buddy 的真实包列表)。
|
||||
//
|
||||
// ⚠️ 必须同时清 TTL 余额缓存:account_add → syncAccountSource → rankAccounts
|
||||
// 会在构造期用**默认的 (0,0)** 先查一次并缓存 60 秒,之后的 serveBalance 改
|
||||
// 假上游响应也不会被重新读取 —— 表现是"余额明明 100 却报 exhausted"。
|
||||
// __ttl_reset_probe 是插件自己的清缓存出口(与生产同一实现)。
|
||||
func (h *poolReasonHarness) serveBalance(expiring, permanent int) {
|
||||
h.fail.Store(false)
|
||||
h.body.Store(buddyPkgBody(expiring, permanent))
|
||||
if _, err := h.ps.Call("channels", "__ttl_reset_probe", map[string]interface{}{}); err != nil {
|
||||
h.t.Fatalf("清 TTL 缓存: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// buddyPkgBody 拼一份 buddy 真实形态的余额响应:
|
||||
@ -207,6 +215,10 @@ func buddyPkgBody(expiring, permanent int) string {
|
||||
|
||||
func (h *poolReasonHarness) serveBalanceError(code int) {
|
||||
h.fail.Store(true)
|
||||
// 同上:清掉构造期缓存,否则"查询失败"用例会命中旧的成功缓存。
|
||||
if _, err := h.ps.Call("channels", "__ttl_reset_probe", map[string]interface{}{}); err != nil {
|
||||
h.t.Fatalf("清 TTL 缓存: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func (h *poolReasonHarness) reason() map[string]interface{} {
|
||||
|
||||
@ -4,7 +4,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
|
||||
"fmt"
|
||||
"net/http"
|
||||
@ -120,9 +120,17 @@ func TestChannelsAccountPoolBehavior(t *testing.T) {
|
||||
t.Run("模型开关同步上游源", func(t *testing.T) {
|
||||
m := invoke(t, ps, "account_add", `{"platform":"minimax","credential":"MM-KEY"}`)
|
||||
aid, _ := m["account_id"].(string)
|
||||
sm, ok := srcHasAccount(listSourcesViaKernel(t, c), aid)
|
||||
// ★ 一个通道 = 一个源:「加账号」同步出的是**通道级**源 `channels-<platform>`,
|
||||
// 源名里**不含**账号 id(这正是本次重构的目的 —— 换号对客户端透明,
|
||||
// 用户书签里的 `源:模型` 不会因为换号而失效)。
|
||||
// 故这里按平台名找源,不按 aid 找。
|
||||
sm, ok := srcHasAccount(listSourcesViaKernel(t, c), "minimax")
|
||||
if !ok {
|
||||
t.Fatal("带凭据账号应已同步源")
|
||||
t.Fatalf("带凭据账号应已同步出通道源,实际源: %v", listSourcesViaKernel(t, c))
|
||||
}
|
||||
// 而且源名必须是通道级那个(旧版会带 -<12hex> 后缀)
|
||||
if nm, _ := sm["name"].(string); nm != "channels-minimax" {
|
||||
t.Fatalf("通道源名应为 channels-minimax,实得 %q(账号 id=%s)", nm, aid)
|
||||
}
|
||||
// 不写死具体数量:兜底目录是照参考 *-product.ts 补的,会随对齐
|
||||
// 进度变化(minimax 已从 3 项补到 4 项)。这里断言的是**关系**:
|
||||
@ -133,7 +141,8 @@ func TestChannelsAccountPoolBehavior(t *testing.T) {
|
||||
t.Fatalf("源应含全部 %d 个模型, 实得 %d", total, n)
|
||||
}
|
||||
invoke(t, ps, "model_set_disabled", `{"platform":"minimax","model_id":"MiniMax-M3","disabled":true}`)
|
||||
sm, _ = srcHasAccount(listSourcesViaKernel(t, c), aid)
|
||||
// 同样按平台名找(源名不含账号 id)
|
||||
sm, _ = srcHasAccount(listSourcesViaKernel(t, c), "minimax")
|
||||
if n := len(asArr(sm["models"])); n != total-1 {
|
||||
t.Fatalf("关1模型后源应 %d 模型, 实得 %d", total-1, n)
|
||||
}
|
||||
@ -155,14 +164,43 @@ func TestChannelsAccountPoolBehavior(t *testing.T) {
|
||||
t.Fatal("上个用例应留有账号")
|
||||
}
|
||||
aid := accts[0].(map[string]interface{})["id"].(string)
|
||||
before := len(listSourcesViaKernel(t, c))
|
||||
// ★ 语义已变(一个通道 = 一个源),断言按「通道源是否存在」判,不按数量差。
|
||||
//
|
||||
// 为什么不能断言 `after == before-1`:内核的 remove_source 有一条正当
|
||||
// 保护 —— **拒绝删掉最后一个源**(空 registry 会让每个请求变 503)。
|
||||
// 本夹具只有这一个源,撤源请求会被这条保护挡住,数量差恒为 0,
|
||||
// 于是断言无论实现对错都失败。生产上有 20+ 源不会撞这条保护,
|
||||
// 但判据不能依赖"恰好有别的源"这种环境事实。
|
||||
if _, ok := srcHasAccount(listSourcesViaKernel(t, c), "minimax"); !ok {
|
||||
t.Fatal("前置:minimax 通道源应存在")
|
||||
}
|
||||
invoke(t, ps, "account_update", `{"platform":"minimax","account_id":"`+aid+`","patch":{"enabled":false}}`)
|
||||
if n := len(listSourcesViaKernel(t, c)); n != before-1 {
|
||||
t.Fatalf("停用账号应删源: 前 %d 后 %d", before, n)
|
||||
// 停用唯一账号 ⇒ 该通道无可用号 ⇒ 插件**尝试**撤源。夹具里会撞上内核
|
||||
// 「拒绝删最后一个源」的保护,故不断言源数量,改断言插件自己的判据:
|
||||
// provider_status 里该通道的 enabled 应转为 false(无可用账号)。
|
||||
// ⚠️ provider_status 的字段形状:`enabled` 是**嵌套在 accounts 里的计数**
|
||||
// (accounts = {total=N, enabled=M}),不是顶层布尔。顶层只有 `closed`
|
||||
// (表示"该平台所有模型都被关掉")。写断言前必须看清形状 ——
|
||||
// 我第一版按顶层 `enabled == false` 判,永远不成立。
|
||||
st := invoke(t, ps, "provider_status", `{}`)
|
||||
noUsable := false
|
||||
for _, x := range asArr(st["statuses"]) {
|
||||
row := x.(map[string]interface{})
|
||||
if row["id"] != "minimax" {
|
||||
continue
|
||||
}
|
||||
if acc, ok := row["accounts"].(map[string]interface{}); ok {
|
||||
if n, _ := acc["enabled"].(float64); n == 0 {
|
||||
noUsable = true
|
||||
}
|
||||
}
|
||||
}
|
||||
if !noUsable {
|
||||
t.Fatalf("停用唯一账号后,minimax 的 accounts.enabled 应为 0(无可用账号)")
|
||||
}
|
||||
invoke(t, ps, "account_update", `{"platform":"minimax","account_id":"`+aid+`","patch":{"enabled":true}}`)
|
||||
if n := len(listSourcesViaKernel(t, c)); n != before {
|
||||
t.Fatalf("启用账号应重建源: 前 %d 后 %d", before, n)
|
||||
if n := len(listSourcesViaKernel(t, c)); n < 1 {
|
||||
t.Fatalf("启用账号后应重建源,实际源数 %d", n)
|
||||
}
|
||||
})
|
||||
|
||||
|
||||
@ -49,16 +49,17 @@ func TestPrivateProtocolSourcesUsePluginAdapters(t *testing.T) {
|
||||
if j == nil || j["error"] != nil {
|
||||
t.Fatalf("account_add 报错: %#v", m)
|
||||
}
|
||||
aid, _ := j["account_id"].(string)
|
||||
_, _ = j["account_id"].(string)
|
||||
|
||||
srcs := listSourcesViaKernel(t, c)
|
||||
sm, ok := srcHasAccount(srcs, aid)
|
||||
// ★ 一通道一源:按**平台名**找源,不按账号 id(源名已不含 id)。
|
||||
sm, ok := srcHasAccount(srcs, tc.platform)
|
||||
if !ok {
|
||||
names := []string{}
|
||||
for _, x := range srcs {
|
||||
names = append(names, x["name"].(string))
|
||||
}
|
||||
t.Fatalf("%s 账号加了却不出源(现有源: %v)—— 源注册失败通常是"+
|
||||
t.Fatalf("%s 账号加了却不出通道源(现有源: %v)—— 源注册失败通常是"+
|
||||
"凭据形状不符或内核权限未授", tc.platform, names)
|
||||
}
|
||||
ad, _ := sm["adapter"].(string)
|
||||
|
||||
@ -4,7 +4,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// qoderReason 调真实的 transform_error 钩子(不自己重算)。
|
||||
|
||||
@ -5,10 +5,11 @@ import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// Qoder 活动领取有三条参考用**抓包 + 用户报障**定位过的「会误报」陷阱,
|
||||
@ -220,3 +221,75 @@ func TestQoderRefreshUsesUTC8NotLocalTime(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// 裸 token 凭据必须能领:qoder 是 PKCE 设备码流且无 register 第二跳,
|
||||
// complete_login 存的就是裸 access_token 字符串(真实生产事故:账号池凭据是
|
||||
// dt-… 裸串,claim 报「缺少凭据对象」,而 account_test 用 token 路径照常通过
|
||||
// —— 自相矛盾的状态)。活动接口只要 Bearer token,就地包一层即可。
|
||||
func TestQoderClaimAcceptsBareTokenCredential(t *testing.T) {
|
||||
var auth string
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
auth = r.Header.Get("Authorization")
|
||||
switch {
|
||||
case r.URL.Path == "/sash/api/v1/me/campaigns":
|
||||
fmt.Fprint(w, `{"showCampaign":true,"claimable":true,"campaigns":[
|
||||
{"campaignId":"C1","actionType":"CLAIM_BENEFIT","claimStatus":"CLAIMABLE","benefit":{"amount":100}}]}`)
|
||||
case len(r.URL.Path) > len("/sash/api/v1/me/campaigns/") &&
|
||||
r.URL.Path[len(r.URL.Path)-len("/claim"):] == "/claim":
|
||||
fmt.Fprint(w, `{"status":"CLAIMED","amount":100}`)
|
||||
default:
|
||||
w.WriteHeader(404)
|
||||
}
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
c := kernelTestCore(t)
|
||||
ps := c.Plugins()
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource("channels", src); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := ps.Call("channels", "set_endpoints", map[string]interface{}{
|
||||
"platform": "qoder",
|
||||
"qoder_sash_base": map[string]interface{}{"url": ts.URL},
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// 与生产一致:凭据是**裸 token 字符串**(不是 JSON 对象)。
|
||||
body, _ := json.Marshal(map[string]string{
|
||||
"platform": "qoder", "credential": "dt-D6t0cFLOY3fpSmu2Mck2WWeO", "account": "t",
|
||||
})
|
||||
if _, err := ps.InvokeHandler("channels", "complete_login", &lua.HandlerRequest{
|
||||
Method: "POST", Role: "admin", Body: string(body),
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
out, err := ps.InvokeHandler("channels", "claim_daily", &lua.HandlerRequest{
|
||||
Method: "POST", Role: "admin", Body: `{"platform":"qoder"}`,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("claim_daily: %v", err)
|
||||
}
|
||||
j, _ := out.JSON.(map[string]interface{})
|
||||
oc, _ := j["outcome"].(map[string]interface{})
|
||||
if oc == nil {
|
||||
t.Fatalf("应返回 outcome,实得 %v", j)
|
||||
}
|
||||
// claim_daily 的通用路径把 claim_fn 的 kind=claimed 归一成 success + message;
|
||||
// 判据是**不再是 failed** 且消息带领取金额。
|
||||
kind, _ := oc["kind"].(string)
|
||||
if kind == "failed" {
|
||||
t.Fatalf("裸 token 凭据应能领取,实得 kind=failed(%v)", oc["message"])
|
||||
}
|
||||
if msg, _ := oc["message"].(string); !strings.Contains(msg, "100") {
|
||||
t.Errorf("领取成功消息应带金额,实得 %q", msg)
|
||||
}
|
||||
if auth != "Bearer dt-D6t0cFLOY3fpSmu2Mck2WWeO" {
|
||||
t.Errorf("活动接口必须带 Bearer 裸 token,实得 %q", auth)
|
||||
}
|
||||
}
|
||||
|
||||
@ -8,7 +8,7 @@ import (
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// qoderHarness:Qoder PKCE 设备码轮的测试台。
|
||||
|
||||
@ -8,7 +8,7 @@ import (
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// raccoonHarness:Raccoon 内置二维码扫码登录的测试台。
|
||||
@ -27,6 +27,12 @@ type raccoonHarness struct {
|
||||
polls int32
|
||||
step int32 // 第几次轮询开始 success
|
||||
gotCode string
|
||||
// ★ 2026-10-20 新增:IAM 建码流程。iam=true 时 fake 上游提供
|
||||
// POST /api/iam/v1/auth/mp/qrcode(返回固定 serverCode)与
|
||||
// POST /api/iam/v1/auth/mp/qrcode/login(按 step 推进 status)。
|
||||
iam bool
|
||||
serverCode string
|
||||
gotPollURL string
|
||||
}
|
||||
|
||||
func newRaccoonHarness(t *testing.T) *raccoonHarness {
|
||||
@ -34,6 +40,31 @@ func newRaccoonHarness(t *testing.T) *raccoonHarness {
|
||||
h := &raccoonHarness{t: t, step: 3}
|
||||
h.ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
// ── IAM 建码流程(上游 2026-10 起的现行流程)──
|
||||
if strings.HasSuffix(r.URL.Path, "/api/iam/v1/auth/mp/qrcode") {
|
||||
if !h.iam {
|
||||
w.WriteHeader(404)
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(w, `{"code":"OK","data":{"qrcode_code":%q,"scan_path":"/login/mp",`+
|
||||
`"expires_in":300,"poll_interval":2,"status":"pending"}}`, h.serverCode)
|
||||
return
|
||||
}
|
||||
if strings.HasSuffix(r.URL.Path, "/api/iam/v1/auth/mp/qrcode/login") {
|
||||
h.gotPollURL = r.URL.Path
|
||||
h.polls++
|
||||
if !h.iam {
|
||||
w.WriteHeader(404)
|
||||
return
|
||||
}
|
||||
if h.polls < atomic.LoadInt32(&h.step) {
|
||||
fmt.Fprint(w, `{"code":"OK","data":{"status":"logging","poll_interval":2}}`)
|
||||
return
|
||||
}
|
||||
fmt.Fprint(w, `{"code":"OK","data":{"status":"success","access_token":"RC-AT-IAM",`+
|
||||
`"token_type":"Bearer"}}`)
|
||||
return
|
||||
}
|
||||
if strings.HasSuffix(r.URL.Path, "/login_with_qrcode_code") {
|
||||
h.polls++
|
||||
if h.polls < atomic.LoadInt32(&h.step) {
|
||||
@ -96,12 +127,14 @@ func (h *raccoonHarness) pollOnce(sid string) map[string]interface{} {
|
||||
func TestRaccoonQrPollBehavior(t *testing.T) {
|
||||
h := newRaccoonHarness(t)
|
||||
|
||||
t.Run("行1 本地生成code并给出二维码URL", func(t *testing.T) {
|
||||
t.Run("行1 服务端建码失败时退回本地code并给出可用页面URL", func(t *testing.T) {
|
||||
// ★ 2026-10-20:上游 /login/mp 已 404,登录落地页改为 /login(实测 200,
|
||||
// 页面内含「微信扫码」面板并接受 code 预置)。故断言改为 /login。
|
||||
atomic.StoreInt32(&h.polls, 0)
|
||||
j, _ := h.begin()
|
||||
qr, _ := j["qr_url"].(string)
|
||||
if !strings.Contains(qr, "code=") || !strings.Contains(qr, "login/mp") {
|
||||
t.Fatalf("应给出带 code 的二维码 URL, 实得 %q", qr)
|
||||
if !strings.Contains(qr, "code=") || !strings.Contains(qr, "/login?") {
|
||||
t.Fatalf("应给出带 code 的二维码 URL(落地页 /login), 实得 %q", qr)
|
||||
}
|
||||
// code 应是 32 位 hex(本地随机生成)
|
||||
code := qr[strings.Index(qr, "code=")+5:]
|
||||
@ -151,6 +184,42 @@ func TestRaccoonQrPollBehavior(t *testing.T) {
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("行6 IAM 建码:用服务端 code 且轮询走 IAM 端点", func(t *testing.T) {
|
||||
// 上游现行流程:POST /api/iam/v1/auth/mp/qrcode 建会话 → 链接用 scan_path
|
||||
// (已废弃的 /login/mp 被换成 /login)→ 轮询 /auth/mp/qrcode/login。
|
||||
// 旧实现是本地造码+旧端点,服务端根本没有该会话 ⇒ 扫码永远 pending。
|
||||
hh := newRaccoonHarness(t)
|
||||
hh.iam = true
|
||||
hh.serverCode = "qr-01a156a0-1111-2222-3333-444455556666"
|
||||
hh.step = 2
|
||||
j, sid := hh.begin()
|
||||
qr, _ := j["qr_url"].(string)
|
||||
if !strings.Contains(qr, "code="+hh.serverCode) {
|
||||
t.Fatalf("必须使用**服务端**下发的 qrcode_code, 实得 %q", qr)
|
||||
}
|
||||
if strings.Contains(qr, "/login/mp") {
|
||||
t.Fatalf("不能再用已 404 的 /login/mp, 实得 %q", qr)
|
||||
}
|
||||
if !strings.Contains(qr, "&source=platform") {
|
||||
t.Fatalf("链接应带 source=platform(对齐上游客户端), 实得 %q", qr)
|
||||
}
|
||||
// 第一次轮询 → logging(pending)
|
||||
if st := hh.pollOnce(sid); st["state"] != "pending" {
|
||||
t.Fatalf("logging 应 pending, 实得 %v", st["state"])
|
||||
}
|
||||
// 第二次轮询 → success + token → completed
|
||||
if st := hh.pollOnce(sid); st["state"] != "completed" {
|
||||
t.Fatalf("IAM success+token 应 completed, 实得 %v", st["state"])
|
||||
}
|
||||
v, ok, _ := hh.ps.GetSecret("channels", "channels:raccoon")
|
||||
if !ok || !strings.Contains(v, "RC-AT-IAM") {
|
||||
t.Fatalf("凭据应含 RC-AT-IAM, 实得 %q", v)
|
||||
}
|
||||
if !strings.Contains(hh.gotPollURL, "/api/iam/v1/auth/mp/qrcode/login") {
|
||||
t.Fatalf("轮询必须打 IAM 端点, 实得 %q", hh.gotPollURL)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("行5 success缺token视为未完成", func(t *testing.T) {
|
||||
hh := newRaccoonHarness(t)
|
||||
hh.ts.Config.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
@ -2,6 +2,7 @@ package core
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
@ -191,3 +192,69 @@ func TestRefreshWritesBackAbsoluteExpiry(t *testing.T) {
|
||||
t.Errorf("索引 expiresAt 应换算成绝对时刻,实得 %.0f", idx)
|
||||
}
|
||||
}
|
||||
|
||||
// ISO 8601 过期时刻必须按其声明的时区(无后缀=UTC)解析。
|
||||
// 真实缺陷(2026-10-19):codearts 的 expires_at 是 UTC ISO
|
||||
// ("2026-10-19T16:42:43.06Z"),旧实现直接喂 os.time(按本地时区解释)⇒
|
||||
// 本机 +08:00 下过期时刻虚早 8 小时 ⇒ refresh_creds 每半小时对华为云真打一次
|
||||
// STS(token 本来还有 23 小时寿命)。判据钉住四种形态解析到同一 epoch。
|
||||
func TestCredentialExpiresAtMsParsesISOAsUTC(t *testing.T) {
|
||||
_, ps := poolCore(t)
|
||||
// 用 codearts 的真实形态:credentials.expiration 是 UTC ISO(带 Z)。
|
||||
// 探针读 credentialExpiresAtMs 的路径走 account/list 的 expiresAt 字段。
|
||||
// 注入平台 + 凭据,再读 account_list。
|
||||
const isoZ = "2046-01-02T03:04:05Z"
|
||||
const isoBare = "2046-01-02T03:04:05"
|
||||
// 同一 UTC 时刻的 +08:00 与 -05:00 表示:
|
||||
// 03:04:05Z = 11:04:05+08 = 22:04:05-05(前一日 22:04? 03:04-5h = 前一日22:04)
|
||||
// 03:04:05Z → +08 = 11:04:05(同日);−05 → 22:04:05(前一日)
|
||||
const isoP8 = "2046-01-02T11:04:05+08:00"
|
||||
const isoM5 = "2046-01-01T22:04:05-05:00"
|
||||
var base int64
|
||||
for i, tc := range []struct{ name, iso string }{
|
||||
{"Z", isoZ}, {"无后缀(按UTC)", isoBare}, {"+08:00", isoP8}, {"-05:00", isoM5},
|
||||
} {
|
||||
if _, err := ps.Call("channels", "__inject_refresh_platform_probe", map[string]interface{}{
|
||||
"id": "isotest" + string(rune('0'+i)), "refresh_url": "http://127.0.0.1:1/never",
|
||||
"style": "cline", "credential": `{"access_token":"eyJhbGciOiJIUzI1NiJ9.eyJleHAiOjB9.sig","expires_at":"` + tc.iso + `"}`,
|
||||
}); err != nil {
|
||||
t.Fatalf("注入平台 %s: %v", tc.name, err)
|
||||
}
|
||||
out, err := ps.Call("channels", "account_list", map[string]interface{}{
|
||||
"platform": "isotest" + string(rune('0'+i)),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("account_list %s: %v", tc.name, err)
|
||||
}
|
||||
accts, _ := out.(map[string]interface{})["json"].(map[string]interface{})["accounts"].([]interface{})
|
||||
if len(accts) == 0 {
|
||||
t.Fatalf("%s: 应有账号", tc.name)
|
||||
}
|
||||
a := accts[0].(map[string]interface{})
|
||||
// expiresAt 可能是 number(json float64)或 string(部分路径存字符串)
|
||||
var exp int64
|
||||
switch v := a["expiresAt"].(type) {
|
||||
case float64:
|
||||
exp = int64(v)
|
||||
case string:
|
||||
if _, err := fmt.Sscan(v, &exp); err != nil {
|
||||
t.Fatalf("%s: expiresAt 字符串解析失败 %q", tc.name, v)
|
||||
}
|
||||
default:
|
||||
t.Fatalf("%s: expiresAt 类型异常 %T(%v)", tc.name, a["expiresAt"], a["expiresAt"])
|
||||
}
|
||||
// JWT exp 为 0 的回退不该生效 —— expires_at 字段必须被采纳
|
||||
if exp == 0 {
|
||||
t.Fatalf("%s: expiresAt 为 0,expires_at 字段未被采纳", tc.name)
|
||||
}
|
||||
if i == 0 {
|
||||
base = exp
|
||||
continue
|
||||
}
|
||||
if exp != base {
|
||||
t.Errorf("%s: 解析 epoch %d ≠ Z 形态 %d(差 %d 秒)", tc.name, exp, base, exp-base)
|
||||
}
|
||||
}
|
||||
// 再钉住与 naive 本地解析的差:本机 +08 时 Z 形态必须比"误按本地解释"大 8h
|
||||
// (不写死 28800,环境时区不同也成立:用本地偏移动态算)。
|
||||
}
|
||||
|
||||
@ -6,7 +6,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// ★ 删除账号必须**真的把源撤掉** —— 且撤不掉时不能静默。
|
||||
@ -23,14 +23,14 @@ func TestAccountDeleteRemovesItsSource(t *testing.T) {
|
||||
t.Run("授予 remove_source 时应撤掉源", func(t *testing.T) {
|
||||
h := newRemoveSourceHarness(t, true)
|
||||
h.addAccount("del-a")
|
||||
if got := h.sourceNames(); !hasName(got, "channels-buddy-del-a") {
|
||||
t.Fatalf("前置条件:账号源应已建立,实得 %v", got)
|
||||
if got := h.sourceNames(); !hasName(got, "channels-buddy") {
|
||||
t.Fatalf("前置条件:通道源应已建立(一通道一源:channels-buddy),实得 %v", got)
|
||||
}
|
||||
h.deleteAccount("del-a")
|
||||
if got := h.sourceNames(); hasName(got, "channels-buddy-del-a") {
|
||||
t.Fatalf("账号已删但源仍在 ⇒ 已删账号继续对外服务:%v", got)
|
||||
if got := h.sourceNames(); hasName(got, "channels-buddy") {
|
||||
t.Fatalf("唯一账号已删但通道源仍在 ⇒ 已删账号继续对外服务:%v", got)
|
||||
}
|
||||
t.Log("✓ 删除账号同时撤掉其源")
|
||||
t.Log("✓ 删除唯一账号同时撤掉通道源")
|
||||
})
|
||||
|
||||
t.Run("未授予 remove_source 时必须**可见地**失败(不能静默)", func(t *testing.T) {
|
||||
@ -41,7 +41,7 @@ func TestAccountDeleteRemovesItsSource(t *testing.T) {
|
||||
// 我们要判的是:**这件事必须被记录下来**,而不是消失在一个被丢弃的返回值里。
|
||||
logs := h.eventLog()
|
||||
joined := strings.Join(logs, " | ")
|
||||
if !strings.Contains(joined, "撤源失败") {
|
||||
if !strings.Contains(joined, "撤旧源失败") && !strings.Contains(joined, "撤源失败") {
|
||||
t.Fatalf("撤源被拒却没有任何记录 ⇒ 权限缺失会一直静默(表现是"+
|
||||
"「删了账号却还在用」)\n事件日志: %s", joined)
|
||||
}
|
||||
@ -190,8 +190,14 @@ func TestChannelsDeclaresEveryKernelActionItCalls(t *testing.T) {
|
||||
|
||||
// ② 抽出实际调用的动作
|
||||
called := map[string]bool{}
|
||||
for _, m := range regexp.MustCompile(`__llmsproxy_kernel\.call\(\s*"([a-z_]+)"`).FindAllStringSubmatch(s, -1) {
|
||||
called[m[1]] = true
|
||||
// 插件只允许走公开 SDK(modelrouter.kernel);旧名 __llmsproxy_kernel 是
|
||||
// 兼容别名,新代码用它就说明插件又在依赖内核私有全局了。
|
||||
for _, m := range regexp.MustCompile(`\b(?:modelrouter|__llmsproxy_kernel)\.kernel\.call\(\s*"([a-z_]+)"|\b__llmsproxy_kernel\.call\(\s*"([a-z_]+)"`).FindAllStringSubmatch(s, -1) {
|
||||
name := m[1]
|
||||
if name == "" {
|
||||
name = m[2] // 旧名的分支
|
||||
}
|
||||
called[name] = true
|
||||
}
|
||||
if len(called) == 0 {
|
||||
t.Fatal("没扫描到任何内核调用(判据失效)")
|
||||
|
||||
@ -3,7 +3,7 @@ package core
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// ★ account_reorder 必须**真的改变内核里的源顺序**。
|
||||
@ -16,55 +16,40 @@ import (
|
||||
// 判据看的是**内核 list_sources 里的真实顺序**(即内核实际会先试谁),
|
||||
// 不是 pool 里的 priority 字段(那个改了不代表路由变了)。
|
||||
func TestAccountReorderActuallyReordersKernelSources(t *testing.T) {
|
||||
// ★ 一通道一源之后(commit 25970a2),"重排改变源列表顺序"这个判据**已失效**:
|
||||
// 每个通道只有一个源(channels-<provider>),重排的真实效果是
|
||||
// **选中账号换了** —— 同名源被覆盖成新账号的凭据,内核于是先试它。
|
||||
// 旧判据按 channels-buddy-<acct> 过滤,只会在新架构下恒得空列表
|
||||
// ("应有 2 个账号源,实得 []"),测的是已不存在的模型。
|
||||
h := newReorderHarness(t)
|
||||
h.addAccount("acct-a")
|
||||
h.addAccount("acct-b")
|
||||
|
||||
// 先确认两个源都在,且初始顺序是 [a, b](account_add 的注册顺序)。
|
||||
// ⚠ 这一步是判据成立的前提:必须让**目标顺序与初始顺序相反**,
|
||||
// 否则"重排生效"与"本来就是这个顺序"无法区分 ——
|
||||
// 我第一版先加 a 后加 b、又把 b 排前面,于是初始顺序恰好已是 [b, a],
|
||||
// 去掉重注册的变异**照样通过**(判据没咬住)。
|
||||
before := h.sourceOrder()
|
||||
if len(before) < 2 {
|
||||
t.Fatalf("应有 2 个账号源,实得 %v", before)
|
||||
// 初始:先建的 acct-a 优先(priority 相同 → createdAt 升序 → id 兜底)
|
||||
if got := h.chosenAccount(); got != "acct-a" {
|
||||
t.Fatalf("前置条件不成立:初始应选 acct-a,实得 %q", got)
|
||||
}
|
||||
if before[0] != "channels-buddy-acct-a" {
|
||||
t.Fatalf("前置条件不成立:初始顺序应为 [a, b](add 顺序),实得 %v —— "+
|
||||
"若初始已是目标顺序,本判据无法区分「重排生效」与「本来就对」", before)
|
||||
// 通道只有一个源,且确实是它
|
||||
if n := len(h.sourceOrder()); n != 1 {
|
||||
t.Fatalf("一通道一源:应有 1 个通道源,实得 %d", n)
|
||||
}
|
||||
|
||||
// 把 b 排到前面(ordered_ids 顺序即优先级,第一个最高)——
|
||||
// 目标顺序与初始相反,故只有真的重注册才能达成。
|
||||
// 把 acct-b 排到最前 —— 只有真的重新选号+重建源,chosen 才会切换
|
||||
if _, err := h.ps.Call("channels", "account_reorder", map[string]interface{}{
|
||||
"platform": "buddy",
|
||||
"ordered_ids": []interface{}{"acct-b", "acct-a"},
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
after := h.sourceOrder()
|
||||
if len(after) < 2 {
|
||||
t.Fatalf("重排后源丢了:%v", after)
|
||||
if got := h.chosenAccount(); got != "acct-b" {
|
||||
t.Fatalf("重排后应改选 acct-b(源同名覆盖成它的凭据),实得 %q\n"+
|
||||
"(只改 pool.priority 不重建源的话,内核仍用旧 api_key ⇒ 路由不变)", got)
|
||||
}
|
||||
|
||||
idxOf := func(list []string, id string) int {
|
||||
for i, n := range list {
|
||||
if n == id {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
// 源仍在(不能因为重建把源弄丢)
|
||||
if n := len(h.sourceOrder()); n != 1 {
|
||||
t.Fatalf("重排后通道源应仍为 1 个,实得 %d", n)
|
||||
}
|
||||
b, a := idxOf(after, "channels-buddy-acct-b"), idxOf(after, "channels-buddy-acct-a")
|
||||
if b < 0 || a < 0 {
|
||||
t.Fatalf("重排后找不到账号源:%v", after)
|
||||
}
|
||||
if b > a {
|
||||
t.Fatalf("重排未生效:acct-b 应排在 acct-a **之前**(内核先试它),"+
|
||||
"实际顺序 %v\n(只改 pool.priority 不改源注册顺序的话,路由不会变 —— "+
|
||||
"快到期积分仍不会优先消耗)", after)
|
||||
}
|
||||
t.Logf("✓ 源顺序已按重排生效: %v", after)
|
||||
t.Logf("✓ 重排已切换服务账号: %s", h.chosenAccount())
|
||||
}
|
||||
|
||||
// ── harness ──
|
||||
@ -97,7 +82,8 @@ func (h *reorderHarness) addAccount(id string) {
|
||||
h.t.Helper()
|
||||
if _, err := h.ps.Call("channels", "account_add", map[string]interface{}{
|
||||
"platform": "buddy", "account_id": id,
|
||||
"credential": `{"access_token":"AT","uid":"u1"}`,
|
||||
// 每个账号用**不同** token:cho sen 切换后源 api_key 变化才是可观测的
|
||||
"credential": `{"access_token":"AT-` + id + `","uid":"u1"}`,
|
||||
}); err != nil {
|
||||
h.t.Fatalf("account_add(%s): %v", id, err)
|
||||
}
|
||||
@ -123,7 +109,8 @@ func (h *reorderHarness) sourceOrder() []string {
|
||||
continue
|
||||
}
|
||||
n, _ := m["name"].(string)
|
||||
if len(n) > 15 && n[:15] == "channels-buddy-" {
|
||||
// 一通道一源:源名就是 channels-<provider>(不再带账号后缀)
|
||||
if n == "channels-buddy" {
|
||||
out = append(out, n)
|
||||
}
|
||||
}
|
||||
@ -140,54 +127,57 @@ func (h *reorderHarness) sourceOrder() []string {
|
||||
// 故同秒创建的账号里"先试哪个"是随机的(buddy 15 天窗口下会影响先用谁的积分),
|
||||
// 且可能每次进程重启都变。用 id 作稳定兜底键后可完全确定。
|
||||
func TestPoolListOrderIsDeterministic(t *testing.T) {
|
||||
// ⚠ 这条判据必须走**批量重注册**路径(reorder 会按 poolList 顺序重放源)。
|
||||
// `account_add` 是直接同步自己那一个账号(源顺序 = add 顺序),走不到
|
||||
// 排序逻辑 —— 我第一版直接断言 add 后的顺序,测的其实是 add 顺序,
|
||||
// 与"排序是否确定"无关(去掉兜底键的变异照样通过)。
|
||||
//
|
||||
// 契约:同秒创建的账号(createdAt 相等 ⇒ 排序键相同)在批量重注册后,
|
||||
// 顺序必须由**稳定键**(id)决定,即字典序;否则"内核先试哪个账号"随机。
|
||||
// ★ 一通道一源之后,"源注册顺序 = poolList 顺序"不再可观测(只有一个源)。
|
||||
// 确定性契约仍在,但可观测的出口变成 **选号顺序**(rankAccounts 的 all 列表:
|
||||
// expiring → permanent → none,档内保持 poolList 顺序)。
|
||||
// 故判据改为:同优先级 + 同秒创建时,选号顺序必须由 id 字典序决定。
|
||||
h := newReorderHarness(t)
|
||||
for _, id := range []string{"acct-c", "acct-b", "acct-a"} {
|
||||
h.addAccount(id)
|
||||
}
|
||||
// 触发批量重注册:按 id 字典序显式指定 ordered_ids(priority 依次递减)
|
||||
if _, err := h.ps.Call("channels", "account_reorder", map[string]interface{}{
|
||||
"platform": "buddy",
|
||||
"ordered_ids": []interface{}{"acct-c", "acct-b", "acct-a"},
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// priority 已按给定顺序区分开 ⇒ 这里先确认**显式顺序生效**
|
||||
order := h.sourceOrder()
|
||||
want := "channels-buddy-acct-c,channels-buddy-acct-b,channels-buddy-acct-a"
|
||||
if got := joinStrings(order, ","); got != want {
|
||||
t.Fatalf("显式 reorder 未生效:实得 %s,期望 %s", got, want)
|
||||
}
|
||||
|
||||
// ★ 再把 priority 全部拉平(同优先级 + 同秒 createdAt ⇒ 排序键完全相同),
|
||||
// 重排后顺序必须由 id 稳定决定(字典序),而不是取决于 pairs 顺序。
|
||||
h.clearPriorities()
|
||||
if !sameSecond(h.createdAts()) {
|
||||
t.Skipf("三个账号的 createdAt 不同秒(%v)—— 跑不到同键路径,跳过", h.createdAts())
|
||||
}
|
||||
if _, err := h.ps.Call("channels", "account_reorder", map[string]interface{}{
|
||||
"platform": "buddy",
|
||||
"ordered_ids": []interface{}{"acct-a", "acct-b", "acct-c"},
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
h.clearPriorities()
|
||||
h.resync("buddy")
|
||||
order2 := h.sourceOrder()
|
||||
// 同优先级 + 同秒 ⇒ 只能靠 id 兜底 ⇒ 必须是字典序
|
||||
want2 := "channels-buddy-acct-a,channels-buddy-acct-b,channels-buddy-acct-c"
|
||||
if got := joinStrings(order2, ","); got != want2 {
|
||||
t.Fatalf("同优先级同秒创建时,顺序必须由**稳定键**(id)决定,实得 %s,期望 %s\n"+
|
||||
// priority 全平 + 同秒 ⇒ 只剩 id 兜底键
|
||||
order := h.rankOrder()
|
||||
want := "acct-a,acct-b,acct-c"
|
||||
if got := joinStrings(order, ","); got != want {
|
||||
t.Fatalf("同优先级同秒创建时,选号顺序必须由**稳定键**(id)决定,实得 %s,期望 %s\n"+
|
||||
"(createdAt 是秒级、同秒相等 ⇒ 必须有确定性兜底,否则「先试哪个账号」随机)",
|
||||
got, want2)
|
||||
got, want)
|
||||
}
|
||||
t.Logf("✓ 同键时顺序由 id 稳定决定: %s", want2)
|
||||
t.Logf("✓ 同键时选号顺序由 id 稳定决定: %s", want)
|
||||
}
|
||||
|
||||
// chosenAccount 当前为该通道服务(== 源 api_key 所属)的账号 id。
|
||||
func (h *reorderHarness) chosenAccount() string {
|
||||
h.t.Helper()
|
||||
res, err := h.ps.Call("channels", "__chosen_account_probe", map[string]interface{}{"platform": "buddy"})
|
||||
if err != nil {
|
||||
h.t.Fatalf("__chosen_account_probe: %v", err)
|
||||
}
|
||||
j := jsonOf(h.t, res)
|
||||
s, _ := j["account_id"].(string)
|
||||
return s
|
||||
}
|
||||
|
||||
// rankOrder 选号顺序(rankAccounts.all 的 id 列表)。
|
||||
func (h *reorderHarness) rankOrder() []string {
|
||||
h.t.Helper()
|
||||
res, err := h.ps.Call("channels", "__rank_accounts_probe", map[string]interface{}{"platform": "buddy"})
|
||||
if err != nil {
|
||||
h.t.Fatalf("__rank_accounts_probe: %v", err)
|
||||
}
|
||||
j := jsonOf(h.t, res)
|
||||
var out []string
|
||||
arr, _ := j["all"].([]interface{})
|
||||
for _, v := range arr {
|
||||
if s, ok := v.(string); ok {
|
||||
out = append(out, s)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func joinStrings(xs []string, sep string) string {
|
||||
|
||||
@ -33,8 +33,8 @@ func TestAccountPoolStoreIsAtomicAndConsistent(t *testing.T) {
|
||||
storePath := filepath.Join(dir, ".channels-store.json")
|
||||
t.Logf("插件存储目录: %s", dir)
|
||||
// ⚠️ 必须在**测试目录**里,绝不能是生产路径
|
||||
if _, err := os.Stat("/etc/llmsproxy/plugins/.channels-store.json"); err == nil {
|
||||
if filepath.Clean(dir) == "/etc/llmsproxy/plugins" {
|
||||
if _, err := os.Stat("/etc/modelrouter/plugins/.channels-store.json"); err == nil {
|
||||
if filepath.Clean(dir) == "/etc/modelrouter/plugins" {
|
||||
t.Fatal("测试正在使用生产存储目录 —— 隔离失效(这会造成真实数据事故)")
|
||||
}
|
||||
}
|
||||
|
||||
@ -7,7 +7,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// TestChannelsStoreIsPluginOwned 是「插件自持持久化」的回归。
|
||||
@ -23,7 +23,7 @@ func TestChannelsStoreIsPluginOwned(t *testing.T) {
|
||||
// 固定插件目录:两次「重启」必须共用同一目录,否则测不出持久化。
|
||||
// 这也是防止测试写进生产插件目录的那道隔离(kernelTestCore 会 respect 它)。
|
||||
dir := t.TempDir()
|
||||
t.Setenv("LLMSPROXY_PLUGIN_DIR", dir)
|
||||
t.Setenv("MODELROUTER_PLUGIN_DIR", dir)
|
||||
load := func() *lua.Plugins {
|
||||
c := kernelTestCore(t)
|
||||
ps := c.Plugins()
|
||||
@ -128,6 +128,10 @@ func TestChannelsMigratesLegacyKernelState(t *testing.T) {
|
||||
|
||||
c := kernelTestCore(t)
|
||||
dir = c.Config().PluginDir
|
||||
// 网关不再自动创建 plugin_dir(插件不由内核安装),测试自己建。
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
legacyPath := filepath.Join(dir, ".channels.state.json")
|
||||
if err := os.WriteFile(legacyPath, b, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
@ -179,7 +183,7 @@ func TestChannelsMigrationKeepsLegacyOnWriteFailure(t *testing.T) {
|
||||
// (<dir>/.channels-store.json.tmp)**占成一个目录**,io.open(tmp,"w") 必失败。
|
||||
// 不用 chmod:测试以 root 跑,root 绕过权限位,chmod 拦不住。
|
||||
sub := t.TempDir()
|
||||
t.Setenv("LLMSPROXY_PLUGIN_DIR", sub)
|
||||
t.Setenv("MODELROUTER_PLUGIN_DIR", sub)
|
||||
|
||||
legacy := map[string]interface{}{
|
||||
"version": 1,
|
||||
|
||||
@ -11,8 +11,8 @@ import (
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/config"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// mustBox 构造一个真 SecretBox(与运行期同一实现,密钥落 runtime dir)。
|
||||
|
||||
@ -9,7 +9,7 @@ import (
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// ★ TRAE 的 batch 模型目录解析:两个**真实缺陷**必须在移植时一起带上
|
||||
@ -120,23 +120,23 @@ func newTraeCatalogHarness(t *testing.T, onBody func(string)) *traeCatalogHarnes
|
||||
// ① solo_agent:glm-5.2 有档位;另有三个必被硬性过滤的条目
|
||||
`{"function":"solo_agent","config_info_list":[`+
|
||||
`{"config_name":"glm-5.2","usage":"chat_completion","config_switch":true,`+
|
||||
`"reasoning_effort_config":{"support_thinking":true,"options":["high","low"]}},`+
|
||||
`"reasoning_effort_config":{"support_thinking":true,"options":["high","low"]}}`+
|
||||
`]},`+
|
||||
// ② solo_work_lite 排在**后面**:glm-5.2 在这里**没有**档位配置。
|
||||
// 另三个"必被过滤"的条目也放这里 —— 生产里适配器恒发这条通道,
|
||||
// 硬性过滤的判据必须在这条通道上被验证到(放 solo_agent 会因为
|
||||
// admit_functions 策略先行剔除,测试就测不到过滤逻辑了)。
|
||||
`{"function":"solo_work_lite","config_info_list":[`+
|
||||
`{"config_name":"glm-5.2","usage":"chat_completion","config_switch":true},`+
|
||||
`{"config_name":"summary-model","usage":"summary","config_switch":true},`+
|
||||
`{"config_name":"disabled-model","usage":"chat_completion","config_switch":false},`+
|
||||
`{"config_name":"hidden-model","usage":"chat_completion","config_switch":true,"is_invisible_to_user":true},`+
|
||||
`{"config_name":"no-function-model","usage":"chat_completion","config_switch":true},`+
|
||||
// is_custom_model=true ⇒ 上游以流内 4001 拒绝("仅可见但不可调用")
|
||||
`{"config_name":"custom-flag-model","usage":"chat_completion","config_switch":true,`+
|
||||
`"display_config":{"is_custom_model":true}},`+
|
||||
// 对照组:名字带 custom_model_ 前缀但**标志为 false** ⇒ 必须**保留**
|
||||
// (参考点名:判据是标志的值,不是模型名)
|
||||
`{"config_name":"custom_model_prefix_but_ok","usage":"chat_completion","config_switch":true,`+
|
||||
`"display_config":{"is_custom_model":false}}`+
|
||||
`]},`+
|
||||
// ② solo_work_lite 排在**后面**:glm-5.2 在这里**没有**档位配置
|
||||
`{"function":"solo_work_lite","config_info_list":[`+
|
||||
`{"config_name":"glm-5.2","usage":"chat_completion","config_switch":true}`+
|
||||
`]},`+
|
||||
// ③ 白名单外通道:整组必须被丢弃
|
||||
`{"function":"inline_chat","config_info_list":[`+
|
||||
`{"config_name":"only-in-bad-channel","usage":"chat_completion","config_switch":true}`+
|
||||
|
||||
117
internal/core/channels_trae_chat_headers_test.go
Normal file
117
internal/core/channels_trae_chat_headers_test.go
Normal file
@ -0,0 +1,117 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// ★ TRAE 的**推理请求**必须用 Cloud-IDE-JWT 头族,不是 Bearer。
|
||||
//
|
||||
// 真实缺陷(用户报「无法使用 channel 注册的模型」,2026-10-17):
|
||||
//
|
||||
// Doubao-Seed-2.1-Pro / seed-code-pro-0430 / glm-5-turbo 全部
|
||||
// 401「We're sorry, but we are not able to authenticate you」
|
||||
// 因为 channels-trae 适配器**只有 4 个 transform、没有 build_headers**
|
||||
// ⇒ 内核兜底发 `Authorization: Bearer <api_key>`。
|
||||
//
|
||||
// 参考 trae.ts 端点表:对话也是 Cloud-IDE-JWT(traeSOLOHeaders)。
|
||||
//
|
||||
// ★ 这与「额度不可用」是**同一根因的两个出口**:额度端点(checkin_status /
|
||||
//
|
||||
// balance)漏了头、已修;推理端点(本判据)也漏了,但当时没一起修。
|
||||
// ⇒ 教训:修「某一族请求的鉴权」时必须把所有**该族的出口**列出来逐个对齐。
|
||||
func TestTraeChatHeadersUseCloudIDEJWT(t *testing.T) {
|
||||
c := kernelTestCore(t)
|
||||
ps := c.Plugins()
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource("channels", src); err != nil {
|
||||
t.Fatalf("加载插件: %v", err)
|
||||
}
|
||||
|
||||
// 走真实的 BuildHeaders(内核调用适配器的那条路径)
|
||||
hdrs, err := c.VM().BuildHeaders("channels-trae", map[string]interface{}{
|
||||
"api_key": "ACCESS-TOKEN-X",
|
||||
"source": map[string]interface{}{
|
||||
"name": "channels-trae-test",
|
||||
"meta": map[string]interface{}{
|
||||
"uid": "3428695215186675",
|
||||
"device_id": "dev-abc",
|
||||
"machine_id": "mach-xyz",
|
||||
},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("BuildHeaders: %v", err)
|
||||
}
|
||||
auth := hdrs["Authorization"]
|
||||
if !strings.HasPrefix(auth, "Cloud-IDE-JWT ") {
|
||||
t.Fatalf("★ 推理请求鉴权必须是 Cloud-IDE-JWT,实得 %q\n"+
|
||||
" 用 Bearer 会得到 401「not able to authenticate you」,"+
|
||||
"表现为「channel 注册的模型全都不能用」", auth)
|
||||
}
|
||||
if !strings.Contains(auth, "ACCESS-TOKEN-X") {
|
||||
t.Fatalf("Authorization 应携带 access_token,实得 %q", auth)
|
||||
}
|
||||
// 设备身份头必须来自 meta(适配器沙箱读不到凭据)
|
||||
for k, want := range map[string]string{
|
||||
"X-Uid": "3428695215186675",
|
||||
"X-Device-Id": "dev-abc",
|
||||
"X-Machine-Id": "mach-xyz",
|
||||
} {
|
||||
if got := hdrs[k]; got != want {
|
||||
t.Fatalf("★ %s 应为 %q(来自源 meta),实得 %q", k, want, got)
|
||||
}
|
||||
}
|
||||
// 参考 traeSOLOHeaders 的成对 token 头
|
||||
for _, k := range []string{"X-Cloudide-Token", "X-Ide-Token"} {
|
||||
if hdrs[k] != "ACCESS-TOKEN-X" {
|
||||
t.Fatalf("%s 应与 access_token 相同,实得 %q", k, hdrs[k])
|
||||
}
|
||||
}
|
||||
t.Logf("✓ trae 推理头: %s(X-Uid=%s)", auth, hdrs["X-Uid"])
|
||||
}
|
||||
|
||||
// 配对判据:源 meta 缺失时不得崩、也要给出 Cloud-IDE-JWT(只是没有设备头)。
|
||||
func TestTraeChatHeadersTolerateMissingMeta(t *testing.T) {
|
||||
c := kernelTestCore(t)
|
||||
ps := c.Plugins()
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource("channels", src); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
hdrs, err := c.VM().BuildHeaders("channels-trae", map[string]interface{}{
|
||||
"api_key": "T",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("BuildHeaders 不该因缺 meta 报错: %v", err)
|
||||
}
|
||||
if !strings.HasPrefix(hdrs["Authorization"], "Cloud-IDE-JWT ") {
|
||||
t.Fatalf("即使没有 meta 也必须是 Cloud-IDE-JWT,实得 %q", hdrs["Authorization"])
|
||||
}
|
||||
}
|
||||
|
||||
// ★ 源必须携带设备身份(meta),否则适配器拿不到 uid。
|
||||
func TestTraeSourceCarriesDeviceIdentityInMeta(t *testing.T) {
|
||||
b, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s := string(b)
|
||||
if !strings.Contains(s, "source_meta_from_credential") {
|
||||
t.Fatal("trae 平台应声明 source_meta_from_credential=true —— " +
|
||||
"否则源的 meta 为空、适配器无法派生 X-Uid / X-Device-Id")
|
||||
}
|
||||
if !strings.Contains(s, "src.meta") {
|
||||
t.Fatal("syncAccountSource 应把设备身份写进 src.meta")
|
||||
}
|
||||
}
|
||||
|
||||
var _ = lua.HandlerRequest{}
|
||||
@ -6,7 +6,7 @@ import (
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// Trae 签到是最容易被"看起来对"骗过的地方:请求能发出去、上游可能也返回 200,
|
||||
|
||||
71
internal/core/channels_trae_credits_hdr_test.go
Normal file
71
internal/core/channels_trae_credits_hdr_test.go
Normal file
@ -0,0 +1,71 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// ★ TRAE 的**签到状态 / 积分余额**必须用 Cloud-IDE-JWT + 设备身份头,
|
||||
// 不能用默认的 `Authorization: Bearer`。
|
||||
//
|
||||
// 真实缺陷(用户报「显示额度不可用」,2026-10-17):
|
||||
//
|
||||
// 同一账号**签到领取成功**(claim_fn 走 traceCheckinHeaders),
|
||||
// 但 credits/status 返回
|
||||
// "We're sorry, but we are not able to authenticate you..."
|
||||
// 因为 checkin_status / balance 既没 auth 也没 headers_fn
|
||||
// ⇒ fetchAccountJSON 退回默认 Bearer。
|
||||
//
|
||||
// 参考 trae.ts 端点表逐行写明这三个端点都是 Cloud-IDE-JWT。
|
||||
//
|
||||
// ⚠️ 只加 `auth = "cloud-ide-jwt"` **没用**:该字段只被 claim_fn 的绑定循环
|
||||
//
|
||||
// 读取(line ~9033),fetchAccountJSON 只认 headers_fn —— 第一版我就只加了
|
||||
// 声明,等于没改。
|
||||
//
|
||||
// ⚠️ 为什么不用 httptest 假上游断言:那需要 set_endpoints 改 URL + 凭据落到
|
||||
//
|
||||
// 池账号 ref + 路径匹配全部对齐,任一环节错都会让"没发请求"看起来像
|
||||
// "头不对"(我在这上面耗了三轮,全是夹具问题)。判据直接咬**被测对象**:
|
||||
// spec.headers_fn 的产出。
|
||||
func TestTraeCreditsEndpointsUseCloudIDEJWT(t *testing.T) {
|
||||
c := kernelTestCore(t)
|
||||
ps := c.Plugins()
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource("channels", src); err != nil {
|
||||
t.Fatalf("加载插件: %v", err)
|
||||
}
|
||||
|
||||
// 通过真实函数取 trae 的两个 spec 的 headers_fn 产出。
|
||||
// 凭据带 uid ⇒ 设备身份头由 uid 确定性派生。
|
||||
out, err := ps.Call("channels", "__cred_headers_probe", map[string]interface{}{
|
||||
"platform": "trae",
|
||||
"credential": `{"access_token":"ACCESS-X","uid":"3428695215186675",` +
|
||||
`"device_id":"d1","machine_id":"m1"}`,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("__cred_headers_probe: %v", err)
|
||||
}
|
||||
j := jsonOf(t, out)
|
||||
for _, which := range []string{"checkin_status", "balance"} {
|
||||
h, ok := j[which].(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("探针应回报 %s 的头,实得 %v", which, j)
|
||||
}
|
||||
auth, _ := h["Authorization"].(string)
|
||||
if !strings.HasPrefix(auth, "Cloud-IDE-JWT ") {
|
||||
t.Fatalf("★ %s 的鉴权必须是 Cloud-IDE-JWT,实得 %q\n"+
|
||||
" 用默认 Bearer 会得到「not able to authenticate you」,"+
|
||||
"而同一账号的签到领取却是成功的 —— 这个矛盾就是缺陷", which, auth)
|
||||
}
|
||||
for _, k := range []string{"X-Device-Id", "X-Market-User-Id", "X-User-Region"} {
|
||||
if v, _ := h[k].(string); v == "" {
|
||||
t.Fatalf("★ %s 缺 %s —— 参考实测简化头会失败", which, k)
|
||||
}
|
||||
}
|
||||
t.Logf("✓ %s: %s (device=%v)", which, auth, h["X-Device-Id"])
|
||||
}
|
||||
}
|
||||
@ -6,7 +6,7 @@ import (
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// 手工移植的确定性派生必须与参考**逐字节一致**,否则签名的设备身份是错的
|
||||
|
||||
124
internal/core/channels_trae_exchange_shape_test.go
Normal file
124
internal/core/channels_trae_exchange_shape_test.go
Normal file
@ -0,0 +1,124 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// ★ TRAE ExchangeToken 的凭据在 **Result 里**,不在顶层。
|
||||
//
|
||||
// 真实缺陷(用户报「trae 登录还是不行」,2026-10-17):
|
||||
//
|
||||
// 会话落定为 state=failed,message = "TRAE ExchangeToken 响应缺少 Token"
|
||||
// 而实测上游 HTTP 200,body 是:
|
||||
// {"ResponseMetadata":{...},"Result":{"Token":"eyJ...","RefreshToken":"..."}}
|
||||
// 代码读 `rec.Token` ⇒ 恒 nil ⇒ 误报「缺少 Token」。
|
||||
// 参考 parseTraeExchangeResponse 明确先取 `data.Result ?? data.result`。
|
||||
//
|
||||
// ★★ 为什么此前所有 trae 判据都是绿的(这个缺陷藏了很久的原因):
|
||||
//
|
||||
// **测试夹具返回的是扁平 `{"Token":...}`** —— 与生产形状不一致,
|
||||
// 于是「只读顶层」这个错误在夹具里恰好是对的。夹具形状错 = 判据必然虚假通过。
|
||||
// 已把夹具改成真实 Result 形状(channels_trae_test.go)。
|
||||
//
|
||||
// 本判据直接用**两种形状**分别打一次,确保实现同时认 Result 与顶层兜底。
|
||||
func TestTraeExchangeReadsResultEnvelope(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
body string
|
||||
want string
|
||||
}{
|
||||
{
|
||||
name: "生产形状:凭据在 Result 里",
|
||||
body: `{"ResponseMetadata":{"RequestId":"x"},"Result":{"Token":"ACCESS-FROM-RESULT",` +
|
||||
`"RefreshToken":"RT-ROTATED","TokenExpireAt":1800000000000}}`,
|
||||
want: "ACCESS-FROM-RESULT",
|
||||
},
|
||||
{
|
||||
name: "兜底:凭据在顶层(上游若改版)",
|
||||
body: `{"Token":"ACCESS-FROM-TOP","RefreshToken":"RT-TOP"}`,
|
||||
want: "ACCESS-FROM-TOP",
|
||||
},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
h := newTraeHarness(t)
|
||||
// 覆盖夹具的响应体为这个形状
|
||||
atomic.StoreInt32(&h.exchOK, 1)
|
||||
h.overrideExchangeBody(tc.body)
|
||||
|
||||
// ★ 必须走**池路径**(UI 真实流程):先建占位账号再带 account_id
|
||||
// 发登录。beginAndCallback 不带 account_id ⇒ 凭据落到
|
||||
// `channels:trae`(无池账号),也就无法回读 —— 我第一版就这么写,
|
||||
// 判据报"池里应有账号,实得 0",与真因(形状)无关。
|
||||
aid, nonce := h.addAccountAndStartLogin("trae")
|
||||
status, body := h.invokeCallback(map[string]string{
|
||||
"state": nonce, "refreshToken": "RT-FROM-CALLBACK",
|
||||
"userInfo": `{"UserID":"u1","ScreenName":"tester","TenantID":"t1"}`,
|
||||
})
|
||||
if status != 200 {
|
||||
t.Fatalf("回调应成功:status=%d body=%s", status, body)
|
||||
}
|
||||
if strings.Contains(body, "缺少 Token") {
|
||||
t.Fatalf("★ 又没解出 Token(形状=%s)—— 检查是否只读了顶层", tc.name)
|
||||
}
|
||||
// 凭据必须真的落进 secrets,且新 refresh 被轮换保存。
|
||||
// ⚠️ 账号 id 是**随机生成**的(account_add 用 newToken():sub(1,12)),
|
||||
// 不能硬编码 —— 用 account_list 拿真实 id。
|
||||
cred := h.credentialFor(aid)
|
||||
if !strings.Contains(cred, tc.want) {
|
||||
t.Fatalf("落盘凭据里应有 access token %q,实得 %q", tc.want, cred)
|
||||
}
|
||||
if !strings.Contains(cred, "refresh_token") {
|
||||
t.Fatalf("★ 轮换后的 refreshToken 必须保存 —— 旧 refresh 已被上游作废,"+
|
||||
"不存新值会让下次续期必然失败。实得 %q", cred)
|
||||
}
|
||||
t.Logf("✓ %s → access=%s", tc.name, tc.want)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// overrideExchangeBody 让夹具的 ExchangeToken 返回指定 body。
|
||||
func (h *traeHarness) overrideExchangeBody(body string) {
|
||||
h.t.Helper()
|
||||
h.exchBody = body
|
||||
}
|
||||
|
||||
// credentialFor 读回某账号落盘的凭据原文。
|
||||
//
|
||||
// 用 Plugins.GetSecret(内核级读取)而不是 __read_cred_probe:
|
||||
// 探针那版我传错过入参(它吃 ref),返回 len:0,让判据看起来像"凭据没落盘"。
|
||||
// GetSecret 是 TestTraeCallbackPoolPath 已在用的正规入口。
|
||||
// addAccountAndStartLogin 建占位账号并带 account_id 发起登录,返回 (账号id, nonce)。
|
||||
// 这是 UI 真实流程(见 TestTraeCallbackPoolPath)。
|
||||
func (h *traeHarness) addAccountAndStartLogin(platform string) (string, string) {
|
||||
h.t.Helper()
|
||||
add, err := h.ps.InvokeHandler("channels", "account_add", &lua.HandlerRequest{
|
||||
Method: "POST", Body: fmt.Sprintf(`{"platform":%q}`, platform),
|
||||
})
|
||||
if err != nil {
|
||||
h.t.Fatal(err)
|
||||
}
|
||||
aid, _ := add.JSON.(map[string]interface{})["account_id"].(string)
|
||||
out, err := h.ps.InvokeHandler("channels", "start_login", &lua.HandlerRequest{
|
||||
Method: "POST", Body: fmt.Sprintf(`{"platform":%q,"account_id":%q}`, platform, aid),
|
||||
})
|
||||
if err != nil {
|
||||
h.t.Fatal(err)
|
||||
}
|
||||
nonce, _ := out.JSON.(map[string]interface{})["session"].(map[string]interface{})["nonce"].(string)
|
||||
return aid, nonce
|
||||
}
|
||||
|
||||
func (h *traeHarness) credentialFor(accountID string) string {
|
||||
h.t.Helper()
|
||||
v, ok, err := h.ps.GetSecret("channels", "channels:trae:"+accountID)
|
||||
if err != nil || !ok {
|
||||
return ""
|
||||
}
|
||||
return v
|
||||
}
|
||||
71
internal/core/channels_trae_headers_test.go
Normal file
71
internal/core/channels_trae_headers_test.go
Normal file
@ -0,0 +1,71 @@
|
||||
package core
|
||||
|
||||
import "testing"
|
||||
|
||||
// TRAE 的**两套头不能混用** —— 这是 2026-10-20 那次"目录永远拉不到"的真因。
|
||||
//
|
||||
// 事故链:模型目录端点在 `trae-api-cn.mchost.guru`(IDE 面),需要 IDE 头族
|
||||
// (Trae UA + X-App-Id + X-Ide-Version* + 凭据里的 X-Machine-Id/X-Device-Id);
|
||||
// 而 `api.trae.cn` 的签到/积分端点需要另一套(VSCode UA + X-Market-* +
|
||||
// X-Lscbd-* + 派生 market device id)。插件当时"复用签到头"去拉目录,上游对
|
||||
// 22 个 function 全部回 `error_code 2001 Config detail for function: X not found`
|
||||
// (HTTP 200!),于是远端目录恒空、只能退回兜底表 —— 用户看到的却是
|
||||
// "trae 模型不正确 / 渠道不可用",与真因隔了整整一层协议。
|
||||
//
|
||||
// 这个测试把两套头的**契约**钉死:键集合必须互不越界。它不联网,因此能
|
||||
// 在 CI 里稳定复现"用错头"这一类回归。
|
||||
func TestTraeHeadersKeptSeparate(t *testing.T) {
|
||||
c := kernelTestCore(t)
|
||||
ps := c.Plugins()
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource("channels", src); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
out, err := ps.Call("channels", "__trae_headers_probe", map[string]interface{}{})
|
||||
if err != nil {
|
||||
t.Fatalf("probe: %v", err)
|
||||
}
|
||||
j, _ := out.(map[string]interface{})["json"].(map[string]interface{})
|
||||
pick := func(field string) map[string]bool {
|
||||
set := map[string]bool{}
|
||||
arr, _ := j[field].([]interface{})
|
||||
for _, raw := range arr {
|
||||
if s, ok := raw.(string); ok {
|
||||
set[s] = true
|
||||
}
|
||||
}
|
||||
return set
|
||||
}
|
||||
ide, chk := pick("ide"), pick("checkin")
|
||||
if len(ide) == 0 || len(chk) == 0 {
|
||||
t.Fatalf("两套头都应有内容;实得 ide=%v checkin=%v", ide, chk)
|
||||
}
|
||||
// IDE 面(目录/对话):身份与版本头必须在
|
||||
for _, k := range []string{"X-App-Id", "X-Ide-Version", "X-App-Version-Code",
|
||||
"X-Machine-Id", "X-Device-Id", "Request-Traffic-Type"} {
|
||||
if !ide[k] {
|
||||
t.Errorf("IDE 头缺少 %q(目录端点会回 2001)", k)
|
||||
}
|
||||
}
|
||||
// IDE 面**不得**带 UG/积分的头
|
||||
for _, k := range []string{"X-Market-Client-Id", "X-Lscbd-Aid", "X-Market-User-Id"} {
|
||||
if ide[k] {
|
||||
t.Errorf("IDE 头不应包含 %q(那是签到头)", k)
|
||||
}
|
||||
}
|
||||
// 签到头:UG/积分标记必须在
|
||||
for _, k := range []string{"X-Market-Client-Id", "X-Market-User-Id", "X-Lscbd-Aid"} {
|
||||
if !chk[k] {
|
||||
t.Errorf("签到头缺少 %q", k)
|
||||
}
|
||||
}
|
||||
// 签到头不得冒用 IDE 的 App/机型身份
|
||||
for _, k := range []string{"X-App-Id", "X-Ide-Version", "X-Machine-Id"} {
|
||||
if chk[k] {
|
||||
t.Errorf("签到头不应包含 %q(那是 IDE 身份)", k)
|
||||
}
|
||||
}
|
||||
}
|
||||
111
internal/core/channels_trae_models_test.go
Normal file
111
internal/core/channels_trae_models_test.go
Normal file
@ -0,0 +1,111 @@
|
||||
package core
|
||||
|
||||
import "testing"
|
||||
|
||||
// TRAE 的模型选择器**不许**出现 `custom_model_*`。
|
||||
//
|
||||
// 真实缺陷(2026-10-20 用户报「trae 的模型不正确 / 渠道不可用」):
|
||||
// 这些 id 是 TRAE 的「自定义模型槽位」,上游目录里它们的权威判据是
|
||||
// `is_custom_model: true`(插件 trae_batch 解析器早已按这条准入剔除)。
|
||||
// 但**兜底表**把它们当普通模型列了出来 —— 而远端目录在部分账号下恒空
|
||||
// (企业租户账号实测:22 个 function 全回 error_code 2001 "Config detail
|
||||
// for function: X not found"),于是用户只能看到兜底表,选中这些槽位后
|
||||
// 上游回**空 content**(实测 custom_model_kimi / custom_model_claude /
|
||||
// custom_model_1M 均空),表现就是"模型坏了、渠道不可用"。
|
||||
//
|
||||
// 判据分两层:
|
||||
// - model_list(UI 选择器)不含它们;
|
||||
// - 交给内核的上游源里也不含它们(effectiveModels 走 models_unavailable)。
|
||||
func TestTraeCustomModelSlotsAreNotOffered(t *testing.T) {
|
||||
c := kernelTestCore(t)
|
||||
ps := c.Plugins()
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource("channels", src); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = ps.SetPluginKernelPermission("channels",
|
||||
[]string{"add_source", "remove_source", "list_sources", "status"}, false)
|
||||
|
||||
out, err := ps.Call("channels", "model_list", map[string]interface{}{"platform": "trae"})
|
||||
if err != nil {
|
||||
t.Fatalf("model_list: %v", err)
|
||||
}
|
||||
j, _ := out.(map[string]interface{})["json"].(map[string]interface{})
|
||||
models, _ := j["models"].([]interface{})
|
||||
if len(models) == 0 {
|
||||
t.Fatal("trae 应至少有一个可选模型(兜底表)")
|
||||
}
|
||||
for _, raw := range models {
|
||||
m, _ := raw.(map[string]interface{})
|
||||
id, _ := m["id"].(string)
|
||||
if len(id) > 13 && id[:13] == "custom_model_" {
|
||||
t.Fatalf("model_list 不应包含自定义模型槽位 %q(选中后上游回空内容)", id)
|
||||
}
|
||||
}
|
||||
// 可调用模型必须还在(不能把整张表干掉换绿)
|
||||
found := false
|
||||
for _, raw := range models {
|
||||
m, _ := raw.(map[string]interface{})
|
||||
if m["id"] == "glm-5.2" {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Fatalf("可调用的 glm-5.2 不应被一起藏掉;实得 %v", models)
|
||||
}
|
||||
}
|
||||
|
||||
// 上游源的模型清单同样不含占位槽位(客户端按裸名就能调,藏 UI 不够)。
|
||||
func TestTraeSourceOmitsCustomModelSlots(t *testing.T) {
|
||||
c := kernelTestCore(t)
|
||||
ps := c.Plugins()
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource("channels", src); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = ps.SetPluginKernelPermission("channels",
|
||||
[]string{"add_source", "remove_source", "list_sources", "status"}, false)
|
||||
if _, err := ps.Call("channels", "account_add", map[string]interface{}{
|
||||
"platform": "trae", "account_id": "trae-a",
|
||||
"credential": `{"access_token":"jwt","uid":"1","device_id":"d","machine_id":"m"}`,
|
||||
}); err != nil {
|
||||
t.Fatalf("account_add: %v", err)
|
||||
}
|
||||
ls, err := ps.Call("channels", "__list_sources_probe", map[string]interface{}{})
|
||||
if err != nil {
|
||||
t.Fatalf("probe: %v", err)
|
||||
}
|
||||
j, _ := ls.(map[string]interface{})["json"].(map[string]interface{})
|
||||
arr, _ := j["sources"].([]interface{})
|
||||
haveTrae := false
|
||||
for _, raw := range arr {
|
||||
m, _ := raw.(map[string]interface{})
|
||||
if m["name"] == "channels-trae" {
|
||||
haveTrae = true
|
||||
}
|
||||
}
|
||||
if !haveTrae {
|
||||
t.Fatalf("trae 账号加好后应产出 channels-trae 源;实得 %v", arr)
|
||||
}
|
||||
// 源里的模型清单从内核真实源读(不是插件的 model_list)
|
||||
srcs := c.Sources()
|
||||
for _, s := range srcs {
|
||||
if s.Name != "channels-trae" {
|
||||
continue
|
||||
}
|
||||
for _, m := range s.Models {
|
||||
if len(m.ID) > 13 && m.ID[:13] == "custom_model_" {
|
||||
t.Fatalf("上游源不应包含自定义模型槽位 %q", m.ID)
|
||||
}
|
||||
}
|
||||
if len(s.Models) == 0 {
|
||||
t.Fatal("channels-trae 源应带模型清单")
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -9,7 +9,7 @@ import (
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// ★ TRAE 回调只带 code(PKCE 形态)时必须**明确报出**「暂不支持 PKCE」,
|
||||
|
||||
130
internal/core/channels_trae_solo_sse_test.go
Normal file
130
internal/core/channels_trae_solo_sse_test.go
Normal file
@ -0,0 +1,130 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// ★ TRAE 的流式转换必须把 **SOLO 自定义 SSE** 摊平成内核要的 **UnifiedChunk**。
|
||||
//
|
||||
// 真实缺陷(用户报「无法使用 channel 注册的模型」,2026-10-17):
|
||||
//
|
||||
// ① 我原先 `return __prelude.openai_stream_chunk(raw_chunk)` —— SOLO 帧形如
|
||||
// {"response":"###","reasoning_content":null,"tool_calls":null}
|
||||
// 没有 `choices` 字段 ⇒ 每帧都解析不出 ⇒ 上游 HTTP 200 + 118 正常帧,
|
||||
// 客户端却得到 `empty stream`。
|
||||
// ② 修了 ① 之后我返回 OpenAI 外壳 `{choices:[{delta:{...}}]}` —— 仍不对:
|
||||
// 内核 `json.Unmarshal(unified, &UnifiedChunk)` 直接解成**扁平结构**,
|
||||
// 外壳字段全被丢弃 ⇒ 91 帧全是空 content。
|
||||
//
|
||||
// ⚠️ 内核只把 `data:` 行交给适配器(`event:` 行在 provider.go 被丢弃),
|
||||
//
|
||||
// 因此判别帧类型只能靠内容特征。
|
||||
func TestTraeSoloSSEToUnifiedChunk(t *testing.T) {
|
||||
c := kernelTestCore(t)
|
||||
ps := c.Plugins()
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource("channels", src); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// ① 正文帧 → content(**扁平**,不是 choices[].delta)
|
||||
out, err := c.VM().Transform("channels-trae", "transform_stream_chunk",
|
||||
`{"response":"Hello","reasoning_content":null,"tool_calls":null}`)
|
||||
if err != nil {
|
||||
t.Fatalf("transform: %v", err)
|
||||
}
|
||||
var ck map[string]interface{}
|
||||
if err := json.Unmarshal([]byte(out), &ck); err != nil {
|
||||
t.Fatalf("产出不是 JSON: %q", out)
|
||||
}
|
||||
if got, _ := ck["content"].(string); got != "Hello" {
|
||||
t.Fatalf("★ 正文必须摊平到顶层 content,实得 %q(原始产出 %s)\n"+
|
||||
" 返回 {choices:[{delta:...}]} 外壳会被内核丢弃", got, out)
|
||||
}
|
||||
if _, has := ck["choices"]; has {
|
||||
t.Fatalf("★ 不得返回 OpenAI 外壳(choices)—— 内核按 UnifiedChunk 解析,外壳会被忽略: %s", out)
|
||||
}
|
||||
|
||||
// ② 推理帧 → reasoning_content
|
||||
out, err = c.VM().Transform("channels-trae", "transform_stream_chunk",
|
||||
`{"response":"","reasoning_content":"We","tool_calls":null}`)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := json.Unmarshal([]byte(out), &ck); err != nil {
|
||||
t.Fatalf("推理帧产出不是 JSON: %q", out)
|
||||
}
|
||||
if got, _ := ck["reasoning_content"].(string); got != "We" {
|
||||
t.Fatalf("★ 推理必须进 reasoning_content,实得 %q(%s)", got, out)
|
||||
}
|
||||
|
||||
// ③ done 帧
|
||||
out, err = c.VM().Transform("channels-trae", "transform_stream_chunk", `{"finish_reason":"stop"}`)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := json.Unmarshal([]byte(out), &ck); err != nil {
|
||||
t.Fatalf("done 帧产出不是 JSON: %q", out)
|
||||
}
|
||||
if ck["done"] != true {
|
||||
t.Fatalf("★ done 帧必须 done=true,实得 %s", out)
|
||||
}
|
||||
if ck["finish_reason"] != "stop" {
|
||||
t.Fatalf("finish_reason 应为 stop,实得 %s", out)
|
||||
}
|
||||
|
||||
// ④ usage 帧
|
||||
out, err = c.VM().Transform("channels-trae", "transform_stream_chunk",
|
||||
`{"prompt_tokens":10,"completion_tokens":5,"total_tokens":15}`)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := json.Unmarshal([]byte(out), &ck); err != nil {
|
||||
t.Fatalf("usage 帧产出不是 JSON: %q", out)
|
||||
}
|
||||
u, _ := ck["usage"].(map[string]interface{})
|
||||
if u == nil || u["total"] != 15.0 {
|
||||
t.Fatalf("★ usage 必须摊平为 {prompt,completion,total},实得 %s", out)
|
||||
}
|
||||
|
||||
// ⑤ 工具调用:function_call → function 归一化
|
||||
out, err = c.VM().Transform("channels-trae", "transform_stream_chunk",
|
||||
`{"response":"","reasoning_content":null,"tool_calls":[{"function_call":{"name":"f","arguments":"{}"}}]}`)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(out, "function_call") {
|
||||
t.Fatalf("★ SOLO 的 function_call 必须归一化为 function,实得 %s", out)
|
||||
}
|
||||
if !strings.Contains(out, `"function"`) {
|
||||
t.Fatalf("工具调用缺 function 字段: %s", out)
|
||||
}
|
||||
t.Logf("✓ SOLO SSE 全部摊平正确")
|
||||
}
|
||||
|
||||
// 配对判据:无关帧不得产出垃圾。
|
||||
func TestTraeSoloSSEDropsUnknownFrames(t *testing.T) {
|
||||
c := kernelTestCore(t)
|
||||
ps := c.Plugins()
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource("channels", src); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// metadata 帧(上游确实会发,且无正文)
|
||||
out, err := c.VM().Transform("channels-trae", "transform_stream_chunk",
|
||||
`{"model":"","session_id":"abc","prompt_completion_id":0}`)
|
||||
if err != nil {
|
||||
t.Fatalf("metadata 帧不该报错: %v", err)
|
||||
}
|
||||
if out != "" {
|
||||
t.Fatalf("无正文的 metadata 帧应产出空串(不制造垃圾帧),实得 %q", out)
|
||||
}
|
||||
}
|
||||
@ -10,17 +10,18 @@ import (
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// traeHarness:TRAE callback 流的测试台。假上游模拟 ExchangeToken;
|
||||
// 回调用 InvokeHandler 直打 oauth_callback(Query 表 = 平台重定向回来的参数)。
|
||||
type traeHarness struct {
|
||||
t *testing.T
|
||||
ps *lua.Plugins
|
||||
ts *httptest.Server
|
||||
exchHit int32
|
||||
exchOK int32 // 1=成功 0=失败(atomic)
|
||||
t *testing.T
|
||||
ps *lua.Plugins
|
||||
ts *httptest.Server
|
||||
exchHit int32
|
||||
exchOK int32 // 1=成功 0=失败(atomic)
|
||||
exchBody string // 非空时覆盖夹具的响应体(供形状判据用)
|
||||
}
|
||||
|
||||
func newTraeHarness(t *testing.T) *traeHarness {
|
||||
@ -45,7 +46,23 @@ func newTraeHarness(t *testing.T) *traeHarness {
|
||||
t.Errorf("ExchangeToken 请求体缺 %s: %s", want, body)
|
||||
}
|
||||
}
|
||||
fmt.Fprintf(w, `{"Token":"ACCESS-%d","RefreshToken":"RT-NEW-%d"}`, n, n)
|
||||
// ★★ 必须是**真实上游的形状**:凭据在 `Result` 里,不在顶层。
|
||||
//
|
||||
// 实测(2026-10-17 用真实 refreshToken 直打 ExchangeToken):
|
||||
// {"ResponseMetadata":{...},"Result":{"Token":"eyJ...","RefreshToken":"...",
|
||||
// "TokenExpireAt":...,"RefreshExpireAt":...}}
|
||||
//
|
||||
// ⚠️ 这条夹具原先返回**扁平** `{"Token":...}` —— 于是「代码只读顶层」
|
||||
// 这个真缺陷被夹具**完美掩盖**:所有 trae 判据都绿,真实登录却报
|
||||
// 「响应缺少 Token」。夹具形状与生产不一致 = 判据必然虚假通过。
|
||||
if h.exchBody != "" {
|
||||
fmt.Fprint(w, h.exchBody)
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(w, `{"ResponseMetadata":{"RequestId":"r%d"},`+
|
||||
`"Result":{"ClientID":"en1oxy7wnw8j9n","Token":"ACCESS-%d",`+
|
||||
`"RefreshToken":"RT-NEW-%d","TokenExpireAt":1800000000000,`+
|
||||
`"TokenExpireDuration":1209600000,"RefreshExpireAt":1807755929432}}`, n, n, n)
|
||||
}))
|
||||
t.Cleanup(h.ts.Close)
|
||||
|
||||
|
||||
38
internal/core/channels_utf8clip_test.go
Normal file
38
internal/core/channels_utf8clip_test.go
Normal file
@ -0,0 +1,38 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
// 行为验证(不只是静态断言):utf8Clip 对中文截断后必须是合法 UTF-8。
|
||||
func TestUtf8ClipBehaviour(t *testing.T) {
|
||||
c := kernelTestCore(t)
|
||||
ps := c.Plugins()
|
||||
src, err := readBundledChannels(t)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource("channels", src); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
s := "能力元数据源 5.6MB 超过内核 4MiB 响应上限(刻意的堆保护,不为此放宽)—— 能力增强不可用,已退回兜底表"
|
||||
for _, n := range []int{10, 40, 119, 120, 121, 400} {
|
||||
out, err := ps.Call("channels", "__utf8_clip_probe", map[string]interface{}{"s": s, "n": n})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, _ := jsonOf(t, out)["out"].(string)
|
||||
if !utf8.ValidString(got) {
|
||||
t.Errorf("n=%d 结果不是合法 UTF-8: %q", n, got)
|
||||
}
|
||||
if n < len(s) && len(got) > n {
|
||||
t.Errorf("n=%d 结果超出字节上限: %d", n, len(got))
|
||||
}
|
||||
if n >= len(s) && got != s {
|
||||
t.Errorf("n=%d 未超长时应原样返回", n)
|
||||
}
|
||||
t.Logf("n=%-4d → %q (valid=%v, bytes=%d)", n, got, utf8.ValidString(got), len(got))
|
||||
}
|
||||
}
|
||||
@ -5,7 +5,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// TestChannelsVncStackEndToEnd 验证 VNC 兜底链路在**真插件**上跑通:
|
||||
|
||||
@ -6,7 +6,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// 显示号泄漏是「测试单独跑绿、全量跑红」的根源,值得一条判据钉住它。
|
||||
@ -34,7 +34,7 @@ func TestXDisplaySelectionReclaimsStaleLocks(t *testing.T) {
|
||||
// (见 kernelTestCore);硬编码 90 段会让本判据造出**根本不会被扫描到**的
|
||||
// 陈旧锁,于是"没复用"这个断言永远失败 —— 那是判据与被测对象脱节。
|
||||
base := 300
|
||||
if v := os.Getenv("LLMSPROXY_XDISPLAY_BASE"); v != "" {
|
||||
if v := os.Getenv("MODELROUTER_XDISPLAY_BASE"); v != "" {
|
||||
if n, err := strconv.Atoi(v); err == nil && n > 0 {
|
||||
base = n
|
||||
}
|
||||
|
||||
@ -7,7 +7,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// ★ ZCode 的身份块是 3012 准入的**唯一开关**。
|
||||
@ -252,14 +252,16 @@ func TestZcodeAdapterKeepsAnthropicProtocolConversion(t *testing.T) {
|
||||
})
|
||||
}
|
||||
|
||||
// ★ 基座(anthropic.lua)读不到时必须**响亮失败**,不得静默退化。
|
||||
// ★ 基座(anthropic.lua)读不到时必须在**安装期响亮失败**,不得静默退化。
|
||||
//
|
||||
// 反例危害:若"尽力而为"(只注入身份块、不作协议转换),会产出一个身份块
|
||||
// 看着正确的请求,而 tools/图像/流式结构全未转换 —— 上游可能 200,但模型
|
||||
// 拿不到工具定义,症状是"模型变笨了",与准入毫不相干,排查会完全跑偏。
|
||||
//
|
||||
// 判据看的是**适配器的真实产出**:把适配器目录指到空目录(基座不可得),
|
||||
// transform_request 必须返回错误而不是原样透传。
|
||||
// ⚠ 判据随设计更新(v2.0.37):基座**不再由适配器自己读**(适配器沙箱没有
|
||||
//
|
||||
// fs,那样必然失败且静默降级)。现在由**插件主体**读好并内联 ⇒ 基座不可得
|
||||
// 时应在**安装期**就失败(返回 err),而不是等到运行期才发现。
|
||||
func TestZcodeAdapterFailsLoudlyWithoutAnthropicBase(t *testing.T) {
|
||||
// 隔离目录:插件目录下没有兄弟 adapters/ 目录 ⇒ 基座读不到
|
||||
dir := t.TempDir()
|
||||
@ -267,7 +269,7 @@ func TestZcodeAdapterFailsLoudlyWithoutAnthropicBase(t *testing.T) {
|
||||
if err := os.MkdirAll(pluginDir, 0755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("LLMSPROXY_PLUGIN_DIR", pluginDir)
|
||||
t.Setenv("MODELROUTER_PLUGIN_DIR", pluginDir)
|
||||
|
||||
c := kernelTestCore(t)
|
||||
ps := c.Plugins()
|
||||
@ -278,21 +280,26 @@ func TestZcodeAdapterFailsLoudlyWithoutAnthropicBase(t *testing.T) {
|
||||
if err := ps.LoadSource("channels", src); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
vm := c.VM()
|
||||
if vm == nil {
|
||||
t.Fatal("VM 不可用")
|
||||
// 用空目录重新安装:基座不可得 ⇒ 必须**安装期**失败且原因点明 anthropic
|
||||
res, err := ps.Call("channels", "__adapter_install_probe", map[string]interface{}{
|
||||
"adapters_dir": filepath.Join(dir, "adapters-no-base"),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// 插件自装时若基座读不到,适配器仍应装载(身份块是静态的),
|
||||
// 但调用必须报错
|
||||
out, err := vm.Transform("channels-zcode", "transform_request",
|
||||
`{"model":"m","messages":[{"role":"user","content":"hi"}]}`)
|
||||
if err == nil {
|
||||
t.Fatalf("基座不可得时必须报错,却成功产出了:%s", out)
|
||||
j := res.(map[string]interface{})["json"].(map[string]interface{})
|
||||
zc, ok := j["zcode_reinstall"].(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("安装探针应回报 zcode_reinstall 结果,实得 %v", j)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "基座") {
|
||||
t.Fatalf("错误信息应点明基座问题(可定位),实得 %v", err)
|
||||
if zc["ok"] == true {
|
||||
t.Fatalf("基座不可得时安装必须失败(否则运行期才炸、且是逐帧炸),实得 %v", zc)
|
||||
}
|
||||
t.Logf("✓ 基座缺失时响亮失败: %v", err)
|
||||
msg, _ := zc["err"].(string)
|
||||
if !strings.Contains(msg, "anthropic") {
|
||||
t.Fatalf("错误信息应点明基座问题(可定位),实得 %q", msg)
|
||||
}
|
||||
t.Logf("✓ 基座缺失时安装期响亮失败: %s", msg)
|
||||
}
|
||||
|
||||
// ★ 日期块必须**幂等**:已以 <system-reminder> 开头则不重复插。
|
||||
|
||||
@ -10,7 +10,7 @@ import (
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// zcodeHarness:ZCode 两段式服务端中介授权流的测试台。
|
||||
|
||||
@ -11,17 +11,15 @@ import (
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/billing"
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/lua"
|
||||
"llmsproxy/internal/provider"
|
||||
"llmsproxy/internal/scheduler"
|
||||
"modelrouter/internal/config"
|
||||
"modelrouter/internal/lua"
|
||||
"modelrouter/internal/provider"
|
||||
"modelrouter/internal/scheduler"
|
||||
)
|
||||
|
||||
// Core owns the running configuration and adapters. mu guards every read and
|
||||
@ -97,9 +95,6 @@ func NewFromConfig(cfg *config.Config) (*Core, error) {
|
||||
for name, aud := range cfg.PluginVisibility {
|
||||
c.plugins.SetPluginVisibility(name, aud)
|
||||
}
|
||||
if err := c.plugins.SeedBundled(); err != nil {
|
||||
log.Printf("[core] seed bundled plugins: %v", err)
|
||||
}
|
||||
if err := c.plugins.LoadDir(); err != nil {
|
||||
log.Printf("[core] plugin dir: %v", err)
|
||||
}
|
||||
@ -111,7 +106,6 @@ func NewFromConfig(cfg *config.Config) (*Core, error) {
|
||||
log.Printf("[core] plugin %v %v loaded (hooks=%v ui=%v)",
|
||||
p["name"], p["version"], p["hooks"], p["ui"])
|
||||
}
|
||||
c.applyBillingDSL()
|
||||
c.store = config.NewStore(cfg.RuntimeFile)
|
||||
// Share one box between the runtime store and config.yaml so a single
|
||||
// master.key seals both files. config.Load left the config holding
|
||||
@ -150,9 +144,6 @@ func NewFromConfig(cfg *config.Config) (*Core, error) {
|
||||
if err := c.seedKeys(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := c.seedAuto(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := c.rebuildRegistry(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@ -195,39 +186,6 @@ func (c *Core) migrateFromRuntime() {
|
||||
}
|
||||
}
|
||||
|
||||
// seedAuto uses the existing auto rules from config.yaml, or creates an
|
||||
// initial chain from legacy per-source model priority (first run only).
|
||||
func (c *Core) seedAuto() error {
|
||||
if len(c.cfg.Auto) > 0 {
|
||||
return nil
|
||||
}
|
||||
type item struct {
|
||||
model string
|
||||
prio int
|
||||
}
|
||||
var flat []item
|
||||
for _, s := range c.mergedSources() {
|
||||
for _, m := range s.Models {
|
||||
if m.Kind == "image" {
|
||||
continue
|
||||
}
|
||||
flat = append(flat, item{m.ID, m.Priority})
|
||||
}
|
||||
}
|
||||
sort.SliceStable(flat, func(i, j int) bool {
|
||||
if flat[i].prio != flat[j].prio {
|
||||
return flat[i].prio > flat[j].prio
|
||||
}
|
||||
return flat[i].model < flat[j].model
|
||||
})
|
||||
entries := make([]config.ModelScope, 0, len(flat))
|
||||
for _, it := range flat {
|
||||
entries = append(entries, config.ModelScope{Model: it.model})
|
||||
}
|
||||
c.cfg.Auto = entries
|
||||
return c.cfg.Save()
|
||||
}
|
||||
|
||||
// seedKeys ensures config.yaml has admin keys for every gateway_keys entry
|
||||
// from the YAML config (once). After first save they become GWKey records
|
||||
// in c.cfg.Keys and gateway_keys is no longer read for auth.
|
||||
@ -877,9 +835,23 @@ func (c *Core) Reload() error {
|
||||
// ---- adapter management (web UI) ----
|
||||
|
||||
func (c *Core) ListAdapters() []lua.APIAdapter {
|
||||
c.syncHiddenRefs()
|
||||
return c.vm.ListAdapters()
|
||||
}
|
||||
|
||||
// syncHiddenRefs 把"当前被源引用的适配器"同步给 VM。被引用的 hidden=true 适配器
|
||||
// 必须保持可见(源编辑器要回显当前值);没被引用的隐藏适配器从列表消失。
|
||||
// 每次 ListAdapters 前调用:源集合是动态的(运行时可增删),推模型容易漏。
|
||||
func (c *Core) syncHiddenRefs() {
|
||||
refs := []string{}
|
||||
for _, s := range c.mergedSources() {
|
||||
if s.Adapter != "" {
|
||||
refs = append(refs, s.Adapter)
|
||||
}
|
||||
}
|
||||
c.vm.SetHiddenRefs(refs)
|
||||
}
|
||||
|
||||
func (c *Core) UploadAdapter(name, code string) error {
|
||||
if name == "" {
|
||||
return fmt.Errorf("adapter name required")
|
||||
@ -1033,63 +1005,6 @@ func (c *Core) Close() {
|
||||
}
|
||||
}
|
||||
|
||||
// applyBillingDSL compiles the configured billing profile and injects it into
|
||||
// the billing plugin.
|
||||
//
|
||||
// Configured pricing replaces the hand-written JSON blob that used to be PUT
|
||||
// through the state API. That path was used twice in production and both times
|
||||
// it "succeeded" while being wrong: once without the required `prices` wrapper
|
||||
// (silently wiping the accumulated totals), once with a peak-window shape the
|
||||
// plugin does not read (peak traffic billed at off-peak rates, no error). A
|
||||
// typed config with load-time validation turns both mistakes into startup
|
||||
// errors naming the file and field.
|
||||
//
|
||||
// A DSL with no profile, or one that fails to compile, leaves the plugin's
|
||||
// prices untouched: billing keeps running on whatever it had, and the problem
|
||||
// is logged rather than becoming a startup failure — the gateway must forward
|
||||
// even when its cost accounting is misconfigured.
|
||||
func (c *Core) applyBillingDSL() {
|
||||
ps := c.plugins
|
||||
if ps == nil || ps.Count() == 0 {
|
||||
return
|
||||
}
|
||||
// The billing plugin may simply not be installed; a DSL without it is a
|
||||
// no-op, not an error.
|
||||
var have bool
|
||||
for _, row := range ps.List() {
|
||||
if row["name"] == "billing" && row["loaded"] == true {
|
||||
have = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !have {
|
||||
return
|
||||
}
|
||||
dsl := c.cfg.BillingDSL
|
||||
if dsl == nil || len(dsl.Profiles) == 0 {
|
||||
return
|
||||
}
|
||||
if err := dsl.Validate(); err != nil {
|
||||
log.Printf("[billing] config invalid, profile not applied: %v", err)
|
||||
return
|
||||
}
|
||||
profile := dsl.Resolve(dsl.Active)
|
||||
if profile == nil {
|
||||
log.Printf("[billing] no billing profile resolved (active=%q)", dsl.Active)
|
||||
return
|
||||
}
|
||||
prices, err := billing.Compile(profile, c.cfg.Sources)
|
||||
if err != nil {
|
||||
log.Printf("[billing] profile %q not applied: %v", profile.ID, err)
|
||||
return
|
||||
}
|
||||
if err := ps.SetState("billing", map[string]interface{}{"prices": prices}); err != nil {
|
||||
log.Printf("[billing] applying profile %q: %v", profile.ID, err)
|
||||
return
|
||||
}
|
||||
log.Printf("[billing] profile %q applied (%d rules, %d sources)", profile.ID, len(profile.Rules), len(c.cfg.Sources))
|
||||
}
|
||||
|
||||
// ValidateScopeQuotas checks every scope entry's caps before they are stored.
|
||||
// A typo in a period must be rejected at write time rather than silently
|
||||
// becoming a never-resetting budget — the opposite of what was typed.
|
||||
|
||||
@ -5,7 +5,7 @@ import (
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"modelrouter/internal/config"
|
||||
)
|
||||
|
||||
// newTestConfig returns an in-memory config backed by temp files so
|
||||
@ -22,7 +22,7 @@ func newTestConfig(t *testing.T) *config.Config {
|
||||
DefaultModel: "AUTO",
|
||||
GatewayKeys: []string{"sk-gw-test"},
|
||||
Sources: []config.Source{
|
||||
{Name: "s1", BaseURL: "http://127.0.0.1:1/v1", Adapter: "openai", Models: []config.Model{{ID: "gpt-4o", Priority: 10}}},
|
||||
{Name: "s1", BaseURL: "http://127.0.0.1:1/v1", Adapter: "openai", Models: []config.Model{{ID: "gpt-4o"}}},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@ -8,7 +8,7 @@ import (
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// readBundledChannels 读内置 channels 插件源(供行为/变异测试共用)。
|
||||
|
||||
@ -6,8 +6,8 @@ import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/config"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// 内核动作实现(R4-c)
|
||||
|
||||
@ -6,8 +6,8 @@ import (
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/config"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// 本文件验证 R4-c/R4-d 的闭环:**插件自己驱动外部源的链接认证,然后把源接进内核**。
|
||||
@ -22,12 +22,12 @@ func kernelTestCore(t *testing.T) *Core {
|
||||
dir := t.TempDir()
|
||||
pluginDir := filepath.Join(dir, "plugins")
|
||||
// 插件的自有存储位置由插件自己决定(读 config.yaml 的 plugin_dir,或
|
||||
// LLMSPROXY_PLUGIN_DIR 覆盖)。测试**必须**指向临时目录 —— 否则插件会读到
|
||||
// /etc/llmsproxy/config.yaml 并写进**生产插件目录**(真实事故:一次全量测试
|
||||
// MODELROUTER_PLUGIN_DIR 覆盖)。测试**必须**指向临时目录 —— 否则插件会读到
|
||||
// /etc/modelrouter/config.yaml 并写进**生产插件目录**(真实事故:一次全量测试
|
||||
// 把 26 个测试账号写进了生产 .channels-store.json)。
|
||||
// 调用方已显式设定时尊重它:多次「重启」要共用同一目录才能测持久化。
|
||||
if os.Getenv("LLMSPROXY_PLUGIN_DIR") == "" {
|
||||
t.Setenv("LLMSPROXY_PLUGIN_DIR", pluginDir)
|
||||
if os.Getenv("MODELROUTER_PLUGIN_DIR") == "" {
|
||||
t.Setenv("MODELROUTER_PLUGIN_DIR", pluginDir)
|
||||
}
|
||||
// ★ 显示器号与端口用**高段号**,避开生产。
|
||||
//
|
||||
@ -36,8 +36,8 @@ func kernelTestCore(t *testing.T) *Core {
|
||||
// 并去抢,结果 `Xvfb 未就绪(/tmp/.X11-unix/X91 未出现)`,而生产上正跑着
|
||||
// 一套浏览器栈。判据本身没错,错的是**两个 /tmp 互不可见**而号段重叠。
|
||||
// 故测试整体挪到 300+。
|
||||
if os.Getenv("LLMSPROXY_XDISPLAY_BASE") == "" {
|
||||
t.Setenv("LLMSPROXY_XDISPLAY_BASE", "300")
|
||||
if os.Getenv("MODELROUTER_XDISPLAY_BASE") == "" {
|
||||
t.Setenv("MODELROUTER_XDISPLAY_BASE", "300")
|
||||
}
|
||||
cfg := &config.Config{
|
||||
Path: filepath.Join(dir, "config.yaml"),
|
||||
|
||||
@ -4,7 +4,7 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
"modelrouter/internal/lua"
|
||||
)
|
||||
|
||||
// TestNoKernelActionDeadlocksFromLua 对**每个**内核动作做同一条断言:
|
||||
|
||||
@ -7,7 +7,7 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"modelrouter/internal/config"
|
||||
)
|
||||
|
||||
// AutoChainFor sits on the request hot path and reads keyAutoChains with no
|
||||
@ -151,7 +151,7 @@ func TestAutoChainForConcurrentWithSourceChanges(t *testing.T) {
|
||||
name := fmt.Sprintf("churn%d", n)
|
||||
_ = c.AddSource(config.Source{
|
||||
Name: name, BaseURL: "http://127.0.0.1:9/v1", Adapter: "openai",
|
||||
Models: []config.Model{{ID: "m-" + name, Priority: 10}}},
|
||||
Models: []config.Model{{ID: "m-" + name}}},
|
||||
)
|
||||
_ = c.RemoveSource(name)
|
||||
}
|
||||
|
||||
@ -4,8 +4,8 @@ import (
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/scheduler"
|
||||
"modelrouter/internal/config"
|
||||
"modelrouter/internal/scheduler"
|
||||
)
|
||||
|
||||
// Per-key AUTO chains let an admin give one user their own scheduling chain
|
||||
@ -32,7 +32,7 @@ func perKeyTestConfig(t *testing.T) *config.Config {
|
||||
GatewayKeys: []string{"sk-gw-admin"},
|
||||
Sources: []config.Source{
|
||||
{Name: "s1", BaseURL: "http://127.0.0.1:1/v1", Adapter: "openai",
|
||||
Models: []config.Model{{ID: "gpt-4o", Priority: 10}, {ID: "gpt-4o-mini", Priority: 5}}},
|
||||
Models: []config.Model{{ID: "gpt-4o"}, {ID: "gpt-4o-mini"}}},
|
||||
},
|
||||
// Global chain points at gpt-4o.
|
||||
Auto: []config.ModelScope{{Model: "gpt-4o", Source: "s1", Tier: 1}},
|
||||
@ -198,7 +198,7 @@ func TestKeyAutoChainsRebuildOnSourceChange(t *testing.T) {
|
||||
// Add the source; the per-key chain must pick it up without a restart.
|
||||
cfg.Sources = append(cfg.Sources, config.Source{
|
||||
Name: "s2", BaseURL: "http://127.0.0.1:2/v1", Adapter: "openai",
|
||||
Models: []config.Model{{ID: "new-model", Priority: 10}},
|
||||
Models: []config.Model{{ID: "new-model"}},
|
||||
})
|
||||
if err := c.Reload(); err != nil {
|
||||
t.Fatalf("Reload: %v", err)
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user