mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-09-20 00:48:00 +00:00
feat: opencode zen adapter + first-run config generation, fix stats/stream bugs
- adapters/opencode.lua: opencode.ai zen free pool adapter — sends the opencode client User-Agent (zen fingerprints clients by UA; non-official clients hit FreeUsageLimitError); pairs with api_key: public - config: no config file ships in the repo; first run generates a default config at the -config path with a random admin key, loopback listen and a keyless zen source (config.EnsureDefault); remove config.example.yaml - lua: seed bundled adapters from the embedded FS instead of a hardcoded name list - ui: widen model kind select (chat was clipped to 'cha') - phase 5 bugfixes: stats ms/s bucket mixing, cleanScopes nil, ctx.Err guards, direct-path ModelAvailable, empty stream body failure, bestImageModel rewrite, transform failure recording, Core.mu, timer, effective model for tool-calls
This commit is contained in:
2
.gitignore
vendored
2
.gitignore
vendored
@ -2,4 +2,6 @@
|
|||||||
/llmsproxy
|
/llmsproxy
|
||||||
/config.yaml
|
/config.yaml
|
||||||
/adapters/
|
/adapters/
|
||||||
|
/runtime.json
|
||||||
|
/master.key
|
||||||
*.logmaster.key
|
*.logmaster.key
|
||||||
|
|||||||
39
README.md
39
README.md
@ -79,26 +79,30 @@ Groq、Mistral、Ollama、KimiCode…)通过 **Lua 适配器** 做协议转换
|
|||||||
## 快速开始
|
## 快速开始
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cp config.example.yaml config.yaml # 编辑你的源与 key
|
|
||||||
GOMODCACHE=... GOPROXY=off go build -tags luajit -o llmsproxy ./cmd/llmsproxy
|
GOMODCACHE=... GOPROXY=off go build -tags luajit -o llmsproxy ./cmd/llmsproxy
|
||||||
./llmsproxy -config config.yaml
|
./llmsproxy -config config.yaml # 首次运行自动生成默认配置并打印随机 admin key
|
||||||
```
|
```
|
||||||
|
|
||||||
> 依赖 [golua](https://github.com/aarzilli/golua)(LuaJIT 绑定)。**必须**带 `-tags luajit`
|
> 依赖 [golua](https://github.com/aarzilli/golua)(LuaJIT 绑定)。**必须**带 `-tags luajit`
|
||||||
> 构建,否则默认走内置超集 gopher-lua 路径(行为略有差异)。
|
> 构建,否则默认走内置超集 gopher-lua 路径(行为略有差异)。
|
||||||
|
|
||||||
|
> **配置不入库**:仓库不携带任何 config 文件(配置文件含密钥)。二进制首次运行时
|
||||||
|
> 会在 `-config` 指定的位置生成默认配置:随机 admin key(打印在启动日志中)、
|
||||||
|
> 仅绑定 `127.0.0.1:8080`、内置一个免 key 的 opencode zen 源可直接对话。
|
||||||
|
> 首次登录后请在 WebUI「密钥」页更换管理员密钥。
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# 无 key -> 401
|
# 无 key -> 401
|
||||||
curl http://127.0.0.1:8080/v1/models
|
curl http://127.0.0.1:8080/v1/models
|
||||||
|
|
||||||
# 单次
|
# 单次($KEY 换成首次启动日志打印的 admin key)
|
||||||
curl -H "Authorization: Bearer sk-gw-local-0001" \
|
curl -H "Authorization: Bearer $KEY" \
|
||||||
-d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"hi"}]}' \
|
-d '{"model":"AUTO","messages":[{"role":"user","content":"hi"}]}' \
|
||||||
http://127.0.0.1:8080/v1/chat/completions
|
http://127.0.0.1:8080/v1/chat/completions
|
||||||
|
|
||||||
# 流式
|
# 流式
|
||||||
curl -N -H "Authorization: Bearer sk-gw-local-0001" \
|
curl -N -H "Authorization: Bearer $KEY" \
|
||||||
-d '{"model":"deepseek-v4-flash","stream":true,"messages":[{"role":"user","content":"hi"}]}' \
|
-d '{"model":"AUTO","stream":true,"messages":[{"role":"user","content":"hi"}]}' \
|
||||||
http://127.0.0.1:8080/v1/chat/completions
|
http://127.0.0.1:8080/v1/chat/completions
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -106,11 +110,11 @@ curl -N -H "Authorization: Bearer sk-gw-local-0001" \
|
|||||||
|
|
||||||
## 配置
|
## 配置
|
||||||
|
|
||||||
见 [`config.example.yaml`](config.example.yaml)。核心字段:
|
首次运行在 `-config` 路径自动生成默认配置(默认 `./config.yaml`),核心字段:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
listen: 127.0.0.1:8080 # 网关监听地址(建议绑内网/回环)
|
listen: 127.0.0.1:8080 # 网关监听地址(建议绑内网/回环)
|
||||||
gateway_keys: [sk-gw-0001] # 初始 admin 密钥种子,仅首启时写入运行时存储用
|
gateway_keys: [sk-gw-<随机>] # 初始 admin 密钥(每次安装随机生成,仅首启写入运行时存储用)
|
||||||
default_model: AUTO # model 无法路由时自动按优先级链选源
|
default_model: AUTO # model 无法路由时自动按优先级链选源
|
||||||
adapter_dir: adapters # Lua 适配目录;目录不存在时首启创建并 seed 内置,存在则只读
|
adapter_dir: adapters # Lua 适配目录;目录不存在时首启创建并 seed 内置,存在则只读
|
||||||
runtime_file: runtime.json # WebUI 编辑的源/密钥/AUTO 链持久化到此文件
|
runtime_file: runtime.json # WebUI 编辑的源/密钥/AUTO 链持久化到此文件
|
||||||
@ -137,6 +141,12 @@ sources:
|
|||||||
timeout: 120s # 请求超时,默认 120s
|
timeout: 120s # 请求超时,默认 120s
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`config.yaml`、`runtime.json`、`master.key`、`adapters/` 均不提交版本库(见
|
||||||
|
`.gitignore`),请随备份带走——`master.key` 丢失后运行时密文无法解密。
|
||||||
|
|
||||||
|
运行文件生成在 `-config` 所在目录:`adapter_dir` / `runtime_file` 自动指向
|
||||||
|
配置文件旁边的 `adapters/` 与 `runtime.json`,从任意工作目录启动都可用。
|
||||||
|
|
||||||
运行时文件(`runtime_file`)中的敏感字段自动加密:
|
运行时文件(`runtime_file`)中的敏感字段自动加密:
|
||||||
|
|
||||||
- 加密算法 AES-256-GCM,格式 `enc:v1:<base64>`。
|
- 加密算法 AES-256-GCM,格式 `enc:v1:<base64>`。
|
||||||
@ -153,7 +163,7 @@ sources:
|
|||||||
- `gateway_keys` 配置只是**初始 admin 密钥种子**:首次启动迁移为运行时 admin
|
- `gateway_keys` 配置只是**初始 admin 密钥种子**:首次启动迁移为运行时 admin
|
||||||
key,之后不再参与鉴权管理。
|
key,之后不再参与鉴权管理。
|
||||||
- **重要:首次启动后请在 WebUI「密钥」页更换管理员密钥**——初始密钥明文写在
|
- **重要:首次启动后请在 WebUI「密钥」页更换管理员密钥**——初始密钥明文写在
|
||||||
`config.yaml` 里,继续使用存在被盗风险;用新密钥登录后删除初始密钥。
|
`config.yaml` 里(每次安装随机生成),继续使用存在被盗风险;用新密钥登录后删除初始密钥。
|
||||||
- WebUI **密钥页**可创建/删除密钥;每个密钥可指定 `admin`(管理全部)或
|
- WebUI **密钥页**可创建/删除密钥;每个密钥可指定 `admin`(管理全部)或
|
||||||
`user`(仅看自己的 key)角色,并配置**模型范围**(模型 + 源 + token 配额 +
|
`user`(仅看自己的 key)角色,并配置**模型范围**(模型 + 源 + token 配额 +
|
||||||
重置周期)。
|
重置周期)。
|
||||||
@ -270,12 +280,19 @@ return {
|
|||||||
|
|
||||||
### 内置适配器
|
### 内置适配器
|
||||||
|
|
||||||
`openai` `deepseek` `anthropic` `gemini` `github` `groq` `mistral` `ollama` `kimicode`。
|
`openai` `deepseek` `anthropic` `gemini` `github` `groq` `mistral` `ollama` `kimicode`
|
||||||
|
`opencode`。
|
||||||
|
|
||||||
`anthropic`/`gemini`/`ollama` 适配器内置多模态转换(`image_url` → 各自上游格式);若
|
`anthropic`/`gemini`/`ollama` 适配器内置多模态转换(`image_url` → 各自上游格式);若
|
||||||
源启用 `disable_thinking`,`deepseek` 适配器会把 `extra_body.thinking.type` 置为
|
源启用 `disable_thinking`,`deepseek` 适配器会把 `extra_body.thinking.type` 置为
|
||||||
`disabled`。
|
`disabled`。
|
||||||
|
|
||||||
|
**opencode** 适配器面向 opencode.ai zen 免费池(`https://opencode.ai/zen/v1`):
|
||||||
|
zen 按 User-Agent 指纹识别官方客户端并分流——非官方 UA 的请求(curl、Go 默认 UA)
|
||||||
|
会被丢进匿名池触发 `FreeUsageLimitError`。该适配器固定发送 opencode 客户端 UA,
|
||||||
|
配合 `api_key: "public"`(官方无 key 客户端实际发送 `Bearer public`)即可走免费池,
|
||||||
|
零成本接入 `deepseek-v4-flash-free` 等免费模型。
|
||||||
|
|
||||||
**kimicode** 是展示 `build_headers` 的样例:云端校验调用方 app,需要按
|
**kimicode** 是展示 `build_headers` 的样例:云端校验调用方 app,需要按
|
||||||
`meta.app_secret` 对时间戳+URL+请求体哈希做 HMAC 签名并附 `X-App-Sign` 等头。
|
`meta.app_secret` 对时间戳+URL+请求体哈希做 HMAC 签名并附 `X-App-Sign` 等头。
|
||||||
配好 `sources[].meta.{app_id, app_secret, app_agent}` 即可。
|
配好 `sources[].meta.{app_id, app_secret, app_agent}` 即可。
|
||||||
|
|||||||
32
README_EN.md
32
README_EN.md
@ -64,27 +64,32 @@ Extracted and independently evolved from the multi-source LLM adapter layer of
|
|||||||
## Quick start
|
## Quick start
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cp config.example.yaml config.yaml # edit your sources & keys
|
|
||||||
GOMODCACHE=... GOPROXY=off go build -tags luajit -o llmsproxy ./cmd/llmsproxy
|
GOMODCACHE=... GOPROXY=off go build -tags luajit -o llmsproxy ./cmd/llmsproxy
|
||||||
./llmsproxy -config config.yaml
|
./llmsproxy -config config.yaml # first run generates a default config and prints a random admin key
|
||||||
```
|
```
|
||||||
|
|
||||||
> Depends on [golua](https://github.com/aarzilli/golua) (LuaJIT bindings).
|
> Depends on [golua](https://github.com/aarzilli/golua) (LuaJIT bindings).
|
||||||
> You **must** build with `-tags luajit`; otherwise the built-in superset
|
> You **must** build with `-tags luajit`; otherwise the built-in superset
|
||||||
> gopher-lua path is used (behavior differs slightly).
|
> gopher-lua path is used (behavior differs slightly).
|
||||||
|
|
||||||
|
> **No config is shipped in the repo** (config files carry real keys). On
|
||||||
|
> first run the binary generates a default config at the `-config` path: a
|
||||||
|
> random admin key (printed to the startup log), loopback-only `127.0.0.1:8080`,
|
||||||
|
> and a keyless opencode zen source ready to chat. Rotate the admin key in the
|
||||||
|
> WebUI after first login.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# no key -> 401
|
# no key -> 401
|
||||||
curl http://127.0.0.1:8080/v1/models
|
curl http://127.0.0.1:8080/v1/models
|
||||||
|
|
||||||
# one-shot
|
# one-shot ($KEY = the admin key printed at first startup)
|
||||||
curl -H "Authorization: Bearer sk-gw-local-0001" \
|
curl -H "Authorization: Bearer $KEY" \
|
||||||
-d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"hi"}]}' \
|
-d '{"model":"AUTO","messages":[{"role":"user","content":"hi"}]}' \
|
||||||
http://127.0.0.1:8080/v1/chat/completions
|
http://127.0.0.1:8080/v1/chat/completions
|
||||||
|
|
||||||
# streaming
|
# streaming
|
||||||
curl -N -H "Authorization: Bearer sk-gw-local-0001" \
|
curl -N -H "Authorization: Bearer $KEY" \
|
||||||
-d '{"model":"deepseek-v4-flash","stream":true,"messages":[{"role":"user","content":"hi"}]}' \
|
-d '{"model":"AUTO","stream":true,"messages":[{"role":"user","content":"hi"}]}' \
|
||||||
http://127.0.0.1:8080/v1/chat/completions
|
http://127.0.0.1:8080/v1/chat/completions
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -93,11 +98,11 @@ gateway key as the API key.
|
|||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
See [`config.example.yaml`](config.example.yaml). Core fields:
|
Generated on first run at the `-config` path (default `./config.yaml`). Core fields:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
listen: 127.0.0.1:8080 # bind address (keep internal/loopback)
|
listen: 127.0.0.1:8080 # bind address (keep internal/loopback)
|
||||||
gateway_keys: [sk-gw-0001] # initial admin key seed, migrated to the store on first start
|
gateway_keys: [sk-gw-<random>] # initial admin key seed (random per install), migrated to the store on first start
|
||||||
default_model: AUTO # when model is unroutable, follow the AUTO chain
|
default_model: AUTO # when model is unroutable, follow the AUTO chain
|
||||||
adapter_dir: adapters # Lua adapter dir; created+seeded if missing, read-only otherwise
|
adapter_dir: adapters # Lua adapter dir; created+seeded if missing, read-only otherwise
|
||||||
runtime_file: runtime.json # WebUI-edited sources/keys/AUTO chain persist here
|
runtime_file: runtime.json # WebUI-edited sources/keys/AUTO chain persist here
|
||||||
@ -282,12 +287,19 @@ Shared helpers: `hmac_sha256_hex(key, data)`, `sha256_hex(data)`,
|
|||||||
### Built-in adapters
|
### Built-in adapters
|
||||||
|
|
||||||
`openai` `deepseek` `anthropic` `gemini` `github` `groq` `mistral` `ollama`
|
`openai` `deepseek` `anthropic` `gemini` `github` `groq` `mistral` `ollama`
|
||||||
`kimicode`.
|
`kimicode` `opencode`.
|
||||||
|
|
||||||
`anthropic`/`gemini`/`ollama` include multimodal conversion
|
`anthropic`/`gemini`/`ollama` include multimodal conversion
|
||||||
(`image_url` → their native format); with `disable_thinking` the `deepseek`
|
(`image_url` → their native format); with `disable_thinking` the `deepseek`
|
||||||
adapter sets `extra_body.thinking.type` to `disabled`.
|
adapter sets `extra_body.thinking.type` to `disabled`.
|
||||||
|
|
||||||
|
**opencode** targets the opencode.ai zen free pool
|
||||||
|
(`https://opencode.ai/zen/v1`): zen fingerprints clients by User-Agent and
|
||||||
|
routes non-official UAs (curl, Go's default) into an anonymous pool that
|
||||||
|
hits `FreeUsageLimitError`. The adapter always sends the opencode client UA;
|
||||||
|
combined with `api_key: "public"` (the keyless official client actually sends
|
||||||
|
`Bearer public`) it gets the free pool, e.g. `deepseek-v4-flash-free`.
|
||||||
|
|
||||||
**kimicode** demonstrates `build_headers`: the cloud validates the calling
|
**kimicode** demonstrates `build_headers`: the cloud validates the calling
|
||||||
app, so you HMAC-sign timestamp+URL+body with `meta.app_secret` and add
|
app, so you HMAC-sign timestamp+URL+body with `meta.app_secret` and add
|
||||||
`X-App-Sign`-style headers. Configure `sources[].meta.{app_id, app_secret,
|
`X-App-Sign`-style headers. Configure `sources[].meta.{app_id, app_secret,
|
||||||
|
|||||||
@ -16,6 +16,7 @@ import (
|
|||||||
"syscall"
|
"syscall"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"llmsproxy/internal/config"
|
||||||
"llmsproxy/internal/core"
|
"llmsproxy/internal/core"
|
||||||
"llmsproxy/internal/gateway"
|
"llmsproxy/internal/gateway"
|
||||||
)
|
)
|
||||||
@ -24,12 +25,23 @@ func main() {
|
|||||||
cfgPath := flag.String("config", "config.yaml", "path to gateway config file")
|
cfgPath := flag.String("config", "config.yaml", "path to gateway config file")
|
||||||
flag.Parse()
|
flag.Parse()
|
||||||
|
|
||||||
|
created, err := config.EnsureDefault(*cfgPath)
|
||||||
|
if err != nil {
|
||||||
|
log.Fatalf("[llmsproxy] config: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
c, err := core.New(*cfgPath)
|
c, err := core.New(*cfgPath)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
log.Fatalf("[llmsproxy] core: %v", err)
|
log.Fatalf("[llmsproxy] core: %v", err)
|
||||||
}
|
}
|
||||||
defer c.Close()
|
defer c.Close()
|
||||||
|
|
||||||
|
if created {
|
||||||
|
for _, k := range c.GatewayKeys() {
|
||||||
|
log.Printf("[llmsproxy] generated default config at %s — admin key: %s (rotate it in the WebUI after first login)", *cfgPath, k)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
gw, err := gateway.New(c, c.GatewayKeys())
|
gw, err := gateway.New(c, c.GatewayKeys())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
log.Fatalf("[llmsproxy] gateway: %v", err)
|
log.Fatalf("[llmsproxy] gateway: %v", err)
|
||||||
|
|||||||
@ -1,84 +0,0 @@
|
|||||||
# llmsproxy — 统一 OpenAI 兼容网关配置
|
|
||||||
|
|
||||||
# 网关监听地址(默认 :8080,建议绑内网/回环)
|
|
||||||
listen: 127.0.0.1:8080
|
|
||||||
|
|
||||||
# 网关自身鉴权密钥。现在为「多密钥」架构:此项仅作为初始 admin 密钥种子,
|
|
||||||
# 首次启动写入运行时存储(runtime_file 的 keys 字段,加密存储)。之后请在
|
|
||||||
# WebUI「密钥」页创建、删除密钥并配置其模型范围。留空 = 首启无 admin 密钥。
|
|
||||||
# 注意:首次启动后应立即在 WebUI 更换管理员密钥,并删除本初始种子——它明文
|
|
||||||
# 写在配置文件里,存在被窃取风险。
|
|
||||||
gateway_keys:
|
|
||||||
- sk-gw-local-0001
|
|
||||||
|
|
||||||
# 默认模型选择:具体模型 id 或 AUTO(AUTO 走优先级页保存的 AUTO 链)
|
|
||||||
default_model: AUTO
|
|
||||||
|
|
||||||
# Lua 适配器目录(默认 adapters/,首次启动自动写入内置适配器)
|
|
||||||
adapter_dir: adapters
|
|
||||||
|
|
||||||
# 运行时持久化文件(WebUI 新增/编辑的源会写入此文件,重启后仍生效)
|
|
||||||
runtime_file: runtime.json
|
|
||||||
|
|
||||||
# 可选 HTTPS:提供 PEM 证书与私钥文件路径后以 HTTPS 提供服务;留空 = 纯 HTTP。
|
|
||||||
# tls_cert_file: /etc/llmsproxy/tls/fullchain.pem
|
|
||||||
# tls_key_file: /etc/llmsproxy/tls/privkey.pem
|
|
||||||
|
|
||||||
# 可选:对外暴露的 base URL,用于“连接配置”复制片段中的 url。
|
|
||||||
# 留空默认根据请求自动推断(http/https + Host)。内网/反代后建议显式配置。
|
|
||||||
# public_base_url: https://gw.example.com/v1
|
|
||||||
|
|
||||||
# 全局并发上限(0 = 不限)
|
|
||||||
max_concurrent: 0
|
|
||||||
|
|
||||||
# ---- 上游 LLM 源列表 ----
|
|
||||||
# 每个源对应一个 Lua 适配器(adapter);同一个适配器可被多个源复用。
|
|
||||||
sources:
|
|
||||||
- name: deepseek
|
|
||||||
base_url: https://api.deepseek.com
|
|
||||||
api_key: sk-your-deepseek-key # 或用 api_key_env: SOME_ENV 引用环境变量
|
|
||||||
adapter: deepseek
|
|
||||||
max_concurrent: 8
|
|
||||||
models:
|
|
||||||
- id: deepseek-v4-flash
|
|
||||||
priority: 100 # YAML 源首次启动 seed 进 AUTO 链的初值
|
|
||||||
kind: chat
|
|
||||||
- id: deepseek-reasoner
|
|
||||||
priority: 60
|
|
||||||
kind: chat
|
|
||||||
meta: { thinking: true }
|
|
||||||
|
|
||||||
- name: ollama
|
|
||||||
base_url: http://127.0.0.1:11434
|
|
||||||
api_key: ""
|
|
||||||
adapter: ollama
|
|
||||||
endpoint: /api/chat
|
|
||||||
models:
|
|
||||||
- id: llama3
|
|
||||||
priority: 50
|
|
||||||
kind: chat
|
|
||||||
|
|
||||||
# 演示:KimiCode 校验调用方 app,通过 Lua build_headers 钩子做签名
|
|
||||||
- name: kimicode
|
|
||||||
base_url: https://api.moonshot.cn
|
|
||||||
api_key: sk-your-kimi-key
|
|
||||||
adapter: kimicode
|
|
||||||
models:
|
|
||||||
- id: kimi-k2
|
|
||||||
priority: 90
|
|
||||||
kind: chat
|
|
||||||
meta:
|
|
||||||
app_id: your-app-id
|
|
||||||
app_secret: your-app-secret
|
|
||||||
app_agent: code-agent
|
|
||||||
api_key: sk-your-kimi-key
|
|
||||||
|
|
||||||
# 生图示例:OpenAI 兼容生图源(flux/dall-e 等)
|
|
||||||
- name: imagegen
|
|
||||||
base_url: https://api.example.com
|
|
||||||
api_key: sk-image
|
|
||||||
adapter: openai
|
|
||||||
models:
|
|
||||||
- id: flux-1
|
|
||||||
priority: 80
|
|
||||||
kind: image
|
|
||||||
@ -6,6 +6,7 @@ import (
|
|||||||
"encoding/json"
|
"encoding/json"
|
||||||
"fmt"
|
"fmt"
|
||||||
"os"
|
"os"
|
||||||
|
"path/filepath"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"gopkg.in/yaml.v3"
|
"gopkg.in/yaml.v3"
|
||||||
@ -63,8 +64,13 @@ type Source struct {
|
|||||||
QueueTimeout time.Duration `yaml:"queue_timeout" json:"-"`
|
QueueTimeout time.Duration `yaml:"queue_timeout" json:"-"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// Load reads and validates a config file.
|
// Load reads and validates a config file. When the file does not exist yet a
|
||||||
|
// default config is generated at that path first (first-run bootstrap), so a
|
||||||
|
// fresh binary just works: `llmsproxy -config /path/to/config.yaml`.
|
||||||
func Load(path string) (*Config, error) {
|
func Load(path string) (*Config, error) {
|
||||||
|
if _, err := EnsureDefault(path); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
data, err := os.ReadFile(path)
|
data, err := os.ReadFile(path)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@ -80,6 +86,63 @@ func Load(path string) (*Config, error) {
|
|||||||
return &cfg, nil
|
return &cfg, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// EnsureDefault creates a default config file at path when it does not exist
|
||||||
|
// yet and returns whether it was created. The repo ships no config file
|
||||||
|
// (config files carry real keys); the binary generates one per install with a
|
||||||
|
// fresh random admin key. An existing file is never touched.
|
||||||
|
func EnsureDefault(path string) (bool, error) {
|
||||||
|
if _, err := os.Stat(path); err == nil {
|
||||||
|
return false, nil
|
||||||
|
} else if !os.IsNotExist(err) {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
if err := writeDefaultConfig(path); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// writeDefaultConfig writes a minimal, safe-by-default config: loopback-only
|
||||||
|
// listen, a fresh random admin key, and a no-key zen source that works out of
|
||||||
|
// the box. adapter_dir / runtime_file live next to the config file so the
|
||||||
|
// binary works regardless of the working directory it is started from.
|
||||||
|
func writeDefaultConfig(path string) error {
|
||||||
|
key, err := NewGatewayKey()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("generate gateway key: %w", err)
|
||||||
|
}
|
||||||
|
dir := filepath.Dir(path)
|
||||||
|
abs, err := filepath.Abs(dir)
|
||||||
|
if err != nil {
|
||||||
|
abs = dir
|
||||||
|
}
|
||||||
|
cfg := Config{
|
||||||
|
Listen: "127.0.0.1:8080",
|
||||||
|
GatewayKeys: []string{key},
|
||||||
|
DefaultModel: "AUTO",
|
||||||
|
AdapterDir: filepath.Join(abs, "adapters"),
|
||||||
|
RuntimeFile: filepath.Join(abs, "runtime.json"),
|
||||||
|
Sources: []Source{
|
||||||
|
{
|
||||||
|
Name: "zen",
|
||||||
|
BaseURL: "https://opencode.ai/zen/v1",
|
||||||
|
APIKey: "public", // zen 免费池:官方无 key 客户端实际发送 Bearer public
|
||||||
|
Adapter: "opencode",
|
||||||
|
Models: []Model{{ID: "deepseek-v4-flash-free", Priority: 100, Kind: "chat"}},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
out, err := yaml.Marshal(&cfg)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("marshal default config: %w", err)
|
||||||
|
}
|
||||||
|
// The config holds the plaintext admin key — restrict permissions.
|
||||||
|
if err := os.MkdirAll(abs, 0755); err != nil {
|
||||||
|
return fmt.Errorf("mkdir config dir: %w", err)
|
||||||
|
}
|
||||||
|
return os.WriteFile(path, out, 0600)
|
||||||
|
}
|
||||||
|
|
||||||
// RemoveSourceFromYAML deletes the named source entry from the config file so
|
// RemoveSourceFromYAML deletes the named source entry from the config file so
|
||||||
// the delete is a real one (no tombstone needed). Uses yaml.Node to preserve
|
// the delete is a real one (no tombstone needed). Uses yaml.Node to preserve
|
||||||
// the rest of the file's comments and formatting.
|
// the rest of the file's comments and formatting.
|
||||||
|
|||||||
@ -3,6 +3,7 @@ package config
|
|||||||
import (
|
import (
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"runtime"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
)
|
)
|
||||||
@ -59,6 +60,89 @@ sources:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestEnsureDefaultGeneratesOnMissingFile(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
path := filepath.Join(dir, "nested", "config.yaml")
|
||||||
|
created, err := EnsureDefault(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ensure: %v", err)
|
||||||
|
}
|
||||||
|
if !created {
|
||||||
|
t.Fatal("expected creation for missing file")
|
||||||
|
}
|
||||||
|
raw, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read generated: %v", err)
|
||||||
|
}
|
||||||
|
if fi, err := os.Stat(path); err != nil || (runtime.GOOS != "windows" && fi.Mode().Perm() != 0600) {
|
||||||
|
t.Fatalf("generated config must be 0600 (holds plaintext key), got %v", fi.Mode().Perm())
|
||||||
|
}
|
||||||
|
// generated file must load and be safe-by-default
|
||||||
|
cfg, err := Load(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("load generated: %v", err)
|
||||||
|
}
|
||||||
|
if len(cfg.GatewayKeys) != 1 || !strings.HasPrefix(cfg.GatewayKeys[0], "sk-gw-") {
|
||||||
|
t.Fatalf("gateway keys = %v", cfg.GatewayKeys)
|
||||||
|
}
|
||||||
|
if strings.Contains(string(raw), cfg.GatewayKeys[0]) == false {
|
||||||
|
t.Fatal("generated key must be written into the file")
|
||||||
|
}
|
||||||
|
if cfg.Listen != "127.0.0.1:8080" {
|
||||||
|
t.Fatalf("listen = %q, want loopback-only", cfg.Listen)
|
||||||
|
}
|
||||||
|
if len(cfg.Sources) != 1 || cfg.Sources[0].Name != "zen" || cfg.Sources[0].Adapter != "opencode" {
|
||||||
|
t.Fatalf("default sources = %+v", cfg.Sources)
|
||||||
|
}
|
||||||
|
if cfg.Sources[0].APIKey != "public" {
|
||||||
|
t.Fatalf("zen api_key = %q", cfg.Sources[0].APIKey)
|
||||||
|
}
|
||||||
|
if cfg.Sources[0].Models[0].ID != "deepseek-v4-flash-free" {
|
||||||
|
t.Fatalf("default model = %+v", cfg.Sources[0].Models)
|
||||||
|
}
|
||||||
|
// adapter_dir / runtime_file resolve next to the config file
|
||||||
|
if !strings.HasPrefix(cfg.AdapterDir, dir) || !strings.HasPrefix(cfg.RuntimeFile, dir) {
|
||||||
|
t.Fatalf("paths must live next to the config: adapter_dir=%s runtime_file=%s", cfg.AdapterDir, cfg.RuntimeFile)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsureDefaultDoesNotOverwriteExisting(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
path := filepath.Join(dir, "cfg.yaml")
|
||||||
|
content := "listen: 127.0.0.1:9999\ngateway_keys: [sk-keep]\n"
|
||||||
|
if err := os.WriteFile(path, []byte(content), 0644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
created, err := EnsureDefault(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ensure: %v", err)
|
||||||
|
}
|
||||||
|
if created {
|
||||||
|
t.Fatal("existing file must not be reported as created")
|
||||||
|
}
|
||||||
|
raw, _ := os.ReadFile(path)
|
||||||
|
if string(raw) != content {
|
||||||
|
t.Fatalf("existing file was overwritten: %q", raw)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewGatewayKeyIsRandom(t *testing.T) {
|
||||||
|
a, err := NewGatewayKey()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
b, err := NewGatewayKey()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if a == b {
|
||||||
|
t.Fatal("two generated keys must differ")
|
||||||
|
}
|
||||||
|
if !strings.HasPrefix(a, "sk-gw-") || len(a) != len("sk-gw-")+32 {
|
||||||
|
t.Fatalf("unexpected key format: %q", a)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestApplyDefaultsDuplicateSource(t *testing.T) {
|
func TestApplyDefaultsDuplicateSource(t *testing.T) {
|
||||||
cfg := Config{Sources: []Source{
|
cfg := Config{Sources: []Source{
|
||||||
{Name: "a", BaseURL: "http://x", Models: []Model{{ID: "m1"}}},
|
{Name: "a", BaseURL: "http://x", Models: []Model{{ID: "m1"}}},
|
||||||
|
|||||||
@ -15,6 +15,16 @@ import (
|
|||||||
|
|
||||||
var encPrefix = "enc:v1:"
|
var encPrefix = "enc:v1:"
|
||||||
|
|
||||||
|
// NewGatewayKey generates a fresh random gateway admin key with the
|
||||||
|
// "sk-gw-" prefix, used when bootstrapping a default config on first run.
|
||||||
|
func NewGatewayKey() (string, error) {
|
||||||
|
b := make([]byte, 16)
|
||||||
|
if _, err := rand.Read(b); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return "sk-gw-" + hex.EncodeToString(b), nil
|
||||||
|
}
|
||||||
|
|
||||||
// SecretBox encrypts and decrypts sensitive values (upstream API keys,
|
// SecretBox encrypts and decrypts sensitive values (upstream API keys,
|
||||||
// custom header values, gateway keys) for persist-time protection. The
|
// custom header values, gateway keys) for persist-time protection. The
|
||||||
// master key comes from the LLMS_PROXY_MASTER_KEY environment variable
|
// master key comes from the LLMS_PROXY_MASTER_KEY environment variable
|
||||||
|
|||||||
@ -11,6 +11,7 @@ import (
|
|||||||
"path/filepath"
|
"path/filepath"
|
||||||
"sort"
|
"sort"
|
||||||
"strings"
|
"strings"
|
||||||
|
"sync"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@ -20,8 +21,12 @@ import (
|
|||||||
"llmsproxy/internal/scheduler"
|
"llmsproxy/internal/scheduler"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Core owns the running configuration and adapters.
|
// Core owns the running configuration and adapters. mu guards every read and
|
||||||
|
// write of c.cfg (keys / sources / auto rules) from the management API while
|
||||||
|
// request paths look keys up concurrently; the AUTO chain itself is swapped
|
||||||
|
// atomically and needs no lock.
|
||||||
type Core struct {
|
type Core struct {
|
||||||
|
mu sync.Mutex
|
||||||
cfg *config.Config
|
cfg *config.Config
|
||||||
vm *lua.VM
|
vm *lua.VM
|
||||||
store *config.Store
|
store *config.Store
|
||||||
@ -192,6 +197,8 @@ func (c *Core) PublicBaseURL() string { return c.cfg.PublicBaseURL }
|
|||||||
|
|
||||||
// ListKeys returns all gateway keys (admin view).
|
// ListKeys returns all gateway keys (admin view).
|
||||||
func (c *Core) ListKeys() []config.GWKey {
|
func (c *Core) ListKeys() []config.GWKey {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
out := make([]config.GWKey, len(c.cfg.Keys))
|
out := make([]config.GWKey, len(c.cfg.Keys))
|
||||||
copy(out, c.cfg.Keys)
|
copy(out, c.cfg.Keys)
|
||||||
return out
|
return out
|
||||||
@ -199,6 +206,8 @@ func (c *Core) ListKeys() []config.GWKey {
|
|||||||
|
|
||||||
// FindKey looks up a gateway key record by its secret value.
|
// FindKey looks up a gateway key record by its secret value.
|
||||||
func (c *Core) FindKey(key string) (config.GWKey, bool) {
|
func (c *Core) FindKey(key string) (config.GWKey, bool) {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
for _, k := range c.cfg.Keys {
|
for _, k := range c.cfg.Keys {
|
||||||
if k.Key == key {
|
if k.Key == key {
|
||||||
return k, true
|
return k, true
|
||||||
@ -209,6 +218,8 @@ func (c *Core) FindKey(key string) (config.GWKey, bool) {
|
|||||||
|
|
||||||
// CreateKey builds a new random gateway key and persists it to config.yaml.
|
// CreateKey builds a new random gateway key and persists it to config.yaml.
|
||||||
func (c *Core) CreateKey(name, role string, models []config.ModelScope, note string) (config.GWKey, error) {
|
func (c *Core) CreateKey(name, role string, models []config.ModelScope, note string) (config.GWKey, error) {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
models = cleanScopes(models)
|
models = cleanScopes(models)
|
||||||
key := make([]byte, 16)
|
key := make([]byte, 16)
|
||||||
if _, err := rand.Read(key); err != nil {
|
if _, err := rand.Read(key); err != nil {
|
||||||
@ -234,6 +245,8 @@ func (c *Core) CreateKey(name, role string, models []config.ModelScope, note str
|
|||||||
|
|
||||||
// UpdateKey mutates a key's name/role/model scope and persists it.
|
// UpdateKey mutates a key's name/role/model scope and persists it.
|
||||||
func (c *Core) UpdateKey(key, name, role string, models []config.ModelScope, note string) (config.GWKey, error) {
|
func (c *Core) UpdateKey(key, name, role string, models []config.ModelScope, note string) (config.GWKey, error) {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
for i, k := range c.cfg.Keys {
|
for i, k := range c.cfg.Keys {
|
||||||
if k.Key == key {
|
if k.Key == key {
|
||||||
if name != "" {
|
if name != "" {
|
||||||
@ -242,7 +255,11 @@ func (c *Core) UpdateKey(key, name, role string, models []config.ModelScope, not
|
|||||||
if role == "admin" || role == "user" {
|
if role == "admin" || role == "user" {
|
||||||
c.cfg.Keys[i].Role = role
|
c.cfg.Keys[i].Role = role
|
||||||
}
|
}
|
||||||
c.cfg.Keys[i].Models = cleanScopes(models)
|
// models == nil means the caller did not provide a scope (leave
|
||||||
|
// the existing one untouched); an explicit [] clears it.
|
||||||
|
if models != nil {
|
||||||
|
c.cfg.Keys[i].Models = cleanScopes(models)
|
||||||
|
}
|
||||||
c.cfg.Keys[i].Note = note
|
c.cfg.Keys[i].Note = note
|
||||||
if err := c.saveConfig(); err != nil {
|
if err := c.saveConfig(); err != nil {
|
||||||
return config.GWKey{}, err
|
return config.GWKey{}, err
|
||||||
@ -255,6 +272,8 @@ func (c *Core) UpdateKey(key, name, role string, models []config.ModelScope, not
|
|||||||
|
|
||||||
// DeleteKey removes a key record; returns false if it did not exist.
|
// DeleteKey removes a key record; returns false if it did not exist.
|
||||||
func (c *Core) DeleteKey(key string) (bool, error) {
|
func (c *Core) DeleteKey(key string) (bool, error) {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
for i, k := range c.cfg.Keys {
|
for i, k := range c.cfg.Keys {
|
||||||
if k.Key == key {
|
if k.Key == key {
|
||||||
c.cfg.Keys = append(c.cfg.Keys[:i], c.cfg.Keys[i+1:]...)
|
c.cfg.Keys = append(c.cfg.Keys[:i], c.cfg.Keys[i+1:]...)
|
||||||
@ -268,13 +287,20 @@ func (c *Core) DeleteKey(key string) (bool, error) {
|
|||||||
|
|
||||||
// AutoRules returns the AUTO scheduling slots in priority order.
|
// AutoRules returns the AUTO scheduling slots in priority order.
|
||||||
func (c *Core) AutoRules() []config.ModelScope {
|
func (c *Core) AutoRules() []config.ModelScope {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
out := make([]config.ModelScope, len(c.cfg.Auto))
|
out := make([]config.ModelScope, len(c.cfg.Auto))
|
||||||
copy(out, c.cfg.Auto)
|
copy(out, c.cfg.Auto)
|
||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// cleanScopes drops empty model entries and normalizes placeholder source
|
||||||
|
// names; it returns nil when no entries survive so an empty scope means
|
||||||
|
// "unrestricted" (nil) instead of a restrictive-but-empty list — a non-nil
|
||||||
|
// empty slice would 403 every model in-process yet become unrestricted again
|
||||||
|
// after a restart (config omits empty models with omitempty).
|
||||||
func cleanScopes(entries []config.ModelScope) []config.ModelScope {
|
func cleanScopes(entries []config.ModelScope) []config.ModelScope {
|
||||||
clean := make([]config.ModelScope, 0, len(entries))
|
var clean []config.ModelScope
|
||||||
for _, e := range entries {
|
for _, e := range entries {
|
||||||
if e.Model == "" {
|
if e.Model == "" {
|
||||||
continue
|
continue
|
||||||
@ -292,6 +318,8 @@ func cleanScopes(entries []config.ModelScope) []config.ModelScope {
|
|||||||
// kept, so a reliably good model keeps its edge while an edited chain applies
|
// kept, so a reliably good model keeps its edge while an edited chain applies
|
||||||
// immediately.
|
// immediately.
|
||||||
func (c *Core) SaveAutoRules(entries []config.ModelScope) error {
|
func (c *Core) SaveAutoRules(entries []config.ModelScope) error {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
c.cfg.Auto = cleanScopes(entries)
|
c.cfg.Auto = cleanScopes(entries)
|
||||||
if err := c.saveConfig(); err != nil {
|
if err := c.saveConfig(); err != nil {
|
||||||
return err
|
return err
|
||||||
@ -396,7 +424,10 @@ func (c *Core) rebuildRegistry() error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// buildAutoChain rebuilds the AUTO chain snapshot from config.yaml rules
|
// buildAutoChain rebuilds the AUTO chain snapshot from config.yaml rules
|
||||||
// against the current providers.
|
// against the current providers. Slot model ids are normalized to the exact
|
||||||
|
// configured spelling (case-insensitive match), otherwise ModelFor would fall
|
||||||
|
// back to the source's best chat model and cooldown/quota bookkeeping would
|
||||||
|
// key on a name that never matches.
|
||||||
func (c *Core) buildAutoChain() {
|
func (c *Core) buildAutoChain() {
|
||||||
prov := func(model, source string) scheduler.Provider {
|
prov := func(model, source string) scheduler.Provider {
|
||||||
p := c.registry.ProviderForSlot(model, source)
|
p := c.registry.ProviderForSlot(model, source)
|
||||||
@ -411,20 +442,26 @@ func (c *Core) buildAutoChain() {
|
|||||||
rules := c.cfg.Auto
|
rules := c.cfg.Auto
|
||||||
sr := make([]scheduler.Rule, 0, len(rules))
|
sr := make([]scheduler.Rule, 0, len(rules))
|
||||||
for _, e := range rules {
|
for _, e := range rules {
|
||||||
r := scheduler.Rule{
|
model, source := e.Model, e.Source
|
||||||
Model: e.Model,
|
if p := c.registry.ProviderForSlot(e.Model, e.Source); p != nil {
|
||||||
Source: e.Source,
|
if exact := p.ModelIDFold(e.Model); exact != "" {
|
||||||
|
model = exact
|
||||||
|
}
|
||||||
|
if m := p.ModelByID(model); m != nil && m.Kind == "image" {
|
||||||
|
continue // image-kind models never join the chat AUTO chain
|
||||||
|
}
|
||||||
|
if source == "" {
|
||||||
|
source = p.Name()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sr = append(sr, scheduler.Rule{
|
||||||
|
Model: model,
|
||||||
|
Source: source,
|
||||||
Tier: e.Tier,
|
Tier: e.Tier,
|
||||||
Quota: e.TokenQuota,
|
Quota: e.TokenQuota,
|
||||||
Period: e.Period,
|
Period: e.Period,
|
||||||
Hours: e.Hours,
|
Hours: e.Hours,
|
||||||
}
|
})
|
||||||
if r.Source == "" {
|
|
||||||
if p := c.registry.ProviderForSlot(e.Model, ""); p != nil {
|
|
||||||
r.Source = p.Name()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
sr = append(sr, r)
|
|
||||||
}
|
}
|
||||||
c.autoChain.Store(scheduler.BuildChain(sr, prov))
|
c.autoChain.Store(scheduler.BuildChain(sr, prov))
|
||||||
}
|
}
|
||||||
@ -469,6 +506,8 @@ func (c *Core) AutoSlotStates() []AutoSlotState {
|
|||||||
|
|
||||||
// Reload re-reads the runtime store and rebuilds sources.
|
// Reload re-reads the runtime store and rebuilds sources.
|
||||||
func (c *Core) Reload() error {
|
func (c *Core) Reload() error {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
if err := c.store.Load(); err != nil {
|
if err := c.store.Load(); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@ -505,6 +544,8 @@ func (c *Core) RemoveAdapter(name string) error {
|
|||||||
// ---- source management (web UI) ----
|
// ---- source management (web UI) ----
|
||||||
|
|
||||||
func (c *Core) AddSource(src config.Source) error {
|
func (c *Core) AddSource(src config.Source) error {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
if err := normalizeSource(&src); err != nil {
|
if err := normalizeSource(&src); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@ -527,6 +568,8 @@ func (c *Core) AddSource(src config.Source) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (c *Core) RemoveSource(name string) error {
|
func (c *Core) RemoveSource(name string) error {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
for _, s := range c.cfg.Sources {
|
for _, s := range c.cfg.Sources {
|
||||||
if s.Name == name {
|
if s.Name == name {
|
||||||
if err := config.RemoveSourceFromYAML(c.cfg.Path, name); err != nil {
|
if err := config.RemoveSourceFromYAML(c.cfg.Path, name); err != nil {
|
||||||
@ -552,7 +595,11 @@ func (c *Core) removeCfgSource(name string) []config.Source {
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Core) Sources() []config.Source { return c.mergedSources() }
|
func (c *Core) Sources() []config.Source {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
|
return c.mergedSources()
|
||||||
|
}
|
||||||
|
|
||||||
func normalizeSource(s *config.Source) error {
|
func normalizeSource(s *config.Source) error {
|
||||||
if s.Name == "" || s.BaseURL == "" {
|
if s.Name == "" || s.BaseURL == "" {
|
||||||
|
|||||||
135
internal/core/core_test.go
Normal file
135
internal/core/core_test.go
Normal file
@ -0,0 +1,135 @@
|
|||||||
|
package core
|
||||||
|
|
||||||
|
import (
|
||||||
|
"path/filepath"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"llmsproxy/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
// newTestConfig returns an in-memory config backed by temp files so
|
||||||
|
// NewFromConfig (store load, key/auto seeding, registry build) is exercised
|
||||||
|
// like production.
|
||||||
|
func newTestConfig(t *testing.T) *config.Config {
|
||||||
|
t.Helper()
|
||||||
|
dir := t.TempDir()
|
||||||
|
return &config.Config{
|
||||||
|
Path: filepath.Join(dir, "config.yaml"),
|
||||||
|
AdapterDir: filepath.Join(dir, "adapters"),
|
||||||
|
RuntimeFile: filepath.Join(dir, "runtime.json"),
|
||||||
|
Listen: "127.0.0.1:0",
|
||||||
|
DefaultModel: "AUTO",
|
||||||
|
GatewayKeys: []string{"sk-gw-test"},
|
||||||
|
Sources: []config.Source{
|
||||||
|
{Name: "s1", BaseURL: "http://127.0.0.1:1/v1", Adapter: "openai", Models: []config.Model{{ID: "gpt-4o", Priority: 10}}},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func newTestCore(t *testing.T, cfg *config.Config) *Core {
|
||||||
|
t.Helper()
|
||||||
|
c, err := NewFromConfig(cfg)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("core: %v", err)
|
||||||
|
}
|
||||||
|
t.Cleanup(c.Close)
|
||||||
|
return c
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCleanScopesNilForEmpty(t *testing.T) {
|
||||||
|
if got := cleanScopes(nil); got != nil {
|
||||||
|
t.Fatalf("nil in must yield nil out, got %#v", got)
|
||||||
|
}
|
||||||
|
if got := cleanScopes([]config.ModelScope{{Model: ""}}); got != nil {
|
||||||
|
t.Fatalf("all-empty entries must yield nil, got %#v", got)
|
||||||
|
}
|
||||||
|
got := cleanScopes([]config.ModelScope{{Model: "m", Source: "undefined"}})
|
||||||
|
if len(got) != 1 || got[0].Source != "" || got[0].Model != "m" {
|
||||||
|
t.Fatalf("placeholder normalization wrong: %#v", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestCreateKeyWithoutScopeUnrestricted: a user key created without a model
|
||||||
|
// scope must be unrestricted in-process (nil), identical to what config.yaml
|
||||||
|
// omitempty produces after a restart — no 403 lock-out, no silent widening.
|
||||||
|
func TestCreateKeyWithoutScopeUnrestricted(t *testing.T) {
|
||||||
|
c := newTestCore(t, newTestConfig(t))
|
||||||
|
k, err := c.CreateKey("svc", "user", nil, "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("create: %v", err)
|
||||||
|
}
|
||||||
|
rec, ok := c.FindKey(k.Key)
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("key not found")
|
||||||
|
}
|
||||||
|
if rec.Models != nil {
|
||||||
|
t.Fatalf("user key without scope must be unrestricted, got %#v", rec.Models)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestUpdateKeyPreservesScopeWhenOmitted: PUT /api/keys without a models field
|
||||||
|
// must keep the existing scope; only an explicit [] clears it.
|
||||||
|
func TestUpdateKeyPreservesScopeWhenOmitted(t *testing.T) {
|
||||||
|
c := newTestCore(t, newTestConfig(t))
|
||||||
|
k, err := c.CreateKey("svc", "user", []config.ModelScope{{Model: "gpt-4o"}}, "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("create: %v", err)
|
||||||
|
}
|
||||||
|
upd, err := c.UpdateKey(k.Key, "renamed", "", nil, "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("update: %v", err)
|
||||||
|
}
|
||||||
|
if len(upd.Models) != 1 || upd.Models[0].Model != "gpt-4o" {
|
||||||
|
t.Fatalf("scope must survive an update without models, got %#v", upd.Models)
|
||||||
|
}
|
||||||
|
cleared, err := c.UpdateKey(k.Key, "", "", []config.ModelScope{}, "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("clear: %v", err)
|
||||||
|
}
|
||||||
|
if cleared.Models != nil {
|
||||||
|
t.Fatalf("explicit empty list must clear the scope, got %#v", cleared.Models)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestSaveAutoRulesNormalizesModelCase: a rule whose model name differs only
|
||||||
|
// in case must be normalized to the exact configured spelling so cooldown,
|
||||||
|
// quota windows and the upstream model id all agree.
|
||||||
|
func TestSaveAutoRulesNormalizesModelCase(t *testing.T) {
|
||||||
|
c := newTestCore(t, newTestConfig(t))
|
||||||
|
if err := c.SaveAutoRules([]config.ModelScope{{Model: "GPT-4O", Source: "s1", Tier: 1}}); err != nil {
|
||||||
|
t.Fatalf("save: %v", err)
|
||||||
|
}
|
||||||
|
ch := c.AutoChain()
|
||||||
|
if ch == nil || len(ch.Tiers) == 0 || len(ch.Tiers[0].Slots) == 0 {
|
||||||
|
t.Fatalf("chain empty: %#v", ch)
|
||||||
|
}
|
||||||
|
if got := ch.Tiers[0].Slots[0].Model; got != "gpt-4o" {
|
||||||
|
t.Fatalf("slot model must be normalized to the config spelling, got %q", got)
|
||||||
|
}
|
||||||
|
if got := ch.Tiers[0].Slots[0].Source; got != "s1" {
|
||||||
|
t.Fatalf("slot source wrong: %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestConcurrentKeyMutation exercises the c.cfg.Keys lock under -race: key
|
||||||
|
// creation/lookup/deletion from many goroutines at once.
|
||||||
|
func TestConcurrentKeyMutation(t *testing.T) {
|
||||||
|
c := newTestCore(t, newTestConfig(t))
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for i := 0; i < 16; i++ {
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
k, err := c.CreateKey("k", "user", nil, "")
|
||||||
|
if err == nil {
|
||||||
|
if rec, ok := c.FindKey(k.Key); ok && rec.Key != k.Key {
|
||||||
|
t.Errorf("wrong key returned")
|
||||||
|
}
|
||||||
|
c.UpdateKey(k.Key, "renamed", "", nil, "")
|
||||||
|
c.DeleteKey(k.Key)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
}
|
||||||
@ -100,7 +100,10 @@ func (g *Gateway) resolveCands(ctx context.Context, req *chatRequest) ([]*provid
|
|||||||
return nil, effective
|
return nil, effective
|
||||||
}
|
}
|
||||||
first := cands[0]
|
first := cands[0]
|
||||||
eff := first.ModelFor(model)
|
// use the prefix-stripped id (effective), never the raw "src:model" form:
|
||||||
|
// ModelFor does exact matching and would fall back to the source's best
|
||||||
|
// chat model for an unknown id
|
||||||
|
eff := first.ModelFor(effective)
|
||||||
if eff == "" {
|
if eff == "" {
|
||||||
eff = firstModel(first)
|
eff = firstModel(first)
|
||||||
}
|
}
|
||||||
|
|||||||
@ -210,20 +210,25 @@ func (s *Stats) aggregateLocked(r Req) {
|
|||||||
}
|
}
|
||||||
incStatus(a, strconv.Itoa(r.Status), r)
|
incStatus(a, strconv.Itoa(r.Status), r)
|
||||||
}
|
}
|
||||||
// window bucket for quota enforcement (per source-model pair, per unix hour)
|
// window bucket for quota enforcement (per source-model pair, per unix
|
||||||
|
// hour). r.Time is unix MILLISECONDS (audit format); hourSec is seconds,
|
||||||
|
// so convert before bucketing — otherwise the bucket width would be
|
||||||
|
// 3.6s and every WindowTokens cutoff comparison would be off by ~1000x.
|
||||||
tok := r.Prompt + r.Compl
|
tok := r.Prompt + r.Compl
|
||||||
if tok > 0 && r.Model != "" {
|
if tok > 0 && r.Model != "" {
|
||||||
key := r.Model
|
key := r.Model
|
||||||
if r.Source != "" {
|
if r.Source != "" {
|
||||||
key = r.Source + "::" + r.Model
|
key = r.Source + "::" + r.Model
|
||||||
}
|
}
|
||||||
h := r.Time / hourSec
|
h := (r.Time / 1000) / hourSec
|
||||||
hm := s.modelHour[key]
|
hm := s.modelHour[key]
|
||||||
if hm == nil {
|
if hm == nil {
|
||||||
hm = map[int64]int64{}
|
hm = map[int64]int64{}
|
||||||
s.modelHour[key] = hm
|
s.modelHour[key] = hm
|
||||||
}
|
}
|
||||||
hm[h] += tok
|
hm[h] += tok
|
||||||
|
// retention: 24*40 = 960 hourly buckets ≈ 40 days of history (covers
|
||||||
|
// the longest "month" quota window)
|
||||||
if len(hm) > 24*40 {
|
if len(hm) > 24*40 {
|
||||||
for k := range hm {
|
for k := range hm {
|
||||||
if k < h-24*40 {
|
if k < h-24*40 {
|
||||||
@ -324,10 +329,10 @@ func AutoPeriodSeconds(period string, hours int64) int64 {
|
|||||||
return 0
|
return 0
|
||||||
}
|
}
|
||||||
|
|
||||||
// WindowTokens returns the tokens billed for the model within the last `sec`
|
|
||||||
// seconds (0 = since forever).
|
|
||||||
// WindowTokens returns the tokens consumed for one model (optionally pinned
|
// WindowTokens returns the tokens consumed for one model (optionally pinned
|
||||||
// to a single source) within the window; sec <= 0 means all time.
|
// to a single source) within the window; sec <= 0 means all time. Buckets are
|
||||||
|
// whole unix hours, so a sliding window overcounts by up to one hour — an
|
||||||
|
// accepted truncation for quota enforcement.
|
||||||
func (s *Stats) WindowTokens(model, source string, sec int64) int64 {
|
func (s *Stats) WindowTokens(model, source string, sec int64) int64 {
|
||||||
s.mu.Lock()
|
s.mu.Lock()
|
||||||
defer s.mu.Unlock()
|
defer s.mu.Unlock()
|
||||||
|
|||||||
@ -6,6 +6,7 @@ import (
|
|||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestStatsByStatus(t *testing.T) {
|
func TestStatsByStatus(t *testing.T) {
|
||||||
@ -91,18 +92,21 @@ func TestAuditRotationRecords(t *testing.T) {
|
|||||||
func TestLoadAuditFullReplay(t *testing.T) {
|
func TestLoadAuditFullReplay(t *testing.T) {
|
||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
path := filepath.Join(dir, "audit.jsonl")
|
path := filepath.Join(dir, "audit.jsonl")
|
||||||
|
// timestamps relative to now: buckets are whole unix hours, so window
|
||||||
|
// assertions must not depend on the minute-of-hour of the test run.
|
||||||
|
now := time.Now()
|
||||||
lines := []string{
|
lines := []string{
|
||||||
`{"obj":"access","time":1699999999000,"key":"k","method":"GET","path":"/api/stats","status":200}`,
|
`{"obj":"access","time":1699999999000,"key":"k","method":"GET","path":"/api/stats","status":200}`,
|
||||||
`{"time":1700000000000,"key":"k","type":"chat","model":"m","source":"s","prompt_tokens":100,"completion_tokens":50,"latency_ms":10,"ok":true,"status":200}`,
|
fmt.Sprintf(`{"time":%d,"key":"k","type":"chat","model":"m","source":"s","prompt_tokens":100,"completion_tokens":50,"latency_ms":10,"ok":true,"status":200}`, now.Add(-65*time.Minute).UnixMilli()),
|
||||||
`{this is not valid json`,
|
`{this is not valid json`,
|
||||||
`{"time":1700003600000,"key":"k","type":"stream","model":"m","source":"s","prompt_tokens":200,"completion_tokens":20,"latency_ms":20,"ok":false,"status":503}`,
|
fmt.Sprintf(`{"time":%d,"key":"k","type":"stream","model":"m","source":"s","prompt_tokens":200,"completion_tokens":20,"latency_ms":20,"ok":false,"status":503}`, now.Add(-30*time.Minute).UnixMilli()),
|
||||||
"garbage-not-json\n",
|
"garbage-not-json\n",
|
||||||
`{"time":1700007200000,"key":"k","type":"chat","model":"m2","source":"s","prompt_tokens":7,"completion_tokens":3,"latency_ms":5,"ok":true,"status":200}`,
|
fmt.Sprintf(`{"time":%d,"key":"k","type":"chat","model":"m2","source":"s","prompt_tokens":7,"completion_tokens":3,"latency_ms":5,"ok":true,"status":200}`, now.Add(-30*time.Minute).UnixMilli()),
|
||||||
}
|
}
|
||||||
loaded := strings.Join(lines, "\n") + "\n" + strings.Repeat("x", 1<<18) + "\n"
|
loaded := strings.Join(lines, "\n") + "\n" + strings.Repeat("x", 1<<18) + "\n"
|
||||||
// oversized row at the END proves the scanner tolerates >64KB lines and
|
// oversized row at the END proves the scanner tolerates >64KB lines and
|
||||||
// still finishes the replay instead of truncating silently.
|
// still finishes the replay instead of truncating silently.
|
||||||
loaded += `{"time":1700010800000,"key":"k","type":"chat","model":"m","source":"s","prompt_tokens":1,"completion_tokens":1,"latency_ms":1,"ok":true,"status":200}` + "\n"
|
loaded += fmt.Sprintf(`{"time":%d,"key":"k","type":"chat","model":"m","source":"s","prompt_tokens":1,"completion_tokens":1,"latency_ms":1,"ok":true,"status":200}`, now.Add(-time.Minute).UnixMilli()) + "\n"
|
||||||
if err := os.WriteFile(path, []byte(loaded), 0644); err != nil {
|
if err := os.WriteFile(path, []byte(loaded), 0644); err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
@ -121,9 +125,24 @@ func TestLoadAuditFullReplay(t *testing.T) {
|
|||||||
if len(s.recs) != 4 {
|
if len(s.recs) != 4 {
|
||||||
t.Fatalf("ring must hold only real requests, got %d rows: %#v", len(s.recs), s.recs)
|
t.Fatalf("ring must hold only real requests, got %d rows: %#v", len(s.recs), s.recs)
|
||||||
}
|
}
|
||||||
// quota window rebuilt from full history
|
// quota window rebuilt from full history. All-time and multi-hour windows
|
||||||
if w := s.WindowTokens("m", "s", hourSec); w != 372 {
|
// must see everything; a 1h window must NOT return all records (the old
|
||||||
t.Fatalf("window tokens want 372, got %d", w)
|
// ms/seconds unit bug made any sec>0 window return everything), and must
|
||||||
|
// always include the row written one minute ago (current hour bucket).
|
||||||
|
if w := s.WindowTokens("m", "s", 0); w != 372 {
|
||||||
|
t.Fatalf("all-time window want 372, got %d", w)
|
||||||
|
}
|
||||||
|
if w := s.WindowTokens("m", "s", 3*hourSec); w != 372 {
|
||||||
|
t.Fatalf("3h window want 372, got %d", w)
|
||||||
|
}
|
||||||
|
if w := s.WindowTokens("m", "s", 24*hourSec); w != 372 {
|
||||||
|
t.Fatalf("24h window want 372, got %d", w)
|
||||||
|
}
|
||||||
|
if w := s.WindowTokens("m", "s", hourSec); w < 2 || w >= 372 {
|
||||||
|
t.Fatalf("1h window want [2,372), got %d", w)
|
||||||
|
}
|
||||||
|
if w := s.WindowTokens("m2", "s", 24*hourSec); w != 10 {
|
||||||
|
t.Fatalf("m2 24h window want 10, got %d", w)
|
||||||
}
|
}
|
||||||
// by_status only from requests (200 x3, 503 x1) — access line must not count
|
// by_status only from requests (200 x3, 503 x1) — access line must not count
|
||||||
if st := s.byStatus[200]; st == nil || st.Reqs != 3 {
|
if st := s.byStatus[200]; st == nil || st.Reqs != 3 {
|
||||||
|
|||||||
@ -205,7 +205,7 @@ label{display:block;font-size:12px;color:var(--muted);margin:12px 0 5px}
|
|||||||
.row{display:flex;gap:12px}.row>div{flex:1}
|
.row{display:flex;gap:12px}.row>div{flex:1}
|
||||||
.model-row{display:flex;gap:6px;align-items:center;width:100%}
|
.model-row{display:flex;gap:6px;align-items:center;width:100%}
|
||||||
.model-row .m-id{flex:1;min-width:0;width:0}
|
.model-row .m-id{flex:1;min-width:0;width:0}
|
||||||
.model-row .m-kind{flex:0 0 70px;width:70px}
|
.model-row .m-kind{flex:0 0 96px;width:96px}
|
||||||
.model-row .del{flex:0 0 auto;padding:4px 8px}
|
.model-row .del{flex:0 0 auto;padding:4px 8px}
|
||||||
.muted{color:var(--muted)}
|
.muted{color:var(--muted)}
|
||||||
.hidden,.hidden#tab-chat{display:none}
|
.hidden,.hidden#tab-chat{display:none}
|
||||||
|
|||||||
95
internal/lua/adapters/opencode.lua
Normal file
95
internal/lua/adapters/opencode.lua
Normal file
@ -0,0 +1,95 @@
|
|||||||
|
local adapter = {}
|
||||||
|
|
||||||
|
adapter.name = "opencode"
|
||||||
|
adapter.version = "1.0.0"
|
||||||
|
adapter.endpoint = "/chat/completions"
|
||||||
|
|
||||||
|
-- opencode.ai zen 网关按 User-Agent 指纹识别官方客户端并把请求分到免费额度池;
|
||||||
|
-- 非官方 UA(curl/Go 默认等)会被分到匿名池并触发 FreeUsageLimitError。
|
||||||
|
-- 因此固定发送 opencode 客户端的 UA;配合源配置 api_key: "public"(官方无 key
|
||||||
|
-- 客户端实际发送 Bearer public)即可走免费池。
|
||||||
|
adapter.headers = {
|
||||||
|
["User-Agent"] = "opencode/0.1.0",
|
||||||
|
}
|
||||||
|
|
||||||
|
-- OpenAI /chat/completions format (pass-through, strip provider-specific fields)
|
||||||
|
function adapter.transform_request(raw_body)
|
||||||
|
local ok, req = pcall(json.decode, raw_body)
|
||||||
|
if not ok then return raw_body end
|
||||||
|
req.disable_thinking = nil
|
||||||
|
req.extra_body = nil
|
||||||
|
if req.messages then
|
||||||
|
for _, msg in ipairs(req.messages) do
|
||||||
|
msg.reasoning_content = nil
|
||||||
|
end
|
||||||
|
end
|
||||||
|
return json.encode(req)
|
||||||
|
end
|
||||||
|
|
||||||
|
function adapter.transform_response(raw_body)
|
||||||
|
local ok, resp = pcall(json.decode, raw_body)
|
||||||
|
if not ok or resp == nil then return raw_body end
|
||||||
|
|
||||||
|
local unified = {
|
||||||
|
content = "",
|
||||||
|
finish_reason = "",
|
||||||
|
token_usage = { prompt = 0, completion = 0, total = 0 }
|
||||||
|
}
|
||||||
|
|
||||||
|
if type(resp.usage) == "table" then
|
||||||
|
unified.token_usage.prompt = resp.usage.prompt_tokens or 0
|
||||||
|
unified.token_usage.completion = resp.usage.completion_tokens or 0
|
||||||
|
unified.token_usage.total = resp.usage.total_tokens or 0
|
||||||
|
end
|
||||||
|
|
||||||
|
if type(resp.choices) == "table" and #resp.choices > 0 then
|
||||||
|
local ch = resp.choices[1]
|
||||||
|
if type(ch.message) == "table" then
|
||||||
|
unified.content = ch.message.content or ""
|
||||||
|
if ch.message.reasoning_content then
|
||||||
|
unified.reasoning_content = ch.message.reasoning_content
|
||||||
|
end
|
||||||
|
if type(ch.message.tool_calls) == "table" then
|
||||||
|
local tcs = {}
|
||||||
|
for _, tc in ipairs(ch.message.tool_calls) do
|
||||||
|
local args_ok, args = pcall(json.decode, tc["function"].arguments)
|
||||||
|
if not args_ok then args = {} end
|
||||||
|
table.insert(tcs, {
|
||||||
|
id = tc.id,
|
||||||
|
type = tc.type or "function",
|
||||||
|
name = tc["function"].name,
|
||||||
|
arguments = args
|
||||||
|
})
|
||||||
|
end
|
||||||
|
unified.tool_calls = tcs
|
||||||
|
end
|
||||||
|
end
|
||||||
|
unified.finish_reason = ch.finish_reason or ""
|
||||||
|
end
|
||||||
|
|
||||||
|
return json.encode(unified)
|
||||||
|
end
|
||||||
|
|
||||||
|
function adapter.transform_stream_chunk(raw_chunk)
|
||||||
|
local ok, chunk = pcall(json.decode, raw_chunk)
|
||||||
|
if not ok then return "" end
|
||||||
|
|
||||||
|
if not chunk.choices or #chunk.choices == 0 then return "" end
|
||||||
|
local delta = chunk.choices[1].delta or {}
|
||||||
|
local fr = chunk.choices[1].finish_reason
|
||||||
|
|
||||||
|
local unified = {
|
||||||
|
content = delta.content or "",
|
||||||
|
done = (fr ~= nil)
|
||||||
|
}
|
||||||
|
if delta.reasoning_content then
|
||||||
|
unified.reasoning_content = delta.reasoning_content
|
||||||
|
end
|
||||||
|
if delta.tool_calls then
|
||||||
|
-- pass raw streaming fragments through; OpenAI clients accumulate index+id+name+arguments
|
||||||
|
unified.tool_calls = delta.tool_calls
|
||||||
|
end
|
||||||
|
return json.encode(unified)
|
||||||
|
end
|
||||||
|
|
||||||
|
return adapter
|
||||||
@ -302,13 +302,21 @@ func (v *VM) pool(name string) *adapterPool {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (v *VM) writeBundledAdapters() error {
|
func (v *VM) writeBundledAdapters() error {
|
||||||
known := []string{"openai", "anthropic", "deepseek", "gemini", "github", "groq", "mistral", "ollama", "kimicode"}
|
// Derive the set from the embedded FS instead of a hardcoded name list so
|
||||||
for _, name := range known {
|
// adding an adapter never requires maintaining a second list.
|
||||||
dst := filepath.Join(v.dir, name+".lua")
|
entries, err := bundledAdapters.ReadDir("adapters")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for _, e := range entries {
|
||||||
|
if e.IsDir() || !strings.HasSuffix(e.Name(), ".lua") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
dst := filepath.Join(v.dir, e.Name())
|
||||||
if _, err := os.Stat(dst); err == nil {
|
if _, err := os.Stat(dst); err == nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
data, err := bundledAdapters.ReadFile("adapters/" + name + ".lua")
|
data, err := bundledAdapters.ReadFile("adapters/" + e.Name())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|||||||
@ -2,6 +2,7 @@ package lua
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
@ -28,13 +29,56 @@ func TestLoadBundledAdapters(t *testing.T) {
|
|||||||
for _, a := range adapters {
|
for _, a := range adapters {
|
||||||
names[a.Name] = true
|
names[a.Name] = true
|
||||||
}
|
}
|
||||||
for _, want := range []string{"openai", "deepseek", "anthropic", "gemini", "ollama", "kimicode"} {
|
for _, want := range []string{"openai", "deepseek", "anthropic", "gemini", "ollama", "kimicode", "opencode"} {
|
||||||
if !names[want] {
|
if !names[want] {
|
||||||
t.Errorf("missing adapter %s (got %v)", want, names)
|
t.Errorf("missing adapter %s (got %v)", want, names)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestFirstRunSeedsEveryBundledAdapter(t *testing.T) {
|
||||||
|
dir := filepath.Join(t.TempDir(), "adapters")
|
||||||
|
vm := NewVM(dir)
|
||||||
|
if err := vm.Start(); err != nil {
|
||||||
|
t.Fatalf("start: %v", err)
|
||||||
|
}
|
||||||
|
defer vm.Stop()
|
||||||
|
// every embedded .lua must have been copied out (and no extra, non-.lua files)
|
||||||
|
embedded, err := bundledAdapters.ReadDir("adapters")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
seeded := 0
|
||||||
|
for _, e := range embedded {
|
||||||
|
if e.IsDir() || !strings.HasSuffix(e.Name(), ".lua") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
seeded++
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, e.Name())); err != nil {
|
||||||
|
t.Errorf("embedded adapter %s not seeded", e.Name())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if seeded == 0 {
|
||||||
|
t.Fatal("no embedded adapters found")
|
||||||
|
}
|
||||||
|
written := 0
|
||||||
|
entries, _ := os.ReadDir(dir)
|
||||||
|
for _, e := range entries {
|
||||||
|
if !e.IsDir() && strings.HasSuffix(e.Name(), ".lua") {
|
||||||
|
written++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if written != seeded {
|
||||||
|
t.Fatalf("seeded %d files but %d embedded adapters exist", written, seeded)
|
||||||
|
}
|
||||||
|
// re-start with the existing dir: authoritative, never rewritten
|
||||||
|
vm2 := NewVM(dir)
|
||||||
|
if err := vm2.Start(); err != nil {
|
||||||
|
t.Fatalf("second start: %v", err)
|
||||||
|
}
|
||||||
|
defer vm2.Stop()
|
||||||
|
}
|
||||||
|
|
||||||
func TestTransformRequest(t *testing.T) {
|
func TestTransformRequest(t *testing.T) {
|
||||||
vm := NewVM(freshAdapterDir(t))
|
vm := NewVM(freshAdapterDir(t))
|
||||||
if err := vm.Start(); err != nil {
|
if err := vm.Start(); err != nil {
|
||||||
@ -65,6 +109,32 @@ func TestBuildHeadersFallbackStatic(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestOpenCodeAdapterFingerprint(t *testing.T) {
|
||||||
|
vm := NewVM(freshAdapterDir(t))
|
||||||
|
if err := vm.Start(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer vm.Stop()
|
||||||
|
hdrs, err := vm.BuildHeaders("opencode", map[string]interface{}{"api_key": "public"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("build headers: %v", err)
|
||||||
|
}
|
||||||
|
if ua := hdrs["User-Agent"]; ua != "opencode/0.1.0" {
|
||||||
|
t.Fatalf("opencode adapter must send the opencode client UA, got %q", ua)
|
||||||
|
}
|
||||||
|
if hdrs["Authorization"] != "" {
|
||||||
|
t.Fatalf("opencode adapter must not hardcode Authorization (config api_key supplies it), got %q", hdrs["Authorization"])
|
||||||
|
}
|
||||||
|
// passthrough behaves like the openai adapter
|
||||||
|
out, err := vm.Transform("opencode", "transform_request", `{"model":"x","disable_thinking":true,"extra_body":{},"messages":[{"role":"user","content":"hi"}]}`)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if strings.Contains(out, "disable_thinking") || strings.Contains(out, "extra_body") {
|
||||||
|
t.Fatalf("opencode transform_request must strip provider fields: %s", out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestBuildHeadersCustomHook(t *testing.T) {
|
func TestBuildHeadersCustomHook(t *testing.T) {
|
||||||
vm := NewVM(freshAdapterDir(t))
|
vm := NewVM(freshAdapterDir(t))
|
||||||
if err := vm.Start(); err != nil {
|
if err := vm.Start(); err != nil {
|
||||||
|
|||||||
@ -208,6 +208,19 @@ func (p *Provider) ModelByID(id string) *config.Model {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ModelIDFold returns the configured model id matching id case-insensitively
|
||||||
|
// ("" when no model matches). AUTO-chain slot models are normalized through
|
||||||
|
// this so cooldown state, quota windows and the upstream model id all refer
|
||||||
|
// to the exact configured spelling.
|
||||||
|
func (p *Provider) ModelIDFold(id string) string {
|
||||||
|
for _, m := range p.cfg.Models {
|
||||||
|
if strings.EqualFold(m.ID, id) {
|
||||||
|
return m.ID
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
// ModelFor resolves the model name this provider should send upstream.
|
// ModelFor resolves the model name this provider should send upstream.
|
||||||
// If the requested model is not owned by this provider (e.g. an AUTO chain
|
// If the requested model is not owned by this provider (e.g. an AUTO chain
|
||||||
// fallback), it returns this provider's highest-priority chat model instead.
|
// fallback), it returns this provider's highest-priority chat model instead.
|
||||||
@ -239,6 +252,26 @@ func (p *Provider) bestChatModel() string {
|
|||||||
return bestID
|
return bestID
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// bestImageModel returns the highest-priority image-kind model of this source
|
||||||
|
// (fallback: first configured model). Used for AUTO image generation so a
|
||||||
|
// mixed source never sends a chat model to /v1/images/generations.
|
||||||
|
func (p *Provider) bestImageModel() string {
|
||||||
|
bestID, bestPrio := "", -1
|
||||||
|
for _, m := range p.cfg.Models {
|
||||||
|
if m.Kind != "" && m.Kind != "image" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if m.Priority > bestPrio {
|
||||||
|
bestPrio = m.Priority
|
||||||
|
bestID = m.ID
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if bestID == "" && len(p.cfg.Models) > 0 {
|
||||||
|
bestID = p.cfg.Models[0].ID
|
||||||
|
}
|
||||||
|
return bestID
|
||||||
|
}
|
||||||
|
|
||||||
// IsAutoID reports whether s is an AUTO routing placeholder.
|
// IsAutoID reports whether s is an AUTO routing placeholder.
|
||||||
func isAutoID(s string) bool {
|
func isAutoID(s string) bool {
|
||||||
s = strings.TrimSpace(s)
|
s = strings.TrimSpace(s)
|
||||||
@ -609,7 +642,11 @@ func (p *Provider) Chat(ctx context.Context, req *types.ChatRequest) (*types.Uni
|
|||||||
}
|
}
|
||||||
raw, status, err := p.do(ctx, p.URL(), body, hdrs)
|
raw, status, err := p.do(ctx, p.URL(), body, hdrs)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
p.RecordFailure(model, 0)
|
// a client disconnect or cancelled context is neither a success nor
|
||||||
|
// a failure for scheduling purposes — only upstream errors count
|
||||||
|
if ctx.Err() == nil {
|
||||||
|
p.RecordFailure(model, 0)
|
||||||
|
}
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if status != 200 {
|
if status != 200 {
|
||||||
@ -618,10 +655,19 @@ func (p *Provider) Chat(ctx context.Context, req *types.ChatRequest) (*types.Uni
|
|||||||
}
|
}
|
||||||
unified, err := p.vm.Transform(p.adapter, "transform_response", raw)
|
unified, err := p.vm.Transform(p.adapter, "transform_response", raw)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
// adapter produced unusable output: a real failure the slot must
|
||||||
|
// back off from, otherwise a broken adapter source is retried at
|
||||||
|
// full latency forever
|
||||||
|
if ctx.Err() == nil {
|
||||||
|
p.RecordFailure(model, 0)
|
||||||
|
}
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
var out types.UnifiedResponse
|
var out types.UnifiedResponse
|
||||||
if err := json.Unmarshal([]byte(unified), &out); err != nil {
|
if err := json.Unmarshal([]byte(unified), &out); err != nil {
|
||||||
|
if ctx.Err() == nil {
|
||||||
|
p.RecordFailure(model, 0)
|
||||||
|
}
|
||||||
return nil, fmt.Errorf("unmarshal unified response: %w (body: %s)", err, unified)
|
return nil, fmt.Errorf("unmarshal unified response: %w (body: %s)", err, unified)
|
||||||
}
|
}
|
||||||
p.RecordSuccess(model)
|
p.RecordSuccess(model)
|
||||||
@ -664,7 +710,11 @@ func (p *Provider) ChatStream(ctx context.Context, req *types.ChatRequest) (<-ch
|
|||||||
ch := make(chan types.UnifiedChunk, 64)
|
ch := make(chan types.UnifiedChunk, 64)
|
||||||
sel := <-rc
|
sel := <-rc
|
||||||
if sel.err != nil {
|
if sel.err != nil {
|
||||||
p.RecordFailure(model, 0)
|
// client disconnect/cancel before the first byte: not a scheduling
|
||||||
|
// failure (a healthy source must not be cooled by client cancellations)
|
||||||
|
if ctx.Err() == nil {
|
||||||
|
p.RecordFailure(model, 0)
|
||||||
|
}
|
||||||
p.Release()
|
p.Release()
|
||||||
return nil, sel.err
|
return nil, sel.err
|
||||||
}
|
}
|
||||||
@ -681,6 +731,8 @@ func (p *Provider) ChatStream(ctx context.Context, req *types.ChatRequest) (<-ch
|
|||||||
defer sel.resp.Body.Close()
|
defer sel.resp.Body.Close()
|
||||||
scanner := bufio.NewScanner(sel.resp.Body)
|
scanner := bufio.NewScanner(sel.resp.Body)
|
||||||
scanner.Buffer(make([]byte, 0, 64*1024), 1024*1024)
|
scanner.Buffer(make([]byte, 0, 64*1024), 1024*1024)
|
||||||
|
var chunks int
|
||||||
|
var doneSeen bool
|
||||||
for scanner.Scan() {
|
for scanner.Scan() {
|
||||||
line := strings.TrimSpace(scanner.Text())
|
line := strings.TrimSpace(scanner.Text())
|
||||||
if line == "" || !strings.HasPrefix(line, "data:") {
|
if line == "" || !strings.HasPrefix(line, "data:") {
|
||||||
@ -691,6 +743,7 @@ func (p *Provider) ChatStream(ctx context.Context, req *types.ChatRequest) (<-ch
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if data == "[DONE]" {
|
if data == "[DONE]" {
|
||||||
|
doneSeen = true
|
||||||
select {
|
select {
|
||||||
case ch <- types.UnifiedChunk{Done: true}:
|
case ch <- types.UnifiedChunk{Done: true}:
|
||||||
case <-ctx.Done():
|
case <-ctx.Done():
|
||||||
@ -711,6 +764,7 @@ func (p *Provider) ChatStream(ctx context.Context, req *types.ChatRequest) (<-ch
|
|||||||
if err := json.Unmarshal([]byte(unified), &ck); err != nil {
|
if err := json.Unmarshal([]byte(unified), &ck); err != nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
chunks++
|
||||||
select {
|
select {
|
||||||
case ch <- ck:
|
case ch <- ck:
|
||||||
case <-ctx.Done():
|
case <-ctx.Done():
|
||||||
@ -720,9 +774,15 @@ func (p *Provider) ChatStream(ctx context.Context, req *types.ChatRequest) (<-ch
|
|||||||
// The stream ended cleanly ([DONE] seen or EOF without an upstream
|
// The stream ended cleanly ([DONE] seen or EOF without an upstream
|
||||||
// read error): record success so a previously cooled model can be
|
// read error): record success so a previously cooled model can be
|
||||||
// retried. A client disconnect or mid-stream read error is neither
|
// retried. A client disconnect or mid-stream read error is neither
|
||||||
// success nor failure for scheduling purposes.
|
// success nor failure for scheduling purposes. A 200 that produced
|
||||||
|
// zero chunks and no [DONE] is an empty stream, i.e. a failure
|
||||||
|
// before the first chunk — record it so the slot can fall back.
|
||||||
if ctx.Err() == nil && scanner.Err() == nil {
|
if ctx.Err() == nil && scanner.Err() == nil {
|
||||||
p.RecordSuccess(model)
|
if doneSeen || chunks > 0 {
|
||||||
|
p.RecordSuccess(model)
|
||||||
|
} else {
|
||||||
|
p.RecordFailure(model, 0)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}()
|
}()
|
||||||
return ch, nil
|
return ch, nil
|
||||||
@ -735,7 +795,15 @@ func (p *Provider) Image(ctx context.Context, req *types.ImageGenRequest) (*type
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
defer p.Release()
|
defer p.Release()
|
||||||
model := p.ModelFor(req.Model)
|
// AUTO/unknown model: resolve to this source's image model, never to the
|
||||||
|
// best chat model (a mixed source would otherwise send a chat id to
|
||||||
|
// /v1/images/generations). An explicitly pinned model is honored as-is.
|
||||||
|
if isAutoID(req.Model) || p.ModelByID(req.Model) == nil {
|
||||||
|
r := *req
|
||||||
|
r.Model = p.bestImageModel()
|
||||||
|
req = &r
|
||||||
|
}
|
||||||
|
model := req.Model
|
||||||
|
|
||||||
b, _ := json.Marshal(req)
|
b, _ := json.Marshal(req)
|
||||||
transformed, err := p.vm.Transform(p.adapter+"_image", "transform_request", string(b))
|
transformed, err := p.vm.Transform(p.adapter+"_image", "transform_request", string(b))
|
||||||
@ -749,7 +817,10 @@ func (p *Provider) Image(ctx context.Context, req *types.ImageGenRequest) (*type
|
|||||||
}
|
}
|
||||||
raw, status, err := p.do(ctx, p.ImageURL(), transformed, hdrs)
|
raw, status, err := p.do(ctx, p.ImageURL(), transformed, hdrs)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
p.RecordFailure(model, 0)
|
// client disconnect/cancel: not a scheduling failure
|
||||||
|
if ctx.Err() == nil {
|
||||||
|
p.RecordFailure(model, 0)
|
||||||
|
}
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if status != 200 {
|
if status != 200 {
|
||||||
@ -767,6 +838,9 @@ func (p *Provider) Image(ctx context.Context, req *types.ImageGenRequest) (*type
|
|||||||
}
|
}
|
||||||
var img types.ImageGenResponse
|
var img types.ImageGenResponse
|
||||||
if err := json.Unmarshal([]byte(raw), &img); err != nil {
|
if err := json.Unmarshal([]byte(raw), &img); err != nil {
|
||||||
|
if ctx.Err() == nil {
|
||||||
|
p.RecordFailure(model, 0)
|
||||||
|
}
|
||||||
return nil, fmt.Errorf("unmarshal image response: %w", err)
|
return nil, fmt.Errorf("unmarshal image response: %w", err)
|
||||||
}
|
}
|
||||||
out.ImageData = img.Data
|
out.ImageData = img.Data
|
||||||
|
|||||||
@ -298,3 +298,106 @@ func TestChatBusyFailsFast(t *testing.T) {
|
|||||||
t.Fatalf("first chat: %v", err)
|
t.Fatalf("first chat: %v", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func eventually(t *testing.T, timeout time.Duration, cond func() bool, msg string) {
|
||||||
|
t.Helper()
|
||||||
|
deadline := time.Now().Add(timeout)
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
|
if cond() {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
time.Sleep(10 * time.Millisecond)
|
||||||
|
}
|
||||||
|
t.Fatalf("timed out: %s", msg)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestChatClientCancelNotRecorded: a client disconnect before the response is
|
||||||
|
// neither success nor failure — the (source, model) state must stay clean.
|
||||||
|
func TestChatClientCancelNotRecorded(t *testing.T) {
|
||||||
|
started := make(chan struct{})
|
||||||
|
release := make(chan struct{})
|
||||||
|
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
close(started)
|
||||||
|
<-release // hold the upstream open; release at cleanup
|
||||||
|
fmt.Fprint(w, `{"choices":[{"message":{"content":"late"}}]}`)
|
||||||
|
}))
|
||||||
|
defer up.Close()
|
||||||
|
p := newTestProvider(t, src("mock", up.URL, "openai", "m"))
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
errCh := make(chan error, 1)
|
||||||
|
go func() {
|
||||||
|
_, err := p.Chat(ctx, &types.ChatRequest{
|
||||||
|
Model: "m",
|
||||||
|
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("x")}},
|
||||||
|
})
|
||||||
|
errCh <- err
|
||||||
|
}()
|
||||||
|
<-started
|
||||||
|
cancel() // client disconnects mid-request
|
||||||
|
if err := <-errCh; err == nil {
|
||||||
|
t.Fatal("cancelled request must return an error")
|
||||||
|
}
|
||||||
|
close(release)
|
||||||
|
eventually(t, 2*time.Second, func() bool {
|
||||||
|
_, fc, _ := p.ModelHealthInfo("m")
|
||||||
|
return fc == 0
|
||||||
|
}, "client cancel must not record a scheduling failure")
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestChatStreamEmptyBodyNotSuccess: a 200 that yields zero chunks and no
|
||||||
|
// [DONE] is a failure before the first chunk — the slot must back off.
|
||||||
|
func TestChatStreamEmptyBodyNotSuccess(t *testing.T) {
|
||||||
|
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.Header().Set("Content-Type", "text/event-stream")
|
||||||
|
// 200 with an empty body: no chunks, no [DONE]
|
||||||
|
}))
|
||||||
|
defer up.Close()
|
||||||
|
p := newTestProvider(t, src("mock", up.URL, "openai", "m"))
|
||||||
|
ch, err := p.ChatStream(context.Background(), &types.ChatRequest{
|
||||||
|
Model: "m",
|
||||||
|
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("x")}},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("stream: %v", err)
|
||||||
|
}
|
||||||
|
got := 0
|
||||||
|
for range ch {
|
||||||
|
got++
|
||||||
|
}
|
||||||
|
if got != 0 {
|
||||||
|
t.Fatalf("want empty stream, got %d chunks", got)
|
||||||
|
}
|
||||||
|
eventually(t, 2*time.Second, func() bool {
|
||||||
|
_, fc, _ := p.ModelHealthInfo("m")
|
||||||
|
return fc >= 1
|
||||||
|
}, "empty stream must record a failure")
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestImageAutoUsesImageModel: AUTO image generation on a mixed source must
|
||||||
|
// send the image-kind model id, never the best chat model.
|
||||||
|
func TestImageAutoUsesImageModel(t *testing.T) {
|
||||||
|
var gotModel string
|
||||||
|
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
var body map[string]interface{}
|
||||||
|
_ = json.NewDecoder(r.Body).Decode(&body)
|
||||||
|
gotModel, _ = body["model"].(string)
|
||||||
|
fmt.Fprint(w, `{"created":1,"data":[{"url":"http://x/1.png"}]}`)
|
||||||
|
}))
|
||||||
|
defer up.Close()
|
||||||
|
s := config.Source{Name: "mix", BaseURL: up.URL, Adapter: "openai", MaxConcurrent: 4}
|
||||||
|
s.Models = []config.Model{
|
||||||
|
{ID: "chat-m", Kind: "chat", Priority: 100},
|
||||||
|
{ID: "img-m", Kind: "image", Priority: 50},
|
||||||
|
}
|
||||||
|
p := newTestProvider(t, s)
|
||||||
|
resp, err := p.Image(context.Background(), &types.ImageGenRequest{Prompt: "cat", Model: "AUTO"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("image: %v", err)
|
||||||
|
}
|
||||||
|
if len(resp.ImageData) == 0 {
|
||||||
|
t.Fatal("no image data returned")
|
||||||
|
}
|
||||||
|
if gotModel != "img-m" {
|
||||||
|
t.Fatalf("AUTO image must use the image-kind model, got %q", gotModel)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@ -266,11 +266,13 @@ func (s *Scheduler) chainDrive(ctx context.Context, chain *Chain, req *types.Cha
|
|||||||
}
|
}
|
||||||
// every candidate was busy or cooling: bounded poll before downgrading
|
// every candidate was busy or cooling: bounded poll before downgrading
|
||||||
deadline := time.Now().Add(busyWait)
|
deadline := time.Now().Add(busyWait)
|
||||||
|
timer := time.NewTimer(busyPoll)
|
||||||
|
defer timer.Stop()
|
||||||
for {
|
for {
|
||||||
select {
|
select {
|
||||||
case <-ctx.Done():
|
case <-ctx.Done():
|
||||||
return nil, nil, "", "", ctx.Err()
|
return nil, nil, "", "", ctx.Err()
|
||||||
case <-time.After(busyPoll):
|
case <-timer.C:
|
||||||
}
|
}
|
||||||
done := time.Now().After(deadline)
|
done := time.Now().After(deadline)
|
||||||
if done {
|
if done {
|
||||||
@ -299,6 +301,7 @@ func (s *Scheduler) chainDrive(ctx context.Context, chain *Chain, req *types.Cha
|
|||||||
ce.Tiers = append(ce.Tiers, res.hard...)
|
ce.Tiers = append(ce.Tiers, res.hard...)
|
||||||
break // hard failure while waiting: stop waiting, fall through
|
break // hard failure while waiting: stop waiting, fall through
|
||||||
}
|
}
|
||||||
|
timer.Reset(busyPoll)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if len(ce.Tiers) == 0 && len(ce.Skipped) == 0 {
|
if len(ce.Tiers) == 0 && len(ce.Skipped) == 0 {
|
||||||
@ -331,6 +334,10 @@ func (s *Scheduler) ChainChatStream(ctx context.Context, chain *Chain, req *type
|
|||||||
// fallback switches the model id per provider instead of reusing the first
|
// fallback switches the model id per provider instead of reusing the first
|
||||||
// candidate's model name. On success it returns the response together with
|
// candidate's model name. On success it returns the response together with
|
||||||
// the name of the provider and the exact model id that served the request.
|
// the name of the provider and the exact model id that served the request.
|
||||||
|
// A candidate whose (source, model) pair is cooling down is skipped like a
|
||||||
|
// busy one — direct paths share the "cooldown is the only hard skip"
|
||||||
|
// semantics of the AUTO chain (plan 2.3/2.5); otherwise persistent direct
|
||||||
|
// traffic would keep renewing a capped auth cooldown forever.
|
||||||
func (s *Scheduler) Chat(ctx context.Context, cands []Provider, req *types.ChatRequest) (*types.UnifiedResponse, string, string, error) {
|
func (s *Scheduler) Chat(ctx context.Context, cands []Provider, req *types.ChatRequest) (*types.UnifiedResponse, string, string, error) {
|
||||||
attempts := s.MaxRetries + 1
|
attempts := s.MaxRetries + 1
|
||||||
var lastErr error
|
var lastErr error
|
||||||
@ -338,6 +345,10 @@ func (s *Scheduler) Chat(ctx context.Context, cands []Provider, req *types.ChatR
|
|||||||
p := cands[i]
|
p := cands[i]
|
||||||
r := *req
|
r := *req
|
||||||
r.Model = p.ModelFor(req.Model)
|
r.Model = p.ModelFor(req.Model)
|
||||||
|
if !p.ModelAvailable(r.Model) {
|
||||||
|
lastErr = fmt.Errorf("provider %s: model %q cooling down", p.Name(), r.Model)
|
||||||
|
continue
|
||||||
|
}
|
||||||
resp, err := p.Chat(ctx, &r)
|
resp, err := p.Chat(ctx, &r)
|
||||||
if ctx.Err() != nil {
|
if ctx.Err() != nil {
|
||||||
return nil, "", "", ctx.Err()
|
return nil, "", "", ctx.Err()
|
||||||
@ -366,6 +377,10 @@ func (s *Scheduler) ChatStream(ctx context.Context, cands []Provider, req *types
|
|||||||
p := cands[i]
|
p := cands[i]
|
||||||
r := *req
|
r := *req
|
||||||
r.Model = p.ModelFor(req.Model)
|
r.Model = p.ModelFor(req.Model)
|
||||||
|
if !p.ModelAvailable(r.Model) {
|
||||||
|
lastErr = fmt.Errorf("provider %s: model %q cooling down", p.Name(), r.Model)
|
||||||
|
continue
|
||||||
|
}
|
||||||
resp, err := p.ChatStream(ctx, &r)
|
resp, err := p.ChatStream(ctx, &r)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
return resp, p.Name(), r.Model, nil
|
return resp, p.Name(), r.Model, nil
|
||||||
@ -385,6 +400,10 @@ func (s *Scheduler) Image(ctx context.Context, cands []Provider, req *types.Imag
|
|||||||
var lastErr error
|
var lastErr error
|
||||||
for i := 0; i < attempts && i < len(cands); i++ {
|
for i := 0; i < attempts && i < len(cands); i++ {
|
||||||
p := cands[i]
|
p := cands[i]
|
||||||
|
if !p.ModelAvailable(p.ModelFor(req.Model)) {
|
||||||
|
lastErr = fmt.Errorf("provider %s: model %q cooling down", p.Name(), p.ModelFor(req.Model))
|
||||||
|
continue
|
||||||
|
}
|
||||||
resp, err := p.Image(ctx, req)
|
resp, err := p.Image(ctx, req)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
return resp, p.Name(), nil
|
return resp, p.Name(), nil
|
||||||
|
|||||||
@ -67,6 +67,40 @@ func (f *fakeProvider) Image(ctx context.Context, req *types.ImageGenRequest) (*
|
|||||||
return nil, errors.New("no image")
|
return nil, errors.New("no image")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestDirectSkipsCooledCandidate: direct paths share the AUTO-chain rule that
|
||||||
|
// cooldown is the only hard skip — a cooling candidate must never be hit, and
|
||||||
|
// a fully cooling set must fail without touching upstream.
|
||||||
|
func TestDirectSkipsCooledCandidate(t *testing.T) {
|
||||||
|
hot := fakeProv("hot", "m")
|
||||||
|
cold := fakeProv("cold", "m")
|
||||||
|
cold.available.Store(false)
|
||||||
|
s := New(2)
|
||||||
|
req := &types.ChatRequest{
|
||||||
|
Model: "m",
|
||||||
|
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("x")}},
|
||||||
|
}
|
||||||
|
resp, src, _, err := s.Chat(context.Background(), []Provider{cold, hot}, req)
|
||||||
|
if err != nil || src != "hot" {
|
||||||
|
t.Fatalf("want hot to serve, got src=%q err=%v", src, err)
|
||||||
|
}
|
||||||
|
if resp == nil || resp.Content != "hot" {
|
||||||
|
t.Fatalf("bad response: %#v", resp)
|
||||||
|
}
|
||||||
|
if cold.chatHits.Load() != 0 {
|
||||||
|
t.Fatalf("cooled candidate must not be hit, got %d hits", cold.chatHits.Load())
|
||||||
|
}
|
||||||
|
// all candidates cooled: direct path reports it instead of hitting upstream
|
||||||
|
hot.available.Store(false)
|
||||||
|
hits := hot.chatHits.Load()
|
||||||
|
_, _, _, err = s.Chat(context.Background(), []Provider{cold, hot}, req)
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "cooling down") {
|
||||||
|
t.Fatalf("want cooling-down error, got %v", err)
|
||||||
|
}
|
||||||
|
if hot.chatHits.Load() != hits || cold.chatHits.Load() != 0 {
|
||||||
|
t.Fatalf("cooled candidates must not be hit, got hot=%d cold=%d", hot.chatHits.Load(), cold.chatHits.Load())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// lookup resolves (model, source) -> provider for chain builders in tests.
|
// lookup resolves (model, source) -> provider for chain builders in tests.
|
||||||
type lookup func(m, s string) Provider
|
type lookup func(m, s string) Provider
|
||||||
|
|
||||||
|
|||||||
34
plan.md
34
plan.md
@ -247,3 +247,37 @@ p==nil → 400/404;!TryAcquire → 429 busy(快速失败)
|
|||||||
- [x] **P4-8(2026-08-11)调度延滞排查结论**:homeagent AUTO 请求"看起来停在 opus"实为调度正常降级——完整 ChainErr 列出 tier1→2→3→4 全部尝试,审计只记 `ce.Tiers[0]`(首档)故 UI 显 opus;zen 源 `opencode.ai/zen/v1` 直连即 403 `[server_error] Upstream response was not valid JSON`(间歇)/ nemotron 404,确认为上游源自身问题,非网关透传或 homeagent 适配器。
|
- [x] **P4-8(2026-08-11)调度延滞排查结论**:homeagent AUTO 请求"看起来停在 opus"实为调度正常降级——完整 ChainErr 列出 tier1→2→3→4 全部尝试,审计只记 `ce.Tiers[0]`(首档)故 UI 显 opus;zen 源 `opencode.ai/zen/v1` 直连即 403 `[server_error] Upstream response was not valid JSON`(间歇)/ nemotron 404,确认为上游源自身问题,非网关透传或 homeagent 适配器。
|
||||||
- [x] **P4-9(2026-08-11)WebUI 统计可视化**(多轮迭代定稿):① 模型用量%改为占总请求的比例(`r.reqs/total`,替代原"相对最大模型"的误导性 100%/97%);② 顶部五宫格统一为同构卡片——**标题 + 大数值 + 副标题 + 86px 图表区**:活跃请求=折线 sparkline(实时)、总请求=按天柱状图、tokens=各模型占比堆叠条(大数值示总用量)、平均延迟=折线 sparkline(实时)、状态码=按码值堆叠条 + chip 副标题;全部 canvas 2D 手绘(无第三方库),去掉灰色空壳占位与独立全宽状态卡/冗余副标题。
|
- [x] **P4-9(2026-08-11)WebUI 统计可视化**(多轮迭代定稿):① 模型用量%改为占总请求的比例(`r.reqs/total`,替代原"相对最大模型"的误导性 100%/97%);② 顶部五宫格统一为同构卡片——**标题 + 大数值 + 副标题 + 86px 图表区**:活跃请求=折线 sparkline(实时)、总请求=按天柱状图、tokens=各模型占比堆叠条(大数值示总用量)、平均延迟=折线 sparkline(实时)、状态码=按码值堆叠条 + chip 副标题;全部 canvas 2D 手绘(无第三方库),去掉灰色空壳占位与独立全宽状态卡/冗余副标题。
|
||||||
- [x] **P4-10(2026-08-11)claude "Third-party apps now draw from your extra usage" 400 结论**:该报错为 **Anthropic 官方配额提示**(账号第三方 app 用量额度耗尽,提示到 Anthropic usage 设置充值),**非流量特征被识破**;伪装流量/换 adaptive 适配器解决不了(请求已被 Anthropic 接受并按账号用量计费)。且 qijiar+openai 适配器下 claude 模型本就能 200 成功(审计 16 次,含 claude-opus-4-8/claude-sonnet-5),AUTO 链/直连的 claude 失败多为上游 `model_not_found`(无 distributor 渠道)、`Concurrency limit exceeded`、`extra usage` 配额——均为上游账号/渠道级问题,非网关。若确需走 Anthropic 原生 `/v1/messages`,可为其源切换内置 `anthropic` 适配器(仅改变量 wire 格式,不影响配额)。
|
- [x] **P4-10(2026-08-11)claude "Third-party apps now draw from your extra usage" 400 结论**:该报错为 **Anthropic 官方配额提示**(账号第三方 app 用量额度耗尽,提示到 Anthropic usage 设置充值),**非流量特征被识破**;伪装流量/换 adaptive 适配器解决不了(请求已被 Anthropic 接受并按账号用量计费)。且 qijiar+openai 适配器下 claude 模型本就能 200 成功(审计 16 次,含 claude-opus-4-8/claude-sonnet-5),AUTO 链/直连的 claude 失败多为上游 `model_not_found`(无 distributor 渠道)、`Concurrency limit exceeded`、`extra usage` 配额——均为上游账号/渠道级问题,非网关。若确需走 Anthropic 原生 `/v1/messages`,可为其源切换内置 `anthropic` 适配器(仅改变量 wire 格式,不影响配额)。
|
||||||
|
|
||||||
|
### Phase 5 — 2026-08-13 全量复查核实 & 修复(本轮会话)
|
||||||
|
|
||||||
|
> 三路并行审查(provider/registry/scheduler、gateway、config/store/core)+ 逐项对照源码核实,全部条目经本人亲自读取代码确认。
|
||||||
|
|
||||||
|
#### 5.1 已核实的未修正 bug(均属实)
|
||||||
|
|
||||||
|
| 编号 | 级别 | 问题 | 证据 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| H1 | 高 | **stats 窗口单位错乱**:`Req.Time` 为毫秒(chat.go:572 `UnixMilli`),`stats.go:220` `h := r.Time/hourSec`(hourSec=3600 秒)→ 桶宽实为 3.6s;`WindowTokens`(stats.go:352)`h*hourSec >= cut`(cut 为秒)毫秒恒 ≥ 秒 → **任何 sec>0 窗口返回全部记录**;剪枝 960 桶 ≈ 仅保留 57.6min(stats.go:227-233)。槽配额(chat.go:326)/key 配额(scopeTokens)/UI 周月曲线全部失真;LoadAudit 回放同路径;`stats_test.go:125` 断言 372=全计,固化错误行为 | 逐行确认 |
|
||||||
|
| H2 | 高 | **空 scope 语义分裂**:`cleanScopes`(core.go:276-288)返回非 nil 空切片 → CreateKey/UpdateKey 后 `allowedModels` 非 nil → 空 scope 无匹配 → **403 锁死**;config 序列化 `models,omitempty`(config.go:275)→ 重启后 nil → **全部放行**。另 `UpdateKey`(core.go:245)对未传 models 的 PUT **静默清空已有 scope** | 逐行确认 |
|
||||||
|
| H3 | 高 | **客户端取消/断连被记失败**:Chat(provider.go:611-613)/Stream 首块前(:666-669)/Image(:751-753)对 `err != nil` 无条件 `RecordFailure`,无 `ctx.Err()==nil` 守卫 → SDK 超时/关页把健康源反复误冷却、退避翻倍至 30min。流式中段(:724)已有守卫 → "断开≠成败"只实现一半 | 逐行确认 |
|
||||||
|
| H4 | 高 | **直连路径无视冷却(P3 残留)**:AUTO 走 `chainDrive` 查 `ModelAvailable`(scheduler.go:242),直连 Chat/ChatStream/Image(scheduler.go:334-398)不查 → 401 打满 30min 后直连照打,`RecordFailure(auth)`(provider.go:79-101)每次重写冷却截止 → **无限续期**,"无永久黑名单"被持续直连流量打破 | 逐行确认 |
|
||||||
|
| M1 | 中 | **AUTO 槽模型名大小写不一致**:`buildAutoChain` 保留规则原文(core.go:415,Image 过滤用精确匹配 :406;`ProviderForSlot` 用 EqualFold)→ `runTier:194` 原样下发 → `ModelFor` 精确匹配失败回退 `bestChatModel`(provider.go:214-222)**静默发错模型**;`ModelAvailable` 未知模型恒 true(:303-310)→ 冷却失效;配额键(chat.go:326 vs 实际记账模型)错位 → 配额永不生效 | 逐行确认 |
|
||||||
|
| M2 | 中 | **200-空流记为成功**:ChatStream 无 chunk 计数(provider.go:678-727),`:724` 对"0 chunk 干净 EOF"照记 `RecordSuccess`→ 空成功流不降级 | 逐行确认 |
|
||||||
|
| M3 | 中 | **AUTO 生图错模型(仅混合源触发)**:`Image`(provider.go:738)`ModelFor`→`bestChatModel`(跳过 image 模型)→ 混合源(chat+image)把 chat 模型发往 /images/generations;`effectiveImageModel`(chat.go:447-459)只用于配额/scope 检查不修正请求体 | 逐行确认 |
|
||||||
|
| M4 | 中 | **transform/unmarshal 失败不上账**:Chat(provider.go:619-626)/Image(:761-771)适配器产出坏数据直接 return → **适配器故障源永不退避**,每请求全额重打 | 逐行确认 |
|
||||||
|
| M5 | 中 | **Core cfg 并发写无锁(潜伏)**:`Core` 仅 `autoChain` 原子(core.go:24-31);`c.cfg.Keys/Sources/Auto` 裸写(CreateKey:228/UpdateKey:237-251/SaveAutoRules:295/AddSource:516-525)与请求路径 `FindKey:201-208`/`mergedSources` 并发 → race;core 包零单测,`go test -race` 全绿属"无证据"非"安全" | 确认+race 全绿 |
|
||||||
|
| L1 | 低 | busyWait 每轮 `time.After` 不 Stop(scheduler.go:273) | 确认 |
|
||||||
|
| L6 | 低 | pinned(`src:model`)tool-call 请求:`resolveCands` 工具分支(chat.go:95-107)用原始带前缀串 `ModelFor`(不解前缀)→ 回退 bestChatModel | 确认 |
|
||||||
|
|
||||||
|
#### 5.2 修复计划(按序推进,每项完成即勾选)
|
||||||
|
|
||||||
|
- [x] **P5-1(H1)stats 窗口单位修正**:`aggregateLocked`(stats.go:220)毫秒转秒后再分桶(`h := r.Time/1000/hourSec`);窗口语义(0=全量;>0 滑动秒窗,整桶计引入截断误差≤1h,注释说明);剪枝阈值 960 小时桶 ≈ 40 天保留(注释注明);重写 `TestLoadAuditFullReplay`(相对时间戳 + 全量/3h/24h=372、1h ∈ [2,372)、m2=10 边界断言,旧断言 372=全计恰固化错误行为);新增 `TestStatsByStatus` 不受影响。
|
||||||
|
- [x] **P5-2(H2)空 scope 归一化**:`cleanScopes` 空输入返回 nil(进程内=重启后语义一致);`UpdateKey` 仅当 `models != nil` 才覆盖 scope(未传保留、显式 `[]` 清空);单测 `TestCleanScopesNilForEmpty`/`TestCreateKeyWithoutScopeUnrestricted`/`TestUpdateKeyPreservesScopeWhenOmitted`。
|
||||||
|
- [x] **P5-3(H3)断开不记失败**:Chat/ChatStream 首块前/Image 三处 `err != nil` 分支改为 `err != nil && ctx.Err() == nil` 才 `RecordFailure`;新增 `TestChatClientCancelNotRecorded`(注意:测试 handler 不能依赖服务端 ctx 取消传播——HTTP/2 下客户端取消不会触发服务端 ctx.Done,用显式 release 通道)。
|
||||||
|
- [x] **P5-4(H4)直连冷却检查**:`Scheduler.Chat/ChatStream/Image` 调用前查 `ModelAvailable`,冷却中记 "cooling down" 错误顺延下一候选;全部冷却即失败返回——与 AUTO 一致"冷却=唯一硬跳过";auth 401 冷却不再被直连流量续期;新增 `TestDirectSkipsCooledCandidate`(冷却候选零命中 + 全冷报错)。已知取舍:AUTO 生图直连的可用性检查用 `ModelFor`(chat 模型)近似,精确 image 模型冷却检查待后续。
|
||||||
|
- [x] **P5-5(M1)槽模型名规范化**:`Provider.ModelIDFold`(大小写无关匹配)+ `buildAutoChain` 用其回填真实模型 ID 到 `r.Model`,image-kind 过滤也基于规范化后的精确 ID;冷却/配额/上游模型三者键一致;单测 `TestSaveAutoRulesNormalizesModelCase`(GPT-4O → gpt-4o)。
|
||||||
|
- [x] **P5-6(M2)空流不记成功**:ChatStream 统计有效 chunk 与 `[DONE]`;干净 EOF 且 0 chunk 且无 `[DONE]` → `RecordFailure`,有 `[DONE]` 或 ≥1 chunk → `RecordSuccess`;新增 `TestChatStreamEmptyBodyNotSuccess`。
|
||||||
|
- [x] **P5-7(M3)AUTO 生图选 image 模型**:`bestImageModel()`(Kind==image 最高 priority,兜底 Models[0]);`Image` 对 AUTO/未知模型**改写请求体**(克隆 req 回填 image 模型 ID,非仅记账);新增 `TestImageAutoUsesImageModel`(混合源断言上游收到 img 模型而非 chat 模型——初版只改记账导致 body 仍为 AUTO,测试当场抓出)。
|
||||||
|
- [x] **P5-8(M4)transform 失败记账**:Chat/Image 的 transform_response/unmarshal 失败分支补 `RecordFailure(model,0)`(组合 ctx.Err 守卫)。
|
||||||
|
- [x] **P5-9(M5)Core 加锁**:`Core.mu sync.Mutex` 包裹全部 cfg 读写公开方法(ListKeys/FindKey/CreateKey/UpdateKey/DeleteKey/AutoRules/SaveAutoRules/Sources/AddSource/RemoveSource/Reload),私有内部函数不持锁防重入;`Config()` 仅测试读 Path 无需锁;新增 `core_test.go`(含 `TestConcurrentKeyMutation` 16 并发增删查)——`-race` 全绿。
|
||||||
|
- [x] **P5-10(L1/L6)**:busyWait 改 `time.NewTimer`+`Reset`/defer Stop(修 20+ 计时器泄漏);`resolveCands` 工具分支改用剥前缀后的 `effective` 调 `ModelFor`(pinned tool-call 不再回退 bestChatModel)。
|
||||||
|
- [x] **P5-11**:`go build ./...`、`go vet ./...`、`go test ./...`(含 e2e)全绿;`go test -race`(core/scheduler/gateway)全绿。待推送 + 生产机 `-tags luajit` 回归部署(P3-7 流程)。
|
||||||
Reference in New Issue
Block a user