Compare commits

...

21 Commits

Author SHA1 Message Date
d5bbb0eca5 chore(gui): 版本号 3.0.0(与内核同版本)
改名为 breaking:go module 路径、二进制名、`/etc` 配置目录、`LLMSPROXY_*`
环境变量全部变更,已部署实例必须跑 `packaging/migrate-to-modelrouter.sh`。
因此内核与桌面端一起升到 3.0.0,而不是继续在 v2.0.x 上打 patch。

`Makefile` 的 VERSION 来自 `git describe`,无需另改。
2026-10-27 00:07:07 +08:00
9cb6254166 docs: 补入 2026-10-19 交接文档
记录当时的 Git 状态、渠道适配器进度、以及"provider 回归测试存在未完成编辑"
这一未决项。作为历史记录保留,不参与构建。
2026-10-27 00:06:41 +08:00
4514270e21 fix(opencodezen): UA 升到 1.19.4 并补回 x-opencode-session-id
- 上游按客户端指纹判定免费层资格:CLI 1.18.29(4 个 x-opencode-* 头)
  稳定 429;CLI 1.19.4(5 个头,含 session-id)200 且任务真实完成。
- 适配器的 UA 与头集合必须**同步**升级:只改 UA 声称的版本、不发对应头,
  指纹反而更不一致,实测重新落回 429。
- 头形状按官方 CLI 抓包校正;`x-opencode-session-id` 与 `x-opencode-session`
  同值,此前被误删。
- live 测试新增 `gatewayBinary()`:优先 `MODELROUTER_BIN`,其次
  `/usr/local/bin/modelrouter`,回退改名前的 `/usr/local/bin/llmsproxy`。

注:本提交是改名/插件整改之外的独立修复,先于 v3.0.0 收口。
2026-10-27 00:06:41 +08:00
ef95bc45d0 docs(provider): 中性化传输代理注释里的第三方客户端表述
`internal/provider/provider.go` 的两处注释用第三方客户端名作代理示例
(`clash on 127.0.0.1:7890`)。改成中性的「a local HTTP proxy on
127.0.0.1:7890」,语义不变:仍然是在说明 per-source `proxy_url` 的用途。

上一轮中性化只做了大小写敏感的 `Clash` 扫描,漏掉了这两处小写;
本次改用 `-i` 全仓复查,源码/文档/打包内已无该表述(`clashing`、
`diverge` 这类同形词不属此列)。
2026-10-27 00:00:19 +08:00
f7e926ccb7 docs: README 拆分为主题文档 + 中性化竞品表述
README 从 642 行降到 131 行,只保留简介、核心能力、最小快速开始、文档索引、
改名说明、目录结构与测试入口。其余内容迁到 docs/,中英文索引一一对应:

  docs/features.md       实现与完整特性清单(原 README 的"实现/特性")
  docs/configuration.md  配置文件、自动探测模型、多密钥与配额
  docs/routing.md        模型路由、AUTO 链、冷却与半冷却、源/适配器、源模板
  docs/performance.md    实测内存占用与调优、thinking 预算
  docs/webui.md          WebUI 使用
  docs/adapters.md       Lua 适配器协议与内置适配器清单
  docs/gui.md            桌面端 Headless/Desktop 两种产物与构建
  docs/development.md    目录职责、开发协作、测试
  docs/deployment.md     部署、发行包、目录权限、llmsproxy 迁移与验证清单

中性化:README / README_EN / GUI 注释里的 `Clash Verge` 类比全部改为
「桌面壳 / desktop shell」,仓库内(源码与文档)已无该表述。
2026-10-26 22:28:24 +08:00
688dce88d1 refactor(rename)!: llmsproxy → modelrouter(module / 二进制 / 路径 / env)
产品名早已是 ModelRouter,但仓库标识符仍沿用旧名。这次统一,读写两侧都做兼容,
已部署的机器不会因为顺序问题立刻失效。

- go module `llmsproxy` → `modelrouter`;全仓 import 与 `cmd/modelrouter` 同步。
- `Makefile BINARY=modelrouter`;发行包名、systemd unit、/usr/share 资源、
  `/usr/local/bin` 目标全部改为 modelrouter。
- 默认配置目录 `/etc/llmsproxy` → `/etc/modelrouter`。**读取侧回退**:插件与
  `storeDir` 仍会检查旧路径,所以"先换二进制、后改 unit"不会立即失效。
- 环境变量 `LLMSPROXY_*`(PLUGIN_DIR / XDISPLAY_BASE / ARCH / LIVE_ZEN /
  BUILD_DIR)改为 `MODELROUTER_*`;`LLMS_PROXY_MASTER_KEY` →
  `MODELROUTER_MASTER_KEY`。旧名在读取侧继续接受。
- 桌面端内嵌核心 `bin/llmsproxy` → `bin/modelrouter`,并保留旧文件名回退,
  避免升级到一半的 profile 被判成"core binary missing"。
- 新增 `packaging/migrate-to-modelrouter.sh`:**默认 dry-run**,复制而非移动、
  旧目录永不删除、支持 `--rollback`;随 release tar 分发并由 core-dist 校验。
- `deploy.sh` 增加改名保护:发现旧 `/etc/llmsproxy` 而新配置不存在时**中止**,
  否则会部署出一个"能启动但没有 key、没有源"的空实例。
- 临时目录前缀 `/tmp/llmsproxy-*` → `/tmp/modelrouter-*`;清理逻辑同时回收两代。

兼容别名(`__llmsproxy_*` 全局、`llmsproxy` SDK 别名、旧路径/旧 env)刻意保留,
它们是升级路径而非遗留;docs/plugins.md 已标注为废弃。

本提交未包含:`adapters/opencodezen.lua`、`adapter_opencodezen_test.go`、
`adapter_opencodezen_live_test.go`(OpenCode UA/免费池的既有改动,另批提交)。
2026-10-26 22:28:08 +08:00
de7553eaa2 feat(lua)!: 插件只通过 modelrouter SDK 与内核交互,且不再随内核安装
- 新增唯一公开接口表 `modelrouter`(sdk_version=1.0):http / secrets /
  kernel / adapters / plugin_dir。成员与内部桥是**同一对象**而非副本 ——
  复制会让凭据保险库分叉(插件写 A、内核读 B)。
- `llmsproxy` 与 `__llmsproxy_*` 降为同对象兼容别名,只为已装插件继续可用。
  仓库内 4 个插件(channels / billing / quota / usage)全部改用
  `modelrouter.*`:此前它们直接依赖内核私有全局,内核无法在不炸插件的前提下
  改实现(重命名、加守卫、换实现都算破坏性变更)。
- 插件不再由网关安装:`SeedBundled()` 变彻底 no-op,`core.New` 不再调用;
  `ReadBundledPlugin` 改为按源文件定位(脱离 cwd),只服务测试与开发工具。
- 桌面端删除 `BUILTIN_PLUGINS` 与 `syncBundledPluginsOnUpgrade`:升级不再
  删除、改名或覆盖用户插件文件。插件是运维资产,不是内核资源。
- `Builtin` 不再靠"源码与仓库示例逐字节相同"推断,插件一律视为外部文件。
- 判据:SDK 存在/版本/形状、别名与内部桥同一张表、适配器 VM 拿不到 SDK、
  经 SDK 真实写读凭据并驱动内核动作、GUI 升级不动插件文件、release 包不含插件。

说明:channels.lua / quota.lua 同时带有此前的渠道与额度改动(含 OpenCode 免费池
监控),与 SDK 改名同处一个文件,无法文件级拆分,故一并纳入。
2026-10-26 22:27:12 +08:00
79bc55cfbe fix(billing): 插件 HTTP 路由按操作拆路径 + 正确解析请求体
- 内核插件路由冲突仲裁按**路径**判重(不看 method),同一路径挂 4 个 method 会被
  当成 4 个插件抢同一路径 ⇒ 规则 API 不可用。拆成 /rules(GET)、/rules/replace
  (PUT)、/rules/add(POST)、/rules/remove(DELETE)。
- 插件 HTTP 分发交给 handler 的是**整个请求表**(body 是原始字符串),而直调导出
  拿到的是参数表;此前 PUT 把请求信封当成规则写进了 billing.json,规则全丢、价格表
  清空。新增 reqData() 统一解出 JSON 载荷。
- 生产复验:GET 返回 4 条规则;PUT 后 billing.json 保持规则形状;localzen(free)
  请求不计入 unpriced,codearts 未列价模型计入(预期收紧)。
2026-10-20 18:59:16 +08:00
4b9fb7b056 chore(gui): 版本号 2.0.49(billing 完全沉入插件) 2026-10-20 18:57:27 +08:00
7941b4110a refactor(billing)!: 计费全部沉入插件,内核零 billing 代码;规则由插件自持
按维护者要求(issue #1 回复方向):内核不处理任何 billing 特有逻辑;插件有完整
OS 权限,自己保存配置文件。据此不再保留任何"内核 ↔ billing 插件"的专用对接。

## 内核(删除)
- `internal/billing`(Go 编译器 + 测试):删除;
- `internal/core` 的 billing 接口/降级应用/规则加载:删除(含 `billingPath`);
- `internal/gateway` 的 `/api/plugins/billing/rules` 端点与测试:删除;
- `config` 的 `BillingDSL/Profile/Rule/Token/Peak/LongContext` 与校验、
  `billing:`/`billing_file:` 配置段:删除。系统配置里不再有 billing 段。
内核只保留所有插件通用的能力:加载 .lua、插件状态持久化、插件目录
(`__llmsproxy_plugin_dir`)、`plugin.http` 路由、插件可用的 io/os。

## 插件(billing.lua)
- 规则文件移到**插件自己的目录**:`<plugin_dir>/billing.json`,由插件用 io 自己
  读写(tmp + rename 原子替换);加载插件时自己读文件并编译;
- 规则以**源名** `source` 为键(`url:"*"` 仅表示全局模型价目表):URL→源名映射
  属于内核配置,插件不为它和内核对接;编辑页在浏览器里取 `/api/sources` 选源名;
- 规则编辑 API 改为插件自己的 `/api/plugins/http/billing/rules`
  (GET/PUT/POST/DELETE,roles=admin),写盘也在插件内完成;
- 计费账目仍在插件状态文件(内核通用持久化)`<plugin_dir>/.billing.state.json`;
- `priceFor` 落地 profile 级 `default` 策略与显式 `mode=unpriced` 的优先级。

## 迁移
生产已迁移:`config.yaml` 的 `billing:` 规则转为 `plugins/billing.json`(按源名
重写),并从系统配置中移除;规则文件与系统配置从此各自独立保存。

## 判据
- `internal/lua/billing_rules_test.go`:插件自写 billing.json 并重启恢复、
  free/subscription/token/通配符编译语义、1e6+1e6 token = 0.75 端到端、
  非法规则被拒且不覆盖旧文件、reasoning/long_context/peak;
- `internal/lua/billing_test.go`:profile default 策略三态 + 显式 unpriced 优先;
- `internal/lua/billing_ui_test.go`:规则编辑页仍挂在 billing 页内、列宽/列数守卫。
2026-10-20 18:57:23 +08:00
5ccf068db3 chore(gui): 版本号 2.0.48(与内核同版本) 2026-10-20 18:33:29 +08:00
b4c5f0f0fc refactor(billing): 计费语义下沉到 Lua(内核只解 YAML/传数据)+ 采纳 PR #2 语义
按 issue #1 的修复方向(维护者评论):内核不再持有任何计费语义,billing 逻辑
全部在 `plugins/billing.lua`;内核只负责解 config.yaml、校验 schema、把
profile + 源 base_url 作为**数据**交给插件,再把插件算出的价目通过 `prices`
侧车注入。计费账目仍持久化在插件状态文件(`plugin_dir/.billing.state.json`)。

## 变更
- **删除** `internal/billing`(compile.go 359 行 + 其 Go 测试);
- 新增 `core.compileBillingPrices`:把 profile/sources JSON 化后调用插件导出
  `compile_prices`,返回价目表 + 未匹配规则清单(供错误日志点名 URL);
- `applyBillingDSL` / 规则编辑器改为走同一个插件编译入口(编辑器 lenient,
  半成品规则可保存、可报告);
- `billing.lua` 新增 `compile_prices`:URL 归一匹配、首条命中优先、
  free/subscription/unpriced/token、USD/M→per-token、peak 窗口、
  long_context 边际价、reasoning 可选价、default 策略、未匹配清单。
  错误通过载荷 `error` 字段回传(插件导出第二返回值通道不可靠)。

## 与 PR #2(valuelesser)合并后的语义
- **token 规则不产生任何源条目**:未列价的模型落到 unpriced 统计(可见),
  不再"静默计 $0"。比 issue 正文里的 `{per_request:0}` 标记更收口;
- `default: free` 由**编译期**给未匹配源写 `{per_request:0}` 标记;
  `unpriced`/缺省不写。删除无人读取的 `default_mode` 死键;
- 保留 `unmatched_rules` 清单(只用于降级应用时点名过期 URL)。

## 判据(原 Go 单测平移到 Lua + PR #2 的用例)
- `internal/lua/billing_compile_test.go`:通配符不标记源、token 规则无源条目、
  任意源条目不得携带 token 价(根因直防)、free/subscription 保留有意零价、
  reasoning 缺席、long_context 三种形态、peak 形状、URL 归一+首条优先、
  lenient 未匹配清单、硬错误 5 例、default 两态;
- `internal/lua/billing_test.go`:编译输出驱动真实 priceFor 的 A/B 用例
  (旧形状 cost=0 / 新形状 0.75 且 unpriced=0)、default 编译期语义 e2e;
- `internal/gateway`:token 规则断言"无源条目"、模型价 per-token 量级;
- `internal/core`:过期规则 URL 降级应用(有效规则仍生效)。
2026-10-20 18:33:26 +08:00
442fa5e3f6 chore(gui): 版本号 2.0.47(与内核同版本) 2026-10-20 18:05:19 +08:00
0caa37ea6a fix(billing): 过期规则 URL 不再让整个 profile 失效(降级应用 + 点名 URL)
生产现网发现(issue #1 排查):billing.rules 里一条指向
https://opencode.ai/inference/v1 的规则,在该源 base_url 变成
.../inference/openai/v1 之后匹配不到任何源;严格编译因此让**整个** profile
不生效 —— 账单恒 0、unpriced_reqs 涨到 3.7 万,唯一痕迹是一行启动日志。

- 严格编译失败时降级为 lenient:应用能匹配的规则,其余按 profile default;
- 日志用 ERROR 点名**具体哪条 URL** 写错(新增 billing.UnmatchedRuleURLs),
  并说明未匹配流量的记账去向;
- 判据 TestBillingStaleRuleURLStillAppliesOtherRules:一条过期规则 + 一条有效
  规则 ⇒ 有效规则必须仍然生效。
2026-10-20 18:04:43 +08:00
33cb2bdb06 fix(billing): 源层零价不再覆盖模型价(issue #1)+ default_mode 生效 + 文档/判据补齐
GitCode issue #1:任何按 `mode: token` 且 URL 非通配符的规则定价的源,计费恒为 0,
且 `unpriced_reqs` 也不增长 —— 完全静默。

## 真因
- `compile.go` 给这类规则写源条目 `{prompt:0, completion:0, per_request:0}`,
  本意只是"标记该源已定价"(同源未列入价目表的模型不算 unpriced);
- `billing.lua` 的 priceFor() 源层在模型层之后、且 `sp.prompt ~= nil` 即覆盖;
- 两者叠加 ⇒ 模型价被清成 0,而 `out.priced` 仍为 true ⇒ 连未定价计数都不涨。

## 修法
1. `compile.go`:token 规则的源条目只写 `{per_request: 0}`(只是标记,不参与 token
   计价)。`free` / `subscription` 保留全零(那里 0 本身就是语义)。
2. `billing.lua`:
   - `priceFor()` 的 `out` 补 `local`(此前是 VM 级全局变量,跨请求残留);
   - 实现 `profiles[].default` 的 `free|unpriced` 语义:`free` = 刻意免费(不进
     unpriced),`unpriced`(默认)= 未定价债务。此前 compile.go 写 `default_mode`
     但**无人读**,两种取值运行时完全等价。
3. `core.go`:应用 profile 时把 `unmatched_sources` 打进日志(WARNING 列出没有
   规则命中的源)。该字段此前只写不读,是第二个静默少报点。
4. `docs/plugins.md`:token 价优先级改为与实际实现一致的
   `keys > sources > models > default`,并写明"标记已定价只能用 per_request:0"。

## 判据(都先红后绿)
- `internal/billing`:token 规则的源条目**不含** prompt/completion;free/subscription
  仍保留显式 0。
- `internal/lua`:同源同模型下旧形状 cost=0(缺陷本身)、新形状 cost=0.75 且
  unpriced_reqs=0;未列入价目表的模型仍算"已定价";default_mode 三态。
- `internal/gateway`:`billing_rules_api_test` 的单位断言原本读
  `cc["models"]`(恒 false,整块被跳过——从未执行过),改为 `prices["models"]`,
  并新增"源条目不得带 token 价"断言。

## 顺带修掉一个"永不执行"的判据
`discover_mutate_test.go` 的变异锚点写成 6 个空格,而 `discover.go` 已是 2 个空格
⇒ 该变异每次都 `anchor not found`,变异 campaign 长期红着。修正锚点后,
"嵌套 metadata 不再读取"这条变异真的被执行并被测试 KILL。
2026-10-20 17:46:52 +08:00
0ae7d2c8be test(retry): 钉住声明式重试判据与流式握手重试语义
- config:ShouldRetry/DelayDuration 表驱动(nil/次数=1/状态不匹配/body 不匹配/
  空子串/多信号/非法间隔)
- provider:命中策略就重试直到 200、body 不匹配**不许**重试(400 也常用于真错误)、
  流式只重试握手(首帧之后不重试,避免重复输出)
2026-10-20 10:45:56 +08:00
02c01582f1 chore(gui): 版本号 2.0.46(与内核同版本) 2026-10-20 10:38:07 +08:00
6454b094bc fix(codearts): 上游限流改为可声明重试(内核通用能力)+ 会话/连接两处真因
用户连续报「codeart 还是不可用 / empty stream」。把上游原始响应打出来逐条对齐,
三个症状同源,且都不是"模型坏了":

| 场景 | 上游实际返回 |
|---|---|
| 流式、短时间连续请求 | HTTP **200** + 只带 error 的帧 ⇒ 内核判 `empty stream` |
| 非流式、同一条件 | HTTP 400 `TM.00001041: 并发会话数已达上限(3个)` |
| 突发请求 | HTTP 429 `TPM limit … 500000 tokens per minute` |

实测:最后一个请求结束后约 **44~60 秒**释放;同一账号连续请求 3~4 次即占满。
参考实现(示例插件 llm-adapter.ts)的做法是**每 10s 轮询后重试 chat**(最多 30
分钟),所以用户那边"看起来没事"。

## 我们的改法(内核只做通用机制,厂商知识留在插件)
1. **内核新增声明式重试策略** `source.retry`(纯数据:状态码列表 + 响应体子串 +
   间隔 + 次数)。内核不认识任何厂商;命中即在同一网关请求内每 10s 重试一次。
   流式只在**握手阶段**重试(首帧尚未下发,不会重复输出),非流式整体重试。
2. **插件为 codearts 声明**:400/429 + 命中 `TM.00001041` / `TPM limit` /
   `并发会话数已达上限` → 10s × 最多 18 次(≈3 分钟)。
3. **适配器补两个会话细节**:`Connection: close`(keep-alive 空闲连接会被上游
   计入并发会话)与稳定 `Session-Id`(同一账号恒定,避免每次请求新建会话)。
4. 错误文案补可行动提示(并发会话上限 / TPM 限流,约 1 分钟释放)。

## 验证(生产)
- 连续 6 次**流式**:6/6 成功(撞上限时自动等 10s 再试,客户端只感觉慢):
  `+0s / +12s / +34s / +86s / +100s / +144s`
- 连续 5 次**非流式**:5/5 成功(`+0 / +11 / +20 / +28 / +38s`)
- 修复前:第 4 个请求起必然 TM.00001041 / empty stream。

> 注:`TM.00001005 无正常可用套餐` 属于账号权益问题(不重试、如实报出),
> 与限流要分开处理。
2026-10-20 10:38:00 +08:00
771b95a133 fix(channels): admit_functions 落到 effectiveModels(解析器不承担产品策略)
上一提交把「本适配器会发的通道」过滤写进了 trae 的目录解析器,导致两个解析器判据
回归:同模型在 solo_agent(有思考档位)与 solo_work_lite(无档位)同时出现时,
整组 solo_agent 被丢弃 ⇒ 档位信息丢光,模型显示"未提供推理等级"。

正确分层:
- **解析器**只负责把目录如实解出来(含跨通道元数据合并、白名单准入);
- **产品策略**(只暴露适配器能发的通道)放到 effectiveModels,作用在对外
  暴露的 id 列表上。

实现要点:合并会选"更优"的那条(往往不在我们要发的通道里),所以判据不能看
合并后条目的 chan —— 解析时额外记 `chans`(该 id 在哪些通道出现过),策略按
chans 求交集。夹具同步调整:硬性过滤的条目放进 solo_work_lite(生产里适配器
恒发这条通道),inline_chat 仍用于白名单判据。
2026-10-20 10:12:51 +08:00
441de434f5 chore(gui): 版本号 2.0.45(与内核同版本) 2026-10-20 09:54:06 +08:00
9feef8f4cb fix(channels): 渠道卡片 logged_in 改成看账号池(此前所有池接入的平台都显示未登录)
## 真因:logged_in 判的是旧单账号键
用户报「codearts 还是不可用」。查下来渠道**本身是通的**(9 个模型逐个实测都返回
内容),不可用的是**UI 的登录态判断**:

`list_platforms` 写的是 `credentialOf("channels:" .. platform)` —— 那是**旧版
单账号**的保险库键。现网凭据键是 `channels:<platform>:<accountId>`(一平台多
账号),于是**所有**通过账号池接入的平台都被判成"未登录":

    修复前(生产):trae=False minimax=False codearts=False qoder=False zcode=False
                    (只有 buddy=True —— 它还留着一条旧格式凭据)
    修复后:          trae=True  minimax=True  codearts=True qoder=True  zcode=True

UI 据此把渠道卡片渲染成"未接入",用户看到的就是"渠道不可用"。底层模型完全能用 ——
这是最容易被误判成"上游挂了"的一类假信号。

## 修法
- `list_platforms`:logged_in 改为遍历账号池(`poolList` + 凭据存在 + enabled≠false),
  旧单账号仅作兜底;顺带回 `account_id` / `accounts` 计数,让 UI 能区分同平台多账号;
- `status_page`:`accounts` 同样合并账号池(此前只回旧表,状态页里池接入的平台
  是空的 —— 同一根因的第二个出口);
- 新增 `__platforms_probe`(`list_platforms` 是 HTTP 路由处理器、不在 exports 里,
  判据走探针,与仓库既有做法一致);
- 回归判据 `TestPlatformsLoggedInFromAccountPool`:池里有凭据账号 ⇒ logged_in=true,
  且 account_id / account / accounts 回池数据。

## 顺带核实(不是代码缺陷)
- codearts 目录里 4 个模型(`deepseek-v4-flash-0731` / `glm-5.3-flash` /
  `deepseek-v4-pro-0813` / `CodeArts-Pangu`)实测上游 404
  `InferHub.002002009.404: The model is not registered` ⇒ 之前被手工停用是对的,
  已按死模型流程收口(不再回到可选列表);
- `GLM-5.3` 同样 404,已标失效并自动从源里剔除(日志:
  「模型已失效,已标记 codearts/GLM-5.3」);
- `deepseek-v4-flash` 早前的 429 是 TPM 限流(我自己连续压测造成),现已恢复 OK。
- 最终 codearts:**9 个模型,9/9 实测返回内容**。
2026-10-20 09:54:06 +08:00
155 changed files with 4172 additions and 3392 deletions

497
HANDOFF_2026-10-19.md Normal file
View File

@ -0,0 +1,497 @@
# ModelRouter / llmsproxy 交接文档
更新时间:2026-10-19
工作目录:`/home/program/llmsproxy`
当前分支:`release/v2.0.x`
## 1. 当前 Git 状态
最近提交:
- `c9f3453 fix(codearts): 识别 InferHub 未注册模型`
- `12bfd53 fix: 收口渠道源迁移与测试页合并`
- `25970a2 refactor(scheduler)!: 移除 priority 废弃字段 + 一通道一源 + discover 复用凭据`
`c9f3453` 已推送到 `origin/release/v2.0.x`。
注意:当前工作树在最后一轮添加 provider 回归测试时出现了一个**未完成且可能无法编译的测试编辑**。最后一次工具报错:
```text
internal/provider/provider_test.go:99: missing ',' in argument list
```
下一位 agent 接手后的第一步必须是:
```bash
cd /home/program/llmsproxy
git status --short
gofmt -w internal/provider/provider_test.go 2>&1 || true
go test -tags luajit ./internal/provider/ 2>&1
```
如果测试文件处于半编辑状态,先检查 `TestProviderChatStreamNormalizesStopAfterToolCall`,修正 Go 字符串转义或删除该未完成测试后再继续。不要直接部署当前工作树。
## 2. 生产服务状态
生产服务:`llmsproxy.service`
- 当前服务 active
- 监听:`0.0.0.0:8081`
- 二进制已通过 `./deploy.sh` 原子替换并重启
- 最近一次部署健康检查通过
- 当前生产插件 SHA256 前缀曾核对为:`1bf26cbb3abcb107`
- 当前生产二进制在最近部署后已更新
不要把密钥写入交接记录。生产 admin key 保存在本机运维环境中,不应复制到聊天、文档或提交。
## 3. Channel 源当前状态
生产源已经从旧的“一账号一源”收口成“一渠道一源”:
```text
channels-trae
channels-minimax
channels-lobsterai
channels-buddy
channels-codearts
channels-zcode
```
旧形态源已经清理:
```text
channels-<provider>-<account_id>
```
此前旧源和新源并存,造成同一渠道同一模型出现多份,只能靠账号 hex 区分。
根因已修复:内核 `list_sources` 返回裸数组,但旧插件代码按 `{sources = [...]}` 信封读取,导致迁移清理循环永远为空。当前 `channels.lua` 已兼容:
- 裸数组
- `{sources = [...]}`
- `{json = {sources = [...]}}`
相关代码:
```text
internal/lua/plugins/channels.lua: syncProviderSource
internal/core/channels_legacy_source_cleanup_test.go
```
当前生产源列表已实测只有 6 个新形态源。
## 4. CodeArts 当前状态
### 已确认可用的部分
CodeArts 使用:
```text
adapter = codearts
base_url = https://snap-access.cn-north-4.myhuaweicloud.com
```
CodeArts 的 chat 请求需要完整 AK/SK/security token 凭据,由插件适配器签名,不是裸 Bearer token。
真实流式工具调用测试已经成功:
请求形状:
```json
{
"model": "channels-codearts:openpangu-2.0-flash",
"messages": [{"role": "user", "content": "Use the weather tool for Beijing."}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get weather",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}
}],
"tool_choice": "auto",
"stream": true,
"max_tokens": 64
}
```
结果:
- HTTP 200
- `Content-Type: text/event-stream`
- 返回结构化 `tool_calls`
- 函数名 `get_weather`
- 参数最终为 `{"city":"Beijing"}`
- `finish_reason = tool_calls`
因此 CodeArts 的流式工具调用链路当前是通的。
### CodeArts 模型刷新
生产调用:
```http
POST /api/plugins/http/channels/model/refresh
{"platform":"codearts"}
```
实测返回:
```json
{
"ok": true,
"results": [
{"platform": "codearts", "ok": true, "count": 11}
]
}
```
CodeArts 模型目录来自两个签名端点并合并:
1. `opengw.developer.huaweicloud.com/api/v1/gateway/config`
- `result.models`
2. `snap-access.cn-north-4.myhuaweicloud.com/v1/model/builtin`
- `builtinModels`
相关解析:
```text
internal/lua/plugins/channels.lua: parseRemoteModels
internal/lua/plugins/channels.lua: refreshRemoteModels
internal/core/channels_codearts_catalog_test.go
```
### 已确认失效并禁用的 CodeArts 模型
下面 4 个模型曾收到明确的上游 404:
```text
CodeArts-Pangu
deepseek-v4-flash-0731
deepseek-v4-pro-0813
glm-5.3-flash
```
生产上已通过 `channels/model/set_disabled` 禁用。它们仍可能在 `model/list` 中出现,但 `disabled=true`,不会作为可用模型路由。
新增的死模型判据:
```lua
model is not registered
```
对应提交:`c9f3453`。
注意:当前插件的 dead-model 机制只在收到真实请求失败后记录模型;刷新目录不会自动根据“远端目录没有该模型”删除模型,这是刻意设计,避免误删远端目录不完整的平台模型。
CodeArts 当前仍有一些模型返回 429/TPM 限流,例如:
```text
deepseek-v4-flash
deepseek-v4.1-flash
```
429 不是模型失效,不能禁用。
## 5. 其他 Channel 实测结果
### Trae
真实流式 + tools 请求:
```text
channels-trae:DeepSeek-V4-Flash
```
结果:
- HTTP 200
- SSE 正常返回
- 结构化 tool call 返回
- 但上游最后的结束原因可能是 `stop`,即使前面已经出现 tool-call fragments
这会让 Agent 把工具调用误判成普通文本完成。
当前准备中的修复在统一 provider 流式层:
```text
internal/provider/provider.go
```
逻辑是:只要本次流出现真实 tool-call 片段,后续 `FinishReason == "stop"` 就归一成 `tool_calls`。
另外,Trae 的非流式接口偶尔返回 SSE。当前 `trae.lua` 对非流式 SSE 只取最后一条 data,不能完整合并多条工具调用/推理片段。这是后续仍需处理的协议问题,但不能用非流式手工请求结果推断 Pi 的真实请求模式。
### Buddy
真实流式 + tools 请求:
```text
channels-buddy:glm-5.3-flash
```
结果:
- HTTP 200
- SSE 正常返回
- tool-call 参数分片可见
- 最终有 `finish_reason = tool_calls`
- 但某些上游 chunk 的 `tool_calls` 可能是空对象 `{}`
空对象不是合法的 OpenAI tool-call 数组,部分 Agent 客户端会拒绝。
当前准备中的修复在:
```text
internal/lua/adapters/buddy.lua
```
只在 `type(delta.tool_calls) == "table" and #delta.tool_calls > 0` 时输出 `tool_calls`,避免 Lua 空表被编码为 `{}`。
Buddy 的非流式错误:
```text
11101: Non-stream chat request is currently not supported
```
之前出现的该错误来自手工 `curl` 测试中显式设置 `stream:false`,不能归因到 Pi。真实 `stream:true + tools` 已经证明 Buddy 流式工具调用可以工作。
### MiniMax
生产日志反复出现:
```text
adapter "anthropic": status=401 body={"error":"invalid access token"}
provider channels-minimax: api error 401
```
这是凭据失效问题,不是 tools 形状问题。
当前源:
```text
channels-minimax
adapter = anthropic
endpoint = /mavis/api/v1/llm/v1/messages
```
需要用户重新登录/提供有效凭据。不要擅自刷新一次性 refresh token,之前已经发生过 refresh token 被消耗后无法再次使用的事故。
## 6. 当前未提交的代码改动
最后一轮已经改动但尚未完成提交:
```text
internal/provider/provider.go
internal/lua/adapters/trae.lua
internal/lua/adapters/buddy.lua
internal/provider/provider_test.go # 可能有半编辑、需先修复
```
已完成的意图:
1. provider 流式层记录 `sawToolCalls`
2. 看到真实 tool-call 后,把后续 `stop` 归一为 `tool_calls`
3. Trae 不输出空工具调用表
4. Buddy 不输出空工具调用表
5. provider 回归测试模拟“tool fragment + stop”场景
但 provider 测试最后一次编辑工具报错:
```text
internal/provider/provider_test.go:99: missing ',' in argument list
```
因此必须先修测试文件,再提交。
## 7. 推荐接手顺序
### 第一步:修复工作树
```bash
cd /home/program/llmsproxy
git status --short
sed -n '1,150p' internal/provider/provider_test.go
gofmt -w internal/provider/provider_test.go 2>&1 || true
go test -tags luajit ./internal/provider/ 2>&1
```
若测试函数字符串转义难以修复,先删除半成品测试,再重新用 raw string 写测试。
### 第二步:跑定向验证
```bash
luac -p internal/lua/adapters/trae.lua
luac -p internal/lua/adapters/buddy.lua
go test -tags luajit ./internal/provider/
go test -tags luajit ./internal/lua/ -run 'Adapter|Buddy|Trae|Stream'
```
### 第三步:构建并部署临时验证版
修复确认后:
```bash
go build ./...
./deploy.sh
```
部署前确认:
```bash
sha256sum internal/lua/plugins/channels.lua
sha256sum /etc/llmsproxy/plugins/channels.lua
systemctl is-active llmsproxy
```
### 第四步:真实流式回归
不要用 `stream:false` 测试流式问题。使用:
```bash
KEY='<从本机安全环境读取,不写入文档>'
TOOL='{"type":"function","function":{"name":"get_weather","description":"Get weather","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}'
curl --no-buffer --max-time 90 \
-H "Authorization: Bearer $KEY" \
-H 'Content-Type: application/json' \
-d "{\"model\":\"channels-codearts:openpangu-2.0-flash\",\"messages\":[{\"role\":\"user\",\"content\":\"Use the weather tool for Beijing.\"}],\"tools\":[$TOOL],\"tool_choice\":\"auto\",\"stream\":true,\"max_tokens\":64}" \
http://127.0.0.1:8081/v1/chat/completions
```
分别将 model 替换为:
```text
channels-codearts:openpangu-2.0-flash
channels-trae:DeepSeek-V4-Flash
channels-buddy:glm-5.3-flash
```
验收:
- HTTP 200
- `Content-Type: text/event-stream`
- 不能出现 `tool_calls: {}`
- 有 tool-call fragments 时最终必须是 `finish_reason: tool_calls`
- CodeArts 当前应继续返回结构化工具调用
### 第五步:CodeArts 刷新验收
```bash
curl -sS -X POST \
-H "Authorization: Bearer $KEY" \
-H 'Content-Type: application/json' \
-d '{"platform":"codearts"}' \
http://127.0.0.1:8081/api/plugins/http/channels/model/refresh
```
预期:
```json
{"ok":true,"results":[{"platform":"codearts","ok":true,"count":11}]}
```
再检查:
```bash
curl -sS -X POST \
-H "Authorization: Bearer $KEY" \
-H 'Content-Type: application/json' \
-d '{"platform":"codearts"}' \
http://127.0.0.1:8081/api/plugins/http/channels/model/list
```
预期:远端目录 11 个 + 兜底补位,已确认 404 的 4 个仍保持 `disabled=true`。
## 8. 发布/安装包状态
目前没有完成“全部安装包”发布闭环。
已经发生的事情:
- 核心包构建成功,但脚本实际使用了 `git describe`,产物版本类似:
`2.0.40~4-gc9f3453`
- Linux GUI 包构建成功,版本为 `1.9.7`
- Windows 主机 Wine 构建失败,生成了一个 264KB 的坏 `ModelRouter Setup 1.9.7.exe`,绝对不能发布
- Docker Windows 构建也失败在拉取 Docker Hub 基础镜像 `golang:1.25` 网络超时
- apt 无法找到 `wine32:i386`
- 旧的 1.9.5/1.9.6 Windows 包是历史产物,不能作为当前版本
不要把旧 Windows exe 当作新包发布。
仓库已有 Docker 方案:
```text
cmd/gui/scripts/dist-win-docker.sh
cmd/gui/docker/win-builder/Dockerfile
```
但当前 Docker Hub 网络不可用。若要继续做 Windows 包:
1. 在可访问 Docker Hub 的构建机/CI 执行 Docker 方案;或
2. 提供包含 Wine32/Wine64 的完整构建环境;或
3. 暂时只发布 Linux/core 包,并明确说明 Windows 包未完成。
版本脚本注意:
- 核心包脚本使用位置参数:
```bash
./packaging/core-dist.sh 2.0.41
```
不是 `VERSION=2.0.41 ./packaging/core-dist.sh`。
- GUI 版本来自 `cmd/gui/package.json`,环境变量 `VERSION` 不会自动修改 Electron 产品版本。
- 清理旧产物后再打包,并在全部包完成后统一计算 `SHA256SUMS`。
## 9. 参考插件状态
用户要求按:
```text
https://github.com/Ebony-Vinyl/dsh-our-free-model
```
逐厂商对照参考插件。
当前本地路径:
```text
/tmp/pi-github-repos/Ebony-Vinyl/dsh-our-free-model
```
但该目录当前不是有效可读源码副本:
- `vendor/channel-pack/src` 下面只有空目录;
- `.git` 没有有效 `HEAD`/refs;
- 无法从本地 Git 对象恢复源码;
- 本地也没有 `buddy.ts`、`trae.ts`、`codearts.ts` 等文件。
因此不能声称当前适配器是逐行照参考插件移植的。若要继续按用户要求完成,必须先重新 clone/恢复该 GitHub 仓库源码,确认以下文件实际存在后再对照:
```text
vendor/channel-pack/src/trae.ts
vendor/channel-pack/src/buddy.ts
vendor/channel-pack/src/codearts.ts
vendor/channel-pack/src/*adapter*
vendor/channel-pack/src/*model*
```
当前仓库代码只能作为临时协议修复基础,不能代替参考插件源码。
## 10. 关键结论
- Pi 是否发送流式请求,历史日志没有保存入站 `stream/tools` 字段,不能靠旧日志判断。
- 本次真实 `stream:true + tools` 测试证明 ModelRouter 流式路径能工作:
- CodeArts:通过;
- Trae:通过但结束原因需归一;
- Buddy:通过但需过滤空 `{}`;
- MiniMax:凭据 401,独立问题。
- 不要再用 `stream:false` 的手工请求推断 Pi 的问题。
- CodeArts 刷新接口当前真实返回 `ok=true,count=11`,刷新链路不是当前阻塞点。
- 当前最紧急的代码状态是修复 `provider_test.go` 的半编辑,然后验证未提交的统一流式修复。
- 未经测试、未重新构建、未重新部署,不要继续改生产账号或刷新一次性凭据。

View File

@ -2,7 +2,7 @@
# ModelRouter - single-binary gateway + optional Electron desktop GUI (embedded core)
export BINARY
BINARY=llmsproxy
BINARY=modelrouter
export BUILD_DIR
BUILD_DIR=build
export GUI_DIR
@ -20,7 +20,7 @@ all: build
# ---- core binary (server users / embedded core) ----
build:
@mkdir -p $(BUILD_DIR)
CGO_ENABLED=1 go build -tags luajit -trimpath -o $(BUILD_DIR)/$(BINARY) ./cmd/llmsproxy
CGO_ENABLED=1 go build -tags luajit -trimpath -o $(BUILD_DIR)/$(BINARY) ./cmd/modelrouter
@echo "Built: $(BUILD_DIR)/$(BINARY) ($(VERSION))"
# ---- core distribution packages (deb + rpm + tar.gz) -----

674
README.md
View File

@ -4,103 +4,46 @@
>
> **AI-assisted**: 本项目使用 AI 辅助编程(代码与文档由 AI 协作完成,经人工复核)。
面向多 llms 订阅者,部署在内网,实现一次配置多个服务共同使用的效果。支持指定模型或 auto 模式,按照配置的优先级选择可用模型提供服务。
面向多 LLM 订阅者的内网网关:一次配置,多个服务共用一组上游额度。支持指定模型或
`AUTO` 模式,按配置的优先级选择可用模型提供服务。
> **轻量且增源无需重编译**:网关本体是单 Go 二进制(约 10MB,零运行时依赖)。新增/切换上游
> 只需在 `config.yaml`(或 WebUI)加一个 `sources` 条目或挂一个 `.lua` 适配器——**不改 Go、
> 不重编译**。源与适配器的改动经 WebUI 提交时即时生效(热更新);直接编辑
> `config.yaml` 或 `adapter_dir` 下的 `.lua` 文件则需要重启进程生效。
网关本体是单个 Go 二进制(约 10 MB,零运行时依赖)。新增或切换上游只需在
`config.yaml`(或 WebUI)加一个 `sources` 条目、或挂一个 `.lua` 适配器——**不改 Go、
不重编译**。源与适配器的改动经 WebUI 提交时即时生效(热更新);直接编辑
`config.yaml` 或 `adapter_dir` 下的 `.lua` 文件需要重启进程生效。
## 核心优势
## 核心能力
### 极致轻量
- **单二进制**:编译后 8~12MB(`-s -w` strip 后约 8MB),零运行时依赖(仅依赖系统 libc),部署即用
- **低内存占用**:与源数量相关,非与运行时长相关——单源 ~10MB、16 源生产实例 ~32–35MB([实测分解与调优](#内存占用实测与调优))
- **日志按需加载**:统计数字来自**全量**审计日志(启动时流式扫一遍即释放,29MB/22 万行约 260ms),但原始记录不常驻内存——默认只加载首屏,下滚自动分页,CSV 导出流式写出(O(1) 内存),离页即释放
- **零运行时依赖**:纯 Go + LuaJIT 静态链接,无需安装 Python/Node/Java 等运行时
- **启动极快**:冷启动 < 200ms,热重载配置 < 10ms
### 强大的多租户调度能力
- **多密钥多租户**:支持无限密钥,每个密钥独立角色、模型范围、Token 配额、重置周期
- **按模型配额**:每把 key 的每个模型单独配 token + 请求数配额与重置周期(小时/周/月/自定义 N 小时);一个模型用满只影响该模型,同 key 其它模型照常;耗尽返 429 + `Retry-After` 并点名模型,admin key 永不受限
- **AUTO 智能调度**:基于优先级档位的分级调度,同优先级源自动轮询负载均衡,故障自动毫秒级故障转移
- **自愈冷却**:冷却上限 5 分钟,过半后放行 1 个探测请求,上游/额度恢复即刻回归轮询,无需等满冷却窗口
- **Token 配额管理**:精确到模型级别的 Token 配额控制,支持小时/周/月/自定义小时周期自动重置
- **同优先级源负载均衡**:同一优先级档位的多个源,请求自动 Round-Robin 均匀分发,故障毫秒级故障转移
- **Source-Model 前缀路由**:支持 `source-model`/`source:model`/`source/model` 精确指定上游源
### 生产级可靠性
- **热加载配置**:WebUI 修改源/密钥/适配器/AUTO链即时生效,无需重启
- **密钥加密落盘**:AES-256-GCM 加密存储 `api_key`、网关密钥、请求头,master.key 0600 权限保护
- **实时源探测**:`GET {base}/models` 定期探测,毫秒级感知上游状态,不污染调度退避状态
- **全链路审计**:HTTP 访问日志、登录/配置变更审计、请求记录 CSV 导出、密钥用量统计 CSV 导出
### 灵活的协议适配
- **LuaJIT VM**:每个适配器独立 VM + **弹性 worker 池**(按需扩缩容,空闲时自动回收),安全并发,Lua 脚本热加载无需重启
- **Lua 适配器协议**:`transform_request` / `transform_response` / `transform_stream_chunk` 双向转换
- **动态请求头钩子**:`build_headers(meta)` 支持 HMAC 签名、动态 Header 注入
- **多模态透传**:`image_url` 等多模态内容在多源间无损透传,Anthropic/Gemini/Ollama 自动转换
## 实现
统一 OpenAI 兼容网关:把多个上游 LLM 源(DeepSeek、Qijiar、OpenAI、Anthropic、Gemini、
Groq、Mistral、Ollama、KimiCode…)通过 **Lua 适配器** 做协议转换,对内网暴露
一个标准 `OpenAI Chat Completions` 接口(`/v1/chat/completions` + `/v1/models`),
支持单次结算、SSE 流式、**AUTO 模型路由**与**多模态**透传。
从 [HomeAgent](https://gitcode.com/JianFeeeee/HomeAgent) 的多源 LLM 适配层
(`internal/agent/api/provider.go` + `internal/lua/adapters/*`)抽离而来并独立演进。
## 特性
- **多源**:一个进程内配置任意多个上游源,按请求的 `model` 自动路由。
- **AUTO 模式**:`default_model: AUTO` 时按优先级页保存的 AUTO 链档位逐档调度并发请求。
- **统一输出**:所有源对外都是 OpenAI 格式(含 `reasoning_content`、`tool_calls`、`usage`)。
- **生图**:`POST /v1/images/generations`,`kind: image` 的模型独立路由。
- **多模态**:`content` 数组(`image_url` 等)在多源间无损透传,Anthropic/Gemini/Ollama 自动转换。
- **LuaJIT VM**:基于 golua 绑定的 LuaJIT,每个适配器独立 VM + worker 池,安全并发。
池是 **弹性** 的:`Σ max_concurrent` 只是上限而非预分配,状态按需创建、空闲时自动回收,
因此空载网关几乎不持有 Lua 状态(启动 0 个)。
- 扩容步长由适配器最大并发决定:`clamp(ceil(max/8), 1, 8)`,且仅在**真实并发争用**
(所有现有状态均已占用)时批量预热,顺序流量始终只用 1 个状态。
- 缩容步长由当前连接数决定:`clamp(ceil(冗余/(1+占用)), 1, 冗余)`——无连接时一轮收到
常驻下限,忙时每轮只释放 1 个,保护热路径。WebUI「适配器」页展示实时池指标。
- **thinking 预算**:请求里的推理开关/预算归一为中立的 `thinking:{enabled,budget_tokens}`,
适配器映射到各自协议(Anthropic `thinking.budget_tokens`、Gemini `thinkingConfig`、
DeepSeek `thinking.type="disabled"`)。
- **Lua 适配协议**:每个源挂一个 `.lua` 适配器,完成 `transform_request` /
`transform_response` / `transform_stream_chunk` 双向转换,协议差异全在 Lua 层。
- **签名 / 请求头钩子**:适配器可定义 `build_headers(meta)`,在 Go 发 HTTP 前
动态注入/签名请求头——用于云端 API 校验调用方 app(如 KimiCode 只放行特定
agent)。提供 `hmac_sha256_hex` / `sha256_hex` / `base64_encode` 等签名辅助。
- **WebUI**:内置管理界面,可在线查看/新增/编辑上游源与模型、配置 AUTO 优先级链、
管理密钥模型范围,写入运行时文件持久化。
- **密钥加密存储**:运行时文件中的上游 `api_key`、自定义请求头值、网关 key 均以
AES-256-GCM 加密落盘(`master.key` 独立 0600,或 `LLMS_PROXY_MASTER_KEY`)。
- **实时源探测**:状态页探测各源可达性(`GET {base}/models`,失败回退最小请求),
不污染正常调度的退避状态,错误信息在 UI 可悬停查看。
- **鉴权**:网关自身用 `gateway_keys` 校验客户端 Bearer key;与上游各自的 key 相互独立。
- **流式**:SSE `chat.completion.chunk`,含角色首包与 `[DONE]` 收尾。
- **全链路审计**:HTTP 访问日志、登录/配置变更审计、请求记录 CSV 导出、密钥用量统计 CSV 导出
- **AES-256-GCM 加密存储**:运行时文件敏感字段加密落盘,master.key 0600 权限,支持环境变量注入主密钥
- **统一接口**:对内暴露 OpenAI 兼容的 `/v1/chat/completions`、`/v1/models`、
`/v1/images/generations`;所有源对外都是同一套格式(含 `reasoning_content`、
`tool_calls`、`usage`)。
- **多源 + AUTO 链**:一个进程内任意多个上游源;`default_model: AUTO` 时按优先级页
保存的档位逐档调度,源故障自动降级并冷却,恢复后半冷却探测自愈。
- **协议适配在 Lua 层**:每个源挂一个 `.lua` 适配器,完成请求/响应/流式分片的双向
转换,并可用 `build_headers(meta)` 注入签名头。协议差异不进 Go 内核。
- **插件独立**:业务扩展(渠道接入、计费、额度巡检)是外置 Lua 插件,由插件目录加载,
不编进二进制、不随发行包分发;插件只通过 `modelrouter` SDK 与内核交互。
- **多租户密钥**:网关 key 与上游 key 相互独立,按 key 限定模型范围、AUTO 链与配额。
- **加密存储**:上游 key、自定义请求头、网关 key 均以 AES-256-GCM 落盘,
`master.key` 单独 0600 权限(或由 `MODELROUTER_MASTER_KEY` 注入)。
- **可观测**:状态页实时探测源可达性;HTTP 访问日志、配置变更审计、请求与用量 CSV 导出;
写操作不落在请求热路径上。
## 快速开始
```bash
GOMODCACHE=... GOPROXY=off go build -tags luajit -o llmsproxy ./cmd/llmsproxy
./llmsproxy -config config.yaml # 首次运行自动生成默认配置并打印随机 admin key
# 需要 LuaJIT 开发库;-tags luajit 是必须的
GOMODCACHE=... GOPROXY=off go build -tags luajit -o modelrouter ./cmd/modelrouter
# 首次运行会生成默认配置,并在日志里打印随机 admin key
./modelrouter -config config.yaml
```
> 依赖 [golua](https://github.com/aarzilli/golua)(LuaJIT 绑定)。**必须**带 `-tags luajit`
> 构建,否则默认走内置超集 gopher-lua 路径(行为略有差异)。
> 依赖 [golua](https://github.com/aarzilli/golua)(LuaJIT 绑定)。不带 `-tags luajit`
> 会走 gopher-lua 路径,行为略有差异。
> **配置不入库**:仓库不携带任何 config 文件(配置文件含密钥)。二进制首次运行时
> 会在 `-config` 指定的位置生成默认配置:随机 admin key(打印在启动日志中)、
> 仅绑定 `127.0.0.1:8080`。首次登录后请在 WebUI「密钥」页更换管理员密钥。
> **配置不入库**:仓库不携带任何 config 文件(其中含密钥)。默认配置只绑定
> `127.0.0.1:8080`,首次登录后请在 WebUI「密钥」页更换管理员密钥。
```bash
# 无 key -> 401
@ -117,526 +60,73 @@ curl -N -H "Authorization: Bearer $KEY" \
http://127.0.0.1:8080/v1/chat/completions
```
任何 OpenAI SDK 把 `base_url` 指到网关地址、`api_key` 用任意一个已创建并授权的网关 key 即可。
任何 OpenAI SDK 把 `base_url` 指向网关、`api_key` 换成任一已授权的网关 key 即可。
## 配置
## 文档
首次运行在 `-config` 路径自动生成默认配置(默认 `./config.yaml`),核心字段:
| 主题 | 文档 |
|---|---|
| 实现与完整特性清单 | [docs/features.md](docs/features.md) |
| 配置:`config.yaml`、自动探测模型、多密钥 | [docs/configuration.md](docs/configuration.md) |
| 路由:模型路由、AUTO 链、冷却与半冷却、源与适配器、源模板 | [docs/routing.md](docs/routing.md) |
| 适配器:Lua 协议、内置适配器清单 | [docs/adapters.md](docs/adapters.md) · [docs/lua-adapters.md](docs/lua-adapters.md)([EN](docs/lua-adapters-en.md)) |
| 插件:SDK 契约、钩子、UI 扩展、HTTP 端点 | [docs/plugins.md](docs/plugins.md) |
| 桌面端:Headless / Desktop 两种产物、构建与运行 | [docs/gui.md](docs/gui.md) |
| 性能:实测内存占用与调优、thinking 预算 | [docs/performance.md](docs/performance.md) |
| WebUI 使用 | [docs/webui.md](docs/webui.md) |
| 开发:目录职责、测试、分支与发布纪律 | [docs/development.md](docs/development.md) · [docs/git-workflow.md](docs/git-workflow.md)([EN](docs/git-workflow-en.md)) |
| 部署与迁移 | [docs/deployment.md](docs/deployment.md) |
```yaml
listen: 127.0.0.1:8080 # 网关监听地址(建议绑内网/回环)
gateway_keys: [sk-gw-<随机>] # 初始 admin 密钥(每次安装随机生成,仅首启写入运行时存储用)
default_model: AUTO # model 无法路由时自动按优先级链选源
adapter_dir: adapters # Lua 适配目录;目录不存在时首启创建并 seed 内置,存在则只读
runtime_file: runtime.json # WebUI 编辑的源/密钥/AUTO 链持久化到此文件
## 从 llmsproxy 改名
sources:
- name: deepseek
base_url: https://api.deepseek.com
api_key: sk-... # 也可用 api_key_env: SOME_ENV 引用环境变量(不落盘明文)
adapter: deepseek
max_concurrent: 8
models:
- id: deepseek-v4-flash
priority: 100 # YAML 源首次启动会 seed 进 AUTO 链的初值
kind: chat
- id: deepseek-v4-pro
priority: 60
kind: chat
# 静态请求头(比适配器默认优先)
headers: { X-Tenant: prod }
# 传给 Lua build_headers 的透传元数据
meta: { app_id: x, app_secret: y }
temperature: 0.7
max_tokens: 4096
timeout: 120s # 请求超时,默认 120s
```
产品名早已是 ModelRouter,二进制、配置目录、服务名与环境变量现已统一为
`modelrouter`:
| 旧 | 新 |
|---|---|
| `llmsproxy`(二进制) | `modelrouter` |
| `/etc/llmsproxy/` | `/etc/modelrouter/` |
| `llmsproxy.service` | `modelrouter.service` |
| `LLMSPROXY_*` | `MODELROUTER_*` |
### 自动探测模型列表
不想手抄模型 id 时,只填 URL 和 key,让网关去扫:
已部署的机器用迁移脚本升级(**默认 dry-run**,复制不移动,可回滚):
```bash
# 只探测,不写任何东西
curl -X POST http://127.0.0.1:8080/api/sources/discover \
-H "Authorization: Bearer sk-gw-…" -H "Content-Type: application/json" \
-d '{"base_url":"https://api.vendor.com/v1","api_key":"sk-…"}'
# → {"ok":true,"base_url":"…","endpoint":"","models":["gpt-x",…],
# "image_models":["gpt-image-2"],"models_path":"/models","latency_ms":1387,…}
# 探测后直接落库
curl -X POST http://127.0.0.1:8080/api/sources/discover/apply \
-H "Authorization: Bearer sk-gw-…" -H "Content-Type: application/json" \
-d '{"base_url":"https://api.vendor.com/v1","api_key":"sk-…","name":"vendor"}'
sudo ./packaging/migrate-to-modelrouter.sh # 预演
sudo ./packaging/migrate-to-modelrouter.sh --apply # 执行
sudo ./packaging/migrate-to-modelrouter.sh --rollback --apply
```
WebUI 源编辑框里也有「自动探测模型」按钮:填好 URL 和 key 点一下,列表自动
填好,仍需点保存才生效。
几点值得知道:
- **URL 形态不用猜。** 厂商对「`/v1` 放 base 还是放 path」没有共识,探测会
依次尝试若干候选并返回答对的那个,同时给出归一化后的 `base_url` +
`endpoint`(`endpoint` 平时留空,因为 openai adapter 自带 `/chat/completions`)。
- **`image_endpoint` 是另一套规则。** 网关的图像默认路径是**绝对**的
`/v1/images/generations`,而 chat 默认是相对的。所以 `base_url` 带 `/v1` 时
必须显式给 `image_endpoint: /images/generations`,否则图像请求 404。
探测会替你填好。
- **图像模型只看上游自己声明的 modality**,绝不从模型名猜。`gpt-image-2`
这类 id 也出现在同时提供对话服务的目录上,猜错会让 AUTO 链选中一个只支持
图像生成的模型,其后每一层都被它拖垮。
- **不会逐模型探活。** 34 个模型的目录跑一轮要花钱且耗时数分钟,`/models`
是足够便宜的信号;模型是否真能应答留给正常请求去发现。
- **对已存在的源,必须显式 `replace_models: true`** 才会覆盖模型列表。重新
探测顺手抹掉为 AUTO 链手工调过的 priority,是最难排查的一类数据丢失。
- **`api_key` 留空表示沿用已存值**,不会把凭据清空。
`config.yaml`、`runtime.json`、`master.key`、`adapters/` 均不提交版本库(见
`.gitignore`),请随备份带走——`master.key` 丢失后运行时密文无法解密。
运行文件生成在 `-config` 所在目录:`adapter_dir` / `runtime_file` 自动指向
配置文件旁边的 `adapters/` 与 `runtime.json`,从任意工作目录启动都可用。
运行时文件(`runtime_file`)中的敏感字段自动加密:
- 加密算法 AES-256-GCM,格式 `enc:v1:<base64>`。
- 主密钥来源:环境变量 `LLMS_PROXY_MASTER_KEY`(64 位 hex);否则读取
`runtime_file` 同目录的 `master.key`;都不存在时首次启动自动生成 `master.key`(0600)。
- 升级时旧明文文件自动兼容:首次运行正常读取,任何 UI 保存操作触发全文件加密迁移。
- 注意:**master.key 丢失后密文无法解密**,请随配置一起备份;切勿提交到版本库。
### 网关密钥(多密钥)
网关鉴权采用「多密钥 + 角色 + 模型范围」架构,密钥持久化在 `runtime_file` 的
`keys` 字段(加密存储):
- `gateway_keys` 配置只是**初始 admin 密钥种子**:首次启动迁移为运行时 admin
key,之后不再参与鉴权管理。
- **重要:首次启动后请在 WebUI「密钥」页更换管理员密钥**——初始密钥明文写在
`config.yaml` 里(每次安装随机生成),继续使用存在被盗风险;用新密钥登录后删除初始密钥。
- WebUI **密钥页**可创建/删除密钥;每个密钥可指定 `admin`(管理全部)或
`user`(仅看自己的 key)角色,并配置**模型范围**(模型 + 源 + token 配额 +
重置周期)。
- 客户端用任意一个已授权的密钥明文作为 Bearer(`Authorization: Bearer <key>`)。
- 删除密钥即从运行时存储移除,立即失效。
#### 密钥用量配额
**配额按模型单独设置**:每个密钥的 `models[]` 里,每一条模型范围各自带一份
token 配额与请求数配额。一个模型用满只影响该模型,同一密钥的其它模型照常工作。
```yaml
keys:
- key: sk-gw-<hex>
role: user
name: agent-alice
models:
- model: deepseek-v4-flash
token_quota: 1000000 # 本周期内该 key 用这个模型的 token 预算
req_quota: 20000 # 本周期内的请求次数
period: nhour # "" | hour | week | month | nhour
hours: 6 # 仅 nhour
- model: AUTO
token_quota: 5000000 # AUTO 也是一条独立配额
period: day # ← 这种写法会被拒绝(词表只有 hour/week/month/nhour)
- model: kimi-k3 # 未列配额 = 无限
```
- **没有「整钥总配额」**:这是刻意的设计。整钥总额会让「把 A 模型的额度挪给
B 模型」变成一次全局重分配;按模型独立计费则每个模型各自可控,运维可以
看出哪个模型吃掉了预算。
- `period` 词表:空 = 永不过期(累计总量),`hour` / `week` / `month` = 固定窗口,
`nhour` + `hours` = 自定义小时数。**拼错的周期在写入时就被拒**,不会静默变成
永不过期。
- 配额严格按密钥隔离,且**同一密钥内按模型隔离**:A 密钥用满 `m1` 不会消耗
B 密钥的额度,同一密钥的 `m2` 也不受影响。
- 配额统计含聊天、流式、生图,跨重启从审计日志回放(保留 40 天,覆盖最长的
month 窗口)。
- 配额耗尽返回 **429 + `Retry-After`**(`rate_limit_exceeded`),消息里点名是哪个
模型用满了,客户端可等窗口重置后自动恢复;模型越权才是 403。
**admin 密钥永不受配额限制**(其 scope 上的配额也不强制),避免把管理员
锁在门外。
- 窗口用量按整点小时分桶统计,实际释放比配置窗口最多晚 1 小时(配额宁可晚释放
也不超发)。
- `PUT /api/keys/{key}` 提交 `models` 即同时提交它们的配额(配额就是 scope 的一部分,
不存在会与模型列表脱节的第二份预算)。显式 `0` = 解除该模型的限制。
##### 配额拒绝 vs 容量拒绝:两种「拒绝」含义不同
配额在**入口**检查,不占用任何上游槽位,因此是廉价拒绝;容量不足则要走
`busyWait` 有界等待后才返回 503。实测(容量 8、0.6s/请求、100 并发):
| 情况 | 状态码 | 延迟 | 是否打到上游 |
|---|---|---|---|
| 配额用尽(`req_quota` 50 → 100 并发) | `429 rate_limit_exceeded` + `Retry-After` | **19–21ms** | 否 |
| 容量打满(同一拓扑,未限配额) | `503 upstream_error` | ~2.6s(`busyWait`) | 超出部分否 |
客户端因此可以区分「等窗口重置」与「等上游腾出容量」:前者只需等待,后者
通常该降并发或换源。
##### 配额的开销
配额桶按 (密钥, 模型, 整点小时) 分桶保留 40 天,实测(AMD 7840HS):
- 窗口查询按窗口长度而非保留总量扫描:1h 窗口 49ns、24h 窗口 55ns、
30d 窗口 3.9μs(此前全扫保留总量,960 桶时 5.9μs)。
- 记录一条请求:283ns、3 分配(与引入配额前相同,分配来自 ring buffer)。
- 内存:生产形态(7 密钥 × 8 模型 × 2 源 × 满 40 天 retention)约 **3.7MB**。
按源 pin 的 `source::model` 桶**惰性创建**——只有当某条配额真的 pin 了
某个源时才维护,否则每条记录多写一份桶,在 20 密钥 × 8 模型 × 3 源下会
多占 18MB。
代价:配置 pinned 配额之前的历史用量无法事后按源拆分,所以 **pinned 配额的
首个窗口可能少算**。
```bash
# 配额耗尽时客户端看到(点名了具体模型)
HTTP/1.1 429 Too Many Requests
Retry-After: 2100
{"error":{"type":"rate_limit_exceeded","message":"token quota exceeded for \"deepseek-v4-flash\" (5000000/5000000)"}}
```
### 模型路由
`/v1/chat/completions` 的 `model` 解析顺序:
1. `source/model` 或 `source:model` 前缀 → 指定源;
2. 精确匹配某个源的 `model`;
3. 配置 `default_model: AUTO` 时 → 走优先级页保存的 AUTO 链(见下);
4. 否则回落(返回错误)。
任何 OpenAI 客户端,只要 `model` 设为某个源的 `name/任意名`,即可锁定走该源;
设为 `AUTO`(或网关配了 `default_model: AUTO`)即自动按优先级链选源。
### AUTO 链(优先级页)
AUTO 调度**只**由优先级页保存的规则决定,源配置里的 `models[].priority` 数字不再
参与调度、也不再显示。**聊天与生图是两套独立的链**,在优先级页面通过「聊天 / 生图」
按钮切换编辑:
- **聊天链**(`runtime_file` 的 `auto` 字段):服务 `POST /v1/chat/completions`
的 `model: AUTO` 请求;只接受 `kind: chat` 模型,配置在链里的生图槽位会被忽略。
- **生图链**(`runtime_file` 的 `auto_image` 字段):服务 `POST /v1/images/generations`
的 `model: AUTO` 请求;只接受 `kind: image` 模型。未配置生图链时,AUTO 生图回落
到「所有含生图模型的源、按注册顺序」的旧行为,兼容旧配置。
每条规则 = 一个「槽位」:`{ model, source, tier, token_quota, period, hours }`。
- `tier` 表示优先级档位:同一档的模型并排、共享该优先级;档位从上到下递减。
- 同一模型可配置多个槽位(如 A 源低配 → B 源低配 → A 源高配 → B 源高配),按档位顺延。
- `token_quota` > 0 时该槽位在重置周期内用满即顺延到下一槽位;`period` 支持
`hour` / `week` / `month` / `nhour`(配合 `hours`),空 = 不限。
- 同 tier 内按「偏好分」排序,失败槽位冷却后自动跳过;冷却 / 配额耗尽 / hard
错误会顺延到下一 tier,全部失败时返回 503 并附上每档失败摘要。
### 冷却与半冷却探测(自愈调度)
槽位失败后进入指数退避冷却(5s → 10s → 20s …,上限 **5 分钟**)。冷却不是全时段
硬闸:
- **前半段完全静默**:刚失败的槽位不接任何流量,避免对故障上游持续施压。
- **后半段放行 1 个探测**:窗口过半后,同一 `(源, 模型)` 最多允许 **一个** 请求作为
探测通过。探测槽数 = `clamp(max_concurrent / 10, 1, 2)`——`max_concurrent: 10`
的源恰好放行 1 个探测请求。
- **探测排在最后**:同 tier 内探测候选永远排在健康候选之后,只有没有正常槽可用时
才承接真实流量,因此探测不会抢走可用容量。
- **探测即真实请求**:成功则立即清零失败计数与冷却,槽位当场回到正常轮询——
上游恢复/额度恢复后不必等满整个冷却窗口。失败则重开一个新窗口,下次探测顺延到
新窗口的中点,不会连续重试。
分类冷却策略:
| 情况 | 冷却 | 说明 |
|---|---|---|
| 传输错误 / 5xx | 5s→10s→20s…,上限 5min | 指数退避 |
| 401 / 403 凭据错误 | 10min | 换 key 后由下一次探测自动接回 |
| 429 限流 | 30s 固定 | 「太快」不等于「坏了」,不进指数阶梯 |
| 配额耗尽(`insufficient_quota` / `余额不足` 等) | 对齐配额窗口,上限 30min | **不进指数阶梯、不涨失败计数**,额度恢复即可用 |
WebUI「优先级」页的槽位健康标签会区分 `冷却` / `待探测` / `探测中`,悬停可看窗口
起点、探测放行时刻与完全恢复时刻。
### 源(Source)与适配器(Adapter)的关系
- **源** 是到某个上游的连接描述:`name`、`base_url`、`api_key`、模型列表与优先级。
- **适配器** 是协议转换逻辑(Lua 脚本):把统一 OpenAI 格式请求转成上游原生格式,
再把上游响应/流式分块转回统一格式。
- 一个适配器可被多个源复用(如 `openai.lua` 同时服务多个 OpenAI 兼容站点);
同一个源也可以切换不同适配器(改 `adapter` 字段即可)。
- 源决定“连谁、暴露哪些模型”,适配器决定“怎么对话”——二者在 `sources[]` 条目中
通过 `adapter` 字段关联。
加载流程(`internal/core` 的 `Core` 负责装配):
1. 启动时 `lua.NewVM(adapter_dir)` 加载全部适配器:内置适配器(编译期 embed)+
`adapter_dir` 下的同名覆盖文件;内置文件写在代码内,覆盖文件要求更高优先级。
2. `config.Load` 读取 `config.yaml`,`config.NewStore(runtime_file)` 读 WebUI 改动的
运行时源,二者按名称合并成完整源列表。
3. `rebuildRegistry` 为每个源创建 `provider.Provider`(持有目标适配器),并按源的
并发上限配置适配器 worker 池**上限**(不预分配,见下)。
4. 请求进来时 `Registry` 按 `model` 路由到 Provider,Provider 调适配器
`transform_request` → HTTP 发送 → `transform_response` / `transform_stream_chunk`。
WebUI 上的"新增/编辑源"、"上传 Lua 适配器"、"改 AUTO 优先级链"与"密钥模型范围"
都即时生效(写入运行时文件或 `adapter_dir` 后重新装配,无需重启);直接编辑
`config.yaml` / `adapter_dir` 下的文件则需要重启进程才会重新加载。
在 WebUI 删除源/适配器时:运行时(WebUI 创建)的源与 `.lua` 文件会真正移除;
`config.yaml` 中定义的基源无法改写配置文件,采用删除标记隐藏(重启后仍隐藏),
在 UI 里重新添加同名源即可恢复。
### 源模板(多 key 负载均衡)
多个上游 key 共享同一套配置(URL / 适配器 / 模型列表 / 并发 / rpm)时,逐一复制
整段 source 太冗余。**源模板**保存除 `name` 和 `api_key` 以外的全部源字段,
从模板一键创建多个仅 key 不同的源,天然支持多 key 负载均衡(每个展开源独立调度、
独立冷却、独立健康状态)。
- WebUI「源」页右上角「**模板管理**」按钮:列出全部模板,可编辑 / 删除 / 新建。
- 添加源弹窗右上角两个按钮:
- **从模板创建** → 选模板 → 表单自动填充(name + key 仍需你填);
- **存为模板** → 把当前表单非 key/name 字段存为新模板。
- 模板持久化在 `runtime_file` 的 `source_templates` 字段,与运行时源一样热管理,
改完即时生效。模板本身不含密钥,可以安全共享 / 纳管。
模板不会自己变成源——它只是「配方」。真正承担流量的是从模板创建出来的、带 key
的具体源。
### 内存占用(实测与调优)
内存不是一个定数:它随**配置的源数量**增长(每个源一个 `http.Transport` 与连接池、
一组健康状态),与运行时长无关。本机实测(Linux x86_64,12 核):
| 部署形态 | 启动 RSS | 稳态 RSS |
|---|---|---|
| 1 源 / 1 适配器(最小配置) | ~4 MB | ~10 MB |
| 1 源 + 29 MB 历史审计日志 | ~19 MB | ~20 MB |
| **16 源 / 13 适配器 / 59 模型(本机生产)** | ~28 MB | **~32–35 MB** |
> 历史参考:本项优化前同一生产配置为 **~105 MB**。降幅来自三处:审计日志的**原始记录**
> 不再常驻内存(约 25 MB;统计聚合仍扫全量,但扫完即释放)、Lua 状态池不再单调增长、
> 以及下面两个运行时开关。
内存构成(生产实例分段测量,`/proc/<pid>/smaps`):
| 区域 | RSS | 说明 |
|---|---|---|
| Go 堆 | ~14 MB | provider/registry/scheduler 结构 + 连接池缓冲 |
| 其他匹名(线程栈 / LuaJIT chunk / runtime) | ~12 MB | 与线程数、已加载适配器数相关 |
| 二进制 text+rodata | ~8 MB | 映射的可执行文件页(只读、可被内核回收) |
| Go runtime 预留 | ~4 MB | VSZ 上看到的 2 GB+ 是地址空间预留,不占物理内存 |
| 共享库 | ~3 MB | libc / libluajit / libm |
**两个推荐的部署开关**(只适用于环境变量,无需改代码):
```ini
# /etc/systemd/system/llmsproxy.service
Environment=GOGC=50
Environment=MALLOC_ARENA_MAX=2
```
- `MALLOC_ARENA_MAX=2`:LuaJIT 的分配走 cgo → glibc malloc,glibc 默认允许 `8×nproc`
个 per-thread arena,每个碰到 malloc 的 OS 线程会占用一个(各约 1 MB,且**不归还给系统**)。
实测从 8–15 个 arena(约 7–12 MB)降到 **0**。
- `GOGC=50`:把 Go 堆增长目标减半。**单独使用无效**(省下的堆会立即被更多 glibc
arena 吃掉,实测 20.3 → 21.5 MB 反而变大),必须与 `MALLOC_ARENA_MAX` 配合,
两者同时开启才降 **~19%**。网关是 I/O 密集型(本机 9 小时仅消耗 1min10s CPU),
多出的 GC 周期与它的空闲 CPU 相比可忽略。
> 桌面版(Electron)已在 `cmd/gui/main.js` 里默认为内嵌核心注入这两个开关;手动 export 同名
> 环境变量可覆盖。如果你用自己的 systemd unit / 容器,建议照上面加上。
可选:`GOMEMLIMIT=48MiB` 作软上限,实测再省 ~1 MB;代价是逐近上限时 GC 转激进,
温和场景不必开。
### thinking 预算
请求体带 `"thinking": {"enabled": false}` 时关闭推理;`"thinking": {"enabled": true,
"budget_tokens": 4096}` 指定预算。内核只定义这个**中立**语义字段,映射到具体协议由
适配器负责(Anthropic `thinking.budget_tokens`、Gemini `thinkingConfig.thinkingBudget`、
DeepSeek `thinking.type = "disabled"`)。内核不再向适配器泄漏 `disable_thinking` /
`extra_body` 这类私有字段。
### WebUI
内置管理界面(`GET /`),登录后可在浏览器完成:
- **状态页**:源在线状态(实时探测 + 悬停看错误)、模型/源/key 用量统计、请求记录,
支持按时间范围导出 CSV;点击模型可生成 pin 到该模型的连接配置。
- **对话页**:流式/非流式调试。
- **密钥页**:创建/编辑网关 key,为每个 key 配模型范围(模型 + 源 + token 配额 + 周期),
管理员管理全部 key,用户只看到自己的 key。每个模型砖块显示自己的配额徽标
(如 `1.0K 77×·1h`,未设配额显示 `∞`),点开可编辑该模型的 token 配额、
请求数配额与重置周期 —— 配额按模型独立生效,一个用满不影响同一 key 的其它模型。
「我的密钥」页对用户展示本 key 的模型范围与各自配额。
- **优先级页**:拖拽积木配置 AUTO 链档位。
- **源页**:在线增删改上游源(API key 等敏感字段加密落盘)。
- **适配器页**:上传 / 删除 Lua 适配器脚本。
改动写入 `runtime_file`(重启仍生效)。
## Lua 适配器协议
完整 API 见 **[Lua 适配器 API 文档](docs/lua-adapters.md)**([English](docs/lua-adapters-en.md))。
每个适配器是一个返回 table 的 Lua 脚本(`internal/lua/adapters/<name>.lua`),
加载时可被 `adapter_dir` 下的同名脚本覆盖——**改 Lua 脚本无需重编译**,
重启即生效;通过 WebUI 上传的适配器与在线编辑的源配置则即时生效。
```lua
return {
name = "mysrc",
version = "1.0.0",
endpoint = "/chat/completions", -- 上游路径(可被 source.endpoint 覆盖)
headers = { ["X-Static"] = "v" }, -- 静态默认请求头(build_headers 缺省时使用)
-- 请求转换:把 OpenAI 格式 req 转成上游原生格式,返回字符串
transform_request = function(raw_json) ... end,
-- 响应转换:把上游原生响应转成统一格式字符串
-- { content, reasoning_content, finish_reason, token_usage{...}, tool_calls[{...}] }
transform_response = function(raw_json) ... end,
-- 流式分块转换:把上游 SSE data 转成 { content, done, ... },返回 "" 则跳过
transform_stream_chunk = function(raw_chunk) ... end,
-- [可选] 动态请求头/签名钩子
-- meta = { url, method, body, api_key, timestamp, source={ name, meta={...} } }
build_headers = function(meta) return { ["X-App-Sign"] = sign } end,
-- [可选] 出向序列化(契约 v2):上游不是 OpenAI 协议时接管写给客户端的字节。
-- 不声明则沿用内核的 OpenAI 兜底;声明即完全取代,内核不再混合字段。
serialize_response = function(result) return body_str end,
serialize_open_preamble = function(result) return bytes end,
serialize_chunk = function(delta) return bytes end,
serialize_close = function(result) return bytes end,
}
```
内置信号辅助:`hmac_sha256_hex(key, data)`、`sha256_hex(data)`、`base64_encode(s)`、
`tohex(s)`、`json.encode/decode`、`log(level, msg)`。
### 内置适配器
`openai` `deepseek` `anthropic` `gemini` `github` `groq` `mistral` `ollama` `kimicode`
`qwen38` `sensenova` `trae`。
`anthropic`/`gemini`/`ollama` 适配器内置多模态转换(`image_url` → 各自上游格式);
推理开关由中立的 `thinking` 字段表达,`deepseek` 适配器把它映射为
`thinking.type = "disabled"`。
**kimicode** 是展示 `build_headers` 的样例:云端校验调用方 app,需要按
`meta.app_secret` 对时间戳+URL+请求体哈希做 HMAC 签名并附 `X-App-Sign` 等头。
配好 `sources[].meta.{app_id, app_secret, app_agent}` 即可。
## 桌面 GUI(Electron)
可选独立产物,面向内网桌面端用户:**内嵌 ModelRouter 核心**(Clash Verge 形态),
一键启停、系统托盘常驻、开机自启、静默启动,内容区一比一内嵌完整 WebUI
(状态/对话/密钥/优先级/源/适配器 六个页面,免登录)。服务器用户继续使用纯 Go 二进制。
### 两种安装包:Headless 与 Desktop
发布到 Release 的安装包分为两类,面向不同用户:
- **Headless(服务器版)**:纯核心二进制(无 Electron),配置驱动,适合服务器、
容器、systemd 等无人值守场景。发布包含 `deb` + `rpm`(含 systemd unit、内存调优
与示例配置)与 `tar.gz`(二进制 + `config.example.yaml` + `adapters/` + README),
Windows 为 `.zip`。
- **Desktop(桌面版)**:Electron GUI 安装包,内嵌核心,适合个人桌面日常使用。
### 选择 GUI 还是纯后端
| 场景 | 推荐 | 理由 |
| ------ | ------ | ------ |
| 服务器 / 内网网关 / 无人值守常驻 | **Headless**(单二进制) | 极轻量(二进制 ~8MB;RSS 随源数量,单源 ~10MB、16 源 ~32–35MB,见[内存占用](#内存占用实测与调优)),零依赖单进程,直接跑在 systemd/任意容器里,远程管理 |
| 个人桌面日常使用 / 多设备内网共享 | **Desktop**(GUI) | 免登录内嵌 WebUI、系统托盘一键启停、开机自启、静默后台,适合不懂命令行的使用者 |
| Windows 桌面 | **Desktop** | 纯后端在 Windows 上需自行注册服务,GUI 提供原生托盘/自启体验 |
| CI 一键出三平台安装包 | **Desktop 打包脚本** | `make gui-*` 系列出 deb/AppImage/NSIS,可进发行流水线 |
两者完全同源:GUI 内嵌的就是纯后端的同一份 `llmsproxy` 核心(LuaJIT 版),
配置/适配器格式完全一致,可随时互换。
### 构建与运行
```bash
cd cmd/gui
npm install
make gui # 或 npm run dev —— 开发运行(需先 make build 生成 bin/llmsproxy)
make gui-deb # 本机构建 deb(分享给其它 Linux 用户)
make gui-dist # 构建 deb + rpm + AppImage(linux,rpm 需系统 rpmbuild)
make core-dist # 构建核心 deb + rpm + tar.gz(服务器版,需 nfpm)
make gui-win # 构建 Windows nsis 安装包(本机 mingw + wine)
make gui-win-docker # 构建 Windows nsis 安装包(全 docker 自足,含 wine)
make gui-dist-dir # 仅解包产物(调试用,不产安装包)
```
产物在 `cmd/build/gui-dist/`:`ModelRouter-<ver>.AppImage`、
`modelrouter-gui_<ver>_amd64.deb`、`ModelRouter Setup <ver>.exe`(win);
核心包在 `cmd/build/dist/`:`llmsproxy_<ver>_amd64.deb`、`llmsproxy-<ver>.x86_64.rpm`。
rpm 需系统 `rpmbuild`。每次打包后都会跑 `packaging/verify-dist.sh` 产物体检
(大小下限门禁——1.3.0 曾因宿主 wine 损坏产出 264KB 的残缺 exe 却没有被拦住)。
发布时会把产物重命名为 `ModelRouter-Desktop-*`(GUI)与 `ModelRouter-Headless-*`(纯后端)
两类上传到 GitCode Release。
#### Windows 打包(docker 固化方案,推荐)
`make gui-win-docker` 一步完成:交叉编译核心 + NSIS 打包**全在 docker 内**,宿主机
**无需 mingw、无需 wine、无需 node**。镜像内置 wine(含 i386 运行时),修复了
此前宿主 wine 损坏(缺 syswow64 → `c0000135`)导致产出 264KB 残缺 exe 的问题。
- 构建镜像:`cmd/gui/docker/win-builder/Dockerfile`(`golang:1.25` + mingw-w64 +
node + wine32/wine64 + 预编译 LuaJIT windows 版 `lua51.dll` 与 import lib)。
LuaJIT 源码取自 gitcode 镜像(国内可达),github 兜底;Go 依赖走 `https://goproxy.cn`。
- 一键发布:`cmd/gui/scripts/dist-win-docker.sh` —— 核心编译 + electron-builder
NSIS 全部在容器内执行(`USE_SYSTEM_WINE=true` 走镜像内的 wine);electron-builder
的 ~400MB 工具链缓存挂到 docker volume,二次构建增量复用。
- 首次运行自动构建 `modelrouter/win-builder` 镜像(约 5–10 分钟);已存在则直接复用。
- 打包完成后自动跑 `packaging/verify-dist.sh` 门禁:exe 低于 5MB / deb·rpm·7z 低于
10MB 即失败——残缺安装包不可能再溜进 Release。
```bash
# 全 Windows 发布(核心重编 + NSIS):
make gui-win-docker
```
### 特性
- **内嵌核心**:自动拉起捆绑的 `llmsproxy`(LuaJIT 版),配置/密钥/adapter 存于
用户数据目录 `<userData>/profile/`,首次运行自动生成随机 admin key 写入 `keys`
(非 seed)并自注入,WebUI 免登录、不会提示"更换初始密钥"。端口可在设置中更改(默认 8787)。
- **系统托盘**:内核状态点、开机自启/静默启动开关、启停/重启内核、退出。
- **静默启动**:`--silent` 参数或设置项,启动仅驻托盘不弹窗;首次运行始终显示窗口。
- **开机自启**:Windows/macOS 用 `setLoginItemSettings`;Linux 写
`~/.config/autostart/modelrouter-gui.desktop`,静默开启时自动附加 `--silent`。
## 目录
```
cmd/llmsproxy # 入口(纯 Go 服务端)
cmd/gui # Electron 桌面 GUI(内嵌核心 + 托盘 + 打包)
internal/config # YAML 配置加载/校验
internal/lua # LuaJIT VM + worker 池 + AdapterCache + 内置适配器 (embed)
旧路径与旧环境变量在读取侧仍被接受,因此升级不会因为顺序问题立刻失效。
`deploy.sh` 在检测到旧 `/etc/llmsproxy` 而新配置不存在时会**中止**,避免部署出
一个「能启动但没有 key / 没有源」的空实例。
插件 SDK 同名调整:唯一公开入口是 `modelrouter`(`llmsproxy` 仅作兼容别名),
详见 [docs/plugins.md](docs/plugins.md)。
## 目录结构
```text
cmd/modelrouter # 入口(纯 Go 服务端)
cmd/gui # Electron 桌面壳(内嵌同一份核心)
internal/config # 配置解析、加密存储、运行时文件
internal/core # 调度、内核动作、状态与审计
internal/lua # LuaJIT VM + 弹性 worker 池 + 内置适配器 (embed)
internal/lua/plugins # 可选插件示例(不随二进制/发行包分发)
internal/provider # Provider(HTTP) + Registry(路由)
internal/gateway # OpenAI 兼容 HTTP 服务 + 鉴权 + SDK/流式 + WebUI
internal/gateway # OpenAI 兼容 HTTP 服务 + 鉴权 + 流式 + WebUI
internal/types # 统一格式 & OpenAI wire 类型
packaging/ # deb/rpm/tar 打包、systemd unit、迁移脚本
docs/ # 主题文档
```
## 开发协作
- **Git 分支工作流**:主分支 / 特性分支 / 发布分支的规范见
**[docs/git-workflow.md](docs/git-workflow.md)**([English](docs/git-workflow-en.md))。
主线:`main` 永可部署 → 特性合入 `main` → 切 `release/vX.Y.Z` 打 tag 发布 →
hotfix 提交发布分支并 cherry-pick 回 `main`。
- **Lua 适配器协议**:见 **[docs/lua-adapters.md](docs/lua-adapters.md)**
([English](docs/lua-adapters-en.md))。
## 测试
## 开发与测试
```bash
go test -tags luajit ./...
go test -tags luajit ./... # 全量
go test -tags luajit ./internal/lua # 单包(LuaJIT 相关改动必跑)
```
覆盖:配置校验、适配器加载/变换、签名钩子、Gateway 鉴权、SDK 结算、SSE 流式、
模型路由、多模态透传与 thinking 预算。
- 分支与发布纪律见 [docs/git-workflow.md](docs/git-workflow.md)。
- 插件放置在 `plugin_dir`,网关启动时**不会**创建、覆盖或删除任何插件文件;
升级桌面端也不会动它们。

View File

@ -2,625 +2,148 @@
> **中文**: [README.md](./README.md)
>
> **AI-assisted**: This project is developed with AI-assisted coding (code and docs are produced in collaboration with AI, reviewed by humans).
> **AI-assisted**: code and documentation were produced with AI assistance and
> reviewed by a human.
A lightweight, unified OpenAI-compatible LLM gateway for internal networks —
configure once, and let multiple services (agents, SDKs, bots) share many
upstreams (DeepSeek, Qijiar, OpenAI, Anthropic, Gemini, Groq, Mistral, Ollama,
KimiCode…) behind a single endpoint. Pick a specific model or use **AUTO** mode,
which routes to the best healthy upstream by configured priority.
An intranet gateway for people with several LLM subscriptions: configure once and
let multiple services share the upstream capacity. Pick a model explicitly, or let
`AUTO` choose by priority among the sources that are currently healthy.
> **Lightweight, no recompile to add sources**: the gateway is a single Go
> binary (8-12 MB, ~8 MB stripped; zero runtime dependencies). Adding or switching an upstream is
> just a `sources` entry in `config.yaml` (or via the WebUI) or a `.lua`
> adapter — **no Go changes, no recompile**. Changes submitted through the
> WebUI take effect immediately (hot reload); editing `config.yaml` or a `.lua`
> file under `adapter_dir` by hand requires a process restart.
The gateway itself is a single Go binary (~10 MB, no runtime dependencies). Adding
or switching an upstream is a `sources` entry in `config.yaml` (or the WebUI), or a
dropped-in `.lua` adapter — **no Go changes, no rebuild**. Changes submitted through
the WebUI take effect immediately; editing `config.yaml` or a `.lua` adapter file
directly requires a restart.
## Overview
## What it does
Exposes a standard `OpenAI Chat Completions` API (`/v1/chat/completions` +
`/v1/models`) to your internal network, translating between multiple upstream
protocols via **Lua adapters**. Supports one-shot calls, SSE streaming,
**AUTO model routing**, multimodal passthrough, and tool calls.
Extracted and independently evolved from the multi-source LLM adapter layer of
[HomeAgent](https://gitcode.com/JianFeeeee/HomeAgent)
(`internal/agent/api/provider.go` + `internal/lua/adapters/*`).
## Features
- **Multi-source**: any number of upstream sources in one process, routed by the
request's `model`.
- **AUTO mode**: with `default_model: AUTO`, scheduling follows the tiered AUTO
chain saved on the Priority page (see below).
- **Unified output**: every source speaks OpenAI format (incl.
`reasoning_content`, `tool_calls`, `usage`).
- **Image generation**: `POST /v1/images/generations`, routed to models with
`kind: image`.
- **Per-model quota**: each key gives every model its own token and request caps
plus a reset period (hour/week/month/custom N hours). One model running out
affects only that model — the key's other models keep working. Exhaustion
answers 429 + `Retry-After` naming the model, so a client resumes when the
window rolls over; admin keys are never capped.
- **Multimodal**: `content` arrays (`image_url` etc.) pass through losslessly;
Anthropic/Gemini/Ollama are translated automatically.
- **LuaJIT VM**: golua-binding LuaJIT; each adapter has its own VM + worker
pool for safe concurrency. Pools are **elastic**: the sum of `max_concurrent`
over an adapter's sources is a ceiling, not a preallocation — states are booted
on demand and reclaimed when demand drops, so an idle gateway holds close to
zero Lua states. The grow step follows the adapter's max concurrency
(`clamp(ceil(max/8), 1, 8)`, and never exceeds the number of queued callers);
the shrink step follows the live connection count
(`clamp(ceil(slack/(1+in_use)), 1, slack)`), so an idle pool collapses in one
round while a busy one gives up a single state at a time.
- **On-demand request logs**: dashboard totals are computed from the **full**
audit history (streamed once at startup and released — 29 MB / 221k lines in
~260 ms), while the raw records are never held in memory: the dashboard loads
one screen, scrolling pages the rest straight off disk, CSV export streams in
O(1) memory, and leaving the page releases everything.
- **Self-healing cooldown**: cooldown is capped at 5 minutes and, past the
window's midpoint, exactly one probe request is allowed through; a recovered
upstream (or a reset quota) returns to full rotation on that probe instead of
waiting out the window.
- **disable_thinking**: `disable_thinking: true` toggles reasoning per-request.
- **Lua adapter protocol**: each source mounts a `.lua` adapter with
`transform_request` / `transform_response` / `transform_stream_chunk` — all
protocol differences live in Lua, Go only schedules and proxies.
- **Signature / header hooks**: adapters may define `build_headers(meta)` to
inject/sign request headers before the HTTP call (e.g. KimiCode style
app-validation), with helpers like `hmac_sha256_hex`, `sha256_hex`,
`base64_encode`.
- **WebUI**: built-in management page to view/add/edit sources & models, edit
the AUTO chain, and manage per-key model scopes, persisted to a runtime file.
- **Encrypted secrets at rest**: upstream `api_key`, custom header values and
gateway keys are stored AES-256-GCM encrypted (`master.key` 0600 next to the
runtime file, or `LLMS_PROXY_MASTER_KEY`).
- **Live source probing**: the status page probes each source
(`GET {base}/models`, falling back to a minimal chat call) without touching
the scheduler's backoff state; errors are shown on hover.
- **Auth**: the gateway validates client keys via `gateway_keys`; independent
from each upstream's own key.
- **Streaming**: SSE `chat.completion.chunk` with a role-first chunk and
`[DONE]` terminator.
- **One OpenAI-compatible API**: `/v1/chat/completions`, `/v1/models`,
`/v1/images/generations`. Every source speaks the same shape outwards, including
`reasoning_content`, `tool_calls` and `usage`.
- **Many sources, one AUTO chain**: configure any number of upstreams; `AUTO`
follows the priority tiers saved in the WebUI, fails over between them, cools
down broken sources and probes them back to health at half-cool.
- **Protocol differences live in Lua**: each source binds a `.lua` adapter that
transforms requests, responses and stream chunks both ways, and can sign headers
via `build_headers(meta)`. Vendor knowledge stays out of the Go kernel.
- **Plugins are independent**: business extensions (channel login, billing, quota
probing) are external Lua plugins loaded from `plugin_dir`. They are not compiled
into the binary and not shipped in release archives, and they talk to the kernel
only through the `modelrouter` SDK.
- **Multi-tenant keys**: gateway keys are separate from upstream keys, and can be
scoped to a model set, an AUTO chain and a quota.
- **Encrypted at rest**: upstream keys, custom header values and gateway keys are
sealed with AES-256-GCM; `master.key` is 0600, or injected via
`MODELROUTER_MASTER_KEY`.
- **Observability**: live source probing, HTTP access log, config-change audit,
request and usage CSV export. Writes are kept off the request hot path.
## Quick start
```bash
GOMODCACHE=... GOPROXY=off go build -tags luajit -o llmsproxy ./cmd/llmsproxy
./llmsproxy -config config.yaml # first run generates a default config and prints a random admin key
# LuaJIT development libraries are required; -tags luajit is mandatory
GOMODCACHE=... GOPROXY=off go build -tags luajit -o modelrouter ./cmd/modelrouter
# First run generates a default config and prints a random admin key
./modelrouter -config config.yaml
```
> Depends on [golua](https://github.com/aarzilli/golua) (LuaJIT bindings).
> You **must** build with `-tags luajit`; otherwise the built-in superset
> gopher-lua path is used (behavior differs slightly).
> Depends on [golua](https://github.com/aarzilli/golua) (LuaJIT bindings). Without
> `-tags luajit` the build falls back to gopher-lua, which behaves slightly
> differently.
> **No config is shipped in the repo** (config files carry real keys). On
> first run the binary generates a default config at the `-config` path: a
> random admin key (printed to the startup log), loopback-only `127.0.0.1:8080`,
> and a keyless source ready to chat. Rotate the admin key in the
> WebUI after first login.
> **No config in the repo**: the repository ships no config file (they contain
> keys). The generated default binds `127.0.0.1:8080` only; rotate the admin key in
> the WebUI after the first login.
```bash
# no key -> 401
# No key -> 401
curl http://127.0.0.1:8080/v1/models
# one-shot ($KEY = the admin key printed at first startup)
# Non-streaming ($KEY = the admin key printed at first start)
curl -H "Authorization: Bearer $KEY" \
-d '{"model":"AUTO","messages":[{"role":"user","content":"hi"}]}' \
http://127.0.0.1:8080/v1/chat/completions
# streaming
# Streaming
curl -N -H "Authorization: Bearer $KEY" \
-d '{"model":"AUTO","stream":true,"messages":[{"role":"user","content":"hi"}]}' \
http://127.0.0.1:8080/v1/chat/completions
```
Any OpenAI SDK works: point `base_url` at the gateway, use any authorized
gateway key as the API key.
## Configuration
Generated on first run at the `-config` path (default `./config.yaml`). Core fields:
```yaml
listen: 127.0.0.1:8080 # bind address (keep internal/loopback)
gateway_keys: [sk-gw-<random>] # initial admin key seed (random per install), migrated to the store on first start
default_model: AUTO # when model is unroutable, follow the AUTO chain
adapter_dir: adapters # Lua adapter dir; created+seeded if missing, read-only otherwise
runtime_file: runtime.json # WebUI-edited sources/keys/AUTO chain persist here
sources:
- name: deepseek
base_url: https://api.deepseek.com
api_key: sk-... # or api_key_env: SOME_ENV to read from an env var
adapter: deepseek
max_concurrent: 8
models:
- id: deepseek-v4-flash
priority: 100 # YAML sources seed the AUTO chain on first start
kind: chat
- id: deepseek-v4-pro
priority: 60
kind: chat
# static headers (take precedence over adapter defaults)
headers: { X-Tenant: prod }
# passthrough metadata for the Lua build_headers hook
meta: { app_id: x, app_secret: y }
temperature: 0.7
max_tokens: 4096
timeout: 120s # request timeout, default 120s
```
Sensitive fields in the runtime file (`runtime_file`) are encrypted at rest:
- AES-256-GCM, stored as `enc:v1:<base64>`.
- Master key source: env var `LLMS_PROXY_MASTER_KEY` (64 hex chars), else a
`master.key` file next to the runtime file; if neither exists it is generated
on first start (mode 0600).
- Old plaintext files still load; the first UI save migrates the whole file to
ciphertext.
- Back up `master.key` with your config — losing it makes the secrets
undecryptable. Do not commit it.
### Gateway keys (multi-key)
Gateway auth uses a "multi-key + role + model scope" model. Keys are persisted
under the `keys` field of the runtime file (encrypted at rest):
- The `gateway_keys` config is only an **initial admin key seed** — it is
migrated into the runtime store on first start and no longer drives auth.
- **Important: after first start, replace the admin key via the WebUI Keys
page.** The seed key is written in plaintext in `config.yaml`, so keeping it
active is a security risk; create a new admin key, log in with it, then
delete the seed key.
- The WebUI **Keys page** creates/deletes keys. Each key has a role (`admin`
manages everything, `user` sees only its own key) and an optional **model
scope** (model + source + token quota + reset period). Each model brick shows
its own budget badge (e.g. `1.0K 77×·1h`, `∞` when uncapped); clicking it
edits that model's token quota, request quota and reset period. Quotas apply
per model, so one model running out never blocks the key's others. The
"My key" view shows a user its model scopes and their budgets.
- Clients authenticate with any authorized key's plaintext as
`Authorization: Bearer <key>`.
- Deleting a key removes it from the store immediately.
#### Per-key usage quota
**Quotas are per model.** Each key's `models[]` list gives every model its own
token budget and request budget. One model running out affects only that model
— the key's other models keep working.
```yaml
keys:
- key: sk-gw-<hex>
role: user
name: agent-alice
models:
- model: deepseek-v4-flash
token_quota: 1000000 # this key's token budget for this model
req_quota: 20000 # requests within the window
period: nhour # "" | hour | week | month | nhour
hours: 6 # n-hour only
- model: AUTO
token_quota: 5000000 # AUTO is a quota entry like any other
period: hour
- model: kimi-k3 # no quota listed = unlimited
```
- **There is deliberately no key-wide total.** A key-wide cap would make
"move A's budget to B" a global reallocation; per-model budgets keep each
model independently controllable, so it stays visible which model is
actually consuming the spend.
- `period`: empty = never resets (lifetime total); `hour` / `week` / `month` =
fixed windows; `nhour` + `hours` = a custom hour count. **A misspelled
period is rejected at write time** rather than silently becoming a
never-resetting quota.
- Quotas are isolated per key *and* per model within a key: one key exhausting
`m1` never draws on another key's budget, and never blocks the same key's
`m2`.
- Usage counts chat, streaming and image requests, and survives a restart by
replaying the audit log (40 days retained, covering the longest `month`
window).
- An exhausted quota returns **429 + `Retry-After`**
(`rate_limit_exceeded`) and the message names the model that ran out, so a
client resumes when the window rolls over; a model the key may not use stays
403. **Admin keys are never capped** (quotas on their scopes are not
enforced either), so a cap can never lock the operator out.
- Buckets are whole unix hours, so a window frees up at most an hour late
(deliberately freeing late rather than overspending).
- `PUT /api/keys/{key}` submits quotas by submitting `models` — the caps are
part of the scope, so there is no second budget that can drift out of sync
with the model list. An explicit `0` lifts that model's cap.
##### Quota rejection vs capacity rejection
A quota is checked at the entry point and occupies no upstream slot, so it is a
cheap rejection. Running out of capacity instead waits out the bounded
`busyWait` before answering 503. Measured (capacity 8, 0.6 s/request, 100
concurrent clients):
| Case | Status | Latency | Reaches upstream |
|---|---|---|---|
| Quota spent (`req_quota` 50, 100 concurrent) | `429 rate_limit_exceeded` + `Retry-After` | **19–21 ms** | no |
| Capacity exhausted (same topology, no quota) | `503 upstream_error` | ~2.6 s (`busyWait`) | overflow: no |
So a client can tell "wait for the window to roll over" apart from "wait for
upstream capacity" — the first only needs patience, the second usually means
back off concurrency or switch sources.
##### What the quota costs
Buckets are kept per (key, model, whole unix hour) for 40 days. Measured on an
AMD 7840HS:
- A window query scans the window, not the whole retention: 49 ns for 1 h,
55 ns for 24 h, 3.9 µs for 30 d.
- Recording one request: 283 ns, 3 allocations (unchanged from before this
feature; the allocations come from the record ring buffer).
- Memory: the production shape (7 keys x 8 models x 2 sources at full 40-day
retention) costs about **3.7 MB**. The `source::model` bucket used by a
source-pinned quota is created **lazily** — it is maintained only once some
quota actually pins that source, because writing it on every record would
cost an extra 18 MB at 20 keys x 8 models x 3 sources.
The trade-off: usage recorded before a pinned quota was configured cannot be
split by source afterwards, so **a pinned quota may under-count its first
window**.
```
HTTP/1.1 429 Too Many Requests
Retry-After: 2100
{"error":{"type":"rate_limit_exceeded","message":"token quota exceeded for \"deepseek-v4-flash\" (5000000/5000000)"}}
```
### Model routing
`/v1/chat/completions` `model` resolution order:
1. `source/model` or `source:model` prefix → pinned source;
2. exact match of a source's `model`;
3. with `default_model: AUTO` → follow the AUTO chain from the Priority page
(see below);
4. otherwise the request errors.
Any OpenAI client can pin to a source by setting `model` to its
`name/anything`; `AUTO` (or the gateway's `default_model: AUTO`) follows the
tiered AUTO chain.
### AUTO chain (Priority page)
AUTO scheduling is driven **only** by the rules saved on the Priority page. The
numeric `models[].priority` in source configs no longer participates in
scheduling and is no longer shown. **Chat and image generation are two
independent chains**, switched in the Priority page via the `Chat / Image` toggle:
- **Chat chain** (`auto` field): serves `POST /v1/chat/completions` with
`model: AUTO`; only `kind: chat` models are honored (image slots are skipped).
- **Image chain** (`auto_image` field): serves `POST /v1/images/generations` with
`model: AUTO`; only `kind: image` models are honored. With no image chain
configured, AUTO image generation falls back to legacy discovery (all sources
exposing an image model, in registry order) for backward compatibility.
Each rule is one "slot": `{ model, source, tier, token_quota, period, hours }`.
- `tier` is the priority tier: models in the same tier sit side by side and
share it; tiers run high → low.
- The same model may appear in several slots (e.g. A low → B low → A high →
B high) and is tried in tier order.
- `token_quota` > 0 means the slot is skipped once its tokens within the reset
window are exhausted; `period` supports `hour` / `week` / `month` / `nhour`
(with `hours`); empty = unlimited.
- Within a tier, slots run by preference score; cooling / quota-exhausted /
hard-failed slots fall through to the next tier, and total failure returns
503 with a per-tier summary.
### Sources vs. adapters
- A **source** describes a connection to an upstream: `name`, `base_url`,
`api_key`, model list and priorities.
- An **adapter** is the protocol translation logic (Lua script): converts the
unified OpenAI request to the upstream's native format and back.
- One adapter serves many sources (e.g. `openai.lua` for any OpenAI-compatible
site); one source can switch adapters via its `adapter` field.
- Sources decide *whom to talk to and which models to expose*; adapters decide
*how to talk*. They are linked by the `adapter` field inside each `sources[]`
entry.
Loading flow (assembled by `Core` in `internal/core`):
1. On startup `lua.NewVM(adapter_dir)` loads all adapters: built-in ones
(embedded at compile time) + same-name override files under `adapter_dir`.
2. `config.Load` reads `config.yaml`; `config.NewStore(runtime_file)` reads
WebUI-edited runtime sources; both are merged by name.
3. `rebuildRegistry` creates a `provider.Provider` per source (holding its
adapter), sizing the adapter's worker pool from the source's concurrency
limit.
4. Incoming requests are routed by `Registry` to a provider, which calls the
adapter's `transform_request` → HTTP call → `transform_response` /
`transform_stream_chunk`.
"Add/Edit source", "Upload adapter", "AUTO chain edits" and "per-key model
scopes" in the WebUI take effect immediately (written to the runtime file /
`adapter_dir`, then reassembled — no restart). Directly editing `config.yaml`
or files under `adapter_dir` requires a process restart.
Deleting a source/adapter in the WebUI truly removes runtime (UI-created)
sources and `.lua` files; base sources defined in `config.yaml` cannot rewrite
that file, so they are hidden via a deletion tombstone (still hidden after
restart) and can be restored by re-adding the same name in the UI.
### Source templates (multi-key balancing)
When several upstream keys share one config (URL / adapter / model list /
concurrency / rpm), copying the whole source block N times is wasteful. A
**source template** stores every source field except `name` and `api_key`, so you
create N key-only-different sources from one shared skeleton — natural multi-key
load balancing (each expanded source schedules, cools down, and reports health
independently).
- WebUI Sources page, top-right **Templates** button: list all templates with
edit / delete / create.
- Add-source dialog has two header buttons:
- **From template** → pick a template → form auto-fills (name + key still
yours to enter);
- **As template** → save the current form's non-key/name fields as a template.
- Templates persist in the runtime file's `source_templates` field alongside
runtime sources (hot-managed, apply immediately). Templates hold no secret
keys, so they're safe to share / version-control.
A template is just a recipe — it doesn't become a source by itself. Only the
key-bearing sources created from it carry real traffic.
### Memory footprint (measured and tuned)
Memory is not a constant: it scales with the **number of configured sources**
(each source owns an `http.Transport` with its connection pool plus per-model
health state), not with uptime. Measured on this machine (Linux x86_64, 12 cores):
| Deployment shape | Startup RSS | Settled RSS |
|---|---|---|
| 1 source / 1 adapter (minimal) | ~4 MB | ~10 MB |
| 1 source + a 29 MB audit history | ~19 MB | ~20 MB |
| **16 sources / 13 adapters / 59 models (this host)** | ~28 MB | **~32-35 MB** |
> For reference, the same production config used **~105 MB** before this round of
> work. The reduction comes from three places: raw audit records are no longer kept
> resident (~25 MB — the aggregates still scan the full history, but the scan
> releases as it goes), Lua state pools no longer grow monotonically, and the two
> runtime knobs below.
Breakdown of the production instance (per-region, from `/proc/<pid>/smaps`):
| Region | RSS | Notes |
|---|---|---|
| Go heap | ~14 MB | provider/registry/scheduler state + connection-pool buffers |
| Other anonymous (thread stacks / LuaJIT chunks / runtime) | ~12 MB | scales with thread count and loaded adapters |
| Binary text+rodata | ~8 MB | mapped executable pages (read-only, reclaimable by the kernel) |
| Go runtime reservations | ~4 MB | the 2 GB+ you see in VSZ is address space, not physical memory |
| Shared libraries | ~3 MB | libc / libluajit / libm |
**Two recommended deployment knobs** (environment only, no code change):
```ini
# /etc/systemd/system/llmsproxy.service
Environment=GOGC=50
Environment=MALLOC_ARENA_MAX=2
```
- `MALLOC_ARENA_MAX=2`: LuaJIT allocates through cgo into glibc malloc, and glibc
allows up to `8 x nproc` per-thread arenas. Every OS thread that touches malloc
claims one (~1 MB each) and **never returns it to the OS**. Measured: 8-15
arenas (7-12 MB) down to **0**.
- `GOGC=50`: halves the Go heap growth target. **It does nothing on its own** —
the heap it saves is immediately consumed by additional glibc arenas (measured
20.3 -> 21.5 MB, i.e. slightly worse) — so it must be paired with
`MALLOC_ARENA_MAX`. Together they cut settled RSS by **~19%**. The gateway is
I/O bound (1min10s of CPU per 9 hours here), so the extra GC cycles are free.
> The desktop build (Electron) already injects both when spawning the embedded
> core (`cmd/gui/main.js`); exporting either variable yourself overrides it. If you
> write your own systemd unit or container spec, add them there.
Optional: `GOMEMLIMIT=48MiB` as a soft ceiling saves roughly 1 MB more, at the
cost of GC turning aggressive as the limit approaches. Not worth it for a relaxed
deployment.
### disable_thinking
With `"disable_thinking": true` in the request body, the gateway passes it to
each adapter; the DeepSeek adapter maps it to `extra_body.thinking.type =
"disabled"`, other sources follow their own protocol.
### WebUI
Built-in admin page at `GET /`; after login you can:
- **Status**: source online state (live probe, error on hover), per
model/source/key usage stats, request records filtered by key, and CSV export
over a time range; click a model to pin the connection snippet to it.
- **Chat**: streaming / non-streaming debug.
- **Keys**: create/edit gateway keys, set per-key model scopes (model + source +
token quota + reset period); admins manage all keys, users see only their own.
- **Priority**: drag blocks to build the tiered AUTO chain.
- **Sources**: add/edit/delete upstream sources online (secrets stored
encrypted).
- **Adapters**: upload / delete Lua adapter scripts.
Changes persist to `runtime_file` (survive restarts).
## Lua adapter protocol
Full API: **[Lua adapter docs](docs/lua-adapters.md)**
([中文](docs/lua-adapters-en.md)).
Each adapter is a Lua script returning a table
(`internal/lua/adapters/<name>.lua`), overridable by a same-name file under
`adapter_dir` — **no recompile needed**, restart to take effect; uploads via
the WebUI take effect immediately.
```lua
return {
name = "mysrc",
version = "1.0.0",
endpoint = "/chat/completions", -- upstream path (source.endpoint overrides)
headers = { ["X-Static"] = "v" }, -- static default headers (fallback)
-- request: convert unified OpenAI req -> upstream native format, return string
transform_request = function(raw_json) ... end,
-- response: convert upstream response to unified format string
-- { content, reasoning_content, finish_reason, token_usage{...}, tool_calls[{...}] }
transform_response = function(raw_json) ... end,
-- streaming chunk: convert upstream SSE data to { content, done, ... }; "" skips
transform_stream_chunk = function(raw_chunk) ... end,
-- [optional] dynamic headers / signature hook
-- meta = { url, method, body, api_key, timestamp, source={ name, meta={...} } }
build_headers = function(meta) return { ["X-App-Sign"] = sign } end,
}
```
Shared helpers: `hmac_sha256_hex(key, data)`, `sha256_hex(data)`,
`base64_encode(s)`, `tohex(s)`, `json.encode/decode`, `log(level, msg)`.
### Built-in adapters
`openai` `deepseek` `anthropic` `gemini` `github` `groq` `mistral` `ollama`
`kimicode` `qwen38` `sensenova` `trae`.
`anthropic`/`gemini`/`ollama` include multimodal conversion
(`image_url` → their native format); with `disable_thinking` the `deepseek`
adapter sets `extra_body.thinking.type` to `disabled`.
**kimicode** demonstrates `build_headers`: the cloud validates the calling
app, so you HMAC-sign timestamp+URL+body with `meta.app_secret` and add
`X-App-Sign`-style headers. Configure `sources[].meta.{app_id, app_secret,
app_agent}`.
## Desktop GUI (Electron)
Optional standalone product for desktop users on your internal network: an
**embedded ModelRouter core** (Clash Verge-style): one-click start/stop, system
tray, autostart, silent launch, and a pixel-for-pixel embedded full WebUI
(Status/Chat/Keys/Priority/Sources/Adapters — no login needed). Server users
keep using the plain Go binary.
### Two installer kinds: Headless and Desktop
Release installers ship in two flavors for different audiences:
- **Headless (server)**: the plain core binary (no Electron), config-driven,
for servers / containers / systemd / unattended runs. Ships as `deb` + `rpm`
(systemd unit with memory tuning + example config) and `tar.gz` (binary +
`config.example.yaml` + `adapters/` + README), Windows as `.zip`.
- **Desktop (desktop app)**: the Electron GUI installer with an embedded core,
for personal daily desktop use.
### GUI vs. plain backend
| Scenario | Pick | Why |
| ---- | ---- | ---- |
| Server / intranet gateway / unattended long-running | **Headless** (single binary) | ~8 MB binary; RSS scales with the number of sources (~10 MB for one, ~32-35 MB for 16 — see [Memory footprint](#memory-footprint-measured-and-tuned)), zero-dep single process — drop it into systemd or any container, remote admin |
| Personal desktop daily use / multi-device intranet sharing | **Desktop** (GUI) | no-login embedded WebUI, tray one-click, autostart, silent background — for non-CLI users |
| Windows desktop | **Desktop** | plain backend needs manual service registration; GUI ships native tray/autostart |
| CI one-shot 3-platform installers | **Desktop packaging scripts** | `make gui-*` emits deb / AppImage / NSIS, drops straight into a release pipeline |
Both share the exact same `llmsproxy` core (LuaJIT build) — configs and
adapters are fully compatible, freely interchangeable.
### Build & run
Any OpenAI SDK works: point `base_url` at the gateway and use an authorized gateway
key as `api_key`.
## Documentation
The detailed documentation is currently written in Chinese.
| Topic | Document |
|---|---|
| Implementation and the full feature list | [docs/features.md](docs/features.md) |
| Configuration: `config.yaml`, model discovery, multiple keys | [docs/configuration.md](docs/configuration.md) |
| Routing: model routing, AUTO chain, cooldown, sources vs adapters, source templates | [docs/routing.md](docs/routing.md) |
| Adapters: Lua protocol and built-in adapters | [docs/adapters.md](docs/adapters.md) · [docs/lua-adapters-en.md](docs/lua-adapters-en.md) (English) |
| Plugins: SDK contract, hooks, UI extensions, HTTP endpoints | [docs/plugins.md](docs/plugins.md) |
| Desktop app: Headless / Desktop artifacts, build and run | [docs/gui.md](docs/gui.md) |
| Performance: measured memory use, tuning, thinking budget | [docs/performance.md](docs/performance.md) |
| WebUI usage | [docs/webui.md](docs/webui.md) |
| Development: layout, tests, branch and release discipline | [docs/development.md](docs/development.md) · [docs/git-workflow-en.md](docs/git-workflow-en.md) (English) |
| Deployment and migration | [docs/deployment.md](docs/deployment.md) |
## Renamed from llmsproxy
The product name has been ModelRouter for a while; the binary, config directory,
service name and environment variables are now `modelrouter` as well:
| Before | After |
|---|---|
| `llmsproxy` (binary) | `modelrouter` |
| `/etc/llmsproxy/` | `/etc/modelrouter/` |
| `llmsproxy.service` | `modelrouter.service` |
| `LLMSPROXY_*` | `MODELROUTER_*` |
Existing installs migrate with a dry-run-by-default script (copies, never moves,
and can roll back):
```bash
cd cmd/gui
npm install
make gui # or npm run dev — dev run (needs make build first for bin/llmsproxy)
make gui-deb # local deb (share with other Linux users)
make gui-dist # deb + rpm + AppImage (linux; rpm needs system rpmbuild)
make core-dist # core deb + rpm + tar.gz (server edition; needs nfpm)
make gui-win # Windows NSIS (host mingw + wine)
make gui-win-docker # Windows NSIS, fully dockerized (wine in image; no host mingw/node/wine)
sudo ./packaging/migrate-to-modelrouter.sh # preview
sudo ./packaging/migrate-to-modelrouter.sh --apply # apply
sudo ./packaging/migrate-to-modelrouter.sh --rollback --apply
```
Artifacts land in `cmd/build/gui-dist/`: `ModelRouter-<ver>.AppImage`,
`modelrouter-gui_<ver>_amd64.deb`, `ModelRouter Setup <ver>.exe` (win); core
packages land in `cmd/build/dist/`: `llmsproxy_<ver>_amd64.deb`,
`llmsproxy-<ver>.x86_64.rpm`. Both target families run
`packaging/verify-dist.sh` afterwards — a size floor that rejects degenerate
installers (1.3.0 shipped a 264 KB NSIS stub because the host wine was broken
and nothing caught it). At release time artifacts are renamed into the
`ModelRouter-Desktop-*` (GUI) and `ModelRouter-Headless-*` (plain backend)
families before upload to GitCode Releases.
Old paths and old environment variable names are still accepted on the read side,
so an upgrade cannot fail merely because of ordering. `deploy.sh` refuses to run
when it finds the old `/etc/llmsproxy` without a new config, because that would
deploy a gateway with no key and no sources.
#### Windows packaging (dockerized, recommended)
`make gui-win-docker` does cross-compile + NSIS **entirely in docker**; the host
needs **no mingw, no wine, no node**. Wine (with the i386 runtime) lives inside
the image, fixing the broken-host-wine failure that produced the 264 KB stub
installer.
- Image: `cmd/gui/docker/win-builder/Dockerfile` (`golang:1.25` + mingw-w64 +
node + wine32/wine64 + prebuilt LuaJIT for Windows: `lua51.dll` + import lib).
LuaJIT source comes from the gitcode mirror (reachable in CN) with github
fallback; Go modules go through `https://goproxy.cn`.
- One-shot release: `cmd/gui/scripts/dist-win-docker.sh` — core build + NSIS
packaging run inside the container (`USE_SYSTEM_WINE=true` uses the image's
wine); electron-builder's ~400 MB toolchain cache lives in a docker volume so
later builds reuse it.
- The `modelrouter/win-builder` image is auto-built on first run (~5-10 min)
and reused afterwards.
- After packaging, `packaging/verify-dist.sh` runs as a gate: an exe under
5 MB (or a deb/rpm/7z under 10 MB) fails the build — a broken installer can
never reach a Release again.
```bash
# Full Windows release (rebuild core + NSIS):
make gui-win-docker
```
### Features
- **Embedded core**: auto-spawns the bundled `llmsproxy` (LuaJIT build);
config/keys/adapters live in `<userData>/profile/`. First run generates a
random admin key written into `keys` (not seed) and self-injects it — WebUI
needs no login and never nags about changing the initial key. Port changeable
in settings (default 8787).
- **System tray**: core status dot, autostart/silent toggles, start/stop/restart
core, quit.
- **Silent launch**: `--silent` flag or setting — starts to the tray only;
first run always shows the window.
- **Autostart**: `setLoginItemSettings` on Windows/macOS; writes
`~/.config/autostart/modelrouter-gui.desktop` on Linux, appending `--silent`
when silent mode is on.
The plugin SDK follows the same naming: the only public entry point is
`modelrouter` (`llmsproxy` remains a compatibility alias). See
[docs/plugins.md](docs/plugins.md).
## Layout
```
cmd/llmsproxy # entry point
internal/config # YAML config loading/validation
internal/lua # LuaJIT VM + worker pool + AdapterCache + built-ins (embed)
```text
cmd/modelrouter # entry point (pure Go server)
cmd/gui # Electron desktop shell (embeds the same core)
internal/config # config parsing, sealed storage, runtime file
internal/core # scheduling, kernel actions, state and audit
internal/lua # LuaJIT VM + elastic worker pool + built-in adapters (embed)
internal/lua/plugins # optional plugin examples (not shipped in binary/archives)
internal/provider # Provider(HTTP) + Registry(routing)
internal/gateway # OpenAI-compatible HTTP + auth + SDK/streaming + WebUI
internal/types # unified format & OpenAI wire types
internal/gateway # OpenAI-compatible HTTP service + auth + streaming + WebUI
internal/types # unified format and OpenAI wire types
packaging/ # deb/rpm/tar packaging, systemd unit, migration script
docs/ # topic documentation
```
## Development & collaboration
- **Git branching workflow**: main / feature / release-branch conventions in
**[docs/git-workflow-en.md](docs/git-workflow-en.md)**
([中文](docs/git-workflow.md)). The line: `main` is always deployable →
features merge into `main` → cut `release/vX.Y.Z` and tag → release → hotfixes
land on the release branch and are cherry-picked back into `main`.
- **Lua adapter protocol**: see
**[docs/lua-adapters-en.md](docs/lua-adapters-en.md)**
([中文](docs/lua-adapters.md)).
## Tests
## Development and tests
```bash
go test -tags luajit ./...
go test -tags luajit ./... # everything
go test -tags luajit ./internal/lua # one package (required for LuaJIT changes)
```
Covers: config validation, adapter load/transform, signature hooks, gateway
auth, SDK round trip, SSE streaming, model routing, multimodal passthrough and
disable_thinking.
- Branch and release discipline: [docs/git-workflow-en.md](docs/git-workflow-en.md).
- Plugins live in `plugin_dir`. The gateway never creates, overwrites or deletes
plugin files, and a desktop upgrade never touches them either.

View File

@ -1,5 +1,5 @@
// ModelRouter Desktop GUI — main process
// Clash-Verge-style: bundles and manages an embedded llmsproxy core instance
// Desktop shell: bundles and manages an embedded modelrouter core instance
// in a per-user profile dir (userData), auto-starts it, keeps a tray icon,
// supports launch-at-login. Server-side users keep using the pure Go binary.
const {
@ -50,11 +50,27 @@ const CONFIG_FILE = path.join(PROFILE_DIR, "config.yaml");
const PLUGINS_DIR = path.join(PROFILE_DIR, "plugins");
// Packaged: extraResource at process.resourcesPath/bin (outside asar, spawnable).
// Dev: cmd/gui/bin, populated by prepare-core.js.
const CORE_EXE = path.join(
// The embedded core is named after the product. Installs produced before the
// llmsproxy -> modelrouter rename still carry the old file name inside their
// profile, so resolution falls back to it instead of declaring the core missing.
const CORE_BIN_DIR = path.join(
app.isPackaged ? process.resourcesPath : __dirname,
"bin",
);
const CORE_EXE = path.join(
CORE_BIN_DIR,
process.platform === "win32" ? "modelrouter.exe" : "modelrouter",
);
const LEGACY_CORE_EXE = path.join(
CORE_BIN_DIR,
process.platform === "win32" ? "llmsproxy.exe" : "llmsproxy",
);
function resolveCoreExe() {
if (fs.existsSync(CORE_EXE)) return CORE_EXE;
if (fs.existsSync(LEGACY_CORE_EXE)) return LEGACY_CORE_EXE;
return CORE_EXE;
}
// Packaged window icon: must be a REAL file on disk (extraResource), not an
// asar path — Electron/GTK pulls BrowserWindow icons from the filesystem and
// crashes (SIGSEGV) when given an asar virtual path on Linux.
@ -163,7 +179,7 @@ function unsealViaCore() {
let key = "";
try {
const out = execFileSync(
CORE_EXE,
resolveCoreExe(),
["-config", CONFIG_FILE, "-show-secrets"],
{
encoding: "utf-8",
@ -215,7 +231,7 @@ function coreStarted() {
function writeDefaultEmbeddedConfig() {
// Generate a self-contained embedded profile with a fresh random admin key.
// No upstream sources are preconfigured — the desktop user adds them via
// the WebUI's sources page (like Clash Verge's empty-by-default profile).
// the WebUI's sources page (empty by default, like the WebUI profile).
// If a config already exists we do NOT overwrite it (keep user's edits).
if (fs.existsSync(CONFIG_FILE)) return;
const key = "sk-gw-" + crypto.randomBytes(16).toString("hex");
@ -272,8 +288,9 @@ function startCore() {
console.log("core already running");
return;
}
if (!fs.existsSync(CORE_EXE)) {
console.error("embedded core binary missing:", CORE_EXE);
const coreExe = resolveCoreExe();
if (!fs.existsSync(coreExe)) {
console.error("embedded core binary missing:", coreExe);
return;
}
// ensure profile dir + config + local adapters exist
@ -281,9 +298,8 @@ function startCore() {
if (!fs.existsSync(CONFIG_FILE)) writeDefaultEmbeddedConfig();
pluginSync.ensurePluginDirConfigured(CONFIG_FILE, PLUGINS_DIR, log);
fs.mkdirSync(PLUGINS_DIR, { recursive: true });
pluginSync.syncBundledPluginsOnUpgrade(PROFILE_DIR, PLUGINS_DIR, app.getVersion(), log);
coreProc = spawn(CORE_EXE, ["-config", CONFIG_FILE], {
coreProc = spawn(coreExe, ["-config", CONFIG_FILE], {
cwd: PROFILE_DIR,
env: Object.assign({}, process.env, {
LLMS_PROXY_GUI: "1",
@ -686,7 +702,7 @@ function createWindow() {
});
}
// Silent start: `--silent` CLI flag or settings.silentStart keeps the app
// in the tray without showing the window (Clash-Verge style). First run
// in the tray without showing the window (desktop-shell behaviour). First run
// always shows the window so the user sees the dashboard.
mainWindow.once("ready-to-show", () => {
const firstRun = !fs.existsSync(
@ -810,8 +826,8 @@ ipcMain.handle("core:state", () => ({
port: settings.port || DEFAULT_PORT,
profileDir: PROFILE_DIR,
configFile: CONFIG_FILE,
coreExe: CORE_EXE,
coreExists: fs.existsSync(CORE_EXE),
coreExe: resolveCoreExe(),
coreExists: fs.existsSync(resolveCoreExe()),
}));
ipcMain.handle("core:start", () => {
app.isQuitting = false;
@ -883,7 +899,7 @@ if (gotLock) {
createWindow();
createTray();
// start embedded core if not already running (another instance of llmsproxy)
// start embedded core if not already running (another instance of modelrouter)
setTimeout(async () => {
const p = await probeEmbedded();
if (!p.ok) {

View File

@ -1,12 +1,12 @@
{
"name": "modelrouter-gui",
"version": "2.0.44",
"version": "3.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "modelrouter-gui",
"version": "2.0.44",
"version": "3.0.0",
"license": "Proprietary",
"devDependencies": {
"electron": "^33.0.0",

View File

@ -1,6 +1,6 @@
{
"name": "modelrouter-gui",
"version": "2.0.44",
"version": "3.0.0",
"description": "ModelRouter Desktop — embedded ModelRouter core with tray",
"author": "ModelRouter",
"main": "main.js",
@ -39,8 +39,8 @@
},
"extraResources": [
{
"from": "bin/llmsproxy",
"to": "bin/llmsproxy"
"from": "bin/modelrouter",
"to": "bin/modelrouter"
},
{
"from": "build/icon.png",
@ -66,8 +66,8 @@
"icon": "build/icon.ico",
"extraResources": [
{
"from": "bin/llmsproxy.exe",
"to": "bin/llmsproxy.exe"
"from": "bin/modelrouter.exe",
"to": "bin/modelrouter.exe"
},
{
"from": "bin/lua51.dll",

View File

@ -1,18 +1,11 @@
// Plugin-tree maintenance for the embedded core (no Electron imports, so this
// Plugin-tree maintenance for the desktop profile (no Electron imports, so this
// is unit-testable with plain node).
//
// Two problems this solves, both invisible from the gateway side:
// 1. The core deliberately does NOT default `plugin_dir` (an operator opts
// in). A desktop profile without it runs with ZERO plugins — the channels
// plugin (account login, credential vault, auto check-in) never appears.
// 2. The core seeds bundled plugins only when the file is ABSENT, so a
// desktop upgrade would keep the user's old channels.lua forever and
// silently skip every channel fix. On version change the shell must drop
// the built-in files so the new binary reseeds them.
// The desktop may configure an operator-selected plugin directory, but it never
// owns the files in that directory. In particular, upgrades must not seed,
// replace, rename, or delete plugins. Plugin installation is an explicit
// operator action (or an external package manager action).
const fs = require("fs");
const path = require("path");
const BUILTIN_PLUGINS = ["billing.lua", "channels.lua", "quota.lua", "usage.lua"];
// ensurePluginDirConfigured back-fills `plugin_dir` into an existing config.
// Appending a top-level YAML key is safe for the shapes this file can have
@ -25,7 +18,7 @@ function ensurePluginDirConfigured(configFile, pluginsDir, log) {
fs.writeFileSync(
configFile,
cur.replace(/\s*$/, "") +
"\n# added by ModelRouter Desktop: enables the plugin tree (channels etc.)\n" +
"\n# added by ModelRouter Desktop: enables the operator plugin directory\n" +
"plugin_dir: " + JSON.stringify(pluginsDir) + "\n",
"utf-8",
);
@ -37,46 +30,4 @@ function ensurePluginDirConfigured(configFile, pluginsDir, log) {
}
}
// syncBundledPluginsOnUpgrade drops built-in plugin files when the app version
// changed, so the core reseeds them from the binary it shipped with. Returns
// { changed, dropped:[...], kept: "prev-<ver>" }.
function syncBundledPluginsOnUpgrade(profileDir, pluginsDir, version, log) {
const stamp = path.join(profileDir, ".plugin-seed-version");
let have = "";
try {
have = fs.readFileSync(stamp, "utf-8").trim();
} catch (e) {
/* first run */
}
if (have === version) return { changed: false, dropped: [] };
const dropped = [];
for (const name of BUILTIN_PLUGINS) {
const f = path.join(pluginsDir, name);
if (!fs.existsSync(f)) continue;
try {
if (have) fs.renameSync(f, f + ".prev-" + have);
else fs.unlinkSync(f);
dropped.push(name);
} catch (e) {
if (log) log("[gui] drop builtin plugin " + name + " failed: " + e.message);
}
}
try {
fs.writeFileSync(stamp, version + "\n", "utf-8");
} catch (e) {
if (log) log("[gui] write plugin seed stamp failed: " + e.message);
}
if (dropped.length && log) {
log(
"[gui] plugins refreshed for " + version + " (" + dropped.length +
" builtin file(s) reseeded by core)",
);
}
return { changed: true, dropped, kept: have ? "prev-" + have : "" };
}
module.exports = {
BUILTIN_PLUGINS,
ensurePluginDirConfigured,
syncBundledPluginsOnUpgrade,
};
module.exports = { ensurePluginDirConfigured };

View File

@ -1,26 +1,22 @@
// Unit tests for the desktop plugin-tree maintenance (plain node, no Electron).
// Run: node --test cmd/gui/plugin-sync.test.js
// Unit tests for desktop plugin-directory configuration (plain node).
const test = require("node:test");
const assert = require("node:assert");
const fs = require("fs");
const os = require("os");
const path = require("path");
const { BUILTIN_PLUGINS, ensurePluginDirConfigured, syncBundledPluginsOnUpgrade } = require("./plugin-sync");
const { ensurePluginDirConfigured } = require("./plugin-sync");
const tmp = () => fs.mkdtempSync(path.join(os.tmpdir(), "plugin-sync-"));
test("back-fills plugin_dir into an existing profile without it", () => {
const dir = tmp();
const cfg = path.join(dir, "config.yaml");
fs.writeFileSync(cfg, 'listen: 127.0.0.1:8787\ndefault_model: AUTO\nkeys:\n - key: k\n role: admin\n');
fs.writeFileSync(cfg, "listen: 127.0.0.1:8787\ndefault_model: AUTO\nkeys:\n - key: k\n role: admin\n");
const plugins = path.join(dir, "plugins");
assert.equal(ensurePluginDirConfigured(cfg, plugins, null), "added");
const out = fs.readFileSync(cfg, "utf-8");
assert.match(out, /^plugin_dir: "/m);
assert.ok(out.includes(plugins.replace(/\\/g, "\\\\")) || out.includes(plugins), "path written");
// the rest of the profile must survive untouched
assert.match(out, /^listen: 127\.0\.0\.1:8787$/m);
assert.match(out, /^ - key: k$/m);
assert.match(out, /^listen: 127.0.0.1:8787$/m);
});
test("leaves a profile that already sets plugin_dir alone", () => {
@ -39,42 +35,14 @@ test("missing config is not created by the back-fill", () => {
assert.equal(fs.existsSync(cfg), false);
});
test("upgrade drops builtin plugins (keeping a backup) but not user plugins", () => {
test("desktop upgrade never modifies operator plugin files", () => {
const dir = tmp();
const plugins = path.join(dir, "plugins");
fs.mkdirSync(plugins, { recursive: true });
for (const f of BUILTIN_PLUGINS) fs.writeFileSync(path.join(plugins, f), "OLD");
fs.writeFileSync(path.join(plugins, "channels.lua"), "OPERATOR FILE");
fs.writeFileSync(path.join(plugins, "my-own.lua"), "MINE");
fs.writeFileSync(path.join(dir, ".plugin-seed-version"), "1.0.0\n");
const r = syncBundledPluginsOnUpgrade(dir, plugins, "2.0.42", null);
assert.equal(r.changed, true);
assert.deepEqual(r.dropped.sort(), [...BUILTIN_PLUGINS].sort());
for (const f of BUILTIN_PLUGINS) {
assert.equal(fs.existsSync(path.join(plugins, f)), false, f + " dropped");
assert.equal(fs.readFileSync(path.join(plugins, f + ".prev-1.0.0"), "utf-8"), "OLD");
}
// There is deliberately no version stamp and no upgrade synchronizer.
assert.equal(fs.readFileSync(path.join(plugins, "channels.lua"), "utf-8"), "OPERATOR FILE");
assert.equal(fs.readFileSync(path.join(plugins, "my-own.lua"), "utf-8"), "MINE");
assert.equal(fs.readFileSync(path.join(dir, ".plugin-seed-version"), "utf-8").trim(), "2.0.42");
});
test("same version is a no-op (nothing dropped)", () => {
const dir = tmp();
const plugins = path.join(dir, "plugins");
fs.mkdirSync(plugins, { recursive: true });
fs.writeFileSync(path.join(plugins, "channels.lua"), "CURRENT");
fs.writeFileSync(path.join(dir, ".plugin-seed-version"), "2.0.42\n");
const r = syncBundledPluginsOnUpgrade(dir, plugins, "2.0.42", null);
assert.equal(r.changed, false);
assert.equal(fs.readFileSync(path.join(plugins, "channels.lua"), "utf-8"), "CURRENT");
});
test("first run on a fresh profile just stamps the version", () => {
const dir = tmp();
const plugins = path.join(dir, "plugins");
fs.mkdirSync(plugins, { recursive: true });
const r = syncBundledPluginsOnUpgrade(dir, plugins, "2.0.42", null);
assert.equal(r.changed, true);
assert.deepEqual(r.dropped, []);
assert.equal(fs.existsSync(path.join(dir, ".plugin-seed-version")), true);
assert.equal(fs.readdirSync(dir).includes(".plugin-seed-version"), false);
});

View File

@ -1,5 +1,5 @@
// ModelRouter Desktop GUI — renderer
// Clash-Verge-style: the content area is a full-screen iframe loading the
// Desktop shell: the content area is a full-screen iframe loading the
// real WebUI 1:1 (status / chat / keys / priority / sources / adapters).
// The shell only adds the titlebar, settings overlay and tray integration.
const state = {

View File

@ -83,7 +83,7 @@
</div>
</div>
<!-- side rail: autostart / silent toggles (Clash-Verge style) -->
<!-- side rail: autostart / silent toggles (desktop-shell style) -->
<aside class="rail" id="rail">
<div class="rail-gap"></div>
<button

View File

@ -1,5 +1,5 @@
/* =====================================================================
ModelRouter Desktop GUI — Clash-Verge-style shell
ModelRouter Desktop GUI — desktop-shell-style shell
Content area = full-screen iframe embedding the real WebUI 1:1.
Palette mirrors the WebUI (sakura/frost/glassmorphism).
===================================================================== */

View File

@ -1,5 +1,5 @@
#!/usr/bin/env node
// Build the embedded llmsproxy core(s) for the GUI package.
// Build the embedded modelrouter core(s) for the GUI package.
// node prepare-core.js -> linux (native, luajit) + win (cross, best effort)
// node prepare-core.js linux -> linux only
// node prepare-core.js win -> win only
@ -12,21 +12,21 @@ const binDir = path.join(guiDir, "bin");
const projectRoot = path.join(guiDir, "..", "..");
function buildLinux() {
const out = path.join(binDir, "llmsproxy");
console.log("[prepare-core] building llmsproxy (linux/luajit)...");
const out = path.join(binDir, "modelrouter");
console.log("[prepare-core] building modelrouter (linux/luajit)...");
fs.mkdirSync(binDir, { recursive: true });
execFileSync(
"go",
["build", "-tags", "luajit", "-trimpath", "-o", out, "./cmd/llmsproxy"],
["build", "-tags", "luajit", "-trimpath", "-o", out, "./cmd/modelrouter"],
{ cwd: projectRoot, stdio: "inherit" },
);
return out;
}
function buildWin() {
const out = path.join(binDir, "llmsproxy.exe");
const out = path.join(binDir, "modelrouter.exe");
console.log(
"[prepare-core] cross-building llmsproxy.exe (windows/amd64, luajit)...",
"[prepare-core] cross-building modelrouter.exe (windows/amd64, luajit)...",
);
fs.mkdirSync(binDir, { recursive: true });
const cc = process.env.MINGW_CC || "x86_64-w64-mingw32-gcc";
@ -43,7 +43,7 @@ function buildWin() {
}
execFileSync(
"go",
["build", "-tags", "luajit", "-trimpath", "-o", out, "./cmd/llmsproxy"],
["build", "-tags", "luajit", "-trimpath", "-o", out, "./cmd/modelrouter"],
{
cwd: projectRoot,
stdio: "inherit",

View File

@ -1,4 +1,4 @@
// Command llmsproxy is a standalone gateway that exposes multiple upstream LLM
// Command modelrouter is a standalone gateway that exposes multiple upstream LLM
// sources behind a unified OpenAI-compatible HTTP API. Protocol differences are
// handled by Lua adapters (per source), optionally signing outgoing requests.
// It supports explicit model selection or AUTO routing by priority, image
@ -16,9 +16,9 @@ import (
"syscall"
"time"
"llmsproxy/internal/config"
"llmsproxy/internal/core"
"llmsproxy/internal/gateway"
"modelrouter/internal/config"
"modelrouter/internal/core"
"modelrouter/internal/gateway"
)
func main() {
@ -33,12 +33,12 @@ func main() {
// error here instead of being silently created.
if *checkOnly {
if _, err := os.Stat(*cfgPath); err != nil {
log.Fatalf("[llmsproxy] check: %v", err)
log.Fatalf("[modelrouter] check: %v", err)
}
if _, err := config.Load(*cfgPath); err != nil {
log.Fatalf("[llmsproxy] check: %v", err)
log.Fatalf("[modelrouter] check: %v", err)
}
log.Printf("[llmsproxy] check: %s is valid", *cfgPath)
log.Printf("[modelrouter] check: %s is valid", *cfgPath)
return
}
@ -48,34 +48,34 @@ func main() {
// program prints a credential, and this path never logs to a file.
if *showSecrets {
if _, err := os.Stat(*cfgPath); err != nil {
log.Fatalf("[llmsproxy] show-secrets: %v", err)
log.Fatalf("[modelrouter] show-secrets: %v", err)
}
if err := config.PrintSecrets(*cfgPath); err != nil {
log.Fatalf("[llmsproxy] show-secrets: %v", err)
log.Fatalf("[modelrouter] show-secrets: %v", err)
}
return
}
created, err := config.EnsureDefault(*cfgPath)
if err != nil {
log.Fatalf("[llmsproxy] config: %v", err)
log.Fatalf("[modelrouter] config: %v", err)
}
c, err := core.New(*cfgPath)
if err != nil {
log.Fatalf("[llmsproxy] core: %v", err)
log.Fatalf("[modelrouter] core: %v", err)
}
defer c.Close()
if created {
for _, k := range c.GatewayKeys() {
log.Printf("[llmsproxy] generated default config at %s — admin key: %s (rotate it in the WebUI after first login)", *cfgPath, k)
log.Printf("[modelrouter] generated default config at %s — admin key: %s (rotate it in the WebUI after first login)", *cfgPath, k)
}
}
gw, err := gateway.New(c)
if err != nil {
log.Fatalf("[llmsproxy] gateway: %v", err)
log.Fatalf("[modelrouter] gateway: %v", err)
}
// Ops hygiene: surface the two most common footguns instead of silently
@ -88,16 +88,16 @@ func main() {
// (e.g. after rotating the starter key away) it reported "every request is
// rejected" while seven working keys, one of them admin, were in service.
if keys := c.ListKeys(); len(keys) == 0 {
log.Printf("[llmsproxy] WARNING: no gateway key configured — every request will be rejected. Add one in the WebUI or set gateway_keys")
log.Printf("[modelrouter] WARNING: no gateway key configured — every request will be rejected. Add one in the WebUI or set gateway_keys")
} else {
for _, k := range keys {
if k.Key == "sk-gw-local-0001" || k.Key == "sk-local-0001" {
log.Printf("[llmsproxy] WARNING: gateway key %q looks like the starter/example key — rotate it before exposing the gateway", k.Key)
log.Printf("[modelrouter] WARNING: gateway key %q looks like the starter/example key — rotate it before exposing the gateway", k.Key)
}
}
}
if l := c.Listen(); strings.HasPrefix(l, "0.0.0.0:") || strings.HasPrefix(l, "::") {
log.Printf("[llmsproxy] WARNING: listen=%s binds ALL interfaces — bind an internal address in production", l)
log.Printf("[modelrouter] WARNING: listen=%s binds ALL interfaces — bind an internal address in production", l)
}
srv := &http.Server{
@ -108,22 +108,22 @@ func main() {
go func() {
cert, key := c.TLS()
if cert != "" && key != "" {
log.Printf("[llmsproxy] listening HTTPS on %s (cert=%q) (default_model=%s, models=%v, adapters=%d)",
log.Printf("[modelrouter] listening HTTPS on %s (cert=%q) (default_model=%s, models=%v, adapters=%d)",
c.Listen(), cert, c.DefaultModel(), c.Registry().ModelList(), len(c.ListAdapters()))
if err := srv.ListenAndServeTLS(cert, key); err != nil && err != http.ErrServerClosed {
log.Fatalf("[llmsproxy] server: %v", err)
log.Fatalf("[modelrouter] server: %v", err)
}
return
}
log.Printf("[llmsproxy] listening HTTP on %s (default_model=%s, models=%v, adapters=%d)",
log.Printf("[modelrouter] listening HTTP on %s (default_model=%s, models=%v, adapters=%d)",
c.Listen(), c.DefaultModel(), c.Registry().ModelList(), len(c.ListAdapters()))
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
log.Fatalf("[llmsproxy] server: %v", err)
log.Fatalf("[modelrouter] server: %v", err)
}
}()
stop := make(chan os.Signal, 1)
signal.Notify(stop, syscall.SIGINT, syscall.SIGTERM)
<-stop
log.Printf("[llmsproxy] shutting down")
log.Printf("[modelrouter] shutting down")
}

View File

@ -1,10 +1,10 @@
#!/usr/bin/env bash
# ============================================================
# llmsproxy 原子化滚动部署脚本
# modelrouter 原子化滚动部署脚本
#
# 策略:
# 1. /tmp (tmpfs) 构建,/usr/local/bin (ext4) 部署
# 2. 构建后先 copy 到目标文件系统(.llmsproxy.tmp),
# 2. 构建后先 copy 到目标文件系统(.modelrouter.tmp),
# 再 rename(2) 做原子替换(同一文件系统内 mv 是原子的)
# 3. runtime.json 不替换(保留运行时密钥、审计状态)
# 4. systemctl restart → 服务中断仅在 Go 进程重启瞬间 (<500ms)
@ -28,20 +28,21 @@
set -euo pipefail
# ---------- 配置 ----------
readonly TARGET_BIN="/usr/local/bin/llmsproxy"
readonly TARGET_CONFIG="/etc/llmsproxy/config.yaml"
readonly TARGET_ADAPTERS="/etc/llmsproxy/adapters"
readonly TARGET_PLUGINS="/etc/llmsproxy/plugins"
readonly REPO_DIR="/home/program/llmsproxy"
readonly TARGET_BIN="/usr/local/bin/modelrouter"
readonly TARGET_CONFIG="/etc/modelrouter/config.yaml"
readonly TARGET_ADAPTERS="/etc/modelrouter/adapters"
readonly TARGET_PLUGINS="/etc/modelrouter/plugins"
# 仓库所在目录不等于产品名:这里保持真实路径,除非用 MODELROUTER_REPO_DIR 覆盖。
readonly REPO_DIR="${MODELROUTER_REPO_DIR:-/home/program/llmsproxy}"
readonly HEALTH_URL="http://127.0.0.1:8081/v1/models"
readonly BIN_NAME="llmsproxy"
readonly BIN_NAME="modelrouter"
# 构建目录。默认落在仓库下的构建工作区,而不是 /tmp:本机 /tmp 是 8.8G
# tmpfs,实测被 Go 构建缓存占满后会让 go test/build 在链接阶段直接失败。
# 可用 LLMSPROXY_BUILD_DIR 覆盖;同一目录也被测试用作 GOTMPDIR。
# 可用 MODELROUTER_BUILD_DIR 覆盖;同一目录也被测试用作 GOTMPDIR。
readonly BUILD_SUBDIR=".build-work"
if [[ -n "${LLMSPROXY_BUILD_DIR:-}" ]]; then
readonly TMP_DIR="${LLMSPROXY_BUILD_DIR}"
if [[ -n "${MODELROUTER_BUILD_DIR:-}" ]]; then
readonly TMP_DIR="${MODELROUTER_BUILD_DIR}"
else
readonly TMP_DIR="${REPO_DIR}/${BUILD_SUBDIR}"
fi
@ -59,7 +60,7 @@ usage() {
cat <<'EOF'
用法: ./deploy.sh [--config <file>]
--config <file> 投放这份配置到 /etc/llmsproxy/config.yaml(原子替换)。
--config <file> 投放这份配置到 /etc/modelrouter/config.yaml(原子替换)。
省略时不改动线上配置,只部署二进制与适配器。
无论是否指定,重启前都会用新二进制 -check 预检线上配置。
@ -103,6 +104,17 @@ precheck() {
command -v systemctl &>/dev/null || fail "未找到 systemctl"
[[ $(id -u) -eq 0 ]] || fail "需要 root 权限"
# 改名保护:本脚本现在部署到 /etc/modelrouter。若机器上仍是改名前的
# /etc/llmsproxy 且新路径尚未建立,直接继续会创建一份**空配置**并让新
# 服务在没有 key/源的情况下起来 —— 看起来成功,实际网关拒掉所有请求。
if [[ -d /etc/llmsproxy && ! -e "$TARGET_CONFIG" ]]; then
fail "检测到改名前的部署 /etc/llmsproxy,但 $TARGET_CONFIG 不存在。
先迁移(dry-run 默认,确认后加 --apply):
sudo ./packaging/migrate-to-modelrouter.sh
sudo ./packaging/migrate-to-modelrouter.sh --apply
如需全新部署,先把旧目录移开:mv /etc/llmsproxy /etc/llmsproxy.bak"
fi
if [[ -n "$NEW_CONFIG_SRC" ]]; then
[[ -f "$NEW_CONFIG_SRC" ]] || fail "待部署配置不存在: $NEW_CONFIG_SRC"
# 同一文件比对:--config 指向线上文件本身时没有"投放"动作可做,
@ -153,7 +165,7 @@ build() {
-trimpath \
-ldflags="-s -w" \
-o "$TMP_DIR/bin/$BIN_NAME" \
./cmd/llmsproxy
./cmd/modelrouter
NEW_BIN="$TMP_DIR/bin/$BIN_NAME"
NEW_SIZE=$(stat -c%s "$NEW_BIN")
@ -254,7 +266,7 @@ verify_config() {
# ---------- 密钥可解封校验 ----------
# config.yaml 里的凭据以 enc:v1: 密文落盘,解密钥匙是 master.key(或
# LLMS_PROXY_MASTER_KEY)。一旦钥匙丢失/错位,配置在**启动时**才会报错,
# MODELROUTER_MASTER_KEY)。一旦钥匙丢失/错位,配置在**启动时**才会报错,
# 那时服务已经处于「二进制已替换 + 配置已替换」的半死状态,只能靠回滚。
# 所以在替换任何东西之前先验:能不能用现有的钥匙解开现有的密文。
#
@ -279,21 +291,22 @@ verify_master_key() {
[[ -n "$runtime_file" ]] || runtime_file="$(dirname "$TARGET_CONFIG")/runtime.json"
key_file="$(dirname "$runtime_file")/master.key"
if [[ -n "${LLMS_PROXY_MASTER_KEY:-}" ]]; then
log " 钥匙来源: LLMS_PROXY_MASTER_KEY(环境变量)"
MASTER_KEY_ENV="${MODELROUTER_MASTER_KEY:-${LLMS_PROXY_MASTER_KEY:-}}"
if [[ -n "$MASTER_KEY_ENV" ]]; then
log " 钥匙来源: MODELROUTER_MASTER_KEY(环境变量)"
# 走真实解密路径验证,而不是只检查格式
if out=$("$TARGET_BIN" -show-secrets -config "$TARGET_CONFIG" 2>&1); then
log "OK 凭据可解封($(echo "$out" | grep -cE '^(source|key) ') 项)"
return 0
fi
err "凭据无法用 LLMS_PROXY_MASTER_KEY 解封,未替换任何文件:"
err "凭据无法用 MODELROUTER_MASTER_KEY 解封,未替换任何文件:"
printf '%s\n' "$out" >&2
return 1
fi
if [[ ! -f "$key_file" ]]; then
err "配置含 $sealed 处密文,但钥匙文件不存在: $key_file"
err " 没有它,部署后服务将无法启动。恢复 master.key 或设置 LLMS_PROXY_MASTER_KEY 后重试。"
err " 没有它,部署后服务将无法启动。恢复 master.key 或设置 MODELROUTER_MASTER_KEY 后重试。"
return 1
fi
log " 钥匙文件: $key_file ($(stat -c '%A' "$key_file" 2>/dev/null || echo '?'),$(wc -c < "$key_file") 字节)"
@ -312,7 +325,7 @@ verify_master_key() {
# ---------- 备份目录保留策略 ----------
# 每次部署都新建一个 <prefix>.bak.<时间戳> 目录,而回滚只用到最近一次
# ($BACKUP_BIN / $BACKUP_CONFIG 都是单文件覆盖)。多出来的目录从没人清理,
# 实测线上已积累 71 个,/etc/llmsproxy 因此涨到 122M。
# 实测线上已积累 71 个,/etc/modelrouter 因此涨到 122M。
#
# 保留最近 KEEP_BACKUPS 份足够回滚,同时给目录数设上限——否则一次误配置
# 就可能在几秒内造出成百上千个目录。
@ -327,7 +340,7 @@ KEEP_BACKUPS=5
prune_backups() {
local prefix="$1"
local base="/etc/llmsproxy"
local base="/etc/modelrouter"
[[ -d "$base" ]] || return 0
# 防误用:前缀必须是已知的两个(避免有人传空串删掉整个目录)
case "$prefix" in
@ -393,7 +406,7 @@ sync_adapters() {
[[ -d "$SRC_ADAPTERS" ]] || fail "内嵌适配器目录不存在: $SRC_ADAPTERS"
# 备份旧的适配器文件(含任何运行时覆盖版本,如 opencode.lua)
BACKUP_DIR="/etc/llmsproxy/adapters.bak.$(date +%Y%m%d%H%M%S)"
BACKUP_DIR="/etc/modelrouter/adapters.bak.$(date +%Y%m%d%H%M%S)"
if [[ -d "$TARGET_ADAPTERS" ]]; then
mkdir -p "$BACKUP_DIR"
cp -rf "$TARGET_ADAPTERS/"*.lua "$BACKUP_DIR/" 2>/dev/null || true
@ -425,7 +438,7 @@ sync_adapters() {
#
# 早期实现是「只补缺,已存在一律不动」,看着安全,实际造成一个功能性缺口:
# 内置插件本身的**修复无法下发**。v2.0.3/v2.0.4/v2.0.5 三个渠道修复就这样
# 全部卡在仓库里 —— 二进制换了、/etc/llmsproxy/plugins/channels.lua 还是旧版,
# 全部卡在仓库里 —— 二进制换了、/etc/modelrouter/plugins/channels.lua 还是旧版,
# 生产行为毫无变化。内置源码的正确性由仓库保证,部署必须与二进制内容对齐。
#
# 「只补缺」原本想保护的是「运维改过 / 运行时新增的插件」,但那些都是**非内置
@ -439,7 +452,7 @@ sync_plugins() {
mkdir -p "$TARGET_PLUGINS"
# 备份将被覆盖/新增的内置插件(非内置名字不进入循环,天然不动)
BACKUP_DIR="/etc/llmsproxy/plugins.bak.$(date +%Y%m%d%H%M%S)"
BACKUP_DIR="/etc/modelrouter/plugins.bak.$(date +%Y%m%d%H%M%S)"
mkdir -p "$BACKUP_DIR"
local added=0 updated=0 same=0
for src in "$SRC_PLUGINS"/*.lua; do
@ -481,8 +494,8 @@ sync_plugins() {
# ---------- 重启服务 ----------
restart_service() {
log "重启服务(中断 <500ms)"
systemctl restart llmsproxy.service || fail "systemd restart 失败"
log "OK llmsproxy.service restarted"
systemctl restart modelrouter.service || fail "systemd restart 失败"
log "OK modelrouter.service restarted"
}
# ---------- 健康检查 ----------
@ -505,7 +518,7 @@ healthcheck() {
(( attempt++ ))
done
warn "健康检查未通过,请检查服务状态: systemctl status llmsproxy.service"
warn "健康检查未通过,请检查服务状态: systemctl status modelrouter.service"
return 1
}
@ -541,7 +554,7 @@ rollback() {
if (( restored_any == 0 )); then
err "无任何回滚点,无法自动恢复"
err "请手动检查: systemctl status llmsproxy.service && journalctl -u llmsproxy -n 50"
err "请手动检查: systemctl status modelrouter.service && journalctl -u modelrouter -n 50"
return 1
fi
@ -555,9 +568,9 @@ rollback() {
fi
log "重启服务(旧版本)"
systemctl restart llmsproxy.service || {
systemctl restart modelrouter.service || {
err "重启失败,服务可能完全宕机"
err "请手动检查: systemctl status llmsproxy.service"
err "请手动检查: systemctl status modelrouter.service"
return 1
}
@ -592,7 +605,7 @@ print_summary() {
# 于是坏配置在服务仍然健康时就被拦下,根本不会触发重启。
main() {
parse_args "$@"
log "═══ llmsproxy 原子部署开始 ═══"
log "═══ modelrouter 原子部署开始 ═══"
precheck
# 密钥校验放在 build/替换之前:它的失败后果最重(配置里的密文永久解不开),
# 必须在动任何文件之前就挡住。此阶段用的是**当前线上二进制**,它已经

60
docs/adapters.md Normal file
View File

@ -0,0 +1,60 @@
# 适配器协议
> 本文档由 README 拆分而来:Lua 适配器协议与内置适配器清单。
> 编写适配器的完整参考见 [lua-adapters.md](lua-adapters.md)。
> 返回 [README](../README.md)。
## Lua 适配器协议
完整 API 见 **[Lua 适配器 API 文档](docs/lua-adapters.md)**([English](docs/lua-adapters-en.md))。
每个适配器是一个返回 table 的 Lua 脚本(`internal/lua/adapters/<name>.lua`),
加载时可被 `adapter_dir` 下的同名脚本覆盖——**改 Lua 脚本无需重编译**,
重启即生效;通过 WebUI 上传的适配器与在线编辑的源配置则即时生效。
```lua
return {
name = "mysrc",
version = "1.0.0",
endpoint = "/chat/completions", -- 上游路径(可被 source.endpoint 覆盖)
headers = { ["X-Static"] = "v" }, -- 静态默认请求头(build_headers 缺省时使用)
-- 请求转换:把 OpenAI 格式 req 转成上游原生格式,返回字符串
transform_request = function(raw_json) ... end,
-- 响应转换:把上游原生响应转成统一格式字符串
-- { content, reasoning_content, finish_reason, token_usage{...}, tool_calls[{...}] }
transform_response = function(raw_json) ... end,
-- 流式分块转换:把上游 SSE data 转成 { content, done, ... },返回 "" 则跳过
transform_stream_chunk = function(raw_chunk) ... end,
-- [可选] 动态请求头/签名钩子
-- meta = { url, method, body, api_key, timestamp, source={ name, meta={...} } }
build_headers = function(meta) return { ["X-App-Sign"] = sign } end,
-- [可选] 出向序列化(契约 v2):上游不是 OpenAI 协议时接管写给客户端的字节。
-- 不声明则沿用内核的 OpenAI 兜底;声明即完全取代,内核不再混合字段。
serialize_response = function(result) return body_str end,
serialize_open_preamble = function(result) return bytes end,
serialize_chunk = function(delta) return bytes end,
serialize_close = function(result) return bytes end,
}
```
内置信号辅助:`hmac_sha256_hex(key, data)`、`sha256_hex(data)`、`base64_encode(s)`、
`tohex(s)`、`json.encode/decode`、`log(level, msg)`。
### 内置适配器
`openai` `deepseek` `anthropic` `gemini` `github` `groq` `mistral` `ollama` `kimicode`
`qwen38` `sensenova` `trae`。
`anthropic`/`gemini`/`ollama` 适配器内置多模态转换(`image_url` → 各自上游格式);
推理开关由中立的 `thinking` 字段表达,`deepseek` 适配器把它映射为
`thinking.type = "disabled"`。
**kimicode** 是展示 `build_headers` 的样例:云端校验调用方 app,需要按
`meta.app_secret` 对时间戳+URL+请求体哈希做 HMAC 签名并附 `X-App-Sign` 等头。
配好 `sources[].meta.{app_id, app_secret, app_agent}` 即可。

View File

@ -389,10 +389,10 @@ flow 类型,不是"登录成功"。
channels 插件用 `storeDir()` 决定存储位置,它有多级兜底:
```
env LLMSPROXY_PLUGIN_DIR → settings.store_dir → 读 /etc/llmsproxy/config.yaml 的 plugin_dir → 硬编码
env MODELROUTER_PLUGIN_DIR → settings.store_dir → 读 /etc/llmsproxy/config.yaml 的 plugin_dir → 硬编码
```
测试台没设 `LLMSPROXY_PLUGIN_DIR`,**后两级直接命中生产**:
测试台没设 `MODELROUTER_PLUGIN_DIR`,**后两级直接命中生产**:
```
测试插件目录 = /tmp/TestXXX/002/plugins
@ -404,7 +404,7 @@ storeDir 结果 = /etc/llmsproxy/plugins ← 探针实测输出
`.channels-store.json` 从 1730 覆盖成 1986 字节、账号池清空(inode 都换了)。
只有 `internal/lua` 复现,`internal/core` 不复现 —— 因为 core 的测试台
(`kernelTestCore`)设了 `LLMSPROXY_PLUGIN_DIR`。**这是隔离做得不一致**的代价。
(`kernelTestCore`)设了 `MODELROUTER_PLUGIN_DIR`。**这是隔离做得不一致**的代价。
同类前科:曾把 26 个测试账号写进生产 store。两次同一类错误。

183
docs/configuration.md Normal file
View File

@ -0,0 +1,183 @@
# 配置
> 本文档由 README 拆分而来:网关配置、模型自动探测、多密钥。
> 返回 [README](../README.md)。
## 配置文件
首次运行在 `-config` 路径自动生成默认配置(默认 `./config.yaml`),核心字段:
```yaml
listen: 127.0.0.1:8080 # 网关监听地址(建议绑内网/回环)
gateway_keys: [sk-gw-<随机>] # 初始 admin 密钥(每次安装随机生成,仅首启写入运行时存储用)
default_model: AUTO # model 无法路由时自动按优先级链选源
adapter_dir: adapters # Lua 适配目录;目录不存在时首启创建并 seed 内置,存在则只读
runtime_file: runtime.json # WebUI 编辑的源/密钥/AUTO 链持久化到此文件
sources:
- name: deepseek
base_url: https://api.deepseek.com
api_key: sk-... # 也可用 api_key_env: SOME_ENV 引用环境变量(不落盘明文)
adapter: deepseek
max_concurrent: 8
models:
- id: deepseek-v4-flash
priority: 100 # YAML 源首次启动会 seed 进 AUTO 链的初值
kind: chat
- id: deepseek-v4-pro
priority: 60
kind: chat
# 静态请求头(比适配器默认优先)
headers: { X-Tenant: prod }
# 传给 Lua build_headers 的透传元数据
meta: { app_id: x, app_secret: y }
temperature: 0.7
max_tokens: 4096
timeout: 120s # 请求超时,默认 120s
```
## 自动探测模型列表
不想手抄模型 id 时,只填 URL 和 key,让网关去扫:
```bash
# 只探测,不写任何东西
curl -X POST http://127.0.0.1:8080/api/sources/discover \
-H "Authorization: Bearer sk-gw-…" -H "Content-Type: application/json" \
-d '{"base_url":"https://api.vendor.com/v1","api_key":"sk-…"}'
# → {"ok":true,"base_url":"…","endpoint":"","models":["gpt-x",…],
# "image_models":["gpt-image-2"],"models_path":"/models","latency_ms":1387,…}
# 探测后直接落库
curl -X POST http://127.0.0.1:8080/api/sources/discover/apply \
-H "Authorization: Bearer sk-gw-…" -H "Content-Type: application/json" \
-d '{"base_url":"https://api.vendor.com/v1","api_key":"sk-…","name":"vendor"}'
```
WebUI 源编辑框里也有「自动探测模型」按钮:填好 URL 和 key 点一下,列表自动
填好,仍需点保存才生效。
几点值得知道:
- **URL 形态不用猜。** 厂商对「`/v1` 放 base 还是放 path」没有共识,探测会
依次尝试若干候选并返回答对的那个,同时给出归一化后的 `base_url` +
`endpoint`(`endpoint` 平时留空,因为 openai adapter 自带 `/chat/completions`)。
- **`image_endpoint` 是另一套规则。** 网关的图像默认路径是**绝对**的
`/v1/images/generations`,而 chat 默认是相对的。所以 `base_url` 带 `/v1` 时
必须显式给 `image_endpoint: /images/generations`,否则图像请求 404。
探测会替你填好。
- **图像模型只看上游自己声明的 modality**,绝不从模型名猜。`gpt-image-2`
这类 id 也出现在同时提供对话服务的目录上,猜错会让 AUTO 链选中一个只支持
图像生成的模型,其后每一层都被它拖垮。
- **不会逐模型探活。** 34 个模型的目录跑一轮要花钱且耗时数分钟,`/models`
是足够便宜的信号;模型是否真能应答留给正常请求去发现。
- **对已存在的源,必须显式 `replace_models: true`** 才会覆盖模型列表。重新
探测顺手抹掉为 AUTO 链手工调过的 priority,是最难排查的一类数据丢失。
- **`api_key` 留空表示沿用已存值**,不会把凭据清空。
`config.yaml`、`runtime.json`、`master.key`、`adapters/` 均不提交版本库(见
`.gitignore`),请随备份带走——`master.key` 丢失后运行时密文无法解密。
运行文件生成在 `-config` 所在目录:`adapter_dir` / `runtime_file` 自动指向
配置文件旁边的 `adapters/` 与 `runtime.json`,从任意工作目录启动都可用。
运行时文件(`runtime_file`)中的敏感字段自动加密:
- 加密算法 AES-256-GCM,格式 `enc:v1:<base64>`。
- 主密钥来源:环境变量 `MODELROUTER_MASTER_KEY`(64 位 hex);否则读取
`runtime_file` 同目录的 `master.key`;都不存在时首次启动自动生成 `master.key`(0600)。
- 升级时旧明文文件自动兼容:首次运行正常读取,任何 UI 保存操作触发全文件加密迁移。
- 注意:**master.key 丢失后密文无法解密**,请随配置一起备份;切勿提交到版本库。
## 网关密钥(多密钥)
网关鉴权采用「多密钥 + 角色 + 模型范围」架构,密钥持久化在 `runtime_file` 的
`keys` 字段(加密存储):
- `gateway_keys` 配置只是**初始 admin 密钥种子**:首次启动迁移为运行时 admin
key,之后不再参与鉴权管理。
- **重要:首次启动后请在 WebUI「密钥」页更换管理员密钥**——初始密钥明文写在
`config.yaml` 里(每次安装随机生成),继续使用存在被盗风险;用新密钥登录后删除初始密钥。
- WebUI **密钥页**可创建/删除密钥;每个密钥可指定 `admin`(管理全部)或
`user`(仅看自己的 key)角色,并配置**模型范围**(模型 + 源 + token 配额 +
重置周期)。
- 客户端用任意一个已授权的密钥明文作为 Bearer(`Authorization: Bearer <key>`)。
- 删除密钥即从运行时存储移除,立即失效。
### 密钥用量配额
**配额按模型单独设置**:每个密钥的 `models[]` 里,每一条模型范围各自带一份
token 配额与请求数配额。一个模型用满只影响该模型,同一密钥的其它模型照常工作。
```yaml
keys:
- key: sk-gw-<hex>
role: user
name: agent-alice
models:
- model: deepseek-v4-flash
token_quota: 1000000 # 本周期内该 key 用这个模型的 token 预算
req_quota: 20000 # 本周期内的请求次数
period: nhour # "" | hour | week | month | nhour
hours: 6 # 仅 nhour
- model: AUTO
token_quota: 5000000 # AUTO 也是一条独立配额
period: day # ← 这种写法会被拒绝(词表只有 hour/week/month/nhour)
- model: kimi-k3 # 未列配额 = 无限
```
- **没有「整钥总配额」**:这是刻意的设计。整钥总额会让「把 A 模型的额度挪给
B 模型」变成一次全局重分配;按模型独立计费则每个模型各自可控,运维可以
看出哪个模型吃掉了预算。
- `period` 词表:空 = 永不过期(累计总量),`hour` / `week` / `month` = 固定窗口,
`nhour` + `hours` = 自定义小时数。**拼错的周期在写入时就被拒**,不会静默变成
永不过期。
- 配额严格按密钥隔离,且**同一密钥内按模型隔离**:A 密钥用满 `m1` 不会消耗
B 密钥的额度,同一密钥的 `m2` 也不受影响。
- 配额统计含聊天、流式、生图,跨重启从审计日志回放(保留 40 天,覆盖最长的
month 窗口)。
- 配额耗尽返回 **429 + `Retry-After`**(`rate_limit_exceeded`),消息里点名是哪个
模型用满了,客户端可等窗口重置后自动恢复;模型越权才是 403。
**admin 密钥永不受配额限制**(其 scope 上的配额也不强制),避免把管理员
锁在门外。
- 窗口用量按整点小时分桶统计,实际释放比配置窗口最多晚 1 小时(配额宁可晚释放
也不超发)。
- `PUT /api/keys/{key}` 提交 `models` 即同时提交它们的配额(配额就是 scope 的一部分,
不存在会与模型列表脱节的第二份预算)。显式 `0` = 解除该模型的限制。
#### 配额拒绝 vs 容量拒绝:两种「拒绝」含义不同
配额在**入口**检查,不占用任何上游槽位,因此是廉价拒绝;容量不足则要走
`busyWait` 有界等待后才返回 503。实测(容量 8、0.6s/请求、100 并发):
| 情况 | 状态码 | 延迟 | 是否打到上游 |
|---|---|---|---|
| 配额用尽(`req_quota` 50 → 100 并发) | `429 rate_limit_exceeded` + `Retry-After` | **19–21ms** | 否 |
| 容量打满(同一拓扑,未限配额) | `503 upstream_error` | ~2.6s(`busyWait`) | 超出部分否 |
客户端因此可以区分「等窗口重置」与「等上游腾出容量」:前者只需等待,后者
通常该降并发或换源。
#### 配额的开销
配额桶按 (密钥, 模型, 整点小时) 分桶保留 40 天,实测(AMD 7840HS):
- 窗口查询按窗口长度而非保留总量扫描:1h 窗口 49ns、24h 窗口 55ns、
30d 窗口 3.9μs(此前全扫保留总量,960 桶时 5.9μs)。
- 记录一条请求:283ns、3 分配(与引入配额前相同,分配来自 ring buffer)。
- 内存:生产形态(7 密钥 × 8 模型 × 2 源 × 满 40 天 retention)约 **3.7MB**。
按源 pin 的 `source::model` 桶**惰性创建**——只有当某条配额真的 pin 了
某个源时才维护,否则每条记录多写一份桶,在 20 密钥 × 8 模型 × 3 源下会
多占 18MB。
代价:配置 pinned 配额之前的历史用量无法事后按源拆分,所以 **pinned 配额的
首个窗口可能少算**。
```bash
# 配额耗尽时客户端看到(点名了具体模型)
HTTP/1.1 429 Too Many Requests
Retry-After: 2100
{"error":{"type":"rate_limit_exceeded","message":"token quota exceeded for \"deepseek-v4-flash\" (5000000/5000000)"}}
```

137
docs/deployment.md Normal file
View File

@ -0,0 +1,137 @@
# 部署与迁移
> 本文档面向运维:单机 systemd 部署、发行包安装、目录与权限、从 `llmsproxy`
> 改名后的迁移与回滚。
> 返回 [README](../README.md)。
## 1. 单机部署(推荐)
仓库根目录的 `deploy.sh` 负责:构建 → 预检配置 → 备份 → 原子替换二进制 →
重启服务 → 后置校验。它**不**修改源码,也**不**假设全新机器。
```bash
sudo ./deploy.sh # 默认读取仓库内 config.yaml
sudo ./deploy.sh --config other.yaml # 投放并部署另一份配置
```
部署目标(改名后):
| 项 | 路径 |
|---|---|
| 二进制 | `/usr/local/bin/modelrouter` |
| 配置 | `/etc/modelrouter/config.yaml` |
| 适配器 | `/etc/modelrouter/adapters/` |
| 插件(可选) | `/etc/modelrouter/plugins/`(需在配置里设 `plugin_dir`) |
| 主密钥 | `/etc/modelrouter/master.key`(0600) |
| systemd unit | `/etc/systemd/system/modelrouter.service` |
`deploy.sh` 会在以下情况**中止**而不是"看起来成功":
- 部署后二进制起不来、或新配置 `-check` 不通过;
- 凭据无法解封(`master.key` 丢失 / 与配置不匹配);
- 检测到改名前的 `/etc/llmsproxy`,但新配置尚未建立(否则会部署出一个
没有 key、没有源的空实例,所有请求都被拒)。
## 2. 发行包安装
三个产物由 `packaging/core-dist.sh` 生成到 `cmd/build/dist/`:
```text
modelrouter_<ver>_amd64.deb
modelrouter-<ver>.x86_64.rpm
modelrouter-<ver>-linux-amd64.tar.gz
```
deb / rpm 安装后:
```bash
sudo systemctl enable --now modelrouter
sudo journalctl -u modelrouter -f
```
tar 包内只有:`modelrouter`、`config.example.yaml`、`modelrouter.service`、
`migrate-to-modelrouter.sh`、`adapters/`。
**发行包不含业务插件**(`channels` / `billing` / `quota` / `usage` 等)。
它们保留在仓库中作为可选扩展,需要时手工放入 `plugin_dir`,网关不会自动安装、
覆盖或删除。这是有意的:插件属于运维资产,生命周期不与内核绑定。
首次启动会在 `/etc/modelrouter/config.yaml` 生成一份**随机 admin key** 的配置,
key 打印在启动日志里。发行包不预置任何可用 key(历史上曾因此泄露过一个真实的
`sk-gw-local-0001`)。
## 3. 目录与权限
- `/etc/modelrouter` 是唯一的读写根(systemd unit 用 `ProtectSystem=strict` +
`ReadWritePaths=/etc/modelrouter` 固化这一点)。
- `master.key` 必须 0600。当前 unit 以 root 运行:降权到普通用户需要先把
`master.key` 交给该用户并统一目录属主,否则服务会"带着明文落盘继续跑",
失败只留一行日志——这是独立变更,不要和加固混在一起。
- `plugin_dir` 由运维维护;插件自带的状态文件(如 `.channels-store.json`)也在其中。
## 4. 从 llmsproxy 迁移
二进制、配置目录、服务名、环境变量已统一为 `modelrouter`:
| 旧 | 新 |
|---|---|
| `/usr/local/bin/llmsproxy` | `/usr/local/bin/modelrouter` |
| `/etc/llmsproxy/` | `/etc/modelrouter/` |
| `llmsproxy.service` | `modelrouter.service` |
| `LLMSPROXY_PLUGIN_DIR` | `MODELROUTER_PLUGIN_DIR` |
| `LLMSPROXY_XDISPLAY_BASE` | `MODELROUTER_XDISPLAY_BASE` |
| `LLMS_PROXY_MASTER_KEY` | `MODELROUTER_MASTER_KEY` |
迁移脚本**默认 dry-run**,复制而非移动,旧目录永不删除:
```bash
sudo ./packaging/migrate-to-modelrouter.sh # 预演(逐条打印将做什么)
sudo ./packaging/migrate-to-modelrouter.sh --apply # 执行
```
它做五件事:复制状态目录并改写配置里的旧绝对路径、复核 `master.key` 0600、
安装新二进制(旧文件保留)、由旧 unit 生成新 unit(改写 ExecStart /
WorkingDirectory / ReadWritePaths)、复制 `/usr/share` 资源;最后跑
`modelrouter -check` 并 `systemctl daemon-reload`。
确认新服务可用后,再手工清理旧布局(脚本不会删):
```bash
sudo systemctl disable --now llmsproxy.service
sudo rm -f /etc/systemd/system/llmsproxy.service
sudo rm -rf /usr/local/bin/llmsproxy /etc/llmsproxy /usr/share/llmsproxy
```
回滚(不需要旧文件被删):
```bash
sudo ./packaging/migrate-to-modelrouter.sh --rollback --apply
```
**读取侧兼容**:旧配置路径与旧环境变量仍被接受,所以"先换二进制、后改 unit"
这种顺序不会立刻失效。但写入永远落在新路径,迁移完成后不要两台布局混用。
## 5. 部署后验证清单
```bash
# 1) 进程与监听
systemctl status modelrouter --no-pager
ss -ltnp | grep modelrouter
# 2) 鉴权生效(无 key 必须 401)
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080/v1/models
# 3) 带 key 能列出模型
curl -s -H "Authorization: Bearer $KEY" http://127.0.0.1:8080/v1/models | head -c 400
# 4) 真实请求走通(证明 LuaJIT 适配器路径未被打断)
curl -s -H "Authorization: Bearer $KEY" \
-d '{"model":"AUTO","messages":[{"role":"user","content":"hi"}]}' \
http://127.0.0.1:8080/v1/chat/completions | head -c 400
# 5) 仓库文件与生产文件一致(部署脚本的收尾检查)
sha256sum /usr/local/bin/modelrouter build/modelrouter
```
第 4 步失败但第 3 步成功,通常是适配器或上游凭据问题,不是部署问题:
查看 `journalctl -u modelrouter -n 100`。

34
docs/development.md Normal file
View File

@ -0,0 +1,34 @@
# 开发与目录结构
> 本文档由 README 拆分而来:目录职责、开发协作、测试。
> 返回 [README](../README.md)。
## 目录
```
cmd/modelrouter # 入口(纯 Go 服务端)
cmd/gui # Electron 桌面 GUI(内嵌核心 + 托盘 + 打包)
internal/config # YAML 配置加载/校验
internal/lua # LuaJIT VM + worker 池 + AdapterCache + 内置适配器 (embed)
internal/provider # Provider(HTTP) + Registry(路由)
internal/gateway # OpenAI 兼容 HTTP 服务 + 鉴权 + SDK/流式 + WebUI
internal/types # 统一格式 & OpenAI wire 类型
```
## 开发协作
- **Git 分支工作流**:主分支 / 特性分支 / 发布分支的规范见
**[docs/git-workflow.md](docs/git-workflow.md)**([English](docs/git-workflow-en.md))。
主线:`main` 永可部署 → 特性合入 `main` → 切 `release/vX.Y.Z` 打 tag 发布 →
hotfix 提交发布分支并 cherry-pick 回 `main`。
- **Lua 适配器协议**:见 **[docs/lua-adapters.md](docs/lua-adapters.md)**
([English](docs/lua-adapters-en.md))。
## 测试
```bash
go test -tags luajit ./...
```
覆盖:配置校验、适配器加载/变换、签名钩子、Gateway 鉴权、SDK 结算、SSE 流式、
模型路由、多模态透传与 thinking 预算。

48
docs/features.md Normal file
View File

@ -0,0 +1,48 @@
# 实现与完整特性
> 本文档由 README 拆分而来:架构来源与完整特性清单(README 只保留摘要)。
> 返回 [README](../README.md)。
## 实现
统一 OpenAI 兼容网关:把多个上游 LLM 源(DeepSeek、Qijiar、OpenAI、Anthropic、Gemini、
Groq、Mistral、Ollama、KimiCode…)通过 **Lua 适配器** 做协议转换,对内网暴露
一个标准 `OpenAI Chat Completions` 接口(`/v1/chat/completions` + `/v1/models`),
支持单次结算、SSE 流式、**AUTO 模型路由**与**多模态**透传。
从 [HomeAgent](https://gitcode.com/JianFeeeee/HomeAgent) 的多源 LLM 适配层
(`internal/agent/api/provider.go` + `internal/lua/adapters/*`)抽离而来并独立演进。
## 特性
- **多源**:一个进程内配置任意多个上游源,按请求的 `model` 自动路由。
- **AUTO 模式**:`default_model: AUTO` 时按优先级页保存的 AUTO 链档位逐档调度并发请求。
- **统一输出**:所有源对外都是 OpenAI 格式(含 `reasoning_content`、`tool_calls`、`usage`)。
- **生图**:`POST /v1/images/generations`,`kind: image` 的模型独立路由。
- **多模态**:`content` 数组(`image_url` 等)在多源间无损透传,Anthropic/Gemini/Ollama 自动转换。
- **LuaJIT VM**:基于 golua 绑定的 LuaJIT,每个适配器独立 VM + worker 池,安全并发。
池是 **弹性** 的:`Σ max_concurrent` 只是上限而非预分配,状态按需创建、空闲时自动回收,
因此空载网关几乎不持有 Lua 状态(启动 0 个)。
- 扩容步长由适配器最大并发决定:`clamp(ceil(max/8), 1, 8)`,且仅在**真实并发争用**
(所有现有状态均已占用)时批量预热,顺序流量始终只用 1 个状态。
- 缩容步长由当前连接数决定:`clamp(ceil(冗余/(1+占用)), 1, 冗余)`——无连接时一轮收到
常驻下限,忙时每轮只释放 1 个,保护热路径。WebUI「适配器」页展示实时池指标。
- **thinking 预算**:请求里的推理开关/预算归一为中立的 `thinking:{enabled,budget_tokens}`,
适配器映射到各自协议(Anthropic `thinking.budget_tokens`、Gemini `thinkingConfig`、
DeepSeek `thinking.type="disabled"`)。
- **Lua 适配协议**:每个源挂一个 `.lua` 适配器,完成 `transform_request` /
`transform_response` / `transform_stream_chunk` 双向转换,协议差异全在 Lua 层。
- **签名 / 请求头钩子**:适配器可定义 `build_headers(meta)`,在 Go 发 HTTP 前
动态注入/签名请求头——用于云端 API 校验调用方 app(如 KimiCode 只放行特定
agent)。提供 `hmac_sha256_hex` / `sha256_hex` / `base64_encode` 等签名辅助。
- **WebUI**:内置管理界面,可在线查看/新增/编辑上游源与模型、配置 AUTO 优先级链、
管理密钥模型范围,写入运行时文件持久化。
- **密钥加密存储**:运行时文件中的上游 `api_key`、自定义请求头值、网关 key 均以
AES-256-GCM 加密落盘(`master.key` 独立 0600,或 `MODELROUTER_MASTER_KEY`)。
- **实时源探测**:状态页探测各源可达性(`GET {base}/models`,失败回退最小请求),
不污染正常调度的退避状态,错误信息在 UI 可悬停查看。
- **鉴权**:网关自身用 `gateway_keys` 校验客户端 Bearer key;与上游各自的 key 相互独立。
- **流式**:SSE `chat.completion.chunk`,含角色首包与 `[DONE]` 收尾。
- **全链路审计**:HTTP 访问日志、登录/配置变更审计、请求记录 CSV 导出、密钥用量统计 CSV 导出
- **AES-256-GCM 加密存储**:运行时文件敏感字段加密落盘,master.key 0600 权限,支持环境变量注入主密钥

86
docs/gui.md Normal file
View File

@ -0,0 +1,86 @@
# 桌面 GUI(Electron)
> 本文档由 README 拆分而来:Headless 与 Desktop 两种产物、构建与运行。
> 返回 [README](../README.md)。
## 桌面 GUI(Electron)
可选独立产物,面向内网桌面端用户:**内嵌 ModelRouter 核心**(桌面壳形态),
一键启停、系统托盘常驻、开机自启、静默启动,内容区一比一内嵌完整 WebUI
(状态/对话/密钥/优先级/源/适配器 六个页面,免登录)。服务器用户继续使用纯 Go 二进制。
### 两种安装包:Headless 与 Desktop
发布到 Release 的安装包分为两类,面向不同用户:
- **Headless(服务器版)**:纯核心二进制(无 Electron),配置驱动,适合服务器、
容器、systemd 等无人值守场景。发布包含 `deb` + `rpm`(含 systemd unit、内存调优
与示例配置)与 `tar.gz`(二进制 + `config.example.yaml` + `adapters/` + README),
Windows 为 `.zip`。
- **Desktop(桌面版)**:Electron GUI 安装包,内嵌核心,适合个人桌面日常使用。
### 选择 GUI 还是纯后端
| 场景 | 推荐 | 理由 |
| ------ | ------ | ------ |
| 服务器 / 内网网关 / 无人值守常驻 | **Headless**(单二进制) | 极轻量(二进制 ~8MB;RSS 随源数量,单源 ~10MB、16 源 ~32–35MB,见[内存占用](#内存占用实测与调优)),零依赖单进程,直接跑在 systemd/任意容器里,远程管理 |
| 个人桌面日常使用 / 多设备内网共享 | **Desktop**(GUI) | 免登录内嵌 WebUI、系统托盘一键启停、开机自启、静默后台,适合不懂命令行的使用者 |
| Windows 桌面 | **Desktop** | 纯后端在 Windows 上需自行注册服务,GUI 提供原生托盘/自启体验 |
| CI 一键出三平台安装包 | **Desktop 打包脚本** | `make gui-*` 系列出 deb/AppImage/NSIS,可进发行流水线 |
两者完全同源:GUI 内嵌的就是纯后端的同一份 `modelrouter` 核心(LuaJIT 版),
配置/适配器格式完全一致,可随时互换。
### 构建与运行
```bash
cd cmd/gui
npm install
make gui # 或 npm run dev —— 开发运行(需先 make build 生成 bin/modelrouter)
make gui-deb # 本机构建 deb(分享给其它 Linux 用户)
make gui-dist # 构建 deb + rpm + AppImage(linux,rpm 需系统 rpmbuild)
make core-dist # 构建核心 deb + rpm + tar.gz(服务器版,需 nfpm)
make gui-win # 构建 Windows nsis 安装包(本机 mingw + wine)
make gui-win-docker # 构建 Windows nsis 安装包(全 docker 自足,含 wine)
make gui-dist-dir # 仅解包产物(调试用,不产安装包)
```
产物在 `cmd/build/gui-dist/`:`ModelRouter-<ver>.AppImage`、
`modelrouter-gui_<ver>_amd64.deb`、`ModelRouter Setup <ver>.exe`(win);
核心包在 `cmd/build/dist/`:`modelrouter_<ver>_amd64.deb`、`modelrouter-<ver>.x86_64.rpm`。
rpm 需系统 `rpmbuild`。每次打包后都会跑 `packaging/verify-dist.sh` 产物体检
(大小下限门禁——1.3.0 曾因宿主 wine 损坏产出 264KB 的残缺 exe 却没有被拦住)。
发布时会把产物重命名为 `ModelRouter-Desktop-*`(GUI)与 `ModelRouter-Headless-*`(纯后端)
两类上传到 GitCode Release。
#### Windows 打包(docker 固化方案,推荐)
`make gui-win-docker` 一步完成:交叉编译核心 + NSIS 打包**全在 docker 内**,宿主机
**无需 mingw、无需 wine、无需 node**。镜像内置 wine(含 i386 运行时),修复了
此前宿主 wine 损坏(缺 syswow64 → `c0000135`)导致产出 264KB 残缺 exe 的问题。
- 构建镜像:`cmd/gui/docker/win-builder/Dockerfile`(`golang:1.25` + mingw-w64 +
node + wine32/wine64 + 预编译 LuaJIT windows 版 `lua51.dll` 与 import lib)。
LuaJIT 源码取自 gitcode 镜像(国内可达),github 兜底;Go 依赖走 `https://goproxy.cn`。
- 一键发布:`cmd/gui/scripts/dist-win-docker.sh` —— 核心编译 + electron-builder
NSIS 全部在容器内执行(`USE_SYSTEM_WINE=true` 走镜像内的 wine);electron-builder
的 ~400MB 工具链缓存挂到 docker volume,二次构建增量复用。
- 首次运行自动构建 `modelrouter/win-builder` 镜像(约 5–10 分钟);已存在则直接复用。
- 打包完成后自动跑 `packaging/verify-dist.sh` 门禁:exe 低于 5MB / deb·rpm·7z 低于
10MB 即失败——残缺安装包不可能再溜进 Release。
```bash
# 全 Windows 发布(核心重编 + NSIS):
make gui-win-docker
```
### 特性
- **内嵌核心**:自动拉起捆绑的 `modelrouter`(LuaJIT 版),配置/密钥/adapter 存于
用户数据目录 `<userData>/profile/`,首次运行自动生成随机 admin key 写入 `keys`
(非 seed)并自注入,WebUI 免登录、不会提示"更换初始密钥"。端口可在设置中更改(默认 8787)。
- **系统托盘**:内核状态点、开机自启/静默启动开关、启停/重启内核、退出。
- **静默启动**:`--silent` 参数或设置项,启动仅驻托盘不弹窗;首次运行始终显示窗口。
- **开机自启**:Windows/macOS 用 `setLoginItemSettings`;Linux 写
`~/.config/autostart/modelrouter-gui.desktop`,静默开启时自动附加 `--silent`。

60
docs/performance.md Normal file
View File

@ -0,0 +1,60 @@
# 性能与内存
> 本文档由 README 拆分而来:实测内存占用、调优手段、thinking 预算。
> 返回 [README](../README.md)。
## 内存占用(实测与调优)
内存不是一个定数:它随**配置的源数量**增长(每个源一个 `http.Transport` 与连接池、
一组健康状态),与运行时长无关。本机实测(Linux x86_64,12 核):
| 部署形态 | 启动 RSS | 稳态 RSS |
|---|---|---|
| 1 源 / 1 适配器(最小配置) | ~4 MB | ~10 MB |
| 1 源 + 29 MB 历史审计日志 | ~19 MB | ~20 MB |
| **16 源 / 13 适配器 / 59 模型(本机生产)** | ~28 MB | **~32–35 MB** |
> 历史参考:本项优化前同一生产配置为 **~105 MB**。降幅来自三处:审计日志的**原始记录**
> 不再常驻内存(约 25 MB;统计聚合仍扫全量,但扫完即释放)、Lua 状态池不再单调增长、
> 以及下面两个运行时开关。
内存构成(生产实例分段测量,`/proc/<pid>/smaps`):
| 区域 | RSS | 说明 |
|---|---|---|
| Go 堆 | ~14 MB | provider/registry/scheduler 结构 + 连接池缓冲 |
| 其他匹名(线程栈 / LuaJIT chunk / runtime) | ~12 MB | 与线程数、已加载适配器数相关 |
| 二进制 text+rodata | ~8 MB | 映射的可执行文件页(只读、可被内核回收) |
| Go runtime 预留 | ~4 MB | VSZ 上看到的 2 GB+ 是地址空间预留,不占物理内存 |
| 共享库 | ~3 MB | libc / libluajit / libm |
**两个推荐的部署开关**(只适用于环境变量,无需改代码):
```ini
# /etc/systemd/system/modelrouter.service
Environment=GOGC=50
Environment=MALLOC_ARENA_MAX=2
```
- `MALLOC_ARENA_MAX=2`:LuaJIT 的分配走 cgo → glibc malloc,glibc 默认允许 `8×nproc`
个 per-thread arena,每个碰到 malloc 的 OS 线程会占用一个(各约 1 MB,且**不归还给系统**)。
实测从 8–15 个 arena(约 7–12 MB)降到 **0**。
- `GOGC=50`:把 Go 堆增长目标减半。**单独使用无效**(省下的堆会立即被更多 glibc
arena 吃掉,实测 20.3 → 21.5 MB 反而变大),必须与 `MALLOC_ARENA_MAX` 配合,
两者同时开启才降 **~19%**。网关是 I/O 密集型(本机 9 小时仅消耗 1min10s CPU),
多出的 GC 周期与它的空闲 CPU 相比可忽略。
> 桌面版(Electron)已在 `cmd/gui/main.js` 里默认为内嵌核心注入这两个开关;手动 export 同名
> 环境变量可覆盖。如果你用自己的 systemd unit / 容器,建议照上面加上。
可选:`GOMEMLIMIT=48MiB` 作软上限,实测再省 ~1 MB;代价是逐近上限时 GC 转激进,
温和场景不必开。
## thinking 预算
请求体带 `"thinking": {"enabled": false}` 时关闭推理;`"thinking": {"enabled": true,
"budget_tokens": 4096}` 指定预算。内核只定义这个**中立**语义字段,映射到具体协议由
适配器负责(Anthropic `thinking.budget_tokens`、Gemini `thinkingConfig.thinkingBudget`、
DeepSeek `thinking.type = "disabled"`)。内核不再向适配器泄漏 `disable_thinking` /
`extra_body` 这类私有字段。

View File

@ -15,6 +15,37 @@ ModelRouter 的插件是**单个 `.lua` 文件**,放在 `config.yaml` 的 `plu
---
## 0. 插件 SDK:唯一的公开接口
插件与网关之间**只有一个**公开接口表 `modelrouter`。它不是 Go 包的导出符号,
而是内核在启动插件 VM 时注入的 Lua 表;插件通过它调用内核能力,永远不需要
(也不应该)依赖任何 `__llmsproxy_*` 全局、Go 结构体或内部函数名。
```lua
modelrouter.sdk_version -- 字符串,契约版本;SDK 形状变化才递增
modelrouter.http(spec) -- 受出站白名单约束的 HTTP 请求
modelrouter.secrets -- 本插件专用的凭据保险库(set/get/delete/keys)
modelrouter.kernel -- 内核动作面(actions/call,受运维授权约束)
modelrouter.adapters -- 安装/更新/删除本插件自己的适配器
modelrouter.plugin_dir -- 本插件所在目录(内核给出的 ground truth)
```
规则:
- **只依赖上表**。内核重命名内部全局、改实现或加守卫时,插件不受影响;
这正是把六个散落的全局收进一张表的原因。
- **能力可能缺失**。未授予的能力(例如出站 HTTP 全禁)在表里是 `nil` 而不是
"存在但一调用就报错",插件应先判断再使用。
- **`sdk_version` 用来做兼容分支**。增量字段不递增版本;字段删除或调用约定
变化才递增。
- **历史别名**:`llmsproxy`(旧产品名)与 `__llmsproxy_{http,secrets,kernel,
adapters,plugin_dir}` 仍然指向同一批对象,只为已安装的插件继续可用而保留,
已废弃;新插件使用 `modelrouter.*`。
- 环境变量:插件目录优先读 `MODELROUTER_PLUGIN_DIR`,回退旧名
`MODELROUTER_PLUGIN_DIR`。
---
## 1. 快速上手
一个最小的插件:
@ -65,16 +96,14 @@ return plugin -- 必须返回一个 table
core.New
└─ lua.NewVM(adapter_dir).Start() 适配器状态
└─ lua.NewPlugins(vm, plugin_dir)
├─ SeedBundled() 仅当目录不存在时写入内置插件(目前是 billing)
└─ LoadDir() 按文件名字典序逐个加载
`plugin_dir` 中的 Lua 文件由网关逐个加载;安装包不携带业务插件,也不会自动创建或写入插件文件。首次启用时,先把需要的 `.lua` 插件放入该目录,再启动网关。
```
**加载失败不影响网关启动。** 一个语法错误的插件会被记录在 `GET /api/plugins`
的 `error` 字段里,永远不会被调用。这与适配器一致,但理由更强:插件是可选的
第三方扩展,因为一个 `.lua` 打错字就让网关起不来是错误的取舍。
**目录一旦存在就是权威的。** 与适配器同规则:首启会 seed 内置插件,之后目录里
的文件说了算,删除或编辑内置插件都是真实生效的操作。
插件目录由运维独立管理。网关不会 seed、覆盖或删除其中的插件;目录缺失时只会跳过插件加载,不影响网关启动。
### 2.1 热更新
@ -294,7 +323,7 @@ PUT /api/plugins/{name}/state 替换状态(admin)
"loaded": true
}],
"hook_errors": { "request_end": { "count": 3, "last_error": "billing: ..." } },
"plugin_dir": "/etc/llmsproxy/plugins"
"plugin_dir": "/etc/modelrouter/plugins"
}
```
@ -428,7 +457,7 @@ Cookie 鉴权都自然生效,无需暴露额外端口。
-- 每小时的定时任务:轮换 token 并重写适配器
function plugin.refresh_token()
local tok = fetch_new_token() -- 插件自己的取 token 逻辑
local name, err = __llmsproxy_adapters.install(
local name, err = modelrouter.adapters.install(
"myauth",
'return { name = "myauth", endpoint = "/chat/completions",' ..
' headers = { ["Authorization"] = "Bearer ' .. tok .. '" } }'
@ -439,9 +468,9 @@ end
| 调用 | 参数 | 返回 |
|---|---|---|
| `__llmsproxy_adapters.install(name, source)` | 适配器名 + Lua 源码 | `(生效名, nil)` 或 `(nil, err)` |
| `__llmsproxy_adapters.update(name, source)` | 同上(只限自己装的) | `(true, nil)` 或 `(nil, err)` |
| `__llmsproxy_adapters.remove(name)` | 适配器名(只限自己装的) | `(true, nil)` 或 `(nil, err)` |
| `modelrouter.adapters.install(name, source)` | 适配器名 + Lua 源码 | `(生效名, nil)` 或 `(nil, err)` |
| `modelrouter.adapters.update(name, source)` | 同上(只限自己装的) | `(true, nil)` 或 `(nil, err)` |
| `modelrouter.adapters.remove(name)` | 适配器名(只限自己装的) | `(true, nil)` 或 `(nil, err)` |
出错时不抛异常,而是返回 `(nil, err)`——与钩子/HTTP 处理器同一套约定,
一个手滑的插件错误不会带走网关。
@ -494,24 +523,66 @@ end
### 6.4 命名
- 全局 `__llmsproxy_plugin` 存放插件返回的 table,**不要占用**
- 全局 `__llmsproxy_plugin` 是内核存放插件返回 table 的**内部位置**,不属于 SDK,**不要占用**
- 插件间状态完全隔离,一个插件改不了另一个的全局
---
## 7. 内置计费插件
## 7. Billing 插件示例
`plugins/billing.lua`,默认随首启 seed 进来,开箱可用。
`internal/lua/plugins/billing.lua` 是仓库中的可选插件示例,不随核心安装包分发,也不会在启动时自动安装。需要使用计费功能时,应将该脚本单独复制到配置的 `plugin_dir` 后再重启网关。
### 7.1 计费维度
### 7.0 分层:billing 完全在插件里,内核没有任何 billing 代码
| 维度 | 用途 | 优先级 |
`billing` 是**插件自己的事**:规则文件、编译、计价、账目、规则编辑页都由
`plugins/billing.lua` 负责。内核里没有 billing 的配置结构、端点、编译或迁移逻辑
—— 它只提供所有插件通用的能力(加载 `.lua`、插件状态持久化、插件目录、
`plugin.http` 路由、插件可用的 `io`/`os`)。
| 项 | 位置 |
|---|---|
| 规则文件 | `<plugin_dir>/billing.json`(插件自己读写;内核不知道这个文件) |
| 计费账目 | 插件状态文件 `<plugin_dir>/.billing.state.json`(内核的通用状态持久化) |
| 规则编辑 API | `/api/plugins/http/billing/rules`(GET/PUT/POST/DELETE,插件路由) |
| 编译/计价 | 插件内 `compileRules()` / `priceFor()` / `costFor()` |
规则以**源名**(`source`)为键,而不是 URL:URL→源名的映射属于内核配置,插件
不需要为了它和内核对接;编辑页在浏览器里,直接取 `/api/sources` 给运营选源名。
`url: "*"` 只表示"全局模型价目表"(模型维度本来就不分源)。
> 历史:早期 `billing:` 写在 `config.yaml`、编译在 Go 的 `internal/billing` 里。
> GitCode issue #1 的根因正是"Go 与 Lua 各写一半覆盖语义",组合起来把模型价
> 静默清零。现在内核侧 billing 代码已全部删除,规则文件也由插件自持。
### 7.1 计费维度### 7.1 计费维度
| 维度 | 用途 | token 价覆盖顺序 |
|---|---|---|
| `sources` | 源的固定价(如按请求计费的转发服务) | 中 |
| `models` | 模型的 per-token 价 | 高 |
| `default` | 兜底价(通常全 0) | 最低 |
| `models` | 模型的 per-token 价 | 中 |
| `sources` | 源级价:固定价,或覆盖该源全部模型的 token 价 | 高 |
| `keys` | 单个网关 key 的覆盖价 | **最高** |
**token 价**优先级:`keys` > `models` > `default`。
**token 价**优先级:`keys` > `sources` > `models` > `default`。
(实现顺序就是"后写覆盖前写":`default` → `models` → `sources` → `keys`;
所以**源级 token 价会覆盖模型价**。这是有意的:运营要能给某个源整体改价。)
⚠️ 正因为 `sources` 会覆盖 `models`:**`mode: token` 的规则不产生任何源条目**。
规则未列出的模型会进入未定价统计(可见),而不是静默计 0 —— 这是 issue #1 / PR #2
的收口语义。只有 `free` / `subscription`("边际成本为零")会写有意为之的
`{prompt:0, completion:0}` 零价条目。
规则写法:
```json
{"active":"current","profiles":[{"id":"current","default":"unpriced","rules":[
{"source":"localzen","mode":"free"},
{"source":"codearts","mode":"token",
"models":{"glm-5.3":{"prompt":"0.15","completion":"0.60"}}},
{"url":"*","mode":"token","models":{"some-model":{"prompt":"1.0","completion":"2.0"}}}
]}]}
```
**`per_request` 固定价是叠加的**(不覆盖),所以一个生图模型可以既算 token 又收固定费。
**`cache_discount`** 见 §7.7。
@ -599,6 +670,17 @@ curl -H "Authorization: Bearer $ADMIN_KEY" \
仪表盘上有 "Unpriced" 卡片。**这个数应该是 0**;不是 0 就去补价目。
**profile 级兜底**:`profiles[].default` 可设为 `free` 或 `unpriced`(默认)。
它是插件运行时的策略(编译期写入 `prices.default_mode`):
| `default` | 没有单独规则的源 | `unpriced_reqs` |
|---|---|---|
| `unpriced`(省略时) | 零价,计入未定价债务 | +1 |
| `free` | 零价,视为**刻意免费** | 不增加 |
显式 `mode: unpriced` 的源即使在 `default: free` 下也仍然计入("只统计不收费"
的声明不能被兜底策略吞掉)。
注意"未定价"不等于"免费":这些请求的 `requests` / token 数**照常计入**
`total` 与各维度,只有金额是 0。
@ -651,7 +733,7 @@ cached = cache_hit_tokens → 全价 × cache_discount
插件的 `log(level, msg)` 输出到网关的日志:
```
journalctl -u llmsproxy -f | grep -i plugin
journalctl -u modelrouter -f | grep -i plugin
```
---

111
docs/routing.md Normal file
View File

@ -0,0 +1,111 @@
# 路由与调度
> 本文档由 README 拆分而来:模型路由、AUTO 链、冷却与半冷却、源与适配器、源模板。
> 返回 [README](../README.md)。
## 模型路由
`/v1/chat/completions` 的 `model` 解析顺序:
1. `source/model` 或 `source:model` 前缀 → 指定源;
2. 精确匹配某个源的 `model`;
3. 配置 `default_model: AUTO` 时 → 走优先级页保存的 AUTO 链(见下);
4. 否则回落(返回错误)。
任何 OpenAI 客户端,只要 `model` 设为某个源的 `name/任意名`,即可锁定走该源;
设为 `AUTO`(或网关配了 `default_model: AUTO`)即自动按优先级链选源。
## AUTO 链(优先级页)
AUTO 调度**只**由优先级页保存的规则决定,源配置里的 `models[].priority` 数字不再
参与调度、也不再显示。**聊天与生图是两套独立的链**,在优先级页面通过「聊天 / 生图」
按钮切换编辑:
- **聊天链**(`runtime_file` 的 `auto` 字段):服务 `POST /v1/chat/completions`
的 `model: AUTO` 请求;只接受 `kind: chat` 模型,配置在链里的生图槽位会被忽略。
- **生图链**(`runtime_file` 的 `auto_image` 字段):服务 `POST /v1/images/generations`
的 `model: AUTO` 请求;只接受 `kind: image` 模型。未配置生图链时,AUTO 生图回落
到「所有含生图模型的源、按注册顺序」的旧行为,兼容旧配置。
每条规则 = 一个「槽位」:`{ model, source, tier, token_quota, period, hours }`。
- `tier` 表示优先级档位:同一档的模型并排、共享该优先级;档位从上到下递减。
- 同一模型可配置多个槽位(如 A 源低配 → B 源低配 → A 源高配 → B 源高配),按档位顺延。
- `token_quota` > 0 时该槽位在重置周期内用满即顺延到下一槽位;`period` 支持
`hour` / `week` / `month` / `nhour`(配合 `hours`),空 = 不限。
- 同 tier 内按「偏好分」排序,失败槽位冷却后自动跳过;冷却 / 配额耗尽 / hard
错误会顺延到下一 tier,全部失败时返回 503 并附上每档失败摘要。
## 冷却与半冷却探测(自愈调度)
槽位失败后进入指数退避冷却(5s → 10s → 20s …,上限 **5 分钟**)。冷却不是全时段
硬闸:
- **前半段完全静默**:刚失败的槽位不接任何流量,避免对故障上游持续施压。
- **后半段放行 1 个探测**:窗口过半后,同一 `(源, 模型)` 最多允许 **一个** 请求作为
探测通过。探测槽数 = `clamp(max_concurrent / 10, 1, 2)`——`max_concurrent: 10`
的源恰好放行 1 个探测请求。
- **探测排在最后**:同 tier 内探测候选永远排在健康候选之后,只有没有正常槽可用时
才承接真实流量,因此探测不会抢走可用容量。
- **探测即真实请求**:成功则立即清零失败计数与冷却,槽位当场回到正常轮询——
上游恢复/额度恢复后不必等满整个冷却窗口。失败则重开一个新窗口,下次探测顺延到
新窗口的中点,不会连续重试。
分类冷却策略:
| 情况 | 冷却 | 说明 |
|---|---|---|
| 传输错误 / 5xx | 5s→10s→20s…,上限 5min | 指数退避 |
| 401 / 403 凭据错误 | 10min | 换 key 后由下一次探测自动接回 |
| 429 限流 | 30s 固定 | 「太快」不等于「坏了」,不进指数阶梯 |
| 配额耗尽(`insufficient_quota` / `余额不足` 等) | 对齐配额窗口,上限 30min | **不进指数阶梯、不涨失败计数**,额度恢复即可用 |
WebUI「优先级」页的槽位健康标签会区分 `冷却` / `待探测` / `探测中`,悬停可看窗口
起点、探测放行时刻与完全恢复时刻。
## 源(Source)与适配器(Adapter)的关系
- **源** 是到某个上游的连接描述:`name`、`base_url`、`api_key`、模型列表与优先级。
- **适配器** 是协议转换逻辑(Lua 脚本):把统一 OpenAI 格式请求转成上游原生格式,
再把上游响应/流式分块转回统一格式。
- 一个适配器可被多个源复用(如 `openai.lua` 同时服务多个 OpenAI 兼容站点);
同一个源也可以切换不同适配器(改 `adapter` 字段即可)。
- 源决定“连谁、暴露哪些模型”,适配器决定“怎么对话”——二者在 `sources[]` 条目中
通过 `adapter` 字段关联。
加载流程(`internal/core` 的 `Core` 负责装配):
1. 启动时 `lua.NewVM(adapter_dir)` 加载全部适配器:内置适配器(编译期 embed)+
`adapter_dir` 下的同名覆盖文件;内置文件写在代码内,覆盖文件要求更高优先级。
2. `config.Load` 读取 `config.yaml`,`config.NewStore(runtime_file)` 读 WebUI 改动的
运行时源,二者按名称合并成完整源列表。
3. `rebuildRegistry` 为每个源创建 `provider.Provider`(持有目标适配器),并按源的
并发上限配置适配器 worker 池**上限**(不预分配,见下)。
4. 请求进来时 `Registry` 按 `model` 路由到 Provider,Provider 调适配器
`transform_request` → HTTP 发送 → `transform_response` / `transform_stream_chunk`。
WebUI 上的"新增/编辑源"、"上传 Lua 适配器"、"改 AUTO 优先级链"与"密钥模型范围"
都即时生效(写入运行时文件或 `adapter_dir` 后重新装配,无需重启);直接编辑
`config.yaml` / `adapter_dir` 下的文件则需要重启进程才会重新加载。
在 WebUI 删除源/适配器时:运行时(WebUI 创建)的源与 `.lua` 文件会真正移除;
`config.yaml` 中定义的基源无法改写配置文件,采用删除标记隐藏(重启后仍隐藏),
在 UI 里重新添加同名源即可恢复。
## 源模板(多 key 负载均衡)
多个上游 key 共享同一套配置(URL / 适配器 / 模型列表 / 并发 / rpm)时,逐一复制
整段 source 太冗余。**源模板**保存除 `name` 和 `api_key` 以外的全部源字段,
从模板一键创建多个仅 key 不同的源,天然支持多 key 负载均衡(每个展开源独立调度、
独立冷却、独立健康状态)。
- WebUI「源」页右上角「**模板管理**」按钮:列出全部模板,可编辑 / 删除 / 新建。
- 添加源弹窗右上角两个按钮:
- **从模板创建** → 选模板 → 表单自动填充(name + key 仍需你填);
- **存为模板** → 把当前表单非 key/name 字段存为新模板。
- 模板持久化在 `runtime_file` 的 `source_templates` 字段,与运行时源一样热管理,
改完即时生效。模板本身不含密钥,可以安全共享 / 纳管。
模板不会自己变成源——它只是「配方」。真正承担流量的是从模板创建出来的、带 key
的具体源。

23
docs/webui.md Normal file
View File

@ -0,0 +1,23 @@
# WebUI
> 本文档由 README 拆分而来。
> 返回 [README](../README.md)。
## WebUI
内置管理界面(`GET /`),登录后可在浏览器完成:
- **状态页**:源在线状态(实时探测 + 悬停看错误)、模型/源/key 用量统计、请求记录,
支持按时间范围导出 CSV;点击模型可生成 pin 到该模型的连接配置。
- **对话页**:流式/非流式调试。
- **密钥页**:创建/编辑网关 key,为每个 key 配模型范围(模型 + 源 + token 配额 + 周期),
管理员管理全部 key,用户只看到自己的 key。每个模型砖块显示自己的配额徽标
(如 `1.0K 77×·1h`,未设配额显示 `∞`),点开可编辑该模型的 token 配额、
请求数配额与重置周期 —— 配额按模型独立生效,一个用满不影响同一 key 的其它模型。
「我的密钥」页对用户展示本 key 的模型范围与各自配额。
- **优先级页**:拖拽积木配置 AUTO 链档位。
- **源页**:在线增删改上游源(API key 等敏感字段加密落盘)。
- **适配器页**:上传 / 删除 Lua 适配器脚本。
改动写入 `runtime_file`(重启仍生效)。

View File

@ -1,4 +1,4 @@
// Package e2e runs the real llmsproxy binary against mocked upstreams over
// Package e2e runs the real modelrouter binary against mocked upstreams over
// real HTTP: startup, auth, chat, streaming SSE, image generation, and
// AUTO failover when a higher-priority source fails.
package e2e
@ -100,15 +100,15 @@ type gatewayUnderTest struct {
func buildBinary(t *testing.T) string {
t.Helper()
dir := t.TempDir()
bin := filepath.Join(dir, "llmsproxy")
bin := filepath.Join(dir, "modelrouter")
if runtime.GOOS == "windows" {
bin += ".exe" // go build writes the exact name; Windows needs the suffix to exec
}
out, err := exec.Command("go", "build", "-tags", "luajit", "-o", bin, "llmsproxy/cmd/llmsproxy").CombinedOutput()
out, err := exec.Command("go", "build", "-tags", "luajit", "-o", bin, "modelrouter/cmd/modelrouter").CombinedOutput()
if err != nil && runtime.GOOS == "windows" {
// local Windows dev box may lack LuaJIT: fall back to the bundled
// Lua runtime (the e2e adapters used here are passthrough-only)
out, err = exec.Command("go", "build", "-o", bin, "llmsproxy/cmd/llmsproxy").CombinedOutput()
out, err = exec.Command("go", "build", "-o", bin, "modelrouter/cmd/modelrouter").CombinedOutput()
}
if err != nil {
t.Fatalf("build: %v\n%s", err, out)

2
go.mod
View File

@ -1,4 +1,4 @@
module llmsproxy
module modelrouter
go 1.25.0

View File

@ -1,323 +0,0 @@
package billing
import (
"fmt"
"strings"
"llmsproxy/internal/config"
)
// Compile turns one billing profile into the prices table the billing plugin
// expects, resolving URL rules against the gateway's actual sources.
//
// WHY URL RULES NEED RESOLVING AT ALL: the operator declares pricing by URL
// because that is what a provider's price list is keyed on, and because several
// sources can point at the same URL. The plugin, however, looks up by
// `payload.source` (a source NAME) and by model — it has no idea what URL a
// request went to. So the URL match happens here, at compile time, where the
// config's name->base_url mapping is known, and the result is expressed in the
// dimensions the plugin already supports.
//
// Profiles are the "let the user choose" axis: the same URL can appear in
// several profiles and switching recomputes this table, so a gateway can be
// repriced without editing the plugin.
func Compile(profile *config.BillingProfile, sources []config.Source) (map[string]interface{}, error) {
return CompileOpts(profile, sources, false)
}
// CompileOpts is Compile with an explicit policy for rules that match no
// source.
//
// The default is STRICT and that is correct for startup: a profile with a
// typo'd URL prices nothing, every request on it is recorded as unpriced, and
// the bill silently comes out wrong. Failing the load is the right response.
//
// The rules editor needs the opposite. An operator editing a half-finished
// profile — adding a rule before the source that will use it exists, or
// renaming a URL while typing — must be able to save and then see the warning,
// not be blocked by an error they can only resolve by guessing. Lenient mode
// keeps the offending rule in the config and drops it from the compiled table,
// so it is preserved as text and clearly reported, without pretending it
// prices anything.
func CompileOpts(profile *config.BillingProfile, sources []config.Source, lenient bool) (map[string]interface{}, error) {
if profile == nil {
return nil, fmt.Errorf("no billing profile")
}
prices := map[string]interface{}{
"currency": "USD",
"default": map[string]interface{}{"prompt": 0.0, "completion": 0.0, "per_request": 0.0},
"sources": map[string]interface{}{},
"models": map[string]interface{}{},
"keys": map[string]interface{}{},
}
if profile.Default != "" {
prices["default_mode"] = profile.Default
}
// Currency is a profile-level statement; a rule may override it.
for i := range profile.Rules {
if c := profile.Rules[i].Currency; c != "" {
prices["currency"] = c
break
}
}
matched := map[string]bool{}
for i := range profile.Rules {
rule := &profile.Rules[i]
targets := matchSources(rule.URL, sources)
if len(targets) == 0 && rule.URL != "*" {
// A rule for a URL no source uses is almost always a typo (or a
// source that was removed). Failing loudly beats a profile that
// silently prices nothing — the whole reason this is a config file
// instead of a hand-written JSON blob.
if lenient {
continue // kept in the config text, reported as a warning
}
return nil, fmt.Errorf("rule url %q matches no configured source base_url", rule.URL)
}
for _, srcName := range targets {
if matched[srcName] {
// First rule wins. Two rules matching one source is ambiguous,
// and silently letting the later one win makes the file's
// meaning depend on ordering the operator cannot see.
continue
}
matched[srcName] = true
if err := applyRule(prices, rule, srcName, sources); err != nil {
return nil, err
}
}
}
// Sources no rule matched fall through to the profile default, which the
// plugin applies via prices.default. Recording them explicitly means the
// UI can say "this source is unpriced" instead of leaving the operator to
// infer it from a zero.
var unmatched []string
for _, src := range sources {
if !matched[src.Name] {
unmatched = append(unmatched, src.Name)
}
}
if len(unmatched) > 0 {
prices["unmatched_sources"] = unmatched
}
return prices, nil
}
// matchSources returns the source names whose base_url matches pattern.
// "*" matches every source (used as a catch-all default rule).
func matchSources(pattern string, sources []config.Source) []string {
var out []string
if pattern == "*" {
for _, s := range sources {
out = append(out, s.Name)
}
return out
}
want := normalizeURL(pattern)
for _, s := range sources {
if normalizeURL(s.BaseURL) == want {
out = append(out, s.Name)
}
}
return out
}
// normalizeURL compares URLs the way an operator expects: trailing slashes and
// case in the host are differences the provider's price list does not care
// about, and requiring an exact byte match would make the config brittle.
func normalizeURL(u string) string {
u = strings.TrimSpace(u)
u = strings.TrimRight(u, "/")
return strings.ToLower(u)
}
// applyRule writes one rule's pricing for one source into the prices table.
func applyRule(prices map[string]interface{}, rule *config.BillingRule, srcName string, sources []config.Source) error {
switch rule.Mode {
case "free":
// Explicitly priced at zero. This is NOT the same as unpriced: a source
// the operator says is free must not appear in the unpriced warnings,
// or those warnings become noise and stop being read.
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
}
return nil
case "subscription":
// A fixed monthly commitment: the per-request MARGINAL cost is zero, and
// the flat fee is reported separately. Spreading a monthly fee across
// requests would invent a per-request number the provider never charges,
// and it would change every time traffic did.
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
}
fixed, _ := prices["subscriptions"].(map[string]interface{})
if fixed == nil {
fixed = map[string]interface{}{}
prices["subscriptions"] = fixed
}
cur := "USD"
if rule.Currency != "" {
cur = rule.Currency
}
fixed[srcName] = map[string]interface{}{
"monthly": rule.Subscription, "currency": cur,
}
return nil
case "unpriced":
// Deliberately left out of `sources` so the plugin's unpriced_models /
// unpriced_reqs counters catch it. That is the point: a subscription
// plan whose credits cannot be converted to tokens must be VISIBLE as
// unbilled, not quietly estimated.
return nil
case "token":
if len(rule.Models) == 0 {
return fmt.Errorf("rule for url %q: mode token requires at least one model", rule.URL)
}
models := prices["models"].(map[string]interface{})
for modelID, t := range rule.Models {
entry := map[string]interface{}{}
prompt, err := usdPerM(t.Prompt)
if err != nil {
return fmt.Errorf("model %q prompt: %w", modelID, err)
}
completion, err := usdPerM(t.Completion)
if err != nil {
return fmt.Errorf("model %q completion: %w", modelID, err)
}
entry["prompt"] = prompt
entry["completion"] = completion
if t.CacheDiscount != nil {
entry["cache_discount"] = *t.CacheDiscount
}
// reasoning is optional and stays ABSENT (not nil, not 0) when the
// rule does not set it, which is what makes the billing plugin skip
// the repricing branch entirely. Writing an explicit 0 here would
// mean "thinking is free" and silently under-bill every request
// from a provider that bills it like ordinary output.
if strings.TrimSpace(t.Reasoning) != "" {
rp, err := usdPerM(t.Reasoning)
if err != nil {
return fmt.Errorf("model %q reasoning: %w", modelID, err)
}
entry["reasoning_price"] = rp
}
if lc := t.LongContext; lc != nil {
if lc.Above <= 0 {
return fmt.Errorf("model %q long_context: above must be > 0", modelID)
}
above, err := longContextRate(lc, prompt)
if err != nil {
return fmt.Errorf("model %q long_context: %w", modelID, err)
}
entry["long_context"] = map[string]interface{}{
"above": lc.Above,
// Marginal, not blended. The plugin charges the tokens past the
// threshold at this rate and the rest at the base rate, so the
// threshold composes with a cache discount (which applies to the
// cached portion) instead of forcing the plugin to guess how
// the two interact.
"prompt": above,
}
}
if rule.Peak != nil {
entry["peak"] = compilePeak(rule.Peak)
}
models[modelID] = entry
}
// A specific-URL token rule needs a source entry so a model served
// from THIS url but absent from `rule.Models` is still recognised as
// belonging to a priced provider.
//
// A WILDCARD rule ("*") must NOT get one. "*" is a model catalogue — it
// says "these model ids have these prices, wherever they come from" —
// and marking every source priced at {0,0} would make priceFor() set
// priced=true for every model the catalogue does NOT list. Paid
// sources (commandcode, alittokenplan, …) would then record $0 instead
// of showing up in unpriced_reqs, which is precisely the silent
// zero-billing failure this whole plugin is built to make visible.
// The model lookup p.models[payload.model] already marks a listed
// model priced on its own, so the wildcard loses nothing.
if rule.URL != "*" {
if _, ok := prices["sources"].(map[string]interface{})[srcName]; !ok {
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
}
}
}
return nil
}
return fmt.Errorf("unknown mode %q", rule.Mode)
}
// usdPerM converts a USD-per-million string into the per-token rate the plugin
// expects. Delegates to config.ParseFloatUSDPerM so the DSL validator and the
// compiler agree byte-for-byte on what a valid price is — a value that passes
// Validate() but fails here (or vice versa) would be the worst kind of drift.
func usdPerM(s string) (float64, error) {
return config.ParseFloatUSDPerM(s)
}
// compilePeak renders the peak window in the shape the plugin reads:
// { multiplier, windows = { { days = {...}, hours = { {lo,hi}, ... } } } }.
//
// The plugin reads `days` / `hours` pairs. The first version of the deployed
// price table used { start, end, weekdays } — a shape nothing reads — so peak
// traffic was billed at off-peak rates with no error anywhere. Compiling from
// typed config fields removes the chance of writing the wrong key names by hand.
func compilePeak(p *config.BillingPeak) map[string]interface{} {
win := map[string]interface{}{}
if len(p.Weekdays) > 0 {
days := make([]interface{}, 0, len(p.Weekdays))
for _, d := range p.Weekdays {
days = append(days, d)
}
win["days"] = days
}
if len(p.Hours) > 0 {
hours := make([]interface{}, 0, len(p.Hours))
for _, h := range p.Hours {
hours = append(hours, []interface{}{h[0], h[1]})
}
win["hours"] = hours
}
out := map[string]interface{}{"multiplier": p.Multiplier}
if len(win) > 0 {
out["windows"] = []interface{}{win}
}
return out
}
// longContextRate resolves the marginal per-token rate for the tokens ABOVE a
// threshold, given the model's base per-million prompt price.
//
// Two ways to express it are supported because providers publish both: an
// absolute rate (OpenAI quotes \$3.00/M past 200k) and a multiple (DeepSeek
// quotes exactly 2x past 128k). Neither may be missing: a step with no rate is a
// silent no-op, and an operator who wrote one meant something.
func longContextRate(lc *config.BillingLongContext, basePerToken float64) (float64, error) {
hasPrompt := strings.TrimSpace(lc.Prompt) != ""
hasMult := lc.Multiplier > 0
switch {
case hasPrompt && hasMult:
p, err := usdPerM(lc.Prompt)
if err != nil {
return 0, err
}
// Both given: the absolute rate wins, because it is the one the provider
// actually quotes. Silently multiplying instead would produce a number
// the operator never wrote.
return p, nil
case hasPrompt:
return usdPerM(lc.Prompt)
case hasMult:
return basePerToken * lc.Multiplier, nil
default:
return 0, fmt.Errorf("needs either prompt or multiplier")
}
}

View File

@ -1,214 +0,0 @@
package billing
import (
"testing"
"llmsproxy/internal/config"
)
// The wildcard token rule must not blanket-mark every source as priced.
//
// Why this is a hazard and not a style question: priceFor() sets priced=true as
// soon as the request's SOURCE has an entry in the price table, and that flag
// is what keeps a request OUT of unpriced_reqs. A "*" rule is a model
// catalogue — it prices specific model ids wherever they come from — but
// compiling it into a {0,0,0} entry on all 21 sources made every model the
// catalogue does not list look "priced at zero". Paid providers (commandcode,
// alittokenplan, …) then stopped appearing in the unpriced warnings, which is
// exactly the silent zero-billing failure the plugin exists to surface.
func TestWildcardTokenRuleDoesNotMarkEverySourcePriced(t *testing.T) {
sources := []config.Source{
{Name: "free1", BaseURL: "https://free1.example.com/v1"},
{Name: "paid1", BaseURL: "https://paid1.example.com/v1"},
{Name: "paid2", BaseURL: "https://paid2.example.com/v1"},
}
profile := &config.BillingProfile{
ID: "p",
Rules: []config.BillingRule{
{URL: "https://free1.example.com/v1", Mode: "free"},
{
URL: "*", Mode: "token",
Models: map[string]config.BillingToken{
"only-catalogued-model": {Prompt: "1.00", Completion: "2.00"},
},
},
},
}
prices, err := Compile(profile, sources)
if err != nil {
t.Fatalf("compile: %v", err)
}
srcs := prices["sources"].(map[string]interface{})
// The explicitly free source keeps its entry (free is a deliberate 0).
if _, ok := srcs["free1"]; !ok {
t.Error("the explicit free rule lost its source entry")
}
// Paid sources must NOT be marked, or their unlisted models bill $0.
for _, name := range []string{"paid1", "paid2"} {
if _, ok := srcs[name]; ok {
t.Errorf("wildcard token rule marked %q as priced at 0 — its unlisted "+
"models will bill $0 instead of appearing as unpriced", name)
}
}
// The catalogue itself must still be present, or the rule prices nothing.
models := prices["models"].(map[string]interface{})
if _, ok := models["only-catalogued-model"]; !ok {
t.Error("the wildcard rule did not publish its model price")
}
}
// The inverse guard: a token rule for a SPECIFIC url still marks that source,
// because it genuinely declares pricing for the models served from it.
func TestSpecificURLTokenRuleStillMarksItsSource(t *testing.T) {
sources := []config.Source{
{Name: "mine", BaseURL: "https://mine.example.com/v1"},
{Name: "other", BaseURL: "https://other.example.com/v1"},
}
profile := &config.BillingProfile{
ID: "p",
Rules: []config.BillingRule{{
URL: "https://mine.example.com/v1", Mode: "token",
Models: map[string]config.BillingToken{
"m": {Prompt: "1.00", Completion: "2.00"},
},
}},
}
prices, err := Compile(profile, sources)
if err != nil {
t.Fatalf("compile: %v", err)
}
srcs := prices["sources"].(map[string]interface{})
if _, ok := srcs["mine"]; !ok {
t.Error("a specific-url token rule must still mark its own source priced")
}
if _, ok := srcs["other"]; ok {
t.Error("an unrelated source was marked priced by a specific-url rule")
}
}
// TestReasoningPriceIsOmittedUnlessSet pins the absence convention for the
// thinking-token rate.
//
// A rule that says nothing about reasoning must produce a price entry with NO
// reasoning_price key, so the billing plugin skips the repricing branch and
// bills thinking at the ordinary output rate — which is what every provider
// actually charges. Writing an explicit 0 instead would read as "thinking is
// free" and silently under-bill every request from that model.
func TestReasoningPriceIsOmittedUnlessSet(t *testing.T) {
sources := []config.Source{{Name: "up", BaseURL: "https://up.example/v1"}}
// Case 1: no reasoning declared → the key must not appear.
p1 := &config.BillingProfile{ID: "p", Rules: []config.BillingRule{{
URL: "https://up.example/v1", Mode: "token",
Models: map[string]config.BillingToken{
"m": {Prompt: "1.00", Completion: "2.00"},
},
}}}
prices, err := Compile(p1, sources)
if err != nil {
t.Fatal(err)
}
entry := prices["models"].(map[string]interface{})["m"].(map[string]interface{})
if _, present := entry["reasoning_price"]; present {
t.Errorf("reasoning_price present (%v) without a rule; the plugin would treat 0 as free",
entry["reasoning_price"])
}
// Case 2: declared → present, and converted to per-token like every other rate.
p2 := &config.BillingProfile{ID: "p", Rules: []config.BillingRule{{
URL: "https://up.example/v1", Mode: "token",
Models: map[string]config.BillingToken{
"m": {Prompt: "1.00", Completion: "2.00", Reasoning: "6.00"},
},
}}}
prices2, err := Compile(p2, sources)
if err != nil {
t.Fatal(err)
}
e2 := prices2["models"].(map[string]interface{})["m"].(map[string]interface{})
rp, ok := e2["reasoning_price"].(float64)
if !ok {
t.Fatalf("reasoning_price missing or wrong type: %#v", e2["reasoning_price"])
}
if rp != 6e-6 {
t.Errorf("reasoning_price = %v, want 6e-6 (USD per TOKEN)", rp)
}
// A malformed rate must be rejected, not silently ignored: an operator who
// typed "6 USD" expects either that value or an error, never a rule that
// looks applied and bills nothing.
p3 := &config.BillingProfile{ID: "p", Rules: []config.BillingRule{{
URL: "https://up.example/v1", Mode: "token",
Models: map[string]config.BillingToken{
"m": {Prompt: "1.00", Completion: "2.00", Reasoning: "six dollars"},
},
}}}
if _, err := Compile(p3, sources); err == nil {
t.Error("a non-numeric reasoning rate must fail the compile")
}
}
// TestLongContextStepCompiles pins the long-context step's two accepted forms
// and its rejection of a step that would silently do nothing.
//
// Providers publish both shapes — DeepSeek says "2x past 128k", OpenAI quotes
// an absolute "$3.00/M past 200k" — so both must compile. Neither may be
// missing: a step with no rate is a rule that LOOKS like it handles long
// context and does not, which under-bills without any visible sign.
func TestLongContextStepCompiles(t *testing.T) {
sources := []config.Source{{Name: "up", BaseURL: "https://up.example/v1"}}
compile := func(lc config.BillingLongContext) (map[string]interface{}, error) {
p := &config.BillingProfile{ID: "p", Rules: []config.BillingRule{{
URL: "https://up.example/v1", Mode: "token",
Models: map[string]config.BillingToken{
"m": {Prompt: "1.00", Completion: "2.00", LongContext: &lc},
},
}}}
prices, err := Compile(p, sources)
if err != nil {
return nil, err
}
e := prices["models"].(map[string]interface{})["m"].(map[string]interface{})
return e["long_context"].(map[string]interface{}), nil
}
// Absolute rate (OpenAI's published shape).
lc1, err := compile(config.BillingLongContext{Above: 200000, Prompt: "3.00"})
if err != nil {
t.Fatalf("absolute long_context rate rejected: %v", err)
}
if lc1["above"] != 200000 || lc1["prompt"] != 3e-6 {
t.Errorf("absolute step = %#v, want above=200000 prompt=3e-6", lc1)
}
// Multiplier (DeepSeek's shape): 2x of the 1.00/M base.
lc2, err := compile(config.BillingLongContext{Above: 128000, Multiplier: 2})
if err != nil {
t.Fatalf("multiplier long_context rejected: %v", err)
}
if lc2["prompt"] != 2e-6 {
t.Errorf("multiplier step = %#v, want prompt=2e-6", lc2)
}
// Both given: the absolute rate wins, because that is the number the
// provider quotes. Multiplying instead would produce a figure the operator
// never wrote.
lc3, err := compile(config.BillingLongContext{Above: 128000, Prompt: "5.00", Multiplier: 2})
if err != nil {
t.Fatal(err)
}
if lc3["prompt"] != 5e-6 {
t.Errorf("both given = %#v, want the quoted absolute rate 5e-6", lc3)
}
// Neither: a no-op step must fail, not compile into a rule that silently
// bills long context at the base rate.
if _, err := compile(config.BillingLongContext{Above: 128000}); err == nil {
t.Error("a step with neither prompt nor multiplier must be rejected")
}
// A zero threshold would make every request "long".
if _, err := compile(config.BillingLongContext{Above: 0, Multiplier: 2}); err == nil {
t.Error("above=0 must be rejected")
}
}

View File

@ -7,7 +7,6 @@ import (
"fmt"
"os"
"path/filepath"
"strconv"
"strings"
"time"
@ -30,15 +29,6 @@ type Config struct {
Auto []ModelScope `yaml:"auto,omitempty"` // AUTO 调度链规则(WebUI 优先级页编辑,chat)
AutoImage []ModelScope `yaml:"auto_image,omitempty"` // AUTO 生图调度链规则(WebUI 优先级页·生图)
Keys []GWKey `yaml:"keys,omitempty"` // 网关密钥(WebUI 密钥页管理)
// BillingDSL declares per-URL pricing profiles for the billing plugin.
//
// Declarative, because hand-writing the prices JSON was error-prone in ways
// that looked successful: the first PUT went out without the required
// `prices` wrapper and silently REPLACED the accumulated totals with the
// price table; the second used a peak-window shape the plugin does not read,
// so peak traffic was billed at off-peak rates — no error, the bill just
// quietly halved. A schema-checked config file catches both at load.
BillingDSL *BillingDSL `yaml:"billing,omitempty"`
// PluginVisibility overrides the audience a plugin declared for itself,
// keyed by plugin name: "admin" hides a plugin (and its state) from user
// keys, "all" widens one that shipped admin-only, "user,admin" names roles
@ -95,7 +85,14 @@ type Source struct {
// ProxyURL routes this source's HTTP(S) traffic through an explicit proxy
// (e.g. http://127.0.0.1:7890). Empty = direct connection. Needed for
// upstreams behind geo/IP blocks that only respond through a proxy.
ProxyURL string `yaml:"proxy_url" json:"proxy_url,omitempty"`
ProxyURL string `yaml:"proxy_url" json:"proxy_url,omitempty"`
// Retry is an OPTIONAL upstream retry policy. It exists for upstreams that
// answer a business-level rejection with a status code the kernel would
// otherwise surface to the client — e.g. a per-account concurrency cap that
// clears by itself after tens of seconds. The kernel stays vendor-agnostic:
// the matcher is pure DATA (status list + response-body substrings), so no
// provider name ever appears in core.
Retry *RetryPolicy `yaml:"retry" json:"retry,omitempty"`
Meta map[string]interface{} `yaml:"meta" json:"meta,omitempty"`
Temperature float64 `yaml:"temperature" json:"temperature,omitempty"`
MaxTokens int `yaml:"max_tokens" json:"max_tokens,omitempty"`
@ -105,9 +102,66 @@ type Source struct {
RPM int `yaml:"rpm" json:"rpm,omitempty"` // optional requests-per-minute cap (0 = unlimited)
}
// RetryPolicy describes when a failed upstream attempt may be retried BEFORE
// anything is returned to the client. All conditions are data:
//
// OnStatus - retry only when the HTTP status is one of these
// OnBodyContains - ... AND the response body contains one of these substrings
// (empty = status alone decides)
// Delay - wait between attempts ("10s" etc.); parsed on demand
// MaxAttempts - TOTAL attempts including the first (1 or 0 = no retry)
//
// Scope: the retry happens inside one gateway request, so a stream is only
// retried before its first chunk reaches the client (the kernel already holds
// the first chunk back), and a non-stream call is retried as a whole.
type RetryPolicy struct {
OnStatus []int `yaml:"on_status,omitempty" json:"on_status,omitempty"`
OnBodyContains []string `yaml:"on_body_contains,omitempty" json:"on_body_contains,omitempty"`
Delay string `yaml:"delay,omitempty" json:"delay,omitempty"`
MaxAttempts int `yaml:"max_attempts,omitempty" json:"max_attempts,omitempty"`
}
// ShouldRetry reports whether one failed attempt matches this policy.
func (r *RetryPolicy) ShouldRetry(status int, body string) bool {
if r == nil || r.MaxAttempts <= 1 {
return false
}
statusOK := len(r.OnStatus) == 0
for _, s := range r.OnStatus {
if s == status {
statusOK = true
break
}
}
if !statusOK {
return false
}
if len(r.OnBodyContains) == 0 {
return true
}
for _, sub := range r.OnBodyContains {
if sub != "" && strings.Contains(body, sub) {
return true
}
}
return false
}
// DelayDuration is the configured pause between attempts (default 10s).
func (r *RetryPolicy) DelayDuration() time.Duration {
if r == nil || r.Delay == "" {
return 10 * time.Second
}
d, err := time.ParseDuration(r.Delay)
if err != nil || d < 0 {
return 10 * time.Second
}
return d
}
// Load reads and validates a config file. When the file does not exist yet a
// default config is generated at that path first (first-run bootstrap), so a
// fresh binary just works: `llmsproxy -config /path/to/config.yaml`.
// fresh binary just works: `modelrouter -config /path/to/config.yaml`.
func Load(path string) (*Config, error) {
if _, err := EnsureDefault(path); err != nil {
return nil, err
@ -423,182 +477,6 @@ type GWKey struct {
// chain" so a user can clear an override back to the global default.
func (k GWKey) HasOwnAuto() bool { return len(k.Auto) > 0 }
// BillingDSL holds declarative per-URL pricing profiles for the billing plugin.
//
// Profiles are the "let the user pick" axis: the same upstream URL can be
// declared under multiple profiles, and the operator chooses which is active.
// Switching recomputes and re-injects the billing plugin's prices table.
type BillingDSL struct {
// Active is the id of the profile in effect. Empty => the first profile.
Active string `yaml:"active,omitempty" json:"active,omitempty"`
Profiles []BillingProfile `yaml:"profiles,omitempty" json:"profiles,omitempty"`
}
// BillingProfile is one named set of per-URL pricing rules.
type BillingProfile struct {
ID string `yaml:"id" json:"id"`
Label string `yaml:"label,omitempty" json:"label,omitempty"`
// Default applies when no rule matches: "free" or "unpriced".
Default string `yaml:"default,omitempty" json:"default,omitempty"`
Rules []BillingRule `yaml:"rules,omitempty" json:"rules,omitempty"`
}
// BillingRule declares the pricing for everything under one URL.
type BillingRule struct {
// URL matches a source's base_url. "*" = any unmatched URL.
URL string `yaml:"url" json:"url"`
// Mode is one of: free, token, subscription, unpriced.
Mode string `yaml:"mode" json:"mode"`
Currency string `yaml:"currency,omitempty" json:"currency,omitempty"`
// Subscription is the flat monthly cost (for mode=subscription).
Subscription float64 `yaml:"subscription,omitempty" json:"subscription,omitempty"`
// Models declares per-token prices when mode=token.
Models map[string]BillingToken `yaml:"models,omitempty" json:"models,omitempty"`
// Free is true when mode=free (kept for explicitness).
// Peak applies to ALL token-priced requests under this URL.
Peak *BillingPeak `yaml:"peak,omitempty" json:"peak,omitempty"`
}
// BillingToken is the per-model price.
type BillingToken struct {
Prompt string `yaml:"prompt" json:"prompt"` // USD per million tokens, e.g. "0.15"
Completion string `yaml:"completion" json:"completion"` // USD per million tokens
CacheDiscount *float64 `yaml:"cache_discount,omitempty" json:"cache_discount,omitempty"`
// Reasoning is the USD-per-million rate for thinking tokens, for the rare
// provider that charges them differently from ordinary output. Omit it and
// reasoning is billed at Completion — which is what OpenAI, DeepSeek and
// Anthropic all do, and what every source on this deployment reports.
//
// It is a rate for a SUBSET of completion tokens, never an extra charge: the
// billing plugin applies only the difference between this and Completion, so
// setting it to a multiple of Completion reprices the thinking share rather
// than billing it twice.
Reasoning string `yaml:"reasoning,omitempty" json:"reasoning,omitempty"`
// LongContext is the prompt-price step above a token threshold. Nil = one
// flat rate, which is what a rule means when it says nothing.
LongContext *BillingLongContext `yaml:"long_context,omitempty" json:"long_context,omitempty"`
}
// BillingPeak declares peak-hour pricing.
type BillingPeak struct {
Multiplier float64 `yaml:"multiplier" json:"multiplier"`
// Weekdays: 1=Mon..5=Fri (Lua os.date !%w, 0=Sun)
Weekdays []int `yaml:"weekdays,omitempty" json:"weekdays,omitempty"`
// Hours: each pair is [lo,hi] inclusive
Hours [][2]int `yaml:"hours,omitempty" json:"hours,omitempty"`
}
// BillingLongContext is a prompt-price step for long inputs.
//
// Every major provider charges the input tokens ABOVE a threshold at a higher
// rate — DeepSeek doubles past 128k, OpenAI and Anthropic step up as well. A
// flat prompt price therefore under-bills exactly the traffic an agent workload
// generates, and it under-bills it silently: the number is plausible, the bill
// adds up, and nothing says the rate was wrong.
//
// The step applies to the PROMPT only. Output and cache reads are billed at
// their own rates regardless of input length, which is how providers publish
// it — the higher rate is for the extra context you made the model read, not for
// what it wrote back.
type BillingLongContext struct {
// Above is the token count the step starts at. A request with exactly this
// many prompt tokens is still at the base rate; the step covers the tokens
// BEYOND it. Inclusive-below matches how the providers document it.
Above int `yaml:"above" json:"above"`
// Prompt is the USD-per-million rate for the tokens above Above.
Prompt string `yaml:"prompt" json:"prompt"`
// Multiplier is an alternative to Prompt: a factor applied to the base rate
// (DeepSeek's published step is exactly 2x). When both are set Prompt wins,
// and setting neither is a validation error rather than a silent no-op.
Multiplier float64 `yaml:"multiplier,omitempty" json:"multiplier,omitempty"`
}
// Validate rejects a BillingDSL that could not work as written.
func (b *BillingDSL) Validate() error {
if b == nil {
return nil
}
ids := map[string]bool{}
for i := range b.Profiles {
p := &b.Profiles[i]
if p.ID == "" {
return fmt.Errorf("billing profile [%d] has no id", i)
}
if ids[p.ID] {
return fmt.Errorf("billing profile id %q appears more than once", p.ID)
}
ids[p.ID] = true
if p.Default != "" && p.Default != "free" && p.Default != "unpriced" {
return fmt.Errorf("profile %q: default must be free or unpriced, got %q", p.ID, p.Default)
}
for j := range p.Rules {
r := &p.Rules[j]
if r.URL == "" {
return fmt.Errorf("profile %q rule [%d]: url is required", p.ID, j)
}
switch r.Mode {
case "free", "token", "subscription", "unpriced":
default:
return fmt.Errorf("profile %q rule [%d]: mode must be free/token/subscription/unpriced, got %q", p.ID, j, r.Mode)
}
if r.Mode == "token" {
for m, t := range r.Models {
if t.Prompt == "" || t.Completion == "" {
return fmt.Errorf("profile %q model %q: prompt and completion are required for token mode", p.ID, m)
}
if _, err := ParseFloatUSDPerM(t.Prompt); err != nil {
return fmt.Errorf("profile %q model %q prompt: %w", p.ID, m, err)
}
if _, err := ParseFloatUSDPerM(t.Completion); err != nil {
return fmt.Errorf("profile %q model %q completion: %w", p.ID, m, err)
}
}
}
}
}
// Active must resolve to a profile id.
if b.Active != "" && !ids[b.Active] {
return fmt.Errorf("billing.active = %q but no profile has that id", b.Active)
}
return nil
}
// Resolve returns the profile with the given id; empty id (or an id that does
// not exist) falls back to the FIRST profile. A config with profiles but no
// active marker should mean "use the one I wrote first" rather than "price
// nothing", which is what a nil here would silently do.
func (b *BillingDSL) Resolve(id string) *BillingProfile {
if b == nil || len(b.Profiles) == 0 {
return nil
}
if id != "" {
for i := range b.Profiles {
if b.Profiles[i].ID == id {
return &b.Profiles[i]
}
}
}
return &b.Profiles[0]
}
// parseFloatUSDPerM parses a USD-per-million price string. Returns the
// per-single-token rate the plugin expects (price / 1e6).
//
// Kept as a string in the DSL rather than a float: "0.15" survives round-trips
// through YAML editors, and a malformed value must fail VALIDATION (with the
// file and field named) rather than parse as 0 and under-bill silently — which
// is exactly the failure mode that motivated the DSL.
func ParseFloatUSDPerM(s string) (float64, error) {
f, err := strconv.ParseFloat(strings.TrimSpace(s), 64)
if err != nil {
return 0, fmt.Errorf("not a number: %q", s)
}
if f < 0 {
return 0, fmt.Errorf("negative price: %q", s)
}
return f / 1e6, nil
}
// KeyQuota is retained only to carry a scope entry's caps through the admin
// API. Quotas are per model, never per key: there is deliberately no key-wide
// total, so exhausting one model's budget never blocks the others.

View File

@ -0,0 +1,59 @@
package config
import (
"testing"
"time"
)
// ShouldRetry 是纯数据判据:状态码列表 + 响应体子串。它决定"这次失败值不值得
// 在网关内部等一会儿重试"。厂商知识绝不能进内核,所以判据必须能用一个表格测清楚。
func TestRetryPolicyShouldRetry(t *testing.T) {
cases := []struct {
name string
policy *RetryPolicy
status int
body string
want bool
}{
{"nil policy", nil, 500, "anything", false},
{"max_attempts=1 disables", &RetryPolicy{OnStatus: []int{400}, MaxAttempts: 1}, 400, "x", false},
{"status matches, no body filter",
&RetryPolicy{OnStatus: []int{429}, MaxAttempts: 3}, 429, "", true},
{"status mismatch",
&RetryPolicy{OnStatus: []int{429}, MaxAttempts: 3}, 400, "429", false},
{"status + body match",
&RetryPolicy{OnStatus: []int{400}, OnBodyContains: []string{"TM.00001041"}, MaxAttempts: 3},
400, `{"error_code":"TM.00001041"}`, true},
{"status matches but body does not",
&RetryPolicy{OnStatus: []int{400}, OnBodyContains: []string{"TM.00001041"}, MaxAttempts: 3},
400, `{"error_code":"TM.00001005"}`, false},
{"empty substring never matches",
&RetryPolicy{OnStatus: []int{400}, OnBodyContains: []string{""}, MaxAttempts: 3}, 400, "x", false},
{"any of several body signals",
&RetryPolicy{OnStatus: []int{400, 429}, OnBodyContains: []string{"TM.00001041", "TPM limit"}, MaxAttempts: 3},
429, `{"message":"TPM limit exceeded"}`, true},
}
for _, tc := range cases {
if got := tc.policy.ShouldRetry(tc.status, tc.body); got != tc.want {
t.Errorf("%s: ShouldRetry(%d, %q) = %v, want %v", tc.name, tc.status, tc.body, got, tc.want)
}
}
}
func TestRetryPolicyDelay(t *testing.T) {
cases := []struct {
name string
policy *RetryPolicy
want time.Duration
}{
{"nil", nil, 10 * time.Second},
{"explicit", &RetryPolicy{Delay: "250ms"}, 250 * time.Millisecond},
{"garbage falls back", &RetryPolicy{Delay: "not-a-duration"}, 10 * time.Second},
{"negative falls back", &RetryPolicy{Delay: "-5s"}, 10 * time.Second},
}
for _, tc := range cases {
if got := tc.policy.DelayDuration(); got != tc.want {
t.Errorf("%s: DelayDuration() = %v, want %v", tc.name, got, tc.want)
}
}
}

View File

@ -27,8 +27,9 @@ func NewGatewayKey() (string, error) {
// SecretBox encrypts and decrypts sensitive values (upstream API keys,
// custom header values, gateway keys) for persist-time protection. The
// master key comes from the LLMS_PROXY_MASTER_KEY environment variable
// (hex, 64 chars) or from a generated <runtime-dir>/master.key file.
// master key comes from the MODELROUTER_MASTER_KEY environment variable
// (hex, 64 chars; the pre-rename LLMS_PROXY_MASTER_KEY is still accepted) or
// from a generated <runtime-dir>/master.key file.
type SecretBox struct {
aead cipher.AEAD
}
@ -37,10 +38,18 @@ type SecretBox struct {
// file next to the runtime file. If neither exists, a fresh random key is
// generated and written to master.key with mode 0600.
func NewSecretBox(runtimeFile string) (*SecretBox, error) {
if v := os.Getenv("LLMS_PROXY_MASTER_KEY"); v != "" {
key, err := hex.DecodeString(strings.TrimSpace(v))
// New name first, then the pre-rename name: an existing unit file that still
// exports LLMS_PROXY_MASTER_KEY must keep starting after an upgrade.
envKey := os.Getenv("MODELROUTER_MASTER_KEY")
envName := "MODELROUTER_MASTER_KEY"
if envKey == "" {
envKey = os.Getenv("LLMS_PROXY_MASTER_KEY")
envName = "LLMS_PROXY_MASTER_KEY"
}
if envKey != "" {
key, err := hex.DecodeString(strings.TrimSpace(envKey))
if err != nil || len(key) != 32 {
return nil, fmt.Errorf("LLMS_PROXY_MASTER_KEY: expected 64 hex chars")
return nil, fmt.Errorf("%s: expected 64 hex chars", envName)
}
return newBox(key)
}

View File

@ -6,7 +6,7 @@ import (
"strings"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// callback 平台的授权 URL 是最容易"看起来对、实际不可用"的地方:URL 能拼出来、

View File

@ -9,7 +9,7 @@ import (
"testing"
"time"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// 三个 callback 平台的回调路径**各是注册值**,且校验规则不同:

View File

@ -3,7 +3,7 @@ package core
import (
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// ★ 无凭据账号的批量清理:**只删没有凭据的**,有凭据的一律不动。

View File

@ -8,7 +8,7 @@ import (
"sync"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// ★ codearts 的模型目录必须从**两个**签名端点拉取并合并。

View File

@ -8,7 +8,7 @@ import (
"strings"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// CodeArts 端到端:起真实回调服务器 → 模上游回 code → 走真实 STS 交换

View File

@ -4,7 +4,7 @@ import (
"fmt"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// CodeArts 的 portal 拒绝 <10000 的回调端口(参考 listenOnCallbackPort 的

View File

@ -7,7 +7,7 @@ import (
"net/http/httptest"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// 有道每日活动领取是**三步流程**,参考注释点名:无可用活动是**正常业务状态**,

View File

@ -2,7 +2,7 @@ package core
import (
"encoding/json"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
"net/http"
"net/http/httptest"
"testing"

View File

@ -10,7 +10,7 @@ import (
"testing"
"time"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// 有道登录闭环的判据:portal 会把授权重定向回 **本机 loopback 接收器**

View File

@ -8,7 +8,7 @@ import (
"sync/atomic"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// loomyHarness:Loomy 微信扫码长轮询的测试台。

View File

@ -8,7 +8,7 @@ import (
"sync"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// 远端模型目录的判据:**远端为主 + 兜底补位**(照参考 reconcileWithFallback)。

View File

@ -9,7 +9,7 @@ import (
"sync/atomic"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// channelsMoreTest:新引入的三条真实流程 + 两个边界流的行为对齐测试。

View File

@ -5,12 +5,12 @@ import (
"path/filepath"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// ★ 插件加载时必须清掉上一进程留下的孤儿回调文件。
//
// 真实事故:生产 `/etc/llmsproxy/plugins/` 攒了 **18 个 callback-*.port**
// 真实事故:生产 `/etc/modelrouter/plugins/` 攒了 **18 个 callback-*.port**
// (对应的 .json 早已消失)。它们永远不会被清,因为:
//
// · loopbacks 追踪表只活在内存(plugin.state.loopbacks);
@ -25,7 +25,7 @@ func TestPluginLoadSweepsOrphanCallbackFiles(t *testing.T) {
if err := os.MkdirAll(pluginDir, 0755); err != nil {
t.Fatal(err)
}
t.Setenv("LLMSPROXY_PLUGIN_DIR", pluginDir)
t.Setenv("MODELROUTER_PLUGIN_DIR", pluginDir)
// 摆上一轮的残留(模拟上一进程被强杀后的现场)
orphans := []string{

View File

@ -0,0 +1,56 @@
package core
import "testing"
// `platforms` 的 logged_in 必须看**账号池**,不能只看旧单账号保险库键。
//
// 真实缺陷(2026-10-20 用户报「codearts 还是不可用」):
// `list_platforms` 当时写的是 `credentialOf("channels:" .. id)` —— 那是旧版
// 单账号的键;而现网凭据键是 `channels:<platform>:<accountId>`(一平台多账号)。
// 于是**所有**通过账号池接入的平台都被判成未登录,UI 把渠道卡片渲染成"未接入",
// 用户看到的正是"渠道不可用",而底层模型其实完全能用。
//
// 判据钉两层:池里的账号让 logged_in=true,且账号标识能被 UI 拿到。
func TestPlatformsLoggedInFromAccountPool(t *testing.T) {
_, ps := poolCore(t)
if _, err := ps.Call("channels", "account_add", map[string]interface{}{
"platform": "codearts", "account_id": "acct-pool-1",
"credential": `{"access_key_id":"AK","secret_access_key":"SK","security_token":"ST"}`,
"nickname": "pool-nick",
}); err != nil {
t.Fatalf("account_add: %v", err)
}
// list_platforms 是 HTTP 路由处理器(不在 plugin.exports 里),走探针取同一份数据
m, err := ps.Call("channels", "__platforms_probe", map[string]interface{}{})
if err != nil {
t.Fatalf("__platforms_probe: %v", err)
}
w, _ := m.(map[string]interface{})
j, _ := w["json"].(map[string]interface{})
arr, _ := j["platforms"].([]interface{})
if len(arr) == 0 {
t.Fatalf("platforms 不应为空")
}
var got map[string]interface{}
for _, raw := range arr {
p, _ := raw.(map[string]interface{})
if p["id"] == "codearts" {
got = p
}
}
if got == nil {
t.Fatalf("platforms 里找不到 codearts")
}
if got["logged_in"] != true {
t.Fatalf("账号池里有凭据的账号,logged_in 必须为 true;实得 %#v", got)
}
if got["account_id"] != "acct-pool-1" {
t.Errorf("account_id 应回池里的账号 id;实得 %#v", got["account_id"])
}
if got["account"] != "pool-nick" {
t.Errorf("account 应回池里的昵称;实得 %#v", got["account"])
}
if s, _ := got["accounts"].(float64); s < 1 {
t.Errorf("accounts 计数应 >=1;实得 %#v", got["accounts"])
}
}

View File

@ -4,7 +4,7 @@ import (
"encoding/json"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// TestChannelsPluginEndToEnd 是「同类插件」的完整验收:

View File

@ -9,7 +9,7 @@ import (
"testing"
"time"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// ★ 「为什么没有可用账号」必须区分**真的用尽**与**查不到**。

View File

@ -4,7 +4,7 @@ import (
"strings"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
"fmt"
"net/http"

View File

@ -4,7 +4,7 @@ import (
"strings"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// qoderReason 调真实的 transform_error 钩子(不自己重算)。

View File

@ -9,7 +9,7 @@ import (
"testing"
"time"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// Qoder 活动领取有三条参考用**抓包 + 用户报障**定位过的「会误报」陷阱,

View File

@ -8,7 +8,7 @@ import (
"sync/atomic"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// qoderHarness:Qoder PKCE 设备码轮的测试台。

View File

@ -8,7 +8,7 @@ import (
"sync/atomic"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// raccoonHarness:Raccoon 内置二维码扫码登录的测试台。

View File

@ -6,7 +6,7 @@ import (
"strings"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// ★ 删除账号必须**真的把源撤掉** —— 且撤不掉时不能静默。
@ -190,8 +190,14 @@ func TestChannelsDeclaresEveryKernelActionItCalls(t *testing.T) {
// ② 抽出实际调用的动作
called := map[string]bool{}
for _, m := range regexp.MustCompile(`__llmsproxy_kernel\.call\(\s*"([a-z_]+)"`).FindAllStringSubmatch(s, -1) {
called[m[1]] = true
// 插件只允许走公开 SDK(modelrouter.kernel);旧名 __llmsproxy_kernel 是
// 兼容别名,新代码用它就说明插件又在依赖内核私有全局了。
for _, m := range regexp.MustCompile(`\b(?:modelrouter|__llmsproxy_kernel)\.kernel\.call\(\s*"([a-z_]+)"|\b__llmsproxy_kernel\.call\(\s*"([a-z_]+)"`).FindAllStringSubmatch(s, -1) {
name := m[1]
if name == "" {
name = m[2] // 旧名的分支
}
called[name] = true
}
if len(called) == 0 {
t.Fatal("没扫描到任何内核调用(判据失效)")

View File

@ -3,7 +3,7 @@ package core
import (
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// ★ account_reorder 必须**真的改变内核里的源顺序**。

View File

@ -33,8 +33,8 @@ func TestAccountPoolStoreIsAtomicAndConsistent(t *testing.T) {
storePath := filepath.Join(dir, ".channels-store.json")
t.Logf("插件存储目录: %s", dir)
// ⚠️ 必须在**测试目录**里,绝不能是生产路径
if _, err := os.Stat("/etc/llmsproxy/plugins/.channels-store.json"); err == nil {
if filepath.Clean(dir) == "/etc/llmsproxy/plugins" {
if _, err := os.Stat("/etc/modelrouter/plugins/.channels-store.json"); err == nil {
if filepath.Clean(dir) == "/etc/modelrouter/plugins" {
t.Fatal("测试正在使用生产存储目录 —— 隔离失效(这会造成真实数据事故)")
}
}

View File

@ -7,7 +7,7 @@ import (
"strings"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// TestChannelsStoreIsPluginOwned 是「插件自持持久化」的回归。
@ -23,7 +23,7 @@ func TestChannelsStoreIsPluginOwned(t *testing.T) {
// 固定插件目录:两次「重启」必须共用同一目录,否则测不出持久化。
// 这也是防止测试写进生产插件目录的那道隔离(kernelTestCore 会 respect 它)。
dir := t.TempDir()
t.Setenv("LLMSPROXY_PLUGIN_DIR", dir)
t.Setenv("MODELROUTER_PLUGIN_DIR", dir)
load := func() *lua.Plugins {
c := kernelTestCore(t)
ps := c.Plugins()
@ -128,6 +128,10 @@ func TestChannelsMigratesLegacyKernelState(t *testing.T) {
c := kernelTestCore(t)
dir = c.Config().PluginDir
// 网关不再自动创建 plugin_dir(插件不由内核安装),测试自己建。
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
legacyPath := filepath.Join(dir, ".channels.state.json")
if err := os.WriteFile(legacyPath, b, 0o600); err != nil {
t.Fatal(err)
@ -179,7 +183,7 @@ func TestChannelsMigrationKeepsLegacyOnWriteFailure(t *testing.T) {
// (<dir>/.channels-store.json.tmp)**占成一个目录**,io.open(tmp,"w") 必失败。
// 不用 chmod:测试以 root 跑,root 绕过权限位,chmod 拦不住。
sub := t.TempDir()
t.Setenv("LLMSPROXY_PLUGIN_DIR", sub)
t.Setenv("MODELROUTER_PLUGIN_DIR", sub)
legacy := map[string]interface{}{
"version": 1,

View File

@ -11,8 +11,8 @@ import (
"sync/atomic"
"testing"
"llmsproxy/internal/config"
"llmsproxy/internal/lua"
"modelrouter/internal/config"
"modelrouter/internal/lua"
)
// mustBox 构造一个真 SecretBox(与运行期同一实现,密钥落 runtime dir)。

View File

@ -9,7 +9,7 @@ import (
"sync"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// ★ TRAE 的 batch 模型目录解析:两个**真实缺陷**必须在移植时一起带上
@ -120,23 +120,23 @@ func newTraeCatalogHarness(t *testing.T, onBody func(string)) *traeCatalogHarnes
// ① solo_agent:glm-5.2 有档位;另有三个必被硬性过滤的条目
`{"function":"solo_agent","config_info_list":[`+
`{"config_name":"glm-5.2","usage":"chat_completion","config_switch":true,`+
`"reasoning_effort_config":{"support_thinking":true,"options":["high","low"]}},`+
`"reasoning_effort_config":{"support_thinking":true,"options":["high","low"]}}`+
`]},`+
// ② solo_work_lite 排在**后面**:glm-5.2 在这里**没有**档位配置。
// 另三个"必被过滤"的条目也放这里 —— 生产里适配器恒发这条通道,
// 硬性过滤的判据必须在这条通道上被验证到(放 solo_agent 会因为
// admit_functions 策略先行剔除,测试就测不到过滤逻辑了)。
`{"function":"solo_work_lite","config_info_list":[`+
`{"config_name":"glm-5.2","usage":"chat_completion","config_switch":true},`+
`{"config_name":"summary-model","usage":"summary","config_switch":true},`+
`{"config_name":"disabled-model","usage":"chat_completion","config_switch":false},`+
`{"config_name":"hidden-model","usage":"chat_completion","config_switch":true,"is_invisible_to_user":true},`+
`{"config_name":"no-function-model","usage":"chat_completion","config_switch":true},`+
// is_custom_model=true ⇒ 上游以流内 4001 拒绝("仅可见但不可调用")
`{"config_name":"custom-flag-model","usage":"chat_completion","config_switch":true,`+
`"display_config":{"is_custom_model":true}},`+
// 对照组:名字带 custom_model_ 前缀但**标志为 false** ⇒ 必须**保留**
// (参考点名:判据是标志的值,不是模型名)
`{"config_name":"custom_model_prefix_but_ok","usage":"chat_completion","config_switch":true,`+
`"display_config":{"is_custom_model":false}}`+
`]},`+
// ② solo_work_lite 排在**后面**:glm-5.2 在这里**没有**档位配置
`{"function":"solo_work_lite","config_info_list":[`+
`{"config_name":"glm-5.2","usage":"chat_completion","config_switch":true}`+
`]},`+
// ③ 白名单外通道:整组必须被丢弃
`{"function":"inline_chat","config_info_list":[`+
`{"config_name":"only-in-bad-channel","usage":"chat_completion","config_switch":true}`+

View File

@ -4,7 +4,7 @@ import (
"strings"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// ★ TRAE 的**推理请求**必须用 Cloud-IDE-JWT 头族,不是 Bearer。

View File

@ -6,7 +6,7 @@ import (
"net/http/httptest"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// Trae 签到是最容易被"看起来对"骗过的地方:请求能发出去、上游可能也返回 200,

View File

@ -6,7 +6,7 @@ import (
"os"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// 手工移植的确定性派生必须与参考**逐字节一致**,否则签名的设备身份是错的

View File

@ -6,7 +6,7 @@ import (
"sync/atomic"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// ★ TRAE ExchangeToken 的凭据在 **Result 里**,不在顶层。

View File

@ -9,7 +9,7 @@ import (
"sync/atomic"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// ★ TRAE 回调只带 code(PKCE 形态)时必须**明确报出**「暂不支持 PKCE」,

View File

@ -10,7 +10,7 @@ import (
"sync/atomic"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// traeHarness:TRAE callback 流的测试台。假上游模拟 ExchangeToken;

View File

@ -5,7 +5,7 @@ import (
"strings"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// TestChannelsVncStackEndToEnd 验证 VNC 兜底链路在**真插件**上跑通:

View File

@ -6,7 +6,7 @@ import (
"strings"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// 显示号泄漏是「测试单独跑绿、全量跑红」的根源,值得一条判据钉住它。
@ -34,7 +34,7 @@ func TestXDisplaySelectionReclaimsStaleLocks(t *testing.T) {
// (见 kernelTestCore);硬编码 90 段会让本判据造出**根本不会被扫描到**的
// 陈旧锁,于是"没复用"这个断言永远失败 —— 那是判据与被测对象脱节。
base := 300
if v := os.Getenv("LLMSPROXY_XDISPLAY_BASE"); v != "" {
if v := os.Getenv("MODELROUTER_XDISPLAY_BASE"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n > 0 {
base = n
}

View File

@ -7,7 +7,7 @@ import (
"strings"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// ★ ZCode 的身份块是 3012 准入的**唯一开关**。
@ -269,7 +269,7 @@ func TestZcodeAdapterFailsLoudlyWithoutAnthropicBase(t *testing.T) {
if err := os.MkdirAll(pluginDir, 0755); err != nil {
t.Fatal(err)
}
t.Setenv("LLMSPROXY_PLUGIN_DIR", pluginDir)
t.Setenv("MODELROUTER_PLUGIN_DIR", pluginDir)
c := kernelTestCore(t)
ps := c.Plugins()

View File

@ -10,7 +10,7 @@ import (
"sync/atomic"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// zcodeHarness:ZCode 两段式服务端中介授权流的测试台。

View File

@ -16,11 +16,10 @@ import (
"sync/atomic"
"time"
"llmsproxy/internal/billing"
"llmsproxy/internal/config"
"llmsproxy/internal/lua"
"llmsproxy/internal/provider"
"llmsproxy/internal/scheduler"
"modelrouter/internal/config"
"modelrouter/internal/lua"
"modelrouter/internal/provider"
"modelrouter/internal/scheduler"
)
// Core owns the running configuration and adapters. mu guards every read and
@ -96,9 +95,6 @@ func NewFromConfig(cfg *config.Config) (*Core, error) {
for name, aud := range cfg.PluginVisibility {
c.plugins.SetPluginVisibility(name, aud)
}
if err := c.plugins.SeedBundled(); err != nil {
log.Printf("[core] seed bundled plugins: %v", err)
}
if err := c.plugins.LoadDir(); err != nil {
log.Printf("[core] plugin dir: %v", err)
}
@ -110,7 +106,6 @@ func NewFromConfig(cfg *config.Config) (*Core, error) {
log.Printf("[core] plugin %v %v loaded (hooks=%v ui=%v)",
p["name"], p["version"], p["hooks"], p["ui"])
}
c.applyBillingDSL()
c.store = config.NewStore(cfg.RuntimeFile)
// Share one box between the runtime store and config.yaml so a single
// master.key seals both files. config.Load left the config holding
@ -1010,63 +1005,6 @@ func (c *Core) Close() {
}
}
// applyBillingDSL compiles the configured billing profile and injects it into
// the billing plugin.
//
// Configured pricing replaces the hand-written JSON blob that used to be PUT
// through the state API. That path was used twice in production and both times
// it "succeeded" while being wrong: once without the required `prices` wrapper
// (silently wiping the accumulated totals), once with a peak-window shape the
// plugin does not read (peak traffic billed at off-peak rates, no error). A
// typed config with load-time validation turns both mistakes into startup
// errors naming the file and field.
//
// A DSL with no profile, or one that fails to compile, leaves the plugin's
// prices untouched: billing keeps running on whatever it had, and the problem
// is logged rather than becoming a startup failure — the gateway must forward
// even when its cost accounting is misconfigured.
func (c *Core) applyBillingDSL() {
ps := c.plugins
if ps == nil || ps.Count() == 0 {
return
}
// The billing plugin may simply not be installed; a DSL without it is a
// no-op, not an error.
var have bool
for _, row := range ps.List() {
if row["name"] == "billing" && row["loaded"] == true {
have = true
break
}
}
if !have {
return
}
dsl := c.cfg.BillingDSL
if dsl == nil || len(dsl.Profiles) == 0 {
return
}
if err := dsl.Validate(); err != nil {
log.Printf("[billing] config invalid, profile not applied: %v", err)
return
}
profile := dsl.Resolve(dsl.Active)
if profile == nil {
log.Printf("[billing] no billing profile resolved (active=%q)", dsl.Active)
return
}
prices, err := billing.Compile(profile, c.cfg.Sources)
if err != nil {
log.Printf("[billing] profile %q not applied: %v", profile.ID, err)
return
}
if err := ps.SetState("billing", map[string]interface{}{"prices": prices}); err != nil {
log.Printf("[billing] applying profile %q: %v", profile.ID, err)
return
}
log.Printf("[billing] profile %q applied (%d rules, %d sources)", profile.ID, len(profile.Rules), len(c.cfg.Sources))
}
// ValidateScopeQuotas checks every scope entry's caps before they are stored.
// A typo in a period must be rejected at write time rather than silently
// becoming a never-resetting budget — the opposite of what was typed.

View File

@ -5,7 +5,7 @@ import (
"sync"
"testing"
"llmsproxy/internal/config"
"modelrouter/internal/config"
)
// newTestConfig returns an in-memory config backed by temp files so

View File

@ -8,7 +8,7 @@ import (
"sync/atomic"
"testing"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// readBundledChannels 读内置 channels 插件源(供行为/变异测试共用)。

View File

@ -6,8 +6,8 @@ import (
"fmt"
"strings"
"llmsproxy/internal/config"
"llmsproxy/internal/lua"
"modelrouter/internal/config"
"modelrouter/internal/lua"
)
// 内核动作实现(R4-c)

View File

@ -6,8 +6,8 @@ import (
"path/filepath"
"testing"
"llmsproxy/internal/config"
"llmsproxy/internal/lua"
"modelrouter/internal/config"
"modelrouter/internal/lua"
)
// 本文件验证 R4-c/R4-d 的闭环:**插件自己驱动外部源的链接认证,然后把源接进内核**。
@ -22,12 +22,12 @@ func kernelTestCore(t *testing.T) *Core {
dir := t.TempDir()
pluginDir := filepath.Join(dir, "plugins")
// 插件的自有存储位置由插件自己决定(读 config.yaml 的 plugin_dir,或
// LLMSPROXY_PLUGIN_DIR 覆盖)。测试**必须**指向临时目录 —— 否则插件会读到
// /etc/llmsproxy/config.yaml 并写进**生产插件目录**(真实事故:一次全量测试
// MODELROUTER_PLUGIN_DIR 覆盖)。测试**必须**指向临时目录 —— 否则插件会读到
// /etc/modelrouter/config.yaml 并写进**生产插件目录**(真实事故:一次全量测试
// 把 26 个测试账号写进了生产 .channels-store.json)。
// 调用方已显式设定时尊重它:多次「重启」要共用同一目录才能测持久化。
if os.Getenv("LLMSPROXY_PLUGIN_DIR") == "" {
t.Setenv("LLMSPROXY_PLUGIN_DIR", pluginDir)
if os.Getenv("MODELROUTER_PLUGIN_DIR") == "" {
t.Setenv("MODELROUTER_PLUGIN_DIR", pluginDir)
}
// ★ 显示器号与端口用**高段号**,避开生产。
//
@ -36,8 +36,8 @@ func kernelTestCore(t *testing.T) *Core {
// 并去抢,结果 `Xvfb 未就绪(/tmp/.X11-unix/X91 未出现)`,而生产上正跑着
// 一套浏览器栈。判据本身没错,错的是**两个 /tmp 互不可见**而号段重叠。
// 故测试整体挪到 300+。
if os.Getenv("LLMSPROXY_XDISPLAY_BASE") == "" {
t.Setenv("LLMSPROXY_XDISPLAY_BASE", "300")
if os.Getenv("MODELROUTER_XDISPLAY_BASE") == "" {
t.Setenv("MODELROUTER_XDISPLAY_BASE", "300")
}
cfg := &config.Config{
Path: filepath.Join(dir, "config.yaml"),

View File

@ -4,7 +4,7 @@ import (
"testing"
"time"
"llmsproxy/internal/lua"
"modelrouter/internal/lua"
)
// TestNoKernelActionDeadlocksFromLua 对**每个**内核动作做同一条断言:

View File

@ -7,7 +7,7 @@ import (
"testing"
"time"
"llmsproxy/internal/config"
"modelrouter/internal/config"
)
// AutoChainFor sits on the request hot path and reads keyAutoChains with no

View File

@ -4,8 +4,8 @@ import (
"path/filepath"
"testing"
"llmsproxy/internal/config"
"llmsproxy/internal/scheduler"
"modelrouter/internal/config"
"modelrouter/internal/scheduler"
)
// Per-key AUTO chains let an admin give one user their own scheduling chain

View File

@ -4,7 +4,7 @@ import (
"net/url"
"testing"
"llmsproxy/internal/config"
"modelrouter/internal/config"
)
// The plugin outbound allowlist is built from the hosts of configured sources.

View File

@ -6,7 +6,7 @@ import (
"strings"
"testing"
"llmsproxy/internal/config"
"modelrouter/internal/config"
)
// Startup must be idempotent, and every credential consumer must see PLAINTEXT.

View File

@ -11,7 +11,7 @@ import (
"strings"
"time"
"llmsproxy/internal/config"
"modelrouter/internal/config"
)
type adapterPayload struct {
@ -488,7 +488,7 @@ func (g *Gateway) handleStatsAPI(w http.ResponseWriter, r *http.Request) {
if to == 0 {
to = time.Now().UnixMilli()
}
csvHeaders(w, "llmsproxy-requests.csv")
csvHeaders(w, "modelrouter-requests.csv")
cw := csv.NewWriter(w)
names := map[string]string{}
for _, k := range g.core.ListKeys() {
@ -537,7 +537,7 @@ func (g *Gateway) handleStatsAPI(w http.ResponseWriter, r *http.Request) {
return
}
if r.URL.Query().Get("export") == "keys-csv" {
csvHeaders(w, "llmsproxy-keys.csv")
csvHeaders(w, "modelrouter-keys.csv")
cw := csv.NewWriter(w)
_ = cw.Write([]string{"key", "key_name", "role", "models", "total_requests", "success_requests", "failed_requests", "prompt_tokens", "completion_tokens", "total_tokens", "avg_latency_ms", "max_latency_ms", "created_at"})
keyFilter := exportKey(r)

View File

@ -31,7 +31,7 @@ import (
"sort"
"strings"
"llmsproxy/internal/config"
"modelrouter/internal/config"
)
// apiV1Routes dispatches /api/v1/*. Kept separate from routes() so the original
@ -144,7 +144,7 @@ func (g *Gateway) apiV1Index(w http.ResponseWriter, r *http.Request) {
}
index := map[string]interface{}{
"api_version": "v1",
"description": "llmsproxy management API. Authenticate with a gateway key: " +
"description": "modelrouter management API. Authenticate with a gateway key: " +
"'Authorization: Bearer <key>', '?api_key=<key>', or the gw_key cookie. " +
"Read endpoints accept any role; mutations require an admin key.",
"discovery": []ep{

View File

@ -7,7 +7,7 @@ import (
"strings"
"testing"
"llmsproxy/internal/config"
"modelrouter/internal/config"
)
// v1Gateway builds a gateway with one source and two keys, and returns the

View File

@ -1,282 +0,0 @@
package gateway
import (
"encoding/json"
"net/http"
"llmsproxy/internal/billing"
"llmsproxy/internal/config"
)
// Billing rules API.
//
// GET /api/plugins/billing/rules the active profile's rules + all profiles
// PUT /api/plugins/billing/rules replace the whole DSL and re-inject prices
// POST /api/plugins/billing/rules { profile, rule } append one rule
// DELETE /api/plugins/billing/rules { profile, url } remove one rule
//
// Why a separate endpoint instead of the generic plugin state: prices are
// plugin state, but they are also a durable, reviewable CONFIGURATION. Routing
// them through /state would let an admin key PUT arbitrary plugin state and
// silently reset the accumulated accounting. Here the server owns the shape:
// it validates the DSL, writes it back to config.yaml, recompiles, and hands
// the plugin its prices table. The operator edits rules; the plugin keeps
// numbers. The two are never mixed in one payload.
//
// Editing means "what the operator typed is what config.yaml holds". A
// billing mistake found next week must be traceable to a reviewable file, not
// to a blob in the plugin's state sidecar.
func (g *Gateway) handleBillingRules(w http.ResponseWriter, r *http.Request) {
if reqRole(r.Context()) != "admin" {
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
return
}
switch r.Method {
case http.MethodGet:
g.getBillingRules(w)
case http.MethodPut:
g.putBillingRules(w, r)
case http.MethodPost:
g.addBillingRule(w, r)
case http.MethodDelete:
g.delBillingRule(w, r)
default:
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET/PUT/POST/DELETE")
}
}
// billingRulesPayload is the UI-facing view: the profiles as declared, plus
// which one is active and the source URLs they can match — the editor needs
// the URL list to offer suggestions, and an operator typing a URL that matches
// no source is exactly the silent-no-pricing failure this feature exists to
// remove.
type billingRulesPayload struct {
DSL *config.BillingDSL `json:"billing"`
Active string `json:"active"`
URLs []string `json:"urls"`
Warnings []string `json:"warnings"`
}
func (g *Gateway) billingRulesPayload() billingRulesPayload {
out := billingRulesPayload{}
cfg := g.core.Config()
if cfg != nil && cfg.BillingDSL != nil {
// Copy so the response cannot be mutated back through the pointer.
cp := *cfg.BillingDSL
out.DSL = &cp
if p := cp.Resolve(""); p != nil {
out.Active = p.ID
}
}
for _, s := range g.sourceURLs() {
out.URLs = append(out.URLs, s)
}
out.Warnings = g.billingRuleWarnings()
return out
}
func (g *Gateway) sourceURLs() []string {
cfg := g.core.Config()
if cfg == nil {
return nil
}
seen := map[string]bool{}
var out []string
for _, s := range cfg.Sources {
if s.BaseURL == "" || seen[s.BaseURL] {
continue
}
seen[s.BaseURL] = true
out = append(out, s.BaseURL)
}
return out
}
// billingRuleWarnings reports rules that match no configured source. This is
// the silent killer of hand-written price tables: the rule parses, validates,
// and prices nothing at all, so every request on that URL lands in unpriced.
// Saying so out loud is the difference between a five-second fix and an
// afternoon wondering why the bill is zero.
func (g *Gateway) billingRuleWarnings() []string {
cfg := g.core.Config()
if cfg == nil || cfg.BillingDSL == nil {
return nil
}
urls := map[string]bool{}
for _, u := range g.sourceURLs() {
urls[u] = true
}
var warns []string
for _, p := range cfg.BillingDSL.Profiles {
for _, r := range p.Rules {
if r.URL == "*" || urls[r.URL] {
continue
}
warns = append(warns, "profile "+p.ID+": rule url "+r.URL+" matches no configured source")
}
}
return warns
}
func (g *Gateway) getBillingRules(w http.ResponseWriter) {
writeJSON(w, http.StatusOK, g.billingRulesPayload())
}
func (g *Gateway) putBillingRules(w http.ResponseWriter, r *http.Request) {
var body struct {
Billing *config.BillingDSL `json:"billing"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
return
}
if body.Billing == nil {
writeError(w, http.StatusBadRequest, "invalid_request", "billing is required")
return
}
g.applyBillingDSLUpdate(w, body.Billing)
}
func (g *Gateway) addBillingRule(w http.ResponseWriter, r *http.Request) {
var body struct {
Profile string `json:"profile"`
Rule config.BillingRule `json:"rule"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
return
}
cfg := g.core.Config()
if cfg == nil || cfg.BillingDSL == nil || len(cfg.BillingDSL.Profiles) == 0 {
writeError(w, http.StatusBadRequest, "no_profile", "no billing profiles configured; create one first")
return
}
next := cloneBillingDSL(cfg.BillingDSL)
idx := profileIndex(next, body.Profile)
if idx < 0 {
writeError(w, http.StatusBadRequest, "unknown_profile", "no such profile: "+body.Profile)
return
}
p := &next.Profiles[idx]
// Replace rather than append when the URL is already declared: two rules
// for one URL would silently make the first unreachable (first match wins),
// which is the exact ambiguity an editor should not be able to create.
replaced := false
for i := range p.Rules {
if p.Rules[i].URL == body.Rule.URL {
p.Rules[i] = body.Rule
replaced = true
break
}
}
if !replaced {
p.Rules = append(p.Rules, body.Rule)
}
g.applyBillingDSLUpdate(w, next)
}
func (g *Gateway) delBillingRule(w http.ResponseWriter, r *http.Request) {
var body struct {
Profile string `json:"profile"`
URL string `json:"url"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
return
}
cfg := g.core.Config()
if cfg == nil || cfg.BillingDSL == nil {
writeError(w, http.StatusBadRequest, "no_profile", "no billing profiles configured")
return
}
next := cloneBillingDSL(cfg.BillingDSL)
idx := profileIndex(next, body.Profile)
if idx < 0 {
writeError(w, http.StatusBadRequest, "unknown_profile", "no such profile: "+body.Profile)
return
}
p := &next.Profiles[idx]
out := p.Rules[:0]
found := false
for _, rule := range p.Rules {
if rule.URL == body.URL {
found = true
continue
}
out = append(out, rule)
}
if !found {
writeError(w, http.StatusNotFound, "not_found", "no rule for url "+body.URL)
return
}
p.Rules = out
g.applyBillingDSLUpdate(w, next)
}
// applyBillingDSLUpdate is the single write path: validate, persist to
// config.yaml, recompile, inject, and only then report success. If the config
// cannot be written the change is NOT applied in memory either — a rules editor
// that says "saved" and loses the edit on the next restart is worse than one
// that refuses.
func (g *Gateway) applyBillingDSLUpdate(w http.ResponseWriter, next *config.BillingDSL) {
if err := next.Validate(); err != nil {
writeError(w, http.StatusBadRequest, "invalid_rules", err.Error())
return
}
cfg := g.core.Config()
if cfg == nil {
writeError(w, http.StatusInternalServerError, "no_config", "no config loaded")
return
}
// Assign BEFORE saving. The previous order saved first and assigned after,
// so every rule edit reported success while writing a config.yaml with no
// billing section at all — the operator's edit vanished on restart and
// nothing in the API response said so.
prev := cfg.BillingDSL
cfg.BillingDSL = next
// Compile before persisting: a profile that cannot be turned into prices
// must not reach the config file, or the next restart fails to load the
// very rules the editor just accepted.
prices, err := billing.CompileOpts(next.Resolve(next.Active), cfg.Sources, true)
if err != nil {
cfg.BillingDSL = prev // no half-applied state
writeError(w, http.StatusBadRequest, "compile_failed", err.Error())
return
}
if err := cfg.Save(); err != nil {
cfg.BillingDSL = prev
writeError(w, http.StatusInternalServerError, "persist_failed", err.Error())
return
}
ps := g.core.Plugins()
if ps != nil {
if err := ps.SetState("billing", map[string]interface{}{"prices": prices}); err != nil {
writeError(w, http.StatusBadRequest, "plugin_error", err.Error())
return
}
}
p := g.billingRulesPayload()
writeJSON(w, http.StatusOK, map[string]interface{}{
"ok": true, "billing": p.DSL, "active": p.Active, "warnings": p.Warnings,
})
}
func cloneBillingDSL(d *config.BillingDSL) *config.BillingDSL {
out := &config.BillingDSL{Active: d.Active}
out.Profiles = make([]config.BillingProfile, len(d.Profiles))
copy(out.Profiles, d.Profiles)
return out
}
func profileIndex(d *config.BillingDSL, id string) int {
if id == "" {
return -1
}
for i, p := range d.Profiles {
if p.ID == id {
return i
}
}
return -1
}

View File

@ -1,281 +0,0 @@
package gateway
import (
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"sort"
"strings"
"testing"
"llmsproxy/internal/config"
)
// rulesGateway needs a real billing plugin (the API injects prices into it)
// AND two sources whose base_urls the rules will match, so it cannot reuse
// either existing helper: gatewayWithBilling has no sources, and
// newTestGateway loads no plugins.
func rulesGateway(t *testing.T) *Gateway {
t.Helper()
g := gatewayWithBilling(t)
cfg := g.core.Config()
cfg.Sources = []config.Source{
{Name: "localzen", BaseURL: "https://free.example.com/v1"},
{Name: "commandcode", BaseURL: "https://api.commandcode.ai/v1"},
}
return g
}
func rulesDo(t *testing.T, g *Gateway, method, body string) *httptest.ResponseRecorder {
t.Helper()
return doReq(t, g, method, "/api/plugins/billing/rules", body)
}
// TestBillingRulesCRUD covers the whole editor loop against the real HTTP
// surface: read, add, replace-by-URL, delete. The replace-by-URL case is the
// one worth pinning — two rules for one URL would make the first unreachable
// (first match wins) without any error, which is exactly the kind of silent
// ambiguity an editor must not be able to create.
func TestBillingRulesCRUD(t *testing.T) {
g := rulesGateway(t)
seed := `{"billing":{"active":"p1","profiles":[{"id":"p1","rules":[
{"url":"https://free.example.com/v1","mode":"free"}]}]}}`
if rr := rulesDo(t, g, http.MethodPut, seed); rr.Code != http.StatusOK {
t.Fatalf("seed rules = %d: %s", rr.Code, rr.Body.String())
}
// Add a token-priced rule for the second URL.
add := `{"profile":"p1","rule":{"url":"https://api.commandcode.ai/v1","mode":"token",
"models":{"deepseek-v4.1-flash":{"prompt":"0.15","completion":"0.60"}}}}`
if rr := rulesDo(t, g, http.MethodPost, add); rr.Code != http.StatusOK {
t.Fatalf("add rule = %d: %s", rr.Code, rr.Body.String())
}
var got billingRulesPayload
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
t.Fatal(err)
}
if len(got.DSL.Profiles[0].Rules) != 2 {
t.Fatalf("after add there are %d rules, want 2", len(got.DSL.Profiles[0].Rules))
}
// Adding the SAME url again must replace, not append.
if rr := rulesDo(t, g, http.MethodPost, `{"profile":"p1","rule":{"url":"https://api.commandcode.ai/v1","mode":"free"}}`); rr.Code != http.StatusOK {
t.Fatalf("re-add same url = %d: %s", rr.Code, rr.Body.String())
}
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
t.Fatal(err)
}
if n := len(got.DSL.Profiles[0].Rules); n != 2 {
t.Fatalf("re-adding an existing url appended instead of replacing: %d rules", n)
}
var mode string
for _, r := range got.DSL.Profiles[0].Rules {
if r.URL == "https://api.commandcode.ai/v1" {
mode = r.Mode
}
}
if mode != "free" {
t.Errorf("re-added rule mode = %q, want free (the replacement must win)", mode)
}
// Delete by URL.
if rr := rulesDo(t, g, http.MethodDelete, `{"profile":"p1","url":"https://api.commandcode.ai/v1"}`); rr.Code != http.StatusOK {
t.Fatalf("delete rule = %d: %s", rr.Code, rr.Body.String())
}
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
t.Fatal(err)
}
if len(got.DSL.Profiles[0].Rules) != 1 {
t.Errorf("after delete there are %d rules, want 1", len(got.DSL.Profiles[0].Rules))
}
}
// TestBillingRulesRejectUnknownURLWithWarning is the silent-failure guard: a
// rule whose URL matches no source prices nothing, so every request on it
// falls into unpriced. The API must say so instead of accepting it silently.
func TestBillingRulesRejectUnknownURLWithWarning(t *testing.T) {
g := rulesGateway(t)
if rr := rulesDo(t, g, http.MethodPut, `{"billing":{"active":"p","profiles":[{"id":"p",
"rules":[{"url":"https://typo.example.com/v1","mode":"free"}]}]}}`); rr.Code != http.StatusOK {
t.Fatalf("seed = %d: %s", rr.Code, rr.Body.String())
}
var got billingRulesPayload
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
t.Fatal(err)
}
if len(got.Warnings) == 0 {
t.Fatal("a rule matching no configured source was accepted with no warning — " +
"it will price nothing and every request will be recorded as unpriced")
}
if !strings.Contains(got.Warnings[0], "typo.example.com") {
t.Errorf("warning does not name the offending url: %v", got.Warnings)
}
// The editor needs the real URL list to offer suggestions.
if len(got.URLs) != 2 {
t.Errorf("urls offered to the editor = %v, want the 2 configured base_urls", got.URLs)
}
}
// TestBillingRulesInvalidIsRejectedNotPersisted checks the write path refuses
// bad input and leaves the previous rules in place — an editor that clears the
// table on a typo loses the price table.
func TestBillingRulesInvalidIsRejectedNotPersisted(t *testing.T) {
g := rulesGateway(t)
seed := `{"billing":{"active":"p","profiles":[{"id":"p","rules":[
{"url":"https://free.example.com/v1","mode":"free"}]}]}}`
if rr := rulesDo(t, g, http.MethodPut, seed); rr.Code != http.StatusOK {
t.Fatalf("seed = %d: %s", rr.Code, rr.Body.String())
}
// An unknown mode must be refused.
if rr := rulesDo(t, g, http.MethodPost, `{"profile":"p","rule":{"url":"https://x/v1","mode":"banana"}}`); rr.Code != http.StatusBadRequest {
t.Errorf("invalid mode accepted with %d, want 400", rr.Code)
}
var got billingRulesPayload
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
t.Fatal(err)
}
if len(got.DSL.Profiles[0].Rules) != 1 {
t.Errorf("a rejected rule was persisted anyway: %d rules", len(got.DSL.Profiles[0].Rules))
}
// An unparseable price must be refused too, not silently priced at zero.
if rr := rulesDo(t, g, http.MethodPost, `{"profile":"p","rule":{"url":"https://y/v1","mode":"token",
"models":{"m":{"prompt":"free","completion":"0.6"}}}}`); rr.Code != http.StatusBadRequest {
t.Errorf("non-numeric price accepted with %d, want 400", rr.Code)
}
}
// TestBillingRulesPersistAndReachThePlugin is the point of the whole
// endpoint, and mutation verification showed the earlier CRUD tests missed it
// twice: dropping cfg.Save() and dropping the SetState injection both left
// every test green. Both failures are invisible in a GET — the response echoes
// the in-memory rules either way — so this test checks the two consequences
// that actually matter:
//
// 1. persistence: the rule must be in the config file on disk, because an
// edit that evaporates on restart is a lie the UI told the operator;
// 2. effect: the billing plugin must now PRICE the source (its published
// prices table has a non-zero entry), because a rule that is stored but
// never injected prices nothing at all.
func TestBillingRulesPersistAndReachThePlugin(t *testing.T) {
g := rulesGateway(t)
cfg := g.core.Config()
seed := `{"billing":{"active":"p","profiles":[{"id":"p","rules":[
{"url":"https://free.example.com/v1","mode":"free"},
{"url":"https://api.commandcode.ai/v1","mode":"token",
"models":{"deepseek-v4.1-flash":{"prompt":"0.15","completion":"0.60"}}}]}]}}`
if rr := rulesDo(t, g, http.MethodPut, seed); rr.Code != http.StatusOK {
t.Fatalf("seed = %d: %s", rr.Code, rr.Body.String())
}
// 1. persisted to the file the process loaded from.
raw, err := os.ReadFile(cfg.Path)
if err != nil {
t.Fatalf("read config: %v", err)
}
if !strings.Contains(string(raw), "commandcode.ai") {
t.Errorf("the rule is not in config.yaml on disk — an edit that does not "+
"survive a restart:\n%s", raw)
}
reloaded, err := config.Load(cfg.Path)
if err != nil {
t.Fatalf("reload saved config: %v", err)
}
if reloaded.BillingDSL == nil || len(reloaded.BillingDSL.Profiles) == 0 {
t.Fatalf("saved config has no billing DSL: %s", raw)
}
// A round-trip through YAML must not lose the price strings.
found := false
for _, r := range reloaded.BillingDSL.Profiles[0].Rules {
if strings.Contains(r.URL, "commandcode") {
found = true
if m, ok := r.Models["deepseek-v4.1-flash"]; !ok || m.Prompt != "0.15" {
t.Errorf("price did not survive the YAML round-trip: %+v", m)
}
}
}
if !found {
t.Errorf("the token rule is missing after reload:\n%s", raw)
}
// 2. reached the plugin. Prices live in a separate Lua field from `state`,
// and the GET /state endpoint returns the published state together with
// the prices sidecar — which is exactly what the plugin's own UI reads.
state := g.core.Plugins().State("billing")
if state == nil {
t.Fatalf("billing plugin published no state")
}
rr := doReq(t, g, http.MethodGet, "/api/plugins/billing/state", "")
if rr.Code != http.StatusOK {
t.Fatalf("GET state = %d: %s", rr.Code, rr.Body.String())
}
var envelope struct {
Prices map[string]interface{} `json:"prices"`
}
if err := json.Unmarshal(rr.Body.Bytes(), &envelope); err != nil {
t.Fatalf("decode state envelope: %v", err)
}
prices := envelope.Prices
if prices == nil {
t.Fatalf("state payload carries no prices")
}
sources, ok := prices["sources"].(map[string]interface{})
if !ok {
t.Fatalf("prices has no sources map: %#v", prices)
}
cc, ok := sources["commandcode"]
if !ok {
t.Fatalf("the rule was saved but never injected: prices has %v, "+
"no commandcode entry. Every request on it stays unpriced.", keysOf(sources))
}
if cc == nil {
t.Error("commandcode price entry is nil")
}
// The free rule must price localzen, and the two must not be conflated.
if _, ok := sources["localzen"]; !ok {
t.Errorf("the free rule did not reach the plugin: %v", keysOf(sources))
}
// The injected price must be per-token, i.e. 0.15 USD/M => 1.5e-7.
// Asserting the exact magnitude catches a unit error (per-million vs
// per-token), which would be off by a factor of a million and still look
// like "a number".
models, ok := cc.(map[string]interface{})["models"].(map[string]interface{})
if ok {
if m, ok := models["deepseek-v4.1-flash"].(map[string]interface{}); ok {
p, _ := m["prompt"].(float64)
if p <= 0 || p > 1e-6 {
t.Errorf("injected prompt price = %v, want per-token (~1.5e-7); "+
"a per-million value here would over-bill by 1e6x", p)
}
}
}
}
func keysOf(m map[string]interface{}) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
// TestBillingRulesRequireAdmin keeps price rewriting behind the admin role,
// like every other plugin write: prices are configuration, not user data.
func TestBillingRulesRequireAdmin(t *testing.T) {
g := rulesGateway(t)
rec, err := g.core.CreateKey("viewer", "user", nil, "")
if err != nil {
t.Fatal(err)
}
userKey := rec.Key
for _, m := range []string{http.MethodPut, http.MethodPost, http.MethodDelete} {
req, _ := http.NewRequest(m, "/api/plugins/billing/rules", strings.NewReader(`{}`))
req.Header.Set("Authorization", "Bearer "+userKey)
req.Header.Set("Content-Type", "application/json")
if rr := newRecorderFor(t, g, req); rr.Code != http.StatusForbidden {
t.Errorf("user %s rules = %d, want 403", m, rr.Code)
}
}
}

View File

@ -13,11 +13,11 @@ import (
"sync/atomic"
"time"
"llmsproxy/internal/config"
"llmsproxy/internal/lua"
"llmsproxy/internal/provider"
"llmsproxy/internal/scheduler"
"llmsproxy/internal/types"
"modelrouter/internal/config"
"modelrouter/internal/lua"
"modelrouter/internal/provider"
"modelrouter/internal/scheduler"
"modelrouter/internal/types"
)
// chatRequest mirrors the OpenAI chat completions request the gateway accepts.

View File

@ -3,7 +3,7 @@ package gateway
import (
"testing"
"llmsproxy/internal/types"
"modelrouter/internal/types"
)
// TestMergeUsageKeepsZeroCacheDetails locks the streaming counterpart of the

View File

@ -3,7 +3,7 @@ package gateway
import (
"testing"
"llmsproxy/internal/types"
"modelrouter/internal/types"
)
// TestRecordChatUsageKeepsReasoningTokens pins the reasoning-token plumbing.

View File

@ -11,7 +11,7 @@ import (
// deploy.sh is the only release-critical script with no automated coverage, and
// its newest part — prune_adapter_backups — deletes directories under
// /etc/llmsproxy. This exercises the function in a throwaway directory with the
// /etc/modelrouter. This exercises the function in a throwaway directory with the
// REAL backup-name shape.
//
// The shape matters and got this wrong twice: the guard is
@ -45,7 +45,7 @@ func TestDeployPruneAdapterBackups(t *testing.T) {
// Redirect ONLY the base path. The logic under test must stay verbatim.
body = strings.Replace(body,
`local base="/etc/llmsproxy"`,
`local base="/etc/modelrouter"`,
`local base="`+base+`"`, 1)
// KEEP_BACKUPS must come from deploy.sh itself, NOT from a literal
// here: hardcoding 5 in the shim overrode whatever the script configured, so
@ -207,7 +207,7 @@ func TestDeployBacksUpTheLiveFileBeforeOverwriting(t *testing.T) {
//
// 事实:backup 清理最初只处理 `adapters.bak.*`,而 `sync_plugins` 同样每次部署
// 建一个 `plugins.bak.<时间戳>` —— 于是 adapters 的堆积被修好,plugins 继续涨。
// 实测线上 plugins.bak 积累到 **36 个**、/etc/llmsproxy 涨到 192M。
// 实测线上 plugins.bak 积累到 **36 个**、/etc/modelrouter 涨到 192M。
//
// 这是「两处只修一处」的老形状。判据钉住:prune_all_backups 必须同时裁两个前缀,
// 且新增任何 .bak.<时间戳> 前缀时都要在这里补上(否则又是一个泄漏点)。
@ -264,7 +264,7 @@ func TestDeployPruneCoversBothBackupPrefixes(t *testing.T) {
k := strings.Index(src, "prune_backups()")
kj := strings.Index(src[k:], "\n}\n")
fnBody := src[k : k+kj+3]
fnBody = strings.Replace(fnBody, `local base="/etc/llmsproxy"`, `local base="`+base+`"`, 1)
fnBody = strings.Replace(fnBody, `local base="/etc/modelrouter"`, `local base="`+base+`"`, 1)
keepRe := regexp.MustCompile(`(?m)^KEEP_BACKUPS=(\d+)`)
km := keepRe.FindStringSubmatch(src)
if km == nil {

View File

@ -11,7 +11,7 @@ import (
"strings"
"time"
"llmsproxy/internal/config"
"modelrouter/internal/config"
)
// Model discovery: turn "here is a URL and a key" into a configured source.

View File

@ -5,7 +5,7 @@ import (
"encoding/json"
"errors"
"io"
"llmsproxy/internal/config"
"modelrouter/internal/config"
"net/http"
"strings"
"time"

View File

@ -9,7 +9,7 @@ import (
"strings"
"testing"
"llmsproxy/internal/config"
"modelrouter/internal/config"
)
// upstream that answers /v1/models with the real captured catalog

View File

@ -126,7 +126,12 @@ func TestRealMutants(t *testing.T) {
},
{
name: "nested metadata no longer read",
from: "\tcopyFields(m.Metadata) // nested first",
// 锚点必须与 discover.go 逐字节一致:此前这里是 6 个空格,源码已
// 变成 2 个空格,于是这条变异永远"找不到锚点"——变异campaign 每次
// 都以 anchor not found 失败,而它本该验证"嵌套 metadata 被丢掉"能被
// 测试抓住。锚点漂移 = 这条变异从未真正跑过(和 issue #1 里
// `cc["models"]` 恒 false 的测试跳过是同一种失效:断言写了但没执行)。
from: "\tcopyFields(m.Metadata) // nested first",
to: "\tcopyFields(json.RawMessage(nil)) // mutant: nested metadata dropped",
},
{

View File

@ -14,8 +14,8 @@ import (
"testing"
"time"
"llmsproxy/internal/config"
"llmsproxy/internal/core"
"modelrouter/internal/config"
"modelrouter/internal/core"
)
func mockUpstream() *httptest.Server {

View File

@ -9,8 +9,8 @@ import (
"strings"
"testing"
"llmsproxy/internal/config"
"llmsproxy/internal/core"
"modelrouter/internal/config"
"modelrouter/internal/core"
)
// Per-key AUTO chain admin API. The behaviours pinned here are the ones that

View File

@ -9,8 +9,8 @@ import (
"strings"
"testing"
"llmsproxy/internal/config"
"llmsproxy/internal/core"
"modelrouter/internal/config"
"modelrouter/internal/core"
)
// adminGateway builds a gateway whose admin key can call /api/keys.

View File

@ -4,7 +4,7 @@ import (
"testing"
"time"
"llmsproxy/internal/config"
"modelrouter/internal/config"
)
// ---- data layer: per-key isolation ----

View File

@ -12,8 +12,8 @@ import (
"testing"
"time"
"llmsproxy/internal/config"
"llmsproxy/internal/core"
"modelrouter/internal/config"
"modelrouter/internal/core"
)
// quotaGateway builds a gateway with two user keys that differ only in their

View File

@ -6,8 +6,8 @@ import (
"net/http"
"strings"
"llmsproxy/internal/config"
"llmsproxy/internal/scheduler"
"modelrouter/internal/config"
"modelrouter/internal/scheduler"
)
// handleKeysAPI manages gateway keys: GET /api/keys (admin: all keys),

View File

@ -10,9 +10,9 @@ import (
"strings"
"time"
"llmsproxy/internal/provider"
"llmsproxy/internal/scheduler"
"llmsproxy/internal/types"
"modelrouter/internal/provider"
"modelrouter/internal/scheduler"
"modelrouter/internal/types"
)
// handleMessages serves the Anthropic Messages protocol (POST /v1/messages).

Some files were not shown because too many files have changed in this diff Show More