diff --git a/README.md b/README.md index 3a3d2e5..1150c9e 100644 --- a/README.md +++ b/README.md @@ -184,16 +184,24 @@ sources: ### AUTO 链(优先级页) -AUTO 调度**只**由优先级页保存的规则(持久化到 `runtime_file` 的 `auto` 字段)决定, -源配置里的 `models[].priority` 数字不再参与调度、也不再显示。 +AUTO 调度**只**由优先级页保存的规则决定,源配置里的 `models[].priority` 数字不再 +参与调度、也不再显示。**聊天与生图是两套独立的链**,在优先级页面通过「聊天 / 生图」 +按钮切换编辑: + +- **聊天链**(`runtime_file` 的 `auto` 字段):服务 `POST /v1/chat/completions` + 的 `model: AUTO` 请求;只接受 `kind: chat` 模型,配置在链里的生图槽位会被忽略。 +- **生图链**(`runtime_file` 的 `auto_image` 字段):服务 `POST /v1/images/generations` + 的 `model: AUTO` 请求;只接受 `kind: image` 模型。未配置生图链时,AUTO 生图回落 + 到「所有含生图模型的源、按注册顺序」的旧行为,兼容旧配置。 + +每条规则 = 一个「槽位」:`{ model, source, tier, token_quota, period, hours }`。 -- 每条规则 = 一个「槽位」:`{ model, source, tier, token_quota, period, hours }`。 - `tier` 表示优先级档位:同一档的模型并排、共享该优先级;档位从上到下递减。 - 同一模型可配置多个槽位(如 A 源低配 → B 源低配 → A 源高配 → B 源高配),按档位顺延。 - `token_quota` > 0 时该槽位在重置周期内用满即顺延到下一槽位;`period` 支持 `hour` / `week` / `month` / `nhour`(配合 `hours`),空 = 不限。 -- 生图模型(`kind: image`)不参与 AUTO 链;生图走 `POST /v1/images/generations` - 的独立路径。 +- 同 tier 内按「偏好分」排序,失败槽位冷却后自动跳过;冷却 / 配额耗尽 / hard + 错误会顺延到下一 tier,全部失败时返回 503 并附上每档失败摘要。 ### 源(Source)与适配器(Adapter)的关系 @@ -224,6 +232,23 @@ WebUI 上的"新增/编辑源"、"上传 Lua 适配器"、"改 AUTO 优先级链 `config.yaml` 中定义的基源无法改写配置文件,采用删除标记隐藏(重启后仍隐藏), 在 UI 里重新添加同名源即可恢复。 +### 源模板(多 key 负载均衡) + +多个上游 key 共享同一套配置(URL / 适配器 / 模型列表 / 并发 / rpm)时,逐一复制 +整段 source 太冗余。**源模板**保存除 `name` 和 `api_key` 以外的全部源字段, +从模板一键创建多个仅 key 不同的源,天然支持多 key 负载均衡(每个展开源独立调度、 +独立冷却、独立健康状态)。 + +- WebUI「源」页右上角「**模板管理**」按钮:列出全部模板,可编辑 / 删除 / 新建。 +- 添加源弹窗右上角两个按钮: + - **从模板创建** → 选模板 → 表单自动填充(name + key 仍需你填); + - **存为模板** → 把当前表单非 key/name 字段存为新模板。 +- 模板持久化在 `runtime_file` 的 `source_templates` 字段,与运行时源一样热管理, + 改完即时生效。模板本身不含密钥,可以安全共享 / 纳管。 + +模板不会自己变成源——它只是「配方」。真正承担流量的是从模板创建出来的、带 key +的具体源。 + ### disable_thinking 请求体带 `"disable_thinking": true` 时,网关透传给各适配器;DeepSeek 适配器将其 @@ -296,14 +321,23 @@ return { 一键启停、系统托盘常驻、开机自启、静默启动,内容区一比一内嵌完整 WebUI (状态/对话/密钥/优先级/源/适配器 六个页面,免登录)。服务器用户继续使用纯 Go 二进制。 +### 两种安装包:Headless 与 Desktop + +发布到 Release 的安装包分为两类,面向不同用户: + +- **Headless(服务器版)**:纯核心二进制(无 Electron),配置驱动,适合服务器、 + 容器、systemd 等无人值守场景。Linux 为 `.tar.gz`(二进制 + `config.example.yaml` + + `adapters/` + README),Windows 为 `.zip`。 +- **Desktop(桌面版)**:Electron GUI 安装包,内嵌核心,适合个人桌面日常使用。 + ### 选择 GUI 还是纯后端 | 场景 | 推荐 | 理由 | | ------ | ------ | ------ | -| 服务器 / 内网网关 / 无人值守常驻 | **纯后端**(单二进制) | 极轻量(~10MB、~15MB RSS),零依赖单进程,直接跑在 systemd/任意容器里,远程管理 | -| 个人桌面日常使用 / 多设备内网共享 | **GUI**(Electron) | 免登录内嵌 WebUI、系统托盘一键启停、开机自启、静默后台,适合不懂命令行的使用者 | -| Windows 桌面 | **GUI** | 纯后端在 Windows 上需自行注册服务,GUI 提供原生托盘/自启体验 | -| CI 一键出三平台安装包 | **GUI 打包脚本** | `make gui-*` 系列出 deb/AppImage/NSIS,可进发行流水线 | +| 服务器 / 内网网关 / 无人值守常驻 | **Headless**(单二进制) | 极轻量(~10MB、~15MB RSS),零依赖单进程,直接跑在 systemd/任意容器里,远程管理 | +| 个人桌面日常使用 / 多设备内网共享 | **Desktop**(GUI) | 免登录内嵌 WebUI、系统托盘一键启停、开机自启、静默后台,适合不懂命令行的使用者 | +| Windows 桌面 | **Desktop** | 纯后端在 Windows 上需自行注册服务,GUI 提供原生托盘/自启体验 | +| CI 一键出三平台安装包 | **Desktop 打包脚本** | `make gui-*` 系列出 deb/AppImage/NSIS,可进发行流水线 | 两者完全同源:GUI 内嵌的就是纯后端的同一份 `llmsproxy` 核心(LuaJIT 版), 配置/适配器格式完全一致,可随时互换。 @@ -321,8 +355,10 @@ make gui-win-docker # 构建 Windows nsis 安装包(全 docker 自足,无 make gui-dist-dir # 仅解包产物(调试用,不产安装包) ``` -产物在 `cmd/build/gui-dist/`:`ModelRouter-1.0.0.AppImage`、 -`modelrouter-gui_1.0.0_amd64.deb`、`ModelRouter Setup 1.0.0.exe`(win)。rpm 需系统 `rpmbuild`。 +产物在 `cmd/build/gui-dist/`:`ModelRouter-.AppImage`、 +`modelrouter-gui__amd64.deb`、`ModelRouter Setup .exe`(win)。rpm 需系统 `rpmbuild`。 +发布时会把产物重命名为 `ModelRouter-Desktop-*`(GUI)与 `ModelRouter-Headless-*`(纯后端) +两类上传到 GitCode Release。 #### Windows 打包(docker 固化方案,推荐) diff --git a/README_EN.md b/README_EN.md index c8f77f4..88faa89 100644 --- a/README_EN.md +++ b/README_EN.md @@ -176,12 +176,20 @@ tiered AUTO chain. ### AUTO chain (Priority page) -AUTO scheduling is driven **only** by the rules saved on the Priority page -(persisted as the `auto` field of the runtime file). The numeric -`models[].priority` in source configs no longer participates in scheduling and -is no longer shown. +AUTO scheduling is driven **only** by the rules saved on the Priority page. The +numeric `models[].priority` in source configs no longer participates in +scheduling and is no longer shown. **Chat and image generation are two +independent chains**, switched in the Priority page via the `Chat / Image` toggle: + +- **Chat chain** (`auto` field): serves `POST /v1/chat/completions` with + `model: AUTO`; only `kind: chat` models are honored (image slots are skipped). +- **Image chain** (`auto_image` field): serves `POST /v1/images/generations` with + `model: AUTO`; only `kind: image` models are honored. With no image chain + configured, AUTO image generation falls back to legacy discovery (all sources + exposing an image model, in registry order) for backward compatibility. + +Each rule is one "slot": `{ model, source, tier, token_quota, period, hours }`. -- Each rule is one "slot": `{ model, source, tier, token_quota, period, hours }`. - `tier` is the priority tier: models in the same tier sit side by side and share it; tiers run high → low. - The same model may appear in several slots (e.g. A low → B low → A high → @@ -189,8 +197,9 @@ is no longer shown. - `token_quota` > 0 means the slot is skipped once its tokens within the reset window are exhausted; `period` supports `hour` / `week` / `month` / `nhour` (with `hours`); empty = unlimited. -- Image models (`kind: image`) are kept out of the chain and are served by the - separate `POST /v1/images/generations` path. +- Within a tier, slots run by preference score; cooling / quota-exhausted / + hard-failed slots fall through to the next tier, and total failure returns + 503 with a per-tier summary. ### Sources vs. adapters @@ -227,6 +236,28 @@ sources and `.lua` files; base sources defined in `config.yaml` cannot rewrite that file, so they are hidden via a deletion tombstone (still hidden after restart) and can be restored by re-adding the same name in the UI. +### Source templates (multi-key balancing) + +When several upstream keys share one config (URL / adapter / model list / +concurrency / rpm), copying the whole source block N times is wasteful. A +**source template** stores every source field except `name` and `api_key`, so you +create N key-only-different sources from one shared skeleton — natural multi-key +load balancing (each expanded source schedules, cools down, and reports health +independently). + +- WebUI Sources page, top-right **Templates** button: list all templates with + edit / delete / create. +- Add-source dialog has two header buttons: + - **From template** → pick a template → form auto-fills (name + key still + yours to enter); + - **As template** → save the current form's non-key/name fields as a template. +- Templates persist in the runtime file's `source_templates` field alongside + runtime sources (hot-managed, apply immediately). Templates hold no secret + keys, so they're safe to share / version-control. + +A template is just a recipe — it doesn't become a source by itself. Only the +key-bearing sources created from it carry real traffic. + ### disable_thinking With `"disable_thinking": true` in the request body, the gateway passes it to @@ -308,14 +339,25 @@ tray, autostart, silent launch, and a pixel-for-pixel embedded full WebUI (Status/Chat/Keys/Priority/Sources/Adapters — no login needed). Server users keep using the plain Go binary. +### Two installer kinds: Headless and Desktop + +Release installers ship in two flavors for different audiences: + +- **Headless (server)**: the plain core binary (no Electron), config-driven, + for servers / containers / systemd / unattended runs. Linux ships as + `.tar.gz` (binary + `config.example.yaml` + `adapters/` + README), Windows as + `.zip`. +- **Desktop (desktop app)**: the Electron GUI installer with an embedded core, + for personal daily desktop use. + ### GUI vs. plain backend | Scenario | Pick | Why | | ---- | ---- | ---- | -| Server / intranet gateway / unattended long-running | **plain backend** (single binary) | ~10 MB, ~15 MB RSS, zero-dep single process — drop it into systemd or any container, remote admin | -| Personal desktop daily use / multi-device intranet sharing | **GUI** (Electron) | no-login embedded WebUI, tray one-click, autostart, silent background — for non-CLI users | -| Windows desktop | **GUI** | plain backend needs manual service registration; GUI ships native tray/autostart | -| CI one-shot 3-platform installers | **GUI packaging scripts** | `make gui-*` emits deb / AppImage / NSIS, drops straight into a release pipeline | +| Server / intranet gateway / unattended long-running | **Headless** (single binary) | ~10 MB, ~15 MB RSS, zero-dep single process — drop it into systemd or any container, remote admin | +| Personal desktop daily use / multi-device intranet sharing | **Desktop** (GUI) | no-login embedded WebUI, tray one-click, autostart, silent background — for non-CLI users | +| Windows desktop | **Desktop** | plain backend needs manual service registration; GUI ships native tray/autostart | +| CI one-shot 3-platform installers | **Desktop packaging scripts** | `make gui-*` emits deb / AppImage / NSIS, drops straight into a release pipeline | Both share the exact same `llmsproxy` core (LuaJIT build) — configs and adapters are fully compatible, freely interchangeable. @@ -332,9 +374,11 @@ make gui-win # Windows NSIS (host mingw + wine) make gui-win-docker # Windows NSIS, fully dockerized (no host mingw needed) ``` -Artifacts land in `cmd/build/gui-dist/`: `ModelRouter-1.0.0.AppImage`, -`modelrouter-gui_1.0.0_amd64.deb`, `ModelRouter Setup 1.0.0.exe` (win). rpm -needs system `rpmbuild`. +Artifacts land in `cmd/build/gui-dist/`: `ModelRouter-.AppImage`, +`modelrouter-gui__amd64.deb`, `ModelRouter Setup .exe` (win). rpm +needs system `rpmbuild`. At release time artifacts are renamed into the +`ModelRouter-Desktop-*` (GUI) and `ModelRouter-Headless-*` (plain backend) +families before upload to GitCode Releases. #### Windows packaging (dockerized, recommended)