mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-09-20 00:48:00 +00:00
fix(ui): clipboard fallback for non-secure contexts; touch-friendly delete button on key scope blocks; append named model after AUTO when adding (+ docs sync: encrypted storage, AUTO chain, live probing, image endpoint)
This commit is contained in:
63
README.md
63
README.md
@ -22,8 +22,9 @@ Groq、Mistral、Ollama、KimiCode…)通过 **Lua 适配器** 做协议转换
|
||||
## 特性
|
||||
|
||||
- **多源**:一个进程内配置任意多个上游源,按请求的 `model` 自动路由。
|
||||
- **AUTO 模式**:`default_model: AUTO` 时按各源模型的 `priority` 自动选最高可用源。
|
||||
- **AUTO 模式**:`default_model: AUTO` 时按优先级页保存的 AUTO 链档位逐档调度并发请求。
|
||||
- **统一输出**:所有源对外都是 OpenAI 格式(含 `reasoning_content`、`tool_calls`、`usage`)。
|
||||
- **生图**:`POST /v1/images/generations`,`kind: image` 的模型独立路由。
|
||||
- **多模态**:`content` 数组(`image_url` 等)在多源间无损透传,Anthropic/Gemini/Ollama 自动转换。
|
||||
- **LuaJIT VM**:基于 golua 绑定的 LuaJIT,每个适配器独立 VM + worker 池,安全并发。
|
||||
- **disable_thinking**:请求 `disable_thinking:true`(或上游对应字段)动态开关推理。
|
||||
@ -32,7 +33,12 @@ Groq、Mistral、Ollama、KimiCode…)通过 **Lua 适配器** 做协议转换
|
||||
- **签名 / 请求头钩子**:适配器可定义 `build_headers(meta)`,在 Go 发 HTTP 前
|
||||
动态注入/签名请求头——用于云端 API 校验调用方 app(如 KimiCode 只放行特定
|
||||
agent)。提供 `hmac_sha256_hex` / `sha256_hex` / `base64_encode` 等签名辅助。
|
||||
- **WebUI**:内置管理界面,可在线查看/新增/编辑上游源与模型,写入运行时文件持久化。
|
||||
- **WebUI**:内置管理界面,可在线查看/新增/编辑上游源与模型、配置 AUTO 优先级链、
|
||||
管理密钥模型范围,写入运行时文件持久化。
|
||||
- **密钥加密存储**:运行时文件中的上游 `api_key`、自定义请求头值、网关 key 均以
|
||||
AES-256-GCM 加密落盘(`master.key` 独立 0600,或 `LLMS_PROXY_MASTER_KEY`)。
|
||||
- **实时源探测**:状态页探测各源可达性(`GET {base}/models`,失败回退最小请求),
|
||||
不污染正常调度的退避状态,错误信息在 UI 可悬停查看。
|
||||
- **鉴权**:网关自身用 `gateway_keys` 校验客户端 Bearer key;与上游各自的 key 相互独立。
|
||||
- **流式**:SSE `chat.completion.chunk`,含角色首包与 `[DONE]` 收尾。
|
||||
|
||||
@ -78,12 +84,12 @@ runtime_file: runtime.json # WebUI 编辑的源持久化到此文件
|
||||
sources:
|
||||
- name: deepseek
|
||||
base_url: https://api.deepseek.com
|
||||
api_key: sk-...
|
||||
api_key: sk-... # 也可用 api_key_env: SOME_ENV 引用环境变量(不落盘明文)
|
||||
adapter: deepseek
|
||||
max_concurrent: 8
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
priority: 100 # 越大越优先被 AUTO 选中
|
||||
priority: 100 # YAML 源首次启动会 seed 进 AUTO 链的初值
|
||||
kind: chat
|
||||
- id: deepseek-v4-pro
|
||||
priority: 60
|
||||
@ -97,17 +103,38 @@ sources:
|
||||
timeout: 120s # 请求超时,默认 120s
|
||||
```
|
||||
|
||||
运行时文件(`runtime_file`)中的敏感字段自动加密:
|
||||
|
||||
- 加密算法 AES-256-GCM,格式 `enc:v1:<base64>`。
|
||||
- 主密钥来源:环境变量 `LLMS_PROXY_MASTER_KEY`(64 位 hex);否则读取
|
||||
`runtime_file` 同目录的 `master.key`;都不存在时首次启动自动生成 `master.key`(0600)。
|
||||
- 升级时旧明文文件自动兼容:首次运行正常读取,任何 UI 保存操作触发全文件加密迁移。
|
||||
- 注意:**master.key 丢失后密文无法解密**,请随配置一起备份;切勿提交到版本库。
|
||||
|
||||
### 模型路由
|
||||
|
||||
`/v1/chat/completions` 的 `model` 解析顺序:
|
||||
|
||||
1. `source/model` 或 `source:model` 前缀 → 指定源;
|
||||
2. 精确匹配某个源的 `model`;
|
||||
3. 配置 `default_model: AUTO` 时 → 按各源模型的 `priority`(数字大优先)选最高可用源;
|
||||
4. 否则回落到 `default_source`。
|
||||
3. 配置 `default_model: AUTO` 时 → 走优先级页保存的 AUTO 链(见下);
|
||||
4. 否则回落(返回错误)。
|
||||
|
||||
任何 OpenAI 客户端,只要 `model` 设为某个源的 `name/任意名`,即可锁定走该源;
|
||||
设为 `AUTO`(或网关配了 `default_model: AUTO`)即自动按优先级选源。
|
||||
设为 `AUTO`(或网关配了 `default_model: AUTO`)即自动按优先级链选源。
|
||||
|
||||
### AUTO 链(优先级页)
|
||||
|
||||
AUTO 调度**只**由优先级页保存的规则(持久化到 `runtime_file` 的 `auto` 字段)决定,
|
||||
源配置里的 `models[].priority` 数字不再参与调度、也不再显示。
|
||||
|
||||
- 每条规则 = 一个「槽位」:`{ model, source, tier, token_quota, period, hours }`。
|
||||
- `tier` 表示优先级档位:同一档的模型并排、共享该优先级;档位从上到下递减。
|
||||
- 同一模型可配置多个槽位(如 A 源低配 → B 源低配 → A 源高配 → B 源高配),按档位顺延。
|
||||
- `token_quota` > 0 时该槽位在重置周期内用满即顺延到下一槽位;`period` 支持
|
||||
`hour` / `week` / `month` / `nhour`(配合 `hours`),空 = 不限。
|
||||
- 生图模型(`kind: image`)不参与 AUTO 链;生图走 `POST /v1/images/generations`
|
||||
的独立路径。
|
||||
|
||||
### 源(Source)与适配器(Adapter)的关系
|
||||
|
||||
@ -130,9 +157,13 @@ sources:
|
||||
4. 请求进来时 `Registry` 按 `model` 路由到 Provider,Provider 调适配器
|
||||
`transform_request` → HTTP 发送 → `transform_response` / `transform_stream_chunk`。
|
||||
|
||||
WebUI 上的"新增/编辑源"与"上传 Lua 适配器"都即时生效(写入运行时文件或
|
||||
`adapter_dir` 后重新装配,无需重启);直接编辑 `config.yaml` / `adapter_dir`
|
||||
下的文件则需要重启进程才会重新加载。
|
||||
WebUI 上的"新增/编辑源"、"上传 Lua 适配器"、"改 AUTO 优先级链"与"密钥模型范围"
|
||||
都即时生效(写入运行时文件或 `adapter_dir` 后重新装配,无需重启);直接编辑
|
||||
`config.yaml` / `adapter_dir` 下的文件则需要重启进程才会重新加载。
|
||||
|
||||
在 WebUI 删除源/适配器时:运行时(WebUI 创建)的源与 `.lua` 文件会真正移除;
|
||||
`config.yaml` 中定义的基源无法改写配置文件,采用删除标记隐藏(重启后仍隐藏),
|
||||
在 UI 里重新添加同名源即可恢复。
|
||||
|
||||
### disable_thinking
|
||||
|
||||
@ -141,7 +172,17 @@ WebUI 上的"新增/编辑源"与"上传 Lua 适配器"都即时生效(写入
|
||||
|
||||
### WebUI
|
||||
|
||||
内置管理界面(`GET /`),登录后可在浏览器查看/新增/编辑上游源与模型,
|
||||
内置管理界面(`GET /`),登录后可在浏览器完成:
|
||||
|
||||
- **状态页**:源在线状态(实时探测 + 悬停看错误)、模型/源/key 用量统计、请求记录,
|
||||
支持按时间范围导出 CSV;点击模型可生成 pin 到该模型的连接配置。
|
||||
- **对话页**:流式/非流式调试。
|
||||
- **密钥页**:创建/编辑网关 key,为每个 key 配模型范围(模型 + 源 + token 配额 + 周期),
|
||||
管理员管理全部 key,用户只看到自己的 key。
|
||||
- **优先级页**:拖拽积木配置 AUTO 链档位。
|
||||
- **源页**:在线增删改上游源(API key 等敏感字段加密落盘)。
|
||||
- **适配器页**:上传 / 删除 Lua 适配器脚本。
|
||||
|
||||
改动写入 `runtime_file`(重启仍生效)。
|
||||
|
||||
## Lua 适配器协议
|
||||
|
||||
87
README_EN.md
87
README_EN.md
@ -30,10 +30,12 @@ Extracted and independently evolved from the multi-source LLM adapter layer of
|
||||
|
||||
- **Multi-source**: any number of upstream sources in one process, routed by the
|
||||
request's `model`.
|
||||
- **AUTO mode**: with `default_model: AUTO`, the highest-priority available
|
||||
source model wins.
|
||||
- **AUTO mode**: with `default_model: AUTO`, scheduling follows the tiered AUTO
|
||||
chain saved on the Priority page (see below).
|
||||
- **Unified output**: every source speaks OpenAI format (incl.
|
||||
`reasoning_content`, `tool_calls`, `usage`).
|
||||
- **Image generation**: `POST /v1/images/generations`, routed to models with
|
||||
`kind: image`.
|
||||
- **Multimodal**: `content` arrays (`image_url` etc.) pass through losslessly;
|
||||
Anthropic/Gemini/Ollama are translated automatically.
|
||||
- **LuaJIT VM**: golua-binding LuaJIT; each adapter has its own VM + worker
|
||||
@ -46,8 +48,14 @@ Extracted and independently evolved from the multi-source LLM adapter layer of
|
||||
inject/sign request headers before the HTTP call (e.g. KimiCode style
|
||||
app-validation), with helpers like `hmac_sha256_hex`, `sha256_hex`,
|
||||
`base64_encode`.
|
||||
- **WebUI**: built-in management page to view/add/edit sources & models,
|
||||
persisted to a runtime file.
|
||||
- **WebUI**: built-in management page to view/add/edit sources & models, edit
|
||||
the AUTO chain, and manage per-key model scopes, persisted to a runtime file.
|
||||
- **Encrypted secrets at rest**: upstream `api_key`, custom header values and
|
||||
gateway keys are stored AES-256-GCM encrypted (`master.key` 0600 next to the
|
||||
runtime file, or `LLMS_PROXY_MASTER_KEY`).
|
||||
- **Live source probing**: the status page probes each source
|
||||
(`GET {base}/models`, falling back to a minimal chat call) without touching
|
||||
the scheduler's backoff state; errors are shown on hover.
|
||||
- **Auth**: the gateway validates client keys via `gateway_keys`; independent
|
||||
from each upstream's own key.
|
||||
- **Streaming**: SSE `chat.completion.chunk` with a role-first chunk and
|
||||
@ -97,12 +105,12 @@ runtime_file: runtime.json # WebUI-edited sources persist here
|
||||
sources:
|
||||
- name: deepseek
|
||||
base_url: https://api.deepseek.com
|
||||
api_key: sk-...
|
||||
api_key: sk-... # or api_key_env: SOME_ENV to read from an env var
|
||||
adapter: deepseek
|
||||
max_concurrent: 8
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
priority: 100 # higher -> preferred by AUTO
|
||||
priority: 100 # YAML sources seed the AUTO chain on first start
|
||||
kind: chat
|
||||
- id: deepseek-v4-pro
|
||||
priority: 60
|
||||
@ -116,18 +124,48 @@ sources:
|
||||
timeout: 120s # request timeout, default 120s
|
||||
```
|
||||
|
||||
Sensitive fields in the runtime file (`runtime_file`) are encrypted at rest:
|
||||
|
||||
- AES-256-GCM, stored as `enc:v1:<base64>`.
|
||||
- Master key source: env var `LLMS_PROXY_MASTER_KEY` (64 hex chars), else a
|
||||
`master.key` file next to the runtime file; if neither exists it is generated
|
||||
on first start (mode 0600).
|
||||
- Old plaintext files still load; the first UI save migrates the whole file to
|
||||
ciphertext.
|
||||
- Back up `master.key` with your config — losing it makes the secrets
|
||||
undecryptable. Do not commit it.
|
||||
|
||||
### Model routing
|
||||
|
||||
`/v1/chat/completions` `model` resolution order:
|
||||
|
||||
1. `source/model` or `source:model` prefix → pinned source;
|
||||
2. exact match of a source's `model`;
|
||||
3. with `default_model: AUTO` → highest-`priority` healthy source model;
|
||||
4. otherwise fall back to `default_source`.
|
||||
3. with `default_model: AUTO` → follow the AUTO chain from the Priority page
|
||||
(see below);
|
||||
4. otherwise the request errors.
|
||||
|
||||
Any OpenAI client can pin to a source by setting `model` to its
|
||||
`name/anything`; `AUTO` (or the gateway's `default_model: AUTO`) picks the
|
||||
healthy source by priority.
|
||||
`name/anything`; `AUTO` (or the gateway's `default_model: AUTO`) follows the
|
||||
tiered AUTO chain.
|
||||
|
||||
### AUTO chain (Priority page)
|
||||
|
||||
AUTO scheduling is driven **only** by the rules saved on the Priority page
|
||||
(persisted as the `auto` field of the runtime file). The numeric
|
||||
`models[].priority` in source configs no longer participates in scheduling and
|
||||
is no longer shown.
|
||||
|
||||
- Each rule is one "slot": `{ model, source, tier, token_quota, period, hours }`.
|
||||
- `tier` is the priority tier: models in the same tier sit side by side and
|
||||
share it; tiers run high → low.
|
||||
- The same model may appear in several slots (e.g. A low → B low → A high →
|
||||
B high) and is tried in tier order.
|
||||
- `token_quota` > 0 means the slot is skipped once its tokens within the reset
|
||||
window are exhausted; `period` supports `hour` / `week` / `month` / `nhour`
|
||||
(with `hours`); empty = unlimited.
|
||||
- Image models (`kind: image`) are kept out of the chain and are served by the
|
||||
separate `POST /v1/images/generations` path.
|
||||
|
||||
### Sources vs. adapters
|
||||
|
||||
@ -154,10 +192,15 @@ Loading flow (assembled by `Core` in `internal/core`):
|
||||
adapter's `transform_request` → HTTP call → `transform_response` /
|
||||
`transform_stream_chunk`.
|
||||
|
||||
"Add/Edit source" and "Upload adapter" in the WebUI take effect immediately
|
||||
(written to the runtime file / `adapter_dir`, then reassembled — no restart).
|
||||
Directly editing `config.yaml` or files under `adapter_dir` requires a process
|
||||
restart.
|
||||
"Add/Edit source", "Upload adapter", "AUTO chain edits" and "per-key model
|
||||
scopes" in the WebUI take effect immediately (written to the runtime file /
|
||||
`adapter_dir`, then reassembled — no restart). Directly editing `config.yaml`
|
||||
or files under `adapter_dir` requires a process restart.
|
||||
|
||||
Deleting a source/adapter in the WebUI truly removes runtime (UI-created)
|
||||
sources and `.lua` files; base sources defined in `config.yaml` cannot rewrite
|
||||
that file, so they are hidden via a deletion tombstone (still hidden after
|
||||
restart) and can be restored by re-adding the same name in the UI.
|
||||
|
||||
### disable_thinking
|
||||
|
||||
@ -167,8 +210,20 @@ each adapter; the DeepSeek adapter maps it to `extra_body.thinking.type =
|
||||
|
||||
### WebUI
|
||||
|
||||
Built-in admin page at `GET /`; after login you can view/edit sources and
|
||||
models in the browser, persisted to `runtime_file` (survives restarts).
|
||||
Built-in admin page at `GET /`; after login you can:
|
||||
|
||||
- **Status**: source online state (live probe, error on hover), per
|
||||
model/source/key usage stats, request records filtered by key, and CSV export
|
||||
over a time range; click a model to pin the connection snippet to it.
|
||||
- **Chat**: streaming / non-streaming debug.
|
||||
- **Keys**: create/edit gateway keys, set per-key model scopes (model + source +
|
||||
token quota + reset period); admins manage all keys, users see only their own.
|
||||
- **Priority**: drag blocks to build the tiered AUTO chain.
|
||||
- **Sources**: add/edit/delete upstream sources online (secrets stored
|
||||
encrypted).
|
||||
- **Adapters**: upload / delete Lua adapter scripts.
|
||||
|
||||
Changes persist to `runtime_file` (survive restarts).
|
||||
|
||||
## Lua adapter protocol
|
||||
|
||||
|
||||
@ -7,7 +7,7 @@ listen: 127.0.0.1:8080
|
||||
gateway_keys:
|
||||
- sk-gw-local-0001
|
||||
|
||||
# 默认模型选择:具体模型 id 或 AUTO(按各源模型的 priority 自动选最高可用源)
|
||||
# 默认模型选择:具体模型 id 或 AUTO(AUTO 走优先级页保存的 AUTO 链)
|
||||
default_model: AUTO
|
||||
|
||||
# Lua 适配器目录(默认 adapters/,首次启动自动写入内置适配器)
|
||||
@ -24,12 +24,12 @@ max_concurrent: 0
|
||||
sources:
|
||||
- name: deepseek
|
||||
base_url: https://api.deepseek.com
|
||||
api_key: sk-your-deepseek-key
|
||||
api_key: sk-your-deepseek-key # 或用 api_key_env: SOME_ENV 引用环境变量
|
||||
adapter: deepseek
|
||||
max_concurrent: 8
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
priority: 100 # 数字越大越优先被 AUTO 选中
|
||||
priority: 100 # YAML 源首次启动 seed 进 AUTO 链的初值
|
||||
kind: chat
|
||||
- id: deepseek-reasoner
|
||||
priority: 60
|
||||
|
||||
@ -172,6 +172,9 @@ td.t-tag { white-space:nowrap; }
|
||||
border-radius:99px; white-space:nowrap; }
|
||||
.mb .copy-b { cursor:pointer; opacity:.8; font-size:12px; padding:0 3px; }
|
||||
.mb .copy-b:hover { opacity:1; }
|
||||
.mb .mb-x { cursor:pointer; font-size:15px; line-height:1; padding:4px 5px; margin:0 2px; border-radius:6px;
|
||||
color:rgba(255,255,255,.85); background:rgba(0,0,0,.18); user-select:none; }
|
||||
.mb .mb-x:hover { background:rgba(220,53,69,.85); color:#fff; }
|
||||
.mb-ghost { position:fixed; pointer-events:none; opacity:.85; z-index:60; transform:rotate(2deg); }
|
||||
.mb-empty { color:var(--muted); font-size:12.5px; }
|
||||
.add-brick { display:inline-flex; align-items:center; gap:6px; padding:9px 16px; border-radius:12px;
|
||||
@ -582,9 +585,29 @@ function toast(m) { const el = $('#toast'); el.textContent = m; el.style.display
|
||||
function esc(s) { return String(s ?? '').replace(/[&<>"']/g, c => ({'&':'&','<':'<','>':'>','"':'"',"'":'''}[c])); }
|
||||
function escAttr(s) { return esc(s).replace(/"/g, '"'); }
|
||||
function copyText(txt, okMsg) {
|
||||
if (navigator.clipboard && navigator.clipboard.writeText) {
|
||||
navigator.clipboard.writeText(txt).then(() => toast(okMsg || t('toastCopied')), () => toast(t('toastCopyFail')));
|
||||
} else { toast(t('toastCopyFail')); }
|
||||
const done = () => toast(okMsg || t('toastCopied'));
|
||||
const fail = () => toast(t('toastCopyFail'));
|
||||
if (navigator.clipboard && navigator.clipboard.writeText && window.isSecureContext) {
|
||||
navigator.clipboard.writeText(txt).then(done, () => legacyCopy(txt) ? done() : fail());
|
||||
return;
|
||||
}
|
||||
legacyCopy(txt) ? done() : fail();
|
||||
}
|
||||
function legacyCopy(txt) {
|
||||
try {
|
||||
const ta = document.createElement('textarea');
|
||||
ta.value = txt;
|
||||
ta.setAttribute('readonly', '');
|
||||
ta.style.position = 'fixed';
|
||||
ta.style.opacity = '0';
|
||||
ta.style.pointerEvents = 'none';
|
||||
document.body.appendChild(ta);
|
||||
ta.select();
|
||||
ta.setSelectionRange(0, ta.value.length);
|
||||
const ok = document.execCommand('copy');
|
||||
document.body.removeChild(ta);
|
||||
return ok;
|
||||
} catch (e) { return false; }
|
||||
}
|
||||
|
||||
/* ---------- status tab ---------- */
|
||||
@ -1779,6 +1802,7 @@ function scopeHtml(key, m) {
|
||||
<span class="mb-name">${esc(m.model)}${src ? `<em class="mb-src">${esc(src)}</em>` : ''}</span>
|
||||
<span class="mb-quota">${esc(quantBadge(m.token_quota, m.period, m.hours))}</span>
|
||||
<span class="copy-b" title="${escAttr(t('kCopyB'))}" onclick="event.stopPropagation();scopeDup('${escAttr(key)}','${escAttr(comb)}')">⧉</span>
|
||||
<span class="mb-x" title="${escAttr(t('kDelB'))}" onclick="event.stopPropagation();scopeRm('${escAttr(key)}','${escAttr(comb)}')">×</span>
|
||||
</span>`;
|
||||
}
|
||||
function normSrc(s) { return (!s || s === 'undefined' || s === 'null') ? '' : s; }
|
||||
@ -1824,10 +1848,18 @@ async function scopePush(key) {
|
||||
const canvas = document.querySelector(`.key-canvas[data-key="${CSS.escape(key)}"]`);
|
||||
if (!canvas) return;
|
||||
const scopes = readScopes(canvas);
|
||||
if (!scopes.some(s => s.model === 'AUTO')) scopes.push({ model: 'AUTO', token_quota: 0 });
|
||||
let added;
|
||||
if (!scopes.some(s => s.model === 'AUTO')) {
|
||||
added = { model: 'AUTO', token_quota: 0 };
|
||||
scopes.push(added);
|
||||
} else {
|
||||
const pick = (allModels || []).find(m => m.id && m.id !== 'AUTO' && !scopes.some(s => s.model === m.id && normSrc(s.source) === m.src));
|
||||
added = pick ? { model: pick.id, source: pick.src, token_quota: 0 } : { model: 'AUTO', token_quota: 0 };
|
||||
scopes.push(added);
|
||||
}
|
||||
try { await putScope(key, scopes); } catch (e) { toast(e.message); return; }
|
||||
await loadKeys();
|
||||
scopeEdit(key, 'AUTO');
|
||||
scopeEdit(key, scopeComb(added));
|
||||
}
|
||||
async function scopeDup(key, comb) {
|
||||
const canvas = document.querySelector(`.key-canvas[data-key="${CSS.escape(key)}"]`);
|
||||
|
||||
Reference in New Issue
Block a user