fix(ui): clipboard fallback for non-secure contexts; touch-friendly delete button on key scope blocks; append named model after AUTO when adding (+ docs sync: encrypted storage, AUTO chain, live probing, image endpoint)

This commit is contained in:
root
2026-08-10 11:10:59 +08:00
parent 62c8a07b88
commit e2dd4d9727
4 changed files with 163 additions and 35 deletions

View File

@ -22,8 +22,9 @@ Groq、Mistral、Ollama、KimiCode…通过 **Lua 适配器** 做协议转换
## 特性 ## 特性
- **多源**:一个进程内配置任意多个上游源,按请求的 `model` 自动路由。 - **多源**:一个进程内配置任意多个上游源,按请求的 `model` 自动路由。
- **AUTO 模式**`default_model: AUTO` 时按各源模型的 `priority` 自动选最高可用源 - **AUTO 模式**`default_model: AUTO` 时按优先级页保存的 AUTO 链档位逐档调度并发请求
- **统一输出**:所有源对外都是 OpenAI 格式(含 `reasoning_content``tool_calls``usage`)。 - **统一输出**:所有源对外都是 OpenAI 格式(含 `reasoning_content``tool_calls``usage`)。
- **生图**`POST /v1/images/generations``kind: image` 的模型独立路由。
- **多模态**`content` 数组(`image_url`在多源间无损透传Anthropic/Gemini/Ollama 自动转换。 - **多模态**`content` 数组(`image_url`在多源间无损透传Anthropic/Gemini/Ollama 自动转换。
- **LuaJIT VM**:基于 golua 绑定的 LuaJIT每个适配器独立 VM + worker 池,安全并发。 - **LuaJIT VM**:基于 golua 绑定的 LuaJIT每个适配器独立 VM + worker 池,安全并发。
- **disable_thinking**:请求 `disable_thinking:true`(或上游对应字段)动态开关推理。 - **disable_thinking**:请求 `disable_thinking:true`(或上游对应字段)动态开关推理。
@ -32,7 +33,12 @@ Groq、Mistral、Ollama、KimiCode…通过 **Lua 适配器** 做协议转换
- **签名 / 请求头钩子**:适配器可定义 `build_headers(meta)`,在 Go 发 HTTP 前 - **签名 / 请求头钩子**:适配器可定义 `build_headers(meta)`,在 Go 发 HTTP 前
动态注入/签名请求头——用于云端 API 校验调用方 app如 KimiCode 只放行特定 动态注入/签名请求头——用于云端 API 校验调用方 app如 KimiCode 只放行特定
agent。提供 `hmac_sha256_hex` / `sha256_hex` / `base64_encode` 等签名辅助。 agent。提供 `hmac_sha256_hex` / `sha256_hex` / `base64_encode` 等签名辅助。
- **WebUI**:内置管理界面,可在线查看/新增/编辑上游源与模型,写入运行时文件持久化。 - **WebUI**:内置管理界面,可在线查看/新增/编辑上游源与模型、配置 AUTO 优先级链、
管理密钥模型范围,写入运行时文件持久化。
- **密钥加密存储**:运行时文件中的上游 `api_key`、自定义请求头值、网关 key 均以
AES-256-GCM 加密落盘(`master.key` 独立 0600`LLMS_PROXY_MASTER_KEY`)。
- **实时源探测**:状态页探测各源可达性(`GET {base}/models`,失败回退最小请求),
不污染正常调度的退避状态,错误信息在 UI 可悬停查看。
- **鉴权**:网关自身用 `gateway_keys` 校验客户端 Bearer key与上游各自的 key 相互独立。 - **鉴权**:网关自身用 `gateway_keys` 校验客户端 Bearer key与上游各自的 key 相互独立。
- **流式**SSE `chat.completion.chunk`,含角色首包与 `[DONE]` 收尾。 - **流式**SSE `chat.completion.chunk`,含角色首包与 `[DONE]` 收尾。
@ -78,12 +84,12 @@ runtime_file: runtime.json # WebUI 编辑的源持久化到此文件
sources: sources:
- name: deepseek - name: deepseek
base_url: https://api.deepseek.com base_url: https://api.deepseek.com
api_key: sk-... api_key: sk-... # 也可用 api_key_env: SOME_ENV 引用环境变量(不落盘明文)
adapter: deepseek adapter: deepseek
max_concurrent: 8 max_concurrent: 8
models: models:
- id: deepseek-v4-flash - id: deepseek-v4-flash
priority: 100 # 越大越优先被 AUTO 选中 priority: 100 # YAML 源首次启动会 seed 进 AUTO 链的初值
kind: chat kind: chat
- id: deepseek-v4-pro - id: deepseek-v4-pro
priority: 60 priority: 60
@ -97,17 +103,38 @@ sources:
timeout: 120s # 请求超时,默认 120s timeout: 120s # 请求超时,默认 120s
``` ```
运行时文件(`runtime_file`)中的敏感字段自动加密:
- 加密算法 AES-256-GCM格式 `enc:v1:<base64>`
- 主密钥来源:环境变量 `LLMS_PROXY_MASTER_KEY`64 位 hex否则读取
`runtime_file` 同目录的 `master.key`;都不存在时首次启动自动生成 `master.key`0600
- 升级时旧明文文件自动兼容:首次运行正常读取,任何 UI 保存操作触发全文件加密迁移。
- 注意:**master.key 丢失后密文无法解密**,请随配置一起备份;切勿提交到版本库。
### 模型路由 ### 模型路由
`/v1/chat/completions``model` 解析顺序: `/v1/chat/completions``model` 解析顺序:
1. `source/model``source:model` 前缀 → 指定源; 1. `source/model``source:model` 前缀 → 指定源;
2. 精确匹配某个源的 `model` 2. 精确匹配某个源的 `model`
3. 配置 `default_model: AUTO` 时 → 按各源模型的 `priority`(数字大优先)选最高可用源 3. 配置 `default_model: AUTO` 时 → 走优先级页保存的 AUTO 链(见下)
4. 否则回落`default_source` 4. 否则回落(返回错误)
任何 OpenAI 客户端,只要 `model` 设为某个源的 `name/任意名`,即可锁定走该源; 任何 OpenAI 客户端,只要 `model` 设为某个源的 `name/任意名`,即可锁定走该源;
设为 `AUTO`(或网关配了 `default_model: AUTO`)即自动按优先级选源。 设为 `AUTO`(或网关配了 `default_model: AUTO`)即自动按优先级选源。
### AUTO 链(优先级页)
AUTO 调度**只**由优先级页保存的规则(持久化到 `runtime_file``auto` 字段)决定,
源配置里的 `models[].priority` 数字不再参与调度、也不再显示。
- 每条规则 = 一个「槽位」:`{ model, source, tier, token_quota, period, hours }`
- `tier` 表示优先级档位:同一档的模型并排、共享该优先级;档位从上到下递减。
- 同一模型可配置多个槽位(如 A 源低配 → B 源低配 → A 源高配 → B 源高配),按档位顺延。
- `token_quota` > 0 时该槽位在重置周期内用满即顺延到下一槽位;`period` 支持
`hour` / `week` / `month` / `nhour`(配合 `hours`),空 = 不限。
- 生图模型(`kind: image`)不参与 AUTO 链;生图走 `POST /v1/images/generations`
的独立路径。
### 源Source与适配器Adapter的关系 ### 源Source与适配器Adapter的关系
@ -130,9 +157,13 @@ sources:
4. 请求进来时 `Registry``model` 路由到 ProviderProvider 调适配器 4. 请求进来时 `Registry``model` 路由到 ProviderProvider 调适配器
`transform_request` → HTTP 发送 → `transform_response` / `transform_stream_chunk` `transform_request` → HTTP 发送 → `transform_response` / `transform_stream_chunk`
WebUI 上的"新增/编辑源""上传 Lua 适配器"都即时生效(写入运行时文件或 WebUI 上的"新增/编辑源""上传 Lua 适配器"、"改 AUTO 优先级链"与"密钥模型范围"
`adapter_dir` 后重新装配,无需重启);直接编辑 `config.yaml` / `adapter_dir` 都即时生效(写入运行时文件或 `adapter_dir` 后重新装配,无需重启);直接编辑
下的文件则需要重启进程才会重新加载。 `config.yaml` / `adapter_dir` 下的文件则需要重启进程才会重新加载。
在 WebUI 删除源/适配器时运行时WebUI 创建)的源与 `.lua` 文件会真正移除;
`config.yaml` 中定义的基源无法改写配置文件,采用删除标记隐藏(重启后仍隐藏),
在 UI 里重新添加同名源即可恢复。
### disable_thinking ### disable_thinking
@ -141,7 +172,17 @@ WebUI 上的"新增/编辑源"与"上传 Lua 适配器"都即时生效(写入
### WebUI ### WebUI
内置管理界面(`GET /`),登录后可在浏览器查看/新增/编辑上游源与模型, 内置管理界面(`GET /`),登录后可在浏览器完成:
- **状态页**:源在线状态(实时探测 + 悬停看错误)、模型/源/key 用量统计、请求记录,
支持按时间范围导出 CSV点击模型可生成 pin 到该模型的连接配置。
- **对话页**:流式/非流式调试。
- **密钥页**:创建/编辑网关 key为每个 key 配模型范围(模型 + 源 + token 配额 + 周期),
管理员管理全部 key用户只看到自己的 key。
- **优先级页**:拖拽积木配置 AUTO 链档位。
- **源页**在线增删改上游源API key 等敏感字段加密落盘)。
- **适配器页**:上传 / 删除 Lua 适配器脚本。
改动写入 `runtime_file`(重启仍生效)。 改动写入 `runtime_file`(重启仍生效)。
## Lua 适配器协议 ## Lua 适配器协议

View File

@ -30,10 +30,12 @@ Extracted and independently evolved from the multi-source LLM adapter layer of
- **Multi-source**: any number of upstream sources in one process, routed by the - **Multi-source**: any number of upstream sources in one process, routed by the
request's `model`. request's `model`.
- **AUTO mode**: with `default_model: AUTO`, the highest-priority available - **AUTO mode**: with `default_model: AUTO`, scheduling follows the tiered AUTO
source model wins. chain saved on the Priority page (see below).
- **Unified output**: every source speaks OpenAI format (incl. - **Unified output**: every source speaks OpenAI format (incl.
`reasoning_content`, `tool_calls`, `usage`). `reasoning_content`, `tool_calls`, `usage`).
- **Image generation**: `POST /v1/images/generations`, routed to models with
`kind: image`.
- **Multimodal**: `content` arrays (`image_url` etc.) pass through losslessly; - **Multimodal**: `content` arrays (`image_url` etc.) pass through losslessly;
Anthropic/Gemini/Ollama are translated automatically. Anthropic/Gemini/Ollama are translated automatically.
- **LuaJIT VM**: golua-binding LuaJIT; each adapter has its own VM + worker - **LuaJIT VM**: golua-binding LuaJIT; each adapter has its own VM + worker
@ -46,8 +48,14 @@ Extracted and independently evolved from the multi-source LLM adapter layer of
inject/sign request headers before the HTTP call (e.g. KimiCode style inject/sign request headers before the HTTP call (e.g. KimiCode style
app-validation), with helpers like `hmac_sha256_hex`, `sha256_hex`, app-validation), with helpers like `hmac_sha256_hex`, `sha256_hex`,
`base64_encode`. `base64_encode`.
- **WebUI**: built-in management page to view/add/edit sources & models, - **WebUI**: built-in management page to view/add/edit sources & models, edit
persisted to a runtime file. the AUTO chain, and manage per-key model scopes, persisted to a runtime file.
- **Encrypted secrets at rest**: upstream `api_key`, custom header values and
gateway keys are stored AES-256-GCM encrypted (`master.key` 0600 next to the
runtime file, or `LLMS_PROXY_MASTER_KEY`).
- **Live source probing**: the status page probes each source
(`GET {base}/models`, falling back to a minimal chat call) without touching
the scheduler's backoff state; errors are shown on hover.
- **Auth**: the gateway validates client keys via `gateway_keys`; independent - **Auth**: the gateway validates client keys via `gateway_keys`; independent
from each upstream's own key. from each upstream's own key.
- **Streaming**: SSE `chat.completion.chunk` with a role-first chunk and - **Streaming**: SSE `chat.completion.chunk` with a role-first chunk and
@ -97,12 +105,12 @@ runtime_file: runtime.json # WebUI-edited sources persist here
sources: sources:
- name: deepseek - name: deepseek
base_url: https://api.deepseek.com base_url: https://api.deepseek.com
api_key: sk-... api_key: sk-... # or api_key_env: SOME_ENV to read from an env var
adapter: deepseek adapter: deepseek
max_concurrent: 8 max_concurrent: 8
models: models:
- id: deepseek-v4-flash - id: deepseek-v4-flash
priority: 100 # higher -> preferred by AUTO priority: 100 # YAML sources seed the AUTO chain on first start
kind: chat kind: chat
- id: deepseek-v4-pro - id: deepseek-v4-pro
priority: 60 priority: 60
@ -116,18 +124,48 @@ sources:
timeout: 120s # request timeout, default 120s timeout: 120s # request timeout, default 120s
``` ```
Sensitive fields in the runtime file (`runtime_file`) are encrypted at rest:
- AES-256-GCM, stored as `enc:v1:<base64>`.
- Master key source: env var `LLMS_PROXY_MASTER_KEY` (64 hex chars), else a
`master.key` file next to the runtime file; if neither exists it is generated
on first start (mode 0600).
- Old plaintext files still load; the first UI save migrates the whole file to
ciphertext.
- Back up `master.key` with your config — losing it makes the secrets
undecryptable. Do not commit it.
### Model routing ### Model routing
`/v1/chat/completions` `model` resolution order: `/v1/chat/completions` `model` resolution order:
1. `source/model` or `source:model` prefix → pinned source; 1. `source/model` or `source:model` prefix → pinned source;
2. exact match of a source's `model`; 2. exact match of a source's `model`;
3. with `default_model: AUTO`highest-`priority` healthy source model; 3. with `default_model: AUTO`follow the AUTO chain from the Priority page
4. otherwise fall back to `default_source`. (see below);
4. otherwise the request errors.
Any OpenAI client can pin to a source by setting `model` to its Any OpenAI client can pin to a source by setting `model` to its
`name/anything`; `AUTO` (or the gateway's `default_model: AUTO`) picks the `name/anything`; `AUTO` (or the gateway's `default_model: AUTO`) follows the
healthy source by priority. tiered AUTO chain.
### AUTO chain (Priority page)
AUTO scheduling is driven **only** by the rules saved on the Priority page
(persisted as the `auto` field of the runtime file). The numeric
`models[].priority` in source configs no longer participates in scheduling and
is no longer shown.
- Each rule is one "slot": `{ model, source, tier, token_quota, period, hours }`.
- `tier` is the priority tier: models in the same tier sit side by side and
share it; tiers run high → low.
- The same model may appear in several slots (e.g. A low → B low → A high →
B high) and is tried in tier order.
- `token_quota` > 0 means the slot is skipped once its tokens within the reset
window are exhausted; `period` supports `hour` / `week` / `month` / `nhour`
(with `hours`); empty = unlimited.
- Image models (`kind: image`) are kept out of the chain and are served by the
separate `POST /v1/images/generations` path.
### Sources vs. adapters ### Sources vs. adapters
@ -154,10 +192,15 @@ Loading flow (assembled by `Core` in `internal/core`):
adapter's `transform_request` → HTTP call → `transform_response` / adapter's `transform_request` → HTTP call → `transform_response` /
`transform_stream_chunk`. `transform_stream_chunk`.
"Add/Edit source" and "Upload adapter" in the WebUI take effect immediately "Add/Edit source", "Upload adapter", "AUTO chain edits" and "per-key model
(written to the runtime file / `adapter_dir`, then reassembled — no restart). scopes" in the WebUI take effect immediately (written to the runtime file /
Directly editing `config.yaml` or files under `adapter_dir` requires a process `adapter_dir`, then reassembled — no restart). Directly editing `config.yaml`
restart. or files under `adapter_dir` requires a process restart.
Deleting a source/adapter in the WebUI truly removes runtime (UI-created)
sources and `.lua` files; base sources defined in `config.yaml` cannot rewrite
that file, so they are hidden via a deletion tombstone (still hidden after
restart) and can be restored by re-adding the same name in the UI.
### disable_thinking ### disable_thinking
@ -167,8 +210,20 @@ each adapter; the DeepSeek adapter maps it to `extra_body.thinking.type =
### WebUI ### WebUI
Built-in admin page at `GET /`; after login you can view/edit sources and Built-in admin page at `GET /`; after login you can:
models in the browser, persisted to `runtime_file` (survives restarts).
- **Status**: source online state (live probe, error on hover), per
model/source/key usage stats, request records filtered by key, and CSV export
over a time range; click a model to pin the connection snippet to it.
- **Chat**: streaming / non-streaming debug.
- **Keys**: create/edit gateway keys, set per-key model scopes (model + source +
token quota + reset period); admins manage all keys, users see only their own.
- **Priority**: drag blocks to build the tiered AUTO chain.
- **Sources**: add/edit/delete upstream sources online (secrets stored
encrypted).
- **Adapters**: upload / delete Lua adapter scripts.
Changes persist to `runtime_file` (survive restarts).
## Lua adapter protocol ## Lua adapter protocol

View File

@ -7,7 +7,7 @@ listen: 127.0.0.1:8080
gateway_keys: gateway_keys:
- sk-gw-local-0001 - sk-gw-local-0001
# 默认模型选择:具体模型 id 或 AUTO按各源模型的 priority 自动选最高可用源 # 默认模型选择:具体模型 id 或 AUTOAUTO 走优先级页保存的 AUTO 链
default_model: AUTO default_model: AUTO
# Lua 适配器目录(默认 adapters/,首次启动自动写入内置适配器) # Lua 适配器目录(默认 adapters/,首次启动自动写入内置适配器)
@ -24,12 +24,12 @@ max_concurrent: 0
sources: sources:
- name: deepseek - name: deepseek
base_url: https://api.deepseek.com base_url: https://api.deepseek.com
api_key: sk-your-deepseek-key api_key: sk-your-deepseek-key # 或用 api_key_env: SOME_ENV 引用环境变量
adapter: deepseek adapter: deepseek
max_concurrent: 8 max_concurrent: 8
models: models:
- id: deepseek-v4-flash - id: deepseek-v4-flash
priority: 100 # 数字越大越优先被 AUTO 选中 priority: 100 # YAML 源首次启动 seed 进 AUTO 链的初值
kind: chat kind: chat
- id: deepseek-reasoner - id: deepseek-reasoner
priority: 60 priority: 60

View File

@ -172,6 +172,9 @@ td.t-tag { white-space:nowrap; }
border-radius:99px; white-space:nowrap; } border-radius:99px; white-space:nowrap; }
.mb .copy-b { cursor:pointer; opacity:.8; font-size:12px; padding:0 3px; } .mb .copy-b { cursor:pointer; opacity:.8; font-size:12px; padding:0 3px; }
.mb .copy-b:hover { opacity:1; } .mb .copy-b:hover { opacity:1; }
.mb .mb-x { cursor:pointer; font-size:15px; line-height:1; padding:4px 5px; margin:0 2px; border-radius:6px;
color:rgba(255,255,255,.85); background:rgba(0,0,0,.18); user-select:none; }
.mb .mb-x:hover { background:rgba(220,53,69,.85); color:#fff; }
.mb-ghost { position:fixed; pointer-events:none; opacity:.85; z-index:60; transform:rotate(2deg); } .mb-ghost { position:fixed; pointer-events:none; opacity:.85; z-index:60; transform:rotate(2deg); }
.mb-empty { color:var(--muted); font-size:12.5px; } .mb-empty { color:var(--muted); font-size:12.5px; }
.add-brick { display:inline-flex; align-items:center; gap:6px; padding:9px 16px; border-radius:12px; .add-brick { display:inline-flex; align-items:center; gap:6px; padding:9px 16px; border-radius:12px;
@ -582,9 +585,29 @@ function toast(m) { const el = $('#toast'); el.textContent = m; el.style.display
function esc(s) { return String(s ?? '').replace(/[&<>"']/g, c => ({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;',"'":'&#39;'}[c])); } function esc(s) { return String(s ?? '').replace(/[&<>"']/g, c => ({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;',"'":'&#39;'}[c])); }
function escAttr(s) { return esc(s).replace(/"/g, '&quot;'); } function escAttr(s) { return esc(s).replace(/"/g, '&quot;'); }
function copyText(txt, okMsg) { function copyText(txt, okMsg) {
if (navigator.clipboard && navigator.clipboard.writeText) { const done = () => toast(okMsg || t('toastCopied'));
navigator.clipboard.writeText(txt).then(() => toast(okMsg || t('toastCopied')), () => toast(t('toastCopyFail'))); const fail = () => toast(t('toastCopyFail'));
} else { toast(t('toastCopyFail')); } if (navigator.clipboard && navigator.clipboard.writeText && window.isSecureContext) {
navigator.clipboard.writeText(txt).then(done, () => legacyCopy(txt) ? done() : fail());
return;
}
legacyCopy(txt) ? done() : fail();
}
function legacyCopy(txt) {
try {
const ta = document.createElement('textarea');
ta.value = txt;
ta.setAttribute('readonly', '');
ta.style.position = 'fixed';
ta.style.opacity = '0';
ta.style.pointerEvents = 'none';
document.body.appendChild(ta);
ta.select();
ta.setSelectionRange(0, ta.value.length);
const ok = document.execCommand('copy');
document.body.removeChild(ta);
return ok;
} catch (e) { return false; }
} }
/* ---------- status tab ---------- */ /* ---------- status tab ---------- */
@ -1779,6 +1802,7 @@ function scopeHtml(key, m) {
<span class="mb-name">${esc(m.model)}${src ? `<em class="mb-src">${esc(src)}</em>` : ''}</span> <span class="mb-name">${esc(m.model)}${src ? `<em class="mb-src">${esc(src)}</em>` : ''}</span>
<span class="mb-quota">${esc(quantBadge(m.token_quota, m.period, m.hours))}</span> <span class="mb-quota">${esc(quantBadge(m.token_quota, m.period, m.hours))}</span>
<span class="copy-b" title="${escAttr(t('kCopyB'))}" onclick="event.stopPropagation();scopeDup('${escAttr(key)}','${escAttr(comb)}')">⧉</span> <span class="copy-b" title="${escAttr(t('kCopyB'))}" onclick="event.stopPropagation();scopeDup('${escAttr(key)}','${escAttr(comb)}')">⧉</span>
<span class="mb-x" title="${escAttr(t('kDelB'))}" onclick="event.stopPropagation();scopeRm('${escAttr(key)}','${escAttr(comb)}')">×</span>
</span>`; </span>`;
} }
function normSrc(s) { return (!s || s === 'undefined' || s === 'null') ? '' : s; } function normSrc(s) { return (!s || s === 'undefined' || s === 'null') ? '' : s; }
@ -1824,10 +1848,18 @@ async function scopePush(key) {
const canvas = document.querySelector(`.key-canvas[data-key="${CSS.escape(key)}"]`); const canvas = document.querySelector(`.key-canvas[data-key="${CSS.escape(key)}"]`);
if (!canvas) return; if (!canvas) return;
const scopes = readScopes(canvas); const scopes = readScopes(canvas);
if (!scopes.some(s => s.model === 'AUTO')) scopes.push({ model: 'AUTO', token_quota: 0 }); let added;
if (!scopes.some(s => s.model === 'AUTO')) {
added = { model: 'AUTO', token_quota: 0 };
scopes.push(added);
} else {
const pick = (allModels || []).find(m => m.id && m.id !== 'AUTO' && !scopes.some(s => s.model === m.id && normSrc(s.source) === m.src));
added = pick ? { model: pick.id, source: pick.src, token_quota: 0 } : { model: 'AUTO', token_quota: 0 };
scopes.push(added);
}
try { await putScope(key, scopes); } catch (e) { toast(e.message); return; } try { await putScope(key, scopes); } catch (e) { toast(e.message); return; }
await loadKeys(); await loadKeys();
scopeEdit(key, 'AUTO'); scopeEdit(key, scopeComb(added));
} }
async function scopeDup(key, comb) { async function scopeDup(key, comb) {
const canvas = document.querySelector(`.key-canvas[data-key="${CSS.escape(key)}"]`); const canvas = document.querySelector(`.key-canvas[data-key="${CSS.escape(key)}"]`);