diff --git a/README.md b/README.md index d9c5178..e25a359 100644 --- a/README.md +++ b/README.md @@ -22,8 +22,9 @@ Groq、Mistral、Ollama、KimiCode…）通过 **Lua 适配器** 做协议转换 ## 特性 - **多源**：一个进程内配置任意多个上游源，按请求的 `model` 自动路由。 -- **AUTO 模式**：`default_model: AUTO` 时按各源模型的 `priority` 自动选最高可用源。 +- **AUTO 模式**：`default_model: AUTO` 时按优先级页保存的 AUTO 链档位逐档调度并发请求。 - **统一输出**：所有源对外都是 OpenAI 格式（含 `reasoning_content`、`tool_calls`、`usage`）。 +- **生图**：`POST /v1/images/generations`，`kind: image` 的模型独立路由。 - **多模态**：`content` 数组（`image_url` 等）在多源间无损透传，Anthropic/Gemini/Ollama 自动转换。 - **LuaJIT VM**：基于 golua 绑定的 LuaJIT，每个适配器独立 VM + worker 池，安全并发。 - **disable_thinking**：请求 `disable_thinking:true`（或上游对应字段）动态开关推理。 @@ -32,7 +33,12 @@ Groq、Mistral、Ollama、KimiCode…）通过 **Lua 适配器** 做协议转换 - **签名 / 请求头钩子**：适配器可定义 `build_headers(meta)`，在 Go 发 HTTP 前 动态注入/签名请求头——用于云端 API 校验调用方 app（如 KimiCode 只放行特定 agent）。提供 `hmac_sha256_hex` / `sha256_hex` / `base64_encode` 等签名辅助。 -- **WebUI**：内置管理界面，可在线查看/新增/编辑上游源与模型，写入运行时文件持久化。 +- **WebUI**：内置管理界面，可在线查看/新增/编辑上游源与模型、配置 AUTO 优先级链、 + 管理密钥模型范围，写入运行时文件持久化。 +- **密钥加密存储**：运行时文件中的上游 `api_key`、自定义请求头值、网关 key 均以 + AES-256-GCM 加密落盘（`master.key` 独立 0600，或 `LLMS_PROXY_MASTER_KEY`）。 +- **实时源探测**：状态页探测各源可达性（`GET {base}/models`，失败回退最小请求）， + 不污染正常调度的退避状态，错误信息在 UI 可悬停查看。 - **鉴权**：网关自身用 `gateway_keys` 校验客户端 Bearer key；与上游各自的 key 相互独立。 - **流式**：SSE `chat.completion.chunk`，含角色首包与 `[DONE]` 收尾。 @@ -78,12 +84,12 @@ runtime_file: runtime.json # WebUI 编辑的源持久化到此文件 sources: - name: deepseek base_url: https://api.deepseek.com - api_key: sk-... + api_key: sk-... # 也可用 api_key_env: SOME_ENV 引用环境变量（不落盘明文） adapter: deepseek max_concurrent: 8 models: - id: deepseek-v4-flash - priority: 100 # 越大越优先被 AUTO 选中 + priority: 100 # YAML 源首次启动会 seed 进 AUTO 链的初值 kind: chat - id: deepseek-v4-pro priority: 60 @@ -97,17 +103,38 @@ sources: timeout: 120s # 请求超时，默认 120s ``` +运行时文件（`runtime_file`）中的敏感字段自动加密： + +- 加密算法 AES-256-GCM，格式 `enc:v1:`。 +- 主密钥来源：环境变量 `LLMS_PROXY_MASTER_KEY`（64 位 hex）；否则读取 + `runtime_file` 同目录的 `master.key`；都不存在时首次启动自动生成 `master.key`（0600）。 +- 升级时旧明文文件自动兼容：首次运行正常读取，任何 UI 保存操作触发全文件加密迁移。 +- 注意：**master.key 丢失后密文无法解密**，请随配置一起备份；切勿提交到版本库。 + ### 模型路由 `/v1/chat/completions` 的 `model` 解析顺序： 1. `source/model` 或 `source:model` 前缀 → 指定源； 2. 精确匹配某个源的 `model`； -3. 配置 `default_model: AUTO` 时 → 按各源模型的 `priority`（数字大优先）选最高可用源； -4. 否则回落到 `default_source`。 +3. 配置 `default_model: AUTO` 时 → 走优先级页保存的 AUTO 链（见下）； +4. 否则回落（返回错误）。 任何 OpenAI 客户端，只要 `model` 设为某个源的 `name/任意名`，即可锁定走该源； -设为 `AUTO`（或网关配了 `default_model: AUTO`）即自动按优先级选源。 +设为 `AUTO`（或网关配了 `default_model: AUTO`）即自动按优先级链选源。 + +### AUTO 链（优先级页） + +AUTO 调度**只**由优先级页保存的规则（持久化到 `runtime_file` 的 `auto` 字段）决定， +源配置里的 `models[].priority` 数字不再参与调度、也不再显示。 + +- 每条规则 = 一个「槽位」：`{ model, source, tier, token_quota, period, hours }`。 +- `tier` 表示优先级档位：同一档的模型并排、共享该优先级；档位从上到下递减。 +- 同一模型可配置多个槽位（如 A 源低配 → B 源低配 → A 源高配 → B 源高配），按档位顺延。 +- `token_quota` > 0 时该槽位在重置周期内用满即顺延到下一槽位；`period` 支持 + `hour` / `week` / `month` / `nhour`（配合 `hours`），空 = 不限。 +- 生图模型（`kind: image`）不参与 AUTO 链；生图走 `POST /v1/images/generations` + 的独立路径。 ### 源（Source）与适配器（Adapter）的关系 @@ -130,9 +157,13 @@ sources: 4. 请求进来时 `Registry` 按 `model` 路由到 Provider，Provider 调适配器 `transform_request` → HTTP 发送 → `transform_response` / `transform_stream_chunk`。 -WebUI 上的"新增/编辑源"与"上传 Lua 适配器"都即时生效（写入运行时文件或 -`adapter_dir` 后重新装配，无需重启）；直接编辑 `config.yaml` / `adapter_dir` -下的文件则需要重启进程才会重新加载。 +WebUI 上的"新增/编辑源"、"上传 Lua 适配器"、"改 AUTO 优先级链"与"密钥模型范围" +都即时生效（写入运行时文件或 `adapter_dir` 后重新装配，无需重启）；直接编辑 +`config.yaml` / `adapter_dir` 下的文件则需要重启进程才会重新加载。 + +在 WebUI 删除源/适配器时：运行时（WebUI 创建）的源与 `.lua` 文件会真正移除； +`config.yaml` 中定义的基源无法改写配置文件，采用删除标记隐藏（重启后仍隐藏）， +在 UI 里重新添加同名源即可恢复。 ### disable_thinking @@ -141,7 +172,17 @@ WebUI 上的"新增/编辑源"与"上传 Lua 适配器"都即时生效（写入 ### WebUI -内置管理界面（`GET /`），登录后可在浏览器查看/新增/编辑上游源与模型， +内置管理界面（`GET /`），登录后可在浏览器完成： + +- **状态页**：源在线状态（实时探测 + 悬停看错误）、模型/源/key 用量统计、请求记录， + 支持按时间范围导出 CSV；点击模型可生成 pin 到该模型的连接配置。 +- **对话页**：流式/非流式调试。 +- **密钥页**：创建/编辑网关 key，为每个 key 配模型范围（模型 + 源 + token 配额 + 周期）， + 管理员管理全部 key，用户只看到自己的 key。 +- **优先级页**：拖拽积木配置 AUTO 链档位。 +- **源页**：在线增删改上游源（API key 等敏感字段加密落盘）。 +- **适配器页**：上传 / 删除 Lua 适配器脚本。 + 改动写入 `runtime_file`（重启仍生效）。 ## Lua 适配器协议 diff --git a/README_EN.md b/README_EN.md index 5e15dca..525c4eb 100644 --- a/README_EN.md +++ b/README_EN.md @@ -30,10 +30,12 @@ Extracted and independently evolved from the multi-source LLM adapter layer of - **Multi-source**: any number of upstream sources in one process, routed by the request's `model`. -- **AUTO mode**: with `default_model: AUTO`, the highest-priority available - source model wins. +- **AUTO mode**: with `default_model: AUTO`, scheduling follows the tiered AUTO + chain saved on the Priority page (see below). - **Unified output**: every source speaks OpenAI format (incl. `reasoning_content`, `tool_calls`, `usage`). +- **Image generation**: `POST /v1/images/generations`, routed to models with + `kind: image`. - **Multimodal**: `content` arrays (`image_url` etc.) pass through losslessly; Anthropic/Gemini/Ollama are translated automatically. - **LuaJIT VM**: golua-binding LuaJIT; each adapter has its own VM + worker @@ -46,8 +48,14 @@ Extracted and independently evolved from the multi-source LLM adapter layer of inject/sign request headers before the HTTP call (e.g. KimiCode style app-validation), with helpers like `hmac_sha256_hex`, `sha256_hex`, `base64_encode`. -- **WebUI**: built-in management page to view/add/edit sources & models, - persisted to a runtime file. +- **WebUI**: built-in management page to view/add/edit sources & models, edit + the AUTO chain, and manage per-key model scopes, persisted to a runtime file. +- **Encrypted secrets at rest**: upstream `api_key`, custom header values and + gateway keys are stored AES-256-GCM encrypted (`master.key` 0600 next to the + runtime file, or `LLMS_PROXY_MASTER_KEY`). +- **Live source probing**: the status page probes each source + (`GET {base}/models`, falling back to a minimal chat call) without touching + the scheduler's backoff state; errors are shown on hover. - **Auth**: the gateway validates client keys via `gateway_keys`; independent from each upstream's own key. - **Streaming**: SSE `chat.completion.chunk` with a role-first chunk and @@ -97,12 +105,12 @@ runtime_file: runtime.json # WebUI-edited sources persist here sources: - name: deepseek base_url: https://api.deepseek.com - api_key: sk-... + api_key: sk-... # or api_key_env: SOME_ENV to read from an env var adapter: deepseek max_concurrent: 8 models: - id: deepseek-v4-flash - priority: 100 # higher -> preferred by AUTO + priority: 100 # YAML sources seed the AUTO chain on first start kind: chat - id: deepseek-v4-pro priority: 60 @@ -116,18 +124,48 @@ sources: timeout: 120s # request timeout, default 120s ``` +Sensitive fields in the runtime file (`runtime_file`) are encrypted at rest: + +- AES-256-GCM, stored as `enc:v1:`. +- Master key source: env var `LLMS_PROXY_MASTER_KEY` (64 hex chars), else a + `master.key` file next to the runtime file; if neither exists it is generated + on first start (mode 0600). +- Old plaintext files still load; the first UI save migrates the whole file to + ciphertext. +- Back up `master.key` with your config — losing it makes the secrets + undecryptable. Do not commit it. + ### Model routing `/v1/chat/completions` `model` resolution order: 1. `source/model` or `source:model` prefix → pinned source; 2. exact match of a source's `model`; -3. with `default_model: AUTO` → highest-`priority` healthy source model; -4. otherwise fall back to `default_source`. +3. with `default_model: AUTO` → follow the AUTO chain from the Priority page + (see below); +4. otherwise the request errors. Any OpenAI client can pin to a source by setting `model` to its -`name/anything`; `AUTO` (or the gateway's `default_model: AUTO`) picks the -healthy source by priority. +`name/anything`; `AUTO` (or the gateway's `default_model: AUTO`) follows the +tiered AUTO chain. + +### AUTO chain (Priority page) + +AUTO scheduling is driven **only** by the rules saved on the Priority page +(persisted as the `auto` field of the runtime file). The numeric +`models[].priority` in source configs no longer participates in scheduling and +is no longer shown. + +- Each rule is one "slot": `{ model, source, tier, token_quota, period, hours }`. +- `tier` is the priority tier: models in the same tier sit side by side and + share it; tiers run high → low. +- The same model may appear in several slots (e.g. A low → B low → A high → + B high) and is tried in tier order. +- `token_quota` > 0 means the slot is skipped once its tokens within the reset + window are exhausted; `period` supports `hour` / `week` / `month` / `nhour` + (with `hours`); empty = unlimited. +- Image models (`kind: image`) are kept out of the chain and are served by the + separate `POST /v1/images/generations` path. ### Sources vs. adapters @@ -154,10 +192,15 @@ Loading flow (assembled by `Core` in `internal/core`): adapter's `transform_request` → HTTP call → `transform_response` / `transform_stream_chunk`. -"Add/Edit source" and "Upload adapter" in the WebUI take effect immediately -(written to the runtime file / `adapter_dir`, then reassembled — no restart). -Directly editing `config.yaml` or files under `adapter_dir` requires a process -restart. +"Add/Edit source", "Upload adapter", "AUTO chain edits" and "per-key model +scopes" in the WebUI take effect immediately (written to the runtime file / +`adapter_dir`, then reassembled — no restart). Directly editing `config.yaml` +or files under `adapter_dir` requires a process restart. + +Deleting a source/adapter in the WebUI truly removes runtime (UI-created) +sources and `.lua` files; base sources defined in `config.yaml` cannot rewrite +that file, so they are hidden via a deletion tombstone (still hidden after +restart) and can be restored by re-adding the same name in the UI. ### disable_thinking @@ -167,8 +210,20 @@ each adapter; the DeepSeek adapter maps it to `extra_body.thinking.type = ### WebUI -Built-in admin page at `GET /`; after login you can view/edit sources and -models in the browser, persisted to `runtime_file` (survives restarts). +Built-in admin page at `GET /`; after login you can: + +- **Status**: source online state (live probe, error on hover), per + model/source/key usage stats, request records filtered by key, and CSV export + over a time range; click a model to pin the connection snippet to it. +- **Chat**: streaming / non-streaming debug. +- **Keys**: create/edit gateway keys, set per-key model scopes (model + source + + token quota + reset period); admins manage all keys, users see only their own. +- **Priority**: drag blocks to build the tiered AUTO chain. +- **Sources**: add/edit/delete upstream sources online (secrets stored + encrypted). +- **Adapters**: upload / delete Lua adapter scripts. + +Changes persist to `runtime_file` (survive restarts). ## Lua adapter protocol diff --git a/config.example.yaml b/config.example.yaml index 5001073..b4aa4f1 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -7,7 +7,7 @@ listen: 127.0.0.1:8080 gateway_keys: - sk-gw-local-0001 -# 默认模型选择：具体模型 id 或 AUTO（按各源模型的 priority 自动选最高可用源） +# 默认模型选择：具体模型 id 或 AUTO（AUTO 走优先级页保存的 AUTO 链） default_model: AUTO # Lua 适配器目录（默认 adapters/，首次启动自动写入内置适配器） @@ -24,12 +24,12 @@ max_concurrent: 0 sources: - name: deepseek base_url: https://api.deepseek.com - api_key: sk-your-deepseek-key + api_key: sk-your-deepseek-key # 或用 api_key_env: SOME_ENV 引用环境变量 adapter: deepseek max_concurrent: 8 models: - id: deepseek-v4-flash - priority: 100 # 数字越大越优先被 AUTO 选中 + priority: 100 # YAML 源首次启动 seed 进 AUTO 链的初值 kind: chat - id: deepseek-reasoner priority: 60 diff --git a/internal/gateway/ui/index.html b/internal/gateway/ui/index.html index 53d5f50..9c0eaa7 100644 --- a/internal/gateway/ui/index.html +++ b/internal/gateway/ui/index.html @@ -172,6 +172,9 @@ td.t-tag { white-space:nowrap; } border-radius:99px; white-space:nowrap; } .mb .copy-b { cursor:pointer; opacity:.8; font-size:12px; padding:0 3px; } .mb .copy-b:hover { opacity:1; } +.mb .mb-x { cursor:pointer; font-size:15px; line-height:1; padding:4px 5px; margin:0 2px; border-radius:6px; + color:rgba(255,255,255,.85); background:rgba(0,0,0,.18); user-select:none; } +.mb .mb-x:hover { background:rgba(220,53,69,.85); color:#fff; } .mb-ghost { position:fixed; pointer-events:none; opacity:.85; z-index:60; transform:rotate(2deg); } .mb-empty { color:var(--muted); font-size:12.5px; } .add-brick { display:inline-flex; align-items:center; gap:6px; padding:9px 16px; border-radius:12px; @@ -582,9 +585,29 @@ function toast(m) { const el = $('#toast'); el.textContent = m; el.style.display function esc(s) { return String(s ?? '').replace(/[&<>"']/g, c => ({'&':'&','<':'<','>':'>','"':'"',"'":'''}[c])); } function escAttr(s) { return esc(s).replace(/"/g, '"'); } function copyText(txt, okMsg) { - if (navigator.clipboard && navigator.clipboard.writeText) { - navigator.clipboard.writeText(txt).then(() => toast(okMsg || t('toastCopied')), () => toast(t('toastCopyFail'))); - } else { toast(t('toastCopyFail')); } + const done = () => toast(okMsg || t('toastCopied')); + const fail = () => toast(t('toastCopyFail')); + if (navigator.clipboard && navigator.clipboard.writeText && window.isSecureContext) { + navigator.clipboard.writeText(txt).then(done, () => legacyCopy(txt) ? done() : fail()); + return; + } + legacyCopy(txt) ? done() : fail(); +} +function legacyCopy(txt) { + try { + const ta = document.createElement('textarea'); + ta.value = txt; + ta.setAttribute('readonly', ''); + ta.style.position = 'fixed'; + ta.style.opacity = '0'; + ta.style.pointerEvents = 'none'; + document.body.appendChild(ta); + ta.select(); + ta.setSelectionRange(0, ta.value.length); + const ok = document.execCommand('copy'); + document.body.removeChild(ta); + return ok; + } catch (e) { return false; } } /* ---------- status tab ---------- */ @@ -1779,6 +1802,7 @@ function scopeHtml(key, m) { ${esc(m.model)}${src ? `${esc(src)}` : ''} ${esc(quantBadge(m.token_quota, m.period, m.hours))} ⧉ + × `; } function normSrc(s) { return (!s || s === 'undefined' || s === 'null') ? '' : s; } @@ -1824,10 +1848,18 @@ async function scopePush(key) { const canvas = document.querySelector(`.key-canvas[data-key="${CSS.escape(key)}"]`); if (!canvas) return; const scopes = readScopes(canvas); - if (!scopes.some(s => s.model === 'AUTO')) scopes.push({ model: 'AUTO', token_quota: 0 }); + let added; + if (!scopes.some(s => s.model === 'AUTO')) { + added = { model: 'AUTO', token_quota: 0 }; + scopes.push(added); + } else { + const pick = (allModels || []).find(m => m.id && m.id !== 'AUTO' && !scopes.some(s => s.model === m.id && normSrc(s.source) === m.src)); + added = pick ? { model: pick.id, source: pick.src, token_quota: 0 } : { model: 'AUTO', token_quota: 0 }; + scopes.push(added); + } try { await putScope(key, scopes); } catch (e) { toast(e.message); return; } await loadKeys(); - scopeEdit(key, 'AUTO'); + scopeEdit(key, scopeComb(added)); } async function scopeDup(key, comb) { const canvas = document.querySelector(`.key-canvas[data-key="${CSS.escape(key)}"]`);