fix(ui): clipboard fallback for non-secure contexts; touch-friendly delete button on key scope blocks; append named model after AUTO when adding (+ docs sync: encrypted storage, AUTO chain, live probing, image endpoint)

This commit is contained in:
root
2026-08-10 11:10:59 +08:00
parent 62c8a07b88
commit e2dd4d9727
4 changed files with 163 additions and 35 deletions

View File

@ -30,10 +30,12 @@ Extracted and independently evolved from the multi-source LLM adapter layer of
- **Multi-source**: any number of upstream sources in one process, routed by the
request's `model`.
- **AUTO mode**: with `default_model: AUTO`, the highest-priority available
source model wins.
- **AUTO mode**: with `default_model: AUTO`, scheduling follows the tiered AUTO
chain saved on the Priority page (see below).
- **Unified output**: every source speaks OpenAI format (incl.
`reasoning_content`, `tool_calls`, `usage`).
- **Image generation**: `POST /v1/images/generations`, routed to models with
`kind: image`.
- **Multimodal**: `content` arrays (`image_url` etc.) pass through losslessly;
Anthropic/Gemini/Ollama are translated automatically.
- **LuaJIT VM**: golua-binding LuaJIT; each adapter has its own VM + worker
@ -46,8 +48,14 @@ Extracted and independently evolved from the multi-source LLM adapter layer of
inject/sign request headers before the HTTP call (e.g. KimiCode style
app-validation), with helpers like `hmac_sha256_hex`, `sha256_hex`,
`base64_encode`.
- **WebUI**: built-in management page to view/add/edit sources & models,
persisted to a runtime file.
- **WebUI**: built-in management page to view/add/edit sources & models, edit
the AUTO chain, and manage per-key model scopes, persisted to a runtime file.
- **Encrypted secrets at rest**: upstream `api_key`, custom header values and
gateway keys are stored AES-256-GCM encrypted (`master.key` 0600 next to the
runtime file, or `LLMS_PROXY_MASTER_KEY`).
- **Live source probing**: the status page probes each source
(`GET {base}/models`, falling back to a minimal chat call) without touching
the scheduler's backoff state; errors are shown on hover.
- **Auth**: the gateway validates client keys via `gateway_keys`; independent
from each upstream's own key.
- **Streaming**: SSE `chat.completion.chunk` with a role-first chunk and
@ -97,12 +105,12 @@ runtime_file: runtime.json # WebUI-edited sources persist here
sources:
- name: deepseek
base_url: https://api.deepseek.com
api_key: sk-...
api_key: sk-... # or api_key_env: SOME_ENV to read from an env var
adapter: deepseek
max_concurrent: 8
models:
- id: deepseek-v4-flash
priority: 100 # higher -> preferred by AUTO
priority: 100 # YAML sources seed the AUTO chain on first start
kind: chat
- id: deepseek-v4-pro
priority: 60
@ -116,18 +124,48 @@ sources:
timeout: 120s # request timeout, default 120s
```
Sensitive fields in the runtime file (`runtime_file`) are encrypted at rest:
- AES-256-GCM, stored as `enc:v1:<base64>`.
- Master key source: env var `LLMS_PROXY_MASTER_KEY` (64 hex chars), else a
`master.key` file next to the runtime file; if neither exists it is generated
on first start (mode 0600).
- Old plaintext files still load; the first UI save migrates the whole file to
ciphertext.
- Back up `master.key` with your config — losing it makes the secrets
undecryptable. Do not commit it.
### Model routing
`/v1/chat/completions` `model` resolution order:
1. `source/model` or `source:model` prefix → pinned source;
2. exact match of a source's `model`;
3. with `default_model: AUTO`highest-`priority` healthy source model;
4. otherwise fall back to `default_source`.
3. with `default_model: AUTO`follow the AUTO chain from the Priority page
(see below);
4. otherwise the request errors.
Any OpenAI client can pin to a source by setting `model` to its
`name/anything`; `AUTO` (or the gateway's `default_model: AUTO`) picks the
healthy source by priority.
`name/anything`; `AUTO` (or the gateway's `default_model: AUTO`) follows the
tiered AUTO chain.
### AUTO chain (Priority page)
AUTO scheduling is driven **only** by the rules saved on the Priority page
(persisted as the `auto` field of the runtime file). The numeric
`models[].priority` in source configs no longer participates in scheduling and
is no longer shown.
- Each rule is one "slot": `{ model, source, tier, token_quota, period, hours }`.
- `tier` is the priority tier: models in the same tier sit side by side and
share it; tiers run high → low.
- The same model may appear in several slots (e.g. A low → B low → A high →
B high) and is tried in tier order.
- `token_quota` > 0 means the slot is skipped once its tokens within the reset
window are exhausted; `period` supports `hour` / `week` / `month` / `nhour`
(with `hours`); empty = unlimited.
- Image models (`kind: image`) are kept out of the chain and are served by the
separate `POST /v1/images/generations` path.
### Sources vs. adapters
@ -154,10 +192,15 @@ Loading flow (assembled by `Core` in `internal/core`):
adapter's `transform_request` → HTTP call → `transform_response` /
`transform_stream_chunk`.
"Add/Edit source" and "Upload adapter" in the WebUI take effect immediately
(written to the runtime file / `adapter_dir`, then reassembled — no restart).
Directly editing `config.yaml` or files under `adapter_dir` requires a process
restart.
"Add/Edit source", "Upload adapter", "AUTO chain edits" and "per-key model
scopes" in the WebUI take effect immediately (written to the runtime file /
`adapter_dir`, then reassembled — no restart). Directly editing `config.yaml`
or files under `adapter_dir` requires a process restart.
Deleting a source/adapter in the WebUI truly removes runtime (UI-created)
sources and `.lua` files; base sources defined in `config.yaml` cannot rewrite
that file, so they are hidden via a deletion tombstone (still hidden after
restart) and can be restored by re-adding the same name in the UI.
### disable_thinking
@ -167,8 +210,20 @@ each adapter; the DeepSeek adapter maps it to `extra_body.thinking.type =
### WebUI
Built-in admin page at `GET /`; after login you can view/edit sources and
models in the browser, persisted to `runtime_file` (survives restarts).
Built-in admin page at `GET /`; after login you can:
- **Status**: source online state (live probe, error on hover), per
model/source/key usage stats, request records filtered by key, and CSV export
over a time range; click a model to pin the connection snippet to it.
- **Chat**: streaming / non-streaming debug.
- **Keys**: create/edit gateway keys, set per-key model scopes (model + source +
token quota + reset period); admins manage all keys, users see only their own.
- **Priority**: drag blocks to build the tiered AUTO chain.
- **Sources**: add/edit/delete upstream sources online (secrets stored
encrypted).
- **Adapters**: upload / delete Lua adapter scripts.
Changes persist to `runtime_file` (survive restarts).
## Lua adapter protocol