mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-09-19 16:39:15 +00:00
fix: tool call anchor & wire format, streaming chunk passthrough, WebUI narrow-screen, docs bilingual
This commit is contained in:
root
36
README.md
36
README.md
@ -1,10 +1,13 @@
|
||||
# ModelRouter
|
||||
|
||||
> **English**: [README_EN.md](./README_EN.md)
|
||||
|
||||
面向多 llms 订阅者，部署在内网，实现一次配置多个服务共同使用的效果。支持指定模型或 auto 模式，按照配置的优先级选择可用模型提供服务。
|
||||
|
||||
> **轻量且增源无需重编译**：网关本体是单 Go 二进制（约 10MB，零运行时依赖）。新增/切换上游
|
||||
> 只需在 `config.yaml`（或 WebUI）加一个 `sources` 条目或挂一个 `.lua` 适配器——**不改 Go、
|
||||
> 不重编译、不重启**。适配器 Lua 脚本热加载，协议差异全部隔离在 Lua 层，Go 只负责调度与透传。
|
||||
> 不重编译**。源与适配器的改动经 WebUI 提交时即时生效（热更新）；直接编辑
|
||||
> `config.yaml` 或 `adapter_dir` 下的 `.lua` 文件则需要重启进程生效。
|
||||
|
||||
## 实现
|
||||
|
||||
@ -106,6 +109,31 @@ sources:
|
||||
任何 OpenAI 客户端，只要 `model` 设为某个源的 `name/任意名`，即可锁定走该源；
|
||||
设为 `AUTO`（或网关配了 `default_model: AUTO`）即自动按优先级选源。
|
||||
|
||||
### 源（Source）与适配器（Adapter）的关系
|
||||
|
||||
- **源** 是到某个上游的连接描述：`name`、`base_url`、`api_key`、模型列表与优先级。
|
||||
- **适配器** 是协议转换逻辑（Lua 脚本）：把统一 OpenAI 格式请求转成上游原生格式，
|
||||
再把上游响应/流式分块转回统一格式。
|
||||
- 一个适配器可被多个源复用（如 `openai.lua` 同时服务多个 OpenAI 兼容站点）；
|
||||
同一个源也可以切换不同适配器（改 `adapter` 字段即可）。
|
||||
- 源决定“连谁、暴露哪些模型”，适配器决定“怎么对话”——二者在 `sources[]` 条目中
|
||||
通过 `adapter` 字段关联。
|
||||
|
||||
加载流程（`internal/core` 的 `Core` 负责装配）：
|
||||
|
||||
1. 启动时 `lua.NewVM(adapter_dir)` 加载全部适配器：内置适配器（编译期 embed）+
|
||||
`adapter_dir` 下的同名覆盖文件；内置文件写在代码内，覆盖文件要求更高优先级。
|
||||
2. `config.Load` 读取 `config.yaml`，`config.NewStore(runtime_file)` 读 WebUI 改动的
|
||||
运行时源，二者按名称合并成完整源列表。
|
||||
3. `rebuildRegistry` 为每个源创建 `provider.Provider`（持有目标适配器），并按源的
|
||||
并发上限配置适配器 worker 池大小。
|
||||
4. 请求进来时 `Registry` 按 `model` 路由到 Provider，Provider 调适配器
|
||||
`transform_request` → HTTP 发送 → `transform_response` / `transform_stream_chunk`。
|
||||
|
||||
WebUI 上的"新增/编辑源"与"上传 Lua 适配器"都即时生效（写入运行时文件或
|
||||
`adapter_dir` 后重新装配，无需重启）；直接编辑 `config.yaml` / `adapter_dir`
|
||||
下的文件则需要重启进程才会重新加载。
|
||||
|
||||
### disable_thinking
|
||||
|
||||
请求体带 `"disable_thinking": true` 时，网关透传给各适配器；DeepSeek 适配器将其
|
||||
@ -118,9 +146,11 @@ sources:
|
||||
|
||||
## Lua 适配器协议
|
||||
|
||||
完整 API 见 **[Lua 适配器 API 文档](docs/lua-adapters.md)**（[English](docs/lua-adapters-en.md)）。
|
||||
|
||||
每个适配器是一个返回 table 的 Lua 脚本（`internal/lua/adapters/<name>.lua`），
|
||||
加载时可被 `adapter_dir` 下的同名脚本覆盖——**改 Lua 脚本同样无需重编译**，
|
||||
重启即生效（甚至可通过 WebUI 在线编辑源配置）。
|
||||
加载时可被 `adapter_dir` 下的同名脚本覆盖——**改 Lua 脚本无需重编译**，
|
||||
重启即生效；通过 WebUI 上传的适配器与在线编辑的源配置则即时生效。
|
||||
|
||||
```lua
|
||||
return {
|
||||
|
||||
242
README_EN.md
Normal file
242
README_EN.md
Normal file
@ -0,0 +1,242 @@
|
||||
# ModelRouter
|
||||
|
||||
> **中文**: [README.md](./README.md)
|
||||
|
||||
A lightweight, unified OpenAI-compatible LLM gateway for internal networks —
|
||||
configure once, and let multiple services (agents, SDKs, bots) share many
|
||||
upstreams (DeepSeek, Qijiar, OpenAI, Anthropic, Gemini, Groq, Mistral, Ollama,
|
||||
KimiCode…) behind a single endpoint. Pick a specific model or use **AUTO** mode,
|
||||
which routes to the best healthy upstream by configured priority.
|
||||
|
||||
> **Lightweight, no recompile to add sources**: the gateway is a single Go
|
||||
> binary (~10 MB, zero runtime dependencies). Adding or switching an upstream is
|
||||
> just a `sources` entry in `config.yaml` (or via the WebUI) or a `.lua`
|
||||
> adapter — **no Go changes, no recompile**. Changes submitted through the
|
||||
> WebUI take effect immediately (hot reload); editing `config.yaml` or a `.lua`
|
||||
> file under `adapter_dir` by hand requires a process restart.
|
||||
|
||||
## Overview
|
||||
|
||||
Exposes a standard `OpenAI Chat Completions` API (`/v1/chat/completions` +
|
||||
`/v1/models`) to your internal network, translating between multiple upstream
|
||||
protocols via **Lua adapters**. Supports one-shot calls, SSE streaming,
|
||||
**AUTO model routing**, multimodal passthrough, and tool calls.
|
||||
|
||||
Extracted and independently evolved from the multi-source LLM adapter layer of
|
||||
[HomeAgent](https://gitcode.com/JianFeeeee/HomeAgent)
|
||||
(`internal/agent/api/provider.go` + `internal/lua/adapters/*`).
|
||||
|
||||
## Features
|
||||
|
||||
- **Multi-source**: any number of upstream sources in one process, routed by the
|
||||
request's `model`.
|
||||
- **AUTO mode**: with `default_model: AUTO`, the highest-priority available
|
||||
source model wins.
|
||||
- **Unified output**: every source speaks OpenAI format (incl.
|
||||
`reasoning_content`, `tool_calls`, `usage`).
|
||||
- **Multimodal**: `content` arrays (`image_url` etc.) pass through losslessly;
|
||||
Anthropic/Gemini/Ollama are translated automatically.
|
||||
- **LuaJIT VM**: golua-binding LuaJIT; each adapter has its own VM + worker
|
||||
pool for safe concurrency.
|
||||
- **disable_thinking**: `disable_thinking: true` toggles reasoning per-request.
|
||||
- **Lua adapter protocol**: each source mounts a `.lua` adapter with
|
||||
`transform_request` / `transform_response` / `transform_stream_chunk` — all
|
||||
protocol differences live in Lua, Go only schedules and proxies.
|
||||
- **Signature / header hooks**: adapters may define `build_headers(meta)` to
|
||||
inject/sign request headers before the HTTP call (e.g. KimiCode style
|
||||
app-validation), with helpers like `hmac_sha256_hex`, `sha256_hex`,
|
||||
`base64_encode`.
|
||||
- **WebUI**: built-in management page to view/add/edit sources & models,
|
||||
persisted to a runtime file.
|
||||
- **Auth**: the gateway validates client keys via `gateway_keys`; independent
|
||||
from each upstream's own key.
|
||||
- **Streaming**: SSE `chat.completion.chunk` with a role-first chunk and
|
||||
`[DONE]` terminator.
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
cp config.example.yaml config.yaml # edit your sources & keys
|
||||
GOMODCACHE=... GOPROXY=off go build -tags luajit -o llmsproxy ./cmd/llmsproxy
|
||||
./llmsproxy -config config.yaml
|
||||
```
|
||||
|
||||
> Depends on [golua](https://github.com/aarzilli/golua) (LuaJIT bindings).
|
||||
> You **must** build with `-tags luajit`; otherwise the built-in superset
|
||||
> gopher-lua path is used (behavior differs slightly).
|
||||
|
||||
```bash
|
||||
# no key -> 401
|
||||
curl http://127.0.0.1:8080/v1/models
|
||||
|
||||
# one-shot
|
||||
curl -H "Authorization: Bearer sk-gw-local-0001" \
|
||||
-d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"hi"}]}' \
|
||||
http://127.0.0.1:8080/v1/chat/completions
|
||||
|
||||
# streaming
|
||||
curl -N -H "Authorization: Bearer sk-gw-local-0001" \
|
||||
-d '{"model":"deepseek-v4-flash","stream":true,"messages":[{"role":"user","content":"hi"}]}' \
|
||||
http://127.0.0.1:8080/v1/chat/completions
|
||||
```
|
||||
|
||||
Any OpenAI SDK works: point `base_url` at the gateway, use one of `gateway_keys`
|
||||
as the API key.
|
||||
|
||||
## Configuration
|
||||
|
||||
See [`config.example.yaml`](config.example.yaml). Core fields:
|
||||
|
||||
```yaml
|
||||
listen: 127.0.0.1:8080 # bind address (keep internal/loopback)
|
||||
gateway_keys: [sk-gw-0001] # keys clients use; empty = no auth
|
||||
default_model: AUTO # when model is unroutable, pick source by priority
|
||||
adapter_dir: adapters # Lua adapter dir; built-ins written on first start
|
||||
runtime_file: runtime.json # WebUI-edited sources persist here
|
||||
|
||||
sources:
|
||||
- name: deepseek
|
||||
base_url: https://api.deepseek.com
|
||||
api_key: sk-...
|
||||
adapter: deepseek
|
||||
max_concurrent: 8
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
priority: 100 # higher -> preferred by AUTO
|
||||
kind: chat
|
||||
- id: deepseek-v4-pro
|
||||
priority: 60
|
||||
kind: chat
|
||||
# static headers (take precedence over adapter defaults)
|
||||
headers: { X-Tenant: prod }
|
||||
# passthrough metadata for the Lua build_headers hook
|
||||
meta: { app_id: x, app_secret: y }
|
||||
temperature: 0.7
|
||||
max_tokens: 4096
|
||||
timeout: 120s # request timeout, default 120s
|
||||
```
|
||||
|
||||
### Model routing
|
||||
|
||||
`/v1/chat/completions` `model` resolution order:
|
||||
|
||||
1. `source/model` or `source:model` prefix → pinned source;
|
||||
2. exact match of a source's `model`;
|
||||
3. with `default_model: AUTO` → highest-`priority` healthy source model;
|
||||
4. otherwise fall back to `default_source`.
|
||||
|
||||
Any OpenAI client can pin to a source by setting `model` to its
|
||||
`name/anything`; `AUTO` (or the gateway's `default_model: AUTO`) picks the
|
||||
healthy source by priority.
|
||||
|
||||
### Sources vs. adapters
|
||||
|
||||
- A **source** describes a connection to an upstream: `name`, `base_url`,
|
||||
`api_key`, model list and priorities.
|
||||
- An **adapter** is the protocol translation logic (Lua script): converts the
|
||||
unified OpenAI request to the upstream's native format and back.
|
||||
- One adapter serves many sources (e.g. `openai.lua` for any OpenAI-compatible
|
||||
site); one source can switch adapters via its `adapter` field.
|
||||
- Sources decide *whom to talk to and which models to expose*; adapters decide
|
||||
*how to talk*. They are linked by the `adapter` field inside each `sources[]`
|
||||
entry.
|
||||
|
||||
Loading flow (assembled by `Core` in `internal/core`):
|
||||
|
||||
1. On startup `lua.NewVM(adapter_dir)` loads all adapters: built-in ones
|
||||
(embedded at compile time) + same-name override files under `adapter_dir`.
|
||||
2. `config.Load` reads `config.yaml`; `config.NewStore(runtime_file)` reads
|
||||
WebUI-edited runtime sources; both are merged by name.
|
||||
3. `rebuildRegistry` creates a `provider.Provider` per source (holding its
|
||||
adapter), sizing the adapter's worker pool from the source's concurrency
|
||||
limit.
|
||||
4. Incoming requests are routed by `Registry` to a provider, which calls the
|
||||
adapter's `transform_request` → HTTP call → `transform_response` /
|
||||
`transform_stream_chunk`.
|
||||
|
||||
"Add/Edit source" and "Upload adapter" in the WebUI take effect immediately
|
||||
(written to the runtime file / `adapter_dir`, then reassembled — no restart).
|
||||
Directly editing `config.yaml` or files under `adapter_dir` requires a process
|
||||
restart.
|
||||
|
||||
### disable_thinking
|
||||
|
||||
With `"disable_thinking": true` in the request body, the gateway passes it to
|
||||
each adapter; the DeepSeek adapter maps it to `extra_body.thinking.type =
|
||||
"disabled"`, other sources follow their own protocol.
|
||||
|
||||
### WebUI
|
||||
|
||||
Built-in admin page at `GET /`; after login you can view/edit sources and
|
||||
models in the browser, persisted to `runtime_file` (survives restarts).
|
||||
|
||||
## Lua adapter protocol
|
||||
|
||||
Full API: **[Lua adapter docs](docs/lua-adapters.md)**
|
||||
([中文](docs/lua-adapters-en.md)).
|
||||
|
||||
Each adapter is a Lua script returning a table
|
||||
(`internal/lua/adapters/<name>.lua`), overridable by a same-name file under
|
||||
`adapter_dir` — **no recompile needed**, restart to take effect; uploads via
|
||||
the WebUI take effect immediately.
|
||||
|
||||
```lua
|
||||
return {
|
||||
name = "mysrc",
|
||||
version = "1.0.0",
|
||||
endpoint = "/chat/completions", -- upstream path (source.endpoint overrides)
|
||||
headers = { ["X-Static"] = "v" }, -- static default headers (fallback)
|
||||
|
||||
-- request: convert unified OpenAI req -> upstream native format, return string
|
||||
transform_request = function(raw_json) ... end,
|
||||
|
||||
-- response: convert upstream response to unified format string
|
||||
-- { content, reasoning_content, finish_reason, token_usage{...}, tool_calls[{...}] }
|
||||
transform_response = function(raw_json) ... end,
|
||||
|
||||
-- streaming chunk: convert upstream SSE data to { content, done, ... }; "" skips
|
||||
transform_stream_chunk = function(raw_chunk) ... end,
|
||||
|
||||
-- [optional] dynamic headers / signature hook
|
||||
-- meta = { url, method, body, api_key, timestamp, source={ name, meta={...} } }
|
||||
build_headers = function(meta) return { ["X-App-Sign"] = sign } end,
|
||||
}
|
||||
```
|
||||
|
||||
Shared helpers: `hmac_sha256_hex(key, data)`, `sha256_hex(data)`,
|
||||
`base64_encode(s)`, `tohex(s)`, `json.encode/decode`, `log(level, msg)`.
|
||||
|
||||
### Built-in adapters
|
||||
|
||||
`openai` `deepseek` `anthropic` `gemini` `github` `groq` `mistral` `ollama`
|
||||
`kimicode`.
|
||||
|
||||
`anthropic`/`gemini`/`ollama` include multimodal conversion
|
||||
(`image_url` → their native format); with `disable_thinking` the `deepseek`
|
||||
adapter sets `extra_body.thinking.type` to `disabled`.
|
||||
|
||||
**kimicode** demonstrates `build_headers`: the cloud validates the calling
|
||||
app, so you HMAC-sign timestamp+URL+body with `meta.app_secret` and add
|
||||
`X-App-Sign`-style headers. Configure `sources[].meta.{app_id, app_secret,
|
||||
app_agent}`.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
cmd/llmsproxy # entry point
|
||||
internal/config # YAML config loading/validation
|
||||
internal/lua # LuaJIT VM + worker pool + AdapterCache + built-ins (embed)
|
||||
internal/provider # Provider(HTTP) + Registry(routing)
|
||||
internal/gateway # OpenAI-compatible HTTP + auth + SDK/streaming + WebUI
|
||||
internal/types # unified format & OpenAI wire types
|
||||
```
|
||||
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
go test -tags luajit ./...
|
||||
```
|
||||
|
||||
Covers: config validation, adapter load/transform, signature hooks, gateway
|
||||
auth, SDK round trip, SSE streaming, model routing, multimodal passthrough and
|
||||
disable_thinking.
|
||||
271
docs/lua-adapters-en.md
Normal file
271
docs/lua-adapters-en.md
Normal file
@ -0,0 +1,271 @@
|
||||
# Lua Adapter API
|
||||
|
||||
> **中文**: [lua-adapters.md](./lua-adapters.md)
|
||||
|
||||
Every upstream source is mounted with a `.lua` adapter that **transforms** the unified
|
||||
OpenAI-format request into the upstream's native format, and converts upstream
|
||||
responses / stream chunks back into the unified format. The Go layer only handles
|
||||
scheduling, concurrency and pass-through — so **adding an adapter or adapting a new
|
||||
protocol never requires recompiling Go**.
|
||||
|
||||
Adapters live in two equivalent places:
|
||||
- Bundled: `internal/lua/adapters/<name>.lua` (embedded at build time)
|
||||
- Override: a same-named script in the configured `adapter_dir` (takes precedence)
|
||||
|
||||
> **When are they loaded?** All adapters are loaded once at startup by
|
||||
> `lua.NewVM(adapter_dir)`. Adapters uploaded via the WebUI
|
||||
> (`Core.UploadAdapter` → `vm.LoadAdapter`) and sources edited online take
|
||||
> effect immediately, without a restart. Directly editing a `.lua` file under
|
||||
> `adapter_dir` requires a process restart to reload.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Script Structure](#1-script-structure)
|
||||
2. [Static Fields](#2-static-fields)
|
||||
3. [Transform Hooks](#3-transform-hooks)
|
||||
4. [Optional Hooks](#4-optional-hooks)
|
||||
5. [Built-in Helper Functions](#5-built-in-helper-functions)
|
||||
6. [meta Contract](#6-meta-contract)
|
||||
7. [A Complete Minimal Adapter](#7-a-complete-minimal-adapter)
|
||||
8. [Multimodal & disable_thinking](#8-multimodal--disable_thinking)
|
||||
|
||||
---
|
||||
|
||||
## 1. Script Structure
|
||||
|
||||
A script is a Lua file that returns a table — it **must** `return` a table:
|
||||
|
||||
```lua
|
||||
local adapter = {}
|
||||
adapter.name = "mysrc"
|
||||
adapter.version = "1.0.0"
|
||||
-- ... fields and functions ...
|
||||
return adapter
|
||||
```
|
||||
|
||||
Scripts are executed by LuaJIT (via golua bindings, cgo). Each adapter owns an
|
||||
**independent VM + worker pool**; adapters never interfere with each other. No mutable
|
||||
state may be shared across workers (`log` to stdout is the only side effect).
|
||||
|
||||
## 2. Static Fields
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `name` | string | yes | Adapter name, shown in the WebUI |
|
||||
| `version` | string | no | Version, shown in the WebUI |
|
||||
| `endpoint` | string | no | Upstream path, default `/chat/completions`; overridable by `source.endpoint` / `source.image_endpoint` |
|
||||
| `headers` | table | no | Static default request headers; used as fallback when no `build_headers` hook is defined |
|
||||
|
||||
These fields are extracted statically at load time (compile-once); reading them never
|
||||
occupies a pooled worker.
|
||||
|
||||
## 3. Transform Hooks
|
||||
|
||||
### `transform_request(raw_body) -> string`
|
||||
|
||||
Input: the unified OpenAI-format JSON string built by the Go layer (the
|
||||
`/v1/chat/completions` request body).
|
||||
Output: the request body string to send upstream.
|
||||
|
||||
```lua
|
||||
function adapter.transform_request(raw_body)
|
||||
local ok, req = pcall(json.decode, raw_body)
|
||||
if not ok then return raw_body end
|
||||
req.model = "upstream-model-name" -- rewrite the model name
|
||||
req.stream = req.stream or false
|
||||
return json.encode(req)
|
||||
end
|
||||
```
|
||||
|
||||
Contract:
|
||||
- The returned string is POSTed verbatim to `base_url + (source.endpoint or adapter.endpoint)`.
|
||||
- Internal gateway fields such as `disable_thinking` and `extra_body` should be cleaned
|
||||
up by the transform (`openai.lua` deletes both).
|
||||
|
||||
### `transform_response(raw_body) -> string`
|
||||
|
||||
Input: the raw JSON string of a non-streaming upstream response.
|
||||
Output: a JSON string in the unified format.
|
||||
|
||||
Unified format fields:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `content` | string | Response text |
|
||||
| `finish_reason` | string | `stop` / `length` / `tool_calls` etc. |
|
||||
| `reasoning_content` | string (optional) | Reasoning text (DeepSeek etc.) |
|
||||
| `token_usage` | table | `{ prompt, completion, total }` (tokens) |
|
||||
| `tool_calls` | array (optional) | Tool calls: `{ id, type, name, arguments }`; `arguments` is a **decoded** table |
|
||||
|
||||
```lua
|
||||
function adapter.transform_response(raw_body)
|
||||
local ok, resp = pcall(json.decode, raw_body)
|
||||
if not ok or resp == nil then return raw_body end
|
||||
local unified = {
|
||||
content = "",
|
||||
finish_reason = "",
|
||||
token_usage = { prompt = 0, completion = 0, total = 0 }
|
||||
}
|
||||
if type(resp.choices) == "table" and #resp.choices > 0 then
|
||||
local ch = resp.choices[1]
|
||||
unified.content = ch.message.content or ""
|
||||
unified.finish_reason = ch.finish_reason or ""
|
||||
if ch.message.reasoning_content then
|
||||
unified.reasoning_content = ch.message.reasoning_content
|
||||
end
|
||||
end
|
||||
return json.encode(unified)
|
||||
end
|
||||
```
|
||||
|
||||
### `transform_stream_chunk(raw_chunk) -> string`
|
||||
|
||||
Input: the raw JSON string of each `data:` line in the SSE stream (without the
|
||||
`data:` prefix).
|
||||
Output: a unified chunk JSON string; **returning `""` skips that chunk**.
|
||||
|
||||
Unified chunk format:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `content` | string | Incremental text for this chunk (may be `""`) |
|
||||
| `done` | boolean (optional) | `true` ends the stream (when `finish_reason` appears) |
|
||||
|
||||
```lua
|
||||
function adapter.transform_stream_chunk(raw_chunk)
|
||||
local ok, chunk = pcall(json.decode, raw_chunk)
|
||||
if not ok then return "" end
|
||||
if not chunk.choices or #chunk.choices == 0 then return "" end
|
||||
local delta = chunk.choices[1].delta or {}
|
||||
local fr = chunk.choices[1].finish_reason
|
||||
return json.encode({
|
||||
content = delta.content or "",
|
||||
done = (fr ~= nil)
|
||||
})
|
||||
end
|
||||
```
|
||||
|
||||
## 4. Optional Hooks
|
||||
|
||||
### `build_headers(meta) -> table<string,string>`
|
||||
|
||||
Dynamically generate / sign request headers (e.g. KimiCode's HMAC signature). If the
|
||||
script does not define this function, the Go layer falls back to the static
|
||||
`adapter.headers` (or `source.headers`).
|
||||
|
||||
```lua
|
||||
function adapter.build_headers(meta)
|
||||
local msg = meta.method .. meta.url .. meta.body
|
||||
return {
|
||||
["X-App-Sign"] = hmac_sha256_hex(meta.source.meta.app_secret, msg),
|
||||
["X-Timestamp"] = meta.timestamp,
|
||||
}
|
||||
end
|
||||
```
|
||||
|
||||
Returning a non-table errors; returning `{}` means no custom headers (no fallback to
|
||||
static headers).
|
||||
|
||||
## 5. Built-in Helper Functions
|
||||
|
||||
Global functions shared by all adapters (injected by Go):
|
||||
|
||||
| Function | Description |
|
||||
|----------|-------------|
|
||||
| `json.encode(v)` | Lua value → JSON string; returns `"null"` on failure |
|
||||
| `json.decode(s)` | JSON string → Lua value; returns `nil` on failure |
|
||||
| `hmac_sha256_hex(key, data)` | HMAC-SHA256, lowercase hex string |
|
||||
| `sha256_hex(data)` | SHA-256, lowercase hex string |
|
||||
| `base64_encode(s)` | Standard Base64 encoding |
|
||||
| `tohex(s)` | Bytes → lowercase hex string |
|
||||
| `log(level, msg)` | Prints `[adapter/<name>] <msg>` to stdout |
|
||||
|
||||
Note: scripts run under LuaJIT with the full standard library (`string`/`table`/`pcall`
|
||||
etc.); `os`/`io` are not exposed (sandbox semantics).
|
||||
|
||||
## 6. meta Contract
|
||||
|
||||
The meta structure shared by `build_headers(meta)` and the request transforms:
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `meta.url` | Full request URL |
|
||||
| `meta.method` | HTTP method (usually `POST`) |
|
||||
| `meta.body` | Request body string |
|
||||
| `meta.api_key` | The source's `api_key` |
|
||||
| `meta.timestamp` | Request timestamp |
|
||||
| `meta.source` | table: `{ name, meta = { ... } }` — the source's `meta` field (e.g. `app_secret`) |
|
||||
|
||||
## 7. A Complete Minimal Adapter
|
||||
|
||||
```lua
|
||||
local adapter = {}
|
||||
|
||||
adapter.name = "mysrc"
|
||||
adapter.version = "1.0.0"
|
||||
adapter.endpoint = "/chat/completions"
|
||||
adapter.headers = { ["X-Tenant"] = "prod" }
|
||||
|
||||
function adapter.transform_request(raw_body)
|
||||
local ok, req = pcall(json.decode, raw_body)
|
||||
if not ok then return raw_body end
|
||||
req.disable_thinking = nil
|
||||
req.extra_body = nil
|
||||
return json.encode(req)
|
||||
end
|
||||
|
||||
function adapter.transform_response(raw_body)
|
||||
local ok, resp = pcall(json.decode, raw_body)
|
||||
if not ok or resp == nil then return raw_body end
|
||||
local unified = {
|
||||
content = "",
|
||||
finish_reason = "",
|
||||
token_usage = { prompt = 0, completion = 0, total = 0 }
|
||||
}
|
||||
if type(resp.choices) == "table" and #resp.choices > 0 then
|
||||
local ch = resp.choices[1]
|
||||
unified.content = ch.message.content or ""
|
||||
unified.finish_reason = ch.finish_reason or ""
|
||||
end
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
function adapter.transform_stream_chunk(raw_chunk)
|
||||
local ok, chunk = pcall(json.decode, raw_chunk)
|
||||
if not ok then return "" end
|
||||
if not chunk.choices or #chunk.choices == 0 then return "" end
|
||||
return json.encode({
|
||||
content = (chunk.choices[1].delta or {}).content or "",
|
||||
done = (chunk.choices[1].finish_reason ~= nil)
|
||||
})
|
||||
end
|
||||
|
||||
return adapter
|
||||
```
|
||||
|
||||
Corresponding `config.yaml` source:
|
||||
|
||||
```yaml
|
||||
sources:
|
||||
- name: mysrc
|
||||
base_url: https://upstream.example.com
|
||||
api_key: sk-xxx
|
||||
adapter: mysrc # or omit = bundled adapter with the same name
|
||||
endpoint: /chat/completions
|
||||
models:
|
||||
- id: my-model
|
||||
priority: 50
|
||||
kind: chat
|
||||
```
|
||||
|
||||
## 8. Multimodal & disable_thinking
|
||||
|
||||
- **Multimodal**: if the request `content` is an array
|
||||
(`[{type:"text"...},{type:"image_url"...}]`), the Go layer passes it through
|
||||
verbatim to the adapter. Sources whose protocol doesn't support it (Anthropic /
|
||||
Gemini / Ollama) must convert inside `transform_request`; the bundled
|
||||
`anthropic.lua` / `gemini.lua` / `ollama.lua` already do.
|
||||
- **disable_thinking**: the gateway passes `disable_thinking:true` from the request
|
||||
body into `transform_request`. The DeepSeek adapter maps it to
|
||||
`extra_body.thinking = { type = "disabled" }` and cleans up its own field; other
|
||||
adapters handle it per their protocol (implement or ignore as needed).
|
||||
257
docs/lua-adapters.md
Normal file
257
docs/lua-adapters.md
Normal file
@ -0,0 +1,257 @@
|
||||
# Lua 适配器 API 文档
|
||||
|
||||
> **English**: [lua-adapters-en.md](./lua-adapters-en.md)
|
||||
|
||||
每个上游源（source）挂载一个 `.lua` 适配器，负责把统一 OpenAI 格式请求**转换**为上游原生
|
||||
格式，并把上游响应/流式分块**转换**回统一格式。Go 层只负责调度、并发与透传——因此**新增
|
||||
适配器、适配新协议，无需重编译 Go**。
|
||||
|
||||
适配器放两份（同构）：
|
||||
- 内置：`internal/lua/adapters/<name>.lua`（编译期 embed）
|
||||
- 可覆盖：配置 `adapter_dir` 目录下的同名脚本（优先级更高）
|
||||
|
||||
> **加载时机**：启动时由 `lua.NewVM(adapter_dir)` 一次性加载全部适配器；通过 WebUI
|
||||
> 上传的适配器（`Core.UploadAdapter` → `vm.LoadAdapter`）与在线编辑的源配置即时生效，
|
||||
> 无需重启。直接修改 `adapter_dir` 下的 `.lua` 文件需要重启进程才会重新加载。
|
||||
|
||||
## 目录
|
||||
|
||||
1. [脚本结构](#1-脚本结构)
|
||||
2. [静态字段](#2-静态字段)
|
||||
3. [转换钩子](#3-转换钩子)
|
||||
4. [可选钩子](#4-可选钩子)
|
||||
5. [内置辅助函数](#5-内置辅助函数)
|
||||
6. [meta 约定](#6-meta-约定)
|
||||
7. [一个完整的最小适配器](#7-一个完整的最小适配器)
|
||||
8. [多模态与 disable_thinking](#8-多模态与-disable_thinking)
|
||||
|
||||
---
|
||||
|
||||
## 1. 脚本结构
|
||||
|
||||
脚本是一个返回 table 的 Lua 文件，必须 `return` 一个表：
|
||||
|
||||
```lua
|
||||
local adapter = {}
|
||||
adapter.name = "mysrc"
|
||||
adapter.version = "1.0.0"
|
||||
-- ... 字段与函数 ...
|
||||
return adapter
|
||||
```
|
||||
|
||||
脚本由 LuaJIT 解析（golua 绑定，cgo）。每个适配器拥有**独立 VM + worker 池**，适配器之间
|
||||
互不干扰；脚本中不允许跨 worker 共享可变状态（`log` 到 stdout 是唯一副作用）。
|
||||
|
||||
## 2. 静态字段
|
||||
|
||||
| 字段 | 类型 | 必填 | 说明 |
|
||||
|------|------|------|------|
|
||||
| `name` | string | 是 | 适配器名，用于 WebUI 展示 |
|
||||
| `version` | string | 否 | 版本号，用于 WebUI 展示 |
|
||||
| `endpoint` | string | 否 | 上游请求路径，默认 `/chat/completions`；可被 `source.endpoint` / `source.image_endpoint` 覆盖 |
|
||||
| `headers` | table | 否 | 静态默认请求头；若未定义 `build_headers` 钩子则作为请求头回退 |
|
||||
|
||||
这些字段在加载时静态提取（compile-once），之后读它们不会占用池内 worker。
|
||||
|
||||
## 3. 转换钩子
|
||||
|
||||
### `transform_request(raw_body) -> string`
|
||||
|
||||
入参：Go 层构造的统一 OpenAI 格式 JSON 字符串（`/v1/chat/completions` 请求体）。
|
||||
返回：发送给上游的请求体字符串。
|
||||
|
||||
```lua
|
||||
function adapter.transform_request(raw_body)
|
||||
local ok, req = pcall(json.decode, raw_body)
|
||||
if not ok then return raw_body end
|
||||
req.model = "upstream-model-name" -- 改写模型名
|
||||
req.stream = req.stream or false
|
||||
return json.encode(req)
|
||||
end
|
||||
```
|
||||
|
||||
约定：
|
||||
- 返回的字符串将被原样 POST 到 `base_url + (source.endpoint or adapter.endpoint)`。
|
||||
- 请求体中包含的 `disable_thinking`、`extra_body` 等网关内部字段，转换时应自行清理
|
||||
（`openai.lua` 即删掉这两个字段）。
|
||||
|
||||
### `transform_response(raw_body) -> string`
|
||||
|
||||
入参：上游非流式响应的原始 JSON 字符串。返回：统一格式 JSON 字符串。
|
||||
|
||||
统一格式字段：
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `content` | string | 回复正文 |
|
||||
| `finish_reason` | string | `stop` / `length` / `tool_calls` 等 |
|
||||
| `reasoning_content` | string（可选） | 推理内容（DeepSeek 等） |
|
||||
| `token_usage` | table | `{ prompt, completion, total }`（tokens） |
|
||||
| `tool_calls` | array（可选） | 工具调用：`{ id, type, name, arguments }`，`arguments` 为**已解码**的 table |
|
||||
|
||||
```lua
|
||||
function adapter.transform_response(raw_body)
|
||||
local ok, resp = pcall(json.decode, raw_body)
|
||||
if not ok or resp == nil then return raw_body end
|
||||
local unified = {
|
||||
content = "",
|
||||
finish_reason = "",
|
||||
token_usage = { prompt = 0, completion = 0, total = 0 }
|
||||
}
|
||||
if type(resp.choices) == "table" and #resp.choices > 0 then
|
||||
local ch = resp.choices[1]
|
||||
unified.content = ch.message.content or ""
|
||||
unified.finish_reason = ch.finish_reason or ""
|
||||
if ch.message.reasoning_content then
|
||||
unified.reasoning_content = ch.message.reasoning_content
|
||||
end
|
||||
end
|
||||
return json.encode(unified)
|
||||
end
|
||||
```
|
||||
|
||||
### `transform_stream_chunk(raw_chunk) -> string`
|
||||
|
||||
入参：SSE 流中每条 `data:` 的原始 JSON 字符串（不含 `data:` 前缀）。
|
||||
返回：统一分块 JSON；**返回 `""` 表示跳过该 chunk**。
|
||||
|
||||
统一分块格式：
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `content` | string | 本次增量文本（可为 `""`） |
|
||||
| `done` | boolean（可选） | `true` 表示流结束（对应 `finish_reason` 出现） |
|
||||
|
||||
```lua
|
||||
function adapter.transform_stream_chunk(raw_chunk)
|
||||
local ok, chunk = pcall(json.decode, raw_chunk)
|
||||
if not ok then return "" end
|
||||
if not chunk.choices or #chunk.choices == 0 then return "" end
|
||||
local delta = chunk.choices[1].delta or {}
|
||||
local fr = chunk.choices[1].finish_reason
|
||||
return json.encode({
|
||||
content = delta.content or "",
|
||||
done = (fr ~= nil)
|
||||
})
|
||||
end
|
||||
```
|
||||
|
||||
## 4. 可选钩子
|
||||
|
||||
### `build_headers(meta) -> table<string,string>`
|
||||
|
||||
动态生成/签名请求头（如 KimiCode 的 HMAC 签名）。若脚本未定义此函数，Go 层回退使用静态
|
||||
`adapter.headers`（或 `source.headers`）。
|
||||
|
||||
```lua
|
||||
function adapter.build_headers(meta)
|
||||
local msg = meta.method .. meta.url .. meta.body
|
||||
return {
|
||||
["X-App-Sign"] = hmac_sha256_hex(meta.source.meta.app_secret, msg),
|
||||
["X-Timestamp"] = meta.timestamp,
|
||||
}
|
||||
end
|
||||
```
|
||||
|
||||
返回非 table 会报错；返回 `{}` 表示无自定义头（不会回退静态头）。
|
||||
|
||||
## 5. 内置辅助函数
|
||||
|
||||
所有适配器共享的全局函数（由 Go 注入）：
|
||||
|
||||
| 函数 | 说明 |
|
||||
|------|------|
|
||||
| `json.encode(v)` | Lua 值 → JSON 字符串；失败返回 `"null"` |
|
||||
| `json.decode(s)` | JSON 字符串 → Lua 值；失败返回 `nil` |
|
||||
| `hmac_sha256_hex(key, data)` | HMAC-SHA256，十六进制小写字符串 |
|
||||
| `sha256_hex(data)` | SHA-256，十六进制小写字符串 |
|
||||
| `base64_encode(s)` | 标准 Base64 编码 |
|
||||
| `tohex(s)` | 字节 → 十六进制小写字符串 |
|
||||
| `log(level, msg)` | 打印 `[adapter/<name>] <msg>` 到 stdout |
|
||||
|
||||
另注：脚本由 LuaJIT 执行，标准库（`string`/`table`/`pcall` 等）完整可用，`os`/`io` 未暴露
|
||||
（保持沙箱语义）。
|
||||
|
||||
## 6. meta 约定
|
||||
|
||||
`build_headers(meta)` 与请求转换共享的 meta 结构：
|
||||
|
||||
| 字段 | 说明 |
|
||||
|------|------|
|
||||
| `meta.url` | 完整请求 URL |
|
||||
| `meta.method` | HTTP 方法（通常 `POST`） |
|
||||
| `meta.body` | 请求体字符串 |
|
||||
| `meta.api_key` | 该源的 `api_key` |
|
||||
| `meta.timestamp` | 请求时间戳 |
|
||||
| `meta.source` | table：`{ name, meta = { ... } }`，即 source 的 `meta` 字段（如 `app_secret`） |
|
||||
|
||||
## 7. 一个完整的最小适配器
|
||||
|
||||
```lua
|
||||
local adapter = {}
|
||||
|
||||
adapter.name = "mysrc"
|
||||
adapter.version = "1.0.0"
|
||||
adapter.endpoint = "/chat/completions"
|
||||
adapter.headers = { ["X-Tenant"] = "prod" }
|
||||
|
||||
function adapter.transform_request(raw_body)
|
||||
local ok, req = pcall(json.decode, raw_body)
|
||||
if not ok then return raw_body end
|
||||
req.disable_thinking = nil
|
||||
req.extra_body = nil
|
||||
return json.encode(req)
|
||||
end
|
||||
|
||||
function adapter.transform_response(raw_body)
|
||||
local ok, resp = pcall(json.decode, raw_body)
|
||||
if not ok or resp == nil then return raw_body end
|
||||
local unified = {
|
||||
content = "",
|
||||
finish_reason = "",
|
||||
token_usage = { prompt = 0, completion = 0, total = 0 }
|
||||
}
|
||||
if type(resp.choices) == "table" and #resp.choices > 0 then
|
||||
local ch = resp.choices[1]
|
||||
unified.content = ch.message.content or ""
|
||||
unified.finish_reason = ch.finish_reason or ""
|
||||
end
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
function adapter.transform_stream_chunk(raw_chunk)
|
||||
local ok, chunk = pcall(json.decode, raw_chunk)
|
||||
if not ok then return "" end
|
||||
if not chunk.choices or #chunk.choices == 0 then return "" end
|
||||
return json.encode({
|
||||
content = (chunk.choices[1].delta or {}).content or "",
|
||||
done = (chunk.choices[1].finish_reason ~= nil)
|
||||
})
|
||||
end
|
||||
|
||||
return adapter
|
||||
```
|
||||
|
||||
对应的 `config.yaml` source：
|
||||
|
||||
```yaml
|
||||
sources:
|
||||
- name: mysrc
|
||||
base_url: https://upstream.example.com
|
||||
api_key: sk-xxx
|
||||
adapter: mysrc # 或省略 = 内置同名适配器
|
||||
endpoint: /chat/completions
|
||||
models:
|
||||
- id: my-model
|
||||
priority: 50
|
||||
kind: chat
|
||||
```
|
||||
|
||||
## 8. 多模态与 disable_thinking
|
||||
|
||||
- **多模态**：请求的 `content` 若是数组（`[{type:"text"...},{type:"image_url"...}]`），Go 层
|
||||
原样透传给适配器。协议不支持的源（Anthropic/Gemini/Ollama）需在
|
||||
`transform_request` 内转换；内置 `anthropic.lua`/`gemini.lua`/`ollama.lua` 已实现。
|
||||
- **disable_thinking**：网关将请求体中的 `disable_thinking:true` 透传进
|
||||
`transform_request`。DeepSeek 适配器将其映射为 `extra_body.thinking = { type = "disabled" }`
|
||||
并清理自身字段；其余适配器按各自协议处理（可自行实现或忽略）。
|
||||
@ -44,10 +44,10 @@ type ChatChoice struct {
|
||||
}
|
||||
|
||||
type RespMessage struct {
|
||||
Role string `json:"role"`
|
||||
Content string `json:"content"`
|
||||
ReasoningContent string `json:"reasoning_content,omitempty"`
|
||||
ToolCalls []types.ToolCall `json:"tool_calls,omitempty"`
|
||||
Role string `json:"role,omitempty"`
|
||||
Content string `json:"content"`
|
||||
ReasoningContent string `json:"reasoning_content,omitempty"`
|
||||
ToolCalls json.RawMessage `json:"tool_calls,omitempty"`
|
||||
}
|
||||
|
||||
type ChatChunk struct {
|
||||
@ -77,13 +77,49 @@ func isAuto(m string) bool {
|
||||
}
|
||||
|
||||
// resolveCands picks the ordered candidate providers for a requested model.
|
||||
func (g *Gateway) resolveCands(model string) ([]*provider.Provider, string) {
|
||||
if model == "" || isAuto(model) {
|
||||
// toolCalling requests are anchored: they resolve to exactly one provider
|
||||
// (highest-priority available) so a tool-call round never switches models.
|
||||
func (g *Gateway) resolveCands(req *chatRequest) ([]*provider.Provider, string) {
|
||||
model := req.Model
|
||||
if model == "" {
|
||||
model = g.core.DefaultModel()
|
||||
}
|
||||
cands, effective := g.resolveByModel(model)
|
||||
if !toolRequest(req) {
|
||||
return cands, effective
|
||||
}
|
||||
// tool-call request: pin to one provider (no AUTO fallback across models)
|
||||
if len(cands) == 0 {
|
||||
return nil, effective
|
||||
}
|
||||
first := cands[0]
|
||||
eff := first.ModelFor(model)
|
||||
if eff == "" {
|
||||
eff = firstModel(first)
|
||||
}
|
||||
return []*provider.Provider{first}, eff
|
||||
}
|
||||
|
||||
func (g *Gateway) resolveByModel(model string) ([]*provider.Provider, string) {
|
||||
if isAuto(model) {
|
||||
return g.core.Registry().Resolve("AUTO"), ""
|
||||
}
|
||||
return g.core.Registry().Resolve(model), model
|
||||
}
|
||||
|
||||
// toolRequest reports whether the request participates in a tool-call round.
|
||||
func toolRequest(req *chatRequest) bool {
|
||||
if len(req.Tools) > 0 || req.ToolChoice != nil {
|
||||
return true
|
||||
}
|
||||
for _, m := range req.Messages {
|
||||
if m.Role == "tool" || len(m.ToolCalls) > 0 {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (g *Gateway) handleChat(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodPost {
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use POST")
|
||||
@ -102,7 +138,7 @@ func (g *Gateway) handleChat(w http.ResponseWriter, r *http.Request) {
|
||||
if model == "" {
|
||||
model = g.core.DefaultModel()
|
||||
}
|
||||
cands, effective := g.resolveCands(model)
|
||||
cands, effective := g.resolveCands(&req)
|
||||
if len(cands) == 0 {
|
||||
writeError(w, http.StatusServiceUnavailable, "no_provider", "no LLM source configured")
|
||||
return
|
||||
@ -159,6 +195,31 @@ func imageOnly(cands []*provider.Provider) []*provider.Provider {
|
||||
return out
|
||||
}
|
||||
|
||||
// toolCallsWire converts unified tool calls to the OpenAI wire format:
|
||||
// tool_calls:[{id,type,function:{name,arguments:StringJSON}}]. Clients expect
|
||||
// arguments to be a JSON string, not an object.
|
||||
func toolCallsWire(tcs []types.ToolCall) json.RawMessage {
|
||||
wire := make([]map[string]interface{}, 0, len(tcs))
|
||||
for _, tc := range tcs {
|
||||
args := "{}"
|
||||
if tc.Arguments != nil {
|
||||
if b, err := json.Marshal(tc.Arguments); err == nil {
|
||||
args = string(b)
|
||||
}
|
||||
}
|
||||
wire = append(wire, map[string]interface{}{
|
||||
"id": tc.ID,
|
||||
"type": tc.Type,
|
||||
"function": map[string]interface{}{
|
||||
"name": tc.Name,
|
||||
"arguments": args,
|
||||
},
|
||||
})
|
||||
}
|
||||
b, _ := json.Marshal(wire)
|
||||
return b
|
||||
}
|
||||
|
||||
func (g *Gateway) singleChat(w http.ResponseWriter, ctx context.Context, cands []*provider.Provider, req *types.ChatRequest, effective string) {
|
||||
resp, err := g.core.Scheduler().Chat(ctx, scheduler.FromRegistry(cands), req)
|
||||
if err != nil {
|
||||
@ -170,7 +231,7 @@ func (g *Gateway) singleChat(w http.ResponseWriter, ctx context.Context, cands [
|
||||
msg.ReasoningContent = resp.ReasoningContent
|
||||
}
|
||||
if len(resp.ToolCalls) > 0 {
|
||||
msg.ToolCalls = resp.ToolCalls
|
||||
msg.ToolCalls = toolCallsWire(resp.ToolCalls)
|
||||
}
|
||||
out := ChatCompletion{
|
||||
ID: newID(),
|
||||
@ -223,7 +284,7 @@ func (g *Gateway) streamChat(w http.ResponseWriter, ctx context.Context, cands [
|
||||
chunk := ChatChunk{
|
||||
ID: id, Object: "chat.completion.chunk", Created: created, Model: effective,
|
||||
}
|
||||
delta := RespMessage{Content: ck.Content}
|
||||
delta := RespMessage{Role: "assistant", Content: ck.Content}
|
||||
if ck.ReasoningContent != "" {
|
||||
delta.ReasoningContent = ck.ReasoningContent
|
||||
}
|
||||
@ -269,7 +330,7 @@ func (g *Gateway) handleImage(w http.ResponseWriter, r *http.Request) {
|
||||
if model == "" {
|
||||
model = g.core.DefaultModel()
|
||||
}
|
||||
cands, _ := g.resolveCands(model)
|
||||
cands, _ := g.resolveByModel(model)
|
||||
cands = imageOnly(cands)
|
||||
if len(cands) == 0 {
|
||||
writeError(w, http.StatusServiceUnavailable, "no_provider", "no image source configured")
|
||||
|
||||
@ -155,6 +155,50 @@ html[data-theme="dark"] .dropzone.dragover, html[data-theme="dark"] .dropzone:ho
|
||||
.empty { color:var(--muted); text-align:center; padding:24px 0; }
|
||||
#modal-wrap { position:fixed; inset:0; background:rgba(15,22,44,.45); display:flex; align-items:flex-start;
|
||||
justify-content:center; overflow:auto; padding:48px 20px; z-index:50; }
|
||||
.twrap { overflow-x:auto; -webkit-overflow-scrolling:touch; }
|
||||
|
||||
/* ---------- responsive / narrow screens ---------- */
|
||||
@media (max-width: 900px) {
|
||||
header { padding:12px 16px; }
|
||||
nav { padding:12px 16px 0; overflow-x:auto; }
|
||||
nav button { padding:7px 12px; white-space:nowrap; }
|
||||
main { padding:16px 16px 40px; }
|
||||
.card { padding:16px; }
|
||||
}
|
||||
@media (max-width: 640px) {
|
||||
header { gap:8px; padding:10px 12px; }
|
||||
.brand h1 { font-size:15px; }
|
||||
.brand .sub { display:none; }
|
||||
.hd-actions button { padding:5px 9px; }
|
||||
nav { gap:4px; padding:10px 12px 0; }
|
||||
nav button { padding:6px 10px; font-size:13px; }
|
||||
main { padding:12px 12px 32px; }
|
||||
.card { padding:13px; border-radius:12px; margin-bottom:14px; }
|
||||
.card h2 { font-size:13px; }
|
||||
th,td { padding:8px 10px; }
|
||||
.row { flex-direction:column; gap:0; }
|
||||
.model-row { flex-wrap:wrap; }
|
||||
.model-row input { flex:1 1 120px; }
|
||||
.model-row select { flex:0 0 auto; }
|
||||
.tab-chat { height:calc(100vh - 150px); }
|
||||
.msg { gap:7px; }
|
||||
.avatar { width:24px; height:24px; font-size:11px; }
|
||||
.bubble { max-width:90%; padding:8px 11px; font-size:13px; }
|
||||
.chat-log { padding:14px 12px 6px; gap:14px; }
|
||||
.chat-tools { flex-wrap:wrap; gap:8px; }
|
||||
.chat-tools .tl { display:none; }
|
||||
.chat-tools select { flex:1 1 auto; min-width:0; }
|
||||
.chat-box { gap:7px; }
|
||||
.chat-box .sendbtn { padding:10px 13px; }
|
||||
.attach-btn { width:38px; height:38px; }
|
||||
.chat-box textarea { font-size:13.5px; }
|
||||
.chat-composer { padding:8px 9px 10px; }
|
||||
#modal-wrap { padding:14px 10px; }
|
||||
.dropzone { padding:18px 14px; }
|
||||
#toast { left:12px; right:12px; bottom:12px; text-align:center; }
|
||||
pre.configbox { font-size:11.5px; padding:11px; }
|
||||
.att img { height:52px; max-width:90px; }
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
|
||||
@ -103,12 +103,41 @@ function adapter.transform_stream_chunk(raw_chunk)
|
||||
if chunk.type == "message_delta" then
|
||||
return json.encode({ content = "", done = (chunk.delta and chunk.delta.stop_reason ~= nil) })
|
||||
end
|
||||
if chunk.type == "content_block_start" and chunk.content_block
|
||||
and chunk.content_block.type == "tool_use" then
|
||||
-- first fragment of a tool call: emit index + id + name, empty args
|
||||
return json.encode({
|
||||
content = "", done = false,
|
||||
tool_calls = { {
|
||||
index = chunk.index or 0,
|
||||
id = chunk.content_block.id or "",
|
||||
type = "function",
|
||||
["function"] = { name = chunk.content_block.name or "", arguments = "" }
|
||||
} }
|
||||
})
|
||||
end
|
||||
if chunk.type == "content_block_delta" and chunk.delta then
|
||||
if chunk.delta.type == "input_json_delta" then
|
||||
-- incremental JSON fragment; clients accumulate across chunks
|
||||
local unified = { content = "", done = false, tool_calls = { {
|
||||
index = chunk.index or 0,
|
||||
id = "",
|
||||
type = "function",
|
||||
["function"] = { name = "", arguments = chunk.delta.partial_json or "" }
|
||||
} } }
|
||||
return json.encode(unified)
|
||||
end
|
||||
if chunk.delta.type == "thinking_delta" and chunk.delta.thinking then
|
||||
return json.encode({ content = "", done = false, reasoning_content = chunk.delta.thinking })
|
||||
end
|
||||
return json.encode({ content = chunk.delta.text or "", done = false })
|
||||
end
|
||||
if chunk.type == "message_stop" then
|
||||
return json.encode({ content = "", done = true })
|
||||
end
|
||||
if chunk.type == "content_block_stop" then
|
||||
return json.encode({ content = "", done = false })
|
||||
end
|
||||
return ""
|
||||
end
|
||||
|
||||
|
||||
@ -20,6 +20,18 @@ function adapter.transform_request(raw_body)
|
||||
req.extra_body.thinking = { type = "disabled" }
|
||||
end
|
||||
req.disable_thinking = nil
|
||||
|
||||
-- V4 thinking 模式要求：带 tool_calls 的 assistant 消息必须回传 reasoning_content。
|
||||
-- OpenAI 兼容客户端不会发该字段，补空串即可通过校验。
|
||||
if req.messages then
|
||||
for _, msg in ipairs(req.messages) do
|
||||
if msg.role == "assistant" and msg.tool_calls and msg.tool_calls[1] then
|
||||
if msg.reasoning_content == nil then
|
||||
msg.reasoning_content = ""
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
return json.encode(req)
|
||||
end
|
||||
|
||||
@ -73,10 +85,18 @@ function adapter.transform_stream_chunk(raw_chunk)
|
||||
if not chunk.choices or #chunk.choices == 0 then return "" end
|
||||
local delta = chunk.choices[1].delta or {}
|
||||
local fr = chunk.choices[1].finish_reason
|
||||
return json.encode({
|
||||
|
||||
local unified = {
|
||||
content = delta.content or "",
|
||||
done = (fr ~= nil)
|
||||
})
|
||||
}
|
||||
if delta.reasoning_content then
|
||||
unified.reasoning_content = delta.reasoning_content
|
||||
end
|
||||
if delta.tool_calls then
|
||||
unified.tool_calls = delta.tool_calls
|
||||
end
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
return adapter
|
||||
|
||||
@ -93,16 +93,31 @@ function adapter.transform_stream_chunk(raw_chunk)
|
||||
|
||||
if not chunk.candidates or #chunk.candidates == 0 then return "" end
|
||||
local cand = chunk.candidates[1]
|
||||
local content = ""
|
||||
local unified = { content = "", done = (cand.finishReason ~= nil) }
|
||||
local reasoning = ""
|
||||
local tools = {}
|
||||
if cand.content and cand.content.parts then
|
||||
for _, part in ipairs(cand.content.parts) do
|
||||
content = content .. (part.text or "")
|
||||
if part.text then
|
||||
unified.content = (unified.content or "") .. part.text
|
||||
elseif part.reasoning_content then
|
||||
reasoning = reasoning .. part.reasoning_content
|
||||
elseif part.functionCall then
|
||||
table.insert(tools, {
|
||||
index = #tools,
|
||||
id = part.functionCall.id or ("call_" .. #tools),
|
||||
type = "function",
|
||||
["function"] = {
|
||||
name = part.functionCall.name or "",
|
||||
arguments = part.functionCall.args or "{}"
|
||||
}
|
||||
})
|
||||
end
|
||||
end
|
||||
end
|
||||
return json.encode({
|
||||
content = content,
|
||||
done = (cand.finishReason ~= nil)
|
||||
})
|
||||
if reasoning ~= "" then unified.reasoning_content = reasoning end
|
||||
if #tools > 0 then unified.tool_calls = tools end
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
return adapter
|
||||
|
||||
@ -66,10 +66,18 @@ function adapter.transform_stream_chunk(raw_chunk)
|
||||
if not chunk.choices or #chunk.choices == 0 then return "" end
|
||||
local delta = chunk.choices[1].delta or {}
|
||||
local fr = chunk.choices[1].finish_reason
|
||||
return json.encode({
|
||||
|
||||
local unified = {
|
||||
content = delta.content or "",
|
||||
done = (fr ~= nil)
|
||||
})
|
||||
}
|
||||
if delta.reasoning_content then
|
||||
unified.reasoning_content = delta.reasoning_content
|
||||
end
|
||||
if delta.tool_calls then
|
||||
unified.tool_calls = delta.tool_calls
|
||||
end
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
return adapter
|
||||
|
||||
@ -65,10 +65,18 @@ function adapter.transform_stream_chunk(raw_chunk)
|
||||
if not chunk.choices or #chunk.choices == 0 then return "" end
|
||||
local delta = chunk.choices[1].delta or {}
|
||||
local fr = chunk.choices[1].finish_reason
|
||||
return json.encode({
|
||||
|
||||
local unified = {
|
||||
content = delta.content or "",
|
||||
done = (fr ~= nil)
|
||||
})
|
||||
}
|
||||
if delta.reasoning_content then
|
||||
unified.reasoning_content = delta.reasoning_content
|
||||
end
|
||||
if delta.tool_calls then
|
||||
unified.tool_calls = delta.tool_calls
|
||||
end
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
return adapter
|
||||
|
||||
@ -98,10 +98,18 @@ function adapter.transform_stream_chunk(raw_chunk)
|
||||
if not chunk.choices or #chunk.choices == 0 then return "" end
|
||||
local delta = chunk.choices[1].delta or {}
|
||||
local fr = chunk.choices[1].finish_reason
|
||||
return json.encode({
|
||||
|
||||
local unified = {
|
||||
content = delta.content or "",
|
||||
done = (fr ~= nil)
|
||||
})
|
||||
}
|
||||
if delta.reasoning_content then
|
||||
unified.reasoning_content = delta.reasoning_content
|
||||
end
|
||||
if delta.tool_calls then
|
||||
unified.tool_calls = delta.tool_calls
|
||||
end
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
return adapter
|
||||
@ -65,10 +65,18 @@ function adapter.transform_stream_chunk(raw_chunk)
|
||||
if not chunk.choices or #chunk.choices == 0 then return "" end
|
||||
local delta = chunk.choices[1].delta or {}
|
||||
local fr = chunk.choices[1].finish_reason
|
||||
return json.encode({
|
||||
|
||||
local unified = {
|
||||
content = delta.content or "",
|
||||
done = (fr ~= nil)
|
||||
})
|
||||
}
|
||||
if delta.reasoning_content then
|
||||
unified.reasoning_content = delta.reasoning_content
|
||||
end
|
||||
if delta.tool_calls then
|
||||
unified.tool_calls = delta.tool_calls
|
||||
end
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
return adapter
|
||||
|
||||
@ -72,10 +72,29 @@ function adapter.transform_stream_chunk(raw_chunk)
|
||||
if not ok then return "" end
|
||||
if not chunk.message then return "" end
|
||||
|
||||
return json.encode({
|
||||
local unified = {
|
||||
content = chunk.message.content or "",
|
||||
done = chunk.done or false
|
||||
})
|
||||
}
|
||||
if chunk.message.reasoning_content then
|
||||
unified.reasoning_content = chunk.message.reasoning_content
|
||||
end
|
||||
if chunk.message.tool_calls then
|
||||
local tools = {}
|
||||
for _, tc in ipairs(chunk.message.tool_calls) do
|
||||
table.insert(tools, {
|
||||
index = #tools,
|
||||
id = tc.id or ("call_" .. #tools),
|
||||
type = "function",
|
||||
["function"] = {
|
||||
name = tc["function"] and tc["function"].name or "",
|
||||
arguments = tc["function"] and (tc["function"].arguments or "{}") or "{}"
|
||||
}
|
||||
})
|
||||
end
|
||||
unified.tool_calls = tools
|
||||
end
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
return adapter
|
||||
|
||||
@ -71,10 +71,18 @@ function adapter.transform_stream_chunk(raw_chunk)
|
||||
local delta = chunk.choices[1].delta or {}
|
||||
local fr = chunk.choices[1].finish_reason
|
||||
|
||||
return json.encode({
|
||||
local unified = {
|
||||
content = delta.content or "",
|
||||
done = (fr ~= nil)
|
||||
})
|
||||
}
|
||||
if delta.reasoning_content then
|
||||
unified.reasoning_content = delta.reasoning_content
|
||||
end
|
||||
if delta.tool_calls then
|
||||
-- pass raw streaming fragments through; OpenAI clients accumulate index+id+name+arguments
|
||||
unified.tool_calls = delta.tool_calls
|
||||
end
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
return adapter
|
||||
|
||||
@ -100,6 +100,43 @@ func (p *Provider) ModelByID(id string) *config.Model {
|
||||
return nil
|
||||
}
|
||||
|
||||
// ModelFor resolves the model name this provider should send upstream.
|
||||
// If the requested model is not owned by this provider (e.g. an AUTO chain
|
||||
// fallback), it returns this provider's highest-priority chat model instead.
|
||||
func (p *Provider) ModelFor(reqModel string) string {
|
||||
if reqModel == "" || isAutoID(reqModel) {
|
||||
return p.bestChatModel()
|
||||
}
|
||||
if p.ModelByID(reqModel) != nil {
|
||||
return reqModel
|
||||
}
|
||||
return p.bestChatModel()
|
||||
}
|
||||
|
||||
// bestChatModel returns the highest-priority chat-kind model of this source.
|
||||
func (p *Provider) bestChatModel() string {
|
||||
bestID, bestPrio := "", -1
|
||||
for _, m := range p.cfg.Models {
|
||||
if m.Kind != "" && m.Kind != "chat" {
|
||||
continue
|
||||
}
|
||||
if m.Priority > bestPrio {
|
||||
bestPrio = m.Priority
|
||||
bestID = m.ID
|
||||
}
|
||||
}
|
||||
if bestID == "" && len(p.cfg.Models) > 0 {
|
||||
bestID = p.cfg.Models[0].ID
|
||||
}
|
||||
return bestID
|
||||
}
|
||||
|
||||
// IsAutoID reports whether s is an AUTO routing placeholder.
|
||||
func isAutoID(s string) bool {
|
||||
s = strings.TrimSpace(s)
|
||||
return s == "" || strings.EqualFold(s, "AUTO")
|
||||
}
|
||||
|
||||
// Endpoint resolves the upstream chat path.
|
||||
func (p *Provider) Endpoint() string {
|
||||
if p.cfg.Endpoint != "" {
|
||||
|
||||
@ -28,6 +28,7 @@ func New(maxRetries int) *Scheduler {
|
||||
type Provider interface {
|
||||
Name() string
|
||||
Available() bool
|
||||
ModelFor(reqModel string) string
|
||||
Chat(ctx context.Context, req *types.ChatRequest) (*types.UnifiedResponse, error)
|
||||
ChatStream(ctx context.Context, req *types.ChatRequest) (<-chan types.UnifiedChunk, error)
|
||||
Image(ctx context.Context, req *types.ImageGenRequest) (*types.UnifiedResponse, error)
|
||||
@ -42,13 +43,18 @@ func FromRegistry(ps []*provider.Provider) []Provider {
|
||||
return out
|
||||
}
|
||||
|
||||
// Chat runs a chat request across cands, falling back on failure.
|
||||
// Chat runs a chat request across cands, falling back on failure. Each
|
||||
// candidate receives a request pinned to its own model (ModelFor), so an AUTO
|
||||
// chain fallback switches the model id per provider instead of reusing the
|
||||
// first candidate's model name.
|
||||
func (s *Scheduler) Chat(ctx context.Context, cands []Provider, req *types.ChatRequest) (*types.UnifiedResponse, error) {
|
||||
attempts := s.MaxRetries + 1
|
||||
var lastErr error
|
||||
for i := 0; i < attempts && i < len(cands); i++ {
|
||||
p := cands[i]
|
||||
resp, err := p.Chat(ctx, req)
|
||||
r := *req
|
||||
r.Model = p.ModelFor(req.Model)
|
||||
resp, err := p.Chat(ctx, &r)
|
||||
if ctx.Err() != nil {
|
||||
return nil, ctx.Err()
|
||||
}
|
||||
@ -68,13 +74,16 @@ func (s *Scheduler) Chat(ctx context.Context, cands []Provider, req *types.ChatR
|
||||
return nil, lastErr
|
||||
}
|
||||
|
||||
// ChatStream runs a streaming chat across cands, falling back early on connect errors.
|
||||
// ChatStream runs a streaming chat across cands, falling back early on connect
|
||||
// errors. The request model is pinned per candidate like Chat.
|
||||
func (s *Scheduler) ChatStream(ctx context.Context, cands []Provider, req *types.ChatRequest) (<-chan types.UnifiedChunk, error) {
|
||||
attempts := s.MaxRetries + 1
|
||||
var lastErr error
|
||||
for i := 0; i < attempts && i < len(cands); i++ {
|
||||
p := cands[i]
|
||||
resp, err := p.ChatStream(ctx, req)
|
||||
r := *req
|
||||
r.Model = p.ModelFor(req.Model)
|
||||
resp, err := p.ChatStream(ctx, &r)
|
||||
if err == nil {
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
@ -47,7 +47,7 @@ type ChatMessage struct {
|
||||
Content json.RawMessage `json:"content,omitempty"`
|
||||
ReasoningContent string `json:"reasoning_content,omitempty"`
|
||||
ToolCallID string `json:"tool_call_id,omitempty"`
|
||||
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
|
||||
ToolCalls json.RawMessage `json:"tool_calls,omitempty"`
|
||||
}
|
||||
|
||||
func StringContent(s string) json.RawMessage { b, _ := json.Marshal(s); return b }
|
||||
@ -100,11 +100,14 @@ type ImageGenResponse struct {
|
||||
|
||||
// ---- Unified streaming chunk produced by adapters ----
|
||||
|
||||
// UnifiedChunk is one streamed delta. ToolCalls carries the raw upstream
|
||||
// streaming tool_calls array (incremental fragments with an index field), which
|
||||
// OpenAI-compatible clients accumulate themselves.
|
||||
type UnifiedChunk struct {
|
||||
Content string `json:"content"`
|
||||
Done bool `json:"done"`
|
||||
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
|
||||
ReasoningContent string `json:"reasoning_content,omitempty"`
|
||||
Content string `json:"content"`
|
||||
Done bool `json:"done"`
|
||||
ToolCalls json.RawMessage `json:"tool_calls,omitempty"`
|
||||
ReasoningContent string `json:"reasoning_content,omitempty"`
|
||||
}
|
||||
|
||||
// Meta passed to Lua build_headers hook
|
||||
|
||||
Reference in New Issue
Block a user