jianf/ModelRouter
1
0
Fork 0
mirror of https://gitcode.com/JianFeeeee/ModelRouter.git synced 2026-09-20 00:48:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: tool call anchor & wire format, streaming chunk passthrough, WebUI narrow-screen, docs bilingual

Browse Source
This commit is contained in:
root
2026-08-08 11:26:58 +08:00
parent 8e9de95274
commit 5d50b69153
18 changed files with 1119 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files

271
docs/lua-adapters-en.md Normal file
View File

@ -0,0 +1,271 @@
# Lua Adapter API
> **中文**: [lua-adapters.md](./lua-adapters.md)
Every upstream source is mounted with a `.lua` adapter that **transforms** the unified
OpenAI-format request into the upstream's native format, and converts upstream
responses / stream chunks back into the unified format. The Go layer only handles
scheduling, concurrency and pass-through — so **adding an adapter or adapting a new
protocol never requires recompiling Go**.
Adapters live in two equivalent places:
- Bundled: `internal/lua/adapters/<name>.lua` (embedded at build time)
- Override: a same-named script in the configured `adapter_dir` (takes precedence)
> **When are they loaded?** All adapters are loaded once at startup by
> `lua.NewVM(adapter_dir)`. Adapters uploaded via the WebUI
> (`Core.UploadAdapter` → `vm.LoadAdapter`) and sources edited online take
> effect immediately, without a restart. Directly editing a `.lua` file under
> `adapter_dir` requires a process restart to reload.
## Table of Contents
1. [Script Structure](#1-script-structure)
2. [Static Fields](#2-static-fields)
3. [Transform Hooks](#3-transform-hooks)
4. [Optional Hooks](#4-optional-hooks)
5. [Built-in Helper Functions](#5-built-in-helper-functions)
6. [meta Contract](#6-meta-contract)
7. [A Complete Minimal Adapter](#7-a-complete-minimal-adapter)
8. [Multimodal & disable_thinking](#8-multimodal--disable_thinking)
---
## 1. Script Structure
A script is a Lua file that returns a table — it **must** `return` a table:
```lua
local adapter = {}
adapter.name = "mysrc"
adapter.version = "1.0.0"
-- ... fields and functions ...
return adapter
```
Scripts are executed by LuaJIT (via golua bindings, cgo). Each adapter owns an
**independent VM + worker pool**; adapters never interfere with each other. No mutable
state may be shared across workers (`log` to stdout is the only side effect).
## 2. Static Fields
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `name` | string | yes | Adapter name, shown in the WebUI |
| `version` | string | no | Version, shown in the WebUI |
| `endpoint` | string | no | Upstream path, default `/chat/completions`; overridable by `source.endpoint` / `source.image_endpoint` |
| `headers` | table | no | Static default request headers; used as fallback when no `build_headers` hook is defined |
These fields are extracted statically at load time (compile-once); reading them never
occupies a pooled worker.
## 3. Transform Hooks
### `transform_request(raw_body) -> string`
Input: the unified OpenAI-format JSON string built by the Go layer (the
`/v1/chat/completions` request body).
Output: the request body string to send upstream.
```lua
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
req.model = "upstream-model-name" -- rewrite the model name
req.stream = req.stream or false
return json.encode(req)
end
```
Contract:
- The returned string is POSTed verbatim to `base_url + (source.endpoint or adapter.endpoint)`.
- Internal gateway fields such as `disable_thinking` and `extra_body` should be cleaned
up by the transform (`openai.lua` deletes both).
### `transform_response(raw_body) -> string`
Input: the raw JSON string of a non-streaming upstream response.
Output: a JSON string in the unified format.
Unified format fields:
| Field | Type | Description |
|-------|------|-------------|
| `content` | string | Response text |
| `finish_reason` | string | `stop` / `length` / `tool_calls` etc. |
| `reasoning_content` | string (optional) | Reasoning text (DeepSeek etc.) |
| `token_usage` | table | `{ prompt, completion, total }` (tokens) |
| `tool_calls` | array (optional) | Tool calls: `{ id, type, name, arguments }`; `arguments` is a **decoded** table |
```lua
function adapter.transform_response(raw_body)
local ok, resp = pcall(json.decode, raw_body)
if not ok or resp == nil then return raw_body end
local unified = {
content = "",
finish_reason = "",
token_usage = { prompt = 0, completion = 0, total = 0 }
}
if type(resp.choices) == "table" and #resp.choices > 0 then
local ch = resp.choices[1]
unified.content = ch.message.content or ""
unified.finish_reason = ch.finish_reason or ""
if ch.message.reasoning_content then
unified.reasoning_content = ch.message.reasoning_content
end
end
return json.encode(unified)
end
```
### `transform_stream_chunk(raw_chunk) -> string`
Input: the raw JSON string of each `data:` line in the SSE stream (without the
`data:` prefix).
Output: a unified chunk JSON string; **returning `""` skips that chunk**.
Unified chunk format:
| Field | Type | Description |
|-------|------|-------------|
| `content` | string | Incremental text for this chunk (may be `""`) |
| `done` | boolean (optional) | `true` ends the stream (when `finish_reason` appears) |
```lua
function adapter.transform_stream_chunk(raw_chunk)
local ok, chunk = pcall(json.decode, raw_chunk)
if not ok then return "" end
if not chunk.choices or #chunk.choices == 0 then return "" end
local delta = chunk.choices[1].delta or {}
local fr = chunk.choices[1].finish_reason
return json.encode({
content = delta.content or "",
done = (fr ~= nil)
})
end
```
## 4. Optional Hooks
### `build_headers(meta) -> table<string,string>`
Dynamically generate / sign request headers (e.g. KimiCode's HMAC signature). If the
script does not define this function, the Go layer falls back to the static
`adapter.headers` (or `source.headers`).
```lua
function adapter.build_headers(meta)
local msg = meta.method .. meta.url .. meta.body
return {
["X-App-Sign"] = hmac_sha256_hex(meta.source.meta.app_secret, msg),
["X-Timestamp"] = meta.timestamp,
}
end
```
Returning a non-table errors; returning `{}` means no custom headers (no fallback to
static headers).
## 5. Built-in Helper Functions
Global functions shared by all adapters (injected by Go):
| Function | Description |
|----------|-------------|
| `json.encode(v)` | Lua value → JSON string; returns `"null"` on failure |
| `json.decode(s)` | JSON string → Lua value; returns `nil` on failure |
| `hmac_sha256_hex(key, data)` | HMAC-SHA256, lowercase hex string |
| `sha256_hex(data)` | SHA-256, lowercase hex string |
| `base64_encode(s)` | Standard Base64 encoding |
| `tohex(s)` | Bytes → lowercase hex string |
| `log(level, msg)` | Prints `[adapter/<name>] <msg>` to stdout |
Note: scripts run under LuaJIT with the full standard library (`string`/`table`/`pcall`
etc.); `os`/`io` are not exposed (sandbox semantics).
## 6. meta Contract
The meta structure shared by `build_headers(meta)` and the request transforms:
| Field | Description |
|-------|-------------|
| `meta.url` | Full request URL |
| `meta.method` | HTTP method (usually `POST`) |
| `meta.body` | Request body string |
| `meta.api_key` | The source's `api_key` |
| `meta.timestamp` | Request timestamp |
| `meta.source` | table: `{ name, meta = { ... } }` — the source's `meta` field (e.g. `app_secret`) |
## 7. A Complete Minimal Adapter
```lua
local adapter = {}
adapter.name = "mysrc"
adapter.version = "1.0.0"
adapter.endpoint = "/chat/completions"
adapter.headers = { ["X-Tenant"] = "prod" }
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
req.disable_thinking = nil
req.extra_body = nil
return json.encode(req)
end
function adapter.transform_response(raw_body)
local ok, resp = pcall(json.decode, raw_body)
if not ok or resp == nil then return raw_body end
local unified = {
content = "",
finish_reason = "",
token_usage = { prompt = 0, completion = 0, total = 0 }
}
if type(resp.choices) == "table" and #resp.choices > 0 then
local ch = resp.choices[1]
unified.content = ch.message.content or ""
unified.finish_reason = ch.finish_reason or ""
end
return json.encode(unified)
end
function adapter.transform_stream_chunk(raw_chunk)
local ok, chunk = pcall(json.decode, raw_chunk)
if not ok then return "" end
if not chunk.choices or #chunk.choices == 0 then return "" end
return json.encode({
content = (chunk.choices[1].delta or {}).content or "",
done = (chunk.choices[1].finish_reason ~= nil)
})
end
return adapter
```
Corresponding `config.yaml` source:
```yaml
sources:
- name: mysrc
base_url: https://upstream.example.com
api_key: sk-xxx
adapter: mysrc # or omit = bundled adapter with the same name
endpoint: /chat/completions
models:
- id: my-model
priority: 50
kind: chat
```
## 8. Multimodal & disable_thinking
- **Multimodal**: if the request `content` is an array
(`[{type:"text"...},{type:"image_url"...}]`), the Go layer passes it through
verbatim to the adapter. Sources whose protocol doesn't support it (Anthropic /
Gemini / Ollama) must convert inside `transform_request`; the bundled
`anthropic.lua` / `gemini.lua` / `ollama.lua` already do.
- **disable_thinking**: the gateway passes `disable_thinking:true` from the request
body into `transform_request`. The DeepSeek adapter maps it to
`extra_body.thinking = { type = "disabled" }` and cleans up its own field; other
adapters handle it per their protocol (implement or ignore as needed).

257
docs/lua-adapters.md Normal file
View File

@ -0,0 +1,257 @@
# Lua 适配器 API 文档
> **English**: [lua-adapters-en.md](./lua-adapters-en.md)
每个上游源（source）挂载一个 `.lua` 适配器，负责把统一 OpenAI 格式请求**转换**为上游原生
格式，并把上游响应/流式分块**转换**回统一格式。Go 层只负责调度、并发与透传——因此**新增
适配器、适配新协议，无需重编译 Go**。
适配器放两份（同构）：
- 内置：`internal/lua/adapters/<name>.lua`（编译期 embed）
- 可覆盖：配置 `adapter_dir` 目录下的同名脚本（优先级更高）
> **加载时机**：启动时由 `lua.NewVM(adapter_dir)` 一次性加载全部适配器；通过 WebUI
> 上传的适配器（`Core.UploadAdapter` → `vm.LoadAdapter`）与在线编辑的源配置即时生效，
> 无需重启。直接修改 `adapter_dir` 下的 `.lua` 文件需要重启进程才会重新加载。
## 目录
1. [脚本结构](#1-脚本结构)
2. [静态字段](#2-静态字段)
3. [转换钩子](#3-转换钩子)
4. [可选钩子](#4-可选钩子)
5. [内置辅助函数](#5-内置辅助函数)
6. [meta 约定](#6-meta-约定)
7. [一个完整的最小适配器](#7-一个完整的最小适配器)
8. [多模态与 disable_thinking](#8-多模态与-disable_thinking)
---
## 1. 脚本结构
脚本是一个返回 table 的 Lua 文件，必须 `return` 一个表：
```lua
local adapter = {}
adapter.name = "mysrc"
adapter.version = "1.0.0"
-- ... 字段与函数 ...
return adapter
```
脚本由 LuaJIT 解析（golua 绑定，cgo）。每个适配器拥有**独立 VM + worker 池**，适配器之间
互不干扰；脚本中不允许跨 worker 共享可变状态（`log` 到 stdout 是唯一副作用）。
## 2. 静态字段
| 字段 | 类型 | 必填 | 说明 |
|------|------|------|------|
| `name` | string | 是 | 适配器名，用于 WebUI 展示 |
| `version` | string | 否 | 版本号，用于 WebUI 展示 |
| `endpoint` | string | 否 | 上游请求路径，默认 `/chat/completions`；可被 `source.endpoint` / `source.image_endpoint` 覆盖 |
| `headers` | table | 否 | 静态默认请求头；若未定义 `build_headers` 钩子则作为请求头回退 |
这些字段在加载时静态提取（compile-once），之后读它们不会占用池内 worker。
## 3. 转换钩子
### `transform_request(raw_body) -> string`
入参：Go 层构造的统一 OpenAI 格式 JSON 字符串（`/v1/chat/completions` 请求体）。
返回：发送给上游的请求体字符串。
```lua
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
req.model = "upstream-model-name" -- 改写模型名
req.stream = req.stream or false
return json.encode(req)
end
```
约定：
- 返回的字符串将被原样 POST 到 `base_url + (source.endpoint or adapter.endpoint)`。
- 请求体中包含的 `disable_thinking`、`extra_body` 等网关内部字段，转换时应自行清理
（`openai.lua` 即删掉这两个字段）。
### `transform_response(raw_body) -> string`
入参：上游非流式响应的原始 JSON 字符串。返回：统一格式 JSON 字符串。
统一格式字段：
| 字段 | 类型 | 说明 |
|------|------|------|
| `content` | string | 回复正文 |
| `finish_reason` | string | `stop` / `length` / `tool_calls` 等 |
| `reasoning_content` | string（可选） | 推理内容（DeepSeek 等） |
| `token_usage` | table | `{ prompt, completion, total }`（tokens） |
| `tool_calls` | array（可选） | 工具调用：`{ id, type, name, arguments }`，`arguments` 为**已解码**的 table |
```lua
function adapter.transform_response(raw_body)
local ok, resp = pcall(json.decode, raw_body)
if not ok or resp == nil then return raw_body end
local unified = {
content = "",
finish_reason = "",
token_usage = { prompt = 0, completion = 0, total = 0 }
}
if type(resp.choices) == "table" and #resp.choices > 0 then
local ch = resp.choices[1]
unified.content = ch.message.content or ""
unified.finish_reason = ch.finish_reason or ""
if ch.message.reasoning_content then
unified.reasoning_content = ch.message.reasoning_content
end
end
return json.encode(unified)
end
```
### `transform_stream_chunk(raw_chunk) -> string`
入参：SSE 流中每条 `data:` 的原始 JSON 字符串（不含 `data:` 前缀）。
返回：统一分块 JSON；**返回 `""` 表示跳过该 chunk**。
统一分块格式：
| 字段 | 类型 | 说明 |
|------|------|------|
| `content` | string | 本次增量文本（可为 `""`） |
| `done` | boolean（可选） | `true` 表示流结束（对应 `finish_reason` 出现） |
```lua
function adapter.transform_stream_chunk(raw_chunk)
local ok, chunk = pcall(json.decode, raw_chunk)
if not ok then return "" end
if not chunk.choices or #chunk.choices == 0 then return "" end
local delta = chunk.choices[1].delta or {}
local fr = chunk.choices[1].finish_reason
return json.encode({
content = delta.content or "",
done = (fr ~= nil)
})
end
```
## 4. 可选钩子
### `build_headers(meta) -> table<string,string>`
动态生成/签名请求头（如 KimiCode 的 HMAC 签名）。若脚本未定义此函数，Go 层回退使用静态
`adapter.headers`（或 `source.headers`）。
```lua
function adapter.build_headers(meta)
local msg = meta.method .. meta.url .. meta.body
return {
["X-App-Sign"] = hmac_sha256_hex(meta.source.meta.app_secret, msg),
["X-Timestamp"] = meta.timestamp,
}
end
```
返回非 table 会报错；返回 `{}` 表示无自定义头（不会回退静态头）。
## 5. 内置辅助函数
所有适配器共享的全局函数（由 Go 注入）：
| 函数 | 说明 |
|------|------|
| `json.encode(v)` | Lua 值 → JSON 字符串；失败返回 `"null"` |
| `json.decode(s)` | JSON 字符串 → Lua 值；失败返回 `nil` |
| `hmac_sha256_hex(key, data)` | HMAC-SHA256，十六进制小写字符串 |
| `sha256_hex(data)` | SHA-256，十六进制小写字符串 |
| `base64_encode(s)` | 标准 Base64 编码 |
| `tohex(s)` | 字节 → 十六进制小写字符串 |
| `log(level, msg)` | 打印 `[adapter/<name>] <msg>` 到 stdout |
另注：脚本由 LuaJIT 执行，标准库（`string`/`table`/`pcall` 等）完整可用，`os`/`io` 未暴露
（保持沙箱语义）。
## 6. meta 约定
`build_headers(meta)` 与请求转换共享的 meta 结构：
| 字段 | 说明 |
|------|------|
| `meta.url` | 完整请求 URL |
| `meta.method` | HTTP 方法（通常 `POST`） |
| `meta.body` | 请求体字符串 |
| `meta.api_key` | 该源的 `api_key` |
| `meta.timestamp` | 请求时间戳 |
| `meta.source` | table：`{ name, meta = { ... } }`，即 source 的 `meta` 字段（如 `app_secret`） |
## 7. 一个完整的最小适配器
```lua
local adapter = {}
adapter.name = "mysrc"
adapter.version = "1.0.0"
adapter.endpoint = "/chat/completions"
adapter.headers = { ["X-Tenant"] = "prod" }
function adapter.transform_request(raw_body)
local ok, req = pcall(json.decode, raw_body)
if not ok then return raw_body end
req.disable_thinking = nil
req.extra_body = nil
return json.encode(req)
end
function adapter.transform_response(raw_body)
local ok, resp = pcall(json.decode, raw_body)
if not ok or resp == nil then return raw_body end
local unified = {
content = "",
finish_reason = "",
token_usage = { prompt = 0, completion = 0, total = 0 }
}
if type(resp.choices) == "table" and #resp.choices > 0 then
local ch = resp.choices[1]
unified.content = ch.message.content or ""
unified.finish_reason = ch.finish_reason or ""
end
return json.encode(unified)
end
function adapter.transform_stream_chunk(raw_chunk)
local ok, chunk = pcall(json.decode, raw_chunk)
if not ok then return "" end
if not chunk.choices or #chunk.choices == 0 then return "" end
return json.encode({
content = (chunk.choices[1].delta or {}).content or "",
done = (chunk.choices[1].finish_reason ~= nil)
})
end
return adapter
```
对应的 `config.yaml` source：
```yaml
sources:
- name: mysrc
base_url: https://upstream.example.com
api_key: sk-xxx
adapter: mysrc # 或省略 = 内置同名适配器
endpoint: /chat/completions
models:
- id: my-model
priority: 50
kind: chat
```
## 8. 多模态与 disable_thinking
- **多模态**：请求的 `content` 若是数组（`[{type:"text"...},{type:"image_url"...}]`），Go 层
原样透传给适配器。协议不支持的源（Anthropic/Gemini/Ollama）需在
`transform_request` 内转换；内置 `anthropic.lua`/`gemini.lua`/`ollama.lua` 已实现。
- **disable_thinking**：网关将请求体中的 `disable_thinking:true` 透传进
`transform_request`。DeepSeek 适配器将其映射为 `extra_body.thinking = { type = "disabled" }`
并清理自身字段；其余适配器按各自协议处理（可自行实现或忽略）。