fix: tool call anchor & wire format, streaming chunk passthrough, WebUI narrow-screen, docs bilingual

2026-09-19 16:39:15 +00:00 · 2026-08-08 11:26:58 +08:00
parent 8e9de95274
commit 5d50b69153
18 changed files with 1119 additions and 42 deletions
--- a/README_EN.md
+++ b/README_EN.md
@ -0,0 +1,242 @@
+# ModelRouter
+
+> **中文**: [README.md](./README.md)
+
+A lightweight, unified OpenAI-compatible LLM gateway for internal networks —
+configure once, and let multiple services (agents, SDKs, bots) share many
+upstreams (DeepSeek, Qijiar, OpenAI, Anthropic, Gemini, Groq, Mistral, Ollama,
+KimiCode…) behind a single endpoint. Pick a specific model or use **AUTO** mode,
+which routes to the best healthy upstream by configured priority.
+
+> **Lightweight, no recompile to add sources**: the gateway is a single Go
+> binary (~10 MB, zero runtime dependencies). Adding or switching an upstream is
+> just a `sources` entry in `config.yaml` (or via the WebUI) or a `.lua`
+> adapter — **no Go changes, no recompile**. Changes submitted through the
+> WebUI take effect immediately (hot reload); editing `config.yaml` or a `.lua`
+> file under `adapter_dir` by hand requires a process restart.
+
+## Overview
+
+Exposes a standard `OpenAI Chat Completions` API (`/v1/chat/completions` +
+`/v1/models`) to your internal network, translating between multiple upstream
+protocols via **Lua adapters**. Supports one-shot calls, SSE streaming,
+**AUTO model routing**, multimodal passthrough, and tool calls.
+
+Extracted and independently evolved from the multi-source LLM adapter layer of
+[HomeAgent](https://gitcode.com/JianFeeeee/HomeAgent)
+(`internal/agent/api/provider.go` + `internal/lua/adapters/*`).
+
+## Features
+
+- **Multi-source**: any number of upstream sources in one process, routed by the
+  request's `model`.
+- **AUTO mode**: with `default_model: AUTO`, the highest-priority available
+  source model wins.
+- **Unified output**: every source speaks OpenAI format (incl.
+  `reasoning_content`, `tool_calls`, `usage`).
+- **Multimodal**: `content` arrays (`image_url` etc.) pass through losslessly;
+  Anthropic/Gemini/Ollama are translated automatically.
+- **LuaJIT VM**: golua-binding LuaJIT; each adapter has its own VM + worker
+  pool for safe concurrency.
+- **disable_thinking**: `disable_thinking: true` toggles reasoning per-request.
+- **Lua adapter protocol**: each source mounts a `.lua` adapter with
+  `transform_request` / `transform_response` / `transform_stream_chunk` — all
+  protocol differences live in Lua, Go only schedules and proxies.
+- **Signature / header hooks**: adapters may define `build_headers(meta)` to
+  inject/sign request headers before the HTTP call (e.g. KimiCode style
+  app-validation), with helpers like `hmac_sha256_hex`, `sha256_hex`,
+  `base64_encode`.
+- **WebUI**: built-in management page to view/add/edit sources & models,
+  persisted to a runtime file.
+- **Auth**: the gateway validates client keys via `gateway_keys`; independent
+  from each upstream's own key.
+- **Streaming**: SSE `chat.completion.chunk` with a role-first chunk and
+  `[DONE]` terminator.
+
+## Quick start
+
+```bash
+cp config.example.yaml config.yaml   # edit your sources & keys
+GOMODCACHE=... GOPROXY=off go build -tags luajit -o llmsproxy ./cmd/llmsproxy
+./llmsproxy -config config.yaml
+```
+
+> Depends on [golua](https://github.com/aarzilli/golua) (LuaJIT bindings).
+> You **must** build with `-tags luajit`; otherwise the built-in superset
+> gopher-lua path is used (behavior differs slightly).
+
+```bash
+# no key -> 401
+curl http://127.0.0.1:8080/v1/models
+
+# one-shot
+curl -H "Authorization: Bearer sk-gw-local-0001" \
+     -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"hi"}]}' \
+     http://127.0.0.1:8080/v1/chat/completions
+
+# streaming
+curl -N -H "Authorization: Bearer sk-gw-local-0001" \
+     -d '{"model":"deepseek-v4-flash","stream":true,"messages":[{"role":"user","content":"hi"}]}' \
+     http://127.0.0.1:8080/v1/chat/completions
+```
+
+Any OpenAI SDK works: point `base_url` at the gateway, use one of `gateway_keys`
+as the API key.
+
+## Configuration
+
+See [`config.example.yaml`](config.example.yaml). Core fields:
+
+```yaml
+listen: 127.0.0.1:8080        # bind address (keep internal/loopback)
+gateway_keys: [sk-gw-0001]    # keys clients use; empty = no auth
+default_model: AUTO           # when model is unroutable, pick source by priority
+adapter_dir: adapters         # Lua adapter dir; built-ins written on first start
+runtime_file: runtime.json    # WebUI-edited sources persist here
+
+sources:
+  - name: deepseek
+    base_url: https://api.deepseek.com
+    api_key: sk-...
+    adapter: deepseek
+    max_concurrent: 8
+    models:
+      - id: deepseek-v4-flash
+        priority: 100          # higher -> preferred by AUTO
+        kind: chat
+      - id: deepseek-v4-pro
+        priority: 60
+        kind: chat
+    # static headers (take precedence over adapter defaults)
+    headers: { X-Tenant: prod }
+    # passthrough metadata for the Lua build_headers hook
+    meta: { app_id: x, app_secret: y }
+    temperature: 0.7
+    max_tokens: 4096
+    timeout: 120s              # request timeout, default 120s
+```
+
+### Model routing
+
+`/v1/chat/completions` `model` resolution order:
+
+1. `source/model` or `source:model` prefix → pinned source;
+2. exact match of a source's `model`;
+3. with `default_model: AUTO` → highest-`priority` healthy source model;
+4. otherwise fall back to `default_source`.
+
+Any OpenAI client can pin to a source by setting `model` to its
+`name/anything`; `AUTO` (or the gateway's `default_model: AUTO`) picks the
+healthy source by priority.
+
+### Sources vs. adapters
+
+- A **source** describes a connection to an upstream: `name`, `base_url`,
+  `api_key`, model list and priorities.
+- An **adapter** is the protocol translation logic (Lua script): converts the
+  unified OpenAI request to the upstream's native format and back.
+- One adapter serves many sources (e.g. `openai.lua` for any OpenAI-compatible
+  site); one source can switch adapters via its `adapter` field.
+- Sources decide *whom to talk to and which models to expose*; adapters decide
+  *how to talk*. They are linked by the `adapter` field inside each `sources[]`
+  entry.
+
+Loading flow (assembled by `Core` in `internal/core`):
+
+1. On startup `lua.NewVM(adapter_dir)` loads all adapters: built-in ones
+   (embedded at compile time) + same-name override files under `adapter_dir`.
+2. `config.Load` reads `config.yaml`; `config.NewStore(runtime_file)` reads
+   WebUI-edited runtime sources; both are merged by name.
+3. `rebuildRegistry` creates a `provider.Provider` per source (holding its
+   adapter), sizing the adapter's worker pool from the source's concurrency
+   limit.
+4. Incoming requests are routed by `Registry` to a provider, which calls the
+   adapter's `transform_request` → HTTP call → `transform_response` /
+   `transform_stream_chunk`.
+
+"Add/Edit source" and "Upload adapter" in the WebUI take effect immediately
+(written to the runtime file / `adapter_dir`, then reassembled — no restart).
+Directly editing `config.yaml` or files under `adapter_dir` requires a process
+restart.
+
+### disable_thinking
+
+With `"disable_thinking": true` in the request body, the gateway passes it to
+each adapter; the DeepSeek adapter maps it to `extra_body.thinking.type =
+"disabled"`, other sources follow their own protocol.
+
+### WebUI
+
+Built-in admin page at `GET /`; after login you can view/edit sources and
+models in the browser, persisted to `runtime_file` (survives restarts).
+
+## Lua adapter protocol
+
+Full API: **[Lua adapter docs](docs/lua-adapters.md)**
+([中文](docs/lua-adapters-en.md)).
+
+Each adapter is a Lua script returning a table
+(`internal/lua/adapters/<name>.lua`), overridable by a same-name file under
+`adapter_dir` — **no recompile needed**, restart to take effect; uploads via
+the WebUI take effect immediately.
+
+```lua
+return {
+  name = "mysrc",
+  version = "1.0.0",
+  endpoint = "/chat/completions",        -- upstream path (source.endpoint overrides)
+  headers = { ["X-Static"] = "v" },      -- static default headers (fallback)
+
+  -- request: convert unified OpenAI req -> upstream native format, return string
+  transform_request = function(raw_json) ... end,
+
+  -- response: convert upstream response to unified format string
+  --   { content, reasoning_content, finish_reason, token_usage{...}, tool_calls[{...}] }
+  transform_response = function(raw_json) ... end,
+
+  -- streaming chunk: convert upstream SSE data to { content, done, ... }; "" skips
+  transform_stream_chunk = function(raw_chunk) ... end,
+
+  -- [optional] dynamic headers / signature hook
+  -- meta = { url, method, body, api_key, timestamp, source={ name, meta={...} } }
+  build_headers = function(meta) return { ["X-App-Sign"] = sign } end,
+}
+```
+
+Shared helpers: `hmac_sha256_hex(key, data)`, `sha256_hex(data)`,
+`base64_encode(s)`, `tohex(s)`, `json.encode/decode`, `log(level, msg)`.
+
+### Built-in adapters
+
+`openai` `deepseek` `anthropic` `gemini` `github` `groq` `mistral` `ollama`
+`kimicode`.
+
+`anthropic`/`gemini`/`ollama` include multimodal conversion
+(`image_url` → their native format); with `disable_thinking` the `deepseek`
+adapter sets `extra_body.thinking.type` to `disabled`.
+
+**kimicode** demonstrates `build_headers`: the cloud validates the calling
+app, so you HMAC-sign timestamp+URL+body with `meta.app_secret` and add
+`X-App-Sign`-style headers. Configure `sources[].meta.{app_id, app_secret,
+app_agent}`.
+
+## Layout
+
+```
+cmd/llmsproxy            # entry point
+internal/config          # YAML config loading/validation
+internal/lua             # LuaJIT VM + worker pool + AdapterCache + built-ins (embed)
+internal/provider        # Provider(HTTP) + Registry(routing)
+internal/gateway         # OpenAI-compatible HTTP + auth + SDK/streaming + WebUI
+internal/types           # unified format & OpenAI wire types
+```
+
+## Tests
+
+```bash
+go test -tags luajit ./...
+```
+
+Covers: config validation, adapter load/transform, signature hooks, gateway
+auth, SDK round trip, SSE streaming, model routing, multimodal passthrough and
+disable_thinking.