# ModelRouter

> **中文**: [README.md](./README.md)
>
> **AI-assisted**: This project is developed with AI-assisted coding (code and docs are produced in collaboration with AI, reviewed by humans).

A lightweight, unified OpenAI-compatible LLM gateway for internal networks —
configure once, and let multiple services (agents, SDKs, bots) share many
upstreams (DeepSeek, Qijiar, OpenAI, Anthropic, Gemini, Groq, Mistral, Ollama,
KimiCode…) behind a single endpoint. Pick a specific model or use **AUTO** mode,
which routes to the best healthy upstream by configured priority.

> **Lightweight, no recompile to add sources**: the gateway is a single Go
> binary (~10 MB, zero runtime dependencies). Adding or switching an upstream is
> just a `sources` entry in `config.yaml` (or via the WebUI) or a `.lua`
> adapter — **no Go changes, no recompile**. Changes submitted through the
> WebUI take effect immediately (hot reload); editing `config.yaml` or a `.lua`
> file under `adapter_dir` by hand requires a process restart.

## Overview

Exposes a standard `OpenAI Chat Completions` API (`/v1/chat/completions` +
`/v1/models`) to your internal network, translating between multiple upstream
protocols via **Lua adapters**. Supports one-shot calls, SSE streaming,
**AUTO model routing**, multimodal passthrough, and tool calls.

Extracted and independently evolved from the multi-source LLM adapter layer of
[HomeAgent](https://gitcode.com/JianFeeeee/HomeAgent)
(`internal/agent/api/provider.go` + `internal/lua/adapters/*`).

## Features

- **Multi-source**: any number of upstream sources in one process, routed by the
  request's `model`.
- **AUTO mode**: with `default_model: AUTO`, scheduling follows the tiered AUTO
  chain saved on the Priority page (see below).
- **Unified output**: every source speaks OpenAI format (incl.
  `reasoning_content`, `tool_calls`, `usage`).
- **Image generation**: `POST /v1/images/generations`, routed to models with
  `kind: image`.
- **Multimodal**: `content` arrays (`image_url` etc.) pass through losslessly;
  Anthropic/Gemini/Ollama are translated automatically.
- **LuaJIT VM**: golua-binding LuaJIT; each adapter has its own VM + worker
  pool for safe concurrency.
- **disable_thinking**: `disable_thinking: true` toggles reasoning per-request.
- **Lua adapter protocol**: each source mounts a `.lua` adapter with
  `transform_request` / `transform_response` / `transform_stream_chunk` — all
  protocol differences live in Lua, Go only schedules and proxies.
- **Signature / header hooks**: adapters may define `build_headers(meta)` to
  inject/sign request headers before the HTTP call (e.g. KimiCode style
  app-validation), with helpers like `hmac_sha256_hex`, `sha256_hex`,
  `base64_encode`.
- **WebUI**: built-in management page to view/add/edit sources & models, edit
  the AUTO chain, and manage per-key model scopes, persisted to a runtime file.
- **Encrypted secrets at rest**: upstream `api_key`, custom header values and
  gateway keys are stored AES-256-GCM encrypted (`master.key` 0600 next to the
  runtime file, or `LLMS_PROXY_MASTER_KEY`).
- **Live source probing**: the status page probes each source
  (`GET {base}/models`, falling back to a minimal chat call) without touching
  the scheduler's backoff state; errors are shown on hover.
- **Auth**: the gateway validates client keys via `gateway_keys`; independent
  from each upstream's own key.
- **Streaming**: SSE `chat.completion.chunk` with a role-first chunk and
  `[DONE]` terminator.

## Quick start

```bash
GOMODCACHE=... GOPROXY=off go build -tags luajit -o llmsproxy ./cmd/llmsproxy
./llmsproxy -config config.yaml    # first run generates a default config and prints a random admin key
```

> Depends on [golua](https://github.com/aarzilli/golua) (LuaJIT bindings).
> You **must** build with `-tags luajit`; otherwise the built-in superset
> gopher-lua path is used (behavior differs slightly).

> **No config is shipped in the repo** (config files carry real keys). On
> first run the binary generates a default config at the `-config` path: a
> random admin key (printed to the startup log), loopback-only `127.0.0.1:8080`,
> and a keyless opencode zen source ready to chat. Rotate the admin key in the
> WebUI after first login.

```bash
# no key -> 401
curl http://127.0.0.1:8080/v1/models

# one-shot ($KEY = the admin key printed at first startup)
curl -H "Authorization: Bearer $KEY" \
     -d '{"model":"AUTO","messages":[{"role":"user","content":"hi"}]}' \
     http://127.0.0.1:8080/v1/chat/completions

# streaming
curl -N -H "Authorization: Bearer $KEY" \
     -d '{"model":"AUTO","stream":true,"messages":[{"role":"user","content":"hi"}]}' \
     http://127.0.0.1:8080/v1/chat/completions
```

Any OpenAI SDK works: point `base_url` at the gateway, use any authorized
gateway key as the API key.

## Configuration

Generated on first run at the `-config` path (default `./config.yaml`). Core fields:

```yaml
listen: 127.0.0.1:8080        # bind address (keep internal/loopback)
gateway_keys: [sk-gw-<random>] # initial admin key seed (random per install), migrated to the store on first start
default_model: AUTO           # when model is unroutable, follow the AUTO chain
adapter_dir: adapters         # Lua adapter dir; created+seeded if missing, read-only otherwise
runtime_file: runtime.json    # WebUI-edited sources/keys/AUTO chain persist here

sources:
  - name: deepseek
    base_url: https://api.deepseek.com
    api_key: sk-...            # or api_key_env: SOME_ENV to read from an env var
    adapter: deepseek
    max_concurrent: 8
    models:
      - id: deepseek-v4-flash
        priority: 100          # YAML sources seed the AUTO chain on first start
        kind: chat
      - id: deepseek-v4-pro
        priority: 60
        kind: chat
    # static headers (take precedence over adapter defaults)
    headers: { X-Tenant: prod }
    # passthrough metadata for the Lua build_headers hook
    meta: { app_id: x, app_secret: y }
    temperature: 0.7
    max_tokens: 4096
    timeout: 120s              # request timeout, default 120s
```

Sensitive fields in the runtime file (`runtime_file`) are encrypted at rest:

- AES-256-GCM, stored as `enc:v1:<base64>`.
- Master key source: env var `LLMS_PROXY_MASTER_KEY` (64 hex chars), else a
  `master.key` file next to the runtime file; if neither exists it is generated
  on first start (mode 0600).
- Old plaintext files still load; the first UI save migrates the whole file to
  ciphertext.
- Back up `master.key` with your config — losing it makes the secrets
  undecryptable. Do not commit it.

### Gateway keys (multi-key)

Gateway auth uses a "multi-key + role + model scope" model. Keys are persisted
under the `keys` field of the runtime file (encrypted at rest):

- The `gateway_keys` config is only an **initial admin key seed** — it is
  migrated into the runtime store on first start and no longer drives auth.
- **Important: after first start, replace the admin key via the WebUI Keys
  page.** The seed key is written in plaintext in `config.yaml`, so keeping it
  active is a security risk; create a new admin key, log in with it, then
  delete the seed key.
- The WebUI **Keys page** creates/deletes keys. Each key has a role (`admin`
  manages everything, `user` sees only its own key) and an optional **model
  scope** (model + source + token quota + reset period).
- Clients authenticate with any authorized key's plaintext as
  `Authorization: Bearer <key>`.
- Deleting a key removes it from the store immediately.

### Model routing

`/v1/chat/completions` `model` resolution order:

1. `source/model` or `source:model` prefix → pinned source;
2. exact match of a source's `model`;
3. with `default_model: AUTO` → follow the AUTO chain from the Priority page
   (see below);
4. otherwise the request errors.

Any OpenAI client can pin to a source by setting `model` to its
`name/anything`; `AUTO` (or the gateway's `default_model: AUTO`) follows the
tiered AUTO chain.

### AUTO chain (Priority page)

AUTO scheduling is driven **only** by the rules saved on the Priority page
(persisted as the `auto` field of the runtime file). The numeric
`models[].priority` in source configs no longer participates in scheduling and
is no longer shown.

- Each rule is one "slot": `{ model, source, tier, token_quota, period, hours }`.
- `tier` is the priority tier: models in the same tier sit side by side and
  share it; tiers run high → low.
- The same model may appear in several slots (e.g. A low → B low → A high →
  B high) and is tried in tier order.
- `token_quota` > 0 means the slot is skipped once its tokens within the reset
  window are exhausted; `period` supports `hour` / `week` / `month` / `nhour`
  (with `hours`); empty = unlimited.
- Image models (`kind: image`) are kept out of the chain and are served by the
  separate `POST /v1/images/generations` path.

### Sources vs. adapters

- A **source** describes a connection to an upstream: `name`, `base_url`,
  `api_key`, model list and priorities.
- An **adapter** is the protocol translation logic (Lua script): converts the
  unified OpenAI request to the upstream's native format and back.
- One adapter serves many sources (e.g. `openai.lua` for any OpenAI-compatible
  site); one source can switch adapters via its `adapter` field.
- Sources decide *whom to talk to and which models to expose*; adapters decide
  *how to talk*. They are linked by the `adapter` field inside each `sources[]`
  entry.

Loading flow (assembled by `Core` in `internal/core`):

1. On startup `lua.NewVM(adapter_dir)` loads all adapters: built-in ones
   (embedded at compile time) + same-name override files under `adapter_dir`.
2. `config.Load` reads `config.yaml`; `config.NewStore(runtime_file)` reads
   WebUI-edited runtime sources; both are merged by name.
3. `rebuildRegistry` creates a `provider.Provider` per source (holding its
   adapter), sizing the adapter's worker pool from the source's concurrency
   limit.
4. Incoming requests are routed by `Registry` to a provider, which calls the
   adapter's `transform_request` → HTTP call → `transform_response` /
   `transform_stream_chunk`.

"Add/Edit source", "Upload adapter", "AUTO chain edits" and "per-key model
scopes" in the WebUI take effect immediately (written to the runtime file /
`adapter_dir`, then reassembled — no restart). Directly editing `config.yaml`
or files under `adapter_dir` requires a process restart.

Deleting a source/adapter in the WebUI truly removes runtime (UI-created)
sources and `.lua` files; base sources defined in `config.yaml` cannot rewrite
that file, so they are hidden via a deletion tombstone (still hidden after
restart) and can be restored by re-adding the same name in the UI.

### disable_thinking

With `"disable_thinking": true` in the request body, the gateway passes it to
each adapter; the DeepSeek adapter maps it to `extra_body.thinking.type =
"disabled"`, other sources follow their own protocol.

### WebUI

Built-in admin page at `GET /`; after login you can:

- **Status**: source online state (live probe, error on hover), per
  model/source/key usage stats, request records filtered by key, and CSV export
  over a time range; click a model to pin the connection snippet to it.
- **Chat**: streaming / non-streaming debug.
- **Keys**: create/edit gateway keys, set per-key model scopes (model + source +
  token quota + reset period); admins manage all keys, users see only their own.
- **Priority**: drag blocks to build the tiered AUTO chain.
- **Sources**: add/edit/delete upstream sources online (secrets stored
  encrypted).
- **Adapters**: upload / delete Lua adapter scripts.

Changes persist to `runtime_file` (survive restarts).

## Lua adapter protocol

Full API: **[Lua adapter docs](docs/lua-adapters.md)**
([中文](docs/lua-adapters-en.md)).

Each adapter is a Lua script returning a table
(`internal/lua/adapters/<name>.lua`), overridable by a same-name file under
`adapter_dir` — **no recompile needed**, restart to take effect; uploads via
the WebUI take effect immediately.

```lua
return {
  name = "mysrc",
  version = "1.0.0",
  endpoint = "/chat/completions",        -- upstream path (source.endpoint overrides)
  headers = { ["X-Static"] = "v" },      -- static default headers (fallback)

  -- request: convert unified OpenAI req -> upstream native format, return string
  transform_request = function(raw_json) ... end,

  -- response: convert upstream response to unified format string
  --   { content, reasoning_content, finish_reason, token_usage{...}, tool_calls[{...}] }
  transform_response = function(raw_json) ... end,

  -- streaming chunk: convert upstream SSE data to { content, done, ... }; "" skips
  transform_stream_chunk = function(raw_chunk) ... end,

  -- [optional] dynamic headers / signature hook
  -- meta = { url, method, body, api_key, timestamp, source={ name, meta={...} } }
  build_headers = function(meta) return { ["X-App-Sign"] = sign } end,
}
```

Shared helpers: `hmac_sha256_hex(key, data)`, `sha256_hex(data)`,
`base64_encode(s)`, `tohex(s)`, `json.encode/decode`, `log(level, msg)`.

### Built-in adapters

`openai` `deepseek` `anthropic` `gemini` `github` `groq` `mistral` `ollama`
`kimicode` `opencode`.

`anthropic`/`gemini`/`ollama` include multimodal conversion
(`image_url` → their native format); with `disable_thinking` the `deepseek`
adapter sets `extra_body.thinking.type` to `disabled`.

**opencode** targets the opencode.ai zen free pool
(`https://opencode.ai/zen/v1`): zen fingerprints clients by User-Agent and
routes non-official UAs (curl, Go's default) into an anonymous pool that
hits `FreeUsageLimitError`. The adapter always sends the opencode client UA;
combined with `api_key: "public"` (the keyless official client actually sends
`Bearer public`) it gets the free pool, e.g. `deepseek-v4-flash-free`.

**kimicode** demonstrates `build_headers`: the cloud validates the calling
app, so you HMAC-sign timestamp+URL+body with `meta.app_secret` and add
`X-App-Sign`-style headers. Configure `sources[].meta.{app_id, app_secret,
app_agent}`.

## Desktop GUI (Electron)

Optional standalone product for desktop users on your internal network: an
**embedded ModelRouter core** (Clash Verge-style): one-click start/stop, system
tray, autostart, silent launch, and a pixel-for-pixel embedded full WebUI
(Status/Chat/Keys/Priority/Sources/Adapters — no login needed). Server users
keep using the plain Go binary.

### GUI vs. plain backend

| Scenario | Pick | Why |
| ---- | ---- | ---- |
| Server / intranet gateway / unattended long-running | **plain backend** (single binary) | ~10 MB, ~15 MB RSS, zero-dep single process — drop it into systemd or any container, remote admin |
| Personal desktop daily use / multi-device intranet sharing | **GUI** (Electron) | no-login embedded WebUI, tray one-click, autostart, silent background — for non-CLI users |
| Windows desktop | **GUI** | plain backend needs manual service registration; GUI ships native tray/autostart |
| CI one-shot 3-platform installers | **GUI packaging scripts** | `make gui-*` emits deb / AppImage / NSIS, drops straight into a release pipeline |

Both share the exact same `llmsproxy` core (LuaJIT build) — configs and
adapters are fully compatible, freely interchangeable.

### Build & run

```bash
cd cmd/gui
npm install
make gui            # or npm run dev — dev run (needs make build first for bin/llmsproxy)
make gui-deb        # local deb (share with other Linux users)
make gui-dist       # deb + AppImage (linux)
make gui-win        # Windows NSIS (host mingw + wine)
make gui-win-docker # Windows NSIS, fully dockerized (no host mingw needed)
```

Artifacts land in `cmd/build/gui-dist/`: `ModelRouter-1.0.0.AppImage`,
`modelrouter-gui_1.0.0_amd64.deb`, `ModelRouter Setup 1.0.0.exe` (win). rpm
needs system `rpmbuild`.

#### Windows packaging (dockerized, recommended)

`make gui-win-docker` does cross-compile + NSIS in one step; the host needs
**no mingw**.

- Image: `cmd/gui/docker/win-builder/Dockerfile` (`golang:1.25` + mingw-w64 +
  prebuilt LuaJIT for Windows: `lua51.dll` + import lib). LuaJIT source comes
  from the gitcode mirror (reachable in CN) with github fallback; Go modules go
  through `https://goproxy.cn`.
- Core build: `cmd/gui/scripts/win-core-docker.sh` — emits
  `cmd/gui/bin/{llmsproxy.exe,lua51.dll}` inside the container.
- One-shot release: `cmd/gui/scripts/dist-win-docker.sh` — after the core,
  runs electron-builder (host needs `wine` for NSIS).
- The `modelrouter/win-builder` image is auto-built on first run and reused
  afterwards.

```bash
# Full Windows release (rebuild core + NSIS):
make gui-win-docker
```

### Features

- **Embedded core**: auto-spawns the bundled `llmsproxy` (LuaJIT build);
  config/keys/adapters live in `<userData>/profile/`. First run generates a
  random admin key written into `keys` (not seed) and self-injects it — WebUI
  needs no login and never nags about changing the initial key. Port changeable
  in settings (default 8787).
- **System tray**: core status dot, autostart/silent toggles, start/stop/restart
  core, quit.
- **Silent launch**: `--silent` flag or setting — starts to the tray only;
  first run always shows the window.
- **Autostart**: `setLoginItemSettings` on Windows/macOS; writes
  `~/.config/autostart/modelrouter-gui.desktop` on Linux, appending `--silent`
  when silent mode is on.

## Layout

```
cmd/llmsproxy            # entry point
internal/config          # YAML config loading/validation
internal/lua             # LuaJIT VM + worker pool + AdapterCache + built-ins (embed)
internal/provider        # Provider(HTTP) + Registry(routing)
internal/gateway         # OpenAI-compatible HTTP + auth + SDK/streaming + WebUI
internal/types           # unified format & OpenAI wire types
```

## Tests

```bash
go test -tags luajit ./...
```

Covers: config validation, adapter load/transform, signature hooks, gateway
auth, SDK round trip, SSE streaming, model routing, multimodal passthrough and
disable_thinking.