mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-11-07 09:47:23 +00:00
f7e926ccb7
README 从 642 行降到 131 行，只保留简介、核心能力、最小快速开始、文档索引、 改名说明、目录结构与测试入口。其余内容迁到 docs/，中英文索引一一对应： docs/features.md 实现与完整特性清单（原 README 的"实现/特性"） docs/configuration.md 配置文件、自动探测模型、多密钥与配额 docs/routing.md 模型路由、AUTO 链、冷却与半冷却、源/适配器、源模板 docs/performance.md 实测内存占用与调优、thinking 预算 docs/webui.md WebUI 使用 docs/adapters.md Lua 适配器协议与内置适配器清单 docs/gui.md 桌面端 Headless/Desktop 两种产物与构建 docs/development.md 目录职责、开发协作、测试 docs/deployment.md 部署、发行包、目录权限、llmsproxy 迁移与验证清单 中性化：README / README_EN / GUI 注释里的 `Clash Verge` 类比全部改为 「桌面壳 / desktop shell」，仓库内（源码与文档）已无该表述。
150 lines
6.8 KiB
Markdown
150 lines
6.8 KiB
Markdown
# ModelRouter
|
|
|
|
> **中文**: [README.md](./README.md)
|
|
>
|
|
> **AI-assisted**: code and documentation were produced with AI assistance and
|
|
> reviewed by a human.
|
|
|
|
An intranet gateway for people with several LLM subscriptions: configure once and
|
|
let multiple services share the upstream capacity. Pick a model explicitly, or let
|
|
`AUTO` choose by priority among the sources that are currently healthy.
|
|
|
|
The gateway itself is a single Go binary (~10 MB, no runtime dependencies). Adding
|
|
or switching an upstream is a `sources` entry in `config.yaml` (or the WebUI), or a
|
|
dropped-in `.lua` adapter — **no Go changes, no rebuild**. Changes submitted through
|
|
the WebUI take effect immediately; editing `config.yaml` or a `.lua` adapter file
|
|
directly requires a restart.
|
|
|
|
## What it does
|
|
|
|
- **One OpenAI-compatible API**: `/v1/chat/completions`, `/v1/models`,
|
|
`/v1/images/generations`. Every source speaks the same shape outwards, including
|
|
`reasoning_content`, `tool_calls` and `usage`.
|
|
- **Many sources, one AUTO chain**: configure any number of upstreams; `AUTO`
|
|
follows the priority tiers saved in the WebUI, fails over between them, cools
|
|
down broken sources and probes them back to health at half-cool.
|
|
- **Protocol differences live in Lua**: each source binds a `.lua` adapter that
|
|
transforms requests, responses and stream chunks both ways, and can sign headers
|
|
via `build_headers(meta)`. Vendor knowledge stays out of the Go kernel.
|
|
- **Plugins are independent**: business extensions (channel login, billing, quota
|
|
probing) are external Lua plugins loaded from `plugin_dir`. They are not compiled
|
|
into the binary and not shipped in release archives, and they talk to the kernel
|
|
only through the `modelrouter` SDK.
|
|
- **Multi-tenant keys**: gateway keys are separate from upstream keys, and can be
|
|
scoped to a model set, an AUTO chain and a quota.
|
|
- **Encrypted at rest**: upstream keys, custom header values and gateway keys are
|
|
sealed with AES-256-GCM; `master.key` is 0600, or injected via
|
|
`MODELROUTER_MASTER_KEY`.
|
|
- **Observability**: live source probing, HTTP access log, config-change audit,
|
|
request and usage CSV export. Writes are kept off the request hot path.
|
|
|
|
## Quick start
|
|
|
|
```bash
|
|
# LuaJIT development libraries are required; -tags luajit is mandatory
|
|
GOMODCACHE=... GOPROXY=off go build -tags luajit -o modelrouter ./cmd/modelrouter
|
|
|
|
# First run generates a default config and prints a random admin key
|
|
./modelrouter -config config.yaml
|
|
```
|
|
|
|
> Depends on [golua](https://github.com/aarzilli/golua) (LuaJIT bindings). Without
|
|
> `-tags luajit` the build falls back to gopher-lua, which behaves slightly
|
|
> differently.
|
|
|
|
> **No config in the repo**: the repository ships no config file (they contain
|
|
> keys). The generated default binds `127.0.0.1:8080` only; rotate the admin key in
|
|
> the WebUI after the first login.
|
|
|
|
```bash
|
|
# No key -> 401
|
|
curl http://127.0.0.1:8080/v1/models
|
|
|
|
# Non-streaming ($KEY = the admin key printed at first start)
|
|
curl -H "Authorization: Bearer $KEY" \
|
|
-d '{"model":"AUTO","messages":[{"role":"user","content":"hi"}]}' \
|
|
http://127.0.0.1:8080/v1/chat/completions
|
|
|
|
# Streaming
|
|
curl -N -H "Authorization: Bearer $KEY" \
|
|
-d '{"model":"AUTO","stream":true,"messages":[{"role":"user","content":"hi"}]}' \
|
|
http://127.0.0.1:8080/v1/chat/completions
|
|
```
|
|
|
|
Any OpenAI SDK works: point `base_url` at the gateway and use an authorized gateway
|
|
key as `api_key`.
|
|
|
|
## Documentation
|
|
|
|
The detailed documentation is currently written in Chinese.
|
|
|
|
| Topic | Document |
|
|
|---|---|
|
|
| Implementation and the full feature list | [docs/features.md](docs/features.md) |
|
|
| Configuration: `config.yaml`, model discovery, multiple keys | [docs/configuration.md](docs/configuration.md) |
|
|
| Routing: model routing, AUTO chain, cooldown, sources vs adapters, source templates | [docs/routing.md](docs/routing.md) |
|
|
| Adapters: Lua protocol and built-in adapters | [docs/adapters.md](docs/adapters.md) · [docs/lua-adapters-en.md](docs/lua-adapters-en.md) (English) |
|
|
| Plugins: SDK contract, hooks, UI extensions, HTTP endpoints | [docs/plugins.md](docs/plugins.md) |
|
|
| Desktop app: Headless / Desktop artifacts, build and run | [docs/gui.md](docs/gui.md) |
|
|
| Performance: measured memory use, tuning, thinking budget | [docs/performance.md](docs/performance.md) |
|
|
| WebUI usage | [docs/webui.md](docs/webui.md) |
|
|
| Development: layout, tests, branch and release discipline | [docs/development.md](docs/development.md) · [docs/git-workflow-en.md](docs/git-workflow-en.md) (English) |
|
|
| Deployment and migration | [docs/deployment.md](docs/deployment.md) |
|
|
|
|
## Renamed from llmsproxy
|
|
|
|
The product name has been ModelRouter for a while; the binary, config directory,
|
|
service name and environment variables are now `modelrouter` as well:
|
|
|
|
| Before | After |
|
|
|---|---|
|
|
| `llmsproxy` (binary) | `modelrouter` |
|
|
| `/etc/llmsproxy/` | `/etc/modelrouter/` |
|
|
| `llmsproxy.service` | `modelrouter.service` |
|
|
| `LLMSPROXY_*` | `MODELROUTER_*` |
|
|
|
|
Existing installs migrate with a dry-run-by-default script (copies, never moves,
|
|
and can roll back):
|
|
|
|
```bash
|
|
sudo ./packaging/migrate-to-modelrouter.sh # preview
|
|
sudo ./packaging/migrate-to-modelrouter.sh --apply # apply
|
|
sudo ./packaging/migrate-to-modelrouter.sh --rollback --apply
|
|
```
|
|
|
|
Old paths and old environment variable names are still accepted on the read side,
|
|
so an upgrade cannot fail merely because of ordering. `deploy.sh` refuses to run
|
|
when it finds the old `/etc/llmsproxy` without a new config, because that would
|
|
deploy a gateway with no key and no sources.
|
|
|
|
The plugin SDK follows the same naming: the only public entry point is
|
|
`modelrouter` (`llmsproxy` remains a compatibility alias). See
|
|
[docs/plugins.md](docs/plugins.md).
|
|
|
|
## Layout
|
|
|
|
```text
|
|
cmd/modelrouter # entry point (pure Go server)
|
|
cmd/gui # Electron desktop shell (embeds the same core)
|
|
internal/config # config parsing, sealed storage, runtime file
|
|
internal/core # scheduling, kernel actions, state and audit
|
|
internal/lua # LuaJIT VM + elastic worker pool + built-in adapters (embed)
|
|
internal/lua/plugins # optional plugin examples (not shipped in binary/archives)
|
|
internal/provider # Provider(HTTP) + Registry(routing)
|
|
internal/gateway # OpenAI-compatible HTTP service + auth + streaming + WebUI
|
|
internal/types # unified format and OpenAI wire types
|
|
packaging/ # deb/rpm/tar packaging, systemd unit, migration script
|
|
docs/ # topic documentation
|
|
```
|
|
|
|
## Development and tests
|
|
|
|
```bash
|
|
go test -tags luajit ./... # everything
|
|
go test -tags luajit ./internal/lua # one package (required for LuaJIT changes)
|
|
```
|
|
|
|
- Branch and release discipline: [docs/git-workflow-en.md](docs/git-workflow-en.md).
|
|
- Plugins live in `plugin_dir`. The gateway never creates, overwrites or deletes
|
|
plugin files, and a desktop upgrade never touches them either.
|