mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-10-03 23:54:06 +00:00
插件 = plugin_dir 下的单个 .lua 文件,做两件事:挂请求流水线的钩子、在启动时
贡献 WebUI 界面(整页或往现有页面追加组件)。两者独立。
## 流水线 stage(三个)
request_start 已解析鉴权、未选源
routed 已选定 (source, model)、未发往上游
request_end 每请求恰好一次,带最终计量
request_end 挂在 gateway.writeRec——四条入口路径(直连/AUTO × 流式/非流式)的
唯一汇合点:既不漏(流式 token 只有流结束才知道)也不重。
## 计费插件(plugins/billing.lua,默认 seed,开箱可用)
源 / 模型 / 密钥三个维度定价。token 价优先级 keys > models > default;per_request
固定价是**叠加**的(生图模型可以既算 token 又收固定费)。单位是 USD/单 token,
即各家 provider 的公布口径。累计 total / by_source / by_model / by_key / by_day。
失败请求保留 token 费用、丢弃固定费(可经 count_failures 翻转)。
界面 = 一个独立页 + 状态页顶部一块总开销 tile。
## 一个明确的设计边界
计费插件**只报表,不执法**。网关自己的配额会计(stats.go,入口强制)才是限额
权威,插件不参与任何路由/配额决策。两套独立会计若对不上,比一套功能略少的
更糟。
## ★ 中途改掉的一个根本设计错误
最初让插件复用适配器的**弹性 worker 池**(多状态)。这对适配器是对的(它们无
状态),对插件是错的:计费插件往 plugin.state 累加,多状态意味着总量被劈成
几份;而 SetState 写价格只写进其中一个 worker,钩子恰好跑到另一个时**所有请求
按 0 计费**。改为**单状态 + 互斥锁**。代价写进文档:钩子必须短、同步、不阻塞,
卡住的钩子会卡住所有插件的钩子。
这个 bug 是测试逼出来的——先写了 SetState+Fire 的用例,数字全是 0 才挖出来。
另一个连带缺陷:只带 prices 的 PUT 会整体替换 state,把累计量清零。改为
prices/state 分离——prices 是配置、state 是历史,改价不动账。
## 撞到的三个 Lua 绑定的坑(都写进注释)
- SetGlobal **会 pop 栈**:连着调两次,第二次从空栈取,赋成 nil
- GetField 索引越界是 **SIGABRT 整个进程**,不是 panic,recover 救不了
- Call(nargs, n) **不接受函数索引**,它调的是 nargs 个参数正下方那个;
传索引会调到参数上("attempt to call a table value")
另外 GetField/SetField 用绝对索引,SetTop(0) 之后必须重取。
## 错误隔离
钩子 error() 不影响转发:捕获 → 记进 hook_errors → 跳下一个插件。适配器出错
会让源进冷却,插件出错**零惩罚**——插件是可选功能。/api/plugins 的 hook_errors
让"坏掉的插件"可见而不是静默消失。
## 界面注入
GET /api/ui-inject 一次返回所有插件的扩展(侧栏需要全部 page 才能建好)。
WebUI 在首次 render **之前** await 注入:先插 HTML 再重建 <script> 让它执行
(innerHTML/template 插入的 script 不会执行,这正是要的效果——避免脚本跑在
自己 DOM 之前)。注入失败不影响仪表盘。
browser 侧 pluginAPI 暴露 fetchState / postState / onTabShown。
## 文档
docs/plugins.md —— 快速上手、加载与热更新、三个 stage 的完整字段表、界面扩展、
状态与 HTTP API、运行时约束(单状态/异常隔离/内置函数)、计费插件的定价与
计费策略、排错表、与适配器的对比表。
## 判据(328 个测试全绿,插件相关 33 个)
- 计费断言的是**具体金额**(0.00625 / 0.0402 / 0.0075…),不是"能加载"
- 4 个变异都红:钩子异常不隔离 / prices 清空累计 / 忽略 key 优先级 /
毫秒时间戳不换算
- UI 侧 6 个判据把注入顺序、script 执行时机、pluginAPI 名称、tab 路由、
anchor 四种形式、失败非致命全钉住
- 鉴权:state 读任意角色、写仅 admin
188 lines
6.3 KiB
Go
188 lines
6.3 KiB
Go
package gateway
|
|
|
|
import (
|
|
"encoding/json"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"os"
|
|
"path/filepath"
|
|
"strings"
|
|
"testing"
|
|
|
|
"llmsproxy/internal/config"
|
|
"llmsproxy/internal/core"
|
|
"llmsproxy/internal/lua"
|
|
)
|
|
|
|
// gatewayWithBilling boots a gateway with the bundled billing plugin loaded, so
|
|
// the UI-injection endpoint is exercised against a real plugin rather than a
|
|
// hand-written stub. The other plugin tests in this package assert on the
|
|
// WebUI source; this one asserts on the HTTP contract the browser consumes.
|
|
func gatewayWithBilling(t *testing.T) *Gateway {
|
|
t.Helper()
|
|
dir := t.TempDir()
|
|
cfgPath := filepath.Join(dir, "config.yaml")
|
|
body := "listen: :0\n" +
|
|
"adapter_dir: " + filepath.Join(dir, "adapters") + "\n" +
|
|
"plugin_dir: " + filepath.Join(dir, "plugins") + "\n" +
|
|
"runtime_file: " + filepath.Join(dir, "runtime.json") + "\n" +
|
|
"gateway_keys:\n - sk-test\n"
|
|
if err := os.WriteFile(cfgPath, []byte(body), 0600); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
cfg, err := config.Load(cfgPath)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
c, err := core.NewFromConfig(cfg)
|
|
if err != nil {
|
|
t.Fatalf("core: %v", err)
|
|
}
|
|
t.Cleanup(c.Close)
|
|
// Load the shipped plugin explicitly: seeding only runs for a directory that
|
|
// does not exist yet, and this test wants a known plugin regardless.
|
|
src, err := lua.ReadBundledPlugin("billing")
|
|
if err != nil {
|
|
t.Fatalf("read bundled billing: %v", err)
|
|
}
|
|
if err := c.Plugins().LoadSource("billing", src); err != nil {
|
|
t.Fatalf("load billing: %v", err)
|
|
}
|
|
g, err := New(c)
|
|
if err != nil {
|
|
t.Fatalf("gateway: %v", err)
|
|
}
|
|
return g
|
|
}
|
|
|
|
// TestUIInjectServesPluginUI: GET /api/ui-inject is the single call the WebUI
|
|
// makes at boot, and it must carry BOTH a contributed page and contributed
|
|
// elements — the browser builds the sidebar from the page and mounts the
|
|
// elements into existing panes from the same payload, so a partial response
|
|
// would produce a page with no body or a missing tile.
|
|
func TestUIInjectServesPluginUI(t *testing.T) {
|
|
g := gatewayWithBilling(t)
|
|
rr := doReq(t, g, http.MethodGet, "/api/ui-inject", "")
|
|
if rr.Code != http.StatusOK {
|
|
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
|
}
|
|
var view struct {
|
|
UI struct {
|
|
Page *struct {
|
|
PageID string `json:"page_id"`
|
|
Title string `json:"title"`
|
|
Mount string `json:"mount"`
|
|
} `json:"page"`
|
|
Elements []struct {
|
|
Target string `json:"target"`
|
|
Mount string `json:"mount"`
|
|
} `json:"elements"`
|
|
} `json:"ui"`
|
|
Stages []string `json:"stages"`
|
|
}
|
|
if err := json.Unmarshal(rr.Body.Bytes(), &view); err != nil {
|
|
t.Fatalf("decode: %v", err)
|
|
}
|
|
if view.UI.Page == nil || view.UI.Page.PageID != "billing" {
|
|
t.Fatalf("no billing page in the inject payload")
|
|
}
|
|
if !strings.Contains(view.UI.Page.Mount, "billing-root") {
|
|
t.Error("the page mount came back empty")
|
|
}
|
|
if len(view.UI.Elements) == 0 {
|
|
t.Error("billing contributes an element to the status page but it is missing")
|
|
}
|
|
for _, e := range view.UI.Elements {
|
|
if e.Target != "status" {
|
|
t.Errorf("element target = %q, want \"status\"", e.Target)
|
|
}
|
|
}
|
|
if len(view.Stages) != 3 {
|
|
t.Errorf("stages = %v, want the three pipeline stages", view.Stages)
|
|
}
|
|
}
|
|
|
|
// TestUIInjectIsEmptyWithoutPlugins: a gateway with no plugins must still answer
|
|
// 200 with an empty (not missing, not null) payload. The WebUI calls this
|
|
// unconditionally at boot, so a 404 or a null `ui` would break every dashboard.
|
|
func TestUIInjectIsEmptyWithoutPlugins(t *testing.T) {
|
|
g := newTestGateway(t)
|
|
rr := doReq(t, g, http.MethodGet, "/api/ui-inject", "")
|
|
if rr.Code != http.StatusOK {
|
|
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
|
}
|
|
if !strings.Contains(rr.Body.String(), `"ui"`) {
|
|
t.Error("no ui key in the response; the WebUI would have nothing to read")
|
|
}
|
|
}
|
|
|
|
// newRecorderFor pushes a request through the full handler chain.
|
|
func newRecorderFor(t *testing.T, g *Gateway, req *http.Request) *httptest.ResponseRecorder {
|
|
t.Helper()
|
|
rr := httptest.NewRecorder()
|
|
g.Handler().ServeHTTP(rr, req)
|
|
return rr
|
|
}
|
|
|
|
// TestPluginsListAndStateAPI covers the management surface the plugin docs
|
|
// promise: listing, and reading a plugin's own published state.
|
|
func TestPluginsListAndStateAPI(t *testing.T) {
|
|
g := gatewayWithBilling(t)
|
|
rr := doReq(t, g, http.MethodGet, "/api/plugins", "")
|
|
if rr.Code != http.StatusOK {
|
|
t.Fatalf("GET /api/plugins = %d: %s", rr.Code, rr.Body.String())
|
|
}
|
|
for _, want := range []string{"billing", "hook_errors", "plugin_dir", "request_end"} {
|
|
if !strings.Contains(rr.Body.String(), want) {
|
|
t.Errorf("/api/plugins response lacks %q", want)
|
|
}
|
|
}
|
|
// state read: the plugin published its (empty) state, so the key exists.
|
|
rr = doReq(t, g, http.MethodGet, "/api/plugins/billing/state", "")
|
|
if rr.Code != http.StatusOK {
|
|
t.Fatalf("GET state = %d: %s", rr.Code, rr.Body.String())
|
|
}
|
|
if !strings.Contains(rr.Body.String(), `"total"`) {
|
|
t.Errorf("billing state lacks the total bucket: %s", rr.Body.String())
|
|
}
|
|
}
|
|
|
|
// TestPluginStatePUTIsAdminOnly: only the WRITE side is gated. A user key must
|
|
// be able to READ its own billing widget's data, but must not be able to
|
|
// rewrite the price table.
|
|
func TestPluginStatePUTIsAdminOnly(t *testing.T) {
|
|
g := gatewayWithBilling(t)
|
|
|
|
// Build a user-role key and remember its secret.
|
|
rec, err := g.core.CreateKey("viewer", "user", nil, "")
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
userKey := rec.Key
|
|
|
|
// A user key may read the state.
|
|
req, _ := http.NewRequest(http.MethodGet, "/api/plugins/billing/state", nil)
|
|
req.Header.Set("Authorization", "Bearer "+userKey)
|
|
rr := newRecorderFor(t, g, req)
|
|
if rr.Code != http.StatusOK {
|
|
t.Errorf("user GET state = %d, want 200 (the billing widget must render for users)", rr.Code)
|
|
}
|
|
|
|
// A user key may NOT write it.
|
|
put, _ := http.NewRequest(http.MethodPut, "/api/plugins/billing/state",
|
|
strings.NewReader(`{"prices":{"default":{"prompt":0}}}`))
|
|
put.Header.Set("Authorization", "Bearer "+userKey)
|
|
put.Header.Set("Content-Type", "application/json")
|
|
prr := newRecorderFor(t, g, put)
|
|
if prr.Code != http.StatusForbidden {
|
|
t.Errorf("user PUT state = %d, want 403 (a user must not rewrite the price table)", prr.Code)
|
|
}
|
|
|
|
// An admin key may.
|
|
adm := doReq(t, g, http.MethodPut, "/api/plugins/billing/state",
|
|
`{"prices":{"default":{"prompt":1e-6}}}`)
|
|
if adm.Code != http.StatusOK {
|
|
t.Errorf("admin PUT state = %d: %s", adm.Code, adm.Body.String())
|
|
}
|
|
}
|