feat(plugins): 分片调用 —— 让阻塞型导出函数真正并行

## 问题

quota 页面 20 个源,刷新要 29 秒。页面用 `Promise.all` 并发发 20 个请求,
**看起来是并发的,实际不是**。

`Plugins.Call` 持 `p.mu.Lock()`,锁一直持有到 `L.Call` 返回 —— **包含 Lua 函数
内部的全部网络等待**。20 个请求到了网关后排队,串行累加。

生产实测:

    单源探测(无竞争)      1.4 – 3.3 s
    20 源 Promise.all       29 s 墙钟
    串行理论值              ≈ 29 s   ← 与实测吻合

代码里其实已经知道这个问题:quota.lua 第 104 行写着「the whole page shares ONE
Lua state under a write lock」,并为此把超时从 10s 压到 4s。但那只把 29s 变成
约 12s,没有解决「刷新不可用」。

而当初改成并发的理由(页面注释)是错的 —— 它假设并发能绕开串行。锁让它退化
回串行了。

## 不能去掉锁

一个 Lua VM 只有一个栈。两个并发调用会破坏栈。锁是必要的,答案只能是**让每个
并发调用有自己的 VM,跑同一份插件源码**。

先量,再写实现(8 分片 × 300ms 本地 HTTP 端点):

    共享 VM,8 次串行      2.407 s
    8 个独立 VM 并行       0.304 s    → 7.9x

顺带一个测量教训:这个 spike 第一版用**忙等**而不是网络 I/O,报出 2.3x。
那是量到了 12 核机器上的 CPU 争抢(负载 8.95),和锁无关。插件做的是网络等待,
spike 就必须做网络等待 —— 和上一条提交里 os.clock 那个错误是同一类。

## 实现

- `callOn(w, fn, args)`:从 `Call` 里抽出的「在某 VM 上调一个导出函数」。`Call`
  从 66 行缩到 18 行,仍负责查找、export 白名单和共享 VM 的锁。
- `shardPool`:私有 VM 池,上限 32。开机约 0.1ms,池化省不了什么但省两次刷新
  的重复开销。
- `ShardCall(name, fn, argsList)`:结果与输入**按位置对齐**。
- `POST /api/plugins/{name}/fanout/{fn}`:`{"args": [...]}`,上限 64 条。
- `pluginAPI.fanout()`:页面一次请求。
- quota.lua 改用 fanout。

## 分片的硬约束(写进注释,不是靠约定)

分片 VM **不能碰 PLUGIN.state** —— state 在 VM 里,每个分片一份副本,billing
的累加会各分叉,saver 写出去的是随机某一份。分片只能用于只读、幂等的工作。
所以 `callOn` 既不查 export 白名单(`Call` 那层才是安全边界)也不落盘。

## 一个分片失败不拖垮整批

一个上游连不上,不该让 20 张卡片变成一个错误。失败的分片在自己的位置留
null,其余照常返回。**但全部分片失败时报错** —— 那不是「某个上游挂了」,
是结构性问题,报成功会把它藏在满屏红卡片后面。

## 顺手修掉一个既有缺陷:热重载是坏的

写分片源码来源时,判据直接把它顶了出来:`LoadSource` 无条件 append,而
`findLocked` 返回**第一个**匹配。实测:

    LoadDir 后 1 条 → LoadSource 后 2 条
    [0] version=1   [1] version=2
    find 返回 version=1

也就是说,**编辑插件并保存看起来什么都没发生** —— 旧版本一直被返回,新版本
躺在旁边不生效。修法是先 Unload 再加载。这不是为分片做的,是分片正好照到了
这条路径,已单独补判据 `TestReloadReplacesRatherThanAppends`。

## 判据

七条:并行性(有界而非计时断言,8×200ms 串行是 1.6s,超过 1.0s 即红)、
位置对齐、未导出与未知插件被拒、单个失败不拖批、全失败报错、用已安装源码而
非磁盘、以及热重载替换而非追加。

全量 9 包通过。

Co-Authored-By: ModelRouter <noreply@modelrouter.dev>
This commit is contained in:
JianFeeeee
2026-10-04 21:56:45 +08:00
parent e5af4ec34c
commit ff8afaa43d
5 changed files with 741 additions and 8 deletions

View File

@ -117,6 +117,20 @@ func (g *Gateway) handlePluginsAPI(w http.ResponseWriter, r *http.Request) {
return
}
// /api/plugins/{name}/fanout/{fn} — one call per argument set, in
// parallel, from one request. See handlePluginFanout for why the
// request-per-item form cannot be made concurrent from the client.
if i := strings.Index(path, "/fanout/"); i >= 0 {
name := path[:i]
fn := path[i+len("/fanout/"):]
if name == "" || fn == "" {
writeError(w, http.StatusBadRequest, "invalid_request", "fanout needs a plugin and a function name")
return
}
g.handlePluginFanout(w, r, name, fn)
return
}
if reqRole(r.Context()) != "admin" {
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
return
@ -133,7 +147,7 @@ func (g *Gateway) handlePluginsAPI(w http.ResponseWriter, r *http.Request) {
// Visibility overrides and the endpoint access table, so the
// plugin page renders what is IN FORCE rather than making the
// operator read .lua files or config.yaml to find out.
"visibility": ps.VisibilityOverrides(),
"visibility": ps.VisibilityOverrides(),
"http_access": ps.EndpointAccess(),
})
case http.MethodPost:
@ -331,6 +345,68 @@ func (g *Gateway) handlePluginCall(w http.ResponseWriter, r *http.Request, name,
writeJSON(w, http.StatusOK, map[string]interface{}{"result": result})
}
// handlePluginFanout runs one plugin export per element of a list, in parallel,
// from a SINGLE request.
//
// Why this exists: the quota page has twenty cards, and it used to fire twenty
// browser requests at /call/. They arrived concurrently and then serialised
// inside Plugins.Call, which holds the plugin mutex across the Lua function —
// including its network wait. Measured on the live gateway: 20 probes took 29s of
// wall clock while a single probe with no contention took 1.4–3.3s, so the page
// sat on "probing…" for half a minute. The browser-side concurrency was real and
// made no difference.
//
// A fan-out cannot be assembled from the client either: any request-per-item
// arrangement is exactly the shape that queues. The parallelism has to happen
// where the fan-out is received.
func (g *Gateway) handlePluginFanout(w http.ResponseWriter, r *http.Request, name, fn string) {
if reqRole(r.Context()) != "admin" {
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
return
}
if r.Method != http.MethodPost {
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "")
return
}
var req struct {
// Args is a LIST of argument objects, one per call. A bare object is
// rejected rather than coerced: "args" meaning "one call" and "args"
// meaning "many" depending on its shape is how a caller ends up probing
// one source and believing it probed twenty.
Args []map[string]interface{} `json:"args"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
return
}
if len(req.Args) == 0 {
writeError(w, http.StatusBadRequest, "invalid_request", "args must be a non-empty list of argument objects")
return
}
// The page legitimately has one card per configured source. The bound is not
// arbitrary: it stops one request from asking for thousands of VMs, and it is
// enforced before any work starts rather than discovered shard by shard.
if len(req.Args) > fanoutMaxArgs {
writeError(w, http.StatusBadRequest, "invalid_request",
fmt.Sprintf("args has %d entries, limit is %d", len(req.Args), fanoutMaxArgs))
return
}
results, err := g.core.Plugins().ShardCall(name, fn, req.Args)
if err != nil {
writeError(w, http.StatusBadGateway, "plugin_error", err.Error())
return
}
// A shard that failed contributes a null in its slot. That is deliberate and
// is NOT the same as the whole request failing: one unreachable upstream must
// not turn a page of twenty cards into a single error.
writeJSON(w, http.StatusOK, map[string]interface{}{"results": results})
}
// fanoutMaxArgs bounds one fan-out request. Twenty is the measured real source
// count on the gateway this was built for; the headroom is for operators with
// more sources configured.
const fanoutMaxArgs = 64
func (g *Gateway) pluginDir() string {
if c := g.core.Config(); c != nil {
return c.PluginDir
@ -369,7 +445,6 @@ func (g *Gateway) removePlugin(name string) error {
return g.core.Plugins().Unload(name)
}
// handleEndpointAccess serves the operator's view and edit surface for one
// plugin endpoint's reachable keys.
//
@ -439,7 +514,6 @@ func (g *Gateway) handleEndpointAccess(w http.ResponseWriter, r *http.Request, n
}
}
// splitStateForKey rewrites a plugin state so that a non-admin caller sees only
// the buckets that belong to them.
//