Files
ModelRouter/internal/lua/fastvalue_test.go
JianFeeeee 30064696b3 perf(plugin): 去掉钩子热路径的 JSON 往返 + 同 stage 跨插件并行
## 1. 去掉 JSON 往返(快路径)
实测单次 Fire 14.6µs,其中 json.Marshal 4.0 + json.Unmarshal 5.6 = 9.6µs,
**67% 花在把 map[string]interface{} 序列化再反序列化**,而紧接着的
pushGoValue 本来就能直接遍历这两种类型。改为按类型直接转换(fastvalue.go),
只对不认识���类型才回落 JSON —— 陌生字段仍然会被送到插件,而不是消失。

快路径与 JSON 路径逐字节等价由 TestFastPathMatchesJSONPath 锁住(8 组载荷,
覆盖 int/uint/float 各宽度、嵌套、slice、map[string]string、未知类型)。
还有一条专门防止「优化悄悄失效」:TestFastPathIsActuallyUsed 用真实的
request_end 载荷断言它确实走快路径。

同一份代码 A/B 实测:JSON 往返 56.4µs → 快路径 35.3µs(省 37%)。

## 2. 同 stage 跨插件并行
参照 /home/program/TrueAgent 的 StageHost.RunStage:
  - **快照后释放锁**再并行 —— 它记录过一次自死锁(p.Stop → onExit → ReclaimOwner
    要拿 registry 锁,持锁并行即死锁)。这里同理:钩子可能经 admin API 增删插件,
    那条路径要拿 ps.mu 写锁,所以并行段内不持任何 ps 锁。
  - 每个 goroutine recover。
  - 单插件走直连路径,不付 goroutine 代价(生产就是这种配置)。

**与 TrueAgent 不同的一点**:它可以放心并行,因为 handler 只写 ctx.Response 并有
IsResponded() 仲裁;我们的钩子返回 table 会合并进 payload,而
docs/plugins.md 明确承诺「payload 原样传给下一个插件」。所以合并**按插件加载
顺序**执行,结果确定,不依赖调度;代价是钩子之间不再互相可见 —— 这是一处
**契约变化**,已在文档里写明,并说明随核心发布的 billing 从不返回任何值
(代码注释就写着 "nobody downstream would read a return value")。

实测收益(真实二进制,三实例对照,3000 请求):

              无插件     1 插件      4 插件
  稳态并发32    849 rps   768 (-9.5%) 741 (-12.7%)
  突发并发64   1524-1893  1182-1676  1064-1443

4 插件只降 10-20%,而并行前实测 4 插件是 63.8µs vs 单插件 14.6µs(-300%)。

## ★ 我自己造成的两次性能事故
**① 持久化把热路径拖慢 26 倍。** 最初的快照在钩子路径上做:走 luaValueToGo +
json.Marshal + json.Unmarshal 三重转换,每请求 264µs,Fire 从 14.6µs 变成 385µs。
改成 saver 按自己节奏拉取(钩子只标记 dirty,flush 时才快照),385µs → 25.7µs。
**这里还踩了第二次 use-after-free**:让后台 goroutine 去读 Lua 表,vm.Stop() 后
那是已释放内存(SIGSEGV)。安全性现在由「Plugins.Close 等 saver 的最后一次
flush 完成后,调用方才停 VM」保证。

**② 基准被自己的后台写入污染。** 关掉 markDirty 反而测出 36µs、比开着还慢,
方向完全反了 —— 是 saver 每 2 秒写盘混进了计时。加了 DisableStatePersistence
后数据才可信。

## 判据(11 项,全部变异验证)
快路径等价/确实生效/不别名输入 + 并行与单插件路径合并一致 + 合并顺序确定 +
抛异常的钩子不拖累同伴 + 每插件恰好执行一次 + 并发 Fire 安全 + Fire 期间不持
注册表锁 + 真实 billing 在并行下正常 + 持久化 7 项。

变异:改坏合并顺序 → 红;去掉单插件路径的合并 → 红(3 个既有测试同时抓到)。
★ 「删掉 recover」这个变异**没有**让判据变红,查下去发现 golua 把 error()、
nil 索引、调用 nil、深递归全部转成 error RETURN,不产生 Go panic —— 那个测试
根本没测到 recover。已改名 TestThrowingHook 并在注释里写明 recover() 当前无法
被 Lua 触达,保留它是为了守 Go 侧。留一个「看起来有覆盖」的断言比没有更糟。

## 端到端(真实二进制 + 真实 billing)
20 万请求全 200,rps 1870,p99 96ms,RSS 37.9→42MB 有界;
负载停止后四个插件计数**完全一致**(231745),hook_errors 为空;
systemctl restart 后 billing 仍是 231745 —— 并行与持久化同时生效。
381 个测试全绿,含 -race。
2026-10-02 10:45:20 +08:00

133 lines
5.1 KiB
Go

package lua
import (
"encoding/json"
"reflect"
"testing"
)
// The hook payload used to be JSON round-tripped on every call. It no longer is,
// so the fast path and the old JSON path must be indistinguishable — otherwise
// a plugin silently sees a different payload than before, which is the worst
// kind of change: it compiles, passes a smoke test, and misprices traffic.
//
// These tests therefore compare the two paths on the SAME inputs rather than
// asserting the fast path's output in isolation.
func viaJSON(t *testing.T, payload map[string]interface{}) interface{} {
t.Helper()
b, err := json.Marshal(payload)
if err != nil {
t.Fatalf("marshal: %v", err)
}
var decoded interface{}
if err := json.Unmarshal(b, &decoded); err != nil {
t.Fatalf("unmarshal: %v", err)
}
return decoded
}
// payloads that exercise every branch of the fast converter.
func payloadCases() []map[string]interface{} {
return []map[string]interface{}{
{"model": "deepseek-v4.1-flash", "source": "commandcode", "ok": true},
// Numbers of every width: a converter that only knows float64 turns
// ints into something else, and a token count that arrives as a string
// makes a Lua hook do arithmetic on nil.
{"i": 42, "i8": int8(8), "i16": int16(16), "i32": int32(32), "i64": int64(1 << 40),
"u": uint(7), "u64": uint64(1 << 50), "f32": float32(1.5), "f64": 2.25},
// Zero, negative, and very large values must stay numbers.
{"zero": 0, "neg": -17, "huge": 1e308, "tiny": 1e-308},
// Slices and string maps: common in payloads and absent from a switch
// that only knows []interface{} / map[string]interface{}.
{"msgs": []interface{}{"a", "b"}, "tags": []string{"x", "y"}},
{"kv": map[string]string{"a": "1", "b": "2"}},
// Nesting, which is where a shallow converter silently drops a level.
{"usage": map[string]interface{}{
"prompt_tokens": 2048, "cache_hit_tokens": 1024,
"nested": map[string]interface{}{"deep": []interface{}{1, "two", true, nil}},
}},
{"nil_field": nil, "empty_map": map[string]interface{}{}, "empty_slice": []interface{}{}},
// A value the fast path does NOT model: it must fall back to JSON and
// still arrive, not disappear.
{"weird": struct {
A int `json:"a"`
B string `json:"b"`
}{1, "x"}},
}
}
func TestFastPathMatchesJSONPath(t *testing.T) {
for i, p := range payloadCases() {
want := viaJSON(t, p)
got := plainForLua(p)
if !reflect.DeepEqual(want, got) {
t.Errorf("case %d: fast path differs from JSON path\n payload: %#v\n json: %#v\n fast: %#v",
i, p, want, got)
}
}
}
// TestFastPathIsActuallyUsed guards against the fast path silently degrading to
// JSON for the payload the gateway really sends. If a future payload gains a
// type the converter does not model, this still works (it falls back) but the
// optimization is gone — and the next person measuring the hook would be
// measuring the old cost without knowing why.
func TestFastPathIsActuallyUsed(t *testing.T) {
// This mirrors the real request_end payload shape from the gateway.
realistic := map[string]interface{}{
"stage": "request_end", "kind": "end", "model": "deepseek-v4.1-flash",
"source": "commandcode", "key": "stress-key", "ok": true,
"status": 200, "duration_ms": 1234,
"usage": map[string]interface{}{
"prompt_tokens": float64(2048), "completion_tokens": float64(512),
"cache_hit_tokens": float64(1024), "total_tokens": float64(3584),
},
"walk": []interface{}{
map[string]interface{}{"kind": "tier_skip", "tier": 1, "source": "", "model": "", "reason": "cooldown"},
map[string]interface{}{"kind": "selected", "tier": 2, "source": "commandcode", "model": "m", "reason": ""},
},
"ts": float64(1700000000),
}
if _, ok := fastToPlain(realistic); !ok {
t.Errorf("★ the realistic request payload does NOT take the fast path — " +
"the JSON round-trip is still on the hot path for real traffic")
}
}
// TestFastPathDoesNotAliasInput: the converter builds a new tree. If it ever
// returned the caller's map directly, a Lua hook's writes could not reach Go —
// but worse, a later mutation of the payload would race with the snapshot the
// persistence saver is holding.
func TestFastPathDoesNotAliasInput(t *testing.T) {
src := map[string]interface{}{
"usage": map[string]interface{}{"prompt_tokens": float64(1)},
"list": []interface{}{"a"},
}
out, ok := fastToPlain(src)
if !ok {
t.Fatal("fast path declined a plain payload")
}
m := out.(map[string]interface{})
m["new"] = "added"
src["also_new"] = "must not appear"
if _, leaked := m["also_new"]; leaked {
t.Error("output map aliases the input map")
}
if _, leaked := src["new"]; leaked {
t.Error("writing to the output mutated the input")
}
// And the nested maps must be copies too.
inner := m["usage"].(map[string]interface{})
inner["prompt_tokens"] = float64(999)
if src["usage"].(map[string]interface{})["prompt_tokens"] != float64(1) {
t.Error("nested map is shared, not copied — a hook could mutate the payload")
}
list := m["list"].([]interface{})
list[0] = "changed"
if src["list"].([]interface{})[0] != "a" {
t.Error("nested slice is shared, not copied")
}
}