mirror of
https://gitcode.com/JianFeeeee/webui4frpc.git
synced 2026-10-15 12:14:37 +00:00
feat(cluster): 负载均衡改为每节点自主「先撤销本地转发，再丢回环」
按用户要求重做：不再由 leader 单独调度，而是**每个节点对自己的转发跑同一条
规则**，超了就自己撤销一条、把任务丢回环上，由环原有的「最闲者认领」决定它
落到哪台。
## 为什么必须是 owner 自己动手（不是洁癖，是实测逼出来的）
真机探针（跑完已删）显示，真实 revoke 之后：
TopologyOwner after revoke = "n1" ← 条目还在（只标记不删除）
n1 Forwards=3 score=310.0 ← 负载根本没降
after step: revoking=true publishing=false
⇒ 一节点的负载**只有它自己的拓扑条目消失时才会下降**，而条目是本地簿记，没有任何
对端可以代它写。leader 从外面只能看见失衡、无法让失衡成为事实——它必须伸手去改
另一个节点的簿记，而这正是这个环到处都在避免的跨节点写（revoke / restart 定向往
owner 也是同一个理由）。
⇒ 而且旧实现**根本跑不起来**：phase 1 判据是「条目消失」，可 mark-don't-remove
让它永远不消失，于是每 120s 打一条 `revoking … timed out; abandoning`，
**phase 2 永不执行，一条转发都不会迁**。开 `W4F_BALANCE_MODE=on` 只会刷日志。
## 规则（替代原来的 ratio 阈值）
当 Forwards >= 2 且 Forwards > ceil(总数/存活节点数) 时，撤销一条
- **自限**：3 条 / 3 台 → owner 2 > 1，卸一条 → 1，`1 > 1` 不成 → 停。
没有上界的话 owner 会把转发**全部卸光**。
- **不会卸空**：低于下限（2）永不撤销。
- 平摊值向上取整（3/3=1，不是 0），否则 1 条 / 3 台的节点也算「超了」。
- **不再用 mem/net 参与判据**：实测 `.30` 空闲时 netPct 能到 55%，Score 会高于
有转发的节点——用 Score 会把纯网络抖动当成负载。
- 节奏用**每节点冷却**（默认 600s）而不是「失衡持续多久」窗口：规则本身已经要求
真实失衡，没有需要平滑的瞬态，只有需要限制的抖动频率。
## 为什么必须是新的 Release 任务类型，不能复用 Revoke
用户停止（Revoke）**保留拓扑条目**作为 disabled 标志的载体；均衡释放必须**删除
条目**，因为 (a) 负载由自己的条目数算出，条目不删负载不降；(b) 每个节点的
duplicate-guard 会拒绝「已有 owner 的转发」的创建任务——保留条目会让**所有节点
都拒领**，转发被永久搁死在刚放弃它的那台机器上。
Release 因此：`Handler.Release`（停 worker，但**不标记 stopped**）→ 删条目 →
`SubmitTask` 以**启用状态**重新发布。它需要自己的 Handler 方法，不能回落到
Revoke：Revoke 会顺手把 link 标成 stopped，而这条转发即将在别的节点上跑。
## 顺带修掉两个既有缺口（实测确认）
1. **`submitMigrate` 发布不出去**：`OnToken` 的 localPending 重合并只保留「从未
publish 过」的本地 pending，而只有 `Add*` 系列会记 `published`。引擎里直接写
`PendingTasks` 的任务会在下一次 adopt 时被静默丢弃。
⇒ 新 `ReleaseTask` 走 `AddPending`。
（`SubmitRestart` 有同样的毛病，但它不在本次范围，未改动——见提交说明末尾。）
2. `Release` 需要绕过 duplicate-claim 守卫（同 Restart），否则「owner 本来就持有
这条转发」这一前提会让每条 Release 都被丢掉。
## 测试（17 条，全绿）
规则：平摊值取整、超了才撤、撤到平摊就停、单节点不撤、低于下限不撤、不撤已停
转发、候选选择确定。
机制：Revoke 留载体 vs Release 删条目（这是两种语义的核心差异）、Release 只由
owner 执行且不消费他人的任务、owner 释放后环的普通认领把它接走。
模式：dry-run 不动任何东西、off 真的什么都不做、未知模式回落 dry-run、
冷却限制抖动频率、确认超时清理。
只读：Report 不变更任何状态、能解释「超了但不动」的原因、均衡节点给出真实答案。
**双向验证 3 条**（改坏→变红→还原→变绿）：
- 去掉 Release 的 duplicate-guard 例外 → `the owner released 0 times, want 1`
- 去掉「不超过平摊值」上界 → `2 owned against share 2 must not release`
- Release 改成只标记不删除 → `n1 should now own 2, got 3`
go build / go vet / go test ./... 全绿（go clean -testcache 后逐包实跑）。
## 未做 / 待办
- `SubmitRestart` 的同类发布缺口未修（同样会被 adopt 丢弃），需要单独确认它
在实机上是否真的丢过 restart。
- UI 尚未跟进新报告结构（下一步）。
This commit is contained in:
JianFeeeee
@ -290,6 +290,21 @@ func main() {
|
||||
log.Printf("ring[%s] revoked task %s: %s→%s:%d", selfID, tk.ID, tk.Local.Name, tk.Remote.Name, tk.Link.RemotePort)
|
||||
return nil
|
||||
},
|
||||
// ReleaseFn stops this forward's worker for a REBALANCE, leaving the
|
||||
// link ENABLED. This must not reuse RevokeFn: that marks the link
|
||||
// stopped, and the released forward is about to run on another node —
|
||||
// a "stopped" flag would make the claim path treat the re-published
|
||||
// task as a disabled forward and refuse to start a worker for it,
|
||||
// leaving the forward dead everywhere.
|
||||
ReleaseFn: func(ctx context.Context, tk *cluster.Task) error {
|
||||
key := process.WorkerKey(tk.Local.Name, tk.Remote.Name, tk.Link.RemotePort)
|
||||
if _, running := pm.Status(key); running {
|
||||
_ = pm.Stop(key)
|
||||
}
|
||||
log.Printf("ring[%s] released worker %s (rebalancing): %s→%s:%d", selfID, key,
|
||||
tk.Local.Name, tk.Remote.Name, tk.Link.RemotePort)
|
||||
return nil
|
||||
},
|
||||
// RestartFn brings an already-claimed forward back after a stop. It
|
||||
// must NOT run the claim bookkeeping (ClaimFn), which appends links and
|
||||
// writes topology state: the forward is already established, and a stop
|
||||
@ -392,7 +407,7 @@ func main() {
|
||||
// Load balancer is created here (before the Handler) so the HTTP layer can
|
||||
// expose its read-only report. Mode/parameters are applied further down, once
|
||||
// the W4F_BALANCE_* environment has been parsed.
|
||||
balancer := cluster.NewBalancer(ring, 2, 1800)
|
||||
balancer := cluster.NewBalancer(ring, 600)
|
||||
|
||||
h := &httpapi.Handler{
|
||||
Store: st,
|
||||
@ -562,41 +577,40 @@ func main() {
|
||||
go ring.WatchTokenLoss(clusterCtx)
|
||||
go ring.WatchLeader(clusterCtx)
|
||||
|
||||
// Load balancing. DRY-RUN BY DEFAULT.
|
||||
// Load balancing — SELF-DIRECTED, and dry-run by default.
|
||||
//
|
||||
// The balancer moves forwards off an overloaded node, which means stopping a
|
||||
// frpc worker and dropping its live connections. That is fine occasionally
|
||||
// (a genuinely hot node) but destructive as a reflex, so it ships off and is
|
||||
// opted into explicitly:
|
||||
// Every node runs this check against its OWN forwards. There is no
|
||||
// coordinator: a node's load only actually falls when its own topology entry
|
||||
// goes away, so only the owner can produce the state change that makes the
|
||||
// ring hand a forward to somebody else. The node gives a forward up and
|
||||
// re-publishes it as an ordinary unclaimed task; the ring's normal
|
||||
// lowest-load claim then places it. (The earlier design had the leader pick a
|
||||
// victim and a destination — but it can only see the imbalance, not resolve
|
||||
// it, so it had to reach into another node's bookkeeping to make its decision
|
||||
// true.)
|
||||
//
|
||||
// W4F_BALANCE_MODE=dry-run report what it would do, move nothing (default)
|
||||
// W4F_BALANCE_MODE=on actually migrate
|
||||
// W4F_BALANCE_RATIO=2 overload = Score above RATIO x the mean
|
||||
// W4F_BALANCE_SUSTAIN=1800 seconds an imbalance must persist
|
||||
// A release stops a worker and drops a live connection, so acting on the ring
|
||||
// is opt-in:
|
||||
//
|
||||
// W4F_BALANCE_MODE=dry-run report the decision, release nothing (default)
|
||||
// W4F_BALANCE_MODE=on actually release
|
||||
// W4F_BALANCE_MODE=off do nothing at all
|
||||
// W4F_BALANCE_COOLDOWN=600 min seconds between two releases by one node
|
||||
// W4F_BALANCE_INTERVAL=10 check every N seconds
|
||||
//
|
||||
// Only the leader balances (WatchBalance no-ops elsewhere): moving forwards is
|
||||
// a cluster-wide decision and two members doing it independently would race on
|
||||
// the same token.
|
||||
balMode := strings.ToLower(strings.TrimSpace(os.Getenv("W4F_BALANCE_MODE")))
|
||||
if balMode == "" {
|
||||
balMode = "dry-run"
|
||||
balMode = cluster.BalanceDryRun
|
||||
}
|
||||
balRatio := envFloat("W4F_BALANCE_RATIO", 2)
|
||||
balSustain := int64(envFloat("W4F_BALANCE_SUSTAIN", 1800))
|
||||
balInterval := time.Duration(envFloat("W4F_BALANCE_INTERVAL", 10)) * time.Second
|
||||
if balInterval < time.Second {
|
||||
balInterval = 10 * time.Second
|
||||
}
|
||||
balancer.SetRatioAndSustain(balRatio, balSustain)
|
||||
if balMode == "on" {
|
||||
log.Printf("ring[%s] load balancing ENABLED (ratio=%.2f sustain=%ds interval=%s)",
|
||||
selfID, balRatio, balSustain, balInterval)
|
||||
} else {
|
||||
log.Printf("ring[%s] load balancing in DRY-RUN (set W4F_BALANCE_MODE=on to enable; "+
|
||||
"ratio=%.2f sustain=%ds interval=%s)", selfID, balRatio, balSustain, balInterval)
|
||||
}
|
||||
go balancer.WatchBalance(clusterCtx, balInterval, balMode != "on")
|
||||
balCooldown := int64(envFloat("W4F_BALANCE_COOLDOWN", 600))
|
||||
balancer.SetCooldown(balCooldown)
|
||||
balancer.SetMode(balMode)
|
||||
log.Printf("ring[%s] load balancing %s (cooldown=%ds interval=%s, self-directed)",
|
||||
selfID, strings.ToUpper(balMode), balCooldown, balInterval)
|
||||
go balancer.Watch(clusterCtx, balInterval)
|
||||
|
||||
// Startup reconcile: every node (leader or not) checks its own topology
|
||||
// entries and re-claims any whose per-forward frpc worker process is
|
||||
|
||||
Reference in New Issue
Block a user