mirror of
https://gitcode.com/JianFeeeee/webui4frpc.git
synced 2026-10-03 23:53:59 +00:00
fix(cluster): 停用的转发会在重启后自己复活 + 令牌轮转日志刷屏
从线上三节点（192.168.2.{30,106,60}）的日志里挖出四个问题，本轮修三个。
## 1. 停用的转发会复活（功能性缺陷，实测仍在发生）
线上现象：`minecraft` 在 store 里 disabled=1，worker 却仍在跑，今天
09:46 还在刷 `connect to local service [192.168.2.60:25565]: connection
refused` —— 对着一个用户刻意没启的本地服务死刷。同时三台 logs 里躺着
13956 / 3769 条 `proxy [x] already exists`，每 33 秒一轮。
四处叠加导致：
- stopForward 先读 link，再 SetLinkDisabled(true)，然后把**改之前**的
副本交给 RevokeTask ⇒ published task 带 disabled=false（实测 id/flag
都对不上：topology 里 link.id=223，store 里同一行是 199）
- ClaimFn **无条件** SetLinkDisabled(...,false)。原意是"重新认领时清掉
停用标记"，但启动 reconcile 只要 worker 不在就重新 Claim ⇒ 每次重启
都是"先清标记再起 worker"
- RevokeFn 停了 worker 却**没删 topology 条目**，条目活过 worker
- 于是下次重启 reconcile 看到"owned 但 worker 不在"→ 再次 Claim → 死循环
修法（把 disabled 的所有权交回两个用户动作）：
- Claim **只读** disabled 决定要不要起 worker；为 true 时连 topology
条目一起摘掉，绝不复活
- 启动 reconcile 先按 store 跳过 disabled 的条目（省掉无谓的
claim→skip 往返）
- RevokeFn 除停 worker 外，同时 RemoveTopologyEntry —— 撤销必须是
完整退役，不能只是"停一下"
- stopForward 把 Disabled=true 随 task 发布出去，让持有该转发的节点
即使本地 store 行陈旧也能判断这次停用是用户主动的
## 2. 令牌轮转日志零信息量却占满磁盘
每轮固定 3 行（OnToken cycle=N / forward cycle=N / token-send -> 200），
2 轮/秒，实测本机 **355 行/分钟、7 天 357 万行**，把真事件全淹了。
同一份信息（cycle / lastSync / roundDelayMs / 成员存活）本来就能从
GET /api/manager/cluster/ring 结构化拿到。
加 W4F_DEBUG 开关（沿用项目既有 W4F_ 前缀约定）：稳态三行降级为 debug、
默认关闭；**失败路径一律保留** —— 发送失败、陈旧令牌、非 2xx 正是别人
grep 的对象，静音它们是坏交易。实测同样 12 秒：36 行 → 4 行。
## 3. Link.ID 在 ReplaceLinks 之后必然失效
ReplaceLinks 是 DELETE + 重新 INSERT，sqlite 给每行**新的自增 id**。
任何在改写前捕获的 Link（典型：随 token 环跑的 Link）手里的 id 要么查无
此行，要么命中另一条转发 —— 实测捕获 alpha id=1，改写后新表是 3/4/5，
GetLink(1) 直接落空。
新增 LinkByTriple(local, remote, port) 按自然键查（业务代码本来就一律用
这个三元组标识转发），并把 claim/reconcile 切过去。查无行返回
(Link{}, false, nil) 而非 error：新建的转发没有行，应当照常启动。
## 4. homeagent_device 孤儿（已澄清，非独立缺陷）
它 disabled=1 且从不在 topology 里，是缺陷 1 的另一面（停用标记没进
token），随本次修复覆盖，无需单独处理。
## 测试
新增 4 个测试文件，重点是**验证测试本身抓得住 bug**：
- 临时回退 `ln.Disabled = true` 这行 → TestStopForwardPublishesDisabled-
FlagInRevokeTask 如期变红，还原后变绿
- ⚠️ 第一版回归测试只断言 store 层，是**假绿**：newTestHandler 的 Ring
为 nil，RevokeTask 那条（真正坏掉的）路根本没执行。补了带 ring 的
newRingTestHandler，直接断言**发布出去的 task 上的 flag**
- LinkByTriple 在 ReplaceLinks 前后保持稳定；GetLink(id) 的失效被固化成
一个可见的说明性测试
- 停用/start 往返、per-forward 停用不误伤兄弟转发
- 错误路径不静音、W4F_DEBUG 各种取值
go build / go vet / go test ./... 全绿，gofmt 干净。
This commit is contained in:
JianFeeeee
198
internal/store/link_test.go
Normal file
198
internal/store/link_test.go
Normal file
@ -0,0 +1,198 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// seed inserts the local + remote rows a link's foreign keys require.
|
||||
// (links.local / links.remote reference their own tables, so a link cannot
|
||||
// exist on its own — the same reason ClaimFn upserts them before ReplaceLinks.)
|
||||
func seed(t *testing.T, st *Store, locals []string, remote string) {
|
||||
t.Helper()
|
||||
for _, n := range locals {
|
||||
if err := st.UpsertLocal(Local{Name: n, IP: "127.0.0.1", Port: 8080, Protocol: "tcp"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := st.UpsertRemote(Remote{Name: remote, IP: "1.2.3.4", Port: 7000, Token: "tok", Enabled: true}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestLinkByTripleSurvivesReplaceLinks pins the reason LinkByTriple exists.
|
||||
//
|
||||
// ReplaceLinks() rewrites the whole table with DELETE + re-INSERT, so sqlite
|
||||
// hands every row a FRESH autoincrement id. A Link captured before such a
|
||||
// write (e.g. one riding inside a ring token) therefore carries an id that
|
||||
// either matches a different forward or matches nothing. The natural key
|
||||
// (local, remote, remotePort) is what every caller actually identifies a
|
||||
// forward by, and it must survive those rewrites.
|
||||
func TestLinkByTripleSurvivesReplaceLinks(t *testing.T) {
|
||||
st, err := New(filepath.Join(t.TempDir(), "test.db"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer st.Close()
|
||||
|
||||
seed(t, st, []string{"alpha", "beta", "gamma"}, "srv")
|
||||
links := []Link{
|
||||
{Local: "alpha", Remote: "srv", RemotePort: 100},
|
||||
{Local: "beta", Remote: "srv", RemotePort: 200},
|
||||
{Local: "gamma", Remote: "srv", RemotePort: 300},
|
||||
}
|
||||
if err := st.ReplaceLinks(links); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Capture the ids as the ring would have them.
|
||||
before := map[string]int64{}
|
||||
all, err := st.ListLinks()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, l := range all {
|
||||
before[l.Local] = l.ID
|
||||
}
|
||||
if len(before) != 3 {
|
||||
t.Fatalf("expected 3 links, got %d", len(before))
|
||||
}
|
||||
|
||||
// Rewrite the table (this is what saveCanvas and ClaimFn both do).
|
||||
if err := st.ReplaceLinks(links); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
after, err := st.ListLinks()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(after) != 3 {
|
||||
t.Fatalf("expected 3 links after rewrite, got %d", len(after))
|
||||
}
|
||||
|
||||
// The natural key must still resolve to the right forward, with its
|
||||
// disabled flag and group intact.
|
||||
for _, l := range after {
|
||||
if l.Disabled {
|
||||
t.Errorf("link %s unexpectedly disabled after a plain rewrite", l.Local)
|
||||
}
|
||||
}
|
||||
got, found, err := st.LinkByTriple("beta", "srv", 200)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !found {
|
||||
t.Fatal("LinkByTriple failed to find beta after ReplaceLinks")
|
||||
}
|
||||
if got.Local != "beta" || got.RemotePort != 200 {
|
||||
t.Fatalf("LinkByTriple returned the wrong row: %+v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestLinkByTripleNotFoundIsNotError documents the contract callers rely on:
|
||||
// "no persisted opinion yet" is (Link{}, false, nil), not an error. A fresh
|
||||
// claim of a link with no row must be allowed to start.
|
||||
func TestLinkByTripleNotFoundIsNotError(t *testing.T) {
|
||||
st, err := New(filepath.Join(t.TempDir(), "test.db"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer st.Close()
|
||||
|
||||
got, found, err := st.LinkByTriple("nope", "srv", 1234)
|
||||
if err != nil {
|
||||
t.Fatalf("missing link must not be an error, got %v", err)
|
||||
}
|
||||
if found {
|
||||
t.Fatalf("missing link reported as found: %+v", got)
|
||||
}
|
||||
if got.Local != "" || got.RemotePort != 0 {
|
||||
t.Fatalf("expected zero Link on miss, got %+v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestLinkByTripleReadsDisabledFlag is the store-level half of the
|
||||
// "stopped forwards resurrect on restart" bug: the claim path asks the store
|
||||
// whether the user disabled this forward, so this lookup must return the flag
|
||||
// as persisted.
|
||||
func TestLinkByTripleReadsDisabledFlag(t *testing.T) {
|
||||
st, err := New(filepath.Join(t.TempDir(), "test.db"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer st.Close()
|
||||
|
||||
seed(t, st, []string{"mc"}, "srv")
|
||||
if err := st.ReplaceLinks([]Link{{Local: "mc", Remote: "srv", RemotePort: 25565, Group: "game"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := st.SetLinkDisabled("mc", "srv", 25565, true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
ln, found, err := st.LinkByTriple("mc", "srv", 25565)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !found {
|
||||
t.Fatal("expected to find the link")
|
||||
}
|
||||
if !ln.Disabled {
|
||||
t.Fatal("expected Disabled=true to be visible through LinkByTriple")
|
||||
}
|
||||
if ln.Group != "game" {
|
||||
t.Fatalf("group should survive, got %q", ln.Group)
|
||||
}
|
||||
|
||||
// ...and the user-facing start path must be able to clear it again.
|
||||
if err := st.SetLinkDisabled("mc", "srv", 25565, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ln, _, _ := st.LinkByTriple("mc", "srv", 25565); ln.Disabled {
|
||||
t.Fatal("expected Disabled=false after clearing")
|
||||
}
|
||||
}
|
||||
|
||||
// TestGetLinkByIDIsStaleAfterReplaceLinks documents WHY callers must not use
|
||||
// GetLink(id) with a previously captured id. It is not a fix — it is the trap
|
||||
// being pinned shut, so the hazard stays visible if someone reintroduces it.
|
||||
func TestGetLinkByIDIsStaleAfterReplaceLinks(t *testing.T) {
|
||||
st, err := New(filepath.Join(t.TempDir(), "test.db"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer st.Close()
|
||||
|
||||
seed(t, st, []string{"alpha", "beta", "gamma"}, "srv")
|
||||
if err := st.ReplaceLinks([]Link{
|
||||
{Local: "alpha", Remote: "srv", RemotePort: 100},
|
||||
{Local: "beta", Remote: "srv", RemotePort: 200},
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
all, _ := st.ListLinks()
|
||||
var staleID int64
|
||||
for _, l := range all {
|
||||
if l.Local == "alpha" {
|
||||
staleID = l.ID
|
||||
}
|
||||
}
|
||||
|
||||
if err := st.ReplaceLinks([]Link{
|
||||
{Local: "alpha", Remote: "srv", RemotePort: 100},
|
||||
{Local: "beta", Remote: "srv", RemotePort: 200},
|
||||
{Local: "gamma", Remote: "srv", RemotePort: 300},
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// The old id may still resolve, but to whatever row now occupies that
|
||||
// id — which is exactly the silent-mis-target hazard. Assert that the
|
||||
// natural key remains the only safe handle.
|
||||
if ln, ok := st.GetLink(staleID); ok && ln.Local != "alpha" {
|
||||
t.Logf("stale id %d now points at %q (hazard confirmed; use LinkByTriple)", staleID, ln.Local)
|
||||
}
|
||||
if got, found, _ := st.LinkByTriple("alpha", "srv", 100); !found || got.Local != "alpha" {
|
||||
t.Fatalf("natural key must stay reliable, got %+v found=%v", got, found)
|
||||
}
|
||||
}
|
||||
@ -560,6 +560,30 @@ func (s *Store) GetLink(id int64) (Link, bool) {
|
||||
return l, true
|
||||
}
|
||||
|
||||
// LinkByTriple looks a link up by its natural key (local, remote, remotePort).
|
||||
//
|
||||
// Prefer this over GetLink(id) whenever the caller only knows the forward's
|
||||
// identity: ReplaceLinks() rewrites the whole table with DELETE + re-INSERT, so
|
||||
// every row gets a fresh autoincrement id. Any id captured before such a write
|
||||
// (e.g. a Link carried inside a ring token) is stale by definition and will
|
||||
// either miss or — worse — match a different forward. The natural key is
|
||||
// stable across those rewrites.
|
||||
//
|
||||
// Returns (link, found). A missing row is (Link{}, false) and is NOT an error:
|
||||
// callers use that to mean "no persisted opinion yet".
|
||||
func (s *Store) LinkByTriple(local, remote string, port int) (Link, bool, error) {
|
||||
var l Link
|
||||
err := s.db.QueryRow("SELECT id, local, remote, remote_port, offset_x, offset_y, grp, disabled FROM links WHERE local = ? AND remote = ? AND remote_port = ?", local, remote, port).
|
||||
Scan(&l.ID, &l.Local, &l.Remote, &l.RemotePort, &l.OffsetX, &l.OffsetY, &l.Group, &l.Disabled)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return Link{}, false, nil
|
||||
}
|
||||
return Link{}, false, err
|
||||
}
|
||||
return l, true, nil
|
||||
}
|
||||
|
||||
// DeleteLink removes a single link by id.
|
||||
func (s *Store) DeleteLink(id int64) error {
|
||||
_, err := s.db.Exec("DELETE FROM links WHERE id = ?", id)
|
||||
|
||||
Reference in New Issue
Block a user