diff --git a/docs/dev-notes-mutation-criteria.md b/docs/dev-notes-mutation-criteria.md index 4382eda..a5784a3 100644 --- a/docs/dev-notes-mutation-criteria.md +++ b/docs/dev-notes-mutation-criteria.md @@ -1,7 +1,7 @@ -# 变异判据的三类失效模式 +# 变异判据的四类失效模式 > 来源:phase B(H 组 W1–W12)三轮返工中实际踩到、并由评审独立复现的坑。 -> 三条都不是本任务特有的,**任何写变异判据的人都会再踩**。 +> 四条都不是本任务特有的,**任何写变异判据的人都会再踩**。 > > 共同点:**它们都让判据绿着,而判据守的东西已经坏了。** 这类问题比被测代码坏了更难发现, > 因为失败的信号本身就是「一切正常」。 @@ -13,7 +13,7 @@ > **变异体必须让 `lib/` 下的代码变坏。** > 一旦变异动的是 `test/`、或者断言的是测试里自己拼的局部函数,它就不是变异,是自测。 -把这一条当硬规则执行,下面三条里至少两条会被自动拦住。 +把这一条当硬规则执行,下面四条里前两条会被自动拦住。 --- @@ -200,6 +200,78 @@ cpSync(join(pkgRoot, 'node_modules'), join(dir, 'node_modules'), { recursive: tr --- +## 4. 判据被替换时,旧判据的覆盖静默消失 + +### 形态 + +一条测试同时承担两件事。你删掉其中一件时,另一件的覆盖跟着没了—— +**而没有任何东西会因为少了这个覆盖而变红。** + +本阶段真实发生:旧的 README 守卫同时含两条断言: + +```js +// ① README 的计数与实现一致(后判定不值得守) +assert.equal(Number(claim[1]), declared, ...) +// ② checkpoints.js 至少注册了一条 ← 值得守 +assert.ok(declared > 0, 'no checkpoint calls found — did the helper get renamed?') +``` + +删①时把整条 test 换掉了,②一起消失。 + +### 实测后果(很安静) + +把 helper 换成一个什么都不做的空操作(调用点全保留): + +``` +$ npm run check +0/0 checkpoints passed +$ echo $? +0 ← 退出码 0 +$ npm test +# tests 119 # pass 119 # fail 0 ← 全套仍全绿 +``` + +**checkpoint 全废了,`npm run check` 报「通过」,119 条测试一条都不红。** + +### 正确形态:守卫要守结果,不要守成因 + +补回一条静态计数只能抓「helper 改名」。放在 runner 里守**结果**,能覆盖所有成因: + +```js +if (results.length === 0) { + console.error('no checkpoints were registered — did the helper get renamed or stubbed?\n') + process.exitCode = 1 +} +``` + +| 成因 | runner 内守卫 | 静态计数 | +|---|---|---| +| helper 改名 | ✔ | ✔ | +| helper 被 stub 成空操作 | ✔ | ✘(调用点还在,计数不变) | +| 条件跳过(`if (skip) return`) | ✔ | ✘ | +| 提前 `process.exit(0)` | ✘(**拦不住**) | ✘ | + +最后一行是关键:**`process.exit(0)` 绕过任何脚本内的检查**。所以还要一条外部断言, +证明 runner 里仍然含有那道守卫: + +```js +assert.match(script, /if \(results\.length === 0\)/) +assert.match(script, /results\.length === 0[\s\S]{0,200}process\.exitCode = 1/) +``` + +### 自查问句 + +> **这条判据除了它名字里说的那件事,还顺手守了什么?** +> +> 删/改之前,把它的断言逐条过一遍,对每一条问: +> 「这条如果不守了,谁会红?」——答案是「没人红」的那些,要么补新守卫,要么明确记下放弃。 + +> 附一条探针经验:验证「零注册」不要用「把 `await checkpoint(` 注释掉」—— +> 那会留下孤立的 `return`,报 `SyntaxError: Illegal return statement`,测的是语法。 +> 用「换成一个仍存在但什么都不做的函数」,调用点保持不变,才是有效探针。 + +--- + ## 通用自查清单 写/改一条变异判据前,逐条过: @@ -213,11 +285,13 @@ cpSync(join(pkgRoot, 'node_modules'), join(dir, 'node_modules'), { recursive: tr - [ ] 跑完确认**无残留**(`.orig` / `.rej` / `/tmp/mut-*`) - [ ] 连跑 **≥5 次**(间歇性问题 3 次绿证明不了什么) - [ ] 文档里**没有**手抄的运行计数 +- [ ] 替换/删除旧判据时,逐条问过「这条覆盖现在谁守」 -## 一句话记住三条 +## 一句话记住四条 | 坑 | 一句话 | |---|---| | 自测型假变异 | 变异不碰 `lib/`,它就永远不会红。 | | 清单写死 | 判据的输入和代码脱节时,绿的是记忆不是事实。 | | 并发撞车 | 变异共享状态不能与代码并发跑 —— 改副本。 | +| 替换带走覆盖 | 一条判据往往守两件事;删一件时另一件不会有人提醒你。 | diff --git a/scripts/checkpoints.js b/scripts/checkpoints.js index 890b4f9..2f3d214 100644 --- a/scripts/checkpoints.js +++ b/scripts/checkpoints.js @@ -421,6 +421,11 @@ console.log(`\n${passed}/${results.length} checkpoints passed\n`) console.log(`plugin: name=${pluginName} inject=[${pluginInject.join(', ')}]`) console.log(`workspace: ${workspace} (removed)\n`) +if (results.length === 0) { + console.error('no checkpoints were registered — did the helper get renamed or stubbed?\n') + process.exitCode = 1 +} + if (failed > 0) { console.error(`${failed} checkpoint(s) FAILED\n`) process.exitCode = 1 diff --git a/test/phase-b-g-static.test.js b/test/phase-b-g-static.test.js index bcf4a61..dfcac56 100644 --- a/test/phase-b-g-static.test.js +++ b/test/phase-b-g-static.test.js @@ -184,6 +184,44 @@ test('G2b†: a disallowed runtime import must be reported with file and package assert.deepEqual(offenders, ['lib/store.js: onnxruntime-node']) }) +// ── the checkpoint runner must not be able to pass with zero checkpoints ── +// +// Replacing the old README-count guard silently removed its second assertion: +// "scripts/checkpoints.js declares at least one checkpoint". Nothing guards that +// any more, and the failure is invisible — stubbing the helper yields +// `0/0 checkpoints passed`, exit code 0, and every test still green. +// +// Two layers, because they catch different causes: +// · the runner itself exits non-zero when `results.length === 0` (rename, +// stub, conditional skip); and +// · this assertion proves the runner still contains that guard, because an +// early `process.exit(0)` bypasses any in-script check entirely. +test('scripts/checkpoints.js registers checkpoints AND refuses to pass with zero', () => { + const root = new URL('..', import.meta.url).pathname + const script = readFileSync(join(root, 'scripts', 'checkpoints.js'), 'utf8') + + // 1. It actually registers something. Guards against a rename of the helper. + const declared = [...script.matchAll(/await checkpoint\('/g)].length + assert.ok( + declared > 0, + 'scripts/checkpoints.js declares no `await checkpoint(...)` calls — did the helper get renamed?', + ) + + // 2. It refuses to report success when nothing ran. This is the guard that a + // stub or a conditional skip trips; an early `process.exit(0)` can only be + // caught here, by asserting the guard exists. + assert.match( + script, + /if \(results\.length === 0\)/, + 'the runner must contain an explicit zero-results guard', + ) + assert.match( + script, + /results\.length === 0[\s\S]{0,200}process\.exitCode = 1/, + 'the zero-results guard must set a non-zero exit code', + ) +}) + // ── README: no unguarded counts ─────────────────────────────────────────── // // This used to assert that README's test/checkpoint counts matched reality. That