test(memory): bound what P1's green does and does not establish (P1 follow-up)

Two discipline holes left after 1309afe / 09be4de, both closed here.

1. The boundary was unstated. A one-hot keyword-family fixture proves the
   PLUMBING — that the computed query vector reaches scoring — and says nothing
   about retrieval quality. Left implicit, a green P1 invites the reading
   "semantic recall works". The measured counter-evidence is now in the file:
   on the real chinese-clip space a true hit scored 0.8345 while an unrelated
   block scored 0.8598, so absolute scores are not calibrated and no threshold
   may be read off them. That is a known unresolved problem outside P1's scope.

2. `dsh` missing used to `t.skip`. A skip keeps `npm test` green while the only
   end-to-end evidence for P1 never runs — the same shape as a criterion that is
   committed but never speaks. It now fails, and says why.

No behavioural change: 272/272 tests, 12/12 checkpoints, 0 skipped on a box
where `dsh` is present. Verified the mutation discipline still bites — deleting
`'hybrid'` from ORDINARY_DEGRADATIONS turns L10, L11, L10† and L11† red.
This commit is contained in:
2026-11-05 01:22:57 +08:00
parent 09be4de0eb
commit 28e18d0258

View File

@ -47,6 +47,15 @@
*
* Without P1a/P1b a passing test would only show that *something* answered.
*
* ★ What this does NOT establish.
* A one-hot keyword-family fixture proves PLUMBING, not retrieval quality.
* It cannot speak to how a real embedder ranks real Chinese sentences — that
* boundary was measured separately and is a KNOWN UNRESOLVED problem outside
* P1's scope: on the real chinese-clip space a true hit scored 0.8345 while an
* unrelated block scored 0.8598, so absolute scores are not calibrated and no
* threshold may be read off them. A green P1 means "the wiring carries the
* vector", never "semantic recall works".
*
* ## The corpus, and why these exact strings
*
* The fake embedder is a deterministic keyword-family projection into 8 unit
@ -62,21 +71,25 @@
* extractor (`[A-Z]{2}[0-9]{2}-[0-9]{3}`) can find it and nothing else in the
* prompt can collide with it.
*
* ## Skip conditions
* ## Missing-harness conditions
*
* The harness needs `dsh` and the two `dsh-base` peers resolvable from this
* package. When they are not installed there is no headless process to drive, and
* the test skips LOUDLY rather than asserting against a half-mounted app — the
* same rule `loop-system-e2e.test.js` follows.
* The harness needs `dsh` resolvable from PATH. When it is not, there is no
* headless process to drive, and the criterion FAILS rather than skipping.
*
* ★ Why `assert.fail` and not `t.skip`: a skip is green. `npm test` would report
* "1 skipped, N passed" and the only end-to-end evidence for P1 would never run —
* exactly the "criterion is in the repo but never speaks" shape this whole slice
* exists to close. It is also worse than a plain hole, because a green suite reads
* as proof. `loop-system-e2e.test.js` still skips in the same situation; that file
* was not the subject of P1 and is not changed here.
*
* @module dsh-memory/test/phase-b-p1-vector-wiring
*/
import assert from 'node:assert/strict'
import { execFileSync, spawn } from 'node:child_process'
import { createReadStream, existsSync, mkdtempSync, rmSync } from 'node:fs'
import { existsSync, mkdtempSync, rmSync } from 'node:fs'
import { createServer } from 'node:http'
import { createServer as createNetServer } from 'node:net'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { fileURLToPath } from 'node:url'
@ -573,8 +586,11 @@ test('P1-0: the corpus is admissible — symbol path blind, vector path reaches
test('P1: a semantic-only question is answered from block memory via the vector route', async (t) => {
if (!dshAvailable()) {
t.skip('no `dsh` binary on PATH — the headless harness cannot be driven here')
return
assert.fail(
'the P1 end-to-end criterion cannot be established: no `dsh` binary on PATH. ' +
'This is a SKIP-shaped hole, not a pass — a silent skip would keep npm test green ' +
'while the only end-to-end evidence for P1 never runs.',
)
}
const workspace = mkdtempSync(join(tmpdir(), 'p1-e2e-'))