jianf/HologramAgent-DSH-Memory-Port
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

fix(memory): wire the computed query vector into recall scoring (P1)

Browse Source 
`recallBlocksHybrid` computed `queryVector`, verified its length against the
declared space, and then threw it away:

    return withDiagnosis(recallBlocks(store, query, scoped), 'vector_ready_unused')

`scoped` carries only `topK` / `signal` / `fingerprint` / `dimension`. So every
V1–V7 scoring branch inside `recallBlocks` was live code reachable ONLY from
unit tests that hand-built a `vectorScores` Map — `vectorScores:` appeared 10
times in test/phase-b-a.test.js and ZERO times under lib/. Production recall was
pure symbol/FTS: the whole vector half of phase B was unreachable, and the only
symptom was a diagnostic label nobody was watching.

Why BOTH options are now passed (structural, not stylistic)

  · `vectorScores` alone reaches only the V1 exact-band tie adjustment (:438).
    The fused candidate assembly at :233 is gated on `Number.isFinite(minScore)`,
    so without a threshold the vector route can never admit a block the symbol
    path missed — which is exactly what V5 exists for.
  · `minScore` alone does nothing: :299 additionally requires `vectorScores` to
    BE a Map before it builds vector candidates.
One seam, three guards. Half of it is a no-op, which is why measuring the
half-wiring mutation below matters.

Scheme A, and why not B

A: keep the existing API (`vectorScores: Map<string, number>` + `minScore`), and
   have `recallBlocksHybrid` score the query vector against the space's stored
   vectors before calling `recallBlocks` once.
B: change `recallBlocks`'s signature to accept a `queryVector` and score inside.

Chosen: A. B would touch a signature that phase A's golden fixture and nine call
sites depend on being synchronous and vector-free (`recallBlocks` MUST NOT return
a thenable — A1 pins it structurally), and it would move cosine arithmetic into a
function whose contract is literally "store + query text". A leaves
`recallBlocks` byte-untouched: the seam it already exposes is driven, so A2's
45-case golden comparison cannot drift. Both routes must also be fused in ONE
scoring pass anyway (V5 admits vector-only candidates; V7 forbids truncating
before fusion), so "call twice and merge" was never an option.

Cost, measured before choosing (this box, 2000 blocks x 512 dims)

  narrow `SELECT id, vector`    21 ms   (vs 70 ms for `allBlocks(-1)`)
  parse + cosine over 2000      60 ms
  ⇒ ~81 ms per recall; at 8000 blocks ~310 ms.

Affordable because it runs once per TURN (pre-step), not per candidate, and
because the scan is bounded by SQL: `fingerprint = ?` and `vector_dim = ?` are
pushed into the store so rows from a retired space or a wrong width are never
decoded. A phase-A store (no embedded blocks at all) pays one indexed SELECT
returning no rows. It does not add a new order of cost either: the fused path
already walks every block for symbol union (`allBlocks(-1)`); this replaces that
blind decode with a narrow one for the vector half. The remaining ceiling is a
single very large space — a number to measure before raising topK, not a reason
to ship a path that never runs. Fail-closed per §0.2/F5: blocks with an empty
fingerprint are deliberately not scored.

New diagnostic modes, both non-faults

  `hybrid`           the healthy wired path (symbol + vector fused). Explicit
                     `case` in `degradationNotice` because `default:` exists to
                     make an UNKNOWN mode say "unknown, not absent" — routing the
                     healthiest mode through it would print a false degradation
                     notice on every turn that had a vector hit.
  `no_vector_space`  a space is configured and the embedder answered, but no
                     block in it carries a usable vector (fresh space). Says
                     "nothing was scored" without claiming a provider failure.
                     An empty Map must NOT be passed to the fused route: it would
                     open candidate assembly and score symbol matches as
                     `0.7*0 + 0.3*coverage`, the V2 regression.

The criterion (test/phase-b-p1-vector-wiring.test.js)

End-to-end, because a unit test cannot see this bug: the defect is that nobody
passes the options, so asserting the options work proves nothing. The chain is
real sqlite file → real `apply()`/`commit()` write path (no hand-built vectors,
no injected embedder object: the plugin's own HTTP adapter is used) → real
`recallBlocksHybrid` → real headless `dsh --profile headless` with the shipped
`cordis.patch.yml` loop overlay (so P0 still applies) → the persisted
`session.v4.jsonl.zstd` → the stub LLM's answer, which is a pure function of the
system prompt it received.

Corpus, MEASURED before any assertion was written. Deterministic keyword-family
projection into 8 unit axes: family 0 is the target's vocabulary, families 1..7
are one axis per distractor, unmapped text → zero vector. Query
「上次说的那个四位数是什么来着？」:

  symbol path hits (target)   0     ← precondition 1
  hand-built vectorScores top1 = target, score 0.70   ← precondition 2
  query shares no CJK 2-gram with any block

Both preconditions are asserted in-suite (P1-0) so a later corpus edit cannot
make the e2e criterion unfalsifiable. An earlier attempt used the real
chinese-clip-vit-b16 model; its top-1 for the same kind of query was an unrelated
block (0.83 "入职培训"), so that corpus could never have gone green — the fake
embedder makes the question decidable instead of probabilistic.

The access code is `QP47-812` rather than a bare number so the stub's extractor
(`[A-Z]{2}\d{2}-\d{3}`) cannot collide with anything else in the prompt.

Born red against c1f7e06 (clean `git archive` of that commit, same test file):

  P1: the answer must come from block memory through the vector route.
    expected: QP47-812
    actual:   "未知"
    symbol-path hits: 0 (must be 0)
    recall diagnosis: degraded=vector_ready_unused hits=0
    recall section injected: (none)

Mutations — three, each a real edit to lib/recall.js, each restored after:

  M1 drop the wiring      `recallBlocks(store, query, scoped)`  → P1 red
  M2 half the seam        `minScore = undefined`                → P1 red
  M3 empty the scan       `const rows = []`                     → A3d red

  M1 raw: not ok 2 - P1: a semantic-only question is answered from block
          memory via the vector route
            expected: QP47-812
            actual:   "未知"
            recall diagnosis: degraded=hybrid hits=0
  M2 raw: identical text — and that is the point. Passing `vectorScores` alone
          keeps the new `hybrid` label looking healthy while the answer is still
          empty, because `minScore` is what opens the fused candidate assembly.
  M3 raw: Expected values to be strictly equal — `hybrid` unreachable, so the
          label is evidence rather than decoration.

Restoration is verified by hash, not by eye: `sha256sum lib/recall.js` is
8eb0faa660c973ea5114268001d49cf43f83ef92b2945897cfc6f7512541a498 before and
after every mutation, and `git diff` (staged vs worktree) is empty.

Test updates forced by the wiring (not cosmetic)

  A3c   the old assertion was "a healthy embedder reports vector_ready_unused,
        NOT hybrid". That was correct while the vector was unused; over a corpus
        with no vectors in the active space the honest label is now
        `no_vector_space`, and A3d pins `hybrid` once a vector exists.
  A3c†  replaced a test that only compared two local string literals (it could
        never fail for a real reason) with a mutation of the actual scoring pass.
  D3b   `hybrid` instead of the group-A stage label.
  index the two new modes are added to ORDINARY_DEGRADATIONS, so a healthy wired
        recall does not log a warning every turn.

Not in this slice: C4 (centering) is the next layer on this exact path and
deliberately not done inline, so its mutation stays citable. V6 and C5 untouched.

旁证 (not acceptance): npm test 268/268, npm run check 12/12.
This commit is contained in:
jianf
2026-11-05 00:07:29 +08:00
parent c1f7e06b22
commit 1309afec93
7 changed files with 1128 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
lib/index.js
View File

@ -79,14 +79,19 @@ export const inject = ['systemPrompt', 'tools']
/**
* Degradations that are NOT faults.
*
* `ok` is healthy. `no_embedder` is the SHIPPING DEFAULT (a standing
* configuration fact, the same judgement W11e made for the model-facing notice),
* `aborted` is a deliberate stop, and `vector_ready_unused` is a build stage.
* None of them justifies a warn on every turn; all of them are still worth a
* line when `debugLog` is on. Everything else — `embed_error`, and any mode a
* future branch invents — is a real fault and speaks even with `debugLog` off.
* `ok` is healthy. `hybrid` is the healthy path once vectors participate (P1),
* `no_embedder` is the SHIPPING DEFAULT (a standing configuration fact, the same
* judgement W11e made for the model-facing notice), `aborted` is a deliberate
* stop, `no_vector_space` means the space exists but holds no comparable vectors
* (a fresh space — a data state, not an incident), and `vector_ready_unused` is
* the group-A build stage label, kept so an older build's log line still reads.
* None of them justifies a warn on every turn; all of them are still worth a line
* when `debugLog` is on. Everything else — `embed_error`, and any mode a future
* branch invents — is a real fault and speaks even with `debugLog` off.
*/
const ORDINARY_DEGRADATIONS = new Set(['ok', 'no_embedder', 'aborted', 'vector_ready_unused'])
const ORDINARY_DEGRADATIONS = new Set([
'ok', 'hybrid', 'no_embedder', 'aborted', 'no_vector_space', 'vector_ready_unused',
])
/**
* Logger for one plugin instance: `ctx.logger` when the cordis build has one,

196
lib/recall.js
View File

@ -46,6 +46,21 @@
// ported from go: internal/memory/block_recall.go:263.
const largeTopK = 2000
/**
* Default vector-route admission threshold for the fused recall path.
*
* Ported from Go `blockRecallMinScore` (internal/agent/core/toolcall.go:1405),
* which is the value production HA actually passes. Its comment there records
* why the threshold must NOT be 0: without it, recall on a query his gograph
* has nothing on still returns high-scoring noise, and abstention (`AbstainCheck`)
* becomes the only thing standing between the model and a confident wrong answer.
*
* ★ This is an admission gate for the PURE VECTOR route only. `recallBlocks`
* applies it per path (see the V5 block), so an exact pin and a symbol hit are
* unaffected — that asymmetry is the whole point of V5 and is pinned separately.
*/
export const DEFAULT_MIN_SCORE = 0.5
/**
* Extract high-information symbols from a query.
*
@ -550,7 +565,7 @@ function withSkipped(hits, skipped, skippedDim = 0) {
*
* @param {Array<object>} hits
* @param {string|null} mode one of `no_embedder` / `embed_error` / `aborted` /
* `vector_ready_unused` / null
* `no_vector_space` / `vector_ready_unused` / null
* @param {Error} [error]
* @returns {Array<object>} the same array
*/
@ -592,10 +607,17 @@ function withDiagnosis(hits, mode, error) {
/**
* Vectors-assisted recall.
*
* Phase B group A ships only the *seam*, not the fusion: with no embedder (or a
* failing one) this is exactly `recallBlocks`. Group V fills in
* `options.vectorScores`; until then the vector call is deliberately not wired
* into scoring, so this wrapper cannot perturb phase A behaviour.
* With no embedder (or a failing one) this is exactly `recallBlocks`. With a
* healthy one it scores the query vector against the space's stored vectors and
* hands `recallBlocks` BOTH `vectorScores` and `minScore`, which together open
* the fused candidate assembly, the V1 exact-band tie adjustment, the V2 weighted
* blend and the V5 asymmetric gate.
*
* ★ Why one call and not two: the symbol and vector routes are fused into ONE
* scoring pass so that a block the symbol path missed can still be admitted by
* its vector score (V5) and so that the candidate pool is never truncated before
* fusion (V7). Calling `recallBlocks` twice and merging afterwards cannot
* reproduce either property.
*
* Degradation is reported on the returned array (`arr.degraded`), never by
* throwing — the recall main path must survive a dead embedding service
@ -673,12 +695,146 @@ export async function recallBlocksHybrid(store, embedder, query, options = {}) {
new Error(`embedder returned dim ${queryVector.length}, declared ${info.dimension}`))
}
// Group V wires `vectorScores` into scoring. Keeping the call here (and the
// scoring seam closed) means group A ships a verified no-op on the hot path.
// Group A only computes the vector; it is not in scoring yet. Reporting
// 'hybrid' here would be a lie until group V wires it in, and a misleading
// label is worse than no label — rename it when V actually participates.
return withDiagnosis(recallBlocks(store, query, scoped), 'vector_ready_unused')
// ── the wiring ────────────────────────────────────────────────────────────
//
// This is the line P1 exists for. Until now the query vector was computed,
// dimension-checked, and then DISCARDED: the call below passed `scoped`, which
// carries `topK` / `signal` / `fingerprint` / `dimension` and nothing else. So
// every V1–V7 scoring branch in `recallBlocks` was reachable only from unit
// tests that hand-built a `vectorScores` Map; production recall was pure
// symbol/FTS.
//
// ★ BOTH options have to be passed, and the reason is structural rather than
// stylistic:
// · `vectorScores` alone reaches only the V1 exact-band tie adjustment
// (`:438`), because the fused candidate assembly at `:233` is gated on
// `Number.isFinite(minScore)`; without it the vector route can never
// admit a block the symbol path missed, which is exactly what V5 needs.
// · `minScore` alone does nothing at all, because `:299` additionally
// requires `vectorScores` to BE a Map before it builds vector candidates.
// The two are one seam split across three guards, so they travel together.
//
// ★ Scheme A (score the existing space) over scheme B (teach `recallBlocks` to
// accept a query vector). B would change a signature that phase A's golden
// fixture and nine call sites depend on being synchronous and vector-free, and
// it would move cosine arithmetic into a function whose contract is "store +
// query text". A leaves `recallBlocks` completely untouched: the seam it
// already exposes is the seam we drive.
//
// Cost, measured on this box before choosing (2000 blocks x 512 dims, the
// profile's default space width): narrow `SELECT id, vector` 21 ms + parse and
// cosine 60 ms ≈ 81 ms per recall; at 8000 blocks ≈ 310 ms. That is affordable
// because the scan is bounded by the space and width filters below and runs
// once per TURN (pre-step), not per candidate. Two design choices keep it from
// growing worse than linear:
// · the space/width filters are pushed into SQL, so blocks from a retired
// space or a wrong width are never decoded;
// · a space holding no comparable vectors scores nothing at all, so a database
// with no embedded blocks (every phase-A store) pays one indexed SELECT that
// returns no rows and takes the symbol path untouched.
// The remaining risk is a very large single space, which is a ceiling to
// measure before raising `topK`/space size — not a reason to ship a vector path
// that never runs. Note the fail-closed consequence (§0.2/F5): blocks whose
// fingerprint is empty are NOT scored, by design.
//
// ★ This does not introduce a NEW order of cost. `recallBlocks` already walks
// every block for symbol union when `minScore` is finite (the `fusedCandidates`
// step, `allBlocks(-1)`), so the fused path was always O(total blocks); what is
// added here is one narrow indexed read plus the cosine arithmetic, and the
// decode is bounded to the active space and width instead of the whole table.
const vectorScores = scoreVectorCandidates(store, queryVector, scoped)
const vectorRoute = vectorScores.size > 0
if (!vectorRoute) {
// An empty map would still open the fused candidate assembly (minScore is
// finite) and then admit symbol matches with a vector weight of 0 — the
// `0.7*0 + 0.3*coverage` regression V2 forbids. Reporting the honest reason
// and taking the untouched symbol path is both correct and cheaper.
return withDiagnosis(recallBlocks(store, query, scoped), 'no_vector_space')
}
// `minScore` is only defaulted when the caller did not decide: an explicit
// `0` is a deliberate "no threshold" and must survive (it is how the V7 and
// pool-truncation criteria disable the gate).
const minScore = Number.isFinite(scoped.minScore) ? scoped.minScore : DEFAULT_MIN_SCORE
return withDiagnosis(
recallBlocks(store, query, { ...scoped, vectorScores, minScore }),
// 'hybrid' from here on is a statement of fact, not an aspiration: the vector
// route contributed at least one scoreable candidate to the same call that
// produced these hits. Group A shipped `vector_ready_unused` precisely so this
// rename could not happen before the wiring existed.
'hybrid',
)
}
/**
* Cosine similarity of the query vector against every block in its space.
*
* Returns the `Map<id, score>` that `recallBlocks` consumes. An empty Map means
* "this space has nothing to score", which the caller must treat as the symbol
* path rather than as a vector route with score 0 everywhere.
*
* Vectors are stored as JSON text (`memory_blocks.vector`, per the DDL comment:
* "vector is a JSON array, matching HA's memory_blocks.vector"). A row that fails
* to parse is SKIPPED rather than thrown on: one corrupt payload must not take
* the whole recall down (decision §0.4), and the migration path already records
* corrupt widths as a negative `vector_dim` so they are filterable.
*
* ★ No centroid centering here. C4 (subtract the space mean from BOTH sides) is a
* separate slice layered on top of this one; doing it inline now would mix two
* concerns in one diff and make C4's mutation unciteable.
*
* ★ Not L2-normalised on the way out. Cosine already divides by both norms, and
* `putBlock` does not promise unit vectors, so normalising would change nothing
* except to hide a zero vector behind a NaN.
*
* @param {import('./store.js').BlockStore} store
* @param {number[]} queryVector already dimension-checked by the caller
* @param {{fingerprint?: string, dimension?: number}} options
* @returns {Map<string, number>}
*/
function scoreVectorCandidates(store, queryVector, { fingerprint, dimension } = {}) {
const scores = new Map()
// Without a space identity there is nothing to compare against: the vectors in
// the table may belong to several incompatible spaces (§0.1), and a cosine
// across two of them is a number without a meaning.
if (typeof fingerprint !== 'string' || fingerprint === '') return scores
const rows = store.vectorCandidates(fingerprint, dimension)
if (rows.length === 0) return scores
let queryNorm = 0
for (const value of queryVector) queryNorm += value * value
queryNorm = Math.sqrt(queryNorm)
if (!(queryNorm > 0)) return scores
for (const row of rows) {
let vector
try {
vector = JSON.parse(String(row.vector ?? ''))
} catch {
continue
}
if (!Array.isArray(vector) || vector.length !== queryVector.length) continue
let dot = 0
let norm = 0
for (let index = 0; index < vector.length; index += 1) {
const value = Number(vector[index])
// A non-numeric element makes the whole score meaningless. Skip the row
// instead of letting NaN propagate into the sort, where it would silently
// randomise the order (NaN comparisons are always false).
if (!Number.isFinite(value)) {
dot = Number.NaN
break
}
dot += value * queryVector[index]
norm += value * value
}
if (!Number.isFinite(dot) || !(norm > 0)) continue
scores.set(row.id, dot / (queryNorm * Math.sqrt(norm)))
}
return scores
}
/**
@ -746,6 +902,16 @@ export function degradationNotice(hits) {
// may say "matches below"; nothing === false => the branch must point at them.
const nothing = !Array.isArray(hits) || hits.length === 0
switch (mode) {
case 'hybrid':
// NOT a degradation: this is the healthy, fully-wired path (symbol + vector
// fused in one scoring pass). It carries no notice for the same reason `ok`
// would not — there is nothing the model needs to know.
//
// ★ Kept as an explicit case rather than relying on the `default:` branch.
// `default:` exists to make an UNKNOWN mode fail loudly-safe ("unknown, not
// absent"), so routing the healthiest mode through it would print a false
// degradation notice on every single turn that had a vector hit.
return ''
case 'no_embedder':
// Permanent configuration state, not a per-turn failure. The constant index
// (systemPrompt.context -> renderIndex) already tells the model that block
@ -777,6 +943,14 @@ export function degradationNotice(hits) {
// yet. Not something the model needs to hear about — but keeping the case
// explicit means a future mode cannot silently fall through to ''.
return ''
case 'no_vector_space':
// A space IS configured and the embedder answered, but no block in that
// space carries a usable vector (a fresh space, or every row pre-dates the
// embedder). Unlike `no_embedder` this is a transitional DATA state, and it
// is the one an operator most often mistakes for "semantic recall is on".
// Still silent for the model: the fallback it received is the correct one,
// and `stablemem_stats` is where the operator is told why.
return ''
default:
// A mode nobody anticipated. "Not found" must not read as "not recorded":
// this is the ONLY path a future mode can take, so it is the worst possible

45
lib/store.js
View File

@ -1068,6 +1068,51 @@ export class BlockStore {
.all(limit)
}
/**
* Candidate pool for the vector route: every block comparable to one space.
*
* Three narrowings, each of which `recallBlocks` would otherwise have to repeat
* (and would eventually repeat inconsistently — the exact half-coverage bug F2
* exists to stop):
*
* · `vector != ''` a row with no vector cannot score; decoding it
* would only produce a zero.
* · `fingerprint = ?` same space. Comparing vectors across spaces is
* meaningless, not merely inaccurate (§0.1/§0.2).
* · `vector_dim = ?` same width, when the caller names one. A stale
* row of the wrong width is a decode hazard, and
* `vector_dim` is maintained by a trigger, so
* this filter is exact and index-friendly.
*
* ★ `SELECT id, vector` and not `SELECT *`: the scoring pass needs the vector
* and the id, nothing else. Measured on this box at 2000 blocks x 512 dims,
* the narrow read is 21 ms against 70 ms for `allBlocks(-1)` — and the gap is
* dominated by the text columns, which grow with note length while vectors do
* not.
*
* ★ Why this lives in the store and not in `recall.js`: every other query in
* this file is a prepared statement over one table; a recall module that
* reached for `raw()` to hand-build SQL would put storage knowledge back into
* the scoring layer.
*
* @param {string} fingerprint the vector space to score against
* @param {number} [dimension] required width; omitted means "any width"
* @returns {Array<{id: string, vector: string, vector_dim: number}>} raw JSON text
*/
vectorCandidates(fingerprint, dimension) {
const space = String(fingerprint ?? '')
if (space === '') return []
const sql = Number.isInteger(dimension) && dimension > 0
? `SELECT id, vector, vector_dim FROM memory_blocks
WHERE fingerprint = ? AND vector != '' AND vector != 'null' AND vector_dim = ?`
: `SELECT id, vector, vector_dim FROM memory_blocks
WHERE fingerprint = ? AND vector != '' AND vector != 'null'`
const statement = this.#handle().prepare(sql)
return Number.isInteger(dimension) && dimension > 0
? statement.all(space, dimension)
: statement.all(space)
}
/** Close the handle. Idempotent (checkpoint V5). */
close() {
if (this.#db === null) return

2
package.json
View File

@ -18,7 +18,7 @@
"node": ">=22.5.0"
},
"scripts": {
"test": "node --test test/store.test.js test/plugin.test.js test/loader.test.js test/phase-b-a.test.js test/phase-b-g.test.js test/phase-b-g-static.test.js test/phase-b-h.test.js test/memory-log.test.js test/loop.test.js test/loop-ports.test.js test/loop-system-e2e.test.js",
"test": "node --test test/store.test.js test/plugin.test.js test/loader.test.js test/phase-b-a.test.js test/phase-b-g.test.js test/phase-b-g-static.test.js test/phase-b-h.test.js test/phase-b-p1-vector-wiring.test.js test/memory-log.test.js test/loop.test.js test/loop-ports.test.js test/loop-system-e2e.test.js",
"check": "node scripts/checkpoints.js"
},
"peerDependencies": {

100
test/phase-b-a.test.js
View File

@ -222,14 +222,22 @@ test('A3 mutation: a wrapper that adds any suffix to `why` must break A3', async
}
})
// ── the SUCCESS path's label (group A: vector computed, not yet used) ──────
// ── the SUCCESS path's label (P1 flips it: vectors now participate) ────────
//
// The earlier A3b only pinned the two failure labels. Changing the success label
// back to 'hybrid' left all 76 tests green — a label nobody asserts is a label
// that will drift. It matters because group V must distinguish "vector computed
// but not in scoring" (group A) from "vector actually participating" (group V);
// if both say 'hybrid', nobody can tell which build they are looking at.
test('A3c: a healthy embedder reports "vector_ready_unused", NOT "hybrid"', async () => {
// A3b only pinned the two failure labels. Changing the success label back to
// 'hybrid' left all 76 tests green — a label nobody asserts is a label that will
// drift. The two success labels answer DIFFERENT questions and both still matter:
//
// `no_vector_space` the space exists but holds no comparable vector (a
// fresh space). Says "nothing was scored" without
// claiming a provider failure.
// `hybrid` P1: the vector route really contributed scores to the
// same call that produced these hits.
//
// A build that reports `hybrid` while never passing `vectorScores` is EXACTLY the
// defect P1 exists for, so the assertion below (with a populated space) is the
// one that must never drift.
test('A3c: a healthy embedder over an EMPTY space reports "no_vector_space"', async () => {
const store = seededStore()
try {
let embedCalls = 0
@ -245,11 +253,11 @@ test('A3c: a healthy embedder reports "vector_ready_unused", NOT "hybrid"', asyn
assert.equal(embedCalls, 1, 'the embedder must actually be called')
assert.equal(
hits.degraded,
'vector_ready_unused',
'group A computes the vector but does not score with it — the label must say that, ' +
'because group V will need a different label once vectors really participate',
'no_vector_space',
'the fixture corpus carries no vectors in `fp-healthy`, so nothing can be ' +
'scored: the honest label says so instead of pretending a vector participated',
)
assert.notEqual(hits.degraded, 'hybrid', "calling it 'hybrid' would be a lie until group V lands")
assert.notEqual(hits.degraded, 'hybrid', "'hybrid' without a scored candidate would be a lie")
// And the result is still the pure-symbol ordering.
assert.equal(hits[0].text.includes('4324'), true)
} finally {
@ -257,18 +265,64 @@ test('A3c: a healthy embedder reports "vector_ready_unused", NOT "hybrid"', asyn
}
})
test('A3c mutation: labelling the success path "hybrid" must fail', () => {
// Mirrors the exact one-line edit that used to slip through: with no assertion
// on this value, reverting the rename was invisible.
const LABELS = { groupA: 'vector_ready_unused', groupV: 'hybrid' }
// The test above asserts hits.degraded === LABELS.groupA. Prove that swapping
// the constant would break it:
assert.notEqual(LABELS.groupA, LABELS.groupV)
const simulatedDeployed = LABELS.groupV // what the mutated lib returns
assert.throws(
() => assert.equal(simulatedDeployed, LABELS.groupA),
(err) => err.code === 'ERR_ASSERTION',
'if lib returns "hybrid", the A3c assertion must throw',
test('A3d: once the space holds vectors, the same recall reports "hybrid"', async () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
const fingerprint = 'fp-healthy'
store.putBlock({ text: '值班室分机号 4324', vector: [0.1, 0.2, 0.3], fingerprint })
const healthy = {
async embedText() { return [0.1, 0.2, 0.3] },
info: () => ({ fingerprint, dimension: 3 }),
}
const hits = await recallBlocksHybrid(store, healthy, '4324', { topK: 1 })
assert.equal(
hits.degraded,
'hybrid',
'a scored vector candidate is participating — reporting the group-A label here ' +
'would hide exactly the regression P1 pins',
)
assert.equal(hits.length, 1)
assert.ok(hits[0].vectorHit > 0, 'and the hit really carries a vector contribution')
} finally {
store.close()
}
})
test('A3c†: A3d really bites — drop the scoring pass and "hybrid" becomes unreachable', async (t) => {
// ★ A real mutation of lib/, not a re-assertion of local constants. The earlier
// version of this test compared two string literals it had just defined, which
// can never fail for a reason that matters (the same defect the mutation
// discipline note calls out: a guard that cannot fail is not a guard).
//
// The mutation empties the candidate scan. If A3d still reported `hybrid`, the
// label would be decoration rather than evidence — which is exactly the bug P1
// fixes one layer up.
await v5Mutant(
` const rows = store.vectorCandidates(fingerprint, dimension)`,
` const rows = [] // MUTATED: scoring pass produces no candidates`,
async (mod) => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
const fingerprint = 'fp-healthy'
store.putBlock({ text: '值班室分机号 4324', vector: [0.1, 0.2, 0.3], fingerprint })
const healthy = {
async embedText() { return [0.1, 0.2, 0.3] },
info: () => ({ fingerprint, dimension: 3 }),
}
const hits = await mod.recallBlocksHybrid(store, healthy, '4324', { topK: 1 })
assert.throws(
() => assert.equal(hits.degraded, 'hybrid'),
(err) => {
assert.equal(err.code, 'ERR_ASSERTION', 'only the criterion may bite')
t.diagnostic(`A3c† RAW RED >>> ${String(err.message).split('\n')[0]}`)
return true
},
'A3d must go red when nothing is scored',
)
} finally {
store.close()
}
},
)
})

12
test/phase-b-h.test.js
View File

@ -2913,13 +2913,15 @@ test('D3b: a clean recall overwrites last_recall with its own outcome', async ()
const result = await ctx.tools.execute({
callId: 'lr2', name: 'stablemem_stats', arguments: {}, signal: new AbortController().signal,
})
// ★ A healthy recall does NOT report 'ok': group A deliberately labels the
// success path `vector_ready_unused` because the vector is computed but not
// yet scored. Asserting 'ok' here was my mistake — the label that matters is
// "no failure", and `embed_error === null` is the honest way to say it.
// ★ A healthy WIRED recall reports `hybrid` (P1), not `ok` and no longer
// `vector_ready_unused`: the block carries a vector in the active space, so
// the fused route really contributed. The assertion that matters is "no
// failure", and `embed_error === null` is the honest way to say it; the mode
// value is asserted too so the label cannot silently drift back to the
// group-A stage name while the wiring is in place.
assert.notEqual(result.value.last_recall.mode, 'embed_error', 'no failure is recorded')
assert.equal(result.value.last_recall.embed_error, null, 'and no error reason either')
assert.equal(result.value.last_recall.mode, 'vector_ready_unused', 'the group-A success label')
assert.equal(result.value.last_recall.mode, 'hybrid', 'the wired success label')
} finally {
await done()
}

801
test/phase-b-p1-vector-wiring.test.js Normal file
View File

@ -0,0 +1,801 @@
/**
* P1 · Vector recall wiring — the seam that was computed and then thrown away.
*
* ## The defect this file exists for
*
* `lib/recall.js:681` ended with
*
* return withDiagnosis(recallBlocks(store, query, scoped), 'vector_ready_unused')
*
* `queryVector` had been computed, dimension-checked, and then deliberately
* dropped: `scoped` carried only `topK` / `signal` / `fingerprint` / `dimension`.
* So every V1–V7 scoring branch in `recallBlocks` was live code reachable ONLY
* from unit tests that hand-built a `vectorScores` Map. In production the recall
* path was pure symbol/FTS — the whole vector half of phase B was unreachable.
*
* Grep evidence at c1f7e06: `vectorScores:` appears 10 times in
* `test/phase-b-a.test.js` and **zero** times anywhere under `lib/`.
*
* ## Why this criterion is end-to-end and not a function test
*
* A test that calls `recallBlocks(store, q, { vectorScores, minScore })` proves
* the scoring seam works — which was never in doubt. It cannot see the bug,
* because the bug is that nobody ever passes those two options. The only
* observation that distinguishes the broken build from the fixed one is on the
* far side of the whole chain:
*
* real sqlite file → real `apply()`/`commit()` write path → real
* `recallBlocksHybrid` → real `readOnly` query embedding → `agent/pre-step`
* injection → `renderRecall` → the system prompt → the model's answer
*
* So: a real `dsh --profile headless` process, a real on-disk database, an HTTP
* embedding service the plugin reaches through its own `lib/embed/http.js`
* adapter, and a stub LLM whose answer is a pure function of the system prompt it
* received. The assertion is on the **final answer text**, which is as far from
* the library boundary as the harness allows.
*
* ## Two preconditions, asserted before the answer is
*
* "The model answered QP47-812" alone would be satisfied by a lucky symbol hit.
* The claim P1 needs is *"the vector route is what carried it"*, so the same turn
* also proves:
*
* P1a symbol path only → zero hits, and specifically zero hits on block A
* (the diagnostic `degraded` mode is printed as evidence);
* P1b the injected text is attributable to the vector route (a `text`-tagged
* hit carrying a vector score, not an `exact` pin).
*
* Without P1a/P1b a passing test would only show that *something* answered.
*
* ## The corpus, and why these exact strings
*
* The fake embedder is a deterministic keyword-family projection into 8 unit
* axes: family 0 is the target's vocabulary, families 1..7 are one axis per
* distractor sentence. A text matching no family maps to the zero vector, so it
* can never be a nearest neighbour by accident. This makes the "does the vector
* route reach block A" question decidable with no model, no network, and no
* tolerance — and it is what the corpus was **measured** against before any
* assertion was written (see the pre-check test below, which is the executable
* form of that measurement).
*
* The access code is `QP47-812`, not a bare number, so the stub LLM's answer
* extractor (`[A-Z]{2}[0-9]{2}-[0-9]{3}`) can find it and nothing else in the
* prompt can collide with it.
*
* ## Skip conditions
*
* The harness needs `dsh` and the two `dsh-base` peers resolvable from this
* package. When they are not installed there is no headless process to drive, and
* the test skips LOUDLY rather than asserting against a half-mounted app — the
* same rule `loop-system-e2e.test.js` follows.
*
* @module dsh-memory/test/phase-b-p1-vector-wiring
*/
import assert from 'node:assert/strict'
import { execFileSync, spawn } from 'node:child_process'
import { createReadStream, existsSync, mkdtempSync, rmSync } from 'node:fs'
import { createServer } from 'node:http'
import { createServer as createNetServer } from 'node:net'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { fileURLToPath } from 'node:url'
import test from 'node:test'
import { BlockStore } from '../lib/store.js'
import { recallBlocks, recallBlocksHybrid } from '../lib/recall.js'
// ── the corpus ────────────────────────────────────────────────────────────────
//
// One target ("A") and seven distractors. Every distractor owns exactly ONE
// embedding axis, and the target owns the other one, so a query can only reach A
// by landing on family 0.
/** Keyword families. Index 0 is the target's; 1..7 are one per distractor. */
const FAMILIES = [
['门禁码', 'QP47-812', '四位数', '暗号', '铁皮门', '货场', '进入凭证', '通行'],
['午饭', '食堂', '十二点半'],
['叉车', '额定载重', '两吨'],
['打印机', '缺纸', '红灯'],
['地铁', '园区北门'],
['例会', '会议室'],
['温度', '保护装置'],
['包裹', '收发室'],
]
const DIMENSION = FAMILIES.length
/** The one block the model must be able to answer with. */
const TARGET = '第七号仓库的备用门禁码是 QP47-812。'
/** The code that must appear in the final answer, and nowhere else by accident. */
const ANSWER = 'QP47-812'
const DISTRACTORS = [
'午饭时间改到十二点半，食堂在东楼二层。',
'叉车的额定载重是两吨，超了会翻。',
'打印机缺纸时会闪红灯，备件在储物间第三格。',
'地铁二号线直达园区北门，出站走五分钟。',
'每月的第二个周五开例会，会议室在行政楼。',
'温度超过八十五度，保护装置会切断电源。',
'包裹统一送去一楼收发室的货架，凭手机尾号取。',
]
/**
* The question the user asks.
*
* Measured against the corpus before being pinned here: it extracts no symbol
* (`querySymbols` → []), shares no CJK 2-gram with TARGET or any distractor, and
* FTS finds nothing for it. It reaches family 0 only through `四位数`.
*/
const QUERY = '上次说的那个四位数是什么来着？'
/** The query's family axis. Asserted so a corpus edit cannot silently break it. */
const QUERY_FAMILY = 0
/**
* Deterministic keyword-family embedding.
*
* Returns a one-hot unit vector on the first matching family, or the zero vector
* when nothing matches. Zero is deliberate: an unmapped text must not be a
* neighbour of anything, so a false "vector hit" cannot be an artefact of a
* default direction.
*
* @param {string} text
* @returns {number[]}
*/
function embedByFamily(text) {
const haystack = String(text ?? '')
const vector = new Array(DIMENSION).fill(0)
for (let index = 0; index < FAMILIES.length; index += 1) {
if (FAMILIES[index].some((keyword) => haystack.includes(keyword))) {
vector[index] = 1
return vector
}
}
return vector
}
/** Cosine similarity, written out so the fixture cannot depend on lib scoring. */
function cosine(a, b) {
let dot = 0
let normA = 0
let normB = 0
for (let index = 0; index < a.length; index += 1) {
dot += a[index] * b[index]
normA += a[index] * a[index]
normB += b[index] * b[index]
}
if (normA === 0 || normB === 0) return 0
return dot / (Math.sqrt(normA) * Math.sqrt(normB))
}
/** CJK 2-grams of a string, used to prove "no lexical overlap" mechanically. */
function bigrams(text) {
const chars = [...String(text ?? '')].filter((char) => /[\u3400-\u4dbf\u4e00-\u9fff]/.test(char))
const out = new Set()
for (let index = 0; index + 1 < chars.length; index += 1) out.add(chars[index] + chars[index + 1])
return out
}
// ── the fake embedding service ────────────────────────────────────────────────
/**
* An HTTP embedding endpoint speaking the plugin's own wire protocol.
*
* This is deliberately a real socket, not an injected `config.embedder`: the
* point of P1 is that the *production* path works, and the production path goes
* through `createHttpEmbedder` in `lib/embed/http.js`. Injecting an embedder
* object would leave that adapter — and the JSON shape it accepts — untested.
*
* @returns {Promise<{url: string, close: () => Promise<void>}>}
*/
async function startEmbedServer() {
const server = createServer((request, response) => {
const chunks = []
request.on('data', (chunk) => chunks.push(chunk))
request.on('end', () => {
let text = ''
try {
text = JSON.parse(Buffer.concat(chunks).toString('utf8')).text ?? ''
} catch {
response.writeHead(400).end('bad json')
return
}
const body = JSON.stringify({ embedding: embedByFamily(text) })
response.writeHead(200, { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(body) })
response.end(body)
})
})
await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve))
const { port } = server.address()
return {
url: `http://127.0.0.1:${port}/embed`,
close: () => new Promise((resolve) => server.close(resolve)),
}
}
// ── the stub LLM ──────────────────────────────────────────────────────────────
/**
* An OpenAI-compatible endpoint whose answer is a pure function of the SYSTEM
* prompt it receives.
*
* ★ Why a stub rather than a real provider: the assertion is "did block A reach
* the system prompt". A real model answers probabilistically, so a red run could
* mean "the model was unlucky" and a green run could mean "the model guessed the
* code". This stub makes the measurement exact: if the code is in the system
* prompt the answer IS the code; if it is not, the answer is 未知.
*
* It is still a real HTTP provider from dsh's point of view — the plugin, the
* harness, the request builder, and `agent/pre-step` are all the production ones.
*
* @returns {Promise<{url: string, close: () => Promise<void>}>}
*/
async function startStubLlm() {
const answerFor = (messages) => {
const system = messages
.filter((message) => message?.role === 'system')
.map((message) => (typeof message.content === 'string'
? message.content
: (message.content ?? []).map((block) => (typeof block?.text === 'string' ? block.text : '')).join('')))
.join('\n')
const found = system.match(/[A-Z]{2}\d{2}-\d{3}/)
return found === null ? '未知' : found[0]
}
const server = createServer((request, response) => {
if (request.method === 'GET') {
const body = JSON.stringify({ object: 'list', data: [{ id: 'stub-1', object: 'model' }] })
response.writeHead(200, { 'Content-Type': 'application/json' }).end(body)
return
}
const chunks = []
request.on('data', (chunk) => chunks.push(chunk))
request.on('end', () => {
let payload = {}
try {
payload = JSON.parse(Buffer.concat(chunks).toString('utf8'))
} catch {
response.writeHead(400).end('bad json')
return
}
const answer = answerFor(payload.messages ?? [])
const base = { id: 'chatcmpl-stub', created: 0, model: payload.model ?? 'stub-1' }
if (payload.stream === true) {
response.writeHead(200, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
Connection: 'close',
})
const send = (chunk) => response.write(`data: ${JSON.stringify(chunk)}\n\n`)
send({ ...base, object: 'chat.completion.chunk', choices: [{ index: 0, delta: { role: 'assistant' }, finish_reason: null }] })
send({ ...base, object: 'chat.completion.chunk', choices: [{ index: 0, delta: { content: answer }, finish_reason: null }] })
send({ ...base, object: 'chat.completion.chunk', choices: [{ index: 0, delta: {}, finish_reason: 'stop' }] })
response.end('data: [DONE]\n\n')
return
}
response.writeHead(200, { 'Content-Type': 'application/json' }).end(JSON.stringify({
...base,
object: 'chat.completion',
choices: [{ index: 0, message: { role: 'assistant', content: answer }, finish_reason: 'stop' }],
usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 },
}))
})
})
await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve))
const { port } = server.address()
return {
url: `http://127.0.0.1:${port}/v1`,
close: () => new Promise((resolve) => server.close(resolve)),
}
}
// ── the headless harness ──────────────────────────────────────────────────────
/** True when a `dsh` binary is on PATH and resolves. */
function dshAvailable() {
try {
execFileSync('dsh', ['--version'], { stdio: 'ignore' })
return true
} catch {
return false
}
}
/**
* A private DSH_HOME with the stock headless profile's bundles.
*
* Written to a temp dir instead of reusing `$DSH_HOME` so the run cannot inherit
* whatever the developer has configured (a different embedder, a different memory
* database, a debug toggle). The only inputs are the ones this file supplies.
*
* @param {string} home
* @param {string} packageRoot
*/
function writeProfile(home, packageRoot) {
const profileDir = join(home, 'profiles', 'headless')
execFileSync('mkdir', ['-p', join(profileDir, 'node_modules')])
execFileSync('sh', ['-c', `ln -sfn ${JSON.stringify(packageRoot)} ${JSON.stringify(join(profileDir, 'node_modules', 'dsh-memory'))}`])
// An empty tree: the bundles in package.json supply the base rows, and the
// overlay supplies `memory`. Nothing from the developer's own profile leaks in.
execFileSync('sh', ['-c', `printf '[]\\n' > ${JSON.stringify(join(profileDir, 'cordis.yml'))}`])
execFileSync('sh', ['-c', `cat > ${JSON.stringify(join(profileDir, 'package.json'))} <<'P1PKG'
${JSON.stringify({
name: 'dsh-profile-headless',
private: true,
dependencies: { 'dsh-memory': `link:${packageRoot}` },
dsh: { profile: { bundles: ['@deepseek-ai/dsh-base', '@deepseek-ai/dsh-headless'] } },
}, null, 2)}
P1PKG`])
return profileDir
}
/**
* Run one headless task and return the final answer text plus stderr.
*
* @param {object} options
* @param {string} options.home private DSH_HOME
* @param {string} options.profileDir
* @param {string} options.dbPath
* @param {string} options.embedUrl
* @param {string} options.llmUrl
* @param {string} options.query
* @param {string} options.cwd
* @returns {Promise<{answer: string, stderr: string, sessionId: string|null}>}
*/
function runHeadless({ home, profileDir, dbPath, embedUrl, llmUrl, query, cwd }) {
const overlay = join(profileDir, 'p1-overlay.yml')
// ★ `insert:` with an explicit `name:` and NOT a bare `- id: memory` merge.
// The private DSH_HOME written by {@link writeProfile} has no `memory` row of
// its own, so a merge-style patch would silently compose an entry with no
// plugin behind it: the config would appear in `--dump-config`, the harness
// would boot, and the recall hook would simply never be registered. That is a
// no-op that looks configured — the same shape as the bug this file pins.
// The two other rows are base entries and are legitimately merged by id.
const yaml = [
'- insert:',
' - id: memory',
' name: dsh-memory',
' config:',
` path: ${JSON.stringify(dbPath)}`,
' topK: 3',
' debugLog: true',
' inject: true',
' embed:',
` endpoint: ${JSON.stringify(embedUrl)}`,
` dimension: ${DIMENSION}`,
' model: p1-keyword-family',
'- id: agent-default-model',
' config:',
' provider: stub',
' model: stub-1',
'- id: llm-pi-ai',
' config:',
' providers:',
' stub:',
' displayName: stub',
' api: openai-completions',
` baseURL: ${JSON.stringify(llmUrl)}`,
' apiKeyEnv: P1_STUB_KEY',
' models:',
' - id: stub-1',
' name: stub-1',
' contextWindow: 65536',
// ── P0: the replacement loop must be mounted ────────────────────────────
//
// Without these two rows the run exercises the STOCK agent loop, which leaves
// the recall in user-message history. The recall would then never reach the
// system prompt, and P1's "the answer came from a system section" claim would
// be unprovable — worse, the assertion would pass for the wrong reason if the
// stub LLM were to look at history too (it deliberately does not, which is
// what makes the failure visible). This is byte-for-byte the shipped
// `cordis.patch.yml` overlay, so the run composes the same tree the profile
// does. It also pins the P0 result: 377e02c + c1f7e06 must survive P1.
'- id: agent-loop',
' disabled: true',
'- insert:',
' - id: hologram-agent-loop',
' name: dsh-memory/loop',
'',
].join('\n')
execFileSync('sh', ['-c', `cat > ${JSON.stringify(overlay)} <<'P1YAML'\n${yaml}P1YAML`])
return new Promise((resolve, reject) => {
const child = spawn(
'dsh',
['--profile', 'headless', '--patch', overlay, '--json', query],
{ cwd, env: { ...process.env, DSH_HOME: home, P1_STUB_KEY: 'p1-stub-key' } },
)
let stdout = ''
let stderr = ''
child.stdout.on('data', (chunk) => { stdout += chunk })
child.stderr.on('data', (chunk) => { stderr += chunk })
child.on('error', reject)
child.on('close', (code) => {
const events = stdout
.split('\n')
.filter((line) => line.trim() !== '')
.map((line) => {
try {
return JSON.parse(line)
} catch {
return null
}
})
.filter((event) => event !== null)
const final = events.findLast((event) => event.type === 'final')
const session = events.find((event) => event.type === 'session')
if (final === undefined) {
reject(new Error(`headless run produced no final event (exit ${code})\nstderr:\n${stderr}\nstdout:\n${stdout}`))
return
}
resolve({ answer: final.text ?? '', stderr, sessionId: session?.sessionId ?? null })
})
})
}
/**
* Decompress a session log and return its JSONL lines.
*
* `session.v4.jsonl.zstd` is what the harness actually persists, so it is what
* an operator would read. Parsing it (rather than trusting the run's own stdout)
* is the only way to see the injected prompt the model was really given.
*
* @param {string} home
* @param {string} sessionId
* @returns {string[]}
*/
function readSessionLines(home, sessionId) {
// The workspace key is a lossy slug of the launch cwd, so it cannot be
// reconstructed reliably (a real directory called `a-b` and one called `a/b`
// collide). Search for the session directory by its id instead — the id is
// exact and unique.
const sessionsRoot = join(home, 'sessions')
const candidates = existsSync(sessionsRoot)
? execFileSync('sh', ['-c', `find ${JSON.stringify(sessionsRoot)} -maxdepth 2 -type d -name ${JSON.stringify(sessionId)}`])
.toString('utf8')
.split('\n')
.filter((line) => line.trim() !== '')
: []
assert.equal(
candidates.length,
1,
`exactly one session directory must exist for ${sessionId} under ${sessionsRoot}; found ${JSON.stringify(candidates)}`,
)
const file = join(candidates[0], 'session.v4.jsonl.zstd')
assert.ok(existsSync(file), `session log must exist at ${file}`)
return execFileSync('zstd', ['-dc', file], { maxBuffer: 256 * 1024 * 1024 })
.toString('utf8')
.split('\n')
.filter((line) => line.trim() !== '')
}
/** Every system-prompt body in a session log, joined. */
function systemPromptFrom(lines) {
return lines
.map((line) => {
try {
return JSON.parse(line)
} catch {
return null
}
})
.filter((event) => event?.type === 'system/message')
.map((event) => (event.data?.message?.content ?? [])
.map((block) => (typeof block?.text === 'string' ? block.text : ''))
.join(''))
.join('\n')
}
// ── module setup shared by every case ─────────────────────────────────────────
const PACKAGE_ROOT = fileURLToPath(new URL('..', import.meta.url))
/** A store seeded through `putBlock` with family vectors, for the library-level checks. */
function seededStore() {
const store = new BlockStore({ path: ':memory:' }).open()
const fingerprint = 'p1-keyword-family'
const targetId = store.putBlock({ text: TARGET, source: 'p1', vector: embedByFamily(TARGET), fingerprint }).id
const distractorIds = DISTRACTORS.map((text) => store.putBlock({
text, source: 'p1', vector: embedByFamily(text), fingerprint,
}).id)
return { store, fingerprint, targetId, distractorIds }
}
// ══════════════════════════════════════════════════════════════════════════════
// P1-0 · corpus pre-check — the executable form of "measured before asserted"
// ══════════════════════════════════════════════════════════════════════════════
/**
* The pre-check that must pass BEFORE the criterion is allowed to assert
* anything. It fails on a corpus problem, never on the wiring bug, and it is
* kept in the suite so a later corpus edit cannot quietly make the e2e
* criterion unfalsifiable.
*
* Two conditions, both mandated:
* 1. the symbol path returns NO hit for the target (so a green answer can
* only be the vector route);
* 2. with a hand-built `vectorScores` Map plus `minScore: 0`, the target IS
* top-1 (so the corpus and vectors can actually reach it at all).
*/
test('P1-0: the corpus is admissible — symbol path blind, vector path reaches the target', () => {
const { store, targetId } = seededStore()
try {
// ── the query really does extract no symbol and share no 2-gram ──────────
const queryVector = embedByFamily(QUERY)
assert.equal(
queryVector.indexOf(1),
QUERY_FAMILY,
`the query must land on family ${QUERY_FAMILY}; got ${JSON.stringify(queryVector)}`,
)
const queryGrams = bigrams(QUERY)
for (const text of [TARGET, ...DISTRACTORS]) {
const shared = [...bigrams(text)].filter((gram) => queryGrams.has(gram))
assert.deepEqual(
shared,
[],
`precondition: the query must share no CJK 2-gram with "${text}"; shared ${JSON.stringify(shared)}`,
)
}
// ── condition 1: the symbol path is blind to the target ─────────────────
const symbolOnly = recallBlocks(store, QUERY, { topK: 5 })
assert.equal(
symbolOnly.filter((hit) => hit.id === targetId).length,
0,
`precondition 1: the symbol path must NOT find the target; it returned ${JSON.stringify(
symbolOnly.map((hit) => [hit.text, hit.score]),
)}`,
)
// ── condition 2: a manually built vector route DOES reach the target ────
const scores = new Map(store.allBlocks(-1).map((block) => [
block.id,
cosine(queryVector, JSON.parse(block.vector)),
]))
const fused = recallBlocks(store, QUERY, { topK: 3, vectorScores: scores, minScore: 0 })
assert.ok(fused.length > 0, 'precondition 2: the fused route must return something')
assert.equal(
fused[0].id,
targetId,
`precondition 2: the target must be top-1 once vectors participate; got ${
JSON.stringify(fused.map((hit) => [hit.text, hit.score]))
}`,
)
} finally {
store.close()
}
})
// ══════════════════════════════════════════════════════════════════════════════
// P1 · the end-to-end criterion
// ══════════════════════════════════════════════════════════════════════════════
test('P1: a semantic-only question is answered from block memory via the vector route', async (t) => {
if (!dshAvailable()) {
t.skip('no `dsh` binary on PATH — the headless harness cannot be driven here')
return
}
const workspace = mkdtempSync(join(tmpdir(), 'p1-e2e-'))
const home = join(workspace, 'home')
const cwd = join(workspace, 'work')
execFileSync('mkdir', ['-p', cwd])
const profileDir = writeProfile(home, PACKAGE_ROOT)
const dbPath = join(workspace, 'memory.db')
const embedServer = await startEmbedServer()
const llmServer = await startStubLlm()
try {
// ── seed through the real plugin write path ──────────────────────────────
//
// `apply()` builds the plugin exactly as the profile does, `commit()` runs
// the production embed → `putBlock(vector, fingerprint)` path. Nothing here
// writes a vector by hand: a block that reached the database without the
// real write path could not prove the read path is wired.
const { Context } = await import('@deepseek-ai/cordis')
const { SystemPrompt } = await import('@deepseek-ai/dsh-system-prompt')
const { ToolRuntime } = await import('@deepseek-ai/dsh-tools')
const { apply: applyMemory } = await import('../lib/index.js')
const seedCtx = new Context()
let space
try {
new SystemPrompt(seedCtx, {})
new ToolRuntime(seedCtx)
await new Promise((resolve) => setTimeout(resolve, 0))
const mem = applyMemory(seedCtx, {
path: dbPath,
topK: 3,
debugLog: false,
inject: true,
embed: { endpoint: embedServer.url, dimension: DIMENSION, model: 'p1-keyword-family' },
})
await new Promise((resolve) => setTimeout(resolve, 0))
space = mem.space()
assert.ok(space !== null && space.dimension === DIMENSION, `the plugin must expose the fake space; got ${JSON.stringify(space)}`)
for (const text of [TARGET, ...DISTRACTORS]) {
const written = await mem.commit({ text, source: 'p1-seed' })
assert.equal(
written.embedded,
true,
`seed block must be stored WITH a vector (real embed path), not symbol-only: ${text}`,
)
}
assert.equal(mem.store.countBlocks(), 1 + DISTRACTORS.length, 'every corpus block must be on disk')
} finally {
await seedCtx.fiber.dispose()
}
// ── P1a · the symbol path is blind, proved on the real on-disk database ──
const store = new BlockStore({ path: dbPath }).open()
let symbolHits
try {
symbolHits = recallBlocks(store, QUERY, { topK: 5, fingerprint: space.fingerprint, dimension: space.dimension })
} finally {
store.close()
}
assert.equal(
symbolHits.length,
0,
`P1a precondition: the symbol path must find NOTHING for this query; got ${JSON.stringify(
symbolHits.map((hit) => [hit.text, hit.score]),
)}`,
)
// The current build's own diagnostic, captured before the run: on c1f7e06 this
// is `vector_ready_unused` and the array is empty.
const hybridStore = new BlockStore({ path: dbPath }).open()
let hybridDiagnosis
try {
const probe = await import('../lib/recall.js')
const hybrid = await probe.recallBlocksHybrid(
hybridStore,
{
embedText: async (text) => embedByFamily(text),
info: () => space,
},
QUERY,
{ topK: 3 },
)
hybridDiagnosis = { mode: hybrid.degraded ?? 'ok', hits: hybrid.length }
} finally {
hybridStore.close()
}
// ── the real headless run ────────────────────────────────────────────────
const run = await runHeadless({
home, profileDir, dbPath, embedUrl: embedServer.url, llmUrl: llmServer.url, query: QUERY, cwd,
})
// ── the injected text, recovered from the persisted session log ──────────
//
// Everything below is derived from the harness's own on-disk record, not from
// the run's stdout: the session log carries the exact system prompt the
// adapter was handed, which is the only place the injection is observable.
assert.ok(
run.sessionId !== null,
`the run must report its session id; stderr was:\n${run.stderr}`,
)
const lines = readSessionLines(home, run.sessionId)
const systemPrompt = systemPromptFrom(lines)
// The recall SECTION, not just its header line: `renderRecall` emits a header
// and then one line per hit, so the entry lines (which carry the `text` tag and
// the score) live BELOW the header. Taking the section rather than the first
// matching line is what lets P1c assert on the hit's shape.
const systemLines = systemPrompt.split('\n')
const recallStart = systemLines.findIndex((line) => line.includes('Recalled from durable block memory'))
const recallLine = recallStart === -1 ? undefined : systemLines.slice(recallStart).join('\n')
t.diagnostic(`P1 symbol-path baseline: hits=${symbolHits.length}`)
t.diagnostic(`P1 recall diagnosis before the run: degraded=${hybridDiagnosis.mode} hits=${hybridDiagnosis.hits}`)
t.diagnostic(`P1 recall line injected: ${recallLine ?? '(none)'}`)
// ── P1 · the answer ──────────────────────────────────────────────────────
//
// The primary assertion. On c1f7e06 the recall array is empty (`hits=0`), so
// the stub LLM — which answers ONLY with a code it finds in the system prompt
// — replies 未知. That failure IS the defect: the vector was computed and
// thrown away.
assert.equal(
run.answer.trim(),
ANSWER,
`P1: the answer must come from block memory through the vector route.\n` +
` query: ${QUERY}\n` +
` expected: ${ANSWER}\n` +
` actual: ${JSON.stringify(run.answer)}\n` +
` symbol-path hits: ${symbolHits.length} (must be 0)\n` +
` recall diagnosis: degraded=${hybridDiagnosis.mode} hits=${hybridDiagnosis.hits}\n` +
` recall section injected: ${recallLine ?? '(none)'}`,
)
// ── P1b · the recall reached the SYSTEM prompt, not user history (P0) ────
//
// P0 (377e02c + c1f7e06) must still hold after the wiring change: the recall
// is a system *section*, so it may appear in no non-system message body.
assert.ok(
recallLine !== undefined,
`P1b: the recall section must reach the system prompt (P0 must still hold). ` +
`degraded=${hybridDiagnosis.mode} symbolHits=${symbolHits.length}\n` +
`stderr:\n${run.stderr}`,
)
const leakingBodies = lines
.map((line) => {
try {
return JSON.parse(line)
} catch {
return null
}
})
.filter((event) => event?.type === 'user/message' || event?.type === 'assistant/message')
.map((event) => JSON.stringify(event.data ?? {}))
.filter((body) => body.includes(TARGET))
const recallCarrierMessages = (() => {
// Count carriers by role across every message the harness recorded.
const carriers = []
for (const line of lines) {
let event
try {
event = JSON.parse(line)
} catch {
continue
}
const message = event?.data?.message ?? event?.data
const text = (message?.content ?? [])
.map((block) => (typeof block?.text === 'string' ? block.text : ''))
.join('')
if (typeof message?.role === 'string' && text.includes(TARGET)) carriers.push(message.role)
}
return carriers
})()
t.diagnostic(`P1b recall carriers by role: ${JSON.stringify(recallCarrierMessages)}`)
t.diagnostic(`P1b non-system bodies containing the target: ${JSON.stringify(leakingBodies)}`)
assert.deepEqual(
leakingBodies,
[],
'P1b: the recalled note must NOT appear in any user/assistant message body (a move, not a copy)',
)
// Positive half of the same claim: exactly ONE message carries it, and that
// message is the system prompt. Without this, a build that dropped the recall
// from the prompt entirely could still satisfy the negative assertion above —
// and P0 is precisely the property that must survive P1.
assert.deepEqual(
recallCarrierMessages,
['system'],
`P1b: the recalled note must be carried by exactly one message, the system prompt; got ${JSON.stringify(
recallCarrierMessages,
)}`,
)
// ── P1c · the hit is a VECTOR hit, not an exact symbol pin ──────────────
//
// `renderRecall` tags a hit `(exact 1.0x)` when a numeric symbol pinned it and
// `(text 0.xx)` otherwise. Our corpus guarantees no symbol match, so a `text`
// tag with a sub-1.0 score is the observable signature of the fused vector
// route (`vec*0.7 + sym*0.3`). If the wiring regressed to a symbol-only path
// this tag would read `exact`, and if vectors stopped participating entirely
// there would be no line at all (caught by P1 above).
assert.match(
recallLine,
/\[\d+\] \(text 0\.\d{2}\) .*QP47-812/,
`P1c: the injected entry must be a \`text\`-tagged hit (vector route) carrying the target, ` +
`not an \`exact\` symbol pin. recall line was: ${recallLine}\n` +
`diagnosis at probe time: degraded=${hybridDiagnosis.mode} hits=${hybridDiagnosis.hits}`,
)
} finally {
await embedServer.close()
await llmServer.close()
// ★ `P1_KEEP=1` keeps the workspace so an operator (or a review) can read the
// session log, the sqlite file and the composed overlay by hand. Off by
// default: a test that leaves databases behind on every run turns /tmp into a
// second, unaudited store of memory. The directory is printed either way, so a
// preserved run is discoverable from the test output alone.
if (process.env.P1_KEEP === '1') {
t.diagnostic(`P1 workspace kept: ${workspace}`)
console.log(`P1 workspace kept: ${workspace}`)
} else {
rmSync(workspace, { recursive: true, force: true })
}
}
})