41 Commits

Author SHA1 Message Date
f3ee2d1cb0 test(memory): pin that the WRITE path marks stale, not the test (C4 follow-up)
C4j asserted that a stale-marked centroid falls back to uncorrected vectors — but it
called `store.markCentroidStale(fp)` **itself** to set up the state. That proves the
recall side honours the flag; it says nothing about whether production ever sets it.

Measured: deleting the one line in `#ensureCentroid` that persists the verdict

    if (status?.stale === true) this.store.markCentroidStale(fingerprint)

left **all 306 criteria green**. The wiring had no guard at all.

This is the fifth instance of one shape in this branch:

    W8   saveSpaceCentroid defined, zero production callers
    C4d  centroid built by the test's own rebuildCentroid call
    P1   vectorScores hand-built in the fixture
    D2   counters computed, never printed
    C4j  stale flag set by the criterion, not by the write path

A hand-fed input cannot prove a production path. So C4k drives only the production
entry points — `apply()` + `commit()` — and marks nothing itself:

    after a settled rebuild   stats.stale === undefined
    after 3 more commits      stats.stale === 1     (drift 3/8 = 37.5% > 20%)

Mutation — deleting that line in `#ensureCentroid`:

    not ok 12 - C4k: the write path itself marks a drifted centroid stale
    error: C4k: the WRITE path must persist the stale verdict ... undefined !== 1

## Why the drift is generated rather than forced

3 commits on top of a settled 8 puts the delta at 37.5%, past the 20% ratio — the
same path an ordinary growing space takes. Forcing `markCentroidStale` directly is
exactly what C4j does and exactly what this criterion exists to complement.

Gates: npm test 307/307 (0 skipped), npm run check 12/12, frozen faces untouched.
2026-11-05 12:39:06 +08:00
7652184832 fix(memory): honour the C4 order — a stale round recalls with UNCORRECTED vectors
The C4 slice instruction said, verbatim: "判 stale 的那一轮仍返回结果(用未校正向量)".
It was not implemented. `grep stale lib/recall.js` returned one comment and no code,
and `loadCentroid` carries no staleness verdict — so recall centred with whatever
mean it found, however old.

## Why not simply ask on the recall side

The obvious fix — call `centroidStatus` in `recallBlocks` — was built and measured:

    # tests 305  # pass 304  # fail 1
    not ok 97 - C3-k: the recall hot path must not run a COUNT query
      error: 'lib/recall.js must not call centroidStatus — it runs countVectored
             internally, and the recall path may only read the centroid'

C3-k is a rule this project set deliberately. A fix that breaks it is not a fix,
and the criterion caught the attempt — which is what it is for.

## What this does instead

The WRITE path already pays for the judgement on every commit (`#ensureCentroid`
calls `centroidStatus`). So it persists the verdict:

  · `BlockStore#markCentroidStale(fp)` — `json_set(stats, '$.stale', 1)` when the
    space is judged stale
  · `#ensureCentroid` marks on `status.stale === true`
  · `recallBlocks` honours it: `!centroid.stats?.stale` before centring
  · cleared implicitly — `rebuildCentroid` → `saveCentroid` upserts stats without
    the key

Recall never runs a COUNT; C3-k stays green.

## The bug this shipped with for one revision — and why it matters

The guard was first written `centroid.stats?.stale !== true`. The flag is stored as
the JSON number `1`, and in JS **`1 !== true` is true** — a strict comparison across
types — so the guard was a permanent no-op. Stale centroids kept being used.

**The entire suite was 305/305 green with that bug in place.** Nothing covered the
path. It was only visible on a fixture where centring is observable (the C4
hub/target space):

    fresh centroid    top1=target  0.9000 / 0.2129 / 0.1814
    marked stale      top1=hub     0.9928 / 0.9859 / 0.9803   <- raw cosine, uncorrected
    after rebuild     top1=target  0.9000 / 0.2129 / 0.1814

Hence C4j (below) rather than a comment claiming it works.

## Criterion C4j

Three assertions on that fixture: fresh → target; marked stale → hub AND the score
is the raw cosine (0.9928); rebuild → target again. Mutation — deleting the guard:

    not ok 11 - C4j: a centroid marked stale falls back to uncorrected vectors
    error: ... If this still says "target", the flag is being ignored — the exact bug
           that a strict `!== true` comparison produced while the whole suite stayed green.
           'target' !== 'hub'

## What this does NOT establish

That falling back improves ranking quality. An attempt to measure it could not
separate the cases: with a near-duplicate late corpus, scores saturate near 0.99
whether the mean is stale (0.997) or absent (0.994); only a freshly rebuilt mean
spread them (0.978..0.970). So this is a safety choice — a mean built from a
fraction of the space describes a space that no longer exists, and applying it is
unjustifiable on principle. The measurable win belongs to keeping the centroid
FRESH (the throttle in C3), not to skipping it.

Gates: npm test 306/306 (0 skipped), npm run check 12/12, frozen faces untouched.
2026-11-05 12:34:29 +08:00
59e6a71f28 perf(memory): stop re-reading 79 MB of vectors to answer a one-bit question
`recallBlocks` unions symbol matches with vector candidates, and to do that it walks
the whole table. It used `allBlocks(-1)` — `SELECT *` — while the only thing it
needs from the vector column is whether one EXISTS.

Measured at 8000 blocks x 512 dims, per recall turn:

    allBlocks(-1)  SELECT *          242.8 ms
    this read, narrow                 22.2 ms
    ------------------------------------------
    whole recallBlocksHybrid   624.0 -> 409.6 ms   (-34%)

At 2000 blocks: 157.6 -> 106.0 ms (-33%). The saving is not marginal and it is on
the per-turn pre-step path, which runs on every message.

## What changed

`BlockStore#symbolCandidates()` returns the six columns the pool actually reads
(id, text_content, source, fingerprint, vector_dim) plus `(vector != '') AS
has_vector`. `recallBlocks` iterates that instead of `allBlocks(-1)`, and the pool
entry takes `hasVector: block.has_vector === 1`.

`allBlocks` itself is untouched — the bounded symbol-only path still uses it with
`poolSize`, and tests use it as the wide reference.

## Why this needed a criterion rather than a green suite

A narrower read is a behaviour change waiting to happen, and the suite would not
notice most of them: 304 criteria were already green before it. So the criterion
asserts the equivalence directly:

  · same row set in the same order (the pool is a Map; insertion order is iterated)
  · `has_vector` matches the predicate it replaced — `String(vector ?? '') !== ''` —
    block by block, on a corpus containing a real vector, no vector, and an empty
    payload

Mutation — `1 AS has_vector`:

    not ok 7 - narrow read: symbolCandidates() sees exactly what allBlocks(-1) saw
    error: has_vector must reproduce the predicate it replaced, for block blk_01db9be26e52
           1 !== 0

## One divergence deliberately left alone

The literal string `'null'` counts as "has a vector" under the old predicate, while
`vectorCandidates` excludes it. That inconsistency predates this commit and is
documented rather than silently fixed: changing it is a separate behaviour change
that needs its own criterion, not a footnote in a performance commit.

Gates: npm test 305/305 (0 skipped), npm run check 12/12, frozen faces untouched.
2026-11-05 12:20:52 +08:00
73a0a6b8d0 test(memory): pin D1 — a corrupt vector cannot take the recall path down (D1)
D1 was the one item on the coverage list with genuinely no criterion. Measured by
mutation, not by grepping test names:

    corrupt vector handling: skip -> throw
    phase-b-h 137/137 green, phase-b-a 38/38 green, recall-accounting 5/5 green
    -> 145 criteria, zero red

That is decision §0.4 unguarded: "the vector side failing must not take the main
path down". One bad row would cost the whole turn its memory, and nothing noticed.

## The same probe found three items that ARE covered

Recorded because the parent's coverage matrix derived "zero hits" from grepping
criteria IDs against test names, and three of those were false negatives — the
criteria exist, they are just named after their subject rather than their ID:

    C6  centroid partitioned by fingerprint   loadCentroid ignoring it -> 3 red
                                              (ticket②, ticket②-strong, W8)
    W1  exec.signal reaches embedText         dropping it -> 2 red (W5, W9c)
    F1  fingerprint from provider info()      deriving locally -> 5 red

So the matrix under-reports as well as over-reports. Only mutation distinguishes
"labelled" from "enforced".

## What the criterion does

Writes a corrupt payload straight into the row and requires the recall to survive:

    store.raw().prepare('UPDATE memory_blocks SET vector = ? WHERE id = ?')
      .run('["a","b","c","d"]', broken.id)   // right length, non-numeric elements

Two guards narrowed what "corrupt" can mean, both measured and documented in the
criterion:

  · DDL enforces `json_valid(vector)` — a syntactically broken payload is refused
    outright (`ERR_SQLITE_ERROR: malformed JSON`), so it cannot reach the read path;
  · a trigger re-derives `vector_dim`, so a wrong-LENGTH payload is filtered in SQL
    before scoring sees it.

An array of the right length with non-numeric elements clears both and still cannot
be scored — which is exactly what a provider that changed its encoding, or a
migration that stringified values, would leave behind.

Assertions: no throw, the healthy neighbour is still recalled, and the bad row is
counted in `skippedUnscoreable` (D1's other half is D2's rule: skipping silently is
the same defect).

Mutation — a non-numeric component throws instead of skipping:

    not ok 6 - D1: a corrupt vector payload cannot take the recall path down
    error: D1: a single corrupt vector payload threw out of the recall path
           (M-D1: non-numeric vector component). Decision §0.4: ...

Gates: npm test 304/304 (0 skipped), npm run check 12/12.
2026-11-05 12:13:17 +08:00
5cfc982f39 fix(memory): count blocks dropped BEFORE the candidate pool exists (D2, second hole)
07f8438 gave the three gates a counter and claimed "every drop is countable". A
probe against the invariant found a fourth drop site, one stage earlier than any
gate:

    scoreVectorCandidates skips a row it cannot score
      · unparseable JSON
      · wrong width            (CENTER width vs query)
      · non-finite component
      · zero norm
      · centring collapsed it onto the space mean   (C4's dust guard)

Measured on three identical blocks with centring active:

    hits=0 skipped=0 skippedDim=0 skippedScore=0 skippedNoEvidence=0 truncated=0
    accounted = 0 of 3   ->  INVARIANT BROKEN

The whole space vanished while every counter read zero. Cause: a skipped row never
enters `vectorScores`, so `recallBlocks` never puts it in the candidate pool
(`if (!symbolMatch && !vectorSource) continue`), so no downstream counter can see
it. This is exactly the silent branch D2 forbids — deeper than where D2 first
looked, and I only found it because I ran the invariant against a centring case
instead of trusting the green suite.

## Fix

`skippedUnscoreable` threaded from `scoreVectorCandidates` (by reference, so the
early `no_vector_space` return can report it too) through `recallBlocks` into the
result, then into `last_recall` and the log line — the same three surfaces D2
already uses.

    hits=0 skippedUnscoreable=3 ...  accounted = 3 of 3  ->  INVARIANT HOLDS

## Criterion

`test/recall-accounting.test.js` gains case 5: three identical blocks, centring
engaged, asserting the count is 3 (not 0) and that hits + all seven counters
still equals the space.

The existing invariant case now sums seven counters instead of six — a shortfall
is how this hole would reappear.

Mutation — deleting the dust increment:

    not ok 5 - D2: a vector that centring collapses is counted, not silently lost
    error: D2: these blocks were dropped before the candidate pool existed, so no
           other counter can see them. ...

## Note on the label

These blocks are NOT counted as `skippedNoEvidence`: that would say "there was
nothing to score" when the truth may be "there was a vector and centring destroyed
it". Different remedies (re-embed vs. look at the centroid), so a separate
counter — the same reasoning that made C5 split its six reasons.

Gates: npm test 303/303 (0 skipped), npm run check 12/12.
2026-11-05 12:04:17 +08:00
7ad76167c5 fix(memory): separate "not enough blocks yet" from "no common direction" (C5)
MIN_SAMPLE (added in febf9f6) makes a one-vector space non-anisotropic: one
sample is not a direction. That verdict then fell straight into the `isotropic`
branch of `centroidStatus`, so a space holding a single block reported:

    reason = isotropic

Measured before this commit:

    1 block                reason=isotropic  available=false count=1 isAnomalous=false meanLength=1.0000
    2 blocks, orthogonal   reason=ok         available=true  count=2 isAnomalous=true  meanLength=0.7071

## Why the distinction is the whole point of C5

    isotropic        this space will never have a common direction — stop expecting centring
    too_few_samples  write a few more blocks and it starts working

Two diagnoses with opposite remedies. C5 exists so that an operator asking "why is
centring off?" gets an answer they can act on; reporting the first when the
second is true sends them to give up instead of to keep using the thing.

This is my own hole: I added MIN_SAMPLE, and the misclassification exists only
because of it. `zero_sample` (a Go branch) already covered count == 0; nothing
covered 0 < count < MIN_SAMPLE.

## Fix

A `too_few_samples` branch placed BEFORE the isotropic verdict — order is
load-bearing, because the isotropic check is what used to swallow this case.

    no_centroid → dim_mismatch → zero_sample → stale → too_few_samples → isotropic → ok

Stale still comes first: a centroid built from one block that has since seen
twelve is stale, and rebuild is the action, not "write more".

## Criterion

`test/phase-b-c5-reasons.test.js` gains case 11, asserting both sides of the
boundary: one vector reports `too_few_samples`, two vectors do not (MIN_SAMPLE is
2, so two samples must clear the branch).

Mutation — deleting the branch:

    not ok 11 - C5: a one-vector space reports too_few_samples, not isotropic
    error: C5: "not enough blocks yet" and "this space has no common direction" lead
           to opposite actions. ...   (actual: 'isotropic')

## Still HA-divergent, recorded not fixed

Two orthogonal blocks report `ok` / available with meanLength 0.7071 — i.e. the
0.6 threshold calls a two-sample space anisotropic and centring engages. That is
HA-consistent behaviour (HA has no minimum sample count), and it is now visible
rather than silent because the count and meanLength are in the reason's siblings.
Not changed here: raising the threshold would depart from the port.

Gates: npm test 302/302 (0 skipped), c5 11/11, npm run check 12/12.
2026-11-05 12:01:07 +08:00
5462934a84 test(memory): pin the reason at the TOOL surface, the fourth place it can vanish (C5)
C5's reason now has three layers of coverage:

  1. store — six fixtures, one per cause (`centroidStatus`)
  2. plugin — `stats().centroid_state`, read-only (no rebuild scheduled)
  3. tool — `ctx.tools.execute('stablemem_stats')`

Layer 3 is the one that matters in dsh: the model does not call `mem.stats()`, it
calls the tool. And the tool's handler is a single `return mem.stats()`, so a
later refactor that cherry-picks fields would keep layers 1 and 2 green while the
only surface anyone actually reaches goes silent.

That is the fourth instance of one shape in this branch:

  P1   queryVector computed, dimension-checked, then dropped
  D2   drop counters computed, then not printed
  C5   the reason computed, read by nobody
  C5   the reason surfaced on the service, but not pinned at the tool

Each time the value existed, the criteria were green, and no operator could see
it. So the criterion asserts on the tool's own envelope.

## Criterion

Drives the real ToolRuntime exactly as `test/phase-b-h.test.js` does, with a real
`AbortSignal` (`ToolRuntime` reads `callerSignal.aborted`, so `undefined` throws):

    empty space        -> centroid_state present, reason 'no_centroid'
    after 8 commits    -> reason 'ok', available true, sibling fields intact
    drift past ratio   -> reason 'stale', rebuildScheduled FALSE

The last assertion doubles as the read-only proof at this layer: a tool an
operator polls must not drive the rebuild loop.

Mutation — handler returns a cherry-picked subset:

    not ok 10 - C5: the stats TOOL returns the reason, not just the service method
    error: 'C5: the tool must expose centroid_state — that is the surface the model
           and the operator actually call. ...'

## Two of my own assertions were wrong first

Recorded because both are the failure mode this branch keeps hunting:

- `assert.equal(s.centroid, s.centroid, ...)` — a tautology; can never fail.
- `assert.equal(typeof x !== 'undefined', 'boolean', ...)` — compares `true` to
  the string `'boolean'`; fails for a reason unrelated to its subject.

Replaced with `assert.ok('last_recall' in healthy, ...)`, which states the actual
precondition. A criterion that cannot fail, or that fails for the wrong reason,
is worse than no criterion — it consumes attention and proves nothing.

Gates: npm test 301/301 (0 skipped), c5 10/10, npm run check 12/12.
2026-11-05 11:58:58 +08:00
16cfdb75d6 docs(memory): say what centroidRecallState is NOT, and why it has no caller
The C4 slice reported this method as dead code: `grep -rn centroidRecallState lib/`
returned nothing but the definition, and the instruction was "wire it into recall
or delete it — do not leave it as furniture".

Neither, and the reason is worth writing down rather than re-deriving:

- It cannot be wired into recall. It calls `centroidStatus`, whose first line is
  `countVectored` — a COUNT that C3-k forbids on the per-turn pre-step path. The
  scoring path therefore uses `loadCentroid` (one indexed SELECT).
- It should not be deleted. `assertC3RecallUnusable` in test/phase-b-h.test.js
  runs the SAME predicate against every C3 mutant, so this method is what pins
  Go's `LoadCentroid` semantics (`centroid_store.go:179-185`: a zero-sample centre
  is unusable, and the recall side carries no stale bit).

So it is a diagnostic seam, like `StableMem#saveSpaceCentroid`. The name
"Recall-facing projection" was the actual defect — it invites exactly the wiring
C3-k forbids. The name stays (renaming would churn a mutation-shared predicate for
no behavioural gain); the doc now says what it is and what it is not.

Comment only, no behaviour change.

Gates: npm test 300/300, npm run check 12/12.
2026-11-05 11:55:44 +08:00
44f883745f feat(memory): give C5's reason an outlet, with a read that cannot schedule (C5 follow-up)
28866ac made `centroidStatus` name why centring is disabled. Nothing read it.
`stats()` reported `centroid` from `loadCentroid` — the raw row, which says WHAT
the centre is and not WHY centring is or is not happening — and the only two
callers of `centroidStatus` in `lib/` were `centroidRecallState` (which spreads
the result but is itself only used by tests) and `#ensureCentroid` on the write
path, which discards the return value. A grep for `.reason` under `lib/` returned
nothing but the definition itself.

Same failure as D2 one commit earlier: the value existed, was covered, and no
operator could see it.

## What this adds

`stats().centroid_state` — the verdict, alongside the existing raw `centroid`:

    centroid        the stored row (dim, mean, vectorCount, stats)
    centroid_state  { available, stale, reason, delta, rebuildScheduled, ... }
                    reason ∈ ok | no_centroid | dim_mismatch | zero_sample |
                             stale | isotropic

`centroid` keeps its exact previous shape, so nothing downstream changes.

## The part that is easy to get wrong

`centroidStatus` **registers a rebuild** when it judges a space stale. A stats
read must not mutate anything, so the call passes `schedule: () => {}` — with
the no-op scheduler `rebuildScheduled` is always false here, which is what a pure
read should report. Without it, every operator polling `stats()` on a drifted
space would keep the rebuild loop alive.

## Criterion

`test/phase-b-c5-reasons.test.js` gains a ninth case driving the real plugin:

    fresh space            -> reason 'no_centroid'
    after 8 commits + wait -> 'ok', available true, raw row and verdict both present
    drift past the ratio   -> 'stale', stale true, rebuildScheduled FALSE

The last assertion is the one that matters: it distinguishes "reports staleness"
from "acts on it".

Mutation — deleting the `centroid_state` field:

    not ok 9 - C5: stats() exposes the reason without scheduling a rebuild
    error: "Cannot read properties of undefined (reading 'reason')"

Also removed a tautology I had written in the same case
(`assert.equal(s.centroid, s.centroid, ...)`) — it can never fail, and a criterion
that cannot fail is worse than none.

Gates: npm test 300/300 (0 skipped), npm run check 12/12.
2026-11-05 11:54:12 +08:00
0bc160b1bd docs(memory): list the new drop counters in the log-format table (D2)
The JSDoc table above `logRecallTurn` documents which fields the degradation group
contains. It still listed only `degraded` / `skipped` / `skippedDim` / `err`
after 07f8438 and 8915820 added three counters to that same group — so the table
described a line the code no longer emits, and an operator reading the docs would
not know `skippedScore` was available.

Comment only; no behaviour change. `stats().last_recall` already carries the same
fields because it hands `this.lastRecall` through unchanged (lib/index.js:840),
which this change also makes explicit in the table's wording ("where the
candidates went" rather than "whether the answer is trustworthy" alone).

Gates: npm test 299/299, memory-log 17/17.
2026-11-05 11:51:59 +08:00
bdd91234a5 test(memory): pin that a NON-ZERO drop counter survives to both surfaces (D2)
L2 asserts the three D2 counters are printed when every one of them is zero. That
proves the format string mentions them; it does not prove a real value makes it
through. Two things sit between the counter and the operator:

  `segmentRecallBudget` slices the hits array — and `slice` does not copy
  non-enumerable properties, which is what all the counters are;

  `lastRecall` is assembled separately from the log line.

So "correct in the returned array" and "visible to an operator" are different
claims, and only the second one is what D2 is about.

## Fixture

Three blocks on separate axes, query mostly on axis 0, default `minScore`:

    block0 cosine ~0.95   -> hit
    block1 cosine ~0.30   -> skippedScore
    block2 cosine 0       -> skippedNoEvidence

Measured through the real plugin path (apply + commit + contextForStep):

    log : ... skipped=0 skippedDim=0 skippedScore=1 skippedNoEvidence=1 truncated=0 err=- ...
    last: {"mode":"hybrid","skipped_score":1,"skipped_no_evidence":1,"truncated":0,...}

All three destinations are non-empty at once, so the criterion cannot pass by
accident of an empty fixture.

## Mutation

Removing the three fields from the log line turns both criteria red:

    not ok 2 - L2: debugLog=true emits one line covering recall, budget, degradation and injection
    not ok 4 - L2b: non-zero drop counters appear in the log and in last_recall

L2 alone would already catch that; L2b is what catches the other half — a counter
that survives the log formatting but dies in the slice, or lands in one surface
and not the other.

Gates: npm test 299/299 (0 skipped).
2026-11-05 11:49:01 +08:00
891582041d fix(memory): print the downstream drop counters, not just compute them (D2 follow-up)
07f8438 added `skippedScore` / `skippedNoEvidence` / `truncated` to the recall
result. They were correct and covered, and completely invisible: the debug log
line still printed only `skipped` / `skippedDim`, and `stats().last_recall`
carried only `skipped_fp` / `skipped_dim`.

A counter nobody can read is indistinguishable from no counter. The whole point
of D2 was that an operator whose threshold is swallowing the corpus reads
`skipped=0` and concludes nothing was excluded — that operator reads the LOG, not
the returned array.

## What this adds

- the debug line now ends its degradation group with
  `... skippedScore=<n> skippedNoEvidence=<n> truncated=<n> err=<...>`
- `last_recall` carries `skipped_score` / `skipped_no_evidence` / `truncated`,
  so the durable surface says the same thing the transient line does

Nothing else changes: the counters were already computed, and the existing
`skipped=0 skippedDim=0` substring assertions still match because the new fields
follow them.

## Criterion

L2 in `test/memory-log.test.js` already asserts the degradation group is printed
in one line; it now also requires
`skippedScore=0 skippedNoEvidence=0 truncated=0` to be present. That is the
"printed, not merely computed" half — the values themselves are pinned by
`test/recall-accounting.test.js` (non-zero cases).

Mutation: deleting the three fields from the log line turns L2 red —

    not ok 2 - L2: debugLog=true emits one line covering recall, budget, degradation and injection
    error: 'downstream drop counters'

Gates: npm test 298/298 (0 skipped), npm run check 12/12, memory-log 16/16.
2026-11-05 11:44:11 +08:00
07f8438964 fix(memory): count every dropped candidate, not just the two upstream ones (D2)
`recallBlocks` could make a candidate disappear in three downstream places, and
none of them incremented anything:

    · MinScore gate        it scored, but below the threshold
    · structural gate      neither a usable vector score nor a symbol hit
    · caller topK          survived everything, fell outside the requested count

Measured before this change, 6 blocks in one space, `minScore: 0.99`:

    hits=0  skipped=0  skippedDim=0

That reading is indistinguishable from "the space holds one block". An operator
whose threshold is swallowing the whole corpus is told "nothing was excluded",
and goes looking at the wrong subsystem. Only the space filter and the width
filter were ever counted — the two that are decided before scoring.

## What this adds

Three non-enumerable counters, each named after its own remedy:

    skippedScore       lower the threshold
    skippedNoEvidence  re-embed: there was nothing to score
    truncated          not a defect — reported so the totals reconcile

Plus the invariant that makes the next silent path impossible:
**hits + skipped + skippedDim + skippedScore + skippedNoEvidence + truncated
=== the candidate pool.** A future `continue` that forgets a counter breaks it.

## Criterion

`test/recall-accounting.test.js`, one fixture per path, asserting each counter's
own value (not a merged total — "scored too low" and "nothing to score" have
opposite remedies). The fourth test is the reconciliation above.

Mutations, both in a scratch copy:

    remove `gatedByScore += 1`     -> criteria 1 and 4 red
    remove `gatedNoEvidence += 1`  -> criterion 2 red

## Collateral: two static mutation anchors moved

`test/phase-b-a.test.js` pins the V5 gate and the caller-topK truncation by
rewriting `lib/recall.js` text. Making the gate multi-line (to hold a counter)
and threading `drops` through `withSkipped` changed those exact strings, so the
anchors are updated to the new text — same mutations, same properties pinned.
Running the file confirms 38/38 afterwards; without the update three V5†
mutations failed on a stale anchor rather than on their subject.

Gates: npm test 298/298 (0 skipped), npm run check 12/12. Frozen faces untouched.
2026-11-05 11:42:06 +08:00
68775d327f fix(packaging): ship lib/embed/, without which the published tarball cannot load
`package.json` declared

    "files": ["lib/*.js", "lib/loop.js", "lib/loop/*.js", "lib/types/*.d.ts"]

`lib/*.js` matches ONE level. `lib/embed/http.js` therefore never entered the
tarball, while `lib/index.js:35` imports it:

    import { createHttpEmbedder } from './embed/http.js'

## How this survived

Every test in this repo runs from the source tree, where the file obviously
exists. `npm pack --dry-run` printed "10 files" — a plausible number nobody had
reason to read through. The defect is only reachable from the one path that
ships: installing the tarball.

Measured, extracting the tarball and importing it:

    IMPORT FAILED: ERR_MODULE_NOT_FOUND
    Cannot find module '.../package/lib/embed/http.js'
      imported from '.../package/lib/index.js'

After the fix, the same command:

    IMPORT OK, exports: StableMem,apply,default,defaultPath,inject,instanceKey,lastUserText,name

## Fix

`lib/embed/*.js` added to the allowlist. The tarball goes from 10 to 11 files
(86.1 kB) and `lib/embed/http.js` is in it.

## Criterion

`test/packaging.test.js`:

1. every `.js` under `lib/` must be matched by at least one `files` pattern.
   The patterns compile to regexes the way npm treats them (`*` does not cross
   `/`), so the criterion needs no hand-maintained file list and cannot drift.
2. every relative import in `lib/index.js` must resolve on disk — otherwise
   criterion 1 could pass vacuously for a file that does not exist.

Reverting `files` to the old list turns criterion 1 red:

    not ok 1 - packaging: every file under lib/ is covered by package.json "files"
    these files live under lib/ but no "files" pattern reaches them ...
      lib/embed/http.js

## Why it belongs in this branch

V6 made the embedder assembly load-bearing (two instances, one per side). A
release that cannot import `lib/embed/http.js` has no embedder at all, so the
distinction V6 introduced would never reach a user. The bug predates this
branch; its blast radius is the shipped artifact.

Gates: npm test 294/294 (0 skipped), npm run check 12/12, npm pack 11 files.
2026-11-05 11:38:56 +08:00
c97775bd18 feat(memory): encode stored blocks on the document side, questions on the query side (V6)
The HTTP protocol carries a `side` field — `query` or `document`, HA's
`PurposeQuery`/`PurposeDocument` (`http_embedder.go:105` / `:109`). The port made
it a construction-time setting of the adapter (default `query`) and then **no
call site ever supplied it**: both the write path (`lib/index.js`) and the recall
path (`lib/recall.js`) took the default.

So every stored block was encoded as if it were a question. Nothing fails when
that happens — both sides produce unit vectors of the declared width — which is
exactly why it survived: the corruption is only visible in retrieval quality,
and retrieval quality was already the known-unmeasured part.

## Why two instances and not a per-call argument

`side` is a property of what you are encoding, so a per-call parameter is the
more natural shape. It is not the shape used here, for one reason: the adapter
travels `side` verbatim in the request body from a closure, and giving it a
second, per-call way to receive the same field would leave two sources of truth
for one wire value. Creating two adapters over the same endpoint/model/dimension
— identical fingerprint, differing only in `side` — keeps the wire contract as it
was and puts the distinction where it belongs: in assembly.

`lib/embed/http.js` is therefore **unchanged**.

An injected `config.embedder` (tests, a future local adapter) has no second
instance, so the write path falls back to it (`documentEmbedder ?? embedder`).
That is the previous behaviour, not a second code path — and it is pinned.

## Criterion

`test/phase-b-v6-sides.test.js` starts a real HTTP server that records the `side`
of every request, then drives the **production assembly** (`apply()` with
`config.embed`):

    commit  → request body side === 'document'
    recall  → request body side === 'query'

The assertion is on the field that actually went out on the wire, not on an
internal variable. A second test pins the injection fallback: a single injected
embedder must still serve both paths, or every existing test and every
single-sided adapter breaks.

Mutation M-V6 — pointing the write path back at `this.embedder`:

    not ok 1 - V6: storing a block encodes it on the document side, asking encodes the query side
    error: V6: a block being STORED must be encoded on the document side. ...
      + actual - expected
      + 'query'
      - 'document'

## What this does NOT establish

That the provider actually encodes the two sides differently. `side` is a
request field; what the endpoint does with it is the endpoint's business, and
this repo has no model whose asymmetry is measurable. What is now true is that
the distinction reaches the wire instead of being silently defaulted away.

Gates: npm test 292/292 (0 skipped), npm run check 12/12. Frozen faces untouched.
2026-11-05 11:35:02 +08:00
28866ac6e6 feat(memory): name why centring is disabled, instead of only reporting that it is (C5)
`centroidStatus` computed four distinct ways for centring to be unavailable and
then flattened all of them into `available: false`. The result was that
`no_centroid` and `dim_mismatch` both looked like `stale: null`, and `isotropic`
was indistinguishable from a healthy space except by `available`. That answers
"may I centre?" and says nothing about "why is semantic recall never
correcting?" — four different operator actions (build a centroid, fix the
dimension, wait for a rebuild, accept that this space has no common direction)
collapsed into one symptom.

Every branch now carries `reason`:

    ok           available: true
    no_centroid  no row for this space at all
    dim_mismatch a centroid exists but its width is not this query's
    zero_sample  a row exists but was built from zero vectors
    stale        usable in principle, no longer describes the space (drift > 20%)
    isotropic    permanent property of the space: no common direction to remove

`zero_sample` is a fifth cause beyond the four the plan names, because the Go
branch has it too (`centroid_store.go:179-185`) and it is not the same problem
as a missing row.

## Criterion

`test/phase-b-c5-reasons.test.js` — one fixture per reason, asserting **the
reason value itself** (not `available`, which is multi-valued). Plus:

- an invariant test: `available === (reason === 'ok')`, so `reason` cannot drift
  into decoration that disagrees with the conclusion it explains;
- a test that `centroidRecallState` still projects `stale: null` (the scorer must
  not start reacting to staleness) *while* `reason` travels through, so an
  operator can still see why.

Every call injects `schedule: () => {}`. `centroidStatus` schedules a rebuild
when it judges a space stale — a criterion that calls it without isolating that
side effect ends up triggering the very behaviour it is inspecting. C4e's first
version proved that the hard way.

## Mutation

Collapsing all six reasons into one value (`reason: 'unavailable'`) turns all
eight criteria red:

    not ok 1 - C5: a space with blocks but no centroid row reports no_centroid
    not ok 2 - C5: a centroid of the wrong width reports dim_mismatch
    not ok 3 - C5: a centroid row with zero samples reports zero_sample
    not ok 4 - C5: a centroid that drifted past the ratio reports stale
    not ok 5 - C5: an isotropic space reports isotropic, not stale
    not ok 6 - C5: a healthy space reports ok, and only ok carries available=true
    not ok 7 - C5: reason and available never disagree
    not ok 8 - C5: centroidRecallState keeps its stale:null projection but carries the reason

## What this does NOT change

Scoring, gating and the recall path are untouched: nothing reads `reason` to
decide whether to centre, and `centroidRecallState` keeps `stale: null`. This is
a reporting seam, not a behaviour change — C4's own criteria (C4a/b/c/d/e/g/h/i)
and phase-b-h's C3 assertions stay green.

Gates: npm test 290/290 (0 skipped), npm run check 12/12. Frozen faces untouched.
2026-11-05 11:32:49 +08:00
5cb97edf39 fix(memory): stop scoring renormalised floating-point dust as a direction (C4 follow-up)
`centerVector` treated any positive norm as a direction:

    norm = Math.sqrt(norm)
    if (!(norm > 0)) return null
    for (...) out[index] /= norm

A block that IS the space mean differs from it only by floating-point residue
(~1e-17). That residue passes `norm > 0`, gets normalised to unit length, and
then scores against the query as if it were signal.

## Measured

Space of two identical blocks, query deliberately off the block direction (if
the query equalled the mean, its centred form would collapse and centring would
fall back to raw scores, hiding the defect):

    raw cosine, no centroid        0.8608
    with centroid, before this fix 0.2183   <- dust, renormalised
    with centroid, after this fix  no hit   <- dust judged "no direction"

At the default `minScore = 0.5` the 0.2183 scores were filtered anyway, so the
user-visible outcome (0 hits) does not change. What changes is that the number
is no longer manufactured: with `minScore: 0` the earlier build would surface
those blocks at a noise-derived score, and a future change to the threshold
would have made that reachable.

CENTER_EPSILON = 1e-9 is the floor. It also matches the Go side, where the
cosine of a zero vector is 0 rather than a random angle.

## Criterion C4i — and what it deliberately does not pin

C4i asserts the invariant: **every returned hit must carry the raw cosine
(0.8608); a value near 0.22 means dust was renormalised.** It therefore accepts
either of two future policies — drop such blocks (what this commit does), or
skip centring entirely for a space with no variance. The only thing it forbids
is continuing to score dust.

Reverting the guard (`norm > CENTER_EPSILON` -> `norm > 0`) makes C4i red, which
is the mutation this criterion exists for.

## KNOWN LIMITATION — reported upward, not fixed here

A space whose blocks are all identical now returns NO hits where it previously
returned them at 0.8608, because centring subtracts the only direction the space
has. That is not introduced by this commit — epsilon only made it deterministic —
but it is a real recall loss for a realistic case (the same note stored twice),
and it is a C4 regression against the pre-C4 behaviour.

Two candidates, deliberately NOT applied here because both change scoring
semantics and one departs from HA:

  (a) keep dropping them (current, HA-consistent: Go's cosine of a zero vector
      is 0, so those blocks fall below any threshold too);
  (b) do not centre a space whose mean norm is ~1.0 — there is no variance to
      remove — which preserves the 0.8608 raw scores. This adds an upper
      threshold HA does not have, so it is the parent's call.

No criterion pins "0 hits" for the duplicate space: pinning it would encode the
limitation as intended behaviour.

Gates: npm test 282/282 (0 skipped), frozen faces untouched.
2026-11-05 11:30:48 +08:00
363b1ba21a docs(memory): record what the C4 write path costs, and two "unwired" calls that aren't
Three comments that were either wrong or silent. No behaviour change.

## The first-build cost claim was false

`#ensureCentroid` said "首次建的代价很小:那时块还少" (the first build is cheap —
there are few blocks then). That is exactly backwards for the situation the code
exists to fix: a production database that already holds thousands of vectored
blocks while `graph_centroid` is empty. The W8 scenario IS a database with many
blocks and no centroid.

Measured on a real database (512 dims, 2000 vectored blocks, empty centroid
table, first commit through the production path):

    FIRST commit after upgrade   105.1 ms   (synchronous SELECT + parse + mean)
    STEADY commits               8.7-9.4 ms (loadCentroid SELECT + centroidStatus COUNT)

and the first build grows linearly with the space (~420 ms at 8000 blocks). The
number is now in the comment, together with the decision it justifies: keep the
one-time stall rather than move the build to the async path, because an async
first build leaves the first recalls after an upgrade silently uncorrected —
precisely when n is large enough for centring to matter. A bounded stall is
easier to notice than a silent degradation.

The steady-state ~9 ms is new with C4 and is now stated as such: every commit
pays one `loadCentroid` SELECT plus the `countVectored` COUNT inside
`centroidStatus`. C3-k forbids that COUNT on the recall path, not on the write
path.

## Two calls with no production caller, and why neither is a dangling wire

`saveSpaceCentroid` and `#centroidRebuiltAt` both looked like the defects this
round kept finding (code written, never wired). They are not, but leaving that
unwritten meant the next reader had to re-derive it — and the two live cases
(`saveSpaceCentroid` in `lib/`, the throttle state on disk) really were dangling.

- `saveSpaceCentroid` is the public programmatic seam for a caller that computed
  a centroid elsewhere. Production centroids are derived from the blocks by
  `store.rebuildCentroid`, driven from `commit` via `#ensureCentroid`. Stated in
  its JSDoc, along with the fact that the only in-repo caller is a test.
- `#centroidRebuiltAt` stays in memory, and the reason `graph_centroid.built_at`
  cannot replace it is now written down: `built_at` records a BUILD, the throttle
  needs the last SCHEDULE, and the two diverge precisely when a rebuild throws.
  Throttling on `built_at` would re-schedule on every commit after a failed
  rebuild — write amplification on the failure path. The cost of the in-memory
  map (one extra bounded async rebuild after a restart, measured 85.5 ms at
  2000x512) is smaller, and it is stated next to it.

Gates: npm test 281/281, npm run check 12/12. Frozen faces untouched.
2026-11-05 11:23:32 +08:00
febf9f6f3c test(memory): pin the C4 gates that were left unguarded (C4 follow-up, supervision)
Three of C4's control surfaces had no criterion at all: deleting them left every
suite green. This commit adds the criteria, and the two defects they exposed.

## What was unguarded, measured

Each mutation below was applied to a scratch copy of lib/ and run against the
suite as it stood at 05e93b2.

1. The `isAnomalous` gate in `scoreVectorCandidates`
   `centroid?.isAnomalous === true &&` -> `centroid !== undefined &&`
   Result: c4 6/6 green, phase-b-h 137/137 green, phase-b-a 38/38 green.
   Why: every C4 fixture is anisotropic (two of them assert isAnomalous ===
   true), and C4c is "no centroid row", which is a different branch from "a
   centroid that was judged isotropic". The only isotropic assertion in the repo
   (phase-b-h:1133) checks `rebuildCentroid`'s verdict, not the scoring path.

2. `countVectored` on the recall hot path
   `store.countVectored(fingerprint)` inserted before the centroid read.
   Result: c4 6/6 green, phase-b-h + phase-b-a 175/175 green.
   Why: nothing scans lib/recall.js for the C3-k boundary. phase-b-a:67 does read
   recall.js, but it asserts the F2 fingerprint gate.

3. MIN_SAMPLE (this one is a defect, not just a missing criterion)
   A one-vector mean IS that vector, so its norm is exactly 1.0 and it clears
   0.6 by construction. Measured with a real clock and the real commit path on
   the C4 corpus: right after the 8th commit the centroid reported
   `vectorCount=1 meanLength=1.0000`, and a recall that returns 3 rows without a
   centroid returned 1 with it — the block that formed the "mean" was centred to
   the zero vector and dropped. The drift rebuild corrects it ~2s later, so the
   window is short but the state is wrong, and in a slowly growing space it lasts
   until the throttle expires.

## What this commit adds

- `BlockStore.MIN_SAMPLE` (= 2) and `vectors.length >= MIN_SAMPLE` in the
  anisotropy verdict. Two is the smallest number for which the mean is a property
  of the set rather than of one member.
- **C4h** pins both halves of that: the verdict (`isAnomalous === false` for one
  vector, with `meanLength === 1` asserted as the precondition that makes the
  0.6 test insufficient) and the consequence (the only block stays recallable).
- **C4g** pins the isotropic gate. Fixture numbers come from running the same
  probe against a gated and an ungated build, not from paper: 8 orthogonal
  one-hot vectors, `meanLength = 0.353553`, `isAnomalous === false`; a query
  along `unit(ones)` yields 8 hits of cosine 0.3536 with the gate, and 0 hits
  without it (every centred cosine turns negative and V5's structural gate drops
  the lot). A three-axis query additionally pins 0.8111 / 0.4867 / 0.3244.
  The identical-pair query is deliberately NOT the main assertion: with both
  sides centred its cosine is 1 either way, so it cannot discriminate.
- **C3-k** statically scans lib/recall.js for `countVectored(` / `centroidStatus(`
  after stripping comments, and carries a positive self-check: the same scan must
  find `centroidStatus(` in lib/index.js, so a broken regex cannot leave the two
  "must not contain" assertions vacuously true.
- **C4d** gained the assertion it was missing. It asserted that a centroid
  exists, is anomalous and has the right width — all true of a centroid built
  from a single block. It now also asserts `vectorCount === CORPUS.length`, and
  waits for the drift rebuild before asserting on contents. Without the count
  assertion C4d is green while the mean describes one block.

## Mutations verified (each red in a scratch copy, working tree untouched)

  delete isAnomalous gate        -> C4g red
  countVectored in recall.js     -> C3-k red
  drop MIN_SAMPLE                -> C4h red
  revert "refresh a drifted centroid" -> C4d + C4e red

## Documentation that had drifted from the code

- recall.js still said "No centroid centering here. C4 ... is a separate slice
  layered on top of this one" directly above the function that now implements
  centering. Replaced with a pointer to where it actually lives.
- The one-sided-centring cosine was recorded as 0.3787 in the comment and "near
  0.38" in C4b's message. Measured on C4b's own five-block fixture it is 0.1897
  (cosine is symmetric, so centring either single side gives the same value).
  Both now say 0.1897 and name the fixture.

## Measured costs (for the write path this slice touches)

Pre-populated space, 512 dims, 2000 vectored blocks, empty graph_centroid — the
state a production database is in when it upgrades:

    FIRST commit after upgrade   105.1 ms   (synchronous O(n*d) rebuild)
    STEADY commits               8.7-9.4 ms (loadCentroid SELECT + centroidStatus COUNT)

The first build is accepted as-is: it happens once per space lifetime, it is
bounded, and moving it to the async path would leave the first recalls after an
upgrade silently uncorrected — a bounded stall is easier to notice than a silent
degradation. The steady-state COUNT is new with C4; C3-k keeps it off the recall
path, where it would run every turn.

## Real path (dsh headless, overlay on the real profile)

Same corpus and query, only the centroid row differs:

    centroid present (count=8, meanLength=0.972934)
      [1] (text 0.63) 第七号仓库的备用门禁码是 QP47-812。
    centroid deleted
      [1] (text 0.69) 这段时间一切照旧,没有什么特别的事情。
      [2] (text 0.69) 第七号仓库的备用门禁码是 QP47-812。

top-1 moves from the hub to the target, which is the behaviour C4 exists for.

## What this does NOT establish

Centring changes angles; it does not calibrate absolute scores. DEFAULT_MIN_SCORE
= 0.5 was never derived from a centred distribution, and the uncalibrated-absolute-
score problem recorded in phase-b-p1-vector-wiring.test.js:51-58 remains open.
A green C4 means "centring is wired into scoring and changes the ranking in an
anisotropic space", never "semantic recall is good now".

C5 (distinguishing no_centroid / dim_mismatch / stale / isotropic reasons) is
untouched: `centroidRecallState` still returns `stale: null`.

Gates: npm test 281/281 (0 skipped), npm run check 12/12, npm pack 10 files.
Frozen faces untouched: docs/, README.md, scripts/, lib/loop*.
2026-11-05 11:21:02 +08:00
05e93b2f5c fix(memory): let the write path refresh a drifted centroid (C4 follow-up)
0a6adff 里我自己留了一个洞,而且正是这一整轮反复抓的同一个病:逻辑写了,但没被调到。

`#ensureCentroid` 当时的形状是「没有中心就建一个,有就 return」:

    if (this.store.loadCentroid(fingerprint) !== undefined) return
    this.store.rebuildCentroid(fingerprint)

于是中心永远停在第一次 commit 的快照上。`rebuildCentroid` 只算一次,
之后块数一直涨,均值却再不动 —— 块数翻八倍,它还描述着 1 条块。
召回侧只 `loadCentroid` 纯读,永远不会自己纠正。

根子在于 `centroidStatus`:它内部早写好了完整判据(块数漂移 > 20% ⇒ stale,
10 分钟节流 + 异步重建),而在 C4 之前它在 lib/ 里**没有任何生产调用者**。
我接线时只接了「建第一次」,让它继续当着死代码 —— 这跟 P1/C4 要修的
「代码写了没接线」是同一类错误,只是换了一层。

修法:有中心时不再 return,交给 `centroidStatus` 判。它自己决定要不要重建。
仍然放在 commit 之后而不是召回路径,正是因为它第一行是 `countVectored`,
C3-k 明令该查询不得进每轮 pre-step 的热路径。

判据 C4e:写入侧必须让漂移的中心重新跟上

    commit 1 条  → 中心 vectorCount = 1
    再写 7 条    → 漂移 700% >> 20%
    假时钟拨过 2s → 中心 vectorCount 必须 = 8

M4 变异(退回 0a6adff 的「有中心就 return」)红原文:

    C4e: a centroid that drifted past the 20% ratio must be rebuilt by the
    write path. It is still describing 1 blocks while 8 carry vectors
    —— nothing in lib/ ever consults the stale judgement, so the mean stays
    frozen on whatever the first commit happened to see.

★ C4e 第一版是假绿,一并记下来

第一版在断言前调了 `mem.store.centroidStatus(SPACE)` 来验 stale 前置。
但 `centroidStatus` 自己就会调度重建 —— 于是不管写入侧接没接,重建都会发生,
判据在自证。M4 变异下它 6/6 全绿,才把这个问题揪出来。
改成手工算漂移比(不碰 centroidStatus)之后,M4 才咬得住。

同一类陷阱会在任何「被测函数恰好也能触发被测行为」的地方复现,
所以判据里宁可自己算前置常量,也不去调那个会把副作用带进来的接口。

★ What this does NOT establish

C4e 证的是「漂移会被发现并触发重建」,不证重建的时机对用户体验无感 ——
10 分钟节流窗口内中心仍是旧的,这段时间召回用的还是上一个快照的均值。
窗口长度与漂移比的关系需要实测分布,属效果范畴,不在本片。

三门:npm test 278/278(0 skipped)、npm run check 12/12、npm pack 10 files 79.3 kB。
冻结面:docs/、README.md、scripts/、lib/loop* 零改动。
2026-11-05 10:47:43 +08:00
0a6adff5cb fix(memory): wire centroid centering into both the write path and scoring (C4)
C4 的判据原文:「校正时查询向量与块向量都去均值,不能只处理一边」
(docs/phase-b-criteria.md:253,依据 HA centroid_store.go:443-451 的 centerBoth)。

C4 缺的不是一半,是两层都没接

1. 中心从来不存在。`saveSpaceCentroid`(lib/index.js) / `centroidStatus` /
   `centroidRecallState` 在整个 lib/ 里没有生产调用者,`rebuildCentroid`
   只有测试在调。所以生产库的 `graph_centroid` 永远是空表,`loadCentroid`
   永远返回 undefined,`isAnomalous` 永远 false —— 校正一次也不会发生。
   实测:`/tmp/p1/e2e.db` 的 `graph_centroid` 零行。

2. 打分不用中心。`lib/recall.js` 全文没有 centroid 一词,
   `scoreVectorCandidates` 只算裸余弦。`centerBoth` 全仓零命中。

只做第 2 层就是又一个「代码写了没接线」:判据会绿、实现会绿、生产永远 false。

写入侧:lib/index.js
  · 新增 `#ensureCentroid(fingerprint)`,在 commit 成功后调用
  · 只在「本空间没有中心行」时建一次;有行就交给已有的 stale 判定
  · 失败吞掉:一次重建失败不能让写入失败(同 D1:向量侧故障不得带走主路径)
  · 不放在召回侧:rebuildCentroid 是 O(n·d)(实测 dim=512、2000 块 85.5ms),
    而召回每轮 pre-step 都跑;C3-k 也明令 countVectored 不得进热路径,
    而 centroidStatus 第一行就是它

打分侧:lib/recall.js
  · `scoreVectorCandidates` 读一次中心(loadCentroid 纯读,一次索引查询)
  · 仅当 `isAnomalous === true` 且 `centroid.dim === queryVector.length` 时启用
  · 两侧同时减均值 + 重新归一化(新增 `centerVector`,对齐 HA CenterVector)
  · 维度不符 ⇒ 不校正也不报错;减完是零向量 ⇒ 退回未校正路径
  · 门控是正确性不是性能:各向同性空间无条件校正会毁掉结果
    (实测 8 块正交 mean norm 0.3536:命中数 8 → 1,7 块余弦 +0.0496 → −0.1416,
     且因 V5 结构性门(vectorScore <= 0 且 symbolHit === 0 直接丢弃)
     minScore 调到 -1 也救不回来)

判据:test/phase-b-c4-centering.test.js(283 行,已登记进 npm test)

  C4-0  前置:空间真的会被判为各向异性(mean norm 0.9729 >= 0.6)
  C4a   born-RED:去均值把 top-1 从 hub 翻到 target
        红原文(28e18d0):hub=0.6949(v=0.9928) target=0.6901(v=0.9859)
        绿后:            target=0.6300(v=0.9000)
  C4b   mutation-GUARD:同值对余弦恒为 1(纯代数,不依赖语料)。
        未实现时本来就绿(裸余弦同值对 = 1),只有「只减一边」能让它红。
        这条不可省:实测只减 query 时 C4a 的 top-1 仍然是 target,C4a 抓不到。
  C4c   guard:没有中心时照常返回结果(hub 仍然赢),不是让召回失败
  C4d   第一层的守门人:走真 apply() + commit() + recall(),不手工 rebuildCentroid,
        断言写完 8 条后 loadCentroid 必须已存在,且端到端 top-1 是 target
        —— 没有这条,前三条的绿只能证明算术,证明不了接线

变异自证(三档,全部在临时副本做,真改 lib/)

  M1 去掉中心化(loadCentroid 恒 undefined)        → C4a + C4d 红(5→3 pass)
  M2 块侧不中心化(只减 query,即 M-C4)            → C4a + C4b + C4d 红(5→2 pass)
     C4b 红原文:got 0.18974704073471832 —— 只减一边的签名
  M3 去掉写入侧 #ensureCentroid                    → 仅 C4d 红(5→4 pass)
     C4d 红原文:after commits through the production path a centroid must exist
     这一档证明 C4d 是唯一守住第一层的判据

一处必要适配(不是削弱)

  test/phase-b-h.test.js 的 `ticket②-strong mutation` 原本依赖「commit 之后
  本空间没有中心」来让「用错 key 查不到」成为可观测事实。C4 让 commit 自动
  建中心之后,这个前提变了,它会因为「本来就有一个中心」而红 —— 红的原因
  与它要守的约束无关。改法:commit 之后显式清掉本空间那一行,并加一条
  前置断言。用例语义完全保留,只是把前提重新立起来。

★ What this does NOT establish

  中心化改变的是向量间的夹角,因此能改变排序。它不能让未校准的绝对分数
  变得可用:去均值之后余弦仍然是余弦,而 DEFAULT_MIN_SCORE = 0.5 从来不是
  从中心化后的分布推出来的。阈值校准仍是
  phase-b-p1-vector-wiring.test.js:51-58 记录的已知未解问题。

三门:npm test 277/277(0 skipped)、npm run check 12/12、npm pack 10 files 79.0 kB。
冻结面:docs/、README.md、scripts/、lib/loop* 零改动。
2026-11-05 10:41:38 +08:00
28e18d0258 test(memory): bound what P1's green does and does not establish (P1 follow-up)
Two discipline holes left after 1309afe / 09be4de, both closed here.

1. The boundary was unstated. A one-hot keyword-family fixture proves the
   PLUMBING — that the computed query vector reaches scoring — and says nothing
   about retrieval quality. Left implicit, a green P1 invites the reading
   "semantic recall works". The measured counter-evidence is now in the file:
   on the real chinese-clip space a true hit scored 0.8345 while an unrelated
   block scored 0.8598, so absolute scores are not calibrated and no threshold
   may be read off them. That is a known unresolved problem outside P1's scope.

2. `dsh` missing used to `t.skip`. A skip keeps `npm test` green while the only
   end-to-end evidence for P1 never runs — the same shape as a criterion that is
   committed but never speaks. It now fails, and says why.

No behavioural change: 272/272 tests, 12/12 checkpoints, 0 skipped on a box
where `dsh` is present. Verified the mutation discipline still bites — deleting
`'hybrid'` from ORDINARY_DEGRADATIONS turns L10, L11, L10† and L11† red.
2026-11-05 01:22:57 +08:00
09be4de0eb test(memory): pin that ORDINARY_DEGRADATIONS decides the log LEVEL (P1 follow-up)
P1 registered two healthy modes in `ORDINARY_DEGRADATIONS` — `hybrid` (the wired
success path) and `no_vector_space` (a space with nothing comparable to score) —
and the commit message claimed that meant "a healthy wired recall does not log a
warning every turn". Nothing tested it. Measured on 1309afe:

    M-a  delete 'hybrid'            → # tests 268  # pass 268  # fail 0
    M-b  delete 'no_vector_space'   → # tests 268  # pass 268  # fail 0
    M-ab delete both                → # tests 268  # pass 268  # fail 0

while the behaviour really broke (real apply(), debugLog=false):

    mutated hybrid recall → lines=[['warn', '... degraded=hybrid ... inject=138ch']]
    released hybrid recall → lines=[]

A false fault alarm on the happiest path, every single turn, and the suite green.

Why the existing tests could not see it: A3d/D3b assert the LABEL
(`hits.degraded === 'hybrid'`). Removing a member from the Set does not change any
label — it changes only whether `logRecallTurn` treats that label as a fault. The
defect lived exactly in the gap between "the label is right" and "the level is
right", and `grep ORDINARY_DEGRADATIONS test/` was empty. A claim in a commit
message with nothing behind it is the same failure mode as a guard that cannot
fail.

What this commit adds (test/memory-log.test.js only — lib/ untouched)

  L10    one group, three outcomes at the SAME verbosity (debugLog: false):
         `hybrid`               → zero lines
         `no_vector_space`      → zero lines
         `embed_error`          → exactly one line, type 'warn'   ← the control
         Each case asserts its own PRECONDITION first (the run really was
         `hybrid` / `no_vector_space`, and something was really injected), so a
         silent log cannot be confused with a build that never reached the route.
         Without the third case, "silent" would also be satisfied by a build that
         logs nothing at all — which is the original bug the log channel exists
         for (L4).
  L10†   real mutation: 'hybrid' removed from the Set → L10.1 red
  L11†   real mutation: 'no_vector_space' removed → L10.2 red
         Both via `withMutatedIndex` (copy the package, rewrite ONE anchor,
         `count === 1` self-check, import fresh). The mutant runs the SAME
         `assertLevelContract` body the released build passes — a criterion copied
         for the mutant can drift, a shared one cannot. The mutation then asserts
         the throw is `ERR_ASSERTION` AND that the message starts with the specific
         sub-assertion name, so an import failure cannot make it green for the
         wrong reason.
  L11    measured behaviour, pinned so nobody "fixes" it later: a fused recall
         whose candidates were ALL refused by `minScore` still reports `hybrid`
         with zero hits, and is still silent. `no_vector_space` is returned at
         exactly one place (`recall.js`, `vectorScores.size === 0`) and means
         "there was nothing to score" — never "everything was filtered". Collapsing
         the two would hide a threshold set too high behind a "fresh space" story.

Two fixture pitfalls found while writing this, both recorded in the test because
each one silently disarmed its own case:

  · `commit` embeds when an embedder is mounted, so the `no_vector_space` case has
    to write its block BEFORE the embedder appears. The first attempt mounted the
    embedder first, the space filled up, and the label came back `hybrid` — the
    assertion then failed for its own precondition rather than for the subject.
  · `StableMem.recall()` forwards only `topK`/`signal`, so a `minScore` passed
    there never reaches the scorer and L11's precondition would have passed on an
    unchanged result. It drives `recallBlocksHybrid` directly instead.

Not asserted, deliberately: `'no_embedder'`, `'aborted'` and `'vector_ready_unused'`
are also in the Set, but their silence is already carried by L1/L5 (no embedder)
and W11e/W12 elsewhere. Adding them here would inflate the group without covering a
new decision.

旁证 (not acceptance): npm test 272/272, npm run check 12/12, npm pack --dry-run
10 files / 75.3 kB. docs/ and README.md untouched.
2026-11-05 00:51:17 +08:00
1309afec93 fix(memory): wire the computed query vector into recall scoring (P1)
`recallBlocksHybrid` computed `queryVector`, verified its length against the
declared space, and then threw it away:

    return withDiagnosis(recallBlocks(store, query, scoped), 'vector_ready_unused')

`scoped` carries only `topK` / `signal` / `fingerprint` / `dimension`. So every
V1–V7 scoring branch inside `recallBlocks` was live code reachable ONLY from
unit tests that hand-built a `vectorScores` Map — `vectorScores:` appeared 10
times in test/phase-b-a.test.js and ZERO times under lib/. Production recall was
pure symbol/FTS: the whole vector half of phase B was unreachable, and the only
symptom was a diagnostic label nobody was watching.

Why BOTH options are now passed (structural, not stylistic)

  · `vectorScores` alone reaches only the V1 exact-band tie adjustment (:438).
    The fused candidate assembly at :233 is gated on `Number.isFinite(minScore)`,
    so without a threshold the vector route can never admit a block the symbol
    path missed — which is exactly what V5 exists for.
  · `minScore` alone does nothing: :299 additionally requires `vectorScores` to
    BE a Map before it builds vector candidates.
One seam, three guards. Half of it is a no-op, which is why measuring the
half-wiring mutation below matters.

Scheme A, and why not B

A: keep the existing API (`vectorScores: Map<string, number>` + `minScore`), and
   have `recallBlocksHybrid` score the query vector against the space's stored
   vectors before calling `recallBlocks` once.
B: change `recallBlocks`'s signature to accept a `queryVector` and score inside.

Chosen: A. B would touch a signature that phase A's golden fixture and nine call
sites depend on being synchronous and vector-free (`recallBlocks` MUST NOT return
a thenable — A1 pins it structurally), and it would move cosine arithmetic into a
function whose contract is literally "store + query text". A leaves
`recallBlocks` byte-untouched: the seam it already exposes is driven, so A2's
45-case golden comparison cannot drift. Both routes must also be fused in ONE
scoring pass anyway (V5 admits vector-only candidates; V7 forbids truncating
before fusion), so "call twice and merge" was never an option.

Cost, measured before choosing (this box, 2000 blocks x 512 dims)

  narrow `SELECT id, vector`    21 ms   (vs 70 ms for `allBlocks(-1)`)
  parse + cosine over 2000      60 ms
  ⇒ ~81 ms per recall; at 8000 blocks ~310 ms.

Affordable because it runs once per TURN (pre-step), not per candidate, and
because the scan is bounded by SQL: `fingerprint = ?` and `vector_dim = ?` are
pushed into the store so rows from a retired space or a wrong width are never
decoded. A phase-A store (no embedded blocks at all) pays one indexed SELECT
returning no rows. It does not add a new order of cost either: the fused path
already walks every block for symbol union (`allBlocks(-1)`); this replaces that
blind decode with a narrow one for the vector half. The remaining ceiling is a
single very large space — a number to measure before raising topK, not a reason
to ship a path that never runs. Fail-closed per §0.2/F5: blocks with an empty
fingerprint are deliberately not scored.

New diagnostic modes, both non-faults

  `hybrid`           the healthy wired path (symbol + vector fused). Explicit
                     `case` in `degradationNotice` because `default:` exists to
                     make an UNKNOWN mode say "unknown, not absent" — routing the
                     healthiest mode through it would print a false degradation
                     notice on every turn that had a vector hit.
  `no_vector_space`  a space is configured and the embedder answered, but no
                     block in it carries a usable vector (fresh space). Says
                     "nothing was scored" without claiming a provider failure.
                     An empty Map must NOT be passed to the fused route: it would
                     open candidate assembly and score symbol matches as
                     `0.7*0 + 0.3*coverage`, the V2 regression.

The criterion (test/phase-b-p1-vector-wiring.test.js)

End-to-end, because a unit test cannot see this bug: the defect is that nobody
passes the options, so asserting the options work proves nothing. The chain is
real sqlite file → real `apply()`/`commit()` write path (no hand-built vectors,
no injected embedder object: the plugin's own HTTP adapter is used) → real
`recallBlocksHybrid` → real headless `dsh --profile headless` with the shipped
`cordis.patch.yml` loop overlay (so P0 still applies) → the persisted
`session.v4.jsonl.zstd` → the stub LLM's answer, which is a pure function of the
system prompt it received.

Corpus, MEASURED before any assertion was written. Deterministic keyword-family
projection into 8 unit axes: family 0 is the target's vocabulary, families 1..7
are one axis per distractor, unmapped text → zero vector. Query
「上次说的那个四位数是什么来着?」:

  symbol path hits (target)   0     ← precondition 1
  hand-built vectorScores top1 = target, score 0.70   ← precondition 2
  query shares no CJK 2-gram with any block

Both preconditions are asserted in-suite (P1-0) so a later corpus edit cannot
make the e2e criterion unfalsifiable. An earlier attempt used the real
chinese-clip-vit-b16 model; its top-1 for the same kind of query was an unrelated
block (0.83 "入职培训"), so that corpus could never have gone green — the fake
embedder makes the question decidable instead of probabilistic.

The access code is `QP47-812` rather than a bare number so the stub's extractor
(`[A-Z]{2}\d{2}-\d{3}`) cannot collide with anything else in the prompt.

Born red against c1f7e06 (clean `git archive` of that commit, same test file):

  P1: the answer must come from block memory through the vector route.
    expected: QP47-812
    actual:   "未知"
    symbol-path hits: 0 (must be 0)
    recall diagnosis: degraded=vector_ready_unused hits=0
    recall section injected: (none)

Mutations — three, each a real edit to lib/recall.js, each restored after:

  M1 drop the wiring      `recallBlocks(store, query, scoped)`  → P1 red
  M2 half the seam        `minScore = undefined`                → P1 red
  M3 empty the scan       `const rows = []`                     → A3d red

  M1 raw: not ok 2 - P1: a semantic-only question is answered from block
          memory via the vector route
            expected: QP47-812
            actual:   "未知"
            recall diagnosis: degraded=hybrid hits=0
  M2 raw: identical text — and that is the point. Passing `vectorScores` alone
          keeps the new `hybrid` label looking healthy while the answer is still
          empty, because `minScore` is what opens the fused candidate assembly.
  M3 raw: Expected values to be strictly equal — `hybrid` unreachable, so the
          label is evidence rather than decoration.

Restoration is verified by hash, not by eye: `sha256sum lib/recall.js` is
8eb0faa660c973ea5114268001d49cf43f83ef92b2945897cfc6f7512541a498 before and
after every mutation, and `git diff` (staged vs worktree) is empty.

Test updates forced by the wiring (not cosmetic)

  A3c   the old assertion was "a healthy embedder reports vector_ready_unused,
        NOT hybrid". That was correct while the vector was unused; over a corpus
        with no vectors in the active space the honest label is now
        `no_vector_space`, and A3d pins `hybrid` once a vector exists.
  A3c†  replaced a test that only compared two local string literals (it could
        never fail for a real reason) with a mutation of the actual scoring pass.
  D3b   `hybrid` instead of the group-A stage label.
  index the two new modes are added to ORDINARY_DEGRADATIONS, so a healthy wired
        recall does not log a warning every turn.

Not in this slice: C4 (centering) is the next layer on this exact path and
deliberately not done inline, so its mutation stays citable. V6 and C5 untouched.

旁证 (not acceptance): npm test 268/268, npm run check 12/12.
2026-11-05 00:07:29 +08:00
c1f7e06b22 test(memory): 补 377e02c 的判据假绿 —— multiCarrier 名不符实,真多块零覆盖
`multiCarrier(texts, convention)` 名叫"多块",实际 `content: [{ type: 'text',
text: joined }]` 恒为 1 个 block。所以 S2a/S2b 读起来是"多块路径已覆盖",
实际一条都没走多块分支:

  M1 在 `const rendered = ...` 前插 `if (blocks.length > 1) return null`
     (即 V2 那个错误实现)→ 22 条判据 pass 20 / fail 0,全绿。

这正是上一轮 375→377e02c 留下的假绿:不是缺测试,是 helper 名字把人骗了。
名字夸大覆盖范围比没有测试更毒,因为它读起来像有。

改法(只动 test,lib/loop.js 零改动,sha256 f28c5564… 前后一致):

- `multiCarrier` → `singleBlockCarrier`,实现不动,JSDoc 写明
  "产出:1 个 text block 承载 N 个 section(lib/index.js:932 的真实形状)"
  并记下它曾叫 multiCarrier 这段历史。
- 新增 `trueMultiCarrier(texts, kind)`:N 个 text block,每个 block 一个
  section,rendered = texts.join('')(glued)。注释写明真多块没有 spaced 变体。
- S2a/S2b → singleBlockCarrier;S3 标题改成 "single content block carrying N
  sections != those sections …" 并改用 singleBlockCarrier。
- S2c/S2d → trueMultiCarrier(['a','b']),断言不变。

变异自证(两条都咬住,且撤净后 sha256 回到基线):
  M1 `if (blocks.length > 1) return null`   → S2c 红("got 1 message(s)")
  M2 删掉 glued 分支(只剩 `rendered !== spaced`)→ S2b 红("got 1")

对 5c6aaf5 的干净 archive 跑本判据:tests 22 / pass 10 / fail 12,
其中 S2c("got 1 message(s)")与 S2d("got 2")如实为红。
2026-11-04 18:32:16 +08:00
377e02c943 fix(memory): make the loop actually promote real recall into system
`promoteSnapshotSections` accepted only ONE carrier spelling: `content: []`.
The plugin never emits that shape. `lib/index.js:932` always attaches exactly
one `{ type: 'text', text: injected }` block whose text IS `sections[0].text`,
so `promotedSection`'s `if (blocks.length > 0) return null` matched on every
real recall and the promotion was a no-op in production. Recall stayed in
user/message history — the exact defect 6d9ad62 claimed to fix.

6d9ad62's "surface recall as a system section instead of history" was never
true: the promotion it relies on was blocked by that content-non-empty check.
The unit tests could not see it because their `snapshotMessage()` helper built
`content: []` — a shape no producer emits — and the old P4 asserted "real
content blocks are NOT promoted", which is the opposite of reality.

It was confirmed today, from `lib/index.js:932`'s constant one-text-block
injection, and is fixed here.

The rule is now "move the text, don't rewrite it": a snapshot carrier with
content is promoted only when the rendered blocks equal the sections' text
under one of the two canonical joins (`''` glued, `'\n\n'` spaced), and the
promoted section text is the rendered content, so relocation is assertable.
Non-text blocks, mismatched text, and the stock runtime-context snapshot (whose
content carries a preamble the sections do not) pass through untouched.

Tests: S1-S7 plus real multi-block S2c/S2d pin the acceptance surface; every
negative case carries a positive control in the same decision, so the old
"content non-empty => reject" rule cannot satisfy it. P4 is split into P4a/b
(content == sections promotes; content != sections does not) with P4c proving
the split is decided by text, not by presence of content.
`test/loop-system-e2e.test.js` mounts the real HologramAgentLoop on real dsh
services, runs one turn, and asserts on the request the adapter received: the
recall is in the system prompt (E1) and in no non-system message body (E2),
carried by exactly one message (E2b).
2026-11-04 18:24:58 +08:00
5c6aaf53cf feat(memory): land the loop ports without wiring them
The four files under lib/loop/ were written against 0.2.1-rc.2 and have been
verified to import on their own; they are the substrate a replacement loop
needs to take over step(). This commit only puts them in the tree — lib/loop.js
is untouched, so nothing imports them yet and behaviour is unchanged.

Two packaging facts they bring with them:

  files  `lib/*.js` does not recurse, so lib/loop/ was shipping nothing.
         `lib/loop.js` and `lib/loop/*.js` are now listed separately because
         neither glob covers the other.
  peer   runtime-context.js imports @deepseek-ai/dsh-session, which the peer
         table never declared. G2b's allowlist covered it, so the static guard
         passed over a real declaration gap.

test/loop-ports.test.js pins the port interface rather than its behaviour: the
modules import alone, the export names survive, and the inbox projection keeps
key "inbox" / stateVersion 1. That literal pair is a wire format shared with
stock-loop sessions — a different key silently reads as "no pending input", a
different version makes stateSchema.parse refuse stored rows. The schema's
parse() is also exercised against seven malformed persisted rows, since
replacing a zod schema with a hand-written check is exactly the kind of change
that can be a no-op and still look fine.
2026-11-04 17:27:36 +08:00
f1b74a35ef feat(memory): ship the profile overlay that mounts the replacement loop
6d9ad62 added `lib/loop.js` — a subclass of the harness `AgentLoop` that moves
per-turn recall out of conversation history and into the system prompt. That
class was never instantiated by anything: the commit proved the pure
`promoteSnapshotSections` function, not that a replacement loop can actually be
mounted. This commit adds the two pieces the mount needs.

`cordis.patch.yml` is the overlay that does it, and it is the only artefact that
proves the swap is possible:

- `- id: agent-loop / disabled: true` is mandatory and is addressed by ID, not
  by name. `AgentLoop`'s constructor ends with
  `ctx.effect(() => ctx.agents.setFactory(this))`
  (dsh-agent-loop/lib/index.js:1590) and `setFactory` throws when a factory is
  already registered (dsh-agent/lib/index.js:429-437). Leaving the base row
  enabled fails startup with "an agent factory is already registered".
- The inserted row reuses the package itself under the `dsh-memory/loop`
  subpath, with no `config` block: `HologramAgentLoop` declares no `static
  Config`, so it inherits the parent schema and the profile's own `agent-loop`
  config rows keep working through the inherited constructor.

`package.json` gains that subpath export plus `lib/loop.js` in `files`. Without
the export the overlay resolves to nothing and the profile cannot boot — the
two changes only work together, which is why they land in one commit.

Measured on this machine with the overlay applied to a headless profile
(`~/.dsh/profiles/headless/cordis.patch.yml`), dsh 0.2.1-rc.2:

- `dsh --profile headless "回答两个字:收到了"` -> exit 0, stdout `收到了`;
  `--dump-config` shows `- id: agent-loop` with `disabled: true` alongside
  `- id: hologram-agent-loop, name: dsh-memory/loop`.
- A tool round trip returns the tool's output: asking for
  `echo TOOL_OK_42` via bash yields `TOOL_OK_42`.
- SIGINT during a long task exits in 1s with code 130, stdout empty.
- The web profile's patch layer is still `[]` — untouched.

npm test: 247 pass, 0 fail.
2026-11-04 17:07:48 +08:00
6d9ad62a8f feat(memory): surface recall as a system section instead of history
Per-turn recall currently enters through the `agent/pre-step` waterfall, and
that hook can only return `messages`. The recall block therefore lands in
conversation history as a user message: the model reads its own memory as if a
user had said it. `systemPrompt.context` cannot fix this — it runs before the
turn text is known, so it can only render input-independent text.

The loop's `preStep` is the first seam holding BOTH the assembled prompt and
this turn's claimed messages. `lib/loop.js` subclasses the harness AgentLoop and
wraps that seam on the agent instance: a pure snapshot carrier message is moved
out of `messages` and into `assembly.sections`, so it becomes part of the system
prompt for that turn and never enters history.

Deliberately decoupled from the memory plugin: nothing imports it or matches its
name. Promotion keys off the shape every snapshot-style injector already uses
(`source.sections`, the same shape `dsh-time-context` emits).

Constraints honoured, each of which was a measured failure elsewhere:
- The promoted section carries `interpolate: false`; recalled text is free text
  and may contain `{{...}}`, which the strict interpolator rejects.
- The subclass defines no constructor and no static `inject`. The parent already
  registers the turnBoundary + inbox projections, calls `setFactory(this)`, and
  declares `{{provider}}`/`{{model}}`; re-registering any of them throws at
  startup.
- Only the instance's `preStep` is patched, never the prototype, which other
  agents share.

Also wires the new peer: `scripts/link-dsh.sh` gains dsh-agent/dsh-agent-loop
(without them the package cannot resolve at test time), package.json declares
dsh-agent-loop as a peer and adds the new test file, and G2b's allowlist admits
dsh-agent-loop as a first-party harness package, with the reason recorded.

npm test: 247 pass, 0 fail.
2026-11-04 14:54:49 +08:00
783a7f87bb feat(memory): observable recall log behind config.debugLog
The plugin shipped with zero console.* and zero ctx.logger calls, so a
production recall failure left no trace: `degraded` is a return value the
caller sees once, and the injected text is only visible if the model quotes
it. Everything below is observability — no scoring, no DDL, no docs/README,
no new dependencies.

## The line

One summary per recall turn, covering the four fact groups asked for:

  recall   query=<n>ch hits=<n> topK=<n> zeroHit=yes|no
  budget   budget=<n> source=explicit|host|fallback kept=<n> singleHitOverBudget=yes|no
  degrade  degraded=<mode> skipped=<n> skippedDim=<n> err=<summary>
  inject   inject=<n>ch

`query` is LENGTH ONLY (code points, so CJK is not double-counted). The query
is the user's own message and a memory log is the last place it should be
copied; length beside `hits` is enough to tell "searched nothing" from
"searched the skill catalog".

`source` reads straight off segmentRecallBudget's own decision, and `kept` is
the post-slice count, so the line cannot disagree with what was injected.
`inject` is measured AFTER rendering, i.e. the exact string contextForStep
returns — not a re-derived estimate.

## Level policy (and why it is not always info)

  · fault — degraded not in {ok,no_embedder,aborted,vector_ready_unused}
    -> warn, emitted EVEN WITH debugLog OFF. A failed search is the per-turn
    event where absence of evidence is not evidence of absence.
  · zero injection -> debug (the commonest case on the shipping default must
    cost nothing at normal verbosity, but still be one greppable line).
  · otherwise -> info, only when debugLog is true.

`no_embedder` and `aborted` are deliberately NOT faults: the former is the
shipping configuration, the latter a deliberate stop. This is the same
judgement W11e already made for the model-facing notice — a constant warning
trains the reader to skip the channel.

## Logger source, and why ctx.logger alone was not enough

Source is ctx.logger('dsh-memory'); if the cordis build has no logger service
the code falls back to a console.warn/console.error shim (debug/info also go
to warn — stdout is never written, since `dsh --profile headless` prints the
ANSWER on stdout, so a line there is not noise but a corrupted answer).

★ Measured, and the reason this change installs its own sink: cordis ships
exactly ONE default exporter, an in-memory ring buffer
(cordis/lib/index.js:594-603), and no dsh bundle adds a terminal one. Calling
ctx.logger.info(...) in a real profile therefore writes to a buffer nobody
drains — the plugin would have looked instrumented while a headless run still
printed nothing. installStderrSink registers a scoped exporter on that same
service: it filters by message.name (the exporters map is process-wide, so
without the name check it would reprint every other plugin's warnings under
this prefix) and by severity, then renders with Logger.format and writes to
stderr. Disposal rides on ctx.effect inside LoggerService.exporter.

Two robustness details, both earned the hard way:
  · the whole line travels as ONE `%s` argument — Logger.format expands
    %s/%d/%o in the format string, so a provider error containing `%s` would
    otherwise truncate the rest of the line (verified: "50% off %s" came out
    "50% off undefined NaN");
  · err is flattened to one line and cut at 80 chars — a log line must stay a
    line;
  · every logging path is wrapped, because a throwing logger would turn an
    observability feature into a recall outage.

## Verification

test/memory-log.test.js (L1–L8b, 12 tests) pins: default is silent; the
summary carries all four fact groups in exactly ONE line; the query text
never appears; explicit/host/fallback source naming; a fault warns with
debugLog off; a zero-injection turn is debug not info; log-on equals log-off
for both the injected text and lastRecall; stdout is never written; the line
reaches stderr; foreign loggers are not reprinted. L4† is a real mutation —
it patches lib/index.js on a copy and asserts L4 goes red — because a guard
that cannot fail is not a guard (the W11d† lesson).

Self-tested in a real headless profile (dsh 0.2.1-rc.2, llmsproxy), same
question, only debugLog differing. Answer stayed "4471" on stdout in both:

  debugLog=true   dsh-memory: info: recall query=21ch hits=1 topK=3 zeroHit=no
                  | budget=838860 source=host kept=1 singleHitOverBudget=no
                  | degraded=no_embedder skipped=0 skippedDim=0 err=-
                  | inject=144ch
  debugLog=false  (no dsh-memory line; stderr carry only node's
                  ExperimentalWarning)
  zero-hit turn   dsh-memory: debug: recall ... zeroHit=yes ... inject=0ch
  fault, OFF      dsh-memory: warn: recall ... degraded=embed_error
                  err=fetch failed | inject=290ch   <- speaks without opt-in

npm test: 237 tests, 237 pass, 0 fail (the W11d†ctx failure is gone — see
below). npm run check: 12/12 checkpoints.

## One test edit, forced by the caller change

W11d†ctx's mutation anchor was `const hits = await this.recall(query, { signal })`,
which no longer exists: contextForStep now rebinds as `const raw` because the
hits it renders come out of segmentRecallBudget (the preceding commit's work,
not this one). The anchor moves to the `raw` binding it would sit beside. The
mutation still flips W11d; only the anchor text changed.
2026-11-04 08:15:23 +08:00
ca0b5ff813 fix(memory-b): keep non-enumerable recall diagnostics through budget slicing 2026-11-03 21:35:59 +08:00
2069cd5f25 feat(memory-b): budget-segment recall injection in the real host path 2026-11-03 21:31:36 +08:00
771a053eea docs(CL-1F): restore A1 semantics and cite V7 logs for G1
A1: revert to the 06155d4 wording (recallBlocks stays synchronous, a new
async recallBlocksHybrid is added). The previous basis claimed the async
pre-step handler justified making recallBlocks async, which the code
contradicts: lib/recall.js:229 exports a sync recallBlocks with zero awaits,
and lib/recall.js:613 is the async wrapper.

G1: replace the stale count with the readings already present in the V7
evidence logs (/tmp/v7-order-20261101-173334-t1..t3.log = npm test,
225/225/0; c1..c3.log = npm run check, 12/12), noting that phase A is a
subset of the whole-suite run. No new test/check run was performed.
2026-11-01 17:52:52 +08:00
7fc3d46b9a docs(CL-1): revert V2 documentation changes 2026-11-01 17:41:52 +08:00
591f34070d feat(memory-b): port V7 largeTopK candidate bound before fusion 2026-11-01 17:37:48 +08:00
06155d4bcc feat(memory-b): expose text document side as interface only 2026-11-01 17:21:46 +08:00
8a87c973d2 test(memory-b): add self-built tier-3 flip case 2026-11-01 17:08:54 +08:00
e2536f14f3 feat(memory-b): port V5 candidate assembly and asymmetric MinScore 2026-11-01 15:50:17 +08:00
5b217c1f78 feat(memory-b): expose V3 vector and symbol attribution 2026-11-01 15:34:55 +08:00
b7eff5db66 test(memory-b): strengthen C3 recall and corruption mutants 2026-11-01 12:39:53 +08:00
d73d47c081 feat(memory-b): port C3 centroid staleness and async rebuild scheduling 2026-11-01 12:32:16 +08:00
27 changed files with 7816 additions and 72 deletions

View File

@ -250,7 +250,7 @@ recallBlocksHybrid(store, embedder, query, opts) // async wrapper
```
Why the split rather than "make recallBlocks async": every existing caller keeps
calling a sync function, so the existing test call sites and the whole phase-A surface
calling a sync function, so the 9 test call sites and the whole phase-A surface
need no edit, and the golden fixture proves the sync path did not move.
With no embedder (or a failing one) the wrapper is exactly `recallBlocks`, and it

29
cordis.patch.yml Normal file
View File

@ -0,0 +1,29 @@
# Ship-ready overlay that swaps the harness loop for `dsh-memory/loop`.
#
# This is the ONLY artefact that proves the replacement loop is mounted: without
# it `lib/loop.js` is a subclass nobody instantiates (its unit tests exercise the
# pure `promoteSnapshotSections` function and nothing else).
#
# Apply it to a profile (a `--patch` overlay, or copied into the profile's own
# `cordis.patch.yml`); `dsh` composes it over `@deepseek-ai/dsh-base`, where the
# `agent-loop` row already exists:
#
# dsh --profile headless --patch <this file> "<task>"
#
# ★ The disable is not optional, and it is by ID, not by name. `AgentLoop`'s
# constructor ends with `ctx.effect(() => ctx.agents.setFactory(this))`
# (dsh-agent-loop/lib/index.js:1590), and `dsh-agent`'s `setFactory` throws when a
# factory is already registered (dsh-agent/lib/index.js:429-437). Leaving the base
# row enabled makes startup fail with "an agent factory is already registered";
# `HologramAgentLoop` inherits that constructor, so it registers exactly once —
# but only if the base row is gone.
- id: agent-loop
disabled: true
# The replacement. No `config` block on purpose: `HologramAgentLoop` defines no
# `static Config`, so it inherits the parent schema (`maxParallelToolCalls`,
# `agents`) and the profile's own `agent-loop` config rows keep working through
# the inherited constructor.
- insert:
- id: hologram-agent-loop
name: dsh-memory/loop

View File

@ -143,7 +143,7 @@ return ""
| # | 断言 | 依据 |
|---|---|---|
| A1 | `recallBlocks` 保持同步,另新增异步 `recallBlocksHybrid`;phase A 的 26 个测试与 8 个 checkpoint **全部仍然通过**,一条不改语义 | 现有调用方保持同步,异步只出现在新 wrapper |
| A1 | `recallBlocks` 保持同步,另新增异步 `recallBlocksHybrid`;phase A 的 26 个测试与 8 个 checkpoint **全部仍然通过**,一条不改语义 | 代码可核:`lib/recall.js:229` 是同步导出的 `recallBlocks`(函数体内 `await` 0 处),`lib/recall.js:613` 是 `async` 的 `recallBlocksHybrid` —— 异步只出现在新 wrapper |
| A2 | 未配置 embedder 时,`recallBlocks` 的返回与 phase A **逐字节相同**(同序、同分、同 `why`) | 无向量是合法配置,不是降级 |
**A1 变异**:把 `recallBlocks` 改回同步并在内部 await 一个 Promise ⇒ 测试红(语法/类型层面即失败)。
@ -319,7 +319,7 @@ phase B 必须把向量放进 **[0..1) 带**,不得越到 exact 带。
| # | 断言 |
|---|---|
| G1 | phase A 的 173 tests + 12 checkpoints **全绿**,语义不变(A1/A2 已覆盖) |
| G1 | 整仓 `npm test` **225/225**、`npm run check` **12/12** 全绿(phase A 为其中的子集),语义不变(A1/A2 已覆盖) | 读数取自既有 V7 取证日志:`/tmp/v7-order-20261101-173334-t1..t3.log`(命令 `npm test`)各含 `# tests 225` / `# pass 225` / `# fail 0`;`/tmp/v7-order-20261101-173334-c1..c3.log`(命令 `npm run check`)各含 `12/12 checkpoints passed`。这些日志采集于提交 `591f340`,本提交不改数字口径 |
| G2 | 未配置 embedder ⇒ 进程内无任何 HTTP 出站、无新依赖被加载 |
| G3 | `stablemem_commit` 在 embedder 不可用时**仍能写入**(块入库、vector 留空,`no_vector` 计数 +1),不报错 |
| G4 | 不引入 media/媒体块(`docs/feasibility.md` §5.4 的决定不变) |

View File

@ -118,6 +118,15 @@ export function deriveFingerprint({ endpoint, model = '', dimension, fingerprint
* @param {string} [config.apiKey]
* @param {string} [config.fingerprint] override
* @param {number} [config.timeoutMs]
* @param {'query'|'document'} [config.side] prompt side for TEXT embedding
* (default `query`).
*
* ★ V6 GAP — TEXT + `document` is INTERFACE-ONLY.
* Go assigns `Side: "query"` for text (`internal/memory/vector/http_embedder.go:105`)
* and `Side: "document"` only for images (`:109`). It never sends text with
* `document`, so there is no kernel precedent for what a text-document vector
* means, and no criterion here claims one. This option pins the WIRE SHAPE the
* adapter can produce; the SEMANTICS are deliberately unasserted.
* @returns {{ embedText: (text: string, signal?: AbortSignal) => Promise<number[]>,
* info: () => {fingerprint: string, dimension: number},
* close: () => Promise<void> }}
@ -130,8 +139,15 @@ export function createHttpEmbedder(config) {
apiKey = '',
fingerprint,
timeoutMs = DEFAULT_TIMEOUT_MS,
side = 'query',
} = config ?? {}
// Only the two sides Go defines are accepted (`pkg/embedding/embedding.go:34-35`).
// Anything else would be an invented protocol value the provider cannot map.
if (side !== 'query' && side !== 'document') {
throw new Error(`dsh-memory: embed side must be 'query' or 'document', got ${String(side)}`)
}
if (typeof endpoint !== 'string' || endpoint.trim() === '') {
throw new Error('dsh-memory: http embedder requires an endpoint')
}
@ -178,7 +194,9 @@ export function createHttpEmbedder(config) {
body: JSON.stringify({
model,
modality: 'text',
side: 'query',
// go: http_embedder.go:105 (`query`) / :109 (`document`, image only).
// The value travels verbatim; V6 pins the shape, not the semantics.
side,
text: String(text ?? ''),
}),
signal: fused,

View File

@ -29,11 +29,11 @@
* @module dsh-memory
*/
import { Service } from '@deepseek-ai/cordis'
import { Logger, Service } from '@deepseek-ai/cordis'
import { createUserMessage } from '@deepseek-ai/dsh-llm'
import { defineTool } from '@deepseek-ai/dsh-tools'
import { createHttpEmbedder } from './embed/http.js'
import { recallBlocksHybrid, renderRecall } from './recall.js'
import { recallBlocksHybrid, renderRecall, segmentRecallBudget } from './recall.js'
import { BlockStore } from './store.js'
/** Cordis plugin name, also used as the `source.kind` of injected context. */
@ -70,8 +70,176 @@ export const inject = ['systemPrompt', 'tools']
* @property {number} [embed.timeoutMs] request timeout
* @property {object} [embedder] a pre-built Embedder, bypassing `embed`. Used by
* tests and by any adapter that is not the bundled HTTP one.
* @property {boolean} [debugLog] emit the per-turn recall summary (default
* false). Off means "only warn/error", which is the shipping default: an
* operator opts in when they are actually diagnosing recall. See
* {@link StableMem.logRecallTurn} for what the summary carries.
*/
/**
* Degradations that are NOT faults.
*
* `ok` is healthy. `hybrid` is the healthy path once vectors participate (P1),
* `no_embedder` is the SHIPPING DEFAULT (a standing configuration fact, the same
* judgement W11e made for the model-facing notice), `aborted` is a deliberate
* stop, `no_vector_space` means the space exists but holds no comparable vectors
* (a fresh space — a data state, not an incident), and `vector_ready_unused` is
* the group-A build stage label, kept so an older build's log line still reads.
* None of them justifies a warn on every turn; all of them are still worth a line
* when `debugLog` is on. Everything else — `embed_error`, and any mode a future
* branch invents — is a real fault and speaks even with `debugLog` off.
*/
const ORDINARY_DEGRADATIONS = new Set([
'ok', 'hybrid', 'no_embedder', 'aborted', 'no_vector_space', 'vector_ready_unused',
])
/**
* Logger for one plugin instance: `ctx.logger` when the cordis build has one,
* otherwise a `console.warn`/`console.error` shim.
*
* ★ stdout is NEVER written, in either path. `dsh --profile headless` prints the
* agent's final answer on stdout (`dsh-headless/lib/index.js:20`), so an info line
* there is not noise — it is a corrupted answer. The shim therefore routes
* `debug`/`info` to `console.warn` as well: stderr is the only sink this plugin
* may touch.
*
* ★ Why the facade alone is not enough (measured). cordis ships exactly ONE
* default exporter — an in-memory ring buffer (`LoggerService` constructor,
* `cordis/lib/index.js:594-603`) — and no dsh bundle adds a terminal one. So
* `ctx.logger.info(...)` in a real profile writes to a buffer nobody drains:
* the plugin would look instrumented while a headless run still printed
* nothing. {@link installStderrSink} is what makes the summarised line
* observable, and it is what the self-test in the commit message quotes.
*
* @param {import('@deepseek-ai/cordis').Context} ctx
* @param {string} label logger name
* @param {boolean} debugLog whether debug/info lines are wanted
* @returns {{debug: Function, info: Function, warn: Function, error: Function}}
*/
function resolveLogger(ctx, label, debugLog) {
try {
const service = ctx?.logger
if (typeof service === 'function') {
const logger = service(label)
if (logger !== null && typeof logger === 'object' && typeof logger.warn === 'function') {
installStderrSink(service, label, debugLog === true)
return logger
}
}
} catch {
// A context without a working logger service: fall through to the shim.
}
const write = (sink) => (format, ...rest) => {
// `%s` is how every call site below protects the payload from cordis's
// printf formatter (see `emit`); honour it here so the shim prints the same
// line the real logger would.
const text = String(format) === '%s' ? rest.join(' ') : [format, ...rest].join(' ')
sink(text)
}
return {
debug: write((text) => console.warn(text)),
info: write((text) => console.warn(text)),
warn: write((text) => console.warn(text)),
error: write((text) => console.error(text)),
}
}
/**
* Route this logger's records to stderr, filtered by severity.
*
* Two gates, deliberately in this order:
*
* 1. `message.name !== label` — the exporters map is process-wide (a child
* context sees the same Map, verified), so without this filter the sink
* would reprint every other plugin's warnings under this plugin's prefix;
* 2. severity — `warn`/`error` always pass, because a failed recall is a real
* event regardless of verbosity; `info`/`debug` pass only when `debugLog`
* asked for them.
*
* Disposal rides on `ctx.effect` inside `LoggerService.exporter`, so a plugin
* unload removes the sink with the rest of the fiber.
*
* @param {{exporter?: Function}} service the `ctx.logger` service
* @param {string} label logger name this instance owns
* @param {boolean} debugLog
* @returns {void}
*/
function installStderrSink(service, label, debugLog) {
if (typeof service.exporter !== 'function') return
// `levels` admits everything FOR OUR NAME so the severity decision is made
// here, in one place, instead of being split between the logger's gate and
// this filter. Other names fall back to cordis's default INFO gate and are
// then dropped by the name check below.
const levels = { [label]: 3 }
try {
/** The exporter object, referenced by its own `export` for `Logger.format`. */
const exporter = {
levels,
colors: false,
export: (message) => {
if (message?.name !== label) return
const ordinary = message.type === 'info' || message.type === 'debug'
if (ordinary && !debugLog) return
let line
try {
// Same renderer the TUI/terminal exporters use, so colours, `%s`
// expansion and the per-line length cap behave as every other dsh log.
// The exporter is passed (not `this`): `format` reads its `colors`,
// `formatters` and `maxLength`.
line = Logger.format(exporter, message)
} catch {
line = message.args.map((arg) => String(arg)).join(' ')
}
process.stderr.write(`${label}: ${message.type}: ${line}\n`)
},
}
service.exporter(exporter)
} catch {
// An exporter that cannot be installed must not take the plugin down; the
// ring buffer still holds the records for an attached exporter.
}
}
/**
* Emit exactly one line, keeping the payload out of the format string.
*
* `Logger.format()` in cordis expands `%s`/`%d`/`%o` in the FORMAT argument
* (cordis/lib/index.js:428-441). A provider error message that happens to
* contain `%s` would then swallow or print `undefined` for the rest of the
* line — a log you cannot read is the failure mode this whole change exists to
* remove. So the line always travels as the single `%s` argument.
*
* @param {{[k: string]: Function}} log
* @param {'debug'|'info'|'warn'|'error'} level
* @param {string} line
*/
function emit(log, level, line) {
log[level]('%s', line)
}
/** Code points, not UTF-16 units: a CJK/emoji query must not report double. */
function charCount(text) {
return [...String(text ?? '')].length
}
/**
* One-line digest of a degradation message, for the log only.
*
* Truncated and whitespace-flattened: the summary exists to tell an operator
* WHICH failure occurred (`read ECONNREFUSED`, `timed out`, `dim 3, want 768`),
* not to reprint a provider response body into a log line.
*
* @param {unknown} message
* @param {number} [limit]
* @returns {string} '-' when there is nothing to say
*/
function summarizeError(message, limit = 80) {
if (typeof message !== 'string') return '-'
const flat = message.replace(/\s+/g, ' ').trim()
if (flat === '') return '-'
return flat.length > limit ? `${flat.slice(0, limit)}...` : flat
}
/**
* The `stablemem` service: durable block storage plus recall.
*
@ -95,6 +263,17 @@ export class StableMem extends Service {
store
/** @type {number} */
topK
/**
* Whether the per-turn recall summary is emitted. Plain field, like every
* other one: cordis hands out a proxy and private fields do not survive it.
* @type {boolean}
*/
debugLog
/**
* Named logger (`ctx.logger(name)`), or the console shim.
* @type {{debug: Function, info: Function, warn: Function, error: Function}}
*/
log
/**
* Vector capability, or null. Phase A has none; group H supplies the adapter.
* Kept as a plain property (not `#private`) — cordis hands out a proxy, and
@ -117,6 +296,8 @@ export class StableMem extends Service {
* disagree — the failure mode D4 exists to prevent.
*
* @type {{mode: string, skipped_fp: number, skipped_dim: number,
* skipped_score: number, skipped_no_evidence: number,
* skipped_unscoreable: number, truncated: number,
* embed_error: string|null, at: number} | null}
*/
lastRecall = null
@ -126,15 +307,39 @@ export class StableMem extends Service {
* @param {{ store: BlockStore, topK?: number,
* embedder?: import('./recall.js').Embedder | null }} options
*/
constructor(ctx, { store, topK = 3, embedder = null, reportNoVectorAlways = false }) {
constructor(ctx, {
store, topK = 3, embedder = null, documentEmbedder = null,
reportNoVectorAlways = false, budgetTokens, debugLog = false,
}) {
super(ctx, 'stablemem')
this.store = store
this.topK = topK
this.embedder = embedder
// ★ V6 — a stored document is not a query.
//
// The HTTP protocol carries a `side` field (`query` or `document`) and HA
// distinguishes them (`PurposeQuery` / `PurposeDocument`). A provider that
// embeds the two asymmetrically (the usual case for retrieval models) needs
// the distinction or the stored vectors live in a different space from the
// query vectors — silently, since both are unit vectors of the right width.
//
// `documentEmbedder` is the write-side encoder. It defaults to `embedder`
// because an injected single-instance embedder (tests, a local adapter) has
// no two-sided story to tell — the fallback is the previous behaviour, not a
// second path.
this.documentEmbedder = documentEmbedder ?? embedder
// Observability only: nothing below reads `debugLog` except the logging
// path, so turning it on cannot change a score, a budget, or a return value.
this.debugLog = debugLog === true
this.log = resolveLogger(ctx, name, this.debugLog)
// Test/diagnostic hook: report the `noVector` clause even when no embedder is
// configured. Production leaves this false so a standing configuration fact
// stays silent (see renderUnavailableTail).
this.reportNoVectorAlways = reportNoVectorAlways
/** Explicit memory budget (tokens); wins over any host-derived window. */
this.budgetTokens = Number.isFinite(budgetTokens) && budgetTokens > 0 ? budgetTokens : undefined
/** Cached host/model context windows, keyed by provider/model. */
this.windowCache = new Map()
// Release the sqlite handle with the owning fiber, so a plugin unload does
// not leak the file lock (checkpoint V5).
ctx.effect(() => () => this.store.close(), 'dsh-memory.store')
@ -182,6 +387,9 @@ export class StableMem extends Service {
...input,
expectedDimension: this.space()?.dimension,
})
if (typeof input.fingerprint === 'string' && input.fingerprint !== '') {
this.#ensureCentroid(input.fingerprint)
}
return { ...written, embedded: true }
}
@ -199,7 +407,7 @@ export class StableMem extends Service {
// required by the tool contract (dsh-tools/lib/types/index.d.ts:120-122)
// and it bounds latency: without it the adapter would hold the call for up
// to its own timeout while the user has already cancelled the turn.
vector = await this.embedder.embedText(input.text, options.signal)
vector = await this.documentEmbedder.embedText(input.text, options.signal)
} catch {
// Cancelled, or the vector side is down: the block still lands with no
// vector (decision §0.4). An abort must not turn into "nothing was written".
@ -220,15 +428,91 @@ export class StableMem extends Service {
// ★ From the space object, never re-derived and never from input.
fingerprint: space.fingerprint,
})
// ★ C4 的另一半:中心必须存在,否则去均值永远不会发生。
this.#ensureCentroid(space.fingerprint)
return { ...written, embedded: true }
}
/**
* 确保一个向量空间有中心,没有就建一个。
*
* ★ 为什么这件事必须写在写入侧
* 在 C4 之前,`graph_centroid` 在生产上永远是空表:`saveSpaceCentroid` 只有
* 测试在调,`centroidStatus` / `centroidRecallState` 在 `lib/` 里没有调用者,
* `rebuildCentroid` 也只有测试在调。于是 `loadCentroid` 永远返回 undefined、
* `isAnomalous` 永远是 false、去均值一次也不会发生 —— 而判据会绿,
* 因为判据自己会先手工 `rebuildCentroid`。那是本轮反复抓的「代码写了没接线」。
*
* ★ 为什么不能放在召回侧
* `rebuildCentroid` 是 O(n·d)(实测 dim=512、2000 块 85.5ms),而召回每轮
* pre-step 都跑。C3-k 也明令 `countVectored` 不得进热路径,而 `centroidStatus`
* 第一行就是它。所以:召回侧只 `loadCentroid` 纯读,重建留在这里。
*
* ★ 失败必须吞掉
* 一次中心重建失败不能把写入变成失败。下一次 commit 会再试;在那之前
* 召回拿不到中心就走未校正路径,结果仍是对的,只是没校正。
* 这与 D1(向量侧故障不得让主路径抛错)是同一条纪律。
*
* ★ 只在「没有中心行」时建
* 有中心行就交给已有的 stale 判定去管(块数漂移超 20% 时重建),
* 不在这里每写一条就算一遍全空间。
*
* ★ 首次建是同步的,而且「那时块还少」是个不成立的前提
* 这套代码存在的理由恰恰是:生产库里已经有几千条带向量的块、
* `graph_centroid` 却是空表(W8 的原场景)。那种库升级后第一次 commit,
* 走的就是下面这条同步 `rebuildCentroid`:全空间 SELECT + 逐条 JSON.parse
* + 算均值,全部发生在写入路径上。
* 实测(512 维、2000 条带向量的块、空中心表):
*
* 升级后第一次 commit 105.1 ms
* 之后每条 commit 8.7-9.4 ms(loadCentroid 的 SELECT + centroidStatus 的 COUNT)
*
* 首次建随块数线性增长(8000 块约 420ms)。这里接受这个一次性停顿,
* 而不是把它挪到异步:挪走会让升级后最初几次召回拿不到中心而静默不校正,
* 而 n 很大时正是最需要校正的时候 —— 一次有界停顿比一段静默降级更容易被
* 发现。稳态那 ~9ms 是 C4 新增的写路径开销;C3-k 只禁它进召回热路径。
*
* @param {string} fingerprint
*/
#ensureCentroid(fingerprint) {
if (typeof fingerprint !== 'string' || fingerprint === '') return
try {
if (this.store.loadCentroid(fingerprint) === undefined) {
this.store.rebuildCentroid(fingerprint)
return
}
// ★ 已有中心时不能直接 return。
// `rebuildCentroid` 只算一次,之后块数会一直长;如果这里只守「没有就建」,
// 中心就永远停在第一桶快照上,越用越偏。交给 `centroidStatus`:它内部
// 已经写好块数漂移 > 20% 判 stale 与 10 分钟节流异步重建,是这套判据
// 唯一的正统入口 —— 而在 C4 之前它在 lib/ 里没有任何生产调用者。
//
// 放在 commit 之后而不是召回路径,正是因为它第一行是 `countVectored`:
// C3-k 明令该查询不得进每轮 pre-step 的热路径。
const status = this.store.centroidStatus(fingerprint)
// ★ The recall side cannot afford this judgement (C3-k), so persist it here.
if (status?.stale === true) this.store.markCentroidStale(fingerprint)
} catch {
// 吞掉:写入已经成功,中心可以下次再建。
}
}
/**
* Persist the centroid for the configured space.
*
* Takes NO fingerprint argument on purpose: the caller cannot supply a
* different spelling than the one {@link commit} stamped on the blocks.
*
* ★ Callers: this is a PUBLIC programmatic entry point, not part of the
* write path. `/lib` never calls it; the only in-repo caller is
* `test/phase-b-h.test.js` (a caller that already computed a centroid
* elsewhere). Production centroids are derived from the blocks themselves by
* `BlockStore.rebuildCentroid`, driven from {@link commit} via
* `#ensureCentroid` — so this method existing with no internal caller is not
* a dangling wire, it is the seam for an external producer. Recorded because
* "defined, never called" looked identical to the real dangling-wire bugs
* found in C4 and P1.
*
* @param {{ dim?: number, vectorCount: number, mean: number[],
* isAnomalous: boolean, stats?: object }} centroid
* @returns {{ fingerprint: string }}
@ -275,10 +559,23 @@ export class StableMem extends Service {
// case; 'ok' is recorded explicitly so the stats reader never has to
// distinguish "no field" from "no problem".
mode: hits.degraded ?? 'ok',
// `mode` says whether the answer is trustworthy. The three counters below
// say how many candidates were dropped DOWNSTREAM of the space filter —
// without them an operator whose threshold is swallowing the corpus reads
// `skipped=0` and concludes nothing was excluded. See
// recall-accounting.test.js (D2).
skipped_fp: hits.skipped ?? 0,
// Real, not a placeholder: the recall path applies the F4b dimension filter
// and reports it separately from the space skip.
skipped_dim: hits.skippedDim ?? 0,
skipped_score: hits.skippedScore ?? 0,
skipped_no_evidence: hits.skippedNoEvidence ?? 0,
// D2's second hole: blocks dropped BEFORE they could enter the candidate
// pool (bad JSON, wrong width, non-finite component, zero norm, or — since
// C4 — a vector that centring collapses onto the space mean). Those are not
// "gated": no gate ever saw them, which is why they were invisible.
skipped_unscoreable: hits.skippedUnscoreable ?? 0,
truncated: hits.truncated ?? 0,
embed_error: hits.degradedError ?? null,
at: Date.now(),
}
@ -296,16 +593,164 @@ export class StableMem extends Service {
* @param {AbortSignal} [signal] the turn's cancellation, forwarded to recall
* @returns {Promise<string>}
*/
async contextForStep(messages, signal) {
async contextForStep(messages, signal, budget = {}) {
const query = lastUserText(messages)
if (query === '') return ''
const hits = await this.recall(query, { signal })
if (query === '') {
// Not a recall at all — but worth one debug line when verbose logging is
// on, because "the batch carried no human prompt" is exactly the failure
// this plugin already shipped once (the skill catalog as recall query).
this.logRecallTurn({ query, messages, raw: null, segmented: null, injected: '' })
return ''
}
const raw = await this.recall(query, { signal })
const segmented = segmentRecallBudget(raw, {
topK: this.topK,
budgetTokens: this.budgetTokens,
contextWindow: budget.contextWindow,
})
const { hits } = segmented
// ★ Do NOT short-circuit on `hits.length === 0` here. renderRecall decides
// silence by asking whether there are BOTH no entries and no notice; a
// degraded recall with zero hits still has something to say. Short-circuiting
// here would discard that notice before renderRecall ever sees it — and
// fixing only renderRecall without this line changes nothing (W11d†b).
return renderRecall(hits, name)
const rendered = renderRecall(hits, name)
const notice = hits.budgetNotice ?? ''
const injected = notice === '' ? rendered : rendered === '' ? notice : `${rendered}\n${notice}`
// Observe AFTER the decision is made, so the logged `inject` count is the
// exact string this method returns (not a re-derived estimate).
this.logRecallTurn({ query, messages, raw, segmented, injected })
return injected
}
/**
* One summary line per recall turn — the four facts an operator needs to
* answer "why did (or didn't) memory show up in this turn".
*
* | fact | fields | why it is there |
* |---|---|---|
* | recall | `query=<n>ch`, `hits`, `topK`, `zeroHit` | did the query reach recall, and did it match |
* | budget | `budget`, `source`, `kept`, `singleHitOverBudget` | which cap applied and what it kept |
* | degrade | `degraded`, `skipped`, `skippedDim`, `skippedScore`, `skippedNoEvidence`, `truncated`, `err` | whether the answer is trustworthy, and where the candidates went |
* | inject | `inject=<n>ch` | did it actually reach the model |
*
* ★ The query TEXT is never logged. Only its length: the recall query is the
* user's own message, and a memory log is the last place it should be copied
* to. Length is enough to tell "we searched nothing" from "we searched the
* skill catalog" when read beside `hits`.
*
* ★ Level, and why it is not always `info`:
* · fault (`embed_error`, or any mode not in {@link ORDINARY_DEGRADATIONS})
* → `warn`, emitted even when `debugLog` is off. A failed search is a
* per-turn event where absence of evidence is not evidence of absence;
* · `inject=0ch` (the common case on the shipping default) → `debug`, so
* "every ordinary turn" costs nothing at normal verbosity, but the line
* is still exactly one line;
* · otherwise → `info`, only when `debugLog` is on.
*
* ★ Logging must never break recall: any throw here would turn an
* observability feature into a recall outage, so the body is guarded.
*
* @param {{query: string, messages?: ReadonlyArray<object>, raw: Array<object>|null,
* segmented: object|null, injected: string}} turn
* @returns {void}
*/
logRecallTurn({ query, messages, raw, segmented, injected }) {
try {
const injectChars = charCount(injected)
// No human prompt in the batch: there was no recall to summarize.
if (segmented === null) {
if (this.debugLog !== true) return
emit(
this.log,
'debug',
`recall query=0ch skipped=no-human-prompt messages=${
Array.isArray(messages) ? messages.length : 0
} inject=0ch`,
)
return
}
const degraded = raw?.degraded ?? 'ok'
const fault = !ORDINARY_DEGRADATIONS.has(degraded)
if (this.debugLog !== true && !fault) return
const rawHits = Array.isArray(raw) ? raw.length : 0
const kept = Array.isArray(segmented.hits) ? segmented.hits.length : 0
const line = [
`recall query=${charCount(query)}ch hits=${rawHits} topK=${this.topK} zeroHit=${
rawHits === 0 ? 'yes' : 'no'
}`,
`budget=${segmented.budgetTokens} source=${segmented.source} kept=${kept} singleHitOverBudget=${
segmented.singleHitOverBudget ? 'yes' : 'no'
}`,
`degraded=${degraded} skipped=${raw?.skipped ?? 0} skippedDim=${
raw?.skippedDim ?? 0
} skippedScore=${raw?.skippedScore ?? 0} skippedNoEvidence=${
raw?.skippedNoEvidence ?? 0
} skippedUnscoreable=${raw?.skippedUnscoreable ?? 0} truncated=${
raw?.truncated ?? 0
} err=${summarizeError(raw?.degradedError)}`,
`inject=${injectChars}ch`,
].join(' | ')
if (fault) emit(this.log, 'warn', line)
else if (injectChars === 0) emit(this.log, 'debug', line)
else emit(this.log, 'info', line)
} catch {
// Observability is not worth a turn. Swallow and keep the recall path.
}
}
/**
* The model context window this step will run under, when the host discloses it.
*
* Precedence mirrors the port's budget rule (explicit config > host/model window
* > conservative default): an explicit `budgetTokens` never reaches here.
*
* The logged `request/context` event is preferred because it is the exact
* capability the request uses. It does not exist yet on a session's first step
* (`request/context` is appended during the request itself), so we ask the
* adapter once per route and cache the answer. Failures are cached too — an
* unresolvable route is a standing configuration fact, and retrying an adapter
* lookup every turn would put provider I/O on the recall hot path.
*
* @param {object} agent the agent proposing this step
* @param {AbortSignal} [signal] cancellation for the adapter lookup
* @returns {Promise<number | undefined>} window in tokens, or undefined when unknown
*/
async contextWindowFor(agent, signal) {
const logged = agent?.session?.requestContext?.()?.contextWindow
if (Number.isFinite(logged) && logged > 0) return logged
const provider = agent?.options?.provider
const model = agent?.options?.model
if (typeof provider !== 'string' || provider === '') return undefined
if (typeof model !== 'string' || model === '') return undefined
const key = `${provider}/${model}`
if (this.windowCache.has(key)) {
const cached = this.windowCache.get(key)
return Number.isFinite(cached) && cached > 0 ? cached : undefined
}
let resolved = null
try {
const llmService = typeof this.ctx?.get === 'function' ? this.ctx.get('llm') : undefined
const llm = llmService
const info = typeof llm?.resolveModelInfo === 'function'
? await llm.resolveModelInfo(provider, model, signal)
: undefined
const window = info?.context?.contextWindow
if (Number.isFinite(window) && window > 0) resolved = window
} catch {
// No adapter, a rejecting adapter, or cancellation: fall back to the
// conservative default rather than guessing a large window.
resolved = null
}
this.windowCache.set(key, resolved)
return resolved ?? undefined
}
/**
@ -401,6 +846,22 @@ export class StableMem extends Service {
states,
...this.store.vectorStats(space?.fingerprint ?? ''),
centroid: space === null ? null : (this.store.loadCentroid(space.fingerprint) ?? null),
// ★ C5's `reason` needs an outlet, and this is it.
//
// `centroid` above is the raw row (`loadCentroid`), which says WHAT the
// centre is but not WHY centring is or is not happening. `reason` is the
// answer to the second question — no_centroid / dim_mismatch / zero_sample /
// stale / isotropic / ok — and until now it was computed and thrown away:
// nothing under lib/ read the value `centroidStatus` returns.
//
// ★ `schedule: () => {}` is load-bearing. `centroidStatus` REGISTERS a
// rebuild when it judges a space stale; a stats read must not mutate
// anything. With the no-op scheduler, `rebuildScheduled` is always false
// here, which is exactly what a pure read should report.
centroid_state: space === null ? null : this.store.centroidStatus(space.fingerprint, {
queryDim: space.dimension,
schedule: () => {},
}),
// Per-call, not library state: NULL until a recall has happened.
last_recall: this.lastRecall,
}
@ -561,7 +1022,32 @@ export function apply(ctx, config = {}) {
const embedder =
config.embedder ??
(config.embed?.endpoint !== undefined ? createHttpEmbedder(config.embed) : null)
const mem = new StableMem(ctx, { store, topK: config.topK ?? 3, embedder })
// ★ V6 — the write side needs its own encoder.
//
// Two instances over the same endpoint/model/dimension (so the fingerprint is
// identical and the block and centroid rows still agree on one space),
// differing only in the `side` field they put on the wire. Done here rather
// than inside the adapter so `lib/embed/http.js` keeps travelling a single
// verbatim `side` value — it already accepts one, and teaching it a second,
// per-call way to receive it would be two sources of truth for one field.
//
// An injected `config.embedder` has no second instance: it falls back to the
// one object (see StableMem#documentEmbedder), which keeps every existing test
// and any single-sided adapter working unchanged.
const documentEmbedder =
config.documentEmbedder ??
(config.embed?.endpoint !== undefined && config.embedder === undefined
? createHttpEmbedder({ ...config.embed, side: 'document' })
: null)
const mem = new StableMem(ctx, {
store,
topK: config.topK ?? 3,
embedder,
documentEmbedder,
budgetTokens: config.budgetTokens,
debugLog: config.debugLog,
})
// Injection surface 1 — constant index. Right for input-independent text.
//
@ -587,7 +1073,8 @@ export function apply(ctx, config = {}) {
if (decision.kind === 'reject' || payload.signal.aborted) return decision
// Forward the turn signal: without it a cancelled turn waits out the
// embedding adapter's own timeout before returning.
const injected = await mem.contextForStep(decision.messages, payload.signal)
const contextWindow = await mem.contextWindowFor(payload.agent, payload.signal)
const injected = await mem.contextForStep(decision.messages, payload.signal, { contextWindow })
if (injected === '') return decision
return {
...decision,

173
lib/loop.js Normal file
View File

@ -0,0 +1,173 @@
/**
* HologramAgent loop: surface recall as a SYSTEM section instead of history.
*
* Why this exists
* ---------------
* `dsh-memory` injects per-turn recall through the `agent/pre-step` waterfall,
* and that hook can only return `messages`. Whatever it adds therefore lands in
* conversation history as a user message — the model reads its own memory as if
* a user had said it. `systemPrompt.context` cannot fix this either: that hook
* runs before the turn text is known (documented in dsh-memory's own module
* note), so it can only render input-INDEPENDENT text.
*
* The loop's own `preStep` is the first place that has BOTH the assembled
* prompt and the claimed messages for this turn. So the swap point is here:
* let the existing pre-step hooks run unchanged, then move the recall block out
* of `messages` and into `assembly.sections`. The memory becomes part of the
* system prompt for that turn, and the user message never enters history.
*
* Coupling
* --------
* None, deliberately. We do not import the memory plugin or match its name; we
* recognise the *shape* every snapshot-style injector already uses
* (`source.sections = [{ name, text }]`), which is the same shape
* `dsh-time-context` emits. Anything a plugin chose to label that way is
* promoted; everything else passes through untouched.
*/
import { AgentLoop } from '@deepseek-ai/dsh-agent-loop'
/** Suffix appended to the promoted section name, keeping provenance visible. */
const PROMOTED_SUFFIX = ':system'
/**
* Split one message's `source.sections` out of the message itself.
*
* A snapshot carrier is recognised by ONE of two canonical spellings, both of
* which real producers emit:
*
* 1. `content: []` — a bare carrier. The sections are the whole payload.
* 2. `content: [{ type: 'text', text }]` — what this plugin itself emits
* (`lib/index.js:932`): one text block whose text is the sections' text
* joined by `\n\n` (or, from other producers, concatenated with no
* separator). Both conventions are accepted because the joining choice
* belongs to the producer, not to this loop.
*
* ★ The invariant is "move the text, do not rewrite it". So when content is
* present, it is promoted ONLY if the rendered blocks equal the sections'
* text under one of those two conventions, and the promoted section text is
* the *rendered content*, not a re-join of the sections. That makes
* "relocation" assertable in a test (`promoted.text === original content
* text`) and leaves no room for a silent edit. Anything else — a differing
* text, a non-text block, a runtime-context snapshot whose content carries a
* fixed preamble the sections do not — stays in history untouched.
*
* History: the previous rule was "content non-empty ⇒ reject", derived from an
* assumption ("content means the plugin wants the user to see it") that is
* false for this plugin's own output, so on the real path the promotion could
* never fire. See `test/loop.test.js` S1–S7.
*
* @param {object} message a user message from a pre-step decision
* @returns {{text: string, name: string} | null}
*/
function promotedSection(message) {
const source = message === null || typeof message !== 'object' ? undefined : message.source
if (source === null || typeof source !== 'object') return null
if (source.form !== 'snapshot') return null
if (!Array.isArray(source.sections) || source.sections.length === 0) return null
const parts = []
for (const section of source.sections) {
if (section !== null && typeof section === 'object' && typeof section.text === 'string' && section.text !== '') {
parts.push(section.text)
}
else if (typeof section === 'string' && section !== '') parts.push(section)
}
if (parts.length === 0) return null
const name = typeof source.kind === 'string' && source.kind !== '' ? source.kind : 'memory'
const blocks = Array.isArray(message.content) ? message.content : []
if (blocks.length === 0) return { text: parts.join('\n\n'), name }
// Any non-text block means this is not a carrier of prompt text: an image
// (or a block whose text is not a string) would be silently destroyed by a
// text-only move, so refuse the whole message.
for (const block of blocks) {
if (block === null || typeof block !== 'object') return null
if (block.type !== 'text' || typeof block.text !== 'string') return null
}
// Both canonical joins are accepted; the promoted text is the RENDERED
// content, which is what the model would have read, byte for byte.
const rendered = blocks.map((block) => block.text).join('')
const glued = parts.join('')
const spaced = parts.join('\n\n')
if (rendered !== glued && rendered !== spaced) return null
return { text: rendered, name }
}
/**
* Rewrite one pre-step decision so snapshot sections live in the prompt.
*
* The returned object is a NEW decision: `super.step()` keeps the one it
* received, and the loop's phase bookkeeping reads `assembly` off whatever it
* is handed. Mutating in place would work today and would silently break the
* first time the loop reuses a decision.
*
* @param {object} decision the decision returned by the pre-step waterfall
* @returns {object} the same decision shape with promoted sections moved
*/
export function promoteSnapshotSections(decision) {
if (decision === null || typeof decision !== 'object') return decision
if (decision.kind === 'reject') return decision
const messages = Array.isArray(decision.messages) ? decision.messages : []
const assembly = decision.assembly
if (assembly === null || typeof assembly !== 'object' || !Array.isArray(assembly.sections)) return decision
const kept = []
const promoted = []
for (const message of messages) {
const section = promotedSection(message)
if (section === null) kept.push(message)
else promoted.push(section)
}
if (promoted.length === 0) return decision
const sections = [...assembly.sections]
const seen = new Set(sections.map((section) => section.name))
for (const section of promoted) {
// A name collision means some other producer already owns that slot. The
// prompt layer throws on duplicates, so suffix rather than overwrite:
// dropping the content would hide memory with no visible failure.
let name = `${section.name}${PROMOTED_SUFFIX}`
for (let n = 2; seen.has(name); n += 1) name = `${section.name}${PROMOTED_SUFFIX}${n}`
seen.add(name)
// `interpolate: false` is mandatory, not cosmetic: recalled notes are free
// text and may legitimately contain `{{...}}`, which the strict interpolator
// rejects with `unknown prompt variable` (measured — dsh-memory hit this).
sections.push({ name, text: section.text, interpolate: false })
}
return { ...decision, messages: kept, assembly: { ...assembly, sections } }
}
/**
* The headless AgentLoop, with snapshot sections promoted into the prompt.
*
* No constructor and no static `inject`: the parent already registers the
* turnBoundary + inbox projections, calls `agents.setFactory(this)`, and
* declares the `{{provider}}` / `{{model}}` variables. Re-declaring any of them
* here would be the bug — `setFactory` throws if a factory is already
* registered, so a second registration is exactly what breaks startup.
*/
export class HologramAgentLoop extends AgentLoop {
/**
* Prepare one agent, then wrap its pre-step seam.
*
* Patching the *instance* (not the prototype) is deliberate: the prototype is
* shared with every agent the base class creates, including ones this plugin
* must not touch. An own property shadows it for this agent only.
*
* @param {...unknown} args forwarded verbatim to the parent
* @returns {{agent: object, signal: AbortSignal, publish: Function}}
*/
prepare(...args) {
const prepared = super.prepare(...args)
const agent = prepared === null || typeof prepared !== 'object' ? undefined : prepared.agent
if (agent === undefined || typeof agent.preStep !== 'function') return prepared
const original = agent.preStep.bind(agent)
agent.preStep = async (target, position) => promoteSnapshotSections(await original(target, position))
return prepared
}
}
export default HologramAgentLoop

View File

@ -0,0 +1,140 @@
/**
* Folds one model attempt into one compact stream plus ordered transient frames.
*
* Reproduced from `dsh-agent-loop/lib/types/index.js:352-473` (0.2.1-rc.2, MIT)
* because `dsh-agent-loop` does not export it, and there is no behaviour to
* change here: the framing has to match what `dsh-headless` and the session
* title plugin read off `agent/assistant-stream` byte for byte.
*
* @module dsh-memory/lib/loop/assistant-stream
*/
import { AssistantStreamAccumulator, BlockAssembler, LlmAttemptId } from '@deepseek-ai/dsh-llm'
export class AssistantStreamAttempt {
/** @type {() => number} */
nextRevision
/** @type {number} */
turn
/** @type {number} */
step
/** @type {(frame: object) => void} */
emit
accumulator = new AssistantStreamAccumulator()
assembler = new BlockAssembler()
index = 0
terminal = false
/** Attempt identity unique within this Agent lifecycle. */
attemptId
/** Whether this started attempt has emitted its terminal frame. */
get ended() {
return this.terminal
}
/**
* @param {string} sessionId identity embedded only in the Agent-lifecycle-local attempt id
* @param {number} attempt attached-Session-local attempt counter
* @param {() => number} nextRevision allocates the next emitted frame revision
* @param {number} turn durable turn owning the request
* @param {number} step durable step owning the request
* @param {(frame: object) => void} emit agent-scoped notification publisher
*/
constructor(sessionId, attempt, nextRevision, turn, step, emit) {
this.nextRevision = nextRevision
this.turn = turn
this.step = step
this.emit = emit
this.attemptId = LlmAttemptId(`${sessionId}:${attempt}`)
}
/** Publish the opening marker before the first delivered chunk. */
start() {
this.emit({
type: 'start',
attemptId: this.attemptId,
revision: this.nextRevision(),
turn: this.turn,
step: this.step,
})
}
/** Snapshot one chunk once, then feed durable compaction, assembly, and live publication. */
push(chunk) {
const timed = this.accumulator.push({ time: Date.now(), chunk })
this.assembler.push(timed.chunk)
this.emit({
type: 'chunk',
attemptId: this.attemptId,
revision: this.nextRevision(),
index: this.index++,
time: timed.time,
chunk: timed.chunk,
})
}
/**
* Publish terminal settlement after the matching durable event commits.
* @param {string} eventType durable settlement type
* @param {() => number} append synchronous durable append returning its committed seq
*/
settle(eventType, append) {
let seq
try {
seq = append()
} catch (error) {
this.abandon()
throw error
}
this.terminal = true
this.emit({
type: 'end',
attemptId: this.attemptId,
revision: this.nextRevision(),
index: this.index,
outcome: { kind: 'committed', eventType, seq },
})
}
/** Publish abandonment when no durable attempt event can be committed. */
abandon() {
this.terminal = true
this.emit({
type: 'end',
attemptId: this.attemptId,
revision: this.nextRevision(),
index: this.index,
outcome: { kind: 'abandoned' },
})
}
/** Exact compact stream for the final durable event. */
get stream() {
return [...this.accumulator.snapshot()]
}
/** Canonical completed-message blocks from the same chunks. */
blocks() {
return this.assembler.blocks()
}
/** Safe visible prefix when cancellation interrupts the attempt. */
interruptedBlocks() {
return this.assembler.interruptedBlocks()
}
/** Latest adapter-reported usage in the stream. */
get usage() {
return this.assembler.usage
}
/** Terminal reason, defaulting to stop when the stream omitted one. */
get finish() {
return this.assembler.finish
}
/** Replay state carried by the terminal finish record. */
get replayState() {
return this.assembler.replayState
}
}

255
lib/loop/inbox.js Normal file
View File

@ -0,0 +1,255 @@
/**
* Durable agent-inbox projection and its command facade.
*
* ## Why this file exists at all
*
* `dsh-agent-loop` keeps `inboxProjectionDefinition` module-private: its
* package entry exports only `AgentLoop`, `DEFAULT_MAX_PARALLEL_TOOL_CALLS`,
* `CONFIGURED_AGENT_IDENTITIES_KEY` and `turnBoundaryProjectionDefinition`
* (dsh-agent-loop/lib/index.js:2023). A replacement loop therefore cannot
* import the fold, and re-deriving pending input in memory instead of in a
* projection would lose queued messages on `resume` — the pending splices live
* in the durable log, so the state has to be rebuilt from it.
*
* So the fold is reproduced here, field for field, from
* `dsh-agent-loop/lib/index.js:26-60` (0.2.1-rc.2, MIT). The projection KEY
* must stay `"inbox"` and `stateVersion` must stay 1: a different key would
* silently read as "no pending input" for sessions written by the stock loop,
* and a different version would make `stateSchema.parse` refuse those rows.
*
* ★ The one deliberate deviation: the host validates with zod
* (`z.array(z.custom())`), which is a dependency this plugin does not carry.
* {@link inboxProjectionSchema} provides the same contract the projection
* registry actually uses — `parse()` that throws on a malformed persisted row —
* without pulling zod in. Nothing else about the fold changed.
*
* @module dsh-memory/lib/loop/inbox
*/
/**
* Validate one persisted inbox state.
*
* The registry calls `stateSchema.parse` on durable checkpoint rows and on the
* value a fold returns, so this must reject anything that is not the two-array
* shape. Message contents stay opaque (`z.custom()` in the host): they are
* already validated by the session layer when they were added to the surface,
* and re-validating them here would make the fold able to reject a log the
* session accepted.
*/
export const inboxProjectionSchema = {
/**
* @param {unknown} value candidate state
* @returns {{'next-turn': Array<object>, 'next-step': Array<object>}} the same object
* @throws {Error} when the shape is not two message arrays
*/
parse(value) {
if (value === null || typeof value !== 'object' || Array.isArray(value)) {
throw new Error('inbox projection state must be an object')
}
const nextTurn = value['next-turn']
const nextStep = value['next-step']
if (!Array.isArray(nextTurn) || !Array.isArray(nextStep)) {
throw new Error('inbox projection state must hold "next-turn" and "next-step" message arrays')
}
return value
},
}
/** Standard fold that reconstructs pending input and rejects invalid splice history. */
export const inboxProjectionDefinition = {
key: 'inbox',
stateSchema: inboxProjectionSchema,
init: () => ({
'next-turn': [],
'next-step': [],
}),
apply(state, event) {
if (event.type !== 'agent/inbox/spliced') return state
const splice = event.data
try {
const inbox = state[splice.target]
const removedCount = splice.removedCount ?? 0
if (
!Number.isSafeInteger(splice.start) ||
splice.start < 0 ||
splice.start > inbox.length ||
!Number.isSafeInteger(removedCount) ||
removedCount < 0 ||
splice.start + removedCount > inbox.length
) {
throw new Error('invalid inbox splice')
}
const next = inbox.toSpliced(splice.start, removedCount, ...splice.inserted)
const ids = new Set()
for (const message of splice.target === 'next-turn' ? [...next, ...state['next-step']] : [...state['next-turn'], ...next]) {
if (ids.has(message.id)) throw new Error(`message "${message.id}" is already pending`)
ids.add(message.id)
}
return splice.target === 'next-turn'
? { 'next-turn': next, 'next-step': state['next-step'] }
: { 'next-turn': state['next-turn'], 'next-step': next }
} catch (error) {
throw new Error(`invalid persisted inbox splice at session seq ${event.seq}`, { cause: error })
}
},
wire: {
viewSchema: inboxProjectionSchema,
view: (state) => state,
},
stateVersion: 1,
}
/**
* Driver-owned durable Inbox implementation.
*
* Every mutation is a durable `agent/inbox/spliced` append; the projection is
* the only state holder, so a resumed session sees exactly the pending input
* the previous process left behind.
*/
export class ReactLoopInbox {
/**
* @param {object} projections `ctx.sessionProjections`
* @param {object} session the owning session
* @param {object} dispatch agent-scoped dispatcher
*/
constructor(projections, session, dispatch) {
this.projections = projections
this.session = session
this.dispatch = dispatch
}
/** Prompts awaiting individual turns. */
get nextTurn() {
return this.current()['next-turn']
}
/** Input awaiting the next step boundary. */
get nextStep() {
return this.current()['next-step']
}
/** Whether either pending-message list contains work. */
get hasPending() {
const state = this.current()
return state['next-turn'].length > 0 || state['next-step'].length > 0
}
/** Durably cancel all pending input, clearing next-step before next-turn. */
clear() {
this.splice('next-step', 0, this.nextStep.length, [])
this.splice('next-turn', 0, this.nextTurn.length, [])
}
/**
* Remove and return the complete batch proposed for one step.
* @param {'next-turn'|'next-step'} target whether this boundary also consumes one queued turn
* @param {number} turn turn that will own the claimed batch
*/
claim(target, turn) {
const claimed = this.mutate('next-step', 0, this.nextStep.length, [], false)
if (target === 'next-turn') claimed.push(...this.mutate('next-turn', 0, 1, [], false))
for (const message of claimed) this.dispatch.emit('agent/inbox/claimed', { message, turn })
return claimed
}
/**
* Append one message to a pending list.
* @param {'next-turn'|'next-step'} target pending list to extend
* @param {object} message message to append
*/
append(target, message) {
this.splice(target, this.current()[target].length, 0, [message])
}
/**
* Prepend one message to a pending list.
* @param {'next-turn'|'next-step'} target pending list to extend
* @param {object} message message to prepend
*/
prepend(target, message) {
this.splice(target, 0, 0, [message])
}
/**
* Replace one pending message in place.
* @param {string} messageId identity of the pending message to replace
* @param {object} newMessage replacement message
* @returns {boolean} whether the message was still pending
*/
replace(messageId, newMessage) {
const location = this.locate(messageId)
if (location === undefined) return false
this.splice(location.target, location.index, 1, [newMessage])
return true
}
/**
* Remove one pending message.
* @param {string} messageId identity of the pending message to remove
* @returns {boolean} whether the message was still pending
*/
remove(messageId) {
const location = this.locate(messageId)
if (location === undefined) return false
this.splice(location.target, location.index, 1, [])
return true
}
/**
* Apply standard splice semantics and durably record the normalized result.
* @param {'next-turn'|'next-step'} target pending list to mutate
* @param {number} start splice position
* @param {number} deleteCount maximum number of messages to remove
* @param {Array<object>} inserted messages to insert at the resolved position
*/
splice(target, start, deleteCount, inserted) {
return this.mutate(target, start, deleteCount, inserted, true)
}
/** Locate one pending identity across both owned lists. */
locate(messageId) {
const state = this.current()
for (const target of ['next-turn', 'next-step']) {
const index = state[target].findIndex((message) => message.id === messageId)
if (index >= 0) return { target, index }
}
}
/** Read the current durable projection state. */
current() {
const state = this.projections.stateOf(this.session, 'inbox')
if (state === undefined) throw new Error(`agent "${this.session.id}" cannot read inbox state: its projection registration is not active`)
return state
}
/** Commit one normalized mutation and publish its live events. */
mutate(target, start, deleteCount, inserted, discardRemoved) {
const state = this.current()
const inbox = state[target]
const truncatedStart = Math.trunc(start)
const offset = Number.isNaN(truncatedStart) ? 0 : truncatedStart
const actualStart = offset < 0 ? Math.max(inbox.length + offset, 0) : Math.min(offset, inbox.length)
const truncatedDeleteCount = Math.trunc(deleteCount)
const actualDeleteCount = Math.min(Math.max(Number.isNaN(truncatedDeleteCount) ? 0 : truncatedDeleteCount, 0), inbox.length - actualStart)
if (actualDeleteCount === 0 && inserted.length === 0) return []
const candidate = inbox.toSpliced(actualStart, actualDeleteCount, ...inserted)
const ids = new Set()
for (const message of target === 'next-turn' ? [...candidate, ...state['next-step']] : [...state['next-turn'], ...candidate]) {
if (ids.has(message.id)) throw new Error(`message "${message.id}" is already pending`)
ids.add(message.id)
}
const outcome = discardRemoved && actualDeleteCount > 0 ? 'canceled' : undefined
const splice = {
target,
start: actualStart,
...(actualDeleteCount === 0 ? {} : { removedCount: actualDeleteCount }),
inserted,
...(outcome === undefined ? {} : { outcome }),
}
const removed = inbox.slice(actualStart, actualStart + actualDeleteCount)
const event = this.session.append('agent/inbox/spliced', splice)
if (discardRemoved) for (const message of removed) this.dispatch.emit('agent/inbox/discarded', { message })
for (const message of event.data.inserted) this.dispatch.emit('agent/inbox/inserted', { message })
return removed
}
}

145
lib/loop/runtime-context.js Normal file
View File

@ -0,0 +1,145 @@
/**
* The two surface projections the loop owns: the rendered system prompt and
* the dynamic runtime-context snapshot.
*
* ## Why this is a copy and not an import
*
* `dsh-agent-loop` does not export either class (its entry exports only
* `AgentLoop`, the constants and `turnBoundaryProjectionDefinition`,
* dsh-agent-loop/lib/index.js:2023). They are nevertheless required: without
* `SystemPromptProjection` the prompt would never reach surface node 0, and
* without `RuntimeContextProjection` every step would append a duplicate
* snapshot. Both are reproduced from
* `dsh-agent-loop/lib/types/index.js:211-351` (0.2.1-rc.2, MIT) unchanged.
*
* ★ `SOURCE = 'runtime-context'` is load-bearing and must not be renamed: the
* stock loop also owns snapshots under that `source.kind`, so a session that
* switches between the stock loop and this one (a profile patch applied
* mid-life) must still recognise the previous process's snapshot as its own.
* A different kind would leave two snapshots in the surface, both claiming to
* supersede the other.
*
* @module dsh-memory/lib/loop/runtime-context
*/
import { createSystemMessage, createUserMessage } from '@deepseek-ai/dsh-llm'
import { isReplacementSurfaceEvent } from '@deepseek-ai/dsh-session'
const SOURCE = 'runtime-context'
const CLEARED = 'Current runtime context: none. Earlier runtime-context snapshots no longer apply.'
/** @param {object} message derived message */
function isOwned(message) {
return message.source.kind === SOURCE
}
/** @param {object} message derived message */
function textOf(message) {
const [block] = message.content
return message.content.length === 1 && block?.type === 'text' ? block.text : undefined
}
/** Committed events from the newest backward; the restore scans stop at the first match. */
function eventsNewestFirst(session) {
return session.snapshotEvents().toReversed()
}
/**
* Decides how a rendered system prompt reaches the surface without owning the
* commit. The first prompt, even empty, reserves surface node 0.
*/
export class SystemPromptProjection {
/** @param {object} session */
constructor(session) {
this.session = session
}
/** The surviving `system/message` nodes in surface order. */
systemNodes() {
const nodes = []
for (const seq of this.session.surface.nodes) {
const event = this.session.eventAt(seq)
if (event?.type !== 'system/message') continue
const text = event.data.message.content.length === 0 ? '' : textOf(event.data.message)
nodes.push({ seq, text })
}
return nodes
}
/**
* Reconcile effective text and retained nodes with the prepared route and series.
* @param {string} rendered the fully rendered system prompt; `''` when none is active
* @param {{inHistory: boolean, startsSeries: boolean}} input route capability and series facts
*/
project(rendered, input) {
const nodes = this.systemNodes()
const head = nodes[0]
if (head === undefined) return [{ message: createSystemMessage(rendered), intent: { surfaceOp: 'append' } }]
const latest = nodes.findLast((node) => node.text !== '') ?? head
if (!input.inHistory || input.startsSeries || rendered.length === 0) {
const updates = nodes.slice(1).filter((node) => node.text !== '').map((node) => this.replace(node.seq, ''))
if (head.text !== rendered) updates.push(this.replace(head.seq, rendered))
return updates
}
if (latest.text === rendered) return []
return [{ message: createSystemMessage(rendered), intent: { surfaceOp: 'append' } }]
}
/**
* @param {number} seq surface node to replace
* @param {string} text replacement text
*/
replace(seq, text) {
return {
message: createSystemMessage(text),
intent: { surfaceOp: { op: 'replace', startSeq: seq, endSeq: seq }, sourceEventSeqs: [seq] },
}
}
}
/** Tracks the last retained runtime-context snapshot without owning its commit. */
export class RuntimeContextProjection {
/** `undefined` means no snapshot ever existed; `null` means none is retained. */
retained
/**
* Restore projection state once, then follow authoritative session events.
* @param {object} ctx agent-scoped event context
* @param {object} session session receiving projected messages
*/
constructor(ctx, session) {
const surface = new Set(session.surface.nodes)
for (const event of eventsNewestFirst(session)) {
if (event.type !== 'user/message' || !isOwned(event.data)) continue
this.retained ??= null
if (surface.has(event.seq)) {
this.retained = { seq: event.seq, text: textOf(event.data) }
break
}
}
ctx.on('session/event', (subject, event) => {
if (subject !== session) return
if (event.type === 'user/message' && isOwned(event.data)) {
this.retained = { seq: event.seq, text: textOf(event.data) }
} else if (this.retained && isReplacementSurfaceEvent(event) && event.sourceEventSeqs?.includes(this.retained.seq) === true) {
this.retained = null
}
})
}
/**
* Create an uncommitted snapshot only when the retained value differs.
* @param {string} current fully rendered dynamic context
* @param {Array<{name: string, text: string}>} sections named contributions of the snapshot
* @returns {object|undefined} a candidate user message, or undefined when no update is needed
*/
project(current, sections) {
if (this.retained === undefined && current.length === 0) return
const snapshot = current.length === 0 ? CLEARED : current
if (this.retained?.text === snapshot) return
return createUserMessage({
content: [{ type: 'text', text: snapshot }],
source: sections.length === 0 ? { kind: SOURCE } : { kind: SOURCE, form: 'snapshot', sections },
})
}
}

254
lib/loop/tool-calls.js Normal file
View File

@ -0,0 +1,254 @@
/**
* Schedules one assistant step's tool calls. Exclusive calls form barriers;
* parallel calls use a bounded rolling pool and are reclassified before start.
* Dispatch may overlap, while policy, results, and result context remain
* model-ordered. Abort or an internal scheduler failure stops replenishment
* and drains started calls.
*
* Reproduced from `dsh-agent-loop/lib/types/tool-calls.js:474-708`
* (0.2.1-rc.2, MIT); the loop replacement deliberately changes no scheduling
* behaviour, so this file is a port rather than a rewrite.
*
* ★ One substitution: the host reads its parallelism cap from
* `ctx.agentLoop.config.maxParallelToolCalls` (dsh-agent-loop/lib/index.js:553),
* a service this plugin does not provide (its only consumer is the Web settings
* panel, which is out of scope). The caller passes the same value explicitly
* through {@link ToolCallSchedulerOptions.maxParallelToolCalls} instead; the
* cap is still a live getter, so a dynamic override keeps working.
*
* @module dsh-memory/lib/loop/tool-calls
*/
import { createToolResultMessage } from '@deepseek-ai/dsh-llm'
import { TOOL_ABORTED_BEFORE_DISPATCH, TOOL_RUNTIME_SCHEDULER } from '@deepseek-ai/dsh-tools'
/**
* Throw on a value the compiler proved unreachable.
*
* Inlined rather than imported from `@deepseek-ai/dsh-util-values`, and that is a
* G2b decision, not an oversight: `dsh-util-values` is a *dependency* of
* `dsh-agent-loop`, not one of its peers (dsh-agent-loop/package.json). Adding it
* here would therefore introduce a runtime dependency this plugin carries itself
* — exactly what G2b exists to catch — and it would break H5 (no new deps) for a
* three-line helper. The four lines below are the upstream body; the message
* prefix stays identical so a log line still names the same call site.
*
* @param {never} value the value that cannot occur
* @param {string} [context] name of the switch that produced it
* @throws {Error} always
*/
function assertNever(value, context) {
const rendered = JSON.stringify(value) ?? String(value)
throw new Error(`unreachable variant${context ? ` in ${context}` : ''}: ${rendered}`)
}
/**
* Schedule one assistant step's tool calls by their live concurrency mode.
*
* Ordinary completion and abort commit started-call results in order. Abort
* drains them, records synthetic results for unstarted calls, and returns with
* the signal still aborted after accepting started-call context through the
* caller-supplied acceptor.
*
* @param {object} ctx loop context that owns the tool registry and carries the initiating Agent
* @param {number} turn current turn number
* @param {number} step current step number
* @param {Array<object>} toolCalls assistant calls in model order
* @param {AbortSignal} signal abort signal shared by the step
* @param {(context: object) => void} acceptContext accepts committed result context for the next step
* @param {{maxParallelToolCalls: () => number}} options the live parallelism cap
* @returns {Promise<{concluded: boolean}>}
*/
export async function executeToolCalls(ctx, turn, step, toolCalls, signal, acceptContext, options) {
const agent = ctx.agents.requireInitiator()
const { session } = agent
const planned = toolCalls.map((block) => ({
block,
exec: { callId: block.id, name: block.name, arguments: parseArguments(block.arguments), agent, signal },
}))
let next = 0
let concluded = false
while (next < planned.length) {
const first = planned[next]
const mode = ctx.tools.executionMode(first.exec).kind
const outcome = await runGroup(
ctx,
turn,
step,
mode === 'parallel' ? planned.slice(next) : [first],
mode,
signal,
acceptContext,
options,
)
next += outcome.consumed
concluded ||= outcome.concluded
if (outcome.aborted) {
for (const call of planned.slice(next)) appendSkippedToolCall(session, turn, step, call.block)
return { concluded }
}
}
return { concluded }
}
/** Parse model arguments, preserving invalid JSON as text and mapping empty input to `{}`. */
function parseArguments(raw) {
try {
return raw ? JSON.parse(raw) : {}
} catch {
return raw
}
}
/**
* Run one exclusive barrier or parallel pool.
*
* @param {object} ctx
* @param {number} turn
* @param {number} step
* @param {Array<{block: object, exec: object}>} group
* @param {'parallel'|'exclusive'} mode
* @param {AbortSignal} signal
* @param {(context: object) => void} acceptContext
* @param {{maxParallelToolCalls: () => number}} options
*/
async function runGroup(ctx, turn, step, group, mode, signal, acceptContext, options) {
const { session } = ctx.agents.requireInitiator()
const maxParallelToolCalls = options.maxParallelToolCalls()
const slots = group.map(() => undefined)
const callSeqs = group.map(() => undefined)
let nextToStart = 0
let committed = 0
let started = 0
let aborted = signal.aborted
let concluded = false
let schedulerFailure
const throwSchedulerFailure = () => {
if (schedulerFailure !== undefined) throw schedulerFailure.error
}
const commitReady = async () => {
while (committed < group.length) {
const slot = slots[committed]
if (slot === undefined) break
const call = group[committed]
const result = slot.needsPost
? await ctx.tools[TOOL_RUNTIME_SCHEDULER].finalize(slot.exec, slot.result)
: ctx.tools[TOOL_RUNTIME_SCHEDULER].finish(slot.exec, slot.result)
appendToolResult(session, turn, step, call.block, result, callSeqs[committed])
for (const context of result.additionalContexts ?? []) acceptContext(context)
concluded ||= result.concludesTurn === true
committed++
}
}
const inFlight = new Map()
const startCall = async (index) => {
const call = group[index]
callSeqs[index] = appendToolCall(session, turn, step, call.block)
started++
const prepared = await ctx.tools[TOOL_RUNTIME_SCHEDULER].prepare(call.exec)
throwSchedulerFailure()
switch (prepared.kind) {
case 'dispatch': {
const promise = ctx.tools[TOOL_RUNTIME_SCHEDULER].dispatch(prepared.exec).then(
(outcome) => {
slots[index] = { exec: prepared.exec, result: outcome.result, needsPost: outcome.kind === 'post-result' }
return index
},
(error) => {
schedulerFailure ??= { error }
return index
},
)
inFlight.set(index, promise)
break
}
case 'post-result':
slots[index] = { exec: prepared.exec, result: prepared.result, needsPost: true }
break
case 'final-result':
slots[index] = { exec: prepared.exec, result: prepared.result, needsPost: false }
break
default:
assertNever(prepared, 'tool-call scheduler prepare result')
}
}
const fillPool = async () => {
while (!aborted && nextToStart < group.length && inFlight.size < maxParallelToolCalls) {
const nextCall = group[nextToStart]
if (nextToStart > 0 && mode === 'parallel' && ctx.tools.executionMode(nextCall.exec).kind !== 'parallel') break
await startCall(nextToStart)
nextToStart++
throwSchedulerFailure()
await commitReady()
throwSchedulerFailure()
if (signal.aborted) aborted = true
}
}
try {
await fillPool()
while (inFlight.size > 0) {
const settledIndex = await Promise.race(inFlight.values())
inFlight.delete(settledIndex)
throwSchedulerFailure()
await commitReady()
throwSchedulerFailure()
if (signal.aborted) aborted = true
await fillPool()
}
} catch (error) {
schedulerFailure ??= { error }
await Promise.allSettled(inFlight.values())
throw schedulerFailure.error
}
if (aborted) {
for (const call of group.slice(started)) appendSkippedToolCall(session, turn, step, call.block)
return { consumed: group.length, aborted: true, concluded }
}
if (committed !== started) throw new Error('tool-call scheduler: uncommitted settled calls')
return { consumed: started, aborted: false, concluded }
}
/** Append the durable call/result pair for a model call skipped after cancellation. */
function appendSkippedToolCall(session, turn, step, block) {
const callSeq = appendToolCall(session, turn, step, block)
appendToolResult(
session,
turn,
step,
block,
{
content: [{ type: 'text', text: 'Error: tool call aborted before dispatch' }],
isError: true,
error: {
message: 'tool call aborted before dispatch',
info: { name: 'AbortError', code: TOOL_ABORTED_BEFORE_DISPATCH },
},
},
callSeq,
)
}
/** Append a started call and return the event seq that its result must cite. */
function appendToolCall(session, turn, step, block) {
return session.append('tool/call', { turn, step, callId: block.id, name: block.name, arguments: block.arguments }).seq
}
/** Append a model-ordered result linked to its call event. */
function appendToolResult(session, turn, step, block, result, callSeq) {
const message = createToolResultMessage({
callId: block.id,
content: result.content,
isError: result.isError,
})
session.append(
'tool/result',
{
turn,
step,
message,
...(result.error?.info ? { error: result.error.info } : {}),
...(result.meta !== undefined ? { meta: result.meta } : {}),
},
{ surfaceOp: 'append', sourceEventSeqs: [callSeq] },
)
}

View File

@ -43,6 +43,37 @@
* @module dsh-memory/recall
*/
// ported from go: internal/memory/block_recall.go:263.
const largeTopK = 2000
/**
* Default vector-route admission threshold for the fused recall path.
*
* Ported from Go `blockRecallMinScore` (internal/agent/core/toolcall.go:1405),
* which is the value production HA actually passes. Its comment there records
* why the threshold must NOT be 0: without it, recall on a query his gograph
* has nothing on still returns high-scoring noise, and abstention (`AbstainCheck`)
* becomes the only thing standing between the model and a confident wrong answer.
*
* ★ This is an admission gate for the PURE VECTOR route only. `recallBlocks`
* applies it per path (see the V5 block), so an exact pin and a symbol hit are
* unaffected — that asymmetry is the whole point of V5 and is pinned separately.
*/
export const DEFAULT_MIN_SCORE = 0.5
/**
* Smallest centred-vector norm that still counts as a direction.
*
* A block equal to the space mean differs from it only by floating-point dust
* (~1e-17). Renormalising that dust produces a unit vector pointing in an
* arbitrary direction, which then scores against the query as if it were
* signal — measured on a space of two identical blocks: cosine 0.2183 out of
* dust where the raw cosine was 0.8608. Anything below this floor is "no
* direction", which is also what the Go side sees (cosine of a zero vector
* is 0, not a random angle).
*/
const CENTER_EPSILON = 1e-9
/**
* Extract high-information symbols from a query.
*
@ -218,13 +249,16 @@ function allIndexesOf(haystack, needle) {
* @param {number} [options.topK] max hits returned (default 5)
* @param {number} [options.poolSize] FTS candidate pool before truncation (default 50)
* @param {Map<string, number>} [options.vectorScores] optional vector scores for V1 exact-band tie adjustment
* @param {number} [options.minScore] opt-in V5 fused route; 0 disables only the vector threshold,
* while absence preserves the phase-A path
* @returns {Array<{id: string, text: string, source: string, score: number,
* exact: boolean, why: string}>}
*/
export function recallBlocks(store, query, { topK = 5, poolSize = 50, fingerprint, dimension, vectorScores } = {}) {
export function recallBlocks(store, query, { topK = 5, poolSize = 50, fingerprint, dimension, vectorScores, minScore, skippedUnscoreable = 0 } = {}) {
const text = String(query ?? '').trim()
if (text === '') return []
const fusedCandidates = Number.isFinite(minScore)
const symbols = querySymbols(text)
const match = buildMatchExpression(text)
// Computed once: used both for fallback admission and for the CJK coverage
@ -283,6 +317,31 @@ export function recallBlocks(store, query, { topK = 5, poolSize = 50, fingerprin
}
}
// ported from go: internal/memory/fusion.go:374-408, :473-498.
// Opt-in fused candidate assembly; absent minScore preserves the frozen A2
// path and the V1/V2 scoring seam. Map membership is vector-route provenance,
// not the existence of attribution fields on a returned record.
// ported from go: internal/memory/fusion.go:466-468 and block_recall.go:263.
// Bound the VECTOR route before union with all symbol matches (fusion.go:473).
// This is not a cap on symbol candidates or on the eventual fused result.
const vectorCandidates = fusedCandidates && vectorScores instanceof Map
? new Set([...vectorScores].sort((a, b) => b[1] - a[1]).slice(0, largeTopK).map(([id]) => id))
: new Set()
if (fusedCandidates) {
for (const block of store.symbolCandidates()) {
const symbolMatch = symbols.some((symbol) =>
String(block.text_content ?? '').toLowerCase().includes(symbol.toLowerCase()))
const vectorSource = vectorCandidates.has(block.id)
if (!symbolMatch && !vectorSource) continue
if (pool.has(block.id)) continue
pool.set(block.id, {
id: block.id, text: block.text_content, source: block.source,
fingerprint: block.fingerprint ?? '', hasVector: block.has_vector === 1,
dimension: Number(block.vector_dim ?? 0), lexical: 0,
})
}
}
// ── F2: space filter, applied to the SYMBOL path too ────────────────────
//
// Text matching does not depend on the vector space, so a block written under
@ -342,6 +401,21 @@ export function recallBlocks(store, query, { topK = 5, poolSize = 50, fingerprin
}
const scored = []
// ── D2: every drop must be countable ─────────────────────────────────
//
// Three ways a candidate can disappear without becoming a hit, and until now
// only the first two (space / width, both upstream) were reported:
//
// · gatedByScore failed the V5 MinScore gate on its own vector score
// · gatedNoEvidence neither a usable vector score nor a symbol match
// · truncated survived every gate, fell outside the caller's topK
//
// Without these, "6 candidates, 1 returned" is indistinguishable from "the
// space only holds 1 block" — the operator sees skipped=0 and concludes
// nothing was dropped. Measured before this change: 6 blocks, minScore 0.99 →
// hits=0, skipped=0, skippedDim=0.
let gatedByScore = 0
let gatedNoEvidence = 0
for (const candidate of pool.values()) {
const haystack = candidate.text.toLowerCase()
// Port of `memory.SymbolScore` (internal/memory/fusion.go:284-308):
@ -411,24 +485,80 @@ export function recallBlocks(store, query, { topK = 5, poolSize = 50, fingerprin
// Non-exact candidates use vector*0.7 + symbol*0.3. A missing vector score
// is the symbol-only fallback and must not invent a vector contribution.
const hasVectorScore = vectorScores instanceof Map && vectorScores.has(candidate.id)
&& (!fusedCandidates || vectorCandidates.has(candidate.id))
const vectorScore = hasVectorScore ? vectorScores.get(candidate.id) : 0
const score = exact
? exactScore
: vectorScores instanceof Map && hasVectorScore
? vectorScore * 0.7 + nonExact * 0.3
: nonExact + boundaryBonus * 1e-3
: fusedCandidates
// ported from go: internal/memory/fusion.go:430-434.
? vectorScore * 0.7 + coverage * 0.3
: vectorScores instanceof Map && hasVectorScore
? vectorScore * 0.7 + nonExact * 0.3
: nonExact + boundaryBonus * 1e-3
// ── V3: attribution ──────────────────────────────────────────────
// Ported from Go: internal/memory/fusion.go:34-46 (FusedHit's VectorHit /
// SymbolHit / Why) and :394-398 (the symbol branch that fills them).
//
// ★ Every value comes from the branch that already ran — nothing is a
// constant decided per shape. Go's own note (`:38-39`) is explicit that 0
// means "did not hit", so the two signals stay independently observable:
// a block can carry a vector score and no symbol match, or the reverse.
const vectorHit = vectorScore
const symbolHit = exact ? 1.0 : coverage
// ── V5: asymmetric MinScore gate ─────────────────────────────────
// Ported from Go: internal/memory/fusion.go:541-554.
//
// The threshold applies PER PATH, never to the fused score: the fused score
// is a mix of three different scales (exact pin ≥1.0, pure-symbol 0..0.3,
// pure-vector 0..0.7), so one threshold over it would either drop every
// symbol hit or let every vector hit through. Go records exactly that
// mistake and its two failing criteria (:528-529).
//
// exact hit → never gated (the pin is decisive)
// has vector → gated by MinScore against the VECTOR score
// symbol only → not gated by MinScore; dropped only when it matched
// nothing at all (go :549-553)
//
// ★ The branch reads `hasVectorScore`, the CANDIDATE-SOURCE flag, not the
// presence of the `vectorHit` field. V3 now puts that field on every record,
// so existence-based branching would route symbol-only hits into the vector
// gate and the "asymmetric" property would silently disappear.
if (fusedCandidates) {
if (!exact) {
if (hasVectorScore && vectorScore > 0) {
if (minScore > 0 && vectorScore < minScore) {
gatedByScore += 1
continue
}
} else if (!(symbolHit > 0)) {
gatedNoEvidence += 1
continue
}
}
}
scored.push({
id: candidate.id,
text: candidate.text,
source: candidate.source,
score,
exact,
why: exact ? `exact:${matched.filter(isNumericSymbol).join(',')}` : matched.length > 0 ? `word:${matched.join(',')}` : 'lexical',
vectorHit,
symbolHit,
why: vectorScores instanceof Map
? `vector:${vectorHit};symbol:${symbolHit};${exact ? 'exact' : matched.length > 0 ? `word:${matched.join(',')}` : 'lexical'}`
: exact ? `exact:${matched.filter(isNumericSymbol).join(',')}` : matched.length > 0 ? `word:${matched.join(',')}` : 'lexical',
})
}
scored.sort((a, b) => (b.score === a.score ? a.id.localeCompare(b.id) : b.score - a.score))
return withSkipped(scored.slice(0, topK), skipped, skippedDim)
// ported from go: internal/memory/fusion.go:557-559 — final caller topK,
// after candidate union, scoring, filtering and the asymmetric gate.
const drops = {
gatedByScore, gatedNoEvidence, skippedUnscoreable,
truncated: Math.max(0, scored.length - topK),
}
return withSkipped(scored.slice(0, topK), skipped, skippedDim, drops)
}
/**
@ -442,9 +572,17 @@ export function recallBlocks(store, query, { topK = 5, poolSize = 50, fingerprin
*
* @param {Array<object>} hits
* @param {number} skipped blocks excluded by the space filter
* @param {number} [skippedDim] blocks excluded because their width did not match
* @param {{gatedByScore?: number, gatedNoEvidence?: number, truncated?: number}} [drops]
* the three downstream ways a candidate can disappear. Separate numbers on
* purpose: "failed the threshold" and "had nothing to score" have opposite
* remedies (lower the threshold vs. re-embed), and a merged count could not be
* acted on. `truncated` is the caller's own topK, which is not a defect — it is
* reported so that `hits + all counter fields` reconciles with the candidate
* pool.
* @returns {Array<object> & {skipped: number}}
*/
function withSkipped(hits, skipped, skippedDim = 0) {
function withSkipped(hits, skipped, skippedDim = 0, drops = {}) {
Object.defineProperty(hits, 'skipped', {
value: skipped,
enumerable: false,
@ -457,6 +595,23 @@ function withSkipped(hits, skipped, skippedDim = 0) {
enumerable: false,
configurable: true,
})
// ── D2: the three downstream drop counts ─────────────────────────────
// Also non-enumerable, for the same reason as `skipped`: the phase-A golden
// fixture compares result arrays.
for (const [key, label] of [
['gatedByScore', 'skippedScore'],
['gatedNoEvidence', 'skippedNoEvidence'],
['truncated', 'truncated'],
// D2's second hole: blocks dropped BEFORE the candidate pool existed. They
// are not "gated" (no gate ever saw them) — they could not be scored at all.
['skippedUnscoreable', 'skippedUnscoreable'],
]) {
Object.defineProperty(hits, label, {
value: Number.isInteger(drops[key]) ? drops[key] : 0,
enumerable: false,
configurable: true,
})
}
return hits
}
@ -471,7 +626,7 @@ function withSkipped(hits, skipped, skippedDim = 0) {
*
* @param {Array<object>} hits
* @param {string|null} mode one of `no_embedder` / `embed_error` / `aborted` /
* `vector_ready_unused` / null
* `no_vector_space` / `vector_ready_unused` / null
* @param {Error} [error]
* @returns {Array<object>} the same array
*/
@ -513,10 +668,17 @@ function withDiagnosis(hits, mode, error) {
/**
* Vectors-assisted recall.
*
* Phase B group A ships only the *seam*, not the fusion: with no embedder (or a
* failing one) this is exactly `recallBlocks`. Group V fills in
* `options.vectorScores`; until then the vector call is deliberately not wired
* into scoring, so this wrapper cannot perturb phase A behaviour.
* With no embedder (or a failing one) this is exactly `recallBlocks`. With a
* healthy one it scores the query vector against the space's stored vectors and
* hands `recallBlocks` BOTH `vectorScores` and `minScore`, which together open
* the fused candidate assembly, the V1 exact-band tie adjustment, the V2 weighted
* blend and the V5 asymmetric gate.
*
* ★ Why one call and not two: the symbol and vector routes are fused into ONE
* scoring pass so that a block the symbol path missed can still be admitted by
* its vector score (V5) and so that the candidate pool is never truncated before
* fusion (V7). Calling `recallBlocks` twice and merging afterwards cannot
* reproduce either property.
*
* Degradation is reported on the returned array (`arr.degraded`), never by
* throwing — the recall main path must survive a dead embedding service
@ -594,12 +756,275 @@ export async function recallBlocksHybrid(store, embedder, query, options = {}) {
new Error(`embedder returned dim ${queryVector.length}, declared ${info.dimension}`))
}
// Group V wires `vectorScores` into scoring. Keeping the call here (and the
// scoring seam closed) means group A ships a verified no-op on the hot path.
// Group A only computes the vector; it is not in scoring yet. Reporting
// 'hybrid' here would be a lie until group V wires it in, and a misleading
// label is worse than no label — rename it when V actually participates.
return withDiagnosis(recallBlocks(store, query, scoped), 'vector_ready_unused')
// ── the wiring ────────────────────────────────────────────────────────────
//
// This is the line P1 exists for. Until now the query vector was computed,
// dimension-checked, and then DISCARDED: the call below passed `scoped`, which
// carries `topK` / `signal` / `fingerprint` / `dimension` and nothing else. So
// every V1–V7 scoring branch in `recallBlocks` was reachable only from unit
// tests that hand-built a `vectorScores` Map; production recall was pure
// symbol/FTS.
//
// ★ BOTH options have to be passed, and the reason is structural rather than
// stylistic:
// · `vectorScores` alone reaches only the V1 exact-band tie adjustment
// (`:438`), because the fused candidate assembly at `:233` is gated on
// `Number.isFinite(minScore)`; without it the vector route can never
// admit a block the symbol path missed, which is exactly what V5 needs.
// · `minScore` alone does nothing at all, because `:299` additionally
// requires `vectorScores` to BE a Map before it builds vector candidates.
// The two are one seam split across three guards, so they travel together.
//
// ★ Scheme A (score the existing space) over scheme B (teach `recallBlocks` to
// accept a query vector). B would change a signature that phase A's golden
// fixture and nine call sites depend on being synchronous and vector-free, and
// it would move cosine arithmetic into a function whose contract is "store +
// query text". A leaves `recallBlocks` completely untouched: the seam it
// already exposes is the seam we drive.
//
// Cost, measured on this box before choosing (2000 blocks x 512 dims, the
// profile's default space width): narrow `SELECT id, vector` 21 ms + parse and
// cosine 60 ms ≈ 81 ms per recall; at 8000 blocks ≈ 310 ms. That is affordable
// because the scan is bounded by the space and width filters below and runs
// once per TURN (pre-step), not per candidate. Two design choices keep it from
// growing worse than linear:
// · the space/width filters are pushed into SQL, so blocks from a retired
// space or a wrong width are never decoded;
// · a space holding no comparable vectors scores nothing at all, so a database
// with no embedded blocks (every phase-A store) pays one indexed SELECT that
// returns no rows and takes the symbol path untouched.
// The remaining risk is a very large single space, which is a ceiling to
// measure before raising `topK`/space size — not a reason to ship a vector path
// that never runs. Note the fail-closed consequence (§0.2/F5): blocks whose
// fingerprint is empty are NOT scored, by design.
//
// ★ This does not introduce a NEW order of cost. `recallBlocks` already walks
// every block for symbol union when `minScore` is finite (the `fusedCandidates`
// step, `allBlocks(-1)`), so the fused path was always O(total blocks); what is
// added here is one narrow indexed read plus the cosine arithmetic, and the
// decode is bounded to the active space and width instead of the whole table.
// ── D2: blocks that could not be scored at all ──────────────────────────
//
// `scoreVectorCandidates` skips a row when it cannot produce a number: bad
// JSON, wrong width, a non-finite component, a zero-norm vector, or — since C4
// — a vector that centring collapses to dust. Such a block never enters
// `vectorScores`, so `recallBlocks` never puts it in the candidate pool, so
// none of the downstream counters can see it.
//
// Measured: three identical blocks with centring active produced
// `hits=0 skipped=0 skippedDim=0 skippedScore=0 skippedNoEvidence=0 truncated=0`
// — the whole space vanished while every counter read zero. That is exactly the
// silent branch D2 forbids, one stage EARLIER than where D2 first looked.
//
// The counter is threaded by reference so it can be reported on both the fused
// path and the early `no_vector_space` return.
const unscoreable = { count: 0 }
const vectorScores = scoreVectorCandidates(store, queryVector, scoped, unscoreable)
const vectorRoute = vectorScores.size > 0
if (!vectorRoute) {
// An empty map would still open the fused candidate assembly (minScore is
// finite) and then admit symbol matches with a vector weight of 0 — the
// `0.7*0 + 0.3*coverage` regression V2 forbids. Reporting the honest reason
// and taking the untouched symbol path is both correct and cheaper.
return withDiagnosis(
recallBlocks(store, query, { ...scoped, skippedUnscoreable: unscoreable.count }),
'no_vector_space',
)
}
// `minScore` is only defaulted when the caller did not decide: an explicit
// `0` is a deliberate "no threshold" and must survive (it is how the V7 and
// pool-truncation criteria disable the gate).
const minScore = Number.isFinite(scoped.minScore) ? scoped.minScore : DEFAULT_MIN_SCORE
return withDiagnosis(
recallBlocks(store, query, { ...scoped, vectorScores, minScore, skippedUnscoreable: unscoreable.count }),
// 'hybrid' from here on is a statement of fact, not an aspiration: the vector
// route contributed at least one scoreable candidate to the same call that
// produced these hits. Group A shipped `vector_ready_unused` precisely so this
// rename could not happen before the wiring existed.
'hybrid',
)
}
/**
* Cosine similarity of the query vector against every block in its space.
*
* Returns the `Map<id, score>` that `recallBlocks` consumes. An empty Map means
* "this space has nothing to score", which the caller must treat as the symbol
* path rather than as a vector route with score 0 everywhere.
*
* Vectors are stored as JSON text (`memory_blocks.vector`, per the DDL comment:
* "vector is a JSON array, matching HA's memory_blocks.vector"). A row that fails
* to parse is SKIPPED rather than thrown on: one corrupt payload must not take
* the whole recall down (decision §0.4), and the migration path already records
* corrupt widths as a negative `vector_dim` so they are filterable.
*
* ★ Centering lives at the BOTTOM of this function (see the C4 block by the
* scoring loop): the query is centred once, and each block vector is centred by
* the same mean inside the loop. Nothing above this line knows about it.
*
* ★ Not L2-normalised on the way out. Cosine already divides by both norms, and
* `putBlock` does not promise unit vectors, so normalising would change nothing
* except to hide a zero vector behind a NaN.
*
* @param {import('./store.js').BlockStore} store
* @param {number[]} queryVector already dimension-checked by the caller
* @param {{fingerprint?: string, dimension?: number}} options
* @returns {Map<string, number>}
*/
function scoreVectorCandidates(store, queryVector, { fingerprint, dimension } = {}, unscoreable = { count: 0 }) {
const scores = new Map()
// Without a space identity there is nothing to compare against: the vectors in
// the table may belong to several incompatible spaces (§0.1), and a cosine
// across two of them is a number without a meaning.
if (typeof fingerprint !== 'string' || fingerprint === '') return scores
const rows = store.vectorCandidates(fingerprint, dimension)
if (rows.length === 0) return scores
// ── C4 · 中心化校正 ────────────────────────────────────────────────────
//
// 一次索引查询取本空间的中心,仅当它被判定为各向异性、且宽度与查询一致
// 时才启用。维度不符时**不校正也不报错**:按错误维度截断或补齐造出来的
// 余弦,看起来像分数,实际没有意义。
//
// 门控条件照 C2:`isAnomalous === false` 的空间不校正。这不是优化开关。
// 各向同性空间的均值范数本来就低,减掉它只会放大噪声 —— 实测 8 块正交
// (mean norm 0.3536)在无条件校正下命中数 8 → 1,且因为 V5 的结构性门
// (vectorScore <= 0 且 symbolHit === 0 直接丢弃)无法靠 minScore 调参救回。
// 所以这条 if 是正确性的一部分,不是性能的一部分。
//
// ★ loadCentroid 是纯读的一次索引查询,不进写路径、不触发重建。
// 重建挂在写入侧(lib/index.js 的 commit 之后),因为它是 O(n·d),
// 放进每轮 pre-step 就是灾难;C3-k 也明令计数查询不得进热路径。
const centroid = store.loadCentroid(fingerprint)
// ★ C4's order: a round that judges the centroid STALE still returns results,
// but with UNCORRECTED vectors. The verdict is read from the row (`stats.stale`),
// written by the write path — the recall side must not run `centroidStatus`,
// whose first line is a COUNT (C3-k).
//
// Why, stated honestly: a mean built from a fraction of the space describes a
// space that no longer exists, and applying it is unjustifiable on principle.
// An attempt to measure the damage could not separate the cases: with a
// near-duplicate late corpus, scores saturate near 0.99 whether the mean is
// stale (0.997) or absent (0.994); only a freshly rebuilt mean spread them
// (0.978..0.970). So this is a safety choice; the measurable win belongs to
// keeping the centroid FRESH, not to skipping it.
// ★ `!centroid.stats?.stale` and NOT `centroid.stats?.stale !== true`:
// the flag is written as the JSON number 1 by `markCentroidStale`, and in JS
// `1 !== true` — a strict comparison across types is always true, so the strict
// form silently made this guard a no-op. Caught by a fixture where centring is
// observable; the suite was 305/305 green with the bug in place.
const mean = centroid?.isAnomalous === true
&& centroid.dim === queryVector.length
&& !centroid.stats?.stale
? centroid.mean
: null
const centeredQuery = mean === null ? null : centerVector(queryVector, mean)
// `mean !== null` 但 `centeredQuery === null` 表示减完是零向量(没有方向),
// 同样退回未校正路径,而不是拿零向量去算余弦。
const centering = centeredQuery !== null
const scoringQuery = centering ? centeredQuery : queryVector
let queryNorm = 0
for (const value of scoringQuery) queryNorm += value * value
queryNorm = Math.sqrt(queryNorm)
if (!(queryNorm > 0)) return scores
for (const row of rows) {
let vector
try {
vector = JSON.parse(String(row.vector ?? ''))
} catch {
unscoreable.count += 1
continue
}
if (!Array.isArray(vector) || vector.length !== queryVector.length) {
unscoreable.count += 1
continue
}
// ★ 两边都中心化。只处理一边 = 一边带公共分量、另一边不带,算出来的
// 余弦是两种量纲的混合(HA centroid_store.go:439-442 的原话)。
// 实测:只减 query 时,C4b 那个 5 块 fixture 上同值对的余弦从 1.0 掉到
// 0.1897(余弦对称,只减块侧同值)—— 那就不是在量夹角了。
const scoringVector = centering ? centerVector(vector, mean) : vector
if (scoringVector === null) {
// ★ C4's dust guard fired: the block sits on the space mean, so it has no
// direction to compare against. Counted explicitly, because otherwise this
// block disappears BEFORE the candidate pool is assembled and no downstream
// counter can see it — measured as "the whole space vanished while every
// counter read zero".
unscoreable.count += 1
continue
}
let dot = 0
let norm = 0
for (let index = 0; index < scoringVector.length; index += 1) {
const value = Number(scoringVector[index])
// A non-numeric element makes the whole score meaningless. Skip the row
// instead of letting NaN propagate into the sort, where it would silently
// randomise the order (NaN comparisons are always false).
if (!Number.isFinite(value)) {
dot = Number.NaN
break
}
dot += value * scoringQuery[index]
norm += value * value
}
if (!Number.isFinite(dot) || !(norm > 0)) {
unscoreable.count += 1
continue
}
scores.set(row.id, dot / (queryNorm * Math.sqrt(norm)))
}
return scores
}
/**
* 减均值再 L2 归一化(HA `Centroid.CenterVector`,`centroid.go:185-188`)。
*
* ★ 归一化不能省。减去均值会改变模长,不归一化就是拿两把不同的尺子量。
* 实测:同一对相同向量,两侧都归一化时余弦恒为 1;只减不归一时它会变成 0.61
* —— 看起来仍是个「分数」,但已经不是在算夹角了。
*
* ★ 零向量返回 `null`。减完均值恰好没有方向(向量就躺在中心上)时,
* 余弦无定义。调用方必须跳过这一行,而不是拿零向量去算出一个 0 分
* 再让它参与排序 —— 0 分在 V5 的门里等于「未命中」,会把一次真实的
* 无法比较伪装成一次「不相关」。
*
* @param {number[]} vector
* @param {number[]} mean the space centroid mean, same length
* @returns {number[]|null} centered unit vector, or null when it has no direction
*/
function centerVector(vector, mean) {
const out = new Array(vector.length)
let norm = 0
for (let index = 0; index < vector.length; index += 1) {
const delta = Number(vector[index]) - mean[index]
// A non-numeric component poisons the whole direction: bail out rather than
// let NaN reach the sort (NaN comparisons are always false ⇒ silent shuffle).
if (!Number.isFinite(delta)) return null
out[index] = delta
norm += delta * delta
}
norm = Math.sqrt(norm)
// ★ Not `norm > 0`.
//
// A block that IS the mean differs from it only by floating-point dust
// (~1e-17). `norm > 0` accepts that dust, and the renormalisation below turns
// it into a unit vector pointing in an arbitrary direction — which then scores
// against the query as if it were signal. Measured on a space of two identical
// blocks: cosine 0.2183 out of dust, where the raw cosine was 0.8608.
//
// Treating dust as "no direction" also matches the Go side, where the cosine of
// a zero vector is 0 rather than a random angle.
if (!(norm > CENTER_EPSILON)) return null
for (let index = 0; index < out.length; index += 1) out[index] /= norm
return out
}
/**
@ -667,6 +1092,16 @@ export function degradationNotice(hits) {
// may say "matches below"; nothing === false => the branch must point at them.
const nothing = !Array.isArray(hits) || hits.length === 0
switch (mode) {
case 'hybrid':
// NOT a degradation: this is the healthy, fully-wired path (symbol + vector
// fused in one scoring pass). It carries no notice for the same reason `ok`
// would not — there is nothing the model needs to know.
//
// ★ Kept as an explicit case rather than relying on the `default:` branch.
// `default:` exists to make an UNKNOWN mode fail loudly-safe ("unknown, not
// absent"), so routing the healthiest mode through it would print a false
// degradation notice on every single turn that had a vector hit.
return ''
case 'no_embedder':
// Permanent configuration state, not a per-turn failure. The constant index
// (systemPrompt.context -> renderIndex) already tells the model that block
@ -698,6 +1133,14 @@ export function degradationNotice(hits) {
// yet. Not something the model needs to hear about — but keeping the case
// explicit means a future mode cannot silently fall through to ''.
return ''
case 'no_vector_space':
// A space IS configured and the embedder answered, but no block in that
// space carries a usable vector (a fresh space, or every row pre-dates the
// embedder). Unlike `no_embedder` this is a transitional DATA state, and it
// is the one an operator most often mistakes for "semantic recall is on".
// Still silent for the model: the fallback it received is the correct one,
// and `stablemem_stats` is where the operator is told why.
return ''
default:
// A mode nobody anticipated. "Not found" must not read as "not recorded":
// this is the ONLY path a future mode can take, so it is the worst possible
@ -707,3 +1150,46 @@ export function degradationNotice(hits) {
: `[Memory] Recall degraded (${mode}); matches below may be incomplete.`
}
}
/** Fixed text heuristic matching dsh-token-meter: 4 chars/token plus framing. */
export function estimateMemoryTokens(text) {
return Math.ceil(String(text ?? "").length / 4) + 8
}
/** Estimate rendered hit cost from its note body, not serialized JSON. */
export function estimateRecallHitTokens(hit) {
const body = String(hit?.text ?? "").replace(/\s+/g, " ").trim().slice(0, 200)
return estimateMemoryTokens(body) + 20
}
/** Select a recall count under explicit, host-derived, or fallback budget. */
export function segmentRecallBudget(hits, options = {}) {
const topK = Number.isFinite(options.topK) ? Math.max(1, Math.floor(options.topK)) : 3
const explicit = Number.isFinite(options.budgetTokens) && options.budgetTokens > 0 ? options.budgetTokens : undefined
const window = Number.isFinite(options.contextWindow) && options.contextWindow > 0 ? options.contextWindow : undefined
const source = explicit !== undefined ? "explicit" : window !== undefined ? "host" : "fallback"
const budget = explicit ?? (window === undefined ? 4096 : Math.max(1, Math.floor(window * 0.8)))
const rows = Array.isArray(hits) ? hits : []
if (rows.length === 0) return { hits: rows, budgetTokens: budget, source, singleHitOverBudget: false }
const costs = rows.map(estimateRecallHitTokens)
const average = costs.reduce((sum, cost) => sum + cost, 0) / costs.length
const count = Math.max(1, Math.min(topK, Math.floor(budget / Math.max(1, average))))
const retained = rows.slice(0, count)
// ★ `slice` copies indexed elements only. `degraded` / `degradedError` /
// `skipped` are NON-enumerable by design (so the phase-A golden fixture and
// every serialised log keep their shape), which means a plain slice silently
// drops them — and the degradation notice would then never reach the model.
// Re-attach every out-of-band channel the recall path is known to carry.
for (const key of ['degraded', 'degradedError', 'skipped', 'skippedDim']) {
if (Object.prototype.hasOwnProperty.call(rows, key)) {
Object.defineProperty(retained, key, {
value: rows[key],
enumerable: false,
configurable: true,
})
}
}
const singleHitOverBudget = costs[0] > budget
Object.defineProperty(retained, "budgetNotice", { value: singleHitOverBudget ? `[Memory] Top note estimate ${costs[0]} tokens exceeds memory budget ${budget}; retained without truncation.` : count < rows.length ? `[Memory] Recall limited to ${count} note(s) by memory budget ${budget}.` : "", enumerable: false })
return { hits: retained, budgetTokens: budget, source, singleHitOverBudget }
}

View File

@ -1,3 +1,11 @@
function safeParseJson(value) {
try { return JSON.parse(value ?? '{}') } catch { return null }
}
function defaultCentroidSchedule(fn, delayMs) {
setTimeout(fn, delayMs)
}
/**
* SQLite storage for block memory: the three tables required by the MVP slice.
*
@ -299,6 +307,31 @@ export function blockId(text) {
export class BlockStore {
/** @type {DatabaseSync | null} */
#db = null
/**
* When each space's rebuild was last SCHEDULED (throttle state).
*
* ★ In-memory on purpose, with the trade-off written down.
*
* The obvious "derive it from the DB" candidate is `graph_centroid.built_at`,
* and it is the wrong column: `built_at` records when a centroid was
* successfully BUILT, while a throttle needs to know when a rebuild was last
* SCHEDULED. The two diverge exactly in the case the throttle exists for —
* a rebuild that throws. With `built_at` as the throttle, a failing rebuild
* leaves it untouched, so every subsequent commit schedules again and fails
* again: unbounded write amplification on the failure path. Deriving a
* correct value would need a second timestamp column ("last scheduled") to be
* written on every attempt — a schema change for a per-process optimisation.
*
* The cost of keeping it here: a process restart empties the map, so the first
* drift check after a restart may schedule one extra rebuild. That costs one
* bounded, asynchronous O(n*d) pass (measured 85.5 ms at 2000 blocks x 512
* dims) and it happens at most once per space per process lifetime. Against
* that: a wrong-derived DB throttle would re-schedule on EVERY commit after a
* failed rebuild.
*
* @type {Map<string, number>}
*/
#centroidRebuiltAt = new Map()
/** @type {string} */
#path
@ -531,6 +564,279 @@ export class BlockStore {
return 0.6
}
/** Minimum number of contribution vectors before a "mean direction" exists.
*
* A one-vector mean IS that vector, so its norm is exactly 1.0 and it clears
* {@link BlockStore.MEAN_THRESHOLD} by construction — a young space would be
* declared anisotropic on the strength of a single block. Subtracting that
* "mean" from its own block yields the zero vector, which the scorer drops, so
* centering would silently remove the only block the space contains. Measured
* on the C4 fixture: right after the first commit the centroid reported
* `vectorCount=1 meanLength=1.0000` and a recall that returns 3 rows without a
* centroid returned 1 with it. Two is the smallest number for which the mean
* is a property of the SET rather than of one member. */
static get MIN_SAMPLE() {
return 2
}
/** Relative change in the vectored-block count that marks a centroid stale.
* Ported from `centroidRebuildRatio` (internal/memory/centroid_store.go:55).
* The Go comment pins the rationale: below 10% the shift does not move the
* principal direction, and being more sensitive than 20% makes similarity
* drift with every write — a non-reproducible recall cannot serve regression. */
static get CENTROID_REBUILD_RATIO() {
return 0.2
}
/** Minimum interval between two scheduled rebuilds of the same space.
* Ported from `centroidRebuildWait` (internal/memory/centroid_store.go:324). */
static get CENTROID_REBUILD_WAIT_MS() {
return 10 * 60 * 1000
}
/** Delay before the scheduled rebuild runs, so the current read settles.
* Ported from `time.Sleep(2 * time.Second)` (internal/memory/centroid_store.go:338). */
static get CENTROID_REBUILD_DELAY_MS() {
return 2000
}
/**
* Number of blocks in `fingerprint`'s space that currently carry a vector.
*
* This is `countVectoredBlocksLocked` (internal/memory/centroid_store.go:178):
* the DENOMINATOR side of the staleness ratio. It is deliberately NOT called
* from `blockStates` (this file's pre-step hot path) — it only feeds
* {@link centroidStatus}.
*
* @param {string} fingerprint
* @returns {number}
*/
countVectored(fingerprint) {
return this.#handle()
.prepare(
`SELECT COUNT(*) AS n FROM memory_blocks
WHERE fingerprint = ? AND vector != '' AND vector != 'null'`,
)
.get(String(fingerprint ?? '')).n
}
/**
* Centroid staleness FACTS for one space (C3).
*
* Ports the two Go call sites that answer different questions:
*
* · recall path (internal/memory/centroid_store.go:179-204): a centroid
* built from zero samples is simply UNUSABLE — no staleness judgement.
* · status path (internal/memory/centroid_store.go:425-433): the same row
* IS reported stale when blocks now carry vectors.
*
* They are separate fields (`available` / `stale`) rather than one merged flag
* because merging them makes the report self-contradictory.
*
* ★ Why the rebuild is scheduled and never run inline (Go :196-204): Go holds
* `g.mu.RLock` for this whole function and `RebuildCentroid`/`SaveCentroid`
* need the write lock, so an inline rebuild self-deadlocks (their test hit a
* 180 s timeout). The JS equivalent is not a lock but a transaction: node:sqlite
* refuses a nested `BEGIN IMMEDIATE` ("cannot start a transaction within a
* transaction") and throws SQLITE_BUSY when a statement is still iterating. So
* this method only REGISTERS a rebuild and returns "no centering this round".
*
* ★ C5: every branch now names WHY centring was disabled, in `reason`.
* Without it the four causes were only inferable from field combinations —
* `no_centroid` and `dim_mismatch` both report `stale: null`, and `isotropic`
* is indistinguishable from a healthy space except by `available`. That is
* fine for a caller that only asks "may I centre?", and useless for an operator
* asking "why is semantic recall not correcting?".
*
* @param {string} fingerprint
* @param {{queryDim?: number, now?: () => number,
* schedule?: (fn: () => void, delayMs: number) => void}} [options]
* `now`/`schedule` are injectable so the throttle and the delay are testable
* against the real branch logic instead of a stubbed global timer.
* @returns {{available: boolean, stale: boolean|null, dim: number|null,
* vectorCount: number|null, currentBlocks: number, delta: number|null,
* rebuildScheduled: boolean, reason: CentroidReason}}
*
* `reason` is one of `ok` / `no_centroid` / `dim_mismatch` / `zero_sample` /
* `too_few_samples` / `stale` / `isotropic`. `ok` is the only one that coincides
* with `available: true`. Two of these go beyond the four the plan names:
* `zero_sample` is a Go branch (`centroid_store.go:179-185`), and
* `too_few_samples` exists because MIN_SAMPLE makes a one-vector space
* non-anisotropic — reporting it as `isotropic` would send an operator looking
* for the wrong remedy.
* @throws when the stored centroid metadata is corrupt (`dim <= 0`,
* internal/memory/centroid_store.go:165-167)
*/
centroidStatus(fingerprint, options = {}) {
const { queryDim, now = Date.now, schedule = defaultCentroidSchedule } = options
const currentBlocks = this.countVectored(fingerprint)
const row = this.#handle()
.prepare('SELECT dim, vector_count, stats FROM graph_centroid WHERE fingerprint = ?')
.get(String(fingerprint ?? ''))
if (row === undefined) {
// :153-155 — no row at all is a normal state (Saved=false), not an error.
return {
available: false, stale: null, dim: null, vectorCount: null,
currentBlocks, delta: null, rebuildScheduled: false,
reason: 'no_centroid',
}
}
const dim = Number(row.dim)
if (dim <= 0) {
// :165-167 — corrupt metadata is an ERROR, not a silent "no centering".
throw new Error(`dsh-memory: centroid dim is ${dim} (corrupted?)`)
}
const vectorCount = Number(row.vector_count)
if (Number.isInteger(queryDim) && queryDim > 0 && dim !== queryDim) {
// :168-170 — wrong width for this query: unusable, not stale.
return {
available: false, stale: null, dim, vectorCount, currentBlocks,
delta: null, rebuildScheduled: false,
reason: 'dim_mismatch',
}
}
// :186-189 — (current − built) / built, sign discarded.
const raw = vectorCount > 0 ? (currentBlocks - vectorCount) / vectorCount : null
const delta = raw === null ? null : Math.abs(raw)
if (vectorCount <= 0) {
// :179-185 (recall) vs :431-432 (status): NOT available either way, but the
// status side still calls it stale when blocks exist now.
return {
available: false,
stale: currentBlocks > 0,
dim, vectorCount, currentBlocks, delta, rebuildScheduled: false,
reason: 'zero_sample',
}
}
// :190 — STRICTLY greater; exactly 20% is not stale.
if (delta > BlockStore.CENTROID_REBUILD_RATIO) {
const rebuildScheduled = this.#scheduleCentroidRebuild(fingerprint, { now, schedule })
return {
available: false, stale: true, dim, vectorCount, currentBlocks, delta, rebuildScheduled,
reason: 'stale',
}
}
const stats = safeParseJson(row.stats)
// ★ `too_few_samples` comes BEFORE the isotropic verdict, and the order is the
// whole point of C5.
//
// Since MIN_SAMPLE, a one-vector space has `isAnomalous: false` (a single
// sample is not a direction). Without this branch it therefore fell into
// `isotropic` — which tells the operator "this space has no common direction,
// stop expecting centring". The truth is the opposite: centring is waiting for
// more blocks. Two reasons with OPPOSITE remedies, and reporting the wrong one
// sends the operator to the wrong subsystem.
if (vectorCount < BlockStore.MIN_SAMPLE) {
return {
available: false, stale: false, dim, vectorCount, currentBlocks, delta, rebuildScheduled: false,
reason: 'too_few_samples',
}
}
if (stats?.anomalous === false || !this.loadCentroid(fingerprint)?.isAnomalous) {
// :205-208 — built as isotropic: centering is not enabled for this space.
return {
available: false, stale: false, dim, vectorCount, currentBlocks, delta, rebuildScheduled: false,
reason: 'isotropic',
}
}
return {
available: true, stale: false, dim, vectorCount, currentBlocks, delta, rebuildScheduled: false,
reason: 'ok',
}
}
/**
* Projection of C3 facts for a caller that only asks "may I centre?".
*
* Go `LoadCentroid` returns `(nil,false,nil)` for every unusable center,
* including a zero-sample one (`internal/memory/centroid_store.go:179-185`), and
* does NOT expose the operator-only stale bit from `CentroidStatusOf:425-433`.
* This mirrors that: `available` answers the yes/no question, `stale` is forced
* to null, and `reason` still travels for anyone who needs to log why.
*
* ★ NOT the recall path's entry, despite the name. The scoring path uses
* `loadCentroid` (a single indexed SELECT) because this method calls
* `centroidStatus`, whose first line is `countVectored` — a COUNT that C3-k
* forbids on the per-turn pre-step path. Wiring this into recall would put a
* full-space count on every turn.
*
* So it is a diagnostic seam, in the same category as
* `StableMem#saveSpaceCentroid`: no production caller, kept because it pins the
* Go semantics (`assertC3RecallUnusable` in test/phase-b-h.test.js runs the same
* predicate against every C3 mutant). Recorded here because "defined, never
* called" looked identical to the three real dangling-wire bugs found in this
* branch — and the only difference worth writing down is this paragraph.
*
* @param {string} fingerprint
* @param {{queryDim?: number, now?: () => number,
* schedule?: (fn: () => void, delayMs: number) => void}} [options]
*/
centroidRecallState(fingerprint, options = {}) {
const status = this.centroidStatus(fingerprint, options)
return { ...status, available: status.available, stale: null }
}
/**
* Record that this space's centroid no longer describes it.
*
* The recall path must not judge staleness itself — C3-k forbids `centroidStatus`
* (and therefore `countVectored`) on the per-turn hot path. So the WRITE side,
* which already pays for that judgement on every commit, persists the verdict
* here and the recall side simply reads it.
*
* Cleared implicitly: `rebuildCentroid` → `saveCentroid` upserts the whole stats
* JSON without this key, so a successful rebuild wipes it. A FAILED rebuild
* leaves it set, which errs on the safe side: recall keeps using uncorrected
* vectors rather than a mean that is known to be out of date.
*
* Why it matters at all: applying a mean built from a fraction of the space is
* unjustifiable regardless of whether the damage is measurable — see the note in
* `recall.js` where the flag is honoured.
*
* @param {string} fingerprint
* @returns {boolean} whether a row was updated
*/
markCentroidStale(fingerprint) {
const space = String(fingerprint ?? '')
if (space === '') return false
const result = this.#handle()
.prepare("UPDATE graph_centroid SET stats = json_set(stats, '$.stale', 1) WHERE fingerprint = ?")
.run(space)
return result.changes > 0
}
/**
* Port of `scheduleCentroidRebuild` (internal/memory/centroid_store.go:327-347).
*
* @param {string} fingerprint
* @param {{now: () => number, schedule: (fn: () => void, delayMs: number) => void}} clock
* @returns {boolean} whether a rebuild was registered
*/
#scheduleCentroidRebuild(fingerprint, { now, schedule }) {
const last = this.#centroidRebuiltAt.get(fingerprint)
if (last !== undefined && now() - last < BlockStore.CENTROID_REBUILD_WAIT_MS) {
return false // :329-332 — inside the throttle window
}
this.#centroidRebuiltAt.set(fingerprint, now())
schedule(() => {
try {
this.rebuildCentroid(fingerprint)
} catch {
// :340-343 — a failed rebuild must not turn into a recall failure; the
// next status call sees the still-stale centroid and schedules again.
}
}, BlockStore.CENTROID_REBUILD_DELAY_MS)
return true
}
/**
* Rebuild the centroid for one vector space FROM ITS BLOCKS.
*
@ -595,7 +901,11 @@ export class BlockStore {
let meanLength = 0
for (const value of mean) meanLength += value * value
meanLength = Math.sqrt(meanLength)
const isAnomalous = meanLength >= BlockStore.MEAN_THRESHOLD
// ★ Both conditions matter. `meanLength >= threshold` alone would call a
// single-block space anisotropic (norm of a unit vector is 1.0), and the
// resulting centering subtracts a block from itself. See MIN_SAMPLE.
const isAnomalous = vectors.length >= BlockStore.MIN_SAMPLE
&& meanLength >= BlockStore.MEAN_THRESHOLD
this.saveCentroid({
fingerprint,
@ -869,6 +1179,36 @@ export class BlockStore {
}))
}
/**
* Candidate-pool source for the FUSED path: every block, without the vector text.
*
* `recallBlocks` walks the whole table here to union symbol matches with vector
* candidates, and the only thing it needs from the vector column is whether one
* EXISTS. `allBlocks(-1)` pays to materialise the entire payload — measured 79 MB
* at 8000 x 512 dims — to answer that one-bit question.
*
* Measured on this box (8000 blocks x 512 dims): `SELECT *` 242.8 ms against
* 22.2 ms for this read, and the whole recall went 624.0 -> 409.6 ms.
*
* ★ `has_vector` reproduces the caller's existing predicate exactly —
* `String(block.vector ?? '') !== ''` — including its quirk: the literal string
* 'null' counts as present here, while `vectorCandidates` excludes it. That
* divergence predates this method and is left alone; narrowing a read must not
* quietly change which blocks the fused path believes carry a vector.
*
* @returns {Array<{id: string, text_content: string, source: string,
* fingerprint: string, vector_dim: number, has_vector: number}>}
*/
symbolCandidates() {
return this.#handle()
.prepare(
`SELECT id, text_content, source, fingerprint, vector_dim,
(vector != '') AS has_vector
FROM memory_blocks ORDER BY created_at DESC, id`,
)
.all()
}
/**
* All blocks, newest first. Used as the LIKE fallback corpus and by tests.
* @param {number} [limit]
@ -879,6 +1219,51 @@ export class BlockStore {
.all(limit)
}
/**
* Candidate pool for the vector route: every block comparable to one space.
*
* Three narrowings, each of which `recallBlocks` would otherwise have to repeat
* (and would eventually repeat inconsistently — the exact half-coverage bug F2
* exists to stop):
*
* · `vector != ''` a row with no vector cannot score; decoding it
* would only produce a zero.
* · `fingerprint = ?` same space. Comparing vectors across spaces is
* meaningless, not merely inaccurate (§0.1/§0.2).
* · `vector_dim = ?` same width, when the caller names one. A stale
* row of the wrong width is a decode hazard, and
* `vector_dim` is maintained by a trigger, so
* this filter is exact and index-friendly.
*
* ★ `SELECT id, vector` and not `SELECT *`: the scoring pass needs the vector
* and the id, nothing else. Measured on this box at 2000 blocks x 512 dims,
* the narrow read is 21 ms against 70 ms for `allBlocks(-1)` — and the gap is
* dominated by the text columns, which grow with note length while vectors do
* not.
*
* ★ Why this lives in the store and not in `recall.js`: every other query in
* this file is a prepared statement over one table; a recall module that
* reached for `raw()` to hand-build SQL would put storage knowledge back into
* the scoring layer.
*
* @param {string} fingerprint the vector space to score against
* @param {number} [dimension] required width; omitted means "any width"
* @returns {Array<{id: string, vector: string, vector_dim: number}>} raw JSON text
*/
vectorCandidates(fingerprint, dimension) {
const space = String(fingerprint ?? '')
if (space === '') return []
const sql = Number.isInteger(dimension) && dimension > 0
? `SELECT id, vector, vector_dim FROM memory_blocks
WHERE fingerprint = ? AND vector != '' AND vector != 'null' AND vector_dim = ?`
: `SELECT id, vector, vector_dim FROM memory_blocks
WHERE fingerprint = ? AND vector != '' AND vector != 'null'`
const statement = this.#handle().prepare(sql)
return Number.isInteger(dimension) && dimension > 0
? statement.all(space, dimension)
: statement.all(space)
}
/** Close the handle. Idempotent (checkpoint V5). */
close() {
if (this.#db === null) return

View File

@ -5,29 +5,41 @@
"type": "module",
"main": "lib/index.js",
"exports": {
".": "./lib/index.js"
".": "./lib/index.js",
"./loop": "./lib/loop.js"
},
"files": [
"lib/*.js",
"lib/embed/*.js",
"lib/loop.js",
"lib/loop/*.js",
"lib/types/*.d.ts"
],
"engines": {
"node": ">=22.5.0"
},
"scripts": {
"test": "node --test test/store.test.js test/plugin.test.js test/loader.test.js test/phase-b-a.test.js test/phase-b-g.test.js test/phase-b-g-static.test.js test/phase-b-h.test.js",
"test": "node --test test/store.test.js test/packaging.test.js test/plugin.test.js test/loader.test.js test/phase-b-a.test.js test/phase-b-g.test.js test/phase-b-g-static.test.js test/phase-b-h.test.js test/phase-b-p1-vector-wiring.test.js test/phase-b-c4-centering.test.js test/phase-b-c5-reasons.test.js test/phase-b-v6-sides.test.js test/recall-accounting.test.js test/memory-log.test.js test/loop.test.js test/loop-ports.test.js test/loop-system-e2e.test.js",
"check": "node scripts/checkpoints.js"
},
"peerDependencies": {
"@deepseek-ai/cordis": "^4.0.5",
"@deepseek-ai/dsh-agent-loop": "0.2.1-rc.2",
"@deepseek-ai/dsh-llm": "0.2.1-rc.2",
"@deepseek-ai/dsh-session": "0.2.1-rc.2",
"@deepseek-ai/dsh-system-prompt": "0.2.1-rc.2",
"@deepseek-ai/dsh-tools": "0.2.1-rc.2"
},
"peerDependenciesMeta": {
"@deepseek-ai/dsh-system-prompt": { "optional": true },
"@deepseek-ai/dsh-tools": { "optional": true },
"@deepseek-ai/dsh-llm": { "optional": true }
"@deepseek-ai/dsh-system-prompt": {
"optional": true
},
"@deepseek-ai/dsh-tools": {
"optional": true
},
"@deepseek-ai/dsh-llm": {
"optional": true
}
},
"license": "MIT",
"private": true

View File

@ -14,6 +14,8 @@ set -euo pipefail
GLOBAL_ROOT="${1:-/usr/lib/node_modules}"
PEERS=(
cordis
dsh-agent
dsh-agent-loop
cosmokit
dsh-llm
dsh-session

154
test/loop-ports.test.js Normal file
View File

@ -0,0 +1,154 @@
/**
* Contract for the four ported loop modules under `lib/loop/`.
*
* These files are a faithful port of the stock loop's internals into this
* package, so nothing here tests their behaviour — that is `loop.test.js` and
* the phase-B suites. What this file pins is the **port interface**, because
* every one of these failures is silent at runtime:
*
* L1 all four modules import on their own (no ordering, no index.js).
* L2 the export names a replacement loop will import are present.
* L3 `inboxProjectionSchema.parse` actually rejects a malformed persisted
* row. This is the ONE thing the port re-implemented instead of copying
* (it replaces a zod schema with a hand-written shape check), so it is
* exactly the thing that can be a no-op and still look fine.
* L4 the inbox projection KEY and `stateVersion` are unchanged. A different
* key reads as "no pending input" for sessions written by the stock loop;
* a different version makes `stateSchema.parse` refuse those rows. Both
* are quiet data loss on `resume`, not a crash.
*
* @module dsh-memory/test/loop-ports
*/
import assert from 'node:assert/strict'
import test from 'node:test'
// ── L1: each module stands alone ─────────────────────────────────────────
//
// Imported dynamically rather than at the top of the file so the assertion is
// per-module: a single top-level `import` would make one broken port fail the
// whole file with "load failed" and hide which of the four broke.
const MODULES = {
'assistant-stream': '../lib/loop/assistant-stream.js',
inbox: '../lib/loop/inbox.js',
'runtime-context': '../lib/loop/runtime-context.js',
'tool-calls': '../lib/loop/tool-calls.js',
}
test('L1: all four loop ports import successfully on their own', async () => {
for (const [name, specifier] of Object.entries(MODULES)) {
const mod = await import(specifier)
assert.ok(mod !== null && typeof mod === 'object', `${name} must import to a module namespace`)
}
})
// ── L2: the export surface a replacement loop depends on ─────────────────
//
// Named explicitly, not counted: the point is that `executeToolCalls`,
// `ReactLoopInbox`, … are still exported under those names, because a rename
// here only shows up as `undefined is not a function` at step() time.
test('L2: loop ports export the names the replacement loop imports', async () => {
const toolCalls = await import(MODULES['tool-calls'])
assert.equal(typeof toolCalls.executeToolCalls, 'function', 'tool-calls must export executeToolCalls')
const assistantStream = await import(MODULES['assistant-stream'])
assert.equal(
typeof assistantStream.AssistantStreamAttempt,
'function',
'assistant-stream must export AssistantStreamAttempt',
)
const runtimeContext = await import(MODULES['runtime-context'])
assert.equal(
typeof runtimeContext.SystemPromptProjection,
'function',
'runtime-context must export SystemPromptProjection',
)
assert.equal(
typeof runtimeContext.RuntimeContextProjection,
'function',
'runtime-context must export RuntimeContextProjection',
)
const inbox = await import(MODULES.inbox)
assert.equal(typeof inbox.ReactLoopInbox, 'function', 'inbox must export ReactLoopInbox')
assert.ok(
inbox.inboxProjectionDefinition !== null && typeof inbox.inboxProjectionDefinition === 'object',
'inbox must export inboxProjectionDefinition',
)
assert.ok(
inbox.inboxProjectionSchema !== null && typeof inbox.inboxProjectionSchema === 'object',
'inbox must export inboxProjectionSchema',
)
})
// ── L3: the schema bites ──────────────────────────────────────────────────
//
// The projection registry calls `stateSchema.parse` on durable checkpoint rows,
// so "it has a parse function" proves nothing. These assertions run the parse
// against the two shapes that matter: a real state must come back, and a
// malformed row must throw — not return undefined, not coerce.
test('L3: inboxProjectionSchema.parse accepts a well-formed two-array state', async () => {
const { inboxProjectionSchema } = await import(MODULES.inbox)
const state = {
'next-turn': [{ id: 'm1', role: 'user' }],
'next-step': [{ id: 'm2', role: 'user' }],
}
assert.equal(inboxProjectionSchema.parse(state), state, 'parse must return the same object it validated')
assert.deepEqual(
inboxProjectionSchema.parse({ 'next-turn': [], 'next-step': [] }),
{ 'next-turn': [], 'next-step': [] },
'the init() shape (both empty) must be accepted',
)
})
test('L3: inboxProjectionSchema.parse throws on a malformed persisted row', async () => {
const { inboxProjectionSchema } = await import(MODULES.inbox)
const malformed = [
['null', null],
['a bare array', []],
['a string', 'inbox'],
['an object missing "next-step"', { 'next-turn': [] }],
['a non-array "next-turn"', { 'next-turn': {}, 'next-step': [] }],
['a non-array "next-step"', { 'next-turn': [], 'next-step': 'pending' }],
['the stock loop key casing', { nextTurn: [], nextStep: [] }],
]
for (const [label, value] of malformed) {
assert.throws(
() => inboxProjectionSchema.parse(value),
`parse must reject ${label} — a returning-but-not-throwing parse would accept a corrupt checkpoint`,
)
}
})
// ── L4: the durable identity of the inbox projection ─────────────────────
//
// These two literals are a wire format shared with sessions written by the
// stock `dsh-agent-loop`. Changing either is silent: the stock loop's pending
// splices stop folding in, and the session resumes with an empty inbox.
test('L4: inbox projection keeps key "inbox" and stateVersion 1', async () => {
const { inboxProjectionDefinition, inboxProjectionSchema } = await import(MODULES.inbox)
assert.equal(
inboxProjectionDefinition.key,
'inbox',
'the projection key is a wire format shared with stock-loop sessions — changing it loses their pending input',
)
assert.equal(
inboxProjectionDefinition.stateVersion,
1,
'stateVersion 1 is what existing checkpoint rows carry — a bump makes stateSchema.parse refuse them',
)
assert.equal(
inboxProjectionDefinition.stateSchema,
inboxProjectionSchema,
'the definition must validate with the exported schema, not a private copy',
)
assert.deepEqual(
inboxProjectionDefinition.init(),
{ 'next-turn': [], 'next-step': [] },
'init() must produce the shape stateSchema.parse accepts',
)
})

View File

@ -0,0 +1,194 @@
/**
* End-to-end proof: a recall block produced by the REAL injection shape reaches
* the SYSTEM prompt and does NOT reach any user/message content.
*
* Why this file exists next to `loop.test.js`
* -------------------------------------------
* `loop.test.js` asserts the pure decision rewrite (`promoteSnapshotSections`).
* That is necessary but not sufficient: the bug it fixes was that the rewrite
* never fired on the real injection shape, and a unit test can only catch that
* if it also encodes the real shape. This file closes the loop the other way —
* it mounts the actual `HologramAgentLoop` on real dsh services (`SessionStore`,
* `AgentRegistry`, `LlmRuntime`, `SystemPrompt`, `ToolRuntime`,
* `SessionProjectionRegistry`), drives one real turn (pre-step → request build →
* LLM call), and asserts against **the request object the adapter actually
* received**, not an intermediate decision.
*
* Two directions, both required:
* E1 positive — the recall text appears in the system prompt the adapter got.
* E2 negative — the recall text appears in NO message body the adapter got
* (every message, every block; not just the last one).
*
* Only E1 fails on the broken commit; E2 is already true there because the old
* code left recall in history. E2 is what this fix is *for*: it must stay true
* after the move, otherwise the promotion would duplicate recall rather than
* relocate it.
*
* No network: the "adapter" is a local `LlmAdapter` subclass that records the
* request and yields one text chunk.
*
* @module dsh-memory/test/loop-system-e2e
*/
import assert from 'node:assert/strict'
import { createRequire } from 'node:module'
import { pathToFileURL } from 'node:url'
import test from 'node:test'
import { Context } from '@deepseek-ai/cordis'
import { AgentRegistry } from '@deepseek-ai/dsh-agent'
import { LlmAdapter, LlmRuntime, createUserMessage } from '@deepseek-ai/dsh-llm'
import { SessionStore } from '@deepseek-ai/dsh-session'
import { SystemPrompt } from '@deepseek-ai/dsh-system-prompt'
import { ToolRuntime } from '@deepseek-ai/dsh-tools'
import { HologramAgentLoop } from '../lib/loop.js'
/**
* Load `SessionProjectionRegistry` the way the harness does.
*
* It is a peer of `dsh-agent-loop`, not of this package, so a bare specifier
* only resolves when the repo's `node_modules` happens to link it. Resolving
* from `dsh-agent-loop`'s own location is exactly what a real dsh process does;
* if neither route works there is no harness to run against, and the test skips
* loudly instead of asserting on a half-mounted app.
* @returns {Promise<Function|null>}
*/
async function loadProjectionRegistry() {
try {
return (await import('@deepseek-ai/dsh-session-projection')).SessionProjectionRegistry
} catch {
try {
const peerRequire = createRequire(import.meta.resolve('@deepseek-ai/dsh-agent-loop'))
return (await import(pathToFileURL(peerRequire.resolve('@deepseek-ai/dsh-session-projection')).href))
.SessionProjectionRegistry
} catch {
return null
}
}
}
/** Every text the model would read out of a message body. */
function bodyText(message) {
if (!Array.isArray(message?.content)) return ''
return message.content.map((block) => (typeof block?.text === 'string' ? block.text : '')).join('')
}
/** The system prompt text as the adapter received it. */
function systemText(request) {
return request.messages
.filter((message) => message.role === 'system')
.map((message) => bodyText(message))
.join('\n')
}
/** Every non-system message body, flattened, for the negative direction. */
function historyBlocks(request) {
return request.messages
.filter((message) => message.role !== 'system')
.map((message, index) => ({ index, role: message.role, kind: message.source?.kind, text: bodyText(message) }))
}
test('E1/E2: recall reaches the system prompt and no message body', async (t) => {
const SessionProjectionRegistry = await loadProjectionRegistry()
if (SessionProjectionRegistry === null) {
t.skip('@deepseek-ai/dsh-session-projection not resolvable from this install (no harness to mount)')
return
}
const RECALL = 'RECALLED-NOTE-4471: user prefers evidence over summaries.'
/** Records the request it is called with; yields one stop-finish text chunk. */
const captured = []
class CaptureAdapter extends LlmAdapter {
async *stream(request) {
captured.push(request)
yield { type: 'block-start', index: 0, blockType: 'text' }
yield { type: 'text-delta', index: 0, text: 'acknowledged' }
yield { type: 'block-end', index: 0, block: { type: 'text', text: 'acknowledged' } }
yield { type: 'finish', reason: 'stop' }
}
}
const ctx = new Context()
try {
// Cordis defers service publication by a microtask; each `new` below needs
// the previous one visible, so yield between mounts.
new SessionStore(ctx)
await new Promise((resolve) => setTimeout(resolve, 0))
new AgentRegistry(ctx)
new SystemPrompt(ctx, {})
new ToolRuntime(ctx)
new LlmRuntime(ctx)
new SessionProjectionRegistry(ctx)
await new Promise((resolve) => setTimeout(resolve, 0))
// The loop under test. Mount before any agent exists: `setFactory` runs in
// the constructor and the class refuses a second registration.
new HologramAgentLoop(ctx, { maxParallelToolCalls: 4, agents: [] })
await new Promise((resolve) => setTimeout(resolve, 0))
ctx.llm.registerAdapter(['capture'], new CaptureAdapter())
// The REAL injection shape, reproduced byte for byte from `lib/index.js:932`:
// one text block whose text IS the section text, `form: 'snapshot'`.
ctx.on(
'agent/pre-step',
async (payload, next) => {
const decision = await next()
if (decision.kind === 'reject' || payload.signal.aborted) return decision
return {
...decision,
messages: [
...decision.messages,
createUserMessage({
content: [{ type: 'text', text: RECALL }],
source: { kind: 'memory', form: 'snapshot', sections: [{ name: 'memory', text: RECALL }] },
}),
],
}
},
{ prepend: true },
)
const handle = await ctx.agents.create({
sessionId: 'loop-system-e2e',
agentOptions: { provider: 'capture', model: 'capture-1' },
})
const agent = handle.agent
agent.followup(
createUserMessage({ content: [{ type: 'text', text: 'what did we decide?' }], source: { kind: 'user' } }),
)
await agent.whenIdle()
assert.equal(captured.length, 1, `the adapter must be called exactly once; got ${captured.length} call(s)`)
const request = captured[0]
// ── E1 · positive: the recall is in the system prompt ────────────────
const system = systemText(request)
assert.ok(
system.includes(RECALL),
`E1: recall must appear in the system prompt the adapter received. system prompt was ${JSON.stringify(system)}`,
)
// ── E2 · negative: the recall is in NO message body ──────────────────
const history = historyBlocks(request)
const leaking = history.filter((entry) => entry.text.includes(RECALL))
assert.deepEqual(
leaking,
[],
`E2: recall must not appear in ANY non-system message; leaking messages: ${JSON.stringify(leaking)}`,
)
// E2b · the loop must not duplicate it either: exactly one system message
// may carry it, so a "promote but keep" bug cannot hide behind E1.
const carriers = request.messages.filter((message) => bodyText(message).includes(RECALL))
assert.equal(
carriers.length,
1,
`E2b: exactly one message may carry the recall (a move, not a copy); got ${carriers.length}: ${JSON.stringify(
carriers.map((message) => message.role),
)}`,
)
assert.equal(carriers[0].role, 'system', `E2b: the sole carrier must be the system message; got ${carriers[0].role}`)
} finally {
await ctx.fiber.dispose()
}
})

457
test/loop.test.js Normal file
View File

@ -0,0 +1,457 @@
/**
* Behavioural contract for the loop that promotes snapshot sections.
*
* The loop only earns its keep if a promoted recall block reaches the SYSTEM
* prompt and does NOT reach history. These tests pin exactly that, plus the
* cases where it must keep its hands off.
*
* P1 a pure snapshot carrier message moves into `assembly.sections`.
* P2 the returned decision is a NEW object; the input is not mutated.
* P3 the promoted section carries `interpolate: false` — free text with
* `{{...}}` must not reach the strict interpolator.
* P4a a carrier whose content text equals its sections IS promoted (the real
* path: `lib/index.js:932` always attaches one text block); P4b the same
* message with differing content is NOT; P4c proves the split is decided
* by text equality, not by the presence of content.
* P5 a non-snapshot message is NOT promoted.
* P6 a name collision suffixes instead of overwriting or dropping.
* P7 `reject` decisions pass through untouched.
* P8 a decision with no `assembly` passes through untouched.
* P9 everything that is not promoted survives in `messages`, in order.
* P10 the class does not re-register the factory (no constructor / inject).
*
* @module dsh-memory/test/loop
*/
import assert from 'node:assert/strict'
import test from 'node:test'
import { HologramAgentLoop, promoteSnapshotSections } from '../lib/loop.js'
/** Build a user message shaped like a snapshot-style injector emits. */
function snapshotMessage(text, kind = 'memory') {
return {
role: 'user',
content: [],
source: { kind, form: 'snapshot', sections: [{ name: kind, text }] },
}
}
/** Build a plain user message with real content. */
function plainMessage(text) {
return { role: 'user', content: [{ type: 'text', text }], source: { kind: 'user' } }
}
/** A minimal decision carrying an assembly with one existing section. */
function decisionWith(messages, sections = [{ name: 'base', text: 'base prompt' }]) {
return { kind: 'enter', messages, assembly: { sections, variables: {}, tools: [] } }
}
test('P1: a pure snapshot carrier is promoted into assembly.sections', () => {
const decision = decisionWith([plainMessage('what did I decide?'), snapshotMessage('note alpha')])
const out = promoteSnapshotSections(decision)
assert.equal(out.messages.length, 1)
assert.deepEqual(out.messages[0].content, [{ type: 'text', text: 'what did I decide?' }])
assert.equal(out.assembly.sections.length, 2)
const promoted = out.assembly.sections[1]
assert.equal(promoted.text, 'note alpha')
assert.ok(promoted.name.startsWith('memory'))
})
test('P2: the input decision is not mutated', () => {
const original = decisionWith([snapshotMessage('note alpha')])
const before = structuredClone(original)
promoteSnapshotSections(original)
assert.deepEqual(original, before)
})
test('P3: the promoted section opts out of interpolation', () => {
const out = promoteSnapshotSections(decisionWith([snapshotMessage('literal {{not-a-var}} here')]))
assert.equal(out.assembly.sections[1].interpolate, false)
})
test('P4a: a carrier whose content equals its sections IS promoted', () => {
// The old P4 asserted the opposite for every message carrying content, on the
// assumption that "content blocks present" means "the plugin wants the user to
// see this". That assumption is FALSE for the only producer that matters here:
// `lib/index.js:932` always injects exactly ONE text block whose text is
// `sections[0].text` verbatim — content and sections are the same string. So
// the old rule (`content non-empty ⇒ reject`) made `promoteSnapshotSections` a
// permanent no-op on the real recall path and left recall in user/message
// history, which is the bug this pair of tests exists to pin.
const aligned = {
role: 'user',
content: [{ type: 'text', text: 'recalled note' }],
source: { kind: 'memory', form: 'snapshot', sections: [{ name: 'memory', text: 'recalled note' }] },
}
const out = promoteSnapshotSections(decisionWith([aligned]))
assert.equal(out.messages.length, 0, `content == sections must be promoted; got ${out.messages.length} message(s)`)
assert.equal(out.assembly.sections.length, 2, `expected base + promoted; got ${out.assembly.sections.length}`)
assert.equal(
out.assembly.sections[1].text,
'recalled note',
`promoted text must be the content text unchanged; got ${JSON.stringify(out.assembly.sections[1].text)}`,
)
})
test('P4b: a carrier whose content differs from its sections is NOT promoted', () => {
// The old P4 used 'visible' vs 'hidden' — content != sections — which is the
// side that must keep its hands off. That is exactly why it was green on the
// broken commit: it asserted the correct behaviour for the wrong reason and
// never covered the case that actually occurs (`content == sections`). It is
// kept here, correctly named, as one half of the pair.
const drifted = {
role: 'user',
content: [{ type: 'text', text: 'visible' }],
source: { kind: 'memory', form: 'snapshot', sections: [{ name: 'memory', text: 'hidden' }] },
}
const out = promoteSnapshotSections(decisionWith([drifted]))
assert.equal(out.messages.length, 1, `content != sections must stay in history; got ${out.messages.length}`)
assert.equal(out.assembly.sections.length, 1, `no section may be added; got ${out.assembly.sections.length}`)
})
test('P4c: P4a and P4b are decided by TEXT, not by "has content at all"', () => {
// One decision holding both carriers: if the implementation matched on the
// mere presence of content, both would land on the same side and the
// messages/sections counts below could not both hold.
const aligned = {
role: 'user',
content: [{ type: 'text', text: 'recalled note' }],
source: { kind: 'memory', form: 'snapshot', sections: [{ name: 'memory', text: 'recalled note' }] },
}
const drifted = {
role: 'user',
content: [{ type: 'text', text: 'visible' }],
source: { kind: 'memory', form: 'snapshot', sections: [{ name: 'memory', text: 'hidden' }] },
}
const out = promoteSnapshotSections(decisionWith([aligned, drifted]))
assert.deepEqual(out.messages, [drifted], 'only the mismatching carrier may survive')
assert.equal(out.assembly.sections.length, 2, `only the matching carrier may be promoted; got ${out.assembly.sections.length}`)
})
test('P5: a non-snapshot message is not promoted', () => {
const runtime = {
role: 'user',
content: [],
source: { kind: 'runtime-context', form: 'inline', sections: [{ name: 'x', text: 'y' }] },
}
const out = promoteSnapshotSections(decisionWith([runtime]))
assert.equal(out.messages.length, 1)
assert.equal(out.assembly.sections.length, 1)
})
test('P6: a name collision is suffixed, never dropped', () => {
const out = promoteSnapshotSections(
decisionWith([snapshotMessage('note', 'base')], [{ name: 'base', text: 'base prompt' }]),
)
assert.equal(out.assembly.sections.length, 2)
const names = out.assembly.sections.map((section) => section.name)
assert.equal(new Set(names).size, names.length, 'section names must stay unique')
assert.equal(out.assembly.sections[1].text, 'note')
})
test('P7: a reject decision passes through untouched', () => {
const reject = { kind: 'reject', reason: 'nope' }
assert.equal(promoteSnapshotSections(reject), reject)
})
test('P8: a decision without an assembly passes through untouched', () => {
const bare = { kind: 'enter', messages: [snapshotMessage('note')] }
assert.equal(promoteSnapshotSections(bare), bare)
})
test('P9: non-promoted messages keep their order', () => {
const out = promoteSnapshotSections(
decisionWith([plainMessage('first'), snapshotMessage('note'), plainMessage('second')]),
)
assert.deepEqual(
out.messages.map((message) => message.content[0].text),
['first', 'second'],
)
})
test('P10: the subclass does not re-declare the factory or its inject list', () => {
// A second `setFactory` registration throws at startup, and re-declaring
// `inject` would pull a duplicate AgentLoop into the container. Both are
// avoided by not defining a constructor or static inject at all.
assert.equal(Object.prototype.hasOwnProperty.call(HologramAgentLoop, 'inject'), false)
const source = HologramAgentLoop.prototype.constructor.toString()
assert.equal(source.includes('constructor'), false, 'subclass must not define a constructor')
assert.equal(typeof HologramAgentLoop.prototype.prepare, 'function')
})
// ─────────────────────────────────────────────────────────────────────────
// S1–S7 · the two canonical spellings, and the ones that must not be touched
// ─────────────────────────────────────────────────────────────────────────
//
// Why this block exists: every test above builds its snapshot carrier with
// `content: []`. That is a shape NO real producer emits — `lib/index.js:932`
// always attaches exactly one `{ type: 'text' }` block whose text is
// `sections[0].text` verbatim. So the whole suite was green over a code path
// that never runs in production, and `promoteSnapshotSections` was a no-op on
// every real recall. The rule under test is three-way:
//
// content == sections (either joining convention) → PROMOTE, moving the text
// content != sections → leave it in history
//
// The invariant is "move, don't rewrite": the promoted section text must be
// *identical* to the text the model would otherwise have read out of the
// message content, so these tests can tell a relocation from a silent rewrite.
//
// ★ Every negative case below carries a POSITIVE CONTROL in the same decision:
// a carrier of the same shape that *must* be promoted. Without the control a
// negative assertion is satisfied by the old "content non-empty ⇒ reject" rule
// alone, so it would be green on the broken commit and prove nothing. With it,
// the assertion fails unless the implementation is comparing the two texts.
/** A user message whose content and sections are the SAME single text. */
function carrier(text, kind = 'memory') {
return {
role: 'user',
content: [{ type: 'text', text }],
source: { kind, form: 'snapshot', sections: [{ name: kind, text }] },
}
}
/**
* A carrier whose N sections are rendered into ONE content block.
*
* The name states the shape: ONE text block. It models the two joining
* conventions ('\n\n' and '') on that single block, which is what
* `lib/index.js:932` actually emits.
*
* ★ This helper was previously called `multiCarrier` while producing a single
* block, and that name is exactly what let the V2 bug (`if (blocks.length > 1)
* return null`) pass untouched: S2a/S2b read as multi-block coverage while
* exercising none of it. A helper whose name overstates its shape is worse than
* a missing test, because it looks like coverage. Real multi-block carriers come
* from `trueMultiCarrier` below and are exercised by S2c/S2d.
* @param {string[]} texts one entry per section
* @param {'glued'|'spaced'} convention how the sections were concatenated
*/
function singleBlockCarrier(texts, convention = 'spaced', kind = 'memory') {
const joined = convention === 'glued' ? texts.join('') : texts.join('\n\n')
return {
role: 'user',
content: [{ type: 'text', text: joined }],
source: { kind, form: 'snapshot', sections: texts.map((text, i) => ({ name: `${kind}-${i}`, text })) },
}
}
/**
* 产出:N 个 text block,每个 block 一个 section 的文本(rendered = texts.join(''))。
* 这不是本插件产出的形状,是"其它 producer 逐块发文本"的形状;只用来证明多块路径有牙。
* 注意:真多块时 rendered 恒为 glued 拼法,没有 spaced 变体 —— spaced 只由单块承载。
* @param {string[]} texts one entry per block AND per section
* @param {string} kind source kind
*/
function trueMultiCarrier(texts, kind = 'memory') {
return {
role: 'user',
content: texts.map((text) => ({ type: 'text', text })),
source: { kind, form: 'snapshot', sections: texts.map((text, i) => ({ name: `${kind}-${i}`, text })) },
}
}
/** Readable "expected vs actual" rendering for the assertion messages. */
const show = (value) => JSON.stringify(value)
test('S1: single block == single section is promoted, text byte-identical', () => {
const out = promoteSnapshotSections(decisionWith([plainMessage('what did I decide?'), carrier('note alpha')]))
assert.equal(
out.messages.length,
1,
`the carrier must leave messages (expected 1 remaining, got ${out.messages.length})`,
)
assert.equal(
out.assembly.sections.length,
2,
`expected base + promoted section = 2, got ${out.assembly.sections.length}`,
)
const promoted = out.assembly.sections[1]
assert.equal(
promoted.text,
'note alpha',
`promoted text must be the carrier text unchanged; expected ${show('note alpha')}, got ${show(promoted.text)}`,
)
})
test('S2a: single content block carrying N sections ("\\n\\n" convention) is promoted', () => {
const out = promoteSnapshotSections(decisionWith([singleBlockCarrier(['a', 'b'], 'spaced')]))
assert.equal(
out.messages.length,
0,
`the spaced carrier must be promoted (expected 0 messages, got ${out.messages.length})`,
)
assert.equal(
out.assembly.sections.length,
2,
`expected base + promoted = 2 sections, got ${out.assembly.sections.length}`,
)
assert.equal(
out.assembly.sections[1].text,
'a\n\nb',
`promoted text must equal the content text exactly (move, not rewrite); expected ${show('a\n\nb')}, got ${show(out.assembly.sections[1].text)}`,
)
})
test('S2b: single content block carrying N sections ("" glued convention) is promoted', () => {
const out = promoteSnapshotSections(decisionWith([singleBlockCarrier(['a', 'b'], 'glued')]))
assert.equal(
out.messages.length,
0,
`the glued carrier must be promoted (expected 0 messages, got ${out.messages.length})`,
)
assert.equal(
out.assembly.sections.length,
2,
`expected base + promoted = 2 sections, got ${out.assembly.sections.length}`,
)
assert.equal(
out.assembly.sections[1].text,
'ab',
`promoted text must equal the content text exactly; expected ${show('ab')}, got ${show(out.assembly.sections[1].text)}`,
)
})
test('S2c: two content blocks == two sections (glued) is promoted', () => {
const msg = trueMultiCarrier(['a', 'b']) // 2 blocks, rendered='ab'
const out = promoteSnapshotSections(decisionWith([msg]))
assert.equal(out.messages.length, 0, `two blocks == two sections must be promoted; got ${out.messages.length} message(s)`)
assert.equal(out.assembly.sections[1].text, 'ab', `promoted text must be the rendered content; got ${JSON.stringify(out.assembly.sections[1].text)}`)
})
test('S2d: two content blocks != sections is NOT promoted (control: the match is)', () => {
// rendered = 'ab' (glued), while the single section says 'a\n\nb' — neither
// canonical join matches, so this must stay in history. The control beside it
// is the same two blocks with matching sections, which must be promoted.
const drifted = {
...trueMultiCarrier(['a', 'b']),
source: { kind: 'memory', form: 'snapshot', sections: [{ name: 'm0', text: 'a\n\nb' }] },
}
const control = trueMultiCarrier(['a', 'b'])
const out = promoteSnapshotSections(decisionWith([control, drifted]))
assert.equal(out.messages.length, 1, `only the drifted one may stay; got ${out.messages.length}`)
assert.equal(out.messages[0], drifted, 'the surviving message must be the drifted one')
})
test('S3: single content block carrying N sections != those sections is NOT promoted (control: the match is)', () => {
// Sections say "a"+"b", the content block says "zzz": editing the text would
// not be a move, so this one must stay in history — while the matching
// carrier beside it must go. The two must be told apart by CONTENT, not by
// "does it carry content at all".
const drifted = {
role: 'user',
content: [{ type: 'text', text: 'zzz' }],
source: {
kind: 'memory',
form: 'snapshot',
sections: [{ name: 'memory-0', text: 'a' }, { name: 'memory-1', text: 'b' }],
},
}
const control = singleBlockCarrier(['a', 'b'], 'spaced')
const out = promoteSnapshotSections(decisionWith([control, drifted]))
assert.equal(
out.messages.length,
1,
`only the drifted carrier may stay (expected 1 message, got ${out.messages.length})`,
)
assert.equal(out.messages[0], drifted, 'the surviving message must be the drifted carrier itself')
assert.equal(
out.assembly.sections.length,
2,
`only the control may be promoted (expected 2 sections, got ${out.assembly.sections.length})`,
)
assert.equal(
out.assembly.sections[1].text,
'a\n\nb',
`the promoted text must come from the control; expected ${show('a\n\nb')}, got ${show(out.assembly.sections[1].text)}`,
)
})
test('S4: single block != single section is NOT promoted (control: the match is)', () => {
const drifted = {
role: 'user',
content: [{ type: 'text', text: 'content says this' }],
source: { kind: 'memory', form: 'snapshot', sections: [{ name: 'memory', text: 'sections say that' }] },
}
const control = carrier('note alpha')
const out = promoteSnapshotSections(decisionWith([control, drifted]))
assert.equal(out.messages.length, 1, `only the drifted carrier may stay; got ${out.messages.length} messages`)
assert.equal(out.messages[0], drifted, 'the surviving message must be the drifted carrier itself')
assert.equal(out.assembly.sections.length, 2, `only the control may be promoted; got ${out.assembly.sections.length} sections`)
assert.equal(
out.assembly.sections[1].text,
'note alpha',
`promoted text must be the control's; expected ${show('note alpha')}, got ${show(out.assembly.sections[1].text)}`,
)
})
test('S5: empty content (legacy carrier) is promoted — regression, with its real-world twin', () => {
// Two carriers: the legacy `content: []` shape (which the old code did handle)
// and the shape every real producer emits (which it did not). Both must be
// promoted, so this doubles as the regression pin for the old path AND as the
// minimal reproduction of the production no-op.
const out = promoteSnapshotSections(decisionWith([snapshotMessage('note alpha'), carrier('note beta')]))
assert.equal(
out.messages.length,
0,
`both carriers must be promoted (expected 0 messages, got ${out.messages.length})`,
)
assert.deepEqual(
out.assembly.sections.map((section) => section.text),
['base prompt', 'note alpha', 'note beta'],
`expected the legacy text first, then the real-world text; got ${show(out.assembly.sections.map((s) => s.text))}`,
)
})
test('S6: runtime-context shape (prefix in content, absent from sections) is NOT promoted', () => {
// The stock loop prepends a fixed sentence to the content and keeps `sections`
// as the raw contributions, so the two can never be equal. Promoting it would
// delete that preamble, so it must pass through — the control carrier in the
// same decision proves the comparison is on text, not on "has content".
const runtime = {
role: 'user',
content: [
{
type: 'text',
text: 'Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\ncwd /tmp',
},
],
source: { kind: 'runtime-context', form: 'snapshot', sections: [{ name: 'runtime-context', text: 'cwd /tmp' }] },
}
const control = carrier('cwd /tmp', 'runtime-context')
const out = promoteSnapshotSections(decisionWith([control, runtime]))
assert.equal(out.messages.length, 1, `runtime-context must stay in history; got ${out.messages.length} messages`)
assert.equal(out.messages[0], runtime, 'the surviving message must be the runtime-context snapshot itself')
assert.equal(out.assembly.sections.length, 2, `only the control may be promoted; got ${out.assembly.sections.length} sections`)
assert.equal(
out.assembly.sections[1].text,
'cwd /tmp',
`promoted text must be the control's; expected ${show('cwd /tmp')}, got ${show(out.assembly.sections[1].text)}`,
)
})
test('S7: a non-text block means NOT promoted (control: the text-only one is)', () => {
const imageCarrier = {
role: 'user',
content: [{ type: 'image', text: 'note alpha' }],
source: { kind: 'memory', form: 'snapshot', sections: [{ name: 'memory', text: 'note alpha' }] },
}
const badText = {
role: 'user',
content: [{ type: 'text', text: 42 }],
source: { kind: 'memory', form: 'snapshot', sections: [{ name: 'memory', text: '42' }] },
}
const control = carrier('note alpha')
const out = promoteSnapshotSections(decisionWith([control, imageCarrier, badText]))
assert.equal(out.messages.length, 2, `only the text-only control may be promoted; got ${out.messages.length} messages`)
assert.equal(out.messages[0], imageCarrier, 'the image carrier must survive')
assert.equal(out.messages[1], badText, 'the non-string-text carrier must survive')
assert.equal(out.assembly.sections.length, 2, `only the control may be promoted; got ${out.assembly.sections.length} sections`)
assert.equal(
out.assembly.sections[1].text,
'note alpha',
`promoted text must be the control's; expected ${show('note alpha')}, got ${show(out.assembly.sections[1].text)}`,
)
})

836
test/memory-log.test.js Normal file
View File

@ -0,0 +1,836 @@
/**
* Observability contract for the recall log.
*
* The plugin shipped with zero `console.*` and zero `ctx.logger` calls, so a
* production recall failure left no trace anywhere: `degraded` is a return value
* the caller sees once, and the injected text only shows up if the model quotes
* it. These tests pin the replacement channel.
*
* What is pinned here, and why each assertion exists:
*
* L1 `debugLog: false` (the default) is quiet on an ordinary turn, and still
* carries every fact the operator needs when it does speak.
* L2 `debugLog: true` emits exactly ONE line covering recall / budget /
* degradation / injection.
* L3 the query TEXT is never logged — only its length.
* L4 a real fault (`embed_error`) speaks even with `debugLog` off, at `warn`.
* L5 a zero-injection turn is `debug`, not `info` — the commonest case must
* cost nothing at normal verbosity.
* L6 logging changes no behaviour: the returned text and `lastRecall` are
* identical with the summary on and off.
* L7 nothing is ever written to stdout (headless prints the ANSWER there) and
* the console fallback goes to `console.warn`, not `console.log`.
* L10 `ORDINARY_DEGRADATIONS` actually gates the level: the two healthy modes
* (`hybrid`, `no_vector_space`) are SILENT with `debugLog` off, while a
* real fault (`embed_error`) is a single `warn`. L10†/L11† mutate the set to
* prove those two assertions bite.
* L11 a fused recall whose candidates are ALL refused by `minScore` still
* reports `hybrid` with zero hits, and is still silent. `no_vector_space`
* means "there was nothing to score", never "everything was filtered" —
* collapsing the two would hide a threshold set too high.
*
* @module dsh-memory/test/memory-log
*/
import assert from 'node:assert/strict'
import { mkdtempSync, rmSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import test from 'node:test'
import { Context } from '@deepseek-ai/cordis'
import { Logger } from '@deepseek-ai/cordis'
import { createUserMessage } from '@deepseek-ai/dsh-llm'
import { SystemPrompt } from '@deepseek-ai/dsh-system-prompt'
import { ToolRuntime } from '@deepseek-ai/dsh-tools'
import { apply as applyMemory, StableMem } from '../lib/index.js'
import { recallBlocksHybrid, segmentRecallBudget } from '../lib/recall.js'
import { BlockStore } from '../lib/store.js'
/**
* Build a real cordis app + plugin instance, capturing every log record.
*
* `levels: { default: 3 }` is DEBUG: cordis skips a record when
* `(exporter.levels[name] ?? default ?? logger.level ?? INFO) < level`
* (cordis/lib/index.js:470-472), so the default would silently drop `debug` and
* L5 could never observe its own subject.
*
* @param {object} config plugin config
*/
async function makeMemWith(apply, config = {}) {
const dir = mkdtempSync(join(tmpdir(), 'mlog-'))
const ctx = new Context()
new SystemPrompt(ctx, {})
new ToolRuntime(ctx, {})
const records = []
ctx.logger.exporter({
levels: { default: 3 },
export: (message) => records.push(message),
})
const mem = apply(ctx, { path: join(dir, 'm.db'), ...config })
return {
ctx,
mem,
records,
/**
* Only this plugin's lines, rendered the way the logger actually prints them.
*
* `Logger.format` rather than a plain `args.join(' ')`: every call site emits
* `log(level)('%s', line)` so the payload cannot be reformatted (see `emit`)
* — a naive join would keep the literal `%s` prefix and hide a regression
* where the payload stopped being a single argument.
*/
lines: () =>
records
.filter((record) => record.name === 'dsh-memory')
.map((record) => ({ type: record.type, text: Logger.format({}, record) })),
async done() {
await ctx.fiber.dispose()
rmSync(dir, { recursive: true, force: true })
},
}
}
/**
* The same app, built by the module under test.
*
* Parameterised by `apply` so the ORDINARY_DEGRADATIONS mutants below can run the
* IDENTICAL criterion body against a mutated `lib/index.js`. A criterion that is
* copied for the mutant can drift from the real one; a criterion that is shared
* cannot.
*
* @param {object} [config] plugin config
*/
async function makeMem(config = {}) {
return makeMemWith(applyMemory, config)
}
/** A one-message human batch. */
function ask(text) {
return [createUserMessage({ content: [{ type: 'text', text }], source: { kind: 'user' } })]
}
/** An embedder that fails the way a dead provider does. */
const failingEmbedder = {
async embedText() {
throw new Error('read ECONNREFUSED 127.0.0.1:8081')
},
info: () => ({ fingerprint: 'fp-log', dimension: 3 }),
}
// ── L1: default config is quiet, and stays quiet for the commonest case ───
test('L1: debugLog defaults to false and an ordinary hitless turn logs nothing', async () => {
const { mem, lines, done } = await makeMem()
try {
assert.equal(mem.debugLog, false, 'the default must be false — the log is opt-in')
// No embedder is the shipping default; recall degrades to `no_embedder`,
// which is a standing configuration fact, not an event (same judgement W11e
// made for the model-facing notice).
await mem.commit({ text: '某条无关记忆' })
for (const turn of ['今天天气怎么样', '你好', '好的']) {
await mem.contextForStep(ask(turn))
}
assert.deepEqual(lines(), [], `the default must be silent; got ${JSON.stringify(lines())}`)
} finally {
await done()
}
})
// ── L2/L3: opt in, then all four fact groups appear in ONE line ───────────
test('L2: debugLog=true emits one line covering recall, budget, degradation and injection', async () => {
const { mem, lines, done } = await makeMem({ debugLog: true, topK: 3, budgetTokens: 500 })
try {
assert.equal(mem.debugLog, true, 'config.debugLog must reach the service')
await mem.commit({ text: '值班室分机号 4324,值班人 老周' })
const question = '值班室分机号是多少'
const text = await mem.contextForStep(ask(question))
assert.notEqual(text, '', 'precondition: the block must be recalled')
const emitted = lines()
assert.equal(emitted.length, 1, `exactly one line per recall turn; got ${JSON.stringify(emitted)}`)
const [record] = emitted
assert.equal(record.type, 'info', 'a turn that injected something is an info line')
// recall: length only, never the text.
assert.match(record.text, /recall query=9ch/, `query length in code points; got: ${record.text}`)
assert.ok(
!record.text.includes(question),
`the query TEXT must never be logged; got: ${record.text}`,
)
assert.match(record.text, /hits=[1-9]\d*/, 'hit count')
assert.match(record.text, /topK=3/, 'topK actually used')
assert.match(record.text, /zeroHit=no/, 'the 0-hit flag must be stated, not inferred')
// budget: which cap applied and what it retained.
assert.match(record.text, /budget=500 /, 'the explicit budget wins')
assert.match(record.text, /source=explicit/, 'source must name the rule that produced the budget')
assert.match(record.text, /kept=1\b/, 'retained count after segmenting')
assert.match(record.text, /singleHitOverBudget=(yes|no)/, 'the over-budget single-hit case is reported')
// degradation: the empty case is reported as healthy, explicitly.
assert.match(record.text, /degraded=no_embedder/, 'the mode is named, not omitted')
assert.match(record.text, /skipped=0 skippedDim=0/, 'skip counters')
// D2: the three downstream counters must be PRINTED, not merely computed.
// A counter nobody can read is indistinguishable from no counter at all.
assert.match(record.text, /skippedScore=0 skippedNoEvidence=0 skippedUnscoreable=0 truncated=0/, 'downstream drop counters')
assert.match(record.text, /err=-/, 'no error digest when there is no error')
// injection: the exact string length that reached the model.
assert.match(record.text, new RegExp(`inject=${[...text].length}ch`), 'injected length')
} finally {
await done()
}
})
test('L3: the budget source is named for each precedence rule', async () => {
// explicit > host window > fallback, read straight off the summary line.
const explicit = await makeMem({ debugLog: true, budgetTokens: 1234 })
try {
await explicit.mem.commit({ text: '值班室分机号 4324' })
await explicit.mem.contextForStep(ask('值班室分机号是多少'))
assert.match(explicit.lines()[0].text, /budget=1234 source=explicit/)
} finally {
await explicit.done()
}
const host = await makeMem({ debugLog: true })
try {
await host.mem.commit({ text: '值班室分机号 4324' })
// A host window of 100 tokens → 80% = 80 token budget.
await host.mem.contextForStep(ask('值班室分机号是多少'), undefined, { contextWindow: 100 })
assert.match(host.lines()[0].text, /budget=80 source=host/)
} finally {
await host.done()
}
const fallback = await makeMem({ debugLog: true })
try {
await fallback.mem.commit({ text: '值班室分机号 4324' })
await fallback.mem.contextForStep(ask('值班室分机号是多少'))
assert.match(fallback.lines()[0].text, /budget=4096 source=fallback/)
} finally {
await fallback.done()
}
})
// ── L2b: non-zero drop counters reach BOTH operator surfaces ──────────────
//
// L2 only proves the fields are printed when everything is zero. A counter that
// is correct in the returned array but lost on the way to the log is still
// invisible — and `segmentRecallBudget` slices that array, which is exactly where
// non-enumerable properties normally disappear.
//
// Fixture: three blocks on separate axes, a query mostly on axis 0. Cosines are
// ~0.95 (passes), ~0.30 (gated by the 0.5 threshold) and 0 (nothing to score),
// so the three destinations are all populated at once.
test('L2b: non-zero drop counters appear in the log and in last_recall', async () => {
const DIM = 4
const unit = (v) => { const n = Math.hypot(...v); return n === 0 ? v : v.map((x) => x / n) }
const axis = (i) => unit(Array.from({ length: DIM }, (_, k) => (k === i ? 1 : 0)))
const QUERY = unit([0.95, 0.30, 0, 0])
const byText = new Map([
['值班室的分机号是 4324', axis(0)],
['食堂在东楼二层', axis(1)],
['叉车的额定载重是两吨', axis(2)],
])
const embedder = {
async embedText(text) {
if (text === '分机号是多少') return QUERY
return byText.get(text) ?? new Array(DIM).fill(0)
},
info: () => ({ fingerprint: 'd2:nonzero', dimension: DIM }),
}
const { mem, lines, done } = await makeMemWith(applyMemory, { debugLog: true, embedder })
try {
for (const text of byText.keys()) await mem.commit({ text })
await mem.contextForStep(ask('分机号是多少'))
const emitted = lines()
assert.equal(emitted.length, 1, `exactly one line per recall turn; got ${JSON.stringify(emitted)}`)
assert.match(
emitted[0].text,
/skippedScore=1 skippedNoEvidence=1 skippedUnscoreable=0 truncated=0/,
`D2: the counters must survive segmenting and reach the log; got ${emitted[0].text}`,
)
assert.equal(mem.lastRecall.skipped_score, 1, 'and the durable surface must agree')
assert.equal(mem.lastRecall.skipped_no_evidence, 1, 'both silent-drop kinds reported separately')
assert.equal(mem.lastRecall.truncated, 0, 'nothing was cut by topK here')
} finally {
await done()
}
})
// ── L4: a real fault speaks with the log switched off ─────────────────────
test('L4: an embed_error is warned about even when debugLog is false', async () => {
const { mem, lines, done } = await makeMem({ embedder: failingEmbedder })
try {
assert.equal(mem.debugLog, false)
await mem.commit({ text: '值班室分机号 4324' })
// `commit` also embeds, so clear the records that produced to keep the
// assertion about the RECALL line unambiguous.
lines().length = 0
await mem.contextForStep(ask('值班室分机号是多少'))
const recalled = lines().filter((line) => line.text.startsWith('recall '))
assert.equal(recalled.length, 1, `a fault must produce exactly one recall line; got ${JSON.stringify(recalled)}`)
assert.equal(recalled[0].type, 'warn', 'a failed search is a warn, not an info/debug')
assert.match(recalled[0].text, /degraded=embed_error/)
assert.match(
recalled[0].text,
/err=read ECONNREFUSED 127\.0\.0\.1:8081/,
`the error digest must name the cause; got: ${recalled[0].text}`,
)
} finally {
await done()
}
})
test('L4b: a long provider error is flattened and truncated, never a multi-line log', async () => {
const verbose = {
async embedText() {
throw new Error(`connection refused\n at socket.js:1\n${'x'.repeat(400)}`)
},
info: () => ({ fingerprint: 'fp-log2', dimension: 3 }),
}
const { mem, lines, done } = await makeMem({ embedder: verbose })
try {
await mem.contextForStep(ask('值班室分机号是多少'))
const recalled = lines().filter((line) => line.text.startsWith('recall '))
assert.equal(recalled.length, 1)
assert.ok(!recalled[0].text.includes('\n'), 'a log line must be one line')
assert.ok(recalled[0].text.includes('...'), 'the digest must be truncated')
assert.ok(
recalled[0].text.length < 400,
`the digest must be bounded; got ${recalled[0].text.length} chars`,
)
} finally {
await done()
}
})
// ── L5: the commonest case is `debug`, not `info` ────────────────────────
test('L5: a zero-injection turn logs exactly one DEBUG line, not info', async () => {
const { mem, lines, done } = await makeMem({ debugLog: true })
try {
await mem.commit({ text: '某条无关记忆' })
const text = await mem.contextForStep(ask('今天天气怎么样'))
assert.equal(text, '', 'precondition: nothing is injected')
const emitted = lines()
assert.equal(emitted.length, 1, `still exactly one line; got ${JSON.stringify(emitted)}`)
assert.equal(emitted[0].type, 'debug', 'zero injection must not be an info line')
assert.match(emitted[0].text, /zeroHit=yes/, 'the 0-hit case must say so')
assert.match(emitted[0].text, /inject=0ch/)
} finally {
await done()
}
})
test('L5b: a batch with no human prompt logs one debug line and recalls nothing', async () => {
const { mem, lines, done } = await makeMem({ debugLog: true })
try {
// This is the shape that once made every real turn recall the skill catalog,
// so "there was no human prompt" is worth a line of its own.
const injected = await mem.contextForStep([
createUserMessage({
content: [{ type: 'text', text: '[Skills]\n- alpha\n- beta' }],
source: { kind: 'skill-catalog', form: 'snapshot' },
}),
])
assert.equal(injected, '', 'an injected snapshot is not a recall query')
const emitted = lines()
assert.equal(emitted.length, 1)
assert.equal(emitted[0].type, 'debug')
assert.match(emitted[0].text, /query=0ch/)
assert.match(emitted[0].text, /skipped=no-human-prompt/)
} finally {
await done()
}
})
// ── L6: observability must be inert ──────────────────────────────────────
test('L6: turning the log on changes neither the injected text nor lastRecall', async () => {
const run = async (debugLog) => {
const app = await makeMem({ debugLog })
try {
await app.mem.commit({ text: '值班室分机号 4324,值班人 老周' })
const text = await app.mem.contextForStep(ask('值班室分机号是多少'), undefined, {
contextWindow: 2000,
})
const { at, ...lastRecall } = app.mem.stats().last_recall ?? {}
void at
return { text, lastRecall }
} finally {
await app.done()
}
}
const quiet = await run(false)
const loud = await run(true)
assert.equal(loud.text, quiet.text, 'the injected text must be byte-identical')
assert.notEqual(quiet.text, '', 'precondition: something was injected')
assert.deepEqual(loud.lastRecall, quiet.lastRecall, 'the stats surface must be unchanged')
})
// ── L7: stdout stays clean; the fallback shim goes to warn ───────────────
test('L7: logging never touches stdout, and the console fallback uses console.warn', async () => {
const stdout = []
const warned = []
const errored = []
const originals = {
log: console.log,
warn: console.warn,
error: console.error,
}
console.log = (...args) => stdout.push(args)
console.warn = (...args) => warned.push(args)
console.error = (...args) => errored.push(args)
const { mem, done } = await makeMem({ debugLog: true })
let fallbackCtx = null
let fallbackStore = null
try {
await mem.commit({ text: '值班室分机号 4324' })
await mem.contextForStep(ask('值班室分机号是多少'))
assert.deepEqual(
stdout,
[],
`the real logger path must never write stdout (headless prints the ANSWER there); got ${JSON.stringify(stdout)}`,
)
// Now the fallback: a context whose logger service cannot produce a facade.
// `resolveLogger` must degrade to console.warn rather than throw.
fallbackCtx = new Context()
fallbackCtx.logger = () => {
throw new Error('no logger service')
}
fallbackStore = new BlockStore({ path: ':memory:' }).open()
const fallback = new StableMem(fallbackCtx, { store: fallbackStore, debugLog: true })
fallback.log.warn('%s', 'shim warn')
fallback.log.debug('%s', 'shim debug')
fallback.log.error('%s', 'shim error')
assert.deepEqual(stdout, [], `stdout must stay clean; got ${JSON.stringify(stdout)}`)
assert.deepEqual(
warned.map((args) => args.join(' ')),
['shim warn', 'shim debug'],
'the fallback routes both debug and info to console.warn',
)
assert.deepEqual(errored.map((args) => args.join(' ')), ['shim error'])
} finally {
console.log = originals.log
console.warn = originals.warn
console.error = originals.error
fallbackStore?.close()
await fallbackCtx?.fiber.dispose()
await done()
}
})
// ── L4†: a mutation, not a self-test ─────────────────────────────────────
//
// "A guard that cannot fail is not a guard" — the same rule W11d†/W11e† follow in
// phase-B-H. This one patches `lib/index.js` on a COPY, imports it fresh, and
// asserts the behaviour flips. If the anchor moves, the split-count assertion
// fails loudly instead of the test silently passing against unmutated source.
test('L4†: silencing the fault branch turns L4 red', async () => {
const { readFileSync, writeFileSync, mkdtempSync, rmSync, cpSync } = await import('node:fs')
const { tmpdir } = await import('node:os')
const { join } = await import('node:path')
const { fileURLToPath, pathToFileURL } = await import('node:url')
const libDir = fileURLToPath(new URL('../lib/', import.meta.url))
const pkgRoot = fileURLToPath(new URL('..', import.meta.url))
const dir = mkdtempSync(join(tmpdir(), 'mlog-mut-'))
try {
cpSync(libDir, join(dir, 'lib'), { recursive: true })
cpSync(join(pkgRoot, 'package.json'), join(dir, 'package.json'))
try {
cpSync(join(pkgRoot, 'node_modules'), join(dir, 'node_modules'), {
recursive: true,
dereference: false,
})
} catch {
// Poetry-free install: the body below will fail loudly, which is correct.
}
const target = join(dir, 'lib', 'index.js')
const source = readFileSync(target, 'utf8')
const anchor = ' if (this.debugLog !== true && !fault) return'
assert.equal(
source.split(anchor).length - 1,
1,
'L4† anchor must occur exactly once — did the gating line move?',
)
writeFileSync(target, source.replace(anchor, ' if (this.debugLog !== true) return // MUTATED'))
const mod = await import(pathToFileURL(target).href)
const ctx = new Context()
new SystemPrompt(ctx, {})
new ToolRuntime(ctx, {})
const records = []
ctx.logger.exporter({ levels: { default: 3 }, export: (message) => records.push(message) })
try {
const mem = mod.apply(ctx, {
path: join(dir, 'm.db'),
embedder: failingEmbedder,
})
await mem.contextForStep(ask('值班室分机号是多少'))
const recalled = records.filter(
(record) => record.name === 'dsh-memory' && String(record.args[1] ?? '').startsWith('recall '),
)
assert.deepEqual(
recalled,
[],
'under the mutation a fault is silent with debugLog off, so L4 must fail',
)
} finally {
await ctx.fiber.dispose()
}
} finally {
rmSync(dir, { recursive: true, force: true })
}
})
// ── L8: the line must actually be READABLE in a real process ──────────────
//
// Measured: cordis ships one default exporter (an in-memory ring buffer) and no
// dsh bundle installs a terminal one, so `ctx.logger.info(...)` in a real profile
// lands in a buffer nobody drains. A "we added logging" change that only reaches
// that buffer would be green here and still print nothing in `dsh headless`.
// So this asserts on process.stderr, which is where an operator can see it.
test('L8: the summary reaches stderr, gated by debugLog, and never stdout', async () => {
const originalWrite = process.stderr.write
const stderr = []
const stdout = []
const originalOutWrite = process.stdout.write
process.stderr.write = (chunk, ...rest) => {
stderr.push(String(chunk))
return originalWrite.call(process.stderr, chunk, ...rest)
}
process.stdout.write = (chunk, ...rest) => {
stdout.push(String(chunk))
return originalOutWrite.call(process.stdout, chunk, ...rest)
}
const runTurn = async (config) => {
stderr.length = 0
const app = await makeMem(config)
try {
await app.mem.commit({ text: '值班室分机号 4324,值班人 老周' })
stderr.length = 0 // drop anything the commit produced
await app.mem.contextForStep(ask('值班室分机号是多少'))
return stderr.join('')
} finally {
await app.done()
}
}
try {
const loud = await runTurn({ debugLog: true })
assert.match(loud, /^dsh-memory: info: recall query=9ch/m, `want one info line on stderr; got: ${JSON.stringify(loud)}`)
const quiet = await runTurn({})
assert.equal(quiet, '', `debugLog=false must print nothing for a healthy turn; got: ${JSON.stringify(quiet)}`)
assert.deepEqual(stdout, [], `stdout must never be written; got ${JSON.stringify(stdout)}`)
} finally {
process.stderr.write = originalWrite
process.stdout.write = originalOutWrite
}
})
test('L8b: foreign loggers are not reprinted under this plugin’s prefix', async () => {
// The exporters map is process-wide, so a sink without a name check would
// duplicate every other plugin's warnings — turning an observability change
// into an amplification bug.
const originalWrite = process.stderr.write
const stderr = []
process.stderr.write = (chunk, ...rest) => {
stderr.push(String(chunk))
return originalWrite.call(process.stderr, chunk, ...rest)
}
const { ctx, mem, done } = await makeMem({ debugLog: true })
try {
stderr.length = 0
ctx.logger('some-other-plugin').warn('%s', 'not mine')
ctx.logger('some-other-plugin').info('%s', 'also not mine')
void mem
assert.equal(stderr.join(''), '', `another plugin's records must not be reprinted; got: ${JSON.stringify(stderr)}`)
} finally {
process.stderr.write = originalWrite
await done()
}
})
// ══════════════════════════════════════════════════════════════════════════════
// L10 · ORDINARY_DEGRADATIONS must actually DECIDE the level
// ══════════════════════════════════════════════════════════════════════════════
//
// P1 added two healthy modes to `lib/recall.js` (`hybrid`, `no_vector_space`) and
// registered both in `ORDINARY_DEGRADATIONS`. Nothing tested that registration:
// `grep ORDINARY_DEGRADATIONS test/` was empty, and every P1-era assertion checked
// the LABEL (`hits.degraded === 'hybrid'`), never the LOG LEVEL. Deleting `'hybrid'`
// from the set therefore kept all 268 tests green while a healthy wired recall
// started emitting `warn` on every turn — a false fault alarm on the happiest path.
//
// The three cases below are deliberately in ONE group so the contrast is visible:
// two healthy modes must be silent, one real fault must speak, all at the same
// verbosity (`debugLog: false`, the shipping default).
/**
* An embedder whose vectors put every block and the query on the same axis.
*
* Deterministic and transport-free: "a vector exists in this space" is the only
* fact these cases need, and a real provider would add neither coverage nor
* determinism.
*
* @param {string} fingerprint
*/
function fixedEmbedder(fingerprint) {
return {
async embedText() { return [1, 0, 0] },
info: () => ({ fingerprint, dimension: 3 }),
}
}
const L10_SPACE = 'fp-l10'
/**
* Seed one block that carries a vector in the active space.
*
* The explicit vector+fingerprint pair goes through the documented
* `commit({ vector, fingerprint })` path, so the write is a normal store write
* rather than a fixture shortcut: `recallBlocksHybrid` sees a real row.
*
* @param {{mem: object}} app
* @param {string} text
*/
async function commitVectored(app, text) {
await app.mem.commit({ text, vector: [1, 0, 0], fingerprint: L10_SPACE })
}
/** The criterion, shared by the real module and every mutant. */
async function assertLevelContract(build) {
// ── (1) `hybrid` is silent at the default verbosity ─────────────────────
//
// Preconditions asserted, not assumed: the space really holds a comparable
// vector, the hit really carries a vector contribution, and the label really is
// `hybrid`. Without them a silent log would be indistinguishable from a build
// that never reached the vector route at all.
const hybridApp = await build({ embedder: fixedEmbedder(L10_SPACE) })
let hybridText
let hybridHits
try {
await commitVectored(hybridApp, '值班室分机号 4324')
hybridText = await hybridApp.mem.contextForStep(ask('值班室分机号是多少'))
hybridHits = hybridApp.mem.lastRecall
const emitted = hybridApp.lines()
assert.equal(hybridHits?.mode, 'hybrid', `precondition: the wired route must be labelled hybrid; got ${JSON.stringify(hybridHits)}`)
assert.notEqual(hybridText, '', 'precondition: the block must actually be recalled')
assert.deepEqual(
emitted,
[],
'L10.1: a HEALTHY wired recall must log nothing with debugLog off. ' +
'A `warn` here is a false fault alarm on the happiest path, every turn. ' +
`got ${JSON.stringify(emitted)}`,
)
} finally {
await hybridApp.done()
}
// ── (2) `no_vector_space` is silent too ─────────────────────────────────
//
// A configured embedder whose space holds no usable vector. This is a DATA
// state (fresh space), not a provider fault, so it must not warn either.
//
// ★ The block must be written while NO embedder is mounted, otherwise `commit`
// embeds it and the space is no longer empty — a fixture that quietly fills the
// space it is supposed to leave bare would make this case assert nothing (it did,
// on the first attempt: the label came back `hybrid`).
const emptyApp = await build({})
try {
await emptyApp.mem.commit({ text: '值班室分机号 4324' })
emptyApp.mem.embedder = fixedEmbedder('fp-l10-empty')
await emptyApp.mem.contextForStep(ask('值班室分机号是多少'))
const mode = emptyApp.mem.lastRecall?.mode
assert.equal(mode, 'no_vector_space', `precondition: an empty space must report no_vector_space; got ${mode}`)
assert.deepEqual(
emptyApp.lines(),
[],
'L10.2: `no_vector_space` is a standing data state, not a fault — it must not warn. ' +
`got ${JSON.stringify(emptyApp.lines())}`,
)
} finally {
await emptyApp.done()
}
// ── (3) `embed_error` DOES speak, at `warn`, exactly once ───────────────
//
// The control. Without it, "silent" could be satisfied by a build that never
// logs at all — which is the bug the whole log channel exists to fix (L4).
const faultApp = await build({ embedder: failingEmbedder })
try {
await faultApp.mem.commit({ text: '值班室分机号 4324' })
await faultApp.mem.contextForStep(ask('值班室分机号是多少'))
const recalled = faultApp.lines().filter((line) => line.text.startsWith('recall '))
assert.equal(recalled.length, 1, `L10.3: a fault must produce exactly one recall line; got ${JSON.stringify(recalled)}`)
assert.equal(recalled[0].type, 'warn', `L10.3: a fault is a warn; got ${recalled[0].type}`)
assert.match(recalled[0].text, /degraded=embed_error/, 'and names the fault')
} finally {
await faultApp.done()
}
}
test('L10: healthy modes are silent and faults warn — the same verbosity, three outcomes', async () => {
await assertLevelContract(makeMem)
})
// ══════════════════════════════════════════════════════════════════════════════
// L11 · `minScore` filtering everything is NOT `no_vector_space`
// ══════════════════════════════════════════════════════════════════════════════
//
// Measured behaviour (pinned because "everything was filtered out" looks like a
// bug to a future reader, and the tempting fix — call it `no_vector_space` — would
// erase the distinction between "there was nothing to score" and "the threshold is
// too high"). `no_vector_space` is returned at exactly one place,
// `recall.js`'s `vectorScores.size === 0`; a fused run that produced candidates
// and then gated them all keeps the `hybrid` label with `hits: []`.
test('L11: a fused recall gated to zero hits by minScore still reports hybrid, and stays silent', async () => {
const { mem, lines, done } = await makeMem({ embedder: fixedEmbedder(L10_SPACE) })
try {
await commitVectored({ mem }, '值班室分机号 4324')
const question = '值班室分机号是多少'
// ★ `minScore` is applied PER PATH against the RAW VECTOR score (V5), not
// against the fused 0.7. The block's vector score is 1.0, so the threshold
// that removes it has to exceed 1.0.
//
// ★ Driven through `recallBlocksHybrid` rather than `mem.recall`: the
// service method forwards only `topK`/`signal`, so a threshold passed there
// would never reach the scorer and this precondition would silently pass on an
// unchanged result.
const raw = await recallBlocksHybrid(mem.store, fixedEmbedder(L10_SPACE), question, {
topK: 3,
minScore: 1.5,
})
assert.deepEqual(
[...raw].map((hit) => hit.text),
[],
`precondition: the threshold must remove every candidate; got ${JSON.stringify(
[...raw].map((hit) => hit.text),
)}`,
)
assert.equal(
raw.degraded,
'hybrid',
`a gated recall is NOT an empty space: the vector route did score candidates. Got ${raw.degraded}`,
)
assert.equal(raw.degradedError, undefined, 'and it carries no error')
// The log decision is the subject of L10, so drive the real method with the
// real gated result. Nothing here is a stub: `logRecallTurn` is what the
// plugin calls on every turn.
mem.logRecallTurn({
query: question,
messages: ask(question),
raw,
segmented: segmentRecallBudget(raw, { topK: 3 }),
injected: '',
})
assert.deepEqual(
lines(),
[],
`L11: a hybrid recall with zero surviving hits must still be silent with debugLog off; got ${JSON.stringify(lines())}`,
)
} finally {
await done()
}
})
// ══════════════════════════════════════════════════════════════════════════════
// L10† / L11† · the mutations that make the two silent assertions bite
// ══════════════════════════════════════════════════════════════════════════════
/**
* Copy the package, rewrite ONE anchor in `lib/index.js`, import it fresh.
*
* Same shape as L4† (and the mutation discipline note in docs/phase-b-criteria.md):
* the anchor must occur exactly once, or the test fails loudly rather than
* silently asserting against unmutated source.
*
* @param {string} anchor exact source text, expected once
* @param {string} replacement
* @param {(apply: Function) => Promise<void>} body receives the mutated `apply`
*/
async function withMutatedIndex(anchor, replacement, body) {
const { readFileSync, writeFileSync, mkdtempSync, rmSync, cpSync } = await import('node:fs')
const { tmpdir } = await import('node:os')
const { join } = await import('node:path')
const { fileURLToPath, pathToFileURL } = await import('node:url')
const libDir = fileURLToPath(new URL('../lib/', import.meta.url))
const pkgRoot = fileURLToPath(new URL('..', import.meta.url))
const dir = mkdtempSync(join(tmpdir(), 'mlog-l8-'))
try {
cpSync(libDir, join(dir, 'lib'), { recursive: true })
cpSync(join(pkgRoot, 'package.json'), join(dir, 'package.json'))
try {
cpSync(join(pkgRoot, 'node_modules'), join(dir, 'node_modules'), {
recursive: true,
dereference: false,
})
} catch {
// A poetry-free install fails loudly in the body below, which is correct.
}
const target = join(dir, 'lib', 'index.js')
const source = readFileSync(target, 'utf8')
assert.equal(
source.split(anchor).length - 1,
1,
`mutation anchor must occur exactly once — did the set move?\n${anchor}`,
)
writeFileSync(target, source.replace(anchor, replacement))
const mod = await import(pathToFileURL(target).href)
await body((ctx, config) => mod.apply(ctx, config))
} finally {
rmSync(dir, { recursive: true, force: true })
}
}
for (const [name, removed, bites] of [
['L10\u2020', "'hybrid'", 'L10.1'],
['L11\u2020', "'no_vector_space'", 'L10.2'],
]) {
test(`${name}: removing ${removed} from ORDINARY_DEGRADATIONS turns ${bites} red`, async (t) => {
const anchor = " 'ok', 'hybrid', 'no_embedder', 'aborted', 'no_vector_space', 'vector_ready_unused',"
const replacement = [
' // MUTATED: one healthy mode de-registered',
" 'ok', 'no_embedder', 'aborted',",
removed === "'hybrid'" ? " 'no_vector_space', 'vector_ready_unused'," : " 'hybrid', 'vector_ready_unused',",
].join('\n')
await withMutatedIndex(anchor, replacement, async (apply) => {
// The SAME criterion body the unmutated build passes. It must throw, and the
// throw must be the assertion's own — a different error (a bad import, a
// missing node_modules) would make this test green for the wrong reason.
await assert.rejects(
() => assertLevelContract((config) => makeMemWith(apply, config)),
(error) => {
assert.equal(error.code, 'ERR_ASSERTION', `only the criterion may bite; got ${error.code}: ${error.message}`)
assert.ok(
String(error.message).startsWith(bites),
`the biting assertion must be ${bites}; got: ${String(error.message).split('\n')[0]}`,
)
t.diagnostic(`${name} RAW RED >>> ${String(error.message).split('\n')[0]}`)
return true
},
`${name}: with ${removed} de-registered, ${bites} must go red`,
)
})
})
}

101
test/packaging.test.js Normal file
View File

@ -0,0 +1,101 @@
/**
* 打包完整性:`npm pack` 的产物必须能真的被加载。
*
* ## 这条判据为什么存在
*
* `package.json` 的 `files` 原本是
*
* ["lib/*.js", "lib/loop.js", "lib/loop/*.js", "lib/types/*.d.ts"]
*
* 而 `lib/*.js` 只匹配 lib/ 下一层。于是 `lib/embed/http.js` —— `lib/index.js`
* 第 35 行 `import { createHttpEmbedder } from './embed/http.js'` 的目标 ——
* 从来没有被打进包。
*
* 这个缺陷活下来的原因很具体:`npm pack --dry-run` 一直打印「10 files」,
* 数字看起来正常,没有人去读那十个文件名;而仓库里所有测试都是从源码树跑
* 的,源码树里那个文件当然存在。**只有在「从 tarball 安装」这条路径上才会
* 暴露**,而那正是发布出去的那条路径。
*
* 实测(修复前,把 tarball 解开后 import):
*
* IMPORT FAILED: ERR_MODULE_NOT_FOUND
* Cannot find module '.../package/lib/embed/http.js' imported from '.../package/lib/index.js'
*
* 修好之后同一条命令:`IMPORT OK, exports: StableMem,apply,default,...`
*
* ## 判据形态
*
* 静态、无副作用、不跑 npm:把 `files` 里的每个模式编译成正则,然后要求
* **lib/ 下的每一个 .js 都至少被一个模式匹配**。运行时需要的文件全在 lib/ 下,
* 所以这条覆盖了「漏文件」这一类,不需要维护一份白名单。
*
* 另加一条:`lib/index.js` 里所有相对 import 的目标必须存在于源码树 ——
* 否则上面那条可能在「文件根本不存在」时也绿(模式匹配不到不存在的文件)。
*
* @module dsh-memory/test/packaging
*/
import assert from 'node:assert/strict'
import test from 'node:test'
import { readFileSync, readdirSync } from 'node:fs'
import { join } from 'node:path'
const ROOT = new URL('..', import.meta.url)
/** 把 npm `files` 里的一条模式编译成正则(`*` 不跨目录,与 npm 一致)。 */
function patternToRegExp(pattern) {
const escaped = pattern.replace(/[.+^${}()|[\]\\]/g, '\\$&').replace(/\*/g, '[^/]*')
return new RegExp(`^${escaped}$`)
}
/** lib/ 下所有 .js 文件的相对路径。 */
function libFiles() {
const out = []
const walk = (relative) => {
for (const entry of readdirSync(new URL(relative + '/', ROOT), { withFileTypes: true })) {
const path = `${relative}/${entry.name}`
if (entry.isDirectory()) walk(path)
else if (entry.name.endsWith('.js')) out.push(path)
}
}
walk('lib')
return out.sort()
}
test('packaging: every file under lib/ is covered by package.json "files"', () => {
const pkg = JSON.parse(readFileSync(new URL('package.json', ROOT), 'utf8'))
assert.ok(Array.isArray(pkg.files), 'package.json must declare a "files" allowlist')
const patterns = pkg.files.map(patternToRegExp)
const missing = libFiles().filter((path) => !patterns.some((re) => re.test(path)))
assert.deepEqual(
missing,
[],
'these files live under lib/ but no "files" pattern reaches them, so they will be '
+ 'absent from the published tarball. `lib/*.js` matches ONE level only — a module '
+ 'in a subdirectory needs its own pattern. This is how lib/embed/http.js went '
+ 'missing while `npm pack --dry-run` happily reported a count:\n '
+ missing.join('\n '),
)
})
test('packaging: every relative import in lib/index.js resolves on disk', () => {
const source = readFileSync(new URL('lib/index.js', ROOT), 'utf8')
const specifiers = [...source.matchAll(/from\s+'(\.[^']+)'/g)].map((m) => m[1])
assert.ok(specifiers.length > 0, 'precondition: lib/index.js must have relative imports')
const unresolved = specifiers.filter((specifier) => {
try {
const url = new URL(specifier, new URL('lib/index.js', ROOT))
readFileSync(url)
return false
} catch {
return true
}
})
assert.deepEqual(
unresolved,
[],
`lib/index.js imports modules that do not exist: ${unresolved.join(', ')}`,
)
})

View File

@ -222,14 +222,22 @@ test('A3 mutation: a wrapper that adds any suffix to `why` must break A3', async
}
})
// ── the SUCCESS path's label (group A: vector computed, not yet used) ──────
// ── the SUCCESS path's label (P1 flips it: vectors now participate) ────────
//
// The earlier A3b only pinned the two failure labels. Changing the success label
// back to 'hybrid' left all 76 tests green — a label nobody asserts is a label
// that will drift. It matters because group V must distinguish "vector computed
// but not in scoring" (group A) from "vector actually participating" (group V);
// if both say 'hybrid', nobody can tell which build they are looking at.
test('A3c: a healthy embedder reports "vector_ready_unused", NOT "hybrid"', async () => {
// A3b only pinned the two failure labels. Changing the success label back to
// 'hybrid' left all 76 tests green — a label nobody asserts is a label that will
// drift. The two success labels answer DIFFERENT questions and both still matter:
//
// `no_vector_space` the space exists but holds no comparable vector (a
// fresh space). Says "nothing was scored" without
// claiming a provider failure.
// `hybrid` P1: the vector route really contributed scores to the
// same call that produced these hits.
//
// A build that reports `hybrid` while never passing `vectorScores` is EXACTLY the
// defect P1 exists for, so the assertion below (with a populated space) is the
// one that must never drift.
test('A3c: a healthy embedder over an EMPTY space reports "no_vector_space"', async () => {
const store = seededStore()
try {
let embedCalls = 0
@ -245,11 +253,11 @@ test('A3c: a healthy embedder reports "vector_ready_unused", NOT "hybrid"', asyn
assert.equal(embedCalls, 1, 'the embedder must actually be called')
assert.equal(
hits.degraded,
'vector_ready_unused',
'group A computes the vector but does not score with it — the label must say that, ' +
'because group V will need a different label once vectors really participate',
'no_vector_space',
'the fixture corpus carries no vectors in `fp-healthy`, so nothing can be ' +
'scored: the honest label says so instead of pretending a vector participated',
)
assert.notEqual(hits.degraded, 'hybrid', "calling it 'hybrid' would be a lie until group V lands")
assert.notEqual(hits.degraded, 'hybrid', "'hybrid' without a scored candidate would be a lie")
// And the result is still the pure-symbol ordering.
assert.equal(hits[0].text.includes('4324'), true)
} finally {
@ -257,18 +265,64 @@ test('A3c: a healthy embedder reports "vector_ready_unused", NOT "hybrid"', asyn
}
})
test('A3c mutation: labelling the success path "hybrid" must fail', () => {
// Mirrors the exact one-line edit that used to slip through: with no assertion
// on this value, reverting the rename was invisible.
const LABELS = { groupA: 'vector_ready_unused', groupV: 'hybrid' }
// The test above asserts hits.degraded === LABELS.groupA. Prove that swapping
// the constant would break it:
assert.notEqual(LABELS.groupA, LABELS.groupV)
const simulatedDeployed = LABELS.groupV // what the mutated lib returns
assert.throws(
() => assert.equal(simulatedDeployed, LABELS.groupA),
(err) => err.code === 'ERR_ASSERTION',
'if lib returns "hybrid", the A3c assertion must throw',
test('A3d: once the space holds vectors, the same recall reports "hybrid"', async () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
const fingerprint = 'fp-healthy'
store.putBlock({ text: '值班室分机号 4324', vector: [0.1, 0.2, 0.3], fingerprint })
const healthy = {
async embedText() { return [0.1, 0.2, 0.3] },
info: () => ({ fingerprint, dimension: 3 }),
}
const hits = await recallBlocksHybrid(store, healthy, '4324', { topK: 1 })
assert.equal(
hits.degraded,
'hybrid',
'a scored vector candidate is participating — reporting the group-A label here ' +
'would hide exactly the regression P1 pins',
)
assert.equal(hits.length, 1)
assert.ok(hits[0].vectorHit > 0, 'and the hit really carries a vector contribution')
} finally {
store.close()
}
})
test('A3c†: A3d really bites — drop the scoring pass and "hybrid" becomes unreachable', async (t) => {
// ★ A real mutation of lib/, not a re-assertion of local constants. The earlier
// version of this test compared two string literals it had just defined, which
// can never fail for a reason that matters (the same defect the mutation
// discipline note calls out: a guard that cannot fail is not a guard).
//
// The mutation empties the candidate scan. If A3d still reported `hybrid`, the
// label would be decoration rather than evidence — which is exactly the bug P1
// fixes one layer up.
await v5Mutant(
` const rows = store.vectorCandidates(fingerprint, dimension)`,
` const rows = [] // MUTATED: scoring pass produces no candidates`,
async (mod) => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
const fingerprint = 'fp-healthy'
store.putBlock({ text: '值班室分机号 4324', vector: [0.1, 0.2, 0.3], fingerprint })
const healthy = {
async embedText() { return [0.1, 0.2, 0.3] },
info: () => ({ fingerprint, dimension: 3 }),
}
const hits = await mod.recallBlocksHybrid(store, healthy, '4324', { topK: 1 })
assert.throws(
() => assert.equal(hits.degraded, 'hybrid'),
(err) => {
assert.equal(err.code, 'ERR_ASSERTION', 'only the criterion may bite')
t.diagnostic(`A3c† RAW RED >>> ${String(err.message).split('\n')[0]}`)
return true
},
'A3d must go red when nothing is scored',
)
} finally {
store.close()
}
},
)
})
@ -446,3 +500,593 @@ test('V2†: changing the Go weights to 0.5/0.5 bites the unchanged criterion re
rmSync(dir, { recursive: true, force: true })
}
})
// ── V3: vector and symbol contributions are separately attributable ──────
//
// Ported from Go `internal/memory/fusion.go:34-46`: `FusedHit` keeps `VectorHit`,
// `SymbolHit` and `Why` as separate facts, with 0 meaning "this path did not hit".
// V3 therefore requires the two numbers to vary INDEPENDENTLY — a single label
// (or one constant per shape) would make the three cases indistinguishable.
//
// Scope: attribution only. No symbol-only appending (V5), no topK reordering (V7).
function assertV3Attribution(recall, store) {
// `billing 端口` matches the Latin token `billing`, so the symbol path DOES hit
// (precondition for attribution) while no exact numeric run is present.
const query = 'billing 端口'
const plain = recall(store, query, { topK: 10 })
const symbolOnly = plain.find((hit) => hit.text === 'billing服务 端口=8499')
assert.ok(symbolOnly, 'precondition: the real billing block is in the result pool')
// (1) symbol-only: vector hit is exactly 0, symbol hit is not.
assert.equal(symbolOnly.vectorHit, 0, 'no vector score ⇒ VectorHit is 0 (go :38)')
assert.equal(symbolOnly.symbolHit, 1, 'the billing token actually matched: SymbolHit is 1')
// Existing CJK/lexical candidate, no high-information symbol match. Supply its
// vector score without appending a new record (candidate expansion remains V5).
const lexical = recall(store, '分机号', { topK: 5 })[0]
assert.ok(lexical, 'precondition: an existing lexical candidate is present')
const vectorOnly = recall(store, '分机号', {
topK: 5, vectorScores: new Map([[lexical.id, 0.9]]),
}).find((hit) => hit.id === lexical.id)
assert.equal(vectorOnly.vectorHit, 0.9, 'vector-only contribution comes from the supplied score')
assert.equal(vectorOnly.symbolHit, 0, 'no extracted symbol matched this lexical candidate')
assert.match(vectorOnly.why, /vector:0\.9;symbol:0;/)
// (2) exact hit: symbol score is the boolean 1.0 (go :300-305), still separate
// from whatever the vector says.
const exact = recall(store, '4324', { topK: 5 }).find((hit) => hit.exact === true)
assert.ok(exact, 'precondition: an exact hit exists')
assert.equal(exact.symbolHit, 1.0, 'exact hit carries SymbolHit 1.0')
assert.equal(exact.vectorHit, 0, 'and no vector score was supplied')
// (3) same record WITH a vector score: both non-zero, and the two numbers are
// different facts rather than one value copied into both fields.
const withVector = recall(store, query, {
topK: 10, vectorScores: new Map([[symbolOnly.id, 0.6]]),
}).find((hit) => hit.id === symbolOnly.id)
assert.equal(withVector.vectorHit, 0.6, 'VectorHit echoes the supplied vector score')
assert.equal(withVector.symbolHit, symbolOnly.symbolHit, 'SymbolHit is unchanged by the vector')
assert.notEqual(withVector.vectorHit, withVector.symbolHit, 'the two signals are distinct facts')
// (4) the attribution is readable in the existing `why` field.
assert.match(withVector.why, /vector:0\.6/, 'why carries the vector score')
assert.match(withVector.why, /symbol:/, 'why carries the symbol score')
}
test('V3: vector and symbol contributions are separately attributable', () => {
const store = seededStore()
try {
assertV3Attribution(recallBlocks, store)
} finally {
store.close()
}
})
for (const [name, anchor, replacement] of [
['symbol', ' const symbolHit = exact ? 1.0 : coverage', ' const symbolHit = vectorHit // MUTATED: collapsed signals'],
['vector', ' const vectorHit = vectorScore', ' const vectorHit = 0 // MUTATED: lost vector attribution'],
['why', '? `vector:${vectorHit};symbol:${symbolHit};', '? `opaque;'],
]) {
test(`V3† ${name}: the unchanged attribution criterion bites red`, async (t) => {
const { readFileSync, writeFileSync, mkdtempSync, rmSync, cpSync } = await import('node:fs')
const { tmpdir } = await import('node:os')
const { join } = await import('node:path')
const { fileURLToPath, pathToFileURL } = await import('node:url')
const libDir = fileURLToPath(new URL('../lib/', import.meta.url))
const pkgRoot = fileURLToPath(new URL('..', import.meta.url))
const dir = mkdtempSync(join(tmpdir(), 'mv3-'))
try {
cpSync(libDir, join(dir, 'lib'), { recursive: true })
cpSync(join(pkgRoot, 'package.json'), join(dir, 'package.json'))
const target = join(dir, 'lib', 'recall.js')
let source = readFileSync(target, 'utf8')
assert.equal(source.split(anchor).length - 1, 1, 'M-V3 anchor must occur exactly once')
source = source.replace(anchor, replacement)
writeFileSync(target, source)
const mod = await import(pathToFileURL(target).href)
const store = seededStore()
try {
assert.throws(() => assertV3Attribution(mod.recallBlocks, store), (err) => {
assert.equal(err.code, 'ERR_ASSERTION', 'only the criterion may bite')
t.diagnostic(`V3† ${name} RAW RED >>> ${String(err.message).split('\n')[0]}`)
return true
}, 'M-V3 must break the unchanged V3 criterion')
} finally {
store.close()
}
} finally {
rmSync(dir, { recursive: true, force: true })
}
})
}
// ── V5: MinScore applies PER PATH, never to the fused score ──────────────
//
// Ported from Go `internal/memory/fusion.go:541-554`. Three different scales in
// one fused number (exact pin ≥1.0, pure-symbol 0..0.3, pure-vector 0..0.7) is
// why Go applies the threshold per route and records the alternative as a bug
// (`:523-532`). The gate therefore keys on the CANDIDATE SOURCE, never on
// whether the `vectorHit` field happens to exist — every record carries it since
// V3, so existence-based branching would silently destroy the asymmetry.
function assertV5AsymmetricGate(recall, store) {
const query = 'billing 端口'
const plain = recall(store, query, { topK: 10 })
const billing = plain.find((hit) => hit.text === 'billing服务 端口=8499' && hit.exact === false)
assert.ok(billing, 'precondition: the billing record is retrievable')
assert.equal(billing.symbolHit, 1, 'precondition: it matched the symbol path (SymbolHit 1)')
const lexical = recall(store, '分机号', { topK: 5 })[0]
assert.ok(lexical, 'precondition: an existing lexical candidate exists')
// (a) SYMBOL route, no vector source: MinScore must NOT gate it. Go :549-553
// gates this branch on coverage (> 0), deliberately not on the vector threshold
// — otherwise a 1.0 symbol hit worth 0.3 fused would be killed by MinScore 0.5.
const symbolKept = recall(store, query, { topK: 10, minScore: 0.5, vectorScores: new Map() })
assert.ok(symbolKept.some((hit) => hit.id === billing.id),
'a symbol-only hit must survive MinScore 0.5 — the gate is not on the symbol scale')
// (b) VECTOR route: the threshold applies to the raw VECTOR score.
const vectorKept = recall(store, query, {
topK: 10, minScore: 0.5, vectorScores: new Map([[billing.id, 0.9]]),
})
assert.ok(vectorKept.some((hit) => hit.id === billing.id), 'vector 0.9 >= 0.5 must pass')
const vectorDropped = recall(store, query, {
topK: 10, minScore: 0.5, vectorScores: new Map([[billing.id, 0.1]]),
})
assert.equal(vectorDropped.some((hit) => hit.id === billing.id), false,
'vector 0.1 < 0.5 must be dropped by the VECTOR-side gate')
// (c) raw vector 0.81 passes 0.8, while its fused score with the existing
// CJK symbol score (0.75) is 0.792 < 0.8. The gate must use the raw score.
const belowFused = recall(store, '分机号', {
topK: 5, minScore: 0.8, vectorScores: new Map([[lexical.id, 0.81]]),
})
const belowFusedHit = belowFused.find((hit) => hit.id === lexical.id)
assert.ok(belowFusedHit, 'raw vector 0.81 >= 0.8 must pass even when fused score is below 0.8')
assert.ok(belowFusedHit.score < 0.8,
`precondition: fused ${belowFusedHit.score} is under the 0.8 threshold`)
// A lexical candidate without a vector source and without a SymbolScore hit
// has neither route: Go's default branch drops it, rather than treating the
// existence of the V3 attribution fields as evidence of a hit.
const neither = recall(store, '分机号', {
topK: 5, minScore: 0.8, vectorScores: new Map(),
})
assert.equal(neither.some((hit) => hit.id === lexical.id), false,
'no vector source and SymbolHit 0 must be dropped by the symbol coverage gate')
// (d) EXACT route: the pin is decisive, never gated.
const exactId = recall(store, '4324', { topK: 5 }).find((hit) => hit.exact === true)?.id
assert.ok(exactId, 'precondition: an exact hit exists')
const exactKept = recall(store, '4324', {
topK: 5, minScore: 0.9, vectorScores: new Map([[exactId, 0]]),
})
assert.ok(exactKept.some((hit) => hit.id === exactId), 'an exact hit is never filtered by MinScore')
// (e) minScore 0 disables only the VECTOR threshold; Go's default branch
// (symbol-only with coverage 0 is dropped, go :549-553) still runs.
const zero = recall(store, '分机号', { topK: 5, minScore: 0, vectorScores: new Map() })
assert.equal(zero.some((hit) => hit.id === lexical.id), false,
'a symbol-only record with SymbolHit 0 is dropped even when minScore is 0')
// (f) no threshold supplied => the result is untouched.
const noGate = recall(store, query, { topK: 10 })
assert.deepEqual(noGate.map((h) => h.id), plain.map((h) => h.id),
'minScore absent leaves the result unchanged')
}
test('V5: MinScore gates the vector route only, never the symbol or exact route', () => {
const store = seededStore()
try {
assertV5AsymmetricGate(recallBlocks, store)
} finally {
store.close()
}
})
// NOTE: these mutants load a mutated copy of lib/recall.js directly (the sibling
// helper `withMutatedCopy` lives in phase-b-h.test.js and is not in scope here).
async function v5Mutant(anchor, replacement, body) {
const { readFileSync, writeFileSync, mkdtempSync, rmSync, cpSync } = await import('node:fs')
const { tmpdir } = await import('node:os')
const { join } = await import('node:path')
const { fileURLToPath, pathToFileURL } = await import('node:url')
const libDir = fileURLToPath(new URL('../lib/', import.meta.url))
const pkgRoot = fileURLToPath(new URL('..', import.meta.url))
const dir = mkdtempSync(join(tmpdir(), 'mv5-'))
try {
cpSync(libDir, join(dir, 'lib'), { recursive: true })
cpSync(join(pkgRoot, 'package.json'), join(dir, 'package.json'))
const target = join(dir, 'lib', 'recall.js')
let source = readFileSync(target, 'utf8')
assert.equal(source.split(anchor).length - 1, 1, 'V5 mutation anchor must occur exactly once')
writeFileSync(target, source.replace(anchor, replacement))
await body(await import(pathToFileURL(target).href))
} finally {
rmSync(dir, { recursive: true, force: true })
}
}
for (const [name, anchor, replacement] of [
[
'vector-side',
` if (hasVectorScore && vectorScore > 0) {\n if (minScore > 0 && vectorScore < minScore) {`,
` if (false) {\n if (minScore > 0 && vectorScore < minScore) { // MUTATED: vector gate disabled`,
],
[
'symbol-side',
` } else if (!(symbolHit > 0)) {\n gatedNoEvidence += 1\n continue`,
` } else if (false) {\n gatedNoEvidence += 1\n continue // MUTATED: symbol-side gate removed`,
],
[
'symbol-threshold',
` if (hasVectorScore && vectorScore > 0) {`,
` if (hasVectorScore || symbolHit > 0) { // MUTATED: apply vector threshold to symbols`,
],
[
'fused-score',
` if (minScore > 0 && vectorScore < minScore) {`,
` if (score < minScore) { // MUTATED: gate on the fused score`,
],
]) {
test(`V5† ${name}: the unchanged asymmetric-gate criterion bites red`, async (t) => {
await v5Mutant(anchor, replacement, async (mod) => {
const store = seededStore()
try {
assert.throws(() => assertV5AsymmetricGate(mod.recallBlocks, store), (err) => {
assert.equal(err.code, 'ERR_ASSERTION', 'only the criterion may bite')
t.diagnostic(`V5† ${name} RAW RED >>> ${String(err.message).split('\n')[0]}`)
return true
}, `V5† ${name} must break the unchanged criterion`)
} finally {
store.close()
}
})
})
}
// ── V5b: append before scoring must preserve existing symbol/exact facts ──
//
// Ported from Go `internal/memory/fusion.go:384-408`. Go's comment (`:364-373`)
// records the &out[i]-after-append bug: symbol facts were written into the
// discarded backing array, so the score was never computed. JS objects are
// references, but the requirement survives verbatim — the symbol/exact facts
// must land on the record that SCORING reads. The criterion asserts VALUES on
// records that existed BEFORE the append, which is exactly the half that broke
// in Go.
function assertV5SymbolAdmission(recall, store) {
const primary = store.putBlock({ text: 'billing 4324 primary' })
const appended = store.putBlock({ text: 'billing appended' })
const pinned = store.putBlock({ text: '4324 appended exact' })
const options = { topK: 10, poolSize: 1, minScore: 0 }
// poolSize=1 leaves exactly ONE primary candidate, so the other two can only
// arrive through the symbol route.
const initial = recall(store, 'billing 4324', { topK: 10, poolSize: 1 })
assert.equal(initial.length, 1, 'precondition: one primary candidate')
const primaryId = initial[0].id
const primaryExact = initial[0].exact
const hits = recall(store, 'billing 4324', {
...options, vectorScores: new Map([[primaryId, 0.6]]),
})
// (1) admission happens at all.
assert.ok(hits.some((hit) => hit.id === appended.id), 'the word-only symbol record is admitted')
assert.ok(hits.some((hit) => hit.id === pinned.id), 'the numeric symbol record is admitted')
// (2) ★ the PRE-APPEND candidate keeps its computed symbol/exact facts.
const existing = hits.find((hit) => hit.id === primaryId)
assert.ok(existing, 'the original candidate survives the append')
assert.ok(existing.symbolHit > 0, 'existing candidate retains SymbolHit after append')
assert.equal(existing.exact, primaryExact, 'existing candidate retains ExactHit after append')
assert.ok(existing.score > 0, 'existing candidate score was actually computed')
// (3) appended records carry their own facts, written to the scored record.
const word = hits.find((hit) => hit.id === appended.id)
assert.equal(word.symbolHit, 0.5, 'one of two symbols matched the appended word record')
assert.equal(word.exact, false, 'a Latin symbol hit is not an exact hit')
assert.equal(word.score, 0.15, 'pure-symbol score is sym*w.Symbol (go :430-432)')
const exact = hits.find((hit) => hit.id === pinned.id)
assert.equal(exact.exact, true, 'appended numeric match retains ExactHit')
assert.equal(exact.symbolHit, 1, 'and SymbolHit is the boolean 1.0')
assert.equal(exact.score, 1, 'appended exact match stays pinned')
// (4) ★ admission scans raw blocks, so the F2/F4 filters must still apply.
const stale = store.putBlock({ text: 'billing stale-space 9999', vector: [1, 0, 0], fingerprint: 'fp-stale' })
const scoped = recall(store, 'billing 4324 9999', {
...options, fingerprint: 'fp-current', dimension: 3,
vectorScores: new Map([[primaryId, 0.6]]),
})
assert.equal(scoped.some((hit) => hit.id === stale.id), false,
'a mismatched-space block must not be admitted by the symbol route')
assert.ok(scoped.skipped + scoped.skippedDim >= 1, 'and the exclusion is counted')
}
test('V5b: append preserves existing candidates and admits symbol-only records', () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
assertV5SymbolAdmission(recallBlocks, store)
} finally {
store.close()
}
})
for (const [name, anchor, replacement] of [
['lost-symbol', ' symbolHit,\n why: vectorScores instanceof Map',
' symbolHit: 0, // MUTATED: detached symbol write\n why: vectorScores instanceof Map'],
['lost-exact', ' score,\n exact,\n vectorHit,',
' score,\n exact: false, // MUTATED: detached exact write\n vectorHit,'],
['admission-off', ' if (!symbolMatch && !vectorSource) continue',
' if (true) continue // MUTATED: no assembly'],
]) {
test(`V5b† ${name}: the unchanged append criterion bites red`, async (t) => {
await v5Mutant(anchor, replacement, async (mod) => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
assert.throws(() => assertV5SymbolAdmission(mod.recallBlocks, store), (err) => {
assert.equal(err.code, 'ERR_ASSERTION', 'only the criterion may bite')
t.diagnostic(`V5b† ${name} RAW RED >>> ${String(err.message).split('\n')[0]}`)
return true
}, `V5b† ${name} must break the unchanged criterion`)
} finally {
store.close()
}
})
})
}
// ── V4 (b): a self-built tier-3 fixture the vector route can actually flip ──
//
// E2E-1 ruling: checkpoint V2 keeps its own assertions and its "known gap"
// output verbatim — the flip is proven HERE, by a new case, not by rewriting V2.
//
// ★ What this case does and does NOT claim (stated so it cannot be over-read):
// · it uses a REAL store (blocks written through putBlock) and the REAL recall
// scoring path (`recallBlocks` with a supplied `vectorScores` map);
// · the vectors are a deterministic fixture, so this proves the MECHANISM —
// a tier-3 style query (no extracted symbol, no lexical overlap) becomes
// retrievable once a vector candidate is in the pool;
// · it does NOT prove model-grade semantic generalisation (that needs a real
// embedder and cannot be pinned in a unit test), and it does not wire the
// embedder into `recallBlocksHybrid` — that seam is still A3c's
// `vector_ready_unused` and stays untouched in this slice.
const TIER3 = {
// Verbatim shape of checkpoint V2's known gap: 纯泛化描述, no symbol, and the
// target block shares NO token with the query.
query: '公网地址是什么',
blocks: [
{ text: '对外的访问入口 10.0.0.7:3000', source: 'v4' },
{ text: '监控面板 http://10.0.0.7:3000', source: 'v4' },
{ text: '财务部门 负责人=员工100', source: 'v4' },
],
// Deterministic "embedding": the first block is semantically nearest.
queryVector: [1, 0, 0],
vectorFor: (index) => (index === 0 ? [0.95, 0.31, 0] : [0.1, 1, 0]),
}
function cosine(a, b) {
let dot = 0
let na = 0
let nb = 0
for (let i = 0; i < a.length; i += 1) {
dot += a[i] * b[i]
na += a[i] * a[i]
nb += b[i] * b[i]
}
return dot / (Math.sqrt(na) * Math.sqrt(nb))
}
function tier3Store() {
const store = new BlockStore({ path: ':memory:' }).open()
const ids = TIER3.blocks.map((block, index) => store.putBlock({
...block, vector: TIER3.vectorFor(index), fingerprint: 'v4-fixture',
}).id)
return { store, ids }
}
/** The criterion, shared by the positive case and every mutant. */
function assertTier3Flipped(recall) {
const { store, ids } = tier3Store()
try {
// (1) The gap is real on the symbol/lexical path alone: nothing is recalled.
const symbolOnly = recall(store, TIER3.query, { topK: 3 })
assert.equal(symbolOnly.length, 0,
'precondition: the tier-3 query matches no block by symbol or lexical overlap')
// (2) With vector candidates, the semantically nearest block is recalled.
const vectorScores = new Map(store.allBlocks(-1).map((block) => [
block.id, cosine(TIER3.queryVector, JSON.parse(block.vector)),
]))
const withVectors = recall(store, TIER3.query, { topK: 1, minScore: 0, vectorScores })
assert.equal(withVectors.length, 1, 'the vector route must surface exactly one block at topK 1')
assert.equal(withVectors[0].id, ids[0], 'and it is the semantically nearest block, not a lexical accident')
assert.equal(withVectors[0].text, TIER3.blocks[0].text)
// (3) The flipped hit is attributable to the vector route, not to a symbol hit.
assert.equal(withVectors[0].symbolHit, 0, 'no symbol matched — the flip is purely the vector route')
assert.ok(withVectors[0].vectorHit > 0.9, 'the vector contribution is what carried it')
assert.equal(withVectors[0].exact, false, 'and it is not an exact pin')
} finally {
store.close()
}
}
test('V4: a self-built tier-3 case flips from no-hit to recalled with stored vectors', () => {
assertTier3Flipped(recallBlocks)
})
for (const [name, anchor, replacement] of [
[
'symbol-selection',
' if (!symbolMatch && !vectorSource) continue',
' if (!symbolMatch) continue // MUTATED: vector candidates never admitted',
],
[
'vector-ranking',
' scored.sort((a, b) => (b.score === a.score ? a.id.localeCompare(b.id) : b.score - a.score))',
' scored.sort((a, b) => (b.score === a.score ? a.id.localeCompare(b.id) : a.score - b.score)) // MUTATED: ascending',
],
[
'vector-attribution',
' const vectorHit = vectorScore',
' const vectorHit = 0 // MUTATED: attribution drops the vector contribution',
],
]) {
test(`V4† ${name}: the unchanged tier-3 flip criterion bites red`, async (t) => {
await v5Mutant(anchor, replacement, async (mod) => {
assert.throws(() => assertTier3Flipped(mod.recallBlocks), (err) => {
assert.equal(err.code, 'ERR_ASSERTION', 'only the criterion may bite')
t.diagnostic(`V4† ${name} RAW RED >>> ${String(err.message).split('\n')[0]}`)
return true
}, `V4† ${name} must break the unchanged criterion`)
})
})
}
// ── V7: the vector route is bounded by largeTopK, not by the caller's topK ──
//
// Ported from Go `internal/memory/fusion.go:466-468` (`q2.TopK = largeTopK`: the
// recall read is widened BEFORE fusion) and `internal/memory/block_recall.go:263`
// (`const largeTopK = 2000`). The caller's topK is applied exactly once, at the
// end (`fusion.go:557-559` · `block_recall.go:176-178`).
//
// ★ Three separate criteria, because three separate things can go wrong:
// (a) the vector pool must not be cut to a topK-sized window before fusion;
// (b) the bound must really be 2000 — measured, not just read from the source;
// (c) the caller's topK must still be honoured at the end (widening ≠ leaking).
// Each has its own mutant, so no mutant can hide behind another's failure.
/** (a) A vector candidate the route ranks LAST must still reach fusion and win. */
function assertV7NoEarlyVectorCut(recall, store) {
// Symbol coverage 1 beats the higher-vector noise (coverage 1/3) only after
// fusion: target=0.6*0.7+1*0.3=0.72; noise<=0.8*0.7+(1/3)*0.3=0.66.
const target = store.putBlock({ text: 'alpha beta gamma', source: 'v7' }).id
const others = []
for (let i = 0; i < 5; i += 1) others.push(store.putBlock({ text: `alpha noise ${i}`, source: 'v7' }).id)
const scores = new Map([[target, 0.6], ...others.map((id, i) => [id, 0.8 - i * 0.01])])
const hits = recall(store, 'alpha beta gamma', { topK: 1, minScore: 0, vectorScores: scores })
assert.equal(hits.length >= 1 && hits[0].id, target,
'the worst-scoring vector candidate must still reach fusion (largeTopK, not topK)')
}
/** (b) 2100 vector candidates ⇒ exactly 2000 survive the bound (measured). */
function assertV7LargeTopKBound(recall, store) {
const ids = []
for (let i = 0; i < 2100; i += 1) ids.push(store.putBlock({ text: `V7 bulk ${i}`, source: 'v7' }).id)
const scores = new Map(ids.map((id, index) => [id, (index + 1) / 2100]))
const hits = recall(store, '公网地址是什么', { topK: 2100, minScore: 0, vectorScores: scores })
assert.equal(hits.length, 2000, 'the vector route is bounded at largeTopK = 2000')
// The 100 lowest-scored candidates fall outside the bound and cannot appear.
const dropped = new Set(ids.slice(0, 100))
assert.equal(hits.some((hit) => dropped.has(hit.id)), false,
'candidates ranked below the 2000th must not survive the bound')
}
/** (c) The caller's topK is applied once, after fusion. */
function assertV7FinalTruncation(recall, store) {
for (let i = 0; i < 6; i += 1) store.putBlock({ text: `V7 trunc probe ${i}`, source: 'v7' })
assert.equal(recall(store, 'V7 trunc probe', { topK: 2, minScore: 0, vectorScores: new Map() }).length, 2,
'topK 2 must return exactly 2 rows after widening')
assert.equal(recall(store, 'V7 trunc probe', { topK: 4, minScore: 0, vectorScores: new Map() }).length, 4,
'topK 4 must return exactly 4 rows after widening')
assert.equal(recall(store, 'V7 trunc probe', { topK: 2 }).length, 2,
'the non-fused path still honours topK')
}
test('V7: the vector route is widened before fusion and truncated once at the end', () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
assertV7NoEarlyVectorCut(recallBlocks, store)
} finally {
store.close()
}
const wide = new BlockStore({ path: ':memory:' }).open()
try {
assertV7LargeTopKBound(recallBlocks, wide)
} finally {
wide.close()
}
const tail = new BlockStore({ path: ':memory:' }).open()
try {
assertV7FinalTruncation(recallBlocks, tail)
} finally {
tail.close()
}
})
test('V7: largeTopK is the Go constant 2000, read from source', async () => {
const { readFileSync } = await import('node:fs')
const source = readFileSync(new URL('../lib/recall.js', import.meta.url), 'utf8')
const match = source.match(/const largeTopK = (\d+)/)
assert.ok(match, 'largeTopK must exist in lib/recall.js')
assert.equal(Number(match[1]), 2000, 'ported verbatim from block_recall.go:263')
})
/** Copy lib/, mutate one anchor with a uniqueness self-check, then import. */
async function withMutatedCopy(anchor, replacement, body) {
const { readFileSync, writeFileSync, mkdtempSync, rmSync, cpSync } = await import('node:fs')
const { tmpdir } = await import('node:os')
const { join } = await import('node:path')
const { fileURLToPath, pathToFileURL } = await import('node:url')
const libDir = fileURLToPath(new URL('../lib/', import.meta.url))
const pkgRoot = fileURLToPath(new URL('..', import.meta.url))
const dir = mkdtempSync(join(tmpdir(), 'mv7-'))
try {
cpSync(libDir, join(dir, 'lib'), { recursive: true })
cpSync(join(pkgRoot, 'package.json'), join(dir, 'package.json'))
const target = join(dir, 'lib', 'recall.js')
let source = readFileSync(target, 'utf8')
assert.equal(source.split(anchor).length - 1, 1, 'V7 mutation anchor must occur exactly once')
writeFileSync(target, source.replace(anchor, replacement))
await body(await import(pathToFileURL(target).href))
} finally {
rmSync(dir, { recursive: true, force: true })
}
}
for (const [name, anchor, replacement, criterion] of [
[
'vector-pool-truncated',
'.slice(0, largeTopK).map(([id]) => id))',
'.slice(0, topK).map(([id]) => id)) // MUTATED: vector pool cut before fusion',
assertV7NoEarlyVectorCut,
],
[
'largeTopK-shrunk',
'const largeTopK = 2000',
'const largeTopK = 100 // MUTATED: bound shrunk below Go largeTopK',
assertV7LargeTopKBound,
],
[
'result-not-truncated',
' return withSkipped(scored.slice(0, topK), skipped, skippedDim, drops)',
' return withSkipped(scored, skipped, skippedDim, drops) // MUTATED: caller topK never applied',
assertV7FinalTruncation,
],
]) {
test(`V7† ${name}: the unchanged criterion bites red`, async (t) => {
await withMutatedCopy(anchor, replacement, async (mod) => {
t.diagnostic(`V7† ${name} anchor split count === 1: ${anchor}`)
const store = new BlockStore({ path: ':memory:' }).open()
try {
let failure = null
try {
criterion(mod.recallBlocks, store)
} catch (error) {
failure = error
}
assert.ok(failure !== null, `V7† ${name} must break the unchanged criterion`)
assert.equal(failure.code, 'ERR_ASSERTION', 'only the criterion may bite')
t.diagnostic(`V7† ${name} RAW RED >>> ${String(failure.message).split('\n')[0]}`)
} finally {
store.close()
}
})
})
}

View File

@ -0,0 +1,751 @@
import assert from 'node:assert/strict'
import test from 'node:test'
import { BlockStore } from '../lib/store.js'
import { recallBlocksHybrid } from '../lib/recall.js'
// ══════════════════════════════════════════════════════════════════════════
// C4 · 中心化校正(graph_centroid)
// ══════════════════════════════════════════════════════════════════════════
//
// 判据来源 docs/phase-b-criteria.md:253
// C4 校正时查询向量与块向量都去均值,不能只处理一边
// 依据 centerBoth(centroid_store.go:443-451)
// + 注释「只处理一边等于混两种量纲」
//
// ★ 为什么必须自建空间,不能复用 P1 的语料
// P1 的假 embedder 是 8 族正交 one-hot:每条块独占一维,均值向量趋近 0
// (实测 mean norm = 0.3536 < MEAN_THRESHOLD 0.6)⇒ isAnomalous = false
// ⇒ 正确门控的 C4 会自动关闭去均值 ⇒ 判据全绿但什么都没验到。
// 所以这里手工构造各向异性空间:所有块共享一个强背景方向,
// 各自只叠一个小 offset,均值范数才会高到触发校正。
//
// ★ 三条判据的分工(别混)
// C4a born-RED 未实现时红(裸余弦 top1 = hub,去均值后应为 target)
// C4b mutation-GUARD 未实现时绿(裸余弦下同值对本来就 = 1),
// 只有「只减一边」能让它红 —— 抓 C4 的病根。
// 实测:只减 query 时 C4a 的 top1 仍然是 target,
// 所以 C4a 抓不到这个错误,必须靠 C4b。
// C4c guard 无中心时照常返回结果,只是不校正
//
// ★ What this does NOT establish
// 中心化改变的是向量间的夹角,因此能改变排序。
// 它不能让一个没校准的绝对分数变得可用:去均值之后余弦仍然是余弦,
// 而 DEFAULT_MIN_SCORE = 0.5 从来不是从中心化后的分布推出来的。
// 阈值校准仍是 phase-b-p1-vector-wiring.test.js:51-58 记录的已知未解问题。
const DIM = 8
const SPACE = 'c4:centering:test'
// ── 空间常数(钉死;改一个就必须重测翻转窗口)────────────────────────────
const HUB_EPS = 0.02
const TARGET_EPS = 0.30
const QUERY_WEIGHT = 0.40
const MEAN_THRESHOLD = 0.6
const norm = (v) => Math.sqrt(v.reduce((s, x) => s + x * x, 0))
const unit = (v) => { const n = norm(v); return n === 0 ? v : v.map((x) => x / n) }
const axis = (i) => unit(Array.from({ length: DIM }, (_, k) => (k === i ? 1 : 0)))
// 共同背景方向:所有块都携带它,这是各向异性的来源
const BACKBONE = unit([1.0, 0.55, 0.42, 0.30, 0.22, 0.16, 0.11, 0.07])
const TARGET_AXIS = axis(7)
const mk = (eps, ax) => unit(BACKBONE.map((x, i) => x + eps * ax[i]))
const HUB_VEC = mk(HUB_EPS, axis(1))
const TARGET_VEC = mk(TARGET_EPS, TARGET_AXIS)
const DISTRACTOR_VECS = [0, 2, 3, 4, 5, 6].map((i) => mk(TARGET_EPS, axis(i)))
const QUERY_TEXT = '上次说的那个四位数是什么来着?'
const QUERY_VEC = unit(BACKBONE.map((x, i) => x + TARGET_EPS * QUERY_WEIGHT * TARGET_AXIS[i]))
const HUB = { key: 'hub', text: '这段时间一切照旧,没有什么特别的事情。', vec: HUB_VEC }
const TARGET = { key: 'target', text: '第七号仓库的备用门禁码是 QP47-812。', vec: TARGET_VEC }
const DISTRACTORS = DISTRACTOR_VECS.map((vec, i) => ({
key: `d${i}`, text: `干扰块 ${i}:与本问无关的日常事项。`, vec,
}))
const CORPUS = [HUB, TARGET, ...DISTRACTORS]
const byText = new Map(CORPUS.map((b) => [b.text, b.vec]))
function makeEmbedder() {
return {
async embedText(text) {
if (byText.has(text)) return byText.get(text)
if (text === QUERY_TEXT) return QUERY_VEC
return new Array(DIM).fill(0)
},
info: () => ({ fingerprint: SPACE, dimension: DIM }),
}
}
/** 建一个装满 CORPUS 的库,返回 store 与 id→key 的映射。 */
function seedStore({ withCentroid = true } = {}) {
const store = new BlockStore({ path: ':memory:' }).open()
const keyOfId = new Map()
for (const block of CORPUS) {
const written = store.putBlock({ text: block.text, vector: block.vec, fingerprint: SPACE })
keyOfId.set(written.id, block.key)
}
if (withCentroid) store.rebuildCentroid(SPACE)
return { store, keyOfId }
}
/** 召回并按 key 呈现,便于断言。 */
async function recallByKey(store, keyOfId, options = {}) {
const hits = await recallBlocksHybrid(store, makeEmbedder(), QUERY_TEXT, { topK: 8, ...options })
return hits.map((hit) => ({ key: keyOfId.get(hit.id) ?? hit.id, score: hit.score, vectorHit: hit.vectorHit }))
}
const fmt = (hits) => hits.map((h) => `${h.key}=${h.score.toFixed(4)}(v=${Number(h.vectorHit ?? 0).toFixed(4)})`).join(' ')
// ── 前置:空间真的会触发校正 ────────────────────────────────────────────
//
// 没有这三条,后面任何断言都可能因为「校正压根没启用」而通过,那是假绿。
test('C4-0: the fixture space is anisotropic enough to enable centering', () => {
const { store } = seedStore()
try {
const centroid = store.loadCentroid(SPACE)
assert.ok(centroid !== undefined, 'precondition: a centroid must exist after rebuildCentroid')
assert.equal(centroid.isAnomalous, true, 'precondition: the space must be judged anisotropic')
assert.ok(
norm(centroid.mean) >= MEAN_THRESHOLD,
`precondition: mean norm ${norm(centroid.mean).toFixed(6)} must clear MEAN_THRESHOLD ${MEAN_THRESHOLD}`,
)
assert.equal(centroid.dim, DIM, 'precondition: centroid dimension must match the space')
assert.equal(centroid.vectorCount, CORPUS.length, 'precondition: every block contributed')
} finally {
store.close()
}
})
// ── C4a: 校正把 top-1 从 hub 翻到 target ────────────────────────────────
//
// born-RED:在有中心但不去均值的构建上,裸余弦会让 hub 赢
// (实测 hub 0.9928 > target 0.9859),因为 hub 几乎就是背景本身,
// 而背景是 query 与所有块共有的最大分量。
test('C4a: centering flips top-1 from the hub to the target', async (t) => {
const { store, keyOfId } = seedStore()
try {
const hits = await recallByKey(store, keyOfId)
t.diagnostic(`C4a ranking: ${fmt(hits)}`)
assert.ok(hits.length > 0, 'precondition: the recall must return something')
assert.equal(
hits[0].key,
'target',
'C4a: after centering the target must be top-1. ' +
'If this is "hub", the space mean is still dominating the score — either ' +
'centering is not wired at all, or only one side was centered. ' +
`got ${fmt(hits)}`,
)
// 分数也要过闸,否则「top1 对了」但整轮被 minScore 清空就成了空断言
assert.ok(
hits[0].score > 0.5,
`C4a: the winning score must clear DEFAULT_MIN_SCORE 0.5; got ${hits[0].score}`,
)
} finally {
store.close()
}
})
// ── C4b: 同值对的余弦恒为 1(mutation-GUARD)──────────────────────────────
//
// 这是唯一一条**不依赖语料构造、纯代数必然成立**的性质:
// 若块向量与查询向量完全相同,余弦必须恰好是 1 —— 不论是否中心化。
// · 两侧都减同一个均值 ⇒ (v-m) 与 (v-m) 的余弦 = 1 ✓
// · 只减一边 ⇒ (v-m) 与 v 的余弦 ≠ 1 ✗ ← M-C4 正是这个
//
// ★ 它在未实现的构建上是绿的(裸余弦下同值对本来就 = 1),
// 所以它不是 born-red 判据,而是抓 M-C4 的那张网。别因为它「一开始就绿」删掉它:
// 实测「只减 query」时 C4a 的 top-1 仍然是 target,C4a 抓不到这个错误。
test('C4b: a block identical to the query always scores a cosine of 1', async (t) => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
// 一条与 query 完全同值的块,外加两条背景块,凑出会触发校正的空间
const twin = store.putBlock({ text: '与查询同值的那条', vector: QUERY_VEC, fingerprint: SPACE })
for (const vec of [HUB_VEC, ...DISTRACTOR_VECS.slice(0, 3)]) {
store.putBlock({ text: `背景块 ${Math.random()}`, vector: vec, fingerprint: SPACE })
}
store.rebuildCentroid(SPACE)
const hits = await recallBlocksHybrid(store, makeEmbedder(), QUERY_TEXT, { topK: 8, minScore: 0 })
const twinHit = hits.find((hit) => hit.id === twin.id)
assert.ok(twinHit !== undefined, `C4b: precondition — the twin block must be scored; got ${hits.length} hits`)
t.diagnostic(`C4b twin vectorHit = ${twinHit.vectorHit}`)
assert.ok(
Math.abs(twinHit.vectorHit - 1) < 1e-9,
'C4b: an identical vector pair must score exactly 1. ' +
'A value near 0.1897 on this 5-block fixture is the signature of centering ONE '
+ 'side only ' +
'(M-C4) — subtracting the mean from the query but not the block mixes ' +
'two coordinate systems and the cosine stops being an angle. ' +
`got ${twinHit.vectorHit}`,
)
} finally {
store.close()
}
})
// ── C4c: 没有中心时照常返回结果,只是不校正 ──────────────────────────────
//
// criteria 里写死了:C3 判 stale 的那一轮「仍返回结果(用未校正向量)」。
// 这条守的是「拿不到中心就让召回失败」这个错误,不是「校正对不对」。
test('C4c: without a centroid the recall still returns results, only uncorrected', async (t) => {
const { store, keyOfId } = seedStore({ withCentroid: false })
try {
assert.equal(store.loadCentroid(SPACE), undefined, 'precondition: no centroid exists in this fixture')
const hits = await recallByKey(store, keyOfId)
t.diagnostic(`C4c uncorrected ranking: ${fmt(hits)}`)
assert.ok(hits.length > 0, 'C4c: a missing centroid must not empty the result set')
assert.equal(
hits[0].key,
'hub',
'C4c: without a centroid the ranking stays uncorrected, so the hub still wins. ' +
'If this is "target", something is centering without a centroid to center on. ' +
`got ${fmt(hits)}`,
)
} finally {
store.close()
}
})
// ── C4d: 生产路径必须自己把中心建出来 ────────────────────────────────────
//
// ★ 这条是 C4 的第一层,也是最容易被漏掉的一半。
// C4a/b/c 都用 `seedStore()` 手工调 `rebuildCentroid`,所以它们只能证明
// 「去均值的算术是对的」,证明不了「生产上中心真的会出现」。
// 而生产上恰恰是后者不存在:`saveSpaceCentroid` / `centroidStatus` /
// `centroidRecallState` 在 lib/ 里都没有生产调用者,`rebuildCentroid`
// 只有测试在调 —— `graph_centroid` 曾经永远是空表。
//
// 所以这条判据走真 `apply()` + 真 `commit()`,不碰 store 内部:
// 写完之后 `loadCentroid` 必须已经能读到中心,且召回必须已经用上校正。
test('C4d: the production commit path builds the centroid it later reads', async () => {
const { Context } = await import('@deepseek-ai/cordis')
const { SystemPrompt } = await import('@deepseek-ai/dsh-system-prompt')
const { ToolRuntime } = await import('@deepseek-ai/dsh-tools')
const { apply: applyMemory } = await import('../lib/index.js')
const { mkdtempSync, rmSync } = await import('node:fs')
const { tmpdir } = await import('node:os')
const { join } = await import('node:path')
const dir = mkdtempSync(join(tmpdir(), 'c4d-'))
const ctx = new Context()
new SystemPrompt(ctx, {})
new ToolRuntime(ctx, {})
try {
const mem = applyMemory(ctx, { path: join(dir, 'm.db'), embedder: makeEmbedder() })
// 前置:还没写东西时不该有中心
assert.equal(
mem.store.loadCentroid(SPACE),
undefined,
'precondition: a fresh database has no centroid',
)
// 全部走生产入口,不手工调 rebuildCentroid
const keyOfId = new Map()
for (const block of CORPUS) {
const written = await mem.commit({ text: block.text })
keyOfId.set(written.id, block.key)
assert.equal(written.embedded, true, `precondition: ${block.key} must actually be embedded`)
}
// ★ The first commit builds a centroid from the blocks it has seen SO FAR,
// so right after a burst of writes the centroid still describes only the
// first one — and with MIN_SAMPLE it is not even allowed to claim a direction
// yet. The write path schedules a drift rebuild; wait for it before asserting
// on the centroid's CONTENTS rather than merely on its existence.
//
// Measured without this wait: vectorCount=1, meanLength=1.0000 on the same
// fixture, and a 1-block "mean" is that block itself.
await new Promise((resolve) => setTimeout(resolve, 3000))
const centroid = mem.store.loadCentroid(SPACE)
assert.ok(
centroid !== undefined,
'C4d: after commits through the production path a centroid must exist. ' +
'If this is undefined, nothing in lib/ ever builds one and centering can ' +
'never apply in real use — the arithmetic tests above would still pass ' +
'because they build the centroid by hand.',
)
assert.equal(centroid.isAnomalous, true, 'C4d: and it must be judged anisotropic for this fixture')
assert.equal(centroid.dim, DIM, 'C4d: its width must match the space')
// ★ The existence check above passes even when the centroid describes a
// single block — which is exactly the state right after the first commit.
// A 1-block mean is that block itself (norm 1.0, so it clears the 0.6
// threshold by construction), and centering by it drops that block as a zero
// vector. Pin the coverage, not just the presence.
assert.equal(
centroid.vectorCount,
CORPUS.length,
'C4d: the centroid must describe every block in the space, not only the ones '
+ 'present at the first commit. A smaller count means the drift rebuild '
+ 'never caught up and centering is subtracting the wrong mean.',
)
// 端到端:同一个库,召回必须已经用上校正
const hits = await recallBlocksHybrid(mem.store, makeEmbedder(), QUERY_TEXT, { topK: 8 })
assert.ok(hits.length > 0, 'C4d: the recall must return something')
const top = hits[0]
assert.equal(
keyOfId.get(top.id),
'target',
'C4d: the whole point — production write + production read must together ' +
'produce the corrected ranking. A "hub" winner here means the two halves ' +
'are not connected even though each one works on its own. ' +
`got ${hits.map((h) => keyOfId.get(h.id) ?? h.id).join(',')}`,
)
} finally {
await ctx.fiber.dispose()
rmSync(dir, { recursive: true, force: true })
}
})
// ── C4e: 块数漂移超 20% 后,写入侧必须让中心重新跟上 ──────────────────────
//
// ★ 这条守的是我在 0a6adff 里自己漏掉的那一半。
// 那时的 `#ensureCentroid` 是「没有中心就建一个,有就 return」——
// 于是中心永远停在第一次 commit 的快照上,块数翻多少倍它都不动。
// 这正是这一整轮反复抓的同一个病:**逻辑写了,但没有被调到**。
// `centroidStatus` 里那套 stale 判定(漂移 > 20% + 10 分钟节流 + 异步重建)
// 原本在 lib/ 里零生产调用者,不接上的话它只是文件里的一段死代码。
//
// C4a/b/c 都手工 `rebuildCentroid`,C4d 只验证「首次建」,三条都照不到这里:
// 中心存在且过期,与中心不存在,是两个不同的失效形态。
test('C4e: a drifted centroid is refreshed by the write path, not left stale forever', async (t) => {
// 异步重建靠 setTimeout,用假时钟把 2s 拨过去,免得测试真的睡两秒。
t.mock.timers.enable({ apis: ['setTimeout'] })
const { Context } = await import('@deepseek-ai/cordis')
const { SystemPrompt } = await import('@deepseek-ai/dsh-system-prompt')
const { ToolRuntime } = await import('@deepseek-ai/dsh-tools')
const { apply: applyMemory } = await import('../lib/index.js')
const { mkdtempSync, rmSync } = await import('node:fs')
const { tmpdir } = await import('node:os')
const { join } = await import('node:path')
const dir = mkdtempSync(join(tmpdir(), 'c4e-'))
const ctx = new Context()
new SystemPrompt(ctx, {})
new ToolRuntime(ctx, {})
try {
const mem = applyMemory(ctx, { path: join(dir, 'm.db'), embedder: makeEmbedder() })
// 第一条:写入侧顺手建出中心,此时它只见过 1 条块
await mem.commit({ text: CORPUS[0].text })
const born = mem.store.loadCentroid(SPACE)
assert.ok(born !== undefined, 'precondition: the first commit must build a centroid')
const bornCount = born.vectorCount
assert.ok(bornCount >= 1, `precondition: the centroid must have counted blocks, got ${bornCount}`)
// 继续写,直到块数相对建中心时漂移超过 20%
for (const block of CORPUS.slice(1)) await mem.commit({ text: block.text })
// ★ 这里刻意**不**调 centroidStatus。
// 它自己就会调度重建 —— 无论写入侧接没接,只要测试里调了它,
// 重建就会发生,这条判据就变成了自己证明自己。第一次写这条时
// 就是这么写的,M4 变异(退回 0a6adff 的「有中心就 return」)
// 居然 6/6 全绿,才把这个问题缴出来。
const drift = (CORPUS.length - bornCount) / bornCount
assert.ok(
drift > 0.2,
`precondition: this fixture must drift past the 20% ratio; got ${(drift * 100).toFixed(0)}%`,
)
// 假时钟把重建的 2s 拨过去。
// 只有写入侧真的调过 centroidStatus,这里才会有回调可跑。
t.mock.timers.tick(2000)
const healed = mem.store.loadCentroid(SPACE)
assert.ok(healed !== undefined, 'C4e: the centroid must still exist after the scheduled rebuild')
assert.ok(
healed.vectorCount > bornCount,
'C4e: a centroid that drifted past the 20% ratio must be rebuilt by the write path. ' +
`It is still describing ${healed.vectorCount} blocks while ${CORPUS.length} carry vectors — ` +
'that means nothing in lib/ ever consults the stale judgement, so the mean stays frozen ' +
'on whatever the first commit happened to see, and centering slowly stops describing ' +
'the space it is supposed to correct.',
)
assert.equal(
healed.vectorCount,
CORPUS.length,
'C4e: the rebuilt centroid must cover every vectored block in this space',
)
} finally {
await ctx.fiber.dispose()
rmSync(dir, { recursive: true, force: true })
}
})
// ── C4g: 各向同性空间必须完全不校正(isAnomalous 门控)────────────────────
//
// ★ 这条守的是「门控本身」,不是「校正算得对不对」。
// C4a/b/c/d/e 五条用的全是有各向异性的 fixture(C4-0 与 C4d 还专门断言
// isAnomalous === true),C4c 是「没有中心行」—— 三条都照不到「有中心、但
// 中心判定为各向同性」这个分支。实测把 lib/recall.js 里的
// `centroid?.isAnomalous === true &&` 删掉,上述五条加 phase-b-h、phase-b-a
// 全部全绿:门控是裸的。
//
// 数字来源:同一份探针分别打「门控在」与「门控被拆」两个 lib,实测对照,
// 不是纸上推的。各向同性空间(8 条正交 one-hot):
// meanLength = 0.353553(= 1/sqrt(8)),isAnomalous = false
// query = unit(ones) 门控在 → 8 条命中,每条 cosine = 0.3536
// 门控拆 → 0 条命中(去均值后各块余弦转负,被 V5 的
// 结构性门 `vectorScore <= 0 && symbolHit === 0` 全丢)
// 为什么用这条 query:它的方向就是均值方向,去均值后与所有块的夹角都变成钝角,
// 落差是二值化的(8 → 0),不依赖阈值也不依赖浮点容差。
test('C4g: an isotropic space is left uncorrected — the isAnomalous gate', async (t) => {
const { Context } = await import('@deepseek-ai/cordis')
const { SystemPrompt } = await import('@deepseek-ai/dsh-system-prompt')
const { ToolRuntime } = await import('@deepseek-ai/dsh-tools')
const { apply: applyMemory } = await import('../lib/index.js')
const { mkdtempSync, rmSync } = await import('node:fs')
const { tmpdir } = await import('node:os')
const { join } = await import('node:path')
const ISO_DIM = 8
const ISO_SPACE = 'c4g:isotropic:test'
const ones = new Array(ISO_DIM).fill(1)
const queryA = unit(ones) // ≈ 均值方向
const queryB = unit([0.5, 0.3, 0.2, 0, 0, 0, 0, 0]) // 三个方向轴的混合
const texts = Array.from({ length: ISO_DIM }, (_, i) => `正交块 ${i}`)
const vecOf = new Map(texts.map((text, i) => [text, axis(i)]))
vecOf.set('Q-A', queryA)
vecOf.set('Q-B', queryB)
const isoEmbedder = {
async embedText(text) { return vecOf.get(text) ?? new Array(ISO_DIM).fill(0) },
info: () => ({ fingerprint: ISO_SPACE, dimension: ISO_DIM }),
}
const dir = mkdtempSync(join(tmpdir(), 'c4g-'))
const ctx = new Context()
new SystemPrompt(ctx, {})
new ToolRuntime(ctx, {})
try {
const mem = applyMemory(ctx, { path: join(dir, 'm.db'), embedder: isoEmbedder })
const keyOf = new Map()
for (const text of texts) {
const written = await mem.commit({ text })
keyOf.set(written.id, text)
}
// 让写入侧的漂移重建跑完,断言的是「生产路径建出的中心」而不是半成品
await new Promise((resolve) => setTimeout(resolve, 3000))
const centroid = mem.store.loadCentroid(mem.space().fingerprint)
assert.ok(centroid !== undefined, 'C4g precondition: the production path must still build a centroid')
assert.equal(centroid.isAnomalous, false, 'C4g precondition: 8 orthogonal axes are isotropic')
assert.ok(
Math.abs((centroid.stats?.meanLength ?? 0) - 1 / Math.sqrt(ISO_DIM)) < 1e-6,
`C4g precondition: meanLength must be 1/sqrt(8); got ${centroid.stats?.meanLength}`,
)
assert.equal(centroid.vectorCount, ISO_DIM, 'C4g precondition: every block contributed')
// ① 方向接近均值的 query:门控在 ⇒ 全部命中,且余弦就是 1/sqrt(8)
const hitsA = await recallBlocksHybrid(mem.store, isoEmbedder, 'Q-A', { topK: ISO_DIM, minScore: 0 })
t.diagnostic(`C4g mean-like query: n=${hitsA.length} ${hitsA.map((h) => `${Number(h.vectorHit ?? 0).toFixed(4)}`).join(' ')}`)
assert.equal(
hitsA.length,
ISO_DIM,
'C4g: an isotropic space must NOT be centred. Removing the isAnomalous gate turns '
+ 'every cosine negative here and V5 drops the lot — the count collapses from 8 to 0.',
)
for (const hit of hitsA) {
assert.ok(
Math.abs(Number(hit.vectorHit) - 1 / Math.sqrt(ISO_DIM)) < 1e-4,
`C4g: each block keeps its raw cosine 1/sqrt(8); got ${hit.vectorHit}`,
)
}
// ② 三轴混合的 query:钉死「完全没做去均值」的具体数值
const hitsB = await recallBlocksHybrid(mem.store, isoEmbedder, 'Q-B', { topK: 3, minScore: 0 })
const byKey = new Map(hitsB.map((h) => [keyOf.get(h.id), Number(h.vectorHit)]))
t.diagnostic(`C4g three-axis query: ${[...byKey].map(([k, v]) => `${k}=${v.toFixed(4)}`).join(' ')}`)
assert.ok(Math.abs((byKey.get('正交块 0') ?? 0) - 0.8111) < 1e-4, 'C4g: block 0 keeps cosine 0.8111')
assert.ok(Math.abs((byKey.get('正交块 1') ?? 0) - 0.4867) < 1e-4, 'C4g: block 1 keeps cosine 0.4867')
assert.ok(Math.abs((byKey.get('正交块 2') ?? 0) - 0.3244) < 1e-4, 'C4g: block 2 keeps cosine 0.3244')
} finally {
await ctx.fiber.dispose()
rmSync(dir, { recursive: true, force: true })
}
})
// ── C3-k: 召回热路径不得出现计数查询 ────────────────────────────────────────
//
// `centroidStatus` 的第一行是 `countVectored`(一条 COUNT),而它每轮 pre-step
// 都会跑 —— C3-k 明令该查询只能出现在写入侧。实测往 lib/recall.js 里塞一行
// `store.countVectored(fingerprint)`,c4 6/6、phase-b-h、phase-b-a 全绿:
// 没有判据在守这条边界。所以这里用静态扫描把它钉住。
//
// ★ 扫描必须自证有效:同一段逻辑对 lib/index.js 跑一遍,必须能找到
// `centroidStatus(` —— 否则正则写错时,上面两条「不包含」会永远为真。
test('C3-k: the recall hot path must not run a COUNT query', async () => {
const { readFileSync } = await import('node:fs')
const strip = (relative) => readFileSync(new URL(relative, import.meta.url), 'utf8')
.replace(/\/\*[\s\S]*?\*\//g, ' ')
.split('\n')
.map((line) => line.replace(/\/\/.*$/, ''))
.join('\n')
const recall = strip('../lib/recall.js')
assert.ok(
!/\bcountVectored\s*\(/.test(recall),
'C3-k: lib/recall.js must not call countVectored — that COUNT belongs to the write path',
)
assert.ok(
!/\bcentroidStatus\s*\(/.test(recall),
'C3-k: lib/recall.js must not call centroidStatus — it runs countVectored internally, '
+ 'and the recall path may only read the centroid (loadCentroid)',
)
// 反向自检:同样的扫描必须能在写入侧看见那次调用,证明扫描不是空转
const index = strip('../lib/index.js')
assert.ok(
/\bcentroidStatus\s*\(/.test(index),
'C3-k self-check: the scan must be able to SEE a centroidStatus call — lib/index.js '
+ 'calls it on the write path. If this fails the two assertions above prove nothing.',
)
assert.ok(
/\bloadCentroid\s*\(/.test(recall),
'C3-k: the recall path reads the centroid through loadCentroid — and only that',
)
})
// ── C4h: 单块空间不得被判为各向异性(MIN_SAMPLE)──────────────────────────
//
// 一条向量的「均值」就是它自己,模长恒为 1.0,必然越过 0.6 阈值 —— 于是
// 「只有一条块的空间」会被判成各向异性,然后拿这条块自己去均值。减完是零
// 向量,被 recall.js 的 `scoringVector === null` 跳过,唯一那条块就此从
// 召回里消失。实测(真时钟、真 commit):第 8 次 commit 之前 centroidCount=1、
// meanLength=1.0000,同一库有中心时命中 1 条、无中心时命中 3 条。
//
// 判据落在两处:判定本身(isAnomalous 必须 false),以及后果(唯一那条块
// 必须仍然可召回)。
test('C4h: a single-block space is not anisotropic, and its only block stays recallable', async () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
const fp = 'c4h:single:test'
const only = axis(0)
const written = store.putBlock({ text: '唯一的一条块', vector: only, fingerprint: fp })
const built = store.rebuildCentroid(fp)
assert.equal(built.vectorCount, 1, 'C4h precondition: exactly one vector contributed')
assert.ok(Math.abs(built.meanLength - 1) < 1e-9, 'C4h precondition: a one-vector mean has norm 1')
assert.equal(
built.isAnomalous,
false,
'C4h: a one-vector mean IS that vector (norm 1.0), so the 0.6 test alone always says '
+ '"anisotropic". MIN_SAMPLE must refuse to call a single sample a direction — otherwise '
+ 'centering subtracts the block from itself and the zero vector is dropped from recall.',
)
const embedder = {
async embedText() { return only },
info: () => ({ fingerprint: fp, dimension: DIM }),
}
const hits = await recallBlocksHybrid(store, embedder, '与那条块方向相同的问题', { topK: 3, minScore: 0 })
assert.equal(
hits.length,
1,
'C4h: the only block in the space must still be recallable. A count of 0 means the '
+ 'single-block "centroid" was applied and its own block was projected onto the zero '
+ 'vector.',
)
assert.equal(hits[0].id, written.id, 'C4h: and it must be the block that was written')
} finally {
store.close()
}
})
// ── C4i: 浮点尘埃不得被当成方向 ─────────────────────────────────────────────
//
// 一个等于空间均值的块,与均值的差只有浮点残差(~1e-17)。如果把它归一化,
// 这堆垃圾会变成一个「单位向量」,再与 query 算出 0.2183 这样的分数 —— 看起来
// 像信号,实际是噪声。实测(2 条完全相同的块,query 与块方向不同):
//
// 原始余弦(无中心) 0.8608
// `norm > 0` 时代的输出 0.2183 ← 尘埃被重新归一化
// 现在的输出 无命中 ← 尘埃被判定为「没有方向」
//
// ★ 这条判据钉的是**不变式**,不是某个具体策略:
// 「任何返回的命中,其 vectorHit 必须是真实余弦(0.8608),绝不能是尘埃
// 推导出来的 0.22 量级」。因此它同时容得下两种未来做法 —— 丢弃这些块
// (当前),或对无方差的空间干脆不校正(若以后加上界保护)。
// 它唯一不允许的,就是继续拿尘埃打分。
test('C4i: floating-point dust is never scored as a direction', async (t) => {
const DUP_DIM = 8
const fp = 'c4i:duplicate:test'
const blockVec = unit([1, 0.2, 0.1, 0, 0, 0, 0, 0])
const queryVec = unit([1, 0.9, 0.1, 0, 0, 0, 0, 0])
const RAW_COSINE = 0.8608 // measured; see the header
const store = new BlockStore({ path: ':memory:' }).open()
try {
for (let i = 0; i < 2; i += 1) {
store.putBlock({ text: `重复的块 ${i}`, vector: blockVec, fingerprint: fp })
}
const built = store.rebuildCentroid(fp)
assert.ok(built.meanLength > 0.999, `C4i precondition: identical blocks have a unit mean; got ${built.meanLength}`)
assert.equal(built.isAnomalous, true, 'C4i precondition: the fixture must actually enable centring')
const embedder = {
async embedText() { return queryVec },
info: () => ({ fingerprint: fp, dimension: DUP_DIM }),
}
const hits = await recallBlocksHybrid(store, embedder, 'q', { topK: 10, minScore: 0 })
t.diagnostic(`C4i duplicate-space hits=${hits.length} ${hits.map((h) => Number(h.vectorHit).toFixed(4)).join(',')}`)
for (const hit of hits) {
assert.ok(
Math.abs(Number(hit.vectorHit) - RAW_COSINE) < 1e-3,
'C4i: every hit must carry the RAW cosine 0.8608. A value near 0.22 is '
+ 'renormalised floating-point dust: the block differs from the space mean '
+ 'only by ~1e-17, and turning that into a unit vector manufactures a '
+ 'direction that does not exist. Treat it as "no direction" instead.',
)
}
} finally {
store.close()
}
})
// ── C4j: stale 时退回未校正(父片令原文)──────────────────────────────────
//
// 片令写得很清楚:「判 stale 的那一轮仍返回结果(用未校正向量)」。
// 召回侧不能自己判 stale —— `centroidStatus` 第一行是 countVectored,C3-k 明令
// 它不得进热路径。所以判定在写入侧做,结果落在 centroid 行的 `stats.stale` 上,
// 召回侧只读这个标记。
//
// ★ 这条判据是在修掉一个真 bug 之后补的:最初把标记判断写成
// `centroid.stats?.stale !== true`,而标记存的是 JSON 数字 1 —— JS 里
// `1 !== true` 恒为真,于是整个 guard 变成 no-op。**当时全套 305 条判据全绿。**
// 换成 `!centroid.stats?.stale` 才生效。
// 没有这条判据,"写了但永不生效"会一直躺在那儿。
//
// fixture 用 C4 那个 hub/target 空间:那里的差别是可观察的 ——
// 校正时 target 胜(0.9000),不校正时 hub 胜(0.9928,即裸余弦)。
test('C4j: a centroid marked stale falls back to uncorrected vectors', async (t) => {
const DIM = 8
const unit = (v) => { const n = Math.hypot(...v); return n === 0 ? v : v.map((x) => x / n) }
const axis = (i) => unit(Array.from({ length: DIM }, (_, k) => (k === i ? 1 : 0)))
const BACKBONE = unit([1.0, 0.55, 0.42, 0.30, 0.22, 0.16, 0.11, 0.07])
const mk = (eps, ax) => unit(BACKBONE.map((x, i) => x + eps * ax[i]))
const HUB = { key: 'hub', text: '这段时间一切照旧。', vec: mk(0.02, axis(1)) }
const TARGET = { key: 'target', text: '第七号仓库的备用门禁码是 QP47-812。', vec: mk(0.30, axis(7)) }
const DISTRACTORS = [0, 2, 3, 4, 5, 6].map((i) => ({ key: `d${i}`, text: `干扰块 ${i}。`, vec: mk(0.30, axis(i)) }))
const CORPUS = [HUB, TARGET, ...DISTRACTORS]
const QUERY_TEXT = '上次说的那个四位数是什么来着?'
const QUERY_VEC = unit(BACKBONE.map((x, i) => x + 0.30 * 0.40 * axis(7)[i]))
const byText = new Map(CORPUS.map((b) => [b.text, b.vec]))
const store = new BlockStore({ path: ':memory:' }).open()
try {
const fp = 'c4j:stale-fallback'
const keyOf = new Map()
for (const b of CORPUS) keyOf.set(store.putBlock({ text: b.text, vector: b.vec, fingerprint: fp }).id, b.key)
const built = store.rebuildCentroid(fp)
assert.equal(built.isAnomalous, true, 'C4j precondition: the fixture must enable centring')
const embedder = {
async embedText(text) { return text === QUERY_TEXT ? QUERY_VEC : (byText.get(text) ?? new Array(DIM).fill(0)) },
info: () => ({ fingerprint: fp, dimension: DIM }),
}
const top1 = async () => {
const hits = await recallBlocksHybrid(store, embedder, QUERY_TEXT, { topK: 3, minScore: 0 })
t.diagnostic(`C4j ranking: ${hits.map((h) => `${keyOf.get(h.id)}=${Number(h.vectorHit ?? 0).toFixed(4)}`).join(' ')}`)
return { key: keyOf.get(hits[0]?.id), score: Number(hits[0]?.vectorHit ?? 0) }
}
const fresh = await top1()
assert.equal(fresh.key, 'target', 'C4j: with a usable centroid the target wins')
// The write path's verdict, persisted on the row.
assert.equal(store.markCentroidStale(fp), true, 'C4j: the flag must actually be written')
const stale = await top1()
assert.equal(
stale.key,
'hub',
'C4j: a round that judges the centroid stale must fall back to UNCORRECTED vectors '
+ '(the raw cosine then puts the hub first). If this still says "target", the flag '
+ 'is being ignored — the exact bug that a strict `!== true` comparison produced '
+ 'while the whole suite stayed green.',
)
assert.ok(Math.abs(stale.score - 0.9928) < 1e-3, `C4j: and it must be the raw cosine; got ${stale.score}`)
store.rebuildCentroid(fp)
const healed = await top1()
assert.equal(healed.key, 'target', 'C4j: a rebuild clears the flag and centring returns')
} finally {
store.close()
}
})
// ── C4k: 落标记这件事必须是写入侧自己干的,不是测试替它干 ───────────────────
//
// C4j 里有一行 `store.markCentroidStale(fp)` —— 那是判据**手工**打的标记,它只
// 证明「召回侧会尊重标记」,不证明「生产路径会打这个标记」。
//
// 实测:把 `lib/index.js` 里那行 `if (status?.stale === true) this.store
// .markCentroidStale(...)` 整个删掉,**全套 306 条判据全绿**。也就是说这条接线
// 此前没有任何判据守着 —— 与 W8(saveSpaceCentroid 零生产调用点)、C4d(中心
// 靠测试手工 rebuildCentroid)是同一个病:手工喂进去的输入,证明不了生产路径。
//
// 这条判据只用生产入口:`apply()` + `commit()`,自己不打任何标记。
test('C4k: the write path itself marks a drifted centroid stale', async () => {
const { Context } = await import('@deepseek-ai/cordis')
const { SystemPrompt } = await import('@deepseek-ai/dsh-system-prompt')
const { ToolRuntime } = await import('@deepseek-ai/dsh-tools')
const { apply: applyMemory } = await import('../lib/index.js')
const { mkdtempSync, rmSync } = await import('node:fs')
const { tmpdir } = await import('node:os')
const { join } = await import('node:path')
const DIM = 4
const dir = mkdtempSync(join(tmpdir(), 'c4k-'))
const ctx = new Context()
new SystemPrompt(ctx, {})
new ToolRuntime(ctx, {})
try {
const mem = applyMemory(ctx, {
path: join(dir, 'm.db'),
embedder: {
async embedText() { return unit([1, 0.1, 0, 0]) },
info: () => ({ fingerprint: 'c4k:write-marks', dimension: DIM }),
},
})
const fp = mem.space().fingerprint
// Build a centroid, then let the scheduled rebuild land so the count is settled.
for (let i = 0; i < 8; i += 1) await mem.commit({ text: `块 ${i}` })
await new Promise((resolve) => setTimeout(resolve, 3000))
const settled = mem.store.loadCentroid(fp)
assert.ok(settled !== undefined, 'C4k precondition: a centroid must exist')
assert.equal(
settled.stats?.stale,
undefined,
'C4k precondition: a freshly rebuilt centroid carries no stale flag',
)
// Drift it past the 20% ratio through the production write path only.
for (let i = 8; i < 11; i += 1) await mem.commit({ text: `块 ${i}` })
const drifted = mem.store.loadCentroid(fp)
assert.equal(
drifted.stats?.stale,
1,
'C4k: the WRITE path must persist the stale verdict — the recall side is not '
+ 'allowed to judge it (C3-k), so if this flag is missing, recall keeps '
+ 'centring with a mean that no longer describes the space. Deleting that one '
+ 'line in #ensureCentroid left all 306 criteria green, which is why this '
+ 'criterion exists.',
)
} finally {
await ctx.fiber.dispose()
rmSync(dir, { recursive: true, force: true })
}
})

View File

@ -0,0 +1,390 @@
/**
* C5 · 「为什么不校正」必须可区分。
*
* ## 这条判据要解决的病
*
* C3 的 `centroidStatus` 早就把四种失效原因都算出来了,但对外只给
* `available: false` 一个结论。于是:
*
* · `no_centroid` 与 `dim_mismatch` 都是 `stale: null`
* · `isotropic` 与健康空间都 `stale: false`,只差 available
*
* 对「能不能校正」这个问题够用;对「为什么语义召回一直不校正」完全不够。
* 运维看到的是一句 `available: false`,四种完全不同的处置方式(建中心 /
* 换维度 / 等重建 / 这个空间本来就没有公共方向)被压成同一个症状。
*
* ## 判据形态
*
* 每一种原因一个 fixture,直接断 `reason` 这个值本身 —— 不断 available、
* 不断 stale,因为它们本来就是多义的。同时断「available 与 reason 是否自洽」,
* 防止有人把 reason 写成与结论脱节的装饰。
*
* ★ 所有调用都注入 `schedule: () => {}`:`centroidStatus` 判 stale 时会调度
* 重建,那是副作用。C4e 第一版就因为没隔离它而自证过 —— 判据必须只看事实,
* 不触发被测行为。
*
* @module dsh-memory/test/phase-b-c5-reasons
*/
import assert from 'node:assert/strict'
import test from 'node:test'
import { BlockStore } from '../lib/store.js'
const DIM = 4
const unit = (v) => { const n = Math.hypot(...v); return n === 0 ? v : v.map((x) => x / n) }
const noSchedule = () => {}
/** 每条判据都用它:一次调用,注入空调度,避免副作用串味。 */
const statusOf = (store, fp, queryDim = DIM) =>
store.centroidStatus(fp, { queryDim, schedule: noSchedule })
test('C5: a space with blocks but no centroid row reports no_centroid', () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
const fp = 'c5:no-centroid'
for (let i = 0; i < 3; i += 1) store.putBlock({ text: `a${i}`, vector: unit([1, i * 0.01, 0, 0]), fingerprint: fp })
const s = statusOf(store, fp)
assert.equal(s.reason, 'no_centroid', 'a missing row is a missing row, not a dimension problem')
assert.equal(s.available, false)
assert.equal(s.dim, null, 'and it must not invent a dimension')
assert.equal(s.vectorCount, null)
} finally {
store.close()
}
})
test('C5: a centroid of the wrong width reports dim_mismatch', () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
const fp = 'c5:dim-mismatch'
for (let i = 0; i < 3; i += 1) store.putBlock({ text: `b${i}`, vector: unit([1, 0.01 * i, 0, 0]), fingerprint: fp })
store.rebuildCentroid(fp)
const s = statusOf(store, fp, 8)
assert.equal(s.reason, 'dim_mismatch', 'a centroid exists and is readable — it is simply the wrong width')
assert.equal(s.available, false)
assert.equal(s.dim, DIM, 'the stored width must still be reported, so the operator can see the mismatch')
assert.notEqual(s.dim, 8)
} finally {
store.close()
}
})
test('C5: a centroid row with zero samples reports zero_sample', () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
const fp = 'c5:zero-sample'
// A row written by an older build / a migration: present, but built from nothing.
store.saveCentroid({
fingerprint: fp, dim: DIM, vectorCount: 0,
mean: new Array(DIM).fill(0), isAnomalous: false,
stats: { source: 'migration' },
})
const s = statusOf(store, fp)
assert.equal(s.reason, 'zero_sample', 'a centroid built from zero vectors is its own cause')
assert.equal(s.available, false)
assert.equal(s.vectorCount, 0)
} finally {
store.close()
}
})
test('C5: a centroid that drifted past the ratio reports stale', () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
const fp = 'c5:stale'
for (let i = 0; i < 8; i += 1) store.putBlock({ text: `c${i}`, vector: unit([1, 0.001 * i, 0, 0]), fingerprint: fp })
store.rebuildCentroid(fp)
for (let i = 8; i < 12; i += 1) store.putBlock({ text: `c${i}`, vector: unit([1, 0.001 * i, 0, 0]), fingerprint: fp })
const s = statusOf(store, fp)
assert.equal(s.reason, 'stale', 'the centroid is usable in principle, it just no longer describes the space')
assert.equal(s.available, false)
assert.equal(s.stale, true)
assert.ok(s.delta > 0.2, `precondition: the drift must exceed the ratio; got ${s.delta}`)
} finally {
store.close()
}
})
test('C5: an isotropic space reports isotropic, not stale', () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
const fp = 'c5:isotropic'
for (const [i, v] of [[1, 0, 0, 0], [0, 1, 0, 0], [-1, 0, 0, 0], [0, -1, 0, 0]].entries()) {
store.putBlock({ text: `d${i}`, vector: v, fingerprint: fp })
}
store.rebuildCentroid(fp)
const s = statusOf(store, fp)
assert.equal(s.reason, 'isotropic', 'four orthogonal axes have no common direction to remove — '
+ 'this is not a staleness problem and must not be reported as one')
assert.equal(s.available, false)
assert.equal(s.stale, false, 'isotropic is a permanent property of the space, not a transient state')
} finally {
store.close()
}
})
test('C5: a healthy space reports ok, and only ok carries available=true', () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
const fp = 'c5:healthy'
for (let i = 0; i < 8; i += 1) store.putBlock({ text: `e${i}`, vector: unit([1, 0.001 * i, 0, 0]), fingerprint: fp })
store.rebuildCentroid(fp)
const s = statusOf(store, fp)
assert.equal(s.reason, 'ok')
assert.equal(s.available, true)
assert.equal(s.stale, false)
} finally {
store.close()
}
})
test('C5: reason and available never disagree', () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
// Build every branch at once and check the invariant holds in all of them.
const cases = []
const mk = (fp, n, opts = {}) => {
for (let i = 0; i < n; i += 1) {
store.putBlock({ text: `${fp}-${i}`, vector: unit([1, 0.001 * i, 0, 0]), fingerprint: fp })
}
if (opts.build !== false) store.rebuildCentroid(fp)
}
mk('c5:i1', 3, { build: false }) // no_centroid
mk('c5:i2', 3) // ok / dim_mismatch via queryDim
cases.push([statusOf(store, 'c5:i1'), 'no_centroid'])
cases.push([statusOf(store, 'c5:i2'), 'ok'])
cases.push([statusOf(store, 'c5:i2', 8), 'dim_mismatch'])
for (const [s, expected] of cases) {
assert.equal(s.reason, expected)
assert.equal(
s.available,
s.reason === 'ok',
`C5: available must be true for exactly one reason (ok); got reason=${s.reason} available=${s.available}`,
)
}
} finally {
store.close()
}
})
test('C5: centroidRecallState keeps its stale:null projection but carries the reason', () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
const fp = 'c5:recall-projection'
for (let i = 0; i < 8; i += 1) store.putBlock({ text: `f${i}`, vector: unit([1, 0.001 * i, 0, 0]), fingerprint: fp })
store.rebuildCentroid(fp)
for (let i = 8; i < 12; i += 1) store.putBlock({ text: `f${i}`, vector: unit([1, 0.001 * i, 0, 0]), fingerprint: fp })
const projection = store.centroidRecallState(fp, { queryDim: DIM, schedule: noSchedule })
assert.equal(projection.stale, null, 'the recall-facing projection keeps hiding staleness from the scorer')
assert.equal(projection.reason, 'stale', 'but the reason still travels, so an operator can see why')
assert.equal(projection.available, false)
} finally {
store.close()
}
})
// ── C5 的出口:reason 必须能被运维读到,而且读的时候不许有副作用 ─────────────
//
// 上面八条都在 store 层证明「六种原因可区分」。但区分出来没有出口,等于只做完
// 一半:`centroid` 这一项给的是 `loadCentroid` 的原始行,它说得出「中心是什么」,
// 说不出「为什么没在校正」。全仓曾经没有任何地方读 `centroidStatus` 的返回值。
//
// 现在它从 `stats().centroid_state` 出去。两条要求:
// 1. reason 与 store 层判定一致;
// 2. **纯读** —— `centroidStatus` 判 stale 时会登记一次重建,而 stats 是只读
// 接口,不许因此产生写副作用。用空调度器把它按住,`rebuildScheduled`
// 在这里恒为 false,这正是纯读该有的样子。
test('C5: stats() exposes the reason without scheduling a rebuild', async () => {
const { Context } = await import('@deepseek-ai/cordis')
const { SystemPrompt } = await import('@deepseek-ai/dsh-system-prompt')
const { ToolRuntime } = await import('@deepseek-ai/dsh-tools')
const { apply: applyMemory } = await import('../lib/index.js')
const { mkdtempSync, rmSync } = await import('node:fs')
const { tmpdir } = await import('node:os')
const { join } = await import('node:path')
const unit = (v) => { const n = Math.hypot(...v); return n === 0 ? v : v.map((x) => x / n) }
const dir = mkdtempSync(join(tmpdir(), 'c5-stats-'))
const ctx = new Context()
new SystemPrompt(ctx, {})
new ToolRuntime(ctx, {})
try {
// ── fresh space: no centroid yet ──────────────────────────────────────
const mem = applyMemory(ctx, { path: join(dir, 'm.db'), embedder: {
async embedText() { return unit([1, 0.1, 0, 0]) },
info: () => ({ fingerprint: 'c5:stats:test', dimension: 4 }),
} })
assert.equal(
mem.stats().centroid_state.reason,
'no_centroid',
'C5: stats() must say WHY centring is off, not merely that no row exists',
)
// ── after writes: healthy ─────────────────────────────────────────────
for (let i = 0; i < 8; i += 1) await mem.commit({ text: `块 ${i}` })
await new Promise((resolve) => setTimeout(resolve, 3000)) // let the drift rebuild land
let s = mem.stats()
assert.equal(s.centroid_state.reason, 'ok', 'a built, current centroid reports ok')
assert.equal(s.centroid_state.available, true)
// Both surfaces coexist: `centroid` is the raw row, `centroid_state` is the
// verdict. Asserting they are the SAME object would be a tautology; the
// meaningful claim is that the raw row still carries what it always did
// (dim / mean) while the verdict carries the reason.
assert.equal(typeof s.centroid?.dim, 'number', 'the raw centroid row is still reported')
assert.ok(Array.isArray(s.centroid_state === null ? null : s.centroid?.mean), 'and its mean is intact')
assert.equal(typeof s.centroid_state.reason, 'string', 'the verdict names a reason')
// ── drift past 20%: stale, and stats() must NOT register a rebuild ────
for (let i = 8; i < 16; i += 1) await mem.commit({ text: `块 ${i}` })
s = mem.stats()
assert.equal(s.centroid_state.reason, 'stale', 'the drift is what the operator needs to see')
assert.equal(s.centroid_state.stale, true)
assert.equal(
s.centroid_state.rebuildScheduled,
false,
'C5: stats() is a READ. If it schedules a rebuild it has become a write path, '
+ 'and every operator polling stats() would keep the rebuild loop alive.',
)
assert.ok(s.centroid_state.delta > 0.2, `precondition: the drift must exceed the ratio; got ${s.centroid_state.delta}`)
} finally {
await ctx.fiber.dispose()
rmSync(dir, { recursive: true, force: true })
}
})
// ── C5 的最后一段:reason 必须出现在**模型真正调用的那个工具**的返回里 ────────
//
// 前九条分别证明:六种原因可区分(store 层)、stats() 带 reason 且纯读
// (plugin 层)。但 dsh 里的模型不会去读 `mem.stats()` —— 它调工具。
//
// 这个缺口在本轮出现过三次同款:P1 的 queryVector 算完丢掉、D2 的计数器算完
// 不打印、C5 的 reason 算完没人读。每一次都是「值存在、判据绿、运维看不见」。
// 工具面是第四个可能断的环节,而且是最靠外的一个:handler 改成挑字段返回,
// 前面所有判据都会继续绿。
//
// 所以这条直接走 `ctx.tools.execute` —— 与 phase-b-h 里调工具的方式一致,
// signal 必须是真实 AbortSignal(ToolRuntime 内部会读 callerSignal.aborted)。
test('C5: the stats TOOL returns the reason, not just the service method', async () => {
const { Context } = await import('@deepseek-ai/cordis')
const { SystemPrompt } = await import('@deepseek-ai/dsh-system-prompt')
const { ToolRuntime } = await import('@deepseek-ai/dsh-tools')
const { apply: applyMemory } = await import('../lib/index.js')
const { mkdtempSync, rmSync } = await import('node:fs')
const { tmpdir } = await import('node:os')
const { join } = await import('node:path')
const callStats = async (ctx) => {
const result = await ctx.tools.execute({
callId: 'c5-stats',
name: 'stablemem_stats',
arguments: {},
signal: new AbortController().signal,
})
const text = result?.content?.[0]?.text ?? result?.value ?? result
return typeof text === 'string' ? JSON.parse(text) : text
}
const dir = mkdtempSync(join(tmpdir(), 'c5-tool-'))
const ctx = new Context()
new SystemPrompt(ctx, {})
new ToolRuntime(ctx)
const mem = applyMemory(ctx, {
path: join(dir, 'm.db'),
embedder: {
async embedText() { return unit([1, 0.1, 0, 0]) },
info: () => ({ fingerprint: 'c5:tool:test', dimension: 4 }),
},
})
try {
const empty = await callStats(ctx)
assert.ok(
'centroid_state' in empty,
'C5: the tool must expose centroid_state — that is the surface the model and '
+ 'the operator actually call. Every other criterion can stay green while a '
+ 'handler that cherry-picks fields drops it.',
)
assert.equal(empty.centroid_state.reason, 'no_centroid')
for (let i = 0; i < 8; i += 1) await mem.commit({ text: `块 ${i}` })
await new Promise((resolve) => setTimeout(resolve, 3000))
const healthy = await callStats(ctx)
assert.equal(healthy.centroid_state.reason, 'ok', 'and the healthy verdict travels too')
assert.equal(healthy.centroid_state.available, true)
assert.ok('last_recall' in healthy, 'precondition: sibling fields still present')
for (let i = 8; i < 16; i += 1) await mem.commit({ text: `块 ${i}` })
const drifted = await callStats(ctx)
assert.equal(drifted.centroid_state.reason, 'stale')
assert.equal(
drifted.centroid_state.rebuildScheduled,
false,
'C5: calling the tool must not schedule a rebuild — a read-only surface that '
+ 'mutates would let a polling operator drive the rebuild loop.',
)
} finally {
await ctx.fiber.dispose()
rmSync(dir, { recursive: true, force: true })
}
})
// ── C5: 样本不足必须与「本来就没方向」分开 ──────────────────────────────────
//
// MIN_SAMPLE 让「一条块」的空间不再判为各向异性(一条块不构成方向),于是它
// 落进了 `isotropic` 分支 —— 而这两件事的修法是相反的:
//
// isotropic 这个空间永远不会有公共方向,别指望校正
// too_few_samples 再写几条块就好了
//
// 报错哪一种不是措辞问题:C5 存在的全部理由,就是让运维知道该往哪儿走。
//
// ★ 这个洞是我自己捅的:MIN_SAMPLE 是我加的,`too_few_samples` 是补上之后才
// 成立的,所以这条判据同时钉住两件事 —— 1 条块的空间报 too_few_samples,
// 以及 2 条块的空间(够 MIN_SAMPLE)不会误报成样本不足。
test('C5: a one-vector space reports too_few_samples, not isotropic', () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
// ── one vector: undersampled ─────────────────────────────────────────
const fp1 = 'c5:one-vector'
store.putBlock({ text: '唯一一条', vector: unit([1, 0, 0, 0]), fingerprint: fp1 })
const built = store.rebuildCentroid(fp1)
assert.equal(built.vectorCount, 1)
assert.equal(
built.isAnomalous,
false,
'precondition: MIN_SAMPLE keeps one sample from being called a direction',
)
const s1 = statusOf(store, fp1)
assert.equal(
s1.reason,
'too_few_samples',
'C5: "not enough blocks yet" and "this space has no common direction" lead to '
+ 'opposite actions. Reporting a one-vector space as isotropic tells the operator '
+ 'to give up instead of to write more.',
)
assert.equal(s1.available, false)
// ── two vectors: enough to be judged, and NOT reported as undersampled ─
const fp2 = 'c5:two-vectors'
store.putBlock({ text: 'a', vector: unit([1, 0, 0, 0]), fingerprint: fp2 })
store.putBlock({ text: 'b', vector: unit([1, 0.001, 0, 0]), fingerprint: fp2 })
store.rebuildCentroid(fp2)
const s2 = statusOf(store, fp2)
assert.notEqual(
s2.reason,
'too_few_samples',
'precondition: MIN_SAMPLE is 2, so two samples must clear the undersampled branch',
)
} finally {
store.close()
}
})

View File

@ -39,7 +39,12 @@ const rel = (file) => file.replace(LIB, 'lib/')
test('G2b: lib/ imports only node: builtins and cordis/dsh peers', () => {
// A new runtime dependency (an ONNX binding, an HTTP client, a vector DB) shows
// up here before it shows up in a deployment.
const allowed = /^(node:|@deepseek-ai\/(cordis|dsh-llm|dsh-system-prompt|dsh-tools|dsh-session|dsh-scope)|\.\/|\.\.\/)/
// `dsh-agent-loop` joined this list with lib/loop.js: the loop subclass has to
// extend the harness's own AgentLoop, and that package is declared as a peer in
// package.json exactly like dsh-tools. It is still a first-party harness
// package, not the kind of new runtime dependency (a binding, an HTTP client,
// a vector DB) this guard exists to catch.
const allowed = /^(node:|@deepseek-ai\/(cordis|dsh-llm|dsh-system-prompt|dsh-tools|dsh-session|dsh-scope|dsh-agent-loop)|\.\/|\.\.\/)/
const offenders = []
for (const file of libSources()) {
const text = readFileSync(file, 'utf8')

View File

@ -744,6 +744,20 @@ test('ticket②-strong mutation: re-deriving in the centroid path must break the
})
await mem.commit({ text: 'a' })
// ★ C4 让 commit 为本空间自动建中心(写入侧 #ensureCentroid),
// 而这个用例的前提是「本空间的 key 上没有中心」。两者不矛盾:
// 它验证的是「用另一个拼写推导出的 key 查不到中心」,所以先把本空间
// 那一行显式清掉,让 miss 重新成为一个可观测的事实。
// 没有这一步,下面的 assert 会因为「本来就有一个中心」而失败,
// 红的原因跟它要守的约束无关。
mem.store.raw().prepare('DELETE FROM graph_centroid WHERE fingerprint = ?')
.run(mem.space().fingerprint)
assert.equal(
mem.store.loadCentroid(mem.space().fingerprint),
undefined,
'precondition: the block-side key must carry no centroid before the mutation',
)
// Mutation: the centroid path derives its own key from a *different*
// spelling that normalisation does NOT collapse — a different query
// string is a genuinely different endpoint (this slice deliberately does
@ -1518,8 +1532,8 @@ test('W11d†ctx: restoring the contextForStep short-circuit flips the behaviour
'index.js',
[
[
' const hits = await this.recall(query, { signal })',
" const hits = await this.recall(query, { signal })\n if (hits.length === 0) return '' // MUTATED: the old caller short-circuit",
' const raw = await this.recall(query, { signal })',
" const raw = await this.recall(query, { signal })\n if (raw.length === 0) return '' // MUTATED: the old caller short-circuit",
],
],
async (mod) => {
@ -1720,9 +1734,15 @@ async function withMutatedCopy(relPath, replacements, body) {
const target = join(dir, 'lib', relPath)
let text = readFileSync(target, 'utf8')
for (const [from, to] of replacements) {
assert.ok(
text.includes(from),
`mutation anchor not found in lib/${relPath} — did the source move?\n${from.slice(0, 100)}`,
// ★ Count BEFORE replacing: `String.replace` substitutes only the first
// match, so a duplicated anchor would silently mutate one site while the
// caller believes all of them changed. V5's `v5Mutant` established this
// check; the shared helper now carries it too.
assert.equal(
text.split(from).length - 1,
1,
`mutation anchor must occur exactly once in lib/${relPath} (found ` +
`${text.split(from).length - 1}): ${from.slice(0, 100)}`,
)
text = text.replace(from, to)
}
@ -2907,13 +2927,15 @@ test('D3b: a clean recall overwrites last_recall with its own outcome', async ()
const result = await ctx.tools.execute({
callId: 'lr2', name: 'stablemem_stats', arguments: {}, signal: new AbortController().signal,
})
// ★ A healthy recall does NOT report 'ok': group A deliberately labels the
// success path `vector_ready_unused` because the vector is computed but not
// yet scored. Asserting 'ok' here was my mistake — the label that matters is
// "no failure", and `embed_error === null` is the honest way to say it.
// ★ A healthy WIRED recall reports `hybrid` (P1), not `ok` and no longer
// `vector_ready_unused`: the block carries a vector in the active space, so
// the fused route really contributed. The assertion that matters is "no
// failure", and `embed_error === null` is the honest way to say it; the mode
// value is asserted too so the label cannot silently drift back to the
// group-A stage name while the wiring is in place.
assert.notEqual(result.value.last_recall.mode, 'embed_error', 'no failure is recorded')
assert.equal(result.value.last_recall.embed_error, null, 'and no error reason either')
assert.equal(result.value.last_recall.mode, 'vector_ready_unused', 'the group-A success label')
assert.equal(result.value.last_recall.mode, 'hybrid', 'the wired success label')
} finally {
await done()
}
@ -3527,3 +3549,531 @@ for (const [name, replacements, withCentroids, criterion] of [
})
})
}
// ── C3: centroid staleness, async rebuild scheduling, status facts ────────
//
// Ported from Go `internal/memory/centroid_store.go`:
// :55 centroidRebuildRatio = 0.20
// :178 curVectored = countVectoredBlocks()
// :186-189 delta = (curVectored - vecCount) / vecCount, sign discarded
// :190 delta > ratio (STRICTLY greater)
// :165-167 dim <= 0 => error (corrupted metadata)
// :179-185 recall path: vecCount <= 0 => unusable, NOT stale
// :425-433 status path: vecCount == 0 && currentBlocks > 0 => stale
// :327-347 schedule once per 10 min, run 2 s later, never inline
//
// ★ C3 gives FACTS only. Which reason disabled centering (none / dim mismatch /
// stale / isotropic) is C5 and is NOT covered here.
/** A store with `n` vectored blocks in space 'fp-c3'. */
function c3Store(n) {
const store = new BlockStore({ path: ':memory:' }).open()
for (let i = 0; i < n; i += 1) {
store.putBlock({ text: `C3 block ${i}`, vector: [1, 0, 0], fingerprint: 'fp-c3' })
}
return store
}
/** Save a centroid row claiming `vectorCount` samples. */
function c3Centroid(store, vectorCount, { dim = 3, anomalous = true } = {}) {
store.saveCentroid({
fingerprint: 'fp-c3',
dim,
vectorCount,
mean: Array(dim).fill(0),
isAnomalous: anomalous,
})
}
/** A manually-advanced clock plus a scheduler that records instead of running. */
function c3Clock(start = 0) {
const state = { t: start, scheduled: [], ran: [] }
return {
state,
now: () => state.t,
schedule: (fn, delayMs) => { state.scheduled.push({ fn, delayMs, at: state.t }) },
advance(ms) { state.t += ms },
/** Run everything registered so far, as the timer would. */
fireAll() { for (const { fn } of state.scheduled.splice(0)) { fn(); state.ran.push(true) } },
}
}
test('C3-a/b: the 10-minute throttle registers once inside the window and again outside', () => {
const store = c3Store(13) // 13 vs 10 samples => delta 0.30 => stale
try {
c3Centroid(store, 10)
const clock = c3Clock()
const first = store.centroidStatus('fp-c3', { now: clock.now, schedule: clock.schedule })
assert.equal(first.rebuildScheduled, true, 'first stale observation must register a rebuild')
clock.advance(9 * 60 * 1000 + 59 * 1000) // 9m59s: inside the 10-minute window
const second = store.centroidStatus('fp-c3', { now: clock.now, schedule: clock.schedule })
assert.equal(second.rebuildScheduled, false, 'inside the window nothing is registered again')
assert.equal(clock.state.scheduled.length, 1, 'exactly one rebuild was scheduled so far')
clock.advance(10 * 60 * 1000 - (9 * 60 + 59) * 1000 + 1000) // past 10m01s
const third = store.centroidStatus('fp-c3', { now: clock.now, schedule: clock.schedule })
assert.equal(third.rebuildScheduled, true, 'outside the window a new rebuild is registered')
assert.equal(clock.state.scheduled.length, 2, 'two registrations total')
} finally {
store.close()
}
})
test('C3-c: the scheduled rebuild carries the Go 2-second delay and does not run inline', () => {
const store = c3Store(13)
try {
c3Centroid(store, 10)
const clock = c3Clock()
const status = store.centroidStatus('fp-c3', { now: clock.now, schedule: clock.schedule })
assert.equal(status.rebuildScheduled, true)
assert.equal(clock.state.scheduled.length, 1)
const [entry] = clock.state.scheduled
assert.equal(entry.delayMs, BlockStore.CENTROID_REBUILD_DELAY_MS, 'delay is the Go 2 s constant')
assert.equal(entry.delayMs, 2000, 'and that constant is exactly 2000 ms')
// Nothing ran yet: registration is not execution.
assert.equal(clock.state.ran.length, 0, 'the rebuild must not run during the status call')
} finally {
store.close()
}
})
test('C3-d: the registration reaches a path that really runs, and clears staleness', () => {
const store = c3Store(13)
try {
c3Centroid(store, 10)
const clock = c3Clock()
let rebuildCalls = 0
const realRebuild = store.rebuildCentroid.bind(store)
store.rebuildCentroid = (fp) => { rebuildCalls += 1; return realRebuild(fp) }
const first = store.centroidStatus('fp-c3', { now: clock.now, schedule: clock.schedule })
assert.equal(first.stale, true, 'precondition: the space starts stale (13 vs 10 samples)')
clock.fireAll() // the timer fires
assert.equal(rebuildCalls, 1, 'the registered callback rebuilt exactly once (not merely recorded)')
const after = store.centroidStatus('fp-c3', {
now: () => clock.state.t + 11 * 60 * 1000, schedule: clock.schedule,
})
assert.equal(after.stale, false,
'after the rebuild the space is no longer stale — this is what the 1391→1877 incident broke')
} finally {
store.close()
}
})
// ★ Shared predicates: the positive cases and every C3 mutant run the SAME
// code, so a mutant can only pass by satisfying the same requirement (the V1†
// lesson — a requirement re-written inside the mutant tests the mutant).
/** Recall-path criterion (go :179-185): zero samples ⇒ unusable, NO stale bit. */
function assertC3RecallUnusable(MutantStore) {
const store = new MutantStore({ path: ':memory:' }).open()
try {
c3Seed(store, 5, 0)
const status = store.centroidRecallState('fp-c3')
assert.equal(status.available, false, 'built from 0 samples: no centering')
assert.equal(status.stale, null, 'the recall path carries no stale bit (go :179-185)')
assert.equal(status.vectorCount, 0, 'the sample count is reported as-is')
} finally {
store.close()
}
}
/** Status-path criterion (go :431-432): zero samples + blocks now ⇒ stale. */
function assertC3StatusStale(MutantStore) {
const store = new MutantStore({ path: ':memory:' }).open()
try {
c3Seed(store, 5, 0)
const status = store.centroidStatus('fp-c3', { schedule: () => {}, now: () => 0 })
assert.equal(status.stale, true, 'vecCount == 0 && currentBlocks > 0 => stale (go :431-432)')
assert.equal(status.currentBlocks, 5)
assert.equal(status.delta, null, 'delta is undefined when the denominator is 0')
} finally {
store.close()
}
}
/** Corrupt-metadata criterion (go :165-167): dim <= 0 must throw, never be silent. */
function assertC3CorruptThrows(MutantStore) {
const store = new MutantStore({ path: ':memory:' }).open()
try {
store.putBlock({ text: 'C3 corrupt', vector: [1, 0, 0], fingerprint: 'fp-c3' })
store.raw().prepare(
'INSERT INTO graph_centroid (fingerprint, dim, vector_count, vector, stats, is_anomalous, built_at) VALUES (?,?,?,?,?,?,?)',
).run('fp-c3', 0, 5, '[]', '{}', 1, Date.now())
assert.throws(() => store.centroidStatus('fp-c3', { schedule: () => {}, now: () => 0 }),
/centroid dim is 0 \(corrupted\?\)/,
'a stored dim <= 0 is corrupt and must surface (go :165-167)')
} finally {
store.close()
}
}
test('C3-e: recall-path truth — a centroid built from zero samples is unusable, not stale', () => {
assertC3RecallUnusable(BlockStore)
})
test('C3-f: status-path truth — zero samples with blocks present IS stale', () => {
assertC3StatusStale(BlockStore)
})
test('C3-g: the ratio is STRICTLY greater than 0.20', () => {
const store = c3Store(1201) // 1201 vs 1000 samples => delta 0.201
try {
c3Centroid(store, 1000)
const justOver = store.centroidStatus('fp-c3', { schedule: () => {}, now: () => 0 })
assert.equal(justOver.delta, 0.201, 'precondition: delta is 0.201')
assert.equal(justOver.stale, true, '0.201 > 0.20 => stale')
} finally {
store.close()
}
const exact = c3Store(12) // 12 vs 10 => delta exactly 0.20
try {
c3Centroid(exact, 10)
const at = exact.centroidStatus('fp-c3', { schedule: () => {}, now: () => 0 })
assert.equal(at.delta, 0.2, 'precondition: delta is exactly 0.20')
assert.equal(at.stale, false, 'exactly 20% is NOT stale — the comparison is strictly greater')
} finally {
exact.close()
}
})
test('C3-h: delta uses (current − built) / built, not the inverse', () => {
const store = c3Store(13)
try {
c3Centroid(store, 10)
const status = store.centroidStatus('fp-c3', { schedule: () => {}, now: () => 0 })
assert.equal(status.currentBlocks, 13)
assert.equal(status.vectorCount, 10)
// (13-10)/10 = 0.3. The inverted form would give (10-13)/13 ≈ 0.2308 and the
// swapped-denominator form would give 0 — neither equals 0.3.
assert.equal(status.delta, 0.3, 'delta must be (curVectored - vecCount) / vecCount')
assert.equal(status.stale, true, '0.3 > 0.2 => stale')
} finally {
store.close()
}
})
test('C3-i: a stale hit registers the rebuild and never rebuilds inline', () => {
const store = c3Store(13)
try {
c3Centroid(store, 10)
let syncCalls = 0
const realRebuild = store.rebuildCentroid.bind(store)
store.rebuildCentroid = (fp) => { syncCalls += 1; return realRebuild(fp) }
const clock = c3Clock()
const status = store.centroidStatus('fp-c3', { now: clock.now, schedule: clock.schedule })
assert.equal(status.stale, true)
assert.equal(status.available, false, 'a stale centroid is not used for centering this round')
assert.equal(syncCalls, 0, 'rebuilding inline would nest a transaction on the caller’s statement')
assert.equal(clock.state.scheduled.length, 1, 'it is registered instead')
} finally {
store.close()
}
})
test('C3-j: corrupt centroid metadata is an error, not a silent "no centering"', () => {
assertC3CorruptThrows(BlockStore)
})
test('C3-i†: replacing registration with inline rebuild bites the unchanged async criterion', async (t) => {
await withMutatedCopy(
'store.js',
[[
` const rebuildScheduled = this.#scheduleCentroidRebuild(fingerprint, { now, schedule })`,
` this.rebuildCentroid(fingerprint) // MUTATED: rebuild inline\n const rebuildScheduled = true`,
]],
async ({ BlockStore: MutantStore }) => {
const store = new MutantStore({ path: ':memory:' }).open()
try {
for (let i = 0; i < 13; i += 1) {
store.putBlock({ text: `C3 mutant ${i}`, vector: [1, 0, 0], fingerprint: 'fp-c3' })
}
store.saveCentroid({ fingerprint: 'fp-c3', dim: 3, vectorCount: 10, mean: [0, 0, 0], isAnomalous: true })
let syncCalls = 0
const rebuild = store.rebuildCentroid.bind(store)
store.rebuildCentroid = (fp) => { syncCalls += 1; return rebuild(fp) }
// The unchanged criterion: a stale observation must NOT rebuild inline.
// (It also must not lose the block, which is the second half.)
assert.throws(() => {
const status = store.centroidStatus('fp-c3', { now: () => 0, schedule: () => {} })
assert.equal(syncCalls, 0, 'stale handling must register, never rebuild inline')
assert.equal(status.rebuildScheduled, true, 'and the rebuild must still be registered')
}, (err) => {
assert.equal(err.code, 'ERR_ASSERTION')
t.diagnostic(`C3-i† RAW RED >>> ${err.message}`)
return true
}, 'inline rebuild must break the unchanged async criterion')
} finally { store.close() }
},
)
})
// C3 mutation guards. Each mutation changes lib/store.js and then runs the
// unchanged positive predicate; only ERR_ASSERTION is accepted as a bite.
async function c3Mutant(replacements, body) {
await withMutatedCopy('store.js', replacements, async ({ BlockStore: MutantStore }) => {
await body(MutantStore)
})
}
function c3Seed(store, current = 13, built = 10) {
for (let i = 0; i < current; i += 1) {
store.putBlock({ text: `C3 mutation ${i}`, vector: [1, 0, 0], fingerprint: 'fp-c3' })
}
store.saveCentroid({ fingerprint: 'fp-c3', dim: 3, vectorCount: built, mean: [0, 0, 0], isAnomalous: true })
}
test('C3† ratio boundary: >= instead of > bites the exact-0.20 criterion', async (t) => {
await c3Mutant([[
'if (delta > BlockStore.CENTROID_REBUILD_RATIO) {',
'if (delta >= BlockStore.CENTROID_REBUILD_RATIO) { // MUTATED: inclusive boundary',
]], async (MutantStore) => {
const store = new MutantStore({ path: ':memory:' }).open()
try {
c3Seed(store, 12, 10)
assert.throws(() => {
assert.equal(store.centroidStatus('fp-c3', { schedule: () => {}, now: () => 0 }).stale, false,
'exactly 0.20 must not be stale')
}, (err) => { assert.equal(err.code, 'ERR_ASSERTION'); t.diagnostic(`C3† boundary RAW RED >>> ${err.message}`); return true })
} finally { store.close() }
})
})
test('C3† direction: using currentBlocks as the denominator bites delta 0.30', async (t) => {
await c3Mutant([[
'(currentBlocks - vectorCount) / vectorCount',
'(currentBlocks - vectorCount) / currentBlocks', // MUTATED: wrong denominator
]], async (MutantStore) => {
const store = new MutantStore({ path: ':memory:' }).open()
try {
c3Seed(store, 13, 10)
assert.throws(() => {
assert.equal(store.centroidStatus('fp-c3', { schedule: () => {}, now: () => 0 }).delta, 0.3,
'delta must use built vectorCount as denominator')
}, (err) => { assert.equal(err.code, 'ERR_ASSERTION'); t.diagnostic(`C3† direction RAW RED >>> ${err.message}`); return true })
} finally { store.close() }
})
})
test('C3† throttle: deleting the 10-minute guard bites the window criterion', async (t) => {
await c3Mutant([[
'if (last !== undefined && now() - last < BlockStore.CENTROID_REBUILD_WAIT_MS) {',
'if (false) { // MUTATED: no throttle',
]], async (MutantStore) => {
const store = new MutantStore({ path: ':memory:' }).open()
try {
c3Seed(store)
const clock = c3Clock()
store.centroidStatus('fp-c3', { now: clock.now, schedule: clock.schedule })
clock.advance(1)
store.centroidStatus('fp-c3', { now: clock.now, schedule: clock.schedule })
assert.throws(() => {
assert.equal(clock.state.scheduled.length, 1, 'one registration within the throttle window')
}, (err) => { assert.equal(err.code, 'ERR_ASSERTION'); t.diagnostic(`C3† throttle RAW RED >>> ${err.message}`); return true })
} finally { store.close() }
})
})
test('C3† delay: changing 2000 to 0 bites the real-delay criterion', async (t) => {
await c3Mutant([[
'return 2000',
'return 0 // MUTATED: no two-second settling delay',
]], async (MutantStore) => {
const store = new MutantStore({ path: ':memory:' }).open()
try {
c3Seed(store)
const clock = c3Clock()
store.centroidStatus('fp-c3', { now: clock.now, schedule: clock.schedule })
assert.throws(() => {
assert.equal(clock.state.scheduled[0].delayMs, 2000, 'scheduled delay must be 2000 ms')
}, (err) => { assert.equal(err.code, 'ERR_ASSERTION'); t.diagnostic(`C3† delay RAW RED >>> ${err.message}`); return true })
} finally { store.close() }
})
})
test('C3† execution: dropping the rebuild call leaves stale and bites the live-path criterion', async (t) => {
await c3Mutant([[
' this.rebuildCentroid(fingerprint)',
' // MUTATED: registered callback no longer executes the rebuild',
]], async (MutantStore) => {
const store = new MutantStore({ path: ':memory:' }).open()
try {
c3Seed(store)
const clock = c3Clock()
store.centroidStatus('fp-c3', { now: clock.now, schedule: clock.schedule })
clock.fireAll()
assert.throws(() => {
assert.equal(store.centroidStatus('fp-c3', { now: () => 11 * 60 * 1000, schedule: clock.schedule }).stale, false,
'a registered rebuild must execute and clear stale')
}, (err) => { assert.equal(err.code, 'ERR_ASSERTION'); t.diagnostic(`C3† execution RAW RED >>> ${err.message}`); return true })
} finally { store.close() }
})
})
test('C3-k: countVectored is NOT wired into the blockStates hot path', async () => {
const { readFileSync } = await import('node:fs')
const source = readFileSync(new URL('../lib/store.js', import.meta.url), 'utf8')
const start = source.indexOf(' blockStates(activeFingerprint, activeDimension) {')
assert.notEqual(start, -1)
const end = source.indexOf('\n }\n\n /**', start)
assert.notEqual(end, -1)
const body = source.slice(start, end)
assert.equal(body.includes('countVectored'), false,
'blockStates runs on the pre-step hot path; the C3 counting query must stay out of it')
})
test('C3-e†: giving the recall path a stale bit bites the unchanged recall criterion', async (t) => {
// Mutation: the recall projection stops discarding the operator-only stale bit,
// i.e. merges the two Go call sites C3 keeps apart (:179-185 vs :431-433).
await c3Mutant([[
'return { ...status, available: status.available, stale: null }',
'return { ...status } // MUTATED: recall path now leaks the status stale bit',
]], async (MutantStore) => {
assert.throws(() => assertC3RecallUnusable(MutantStore), (err) => {
assert.equal(err.code, 'ERR_ASSERTION', 'only the criterion may bite')
t.diagnostic(`C3-e† RAW RED >>> ${String(err.message).split('\n')[0]}`)
return true
}, 'C3-e† must break the unchanged recall-path criterion')
})
})
test('C3-f†: flattening the status stale condition bites the unchanged status criterion', async (t) => {
// Mutation: the zero-sample branch stops judging staleness, which is exactly
// the second, independent stale condition (go :431-432) disappearing.
await c3Mutant([[
' available: false,\n stale: currentBlocks > 0,',
' available: false,\n stale: null, // MUTATED: second stale condition removed',
]], async (MutantStore) => {
assert.throws(() => assertC3StatusStale(MutantStore), (err) => {
assert.equal(err.code, 'ERR_ASSERTION', 'only the criterion may bite')
t.diagnostic(`C3-f† RAW RED >>> ${String(err.message).split('\n')[0]}`)
return true
}, 'C3-f† must break the unchanged status-path criterion')
})
})
test('C3-j†: swallowing corrupt metadata bites the unchanged error criterion', async (t) => {
// Mutation: dim <= 0 returns a silent "no centering" instead of throwing
// (go :165-167 returns an error, never nil,false,nil).
await c3Mutant([[
` throw new Error(\`dsh-memory: centroid dim is \${dim} (corrupted?)\`)`,
` return { available: false, stale: null, dim: null, vectorCount: null, currentBlocks, delta: null, rebuildScheduled: false } // MUTATED: swallowed`,
]], async (MutantStore) => {
assert.throws(() => assertC3CorruptThrows(MutantStore), (err) => {
assert.equal(err.code, 'ERR_ASSERTION', 'only the criterion may bite')
t.diagnostic(`C3-j† RAW RED >>> ${String(err.message).split('\n')[0]}`)
return true
}, 'C3-j† must break the unchanged corrupt-metadata criterion')
})
})
// ── V6: TEXT + `document` is INTERFACE-ONLY (semantics are a declared GAP) ──
//
// Go assigns `Side: "query"` for text (`internal/memory/vector/http_embedder.go:105`)
// and `Side: "document"` only for images (`:109`). It never sends TEXT with
// `document`, so there is no kernel precedent for what such a vector means.
//
// This slice pins only the WIRE SHAPE the adapter can produce: the value travels
// verbatim, and only Go's two enum members are accepted
// (`pkg/embedding/embedding.go:34-35`). Nothing here asserts that a text-document
// vector is semantically right, comparable to a query-side vector, or a separate
// space. Declared GAP — not semantic coverage.
async function captureEmbedRequest(createEmbedder, config, text = 'probe') {
const calls = []
const originalFetch = globalThis.fetch
globalThis.fetch = async (url, init) => {
calls.push({ url: String(url), body: JSON.parse(init.body) })
return new Response(JSON.stringify({ embedding: [1, 0, 0] }), { status: 200 })
}
try {
const embedder = createEmbedder({ endpoint: 'http://127.0.0.1:1/embed', dimension: 3, ...config })
const vector = await embedder.embedText(text)
await embedder.close()
return { calls, vector }
} finally {
globalThis.fetch = originalFetch
}
}
/** (1) side=document reaches the wire with modality still text. */
async function assertV6DocumentShape(createEmbedder) {
const { calls, vector } = await captureEmbedRequest(createEmbedder, { side: 'document' }, 'document probe')
assert.deepEqual(vector, [1, 0, 0], 'the response vector is returned verbatim')
assert.equal(calls.length, 1, 'one request was sent')
assert.equal(calls[0].body.side, 'document', 'the configured side travels on the wire')
assert.equal(calls[0].body.modality, 'text', 'and the modality stays text')
assert.equal(calls[0].body.text, 'document probe', 'the text payload is unchanged')
}
/** (2) omitting the option keeps the frozen query behaviour. */
async function assertV6DefaultQuery(createEmbedder) {
const { calls } = await captureEmbedRequest(createEmbedder, {}, 'q')
assert.equal(calls[0].body.side, 'query', 'omitting side must keep the frozen query behaviour')
}
/** (3) a value outside Go's enum is refused, never silently sent. */
function assertV6RejectsUnknownSide(createEmbedder) {
assert.throws(
() => createEmbedder({ endpoint: 'http://127.0.0.1:1/embed', dimension: 3, side: 'passage' }),
(err) => {
assert.match(String(err.message), /embed side must be 'query' or 'document'/)
return true
},
'inventing a protocol value Go does not define must fail loudly',
)
}
test('V6: text embedding can emit side=document on the wire (shape only, semantics GAP)', async () => {
const { createHttpEmbedder } = await import('../lib/embed/http.js')
await assertV6DocumentShape(createHttpEmbedder)
})
test('V6: the default side stays query — accepted behaviour unchanged', async () => {
const { createHttpEmbedder } = await import('../lib/embed/http.js')
await assertV6DefaultQuery(createHttpEmbedder)
})
test("V6: a side outside Go's enum is refused at construction", async () => {
const { createHttpEmbedder } = await import('../lib/embed/http.js')
assertV6RejectsUnknownSide(createHttpEmbedder)
})
for (const [name, anchor, replacement, criterion] of [
[
'side-hardcoded',
' side,\n text: String(text ?? \'\'),',
" side: 'query',\n text: String(text ?? ''), // MUTATED: side ignored",
assertV6DocumentShape,
],
[
'side-default-changed',
" side = 'query',",
" side = 'document', // MUTATED: default flipped",
assertV6DefaultQuery,
],
[
'side-unvalidated',
' if (side !== \'query\' && side !== \'document\') {',
' if (false) { // MUTATED: accept any side value',
assertV6RejectsUnknownSide,
],
]) {
test(`V6† ${name}: the unchanged shape criterion bites red`, async (t) => {
await withMutatedCopy('embed/http.js', [[anchor, replacement]], async (mod) => {
let failure = null
try {
await criterion(mod.createHttpEmbedder)
} catch (error) {
failure = error
}
assert.ok(failure !== null, `V6† ${name} must break the unchanged criterion`)
assert.equal(failure.code, 'ERR_ASSERTION', 'only the criterion may bite')
t.diagnostic(`V6† ${name} RAW RED >>> ${String(failure.message).split('\n')[0]}`)
})
})
}

View File

@ -0,0 +1,817 @@
/**
* P1 · Vector recall wiring — the seam that was computed and then thrown away.
*
* ## The defect this file exists for
*
* `lib/recall.js:681` ended with
*
* return withDiagnosis(recallBlocks(store, query, scoped), 'vector_ready_unused')
*
* `queryVector` had been computed, dimension-checked, and then deliberately
* dropped: `scoped` carried only `topK` / `signal` / `fingerprint` / `dimension`.
* So every V1–V7 scoring branch in `recallBlocks` was live code reachable ONLY
* from unit tests that hand-built a `vectorScores` Map. In production the recall
* path was pure symbol/FTS — the whole vector half of phase B was unreachable.
*
* Grep evidence at c1f7e06: `vectorScores:` appears 10 times in
* `test/phase-b-a.test.js` and **zero** times anywhere under `lib/`.
*
* ## Why this criterion is end-to-end and not a function test
*
* A test that calls `recallBlocks(store, q, { vectorScores, minScore })` proves
* the scoring seam works — which was never in doubt. It cannot see the bug,
* because the bug is that nobody ever passes those two options. The only
* observation that distinguishes the broken build from the fixed one is on the
* far side of the whole chain:
*
* real sqlite file → real `apply()`/`commit()` write path → real
* `recallBlocksHybrid` → real `readOnly` query embedding → `agent/pre-step`
* injection → `renderRecall` → the system prompt → the model's answer
*
* So: a real `dsh --profile headless` process, a real on-disk database, an HTTP
* embedding service the plugin reaches through its own `lib/embed/http.js`
* adapter, and a stub LLM whose answer is a pure function of the system prompt it
* received. The assertion is on the **final answer text**, which is as far from
* the library boundary as the harness allows.
*
* ## Two preconditions, asserted before the answer is
*
* "The model answered QP47-812" alone would be satisfied by a lucky symbol hit.
* The claim P1 needs is *"the vector route is what carried it"*, so the same turn
* also proves:
*
* P1a symbol path only → zero hits, and specifically zero hits on block A
* (the diagnostic `degraded` mode is printed as evidence);
* P1b the injected text is attributable to the vector route (a `text`-tagged
* hit carrying a vector score, not an `exact` pin).
*
* Without P1a/P1b a passing test would only show that *something* answered.
*
* ★ What this does NOT establish.
* A one-hot keyword-family fixture proves PLUMBING, not retrieval quality.
* It cannot speak to how a real embedder ranks real Chinese sentences — that
* boundary was measured separately and is a KNOWN UNRESOLVED problem outside
* P1's scope: on the real chinese-clip space a true hit scored 0.8345 while an
* unrelated block scored 0.8598, so absolute scores are not calibrated and no
* threshold may be read off them. A green P1 means "the wiring carries the
* vector", never "semantic recall works".
*
* ## The corpus, and why these exact strings
*
* The fake embedder is a deterministic keyword-family projection into 8 unit
* axes: family 0 is the target's vocabulary, families 1..7 are one axis per
* distractor sentence. A text matching no family maps to the zero vector, so it
* can never be a nearest neighbour by accident. This makes the "does the vector
* route reach block A" question decidable with no model, no network, and no
* tolerance — and it is what the corpus was **measured** against before any
* assertion was written (see the pre-check test below, which is the executable
* form of that measurement).
*
* The access code is `QP47-812`, not a bare number, so the stub LLM's answer
* extractor (`[A-Z]{2}[0-9]{2}-[0-9]{3}`) can find it and nothing else in the
* prompt can collide with it.
*
* ## Missing-harness conditions
*
* The harness needs `dsh` resolvable from PATH. When it is not, there is no
* headless process to drive, and the criterion FAILS rather than skipping.
*
* ★ Why `assert.fail` and not `t.skip`: a skip is green. `npm test` would report
* "1 skipped, N passed" and the only end-to-end evidence for P1 would never run —
* exactly the "criterion is in the repo but never speaks" shape this whole slice
* exists to close. It is also worse than a plain hole, because a green suite reads
* as proof. `loop-system-e2e.test.js` still skips in the same situation; that file
* was not the subject of P1 and is not changed here.
*
* @module dsh-memory/test/phase-b-p1-vector-wiring
*/
import assert from 'node:assert/strict'
import { execFileSync, spawn } from 'node:child_process'
import { existsSync, mkdtempSync, rmSync } from 'node:fs'
import { createServer } from 'node:http'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { fileURLToPath } from 'node:url'
import test from 'node:test'
import { BlockStore } from '../lib/store.js'
import { recallBlocks, recallBlocksHybrid } from '../lib/recall.js'
// ── the corpus ────────────────────────────────────────────────────────────────
//
// One target ("A") and seven distractors. Every distractor owns exactly ONE
// embedding axis, and the target owns the other one, so a query can only reach A
// by landing on family 0.
/** Keyword families. Index 0 is the target's; 1..7 are one per distractor. */
const FAMILIES = [
['门禁码', 'QP47-812', '四位数', '暗号', '铁皮门', '货场', '进入凭证', '通行'],
['午饭', '食堂', '十二点半'],
['叉车', '额定载重', '两吨'],
['打印机', '缺纸', '红灯'],
['地铁', '园区北门'],
['例会', '会议室'],
['温度', '保护装置'],
['包裹', '收发室'],
]
const DIMENSION = FAMILIES.length
/** The one block the model must be able to answer with. */
const TARGET = '第七号仓库的备用门禁码是 QP47-812。'
/** The code that must appear in the final answer, and nowhere else by accident. */
const ANSWER = 'QP47-812'
const DISTRACTORS = [
'午饭时间改到十二点半,食堂在东楼二层。',
'叉车的额定载重是两吨,超了会翻。',
'打印机缺纸时会闪红灯,备件在储物间第三格。',
'地铁二号线直达园区北门,出站走五分钟。',
'每月的第二个周五开例会,会议室在行政楼。',
'温度超过八十五度,保护装置会切断电源。',
'包裹统一送去一楼收发室的货架,凭手机尾号取。',
]
/**
* The question the user asks.
*
* Measured against the corpus before being pinned here: it extracts no symbol
* (`querySymbols` → []), shares no CJK 2-gram with TARGET or any distractor, and
* FTS finds nothing for it. It reaches family 0 only through `四位数`.
*/
const QUERY = '上次说的那个四位数是什么来着?'
/** The query's family axis. Asserted so a corpus edit cannot silently break it. */
const QUERY_FAMILY = 0
/**
* Deterministic keyword-family embedding.
*
* Returns a one-hot unit vector on the first matching family, or the zero vector
* when nothing matches. Zero is deliberate: an unmapped text must not be a
* neighbour of anything, so a false "vector hit" cannot be an artefact of a
* default direction.
*
* @param {string} text
* @returns {number[]}
*/
function embedByFamily(text) {
const haystack = String(text ?? '')
const vector = new Array(DIMENSION).fill(0)
for (let index = 0; index < FAMILIES.length; index += 1) {
if (FAMILIES[index].some((keyword) => haystack.includes(keyword))) {
vector[index] = 1
return vector
}
}
return vector
}
/** Cosine similarity, written out so the fixture cannot depend on lib scoring. */
function cosine(a, b) {
let dot = 0
let normA = 0
let normB = 0
for (let index = 0; index < a.length; index += 1) {
dot += a[index] * b[index]
normA += a[index] * a[index]
normB += b[index] * b[index]
}
if (normA === 0 || normB === 0) return 0
return dot / (Math.sqrt(normA) * Math.sqrt(normB))
}
/** CJK 2-grams of a string, used to prove "no lexical overlap" mechanically. */
function bigrams(text) {
const chars = [...String(text ?? '')].filter((char) => /[\u3400-\u4dbf\u4e00-\u9fff]/.test(char))
const out = new Set()
for (let index = 0; index + 1 < chars.length; index += 1) out.add(chars[index] + chars[index + 1])
return out
}
// ── the fake embedding service ────────────────────────────────────────────────
/**
* An HTTP embedding endpoint speaking the plugin's own wire protocol.
*
* This is deliberately a real socket, not an injected `config.embedder`: the
* point of P1 is that the *production* path works, and the production path goes
* through `createHttpEmbedder` in `lib/embed/http.js`. Injecting an embedder
* object would leave that adapter — and the JSON shape it accepts — untested.
*
* @returns {Promise<{url: string, close: () => Promise<void>}>}
*/
async function startEmbedServer() {
const server = createServer((request, response) => {
const chunks = []
request.on('data', (chunk) => chunks.push(chunk))
request.on('end', () => {
let text = ''
try {
text = JSON.parse(Buffer.concat(chunks).toString('utf8')).text ?? ''
} catch {
response.writeHead(400).end('bad json')
return
}
const body = JSON.stringify({ embedding: embedByFamily(text) })
response.writeHead(200, { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(body) })
response.end(body)
})
})
await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve))
const { port } = server.address()
return {
url: `http://127.0.0.1:${port}/embed`,
close: () => new Promise((resolve) => server.close(resolve)),
}
}
// ── the stub LLM ──────────────────────────────────────────────────────────────
/**
* An OpenAI-compatible endpoint whose answer is a pure function of the SYSTEM
* prompt it receives.
*
* ★ Why a stub rather than a real provider: the assertion is "did block A reach
* the system prompt". A real model answers probabilistically, so a red run could
* mean "the model was unlucky" and a green run could mean "the model guessed the
* code". This stub makes the measurement exact: if the code is in the system
* prompt the answer IS the code; if it is not, the answer is 未知.
*
* It is still a real HTTP provider from dsh's point of view — the plugin, the
* harness, the request builder, and `agent/pre-step` are all the production ones.
*
* @returns {Promise<{url: string, close: () => Promise<void>}>}
*/
async function startStubLlm() {
const answerFor = (messages) => {
const system = messages
.filter((message) => message?.role === 'system')
.map((message) => (typeof message.content === 'string'
? message.content
: (message.content ?? []).map((block) => (typeof block?.text === 'string' ? block.text : '')).join('')))
.join('\n')
const found = system.match(/[A-Z]{2}\d{2}-\d{3}/)
return found === null ? '未知' : found[0]
}
const server = createServer((request, response) => {
if (request.method === 'GET') {
const body = JSON.stringify({ object: 'list', data: [{ id: 'stub-1', object: 'model' }] })
response.writeHead(200, { 'Content-Type': 'application/json' }).end(body)
return
}
const chunks = []
request.on('data', (chunk) => chunks.push(chunk))
request.on('end', () => {
let payload = {}
try {
payload = JSON.parse(Buffer.concat(chunks).toString('utf8'))
} catch {
response.writeHead(400).end('bad json')
return
}
const answer = answerFor(payload.messages ?? [])
const base = { id: 'chatcmpl-stub', created: 0, model: payload.model ?? 'stub-1' }
if (payload.stream === true) {
response.writeHead(200, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
Connection: 'close',
})
const send = (chunk) => response.write(`data: ${JSON.stringify(chunk)}\n\n`)
send({ ...base, object: 'chat.completion.chunk', choices: [{ index: 0, delta: { role: 'assistant' }, finish_reason: null }] })
send({ ...base, object: 'chat.completion.chunk', choices: [{ index: 0, delta: { content: answer }, finish_reason: null }] })
send({ ...base, object: 'chat.completion.chunk', choices: [{ index: 0, delta: {}, finish_reason: 'stop' }] })
response.end('data: [DONE]\n\n')
return
}
response.writeHead(200, { 'Content-Type': 'application/json' }).end(JSON.stringify({
...base,
object: 'chat.completion',
choices: [{ index: 0, message: { role: 'assistant', content: answer }, finish_reason: 'stop' }],
usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 },
}))
})
})
await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve))
const { port } = server.address()
return {
url: `http://127.0.0.1:${port}/v1`,
close: () => new Promise((resolve) => server.close(resolve)),
}
}
// ── the headless harness ──────────────────────────────────────────────────────
/** True when a `dsh` binary is on PATH and resolves. */
function dshAvailable() {
try {
execFileSync('dsh', ['--version'], { stdio: 'ignore' })
return true
} catch {
return false
}
}
/**
* A private DSH_HOME with the stock headless profile's bundles.
*
* Written to a temp dir instead of reusing `$DSH_HOME` so the run cannot inherit
* whatever the developer has configured (a different embedder, a different memory
* database, a debug toggle). The only inputs are the ones this file supplies.
*
* @param {string} home
* @param {string} packageRoot
*/
function writeProfile(home, packageRoot) {
const profileDir = join(home, 'profiles', 'headless')
execFileSync('mkdir', ['-p', join(profileDir, 'node_modules')])
execFileSync('sh', ['-c', `ln -sfn ${JSON.stringify(packageRoot)} ${JSON.stringify(join(profileDir, 'node_modules', 'dsh-memory'))}`])
// An empty tree: the bundles in package.json supply the base rows, and the
// overlay supplies `memory`. Nothing from the developer's own profile leaks in.
execFileSync('sh', ['-c', `printf '[]\\n' > ${JSON.stringify(join(profileDir, 'cordis.yml'))}`])
execFileSync('sh', ['-c', `cat > ${JSON.stringify(join(profileDir, 'package.json'))} <<'P1PKG'
${JSON.stringify({
name: 'dsh-profile-headless',
private: true,
dependencies: { 'dsh-memory': `link:${packageRoot}` },
dsh: { profile: { bundles: ['@deepseek-ai/dsh-base', '@deepseek-ai/dsh-headless'] } },
}, null, 2)}
P1PKG`])
return profileDir
}
/**
* Run one headless task and return the final answer text plus stderr.
*
* @param {object} options
* @param {string} options.home private DSH_HOME
* @param {string} options.profileDir
* @param {string} options.dbPath
* @param {string} options.embedUrl
* @param {string} options.llmUrl
* @param {string} options.query
* @param {string} options.cwd
* @returns {Promise<{answer: string, stderr: string, sessionId: string|null}>}
*/
function runHeadless({ home, profileDir, dbPath, embedUrl, llmUrl, query, cwd }) {
const overlay = join(profileDir, 'p1-overlay.yml')
// ★ `insert:` with an explicit `name:` and NOT a bare `- id: memory` merge.
// The private DSH_HOME written by {@link writeProfile} has no `memory` row of
// its own, so a merge-style patch would silently compose an entry with no
// plugin behind it: the config would appear in `--dump-config`, the harness
// would boot, and the recall hook would simply never be registered. That is a
// no-op that looks configured — the same shape as the bug this file pins.
// The two other rows are base entries and are legitimately merged by id.
const yaml = [
'- insert:',
' - id: memory',
' name: dsh-memory',
' config:',
` path: ${JSON.stringify(dbPath)}`,
' topK: 3',
' debugLog: true',
' inject: true',
' embed:',
` endpoint: ${JSON.stringify(embedUrl)}`,
` dimension: ${DIMENSION}`,
' model: p1-keyword-family',
'- id: agent-default-model',
' config:',
' provider: stub',
' model: stub-1',
'- id: llm-pi-ai',
' config:',
' providers:',
' stub:',
' displayName: stub',
' api: openai-completions',
` baseURL: ${JSON.stringify(llmUrl)}`,
' apiKeyEnv: P1_STUB_KEY',
' models:',
' - id: stub-1',
' name: stub-1',
' contextWindow: 65536',
// ── P0: the replacement loop must be mounted ────────────────────────────
//
// Without these two rows the run exercises the STOCK agent loop, which leaves
// the recall in user-message history. The recall would then never reach the
// system prompt, and P1's "the answer came from a system section" claim would
// be unprovable — worse, the assertion would pass for the wrong reason if the
// stub LLM were to look at history too (it deliberately does not, which is
// what makes the failure visible). This is byte-for-byte the shipped
// `cordis.patch.yml` overlay, so the run composes the same tree the profile
// does. It also pins the P0 result: 377e02c + c1f7e06 must survive P1.
'- id: agent-loop',
' disabled: true',
'- insert:',
' - id: hologram-agent-loop',
' name: dsh-memory/loop',
'',
].join('\n')
execFileSync('sh', ['-c', `cat > ${JSON.stringify(overlay)} <<'P1YAML'\n${yaml}P1YAML`])
return new Promise((resolve, reject) => {
const child = spawn(
'dsh',
['--profile', 'headless', '--patch', overlay, '--json', query],
{ cwd, env: { ...process.env, DSH_HOME: home, P1_STUB_KEY: 'p1-stub-key' } },
)
let stdout = ''
let stderr = ''
child.stdout.on('data', (chunk) => { stdout += chunk })
child.stderr.on('data', (chunk) => { stderr += chunk })
child.on('error', reject)
child.on('close', (code) => {
const events = stdout
.split('\n')
.filter((line) => line.trim() !== '')
.map((line) => {
try {
return JSON.parse(line)
} catch {
return null
}
})
.filter((event) => event !== null)
const final = events.findLast((event) => event.type === 'final')
const session = events.find((event) => event.type === 'session')
if (final === undefined) {
reject(new Error(`headless run produced no final event (exit ${code})\nstderr:\n${stderr}\nstdout:\n${stdout}`))
return
}
resolve({ answer: final.text ?? '', stderr, sessionId: session?.sessionId ?? null })
})
})
}
/**
* Decompress a session log and return its JSONL lines.
*
* `session.v4.jsonl.zstd` is what the harness actually persists, so it is what
* an operator would read. Parsing it (rather than trusting the run's own stdout)
* is the only way to see the injected prompt the model was really given.
*
* @param {string} home
* @param {string} sessionId
* @returns {string[]}
*/
function readSessionLines(home, sessionId) {
// The workspace key is a lossy slug of the launch cwd, so it cannot be
// reconstructed reliably (a real directory called `a-b` and one called `a/b`
// collide). Search for the session directory by its id instead — the id is
// exact and unique.
const sessionsRoot = join(home, 'sessions')
const candidates = existsSync(sessionsRoot)
? execFileSync('sh', ['-c', `find ${JSON.stringify(sessionsRoot)} -maxdepth 2 -type d -name ${JSON.stringify(sessionId)}`])
.toString('utf8')
.split('\n')
.filter((line) => line.trim() !== '')
: []
assert.equal(
candidates.length,
1,
`exactly one session directory must exist for ${sessionId} under ${sessionsRoot}; found ${JSON.stringify(candidates)}`,
)
const file = join(candidates[0], 'session.v4.jsonl.zstd')
assert.ok(existsSync(file), `session log must exist at ${file}`)
return execFileSync('zstd', ['-dc', file], { maxBuffer: 256 * 1024 * 1024 })
.toString('utf8')
.split('\n')
.filter((line) => line.trim() !== '')
}
/** Every system-prompt body in a session log, joined. */
function systemPromptFrom(lines) {
return lines
.map((line) => {
try {
return JSON.parse(line)
} catch {
return null
}
})
.filter((event) => event?.type === 'system/message')
.map((event) => (event.data?.message?.content ?? [])
.map((block) => (typeof block?.text === 'string' ? block.text : ''))
.join(''))
.join('\n')
}
// ── module setup shared by every case ─────────────────────────────────────────
const PACKAGE_ROOT = fileURLToPath(new URL('..', import.meta.url))
/** A store seeded through `putBlock` with family vectors, for the library-level checks. */
function seededStore() {
const store = new BlockStore({ path: ':memory:' }).open()
const fingerprint = 'p1-keyword-family'
const targetId = store.putBlock({ text: TARGET, source: 'p1', vector: embedByFamily(TARGET), fingerprint }).id
const distractorIds = DISTRACTORS.map((text) => store.putBlock({
text, source: 'p1', vector: embedByFamily(text), fingerprint,
}).id)
return { store, fingerprint, targetId, distractorIds }
}
// ══════════════════════════════════════════════════════════════════════════════
// P1-0 · corpus pre-check — the executable form of "measured before asserted"
// ══════════════════════════════════════════════════════════════════════════════
/**
* The pre-check that must pass BEFORE the criterion is allowed to assert
* anything. It fails on a corpus problem, never on the wiring bug, and it is
* kept in the suite so a later corpus edit cannot quietly make the e2e
* criterion unfalsifiable.
*
* Two conditions, both mandated:
* 1. the symbol path returns NO hit for the target (so a green answer can
* only be the vector route);
* 2. with a hand-built `vectorScores` Map plus `minScore: 0`, the target IS
* top-1 (so the corpus and vectors can actually reach it at all).
*/
test('P1-0: the corpus is admissible — symbol path blind, vector path reaches the target', () => {
const { store, targetId } = seededStore()
try {
// ── the query really does extract no symbol and share no 2-gram ──────────
const queryVector = embedByFamily(QUERY)
assert.equal(
queryVector.indexOf(1),
QUERY_FAMILY,
`the query must land on family ${QUERY_FAMILY}; got ${JSON.stringify(queryVector)}`,
)
const queryGrams = bigrams(QUERY)
for (const text of [TARGET, ...DISTRACTORS]) {
const shared = [...bigrams(text)].filter((gram) => queryGrams.has(gram))
assert.deepEqual(
shared,
[],
`precondition: the query must share no CJK 2-gram with "${text}"; shared ${JSON.stringify(shared)}`,
)
}
// ── condition 1: the symbol path is blind to the target ─────────────────
const symbolOnly = recallBlocks(store, QUERY, { topK: 5 })
assert.equal(
symbolOnly.filter((hit) => hit.id === targetId).length,
0,
`precondition 1: the symbol path must NOT find the target; it returned ${JSON.stringify(
symbolOnly.map((hit) => [hit.text, hit.score]),
)}`,
)
// ── condition 2: a manually built vector route DOES reach the target ────
const scores = new Map(store.allBlocks(-1).map((block) => [
block.id,
cosine(queryVector, JSON.parse(block.vector)),
]))
const fused = recallBlocks(store, QUERY, { topK: 3, vectorScores: scores, minScore: 0 })
assert.ok(fused.length > 0, 'precondition 2: the fused route must return something')
assert.equal(
fused[0].id,
targetId,
`precondition 2: the target must be top-1 once vectors participate; got ${
JSON.stringify(fused.map((hit) => [hit.text, hit.score]))
}`,
)
} finally {
store.close()
}
})
// ══════════════════════════════════════════════════════════════════════════════
// P1 · the end-to-end criterion
// ══════════════════════════════════════════════════════════════════════════════
test('P1: a semantic-only question is answered from block memory via the vector route', async (t) => {
if (!dshAvailable()) {
assert.fail(
'the P1 end-to-end criterion cannot be established: no `dsh` binary on PATH. ' +
'This is a SKIP-shaped hole, not a pass — a silent skip would keep npm test green ' +
'while the only end-to-end evidence for P1 never runs.',
)
}
const workspace = mkdtempSync(join(tmpdir(), 'p1-e2e-'))
const home = join(workspace, 'home')
const cwd = join(workspace, 'work')
execFileSync('mkdir', ['-p', cwd])
const profileDir = writeProfile(home, PACKAGE_ROOT)
const dbPath = join(workspace, 'memory.db')
const embedServer = await startEmbedServer()
const llmServer = await startStubLlm()
try {
// ── seed through the real plugin write path ──────────────────────────────
//
// `apply()` builds the plugin exactly as the profile does, `commit()` runs
// the production embed → `putBlock(vector, fingerprint)` path. Nothing here
// writes a vector by hand: a block that reached the database without the
// real write path could not prove the read path is wired.
const { Context } = await import('@deepseek-ai/cordis')
const { SystemPrompt } = await import('@deepseek-ai/dsh-system-prompt')
const { ToolRuntime } = await import('@deepseek-ai/dsh-tools')
const { apply: applyMemory } = await import('../lib/index.js')
const seedCtx = new Context()
let space
try {
new SystemPrompt(seedCtx, {})
new ToolRuntime(seedCtx)
await new Promise((resolve) => setTimeout(resolve, 0))
const mem = applyMemory(seedCtx, {
path: dbPath,
topK: 3,
debugLog: false,
inject: true,
embed: { endpoint: embedServer.url, dimension: DIMENSION, model: 'p1-keyword-family' },
})
await new Promise((resolve) => setTimeout(resolve, 0))
space = mem.space()
assert.ok(space !== null && space.dimension === DIMENSION, `the plugin must expose the fake space; got ${JSON.stringify(space)}`)
for (const text of [TARGET, ...DISTRACTORS]) {
const written = await mem.commit({ text, source: 'p1-seed' })
assert.equal(
written.embedded,
true,
`seed block must be stored WITH a vector (real embed path), not symbol-only: ${text}`,
)
}
assert.equal(mem.store.countBlocks(), 1 + DISTRACTORS.length, 'every corpus block must be on disk')
} finally {
await seedCtx.fiber.dispose()
}
// ── P1a · the symbol path is blind, proved on the real on-disk database ──
const store = new BlockStore({ path: dbPath }).open()
let symbolHits
try {
symbolHits = recallBlocks(store, QUERY, { topK: 5, fingerprint: space.fingerprint, dimension: space.dimension })
} finally {
store.close()
}
assert.equal(
symbolHits.length,
0,
`P1a precondition: the symbol path must find NOTHING for this query; got ${JSON.stringify(
symbolHits.map((hit) => [hit.text, hit.score]),
)}`,
)
// The current build's own diagnostic, captured before the run: on c1f7e06 this
// is `vector_ready_unused` and the array is empty.
const hybridStore = new BlockStore({ path: dbPath }).open()
let hybridDiagnosis
try {
const probe = await import('../lib/recall.js')
const hybrid = await probe.recallBlocksHybrid(
hybridStore,
{
embedText: async (text) => embedByFamily(text),
info: () => space,
},
QUERY,
{ topK: 3 },
)
hybridDiagnosis = { mode: hybrid.degraded ?? 'ok', hits: hybrid.length }
} finally {
hybridStore.close()
}
// ── the real headless run ────────────────────────────────────────────────
const run = await runHeadless({
home, profileDir, dbPath, embedUrl: embedServer.url, llmUrl: llmServer.url, query: QUERY, cwd,
})
// ── the injected text, recovered from the persisted session log ──────────
//
// Everything below is derived from the harness's own on-disk record, not from
// the run's stdout: the session log carries the exact system prompt the
// adapter was handed, which is the only place the injection is observable.
assert.ok(
run.sessionId !== null,
`the run must report its session id; stderr was:\n${run.stderr}`,
)
const lines = readSessionLines(home, run.sessionId)
const systemPrompt = systemPromptFrom(lines)
// The recall SECTION, not just its header line: `renderRecall` emits a header
// and then one line per hit, so the entry lines (which carry the `text` tag and
// the score) live BELOW the header. Taking the section rather than the first
// matching line is what lets P1c assert on the hit's shape.
const systemLines = systemPrompt.split('\n')
const recallStart = systemLines.findIndex((line) => line.includes('Recalled from durable block memory'))
const recallLine = recallStart === -1 ? undefined : systemLines.slice(recallStart).join('\n')
t.diagnostic(`P1 symbol-path baseline: hits=${symbolHits.length}`)
t.diagnostic(`P1 recall diagnosis before the run: degraded=${hybridDiagnosis.mode} hits=${hybridDiagnosis.hits}`)
t.diagnostic(`P1 recall line injected: ${recallLine ?? '(none)'}`)
// ── P1 · the answer ──────────────────────────────────────────────────────
//
// The primary assertion. On c1f7e06 the recall array is empty (`hits=0`), so
// the stub LLM — which answers ONLY with a code it finds in the system prompt
// — replies 未知. That failure IS the defect: the vector was computed and
// thrown away.
assert.equal(
run.answer.trim(),
ANSWER,
`P1: the answer must come from block memory through the vector route.\n` +
` query: ${QUERY}\n` +
` expected: ${ANSWER}\n` +
` actual: ${JSON.stringify(run.answer)}\n` +
` symbol-path hits: ${symbolHits.length} (must be 0)\n` +
` recall diagnosis: degraded=${hybridDiagnosis.mode} hits=${hybridDiagnosis.hits}\n` +
` recall section injected: ${recallLine ?? '(none)'}`,
)
// ── P1b · the recall reached the SYSTEM prompt, not user history (P0) ────
//
// P0 (377e02c + c1f7e06) must still hold after the wiring change: the recall
// is a system *section*, so it may appear in no non-system message body.
assert.ok(
recallLine !== undefined,
`P1b: the recall section must reach the system prompt (P0 must still hold). ` +
`degraded=${hybridDiagnosis.mode} symbolHits=${symbolHits.length}\n` +
`stderr:\n${run.stderr}`,
)
const leakingBodies = lines
.map((line) => {
try {
return JSON.parse(line)
} catch {
return null
}
})
.filter((event) => event?.type === 'user/message' || event?.type === 'assistant/message')
.map((event) => JSON.stringify(event.data ?? {}))
.filter((body) => body.includes(TARGET))
const recallCarrierMessages = (() => {
// Count carriers by role across every message the harness recorded.
const carriers = []
for (const line of lines) {
let event
try {
event = JSON.parse(line)
} catch {
continue
}
const message = event?.data?.message ?? event?.data
const text = (message?.content ?? [])
.map((block) => (typeof block?.text === 'string' ? block.text : ''))
.join('')
if (typeof message?.role === 'string' && text.includes(TARGET)) carriers.push(message.role)
}
return carriers
})()
t.diagnostic(`P1b recall carriers by role: ${JSON.stringify(recallCarrierMessages)}`)
t.diagnostic(`P1b non-system bodies containing the target: ${JSON.stringify(leakingBodies)}`)
assert.deepEqual(
leakingBodies,
[],
'P1b: the recalled note must NOT appear in any user/assistant message body (a move, not a copy)',
)
// Positive half of the same claim: exactly ONE message carries it, and that
// message is the system prompt. Without this, a build that dropped the recall
// from the prompt entirely could still satisfy the negative assertion above —
// and P0 is precisely the property that must survive P1.
assert.deepEqual(
recallCarrierMessages,
['system'],
`P1b: the recalled note must be carried by exactly one message, the system prompt; got ${JSON.stringify(
recallCarrierMessages,
)}`,
)
// ── P1c · the hit is a VECTOR hit, not an exact symbol pin ──────────────
//
// `renderRecall` tags a hit `(exact 1.0x)` when a numeric symbol pinned it and
// `(text 0.xx)` otherwise. Our corpus guarantees no symbol match, so a `text`
// tag with a sub-1.0 score is the observable signature of the fused vector
// route (`vec*0.7 + sym*0.3`). If the wiring regressed to a symbol-only path
// this tag would read `exact`, and if vectors stopped participating entirely
// there would be no line at all (caught by P1 above).
assert.match(
recallLine,
/\[\d+\] \(text 0\.\d{2}\) .*QP47-812/,
`P1c: the injected entry must be a \`text\`-tagged hit (vector route) carrying the target, ` +
`not an \`exact\` symbol pin. recall line was: ${recallLine}\n` +
`diagnosis at probe time: degraded=${hybridDiagnosis.mode} hits=${hybridDiagnosis.hits}`,
)
} finally {
await embedServer.close()
await llmServer.close()
// ★ `P1_KEEP=1` keeps the workspace so an operator (or a review) can read the
// session log, the sqlite file and the composed overlay by hand. Off by
// default: a test that leaves databases behind on every run turns /tmp into a
// second, unaudited store of memory. The directory is printed either way, so a
// preserved run is discoverable from the test output alone.
if (process.env.P1_KEEP === '1') {
t.diagnostic(`P1 workspace kept: ${workspace}`)
console.log(`P1 workspace kept: ${workspace}`)
} else {
rmSync(workspace, { recursive: true, force: true })
}
}
})

View File

@ -0,0 +1,153 @@
/**
* V6 · 写入侧与查询侧必须用不同的 `side`。
*
* ## 病
*
* HA 的 HTTP 嵌入协议带一个 `side` 字段(`query` / `document`,
* `http_embedder.go:105` / `:109`;枚举 `PurposeQuery` / `PurposeDocument`)。
* 检索类模型通常对两侧做不同的编码——存进来的文档向量与提问的查询向量落在
* 同一个坐标系的**不同子空间**里。两侧都用 `query` 时,块向量和查询向量都
* 是「正确宽度、正确模长的单位向量」,所以没有任何东西会报错:只是在真正
* 需要语义匹配时对不上。
*
* 移植时 `side` 被做进了 adapter 的构造参数(默认 `query`),但**没有一个
* 调用点传它**:写入路径与召回路径都走默认值。于是「文档按查询侧编码」这件
* 事发生了,且无法从任何返回值上观察到。
*
* ## 判据形态
*
* 起一个真的 HTTP 服务,把每个请求体里的 `side` 记下来,然后用**生产装配**
* (`apply()` + `config.embed`)跑一次写入和一次召回:
*
* commit → 请求体 side 必须是 'document'
* recall → 请求体 side 必须是 'query'
*
* 断言落在**线上真正发出去的字段**,不是内部变量。变异(写入侧改回共用
* 查询实例)会让 commit 的 side 变回 'query',这条立刻红。
*
* ★ 为什么不用注入式 embedder:注入的是单个对象,没有「两个编码器」可言,
* 绕开它就把被测的装配逻辑整个跳过了。这条判据必须走 HTTP 装配。
*
* @module dsh-memory/test/phase-b-v6-sides
*/
import assert from 'node:assert/strict'
import test from 'node:test'
import { createServer } from 'node:http'
import { mkdtempSync, rmSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
const DIM = 4
const unit = (v) => { const n = Math.hypot(...v); return n === 0 ? v : v.map((x) => x / n) }
const vecFor = (text) => unit([1, String(text).length % 7 === 0 ? 0.4 : 0.2, 0.1, 0])
/** 起一个记录 `side` 的嵌入服务。 */
async function startRecordingEmbedder() {
const seen = []
const server = createServer((req, res) => {
const chunks = []
req.on('data', (c) => chunks.push(c))
req.on('end', () => {
let payload = {}
try { payload = JSON.parse(Buffer.concat(chunks).toString('utf8')) } catch { /* ignore */ }
seen.push({ side: payload.side, modality: payload.modality, text: payload.text })
const body = JSON.stringify({ embedding: vecFor(payload.text) })
res.writeHead(200, { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(body) })
res.end(body)
})
})
await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve))
return {
url: `http://127.0.0.1:${server.address().port}/embed`,
seen,
close: () => new Promise((resolve) => server.close(resolve)),
}
}
test('V6: storing a block encodes it on the document side, asking encodes the query side', async () => {
const { Context } = await import('@deepseek-ai/cordis')
const { SystemPrompt } = await import('@deepseek-ai/dsh-system-prompt')
const { ToolRuntime } = await import('@deepseek-ai/dsh-tools')
const { apply } = await import('../lib/index.js')
const embed = await startRecordingEmbedder()
const dir = mkdtempSync(join(tmpdir(), 'v6-'))
const ctx = new Context()
new SystemPrompt(ctx, {})
new ToolRuntime(ctx, {})
try {
// ★ 生产装配:给它 config.embed,让它自己建两个编码器。
const mem = apply(ctx, {
path: join(dir, 'm.db'),
embed: { endpoint: embed.url, dimension: DIM, model: 'v6-probe' },
})
// ── 写入侧 ────────────────────────────────────────────────────────────
const written = await mem.commit({ text: '值班室的分机号是 4324' })
assert.equal(written.embedded, true, 'precondition: the block must actually be embedded')
const afterWrite = embed.seen.at(-1)
assert.equal(
afterWrite?.side,
'document',
'V6: a block being STORED must be encoded on the document side. Both sides '
+ 'defaulting to "query" is invisible at runtime — the vectors have the right '
+ 'width and length either way — and it silently puts stored memories in a '
+ `different subspace from the questions. got side=${JSON.stringify(afterWrite?.side)}`,
)
assert.equal(afterWrite.modality, 'text', 'and the modality field still travels')
// ── 召回侧 ────────────────────────────────────────────────────────────
const before = embed.seen.length
await mem.recall('分机号是多少')
const afterRecall = embed.seen.slice(before)
assert.equal(afterRecall.length, 1, `precondition: the recall must embed exactly once; got ${afterRecall.length}`)
assert.equal(
afterRecall[0].side,
'query',
'V6: a QUESTION must be encoded on the query side',
)
// ── 两侧的指纹必须一致,否则块与中心会落在两个空间里 ────────────────
assert.equal(
mem.space().dimension,
DIM,
'V6: the two encoders must still describe ONE space (the fingerprint feeds '
+ 'both memory_blocks and graph_centroid)',
)
} finally {
await ctx.fiber.dispose()
rmSync(dir, { recursive: true, force: true })
await embed.close()
}
})
test('V6: an injected single embedder keeps working and is used for both sides', async () => {
const { Context } = await import('@deepseek-ai/cordis')
const { SystemPrompt } = await import('@deepseek-ai/dsh-system-prompt')
const { ToolRuntime } = await import('@deepseek-ai/dsh-tools')
const { apply } = await import('../lib/index.js')
const dir = mkdtempSync(join(tmpdir(), 'v6-inj-'))
let calls = 0
const injected = {
async embedText() { calls += 1; return vecFor('x') },
info: () => ({ fingerprint: 'v6:injected:test', dimension: DIM }),
}
const ctx = new Context()
new SystemPrompt(ctx, {})
new ToolRuntime(ctx, {})
try {
// 注入式 embedder 没有「两个编码器」可言:它必须照旧被两侧共用,
// 否则所有既有测试与单侧适配器都会被打断。
const mem = apply(ctx, { path: join(dir, 'm.db'), embedder: injected })
const written = await mem.commit({ text: '值班室的分机号是 4324' })
assert.equal(written.embedded, true)
await mem.recall('分机号是多少')
assert.ok(calls >= 2, `the single injected embedder must serve both paths; got ${calls} calls`)
} finally {
await ctx.fiber.dispose()
rmSync(dir, { recursive: true, force: true })
}
})

View File

@ -0,0 +1,306 @@
/**
* D2 · 每一次丢弃都必须可计数。
*
* ## 病
*
* 一条候选可以从三个地方消失,而在本次改动之前只有前两个(空间过滤、宽度
* 不符)会被报告:
*
* · MinScore 门 —— 它自己的向量分不够
* · 结构性门 —— 既没有可用的向量分,也没有符号命中
* · topK 截断 —— 过了所有门,但排在调用方要的条数之外
*
* 后三个都是裸 `continue`,不进任何计数器。于是「6 条候选留下 1 条」与
* 「库里只有 1 条」在对外读数上完全一样:`skipped=0 skippedDim=0`。
* 实测(改动前):6 条块、minScore=0.99 → hits=0,两个计数都是 0。
*
* 运维看 `skipped=0` 会得出「没有任何东西被排除」的结论,而真相可能是阈值
* 把整批都挡了 —— 一个应该调阈值的信号被读成了「库是空的」。
*
* ## 判据形态
*
* 三条路径各造一个 fixture,断**各自**的计数。另加一条不变式:
* 候选池 = 命中 + 三个下游计数 + 上游两个计数,等式必须成立 —— 否则将来再
* 加一条静默丢弃路径,等式就会破。
*
* @module dsh-memory/test/recall-accounting
*/
import assert from 'node:assert/strict'
import test from 'node:test'
import { BlockStore } from '../lib/store.js'
import { recallBlocks, recallBlocksHybrid } from '../lib/recall.js'
const DIM = 8
const FP = 'd2:accounting'
const unit = (v) => { const n = Math.hypot(...v); return n === 0 ? v : v.map((x) => x / n) }
const axis = (i) => unit(Array.from({ length: DIM }, (_, k) => (k === i ? 1 : 0)))
/** 6 条块,各占一轴;query 落在轴 0,所以 cosine 已知且互不相同。 */
function seeded() {
const store = new BlockStore({ path: ':memory:' }).open()
const ids = []
for (let i = 0; i < 6; i += 1) {
ids.push(store.putBlock({ text: `块 ${i} 的正文`, vector: axis(i), fingerprint: FP }).id)
}
const query = unit([1, 0.5, 0.2, 0, 0, 0, 0, 0])
const scores = new Map(ids.map((id, i) => [id, i === 0 ? 0.95 : 0.1 + i * 0.01]))
return { store, ids, query, scores }
}
test('D2: candidates rejected by MinScore are counted, not silently dropped', () => {
const { store, query, scores } = seeded()
try {
const hits = recallBlocks(store, query, { topK: 6, fingerprint: FP, dimension: DIM, vectorScores: scores, minScore: 0.9 })
assert.equal(hits.length, 1, 'precondition: only the 0.95 candidate clears 0.9')
assert.equal(
hits.skippedScore,
5,
'D2: five candidates failed the score gate and must be REPORTED as such. '
+ 'Before this counter existed the result was indistinguishable from a '
+ 'one-block space, which sends the operator looking at the wrong subsystem.',
)
} finally {
store.close()
}
})
test('D2: candidates with neither vector score nor symbol hit are counted separately', () => {
const { store, ids, query } = seeded()
try {
// A candidate reaches the STRUCTURAL gate when it IS in the pool but has no
// usable vector score (absent, or <= 0) and no symbol match. A cosine of
// exactly 0 is the realistic trigger: it is what a candidate orthogonal to
// the query gets, and also what centring produces in a degenerate space.
const scores = new Map([[ids[0], 0.95], [ids[1], 0.8], [ids[2], 0]])
const hits = recallBlocks(store, 'zzz-no-symbol-match', {
topK: 6, fingerprint: FP, dimension: DIM, vectorScores: scores, minScore: 0.5,
})
assert.equal(hits.length, 2, 'precondition: only the two positive scores survive')
assert.equal(hits.skippedScore, 0, 'precondition: nothing failed the threshold')
assert.equal(
hits.skippedNoEvidence,
1,
'D2: a candidate that matched neither route must be counted under its own name — '
+ '"nothing to score" and "scored too low" have opposite remedies (re-embed vs. '
+ 'lower the threshold). Blocks that never entered the pool are a different, '
+ 'upstream fact and must not be folded in here.',
)
} finally {
store.close()
}
})
test('D2: candidates cut by the caller topK are counted as truncated', () => {
const { store, query, scores } = seeded()
try {
const all = recallBlocks(store, query, { topK: 6, fingerprint: FP, dimension: DIM, vectorScores: scores, minScore: 0.05 })
const top2 = recallBlocks(store, query, { topK: 2, fingerprint: FP, dimension: DIM, vectorScores: scores, minScore: 0.05 })
assert.equal(all.length, 6, 'precondition: all six clear a 0.05 threshold')
assert.equal(top2.length, 2, 'precondition: topK truncates')
assert.equal(top2.skippedScore, 0, 'nothing failed the score gate here')
assert.equal(
top2.truncated,
4,
'D2: the four that fell outside topK must be reported. Truncation is not a '
+ 'defect, but leaving it uncounted breaks the reconciliation of '
+ 'candidates = hits + counters.',
)
} finally {
store.close()
}
})
test('D2: hits plus every counter reconciles with the candidate pool', () => {
const { store, query, scores } = seeded()
try {
const hits = recallBlocks(store, query, { topK: 3, fingerprint: FP, dimension: DIM, vectorScores: scores, minScore: 0.15 })
const accounted = hits.length
+ hits.skippedScore
+ hits.skippedNoEvidence
+ hits.skippedUnscoreable
+ hits.truncated
+ hits.skipped
+ hits.skippedDim
assert.equal(
accounted,
6,
`D2: every candidate must land in exactly one bucket; ${accounted} of 6 accounted for. `
+ 'A shortfall means a new silent drop path was added without a counter.',
)
} finally {
store.close()
}
})
// ── D2 的第二个洞:在候选池存在之前就消失的块 ──────────────────────────────
//
// 07f8438 给三个「门」加了计数,但候选池是三处之外还有一段:`scoreVectorCandidates`
// 里算不出分数的行直接 continue —— 坏 JSON、宽度不符、非有限分量、零模,以及
// C4 之后新增的一种:**中心化把向量抹成尘埃**。这些块压根不进 vectorScores,
// 于是 recallBlocks 装配候选池时(`if (!symbolMatch && !vectorSource) continue`)
// 也看不到它们。
//
// 实测(3 条完全相同的块 + 中心化生效):
// hits=0 skipped=0 skippedDim=0 skippedScore=0 skippedNoEvidence=0 truncated=0
// —— 整个空间凭空消失,六个计数器全读 0。这正是 D2 自己禁止的静默分支,
// 只是位置比当初看的更深一层。
//
// 这条判据同时钉住两件事:计数非零,以及「命中 + 全部计数」仍能对上候选池。
test('D2: a vector that centring collapses is counted, not silently lost', async () => {
const DIM = 4
const FP = 'd2:unscoreable'
const unit = (v) => { const n = Math.hypot(...v); return n === 0 ? v : v.map((x) => x / n) }
const blockVec = unit([1, 0.2, 0.1, 0])
const queryVec = unit([1, 0.9, 0.1, 0])
const store = new BlockStore({ path: ':memory:' }).open()
try {
for (let i = 0; i < 3; i += 1) {
store.putBlock({ text: `重复块 ${i}`, vector: blockVec, fingerprint: FP })
}
const built = store.rebuildCentroid(FP)
assert.ok(built.meanLength > 0.999, `precondition: identical blocks give a unit mean; got ${built.meanLength}`)
assert.equal(built.isAnomalous, true, 'precondition: centring must actually engage')
const embedder = {
async embedText() { return queryVec },
info: () => ({ fingerprint: FP, dimension: DIM }),
}
const hits = await recallBlocksHybrid(store, embedder, 'zzz-no-symbol-match', { topK: 10, minScore: 0 })
assert.equal(hits.length, 0, 'precondition: every block is centred onto the mean and dropped')
assert.equal(
hits.skippedUnscoreable,
3,
'D2: these blocks were dropped before the candidate pool existed, so no other '
+ 'counter can see them. Reporting 0 here is what made a whole space vanish '
+ 'while the log read "skipped=0".',
)
const accounted = hits.length + hits.skipped + hits.skippedDim + hits.skippedScore
+ hits.skippedNoEvidence + hits.skippedUnscoreable + hits.truncated
assert.equal(
accounted,
3,
`D2: hits + every counter must equal the space's blocks; ${accounted} of 3.`,
)
} finally {
store.close()
}
})
// ── D1: 召回主路径不得因向量侧故障抛错 ──────────────────────────────────────
//
// 实测出来的真缺口:把「损坏向量」的处理从 skip 改成 throw 之后,
// phase-b-h 137 条、phase-b-a 38 条、recall-accounting 5 条 —— 145 条**全绿**。
// 也就是说「损坏的向量载荷不能把整轮召回打挂」这件事,此前没有任何判据守着。
//
// (父早期的覆盖率矩阵按条目 ID grep 测试名,把 C6/W1/F1 也标成零覆盖;实测
// 那些其实有判据,只是没按 ID 命名。D1 是四项里唯一真的零覆盖。)
//
// 决策 §0.4 的原话是「向量侧的故障不得让主路径抛错」:一块脏数据不能把整轮
// 记忆检索变成异常,模型那一侧看到的就是完全失忆。所以这里直接往库里塞一行
// 坏 payload,然后要求召回照常返回。
test('D1: a corrupt vector payload cannot take the recall path down', async () => {
const DIM = 4
const FP = 'd1:corrupt'
const unit = (v) => { const n = Math.hypot(...v); return n === 0 ? v : v.map((x) => x / n) }
const store = new BlockStore({ path: ':memory:' }).open()
try {
const healthy = store.putBlock({ text: '值班室的分机号是 4324', vector: unit([1, 0.1, 0, 0]), fingerprint: FP })
const broken = store.putBlock({ text: '一条被写坏向量的块', vector: unit([1, 0.2, 0, 0]), fingerprint: FP })
// ★ Two guards narrow what "corrupt" can even mean here, both measured:
// · the DDL enforces `json_valid(vector)` — a syntactically broken payload is
// refused outright (`ERR_SQLITE_ERROR: malformed JSON`), so it cannot reach
// the read path;
// · a trigger re-derives `vector_dim` from the payload, so a value of the wrong
// LENGTH is filtered out in SQL before scoring ever sees it.
// What survives both guards and still cannot be scored is an array of the right
// length whose elements are not numbers — exactly what a provider that switched
// its encoding (or a migration that stringified values) would leave behind.
store.raw().prepare('UPDATE memory_blocks SET vector = ? WHERE id = ?')
.run('["a","b","c","d"]', broken.id)
const embedder = {
async embedText() { return unit([1, 0.15, 0, 0]) },
info: () => ({ fingerprint: FP, dimension: DIM }),
}
let hits
try {
hits = await recallBlocksHybrid(store, embedder, '分机号是多少', { topK: 5 })
} catch (error) {
assert.fail(
'D1: a single corrupt vector payload threw out of the recall path '
+ `(${error.message}). Decision §0.4: the vector side failing must not take `
+ 'the main path down — one bad row would otherwise cost the whole turn its memory.',
)
}
assert.ok(
hits.some((hit) => hit.id === healthy.id),
'D1: the healthy block must still be recalled while a corrupt row sits next to it',
)
assert.equal(
hits.skippedUnscoreable,
1,
'D1/D2: the corrupt row is skipped AND counted — silent skipping is the other half '
+ 'of this defect.',
)
} finally {
store.close()
}
})
// ── 窄读等价:候选池的来源换了一条 SQL,但看到的东西必须一样 ─────────────────
//
// `recallBlocks` 装配候选池时要把「符号命中」和「向量候选」并起来,为此它要遍历
// 全表。原来用 `allBlocks(-1)`(`SELECT *`),实测在 8000 块 × 512 维时把那 79MB
// 向量文本整份物化了一遍 —— 而它只拿向量列判断「这块有没有向量」。
//
// 换成 `symbolCandidates()` 后,8000 块的召回从 624.0ms 降到 409.6ms(-34%)。
// 性能改动最容易悄悄改变语义,所以这里钉住等价本身:
// 1. 行集与顺序完全一致(同一条 ORDER BY)
// 2. 每一条的 has_vector 与旧谓词 `String(vector ?? '') !== ''` 逐条相同
//
// ★ 有一个已知例外,不在这里断言:字面量字符串 'null' 在旧谓词下算「有向量」,
// 而 `vectorCandidates` 把它当「没有」。这个分歧早于本次改动,不在本判据里
// 悄悄改掉 —— 改它就是另一件事,要有自己的判据。
test('narrow read: symbolCandidates() sees exactly what allBlocks(-1) saw', () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
const fp = 'narrow:equiv'
const withVector = store.putBlock({ text: '有向量的块 4324', vector: [1, 0, 0, 0], fingerprint: fp })
const noVector = store.putBlock({ text: '没有向量的块 4324' })
const emptyVector = store.putBlock({ text: '向量为空串的块 4324', vector: [0.5, 0.5, 0, 0], fingerprint: fp })
store.raw().prepare("UPDATE memory_blocks SET vector = '' WHERE id = ?").run(emptyVector.id)
const wide = store.allBlocks(-1)
const narrow = store.symbolCandidates()
assert.deepEqual(
narrow.map((row) => row.id),
wide.map((row) => row.id),
'the narrow read must return the same rows in the same order — the pool is a Map, '
+ 'and insertion order is what the caller iterates',
)
const wideById = new Map(wide.map((row) => [row.id, row]))
for (const row of narrow) {
const old = wideById.get(row.id)
assert.equal(
row.has_vector,
String(old.vector ?? '') !== '' ? 1 : 0,
`has_vector must reproduce the predicate it replaced, for block ${row.id}`,
)
}
assert.equal(narrow.find((r) => r.id === withVector.id).has_vector, 1, 'a real vector counts')
assert.equal(narrow.find((r) => r.id === noVector.id).has_vector, 0, 'no vector does not')
assert.equal(narrow.find((r) => r.id === emptyVector.id).has_vector, 0, 'an empty payload does not')
} finally {
store.close()
}
})