`package.json` declared
"files": ["lib/*.js", "lib/loop.js", "lib/loop/*.js", "lib/types/*.d.ts"]
`lib/*.js` matches ONE level. `lib/embed/http.js` therefore never entered the
tarball, while `lib/index.js:35` imports it:
import { createHttpEmbedder } from './embed/http.js'
## How this survived
Every test in this repo runs from the source tree, where the file obviously
exists. `npm pack --dry-run` printed "10 files" — a plausible number nobody had
reason to read through. The defect is only reachable from the one path that
ships: installing the tarball.
Measured, extracting the tarball and importing it:
IMPORT FAILED: ERR_MODULE_NOT_FOUND
Cannot find module '.../package/lib/embed/http.js'
imported from '.../package/lib/index.js'
After the fix, the same command:
IMPORT OK, exports: StableMem,apply,default,defaultPath,inject,instanceKey,lastUserText,name
## Fix
`lib/embed/*.js` added to the allowlist. The tarball goes from 10 to 11 files
(86.1 kB) and `lib/embed/http.js` is in it.
## Criterion
`test/packaging.test.js`:
1. every `.js` under `lib/` must be matched by at least one `files` pattern.
The patterns compile to regexes the way npm treats them (`*` does not cross
`/`), so the criterion needs no hand-maintained file list and cannot drift.
2. every relative import in `lib/index.js` must resolve on disk — otherwise
criterion 1 could pass vacuously for a file that does not exist.
Reverting `files` to the old list turns criterion 1 red:
not ok 1 - packaging: every file under lib/ is covered by package.json "files"
these files live under lib/ but no "files" pattern reaches them ...
lib/embed/http.js
## Why it belongs in this branch
V6 made the embedder assembly load-bearing (two instances, one per side). A
release that cannot import `lib/embed/http.js` has no embedder at all, so the
distinction V6 introduced would never reach a user. The bug predates this
branch; its blast radius is the shipped artifact.
Gates: npm test 294/294 (0 skipped), npm run check 12/12, npm pack 11 files.
The HTTP protocol carries a `side` field — `query` or `document`, HA's
`PurposeQuery`/`PurposeDocument` (`http_embedder.go:105` / `:109`). The port made
it a construction-time setting of the adapter (default `query`) and then **no
call site ever supplied it**: both the write path (`lib/index.js`) and the recall
path (`lib/recall.js`) took the default.
So every stored block was encoded as if it were a question. Nothing fails when
that happens — both sides produce unit vectors of the declared width — which is
exactly why it survived: the corruption is only visible in retrieval quality,
and retrieval quality was already the known-unmeasured part.
## Why two instances and not a per-call argument
`side` is a property of what you are encoding, so a per-call parameter is the
more natural shape. It is not the shape used here, for one reason: the adapter
travels `side` verbatim in the request body from a closure, and giving it a
second, per-call way to receive the same field would leave two sources of truth
for one wire value. Creating two adapters over the same endpoint/model/dimension
— identical fingerprint, differing only in `side` — keeps the wire contract as it
was and puts the distinction where it belongs: in assembly.
`lib/embed/http.js` is therefore **unchanged**.
An injected `config.embedder` (tests, a future local adapter) has no second
instance, so the write path falls back to it (`documentEmbedder ?? embedder`).
That is the previous behaviour, not a second code path — and it is pinned.
## Criterion
`test/phase-b-v6-sides.test.js` starts a real HTTP server that records the `side`
of every request, then drives the **production assembly** (`apply()` with
`config.embed`):
commit → request body side === 'document'
recall → request body side === 'query'
The assertion is on the field that actually went out on the wire, not on an
internal variable. A second test pins the injection fallback: a single injected
embedder must still serve both paths, or every existing test and every
single-sided adapter breaks.
Mutation M-V6 — pointing the write path back at `this.embedder`:
not ok 1 - V6: storing a block encodes it on the document side, asking encodes the query side
error: V6: a block being STORED must be encoded on the document side. ...
+ actual - expected
+ 'query'
- 'document'
## What this does NOT establish
That the provider actually encodes the two sides differently. `side` is a
request field; what the endpoint does with it is the endpoint's business, and
this repo has no model whose asymmetry is measurable. What is now true is that
the distinction reaches the wire instead of being silently defaulted away.
Gates: npm test 292/292 (0 skipped), npm run check 12/12. Frozen faces untouched.
`centroidStatus` computed four distinct ways for centring to be unavailable and
then flattened all of them into `available: false`. The result was that
`no_centroid` and `dim_mismatch` both looked like `stale: null`, and `isotropic`
was indistinguishable from a healthy space except by `available`. That answers
"may I centre?" and says nothing about "why is semantic recall never
correcting?" — four different operator actions (build a centroid, fix the
dimension, wait for a rebuild, accept that this space has no common direction)
collapsed into one symptom.
Every branch now carries `reason`:
ok available: true
no_centroid no row for this space at all
dim_mismatch a centroid exists but its width is not this query's
zero_sample a row exists but was built from zero vectors
stale usable in principle, no longer describes the space (drift > 20%)
isotropic permanent property of the space: no common direction to remove
`zero_sample` is a fifth cause beyond the four the plan names, because the Go
branch has it too (`centroid_store.go:179-185`) and it is not the same problem
as a missing row.
## Criterion
`test/phase-b-c5-reasons.test.js` — one fixture per reason, asserting **the
reason value itself** (not `available`, which is multi-valued). Plus:
- an invariant test: `available === (reason === 'ok')`, so `reason` cannot drift
into decoration that disagrees with the conclusion it explains;
- a test that `centroidRecallState` still projects `stale: null` (the scorer must
not start reacting to staleness) *while* `reason` travels through, so an
operator can still see why.
Every call injects `schedule: () => {}`. `centroidStatus` schedules a rebuild
when it judges a space stale — a criterion that calls it without isolating that
side effect ends up triggering the very behaviour it is inspecting. C4e's first
version proved that the hard way.
## Mutation
Collapsing all six reasons into one value (`reason: 'unavailable'`) turns all
eight criteria red:
not ok 1 - C5: a space with blocks but no centroid row reports no_centroid
not ok 2 - C5: a centroid of the wrong width reports dim_mismatch
not ok 3 - C5: a centroid row with zero samples reports zero_sample
not ok 4 - C5: a centroid that drifted past the ratio reports stale
not ok 5 - C5: an isotropic space reports isotropic, not stale
not ok 6 - C5: a healthy space reports ok, and only ok carries available=true
not ok 7 - C5: reason and available never disagree
not ok 8 - C5: centroidRecallState keeps its stale:null projection but carries the reason
## What this does NOT change
Scoring, gating and the recall path are untouched: nothing reads `reason` to
decide whether to centre, and `centroidRecallState` keeps `stale: null`. This is
a reporting seam, not a behaviour change — C4's own criteria (C4a/b/c/d/e/g/h/i)
and phase-b-h's C3 assertions stay green.
Gates: npm test 290/290 (0 skipped), npm run check 12/12. Frozen faces untouched.
`centerVector` treated any positive norm as a direction:
norm = Math.sqrt(norm)
if (!(norm > 0)) return null
for (...) out[index] /= norm
A block that IS the space mean differs from it only by floating-point residue
(~1e-17). That residue passes `norm > 0`, gets normalised to unit length, and
then scores against the query as if it were signal.
## Measured
Space of two identical blocks, query deliberately off the block direction (if
the query equalled the mean, its centred form would collapse and centring would
fall back to raw scores, hiding the defect):
raw cosine, no centroid 0.8608
with centroid, before this fix 0.2183 <- dust, renormalised
with centroid, after this fix no hit <- dust judged "no direction"
At the default `minScore = 0.5` the 0.2183 scores were filtered anyway, so the
user-visible outcome (0 hits) does not change. What changes is that the number
is no longer manufactured: with `minScore: 0` the earlier build would surface
those blocks at a noise-derived score, and a future change to the threshold
would have made that reachable.
CENTER_EPSILON = 1e-9 is the floor. It also matches the Go side, where the
cosine of a zero vector is 0 rather than a random angle.
## Criterion C4i — and what it deliberately does not pin
C4i asserts the invariant: **every returned hit must carry the raw cosine
(0.8608); a value near 0.22 means dust was renormalised.** It therefore accepts
either of two future policies — drop such blocks (what this commit does), or
skip centring entirely for a space with no variance. The only thing it forbids
is continuing to score dust.
Reverting the guard (`norm > CENTER_EPSILON` -> `norm > 0`) makes C4i red, which
is the mutation this criterion exists for.
## KNOWN LIMITATION — reported upward, not fixed here
A space whose blocks are all identical now returns NO hits where it previously
returned them at 0.8608, because centring subtracts the only direction the space
has. That is not introduced by this commit — epsilon only made it deterministic —
but it is a real recall loss for a realistic case (the same note stored twice),
and it is a C4 regression against the pre-C4 behaviour.
Two candidates, deliberately NOT applied here because both change scoring
semantics and one departs from HA:
(a) keep dropping them (current, HA-consistent: Go's cosine of a zero vector
is 0, so those blocks fall below any threshold too);
(b) do not centre a space whose mean norm is ~1.0 — there is no variance to
remove — which preserves the 0.8608 raw scores. This adds an upper
threshold HA does not have, so it is the parent's call.
No criterion pins "0 hits" for the duplicate space: pinning it would encode the
limitation as intended behaviour.
Gates: npm test 282/282 (0 skipped), frozen faces untouched.
Three comments that were either wrong or silent. No behaviour change.
## The first-build cost claim was false
`#ensureCentroid` said "首次建的代价很小：那时块还少" (the first build is cheap —
there are few blocks then). That is exactly backwards for the situation the code
exists to fix: a production database that already holds thousands of vectored
blocks while `graph_centroid` is empty. The W8 scenario IS a database with many
blocks and no centroid.
Measured on a real database (512 dims, 2000 vectored blocks, empty centroid
table, first commit through the production path):
FIRST commit after upgrade 105.1 ms (synchronous SELECT + parse + mean)
STEADY commits 8.7-9.4 ms (loadCentroid SELECT + centroidStatus COUNT)
and the first build grows linearly with the space (~420 ms at 8000 blocks). The
number is now in the comment, together with the decision it justifies: keep the
one-time stall rather than move the build to the async path, because an async
first build leaves the first recalls after an upgrade silently uncorrected —
precisely when n is large enough for centring to matter. A bounded stall is
easier to notice than a silent degradation.
The steady-state ~9 ms is new with C4 and is now stated as such: every commit
pays one `loadCentroid` SELECT plus the `countVectored` COUNT inside
`centroidStatus`. C3-k forbids that COUNT on the recall path, not on the write
path.
## Two calls with no production caller, and why neither is a dangling wire
`saveSpaceCentroid` and `#centroidRebuiltAt` both looked like the defects this
round kept finding (code written, never wired). They are not, but leaving that
unwritten meant the next reader had to re-derive it — and the two live cases
(`saveSpaceCentroid` in `lib/`, the throttle state on disk) really were dangling.
- `saveSpaceCentroid` is the public programmatic seam for a caller that computed
a centroid elsewhere. Production centroids are derived from the blocks by
`store.rebuildCentroid`, driven from `commit` via `#ensureCentroid`. Stated in
its JSDoc, along with the fact that the only in-repo caller is a test.
- `#centroidRebuiltAt` stays in memory, and the reason `graph_centroid.built_at`
cannot replace it is now written down: `built_at` records a BUILD, the throttle
needs the last SCHEDULE, and the two diverge precisely when a rebuild throws.
Throttling on `built_at` would re-schedule on every commit after a failed
rebuild — write amplification on the failure path. The cost of the in-memory
map (one extra bounded async rebuild after a restart, measured 85.5 ms at
2000x512) is smaller, and it is stated next to it.
Gates: npm test 281/281, npm run check 12/12. Frozen faces untouched.
Three of C4's control surfaces had no criterion at all: deleting them left every
suite green. This commit adds the criteria, and the two defects they exposed.
## What was unguarded, measured
Each mutation below was applied to a scratch copy of lib/ and run against the
suite as it stood at 05e93b2.
1. The `isAnomalous` gate in `scoreVectorCandidates`
`centroid?.isAnomalous === true &&` -> `centroid !== undefined &&`
Result: c4 6/6 green, phase-b-h 137/137 green, phase-b-a 38/38 green.
Why: every C4 fixture is anisotropic (two of them assert isAnomalous ===
true), and C4c is "no centroid row", which is a different branch from "a
centroid that was judged isotropic". The only isotropic assertion in the repo
(phase-b-h:1133) checks `rebuildCentroid`'s verdict, not the scoring path.
2. `countVectored` on the recall hot path
`store.countVectored(fingerprint)` inserted before the centroid read.
Result: c4 6/6 green, phase-b-h + phase-b-a 175/175 green.
Why: nothing scans lib/recall.js for the C3-k boundary. phase-b-a:67 does read
recall.js, but it asserts the F2 fingerprint gate.
3. MIN_SAMPLE (this one is a defect, not just a missing criterion)
A one-vector mean IS that vector, so its norm is exactly 1.0 and it clears
0.6 by construction. Measured with a real clock and the real commit path on
the C4 corpus: right after the 8th commit the centroid reported
`vectorCount=1 meanLength=1.0000`, and a recall that returns 3 rows without a
centroid returned 1 with it — the block that formed the "mean" was centred to
the zero vector and dropped. The drift rebuild corrects it ~2s later, so the
window is short but the state is wrong, and in a slowly growing space it lasts
until the throttle expires.
## What this commit adds
- `BlockStore.MIN_SAMPLE` (= 2) and `vectors.length >= MIN_SAMPLE` in the
anisotropy verdict. Two is the smallest number for which the mean is a property
of the set rather than of one member.
- **C4h** pins both halves of that: the verdict (`isAnomalous === false` for one
vector, with `meanLength === 1` asserted as the precondition that makes the
0.6 test insufficient) and the consequence (the only block stays recallable).
- **C4g** pins the isotropic gate. Fixture numbers come from running the same
probe against a gated and an ungated build, not from paper: 8 orthogonal
one-hot vectors, `meanLength = 0.353553`, `isAnomalous === false`; a query
along `unit(ones)` yields 8 hits of cosine 0.3536 with the gate, and 0 hits
without it (every centred cosine turns negative and V5's structural gate drops
the lot). A three-axis query additionally pins 0.8111 / 0.4867 / 0.3244.
The identical-pair query is deliberately NOT the main assertion: with both
sides centred its cosine is 1 either way, so it cannot discriminate.
- **C3-k** statically scans lib/recall.js for `countVectored(` / `centroidStatus(`
after stripping comments, and carries a positive self-check: the same scan must
find `centroidStatus(` in lib/index.js, so a broken regex cannot leave the two
"must not contain" assertions vacuously true.
- **C4d** gained the assertion it was missing. It asserted that a centroid
exists, is anomalous and has the right width — all true of a centroid built
from a single block. It now also asserts `vectorCount === CORPUS.length`, and
waits for the drift rebuild before asserting on contents. Without the count
assertion C4d is green while the mean describes one block.
## Mutations verified (each red in a scratch copy, working tree untouched)
delete isAnomalous gate -> C4g red
countVectored in recall.js -> C3-k red
drop MIN_SAMPLE -> C4h red
revert "refresh a drifted centroid" -> C4d + C4e red
## Documentation that had drifted from the code
- recall.js still said "No centroid centering here. C4 ... is a separate slice
layered on top of this one" directly above the function that now implements
centering. Replaced with a pointer to where it actually lives.
- The one-sided-centring cosine was recorded as 0.3787 in the comment and "near
0.38" in C4b's message. Measured on C4b's own five-block fixture it is 0.1897
(cosine is symmetric, so centring either single side gives the same value).
Both now say 0.1897 and name the fixture.
## Measured costs (for the write path this slice touches)
Pre-populated space, 512 dims, 2000 vectored blocks, empty graph_centroid — the
state a production database is in when it upgrades:
FIRST commit after upgrade 105.1 ms (synchronous O(n*d) rebuild)
STEADY commits 8.7-9.4 ms (loadCentroid SELECT + centroidStatus COUNT)
The first build is accepted as-is: it happens once per space lifetime, it is
bounded, and moving it to the async path would leave the first recalls after an
upgrade silently uncorrected — a bounded stall is easier to notice than a silent
degradation. The steady-state COUNT is new with C4; C3-k keeps it off the recall
path, where it would run every turn.
## Real path (dsh headless, overlay on the real profile)
Same corpus and query, only the centroid row differs:
centroid present (count=8, meanLength=0.972934)
[1] (text 0.63) 第七号仓库的备用门禁码是 QP47-812。
centroid deleted
[1] (text 0.69) 这段时间一切照旧，没有什么特别的事情。
[2] (text 0.69) 第七号仓库的备用门禁码是 QP47-812。
top-1 moves from the hub to the target, which is the behaviour C4 exists for.
## What this does NOT establish
Centring changes angles; it does not calibrate absolute scores. DEFAULT_MIN_SCORE
= 0.5 was never derived from a centred distribution, and the uncalibrated-absolute-
score problem recorded in phase-b-p1-vector-wiring.test.js:51-58 remains open.
A green C4 means "centring is wired into scoring and changes the ranking in an
anisotropic space", never "semantic recall is good now".
C5 (distinguishing no_centroid / dim_mismatch / stale / isotropic reasons) is
untouched: `centroidRecallState` still returns `stale: null`.
Gates: npm test 281/281 (0 skipped), npm run check 12/12, npm pack 10 files.
Frozen faces untouched: docs/, README.md, scripts/, lib/loop*.
Two discipline holes left after 1309afe / 09be4de, both closed here.
1. The boundary was unstated. A one-hot keyword-family fixture proves the
PLUMBING — that the computed query vector reaches scoring — and says nothing
about retrieval quality. Left implicit, a green P1 invites the reading
"semantic recall works". The measured counter-evidence is now in the file:
on the real chinese-clip space a true hit scored 0.8345 while an unrelated
block scored 0.8598, so absolute scores are not calibrated and no threshold
may be read off them. That is a known unresolved problem outside P1's scope.
2. `dsh` missing used to `t.skip`. A skip keeps `npm test` green while the only
end-to-end evidence for P1 never runs — the same shape as a criterion that is
committed but never speaks. It now fails, and says why.
No behavioural change: 272/272 tests, 12/12 checkpoints, 0 skipped on a box
where `dsh` is present. Verified the mutation discipline still bites — deleting
`'hybrid'` from ORDINARY_DEGRADATIONS turns L10, L11, L10† and L11† red.
P1 registered two healthy modes in `ORDINARY_DEGRADATIONS` — `hybrid` (the wired
success path) and `no_vector_space` (a space with nothing comparable to score) —
and the commit message claimed that meant "a healthy wired recall does not log a
warning every turn". Nothing tested it. Measured on 1309afe:
M-a delete 'hybrid' → # tests 268 # pass 268 # fail 0
M-b delete 'no_vector_space' → # tests 268 # pass 268 # fail 0
M-ab delete both → # tests 268 # pass 268 # fail 0
while the behaviour really broke (real apply(), debugLog=false):
mutated hybrid recall → lines=[['warn', '... degraded=hybrid ... inject=138ch']]
released hybrid recall → lines=[]
A false fault alarm on the happiest path, every single turn, and the suite green.
Why the existing tests could not see it: A3d/D3b assert the LABEL
(`hits.degraded === 'hybrid'`). Removing a member from the Set does not change any
label — it changes only whether `logRecallTurn` treats that label as a fault. The
defect lived exactly in the gap between "the label is right" and "the level is
right", and `grep ORDINARY_DEGRADATIONS test/` was empty. A claim in a commit
message with nothing behind it is the same failure mode as a guard that cannot
fail.
What this commit adds (test/memory-log.test.js only — lib/ untouched)
L10 one group, three outcomes at the SAME verbosity (debugLog: false):
`hybrid` → zero lines
`no_vector_space` → zero lines
`embed_error` → exactly one line, type 'warn' ← the control
Each case asserts its own PRECONDITION first (the run really was
`hybrid` / `no_vector_space`, and something was really injected), so a
silent log cannot be confused with a build that never reached the route.
Without the third case, "silent" would also be satisfied by a build that
logs nothing at all — which is the original bug the log channel exists
for (L4).
L10† real mutation: 'hybrid' removed from the Set → L10.1 red
L11† real mutation: 'no_vector_space' removed → L10.2 red
Both via `withMutatedIndex` (copy the package, rewrite ONE anchor,
`count === 1` self-check, import fresh). The mutant runs the SAME
`assertLevelContract` body the released build passes — a criterion copied
for the mutant can drift, a shared one cannot. The mutation then asserts
the throw is `ERR_ASSERTION` AND that the message starts with the specific
sub-assertion name, so an import failure cannot make it green for the
wrong reason.
L11 measured behaviour, pinned so nobody "fixes" it later: a fused recall
whose candidates were ALL refused by `minScore` still reports `hybrid`
with zero hits, and is still silent. `no_vector_space` is returned at
exactly one place (`recall.js`, `vectorScores.size === 0`) and means
"there was nothing to score" — never "everything was filtered". Collapsing
the two would hide a threshold set too high behind a "fresh space" story.
Two fixture pitfalls found while writing this, both recorded in the test because
each one silently disarmed its own case:
· `commit` embeds when an embedder is mounted, so the `no_vector_space` case has
to write its block BEFORE the embedder appears. The first attempt mounted the
embedder first, the space filled up, and the label came back `hybrid` — the
assertion then failed for its own precondition rather than for the subject.
· `StableMem.recall()` forwards only `topK`/`signal`, so a `minScore` passed
there never reaches the scorer and L11's precondition would have passed on an
unchanged result. It drives `recallBlocksHybrid` directly instead.
Not asserted, deliberately: `'no_embedder'`, `'aborted'` and `'vector_ready_unused'`
are also in the Set, but their silence is already carried by L1/L5 (no embedder)
and W11e/W12 elsewhere. Adding them here would inflate the group without covering a
new decision.
旁证 (not acceptance): npm test 272/272, npm run check 12/12, npm pack --dry-run
10 files / 75.3 kB. docs/ and README.md untouched.
`recallBlocksHybrid` computed `queryVector`, verified its length against the
declared space, and then threw it away:
return withDiagnosis(recallBlocks(store, query, scoped), 'vector_ready_unused')
`scoped` carries only `topK` / `signal` / `fingerprint` / `dimension`. So every
V1–V7 scoring branch inside `recallBlocks` was live code reachable ONLY from
unit tests that hand-built a `vectorScores` Map — `vectorScores:` appeared 10
times in test/phase-b-a.test.js and ZERO times under lib/. Production recall was
pure symbol/FTS: the whole vector half of phase B was unreachable, and the only
symptom was a diagnostic label nobody was watching.
Why BOTH options are now passed (structural, not stylistic)
· `vectorScores` alone reaches only the V1 exact-band tie adjustment (:438).
The fused candidate assembly at :233 is gated on `Number.isFinite(minScore)`,
so without a threshold the vector route can never admit a block the symbol
path missed — which is exactly what V5 exists for.
· `minScore` alone does nothing: :299 additionally requires `vectorScores` to
BE a Map before it builds vector candidates.
One seam, three guards. Half of it is a no-op, which is why measuring the
half-wiring mutation below matters.
Scheme A, and why not B
A: keep the existing API (`vectorScores: Map<string, number>` + `minScore`), and
have `recallBlocksHybrid` score the query vector against the space's stored
vectors before calling `recallBlocks` once.
B: change `recallBlocks`'s signature to accept a `queryVector` and score inside.
Chosen: A. B would touch a signature that phase A's golden fixture and nine call
sites depend on being synchronous and vector-free (`recallBlocks` MUST NOT return
a thenable — A1 pins it structurally), and it would move cosine arithmetic into a
function whose contract is literally "store + query text". A leaves
`recallBlocks` byte-untouched: the seam it already exposes is driven, so A2's
45-case golden comparison cannot drift. Both routes must also be fused in ONE
scoring pass anyway (V5 admits vector-only candidates; V7 forbids truncating
before fusion), so "call twice and merge" was never an option.
Cost, measured before choosing (this box, 2000 blocks x 512 dims)
narrow `SELECT id, vector` 21 ms (vs 70 ms for `allBlocks(-1)`)
parse + cosine over 2000 60 ms
⇒ ~81 ms per recall; at 8000 blocks ~310 ms.
Affordable because it runs once per TURN (pre-step), not per candidate, and
because the scan is bounded by SQL: `fingerprint = ?` and `vector_dim = ?` are
pushed into the store so rows from a retired space or a wrong width are never
decoded. A phase-A store (no embedded blocks at all) pays one indexed SELECT
returning no rows. It does not add a new order of cost either: the fused path
already walks every block for symbol union (`allBlocks(-1)`); this replaces that
blind decode with a narrow one for the vector half. The remaining ceiling is a
single very large space — a number to measure before raising topK, not a reason
to ship a path that never runs. Fail-closed per §0.2/F5: blocks with an empty
fingerprint are deliberately not scored.
New diagnostic modes, both non-faults
`hybrid` the healthy wired path (symbol + vector fused). Explicit
`case` in `degradationNotice` because `default:` exists to
make an UNKNOWN mode say "unknown, not absent" — routing the
healthiest mode through it would print a false degradation
notice on every turn that had a vector hit.
`no_vector_space` a space is configured and the embedder answered, but no
block in it carries a usable vector (fresh space). Says
"nothing was scored" without claiming a provider failure.
An empty Map must NOT be passed to the fused route: it would
open candidate assembly and score symbol matches as
`0.7*0 + 0.3*coverage`, the V2 regression.
The criterion (test/phase-b-p1-vector-wiring.test.js)
End-to-end, because a unit test cannot see this bug: the defect is that nobody
passes the options, so asserting the options work proves nothing. The chain is
real sqlite file → real `apply()`/`commit()` write path (no hand-built vectors,
no injected embedder object: the plugin's own HTTP adapter is used) → real
`recallBlocksHybrid` → real headless `dsh --profile headless` with the shipped
`cordis.patch.yml` loop overlay (so P0 still applies) → the persisted
`session.v4.jsonl.zstd` → the stub LLM's answer, which is a pure function of the
system prompt it received.
Corpus, MEASURED before any assertion was written. Deterministic keyword-family
projection into 8 unit axes: family 0 is the target's vocabulary, families 1..7
are one axis per distractor, unmapped text → zero vector. Query
「上次说的那个四位数是什么来着？」:
symbol path hits (target) 0 ← precondition 1
hand-built vectorScores top1 = target, score 0.70 ← precondition 2
query shares no CJK 2-gram with any block
Both preconditions are asserted in-suite (P1-0) so a later corpus edit cannot
make the e2e criterion unfalsifiable. An earlier attempt used the real
chinese-clip-vit-b16 model; its top-1 for the same kind of query was an unrelated
block (0.83 "入职培训"), so that corpus could never have gone green — the fake
embedder makes the question decidable instead of probabilistic.
The access code is `QP47-812` rather than a bare number so the stub's extractor
(`[A-Z]{2}\d{2}-\d{3}`) cannot collide with anything else in the prompt.
Born red against c1f7e06 (clean `git archive` of that commit, same test file):
P1: the answer must come from block memory through the vector route.
expected: QP47-812
actual: "未知"
symbol-path hits: 0 (must be 0)
recall diagnosis: degraded=vector_ready_unused hits=0
recall section injected: (none)
Mutations — three, each a real edit to lib/recall.js, each restored after:
M1 drop the wiring `recallBlocks(store, query, scoped)` → P1 red
M2 half the seam `minScore = undefined` → P1 red
M3 empty the scan `const rows = []` → A3d red
M1 raw: not ok 2 - P1: a semantic-only question is answered from block
memory via the vector route
expected: QP47-812
actual: "未知"
recall diagnosis: degraded=hybrid hits=0
M2 raw: identical text — and that is the point. Passing `vectorScores` alone
keeps the new `hybrid` label looking healthy while the answer is still
empty, because `minScore` is what opens the fused candidate assembly.
M3 raw: Expected values to be strictly equal — `hybrid` unreachable, so the
label is evidence rather than decoration.
Restoration is verified by hash, not by eye: `sha256sum lib/recall.js` is
8eb0faa660c973ea5114268001d49cf43f83ef92b2945897cfc6f7512541a498 before and
after every mutation, and `git diff` (staged vs worktree) is empty.
Test updates forced by the wiring (not cosmetic)
A3c the old assertion was "a healthy embedder reports vector_ready_unused,
NOT hybrid". That was correct while the vector was unused; over a corpus
with no vectors in the active space the honest label is now
`no_vector_space`, and A3d pins `hybrid` once a vector exists.
A3c† replaced a test that only compared two local string literals (it could
never fail for a real reason) with a mutation of the actual scoring pass.
D3b `hybrid` instead of the group-A stage label.
index the two new modes are added to ORDINARY_DEGRADATIONS, so a healthy wired
recall does not log a warning every turn.
Not in this slice: C4 (centering) is the next layer on this exact path and
deliberately not done inline, so its mutation stays citable. V6 and C5 untouched.
旁证 (not acceptance): npm test 268/268, npm run check 12/12.
`promoteSnapshotSections` accepted only ONE carrier spelling: `content: []`.
The plugin never emits that shape. `lib/index.js:932` always attaches exactly
one `{ type: 'text', text: injected }` block whose text IS `sections[0].text`,
so `promotedSection`'s `if (blocks.length > 0) return null` matched on every
real recall and the promotion was a no-op in production. Recall stayed in
user/message history — the exact defect 6d9ad62 claimed to fix.
6d9ad62's "surface recall as a system section instead of history" was never
true: the promotion it relies on was blocked by that content-non-empty check.
The unit tests could not see it because their `snapshotMessage()` helper built
`content: []` — a shape no producer emits — and the old P4 asserted "real
content blocks are NOT promoted", which is the opposite of reality.
It was confirmed today, from `lib/index.js:932`'s constant one-text-block
injection, and is fixed here.
The rule is now "move the text, don't rewrite it": a snapshot carrier with
content is promoted only when the rendered blocks equal the sections' text
under one of the two canonical joins (`''` glued, `'\n\n'` spaced), and the
promoted section text is the rendered content, so relocation is assertable.
Non-text blocks, mismatched text, and the stock runtime-context snapshot (whose
content carries a preamble the sections do not) pass through untouched.
Tests: S1-S7 plus real multi-block S2c/S2d pin the acceptance surface; every
negative case carries a positive control in the same decision, so the old
"content non-empty => reject" rule cannot satisfy it. P4 is split into P4a/b
(content == sections promotes; content != sections does not) with P4c proving
the split is decided by text, not by presence of content.
`test/loop-system-e2e.test.js` mounts the real HologramAgentLoop on real dsh
services, runs one turn, and asserts on the request the adapter received: the
recall is in the system prompt (E1) and in no non-system message body (E2),
carried by exactly one message (E2b).
The four files under lib/loop/ were written against 0.2.1-rc.2 and have been
verified to import on their own; they are the substrate a replacement loop
needs to take over step(). This commit only puts them in the tree — lib/loop.js
is untouched, so nothing imports them yet and behaviour is unchanged.
Two packaging facts they bring with them:
files `lib/*.js` does not recurse, so lib/loop/ was shipping nothing.
`lib/loop.js` and `lib/loop/*.js` are now listed separately because
neither glob covers the other.
peer runtime-context.js imports @deepseek-ai/dsh-session, which the peer
table never declared. G2b's allowlist covered it, so the static guard
passed over a real declaration gap.
test/loop-ports.test.js pins the port interface rather than its behaviour: the
modules import alone, the export names survive, and the inbox projection keeps
key "inbox" / stateVersion 1. That literal pair is a wire format shared with
stock-loop sessions — a different key silently reads as "no pending input", a
different version makes stateSchema.parse refuse stored rows. The schema's
parse() is also exercised against seven malformed persisted rows, since
replacing a zod schema with a hand-written check is exactly the kind of change
that can be a no-op and still look fine.
6d9ad62 added `lib/loop.js` — a subclass of the harness `AgentLoop` that moves
per-turn recall out of conversation history and into the system prompt. That
class was never instantiated by anything: the commit proved the pure
`promoteSnapshotSections` function, not that a replacement loop can actually be
mounted. This commit adds the two pieces the mount needs.
`cordis.patch.yml` is the overlay that does it, and it is the only artefact that
proves the swap is possible:
- `- id: agent-loop / disabled: true` is mandatory and is addressed by ID, not
by name. `AgentLoop`'s constructor ends with
`ctx.effect(() => ctx.agents.setFactory(this))`
(dsh-agent-loop/lib/index.js:1590) and `setFactory` throws when a factory is
already registered (dsh-agent/lib/index.js:429-437). Leaving the base row
enabled fails startup with "an agent factory is already registered".
- The inserted row reuses the package itself under the `dsh-memory/loop`
subpath, with no `config` block: `HologramAgentLoop` declares no `static
Config`, so it inherits the parent schema and the profile's own `agent-loop`
config rows keep working through the inherited constructor.
`package.json` gains that subpath export plus `lib/loop.js` in `files`. Without
the export the overlay resolves to nothing and the profile cannot boot — the
two changes only work together, which is why they land in one commit.
Measured on this machine with the overlay applied to a headless profile
(`~/.dsh/profiles/headless/cordis.patch.yml`), dsh 0.2.1-rc.2:
- `dsh --profile headless "回答两个字：收到了"` -> exit 0, stdout `收到了`;
`--dump-config` shows `- id: agent-loop` with `disabled: true` alongside
`- id: hologram-agent-loop, name: dsh-memory/loop`.
- A tool round trip returns the tool's output: asking for
`echo TOOL_OK_42` via bash yields `TOOL_OK_42`.
- SIGINT during a long task exits in 1s with code 130, stdout empty.
- The web profile's patch layer is still `[]` — untouched.
npm test: 247 pass, 0 fail.
Per-turn recall currently enters through the `agent/pre-step` waterfall, and
that hook can only return `messages`. The recall block therefore lands in
conversation history as a user message: the model reads its own memory as if a
user had said it. `systemPrompt.context` cannot fix this — it runs before the
turn text is known, so it can only render input-independent text.
The loop's `preStep` is the first seam holding BOTH the assembled prompt and
this turn's claimed messages. `lib/loop.js` subclasses the harness AgentLoop and
wraps that seam on the agent instance: a pure snapshot carrier message is moved
out of `messages` and into `assembly.sections`, so it becomes part of the system
prompt for that turn and never enters history.
Deliberately decoupled from the memory plugin: nothing imports it or matches its
name. Promotion keys off the shape every snapshot-style injector already uses
(`source.sections`, the same shape `dsh-time-context` emits).
Constraints honoured, each of which was a measured failure elsewhere:
- The promoted section carries `interpolate: false`; recalled text is free text
and may contain `{{...}}`, which the strict interpolator rejects.
- The subclass defines no constructor and no static `inject`. The parent already
registers the turnBoundary + inbox projections, calls `setFactory(this)`, and
declares `{{provider}}`/`{{model}}`; re-registering any of them throws at
startup.
- Only the instance's `preStep` is patched, never the prototype, which other
agents share.
Also wires the new peer: `scripts/link-dsh.sh` gains dsh-agent/dsh-agent-loop
(without them the package cannot resolve at test time), package.json declares
dsh-agent-loop as a peer and adds the new test file, and G2b's allowlist admits
dsh-agent-loop as a first-party harness package, with the reason recorded.
npm test: 247 pass, 0 fail.
The plugin shipped with zero console.* and zero ctx.logger calls, so a
production recall failure left no trace: `degraded` is a return value the
caller sees once, and the injected text is only visible if the model quotes
it. Everything below is observability — no scoring, no DDL, no docs/README,
no new dependencies.
## The line
One summary per recall turn, covering the four fact groups asked for:
recall query=<n>ch hits=<n> topK=<n> zeroHit=yes|no
budget budget=<n> source=explicit|host|fallback kept=<n> singleHitOverBudget=yes|no
degrade degraded=<mode> skipped=<n> skippedDim=<n> err=<summary>
inject inject=<n>ch
`query` is LENGTH ONLY (code points, so CJK is not double-counted). The query
is the user's own message and a memory log is the last place it should be
copied; length beside `hits` is enough to tell "searched nothing" from
"searched the skill catalog".
`source` reads straight off segmentRecallBudget's own decision, and `kept` is
the post-slice count, so the line cannot disagree with what was injected.
`inject` is measured AFTER rendering, i.e. the exact string contextForStep
returns — not a re-derived estimate.
## Level policy (and why it is not always info)
· fault — degraded not in {ok,no_embedder,aborted,vector_ready_unused}
-> warn, emitted EVEN WITH debugLog OFF. A failed search is the per-turn
event where absence of evidence is not evidence of absence.
· zero injection -> debug (the commonest case on the shipping default must
cost nothing at normal verbosity, but still be one greppable line).
· otherwise -> info, only when debugLog is true.
`no_embedder` and `aborted` are deliberately NOT faults: the former is the
shipping configuration, the latter a deliberate stop. This is the same
judgement W11e already made for the model-facing notice — a constant warning
trains the reader to skip the channel.
## Logger source, and why ctx.logger alone was not enough
Source is ctx.logger('dsh-memory'); if the cordis build has no logger service
the code falls back to a console.warn/console.error shim (debug/info also go
to warn — stdout is never written, since `dsh --profile headless` prints the
ANSWER on stdout, so a line there is not noise but a corrupted answer).
★ Measured, and the reason this change installs its own sink: cordis ships
exactly ONE default exporter, an in-memory ring buffer
(cordis/lib/index.js:594-603), and no dsh bundle adds a terminal one. Calling
ctx.logger.info(...) in a real profile therefore writes to a buffer nobody
drains — the plugin would have looked instrumented while a headless run still
printed nothing. installStderrSink registers a scoped exporter on that same
service: it filters by message.name (the exporters map is process-wide, so
without the name check it would reprint every other plugin's warnings under
this prefix) and by severity, then renders with Logger.format and writes to
stderr. Disposal rides on ctx.effect inside LoggerService.exporter.
Two robustness details, both earned the hard way:
· the whole line travels as ONE `%s` argument — Logger.format expands
%s/%d/%o in the format string, so a provider error containing `%s` would
otherwise truncate the rest of the line (verified: "50% off %s" came out
"50% off undefined NaN");
· err is flattened to one line and cut at 80 chars — a log line must stay a
line;
· every logging path is wrapped, because a throwing logger would turn an
observability feature into a recall outage.
## Verification
test/memory-log.test.js (L1–L8b, 12 tests) pins: default is silent; the
summary carries all four fact groups in exactly ONE line; the query text
never appears; explicit/host/fallback source naming; a fault warns with
debugLog off; a zero-injection turn is debug not info; log-on equals log-off
for both the injected text and lastRecall; stdout is never written; the line
reaches stderr; foreign loggers are not reprinted. L4† is a real mutation —
it patches lib/index.js on a copy and asserts L4 goes red — because a guard
that cannot fail is not a guard (the W11d† lesson).
Self-tested in a real headless profile (dsh 0.2.1-rc.2, llmsproxy), same
question, only debugLog differing. Answer stayed "4471" on stdout in both:
debugLog=true dsh-memory: info: recall query=21ch hits=1 topK=3 zeroHit=no
| budget=838860 source=host kept=1 singleHitOverBudget=no
| degraded=no_embedder skipped=0 skippedDim=0 err=-
| inject=144ch
debugLog=false (no dsh-memory line; stderr carry only node's
ExperimentalWarning)
zero-hit turn dsh-memory: debug: recall ... zeroHit=yes ... inject=0ch
fault, OFF dsh-memory: warn: recall ... degraded=embed_error
err=fetch failed | inject=290ch <- speaks without opt-in
npm test: 237 tests, 237 pass, 0 fail (the W11d†ctx failure is gone — see
below). npm run check: 12/12 checkpoints.
## One test edit, forced by the caller change
W11d†ctx's mutation anchor was `const hits = await this.recall(query, { signal })`,
which no longer exists: contextForStep now rebinds as `const raw` because the
hits it renders come out of segmentRecallBudget (the preceding commit's work,
not this one). The anchor moves to the `raw` binding it would sit beside. The
mutation still flips W11d; only the anchor text changed.
A1: revert to the 06155d4 wording (recallBlocks stays synchronous, a new
async recallBlocksHybrid is added). The previous basis claimed the async
pre-step handler justified making recallBlocks async, which the code
contradicts: lib/recall.js:229 exports a sync recallBlocks with zero awaits,
and lib/recall.js:613 is the async wrapper.
G1: replace the stale count with the readings already present in the V7
evidence logs (/tmp/v7-order-20261101-173334-t1..t3.log = npm test,
225/225/0; c1..c3.log = npm run check, 12/12), noting that phase A is a
subset of the whole-suite run. No new test/check run was performed.
