jianf
5cfc982f39
07f8438 gave the three gates a counter and claimed "every drop is countable". A
probe against the invariant found a fourth drop site, one stage earlier than any
gate:
scoreVectorCandidates skips a row it cannot score
· unparseable JSON
· wrong width (CENTER width vs query)
· non-finite component
· zero norm
· centring collapsed it onto the space mean (C4's dust guard)
Measured on three identical blocks with centring active:
hits=0 skipped=0 skippedDim=0 skippedScore=0 skippedNoEvidence=0 truncated=0
accounted = 0 of 3 -> INVARIANT BROKEN
The whole space vanished while every counter read zero. Cause: a skipped row never
enters `vectorScores`, so `recallBlocks` never puts it in the candidate pool
(`if (!symbolMatch && !vectorSource) continue`), so no downstream counter can see
it. This is exactly the silent branch D2 forbids — deeper than where D2 first
looked, and I only found it because I ran the invariant against a centring case
instead of trusting the green suite.
## Fix
`skippedUnscoreable` threaded from `scoreVectorCandidates` (by reference, so the
early `no_vector_space` return can report it too) through `recallBlocks` into the
result, then into `last_recall` and the log line — the same three surfaces D2
already uses.
hits=0 skippedUnscoreable=3 ... accounted = 3 of 3 -> INVARIANT HOLDS
## Criterion
`test/recall-accounting.test.js` gains case 5: three identical blocks, centring
engaged, asserting the count is 3 (not 0) and that hits + all seven counters
still equals the space.
The existing invariant case now sums seven counters instead of six — a shortfall
is how this hole would reappear.
Mutation — deleting the dust increment:
not ok 5 - D2: a vector that centring collapses is counted, not silently lost
error: D2: these blocks were dropped before the candidate pool existed, so no
other counter can see them. ...
## Note on the label
These blocks are NOT counted as `skippedNoEvidence`: that would say "there was
nothing to score" when the truth may be "there was a vector and centring destroyed
it". Different remedies (re-embed vs. look at the centroid), so a separate
counter — the same reasoning that made C5 split its six reasons.
Gates: npm test 303/303 (0 skipped), npm run check 12/12.
1147 lines
50 KiB
JavaScript
1147 lines
50 KiB
JavaScript
/**
|
||
* dsh-memory — block memory for dsh, entry point.
|
||
*
|
||
* This is **phase A** of `docs/feasibility.md` §7.1: prove that a block can go
|
||
* into dsh, be recalled, and reach the prompt, with **no embedding provider**.
|
||
*
|
||
* ## The one thing to read before touching injection
|
||
*
|
||
* dsh has two obvious-looking injection points, and only one of them can reach
|
||
* the current turn's input.
|
||
*
|
||
* | hook | gets this turn's input? | durable? |
|
||
* |---|---|---|
|
||
* | `ctx.systemPrompt.context({ text })` | no — `AssembleContext` is `{scope, signal}` plus a merged-in `agent` | yes (appended as a runtime-context snapshot) |
|
||
* | `ctx.on('agent/pre-step')` | yes — receives `decision.messages` | yes (the loop appends whatever you return) |
|
||
*
|
||
* Verified in the harness, not assumed:
|
||
* · `dsh-system-prompt/lib/types/index.d.ts:37-44` — `AssembleContext` has only scope+signal
|
||
* · `dsh-agent-loop/lib/index.js:902-934` — `preStep` claims the inbox, assembles the
|
||
* prompt, then runs the `agent/pre-step` waterfall with those messages
|
||
* · `dsh-agent-loop/lib/index.js:1066-1073` — **every returned message is appended
|
||
* as a durable `user/message`**, so "shadow" injection there is not shadow: it
|
||
* becomes conversation history (see `docs/shadow-context.md` §1.3)
|
||
*
|
||
* Therefore: the input-independent index uses `systemPrompt.context`; per-turn
|
||
* recall uses `agent/pre-step`, prepended so it runs before listeners that may
|
||
* want the final word.
|
||
*
|
||
* @module dsh-memory
|
||
*/
|
||
|
||
import { Logger, Service } from '@deepseek-ai/cordis'
|
||
import { createUserMessage } from '@deepseek-ai/dsh-llm'
|
||
import { defineTool } from '@deepseek-ai/dsh-tools'
|
||
import { createHttpEmbedder } from './embed/http.js'
|
||
import { recallBlocksHybrid, renderRecall, segmentRecallBudget } from './recall.js'
|
||
import { BlockStore } from './store.js'
|
||
|
||
/** Cordis plugin name, also used as the `source.kind` of injected context. */
|
||
export const name = 'dsh-memory'
|
||
|
||
/**
|
||
* Services this plugin needs.
|
||
*
|
||
* `systemPrompt` and `tools` are both present in any standard dsh composition
|
||
* (`dsh-base/cordis.patch.yml`), so requiring them narrows nothing.
|
||
*/
|
||
export const inject = ['systemPrompt', 'tools']
|
||
|
||
/**
|
||
* Plugin configuration.
|
||
*
|
||
* @typedef {object} Config
|
||
* @property {string} [path] sqlite file path; defaults to `<cwd>/.dsh-memory/memory.db`
|
||
* @property {number} [topK] recall hits injected per turn (default 3)
|
||
* @property {boolean} [inject] false = register only the tool (default true)
|
||
* @property {boolean} [wal] sqlite WAL mode (default true)
|
||
* @property {string} [toolName] model-facing commit tool name (default `stablemem_commit`).
|
||
* @property {string} [statsToolName] model-facing stats tool name (default `stablemem_stats`).
|
||
* Tool names are unique per tool layer and `isolate()` does not partition them
|
||
* (`ToolRuntime.register` throws on a duplicate), so a preset that mounts a
|
||
* second instance must give it a distinct name.
|
||
* @property {object} [embed] vector-space configuration. Omit to run without
|
||
* vectors (phase A behaviour, symbol path only).
|
||
* @property {string} [embed.endpoint] embedding endpoint
|
||
* @property {number} [embed.dimension] vector width; required when `endpoint` is set
|
||
* @property {string} [embed.model] model name forwarded in the request body
|
||
* @property {string} [embed.apiKey] bearer credential
|
||
* @property {string} [embed.fingerprint] explicit space identity (overrides derivation)
|
||
* @property {number} [embed.timeoutMs] request timeout
|
||
* @property {object} [embedder] a pre-built Embedder, bypassing `embed`. Used by
|
||
* tests and by any adapter that is not the bundled HTTP one.
|
||
* @property {boolean} [debugLog] emit the per-turn recall summary (default
|
||
* false). Off means "only warn/error", which is the shipping default: an
|
||
* operator opts in when they are actually diagnosing recall. See
|
||
* {@link StableMem.logRecallTurn} for what the summary carries.
|
||
*/
|
||
|
||
/**
|
||
* Degradations that are NOT faults.
|
||
*
|
||
* `ok` is healthy. `hybrid` is the healthy path once vectors participate (P1),
|
||
* `no_embedder` is the SHIPPING DEFAULT (a standing configuration fact, the same
|
||
* judgement W11e made for the model-facing notice), `aborted` is a deliberate
|
||
* stop, `no_vector_space` means the space exists but holds no comparable vectors
|
||
* (a fresh space — a data state, not an incident), and `vector_ready_unused` is
|
||
* the group-A build stage label, kept so an older build's log line still reads.
|
||
* None of them justifies a warn on every turn; all of them are still worth a line
|
||
* when `debugLog` is on. Everything else — `embed_error`, and any mode a future
|
||
* branch invents — is a real fault and speaks even with `debugLog` off.
|
||
*/
|
||
const ORDINARY_DEGRADATIONS = new Set([
|
||
'ok', 'hybrid', 'no_embedder', 'aborted', 'no_vector_space', 'vector_ready_unused',
|
||
])
|
||
|
||
/**
|
||
* Logger for one plugin instance: `ctx.logger` when the cordis build has one,
|
||
* otherwise a `console.warn`/`console.error` shim.
|
||
*
|
||
* ★ stdout is NEVER written, in either path. `dsh --profile headless` prints the
|
||
* agent's final answer on stdout (`dsh-headless/lib/index.js:20`), so an info line
|
||
* there is not noise — it is a corrupted answer. The shim therefore routes
|
||
* `debug`/`info` to `console.warn` as well: stderr is the only sink this plugin
|
||
* may touch.
|
||
*
|
||
* ★ Why the facade alone is not enough (measured). cordis ships exactly ONE
|
||
* default exporter — an in-memory ring buffer (`LoggerService` constructor,
|
||
* `cordis/lib/index.js:594-603`) — and no dsh bundle adds a terminal one. So
|
||
* `ctx.logger.info(...)` in a real profile writes to a buffer nobody drains:
|
||
* the plugin would look instrumented while a headless run still printed
|
||
* nothing. {@link installStderrSink} is what makes the summarised line
|
||
* observable, and it is what the self-test in the commit message quotes.
|
||
*
|
||
* @param {import('@deepseek-ai/cordis').Context} ctx
|
||
* @param {string} label logger name
|
||
* @param {boolean} debugLog whether debug/info lines are wanted
|
||
* @returns {{debug: Function, info: Function, warn: Function, error: Function}}
|
||
*/
|
||
function resolveLogger(ctx, label, debugLog) {
|
||
try {
|
||
const service = ctx?.logger
|
||
if (typeof service === 'function') {
|
||
const logger = service(label)
|
||
if (logger !== null && typeof logger === 'object' && typeof logger.warn === 'function') {
|
||
installStderrSink(service, label, debugLog === true)
|
||
return logger
|
||
}
|
||
}
|
||
} catch {
|
||
// A context without a working logger service: fall through to the shim.
|
||
}
|
||
const write = (sink) => (format, ...rest) => {
|
||
// `%s` is how every call site below protects the payload from cordis's
|
||
// printf formatter (see `emit`); honour it here so the shim prints the same
|
||
// line the real logger would.
|
||
const text = String(format) === '%s' ? rest.join(' ') : [format, ...rest].join(' ')
|
||
sink(text)
|
||
}
|
||
return {
|
||
debug: write((text) => console.warn(text)),
|
||
info: write((text) => console.warn(text)),
|
||
warn: write((text) => console.warn(text)),
|
||
error: write((text) => console.error(text)),
|
||
}
|
||
}
|
||
|
||
/**
|
||
* Route this logger's records to stderr, filtered by severity.
|
||
*
|
||
* Two gates, deliberately in this order:
|
||
*
|
||
* 1. `message.name !== label` — the exporters map is process-wide (a child
|
||
* context sees the same Map, verified), so without this filter the sink
|
||
* would reprint every other plugin's warnings under this plugin's prefix;
|
||
* 2. severity — `warn`/`error` always pass, because a failed recall is a real
|
||
* event regardless of verbosity; `info`/`debug` pass only when `debugLog`
|
||
* asked for them.
|
||
*
|
||
* Disposal rides on `ctx.effect` inside `LoggerService.exporter`, so a plugin
|
||
* unload removes the sink with the rest of the fiber.
|
||
*
|
||
* @param {{exporter?: Function}} service the `ctx.logger` service
|
||
* @param {string} label logger name this instance owns
|
||
* @param {boolean} debugLog
|
||
* @returns {void}
|
||
*/
|
||
function installStderrSink(service, label, debugLog) {
|
||
if (typeof service.exporter !== 'function') return
|
||
// `levels` admits everything FOR OUR NAME so the severity decision is made
|
||
// here, in one place, instead of being split between the logger's gate and
|
||
// this filter. Other names fall back to cordis's default INFO gate and are
|
||
// then dropped by the name check below.
|
||
const levels = { [label]: 3 }
|
||
try {
|
||
/** The exporter object, referenced by its own `export` for `Logger.format`. */
|
||
const exporter = {
|
||
levels,
|
||
colors: false,
|
||
export: (message) => {
|
||
if (message?.name !== label) return
|
||
const ordinary = message.type === 'info' || message.type === 'debug'
|
||
if (ordinary && !debugLog) return
|
||
let line
|
||
try {
|
||
// Same renderer the TUI/terminal exporters use, so colours, `%s`
|
||
// expansion and the per-line length cap behave as every other dsh log.
|
||
// The exporter is passed (not `this`): `format` reads its `colors`,
|
||
// `formatters` and `maxLength`.
|
||
line = Logger.format(exporter, message)
|
||
} catch {
|
||
line = message.args.map((arg) => String(arg)).join(' ')
|
||
}
|
||
process.stderr.write(`${label}: ${message.type}: ${line}\n`)
|
||
},
|
||
}
|
||
service.exporter(exporter)
|
||
} catch {
|
||
// An exporter that cannot be installed must not take the plugin down; the
|
||
// ring buffer still holds the records for an attached exporter.
|
||
}
|
||
}
|
||
|
||
/**
|
||
* Emit exactly one line, keeping the payload out of the format string.
|
||
*
|
||
* `Logger.format()` in cordis expands `%s`/`%d`/`%o` in the FORMAT argument
|
||
* (cordis/lib/index.js:428-441). A provider error message that happens to
|
||
* contain `%s` would then swallow or print `undefined` for the rest of the
|
||
* line — a log you cannot read is the failure mode this whole change exists to
|
||
* remove. So the line always travels as the single `%s` argument.
|
||
*
|
||
* @param {{[k: string]: Function}} log
|
||
* @param {'debug'|'info'|'warn'|'error'} level
|
||
* @param {string} line
|
||
*/
|
||
function emit(log, level, line) {
|
||
log[level]('%s', line)
|
||
}
|
||
|
||
/** Code points, not UTF-16 units: a CJK/emoji query must not report double. */
|
||
function charCount(text) {
|
||
return [...String(text ?? '')].length
|
||
}
|
||
|
||
/**
|
||
* One-line digest of a degradation message, for the log only.
|
||
*
|
||
* Truncated and whitespace-flattened: the summary exists to tell an operator
|
||
* WHICH failure occurred (`read ECONNREFUSED`, `timed out`, `dim 3, want 768`),
|
||
* not to reprint a provider response body into a log line.
|
||
*
|
||
* @param {unknown} message
|
||
* @param {number} [limit]
|
||
* @returns {string} '-' when there is nothing to say
|
||
*/
|
||
function summarizeError(message, limit = 80) {
|
||
if (typeof message !== 'string') return '-'
|
||
const flat = message.replace(/\s+/g, ' ').trim()
|
||
if (flat === '') return '-'
|
||
return flat.length > limit ? `${flat.slice(0, limit)}...` : flat
|
||
}
|
||
|
||
/**
|
||
* The `stablemem` service: durable block storage plus recall.
|
||
*
|
||
* Registered as `ctx.stablemem` so other plugins can use it without going through
|
||
* tools, and so a preset subtree can `isolate: { stablemem: true }` it later
|
||
* (docs/shadow-context.md §5.2).
|
||
*/
|
||
export class StableMem extends Service {
|
||
/**
|
||
* ★ These are plain fields, NOT `#private`.
|
||
*
|
||
* cordis resolves a service through a **proxy** (`ctx.stablemem !== instance`),
|
||
* and JS private fields are bound to the declaring object, so any method that
|
||
* touches `this.#store` throws
|
||
* `Cannot read private member #store from an object whose class did not declare it`
|
||
* when reached through `ctx.<name>`. Verified against the real Context.
|
||
* A cordis service must therefore keep its state in ordinary properties.
|
||
*
|
||
* @type {BlockStore}
|
||
*/
|
||
store
|
||
/** @type {number} */
|
||
topK
|
||
/**
|
||
* Whether the per-turn recall summary is emitted. Plain field, like every
|
||
* other one: cordis hands out a proxy and private fields do not survive it.
|
||
* @type {boolean}
|
||
*/
|
||
debugLog
|
||
/**
|
||
* Named logger (`ctx.logger(name)`), or the console shim.
|
||
* @type {{debug: Function, info: Function, warn: Function, error: Function}}
|
||
*/
|
||
log
|
||
/**
|
||
* Vector capability, or null. Phase A has none; group H supplies the adapter.
|
||
* Kept as a plain property (not `#private`) — cordis hands out a proxy, and
|
||
* private fields do not survive it.
|
||
* @type {import('./recall.js').Embedder | null}
|
||
*/
|
||
embedder
|
||
|
||
/**
|
||
* Outcome of the most recent recall, for the stats surface.
|
||
*
|
||
* ★ Why this exists in addition to the per-call `degraded` on the result: those
|
||
* answer different questions. `degraded` is a RETURN VALUE — the caller sees it
|
||
* once and it is gone. An operator asking "did the last recall fail, and why"
|
||
* needs something durable, and criteria:282 specifies `last_recall` in the stats
|
||
* shape for exactly that.
|
||
*
|
||
* Deliberately NOT part of the four states: those are derived from rows, while
|
||
* this is a per-call event. Mixing them would make two reads at the same instant
|
||
* disagree — the failure mode D4 exists to prevent.
|
||
*
|
||
* @type {{mode: string, skipped_fp: number, skipped_dim: number,
|
||
* skipped_score: number, skipped_no_evidence: number,
|
||
* skipped_unscoreable: number, truncated: number,
|
||
* embed_error: string|null, at: number} | null}
|
||
*/
|
||
lastRecall = null
|
||
|
||
/**
|
||
* @param {import('@deepseek-ai/cordis').Context} ctx
|
||
* @param {{ store: BlockStore, topK?: number,
|
||
* embedder?: import('./recall.js').Embedder | null }} options
|
||
*/
|
||
constructor(ctx, {
|
||
store, topK = 3, embedder = null, documentEmbedder = null,
|
||
reportNoVectorAlways = false, budgetTokens, debugLog = false,
|
||
}) {
|
||
super(ctx, 'stablemem')
|
||
this.store = store
|
||
this.topK = topK
|
||
this.embedder = embedder
|
||
// ★ V6 — a stored document is not a query.
|
||
//
|
||
// The HTTP protocol carries a `side` field (`query` or `document`) and HA
|
||
// distinguishes them (`PurposeQuery` / `PurposeDocument`). A provider that
|
||
// embeds the two asymmetrically (the usual case for retrieval models) needs
|
||
// the distinction or the stored vectors live in a different space from the
|
||
// query vectors — silently, since both are unit vectors of the right width.
|
||
//
|
||
// `documentEmbedder` is the write-side encoder. It defaults to `embedder`
|
||
// because an injected single-instance embedder (tests, a local adapter) has
|
||
// no two-sided story to tell — the fallback is the previous behaviour, not a
|
||
// second path.
|
||
this.documentEmbedder = documentEmbedder ?? embedder
|
||
// Observability only: nothing below reads `debugLog` except the logging
|
||
// path, so turning it on cannot change a score, a budget, or a return value.
|
||
this.debugLog = debugLog === true
|
||
this.log = resolveLogger(ctx, name, this.debugLog)
|
||
// Test/diagnostic hook: report the `noVector` clause even when no embedder is
|
||
// configured. Production leaves this false so a standing configuration fact
|
||
// stays silent (see renderUnavailableTail).
|
||
this.reportNoVectorAlways = reportNoVectorAlways
|
||
/** Explicit memory budget (tokens); wins over any host-derived window. */
|
||
this.budgetTokens = Number.isFinite(budgetTokens) && budgetTokens > 0 ? budgetTokens : undefined
|
||
/** Cached host/model context windows, keyed by provider/model. */
|
||
this.windowCache = new Map()
|
||
// Release the sqlite handle with the owning fiber, so a plugin unload does
|
||
// not leak the file lock (checkpoint V5).
|
||
ctx.effect(() => () => this.store.close(), 'dsh-memory.store')
|
||
}
|
||
|
||
/**
|
||
* The configured vector space, or null when running symbol-only.
|
||
*
|
||
* ★ This single object is the ONLY source of `fingerprint`. Both write paths
|
||
* (block rows and centroid rows) take the string from here rather than
|
||
* deriving it themselves: HA looks the centroid up by fingerprint
|
||
* (`block_recall.go:92`), so two independently-derived spellings would make
|
||
* that lookup miss with no error — centering silently never applies.
|
||
*
|
||
* `info()` is synchronous and memoised in the adapter, so the string is
|
||
* computed once per process and reused forever.
|
||
*
|
||
* @returns {{fingerprint: string, dimension: number} | null}
|
||
*/
|
||
space() {
|
||
return this.embedder === null ? null : this.embedder.info()
|
||
}
|
||
|
||
/**
|
||
* Commit one text block.
|
||
*
|
||
* When a vector space is configured the block is embedded and stamped with
|
||
* `space().fingerprint`. When it is not — or the embedder fails — the block is
|
||
* still written, with an empty vector. Storing an invented vector would make it
|
||
* participate in scoring and always rank last, which is a silently wrong
|
||
* memory; an empty one honestly says "not embedded yet".
|
||
*
|
||
* @param {{ text: string, source?: string, scene?: string, semanticType?: string,
|
||
* vector?: number[], fingerprint?: string }} input
|
||
* @returns {Promise<{ id: string, created: boolean, embedded: boolean }>}
|
||
*/
|
||
async commit(input, options = {}) {
|
||
// An explicit vector+fingerprint pair (tests, migrations, or a caller that
|
||
// already embedded) is written verbatim.
|
||
if (Array.isArray(input.vector) && input.vector.length > 0) {
|
||
// An explicit vector is validated against the declared space (F8): a caller
|
||
// handing over a mis-sized vector has a bug, and refusing is the only way it
|
||
// surfaces at the call site instead of as an unrecallable row.
|
||
const written = this.store.putBlock({
|
||
...input,
|
||
expectedDimension: this.space()?.dimension,
|
||
})
|
||
if (typeof input.fingerprint === 'string' && input.fingerprint !== '') {
|
||
this.#ensureCentroid(input.fingerprint)
|
||
}
|
||
return { ...written, embedded: true }
|
||
}
|
||
|
||
const space = this.space()
|
||
// No space configured, or the space cannot name itself: store the block with
|
||
// no vector. Checking the fingerprint BEFORE calling embedText avoids paying
|
||
// for a round-trip whose result could never be labelled with a space.
|
||
if (space === null || typeof space.fingerprint !== 'string' || space.fingerprint === '') {
|
||
return { ...this.store.putBlock(input), embedded: false }
|
||
}
|
||
|
||
let vector
|
||
try {
|
||
// `options.signal` is the tool call's cancellation. Forwarding it is
|
||
// required by the tool contract (dsh-tools/lib/types/index.d.ts:120-122)
|
||
// and it bounds latency: without it the adapter would hold the call for up
|
||
// to its own timeout while the user has already cancelled the turn.
|
||
vector = await this.documentEmbedder.embedText(input.text, options.signal)
|
||
} catch {
|
||
// Cancelled, or the vector side is down: the block still lands with no
|
||
// vector (decision §0.4). An abort must not turn into "nothing was written".
|
||
return { ...this.store.putBlock(input), embedded: false }
|
||
}
|
||
|
||
// Write-side dimension check. Recall already skips mismatched vectors
|
||
// (recall.js), so storing one would produce a block that exists, looks fine,
|
||
// and can never be recalled — the exact silent failure group F exists to
|
||
// eliminate. Treat it as an embed failure instead of storing a bad vector.
|
||
if (!Array.isArray(vector) || vector.length !== space.dimension) {
|
||
return { ...this.store.putBlock(input), embedded: false }
|
||
}
|
||
|
||
const written = this.store.putBlock({
|
||
...input,
|
||
vector,
|
||
// ★ From the space object, never re-derived and never from input.
|
||
fingerprint: space.fingerprint,
|
||
})
|
||
// ★ C4 的另一半：中心必须存在，否则去均值永远不会发生。
|
||
this.#ensureCentroid(space.fingerprint)
|
||
return { ...written, embedded: true }
|
||
}
|
||
|
||
/**
|
||
* 确保一个向量空间有中心，没有就建一个。
|
||
*
|
||
* ★ 为什么这件事必须写在写入侧
|
||
* 在 C4 之前，`graph_centroid` 在生产上永远是空表：`saveSpaceCentroid` 只有
|
||
* 测试在调，`centroidStatus` / `centroidRecallState` 在 `lib/` 里没有调用者，
|
||
* `rebuildCentroid` 也只有测试在调。于是 `loadCentroid` 永远返回 undefined、
|
||
* `isAnomalous` 永远是 false、去均值一次也不会发生 —— 而判据会绿，
|
||
* 因为判据自己会先手工 `rebuildCentroid`。那是本轮反复抓的「代码写了没接线」。
|
||
*
|
||
* ★ 为什么不能放在召回侧
|
||
* `rebuildCentroid` 是 O(n·d)（实测 dim=512、2000 块 85.5ms），而召回每轮
|
||
* pre-step 都跑。C3-k 也明令 `countVectored` 不得进热路径，而 `centroidStatus`
|
||
* 第一行就是它。所以：召回侧只 `loadCentroid` 纯读，重建留在这里。
|
||
*
|
||
* ★ 失败必须吞掉
|
||
* 一次中心重建失败不能把写入变成失败。下一次 commit 会再试；在那之前
|
||
* 召回拿不到中心就走未校正路径，结果仍是对的，只是没校正。
|
||
* 这与 D1（向量侧故障不得让主路径抛错）是同一条纪律。
|
||
*
|
||
* ★ 只在「没有中心行」时建
|
||
* 有中心行就交给已有的 stale 判定去管（块数漂移超 20% 时重建），
|
||
* 不在这里每写一条就算一遍全空间。
|
||
*
|
||
* ★ 首次建是同步的，而且「那时块还少」是个不成立的前提
|
||
* 这套代码存在的理由恰恰是：生产库里已经有几千条带向量的块、
|
||
* `graph_centroid` 却是空表（W8 的原场景）。那种库升级后第一次 commit，
|
||
* 走的就是下面这条同步 `rebuildCentroid`：全空间 SELECT + 逐条 JSON.parse
|
||
* + 算均值，全部发生在写入路径上。
|
||
* 实测（512 维、2000 条带向量的块、空中心表）：
|
||
*
|
||
* 升级后第一次 commit 105.1 ms
|
||
* 之后每条 commit 8.7-9.4 ms（loadCentroid 的 SELECT + centroidStatus 的 COUNT）
|
||
*
|
||
* 首次建随块数线性增长（8000 块约 420ms）。这里接受这个一次性停顿，
|
||
* 而不是把它挪到异步：挪走会让升级后最初几次召回拿不到中心而静默不校正，
|
||
* 而 n 很大时正是最需要校正的时候 —— 一次有界停顿比一段静默降级更容易被
|
||
* 发现。稳态那 ~9ms 是 C4 新增的写路径开销；C3-k 只禁它进召回热路径。
|
||
*
|
||
* @param {string} fingerprint
|
||
*/
|
||
#ensureCentroid(fingerprint) {
|
||
if (typeof fingerprint !== 'string' || fingerprint === '') return
|
||
try {
|
||
if (this.store.loadCentroid(fingerprint) === undefined) {
|
||
this.store.rebuildCentroid(fingerprint)
|
||
return
|
||
}
|
||
// ★ 已有中心时不能直接 return。
|
||
// `rebuildCentroid` 只算一次，之后块数会一直长；如果这里只守「没有就建」，
|
||
// 中心就永远停在第一桶快照上，越用越偏。交给 `centroidStatus`：它内部
|
||
// 已经写好块数漂移 > 20% 判 stale 与 10 分钟节流异步重建，是这套判据
|
||
// 唯一的正统入口 —— 而在 C4 之前它在 lib/ 里没有任何生产调用者。
|
||
//
|
||
// 放在 commit 之后而不是召回路径，正是因为它第一行是 `countVectored`：
|
||
// C3-k 明令该查询不得进每轮 pre-step 的热路径。
|
||
this.store.centroidStatus(fingerprint)
|
||
} catch {
|
||
// 吞掉：写入已经成功，中心可以下次再建。
|
||
}
|
||
}
|
||
|
||
/**
|
||
* Persist the centroid for the configured space.
|
||
*
|
||
* Takes NO fingerprint argument on purpose: the caller cannot supply a
|
||
* different spelling than the one {@link commit} stamped on the blocks.
|
||
*
|
||
* ★ Callers: this is a PUBLIC programmatic entry point, not part of the
|
||
* write path. `/lib` never calls it; the only in-repo caller is
|
||
* `test/phase-b-h.test.js` (a caller that already computed a centroid
|
||
* elsewhere). Production centroids are derived from the blocks themselves by
|
||
* `BlockStore.rebuildCentroid`, driven from {@link commit} via
|
||
* `#ensureCentroid` — so this method existing with no internal caller is not
|
||
* a dangling wire, it is the seam for an external producer. Recorded because
|
||
* "defined, never called" looked identical to the real dangling-wire bugs
|
||
* found in C4 and P1.
|
||
*
|
||
* @param {{ dim?: number, vectorCount: number, mean: number[],
|
||
* isAnomalous: boolean, stats?: object }} centroid
|
||
* @returns {{ fingerprint: string }}
|
||
*/
|
||
saveSpaceCentroid(centroid) {
|
||
const space = this.space()
|
||
if (space === null) throw new Error('dsh-memory: no vector space configured')
|
||
this.store.saveCentroid({
|
||
fingerprint: space.fingerprint,
|
||
dim: centroid.dim ?? space.dimension,
|
||
vectorCount: centroid.vectorCount,
|
||
mean: centroid.mean,
|
||
isAnomalous: centroid.isAnomalous,
|
||
stats: centroid.stats ?? {},
|
||
})
|
||
return { fingerprint: space.fingerprint }
|
||
}
|
||
|
||
/**
|
||
* Link two blocks.
|
||
* @param {object} edge
|
||
* @returns {{ created: boolean }}
|
||
*/
|
||
link(edge) {
|
||
return this.store.addEdge(edge)
|
||
}
|
||
|
||
/**
|
||
* Recall hits for one query.
|
||
* @param {string} query
|
||
* @param {object} [options]
|
||
* @param {number} [options.topK]
|
||
* @param {AbortSignal} [options.signal] cancellation, forwarded to the embedder
|
||
*/
|
||
async recall(query, options = {}) {
|
||
// Group A ships the async seam only: with no embedder configured this is
|
||
// exactly `recallBlocks` (pinned by the golden fixture in test/phase-b-a).
|
||
const hits = await recallBlocksHybrid(this.store, this.embedder, query, {
|
||
topK: options.topK ?? this.topK,
|
||
signal: options.signal,
|
||
})
|
||
this.lastRecall = {
|
||
// `degraded` is absent when the recall was healthy, which is the ordinary
|
||
// case; 'ok' is recorded explicitly so the stats reader never has to
|
||
// distinguish "no field" from "no problem".
|
||
mode: hits.degraded ?? 'ok',
|
||
// `mode` says whether the answer is trustworthy. The three counters below
|
||
// say how many candidates were dropped DOWNSTREAM of the space filter —
|
||
// without them an operator whose threshold is swallowing the corpus reads
|
||
// `skipped=0` and concludes nothing was excluded. See
|
||
// recall-accounting.test.js (D2).
|
||
skipped_fp: hits.skipped ?? 0,
|
||
// Real, not a placeholder: the recall path applies the F4b dimension filter
|
||
// and reports it separately from the space skip.
|
||
skipped_dim: hits.skippedDim ?? 0,
|
||
skipped_score: hits.skippedScore ?? 0,
|
||
skipped_no_evidence: hits.skippedNoEvidence ?? 0,
|
||
// D2's second hole: blocks dropped BEFORE they could enter the candidate
|
||
// pool (bad JSON, wrong width, non-finite component, zero norm, or — since
|
||
// C4 — a vector that centring collapses onto the space mean). Those are not
|
||
// "gated": no gate ever saw them, which is why they were invisible.
|
||
skipped_unscoreable: hits.skippedUnscoreable ?? 0,
|
||
truncated: hits.truncated ?? 0,
|
||
embed_error: hits.degradedError ?? null,
|
||
at: Date.now(),
|
||
}
|
||
return hits
|
||
}
|
||
|
||
/**
|
||
* Text to inject for this step, or '' when nothing is relevant.
|
||
*
|
||
* The query is the **last user-role text** in the batch: `decision.messages` is
|
||
* the batch admitted for this step, and steering appends rather than replaces,
|
||
* so the last user message is the actual prompt.
|
||
*
|
||
* @param {ReadonlyArray<{role: string, content: ReadonlyArray<{type: string, text?: string}>}>} messages
|
||
* @param {AbortSignal} [signal] the turn's cancellation, forwarded to recall
|
||
* @returns {Promise<string>}
|
||
*/
|
||
async contextForStep(messages, signal, budget = {}) {
|
||
const query = lastUserText(messages)
|
||
if (query === '') {
|
||
// Not a recall at all — but worth one debug line when verbose logging is
|
||
// on, because "the batch carried no human prompt" is exactly the failure
|
||
// this plugin already shipped once (the skill catalog as recall query).
|
||
this.logRecallTurn({ query, messages, raw: null, segmented: null, injected: '' })
|
||
return ''
|
||
}
|
||
const raw = await this.recall(query, { signal })
|
||
const segmented = segmentRecallBudget(raw, {
|
||
topK: this.topK,
|
||
budgetTokens: this.budgetTokens,
|
||
contextWindow: budget.contextWindow,
|
||
})
|
||
const { hits } = segmented
|
||
// ★ Do NOT short-circuit on `hits.length === 0` here. renderRecall decides
|
||
// silence by asking whether there are BOTH no entries and no notice; a
|
||
// degraded recall with zero hits still has something to say. Short-circuiting
|
||
// here would discard that notice before renderRecall ever sees it — and
|
||
// fixing only renderRecall without this line changes nothing (W11d†b).
|
||
const rendered = renderRecall(hits, name)
|
||
const notice = hits.budgetNotice ?? ''
|
||
const injected = notice === '' ? rendered : rendered === '' ? notice : `${rendered}\n${notice}`
|
||
// Observe AFTER the decision is made, so the logged `inject` count is the
|
||
// exact string this method returns (not a re-derived estimate).
|
||
this.logRecallTurn({ query, messages, raw, segmented, injected })
|
||
return injected
|
||
}
|
||
|
||
/**
|
||
* One summary line per recall turn — the four facts an operator needs to
|
||
* answer "why did (or didn't) memory show up in this turn".
|
||
*
|
||
* | fact | fields | why it is there |
|
||
* |---|---|---|
|
||
* | recall | `query=<n>ch`, `hits`, `topK`, `zeroHit` | did the query reach recall, and did it match |
|
||
* | budget | `budget`, `source`, `kept`, `singleHitOverBudget` | which cap applied and what it kept |
|
||
* | degrade | `degraded`, `skipped`, `skippedDim`, `skippedScore`, `skippedNoEvidence`, `truncated`, `err` | whether the answer is trustworthy, and where the candidates went |
|
||
* | inject | `inject=<n>ch` | did it actually reach the model |
|
||
*
|
||
* ★ The query TEXT is never logged. Only its length: the recall query is the
|
||
* user's own message, and a memory log is the last place it should be copied
|
||
* to. Length is enough to tell "we searched nothing" from "we searched the
|
||
* skill catalog" when read beside `hits`.
|
||
*
|
||
* ★ Level, and why it is not always `info`:
|
||
* · fault (`embed_error`, or any mode not in {@link ORDINARY_DEGRADATIONS})
|
||
* → `warn`, emitted even when `debugLog` is off. A failed search is a
|
||
* per-turn event where absence of evidence is not evidence of absence;
|
||
* · `inject=0ch` (the common case on the shipping default) → `debug`, so
|
||
* "every ordinary turn" costs nothing at normal verbosity, but the line
|
||
* is still exactly one line;
|
||
* · otherwise → `info`, only when `debugLog` is on.
|
||
*
|
||
* ★ Logging must never break recall: any throw here would turn an
|
||
* observability feature into a recall outage, so the body is guarded.
|
||
*
|
||
* @param {{query: string, messages?: ReadonlyArray<object>, raw: Array<object>|null,
|
||
* segmented: object|null, injected: string}} turn
|
||
* @returns {void}
|
||
*/
|
||
logRecallTurn({ query, messages, raw, segmented, injected }) {
|
||
try {
|
||
const injectChars = charCount(injected)
|
||
|
||
// No human prompt in the batch: there was no recall to summarize.
|
||
if (segmented === null) {
|
||
if (this.debugLog !== true) return
|
||
emit(
|
||
this.log,
|
||
'debug',
|
||
`recall query=0ch skipped=no-human-prompt messages=${
|
||
Array.isArray(messages) ? messages.length : 0
|
||
} inject=0ch`,
|
||
)
|
||
return
|
||
}
|
||
|
||
const degraded = raw?.degraded ?? 'ok'
|
||
const fault = !ORDINARY_DEGRADATIONS.has(degraded)
|
||
if (this.debugLog !== true && !fault) return
|
||
|
||
const rawHits = Array.isArray(raw) ? raw.length : 0
|
||
const kept = Array.isArray(segmented.hits) ? segmented.hits.length : 0
|
||
const line = [
|
||
`recall query=${charCount(query)}ch hits=${rawHits} topK=${this.topK} zeroHit=${
|
||
rawHits === 0 ? 'yes' : 'no'
|
||
}`,
|
||
`budget=${segmented.budgetTokens} source=${segmented.source} kept=${kept} singleHitOverBudget=${
|
||
segmented.singleHitOverBudget ? 'yes' : 'no'
|
||
}`,
|
||
`degraded=${degraded} skipped=${raw?.skipped ?? 0} skippedDim=${
|
||
raw?.skippedDim ?? 0
|
||
} skippedScore=${raw?.skippedScore ?? 0} skippedNoEvidence=${
|
||
raw?.skippedNoEvidence ?? 0
|
||
} skippedUnscoreable=${raw?.skippedUnscoreable ?? 0} truncated=${
|
||
raw?.truncated ?? 0
|
||
} err=${summarizeError(raw?.degradedError)}`,
|
||
`inject=${injectChars}ch`,
|
||
].join(' | ')
|
||
|
||
if (fault) emit(this.log, 'warn', line)
|
||
else if (injectChars === 0) emit(this.log, 'debug', line)
|
||
else emit(this.log, 'info', line)
|
||
} catch {
|
||
// Observability is not worth a turn. Swallow and keep the recall path.
|
||
}
|
||
}
|
||
|
||
/**
|
||
* The model context window this step will run under, when the host discloses it.
|
||
*
|
||
* Precedence mirrors the port's budget rule (explicit config > host/model window
|
||
* > conservative default): an explicit `budgetTokens` never reaches here.
|
||
*
|
||
* The logged `request/context` event is preferred because it is the exact
|
||
* capability the request uses. It does not exist yet on a session's first step
|
||
* (`request/context` is appended during the request itself), so we ask the
|
||
* adapter once per route and cache the answer. Failures are cached too — an
|
||
* unresolvable route is a standing configuration fact, and retrying an adapter
|
||
* lookup every turn would put provider I/O on the recall hot path.
|
||
*
|
||
* @param {object} agent the agent proposing this step
|
||
* @param {AbortSignal} [signal] cancellation for the adapter lookup
|
||
* @returns {Promise<number | undefined>} window in tokens, or undefined when unknown
|
||
*/
|
||
async contextWindowFor(agent, signal) {
|
||
const logged = agent?.session?.requestContext?.()?.contextWindow
|
||
if (Number.isFinite(logged) && logged > 0) return logged
|
||
|
||
const provider = agent?.options?.provider
|
||
const model = agent?.options?.model
|
||
if (typeof provider !== 'string' || provider === '') return undefined
|
||
if (typeof model !== 'string' || model === '') return undefined
|
||
|
||
const key = `${provider}/${model}`
|
||
if (this.windowCache.has(key)) {
|
||
const cached = this.windowCache.get(key)
|
||
return Number.isFinite(cached) && cached > 0 ? cached : undefined
|
||
}
|
||
|
||
let resolved = null
|
||
try {
|
||
const llmService = typeof this.ctx?.get === 'function' ? this.ctx.get('llm') : undefined
|
||
const llm = llmService
|
||
const info = typeof llm?.resolveModelInfo === 'function'
|
||
? await llm.resolveModelInfo(provider, model, signal)
|
||
: undefined
|
||
const window = info?.context?.contextWindow
|
||
if (Number.isFinite(window) && window > 0) resolved = window
|
||
} catch {
|
||
// No adapter, a rejecting adapter, or cancellation: fall back to the
|
||
// conservative default rather than guessing a large window.
|
||
resolved = null
|
||
}
|
||
this.windowCache.set(key, resolved)
|
||
return resolved ?? undefined
|
||
}
|
||
|
||
/**
|
||
* Constant index text for `systemPrompt.context`.
|
||
*
|
||
* Deliberately input-independent: that hook cannot see the current turn's text
|
||
* (module note above). A count is still worth its tokens — the model learns that
|
||
* durable memory exists without paying a per-turn recall cost.
|
||
*
|
||
* @returns {string}
|
||
*/
|
||
renderIndex() {
|
||
const count = this.store.countBlocks()
|
||
if (count === 0) return ''
|
||
|
||
// ★ Reads the SAME derived accounting the stats tool reports — not a second
|
||
// implementation. If either kept its own tally, the model would be told a
|
||
// number that contradicts the operator's report.
|
||
const space = this.space()
|
||
const states = this.store.blockStates(space?.fingerprint, space?.dimension)
|
||
const tail = this.renderUnavailableTail(states, space)
|
||
|
||
return (
|
||
`[Memory · ${name}] ${count} durable note(s) in block memory.${tail} ` +
|
||
'Relevant ones are recalled into your timeline automatically; ' +
|
||
'record new facts with the `stablemem_commit` tool.'
|
||
)
|
||
}
|
||
|
||
/**
|
||
* One clause per unavailable state, because the states are NOT interchangeable.
|
||
*
|
||
* The earlier single sentence ended "they are still searchable by exact text",
|
||
* which is true only for `noVector`. Measured for `spaceMismatch`:
|
||
*
|
||
* production recall (with fingerprint): hits=0, skipped=1 ← not searchable
|
||
* store-level browse (no fingerprint): hits=1 ← reachable
|
||
*
|
||
* So a mismatched block is reachable ONLY by browse, and saying otherwise
|
||
* invites the model to keep retrying a query that cannot succeed. This is the
|
||
* F6 distinction (browse ≠ recall) applied to the renderer.
|
||
*
|
||
* @param {object} states from `blockStates`
|
||
* @param {{fingerprint: string, dimension: number} | null} space
|
||
* @returns {string} '' when nothing is unavailable OR when the reason is a
|
||
* standing configuration fact (see below)
|
||
*/
|
||
renderUnavailableTail(states, space) {
|
||
// ★ No embedder configured is the SHIPPING DEFAULT, not a per-turn incident.
|
||
// Repeating it every turn is the same mistake W11e settled for degradation
|
||
// notices: a constant line trains the model to skip the memory section. The
|
||
// operator learns it from `stablemem_stats`, which is the right channel for a
|
||
// standing fact.
|
||
if (space === null && this.reportNoVectorAlways !== true) return ''
|
||
|
||
const clauses = []
|
||
if (states.spaceMismatch > 0) {
|
||
clauses.push(
|
||
`${states.spaceMismatch} from a retired space (reachable by browse only, not by recall)`,
|
||
)
|
||
}
|
||
if (states.unknownSpace > 0) {
|
||
clauses.push(`${states.unknownSpace} without a space label (semantic recall unavailable)`)
|
||
}
|
||
if (states.dimensionMismatch > 0) {
|
||
clauses.push(
|
||
`${states.dimensionMismatch} whose width does not match this provider (fix the config, then re-embed)`,
|
||
)
|
||
}
|
||
if (states.noVector > 0) {
|
||
// This one IS text-recallable: no vector, but the text index still matches.
|
||
clauses.push(`${states.noVector} not yet embedded (still reachable by exact text)`)
|
||
}
|
||
if (clauses.length === 0) return ''
|
||
return ` Unavailable for semantic recall: ${clauses.join('; ')}.`
|
||
}
|
||
|
||
/**
|
||
* Space accounting for the tool surface (D3).
|
||
*
|
||
* Reports the derived counts plus the per-space breakdown, so an operator can
|
||
* tell "this block is unlabelled" (re-embed) apart from "this block is from a
|
||
* retired space" (re-embed into the active one) apart from "the provider is
|
||
* misconfigured for this width" (fix config).
|
||
*
|
||
* @returns {object} JSON-safe report
|
||
*/
|
||
stats() {
|
||
const space = this.space()
|
||
const states = this.store.blockStates(space?.fingerprint, space?.dimension)
|
||
return {
|
||
space: space === null ? null : { fingerprint: space.fingerprint, dimension: space.dimension },
|
||
states,
|
||
...this.store.vectorStats(space?.fingerprint ?? ''),
|
||
centroid: space === null ? null : (this.store.loadCentroid(space.fingerprint) ?? null),
|
||
// ★ C5's `reason` needs an outlet, and this is it.
|
||
//
|
||
// `centroid` above is the raw row (`loadCentroid`), which says WHAT the
|
||
// centre is but not WHY centring is or is not happening. `reason` is the
|
||
// answer to the second question — no_centroid / dim_mismatch / zero_sample /
|
||
// stale / isotropic / ok — and until now it was computed and thrown away:
|
||
// nothing under lib/ read the value `centroidStatus` returns.
|
||
//
|
||
// ★ `schedule: () => {}` is load-bearing. `centroidStatus` REGISTERS a
|
||
// rebuild when it judges a space stale; a stats read must not mutate
|
||
// anything. With the no-op scheduler, `rebuildScheduled` is always false
|
||
// here, which is exactly what a pure read should report.
|
||
centroid_state: space === null ? null : this.store.centroidStatus(space.fingerprint, {
|
||
queryDim: space.dimension,
|
||
schedule: () => {},
|
||
}),
|
||
// Per-call, not library state: NULL until a recall has happened.
|
||
last_recall: this.lastRecall,
|
||
}
|
||
}
|
||
}
|
||
|
||
/**
|
||
* Source kinds that carry a HUMAN prompt.
|
||
*
|
||
* Everything else a plugin injects is user-ROLE but not user-SPEECH:
|
||
* `runtime-context` (dsh-agent-loop's working-directory/sandbox snapshot),
|
||
* `skill-catalog` / `skill-invocation` (dsh-tool-skill), `agent-instructions`,
|
||
* `time-context`, `goal`, and any snapshot-style contribution.
|
||
*
|
||
* ★ Why this allowlist exists (regression, found in a real profile run):
|
||
* A real step batch is not `[question]` — it is
|
||
*
|
||
* [ user question ][ runtime-context snapshot ][ skill catalog reminder ]
|
||
*
|
||
* and the injected ones are `role: 'user'` too. Picking "the last user-role
|
||
* message" therefore returned the SKILL CATALOG, so every real turn used a list
|
||
* of skill names as its recall query and matched nothing. Auto-recall was 100%
|
||
* dead in production while all unit checkpoints were green, because the tests
|
||
* built 2-message batches.
|
||
*/
|
||
const HUMAN_PROMPT_KINDS = new Set(['user'])
|
||
|
||
/**
|
||
* Pull the human prompt out of a step's message batch.
|
||
*
|
||
* Scans backwards (the newest human turn wins) but filters by
|
||
* `source.kind` — see {@link HUMAN_PROMPT_KINDS} for why role alone is wrong.
|
||
* Unknown kinds are treated as injected: a plugin's own snapshot must never
|
||
* become the recall query, and a genuine human message always carries
|
||
* `kind: 'user'` (`dsh-llm/lib/types/message.d.ts`, `MessageSourceMap.user`).
|
||
*
|
||
* @param {ReadonlyArray<{role: string, source?: {kind?: string},
|
||
* content: ReadonlyArray<{type: string, text?: string}>}>} messages
|
||
* @returns {string} the question text, or '' when the batch has no human prompt
|
||
*/
|
||
export function lastUserText(messages) {
|
||
if (!Array.isArray(messages)) return ''
|
||
for (let index = messages.length - 1; index >= 0; index -= 1) {
|
||
const message = messages[index]
|
||
if (message?.role !== 'user') continue
|
||
const kind = message.source?.kind
|
||
if (typeof kind !== 'string' || !HUMAN_PROMPT_KINDS.has(kind)) continue
|
||
const text = (message.content ?? [])
|
||
.filter((block) => block?.type === 'text' && typeof block.text === 'string')
|
||
.map((block) => block.text)
|
||
.join('\n')
|
||
.trim()
|
||
if (text !== '') return text
|
||
}
|
||
return ''
|
||
}
|
||
|
||
/**
|
||
* Register the `stablemem_commit` tool.
|
||
*
|
||
* `output.schema` + `output.render` are mandatory in dsh
|
||
* (`dsh-tools/lib/types/index.d.ts:117`), unlike HologramAgent's
|
||
* description-only `toolDefWith` (`internal/agent/core/tooldefs.go`).
|
||
*
|
||
* @param {import('@deepseek-ai/cordis').Context} ctx
|
||
* @param {StableMem} mem
|
||
* @param {string} toolName model-facing commit tool name
|
||
* @param {string} statsToolName model-facing stats tool name
|
||
* @returns {() => void} disposer
|
||
*/
|
||
function registerTools(ctx, mem, toolName, statsToolName) {
|
||
// ★ No `return` here: a second registration follows. An early return would make
|
||
// every later tool unreachable while the file still reads as if it registers
|
||
// them — the "green because it never ran" shape this project keeps finding.
|
||
ctx.tools.register(
|
||
defineTool({
|
||
name: toolName,
|
||
description:
|
||
'Store one durable fact or note in long-term block memory. Use it for ' +
|
||
'stable facts worth recalling later (addresses, ports, paths, decisions, ' +
|
||
'preferences). Keep each note short and self-contained: recall matches ' +
|
||
'on the note text itself, so include the subject, not just the value.',
|
||
parameters: {
|
||
text: {
|
||
type: 'string',
|
||
description: 'The note. Self-contained: include subject, not just the value.',
|
||
required: true,
|
||
},
|
||
source: {
|
||
type: 'string',
|
||
description: 'Optional origin label (channel, tool, file).',
|
||
},
|
||
scene: {
|
||
type: 'string',
|
||
description: 'Optional scene key for prefix-matching recall.',
|
||
},
|
||
},
|
||
output: {
|
||
schema: { type: 'string' },
|
||
render: (_args, value) => [{ type: 'text', text: value }],
|
||
},
|
||
// Commits are independent and idempotent (content-addressed id), so they
|
||
// regroup safely with sibling calls.
|
||
isConcurrencySafe: () => true,
|
||
async execute(args, exec) {
|
||
const { id, created, embedded } = await mem.commit(
|
||
{
|
||
text: args.text,
|
||
source: args.source ?? 'tool',
|
||
scene: args.scene ?? '',
|
||
},
|
||
// Forward the call's cancellation into the embedding request.
|
||
{ signal: exec.signal },
|
||
)
|
||
const suffix = embedded ? '' : ' (stored without a vector)'
|
||
return `${created ? 'Committed' : 'Already present'} block ${id}${suffix}.`
|
||
},
|
||
}),
|
||
)
|
||
|
||
// D3: the space accounting, queryable. Numbers here are DERIVED from the rows
|
||
// on every call (`blockStates`), so they cannot drift from the database the way
|
||
// an accumulator would.
|
||
ctx.tools.register(
|
||
defineTool({
|
||
name: statsToolName,
|
||
description:
|
||
'Report block-memory health: how many blocks are recallable, and why the ' +
|
||
'rest are not (no vector yet / unlabelled space / retired space / wrong ' +
|
||
'width). Use it when memory seems to be missing entries.',
|
||
parameters: {},
|
||
output: {
|
||
schema: { type: 'json' },
|
||
render: (_args, value) => [{ type: 'text', text: JSON.stringify(value, null, 2) }],
|
||
},
|
||
async execute() {
|
||
return mem.stats()
|
||
},
|
||
}),
|
||
)
|
||
}
|
||
|
||
/**
|
||
* Apply the plugin.
|
||
*
|
||
* @param {import('@deepseek-ai/cordis').Context} ctx
|
||
* @param {Config} [config]
|
||
*/
|
||
export function apply(ctx, config = {}) {
|
||
const path = config.path ?? defaultPath()
|
||
const store = new BlockStore({ path, wal: config.wal ?? true }).open()
|
||
// The adapter lives in exactly one file (lib/embed/http.js) so swapping the
|
||
// transport never touches recall, scoring, or storage (guarded by G5).
|
||
//
|
||
// `config.embedder` accepts an already-built Embedder. It exists so tests can
|
||
// inject a failing/slow/nameless provider without standing up an HTTP server —
|
||
// and so a future local-ONNX adapter can be mounted without touching this file.
|
||
const embedder =
|
||
config.embedder ??
|
||
(config.embed?.endpoint !== undefined ? createHttpEmbedder(config.embed) : null)
|
||
|
||
// ★ V6 — the write side needs its own encoder.
|
||
//
|
||
// Two instances over the same endpoint/model/dimension (so the fingerprint is
|
||
// identical and the block and centroid rows still agree on one space),
|
||
// differing only in the `side` field they put on the wire. Done here rather
|
||
// than inside the adapter so `lib/embed/http.js` keeps travelling a single
|
||
// verbatim `side` value — it already accepts one, and teaching it a second,
|
||
// per-call way to receive it would be two sources of truth for one field.
|
||
//
|
||
// An injected `config.embedder` has no second instance: it falls back to the
|
||
// one object (see StableMem#documentEmbedder), which keeps every existing test
|
||
// and any single-sided adapter working unchanged.
|
||
const documentEmbedder =
|
||
config.documentEmbedder ??
|
||
(config.embed?.endpoint !== undefined && config.embedder === undefined
|
||
? createHttpEmbedder({ ...config.embed, side: 'document' })
|
||
: null)
|
||
const mem = new StableMem(ctx, {
|
||
store,
|
||
topK: config.topK ?? 3,
|
||
embedder,
|
||
documentEmbedder,
|
||
budgetTokens: config.budgetTokens,
|
||
debugLog: config.debugLog,
|
||
})
|
||
|
||
// Injection surface 1 — constant index. Right for input-independent text.
|
||
//
|
||
// ★ The prompt-context NAME must be unique per prompt layer, and `isolate()`
|
||
// does NOT partition prompt names: `SystemPrompt` keeps one `NamedEntries` per
|
||
// layer and throws `prompt context "..." is already registered`
|
||
// (dsh-system-prompt/lib/index.js:204). So a preset mounting a second instance
|
||
// must not reuse the literal name — deriving it from the database path keeps
|
||
// two instances in one realm distinguishable, and is stable across restarts.
|
||
ctx.systemPrompt.context({
|
||
name: `memory:index:${instanceKey(path)}`,
|
||
order: ctx.systemPrompt.getContextOrder('WEB_SURFACE'),
|
||
interpolate: false,
|
||
text: () => mem.renderIndex(),
|
||
})
|
||
|
||
// Injection surface 2 — per-turn recall. The only hook that sees the input.
|
||
if (config.inject ?? true) {
|
||
ctx.on(
|
||
'agent/pre-step',
|
||
async (payload, next) => {
|
||
const decision = await next()
|
||
if (decision.kind === 'reject' || payload.signal.aborted) return decision
|
||
// Forward the turn signal: without it a cancelled turn waits out the
|
||
// embedding adapter's own timeout before returning.
|
||
const contextWindow = await mem.contextWindowFor(payload.agent, payload.signal)
|
||
const injected = await mem.contextForStep(decision.messages, payload.signal, { contextWindow })
|
||
if (injected === '') return decision
|
||
return {
|
||
...decision,
|
||
messages: [
|
||
...decision.messages,
|
||
createUserMessage({
|
||
content: [{ type: 'text', text: injected }],
|
||
// `form: 'snapshot'` matches how dsh-time-context labels its own
|
||
// injected context (dsh-time-context/lib/index.js:238). `kind` is
|
||
// this plugin's own — MessageSourceMap is merge-extensible
|
||
// (dsh-llm/lib/types/message.d.ts:96-105) and user messages accept
|
||
// unknown kinds (dsh-session/lib/index.js:1216).
|
||
source: {
|
||
kind: name,
|
||
form: 'snapshot',
|
||
sections: [{ name, text: injected }],
|
||
},
|
||
}),
|
||
],
|
||
}
|
||
},
|
||
// Prepend: recall should be visible to listeners that run later.
|
||
{ prepend: true },
|
||
)
|
||
}
|
||
|
||
registerTools(
|
||
ctx,
|
||
mem,
|
||
config.toolName ?? 'stablemem_commit',
|
||
// ★ Also overridable: tool names are unique per tool layer, and `isolate()`
|
||
// does not partition them, so two instances in one process collide unless
|
||
// BOTH names differ. Hardcoding this one re-broke the preset route as soon as
|
||
// the tool was added.
|
||
config.statsToolName ?? 'stablemem_stats',
|
||
)
|
||
|
||
return mem
|
||
}
|
||
|
||
/**
|
||
* A stable, filesystem-safe key distinguishing one plugin instance.
|
||
*
|
||
* Used to keep prompt-context names unique when a realm hosts more than one
|
||
* instance (a preset isolating `stablemem` but not the prompt layer). Derived
|
||
* from the database path so it survives restarts and stays readable in
|
||
* `/dump-config`.
|
||
*
|
||
* @param {string} path the configured database path
|
||
* @returns {string}
|
||
*/
|
||
export function instanceKey(path) {
|
||
let hash = 0
|
||
for (const char of String(path)) hash = (hash * 31 + char.codePointAt(0)) >>> 0
|
||
return hash.toString(36)
|
||
}
|
||
|
||
/**
|
||
* Default database location.
|
||
*
|
||
* Under the process cwd so a profile sees one memory per workspace, matching how
|
||
* `dsh-storage-json` refuses to guess a location by default
|
||
* (`dsh-storage-json/lib/types/index.d.ts:14-18`). Overridable via `config.path`.
|
||
*
|
||
* @returns {string}
|
||
*/
|
||
export function defaultPath() {
|
||
return `${process.cwd()}/.dsh-memory/memory.db`
|
||
}
|
||
|
||
export default { name, inject, apply, StableMem }
|