HologramAgent-DSH-Memory-Port

把 HologramAgent 的记忆系统移植到 dsh 插件。本仓分两部分：

1. 调研报告（已交付）
   • docs/feasibility.md — 三层记忆移植可行性评估（结论：只能做子集，L2+L3 可移植，L1 抓不住）
   • docs/shadow-context.md — 影子上下文方案核实（结论：路线② preset 可行，路线① 只是看着可行）
2. MVP 阶段 A 实现（本文件以下内容）— 1–2 天切片：块能进 dsh、能被召回、能进 prompt，不引入 embedding

---

dsh-memory（阶段 A）

HologramAgent-style block memory for dsh, as a dsh plugin. This is phase A of the MVP described in docs/feasibility.md §7.1 — the smallest slice that proves the whole chain works:

commit a block  →  store it durably  →  recall it  →  get it into the model's prompt

Deliberately not included: embedding providers, vectors, the L1 eviction flow, media blocks. Each is blocked or out of scope for a 1–2 day slice; see What this slice does not do.

---

★ Read this before touching injection

dsh has two injection hooks that look interchangeable and are not. Getting this wrong silently corrupts the conversation log.

hook	receives this turn's input?	durable side effect
ctx.systemPrompt.context({ text })	No — AssembleContext is {scope, signal} + a merged-in agent	appends a runtime-context snapshot
ctx.on('agent/pre-step')	Yes — payload carries decision.messages	every message you return is appended as a durable user/message

Evidence, verified in the installed harness (not inferred):

• dsh-system-prompt/lib/types/index.d.ts:37-44 — AssembleContext has only scope and signal (agent is added by dsh-agent's declaration merging; still no messages).
• dsh-agent-loop/lib/index.js:902-934 — preStep() claims the inbox, calls systemPrompt.assemble(), then runs the agent/pre-step waterfall with those messages.
• dsh-agent-loop/lib/index.js:1066-1073 — the loop then does session.append('user/message', message, { surfaceOp: 'append' }) for each returned message.

Consequences, and how this plugin uses them:

• The input-independent index (a block count, "memory exists, here's the tool") goes through systemPrompt.context. It cannot see the current question, so it must not pretend to be a recall result.
• Per-turn recall goes through agent/pre-step, because that is the only hook that sees the input. Recall is prepend: true so it runs before listeners that may want the final word.
• Because pre-step output is durable, injected recall becomes part of the conversation history. That is acceptable here (it is honest: the model saw it), but it means this plugin is not a shadow-context mechanism. For that distinction see docs/shadow-context.md §1.3.

★ A real batch is not just the question

decision.messages is the batch admitted for the step, and in a real dsh profile the human question is followed by other user-role messages:

[ user question ][ runtime-context snapshot ][ skill catalog reminder ]

All three carry role: 'user'. So "use the last user message as the recall query" returns the skill catalog, and every real turn recalls nothing.

This plugin's first release had exactly that bug. It was found by running the plugin in a real headless profile, not by the test suite — the unit tests built 2-message batches, so they never saw it (all checkpoints were green while auto-recall was 100% dead).

lastUserText therefore filters by source.kind, not by role:

Allowed (the recall query may come from these):

kind	what
user	every human prompt, including remote/browser ones

Everything else is injected. The rule is an allowlist over source.kind, so an unlisted — including a future — kind fails closed.

Two facts make that strict allowlist safe:

1. user is the kind for all human prompts, local and remote. A browser prompt arrives under a different source key (user-rpc) but the same kind:

   // dsh-api-session-controller/lib/types/types.d.ts:395-400
   'user-rpc': { kind: 'user'; rpcId: SessionRequestId;
                 clientTimeZone?: string; principal?: Principal }
   

   So filtering on kind (not on source key, and not on "has no extra fields") cannot lock out web users. G6b pins this.
2. Injected context always declares its own kind. Examples verified as literal declarations in the installed harness:

   kind	source
   runtime-context	dsh-agent-loop/lib/types/runtime-context.d.ts:13
   skill-catalog	dsh-tool-skill/lib/types/index.d.ts:29
   skill-invocation	dsh-skill/lib/types/index.d.ts:131
   schedule	dsh-schedule/lib/types/runtime.d.ts:5
   agent-message	dsh-subagent/lib/types/continuation-messages.d.ts:37
   subagent-settled	dsh-subagent/lib/types/continuation-messages.d.ts:38
   agent-instructions	dsh-agent-instructions/lib/types/state.d.ts:27
   time-context	dsh-time-context/lib/types/index.d.ts:14
   tool-goal, tool-jobs	dsh-tool-goal/…/index.d.ts:12, dsh-tool-jobs/…/index.d.ts:16
   user-approval	dsh-user-approval/lib/types/index.d.ts:14
   plan-mode, ptc-mode	dsh-plan-mode/…/index.d.ts:31, dsh-tools/lib/types/ptc.d.ts:14
   …and more	see the G6 table

Scope of the test table — read this before trusting it. G6 covers every user-role injected kind declared in the currently installed harness (23 of them). It is not a claim of exhaustiveness: the full MessageSourceMap surface is larger and grows with each release. That is fine, because the allowlist fails closed, so an unlisted kind is safe by construction. What G6 actually buys is: a rename in the harness breaks a test here rather than silently killing recall in production.

G7 separately pins that a message with no source at all yields ''. Reachability, stated precisely: currently unreachable — assertMessageEventShape requires a non-empty source.kind and runs on both the append path (dsh-session/lib/index.js:1089) and the seed/restore path (:1165). It is therefore defense in depth against a future format change, not a live bug.

Measured effect (4 seeded notes, three batch shapes):

batch shape	old rule (last role=user)	fixed rule (last human kind)
2-part (the old test shape)	1 hit	1 hit
3-part (+runtime-context)	0 hits	1 hit
4-part (+runtime-context +skill-catalog)	0 hits	1 hit
total	1 / 3	3 / 3

Guarded by these tests in test/plugin.test.js:

test	what it pins
regression: extraction	the question, not the injection, is returned
regression: end-to-end	a real 3-part batch actually recalls (this was 0 hits)
regression mutation	reintroducing "last role=user" drops hits to 0
G6	all 23 user-role injected kinds in the installed harness are rejected (parameterised over literal declarations)
G6 mutation	"role is user ⇒ accept" must fail on all 23
G6b	a user-rpc prompt (kind: 'user' + rpcId/principal) is accepted — the allowlist does not lock out remote users
G7	a message with no source yields ''
G7 mutation	dropping the guard admits it, so the guard is load-bearing

★ The lesson: a 2-message batch is a fake batch

The bug above shipped with every checkpoint green. The reason is narrower than "tests were missing":

V3b built its batch as [question] — or [question, injected]. A real dsh step appends injected context after the question, so the very shape that made the bug invisible was the shape the test chose.

Hence a rule for anything added from here on: any test that exercises agent/pre-step must build its batch through realBatch(), which always appends at least a runtime-context and a skill-catalog message after the question. A batch of one human message proves the hook fires; it does not prove the hook reads the right thing. V3b is retained (it still pins injection attribution) but is flagged V3b⚠ as a known blind spot.

There is no hook for "the context is about to overflow, please evict something", and no waterfall that can rewrite a loop-built request (llm/stream receives a deep-frozen GenerateOptions for loop requests — dsh-llm/lib/types/index.d.ts:37-42).

---

Install

# from this repo
npm install                    # nothing to install; dsh packages are peers

The plugin needs node:sqlite (built into Node ≥ 22.5). On Node 22.23 it works behind an ExperimentalWarning only — no --experimental-sqlite flag.

Mount it in a dsh composition:

- id: memory
  name: dsh-memory
  config:
    path: /path/to/memory.db     # optional; defaults to <cwd>/.dsh-memory/memory.db
    topK: 3                      # optional; recall hits injected per turn
    inject: true                 # optional; false = tool only, no auto-recall
    toolName: stablemem_commit   # optional; must be unique per tool layer

Use

The model gets one tool:

stablemem_commit({ text, source?, scene? })

and, once at least one block exists, a constant prompt context line telling it that durable memory is available.

Recall runs automatically each turn: the last user message is used as the query, FTS5 + exact-symbol pinning selects up to topK blocks, and they are injected as one attributed user message:

[Memory · dsh-memory] Recalled from durable block memory — these are PAST notes, not the current conversation.
[1] (exact 1.01) 值班室分机号 4324，值班人 老周

Programmatic use (another plugin, or a preset subtree):

ctx.stablemem.commit({ text: 'etcd 集群端口 2379', source: 'ops' })
ctx.stablemem.recall('etcd 端口', 5)
ctx.stablemem.link({ sourceKind: 'block', sourceId: a, targetKind: 'block', targetId: b, type: 'supersedes' })

Phase B · group A — the async seam

recallBlocks stays synchronous and unmodified; vectors get a separate async wrapper beside it:

recallBlocks(store, query, opts)                  // sync, unchanged, no embedder
recallBlocksHybrid(store, embedder, query, opts)  // async wrapper

Why the split rather than "make recallBlocks async": every existing caller keeps calling a sync function, so the 9 test call sites and the whole phase-A surface need no edit, and the golden fixture proves the sync path did not move.

With no embedder (or a failing one) the wrapper is exactly recallBlocks, and it reports why it degraded on the result array itself:

const hits = await recallBlocksHybrid(store, null, q)
hits.degraded        // 'no_embedder' | 'embed_error' | 'vector_ready_unused'

degraded is non-enumerable, so it does not appear in JSON.stringify, Object.keys, or deep-equality against phase-A output. It never throws — a dead embedding service must not take recall down (§0.4).

'vector_ready_unused' is deliberate naming: group A computes the query vector but does not feed it into scoring yet, so calling that state 'hybrid' would be a lie that outlives its reason. Group V renames it when vectors actually participate.

Because degraded is non-enumerable it is invisible to any serialization. That makes it fine for tests and useless for the "degradation must be visible" requirement — group D therefore renders counters into the prompt text, not onto an object property.

Guarded by test/phase-b-a.test.js (6 tests) against test/fixtures/golden-phase-a.json: 12 blocks × 15 queries × topK 1/3/5 = 45 cases, comparing id, order, score (exact, not approximate), exact flag, and the why string. A tolerance-based comparison would let a scoring change through, which is the one thing this group exists to stop.

A trap this group found: making Svc.recall() async silently disarmed a regression mutation — await-ing was missing, undefined.length is undefined, and undefined < undefined is false, so an assertion comparing hit counts passed while measuring nothing. Both that test and checkpoint V3c now await. If a future change makes anything else async, grep for .length on a value that may have become a Promise.

Phase B · group G — guards

test/phase-b-g.test.js (behavioural) and test/phase-b-g-static.test.js (source-text) pin the properties that keep vectors from damaging phase A:

#	guard	mutation that must turn it red
G2	no embedder ⇒ zero network calls (global fetch replaced with a tripwire)	add a fetch to the no-embedder branch
G2b	lib/ imports only node: builtins + dsh peers	add onnxruntime-node
G3	stablemem_commit still writes when the embedder throws; symbol recall still finds it	make commit await the embedder and reject
G4	only the text modality; no media columns exist	add a data BLOB column
G5	transport identifiers (fetch(, Authorization, Bearer, http://, model_dir, api_key) may appear only under lib/embed/	put an endpoint literal in recall.js
G2†	every early-degradation path of recallBlocksHybrid is network-free (6 paths, data-driven)	add a fetch to any one of them
G2†b	the fingerprint check runs before embedText	move the call earlier

Two things this group taught us, both about guards rather than about vectors:

1. A source-scanning guard must not import the code it scans. G2b/G5 first lived beside the behavioural guards, which import lib/index.js. A broken lib/ then crashed the whole file, so the report was "load failed" instead of "you added a disallowed dependency". They now live in test/phase-b-g-static.test.js, which imports nothing from lib/ and still names the offender when lib/ cannot load at all.
2. Comment-aware scanning or none. G5's first version flagged "no endpoint" in a comment and "edge endpoint" in store.js. A guard that cries wolf gets disabled, which is worse than not having it. It now strips comments before scanning.

Phase B · group H — HTTP embedding adapter

lib/embed/http.js is the only file allowed to know a transport (enforced by G5). Protocol is byte-compatible with HologramAgent's http provider (internal/memory/vector/http_embedder.go:72-87), so an existing HA sidecar works without a shim:

// request
{ "model": "…", "modality": "text", "side": "query", "text": "…" }
// response — either shape accepted
{ "embedding": [ … ] }        { "data": [ { "embedding": [ … ] } ] }

Enable it:

- id: memory
  name: dsh-memory
  config:
    embed:
      endpoint: http://127.0.0.1:18999/embed
      dimension: 2048
      model: jina-v5
      # apiKey / fingerprint / timeoutMs optional

Omit embed entirely to run symbol-only, exactly as phase A did.

Fingerprint: http:<model>:<dim>:<sha256(normalised endpoint)[:8]>

HA derives http:<model>:<dim> (http_embedder.go:99) — model and dimension only. That is unsafe once the endpoint is configurable: two deployments of the same model name at the same dimension produce the same fingerprint but different vectors, so their blocks compare silently and wrongly. The endpoint hash closes that. It is computed once at construction and reused, because the same string must reach both tables:

Ticket ② constraint. memory_blocks.fingerprint and graph_centroid.fingerprint must be byte-identical for the same space. HA looks the centroid up by fingerprint (block_recall.go:92), so two spellings make the lookup miss with no error — centering just never applies and scores quietly degrade. Pinned by the two-table fingerprint test, whose mutation adds a single trailing space to prove the lookup misses.

Endpoint normalisation (before hashing, so cosmetic differences don't each cause a full re-embed): trim, lowercase scheme+host, drop trailing /, drop an explicit default port. It deliberately does not reorder query strings or resolve ..: those differences are usually real.

Failures are hard at the adapter and soft at the recall layer:

layer	behaviour
adapter	non-2xx / no embedding array / wrong dimension / oversized body / timeout ⇒ throws
recallBlocksHybrid	catches, sets degraded: 'embed_error', answers from FTS5

A wrong dimension is never truncated or padded — a silently mis-sized vector produces a plausible-looking but meaningless score (block_recall.go:135-138).

---

Schema

Three tables, mirroring HologramAgent's (internal/memory/graph.go:2577, :320, centroid_store.go:58), restricted to the text modality:

table	purpose	notes
memory_blocks	the block payload	text_content is the body; source / scene / semantic_type mirror HA's columns
memory_block_edges	structural + semantic edges	unique per (source_kind, source_id, target_kind, target_id, type)
graph_centroid	vector-space centering	created empty — phase B writes it; creating it now avoids a migration
memory_blocks_fts	FTS5 index over text_content	unicode61 remove_diacritics 2

memory_blocks.id is content-addressed (blk_ + sha256(text)[:12], ported from distill.BlockID, internal/memory/distill/blocks.go:99), so re-committing identical text is idempotent and never blanks existing metadata.

---

Recall design, and why there are no vectors yet

This slice ports HologramAgent's symbol path, not its vector path.

docs/zh/recall-capability-tiers.md measures four query shapes. The one that works best in production is 形态 1 (queries carrying an exact token: port, IP, path, version) — which is pure symbol matching. The one that needs vectors is 形态 3 (pure generic description), and it is the weakest even with vectors (target block ranked 389/2686). So dropping embeddings here costs the tier that was already marginal, not the tier that carries the product.

Scoring follows HA exactly where it matters:

• An exact numeric hit is a boolean pin (1.0 + lexical*0.01), never a weight. Folding it into a weighted sum makes it lose to noise — the regression documented at internal/memory/fusion.go:43-49.
• A matched word is a coverage signal, not an exact hit.
• Single-character symbols are rejected (abstain.go:115 "单字不成符号"); without that gate a query like billing0 yields a bare 0 that "exactly" matches every block containing a zero.
• A matched word prefers blocks where it is not a prefix of a longer word (billing should beat billing2), loosely mirroring isCrossWordTailMatch (distill/split.go:383).

Two documented gaps, asserted rather than hidden (see checkpoint V2):

gap	why	when
纯泛化描述 queries ("公网地址是什么")	needs vectors	phase B
「库里没有」detection (abstention)	needs either vectors or a real CJK tokenizer	phase B/C
CJK segmentation	unicode61 keeps a CJK run as ONE token; compensated by a 2..4-char sliding-window fallback ported from cjkWindows (fusion.go:240)	phase C

---

Three harness constraints this slice discovered

Each was found by a failing test, not by reading types — they are recorded here because they will bite anyone writing a second dsh service plugin.

1. A cordis service must not use JS #private fields. ctx.stablemem is a proxy, not the instance you constructed, and private fields are bound to the declaring object: Cannot read private member #store from an object whose class did not declare it. Keep service state in ordinary properties.

2. isolate() does not partition prompt-context names or tool names. It partitions service lookups. Two instances in one realm still collide on systemPrompt.context({name}) (dsh-system-prompt/lib/index.js:204) and tools.register unless the names differ. Hence the configurable toolName, and the prompt-context name derived from the database path (memory:index:<key>), which stays unique and stable.

3. ctx.systemPrompt.getContextOrder() is an allocator, not a free constant. Prefer a named slot (WEB_SURFACE) over a magic number; the numeric constants are private to the package.

---

What this slice does not do

not done	why	where it lands
embeddings / vectors	no embedding service in dsh (docs/feasibility.md §5.1)	phase B
evicting L1 context into memory	dsh's compaction is summary-replacement, and CompactionTrigger carries no query (§5.3)	replaced by shadow-context + preset
media blocks	dsh attachment is CAS; HA requires bytes inside the block (§5.4)	not planned
cross-session / cross-agent memory	one sqlite file per config.path	phase D
abstention	needs vectors or a real tokenizer	phase B/C

---

Verify

npm test          # 76 tests: store, recall, wiring, loader, isolation, regression, G6/G6b/G7, phase-B A + G + H
npm run check     # 8 checkpoints with human-readable evidence (V1–V5 + mutations)

npm run check prints one line of evidence per checkpoint. The mapping to docs/feasibility.md §7.1:

checkpoint	proves
V1	node:sqlite opens, WAL applies, 100 writes round-trip
V2	recall quality per capability tier: tiers 1–2 must pass, tiers 3–4 pinned as gaps
V3	systemPrompt.context exposes agent and cannot see this step's messages
V3b	agent/pre-step injection lands in the accepted batch with our source.kind
V3b⚠	known blind spot: V3b's batch is 2 messages, so it cannot detect the real-batch bug. The regression tests in test/plugin.test.js cover it — see above
V3b†	mutation — silencing recall makes V3b fail (the test can turn red)
V4	tools.register schema + render contract, executed end to end
V5	fiber disposal closes the handle; the file is reopenable
A5†	mutation — containment-only recall loses discrimination (why FTS is primary)

Plus three regression tests for the real-batch bug described above (test/plugin.test.js): extraction, end-to-end 3-part batch, and a mutation that reintroduces the old rule and asserts the hit count falls to zero.

Expectations from a real run

  ✔ V1  journal=wal, rows=100, mid-batch row=blk_cad13afe6824, file on disk
  ✔ V2  tier1(exact) 30/30 = 100% · tier2(domain) 20/20 = 100%
        · known gaps: 泛化查询→0 hit, 库里没有→0 hit (需向量，phase B)
  ✔ V3  AssembleContext keys = [agent] — no messages/input field
  ✔ V3b batch 2 msgs; injected kind=dsh-memory, carries "4324"
  ✔ V4  1 tool(s) visible; execute → "Committed block blk_0be019e8b0e4."
  ✔ V5  fiber dispose → handle closed; reopened same file successfully

8/8 checkpoints passed

---

Ported from HologramAgent

Source is read-only reference (/home/program/TrueAgent); nothing is vendored.

this repo	HologramAgent
lib/store.js BlockStore	memory.GraphDB.PutMemoryBlocks (internal/memory/block.go:203)
lib/store.js blockId	distill.BlockID (internal/memory/distill/blocks.go:99)
lib/recall.js querySymbols	memory.QuerySymbols (internal/memory/fusion.go:145)
lib/recall.js scoring / pin	memory.SymbolScore (fusion.go:284), pin at :427
lib/recall.js isNumericSymbol	fusion.go:316
lib/recall.js cjkNeedles	cjkWindows (fusion.go:240)
lib/recall.js single-char gate	AbstainCheck guard (abstain.go:115)
lib/index.js renderIndex header	crossModalMarkdown (internal/agent/core/crossmodal.go:256)

License

MIT