fix(memory-b): phase A 存量库迁移 + 成功路径 degraded 标签断言

⚠️ 先说明顺序问题:H 组我已在 00a4a5f 提交并推送,在你这条指令之前。
你要求「迁移是 H 的前置」,我同意,但已经发出去了。
本提交补上迁移,00a4a5f..本提交 之间是一个真实的破窗状态
(存量库会被打成只读),已记在这里,不掩饰。

阻塞① 迁移(硬 bug,我复现了)
复现:用 phase-A 形库(无 vector/fingerprint 列,user_version=0)
  opened OK                        ← 加载不报错
  PUTBLOCK THREW: table memory_blocks has no column named vector
  blockFingerprints THREW: no such column: fingerprint
根因:DDL 全是 CREATE TABLE IF NOT EXISTS,对已存在表静默 no-op。

实现(lib/store.js)
- 新增 SCHEMA_VERSION = 1 与 migrate(db)
- 探测而非猜:读 pragma_table_info('memory_blocks'),缺哪列 ALTER 哪列,
  不靠 try/catch 吞异常(半个迁移被吞掉是最难查的形态)
- user_version 为 0 不等于缺列(phase B 崩溃前未盖戳的库会是这样),
  所以判据是列存在性,不是版本号
- graph_centroid:phase A 的空表直接 DROP 重建;有数据则**显式拒绝并报错**,
  不销毁数据
- 新增 userVersion() 供读取
- open() 在 DDL 之后调用 migrate():新库已具当前形状,只有老库有活干

判据
  MIG1  phase-A 库:列补齐 → 老块 getBlock 读得到 → FTS 仍命中
        → putBlock 成功 → user_version==1 → 重开幂等
  MIG2  新库直接是 version 1
  MIG1† 不迁移时那条真实写入必须失败(并断言 phase-A 表确实无 vector 列)

变异打红原文(把 migrate(db) 注释掉)
  not ok 21 - MIG1: a phase-A database gains the vector columns and stays writable
    error: 'missing vector column; have id,modality,text_content,source,scene,semantic_type,created_at,updated_at'
  not ok 22 - MIG2: a fresh database is created at version 1
恢复 → # pass 23 / # fail 0

★ 我第一版 MIG1† 写错了:用了一条不提到 vector 的 INSERT 去断言会失败,
  而那种 INSERT 本来就会成功。已改成 phase B 实际执行的那种写(显式带
  vector/fingerprint),并加断言确认 phase-A 表确实没有该列。

阻塞② 成功路径的 degraded 标签
你复现过:把 lib/recall.js 的 'vector_ready_unused' 改回 'hybrid',
76 条全绿 —— 成功路径的标签没有任何断言。已补:

  A3c: 健康 embedder ⇒ degraded === 'vector_ready_unused',
       且断言 != 'hybrid',并确认 embedText 真被调用了一次
  A3c mutation: 交换常量必须抛 ERR_ASSERTION

真实代码变异(把标签改回 'hybrid',跑全量)
  not ok 12 - A3c: a healthy embedder reports "vector_ready_unused", NOT "hybrid"
    error: group A computes the vector but does not score with it — the label
           must say that, because group V will need a different label once
           vectors really participate
    expected: 'vector_ready_unused'
  # tests 81 # pass 80 # fail 1
恢复 → # tests 81 # pass 81 # fail 0

端到端复验(真实 phase-A 库过一遍插件)
  loaded on phase-A DB, user_version = 1
  commit: Committed block blk_5e3653f55544.
  legacy block recallable: true -> 值班室分机号 4324

验证
- npm test 81/81(76 + MIG 三条 + A3c 两条)
- npm run check 9/9
- 提交前连跑 3 次 test + 3 次 check,6 次全绿

待办
- 票③ D7 fail-closed(中心表空但有向量行 ⇒ dim 报 null)跟 F 组
This commit is contained in:
2026-10-30 10:20:10 +08:00
parent 00a4a5f6ee
commit f84011d9ef
4 changed files with 280 additions and 1 deletions

View File

@ -361,6 +361,32 @@ Failures are **hard at the adapter** and **soft at the recall layer**:
A wrong dimension is never truncated or padded — a silently mis-sized vector
produces a plausible-looking but meaningless score (`block_recall.go:135-138`).
### Schema migration
`PRAGMA user_version` gates the upgrade (currently `0 → 1`).
Phase A shipped only `CREATE TABLE IF NOT EXISTS`, which is a **silent no-op** on
an existing table. A database created by the phase-A build therefore opened
without error, recalled fine through FTS, and then threw on the **first commit**:
```
PUTBLOCK THREW: table memory_blocks has no column named vector
```
The memory store became read-only with no diagnostic at load time — a worse
failure shape than a loud error, because nothing looks wrong until someone tries
to write.
The migration probes `pragma_table_info` instead of guessing from the version
number (a version-0 database created by a crashed phase-B build may already have
the columns), then `ALTER TABLE ... ADD COLUMN` with `DEFAULT ''` — the same
"not embedded yet" sentinel `putBlock` uses. `graph_centroid` from phase A is
dropped and rebuilt, but only when it is empty; a populated one refuses loudly
rather than destroying data.
Verified end to end: a real phase-A database opens, reports `user_version = 1`,
accepts a commit, and its legacy block is still recallable.
---
## Schema
@ -455,7 +481,7 @@ because they will bite anyone writing a second dsh service plugin.
## Verify
```bash
npm test # 76 tests: store, recall, wiring, loader, isolation, regression, G6/G6b/G7, phase-B A + G + H
npm test # 81 tests: store, recall, wiring, loader, isolation, regression, G6/G6b/G7, phase-B A + G + H
npm run check # 8 checkpoints with human-readable evidence (V1–V5 + mutations)
```

View File

@ -26,6 +26,83 @@ export const MODALITY_TEXT = 'text'
/** The three tables. Kept as one exec to keep schema creation atomic-ish and
* readable next to its HologramAgent counterpart. */
/**
* Current schema version, stamped into `PRAGMA user_version`.
*
* 0 phase A: memory_blocks without vector/fingerprint
* 1 phase B: + memory_blocks.vector, + memory_blocks.fingerprint,
* graph_centroid aligned with HA + is_anomalous column
*
* ★ Why a version and not just ALTER-and-hope:
* every phase-A statement is `CREATE TABLE IF NOT EXISTS`, which is a **silent
* no-op** on an existing table. An old database therefore opens without error,
* recalls fine through FTS, and throws on the FIRST commit
* (`table memory_blocks has no column named vector`). The store becomes
* read-only with no diagnostic at load time — the worst shape a failure can
* take. The version makes the upgrade explicit and testable.
*/
export const SCHEMA_VERSION = 1
/**
* Bring an existing database up to {@link SCHEMA_VERSION}.
*
* Probes the actual column list rather than guessing or try/catching: a
* `user_version` of 0 is not proof that the columns are absent (a database
* created by a phase-B build that crashed before stamping would look
* version-0 while already having them), and relying on ALTER throwing is how
* half-applied migrations become invisible.
*
* @param {import('node:sqlite').DatabaseSync} db an open handle
* @returns {number} the version after migrating
*/
function migrate(db) {
let version = db.prepare('PRAGMA user_version').get().user_version
if (version < 1) {
const existing = new Set(
db.prepare(`SELECT name FROM pragma_table_info('memory_blocks')`).all().map((row) => row.name),
)
if (existing.size === 0) {
// No table at all: DDL below creates it at the current shape. Nothing to
// migrate, and stamping the version is still correct.
version = SCHEMA_VERSION
} else {
// SQLite requires a non-NULL default when adding a NOT NULL column to a
// populated table; '' is the same "not embedded yet" sentinel putBlock uses.
if (!existing.has('vector')) {
db.exec("ALTER TABLE memory_blocks ADD COLUMN vector TEXT NOT NULL DEFAULT ''")
}
if (!existing.has('fingerprint')) {
db.exec("ALTER TABLE memory_blocks ADD COLUMN fingerprint TEXT NOT NULL DEFAULT ''")
}
version = 1
}
}
// graph_centroid changed shape between "created empty in phase A" and phase B.
// Phase A created it with (fingerprint, dim, count, sum_json, updated_at); an
// empty one is simply dropped so DDL rebuilds it. A populated one never
// happens in practice (phase A never wrote a centroid), so refuse loudly
// instead of destroying data.
const centroidColumns = db
.prepare(`SELECT name FROM pragma_table_info('graph_centroid')`)
.all()
.map((row) => row.name)
if (centroidColumns.length > 0 && centroidColumns.includes('sum_json')) {
const rows = db.prepare('SELECT COUNT(*) AS n FROM graph_centroid').get().n
if (rows > 0) {
throw new Error(
`dsh-memory: refusing to migrate graph_centroid: phase-A shape with ${rows} row(s). ` +
'Phase A never wrote a centroid, so this is unexpected — export the data before upgrading.',
)
}
db.exec('DROP TABLE graph_centroid')
}
db.exec(`PRAGMA user_version = ${SCHEMA_VERSION}`)
return SCHEMA_VERSION
}
const DDL = `
CREATE TABLE IF NOT EXISTS memory_blocks (
id TEXT PRIMARY KEY,
@ -170,6 +247,9 @@ export class BlockStore {
if (this.#wantsWal && this.#path !== ':memory:') db.exec('PRAGMA journal_mode = WAL')
db.exec('PRAGMA foreign_keys = ON')
db.exec(DDL)
// After DDL: a brand-new database now has the current shape, so the
// migration only has work to do for databases that predate phase B.
migrate(db)
this.#db = db
return this
}
@ -397,6 +477,11 @@ export class BlockStore {
return db.prepare(`SELECT name FROM pragma_table_info(?)`).all(table).map((row) => row.name)
}
/** @returns {number} the schema version recorded in `PRAGMA user_version` */
userVersion() {
return this.#handle().prepare('PRAGMA user_version').get().user_version
}
/** @returns {string} the journal mode actually in force */
journalMode() {
return this.#handle().prepare('PRAGMA journal_mode').get().journal_mode

View File

@ -194,3 +194,53 @@ test('A3 mutation: a wrapper that adds any suffix to `why` must break A3', async
store.close()
}
})
// ── the SUCCESS path's label (group A: vector computed, not yet used) ──────
//
// The earlier A3b only pinned the two failure labels. Changing the success label
// back to 'hybrid' left all 76 tests green — a label nobody asserts is a label
// that will drift. It matters because group V must distinguish "vector computed
// but not in scoring" (group A) from "vector actually participating" (group V);
// if both say 'hybrid', nobody can tell which build they are looking at.
test('A3c: a healthy embedder reports "vector_ready_unused", NOT "hybrid"', async () => {
const store = seededStore()
try {
let embedCalls = 0
const healthy = {
async embedText() {
embedCalls += 1
return [0.1, 0.2, 0.3]
},
info: () => ({ fingerprint: 'fp-healthy', dimension: 3 }),
}
const hits = await recallBlocksHybrid(store, healthy, '4324', { topK: 1 })
assert.equal(embedCalls, 1, 'the embedder must actually be called')
assert.equal(
hits.degraded,
'vector_ready_unused',
'group A computes the vector but does not score with it — the label must say that, ' +
'because group V will need a different label once vectors really participate',
)
assert.notEqual(hits.degraded, 'hybrid', "calling it 'hybrid' would be a lie until group V lands")
// And the result is still the pure-symbol ordering.
assert.equal(hits[0].text.includes('4324'), true)
} finally {
store.close()
}
})
test('A3c mutation: labelling the success path "hybrid" must fail', () => {
// Mirrors the exact one-line edit that used to slip through: with no assertion
// on this value, reverting the rename was invisible.
const LABELS = { groupA: 'vector_ready_unused', groupV: 'hybrid' }
// The test above asserts hits.degraded === LABELS.groupA. Prove that swapping
// the constant would break it:
assert.notEqual(LABELS.groupA, LABELS.groupV)
const simulatedDeployed = LABELS.groupV // what the mutated lib returns
assert.throws(
() => assert.equal(simulatedDeployed, LABELS.groupA),
(err) => err.code === 'ERR_ASSERTION',
'if lib returns "hybrid", the A3c assertion must throw',
)
})

View File

@ -22,7 +22,11 @@
*/
import assert from 'node:assert/strict'
import { mkdtempSync, rmSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import test from 'node:test'
import { DatabaseSync } from 'node:sqlite'
import { createHttpEmbedder, deriveFingerprint, normalizeEndpoint } from '../lib/embed/http.js'
import { BlockStore } from '../lib/store.js'
import { recallBlocksHybrid } from '../lib/recall.js'
@ -345,3 +349,117 @@ test('ticket②: re-committing without a vector must not erase the stored one',
store.close()
}
})
// ── migration: a phase-A database must keep working ───────────────────────
//
// Phase A shipped `CREATE TABLE IF NOT EXISTS` with no `vector`/`fingerprint`
// columns. `open()` re-runs that DDL, which is a SILENT no-op on an existing
// table. So a database created by the phase-A build opens fine, recalls fine via
// FTS, and then throws on the FIRST commit:
//
// PUTBLOCK THREW: table memory_blocks has no column named vector
//
// The memory store becomes read-only with no error at load time. Worse than the
// G5 bug: a false alarm gets noticed, this one does not.
/** Build a database with exactly the phase-A schema (no vector/fingerprint). */
function buildPhaseADatabase(path) {
const db = new DatabaseSync(path)
db.exec(`CREATE TABLE IF NOT EXISTS memory_blocks (
id TEXT PRIMARY KEY,
modality TEXT NOT NULL,
text_content TEXT NOT NULL DEFAULT '',
source TEXT NOT NULL DEFAULT '',
scene TEXT NOT NULL DEFAULT '',
semantic_type TEXT NOT NULL DEFAULT '',
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL)`)
db.exec(`CREATE TABLE IF NOT EXISTS memory_block_edges (
id INTEGER PRIMARY KEY AUTOINCREMENT, source_kind TEXT NOT NULL, source_id TEXT NOT NULL,
target_kind TEXT NOT NULL, target_id TEXT NOT NULL, type TEXT NOT NULL,
confidence REAL NOT NULL DEFAULT 0, status TEXT NOT NULL DEFAULT '', created_at INTEGER NOT NULL)`)
db.exec(`CREATE VIRTUAL TABLE IF NOT EXISTS memory_blocks_fts
USING fts5(text_content, block_id UNINDEXED, tokenize='unicode61 remove_diacritics 2')`)
db.exec(`INSERT INTO memory_blocks (id, modality, text_content, source, created_at, updated_at)
VALUES ('blk_legacy', 'text', '值班室分机号 4324', 'legacy', 1, 1)`)
db.exec(`INSERT INTO memory_blocks_fts (text_content, block_id) VALUES ('值班室分机号 4324', 'blk_legacy')`)
db.close()
}
test('MIG1: a phase-A database gains the vector columns and stays writable', () => {
const dir = mkdtempSync(join(tmpdir(), 'mig-'))
const path = join(dir, 'phase-a.db')
try {
buildPhaseADatabase(path)
const store = new BlockStore({ path }).open()
// 1. Columns exist after open.
const columns = store.columns('memory_blocks')
assert.ok(columns.includes('vector'), `missing vector column; have ${columns.join(',')}`)
assert.ok(columns.includes('fingerprint'), `missing fingerprint column; have ${columns.join(',')}`)
// 2. The legacy block is still readable, and still findable by FTS.
const legacy = store.getBlock('blk_legacy')
assert.ok(legacy, 'the phase-A block must survive')
assert.equal(legacy.text_content, '值班室分机号 4324')
assert.equal(legacy.vector, '', 'a pre-existing block has no vector, not a fabricated one')
const hits = store.searchFts('"4324"', 5)
assert.ok(hits.length >= 1, 'FTS must still find the legacy block')
// 3. Writing works — this is what used to throw.
const { created } = store.putBlock({ text: '迁移之后的新块 8499' })
assert.equal(created, true, 'commit must work after migration')
// 4. The version is stamped.
assert.equal(store.userVersion(), 1, 'schema version must be recorded')
// 5. Re-opening is idempotent (migration must not re-run destructively).
store.close()
const again = new BlockStore({ path }).open()
assert.equal(again.userVersion(), 1)
assert.equal(again.countBlocks(), 2)
again.close()
} finally {
rmSync(dir, { recursive: true, force: true })
}
})
test('MIG2: a fresh database is created at version 1', () => {
const store = new BlockStore({ path: ':memory:' }).open()
try {
assert.equal(store.userVersion(), 1, 'a new database starts current, not at 0')
} finally {
store.close()
}
})
test('MIG1 mutation: skipping the ALTER leaves the store unwritable', () => {
// Prove the migration is load-bearing: build the phase-A schema, do NOT
// migrate, and show the write fails with the exact error users would hit.
const dir = mkdtempSync(join(tmpdir(), 'mig-mut-'))
const path = join(dir, 'phase-a.db')
try {
buildPhaseADatabase(path)
const raw = new DatabaseSync(path)
// The FAILING write is the one phase B actually performs — it always names
// `vector` and `fingerprint`. That is why the failure is a *runtime* error
// a long way from load, and why an unwritable store hides behind a clean
// start. (A plain insert that names neither column succeeds, which is
// exactly the trap: the store looks alive right up to the first real write.)
assert.throws(
() =>
raw.exec(
"INSERT INTO memory_blocks (id, modality, text_content, vector, fingerprint, created_at, updated_at) " +
"VALUES ('x','text','y','','fp',1,1)",
),
(err) => /no column named|has no column/i.test(err.message),
'skipping the migration must fail at write time',
)
// And prove the surviving column list is the phase-A one.
const cols = raw.prepare(`SELECT name FROM pragma_table_info('memory_blocks')`).all().map((r) => r.name)
assert.ok(!cols.includes('vector'), 'a phase-A table has no vector column')
raw.close()
} finally {
rmSync(dir, { recursive: true, force: true })
}
})