diff --git a/assets/docs/en/ARCHITECTURE.md b/assets/docs/en/ARCHITECTURE.md index fdb9b90e..32a64361 100644 --- a/assets/docs/en/ARCHITECTURE.md +++ b/assets/docs/en/ARCHITECTURE.md @@ -207,19 +207,29 @@ single typo silently breaks reference binding with no error anywhere in the chai also gained `sentence_text`: media references hang off a sentence, so with no sentence there is nowhere to attach them. -### Media Memory (since v1.2.0: first-class memory blocks) +### Media Memory (block union: media *is* a block) -Media is not attached content but a **first-class memory node**: `internal/memory/media/` is a -content-addressed store (CAS), and graph `block` nodes carry its digest plus its own vector, while -structural edges (e.g. `sentence --contains--> block`) express ownership. +Media is neither attached content nor a separate storage layer — it is **one of the values a block +can take**. `MemoryBlock` is a union: a text block carries its payload in `Text`, a media block in +`Data` (plus MIME/width/height/size). The bytes live and die with the block — same row, same block +value, with no independent addressing layer. + +> ★ 2026-10-11 CAS removal: media bytes used to live in a separate **content-addressed store** +> (`internal/memory/media/`, with `media.db` + `blobs/`, a `ref_count` ledger, a second vector set +> and a second retrieval path `QueryMediaScored`), with blocks holding only a digest pointing at it. +> Two stores inevitably demand patches (`crossmodal.go` needed `heldMediaDigests()` to stop +> retrieving "media with nowhere to belong"), and the dual digest/block-ID identity made +> cross-layer references easy to misalign. That layer was removed wholesale: media travels with +> the block, and the block ID *is* the media's identity. | Concern | Approach | Why | |---|---|---| -| Addressing | sha256 digest; metadata in SQLite, blobs on disk (`blobs//`, two-level fanout) | Identical bytes stored once; metadata must be queryable, blobs must not live in the database | -| Integrity | Every `Get` re-verifies the digest | Silently returning corrupt data on disk damage is far worse than an error | -| Write atomicity | `.tmp` + rename | A half-written file taken as complete content would permanently poison that digest | -| Retrieval | Blocks carry **their own multimodal vector and fingerprint** and are searched directly | No description text is needed as an intermediary | -| Lifecycle | **No separate GC, no refcounts, no keep-set**; deleting the block deletes the content | Media is a memory node, not a cache that needs keeping alive | +| Payload ownership | Bytes in `memory_blocks.data` (BLOB), living and dying with the block | A block carries either text or bytes; there is no byte store outside the block | +| Block identity | **Content-derived** `blk_media_` | Same content ⇒ same ID ⇒ `ON CONFLICT` dedupes naturally (the benefit CAS once provided, now supplied directly by the block ID) | +| Retrieval | Blocks carry **their own multimodal vector and fingerprint**, participating directly in `RecallBlocks` vector search | Pure vector retrieval never touches payloads; a payload is fetched on demand via `BlockData` only after a block is selected | +| Fetching bytes | `BlockData(blockID)` loads one block on demand; plugins call `Memory().MediaBytes(ref)` with a block reference | `blockColumns` deliberately excludes `data`: including payloads would read tens of MB on every full-table recall | +| Lifecycle | **No separate GC, no refcounts, no keep-set**; deleting the block deletes the content | Media is a memory block, not a cache that needs keeping alive | +| Knowledge media | Bytes travel as data carried by the entry, stored in the entry directory (beside `content.md` and `.media.json`) | `Remove`'s `RemoveAll` cleans it up naturally, leaving no orphans | **Description-based indexing is gone**: the old implementation embedded a `[ ] ` marker in the body and treated the description as the @@ -228,14 +238,11 @@ second-hand model output, and retrieving "someone else's paraphrase of an image" than retrieving the image's own vector. Images are now retrieved only by their own vector in the unified space, and no media marker is written into the body. -**Cross-space vector migration**: media rows store their vector together with `vec_model` (the space -fingerprint). At startup `reembedStaleMedia()` recomputes and **writes back** every row whose -`vec_model` is empty (never embedded) or differs from the current space (model/dimension switched). -Modalities outside the space return `ErrModalityUnsupported` — the kernel **never substitutes -another model's vector**. - -The media store is **optional throughout**: with `core.memory.media.enabled=false` or no -configuration, the whole chain silently degrades to plain-text behaviour — no errors, no panics. +**Cross-space vector migration**: blocks store their vector together with a `fingerprint` (the space +fingerprint). At startup `reembedStaleMedia()` recomputes and **writes back** every block whose +`vector` is empty/`'null'` or whose fingerprint differs from the current space (model/dimension +switched), fetching bytes on demand via `BlockData`. Modalities outside the space return +`ErrModalityUnsupported` — the kernel **never substitutes another model's vector**. ### Other Memory Layers diff --git a/assets/docs/zh/ARCHITECTURE.md b/assets/docs/zh/ARCHITECTURE.md index caa02348..05ad46d9 100644 --- a/assets/docs/zh/ARCHITECTURE.md +++ b/assets/docs/zh/ARCHITECTURE.md @@ -203,32 +203,37 @@ eventLoop() → processTextInput() 要求调用方知道格式，等于让一个拼写错误静默切断引用绑定而全链路无人报错。 `memory_commit` 同时新增 `sentence_text`：媒体引用挂在句子上，没有句子就无处可挂。 -### 媒体记忆（v1.2.0 起：一等记忆块） +### 媒体记忆（块 union：媒体即块） -媒体不是外挂内容，而是**记忆的一等节点**：`internal/memory/media/` 是内容寻址仓储（CAS）， -图数据库里的 block 节点携带它的 digest 与向量，结构边（如 `sentence --contains--> block`）表达归属。 +媒体不是外挂内容，也不是另建的存储层，而是**块自身的一种取值**。 +`MemoryBlock` 是一个 union：文本块的负载在 `Text`，媒体块的负载在 `Data`（配 MIME/宽高/大小）。 +字节随块同生同灭——同一个 row、同一个块值，没有独立的寻址层。 + +> ★ 2026-10-11 CAS 淘汰：此前媒体字节存在独立的**内容寻址仓储**（`internal/memory/media/`， +> 带 `media.db` + `blobs/`、`ref_count` 引用账本、第二套向量与第二套召回 `QueryMediaScored`）， +> 块只留一个 digest 指向它。两套存储必然要打补丁（`crossmodal.go` 曾靠 `heldMediaDigests()` +> 防召回“无处可归的媒体”），且 digest 与块 ID 双身份让跨层引用容易错位。 +> 该层已整体拆除：媒体跟着块走，块 ID 就是媒体的身份。 | 关注点 | 做法 | 为何 | |---|---|---| -| 寻址 | sha256 digest；元数据在 SQLite，blob 在磁盘（`blobs/<前2位>/<其余>` 两级分桶） | 相同字节只存一份；元数据要可查询，blob 不该进数据库 | -| 完整性 | 每次 `Get` 重校 digest | 磁盘损坏时静默返回脏数据比报错危险得多 | -| 写入原子性 | `.tmp` + rename | 半个文件被当成完整内容会永久污染那个 digest | -| 检索 | 块携带**自己的多模态向量与指纹**，直接参与向量检索 | 不需要描述文本做中介 | -| 生命周期 | **无独立 GC、无引用计数、无 keep-set**；删除块即删内容 | 媒体是记忆节点，不是需要保活的缓存 | +| 负载归属 | 字节在 `memory_blocks.data`（BLOB），随块同生共灭 | 一个块要么带文本、要么带字节；不存在块外的字节存储层 | +| 块身份 | **内容派生** `blk_media_` | 同内容⇒同 ID⇒`ON CONFLICT` 天然去重（内容寻址当年的好处，现在由块 ID 直接提供） | +| 检索 | 块携带**自己的多模态向量与指纹**，直接参与 `RecallBlocks` 向量召回 | 纯向量检索，全程不碰负载；负载只在块被选中后按需 `BlockData` 取用 | +| 取字节 | `BlockData(blockID)` 单块按需加载；插件经 `Memory().MediaBytes(ref)` 按块引用取回 | `blockColumns` 刻意不含 `data`：召回带上负载会让每次全表读入几十 MB | +| 生命周期 | **无独立 GC、无引用计数、无 keep-set**；删除块即删内容 | 媒体是记忆块，不是需要保活的缓存 | +| 知识库媒体 | 字节作为条目自带数据落在 entry 目录（与 `content.md`、`.media.json` 并列） | `Remove` 的 `RemoveAll` 天然清掉，不留孤儿 | **不再有描述式索引**：旧实现在正文里写 `[ <短digest>] <描述>` 标记，并把描述文本当作语义记忆 （检索靠描述）。该机制已在 v1.2.0 整体拆除：描述是模型生成的二手信息， 检索“别人转述的图片”不如检索图片自己的向量。现在图片只按自己的统一空间向量被检索， 正文里不再有 media marker。 -**跨空间向量迁移**：媒体行的向量带 `vec_model`（空间指纹）。启动时 -`reembedStaleMedia()` 把 `vec_model` 为空（从未嵌入）或与当前空间不一致（换过模型/维度）的行 -批量重算并**写回库**；模态不在本空间覆盖范围时返回 `ErrModalityUnsupported`， +**跨空间向量迁移**：块的向量带 `fingerprint`（空间指纹）。启动时 +`reembedStaleMedia()` 把 `vector` 为空/`'null'` 或 fingerprint 与当前空间不一致（换过模型/维度）的块 +批量重算并**写回库**（字节经 `BlockData` 按需取回）；模态不在本空间覆盖范围时返回 `ErrModalityUnsupported`， **绝不拿别的模型的向量顶替**。 -媒体存储全程可选：`core.memory.media.enabled=false` 或未配置时，整条链路静默退化为纯文本行为， -不报错不 panic。 - ### 其他记忆层 - **Social** (`internal/memory/social/social.go`) — 人格特质和关系网，包装 GraphDB 实体类型 diff --git a/internal/config/registry.go b/internal/config/registry.go index 5fd45e25..997952f5 100644 --- a/internal/config/registry.go +++ b/internal/config/registry.go @@ -721,7 +721,6 @@ func (r *ConfigRegistry) seedDBValues(dataDir string) { set("core.memory.graph", filepath.Join(dataDir, "memory", "graph.db")) set("core.memory.text", filepath.Join(dataDir, "memory", "text")) set("core.memory.documents", filepath.Join(dataDir, "memory", "documents")) - set("core.memory.media.dir", filepath.Join(dataDir, "memory", "media")) // 发行版默认启用本地向量空间。用 chineseclip（text+image、512 维、实测 // 稳态约 0.89GB、Apache-2.0）而不是 qwen3vl（视觉塔按需加载，峰值约 9.4GB）： // 多数机器装不下后者。 @@ -834,8 +833,8 @@ func (r *ConfigRegistry) seedCoreDefs(dataDir string) { reg(ConfigDef{Key: "core.memory.graph", Default: filepath.Join(dataDir, "memory", "graph.db"), Type: "string", DisplayName: "图数据库路径", Description: "长期记忆（图数据库）存储路径", Category: "paths"}) reg(ConfigDef{Key: "core.memory.text", Default: filepath.Join(dataDir, "memory", "text"), Type: "string", DisplayName: "文本记忆路径", Description: "短期文本记忆存储目录", Category: "paths"}) reg(ConfigDef{Key: "core.memory.documents", Default: filepath.Join(dataDir, "memory", "documents"), Type: "string", DisplayName: "文档记忆路径", Description: "文档记忆存储目录", Category: "paths"}) - reg(ConfigDef{Key: "core.memory.media.enabled", Default: "true", Type: "bool", DisplayName: "媒体记忆", Description: "把对话里出现的图片/音频变成一等记忆块，内容按 sha256 落盘去重。关闭后媒体仅在当前对话内可见，下一轮起只剩路径或 alt 文本", Category: "memory"}) - reg(ConfigDef{Key: "core.memory.media.dir", Default: filepath.Join(dataDir, "memory", "media"), Type: "string", DisplayName: "媒体存储路径", Description: "媒体内容寻址存储目录（内含 media.db 与 blobs/）", Category: "paths"}) + // ★ core.memory.media.* 已随 CAS 淘汰删除（2026-10-11）：媒体字节现在是 + // 块自带负载（MemoryBlock.Data），媒体记忆不再有独立的存储开关/路径。 reg(ConfigDef{Key: "core.memory.multimodal_space.provider", Default: "chineseclip", Type: "string", DisplayName: "多模态向量 provider", Description: "从公共 provider 注册表（pkg/embedding）按名字打开的多模态向量空间。内置：chineseclip（默认，text+image，512 维，实测稳态约 0.89GB（加载峰值 1.59GB），Apache-2.0）、qwen3vl（text+image，2048 维，峰值约 9.4GB，视觉塔按需加载；视频已实现但未纳入契约）、http（外部向量 API）。也可是第三方注册的名字。两者均需 onnxruntime 构建标签。留空禁用多模态向量检索，只保留 fastText 文本路径。provider 的模型文件、预处理与运行时全在 provider 内部，核心不做任何模型假设。修改后需重启生效。", Category: "memory"}) reg(ConfigDef{Key: "core.memory.multimodal_space.options.model_dir", Default: filepath.Join(dataDir, "models", "chinese-clip-vit-b16-onnx"), Type: "string", DisplayName: "provider 模型目录", Description: "provider 自定义选项（以 options. 开头的键会去掉前缀后原样传给 provider，核心不解释其含义）。对内置 chineseclip：Chinese-CLIP 产物目录（用 scripts/export_chineseclip_onnx.py 生成）。对内置 qwen3vl：Qwen3-VL ONNX 产物目录（用 scripts/export_qwen3vl_embedding_onnx.py 生成）。", Category: "memory"}) reg(ConfigDef{Key: "core.memory.multimodal_space.options.endpoint", Default: "", Type: "string", DisplayName: "provider 服务端点", Description: "provider 自定义选项。对内置 http：外部多模态向量服务的端点 URL（POST，接受 modality/side/text/data/mime，返回 embedding）。", Category: "memory"})