mirror of
https://gitcode.com/JianFeeeee/HomeAgent.git
synced 2026-09-21 17:38:10 +00:00
docs: 同步仓库文档到 v1.1.1,补媒体记忆与接口扩展规则
五处文档此前停在 v1.0.0,而 v1.1.0/v1.1.1 都已发布并在现网运行。 本轮补齐三层记忆的媒体架构、模型工具的媒体参数、通信面 method 数, 以及冻结解除后的替代约束。 ## 中英双语 OVERVIEW.md 三层记忆段只写了 Context/Document/Graph 三层,而 v1.1.0 起图片/音频是 三层里的一类节点。补上媒体记忆的架构概要:CAS + 引用计数 GC + 标记格式 (描述文本才是持久语义记忆,blob 是可淘汰的缓存)+ 插件边界贯通。 ## 中英双语 ARCHITECTURE.md - 「记忆工具」表补 `memory_commit`/`doc_commit` 的 `media_digests` 与 `sentence_text`(后者从未暴露给模型,而它是媒体绑定链的必经环节) - 「三层记忆」段新增媒体记忆子节:CAS 设计表(寻址/完整性/写入原子性/引用/GC)、 标记格式、可选性(`enabled=false` 时整条链路静默退化) - 「三个通信面」从 51 改为 55 个 method,新增四个 media method 的说明 - 「SDK 四通道」代码示例补媒体注入三方法 + 与 SetToolBlocks 的区别 ## plugin-interface-matrix.md - 状态从「完成 v2」改「完成 v3」,v3 记录 v1.1.1 的接口扩展 - §二 A3 DocMemoryAPI 补 InsertWithMedia - §二 A4 补三个媒体注入方法 + 为何不能搭 SetToolBlocks 的车 - §二 A5 补 MediaAttachment 类型 + Triple/Doc/TextEvent 的扩展字段 - §六「新获得的能力」补 SetToolBlocks 已落地、媒体入记忆、插件主动发起 带媒体的对话 - §七 冻结检查点补第 5 条(冻结已解除,取代它的是 §九) - 新增 §九「v1.1.x 的接口扩展规则」:冻结解除后的三条硬约束 (只增不减签名不改 / 新增方法方向 / 模板接线六处失败链)+ 验证方式 (存量插件 17/17、旧产物 4/4 建链、模板断言、压测 -race) ## 为何分立两笔 commit 前一笔(SDK 仓 README + 内核 README)改的是给**插件开发者**看的文本, 本笔改的是给**架构师与维护者**看的技术文档。受众与改动层次不同, 放在同一个 commit 会让追溯时看不出"文档在哪一层跟上了代码"。
This commit is contained in:
@ -189,6 +189,39 @@ All vectorization unified under `StaticEmbedder` (`internal/memory/static_embedd
|
||||
| `doc_query` | Search from Document |
|
||||
| `doc_commit` | Write to Document |
|
||||
|
||||
Since v1.1.1 `memory_commit` and `doc_commit` accept `media_digests`, and the kernel appends the
|
||||
`[<mime> <short digest>] <description>` marker into the sentence/body — **the kernel builds the
|
||||
marker, the model only supplies the digest**. Requiring the caller to know the format would mean a
|
||||
single typo silently breaks reference binding with no error anywhere in the chain. `memory_commit`
|
||||
also gained `sentence_text`: media references hang off a sentence, so with no sentence there is
|
||||
nowhere to attach them.
|
||||
|
||||
### Media Memory (since v1.1.0)
|
||||
|
||||
`internal/memory/media/` — `Store`, content-addressed (CAS)
|
||||
|
||||
| Concern | Approach | Why |
|
||||
|---|---|---|
|
||||
| Addressing | sha256 digest; metadata in SQLite, blobs on disk | Identical bytes stored once; metadata must be queryable, blobs must not live in the database |
|
||||
| Integrity | Every `Get` re-verifies the digest | Silently returning corrupt data on disk damage is far worse than an error |
|
||||
| Write atomicity | `.tmp` + rename | A half-written file taken as complete content would permanently poison that digest |
|
||||
| References | `owner_kind/owner_id/digest` composite primary key, `AddRef` idempotent | Three owner kinds: `context` (context events), `document`, `graph_sentence` |
|
||||
| GC | Two-stage with `minAge`, **referenced items are never deleted** | Description text stays in the text layers while blobs may be evicted — semantic memory and byte cache are decoupled |
|
||||
|
||||
**How media is represented in plain-text memory** is the marker `[<mime> <short digest>] <description>`:
|
||||
|
||||
```
|
||||
[image/png a1b2c3d4e5f6] a purple-blue-red three-band chart
|
||||
```
|
||||
|
||||
Why it must ride on text: `Doc.Content`, `sentences.text` and text memory's `Input` are all strings
|
||||
— there is no field to carry structured data. **The description text is the durable semantic
|
||||
memory** (retrieval uses it); the digest is the key back to the bytes (reverse lookup uses it).
|
||||
After capacity GC evicts a blob, the description remains in the L0/L2/L3 text.
|
||||
|
||||
The media store is **optional throughout**: with `core.memory.media.enabled=false` or no
|
||||
configuration, the whole chain silently degrades to plain-text behaviour — no errors, no panics.
|
||||
|
||||
### Other Memory Layers
|
||||
|
||||
- **Social** (`internal/memory/social/social.go`) — Persona traits and relationship network, wraps GraphDB entity types
|
||||
@ -296,7 +329,7 @@ Common ground:
|
||||
|
||||
| Plane | Mechanism | Why this choice |
|
||||
|---|---|---|
|
||||
| Control | stdio JSON-RPC (NDJSON frames), 51 `core.*` methods | The process boundary *is* the ABI boundary—no need to maintain three platform-specific dynamic-library loaders |
|
||||
| Control | stdio JSON-RPC (NDJSON frames), 55 `core.*` methods | The process boundary *is* the ABI boundary—no need to maintain three platform-specific dynamic-library loaders |
|
||||
| Data | Shared memory segment, **one segment shared by all subprocesses** | One segment per plugin would degrade "kernel ctx → segment → plugin mutates → read back" into the copy model under concurrency, reproducing lost updates exactly |
|
||||
| Notification | Event ring + platform notify (Linux eventfd / macOS pipe / Windows Event) | The kernel must never block on a consumer: streaming output publishes per token, so any wait shows up as stutter |
|
||||
|
||||
@ -334,8 +367,23 @@ sdk.Memory().Recall/Commit
|
||||
sdk.Knowledge().Search/Create
|
||||
sdk.Settings().Get/Set/List
|
||||
sdk.RegisterOutputChannel("qq", sdk.CapText|sdk.CapAudio|sdk.CapImage, "QQ channel, see output_send__qq_help for details", handler)
|
||||
|
||||
// v1.1.1 media APIs (all additive, no signature changes)
|
||||
sdk.DocMemory().InsertWithMedia(doc, attachments) // attachments with Data land in CAS; Digest-only ones reference existing content
|
||||
sdk.InjectInputMedia(source, channel, text, blocks) // media reaches the model in *this* turn
|
||||
sdk.InjectInputMediaSync(...) // same, and waits for the reply
|
||||
sdk.InjectInterruptMedia(...) // media-bearing interrupt, can preempt current processing
|
||||
```
|
||||
|
||||
How media injection differs from `SetToolBlocks`: the latter is only callable inside a tool handler
|
||||
and its media reaches the model with the **next** tool message; these three let a plugin
|
||||
**initiate a turn that carries media** — it goes out with this turn's message and is automatically
|
||||
stored in CAS with a memory reference attached. `Triple` and `Doc` gained `MediaDigests` /
|
||||
`Attachments` correspondingly.
|
||||
|
||||
`internal/sdk/` is the bridge implementation for this layer and is not subject to the public
|
||||
interface freeze (see `docs/git-branching.md` §6).
|
||||
|
||||
### Plugin Interface
|
||||
|
||||
```go
|
||||
|
||||
Reference in New Issue
Block a user