merge: 架构节补上下文管理与记忆

2026-11-01 14:57:38 +00:00 · 2026-10-31 11:30:22 +08:00
parent 7b4c6ead7f 152dc680ad
commit 6a2c907e04
2 changed files with 194 additions and 36 deletions
--- a/README.md
+++ b/README.md
@ -4,37 +4,109 @@

 > **English**: [README_EN.md](./README_EN.md)

-自托管 Agent 内核。[文档](assets/docs/zh/ARCHITECTURE.md) · [English](README_EN.md)
-
-**启动顺序是编译期算出来的，不是运行期猜的。** 20 个内核模块各自声明 `Requires`/`Provides`，
-构建时算出 6 层依赖图写进 `depgraph.bin`，并把图的 digest 用 `-ldflags` 注入二进制。
-运行期只读图，不重算。图与二进制不配套就拒绝启动，不猜哪边对。
+自托管 Agent 内核。跑在自己的机器上，插件以独立进程接入。

 ```bash
 make build build-cli
 ./build/homed -data /tmp/ha
 ```

-**插件按设施就绪时刻启动，不等内核全装完。** 插件声明 `requires_facilities: [config]`，
-config 层（第 1 层）完成时它就起来了，不必等 agent 层。实测省掉 35 秒：agent 层要做
-context dense backfill 750 条事件、重建文档与知识库索引。
+API 密钥在 WebUI（`http://localhost:8080`）里填。需要 Linux。

-**输入分两类，中断分四级。** 排队输入没有级别，任何中断都能打断它。中断带 L1–L4，
-高抢占低，被抢占的现场压栈待恢复。同级不互相抢占。L4 归内核独占（panic、内核事件），
-插件声明 L4 会被夹到 L3。抢占计数能提级，但最多两档（`preemptPromotionCap = 2`），
-防低优先级任务被无限饿死。
+跟常见 agent 框架的主要差别在上下文管理：不是超窗就截断或让模型摘要，
+而是按相关度取舍，低分的下沉成文档、再蒸馏进图数据库。另外两处不太一样的是
+启动顺序在编译期定序，以及插件按设施就绪提前启动（详见下方架构）。

-**记忆按场面分，不按会话。** 场面指纹由通道、对话对象、工具、话题、时段构成，
-权重依次 1.0 / 1.0 / 0.8 / 0.4 / 0.2。重复出现到第 2 次才长成场面
-（`minSceneEvidence = 2`），只发生一次的不算。归属阈值 0.5，唤起阈值 0.35，
-唤起比归属宽松——想不起来是损失，多想一条只是多几行上下文。
+## 架构
+
+```
+                    homed（内核）
+        LLM 编排 · 三层记忆 · 检索 · 调度
+                       │
+        ┌──────────────┼──────────────┐
+        │              │              │
+   stdio RPC       共享内存        事件环
+    控制          大块数据         通知
+        │              │              │
+        └──────────────┼──────────────┘
+                       │
+              插件（各自独立进程）
+   WebUI · QQ · 命令行 · 文件 · 浏览器
+```
+
+内核不发 HTTP、不读文件。所有接触外部世界的能力都在插件里，插件崩溃不影响内核，
+重启退避 1s / 2s / 3s，5 分钟内第 4 次崩溃停止自动重启等人工介入。
+
+### 上下文不是「截断 + 摘要」
+
+常见做法是把超窗的对话裁掉或让模型自己摘要，两者的共同问题是**丢什么由位置决定**。
+这里换成了按相关度取舍：用当前输入给每条历史事件打分，低分的下沉到文档层。
+
+打分用稠密向量余弦（与媒体、文档共享同一向量空间），模型未就绪时退化到 TF-IDF。
+裁剪时无条件保住最近 N 条，但 N 本身受预算约束：
+
+```go
+// 保护条数不超过预算能装下的条数
+budgetCap = ContextTokens / avgEventTokens
+pCount = min(配置值, budgetCap)
+```
+
+这条约束来自一次实测：50k 窗口下单条事件约 5800 token（工具回灌型），预算约 40000，
+「钉住 10 条」本身就装不下，于是每轮裁剪都撞保护下限，两轮后预算耗尽，任务直接终止。
+召回率从 44% 掉到 22%。现在窗口越小越自动收紧，1M 窗口下则能用满配置值。
+
+被裁掉的事件不是丢弃，是迁移：事件连同它持有的媒体块一起进文档层，块的 ID 不变、
+只换持有层。回灌时用 `coldcontext_query` 取回，两阶段（先 Peek 确认、再 Remove），
+不用「先删后灌」——那会在回灌失败时永久丢数据。
+
+### 三层记忆是迁移，不是复制
+
+```
+L0 上下文   相关性裁剪，最近 N 条恒受保护
+     ↓      低分事件下沉（带媒体块）
+L1 文档     稠密检索，访问计数 ≤2 且久未访问则继续下沉
+     ↓      蒸馏成三元组
+L2 图       实体关系 + 场面索引，支持按场面召回
+```
+
+块在任一层只有一份持有者，不靠引用计数保活。媒体不是附件：图片、音频是图里的一等节点，
+有自己的向量，和文本在同一空间里检索。
+
+### 记忆按场面分，不按会话
+
+普通 agent 通常一条会话一条历史，换个话题就失忆，或者全塞进一个上下文里互相干扰。
+这里按「可观察的场所指纹」分：通道、对话对象、工具、话题、时段，权重 1.0/1.0/0.8/0.4/0.2。
+
+同一指纹出现到第 2 次才建场景（`minSceneEvidence = 2`），只发生一次的不算。
+归属阈值 0.5、唤起阈值 0.35，唤起比归属宽松——想不起来是损失，多想一条只是多几行上下文。
+场景每次重现强度 +1，久不重现按半衰期衰减。
+
+两个来源并行：**声明**（插件在注入点说明"这是哪个场面"）与**涌现**（指纹自己聚类）。
+声明场景不吸收整轮指纹，否则它会以接近 1.0 的相似度吃掉后续所有同类轮次，
+被动路再也长不出更细的场面。
+
+### 七个阶段
+
+`internal/agent/core/stages.go`
+
+| 阶段 | 时机 | 插件可做 |
+|---|---|---|
+| `on_input` | 消息到达 | 黑名单、限流、短路回复 |
+| `pre_action` | 上下文就绪，调 LLM 前 | 注入外部数据 |
+| `post_action` | LLM 返回文本与工具调用 | 敏感词过滤、强制改向 |
+| `before_toolcall` | 单个工具执行前 | 审计、拒绝、改参 |
+| `after_toolcall` | 单个工具执行完 | 脱敏、排序 |
+| `before_output` | 最终文本就绪 | 格式适配 |
+| `after_output` | 已发送 | 统计 |
+
+`post_action → [before_toolcall → 执行 → after_toolcall] → post_action` 是内循环，
+退出条件：模型不再调工具、全部被拒、或达到上限。

 ## 其余

 - **通道**：WebUI、命令行、QQ 群与私聊、A2A、IDE 里的 ACP
 - **工具**：文件、命令、代码、浏览器（真实 Chrome）、日程、备忘、RSS、音乐
 - **模态**：图片、视频、语音、截图、网页
- **内核零 IO**：不发 HTTP、不读文件，全部能力在插件里

 28 个内置插件、89 个模型可调工具。外部插件一插件一仓，见 [github.com/HologramAgent](https://github.com/HologramAgent)。

@ -70,20 +142,13 @@ Windows 上构建 `waiter.exe`，`homed` 跑在 WSL2 里；macOS 要原生构建

 ## 许可

-本项目以 **GNU Affero 通用公共许可证第 3 版（AGPL-3.0-only）** 发布，全文见 [LICENSE](LICENSE)。
+内核以 **AGPL-3.0-only** 发布（[LICENSE](LICENSE)）。改过的版本如果对外提供网络服务，
+必须向使用者提供源码（§13）。

-它是 GPL 家族里**传染性最强**的一档：不仅分发时须提供完整对应源码，
-**通过网络提供服务时也要向使用者提供源码**（§13 Remote Network Interaction）。
-即：任何人把改过的 HologramAgent 对外提供网络服务，都必须让该服务的使用者拿到改动后的源码。
+自己写的插件不受此约束。插件静态链接的 [hologramagentsdk](https://github.com/JianFeeeee/hologramagentsdk)
+是 **MIT**，宽松许可不继承 AGPL，所以外部插件可以闭源、可以商用，也不必开源自改。

-插件与本项目通过公开 [hologramagentsdk](https://github.com/JianFeeeee/hologramagentsdk) 静态链接
-（SDK 源码会进入插件二进制），但那个仓以 **MIT** 发布。MIT 是宽松许可，拿到授权的代码不继承
-本项目的 AGPL。因此**外部插件不是本项目的衍生作品**，作者可自行选择许可（含闭源、商业、私有），
-既不必同许可、也不受 §13 网络条款约束。第三方插件生态的安全与活跃正建立在这条之上。
-
-边界很清楚：**AGPL 覆盖内核与随包内置插件**（`homed`、`internal/`、`internal/plugins/` 下的内置插件）；
-**MIT 覆盖公开 SDK**（`sdk/`，`go.mod` 零外部依赖、只依赖 Go 标准库，不引用内核任何代码）。
-子进程隔离在这里不重要了。决定许可的是被链接的 SDK 代码本身，而它是 MIT。
+随包第三方组件保持各自许可，全文在 `/usr/share/doc/hologramagent/licenses/`。

 ### 随包分发的第三方组件

--- a/README_EN.md
+++ b/README_EN.md
@ -4,27 +4,120 @@

 > **中文**: [README.md](./README.md)

-Self-hosted agent kernel. [Docs](assets/docs/en/ARCHITECTURE.md) · [中文](README.md)
-
-**Startup order is computed at build time, not guessed at runtime.** Twenty kernel modules each declare `Requires`/`Provides`. The build resolves a 6-layer dependency graph into `depgraph.bin` and injects the graph digest into the binary via `-ldflags`. At runtime the kernel reads the graph and does not recompute it. If the graph and the binary don't match, startup is refused rather than guessing which one is right.
+Self-hosted agent kernel. Runs on your own hardware; plugins attach as separate processes.

 ```bash
 make build build-cli
 ./build/homed -data /tmp/ha
 ```

-**Plugins start when their facilities are ready, not when the kernel finishes.** A plugin declaring `requires_facilities: [config]` launches as layer 1 completes instead of waiting for the agent layer. Measured saving: 35 seconds, which is how long the agent layer spends on dense backfill of 750 events plus rebuilding document and knowledge indexes.
+API keys go in the web UI at `http://localhost:8080`. Requires Linux.

-**Two input classes, four interrupt levels.** Queued input carries no level, so any interrupt can preempt it. Interrupts carry L1–L4; higher preempts lower and the preempted state is pushed onto a stack for later resume. Same level never preempts. L4 belongs to the kernel alone (panics, kernel events); a plugin declaring L4 gets clamped to L3. Preemption count can raise a task's level, but by at most two steps (`preemptPromotionCap = 2`), so low-priority work can't starve indefinitely.
+The main difference from typical agent frameworks is context handling: instead of truncating
+on overflow or asking the model to summarize, it scores history by relevance and sinks the
+low-scoring events into documents, then distills those into a graph. Two other things differ:
+startup order is resolved at build time, and plugins launch as soon as their facilities are ready.

-**Memory is partitioned by scene, not by session.** A scene fingerprint is built from channel, conversation partner, tools, topic, and time of day, weighted 1.0 / 1.0 / 0.8 / 0.4 / 0.2. A scene only comes into being on the second occurrence (`minSceneEvidence = 2`); one-off interactions don't count. Join threshold is 0.5, recall threshold 0.35. Recall is looser than join on purpose: failing to recall is a loss, while recalling one extra scene costs a few lines of context.
+## Architecture
+
+```
+                    homed (kernel)
+        LLM orchestration · memory · retrieval
+                       │
+        ┌──────────────┼──────────────┐
+        │              │              │
+   stdio RPC       shared mem     event ring
+     control       bulk data      notify
+        │              │              │
+        └──────────────┼──────────────┘
+                       │
+              plugins (separate processes)
+   web UI · QQ · CLI · files · browser
+```
+
+The kernel makes no HTTP calls and reads no files. Everything touching the outside world is a
+plugin. A plugin crash leaves the kernel running; restarts back off 1s / 2s / 3s, and the 4th
+crash within 5 minutes stops automatic restarting for human intervention.
+
+### Context is not "truncate and summarize"
+
+The usual approach drops old turns or asks the model to summarize them. Both share a flaw: what
+gets lost is decided by position. Here each historical event is scored against the current input,
+and low scorers sink into the document layer.
+
+Scoring uses dense cosine similarity in the same vector space as media and documents, falling
+back to TF-IDF when the model isn't loaded. The most recent N events are always kept, but N is
+itself bounded by budget:
+
+```go
+// protected count cannot exceed what the budget can hold
+budgetCap = ContextTokens / avgEventTokens
+pCount = min(configured, budgetCap)
+```
+
+That bound came from a measurement: with a 50k window, one event averaged 5800 tokens (tool
+results), while the budget was 40000, so "pin 10 events" didn't fit. Every prune hit the
+protection floor, the budget was exhausted after two rounds, and the task terminated. Recall
+dropped from 44% to 22%. Smaller windows now tighten automatically, and a 1M window can use the
+full configured value.
+
+Pruned events aren't discarded, they migrate: the event and the media blocks it holds move into
+the document layer with block IDs unchanged, only the holding layer changes. Retrieval uses
+`coldcontext_query` in two phases (Peek to confirm, then Remove) rather than delete-then-refill,
+which would lose data permanently if the refill failed.
+
+### Three layers migrate, they don't copy
+
+```
+L0 context   relevance pruning, last N always protected
+     ↓       low scorers sink (with media blocks)
+L1 documents dense retrieval; access count ≤2 and stale means sink further
+     ↓       distilled into triples
+L2 graph     entities, relations, scene index; recall by scene
+```
+
+A block has exactly one holder at any layer, with no reference counting to keep it alive. Media
+are not attachments: images and audio are first-class graph nodes with their own vectors,
+searchable in the same space as text.
+
+### Memory is partitioned by scene, not by session
+
+Typical agents keep one history per session, so switching topics loses context or crams
+everything into one window. Here memory is partitioned by observable scene fingerprint: channel,
+conversation partner, tools, topic, time of day, weighted 1.0/1.0/0.8/0.4/0.2.
+
+A scene comes into being only on the second matching fingerprint (`minSceneEvidence = 2`).
+Join threshold 0.5, recall 0.35 — recall is looser because failing to recall loses information
+while an extra scene costs a few lines of context. Each recurrence adds to scene strength;
+scenes that stop recurring decay by half-life.
+
+Two paths run in parallel: **declared** (a plugin states at the injection point which scene this
+is) and **emergent** (fingerprints cluster on their own). Declared scenes deliberately don't
+absorb the full fingerprint, otherwise they'd score near 1.0 against every later turn and the
+emergent path could never grow finer-grained scenes.
+
+### Seven stages
+
+`internal/agent/core/stages.go`
+
+| Stage | When | Plugins can |
+|---|---|---|
+| `on_input` | message arrives | blocklist, rate limit, short-circuit reply |
+| `pre_action` | context ready, before LLM | inject external data |
+| `post_action` | LLM returns text and tool calls | filter, force redirect |
+| `before_toolcall` | before one tool runs | audit, deny, rewrite args |
+| `after_toolcall` | after one tool runs | redact, sort results |
+| `before_output` | final text ready | format adaptation |
+| `after_output` | after sending | statistics |
+
+`post_action → [before_toolcall → run → after_toolcall] → post_action` is the inner loop. It exits
+when the model stops calling tools, all calls are denied, or the limit is reached.

 ## Everything else

 - **Channels**: web UI, CLI, QQ groups and DMs, A2A, ACP in your IDE
 - **Tools**: files, shell, code, browser (real Chrome), calendar, notes, RSS, music
 - **Modalities**: images, video, voice, screenshots, web pages
- **Zero-IO kernel**: no HTTP calls, no file reads; every capability lives in a plugin

 28 built-in plugins, 89 model-callable tools. External plugins live one per repo under [github.com/HologramAgent](https://github.com/HologramAgent).