feat: NoMemory/Cleaner memory system + doc update

- _sdk_local/ removed (moved to standalone sdk repo)
- internal/agent/core: NoMemory/Cleaner data-flow breakpoints
- internal/memory: clean_text, document store refactor
- internal/plugin/registry.go: plugin API alignment
- docs: PLUGIN_DEV.md, ARCHITECTURE.md NoMemory/Cleaner docs
- plan.md, review.md: status update
This commit is contained in:
JianFeeeee
2026-07-25 11:17:31 +08:00
parent 4a2df503ed
commit 31664a7853
36 changed files with 639 additions and 2488 deletions

View File

@ -8,7 +8,7 @@ HomeAgent's cognitive architecture consists of three subsystems: the event loop
**The event loop (eventLoop)** is a three-way select: `a.io.InputChan()` receives external user input and dispatches to `processTextInput` / `processMediaInput`; `a.selfInputCh` receives internal system tasks (memory merges, distillation callbacks) routed through `processConsolidation` under the `_consolidation_` output channel; `a.ctx.Done()` accepts shutdown signals. A concurrently running `interceptLoop` goroutine independently reads `a.io.InputInterruptChan()` — on receiving a high-priority interrupt, it cancels the in-flight LLM HTTP request (`a.cancelLLM()`), then writes the event to `a.interceptCh`. This channel is drained non-blockingly by `drainInterrupts()` before each LLM call in `process()`, injecting interrupts as `[打断消息]` formatted entries into message history. The three interrupt delivery paths carry distinct semantics: `cancelLLM` terminates the current HTTP request, `interceptCh` injects text before the next LLM turn, and `InjectInput` triggers a new processing cycle when the event loop is idle.
**The stage pipeline (StageHost)** manages two registration categories: tool definitions (ToolDef) and stage handlers (StageHandler). `RegisterTool` rejects duplicate names, infers the owning plugin name from the tool name prefix, and maintains a `toolPlugins` mapping. `RegisterStage` appends handlers to the corresponding stage list. On stage execution (`RunStage`), **all registered handlers execute in parallel via goroutines**, sharing a single `*StageContext` protected by `sync.RWMutex`. Individual handler panics are recovered independently without affecting other handlers. Short-circuit semantics are implemented by checking `ctx.Response != nil` — any stage handler can set this value to terminate the pipeline early. `ExecuteTool` includes built-in panic recovery with stack-trace recording. `UnregisterPluginTools` removes a plugin's tool set during hot-reload.
**The stage pipeline (StageHost)** manages two registration categories: tool definitions (ToolDef) and stage handlers (StageHandler). ToolDef includes two optional memory control fields: `NoMemory bool` — when true, the tool's output is excluded from vectorization/jieba/distillation (original text preserved); and `Cleaner func(string) string` — a filter applied before the output enters the computation layer (e.g., extracting a `content` field from JSON). Neither modifies the original output; both only affect the computation layer input. `RegisterTool` rejects duplicate names, infers the owning plugin name from the tool name prefix, and maintains a `toolPlugins` mapping. `RegisterStage` appends handlers to the corresponding stage list. On stage execution (`RunStage`), **all registered handlers execute in parallel via goroutines**, sharing a single `*StageContext` protected by `sync.RWMutex`. Individual handler panics are recovered independently without affecting other handlers. Short-circuit semantics are implemented by checking `ctx.Response != nil` — any stage handler can set this value to terminate the pipeline early. `ExecuteTool` includes built-in panic recovery with stack-trace recording. `UnregisterPluginTools` removes a plugin's tool set during hot-reload.
**The context window (RelevanceContext)** maintains a chronologically ordered event list. `Append` applies `CleanTemplateText` to strip QQ templates and timestamp noise before computing the embedding vector using a three-branch strategy (agent events use Response, user events use Input, cold_storage uses Input+Response). `Prune` triggers when the event count exceeds `topK`: it **unconditionally protects the last 10 events from eviction** (recency bias), scores remaining candidates against the current input via CosineSimilarity, keeps `topK - 10` highest-scoring entries (floor at 0), then re-sorts chronologically. Pruned events from sources other than `agentcli` and `terminal` are archived to the Document layer via `docStore.ContextToDoc`, retaining original timestamps. Persistence uses 5-second debounced writes to a JSON file.

View File

@ -250,6 +250,10 @@ shared by both Windows DLL and Linux/macOS .so builds. No manual bridge code nee
s.RegisterTool("weather_query", sdk.ToolDef{
Name: "weather_query",
Description: "Query weather for a specified city",
NoMemory: false, // false=output participates in memory, true=skip
// Cleaner: func(output string) string { // Optional: clean output before vector/jieba/distill
// return extractJSON(output, "content")
// },
Parameters: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
@ -270,6 +274,26 @@ s.RegisterTool("weather_query", sdk.ToolDef{
})
```
##### NoMemory and Cleaner
`NoMemory` and `Cleaner` are optional fields on `ToolDef` that control how tool output participates in the **memory computation layer** (vectorization, jieba tokenization, distillation):
- **`NoMemory`** (default `false`): When `true`, the tool's output is excluded from all memory computation (vector, tokenization, distillation), but the original text is preserved in Context and Document. LLM attention is unaffected. Use cases: `cmd_run` (unpredictable noise in command output), pure operation tools like file upload/delete.
- **`Cleaner`** (optional): A function `func(output string) string`. When set, the tool output is filtered through this function before participating in vectorization/jieba/distillation. Typical use: stripping SQL prefixes, extracting a `content` field from JSON. The original output is never modified — Cleaner only affects the computation layer input.
Decision matrix:
```
Tool output → valuable for LLM attention?
├── No → NoMemory=true (output preserved, skipped in computation)
└── Yes → Contains cleanable noise?
├── Yes → Cleaner filters before computation
└── No → Normal memory, no extra handling
```
> **Note**: `Cleaner` is a Go `func` type (`json:"-"`), cannot cross C ABI boundaries. Not available for Lua plugins or remote plugins.
#### Stage Hooks — Intervene in message processing flow
7 stages:

View File

@ -8,7 +8,7 @@ HomeAgent 的认知架构由三个核心子系统构成事件循环eventLo
**事件循环eventLoop** 是一个三路 select 循环:`a.io.InputChan()` 接收外部用户输入并分发至 `processTextInput` / `processMediaInput``a.selfInputCh` 接收内部系统任务(如记忆合并、蒸馏回调),以 `_consolidation_` 输出通道标识区分,走 `processConsolidation` 路径;`a.ctx.Done()` 接受关闭信号。与之并行运行的 `interceptLoop` 协程独立监听 `a.io.InputInterruptChan()`,收到高优先级中断时先取消当前 LLM HTTP 请求(`a.cancelLLM()`),再将事件写入 `a.interceptCh`——该通道在 `process()` 每次 LLM 调用前由 `drainInterrupts()` 非阻塞排空,以 `[打断消息]` 格式注入消息历史。三条中断投递路径各具语义:`cancelLLM` 终结当前 HTTP 请求,`interceptCh` 在下一轮 LLM 调用前注入文本,`InjectInput` 在 eventLoop 空闲时触发新一轮处理。
**阶段管道StageHost** 管理两类注册工具定义ToolDef与阶段处理器StageHandler`RegisterTool` 拒绝同名注册,推断工具所属插件名,并维护工具到插件的映射表 `toolPlugins``RegisterStage` 将处理器追加至对应阶段的处理器列表。触发阶段执行时(`RunStage`**所有已注册处理器通过 goroutine 并行执行**,共享同一 `*StageContext` 实例(通过 `sync.RWMutex` 保护并发访问)。单个处理器的 panic 被独立恢复,不影响其他处理器。短路语义通过检查 `ctx.Response != nil` 实现——任一阶段处理器可设置此值提前终止当前链路。工具执行 `ExecuteTool` 内置 panic 恢复与栈追踪记录。`UnregisterPluginTools` 在插件热重载时移除对应工具集。
**阶段管道StageHost** 管理两类注册工具定义ToolDef与阶段处理器StageHandlerToolDef 包含 `NoMemory bool``Cleaner func(string) string` 两个可选的记忆控制字段:`NoMemory=true` 时工具输出不参与向量化/jieba/蒸馏计算(原文保留);`Cleaner` 在输出进入计算层前执行过滤(如提取 JSON 的 `content` 字段)。两者均不修改原文,只影响计算层输入。`RegisterTool` 拒绝同名注册,推断工具所属插件名,并维护工具到插件的映射表 `toolPlugins``RegisterStage` 将处理器追加至对应阶段的处理器列表。触发阶段执行时(`RunStage`**所有已注册处理器通过 goroutine 并行执行**,共享同一 `*StageContext` 实例(通过 `sync.RWMutex` 保护并发访问)。单个处理器的 panic 被独立恢复,不影响其他处理器。短路语义通过检查 `ctx.Response != nil` 实现——任一阶段处理器可设置此值提前终止当前链路。工具执行 `ExecuteTool` 内置 panic 恢复与栈追踪记录。`UnregisterPluginTools` 在插件热重载时移除对应工具集。
**上下文窗口RelevanceContext** 维护一个按时间排序的事件列表。`Append` 在录入前经 `CleanTemplateText` 剥离 QQ 模板与时间戳噪声再通过三分支向量策略agent 事件用 Response用户事件用 Inputcold_storage 用 Input+Response计算嵌入向量。`Prune` 在事件数超过 `topK` 时触发,**无条件保护最近 10 条事件不被裁剪**recency bias对剩余候选事件计算与当前输入的 CosineSimilarity按评分降序保留 `topK - 10` 条(下限为 0之后按时间戳重排序。裁剪出的事件中过滤掉 `agentcli``terminal` 来源后,其余通过 `docStore.ContextToDoc` 归档至 Document 层,保留原始时间戳。持久化采用 5 秒防抖写入磁盘 JSON 文件。

View File

@ -248,6 +248,10 @@ func NewPlugin(name string, config map[string]interface{}) (sdk.Plugin, error) {
s.RegisterTool("weather_query", sdk.ToolDef{
Name: "weather_query",
Description: "查询指定城市的天气",
NoMemory: false, // false=输出参与记忆计算true=跳过计算
// Cleaner: func(output string) string { // 可选:输出参与向量化/jieba/蒸馏前的清洗
// return extractJSON(output, "content")
// },
Parameters: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
@ -268,6 +272,26 @@ s.RegisterTool("weather_query", sdk.ToolDef{
})
```
##### NoMemory 与 Cleaner 说明
`NoMemory``Cleaner``ToolDef` 上的两个可选字段,控制工具输出在**记忆计算层**向量化、jieba 分词、蒸馏)中的行为:
- **`NoMemory`**(默认 `false`):设为 `true` 时,工具输出不参与任何记忆计算(向量、分词、蒸馏),但原文保留在 Context 和 Document 中LLM 注意力不受影响。适用场景:`cmd_run`(命令输出含不可控噪音)、文件上传/删除等纯操作工具。
- **`Cleaner`**(可选):函数签名 `func(output string) string`。注册后,工具输出在参与向量化/jieba/蒸馏前先经过此函数过滤。典型用途SQL 查询去前缀、JSON 包裹提取 `content` 字段。原文始终不变Cleaner 只影响计算层输入。
决策矩阵:
```
工具输出 → 对 LLM 注意力有信号价值?
├── 否 → NoMemory=true输出保留原文跳过计算层
└── 是 → 有可控噪音?
├── 是 → Cleaner 过滤后参与计算
└── 否 → 正常记忆,无需额外处理
```
> **注意**`Cleaner` 是 Go `func` 类型(`json:"-"`),不能跨 C ABI 边界序列化。Lua 插件和远程插件无法使用。
#### 阶段钩子 — 干预消息处理流
7 个阶段: