mirror of
https://gitcode.com/JianFeeeee/HomeAgent.git
synced 2026-09-21 17:38:10 +00:00
feat: NoMemory/Cleaner memory system + doc update
- _sdk_local/ removed (moved to standalone sdk repo) - internal/agent/core: NoMemory/Cleaner data-flow breakpoints - internal/memory: clean_text, document store refactor - internal/plugin/registry.go: plugin API alignment - docs: PLUGIN_DEV.md, ARCHITECTURE.md NoMemory/Cleaner docs - plan.md, review.md: status update
This commit is contained in:
@ -8,7 +8,7 @@ HomeAgent's cognitive architecture consists of three subsystems: the event loop
|
||||
|
||||
**The event loop (eventLoop)** is a three-way select: `a.io.InputChan()` receives external user input and dispatches to `processTextInput` / `processMediaInput`; `a.selfInputCh` receives internal system tasks (memory merges, distillation callbacks) routed through `processConsolidation` under the `_consolidation_` output channel; `a.ctx.Done()` accepts shutdown signals. A concurrently running `interceptLoop` goroutine independently reads `a.io.InputInterruptChan()` — on receiving a high-priority interrupt, it cancels the in-flight LLM HTTP request (`a.cancelLLM()`), then writes the event to `a.interceptCh`. This channel is drained non-blockingly by `drainInterrupts()` before each LLM call in `process()`, injecting interrupts as `[打断消息]` formatted entries into message history. The three interrupt delivery paths carry distinct semantics: `cancelLLM` terminates the current HTTP request, `interceptCh` injects text before the next LLM turn, and `InjectInput` triggers a new processing cycle when the event loop is idle.
|
||||
|
||||
**The stage pipeline (StageHost)** manages two registration categories: tool definitions (ToolDef) and stage handlers (StageHandler). `RegisterTool` rejects duplicate names, infers the owning plugin name from the tool name prefix, and maintains a `toolPlugins` mapping. `RegisterStage` appends handlers to the corresponding stage list. On stage execution (`RunStage`), **all registered handlers execute in parallel via goroutines**, sharing a single `*StageContext` protected by `sync.RWMutex`. Individual handler panics are recovered independently without affecting other handlers. Short-circuit semantics are implemented by checking `ctx.Response != nil` — any stage handler can set this value to terminate the pipeline early. `ExecuteTool` includes built-in panic recovery with stack-trace recording. `UnregisterPluginTools` removes a plugin's tool set during hot-reload.
|
||||
**The stage pipeline (StageHost)** manages two registration categories: tool definitions (ToolDef) and stage handlers (StageHandler). ToolDef includes two optional memory control fields: `NoMemory bool` — when true, the tool's output is excluded from vectorization/jieba/distillation (original text preserved); and `Cleaner func(string) string` — a filter applied before the output enters the computation layer (e.g., extracting a `content` field from JSON). Neither modifies the original output; both only affect the computation layer input. `RegisterTool` rejects duplicate names, infers the owning plugin name from the tool name prefix, and maintains a `toolPlugins` mapping. `RegisterStage` appends handlers to the corresponding stage list. On stage execution (`RunStage`), **all registered handlers execute in parallel via goroutines**, sharing a single `*StageContext` protected by `sync.RWMutex`. Individual handler panics are recovered independently without affecting other handlers. Short-circuit semantics are implemented by checking `ctx.Response != nil` — any stage handler can set this value to terminate the pipeline early. `ExecuteTool` includes built-in panic recovery with stack-trace recording. `UnregisterPluginTools` removes a plugin's tool set during hot-reload.
|
||||
|
||||
**The context window (RelevanceContext)** maintains a chronologically ordered event list. `Append` applies `CleanTemplateText` to strip QQ templates and timestamp noise before computing the embedding vector using a three-branch strategy (agent events use Response, user events use Input, cold_storage uses Input+Response). `Prune` triggers when the event count exceeds `topK`: it **unconditionally protects the last 10 events from eviction** (recency bias), scores remaining candidates against the current input via CosineSimilarity, keeps `topK - 10` highest-scoring entries (floor at 0), then re-sorts chronologically. Pruned events from sources other than `agentcli` and `terminal` are archived to the Document layer via `docStore.ContextToDoc`, retaining original timestamps. Persistence uses 5-second debounced writes to a JSON file.
|
||||
|
||||
|
||||
@ -250,6 +250,10 @@ shared by both Windows DLL and Linux/macOS .so builds. No manual bridge code nee
|
||||
s.RegisterTool("weather_query", sdk.ToolDef{
|
||||
Name: "weather_query",
|
||||
Description: "Query weather for a specified city",
|
||||
NoMemory: false, // false=output participates in memory, true=skip
|
||||
// Cleaner: func(output string) string { // Optional: clean output before vector/jieba/distill
|
||||
// return extractJSON(output, "content")
|
||||
// },
|
||||
Parameters: map[string]interface{}{
|
||||
"type": "object",
|
||||
"properties": map[string]interface{}{
|
||||
@ -270,6 +274,26 @@ s.RegisterTool("weather_query", sdk.ToolDef{
|
||||
})
|
||||
```
|
||||
|
||||
##### NoMemory and Cleaner
|
||||
|
||||
`NoMemory` and `Cleaner` are optional fields on `ToolDef` that control how tool output participates in the **memory computation layer** (vectorization, jieba tokenization, distillation):
|
||||
|
||||
- **`NoMemory`** (default `false`): When `true`, the tool's output is excluded from all memory computation (vector, tokenization, distillation), but the original text is preserved in Context and Document. LLM attention is unaffected. Use cases: `cmd_run` (unpredictable noise in command output), pure operation tools like file upload/delete.
|
||||
|
||||
- **`Cleaner`** (optional): A function `func(output string) string`. When set, the tool output is filtered through this function before participating in vectorization/jieba/distillation. Typical use: stripping SQL prefixes, extracting a `content` field from JSON. The original output is never modified — Cleaner only affects the computation layer input.
|
||||
|
||||
Decision matrix:
|
||||
|
||||
```
|
||||
Tool output → valuable for LLM attention?
|
||||
├── No → NoMemory=true (output preserved, skipped in computation)
|
||||
└── Yes → Contains cleanable noise?
|
||||
├── Yes → Cleaner filters before computation
|
||||
└── No → Normal memory, no extra handling
|
||||
```
|
||||
|
||||
> **Note**: `Cleaner` is a Go `func` type (`json:"-"`), cannot cross C ABI boundaries. Not available for Lua plugins or remote plugins.
|
||||
|
||||
#### Stage Hooks — Intervene in message processing flow
|
||||
|
||||
7 stages:
|
||||
|
||||
@ -8,7 +8,7 @@ HomeAgent 的认知架构由三个核心子系统构成:事件循环(eventLo
|
||||
|
||||
**事件循环(eventLoop)** 是一个三路 select 循环:`a.io.InputChan()` 接收外部用户输入并分发至 `processTextInput` / `processMediaInput`;`a.selfInputCh` 接收内部系统任务(如记忆合并、蒸馏回调),以 `_consolidation_` 输出通道标识区分,走 `processConsolidation` 路径;`a.ctx.Done()` 接受关闭信号。与之并行运行的 `interceptLoop` 协程独立监听 `a.io.InputInterruptChan()`,收到高优先级中断时先取消当前 LLM HTTP 请求(`a.cancelLLM()`),再将事件写入 `a.interceptCh`——该通道在 `process()` 每次 LLM 调用前由 `drainInterrupts()` 非阻塞排空,以 `[打断消息]` 格式注入消息历史。三条中断投递路径各具语义:`cancelLLM` 终结当前 HTTP 请求,`interceptCh` 在下一轮 LLM 调用前注入文本,`InjectInput` 在 eventLoop 空闲时触发新一轮处理。
|
||||
|
||||
**阶段管道(StageHost)** 管理两类注册:工具定义(ToolDef)与阶段处理器(StageHandler)。`RegisterTool` 拒绝同名注册,推断工具所属插件名,并维护工具到插件的映射表 `toolPlugins`。`RegisterStage` 将处理器追加至对应阶段的处理器列表。触发阶段执行时(`RunStage`),**所有已注册处理器通过 goroutine 并行执行**,共享同一 `*StageContext` 实例(通过 `sync.RWMutex` 保护并发访问)。单个处理器的 panic 被独立恢复,不影响其他处理器。短路语义通过检查 `ctx.Response != nil` 实现——任一阶段处理器可设置此值提前终止当前链路。工具执行 `ExecuteTool` 内置 panic 恢复与栈追踪记录。`UnregisterPluginTools` 在插件热重载时移除对应工具集。
|
||||
**阶段管道(StageHost)** 管理两类注册:工具定义(ToolDef)与阶段处理器(StageHandler)。ToolDef 包含 `NoMemory bool` 和 `Cleaner func(string) string` 两个可选的记忆控制字段:`NoMemory=true` 时工具输出不参与向量化/jieba/蒸馏计算(原文保留);`Cleaner` 在输出进入计算层前执行过滤(如提取 JSON 的 `content` 字段)。两者均不修改原文,只影响计算层输入。`RegisterTool` 拒绝同名注册,推断工具所属插件名,并维护工具到插件的映射表 `toolPlugins`。`RegisterStage` 将处理器追加至对应阶段的处理器列表。触发阶段执行时(`RunStage`),**所有已注册处理器通过 goroutine 并行执行**,共享同一 `*StageContext` 实例(通过 `sync.RWMutex` 保护并发访问)。单个处理器的 panic 被独立恢复,不影响其他处理器。短路语义通过检查 `ctx.Response != nil` 实现——任一阶段处理器可设置此值提前终止当前链路。工具执行 `ExecuteTool` 内置 panic 恢复与栈追踪记录。`UnregisterPluginTools` 在插件热重载时移除对应工具集。
|
||||
|
||||
**上下文窗口(RelevanceContext)** 维护一个按时间排序的事件列表。`Append` 在录入前经 `CleanTemplateText` 剥离 QQ 模板与时间戳噪声,再通过三分支向量策略(agent 事件用 Response,用户事件用 Input,cold_storage 用 Input+Response)计算嵌入向量。`Prune` 在事件数超过 `topK` 时触发,**无条件保护最近 10 条事件不被裁剪**(recency bias),对剩余候选事件计算与当前输入的 CosineSimilarity,按评分降序保留 `topK - 10` 条(下限为 0),之后按时间戳重排序。裁剪出的事件中,过滤掉 `agentcli` 和 `terminal` 来源后,其余通过 `docStore.ContextToDoc` 归档至 Document 层,保留原始时间戳。持久化采用 5 秒防抖写入磁盘 JSON 文件。
|
||||
|
||||
|
||||
@ -248,6 +248,10 @@ func NewPlugin(name string, config map[string]interface{}) (sdk.Plugin, error) {
|
||||
s.RegisterTool("weather_query", sdk.ToolDef{
|
||||
Name: "weather_query",
|
||||
Description: "查询指定城市的天气",
|
||||
NoMemory: false, // false=输出参与记忆计算,true=跳过计算
|
||||
// Cleaner: func(output string) string { // 可选:输出参与向量化/jieba/蒸馏前的清洗
|
||||
// return extractJSON(output, "content")
|
||||
// },
|
||||
Parameters: map[string]interface{}{
|
||||
"type": "object",
|
||||
"properties": map[string]interface{}{
|
||||
@ -268,6 +272,26 @@ s.RegisterTool("weather_query", sdk.ToolDef{
|
||||
})
|
||||
```
|
||||
|
||||
##### NoMemory 与 Cleaner 说明
|
||||
|
||||
`NoMemory` 和 `Cleaner` 是 `ToolDef` 上的两个可选字段,控制工具输出在**记忆计算层**(向量化、jieba 分词、蒸馏)中的行为:
|
||||
|
||||
- **`NoMemory`**(默认 `false`):设为 `true` 时,工具输出不参与任何记忆计算(向量、分词、蒸馏),但原文保留在 Context 和 Document 中,LLM 注意力不受影响。适用场景:`cmd_run`(命令输出含不可控噪音)、文件上传/删除等纯操作工具。
|
||||
|
||||
- **`Cleaner`**(可选):函数签名 `func(output string) string`。注册后,工具输出在参与向量化/jieba/蒸馏前先经过此函数过滤。典型用途:SQL 查询去前缀、JSON 包裹提取 `content` 字段。原文始终不变,Cleaner 只影响计算层输入。
|
||||
|
||||
决策矩阵:
|
||||
|
||||
```
|
||||
工具输出 → 对 LLM 注意力有信号价值?
|
||||
├── 否 → NoMemory=true(输出保留原文,跳过计算层)
|
||||
└── 是 → 有可控噪音?
|
||||
├── 是 → Cleaner 过滤后参与计算
|
||||
└── 否 → 正常记忆,无需额外处理
|
||||
```
|
||||
|
||||
> **注意**:`Cleaner` 是 Go `func` 类型(`json:"-"`),不能跨 C ABI 边界序列化。Lua 插件和远程插件无法使用。
|
||||
|
||||
#### 阶段钩子 — 干预消息处理流
|
||||
|
||||
7 个阶段:
|
||||
|
||||
Reference in New Issue
Block a user