From f9ea80abbfef135ca54476ebd79bde39dd9dce27 Mon Sep 17 00:00:00 2001 From: jianf <2198972886@qq.com> Date: Fri, 17 Jul 2026 21:51:03 +0800 Subject: [PATCH] docs: update sequence diagram, memory flow, vectorization strategy for StaticEmbedder + CleanTemplateText + three-branch vector --- README.md | 30 +++++++++--------- README_EN.md | 30 +++++++++--------- docs/en/ARCHITECTURE.md | 67 ++++++++++++++++++++++------------------- docs/en/OVERVIEW.md | 2 +- docs/zh/ARCHITECTURE.md | 67 ++++++++++++++++++++++------------------- docs/zh/OVERVIEW.md | 2 +- 6 files changed, 104 insertions(+), 94 deletions(-) diff --git a/README.md b/README.md index 176f84a..bbb0365 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ homed（内核零 IO） ← PluginSDK → 插件（所有 IO 能力） **核心域与应用域分离** — 内核只做 LLM 编排、记忆管理、知识检索；所有 IO 能力（收发消息、读写文件、网络请求、硬件交互）全由插件实现。插件可热加载、独立开发、独立发布。这不是 RPC 框架的微服务拆分，而是 Agent 框架层次的领域划分。 **三层记忆架构** — 解决 Agent 长期运行的记忆衰减问题： -- **Context 层**：TF-IDF 相关性评分的事件窗口，维护最近 topK 条上下文 +- **Context 层**：预训练词嵌入 / TF-IDF 回退的相关性评分事件窗口，保护最近 10 条，维护 topK 条上下文 - **Document 层**：临时记忆，冷数据自动下沉，也支持用户主动提交 - **Graph 层**：SQLite 图数据库，持久化实体关系和语义记忆，支持蒸馏管道从原始对话中提取三元组 @@ -40,14 +40,14 @@ sequenceDiagram rect lavender Note over EV: processTextInput EV->>ST: StageOnInput 插件可改写/短路 - EV->>CTX: Prune(input,topK) TF-IDF裁剪 - CTX->>MEM: 低分事件归档 Document - EV->>CTX: Append(input) 5s写盘 + EV->>CTX: Prune(input,topK) StaticEmbedder/TF-IDF余弦相似度裁剪 + CTX->>MEM: 低分事件归档 Document (原始时间戳) + EV->>CTX: Append(input) CleanTemplateText→三分支向量→5s写盘 end rect lightgreen Note over EV,LLM: process() - EV->>MEM: buildMemoryContext Indexer召回Graph - EV->>MEM: buildSystemPrompt 人格+记忆+技能注入 + EV->>MEM: buildMemoryContext Indexer召回Graph(向量+jieba→BFS depth=2) + EV->>MEM: buildSystemPrompt DocQuery摘要+Graph记忆索引+人格+技能 EV->>ST: StagePreAction 插件可预拦截 loop 工具循环 LLM->>LLM: drainInterrupts @@ -99,21 +99,21 @@ flowchart LR flowchart TB subgraph C[① Context 工作窗口] RC[RelevanceContext] - A[Append] -->|Vectorize char 1-2gram| RC - P[Prune TF-IDF Cosine] -->|低分| D - P -->|保留| TL[timeline→system prompt] + A[Append] -->|CleanTemplateText→三分支向量| RC + P[Prune StaticEmbedder/TF-IDF Cosine] -->|低分原始时间戳| D + P -->|保留| TL[timeline→按时间排序→system prompt] end subgraph D[② Document 文件记忆] DS[DocStore JSON+TF-IDF] - Q1[Query 自动注入] -->|【相关记忆文档】| SP - Q2[doc_query LLM主动] -->|Consume+删除| DS - Q2 -->|原始时间戳写入| RC + Q1[Query 摘要自动注入] -->|【相关记忆文档】| SP + Q2[doc_query LLM主动召回] -->|Consume+删除源| DS + Q2 -->|原始时间戳写入上下文| RC CD[FindColdDocs 72h] -->|docToTriples| G end subgraph G[③ Graph 图数据库] DB[(SQLite)] - IDX[Indexer BFS depth=2] -->|【记忆索引】| SP - MEM[memory_recall/commit] + IDX[Indexer 向量+jieba→BFS depth=2] -->|【记忆索引】| SP + MEM[memory_recall/commit/merge/purge/edit] SOC[person_query/set_trait] end subgraph H[④ 心跳蒸馏] @@ -160,7 +160,7 @@ cmd/waiter/ CLI 客户端（Unix socket） internal/ ├── agent/core/ Agent 核心：事件循环、LLM 工具循环、7 阶段管道 ├── agent/api/ LLM Provider + 8 个 Lua 适配器 -├── memory/ 三层记忆：Graph(SQLite) / Document(JSON+TF-IDF) / Text(JSONL) +├── memory/ 三层记忆：Graph(SQLite) / Document(JSON+TF-IDF) / Text(JSONL) + StaticEmbedder(预训练词嵌入/TF-IDF回退) + CleanTemplateText(去模版) ├── knowledge/ 知识库（文件系统 + TF-IDF） ├── plugin/ 插件注册表 + .so 动态加载器 ├── plugins/ 内置 10 个插件（webui/cli/timer/cmd/mcp/openclaw/agentcli/healthcheck/pluginmgr/files） diff --git a/README_EN.md b/README_EN.md index df4393b..2511ddb 100644 --- a/README_EN.md +++ b/README_EN.md @@ -17,7 +17,7 @@ homed (kernel, zero IO) ← PluginSDK → plugins (all IO capabilities) **Separation of Core Domain and Application Domain** — The kernel only handles LLM orchestration, memory management, and knowledge retrieval; all IO capabilities (sending/receiving messages, reading/writing files, network requests, hardware interaction) are implemented by plugins. Plugins can be hot-loaded, independently developed, and independently released. This is not a microservice split of an RPC framework, but a domain-level separation in Agent framework design. **Three-Layer Memory Architecture** — Solves the memory decay problem for long-running agents: -- **Context Layer**: TF-IDF relevance-scored event window, maintains recent topK context entries +- **Context Layer**: Pretrained word embedding / TF-IDF fallback relevance-scored event window, protects last 10 entries, maintains topK context entries - **Document Layer**: Temporary memory with automatic cold data sinking, also supports user-initiated submissions - **Graph Layer**: SQLite graph database, persists entity relationships and semantic memory, supports distillation pipelines to extract triples from conversations @@ -40,14 +40,14 @@ sequenceDiagram rect lavender Note over EV: processTextInput EV->>ST: StageOnInput Plugin can rewrite/short-circuit - EV->>CTX: Prune(input,topK) TF-IDF pruning - CTX->>MEM: Low-score events archived to Document - EV->>CTX: Append(input) 5s debounce write + EV->>CTX: Prune(input,topK) StaticEmbedder/TF-IDF cosine pruning + CTX->>MEM: Low-score events archived to Document (original timestamp) + EV->>CTX: Append(input) CleanTemplateText→three-branch vector→5s write end rect lightgreen Note over EV,LLM: process() - EV->>MEM: buildMemoryContext Indexer recalls from Graph - EV->>MEM: buildSystemPrompt Persona+Memory+Skills injection + EV->>MEM: buildMemoryContext Indexer recalls from Graph (vector+jieba→BFS depth=2) + EV->>MEM: buildSystemPrompt DocQuery summary+Graph memory index+Persona+Skills EV->>ST: StagePreAction Plugin can pre-intercept loop Tool loop LLM->>LLM: drainInterrupts @@ -99,21 +99,21 @@ flowchart LR flowchart TB subgraph C[① Context Working Window] RC[RelevanceContext] - A[Append] -->|Vectorize char 1-2gram| RC - P[Prune TF-IDF Cosine] -->|Low score| D - P -->|Keep| TL[timeline→system prompt] + A[Append] -->|CleanTemplateText→three-branch vector| RC + P[Prune StaticEmbedder/TF-IDF Cosine] -->|Low score original timestamp| D + P -->|Keep| TL[timeline→chronological→system prompt] end subgraph D[② Document File Memory] DS[DocStore JSON+TF-IDF] - Q1[Query auto-inject] -->|[Related Memory Docs]| SP - Q2[doc_query LLM active] -->|Consume+delete| DS - Q2 -->|Original timestamp write| RC + Q1[Query summary auto-inject] -->|[Related Memory Docs]| SP + Q2[doc_query LLM active recall] -->|Consume+delete source| DS + Q2 -->|Original timestamp write to context| RC CD[FindColdDocs 72h] -->|docToTriples| G end subgraph G[③ Graph Database] DB[(SQLite)] - IDX[Indexer BFS depth=2] -->|[Memory Index]| SP - MEM[memory_recall/commit] + IDX[Indexer vector+jieba→BFS depth=2] -->|[Memory Index]| SP + MEM[memory_recall/commit/merge/purge/edit] SOC[person_query/set_trait] end subgraph H[④ Heartbeat Distillation] @@ -160,7 +160,7 @@ cmd/waiter/ CLI client (Unix socket) internal/ ├── agent/core/ Agent core: event loop, LLM tool loop, 7-stage pipeline ├── agent/api/ LLM Provider + 8 Lua adapters -├── memory/ Three-layer memory: Graph(SQLite) / Document(JSON+TF-IDF) / Text(JSONL) +├── memory/ Three-layer memory: Graph(SQLite) / Document(JSON+TF-IDF) / Text(JSONL) + StaticEmbedder(pretrained word embedding/TF-IDF fallback) + CleanTemplateText(de-template) ├── knowledge/ Knowledge base (filesystem + TF-IDF) ├── plugin/ Plugin registry + .so/.dll dynamic loader ├── plugins/ 10 built-in plugins (webui/cli/timer/cmd/mcp/openclaw/agentcli/healthcheck/pluginmgr/files) diff --git a/docs/en/ARCHITECTURE.md b/docs/en/ARCHITECTURE.md index 1fd13b4..4f42a7e 100644 --- a/docs/en/ARCHITECTURE.md +++ b/docs/en/ARCHITECTURE.md @@ -75,42 +75,44 @@ Setting `ctx.Response` at any stage jumps to `after_output`. ``` ① Context (Working Window) RelevanceContext — In-memory events[] + JSON persistence - Append: Each input, Vectorize(char 1-2gram TF-IDF) - Prune: TF-IDF CosineSimilarity, keep topK + last 10 - ├── Keep → timeline → system prompt (sorted by time) - └── Low score → Document layer archive (ContextToDoc) + Append: Each input, CleanTemplateText → three-branch vector(textForVector) + agent→Response, user→Input, cold_storage→Input+Response + StaticEmbedder pretrained word embedding / TF-IDF fallback + Prune: StaticEmbedder CosineSimilarity, keep topK + last 10 + ├── Keep → timeline → chronologically sorted → system prompt + └── Low score → Document layer archive (original timestamp) Save: 5s debounce write to disk - ↓ Prune archive ↑ LLM active recall + ↓ Prune archive ↑ LLM active recall ② Document (File Memory) DocStore — JSON files + TF-IDF InvertedIndex Write: Prune archive / doc_commit / Graph snapshot (syncGraphToDocs) Read: - ├── Auto-inject: Query(input, top3) → [Related Memory Docs] → system prompt (read-only, update AccessCount) + ├── Auto-inject: Query(input, top3) → similarity summary → [Related Memory Docs] → system prompt (read-only) └── LLM active: doc_query → Consume(read and delete) → context.Append{Timestamp: d.CreatedAt, Source: "cold_storage"} per doc → Docs written to context timeline with original timestamps, deleted from docStore Cold: FindColdDocs(72h, ≤2 accesses) → docToTriples → Graph - ↓ Cold doc distillation ↑ Auto recall + ↓ Cold doc distillation ↑ Auto recall ③ Graph (Graph Database) SQLite — entities + relations tables - Write: memory_commit / cold doc distillation / Pipeline rule distillation + Write: memory_commit / cold doc distillation / Pipeline rule distillation / memory_merge Read: ├── Auto recall: Indexer.BuildContext(input) - │ → TF-IDF entity name search → BFS depth=2 + │ → CleanTemplateText → vector entity search + jieba keywords → SQLite LIKE + BFS depth=2 │ → [Memory Index] → system prompt - └── LLM active: memory_recall / doc_query + └── LLM active: memory_recall / memory_merge / memory_purge / memory_edit / memory_delete_entity Social: person_query / set_trait / relate (wraps GraphDB) ④ Distillation Pipeline (30min heartbeat) distillContext → window > 2×maxSize → force Prune syncGraphToDocs → Graph snapshot to Document (cross-layer searchable) reorgGraph: - Step1: indexer.Sync — rebuild entity TF-IDF vector index - Step2: docStore.Reindex — rebuild document TF-IDF vector index + Step1: indexer.Sync — rebuild entity vector index + Step2: docStore.Reindex — rebuild document vector index Step3: Cold docs → docToTriples → GraphDB.Commit Step4: Entity similarity (Bigram Jaccard > 0.75) → consolidation → LLM decides merge Step5: evaluateGraphQuality → LLM decides keep/delete @@ -121,36 +123,35 @@ Setting `ctx.Response` at any stage jumps to `after_output`. → triples → GraphDB.Commit ``` -### Vectorization: Two Strategies +### Vectorization: Pretrained Word Embedding + TF-IDF Fallback -Vectorization is used in 4 independent locations with different strategies: +All vectorization unified under `StaticEmbedder` (`internal/memory/static_embedder.go`): -**Strategy A — Local Word Embedding** (`LocalWordEmbedder`, `internal/memory/embedder.go`), used by Context layer: - -- **jieba tokenization** → removes stop words and single characters -- **TF-IDF** as base word weight -- **Sliding window (size=5)** counts word co-occurrence → **PMI (Pointwise Mutual Information)** → keeps top 50 -- **Vectorization**: `vec[ctx] += TF-IDF × PMI` + self-tag `__w__` + TF-IDF - -**Strategy B — char-bigram TF-IDF + jieba keyword extraction** (`TFIDFVectorizer` + `ExtractKeywords`), used by Document and Indexer layers: - -- **char bigram tokenization** (1-2 gram) for entity name vector search -- **jieba tokenization** for keyword extraction, paired with SQLite LIKE + BFS traversal -- **TF-IDF weights** + **inverted index** +**Primary Strategy — Pretrained Word Embedding (aligned 300d)** +- Model sources: ConceptNet Numberbatch (77-language aligned) / fastText Chinese / fastText English +- Configured via `core.agent.embedding_model_path` (comma-separated multi-model) +- Path containing `numberbatch` → auto-download ConceptNet; `cc.zh.` → fastText Chinese; `cc.en.` → fastText English +- Falls back to ConceptNet by default if no match +- **Pre-processing**: `CleanTemplateText` strips QQ tool-call templates and timestamp noise +- **Three-branch vector source**: agent→Response, user→Input, cold_storage→Input+Response +- **TF-IDF fallback**: auto-fallback to bag-of-words TF-IDF if model download fails or not configured | Location | File | Purpose | Algorithm | |----------|------|---------|-----------| -| Context Prune | `context.go:161` | Trim low-relevance context events | LocalWordEmbedder → CosineSimilarity(queryVec, evt.Vector) | -| DocStore Query | `document.go:198` | Recall related content from document memory | char-bigram TF-IDF + jieba keywords → InvertedIndex + CosineSimilarity | -| Indexer Entity Search | `indexer.go:149` | Recall related entities from Graph | char-bigram TF-IDF vector search + jieba keywords → InvertedIndex + CosineSimilarity + SQLite BFS | -| Entity Similarity Detection | `agent.go:2297` | Detect similar entities in Graph | Bigram Jaccard (>0.75 → consolidation) | +| Context Prune | `context.go:155` | Trim low-relevance context events | VectorizeClean → CosineSimilarity(queryVec, evt.Vector) | +| DocStore Query | `document.go:206` | Recall from document memory | TF-IDF Vectorize → vec.Search | +| Indexer Entity Search | `indexer.go:96+111` | Recall from Graph | vector entity search + jieba keywords → SQLite LIKE + BFS | +| Entity Similarity Detection | `agent.go` | Detect similar entities in Graph | Bigram Jaccard (>0.75 → consolidation) | ### Context Layer `internal/agent/core/context.go` — `RelevanceContext` - Maintains recent event list, writes JSON on each Append/Prune to prevent data loss -- Word embedding relevance scoring on user input (LocalWordEmbedder → CosineSimilarity), keeps topK +- Pre-vectorization pipeline runs through `CleanTemplateText` to remove template noise +- Three-branch `textForVector`: agent events → Response, user events → Input, cold_storage → Input+Response +- Pretrained word embedding `StaticEmbedder` → CosineSimilarity, auto-fallback to TF-IDF if unavailable - Protects last 10 events from eviction; excess candidates are sorted by relevance and archived to document memory +- Archived events retain original timestamps; on `doc_query` recall they re-insert into the context timeline at their original position ### Document Layer @@ -171,6 +172,10 @@ Vectorization is used in 4 independent locations with different strategies: |------|---------| | `memory_recall` | Recall from Graph | | `memory_commit` | Write triples to Graph | +| `memory_merge` | Merge two entity nodes | +| `memory_purge` | Delete entity node | +| `memory_edit` | Edit existing entity/relation | +| `memory_delete_entity` | Delete entity and all its relations | | `memory_introspect` | View memory statistics | | `doc_query` | Search from Document | | `doc_commit` | Write to Document | diff --git a/docs/en/OVERVIEW.md b/docs/en/OVERVIEW.md index 650b4d9..06e6a34 100644 --- a/docs/en/OVERVIEW.md +++ b/docs/en/OVERVIEW.md @@ -37,7 +37,7 @@ Code is in the project root, implemented in Go. - Maintains a message loop (`eventLoop`), queuing input from the IO layer - Each input goes through the full processing pipeline: memory recall → persona injection → LLM call → tool execution → output delivery - LLM calls abstracted through Provider interface, supports 8 LLM sources with automatic fallback -- Context management (`context.go`) based on word embedding scoring (LocalWordEmbedder → CosineSimilarity), automatic pruning of low-relevance events +- Context management (`context.go`) based on pretrained word embedding scoring (StaticEmbedder → CosineSimilarity, TF-IDF fallback), automatic pruning of low-relevance events **Memory System** (`internal/memory/`): - **GraphDB** (`graph.go`) — SQLite, entities + relations tables, BFS traversal diff --git a/docs/zh/ARCHITECTURE.md b/docs/zh/ARCHITECTURE.md index 6df707b..127b445 100644 --- a/docs/zh/ARCHITECTURE.md +++ b/docs/zh/ARCHITECTURE.md @@ -75,42 +75,44 @@ eventLoop() → processTextInput() ``` ① Context (工作窗口) RelevanceContext — 内存 events[] + JSON持久化 - Append: 每次输入, Vectorize(char 1-2gram TF-IDF) - Prune: TF-IDF CosineSimilarity, 保留 topK + 最近10条 - ├── 保留 → timeline → system prompt (按时间排序) - └── 低分 → Document 层归档 (ContextToDoc) + Append: 每次输入, CleanTemplateText → 三分支向量(textForVector) + agent事件→Response, 用户事件→Input, cold_storage→Input+Response + StaticEmbedder 预训练词嵌入 / TF-IDF 回退 + Prune: StaticEmbedder CosineSimilarity, 保留 topK + 最近10条 + ├── 保留 → timeline → 按时间排序 → system prompt + └── 低分 → Document 层归档 (原始时间戳) Save: 5s debounce 写盘 - ↓ Prune 归档 ↑ LLM 主动召回 + ↓ Prune 归档 ↑ LLM 主动召回 ② Document (文件记忆) DocStore — JSON文件 + TF-IDF InvertedIndex 写入: Prune归档 / doc_commit / Graph快照(syncGraphToDocs) 读取: - ├── 自动注入: Query(input, top3) → 【相关记忆文档】→ system prompt (只读, 更新 AccessCount) + ├── 自动注入: Query(input, top3) → 相似度摘要 → 【相关记忆文档】→ system prompt (只读) └── LLM主动: doc_query → Consume(读取并删除) → 逐条 context.Append{Timestamp: d.CreatedAt, Source: "cold_storage"} → 文档以原始时间戳写入 context 时间线, 从 docStore 删除 冷化: FindColdDocs(72h, ≤2次访问) → docToTriples → Graph - ↓ 冷文档蒸馏 ↑ 自动召回 + ↓ 冷文档蒸馏 ↑ 自动召回 ③ Graph (图数据库) SQLite — entities + relations 表 - 写入: memory_commit / 冷文档蒸馏 / Pipeline 规则蒸馏 + 写入: memory_commit / 冷文档蒸馏 / Pipeline 规则蒸馏 / memory_merge 读取: ├── 自动召回: Indexer.BuildContext(input) - │ → TF-IDF 实体名搜索 → BFS depth=2 + │ → CleanTemplateText → 向量实体搜索 + jieba关键词 → SQLite LIKE + BFS depth=2 │ → 【记忆索引】→ system prompt - └── LLM主动: memory_recall / doc_query + └── LLM主动: memory_recall / memory_merge / memory_purge / memory_edit / memory_delete_entity Social: person_query / set_trait / relate (包装 GraphDB) ④ 蒸馏管道 (每30min心跳) distillContext → 窗口>2×maxSize → 强制Prune syncGraphToDocs → Graph 快照写入 Document(跨层可搜索) reorgGraph: - Step1: indexer.Sync — 重建实体TF-IDF向量索引 - Step2: docStore.Reindex — 重建文档TF-IDF向量索引 + Step1: indexer.Sync — 重建实体向量索引 + Step2: docStore.Reindex — 重建文档向量索引 Step3: 冷文档 → docToTriples → GraphDB.Commit Step4: 实体相似度(Bigram Jaccard>0.75) → consolidation → LLM判断合并 Step5: evaluateGraphQuality → LLM判断保留/删除 @@ -121,36 +123,35 @@ eventLoop() → processTextInput() → 三元组 → GraphDB.Commit ``` -### 向量化算法：两种策略 +### 向量化：预训练词嵌入 + TF-IDF 回退 -向量化在 4 个独立位置以不同方式使用： +所有向量化统一使用 `StaticEmbedder`（`internal/memory/static_embedder.go`）： -**策略 A — 局部词嵌入**（`LocalWordEmbedder`, `internal/memory/embedder.go`），用于 Context 层： - -- **jieba 分词** → 去除停用词和单字 -- **TF-IDF** 作为基础词权重 -- **滑动窗口（size=5）** 统计词对共现 → **PMI（点互信息）** → 保留 top 50 -- **向量化**：`vec[ctx] += TF-IDF × PMI` + 自身上标 `__w__` + TF-IDF - -**策略 B — char-bigram TF-IDF + jieba 关键词提取**（`TFIDFVectorizer` + `ExtractKeywords`），用于 Document 和 Indexer 层： - -- **char bigram 分词**（1-2 gram）用于实体名向量搜索 -- **jieba 分词**用于关键词提取，配合 SQLite LIKE + BFS 遍历 -- **TF-IDF 权重** + **倒排索引** +**主策略 — 预训练词嵌入（词对齐 300 维）** +- 模型来源：ConceptNet Numberbatch（77 语对齐）/ fastText 中文 / fastText 英文 +- 通过 `core.agent.embedding_model_path` 配置（逗号分隔多模型） +- 路径名含 `numberbatch` → 自动下载 ConceptNet，含 `cc.zh.` → fastText 中文，含 `cc.en.` → fastText 英文 +- 不匹配则默认 ConceptNet +- **前处理**：`CleanTemplateText` 剥离 QQ 工具调用模版、时间戳噪声，避免垃圾干扰相似度 +- **三分支向量来源**：agent→Response，用户→Input，cold_storage→Input+Response +- **TF-IDF 回退**：模型下载失败或未配置时自动回退词袋 TF-IDF，服务不中断 | 位置 | 文件 | 用途 | 算法 | |------|------|------|------| -| Context Prune | `context.go:161` | 裁剪低相关性上下文事件 | LocalWordEmbedder → CosineSimilarity(queryVec, evt.Vector) | -| DocStore Query | `document.go:198` | 从文档记忆召回相关内容 | char-bigram TF-IDF + jieba 关键词 → InvertedIndex + CosineSimilarity | -| Indexer 实体搜索 | `indexer.go:149` | 从Graph召回相关实体 | char-bigram TF-IDF 向量搜索 + jieba 关键词 → InvertedIndex + CosineSimilarity + SQLite BFS | -| 实体相似度检测 | `agent.go:2297` | 检测Graph中相似实体 | Bigram Jaccard (>0.75 → consolidation) | +| Context Prune | `context.go:155` | 裁剪低相关性上下文事件 | VectorizeClean → CosineSimilarity(queryVec, evt.Vector) | +| DocStore Query | `document.go:206` | 文档记忆召回 | TF-IDF Vectorize → vec.Search | +| Indexer 实体搜索 | `indexer.go:96+111` | Graph实体召回 | 向量实体搜索 + jieba关键词 → SQLite LIKE + BFS | +| 实体相似度检测 | `agent.go` | Graph中相似实体 | Bigram Jaccard (>0.75 → consolidation) | ### Context 层 `internal/agent/core/context.go` — `RelevanceContext` - 维护最近事件列表，每次 Append/Prune 写入 JSON 防丢 -- 用户输入时做词嵌入相关性评分（LocalWordEmbedder → CosineSimilarity），保留 topK +- 向量化前统一经 `CleanTemplateText` 去模版噪声 +- 三分支 `textForVector`：agent 事件用 Response、用户事件用 Input、cold_storage 用 Input+Response +- 预训练词嵌入 `StaticEmbedder` → CosineSimilarity，模型不可用时自动回退 TF-IDF - 保护最近 10 条记录免于淘汰，超出部分按相关性排序归档到文档记忆 +- 归档事件以原始时间戳写入文档记忆，后续 `doc_query` 召回时按原始时间戳插回时序 ### Document 层 @@ -171,6 +172,10 @@ eventLoop() → processTextInput() |------|------| | `memory_recall` | 从 Graph 召回 | | `memory_commit` | 写入 Graph 三元组 | +| `memory_merge` | 合并两个实体节点 | +| `memory_purge` | 删除指定实体 | +| `memory_edit` | 编辑已有实体/关系 | +| `memory_delete_entity` | 删除实体及其所有关系 | | `memory_introspect` | 查看记忆统计 | | `doc_query` | 从 Document 搜索 | | `doc_commit` | 写入 Document | diff --git a/docs/zh/OVERVIEW.md b/docs/zh/OVERVIEW.md index 70900b4..deef6c8 100644 --- a/docs/zh/OVERVIEW.md +++ b/docs/zh/OVERVIEW.md @@ -37,7 +37,7 @@ HomeAgent 是一个持续运行的个人智能 Agent 框架。 - 维护一个消息循环（`eventLoop`），从 IO 层排队接收输入 - 每次输入走完整的处理管道：记忆召回 → 人格注入 → LLM 调用 → 工具执行 → 输出发送 - LLM 调用通过 Provider 接口抽象，支持 8 个 LLM 源自动降级 -- 上下文管理（`context.go`）基于词嵌入评分（LocalWordEmbedder → CosineSimilarity），自动剪枝低相关性事件 +- 上下文管理（`context.go`）基于预训练词嵌入评分（StaticEmbedder → CosineSimilarity，TF-IDF回退），自动剪枝低相关性事件 **记忆系统** (`internal/memory/`)： - **GraphDB** (`graph.go`) — SQLite，entities + relations 表，BFS 遍历