From 8c6c8e71d26d388f05eb873fe0ebd23eb268051d Mon Sep 17 00:00:00 2001
From: root <root@qq.com>
Date: Fri, 17 Jul 2026 07:04:05 +0800
Subject: [PATCH] docs: graph memory uses jieba keyword extraction via
 ExtractKeywords

- Indexer uses dual recall: char-bigram TF-IDF vector search + jieba keywords
- Document layer: char-bigram TF-IDF + jieba keyword extraction
- Memory Indexer description: add jieba keyword extraction
---
 docs/en/ARCHITECTURE.md | 13 +++++++------
 docs/zh/ARCHITECTURE.md | 13 +++++++------
 2 files changed, 14 insertions(+), 12 deletions(-)

diff --git a/docs/en/ARCHITECTURE.md b/docs/en/ARCHITECTURE.md
index 795e5f1..7ec870b 100644
--- a/docs/en/ARCHITECTURE.md
+++ b/docs/en/ARCHITECTURE.md
@@ -128,16 +128,17 @@ Vectorization is used in 4 independent locations with different strategies:
 - **Sliding window (size=5)** counts word co-occurrence → **PMI (Pointwise Mutual Information)** → keeps top 50
 - **Vectorization**: `vec[ctx] += TF-IDF × PMI` + self-tag `__w__` + TF-IDF
 
-**Strategy B — char-bigram TF-IDF** (`TFIDFVectorizer`, `internal/memory/vector/`), used by Document and Indexer layers:
+**Strategy B — char-bigram TF-IDF + jieba keyword extraction** (`TFIDFVectorizer` + `ExtractKeywords`), used by Document and Indexer layers:
 
-- **char bigram tokenization** (1-2 gram)
+- **char bigram tokenization** (1-2 gram) for entity name vector search
+- **jieba tokenization** for keyword extraction, paired with SQLite LIKE + BFS traversal
 - **TF-IDF weights** + **inverted index**
 
 | Location | File | Purpose | Algorithm |
 |----------|------|---------|-----------|
 | Context Prune | `context.go:161` | Trim low-relevance context events | LocalWordEmbedder → CosineSimilarity(queryVec, evt.Vector) |
-| DocStore Query | `document.go:198` | Recall related content from document memory | InvertedIndex + CosineSimilarity |
-| Indexer Entity Search | `indexer.go:149` | Recall related entities from Graph | InvertedIndex + CosineSimilarity |
+| DocStore Query | `document.go:198` | Recall related content from document memory | char-bigram TF-IDF + jieba keywords → InvertedIndex + CosineSimilarity |
+| Indexer Entity Search | `indexer.go:149` | Recall related entities from Graph | char-bigram TF-IDF vector search + jieba keywords → InvertedIndex + CosineSimilarity + SQLite BFS |
 | Entity Similarity Detection | `agent.go:2297` | Detect similar entities in Graph | Bigram Jaccard (>0.75 → consolidation) |
 
 ### Context Layer
@@ -151,7 +152,7 @@ Vectorization is used in 4 independent locations with different strategies:
 
 `internal/memory/document/document.go` — `Store`
 - Consume-on-read mode: deleted after `doc_query` retrieval
-- TF-IDF index with character bigram + inverted index
+- Dual recall: char-bigram TF-IDF vector search + jieba keyword extraction
 
 ### Graph Layer
 
@@ -174,7 +175,7 @@ Vectorization is used in 4 independent locations with different strategies:
 
 - **Social** (`internal/memory/social/social.go`) — Persona traits and relationship network, wraps GraphDB entity types
 - **Text Memory** (`internal/memory/text/text.go`) — Raw conversation JSONL logs, rotation strategy
-- **Memory Indexer** (`internal/memory/indexer.go`) — Entity vectorization, auto-inject into system prompt
+- **Memory Indexer** (`internal/memory/indexer.go`) — Entity vectorization + jieba keyword extraction, auto-inject into system prompt
 
 ### Distillation Pipeline
 
diff --git a/docs/zh/ARCHITECTURE.md b/docs/zh/ARCHITECTURE.md
index 5360d07..eac775c 100644
--- a/docs/zh/ARCHITECTURE.md
+++ b/docs/zh/ARCHITECTURE.md
@@ -128,16 +128,17 @@ eventLoop() → processTextInput()
 - **滑动窗口（size=5）** 统计词对共现 → **PMI（点互信息）** → 保留 top 50
 - **向量化**：`vec[ctx] += TF-IDF × PMI` + 自身上标 `__w__` + TF-IDF
 
-**策略 B — char-bigram TF-IDF**（`TFIDFVectorizer`, `internal/memory/vector/`），用于 Document 和 Indexer 层：
+**策略 B — char-bigram TF-IDF + jieba 关键词提取**（`TFIDFVectorizer` + `ExtractKeywords`），用于 Document 和 Indexer 层：
 
-- **char bigram 分词**（1-2 gram）
+- **char bigram 分词**（1-2 gram）用于实体名向量搜索
+- **jieba 分词**用于关键词提取，配合 SQLite LIKE + BFS 遍历
 - **TF-IDF 权重** + **倒排索引**
 
 | 位置 | 文件 | 用途 | 算法 |
 |------|------|------|------|
 | Context Prune | `context.go:161` | 裁剪低相关性上下文事件 | LocalWordEmbedder → CosineSimilarity(queryVec, evt.Vector) |
-| DocStore Query | `document.go:198` | 从文档记忆召回相关内容 | InvertedIndex + CosineSimilarity |
-| Indexer 实体搜索 | `indexer.go:149` | 从Graph召回相关实体 | InvertedIndex + CosineSimilarity |
+| DocStore Query | `document.go:198` | 从文档记忆召回相关内容 | char-bigram TF-IDF + jieba 关键词 → InvertedIndex + CosineSimilarity |
+| Indexer 实体搜索 | `indexer.go:149` | 从Graph召回相关实体 | char-bigram TF-IDF 向量搜索 + jieba 关键词 → InvertedIndex + CosineSimilarity + SQLite BFS |
 | 实体相似度检测 | `agent.go:2297` | 检测Graph中相似实体 | Bigram Jaccard (>0.75 → consolidation) |
 
 ### Context 层
@@ -151,7 +152,7 @@ eventLoop() → processTextInput()
 
 `internal/memory/document/document.go` — `Store`
 - 消费即删模式：`doc_query` 检索到后删除
-- TF-IDF 索引 character bigram + 倒排
+- 双路召回：char-bigram TF-IDF 向量搜索 + jieba 关键词提取
 
 ### Graph 层
 
@@ -174,7 +175,7 @@ eventLoop() → processTextInput()
 
 - **Social** (`internal/memory/social/social.go`) — 人格特质和关系网，包装 GraphDB 实体类型
 - **Text Memory** (`internal/memory/text/text.go`) — 原始对话 JSONL 日志，轮转策略
-- **Memory Indexer** (`internal/memory/indexer.go`) — 实体向量化，自动注入 system prompt
+- **Memory Indexer** (`internal/memory/indexer.go`) — 实体向量化 + jieba 关键词提取，自动注入 system prompt
 
 ### 蒸馏管道