diff --git a/README.md b/README.md index 16055308..23158696 100644 --- a/README.md +++ b/README.md @@ -14,12 +14,12 @@ make build build-cli API 密钥在 WebUI（`http://localhost:8080`）里填。需要 Linux。 跟常见 agent 框架的主要差别在上下文管理：不是超窗就截断或让模型摘要， -而是按相关度取舍，低分的下沉成文档、再蒸馏进图数据库。另外两处不太一样的是 +而是按相关度取舍，低分的下沉成冷上下文、再蒸馏进图数据库。另外两处不太一样的是 启动顺序在编译期定序，以及插件按设施就绪提前启动（详见下方架构）。 ## 架构 -``` +```text homed（内核） LLM 编排 · 三层记忆 · 检索 · 调度 │ @@ -40,9 +40,9 @@ API 密钥在 WebUI（`http://localhost:8080`）里填。需要 Linux。 ### 上下文不是「截断 + 摘要」 常见做法是把超窗的对话裁掉或让模型自己摘要，两者的共同问题是**丢什么由位置决定**。 -这里换成了按相关度取舍：用当前输入给每条历史事件打分，低分的下沉到文档层。 +这里换成了按相关度取舍：用当前输入给每条历史事件打分，低分事件下沉到冷上下文层。 -打分用稠密向量余弦（与媒体、文档共享同一向量空间），模型未就绪时退化到 TF-IDF。 +打分用稠密向量余弦（与媒体、冷上下文共享同一向量空间），模型未就绪时退化到 TF-IDF。 裁剪时无条件保住最近 N 条，但 N 本身受预算约束： ```go @@ -55,18 +55,18 @@ pCount = min(配置值, budgetCap) 「钉住 10 条」本身就装不下，于是每轮裁剪都撞保护下限，两轮后预算耗尽，任务直接终止。 召回率从 44% 掉到 22%。现在窗口越小越自动收紧，1M 窗口下则能用满配置值。 -被裁掉的事件不是丢弃，是迁移：事件连同它持有的媒体块一起进文档层，块的 ID 不变、 +被裁掉的事件不是丢弃，是迁移：事件连同它持有的媒体块一起进冷上下文层，块的 ID 不变、 只换持有层。回灌时用 `coldcontext_query` 取回，两阶段（先 Peek 确认、再 Remove）， 不用「先删后灌」——那会在回灌失败时永久丢数据。 ### 三层记忆是迁移，不是复制 -``` -L0 上下文 相关性裁剪，最近 N 条恒受保护 - ↓ 低分事件下沉（带媒体块） -L1 文档 稠密检索，访问计数 ≤2 且久未访问则继续下沉 - ↓ 蒸馏成三元组 -L2 图 实体关系 + 场面索引，支持按场面召回 +```text +L1 上下文 Context 相关性裁剪，最近 N 条恒受保护 + ↓ 低分事件下沉（带媒体块） +L2 冷上下文 Cold Context 稠密检索，≤2 次访问且 72h 未访问则继续下沉 + ↓ 蒸馏成三元组 +L3 稳定记忆 Stable 实体关系 + 场面索引，支持按场面召回 ``` 块在任一层只有一份持有者，不靠引用计数保活。媒体不是附件：图片、音频是图里的一等节点， @@ -142,13 +142,9 @@ Windows 上构建 `waiter.exe`，`homed` 跑在 WSL2 里；macOS 要原生构建 ## 许可 -内核以 **AGPL-3.0-only** 发布（[LICENSE](LICENSE)）。改过的版本如果对外提供网络服务， -必须向使用者提供源码（§13）。 +以 **AGPL-3.0-only** 发布，全文见 [LICENSE](LICENSE)。 -自己写的插件不受此约束。插件静态链接的 [hologramagentsdk](https://github.com/JianFeeeee/hologramagentsdk) -是 **MIT**，宽松许可不继承 AGPL，所以外部插件可以闭源、可以商用，也不必开源自改。 - -随包第三方组件保持各自许可，全文在 `/usr/share/doc/hologramagent/licenses/`。 +随包的第三方组件保持各自许可，全文在 `/usr/share/doc/hologramagent/licenses/`。 ### 随包分发的第三方组件 @@ -159,5 +155,4 @@ Windows 上构建 `waiter.exe`，`homed` 跑在 WSL2 里；macOS 要原生构建 | jieba 词库（内嵌进二进制） | MIT | 源码 `internal/memory/jiebadict/` | | Go 依赖（go-sqlite3、gojieba、bubbletea 等） | MIT / BSD-3 / Apache-2.0 | 均为宽松许可，与 AGPL-3.0 兼容 | -这些组件**保持各自原有许可**，不在本项目的 AGPL 授权范围内；发行包把它们的许可全文放在 -`/usr/share/doc/hologramagent/licenses/`，并在 dep/rpm 元数据里声明本包许可为 `AGPL-3.0-only`。 +发行包在 dep/rpm 元数据里声明本包许可为 `AGPL-3.0-only`。 diff --git a/README_EN.md b/README_EN.md index cdcb5db1..e81b58de 100644 --- a/README_EN.md +++ b/README_EN.md @@ -15,12 +15,12 @@ API keys go in the web UI at `http://localhost:8080`. Requires Linux. The main difference from typical agent frameworks is context handling: instead of truncating on overflow or asking the model to summarize, it scores history by relevance and sinks the -low-scoring events into documents, then distills those into a graph. Two other things differ: +low-scoring events into cold context, then distills those into a graph. Two other things differ: startup order is resolved at build time, and plugins launch as soon as their facilities are ready. ## Architecture -``` +```text homed (kernel) LLM orchestration · memory · retrieval │ @@ -43,9 +43,9 @@ crash within 5 minutes stops automatic restarting for human intervention. The usual approach drops old turns or asks the model to summarize them. Both share a flaw: what gets lost is decided by position. Here each historical event is scored against the current input, -and low scorers sink into the document layer. +and low scorers sink into the cold-context layer. -Scoring uses dense cosine similarity in the same vector space as media and documents, falling +Scoring uses dense cosine similarity in the same vector space as media and cold context, falling back to TF-IDF when the model isn't loaded. The most recent N events are always kept, but N is itself bounded by budget: @@ -62,18 +62,18 @@ dropped from 44% to 22%. Smaller windows now tighten automatically, and a 1M win full configured value. Pruned events aren't discarded, they migrate: the event and the media blocks it holds move into -the document layer with block IDs unchanged, only the holding layer changes. Retrieval uses +the cold-context layer with block IDs unchanged, only the holding layer changes. Retrieval uses `coldcontext_query` in two phases (Peek to confirm, then Remove) rather than delete-then-refill, which would lose data permanently if the refill failed. ### Three layers migrate, they don't copy -``` -L0 context relevance pruning, last N always protected - ↓ low scorers sink (with media blocks) -L1 documents dense retrieval; access count ≤2 and stale means sink further - ↓ distilled into triples -L2 graph entities, relations, scene index; recall by scene +```text +L1 Context relevance pruning, last N always protected + ↓ low scorers sink (with media blocks) +L2 Cold Context dense retrieval; ≤2 accesses and 72h stale means sink further + ↓ distilled into triples +L3 Stable entities, relations, scene index; recall by scene ``` A block has exactly one holder at any layer, with no reference counting to keep it alive. Media @@ -155,27 +155,10 @@ macOS can build `waiter`/`initconfig`; `homed` must be built on native macOS. ## License -This project is released under the **GNU Affero General Public License, version 3 -(AGPL-3.0-only)** — see [LICENSE](LICENSE). +**AGPL-3.0-only**. See [LICENSE](LICENSE). -This is the strongest copyleft in the GPL family: besides shipping the complete corresponding -source when you distribute the software, **you must also offer the source to users who interact -with it over a network** (§13, Remote Network Interaction). Anyone running a modified HologramAgent -as a network service therefore has to make the modified source available to that service's users. - -Plugins are **statically linked** against this project through the public -[hologramagentsdk](https://github.com/JianFeeeee/hologramagentsdk) (the SDK source ends up inside the -plugin binary), but that repository is released under **MIT**. That is a permissive license, so code -received under it does **not** inherit this project's AGPL. External plugins are therefore **not -derivative works of this project**: authors pick their own license (closed-source, commercial or -private included), with no same-license obligation and no §13 network clause. The safety and -vitality of the third-party plugin ecosystem rest on this. - -The boundary is clean: **AGPL covers the kernel and the bundled plugins** (`homed`, `internal/`, -the 13 built-in plugins under `internal/plugins/`); **MIT covers the public SDK** (`sdk/`, whose -`go.mod` has zero external dependencies and imports only the Go standard library, never -any kernel code). Process isolation is beside the point here. What decides the -license is the linked SDK code itself, and that code is MIT. +Bundled third-party components keep their own licenses; full texts ship in +`/usr/share/doc/hologramagent/licenses/`. ### Third-party components shipped with the packages @@ -186,6 +169,4 @@ license is the linked SDK code itself, and that code is MIT. | jieba dictionary (embedded in the binary) | MIT | `internal/memory/jiebadict/` | | Go dependencies (go-sqlite3, gojieba, bubbletea, …) | MIT / BSD-3 / Apache-2.0 | permissive, AGPL-3.0-compatible | -These components keep their own licenses and are not relicensed by this project. Full texts are -shipped in `/usr/share/doc/hologramagent/licenses/`, and the package metadata declares this package -as `AGPL-3.0-only`. +The package metadata declares this package as `AGPL-3.0-only`. diff --git a/assets/branding/mascot-xiaozhai-guide.md b/assets/branding/mascot-xiaozhai-guide.md index 82c27979..0caa229c 100644 --- a/assets/branding/mascot-xiaozhai-guide.md +++ b/assets/branding/mascot-xiaozhai-guide.md @@ -29,7 +29,7 @@ 连衣裙 #FFFFFF → #F1F5F9 白→浅灰 丝带蓝 #3B82F6 Context 层 -丝带青 #06B6D4 Document 层 +丝带青 #06B6D4 冷上下文层 丝带金 #F59E0B Graph 层 徽章金 #F59E0B Kernel diff --git a/assets/docs/en/ARCHITECTURE.md b/assets/docs/en/ARCHITECTURE.md index 76926fcd..a5b90f6c 100644 --- a/assets/docs/en/ARCHITECTURE.md +++ b/assets/docs/en/ARCHITECTURE.md @@ -98,13 +98,13 @@ HologramAgent's cognitive architecture consists of three subsystems: the event l **The stage pipeline (StageHost)** manages two registration categories: tool definitions (ToolDef) and stage handlers (StageHandler). ToolDef includes two optional memory control fields: `NoMemory bool` — when true, the tool's output is excluded from vectorization/jieba/distillation (original text preserved); and `Cleaner func(string) string` — a filter applied before the output enters the computation layer (e.g., extracting a `content` field from JSON). Neither modifies the original output; both only affect the computation layer input. `RegisterTool` rejects duplicate names, infers the owning plugin name from the tool name prefix, and maintains a `toolPlugins` mapping. `RegisterStage` appends handlers to the corresponding stage list. On stage execution (`RunStage`), **all registered handlers execute in parallel via goroutines**, sharing a single `*StageContext` protected by `sync.RWMutex`. Individual handler panics are recovered independently without affecting other handlers. Short-circuit semantics are implemented by checking `ctx.Response != nil` — any stage handler can set this value to terminate the pipeline early. `ExecuteTool` includes built-in panic recovery with stack-trace recording. `UnregisterPluginTools` removes a plugin's tool set during hot-reload. -**The context window (RelevanceContext)** maintains a chronologically ordered event list. `Append` aggregates tool outputs through `textForVector` before computing the embedding vector: `NoMemory` skips, `Cleaner` filters (per-tool cleaners registered by plugins — e.g. the QQ plugin strips its own tool-call templates), and `CleanText` finalizes with basic whitespace normalization. A three-branch strategy selects the text source (agent events use Response, user events use Input, cold_storage uses Input+Response). `Prune` triggers when the event count exceeds `topK`: it **unconditionally protects the last 10 events from eviction** (recency bias), scores remaining candidates against the current input via CosineSimilarity, keeps `topK - 10` highest-scoring entries (floor at 0), then re-sorts chronologically. Pruned events from sources other than `agentcli` and `terminal` are archived to the Document layer via `docStore.ContextToDoc`, retaining original timestamps. Persistence uses 5-second debounced writes to a JSON file. +**The context window (RelevanceContext)** maintains a chronologically ordered event list. `Append` aggregates tool outputs through `textForVector` before computing the embedding vector: `NoMemory` skips, `Cleaner` filters (per-tool cleaners registered by plugins — e.g. the QQ plugin strips its own tool-call templates), and `CleanText` finalizes with basic whitespace normalization. A three-branch strategy selects the text source (agent events use Response, user events use Input, cold_storage uses Input+Response). `Prune` triggers when the event count exceeds `topK`: it **unconditionally protects the last 10 events from eviction** (recency bias), scores remaining candidates against the current input via CosineSimilarity, keeps `topK - 10` highest-scoring entries (floor at 0), then re-sorts chronologically. Pruned events from sources other than `agentcli` and `terminal` are archived to the Cold Context layer via `coldcontextStore.ContextToColdContext`, retaining original timestamps. Persistence uses 5-second debounced writes to a JSON file. **Tool definitions are aggregated from five sources**: IOManager-registered plugin tools; StageHost-registered SDK tools; Indexer-provided memory index tools; conditionally added built-in tools (depending on non-nil state of memory/knowledge/docStore/social/pluginReg/providerManager modules — including memory operations, knowledge retrieval, document queries, social networking, plugin reloading, child-agent spawning, per-output-channel send tools, and LLM source switching); and media processing tools added based on `pendingMedia` state. `buildToolDefs()` re-aggregates all sources on each process cycle. **Provider invocation follows an ordered fallback strategy**: `ProviderManager.OrderedProviders()` returns the provider list in registration order. The `process()` inner loop iterates this list attempting `Chat()` on each. HTTP 401/403 responses mark the provider as permanently unavailable; other error types also mark unavailability but with higher tolerance. If all providers fail, an error is returned to the caller. If a call is interrupted by context cancellation while the agent is still running, it is retried (only on non-consolidation paths). -**The memory system adopts a three-tier storage hierarchy (Context → Document → Graph), tiering data by access locality and persistence requirements**: the Context layer is a fast-volatile working window using StaticEmbedder (pretrained word embeddings with TF-IDF fallback) for semantic relevance scoring; the Document layer **shares the same StaticEmbedder vector space with Context** (the embedder is injected into the Document Store at agent startup via `docStore.SetVectorizer(embedder)`), ensuring that relevance scores during Context pruning and semantic retrieval during Document queries operate within the same vector space — TF-IDF serves only as a fallback when the embedder is unavailable; the Graph layer uses SQLite as its persistence substrate with an entities table (nodes) and a relations table (directed edges), supporting BFS traversal recall. Data migration policies govern movement across tiers: low-scoring events sink from Context to Document (vectorized using the same embedder at archival time); cold documents, after a 72-hour no-access threshold, are distilled into triples via `docToTriples` and committed to Graph. The Indexer uses dual retrieval (entity vector similarity search + jieba keyword extraction) to construct Graph query seeds, and the `MarkRecalled` mechanism prevents entities already fetched via tool calls from being re-injected into the system prompt. +**The memory system adopts a three-tier storage hierarchy (Context → Cold Context → Graph), tiering data by access locality and persistence requirements**: the Context layer is a fast-volatile working window using StaticEmbedder (pretrained word embeddings with TF-IDF fallback) for semantic relevance scoring; the Cold Context layer **shares the same StaticEmbedder vector space with Context** (the embedder is injected into the coldcontext Store at agent startup), ensuring that relevance scores during Context pruning and semantic retrieval during cold-context queries operate within the same vector space — TF-IDF serves only as a fallback when the embedder is unavailable; the Graph layer uses SQLite as its persistence substrate with an entities table (nodes) and a relations table (directed edges), supporting BFS traversal recall. Data migration policies govern movement across tiers: low-scoring events sink from Context to Cold Context (vectorized using the same embedder at archival time); cold-context docs, after a 72-hour no-access threshold, are distilled into triples via `coldDocToTriples` and committed to Graph. The Indexer uses dual retrieval (entity vector similarity search + jieba keyword extraction) to construct Graph query seeds, and the `MarkRecalled` mechanism prevents entities already fetched via tool calls from being re-injected into the system prompt. **The separation of core domain and application domain** constrains the kernel's responsibilities to LLM orchestration, memory management, and knowledge retrieval — no direct IO operations; all external interaction is mediated through the plugin domain. This separation limits the kernel's complexity to a verifiable scope while granting the plugin domain independent evolution: plugins can be independently developed, independently released, hot-loaded, and do not directly affect the stability of the core domain. @@ -120,7 +120,7 @@ eventLoop() → processTextInput() │ ├── on_input stage Plugins can intercept/rewrite/short-circuit ├── Context.Append Record to context window - ├── Context.Prune Low-relevance events archived to Document + ├── Context.Prune Low-relevance events archived to Cold Context ├── buildMemoryContext() Indexer recall → GraphDB BFS traversal │ ├── pre_action stage Plugins can inject system messages @@ -181,20 +181,20 @@ Setting `ctx.Response` at any stage jumps to `after_output`. Vector layers: unified multimodal space (primary, with fingerprint) → StaticEmbedder word embedding → TF-IDF (fallback) Prune: DenseCosine (compared only within the same fingerprint) → StaticEmbedder CosineSimilarity fallback; keep topK + last 10 ├── Keep → timeline → chronologically sorted → system prompt - └── Low score → Document layer archive (original timestamp) + └── Low score → Cold Context layer archive (original timestamp) Save: 5s debounce write to disk ↓ Prune archive ↑ LLM active recall -② Document (File Memory) +② Cold Context (File Memory) DocStore — JSON files + dense vectors (unified multimodal space; dense_fp must match the current space fingerprint or the doc is recomputed; fallback: StaticEmbedder / TF-IDF InvertedIndex) Write: Prune archive / coldcontext_commit / stablemem snapshot (syncGraphToDocs) Read: ├── Auto-inject: Query(input, top3) → similarity summary under same vector space → [Related Memory Docs] → system prompt (read-only) └── LLM active: coldcontext_query → Consume(read and delete) → context.Append{Timestamp: d.CreatedAt, Source: "cold_storage"} per doc - → Docs written to context timeline with original timestamps, deleted from docStore - Cold: FindColdDocs(72h, ≤2 accesses) → docToTriples → Graph + → Docs written to context timeline with original timestamps, deleted from coldcontextStore + Cold: FindColdContexts(72h, ≤2 accesses) → coldDocToTriples → Graph ↓ Cold doc distillation ↑ Auto recall @@ -209,8 +209,8 @@ Setting `ctx.Response` at any stage jumps to `after_output`. Social: person_query / set_trait / relate (wraps GraphDB) ④ Four Independent Heartbeat Loops (separate tickers and config intervals) - distillLoop (distillInterval, default 30m): Context pruning — Context.Prune → Document - archiveLoop (archiveInterval, default 60m): Cold doc archival — docToTriples → GraphDB + distillLoop (distillInterval, default 30m): Context pruning — Context.Prune → Cold Context + archiveLoop (archiveInterval, default 60m): Cold context archival — coldDocToTriples → GraphDB mergeLoop (mergeInterval, default 120m): Entity merge detection — similarity → LLM decision reviewLoop (reviewInterval, default 120m): Relation review — SentenceRef recall → LLM fix @@ -268,7 +268,7 @@ dimension** participate in fusion (mixing coordinate systems yields a direction - Protects last 10 events from eviction; excess candidates are sorted by relevance and archived to document memory - Archived events retain original timestamps; on `coldcontext_query` recall they re-insert into the context timeline at their original position -### Document Layer +### Cold Context Layer `internal/memory/document/document.go` — `Store` - Consume-on-read mode: deleted after `coldcontext_query` retrieval @@ -389,9 +389,9 @@ switched), fetching bytes on demand via `BlockData`. Modalities outside the spac ```text Four independent heartbeat loops (each with configurable interval): ├── distillLoop (distillInterval, default 30m) - │ └── distillContext() — Context.Prune → Document + │ └── distillContext() — Context.Prune → Cold Context ├── archiveLoop (archiveInterval, default 60m) - │ └── archiveColdDocs() — Cold docs → docToTriples → GraphDB + │ └── archiveColdDocs() — Cold docs → coldDocToTriples → GraphDB ├── mergeLoop (mergeInterval, default 120m) │ └── detectEntityMerge() — Entity similarity detection → LLM decision └── reviewLoop (reviewInterval, default 120m) @@ -857,7 +857,7 @@ internal/ │ ├── graph.go — SQLite graph database │ ├── indexer.go — Graph → vector index │ ├── vector/store.go — TF-IDF vector engine -│ ├── document/document.go — Document memory +│ ├── coldcontext/coldcontext.go — Cold Context memory │ ├── text/text.go — Text logs │ └── pipeline/ — Distiller ├── knowledge/knowledge.go — Knowledge base diff --git a/assets/docs/en/OVERVIEW.md b/assets/docs/en/OVERVIEW.md index d1424e4f..f400eb86 100644 --- a/assets/docs/en/OVERVIEW.md +++ b/assets/docs/en/OVERVIEW.md @@ -29,7 +29,7 @@ require touching the kernel, and no second version of business truth accumulates **Three-Layer Memory Architecture** — Manages information retention across long agent runtimes through a tiered storage strategy: - **Context Layer**: In-memory pretrained word embedding scored event window (StaticEmbedder word vectors → CosineSimilarity, TF-IDF fallback), maintains recent context in real-time, low-relevance events automatically sink to the next layer -- **Document Layer**: JSON files + TF-IDF vector-indexed temporary memory, supports explicit submission and implicit archival, cold data distills to Graph +- **Cold Context Layer (L2)**: JSON-file-backed cold memory with dense + TF-IDF retrieval, supports explicit submission and implicit archival, cold data distills to Graph - **Stablemem Layer**: SQLite graph database, persists entities and relations, BFS traversal recall, distillation pipeline extracts triples from conversations Three progressive layers — context, cold archive, long-term stable memory — form an information decay and consolidation pipeline from short-term to persistent storage. @@ -41,7 +41,7 @@ Three progressive layers — context, cold archive, long-term stable memory — - **Blocks carry their own multimodal vector and fingerprint**, and are retrieved directly — there is no media marker in the body text any more. - **No separate GC, no reference counting, no keep-set**: blocks follow the single-layer invariant — - Context → Document → Graph is a **migration** of a block, not a copy, and nothing is kept alive by + Context → Cold Context → Graph is a **migration** of a block, not a copy, and nothing is kept alive by references; **deleting a block deletes its content.** - **Reaches the plugin boundary since v1.1.1**: plugins read and write media through `InsertWithMedia` / `InjectInputMedia`; the model attaches media via the `media_digests` argument of `memory_commit` / `coldcontext_commit`. @@ -64,7 +64,7 @@ Code is in the project root, implemented in Go. **Memory System** (`internal/memory/`): - **GraphDB** (`graph.go`) — SQLite, entities + relations tables, BFS traversal -- **Document Store** (`document/document.go`) — Temporary memory, JSON files + TF-IDF vector index, consume-on-read +- **Cold Context Store** (`internal/memory/coldcontext/`) — cold memory with two-phase retrieval (Peek, then Remove) - **Text Memory** (`text/text.go`) — Raw conversation logs, JSONL file rotation - **Social Store** (`social/social.go`) — Persona traits + relationship network, wraps GraphDB - **Memory Indexer** (`indexer.go`) — Auto-vectorizes GraphDB entities, recalls and injects into system prompt on user input diff --git a/assets/docs/zh/ARCHITECTURE.md b/assets/docs/zh/ARCHITECTURE.md index d7a619de..0d9f65a5 100644 --- a/assets/docs/zh/ARCHITECTURE.md +++ b/assets/docs/zh/ARCHITECTURE.md @@ -87,7 +87,7 @@ HologramAgent 的认知架构由三个核心子系统构成：事件循环（eve **阶段管道（StageHost）** 管理两类注册：工具定义（ToolDef）与阶段处理器（StageHandler）。ToolDef 包含 `NoMemory bool` 和 `Cleaner func(string) string` 两个可选的记忆控制字段：`NoMemory=true` 时工具输出不参与向量化/jieba/蒸馏计算（原文保留）；`Cleaner` 在输出进入计算层前执行过滤（如提取 JSON 的 `content` 字段）。两者均不修改原文，只影响计算层输入。`RegisterTool` 拒绝同名注册，推断工具所属插件名，并维护工具到插件的映射表 `toolPlugins`。`RegisterStage` 将处理器追加至对应阶段的处理器列表。触发阶段执行时（`RunStage`），**所有已注册处理器通过 goroutine 并行执行**，共享同一 `*StageContext` 实例（通过 `sync.RWMutex` 保护并发访问）。单个处理器的 panic 被独立恢复，不影响其他处理器。短路语义通过检查 `ctx.Response != nil` 实现——任一阶段处理器可设置此值提前终止当前链路。工具执行 `ExecuteTool` 内置 panic 恢复与栈追踪记录。`UnregisterPluginTools` 在插件热重载时移除对应工具集。 -**上下文窗口（RelevanceContext）** 维护一个按时间排序的事件列表。`Append` 在录入前经 `textForVector` 聚合工具输出：按 `NoMemory` 跳过、`Cleaner` 过滤（插件为各自工具注册的清洗函数），最后经 `CleanText` 做基本空白规范化，再通过三分支向量策略（agent 事件用 Response，用户事件用 Input，cold_storage 用 Input+Response）计算嵌入向量。`Prune` 在事件数超过 `topK` 时触发，**无条件保护最近 10 条事件不被裁剪**（recency bias），对剩余候选事件计算与当前输入的 CosineSimilarity，按评分降序保留 `topK - 10` 条（下限为 0），之后按时间戳重排序。裁剪出的事件通过 `docStore.ContextToDoc` **迁移**至 Document 层：事件连同它持有的块一起下沉，保留原始时间戳与完整正文。持久化采用 5 秒防抖写入磁盘 JSON 文件。 +**上下文窗口（RelevanceContext）** 维护一个按时间排序的事件列表。`Append` 在录入前经 `textForVector` 聚合工具输出：按 `NoMemory` 跳过、`Cleaner` 过滤（插件为各自工具注册的清洗函数），最后经 `CleanText` 做基本空白规范化，再通过三分支向量策略（agent 事件用 Response，用户事件用 Input，cold_storage 用 Input+Response）计算嵌入向量。`Prune` 在事件数超过 `topK` 时触发，**无条件保护最近 10 条事件不被裁剪**（recency bias），对剩余候选事件计算与当前输入的 CosineSimilarity，按评分降序保留 `topK - 10` 条（下限为 0），之后按时间戳重排序。裁剪出的事件通过 `coldcontextStore.ContextToColdContext` **迁移**至冷上下文层：事件连同它持有的块一起下沉，保留原始时间戳与完整正文。持久化采用 5 秒防抖写入磁盘 JSON 文件。 > ★ **哪些事件不该进 L2，由插件自己声明，内核不认插件名**。 > @@ -104,7 +104,7 @@ HologramAgent 的认知架构由三个核心子系统构成：事件循环（eve **Provider 调用采用有序降级策略**：`ProviderManager.OrderedProviders()` 返回按注册顺序排列的 provider 列表。`process()` 内循环遍历该列表依次尝试 `Chat()` 调用。401/403 状态码将对应 provider 标记为永久不可用；其他错误类型同样标记不可用但容忍度更高。全部 provider 失败时返回错误返回调用方。若调用因 context 取消而中断且 Agent 仍在运行，则重试（仅当处理非 consolidation 路径时）。 -**记忆体系采用三级存储层级结构（Context → Document → Graph），按访问局部性与持久化需求进行数据分置**：Context 层为高速易失工作窗口，使用 StaticEmbedder（预训练词嵌入）进行语义相关性评分，以 TF-IDF 为回退策略；Document 层与 Context **共享同一 StaticEmbedder 向量空间**（agent 启动时将 embedder 注入 Document Store），使 Context 裁剪时的事件相关性评分与 Document 查询时的语义检索处于同一向量空间中，确保冷热数据之间的相似度可比——TF-IDF 仅在 embedder 未加载时作为兜底方案；Graph 层以 SQLite 为持久化载体，entities 表与 relations 表分别存储节点与有向边，支持 BFS 遍历召回。三级之间定义数据迁移策略：低分事件从 Context 下沉至 Document（以归档时相同的 embedder 向量化写入），冷文档经 72 小时未访问阈值判定后通过 `docToTriples` 蒸馏为三元组写入 Graph。Indexer 通过双路召回（实体向量相似度搜索 + jieba 关键词提取）构建 Graph 查询种子，结合 `MarkRecalled` 机制避免已被工具调用取回的实体重复注入系统提示。 +**记忆体系采用三级存储层级结构（Context → Cold Context → Graph），按访问局部性与持久化需求进行数据分置**：Context 层为高速易失工作窗口，使用 StaticEmbedder（预训练词嵌入）进行语义相关性评分，以 TF-IDF 为回退策略；冷上下文层与 Context **共享同一 StaticEmbedder 向量空间**（agent 启动时将 embedder 注入 coldcontext Store），使 Context 裁剪时的事件相关性评分与冷上下文查询时的语义检索处于同一向量空间中，确保冷热数据之间的相似度可比——TF-IDF 仅在 embedder 未加载时作为兜底方案；Graph 层以 SQLite 为持久化载体，entities 表与 relations 表分别存储节点与有向边，支持 BFS 遍历召回。三级之间定义数据迁移策略：低分事件从 Context 下沉至冷上下文（以归档时相同的 embedder 向量化写入），冷上下文文档经 72 小时未访问阈值判定后通过 `coldDocToTriples` 蒸馏为三元组写入 Graph。Indexer 通过双路召回（实体向量相似度搜索 + jieba 关键词提取）构建 Graph 查询种子，结合 `MarkRecalled` 机制避免已被工具调用取回的实体重复注入系统提示。 **核心域与应用域的职责域分离**是一项系统级架构决策：内核的责任边界限定在 LLM 编排、记忆管理与知识检索三个维度内，不直接承载任何 IO 操作；所有外部交互通过插件域接入。该分离将核心域的复杂度控制在可验证范围内，同时赋予插件域独立的演化自由度——后者可独立开发、独立发布、热加载，且不对核心域的稳定性构成直接影响。 @@ -120,7 +120,7 @@ eventLoop() → processTextInput() │ ├── on_input stage 插件可拦截/改写/短路 ├── Context.Append 记录到上下文窗口 - ├── Context.Prune 低相关性事件归档到 Document + ├── Context.Prune 低相关性事件归档到冷上下文 ├── buildMemoryContext() Indexer 召回 → GraphDB BFS 遍历 │ ├── pre_action stage 插件可注入 system 消息 @@ -185,7 +185,7 @@ eventLoop() → processTextInput() | 层 | 承载单元 | 与模型的关系 | 容量 | 流动性 | | --- | --- | --- | --- | --- | | **L1 Context** | 事件（内联正文 + 块） | **直接投喂**（进 system prompt 时序） | 最近 n 条 / 上限 m 条 | 超限时按相似度迁出 | -| **L2 Document** | 事件（内联正文 + 块） | **不直接投喂**（需召回才进 L1） | 无上限 | 冷到阈值后蒸馏进 L3 | +| **L2 Cold Context** | 事件（内联正文 + 块） | **不直接投喂**（需召回才进 L1） | 无上限 | 冷到阈值后蒸馏进 L3 | | **L3 Stable** | 三元组（节点 + 关系边），节点即记忆块 | 不投喂（经召回进入 L1） | 持久 | 不流出 | ★ **L1 与 L2 结构完全同构**，唯一区别是「是否被直接投喂给模型」。 @@ -208,12 +208,12 @@ L2 不是 L1 的摘要层，也不是另一种数据形态——它就是同样 向量层级：统一多模态空间（主，带 fingerprint）→ StaticEmbedder 词嵌入 → TF-IDF（回退） Prune: DenseCosine（仅同指纹才比较）→ 退化 StaticEmbedder CosineSimilarity；保留 topK + 最近10条 ├── 保留 → timeline → 按时间排序 → system prompt - └── 低分 → Document 层归档（迁移：事件连同块一起下沉，时间戳与正文不变） + └── 低分 → 冷上下文层归档（迁移：事件连同块一起下沉，时间戳与正文不变） Save: 5s debounce 写盘 ↓ Prune 归档（迁移） ↑ 召回（迁回） -② Document (文件记忆 — 不直接投喂，结构与 L1 同构) +② 冷上下文 Cold Context (文件记忆 — 不直接投喂，结构与 L1 同构) DocStore — 每篇文档 = 检索元数据 + 事件列表（内联正文 + 块） 写入: Prune归档 / coldcontext_commit / stablemem 快照(syncGraphToDocs) 读取: @@ -221,7 +221,7 @@ L2 不是 L1 的摘要层，也不是另一种数据形态——它就是同样 └── LLM主动: coldcontext_query → Consume(取走) → 逐条按各自原始时间戳把事件插回 L1 时序 → 块随事件迁回（同一块 ID，向量不重算），L2 不再持有 - 冷化: FindColdDocs(72h, ≤2次访问) → docToTriples → Graph + 冷化: FindColdContexts(72h, ≤2次访问) → coldDocToTriples → Graph ↓ 冷文档蒸馏 ↑ 自动召回 @@ -238,8 +238,8 @@ L2 不是 L1 的摘要层，也不是另一种数据形态——它就是同样 Social: person_query / set_trait / relate (包装 GraphDB) ④ 四个独立心跳循环（各自独立的 ticker 和配置间隔） - distillLoop (distillInterval, 默认30m): 上下文裁剪 — Context.Prune → Document - archiveLoop (archiveInterval, 默认60m): 冷文档归档 — docToTriples → GraphDB + distillLoop (distillInterval, 默认30m): 上下文裁剪 — Context.Prune → 冷上下文 + archiveLoop (archiveInterval, 默认60m): 冷上下文归档 — coldDocToTriples → GraphDB mergeLoop (mergeInterval, 默认120m): 实体合并检测 — 相似度 → LLM 裁决 reviewLoop (reviewInterval, 默认120m): 关系复审 — SentenceRef 回溯 → LLM 修正 @@ -297,7 +297,7 @@ L2 不是 L1 的摘要层，也不是另一种数据形态——它就是同样 - 保护最近 10 条记录免于淘汰，超出部分按相关性排序**迁移**到文档记忆 - 迁移是事件**连同它持有的块**一起下沉，时间戳与正文不变；源事件清空块（同一块不同时留两层） -### Document 层 +### 冷上下文层 `internal/memory/document/document.go` — `Store` @@ -309,7 +309,7 @@ L2 不是 L1 的摘要层，也不是另一种数据形态——它就是同样 - `Doc.Content` 只是**渲染视图**（供摘要/人工阅读/SDK 兼容），不是原文载体 - 自动预填充：与 Context 共享的向量空间做相似度检索 → 摘要进 system prompt（**只读**，不动层） - 主动召回：`coldcontext_query` 取走后，**逐条按各自原始时间戳**把事件插回 L1 时序，块随事件迁回 -- 冷化：`FindColdDocs(72h, ≤2次访问)` → `docToTriples` → Graph +- 冷化：`FindColdContexts(72h, ≤2次访问)` → `coldDocToTriples` → Graph ## 安全防火墙与设备数据面（内核模块） @@ -428,9 +428,9 @@ Core ◀─入向审查── 外部 ```text 四个独立心跳循环（各自可配置间隔）: ├── distillLoop (distillInterval, 默认30m) - │ └── distillContext() — Context.Prune → Document + │ └── distillContext() — Context.Prune → 冷上下文 ├── archiveLoop (archiveInterval, 默认60m) - │ └── archiveColdDocs() — 冷文档 → docToTriples → GraphDB + │ └── archiveColdDocs() — 冷上下文 → coldDocToTriples → GraphDB ├── mergeLoop (mergeInterval, 默认120m) │ └── detectEntityMerge() — 实体相似度检测 → LLM 裁决 └── reviewLoop (reviewInterval, 默认120m) diff --git a/assets/docs/zh/OVERVIEW.md b/assets/docs/zh/OVERVIEW.md index 5df0572d..4f024c3c 100644 --- a/assets/docs/zh/OVERVIEW.md +++ b/assets/docs/zh/OVERVIEW.md @@ -26,7 +26,7 @@ CLI 需要终端面板时，**订阅事件总线**（`EventTerminalOutput` + `Ev **三层记忆架构** — 三层不是三种数据结构，而是**同一批记忆块的三层持有关系**： - **Context 层（L1，直接投喂）**：内存中预训练词嵌入评分的事件窗口（StaticEmbedder 词向量 → CosineSimilarity，TF-IDF 回退），实时维护最近上下文，低相关性事件自动下沉到下一层 -- **Document 层（L2，不直接投喂）**：与 L1 **结构同构**——同样的事件 + 同样的块，只是不直接进模型视野；支持显式提交和隐式归档，冷数据蒸馏到稳定记忆（stablemem） +- **冷上下文层 Cold Context（L2，不直接投喂）**：与 L1 **结构同构**——同样的事件 + 同样的块，只是不直接进模型视野；支持显式提交和隐式归档，冷数据蒸馏到稳定记忆（stablemem） - **Stablemem 层（L3，持久）**：SQLite 图数据库，**顶层仍是三元组**（节点 + 关系边），但节点从 `entities` 表升级为 `memory_blocks`；BFS 遍历召回，蒸馏管道从对话中提取三元组 ★ **L1 与 L2 唯一区别是「是否被直接投喂给模型」**：L2 不是 L1 的摘要层，也不是 @@ -43,7 +43,7 @@ CLI 需要终端面板时，**订阅事件总线**（`EventTerminalOutput` + `Ev - **内容寻址存储（CAS）**：digest 寻址，元数据在 SQLite、blob 在磁盘（两级分桶），相同字节只存一份， 每次 `Get` 重校 digest（磁盘损坏静默返回脏数据比报错更危险） - **媒体块直接参与向量检索**：块携带**自己的多模态向量与指纹**，正文里不再有任何媒体标记 -- **无独立 GC、无引用计数、无 keep-set**：记忆块遵循单层不变量——Context → Document → Graph +- **无独立 GC、无引用计数、无 keep-set**：记忆块遵循单层不变量——Context → Cold Context → Graph 是块的**迁移**，不是复制、也不靠引用保活；**删除块即删内容** - **v1.1.1 起贯通插件边界**：插件可通过 `InsertWithMedia` / `InjectInputMedia` 读写媒体， 模型可用 `stablemem_commit` / `coldcontext_commit` 的 `media_digests` 参数关联媒体 @@ -64,7 +64,7 @@ CLI 需要终端面板时，**订阅事件总线**（`EventTerminalOutput` + `Ev **记忆系统** (`internal/memory/`)： - **GraphDB** (`graph.go`) — SQLite，entities + relations 表，BFS 遍历 -- **Document Store** (`document/document.go`) — 临时记忆，JSON 文件 + TF-IDF 向量索引，消费即删 +- **Cold Context Store** (`internal/memory/coldcontext/`) — 冷上下文，JSON 文件 + 稠密/TF-IDF 双路检索，两阶段取回 - **Text Memory** (`text/text.go`) — 原始对话日志，JSONL 文件轮转 - **Social Store** (`social/social.go`) — 人格特质 + 关系网，包装 GraphDB - **Memory Indexer** (`indexer.go`) — 自动将 GraphDB 实体向量化，用户输入时召回注入 system prompt diff --git a/internal/agent/core/recallcoldcontext_test.go b/internal/agent/core/recallcoldcontext_test.go index 9aee6f66..ef7baef2 100644 --- a/internal/agent/core/recallcoldcontext_test.go +++ b/internal/agent/core/recallcoldcontext_test.go @@ -2,7 +2,7 @@ package core // L2 → L1 召回协议的判据。 // -// 依据（assets/docs/zh/ARCHITECTURE.md「记忆流转」② Document 层的读取分支）： +// 依据（assets/docs/zh/ARCHITECTURE.md「记忆流转」② 冷上下文 Cold Context 的读取分支）： // // coldcontext_query → Consume(读取并删除) // → 逐条 context.Append{Timestamp: 原始时间戳, Source: 原来源} diff --git a/internal/agent/core/toolcall.go b/internal/agent/core/toolcall.go index 2656c77e..40e19517 100644 --- a/internal/agent/core/toolcall.go +++ b/internal/agent/core/toolcall.go @@ -1159,7 +1159,7 @@ func (a *Agent) executeDocTool(tc agentAPI.ToolCall) string { for i, d := range docs { // 召回 = 把事件（连同它持有的块）迁回 L1，不是造一条新事件。 // - // ARCHITECTURE「记忆流转」② Document 层： + // ARCHITECTURE「记忆流转」② 冷上下文 Cold Context： // coldcontext_query → Consume(读取并删除) // → 逐条 context.Append{Timestamp: 原始时间戳, Source: 原来源} n := a.restoreDocEvents(d, plan.Evict[d.ID]) @@ -1414,7 +1414,7 @@ func (a *Agent) currentFingerprint() string { // // ## 依据 // -// ARCHITECTURE「记忆流转」② Document 层的读取分支： +// ARCHITECTURE「记忆流转」② 冷上下文 Cold Context 的读取分支： // // coldcontext_query → Consume(读取并删除) // → 逐条 context.Append{Timestamp: 原始时间戳, Source: 原来源} diff --git a/internal/memory/coldcontext/coldcontext_test.go b/internal/memory/coldcontext/coldcontext_test.go index c354a008..f7bc74e4 100644 --- a/internal/memory/coldcontext/coldcontext_test.go +++ b/internal/memory/coldcontext/coldcontext_test.go @@ -67,7 +67,7 @@ func TestQuery(t *testing.T) { } } -func TestContextToDoc(t *testing.T) { +func TestContextToColdContext(t *testing.T) { dir, err := os.MkdirTemp("", "doc_ctx_*") if err != nil { t.Fatal(err) @@ -98,7 +98,7 @@ func TestContextToDoc(t *testing.T) { } } -func TestFindColdDocs(t *testing.T) { +func TestFindColdContexts(t *testing.T) { dir, err := os.MkdirTemp("", "doc_cold_*") if err != nil { t.Fatal(err) @@ -361,7 +361,7 @@ func TestRemoveNonexistent(t *testing.T) { } } -func TestContextToDocContentPreservesRawToolOutput(t *testing.T) { +func TestContextToColdContextContentPreservesRawToolOutput(t *testing.T) { dir, err := os.MkdirTemp("", "doc_toolclean_*") if err != nil { t.Fatal(err)