diff --git a/assets/docs/en/ARCHITECTURE.md b/assets/docs/en/ARCHITECTURE.md index f8feede3..fdb9b90e 100644 --- a/assets/docs/en/ARCHITECTURE.md +++ b/assets/docs/en/ARCHITECTURE.md @@ -283,9 +283,7 @@ Agent ▼ Provider Interface (Name / Chat / ChatStream) │ - ├── OpenAIProvider — Standard OpenAI API - ├── OllamaProvider — Local Ollama - └── LuaAdaptedProvider (primary) + └── LuaAdaptedProvider (the only implementation) ├── Serialize CompletionRequest → JSON ├── adapter.transform_request() → API format ├── HTTP request + adapter.headers @@ -562,7 +560,7 @@ budget, actual injection was still a few hundred characters. - SQLite storage, `config` table + `config_` independent tables - Namespaces: `core.*` / `plugin..*` -- `RegisterDefault` inserts ~80 default keys (seeds for 8 LLM sources) +- `RegisterDefault` inserts ~80 default keys (including 1 LLM source seed; further sources are added at deploy time) - WebUI settings page `/api/v1/settings` for read/write @@ -580,7 +578,7 @@ internal/ │ └── personal.go — Persona loading ├── plugin/ │ ├── registry.go — Registry + lifecycle -│ ├── dynamic.go — .so dynamic loader +│ ├── dynamic.go — Load-channel dispatch (proc / lua / skill); .so only remains to error loudly │ └── manifest.go — plugin.json metadata ├── plugins/ — Built-in plugin implementations │ ├── all.go — Blank imports @@ -594,6 +592,7 @@ internal/ │ ├── agentcli/ — PTY terminal │ ├── healthcheck/ — Health check │ ├── pluginmgr/ — Plugin manager +│ ├── skillmgr/ — Skill plugin management │ └── cfgmgr/ — Config manager ├── sdk/ — PluginSDK definitions │ ├── plugin.go — Plugin interface + PluginSDK @@ -611,13 +610,11 @@ internal/ ├── knowledge/knowledge.go — Knowledge base ├── lua/ │ ├── vm.go — Lua VM (json/log/http) -│ └── adapters/ — 8 LLM adapter scripts +│ └── adapters/ — 10 LLM protocol adapter scripts ├── config/registry.go — SQLite config center ├── events/bus.go — Event bus ├── tracker/ — OverlayFS change tracking ├── supervisor/ — Daemon management -├── skill/ — Skill plugin management -│ └── manager.go — Skill loading/matching └── meta/ — Meta information └── meta.go — Agent metadata ``` diff --git a/assets/docs/en/OVERVIEW.md b/assets/docs/en/OVERVIEW.md index c71263bc..8a891a7a 100644 --- a/assets/docs/en/OVERVIEW.md +++ b/assets/docs/en/OVERVIEW.md @@ -25,18 +25,22 @@ The significance lies in clear responsibility boundaries: the kernel focuses on Three progressive layers — context, cold archive, long-term graph memory — form an information decay and consolidation pipeline from short-term to persistent storage. -**Media Memory (since v1.1.0)** — Images and audio are not attachments; they are a kind of node in all three layers: -- **Content-addressed store (CAS)**: addressed by digest, metadata in SQLite and blobs on disk, identical bytes - stored once. Every `Get` re-verifies the digest (silently returning corrupt data is worse than an error). -- **Reference-counted GC**: `owner_kind/owner_id/digest` is the primary key; context events, documents and graph - sentences each hold their own references. **Referenced items are never deleted** — only unowned content past - `minAge` is reclaimed. -- **The description text is the durable semantic memory**: what the vision model produced is written into - plain-text memory as a `[ ] ` marker and participates in vector retrieval and - distillation; the blob is only a cache that capacity GC may evict. Months later "that purple-blue-red - three-band chart" is still findable — via the description, not the bytes. +**Media Memory (first-class memory blocks since v1.2.0)** — Images and audio are not attachments; they are first-class nodes of memory: +- **Content-addressed store (CAS)**: addressed by digest, metadata in SQLite and blobs on disk (two-level + bucketing), identical bytes stored once. Every `Get` re-verifies the digest (silently returning corrupt + data is worse than an error). +- **Blocks carry their own multimodal vector and fingerprint**, and are retrieved directly — there is no + media marker in the body text any more. +- **No separate GC, no reference counting, no keep-set**: blocks follow the single-layer invariant — + Context → Document → Graph is a **migration** of a block, not a copy, and nothing is kept alive by + references; **deleting a block deletes its content.** - **Reaches the plugin boundary since v1.1.1**: plugins read and write media through `InsertWithMedia` / `InjectInputMedia`; the model attaches media via the `media_digests` argument of `memory_commit` / `doc_commit`. +- **The old mechanism was removed entirely**: the **descriptive index** — writing the vision model's + description into the body text as a `[ ] ` marker and retrieving by it — + together with `media_refs` reference counting, was deleted in v1.2.0. A description is second-hand + information produced by a model; retrieving "someone else's rendering of the image" is worse than + retrieving the image's own vector. ## What It Actually Does @@ -47,7 +51,7 @@ Code is in the project root, implemented in Go. - `process.go` / `stages.go` — Processing pipeline: memory recall → persona injection → LLM call → tool execution → output delivery, 7 stage hooks - `toolcall.go` — Tool scheduling and execution - `context.go` — Context management (pretrained word embedding scoring StaticEmbedder → CosineSimilarity, TF-IDF fallback), automatic pruning of low-relevance events -- LLM calls abstracted through Provider interface, supports 8 LLM sources with automatic fallback +- LLM calls abstracted through the Provider interface, with multi-source automatic fallback **Memory System** (`internal/memory/`): - **GraphDB** (`graph.go`) — SQLite, entities + relations tables, BFS traversal @@ -72,9 +76,11 @@ Code is in the project root, implemented in Go. **LLM Provider** (`internal/agent/api/provider.go`): - Provider interface: Name / Chat / ChatStream -- Three implementations: OpenAIProvider (standard OpenAI API), OllamaProvider (local), LuaAdaptedProvider (Lua adapter) -- LuaAdapter located at `internal/lua/adapters/`, each LLM source has a corresponding `.lua` script -- 8 built-in adapters: deepseek / openai / anthropic / gemini / mistral / groq / github / ollama +- Single implementation, `LuaAdaptedProvider`: every source goes through a Lua adapter script that + transforms requests/responses; adapter scripts live in `internal/lua/adapters/`, one `.lua` per protocol +- 10 built-in adapters: deepseek / openai / anthropic / gemini / mistral / groq / github / kimicode / ollama / server +- Sources (`core.llm.sources..*`) are configured at deploy time — the count is deployment-specific + and is not tied to the number of built-in adapters **WebUI** (`internal/plugins/webui/`): - Embedded SPA dashboard (`dashboard.html` packaged via `//go:embed`) diff --git a/assets/docs/zh/ARCHITECTURE.md b/assets/docs/zh/ARCHITECTURE.md index 258bacbb..caa02348 100644 --- a/assets/docs/zh/ARCHITECTURE.md +++ b/assets/docs/zh/ARCHITECTURE.md @@ -539,7 +539,7 @@ targetUsage = min(maxCtx × 0.8, 600000) // 工作面：封顶 600K - SQLite 存储，`config` 表 + `config_` 独立表 - 命名空间：`core.*` / `plugin..*` -- `RegisterDefault` 插入 ~80 个默认键（8 个 LLM 源的 seeds） +- `RegisterDefault` 插入 ~80 个默认键（含 1 个 LLM 源 seed，其余源由部署时添加） - WebUI 设置页 `/api/v1/settings` 读写 @@ -557,7 +557,7 @@ internal/ │ └── personal.go — 人格加载 ├── plugin/ │ ├── registry.go — 注册表 + 生命周期 -│ ├── dynamic.go — .so 动态加载器 +│ ├── dynamic.go — 加载通道分派（proc / lua / skill）；.so 仅剩给明确报错 │ └── manifest.go — plugin.json 元数据 ├── plugins/ — 内置插件实现 │ ├── all.go — 空白导入 @@ -571,6 +571,7 @@ internal/ │ ├── agentcli/ — PTY 终端 │ ├── healthcheck/ — 健康检查 │ ├── pluginmgr/ — 插件管理器 +│ ├── skillmgr/ — Skill 插件管理 │ └── cfgmgr/ — 配置管理 ├── sdk/ — PluginSDK 定义 │ ├── plugin.go — Plugin 接口 + PluginSDK @@ -588,13 +589,11 @@ internal/ ├── knowledge/knowledge.go — 知识库 ├── lua/ │ ├── vm.go — Lua VM (json/log/http) -│ └── adapters/ — 8 个 LLM 适配器脚本 +│ └── adapters/ — 10 个 LLM 协议适配器脚本 ├── config/registry.go — SQLite 配置中心 ├── events/bus.go — 事件总线 ├── tracker/ — OverlayFS 变更追踪 ├── supervisor/ — 守护进程管理 -├── skill/ — Skill 插件管理 -│ └── manager.go — Skill 加载/匹配 └── meta/ — 元信息 └── meta.go — Agent 元数据 ``` diff --git a/assets/docs/zh/OVERVIEW.md b/assets/docs/zh/OVERVIEW.md index bfa1a9c0..48006286 100644 --- a/assets/docs/zh/OVERVIEW.md +++ b/assets/docs/zh/OVERVIEW.md @@ -25,16 +25,17 @@ HomeAgent 是一个持续运行的个人智能 Agent 框架。 三层递进：上下文 → 冷归档 → 长期图记忆，构成从短期到持久的信息衰减与整合管道。 -**媒体记忆（v1.1.0 起）** — 图片/音频不是附属物，而是三层里的一类节点： -- **内容寻址存储（CAS）**：digest 寻址，元数据在 SQLite、blob 在磁盘，相同字节只存一份， +**媒体记忆（v1.2.0 起：一等记忆块）** — 图片/音频不是附属物，而是记忆的一等节点： +- **内容寻址存储（CAS）**：digest 寻址，元数据在 SQLite、blob 在磁盘（两级分桶），相同字节只存一份， 每次 `Get` 重校 digest（磁盘损坏静默返回脏数据比报错更危险） -- **引用计数 GC**：`owner_kind/owner_id/digest` 三元组为主键，上下文事件/文档/图谱句子各自持引用； - **有引用者绝不删除**，仅回收无主且超过 `minAge` 的内容 -- **描述文本才是持久语义记忆**：视觉模型生成的描述以 - `[ <短digest>] <描述>` 标记形式写进纯文本记忆，参与向量检索与蒸馏； - blob 只是可被容量 GC 淘汰的缓存。几个月后“那张紫蓝红三色带图”仍可检索，靠的是描述而不是字节 +- **媒体块直接参与向量检索**：块携带**自己的多模态向量与指纹**，正文里不再有任何媒体标记 +- **无独立 GC、无引用计数、无 keep-set**：记忆块遵循单层不变量——Context → Document → Graph + 是块的**迁移**，不是复制、也不靠引用保活；**删除块即删内容** - **v1.1.1 起贯通插件边界**：插件可通过 `InsertWithMedia` / `InjectInputMedia` 读写媒体， 模型可用 `memory_commit` / `doc_commit` 的 `media_digests` 参数关联媒体 +- **旧机制已整体拆除**：此前把视觉模型生成的描述写进正文、以 `[ <短digest>] <描述>` 标记 + 参与检索的**描述式索引**，以及 `media_refs` 引用计数，均已在 v1.2.0 删除——描述是模型生成的 + 二手信息，检索「别人转述的图片」不如检索图片自己的向量 ## 它实际做了什么 @@ -45,7 +46,7 @@ HomeAgent 是一个持续运行的个人智能 Agent 框架。 - `process.go` / `stages.go` — 处理管道：记忆召回 → 人格注入 → LLM 调用 → 工具执行 → 输出发送，7 阶段钩子 - `toolcall.go` — 工具调度与执行 - `context.go` — 上下文管理（预训练词嵌入评分 StaticEmbedder → CosineSimilarity，TF-IDF 回退），自动剪枝低相关性事件 -- LLM 调用通过 Provider 接口抽象，支持 8 个 LLM 源自动降级 +- LLM 调用通过 Provider 接口抽象，多源自动降级 **记忆系统** (`internal/memory/`)： - **GraphDB** (`graph.go`) — SQLite，entities + relations 表，BFS 遍历 @@ -69,9 +70,10 @@ HomeAgent 是一个持续运行的个人智能 Agent 框架。 **LLM Provider** (`internal/agent/api/provider.go`)： - Provider 接口：Name / Chat / ChatStream -- 三种实现：OpenAIProvider（标准 OpenAI API）、OllamaProvider（本地）、LuaAdaptedProvider（Lua 胶水适配） -- LuaAdapter 位于 `internal/lua/adapters/`，每个 LLM 源对应一个 `.lua` 脚本 -- 内置 8 个适配器：deepseek / openai / anthropic / gemini / mistral / groq / github / ollama +- 唯一实现 `LuaAdaptedProvider`：所有源都经 Lua 适配脚本做请求/响应变换； + 适配器脚本位于 `internal/lua/adapters/`，每类协议一个 `.lua` 脚本 +- 内置 10 个适配器：deepseek / openai / anthropic / gemini / mistral / groq / github / kimicode / ollama / server +- 源（`core.llm.sources..*`）由部署时配置，数量不固定，不随内置适配器数绑定 **WebUI** (`internal/plugins/webui/`)： - 嵌入式 SPA 仪表盘（`dashboard.html` 通过 `//go:embed` 打包）