mirror of
https://gitcode.com/JianFeeeee/HomeAgent.git
synced 2026-09-21 09:28:14 +00:00
五处文档此前停在 v1.0.0,而 v1.1.0/v1.1.1 都已发布并在现网运行。 本轮补齐三层记忆的媒体架构、模型工具的媒体参数、通信面 method 数, 以及冻结解除后的替代约束。 ## 中英双语 OVERVIEW.md 三层记忆段只写了 Context/Document/Graph 三层,而 v1.1.0 起图片/音频是 三层里的一类节点。补上媒体记忆的架构概要:CAS + 引用计数 GC + 标记格式 (描述文本才是持久语义记忆,blob 是可淘汰的缓存)+ 插件边界贯通。 ## 中英双语 ARCHITECTURE.md - 「记忆工具」表补 `memory_commit`/`doc_commit` 的 `media_digests` 与 `sentence_text`(后者从未暴露给模型,而它是媒体绑定链的必经环节) - 「三层记忆」段新增媒体记忆子节:CAS 设计表(寻址/完整性/写入原子性/引用/GC)、 标记格式、可选性(`enabled=false` 时整条链路静默退化) - 「三个通信面」从 51 改为 55 个 method,新增四个 media method 的说明 - 「SDK 四通道」代码示例补媒体注入三方法 + 与 SetToolBlocks 的区别 ## plugin-interface-matrix.md - 状态从「完成 v2」改「完成 v3」,v3 记录 v1.1.1 的接口扩展 - §二 A3 DocMemoryAPI 补 InsertWithMedia - §二 A4 补三个媒体注入方法 + 为何不能搭 SetToolBlocks 的车 - §二 A5 补 MediaAttachment 类型 + Triple/Doc/TextEvent 的扩展字段 - §六「新获得的能力」补 SetToolBlocks 已落地、媒体入记忆、插件主动发起 带媒体的对话 - §七 冻结检查点补第 5 条(冻结已解除,取代它的是 §九) - 新增 §九「v1.1.x 的接口扩展规则」:冻结解除后的三条硬约束 (只增不减签名不改 / 新增方法方向 / 模板接线六处失败链)+ 验证方式 (存量插件 17/17、旧产物 4/4 建链、模板断言、压测 -race) ## 为何分立两笔 commit 前一笔(SDK 仓 README + 内核 README)改的是给**插件开发者**看的文本, 本笔改的是给**架构师与维护者**看的技术文档。受众与改动层次不同, 放在同一个 commit 会让追溯时看不出"文档在哪一层跟上了代码"。
529 lines
32 KiB
Markdown
529 lines
32 KiB
Markdown
**中文** | [English](../en/ARCHITECTURE.md)
|
|
|
|
# HomeAgent Architecture
|
|
|
|
## Architectural Principles
|
|
|
|
HomeAgent's cognitive architecture consists of three subsystems: the event loop (eventLoop), the context window (RelevanceContext), and the stage pipeline (StageHost). Together they form the orchestration framework. Within this framework, the LLM serves as a scheduled reasoning unit; cognitive continuity is maintained by the event loop, context window, and stage pipeline.
|
|
|
|
**The event loop (eventLoop)** is a three-way select: `a.io.InputChan()` receives external user input and dispatches to `processTextInput` / `processMediaInput`; `a.selfInputCh` receives internal system tasks (memory merges, distillation callbacks) routed through `processConsolidation` under the `_consolidation_` output channel; `a.ctx.Done()` accepts shutdown signals. A concurrently running `interceptLoop` goroutine independently reads `a.io.InputInterruptChan()` — on receiving a high-priority interrupt, it cancels the in-flight LLM HTTP request (`a.cancelLLM()`), then writes the event to `a.interceptCh`. This channel is drained non-blockingly by `drainInterrupts()` before each LLM call in `process()`, injecting interrupts as `[打断消息]` formatted entries into message history. The three interrupt delivery paths carry distinct semantics: `cancelLLM` terminates the current HTTP request, `interceptCh` injects text before the next LLM turn, and `InjectInput` triggers a new processing cycle when the event loop is idle.
|
|
|
|
**The stage pipeline (StageHost)** manages two registration categories: tool definitions (ToolDef) and stage handlers (StageHandler). ToolDef includes two optional memory control fields: `NoMemory bool` — when true, the tool's output is excluded from vectorization/jieba/distillation (original text preserved); and `Cleaner func(string) string` — a filter applied before the output enters the computation layer (e.g., extracting a `content` field from JSON). Neither modifies the original output; both only affect the computation layer input. `RegisterTool` rejects duplicate names, infers the owning plugin name from the tool name prefix, and maintains a `toolPlugins` mapping. `RegisterStage` appends handlers to the corresponding stage list. On stage execution (`RunStage`), **all registered handlers execute in parallel via goroutines**, sharing a single `*StageContext` protected by `sync.RWMutex`. Individual handler panics are recovered independently without affecting other handlers. Short-circuit semantics are implemented by checking `ctx.Response != nil` — any stage handler can set this value to terminate the pipeline early. `ExecuteTool` includes built-in panic recovery with stack-trace recording. `UnregisterPluginTools` removes a plugin's tool set during hot-reload.
|
|
|
|
**The context window (RelevanceContext)** maintains a chronologically ordered event list. `Append` aggregates tool outputs through `textForVector` before computing the embedding vector: `NoMemory` skips, `Cleaner` filters (per-tool cleaners registered by plugins — e.g. the QQ plugin strips its own tool-call templates), and `CleanText` finalizes with basic whitespace normalization. A three-branch strategy selects the text source (agent events use Response, user events use Input, cold_storage uses Input+Response). `Prune` triggers when the event count exceeds `topK`: it **unconditionally protects the last 10 events from eviction** (recency bias), scores remaining candidates against the current input via CosineSimilarity, keeps `topK - 10` highest-scoring entries (floor at 0), then re-sorts chronologically. Pruned events from sources other than `agentcli` and `terminal` are archived to the Document layer via `docStore.ContextToDoc`, retaining original timestamps. Persistence uses 5-second debounced writes to a JSON file.
|
|
|
|
**Tool definitions are aggregated from five sources**: IOManager-registered plugin tools; StageHost-registered SDK tools; Indexer-provided memory index tools; conditionally added built-in tools (depending on non-nil state of memory/knowledge/docStore/social/pluginReg/providerManager modules — including memory operations, knowledge retrieval, document queries, social networking, plugin reloading, child-agent spawning, per-output-channel send tools, and LLM source switching); and media processing tools added based on `pendingMedia` state. `buildToolDefs()` re-aggregates all sources on each process cycle.
|
|
|
|
**Provider invocation follows an ordered fallback strategy**: `ProviderManager.OrderedProviders()` returns the provider list in registration order. The `process()` inner loop iterates this list attempting `Chat()` on each. HTTP 401/403 responses mark the provider as permanently unavailable; other error types also mark unavailability but with higher tolerance. If all providers fail, an error is returned to the caller. If a call is interrupted by context cancellation while the agent is still running, it is retried (only on non-consolidation paths).
|
|
|
|
**The memory system adopts a three-tier storage hierarchy (Context → Document → Graph), tiering data by access locality and persistence requirements**: the Context layer is a fast-volatile working window using StaticEmbedder (pretrained word embeddings with TF-IDF fallback) for semantic relevance scoring; the Document layer **shares the same StaticEmbedder vector space with Context** (the embedder is injected into the Document Store at agent startup via `docStore.SetVectorizer(embedder)`), ensuring that relevance scores during Context pruning and semantic retrieval during Document queries operate within the same vector space — TF-IDF serves only as a fallback when the embedder is unavailable; the Graph layer uses SQLite as its persistence substrate with an entities table (nodes) and a relations table (directed edges), supporting BFS traversal recall. Data migration policies govern movement across tiers: low-scoring events sink from Context to Document (vectorized using the same embedder at archival time); cold documents, after a 72-hour no-access threshold, are distilled into triples via `docToTriples` and committed to Graph. The Indexer uses dual retrieval (entity vector similarity search + jieba keyword extraction) to construct Graph query seeds, and the `MarkRecalled` mechanism prevents entities already fetched via tool calls from being re-injected into the system prompt.
|
|
|
|
**The separation of core domain and application domain** constrains the kernel's responsibilities to LLM orchestration, memory management, and knowledge retrieval — no direct IO operations; all external interaction is mediated through the plugin domain. This separation limits the kernel's complexity to a verifiable scope while granting the plugin domain independent evolution: plugins can be independently developed, independently released, hot-loaded, and do not directly affect the stability of the core domain.
|
|
|
|
## Message Processing Flow
|
|
|
|
### Full Pipeline
|
|
|
|
```
|
|
External input (via plugin InjectInput)
|
|
│
|
|
▼
|
|
eventLoop() → processTextInput()
|
|
│
|
|
├── on_input stage Plugins can intercept/rewrite/short-circuit
|
|
├── Context.Append Record to context window
|
|
├── Context.Prune Low-relevance events archived to Document
|
|
├── buildMemoryContext() Indexer recall → GraphDB BFS traversal
|
|
│
|
|
├── pre_action stage Plugins can inject system messages
|
|
│
|
|
├── [Tool Loop] process()
|
|
│ ├── buildSystemPrompt Persona + Memory + Knowledge + Context
|
|
│ ├── buildToolDefs Built-in tools + Plugin tools
|
|
│ ├── provider.Chat() LLM call
|
|
│ ├── post_action stage Plugins see LLM output + tool list
|
|
│ ├── Has tools?
|
|
│ │ ├── before_toolcall Plugins can reject/modify params
|
|
│ │ ├── executeToolCall Route to plugin/built-in
|
|
│ │ ├── after_toolcall Plugins can modify results
|
|
│ │ └── → back to post_action
|
|
│ └── No tools → exit loop
|
|
│
|
|
├── Context.Append(response)
|
|
├── before_output stage Plugins can modify final text
|
|
├── emitResponse() Send via output_send
|
|
└── after_output stage Read-only, cleanup
|
|
```
|
|
|
|
Code: `internal/agent/core/process.go` — `process()` is the main tool loop
|
|
|
|
### 7 Stage Hooks
|
|
|
|
| Stage | Trigger | Plugin Capabilities |
|
|
|-------|---------|---------------------|
|
|
| `on_input` | Message arrives at Agent, zero processing | Blacklist/rate-limit/short-circuit reply |
|
|
| `pre_action` | Context ready, before LLM call | Inject external data into context |
|
|
| `post_action` | LLM returns text + tool list | Sensitive word filter/forced redirect |
|
|
| `before_toolcall` | Before single tool execution | Audit/reject/modify params |
|
|
| `after_toolcall` | After single tool execution | Desensitize/sort results |
|
|
| `before_output` | Final text ready, before sending | Format adaptation |
|
|
| `after_output` | Already sent | Statistics/logging |
|
|
|
|
Code: `internal/agent/core/stages.go` — `StageHost` orchestration
|
|
|
|
### Loop Rules
|
|
|
|
`post_action → [before_toolcall → execute → after_toolcall] → post_action` forms the inner loop.
|
|
Exit conditions: LLM has no tool calls / all rejected / exceeded limit.
|
|
|
|
### Short-Circuit Rules
|
|
|
|
Setting `ctx.Response` at any stage jumps to `after_output`.
|
|
|
|
|
|
## Three-Layer Memory
|
|
|
|
### Memory Flow
|
|
|
|
```
|
|
① Context (Working Window)
|
|
RelevanceContext — In-memory events[] + JSON persistence
|
|
Append: Each input, CleanTemplateText → three-branch vector(textForVector)
|
|
agent→Response, user→Input, cold_storage→Input+Response
|
|
StaticEmbedder pretrained word embedding / TF-IDF fallback
|
|
Prune: StaticEmbedder CosineSimilarity, keep topK + last 10
|
|
├── Keep → timeline → chronologically sorted → system prompt
|
|
└── Low score → Document layer archive (original timestamp)
|
|
Save: 5s debounce write to disk
|
|
|
|
↓ Prune archive ↑ LLM active recall
|
|
|
|
② Document (File Memory)
|
|
DocStore — JSON files + shared StaticEmbedder vector space with Context (fallback: TF-IDF InvertedIndex)
|
|
Write: Prune archive / doc_commit / Graph snapshot (syncGraphToDocs)
|
|
Read:
|
|
├── Auto-inject: Query(input, top3) → similarity summary under same vector space → [Related Memory Docs] → system prompt (read-only)
|
|
└── LLM active: doc_query → Consume(read and delete)
|
|
→ context.Append{Timestamp: d.CreatedAt, Source: "cold_storage"} per doc
|
|
→ Docs written to context timeline with original timestamps, deleted from docStore
|
|
Cold: FindColdDocs(72h, ≤2 accesses) → docToTriples → Graph
|
|
|
|
↓ Cold doc distillation ↑ Auto recall
|
|
|
|
③ Graph (Graph Database)
|
|
SQLite — entities + relations tables
|
|
Write: memory_commit / cold doc distillation / Pipeline rule distillation / memory_merge
|
|
Read:
|
|
├── Auto recall: Indexer.BuildContext(input)
|
|
│ → CleanTemplateText → vector entity search + jieba keywords → SQLite LIKE + BFS depth=2
|
|
│ → [Memory Index] → system prompt
|
|
└── LLM active: memory_recall / memory_merge / memory_purge / memory_edit / memory_delete_entity
|
|
Social: person_query / set_trait / relate (wraps GraphDB)
|
|
|
|
④ Four Independent Heartbeat Loops (separate tickers and config intervals)
|
|
distillLoop (distillInterval, default 30m): Context pruning — Context.Prune → Document
|
|
archiveLoop (archiveInterval, default 60m): Cold doc archival — docToTriples → GraphDB
|
|
mergeLoop (mergeInterval, default 120m): Entity merge detection — similarity → LLM decision
|
|
reviewLoop (reviewInterval, default 120m): Relation review — SentenceRef recall → LLM fix
|
|
|
|
⑤ Pipeline Rule Distiller (every 10min heartbeat)
|
|
distillOnce → regex match personal info:
|
|
我叫X / 我住在X / 我喜欢X / 我X岁 / 我的工作是X
|
|
→ triples → GraphDB.Commit
|
|
```
|
|
|
|
### Vectorization: Pretrained Word Embedding + TF-IDF Fallback
|
|
|
|
All vectorization unified under `StaticEmbedder` (`internal/memory/static_embedder.go`):
|
|
|
|
**Primary Strategy — Pretrained Word Embedding (aligned 300d)**
|
|
- Model sources: ConceptNet Numberbatch (77-language aligned) / fastText Chinese / fastText English
|
|
- Configured via `core.agent.embedding_model_path` (comma-separated multi-model)
|
|
- Path containing `numberbatch` → auto-download ConceptNet; `cc.zh.` → fastText Chinese; `cc.en.` → fastText English
|
|
- Falls back to ConceptNet by default if no match
|
|
- **Pre-processing**: plugins register per-tool `Cleaner` functions; `textForVector` applies them before `CleanText` final normalization
|
|
- **Three-branch vector source**: agent→Response, user→Input, cold_storage→Input+Response
|
|
- **TF-IDF fallback**: auto-fallback to bag-of-words TF-IDF if model download fails or not configured
|
|
|
|
| Location | File | Purpose | Algorithm |
|
|
|----------|------|---------|-----------|
|
|
| Context Prune | `context.go:155` | Trim low-relevance context events | VectorizeClean → CosineSimilarity(queryVec, evt.Vector) |
|
|
| DocStore Query | `document.go:206` | Recall from document memory | StaticEmbedder.Vectorize (primary) / TF-IDF (fallback) → vec.Search |
|
|
| Indexer Entity Search | `indexer.go:96+111` | Recall from Graph | vector entity search + jieba keywords → SQLite LIKE + BFS |
|
|
| Entity Similarity Detection | `distill.go` | Detect similar entities in Graph | Bigram Jaccard (>0.75 → consolidation) |
|
|
|
|
### Context Layer
|
|
|
|
`internal/agent/core/context.go` — `RelevanceContext`
|
|
- Maintains recent event list, writes JSON on each Append/Prune to prevent data loss
|
|
- Pre-vectorization pipeline: per-tool `Cleaner` functions strip template noise, then `CleanText` for basic whitespace normalization
|
|
- Three-branch `textForVector`: agent events → Response, user events → Input, cold_storage → Input+Response
|
|
- Pretrained word embedding `StaticEmbedder` → CosineSimilarity, auto-fallback to TF-IDF if unavailable
|
|
- Protects last 10 events from eviction; excess candidates are sorted by relevance and archived to document memory
|
|
- Archived events retain original timestamps; on `doc_query` recall they re-insert into the context timeline at their original position
|
|
|
|
### Document Layer
|
|
|
|
`internal/memory/document/document.go` — `Store`
|
|
- Consume-on-read mode: deleted after `doc_query` retrieval
|
|
- Dual recall: shared StaticEmbedder semantic vector search + jieba keyword extraction (falls back to char-bigram TF-IDF when model is not loaded)
|
|
|
|
### Graph Layer
|
|
|
|
`internal/memory/graph.go` — `GraphDB`
|
|
- SQLite WAL mode, two tables (driver: mattn/go-sqlite3, CGo)
|
|
- `Commit(triples)` — UPSERT entities + INSERT relations
|
|
- `Recall(keywords, depth)` — Keyword LIKE search + BFS traversal
|
|
|
|
### Memory Tools (LLM-callable)
|
|
|
|
| Tool | Purpose |
|
|
|------|---------|
|
|
| `memory_recall` | Recall from Graph |
|
|
| `memory_commit` | Write triples to Graph |
|
|
| `memory_merge` | Merge two entity nodes |
|
|
| `memory_purge` | Delete entity node |
|
|
| `memory_edit` | Edit existing entity/relation |
|
|
| `memory_delete_entity` | Delete entity and all its relations |
|
|
| `memory_introspect` | View memory statistics |
|
|
| `doc_query` | Search from Document |
|
|
| `doc_commit` | Write to Document |
|
|
|
|
Since v1.1.1 `memory_commit` and `doc_commit` accept `media_digests`, and the kernel appends the
|
|
`[<mime> <short digest>] <description>` marker into the sentence/body — **the kernel builds the
|
|
marker, the model only supplies the digest**. Requiring the caller to know the format would mean a
|
|
single typo silently breaks reference binding with no error anywhere in the chain. `memory_commit`
|
|
also gained `sentence_text`: media references hang off a sentence, so with no sentence there is
|
|
nowhere to attach them.
|
|
|
|
### Media Memory (since v1.1.0)
|
|
|
|
`internal/memory/media/` — `Store`, content-addressed (CAS)
|
|
|
|
| Concern | Approach | Why |
|
|
|---|---|---|
|
|
| Addressing | sha256 digest; metadata in SQLite, blobs on disk | Identical bytes stored once; metadata must be queryable, blobs must not live in the database |
|
|
| Integrity | Every `Get` re-verifies the digest | Silently returning corrupt data on disk damage is far worse than an error |
|
|
| Write atomicity | `.tmp` + rename | A half-written file taken as complete content would permanently poison that digest |
|
|
| References | `owner_kind/owner_id/digest` composite primary key, `AddRef` idempotent | Three owner kinds: `context` (context events), `document`, `graph_sentence` |
|
|
| GC | Two-stage with `minAge`, **referenced items are never deleted** | Description text stays in the text layers while blobs may be evicted — semantic memory and byte cache are decoupled |
|
|
|
|
**How media is represented in plain-text memory** is the marker `[<mime> <short digest>] <description>`:
|
|
|
|
```
|
|
[image/png a1b2c3d4e5f6] a purple-blue-red three-band chart
|
|
```
|
|
|
|
Why it must ride on text: `Doc.Content`, `sentences.text` and text memory's `Input` are all strings
|
|
— there is no field to carry structured data. **The description text is the durable semantic
|
|
memory** (retrieval uses it); the digest is the key back to the bytes (reverse lookup uses it).
|
|
After capacity GC evicts a blob, the description remains in the L0/L2/L3 text.
|
|
|
|
The media store is **optional throughout**: with `core.memory.media.enabled=false` or no
|
|
configuration, the whole chain silently degrades to plain-text behaviour — no errors, no panics.
|
|
|
|
### Other Memory Layers
|
|
|
|
- **Social** (`internal/memory/social/social.go`) — Persona traits and relationship network, wraps GraphDB entity types
|
|
- **Text Memory** (`internal/memory/text/text.go`) — Raw conversation JSONL logs, rotation strategy
|
|
- **Memory Indexer** (`internal/memory/indexer.go`) — Entity vectorization + jieba keyword extraction, auto-inject into system prompt
|
|
|
|
### Distillation Pipeline
|
|
|
|
`internal/memory/pipeline/pipeline.go`
|
|
- 10-minute tick, 7-day retention
|
|
- Rule-based triple extraction (name / location / likes / age / job patterns)
|
|
- Writes to GraphDB
|
|
|
|
### Context Pruning
|
|
|
|
```
|
|
Four independent heartbeat loops (each with configurable interval):
|
|
├── distillLoop (distillInterval, default 30m)
|
|
│ └── distillContext() — Context.Prune → Document
|
|
├── archiveLoop (archiveInterval, default 60m)
|
|
│ └── archiveColdDocs() — Cold docs → docToTriples → GraphDB
|
|
├── mergeLoop (mergeInterval, default 120m)
|
|
│ └── detectEntityMerge() — Entity similarity detection → LLM decision
|
|
└── reviewLoop (reviewInterval, default 120m)
|
|
└── reviewRelations() — Relation review → SentenceRef recall → LLM fix
|
|
```
|
|
|
|
Entity conflict detection heuristic (bigram Jaccard > 0.75), routed through `selfInputCh` internal channel, LLM makes the final merge decision.
|
|
|
|
|
|
## Knowledge Base
|
|
|
|
`internal/knowledge/knowledge.go`
|
|
- File directory `knowledge/<name>/content.md`
|
|
- Independent TF-IDF index, separate from memory system
|
|
- `knowledge_search` / `knowledge_create` / `knowledge_list`
|
|
|
|
|
|
## Provider & Lua Adapter Layer
|
|
|
|
```
|
|
Agent
|
|
│
|
|
▼
|
|
Provider Interface (Name / Chat / ChatStream)
|
|
│
|
|
├── OpenAIProvider — Standard OpenAI API
|
|
├── OllamaProvider — Local Ollama
|
|
└── LuaAdaptedProvider (primary)
|
|
├── Serialize CompletionRequest → JSON
|
|
├── adapter.transform_request() → API format
|
|
├── HTTP request + adapter.headers
|
|
├── adapter.transform_response() → unified format
|
|
└── Deserialize
|
|
```
|
|
|
|
Code: `internal/agent/api/provider.go`
|
|
|
|
ProviderManager manages multiple sources, fallback in registration order. Lua adapters at `internal/lua/adapters/`, each `.lua` script defines `transform_request` / `transform_response` / `transform_stream_chunk`.
|
|
|
|
VM built-ins: `json.encode` / `json.decode` / `log` / `http_get` / `http_post`.
|
|
|
|
|
|
## Plugin System
|
|
|
|
### Four Loading Methods
|
|
|
|
| Method | Registration Mechanism | Compilation | Usage |
|
|
|--------|----------------------|-------------|-------|
|
|
| Built-in | `init()` → `RegisterFactory` | `internal/plugins/` compiled into kernel | webui/cli/timer/mcp etc. |
|
|
| External subprocess plugin | Handshake + stdio JSON-RPC reverse registration | `plugindev build` → `plugin.bin` (ordinary Go binary) | qq/browser/files etc. |
|
|
| Lua script plugin | Execute `main.lua` to register tools | No compilation, takes effect after restart/reload | luademo etc. |
|
|
| SKILL plugin | Parse `SKILL.md` | Markdown definition | Loaded via clawhubadapter |
|
|
|
|
Built-in plugin registration: `internal/plugins/all.go` blank imports → each plugin `init()` → `Registry.Load()` scans directory to match factory.
|
|
|
|
External plugin loading (since v1.0.0): `internal/plugin/dynamic_proc.go` → `exec.Command(plugin.bin)`
|
|
→ inherit shared-segment fds → handshake (protocol version check) → `plugin.init` → `plugin.start`
|
|
(the plugin reverse-registers tools/stages/channels during this window).
|
|
**The C ABI channel (`-buildmode=c-shared` + bridge) was removed entirely in v1.0.0**—
|
|
the old `plugin.Open` path-cache workarounds (SHA256 temp-path copies) retired with it.
|
|
|
|
Lua script plugin loading: `internal/plugin/` → the gopher-lua interpreter executes `main.lua` (at load time `sdk.register_*` only buffers handlers), then `Start()` swaps in the real SDK implementation and registers them in batch. The script is read only once at load time; runtime execution happens via callbacks.
|
|
|
|
### Built-in vs External Plugins
|
|
|
|
| Dimension | Built-in Plugin | External Plugin |
|
|
|-----------|----------------|-----------------|
|
|
| Registration | `init()` calls `plugin.RegisterFactory(name, factory)` | Implements `NewPluginFactory(name, config) (sdk.Plugin, error)` entry function |
|
|
| Compilation | Compiled into `homed` binary, no separate build | Compiled via `plugindev build` to `plugin.bin` (ordinary Go binary, zero cgo); the kernel spawns it as a subprocess |
|
|
| Distribution | Bundled with kernel, not independently installable | `.hmap` package (ZIP archive), installed via WebUI or pluginmgr API |
|
|
| Metadata | `plugin.RegisterPluginMeta()` for display name | `plugin.json` manifest file (name, version, entry, platforms, capabilities, etc.) |
|
|
| Plugin directory | No separate directory, compiled into binary | `plugins/<name>/` independent directory with `plugin.json` + `plugin.bin` |
|
|
| SDK permissions | Full PluginSDK (SocialAPI read/write, Publish events) | Narrowed `procCore` surface + manifest capabilities declaration + RPC boundary rejection |
|
|
| Lifecycle | Starts/stops with kernel, no individual hot-reload | Independent process; true hot-reload by swapping `plugin.bin` (ReloadOne) plus enable/disable |
|
|
| Crash recovery | No independent recovery | Process-level isolation: a crash cannot take down the kernel; the kernel detaches its registrations then restarts it with backoff (`SetAutoRestart(false)` opts out) |
|
|
|
|
Common ground:
|
|
- Built-in `RegisterFactory` and external `NewPluginFactory` share the same `NativeFactory` type signature
|
|
- `Registry.Load()` handles both uniformly: checks the factory table first (built-in), otherwise dispatches by the manifest `entry` to the proc / lua / skill channel
|
|
- Both use the same `Plugin` interface and public SDK API; tool registration, stage hooks, and output channel APIs are identical
|
|
- Both share the same tool registry (`StageHost`); LLM invocations treat them identically
|
|
|
|
### The Three Communication Planes of Subprocess Plugins (v1.0.0)
|
|
|
|
| Plane | Mechanism | Why this choice |
|
|
|---|---|---|
|
|
| Control | stdio JSON-RPC (NDJSON frames), 55 `core.*` methods | The process boundary *is* the ABI boundary—no need to maintain three platform-specific dynamic-library loaders |
|
|
| Data | Shared memory segment, **one segment shared by all subprocesses** | One segment per plugin would degrade "kernel ctx → segment → plugin mutates → read back" into the copy model under concurrency, reproducing lost updates exactly |
|
|
| Notification | Event ring + platform notify (Linux eventfd / macOS pipe / Windows Event) | The kernel must never block on a consumer: streaming output publishes per token, so any wait shows up as stutter |
|
|
|
|
**Subprocess lifecycle management**:
|
|
- One dedicated `waitLoop` per subprocess (the sole `cmd.Wait()` call site)—it does not rely on
|
|
stdout EOF, because grandchild processes forked by a plugin (browser spawning chromium,
|
|
editdoc spawning python) inherit the same stdout, so EOF never arrives after the plugin itself dies
|
|
- Central ledger `proc.Supervisor`: registered on successful handshake, unregistered on exit;
|
|
`Host.Close()` runs StopAll before tearing down the segment (reversing that order leaves plugins
|
|
holding a mapping that has been unmapped—SIGBUS on their next access)
|
|
- Crash self-healing: detach registrations (tools + stage handlers + IO channels) → remove from
|
|
the registry → restart with backoff
|
|
- Linux `Pdeathsig` is the last-resort guard so subprocesses do not linger as orphans when homed is SIGKILLed
|
|
|
|
### PluginSDK Four Channels
|
|
|
|
```
|
|
Plugin ──→ Kernel
|
|
|
|
RegisterTool(name, fn) ──→ buildToolDefs() / executeToolCall()
|
|
RegisterStage(stage, fn, scope...) ──→ runStage() called at corresponding phase (scope: global / own-tools-only)
|
|
Subscribe(event, fn) ──→ Publish() notify all subscribers
|
|
RegisterOutputChannel(name, caps, desc, handler) ──→ output_send__{name} tool generation
|
|
```
|
|
|
|
`internal/sdk/` bridges external SDK interface to kernel, defines complete PluginSDK:
|
|
|
|
```go
|
|
sdk.RegisterTool(name, def, handler)
|
|
sdk.RegisterStage(stage, handler, scope...)
|
|
sdk.Publish(event)
|
|
sdk.InjectInput(source, channel, payload)
|
|
sdk.InjectInterrupt(source, channel, payload)
|
|
sdk.Memory().Recall/Commit
|
|
sdk.Knowledge().Search/Create
|
|
sdk.Settings().Get/Set/List
|
|
sdk.RegisterOutputChannel("qq", sdk.CapText|sdk.CapAudio|sdk.CapImage, "QQ channel, see output_send__qq_help for details", handler)
|
|
|
|
// v1.1.1 media APIs (all additive, no signature changes)
|
|
sdk.DocMemory().InsertWithMedia(doc, attachments) // attachments with Data land in CAS; Digest-only ones reference existing content
|
|
sdk.InjectInputMedia(source, channel, text, blocks) // media reaches the model in *this* turn
|
|
sdk.InjectInputMediaSync(...) // same, and waits for the reply
|
|
sdk.InjectInterruptMedia(...) // media-bearing interrupt, can preempt current processing
|
|
```
|
|
|
|
How media injection differs from `SetToolBlocks`: the latter is only callable inside a tool handler
|
|
and its media reaches the model with the **next** tool message; these three let a plugin
|
|
**initiate a turn that carries media** — it goes out with this turn's message and is automatically
|
|
stored in CAS with a memory reference attached. `Triple` and `Doc` gained `MediaDigests` /
|
|
`Attachments` correspondingly.
|
|
|
|
`internal/sdk/` is the bridge implementation for this layer and is not subject to the public
|
|
interface freeze (see `docs/git-branching.md` §6).
|
|
|
|
### Plugin Interface
|
|
|
|
```go
|
|
type Plugin interface {
|
|
Name() string
|
|
Start(sdk *PluginSDK) error
|
|
Stop() error
|
|
}
|
|
```
|
|
|
|
|
|
## Output Channel System
|
|
|
|
Each output channel generates two tools:
|
|
|
|
| Tool | Type | Purpose |
|
|
|------|------|---------|
|
|
| `output_send__{name}` | function | Accepts `payload` (content), `meta` (JSON routing metadata), `type` (enum) — routed to plugin handler |
|
|
| `output_send__{name}_help` | function | Returns the channel's meta format and type enum documentation |
|
|
|
|
Capability flags:
|
|
|
|
| Flag | Value | Meaning |
|
|
|------|-------|---------|
|
|
| CapText | 1 | Plain text |
|
|
| CapFile | 2 | File |
|
|
| CapImage | 4 | Image |
|
|
| CapAudio | 8 | Audio |
|
|
| CapStructured | 16 | Structured data |
|
|
|
|
System prompt injection: output gate rules, multi-call support, long message splitting.
|
|
Child agent permission: `output_send__` prefix tools are allowed.
|
|
|
|
|
|
## EventAgentLLMChain Event
|
|
|
|
- Event type `agent_llm_chain` emitted after each LLM turn
|
|
- Contains the full LLM response (text + tool calls + reasoning)
|
|
- WebUI subscribes to this event via SSE for real-time display
|
|
- Plugins can subscribe via EventSubscriber (read-only for external plugins)
|
|
|
|
|
|
## Restricted External Plugin API
|
|
|
|
Layered architecture: internal plugins get full PluginSDK, external plugins get restricted SDK.
|
|
|
|
| API | Internal Plugin | External Plugin |
|
|
|-----|-----------------|-----------------|
|
|
| SocialAPI | Full read/write | Read-only (GetPerson / GetTrait / GetRelations / GetNetwork / ListPersons) |
|
|
| EventSubscriber | Subscribe + Publish | Subscribe-only (no Publish capability) |
|
|
|
|
Extended fields:
|
|
- Triple extensions: Confidence, SubjectType, ObjectType
|
|
- Relation extension: Confidence
|
|
|
|
|
|
## Interrupt Mechanism
|
|
|
|
```
|
|
interceptLoop (goroutine)
|
|
├── InputInterruptChan() ← Timer/message notifications
|
|
├── (a) cancelLLM() → Cancel Provider HTTP request
|
|
├── (b) interceptCh → process() pre-loop read [interrupt message]
|
|
└── (c) InjectInput() → Trigger new processing when idle
|
|
```
|
|
|
|
Three delivery paths:
|
|
|
|
| Path | Effect | Timing |
|
|
|------|--------|--------|
|
|
| cancelLLM | Cancel current HTTP request | On context.Canceled |
|
|
| interceptCh | Insert `[interrupt message]` in process() | Before each LLM call |
|
|
| InjectInput | Trigger new processing when eventLoop is idle | No ongoing request |
|
|
|
|
Code: `internal/agent/core/eventloop.go` — `interceptLoop` / `drainInterrupts`
|
|
|
|
|
|
## Configuration System
|
|
|
|
`internal/config/registry.go` — ConfigRegistry
|
|
|
|
- SQLite storage, `config` table + `config_<plugin>` independent tables
|
|
- Namespaces: `core.*` / `plugin.<name>.*`
|
|
- `RegisterDefault` inserts ~80 default keys (seeds for 8 LLM sources)
|
|
- WebUI settings page `/api/v1/settings` for read/write
|
|
|
|
|
|
## Code Structure
|
|
|
|
```
|
|
cmd/homed/main.go — Entry: assembles all subsystems
|
|
cmd/waiter/main.go — CLI client (Unix socket)
|
|
internal/
|
|
├── agent/
|
|
│ ├── core/ — Agent core (eventLoop/process/stages/context)
|
|
│ │ └── plugin_health.go — Plugin health monitoring and auto-restart
|
|
│ ├── api/ — Provider interface + LuaAdaptedProvider
|
|
│ ├── io/ — IOManager (queue/interrupt/output)
|
|
│ └── personal.go — Persona loading
|
|
├── plugin/
|
|
│ ├── registry.go — Registry + lifecycle
|
|
│ ├── dynamic.go — .so dynamic loader
|
|
│ └── manifest.go — plugin.json metadata
|
|
├── plugins/ — Built-in plugin implementations
|
|
│ ├── all.go — Blank imports
|
|
│ ├── webui/ — HTTP server + embedded SPA
|
|
│ ├── cli/ — Unix socket CLI
|
|
│ ├── timer/ — Timer
|
|
│ ├── cmd/ — Command execution
|
|
│ ├── mcp/ — MCP protocol
|
|
│ ├── files/ — File operations
|
|
│ ├── clawhubadapter/ — ClawHub adapter (OC plugin/SKILL/JS/Python sidecar)
|
|
│ ├── agentcli/ — PTY terminal
|
|
│ ├── healthcheck/ — Health check
|
|
│ ├── pluginmgr/ — Plugin manager
|
|
│ └── cfgmgr/ — Config manager
|
|
├── sdk/ — PluginSDK definitions
|
|
│ ├── plugin.go — Plugin interface + PluginSDK
|
|
│ ├── memory.go — MemoryAPI
|
|
│ ├── knowledge.go — KnowledgeAPI
|
|
│ ├── settings.go — SettingsAPI
|
|
│ └── llm.go — LLMAPI
|
|
├── memory/
|
|
│ ├── graph.go — SQLite graph database
|
|
│ ├── indexer.go — Graph → vector index
|
|
│ ├── vector/store.go — TF-IDF vector engine
|
|
│ ├── document/document.go — Document memory
|
|
│ ├── text/text.go — Text logs
|
|
│ └── pipeline/ — Distiller
|
|
├── knowledge/knowledge.go — Knowledge base
|
|
├── lua/
|
|
│ ├── vm.go — Lua VM (json/log/http)
|
|
│ └── adapters/ — 8 LLM adapter scripts
|
|
├── config/registry.go — SQLite config center
|
|
├── events/bus.go — Event bus
|
|
├── tracker/ — OverlayFS change tracking
|
|
├── supervisor/ — Daemon management
|
|
├── skill/ — Skill plugin management
|
|
│ └── manager.go — Skill loading/matching
|
|
└── meta/ — Meta information
|
|
└── meta.go — Agent metadata
|
|
```
|