在 a822674 的 CAS 层之上把媒体真正接进记忆链路。此前 CAS 只是个孤立的
存储包,没有任何写入方。
## 媒体进入对话有两条路,两条都只把文字留给记忆
1. 用户直接发图 → processMediaInput → mediaToBlocks
ContextEvent.Input 只存 alt 文本("[从 qq 收到了 image]"),
base64 随 message 数组发给模型后就丢了。
2. 插件注入 → SetToolBlocks → process.go 的 mediaMsg
ToolResultItem.Output 只存那句 "[已将图片注入后续对话] /tmp/x.png"。
于是下一轮起,模型能看到的只剩一句路径或一句 alt。那个文件被删、被覆盖,
或者本来就是 /tmp 下的临时产物,连线索都断了。
现在两条路在同一处收口(captureBlockMedia):从 ContentBlock 的 data URL
取出字节存进 CAS,digest 挂到当轮 ContextEvent。
## 改动
internal/agent/core/mediaref.go(新)
- captureBlockMedia:ContentBlock → CAS。只处理 data URL——http(s) URL
拿不到字节就无法内容寻址,而「下载它再存」会把一次对话变成一次网络
请求(超时、鉴权、SSRF 全来了),不在本层解决。
- stage/drainMediaDigests:媒体在 process() 期间被捕获,而承载它的
ContextEvent 要等 process() 返回后才 Append——此刻还没有 owner_id,
故先缓存。与既有 pendingMedia 同一手法,同受 a.mu 保护。
- bindEventMedia:双向落地。evt.Media 让事件记得引了什么(随
context.json 持久化),media_refs 让 CAS 知道谁在引用(GC 的判断依据)。
只写一边的话,要么 GC 误删仍被引用的内容,要么孤儿永远清不掉。
- mediaSummaryForEvent:把已有描述拼成一行写进 Input。这是方案 C 的
落点——**描述文本才是持久语义记忆,blob 只是缓存**。blob 可能被容量
GC 淘汰,但描述会一直留在 L0/L2/L3 的文本里,让「那张紫蓝红三色带图」
几个月后仍可被检索。
ContextEvent 新增 ID 与 Media 两个字段,都是 omitempty:
- ID 懒生成,只有真要挂媒体时才赋值。绝大多数对话没有媒体,全量生成
会让每条事件都多一个字段进 context.json。
- 存量 context.json 读回来两字段皆空,不影响任何既有行为(有测试)。
RelevanceContext.Prune 归档时转移引用(transferMediaRefs):
**先挂到归档文档、再注销原事件引用**。顺序不能反——先销后挂会让引用
计数瞬时归零,若此刻后台 GC 正在跑就会把仍被记忆引用的内容当孤儿清掉。
为此把 Prune 内的局部类型 scored 提为包级 scoredEvent(局部类型无法
出现在方法签名上)。
media 包新增 OwnerContext/OwnerDocument/OwnerGraphSentence 常量:
owner_kind 进了主键,拼错一个字符就是一条永远对不上的孤立引用——
AddRef 不报错,DropOwner 也永远匹配不到。
## 配置
core.memory.media.enabled(默认 true)、.dir、.max_mb(2048)、
.gc_interval(6h)、.gc_min_age(1h)。
关闭后全链路静默跳过,对话行为与本特性上线前完全一致(有测试)。
mediaStore 为 nil 时同理——它是记忆增强,不是对话必需品,开不起来
只记一条 warning 不阻止启动。
## 测试(11 例)
入库与 MIME 归类、http URL 跳过、nil store 全链路 no-op、音视频混合、
stage/drain 清空语义、懒生成 ID、描述作为持久记忆、**归档转移期间内容
始终可读且 refcount 不归零**、无媒体存储时归档照常、context.json
向后兼容往返。
全仓 go build / go vet / go test 通过,SDK 冻结 diff = 0。
## 尚未接入
L3 图库的 graph_sentence owner(常量已备好,无写入方)、
描述生成的后台任务(Pending() 已就绪,尚无消费者)、
媒体 GC 的定时触发(配置项已注册,尚未接 ticker)。
⚠️ AI-Assisted Programming Notice: Parts of this project's code, documentation, and commit history were generated or modified with AI assistance. Key changes have been human-reviewed, but please evaluate and verify before use.
HomeAgent
中文: README.md
An Agent framework designed around separation of core domain and application domain. The kernel enforces a zero-IO policy — all external interaction (WebUI, QQ, CLI, file operations, web search, memos, etc.) is handled by the plugin layer; the kernel performs no direct IO operations.
Combined with a three-layer memory architecture (Context → Document → Graph), it maintains contextual coherence across long-running single-conversation sessions through tiered storage and automated archival.
homed (kernel, zero IO) ← PluginSDK → plugins (all IO capabilities)
Since v1.0.0 external plugins are independent subprocesses, communicating with the kernel over
stdio JSON-RPC (control plane) + a shared memory segment (data plane) + an event ring (notification
plane). A plugin crash cannot take down the kernel and it restarts automatically; swapping
plugin.bin gives true hot-reload.
Design Principles
Separation of Core Domain and Application Domain — The kernel's responsibilities are limited to LLM orchestration, memory management, and knowledge retrieval; all IO capabilities (message send/receive, file read/write, network requests, hardware interaction, etc.) are implemented by plugins. This separation defines domain boundaries at the Agent framework level, with distinct responsibility scopes for the kernel and plugins.
Three-Layer Memory Architecture — Manages information retention in long-running agents through a tiered storage strategy:
- Context Layer: Pretrained word embedding / TF-IDF fallback relevance-scored event window, protects last 10 entries, maintains topK context entries
- Document Layer: Temporary memory with automatic cold data sinking, also supports user-initiated submissions
- Graph Layer: SQLite graph database, persists entity relationships and semantic memory, supports distillation pipelines to extract triples from conversations
Architecture Diagrams
1. Message Processing Sequence
sequenceDiagram
participant U as User/Plugin
participant IO as IOManager
participant EV as eventLoop
participant CTX as RelevanceContext
participant LLM as LLM+Tool Loop
participant ST as StageHost
participant MEM as Three-Layer Memory
U->>IO: InjectInput(type, payload)
IO->>EV: inputCh
rect lavender
Note over EV: processTextInput
EV->>ST: StageOnInput Plugin can rewrite/short-circuit
EV->>CTX: Prune(input,topK) StaticEmbedder/TF-IDF cosine pruning
CTX->>MEM: Low-score events archived to Document (original timestamp)
EV->>CTX: Append(input) CleanTemplateText→three-branch vector→5s write
end
rect lightgreen
Note over EV,LLM: process()
EV->>MEM: buildMemoryContext Indexer recalls from Graph (vector+jieba→BFS depth=2)
EV->>MEM: buildSystemPrompt DocQuery summary+Graph memory index+Persona+Skills
EV->>ST: StagePreAction Plugin can pre-intercept
loop Tool loop
LLM->>LLM: drainInterrupts
LLM->>LLM: LLM Chat
LLM->>ST: StagePostAction Plugin can modify/short-circuit
alt No tool call
LLM-->>EV: Returns response
else
loop Each tool
ST->>ST: StageBeforeToolcall Plugin can reject
LLM->>LLM: executeToolCall
ST->>ST: StageAfterToolcall
end
end
end
end
rect lightpink
Note over EV: emitResponse
CTX->>CTX: Append(response)
ST->>ST: StageBeforeOutput Plugin can rewrite
EV-->>U: ResponseCh CLI sync
EV-->>EV: Event bus WebUI SSE
ST->>ST: StageAfterOutput Read-only
EV->>MEM: emitMemoryCandidate
end
2. Stage Pipeline
flowchart LR
S1[① on_input] --> S2[② pre_action]
S2 --> S3[③ post_action]
S3 --> Q{Has tool?}
Q -->|Yes| S4[④ before_toolcall]
S4 --> T[executeToolCall]
T --> S5[⑤ after_toolcall]
S5 --> S3
Q -->|No| S6[⑥ before_output]
S6 --> S7[⑦ after_output]
style S1 fill:#e1f5fe
style S3 fill:#fff3e0
style S6 fill:#e8f5e9
3. Three-Layer Memory
flowchart TB
subgraph C[① Context Working Window]
RC[RelevanceContext]
A[Append] -->|CleanTemplateText→three-branch vector| RC
P[Prune StaticEmbedder/TF-IDF Cosine] -->|Low score original timestamp| D
P -->|Keep| TL[timeline→chronological→system prompt]
end
subgraph D[② Document File Memory]
DS[DocStore JSON+TF-IDF]
Q1[Query summary auto-inject] -->|[Related Memory Docs]| SP
Q2[doc_query LLM active recall] -->|Consume+delete source| DS
Q2 -->|Original timestamp write to context| RC
CD[FindColdDocs 72h] -->|docToTriples| G
end
subgraph G[③ Graph Database]
DB[(SQLite)]
IDX[Indexer vector+jieba→BFS depth=2] -->|[Memory Index]| SP
MEM[memory_recall/commit/merge/purge/edit]
SOC[person_query/set_trait]
end
subgraph H[④ Heartbeat Distillation]
REORG -->|Step3 Cold docs| CD
REORG -->|Step4 Bigram Jaccard| CONS[consolidation]
PIPE[Pipeline regex] -->|Name/Address/Likes/Age/Job| DB
end
SP[System Prompt] -->|Sequential assembly| LLM
LLM[LLM] -->|doc_query| Q2
LLM -->|memory_recall| MEM
See assets/docs/en/ARCHITECTURE.md for details.
Web Mascot
Quick Start
make build build-cli
./build/homed -data /tmp/ha
# Interactive mode
./build/waiter
# Or single message
echo "Hello, remember that I like coffee" | ./build/waiter
API keys are configured via WebUI http://localhost:8080 settings page, persisted in SQLite.
Code Structure
cmd/homed/ Daemon entry, assembles all subsystems
cmd/waiter/ CLI client (Unix socket)
internal/
├── agent/core/ Agent core: event loop, LLM tool loop, 7-stage pipeline
├── agent/api/ LLM Provider + 8 Lua adapters
├── memory/ Three-layer memory: Graph(SQLite) / Document(JSON+TF-IDF) / Text(JSONL) + StaticEmbedder(pretrained word embedding/TF-IDF fallback) + CleanTemplateText(de-template)
├── knowledge/ Knowledge base (filesystem + TF-IDF)
├── plugin/ Plugin registry + subprocess loader (stdio RPC + shared memory segment + event ring)
├── plugins/ 11 built-in plugins (webui/cli/timer/cmd/mcp/clawhubadapter/agentcli/healthcheck/pluginmgr/files/cfgmgr)
├── sdk/ PluginSDK (Tool/Stage/Event three channels)
├── config/ SQLite config center
├── events/ Event bus
└── internal/lua/adapters/ 8 LLM protocol adapter scripts
External plugin development: see [homeagent-sdk](https://gitcode.com/JianFeeeee/homeagent-sdk) repo, use `plugindev` toolchain, refer to Go and Lua examples in `example/`
Project Status
v1.0.0 — External plugins moved from C ABI shared libraries to subprocess + shared memory. The first release that no longer loads .so/.dll, and it is incompatible with 0.9.x (existing plugins must be rebuilt into plugin.bin with the new plugindev, though business code needs zero changes). Eliminates 6 classes of defects that had caused production incidents: hot-reload silently failing (DF_1_NODELETE making dlclose a no-op), no crash isolation (a plugin panic took down homed), stage lost updates (35.8~36.8% loss under the copy model), uncancellable cgo timeouts (linear OS-thread leaks), output_send reporting false success (the model was told "sent" while the message never went out), and Windows capability degradation (only 3 stage fields visible, no write-back). Three communication planes: stdio JSON-RPC (control) + shared memory segment (data) + event ring (notification); the privilege gradient is now enforced by three explicit gates. RPC round-trip p50 24.1µs; crash-to-recovery under 1s.
v0.9.0 — C ABI v2: external plugin Stage callbacks can now write back (invoke_stage gained a result out-param; plugins may mutate RawMessage/LLMText/ToolResults etc. in OnInput/AfterToolcall/PostAction and have them synced to the core). ABI version now tracks core minor releases (v0.9.x → ABIVersion=2, version_min=1 keeps old plugins loadable). Also fixes the tool-loop zen-compat placeholder that wrongly fired on first-turn system context tail. The SDK ships an enhanced sanitizer example (bad-UTF-8 / U+FFFD / ANSI-escape scrub across the whole pipeline). This ABI retired with v1.0.0.
v0.8.0 — Core is functional, plugin system enhanced. 20+ built-in plugins. External plugin development via homeagent-sdk repo. Added input channel NoMemory/Cleaner, ChannelDef, plugin disable/enable system (CLI + WebUI), plugindev toolchain C ABI ChannelDef support.
Documentation
- Project Overview | 中文
- Technical Architecture | 中文
- Plugin Development Guide | 中文
- Lua Adapter | 中文
- Knowledge Base Demo
Downloads
Releases ship three variants:
| Variant | Contents | For |
|---|---|---|
| full | homed + waiter + desktop GUI + systemd unit | Single-machine, everything |
| server | homed + waiter + systemd unit | Servers (no desktop environment) |
| client | waiter + desktop GUI | Connecting to a remote HomeAgent |
- Linux:
.deb(amd64/arm64),.rpm(x86_64),.tar.gz - Windows:
HomeAgent_v1.0.0_{Full,Server,Client}_win64.exe(NSIS installer) - Portable:
homeagent-bin-<os>_<arch>.tar.gz(homed/waiter/initconfig) - Verification:
SHA256SUMS
The macOS homed requires a native macOS build (CGO + sqlite3), so release packages ship only waiter/initconfig.
Build
make build build-cli # Build daemon + CLI
make test # go test ./...
make install # Install to system
Dependencies: Go 1.25+, CGo (go-sqlite3), Linux/Windows.
