JianFeeeee 1fbef8b53e fix(memory): 媒体归档三缺陷——数据丢失、L3 入库不可靠、L3 检索未接线
自动触发链实测(medialive)连续暴露的三个缺陷,全部是「手工调 API 的
单测无法发现」的类型。附带该实测本身。

## 缺陷一:三元组全被拒时仍释放引用并删文档(数据丢失)

archiveColdDocs 只检查 len(triples) > 0 就释放媒体引用、删除文档。
但 Commit 会静默跳过实体名不合法的三元组(validEntityName 要求
2–50 字符),于是「无错但一条也没写进去」真实发生:

  [agent] doc→graph: doc_xxx → 0 entities, 0 relations
  [media] 文档 doc_xxx 入图库,释放 1 个媒体引用(描述已留在图库)
  被记忆引用的内容被 GC 删除了(清 1 条/318 字节)

图库里没有任何句子承载引用,文档也被删,blob 被 GC 回收 → 图片与描述
彻底消失。我在上一层写的注释「Commit 之后引用已挂到 graph_sentence」
是错的:ec=0 rc=0 时它什么也没挂。

修法(用户选定 B+A):
  B. ec==0 && rc==0 时保留文档、跳过归档——归档的实质是「信息从 L2
     搬到 L3」,搬不过去就不该删源,下轮再试。
  A. bindSentenceMedia 返回实际绑定数,commitTriplesWithMedia 透出为
     mediaBound;释放前四路判断(查引用出错→保守不释放/本无引用→无需
     释放/mediaBound==0→保留并记录原因/否则释放)。宁可留一条悬空
     引用(内容还在,可由后续一致性检查清理)也不能丢内容。

反向验证:旧行为下新测试确实 FAIL,报「引用被释放了」+「GC 删掉了本该
保留的内容」;恢复修复后 PASS。

## 缺陷二:媒体入 L3 依赖 NLP 提取器运气(可靠性)

媒体能否进图库,取决于提取器碰巧从描述文本里提出合规三元组。实测 LLM
的 477 字图片描述只产出「水平 -分割-> 成」,obj 仅 1 字被拒 → 整条媒体
记忆进不了图库。表现为「阶段 5 时好时坏」,取决于描述文本。

但媒体自身的 digest / mime / 描述都是确定的,不该受提取器支配。

新增 parseMediaMarkers + mediaTriplesFromText:从文档正文的媒体标记
直接产出确定三元组,先于 NLP 提取。同一份真实文档由 0 entities 0
relations 变为 ec=4 rc=2 且拿到句子 id。

三个设计点:
  - 实体名用「图片 <短digest>」而非描述:描述会被重新生成(换视觉模型、
    补描述),若名字取自描述,同一张图会在图谱上留下多个节点。digest
    不变则名字不变,长度也天然合规。
  - SentenceText 用原始标记段,保证 bindSentenceMedia 的正则必然能反解
    到 digest——绑定从概率事件变成确定行为。
  - 描述为空时仍产出「类型」三元组:描述是后台异步补的,媒体节点不该
    因为还没描述就不存在于图谱。
  - summarizeForEntity 按 rune 截断而非字节:按字节切会破坏 UTF-8,
    图库里会留下乱码实体名。同时清 Markdown 强调符。

这是过渡方案,用户已定:下个 feature 换多模态嵌入后不再依赖
「描述文本 → 提取三元组 → 图谱节点」这条链路。

## 缺陷三:L3 媒体检索没有任何调用方(接线缺失)

第四层实现的 RecallMediaForSentence / mediaContextForSentences 从未被
调用——媒体能存进 L3、能反查,但 agent 拿不出来。实测第二轮 agent 显式
调了 doc_query,回答「没有找到那张图片的任何记录」。

接两个入口:
  - buildMemoryContext(自动注入,每次 LLM 调用都走)
  - memory_recall 工具结果末尾(显式查询)

关系行只有实体名和关系类型,看不出「这条记忆当时还带了一张图」,
媒体挂在句子上,必须经 关系→句子→media_refs 反查。

一处折返:最初直接用 injected.Relations 取 sentence_id,测试失败。
Indexer.BuildContext 刻意把 Relations 置 nil(自动注入只给实体索引以省
token,细节留给 memory_recall)。改为用命中的实体名再查一次关系,
深度固定 1——媒体是「这条记忆当时带的图」,顺关系网扩散只会带出无关
媒体并挤占 token。

## medialive 自动触发链实测

internal/agent/core/medialive_test.go,medialive build tag,默认
go test 不收录。源/模型/密钥全部由调用方经环境变量显式指定,缺任何一项
Skip 并列出缺哪个——刻意不提供 fallback,猜一个 base_url 可能打到调用者
机器上不相干的服务,而失败会被误报成「媒体记忆有问题」。

  MEDIALIVE_BASE_URL=... MEDIALIVE_API_KEY=... \
  MEDIALIVE_MODEL=... MEDIALIVE_ADAPTER=... \
  go test -tags medialive ./internal/agent/core/ -run TestMediaLive -v

只注入一个 image 事件,之后七个阶段全由生产代码自己触发:CAS 落盘 →
引用绑定 → 描述生成 → L0→L2 转移 → L2→L3 绑定 → GC 保护 → 第二轮召回。
另有阴性对照:不给记忆时不该「记得」,否则阳性用例的通过可能只是模型
猜常见配色。上游不可用时 Skip 而非假 PASS。

真实 claude-opus-5 实测通过:第二轮不给图,agent 答出
「上:紫罗兰色 #8800DD / 中:蓝色 #0055EE / 下:纯红 #EE0000」。

## 测试

graphmedia_test.go 新增 8 例:数据丢失回归(反向验证过)、mediaBound
计数、媒体标记解析、实体名生成、描述截断、确定性三元组必然可入库、
关系→句子映射、L3 检索接线(自动注入与显式查询两路)。

全仓 go build / go vet / go test 通过,internal/agent/core 与
internal/memory 全绿,SDK 冻结 diff = 0。
2026-09-05 12:03:30 +08:00
2026-07-29 15:45:24 +08:00

⚠️ AI-Assisted Programming Notice: Parts of this project's code, documentation, and commit history were generated or modified with AI assistance. Key changes have been human-reviewed, but please evaluate and verify before use.

HomeAgent

中文: README.md

An Agent framework designed around separation of core domain and application domain. The kernel enforces a zero-IO policy — all external interaction (WebUI, QQ, CLI, file operations, web search, memos, etc.) is handled by the plugin layer; the kernel performs no direct IO operations.

Combined with a three-layer memory architecture (Context → Document → Graph), it maintains contextual coherence across long-running single-conversation sessions through tiered storage and automated archival.

homed (kernel, zero IO) ← PluginSDK → plugins (all IO capabilities)

Since v1.0.0 external plugins are independent subprocesses, communicating with the kernel over stdio JSON-RPC (control plane) + a shared memory segment (data plane) + an event ring (notification plane). A plugin crash cannot take down the kernel and it restarts automatically; swapping plugin.bin gives true hot-reload.

Design Principles

Separation of Core Domain and Application Domain — The kernel's responsibilities are limited to LLM orchestration, memory management, and knowledge retrieval; all IO capabilities (message send/receive, file read/write, network requests, hardware interaction, etc.) are implemented by plugins. This separation defines domain boundaries at the Agent framework level, with distinct responsibility scopes for the kernel and plugins.

Three-Layer Memory Architecture — Manages information retention in long-running agents through a tiered storage strategy:

  • Context Layer: Pretrained word embedding / TF-IDF fallback relevance-scored event window, protects last 10 entries, maintains topK context entries
  • Document Layer: Temporary memory with automatic cold data sinking, also supports user-initiated submissions
  • Graph Layer: SQLite graph database, persists entity relationships and semantic memory, supports distillation pipelines to extract triples from conversations

Architecture Diagrams

1. Message Processing Sequence

sequenceDiagram
    participant U as User/Plugin
    participant IO as IOManager
    participant EV as eventLoop
    participant CTX as RelevanceContext
    participant LLM as LLM+Tool Loop
    participant ST as StageHost
    participant MEM as Three-Layer Memory

    U->>IO: InjectInput(type, payload)
    IO->>EV: inputCh
    rect lavender
        Note over EV: processTextInput
        EV->>ST: StageOnInput  Plugin can rewrite/short-circuit
        EV->>CTX: Prune(input,topK)  StaticEmbedder/TF-IDF cosine pruning
        CTX->>MEM: Low-score events archived to Document (original timestamp)
        EV->>CTX: Append(input)  CleanTemplateText→three-branch vector→5s write
    end
    rect lightgreen
        Note over EV,LLM: process()
        EV->>MEM: buildMemoryContext  Indexer recalls from Graph (vector+jieba→BFS depth=2)
        EV->>MEM: buildSystemPrompt  DocQuery summary+Graph memory index+Persona+Skills
        EV->>ST: StagePreAction  Plugin can pre-intercept
        loop Tool loop
            LLM->>LLM: drainInterrupts
            LLM->>LLM: LLM Chat
            LLM->>ST: StagePostAction  Plugin can modify/short-circuit
            alt No tool call
                LLM-->>EV: Returns response
            else
                loop Each tool
                    ST->>ST: StageBeforeToolcall  Plugin can reject
                    LLM->>LLM: executeToolCall
                    ST->>ST: StageAfterToolcall
                end
            end
        end
    end
    rect lightpink
        Note over EV: emitResponse
        CTX->>CTX: Append(response)
        ST->>ST: StageBeforeOutput  Plugin can rewrite
        EV-->>U: ResponseCh CLI sync
        EV-->>EV: Event bus WebUI SSE
        ST->>ST: StageAfterOutput  Read-only
        EV->>MEM: emitMemoryCandidate
    end

2. Stage Pipeline

flowchart LR
    S1[① on_input] --> S2[② pre_action]
    S2 --> S3[③ post_action]
    S3 --> Q{Has tool?}
    Q -->|Yes| S4[④ before_toolcall]
    S4 --> T[executeToolCall]
    T --> S5[⑤ after_toolcall]
    S5 --> S3
    Q -->|No| S6[⑥ before_output]
    S6 --> S7[⑦ after_output]
    style S1 fill:#e1f5fe
    style S3 fill:#fff3e0
    style S6 fill:#e8f5e9

3. Three-Layer Memory

flowchart TB
    subgraph C[① Context Working Window]
        RC[RelevanceContext]
        A[Append] -->|CleanTemplateText→three-branch vector| RC
        P[Prune StaticEmbedder/TF-IDF Cosine] -->|Low score original timestamp| D
        P -->|Keep| TL[timeline→chronological→system prompt]
    end
    subgraph D[② Document File Memory]
        DS[DocStore JSON+TF-IDF]
        Q1[Query summary auto-inject] -->|[Related Memory Docs]| SP
        Q2[doc_query LLM active recall] -->|Consume+delete source| DS
        Q2 -->|Original timestamp write to context| RC
        CD[FindColdDocs 72h] -->|docToTriples| G
    end
    subgraph G[③ Graph Database]
        DB[(SQLite)]
        IDX[Indexer vector+jieba→BFS depth=2] -->|[Memory Index]| SP
        MEM[memory_recall/commit/merge/purge/edit]
        SOC[person_query/set_trait]
    end
    subgraph H[④ Heartbeat Distillation]
        REORG -->|Step3 Cold docs| CD
        REORG -->|Step4 Bigram Jaccard| CONS[consolidation]
        PIPE[Pipeline regex] -->|Name/Address/Likes/Age/Job| DB
    end
    SP[System Prompt] -->|Sequential assembly| LLM
    LLM[LLM] -->|doc_query| Q2
    LLM -->|memory_recall| MEM

See assets/docs/en/ARCHITECTURE.md for details.

Web Mascot

HomeAgent Web Mascot Xiaozhai

Xiaozhai — HomeAgent Web Mascot

Quick Start

make build build-cli
./build/homed -data /tmp/ha
# Interactive mode
./build/waiter

# Or single message
echo "Hello, remember that I like coffee" | ./build/waiter

API keys are configured via WebUI http://localhost:8080 settings page, persisted in SQLite.

Code Structure

cmd/homed/          Daemon entry, assembles all subsystems
cmd/waiter/         CLI client (Unix socket)
internal/
├── agent/core/     Agent core: event loop, LLM tool loop, 7-stage pipeline
├── agent/api/      LLM Provider + 8 Lua adapters
├── memory/         Three-layer memory: Graph(SQLite) / Document(JSON+TF-IDF) / Text(JSONL) + StaticEmbedder(pretrained word embedding/TF-IDF fallback) + CleanTemplateText(de-template)
├── knowledge/      Knowledge base (filesystem + TF-IDF)
├── plugin/         Plugin registry + subprocess loader (stdio RPC + shared memory segment + event ring)
├── plugins/        11 built-in plugins (webui/cli/timer/cmd/mcp/clawhubadapter/agentcli/healthcheck/pluginmgr/files/cfgmgr)
├── sdk/            PluginSDK (Tool/Stage/Event three channels)
├── config/         SQLite config center
├── events/         Event bus
└── internal/lua/adapters/   8 LLM protocol adapter scripts
External plugin development: see [homeagent-sdk](https://gitcode.com/JianFeeeee/homeagent-sdk) repo, use `plugindev` toolchain, refer to Go and Lua examples in `example/`

Project Status

v1.0.0 — External plugins moved from C ABI shared libraries to subprocess + shared memory. The first release that no longer loads .so/.dll, and it is incompatible with 0.9.x (existing plugins must be rebuilt into plugin.bin with the new plugindev, though business code needs zero changes). Eliminates 6 classes of defects that had caused production incidents: hot-reload silently failing (DF_1_NODELETE making dlclose a no-op), no crash isolation (a plugin panic took down homed), stage lost updates (35.8~36.8% loss under the copy model), uncancellable cgo timeouts (linear OS-thread leaks), output_send reporting false success (the model was told "sent" while the message never went out), and Windows capability degradation (only 3 stage fields visible, no write-back). Three communication planes: stdio JSON-RPC (control) + shared memory segment (data) + event ring (notification); the privilege gradient is now enforced by three explicit gates. RPC round-trip p50 24.1µs; crash-to-recovery under 1s.

v0.9.0 — C ABI v2: external plugin Stage callbacks can now write back (invoke_stage gained a result out-param; plugins may mutate RawMessage/LLMText/ToolResults etc. in OnInput/AfterToolcall/PostAction and have them synced to the core). ABI version now tracks core minor releases (v0.9.x → ABIVersion=2, version_min=1 keeps old plugins loadable). Also fixes the tool-loop zen-compat placeholder that wrongly fired on first-turn system context tail. The SDK ships an enhanced sanitizer example (bad-UTF-8 / U+FFFD / ANSI-escape scrub across the whole pipeline). This ABI retired with v1.0.0.

v0.8.0 — Core is functional, plugin system enhanced. 20+ built-in plugins. External plugin development via homeagent-sdk repo. Added input channel NoMemory/Cleaner, ChannelDef, plugin disable/enable system (CLI + WebUI), plugindev toolchain C ABI ChannelDef support.

Documentation

Downloads

Releases ship three variants:

Variant Contents For
full homed + waiter + desktop GUI + systemd unit Single-machine, everything
server homed + waiter + systemd unit Servers (no desktop environment)
client waiter + desktop GUI Connecting to a remote HomeAgent
  • Linux: .deb (amd64/arm64), .rpm (x86_64), .tar.gz
  • Windows: HomeAgent_v1.0.0_{Full,Server,Client}_win64.exe (NSIS installer)
  • Portable: homeagent-bin-<os>_<arch>.tar.gz (homed/waiter/initconfig)
  • Verification: SHA256SUMS

The macOS homed requires a native macOS build (CGO + sqlite3), so release packages ship only waiter/initconfig.

Build

make build build-cli    # Build daemon + CLI
make test               # go test ./...
make install            # Install to system

Dependencies: Go 1.25+, CGo (go-sqlite3), Linux/Windows.

Description
No description provided
Readme AGPL-3.0 87 MiB
Languages
Go 74.8%
JavaScript 11.1%
HTML 3%
Python 2.8%
CSS 2.4%
Other 5.7%