JianFeeeee 387b28ce09 fix(memory): 媒体归档三缺陷——数据丢失、L3 入库不可靠、L3 检索未接线
自动触发链实测(medialive)连续暴露的三个缺陷,全部是「手工调 API 的
单测无法发现」的类型。附带该实测本身。

## 缺陷一:三元组全被拒时仍释放引用并删文档(数据丢失)

archiveColdDocs 只检查 len(triples) > 0 就释放媒体引用、删除文档。
但 Commit 会静默跳过实体名不合法的三元组(validEntityName 要求
2–50 字符),于是「无错但一条也没写进去」真实发生:

  [agent] doc→graph: doc_xxx → 0 entities, 0 relations
  [media] 文档 doc_xxx 入图库,释放 1 个媒体引用(描述已留在图库)
  被记忆引用的内容被 GC 删除了(清 1 条/318 字节)

图库里没有任何句子承载引用,文档也被删,blob 被 GC 回收 → 图片与描述
彻底消失。我在上一层写的注释「Commit 之后引用已挂到 graph_sentence」
是错的:ec=0 rc=0 时它什么也没挂。

修法(用户选定 B+A):
  B. ec==0 && rc==0 时保留文档、跳过归档——归档的实质是「信息从 L2
     搬到 L3」,搬不过去就不该删源,下轮再试。
  A. bindSentenceMedia 返回实际绑定数,commitTriplesWithMedia 透出为
     mediaBound;释放前四路判断(查引用出错→保守不释放/本无引用→无需
     释放/mediaBound==0→保留并记录原因/否则释放)。宁可留一条悬空
     引用(内容还在,可由后续一致性检查清理)也不能丢内容。

反向验证:旧行为下新测试确实 FAIL,报「引用被释放了」+「GC 删掉了本该
保留的内容」;恢复修复后 PASS。

## 缺陷二:媒体入 L3 依赖 NLP 提取器运气(可靠性)

媒体能否进图库,取决于提取器碰巧从描述文本里提出合规三元组。实测 LLM
的 477 字图片描述只产出「水平 -分割-> 成」,obj 仅 1 字被拒 → 整条媒体
记忆进不了图库。表现为「阶段 5 时好时坏」,取决于描述文本。

但媒体自身的 digest / mime / 描述都是确定的,不该受提取器支配。

新增 parseMediaMarkers + mediaTriplesFromText:从文档正文的媒体标记
直接产出确定三元组,先于 NLP 提取。同一份真实文档由 0 entities 0
relations 变为 ec=4 rc=2 且拿到句子 id。

三个设计点:
  - 实体名用「图片 <短digest>」而非描述:描述会被重新生成(换视觉模型、
    补描述),若名字取自描述,同一张图会在图谱上留下多个节点。digest
    不变则名字不变,长度也天然合规。
  - SentenceText 用原始标记段,保证 bindSentenceMedia 的正则必然能反解
    到 digest——绑定从概率事件变成确定行为。
  - 描述为空时仍产出「类型」三元组:描述是后台异步补的,媒体节点不该
    因为还没描述就不存在于图谱。
  - summarizeForEntity 按 rune 截断而非字节:按字节切会破坏 UTF-8,
    图库里会留下乱码实体名。同时清 Markdown 强调符。

这是过渡方案,用户已定:下个 feature 换多模态嵌入后不再依赖
「描述文本 → 提取三元组 → 图谱节点」这条链路。

## 缺陷三:L3 媒体检索没有任何调用方(接线缺失)

第四层实现的 RecallMediaForSentence / mediaContextForSentences 从未被
调用——媒体能存进 L3、能反查,但 agent 拿不出来。实测第二轮 agent 显式
调了 doc_query,回答「没有找到那张图片的任何记录」。

接两个入口:
  - buildMemoryContext(自动注入,每次 LLM 调用都走)
  - memory_recall 工具结果末尾(显式查询)

关系行只有实体名和关系类型,看不出「这条记忆当时还带了一张图」,
媒体挂在句子上,必须经 关系→句子→media_refs 反查。

一处折返:最初直接用 injected.Relations 取 sentence_id,测试失败。
Indexer.BuildContext 刻意把 Relations 置 nil(自动注入只给实体索引以省
token,细节留给 memory_recall)。改为用命中的实体名再查一次关系,
深度固定 1——媒体是「这条记忆当时带的图」,顺关系网扩散只会带出无关
媒体并挤占 token。

## medialive 自动触发链实测

internal/agent/core/medialive_test.go,medialive build tag,默认
go test 不收录。源/模型/密钥全部由调用方经环境变量显式指定,缺任何一项
Skip 并列出缺哪个——刻意不提供 fallback,猜一个 base_url 可能打到调用者
机器上不相干的服务,而失败会被误报成「媒体记忆有问题」。

  MEDIALIVE_BASE_URL=... MEDIALIVE_API_KEY=... \
  MEDIALIVE_MODEL=... MEDIALIVE_ADAPTER=... \
  go test -tags medialive ./internal/agent/core/ -run TestMediaLive -v

只注入一个 image 事件,之后七个阶段全由生产代码自己触发:CAS 落盘 →
引用绑定 → 描述生成 → L0→L2 转移 → L2→L3 绑定 → GC 保护 → 第二轮召回。
另有阴性对照:不给记忆时不该「记得」,否则阳性用例的通过可能只是模型
猜常见配色。上游不可用时 Skip 而非假 PASS。

真实 claude-opus-5 实测通过:第二轮不给图,agent 答出
「上:紫罗兰色 #8800DD / 中:蓝色 #0055EE / 下:纯红 #EE0000」。

## 测试

graphmedia_test.go 新增 8 例:数据丢失回归(反向验证过)、mediaBound
计数、媒体标记解析、实体名生成、描述截断、确定性三元组必然可入库、
关系→句子映射、L3 检索接线(自动注入与显式查询两路)。

全仓 go build / go vet / go test 通过,internal/agent/core 与
internal/memory 全绿,SDK 冻结 diff = 0。
2026-09-05 12:03:30 +08:00
2026-07-29 15:45:24 +08:00

⚠️ AI-Assisted Programming Notice: Parts of this project's code, documentation, and commit history were generated or modified with AI assistance. Key changes have been human-reviewed, but please evaluate and verify before use.

HomeAgent

中文: README.md

An Agent framework designed around separation of core domain and application domain. The kernel enforces a zero-IO policy — all external interaction (WebUI, QQ, CLI, file operations, web search, memos, etc.) is handled by the plugin layer; the kernel performs no direct IO operations.

Combined with a three-layer memory architecture (Context → Document → Graph), it maintains contextual coherence across long-running single-conversation sessions through tiered storage and automated archival.

homed (kernel, zero IO)  PluginSDK  plugins (all IO capabilities)

Since v1.0.0 external plugins are independent subprocesses, communicating with the kernel over stdio JSON-RPC (control plane) + a shared memory segment (data plane) + an event ring (notification plane). A plugin crash cannot take down the kernel and it restarts automatically; swapping plugin.bin gives true hot-reload.

Design Principles

Separation of Core Domain and Application Domain — The kernel's responsibilities are limited to LLM orchestration, memory management, and knowledge retrieval; all IO capabilities (message send/receive, file read/write, network requests, hardware interaction, etc.) are implemented by plugins. This separation defines domain boundaries at the Agent framework level, with distinct responsibility scopes for the kernel and plugins.

Three-Layer Memory Architecture — Manages information retention in long-running agents through a tiered storage strategy:

  • Context Layer: Pretrained word embedding / TF-IDF fallback relevance-scored event window, protects last 10 entries, maintains topK context entries
  • Document Layer: Temporary memory with automatic cold data sinking, also supports user-initiated submissions
  • Graph Layer: SQLite graph database, persists entity relationships and semantic memory, supports distillation pipelines to extract triples from conversations

Architecture Diagrams

1. Message Processing Sequence

sequenceDiagram
    participant U as User/Plugin
    participant IO as IOManager
    participant EV as eventLoop
    participant CTX as RelevanceContext
    participant LLM as LLM+Tool Loop
    participant ST as StageHost
    participant MEM as Three-Layer Memory

    U->>IO: InjectInput(type, payload)
    IO->>EV: inputCh
    rect lavender
        Note over EV: processTextInput
        EV->>ST: StageOnInput  Plugin can rewrite/short-circuit
        EV->>CTX: Prune(input,topK)  StaticEmbedder/TF-IDF cosine pruning
        CTX->>MEM: Low-score events archived to Document (original timestamp)
        EV->>CTX: Append(input)  CleanTemplateText→three-branch vector→5s write
    end
    rect lightgreen
        Note over EV,LLM: process()
        EV->>MEM: buildMemoryContext  Indexer recalls from Graph (vector+jieba→BFS depth=2)
        EV->>MEM: buildSystemPrompt  DocQuery summary+Graph memory index+Persona+Skills
        EV->>ST: StagePreAction  Plugin can pre-intercept
        loop Tool loop
            LLM->>LLM: drainInterrupts
            LLM->>LLM: LLM Chat
            LLM->>ST: StagePostAction  Plugin can modify/short-circuit
            alt No tool call
                LLM-->>EV: Returns response
            else
                loop Each tool
                    ST->>ST: StageBeforeToolcall  Plugin can reject
                    LLM->>LLM: executeToolCall
                    ST->>ST: StageAfterToolcall
                end
            end
        end
    end
    rect lightpink
        Note over EV: emitResponse
        CTX->>CTX: Append(response)
        ST->>ST: StageBeforeOutput  Plugin can rewrite
        EV-->>U: ResponseCh CLI sync
        EV-->>EV: Event bus WebUI SSE
        ST->>ST: StageAfterOutput  Read-only
        EV->>MEM: emitMemoryCandidate
    end

2. Stage Pipeline

flowchart LR
    S1[① on_input] --> S2[② pre_action]
    S2 --> S3[③ post_action]
    S3 --> Q{Has tool?}
    Q -->|Yes| S4[④ before_toolcall]
    S4 --> T[executeToolCall]
    T --> S5[⑤ after_toolcall]
    S5 --> S3
    Q -->|No| S6[⑥ before_output]
    S6 --> S7[⑦ after_output]
    style S1 fill:#e1f5fe
    style S3 fill:#fff3e0
    style S6 fill:#e8f5e9

3. Three-Layer Memory

flowchart TB
    subgraph C[① Context Working Window]
        RC[RelevanceContext]
        A[Append] -->|CleanTemplateText→three-branch vector| RC
        P[Prune StaticEmbedder/TF-IDF Cosine] -->|Low score original timestamp| D
        P -->|Keep| TL[timeline→chronological→system prompt]
    end
    subgraph D[② Document File Memory]
        DS[DocStore JSON+TF-IDF]
        Q1[Query summary auto-inject] -->|[Related Memory Docs]| SP
        Q2[doc_query LLM active recall] -->|Consume+delete source| DS
        Q2 -->|Original timestamp write to context| RC
        CD[FindColdDocs 72h] -->|docToTriples| G
    end
    subgraph G[③ Graph Database]
        DB[(SQLite)]
        IDX[Indexer vector+jieba→BFS depth=2] -->|[Memory Index]| SP
        MEM[memory_recall/commit/merge/purge/edit]
        SOC[person_query/set_trait]
    end
    subgraph H[④ Heartbeat Distillation]
        REORG -->|Step3 Cold docs| CD
        REORG -->|Step4 Bigram Jaccard| CONS[consolidation]
        PIPE[Pipeline regex] -->|Name/Address/Likes/Age/Job| DB
    end
    SP[System Prompt] -->|Sequential assembly| LLM
    LLM[LLM] -->|doc_query| Q2
    LLM -->|memory_recall| MEM

See assets/docs/en/ARCHITECTURE.md for details.

Web Mascot

HomeAgent Web Mascot Xiaozhai

Xiaozhai — HomeAgent Web Mascot

Quick Start

make build build-cli
./build/homed -data /tmp/ha
# Interactive mode
./build/waiter

# Or single message
echo "Hello, remember that I like coffee" | ./build/waiter

API keys are configured via WebUI http://localhost:8080 settings page, persisted in SQLite.

Code Structure

cmd/homed/          Daemon entry, assembles all subsystems
cmd/waiter/         CLI client (Unix socket)
internal/
├── agent/core/     Agent core: event loop, LLM tool loop, 7-stage pipeline
├── agent/api/      LLM Provider + 8 Lua adapters
├── memory/         Three-layer memory: Graph(SQLite) / Document(JSON+TF-IDF) / Text(JSONL) + StaticEmbedder(pretrained word embedding/TF-IDF fallback) + CleanTemplateText(de-template)
├── knowledge/      Knowledge base (filesystem + TF-IDF)
├── plugin/         Plugin registry + subprocess loader (stdio RPC + shared memory segment + event ring)
├── plugins/        11 built-in plugins (webui/cli/timer/cmd/mcp/clawhubadapter/agentcli/healthcheck/pluginmgr/files/cfgmgr)
├── sdk/            PluginSDK (Tool/Stage/Event three channels)
├── config/         SQLite config center
├── events/         Event bus
└── internal/lua/adapters/   8 LLM protocol adapter scripts
External plugin development: see [homeagent-sdk](https://gitcode.com/JianFeeeee/homeagent-sdk) repo, use `plugindev` toolchain, refer to Go and Lua examples in `example/`

Project Status

v1.0.0 — External plugins moved from C ABI shared libraries to subprocess + shared memory. The first release that no longer loads .so/.dll, and it is incompatible with 0.9.x (existing plugins must be rebuilt into plugin.bin with the new plugindev, though business code needs zero changes). Eliminates 6 classes of defects that had caused production incidents: hot-reload silently failing (DF_1_NODELETE making dlclose a no-op), no crash isolation (a plugin panic took down homed), stage lost updates (35.8~36.8% loss under the copy model), uncancellable cgo timeouts (linear OS-thread leaks), output_send reporting false success (the model was told "sent" while the message never went out), and Windows capability degradation (only 3 stage fields visible, no write-back). Three communication planes: stdio JSON-RPC (control) + shared memory segment (data) + event ring (notification); the privilege gradient is now enforced by three explicit gates. RPC round-trip p50 24.1µs; crash-to-recovery under 1s.

v0.9.0 — C ABI v2: external plugin Stage callbacks can now write back (invoke_stage gained a result out-param; plugins may mutate RawMessage/LLMText/ToolResults etc. in OnInput/AfterToolcall/PostAction and have them synced to the core). ABI version now tracks core minor releases (v0.9.x → ABIVersion=2, version_min=1 keeps old plugins loadable). Also fixes the tool-loop zen-compat placeholder that wrongly fired on first-turn system context tail. The SDK ships an enhanced sanitizer example (bad-UTF-8 / U+FFFD / ANSI-escape scrub across the whole pipeline). This ABI retired with v1.0.0.

v0.8.0 — Core is functional, plugin system enhanced. 20+ built-in plugins. External plugin development via homeagent-sdk repo. Added input channel NoMemory/Cleaner, ChannelDef, plugin disable/enable system (CLI + WebUI), plugindev toolchain C ABI ChannelDef support.

Documentation

Downloads

Releases ship three variants:

Variant Contents For
full homed + waiter + desktop GUI + systemd unit Single-machine, everything
server homed + waiter + systemd unit Servers (no desktop environment)
client waiter + desktop GUI Connecting to a remote HomeAgent
  • Linux: .deb (amd64/arm64), .rpm (x86_64), .tar.gz
  • Windows: HomeAgent_v1.0.0_{Full,Server,Client}_win64.exe (NSIS installer)
  • Portable: homeagent-bin-<os>_<arch>.tar.gz (homed/waiter/initconfig)
  • Verification: SHA256SUMS

The macOS homed requires a native macOS build (CGO + sqlite3), so release packages ship only waiter/initconfig.

Build

make build build-cli    # Build daemon + CLI
make test               # go test ./...
make install            # Install to system

Dependencies: Go 1.25+, CGo (go-sqlite3), Linux/Windows.

Description
No description provided
Readme AGPL-3.0 83 MiB
Languages
Go 74.6%
JavaScript 12.8%
HTML 3.9%
CSS 3.1%
Python 2.4%
Other 3.1%