jianf/HomeAgent
1
0
Fork 0
mirror of https://gitcode.com/JianFeeeee/HomeAgent.git synced 2026-09-21 17:38:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
420 Commits 6 Branches 27 Tags
f3232000f4901209c4dbf7c655f36e6df4b4df43
Commit Graph

26 Commits

Author SHA1 Message Date
JianFeeeee
bfdb395731 feat(memory): 新增 chineseclip provider —— text+image 的小体积可商用向量空间 
## 为什么

用户决定「本轮不覆盖 video，先支持 text+image」。这一刀正好解锁了此前
「小 + 可商用 + 覆盖视频」三者不可兼得的僵局：不要求视频后，唯一同时满足
**小、可商用、中文原生** 的选项是 Chinese-CLIP ViT-B/16。

实测对比（同机、真实跑出来的数字）：

| | Chinese-CLIP | jina-v5-omni-nano | Qwen3-VL-Emb-2B |
|---|---|---|---|
| 参数量 | 188M | 1.04B | 2B |
| 产物 / 常驻内存 | 754MB / **1.15GB** | ~2GB / 2.23GB | 8GB / 9.4GB |
| 维度 | 512 | 768 | 2048 |
| 许可 | **Apache-2.0** | CC BY-NC（不可商用） | Apache-2.0 |
| 视频 | 无 | 有 | 有 |

本机可用内存只有 5.3GB，Qwen 的 9.4GB 无法进程内使用；而 ORT format + mmap
那条路被证实当前不通（转换器对三段图段错误；走通还需同时升 ORT 运行时与
Go 绑定，v1.36 要求 API 29 而本机只有 28）。1.15GB 则可以直接进程内跑。

**代价已写进包注释与文档**：CLIP 是双塔对比学习，text↔image 是强项，但纯文本
语义明显弱于 MLLM 型嵌入器；文本检索仍由既有词向量/TF-IDF 路径兜底。
需要更强文本语义或视频时切回 qwen3vl。

## 内容

- `providers/chineseclip/`：按公共 SPI 实现的 provider（注册名 `chineseclip`），
  含 BERT WordPiece 分词器、图像预处理、ONNX 双塔推理、无标签 stub。
- `scripts/export_chineseclip_onnx.py`：从官方权重导出规范产物 + 冻结参考，
  自带逐用例 PyTorch 对比与覆盖度断言（计划集合≠执行集合即非零退出）。
- `cmd/homed/main.go`：空白导入两个 provider，由配置选其一。
- `go.mod`：`golang.org/x/text` 由间接依赖转为直接依赖（删音标需要 NFD）。

## 实现要点

- **分词器逐 token 对齐官方**。第一版探针自己拼 BertTokenizer（只给 vocab.txt、
  没删音标、中文没逐字切），中文被整体切成 [UNK]，三个不同句子产出几乎相同的
  向量（余弦 0.98）——差点把「模型坏了」当成结论。官方配置是 do_lower_case=true
  + 删音标生效 + 中文逐字切分；`TestTokenizerMatchesOfficialReference` 钉住
  逐 token 一致。
- **图像缩放自写 bicubic**（复刻 PIL 的 precompute_coeffs + a=-0.5 核），不引
  golang.org/x/image：它未进本机模块缓存，且最新版要求把整个工具链升到 Go 1.26，
  为一个缩放函数动工具链不划算。
- **归一化在 provider 侧**（两个塔的图里都没归一化），检索按余弦。
- **指纹覆盖全部影响语义的产物**：两个 ONNX 图 + vocab.txt + embed_config.json，
  读不到就写 MISSING（跳过等于对缺件不敏感）。
- 会话 Run 用 runMu 串行化（ORT 会话不保证并发安全），创建/销毁用 mu。

## 模态范围

只声明 `text` 与 `image`；`audio`/`video` 明确返回 `ErrUnsupportedModality`，
绝不用别的模型向量冒充（这是「音频明确 unsupported」纪律的落地）。

## 验证（实测）

导出侧：10 个用例（5 文本 + 5 图像）ONNX vs 官方 PyTorch 全部
`cos = 1.000000000`，覆盖度断言 10/10 通过。

Go 侧（`CHINESECLIP_MODEL_DIR=... go test -tags onnxruntime ./providers/chineseclip/ -v`）：
11/11 通过，其中
- 文本 5 用例 `cos = 1.000000000000`（逐位一致）
- 图像 4 纯色用例 `cos = 1.000000`（与官方预处理在 6 位小数内一致）
- 跨模态判别：红图对「红色」文本高于「蓝色」文本
- 模态拒绝 / 空输入 / 指纹稳定 / 产物缺失报错

顺带修掉测试自身的一个假通过：参考向量是**未归一化**的原始输出（模长 10~36），
原先「点积当余弦 + 单侧下界」会让 13.6 也判过，已改为真余弦 + 双侧容差。

构建矩阵：`go build/vet ./...` 与 `-tags onnxruntime` 两种都过；
`providers/... pkg/... internal/config/... internal/memory/vector/...` 回归通过
（qwen3vl 的 TestVideoModelInputMRope 需要 QWEN_ONNX_MODEL_DIR 指向含视频档的
v3 目录，缺该环境变量时用的是只有文本+图像的目录，与本改动无关）。

## 未做（明确记录）

- 发行版默认 provider 与构建标签变更：留下一提交（涉及打包与模型分发策略）。
- 模型产物（754MB）不进仓库，由导出脚本生成。
 2026-09-11 23:58:53 +08:00
JianFeeeee
14f3605f6e chore: sdk v0.9.2 vendored副本同步 + go.mod 依赖更新 
- third_party/homeagent-sdk/meta/meta.go: 0.9.1 → 0.9.2
- go.mod: require v0.9.2（replace 保留，指向本地 vendored 目录确保可编译性）
- CoreVersion 同步更新
 2026-08-27 09:36:49 +08:00
JianFeeeee
8c5b35a6b1 chore: bump version to v0.9.1 
- Version: 0.9.0 -> 0.9.1
- SDKCompatibleVersion: 0.9.0 -> 0.9.1
- go.mod require homeagent-sdk: v0.8.0 -> v0.9.1
- SDK 版本独立提交（third_party/homeagent-sdk/meta/meta.go）

CABINum 仍为 900（minor=9 不变），ABI 向前兼容。
 2026-08-25 13:15:30 +08:00
dev
22de000f23 fix: bump llm http client timeout 120s→180s for llmsproxy AUTO chain failover 
The local llmsproxy AUTO chain tries 6+ slots across 3 tiers sequentially.
Each failed tier incurs busyWait (2s) + upstream timeout, so a full chain
exhaustion can exceed 120s. The llmsproxy logs showed 143 'context canceled'
errors for the homeagent key — the client gave up before the chain finished.

180s gives the chain enough room to complete before the client timeout fires.
Also remove stale backup files under /usr/local/bin/.
 2026-08-25 00:34:25 +08:00
JianFeeeee
147d0baaf9 fix: LLM 工具循环 400、中断消息注入、ConPTY 终端支持 
- agent: 工具轮请求尾部补 user 占位(zen 网关强制),tool 消息正确配对
- agent: 工具提醒/中断以 system 角色注入并带 [中断消息] 前缀,不进用户履历;系统提示词说明中断消息格式
- agentcli: 基于 ConPTY 的交互式终端(ptywin fork),terminal_create/read/write/resize/close/watch
- webui: server 输出通道适配器(保留 reasoning_content/disable_thinking)
- GUI: 沉浸式标题栏、icon 圆角重制、mascot 等打磨
 2026-08-14 00:48:40 +08:00
root
9d5a914941 Phase 0.1/1/3/6: 核心生产问题修复 
Healthcheck 隔离 (Phase 0.1):
- 新增 internal/sdk/selftest.go: VirtualInstance 完全隔离自检空间
- PluginSDK.Selftest()/SelftestReset() 暴露隔离实例 (含 mutex)
- LLM 自检只读白名单 isSafeReadonlyTool 防写类工具污染生产
- 单测验证: healthcheck 后生产实例内容不变 + 无残留
- 存量清理: 删除 gotest/luatest 残留目录

GraphDB 去重 (Phase 1):
- migrateRelationUnique: 启动自动重建 relations 表加 UNIQUE 约束并去重
- Commit 改为存在性检查, 重复三元组仅刷新 confidence 不重复插入
- 3 个 dedup 单测全绿

配置时长解析 (Phase 3):
- parseDurationExtended 支持 2d/1w/3h 等人类可读单位
- GetDuration 全局生效, 防 2d 静默回退 30m

Agentcli 通知风暴治理 (Phase 6):
- 语义通知: 累积 notify_bytes(2KB) 或间隔 notify_interval(2s) 触发
- 生命周期即时通知: 启动/进程退出/EOF 立即通知
- 可配置 settings, 保留通知机制保证 agent 感知终端存在
- 运维止血: 已杀掉幽灵 PID 3716282 (bash git sparse clone 运行 16h)

Plan.md: 新增设计意图备忘(插件即App/分层记忆), 更新各 Phase 进度
 2026-08-12 13:51:40 +08:00
root
3a5a16c916 bump version to v0.8.0 2026-07-29 15:45:24 +08:00
root
a899d777c3 sdk: embed non-toolchain SDK in third_party, add NoMemory/Cleaner support 
- Embed sdk/, example/, meta/, go.mod from homeagent-sdk (no .git)
- Core .gitignore excludes SDK toolchain: bin/, tools/, package/
- RegisterInputChannel + ChannelDef(NoMemory, Cleaner) in SDK
- IOManager input channel registry with GetInputChannelDef
- eventloop: apply channel Cleaner/NoMemory to interrupt text
- context engine: channelDefLookup applied in textForVector
- document store: ChannelCleaner param for archive functions
- All callers/adapters updated with ChannelDef{} default
 2026-07-29 14:48:23 +08:00
root
51f190e0b6 core: pass ReasoningContent to assistant messages in process.go 
- process.go: attach resp.ReasoningContent when building assistant messages
- deepseek.lua v2.1.0: remove last_reasoning closure hack; rely on
  core-provided reasoning_content in messages
- openai.lua: strip reasoning_content from messages in transform_request
  (not supported by OpenAI API)
 2026-07-28 17:07:17 +08:00
JianFeeeee
2c5f9ff262 v0.7.2: 根目录清理 + Agent 心跳重构 + 内嵌 ONNX 模型 
- 根目录清理: branding/docs/knowledge -> assets/, package/tools/deploy -> deploy/
- meta.go: Version 0.7.2, SDKCompatibleVersion 语义改为最高兼容
- Makefile: 版本回退 0.7.2
- registry.go: 系统提示词改用 meta.Version 格式化
- Agent 心跳: reorgGraph 拆分为三个独立循环(archive/merge/review)，各自可配间隔
- GraphDB: 新增 sentences 表 + 关系句子溯源 + ClearSentenceID + CleanupOrphanedSentences
- Knowledge: 支持词嵌入向量化器
- NLP 四阶段流水线: Parse -> Extract -> Verify -> Fuse + SentenceRef
- 移除远程 HTTP 解析器(remote_parser.go)
- 新增内嵌 ONNX 模型(vocab + dep_parser.onnx):
  +build onnxruntime: 全量 ONNX Runtime 推理
  !build onnxruntime: 内嵌词表规则式降级解析器
- config: core.agent.onnx_model_path 替代 dep_parser_url
 2026-07-28 09:56:26 +08:00
JianFeeeee
31664a7853 feat: NoMemory/Cleaner memory system + doc update 
- _sdk_local/ removed (moved to standalone sdk repo)
- internal/agent/core: NoMemory/Cleaner data-flow breakpoints
- internal/memory: clean_text, document store refactor
- internal/plugin/registry.go: plugin API alignment
- docs: PLUGIN_DEV.md, ARCHITECTURE.md NoMemory/Cleaner docs
- plan.md, review.md: status update
 2026-07-25 11:17:31 +08:00
root
3fc2151588 refactor: pluginize text cleaning and tool NoMemory control 
- SDK: ToolDef.NoMemory field, PluginSDK.RegisterTextCleaner/TextCleaners
- Registry: aggregate text cleaners from plugins, expose CleanText()
- Memory: replace hardcoded QQ regex CleanTemplateText with dynamic CleanText/SetTextCleaner
- StageHost: add ToolDef(name) lookup
- eventloop: check ToolDef.NoMemory before emitMemoryCandidate
- context/Prune: replace hardcoded agentcli/terminal source filter with ToolsUsed NoMemory check
- agentcli/cmd: mark tools with NoMemory: true
- main.go: wire memory.SetTextCleaner(pluginReg.CleanText)
 2026-07-24 14:49:08 +08:00
root
23082fa92d chore: remove local replace directive for homeagent-sdk 2026-07-21 17:58:04 +08:00
jianf
4b0f2f332a feat: Linux packaging, SDK v0.7.1, fix rawmode conflict 
- go.mod: SDK pseudo-version -> v0.7.1 (remote tag), go.sum updated
- cmd/waiter/rawmode_linux.go: remove (conflicts with raw_unix.go)
- cmd/gui/package.json: add rpm/AppImage Linux targets
- package/package-linux.sh: cross-distro packaging script (deb/tar.gz/rpm)
- package/linux/: DEBIAN control files, postinst/prerm, RPM spec
 2026-07-20 11:03:43 +08:00
JianFeeeee
0edd7c0b47 feat: Windows NSIS installer, embedded icons, GUI auto-launch backend 
- Add package/installer.nsi (Full/Server/Client) and toolchain.nsi
- Embed icon.ico (rounded corners) into homed.exe/waiter.exe via .syso
- GUI auto-launches homed.exe from parent dir (Windows only)
- GUI fallback connections.json from app resource dir
- Add initconfig command for config.db seeding
- Replace waiter rawmode* with raw_{unix,windows,other}.go
- Fix .gitignore: /homed instead of homed, add login.json
- Update icon.svg with rounded rect, new mascot.svg
 2026-07-19 23:46:16 +08:00
root
8ae1d19869 refactor: output channel interface (payload/meta/type) + memory fixes 
- Redesign output_send__ tools: content JSON string -> structured
  payload/meta/type params for LLM reliability
- executeOutputSendTool: route by type with capability check
- executeOutputSendHelp: show meta format + type enum
- Updated system prompt rules for new interface
- docToTriples: use jieba exact mode adjacent co-occurrence
- Unify vector space: Doc.Vector field, ContextToDoc vectorizer,
  ReindexWithVectorizer on startup
 2026-07-19 10:29:21 +08:00
root
c9e67d3d55 docs: 修正全部文档使其与源码实现一致 
主仓库：
- 修复 4 份英文文档语言切换链接指向错误 (../zh/ → ../en/)
- ARCHITECTURE.md 标题 "三种加载方式" → "四种加载方式" (实际表格4行)
- PLUGIN_DEV.md 示例表: 添加 webfetch, 移除不存在的 luaplugintest/testlua
- PLUGIN_DEV.md 代码示例: InjectInput/InjectInterrupt → InjectText/InjectInterruptText
- PLUGIN_DEV.md 代码示例: Memory/Knowledge/LLM/Events 接口签名修正
- PLUGIN_DEV.md .hmap 内容统一, plugindev 编译去除 .exe 后缀

SDK 仓库:
- Plugin.Start(sdk *PluginSDK) 接口签名改为指针
- 方法表重写: 移除 CallLLM/QueryKnowledge/SetMemory 等不存在方法
- IOInjector 参数顺序修正为 (source, channel, text)
- 删除虚构 SDKConfig, 替换为实际 New() 构造函数签名
- .hmap 内容描述一致化

修正前一次会话中的 QQ/Bili 插件问题:
- qq napcat() 超时, fetchBotInfo 竞态, handleWebhook 同步阻塞
- bili CDN 直连失败, 添加 HTTP_PROXY 代理
 2026-07-18 20:46:58 +08:00
root
7f28b997e6 feat: output channel redesign - per-channel output gates, LLM chain events, SDKConfig 
- Output channels generate per-channel tools: output_send__{name} (type=output) + output_send__{name}_help
- content is JSON string transparently passed to plugin handler for routing
- EventAgentLLMChain: full LLM response forwarded after each turn for webui/logs
- sdk.New refactored to SDKConfig struct (no more 13 positional args)
- RegisterOutputChannel adds desc param for JSON format documentation
- channelDevice simplified (no Tools method), desc field added
- Child agent permission updated for output_send__ prefix
- System prompt: output gates, multi-call, long messages split
- WebUI: subscribes to EventAgentLLMChain in SSE, no output channel
- Tests updated for new naming convention
 2026-07-16 12:11:16 +08:00
root
fa91b24460 refactor: switch from modernc/sqlite back to mattn/go-sqlite3 (CGo) 2026-07-14 11:43:59 +08:00
JianFeeeee
c7e22fbe30 refactor: move webui.listen_addr from core.daemon to webui category 2026-07-14 11:22:46 +08:00
JianFeeeee
950090959f feat: restructure plugin system, add Lua plugin support, update docs 2026-07-13 21:48:13 +08:00
root
1f1233b823 refactor: P0-P3 fixes, C1 cleanup, architecture diagrams, go.work upgrade 
- P0-1: ProviderError type + ReportStatus for precise 401/403 detection
- P0-2: Remove -config flag from deploy/homeagent.service
- P2-1: 5s debounce on context.go Save()
- P2-2→C1: Delete output_set_channel entirely
- P2-3: Extract mediaDataURL/mediaChat helpers
- P2-4: Dedup defaultSources var
- P3: Delete dead packages (embed/tokenizer/container/snapshot)
- P3: Delete dead functions (messagesToMap, RunStageAll)
- CL: Update .gitignore, docs, Makefile, gojieba removal
- Config: Delete config/config.yaml, update docs
- Arch: Remove EmitOutputTo from emitResponse
- CL-1: go.work 1.19→1.21
- Docs: Add Mermaid architecture diagrams to README
- Docs: Add kernel-rebuild requires plugin-rebuild note to PLUGIN_DEV.md
 2026-07-12 11:42:56 +08:00
root
b55f1dcf0e fix: cap distiller startup scan and switch to formal SDK module dependency 
- replace vendored third_party/homeagent-sdk with formal module dependency
- keep canonical SDK via go.mod pinned commit
- optimize distiller startup loading to stream recent raw records only
- parse timestamps correctly and cap startup load to 5000 records
- remove quadratic string building in raw record parsing
- restore fast startup while preserving memory pipeline
 2026-07-06 20:18:13 +08:00
root
50e5dec745 feat: converge dynamic plugins onto canonical homeagent-sdk 
- Separate built-in plugin interface from external plugin interface
- Route dynamic plugin loading through homeagent-sdk/sdk using reflection
- Turn internal/sdk into an enhanced wrapper over canonical SDK types
- Vendor SDK repo snapshot under third_party/homeagent-sdk for stable builds
- Keep internal constructors/adapters for memory, knowledge, llm, settings
- Align dynamic QQ loading with canonical SDK chain
 2026-07-06 19:27:51 +08:00
root
fab58e709a feat: complete P0/P1/P2 — WebUI SPA, OpenClaw sidecar+simulator, healthcheck auto-sched+perf 
P0: WebUI重构
- 完整 SPA 仪表盘 (7标签页), //go:embed dashboard.html
P1: OpenClaw兼容 (三通道: SKILL.md / sidecar / simulator)
- Node.js 模拟进程统一加载任意 OpenClaw 插件
- JSON-RPC 2.0 over stdio 协议, go:embed 内嵌
P2: Healthcheck 优化
- 定时自动执行 (startAutoCheck, 30min)
- healthcheck_perf 性能监控工具

其他: agentcli/cmd 插件, integration_test, status.go,
      test_deepseek 清理, 多项 bug 修复
 2026-07-04 12:51:32 +08:00
root
bc26850b50 feat: complete HomeAgent architecture v2 
- IO abstraction layer with OutputChannel routing and capability validation
- Three-layer memory (Context-Document-Graph) with TF-IDF relevance pruning
- OneBot V11 QQ protocol plugin with Reverse WebSocket client
- Plugin system with hot-reload (SKILL.md + native factories)
- Knowledge system with TF-IDF vector indexing
- Personality system (personal.md)
- Text memory (JSONL with rotation)
- Change tracker (overlayfs) with rollback
- Lua adapter VM
- Design document (DESIGN.md)

Module: gitcode.com/JianFeeeee/HomeAgent
 2026-07-02 12:04:36 +08:00