mirror of
https://gitcode.com/JianFeeeee/HomeAgent.git
synced 2026-09-22 18:08:04 +00:00
docs: 文档与发布脚本同步到 v1.0.0 子进程架构
README/架构文档仍在描述 C ABI 动态库加载,与 v1.0.0 实际实现不符。
新用户按文档走会去做 -buildmode=c-shared,产物新内核根本不加载。
README.md / README_EN.md:
- 设计要点补子进程架构段(三面通信、崩溃自愈、真热重载)
- 代码结构 plugin/ 描述:.so 动态加载器 → 子进程加载器
- 项目状态补 v1.0.0 条目(6 类缺陷 + 实测数字),v0.9.0 标注 ABI 已退场
- 新增「下载」章节:三变体对照 + 各平台包格式 + macOS 限制
assets/docs/{zh,en}/ARCHITECTURE.md:
- 四种加载方式表:外部 .so/C ABI → 外部子进程/握手+stdio JSON-RPC
- 加载流程改写为 exec.Command → 继承 fd → 握手 → init → start
- 内置 vs 外部对照表 7 行更新
- 新增「子进程插件的三个通信面」小节,含每个面的选择理由
assets/docs/{zh,en}/OVERVIEW.md:插件系统段落改写
deploy/ 发布脚本三处回归(v0.7.2 的 2c5f9ff 把 package/ 移到
deploy/packaging/ 使目录深度 1→2,但没改相对路径,此后两个版本
的发布都没有二进制资产):
- build.sh:.syso 按目标平台 hide/restore(trap 兜底),恢复
windows 目标的 CXX,arm64 刻意不带 CXX
- installer.nsi:5 处 ..\build → ..\..\build,PRODUCT_VERSION 可注入
(原先硬编码 0.8.0)
- homeagent.spec:server 变体补装 waiter(control-server 声明了 CLI 却没装)
deploy/scripts/upload_assets.py:release 资产上传(两步签名 URL → OBS
PUT)。放 deploy/scripts/ 而非 scripts/,因为后者在 .gitignore 里。
支持 GITCODE_REPO/ASSET_DIR 环境变量以复用于 SDK 仓。
This commit is contained in:
25
README.md
25
README.md
@ -12,6 +12,8 @@
|
||||
homed(内核零 IO) ← PluginSDK → 插件(所有 IO 能力)
|
||||
```
|
||||
|
||||
**v1.0.0 起外部插件是独立子进程**:经 stdio JSON-RPC(控制面)+ 共享内存段(数据面)+ 事件环(通知面)与内核通信。插件崩溃不影响内核且自动重启,换 `plugin.bin` 即生效的真热重载。
|
||||
|
||||
## 设计要点
|
||||
|
||||
**核心域与应用域分离** — 内核职责限定为 LLM 编排、记忆管理与知识检索;所有 IO 能力(消息收发、文件读写、网络请求、硬件交互等)由插件实现。这种划分在 Agent 框架层面进行领域边界界定,内核与插件各有其责任范围。
|
||||
@ -180,7 +182,7 @@ internal/
|
||||
├── agent/api/ LLM Provider + 8 个 Lua 适配器
|
||||
├── memory/ 三层记忆:Graph(SQLite) / Document(JSON+TF-IDF) / Text(JSONL) + StaticEmbedder(预训练词嵌入/TF-IDF回退) + CleanTemplateText(去模版)
|
||||
├── knowledge/ 知识库(文件系统 + TF-IDF)
|
||||
├── plugin/ 插件注册表 + .so 动态加载器
|
||||
├── plugin/ 插件注册表 + 子进程加载器(stdio RPC + 共享内存段 + 事件环)
|
||||
├── plugins/ 内置 11 个插件(webui/cli/timer/cmd/mcp/clawhubadapter/agentcli/healthcheck/pluginmgr/files/cfgmgr)
|
||||
├── sdk/ PluginSDK(Tool/Stage/Event 三通道)
|
||||
├── config/ SQLite 配置中心
|
||||
@ -191,7 +193,9 @@ internal/
|
||||
|
||||
## 项目状态
|
||||
|
||||
**v0.9.0** — C ABI v2:外部插件 Stage 回调支持写回(`invoke_stage` 增加 result 输出,插件可在 OnInput/AfterToolcall/PostAction 修改 RawMessage/LLMText/ToolResults 等并同步回内核),ABI 版本随内核 minor 对齐(v0.9.x → ABIVersion=2,`version_min=1` 向后兼容旧插件)。同步修复工具循环 zen 兼容补位误伤首轮 system 上下文的问题。配套 SDK 提供增强版 sanitizer 示例(坏 UTF-8/U+FFFD/ANSI 转义全链路清洗)。
|
||||
**v1.0.0** — 外部插件从 C ABI 动态库迁移到**子进程 + 共享内存**。首个不再加载 `.so`/`.dll` 的版本,与 0.9.x 不兼容(存量插件须用新版 `plugindev` 重编为 `plugin.bin`,**业务代码零改动**)。消除 6 类此前在生产造成故障的缺陷:热重载失效(`DF_1_NODELETE` 让 `dlclose` 成 no-op)、崩溃隔离缺失(插件 panic 带崩 homed)、stage lost update(副本模型丢失 35.8~36.8%)、cgo 超时不可中断(线程线性泄漏)、`output_send` 假成功(模型收到「已发送」而消息未送达)、Windows 能力断层(只见 3 个 stage 字段且无法写回)。三面通信:stdio JSON-RPC(控制)+ 共享内存段(数据)+ 事件环(通知);权限梯度显式化为三道闸。RPC 往返 p50 24.1µs,崩溃到恢复 <1s。
|
||||
|
||||
**v0.9.0** — C ABI v2:外部插件 Stage 回调支持写回(`invoke_stage` 增加 result 输出,插件可在 OnInput/AfterToolcall/PostAction 修改 RawMessage/LLMText/ToolResults 等并同步回内核),ABI 版本随内核 minor 对齐(v0.9.x → ABIVersion=2,`version_min=1` 向后兼容旧插件)。同步修复工具循环 zen 兼容补位误伤首轮 system 上下文的问题。配套 SDK 提供增强版 sanitizer 示例(坏 UTF-8/U+FFFD/ANSI 转义全链路清洗)。**该 ABI 已随 v1.0.0 退场。**
|
||||
|
||||
**v0.8.0** — 核心可用,插件系统增强。内置 20+ 插件,外部插件开发见 [homeagent-sdk](https://gitcode.com/JianFeeeee/homeagent-sdk) 仓库。新增输入通道 `NoMemory`/`Cleaner`、`ChannelDef`、插件禁用/启用系统(CLI + WebUI),`plugindev` 工具链完成 C ABI `ChannelDef` 传递。
|
||||
|
||||
@ -203,6 +207,23 @@ internal/
|
||||
- [Lua Adapter](assets/docs/zh/ADAPTER.md) | [English](assets/docs/en/ADAPTER.md)
|
||||
- [知识库演示](assets/knowledge/homeagent_architecture/content.md)
|
||||
|
||||
## 下载
|
||||
|
||||
[Releases](https://gitcode.com/JianFeeeee/HomeAgent/releases) 提供三种变体:
|
||||
|
||||
| 变体 | 内容 | 适用 |
|
||||
|---|---|---|
|
||||
| **full** | homed + waiter + 桌面 GUI + systemd unit | 单机全功能 |
|
||||
| **server** | homed + waiter + systemd unit | 服务器(无桌面环境) |
|
||||
| **client** | waiter + 桌面 GUI | 连接远程 HomeAgent |
|
||||
|
||||
- Linux:`.deb`(amd64/arm64)、`.rpm`(x86_64)、`.tar.gz`
|
||||
- Windows:`HomeAgent_v1.0.0_{Full,Server,Client}_win64.exe`(NSIS 安装向导)
|
||||
- 免安装:`homeagent-bin-<os>_<arch>.tar.gz`(含 homed/waiter/initconfig)
|
||||
- 校验:`SHA256SUMS`
|
||||
|
||||
macOS 的 `homed` 需在原生 macOS 构建(CGO + sqlite3),发布包仅含 `waiter`/`initconfig`。
|
||||
|
||||
## 构建
|
||||
|
||||
```bash
|
||||
|
||||
28
README_EN.md
28
README_EN.md
@ -12,6 +12,11 @@ Combined with a **three-layer memory architecture** (Context → Document → Gr
|
||||
homed (kernel, zero IO) ← PluginSDK → plugins (all IO capabilities)
|
||||
```
|
||||
|
||||
**Since v1.0.0 external plugins are independent subprocesses**, communicating with the kernel over
|
||||
stdio JSON-RPC (control plane) + a shared memory segment (data plane) + an event ring (notification
|
||||
plane). A plugin crash cannot take down the kernel and it restarts automatically; swapping
|
||||
`plugin.bin` gives true hot-reload.
|
||||
|
||||
## Design Principles
|
||||
|
||||
**Separation of Core Domain and Application Domain** — The kernel's responsibilities are limited to LLM orchestration, memory management, and knowledge retrieval; all IO capabilities (message send/receive, file read/write, network requests, hardware interaction, etc.) are implemented by plugins. This separation defines domain boundaries at the Agent framework level, with distinct responsibility scopes for the kernel and plugins.
|
||||
@ -163,7 +168,7 @@ internal/
|
||||
├── agent/api/ LLM Provider + 8 Lua adapters
|
||||
├── memory/ Three-layer memory: Graph(SQLite) / Document(JSON+TF-IDF) / Text(JSONL) + StaticEmbedder(pretrained word embedding/TF-IDF fallback) + CleanTemplateText(de-template)
|
||||
├── knowledge/ Knowledge base (filesystem + TF-IDF)
|
||||
├── plugin/ Plugin registry + .so/.dll dynamic loader
|
||||
├── plugin/ Plugin registry + subprocess loader (stdio RPC + shared memory segment + event ring)
|
||||
├── plugins/ 11 built-in plugins (webui/cli/timer/cmd/mcp/clawhubadapter/agentcli/healthcheck/pluginmgr/files/cfgmgr)
|
||||
├── sdk/ PluginSDK (Tool/Stage/Event three channels)
|
||||
├── config/ SQLite config center
|
||||
@ -174,7 +179,9 @@ External plugin development: see [homeagent-sdk](https://gitcode.com/JianFeeeee/
|
||||
|
||||
## Project Status
|
||||
|
||||
**v0.9.0** — C ABI v2: external plugin Stage callbacks can now write back (`invoke_stage` gained a result out-param; plugins may mutate RawMessage/LLMText/ToolResults etc. in OnInput/AfterToolcall/PostAction and have them synced to the core). ABI version now tracks core minor releases (v0.9.x → ABIVersion=2, `version_min=1` keeps old plugins loadable). Also fixes the tool-loop zen-compat placeholder that wrongly fired on first-turn system context tail. The SDK ships an enhanced sanitizer example (bad-UTF-8 / U+FFFD / ANSI-escape scrub across the whole pipeline).
|
||||
**v1.0.0** — External plugins moved from C ABI shared libraries to **subprocess + shared memory**. The first release that no longer loads `.so`/`.dll`, and it is incompatible with 0.9.x (existing plugins must be rebuilt into `plugin.bin` with the new `plugindev`, though **business code needs zero changes**). Eliminates 6 classes of defects that had caused production incidents: hot-reload silently failing (`DF_1_NODELETE` making `dlclose` a no-op), no crash isolation (a plugin panic took down homed), stage lost updates (35.8~36.8% loss under the copy model), uncancellable cgo timeouts (linear OS-thread leaks), `output_send` reporting false success (the model was told "sent" while the message never went out), and Windows capability degradation (only 3 stage fields visible, no write-back). Three communication planes: stdio JSON-RPC (control) + shared memory segment (data) + event ring (notification); the privilege gradient is now enforced by three explicit gates. RPC round-trip p50 24.1µs; crash-to-recovery under 1s.
|
||||
|
||||
**v0.9.0** — C ABI v2: external plugin Stage callbacks can now write back (`invoke_stage` gained a result out-param; plugins may mutate RawMessage/LLMText/ToolResults etc. in OnInput/AfterToolcall/PostAction and have them synced to the core). ABI version now tracks core minor releases (v0.9.x → ABIVersion=2, `version_min=1` keeps old plugins loadable). Also fixes the tool-loop zen-compat placeholder that wrongly fired on first-turn system context tail. The SDK ships an enhanced sanitizer example (bad-UTF-8 / U+FFFD / ANSI-escape scrub across the whole pipeline). **This ABI retired with v1.0.0.**
|
||||
|
||||
**v0.8.0** — Core is functional, plugin system enhanced. 20+ built-in plugins. External plugin development via [homeagent-sdk](https://gitcode.com/JianFeeeee/homeagent-sdk) repo. Added input channel `NoMemory`/`Cleaner`, `ChannelDef`, plugin disable/enable system (CLI + WebUI), `plugindev` toolchain C ABI `ChannelDef` support.
|
||||
|
||||
@ -186,6 +193,23 @@ External plugin development: see [homeagent-sdk](https://gitcode.com/JianFeeeee/
|
||||
- [Lua Adapter](assets/docs/en/ADAPTER.md) | [中文](assets/docs/zh/ADAPTER.md)
|
||||
- [Knowledge Base Demo](assets/knowledge/homeagent_architecture/content.md)
|
||||
|
||||
## Downloads
|
||||
|
||||
[Releases](https://gitcode.com/JianFeeeee/HomeAgent/releases) ship three variants:
|
||||
|
||||
| Variant | Contents | For |
|
||||
|---|---|---|
|
||||
| **full** | homed + waiter + desktop GUI + systemd unit | Single-machine, everything |
|
||||
| **server** | homed + waiter + systemd unit | Servers (no desktop environment) |
|
||||
| **client** | waiter + desktop GUI | Connecting to a remote HomeAgent |
|
||||
|
||||
- Linux: `.deb` (amd64/arm64), `.rpm` (x86_64), `.tar.gz`
|
||||
- Windows: `HomeAgent_v1.0.0_{Full,Server,Client}_win64.exe` (NSIS installer)
|
||||
- Portable: `homeagent-bin-<os>_<arch>.tar.gz` (homed/waiter/initconfig)
|
||||
- Verification: `SHA256SUMS`
|
||||
|
||||
The macOS `homed` requires a native macOS build (CGO + sqlite3), so release packages ship only `waiter`/`initconfig`.
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
|
||||
@ -259,12 +259,18 @@ VM built-ins: `json.encode` / `json.decode` / `log` / `http_get` / `http_post`.
|
||||
| Method | Registration Mechanism | Compilation | Usage |
|
||||
|--------|----------------------|-------------|-------|
|
||||
| Built-in | `init()` → `RegisterFactory` | `internal/plugins/` compiled into kernel | webui/cli/timer/mcp etc. |
|
||||
| External `.so` | C ABI dynamic loading | `-buildmode=c-shared` + bridge | qq/browser/files etc. |
|
||||
| External subprocess plugin | Handshake + stdio JSON-RPC reverse registration | `plugindev build` → `plugin.bin` (ordinary Go binary) | qq/browser/files etc. |
|
||||
| Lua script plugin | Execute `main.lua` to register tools | No compilation, takes effect after restart/reload | luademo etc. |
|
||||
| SKILL plugin | Parse `SKILL.md` | Markdown definition | Loaded via clawhubadapter |
|
||||
|
||||
Built-in plugin registration: `internal/plugins/all.go` blank imports → each plugin `init()` → `Registry.Load()` scans directory to match factory.
|
||||
External plugin loading: `internal/plugin/dynamic.go` → copy to SHA256 temp path (bypass `plugin.Open` path cache) → `Open` + `Lookup("NewPlugin")`.
|
||||
|
||||
External plugin loading (since v1.0.0): `internal/plugin/dynamic_proc.go` → `exec.Command(plugin.bin)`
|
||||
→ inherit shared-segment fds → handshake (protocol version check) → `plugin.init` → `plugin.start`
|
||||
(the plugin reverse-registers tools/stages/channels during this window).
|
||||
**The C ABI channel (`-buildmode=c-shared` + bridge) was removed entirely in v1.0.0**—
|
||||
the old `plugin.Open` path-cache workarounds (SHA256 temp-path copies) retired with it.
|
||||
|
||||
Lua script plugin loading: `internal/plugin/` → the gopher-lua interpreter executes `main.lua` (at load time `sdk.register_*` only buffers handlers), then `Start()` swaps in the real SDK implementation and registers them in batch. The script is read only once at load time; runtime execution happens via callbacks.
|
||||
|
||||
### Built-in vs External Plugins
|
||||
@ -272,20 +278,39 @@ Lua script plugin loading: `internal/plugin/` → the gopher-lua interpreter exe
|
||||
| Dimension | Built-in Plugin | External Plugin |
|
||||
|-----------|----------------|-----------------|
|
||||
| Registration | `init()` calls `plugin.RegisterFactory(name, factory)` | Implements `NewPluginFactory(name, config) (sdk.Plugin, error)` entry function |
|
||||
| Compilation | Compiled into `homed` binary, no separate build | Compiled via `plugindev build` to `.so`/`.dll` (`-buildmode=c-shared`), loaded via C ABI bridge |
|
||||
| Compilation | Compiled into `homed` binary, no separate build | Compiled via `plugindev build` to `plugin.bin` (ordinary Go binary, zero cgo); the kernel spawns it as a subprocess |
|
||||
| Distribution | Bundled with kernel, not independently installable | `.hmap` package (ZIP archive), installed via WebUI or pluginmgr API |
|
||||
| Metadata | `plugin.RegisterPluginMeta()` for display name | `plugin.json` manifest file (name, version, entry, platforms, etc.) |
|
||||
| Plugin directory | No separate directory, compiled into binary | `plugins/<name>/` independent directory with `plugin.json` + binary |
|
||||
| SDK permissions | Full PluginSDK (SocialAPI read/write, Publish events) | Restricted SDK (SocialAPI read-only, Subscribe-only events) |
|
||||
| Lifecycle | Starts/stops with kernel, no individual hot-reload | Independent Start/Stop, supports hot-reload (ReloadOne) and enable/disable |
|
||||
| Crash recovery | No independent recovery | Supports `SetAutoRestart(true)` for automatic crash restart |
|
||||
| Metadata | `plugin.RegisterPluginMeta()` for display name | `plugin.json` manifest file (name, version, entry, platforms, capabilities, etc.) |
|
||||
| Plugin directory | No separate directory, compiled into binary | `plugins/<name>/` independent directory with `plugin.json` + `plugin.bin` |
|
||||
| SDK permissions | Full PluginSDK (SocialAPI read/write, Publish events) | Narrowed `procCore` surface + manifest capabilities declaration + RPC boundary rejection |
|
||||
| Lifecycle | Starts/stops with kernel, no individual hot-reload | Independent process; true hot-reload by swapping `plugin.bin` (ReloadOne) plus enable/disable |
|
||||
| Crash recovery | No independent recovery | Process-level isolation: a crash cannot take down the kernel; the kernel detaches its registrations then restarts it with backoff (`SetAutoRestart(false)` opts out) |
|
||||
|
||||
Common ground:
|
||||
- Built-in `RegisterFactory` and external `NewPluginFactory` share the same `NativeFactory` type signature
|
||||
- `Registry.Load()` handles both uniformly: checks factory table first (built-in), falls back to dynamic loading (external)
|
||||
- Both use the same `Plugin` interface and `PluginSDK`; tool registration, stage hooks, and output channel APIs are identical
|
||||
- `Registry.Load()` handles both uniformly: checks the factory table first (built-in), otherwise dispatches by the manifest `entry` to the proc / lua / skill channel
|
||||
- Both use the same `Plugin` interface and public SDK API; tool registration, stage hooks, and output channel APIs are identical
|
||||
- Both share the same tool registry (`StageHost`); LLM invocations treat them identically
|
||||
|
||||
### The Three Communication Planes of Subprocess Plugins (v1.0.0)
|
||||
|
||||
| Plane | Mechanism | Why this choice |
|
||||
|---|---|---|
|
||||
| Control | stdio JSON-RPC (NDJSON frames), 51 `core.*` methods | The process boundary *is* the ABI boundary—no need to maintain three platform-specific dynamic-library loaders |
|
||||
| Data | Shared memory segment, **one segment shared by all subprocesses** | One segment per plugin would degrade "kernel ctx → segment → plugin mutates → read back" into the copy model under concurrency, reproducing lost updates exactly |
|
||||
| Notification | Event ring + platform notify (Linux eventfd / macOS pipe / Windows Event) | The kernel must never block on a consumer: streaming output publishes per token, so any wait shows up as stutter |
|
||||
|
||||
**Subprocess lifecycle management**:
|
||||
- One dedicated `waitLoop` per subprocess (the sole `cmd.Wait()` call site)—it does not rely on
|
||||
stdout EOF, because grandchild processes forked by a plugin (browser spawning chromium,
|
||||
editdoc spawning python) inherit the same stdout, so EOF never arrives after the plugin itself dies
|
||||
- Central ledger `proc.Supervisor`: registered on successful handshake, unregistered on exit;
|
||||
`Host.Close()` runs StopAll before tearing down the segment (reversing that order leaves plugins
|
||||
holding a mapping that has been unmapped—SIGBUS on their next access)
|
||||
- Crash self-healing: detach registrations (tools + stage handlers + IO channels) → remove from
|
||||
the registry → restart with backoff
|
||||
- Linux `Pdeathsig` is the last-resort guard so subprocesses do not linger as orphans when homed is SIGKILLed
|
||||
|
||||
### PluginSDK Four Channels
|
||||
|
||||
```
|
||||
|
||||
@ -50,7 +50,10 @@ Code is in the project root, implemented in Go.
|
||||
|
||||
**Plugin System** (`internal/plugin/`):
|
||||
- Built-in plugins: Go `init()` self-registration, compiled into kernel
|
||||
- External plugins: Go `-buildmode=c-shared` compiled to `.so`, dynamically loaded via C ABI bridge; also supports Lua script plugins
|
||||
- External plugins (since v1.0.0): compiled to an ordinary Go binary `plugin.bin`, spawned by the
|
||||
kernel as an **independent subprocess**, communicating over stdio JSON-RPC (control plane) +
|
||||
a shared memory segment (data plane) + an event ring (notification plane); Lua script plugins are
|
||||
also supported (the C ABI shared-library channel, `-buildmode=c-shared`, was removed entirely in v1.0.0)
|
||||
- PluginSDK (`internal/sdk/`) defines four channels: RegisterTool / RegisterStage / Subscribe / RegisterOutputChannel
|
||||
- 7 stage hooks: on_input → pre_action → post_action → before_toolcall → after_toolcall → before_output → after_output
|
||||
|
||||
|
||||
@ -257,12 +257,18 @@ VM 内置 `json.encode` / `json.decode` / `log` / `http_get` / `http_post`。
|
||||
| 方式 | 注册机制 | 编译 | 用途 |
|
||||
|------|----------|------|------|
|
||||
| 内置插件 | `init()` → `RegisterFactory` | `internal/plugins/` 编译进内核 | webui/cli/timer/mcp 等 |
|
||||
| 外部 `.so` | C ABI 动态加载 | `-buildmode=c-shared` + bridge | qq/browser/files 等 |
|
||||
| 外部子进程插件 | 握手 + stdio JSON-RPC 反向注册 | `plugindev build` → `plugin.bin`(普通 Go 二进制) | qq/browser/files 等 |
|
||||
| Lua 脚本插件 | 执行 `main.lua` 注册工具 | 无需编译,重启/重载生效 | luademo 等 |
|
||||
| SKILL 插件 | 解析 `SKILL.md` | Markdown 定义 | clawhubadapter 兼容加载 |
|
||||
|
||||
内置插件注册:`internal/plugins/all.go` 空白导入 → 各插件 `init()` → `Registry.Load()` 扫描目录匹配工厂。
|
||||
外部插件加载:`internal/plugin/dynamic.go` → 复制到 SHA256 临时路径(绕过 `plugin.Open` 路径缓存)→ `Open` + `Lookup("NewPlugin")`。
|
||||
|
||||
外部插件加载(v1.0.0 起):`internal/plugin/dynamic_proc.go` → `exec.Command(plugin.bin)`
|
||||
→ 继承共享段 fd → 握手(比对 protocol 版本)→ `plugin.init` → `plugin.start`
|
||||
(插件在此期间反向注册工具/阶段/通道)。
|
||||
**C ABI 通道(`-buildmode=c-shared` + bridge)已在 v1.0.0 整体删除**——
|
||||
旧的 `plugin.Open` 路径缓存绕行、SHA256 临时路径复制等手法随之退场。
|
||||
|
||||
Lua 脚本插件加载:`internal/plugin/` → gopher-lua 解释器执行 `main.lua`(加载期 `sdk.register_*` 仅暂存 handler),`Start()` 时替换为真实 SDK 实现并批量注册。脚本只在加载时读取一次,运行期通过回调执行。
|
||||
|
||||
### 内置插件 vs 外部插件
|
||||
@ -270,20 +276,37 @@ Lua 脚本插件加载:`internal/plugin/` → gopher-lua 解释器执行 `main
|
||||
| 维度 | 内置插件 | 外部插件 |
|
||||
|------|----------|----------|
|
||||
| 注册方式 | `init()` 调用 `plugin.RegisterFactory(name, factory)` | 实现 `NewPluginFactory(name, config) (sdk.Plugin, error)` 入口函数 |
|
||||
| 编译方式 | 编译进 `homed` 二进制,无需独立编译 | 通过 `plugindev build` 编译为 `.so`/`.dll`(`-buildmode=c-shared`),C ABI bridge 加载 |
|
||||
| 编译方式 | 编译进 `homed` 二进制,无需独立编译 | 通过 `plugindev build` 编译为 `plugin.bin`(普通 Go 二进制,零 cgo),内核 spawn 为子进程 |
|
||||
| 分发方式 | 随内核分发,不可独立安装/卸载 | `.hmap` 包(ZIP 归档),通过 WebUI 或 pluginmgr API 安装 |
|
||||
| 元数据 | 通过 `plugin.RegisterPluginMeta()` 注册显示名 | `plugin.json` manifest 文件(name, version, entry, platforms 等) |
|
||||
| 插件目录 | 无独立目录,编译进二进制 | `plugins/<name>/` 独立目录,包含 `plugin.json` + 二进制 |
|
||||
| SDK 权限 | 完整 PluginSDK(SocialAPI 读写、Publish 事件) | 受限 SDK(SocialAPI 只读、仅 Subscribe 事件) |
|
||||
| 生命周期 | 随内核启动/停止,不可单独热重载 | 独立 Start/Stop,支持热重载(ReloadOne)和禁用/启用 |
|
||||
| 崩溃恢复 | 无独立恢复机制 | 支持 `SetAutoRestart(true)` 崩溃自动重启 |
|
||||
| 元数据 | 通过 `plugin.RegisterPluginMeta()` 注册显示名 | `plugin.json` manifest 文件(name, version, entry, platforms, capabilities 等) |
|
||||
| 插件目录 | 无独立目录,编译进二进制 | `plugins/<name>/` 独立目录,包含 `plugin.json` + `plugin.bin` |
|
||||
| SDK 权限 | 完整 PluginSDK(SocialAPI 读写、Publish 事件) | 收窄的 `procCore` 能力面 + manifest capabilities 声明 + RPC 边界拒绝 |
|
||||
| 生命周期 | 随内核启动/停止,不可单独热重载 | 独立进程,换 `plugin.bin` 即生效的真热重载(ReloadOne)和禁用/启用 |
|
||||
| 崩溃恢复 | 无独立恢复机制 | 进程级隔离:崩溃不影响内核,内核摘除其注册面后按退避自动重启(`SetAutoRestart(false)` 可关) |
|
||||
|
||||
两者的联系:
|
||||
- 内置插件的工厂函数 `RegisterFactory` 与外部插件的 `NewPluginFactory` 共用同一个 `NativeFactory` 类型签名
|
||||
- `Registry.Load()` 统一处理两者的加载:先查工厂表(内置),无工厂则尝试动态加载(外部)
|
||||
- 两者使用相同的 `Plugin` 接口和 `PluginSDK`,工具注册、阶段钩子、输出通道等 API 完全一致
|
||||
- `Registry.Load()` 统一处理两者的加载:先查工厂表(内置),无工厂则按 manifest 的 `entry` 分派到 proc / lua / skill 通道
|
||||
- 两者使用相同的 `Plugin` 接口和公开 SDK API,工具注册、阶段钩子、输出通道等完全一致
|
||||
- 两者共享同一个工具注册表(`StageHost`),LLM 调用时无差别
|
||||
|
||||
### 子进程插件的三个通信面(v1.0.0)
|
||||
|
||||
| 面 | 机制 | 为何这么选 |
|
||||
|---|---|---|
|
||||
| 控制面 | stdio JSON-RPC(NDJSON 帧),51 个 `core.*` method | 进程边界即 ABI 边界,无需维护三套平台特定的动态库加载代码 |
|
||||
| 数据面 | 共享内存段,**全部子进程共用一块** | 每插件一段会让「内核 ctx → 段 → 插件改 → 回读 ctx」在多插件下退化成副本模型,lost update 原样复现 |
|
||||
| 通知面 | 事件环 + 平台通知(Linux eventfd / macOS pipe / Windows Event) | 内核发事件绕不等消费者,流式输出逐 token 发布时任何等待都会造成卡顿 |
|
||||
|
||||
**子进程生命周期管理**:
|
||||
- 每子进程一根专职 `waitLoop`(`cmd.Wait()` 唯一调用点)——不依赖 stdout EOF,
|
||||
因为插件 fork 的孙子进程(browser 拉 chromium、editdoc 拉 python)继承同一 stdout,
|
||||
插件本体死后 EOF 永不到来
|
||||
- 集中台账 `proc.Supervisor`:握手成功即登记,退出即注销;`Host.Close()` 先 StopAll 再拆段
|
||||
(顺序反了插件还持有映射而段已 unmap,下次访问就是 SIGBUS)
|
||||
- 崩溃自愈:摘注册面(工具 + stage handler + IO 通道)→ 移出注册表 → 退避重启
|
||||
- Linux `Pdeathsig` 兜底 homed 被强杀时子进程不滞留为孤儿
|
||||
|
||||
### PluginSDK 四通道
|
||||
|
||||
```
|
||||
|
||||
@ -50,7 +50,9 @@ HomeAgent 是一个持续运行的个人智能 Agent 框架。
|
||||
|
||||
**插件系统** (`internal/plugin/`):
|
||||
- 内置插件:Go `init()` 自注册,编译进内核
|
||||
- 外部插件:Go `-buildmode=c-shared` 编译为 `.so`,通过 C ABI bridge 动态加载;也支持 Lua 脚本插件
|
||||
- 外部插件(v1.0.0 起):编译为普通 Go 二进制 `plugin.bin`,内核 spawn 为**独立子进程**,
|
||||
经 stdio JSON-RPC(控制面)+ 共享内存段(数据面)+ 事件环(通知面)通信;也支持 Lua 脚本插件
|
||||
(C ABI 动态库通道 `-buildmode=c-shared` 已在 v1.0.0 整体删除)
|
||||
- PluginSDK (`internal/sdk/`) 定义四通道:RegisterTool / RegisterStage / Subscribe / RegisterOutputChannel
|
||||
- 阶段钩子 7 个:on_input → pre_action → post_action → before_toolcall → after_toolcall → before_output → after_output
|
||||
|
||||
|
||||
@ -27,10 +27,15 @@ COMPONENT="${2:-all}"
|
||||
case "$TARGET" in
|
||||
native) GOOS="" GOARCH="" ;;
|
||||
linux/amd64) GOOS=linux GOARCH=amd64 CC="${CC:-}" ;;
|
||||
# arm64 刻意不设 CXX:设了会让 Go 用 aarch64 的 g++ 去链接,
|
||||
# 而它对 host 产生的 .o 报 "file format not recognized"。
|
||||
# gojieba 的 C++ 源仍由 CC 对应的 gcc 驱动编译(gcc 能编 C++)。
|
||||
linux/arm64) GOOS=linux GOARCH=arm64 CC="${CC:-aarch64-linux-gnu-gcc}" ;;
|
||||
darwin/amd64) GOOS=darwin GOARCH=amd64 CC="${CC:-}" ;;
|
||||
darwin/arm64) GOOS=darwin GOARCH=arm64 CC="${CC:-}" ;;
|
||||
windows/amd64) GOOS=windows GOARCH=amd64 CC="${CC:-x86_64-w64-mingw32-gcc}" ;;
|
||||
# Windows 必须同时给 CXX:gojieba 是 C++,缺 CXX 时 cgo 回退到宿主 g++,
|
||||
# 而宿主 g++ 不认 mingw 的 -mthreads,报 unrecognized command-line option。
|
||||
windows/amd64) GOOS=windows GOARCH=amd64 CC="${CC:-x86_64-w64-mingw32-gcc}" CXX="${CXX:-x86_64-w64-mingw32-g++}" ;;
|
||||
all)
|
||||
"$0" linux/amd64 "$COMPONENT"
|
||||
"$0" linux/arm64 "$COMPONENT"
|
||||
@ -60,6 +65,34 @@ export CGO_ENABLED="${CGO_ENABLED:-1}"
|
||||
|
||||
mkdir -p "$BUILD_DIR"
|
||||
|
||||
# ---- .syso 隔离 ----
|
||||
#
|
||||
# cmd/{homed,waiter}/*.syso 是 Windows 资源对象(COFF,含图标/版本信息)。
|
||||
# Go 会把同目录的 .syso 无条件链进任何目标,于是交叉编译到非 Windows 平台时:
|
||||
# - linux/arm64、darwin/arm64 报 "unknown ARM64 relocation type 3"
|
||||
# - 其他架构报 "file format not recognized"
|
||||
# package-linux.sh 有 hide_syso(),但直接调本脚本时没有那层保护——
|
||||
# 这正是 arm64 产物长期缺失的原因(曾被误判为缺 g++ 交叉编译器)。
|
||||
SYSO_HIDDEN=()
|
||||
hide_syso_for_target() {
|
||||
[ "${GOOS:-}" = "windows" ] && return 0
|
||||
local f
|
||||
for f in "$PROJECT_ROOT"/cmd/homed/*.syso "$PROJECT_ROOT"/cmd/waiter/*.syso; do
|
||||
[ -f "$f" ] || continue
|
||||
mv "$f" "$f.hidden"
|
||||
SYSO_HIDDEN+=("$f")
|
||||
done
|
||||
}
|
||||
restore_syso_for_target() {
|
||||
local f
|
||||
for f in "${SYSO_HIDDEN[@]:-}"; do
|
||||
[ -n "$f" ] && [ -f "$f.hidden" ] && mv "$f.hidden" "$f"
|
||||
done
|
||||
SYSO_HIDDEN=()
|
||||
}
|
||||
trap restore_syso_for_target EXIT
|
||||
hide_syso_for_target
|
||||
|
||||
# ---- homed (CGO, sqlite3) ----
|
||||
build_homed() {
|
||||
local out="$BUILD_DIR/homed${SUFFIX:+_$SUFFIX}"
|
||||
|
||||
@ -10,7 +10,12 @@
|
||||
|
||||
!define PRODUCT_NAME "HomeAgent"
|
||||
!define PRODUCT_PUBLISHER "HomeAgent Team"
|
||||
!define PRODUCT_VERSION "0.8.0"
|
||||
# 版本号由 makensis -DPRODUCT_VERSION=X.Y.Z 注入;缺省值仅供本地手工构建。
|
||||
# 此前硬编码 0.8.0 而 release 已到 1.0.0,装出来的包在「添加/删除程序」里
|
||||
# 会显示错误版本(DisplayVersion 也取自这个宏)。
|
||||
!ifndef PRODUCT_VERSION
|
||||
!define PRODUCT_VERSION "1.0.0"
|
||||
!endif
|
||||
|
||||
!if "${VARIANT}" == "full"
|
||||
!define PRODUCT_DISPLAY_NAME "HomeAgent 完整版"
|
||||
@ -38,7 +43,7 @@
|
||||
!endif
|
||||
|
||||
Name "${PRODUCT_DISPLAY_NAME}"
|
||||
OutFile "..\build\${OUTPUT_FILE}"
|
||||
OutFile "..\..\build\${OUTPUT_FILE}"
|
||||
InstallDir "$PROGRAMFILES64\${PRODUCT_NAME}"
|
||||
InstallDirRegKey HKLM "Software\${PRODUCT_NAME}" ""
|
||||
RequestExecutionLevel admin
|
||||
@ -217,17 +222,17 @@ Section "Install" SEC_INSTALL
|
||||
CreateDirectory "$INSTDIR\data\adapters"
|
||||
|
||||
!if "${HAS_CORE}" == "1"
|
||||
File "..\build\initconfig.exe"
|
||||
File "..\build\homed.exe"
|
||||
File "..\..\build\initconfig.exe"
|
||||
File "..\..\build\homed.exe"
|
||||
!endif
|
||||
|
||||
!if "${HAS_WAITER}" == "1"
|
||||
File "..\build\waiter.exe"
|
||||
File "..\..\build\waiter.exe"
|
||||
!endif
|
||||
|
||||
!if "${HAS_GUI}" == "1"
|
||||
SetOutPath "$INSTDIR\homeagent-gui-win32-x64"
|
||||
File /r "..\build\homeagent-gui-win32-x64\*.*"
|
||||
File /r "..\..\build\homeagent-gui-win32-x64\*.*"
|
||||
SetOutPath "$INSTDIR"
|
||||
!endif
|
||||
|
||||
|
||||
@ -63,9 +63,9 @@ install -m 755 %{_sourcedir}/homed %{buildroot}%{_bindir}/homed
|
||||
install -m 644 %{_sourcedir}/homeagent.service %{buildroot}%{_unitdir}/homeagent.service
|
||||
%endif
|
||||
|
||||
%if "%{variant}" == "full" || "%{variant}" == "client"
|
||||
# waiter 三个变体都要:server 也含 CLI(对齐 deb 的 stage_variant
|
||||
# 与 control-server 的 "waiter: command-line client" 描述)。
|
||||
install -m 755 %{_sourcedir}/waiter %{buildroot}%{_bindir}/waiter
|
||||
%endif
|
||||
|
||||
%if "%{variant}" == "full" || "%{variant}" == "client"
|
||||
mkdir -p %{buildroot}%{_datadir}/homeagent-gui
|
||||
@ -81,9 +81,7 @@ cp -r %{_sourcedir}/homeagent-gui-linux-*/* %{buildroot}%{_datadir}/homeagent-gu
|
||||
%dir %{_varlibdir}
|
||||
%endif
|
||||
|
||||
%if "%{variant}" == "full" || "%{variant}" == "client"
|
||||
%{_bindir}/waiter
|
||||
%endif
|
||||
|
||||
%if "%{variant}" == "full" || "%{variant}" == "client"
|
||||
%dir %{_datadir}/homeagent-gui
|
||||
|
||||
107
deploy/scripts/upload_assets.py
Executable file
107
deploy/scripts/upload_assets.py
Executable file
@ -0,0 +1,107 @@
|
||||
#!/usr/bin/env python3
|
||||
"""上传 release 资产到 gitcode(两步:取签名 URL → PUT 到 OBS)。
|
||||
|
||||
用法: upload_assets.py <tag> <token> [file...]
|
||||
不传 file 时上传 dist/release/ 下全部发布产物。
|
||||
|
||||
环境变量:
|
||||
GITCODE_REPO 目标仓库,默认 JianFeeeee/HomeAgent(SDK 仓传 JianFeeeee/homeagent-sdk)
|
||||
ASSET_DIR 资产目录,默认 <repo>/dist/release
|
||||
|
||||
为何两步:gitcode 的 release 附件不走 API 直传,而是先向
|
||||
`releases/<tag>/upload_url` 要一个 OBS 预签名 URL(带 x-obs-* 回调头),
|
||||
再把文件 PUT 到那个 URL。回调头必须原样透传,否则 OBS 收下了文件但
|
||||
gitcode 侧不会登记为 release 附件。
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
REPO = os.environ.get("GITCODE_REPO", "JianFeeeee/HomeAgent")
|
||||
API = "https://gitcode.com/api/v5/repos"
|
||||
|
||||
# 发布产物后缀。注意 Windows 安装器是 HomeAgent_v*_win64.exe,
|
||||
# 与 bin/ 里的裸 .exe 靠 _win64.exe 后缀区分。
|
||||
ARTIFACT_SUFFIXES = (
|
||||
".tar.gz",
|
||||
".zip",
|
||||
".deb",
|
||||
".rpm",
|
||||
".pkg",
|
||||
"_win64.exe",
|
||||
)
|
||||
|
||||
|
||||
def is_artifact(name: str) -> bool:
|
||||
return name == "SHA256SUMS" or name.endswith(ARTIFACT_SUFFIXES)
|
||||
|
||||
|
||||
def get_upload_url(tag: str, token: str, filename: str) -> tuple[str, dict]:
|
||||
q = urllib.parse.urlencode({"file_name": filename})
|
||||
url = f"{API}/{REPO}/releases/{tag}/upload_url?{q}"
|
||||
req = urllib.request.Request(url, headers={"private-token": token})
|
||||
with urllib.request.urlopen(req, timeout=30) as r:
|
||||
data = json.loads(r.read())
|
||||
return data["url"], data.get("headers", {})
|
||||
|
||||
|
||||
def put_file(url: str, headers: dict, path: str) -> tuple[int, str]:
|
||||
size = os.path.getsize(path)
|
||||
with open(path, "rb") as f:
|
||||
body = f.read()
|
||||
req = urllib.request.Request(url, data=body, method="PUT")
|
||||
for k, v in headers.items():
|
||||
req.add_header(k, v)
|
||||
req.add_header("Content-Length", str(size))
|
||||
try:
|
||||
# 大文件(Full 变体安装包近 100MB)给足超时。
|
||||
with urllib.request.urlopen(req, timeout=900) as r:
|
||||
return r.status, r.read().decode("utf-8", "replace")[:300]
|
||||
except urllib.error.HTTPError as e:
|
||||
return e.code, e.read().decode("utf-8", "replace")[:300]
|
||||
except Exception as e: # noqa: BLE001
|
||||
return 0, f"{type(e).__name__}: {e}"
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if len(sys.argv) < 3:
|
||||
print(__doc__)
|
||||
return 2
|
||||
tag, token = sys.argv[1], sys.argv[2]
|
||||
outdir = os.environ.get("ASSET_DIR") or os.path.join(
|
||||
os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
|
||||
"dist",
|
||||
"release",
|
||||
)
|
||||
files = sys.argv[3:] or sorted(
|
||||
f for f in os.listdir(outdir) if is_artifact(f)
|
||||
)
|
||||
print(f"repo={REPO} tag={tag} dir={outdir}", flush=True)
|
||||
failed = []
|
||||
for name in files:
|
||||
path = os.path.join(outdir, name)
|
||||
if not os.path.isfile(path):
|
||||
print(f"skip (missing): {name}", flush=True)
|
||||
continue
|
||||
mib = os.path.getsize(path) / 1048576
|
||||
print(f"==> {name} ({mib:.1f} MiB)", flush=True)
|
||||
try:
|
||||
url, headers = get_upload_url(tag, token, name)
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f" upload_url FAILED: {e}", flush=True)
|
||||
failed.append(name)
|
||||
continue
|
||||
status, body = put_file(url, headers, path)
|
||||
ok = 200 <= status < 300
|
||||
print(f" PUT -> {status} {'OK' if ok else body}", flush=True)
|
||||
if not ok:
|
||||
failed.append(name)
|
||||
print(f"\n{'ALL OK' if not failed else f'{len(failed)} FAILED: ' + ', '.join(failed)}")
|
||||
return 1 if failed else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user