5b0cd45093
feat: screensee 工具 — agent 查看远程设备屏幕内容
...
与 screensue(向用户屏幕显示)配对: screensue 是给用户看, screensee 是 agent 看。
服务端实现:
1. remotedevice 新增 screensee 工具:
- 下发 homeagent-screensee 命令 → 设备截屏回传 jpeg base64
- seeHandler 回调(agent 核心注入)用视觉模型自动描述屏幕内容
- 未授权/离线/超时完整错误路径; 结果留档 cmdresult
2. SDK LLMMessage 扩展多模态 Blocks(text/image_url):
- llm_impl 转换为 agentAPI.ContentBlock, 视觉模型可看图
3. describeScreen: 默认提示词描述窗口/文字/界面状态;
provider 参数可指定视觉源(临时切换后恢复)
GUI 端需配套(已发群): onDeviceMsg 加 case "screensee",
desktopCapturer 截屏 → jpeg base64 data URL 回执(同 camerasue 抓拍模式)。
测试: 端到端模拟设备截屏回传+视觉回调验证; 全项目 go test 通过
2026-08-21 11:31:45 +08:00
dbbd73b930
refactor: migrate built-in plugins to SDK-only interface
...
- Six-phase plan complete: webui/cli/healthcheck/pluginmgr/clawhubadapter
now interact with the kernel exclusively via internal/sdk interfaces;
all Configure() calls and package-level global injection removed
- buildSDK in internal/plugin/registry.go is the single assembly point
- Add internal/sdk/events.go exporting event types/constants
- Fix ProviderManager cooldown sharing: LuaAdaptedProvider.Name() now
returns the source name instead of lua_<adapter>, so multiple sources
sharing an adapter (single script load via shared VM AdapterCache) no
longer share failure-cooldown state
- Verified: build/vet/tests green, deployed to homeagent.service with
full plugin capability testing via local OpenAI-compatible mock
2026-08-01 12:17:17 +08:00
1f1233b823
refactor: P0-P3 fixes, C1 cleanup, architecture diagrams, go.work upgrade
...
- P0-1: ProviderError type + ReportStatus for precise 401/403 detection
- P0-2: Remove -config flag from deploy/homeagent.service
- P2-1: 5s debounce on context.go Save()
- P2-2→C1: Delete output_set_channel entirely
- P2-3: Extract mediaDataURL/mediaChat helpers
- P2-4: Dedup defaultSources var
- P3: Delete dead packages (embed/tokenizer/container/snapshot)
- P3: Delete dead functions (messagesToMap, RunStageAll)
- CL: Update .gitignore, docs, Makefile, gojieba removal
- Config: Delete config/config.yaml, update docs
- Arch: Remove EmitOutputTo from emitResponse
- CL-1: go.work 1.19→1.21
- Docs: Add Mermaid architecture diagrams to README
- Docs: Add kernel-rebuild requires plugin-rebuild note to PLUGIN_DEV.md
2026-07-12 11:42:56 +08:00
50e5dec745
feat: converge dynamic plugins onto canonical homeagent-sdk
...
- Separate built-in plugin interface from external plugin interface
- Route dynamic plugin loading through homeagent-sdk/sdk using reflection
- Turn internal/sdk into an enhanced wrapper over canonical SDK types
- Vendor SDK repo snapshot under third_party/homeagent-sdk for stable builds
- Keep internal constructors/adapters for memory, knowledge, llm, settings
- Align dynamic QQ loading with canonical SDK chain
2026-07-06 19:27:51 +08:00
2d314b3e9c
重构: 插件自注册 + .so 动态加载 + 中断打断机制
...
- 所有内置插件 init() 自注册 (plugin.RegisterFactory), 移除 main.go 硬编码
- 新增 .so 动态加载器 (internal/plugin/dynamic.go), 插件可编译为 plugin.so
- 新增 plugin.json 元数据 (internal/plugin/manifest.go)
- 新增 interceptLoop 独立 goroutine:
(a) cancelLLM() 取消进行中的 HTTP 请求
(b) interceptCh → drainInterrupt() 注入 [打断消息] 到 LLM 上下文
(c) InjectInput 空闲时触发新处理循环
- 新增 internal/plugins/all.go 空白导入触发所有内置插件 init()
- internal/sdk/ 作为 PluginSDK 正式 Go API
- internal/api/ → internal/plugins/webui/ 迁移
- 删除旧 cmd/cli/, 使用 cmd/waiter/ 替代
- 更新 PLAN.md / ARCHITECTURE.md / README.md 文档
2026-07-03 16:53:34 +08:00
