JianFeeeee 5b0cd45093 feat: screensee 工具 — agent 查看远程设备屏幕内容 ...

与 screensue(向用户屏幕显示)配对: screensue 是给用户看, screensee 是 agent 看。

服务端实现:
1. remotedevice 新增 screensee 工具:
   - 下发 homeagent-screensee 命令 → 设备截屏回传 jpeg base64
   - seeHandler 回调(agent 核心注入)用视觉模型自动描述屏幕内容
   - 未授权/离线/超时完整错误路径; 结果留档 cmdresult
2. SDK LLMMessage 扩展多模态 Blocks(text/image_url):
   - llm_impl 转换为 agentAPI.ContentBlock, 视觉模型可看图
3. describeScreen: 默认提示词描述窗口/文字/界面状态;
   provider 参数可指定视觉源(临时切换后恢复)

GUI 端需配套(已发群): onDeviceMsg 加 case "screensee",
desktopCapturer 截屏 → jpeg base64 data URL 回执(同 camerasue 抓拍模式)。

测试: 端到端模拟设备截屏回传+视觉回调验证; 全项目 go test 通过

2026-08-21 11:31:45 +08:00

..

adapter_impl.go

refactor: migrate built-in plugins to SDK-only interface

2026-08-01 12:17:17 +08:00

adapter.go

refactor: migrate built-in plugins to SDK-only interface

2026-08-01 12:17:17 +08:00

config_impl.go

refactor: migrate built-in plugins to SDK-only interface

2026-08-01 12:17:17 +08:00

config.go

refactor: migrate built-in plugins to SDK-only interface

2026-08-01 12:17:17 +08:00

events.go

fix: LLM 工具循环 400、中断消息注入、ConPTY 终端支持

2026-08-14 00:48:40 +08:00

indexer_impl.go

refactor: migrate built-in plugins to SDK-only interface

2026-08-01 12:17:17 +08:00

indexer.go

refactor: migrate built-in plugins to SDK-only interface

2026-08-01 12:17:17 +08:00

knowledge_impl.go

refactor: migrate built-in plugins to SDK-only interface

2026-08-01 12:17:17 +08:00

knowledge.go

refactor: migrate built-in plugins to SDK-only interface

2026-08-01 12:17:17 +08:00

llm_impl.go

feat: screensee 工具 — agent 查看远程设备屏幕内容

2026-08-21 11:31:45 +08:00

llm.go

feat: screensee 工具 — agent 查看远程设备屏幕内容

2026-08-21 11:31:45 +08:00

memory_impl.go

refactor: migrate built-in plugins to SDK-only interface

2026-08-01 12:17:17 +08:00

memory.go

refactor: migrate built-in plugins to SDK-only interface

2026-08-01 12:17:17 +08:00

plugin.go

feat: 导出单插件重载到 SDK (CORE_PLUGIN_RELOAD_ONE/48 + LIST/49 + IS_DISABLED/50)

2026-08-16 18:27:25 +08:00

selftest.go

Phase 0.1/1/3/6: 核心生产问题修复

2026-08-12 13:51:40 +08:00

settings_impl.go

webui: 修复 '删除源/服务器' 只置 <nil> 不真删的问题

2026-08-10 09:35:29 +08:00

settings.go

webui: 修复 '删除源/服务器' 只置 <nil> 不真删的问题

2026-08-10 09:35:29 +08:00

status.go

refactor: remove core skill direct loading, skills owned by clawhubadapter only

2026-08-02 11:40:13 +08:00

supervisor.go

refactor: migrate built-in plugins to SDK-only interface

2026-08-01 12:17:17 +08:00

tool_impl.go

refactor: migrate built-in plugins to SDK-only interface

2026-08-01 12:17:17 +08:00

tool.go

refactor: migrate built-in plugins to SDK-only interface

2026-08-01 12:17:17 +08:00

tracker.go

refactor: migrate built-in plugins to SDK-only interface

2026-08-01 12:17:17 +08:00