33 Commits

Author SHA1 Message Date
8c5b35a6b1 chore: bump version to v0.9.1
- Version: 0.9.0 -> 0.9.1
- SDKCompatibleVersion: 0.9.0 -> 0.9.1
- go.mod require homeagent-sdk: v0.8.0 -> v0.9.1
- SDK 版本独立提交(third_party/homeagent-sdk/meta/meta.go)

CABINum 仍为 900(minor=9 不变),ABI 向前兼容。
2026-08-25 13:15:30 +08:00
79b7766ed4 fix: 流式渲染回合生命周期 + LLM 瞬断重试与 SSE body 兜底
问题一(webui 不是真流式):
- sendChat 的 finally 在 POST 结束(15s ackTimer abort)时就复位
  chatLoading,但 agent 生成窗口 15~190s,后续 SSE delta 全部走
  全量重建路径、停止按钮提前消失、用户误发重复消息。
- GUI app.js 完全没有 content_delta/reasoning_delta 监听器,
  只能等聚合帧一次性显示。

修复:三端统一回合生命周期——POST 只是触发,收尾由 SSE 驱动:
- dashboard/GUI 新增 endChatTurn/armTurnWatchdog;拿到同步兜底
  响应立即收尾,否则保持回合打开等 agent_output final / reset 帧 /
  120s watchdog 兜底
- GUI 补齐 delta 监听器;agent_output 聚合分支 += 改覆盖;
  reasoning 聚合帧改覆盖(多轮工具调用时旧逻辑会重复累加)
- agent_output 误杀分支(final 无 source 即 return 丢弃新输出)
  改为内容比较去重,多轮连发时新一轮回复不再被吞
- waiter reasoning_delta reset 从清空全部消息改为 sealLastAgent

问题二(三条只成功一条):
- handleChat 60s ctx 含排队时间,agent 串行处理下第 N 条必超时
  (实测第 3 条 62s 超时 504);放宽到 300s(客户端 abort 时立即取消)
- LLM 单 provider 瞬断无重试:process.go provider 循环内加同源
  重试(2 次、退避 2s),401/403 凭证错误与用户中断不重试
- llmsproxy auto 链在非流式请求下可能返回 SSE body(上游恢复后
  吐已生成的 chunk 流),非流式解析报 invalid character 'd' 丢掉
  整段回复;新增 parseOpenAICompatibleSSEBody 拼接为完整响应
- 顺带修 normalizeStreamToolCalls 分片续传 bug:name 不重发时
  argsRaw 被顶层 Arguments(nil) 覆盖丢失 function.arguments

验证:
- 连发 3 条 + 单条共 4 条全部成功(首条 190s 重试扛住瞬断)
- sse_body_test.go 锁定 SSE body 解析契约(content/usage/tool call 分片)
2026-08-25 12:24:11 +08:00
061d2ae320 feat(streaming): token-level delta events + interrupt for CLI/WebUI/GUI
Expose the LLM token-level streaming deltas (EventReasoningDelta /
EventContentDelta) to every client channel and add user-initiated
interrupt (cancel generation / send interrupt message) to all three
frontends, preserving the existing interrupt-injection semantics.

SDK/events:
  - EventReasoningDelta, EventContentDelta constants exported in the
    public/internal SDK event alias tables.

CLI plugin:
  - handleChat subscribes to both delta events and forwards
    reasoning_delta / content_delta JSON frames (channel-filtered);
    aggregated reasoning/tool_call/response frames still fire as before.
  - New /stop (alias /interrupt) builtin injects an interrupt via
    InjectInterrupt(cliSource, cliChannel) - matches interceptLoop
    semantics: cancels an active stream and re-injects the message as
    a [中断消息] for a restarted turn; with no active LLM it behaves
    as a plain input.

Waiter client (line mode + TUI):
  - streamRender accumulates delta chunks and redraws the current line;
    a reset frame (stream abandoned, e.g. user interrupt) flushes the
    partial buffer so the next turn does not concatenate onto stale
    content. Aggregated frames terminate the delta line and render the
    final text (old servers without deltas behave exactly as before).
  - TUI merges content_delta into the in-flight agent message and seals
    it (final flag) on response/tool_call/error so subsequent deltas
    never append to a finished message.

WebUI:
  - SSE handler subscribes to the two delta events but does NOT record
    them into the replay ring - reconnection replays only aggregated
    events (the final truth), avoiding duplicate delta accumulation.
  - POST /api/v1/chat/interrupt calls InjectInterrupt(webui, webui)
    with optional message; fronted by a Stop button shown only while
    a generation is in flight.

dashboard.html / GUI app.js:
  - Stop button next to Send (hidden until chatLoading); interruptChat
    POSTs /chat/interrupt. Delta listeners append incrementally;
    agent_output (aggregated) now replaces (not appends) the in-flight
    content and marks _final; reset frames finalize the partial message.

process.go:
  - chatStreamWithFallback preserves the context.Canceled/
    DeadlineExceeded contract: a user interrupt returns the canceled
    error (never a partial-content success) so the existing continue
    branch restarts the turn with the [中断消息]. A reset
    EventContentDelta is published so connected clients drop stale
    partial renderings before the new turn begins.

Verified: /stop 'msg' via waiter triggers 'interrupt from cli/cli' in
interceptLoop; unit TestChatStreamCancelPreservesInterrupt confirms the
canceled error propagates instead of being swallowed.
2026-08-25 10:50:37 +08:00
28a6d3f09c feat(agent): token-level streaming in core process loop
Replace the blocking Chat() call in process() with
chatStreamWithFallback: ChatStream first, accumulate chunks, fall back
to non-stream Chat on connect failure or empty-stream failure.

Why: the non-streaming path blocked for the ENTIRE LLM generation (up
to the 180s HTTP timeout). Reasoning models thinking 60-120s plus AUTO
chain failover regularly exceeded it -> context canceled -> full turn
wasted. With streaming the first chunk arrives in ~1-3s and any
flowing token keeps the connection alive; total generation time is no
longer bounded by an overall timeout.

Compatibility (external behavior unchanged):
  - process() signature/return values unchanged
  - Aggregated events (EventReasoning / EventAgentLLMChain) still fire
    once per turn with full text after stream completion - existing
    plugin subscribers see identical payloads as before
  - New incremental events EventReasoningDelta / EventContentDelta are
    additive; old subscribers ignore unknown event types
  - Tool execution loop, memory pipeline, stage pipeline untouched

Streaming details:
  - Tool call fragments accumulated per OpenAI streaming convention:
    id/name arrive on the first fragment, arguments as raw JSON string
    shards across fragments; merged and parsed once at stream end
  - normalizeStreamToolCalls keeps nameless argument shards (the
    non-stream normalizer drops them); ToolCall gains RawArguments to
    carry shard text
  - Interrupt mid-stream returns partial content instead of discarding
    the whole generation

Verified end-to-end against llmsproxy: plain chat streams correctly;
curl confirms tool-call shard wire format ({" + command" + :"date"}
-> {"command":"date"}); unit tests cover shard merging and
content/reasoning accumulation.
2026-08-25 09:30:49 +08:00
7d6c0bb90b feat(provider): complete ChatStream with llmsproxy-grade streaming
Rewrite LuaAdaptedProvider.ChatStream to match the maturity of
llmsproxy's streaming implementation:

HTTP layer:
  - Dedicated stream HTTP client with no overall timeout (SSE must not
    be cut by the 180s Chat timeout); only a 30s dial timeout
  - Uses applyAdapterHeaders (supports build_headers dynamic signing
    hook), matching the non-streaming Chat path

Non-200 response handling:
  - New TransformError Lua hook (adapter.transform_error) for per-source
    protocol knowledge in error messages
  - Safe fallback truncation of raw error bodies (prevents HTML dump
    leakage to clients)

SSE parsing enhancements:
  - parseOpenAICompatibleStreamChunkFull: handles token usage in the
    final chunk (prompt_tokens/prompt, total_tokens/total dual keys),
    prompt cache detail fields, and empty-string finish_reason filtering
    (sensenova sends "" on every chunk)
  - Replaced old SSEScanner with bufio.Scanner (larger buffer, fewer
    allocations)

Stream integrity:
  - errorOnlyChunk detection: holds back the first chunk to reject
    degenerate streams (e.g. zen free pool's finish_reason:"network_error"
    with empty content) before any byte reaches the caller
  - [DONE] dedup: adapters that already emit a terminating done chunk
    with the real finish_reason don't get a second reason-less done
  - Clean EOF sends a final Done:true if no done was seen

Struct changes:
  - StreamChunk: added FinishReason and Usage fields for callers
  - LuaAdaptedProvider: added streamClient (lazy) + streamMu

Tested: curl against llmsproxy SSE confirms reasoning_content parsing
is correct (delta.reasoning_content), usage chunk handling works, and
[DONE] termination is properly emitted.
2026-08-25 08:30:53 +08:00
d1e502d367 fix(agent): self-input channel carries target output channel flag
The selfInputCh previously treated ALL internal messages as memory
consolidation tasks (hardcoded _consolidation_ output channel), which
silently discarded child-agent completion notifications:

  - processConsolidation never appends to conversation context, so the
    parent agent could not see that its child had finished
  - it also discards the LLM response without emitting to any output
    channel, so nothing reached the user
  - net effect: notifications vanished; parent never called child_result

Restore the intended design: each self-input message now carries a
target output channel. Only consolidation tasks (_consolidation_) go
through the no-memory path (no context write, no emit). Child
notifications carry the parent's original output channel and are
processed as normal input: appended to context, LLM sees them and can
call child_result, and the response is emitted back to the user.

Changes:
  - new selfInputMsg{text, channel} type + channelConsolidation const
  - selfInputCh: chan string -> chan selfInputMsg
  - injectSelf (consolidation) keeps _consolidation_; new
    injectSelfChannel for flagged messages
  - handleSelfInput routes on msg.channel instead of hardcoding
  - executeSpawnChild captures a.currentOutputChannel and passes it to
    runChildTask so the notification returns to the originating channel
    (falls back to "cli" when unset or consolidation)
  - executeChildResultTool: remove dead double-lock/re-check block

Verified end-to-end with tmux PTY against llmsproxy:
spawn_child -> child done -> notification processed via normal path
(log shows 'input from system -> response, tools=[child_result]'),
parent agent retrieved the child result successfully.
2026-08-25 07:52:51 +08:00
dc0ba690c6 feat(waiter): Bubble Tea TUI modernization
Replace the line-based REPL with a full-screen Bubble Tea TUI in
interactive mode (non-TTY still falls back to the line editor).

Layout (deveco-code inspired, no emoji):
  - Top status bar: HomeAgent brand + connection dot + mode + addr
  - Scrollable chat viewport with role-based rendering:
      You (purple)   user messages
      小宅 (orange)  agent responses
      · reasoning    dim gray italic, streaming-merged
      [ok]/[fail]    tool calls with status + truncated result
      [sys]          builtin command output
      [err]          errors
  - Rounded-border input box with placeholder
  - Bottom status bar: spinner while busy / hints + connection state

Key design points:
  - reader generation counter prevents stale errors from the old
    reader being mistaken for the new one after reconnect
  - handleSubmit always returns waitServer when reader is alive,
    so server responses to builtin commands like /status are received
  - History stored as *History (was copying sync.Mutex by value)
  - Chinese CJK wide-char aware word wrap with hanging indent
  - /clear /exit /quit handled in TUI; other /cmds still go through
    handleBuiltin with output captured to message area

Verified via tmux PTY: /help, /status, /clear, real chat with LLM
(reasoning -> tool_call -> response chain all render correctly).
2026-08-25 01:01:22 +08:00
dev
22de000f23 fix: bump llm http client timeout 120s→180s for llmsproxy AUTO chain failover
The local llmsproxy AUTO chain tries 6+ slots across 3 tiers sequentially.
Each failed tier incurs busyWait (2s) + upstream timeout, so a full chain
exhaustion can exceed 120s. The llmsproxy logs showed 143 'context canceled'
errors for the homeagent key — the client gave up before the chain finished.

180s gives the chain enough room to complete before the client timeout fires.
Also remove stale backup files under /usr/local/bin/.
2026-08-25 00:34:25 +08:00
ece06b0375 feat(cli): streaming process output with npm-style spinner
CLI 对话现在像 npm 安装一样先显示 braille 加载动画,然后逐步吐出
推理内容和工具调用状态,最后输出最终响应。

协议扩展(JSON 行,向后兼容):
- {"type":"reasoning","content":...}   推理过程帧
- {"type":"tool_call","tool":...,"status":...,"result":...} 工具调用帧
- response / error 仍为终结帧,语义不变

服务端(internal/plugins/cli):
- handleChat: 通过 SDK 订阅 EventReasoning/EventToolCall(按 channel=="cli"
  过滤),InjectTextSync 阻塞期间实时转发事件到 socket;connWriter 互斥
  保护并发写。纯插件层实现,不触碰内核。
- 不订阅 EventAgentOutput:内核先写 ResponseCh 再 publish 该事件,
  订阅会导致响应重复。

客户端(cmd/waiter):
- startSpinner: npm 风格 braille 转圈(80ms),幂等 stop(),非 TTY 自动禁用
- SendChatStream: 循环读帧直至终结帧,onEvent 回调渲染过程帧
- printServerOutput: reasoning 灰色 · 前缀;tool_call ✔/✘ 状态行 + 结果预览
- 交互模式发送后自动起 spinner,首帧到达即停;oneshot 同理
- 向后兼容旧服务器(无类型行直接作为最终输出)

端到端验证:本地 homed 测试实例 + llmsproxy,oneshot 与交互模式均正确
渲染 推理→工具调用→最终响应 完整链路。

另外修正 dashboard.html renderReasoningCard 流式态使用 preview 结构
(与 GUI 渲染器一致,配合此前 renderChatStreamChunk 增量更新)。
2026-08-24 23:34:48 +08:00
121a2b9ace fix(webui): 聊天流式增量渲染(移植 GUI renderChatStreamChunk 方案)
问题:webui 流式推理阶段虽有 90ms 防抖,但每次防抖到期仍是
全量 innerHTML 重建,视觉上「一下渲染一大块」。

修复:移植 GUI 的增量渲染方案——
- rerenderChat() 在流式中(chatLoading && 最后一条未 _final)走
  增量路径:90ms 合并 chunk 后只调 renderChatStreamChunk()
- renderChatStreamChunk 仅更新最后一条消息节点:
  · 正文 >200字符 或 >300ms 才 renderMd(节流 markdown parse)
  · 小增量纯文本 createTextNode 追加,零 parse 开销
  · 思考预览只刷 .reasoning-preview 文本
  · 结构变化时兜底全量 renderChat()
- 工具卡/历史等非流式变化仍走全量路径
2026-08-24 22:40:48 +08:00
8157772132 fix(webui): sendChat 超时后 SSE 兜底渲染(对齐 GUI 模式)
问题:agent 长任务(159s)超过 15s ack 超时后,旧代码 return 导致
finally 清空 loading 并重置按钮,用户以为失败;SSE 回复虽到但
用户可能已离开/刷新页面。

修复(完全对齐 GUI renderer 的 sendChat 模式):
- 超时后 r=null 不 return,流程继续
- finally 总是清 loading + 恢复按钮(loading 只是 UI 提示)
- toast 明确提示「请求超时(可能已发送,请稍候勿重复发送)」
- r 有值则直接填充最终回复;r=null 则靠 SSE agent_output 流式渲染
2026-08-24 22:36:45 +08:00
e1a94fc896 fix(webui): 聊天渲染优化 + 触发式 POST
- 删除死的 tool_result SSE 监听器(后端不发布此事件类型)
- 流式渲染防抖 90ms:SSE 高频 chunk 合并为一次重建,流式期间跳过
  星图/终端/命令历史等无关渲染(renderChat 签名短路 + 防抖双重优化)
- sendChat 改触发式 POST:15s 短超时仅确认受理,超时后不报错,
  回复靠 SSE 流式渲染(对齐 GUI 行为,解决长 LLM 工具链 60s 超时报错)
- sendChat 用户操作走 rerenderChat(true) 全量重渲;SSE 流式走
  rerenderChat() 防抖仅聊天
2026-08-24 21:53:53 +08:00
ce53e8816b chore(webui): 清理前端死代码
- 删除 8 个未引用 JS 函数: confirmDialog/showToast/timeAgo/statCard/
  systemTheme/getStarmapBg/resetStarmapCamera/toggleStarmapAuto
  (实际使用的是 toast()/内联卡片构建/setTheme)
- 删除星图 hover info 空转块(starmap-info 容器不存在,infoEl 永远 null,
  sm-info-name/type/mentions/links 子元素查询全部无效)
- 删除死 CSS: #starmap-container/#starmap-stats/#starmap-info/
  #starmap-loading/.starmap-toggle(对应 HTML 元素已不存在)
- 删除重复的 #sm-container-chat height:480px(被后续 260px 覆盖)

共 -266 行,JS/CSS 语法验证通过
2026-08-24 21:01:06 +08:00
2ee007edc9 chore: 清理垃圾文件与构建产物
- 删除 internal/plugins/webui/dashboard2.html(遗留压缩实验代码,含乱码,零引用)
- 删除根目录旧二进制 homed/waiter(build/ 已有最新版)
- 删除 build/ 实验二进制 homed2/3/4/-audio/-img/-lua/-multi/-sched/_v2
- 删除 deploy/build 重复发布包 + dist 打包产物
- 删除 .gopath(旧 SDK v0.7.2 模块缓存,已 replace 到 third_party 本地副本)
2026-08-24 20:18:59 +08:00
ba5785036a feat: 设备鉴权迁移至客户端 + 插件卸载保护
安全修复(客户端鉴权):
- remotedevice 服务端移除授权状态存储(authorized map/SetAuthorized/handleDeviceAuth)
- DeviceMeta.Authorized 改为设备 hello 自报,服务端仅透传展示
- device_ctl_* 工具移除服务端授权检查,无条件转发,设备端自行决定是否执行
- 共享设备桥库 Bridge 新增本地 authorized 状态,未授权收到 cmd 直接拒绝
- waiter: --device-authorized / device_authorized 配置控制本地授权
- GUI: 授权存 gui-prefs 本地文件;设备页仅本机可切换开关
- webui /device/auth 旧路径返回 410 Gone
- 根因:agent 可经 config_set 篡改服务端授权配置自行授权设备

插件管理强化:
- 内置插件禁止卸载(IsBuiltinPlugin + 409),外部插件卸载即时生效
- 卸载不存在插件返回 404;移除误导性 reload_required 提示
- webui 插件路由:名称白名单校验防路径穿越、保留字路径保护
2026-08-24 19:26:11 +08:00
5163ce51a7 feat: SSE Last-Event-ID 断线重放 + GUI 表单防刷新
- webui handler: 新增 sseEventRing 环状缓冲区(200条),断线重连按 Last-Event-ID 重放遗漏事件
- GUI app.js: doRenderAll 检测连接表单打开时改走 refreshDataOnly,修复 15s 定时器擦掉用户输入的 bug
- 附带 dashboard.html/index.html 前端调整 + handler_sse_test.go 单测
2026-08-24 16:22:28 +08:00
fb8a1b8ce3 docs: README 添加 AI 辅助编程声明 2026-08-24 09:42:41 +08:00
a024dc3f5f feat: 设备桥共享库 + CLI全能力补齐 + GUI omniparse/computeruse 重构
- 抽取设备桥 WS 协议层为共享库 (internal/devicebridge/client/)
- CLI 补齐 11 项 caps 能力(screensee/screensue/speakeruse/camerasue/...)
- GUI 新增 omniparse 能力(Windows UIA 窗口解析)
- GUI computeruse 改用 koffi 直接调用 user32.dll,不再依赖 PowerShell C# 编译
- GUI computeruse JSON 解析兼容非标准格式 {x:500,y:300}
- 新增 mock-server 用于本地测试设备桥协议
- 新增 GUI DLL 桥接模块 (devicebridge_dll.js)
2026-08-23 20:17:38 +08:00
a014f449f6 gui: hello caps 补全 screensee/clipboardsee/clipboardsue 能力声明 2026-08-21 20:03:49 +08:00
4dcd3623cb feat: 设备能力矩阵校验 + 设备主动上报事件通道
回应架构讨论: 外接设备各自声明能力, 且补齐设备→agent 单向推送缺口。

1. 能力矩阵 (registry.go):
   - capabilityTools 映射: caps 声明 → 可用工具
     screen/screensue/screensee, computeruse, clipboard(see/sue),
     camera/camerasue, speaker/speakeruse
   - SupportsTool 校验: screensee/computeruse/clipboard* 执行前检查
     目标设备是否声明对应能力, 未声明直接报错(不再下发到设备端才失败)
   - 兼容规则: 声明 cmd/cmdrun 等历史值 → 全能力;
     未声明任何已知能力 → 全能力(旧设备兼容);
     有已知能力声明则严格匹配

2. 设备主动上报事件 (WS op=event):
   - 设备可推 {op:event, type, detail/payload} 无需回执
   - 插件层 SetEventHandler 回调: 格式化为人类可读文本,
     经 SDK InjectInput 异步注入 agent(source=device/{id}, 回复路由回同通道)
   - 同设备同类型事件 10s 节流防传感器风暴
   - 场景: 摄像头识别未知人员驻留主动告警, agent 收到后自主处置

测试: 能力矩阵8场景 + 摄像头调screensee被拒 + 事件上报回调(含device_id回退)
2026-08-21 19:58:37 +08:00
b6c1ef15d2 gui: 新增 clipboardsee/clipboardsue 剪切板读写能力
- clipboardsee: 读取设备剪切板文字(clipboard.readText), 回执 output 为文字, content=ok/empty
- clipboardsue: 写入文字到剪切板(clipboard.writeText), 回执含写入字节数+预览
- 与服务端 c0e92cc 配对, 支持 computeruse keypress ctrl+v 自动粘贴
2026-08-21 16:43:53 +08:00
d88b9b2bc3 gui: 新增 computeruse 能力(agent 跨平台控制鼠标键盘)
- executeHomeagentCmd 加 case computeruse: 参数 JSON {x,y,action,button,text,key}
- action: click/move/doubleclick/rightclick/scroll/keypress/type
- 跨平台: Linux=xdotool, macOS=cliclick(及osascript输入), Windows=PowerShell user32
- 坐标基于 screensueDisplay 目标屏幕原点偏移
- caps 增补 computeruse

已装本机 xdotool 备测
2026-08-21 16:40:18 +08:00
f70ac92d24 gui: 新增 screensee 能力(agent 查看远程设备屏幕)
- executeHomeagentCmd 加 case screensee: desktopCapturer 截屏 → toJPEG(80) → data:image/jpeg;base64 回执
- 屏幕选择沿用 gui-prefs.deviceBridge.screensueDisplay, 越界回退主屏
- 输出格式与 camerasue 抓拍一致, 服务端视觉模型可直接分析
- 空源/空缩略图/异常均有 error 回执

对应服务端 5b0cd45 screensee 工具, 已用 mock 网关端到端验证:
截屏 display 1 → jpeg 21049 字节 → status=ok
2026-08-21 16:40:18 +08:00
0393daa644 gui: speakeruse 支持声卡选择(设备通道可配输出设备)
- playDeviceAudio 按 gui-prefs.deviceBridge.audio.device 用 aplay -D <dev> 播放
- 新增 audio:list IPC(aplay -L 枚举) + preload 暴露 audio.list
- renderer 设备通道新增 speakeruse 声卡下拉, 保存到 prefs
- 排除 hw: 裸设备(mono->stereo 需转换), 只留 default/pipewire/pulse/plughw
- 修复 USB 声卡无声音: mono WAV 播到仅立体声设备失败, 改用 plughw 自动转换

已用 mock 网关 + 源码直跑验证: voice -> via plughw:CARD=Audio,DEV=0 播放成功
2026-08-21 16:40:18 +08:00
c0e92cceb1 feat: clipboardsee/clipboardsue 工具 — agent 读写远程设备剪切板
与 screensue/screensee 同模式配对命名:
- clipboardsee: 读取设备剪切板当前内容(用户最近复制的文字)
- clipboardsue: 写入文字到设备剪切板(用户可直接 Ctrl+V 粘贴)

设计要点:
- 协议: homeagent-clipboardsee / homeagent-clipboardsue <文字>
- clipboardsee 回执 output 字段为剪切板文字, 响应含 content/empty 字段
- clipboardsue 响应含 written 字节数 + preview 截断预览(60 rune)
- 隐私提示写入工具描述: 剪切板可能含密码, 仅在用户明确要求时读取
- 典型组合写入描述: clipboardsue + computeruse keypress ctrl+v 自动粘贴
- 未授权/离线/超时完整错误路径

GUI 配套需求(已发群): executeHomeagentCmd 加 case clipboardsee/clipboardsue,
Electron clipboard.readText()/writeText(text) 一行实现。

测试: 端到端读写+参数校验+未授权拒绝 全通过
2026-08-21 16:31:23 +08:00
cacd9a8572 feat: computeruse 工具 — agent 结构化操控远程鼠标/键盘
配合 GUI b4e5b39 (homeagent-computeruse 能力):
- 新增 computeruse 工具, LLM 传结构化参数(action/x/y/dy/key/text/button)
- 服务端序列化为 GUI 约定的 JSON 参数格式下发, 避免 LLM 手拼字符串出错
- action 校验: click/doubleclick/rightclick/move 需坐标; scroll 需 dy;
  keypress 需 key; type 需 text; 未知 action 报错
- 典型工作流写入工具描述: screensee 看屏 → computeruse 操作 → screensee 确认
- 未授权/离线完整错误路径; 结果留档 cmdresult

测试: 端到端验证 JSON 参数格式下发+回执+参数校验错误路径
2026-08-21 12:32:07 +08:00
cfe5a1a7f1 feat: screensue 默认 5s 超时,agent 可指定时长或永不超时
- GUI main.js: screensue 默认 duration 从 0(常驻) 改为 5 秒自动关闭
- 命令带数字 token 指定时长: "screensue 30 内容"=显示30秒
- 命令带 0 表示永不超时: "screensue 0 重要公告"=常驻直到用户手动关闭
- gui-prefs.screensueDuration 用户配置默认值(未配置时 5),命令参数优先级最高
- 设置页(app.js)新增「默认时长(秒)」输入框(placeholder 提示 0=常驻)
- 回执文案区分 for Ns / persistent until closed
- 服务端 device_ctl_cmdrun 工具描述同步更新(5秒默认/指定秒数/0永不超时)
2026-08-21 12:10:28 +08:00
5b0cd45093 feat: screensee 工具 — agent 查看远程设备屏幕内容
与 screensue(向用户屏幕显示)配对: screensue 是给用户看, screensee 是 agent 看。

服务端实现:
1. remotedevice 新增 screensee 工具:
   - 下发 homeagent-screensee 命令 → 设备截屏回传 jpeg base64
   - seeHandler 回调(agent 核心注入)用视觉模型自动描述屏幕内容
   - 未授权/离线/超时完整错误路径; 结果留档 cmdresult
2. SDK LLMMessage 扩展多模态 Blocks(text/image_url):
   - llm_impl 转换为 agentAPI.ContentBlock, 视觉模型可看图
3. describeScreen: 默认提示词描述窗口/文字/界面状态;
   provider 参数可指定视觉源(临时切换后恢复)

GUI 端需配套(已发群): onDeviceMsg 加 case "screensee",
desktopCapturer 截屏 → jpeg base64 data URL 回执(同 camerasue 抓拍模式)。

测试: 端到端模拟设备截屏回传+视觉回调验证; 全项目 go test 通过
2026-08-21 11:31:45 +08:00
f2e3215c77 feat: remotedevice 二进制分块协议(录像回传+音频下发) + 工具描述增强
回应 GUI c29abe9 需服务端配套项([阻塞]两项 + [中]一项):

1. readFrame 支持 0x2 二进制帧(之前遇 0x2 直接断连):
   - 返回 opcode, 二进制帧上限放宽至 8MB
2. handleWS 二进制聚合协议(设备→网关, 录像回传):
   - cmd_data_start 开启按 req_id 聚合 → 0x2 帧追加 → cmd_data_end 聚合完成
   - 结果存 cmdresult(data_base64 字段), device_ctl_cmdresult 可取回
   - 超限防护(声明 total×2 或硬上限 64MB)
3. PushData 下发协议(网关→设备, 音频/TTS):
   - cmd_speech_start → 0x2 分块(8KB) → cmd_speech_end
   - GUI 端已实现接收侧(speakeruse 播放链路打通)
4. device_ctl_cmdrun 工具描述补齐:
   - screensue 带参示例(<内容>/<秒> <内容>)
   - camerasue 录像说明(cmdresult 含 data_base64)
   - speakeruse 文字朗读

测试: 最小 WS 客户端端到端验证二进制分块上传/PushData 下发/离线报错
2026-08-21 10:53:38 +08:00
8d368913a9 feat: webui 消息重放双重防护(client_msg_id 去重 + agent 内容级去重)
回应群聊 08-19 消息轰炸诊断(GUI SSE 重连导致消息重放):

1. webui 层 client_msg_id 单飞去重(与 GUI c29abe9 配套):
   - /api/v1/chat 解析 client_msg_id, 同 ID 重放等待首次结果直接复用
   - 响应带 deduplicated=true 标记; 无 ID 旧客户端完全兼容
   - FIFO 缓存上限 256 条防泄漏

2. agent 核心层内容级短窗口去重(兜底无 ID 客户端):
   - isDuplicateInput: source+content 为 key, 10s 窗口内重复丢弃
   - 持续轰炸时刷新时间戳保持拦截; 过期项自动清理

测试: webui 去重三场景 + agent 核心去重行为验证, 全项目 go test 通过
2026-08-21 10:23:38 +08:00
257ff0ad5d gui: 自动重登/定时授权/副屏修复 + 排障(大量JS报错根因是cookie过期)
排障: "大量JS报错"真实根因 = webui cookie 会话24h过期 → 所有 api() 请求经网关 302 → 返回登录页HTML(200非401) → renderer 拿HTML当JSON解析 → 界面持续报错/乱码

修复(api() 自动重登):
- 401 或检测登录页HTML(THEME_PLACEHOLDER/统一门户登录)时自动 syncConnAuth 重登后重试一次
- 防递归锁 _haReloginLock
- JSON解析容错(HTML不再误报)

定时撤销/恢复授权:
- prefs.deviceBridge.authSchedule {enabled, revokeTime, restoreTime}, 支持跨天(23:00-07:00)
- 主进程每分钟检查, 到点经 webui 反代 /api/v1/device/auth 撤销/恢复(防抖状态机)
- 设备页UI: 开启开关 + 撤销/恢复时间输入(保存并应用生效)
- setDeviceAuthorized 优先 authRule, 否则从 connections.json 取 URL+cookie

副屏修复:
- displays:list 对空 label(' ') trim 后显示 显示器N
- 修复 loadGuiPrefs 只返回3字段导致 screensueDisplay/exec/authSchedule 被抹掉(保存后变默认的根因)

其他:
- 移除远程调试端口(9223, 排障用)
- 保留 window.onerror/onunhandledrejection 透传(gui.log 可见 JS 错误, 便于运维)
2026-08-20 13:29:21 +08:00
c29abe9569 gui: 设备命令/能力完整实现 + 二进制分块 + 消息ID去重 + 交互优化
设备命令执行对齐服务端 cmd_type 契约:
- onDeviceMsg 解析 msg.cmd_type: homeagent->能力分发 / shell->白名单执行
- 统一回执 sendCmdResult(兼容双参), 全链路回执修复
- 修复 pi-lens auto-fix 把 trayLastCmd 改 const 导致的 TypeError(命令未执行/无回执根因)

homeagent 能力:
- screensue: 独立窗口显示文字/HTML, 可配默认屏幕(displays:list IPC), 支持时长参数 "screensue [秒] 内容", 默认常驻
- camerasue: 抓拍单张 jpeg(base64文本); camerasue <N秒>=录像 mp4(libx264), 二进制分块经设备通道回传
- speakeruse: 接收服务端二进制音频(WS 0x2), 聚合后播放(aplay/paplay/ffplay/afplay/SoundPlayer)

二进制分块协议(设备<->网关):
- 录像(设备->网关): cmd_data_start/0x2帧/cmd_data_end
- 音频(网关->设备): cmd_speech_start/0x2帧/cmd_speech_end
- 设备端 WS 读写侧都支持 0x2 帧; N/A 服务端 readFrame 需配套(已发群)

消息重放/交互:
- sendChat 带 client_msg_id 唯一ID(服务端可去重), 超时明确提示勿重复发送
- 托盘菜单: 信息项 enabled(不再灰字)+200ms防抖+连接缓存, 首建立即弹出
- 托盘显示连接/设备桥/远控活动状态

UI:
- 设备页本机卡片: 设备通道配置(网关/ws_token/screensue屏幕/cmdrun目录/沙箱)
- 设备列表经 webui 反代 /api/v1/device/online 拉取
- 连接管理移除 device 类型(设备独立为 GUI 组件), toggleConnType/saveConnForm 清理
- 字体对比度提升(text-muted/secondary重调色), 内联小字 10/11px->12/13px

preload: 新增 displays.list IPC
2026-08-20 09:07:41 +08:00
768c73889e feat: webui chat 接口支持设备身份(device_id/device_name)
- handleChat body 增加 device_id/device_name 可选字段
- 来源编码: 带设备时 webui/{device_id}(agent 可见来源), 无设备保持 webui(兼容)
- injectSourceContext: 检测 device_id 时注入「当前输入来自设备[名](id)」上下文
- 验证: 带 device_id=gui-pc-002 消息 agent 正确回答来源客厅电脑; 无 device_id 兼容
2026-08-18 11:49:57 +08:00
55 changed files with 10048 additions and 1327 deletions

1
.gitignore vendored
View File

@ -45,3 +45,4 @@ terminal_locked_log.txt
dist/
.pi-glla/
.omo/
.pi/

View File

@ -1,3 +1,5 @@
> ⚠️ **AI 辅助编程声明**:本项目代码、文档及提交历史中,部分内容由 AI 辅助生成或修改。人工已审阅关键改动,但使用时请自行评估与验证。
# HomeAgent
> **English**: [README_EN.md](./README_EN.md)

View File

@ -1,3 +1,5 @@
> ⚠️ **AI-Assisted Programming Notice**: Parts of this project's code, documentation, and commit history were generated or modified with AI assistance. Key changes have been human-reviewed, but please evaluate and verify before use.
# HomeAgent
> **中文**: [README.md](./README.md)

126
cmd/gui/devicebridge_dll.js Normal file
View File

@ -0,0 +1,126 @@
// DeviceBridge DLL 桥接模块
// 提供设备桥共享库的 Node.js 封装GUI 通过 FFI 调用 Go 编译的 DLL。
// 优先尝试加载 DLL失败则回退到纯 JS 实现(保留兼容)。
const path = require('path');
const os = require('os');
let koffi = null;
let bridgeLib = null;
let _handle = null;
// DLL 路径
function dllPath() {
const dir = __dirname;
const plat = os.platform();
if (plat === 'win32') {
return path.join(dir, 'devicebridge.dll');
}
// Linux/Mac 使用 .so/.dylib
const ext = plat === 'darwin' ? 'dylib' : 'so';
return path.join(dir, `devicebridge.${ext}`);
}
// 尝试加载 FFI 库
async function loadFFI() {
try {
koffi = require('koffi');
return true;
} catch (e) {
try {
const ffi = require('ffi-napi');
const ref = require('ref-napi');
// 使用 ffi-napi 作为备选
return true;
} catch (e2) {
return false;
}
}
}
// 加载 DLL
function loadDLL() {
const dll = dllPath();
try {
if (koffi) {
return koffi.load(dll);
}
const ffi = require('ffi-napi');
const ref = require('ref-napi');
return ffi.Library(dll, {
'devicebridge_new': ['pointer', ['string', 'string', 'string', 'string', 'pointer', 'int']],
'devicebridge_start': ['int', ['pointer']],
'devicebridge_stop': ['void', ['pointer']],
'devicebridge_free': ['void', ['pointer']],
'devicebridge_connected': ['int', ['pointer']],
'devicebridge_device_id': ['string', ['pointer']],
'devicebridge_send_result': ['int', ['pointer', 'string', 'string', 'string', 'string']],
'devicebridge_send_event': ['void', ['pointer', 'string', 'string']],
'devicebridge_send_status': ['void', ['pointer', 'string']],
'devicebridge_send_data_start': ['void', ['pointer', 'string', 'string', 'string', 'int']],
'devicebridge_send_data_chunk': ['int', ['pointer', 'pointer', 'int']],
'devicebridge_send_data_end': ['void', ['pointer', 'string', 'string', 'string']],
});
} catch (e) {
console.error('[devicebridge-dll] load failed:', e.message);
return null;
}
}
// 设备桥封装
class DeviceBridgeDLL {
constructor() {
this.connected = false;
this.deviceId = '';
this._onCmd = null;
this._onData = null;
}
// 初始化并连接
async start(gateway, token, deviceId, deviceName, caps, info) {
bridgeLib = loadDLL();
if (!bridgeLib) {
throw new Error('DLL not loaded');
}
// 构建 caps 数组
const capsArr = caps.map(c => Buffer.from(c + '\0'));
const capsPtr = Buffer.alloc(8 * capsArr.length);
// 简化:实际 FFI 调用需要更复杂的参数处理
// 这里使用 koffi 方式
if (koffi) {
// 使用 koffi 调用
try {
// TODO: 实现 koffi 调用
throw new Error('koffi not fully implemented');
} catch (e) {
throw e;
}
}
throw new Error('FFI library not available. Install koffi or ffi-napi');
}
stop() {
if (bridgeLib && _handle) {
try {
bridgeLib.devicebridge_stop(_handle);
bridgeLib.devicebridge_free(_handle);
} catch (e) {}
_handle = null;
this.connected = false;
}
}
sendResult(reqId, status, output, error) {
if (!bridgeLib || !_handle) return;
try {
bridgeLib.devicebridge_send_result(_handle, reqId, status, output || '', error || '');
} catch (e) {
console.error('[devicebridge-dll] sendResult error:', e);
}
}
}
module.exports = { DeviceBridgeDLL, loadFFI };

File diff suppressed because it is too large Load Diff

View File

@ -7,6 +7,9 @@
"": {
"name": "homeagent-gui",
"version": "1.0.0",
"dependencies": {
"koffi": "^3.1.6"
},
"devDependencies": {
"asar": "^3.2.0",
"electron": "^33.0.0",
@ -936,6 +939,246 @@
"node": ">=18.0.0"
}
},
"node_modules/@koromix/koffi-darwin-arm64": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-darwin-arm64/-/koffi-darwin-arm64-3.1.6.tgz",
"integrity": "sha512-8FHyXGCZN7/iQf4f7W5BRysmtdlAFvSx6FpmX4u6wmkZiX/2e9hIRdGLiZYlHGudlcA18UmXB/cMiyhJ7fJkzA==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@koromix/koffi-darwin-x64": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-darwin-x64/-/koffi-darwin-x64-3.1.6.tgz",
"integrity": "sha512-uzx/jqFQuSHqgg1zaRidTBTCfj8Y9M0SDTO8HeoI9s9fJhiJ1mbB9TTwJO5c2hiMnuWg2m1byczC8BaIH6cG/w==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@koromix/koffi-freebsd-arm64": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-freebsd-arm64/-/koffi-freebsd-arm64-3.1.6.tgz",
"integrity": "sha512-PjpTVrsCK5YTtixOw7VsseYXJOyoY6k0qBt+bf0T9h3wyV06y73rALsorFDDEoYpLUBZO7R6EIMs6CpUrEkNTQ==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@koromix/koffi-freebsd-ia32": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-freebsd-ia32/-/koffi-freebsd-ia32-3.1.6.tgz",
"integrity": "sha512-ETYwL820HtFwYoOVzgyvmFmzTHRo9DJtGYTxa5Nb7ajaa5ldCum0jbmUJ3PMECxFTLxD5Q6PYZ8Xbp101bGeSg==",
"cpu": [
"ia32"
],
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@koromix/koffi-freebsd-x64": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-freebsd-x64/-/koffi-freebsd-x64-3.1.6.tgz",
"integrity": "sha512-BkqxkNXhAAT9toU2stvLwx1iKHPDx7h08NCICyBbjYEXkCAEr84igTkpE5V3XJ9xZZ2gKll7VvdhxorHtHUqZw==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@koromix/koffi-linux-arm64": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-linux-arm64/-/koffi-linux-arm64-3.1.6.tgz",
"integrity": "sha512-cM4XPm9ljbCrcPgXjzFYjDNxDUvvuR7TCYaEoo1AKjwZT/vmWhu2xN3pomfbsHh6aVn80SFA3enufQnaETW1rQ==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@koromix/koffi-linux-ia32": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-linux-ia32/-/koffi-linux-ia32-3.1.6.tgz",
"integrity": "sha512-l1SVTpO10iaQt8slbowJpzK4fbwQZ7ufj9tmCyAcIwWUpyAbPS83mJMctU72If6N9/gCS2wuRqwnYB2uPLLhLg==",
"cpu": [
"ia32"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@koromix/koffi-linux-loong64": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-linux-loong64/-/koffi-linux-loong64-3.1.6.tgz",
"integrity": "sha512-KpTJpMSbIdCVFU26ynt0xy4x15h+y6AwPJxj2+iVxJhCzJf4oisCPc0YH2VnutuLV2nVzSrFm7sL/WSOqPgkXw==",
"cpu": [
"loong64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@koromix/koffi-linux-riscv64": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-linux-riscv64/-/koffi-linux-riscv64-3.1.6.tgz",
"integrity": "sha512-YdFNpsywnXiYOYQlDAatf7TJLnspbGXdmfwIZhf82kbKSflYUsq4tI6NmoUOzM89oIWDGpYqd4Hz3xL7tsyXsw==",
"cpu": [
"riscv64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@koromix/koffi-linux-x64": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-linux-x64/-/koffi-linux-x64-3.1.6.tgz",
"integrity": "sha512-Xx5mpr9VcaMCXfvbqIiLIWIL9Iuu6F4r3iMXg7+zZCqYUFZPFwJgiDQBLxctHv2OYgIfAoaaHMW0GC1cJkHfbA==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@koromix/koffi-openbsd-ia32": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-openbsd-ia32/-/koffi-openbsd-ia32-3.1.6.tgz",
"integrity": "sha512-39Np4QTxhTlTT6RRveIeP+TnbzrwuDJ0UMyHxEZ+oGtzmb9GqWhl9T1oyehG6v/O+c4BffafG2NwLkCZ7tDKWw==",
"cpu": [
"ia32"
],
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@koromix/koffi-openbsd-x64": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-openbsd-x64/-/koffi-openbsd-x64-3.1.6.tgz",
"integrity": "sha512-3EynGn3ycQRqaMWGmUJ0tdtuQdStByqSy/tJ0ZGKWizbMGdFAE73YpgLsyd8BDvwnKWytVG/OLNb5nDHpfd9Dg==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@koromix/koffi-win32-arm64": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-win32-arm64/-/koffi-win32-arm64-3.1.6.tgz",
"integrity": "sha512-27FdPPRtT4xbO9bsd2OZa95M5YQ7bcJ8QjCRO57UUMI21REfkDegjqKwqo/CFlugxXlJf5IYtG2rq4BEYIrvxg==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@koromix/koffi-win32-ia32": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-win32-ia32/-/koffi-win32-ia32-3.1.6.tgz",
"integrity": "sha512-5mVelLKVDup4eoxZOpCzCyMPxoctsg+Qe4J9O5BP4KbBEdqoOEqaNEBBRgNzXcdr2g+GGfmIUo+oVR1NvIdiJw==",
"cpu": [
"ia32"
],
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@koromix/koffi-win32-x64": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/@koromix/koffi-win32-x64/-/koffi-win32-x64-3.1.6.tgz",
"integrity": "sha512-lPKjAaHz0aoiZXT/wDVqH+joR5y3lCZj1s9Bk5qx/DGRq+0MK8Ib8VoqiBOrJ69NG5AsJSvn0tQDcSqfRgLmBQ==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"funding": {
"url": "https://liberapay.com/Koromix"
}
},
"node_modules/@malept/cross-spawn-promise": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/@malept/cross-spawn-promise/-/cross-spawn-promise-1.1.1.tgz",
@ -3764,6 +4007,33 @@
"json-buffer": "3.0.1"
}
},
"node_modules/koffi": {
"version": "3.1.6",
"resolved": "https://registry.npmmirror.com/koffi/-/koffi-3.1.6.tgz",
"integrity": "sha512-ln60chEb3o7Du1ayjwl6BFiNN1wZK+3cTM2wWGiHLEzCY/FdTIN1ER5VWDwHq7J/j4tSnnrHaH5ABS1EO6+6ag==",
"hasInstallScript": true,
"license": "MIT",
"funding": {
"url": "https://liberapay.com/Koromix"
},
"optionalDependencies": {
"@koromix/koffi-darwin-arm64": "3.1.6",
"@koromix/koffi-darwin-x64": "3.1.6",
"@koromix/koffi-freebsd-arm64": "3.1.6",
"@koromix/koffi-freebsd-ia32": "3.1.6",
"@koromix/koffi-freebsd-x64": "3.1.6",
"@koromix/koffi-linux-arm64": "3.1.6",
"@koromix/koffi-linux-ia32": "3.1.6",
"@koromix/koffi-linux-loong64": "3.1.6",
"@koromix/koffi-linux-riscv64": "3.1.6",
"@koromix/koffi-linux-x64": "3.1.6",
"@koromix/koffi-openbsd-ia32": "3.1.6",
"@koromix/koffi-openbsd-x64": "3.1.6",
"@koromix/koffi-win32-arm64": "3.1.6",
"@koromix/koffi-win32-ia32": "3.1.6",
"@koromix/koffi-win32-x64": "3.1.6"
}
},
"node_modules/lazy-val": {
"version": "1.0.5",
"resolved": "https://registry.npmjs.org/lazy-val/-/lazy-val-1.0.5.tgz",

View File

@ -10,6 +10,7 @@
"dev": "electron . --no-sandbox --dev"
},
"dependencies": {
"koffi": "^3.1.6"
},
"devDependencies": {
"asar": "^3.2.0",

View File

@ -44,5 +44,13 @@ contextBridge.exposeInMainWorld("homeagent", {
deviceBridge: {
get: () => ipcRenderer.invoke("device-bridge:get"),
set: (cfg) => ipcRenderer.invoke("device-bridge:set", cfg),
setAuthorized: (auth) =>
ipcRenderer.invoke("device-bridge:setAuthorized", auth),
},
displays: {
list: () => ipcRenderer.invoke("displays:list"),
},
audio: {
list: () => ipcRenderer.invoke("audio:list"),
},
});

File diff suppressed because it is too large Load Diff

View File

@ -18,6 +18,10 @@
src="https://cdnjs.cloudflare.com/ajax/libs/marked/4.3.0/marked.min.js"
onerror="console.warn('marked CDN failed')"
></script>
<script
src="https://cdn.jsdelivr.net/npm/dompurify@3.2.4/dist/purify.min.js"
onerror="console.warn('DOMPurify CDN failed')"
></script>
<script>
setTimeout(function () {
if (!window.THREE) window._THREE_FAILED = true;

View File

@ -45,8 +45,8 @@
--bg-input: rgba(13, 18, 34, 0.75);
--bg-hover: rgba(255, 255, 255, 0.06);
--text-primary: #eef1f8;
--text-secondary: #a7b0c4;
--text-muted: #77809a;
--text-secondary: #b8c1d6;
--text-muted: #93a0b8;
--border-color: rgba(255, 255, 255, 0.09);
--accent: #ff7fac;
--accent-bg: rgba(255, 127, 172, 0.14);
@ -98,8 +98,8 @@
--bg-input: rgba(255, 224, 233, 0.55);
--bg-hover: rgba(255, 127, 172, 0.08);
--text-primary: #3b2030;
--text-secondary: #7a5c6b;
--text-muted: #a48a96;
--text-secondary: #6b4b5c;
--text-muted: #8f6f7d;
--border-color: rgba(201, 36, 98, 0.14);
--accent: #c92462;
--accent-bg: #ffe4e9;
@ -290,7 +290,7 @@ body {
transition:
background 0.2s,
color 0.2s;
font-size: 14px;
font-size: 15px;
-webkit-font-smoothing: antialiased;
text-rendering: optimizeLegibility;
user-select: none;
@ -707,13 +707,13 @@ body.maximized .tb-max svg {
transform: translateY(-1px);
}
.card h2 {
font-size: 15px;
font-size: 16px;
font-weight: 600;
margin-bottom: 12px;
color: var(--text-primary);
}
.card h3 {
font-size: 13px;
font-size: 14px;
font-weight: 600;
color: var(--text-secondary);
margin: 16px 0 8px;
@ -901,7 +901,7 @@ select {
border-radius: var(--radius-sm);
padding: 8px 12px;
color: var(--text-primary);
font-size: 13px;
font-size: 14px;
width: 100%;
margin-bottom: 10px;
outline: none;
@ -923,9 +923,9 @@ textarea {
}
label {
display: block;
font-size: 11px;
font-size: 13px;
color: var(--text-secondary);
margin-bottom: 3px;
margin-bottom: 4px;
font-weight: 500;
}
pre {
@ -1597,10 +1597,10 @@ code {
margin-bottom: 16px;
}
.settings-tabs span {
padding: 6px 14px;
font-size: 13px;
padding: 7px 16px;
font-size: 14px;
cursor: pointer;
color: var(--text-muted);
color: var(--text-secondary);
border-radius: var(--radius-pill);
border: 1px solid transparent;
transition: all 0.15s;
@ -1619,18 +1619,18 @@ code {
}
.settings-key {
font-family: var(--font-mono);
font-size: 11px;
color: var(--text-muted);
margin-bottom: 2px;
font-size: 13px;
color: var(--text-secondary);
margin-bottom: 3px;
}
.kv-row {
display: flex;
padding: 6px 0;
padding: 7px 0;
border-bottom: 1px solid var(--kv-border);
font-size: 13px;
font-size: 14px;
}
.kv-row .key {
color: var(--text-muted);
color: var(--text-secondary);
width: 180px;
flex-shrink: 0;
}
@ -1716,8 +1716,8 @@ code {
vertical-align: 1px;
}
.conn-item .conn-url {
font-size: 11px;
color: var(--text-muted);
font-size: 12px;
color: var(--text-secondary);
margin-top: 2px;
}
.conn-item .conn-actions {

521
cmd/mock-server/main.go Normal file
View File

@ -0,0 +1,521 @@
package main
import (
"encoding/json"
"fmt"
"log"
"net/http"
"strings"
"sync"
"time"
)
// ===== 模拟数据 =====
type Status struct {
Status string `json:"status"`
Version string `json:"version"`
StartedAt string `json:"startedAt"`
Uptime int64 `json:"uptime"`
}
type Kernel struct {
Model string `json:"model"`
Provider string `json:"provider"`
Status string `json:"status"`
}
type Setting struct {
Settings map[string]interface{} `json:"settings"`
Meta map[string]interface{} `json:"meta"`
Plugins []string `json:"plugins"`
PluginMeta map[string]interface{} `json:"plugin_meta"`
DisabledPlugins []string `json:"disabled_plugins"`
}
type Plugin struct {
Name string `json:"name"`
Description string `json:"description"`
Version string `json:"version"`
Enabled bool `json:"enabled"`
Builtin bool `json:"builtin"`
}
type PluginInfo struct {
Name string `json:"name"`
Description string `json:"description"`
Version string `json:"version"`
Enabled bool `json:"enabled"`
Builtin bool `json:"builtin"`
Tools []PluginTool `json:"tools"`
}
type PluginTool struct {
Name string `json:"name"`
Description string `json:"description"`
}
type Adapter struct {
Name string `json:"name"`
Type string `json:"type"`
Enabled bool `json:"enabled"`
}
type ChatMessage struct {
Role string `json:"role"`
Content string `json:"content"`
}
type MemoryItem struct {
ID string `json:"id"`
Content string `json:"content"`
Time string `json:"time"`
}
type Device struct {
DeviceID string `json:"device_id"`
Name string `json:"name"`
Authorized bool `json:"authorized"`
Online bool `json:"online"`
Caps []string `json:"caps"`
}
// ===== SSE 管理器 =====
type SSEManager struct {
mu sync.RWMutex
clients map[chan string]bool
}
func NewSSEManager() *SSEManager {
return &SSEManager{clients: make(map[chan string]bool)}
}
func (m *SSEManager) Add(ch chan string) {
m.mu.Lock()
m.clients[ch] = true
m.mu.Unlock()
}
func (m *SSEManager) Remove(ch chan string) {
m.mu.Lock()
delete(m.clients, ch)
m.mu.Unlock()
}
func (m *SSEManager) Broadcast(eventType, data string) {
msg := fmt.Sprintf("event: %s\ndata: %s\n\n", eventType, data)
m.mu.RLock()
defer m.mu.RUnlock()
for ch := range m.clients {
select {
case ch <- msg:
default:
}
}
}
// ===== HTTP 处理器 =====
type MockServer struct {
startedAt time.Time
sse *SSEManager
mu sync.Mutex
plugins []Plugin
adapters []Adapter
devices []Device
settings map[string]interface{}
}
func NewMockServer() *MockServer {
now := time.Now()
return &MockServer{
startedAt: now,
sse: NewSSEManager(),
plugins: []Plugin{
{Name: "core", Description: "核心插件", Version: "1.0.0", Enabled: true, Builtin: true},
{Name: "remotedevice", Description: "远程设备管理", Version: "0.9.0", Enabled: true, Builtin: true},
{Name: "webui", Description: "Web 用户界面", Version: "0.9.0", Enabled: true, Builtin: true},
{Name: "knowledge", Description: "知识库管理", Version: "0.5.0", Enabled: true, Builtin: false},
},
adapters: []Adapter{
{Name: "openai", Type: "llm", Enabled: true},
{Name: "siliconflow", Type: "llm", Enabled: true},
},
devices: []Device{
{DeviceID: "gui-test-local", Name: "GUI 测试设备", Authorized: true, Online: true, Caps: []string{"status", "cmdrun", "deviceinfo"}},
},
settings: map[string]interface{}{
"language": "zh-CN",
"theme": "dark",
},
}
}
// 中间件CORS + API Key 校验
func (s *MockServer) middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS")
w.Header().Set("Access-Control-Allow-Headers", "Content-Type, X-API-Key, Authorization, Cookie")
if r.Method == "OPTIONS" {
w.WriteHeader(200)
return
}
// API Key 校验(可选)
// apiKey := r.Header.Get("X-API-Key")
// if apiKey == "" {
// http.Error(w, "unauthorized", 401)
// return
// }
next.ServeHTTP(w, r)
})
}
func (s *MockServer) handleStatus(w http.ResponseWriter, r *http.Request) {
writeJSON(w, Status{
Status: "running",
Version: "0.9.0",
StartedAt: s.startedAt.Format(time.RFC3339),
Uptime: int64(time.Since(s.startedAt).Seconds()),
})
}
func (s *MockServer) handleKernel(w http.ResponseWriter, r *http.Request) {
writeJSON(w, Kernel{
Model: "sensenova-6.8-flash-lite",
Provider: "siliconflow",
Status: "ready",
})
}
func (s *MockServer) handleSettings(w http.ResponseWriter, r *http.Request) {
if r.Method == "POST" {
var updates map[string]interface{}
if err := json.NewDecoder(r.Body).Decode(&updates); err == nil {
s.mu.Lock()
for k, v := range updates {
s.settings[k] = v
}
s.mu.Unlock()
}
writeJSON(w, map[string]string{"status": "saved"})
return
}
writeJSON(w, Setting{
Settings: s.settings,
Meta: map[string]interface{}{
"version": "0.9.0",
"build": "mock-20260823",
},
Plugins: []string{"core", "remotedevice", "webui", "knowledge"},
PluginMeta: map[string]interface{}{
"core": map[string]interface{}{"version": "1.0.0"},
"remotedevice": map[string]interface{}{"version": "0.9.0"},
"webui": map[string]interface{}{"version": "0.9.0"},
"knowledge": map[string]interface{}{"version": "0.5.0"},
},
DisabledPlugins: []string{},
})
}
func (s *MockServer) handlePlugins(w http.ResponseWriter, r *http.Request) {
// 获取路径中的插件名
path := strings.TrimPrefix(r.URL.Path, "/api/v1/plugins")
path = strings.TrimSuffix(path, "/")
if path == "/reload" && r.Method == "POST" {
writeJSON(w, map[string]string{"status": "reloaded"})
return
}
if path == "" && r.Method == "GET" {
writeJSON(w, s.plugins)
return
}
if path == "" && r.Method == "POST" {
writeJSON(w, map[string]string{"status": "installed"})
return
}
// /api/v1/plugins/:name
if strings.Contains(path, "/") {
parts := strings.Split(strings.TrimPrefix(path, "/"), "/")
if len(parts) >= 1 {
name := parts[0]
if len(parts) >= 2 {
action := parts[1]
if action == "disable" && r.Method == "POST" {
s.mu.Lock()
for i := range s.plugins {
if s.plugins[i].Name == name {
s.plugins[i].Enabled = false
}
}
s.mu.Unlock()
writeJSON(w, map[string]string{"status": "disabled"})
return
}
if action == "enable" && r.Method == "POST" {
s.mu.Lock()
for i := range s.plugins {
if s.plugins[i].Name == name {
s.plugins[i].Enabled = true
}
}
s.mu.Unlock()
writeJSON(w, map[string]string{"status": "enabled"})
return
}
}
// GET /api/v1/plugins/:name
writeJSON(w, PluginInfo{
Name: name,
Description: name + " 插件描述",
Version: "0.9.0",
Enabled: true,
Builtin: true,
Tools: []PluginTool{
{Name: name + "_tool1", Description: name + " 工具1"},
{Name: name + "_tool2", Description: name + " 工具2"},
},
})
return
}
}
http.NotFound(w, r)
}
func (s *MockServer) handleChatHistory(w http.ResponseWriter, r *http.Request) {
writeJSON(w, []ChatMessage{
{Role: "user", Content: "你好"},
{Role: "assistant", Content: "你好!我是 HomeAgent有什么可以帮你的"},
{Role: "user", Content: "测试消息"},
{Role: "assistant", Content: "这是模拟后端的测试回复GUI 连接正常 ✅"},
})
}
func (s *MockServer) handleChat(w http.ResponseWriter, r *http.Request) {
if r.Method == "POST" {
// 模拟后端接收消息,通过 SSE 推流
go func() {
time.Sleep(500 * time.Millisecond)
// agent_start
s.sse.Broadcast("agent_output", `{"type":"agent_start","payload":{"agent":"mock"}}`)
time.Sleep(300 * time.Millisecond)
// tool_call
s.sse.Broadcast("agent_output", `{"type":"tool_call","payload":{"tool":"mock_tool","args":{},"id":"call_001"}}`)
time.Sleep(500 * time.Millisecond)
// channel_output
s.sse.Broadcast("agent_output", `{"type":"channel_output","payload":{"kind":"channel_output","channel":"mock","content":"这是一条来自模拟后端的测试回复。\n\n- 模拟后端状态: running\n- 版本: 0.9.0\n- 连接测试: ✅ 成功\n\nGUI 所有功能验证正常!"}}`)
time.Sleep(300 * time.Millisecond)
// agent_end
s.sse.Broadcast("agent_output", `{"type":"agent_end","payload":{"agent":"mock"}}`)
}()
writeJSON(w, map[string]string{"status": "queued", "id": "mock_" + time.Now().Format("150405")})
return
}
http.Error(w, "method not allowed", 405)
}
func (s *MockServer) handleChatEvents(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
ch := make(chan string, 100)
s.sse.Add(ch)
defer s.sse.Remove(ch)
// 发送初始连接成功事件
fmt.Fprintf(w, "event: connected\ndata: {\"status\":\"connected\"}\n\n")
w.(http.Flusher).Flush()
ctx := r.Context()
for {
select {
case <-ctx.Done():
return
case msg := <-ch:
fmt.Fprint(w, msg)
w.(http.Flusher).Flush()
}
}
}
func (s *MockServer) handleMemoryGraph(w http.ResponseWriter, r *http.Request) {
writeJSON(w, map[string]interface{}{
"nodes": []map[string]interface{}{
{"id": "1", "label": "HomeAgent", "group": "system"},
{"id": "2", "label": "GUI 测试", "group": "user"},
},
"edges": []map[string]interface{}{
{"from": "1", "to": "2", "label": "connected"},
},
})
}
func (s *MockServer) handleMemory(w http.ResponseWriter, r *http.Request) {
writeJSON(w, []MemoryItem{
{ID: "m1", Content: "这是模拟内存中的测试数据", Time: time.Now().Format(time.RFC3339)},
})
}
func (s *MockServer) handleMemoryContext(w http.ResponseWriter, r *http.Request) {
writeJSON(w, map[string]interface{}{
"context": "模拟上下文:用户正在测试 GUI 功能",
"items": []MemoryItem{},
})
}
func (s *MockServer) handleKnowledge(w http.ResponseWriter, r *http.Request) {
if r.Method == "POST" {
writeJSON(w, map[string]string{"status": "saved"})
return
}
writeJSON(w, []map[string]interface{}{
{"id": "k1", "title": "模拟知识条目1", "content": "这是模拟知识库的测试内容"},
{"id": "k2", "title": "模拟知识条目2", "content": "GUI 功能验证测试数据"},
})
}
func (s *MockServer) handleTerminals(w http.ResponseWriter, r *http.Request) {
writeJSON(w, []map[string]interface{}{
{"id": "t1", "name": "终端 1", "status": "running"},
{"id": "t2", "name": "终端 2", "status": "idle"},
})
}
func (s *MockServer) handleCmdHistory(w http.ResponseWriter, r *http.Request) {
writeJSON(w, []map[string]interface{}{
{"cmd": "echo hello", "time": time.Now().Add(-5 * time.Minute).Format(time.RFC3339)},
{"cmd": "ls -la", "time": time.Now().Add(-10 * time.Minute).Format(time.RFC3339)},
})
}
func (s *MockServer) handleDevices(w http.ResponseWriter, r *http.Request) {
path := strings.TrimPrefix(r.URL.Path, "/api/v1/device")
path = strings.TrimSuffix(path, "/")
switch {
case path == "/online" || path == "":
writeJSON(w, s.devices)
case path == "/auth" && r.Method == "POST":
var req struct {
DeviceID string `json:"device_id"`
Authorized bool `json:"authorize"`
}
json.NewDecoder(r.Body).Decode(&req)
s.mu.Lock()
for i := range s.devices {
if s.devices[i].DeviceID == req.DeviceID {
s.devices[i].Authorized = req.Authorized
}
}
s.mu.Unlock()
writeJSON(w, map[string]interface{}{
"authorized": true,
"device_id": req.DeviceID,
})
case path == "/push" && r.Method == "POST":
writeJSON(w, map[string]string{"status": "pushed"})
default:
http.NotFound(w, r)
}
}
func (s *MockServer) handleAdapters(w http.ResponseWriter, r *http.Request) {
path := strings.TrimPrefix(r.URL.Path, "/api/v1/adapters")
path = strings.TrimSuffix(path, "/")
switch {
case path == "" && r.Method == "GET":
writeJSON(w, s.adapters)
case path == "" && r.Method == "POST":
var a Adapter
if err := json.NewDecoder(r.Body).Decode(&a); err == nil {
s.mu.Lock()
s.adapters = append(s.adapters, a)
s.mu.Unlock()
}
writeJSON(w, map[string]string{"status": "added"})
case strings.Count(path, "/") == 1 && r.Method == "DELETE":
name := strings.TrimPrefix(path, "/")
s.mu.Lock()
for i := range s.adapters {
if s.adapters[i].Name == name {
s.adapters = append(s.adapters[:i], s.adapters[i+1:]...)
break
}
}
s.mu.Unlock()
writeJSON(w, map[string]string{"status": "deleted"})
default:
http.NotFound(w, r)
}
}
func (s *MockServer) handleWebSocket(w http.ResponseWriter, r *http.Request) {
// 简单返回 400GUI 的 main.js 会尝试连接设备桥 WS
// 这里只验证 HTTP 路由可达
http.Error(w, "WebSocket upgrade required (mock server)", 400)
}
// ===== 路由注册 =====
func (s *MockServer) registerRoutes(mux *http.ServeMux) {
mux.HandleFunc("/api/v1/status", s.handleStatus)
mux.HandleFunc("/api/v1/kernel", s.handleKernel)
mux.HandleFunc("/api/v1/settings", s.handleSettings)
mux.HandleFunc("/api/v1/plugins", s.handlePlugins)
mux.HandleFunc("/api/v1/plugins/", s.handlePlugins)
mux.HandleFunc("/api/v1/chat/history", s.handleChatHistory)
mux.HandleFunc("/api/v1/chat", s.handleChat)
mux.HandleFunc("/api/v1/chat/events", s.handleChatEvents)
mux.HandleFunc("/api/v1/memory/graph", s.handleMemoryGraph)
mux.HandleFunc("/api/v1/memory", s.handleMemory)
mux.HandleFunc("/api/v1/memory/context", s.handleMemoryContext)
mux.HandleFunc("/api/v1/knowledge", s.handleKnowledge)
mux.HandleFunc("/api/v1/terminals", s.handleTerminals)
mux.HandleFunc("/api/v1/cmd/history", s.handleCmdHistory)
mux.HandleFunc("/api/v1/device", s.handleDevices)
mux.HandleFunc("/api/v1/device/", s.handleDevices)
mux.HandleFunc("/api/v1/adapters", s.handleAdapters)
mux.HandleFunc("/api/v1/adapters/", s.handleAdapters)
mux.HandleFunc("/api/v1/device/ws", s.handleDeviceWS)
}
func writeJSON(w http.ResponseWriter, v interface{}) {
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(v)
}
func main() {
server := NewMockServer()
mux := http.NewServeMux()
server.registerRoutes(mux)
addr := ":9099"
log.Printf("=== HomeAgent Mock Server ====")
log.Printf("监听地址: http://0.0.0.0%s", addr)
log.Printf("API 基础路径: http://0.0.0.0%s/api/v1/", addr)
log.Printf("SSE 端点: http://0.0.0.0%s/api/v1/chat/events", addr)
log.Printf("设备桥 WS: ws://0.0.0.0%s/api/v1/device/ws", addr)
log.Printf("==============================")
log.Fatal(http.ListenAndServe(addr, server.middleware(mux)))
}

472
cmd/mock-server/ws.go Normal file
View File

@ -0,0 +1,472 @@
package main
import (
"bufio"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/binary"
"encoding/json"
"fmt"
"io"
"log"
"net"
"net/http"
"sync"
"time"
"gitcode.com/JianFeeeee/HomeAgent/internal/devicebridge/client"
)
// ===== WebSocket 帧编码/解码RFC 6455 =====
const wsGUID = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
type WSConn struct {
conn net.Conn
rw *bufio.ReadWriter
mu sync.Mutex
}
func upgradeWS(w http.ResponseWriter, r *http.Request) (*WSConn, error) {
if r.Header.Get("Upgrade") != "websocket" {
http.Error(w, "not websocket", 400)
return nil, fmt.Errorf("not websocket upgrade")
}
key := r.Header.Get("Sec-WebSocket-Key")
if key == "" {
http.Error(w, "missing key", 400)
return nil, fmt.Errorf("missing Sec-WebSocket-Key")
}
h := sha256.Sum256([]byte(key + wsGUID))
accept := base64.StdEncoding.EncodeToString(h[:])
hijacker, ok := w.(http.Hijacker)
if !ok {
http.Error(w, "hijack not supported", 500)
return nil, fmt.Errorf("hijack not supported")
}
conn, bufrw, err := hijacker.Hijack()
if err != nil {
http.Error(w, "hijack failed", 500)
return nil, err
}
resp := "HTTP/1.1 101 Switching Protocols\r\n" +
"Upgrade: websocket\r\n" +
"Connection: Upgrade\r\n" +
"Sec-WebSocket-Accept: " + accept + "\r\n\r\n"
if _, err := bufrw.WriteString(resp); err != nil {
conn.Close()
return nil, err
}
if err := bufrw.Flush(); err != nil {
conn.Close()
return nil, err
}
return &WSConn{conn: conn, rw: bufrw}, nil
}
func (ws *WSConn) ReadFrame() (opcode byte, payload []byte, err error) {
for {
b0, err := ws.rw.ReadByte()
if err != nil {
return 0, nil, err
}
opcode = b0 & 0x0F
b1, err := ws.rw.ReadByte()
if err != nil {
return 0, nil, err
}
masked := b1&0x80 != 0
length := int64(b1 & 0x7F)
switch {
case length == 126:
var b [2]byte
if _, err := io.ReadFull(ws.rw, b[:]); err != nil {
return 0, nil, err
}
length = int64(binary.BigEndian.Uint16(b[:]))
case length == 127:
var b [8]byte
if _, err := io.ReadFull(ws.rw, b[:]); err != nil {
return 0, nil, err
}
length = int64(binary.BigEndian.Uint64(b[:]))
}
var maskKey [4]byte
if masked {
if _, err := io.ReadFull(ws.rw, maskKey[:]); err != nil {
return 0, nil, err
}
}
payload = make([]byte, length)
if _, err := io.ReadFull(ws.rw, payload); err != nil {
return 0, nil, err
}
if masked {
for i := range payload {
payload[i] ^= maskKey[i%4]
}
}
if opcode == 0x8 { // Close
ws.sendFrame(0x8, nil, false)
return opcode, payload, fmt.Errorf("ws closed")
}
if opcode == 0x9 { // Ping
ws.sendFrame(0xA, payload, false) // Pong
continue
}
if opcode == 0xA { // Pong
continue
}
return opcode, payload, nil
}
}
func (ws *WSConn) sendFrame(opcode byte, payload []byte, masked bool) error {
ws.mu.Lock()
defer ws.mu.Unlock()
buf := []byte{0x80 | opcode} // FIN + opcode
length := len(payload)
switch {
case length <= 125:
if masked {
buf = append(buf, byte(length)|0x80)
} else {
buf = append(buf, byte(length))
}
case length <= 65535:
if masked {
buf = append(buf, 126|0x80)
} else {
buf = append(buf, 126)
}
b := make([]byte, 2)
binary.BigEndian.PutUint16(b, uint16(length))
buf = append(buf, b...)
default:
if masked {
buf = append(buf, 127|0x80)
} else {
buf = append(buf, 127)
}
b := make([]byte, 8)
binary.BigEndian.PutUint64(b, uint64(length))
buf = append(buf, b...)
}
var maskKey [4]byte
if masked {
rand.Read(maskKey[:])
buf = append(buf, maskKey[:]...)
maskedPayload := make([]byte, length)
copy(maskedPayload, payload)
for i := range maskedPayload {
maskedPayload[i] ^= maskKey[i%4]
}
buf = append(buf, maskedPayload...)
} else {
buf = append(buf, payload...)
}
_, err := ws.conn.Write(buf)
return err
}
func (ws *WSConn) WriteJSON(v interface{}) error {
b, err := json.Marshal(v)
if err != nil {
return err
}
return ws.sendFrame(0x1, b, false) // Text frame
}
func (ws *WSConn) ReadJSON(v interface{}) error {
_, payload, err := ws.ReadFrame()
if err != nil {
return err
}
return json.Unmarshal(payload, v)
}
func (ws *WSConn) Close() {
ws.sendFrame(0x8, nil, false)
ws.conn.Close()
}
// ===== Mock Remotedevice 设备桥 =====
type MockDevice struct {
DeviceID string
Name string
Caps []string
Conn *WSConn
Online bool
}
type MockRemoteDevice struct {
mu sync.Mutex
devices map[string]*MockDevice
}
func NewMockRemoteDevice() *MockRemoteDevice {
return &MockRemoteDevice{
devices: make(map[string]*MockDevice),
}
}
func (s *MockServer) handleDeviceWS(w http.ResponseWriter, r *http.Request) {
ws, err := upgradeWS(w, r)
if err != nil {
log.Printf("[ws] upgrade failed: %v", err)
return
}
defer ws.Close()
log.Printf("[ws] 新设备连接")
// 处理 hello/bind/cmd 协议
var device *MockDevice
for {
var msg struct {
Op string `json:"op"`
DeviceID string `json:"device_id"`
Token string `json:"token"`
ReqID string `json:"req_id"`
Command string `json:"command"`
CmdType string `json:"cmd_type"`
Status string `json:"status"`
Output string `json:"output"`
Error string `json:"error"`
Device json.RawMessage `json:"device"`
Payload json.RawMessage `json:"payload"`
}
if err := ws.ReadJSON(&msg); err != nil {
log.Printf("[ws] read error: %v", err)
break
}
switch msg.Op {
case "hello":
var devMeta struct {
DeviceID string `json:"device_id"`
Name string `json:"name"`
Kind string `json:"kind"`
Caps []string `json:"caps"`
}
json.Unmarshal(msg.Device, &devMeta)
device = &MockDevice{
DeviceID: devMeta.DeviceID,
Name: devMeta.Name,
Caps: devMeta.Caps,
Conn: ws,
Online: true,
}
s.mu.Lock()
// 更新设备列表
found := false
for i := range s.devices {
if s.devices[i].DeviceID == devMeta.DeviceID {
s.devices[i].Online = true
s.devices[i].Caps = devMeta.Caps
found = true
break
}
}
if !found {
s.devices = append(s.devices, Device{
DeviceID: devMeta.DeviceID,
Name: devMeta.Name,
Authorized: false,
Online: true,
Caps: devMeta.Caps,
})
}
s.mu.Unlock()
ws.WriteJSON(map[string]interface{}{
"op": "hello_ack",
"code": 0,
})
log.Printf("[ws] 设备登记: %s (%s) caps=%v", devMeta.DeviceID, devMeta.Name, devMeta.Caps)
case "bind":
if msg.DeviceID == "" || msg.Token == "" {
ws.WriteJSON(map[string]interface{}{
"op": "bind_ack",
"code": 1,
"error": "missing device_id or token",
})
break
}
s.mu.Lock()
for i := range s.devices {
if s.devices[i].DeviceID == msg.DeviceID {
s.devices[i].Authorized = true
}
}
s.mu.Unlock()
ws.WriteJSON(map[string]interface{}{
"op": "bind_ack",
"code": 0,
})
log.Printf("[ws] 设备授权: %s", msg.DeviceID)
// 绑定成功后,发送全量能力测试命令序列
go func() {
time.Sleep(500 * time.Millisecond)
// 测试 1: screensee 截图
log.Printf("[ws] 发命令 1/7: homeagent-screensee")
ws.WriteJSON(client.CmdMsg{
Op: "cmd",
ReqID: "test_1_screensee_" + fmt.Sprintf("%d", time.Now().UnixMilli()),
Command: "homeagent-screensee",
CmdType: "homeagent",
})
time.Sleep(800 * time.Millisecond)
// 测试 2: clipboardsue 写入剪贴板
log.Printf("[ws] 发命令 2/7: homeagent-clipboardsue")
ws.WriteJSON(client.CmdMsg{
Op: "cmd",
ReqID: "test_2_clipboardsue_" + fmt.Sprintf("%d", time.Now().UnixMilli()),
Command: "homeagent-clipboardsue HomeAgent远程测试_" + fmt.Sprintf("%d", time.Now().UnixMilli()),
CmdType: "homeagent",
})
time.Sleep(800 * time.Millisecond)
// 测试 3: clipboardsee 读取剪贴板
log.Printf("[ws] 发命令 3/7: homeagent-clipboardsee")
ws.WriteJSON(client.CmdMsg{
Op: "cmd",
ReqID: "test_3_clipboardsee_" + fmt.Sprintf("%d", time.Now().UnixMilli()),
Command: "homeagent-clipboardsee",
CmdType: "homeagent",
})
time.Sleep(800 * time.Millisecond)
// 测试 4: computeruse 鼠标移动(使用非标准 JSON 格式测试兼容性)
log.Printf("[ws] 发命令 4/7: homeagent-computeruse move")
ws.WriteJSON(client.CmdMsg{
Op: "cmd",
ReqID: "test_4_computeruse_" + fmt.Sprintf("%d", time.Now().UnixMilli()),
Command: `homeagent-computeruse move {x:500,y:300}`,
CmdType: "homeagent",
})
time.Sleep(800 * time.Millisecond)
// 测试 4b: computeruse 鼠标点击(标准 JSON 格式)
log.Printf("[ws] 发命令 4b/7: homeagent-computeruse click")
ws.WriteJSON(client.CmdMsg{
Op: "cmd",
ReqID: "test_4b_click_" + fmt.Sprintf("%d", time.Now().UnixMilli()),
Command: `homeagent-computeruse {"x":800,"y":500,"action":"click","button":"left"}`,
CmdType: "homeagent",
})
time.Sleep(800 * time.Millisecond)
// 测试 5: speakeruse TTS 播报
log.Printf("[ws] 发命令 5/7: homeagent-speakeruse")
ws.WriteJSON(client.CmdMsg{
Op: "cmd",
ReqID: "test_5_speakeruse_" + fmt.Sprintf("%d", time.Now().UnixMilli()),
Command: "homeagent-speakeruse 你好这是来自远程Mock服务器的测试播报",
CmdType: "homeagent",
})
time.Sleep(800 * time.Millisecond)
// 测试 6: screensue 弹窗显示
log.Printf("[ws] 发命令 6/7: homeagent-screensueHTML")
ws.WriteJSON(client.CmdMsg{
Op: "cmd",
ReqID: "test_6_screensue_" + fmt.Sprintf("%d", time.Now().UnixMilli()),
Command: `homeagent-screensue 10 <!DOCTYPE html><html><head><meta charset="utf-8"><style>body{background:linear-gradient(135deg,#667eea,#764ba2);color:white;font-family:sans-serif;padding:30px;margin:0}h1{font-size:32px;text-shadow:0 2px 8px rgba(0,0,0,0.3)}.card{background:rgba(255,255,255,0.15);border-radius:12px;padding:20px;margin:12px 0;backdrop-filter:blur(8px)}.badge{display:inline-block;background:#4ade80;color:#000;padding:3px 10px;border-radius:16px;font-weight:bold}</style></head><body><h1>HomeAgent GUI 全量测试</h1><div class="card"><h2>能力测试结果</h2><table border="1" cellpadding="6" style="border-collapse:collapse;width:100%"><tr><th>能力</th><th>结果</th></tr><tr><td>screensee 截图</td><td><span class="badge">通过</span></td></tr><tr><td>clipboard 读写</td><td><span class="badge">通过</span></td></tr><tr><td>computeruse 操控</td><td><span class="badge">通过</span></td></tr><tr><td>speakeruse TTS</td><td><span class="badge">通过</span></td></tr><tr><td>screensue 渲染</td><td><span class="badge">通过</span></td></tr></table></div><p style="text-align:center;color:rgba(255,255,255,0.7)">2026-08-23 19:50</p></body></html>`,
CmdType: "homeagent",
})
time.Sleep(800 * time.Millisecond)
// 测试 7: omniparse 解析当前窗口 UI 元素
log.Printf("[ws] 发命令 7/7: homeagent-omniparse")
ws.WriteJSON(client.CmdMsg{
Op: "cmd",
ReqID: "test_7_omniparse_" + fmt.Sprintf("%d", time.Now().UnixMilli()),
Command: "homeagent-omniparse",
CmdType: "homeagent",
})
}()
case "cmd_result":
log.Printf("[ws] 命令结果: req=%s status=%s", msg.ReqID, msg.Status)
if msg.Output != "" {
output := msg.Output
if len(output) > 100 {
output = output[:100] + "..."
}
log.Printf("[ws] 输出: %s", output)
}
if msg.Error != "" {
log.Printf("[ws] 错误: %s", msg.Error)
}
case "data_start":
var ds client.DataStart
json.Unmarshal(msg.Payload, &ds)
log.Printf("[ws] 二进制数据开始: req=%s kind=%s mime=%s total=%d", ds.ReqID, ds.Kind, ds.MIME, ds.Total)
case "data_end":
log.Printf("[ws] 二进制数据结束: req=%s status=%s", msg.ReqID, msg.Status)
case "speech_start":
log.Printf("[ws] TTS 音频开始: req=%s", msg.ReqID)
case "speech_end":
log.Printf("[ws] TTS 音频结束: req=%s", msg.ReqID)
case "status":
log.Printf("[ws] 状态上报: %s -> %s", msg.DeviceID, msg.Status)
case "event":
log.Printf("[ws] 事件上报: %s type=%s", msg.DeviceID, string(msg.Payload))
default:
log.Printf("[ws] 未知消息类型: %s", msg.Op)
}
}
if device != nil {
device.Online = false
s.mu.Lock()
for i := range s.devices {
if s.devices[i].DeviceID == device.DeviceID {
s.devices[i].Online = false
}
}
s.mu.Unlock()
}
log.Printf("[ws] 设备断开")
}

View File

@ -3,14 +3,16 @@ package main
import (
"encoding/json"
"fmt"
"io"
"strings"
)
func handleBuiltin(cmd string, cfg *Config, state *State, reconnect func()) bool {
func handleBuiltin(cmd string, cfg *Config, state *State, reconnect func(), out io.Writer) bool {
switch {
case cmd == "/help":
fmt.Println(`Built-in commands:
fmt.Fprintln(out, `Built-in commands:
/help show this help
/stop [msg] stop generation / send interrupt (alias /interrupt)
/exit, /quit exit waiter
/clear clear screen
/reconnect force reconnection
@ -44,7 +46,7 @@ Any other text is sent to the agent directly.`)
return true
case cmd == "/clear":
fmt.Print("\033[H\033[2J")
fmt.Fprint(out, "\033[H\033[2J")
return true
case cmd == "/reconnect":
@ -72,7 +74,7 @@ Any other text is sent to the agent directly.`)
case cmd == "/conn list":
if len(cfg.Connections) == 0 {
fmt.Println("no saved connections")
fmt.Fprintln(out, "no saved connections")
}
for _, c := range cfg.Connections {
mark := " "
@ -83,39 +85,39 @@ Any other text is sent to the agent directly.`)
if addr == "" {
addr = c.Socket
}
fmt.Printf(" %s %-15s %s\n", mark, c.Name, addr)
fmt.Fprintf(out, " %s %-15s %s\n", mark, c.Name, addr)
}
return true
case strings.HasPrefix(cmd, "/conn save "):
name := strings.TrimSpace(cmd[11:])
cfg.SaveConnection(name)
fmt.Printf("connection saved as '%s' (default)\n", name)
fmt.Fprintf(out, "connection saved as '%s' (default)\n", name)
return true
case strings.HasPrefix(cmd, "/conn use "):
name := strings.TrimSpace(cmd[10:])
if cfg.SwitchConnection(name) {
fmt.Printf("switched to '%s'\n", name)
fmt.Fprintf(out, "switched to '%s'\n", name)
reconnect()
} else {
fmt.Printf("connection '%s' not found\n", name)
fmt.Fprintf(out, "connection '%s' not found\n", name)
}
return true
case strings.HasPrefix(cmd, "/conn del "):
name := strings.TrimSpace(cmd[10:])
if cfg.DeleteConnection(name) {
fmt.Printf("connection '%s' deleted\n", name)
fmt.Fprintf(out, "connection '%s' deleted\n", name)
} else {
fmt.Printf("connection '%s' not found\n", name)
fmt.Fprintf(out, "connection '%s' not found\n", name)
}
return true
case cmd == "/status":
if rc := state.RemoteConn(); rc != nil {
d, _ := rc.DoAPI("GET", "/api/v1/status", "")
printJSON(d)
printJSON(out, d)
} else {
state.Send("/status")
}
@ -124,7 +126,7 @@ Any other text is sent to the agent directly.`)
case cmd == "/kernel":
if rc := state.RemoteConn(); rc != nil {
d, _ := rc.DoAPI("GET", "/api/v1/kernel", "")
printJSON(d)
printJSON(out, d)
} else {
state.Send("/kernel")
}
@ -133,13 +135,13 @@ Any other text is sent to the agent directly.`)
case strings.HasPrefix(cmd, "/settings set "):
parts := strings.SplitN(cmd[14:], " ", 2)
if len(parts) < 2 {
fmt.Println("usage: /settings set <key> <value>")
fmt.Fprintln(out, "usage: /settings set <key> <value>")
return true
}
if rc := state.RemoteConn(); rc != nil {
body := fmt.Sprintf(`{"%s":%q}`, parts[0], parts[1])
rc.DoAPI("PUT", "/api/v1/settings", body)
fmt.Println("ok")
fmt.Fprintln(out, "ok")
} else {
state.Send(cmd[1:])
}
@ -148,7 +150,7 @@ Any other text is sent to the agent directly.`)
case strings.HasPrefix(cmd, "/settings"):
if rc := state.RemoteConn(); rc != nil {
d, _ := rc.DoAPI("GET", "/api/v1/settings", "")
printJSON(d)
printJSON(out, d)
} else {
state.Send(cmd[1:])
}
@ -157,7 +159,7 @@ Any other text is sent to the agent directly.`)
case cmd == "/plugin list":
if rc := state.RemoteConn(); rc != nil {
d, _ := rc.DoAPI("GET", "/api/v1/plugins", "")
printJSON(d)
printJSON(out, d)
} else {
state.Send("/plugin list")
}
@ -168,7 +170,7 @@ Any other text is sent to the agent directly.`)
if rc := state.RemoteConn(); rc != nil {
body := fmt.Sprintf(`{"url":%q}`, url)
d, _ := rc.DoAPI("POST", "/api/v1/plugins", body)
printJSON(d)
printJSON(out, d)
} else {
state.Send(cmd[1:])
}
@ -178,7 +180,7 @@ Any other text is sent to the agent directly.`)
name := strings.TrimSpace(cmd[15:])
if rc := state.RemoteConn(); rc != nil {
d, _ := rc.DoAPI("DELETE", "/api/v1/plugins/"+name, "")
printJSON(d)
printJSON(out, d)
} else {
state.Send(cmd[1:])
}
@ -188,7 +190,7 @@ Any other text is sent to the agent directly.`)
name := strings.TrimSpace(cmd[13:])
if rc := state.RemoteConn(); rc != nil {
d, _ := rc.DoAPI("GET", "/api/v1/plugins/"+name, "")
printJSON(d)
printJSON(out, d)
} else {
state.Send(cmd[1:])
}
@ -198,7 +200,7 @@ Any other text is sent to the agent directly.`)
q := strings.TrimSpace(cmd[14:])
if rc := state.RemoteConn(); rc != nil {
d, _ := rc.DoAPI("GET", "/api/v1/memory?query="+q, "")
printJSON(d)
printJSON(out, d)
} else {
state.Send(cmd[1:])
}
@ -208,7 +210,7 @@ Any other text is sent to the agent directly.`)
name := strings.TrimSpace(cmd[18:])
if rc := state.RemoteConn(); rc != nil {
d, _ := rc.DoAPI("DELETE", "/api/v1/knowledge/"+name, "")
printJSON(d)
printJSON(out, d)
} else {
state.Send(cmd[1:])
}
@ -217,7 +219,7 @@ Any other text is sent to the agent directly.`)
case cmd == "/knowledge":
if rc := state.RemoteConn(); rc != nil {
d, _ := rc.DoAPI("GET", "/api/v1/knowledge", "")
printJSON(d)
printJSON(out, d)
} else {
state.Send("/knowledge")
}
@ -226,7 +228,7 @@ Any other text is sent to the agent directly.`)
case cmd == "/agents":
if rc := state.RemoteConn(); rc != nil {
d, _ := rc.DoAPI("GET", "/api/v1/agents", "")
printJSON(d)
printJSON(out, d)
} else {
state.Send("/agents")
}
@ -237,11 +239,11 @@ Any other text is sent to the agent directly.`)
}
}
func printJSON(d map[string]interface{}) {
func printJSON(out io.Writer, d map[string]interface{}) {
if d == nil {
fmt.Println("(no data)")
fmt.Fprintln(out, "(no data)")
return
}
b, _ := json.MarshalIndent(d, "", " ")
fmt.Println(string(b))
fmt.Fprintln(out, string(b))
}

View File

@ -16,13 +16,14 @@ type Connection struct {
}
type Config struct {
Socket string `yaml:"socket"`
Remote string `yaml:"remote"`
APIKey string `yaml:"api_key"`
Default string `yaml:"default"`
Connections []Connection `yaml:"connections,omitempty"`
DeviceGateway string `yaml:"device_gateway,omitempty"` // remotedevice 网关地址(如 127.0.0.1:9890
DeviceToken string `yaml:"device_token,omitempty"` // 设备接入 token
Socket string `yaml:"socket"`
Remote string `yaml:"remote"`
APIKey string `yaml:"api_key"`
Default string `yaml:"default"`
Connections []Connection `yaml:"connections,omitempty"`
DeviceGateway string `yaml:"device_gateway,omitempty"` // remotedevice 网关地址(如 127.0.0.1:9890
DeviceToken string `yaml:"device_token,omitempty"` // 设备接入 token
DeviceAuthorized bool `yaml:"device_authorized,omitempty"` // 客户端本地授权(用户手动开启,服务端无法篡改)
}
func (c *Config) Active() *Connection {

File diff suppressed because it is too large Load Diff

View File

@ -13,8 +13,8 @@ type History struct {
mu sync.Mutex
}
func newHistory(path string, max int) History {
return History{path: path, max: max}
func newHistory(path string, max int) *History {
return &History{path: path, max: max}
}
func (h *History) load() {

View File

@ -2,13 +2,13 @@ package main
import (
"context"
"encoding/json"
"flag"
"fmt"
"os"
"os/signal"
"path/filepath"
"strings"
"sync"
"syscall"
"time"
)
@ -26,6 +26,48 @@ const clearLine = "\033[2K\r"
var colors = true
var spinnerFrames = []string{"⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"}
// isTTYFile 判断文件是否为字符终端(非终端时禁用 spinner 转圈)。
func isTTYFile(f *os.File) bool {
fi, err := f.Stat()
if err != nil {
return false
}
return fi.Mode()&os.ModeCharDevice != 0
}
// startSpinner 启动 npm 风格的加载动画,返回停止函数。
// stop() 幂等:终止动画并清除当前行。非终端环境直接空操作。
func startSpinner(label string) func() {
if !colors || !isTTYFile(os.Stdout) {
return func() {}
}
done := make(chan struct{})
var once sync.Once
go func() {
ticker := time.NewTicker(80 * time.Millisecond)
defer ticker.Stop()
i := 0
for {
select {
case <-done:
return
case <-ticker.C:
fmt.Printf("%s%s %s%s\n", clearLine, colorDim, spinnerFrames[i%len(spinnerFrames)]+" "+label, colorReset)
i++
}
}
}()
stop := func() {
once.Do(func() {
close(done)
fmt.Print(clearLine)
})
}
return stop
}
func init() {
if os.Getenv("NO_COLOR") != "" {
colors = false
@ -83,8 +125,17 @@ func main() {
say := flag.String("say", "", "deprecated alias of -chat")
deviceGateway := flag.String("device", "", "remotedevice 网关地址(如 127.0.0.1:9890启动设备桥")
deviceToken := flag.String("device-token", "", "设备接入 token")
deviceAuthorized := flag.Bool("device-authorized", false, "客户端本地授权(允许远程操控本机;也可在 waiter.yaml 配 device_authorized: true")
testCap := flag.String("test-cap", "", "测试本地能力screensue/speakeruse/screensee/clipboardsee/clipboardsue/computeruse/camerasue如 --test-cap screensue")
testCapArgs := flag.String("test-cap-args", "", "测试能力的参数")
flag.Parse()
// 本地能力测试模式(无需连接服务器)
if *testCap != "" {
runCapTest(*testCap, *testCapArgs)
return
}
cfg := discoverConfig(*configPath)
cfg.MergeCLI(*socket, *remote, *apiKey)
cfg.ApplyDefault()
@ -106,7 +157,6 @@ func main() {
defer state.Disconnect()
// 设备桥:--device 或配置 device_gateway 时waiter 作为被控设备接入 remotedevice
var bridge *deviceBridge
dg := *deviceGateway
if dg == "" {
dg = cfg.DeviceGateway
@ -116,12 +166,14 @@ func main() {
dt = cfg.DeviceToken
}
if dg != "" && dt != "" {
bridge = newDeviceBridge(dg, dt)
if err := bridge.Start(); err != nil {
if err := startDeviceBridge(dg, dt); err != nil {
printlnC(colorYellow, fmt.Sprintf("device bridge: %v (continue without)", err))
} else {
printlnC(colorGreen, "device bridge active: "+bridge.deviceID)
defer bridge.Stop()
// 客户端本地授权:命令行 --device-authorized 或 waiter.yaml device_authorized
auth := *deviceAuthorized || cfg.DeviceAuthorized
deviceBridge.SetAuthorized(auth)
printlnC(colorGreen, "device bridge active: "+deviceBridgeID+" authorized="+fmt.Sprint(auth))
defer stopDeviceBridge()
}
}
@ -129,26 +181,74 @@ func main() {
oneshot(state, oneShotMsg)
return
}
runInteractive(state, cfg)
}
// runInteractive 交互入口TTY 下走 Bubble Tea 全屏 TUI非 TTY 回退行式 REPL。
func runInteractive(state *State, cfg *Config) {
history := newHistory(historyPath(), 1000)
history.load()
if isTTYFile(os.Stdin) && isTTYFile(os.Stdout) && colors {
if err := runTUI(state, cfg, history); err != nil {
printlnC(colorRed, fmt.Sprintf("tui: %v", err))
printlnC(colorYellow, "falling back to line mode")
runLineMode(state, cfg, history)
}
return
}
runLineMode(state, cfg, history)
}
func oneshot(state *State, msg string) {
resp, err := state.SendChat(msg)
var sr streamRender
stop := startSpinner("thinking...")
resp, err := state.SendChatStream(msg, func(rl respLine) {
// 第一个过程帧到达即停转,后续帧直接渲染
stop()
if sr.handleDelta(rl) {
return // delta 已增量渲染
}
sr.reset() // 聚合帧/工具帧:结束 delta 流,换行输出
printServerEvent(rl)
})
stop()
if err != nil {
printlnC(colorRed, fmt.Sprintf("error: %v", err))
os.Exit(1)
}
fmt.Println(resp)
printlnC(colorGreen, resp)
}
func runInteractive(state *State, cfg *Config) {
// runCapTest 本地能力测试(无需连接服务器)
func runCapTest(capName, args string) {
fmt.Printf("=== 测试能力: %s ===\n", capName)
fmt.Printf("参数: %s\n", args)
fmt.Println("===========================")
// 覆盖 sendBridgeResult 为本地打印
sendBridgeResult = func(reqID, status, output, errMsg string) {
fmt.Printf("结果状态: %s\n", status)
if output != "" {
fmt.Printf("输出: %s\n", output)
}
if errMsg != "" {
fmt.Printf("错误: %s\n", errMsg)
}
}
handleHomeagentCmd("test-001", "homeagent-"+capName+" "+args)
fmt.Println("===========================")
fmt.Println("测试完成")
}
// runLineMode 传统行式 REPL非 TTY 回退 / TUI 启动失败时使用)。
func runLineMode(state *State, cfg *Config, history *History) {
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)
history := newHistory(historyPath(), 1000)
history.load()
line := newLineEditor(&history)
line := newLineEditor(history)
restore, err := setRawMode(0)
if err != nil {
@ -170,10 +270,18 @@ func runInteractive(state *State, cfg *Config) {
fmt.Println("Type /help for commands.")
var readerCancel func()
var spinnerStopMu sync.Mutex
var spinnerStop = func() {}
startReader := func() {
ctx, cancel := context.WithCancel(context.Background())
readerCancel = cancel
go state.ReadLoop(ctx, printServerOutput)
go state.ReadLoop(ctx, func(line string) {
spinnerStopMu.Lock()
stop := spinnerStop
spinnerStopMu.Unlock()
stop()
printServerOutput(line)
})
}
startReader()
@ -209,7 +317,7 @@ loop:
}
if cmd[0] == '/' {
if handleBuiltin(cmd, cfg, state, reconnect) {
if handleBuiltin(cmd, cfg, state, reconnect, os.Stdout) {
if cmd == "/exit" || cmd == "/quit" {
break loop
}
@ -227,6 +335,11 @@ loop:
state.Send(cmd)
}
// 发送成功后启动加载动画,收到第一帧服务器输出时自动停止
spinnerStopMu.Lock()
spinnerStop = startSpinner("thinking...")
spinnerStopMu.Unlock()
select {
case <-sigCh:
break loop
@ -239,22 +352,123 @@ loop:
}
}
func printServerOutput(content string) {
if !colors {
fmt.Printf("%s%s\n", clearLine, content)
return
}
var rl respLine
if err := json.Unmarshal([]byte(content), &rl); err != nil {
fmt.Printf("%s%s%s\n", clearLine, content, colorReset)
return
}
// streamRender 累积 token 级 delta 帧并增量重绘当前行。
// 聚合帧reasoning/tool_call/response到达时清空累积状态该轮已结束
// 旧服务器不发 delta此结构始终为空行为与原来完全一致。
type streamRender struct {
reasoning strings.Builder
content strings.Builder
}
// handleDelta 处理 delta 帧;返回是否消费了该帧。
// reset=true 的空帧表示服务端轮次作废(用户中断):清空累积并定格已显示内容。
func (sr *streamRender) handleDelta(rl respLine) bool {
switch rl.Type {
case "reasoning_delta":
if rl.Reset {
sr.reasoning.Reset()
fmt.Print(clearLine)
return true
}
sr.reasoning.WriteString(rl.Content)
if colors {
fmt.Printf("%s%s· %s%s", clearLine, colorDim, sr.reasoning.String(), colorReset)
} else {
fmt.Printf("%s[思考] %s", clearLine, sr.reasoning.String())
}
return true
case "content_delta":
if rl.Reset {
sr.content.Reset()
fmt.Print(clearLine + "\n") // 定格已显示的部分内容,换行
return true
}
sr.content.WriteString(rl.Content)
if colors {
fmt.Printf("%s%s%s%s", clearLine, colorGreen, sr.content.String(), colorReset)
} else {
fmt.Printf("%s%s", clearLine, sr.content.String())
}
return true
}
return false
}
// reset 在收到聚合帧/工具帧时调用delta 流被打断或结束,
// 下一行输出不再覆盖 delta 内容。
func (sr *streamRender) reset() {
sr.reasoning.Reset()
sr.content.Reset()
fmt.Print(clearLine + "\n")
}
// printServerOutput 渲染一行服务器输出JSON 帧)。
func printServerOutput(content string) {
rl := parseRespLineStruct(content)
if !colors {
fmt.Printf("%s%s\n", clearLine, renderPlain(rl, content))
return
}
printServerEventColored(rl, content)
}
// printServerEvent 渲染一个已解析的过程/终结事件。
func printServerEvent(rl respLine) {
if !colors {
fmt.Printf("%s%s\n", clearLine, renderPlain(rl, ""))
return
}
printServerEventColored(rl, "")
}
// renderPlain 无色模式下的纯文本渲染。
func renderPlain(rl respLine, raw string) string {
switch rl.Type {
case "reasoning", "reasoning_delta":
return "[思考] " + rl.Content
case "content_delta":
return rl.Content
case "tool_call":
return fmt.Sprintf("[工具] %s (%s) %s", rl.Tool, rl.Status, rl.Result)
case "response":
return rl.Content
case "error":
return "[错误] " + rl.Error
default:
if raw != "" {
return raw
}
return rl.Content
}
}
// printServerEventColored 彩色模式下的帧渲染。
func printServerEventColored(rl respLine, raw string) {
switch rl.Type {
case "reasoning":
fmt.Printf("%s%s· %s%s\n", clearLine, colorDim, rl.Content, colorReset)
case "tool_call":
mark, markColor := "⚙", colorYellow
switch rl.Status {
case "ok":
mark, markColor = "✔", colorGreen
case "denied", "interrupted", "error":
mark, markColor = "✘", colorRed
}
preview := rl.Result
if preview != "" {
preview = " " + preview
}
fmt.Printf("%s%s%s %s [%s]%s%s\n", clearLine, markColor, mark, rl.Tool, rl.Status, preview, colorReset)
case "response":
fmt.Printf("%s%s%s%s\n", clearLine, colorGreen, rl.Content, colorReset)
case "error":
fmt.Printf("%s%s%s%s\n", clearLine, colorRed, rl.Error, colorReset)
default:
fmt.Printf("%s%s%s\n", clearLine, content, colorReset)
text := raw
if text == "" {
text = rl.Content
}
fmt.Printf("%s%s%s\n", clearLine, text, colorReset)
}
}

View File

@ -61,14 +61,41 @@ func (s *State) Send(line string) error {
}
func (s *State) SendChat(msg string) (string, error) {
return s.SendChatStream(msg, nil)
}
// SendChatStream 发送一条对话消息并循环读取响应行直至终结帧。
// onEvent 回调在每收到一个过程帧reasoning/tool_call时被调用
// 可为 nil返回值为最终响应内容或错误。
func (s *State) SendChatStream(msg string, onEvent func(respLine)) (string, error) {
if err := s.Send(msg); err != nil {
return "", err
}
line, err := s.readLine()
if err != nil {
return "", err
for {
line, err := s.readLine()
if err != nil {
return "", err
}
rl := parseRespLineStruct(line)
switch rl.Type {
case "response":
return rl.Content, nil
case "error":
if rl.Error == "" {
rl.Error = line
}
return "", fmt.Errorf("%s", rl.Error)
default:
// 过程帧reasoning / tool_call / 旧版服务器的普通文本
if rl.Type == "" && onEvent == nil && rl.Content == "" && rl.Error == "" {
// 非JSON旧行且无回调直接当最终输出向后兼容旧服务器
return line, nil
}
if onEvent != nil {
onEvent(rl)
}
}
}
return parseRespLine(line)
}
func (s *State) SendBuiltin(cmd string) (string, error) {
@ -96,13 +123,23 @@ type respLine struct {
Type string `json:"type"`
Content string `json:"content"`
Error string `json:"error"`
Tool string `json:"tool"`
Status string `json:"status"`
Result string `json:"result"`
Reset bool `json:"reset,omitempty"` // delta 帧:服务端轮次作废,清空累积
}
// parseRespLineStruct 解析一行 JSON 响应帧,解析失败时将原文放入 Content。
func parseRespLineStruct(line string) respLine {
var rl respLine
if err := json.Unmarshal([]byte(line), &rl); err != nil {
return respLine{Content: line}
}
return rl
}
func parseRespLine(line string) (string, error) {
var rl respLine
if err := json.Unmarshal([]byte(line), &rl); err != nil {
return line, nil
}
rl := parseRespLineStruct(line)
switch rl.Type {
case "response":
return rl.Content, nil

734
cmd/waiter/tui.go Normal file
View File

@ -0,0 +1,734 @@
package main
import (
"fmt"
"strings"
"time"
"github.com/charmbracelet/bubbles/textarea"
"github.com/charmbracelet/bubbles/viewport"
tea "github.com/charmbracelet/bubbletea"
"github.com/charmbracelet/lipgloss"
)
// ---------------------------------------------------------------------------
// 配色(对齐 deveco-code 深色主题12 阶灰阶 + 语义色,无 emoji纯文本标记
// ---------------------------------------------------------------------------
var (
cStep3 = lipgloss.Color("#1e1e1e")
cStep7 = lipgloss.Color("#484848")
cStep11 = lipgloss.Color("#808080")
cStep12 = lipgloss.Color("#eeeeee")
cPrimary = lipgloss.Color("#fab283") // 主色(暖橙)
cAccent = lipgloss.Color("#9d7cd8") // 紫
cGreen = lipgloss.Color("#7fd88f")
cRed = lipgloss.Color("#e06c75")
cYellow = lipgloss.Color("#e5c07b")
cCyan = lipgloss.Color("#56b6c2")
)
var (
styleHeaderBox = lipgloss.NewStyle().Foreground(cStep12).Background(cStep3).Padding(0, 1)
styleTitle = lipgloss.NewStyle().Bold(true).Foreground(cPrimary)
styleDotOn = lipgloss.NewStyle().Foreground(cGreen)
styleDotOff = lipgloss.NewStyle().Foreground(cRed)
styleDim = lipgloss.NewStyle().Foreground(cStep11)
styleUserTag = lipgloss.NewStyle().Bold(true).Foreground(cAccent)
styleAgentTag = lipgloss.NewStyle().Bold(true).Foreground(cPrimary)
styleSysTag = lipgloss.NewStyle().Bold(true).Foreground(cCyan)
styleErrTag = lipgloss.NewStyle().Bold(true).Foreground(cRed)
styleReason = lipgloss.NewStyle().Foreground(cStep11)
styleInputBox = lipgloss.NewStyle().Border(lipgloss.RoundedBorder()).BorderForeground(cStep7)
styleInputFocus = lipgloss.NewStyle().Border(lipgloss.RoundedBorder()).BorderForeground(cPrimary)
styleStatusBar = lipgloss.NewStyle().Foreground(cStep11).Background(cStep3).Padding(0, 1)
styleSep = lipgloss.NewStyle().Foreground(cStep7)
styleToolPending = lipgloss.NewStyle().Foreground(cYellow)
styleToolOK = lipgloss.NewStyle().Foreground(cGreen)
styleToolFail = lipgloss.NewStyle().Foreground(cRed)
)
// ---------------------------------------------------------------------------
// 消息模型
// ---------------------------------------------------------------------------
type msgKind int
const (
msgUser msgKind = iota
msgAgent
msgReasoning
msgTool
msgSystem
msgError
)
type chatMsg struct {
kind msgKind
text string
tool string // msgTool: 工具名
status string // msgTool: ok/denied/interrupted/error/running
result string // msgTool: 结果预览
final bool // msgAgent: 流式消息已完成(后续 delta 不再追加)
}
// ---- tea.Msg ----
type spinnerTickMsg struct{}
type serverLineMsg struct{ line string }
type readerErrMsg struct {
err error
gen int
}
type reconnectDoneMsg struct{ ok bool }
// ---------------------------------------------------------------------------
// Model
// ---------------------------------------------------------------------------
type tuiModel struct {
state *State
cfg *Config
vp viewport.Model
input textarea.Model
messages []chatMsg
width int
height int
ready bool
busy bool
spinnerIdx int
history *History
historyIdx int // -1 = 无导航0..n-1 = history.all() 下标(越大越新)
historyDraft string
lines chan string
errs chan readerErrMsg
readerGen int // 当前 reader 世代;重启时递增
readerAlive bool
reconnecting bool
}
func newTuiModel(state *State, cfg *Config, history *History, lines chan string, errs chan readerErrMsg) tuiModel {
ti := textarea.New()
ti.Placeholder = "输入消息,/help 查看命令"
ti.Prompt = ""
ti.CharLimit = -1
ti.SetHeight(1)
ti.ShowLineNumbers = false
ti.Focus()
return tuiModel{
state: state,
cfg: cfg,
input: ti,
vp: viewport.New(80, 20),
history: history,
historyIdx: -1,
lines: lines,
errs: errs,
readerAlive: true,
}
}
func (m tuiModel) Init() tea.Cmd {
return tea.Batch(textarea.Blink, waitServer(m.lines, m.errs))
}
// waitServer 阻塞等待下一行服务器输出或读错误。
func waitServer(lines chan string, errs chan readerErrMsg) tea.Cmd {
return func() tea.Msg {
select {
case l := <-lines:
return serverLineMsg{l}
case e := <-errs:
return e
}
}
}
func spinTick() tea.Cmd {
return tea.Tick(90*time.Millisecond, func(time.Time) tea.Msg { return spinnerTickMsg{} })
}
// spinCmd busy 时启动 spinner tick 循环。
func (m tuiModel) spinCmd() tea.Cmd {
if !m.busy {
return nil
}
return spinTick()
}
// reconnectCmd 后台重连State 自带锁goroutine 安全)。
func (m tuiModel) reconnectCmd() tea.Cmd {
return func() tea.Msg {
m.state.Disconnect()
for i := 0; i < 15; i++ {
if err := m.state.Connect(m.cfg); err == nil {
return reconnectDoneMsg{ok: true}
}
time.Sleep(time.Second)
}
return reconnectDoneMsg{ok: false}
}
}
// restartReader 重启读循环 goroutine。
func (m *tuiModel) restartReader() {
m.readerGen++ // 使旧 reader 的迟到错误失效
m.readerAlive = true
go m.readPump(m.readerGen)
}
// readPump 持续读服务器输出并投递到 channel出错时投递带世代号的 err 后退出。
func (m *tuiModel) readPump(gen int) {
for {
line, err := m.state.readLine()
if err != nil {
select {
case m.errs <- readerErrMsg{err: err, gen: gen}:
default:
}
return
}
select {
case m.lines <- line:
case <-time.After(30 * time.Second):
return
}
}
}
// ---------------------------------------------------------------------------
// Update
// ---------------------------------------------------------------------------
func (m tuiModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
switch msg := msg.(type) {
case tea.WindowSizeMsg:
m.width = msg.Width
m.height = msg.Height
if !m.ready {
m.ready = true
m.append(chatMsg{kind: msgSystem,
text: fmt.Sprintf("connected %s://%s", m.modeLabel(), m.addrLabel())})
}
m.layout()
case tea.KeyMsg:
switch msg.Type {
case tea.KeyCtrlC, tea.KeyCtrlD:
return m, tea.Quit
case tea.KeyEnter:
return m.handleSubmit()
case tea.KeyUp:
lines := m.history.all()
if len(lines) == 0 {
return m, nil
}
if m.historyIdx == -1 {
m.historyDraft = m.input.Value()
m.historyIdx = len(lines) - 1
} else if m.historyIdx > 0 {
m.historyIdx--
}
m.input.SetValue(lines[m.historyIdx])
return m, nil
case tea.KeyDown:
if m.historyIdx >= 0 {
m.historyIdx++
if m.historyIdx >= len(m.history.all()) {
m.historyIdx = -1
m.input.SetValue(m.historyDraft)
} else {
m.input.SetValue(m.history.all()[m.historyIdx])
}
}
return m, nil
case tea.KeyPgUp:
m.vp.HalfPageUp()
return m, nil
case tea.KeyPgDown:
m.vp.HalfPageDown()
return m, nil
}
case spinnerTickMsg:
if m.busy {
m.spinnerIdx++
return m, spinTick()
}
case serverLineMsg:
m.handleServerLine(msg.line)
var cmds []tea.Cmd
if m.readerAlive {
cmds = append(cmds, waitServer(m.lines, m.errs))
}
return m, tea.Batch(cmds...)
case readerErrMsg:
if msg.gen != m.readerGen {
// 旧 reader 的迟到错误:新 reader 已在运行,忽略
return m, nil
}
m.readerAlive = false
if !m.reconnecting {
m.append(chatMsg{kind: msgError, text: "connection lost: " + msg.err.Error()})
}
return m, nil
case reconnectDoneMsg:
m.reconnecting = false
if msg.ok {
m.restartReader()
m.append(chatMsg{kind: msgSystem, text: "reconnected"})
return m, waitServer(m.lines, m.errs)
}
m.append(chatMsg{kind: msgError, text: "reconnect failed after 15 attempts"})
return m, nil
}
// 其余按键交给输入框
var icmd tea.Cmd
m.input, icmd = m.input.Update(msg)
var vcmd tea.Cmd
m.vp, vcmd = m.vp.Update(msg)
return m, tea.Batch(icmd, vcmd)
}
func (m tuiModel) handleSubmit() (tea.Model, tea.Cmd) {
text := strings.TrimSpace(m.input.Value())
m.input.Reset()
m.historyIdx = -1
if text == "" {
return m, nil
}
// TUI 自己处理的内置命令
switch text {
case "/exit", "/quit":
return m, tea.Quit
case "/clear":
m.messages = nil
m.refreshViewport()
return m, nil
case "/reconnect":
m.append(chatMsg{kind: msgSystem, text: "reconnecting..."})
m.reconnecting = true
return m, m.reconnectCmd()
}
// 其余内置命令:捕获输出进消息区
if strings.HasPrefix(text, "/") {
needReconnect := false
var buf strings.Builder
handled := handleBuiltin(text, m.cfg, m.state, func() { needReconnect = true }, &buf)
if handled {
if out := strings.TrimRight(buf.String(), "\n"); out != "" {
m.append(chatMsg{kind: msgSystem, text: out})
}
if needReconnect {
m.reconnecting = true
return m, m.reconnectCmd()
}
// 即使内置命令也可能触发服务器回复(如 /status所以继续保持监听
if m.readerAlive {
return m, waitServer(m.lines, m.errs)
}
return m, nil
}
}
// 普通消息:发给 agent
m.append(chatMsg{kind: msgUser, text: text})
m.history.add(text)
m.history.save()
m.busy = true
if err := m.state.Send(text); err != nil {
m.busy = false
m.append(chatMsg{kind: msgError, text: "send failed: " + err.Error()})
m.reconnecting = true
return m, m.reconnectCmd()
}
if m.readerAlive {
return m, tea.Batch(waitServer(m.lines, m.errs), m.spinCmd())
}
return m, m.spinCmd()
}
func (m *tuiModel) handleServerLine(line string) {
rl := parseRespLineStruct(line)
switch rl.Type {
case "reasoning":
cm := chatMsg{kind: msgReasoning, text: rl.Content}
// 连续 reasoning 增量合并到最后一条,形成流式效果
if n := len(m.messages); n > 0 && m.messages[n-1].kind == msgReasoning {
m.messages[n-1].text += rl.Content
} else {
m.append(cm)
}
case "reasoning_delta":
// token 级增量:与 reasoning 同样合并到最后一条 reasoning 消息
if rl.Reset {
m.sealLastAgent()
break
}
if n := len(m.messages); n > 0 && m.messages[n-1].kind == msgReasoning {
m.messages[n-1].text += rl.Content
} else if rl.Content != "" {
m.append(chatMsg{kind: msgReasoning, text: rl.Content})
}
case "content_delta":
// token 级增量:追加到最后一条 agent 消息(流式生成中的回复)
if rl.Reset {
m.sealLastAgent()
break
}
if n := len(m.messages); n > 0 && m.messages[n-1].kind == msgAgent && !m.messages[n-1].final {
m.messages[n-1].text += rl.Content
} else if rl.Content != "" {
m.append(chatMsg{kind: msgAgent, text: rl.Content})
}
case "tool_call":
// 工具调用打断内容流:置 final 防止后续 delta 误追加到旧消息
m.sealLastAgent()
m.append(chatMsg{kind: msgTool, tool: rl.Tool, status: rl.Status, result: rl.Result})
case "response":
// 聚合最终响应:覆盖/替换 delta 累积的最后一条 agent 消息(内容相同),
// 或在无 delta 时新建。置 final 标记本轮完成。
m.busy = false
if n := len(m.messages); n > 0 && m.messages[n-1].kind == msgAgent && !m.messages[n-1].final {
if rl.Content != "" {
m.messages[n-1].text = rl.Content // 以聚合为准(含 stage 插件改写后的最终文本)
}
m.messages[n-1].final = true
} else {
cm := chatMsg{kind: msgAgent, text: rl.Content, final: true}
m.append(cm)
}
case "error":
m.busy = false
m.sealLastAgent()
m.append(chatMsg{kind: msgError, text: rl.Error})
default:
// 非 JSON 旧行(旧服务器):当最终输出
m.busy = false
m.append(chatMsg{kind: msgAgent, text: line, final: true})
}
m.refreshViewport()
}
// sealLastAgent 将最后一条未完成的 agent 流式消息标记为完成。
func (m *tuiModel) sealLastAgent() {
if n := len(m.messages); n > 0 && m.messages[n-1].kind == msgAgent {
m.messages[n-1].final = true
}
}
func (m *tuiModel) append(cm chatMsg) {
m.messages = append(m.messages, cm)
m.refreshViewport()
}
func (m *tuiModel) modeLabel() string {
if m.cfg.Remote != "" {
return "remote"
}
return "local"
}
func (m *tuiModel) addrLabel() string {
if m.cfg.Remote != "" {
return m.cfg.Remote
}
return m.cfg.Socket
}
// ---------------------------------------------------------------------------
// 渲染
// ---------------------------------------------------------------------------
const tagWidth = 5
func (m tuiModel) renderMessage(cm chatMsg, width int) string {
switch cm.kind {
case msgUser:
return hangingIndent(styleUserTag.Render("You"), cm.text, width)
case msgAgent:
return hangingIndent(styleAgentTag.Render("小宅"), cm.text, width)
case msgReasoning:
tail := lastNonEmptyLine(cm.text)
if w := width - tagWidth - 4; w > 8 && lipgloss.Width(tail) > w {
tail = truncateTail(tail, w)
}
return styleDim.Render(" · " + tail)
case msgTool:
var mark string
var st lipgloss.Style
switch cm.status {
case "ok":
mark, st = "[ok]", styleToolOK
case "denied", "interrupted", "error":
mark, st = "[fail]", styleToolFail
default:
mark, st = "[..]", styleToolPending
}
preview := ""
if cm.result != "" {
first := firstLine(cm.result)
if w := width - tagWidth - 24; w > 8 && lipgloss.Width(first) > w {
first = truncateTail(first, w)
}
preview = styleDim.Render(" " + first)
}
return " " + st.Render(fmt.Sprintf("%-6s", mark)) +
styleDim.Render(fmt.Sprintf("%-*s", 18, cm.tool)) + preview
case msgSystem:
return styleSysTag.Render("[sys]") + " " + styleDim.Render(cm.text)
case msgError:
return styleErrTag.Render("[err]") + " " +
lipgloss.NewStyle().Foreground(cRed).Render(cm.text)
}
return ""
}
// hangingIndent 两列布局:首行 "tag body",续行缩进对齐 body。
func hangingIndent(tag, body string, width int) string {
indent := tagWidth + 1
avail := width - indent
if avail < 10 {
avail = 10
}
pad := indent - lipgloss.Width(tag)
if pad < 1 {
pad = 1
}
prefix := tag + strings.Repeat(" ", pad)
var b strings.Builder
first := true
for _, ln := range strings.Split(body, "\n") {
for j, seg := range wordWrap(ln, avail) {
if first && j == 0 {
b.WriteString(prefix)
} else {
b.WriteString(strings.Repeat(" ", indent))
}
b.WriteString(seg)
b.WriteString("\n")
}
first = false
}
return strings.TrimSuffix(b.String(), "\n")
}
// wordWrap 按 display 宽度断行(宽字符按 2 列计)。
func wordWrap(s string, limit int) []string {
if s == "" {
return []string{""}
}
if lipgloss.Width(s) <= limit {
return []string{s}
}
var out []string
var cur strings.Builder
curW := 0
for _, r := range s {
w := runeWidth(r)
if curW+w > limit && cur.Len() > 0 {
out = append(out, cur.String())
cur.Reset()
curW = 0
}
cur.WriteRune(r)
curW += w
}
if cur.Len() > 0 {
out = append(out, cur.String())
}
return out
}
func runeWidth(r rune) int {
if r >= 0x1100 && (r <= 0x115F ||
r == 0x2329 || r == 0x232A ||
(r >= 0x2E80 && r <= 0xA4CF) ||
(r >= 0xAC00 && r <= 0xD7A3) ||
(r >= 0xF900 && r <= 0xFAFF) ||
(r >= 0xFE30 && r <= 0xFE4F) ||
(r >= 0xFF00 && r <= 0xFF60) ||
(r >= 0xFFE0 && r <= 0xFFE6) ||
(r >= 0x20000 && r <= 0x3FFFD)) {
return 2
}
return 1
}
func firstLine(s string) string {
if i := strings.IndexByte(s, '\n'); i >= 0 {
return s[:i]
}
return s
}
func lastNonEmptyLine(s string) string {
lines := strings.Split(strings.TrimRight(s, "\n"), "\n")
for i := len(lines) - 1; i >= 0; i-- {
if strings.TrimSpace(lines[i]) != "" {
return lines[i]
}
}
return ""
}
// truncateTail 尾部省略(保留开头)。
func truncateTail(s string, max int) string {
out := ""
w := 0
for _, r := range s {
rw := runeWidth(r)
if w+rw > max-1 {
break
}
out += string(r)
w += rw
}
return out + "…"
}
// ---------------------------------------------------------------------------
// 布局与视图
// ---------------------------------------------------------------------------
const (
headerRows = 1
inputRows = 3 // 圆角边框上下 + 1 行输入
statusRows = 1
gapRows = 2 // 头部与消息区、消息区与输入框之间的空行
minVpHeight = 4
)
func (m *tuiModel) layout() {
h := m.height - headerRows - statusRows - inputRows - gapRows
if h < minVpHeight {
h = minVpHeight
}
m.vp.Width = m.width
m.vp.Height = h
}
func (m *tuiModel) refreshViewport() {
var b strings.Builder
w := m.width - 2
if w < 40 {
w = 40
}
for i, cm := range m.messages {
b.WriteString(m.renderMessage(cm, w))
if i < len(m.messages)-1 {
b.WriteString("\n\n") // 消息间空行分隔
}
}
m.vp.SetContent(b.String())
m.vp.GotoBottom()
}
func (m tuiModel) View() string {
if !m.ready {
return ""
}
// 顶栏
dot := styleDotOn.Render("●")
connText := "connected"
if !m.state.Connected() {
dot = styleDotOff.Render("●")
connText = "disconnected"
}
left := styleTitle.Render("HomeAgent") + " " + dot + " " + connText
right := styleDim.Render(truncMid(m.addrLabel(), maxInt(10, m.width-lipgloss.Width(left)-8)))
header := styleHeaderBox.Width(m.width).MaxWidth(m.width).Render(left + " " + right)
// 输入区
boxStyle := styleInputBox
if m.input.Focused() {
boxStyle = styleInputFocus
}
inputBox := boxStyle.Width(m.width - 2).Render(m.input.View())
statusBar := styleStatusBar.Width(m.width).MaxWidth(m.width).Render(m.statusLine())
return lipgloss.JoinVertical(lipgloss.Left,
header,
"",
m.vp.View(),
"",
inputBox,
statusBar,
)
}
func (m tuiModel) statusLine() string {
var leftSeg string
if m.busy {
leftSeg = styleTitle.Render(spinnerFrames[m.spinnerIdx%len(spinnerFrames)] + " thinking...")
} else {
leftSeg = styleDim.Render("enter 发送 · PgUp/PgDn 翻页 · ctrl+c 退出")
}
right := styleDim.Render(connSummary(m.state, m.modeLabel()))
gap := m.width - lipgloss.Width(leftSeg) - lipgloss.Width(right) - 2
if gap < 1 {
gap = 1
}
return leftSeg + strings.Repeat(" ", gap) + right
}
func connSummary(state *State, mode string) string {
s := "● " + mode
if state.Connected() {
return styleDotOn.Render(s)
}
return styleDotOff.Render(s + " disconnected")
}
func truncMid(s string, limit int) string {
if lipgloss.Width(s) <= limit {
return s
}
half := (limit - 1) / 2
r := []rune(s)
if half < 1 {
return "…"
}
return string(r[:half]) + "…" + string(r[len(r)-half:])
}
func maxInt(a, b int) int {
if a > b {
return a
}
return b
}
// ---------------------------------------------------------------------------
// 入口
// ---------------------------------------------------------------------------
// runTUI 启动 Bubble Tea 全屏界面。
func runTUI(state *State, cfg *Config, history *History) error {
m := newTuiModel(state, cfg, history, make(chan string, 128), make(chan readerErrMsg, 8))
m.readerAlive = true
go m.readPump(m.readerGen) // 初始读循环
p := tea.NewProgram(m, tea.WithAltScreen())
_, err := p.Run()
return err
}

30
go.mod
View File

@ -13,8 +13,34 @@ require github.com/yanyiwu/gojieba v1.4.7
require github.com/yalue/onnxruntime_go v1.13.0
require (
gitcode.com/JianFeeeee/homeagent-sdk v0.8.0
golang.org/x/sys v0.8.0
gitcode.com/JianFeeeee/homeagent-sdk v0.9.1
github.com/charmbracelet/bubbles v1.0.0
github.com/charmbracelet/bubbletea v1.3.10
github.com/charmbracelet/lipgloss v1.1.0
golang.org/x/sys v0.38.0
)
require (
github.com/atotto/clipboard v0.1.4 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/charmbracelet/colorprofile v0.4.1 // indirect
github.com/charmbracelet/x/ansi v0.11.6 // indirect
github.com/charmbracelet/x/cellbuf v0.0.15 // indirect
github.com/charmbracelet/x/term v0.2.2 // indirect
github.com/clipperhouse/displaywidth v0.9.0 // indirect
github.com/clipperhouse/stringish v0.1.1 // indirect
github.com/clipperhouse/uax29/v2 v2.5.0 // indirect
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
github.com/lucasb-eyer/go-colorful v1.3.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-localereader v0.0.1 // indirect
github.com/mattn/go-runewidth v0.0.19 // indirect
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect
github.com/muesli/termenv v0.16.0 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
golang.org/x/text v0.3.8 // indirect
)
replace gitcode.com/JianFeeeee/homeagent-sdk => ./third_party/homeagent-sdk

58
go.sum
View File

@ -1,13 +1,67 @@
github.com/MakeNowJust/heredoc v1.0.0 h1:cXCdzVdstXyiTqTvfqk9SDHpKNjxuom+DOlyEeQ4pzQ=
github.com/MakeNowJust/heredoc v1.0.0/go.mod h1:mG5amYoWBHf8vpLOuehzbGGw0EHxpZZ6lCpQ4fNJ8LE=
github.com/atotto/clipboard v0.1.4 h1:EH0zSVneZPSuFR11BlR9YppQTVDbh5+16AmcJi4g1z4=
github.com/atotto/clipboard v0.1.4/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn0Yu86PYI=
github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k=
github.com/aymanbagabas/go-osc52/v2 v2.0.1/go.mod h1:uYgXzlJ7ZpABp8OJ+exZzJJhRNQ2ASbcXHWsFqH8hp8=
github.com/aymanbagabas/go-udiff v0.3.1 h1:LV+qyBQ2pqe0u42ZsUEtPiCaUoqgA9gYRDs3vj1nolY=
github.com/aymanbagabas/go-udiff v0.3.1/go.mod h1:G0fsKmG+P6ylD0r6N/KgQD/nWzgfnl8ZBcNLgcbrw8E=
github.com/charmbracelet/bubbles v1.0.0 h1:12J8/ak/uCZEMQ6KU7pcfwceyjLlWsDLAxB5fXonfvc=
github.com/charmbracelet/bubbles v1.0.0/go.mod h1:9d/Zd5GdnauMI5ivUIVisuEm3ave1XwXtD1ckyV6r3E=
github.com/charmbracelet/bubbletea v1.3.10 h1:otUDHWMMzQSB0Pkc87rm691KZ3SWa4KUlvF9nRvCICw=
github.com/charmbracelet/bubbletea v1.3.10/go.mod h1:ORQfo0fk8U+po9VaNvnV95UPWA1BitP1E0N6xJPlHr4=
github.com/charmbracelet/colorprofile v0.4.1 h1:a1lO03qTrSIRaK8c3JRxJDZOvhvIeSco3ej+ngLk1kk=
github.com/charmbracelet/colorprofile v0.4.1/go.mod h1:U1d9Dljmdf9DLegaJ0nGZNJvoXAhayhmidOdcBwAvKk=
github.com/charmbracelet/lipgloss v1.1.0 h1:vYXsiLHVkK7fp74RkV7b2kq9+zDLoEU4MZoFqR/noCY=
github.com/charmbracelet/lipgloss v1.1.0/go.mod h1:/6Q8FR2o+kj8rz4Dq0zQc3vYf7X+B0binUUBwA0aL30=
github.com/charmbracelet/x/ansi v0.11.6 h1:GhV21SiDz/45W9AnV2R61xZMRri5NlLnl6CVF7ihZW8=
github.com/charmbracelet/x/ansi v0.11.6/go.mod h1:2JNYLgQUsyqaiLovhU2Rv/pb8r6ydXKS3NIttu3VGZQ=
github.com/charmbracelet/x/cellbuf v0.0.15 h1:ur3pZy0o6z/R7EylET877CBxaiE1Sp1GMxoFPAIztPI=
github.com/charmbracelet/x/cellbuf v0.0.15/go.mod h1:J1YVbR7MUuEGIFPCaaZ96KDl5NoS0DAWkskup+mOY+Q=
github.com/charmbracelet/x/term v0.2.2 h1:xVRT/S2ZcKdhhOuSP4t5cLi5o+JxklsoEObBSgfgZRk=
github.com/charmbracelet/x/term v0.2.2/go.mod h1:kF8CY5RddLWrsgVwpw4kAa6TESp6EB5y3uxGLeCqzAI=
github.com/clipperhouse/displaywidth v0.9.0 h1:Qb4KOhYwRiN3viMv1v/3cTBlz3AcAZX3+y9OLhMtAtA=
github.com/clipperhouse/displaywidth v0.9.0/go.mod h1:aCAAqTlh4GIVkhQnJpbL0T/WfcrJXHcj8C0yjYcjOZA=
github.com/clipperhouse/stringish v0.1.1 h1:+NSqMOr3GR6k1FdRhhnXrLfztGzuG+VuFDfatpWHKCs=
github.com/clipperhouse/stringish v0.1.1/go.mod h1:v/WhFtE1q0ovMta2+m+UbpZ+2/HEXNWYXQgCt4hdOzA=
github.com/clipperhouse/uax29/v2 v2.5.0 h1:x7T0T4eTHDONxFJsL94uKNKPHrclyFI0lm7+w94cO8U=
github.com/clipperhouse/uax29/v2 v2.5.0/go.mod h1:Wn1g7MK6OoeDT0vL+Q0SQLDz/KpfsVRgg6W7ihQeh4g=
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f h1:Y/CXytFA4m6baUTXGLOoWe4PQhGxaX0KpnayAqC48p4=
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f/go.mod h1:vw97MGsxSvLiUE2X8qFplwetxpGLQrlU1Q9AUEIzCaM=
github.com/lucasb-eyer/go-colorful v1.3.0 h1:2/yBRLdWBZKrf7gB40FoiKfAWYQ0lqNcbuQwVHXptag=
github.com/lucasb-eyer/go-colorful v1.3.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-localereader v0.0.1 h1:ygSAOl7ZXTx4RdPYinUpg6W99U8jWvWi9Ye2JC/oIi4=
github.com/mattn/go-localereader v0.0.1/go.mod h1:8fBrzywKY7BI3czFoHkuzRoWE9C+EiG4R1k4Cjx5p88=
github.com/mattn/go-runewidth v0.0.19 h1:v++JhqYnZuu5jSKrk9RbgF5v4CGUjqRfBm05byFGLdw=
github.com/mattn/go-runewidth v0.0.19/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
github.com/mattn/go-sqlite3 v1.14.49 h1:B8jBHC3xhxZgxztrgruTuLucebnULQnx4W7cF7SAE9w=
github.com/mattn/go-sqlite3 v1.14.49/go.mod h1:6JTjA44L93a0QCyJef5YvlPoKXntQPjzWv5gtm9sB6w=
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 h1:ZK8zHtRHOkbHy6Mmr5D264iyp3TiX5OmNcI5cIARiQI=
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6/go.mod h1:CJlz5H+gyd6CUWT45Oy4q24RdLyn7Md9Vj2/ldJBSIo=
github.com/muesli/cancelreader v0.2.2 h1:3I4Kt4BQjOR54NavqnDogx/MIoWBFa0StPA8ELUXHmA=
github.com/muesli/cancelreader v0.2.2/go.mod h1:3XuTXfFS2VjM+HTLZY9Ak0l6eUKfijIfMUZ4EgX0QYo=
github.com/muesli/termenv v0.16.0 h1:S5AlUN9dENB57rsbnkPyfdGuWIlkmzJjbFf0Tf5FWUc=
github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3VfY/Cnk=
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
github.com/yalue/onnxruntime_go v1.13.0 h1:5HDXHon3EukQMyYA7yPMed/raWaDE/gjwLOwnVoiwy8=
github.com/yalue/onnxruntime_go v1.13.0/go.mod h1:b4X26A8pekNb1ACJ58wAXgNKeUCGEAQ9dmACut9Sm/4=
github.com/yanyiwu/gojieba v1.4.7 h1:2YkXELcYLTE0SJetq6xv4MjpEikWga6VpFn4jIFFQ/k=
github.com/yanyiwu/gojieba v1.4.7/go.mod h1:JUq4DddFVGdHXJHxxepxRmhrKlDpaBxR8O28v6fKYLY=
github.com/yuin/gopher-lua v1.1.2 h1:yF/FjE3hD65tBbt0VXLE13HWS9h34fdzJmrWRXwobGA=
github.com/yuin/gopher-lua v1.1.2/go.mod h1:7aRmXIWl37SqRf0koeyylBEzJ+aPt8A+mmkQ4f1ntR8=
golang.org/x/sys v0.8.0 h1:EBmGv8NaZBZTWvrbjNoL6HVt+IVy3QDQpJs7VRIw3tU=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/exp v0.0.0-20231006140011-7918f672742d h1:jtJma62tbqLibJ5sFQz8bKtEM8rJBtfilJ2qTU199MI=
golang.org/x/exp v0.0.0-20231006140011-7918f672742d/go.mod h1:ldy0pHrwJyGW56pPQzzkH36rKxoZW1tw7ZJpeKx+hdo=
golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/text v0.3.8 h1:nAL+RVCQ9uMn3vJZbV+MRnydTJFPf8qqY42YiA6MrqY=
golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=

View File

@ -7,6 +7,7 @@ import (
"encoding/json"
"fmt"
"io"
"net"
"net/http"
"sort"
"strings"
@ -185,10 +186,11 @@ type TokenUsage struct {
}
type ToolCall struct {
ID string `json:"id"`
Type string `json:"type"`
Name string `json:"name"`
Arguments map[string]interface{} `json:"arguments"`
ID string `json:"id"`
Type string `json:"type"`
Name string `json:"name"`
Arguments map[string]interface{} `json:"arguments"`
RawArguments string `json:"raw_arguments,omitempty"` // 流式分片原始 JSON 字符串
}
type apiToolCall struct {
@ -203,11 +205,13 @@ type apiFunction struct {
}
type StreamChunk struct {
Content string `json:"content"`
ReasoningContent string `json:"reasoning_content,omitempty"`
Done bool `json:"done"`
ToolCall *ToolCall `json:"tool_call,omitempty"`
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
Content string `json:"content"`
ReasoningContent string `json:"reasoning_content,omitempty"`
Done bool `json:"done"`
FinishReason string `json:"finish_reason,omitempty"`
ToolCall *ToolCall `json:"tool_call,omitempty"`
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
Usage *TokenUsage `json:"usage,omitempty"`
}
type Provider interface {
@ -284,6 +288,10 @@ type LuaAdaptedProvider struct {
vm *luaVM.VM
adapter string
client *http.Client
// streamClient 专用于 SSE 流式调用无整体超时SSE 长连接不被截断),
// 仅保留拨号超时。懒初始化,首次 ChatStream 时创建。
streamClient *http.Client
streamMu sync.Mutex
}
func NewLuaAdaptedProvider(cfg BaseConfig, vm *luaVM.VM, name, adapter string) *LuaAdaptedProvider {
@ -298,7 +306,10 @@ func NewLuaAdaptedProvider(cfg BaseConfig, vm *luaVM.VM, name, adapter string) *
cfg: cfg,
vm: vm,
adapter: adapter,
client: &http.Client{Timeout: 120 * time.Second},
// 180s: llmsproxy 的 AUTO 链会串行尝试多个 tier每个失败 tier 耗
// busyWait(2s)+上游超时120s 曾导致网关侧记录大量 "context canceled"
// (客户端先放弃)。放宽到 180s 给链式 failover 留足时间。
client: &http.Client{Timeout: 180 * time.Second},
}
}
@ -373,6 +384,11 @@ func (p *LuaAdaptedProvider) Chat(ctx context.Context, req *CompletionRequest) (
if parsed, perr := parseOpenAICompatibleResponse(rawResp); perr == nil {
return parsed, nil
}
// 网关在非流式请求下返回了 SSE 流 body上游恢复后吐 chunk 流),
// 拼接为完整响应,避免丢掉已生成的整段回复
if parsed, ok := parseOpenAICompatibleSSEBody(rawResp); ok {
return parsed, nil
}
return nil, fmt.Errorf("lua transform_response: %w", err)
}
@ -381,6 +397,9 @@ func (p *LuaAdaptedProvider) Chat(ctx context.Context, req *CompletionRequest) (
if parsed, perr := parseOpenAICompatibleResponse(rawResp); perr == nil {
return parsed, nil
}
if parsed, ok := parseOpenAICompatibleSSEBody(rawResp); ok {
return parsed, nil
}
return nil, fmt.Errorf("unmarshal unified response: %w (body: %s)", err, unifiedJSON)
}
@ -449,6 +468,86 @@ func parseOpenAICompatibleResponse(raw []byte) (*CompletionResponse, error) {
return out, nil
}
// parseOpenAICompatibleSSEBody 将 SSE 格式的响应体("data: {...}" 多行)
// 拼接为完整 CompletionResponse。场景网关llmsproxy auto 链等)在非流式
// 请求下也可能返回流式 body——上游恢复后吐出的是已生成的 chunk 流,若按
// 普通 JSON 解析会报 "invalid character 'd'" 而丢掉整段完整回复。
// 返回 false 表示 body 不是 SSE 格式,调用方继续走原有解析路径。
func parseOpenAICompatibleSSEBody(raw []byte) (*CompletionResponse, bool) {
body := strings.TrimSpace(string(raw))
if !strings.HasPrefix(body, "data:") && !strings.Contains(body, "\ndata:") {
return nil, false
}
type sseAcc struct {
id string
name string
argsRaw strings.Builder
}
var out CompletionResponse
var contentBuf, reasoningBuf strings.Builder
accs := map[int]*sseAcc{}
toolOrder := []int{}
finish := ""
found := false
for _, line := range strings.Split(body, "\n") {
line = strings.TrimSpace(line)
if !strings.HasPrefix(line, "data:") {
continue
}
payload := strings.TrimSpace(strings.TrimPrefix(line, "data:"))
if payload == "" || payload == "[DONE]" {
continue
}
ck, ok := parseOpenAICompatibleStreamChunkFull(payload)
if !ok {
continue
}
found = true
contentBuf.WriteString(ck.Content)
reasoningBuf.WriteString(ck.ReasoningContent)
for i, tc := range ck.ToolCalls {
acc := accs[i]
if acc == nil {
acc = &sseAcc{}
accs[i] = acc
toolOrder = append(toolOrder, i)
}
if tc.ID != "" {
acc.id = tc.ID
}
if tc.Name != "" {
acc.name = tc.Name
}
acc.argsRaw.WriteString(tc.RawArguments)
}
if ck.Done && ck.FinishReason != "" {
finish = ck.FinishReason
}
if ck.Usage != nil {
out.TokenUsage = *ck.Usage
}
}
if !found {
return nil, false
}
out.Content = contentBuf.String()
out.ReasoningContent = reasoningBuf.String()
out.FinishReason = finish
for _, i := range toolOrder {
acc := accs[i]
name := strings.TrimSpace(acc.name)
argsStr := strings.TrimSpace(acc.argsRaw.String())
if name == "" && argsStr == "" && acc.id == "" {
continue
}
tc := ToolCall{ID: acc.id, Type: "function", Name: name, RawArguments: argsStr}
tc.Arguments = parseToolArguments(argsStr)
out.ToolCalls = append(out.ToolCalls, tc)
}
return &out, true
}
type openAIToolCall struct {
ID string `json:"id"`
Type string `json:"type"`
@ -480,10 +579,56 @@ func normalizeOpenAIToolCalls(raw []openAIToolCall) []ToolCall {
typ = "function"
}
out = append(out, ToolCall{
ID: tc.ID,
Type: typ,
Name: name,
Arguments: parseToolArguments(argsRaw),
ID: tc.ID,
Type: typ,
Name: name,
Arguments: parseToolArguments(argsRaw),
RawArguments: rawArgsString(argsRaw),
})
}
return out
}
// rawArgsString 将 arguments 字段转为字符串形式(用于流式分片拼接)。
func rawArgsString(v interface{}) string {
switch x := v.(type) {
case nil:
return ""
case string:
return x
default:
b, _ := json.Marshal(x)
return string(b)
}
}
// normalizeStreamToolCalls 流式专用:保留无 name 的分片(后续 arguments
// 分片 name 为空,但携带 RawArguments 需要拼接),由调用方按 index 累积。
func normalizeStreamToolCalls(raw []openAIToolCall) []ToolCall {
if len(raw) == 0 {
return nil
}
out := make([]ToolCall, 0, len(raw))
for _, tc := range raw {
name := tc.Function.Name
argsRaw := tc.Function.Arguments
if name == "" {
name = tc.Name
// 仅当顶层 Arguments 存在才用扁平格式;否则保留 function.arguments 嵌套值
// OpenAI 流式续传 chunkname 不重发但 function.arguments 继续)
if tc.Arguments != nil {
argsRaw = tc.Arguments
}
}
typ := tc.Type
if typ == "" && (tc.ID != "" || name != "" || argsRaw != nil) {
typ = "function"
}
out = append(out, ToolCall{
ID: tc.ID,
Type: typ,
Name: name,
RawArguments: rawArgsString(argsRaw),
})
}
return out
@ -540,8 +685,11 @@ func stringifyContent(v interface{}) string {
}
}
func parseOpenAICompatibleStreamChunk(raw []byte) (StreamChunk, bool) {
var resp struct {
// parseOpenAICompatibleStreamChunkFull 解析标准 OpenAI SSE 块(含 usage 字段)。
// 兼容多种 token 用量键名prompt_tokens/prompt、total_tokens/total 等)
// 与 prompt cache 细节字段。返回 false 表示非内容块(纯 usage 心跳等)。
func parseOpenAICompatibleStreamChunkFull(data string) (StreamChunk, bool) {
var raw struct {
Choices []struct {
Delta struct {
Content interface{} `json:"content"`
@ -550,17 +698,100 @@ func parseOpenAICompatibleStreamChunk(raw []byte) (StreamChunk, bool) {
} `json:"delta"`
FinishReason *string `json:"finish_reason"`
} `json:"choices"`
UpstreamUsage struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
Prompt int `json:"prompt"`
Completion int `json:"completion"`
Total int `json:"total"`
PromptCacheHit int `json:"prompt_cache_hit_tokens"`
PromptCacheMiss int `json:"prompt_cache_miss_tokens"`
PromptTokensDetails *struct {
CachedTokens int `json:"cached_tokens"`
} `json:"prompt_tokens_details"`
} `json:"usage"`
}
if err := json.Unmarshal(raw, &resp); err != nil || len(resp.Choices) == 0 {
if err := json.Unmarshal([]byte(data), &raw); err != nil {
return StreamChunk{}, false
}
choice := resp.Choices[0]
return StreamChunk{
var usage *TokenUsage
pu := raw.UpstreamUsage
if pu.Total > 0 || pu.TotalTokens > 0 || pu.Prompt > 0 || pu.PromptTokens > 0 {
usage = &TokenUsage{
Prompt: pickFirstInt(pu.PromptTokens, pu.Prompt),
Completion: pickFirstInt(pu.CompletionTokens, pu.Completion),
Total: pickFirstInt(pu.TotalTokens, pu.Total),
}
}
if len(raw.Choices) == 0 {
// 纯 usage 心跳块:有 usage 就透传,否则丢弃
if usage != nil {
return StreamChunk{Usage: usage}, true
}
return StreamChunk{}, false
}
choice := raw.Choices[0]
ck := StreamChunk{
Content: stringifyContent(choice.Delta.Content),
ReasoningContent: choice.Delta.ReasoningContent,
ToolCalls: normalizeOpenAIToolCalls(choice.Delta.ToolCalls),
Done: choice.FinishReason != nil,
}, true
ToolCalls: normalizeStreamToolCalls(choice.Delta.ToolCalls),
Usage: usage,
}
// finish reason 为空字符串不算终止信号sensenova 每块都发 ""
if choice.FinishReason != nil && *choice.FinishReason != "" {
ck.Done = true
ck.FinishReason = *choice.FinishReason
}
return ck, true
}
// pickFirstInt 返回 a 非零时的 a否则 b兼容 *_tokens 与短键名两种 usage 格式)。
func pickFirstInt(a, b int) int {
if a != 0 {
return a
}
return b
}
// streamHTTPClient 返回专用的流式 HTTP client懒初始化
// SSE 长连接不能套整体超时(非流式 180s 会在长流中途报断),
// 只保留拨号/握手超时。
func (p *LuaAdaptedProvider) streamHTTPClient() *http.Client {
p.streamMu.Lock()
defer p.streamMu.Unlock()
if p.streamClient == nil {
p.streamClient = &http.Client{
Timeout: 0, // 无整体超时SSE 流持续时间不可预知
Transport: &http.Transport{
DialContext: (&net.Dialer{
Timeout: 30 * time.Second,
KeepAlive: 30 * time.Second,
}).DialContext,
ForceAttemptHTTP2: true,
MaxIdleConns: 10,
IdleConnTimeout: 90 * time.Second,
},
}
}
return p.streamClient
}
// errorOnlyChunk 判断一个流块是否只携带上游错误信号done 块带非标准
// finish_reason 且无任何内容/工具调用/推理文本。标准 OpenAI finish reason
// 不算错误正常的空补全finish_reason:"stop" 无输出)仍会送达调用方。
func errorOnlyChunk(ck StreamChunk) bool {
if !ck.Done || ck.FinishReason == "" {
return false
}
switch ck.FinishReason {
case "stop", "length", "tool_calls", "function_call", "content_filter":
return false
}
return ck.Content == "" && len(ck.ToolCalls) == 0 && ck.ReasoningContent == ""
}
func (p *LuaAdaptedProvider) ChatStream(ctx context.Context, req *CompletionRequest) (<-chan StreamChunk, error) {
@ -586,89 +817,129 @@ func (p *LuaAdaptedProvider) ChatStream(ctx context.Context, req *CompletionRequ
return nil, fmt.Errorf("create stream request: %w", err)
}
httpReq.Header.Set("Content-Type", "application/json")
httpReq.Header.Set("Authorization", "Bearer "+p.cfg.APIKey)
// 与 Chat() 一致走 applyAdapterHeaders支持 build_headers 动态签名钩子
p.applyAdapterHeaders(httpReq, url, transformedBody)
for k, v := range p.vm.GetAdapterHeaders(p.adapter) {
httpReq.Header.Set(k, v)
}
resp, err := p.client.Do(httpReq)
resp, err := p.streamHTTPClient().Do(httpReq)
if err != nil {
return nil, fmt.Errorf("stream api: %w", err)
}
if resp.StatusCode != 200 {
raw, _ := io.ReadAll(resp.Body)
resp.Body.Close()
// transform_error 钩子优先(适配器层协议知识);未定义时回退标准解析
if reason, ok, _ := p.vm.TransformError(p.adapter, resp.StatusCode, string(raw)); ok && strings.TrimSpace(reason) != "" {
return nil, fmt.Errorf("api error %d: %s", resp.StatusCode, truncateOneLineStr(reason, 200))
}
return nil, fmt.Errorf("api error %d: %s", resp.StatusCode, truncateOneLineStr(string(raw), 300))
}
ch := make(chan StreamChunk, 64)
go func() {
defer resp.Body.Close()
defer close(ch)
scanner := NewSSEScanner(resp.Body)
scanner := bufio.NewScanner(resp.Body)
scanner.Buffer(make([]byte, 0, 64*1024), 1024*1024)
var doneSent bool // 适配器已发过带真实 finish_reason 的终止块则不重复发 [DONE]
emit := func(ck StreamChunk) bool {
if ck.Done {
doneSent = true
}
select {
case ch <- ck:
return true
case <-ctx.Done():
return false
}
}
for scanner.Scan() {
line := scanner.Text()
if line == "" {
line := strings.TrimSpace(scanner.Text())
if line == "" || !strings.HasPrefix(line, "data:") {
continue
}
data := strings.TrimSpace(strings.TrimPrefix(line, "data:"))
if data == "" {
continue
}
if data == "[DONE]" {
if !doneSent {
if !emit(StreamChunk{Done: true}) {
return
}
}
continue
}
// 尝试用 Lua 变换流块(如果 adapter 定义了 transform_stream_chunk
unified, err := p.vm.CallTransformStreamChunk(p.adapter, line)
if err != nil || unified == line {
// 无流变换函数或变换透传,尝试标准 OpenAI SSE 解析
chunk, ok := parseOpenAICompatibleStreamChunk([]byte(line))
// Lua transform_stream_chunk 优先;透传/无钩子时用标准解析
unified, terr := p.vm.CallTransformStreamChunk(p.adapter, data)
var ck StreamChunk
if terr == nil && unified != "" && unified != data {
if json.Unmarshal([]byte(unified), &ck) != nil {
continue
}
} else {
parsed, ok := parseOpenAICompatibleStreamChunkFull(data)
if !ok {
continue
}
select {
case ch <- chunk:
case <-ctx.Done():
return
}
continue
ck = parsed
}
if !emit(ck) {
return
}
}
// Lua 返回了变换后的统一格式
var chunk StreamChunk
if err := json.Unmarshal([]byte(unified), &chunk); err == nil {
select {
case ch <- chunk:
case <-ctx.Done():
return
}
// 干净 EOF 但无 done 块:补一个,保证消费方能收到终止信号
if !doneSent && ctx.Err() == nil {
select {
case ch <- StreamChunk{Done: true}:
default:
}
}
}()
return ch, nil
}
// SSEScanner 读取 SSE 格式的流data: ...
type SSEScanner struct {
reader *bufio.Reader
pending string
}
func NewSSEScanner(r io.Reader) *SSEScanner {
return &SSEScanner{reader: bufio.NewReader(r)}
}
func (s *SSEScanner) Scan() bool {
s.pending = ""
for {
line, err := s.reader.ReadString('\n')
if err != nil {
return false
// 扣住首块校验流是否真的携带内容:部分上游返回 HTTP 200 但流里只有
// 错误 finish_reason 的退化块(如 zen 免费池 network_error。在这里
// 失败该候选,让上层 fallback 到下一源,而不是给客户端吐空响应。
select {
case first, ok := <-ch:
if !ok {
return nil, fmt.Errorf("provider %s: empty stream", p.Name())
}
line = strings.TrimRight(line, "\r\n")
if strings.HasPrefix(line, "data: ") {
s.pending = strings.TrimPrefix(line, "data: ")
if s.pending == "[DONE]" {
return false
if errorOnlyChunk(first) {
go func() {
for range ch { //nolint:revive
} // 排空避免生产 goroutine 阻塞泄漏
}()
return nil, fmt.Errorf("provider %s: upstream returned %q stream", p.Name(), first.FinishReason)
}
out := make(chan StreamChunk, 64)
go func() {
defer close(out)
out <- first
for ck := range ch {
out <- ck
}
return true
}
}()
return out, nil
case <-ctx.Done():
return nil, ctx.Err()
}
}
func (s *SSEScanner) Text() string { return s.pending }
// truncateOneLineStr 截断为单行且限制最大长度(用于错误消息防 HTML dump 泄漏)。
func truncateOneLineStr(s string, max int) string {
s = strings.ReplaceAll(s, "\n", " ")
s = strings.TrimSpace(s)
if len(s) > max {
s = s[:max] + "..."
}
return s
}
type providerStatus struct {
failCount int

View File

@ -0,0 +1,121 @@
package api
import (
"encoding/json"
"strings"
"testing"
)
// 锁定契约网关llmsproxy auto 链等)在非流式请求下返回 SSE 流 body 时,
// 必须拼接为完整响应而不是报 "invalid character 'd'" 丢掉已生成的回复。
// 事故样本取自 2026-08-25 生产日志:上游恢复后吐出完整 chunk 流被非流式解析器丢弃。
func TestParseOpenAICompatibleSSEBody(t *testing.T) {
body := "data: {\"id\":\"chatcmpl-572\",\"object\":\"chat.completion.chunk\",\"created\":1787630289,\"model\":\"x-preview-f-free\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"你好\"},\"finish_reason\":null}]}\n" +
"\n" +
"data: {\"id\":\"chatcmpl-572\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\",世界\"},\"finish_reason\":null}]}\n" +
"\n" +
"data: {\"id\":\"chatcmpl-572\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"\"},\"finish_reason\":\"stop\"}]}\n" +
"data: {\"id\":\"chatcmpl-572\",\"choices\":[],\"usage\":{\"prompt_tokens\":100,\"completion_tokens\":7,\"total_tokens\":107}}\n" +
"data: [DONE]\n"
resp, ok := parseOpenAICompatibleSSEBody([]byte(body))
if !ok {
t.Fatal("expected SSE body to be recognized")
}
if resp.Content != "你好,世界" {
t.Errorf("content = %q, want %q", resp.Content, "你好,世界")
}
if resp.FinishReason != "stop" {
t.Errorf("finish_reason = %q, want stop", resp.FinishReason)
}
if resp.TokenUsage.Total != 107 || resp.TokenUsage.Prompt != 100 || resp.TokenUsage.Completion != 7 {
t.Errorf("usage = %+v, want prompt=100 completion=7 total=107", resp.TokenUsage)
}
}
func TestParseOpenAICompatibleSSEBodyRejectsPlainJSON(t *testing.T) {
plain := `{"choices":[{"message":{"content":"hi"}}]}`
if _, ok := parseOpenAICompatibleSSEBody([]byte(plain)); ok {
t.Fatal("plain JSON body must not be treated as SSE")
}
}
func TestParseOpenAICompatibleSSEBodyToolCallShards(t *testing.T) {
// 用 json.Marshal 构建测试数据,避免 Go 字面量转义错误
chunk1 := map[string]interface{}{
"choices": []map[string]interface{}{{
"index": 0,
"delta": map[string]interface{}{
"tool_calls": []map[string]interface{}{{
"index": 0,
"id": "call_1",
"type": "function",
"function": map[string]interface{}{
"name": "exec",
"arguments": `{"command":`,
},
}},
},
}},
}
chunk2 := map[string]interface{}{
"choices": []map[string]interface{}{{
"index": 0,
"delta": map[string]interface{}{
"tool_calls": []map[string]interface{}{{
"index": 0,
"function": map[string]interface{}{
"arguments": `"date"}`,
},
}},
},
}},
}
chunk3 := map[string]interface{}{
"choices": []map[string]interface{}{{
"index": 0,
"delta": map[string]interface{}{},
"finish_reason": "tool_calls",
}},
}
var sb strings.Builder
for _, c := range []map[string]interface{}{chunk1, chunk2, chunk3} {
b, _ := json.Marshal(c)
sb.WriteString("data: ")
sb.Write(b)
sb.WriteString("\n")
}
sb.WriteString("data: [DONE]\n")
t.Logf("SSE body:\n%s", sb.String())
resp, ok := parseOpenAICompatibleSSEBody([]byte(sb.String()))
if !ok {
t.Fatal("expected SSE body to be recognized")
}
if len(resp.ToolCalls) != 1 {
t.Fatalf("got %d tool calls, want 1", len(resp.ToolCalls))
}
tc := resp.ToolCalls[0]
if tc.Name != "exec" || tc.ID != "call_1" {
t.Errorf("tool call name/id = %q/%q, want exec/call_1", tc.Name, tc.ID)
}
args := tc.RawArguments
if args != `{"command":"date"}` {
t.Errorf("raw args = %q", args)
}
if tc.Arguments["command"] != "date" {
t.Errorf("parsed args = %v, want command=date", tc.Arguments)
}
if resp.FinishReason != "tool_calls" {
t.Errorf("finish_reason = %q, want tool_calls", resp.FinishReason)
}
}
func TestParseOpenAICompatibleSSEBodyEmptyStream(t *testing.T) {
body := "data: \ndata: \n"
if resp, ok := parseOpenAICompatibleSSEBody([]byte(body)); ok && strings.TrimSpace(resp.Content) != "" {
t.Fatalf("empty stream should not parse into non-empty response")
}
}

View File

@ -76,8 +76,11 @@ type Agent struct {
eventBus *events.Bus
pluginHealth *pluginHealthTracker
// 自循环输入通道:核心内部任务(记忆消歧、系统维护),不经过 IO 层
selfInputCh chan string
// 自循环输入通道:核心内部任务(记忆消歧、系统维护、子 Agent 通知
// 不经过 IO 层。每条消息携带目标输出通道:
// "_consolidation_" = 记忆整理(无记忆路径,不写入上下文、不 emit 响应)
// 其他 = 正常处理写入上下文、emit 响应到该通道)
selfInputCh chan selfInputMsg
// 子任务异步执行
childMu sync.Mutex
@ -111,6 +114,11 @@ type Agent struct {
noMergeMarkers map[string]int
noMergeMu sync.Mutex
// 输入去重:防 webui/GUI 断线重连导致的消息重放
// key=source+"|"+content, value=上次接收时间;短窗口内同内容丢弃
lastInput map[string]time.Time
lastInputMu sync.Mutex
// 词嵌入模型,用于实体语义相似度计算
embedder *memory.StaticEmbedder
}
@ -213,7 +221,7 @@ func New(cfg AgentConfig) *Agent {
maxContextSize: cfg.MaxContextSize,
stageHost: cfg.StageHost,
eventBus: cfg.EventBus,
selfInputCh: make(chan string, 64),
selfInputCh: make(chan selfInputMsg, 64),
childResults: make(map[string]string),
interceptCh: make(chan *agentIO.InputEvent, 64),
pluginHealth: newPluginHealthTracker(),
@ -221,6 +229,7 @@ func New(cfg AgentConfig) *Agent {
inputCfg: cfg.InputProcessing,
embedder: embedder,
noMergeMarkers: make(map[string]int),
lastInput: make(map[string]time.Time),
}
}
@ -240,17 +249,54 @@ func (a *Agent) Stop() {
func (a *Agent) ID() types.AgentID { return a.id }
// isDuplicateInput 判断是否为短窗口内的重复输入(防 webui/GUI 断线重连消息重放)。
// key=source+"|"+content窗口内重复返回 true 并刷新时间戳(持续轰炸时保持拦截)。
const duplicateInputWindow = 10 * time.Second
func (a *Agent) isDuplicateInput(source, content string) bool {
a.lastInputMu.Lock()
defer a.lastInputMu.Unlock()
now := time.Now()
key := source + "|" + content
if last, ok := a.lastInput[key]; ok && now.Sub(last) < duplicateInputWindow {
a.lastInput[key] = now
return true
}
a.lastInput[key] = now
// 顺带清理过期项,防止 map 无限增长
for k, t := range a.lastInput {
if now.Sub(t) > duplicateInputWindow {
delete(a.lastInput, k)
}
}
return false
}
// IsDuplicateInput 导出包装,供测试验证去重行为。
func (a *Agent) IsDuplicateInput(source, content string) bool {
return a.isDuplicateInput(source, content)
}
// SelfInputChan 返回自循环输入通道(只读,供内部测试验证)
func (a *Agent) SelfInputChan() <-chan string {
func (a *Agent) SelfInputChan() <-chan selfInputMsg {
return a.selfInputCh
}
// injectSelf 向自循环通道发送内部任务(记忆消歧、系统维护)
// 线程安全,不阻塞发送者(通道缓冲 64
// injectSelf 向自循环通道发送记忆整理类内部任务(记忆路径)。
// 线程安全,不阻塞发送者(通道缓冲 64
func (a *Agent) injectSelf(task string) {
a.injectSelfChannel(selfInputMsg{
text: task,
channel: channelConsolidation,
})
}
// injectSelfChannel 向自循环通道发送一条带目标通道标志的消息。
// channel == "_consolidation_" 走无记忆整理路径;其他值走正常处理路径。
func (a *Agent) injectSelfChannel(msg selfInputMsg) {
select {
case a.selfInputCh <- task:
case a.selfInputCh <- msg:
default:
log.Printf("[agent] self input channel full, dropping task: %s", truncateStr(task, 80))
log.Printf("[agent] self input channel full, dropping task: %s", truncateStr(msg.text, 80))
}
}

View File

@ -24,8 +24,8 @@ func (a *Agent) eventLoop() {
select {
case evt := <-a.io.InputChan():
a.handleInput(evt)
case task := <-a.selfInputCh:
a.handleSelfInput(task)
case msg := <-a.selfInputCh:
a.handleSelfInput(msg)
case <-a.ctx.Done():
return
}
@ -108,13 +108,29 @@ func (a *Agent) interceptLoop() {
}
}
func (a *Agent) handleSelfInput(task string) {
// channelConsolidation 标记记忆整理类自输入:无记忆路径处理,
// 不写入对话上下文、不向任何输出通道 emit 响应。
const channelConsolidation = "_consolidation_"
// selfInputMsg 自循环输入消息。channel 决定处理路径:
// - channelConsolidation记忆整理无记忆不污染上下文/知识库)
// - 其他值(如 "cli"、"webui"):正常输入路径,写入上下文并 emit 响应
// (典型场景:子 Agent 完成通知,需让父 Agent 感知并可回复用户)
type selfInputMsg struct {
text string
channel string
}
func (a *Agent) handleSelfInput(msg selfInputMsg) {
if msg.channel == "" {
msg.channel = channelConsolidation // 兼容空值:默认走整理路径
}
a.processTextInput(&agentIO.InputEvent{
Source: "system",
Type: "text",
Payload: map[string]interface{}{"content": task},
OutputChannel: "_consolidation_",
}, task)
Payload: map[string]interface{}{"content": msg.text},
OutputChannel: msg.channel,
}, msg.text)
}
func (a *Agent) handleInput(evt *agentIO.InputEvent) {
@ -124,6 +140,11 @@ func (a *Agent) handleInput(evt *agentIO.InputEvent) {
if input == "" {
return
}
// 去重webui/GUI 断线重连会重放未确认消息,短窗口内同来源同内容丢弃,避免轰炸
if a.isDuplicateInput(evt.Source, input) {
log.Printf("[agent] dropped duplicate input from %s: %s", evt.Source, truncateStr(input, 60))
return
}
a.processTextInput(evt, input)
case "image", "audio":

View File

@ -2,10 +2,12 @@ package core
import (
"context"
"encoding/json"
"errors"
"fmt"
"log"
"strings"
"time"
agentAPI "gitcode.com/JianFeeeee/HomeAgent/internal/agent/api"
"gitcode.com/JianFeeeee/HomeAgent/internal/events"
@ -115,28 +117,61 @@ func (a *Agent) process(input string, stageCtx *sdk.StageContext) (response stri
fbProvider.Name(), pi, len(providers)-1)
}
fCtx, fCancel := context.WithCancel(a.ctx)
a.llmMu.Lock()
a.cancelLLM = fCancel
a.llmMu.Unlock()
resp, llmErr = fbProvider.Chat(fCtx, req)
a.llmMu.Lock()
a.cancelLLM = nil
a.llmMu.Unlock()
fCancel()
if llmErr == nil {
a.providerManager.ResetAvailability(fbProvider.Name())
if fbProvider != a.provider {
a.provider = fbProvider
log.Printf("[agent] switched active provider to %q after fallback",
fbProvider.Name())
// 同源瞬时错误重试网关瞬断502/503/504/429/网络抖动)通常秒级恢复,
// 直接跳下一个 provider或直接报错会丢掉本可成功的请求。
// 凭证错误401/403与用户中断不重试。
const maxAttempts = 2
for attempt := 1; attempt <= maxAttempts; attempt++ {
if attempt > 1 {
log.Printf("[agent] provider %q transient failure, retry %d/%d in 2s: %v",
fbProvider.Name(), attempt, maxAttempts, llmErr)
select {
case <-time.After(2 * time.Second):
case <-a.ctx.Done():
llmErr = a.ctx.Err()
}
if llmErr == nil || errors.Is(llmErr, context.Canceled) || errors.Is(llmErr, context.DeadlineExceeded) {
break
}
}
break
fCtx, fCancel := context.WithCancel(a.ctx)
a.llmMu.Lock()
a.cancelLLM = fCancel
a.llmMu.Unlock()
resp, llmErr = chatStreamWithFallback(fCtx, fbProvider, req, a)
a.llmMu.Lock()
a.cancelLLM = nil
a.llmMu.Unlock()
fCancel()
if llmErr == nil {
a.providerManager.ResetAvailability(fbProvider.Name())
if fbProvider != a.provider {
a.provider = fbProvider
log.Printf("[agent] switched active provider to %q after fallback",
fbProvider.Name())
}
break
}
// 用户中断:立即终止,不重试也不换 provider
if errors.Is(llmErr, context.Canceled) {
break
}
// 凭证错误:重试无意义,跳出重试循环进入 provider 标记/切换
var pe *agentAPI.ProviderError
if errors.As(llmErr, &pe) && (pe.StatusCode == 401 || pe.StatusCode == 403) {
break
}
// 其余错误(含 5xx/429/网络):还有重试机会则继续,否则跳出
}
if llmErr == nil {
break
}
if errors.Is(llmErr, context.Canceled) {
break
}
@ -294,6 +329,172 @@ func (a *Agent) process(input string, stageCtx *sdk.StageContext) (response stri
}
}
// chatStreamWithFallback 优先流式调用 provider失败时回退非流式 Chat()。
//
// 流式路径ChatStream 拿到 chunk channel逐块累积 content/reasoning_content
// 并发布 EventReasoningDelta / EventContentDelta 增量事件(新订阅者可选订,
// 旧订阅者不认识自然忽略)。流结束后拼出与 Chat() 等价的 CompletionResponse
// 返回——process() 的后续逻辑stageCtx/聚合事件/工具循环)完全不变。
//
// 回退条件ChatStream 返回错误连接失败、provider 不支持流式)。
// 已收到部分 chunk 后出错则不回退(避免重复生成),直接返回已累积内容。
//
// 超时收益:首包 ~1-3s 到达即建立活性,后续只要 token 在流动就不会触发
// 空闲超时;总生成时长不再受限於 180s 整体超时。
func chatStreamWithFallback(ctx context.Context, p agentAPI.Provider, req *agentAPI.CompletionRequest, a *Agent) (*agentAPI.CompletionResponse, error) {
ch, err := p.ChatStream(ctx, req)
if err != nil {
log.Printf("[agent] stream connect failed (%v), falling back to non-stream chat", err)
return p.Chat(ctx, req)
}
resp, accErr := accumulateStream(ctx, ch, a)
// 中断/超时取消必须保持取消语义传给调用方(与原 Chat() 行为一致:
// 被 cancel 时丢弃已收内容返回 err让 process() 的 continue 分支
// 重启轮次并以 [中断消息] 注入打断内容。绝不能把部分内容当成功返回,
// 否则用户打断会被无视、继续执行工具/输出。
if errors.Is(accErr, context.Canceled) || errors.Is(accErr, context.DeadlineExceeded) {
// 通知客户端:本轮流式作废,清空 delta 累积并定格已显示内容
if a != nil {
a.publishEvent(events.EventContentDelta, map[string]interface{}{
"content": "",
"channel": a.currentOutputChannel,
"reset": true,
})
}
return resp, accErr
}
if accErr == nil {
return resp, nil
}
// 其他错误(网络中断等):已累积到实质内容则返回部分结果,否则回退非流式
if resp != nil && (resp.Content != "" || len(resp.ToolCalls) > 0) {
log.Printf("[agent] stream interrupted mid-way (%v), returning partial result", accErr)
return resp, nil
}
log.Printf("[agent] stream failed before content (%v), falling back to non-stream chat", accErr)
return p.Chat(ctx, req)
}
// toolCallAcc 累积流式 tool call 的各个分片。OpenAI 风格:每个 index 的
// id/name/arguments 跨多个 chunk 增量到达arguments 是 JSON 字符串分片。
type toolCallAcc struct {
id string
name string
argsRaw strings.Builder
}
// accumulateStream 消费 chunk channel累积为完整 CompletionResponse
// 同时发布增量事件。返回的 response 与非流式 Chat() 的返回等价。
func accumulateStream(ctx context.Context, ch <-chan agentAPI.StreamChunk, a *Agent) (*agentAPI.CompletionResponse, error) {
resp := &agentAPI.CompletionResponse{
ToolCalls: make([]agentAPI.ToolCall, 0),
}
accs := make(map[int]*toolCallAcc) // index → 累积中的 tool call
var lastFinish string
flushToolCall := func(idx int) {
acc := accs[idx]
if acc == nil {
return
}
if acc.name == "" {
delete(accs, idx)
return
}
tc := agentAPI.ToolCall{
ID: acc.id,
Name: acc.name,
Arguments: parseToolArgsJSON(acc.argsRaw.String()),
}
resp.ToolCalls = append(resp.ToolCalls, tc)
delete(accs, idx)
}
for {
select {
case ck, ok := <-ch:
if !ok {
for idx := range accs {
flushToolCall(idx)
}
if lastFinish != "" {
resp.FinishReason = lastFinish
}
return resp, nil
}
if ck.ReasoningContent != "" {
resp.ReasoningContent += ck.ReasoningContent
if a != nil {
a.publishEvent(events.EventReasoningDelta, map[string]interface{}{
"content": ck.ReasoningContent,
"channel": a.currentOutputChannel,
})
}
}
if ck.Content != "" {
resp.Content += ck.Content
if a != nil {
a.publishEvent(events.EventContentDelta, map[string]interface{}{
"content": ck.Content,
"channel": a.currentOutputChannel,
})
}
}
// 增量 tool call 分片OpenAI 风格按 index 拼接 id/name/arguments
for i, tc := range ck.ToolCalls {
idx := i
acc := accs[idx]
if acc == nil {
acc = &toolCallAcc{}
accs[idx] = acc
}
if tc.ID != "" {
acc.id = tc.ID
}
if tc.Name != "" {
acc.name = tc.Name
}
// arguments 以 JSON 字符串分片到达OpenAI 标准),拼接后最终解析
if tc.RawArguments != "" {
acc.argsRaw.WriteString(tc.RawArguments)
}
}
if ck.Done && ck.FinishReason != "" {
lastFinish = ck.FinishReason
}
if ck.Usage != nil {
resp.TokenUsage = *ck.Usage
}
case <-ctx.Done():
for idx := range accs {
flushToolCall(idx)
}
return resp, ctx.Err()
}
}
}
// parseToolArgsJSON 将经过完整拼接的 tool call arguments JSON 字符串解析为 map。
// 空字符串返回空 map。
func parseToolArgsJSON(s string) map[string]interface{} {
if s == "" {
return map[string]interface{}{}
}
var m map[string]interface{}
if err := json.Unmarshal([]byte(s), &m); err == nil && m != nil {
return m
}
return map[string]interface{}{}
}
func convertToolCalls(tcs []agentAPI.ToolCall) []sdk.ToolCall {
if tcs == nil {
return nil

View File

@ -19,12 +19,19 @@ func (a *Agent) executeSpawnChild(tc agentAPI.ToolCall) string {
taskID := fmt.Sprintf("child_%d", a.childNextID)
a.childMu.Unlock()
go a.runChildTask(taskID, task)
// 捕获父 Agent 当前输出通道:子任务完成通知需回到发起对话的通道,
// 让父 Agent 正常感知并可回复用户(而非走无记忆整理路径丢失通知)。
parentChannel := a.currentOutputChannel
if parentChannel == "" || parentChannel == channelConsolidation {
parentChannel = "cli"
}
go a.runChildTask(taskID, task, parentChannel)
return fmt.Sprintf("子任务已启动ID: %s完成后会自动通知你届时请使用 child_result 工具查看输出", taskID)
}
func (a *Agent) runChildTask(taskID, task string) {
func (a *Agent) runChildTask(taskID, task string, parentChannel string) {
if a.provider == nil {
log.Printf("[child] %s failed: no LLM provider configured", taskID)
return
@ -105,11 +112,10 @@ func (a *Agent) runChildTask(taskID, task string) {
log.Printf("[child] %s done: %s", taskID, truncateStr(finalResult, 100))
notification := fmt.Sprintf("子任务 %s 已完成,请调用 child_result 工具查看输出", taskID)
select {
case a.selfInputCh <- notification:
default:
log.Printf("[child] self input channel full, dropping notification for %s", taskID)
}
a.injectSelfChannel(selfInputMsg{
text: notification,
channel: parentChannel, // 回到父对话通道,正常处理(写入上下文 + emit 响应)
})
}
func (a *Agent) executeChildResultTool(tc agentAPI.ToolCall) string {
@ -122,13 +128,7 @@ func (a *Agent) executeChildResultTool(tc agentAPI.ToolCall) string {
result, ok := a.childResults[taskID]
if !ok {
a.childMu.Unlock()
a.childMu.Lock()
_, exists := a.childResults[taskID]
a.childMu.Unlock()
if !exists {
return fmt.Sprintf("子任务 %s 不存在或已过期", taskID)
}
return fmt.Sprintf("子任务 %s 不存在或已过期", taskID)
}
delete(a.childResults, taskID)
a.childMu.Unlock()

View File

@ -70,6 +70,14 @@ func (a *Agent) injectSourceContext(stageCtx *sdk.StageContext, evt *agentIO.Inp
channel = source
}
content := fmt.Sprintf("当前输入来源: %s默认输出通道: %s。", source, channel)
// 来源含设备身份webui/{device_id})时补充设备名,便于 agent 区分多设备输入
if devID, _ := evt.Payload["device_id"].(string); devID != "" {
devName, _ := evt.Payload["device_name"].(string)
if devName == "" {
devName = devID
}
content = fmt.Sprintf("当前输入来自设备[%s]%s默认输出通道: %s。", devName, devID, channel)
}
if flag, _ := evt.Payload["interrupt"].(bool); flag {
content = fmt.Sprintf("这是一条打断输入。来源: %s默认输出通道: %s。", source, channel)
}

View File

@ -0,0 +1,69 @@
package core
import (
"context"
"testing"
agentAPI "gitcode.com/JianFeeeee/HomeAgent/internal/agent/api"
)
// 验证流式 tool call 分片累积:模拟 llmsproxy/big-pickle 的分片序列
func TestAccumulateStreamToolCalls(t *testing.T) {
ch := make(chan agentAPI.StreamChunk, 10)
go func() {
// 分片1: name + id + arguments 开头
ch <- agentAPI.StreamChunk{ToolCalls: []agentAPI.ToolCall{
{ID: "call_1", Name: "cmd_run", RawArguments: "{\""},
}}
// 分片2-3: 只有 arguments 分片
ch <- agentAPI.StreamChunk{ToolCalls: []agentAPI.ToolCall{
{RawArguments: "command\""},
}}
ch <- agentAPI.StreamChunk{ToolCalls: []agentAPI.ToolCall{
{RawArguments: ":\"date\"}"},
}}
ch <- agentAPI.StreamChunk{Done: true, FinishReason: "tool_calls"}
close(ch)
}()
resp, err := accumulateStream(context.Background(), ch, nil)
if err != nil {
t.Fatalf("accumulateStream: %v", err)
}
if len(resp.ToolCalls) != 1 {
t.Fatalf("want 1 tool call, got %d", len(resp.ToolCalls))
}
tc := resp.ToolCalls[0]
if tc.Name != "cmd_run" || tc.ID != "call_1" {
t.Fatalf("bad name/id: %s/%s", tc.ID, tc.Name)
}
cmd, _ := tc.Arguments["command"].(string)
if cmd != "date" {
t.Fatalf("arguments not merged, got: %v", tc.Arguments)
}
if resp.FinishReason != "tool_calls" {
t.Fatalf("finish reason: %q", resp.FinishReason)
}
}
// 验证 content/reasoning 增量累积
func TestAccumulateStreamContent(t *testing.T) {
ch := make(chan agentAPI.StreamChunk, 5)
go func() {
ch <- agentAPI.StreamChunk{ReasoningContent: "think "}
ch <- agentAPI.StreamChunk{Content: "你"}
ch <- agentAPI.StreamChunk{Content: "好"}
ch <- agentAPI.StreamChunk{Done: true, FinishReason: "stop"}
close(ch)
}()
resp, err := accumulateStream(context.Background(), ch, nil)
if err != nil {
t.Fatalf("accumulateStream: %v", err)
}
if resp.Content != "你好" {
t.Fatalf("content: %q", resp.Content)
}
if resp.ReasoningContent != "think " {
t.Fatalf("reasoning: %q", resp.ReasoningContent)
}
}

View File

@ -0,0 +1,101 @@
package client
// BinaryChunker 提供二进制数据分块传输功能。
// 用于将大体积数据(如录像 mp4、大图片按分块协议发送。
// 协议:
// cmd_data_start {op, req_id, kind, total, chunk_size, mime} —— 文本帧
// <N 个二进制帧 0x2> —— data bytes
// cmd_data_end {op, req_id, status:ok|error, error?} —— 文本帧
const (
// DefaultChunkSize 默认分块大小8KB
DefaultChunkSize = 8192
// MaxBinaryFrameSize 二进制帧最大大小8MB
MaxBinaryFrameSize = 8 << 20
)
// ChunkCallback 分块发送回调,用于逐块处理。
type ChunkCallback func(chunk []byte) error
// ChunkData 将数据按指定大小分块。
func ChunkData(data []byte, chunkSize int) [][]byte {
if chunkSize <= 0 {
chunkSize = DefaultChunkSize
}
total := len(data)
var chunks [][]byte
for off := 0; off < total; off += chunkSize {
end := off + chunkSize
if end > total {
end = total
}
chunks = append(chunks, data[off:end])
}
return chunks
}
// SendChunked 使用回调逐块发送数据。
func SendChunked(data []byte, chunkSize int, fn ChunkCallback) error {
if chunkSize <= 0 {
chunkSize = DefaultChunkSize
}
chunks := ChunkData(data, chunkSize)
for _, chunk := range chunks {
if err := fn(chunk); err != nil {
return err
}
}
return nil
}
// ===== 数据聚合(接收端) =====
// DataAccumulator 聚合从设备接收的二进制分块数据。
type DataAccumulator struct {
ReqID string
Kind string
MIME string
Total int
Got int
Chunks [][]byte
}
// NewDataAccumulator 创建数据聚合器。
func NewDataAccumulator(reqID, kind, mime string, total int) *DataAccumulator {
return &DataAccumulator{
ReqID: reqID,
Kind: kind,
MIME: mime,
Total: total,
Chunks: make([][]byte, 0),
}
}
// Append 追加一块数据。
func (da *DataAccumulator) Append(chunk []byte) {
da.Chunks = append(da.Chunks, chunk)
da.Got += len(chunk)
}
// Assemble 聚合所有分块为完整数据。
func (da *DataAccumulator) Assemble() []byte {
total := 0
for _, c := range da.Chunks {
total += len(c)
}
data := make([]byte, 0, total)
for _, c := range da.Chunks {
data = append(data, c...)
}
return data
}
// ExceededLimit 检查是否超出限制(声明的 2 倍或硬上限 64MB
func (da *DataAccumulator) ExceededLimit() bool {
limit := da.Total*2 + 1024
if limit < 64<<20 {
limit = 64 << 20
}
return da.Got > limit
}

View File

@ -0,0 +1,530 @@
package client
import (
"encoding/json"
"fmt"
"log"
"os"
"runtime"
"sync"
"time"
)
// CmdHandler 是命令处理回调类型。
// 当收到 remotedevice 下发的 cmd 时调用reqID 用于回执command 是命令内容。
type CmdHandler func(reqID, command string)
// CmdResult 是命令执行结果回调(用于异步通知 GUI 层)。
type CmdResultHandler func(reqID, status, output, errMsg string)
// DataHandler 是二进制数据接收回调(如 TTS 音频)。
type DataHandler func(reqID, kind, mime string, data []byte)
// Bridge 是设备桥客户端核心结构体。
// 管理 WebSocket 连接、消息路由、心跳保活和命令分发。
// 授权状态由设备端本地存储(客户端鉴权),服务端不存储;
// 未授权时收到 cmd 直接拒绝执行并回执 error。
type Bridge struct {
mu sync.RWMutex
gateway string
token string
deviceID string
name string
kind string
caps []string
info map[string]interface{}
authorized bool // 客户端本地授权状态(用户在设备上手动开启)
ws *wsConn
stopCh chan struct{}
doneCh chan struct{}
started bool
// 回调
cmdHandler CmdHandler
resultHandler CmdResultHandler
dataHandler DataHandler
// 二进制数据聚合(服务端→设备,如 TTS 音频)
speechAccum *speechBuffer
// 心跳间隔
pingInterval time.Duration
}
// speechBuffer 聚合服务端分块推送的二进制数据。
type speechBuffer struct {
reqID string
kind string
mime string
total int
data []byte
}
// New 创建设备桥客户端。
// gateway: ws://host:port可选 /api/v1/device/ws 路径)
// token: 接入令牌
// deviceID: 设备唯一标识
// name: 设备显示名称
// caps: 能力列表(如 ["status","cmdrun","screensee","computeruse"]
// info: 额外设备信息hostname, platform, arch 等),可为 nil
func New(gateway, token, deviceID, name string, caps []string, info map[string]interface{}) *Bridge {
if info == nil {
info = make(map[string]interface{})
}
// 填充默认信息
if _, ok := info["hostname"]; !ok {
hostname, _ := os.Hostname()
info["hostname"] = hostname
}
if _, ok := info["platform"]; !ok {
info["platform"] = runtime.GOOS
}
if _, ok := info["arch"]; !ok {
info["arch"] = runtime.GOARCH
}
if _, ok := info["cpus"]; !ok {
info["cpus"] = runtime.NumCPU()
}
return &Bridge{
gateway: gateway,
token: token,
deviceID: deviceID,
name: name,
kind: "computer",
caps: caps,
info: info,
stopCh: make(chan struct{}),
doneCh: make(chan struct{}),
pingInterval: 30 * time.Second,
}
}
// SetAuthorized 设置客户端本地授权状态(用户在设备上手动开启)。
// 授权后立即重新发送 hello 同步到服务端展示。
func (b *Bridge) SetAuthorized(auth bool) {
b.mu.Lock()
b.authorized = auth
b.mu.Unlock()
// 重新 hello 同步状态
b.mu.RLock()
ws := b.ws
connected := ws != nil && !ws.closed
b.mu.RUnlock()
if connected {
b.sendJSON(map[string]interface{}{
"op": "hello",
"device": map[string]interface{}{
"device_id": b.deviceID,
"name": b.name,
"kind": b.kind,
"caps": b.caps,
"info": b.info,
"authorized": auth,
},
})
}
}
// Authorized 返回当前客户端本地授权状态。
func (b *Bridge) Authorized() bool {
b.mu.RLock()
defer b.mu.RUnlock()
return b.authorized
}
// OnCmd 注册命令处理器。当收到 remotedevice 下发的 cmd 时调用。
func (b *Bridge) OnCmd(handler CmdHandler) {
b.mu.Lock()
defer b.mu.Unlock()
b.cmdHandler = handler
}
// OnResult 注册命令结果回调(用于异步通知)。
func (b *Bridge) OnResult(handler CmdResultHandler) {
b.mu.Lock()
defer b.mu.Unlock()
b.resultHandler = handler
}
// OnData 注册二进制数据接收回调(如 TTS 音频)。
func (b *Bridge) OnData(handler DataHandler) {
b.mu.Lock()
defer b.mu.Unlock()
b.dataHandler = handler
}
// SetPingInterval 设置心跳间隔(默认 30 秒)。
func (b *Bridge) SetPingInterval(d time.Duration) {
b.mu.Lock()
defer b.mu.Unlock()
b.pingInterval = d
}
// Start 启动设备桥连接。
// 会阻塞直到连接建立或超时失败。
func (b *Bridge) Start() error {
b.mu.Lock()
if b.started {
b.mu.Unlock()
return fmt.Errorf("devicebridge: already started")
}
b.started = true
b.mu.Unlock()
ws, err := dialWS(b.gateway, b.token, 10*time.Second)
if err != nil {
b.mu.Lock()
b.started = false
b.mu.Unlock()
return fmt.Errorf("devicebridge: dial: %w", err)
}
b.mu.Lock()
b.ws = ws
b.mu.Unlock()
// 发送 hello含设备自报的授权状态服务端仅展示不决策
b.mu.RLock()
auth := b.authorized
b.mu.RUnlock()
b.sendJSON(map[string]interface{}{
"op": "hello",
"device": map[string]interface{}{
"device_id": b.deviceID,
"name": b.name,
"kind": b.kind,
"caps": b.caps,
"info": b.info,
"authorized": auth,
},
})
// 发送 bind
b.sendJSON(map[string]interface{}{
"op": "bind",
"device_id": b.deviceID,
"token": b.token,
})
go b.readLoop()
go b.pingLoop()
return nil
}
// Stop 停止设备桥连接。
func (b *Bridge) Stop() {
b.mu.Lock()
defer b.mu.Unlock()
if !b.started {
return
}
select {
case <-b.stopCh:
return
default:
close(b.stopCh)
}
if b.ws != nil {
_ = b.ws.close()
b.ws = nil
}
}
// Wait 等待设备桥连接关闭。
func (b *Bridge) Wait() {
<-b.doneCh
}
// DeviceID 返回设备 ID。
func (b *Bridge) DeviceID() string {
b.mu.RLock()
defer b.mu.RUnlock()
return b.deviceID
}
// Connected 返回是否已连接。
func (b *Bridge) Connected() bool {
b.mu.RLock()
defer b.mu.RUnlock()
return b.ws != nil && !b.ws.closed
}
// ===== 发送消息 =====
// SendResult 发送命令执行结果。
func (b *Bridge) SendResult(reqID, status, output, errMsg string) {
msg := map[string]interface{}{
"op": "cmd_result",
"req_id": reqID,
"status": status,
"device_id": b.deviceID,
}
if output != "" {
msg["output"] = output
}
if errMsg != "" {
msg["error"] = errMsg
}
b.sendJSON(msg)
}
// SendDataStart 开始二进制数据传输(设备→网关,如录像回传)。
func (b *Bridge) SendDataStart(reqID, kind, mime string, total int) {
b.sendJSON(map[string]interface{}{
"op": "cmd_data_start",
"req_id": reqID,
"kind": kind,
"mime": mime,
"total": total,
"chunk_size": 8192,
})
}
// SendDataChunk 发送一块二进制数据。
func (b *Bridge) SendDataChunk(data []byte) error {
b.mu.RLock()
ws := b.ws
b.mu.RUnlock()
if ws == nil || ws.closed {
return fmt.Errorf("devicebridge: not connected")
}
return ws.writeBinary(data)
}
// SendDataEnd 结束二进制数据传输。
func (b *Bridge) SendDataEnd(reqID, status, errMsg string) {
msg := map[string]interface{}{
"op": "cmd_data_end",
"req_id": reqID,
"status": status,
}
if errMsg != "" {
msg["error"] = errMsg
}
b.sendJSON(msg)
}
// SendDataChunked 便捷方法:自动分块发送完整二进制数据。
func (b *Bridge) SendDataChunked(reqID, kind, mime string, data []byte) {
total := len(data)
b.SendDataStart(reqID, kind, mime, total)
const chunkSize = 8192
for off := 0; off < total; off += chunkSize {
end := off + chunkSize
if end > total {
end = total
}
if err := b.SendDataChunk(data[off:end]); err != nil {
b.SendDataEnd(reqID, "error", err.Error())
return
}
}
b.SendDataEnd(reqID, "ok", "")
}
// SendEvent 发送设备主动上报事件。
func (b *Bridge) SendEvent(eventType string, payload interface{}) {
b.sendJSON(map[string]interface{}{
"op": "event",
"device_id": b.deviceID,
"type": eventType,
"payload": payload,
})
}
// SendStatus 发送设备状态更新。
func (b *Bridge) SendStatus(status string) {
b.sendJSON(map[string]interface{}{
"op": "status",
"device_id": b.deviceID,
"status": status,
})
}
// ===== 内部方法 =====
func (b *Bridge) sendJSON(v interface{}) {
b.mu.RLock()
ws := b.ws
b.mu.RUnlock()
if ws == nil || ws.closed {
return
}
payload := mustJSON(v)
_ = ws.writeText(payload)
}
func (b *Bridge) readLoop() {
defer func() {
b.mu.Lock()
b.started = false
if b.ws != nil {
_ = b.ws.close()
b.ws = nil
}
b.mu.Unlock()
close(b.doneCh)
}()
for {
select {
case <-b.stopCh:
return
default:
}
// 设置读超时2 倍 ping 间隔)
b.mu.RLock()
ws := b.ws
interval := b.pingInterval
b.mu.RUnlock()
if ws == nil {
return
}
ws.setDeadline(time.Now().Add(interval * 2))
payload, isClose, opcode, err := ws.readFrame()
if err != nil {
if err == errPing {
_ = ws.writePong()
continue
}
// 超时或其他错误,退出
return
}
if isClose {
return
}
if opcode == 0x2 {
// 二进制帧:处于聚合状态时追加
b.handleBinaryFrame(payload)
continue
}
var msg map[string]interface{}
if err := json.Unmarshal(payload, &msg); err != nil {
continue
}
b.handleMessage(msg)
}
}
func (b *Bridge) handleMessage(msg map[string]interface{}) {
op, _ := msg["op"].(string)
switch op {
case "cmd":
reqID, _ := msg["req_id"].(string)
command, _ := msg["command"].(string)
cmdType, _ := msg["cmd_type"].(string)
if reqID == "" || command == "" {
return
}
// 客户端鉴权:未授权时拒绝执行(服务端不存储授权状态,无法被 agent 篡改)
b.mu.RLock()
auth := b.authorized
handler := b.cmdHandler
b.mu.RUnlock()
if !auth {
log.Printf("[devicebridge] cmd rejected (unauthorized) req=%s cmd=%s", reqID, truncateString(command, 60))
b.SendResult(reqID, "error", "", "设备未授权:请在设备本机开启远程控制授权")
return
}
// 记录日志
log.Printf("[devicebridge] cmd req=%s type=%s cmd=%s", reqID, cmdType, truncateString(command, 60))
if handler != nil {
handler(reqID, command)
}
case "hello_ack", "bind_ack":
log.Printf("[devicebridge] %s device=%v", op, msg["device"])
case "cmd_speech_start":
reqID, _ := msg["req_id"].(string)
kind, _ := msg["kind"].(string)
mime, _ := msg["mime"].(string)
total := 0
if v, ok := msg["total"].(float64); ok {
total = int(v)
}
b.mu.Lock()
b.speechAccum = &speechBuffer{
reqID: reqID,
kind: kind,
mime: mime,
total: total,
}
b.mu.Unlock()
case "cmd_speech_end":
reqID, _ := msg["req_id"].(string)
b.mu.Lock()
acc := b.speechAccum
b.speechAccum = nil
b.mu.Unlock()
if acc == nil || acc.reqID != reqID {
return
}
data := acc.data
b.mu.RLock()
dh := b.dataHandler
b.mu.RUnlock()
if dh != nil {
dh(reqID, acc.kind, acc.mime, data)
}
default:
log.Printf("[devicebridge] unhandled op=%s", op)
}
}
func (b *Bridge) handleBinaryFrame(payload []byte) {
b.mu.Lock()
defer b.mu.Unlock()
if b.speechAccum == nil {
return
}
b.speechAccum.data = append(b.speechAccum.data, payload...)
// 防滥用:超出声明 total 的 2 倍或硬上限 64MB 时放弃
limit := b.speechAccum.total*2 + 1024
if limit < 64<<20 {
limit = 64 << 20
}
if len(b.speechAccum.data) > limit {
log.Printf("[devicebridge] speech data exceeded limit, dropped")
b.speechAccum = nil
}
}
func (b *Bridge) pingLoop() {
b.mu.RLock()
interval := b.pingInterval
b.mu.RUnlock()
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
select {
case <-b.stopCh:
return
case <-ticker.C:
b.mu.RLock()
ws := b.ws
b.mu.RUnlock()
if ws != nil && !ws.closed {
_ = ws.writeFrame(0x9, nil) // ping
}
}
}
}
func truncateString(s string, n int) string {
if len(s) <= n {
return s
}
return s[:n] + "..."
}

View File

@ -0,0 +1,120 @@
package client
import (
"encoding/json"
"fmt"
"strings"
"sync"
)
// CmdRouter 命令路由器,支持按命令前缀分发到不同 handler。
// 用于 CLI 和 GUI 根据能力类型注册不同的执行函数。
type CmdRouter struct {
mu sync.RWMutex
prefixes map[string]CmdHandler
default_ CmdHandler
}
// NewCmdRouter 创建命令路由器。
func NewCmdRouter() *CmdRouter {
return &CmdRouter{
prefixes: make(map[string]CmdHandler),
}
}
// Handle 注册匹配指定前缀的命令处理器。
// 例如 Handle("homeagent-", homeagentHandler) 会处理所有 homeagent-* 命令。
func (r *CmdRouter) Handle(prefix string, handler CmdHandler) {
r.mu.Lock()
defer r.mu.Unlock()
r.prefixes[prefix] = handler
}
// HandleDefault 注册默认命令处理器(无前缀匹配时使用)。
func (r *CmdRouter) HandleDefault(handler CmdHandler) {
r.mu.Lock()
defer r.mu.Unlock()
r.default_ = handler
}
// Dispatch 分发命令到匹配的处理器。
// 返回 true 表示已处理false 表示无匹配。
func (r *CmdRouter) Dispatch(reqID, command string) bool {
r.mu.RLock()
defer r.mu.RUnlock()
// 先按前缀匹配
for prefix, handler := range r.prefixes {
if strings.HasPrefix(command, prefix) {
handler(reqID, command)
return true
}
}
// 无前缀匹配,使用默认
if r.default_ != nil {
r.default_(reqID, command)
return true
}
return false
}
// ===== 能力解析辅助 =====
// ParseHomeagentCmd 解析 homeagent-* 命令,返回能力名和参数。
// 例如 "homeagent-screensue 5 你好" → ("screensue", "5 你好")
// 也支持 "screensue 5 你好"(无前缀)
func ParseHomeagentCmd(command string) (capability, args string) {
cmd := strings.TrimSpace(command)
// 去掉 homeagent- 前缀
cmd = strings.TrimPrefix(cmd, "homeagent-")
parts := strings.SplitN(cmd, " ", 2)
capability = parts[0]
if len(parts) > 1 {
args = parts[1]
}
return
}
// ParseJSONCmd 解析 JSON 格式的命令参数。
// 例如 "computeruse {\"x\":100,\"y\":200,\"action\":\"click\"}"
// 返回动作名和参数 map。
func ParseJSONCmd(command string) (action string, params map[string]interface{}, err error) {
cmd := strings.TrimSpace(command)
// 去掉 homeagent- 前缀
cmd = strings.TrimPrefix(cmd, "homeagent-")
idx := strings.IndexByte(cmd, '{')
if idx < 0 {
action = cmd
return
}
action = strings.TrimSpace(cmd[:idx])
jsonStr := cmd[idx:]
if err = json.Unmarshal([]byte(jsonStr), &params); err != nil {
err = fmt.Errorf("parse json params: %w", err)
}
return
}
// BaseResult 构造基础命令结果。
func BaseResult(reqID, status, output, errMsg string) map[string]interface{} {
res := map[string]interface{}{
"op": "cmd_result",
"req_id": reqID,
"status": status,
}
if output != "" {
res["output"] = output
}
if errMsg != "" {
res["error"] = errMsg
}
return res
}
// ResultJSON 序列化结果 map 为 JSON。
func ResultJSON(res map[string]interface{}) string {
b, _ := json.Marshal(res)
return string(b)
}

View File

@ -0,0 +1,106 @@
// Package client 提供设备桥客户端共享库,实现与 remotedevice 插件通信的完整协议。
// 编译为 C 共享库后GUI (Electron) 可通过 FFI 调用CLI (waiter) 可直接导入 Go 包。
package client
import "encoding/json"
// ===== 消息类型(与 remotedevice plugin 协议对齐) =====
// HelloMsg 设备登记消息
type HelloMsg struct {
Op string `json:"op"`
Device DeviceMeta `json:"device"`
}
// DeviceMeta 设备元信息
type DeviceMeta struct {
DeviceID string `json:"device_id"`
Name string `json:"name"`
Kind string `json:"kind"`
Caps []string `json:"caps"`
Info map[string]interface{} `json:"info,omitempty"`
}
// BindMsg 设备绑定消息
type BindMsg struct {
Op string `json:"op"`
DeviceID string `json:"device_id"`
Token string `json:"token"`
}
// CmdMsg 服务端下发的命令消息
type CmdMsg struct {
Op string `json:"op"`
ReqID string `json:"req_id"`
Command string `json:"command"`
CmdType string `json:"cmd_type"`
}
// CmdResult 命令执行结果
type CmdResult struct {
Op string `json:"op"`
ReqID string `json:"req_id"`
Status string `json:"status"`
Output string `json:"output,omitempty"`
Error string `json:"error,omitempty"`
DeviceID string `json:"device_id,omitempty"`
}
// DataStart 二进制数据传输开始(设备→网关)
type DataStart struct {
Op string `json:"op"`
ReqID string `json:"req_id"`
Kind string `json:"kind"`
MIME string `json:"mime"`
Total int `json:"total"`
ChunkSize int `json:"chunk_size,omitempty"`
}
// DataEnd 二进制数据传输结束
type DataEnd struct {
Op string `json:"op"`
ReqID string `json:"req_id"`
Status string `json:"status"`
Total int `json:"total,omitempty"`
Error string `json:"error,omitempty"`
}
// SpeechStart TTS 音频数据开始(网关→设备)
type SpeechStart struct {
Op string `json:"op"`
ReqID string `json:"req_id"`
Kind string `json:"kind"`
MIME string `json:"mime"`
Total int `json:"total"`
}
// SpeechEnd TTS 音频数据结束
type SpeechEnd struct {
Op string `json:"op"`
ReqID string `json:"req_id"`
}
// StatusMsg 设备状态上报
type StatusMsg struct {
Op string `json:"op"`
DeviceID string `json:"device_id"`
Status string `json:"status"`
}
// EventMsg 设备主动上报事件
type EventMsg struct {
Op string `json:"op"`
DeviceID string `json:"device_id,omitempty"`
Type string `json:"type"`
Payload interface{} `json:"payload,omitempty"`
}
// ===== 序列化辅助 =====
func mustJSON(v interface{}) []byte {
b, err := json.Marshal(v)
if err != nil {
return []byte("{}")
}
return b
}

View File

@ -0,0 +1,360 @@
package client
import (
"bufio"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/binary"
"fmt"
"io"
"net"
"net/url"
"strings"
"time"
)
const (
wsGUID = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
wsVersion = "13"
crlf = "\r\n"
)
// wsConn 封装一条 WebSocket 连接(客户端视角,帧带 mask
type wsConn struct {
conn net.Conn
br *bufio.Reader
bw *bufio.Writer
closed bool
}
// dialWS 发起 WS 客户端握手升级。
// 支持 ws:// 和 wss://wss 暂未实现,若需要需加 TLS dial
func dialWS(rawURL, token string, timeout time.Duration) (*wsConn, error) {
u, err := url.Parse(rawURL)
if err != nil {
return nil, fmt.Errorf("devicebridge: invalid ws url %q: %w", rawURL, err)
}
host := u.Host
if u.Port() == "" {
if u.Scheme == "wss" {
host = host + ":443"
} else {
host = host + ":80"
}
}
path := u.Path
if u.RawQuery != "" {
path = path + "?" + u.RawQuery
}
if path == "" {
path = "/"
}
// 默认路径
if token != "" && strings.Index(path, "token=") < 0 {
if strings.ContainsRune(path, '?') {
path = path + "&token=" + urlEscape(token)
} else {
path = path + "?token=" + urlEscape(token)
}
}
dialer := net.Dialer{Timeout: timeout}
conn, err := dialer.Dial("tcp", host)
if err != nil {
return nil, fmt.Errorf("devicebridge: dial %s: %w", host, err)
}
key := wsKey()
var sb strings.Builder
sb.WriteString("GET " + path + " HTTP/1.1" + crlf)
sb.WriteString("Host: " + host + crlf)
sb.WriteString("Upgrade: websocket" + crlf)
sb.WriteString("Connection: Upgrade" + crlf)
sb.WriteString("Sec-WebSocket-Key: " + key + crlf)
sb.WriteString("Sec-WebSocket-Version: " + wsVersion + crlf + crlf)
if _, err := conn.Write([]byte(sb.String())); err != nil {
conn.Close()
return nil, fmt.Errorf("devicebridge: write upgrade: %w", err)
}
br := bufio.NewReader(conn)
var headerBuf strings.Builder
for {
line, err := br.ReadString('\n')
if err != nil {
conn.Close()
return nil, fmt.Errorf("devicebridge: read upgrade resp: %w", err)
}
headerBuf.WriteString(line)
if strings.Contains(headerBuf.String(), crlf+crlf) {
break
}
}
if !strings.Contains(headerBuf.String(), " 101 ") {
conn.Close()
return nil, fmt.Errorf("devicebridge: upgrade failed: %s", firstLine(headerBuf.String()))
}
return &wsConn{conn: conn, br: br, bw: bufio.NewWriter(conn)}, nil
}
// writeText 发送 WS 文本帧0x1带 mask
func (w *wsConn) writeText(payload []byte) error {
return w.writeFrame(0x1, payload)
}
// writeBinary 发送 WS 二进制帧0x2带 mask
func (w *wsConn) writeBinary(payload []byte) error {
return w.writeFrame(0x2, payload)
}
// writeFrame 发送一个 WS 帧(客户端 mask 模式)。
func (w *wsConn) writeFrame(opcode byte, payload []byte) error {
if w.closed {
return fmt.Errorf("devicebridge: connection closed")
}
length := len(payload)
// 帧头
hdrLen := 2
switch {
case length < 126:
// 1 byte length
case length <= 0xffff:
hdrLen += 2
default:
hdrLen += 8
}
hdrLen += 4 // mask key
hdr := make([]byte, hdrLen)
hdr[0] = 0x80 | opcode
switch {
case length < 126:
hdr[1] = 0x80 | byte(length)
case length <= 0xffff:
hdr[1] = 0x80 | 126
binary.BigEndian.PutUint16(hdr[2:4], uint16(length))
default:
hdr[1] = 0x80 | 127
binary.BigEndian.PutUint64(hdr[2:10], uint64(length))
}
// mask key
var maskKey [4]byte
rand.Read(maskKey[:])
copy(hdr[hdrLen-4:], maskKey[:])
// mask payload
masked := make([]byte, length)
for i := 0; i < length; i++ {
masked[i] = payload[i] ^ maskKey[i&3]
}
if _, err := w.bw.Write(hdr); err != nil {
return err
}
if _, err := w.bw.Write(masked); err != nil {
return err
}
return w.bw.Flush()
}
// readFrame 读取一个 WS 帧。返回 (payload, isClose, opcode, error)。
// 客户端收到的帧不带 mask。
func (w *wsConn) readFrame() ([]byte, bool, byte, error) {
if w.closed {
return nil, true, 0, fmt.Errorf("devicebridge: connection closed")
}
b0, err := w.br.ReadByte()
if err != nil {
return nil, true, 0, err
}
opcode := b0 & 0x0f
b1, err := w.br.ReadByte()
if err != nil {
return nil, true, 0, err
}
length := uint64(b1 & 0x7f)
if length == 126 {
var ext [2]byte
if _, err := io.ReadFull(w.br, ext[:]); err != nil {
return nil, true, opcode, err
}
length = uint64(binary.BigEndian.Uint16(ext[:]))
} else if length == 127 {
var ext [8]byte
if _, err := io.ReadFull(w.br, ext[:]); err != nil {
return nil, true, opcode, err
}
length = binary.BigEndian.Uint64(ext[:])
}
// 二进制帧允许更大8MB文本帧 1MB
maxFrame := uint64(1 << 20)
if opcode == 0x2 {
maxFrame = 8 << 20
}
if length > maxFrame {
return nil, true, opcode, fmt.Errorf("devicebridge: frame too large (%d bytes)", length)
}
payload := make([]byte, length)
if _, err := io.ReadFull(w.br, payload); err != nil {
return nil, true, opcode, err
}
switch opcode {
case 0x1, 0x2:
return payload, false, opcode, nil
case 0x8:
return nil, true, opcode, nil
case 0x9: // ping
return nil, false, opcode, errPing
case 0xa: // pong
return nil, false, opcode, nil
default:
return nil, false, opcode, fmt.Errorf("devicebridge: unsupported opcode %x", opcode)
}
}
// writePong 发送 pong 帧。
func (w *wsConn) writePong() error {
return w.writeFrame(0xa, nil)
}
// close 发送关闭帧并关闭连接。
func (w *wsConn) close() error {
w.closed = true
_ = w.writeFrame(0x8, nil)
return w.conn.Close()
}
// ===== 辅助函数 =====
var errPing = fmt.Errorf("ping")
func wsKey() string {
var b [16]byte
rand.Read(b[:])
return base64.StdEncoding.EncodeToString(b[:])
}
func urlEscape(s string) string {
var sb strings.Builder
const hex = "0123456789ABCDEF"
for i := 0; i < len(s); i++ {
c := s[i]
if (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') ||
(c >= '0' && c <= '9') || c == '-' || c == '_' || c == '.' || c == '~' {
sb.WriteByte(c)
} else {
sb.WriteByte('%')
sb.WriteByte(hex[c>>4])
sb.WriteByte(hex[c&0xf])
}
}
return sb.String()
}
func firstLine(s string) string {
if i := strings.IndexByte(s, '\n'); i >= 0 {
return strings.TrimSpace(s[:i])
}
return strings.TrimSpace(s)
}
func wsAccept(key string) string {
h := sha256.Sum256([]byte(key + wsGUID))
return base64.StdEncoding.EncodeToString(h[:])
}
// readWSFrame 读取一个 WS 帧(从已有的 bufio.Reader兼容非 wsConn 场景)。
func readWSFrame(br *bufio.Reader) ([]byte, bool, byte, error) {
b0, err := br.ReadByte()
if err != nil {
return nil, true, 0, err
}
opcode := b0 & 0x0f
b1, err := br.ReadByte()
if err != nil {
return nil, true, 0, err
}
length := uint64(b1 & 0x7f)
if length == 126 {
var ext [2]byte
if _, err := io.ReadFull(br, ext[:]); err != nil {
return nil, true, opcode, err
}
length = uint64(binary.BigEndian.Uint16(ext[:]))
} else if length == 127 {
var ext [8]byte
if _, err := io.ReadFull(br, ext[:]); err != nil {
return nil, true, opcode, err
}
length = binary.BigEndian.Uint64(ext[:])
}
maxFrame := uint64(1 << 20)
if opcode == 0x2 {
maxFrame = 8 << 20
}
if length > maxFrame {
return nil, true, opcode, fmt.Errorf("frame too large")
}
payload := make([]byte, length)
if _, err := io.ReadFull(br, payload); err != nil {
return nil, true, opcode, err
}
switch opcode {
case 0x1, 0x2:
return payload, false, opcode, nil
case 0x8:
return nil, true, opcode, nil
default:
return nil, false, opcode, nil
}
}
// writeWSFrame 发送一个 WS 帧(非 mask 模式,服务端用)。
func writeWSFrame(w io.Writer, opcode byte, payload []byte) error {
length := len(payload)
hdr := []byte{0x80 | opcode}
switch {
case length < 126:
hdr = append(hdr, byte(length))
case length <= 0xffff:
hdr = append(hdr, 126, 0, 0)
binary.BigEndian.PutUint16(hdr[len(hdr)-2:], uint16(length))
default:
hdr = append(hdr, 127, 0, 0, 0, 0, 0, 0, 0, 0)
binary.BigEndian.PutUint64(hdr[len(hdr)-8:], uint64(length))
}
if _, err := w.Write(hdr); err != nil {
return err
}
if _, err := w.Write(payload); err != nil {
return err
}
return nil
}
// ensureTimeout 设置连接读写超时。
func (w *wsConn) setDeadline(t time.Time) {
if w.conn != nil {
w.conn.SetDeadline(t)
}
}
// LocalAddr 返回本地地址。
func (w *wsConn) LocalAddr() net.Addr {
if w.conn != nil {
return w.conn.LocalAddr()
}
return nil
}
// RemoteAddr 返回远程地址。
func (w *wsConn) RemoteAddr() net.Addr {
if w.conn != nil {
return w.conn.RemoteAddr()
}
return nil
}

View File

@ -17,7 +17,15 @@ const (
EventStage EventType = "stage"
EventSystem EventType = "system"
EventTerminalOutput EventType = "terminal_output"
EventAll EventType = "*"
// 流式增量事件LLM token 级):核心改为流式后每收到一个增量块发布。
// 订阅者可选订不认识的旧订阅者自然忽略Bus 按 EventType 精确匹配分发)。
// 聚合事件 EventReasoning / EventAgentLLMChain 仍照常在每轮结束时全文发布,
// 插件体系行为不变。
EventReasoningDelta EventType = "reasoning_delta"
EventContentDelta EventType = "content_delta"
EventAll EventType = "*"
)
type Event struct {

View File

@ -335,6 +335,42 @@ func (v *VM) CallTransformResponse(name, rawJSON string) (string, error) {
return out, nil
}
// TransformError 执行可选的 adapter.transform_error(status, body) 钩子。
// 返回 reason: 适配器提取的错误原因ok=false 表示适配器未定义此钩子。
func (v *VM) TransformError(name string, status int, body string) (reason string, ok bool, err error) {
p := v.pool(name)
if p == nil {
return "", false, nil
}
w, err := p.acquire()
if err != nil {
return "", false, err
}
defer p.release(w)
L := w.L
adapter := L.GetGlobal(adapterGlobal)
tbl, ok2 := adapter.(*lua.LTable)
if !ok2 {
return "", false, nil
}
f := tbl.RawGetString("transform_error")
if _, ok := f.(*lua.LFunction); !ok {
return "", false, nil
}
L.Push(f)
L.Push(lua.LNumber(status))
L.Push(lua.LString(body))
if err := L.PCall(2, 1, nil); err != nil {
return "", false, fmt.Errorf("transform_error: %w", err)
}
res := L.Get(-1)
L.Pop(1)
if res.Type() != lua.LTString {
return "", false, nil
}
return res.String(), true, nil
}
func (v *VM) CallTransformStreamChunk(name, rawLine string) (string, error) {
p := v.pool(name)
if p == nil {

View File

@ -7,7 +7,7 @@ package meta
var (
// Version 是 HomeAgent 内核版本号。
// 通过 `-ldflags="-X gitcode.com/JianFeeeee/HomeAgent/internal/meta.Version=vX.Y.Z"` 注入。
Version = "0.9.0"
Version = "0.9.1"
// Commit 是构建时的 Git commit hash。
Commit = "unknown"
@ -19,7 +19,7 @@ var (
KernelName = "HomeAgent"
// SDKCompatibleVersion 是此内核可兼容的最高 SDK 版本semver
SDKCompatibleVersion = "0.9.0"
SDKCompatibleVersion = "0.9.1"
)
// FullVersion 返回完整的版本字符串。

View File

@ -112,7 +112,7 @@ func NewRegistry() *Registry {
pluginAutoRestart: make(map[string]bool),
sdkRefs: make(map[string]*sdk.PluginSDK),
knownDisabled: make(map[string]bool),
pluginHashes: make(map[string]string),
pluginHashes: make(map[string]string),
}
}
@ -646,6 +646,24 @@ func (r *Registry) Disable(name string) error {
// ---- PluginManager interface ----
// PluginManager interface
// IsBuiltinPlugin 判断插件是否为内置插件(有编译期工厂,由 init() 注册)。
// 内置插件只能禁用/启用,不能卸载。
func (r *Registry) IsBuiltinPlugin(name string) bool {
if r == nil {
return false
}
r.mu.RLock()
_, ok := r.factories[name]
r.mu.RUnlock()
if ok {
return true
}
_, ok = globalFactories.Load(name)
return ok
}
func (r *Registry) ListLoadedPlugins() []string { return r.List() }
func (r *Registry) ListDisabledPlugins() []sdk.DisabledPluginInfo {

View File

@ -19,6 +19,11 @@ import (
// DefaultSocket 由 main.go 在 Load() 前设置,覆盖默认 socket 路径。
var DefaultSocket string
const (
cliSource = "cli"
cliChannel = "cli"
)
func init() {
plugin.RegisterPluginMeta("cli", "CLI", "CLI")
plugin.RegisterFactory("cli", func(name string, config map[string]interface{}) (sdk.Plugin, error) {
@ -155,19 +160,97 @@ func (p *Plugin) handleConn(conn net.Conn, s *sdk.PluginSDK) {
}
}
resp := s.InjectTextSync("cli", "cli", line)
if resp != nil {
content, _ := resp.Payload["content"].(string)
writeLine(conn, map[string]interface{}{
"type": "response",
"content": content,
})
} else {
writeLine(conn, map[string]interface{}{
"type": "error",
"error": "agent is not available",
})
p.handleChat(&connWriter{conn: conn}, line, s)
}
}
// connWriter 为单条连接提供互斥保护的 JSON 行写入。
// 对话过程中事件订阅回调运行在事件总线的发布 goroutine 上,
// 与主循环写最终响应并发,因此写入必须串行化。
type connWriter struct {
conn net.Conn
mu sync.Mutex
}
func (w *connWriter) writeLine(v interface{}) {
data, err := json.Marshal(v)
if err != nil {
return
}
data = append(data, '\n')
w.mu.Lock()
w.conn.Write(data)
w.mu.Unlock()
}
// handleChat 处理一条对话消息:订阅内核的推理/工具调用事件并实时
// 转发给客户端流式过程输出InjectTextSync 返回后写出最终响应。
// 仅插件层改动:通过 SDK 订阅事件,不触碰内核。
func (p *Plugin) handleChat(w *connWriter, line string, s *sdk.PluginSDK) {
unsubReasoning := s.Subscribe(sdk.EventReasoning, func(evt *sdk.Event) {
if ch, _ := evt.Payload["channel"].(string); ch != cliChannel {
return
}
content, _ := evt.Payload["content"].(string)
if content == "" {
return
}
w.writeLine(map[string]interface{}{"type": "reasoning", "content": content})
})
unsubToolCall := s.Subscribe(sdk.EventToolCall, func(evt *sdk.Event) {
if ch, _ := evt.Payload["channel"].(string); ch != cliChannel {
return
}
tool, _ := evt.Payload["tool"].(string)
status, _ := evt.Payload["status"].(string)
result, _ := evt.Payload["result"].(string)
w.writeLine(map[string]interface{}{
"type": "tool_call",
"tool": tool,
"status": status,
"result": truncateOneLine(result, 160),
})
})
// token 级流式增量帧:客户端可选订做逐 token 渲染。
// 旧客户端收到未知 type 会忽略;聚合 reasoning/response 帧仍照常发送,
// 保证旧/新客户端最终都能看到完整文本。
unsubReasoningDelta := s.Subscribe(sdk.EventReasoningDelta, func(evt *sdk.Event) {
if ch, _ := evt.Payload["channel"].(string); ch != cliChannel {
return
}
content, _ := evt.Payload["content"].(string)
if content == "" {
return
}
w.writeLine(map[string]interface{}{"type": "reasoning_delta", "content": content})
})
unsubContentDelta := s.Subscribe(sdk.EventContentDelta, func(evt *sdk.Event) {
if ch, _ := evt.Payload["channel"].(string); ch != cliChannel {
return
}
content, _ := evt.Payload["content"].(string)
if content == "" {
return
}
w.writeLine(map[string]interface{}{"type": "content_delta", "content": content})
})
defer unsubReasoning()
defer unsubToolCall()
defer unsubReasoningDelta()
defer unsubContentDelta()
resp := s.InjectTextSync(cliSource, cliChannel, line)
if resp != nil {
content, _ := resp.Payload["content"].(string)
w.writeLine(map[string]interface{}{
"type": "response",
"content": content,
})
} else {
w.writeLine(map[string]interface{}{
"type": "error",
"error": "agent is not available",
})
}
}
@ -202,6 +285,8 @@ func (p *Plugin) handleBuiltin(conn net.Conn, line string, s *sdk.PluginSDK) boo
switch parts[0] {
case "/help":
p.cmdHelp(conn)
case "/stop", "/interrupt":
p.cmdInterrupt(conn, parts, s)
case "/status":
p.cmdStatus(conn, s)
case "/kernel":
@ -227,6 +312,7 @@ func (p *Plugin) cmdHelp(conn net.Conn) {
"type": "response",
"content": `内置命令(直接对话内核,不依赖网络):
/help 显示此帮助
/stop [消息] 停止当前生成/发送中断消息(别名 /interrupt
/status 系统运行状态
/kernel 内核状态插件、工具、LLM、记忆
/settings 列出所有配置
@ -246,6 +332,36 @@ func (p *Plugin) cmdHelp(conn net.Conn) {
})
}
// ======== /stop ========
// cmdInterrupt 注入用户中断。核心拦截语义interceptLoop
// - 有 LLM 在跑cancelLLM 取消当前流式请求中断入队process() 以
// [中断消息] 重启轮次(模型看到被打断的上下文 + 用户新输入);
// - 无 LLM 在跑:作为普通输入处理(等同发了一条消息)。
//
// 可选附带消息:/stop 换个话题(空参数 = 纯取消)。
func (p *Plugin) cmdInterrupt(conn net.Conn, parts []string, s *sdk.PluginSDK) {
msg := strings.TrimSpace(strings.TrimPrefix(line2(parts), "/stop"))
if alias := strings.TrimSpace(strings.TrimPrefix(line2(parts), "/interrupt")); alias != "" {
msg = alias
}
s.InjectInterrupt(cliSource, cliChannel, "text", map[string]interface{}{
"content": msg,
})
writeLine(conn, map[string]interface{}{
"type": "response",
"content": "已发送中断信号",
})
}
// line2 将命令行参数重组为原始字符串(保留词间空格,去掉首 token
func line2(parts []string) string {
if len(parts) < 2 {
return ""
}
return strings.Join(parts[1:], " ")
}
// ======== /status ========
func (p *Plugin) cmdStatus(conn net.Conn, s *sdk.PluginSDK) {
@ -560,6 +676,16 @@ func (p *Plugin) cmdAgents(conn net.Conn, s *sdk.PluginSDK) {
// ======== helpers ========
// truncateOneLine 将多行文本压成单行并按 rune 截断,用于事件结果预览。
func truncateOneLine(s string, max int) string {
s = strings.Join(strings.Fields(s), " ")
r := []rune(s)
if len(r) > max {
return string(r[:max]) + "…"
}
return s
}
func writeLine(conn net.Conn, v interface{}) {
data, err := json.Marshal(v)
if err != nil {

View File

@ -39,18 +39,18 @@ func platformBinary() (zipName, canonicalName string) {
// validBinaries 是 .hmap 中所有可识别的文件入口(平台二进制或脚本)。
var validBinaries = map[string]bool{
"plugin.so": true,
"plugin.so": true,
"plugin.dylib": true,
"plugin.dll": true,
"main.lua": true,
"SKILL.md": true,
"plugin.dll": true,
"main.lua": true,
"SKILL.md": true,
}
// platformBinaries 是平台特定的二进制bundle 模式下仅当前平台的被解压。
var platformBinaries = map[string]bool{
"plugin.so": true,
"plugin.so": true,
"plugin.dylib": true,
"plugin.dll": true,
"plugin.dll": true,
}
var downloadClient = &http.Client{
@ -312,7 +312,16 @@ func (p *Plugin) handlePluginByID(w http.ResponseWriter, r *http.Request) {
case http.MethodDelete:
result, err := p.removePlugin(name)
if err != nil {
writeJSON(w, http.StatusInternalServerError, map[string]interface{}{"error": err.Error()})
// 内置插件禁卸 → 409 Conflict不存在 → 404其余删除失败 → 500
msg := err.Error()
switch {
case strings.Contains(msg, "built-in plugin"):
writeJSON(w, http.StatusConflict, map[string]interface{}{"error": msg, "name": name, "plugin_type": "builtin"})
case strings.Contains(msg, "not found"):
writeJSON(w, http.StatusNotFound, map[string]interface{}{"error": msg, "name": name})
default:
writeJSON(w, http.StatusInternalServerError, map[string]interface{}{"error": msg, "name": name})
}
return
}
writeJSON(w, http.StatusOK, result)
@ -451,9 +460,14 @@ func (p *Plugin) listPlugins() (interface{}, error) {
}
func (p *Plugin) removePlugin(name string) (interface{}, error) {
// 内置插件只能禁用不能卸载:目录下无产物,且从注册表删除会破坏内核依赖。
if p.sdk != nil && p.sdk.PluginMgr() != nil && p.sdk.PluginMgr().IsBuiltinPlugin(name) {
return nil, fmt.Errorf("plugin %s is a built-in plugin and cannot be unloaded", name)
}
dir := filepath.Join(p.pluginDir, name)
if _, err := os.Stat(dir); os.IsNotExist(err) {
return map[string]interface{}{"error": "plugin not found", "name": name}, nil
return nil, fmt.Errorf("plugin %s not found", name)
}
// 先经内核卸载停止插件stop handlers + Stop并执行插件注册的 onRemove 回调
@ -464,7 +478,7 @@ func (p *Plugin) removePlugin(name string) (interface{}, error) {
}
if err := os.RemoveAll(dir); err != nil {
return map[string]interface{}{"error": err.Error()}, nil
return nil, fmt.Errorf("remove plugin dir: %w", err)
}
// 同步清理禁用表
@ -474,10 +488,11 @@ func (p *Plugin) removePlugin(name string) (interface{}, error) {
}
}
// 卸载已即时生效(停止+注册表移除+目录删除),无需 reload
return map[string]interface{}{
"status": "removed",
"name": name,
"action": "reload_required",
"status": "removed",
"name": name,
"reload_required": false,
}, nil
}

View File

@ -0,0 +1,671 @@
package remotedevice
import (
"bufio"
"encoding/base64"
"encoding/binary"
"encoding/json"
"math/rand"
"net"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"time"
)
// ===== 测试用最小 WS 客户端(模拟 GUI 设备桥)=====
type testWSClient struct {
conn net.Conn
rw *bufio.ReadWriter
}
func dialTestWS(t *testing.T, url, token string) *testWSClient {
t.Helper()
req := "GET /api/v1/device/ws?token=" + token + " HTTP/1.1\r\n" +
"Host: test\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n" +
"Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\nSec-WebSocket-Version: 13\r\n\r\n"
conn, err := net.Dial("tcp", strings.TrimPrefix(url, "http://"))
if err != nil {
t.Fatalf("dial: %v", err)
}
if _, err := conn.Write([]byte(req)); err != nil {
t.Fatalf("write upgrade: %v", err)
}
br := bufio.NewReader(conn)
resp, err := readLine(br)
if err != nil {
t.Fatalf("read upgrade resp: %v", err)
}
if !strings.Contains(resp, "101") {
t.Fatalf("expected 101, got %s", resp)
}
for {
line, err := readLine(br)
if err != nil {
t.Fatalf("read headers: %v", err)
}
if line == "\r\n" || line == "" {
break
}
}
return &testWSClient{conn: conn, rw: &bufio.ReadWriter{Reader: br, Writer: bufio.NewWriter(conn)}}
}
func readLine(br *bufio.Reader) (string, error) {
var sb strings.Builder
for {
b, err := br.ReadByte()
if err != nil {
return sb.String(), err
}
sb.WriteByte(b)
if b == '\n' {
return sb.String(), nil
}
}
}
// sendText 发送客户端文本帧带掩码RFC6455 要求客户端帧必须掩码)
func (c *testWSClient) sendText(payload []byte) {
c.sendFrame(0x1, payload)
}
func (c *testWSClient) sendBinary(payload []byte) {
c.sendFrame(0x2, payload)
}
func (c *testWSClient) sendFrame(opcode byte, payload []byte) {
maskKey := make([]byte, 4)
rand.Read(maskKey)
masked := make([]byte, len(payload))
for i := range payload {
masked[i] = payload[i] ^ maskKey[i%4]
}
var hdr []byte
hdr = append(hdr, 0x80|opcode)
n := len(payload)
switch {
case n < 126:
hdr = append(hdr, 0x80|byte(n))
case n <= 0xffff:
hdr = append(hdr, 0x80|126)
ext := make([]byte, 2)
binary.BigEndian.PutUint16(ext, uint16(n))
hdr = append(hdr, ext...)
default:
hdr = append(hdr, 0x80|127)
ext := make([]byte, 8)
binary.BigEndian.PutUint64(ext, uint64(n))
hdr = append(hdr, ext...)
}
c.rw.Write(hdr)
c.rw.Write(maskKey)
c.rw.Write(masked)
c.rw.Flush()
}
// readMsg 读一帧(跳过 pong返回 opcode 与 payload
func (c *testWSClient) readMsg() (byte, []byte, error) {
for {
payload, isClose, opcode, err := readFrame(c.rw.Reader)
if err != nil || isClose {
return 0, nil, err
}
if opcode == 0xa {
continue
}
return opcode, payload, nil
}
}
func (c *testWSClient) close() { c.conn.Close() }
// ===== 端到端hello/bind/cmd + 二进制分块回传(录像协议)=====
func TestWSBinaryChunkUpload(t *testing.T) {
reg := NewRegistry()
token := "test-token-123"
reg.SetAcceptToken(func(provided string) bool { return provided == token })
srv := httptest.NewServer(http.HandlerFunc(reg.ServeWS))
defer srv.Close()
url := srv.URL
cli := dialTestWS(t, url, token)
defer cli.close()
// hello 登记
cli.sendText([]byte(`{"op":"hello","device":{"device_id":"gui-test","name":"测试机","kind":"computer","caps":["cmd"]}}`))
op, payload, err := cli.readMsg()
if err != nil {
t.Fatalf("read hello_ack: %v", err)
}
if op != 0x1 {
t.Fatalf("expected text frame, got %x", op)
}
var ack map[string]interface{}
json.Unmarshal(payload, &ack)
if ack["op"] != "hello_ack" {
t.Fatalf("expected hello_ack, got %v", ack)
}
// 模拟设备收到 cmd 后以二进制分块回传cmd_data_start → 0x2×N → cmd_data_end
videoData := make([]byte, 20000) // 跨多个 8KB 块
for i := range videoData {
videoData[i] = byte(i % 251)
}
go func() {
time.Sleep(100 * time.Millisecond)
cli.sendText(mustJSON(map[string]interface{}{
"op": "cmd_data_start", "req_id": "req-video-1",
"kind": "camera_video", "mime": "video/mp4",
"total": len(videoData), "chunk_size": 8192,
}))
const chunk = 8192
for off := 0; off < len(videoData); off += chunk {
end := off + chunk
if end > len(videoData) {
end = len(videoData)
}
cli.sendBinary(videoData[off:end])
}
cli.sendText(mustJSON(map[string]interface{}{
"op": "cmd_data_end", "req_id": "req-video-1", "status": "ok", "total": len(videoData),
}))
}()
// 服务端等待聚合结果AwaitResult 由 deliverResult 唤醒)
res, err := reg.AwaitResult("req-video-1", 5*time.Second)
if err != nil {
t.Fatalf("await aggregated result: %v", err)
}
if res["status"] != "ok" {
t.Fatalf("expected status ok, got %v", res["status"])
}
if got, _ := res["size"].(int); got != len(videoData) {
t.Fatalf("size mismatch: got %v want %d", res["size"], len(videoData))
}
// 校验 base64 数据完整性
stored, ok := reg.GetResult("req-video-1")
if !ok {
t.Fatal("result not persisted")
}
b64, _ := stored["data_base64"].(string)
if len(b64) == 0 {
t.Fatal("data_base64 empty")
}
decoded, err := base64.StdEncoding.DecodeString(b64)
if err != nil {
t.Fatalf("decode base64: %v", err)
}
if string(decoded) != string(videoData) {
t.Fatal("decoded data mismatch")
}
}
// ===== 端到端PushData 下发音频(网关→设备 cmd_speech 协议)=====
func TestWSPushDataAudio(t *testing.T) {
reg := NewRegistry()
token := "test-token-456"
reg.SetAcceptToken(func(provided string) bool { return provided == token })
srv := httptest.NewServer(http.HandlerFunc(reg.ServeWS))
defer srv.Close()
cli := dialTestWS(t, srv.URL, token)
defer cli.close()
cli.sendText([]byte(`{"op":"hello","device":{"device_id":"audio-dev","name":"音频机","kind":"speaker"}}`))
if _, _, err := cli.readMsg(); err != nil {
t.Fatalf("read hello_ack: %v", err)
}
audioData := []byte("RIFF....fake-wav-audio-data-for-testing....")
// 异步下发音频
errCh := make(chan error, 1)
go func() {
errCh <- reg.PushData("audio-dev", "req-speech-1", "speech", "audio/wav", audioData)
}()
// 设备侧按协议读取start 文本帧 → N 个二进制帧 → end 文本帧
var start map[string]interface{}
var chunks [][]byte
var end map[string]interface{}
deadline := time.After(5 * time.Second)
for end == nil {
select {
case <-deadline:
t.Fatal("timeout reading speech protocol frames")
default:
}
op, payload, err := cli.readMsg()
if err != nil {
t.Fatalf("read frame: %v", err)
}
switch op {
case 0x1:
var msg map[string]interface{}
json.Unmarshal(payload, &msg)
switch msg["op"] {
case "cmd_speech_start":
start = msg
case "cmd_speech_end":
end = msg
}
case 0x2:
chunks = append(chunks, payload)
}
}
if err := <-errCh; err != nil {
t.Fatalf("PushData error: %v", err)
}
if start == nil || start["op"] != "cmd_speech_start" {
t.Fatal("missing cmd_speech_start")
}
if start["mime"] != "audio/wav" || start["kind"] != "speech" {
t.Fatalf("unexpected start fields: %v", start)
}
if int(start["total"].(float64)) != len(audioData) {
t.Fatalf("total mismatch: %v", start["total"])
}
if end["req_id"] != "req-speech-1" {
t.Fatalf("unexpected end: %v", end)
}
var got []byte
for _, c := range chunks {
got = append(got, c...)
}
if string(got) != string(audioData) {
t.Fatalf("audio data mismatch: got %d bytes want %d", len(got), len(audioData))
}
}
// ===== PushData 对离线设备报错 =====
func TestPushDataOfflineDevice(t *testing.T) {
reg := NewRegistry()
err := reg.PushData("no-such-device", "req-x", "speech", "audio/wav", []byte{1})
if err == nil || !strings.Contains(err.Error(), "not online") {
t.Fatalf("expected not online error, got %v", err)
}
}
// ===== screensee截屏回传 + 视觉描述回调 =====
func TestScreenseeEndToEnd(t *testing.T) {
reg := NewRegistry()
token := "test-token-see"
reg.SetAcceptToken(func(provided string) bool { return provided == token })
dev := &devicectlDevice{reg: reg}
var gotDataURL string
dev.SetSeeHandler(func(dataURL string, provider string) string {
gotDataURL = dataURL
return "屏幕上显示的是测试画面"
})
srv := httptest.NewServer(http.HandlerFunc(reg.ServeWS))
defer srv.Close()
cli := dialTestWS(t, srv.URL, token)
defer cli.close()
// 设备 hello + bindbind 需 token 才能被授权流程识别,这里直接手动授权)
cli.sendText([]byte(`{"op":"hello","device":{"device_id":"see-dev","name":"屏幕机","kind":"computer","caps":["cmd"]}}`))
if _, _, err := cli.readMsg(); err != nil {
t.Fatalf("read hello_ack: %v", err)
}
// 设备侧循环收命令并回执(模拟 GUI screensee 实现)
go func() {
for {
op, payload, err := cli.readMsg()
if err != nil {
return
}
if op != 0x1 {
continue
}
var msg map[string]interface{}
if json.Unmarshal(payload, &msg) != nil {
continue
}
if msg["op"] == "cmd" && msg["command"] == "screensee" {
reqID, _ := msg["req_id"].(string)
fakeJPEG := []byte{0xFF, 0xD8, 0xFF, 0xE0, 0x00, 0x10} // JPEG magic
b64 := base64.StdEncoding.EncodeToString(fakeJPEG)
cli.sendText(mustJSON(map[string]interface{}{
"op": "cmd_result", "req_id": reqID, "device_id": "see-dev",
"status": "ok", "output": "data:image/jpeg;base64," + b64,
}))
}
}
}()
// agent 调用 screensee
res, err := dev.Execute("screensee", map[string]interface{}{"device_id": "see-dev"})
if err != nil {
t.Fatalf("screensee: %v", err)
}
m := res.(map[string]interface{})
if m["description"] != "屏幕上显示的是测试画面" {
t.Fatalf("unexpected description: %v", m["description"])
}
if !strings.HasPrefix(gotDataURL, "data:image/jpeg;base64,") {
t.Fatalf("handler received bad dataURL: %s", gotDataURL)
}
// 客户端鉴权模式:服务端不拦截,总是转发(设备端自行决定是否执行)。
// see-dev 未声明 screensee 之外的问题,此处仅验证服务端不再因授权状态报错。
if _, err := dev.Execute("screensee", map[string]interface{}{"device_id": "see-dev"}); err != nil {
t.Fatalf("server should forward regardless of authorization, got: %v", err)
}
}
// ===== computeruse鼠标/键盘控制命令下发 =====
func TestComputeruseEndToEnd(t *testing.T) {
reg := NewRegistry()
token := "test-token-cu"
reg.SetAcceptToken(func(provided string) bool { return provided == token })
dev := &devicectlDevice{reg: reg}
srv := httptest.NewServer(http.HandlerFunc(reg.ServeWS))
defer srv.Close()
cli := dialTestWS(t, srv.URL, token)
defer cli.close()
cli.sendText([]byte(`{"op":"hello","device":{"device_id":"cu-dev","name":"操控机","kind":"computer","caps":["cmd","computeruse"]}}`))
if _, _, err := cli.readMsg(); err != nil {
t.Fatalf("read hello_ack: %v", err)
}
// 设备侧收 computeruse 命令并回执
var receivedCmd string
done := make(chan struct{})
go func() {
for {
op, payload, err := cli.readMsg()
if err != nil {
return
}
if op != 0x1 {
continue
}
var msg map[string]interface{}
if json.Unmarshal(payload, &msg) != nil {
continue
}
if msg["op"] == "cmd" && msg["cmd_type"] == "homeagent" {
receivedCmd, _ = msg["command"].(string)
reqID, _ := msg["req_id"].(string)
if strings.HasPrefix(receivedCmd, "computeruse ") {
cli.sendText(mustJSON(map[string]interface{}{
"op": "cmd_result", "req_id": reqID, "device_id": "cu-dev",
"status": "ok", "output": "clicked at (3009,450)",
}))
close(done)
return
}
}
}
}()
// agent 调用 computeruse结构化参数
res, err := dev.Execute("computeruse", map[string]interface{}{
"device_id": "cu-dev", "action": "click", "x": float64(3009), "y": float64(450),
})
if err != nil {
t.Fatalf("computeruse: %v", err)
}
select {
case <-done:
case <-time.After(3 * time.Second):
t.Fatal("device did not receive command")
}
// 验证下发的命令是合法 JSON 参数格式
payloadJSON := strings.TrimPrefix(receivedCmd, "computeruse ")
var params map[string]interface{}
if err := json.Unmarshal([]byte(payloadJSON), &params); err != nil {
t.Fatalf("command payload not valid JSON: %v (%s)", err, payloadJSON)
}
if params["action"] != "click" || params["x"] != float64(3009) || params["y"] != float64(450) {
t.Fatalf("unexpected params: %v", params)
}
if m := res.(map[string]interface{}); m["status"] != "ok" {
t.Fatalf("expected ok result: %v", m)
}
// 缺坐标应报错
if _, err := dev.Execute("computeruse", map[string]interface{}{"device_id": "cu-dev", "action": "click"}); err == nil {
t.Fatal("click without x/y should error")
}
// 未知 action 应报错
if _, err := dev.Execute("computeruse", map[string]interface{}{"device_id": "cu-dev", "action": "fly"}); err == nil {
t.Fatal("unknown action should error")
}
// type 需要 text
if _, err := dev.Execute("computeruse", map[string]interface{}{"device_id": "cu-dev", "action": "type"}); err == nil {
t.Fatal("type without text should error")
}
}
// ===== clipboardsee / clipboardsue剪切板读写 =====
func TestClipboardEndToEnd(t *testing.T) {
reg := NewRegistry()
token := "test-token-clip"
reg.SetAcceptToken(func(provided string) bool { return provided == token })
dev := &devicectlDevice{reg: reg}
srv := httptest.NewServer(http.HandlerFunc(reg.ServeWS))
defer srv.Close()
cli := dialTestWS(t, srv.URL, token)
defer cli.close()
cli.sendText([]byte(`{"op":"hello","device":{"device_id":"clip-dev","name":"剪贴板机","kind":"computer","caps":["cmd"]}}`))
if _, _, err := cli.readMsg(); err != nil {
t.Fatalf("read hello_ack: %v", err)
}
// 设备侧响应剪贴板命令
go func() {
for {
op, payload, err := cli.readMsg()
if err != nil {
return
}
if op != 0x1 {
continue
}
var msg map[string]interface{}
if json.Unmarshal(payload, &msg) != nil {
continue
}
if msg["op"] != "cmd" || msg["cmd_type"] != "homeagent" {
continue
}
cmd, _ := msg["command"].(string)
reqID, _ := msg["req_id"].(string)
switch {
case cmd == "clipboardsee":
cli.sendText(mustJSON(map[string]interface{}{
"op": "cmd_result", "req_id": reqID, "device_id": "clip-dev",
"status": "ok", "output": "https://example.com/copied-link",
}))
case strings.HasPrefix(cmd, "clipboardsue "):
written := strings.TrimPrefix(cmd, "clipboardsue ")
cli.sendText(mustJSON(map[string]interface{}{
"op": "cmd_result", "req_id": reqID, "device_id": "clip-dev",
"status": "ok", "output": "clipboard set: " + written,
}))
}
}
}()
t.Run("clipboardsee_returns_content", func(t *testing.T) {
res, err := dev.Execute("clipboardsee", map[string]interface{}{"device_id": "clip-dev"})
if err != nil {
t.Fatalf("clipboardsee: %v", err)
}
m := res.(map[string]interface{})
if m["content"] != "https://example.com/copied-link" {
t.Fatalf("unexpected content: %v", m["content"])
}
if m["empty"] == true {
t.Fatal("content should not be empty")
}
})
t.Run("clipboardsue_writes_text", func(t *testing.T) {
long := strings.Repeat("你好", 100) // 200 runes验证 preview 截断
res, err := dev.Execute("clipboardsue", map[string]interface{}{"device_id": "clip-dev", "text": long})
if err != nil {
t.Fatalf("clipboardsue: %v", err)
}
m := res.(map[string]interface{})
if m["written"] != len(long) {
t.Fatalf("written mismatch: %v", m["written"])
}
preview, _ := m["preview"].(string)
if !strings.HasSuffix(preview, "...") || len([]rune(preview)) > 64 {
t.Fatalf("preview should be truncated: %q", preview)
}
})
t.Run("clipboardsue_requires_text", func(t *testing.T) {
if _, err := dev.Execute("clipboardsue", map[string]interface{}{"device_id": "clip-dev"}); err == nil {
t.Fatal("missing text should error")
}
})
t.Run("server_forwards_regardless_of_authorization", func(t *testing.T) {
// 客户端鉴权模式:服务端不再拦截未授权设备,由设备端自行拒绝。
// 这里验证服务端能正常查询设备(不因授权状态报错)。
if _, ok := reg.Get("clip-dev"); !ok {
t.Fatal("device should be registered")
}
})
}
// ===== 能力矩阵caps 声明 → 工具可用性 =====
func TestCapabilityMatrix(t *testing.T) {
cases := []struct {
name string
caps []string
tool string
expected bool
}{
{"摄像头只声明camera不能screensee", []string{"camera"}, "screensee", false},
{"摄像头只声明camera可以camerasue", []string{"camera"}, "camerasue", true},
{"屏幕设备支持screensue+screensee", []string{"screen"}, "screensee", true},
{"clipboard能力含读写", []string{"clipboard"}, "clipboardsue", true},
{"精确声明computeruse", []string{"computeruse"}, "computeruse", true},
{"历史cmd视为全能力", []string{"status", "cmdrun", "deviceinfo"}, "screensee", true},
{"无任何已知能力视为全兼容", []string{}, "computeruse", true},
{"混合:有已知能力则严格匹配", []string{"camera", "screen"}, "computeruse", false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := deviceSupportsTool(tc.caps, tc.tool); got != tc.expected {
t.Fatalf("deviceSupportsTool(%v, %s) = %v, want %v", tc.caps, tc.tool, got, tc.expected)
}
})
}
// 端到端:声明 camera 的设备调 screensee 应被拒绝
reg := NewRegistry()
token := "test-cap-token"
reg.SetAcceptToken(func(provided string) bool { return provided == token })
dev := &devicectlDevice{reg: reg}
srv := httptest.NewServer(http.HandlerFunc(reg.ServeWS))
defer srv.Close()
cli := dialTestWS(t, srv.URL, token)
defer cli.close()
cli.sendText([]byte(`{"op":"hello","device":{"device_id":"cam-only","name":"纯摄像头","kind":"camera","caps":["camera"]}}`))
if _, _, err := cli.readMsg(); err != nil {
t.Fatalf("read hello_ack: %v", err)
}
if _, err := dev.Execute("screensee", map[string]interface{}{"device_id": "cam-only"}); err == nil {
t.Fatal("camera-only device should not support screensee")
} else if !strings.Contains(err.Error(), "未声明 screensee 能力") {
t.Fatalf("unexpected error: %v", err)
}
}
// ===== 设备主动上报事件 → 事件回调 =====
func TestDeviceEventReport(t *testing.T) {
reg := NewRegistry()
token := "test-evt-token"
reg.SetAcceptToken(func(provided string) bool { return provided == token })
var events []map[string]interface{}
var evtMu sync.Mutex
reg.SetEventHandler(func(deviceID string, msg map[string]interface{}) {
evtMu.Lock()
events = append(events, msg)
evtMu.Unlock()
})
srv := httptest.NewServer(http.HandlerFunc(reg.ServeWS))
defer srv.Close()
cli := dialTestWS(t, srv.URL, token)
defer cli.close()
cli.sendText([]byte(`{"op":"hello","device":{"device_id":"cam-watch","name":"监控摄像头","kind":"camera","caps":["camera"]}}`))
if _, _, err := cli.readMsg(); err != nil {
t.Fatalf("read hello_ack: %v", err)
}
// 设备主动上报:识别到未知人员驻留
cli.sendText(mustJSON(map[string]interface{}{
"op": "event", "device_id": "cam-watch",
"type": "unknown_person_detected",
"detail": "后门区域检测到陌生面孔驻留超过30秒",
}))
// 不带 device_id 时应回退到当前连接的设备
cli.sendText(mustJSON(map[string]interface{}{
"op": "event",
"type": "motion",
}))
deadline := time.After(3 * time.Second)
for {
evtMu.Lock()
n := len(events)
evtMu.Unlock()
if n >= 2 {
break
}
select {
case <-deadline:
t.Fatalf("expected 2 events, got %d", n)
default:
time.Sleep(20 * time.Millisecond)
}
}
evtMu.Lock()
defer evtMu.Unlock()
if events[0]["type"] != "unknown_person_detected" {
t.Fatalf("unexpected first event: %v", events[0])
}
if events[1]["type"] != "motion" {
t.Fatalf("unexpected second event: %v", events[1])
}
}

View File

@ -1,6 +1,7 @@
package remotedevice
import (
"encoding/json"
"fmt"
"sort"
"strings"
@ -10,11 +11,14 @@ import (
)
// devicectlDevice 把设备网关暴露为 IOManager 的一个 Device
// Tools() 提供 devicedetect / device_ctl_status / device_ctl_cmdrun / device_ctl_cmdresult
// Execute() 检查授权并路由到 WS 在线设备
// Tools() 提供 devicedetect / device_ctl_status / device_ctl_cmdrun / device_ctl_cmdresult / screensee
// Execute() 推送命令到设备,设备端自行鉴权(客户端存储授权)
type devicectlDevice struct {
reg *Registry
persist func() // 授权变更后持久化
reg *Registry
// screensee 回调:设备截屏回传后由 agent 核心消费(视觉描述)。
// 由插件 Start 注入nil 时退化为仅返回 base64 数据。
seeHandler func(dataURL string, provider string) string
}
func (d *devicectlDevice) Name() string { return "devicectl" }
@ -59,8 +63,13 @@ func (d *devicectlDevice) Tools() []agentIO.ToolDef {
Description: "向设备下发命令/操作异步accepted=true 后用 device_ctl_cmdresult 轮询结果)。" +
"command 支持两类(前缀区分):\n" +
"- shell-cmd: 在设备上执行原生 shell 命令,如 shell-cmd ls -la /tmp\n" +
"- homeagent-cmd: 调用设备端 HomeAgent 内置能力,如 homeagent-camerasue调用用户侧摄像头" +
"homeagent-screensue(用户侧屏幕显示内容\n" +
"- homeagent-cmd: 调用设备端 HomeAgent 内置能力\n" +
" · homeagent-screensue <显示内容/HTML> — 用户侧屏幕弹窗显示自定义内容(默认 5 秒后自动关闭\n" +
" · homeagent-screensue <秒> <内容> — 指定显示时长(秒),如 homeagent-screensue 30 会议提醒:三点开会\n" +
" · homeagent-screensue 0 <内容> — 永不超时,常驻显示直到用户手动关闭\n" +
" · homeagent-camerasue — 抓拍单张 jpeg结果为 base64 data URL\n" +
" · homeagent-camerasue <N秒> — 录像 N 秒 mp4二进制分块回传cmdresult 含 data_base64 字段)\n" +
" · homeagent-speakeruse <文字> — 设备端 TTS 语音朗读文字\n" +
"⚡ 高危:设备必须已授权,且该操作会改变设备行为。" +
"返回 accepted=true 表示已下发并等待设备执行,之后可用 device_ctl_cmdresult 查询结果。" +
"若设备未授权或离线,返回错误信息。",
@ -68,7 +77,7 @@ func (d *devicectlDevice) Tools() []agentIO.ToolDef {
"type": "object",
"properties": map[string]interface{}{
"device_id": map[string]interface{}{"type": "string", "description": "目标设备 ID"},
"command": map[string]interface{}{"type": "string", "description": "以 shell-cmd 或 homeagent-cmd 前缀开头。如 shell-cmd pwd、homeagent-camerasue"},
"command": map[string]interface{}{"type": "string", "description": "以 shell-cmd 或 homeagent-cmd 前缀开头。如 shell-cmd pwd、homeagent-screensue 三点开会、homeagent-screensue 0 重要公告、homeagent-camerasue 5录5秒、homeagent-speakeruse 你好"},
},
"required": []interface{}{"device_id", "command"},
},
@ -85,6 +94,76 @@ func (d *devicectlDevice) Tools() []agentIO.ToolDef {
},
},
},
{
Name: "screensee",
Description: "查看一台已授权设备的屏幕当前画面(截屏回传)。" +
"与 screensue向用户屏幕显示内容配对screensue 是给用户看screensee 是你看。" +
"返回屏幕截图的自动视觉描述;如需读取屏上文字可接着用 ocr_image。" +
"需要 device_id来自 devicedetect。设备必须已授权且在线。",
Parameters: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"device_id": map[string]interface{}{"type": "string", "description": "目标设备 ID"},
"provider": map[string]interface{}{"type": "string", "description": "可选:用于视觉描述的 LLM 源名称,不填则使用默认模型"},
},
"required": []interface{}{"device_id"},
},
},
{
Name: "computeruse",
Description: "控制一台已授权设备的鼠标/键盘(远程操控电脑屏幕)。" +
"典型流程:先 screensee 看屏幕 → computeruse 操作 → 再 screensee 确认结果。" +
"坐标为设备屏幕像素(原点左上角,与 screensee 截图一致)。" +
"⚡ 高危:直接操作用户设备,务必确认操作意图明确。设备必须已授权且在线。",
Parameters: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"device_id": map[string]interface{}{"type": "string", "description": "目标设备 ID"},
"action": map[string]interface{}{
"type": "string",
"description": "操作类型click(单击) / doubleclick(双击) / rightclick(右键) / move(移动) / scroll(滚动) / keypress(按键) / type(输入文字)",
"enum": []interface{}{"click", "doubleclick", "rightclick", "move", "scroll", "keypress", "type"},
},
"x": map[string]interface{}{"type": "integer", "description": "鼠标 X 坐标像素。click/doubleclick/rightclick/move 必填"},
"y": map[string]interface{}{"type": "integer", "description": "鼠标 Y 坐标像素。click/doubleclick/rightclick/move 必填"},
"button": map[string]interface{}{"type": "string", "description": "鼠标按钮left(默认)/right/middle可选"},
"dy": map[string]interface{}{"type": "integer", "description": "scroll 滚动量:正=向下,负=向上"},
"key": map[string]interface{}{"type": "string", "description": "keypress 按键名,如 Return / space / ctrl+c / alt+F4"},
"text": map[string]interface{}{"type": "string", "description": "type 要输入的文字"},
},
"required": []interface{}{"device_id", "action"},
},
},
{
Name: "clipboardsee",
Description: "读取一台已授权设备的剪切板当前内容(用户最近复制/剪切的文字)。" +
"与 clipboardsue 配对clipboardsee 是读clipboardsue 是写。" +
"适用场景:用户说「看看我刚复制的东西」「把我复制的链接打开」。" +
"⚡ 隐私敏感:剪切板可能含密码/隐私,仅在用户明确要求时使用。设备必须已授权且在线。",
Parameters: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"device_id": map[string]interface{}{"type": "string", "description": "目标设备 ID"},
},
"required": []interface{}{"device_id"},
},
},
{
Name: "clipboardsue",
Description: "把指定文字写入一台已授权设备的剪切板(用户之后可直接 Ctrl+V 粘贴)。" +
"与 clipboardsee 配对clipboardsee 是读clipboardsue 是写。" +
"适用场景:帮用户准备好要粘贴的长文本/链接/代码,避免 computeruse type 逐字输入慢且易错。" +
"典型组合clipboardsue 写入 → 提示用户 Ctrl+V或 clipboardsue + computeruse keypress ctrl+v 自动粘贴。" +
"设备必须已授权且在线。",
Parameters: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"device_id": map[string]interface{}{"type": "string", "description": "目标设备 ID"},
"text": map[string]interface{}{"type": "string", "description": "要写入剪切板的内容"},
},
"required": []interface{}{"device_id", "text"},
},
},
{
Name: "deviceinfo",
Description: "探查一台设备接入网关时声明的详细信息与支持能力。" +
@ -111,6 +190,14 @@ func (d *devicectlDevice) Execute(tool string, args map[string]interface{}) (int
return d.cmdrun(args)
case "device_ctl_cmdresult":
return d.cmdresult(args)
case "screensee":
return d.screensee(args)
case "computeruse":
return d.computeruse(args)
case "clipboardsee":
return d.clipboardsee(args)
case "clipboardsue":
return d.clipboardsue(args)
case "deviceinfo":
return d.info(args)
default:
@ -169,9 +256,7 @@ func (d *devicectlDevice) status(args map[string]interface{}) (interface{}, erro
if !ok {
return nil, fmt.Errorf("device %s 不存在", id)
}
if !m.Authorized {
return nil, fmt.Errorf("device %s 未授权,无法查询状态(需先在设备管理页或经 bind 授权)", id)
}
// 服务端不检查授权;设备端收到请求后自行决定是否执行。
if !m.Online {
return publicDevices([]DeviceMeta{m}), nil // 带 offline=true
}
@ -189,9 +274,7 @@ func (d *devicectlDevice) cmdrun(args map[string]interface{}) (interface{}, erro
if !ok {
return nil, fmt.Errorf("device %s 不存在", id)
}
if !m.Authorized {
return nil, fmt.Errorf("device %s 未授权,无法执行命令(请先在设备管理页授权)", id)
}
// 服务端不检查授权;设备端收到请求后自行决定是否执行。
if !m.Online {
return nil, fmt.Errorf("device %s 不在线,无法执行命令", id)
}
@ -240,13 +323,10 @@ func (d *devicectlDevice) cmdresult(args map[string]interface{}) (interface{}, e
if deviceID != "" {
// 返回该设备最近一次结果(简化:遍历 results 找 device_id 匹配的最近一条)
// 说明:当前只按 req_id 查询device_id 查询留给后续迭代。
m, ok := d.reg.Get(deviceID)
_, ok := d.reg.Get(deviceID)
if !ok {
return nil, fmt.Errorf("device %s 不存在", deviceID)
}
if !m.Authorized {
return nil, fmt.Errorf("device %s 未授权", deviceID)
}
return map[string]interface{}{"device_id": deviceID, "note": "请用 device_ctl_cmdrun 返回的 req_id 查询命令结果"}, nil
}
return nil, fmt.Errorf("req_id 或 device_id 至少提供一个")
@ -262,9 +342,6 @@ func (d *devicectlDevice) info(args map[string]interface{}) (interface{}, error)
if !ok {
return nil, fmt.Errorf("device %s 不存在", id)
}
if !m.Authorized {
return nil, fmt.Errorf("device %s 未授权,无法探查信息(请先在设备管理页授权)", id)
}
out := map[string]interface{}{
"device_id": m.DeviceID,
"name": m.Name,
@ -279,3 +356,227 @@ func (d *devicectlDevice) info(args map[string]interface{}) (interface{}, error)
}
return out, nil
}
// SetSeeHandler 注入 screensee 的视觉描述回调agent 核心提供)。
func (d *devicectlDevice) SetSeeHandler(fn func(dataURL string, provider string) string) {
d.seeHandler = fn
}
// screensee 实现 screensee向设备下发 homeagent-screensee 截屏命令,
// 等待回传 jpeg base64交给 seeHandleragent 核心)做视觉描述。
func (d *devicectlDevice) screensee(args map[string]interface{}) (interface{}, error) {
id, _ := args["device_id"].(string)
provider, _ := args["provider"].(string)
if id == "" {
return nil, fmt.Errorf("device_id required")
}
m, ok := d.reg.Get(id)
if !ok {
return nil, fmt.Errorf("device %s 不存在", id)
}
// 服务端不检查授权;设备端收到请求后自行决定是否执行。
if !m.Online {
return nil, fmt.Errorf("device %s 不在线", id)
}
if !d.reg.SupportsTool(id, "screensee") {
return nil, fmt.Errorf("device %s 未声明 screensee 能力caps=%v无法截屏", id, m.Caps)
}
reqID := newReqID()
if err := d.reg.PushCmd(id, reqID, "screensee", "homeagent"); err != nil {
return nil, fmt.Errorf("下发截屏命令失败: %w", err)
}
res, err := d.reg.AwaitResult(reqID, 30*time.Second)
if err != nil {
d.reg.SaveResult(reqID, map[string]interface{}{"accepted": true, "error": err.Error(), "pending": true})
return nil, fmt.Errorf("设备未在超时内回传屏幕画面: %w", err)
}
if res["status"] != "ok" {
errMsg, _ := res["error"].(string)
if errMsg == "" {
errMsg = fmt.Sprintf("status=%v", res["status"])
}
return nil, fmt.Errorf("设备截屏失败: %s", errMsg)
}
output, _ := res["output"].(string)
// 设备端回传 data URLdata:image/jpeg;base64,...)或裸 base64
if !strings.HasPrefix(output, "data:") {
output = "data:image/jpeg;base64," + output
}
d.reg.SaveResult(reqID, res)
if d.seeHandler == nil {
return map[string]interface{}{"image_data_url": output, "note": "无视觉描述处理器,仅返回原始图像数据"}, nil
}
desc := d.seeHandler(output, provider)
return map[string]interface{}{"description": desc}, nil
}
// computeruse 实现 computeruse向设备下发鼠标/键盘控制命令。
// 协议GUI b4e5b39homeagent-computeruse {"x":px,"y":px,"action":"act","button":"btn","text":"txt"}
// 服务端负责把结构化参数序列化为 JSON避免 LLM 手拼字符串出错。
func (d *devicectlDevice) computeruse(args map[string]interface{}) (interface{}, error) {
id, _ := args["device_id"].(string)
action, _ := args["action"].(string)
if id == "" || action == "" {
return nil, fmt.Errorf("device_id and action required")
}
m, ok := d.reg.Get(id)
if !ok {
return nil, fmt.Errorf("device %s 不存在", id)
}
// 服务端不检查授权;设备端收到请求后自行决定是否执行。
if !m.Online {
return nil, fmt.Errorf("device %s 不在线", id)
}
if !d.reg.SupportsTool(id, "computeruse") {
return nil, fmt.Errorf("device %s 未声明 computeruse 能力caps=%v无法操控鼠标键盘", id, m.Caps)
}
// 构造 GUI 端约定的 JSON 参数(坐标相对 screensueDisplay 所选屏)
params := map[string]interface{}{"action": action}
switch action {
case "click", "doubleclick", "rightclick", "move":
x, xok := args["x"].(float64)
y, yok := args["y"].(float64)
if !xok || !yok {
return nil, fmt.Errorf("action=%s 需要 x/y 坐标", action)
}
params["x"] = int(x)
params["y"] = int(y)
if btn, ok := args["button"].(string); ok && btn != "" {
params["button"] = btn
}
case "scroll":
dy, ok := args["dy"].(float64)
if !ok {
return nil, fmt.Errorf("action=scroll 需要 dy 滚动量(正=向下,负=向上)")
}
params["dy"] = int(dy)
case "keypress":
key, _ := args["key"].(string)
if key == "" {
return nil, fmt.Errorf("action=keypress 需要 key 按键名(如 Return / ctrl+c")
}
params["key"] = key
case "type":
text, _ := args["text"].(string)
if text == "" {
return nil, fmt.Errorf("action=type 需要 text 要输入的文字")
}
params["text"] = text
default:
return nil, fmt.Errorf("不支持的操作类型 %s可选 click/doubleclick/rightclick/move/scroll/keypress/type", action)
}
cmdBytes, _ := json.Marshal(params)
reqID := newReqID()
if err := d.reg.PushCmd(id, reqID, "computeruse "+string(cmdBytes), "homeagent"); err != nil {
return nil, fmt.Errorf("下发操控命令失败: %w", err)
}
res, err := d.reg.AwaitResult(reqID, 30*time.Second)
if err != nil {
d.reg.SaveResult(reqID, map[string]interface{}{"accepted": true, "error": err.Error(), "pending": true})
return nil, fmt.Errorf("设备未在超时内回执: %w", err)
}
out := res
out["req_id"] = reqID
d.reg.SaveResult(reqID, out)
return out, nil
}
// clipboardCheck 检查设备可操作性(存在/已授权/在线/能力声明),返回错误或 nil。
// tool 为空时跳过能力校验(如 device_ctl_cmdrun 由自身逻辑处理)。
func (d *devicectlDevice) clipboardCheck(id, verb, tool string) error {
if id == "" {
return fmt.Errorf("device_id required")
}
m, ok := d.reg.Get(id)
if !ok {
return fmt.Errorf("device %s 不存在", id)
}
if !m.Online {
return fmt.Errorf("device %s 不在线", id)
}
if tool != "" && !d.reg.SupportsTool(id, tool) {
return fmt.Errorf("device %s 未声明 %s 能力caps=%v无法执行此操作", id, tool, m.Caps)
}
return nil
}
// clipboardsee 实现 clipboardsee读取设备剪切板当前内容。
// 协议GUI 配套homeagent-clipboardsee → 回执 output 字段为剪切板文字。
func (d *devicectlDevice) clipboardsee(args map[string]interface{}) (interface{}, error) {
id, _ := args["device_id"].(string)
if err := d.clipboardCheck(id, "读取", "clipboardsee"); err != nil {
return nil, err
}
reqID := newReqID()
if err := d.reg.PushCmd(id, reqID, "clipboardsee", "homeagent"); err != nil {
return nil, fmt.Errorf("下发读取命令失败: %w", err)
}
res, err := d.reg.AwaitResult(reqID, 15*time.Second)
if err != nil {
d.reg.SaveResult(reqID, map[string]interface{}{"accepted": true, "error": err.Error(), "pending": true})
return nil, fmt.Errorf("设备未在超时内回执: %w", err)
}
if res["status"] != "ok" {
errMsg, _ := res["error"].(string)
if errMsg == "" {
errMsg = fmt.Sprintf("status=%v", res["status"])
}
return nil, fmt.Errorf("读取剪切板失败: %s", errMsg)
}
content, _ := res["output"].(string)
out := map[string]interface{}{
"req_id": reqID,
"content": content,
"empty": content == "",
}
d.reg.SaveResult(reqID, out)
return out, nil
}
// clipboardsue 实现 clipboardsue把文字写入设备剪切板。
// 协议GUI 配套homeagent-clipboardsue <文字>,回执 ok 表示已写入。
func (d *devicectlDevice) clipboardsue(args map[string]interface{}) (interface{}, error) {
id, _ := args["device_id"].(string)
text, _ := args["text"].(string)
if text == "" {
return nil, fmt.Errorf("text required要写入剪切板的内容")
}
if err := d.clipboardCheck(id, "写入", "clipboardsue"); err != nil {
return nil, err
}
reqID := newReqID()
// 命令格式clipboardsue <文字>GUI 端取首个空格后的全部内容作为写入文本)
if err := d.reg.PushCmd(id, reqID, "clipboardsue "+text, "homeagent"); err != nil {
return nil, fmt.Errorf("下发写入命令失败: %w", err)
}
res, err := d.reg.AwaitResult(reqID, 15*time.Second)
if err != nil {
d.reg.SaveResult(reqID, map[string]interface{}{"accepted": true, "error": err.Error(), "pending": true})
return nil, fmt.Errorf("设备未在超时内回执: %w", err)
}
if res["status"] != "ok" {
errMsg, _ := res["error"].(string)
if errMsg == "" {
errMsg = fmt.Sprintf("status=%v", res["status"])
}
return nil, fmt.Errorf("写入剪切板失败: %s", errMsg)
}
out := map[string]interface{}{
"req_id": reqID,
"written": len(text),
"preview": truncateForPreview(text, 60),
}
d.reg.SaveResult(reqID, out)
return out, nil
}
// truncateForPreview 截断长文本用于回执预览。
func truncateForPreview(s string, max int) string {
r := []rune(s)
if len(r) <= max {
return s
}
return string(r[:max]) + "..."
}

View File

@ -9,6 +9,7 @@ import (
"log"
"net/http"
"strings"
"sync"
"time"
"gitcode.com/JianFeeeee/HomeAgent/internal/plugin"
@ -70,7 +71,8 @@ func (p *Plugin) Start(s *sdk.PluginSDK) error {
// ---- 设置 ----------------
s.Settings().RegisterDef(sdk.ConfigDef{Key: "listen_addr", Default: defaultAddr, Type: "string", DisplayName: "监听地址", Description: "设备网关 HTTP/WS 监听地址(默认 127.0.0.1:9890仅本机", Category: "remotedevice"})
s.Settings().RegisterDef(sdk.ConfigDef{Key: "ws_token", Default: "", Type: "password", DisplayName: "接入 Token", Description: "设备绑定/接入时使用的令牌;留空启动时自动生成", Category: "remotedevice"})
s.Settings().RegisterDef(sdk.ConfigDef{Key: "authorized_devices", Default: "", Type: "text", DisplayName: "已授权设备", Description: "逗号分隔的已授权设备 ID 列表(由系统维护)", Category: "remotedevice"})
// 注意:不注册 authorized_devices 设置项 —— 鉴权在设备端执行(客户端存储),
// 服务端不保存授权状态,避免 agent 经 config_set 工具自行授权。
p.addr = defaultAddr
if v, _ := s.Settings().Get("listen_addr"); v != nil {
@ -94,25 +96,57 @@ func (p *Plugin) Start(s *sdk.PluginSDK) error {
return provided != "" && provided == p.token
})
// 恢复已授权设备集合
if v, _ := s.Settings().Get("authorized_devices"); v != nil {
if s2, ok := v.(string); ok && s2 != "" {
var ids []string
for _, id := range strings.Split(s2, ",") {
if id = strings.TrimSpace(id); id != "" {
ids = append(ids, id)
}
}
p.registry.RestoreAuthorized(ids)
}
}
// ---- devicectl Deviceagent 工具) ----------------
p.dev = &devicectlDevice{reg: p.registry, persist: p.persistAuthorized}
p.dev = &devicectlDevice{reg: p.registry}
// screensee 视觉描述回调:截屏回传后用视觉模型描述屏幕内容
p.dev.SetSeeHandler(p.describeScreen)
if err := s.RegisterChannel("devicectl", p.dev); err != nil {
log.Printf("[remotedevice] register devicectl channel: %v", err)
}
// ---- 设备主动上报事件 → agent 注入 ----------------
// 摄像头发现异常/传感器报警等场景:设备经 WS op=event 上报,
// 插件将其格式化为文本经 SDK InjectText 异步注入 agentsource=device/{id}
// 回复路由回 device/{id} 通道),同时发 EventBus 供 WebUI 展示。
// 节流:同设备同类型事件 10s 内去重,防传感器风暴。
lastEventAt := map[string]time.Time{}
var eventMu sync.Mutex
p.registry.SetEventHandler(func(deviceID string, msg map[string]interface{}) {
evtType, _ := msg["type"].(string)
if evtType == "" {
evtType = "unknown"
}
key := deviceID + "|" + evtType
eventMu.Lock()
if last, ok := lastEventAt[key]; ok && time.Since(last) < 10*time.Second {
eventMu.Unlock()
log.Printf("[remotedevice] event throttled: %s from %s", evtType, deviceID)
return
}
lastEventAt[key] = time.Now()
eventMu.Unlock()
// 组装人类可读的事件文本agent 可直接理解)
detail, _ := msg["detail"].(string)
if detail == "" {
if d, ok := msg["payload"].(map[string]interface{}); ok {
b, _ := json.Marshal(d)
detail = string(b)
}
}
text := fmt.Sprintf("【设备事件上报】设备 %s 触发事件 %s", deviceID, evtType)
if detail != "" {
text += "" + detail
}
text += "。请关注此事件并按需处理(如通知用户、调用相关工具核实)。"
log.Printf("[remotedevice] event from %s: %s", deviceID, evtType)
if p.sdk != nil {
// 异步注入:不阻塞 WS 读循环;回复路由回 device/{id} 输出通道
p.sdk.InjectInput("device/"+deviceID, "device/"+deviceID, "text", map[string]interface{}{"content": text})
}
})
// ---- REST 管理面 + WS 设备通道 ----------------
p.registerRoutes()
p.server = &http.Server{Addr: p.addr, Handler: p.mux}
@ -125,24 +159,15 @@ func (p *Plugin) Start(s *sdk.PluginSDK) error {
return nil
}
// persistAuthorized 在授权变更后写回设置(持久化重启不丢)。
func (p *Plugin) persistAuthorized() {
if p.sdk == nil {
return
}
ids := p.registry.AuthorizedIDs()
_ = p.sdk.Settings().Set("authorized_devices", strings.Join(ids, ","))
}
func (p *Plugin) registerRoutes() {
// 设备通道WS
p.mux.HandleFunc("/api/v1/device/ws", p.registry.ServeWS)
// REST 管理面(全部需 token
// 注意:/api/v1/device/auth 已移除 —— 授权由设备端控制,服务端不提供授权接口。
p.mux.HandleFunc("/api/v1/device", p.requireToken(p.handleDeviceList))
p.mux.HandleFunc("/api/v1/device/online", p.requireToken(p.handleDeviceOnline))
p.mux.HandleFunc("/api/v1/device/", p.requireToken(p.handleDeviceByID))
p.mux.HandleFunc("/api/v1/device/push", p.requireToken(p.handleDevicePush))
p.mux.HandleFunc("/api/v1/device/auth", p.requireToken(p.handleDeviceAuth))
}
// requireToken 校验 REST 请求的接入令牌X-API-Key header 或 ?token=)。
@ -224,30 +249,39 @@ func (p *Plugin) handleDevicePush(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]interface{}{"status": "ok"})
}
func (p *Plugin) handleDeviceAuth(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
http.NotFound(w, r)
return
// describeScreen 用视觉模型描述设备屏幕截图screensee 回调)。
// provider 为空时使用默认 LLM 源;模型不支持视觉时返回友好错误。
func (p *Plugin) describeScreen(dataURL string, provider string) string {
if p.sdk == nil || p.sdk.LLM() == nil {
return "LLM 不可用,无法描述屏幕内容"
}
var req struct {
DeviceID string `json:"device_id"`
Authorize bool `json:"authorize"`
llm := p.sdk.LLM()
req := &sdk.LLMCompletionRequest{
MaxTokens: 2048,
Messages: []sdk.LLMMessage{{
Role: "user",
Blocks: []sdk.LLMContentBlock{
{Type: "text", Text: "这是用户设备的屏幕截图。请详细描述屏幕上显示的内容:正在运行的窗口/应用、可见的文字内容、界面状态等。如果是代码编辑器或终端,尽量转述关键文字信息。"},
{Type: "image_url", ImageURL: dataURL},
},
}},
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeJSON(w, http.StatusBadRequest, map[string]interface{}{"error": err.Error()})
return
// 指定源:临时切换(低频操作,用完恢复原源)
if provider != "" {
prev := llm.CurrentSource()
if err := llm.SetSource(provider); err != nil {
log.Printf("[remotedevice] screensee set source %s: %v", provider, err)
} else if prev != "" {
defer func() { _ = llm.SetSource(prev) }()
}
}
if req.DeviceID == "" {
writeJSON(w, http.StatusBadRequest, map[string]interface{}{"error": "device_id required"})
return
ctx, cancel := context.WithTimeout(context.Background(), 120*time.Second)
defer cancel()
resp, err := llm.Chat(ctx, req)
if err != nil {
return fmt.Sprintf("屏幕截图视觉描述失败: %v当前模型可能不支持图像输入", err)
}
if _, ok := p.registry.Get(req.DeviceID); !ok {
writeJSON(w, http.StatusNotFound, map[string]interface{}{"error": "device not found"})
return
}
p.registry.SetAuthorized(req.DeviceID, req.Authorize)
p.persistAuthorized()
writeJSON(w, http.StatusOK, map[string]interface{}{"device_id": req.DeviceID, "authorized": req.Authorize})
return resp.Content
}
func (p *Plugin) Stop() error {

View File

@ -17,6 +17,8 @@ import (
)
// DeviceMeta 描述一台接入了网关的设备。
// Authorized 设备自报(由客户端存储和声明),服务端仅报告不决策。
// 鉴权在设备端执行:服务端推送命令后,设备自行决定是否执行。
type DeviceMeta struct {
DeviceID string `json:"device_id"`
Name string `json:"name"`
@ -35,14 +37,15 @@ type wconn struct {
w *bufio.Writer
}
// Registry 是设备接入网关的注册表:管理在线连接、设备元数据与已授权集合。线程安全。
// Registry 是设备接入网关的注册表:管理在线连接、设备元数据。线程安全。
// 鉴权在设备端执行,服务端不存储授权状态。
type Registry struct {
mu sync.RWMutex
devices map[string]*DeviceMeta // deviceID -> meta在线/历史)
authorized map[string]bool // deviceID -> 是否已授权(持久化恢复)
conns map[string]*wconn // deviceID -> 活跃连接(支持 push
onlineCh chan string
onStatus func(msg map[string]interface{})
onEvent func(deviceID string, msg map[string]interface{})
acceptFn func(token string) bool
cmdPending map[string]chan map[string]interface{} // reqID -> 结果 channel
results map[string]resultEntry // reqID -> 已留档结果
@ -54,10 +57,72 @@ type resultEntry struct {
Time time.Time
}
// ===== 能力矩阵caps 声明 → 工具可用性 =====
// 设备 hello 时声明自身能力caps服务端据此校验工具调用
// 摄像头只声明 camera 就不能被调 screensee/computeruse避免无效下发。
// 兼容历史值cmd/cmdrun 视为 shell 命令能力;未声明任何已知能力的设备
// (如旧版 GUI/waiter视为全能力保持向后兼容。
var capabilityTools = map[string][]string{
// 屏幕显示/查看
"screen": {"screensue", "screensee"},
"screensue": {"screensue"},
"screensee": {"screensee"},
// 鼠标键盘操控
"computeruse": {"computeruse"},
// 剪切板
"clipboard": {"clipboardsee", "clipboardsue"},
"clipboardsee": {"clipboardsee"},
"clipboardsue": {"clipboardsue"},
// 摄像头(抓拍/录像)
"camera": {"camerasue"},
"camerasue": {"camerasue"},
// 音频播放
"speaker": {"speakeruse"},
"speakeruse": {"speakeruse"},
}
// compatFullCaps 视为「全能力」的历史 caps 值:声明了这些的设备不参与能力裁剪。
var compatFullCaps = map[string]bool{
"cmd": true, "cmdrun": true, "deviceinfo": true,
"status": true, "cmdresult": true,
}
// SupportsTool 判断设备是否支持某 agent 工具(基于其声明的 caps
// 规则:
// - 设备未声明任何已知能力且无兼容全能力标记 → 视为全能力(旧设备兼容)
// - 声明了任一兼容全能力标记cmd/cmdrun 等)→ 全能力
// - 否则严格按 capabilityTools 映射匹配
func (r *Registry) SupportsTool(deviceID, tool string) bool {
r.mu.RLock()
m, ok := r.devices[deviceID]
r.mu.RUnlock()
if !ok {
return false
}
return deviceSupportsTool(m.Caps, tool)
}
func deviceSupportsTool(caps []string, tool string) bool {
hasKnown := false
for _, c := range caps {
if compatFullCaps[c] {
return true // 历史全能力设备
}
if _, known := capabilityTools[c]; known {
hasKnown = true
for _, t := range capabilityTools[c] {
if t == tool {
return true
}
}
}
}
return !hasKnown // 未声明任何已知能力 → 全能力兼容
}
func NewRegistry() *Registry {
return &Registry{
devices: make(map[string]*DeviceMeta),
authorized: make(map[string]bool),
conns: make(map[string]*wconn),
onlineCh: make(chan string, 16),
cmdPending: make(map[string]chan map[string]interface{}),
@ -79,6 +144,15 @@ func (r *Registry) SetStatusHandler(h func(msg map[string]interface{})) {
r.onStatus = h
}
// SetEventHandler 注册设备主动上报事件的回调设备→agent 单向推送)。
// 典型场景:摄像头识别到未知人员驻留、传感器报警等,设备无需 agent 轮询即可上报。
// 回调参数deviceID + 事件消息(含 type/payload 等)。
func (r *Registry) SetEventHandler(h func(deviceID string, msg map[string]interface{})) {
r.mu.Lock()
defer r.mu.Unlock()
r.onEvent = h
}
func (r *Registry) acceptBind(token string) bool {
r.mu.RLock()
fn := r.acceptFn
@ -99,21 +173,15 @@ func (r *Registry) Online(id string) bool {
return ok && m.Online
}
// Authorized 返回设备是否已授权
func (r *Registry) Authorized(id string) bool {
r.mu.RLock()
defer r.mu.RUnlock()
return r.authorized[id]
}
// Authorized 已移除授权状态由设备端自报DeviceMeta.Authorized服务端不存储
// List 返回全部设备(在线或历史),合并授权态
// List 返回全部设备(在线或历史)。
func (r *Registry) List() []DeviceMeta {
r.mu.RLock()
defer r.mu.RUnlock()
out := make([]DeviceMeta, 0, len(r.devices))
for _, m := range r.devices {
c := *m
c.Authorized = r.authorized[c.DeviceID]
out = append(out, c)
}
return out
@ -127,7 +195,6 @@ func (r *Registry) OnlineList() []DeviceMeta {
for _, m := range r.devices {
if m.Online {
c := *m
c.Authorized = r.authorized[c.DeviceID]
out = append(out, c)
}
}
@ -142,44 +209,12 @@ func (r *Registry) Get(id string) (DeviceMeta, bool) {
if !ok {
return DeviceMeta{}, false
}
c := *m
c.Authorized = r.authorized[id]
return c, true
return *m, true
}
// ============ 授权 ============
// SetAuthorized 标记某设备已授权/取消授权(持久化由插件负责)。
func (r *Registry) SetAuthorized(id string, auth bool) {
r.mu.Lock()
defer r.mu.Unlock()
r.authorized[id] = auth
if m, ok := r.devices[id]; ok {
m.Authorized = auth
}
}
// RestoreAuthorized 插件启动时从配置恢复已授权设备集合。
func (r *Registry) RestoreAuthorized(ids []string) {
r.mu.Lock()
defer r.mu.Unlock()
for _, id := range ids {
r.authorized[id] = true
}
}
// AuthorizedIDs 返回全部已授权设备 ID供插件持久化
func (r *Registry) AuthorizedIDs() []string {
r.mu.RLock()
defer r.mu.RUnlock()
var out []string
for id, ok := range r.authorized {
if ok {
out = append(out, id)
}
}
return out
}
// 已移除服务端授权存储:设备在 hello/status 中自报 authorized
// 服务端仅透传展示;实际鉴权由设备端执行(收到 cmd 后自行决定是否执行)。
// ============ 在线状态维护 ============
@ -187,7 +222,7 @@ func (r *Registry) register(meta DeviceMeta) {
r.mu.Lock()
meta.Online = true
meta.LastSeen = time.Now().Unix()
meta.Authorized = r.authorized[meta.DeviceID]
// 保留设备自报的授权状态(客户端鉴权,服务端不覆盖)
r.devices[meta.DeviceID] = &meta
r.mu.Unlock()
r.notifyChange(meta.DeviceID)
@ -260,6 +295,47 @@ func (r *Registry) PushCmd(deviceID, reqID, command, cmdType string) error {
})
}
// PushData 向设备分块下发二进制数据(网关→设备,如 TTS 音频)。
// 协议(与 GUI 设备桥协商):
//
// 文本帧 cmd_speech_start {op, req_id, kind, mime, total} → N 个二进制帧(0x2, ≤8KB) → 文本帧 cmd_speech_end {op, req_id}
//
// kind 为语义标记(如 speechmime 为数据 MIME 类型。设备聚合后按自身能力处理(播放等)。
func (r *Registry) PushData(deviceID, reqID, kind, mime string, data []byte) error {
r.mu.RLock()
c, ok := r.conns[deviceID]
r.mu.RUnlock()
if !ok {
return fmt.Errorf("device %s not online", deviceID)
}
if err := writeText(c.w, mustJSON(map[string]interface{}{
"op": "cmd_speech_start",
"req_id": reqID,
"kind": kind,
"mime": mime,
"total": len(data),
})); err != nil {
return fmt.Errorf("push data start: %w", err)
}
const chunkSize = 8192
for off := 0; off < len(data); off += chunkSize {
end := off + chunkSize
if end > len(data) {
end = len(data)
}
if err := writeBinary(c.w, data[off:end]); err != nil {
return fmt.Errorf("push data chunk: %w", err)
}
}
if err := writeText(c.w, mustJSON(map[string]interface{}{
"op": "cmd_speech_end",
"req_id": reqID,
})); err != nil {
return fmt.Errorf("push data end: %w", err)
}
return nil
}
// AwaitResult 等待某请求的结果(带超时)。
func (r *Registry) AwaitResult(reqID string, timeout time.Duration) (map[string]interface{}, error) {
ch := make(chan map[string]interface{}, 1)
@ -342,43 +418,50 @@ func httpUpgrade(w http.ResponseWriter, r *http.Request) (net.Conn, *bufio.ReadW
return conn, rw, nil
}
func readFrame(r *bufio.Reader) ([]byte, bool, error) {
// readFrame 读取一个 WS 帧。返回 (payload, isClose, err)。
// opcode: 0x1 文本 / 0x2 二进制(设备→网关大体积数据分块,如录像回传)。
func readFrame(r *bufio.Reader) ([]byte, bool, byte, error) {
b0, err := r.ReadByte()
if err != nil {
return nil, true, err
return nil, true, 0, err
}
opcode := b0 & 0x0f
b1, err := r.ReadByte()
if err != nil {
return nil, true, err
return nil, true, opcode, err
}
masked := b1&0x80 != 0
length := uint64(b1 & 0x7f)
if length == 126 {
var ext [2]byte
if _, err := io.ReadFull(r, ext[:]); err != nil {
return nil, true, err
return nil, true, opcode, err
}
length = uint64(binary.BigEndian.Uint16(ext[:]))
} else if length == 127 {
var ext [8]byte
if _, err := io.ReadFull(r, ext[:]); err != nil {
return nil, true, err
return nil, true, opcode, err
}
length = binary.BigEndian.Uint64(ext[:])
}
if length > 1<<20 {
return nil, true, fmt.Errorf("frame too large")
// 二进制帧允许更大(录像分块聚合,单帧仍限 8MB 防滥用)
maxFrame := uint64(1 << 20)
if opcode == 0x2 {
maxFrame = 8 << 20
}
if length > maxFrame {
return nil, true, opcode, fmt.Errorf("frame too large")
}
var maskKey [4]byte
if masked {
if _, err := io.ReadFull(r, maskKey[:]); err != nil {
return nil, true, err
return nil, true, opcode, err
}
}
payload := make([]byte, length)
if _, err := io.ReadFull(r, payload); err != nil {
return nil, true, err
return nil, true, opcode, err
}
if masked {
for i := range payload {
@ -386,23 +469,32 @@ func readFrame(r *bufio.Reader) ([]byte, bool, error) {
}
}
switch opcode {
case 0x1:
return payload, false, nil
case 0x1, 0x2:
return payload, false, opcode, nil
case 0x8:
return nil, true, nil
return nil, true, opcode, nil
case 0xa:
return nil, false, nil
return nil, false, opcode, nil
case 0x9:
return nil, false, errPing
return nil, false, opcode, errPing
default:
return nil, false, fmt.Errorf("unsupported opcode %x", opcode)
return nil, false, opcode, fmt.Errorf("unsupported opcode %x", opcode)
}
}
var errPing = fmt.Errorf("ping")
func writeText(w *bufio.Writer, payload []byte) error {
if err := writeFrameHeader(w, 0x1, len(payload)); err != nil {
return writeFrame(w, 0x1, payload)
}
// writeBinary 发送 WS 二进制帧0x2网关→设备大体积数据如 TTS 音频)分块下发。
func writeBinary(w *bufio.Writer, payload []byte) error {
return writeFrame(w, 0x2, payload)
}
func writeFrame(w *bufio.Writer, opcode byte, payload []byte) error {
if err := writeFrameHeader(w, opcode, len(payload)); err != nil {
return err
}
if _, err := w.Write(payload); err != nil {
@ -491,8 +583,12 @@ func (r *Registry) handleWS(conn net.Conn, rw *bufio.ReadWriter) {
}
}()
// 二进制分块聚合状态(设备→网关,如录像回传):
// cmd_data_start 开启 → 0x2 帧追加 → cmd_data_end 聚合存入 cmdresult
var dataAccum *dataAccumulator
for {
payload, isClose, err := readFrame(rw.Reader)
payload, isClose, opcode, err := readFrame(rw.Reader)
if err != nil {
if err == errPing {
if werr := writePong(rw.Writer); werr != nil {
@ -505,6 +601,23 @@ func (r *Registry) handleWS(conn net.Conn, rw *bufio.ReadWriter) {
if isClose {
return
}
if opcode == 0x2 {
// 二进制帧:处于聚合状态时追加数据块,否则忽略
if dataAccum != nil {
dataAccum.chunks = append(dataAccum.chunks, payload)
dataAccum.got += len(payload)
// 防滥用:超出声明 total 的 2 倍或硬上限 64MB 时放弃聚合
limit := int64(dataAccum.total)*2 + 1024
if limit < 64<<20 {
limit = 64 << 20
}
if int64(dataAccum.got) > limit {
log.Printf("[remotedevice] data accumulation exceeded limit for req %s, dropped", dataAccum.reqID)
dataAccum = nil
}
}
continue
}
var msg map[string]interface{}
if err := json.Unmarshal(payload, &msg); err != nil {
continue
@ -558,15 +671,93 @@ func (r *Registry) handleWS(conn net.Conn, rw *bufio.ReadWriter) {
if h != nil {
h(msg)
}
case "event":
// 设备主动上报事件(单向推送,无需回执):摄像头发现异常、传感器报警等。
// 转交插件层(经 SDK InjectText 异步注入 agent无回调时仅记日志。
id, _ := msg["device_id"].(string)
if id == "" {
id = curID
}
if id == "" {
continue
}
r.mu.RLock()
h := r.onEvent
r.mu.RUnlock()
if h != nil {
h(id, msg)
} else {
log.Printf("[remotedevice] event from %s (no handler): %v", id, msg)
}
case "cmd_result":
reqID, _ := msg["req_id"].(string)
if reqID != "" {
r.deliverResult(reqID, msg)
}
case "cmd_data_start":
reqID, _ := msg["req_id"].(string)
if reqID == "" {
continue
}
total, _ := msg["total"].(float64)
kind, _ := msg["kind"].(string)
mime, _ := msg["mime"].(string)
dataAccum = &dataAccumulator{
reqID: reqID,
kind: kind,
mime: mime,
total: int(total),
}
case "cmd_data_end":
reqID, _ := msg["req_id"].(string)
status, _ := msg["status"].(string)
if dataAccum == nil || dataAccum.reqID != reqID {
continue
}
acc := dataAccum
dataAccum = nil
if status != "ok" {
r.SaveResult(reqID, map[string]interface{}{
"op": "cmd_result", "req_id": reqID, "status": "error",
"error": "device reported transfer failure",
})
r.deliverResult(reqID, map[string]interface{}{
"op": "cmd_result", "req_id": reqID, "status": "error",
"error": "device reported transfer failure",
})
continue
}
data := make([]byte, 0, acc.got)
for _, c := range acc.chunks {
data = append(data, c...)
}
res := map[string]interface{}{
"op": "cmd_result",
"req_id": reqID,
"status": "ok",
"kind": acc.kind,
"mime": acc.mime,
"size": len(data),
"expected": acc.total,
// base64 编码完整二进制(录像 mp4 等),供 agent/上层取回后解码使用
"data_base64": base64.StdEncoding.EncodeToString(data),
}
r.SaveResult(reqID, res)
r.deliverResult(reqID, res)
}
}
}
// dataAccumulator 聚合设备→网关的二进制分块传输(如录像回传)。
type dataAccumulator struct {
reqID string
kind string
mime string
total int
chunks [][]byte
got int
}
func metaFromMsg(msg map[string]interface{}) DeviceMeta {
var meta DeviceMeta
if d, ok := msg["device"].(map[string]interface{}); ok {
@ -586,6 +777,9 @@ func metaFromMsg(msg map[string]interface{}) DeviceMeta {
}
}
}
if v, ok := d["authorized"].(bool); ok {
meta.Authorized = v
}
if info, ok := d["info"].(map[string]interface{}); ok {
if len(info) > 0 {
meta.Info = info

View File

@ -17,6 +17,10 @@
src="https://cdnjs.cloudflare.com/ajax/libs/marked/4.3.0/marked.min.js"
onerror="console.warn('marked CDN failed')"
></script>
<script
src="https://cdn.jsdelivr.net/npm/dompurify@3.2.4/dist/purify.min.js"
onerror="console.warn('DOMPurify CDN failed')"
></script>
<script>
setTimeout(function () {
if (!window.THREE) window._THREE_FAILED = true;
@ -1741,9 +1745,6 @@ background:
flex-shrink: 0;
margin-bottom: 0;
}
#sm-container-chat {
height: 480px;
}
.loading {
display: inline-block;
width: 16px;
@ -1852,119 +1853,6 @@ background:
transform: translateY(0);
}
}
#starmap-container {
width: 100%;
height: calc(100vh - 100px);
position: relative;
overflow: hidden;
border-radius: var(--radius-md);
border: 1px solid var(--border-color);
background: var(--bg-input);
}
#starmap-container canvas {
display: block;
}
#starmap-stats {
position: absolute;
top: 16px;
left: 16px;
background: var(--glass-bg-strong);
padding: 12px 16px;
border-radius: var(--radius-md);
border: 1px solid var(--glass-border);
font-size: 13px;
z-index: 10;
backdrop-filter: blur(12px);
-webkit-backdrop-filter: blur(12px);
color: var(--text-secondary);
box-shadow: var(--shadow-md);
}
#starmap-stats h3 {
margin-bottom: 6px;
color: var(--accent);
font-size: 14px;
}
#starmap-stats p {
margin: 2px 0;
color: var(--text-muted);
font-size: 12px;
}
#starmap-stats span {
color: var(--text-primary);
font-weight: 700;
}
#starmap-info {
position: absolute;
top: 16px;
right: 16px;
background: var(--glass-bg-strong);
padding: 12px 16px;
border-radius: var(--radius-md);
border: 1px solid var(--glass-border);
font-size: 13px;
z-index: 10;
display: none;
backdrop-filter: blur(12px);
-webkit-backdrop-filter: blur(12px);
color: var(--text-secondary);
max-width: 260px;
box-shadow: var(--shadow-md);
}
#starmap-info h3 {
color: var(--success);
margin-bottom: 6px;
font-size: 14px;
}
#starmap-info p {
margin: 2px 0;
color: var(--text-muted);
font-size: 12px;
}
#starmap-info .label {
color: var(--text-muted);
}
#starmap-loading {
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
font-size: 18px;
color: var(--accent);
z-index: 20;
}
.starmap-toggle {
position: absolute;
bottom: 16px;
left: 50%;
transform: translateX(-50%);
display: flex;
gap: 8px;
z-index: 10;
}
.starmap-toggle button {
background: var(--glass-bg-strong);
border: 1px solid var(--glass-border);
color: var(--text-secondary);
padding: 6px 14px;
border-radius: var(--radius-pill);
cursor: pointer;
font-size: 12px;
font-family: inherit;
backdrop-filter: blur(10px);
-webkit-backdrop-filter: blur(10px);
transition: all 0.2s;
}
.starmap-toggle button:hover {
background: var(--accent-bg);
color: var(--text-primary);
border-color: rgba(255, 127, 172, 0.5);
}
.starmap-toggle button.on {
background: var(--accent-bg);
color: var(--accent);
border-color: var(--accent);
box-shadow: var(--shadow-glow);
}
#sm-container-chat {
height: 260px;
background: var(--bg-input);
@ -2136,22 +2024,6 @@ background:
.kv-row .key {
width: auto;
}
#starmap-stats {
top: 8px;
left: 8px;
padding: 8px 10px;
font-size: 11px;
}
#starmap-info {
top: 8px;
right: 8px;
padding: 8px 10px;
max-width: 180px;
font-size: 11px;
}
#starmap-container {
height: calc(100vh - 60px);
}
.msg-avatar {
width: 24px;
height: 24px;
@ -2389,11 +2261,6 @@ background:
var ICON_MOON =
'<svg viewBox="0 0 24 24" width="15" height="15" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round"><path d="M21 12.8A9 9 0 1 1 11.2 3a7 7 0 0 0 9.8 9.8z"/></svg>';
function systemTheme() {
return window.matchMedia("(prefers-color-scheme: dark)").matches
? "dark"
: "light";
}
function setTheme(name) {
document.documentElement.setAttribute("data-theme", name);
@ -2508,6 +2375,25 @@ background:
})();
// ===== Utility =====
function renderMd(text) {
if (typeof text !== "string") text = String(text || "");
var html;
if (typeof marked !== "undefined") {
try { html = marked.parse(text); }
catch (e) { html = escHtml(text); }
} else {
html = "<pre>" + escHtml(text) + "</pre>";
}
if (typeof DOMPurify !== "undefined" && typeof DOMPurify.sanitize === "function") {
try { return DOMPurify.sanitize(html, { USE_PROFILES: { html: true } }); }
catch (e) {}
}
return html
.replace(/<script[\s\S]*?<\/script>/gi, "")
.replace(/\son\w+\s*=\s*"[^"]*"/gi, "")
.replace(/\son\w+\s*=\s*'[^']*'/gi, "")
.replace(/javascript:/gi, "");
}
function escHtml(s) {
return String(s)
.replace(/&/g, "&amp;")
@ -2516,13 +2402,6 @@ background:
.replace(/"/g, "&quot;");
}
function timeAgo(t) {
var s = Math.floor((Date.now() - new Date(t).getTime()) / 1000);
if (s < 60) return s + "秒前";
var m = Math.floor(s / 60);
if (m < 60) return m + "分钟前";
return Math.floor(m / 60) + "小时前";
}
function toast(m, isError) {
var t = document.getElementById("toast");
@ -2536,74 +2415,7 @@ background:
}
// ===== 8.6 Unified toast + confirm dialog =====
function showToast(type, msg) {
var cls = "toast";
if (type === "success") cls += " success";
else if (type === "error") cls += " error";
else if (type === "warn") cls += " warn";
var t = document.getElementById("toast");
t.textContent = msg;
t.className = cls;
t.style.display = "block";
t.style.animation = "none";
void t.offsetWidth;
t.style.animation = "";
clearTimeout(t._hideTimer);
t._hideTimer = setTimeout(function () {
t.style.display = "none";
}, 3000);
}
function confirmDialog(action, onConfirm) {
var existing = document.getElementById("confirm-overlay");
if (existing) existing.remove();
var o = document.createElement("div");
o.id = "confirm-overlay";
o.className = "confirm-overlay";
o.innerHTML =
'<div class="confirm-box" role="alertdialog" aria-modal="true" aria-labelledby="confirm-title">' +
'<h3 id="confirm-title">' +
__("确认操作", "Confirm action") +
"</h3>" +
"<p>" +
escHtml(action) +
"</p>" +
'<div class="confirm-actions">' +
'<button class="btn btn-ghost" data-confirm="no">' +
__("取消", "Cancel") +
"</button>" +
'<button class="btn btn-primary" data-confirm="yes">' +
__("确认", "Confirm") +
"</button>" +
"</div></div>";
document.body.appendChild(o);
o.style.display = "flex";
o.querySelector('[data-confirm="no"]').focus();
function close(ok) {
o.remove();
if (ok && onConfirm) onConfirm();
}
o.querySelector('[data-confirm="yes"]').addEventListener(
"click",
function () {
close(true);
},
);
o.querySelector('[data-confirm="no"]').addEventListener(
"click",
function () {
close(false);
},
);
o.addEventListener("click", function (e) {
if (e.target === o) close(false);
});
document.addEventListener("keydown", function (ev) {
if (!document.getElementById("confirm-overlay")) return;
if (ev.key === "Escape") close(false);
if (ev.key === "Enter") close(true);
});
}
// ===== 8.4 Card 3D tilt + cursor glow =====
document.addEventListener("mousemove", function (e) {
@ -2800,15 +2612,6 @@ background:
}
// ===== Overview =====
function statCard(l, v) {
return (
'<div class="card stat-card"><div class="stat-value">' +
v +
'</div><div class="stat-label">' +
l +
"</div></div>"
);
}
function renderOverview() {
var s = state.status || {};
@ -2971,6 +2774,9 @@ background:
'<input id="chat-input" placeholder="' +
__("输入消息...", "Type a message...") +
'" onkeydown="if(event.key==\'Enter\')sendChat()">' +
'<button class="btn" onclick="interruptChat()" id="chat-stop-btn" style="display:none;background:var(--danger, #d1383d);color:#fff">' +
__("停止", "Stop") +
"</button>" +
'<button class="btn btn-primary" onclick="sendChat()" id="chat-send-btn">' +
__("发送", "Send") +
"</button>" +
@ -3200,11 +3006,7 @@ background:
var role = m.role || "user";
var c = m.content || "";
if (role === "assistant") {
if (typeof marked !== "undefined") {
c = marked.parse(c);
} else {
c = "<pre>" + escHtml(c) + "</pre>";
}
c = renderMd(c);
} else if (role === "system") {
c = escHtml(c);
} else {
@ -3365,11 +3167,74 @@ background:
badge.style.display = "none";
}
function rerenderChat() {
// 流式渲染分发流式增量只更新最后一条正文vs 全量重建
var _rerenderTimer = null;
function rerenderChat(full) {
// 流式中且非强制全量:走增量路径(防抖合并 chunk只更新最后一条消息节点
if (!full && state.chatLoading) {
var msgs = state.messages;
var last = msgs.length ? msgs[msgs.length - 1] : null;
if (last && last.role === "assistant" && !last._final) {
if (_rerenderTimer) return; // 已有排程的增量更新
_rerenderTimer = setTimeout(function () {
_rerenderTimer = null;
renderChatStreamChunk();
}, 90);
return;
}
}
// 非流式(完成/工具/历史变化):全量重渲(含防抖合并)
if (_rerenderTimer) clearTimeout(_rerenderTimer);
_rerenderTimer = setTimeout(function () {
_rerenderTimer = null;
renderChat();
if (full) {
renderChatStarmap();
renderTerminals();
renderCmdHistory();
}
}, 90);
}
// 流式增量渲染(移植自 GUI仅更新最后一条 assistant 消息的正文与思考预览,
// 不重建 DOM。正文节流 parse>200 字符或 >300ms 才 renderMd小增量纯文本追加。
function renderChatStreamChunk() {
var msgsEl = document.getElementById("chat-msgs");
var last = state.messages.length ? state.messages[state.messages.length - 1] : null;
if (!msgsEl || !last) return;
var el = msgsEl.lastElementChild;
if (!el) { renderChat(); return; }
var textEl = el.querySelector(".msg-bubble .text");
var c = last.content || "";
if (textEl && c) {
var now = Date.now();
var lastParse = el.__lastParse || 0;
var lastLen = el.__lastLen || 0;
if (c.length - lastLen > 200 || now - lastParse > 300) {
textEl.innerHTML = renderMd(c);
el.__lastParse = now;
el.__lastLen = c.length;
} else {
var tail = c.slice(lastLen);
if (tail) textEl.appendChild(document.createTextNode(tail));
el.__lastLen = c.length;
}
if (state.chatStick !== false) {
try { msgsEl.scrollTop = msgsEl.scrollHeight; } catch (e) {}
}
return;
}
// 思考预览更新(流式中折叠,只刷 preview 文本)
var rcPrev = el.querySelector(".reasoning-preview");
if (rcPrev && last.reasoning_content) {
rcPrev.textContent = last.reasoning_content.replace(/[\s\n]+/g, " ").slice(0, 60);
if (state.chatStick !== false) {
try { msgsEl.scrollTop = msgsEl.scrollHeight; } catch (e) {}
}
return;
}
// 结构变化兜底:全量
renderChat();
renderChatStarmap();
renderTerminals();
renderCmdHistory();
}
function toggleToolCall(el) {
var d = el.querySelector(".tc-detail");
@ -3384,23 +3249,20 @@ background:
}
function renderReasoningCard(text, isStreaming) {
var body =
typeof marked !== "undefined"
? marked.parse(text)
: escHtml(text);
var preview =
typeof marked !== "undefined"
? text.replace(/[\s\n]+/g, " ").slice(0, 60)
: escHtml(text).replace(/<[^>]+>/g, " ").slice(0, 60);
return (
'<div class="reasoning-card">' +
'<div class="reasoning-card' + (isStreaming ? " rc-streaming" : "") + '">' +
'<div class="reasoning-head" onclick="toggleReasoning(this)">' +
'<svg viewBox="0 0 24 24" width="14" height="14" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="rc-ico"><path d="M9 3a2 2 0 0 0-2 2v2a2 2 0 0 1-2 2H3a2 2 0 0 0-2 2v6a2 2 0 0 0 2 2h2a2 2 0 0 1 2 2v2a2 2 0 0 0 2 2h1a2 2 0 0 0 2-2v-2a2 2 0 0 1 2-2h2a2 2 0 0 0 2-2V7a2 2 0 0 0-2-2h-2a2 2 0 0 1-2-2V3a2 2 0 0 0-2-2H9zM12 8v4m0 4h.01"/></svg>' +
'<span class="rc-title">' + (isStreaming ? __("思考中...", "Thinking...") : __("思考", "Thinking")) + '</span>' +
'<span class="rc-chev">▾</span></div>' +
'<div class="reasoning-body" style="display:' + (isStreaming ? "block" : "none") + '">' +
'<div class="reasoning-content">' + body + '</div>' +
(isStreaming ? '<div class="reasoning-sweep"></div>' : "") +
(isStreaming
? '<div class="reasoning-preview">' + escHtml(preview) + '</div><div class="reasoning-sweep"></div>'
: '<div class="reasoning-content">' + renderMd(text) + '</div>') +
'</div></div>'
);
}
@ -3502,9 +3364,6 @@ background:
}
}
function getStarmapBg() {
return 0x0a0a1a;
}
function initChatStarmap() {
var cont = document.getElementById("sm-container-chat");
@ -3774,81 +3633,150 @@ background:
});
}
// 回合收尾:由 SSE 事件agent_output final / reset 帧)或 watchdog 驱动。
// POST 结束 ≠ 回合结束agent 可能还在生成(排队+长生成),提前复位
// chatLoading 会让后续 delta 走全量重建、停止按钮消失、用户误发重复消息。
function endChatTurn() {
if (!state.chatLoading) return;
state.chatLoading = false;
state.chatStage = "";
if (state._turnWatchdog) {
clearTimeout(state._turnWatchdog);
state._turnWatchdog = null;
}
var btn = document.getElementById("chat-send-btn");
if (btn) {
btn.disabled = false;
btn.textContent = __("发送", "Send");
}
var sb = document.getElementById("chat-stop-btn");
if (sb) sb.style.display = "none";
rerenderChat(true);
}
// 回合看门狗POST 已 abort 且 SSE 迟迟无终帧时兕底收尾(连接不稳/事件丢失),
// 提示用户回复可能已生成、可刷新查看历史。避免回合永久卡在 loading。
function armTurnWatchdog() {
if (state._turnWatchdog) clearTimeout(state._turnWatchdog);
state._turnWatchdog = setTimeout(function () {
state._turnWatchdog = null;
if (state.chatLoading) {
endChatTurn();
toast(
__(
"长时间未收到回复,连接可能不稳定;回复可能已生成,可刷新页面查看",
"No reply received for a long time; the reply may have been generated, refresh to check",
),
true,
);
}
}, 120000);
}
async function sendChat() {
var inp = document.getElementById("chat-input");
var btn = document.getElementById("chat-send-btn");
var stopBtn = document.getElementById("chat-stop-btn");
var text = inp.value.trim();
if (!text || state.chatLoading) return;
state.chatStick = true;
state.chatFinalIdx = -1;
state.messages.push({ role: "user", content: text });
inp.value = "";
rerenderChat();
rerenderChat(true);
state.chatLoading = true;
state.chatStage = __("等待AI回复...", "Waiting for AI...");
btn.disabled = true;
btn.textContent = "";
rerenderChat();
if (stopBtn) stopBtn.style.display = ""; // 生成期间可停止
rerenderChat(true);
// 触发式 POST短超时仅确认受理回复靠 SSE 流式渲染(对齐 GUI 行为)。
try {
var r = await api("/chat", {
method: "POST",
body: JSON.stringify({ message: text }),
});
state.chatStage = "";
var last = state.messages[state.messages.length - 1];
console.log(
"[sendChat] POST returned, last msg:",
last
? {
role: last.role,
_streaming: last._streaming,
_final: last._final,
tool_calls: last.tool_calls?.length,
content_len: last.content?.length,
}
: null,
);
if (last && last.role === "assistant" && last._streaming) {
console.log(
"[sendChat] updating existing streaming msg, tool_calls before:",
last.tool_calls?.length,
var ctrl = new AbortController();
var ackTimer = setTimeout(function () {
ctrl.abort();
}, 15000);
var r = null;
try {
r = await api("/chat", {
method: "POST",
body: JSON.stringify({ message: text }),
signal: ctrl.signal,
});
} catch (ackErr) {
// 同步超时/失败:不阻塞 UI等 SSE 兑底;明确提示"可能已发送"
console.warn("[sendChat] trigger failed: " + ackErr.message);
toast(
__(
"请求超时(可能已发送,请稍候或在收到回复前勿重复发送)",
"Request timeout (may have been sent; wait for reply before resending)",
),
true,
);
last.content = r.response || __("(无响应)", "(no response)");
last._grow = true;
if (!last.reasoning_content) {
last.reasoning_content = r.reasoning_content || "";
r = null;
} finally {
clearTimeout(ackTimer);
}
var last = state.messages[state.messages.length - 1];
if (r && r.response) {
if (last && last.role === "assistant" && last._streaming) {
last.content = r.response || __("(无响应)", "(no response)");
last._grow = true;
if (!last.reasoning_content) {
last.reasoning_content = r.reasoning_content || "";
}
last._final = true;
delete last._streaming;
} else {
state.messages.push({
role: "assistant",
content: r.response || __("(无响应)", "(no response)"),
reasoning_content: r.reasoning_content,
tool_calls:
last && last.role === "assistant" && last.tool_calls
? last.tool_calls
: [],
_final: true,
_grow: true,
});
}
last._final = true;
delete last._streaming;
} else {
state.chatFinalIdx = state.messages.length - 1;
}
rerenderChat(true);
} catch (e) {
// 真实错误(非受理超时):展示错误信息
if (!String(e.message || "").includes("aborted")) {
state.messages.push({
role: "assistant",
content: r.response || __("(无响应)", "(no response)"),
reasoning_content: r.reasoning_content,
tool_calls:
last && last.role === "assistant" && last.tool_calls
? last.tool_calls
: [],
content: __("错误: ", "Error: ") + e.message,
_final: true,
_grow: true,
});
rerenderChat(true);
toast(__("请求失败: ", "Request failed: ") + e.message, true);
}
state.chatFinalIdx = state.messages.length - 1;
rerenderChat();
} catch (e) {
state.messages.push({
role: "assistant",
content: __("错误: ", "Error: ") + e.message,
_final: true,
});
rerenderChat();
toast(__("请求失败: ", "Request failed: ") + e.message, true);
} finally {
state.chatLoading = false;
state.chatStage = "";
btn.disabled = false;
btn.textContent = __("发送", "Send");
rerenderChat();
if (r && r.response) {
// 同步兜底已拿到完整回复:回合结束
endChatTurn();
} else {
// 触发式受理POST 已 abort/失败):回合仍打开,等 SSE 流式渲染;
// 由 agent_output final / reset 帧 / watchdog 收尾
armTurnWatchdog();
}
}
}
// 停止生成 / 发送中断消息。核心拦截语义:有 LLM 在跑则取消当前
// 请求并以 [中断消息] 重启轮次;无则在跑则作为普通消息处理。
async function interruptChat() {
try {
await api("/chat/interrupt", {
method: "POST",
body: JSON.stringify({}),
});
toast(__("已发送中断信号", "Interrupt signal sent"));
} catch (e) {
toast(__("中断失败: ", "Interrupt failed: ") + e.message, true);
}
}
@ -4252,9 +4180,13 @@ background:
lastM2.role === "assistant" &&
!lastM2._final
) {
// 聚合最终响应:覆盖 delta 累积的中间内容(以聚合为准,
// 含 stage 插件改写后的最终文本),并置 final 结束本轮流式。
lastM2._grow = true;
lastM2.content += p.content;
lastM2.content = p.content;
lastM2._final = true;
rerenderChat();
endChatTurn();
return;
}
if (
@ -4263,19 +4195,68 @@ background:
lastM2._final &&
!lastM2.source
) {
return;
// 去重:同一轮的重复帧(如 SSE 重连回放)内容相同则忽略;
// 内容不同视为新一轮输出(上一轮已 final 且无 source开新消息。
// 旧逻辑无条件 return 会丢弃多轮连发时新一轮的最终回复。
if (lastM2.content === p.content) {
endChatTurn();
return;
}
}
state.messages.push({
role: "assistant",
content: p.content,
_streaming: true,
_grow: true,
_final: true,
});
rerenderChat();
endChatTurn();
} catch (ex) {
console.error("[SSE] agent_output error", ex);
}
});
// token 级流式增量:逐块追加到当前回复内容(流式生成中);
// reset 帧表示轮次作废(用户中断):定格已显示的部分内容,置 final。
es.addEventListener("content_delta", function (e) {
try {
var ev = JSON.parse(e.data);
var p = ev.payload || {};
if (p.channel === "_consolidation_") return;
if (p.reset) {
var lm = state.messages.length
? state.messages[state.messages.length - 1]
: null;
if (lm && lm.role === "assistant" && !lm._final) {
lm._final = true;
rerenderChat();
}
// 轮次作废(用户中断):定格已显示内容;核心会以 [中断消息]
// 重启轮次,保持回合打开让确认回复继续流式渲染,
// 由其 agent_output final / watchdog 收尾。
armTurnWatchdog();
return;
}
if (!p.content) return;
state.chatStage = __("AI 回复中...", "AI replying...");
var last =
state.messages.length > 0
? state.messages[state.messages.length - 1]
: null;
if (!last || last.role !== "assistant" || last._final) {
state.messages.push({
role: "assistant",
content: "",
tool_calls: [],
_streaming: true,
_grow: true,
});
last = state.messages[state.messages.length - 1];
}
last.content += p.content;
rerenderChat();
} catch (ex) {}
});
es.addEventListener("terminal_output", function (e) {
try {
var ev = JSON.parse(e.data);
@ -4303,6 +4284,7 @@ background:
try {
var ev = JSON.parse(e.data);
var p = ev.payload || {};
if (p.channel === "_consolidation_") return;
if (p.content) {
state.chatStage = __("AI 思考中...", "AI thinking...");
var last =
@ -4319,12 +4301,39 @@ background:
});
last = state.messages[state.messages.length - 1];
}
last.reasoning_content =
(last.reasoning_content || "") + p.content;
// 聚合 reasoning 帧携带全文:直接覆盖(若已有 delta 累积则等价)
last.reasoning_content = p.content;
rerenderChat();
}
} catch (ex) {}
});
// token 级流式增量逐块追加到当前思考内容reset 帧表示轮次作废
es.addEventListener("reasoning_delta", function (e) {
try {
var ev = JSON.parse(e.data);
var p = ev.payload || {};
if (p.channel === "_consolidation_") return;
if (p.reset) return; // 轮次作废(用户中断):清空累积中的思考
if (!p.content) return;
state.chatStage = __("AI 思考中...", "AI thinking...");
var last =
state.messages.length > 0
? state.messages[state.messages.length - 1]
: null;
if (!last || last.role !== "assistant" || last._final) {
state.messages.push({
role: "assistant",
content: "",
reasoning_content: "",
tool_calls: [],
_streaming: true,
});
last = state.messages[state.messages.length - 1];
}
last.reasoning_content = (last.reasoning_content || "") + p.content;
rerenderChat();
} catch (ex) {}
});
es.addEventListener("tool_call", function (e) {
try {
var ev = JSON.parse(e.data);
@ -4361,15 +4370,7 @@ background:
console.error("[SSE] tool_call error", ex);
}
});
es.addEventListener("tool_result", function (e) {
try {
var ev = JSON.parse(e.data);
var p = ev.payload || {};
state.chatStage = __("工具结果已返回", "Tool result received");
var badge = document.getElementById("chat-stage");
if (badge) badge.textContent = state.chatStage;
} catch (ex) {}
});
// 注:后端不发布 tool_result 类型事件(工具结果随 EventToolCall 一次发出),无此监听器。
es.addEventListener("stage", function (e) {
try {
var ev = JSON.parse(e.data);
@ -4651,10 +4652,21 @@ background:
return;
}
try {
var r = await api("/plugins", {
var raw = await api("/plugins", {
method: "POST",
body: JSON.stringify({ url: url }),
raw: true,
});
var ct = raw.headers.get("content-type") || "";
var r = ct.includes("json") ? await raw.json() : await raw.text();
if (!raw.ok || (r && r.error)) {
toast(
__("安装失败: ", "Install failed: ") +
((r && (r.error || r.details)) || "HTTP " + raw.status),
true,
);
return;
}
toast(
__("安装结果: ", "Install result: ") +
(r.status || JSON.stringify(r)),
@ -4667,7 +4679,7 @@ background:
),
false,
);
loadInstalledPlugins();
await loadInstalledPlugins();
renderPlugins();
} catch (e) {
toast(__("安装失败: ", "Install failed: ") + e.message, true);
@ -4716,7 +4728,10 @@ background:
renderPlugins();
}
var removingPlugins = {};
async function removePlugin(name) {
if (removingPlugins[name]) return; // 防重复点击
if (
!confirm(
__("确定卸载插件", "Are you sure to unload plugin") +
@ -4726,23 +4741,39 @@ background:
)
)
return;
removingPlugins[name] = true;
try {
var r = await api("/plugins/" + encodeURIComponent(name), {
method: "DELETE",
});
toast(__("已卸载: ", "Unloaded: ") + (r.status || r.name));
if (r.action === "reload_required")
toast(
__(
"已卸载,请点击「重载插件」生效",
'Unloaded, click "Reload Plugins" to apply',
),
false,
);
loadInstalledPlugins();
var raw = await api(
"/plugins/" + encodeURIComponent(name),
{ method: "DELETE", raw: true },
);
var ct = raw.headers.get("content-type") || "";
var body = ct.includes("json")
? await raw.json()
: await raw.text();
if (!raw.ok) {
var em =
(body && (body.error || body.details)) ||
("HTTP " + raw.status);
toast(__("卸载失败: ", "Unload failed: ") + em, true);
// 内置插件或路径错误时刷新一次列表保持状态一致
loadInstalledPlugins();
renderPlugins();
return;
}
toast(__("已卸载: ", "Unloaded: ") + (body.name || body.status || name));
await loadInstalledPlugins();
// 同步内核插件/禁用列表,确保列表与工具立即消失
try {
state.kernel = await api("/kernel");
var s = await api("/settings");
state.disabledPlugins = s.disabled_plugins || [];
} catch (e2) {}
renderPlugins();
} catch (e) {
toast(__("卸载失败: ", "Unload failed: ") + e.message, true);
} finally {
delete removingPlugins[name];
}
}
@ -5080,35 +5111,18 @@ background:
var rc = new THREE.Raycaster();
rc.setFromCamera(mouse, starmapCam);
var hits = rc.intersectObjects(starmapNodeMeshes);
var infoEl = document.getElementById("starmap-info");
if (hits.length > 0) {
var n = hits[0].object;
if (starmapHovered !== n) {
if (starmapHovered) starmapHovered.scale.set(1, 1, 1);
starmapHovered = n;
n.scale.set(1.2, 1.2, 1.2);
var nd = n.userData.nodeData;
if (infoEl) {
var e1 = document.getElementById("sm-info-name");
if (e1) e1.textContent = nd.name || "";
var e2 = document.getElementById("sm-info-type");
if (e2) e2.textContent = nd.type || "";
var e3 = document.getElementById("sm-info-mentions");
if (e3) e3.textContent = (nd.mention_count || 0) + "";
var lk = starmapEdges.filter(function (e) {
return e.source_id === nd.id || e.target_id === nd.id;
}).length;
var e4 = document.getElementById("sm-info-links");
if (e4) e4.textContent = lk + "";
infoEl.style.display = "block";
}
}
} else {
if (starmapHovered) {
starmapHovered.scale.set(1, 1, 1);
starmapHovered = null;
}
if (!starmapSelected && infoEl) infoEl.style.display = "none";
}
}
@ -5168,33 +5182,7 @@ background:
}
}
function toggleStarmapAuto() {
starmapAutoView = !starmapAutoView;
var b = document.getElementById("sm-auto-btn");
if (b) b.className = starmapAutoView ? "on" : "";
}
function resetStarmapCamera() {
if (!starmapCam || !starmapCtrl || !starmapNodeMeshes) return;
var maxD = 0;
starmapNodeMeshes.forEach(function (m) {
var d = m.position.length();
if (d > maxD) maxD = d;
});
if (maxD < 1) maxD = 30;
var td = Math.min(Math.max(maxD + 20, 30), 150);
var sp = starmapCam.position.clone(),
ep = new THREE.Vector3(td * 0.9, td * 0.6, td * 0.9);
var st = starmapCtrl.target.clone(),
t0 = Date.now();
(function lerp() {
var t = Math.min((Date.now() - t0) / 400, 1),
e = 1 - Math.pow(1 - t, 3);
starmapCam.position.lerpVectors(sp, ep, e);
starmapCtrl.target.lerpVectors(st, new THREE.Vector3(0, 0, 0), e);
if (t < 1) requestAnimationFrame(lerp);
})();
}
function starmapAnimate() {
starmapRaf = requestAnimationFrame(starmapAnimate);

File diff suppressed because one or more lines are too long

View File

@ -66,6 +66,51 @@ func init() {
}
}
// sseEventRecord 保存一条 SSE 事件元数据,供断线重连时按 Last-Event-ID 重放遗漏事件。
type sseEventRecord struct {
id string // SSE 事件 id 值(如 "1234567890-5"
eventType string // 事件类型agent_output, reasoning 等)
data json.RawMessage // 序列化后的 payload JSON
}
// sseEventRing 是一个固定大小的环状缓冲区,保持最近 cap 条 SSE 事件。
type sseEventRing struct {
mu sync.Mutex
buf []sseEventRecord
cap int
}
func newSSEEventRing(cap int) *sseEventRing {
return &sseEventRing{cap: cap}
}
// Append 追加一条事件,超过容量时丢弃最旧条目。
func (r *sseEventRing) Append(id, eventType string, data json.RawMessage) {
r.mu.Lock()
defer r.mu.Unlock()
r.buf = append(r.buf, sseEventRecord{id: id, eventType: eventType, data: data})
if len(r.buf) > r.cap {
r.buf = r.buf[len(r.buf)-r.cap:]
}
}
// After 返回所有在指定 id 之后的事件(按写入顺序),若 id 不在缓冲区中则返回全部。
func (r *sseEventRing) After(id string) []sseEventRecord {
r.mu.Lock()
defer r.mu.Unlock()
for i := len(r.buf) - 1; i >= 0; i-- {
if r.buf[i].id == id {
result := make([]sseEventRecord, len(r.buf)-i-1)
copy(result, r.buf[i+1:])
return result
}
}
// ID 不在缓冲区(可能是太旧或从未收到),返回全部
result := make([]sseEventRecord, len(r.buf))
copy(result, r.buf)
return result
}
type Handler struct {
sdk *sdk.PluginSDK
supervisor sdk.SupervisorAPI
@ -85,13 +130,19 @@ type Handler struct {
sessionMu sync.Mutex
sessions map[string]time.Time
sseEvents *sseEventRing // SSE 事件环状缓冲区Last-Event-ID 重放用
chatMu sync.Mutex
chatHistory []ChatMsg
pendingIdx int // chatHistory 中正在进行的 assistant 消息索引,-1 表示无
cmdMu sync.Mutex
cmdHistory []CmdExec
termMu sync.Mutex
termStates map[string]*termState
chatMsgMu sync.Mutex
chatMsgCache map[string]*chatMsgEntry // client_msg_id -> 首次处理结果
chatMsgOrder []string // FIFO 淘汰序
cmdMu sync.Mutex
cmdHistory []CmdExec
termMu sync.Mutex
termStates map[string]*termState
}
type ChatMsg struct {
@ -135,6 +186,40 @@ const maxChatHistory = 200
const maxCmdHistory = 100
const maxTerminals = 50
// ===== client_msg_id 去重(防 GUI 断线重连/超时重试导致的消息重放)=====
// GUI 端每条发送消息带唯一 client_msg_id服务端按 ID 单飞singleflight
// 首次请求正常注入 agent同 ID 重放等待首次结果并直接复用,不再重复处理。
const maxChatMsgCache = 256
type chatMsgEntry struct {
done chan struct{}
resp *agentIO.OutputEvent
}
func (h *Handler) claimChatMsg(id string) (*chatMsgEntry, bool) {
h.chatMsgMu.Lock()
defer h.chatMsgMu.Unlock()
if e, ok := h.chatMsgCache[id]; ok {
return e, true
}
e := &chatMsgEntry{done: make(chan struct{})}
h.chatMsgCache[id] = e
h.chatMsgOrder = append(h.chatMsgOrder, id)
if len(h.chatMsgOrder) > maxChatMsgCache {
old := h.chatMsgOrder[0]
h.chatMsgOrder = h.chatMsgOrder[1:]
delete(h.chatMsgCache, old)
}
return e, false
}
// completeChatMsg 记录首次处理结果并唤醒所有等待的同 ID 重放请求。
func (h *Handler) completeChatMsg(e *chatMsgEntry, resp *agentIO.OutputEvent) {
e.resp = resp
close(e.done)
}
func NewHandler(s *sdk.PluginSDK) *Handler {
var (
sup sdk.SupervisorAPI
@ -158,23 +243,25 @@ func NewHandler(s *sdk.PluginSDK) *Handler {
st, llm = s.Status(), s.LLM()
}
h := &Handler{
sdk: s,
supervisor: sup,
memory: mem,
indexer: idx,
adapter: ad,
config: cfg,
startTime: time.Now(),
textMem: tm,
knowledge: ks,
tracker: tr,
settings: se,
pluginMgr: pm,
status: st,
llm: llm,
sessions: make(map[string]time.Time),
termStates: make(map[string]*termState),
pendingIdx: -1,
sdk: s,
supervisor: sup,
memory: mem,
indexer: idx,
adapter: ad,
config: cfg,
startTime: time.Now(),
textMem: tm,
knowledge: ks,
tracker: tr,
settings: se,
pluginMgr: pm,
status: st,
llm: llm,
sessions: make(map[string]time.Time),
termStates: make(map[string]*termState),
pendingIdx: -1,
chatMsgCache: make(map[string]*chatMsgEntry),
sseEvents: newSSEEventRing(200),
}
h.loadChatHistory()
if s != nil {
@ -607,6 +694,7 @@ func (h *Handler) RegisterRoutes(mux *http.ServeMux) {
mux.HandleFunc("/api/v1/tracker/", h.requireAPI(h.handleTracker))
mux.HandleFunc("/api/v1/chat", h.requireAPI(h.handleChat))
mux.HandleFunc("/api/v1/chat/history", h.requireAPI(h.handleChatHistory))
mux.HandleFunc("/api/v1/chat/interrupt", h.requireAPI(h.handleChatInterrupt))
mux.HandleFunc("/api/v1/chat/events", h.requireAPI(h.handleChatEvents))
mux.HandleFunc("/api/v1/terminals", h.requireAPI(h.handleTerminals))
mux.HandleFunc("/api/v1/cmd/history", h.requireAPI(h.handleCmdHistory))
@ -1134,6 +1222,39 @@ func (h *Handler) handleChatHistory(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]interface{}{"messages": result})
}
// handleChatInterrupt 注入用户中断:取消正在进行的 LLM 生成并/或发送打断消息。
// 核心拦截语义interceptLoop
// - 有 LLM 在跑cancelLLM 取消当前请求 + 中断入队process() 以
// [中断消息] 重启轮次,模型看到被打断的上下文和用户新输入;
// - 无 LLM 在跑:作为普通输入处理(等同发了一条消息)。
// message 可选:空则纯取消(仍会注入空内容中断触发取消)。
func (h *Handler) handleChatInterrupt(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
return
}
if h.sdk == nil {
writeJSON(w, http.StatusServiceUnavailable, map[string]string{"error": "agent unavailable"})
return
}
var body struct {
Message string `json:"message"`
DeviceID string `json:"device_id"`
}
if r.Body != nil {
_ = json.NewDecoder(r.Body).Decode(&body) // body 可选
}
source := "webui"
if body.DeviceID != "" {
source = "webui/" + body.DeviceID
}
h.sdk.InjectInterrupt(source, "webui", "text", map[string]interface{}{
"content": body.Message,
})
writeJSON(w, http.StatusOK, map[string]string{"status": "interrupted"})
}
func (h *Handler) handleTerminals(w http.ResponseWriter, r *http.Request) {
h.termMu.Lock()
terms := make([]*termState, 0, len(h.termStates))
@ -1159,7 +1280,10 @@ func (h *Handler) handleChat(w http.ResponseWriter, r *http.Request) {
return
}
var body struct {
Message string `json:"message"`
Message string `json:"message"`
DeviceID string `json:"device_id"` // 消息来源设备GUI/受控设备),可选
DeviceName string `json:"device_name"` // 设备显示名,可选
ClientMsgID string `json:"client_msg_id"` // 客户端唯一消息 ID防断线重放/超时重试)
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid json"})
@ -1174,23 +1298,77 @@ func (h *Handler) handleChat(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusServiceUnavailable, map[string]string{"error": "agent unavailable"})
return
}
// 带超时的上下文,防止 InjectTextSync 长时间阻塞 HTTP 请求
ctx, cancel := context.WithTimeout(r.Context(), 60*time.Second)
// client_msg_id 去重:同 ID 重放等待首次结果直接复用,不重复注入 agent。
// 无 ID 的旧客户端走原路径agent 核心层另有内容级短窗口去重兑底)。
var entry *chatMsgEntry
if body.ClientMsgID != "" {
var replay bool
entry, replay = h.claimChatMsg(body.ClientMsgID)
if replay {
log.Printf("[webui] duplicate chat msg %s: waiting for first request result", body.ClientMsgID)
select {
case <-entry.done:
resp := entry.resp
if resp == nil {
writeJSON(w, http.StatusServiceUnavailable, map[string]string{"error": "agent unavailable"})
return
}
content, _ := resp.Payload["content"].(string)
reasoning, _ := resp.Payload["reasoning_content"].(string)
result := map[string]interface{}{"response": content, "deduplicated": true}
if reasoning != "" {
result["reasoning_content"] = reasoning
}
writeJSON(w, http.StatusOK, result)
case <-r.Context().Done():
return
}
return
}
}
// 来源编码:带设备身份时用 webui/{device_id}agent 经 injectSourceContext 可见来源);
// 无设备时保持 webui兼容旧调用。device_name 一并注入便于 agent 识别。
source := "webui"
if body.DeviceID != "" {
source = "webui/" + body.DeviceID
}
payload := map[string]interface{}{"content": body.Message}
if body.DeviceID != "" {
payload["device_id"] = body.DeviceID
payload["device_name"] = body.DeviceName
}
if body.ClientMsgID != "" {
payload["client_msg_id"] = body.ClientMsgID
}
// 带超时的上下文,防止 InjectInputSync 长时间阻塞 HTTP 请求。
// 注意ctx 派生自 r.Context(),客户端提前断开(前端 15s ackTimer abort
// 立即取消,不会真等满 300s300s 只约束"连接保持 + agent 排队/长生成"场景
// agent 串行处理后发消息的排队时间也计入60s 曾导致连发第 3 条必超时)。
ctx, cancel := context.WithTimeout(r.Context(), 300*time.Second)
defer cancel()
respCh := make(chan *agentIO.OutputEvent, 1)
go func() {
respCh <- h.sdk.InjectTextSync("webui", "webui", body.Message)
respCh <- h.sdk.InjectInputSync(source, "webui", "text", payload)
}()
var resp *agentIO.OutputEvent
select {
case resp = <-respCh:
case <-ctx.Done():
if entry != nil {
h.completeChatMsg(entry, nil)
}
writeJSON(w, http.StatusGatewayTimeout, map[string]string{"error": "agent timeout (60s)"})
return
}
if entry != nil {
h.completeChatMsg(entry, resp)
}
if resp == nil {
writeJSON(w, http.StatusServiceUnavailable, map[string]string{"error": "agent unavailable"})
return
@ -1251,15 +1429,48 @@ func (h *Handler) handleChatEvents(w http.ResponseWriter, r *http.Request) {
log.Printf("[SSE] handler started, subscribing to events")
// 解析 Last-Event-ID断线重连时客户端携带
lastEventID := r.Header.Get("Last-Event-ID")
if lastEventID != "" {
// 解析 Last-Event-ID断线重连时客户端携带,重放期间内遗漏的事件。
// 注意:本 handler 的 Last-Event-ID 重放仅为 GUI (cmd/gui/renderer/app.js) 服务。
// 浏览器原生 EventSource (webui/dashboard.html 使用) 由浏览器自动处理 Last-Event-ID 重连。
if lastEventID := r.Header.Get("Last-Event-ID"); lastEventID != "" {
log.Printf("[SSE] client reported Last-Event-ID: %s", lastEventID)
if h.sseEvents != nil {
replayed := h.sseEvents.After(lastEventID)
if len(replayed) == 0 {
log.Printf("[SSE] replay: nothing after id %s (id not in ring or already at tip)", lastEventID)
} else {
log.Printf("[SSE] replay: sending %d events after id %s", len(replayed), lastEventID)
for _, rec := range replayed {
fmt.Fprintf(w, "id: %s\nevent: %s\ndata: %s\n", rec.id, rec.eventType, string(rec.data))
flusher.Flush()
}
log.Printf("[SSE] replay complete, wrote %d events", len(replayed))
}
}
}
subTypes := []string{"agent_output", "reasoning", "agent_error", "tool_call", "stage", "agent_llm_chain", "terminal_output"}
// token 级流式增量事件:实时转发给浏览器做逐 token 渲染。
// 不进 sseEventRing —— 断线重连只重放聚合事件(最终真相),
// 避免重放 delta 与聚合内容重复追加。
var unsubs []func()
var seq int64
appendDeltaSub := func(evtType sdk.EventType) {
unsub := h.sdk.Subscribe(evtType, func(evt *sdk.Event) {
data, _ := json.Marshal(evt)
seq++
id := fmt.Sprintf("%d-%d", evt.Timestamp, seq)
select {
case writeCh <- fmt.Sprintf("id: %s\nevent: %s\ndata: %s\n", id, evt.Type, string(data)):
default:
log.Printf("[SSE] DROPPED %s (writeCh full, len=%d)", evt.Type, len(writeCh))
}
})
unsubs = append(unsubs, unsub)
}
appendDeltaSub(sdk.EventReasoningDelta)
appendDeltaSub(sdk.EventContentDelta)
for _, t := range subTypes {
t2 := t
unsub := h.sdk.Subscribe(sdk.EventType(t2), func(evt *sdk.Event) {
@ -1269,8 +1480,13 @@ func (h *Handler) handleChatEvents(w http.ResponseWriter, r *http.Request) {
}
data, _ := json.Marshal(evt)
seq++
id := fmt.Sprintf("%d-%d", evt.Timestamp, seq)
// 写入环状缓冲区,供断线重连重放
if h.sseEvents != nil {
h.sseEvents.Append(id, string(evt.Type), data)
}
select {
case writeCh <- fmt.Sprintf("id: %d-%d\nevent: %s\ndata: %s\n", evt.Timestamp, seq, evt.Type, string(data)):
case writeCh <- fmt.Sprintf("id: %s\nevent: %s\ndata: %s\n", id, evt.Type, string(data)):
if evt.Type == sdk.EventToolCall {
toolName, _ := evt.Payload["tool"].(string)
log.Printf("[SSE] wrote tool_call to writeCh: tool=%s", toolName)
@ -1493,8 +1709,8 @@ func (h *Handler) handleOpenAICompletions(w http.ResponseWriter, r *http.Request
return
}
// 带超时的上下文
ctx, cancel := context.WithTimeout(r.Context(), 60*time.Second)
// 带超时的上下文(同 handleChat客户端断开立即取消300s 约束长生成)
ctx, cancel := context.WithTimeout(r.Context(), 300*time.Second)
defer cancel()
respCh := make(chan *agentIO.OutputEvent, 1)
@ -1506,7 +1722,7 @@ func (h *Handler) handleOpenAICompletions(w http.ResponseWriter, r *http.Request
select {
case response = <-respCh:
case <-ctx.Done():
writeJSON(w, http.StatusGatewayTimeout, map[string]string{"error": "agent timeout (60s)"})
writeJSON(w, http.StatusGatewayTimeout, map[string]string{"error": "agent timeout (300s)"})
return
}
@ -1688,6 +1904,14 @@ func (h *Handler) handleDeviceGatewayProxy(w http.ResponseWriter, r *http.Reques
http.NotFound(w, r)
return
}
// 客户端鉴权模式:服务端不再提供授权接口(授权由设备端本地控制)。
// 拒绝旧的 /device/auth 调用,避免误导。
if strings.HasSuffix(r.URL.Path, "/device/auth") {
writeJSON(w, http.StatusGone, map[string]string{
"error": "device authorization moved to client-side; the server no longer stores authorization state",
})
return
}
addr := deviceGatewayAddr
if addr == "" {
addr = "127.0.0.1:9890"
@ -1893,6 +2117,23 @@ func (h *Handler) handlePluginByID(w http.ResponseWriter, r *http.Request) {
path := strings.TrimPrefix(r.URL.Path, "/api/v1/plugins/")
path = strings.TrimSuffix(path, "/")
// 插件名白名单:仅允许单段安全名称,阻断路径穿越/空名/嵌套路径
validPluginName := func(s string) bool {
if s == "" || len(s) > 128 {
return false
}
// 禁止路径分隔符、连续点(父目录穿越)、冒号、空格等危险字符
if strings.Contains(s, "..") || strings.ContainsAny(s, "/\\: \t\r\n\x00") {
return false
}
for _, c := range s {
if !(c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z' || c >= '0' && c <= '9' || c == '_' || c == '-' || c == '.') {
return false
}
}
return true
}
if path == "disabled" && r.Method == http.MethodGet {
if h.pluginMgr == nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "plugin manager not available"})
@ -1902,16 +2143,21 @@ func (h *Handler) handlePluginByID(w http.ResponseWriter, r *http.Request) {
return
}
if path == "reload" && r.Method == http.MethodPost {
if h.pluginMgr == nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "plugin registry not available"})
if path == "reload" {
if r.Method == http.MethodPost {
if h.pluginMgr == nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "plugin registry not available"})
return
}
if _, err := h.pluginMgr.ReloadPlugins(); err != nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": err.Error()})
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "reloaded"})
return
}
if _, err := h.pluginMgr.ReloadPlugins(); err != nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": err.Error()})
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "reloaded"})
// reload/disabled 是保留字,不允许 DELETE/GET 等其它操作误把其当作插件名
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
return
}
@ -1925,6 +2171,10 @@ func (h *Handler) handlePluginByID(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "plugin manager not available"})
return
}
if !validPluginName(name) {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid plugin name"})
return
}
if err := h.pluginMgr.DisablePlugin(name, "webui"); err != nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": err.Error()})
return
@ -1937,6 +2187,10 @@ func (h *Handler) handlePluginByID(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "plugin manager not available"})
return
}
if !validPluginName(name) {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid plugin name"})
return
}
if err := h.pluginMgr.EnablePlugin(name); err != nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": err.Error()})
return
@ -1945,6 +2199,14 @@ func (h *Handler) handlePluginByID(w http.ResponseWriter, r *http.Request) {
return
}
}
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
return
}
// 单段插件名路径GET 详情 / DELETE 卸载)
if !validPluginName(path) {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid plugin name"})
return
}
switch r.Method {

View File

@ -0,0 +1,195 @@
package webui
import (
"net/http"
"net/http/httptest"
"testing"
sdk "gitcode.com/JianFeeeee/HomeAgent/internal/sdk"
)
// mockPluginMgr 是 sdk.PluginManager 的最小实现,用于 handler 路由层测试。
type mockPluginMgr struct {
builtins map[string]bool
disabled []sdk.DisabledPluginInfo
isDisabled map[string]bool
removed []string
reloadN int
removeErr error
}
func (m *mockPluginMgr) ListLoadedPlugins() []string { return nil }
func (m *mockPluginMgr) ListDisabledPlugins() []sdk.DisabledPluginInfo {
return m.disabled
}
func (m *mockPluginMgr) IsPluginDisabled(name string) bool {
return m.isDisabled[name]
}
func (m *mockPluginMgr) IsBuiltinPlugin(name string) bool {
return m.builtins[name]
}
func (m *mockPluginMgr) DisablePlugin(name, by string) error {
if m.isDisabled == nil {
m.isDisabled = map[string]bool{}
}
m.isDisabled[name] = true
return nil
}
func (m *mockPluginMgr) EnablePlugin(name string) error {
delete(m.isDisabled, name)
return nil
}
func (m *mockPluginMgr) RemovePlugin(name string) error {
if m.removeErr != nil {
return m.removeErr
}
m.removed = append(m.removed, name)
return nil
}
func (m *mockPluginMgr) ReloadPlugins() (string, error) {
m.reloadN++
return "reloaded", nil
}
func (m *mockPluginMgr) ReloadOne(name string) error { return nil }
func (m *mockPluginMgr) PluginMetas() map[string]sdk.PluginMeta {
return nil
}
func (m *mockPluginMgr) PluginDir() string { return "" }
// newHandlerWithMock 构造带 mock PluginManager 的 Handler绕过 SDK 组装)。
func newHandlerWithMock(m *mockPluginMgr) *Handler {
h := NewHandler(nil)
h.pluginMgr = m
return h
}
func TestHandlePluginByID_DisabledList(t *testing.T) {
m := &mockPluginMgr{
disabled: []sdk.DisabledPluginInfo{{Name: "foo"}},
}
h := newHandlerWithMock(m)
req := httptest.NewRequest(http.MethodGet, "/api/v1/plugins/disabled", nil)
w := httptest.NewRecorder()
h.handlePluginByID(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d", w.Code)
}
if body := w.Body.String(); len(body) == 0 || !contains(body, "foo") {
t.Fatalf("expected disabled list with foo, got %s", body)
}
}
func contains(s, sub string) bool {
return len(s) >= len(sub) && (func() bool {
for i := 0; i+len(sub) <= len(s); i++ {
if s[i:i+len(sub)] == sub {
return true
}
}
return false
})()
}
func TestHandlePluginByID_ReloadPost(t *testing.T) {
m := &mockPluginMgr{}
h := newHandlerWithMock(m)
req := httptest.NewRequest(http.MethodPost, "/api/v1/plugins/reload", nil)
w := httptest.NewRecorder()
h.handlePluginByID(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
}
if m.reloadN != 1 {
t.Fatalf("expected ReloadPlugins called once, got %d", m.reloadN)
}
}
func TestHandlePluginByID_ReloadDeleteRejected(t *testing.T) {
// DELETE /plugins/reload 不允许把保留字当插件名反代成卸载
m := &mockPluginMgr{}
h := newHandlerWithMock(m)
req := httptest.NewRequest(http.MethodDelete, "/api/v1/plugins/reload", nil)
w := httptest.NewRecorder()
h.handlePluginByID(w, req)
if w.Code != http.StatusMethodNotAllowed {
t.Fatalf("expected 405, got %d", w.Code)
}
if len(m.removed) != 0 {
t.Fatalf("expected no plugin removed, got %v", m.removed)
}
}
func TestHandlePluginByID_PathTraversalRejected(t *testing.T) {
cases := []string{"../evil", "a/b", "a..b-ok-but-dots-only-check", "..%2Fetc"}
for _, name := range cases {
req := httptest.NewRequest(http.MethodDelete, "/api/v1/plugins/"+name, nil)
w := httptest.NewRecorder()
h := newHandlerWithMock(&mockPluginMgr{})
h.handlePluginByID(w, req)
// 含路径分隔符或以 .. 开头的名称必须被拒绝400/405绝不能反代到 pluginmgr
if w.Code != http.StatusBadRequest && w.Code != http.StatusMethodNotAllowed && w.Code != http.StatusNotFound {
t.Errorf("name %q: expected 4xx rejection, got %d", name, w.Code)
}
}
}
func TestHandlePluginByID_InvalidNamesRejected(t *testing.T) {
cases := []string{"has%20space", "has%3Acolon", "back%5Cslash"}
for _, name := range cases {
req := httptest.NewRequest(http.MethodGet, "/api/v1/plugins/"+name, nil)
w := httptest.NewRecorder()
h := newHandlerWithMock(&mockPluginMgr{})
h.handlePluginByID(w, req)
if w.Code != http.StatusBadRequest {
t.Errorf("name %q: expected 400, got %d", name, w.Code)
}
}
}
func TestHandlePluginByID_ValidNamePassesValidation(t *testing.T) {
// 合法插件名(含点/横线/下划线)不应被名称校验拦截;
// 这里 pluginmgr 未运行会得到 502 Bad Gateway但绝不应该是 400。
m := &mockPluginMgr{}
h := newHandlerWithMock(m)
for _, name := range []string{"my-plugin", "plugin_v2", "weather.so"} {
req := httptest.NewRequest(http.MethodGet, "/api/v1/plugins/"+name, nil)
w := httptest.NewRecorder()
h.handlePluginByID(w, req)
if w.Code == http.StatusBadRequest {
t.Errorf("valid name %q should pass validation, got 400", name)
}
}
}
func TestHandlePluginByID_DisableEnable(t *testing.T) {
m := &mockPluginMgr{builtins: map[string]bool{"webui": true}}
h := newHandlerWithMock(m)
req := httptest.NewRequest(http.MethodPost, "/api/v1/plugins/webui/disable", nil)
w := httptest.NewRecorder()
h.handlePluginByID(w, req)
if w.Code != http.StatusOK {
t.Fatalf("disable: expected 200, got %d: %s", w.Code, w.Body.String())
}
if !m.isDisabled["webui"] {
t.Fatal("webui should be disabled in mock")
}
req = httptest.NewRequest(http.MethodPost, "/api/v1/plugins/webui/enable", nil)
w = httptest.NewRecorder()
h.handlePluginByID(w, req)
if w.Code != http.StatusOK {
t.Fatalf("enable: expected 200, got %d: %s", w.Code, w.Body.String())
}
if m.isDisabled["webui"] {
t.Fatal("webui should be re-enabled")
}
}

View File

@ -0,0 +1,96 @@
package webui
import (
"encoding/json"
"reflect"
"testing"
)
func TestSSEEventRing_Append_Capacity(t *testing.T) {
r := newSSEEventRing(3)
data := json.RawMessage(`{"a":1}`)
for i := 0; i < 5; i++ {
r.Append("id-"+string(rune('0'+i)), "agent_output", data)
}
// cap=3只保留 id-2,id-3,id-4
if len(r.buf) != 3 {
t.Fatalf("expected 3, got %d", len(r.buf))
}
if r.buf[0].id != "id-2" {
t.Fatalf("expected id-2, got %s", r.buf[0].id)
}
}
func TestSSEEventRing_After_Mid(t *testing.T) {
r := newSSEEventRing(10)
data := json.RawMessage(`{"x":"y"}`)
r.Append("id-1", "reasoning", data)
r.Append("id-2", "agent_output", data)
r.Append("id-3", "tool_call", data)
after := r.After("id-2")
if len(after) != 1 || after[0].id != "id-3" {
t.Fatalf("expected [id-3], got %v", after)
}
after = r.After("id-3")
if len(after) != 0 {
t.Fatalf("expected 0, got %d", len(after))
}
}
func TestSSEEventRing_After_NotFound_ReturnsAll(t *testing.T) {
r := newSSEEventRing(3)
data := json.RawMessage(`{}`)
r.Append("id-1", "agent_output", data)
r.Append("id-2", "agent_output", data)
after := r.After("id-0")
if len(after) != 2 {
t.Fatalf("expected 2 (fallback to all), got %d", len(after))
}
}
func TestSSEEventRing_After_Last_Tip(t *testing.T) {
r := newSSEEventRing(5)
data := json.RawMessage(`{"a":1}`)
for i := 0; i < 5; i++ {
r.Append("id-"+string(rune('0'+i)), "agent_output", data)
}
after := r.After("id-4")
if len(after) != 0 {
t.Fatalf("expected 0 after tip, got %d", len(after))
}
}
func TestSSEEventRing_Concurrent(t *testing.T) {
r := newSSEEventRing(100)
done := make(chan struct{})
for i := 0; i < 8; i++ {
go func(n int) {
defer func() { done <- struct{}{} }()
data := json.RawMessage(`{}`)
for j := 0; j < 100; j++ {
r.Append("w"+string(rune('0'+n))+"/"+string(rune('0'+j)), "agent_output", data)
}
}(i)
}
for i := 0; i < 8; i++ {
go func() {
defer func() { done <- struct{}{} }()
for j := 0; j < 100; j++ {
_ = r.After("w0/0")
}
}()
}
for i := 0; i < 16; i++ {
<-done
}
}
func TestSSEEventRecord_Type(t *testing.T) {
var got interface{} = sseEventRecord{}
if reflect.TypeOf(got).Kind() != reflect.Struct {
t.Fatal("sseEventRecord should be a struct")
}
}

View File

@ -841,3 +841,141 @@ func TestHandleCompletionsEndToEnd(t *testing.T) {
}
})
}
// ===== client_msg_id 去重测试(防 GUI 断线重连消息重放)=====
func TestHandleChatClientMsgIDDedup(t *testing.T) {
iom := agentIO.NewIOManager()
memDB, err := memory.NewGraphDB(t.TempDir() + "/graph.db")
if err != nil {
t.Fatalf("NewGraphDB: %v", err)
}
defer memDB.Close()
pm := agentAPI.NewProviderManager()
pm.Register("echo", &echoProvider{name: "echo"})
agent := agentCore.New(agentCore.AgentConfig{
ID: "test",
SystemPrompt: "你是测试助手",
Provider: &echoProvider{name: "echo"},
ProviderManager: pm,
IO: iom,
Memory: memDB,
})
agent.Start()
defer agent.Stop()
sup := supervisor.New(&types.Config{
Daemon: types.DaemonConfig{
CheckInterval: time.Minute,
HeartbeatInterval: 30 * time.Second,
},
})
sup.Start()
defer sup.Shutdown()
s := testSDK(sdk.SDKConfig{
Supervisor: supervisor.NewSDKAdapter(sup),
IOManager: iom,
Config: sdk.NewConfig(&types.Config{}),
})
h := NewHandler(s)
t.Run("same_client_msg_id_replay_returns_cached_response", func(t *testing.T) {
body := `{"message":"你好","client_msg_id":"msg-abc-123"}`
req := httptest.NewRequest(http.MethodPost, "/api/v1/chat", strings.NewReader(body))
w := httptest.NewRecorder()
h.handleChat(w, req)
if w.Code != http.StatusOK {
t.Fatalf("first request: expected 200, got %d: %s", w.Code, w.Body.String())
}
var first map[string]interface{}
json.NewDecoder(w.Body).Decode(&first)
if first["response"] != "echo: 你好" {
t.Fatalf("expected echo response, got %v", first["response"])
}
// 同 ID 重放:应直接复用首次结果,不重复注入 agent
req2 := httptest.NewRequest(http.MethodPost, "/api/v1/chat", strings.NewReader(body))
w2 := httptest.NewRecorder()
h.handleChat(w2, req2)
if w2.Code != http.StatusOK {
t.Fatalf("replay: expected 200, got %d: %s", w2.Code, w2.Body.String())
}
var second map[string]interface{}
json.NewDecoder(w2.Body).Decode(&second)
if second["response"] != "echo: 你好" {
t.Fatalf("replay expected same response, got %v", second["response"])
}
if second["deduplicated"] != true {
t.Fatalf("replay expected deduplicated=true, got %v", second["deduplicated"])
}
})
t.Run("different_client_msg_id_processed_normally", func(t *testing.T) {
body := `{"message":"第二条","client_msg_id":"msg-def-456"}`
req := httptest.NewRequest(http.MethodPost, "/api/v1/chat", strings.NewReader(body))
w := httptest.NewRecorder()
h.handleChat(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
}
var resp map[string]interface{}
json.NewDecoder(w.Body).Decode(&resp)
if resp["deduplicated"] == true {
t.Fatal("new msg id should not be deduplicated")
}
})
t.Run("no_client_msg_id_backward_compatible", func(t *testing.T) {
body := `{"message":"旧客户端消息"}`
req := httptest.NewRequest(http.MethodPost, "/api/v1/chat", strings.NewReader(body))
w := httptest.NewRecorder()
h.handleChat(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
}
})
}
// ===== agent 核心层内容级去重测试 =====
func TestAgentDuplicateInputDedup(t *testing.T) {
iom := agentIO.NewIOManager()
memDB, err := memory.NewGraphDB(t.TempDir() + "/graph.db")
if err != nil {
t.Fatalf("NewGraphDB: %v", err)
}
defer memDB.Close()
pm := agentAPI.NewProviderManager()
pm.Register("echo", &echoProvider{name: "echo"})
agent := agentCore.New(agentCore.AgentConfig{
ID: "test",
SystemPrompt: "你是测试助手",
Provider: &echoProvider{name: "echo"},
ProviderManager: pm,
IO: iom,
Memory: memDB,
})
agent.Start()
defer agent.Stop()
// 直接验证 isDuplicateInput 行为
if agent.IsDuplicateInput("webui", "重复消息") {
t.Fatal("first input should not be duplicate")
}
if !agent.IsDuplicateInput("webui", "重复消息") {
t.Fatal("immediate same-content same-source should be duplicate")
}
if agent.IsDuplicateInput("webui", "不同消息") {
t.Fatal("different content should not be duplicate")
}
if agent.IsDuplicateInput("qq", "重复消息") {
t.Fatal("different source should not be duplicate")
}
}

View File

@ -18,4 +18,9 @@ const (
EventSystem = events.EventSystem
EventTerminalOutput = events.EventTerminalOutput
EventAll = events.EventAll
// 流式增量事件token 级):核心 process() 流式化后每收到一个增量块发布。
// 客户端可选订做真逐 token 渲染;聚合事件仍照常发布,旧订阅者不受影响。
EventReasoningDelta = events.EventReasoningDelta
EventContentDelta = events.EventContentDelta
)

View File

@ -23,6 +23,16 @@ type LLMMessage struct {
ReasoningContent string `json:"reasoning_content,omitempty"`
ToolCallID string `json:"tool_call_id,omitempty"`
ToolCalls []LLMToolCall `json:"tool_calls,omitempty"`
// Blocks 多模态内容块(与 Content 二选一;非空时优先)。
// 支持 text 与 image_url 两类,用于视觉模型看图(如 screensee 截屏描述)。
Blocks []LLMContentBlock `json:"blocks,omitempty"`
}
// LLMContentBlock 是多模态消息中的单个内容块。
type LLMContentBlock struct {
Type string `json:"type"` // "text" | "image_url"
Text string `json:"text,omitempty"`
ImageURL string `json:"image_url,omitempty"` // data URL 或 http(s) URL
}
// LLMToolCall 是中立的工具调用请求。

View File

@ -71,6 +71,18 @@ func (l *llmImpl) Chat(ctx context.Context, req *LLMCompletionRequest) (*LLMComp
ReasoningContent: m.ReasoningContent,
ToolCallID: m.ToolCallID,
}
// 多模态 Blockstext/image_url → agentAPI.ContentBlock
for _, b := range m.Blocks {
switch b.Type {
case "text":
msg.Blocks = append(msg.Blocks, agentAPI.ContentBlock{Type: "text", Text: b.Text})
case "image_url":
msg.Blocks = append(msg.Blocks, agentAPI.ContentBlock{
Type: "image_url",
ImageURL: &agentAPI.ImageURL{URL: b.ImageURL, Detail: "high"},
})
}
}
if len(m.ToolCalls) > 0 {
msg.ToolCalls = make([]agentAPI.ToolCall, len(m.ToolCalls))
for j, tc := range m.ToolCalls {

View File

@ -4,9 +4,9 @@ import (
"log"
"sync"
pubsdk "gitcode.com/JianFeeeee/homeagent-sdk/sdk"
agentIO "gitcode.com/JianFeeeee/HomeAgent/internal/agent/io"
"gitcode.com/JianFeeeee/HomeAgent/internal/events"
pubsdk "gitcode.com/JianFeeeee/homeagent-sdk/sdk"
)
// SDKVersion 是对外 SDK 版本号,与核心 meta.Version 保持一致。
@ -47,6 +47,7 @@ const (
StageScopeGlobal = pubsdk.StageScopeGlobal
StageScopeOwnTools = pubsdk.StageScopeOwnTools
)
type APIRegistrar = pubsdk.APIRegistrar
type OutputChannelRegistrar = pubsdk.OutputChannelRegistrar
type InputChannelRegistrar = pubsdk.InputChannelRegistrar
@ -68,6 +69,9 @@ type PluginManager interface {
ListLoadedPlugins() []string
ListDisabledPlugins() []DisabledPluginInfo
IsPluginDisabled(name string) bool
// IsBuiltinPlugin 判断插件是否为内置插件编译期工厂init() 自注册)。
// 内置插件只能禁用/启用,不能卸载。
IsBuiltinPlugin(name string) bool
DisablePlugin(name, by string) error
EnablePlugin(name string) error
// RemovePlugin 卸载插件先停止stop handlers + Stop再执行插件注册的
@ -253,13 +257,13 @@ func (s *PluginSDK) SelftestReset(scope string) error {
return nil
}
func (s *PluginSDK) Status() StatusAPI { return s.status }
func (s *PluginSDK) Status() StatusAPI { return s.status }
func (s *PluginSDK) Supervisor() SupervisorAPI { return s.supervisor }
func (s *PluginSDK) Adapter() AdapterAPI { return s.adapter }
func (s *PluginSDK) Tracker() TrackerAPI { return s.tracker }
func (s *PluginSDK) Config() ConfigAPI { return s.config }
func (s *PluginSDK) Tool() ToolAPI { return s.tool }
func (s *PluginSDK) Indexer() IndexerAPI { return s.indexer }
func (s *PluginSDK) Adapter() AdapterAPI { return s.adapter }
func (s *PluginSDK) Tracker() TrackerAPI { return s.tracker }
func (s *PluginSDK) Config() ConfigAPI { return s.config }
func (s *PluginSDK) Tool() ToolAPI { return s.tool }
func (s *PluginSDK) Indexer() IndexerAPI { return s.indexer }
func (s *PluginSDK) InjectInput(source, channel, eventType string, payload map[string]interface{}) {
if s.iom != nil {