Files
HomeAgent/internal/agent/io/channel.go
JianFeeeee d1959cbe80 feat(core): 注入行为的记忆/裁剪标志位落地 + jieba 词库内嵌 + Windows 改走 WSL
配套 SDK 提交:homeagent-sdk ba49dfd(公开 API 纯追加,无签名变更)。
本仓第三方的库镜像同步至该版本,以保证全新 clone 能编译。

## 1. 注入标志位(内核侧)

- 7 条注入路径(排队/中断/同步 × 纯文本/带媒体 + 旧 NoMem 变体)解析并转发
  no_memory / context_policy / cleaner_name;策略在入口**校验**,
  非法值报错而不是静默降级成 none(降级会让调用方以为自己声明的裁剪在生效)。
- 新增 validateContextPolicy(与 tool.register 同一套规则)与 pubSdkInjectOpts。
- input.register 不再手写字段白名单重建 ChannelDef,改为整体传递 + 补 ContextPolicy。
- io 层:applyInjectOpts 把标志位写进事件 payload,仅非零时写
  (零值与旧 payload 逐字节一致,事件订阅方与旧内核都不受影响)。
- ioAdapter / procCore / internal-sdk 别名补齐六个 *Opts 实现。

## 2. 修掉「输入无条件裁剪」这个真缺陷

eventloop 此前对**每条非中断输入**都调 `context.Prune(...)`:破坏性(低相关事件被
归档移出上下文)且无法从调用点看出是谁触发的。改为 pruneOnInput/pruneDeclared:

  优先级:注入点声明(payload.context_policy)> 通道声明(ChannelDef.ContextPolicy)
          > 默认**不裁剪**

查询向量仍取清洗后的内容;新增 cleanInputFor 解析清洗文本,优先级为
注入点声明的 cleaner(cleaner_name)> 按 source 查到的通道 cleaner > 原文,
名字查不到时**记日志再回退**(注入是 fire-and-forget,插件看不到错误,
至少要在内核日志留下「你声明的清洗没生效」的痕迹)。

## 3. jieba 词库内嵌(修「猜 GOMODCACHE → 静默失效」)

原 jiebaDictDir() 去猜 GOMODCACHE/GOPATH/~/go/pkg/mod,部署机上通常没有 Go 模块
缓存 → GetJieba() 返回 nil → 分词/关键词提取/NLP 依存解析(进而 doc→graph 三元组
抽取)/静态词向量 tokenizer **一律静默返回空列表**,只有一行日志。本机看起来正常
只因开发机与生产机重合、恰好有那份缓存。

现在词库随二进制分发:internal/memory/jiebadict/ 5 文件约 11.6MB + go:embed,
按**内容哈希**命名缓存目录落盘(词库升级不复用旧文件),已齐全则跳过写入。
模块缓存降为兜底。homed 体积 32MB。

顺带确认(并有测试佐证):gojieba 的 Tag() 不需要 pos_dict/ 目录——
cppjieba 的 PosTagger 从主词典每行的词性列取 tag。

## 4. homed 放弃 Windows 原生,改走 WSL2

插件体系依赖「继承的 fd」+「统一共享内存区的段内偏移解引用」,Windows 既无 fd
继承语义,其句柄模型也无法表达后者;强行适配等于再维护一套平台专属 ABI
(C ABI 时代三套 ABI 并存曾导致改写型插件在某平台静默失效)。

- cmd/homed/platform_{windows,other}.go:原生 Windows 启动即拒绝并打印 WSL2 指引。
- internal/plugin/proc/shmalloc_windows.go:allocShm 直接返回「请用 WSL2」,
  **不返回半可用的段**(与 shmalloc_other.go 同风格:未支持平台显式报错);
  procEnvForShm 返回 nil。顺手修掉两处长期编译错误
  (cryptorand→rand、h.evData→h.unified.evtData),使 GOOS=windows 至少能编译。
  注:homed 本就编不出 Windows——internal/memory 依赖 cgo-only 的 gojieba。
- deploy/packaging/installer.nsi:不再安装 homed.exe/initconfig.exe,改为携带
  **linux payload** 并调用新的 install-via-wsl.ps1;退出码 20/21 表示
  「需先装 WSL/发行版」,走指引而非报错。
- deploy/packaging/windows/install-via-wsl.ps1(新):检测 WSL → 引导安装 →
  确保 WSL2 → 送包进发行版 → 在 WSL 内按 Linux 方式安装。**复用 Linux 包与
  linux/setup.sh**,不另写一套安装逻辑;落点与 deb 布局统一
  (/usr/bin/homed + /usr/lib/homeagent/setup.sh)。
- deploy/packaging/linux/setup.sh:API Key 允许 HOMEAGENT_API_KEY 覆盖
  (否则安装器界面显示一份、config.db 里另一份 → 登录不上)。
- deploy/packaging/build.sh:windows 目标只构建 waiter + gui,并新增
  stage_linux_payload 把 Linux 包暂存给安装器;homed/initconfig 在 windows
  目标下明确拒绝。

## 5. 插件调用点统一写明意图

- webui 的 OpenAI 兼容端点(固定提示词模板)→ InjectTextSyncNoMemory。
- agentcli 的 5 处纯状态通知(已启动/超时/执行结束/进程退出/读取结束)→ NoMemory;
  **带输出**的 2 处(定时反馈、有新输出)刻意保留记忆并注明理由。
- timer 的定时提醒 → NoMemory(中断本来也隐含 NoMemory,这里是写明意图)。

## 6. 版本

meta.Version 仍为 1.2.0(main 是下一个未发布中版本);
SDKCompatibleVersion 1.1.0 → **1.2.0**(本内核已实现 SDK 1.2.0 全部新增方法)。

## 测试

- core:默认不裁剪(无声明/none/空)、通道 opt-in、注入点双向覆盖通道、
  nil context/io 安全、cleaner 优先级与未知名回退。
- io:零值 opts 与历史 payload 逐键相同;text/中断/媒体三类注入标志位都落到
  payload;旧方法仍生效。
- proc:validateContextPolicy 只接受 ""/none/prune,报错含位置与实际值;
  **跨进程** e2e——testdata 插件经 io.injectText 送出三个标志位,断言它们穿过 RPC
  到达内核。
- memory:模块缓存不可见时内嵌词库仍可用(分词与 POS 内容词均非空)、
  落盘幂等、内容哈希稳定。

验证:go build ./... / go vet ./... / go vet -tags onnxruntime ./...
      go test -short ./internal/memory/... ./internal/nlp/... ./internal/plugin/...
      ./internal/agent/{core,io}/... ./pkg/...
2026-09-11 20:31:50 +08:00

790 lines
24 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

package io
import (
"fmt"
"log"
"runtime/debug"
"sync"
"time"
pubsdk "gitcode.com/JianFeeeee/homeagent-sdk/sdk"
)
// ChannelDef 描述通道在记忆计算层的行为,与 ToolDef.NoMemory/Cleaner 语义一致。
type ChannelDef = pubsdk.ChannelDef
type DeviceType int
const (
DeviceInput DeviceType = 0
DeviceOutput DeviceType = 1
DeviceIO DeviceType = 2
)
// OutputCapability 定义通道支持的输出格式
type OutputCapability int
const (
CapText OutputCapability = 1 << iota // 文本
CapFile // 文件
CapImage // 图片
CapAudio // 音频
CapStructured // 结构化数据JSON/卡片)
)
func (c OutputCapability) Supports(cap OutputCapability) bool {
return c&cap != 0
}
func (c OutputCapability) String() string {
var flags []string
if c&CapText != 0 {
flags = append(flags, "text")
}
if c&CapFile != 0 {
flags = append(flags, "file")
}
if c&CapImage != 0 {
flags = append(flags, "image")
}
if c&CapAudio != 0 {
flags = append(flags, "audio")
}
if c&CapStructured != 0 {
flags = append(flags, "structured")
}
return fmt.Sprintf("%v", flags)
}
type Device interface {
Name() string
Type() DeviceType
Description() string
Tools() []ToolDef
Execute(tool string, args map[string]interface{}) (interface{}, error)
Start() error
Stop() error
OutputCapabilities() OutputCapability
ChannelDef() ChannelDef
}
type ToolHandler func(args map[string]interface{}) (interface{}, error)
type ToolDef struct {
Name string `json:"name"`
Description string `json:"description"`
Parameters map[string]interface{} `json:"parameters"`
Handler ToolHandler `json:"-"` // 可选插件工具的直接处理器Device 通过 Execute() 分发
}
type InputEvent struct {
RequestID string `json:"request_id"`
Source string `json:"source"`
Type string `json:"type"`
Payload map[string]interface{} `json:"payload"`
ResponseCh chan<- *OutputEvent `json:"-"`
OutputChannel string `json:"output_channel"` // 默认输出通道(不传则等于 Source
}
type OutputEvent struct {
RequestID string `json:"request_id"`
Target string `json:"target"`
Type string `json:"type"`
Payload map[string]interface{} `json:"payload"`
Done bool `json:"done,omitempty"`
OutputChannel string `json:"output_channel"` // 路由到此通道
}
type IOManager struct {
mu sync.RWMutex
devices map[string]Device
inputCh chan *InputEvent
interruptCh chan *InputEvent
outputCh chan *OutputEvent
nextReqID int64
inputChannels map[string]ChannelDef
// toolBlocks插件工具注入多模态内容块process.go 在下一条 tool message 时消费。
// 用 interface{}[] 避免 import api.ContentBlock 导致的循环依赖。
toolBlocksMu sync.Mutex
toolPendingBlocks []interface{}
}
func NewIOManager() *IOManager {
return &IOManager{
devices: make(map[string]Device),
inputCh: make(chan *InputEvent, 256),
interruptCh: make(chan *InputEvent, 64),
outputCh: make(chan *OutputEvent, 256),
inputChannels: make(map[string]ChannelDef),
}
}
func (m *IOManager) UnregisterDevice(name string) {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.devices, name)
}
func (m *IOManager) nextRequestID() string {
m.mu.Lock()
defer m.mu.Unlock()
m.nextReqID++
return fmt.Sprintf("req_%d_%d", time.Now().UnixNano(), m.nextReqID)
}
// AtomicSwapDevices 原子化替换全部 IO 设备
// 1. 新设备必须在调用前已完成 Start()
// 2. 调用后旧设备立即摘除,新请求走向新设备
// 3. 返回旧设备列表,由调用方负责 Stop()
func (m *IOManager) AtomicSwapDevices(newDevices map[string]Device) map[string]Device {
m.mu.Lock()
defer m.mu.Unlock()
oldDevices := m.devices
m.devices = newDevices
return oldDevices
}
func (m *IOManager) RegisterDevice(dev Device) error {
m.mu.Lock()
defer m.mu.Unlock()
if _, ok := m.devices[dev.Name()]; ok {
return fmt.Errorf("device %s already registered", dev.Name())
}
m.devices[dev.Name()] = dev
return nil
}
func (m *IOManager) GetDevice(name string) Device {
m.mu.RLock()
defer m.mu.RUnlock()
return m.devices[name]
}
func (m *IOManager) StartAll() error {
m.mu.RLock()
devices := make([]Device, 0, len(m.devices))
for _, dev := range m.devices {
devices = append(devices, dev)
}
m.mu.RUnlock()
for _, dev := range devices {
if err := dev.Start(); err != nil {
return fmt.Errorf("start device %s: %w", dev.Name(), err)
}
}
return nil
}
func (m *IOManager) StopAll() {
m.mu.RLock()
devices := make([]Device, 0, len(m.devices))
for _, dev := range m.devices {
devices = append(devices, dev)
}
m.mu.RUnlock()
for _, dev := range devices {
if err := dev.Stop(); err != nil {
log.Printf("[io] stop device %s error: %v", dev.Name(), err)
}
}
}
func (m *IOManager) InjectInput(source string, eventType string, payload map[string]interface{}) {
m.inputCh <- &InputEvent{
RequestID: m.nextRequestID(),
Source: source,
Type: eventType,
Payload: payload,
OutputChannel: source,
}
}
func (m *IOManager) InjectInputSync(source string, eventType string, payload map[string]interface{}) *OutputEvent {
ch := make(chan *OutputEvent, 1)
m.inputCh <- &InputEvent{
RequestID: m.nextRequestID(),
Source: source,
Type: eventType,
Payload: payload,
ResponseCh: ch,
OutputChannel: source,
}
return <-ch
}
// InjectInputTo 注入输入事件并指定输出通道
func (m *IOManager) InjectInputTo(source, outputChannel, eventType string, payload map[string]interface{}) {
m.inputCh <- &InputEvent{
RequestID: m.nextRequestID(),
Source: source,
Type: eventType,
Payload: payload,
OutputChannel: outputChannel,
}
}
// InjectInputSyncTo 注入输入事件(同步等待)并指定输出通道
func (m *IOManager) InjectInputSyncTo(source, outputChannel, eventType string, payload map[string]interface{}) *OutputEvent {
ch := make(chan *OutputEvent, 1)
m.inputCh <- &InputEvent{
RequestID: m.nextRequestID(),
Source: source,
Type: eventType,
Payload: payload,
ResponseCh: ch,
OutputChannel: outputChannel,
}
return <-ch
}
// InjectOptions 声明一次注入在记忆层与上下文层的表现。
//
// 零值 = 记入记忆 + 不裁剪上下文,与历史的三参数注入方法完全一致。
// 别名到公共 SDK 而非另建一套:内置插件与外部插件必须用同一套结构,
// 否则内核要认两种类型,而漏认会静默丢失标志位。
type InjectOptions = pubsdk.InjectOptions
// applyInjectOpts 把注入标志位写进事件 payload。
//
// 只在非零时写:零值与旧 payload 逐字节一致,事件订阅方与旧内核
// (不认识这两个键)都不会受影响。
//
// 为什么不把标志位当独立参数传到底eventloop 与各注入路径都按 payload 取字段
// no_memory 本来就是这么走的payload 是这里唯一已有的携带面。
func applyInjectOpts(payload map[string]interface{}, opts InjectOptions) {
if opts.NoMemory {
payload["no_memory"] = true
}
if opts.ContextPolicy != "" {
payload["context_policy"] = opts.ContextPolicy
}
if opts.CleanerName != "" {
payload["cleaner_name"] = opts.CleanerName
}
}
func (m *IOManager) InjectInputOpts(source, eventType string, payload map[string]interface{}, opts InjectOptions) {
applyInjectOpts(payload, opts)
m.InjectInput(source, eventType, payload)
}
func (m *IOManager) InjectInputToOpts(source, outputChannel, eventType string, payload map[string]interface{}, opts InjectOptions) {
applyInjectOpts(payload, opts)
m.InjectInputTo(source, outputChannel, eventType, payload)
}
func (m *IOManager) InjectInputSyncToOpts(source, outputChannel, eventType string, payload map[string]interface{}, opts InjectOptions) *OutputEvent {
applyInjectOpts(payload, opts)
return m.InjectInputSyncTo(source, outputChannel, eventType, payload)
}
func (m *IOManager) InjectInterruptOpts(source, channel string, payload map[string]interface{}, opts InjectOptions) {
applyInjectOpts(payload, opts)
m.InjectInterrupt(source, channel, payload)
}
func (m *IOManager) InjectText(source string, text string) {
m.InjectInput(source, "text", map[string]interface{}{
"content": text,
})
}
func (m *IOManager) InjectTextSync(source string, text string) *OutputEvent {
return m.InjectInputSync(source, "text", map[string]interface{}{
"content": text,
})
}
// InjectTextTo 注入文本输入并指定输出通道
func (m *IOManager) InjectTextTo(source, outputChannel, text string) {
m.InjectInputTo(source, outputChannel, "text", map[string]interface{}{
"content": text,
})
}
// InjectTextNoMemoryTo 注入文本输入(不产生记忆)并指定输出通道
func (m *IOManager) InjectTextNoMemoryTo(source, outputChannel, text string) {
m.InjectInputTo(source, outputChannel, "text", map[string]interface{}{
"content": text,
"no_memory": true,
})
}
// InjectTextSyncNoMemoryTo 注入文本输入(同步等待,不产生记忆)并指定输出通道
func (m *IOManager) InjectTextSyncNoMemoryTo(source, outputChannel, text string) *OutputEvent {
return m.InjectInputSyncTo(source, outputChannel, "text", map[string]interface{}{
"content": text,
"no_memory": true,
})
}
// InjectInterrupt 向中断通道发送输入
func (m *IOManager) InjectInterrupt(source, channel string, payload map[string]interface{}) {
if payload == nil {
payload = map[string]interface{}{}
}
evtType, _ := payload["type"].(string)
m.interruptCh <- &InputEvent{
RequestID: m.nextRequestID(),
Source: source,
Type: evtType,
Payload: payload,
OutputChannel: channel,
}
}
func (m *IOManager) InjectInterruptText(source, channel, text string) {
m.InjectInterruptTextOpts(source, channel, text, InjectOptions{})
}
// InjectInterruptTextOpts 注入中断文本,并声明本次注入的记忆/裁剪行为。
//
// 中断也允许声明 ContextPolicyPrune中断同样携带内容进入上下文。
func (m *IOManager) InjectInterruptTextOpts(source, channel, text string, opts InjectOptions) {
m.InjectInterruptOpts(source, channel, map[string]interface{}{
"type": "text",
"content": text,
}, opts)
}
// InjectTextOpts 注入排队文本,并声明本次注入的记忆/裁剪行为。
func (m *IOManager) InjectTextOpts(source, channel, text string, opts InjectOptions) {
m.InjectInputToOpts(source, channel, "text", map[string]interface{}{
"content": text,
}, opts)
}
// InjectTextSyncOpts 同步注入文本并声明记忆/裁剪行为。
func (m *IOManager) InjectTextSyncOpts(source, outputChannel, text string, opts InjectOptions) *OutputEvent {
return m.InjectInputSyncToOpts(source, outputChannel, "text", map[string]interface{}{
"content": text,
}, opts)
}
func (m *IOManager) InputInterruptChan() <-chan *InputEvent { return m.interruptCh }
// InjectTextSyncTo 注入文本输入(同步等待)并指定输出通道
func (m *IOManager) InjectTextSyncTo(source, outputChannel, text string) *OutputEvent {
return m.InjectInputSyncTo(source, outputChannel, "text", map[string]interface{}{
"content": text,
})
}
// ---- 带标志位的注入(记忆/裁剪行为由调用点声明)----
// InjectInputMediaOpts 注入带媒体块的输入,并声明记忆/裁剪行为。
func (m *IOManager) InjectInputMediaOpts(source, outputChannel, text string, blocks []pubsdk.ContentBlock, opts InjectOptions) {
m.InjectInputToOpts(source, outputChannel, "text", map[string]interface{}{
"content": text,
"media_blocks": blocks,
}, opts)
}
// InjectInputMediaSyncOpts 注入带媒体块的输入并同步等待回复,同时声明记忆/裁剪行为。
func (m *IOManager) InjectInputMediaSyncOpts(source, outputChannel, text string, blocks []pubsdk.ContentBlock, opts InjectOptions) *OutputEvent {
return m.InjectInputSyncToOpts(source, outputChannel, "text", map[string]interface{}{
"content": text,
"media_blocks": blocks,
}, opts)
}
// InjectInterruptMediaOpts 注入带媒体块的中断,并声明记忆/裁剪行为。
func (m *IOManager) InjectInterruptMediaOpts(source, channel, text string, blocks []pubsdk.ContentBlock, opts InjectOptions) {
m.InjectInterruptOpts(source, channel, map[string]interface{}{
"type": "text",
"content": text,
"media_blocks": blocks,
}, opts)
}
func (m *IOManager) EmitOutput(target string, outputType string, payload map[string]interface{}) {
m.outputCh <- &OutputEvent{
RequestID: "",
Target: target,
Type: outputType,
Payload: payload,
Done: true,
}
}
// EmitOutputTo 通过指定输出通道发送
func (m *IOManager) EmitOutputTo(target, outputChannel, outputType string, payload map[string]interface{}) {
m.outputCh <- &OutputEvent{
RequestID: "",
Target: target,
Type: outputType,
Payload: payload,
Done: true,
OutputChannel: outputChannel,
}
}
func (m *IOManager) EmitText(target string, text string) {
m.EmitOutput(target, "text", map[string]interface{}{
"content": text,
})
}
// EmitTextTo 通过指定输出通道发送文本
func (m *IOManager) EmitTextTo(target, outputChannel, text string) {
m.EmitOutputTo(target, outputChannel, "text", map[string]interface{}{
"content": text,
})
}
func (m *IOManager) InputChan() <-chan *InputEvent { return m.inputCh }
func (m *IOManager) OutputChan() <-chan *OutputEvent { return m.outputCh }
// RegisterInputChannel 注册输入通道的记忆行为
func (m *IOManager) RegisterInputChannel(name string, def ChannelDef) {
m.mu.Lock()
defer m.mu.Unlock()
m.inputChannels[name] = def
}
// UnregisterInputChannel 注销输入通道
func (m *IOManager) UnregisterInputChannel(name string) {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.inputChannels, name)
}
// GetInputChannelDef 查询输入通道的记忆行为定义
func (m *IOManager) GetInputChannelDef(name string) (ChannelDef, bool) {
m.mu.RLock()
defer m.mu.RUnlock()
def, ok := m.inputChannels[name]
return def, ok
}
func (m *IOManager) GetAllTools() []ToolDef {
m.mu.RLock()
defer m.mu.RUnlock()
var tools []ToolDef
for _, dev := range m.devices {
tools = append(tools, dev.Tools()...)
}
return tools
}
func (m *IOManager) ExecuteTool(name string, args map[string]interface{}) (ret interface{}, err error) {
m.mu.RLock()
type nameDevice struct {
name string
dev Device
}
var candidates []nameDevice
for _, dev := range m.devices {
for _, t := range dev.Tools() {
if t.Name == name {
candidates = append(candidates, nameDevice{name: dev.Name(), dev: dev})
break
}
}
}
m.mu.RUnlock()
if len(candidates) == 0 {
return nil, fmt.Errorf("tool %s not found", name)
}
defer func() {
if r := recover(); r != nil {
log.Printf("[io] tool %s execute panic: %v\n%s", name, r, debug.Stack())
err = fmt.Errorf("tool %s execute panic: %v", name, r)
}
}()
return candidates[0].dev.Execute(name, args)
}
func (m *IOManager) ListDevices() []Device {
m.mu.RLock()
defer m.mu.RUnlock()
list := make([]Device, 0, len(m.devices))
for _, d := range m.devices {
list = append(list, d)
}
return list
}
// ChannelInfo 返回 IOManager 中已注册的所有通道信息
type ChannelInfo struct {
Name string `json:"name"`
Type DeviceType `json:"type"`
Description string `json:"description"`
Tools []ToolDef `json:"tools"`
OutputCaps OutputCapability `json:"output_capabilities"`
}
func (m *IOManager) ListChannels() []ChannelInfo {
m.mu.RLock()
defer m.mu.RUnlock()
var list []ChannelInfo
for _, dev := range m.devices {
list = append(list, ChannelInfo{
Name: dev.Name(),
Type: dev.Type(),
Description: dev.Description(),
Tools: dev.Tools(),
OutputCaps: dev.OutputCapabilities(),
})
}
return list
}
func (m *IOManager) GetChannelCapabilities(channel string) OutputCapability {
m.mu.RLock()
defer m.mu.RUnlock()
if dev, ok := m.devices[channel]; ok {
return dev.OutputCapabilities()
}
return 0
}
// Microphone
type Microphone struct {
name string
sampleRate int
io *IOManager
}
func NewMicrophone(name string, sampleRate int, io *IOManager) *Microphone {
return &Microphone{name: name, sampleRate: sampleRate, io: io}
}
func (d *Microphone) Name() string { return d.name }
func (d *Microphone) Type() DeviceType { return DeviceInput }
func (d *Microphone) OutputCapabilities() OutputCapability { return 0 } // 纯输入
func (d *Microphone) Description() string {
return fmt.Sprintf("麦克风 (%s, %dHz)", d.name, d.sampleRate)
}
func (d *Microphone) Start() error { return nil }
func (d *Microphone) Stop() error { return nil }
func (d *Microphone) ChannelDef() ChannelDef { return ChannelDef{} }
func (d *Microphone) Tools() []ToolDef {
return []ToolDef{{
Name: d.name + "_capture",
Description: fmt.Sprintf("从 %s 录制音频", d.name),
Parameters: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"duration": map[string]interface{}{"type": "number", "description": "录制时长(秒)", "default": 3},
},
},
}}
}
func (d *Microphone) Execute(tool string, args map[string]interface{}) (interface{}, error) {
return map[string]interface{}{"device": d.name, "status": "recorded", "format": "wav", "sample_rate": d.sampleRate}, nil
}
// Speaker
type Speaker struct {
name string
io *IOManager
}
func NewSpeaker(name string, io *IOManager) *Speaker {
return &Speaker{name: name, io: io}
}
func (d *Speaker) Name() string { return d.name }
func (d *Speaker) Type() DeviceType { return DeviceOutput }
func (d *Speaker) OutputCapabilities() OutputCapability { return CapText | CapAudio }
func (d *Speaker) Description() string { return fmt.Sprintf("扬声器 (%s)", d.name) }
func (d *Speaker) Start() error { return nil }
func (d *Speaker) Stop() error { return nil }
func (d *Speaker) ChannelDef() ChannelDef { return ChannelDef{} }
func (d *Speaker) Tools() []ToolDef {
return []ToolDef{{
Name: d.name + "_speak",
Description: fmt.Sprintf("通过 %s 播放语音", d.name),
Parameters: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"text": map[string]interface{}{"type": "string", "description": "播放文本"},
},
"required": []string{"text"},
},
}}
}
func (d *Speaker) Execute(tool string, args map[string]interface{}) (interface{}, error) {
text, _ := args["text"].(string)
return map[string]interface{}{"device": d.name, "status": "playing", "text": text}, nil
}
// Camera
type Camera struct {
name string
io *IOManager
}
func NewCamera(name string, io *IOManager) *Camera {
return &Camera{name: name, io: io}
}
func (d *Camera) Name() string { return d.name }
func (d *Camera) Type() DeviceType { return DeviceInput }
func (d *Camera) OutputCapabilities() OutputCapability { return CapImage } // 可返回图片
func (d *Camera) Description() string { return fmt.Sprintf("摄像头 (%s)", d.name) }
func (d *Camera) Start() error { return nil }
func (d *Camera) Stop() error { return nil }
func (d *Camera) ChannelDef() ChannelDef { return ChannelDef{} }
func (d *Camera) Tools() []ToolDef {
return []ToolDef{
{
Name: d.name + "_capture",
Description: fmt.Sprintf("使用 %s 拍照", d.name),
Parameters: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"quality": map[string]interface{}{"type": "integer", "description": "质量1-100", "default": 90},
},
},
},
{
Name: d.name + "_stream",
Description: fmt.Sprintf("控制 %s 视频流", d.name),
Parameters: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"action": map[string]interface{}{"type": "string", "enum": []interface{}{"start", "stop"}},
},
"required": []string{"action"},
},
},
}
}
func (d *Camera) Execute(tool string, args map[string]interface{}) (interface{}, error) {
return map[string]interface{}{"device": d.name, "status": "captured"}, nil
}
// RobotArm
type RobotArm struct {
name string
io *IOManager
}
func NewRobotArm(name string, io *IOManager) *RobotArm {
return &RobotArm{name: name, io: io}
}
func (d *RobotArm) Name() string { return d.name }
func (d *RobotArm) Type() DeviceType { return DeviceIO }
func (d *RobotArm) OutputCapabilities() OutputCapability { return CapStructured }
func (d *RobotArm) Description() string { return fmt.Sprintf("机械臂 (%s)", d.name) }
func (d *RobotArm) Start() error { return nil }
func (d *RobotArm) Stop() error { return nil }
func (d *RobotArm) ChannelDef() ChannelDef { return ChannelDef{} }
func (d *RobotArm) Tools() []ToolDef {
return []ToolDef{
{
Name: d.name + "_move",
Description: fmt.Sprintf("移动 %s 到坐标", d.name),
Parameters: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"x": map[string]interface{}{"type": "number", "description": "X 轴"},
"y": map[string]interface{}{"type": "number", "description": "Y 轴"},
"z": map[string]interface{}{"type": "number", "description": "Z 轴"},
},
"required": []string{"x", "y", "z"},
},
},
{
Name: d.name + "_grip",
Description: fmt.Sprintf("控制 %s 夹爪", d.name),
Parameters: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"action": map[string]interface{}{"type": "string", "enum": []interface{}{"open", "close"}},
},
"required": []string{"action"},
},
},
}
}
func (d *RobotArm) Execute(tool string, args map[string]interface{}) (interface{}, error) {
return map[string]interface{}{"device": d.name, "tool": tool, "status": "executed"}, nil
}
// GPIODevice
type GPIODevice struct {
name string
pins []int
io *IOManager
}
func NewGPIODevice(name string, pins []int, io *IOManager) *GPIODevice {
return &GPIODevice{name: name, pins: pins, io: io}
}
func (d *GPIODevice) Name() string { return d.name }
func (d *GPIODevice) Type() DeviceType { return DeviceIO }
func (d *GPIODevice) OutputCapabilities() OutputCapability { return CapStructured }
func (d *GPIODevice) Description() string { return "GPIO 通用引脚" }
func (d *GPIODevice) Start() error { return nil }
func (d *GPIODevice) Stop() error { return nil }
func (d *GPIODevice) ChannelDef() ChannelDef { return ChannelDef{} }
func (d *GPIODevice) Tools() []ToolDef {
return []ToolDef{
{
Name: d.name + "_gpio_write",
Description: "设置引脚电平",
Parameters: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"pin": map[string]interface{}{"type": "integer"},
"value": map[string]interface{}{"type": "integer", "enum": []interface{}{0, 1}},
},
"required": []string{"pin", "value"},
},
},
{
Name: d.name + "_gpio_read",
Description: "读取引脚电平",
Parameters: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"pin": map[string]interface{}{"type": "integer"},
},
"required": []string{"pin"},
},
},
}
}
func (d *GPIODevice) Execute(tool string, args map[string]interface{}) (interface{}, error) {
return map[string]interface{}{"device": d.name, "tool": tool, "status": "ok"}, nil
}
// SetToolBlocks 插件工具调用时注入多模态内容块image_url/audio_url 等),
// 下一条 tool message 追加这些块到 content 数组OpenAI 多模态格式)。
func (m *IOManager) SetToolBlocks(blocks []interface{}) {
m.toolBlocksMu.Lock()
m.toolPendingBlocks = blocks
m.toolBlocksMu.Unlock()
}
// ConsumeToolBlocks 返回并清空 pending blocksprocess.go 在 append tool message 时调用。
func (m *IOManager) ConsumeToolBlocks() []interface{} {
m.toolBlocksMu.Lock()
blocks := m.toolPendingBlocks
m.toolPendingBlocks = nil
m.toolBlocksMu.Unlock()
return blocks
}