mirror of
https://gitcode.com/JianFeeeee/HomeAgent.git
synced 2026-09-22 09:58:06 +00:00
IO 抽象层增强:非文本输入支持 + waiter CLI 重写
PluginSDK: - 添加 InjectInput / InjectInputSync / InjectInterrupt 泛型接口 - 插件现在可注入 image/audio/file 等任意类型输入 Provider: - 添加 ContentBlock / ImageURL / AudioURL 类型 - Message 增加 Blocks 字段,Content 在非空 Blocks 时序列化为数组(多模态格式) Agent: - handleInput 新增 image/audio 类型分发 → processMediaInput - processMediaInput 将媒体数据附着到对话上下文,LLM 自主决策处理策略 - 新增内置工具:describe_image / transcribe_audio / ocr_image(pendingMedia 驱动) - 工具仅当有未处理媒体数据时注册,通过 Provider 直接调用多模态模型 Config: - 新增 InputProcessingConfig(image/audio 处理配置) - 含 fallback_provider / describe_prompt / ocr_enabled 等选项 Waiter CLI 重写: - 配置文件 ~/.config/homeagent/cli.yaml(自动发现 socket) - 原始终端行编辑 + 命令历史持久化 + 彩色输出 - 内置命令:/help /reconnect /connect /remote /local /prompt - 断线自动重连
This commit is contained in:
@ -40,6 +40,19 @@ func DefaultConfig() types.Config {
|
||||
{Name: "ollama", BaseURL: "http://localhost:11434", Model: "llama3", Adapter: "ollama", AdapterPath: "adapters/ollama.lua"},
|
||||
},
|
||||
},
|
||||
InputProcessing: types.InputProcessingConfig{
|
||||
Image: types.ImageProcessingConfig{
|
||||
FallbackProvider: "",
|
||||
FallbackModel: "",
|
||||
DescribePrompt: "请详细描述这张图片的内容",
|
||||
OCREnabled: true,
|
||||
},
|
||||
Audio: types.AudioProcessingConfig{
|
||||
FallbackProvider: "",
|
||||
FallbackModel: "",
|
||||
DescribePrompt: "请描述这段音频的内容",
|
||||
},
|
||||
},
|
||||
Defaults: types.AgentConfig{
|
||||
Image: "homeagent/agent-base:latest",
|
||||
LLMEndpoints: []string{"https://api.openai.com/v1"},
|
||||
|
||||
@ -55,6 +55,17 @@ llm:
|
||||
adapter: "ollama"
|
||||
adapter_path: "adapters/ollama.lua"
|
||||
|
||||
input_processing:
|
||||
image:
|
||||
fallback_provider: ""
|
||||
fallback_model: ""
|
||||
describe_prompt: "请详细描述这张图片的内容"
|
||||
ocr_enabled: true
|
||||
audio:
|
||||
fallback_provider: ""
|
||||
fallback_model: ""
|
||||
describe_prompt: "请描述这段音频的内容"
|
||||
|
||||
defaults:
|
||||
image: "homeagent/agent-base:latest"
|
||||
llm_endpoints:
|
||||
|
||||
Reference in New Issue
Block a user