mirror of
https://gitcode.com/JianFeeeee/HomeAgent.git
synced 2026-09-22 09:58:06 +00:00
feat(agent): token-level streaming in core process loop
Replace the blocking Chat() call in process() with
chatStreamWithFallback: ChatStream first, accumulate chunks, fall back
to non-stream Chat on connect failure or empty-stream failure.
Why: the non-streaming path blocked for the ENTIRE LLM generation (up
to the 180s HTTP timeout). Reasoning models thinking 60-120s plus AUTO
chain failover regularly exceeded it -> context canceled -> full turn
wasted. With streaming the first chunk arrives in ~1-3s and any
flowing token keeps the connection alive; total generation time is no
longer bounded by an overall timeout.
Compatibility (external behavior unchanged):
- process() signature/return values unchanged
- Aggregated events (EventReasoning / EventAgentLLMChain) still fire
once per turn with full text after stream completion - existing
plugin subscribers see identical payloads as before
- New incremental events EventReasoningDelta / EventContentDelta are
additive; old subscribers ignore unknown event types
- Tool execution loop, memory pipeline, stage pipeline untouched
Streaming details:
- Tool call fragments accumulated per OpenAI streaming convention:
id/name arrive on the first fragment, arguments as raw JSON string
shards across fragments; merged and parsed once at stream end
- normalizeStreamToolCalls keeps nameless argument shards (the
non-stream normalizer drops them); ToolCall gains RawArguments to
carry shard text
- Interrupt mid-stream returns partial content instead of discarding
the whole generation
Verified end-to-end against llmsproxy: plain chat streams correctly;
curl confirms tool-call shard wire format ({" + command" + :"date"}
-> {"command":"date"}); unit tests cover shard merging and
content/reasoning accumulation.
This commit is contained in:
@ -17,7 +17,15 @@ const (
|
||||
EventStage EventType = "stage"
|
||||
EventSystem EventType = "system"
|
||||
EventTerminalOutput EventType = "terminal_output"
|
||||
EventAll EventType = "*"
|
||||
|
||||
// 流式增量事件(LLM token 级):核心改为流式后每收到一个增量块发布。
|
||||
// 订阅者可选订;不认识的旧订阅者自然忽略(Bus 按 EventType 精确匹配分发)。
|
||||
// 聚合事件 EventReasoning / EventAgentLLMChain 仍照常在每轮结束时全文发布,
|
||||
// 插件体系行为不变。
|
||||
EventReasoningDelta EventType = "reasoning_delta"
|
||||
EventContentDelta EventType = "content_delta"
|
||||
|
||||
EventAll EventType = "*"
|
||||
)
|
||||
|
||||
type Event struct {
|
||||
|
||||
Reference in New Issue
Block a user