Commit Graph

7 Commits

Author SHA1 Message Date
2e3d5b79ad fix(adapters): stop dropping non-streaming tool calls (agent loops died on turn 2)
Four adapters handled tool_calls in transform_stream_chunk but lost them in
transform_response, so any NON-streaming tool-using conversation broke on its
second request: the client received finish_reason:"tool_calls" with no
tool_calls payload, replayed an assistant message whose function
name/arguments were empty, and the upstream rejected the next turn with

    400 invalid tool_call function, function/name/arguments cannot be empty

The production audit trail shows 46 such failures on sensenova alone.

- sensenova.lua: forward message.tool_calls, decoding the arguments JSON string
  into an object as the unified shape expects.
- gemini.lua: collect functionCall parts from candidates[].content.parts. Also
  correct finish_reason, since Gemini reports "STOP" even when it emitted a
  function call and clients keyed on it treat that as a finished answer.
- ollama.lua: the field was initialized to an empty table and never filled;
  fill it and likewise correct done_reason "stop" -> "tool_calls".

trae is a different failure with the same symptom: trae-local-api's OpenAI
endpoint (/v1/chat/completions, src/server.js:353) never reads the request's
`tools` array — only its Anthropic endpoint does — so the relayed model is never
told the tool schema and instead PRINTS a <tool_call>{...}</tool_call> block into
content, leaving message.tool_calls null and finish_reason "stop". An OpenAI
client sees an ordinary completion and its agent loop ends mid-conversation.
trae.lua now recovers the structured call from that text, strips the block from
user-visible content, and corrects finish_reason. Both tag spellings
(<tool_call>/<toolcall>, the latter is what the same codebase's Anthropic prompt
asks for) and all three argument key names (arguments/params/input) are accepted.
This is a defensive fallback: fixing the upstream shim to honour `tools` remains
the real fix, since the model still guesses parameter names.

Tests: TestNonStreamToolCallsPreserved covers all ten OpenAI-shaped adapters,
TestGeminiNonStreamToolCalls and TestOllamaNonStreamToolCalls cover their native
shapes, TestTraeTextToolCallRecovery covers both tag spellings, prose around the
block, and asserts a plain text answer never gains tool_calls.

Verified end-to-end against mock upstreams reproducing each shape: a full
two-round agent loop (tool call -> tool result -> final answer) now completes for
both the structured and the text-emitted variants.
2026-08-31 10:23:35 +08:00
b2183df1e8 feat(adapter): move per-source error condensing into transform_error hooks
Every upstream formats errors differently, which is adapter territory:
the protocol gains an optional transform_error(status, body) hook and all
built-in adapters implement their own envelope parsing (zen free-pool
labels, anthropic/gemini/ollama/mistral shapes, sensenova quota notes,
agentrouter WAF pages). The core keeps a single uniform fallback: when no
hook yields a reason clients get "api error <status>: unknown error" and
the raw body goes to server logs only.
2026-08-24 19:17:36 +08:00
bb3af3bdb3 feat(gateway): pass through upstream finish_reason end-to-end
The gateway hardcoded "stop" on every terminating stream chunk, so
tool-call rounds reported finish_reason=stop and length caps were
invisible to clients. UnifiedChunk now carries finish_reason; adapters
emit it (with empty-string finish reasons like sensenova treated as
non-terminal), standardSSEChunk passes it through for un-adapted
upstreams, [DONE] no longer emits a duplicate reason-less done chunk,
and both streaming paths emit the real reason with "stop" as fallback.

Also vendor sensenova/agentrouter adapters into the repo: they were
WebUI-only uploads and a deploy sync silently removed them while live
AUTO-chain slots still referenced them.
2026-08-24 15:05:50 +08:00
3069cfce4e fix(gateway): pass through upstream token usage in streams for all adapters
The prior usage-passthrough fix only covered openai/opencode; the same
empty-choices+usage drop bug remained in the 5 sibling OpenAI-compatible
adapters, and non-OpenAI providers (anthropic/gemini/ollama) never surfaced
streaming usage at all.

- deepseek/github/groq/kimicode/mistral: preserve usage on empty-choices
  chunks and attach it to normal chunks (same pattern as openai.lua)
- anthropic: emit usage from message_start (prompt) and message_delta
  (completion); gateway merges split usage additively
- gemini: read usageMetadata in the stream path
- ollama: fix non-streaming key (usage -> token_usage, matches
  UnifiedResponse json tag) and read prompt_eval_count/eval_count;
  surface counts from the done stream chunk
- gateway: mergeUsage combines usage across chunks (non-zero fields win,
  total recomputed from prompt+completion) so split usage doesn't lose
  the prompt half; single-chunk case (OpenAI) preserved exactly
- usage-only chunks: done=false (no redundant terminal stop), matching
  the Go fallback standardSSEChunk
2026-08-18 23:24:01 +08:00
5d50b69153 fix: tool call anchor & wire format, streaming chunk passthrough, WebUI narrow-screen, docs bilingual 2026-08-08 11:26:58 +08:00
ccbcede581 feat: LuaJIT worker-pool VM, multimodal/disable-thinking passthrough, WebUI redesign, DeepSeek V4 2026-08-07 13:34:16 +08:00
f7f76e097d feat: ModelRouter — unified OpenAI-compatible multi-source LLM gateway
- Lua adapters per upstream (transform_request/response/stream_chunk, build_headers signing hooks)
- AUTO priority routing with per-model kind (chat/image), explicit source/model routing
- Per-source concurrency caps with queueing, exponential backoff, AUTO failover
- OpenAI-compatible API: chat completions, SSE streaming, image generations, models
- Gateway key auth, web UI for adapter/source management, runtime persistence
- e2e test running the real binary against mocked upstreams
2026-08-05 15:25:47 +08:00