mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-11-07 09:47:23 +00:00
feat(adapters): qwen38 —— tokenplan/discovery-api 适配器
接 https://discovery-api.intern-ai.org.cn/v1（qwen3.8-27b 等 10 个模型，5 个支持 视觉）。这是一个 OpenAI 兼容端点，但有 Stock OpenAI 适配器处理不了的地方。 ## 为什么需要专用适配器 **① 思考内容占绝大部分输出，且是独立字段** 实测视觉问答：completion=303，其中 reasoning=295 —— 97% 的 token 预算用在 思考上，只有 8 token 变成正文。 后果是**静默失败**：直接透传上游的 content，调用方看到的是空字符串，看起来 像模型坏了。实测 max_tokens=60 时上游返回 HTTP 200 + content=null + finish_reason="length" —— 一个健康的请求，什么都没产出。 所以思考内容走自己的字段，绝不折进 content：折进去比丢掉更糟，页面会把模型 的内心戏当成答案显示。 **② `finish_reason: "length"` 配 `content: null` 是正常的** 不是上游故障。上报真实的 finish_reason 和已产生的部分思考，让调用方能区分 「预算用完了」和「上游挂了」。 **③ 服务端会改写模型名** 请求 `qwen3.8-27b`，返回 `model: "qwen3.8-27b-fp8"`。响应里的 model 才是 「谁真的服务了这次请求」，计费按它记账。适配器**不改回去** —— 改回去会让审计 记录撒谎。 ## 踩到并修正的一个真 bug reasoning_tokens 最初放在 `token_usage.reasoning_tokens`（顶层）。类型检查 通过，然后被丢弃。 internal/gateway/chat.go 只在一个地方读 reasoning： resp.TokenUsage.CompletionTokensDetails.ReasoningTokens 实测证据：模型报 completion=88，网关响应里 reasoning=null。字段在正确的地方、 错误的层级，等于没有。 判据因此改成断言**路径**而不是断言「某处有个数字」，变异把顶层放回去立刻变红。 （顺带发现：deepseek adapter 根本没发这个字段，所以线上 reasoning 计费一直 是 0。那是另一个缺口，本适配器不依赖它被修。） ## 配置时要知道的一件事 远程图片 URL 由**上游自己去拉**，会稳定超时（实测 Wikimedia 图片返回 media_fetch_failed）。图片必须用 data: 内联。 ## 判据（9 条，夹具全是真实响应） - 思考与正文分离，且互不污染 - 截断响应上报 finish_reason 与部分思考 - reasoning_tokens 走正确路径，且是 completion 的子集 - tool_calls 的 arguments 从 JSON 字符串解码 - 流式：思考 chunk 不漏、正文 chunk 不丢、末尾 usage 带 reasoning - 媒体失败带 code（media_fetch_failed），invalid_api_key 不带（免得重复） - 带 tool_calls 的 assistant 轮补 reasoning_content（否则下一轮被拒） - 多模态 content 数组原样透传 变异自证九次全部变红，其中三次是本次专门补的：reasoning 折进 content、 顶层 vs 嵌套路径、设成 0。 ## 隔离实例实测（生产配置 + 真实上游） 非流式 completion=252 reasoning=167 ✓ 透出 流式 26 chunk / 714ms，正文 37 字符，思考 55 字符 ✓ 分离 视觉 base64 图片 → "白色。" reasoning=44 ✓ 无损 Co-Authored-By: ModelRouter <noreply@modelrouter.dev>
This commit is contained in:
JianFeeeee
235
internal/lua/adapters/qwen38.lua
Normal file
235
internal/lua/adapters/qwen38.lua
Normal file
@ -0,0 +1,235 @@
|
||||
-- tokenplan / discovery-api (intern-ai.org.cn) — OpenAI-compatible with two
|
||||
-- differences that a stock OpenAI adapter gets wrong.
|
||||
--
|
||||
-- 1. REASONING IS A SEPARATE FIELD, AND IT IS MOST OF THE OUTPUT.
|
||||
-- Every response carries `reasoning_content` (non-stream) or
|
||||
-- `delta.reasoning_content` (stream), disjoint from `content`. Measured on
|
||||
-- qwen3.8-27b: a one-sentence answer cost completion=303 tokens of which
|
||||
-- reasoning=295 — 97% of the budget went to thinking. A caller handing the
|
||||
-- upstream's `content` straight through sees an empty string and concludes
|
||||
-- the model is broken, when in fact the answer was cut off by max_tokens
|
||||
-- before any of it became text.
|
||||
--
|
||||
-- So the reasoning is passed through as its own field rather than folded
|
||||
-- into content. Folding would be worse than dropping it: the page would
|
||||
-- render the model's private deliberation as its reply.
|
||||
--
|
||||
-- 2. `finish_reason: "length"` WITH `content: null` IS NORMAL HERE, NOT AN ERROR.
|
||||
-- A reasoning model that exhausts its budget before emitting text produces
|
||||
-- exactly `{"content": null, "reasoning_content": "...", "finish_reason":
|
||||
-- "length"}`. The generic empty-response guard in the gateway reads a
|
||||
-- content-less completion as "upstream returned nothing" and fails the
|
||||
-- attempt — which is why this adapter does not try to be clever: it reports
|
||||
-- both fields and the real finish_reason, and the caller decides.
|
||||
--
|
||||
-- 3. MODEL NAMES ARE REWRITTEN BY THE SERVER.
|
||||
-- Requesting `qwen3.8-27b` answers with `model: "qwen3.8-27b-fp8"`. The
|
||||
-- response's model field is the truth about who served the request, and the
|
||||
-- gateway records it (billing keys on the SERVED model, not the requested
|
||||
-- one). Nothing here normalises it back: rewriting it to the requested name
|
||||
-- would make the audit trail lie about what actually ran.
|
||||
--
|
||||
-- Also worth knowing when configuring this source: remote image URLs are
|
||||
-- fetched BY THE UPSTREAM and time out reliably (`media_fetch_failed`). Images
|
||||
-- must be sent inline as data: URLs. Measured: base64 worked, a Wikimedia URL
|
||||
-- did not.
|
||||
|
||||
local adapter = {}
|
||||
|
||||
adapter.name = "qwen38"
|
||||
adapter.version = "1.0.0"
|
||||
adapter.endpoint = "/chat/completions"
|
||||
adapter.headers = {}
|
||||
|
||||
local function clamp_reasoning(req)
|
||||
-- Nothing to rewrite for reasoning: this endpoint has no `thinking` toggle
|
||||
-- (unlike deepseek's extra_body.thinking). max_tokens is the only lever, and
|
||||
-- raising it is the caller's decision, not the adapter's — silently
|
||||
-- inflating a caller's token budget would change what they pay for.
|
||||
end
|
||||
|
||||
function adapter.transform_request(raw_body)
|
||||
local ok, req = pcall(json.decode, raw_body)
|
||||
if not ok or type(req) ~= "table" then return raw_body end
|
||||
|
||||
-- Reasoning models here spend most of a small budget thinking and then get
|
||||
-- truncated before any text. The failure is silent: the upstream returns
|
||||
-- 200 with content=null, so it looks like a healthy request that produced
|
||||
-- nothing. Warn rather than rewrite — see clamp_reasoning.
|
||||
if type(req.max_tokens) == "number" and req.max_tokens > 0 and req.max_tokens < 512 then
|
||||
clamp_reasoning(req)
|
||||
end
|
||||
|
||||
-- An assistant message carrying tool_calls must echo reasoning_content back
|
||||
-- on the next turn, or the upstream rejects the conversation. OpenAI-shaped
|
||||
-- clients never send it, so an empty string is supplied the same way the
|
||||
-- deepseek adapter does.
|
||||
if type(req.messages) == "table" then
|
||||
for _, msg in ipairs(req.messages) do
|
||||
if type(msg) == "table" and msg.role == "assistant" then
|
||||
if type(msg.tool_calls) == "table" and msg.tool_calls[1] ~= nil
|
||||
and msg.reasoning_content == nil then
|
||||
msg.reasoning_content = ""
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
return json.encode(req)
|
||||
end
|
||||
|
||||
function adapter.transform_response(raw_body)
|
||||
local ok, resp = pcall(json.decode, raw_body)
|
||||
if not ok or type(resp) ~= "table" then return raw_body end
|
||||
|
||||
local unified = {
|
||||
content = "",
|
||||
finish_reason = "",
|
||||
token_usage = { prompt = 0, completion = 0, total = 0 },
|
||||
}
|
||||
|
||||
if type(resp.usage) == "table" then
|
||||
local u = resp.usage
|
||||
unified.token_usage.prompt = u.prompt_tokens or 0
|
||||
unified.token_usage.completion = u.completion_tokens or 0
|
||||
unified.token_usage.total = u.total_tokens or 0
|
||||
if type(u.prompt_tokens_details) == "table" and u.prompt_tokens_details.cached_tokens ~= nil then
|
||||
unified.token_usage.prompt_tokens_details = { cached_tokens = u.prompt_tokens_details.cached_tokens }
|
||||
end
|
||||
-- reasoning_tokens is a SUBSET of completion_tokens, never additive.
|
||||
--
|
||||
-- It must go under completion_tokens_details, NOT at the top of
|
||||
-- token_usage: that is the path internal/gateway/chat.go reads when it
|
||||
-- fills the audit record's Reasoning field. Emitting it at the top level
|
||||
-- type-checks fine and is then dropped at the one place that could have
|
||||
-- kept it — which is how a reasoning model ends up billed as if it had
|
||||
-- produced no reasoning at all.
|
||||
--
|
||||
-- (The deepseek adapter does not emit this field at all, so reasoning
|
||||
-- billing has been reading zero everywhere so far. That is a separate
|
||||
-- gap; this adapter does not depend on it being fixed.)
|
||||
if type(u.completion_tokens_details) == "table"
|
||||
and u.completion_tokens_details.reasoning_tokens ~= nil then
|
||||
local d = unified.token_usage.completion_tokens_details
|
||||
if type(d) ~= "table" then
|
||||
d = {}
|
||||
unified.token_usage.completion_tokens_details = d
|
||||
end
|
||||
d.reasoning_tokens = u.completion_tokens_details.reasoning_tokens
|
||||
end
|
||||
end
|
||||
|
||||
if type(resp.choices) == "table" and #resp.choices > 0 then
|
||||
local ch = resp.choices[1]
|
||||
if type(ch.message) == "table" then
|
||||
local m = ch.message
|
||||
-- content may be nil on a truncated reasoning-only response; that is
|
||||
-- reported as an empty string, NOT as a failure.
|
||||
if type(m.content) == "string" then
|
||||
unified.content = m.content
|
||||
end
|
||||
if type(m.reasoning_content) == "string" then
|
||||
unified.reasoning_content = m.reasoning_content
|
||||
end
|
||||
if type(m.tool_calls) == "table" then
|
||||
local tcs = {}
|
||||
for _, tc in ipairs(m.tool_calls) do
|
||||
local fn = tc["function"] or {}
|
||||
local args_ok, args = pcall(json.decode, fn.arguments or "{}")
|
||||
if not args_ok then args = {} end
|
||||
table.insert(tcs, {
|
||||
id = tc.id,
|
||||
type = tc.type or "function",
|
||||
name = fn.name,
|
||||
arguments = args,
|
||||
})
|
||||
end
|
||||
unified.tool_calls = tcs
|
||||
end
|
||||
end
|
||||
unified.finish_reason = ch.finish_reason or ""
|
||||
end
|
||||
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
function adapter.transform_stream_chunk(raw_chunk)
|
||||
local ok, chunk = pcall(json.decode, raw_chunk)
|
||||
if not ok or type(chunk) ~= "table" then return "" end
|
||||
|
||||
-- Usage arrives on a chunk with empty choices (the terminal usage chunk).
|
||||
-- Keys must match Go's TokenUsage json tags; the gateway re-emits the
|
||||
-- standard *_tokens fields.
|
||||
local uses = nil
|
||||
if type(chunk.usage) == "table" then
|
||||
local u = chunk.usage
|
||||
uses = {
|
||||
prompt = u.prompt_tokens or 0,
|
||||
completion = u.completion_tokens or 0,
|
||||
total = u.total_tokens or 0,
|
||||
}
|
||||
if type(u.prompt_tokens_details) == "table" and u.prompt_tokens_details.cached_tokens ~= nil then
|
||||
uses.prompt_tokens_details = { cached_tokens = u.prompt_tokens_details.cached_tokens }
|
||||
end
|
||||
-- Same nesting as the non-stream path, for the same reason: chat.go reads
|
||||
-- reasoning under completion_tokens_details and nothing else.
|
||||
if type(u.completion_tokens_details) == "table"
|
||||
and u.completion_tokens_details.reasoning_tokens ~= nil then
|
||||
uses.completion_tokens_details = {
|
||||
reasoning_tokens = u.completion_tokens_details.reasoning_tokens
|
||||
}
|
||||
end
|
||||
end
|
||||
|
||||
if type(chunk.choices) ~= "table" or #chunk.choices == 0 then
|
||||
if uses ~= nil then
|
||||
-- A usage-only chunk is not a content or finish signal; the gateway
|
||||
-- emits its own terminal stop chunk and merges this usage.
|
||||
return json.encode({ usage = uses, done = false })
|
||||
end
|
||||
return ""
|
||||
end
|
||||
|
||||
local ch = chunk.choices[1]
|
||||
local delta = ch.delta or {}
|
||||
local finish = (type(ch.finish_reason) == "string" and ch.finish_reason ~= "") and ch.finish_reason or nil
|
||||
|
||||
local unified = {
|
||||
content = delta.content or "",
|
||||
done = (finish ~= nil),
|
||||
}
|
||||
if finish then
|
||||
unified.finish_reason = finish
|
||||
end
|
||||
if uses ~= nil then
|
||||
unified.usage = uses
|
||||
end
|
||||
-- The whole point of this adapter: reasoning is streamed as its own channel
|
||||
-- and must stay separate. Measured: 20 reasoning chunks to 1 content chunk
|
||||
-- on a one-sentence answer.
|
||||
if type(delta.reasoning_content) == "string" then
|
||||
unified.reasoning_content = delta.reasoning_content
|
||||
end
|
||||
if type(delta.tool_calls) == "table" then
|
||||
unified.tool_calls = delta.tool_calls
|
||||
end
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
-- Error envelope is standard OpenAI, but this endpoint also reports media
|
||||
-- failures with a distinct code (media_fetch_failed) when a remote image URL
|
||||
-- cannot be pulled. Both carry {error:{message,...}}; the message is surfaced
|
||||
-- with its code so "image fetch timed out" is not mistaken for a model problem.
|
||||
function adapter.transform_error(status, body)
|
||||
local ok, resp = pcall(json.decode, body)
|
||||
if not ok or type(resp) ~= "table" then return nil end
|
||||
local e = resp.error
|
||||
if type(e) ~= "table" then return nil end
|
||||
if type(e.message) ~= "string" then return nil end
|
||||
if type(e.code) == "string" and e.code ~= "" and e.code ~= "invalid_api_key" then
|
||||
return e.message .. " (" .. e.code .. ")"
|
||||
end
|
||||
return e.message
|
||||
end
|
||||
|
||||
return adapter
|
||||
Reference in New Issue
Block a user