fix(codearts): 上游限流改为可声明重试(内核通用能力)+ 会话/连接两处真因

用户连续报「codeart 还是不可用 / empty stream」。把上游原始响应打出来逐条对齐,
三个症状同源,且都不是"模型坏了":

| 场景 | 上游实际返回 |
|---|---|
| 流式、短时间连续请求 | HTTP **200** + 只带 error 的帧 ⇒ 内核判 `empty stream` |
| 非流式、同一条件 | HTTP 400 `TM.00001041: 并发会话数已达上限(3个)` |
| 突发请求 | HTTP 429 `TPM limit … 500000 tokens per minute` |

实测:最后一个请求结束后约 **44~60 秒**释放;同一账号连续请求 3~4 次即占满。
参考实现(示例插件 llm-adapter.ts)的做法是**每 10s 轮询后重试 chat**(最多 30
分钟),所以用户那边"看起来没事"。

## 我们的改法(内核只做通用机制,厂商知识留在插件)
1. **内核新增声明式重试策略** `source.retry`(纯数据:状态码列表 + 响应体子串 +
   间隔 + 次数)。内核不认识任何厂商;命中即在同一网关请求内每 10s 重试一次。
   流式只在**握手阶段**重试(首帧尚未下发,不会重复输出),非流式整体重试。
2. **插件为 codearts 声明**:400/429 + 命中 `TM.00001041` / `TPM limit` /
   `并发会话数已达上限` → 10s × 最多 18 次(≈3 分钟)。
3. **适配器补两个会话细节**:`Connection: close`(keep-alive 空闲连接会被上游
   计入并发会话)与稳定 `Session-Id`(同一账号恒定,避免每次请求新建会话)。
4. 错误文案补可行动提示(并发会话上限 / TPM 限流,约 1 分钟释放)。

## 验证(生产)
- 连续 6 次**流式**:6/6 成功(撞上限时自动等 10s 再试,客户端只感觉慢):
  `+0s / +12s / +34s / +86s / +100s / +144s`
- 连续 5 次**非流式**:5/5 成功(`+0 / +11 / +20 / +28 / +38s`)
- 修复前:第 4 个请求起必然 TM.00001041 / empty stream。

> 注:`TM.00001005 无正常可用套餐` 属于账号权益问题(不重试、如实报出),
> 与限流要分开处理。
This commit is contained in:
root
2026-10-20 10:38:00 +08:00
parent 771b95a133
commit 6454b094bc
3 changed files with 201 additions and 5 deletions

View File

@ -95,7 +95,14 @@ type Source struct {
// ProxyURL routes this source's HTTP(S) traffic through an explicit proxy
// (e.g. http://127.0.0.1:7890). Empty = direct connection. Needed for
// upstreams behind geo/IP blocks that only respond through a proxy.
ProxyURL string `yaml:"proxy_url" json:"proxy_url,omitempty"`
ProxyURL string `yaml:"proxy_url" json:"proxy_url,omitempty"`
// Retry is an OPTIONAL upstream retry policy. It exists for upstreams that
// answer a business-level rejection with a status code the kernel would
// otherwise surface to the client — e.g. a per-account concurrency cap that
// clears by itself after tens of seconds. The kernel stays vendor-agnostic:
// the matcher is pure DATA (status list + response-body substrings), so no
// provider name ever appears in core.
Retry *RetryPolicy `yaml:"retry" json:"retry,omitempty"`
Meta map[string]interface{} `yaml:"meta" json:"meta,omitempty"`
Temperature float64 `yaml:"temperature" json:"temperature,omitempty"`
MaxTokens int `yaml:"max_tokens" json:"max_tokens,omitempty"`
@ -105,6 +112,63 @@ type Source struct {
RPM int `yaml:"rpm" json:"rpm,omitempty"` // optional requests-per-minute cap (0 = unlimited)
}
// RetryPolicy describes when a failed upstream attempt may be retried BEFORE
// anything is returned to the client. All conditions are data:
//
// OnStatus - retry only when the HTTP status is one of these
// OnBodyContains - ... AND the response body contains one of these substrings
// (empty = status alone decides)
// Delay - wait between attempts ("10s" etc.); parsed on demand
// MaxAttempts - TOTAL attempts including the first (1 or 0 = no retry)
//
// Scope: the retry happens inside one gateway request, so a stream is only
// retried before its first chunk reaches the client (the kernel already holds
// the first chunk back), and a non-stream call is retried as a whole.
type RetryPolicy struct {
OnStatus []int `yaml:"on_status,omitempty" json:"on_status,omitempty"`
OnBodyContains []string `yaml:"on_body_contains,omitempty" json:"on_body_contains,omitempty"`
Delay string `yaml:"delay,omitempty" json:"delay,omitempty"`
MaxAttempts int `yaml:"max_attempts,omitempty" json:"max_attempts,omitempty"`
}
// ShouldRetry reports whether one failed attempt matches this policy.
func (r *RetryPolicy) ShouldRetry(status int, body string) bool {
if r == nil || r.MaxAttempts <= 1 {
return false
}
statusOK := len(r.OnStatus) == 0
for _, s := range r.OnStatus {
if s == status {
statusOK = true
break
}
}
if !statusOK {
return false
}
if len(r.OnBodyContains) == 0 {
return true
}
for _, sub := range r.OnBodyContains {
if sub != "" && strings.Contains(body, sub) {
return true
}
}
return false
}
// DelayDuration is the configured pause between attempts (default 10s).
func (r *RetryPolicy) DelayDuration() time.Duration {
if r == nil || r.Delay == "" {
return 10 * time.Second
}
d, err := time.ParseDuration(r.Delay)
if err != nil || d < 0 {
return 10 * time.Second
}
return d
}
// Load reads and validates a config file. When the file does not exist yet a
// default config is generated at that path first (first-run bootstrap), so a
// fresh binary just works: `llmsproxy -config /path/to/config.yaml`.