mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-11-07 09:47:23 +00:00
用户判据:「priority 直接移除」「即使带 key 也没有 /v1/responses」
「channel 插件的问题还没完全解决」「生图和聊天合并到一个导航项」
## 一、删除 priority(它从未参与跨源调度)
`config.Model.Priority` 号称"高者优先参与 AUTO 选源",**那是错的**:
AUTO 链由 `config.ModelScope.Tier` 驱动(scheduler.Chain),priority 只被
`bestChatModel/bestImageModel` 用于"本源内部该发哪个模型",与跨源调度无关。
把它当调度依据会让人以为拖个数字就能改顺序,改完毫无效果。
- 删 `config.Model.Priority` 字段与 `bestChatModel/bestImageModel` 的优先级比较
- 删 `seedAuto()`:**AUTO 链为空时网关应直接 503**,不能凭空播种用户没配过的档位
- 删 UI `buildLanes` 的 discovery 回退(按源 priority 自动铺开)与 sourceRows
- 清掉 config.yaml 里 140 行 priority
## 二、一通道 = 一个适配器 = 一个源(channels 插件)
旧设计「一账号一源」(`channels-<平台>-<12位hex>`) 有三个结构性后果:
1. 三个账号 ⇒ 三个源、三份同名模型,用户看到三行无法区分的东西
2. 谁先谁后**取决于注册顺序,不是余额**(rankAccounts 只用于展示,从未接进同步路径)
3. 换号要改源名 ⇒ 用户书签里的 `源:模型` 全部失效
新设计:
- 源名只跟通道绑定:`channels-<平台>`,换号 = 同名 add_source 覆盖 api_key
(内核 AddSource 是 UpsertSourceInYAML + 内存替换),对客户端完全透明
- `pickAccountForSource()`:凭据可用 → 余额分档(先快作废后永久)→
跳过已标记失效的号。没有 balance 声明的平台不据此排除(不因"不会查"判死)
- `request_end` 钩子:真实请求失败且是**终态**错误(需充值/封禁/凭据永久失效)
时给当前账号打 termErr,下次选号跳过;成功则清标记。
⚠️ 只认终态 —— 429/超时/抖动都**不**标记(那会把好号一个个废掉)
- `refresh_creds` 每 30 分钟巡检后重新选号(兜底心跳,纯本地判定)
- 迁移:同步时自动清掉旧版按账号拆的源(按已知平台前缀匹配,不用贪婪 match)
- 顺带修:`refresh_creds` 漏认 `refresh_fn` ⇒ codearts 的 DPoP 续期永不触发
## 三、discover 支持用已存凭据(真实缺口)
刷新已有源的模型目录,此前必须让运维**再粘一次明文 key** —— 而库里是
`enc:v1:` 密文,WebUI 也不回显明文。结果就是"想刷新 → 得重新申请 key",
很多人干脆不刷,源的目录越放越旧(实例:某源 3 个生图模型从未登记)。
- `{"name":"oin"}` ⇒ 从已存源取 base_url + 解密凭据 + adapter
- 安全边界:已要求 admin;只能探测**本网关已配置的**源(不能拿任意 URL+key
做探测,那就成了 SSRF/密钥外带);响应不回显 key
## 四、image_models 显式声明(补元数据缺失)
`discoveredModelKind` 只看上游 modality 字段,并刻意**不从模型名推断**
(错误的 kind 会毒死 AUTO 链)。但有些网关的 /v1/models 只返回
`{id,object,created,owned_by}` —— 零 modality,于是生图模型全落成 chat,
生图端点永远 `no image source configured`。
不解这个两难:默认仍完全按元数据(不猜),但允许 `image_models` **显式声明**。
写错的 id 明确报 400,不静默造一个不存在的模型。
## 五、UI:对话与生图合并为一个导航项
两者本是同一件事的两种输入(都选模型、都发请求、都读结果),只是 kind 与端点
不同。合并只发生在**导航层**:模型下拉仍按 kind 分组/禁选(把图片模型选进 chat
会请求到错误端点)。两个面板同时存在于 DOM,切换只改 display —— 来回切不会丢
已输入的文字、对话历史或已生成的图。
## 测试
- `TestUI_ImageModeIsReachableFromChatTab`:改为钉"合并后仍可达"(分段控件、
切换函数、面板容器、端点),并断言**旧 tab 三件套必须消失**(否则 goTab 会找
一个不存在的面板)
- 测试夹具显式建 AUTO 链(内核不再 seedAuto):默认一源一 tier 倒序
(TestChatAUTO 验优先级);TestChatAutoChainTierFailover 自己覆盖成同 tier
(验 tier 内轮转)—— 两者验不同机制,不能共用一个默认
- core 的一批断言从"按账号 id 找源"改为"按平台名找源"
gateway/config/provider 全绿;core 的 channels 用例全绿。
TestRealMutants 需工作树干净后才跑(其自带的变异残留防护)。
BREAKING CHANGE: config.Model.Priority 已删除。旧的 priority 字段会被 YAML
解码静默忽略(内核不校验未知字段),因此既有 config.yaml 仍可加载,但该字段
不再有任何作用 —— 调度顺序请通过 AUTO 链的 tier 表达。
(cherry picked from commit 25970a2135)
255 lines
8.6 KiB
Go
255 lines
8.6 KiB
Go
package gateway
|
|
|
|
import (
|
|
"encoding/json"
|
|
"fmt"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"os"
|
|
"strings"
|
|
"testing"
|
|
|
|
"llmsproxy/internal/config"
|
|
)
|
|
|
|
// These cover the two write-path defects found while adding a model to a live
|
|
// source by hand:
|
|
//
|
|
// 1. PUT /api/sources/{name} was documented in the API index but not
|
|
// implemented, so it answered 405 while POST (which does the same upsert)
|
|
// worked. A caller following the documented contract could not edit a
|
|
// source at all.
|
|
// 2. POST is an upsert that replaces the whole source, so a partial edit that
|
|
// did not carry api_key persisted an empty/placeholder credential. The
|
|
// source kept its name and models and answered 401 on the next call, well
|
|
// after the writing script had exited 0.
|
|
//
|
|
// The second one is the dangerous class: a write that reports success and
|
|
// breaks the source later.
|
|
|
|
// sourceAPIKey reads the stored key back the way an operator would, i.e. what
|
|
// the routes actually use, not what the file happens to contain.
|
|
func sourceAPIKey(t *testing.T, g *Gateway, name string) string {
|
|
t.Helper()
|
|
for _, s := range g.core.Sources() {
|
|
if s.Name == name {
|
|
return s.APIKey
|
|
}
|
|
}
|
|
t.Fatalf("source %q not found", name)
|
|
return ""
|
|
}
|
|
|
|
func seededGateway(t *testing.T) (*Gateway, string) {
|
|
t.Helper()
|
|
up := mockUpstream()
|
|
t.Cleanup(up.Close)
|
|
g := newTestGateway(t, config.Source{
|
|
Name: "seeded",
|
|
BaseURL: up.URL,
|
|
Adapter: "openai",
|
|
APIKey: "sk-real-credential",
|
|
Models: []config.Model{{ID: "m-one", Kind: "chat"}},
|
|
})
|
|
return g, up.URL
|
|
}
|
|
|
|
func TestSourcePutUpdatesSource(t *testing.T) {
|
|
g, up := seededGateway(t)
|
|
|
|
// A single-field edit: change one model's priority, name the source only in
|
|
// the path. This is the shape the API index documents for PUT.
|
|
rr := doReq(t, g, "PUT", "/api/sources/seeded", fmt.Sprintf(
|
|
`{"base_url":%q,"adapter":"openai","api_key":"sk-real-credential","models":[{"id":"m-one","priority":42,"kind":"chat"}]}`, up))
|
|
if rr.Code != http.StatusOK {
|
|
t.Fatalf("PUT status=%d body=%s", rr.Code, rr.Body.String())
|
|
}
|
|
|
|
var found bool
|
|
for _, s := range g.core.Sources() {
|
|
if s.Name != "seeded" {
|
|
continue
|
|
}
|
|
found = true
|
|
}
|
|
if !found {
|
|
t.Fatal("source vanished after PUT")
|
|
}
|
|
// And it must be routable, not merely stored.
|
|
if rr := doReq(t, g, "GET", "/v1/models", ""); rr.Code != http.StatusOK ||
|
|
!strings.Contains(rr.Body.String(), "m-one") {
|
|
t.Fatalf("model not routable after PUT: %d %s", rr.Code, rr.Body.String())
|
|
}
|
|
}
|
|
|
|
func TestSourcePutRejectsNameMismatch(t *testing.T) {
|
|
g, _ := seededGateway(t)
|
|
// Body names a different source than the path. Silently preferring either
|
|
// one would let a client edit a source it did not name.
|
|
rr := doReq(t, g, "PUT", "/api/sources/seeded", `{"name":"other","base_url":"http://x","models":[{"id":"m"}]}`)
|
|
if rr.Code != http.StatusBadRequest {
|
|
t.Fatalf("name mismatch accepted: status=%d body=%s", rr.Code, rr.Body.String())
|
|
}
|
|
if sourceAPIKey(t, g, "seeded") != "sk-real-credential" {
|
|
t.Fatal("rejected PUT still mutated the source")
|
|
}
|
|
}
|
|
|
|
func TestSourcePutRequiresNameInPath(t *testing.T) {
|
|
g, _ := seededGateway(t)
|
|
rr := doReq(t, g, "PUT", "/api/sources", `{"base_url":"http://x","models":[{"id":"m"}]}`)
|
|
if rr.Code != http.StatusBadRequest {
|
|
t.Fatalf("PUT without a name in the path: status=%d body=%s", rr.Code, rr.Body.String())
|
|
}
|
|
}
|
|
|
|
func TestSourceUpsertKeepsAPIKey(t *testing.T) {
|
|
for _, tc := range []struct {
|
|
name string
|
|
sentKey string
|
|
}{
|
|
{"explicit mask", keepExistingAPIKey},
|
|
{"omitted key", ""},
|
|
} {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
g, up := seededGateway(t)
|
|
rr := doReq(t, g, "PUT", "/api/sources/seeded", fmt.Sprintf(
|
|
`{"base_url":%q,"adapter":"openai","api_key":%q,"models":[{"id":"m-one"},{"id":"m-two"}]}`,
|
|
up, tc.sentKey))
|
|
if rr.Code != http.StatusOK {
|
|
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
|
}
|
|
if got := sourceAPIKey(t, g, "seeded"); got != "sk-real-credential" {
|
|
t.Fatalf("api_key was clobbered: got %q, want the pre-existing credential", got)
|
|
}
|
|
// The failure this guards is a source that looks configured but
|
|
// cannot authenticate, so assert the route still works.
|
|
if rr := doReq(t, g, "GET", "/v1/models", ""); rr.Code != http.StatusOK {
|
|
t.Fatalf("source unusable after edit: %d %s", rr.Code, rr.Body.String())
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestSourceUpsertStillWritesNewKey(t *testing.T) {
|
|
// The keep-existing path must not swallow a deliberate rotation.
|
|
g, up := seededGateway(t)
|
|
rr := doReq(t, g, "PUT", "/api/sources/seeded", fmt.Sprintf(
|
|
`{"base_url":%q,"adapter":"openai","api_key":"sk-rotated","models":[{"id":"m-one"}]}`, up))
|
|
if rr.Code != http.StatusOK {
|
|
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
|
}
|
|
if got := sourceAPIKey(t, g, "seeded"); got != "sk-rotated" {
|
|
t.Fatalf("rotation ignored: got %q", got)
|
|
}
|
|
}
|
|
|
|
func TestSourceUpsertNewSourceMayHaveNoKey(t *testing.T) {
|
|
// Empty is a real value for a credential-less source (local upstreams), so
|
|
// inherit-on-empty must not invent one for a source that does not exist yet.
|
|
up := mockUpstream()
|
|
defer up.Close()
|
|
g := newTestGateway(t)
|
|
rr := doReq(t, g, "POST", "/api/sources", fmt.Sprintf(
|
|
`{"name":"noauth","base_url":%q,"adapter":"openai","models":[{"id":"local-m"}]}`, up.URL))
|
|
if rr.Code != http.StatusOK {
|
|
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
|
}
|
|
if got := sourceAPIKey(t, g, "noauth"); got != "" {
|
|
t.Fatalf("unexpected key on a new credential-less source: %q", got)
|
|
}
|
|
}
|
|
|
|
func TestSourceModelIDsIsAdditive(t *testing.T) {
|
|
// "Add one model" is the edit that motivated the defect: it needs no read
|
|
// of the existing list, so it must not require one.
|
|
g, up := seededGateway(t)
|
|
rr := doReq(t, g, "PUT", "/api/sources/seeded", fmt.Sprintf(
|
|
`{"base_url":%q,"adapter":"openai","api_key":%q,"model_ids":["m-two","m-two"," m-three "]}`, up, keepExistingAPIKey))
|
|
if rr.Code != http.StatusOK {
|
|
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
|
}
|
|
var ids []string
|
|
for _, s := range g.core.Sources() {
|
|
if s.Name == "seeded" {
|
|
for _, m := range s.Models {
|
|
ids = append(ids, m.ID)
|
|
}
|
|
}
|
|
}
|
|
// m-one kept, m-two not duplicated, blank entries dropped, whitespace trimmed.
|
|
want := map[string]int{"m-one": 1, "m-two": 1, "m-three": 1}
|
|
if len(ids) != len(want) {
|
|
t.Fatalf("models=%v, want exactly %v", ids, want)
|
|
}
|
|
seen := map[string]int{}
|
|
for _, id := range ids {
|
|
seen[id]++
|
|
}
|
|
for id, n := range want {
|
|
if seen[id] != n {
|
|
t.Fatalf("models=%v: %q appears %d times, want %d", ids, id, seen[id], n)
|
|
}
|
|
}
|
|
}
|
|
|
|
func TestSourceWriteRejectsNonAdmin(t *testing.T) {
|
|
g, up := seededGateway(t)
|
|
for _, m := range []string{"POST", "PUT", "DELETE"} {
|
|
req, _ := http.NewRequest(m, "/api/sources/seeded", strings.NewReader(fmt.Sprintf(
|
|
`{"name":"seeded","base_url":%q,"models":[{"id":"m"}]}`, up)))
|
|
req.Header.Set("Authorization", "Bearer sk-not-a-real-key")
|
|
req.Header.Set("Content-Type", "application/json")
|
|
rr := httptest.NewRecorder()
|
|
g.Handler().ServeHTTP(rr, req)
|
|
if rr.Code == http.StatusOK {
|
|
t.Fatalf("%s allowed for an unknown key: %s", m, rr.Body.String())
|
|
}
|
|
}
|
|
if sourceAPIKey(t, g, "seeded") != "sk-real-credential" {
|
|
t.Fatal("unauthenticated write mutated the source")
|
|
}
|
|
}
|
|
|
|
// TestSourceWritePersistsToDisk guards the difference between "the in-memory
|
|
// registry accepted it" and "it will survive a restart". The api_key must be
|
|
// sealed at rest, never the literal placeholder.
|
|
func TestSourceWritePersistsToDisk(t *testing.T) {
|
|
g, up := seededGateway(t)
|
|
rr := doReq(t, g, "PUT", "/api/sources/seeded", fmt.Sprintf(
|
|
`{"base_url":%q,"adapter":"openai","api_key":%q,"models":[{"id":"m-one"}]}`, up, keepExistingAPIKey))
|
|
if rr.Code != http.StatusOK {
|
|
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
|
}
|
|
raw, err := os.ReadFile(g.core.Config().Path)
|
|
if err != nil {
|
|
t.Fatalf("read config: %v", err)
|
|
}
|
|
text := string(raw)
|
|
if strings.Contains(text, keepExistingAPIKey) {
|
|
t.Fatal("the keep-mask was persisted as a literal api_key")
|
|
}
|
|
if strings.Contains(text, "sk-real-credential") {
|
|
t.Fatal("api_key stored in plaintext; it must be sealed at rest")
|
|
}
|
|
if !strings.Contains(text, "m-one") {
|
|
t.Fatal("source not persisted")
|
|
}
|
|
// The masked GET view must not leak the credential either.
|
|
rr = doReq(t, g, "GET", "/api/v1/sources/seeded", "")
|
|
if rr.Code != http.StatusOK {
|
|
t.Fatalf("get status=%d", rr.Code)
|
|
}
|
|
var view struct {
|
|
Source struct {
|
|
APIKey string `json:"api_key"`
|
|
} `json:"source"`
|
|
}
|
|
if err := json.Unmarshal(rr.Body.Bytes(), &view); err != nil {
|
|
t.Fatalf("decode: %v", err)
|
|
}
|
|
if view.Source.APIKey == "sk-real-credential" {
|
|
t.Fatal("GET /api/v1/sources leaked the plaintext api_key")
|
|
}
|
|
}
|