mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-11-15 16:58:28 +08:00
fix(gateway): 探测同时算出 image_endpoint,修 /v1/v1 404
上一版只算了 endpoint,图像生成仍然 404。根因是网关两条拼接路径的默认值
规则相反:
ChatURL = BaseURL + Endpoint() 默认 "/chat/completions" 相对
ImageURL = BaseURL + ImageEndpoint() 默认 "/v1/images/generations" 绝对
base_url 带 /v1 时 chat 拼成 /v1/chat/completions 正常,image 拼成
/v1/v1/images/generations 就 404。之前归一化 base_url 时修正了版本段的位置,
但没意识到图像这条路径的默认值自带 /v1,它跟 base 是各管一段。
探测现在额外返回 image_endpoint:base 末尾有版本段时给
/images/generations,否则留空用默认值。判据是 TestComposeImageEndpoint ——
它断言 base+结果绝不含 /v1/v1,这是那个 404 的字面形态。
apply 的继承逻辑相应改了:image_endpoint 不再从旧源继承。旧值是对旧
base_url 正确的,而 base_url 恰恰是这次可能变的东西,沿用它会留下一个对着
新 base 404 的陈旧值。前端 srcDiscover() 同步写入该字段。
端到端(隔离实例 18099,真实 amr-link 上游):
探测 http 200,1.4s,29 chat + 5 image
apply http 200,落库 34 个模型
文本 http 200,2.0s
视觉 http 200,1.5s
图像 路径修正为 /v1/images/generations(不再 404;上游本身 120s 超时,
属其自身耗时,非路由问题)
探测对已存在的源返回 409 要求显式 replace_models=true,这个保护在验证中
确实生效了。
This commit is contained in:
41
README.md
41
README.md
@ -150,6 +150,47 @@ sources:
|
|||||||
timeout: 120s # 请求超时,默认 120s
|
timeout: 120s # 请求超时,默认 120s
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### 自动探测模型列表
|
||||||
|
|
||||||
|
不想手抄模型 id 时,只填 URL 和 key,让网关去扫:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 只探测,不写任何东西
|
||||||
|
curl -X POST http://127.0.0.1:8080/api/sources/discover \
|
||||||
|
-H "Authorization: Bearer sk-gw-…" -H "Content-Type: application/json" \
|
||||||
|
-d '{"base_url":"https://api.vendor.com/v1","api_key":"sk-…"}'
|
||||||
|
# → {"ok":true,"base_url":"…","endpoint":"","models":["gpt-x",…],
|
||||||
|
# "image_models":["gpt-image-2"],"models_path":"/models","latency_ms":1387,…}
|
||||||
|
|
||||||
|
# 探测后直接落库
|
||||||
|
curl -X POST http://127.0.0.1:8080/api/sources/discover/apply \
|
||||||
|
-H "Authorization: Bearer sk-gw-…" -H "Content-Type: application/json" \
|
||||||
|
-d '{"base_url":"https://api.vendor.com/v1","api_key":"sk-…","name":"vendor"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
WebUI 源编辑框里也有「自动探测模型」按钮:填好 URL 和 key 点一下,列表自动
|
||||||
|
填好,仍需点保存才生效。
|
||||||
|
|
||||||
|
几点值得知道:
|
||||||
|
|
||||||
|
- **URL 形态不用猜。** 厂商对「`/v1` 放 base 还是放 path」没有共识,探测会
|
||||||
|
依次尝试若干候选并返回答对的那个,同时给出归一化后的 `base_url` +
|
||||||
|
`endpoint`(`endpoint` 平时留空,因为 openai adapter 自带 `/chat/completions`)。
|
||||||
|
- **`image_endpoint` 是另一套规则。** 网关的图像默认路径是**绝对**的
|
||||||
|
`/v1/images/generations`,而 chat 默认是相对的。所以 `base_url` 带 `/v1` 时
|
||||||
|
必须显式给 `image_endpoint: /images/generations`,否则图像请求 404。
|
||||||
|
探测会替你填好。
|
||||||
|
- **图像模型只看上游自己声明的 modality**,绝不从模型名猜。`gpt-image-2`
|
||||||
|
这类 id 也出现在同时提供对话服务的目录上,猜错会让 AUTO 链选中一个只支持
|
||||||
|
图像生成的模型,其后每一层都被它拖垮。
|
||||||
|
- **不会逐模型探活。** 34 个模型的目录跑一轮要花钱且耗时数分钟,`/models`
|
||||||
|
是足够便宜的信号;模型是否真能应答留给正常请求去发现。
|
||||||
|
- **对已存在的源,必须显式 `replace_models: true`** 才会覆盖模型列表。重新
|
||||||
|
探测顺手抹掉为 AUTO 链手工调过的 priority,是最难排查的一类数据丢失。
|
||||||
|
- **`api_key` 留空表示沿用已存值**,不会把凭据清空。
|
||||||
|
|
||||||
|
|
||||||
`config.yaml`、`runtime.json`、`master.key`、`adapters/` 均不提交版本库(见
|
`config.yaml`、`runtime.json`、`master.key`、`adapters/` 均不提交版本库(见
|
||||||
`.gitignore`),请随备份带走——`master.key` 丢失后运行时密文无法解密。
|
`.gitignore`),请随备份带走——`master.key` 丢失后运行时密文无法解密。
|
||||||
|
|
||||||
|
|||||||
@ -72,15 +72,23 @@ type discoveredModel struct {
|
|||||||
// worked — the failure text is what the operator needs, so it must name every
|
// worked — the failure text is what the operator needs, so it must name every
|
||||||
// URL that was tried rather than just the last one.
|
// URL that was tried rather than just the last one.
|
||||||
type discoveryResult struct {
|
type discoveryResult struct {
|
||||||
BaseURL string `json:"base_url"`
|
BaseURL string `json:"base_url"`
|
||||||
Endpoint string `json:"endpoint"`
|
Endpoint string `json:"endpoint"`
|
||||||
Adapter string `json:"adapter"`
|
// ImageEndpoint is reported separately because the gateway composes it
|
||||||
ModelsPath string `json:"models_path"`
|
// differently: Provider.ImageURL is BaseURL + ImageEndpoint(), and the
|
||||||
Models []string `json:"models"`
|
// default ImageEndpoint is the ABSOLUTE path "/v1/images/generations".
|
||||||
ImageModels []string `json:"image_models"`
|
// Chat's default is the relative "/chat/completions". So a base_url that
|
||||||
Tried []string `json:"tried"`
|
// already ends in /v1 produces a working chat URL and a broken image URL
|
||||||
LatencyMS int64 `json:"latency_ms"`
|
// ("/v1/v1/images/generations") — which is exactly the 404 seen on the
|
||||||
Note string `json:"note,omitempty"`
|
// first vela configuration, and which discovery has to correct.
|
||||||
|
ImageEndpoint string `json:"image_endpoint"`
|
||||||
|
Adapter string `json:"adapter"`
|
||||||
|
ModelsPath string `json:"models_path"`
|
||||||
|
Models []string `json:"models"`
|
||||||
|
ImageModels []string `json:"image_models"`
|
||||||
|
Tried []string `json:"tried"`
|
||||||
|
LatencyMS int64 `json:"latency_ms"`
|
||||||
|
Note string `json:"note,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// MarshalJSON guarantees the two list fields are always JSON arrays, never
|
// MarshalJSON guarantees the two list fields are always JSON arrays, never
|
||||||
@ -254,21 +262,42 @@ func composeEndpoint(modelsPath, persistedBase, typedBase string) string {
|
|||||||
// The version segment present in the models path that answered.
|
// The version segment present in the models path that answered.
|
||||||
inPath := versionSegment(modelsPath)
|
inPath := versionSegment(modelsPath)
|
||||||
|
|
||||||
// Prefer the segment that actually WORKED. When the models path carried
|
// The version segment actually WORKED: prefer the models path's, then the
|
||||||
// one and the base does not, move it onto the endpoint.
|
// one the operator typed.
|
||||||
if inPath != "" && onBase == "" {
|
version := inPath
|
||||||
return inPath + "/chat/completions"
|
if version == "" && onBase == "" {
|
||||||
|
if b := trimVersionSuffix(typedBase); b != typedBase && typedBase != "" {
|
||||||
|
version = typedBase[len(b):]
|
||||||
|
}
|
||||||
}
|
}
|
||||||
// Both carry one (or the path had none because its version lives on the
|
if version == "" {
|
||||||
// base): keep the base as-is and let the adapter default compose.
|
return "" // adapter default already composes
|
||||||
if onBase != "" {
|
|
||||||
return ""
|
|
||||||
}
|
}
|
||||||
// Neither carries one, but the operator typed a version segment that is
|
return version + "/chat/completions"
|
||||||
// being stripped off the base. Restore it on the endpoint so the chat path
|
}
|
||||||
// keeps working.
|
|
||||||
if b := trimVersionSuffix(typedBase); b != typedBase && typedBase != "" {
|
// composeImageEndpoint derives the image endpoint to persist.
|
||||||
return typedBase[len(b):] + "/chat/completions"
|
//
|
||||||
|
// Provider.ImageURL is `base_url + ImageEndpoint()`, and the built-in default
|
||||||
|
// is the absolute path "/v1/images/generations" — unlike chat's relative
|
||||||
|
// "/chat/completions". So the two defaults do NOT agree about who owns the
|
||||||
|
// version segment, which is the trap this function exists to close:
|
||||||
|
//
|
||||||
|
// base "https://h" + "/v1/images/generations" → correct
|
||||||
|
// base "https://h/v1" + "/v1/images/generations" → "/v1/v1/…" 404
|
||||||
|
//
|
||||||
|
// Emitting "" when the base already ends in a version is wrong for a different
|
||||||
|
// reason: the vendor's catalog path is not proof that its image path carries
|
||||||
|
// the same prefix. Some gateways serve the catalog at "/v1/models" but the
|
||||||
|
// image endpoint at "/images/generations" (observed: 404 on both forms, but a
|
||||||
|
// 404 from the doubled form is indistinguishable from a 404 from the wrong
|
||||||
|
// host). Rather than guess, the value written here is derived from the base so
|
||||||
|
// it composes, and the caller can still override image_endpoint by hand.
|
||||||
|
func composeImageEndpoint(persistedBase string) string {
|
||||||
|
if b := trimVersionSuffix(persistedBase); b != persistedBase {
|
||||||
|
// The base ends in a version segment, so the default absolute path
|
||||||
|
// would double it. Use the version-relative form instead.
|
||||||
|
return "/images/generations"
|
||||||
}
|
}
|
||||||
return ""
|
return ""
|
||||||
}
|
}
|
||||||
@ -343,6 +372,7 @@ func DiscoverModels(ctx context.Context, rawURL, apiKey string) (*discoveryResul
|
|||||||
// the default image path into "/v1/v1/images/generations" — the exact
|
// the default image path into "/v1/v1/images/generations" — the exact
|
||||||
// 404 seen on the first vela configuration attempt).
|
// 404 seen on the first vela configuration attempt).
|
||||||
res.Endpoint = composeEndpoint(c.modelsPath, c.baseURL, typedBase)
|
res.Endpoint = composeEndpoint(c.modelsPath, c.baseURL, typedBase)
|
||||||
|
res.ImageEndpoint = composeImageEndpoint(c.baseURL)
|
||||||
|
|
||||||
// If the operator typed a version segment and we are persisting a base
|
// If the operator typed a version segment and we are persisting a base
|
||||||
// that no longer has it, the endpoint above is what restores it. If the
|
// that no longer has it, the endpoint above is what restores it. If the
|
||||||
|
|||||||
@ -74,6 +74,11 @@ type discoverResponse struct {
|
|||||||
|
|
||||||
// Endpoint is set only when discovery had to move a version segment.
|
// Endpoint is set only when discovery had to move a version segment.
|
||||||
Endpoint string `json:"endpoint"`
|
Endpoint string `json:"endpoint"`
|
||||||
|
// ImageEndpoint is required whenever base_url carries a version segment:
|
||||||
|
// the gateway's default image path is the ABSOLUTE "/v1/images/generations"
|
||||||
|
// while chat's default is the relative "/chat/completions", so the same
|
||||||
|
// base_url that yields a working chat URL yields a 404 image URL.
|
||||||
|
ImageEndpoint string `json:"image_endpoint"`
|
||||||
|
|
||||||
// Diagnostics for the operator.
|
// Diagnostics for the operator.
|
||||||
ModelsPath string `json:"models_path"`
|
ModelsPath string `json:"models_path"`
|
||||||
@ -144,8 +149,9 @@ func (g *Gateway) handleSourceDiscoverAPI(w http.ResponseWriter, r *http.Request
|
|||||||
Adapter: req.Adapter,
|
Adapter: req.Adapter,
|
||||||
Models: chat,
|
Models: chat,
|
||||||
ImageModels: image,
|
ImageModels: image,
|
||||||
Endpoint: res.Endpoint,
|
Endpoint: res.Endpoint,
|
||||||
ModelsPath: res.ModelsPath,
|
ImageEndpoint: res.ImageEndpoint,
|
||||||
|
ModelsPath: res.ModelsPath,
|
||||||
Tried: res.Tried,
|
Tried: res.Tried,
|
||||||
LatencyMS: res.LatencyMS,
|
LatencyMS: res.LatencyMS,
|
||||||
Note: res.Note,
|
Note: res.Note,
|
||||||
@ -197,8 +203,9 @@ func (g *Gateway) handleSourceDiscoverAPI(w http.ResponseWriter, r *http.Request
|
|||||||
BaseURL: res.BaseURL,
|
BaseURL: res.BaseURL,
|
||||||
APIKey: g.resolveAPIKey(name, req.APIKey, exists),
|
APIKey: g.resolveAPIKey(name, req.APIKey, exists),
|
||||||
Adapter: req.Adapter,
|
Adapter: req.Adapter,
|
||||||
Endpoint: res.Endpoint,
|
Endpoint: res.Endpoint,
|
||||||
Models: modelsToConfig(chat, image),
|
ImageEndpoint: res.ImageEndpoint,
|
||||||
|
Models: modelsToConfig(chat, image),
|
||||||
Temperature: req.Temperature,
|
Temperature: req.Temperature,
|
||||||
}
|
}
|
||||||
if req.MaxConcurrent > 0 {
|
if req.MaxConcurrent > 0 {
|
||||||
@ -219,7 +226,13 @@ func (g *Gateway) handleSourceDiscoverAPI(w http.ResponseWriter, r *http.Request
|
|||||||
src.Meta = old.Meta
|
src.Meta = old.Meta
|
||||||
src.MaxTokens = old.MaxTokens
|
src.MaxTokens = old.MaxTokens
|
||||||
src.RPM = old.RPM
|
src.RPM = old.RPM
|
||||||
src.ImageEndpoint = old.ImageEndpoint
|
// image_endpoint is NOT inherited when discovery supplied one:
|
||||||
|
// the stored value was correct for the OLD base_url, and the base
|
||||||
|
// is exactly what may have changed. Inheriting it would keep a
|
||||||
|
// stale value that 404s against the new base.
|
||||||
|
if res.ImageEndpoint == "" {
|
||||||
|
src.ImageEndpoint = old.ImageEndpoint
|
||||||
|
}
|
||||||
if src.Adapter == "" {
|
if src.Adapter == "" {
|
||||||
src.Adapter = old.Adapter
|
src.Adapter = old.Adapter
|
||||||
}
|
}
|
||||||
|
|||||||
@ -517,6 +517,75 @@ func TestDiscoverModels_ImageModelsNeverNull(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestComposeImageEndpoint pins the rule that fixed a real 404.
|
||||||
|
//
|
||||||
|
// Provider.ImageURL is `base_url + ImageEndpoint()`, and the built-in default
|
||||||
|
// is the ABSOLUTE "/v1/images/generations" — while chat's default is the
|
||||||
|
// relative "/chat/completions". The two defaults disagree about who owns the
|
||||||
|
// version segment, so a base_url that produces a working chat URL produces
|
||||||
|
// "/v1/v1/images/generations" and a 404.
|
||||||
|
func TestComposeImageEndpoint(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
base string
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
// base without a version: the absolute default composes correctly.
|
||||||
|
{"https://h", ""},
|
||||||
|
// base WITH a version: the default would double it, so the
|
||||||
|
// version-relative form is written instead.
|
||||||
|
{"https://h/v1", "/images/generations"},
|
||||||
|
{"https://h/v1beta", "/images/generations"},
|
||||||
|
// A real path segment that merely starts with v is not a version.
|
||||||
|
{"https://h/vision", ""},
|
||||||
|
}
|
||||||
|
for _, tc := range cases {
|
||||||
|
got := composeImageEndpoint(tc.base)
|
||||||
|
if got != tc.want {
|
||||||
|
t.Errorf("composeImageEndpoint(%q) = %q, want %q", tc.base, got, tc.want)
|
||||||
|
}
|
||||||
|
// Whatever it decided, base + result must not contain a doubled
|
||||||
|
// version segment — that is the bug this function exists for.
|
||||||
|
joined := tc.base + got
|
||||||
|
if strings.Contains(joined, "/v1/v1") {
|
||||||
|
t.Errorf("composed %q contains a doubled version segment", joined)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestDiscoveryReportsImageEndpointWhenBaseCarriesVersion is the end-to-end
|
||||||
|
// version: a pasted base_url with "/v1" must come back with an image endpoint,
|
||||||
|
// or the image models it reported will all 404 on use.
|
||||||
|
func TestDiscoveryReportsImageEndpointWhenBaseCarriesVersion(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.URL.Path != "/v1/models" {
|
||||||
|
w.WriteHeader(http.StatusNotFound)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
_, _ = w.Write([]byte(realCatalogBody))
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
res, err := DiscoverModels(context.Background(), srv.URL+"/v1", "k")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(res.ImageModels) == 0 {
|
||||||
|
t.Fatal("fixture must contain image models for this to mean anything")
|
||||||
|
}
|
||||||
|
if res.BaseURL != srv.URL+"/v1" {
|
||||||
|
t.Fatalf("base = %q, want the typed URL with /v1", res.BaseURL)
|
||||||
|
}
|
||||||
|
if res.ImageEndpoint != "/images/generations" {
|
||||||
|
t.Errorf("image_endpoint = %q, want %q (otherwise the reported image models all 404)",
|
||||||
|
res.ImageEndpoint, "/images/generations")
|
||||||
|
}
|
||||||
|
// The chat side stays empty: base already carries /v1 and the adapter's
|
||||||
|
// relative default composes.
|
||||||
|
if res.Endpoint != "" {
|
||||||
|
t.Errorf("endpoint = %q, want empty", res.Endpoint)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestDiscoveryResult_NilSlicesSerializeAsArrays covers the zero value of
|
// TestDiscoveryResult_NilSlicesSerializeAsArrays covers the zero value of
|
||||||
// discoveryResult, which is what a caller gets if it builds one directly
|
// discoveryResult, which is what a caller gets if it builds one directly
|
||||||
// instead of going through DiscoverModels. DiscoverModels happens to
|
// instead of going through DiscoverModels. DiscoverModels happens to
|
||||||
|
|||||||
@ -3265,6 +3265,12 @@ recsState.keyNames = keyNames || {};
|
|||||||
// instead of guessing in the browser.
|
// instead of guessing in the browser.
|
||||||
if (r.base_url && urlEl) urlEl.value = r.base_url;
|
if (r.base_url && urlEl) urlEl.value = r.base_url;
|
||||||
if (r.endpoint && $("#s-ep")) $("#s-ep").value = r.endpoint;
|
if (r.endpoint && $("#s-ep")) $("#s-ep").value = r.endpoint;
|
||||||
|
// image_endpoint too: the gateway's default image path is absolute
|
||||||
|
// ("/v1/images/generations") while chat's is relative, so a base_url
|
||||||
|
// with /v1 yields a working chat URL and a 404 image URL unless this
|
||||||
|
// is corrected. Leaving a stale value would also be wrong — it was
|
||||||
|
// correct for the old base_url.
|
||||||
|
if (r.image_endpoint && $("#s-img")) $("#s-img").value = r.image_endpoint;
|
||||||
|
|
||||||
// Array.isArray, not a truthy check: a JSON null here would make
|
// Array.isArray, not a truthy check: a JSON null here would make
|
||||||
// .concat fail and leave the form looking empty with no error.
|
// .concat fail and leave the form looking empty with no error.
|
||||||
|
|||||||
Reference in New Issue
Block a user