mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-11-07 17:57:23 +00:00
Compare commits
11 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 79bc55cfbe | |||
| 4b9fb7b056 | |||
| 7941b4110a | |||
| 5ccf068db3 | |||
| b4c5f0f0fc | |||
| 442fa5e3f6 | |||
| 0caa37ea6a | |||
| 33cb2bdb06 | |||
| 0ae7d2c8be | |||
| 02c01582f1 | |||
| 6454b094bc |
4
cmd/gui/package-lock.json
generated
4
cmd/gui/package-lock.json
generated
@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "modelrouter-gui",
|
||||
"version": "2.0.45",
|
||||
"version": "2.0.49",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "modelrouter-gui",
|
||||
"version": "2.0.45",
|
||||
"version": "2.0.49",
|
||||
"license": "Proprietary",
|
||||
"devDependencies": {
|
||||
"electron": "^33.0.0",
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "modelrouter-gui",
|
||||
"version": "2.0.45",
|
||||
"version": "2.0.49",
|
||||
"description": "ModelRouter Desktop — embedded ModelRouter core with tray",
|
||||
"author": "ModelRouter",
|
||||
"main": "main.js",
|
||||
|
||||
@ -503,15 +503,57 @@ end
|
||||
|
||||
`plugins/billing.lua`，默认随首启 seed 进来，开箱可用。
|
||||
|
||||
### 7.1 计费维度
|
||||
### 7.0 分层：billing 完全在插件里，内核没有任何 billing 代码
|
||||
|
||||
| 维度 | 用途 | 优先级 |
|
||||
`billing` 是**插件自己的事**：规则文件、编译、计价、账目、规则编辑页都由
|
||||
`plugins/billing.lua` 负责。内核里没有 billing 的配置结构、端点、编译或迁移逻辑
|
||||
—— 它只提供所有插件通用的能力（加载 `.lua`、插件状态持久化、插件目录、
|
||||
`plugin.http` 路由、插件可用的 `io`/`os`）。
|
||||
|
||||
| 项 | 位置 |
|
||||
|---|---|
|
||||
| 规则文件 | `<plugin_dir>/billing.json`（插件自己读写；内核不知道这个文件） |
|
||||
| 计费账目 | 插件状态文件 `<plugin_dir>/.billing.state.json`（内核的通用状态持久化） |
|
||||
| 规则编辑 API | `/api/plugins/http/billing/rules`（GET/PUT/POST/DELETE，插件路由） |
|
||||
| 编译/计价 | 插件内 `compileRules()` / `priceFor()` / `costFor()` |
|
||||
|
||||
规则以**源名**（`source`）为键，而不是 URL：URL→源名的映射属于内核配置，插件
|
||||
不需要为了它和内核对接；编辑页在浏览器里，直接取 `/api/sources` 给运营选源名。
|
||||
`url: "*"` 只表示"全局模型价目表"（模型维度本来就不分源）。
|
||||
|
||||
> 历史：早期 `billing:` 写在 `config.yaml`、编译在 Go 的 `internal/billing` 里。
|
||||
> GitCode issue #1 的根因正是"Go 与 Lua 各写一半覆盖语义"，组合起来把模型价
|
||||
> 静默清零。现在内核侧 billing 代码已全部删除，规则文件也由插件自持。
|
||||
|
||||
### 7.1 计费维度### 7.1 计费维度
|
||||
|
||||
| 维度 | 用途 | token 价覆盖顺序 |
|
||||
|---|---|---|
|
||||
| `sources` | 源的固定价（如按请求计费的转发服务） | 中 |
|
||||
| `models` | 模型的 per-token 价 | 高 |
|
||||
| `default` | 兜底价（通常全 0） | 最低 |
|
||||
| `models` | 模型的 per-token 价 | 中 |
|
||||
| `sources` | 源级价：固定价，或覆盖该源全部模型的 token 价 | 高 |
|
||||
| `keys` | 单个网关 key 的覆盖价 | **最高** |
|
||||
|
||||
**token 价**优先级：`keys` > `models` > `default`。
|
||||
**token 价**优先级：`keys` > `sources` > `models` > `default`。
|
||||
（实现顺序就是"后写覆盖前写"：`default` → `models` → `sources` → `keys`；
|
||||
所以**源级 token 价会覆盖模型价**。这是有意的：运营要能给某个源整体改价。）
|
||||
|
||||
⚠️ 正因为 `sources` 会覆盖 `models`：**`mode: token` 的规则不产生任何源条目**。
|
||||
规则未列出的模型会进入未定价统计（可见），而不是静默计 0 —— 这是 issue #1 / PR #2
|
||||
的收口语义。只有 `free` / `subscription`（"边际成本为零"）会写有意为之的
|
||||
`{prompt:0, completion:0}` 零价条目。
|
||||
|
||||
规则写法：
|
||||
|
||||
```json
|
||||
{"active":"current","profiles":[{"id":"current","default":"unpriced","rules":[
|
||||
{"source":"localzen","mode":"free"},
|
||||
{"source":"codearts","mode":"token",
|
||||
"models":{"glm-5.3":{"prompt":"0.15","completion":"0.60"}}},
|
||||
{"url":"*","mode":"token","models":{"some-model":{"prompt":"1.0","completion":"2.0"}}}
|
||||
]}]}
|
||||
```
|
||||
|
||||
**`per_request` 固定价是叠加的**（不覆盖），所以一个生图模型可以既算 token 又收固定费。
|
||||
**`cache_discount`** 见 §7.7。
|
||||
|
||||
@ -599,6 +641,17 @@ curl -H "Authorization: Bearer $ADMIN_KEY" \
|
||||
|
||||
仪表盘上有 "Unpriced" 卡片。**这个数应该是 0**；不是 0 就去补价目。
|
||||
|
||||
**profile 级兜底**：`profiles[].default` 可设为 `free` 或 `unpriced`（默认）。
|
||||
它是插件运行时的策略（编译期写入 `prices.default_mode`）：
|
||||
|
||||
| `default` | 没有单独规则的源 | `unpriced_reqs` |
|
||||
|---|---|---|
|
||||
| `unpriced`（省略时） | 零价，计入未定价债务 | +1 |
|
||||
| `free` | 零价，视为**刻意免费** | 不增加 |
|
||||
|
||||
显式 `mode: unpriced` 的源即使在 `default: free` 下也仍然计入（"只统计不收费"
|
||||
的声明不能被兜底策略吞掉）。
|
||||
|
||||
注意"未定价"不等于"免费"：这些请求的 `requests` / token 数**照常计入**
|
||||
`total` 与各维度，只有金额是 0。
|
||||
|
||||
|
||||
@ -1,323 +0,0 @@
|
||||
package billing
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// Compile turns one billing profile into the prices table the billing plugin
|
||||
// expects, resolving URL rules against the gateway's actual sources.
|
||||
//
|
||||
// WHY URL RULES NEED RESOLVING AT ALL: the operator declares pricing by URL
|
||||
// because that is what a provider's price list is keyed on, and because several
|
||||
// sources can point at the same URL. The plugin, however, looks up by
|
||||
// `payload.source` (a source NAME) and by model — it has no idea what URL a
|
||||
// request went to. So the URL match happens here, at compile time, where the
|
||||
// config's name->base_url mapping is known, and the result is expressed in the
|
||||
// dimensions the plugin already supports.
|
||||
//
|
||||
// Profiles are the "let the user choose" axis: the same URL can appear in
|
||||
// several profiles and switching recomputes this table, so a gateway can be
|
||||
// repriced without editing the plugin.
|
||||
func Compile(profile *config.BillingProfile, sources []config.Source) (map[string]interface{}, error) {
|
||||
return CompileOpts(profile, sources, false)
|
||||
}
|
||||
|
||||
// CompileOpts is Compile with an explicit policy for rules that match no
|
||||
// source.
|
||||
//
|
||||
// The default is STRICT and that is correct for startup: a profile with a
|
||||
// typo'd URL prices nothing, every request on it is recorded as unpriced, and
|
||||
// the bill silently comes out wrong. Failing the load is the right response.
|
||||
//
|
||||
// The rules editor needs the opposite. An operator editing a half-finished
|
||||
// profile — adding a rule before the source that will use it exists, or
|
||||
// renaming a URL while typing — must be able to save and then see the warning,
|
||||
// not be blocked by an error they can only resolve by guessing. Lenient mode
|
||||
// keeps the offending rule in the config and drops it from the compiled table,
|
||||
// so it is preserved as text and clearly reported, without pretending it
|
||||
// prices anything.
|
||||
func CompileOpts(profile *config.BillingProfile, sources []config.Source, lenient bool) (map[string]interface{}, error) {
|
||||
if profile == nil {
|
||||
return nil, fmt.Errorf("no billing profile")
|
||||
}
|
||||
prices := map[string]interface{}{
|
||||
"currency": "USD",
|
||||
"default": map[string]interface{}{"prompt": 0.0, "completion": 0.0, "per_request": 0.0},
|
||||
"sources": map[string]interface{}{},
|
||||
"models": map[string]interface{}{},
|
||||
"keys": map[string]interface{}{},
|
||||
}
|
||||
if profile.Default != "" {
|
||||
prices["default_mode"] = profile.Default
|
||||
}
|
||||
|
||||
// Currency is a profile-level statement; a rule may override it.
|
||||
for i := range profile.Rules {
|
||||
if c := profile.Rules[i].Currency; c != "" {
|
||||
prices["currency"] = c
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
matched := map[string]bool{}
|
||||
for i := range profile.Rules {
|
||||
rule := &profile.Rules[i]
|
||||
targets := matchSources(rule.URL, sources)
|
||||
if len(targets) == 0 && rule.URL != "*" {
|
||||
// A rule for a URL no source uses is almost always a typo (or a
|
||||
// source that was removed). Failing loudly beats a profile that
|
||||
// silently prices nothing — the whole reason this is a config file
|
||||
// instead of a hand-written JSON blob.
|
||||
if lenient {
|
||||
continue // kept in the config text, reported as a warning
|
||||
}
|
||||
return nil, fmt.Errorf("rule url %q matches no configured source base_url", rule.URL)
|
||||
}
|
||||
for _, srcName := range targets {
|
||||
if matched[srcName] {
|
||||
// First rule wins. Two rules matching one source is ambiguous,
|
||||
// and silently letting the later one win makes the file's
|
||||
// meaning depend on ordering the operator cannot see.
|
||||
continue
|
||||
}
|
||||
matched[srcName] = true
|
||||
if err := applyRule(prices, rule, srcName, sources); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Sources no rule matched fall through to the profile default, which the
|
||||
// plugin applies via prices.default. Recording them explicitly means the
|
||||
// UI can say "this source is unpriced" instead of leaving the operator to
|
||||
// infer it from a zero.
|
||||
var unmatched []string
|
||||
for _, src := range sources {
|
||||
if !matched[src.Name] {
|
||||
unmatched = append(unmatched, src.Name)
|
||||
}
|
||||
}
|
||||
if len(unmatched) > 0 {
|
||||
prices["unmatched_sources"] = unmatched
|
||||
}
|
||||
return prices, nil
|
||||
}
|
||||
|
||||
// matchSources returns the source names whose base_url matches pattern.
|
||||
// "*" matches every source (used as a catch-all default rule).
|
||||
func matchSources(pattern string, sources []config.Source) []string {
|
||||
var out []string
|
||||
if pattern == "*" {
|
||||
for _, s := range sources {
|
||||
out = append(out, s.Name)
|
||||
}
|
||||
return out
|
||||
}
|
||||
want := normalizeURL(pattern)
|
||||
for _, s := range sources {
|
||||
if normalizeURL(s.BaseURL) == want {
|
||||
out = append(out, s.Name)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// normalizeURL compares URLs the way an operator expects: trailing slashes and
|
||||
// case in the host are differences the provider's price list does not care
|
||||
// about, and requiring an exact byte match would make the config brittle.
|
||||
func normalizeURL(u string) string {
|
||||
u = strings.TrimSpace(u)
|
||||
u = strings.TrimRight(u, "/")
|
||||
return strings.ToLower(u)
|
||||
}
|
||||
|
||||
// applyRule writes one rule's pricing for one source into the prices table.
|
||||
func applyRule(prices map[string]interface{}, rule *config.BillingRule, srcName string, sources []config.Source) error {
|
||||
switch rule.Mode {
|
||||
case "free":
|
||||
// Explicitly priced at zero. This is NOT the same as unpriced: a source
|
||||
// the operator says is free must not appear in the unpriced warnings,
|
||||
// or those warnings become noise and stop being read.
|
||||
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
|
||||
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
|
||||
}
|
||||
return nil
|
||||
|
||||
case "subscription":
|
||||
// A fixed monthly commitment: the per-request MARGINAL cost is zero, and
|
||||
// the flat fee is reported separately. Spreading a monthly fee across
|
||||
// requests would invent a per-request number the provider never charges,
|
||||
// and it would change every time traffic did.
|
||||
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
|
||||
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
|
||||
}
|
||||
fixed, _ := prices["subscriptions"].(map[string]interface{})
|
||||
if fixed == nil {
|
||||
fixed = map[string]interface{}{}
|
||||
prices["subscriptions"] = fixed
|
||||
}
|
||||
cur := "USD"
|
||||
if rule.Currency != "" {
|
||||
cur = rule.Currency
|
||||
}
|
||||
fixed[srcName] = map[string]interface{}{
|
||||
"monthly": rule.Subscription, "currency": cur,
|
||||
}
|
||||
return nil
|
||||
|
||||
case "unpriced":
|
||||
// Deliberately left out of `sources` so the plugin's unpriced_models /
|
||||
// unpriced_reqs counters catch it. That is the point: a subscription
|
||||
// plan whose credits cannot be converted to tokens must be VISIBLE as
|
||||
// unbilled, not quietly estimated.
|
||||
return nil
|
||||
|
||||
case "token":
|
||||
if len(rule.Models) == 0 {
|
||||
return fmt.Errorf("rule for url %q: mode token requires at least one model", rule.URL)
|
||||
}
|
||||
models := prices["models"].(map[string]interface{})
|
||||
for modelID, t := range rule.Models {
|
||||
entry := map[string]interface{}{}
|
||||
prompt, err := usdPerM(t.Prompt)
|
||||
if err != nil {
|
||||
return fmt.Errorf("model %q prompt: %w", modelID, err)
|
||||
}
|
||||
completion, err := usdPerM(t.Completion)
|
||||
if err != nil {
|
||||
return fmt.Errorf("model %q completion: %w", modelID, err)
|
||||
}
|
||||
entry["prompt"] = prompt
|
||||
entry["completion"] = completion
|
||||
if t.CacheDiscount != nil {
|
||||
entry["cache_discount"] = *t.CacheDiscount
|
||||
}
|
||||
// reasoning is optional and stays ABSENT (not nil, not 0) when the
|
||||
// rule does not set it, which is what makes the billing plugin skip
|
||||
// the repricing branch entirely. Writing an explicit 0 here would
|
||||
// mean "thinking is free" and silently under-bill every request
|
||||
// from a provider that bills it like ordinary output.
|
||||
if strings.TrimSpace(t.Reasoning) != "" {
|
||||
rp, err := usdPerM(t.Reasoning)
|
||||
if err != nil {
|
||||
return fmt.Errorf("model %q reasoning: %w", modelID, err)
|
||||
}
|
||||
entry["reasoning_price"] = rp
|
||||
}
|
||||
if lc := t.LongContext; lc != nil {
|
||||
if lc.Above <= 0 {
|
||||
return fmt.Errorf("model %q long_context: above must be > 0", modelID)
|
||||
}
|
||||
above, err := longContextRate(lc, prompt)
|
||||
if err != nil {
|
||||
return fmt.Errorf("model %q long_context: %w", modelID, err)
|
||||
}
|
||||
entry["long_context"] = map[string]interface{}{
|
||||
"above": lc.Above,
|
||||
// Marginal, not blended. The plugin charges the tokens past the
|
||||
// threshold at this rate and the rest at the base rate, so the
|
||||
// threshold composes with a cache discount (which applies to the
|
||||
// cached portion) instead of forcing the plugin to guess how
|
||||
// the two interact.
|
||||
"prompt": above,
|
||||
}
|
||||
}
|
||||
if rule.Peak != nil {
|
||||
entry["peak"] = compilePeak(rule.Peak)
|
||||
}
|
||||
models[modelID] = entry
|
||||
}
|
||||
// A specific-URL token rule needs a source entry so a model served
|
||||
// from THIS url but absent from `rule.Models` is still recognised as
|
||||
// belonging to a priced provider.
|
||||
//
|
||||
// A WILDCARD rule ("*") must NOT get one. "*" is a model catalogue — it
|
||||
// says "these model ids have these prices, wherever they come from" —
|
||||
// and marking every source priced at {0,0} would make priceFor() set
|
||||
// priced=true for every model the catalogue does NOT list. Paid
|
||||
// sources (commandcode, alittokenplan, …) would then record $0 instead
|
||||
// of showing up in unpriced_reqs, which is precisely the silent
|
||||
// zero-billing failure this whole plugin is built to make visible.
|
||||
// The model lookup p.models[payload.model] already marks a listed
|
||||
// model priced on its own, so the wildcard loses nothing.
|
||||
if rule.URL != "*" {
|
||||
if _, ok := prices["sources"].(map[string]interface{})[srcName]; !ok {
|
||||
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
|
||||
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("unknown mode %q", rule.Mode)
|
||||
}
|
||||
|
||||
// usdPerM converts a USD-per-million string into the per-token rate the plugin
|
||||
// expects. Delegates to config.ParseFloatUSDPerM so the DSL validator and the
|
||||
// compiler agree byte-for-byte on what a valid price is — a value that passes
|
||||
// Validate() but fails here (or vice versa) would be the worst kind of drift.
|
||||
func usdPerM(s string) (float64, error) {
|
||||
return config.ParseFloatUSDPerM(s)
|
||||
}
|
||||
|
||||
// compilePeak renders the peak window in the shape the plugin reads:
|
||||
// { multiplier, windows = { { days = {...}, hours = { {lo,hi}, ... } } } }.
|
||||
//
|
||||
// The plugin reads `days` / `hours` pairs. The first version of the deployed
|
||||
// price table used { start, end, weekdays } — a shape nothing reads — so peak
|
||||
// traffic was billed at off-peak rates with no error anywhere. Compiling from
|
||||
// typed config fields removes the chance of writing the wrong key names by hand.
|
||||
func compilePeak(p *config.BillingPeak) map[string]interface{} {
|
||||
win := map[string]interface{}{}
|
||||
if len(p.Weekdays) > 0 {
|
||||
days := make([]interface{}, 0, len(p.Weekdays))
|
||||
for _, d := range p.Weekdays {
|
||||
days = append(days, d)
|
||||
}
|
||||
win["days"] = days
|
||||
}
|
||||
if len(p.Hours) > 0 {
|
||||
hours := make([]interface{}, 0, len(p.Hours))
|
||||
for _, h := range p.Hours {
|
||||
hours = append(hours, []interface{}{h[0], h[1]})
|
||||
}
|
||||
win["hours"] = hours
|
||||
}
|
||||
out := map[string]interface{}{"multiplier": p.Multiplier}
|
||||
if len(win) > 0 {
|
||||
out["windows"] = []interface{}{win}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// longContextRate resolves the marginal per-token rate for the tokens ABOVE a
|
||||
// threshold, given the model's base per-million prompt price.
|
||||
//
|
||||
// Two ways to express it are supported because providers publish both: an
|
||||
// absolute rate (OpenAI quotes \$3.00/M past 200k) and a multiple (DeepSeek
|
||||
// quotes exactly 2x past 128k). Neither may be missing: a step with no rate is a
|
||||
// silent no-op, and an operator who wrote one meant something.
|
||||
func longContextRate(lc *config.BillingLongContext, basePerToken float64) (float64, error) {
|
||||
hasPrompt := strings.TrimSpace(lc.Prompt) != ""
|
||||
hasMult := lc.Multiplier > 0
|
||||
switch {
|
||||
case hasPrompt && hasMult:
|
||||
p, err := usdPerM(lc.Prompt)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
// Both given: the absolute rate wins, because it is the one the provider
|
||||
// actually quotes. Silently multiplying instead would produce a number
|
||||
// the operator never wrote.
|
||||
return p, nil
|
||||
case hasPrompt:
|
||||
return usdPerM(lc.Prompt)
|
||||
case hasMult:
|
||||
return basePerToken * lc.Multiplier, nil
|
||||
default:
|
||||
return 0, fmt.Errorf("needs either prompt or multiplier")
|
||||
}
|
||||
}
|
||||
@ -1,214 +0,0 @@
|
||||
package billing
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// The wildcard token rule must not blanket-mark every source as priced.
|
||||
//
|
||||
// Why this is a hazard and not a style question: priceFor() sets priced=true as
|
||||
// soon as the request's SOURCE has an entry in the price table, and that flag
|
||||
// is what keeps a request OUT of unpriced_reqs. A "*" rule is a model
|
||||
// catalogue — it prices specific model ids wherever they come from — but
|
||||
// compiling it into a {0,0,0} entry on all 21 sources made every model the
|
||||
// catalogue does not list look "priced at zero". Paid providers (commandcode,
|
||||
// alittokenplan, …) then stopped appearing in the unpriced warnings, which is
|
||||
// exactly the silent zero-billing failure the plugin exists to surface.
|
||||
func TestWildcardTokenRuleDoesNotMarkEverySourcePriced(t *testing.T) {
|
||||
sources := []config.Source{
|
||||
{Name: "free1", BaseURL: "https://free1.example.com/v1"},
|
||||
{Name: "paid1", BaseURL: "https://paid1.example.com/v1"},
|
||||
{Name: "paid2", BaseURL: "https://paid2.example.com/v1"},
|
||||
}
|
||||
profile := &config.BillingProfile{
|
||||
ID: "p",
|
||||
Rules: []config.BillingRule{
|
||||
{URL: "https://free1.example.com/v1", Mode: "free"},
|
||||
{
|
||||
URL: "*", Mode: "token",
|
||||
Models: map[string]config.BillingToken{
|
||||
"only-catalogued-model": {Prompt: "1.00", Completion: "2.00"},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
prices, err := Compile(profile, sources)
|
||||
if err != nil {
|
||||
t.Fatalf("compile: %v", err)
|
||||
}
|
||||
srcs := prices["sources"].(map[string]interface{})
|
||||
|
||||
// The explicitly free source keeps its entry (free is a deliberate 0).
|
||||
if _, ok := srcs["free1"]; !ok {
|
||||
t.Error("the explicit free rule lost its source entry")
|
||||
}
|
||||
// Paid sources must NOT be marked, or their unlisted models bill $0.
|
||||
for _, name := range []string{"paid1", "paid2"} {
|
||||
if _, ok := srcs[name]; ok {
|
||||
t.Errorf("wildcard token rule marked %q as priced at 0 — its unlisted "+
|
||||
"models will bill $0 instead of appearing as unpriced", name)
|
||||
}
|
||||
}
|
||||
// The catalogue itself must still be present, or the rule prices nothing.
|
||||
models := prices["models"].(map[string]interface{})
|
||||
if _, ok := models["only-catalogued-model"]; !ok {
|
||||
t.Error("the wildcard rule did not publish its model price")
|
||||
}
|
||||
}
|
||||
|
||||
// The inverse guard: a token rule for a SPECIFIC url still marks that source,
|
||||
// because it genuinely declares pricing for the models served from it.
|
||||
func TestSpecificURLTokenRuleStillMarksItsSource(t *testing.T) {
|
||||
sources := []config.Source{
|
||||
{Name: "mine", BaseURL: "https://mine.example.com/v1"},
|
||||
{Name: "other", BaseURL: "https://other.example.com/v1"},
|
||||
}
|
||||
profile := &config.BillingProfile{
|
||||
ID: "p",
|
||||
Rules: []config.BillingRule{{
|
||||
URL: "https://mine.example.com/v1", Mode: "token",
|
||||
Models: map[string]config.BillingToken{
|
||||
"m": {Prompt: "1.00", Completion: "2.00"},
|
||||
},
|
||||
}},
|
||||
}
|
||||
prices, err := Compile(profile, sources)
|
||||
if err != nil {
|
||||
t.Fatalf("compile: %v", err)
|
||||
}
|
||||
srcs := prices["sources"].(map[string]interface{})
|
||||
if _, ok := srcs["mine"]; !ok {
|
||||
t.Error("a specific-url token rule must still mark its own source priced")
|
||||
}
|
||||
if _, ok := srcs["other"]; ok {
|
||||
t.Error("an unrelated source was marked priced by a specific-url rule")
|
||||
}
|
||||
}
|
||||
|
||||
// TestReasoningPriceIsOmittedUnlessSet pins the absence convention for the
|
||||
// thinking-token rate.
|
||||
//
|
||||
// A rule that says nothing about reasoning must produce a price entry with NO
|
||||
// reasoning_price key, so the billing plugin skips the repricing branch and
|
||||
// bills thinking at the ordinary output rate — which is what every provider
|
||||
// actually charges. Writing an explicit 0 instead would read as "thinking is
|
||||
// free" and silently under-bill every request from that model.
|
||||
func TestReasoningPriceIsOmittedUnlessSet(t *testing.T) {
|
||||
sources := []config.Source{{Name: "up", BaseURL: "https://up.example/v1"}}
|
||||
|
||||
// Case 1: no reasoning declared → the key must not appear.
|
||||
p1 := &config.BillingProfile{ID: "p", Rules: []config.BillingRule{{
|
||||
URL: "https://up.example/v1", Mode: "token",
|
||||
Models: map[string]config.BillingToken{
|
||||
"m": {Prompt: "1.00", Completion: "2.00"},
|
||||
},
|
||||
}}}
|
||||
prices, err := Compile(p1, sources)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
entry := prices["models"].(map[string]interface{})["m"].(map[string]interface{})
|
||||
if _, present := entry["reasoning_price"]; present {
|
||||
t.Errorf("reasoning_price present (%v) without a rule; the plugin would treat 0 as free",
|
||||
entry["reasoning_price"])
|
||||
}
|
||||
|
||||
// Case 2: declared → present, and converted to per-token like every other rate.
|
||||
p2 := &config.BillingProfile{ID: "p", Rules: []config.BillingRule{{
|
||||
URL: "https://up.example/v1", Mode: "token",
|
||||
Models: map[string]config.BillingToken{
|
||||
"m": {Prompt: "1.00", Completion: "2.00", Reasoning: "6.00"},
|
||||
},
|
||||
}}}
|
||||
prices2, err := Compile(p2, sources)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
e2 := prices2["models"].(map[string]interface{})["m"].(map[string]interface{})
|
||||
rp, ok := e2["reasoning_price"].(float64)
|
||||
if !ok {
|
||||
t.Fatalf("reasoning_price missing or wrong type: %#v", e2["reasoning_price"])
|
||||
}
|
||||
if rp != 6e-6 {
|
||||
t.Errorf("reasoning_price = %v, want 6e-6 (USD per TOKEN)", rp)
|
||||
}
|
||||
// A malformed rate must be rejected, not silently ignored: an operator who
|
||||
// typed "6 USD" expects either that value or an error, never a rule that
|
||||
// looks applied and bills nothing.
|
||||
p3 := &config.BillingProfile{ID: "p", Rules: []config.BillingRule{{
|
||||
URL: "https://up.example/v1", Mode: "token",
|
||||
Models: map[string]config.BillingToken{
|
||||
"m": {Prompt: "1.00", Completion: "2.00", Reasoning: "six dollars"},
|
||||
},
|
||||
}}}
|
||||
if _, err := Compile(p3, sources); err == nil {
|
||||
t.Error("a non-numeric reasoning rate must fail the compile")
|
||||
}
|
||||
}
|
||||
|
||||
// TestLongContextStepCompiles pins the long-context step's two accepted forms
|
||||
// and its rejection of a step that would silently do nothing.
|
||||
//
|
||||
// Providers publish both shapes — DeepSeek says "2x past 128k", OpenAI quotes
|
||||
// an absolute "$3.00/M past 200k" — so both must compile. Neither may be
|
||||
// missing: a step with no rate is a rule that LOOKS like it handles long
|
||||
// context and does not, which under-bills without any visible sign.
|
||||
func TestLongContextStepCompiles(t *testing.T) {
|
||||
sources := []config.Source{{Name: "up", BaseURL: "https://up.example/v1"}}
|
||||
|
||||
compile := func(lc config.BillingLongContext) (map[string]interface{}, error) {
|
||||
p := &config.BillingProfile{ID: "p", Rules: []config.BillingRule{{
|
||||
URL: "https://up.example/v1", Mode: "token",
|
||||
Models: map[string]config.BillingToken{
|
||||
"m": {Prompt: "1.00", Completion: "2.00", LongContext: &lc},
|
||||
},
|
||||
}}}
|
||||
prices, err := Compile(p, sources)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
e := prices["models"].(map[string]interface{})["m"].(map[string]interface{})
|
||||
return e["long_context"].(map[string]interface{}), nil
|
||||
}
|
||||
|
||||
// Absolute rate (OpenAI's published shape).
|
||||
lc1, err := compile(config.BillingLongContext{Above: 200000, Prompt: "3.00"})
|
||||
if err != nil {
|
||||
t.Fatalf("absolute long_context rate rejected: %v", err)
|
||||
}
|
||||
if lc1["above"] != 200000 || lc1["prompt"] != 3e-6 {
|
||||
t.Errorf("absolute step = %#v, want above=200000 prompt=3e-6", lc1)
|
||||
}
|
||||
|
||||
// Multiplier (DeepSeek's shape): 2x of the 1.00/M base.
|
||||
lc2, err := compile(config.BillingLongContext{Above: 128000, Multiplier: 2})
|
||||
if err != nil {
|
||||
t.Fatalf("multiplier long_context rejected: %v", err)
|
||||
}
|
||||
if lc2["prompt"] != 2e-6 {
|
||||
t.Errorf("multiplier step = %#v, want prompt=2e-6", lc2)
|
||||
}
|
||||
|
||||
// Both given: the absolute rate wins, because that is the number the
|
||||
// provider quotes. Multiplying instead would produce a figure the operator
|
||||
// never wrote.
|
||||
lc3, err := compile(config.BillingLongContext{Above: 128000, Prompt: "5.00", Multiplier: 2})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if lc3["prompt"] != 5e-6 {
|
||||
t.Errorf("both given = %#v, want the quoted absolute rate 5e-6", lc3)
|
||||
}
|
||||
|
||||
// Neither: a no-op step must fail, not compile into a rule that silently
|
||||
// bills long context at the base rate.
|
||||
if _, err := compile(config.BillingLongContext{Above: 128000}); err == nil {
|
||||
t.Error("a step with neither prompt nor multiplier must be rejected")
|
||||
}
|
||||
// A zero threshold would make every request "long".
|
||||
if _, err := compile(config.BillingLongContext{Above: 0, Multiplier: 2}); err == nil {
|
||||
t.Error("above=0 must be rejected")
|
||||
}
|
||||
}
|
||||
@ -7,7 +7,6 @@ import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@ -30,15 +29,6 @@ type Config struct {
|
||||
Auto []ModelScope `yaml:"auto,omitempty"` // AUTO 调度链规则（WebUI 优先级页编辑，chat）
|
||||
AutoImage []ModelScope `yaml:"auto_image,omitempty"` // AUTO 生图调度链规则（WebUI 优先级页·生图）
|
||||
Keys []GWKey `yaml:"keys,omitempty"` // 网关密钥（WebUI 密钥页管理）
|
||||
// BillingDSL declares per-URL pricing profiles for the billing plugin.
|
||||
//
|
||||
// Declarative, because hand-writing the prices JSON was error-prone in ways
|
||||
// that looked successful: the first PUT went out without the required
|
||||
// `prices` wrapper and silently REPLACED the accumulated totals with the
|
||||
// price table; the second used a peak-window shape the plugin does not read,
|
||||
// so peak traffic was billed at off-peak rates — no error, the bill just
|
||||
// quietly halved. A schema-checked config file catches both at load.
|
||||
BillingDSL *BillingDSL `yaml:"billing,omitempty"`
|
||||
// PluginVisibility overrides the audience a plugin declared for itself,
|
||||
// keyed by plugin name: "admin" hides a plugin (and its state) from user
|
||||
// keys, "all" widens one that shipped admin-only, "user,admin" names roles
|
||||
@ -95,7 +85,14 @@ type Source struct {
|
||||
// ProxyURL routes this source's HTTP(S) traffic through an explicit proxy
|
||||
// (e.g. http://127.0.0.1:7890). Empty = direct connection. Needed for
|
||||
// upstreams behind geo/IP blocks that only respond through a proxy.
|
||||
ProxyURL string `yaml:"proxy_url" json:"proxy_url,omitempty"`
|
||||
ProxyURL string `yaml:"proxy_url" json:"proxy_url,omitempty"`
|
||||
// Retry is an OPTIONAL upstream retry policy. It exists for upstreams that
|
||||
// answer a business-level rejection with a status code the kernel would
|
||||
// otherwise surface to the client — e.g. a per-account concurrency cap that
|
||||
// clears by itself after tens of seconds. The kernel stays vendor-agnostic:
|
||||
// the matcher is pure DATA (status list + response-body substrings), so no
|
||||
// provider name ever appears in core.
|
||||
Retry *RetryPolicy `yaml:"retry" json:"retry,omitempty"`
|
||||
Meta map[string]interface{} `yaml:"meta" json:"meta,omitempty"`
|
||||
Temperature float64 `yaml:"temperature" json:"temperature,omitempty"`
|
||||
MaxTokens int `yaml:"max_tokens" json:"max_tokens,omitempty"`
|
||||
@ -105,6 +102,63 @@ type Source struct {
|
||||
RPM int `yaml:"rpm" json:"rpm,omitempty"` // optional requests-per-minute cap (0 = unlimited)
|
||||
}
|
||||
|
||||
// RetryPolicy describes when a failed upstream attempt may be retried BEFORE
|
||||
// anything is returned to the client. All conditions are data:
|
||||
//
|
||||
// OnStatus - retry only when the HTTP status is one of these
|
||||
// OnBodyContains - ... AND the response body contains one of these substrings
|
||||
// (empty = status alone decides)
|
||||
// Delay - wait between attempts ("10s" etc.); parsed on demand
|
||||
// MaxAttempts - TOTAL attempts including the first (1 or 0 = no retry)
|
||||
//
|
||||
// Scope: the retry happens inside one gateway request, so a stream is only
|
||||
// retried before its first chunk reaches the client (the kernel already holds
|
||||
// the first chunk back), and a non-stream call is retried as a whole.
|
||||
type RetryPolicy struct {
|
||||
OnStatus []int `yaml:"on_status,omitempty" json:"on_status,omitempty"`
|
||||
OnBodyContains []string `yaml:"on_body_contains,omitempty" json:"on_body_contains,omitempty"`
|
||||
Delay string `yaml:"delay,omitempty" json:"delay,omitempty"`
|
||||
MaxAttempts int `yaml:"max_attempts,omitempty" json:"max_attempts,omitempty"`
|
||||
}
|
||||
|
||||
// ShouldRetry reports whether one failed attempt matches this policy.
|
||||
func (r *RetryPolicy) ShouldRetry(status int, body string) bool {
|
||||
if r == nil || r.MaxAttempts <= 1 {
|
||||
return false
|
||||
}
|
||||
statusOK := len(r.OnStatus) == 0
|
||||
for _, s := range r.OnStatus {
|
||||
if s == status {
|
||||
statusOK = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !statusOK {
|
||||
return false
|
||||
}
|
||||
if len(r.OnBodyContains) == 0 {
|
||||
return true
|
||||
}
|
||||
for _, sub := range r.OnBodyContains {
|
||||
if sub != "" && strings.Contains(body, sub) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// DelayDuration is the configured pause between attempts (default 10s).
|
||||
func (r *RetryPolicy) DelayDuration() time.Duration {
|
||||
if r == nil || r.Delay == "" {
|
||||
return 10 * time.Second
|
||||
}
|
||||
d, err := time.ParseDuration(r.Delay)
|
||||
if err != nil || d < 0 {
|
||||
return 10 * time.Second
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
// Load reads and validates a config file. When the file does not exist yet a
|
||||
// default config is generated at that path first (first-run bootstrap), so a
|
||||
// fresh binary just works: `llmsproxy -config /path/to/config.yaml`.
|
||||
@ -423,182 +477,6 @@ type GWKey struct {
|
||||
// chain" so a user can clear an override back to the global default.
|
||||
func (k GWKey) HasOwnAuto() bool { return len(k.Auto) > 0 }
|
||||
|
||||
// BillingDSL holds declarative per-URL pricing profiles for the billing plugin.
|
||||
//
|
||||
// Profiles are the "let the user pick" axis: the same upstream URL can be
|
||||
// declared under multiple profiles, and the operator chooses which is active.
|
||||
// Switching recomputes and re-injects the billing plugin's prices table.
|
||||
type BillingDSL struct {
|
||||
// Active is the id of the profile in effect. Empty => the first profile.
|
||||
Active string `yaml:"active,omitempty" json:"active,omitempty"`
|
||||
Profiles []BillingProfile `yaml:"profiles,omitempty" json:"profiles,omitempty"`
|
||||
}
|
||||
|
||||
// BillingProfile is one named set of per-URL pricing rules.
|
||||
type BillingProfile struct {
|
||||
ID string `yaml:"id" json:"id"`
|
||||
Label string `yaml:"label,omitempty" json:"label,omitempty"`
|
||||
// Default applies when no rule matches: "free" or "unpriced".
|
||||
Default string `yaml:"default,omitempty" json:"default,omitempty"`
|
||||
Rules []BillingRule `yaml:"rules,omitempty" json:"rules,omitempty"`
|
||||
}
|
||||
|
||||
// BillingRule declares the pricing for everything under one URL.
|
||||
type BillingRule struct {
|
||||
// URL matches a source's base_url. "*" = any unmatched URL.
|
||||
URL string `yaml:"url" json:"url"`
|
||||
// Mode is one of: free, token, subscription, unpriced.
|
||||
Mode string `yaml:"mode" json:"mode"`
|
||||
Currency string `yaml:"currency,omitempty" json:"currency,omitempty"`
|
||||
// Subscription is the flat monthly cost (for mode=subscription).
|
||||
Subscription float64 `yaml:"subscription,omitempty" json:"subscription,omitempty"`
|
||||
// Models declares per-token prices when mode=token.
|
||||
Models map[string]BillingToken `yaml:"models,omitempty" json:"models,omitempty"`
|
||||
// Free is true when mode=free (kept for explicitness).
|
||||
// Peak applies to ALL token-priced requests under this URL.
|
||||
Peak *BillingPeak `yaml:"peak,omitempty" json:"peak,omitempty"`
|
||||
}
|
||||
|
||||
// BillingToken is the per-model price.
|
||||
type BillingToken struct {
|
||||
Prompt string `yaml:"prompt" json:"prompt"` // USD per million tokens, e.g. "0.15"
|
||||
Completion string `yaml:"completion" json:"completion"` // USD per million tokens
|
||||
CacheDiscount *float64 `yaml:"cache_discount,omitempty" json:"cache_discount,omitempty"`
|
||||
// Reasoning is the USD-per-million rate for thinking tokens, for the rare
|
||||
// provider that charges them differently from ordinary output. Omit it and
|
||||
// reasoning is billed at Completion — which is what OpenAI, DeepSeek and
|
||||
// Anthropic all do, and what every source on this deployment reports.
|
||||
//
|
||||
// It is a rate for a SUBSET of completion tokens, never an extra charge: the
|
||||
// billing plugin applies only the difference between this and Completion, so
|
||||
// setting it to a multiple of Completion reprices the thinking share rather
|
||||
// than billing it twice.
|
||||
Reasoning string `yaml:"reasoning,omitempty" json:"reasoning,omitempty"`
|
||||
// LongContext is the prompt-price step above a token threshold. Nil = one
|
||||
// flat rate, which is what a rule means when it says nothing.
|
||||
LongContext *BillingLongContext `yaml:"long_context,omitempty" json:"long_context,omitempty"`
|
||||
}
|
||||
|
||||
// BillingPeak declares peak-hour pricing.
|
||||
type BillingPeak struct {
|
||||
Multiplier float64 `yaml:"multiplier" json:"multiplier"`
|
||||
// Weekdays: 1=Mon..5=Fri (Lua os.date !%w, 0=Sun)
|
||||
Weekdays []int `yaml:"weekdays,omitempty" json:"weekdays,omitempty"`
|
||||
// Hours: each pair is [lo,hi] inclusive
|
||||
Hours [][2]int `yaml:"hours,omitempty" json:"hours,omitempty"`
|
||||
}
|
||||
|
||||
// BillingLongContext is a prompt-price step for long inputs.
|
||||
//
|
||||
// Every major provider charges the input tokens ABOVE a threshold at a higher
|
||||
// rate — DeepSeek doubles past 128k, OpenAI and Anthropic step up as well. A
|
||||
// flat prompt price therefore under-bills exactly the traffic an agent workload
|
||||
// generates, and it under-bills it silently: the number is plausible, the bill
|
||||
// adds up, and nothing says the rate was wrong.
|
||||
//
|
||||
// The step applies to the PROMPT only. Output and cache reads are billed at
|
||||
// their own rates regardless of input length, which is how providers publish
|
||||
// it — the higher rate is for the extra context you made the model read, not for
|
||||
// what it wrote back.
|
||||
type BillingLongContext struct {
|
||||
// Above is the token count the step starts at. A request with exactly this
|
||||
// many prompt tokens is still at the base rate; the step covers the tokens
|
||||
// BEYOND it. Inclusive-below matches how the providers document it.
|
||||
Above int `yaml:"above" json:"above"`
|
||||
// Prompt is the USD-per-million rate for the tokens above Above.
|
||||
Prompt string `yaml:"prompt" json:"prompt"`
|
||||
// Multiplier is an alternative to Prompt: a factor applied to the base rate
|
||||
// (DeepSeek's published step is exactly 2x). When both are set Prompt wins,
|
||||
// and setting neither is a validation error rather than a silent no-op.
|
||||
Multiplier float64 `yaml:"multiplier,omitempty" json:"multiplier,omitempty"`
|
||||
}
|
||||
|
||||
// Validate rejects a BillingDSL that could not work as written.
|
||||
func (b *BillingDSL) Validate() error {
|
||||
if b == nil {
|
||||
return nil
|
||||
}
|
||||
ids := map[string]bool{}
|
||||
for i := range b.Profiles {
|
||||
p := &b.Profiles[i]
|
||||
if p.ID == "" {
|
||||
return fmt.Errorf("billing profile [%d] has no id", i)
|
||||
}
|
||||
if ids[p.ID] {
|
||||
return fmt.Errorf("billing profile id %q appears more than once", p.ID)
|
||||
}
|
||||
ids[p.ID] = true
|
||||
if p.Default != "" && p.Default != "free" && p.Default != "unpriced" {
|
||||
return fmt.Errorf("profile %q: default must be free or unpriced, got %q", p.ID, p.Default)
|
||||
}
|
||||
for j := range p.Rules {
|
||||
r := &p.Rules[j]
|
||||
if r.URL == "" {
|
||||
return fmt.Errorf("profile %q rule [%d]: url is required", p.ID, j)
|
||||
}
|
||||
switch r.Mode {
|
||||
case "free", "token", "subscription", "unpriced":
|
||||
default:
|
||||
return fmt.Errorf("profile %q rule [%d]: mode must be free/token/subscription/unpriced, got %q", p.ID, j, r.Mode)
|
||||
}
|
||||
if r.Mode == "token" {
|
||||
for m, t := range r.Models {
|
||||
if t.Prompt == "" || t.Completion == "" {
|
||||
return fmt.Errorf("profile %q model %q: prompt and completion are required for token mode", p.ID, m)
|
||||
}
|
||||
if _, err := ParseFloatUSDPerM(t.Prompt); err != nil {
|
||||
return fmt.Errorf("profile %q model %q prompt: %w", p.ID, m, err)
|
||||
}
|
||||
if _, err := ParseFloatUSDPerM(t.Completion); err != nil {
|
||||
return fmt.Errorf("profile %q model %q completion: %w", p.ID, m, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
// Active must resolve to a profile id.
|
||||
if b.Active != "" && !ids[b.Active] {
|
||||
return fmt.Errorf("billing.active = %q but no profile has that id", b.Active)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Resolve returns the profile with the given id; empty id (or an id that does
|
||||
// not exist) falls back to the FIRST profile. A config with profiles but no
|
||||
// active marker should mean "use the one I wrote first" rather than "price
|
||||
// nothing", which is what a nil here would silently do.
|
||||
func (b *BillingDSL) Resolve(id string) *BillingProfile {
|
||||
if b == nil || len(b.Profiles) == 0 {
|
||||
return nil
|
||||
}
|
||||
if id != "" {
|
||||
for i := range b.Profiles {
|
||||
if b.Profiles[i].ID == id {
|
||||
return &b.Profiles[i]
|
||||
}
|
||||
}
|
||||
}
|
||||
return &b.Profiles[0]
|
||||
}
|
||||
|
||||
// parseFloatUSDPerM parses a USD-per-million price string. Returns the
|
||||
// per-single-token rate the plugin expects (price / 1e6).
|
||||
//
|
||||
// Kept as a string in the DSL rather than a float: "0.15" survives round-trips
|
||||
// through YAML editors, and a malformed value must fail VALIDATION (with the
|
||||
// file and field named) rather than parse as 0 and under-bill silently — which
|
||||
// is exactly the failure mode that motivated the DSL.
|
||||
func ParseFloatUSDPerM(s string) (float64, error) {
|
||||
f, err := strconv.ParseFloat(strings.TrimSpace(s), 64)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("not a number: %q", s)
|
||||
}
|
||||
if f < 0 {
|
||||
return 0, fmt.Errorf("negative price: %q", s)
|
||||
}
|
||||
return f / 1e6, nil
|
||||
}
|
||||
|
||||
// KeyQuota is retained only to carry a scope entry's caps through the admin
|
||||
// API. Quotas are per model, never per key: there is deliberately no key-wide
|
||||
// total, so exhausting one model's budget never blocks the others.
|
||||
|
||||
59
internal/config/retry_policy_test.go
Normal file
59
internal/config/retry_policy_test.go
Normal file
@ -0,0 +1,59 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ShouldRetry 是纯数据判据：状态码列表 + 响应体子串。它决定"这次失败值不值得
|
||||
// 在网关内部等一会儿重试"。厂商知识绝不能进内核，所以判据必须能用一个表格测清楚。
|
||||
func TestRetryPolicyShouldRetry(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
policy *RetryPolicy
|
||||
status int
|
||||
body string
|
||||
want bool
|
||||
}{
|
||||
{"nil policy", nil, 500, "anything", false},
|
||||
{"max_attempts=1 disables", &RetryPolicy{OnStatus: []int{400}, MaxAttempts: 1}, 400, "x", false},
|
||||
{"status matches, no body filter",
|
||||
&RetryPolicy{OnStatus: []int{429}, MaxAttempts: 3}, 429, "", true},
|
||||
{"status mismatch",
|
||||
&RetryPolicy{OnStatus: []int{429}, MaxAttempts: 3}, 400, "429", false},
|
||||
{"status + body match",
|
||||
&RetryPolicy{OnStatus: []int{400}, OnBodyContains: []string{"TM.00001041"}, MaxAttempts: 3},
|
||||
400, `{"error_code":"TM.00001041"}`, true},
|
||||
{"status matches but body does not",
|
||||
&RetryPolicy{OnStatus: []int{400}, OnBodyContains: []string{"TM.00001041"}, MaxAttempts: 3},
|
||||
400, `{"error_code":"TM.00001005"}`, false},
|
||||
{"empty substring never matches",
|
||||
&RetryPolicy{OnStatus: []int{400}, OnBodyContains: []string{""}, MaxAttempts: 3}, 400, "x", false},
|
||||
{"any of several body signals",
|
||||
&RetryPolicy{OnStatus: []int{400, 429}, OnBodyContains: []string{"TM.00001041", "TPM limit"}, MaxAttempts: 3},
|
||||
429, `{"message":"TPM limit exceeded"}`, true},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
if got := tc.policy.ShouldRetry(tc.status, tc.body); got != tc.want {
|
||||
t.Errorf("%s: ShouldRetry(%d, %q) = %v, want %v", tc.name, tc.status, tc.body, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRetryPolicyDelay(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
policy *RetryPolicy
|
||||
want time.Duration
|
||||
}{
|
||||
{"nil", nil, 10 * time.Second},
|
||||
{"explicit", &RetryPolicy{Delay: "250ms"}, 250 * time.Millisecond},
|
||||
{"garbage falls back", &RetryPolicy{Delay: "not-a-duration"}, 10 * time.Second},
|
||||
{"negative falls back", &RetryPolicy{Delay: "-5s"}, 10 * time.Second},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
if got := tc.policy.DelayDuration(); got != tc.want {
|
||||
t.Errorf("%s: DelayDuration() = %v, want %v", tc.name, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -16,7 +16,6 @@ import (
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/billing"
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/lua"
|
||||
"llmsproxy/internal/provider"
|
||||
@ -110,7 +109,6 @@ func NewFromConfig(cfg *config.Config) (*Core, error) {
|
||||
log.Printf("[core] plugin %v %v loaded (hooks=%v ui=%v)",
|
||||
p["name"], p["version"], p["hooks"], p["ui"])
|
||||
}
|
||||
c.applyBillingDSL()
|
||||
c.store = config.NewStore(cfg.RuntimeFile)
|
||||
// Share one box between the runtime store and config.yaml so a single
|
||||
// master.key seals both files. config.Load left the config holding
|
||||
@ -1010,63 +1008,6 @@ func (c *Core) Close() {
|
||||
}
|
||||
}
|
||||
|
||||
// applyBillingDSL compiles the configured billing profile and injects it into
|
||||
// the billing plugin.
|
||||
//
|
||||
// Configured pricing replaces the hand-written JSON blob that used to be PUT
|
||||
// through the state API. That path was used twice in production and both times
|
||||
// it "succeeded" while being wrong: once without the required `prices` wrapper
|
||||
// (silently wiping the accumulated totals), once with a peak-window shape the
|
||||
// plugin does not read (peak traffic billed at off-peak rates, no error). A
|
||||
// typed config with load-time validation turns both mistakes into startup
|
||||
// errors naming the file and field.
|
||||
//
|
||||
// A DSL with no profile, or one that fails to compile, leaves the plugin's
|
||||
// prices untouched: billing keeps running on whatever it had, and the problem
|
||||
// is logged rather than becoming a startup failure — the gateway must forward
|
||||
// even when its cost accounting is misconfigured.
|
||||
func (c *Core) applyBillingDSL() {
|
||||
ps := c.plugins
|
||||
if ps == nil || ps.Count() == 0 {
|
||||
return
|
||||
}
|
||||
// The billing plugin may simply not be installed; a DSL without it is a
|
||||
// no-op, not an error.
|
||||
var have bool
|
||||
for _, row := range ps.List() {
|
||||
if row["name"] == "billing" && row["loaded"] == true {
|
||||
have = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !have {
|
||||
return
|
||||
}
|
||||
dsl := c.cfg.BillingDSL
|
||||
if dsl == nil || len(dsl.Profiles) == 0 {
|
||||
return
|
||||
}
|
||||
if err := dsl.Validate(); err != nil {
|
||||
log.Printf("[billing] config invalid, profile not applied: %v", err)
|
||||
return
|
||||
}
|
||||
profile := dsl.Resolve(dsl.Active)
|
||||
if profile == nil {
|
||||
log.Printf("[billing] no billing profile resolved (active=%q)", dsl.Active)
|
||||
return
|
||||
}
|
||||
prices, err := billing.Compile(profile, c.cfg.Sources)
|
||||
if err != nil {
|
||||
log.Printf("[billing] profile %q not applied: %v", profile.ID, err)
|
||||
return
|
||||
}
|
||||
if err := ps.SetState("billing", map[string]interface{}{"prices": prices}); err != nil {
|
||||
log.Printf("[billing] applying profile %q: %v", profile.ID, err)
|
||||
return
|
||||
}
|
||||
log.Printf("[billing] profile %q applied (%d rules, %d sources)", profile.ID, len(profile.Rules), len(c.cfg.Sources))
|
||||
}
|
||||
|
||||
// ValidateScopeQuotas checks every scope entry's caps before they are stored.
|
||||
// A typo in a period must be rejected at write time rather than silently
|
||||
// becoming a never-resetting budget — the opposite of what was typed.
|
||||
|
||||
@ -1,282 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
|
||||
"llmsproxy/internal/billing"
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// Billing rules API.
|
||||
//
|
||||
// GET /api/plugins/billing/rules the active profile's rules + all profiles
|
||||
// PUT /api/plugins/billing/rules replace the whole DSL and re-inject prices
|
||||
// POST /api/plugins/billing/rules { profile, rule } append one rule
|
||||
// DELETE /api/plugins/billing/rules { profile, url } remove one rule
|
||||
//
|
||||
// Why a separate endpoint instead of the generic plugin state: prices are
|
||||
// plugin state, but they are also a durable, reviewable CONFIGURATION. Routing
|
||||
// them through /state would let an admin key PUT arbitrary plugin state and
|
||||
// silently reset the accumulated accounting. Here the server owns the shape:
|
||||
// it validates the DSL, writes it back to config.yaml, recompiles, and hands
|
||||
// the plugin its prices table. The operator edits rules; the plugin keeps
|
||||
// numbers. The two are never mixed in one payload.
|
||||
//
|
||||
// Editing means "what the operator typed is what config.yaml holds". A
|
||||
// billing mistake found next week must be traceable to a reviewable file, not
|
||||
// to a blob in the plugin's state sidecar.
|
||||
func (g *Gateway) handleBillingRules(w http.ResponseWriter, r *http.Request) {
|
||||
if reqRole(r.Context()) != "admin" {
|
||||
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
|
||||
return
|
||||
}
|
||||
switch r.Method {
|
||||
case http.MethodGet:
|
||||
g.getBillingRules(w)
|
||||
case http.MethodPut:
|
||||
g.putBillingRules(w, r)
|
||||
case http.MethodPost:
|
||||
g.addBillingRule(w, r)
|
||||
case http.MethodDelete:
|
||||
g.delBillingRule(w, r)
|
||||
default:
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET/PUT/POST/DELETE")
|
||||
}
|
||||
}
|
||||
|
||||
// billingRulesPayload is the UI-facing view: the profiles as declared, plus
|
||||
// which one is active and the source URLs they can match — the editor needs
|
||||
// the URL list to offer suggestions, and an operator typing a URL that matches
|
||||
// no source is exactly the silent-no-pricing failure this feature exists to
|
||||
// remove.
|
||||
type billingRulesPayload struct {
|
||||
DSL *config.BillingDSL `json:"billing"`
|
||||
Active string `json:"active"`
|
||||
URLs []string `json:"urls"`
|
||||
Warnings []string `json:"warnings"`
|
||||
}
|
||||
|
||||
func (g *Gateway) billingRulesPayload() billingRulesPayload {
|
||||
out := billingRulesPayload{}
|
||||
cfg := g.core.Config()
|
||||
if cfg != nil && cfg.BillingDSL != nil {
|
||||
// Copy so the response cannot be mutated back through the pointer.
|
||||
cp := *cfg.BillingDSL
|
||||
out.DSL = &cp
|
||||
if p := cp.Resolve(""); p != nil {
|
||||
out.Active = p.ID
|
||||
}
|
||||
}
|
||||
for _, s := range g.sourceURLs() {
|
||||
out.URLs = append(out.URLs, s)
|
||||
}
|
||||
out.Warnings = g.billingRuleWarnings()
|
||||
return out
|
||||
}
|
||||
|
||||
func (g *Gateway) sourceURLs() []string {
|
||||
cfg := g.core.Config()
|
||||
if cfg == nil {
|
||||
return nil
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
var out []string
|
||||
for _, s := range cfg.Sources {
|
||||
if s.BaseURL == "" || seen[s.BaseURL] {
|
||||
continue
|
||||
}
|
||||
seen[s.BaseURL] = true
|
||||
out = append(out, s.BaseURL)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// billingRuleWarnings reports rules that match no configured source. This is
|
||||
// the silent killer of hand-written price tables: the rule parses, validates,
|
||||
// and prices nothing at all, so every request on that URL lands in unpriced.
|
||||
// Saying so out loud is the difference between a five-second fix and an
|
||||
// afternoon wondering why the bill is zero.
|
||||
func (g *Gateway) billingRuleWarnings() []string {
|
||||
cfg := g.core.Config()
|
||||
if cfg == nil || cfg.BillingDSL == nil {
|
||||
return nil
|
||||
}
|
||||
urls := map[string]bool{}
|
||||
for _, u := range g.sourceURLs() {
|
||||
urls[u] = true
|
||||
}
|
||||
var warns []string
|
||||
for _, p := range cfg.BillingDSL.Profiles {
|
||||
for _, r := range p.Rules {
|
||||
if r.URL == "*" || urls[r.URL] {
|
||||
continue
|
||||
}
|
||||
warns = append(warns, "profile "+p.ID+": rule url "+r.URL+" matches no configured source")
|
||||
}
|
||||
}
|
||||
return warns
|
||||
}
|
||||
|
||||
func (g *Gateway) getBillingRules(w http.ResponseWriter) {
|
||||
writeJSON(w, http.StatusOK, g.billingRulesPayload())
|
||||
}
|
||||
|
||||
func (g *Gateway) putBillingRules(w http.ResponseWriter, r *http.Request) {
|
||||
var body struct {
|
||||
Billing *config.BillingDSL `json:"billing"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
|
||||
return
|
||||
}
|
||||
if body.Billing == nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "billing is required")
|
||||
return
|
||||
}
|
||||
g.applyBillingDSLUpdate(w, body.Billing)
|
||||
}
|
||||
|
||||
func (g *Gateway) addBillingRule(w http.ResponseWriter, r *http.Request) {
|
||||
var body struct {
|
||||
Profile string `json:"profile"`
|
||||
Rule config.BillingRule `json:"rule"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
|
||||
return
|
||||
}
|
||||
cfg := g.core.Config()
|
||||
if cfg == nil || cfg.BillingDSL == nil || len(cfg.BillingDSL.Profiles) == 0 {
|
||||
writeError(w, http.StatusBadRequest, "no_profile", "no billing profiles configured; create one first")
|
||||
return
|
||||
}
|
||||
next := cloneBillingDSL(cfg.BillingDSL)
|
||||
idx := profileIndex(next, body.Profile)
|
||||
if idx < 0 {
|
||||
writeError(w, http.StatusBadRequest, "unknown_profile", "no such profile: "+body.Profile)
|
||||
return
|
||||
}
|
||||
p := &next.Profiles[idx]
|
||||
// Replace rather than append when the URL is already declared: two rules
|
||||
// for one URL would silently make the first unreachable (first match wins),
|
||||
// which is the exact ambiguity an editor should not be able to create.
|
||||
replaced := false
|
||||
for i := range p.Rules {
|
||||
if p.Rules[i].URL == body.Rule.URL {
|
||||
p.Rules[i] = body.Rule
|
||||
replaced = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !replaced {
|
||||
p.Rules = append(p.Rules, body.Rule)
|
||||
}
|
||||
g.applyBillingDSLUpdate(w, next)
|
||||
}
|
||||
|
||||
func (g *Gateway) delBillingRule(w http.ResponseWriter, r *http.Request) {
|
||||
var body struct {
|
||||
Profile string `json:"profile"`
|
||||
URL string `json:"url"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
|
||||
return
|
||||
}
|
||||
cfg := g.core.Config()
|
||||
if cfg == nil || cfg.BillingDSL == nil {
|
||||
writeError(w, http.StatusBadRequest, "no_profile", "no billing profiles configured")
|
||||
return
|
||||
}
|
||||
next := cloneBillingDSL(cfg.BillingDSL)
|
||||
idx := profileIndex(next, body.Profile)
|
||||
if idx < 0 {
|
||||
writeError(w, http.StatusBadRequest, "unknown_profile", "no such profile: "+body.Profile)
|
||||
return
|
||||
}
|
||||
p := &next.Profiles[idx]
|
||||
out := p.Rules[:0]
|
||||
found := false
|
||||
for _, rule := range p.Rules {
|
||||
if rule.URL == body.URL {
|
||||
found = true
|
||||
continue
|
||||
}
|
||||
out = append(out, rule)
|
||||
}
|
||||
if !found {
|
||||
writeError(w, http.StatusNotFound, "not_found", "no rule for url "+body.URL)
|
||||
return
|
||||
}
|
||||
p.Rules = out
|
||||
g.applyBillingDSLUpdate(w, next)
|
||||
}
|
||||
|
||||
// applyBillingDSLUpdate is the single write path: validate, persist to
|
||||
// config.yaml, recompile, inject, and only then report success. If the config
|
||||
// cannot be written the change is NOT applied in memory either — a rules editor
|
||||
// that says "saved" and loses the edit on the next restart is worse than one
|
||||
// that refuses.
|
||||
func (g *Gateway) applyBillingDSLUpdate(w http.ResponseWriter, next *config.BillingDSL) {
|
||||
if err := next.Validate(); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_rules", err.Error())
|
||||
return
|
||||
}
|
||||
cfg := g.core.Config()
|
||||
if cfg == nil {
|
||||
writeError(w, http.StatusInternalServerError, "no_config", "no config loaded")
|
||||
return
|
||||
}
|
||||
// Assign BEFORE saving. The previous order saved first and assigned after,
|
||||
// so every rule edit reported success while writing a config.yaml with no
|
||||
// billing section at all — the operator's edit vanished on restart and
|
||||
// nothing in the API response said so.
|
||||
prev := cfg.BillingDSL
|
||||
cfg.BillingDSL = next
|
||||
|
||||
// Compile before persisting: a profile that cannot be turned into prices
|
||||
// must not reach the config file, or the next restart fails to load the
|
||||
// very rules the editor just accepted.
|
||||
prices, err := billing.CompileOpts(next.Resolve(next.Active), cfg.Sources, true)
|
||||
if err != nil {
|
||||
cfg.BillingDSL = prev // no half-applied state
|
||||
writeError(w, http.StatusBadRequest, "compile_failed", err.Error())
|
||||
return
|
||||
}
|
||||
if err := cfg.Save(); err != nil {
|
||||
cfg.BillingDSL = prev
|
||||
writeError(w, http.StatusInternalServerError, "persist_failed", err.Error())
|
||||
return
|
||||
}
|
||||
ps := g.core.Plugins()
|
||||
if ps != nil {
|
||||
if err := ps.SetState("billing", map[string]interface{}{"prices": prices}); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "plugin_error", err.Error())
|
||||
return
|
||||
}
|
||||
}
|
||||
p := g.billingRulesPayload()
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{
|
||||
"ok": true, "billing": p.DSL, "active": p.Active, "warnings": p.Warnings,
|
||||
})
|
||||
}
|
||||
|
||||
func cloneBillingDSL(d *config.BillingDSL) *config.BillingDSL {
|
||||
out := &config.BillingDSL{Active: d.Active}
|
||||
out.Profiles = make([]config.BillingProfile, len(d.Profiles))
|
||||
copy(out.Profiles, d.Profiles)
|
||||
return out
|
||||
}
|
||||
|
||||
func profileIndex(d *config.BillingDSL, id string) int {
|
||||
if id == "" {
|
||||
return -1
|
||||
}
|
||||
for i, p := range d.Profiles {
|
||||
if p.ID == id {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
@ -1,281 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"sort"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// rulesGateway needs a real billing plugin (the API injects prices into it)
|
||||
// AND two sources whose base_urls the rules will match, so it cannot reuse
|
||||
// either existing helper: gatewayWithBilling has no sources, and
|
||||
// newTestGateway loads no plugins.
|
||||
func rulesGateway(t *testing.T) *Gateway {
|
||||
t.Helper()
|
||||
g := gatewayWithBilling(t)
|
||||
cfg := g.core.Config()
|
||||
cfg.Sources = []config.Source{
|
||||
{Name: "localzen", BaseURL: "https://free.example.com/v1"},
|
||||
{Name: "commandcode", BaseURL: "https://api.commandcode.ai/v1"},
|
||||
}
|
||||
return g
|
||||
}
|
||||
|
||||
func rulesDo(t *testing.T, g *Gateway, method, body string) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
return doReq(t, g, method, "/api/plugins/billing/rules", body)
|
||||
}
|
||||
|
||||
// TestBillingRulesCRUD covers the whole editor loop against the real HTTP
|
||||
// surface: read, add, replace-by-URL, delete. The replace-by-URL case is the
|
||||
// one worth pinning — two rules for one URL would make the first unreachable
|
||||
// (first match wins) without any error, which is exactly the kind of silent
|
||||
// ambiguity an editor must not be able to create.
|
||||
func TestBillingRulesCRUD(t *testing.T) {
|
||||
g := rulesGateway(t)
|
||||
seed := `{"billing":{"active":"p1","profiles":[{"id":"p1","rules":[
|
||||
{"url":"https://free.example.com/v1","mode":"free"}]}]}}`
|
||||
if rr := rulesDo(t, g, http.MethodPut, seed); rr.Code != http.StatusOK {
|
||||
t.Fatalf("seed rules = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
|
||||
// Add a token-priced rule for the second URL.
|
||||
add := `{"profile":"p1","rule":{"url":"https://api.commandcode.ai/v1","mode":"token",
|
||||
"models":{"deepseek-v4.1-flash":{"prompt":"0.15","completion":"0.60"}}}}`
|
||||
if rr := rulesDo(t, g, http.MethodPost, add); rr.Code != http.StatusOK {
|
||||
t.Fatalf("add rule = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var got billingRulesPayload
|
||||
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got.DSL.Profiles[0].Rules) != 2 {
|
||||
t.Fatalf("after add there are %d rules, want 2", len(got.DSL.Profiles[0].Rules))
|
||||
}
|
||||
|
||||
// Adding the SAME url again must replace, not append.
|
||||
if rr := rulesDo(t, g, http.MethodPost, `{"profile":"p1","rule":{"url":"https://api.commandcode.ai/v1","mode":"free"}}`); rr.Code != http.StatusOK {
|
||||
t.Fatalf("re-add same url = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if n := len(got.DSL.Profiles[0].Rules); n != 2 {
|
||||
t.Fatalf("re-adding an existing url appended instead of replacing: %d rules", n)
|
||||
}
|
||||
var mode string
|
||||
for _, r := range got.DSL.Profiles[0].Rules {
|
||||
if r.URL == "https://api.commandcode.ai/v1" {
|
||||
mode = r.Mode
|
||||
}
|
||||
}
|
||||
if mode != "free" {
|
||||
t.Errorf("re-added rule mode = %q, want free (the replacement must win)", mode)
|
||||
}
|
||||
|
||||
// Delete by URL.
|
||||
if rr := rulesDo(t, g, http.MethodDelete, `{"profile":"p1","url":"https://api.commandcode.ai/v1"}`); rr.Code != http.StatusOK {
|
||||
t.Fatalf("delete rule = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got.DSL.Profiles[0].Rules) != 1 {
|
||||
t.Errorf("after delete there are %d rules, want 1", len(got.DSL.Profiles[0].Rules))
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingRulesRejectUnknownURLWithWarning is the silent-failure guard: a
|
||||
// rule whose URL matches no source prices nothing, so every request on it
|
||||
// falls into unpriced. The API must say so instead of accepting it silently.
|
||||
func TestBillingRulesRejectUnknownURLWithWarning(t *testing.T) {
|
||||
g := rulesGateway(t)
|
||||
if rr := rulesDo(t, g, http.MethodPut, `{"billing":{"active":"p","profiles":[{"id":"p",
|
||||
"rules":[{"url":"https://typo.example.com/v1","mode":"free"}]}]}}`); rr.Code != http.StatusOK {
|
||||
t.Fatalf("seed = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var got billingRulesPayload
|
||||
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got.Warnings) == 0 {
|
||||
t.Fatal("a rule matching no configured source was accepted with no warning — " +
|
||||
"it will price nothing and every request will be recorded as unpriced")
|
||||
}
|
||||
if !strings.Contains(got.Warnings[0], "typo.example.com") {
|
||||
t.Errorf("warning does not name the offending url: %v", got.Warnings)
|
||||
}
|
||||
// The editor needs the real URL list to offer suggestions.
|
||||
if len(got.URLs) != 2 {
|
||||
t.Errorf("urls offered to the editor = %v, want the 2 configured base_urls", got.URLs)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingRulesInvalidIsRejectedNotPersisted checks the write path refuses
|
||||
// bad input and leaves the previous rules in place — an editor that clears the
|
||||
// table on a typo loses the price table.
|
||||
func TestBillingRulesInvalidIsRejectedNotPersisted(t *testing.T) {
|
||||
g := rulesGateway(t)
|
||||
seed := `{"billing":{"active":"p","profiles":[{"id":"p","rules":[
|
||||
{"url":"https://free.example.com/v1","mode":"free"}]}]}}`
|
||||
if rr := rulesDo(t, g, http.MethodPut, seed); rr.Code != http.StatusOK {
|
||||
t.Fatalf("seed = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
// An unknown mode must be refused.
|
||||
if rr := rulesDo(t, g, http.MethodPost, `{"profile":"p","rule":{"url":"https://x/v1","mode":"banana"}}`); rr.Code != http.StatusBadRequest {
|
||||
t.Errorf("invalid mode accepted with %d, want 400", rr.Code)
|
||||
}
|
||||
var got billingRulesPayload
|
||||
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got.DSL.Profiles[0].Rules) != 1 {
|
||||
t.Errorf("a rejected rule was persisted anyway: %d rules", len(got.DSL.Profiles[0].Rules))
|
||||
}
|
||||
// An unparseable price must be refused too, not silently priced at zero.
|
||||
if rr := rulesDo(t, g, http.MethodPost, `{"profile":"p","rule":{"url":"https://y/v1","mode":"token",
|
||||
"models":{"m":{"prompt":"free","completion":"0.6"}}}}`); rr.Code != http.StatusBadRequest {
|
||||
t.Errorf("non-numeric price accepted with %d, want 400", rr.Code)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingRulesPersistAndReachThePlugin is the point of the whole
|
||||
// endpoint, and mutation verification showed the earlier CRUD tests missed it
|
||||
// twice: dropping cfg.Save() and dropping the SetState injection both left
|
||||
// every test green. Both failures are invisible in a GET — the response echoes
|
||||
// the in-memory rules either way — so this test checks the two consequences
|
||||
// that actually matter:
|
||||
//
|
||||
// 1. persistence: the rule must be in the config file on disk, because an
|
||||
// edit that evaporates on restart is a lie the UI told the operator;
|
||||
// 2. effect: the billing plugin must now PRICE the source (its published
|
||||
// prices table has a non-zero entry), because a rule that is stored but
|
||||
// never injected prices nothing at all.
|
||||
func TestBillingRulesPersistAndReachThePlugin(t *testing.T) {
|
||||
g := rulesGateway(t)
|
||||
cfg := g.core.Config()
|
||||
|
||||
seed := `{"billing":{"active":"p","profiles":[{"id":"p","rules":[
|
||||
{"url":"https://free.example.com/v1","mode":"free"},
|
||||
{"url":"https://api.commandcode.ai/v1","mode":"token",
|
||||
"models":{"deepseek-v4.1-flash":{"prompt":"0.15","completion":"0.60"}}}]}]}}`
|
||||
if rr := rulesDo(t, g, http.MethodPut, seed); rr.Code != http.StatusOK {
|
||||
t.Fatalf("seed = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
|
||||
// 1. persisted to the file the process loaded from.
|
||||
raw, err := os.ReadFile(cfg.Path)
|
||||
if err != nil {
|
||||
t.Fatalf("read config: %v", err)
|
||||
}
|
||||
if !strings.Contains(string(raw), "commandcode.ai") {
|
||||
t.Errorf("the rule is not in config.yaml on disk — an edit that does not "+
|
||||
"survive a restart:\n%s", raw)
|
||||
}
|
||||
reloaded, err := config.Load(cfg.Path)
|
||||
if err != nil {
|
||||
t.Fatalf("reload saved config: %v", err)
|
||||
}
|
||||
if reloaded.BillingDSL == nil || len(reloaded.BillingDSL.Profiles) == 0 {
|
||||
t.Fatalf("saved config has no billing DSL: %s", raw)
|
||||
}
|
||||
// A round-trip through YAML must not lose the price strings.
|
||||
found := false
|
||||
for _, r := range reloaded.BillingDSL.Profiles[0].Rules {
|
||||
if strings.Contains(r.URL, "commandcode") {
|
||||
found = true
|
||||
if m, ok := r.Models["deepseek-v4.1-flash"]; !ok || m.Prompt != "0.15" {
|
||||
t.Errorf("price did not survive the YAML round-trip: %+v", m)
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Errorf("the token rule is missing after reload:\n%s", raw)
|
||||
}
|
||||
|
||||
// 2. reached the plugin. Prices live in a separate Lua field from `state`,
|
||||
// and the GET /state endpoint returns the published state together with
|
||||
// the prices sidecar — which is exactly what the plugin's own UI reads.
|
||||
state := g.core.Plugins().State("billing")
|
||||
if state == nil {
|
||||
t.Fatalf("billing plugin published no state")
|
||||
}
|
||||
rr := doReq(t, g, http.MethodGet, "/api/plugins/billing/state", "")
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("GET state = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var envelope struct {
|
||||
Prices map[string]interface{} `json:"prices"`
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &envelope); err != nil {
|
||||
t.Fatalf("decode state envelope: %v", err)
|
||||
}
|
||||
prices := envelope.Prices
|
||||
if prices == nil {
|
||||
t.Fatalf("state payload carries no prices")
|
||||
}
|
||||
sources, ok := prices["sources"].(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("prices has no sources map: %#v", prices)
|
||||
}
|
||||
cc, ok := sources["commandcode"]
|
||||
if !ok {
|
||||
t.Fatalf("the rule was saved but never injected: prices has %v, "+
|
||||
"no commandcode entry. Every request on it stays unpriced.", keysOf(sources))
|
||||
}
|
||||
if cc == nil {
|
||||
t.Error("commandcode price entry is nil")
|
||||
}
|
||||
// The free rule must price localzen, and the two must not be conflated.
|
||||
if _, ok := sources["localzen"]; !ok {
|
||||
t.Errorf("the free rule did not reach the plugin: %v", keysOf(sources))
|
||||
}
|
||||
// The injected price must be per-token, i.e. 0.15 USD/M => 1.5e-7.
|
||||
// Asserting the exact magnitude catches a unit error (per-million vs
|
||||
// per-token), which would be off by a factor of a million and still look
|
||||
// like "a number".
|
||||
models, ok := cc.(map[string]interface{})["models"].(map[string]interface{})
|
||||
if ok {
|
||||
if m, ok := models["deepseek-v4.1-flash"].(map[string]interface{}); ok {
|
||||
p, _ := m["prompt"].(float64)
|
||||
if p <= 0 || p > 1e-6 {
|
||||
t.Errorf("injected prompt price = %v, want per-token (~1.5e-7); "+
|
||||
"a per-million value here would over-bill by 1e6x", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func keysOf(m map[string]interface{}) []string {
|
||||
out := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
out = append(out, k)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// TestBillingRulesRequireAdmin keeps price rewriting behind the admin role,
|
||||
// like every other plugin write: prices are configuration, not user data.
|
||||
func TestBillingRulesRequireAdmin(t *testing.T) {
|
||||
g := rulesGateway(t)
|
||||
rec, err := g.core.CreateKey("viewer", "user", nil, "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
userKey := rec.Key
|
||||
for _, m := range []string{http.MethodPut, http.MethodPost, http.MethodDelete} {
|
||||
req, _ := http.NewRequest(m, "/api/plugins/billing/rules", strings.NewReader(`{}`))
|
||||
req.Header.Set("Authorization", "Bearer "+userKey)
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
if rr := newRecorderFor(t, g, req); rr.Code != http.StatusForbidden {
|
||||
t.Errorf("user %s rules = %d, want 403", m, rr.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -126,7 +126,12 @@ func TestRealMutants(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "nested metadata no longer read",
|
||||
from: "\tcopyFields(m.Metadata) // nested first",
|
||||
// 锚点必须与 discover.go 逐字节一致：此前这里是 6 个空格，源码已
|
||||
// 变成 2 个空格，于是这条变异永远"找不到锚点"——变异campaign 每次
|
||||
// 都以 anchor not found 失败，而它本该验证"嵌套 metadata 被丢掉"能被
|
||||
// 测试抓住。锚点漂移 = 这条变异从未真正跑过（和 issue #1 里
|
||||
// `cc["models"]` 恒 false 的测试跳过是同一种失效：断言写了但没执行）。
|
||||
from: "\tcopyFields(m.Metadata) // nested first",
|
||||
to: "\tcopyFields(json.RawMessage(nil)) // mutant: nested metadata dropped",
|
||||
},
|
||||
{
|
||||
|
||||
@ -247,11 +247,6 @@ func (g *Gateway) routes(w http.ResponseWriter, r *http.Request) {
|
||||
g.handleSourceTemplatesAPI(w, r)
|
||||
case r.URL.Path == "/api/ui-inject" || strings.HasPrefix(r.URL.Path, "/api/ui-inject/"):
|
||||
g.handlePluginUI(w, r)
|
||||
case r.URL.Path == "/api/plugins/billing/rules":
|
||||
// Checked BEFORE the generic /api/plugins/ prefix: the plugin router
|
||||
// treats anything after the name as a plugin sub-resource and would
|
||||
// 404 on "rules" instead of reaching the billing editor.
|
||||
g.handleBillingRules(w, r)
|
||||
case strings.HasPrefix(r.URL.Path, pluginHTTPPrefix):
|
||||
// Before the generic plugin router: these paths belong to a PLUGIN, not to
|
||||
// the plugin manager, and "http" is a valid sub-resource name otherwise.
|
||||
|
||||
15
internal/gateway/test_helpers_test.go
Normal file
15
internal/gateway/test_helpers_test.go
Normal file
@ -0,0 +1,15 @@
|
||||
package gateway
|
||||
|
||||
import "sort"
|
||||
|
||||
// keysOf returns the sorted keys of a map, for readable failure messages.
|
||||
// (It used to live in the deleted billing-rules test file; other gateway tests
|
||||
// still use it.)
|
||||
func keysOf(m map[string]interface{}) []string {
|
||||
out := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
out = append(out, k)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
227
internal/lua/billing_rules_test.go
Normal file
227
internal/lua/billing_rules_test.go
Normal file
@ -0,0 +1,227 @@
|
||||
package lua
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// billing 的规则**完全在插件内**：以源名为键、由插件自己编译、自己写
|
||||
// `<plugin_dir>/billing.json`。内核没有 billing 代码，也不提供 billing 端点；
|
||||
// 这些判据直接打插件的导出（对应插件自己的 /api/plugins/http/billing/rules）。
|
||||
func putRules(t *testing.T, ps *Plugins, rules map[string]interface{}) map[string]interface{} {
|
||||
t.Helper()
|
||||
out, err := ps.Call("billing", "rules_put", map[string]interface{}{"billing": rules})
|
||||
if err != nil {
|
||||
t.Fatalf("rules_put: %v", err)
|
||||
}
|
||||
m, _ := out.(map[string]interface{})
|
||||
return m
|
||||
}
|
||||
|
||||
func pricesOf(t *testing.T, ps *Plugins) map[string]interface{} {
|
||||
t.Helper()
|
||||
raw := ps.Prices("billing")
|
||||
p, ok := raw.(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("billing has no prices: %#v", raw)
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
func oneProfile(rules []interface{}, def interface{}) map[string]interface{} {
|
||||
p := map[string]interface{}{"id": "p", "rules": rules}
|
||||
if def != nil {
|
||||
p["default"] = def
|
||||
}
|
||||
return map[string]interface{}{"active": "p", "profiles": []interface{}{p}}
|
||||
}
|
||||
|
||||
// 插件自己把规则写到 plugin_dir/billing.json，且读取回来一致。
|
||||
func TestBillingRulesPersistInPluginOwnedFile(t *testing.T) {
|
||||
ps, pdir := billingVM(t)
|
||||
rules := oneProfile([]interface{}{
|
||||
map[string]interface{}{"source": "localzen", "mode": "free"},
|
||||
}, "unpriced")
|
||||
if m := putRules(t, ps, rules); m["ok"] != true {
|
||||
t.Fatalf("rules_put 失败: %#v", m)
|
||||
}
|
||||
path := filepath.Join(pdir, "billing.json")
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("插件没有自己写规则文件 %s: %v", path, err)
|
||||
}
|
||||
var onDisk map[string]interface{}
|
||||
if err := json.Unmarshal(raw, &onDisk); err != nil {
|
||||
t.Fatalf("规则文件不是 JSON: %v\n%s", err, raw)
|
||||
}
|
||||
if !strings.Contains(string(raw), "localzen") {
|
||||
t.Fatalf("规则文件内容不对: %s", raw)
|
||||
}
|
||||
// 重新加载插件（模拟重启）：规则应从文件恢复。
|
||||
ps2 := reloadBilling(t, pdir)
|
||||
got, err := ps2.Call("billing", "rules_get", map[string]interface{}{})
|
||||
if err != nil {
|
||||
t.Fatalf("rules_get: %v", err)
|
||||
}
|
||||
b, _ := json.Marshal(got)
|
||||
if !strings.Contains(string(b), "localzen") {
|
||||
t.Fatalf("重启后没有从 billing.json 恢复规则: %s", b)
|
||||
}
|
||||
// 编译结果也恢复了：free 源条目存在。
|
||||
if p := ps2.Prices("billing").(map[string]interface{}); p != nil {
|
||||
if _, ok := p["sources"].(map[string]interface{})["localzen"]; !ok {
|
||||
t.Errorf("恢复后价格表缺 localzen: %#v", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func reloadBilling(t *testing.T, pdir string) *Plugins {
|
||||
t.Helper()
|
||||
vm := NewVM(filepath.Join(t.TempDir(), "adapters"))
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatalf("vm: %v", err)
|
||||
}
|
||||
t.Cleanup(vm.Stop)
|
||||
ps := NewPlugins(vm, pdir)
|
||||
if err := ps.LoadDir(); err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
return ps
|
||||
}
|
||||
|
||||
// 编译语义：free/subscription 写"有意零价"，token 规则**不写源条目**（未列价
|
||||
// 的模型要可见地进入 unpriced），通配符只发布模型价。
|
||||
func TestBillingRulesCompileSemantics(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
rules := oneProfile([]interface{}{
|
||||
map[string]interface{}{"source": "free-src", "mode": "free"},
|
||||
map[string]interface{}{"source": "sub-src", "mode": "subscription", "subscription": 20},
|
||||
map[string]interface{}{"source": "tok-src", "mode": "token", "models": map[string]interface{}{
|
||||
"m1": map[string]interface{}{"prompt": "0.15", "completion": "0.60"},
|
||||
}},
|
||||
map[string]interface{}{"url": "*", "mode": "token", "models": map[string]interface{}{
|
||||
"m2": map[string]interface{}{"prompt": "1", "completion": "2"},
|
||||
}},
|
||||
}, nil)
|
||||
if m := putRules(t, ps, rules); m["ok"] != true {
|
||||
t.Fatalf("rules_put: %#v", m)
|
||||
}
|
||||
p := pricesOf(t, ps)
|
||||
srcs := p["sources"].(map[string]interface{})
|
||||
for _, n := range []string{"free-src", "sub-src"} {
|
||||
e, ok := srcs[n].(map[string]interface{})
|
||||
if !ok || e["prompt"] != 0.0 || e["completion"] != 0.0 {
|
||||
t.Errorf("%s 应写有意零价: %#v", n, srcs[n])
|
||||
}
|
||||
}
|
||||
if _, ok := srcs["tok-src"]; ok {
|
||||
t.Errorf("token 规则不得产生源条目: %#v", srcs["tok-src"])
|
||||
}
|
||||
if subs, ok := p["subscriptions"].(map[string]interface{})["sub-src"].(map[string]interface{}); !ok || subs["monthly"] != 20.0 {
|
||||
t.Errorf("subscription 固定费丢失: %#v", p["subscriptions"])
|
||||
}
|
||||
models := p["models"].(map[string]interface{})
|
||||
if models["m1"].(map[string]interface{})["prompt"] != 1.5e-7 {
|
||||
t.Errorf("m1 prompt = %#v, want 1.5e-7", models["m1"])
|
||||
}
|
||||
if _, ok := models["m2"]; !ok {
|
||||
t.Error("通配符规则的模型价丢失")
|
||||
}
|
||||
}
|
||||
|
||||
// 端到端：token 规则 + 1e6/1e6 tokens @ 0.15/0.60 USD/M = 0.75，
|
||||
// 且 unpriced 计数不涨（模型本身被标价）。
|
||||
func TestBillingRulesTokenCostEndToEnd(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
rules := oneProfile([]interface{}{
|
||||
map[string]interface{}{"source": "tok-src", "mode": "token", "models": map[string]interface{}{
|
||||
"m": map[string]interface{}{"prompt": "0.15", "completion": "0.60"},
|
||||
}},
|
||||
}, nil)
|
||||
if m := putRules(t, ps, rules); m["ok"] != true {
|
||||
t.Fatalf("rules_put: %#v", m)
|
||||
}
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m", "source": "tok-src", "key": "***p", "ok": true,
|
||||
"prompt_tokens": 1000000.0, "completion_tokens": 1000000.0, "time": 1750000000000,
|
||||
})
|
||||
st := stateOf(t, ps)
|
||||
approx(t, "cost", st["total"].(map[string]interface{})["cost"].(float64), 0.75)
|
||||
if got := st["unpriced_reqs"].(float64); got != 0 {
|
||||
t.Errorf("unpriced_reqs = %v, want 0", got)
|
||||
}
|
||||
// 同源未列价的模型必须落入 unpriced（可见），而不是静默计 0。
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "unlisted", "source": "tok-src", "key": "***p", "ok": true,
|
||||
"prompt_tokens": 100.0, "completion_tokens": 1.0, "time": 1750000000000,
|
||||
})
|
||||
if got := stateOf(t, ps)["unpriced_reqs"].(float64); got != 1 {
|
||||
t.Errorf("未列价模型应计入 unpriced，got %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// 非法规则必须被拒绝，且**不覆盖**已有文件（保存失败不能丢旧配置）。
|
||||
func TestBillingRulesRejectInvalidWithoutClobberingFile(t *testing.T) {
|
||||
ps, pdir := billingVM(t)
|
||||
good := oneProfile([]interface{}{
|
||||
map[string]interface{}{"source": "s1", "mode": "free"},
|
||||
}, nil)
|
||||
if m := putRules(t, ps, good); m["ok"] != true {
|
||||
t.Fatalf("seed: %#v", m)
|
||||
}
|
||||
path := filepath.Join(pdir, "billing.json")
|
||||
before, _ := os.ReadFile(path)
|
||||
|
||||
bad := oneProfile([]interface{}{
|
||||
map[string]interface{}{"source": "s1", "mode": "nope"},
|
||||
}, nil)
|
||||
out, err := ps.Call("billing", "rules_put", map[string]interface{}{"billing": bad})
|
||||
if err != nil {
|
||||
t.Fatalf("call: %v", err)
|
||||
}
|
||||
m := out.(map[string]interface{})
|
||||
if m["error"] == nil {
|
||||
t.Fatalf("非法 mode 应被拒绝: %#v", m)
|
||||
}
|
||||
after, _ := os.ReadFile(path)
|
||||
if string(before) != string(after) {
|
||||
t.Errorf("被拒绝的保存改动了规则文件:\nbefore=%s\nafter=%s", before, after)
|
||||
}
|
||||
}
|
||||
|
||||
// reasoning 缺席 = 按输出价计；显式 long_context/peak 必须编译成插件读取的形状。
|
||||
func TestBillingRulesOptionalPricesCompile(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
rules := oneProfile([]interface{}{
|
||||
map[string]interface{}{"source": "s1", "mode": "token",
|
||||
"models": map[string]interface{}{
|
||||
"plain": map[string]interface{}{"prompt": "1", "completion": "2"},
|
||||
"reasoning": map[string]interface{}{"prompt": "1", "completion": "2", "reasoning": "3"},
|
||||
"long": map[string]interface{}{"prompt": "1", "completion": "2",
|
||||
"long_context": map[string]interface{}{"above": 128000, "multiplier": 2}},
|
||||
},
|
||||
"peak": map[string]interface{}{"multiplier": 2, "hours": []interface{}{[]interface{}{1, 4}}},
|
||||
},
|
||||
}, nil)
|
||||
if m := putRules(t, ps, rules); m["ok"] != true {
|
||||
t.Fatalf("rules_put: %#v", m)
|
||||
}
|
||||
models := pricesOf(t, ps)["models"].(map[string]interface{})
|
||||
if _, present := models["plain"].(map[string]interface{})["reasoning_price"]; present {
|
||||
t.Error("未配置 reasoning 不应出现 reasoning_price（0 会被当成免费）")
|
||||
}
|
||||
if got := models["reasoning"].(map[string]interface{})["reasoning_price"]; got != 3e-6 {
|
||||
t.Errorf("reasoning_price = %v, want 3e-6", got)
|
||||
}
|
||||
lc := models["long"].(map[string]interface{})["long_context"].(map[string]interface{})
|
||||
if lc["above"] != 128000.0 || lc["prompt"] != 2e-6 {
|
||||
t.Errorf("long_context = %#v", lc)
|
||||
}
|
||||
pk := models["long"].(map[string]interface{})["peak"].(map[string]interface{})
|
||||
if pk["multiplier"] != 2.0 {
|
||||
t.Errorf("peak = %#v", pk)
|
||||
}
|
||||
}
|
||||
@ -340,7 +340,7 @@ func TestBillingRuleEditorLivesInsideTheBillingPage(t *testing.T) {
|
||||
for _, needle := range []string{
|
||||
`id="br-body"`,
|
||||
"data-act='save'", "data-act='export'", "data-act='newprofile'",
|
||||
".r-url", ".r-mode",
|
||||
".r-src", ".r-mode",
|
||||
} {
|
||||
if !strings.Contains(mount, needle) {
|
||||
t.Errorf("the rule editor is missing %s", needle)
|
||||
@ -1236,3 +1236,136 @@ func TestLongContextComposesWithCacheDiscount(t *testing.T) {
|
||||
// total = 0.2325
|
||||
approx(t, "step composes with cache discount", got, 0.2325)
|
||||
}
|
||||
|
||||
// ---- 源层零价覆盖模型价（issue #1）---------------------------------------
|
||||
//
|
||||
// 真实缺陷（GitCode issue #1）：`compile.go` 给每条**具体 URL** 的 token 规则
|
||||
// 写一条 `{prompt:0, completion:0, per_request:0}` 的源条目，本意只是"标记该源
|
||||
// 已定价"；而插件 priceFor() 的源层是**无条件覆盖**模型层：`sp.prompt ~= nil`
|
||||
// 就把模型价改成 0。两条各自正当的规则叠加 ⇒ 每个请求 cost:0，且因为
|
||||
// out.priced=true，连 unpriced_reqs 都不涨 —— 完全静默。
|
||||
//
|
||||
// 这里把两种条目形状都钉住：旧形状必须**真的是错的**（A 行），新形状必须
|
||||
// **保住模型价同时仍算"已定价"**（B 行）。只断言"键存在"是不够的 —— 那正是
|
||||
// 现有测试把错误形状固化成契约的原因。
|
||||
func TestBillingSourceZeroPriceMustNotZeroModelPrice(t *testing.T) {
|
||||
const promptPrice = 1.5e-7 // 0.15 USD/M
|
||||
const completionPrice = 6e-7 // 0.60 USD/M
|
||||
const want = 0.75 // 1e6 * 1.5e-7 + 1e6 * 6e-7
|
||||
|
||||
newPrices := func(sourceEntry map[string]interface{}) map[string]interface{} {
|
||||
return map[string]interface{}{
|
||||
"default": map[string]interface{}{"prompt": 0.0, "completion": 0.0, "per_request": 0.0},
|
||||
"models": map[string]interface{}{
|
||||
"m": map[string]interface{}{"prompt": promptPrice, "completion": completionPrice},
|
||||
},
|
||||
"sources": map[string]interface{}{"src": sourceEntry},
|
||||
}
|
||||
}
|
||||
run := func(t *testing.T, sourceEntry map[string]interface{}) (cost, unpriced float64) {
|
||||
t.Helper()
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{"prices": newPrices(sourceEntry)})
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m", "source": "src", "key": "***p", "ok": true,
|
||||
"prompt_tokens": 1000000.0, "completion_tokens": 1000000.0,
|
||||
"time": 1750000000000,
|
||||
})
|
||||
st := stateOf(t, ps)
|
||||
total := st["total"].(map[string]interface{})
|
||||
return total["cost"].(float64), st["unpriced_reqs"].(float64)
|
||||
}
|
||||
|
||||
// A 行：compile.go 修复前的形状 —— 这是缺陷本身，必须算成 0。
|
||||
cost, unpriced := run(t, map[string]interface{}{
|
||||
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
|
||||
})
|
||||
approx(t, "旧形状(全零源条目) cost", cost, 0)
|
||||
if unpriced != 0 {
|
||||
t.Errorf("旧形状下 unpriced_reqs = %v，缺陷正是它也为 0（静默）", unpriced)
|
||||
}
|
||||
|
||||
// B 行：修复后的形状 —— 只带 per_request，不得覆盖模型价。
|
||||
cost, unpriced = run(t, map[string]interface{}{"per_request": 0.0})
|
||||
approx(t, "新形状(仅 per_request) cost", cost, want)
|
||||
if unpriced != 0 {
|
||||
t.Errorf("新形状下 unpriced_reqs = %v，want 0（该源仍算已定价）", unpriced)
|
||||
}
|
||||
}
|
||||
|
||||
// 同一源上一个**未列入价目表**的模型：源条目（仅 per_request）让它保持
|
||||
// "已定价"，所以不会被计入 unpriced_reqs —— 这正是 compile 写源条目的原意。
|
||||
func TestBillingSourceEntryStillMarksUnlistedModelPriced(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"default": map[string]interface{}{"prompt": 0.0, "completion": 0.0, "per_request": 0.0},
|
||||
"models": map[string]interface{}{
|
||||
"priced-model": map[string]interface{}{"prompt": 1.5e-7, "completion": 6e-7},
|
||||
},
|
||||
"sources": map[string]interface{}{
|
||||
"src": map[string]interface{}{"per_request": 0.0},
|
||||
},
|
||||
},
|
||||
})
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "unlisted-model", "source": "src", "key": "***p", "ok": true,
|
||||
"prompt_tokens": 1000.0, "completion_tokens": 0.0, "time": 1750000000000,
|
||||
})
|
||||
st := stateOf(t, ps)
|
||||
if got := st["unpriced_reqs"].(float64); got != 0 {
|
||||
t.Errorf("unpriced_reqs = %v，want 0：源条目存在即为已定价源", got)
|
||||
}
|
||||
}
|
||||
|
||||
// profile 级 default 策略：规则以**源名**为键，编译在插件内完成；
|
||||
// `default: free` 表示"没有单独规则的源一律免费"（不进 unpriced 统计），
|
||||
// `unpriced`/缺省则让这些源保持可统计；显式 mode=unpriced 的源即使
|
||||
// default=free 也必须仍被统计。
|
||||
func TestBillingProfileDefaultPolicy(t *testing.T) {
|
||||
newVM := func(t *testing.T, rules map[string]interface{}) *Plugins {
|
||||
t.Helper()
|
||||
ps, _ := billingVM(t)
|
||||
if _, err := ps.Call("billing", "rules_put", map[string]interface{}{"billing": rules}); err != nil {
|
||||
t.Fatalf("rules_put: %v", err)
|
||||
}
|
||||
return ps
|
||||
}
|
||||
fire := func(t *testing.T, ps *Plugins, source string) float64 {
|
||||
t.Helper()
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "some-unlisted-model", "source": source, "key": "***p", "ok": true,
|
||||
"prompt_tokens": 1000.0, "completion_tokens": 10.0, "time": 1750000000000,
|
||||
})
|
||||
return stateOf(t, ps)["unpriced_reqs"].(float64)
|
||||
}
|
||||
base := func(def interface{}, rules []interface{}) map[string]interface{} {
|
||||
p := map[string]interface{}{"id": "p", "rules": rules}
|
||||
if def != nil {
|
||||
p["default"] = def
|
||||
}
|
||||
return map[string]interface{}{"active": "p", "profiles": []interface{}{p}}
|
||||
}
|
||||
|
||||
// default=free：未列规则的源算"刻意免费"
|
||||
ps := newVM(t, base("free", []interface{}{
|
||||
map[string]interface{}{"source": "priced-src", "mode": "free"},
|
||||
}))
|
||||
if got := fire(t, ps, "unlisted-src"); got != 0 {
|
||||
t.Errorf("default=free: unpriced_reqs = %v, want 0", got)
|
||||
}
|
||||
// default=free 但显式 unpriced 的源仍要统计
|
||||
ps = newVM(t, base("free", []interface{}{
|
||||
map[string]interface{}{"source": "counted-src", "mode": "unpriced"},
|
||||
}))
|
||||
if got := fire(t, ps, "counted-src"); got != 1 {
|
||||
t.Errorf("显式 unpriced 在 default=free 下仍应统计，got %v", got)
|
||||
}
|
||||
// default 缺省/unpriced：未列规则的源计入 unpriced
|
||||
for _, def := range []interface{}{nil, "unpriced"} {
|
||||
ps := newVM(t, base(def, []interface{}{}))
|
||||
if got := fire(t, ps, "unlisted-src"); got != 1 {
|
||||
t.Errorf("default=%v: unpriced_reqs = %v, want 1", def, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -315,7 +315,7 @@ func TestBillingTableHeaderMatchesRowColumns(t *testing.T) {
|
||||
// headers were counted too and the check reported 17 vs 8 — a failure about
|
||||
// two unrelated tables. Stop at the rule editor's script.
|
||||
header := js[hStart:]
|
||||
if cut := strings.Index(header, "/api/plugins/billing/rules"); cut > 0 {
|
||||
if cut := strings.Index(header, "/api/plugins/http/billing/rules"); cut > 0 {
|
||||
header = header[:cut]
|
||||
}
|
||||
// Count <th> with OR without attributes: the name column now carries
|
||||
|
||||
@ -72,6 +72,411 @@ local DEFAULT_PRICES = {
|
||||
}
|
||||
plugin.prices = DEFAULT_PRICES
|
||||
|
||||
-- ===== 规则 → 价格表编译（全部在插件内） ==============================
|
||||
--
|
||||
-- billing 没有任何内核逻辑（issue #1 的方向）：规则文件、编译、计价、账目、
|
||||
-- 规则编辑页全部由本插件负责。内核既不知道 billing.json，也不知道
|
||||
-- free/token/subscription 这些语义 —— 它只做通用的"加载插件 / 持久化插件状态 /
|
||||
-- 提供通用 HTTP 与文件能力"。
|
||||
--
|
||||
-- 规则以**源名**（`source`）为键，而不是 URL：URL→源名的映射属于内核配置，
|
||||
-- 插件不该为了它去和内核对接；而编辑页跑在浏览器里，可以直接取 /api/sources
|
||||
-- 让运营选源名。`url: "*"` 只保留为"全局模型价目表"（模型维度本来就不分源）。
|
||||
local VALID_MODES = { free = true, token = true, subscription = true, unpriced = true }
|
||||
|
||||
local function trim(v)
|
||||
local s = tostring(v == nil and "" or v)
|
||||
return (s:gsub("^%s+", ""):gsub("%s+$", ""))
|
||||
end
|
||||
|
||||
-- USD per million → USD per token.
|
||||
local function usdPerM(v)
|
||||
local n = tonumber(trim(v))
|
||||
if n == nil then return nil, "not a number: \"" .. tostring(v) .. "\"" end
|
||||
if n < 0 then return nil, "negative price: \"" .. tostring(v) .. "\"" end
|
||||
return n / 1e6, nil
|
||||
end
|
||||
|
||||
local function compilePeak(pk)
|
||||
local win = {}
|
||||
if type(pk.weekdays) == "table" and #pk.weekdays > 0 then
|
||||
local days = {}
|
||||
for i, d in ipairs(pk.weekdays) do days[i] = d end
|
||||
win.days = days
|
||||
end
|
||||
if type(pk.hours) == "table" and #pk.hours > 0 then
|
||||
local hours = {}
|
||||
for i, h in ipairs(pk.hours) do hours[i] = { h[1], h[2] } end
|
||||
win.hours = hours
|
||||
end
|
||||
local out = { multiplier = tonumber(pk.multiplier) or 1 }
|
||||
if next(win) ~= nil then out.windows = { win } end
|
||||
return out
|
||||
end
|
||||
|
||||
local function longContextRate(lc, basePerToken)
|
||||
local prompt = trim(lc.prompt)
|
||||
local mult = tonumber(lc.multiplier) or 0
|
||||
if prompt ~= "" then
|
||||
local v, err = usdPerM(prompt)
|
||||
if not v then return nil, err end
|
||||
return v, nil
|
||||
end
|
||||
if mult > 0 then return basePerToken * mult, nil end
|
||||
return nil, "needs either prompt or multiplier"
|
||||
end
|
||||
|
||||
-- 一条规则作用于哪个源。`source` 是正式写法；`url` 若不是 "*"，仅在它恰好
|
||||
-- 等于源名时兼容（旧配置迁移用）。
|
||||
local function ruleSource(rule)
|
||||
local src = trim(rule.source)
|
||||
if src ~= "" then return src end
|
||||
local u = trim(rule.url)
|
||||
if u ~= "" and u ~= "*" then return u end
|
||||
return nil
|
||||
end
|
||||
|
||||
local function applyRule(prices, rule, srcName)
|
||||
local mode = rule.mode
|
||||
if mode == "free" or mode == "subscription" then
|
||||
prices.sources[srcName] = { prompt = 0, completion = 0, per_request = 0 }
|
||||
if mode == "subscription" then
|
||||
if prices.subscriptions == nil then prices.subscriptions = {} end
|
||||
local cur = trim(rule.currency)
|
||||
if cur == "" then cur = "USD" end
|
||||
prices.subscriptions[srcName] = { monthly = tonumber(rule.subscription) or 0, currency = cur }
|
||||
end
|
||||
return true
|
||||
end
|
||||
if mode == "unpriced" then
|
||||
-- 刻意不写条目并显式记录：profile 的 default=free 也不能把它当"免费"。
|
||||
prices.explicit_unpriced[srcName] = true
|
||||
return true
|
||||
end
|
||||
if mode == "token" then
|
||||
if type(rule.models) ~= "table" or next(rule.models) == nil then
|
||||
return nil, string.format("token rule for %q requires at least one model",
|
||||
tostring(rule.source or rule.url))
|
||||
end
|
||||
for modelID, t in pairs(rule.models) do
|
||||
local prompt, perr = usdPerM(t.prompt)
|
||||
if not prompt then return nil, string.format("model %q prompt: %s", tostring(modelID), perr) end
|
||||
local completion, cerr = usdPerM(t.completion)
|
||||
if not completion then return nil, string.format("model %q completion: %s", tostring(modelID), cerr) end
|
||||
local entry = { prompt = prompt, completion = completion }
|
||||
if t.cache_discount ~= nil then entry.cache_discount = tonumber(t.cache_discount) end
|
||||
if trim(t.reasoning) ~= "" then
|
||||
local rp, rerr = usdPerM(t.reasoning)
|
||||
if not rp then return nil, string.format("model %q reasoning: %s", tostring(modelID), rerr) end
|
||||
entry.reasoning_price = rp
|
||||
end
|
||||
if type(t.long_context) == "table" then
|
||||
local above = tonumber(t.long_context.above) or 0
|
||||
if above <= 0 then
|
||||
return nil, string.format("model %q long_context: above must be > 0", tostring(modelID))
|
||||
end
|
||||
local rate, lerr = longContextRate(t.long_context, prompt)
|
||||
if not rate then return nil, string.format("model %q long_context: %s", tostring(modelID), lerr) end
|
||||
entry.long_context = { above = above, prompt = rate }
|
||||
end
|
||||
if type(rule.peak) == "table" then entry.peak = compilePeak(rule.peak) end
|
||||
prices.models[modelID] = entry
|
||||
end
|
||||
-- token 规则**不写源条目**：未列价的模型进入 unpriced 统计（可见）。
|
||||
return true
|
||||
end
|
||||
return nil, string.format("unknown mode %q", tostring(mode))
|
||||
end
|
||||
|
||||
-- compileRules(profile) → { prices, warnings } 或 nil, err。
|
||||
-- 不接收源列表：规则以源名为键，编译器不需要知道网关上有哪些源。
|
||||
local function compileRules(profile)
|
||||
if type(profile) ~= "table" then return nil, "no billing profile" end
|
||||
local prices = {
|
||||
currency = "USD",
|
||||
default = { prompt = 0, completion = 0, per_request = 0 },
|
||||
sources = {}, models = {}, keys = {}, explicit_unpriced = {},
|
||||
}
|
||||
if profile.default == "free" or profile.default == "unpriced" then
|
||||
prices.default_mode = profile.default
|
||||
end
|
||||
local rules = type(profile.rules) == "table" and profile.rules or {}
|
||||
for i = 1, #rules do
|
||||
local c = trim(rules[i].currency)
|
||||
if c ~= "" then prices.currency = c break end
|
||||
end
|
||||
local warnings = {}
|
||||
for i = 1, #rules do
|
||||
local rule = rules[i]
|
||||
local url = trim(rule.url)
|
||||
local src = ruleSource(rule)
|
||||
if url == "*" then
|
||||
local ok, err = applyRule(prices, rule, nil)
|
||||
if not ok then return nil, err end
|
||||
elseif src == nil then
|
||||
warnings[#warnings + 1] = "rule " .. i .. " 既没有 source 也没有 url=\"*\""
|
||||
else
|
||||
local ok, err = applyRule(prices, rule, src)
|
||||
if not ok then return nil, err end
|
||||
end
|
||||
end
|
||||
return { prices = prices, warnings = warnings }
|
||||
end
|
||||
|
||||
-- ===== 规则持久化：插件自己读写自己的配置文件 ========================
|
||||
--
|
||||
-- 规则文件放在**插件自己的目录**（内核通过通用的 __llmsproxy_plugin_dir 暴露，
|
||||
-- 任何插件都能用；内核不知道 billing 的存在）：`<plugin_dir>/billing.json`。
|
||||
-- 用 JSON 因为插件沙箱只有 json 库；用独立文件因为"保存系统配置"绝不该改动
|
||||
-- 价目表，反之亦然。
|
||||
plugin.rules = { active = "", profiles = {} }
|
||||
plugin.rules_path = ""
|
||||
|
||||
local function defaultRulesPath()
|
||||
local dir = rawget(_G, "__llmsproxy_plugin_dir")
|
||||
if type(dir) ~= "string" or dir == "" then return "" end
|
||||
return dir .. "/billing.json"
|
||||
end
|
||||
|
||||
local function readRulesFile(path)
|
||||
if path == "" then return nil end
|
||||
local f = io.open(path, "r")
|
||||
if f == nil then return nil end
|
||||
local data = f:read("*a")
|
||||
f:close()
|
||||
if data == nil or data == "" then return nil end
|
||||
local ok, dec = pcall(json.decode, data)
|
||||
if not ok or type(dec) ~= "table" then
|
||||
return nil, "规则文件不是合法 JSON: " .. tostring(path)
|
||||
end
|
||||
return dec
|
||||
end
|
||||
|
||||
local function writeRulesFile(path, rules)
|
||||
if path == "" then return nil, "插件目录不可用，无法保存规则文件" end
|
||||
local ok, enc = pcall(json.encode, rules)
|
||||
if not ok then return nil, "规则无法序列化" end
|
||||
local tmp = path .. ".tmp"
|
||||
local f, err = io.open(tmp, "w")
|
||||
if f == nil then return nil, "无法写规则文件: " .. tostring(err) end
|
||||
local wok, werr = f:write(enc)
|
||||
f:close()
|
||||
if not wok then os.remove(tmp) return nil, "写规则文件失败: " .. tostring(werr) end
|
||||
local rok, rerr = os.rename(tmp, path)
|
||||
if not rok then
|
||||
os.remove(tmp)
|
||||
return nil, "替换规则文件失败: " .. tostring(rerr)
|
||||
end
|
||||
return true
|
||||
end
|
||||
|
||||
local function validateRulesShape(rules)
|
||||
if type(rules) ~= "table" then return nil, "规则必须是对象" end
|
||||
if rules.active ~= nil and type(rules.active) ~= "string" then return nil, "active 必须是字符串" end
|
||||
local profiles = rules.profiles
|
||||
if profiles == nil then profiles = {} rules.profiles = profiles end
|
||||
if type(profiles) ~= "table" then return nil, "profiles 必须是数组" end
|
||||
local seen = {}
|
||||
for i, p in ipairs(profiles) do
|
||||
if type(p) ~= "table" then return nil, "profiles[" .. i .. "] 必须是对象" end
|
||||
local id = trim(p.id)
|
||||
if id == "" then return nil, "profiles[" .. i .. "] 缺少 id" end
|
||||
if seen[id] then return nil, "profile id 重复: " .. id end
|
||||
seen[id] = true
|
||||
if p.default ~= nil and p.default ~= "free" and p.default ~= "unpriced" then
|
||||
return nil, "profile " .. id .. " 的 default 只能是 free 或 unpriced"
|
||||
end
|
||||
if p.rules ~= nil and type(p.rules) ~= "table" then
|
||||
return nil, "profile " .. id .. " 的 rules 必须是数组"
|
||||
end
|
||||
for j, r in ipairs(p.rules or {}) do
|
||||
if type(r) ~= "table" then return nil, "profile " .. id .. " 的 rules[" .. j .. "] 必须是对象" end
|
||||
local mode = trim(r.mode)
|
||||
if not VALID_MODES[mode] then
|
||||
return nil, "profile " .. id .. " 的 rules[" .. j .. "] 的 mode 非法: " .. mode
|
||||
end
|
||||
local wildcard = trim(r.url) == "*"
|
||||
if ruleSource(r) == nil and not wildcard then
|
||||
return nil, "profile " .. id .. " 的 rules[" .. j .. "] 必须有 source（或 url=\"*\" 作全局模型价目）"
|
||||
end
|
||||
if mode == "token" and (type(r.models) ~= "table" or next(r.models) == nil) then
|
||||
return nil, "profile " .. id .. " 的 token 规则缺少 models"
|
||||
end
|
||||
end
|
||||
end
|
||||
if rules.active ~= nil and rules.active ~= "" and not seen[rules.active] then
|
||||
return nil, "active 指向不存在的 profile: " .. rules.active
|
||||
end
|
||||
return true
|
||||
end
|
||||
|
||||
local function activeProfile(rules)
|
||||
local profile = nil
|
||||
for _, p in ipairs(rules.profiles or {}) do
|
||||
if rules.active ~= nil and rules.active ~= "" and p.id == rules.active then
|
||||
profile = p
|
||||
break
|
||||
end
|
||||
end
|
||||
if profile == nil then profile = (rules.profiles or {})[1] end
|
||||
return profile
|
||||
end
|
||||
|
||||
-- applyRules：用给定规则重编译并注入 plugin.prices。启动、保存、切方案都走它。
|
||||
local function applyRules(rules)
|
||||
local res, err = compileRules(activeProfile(rules) or {})
|
||||
if not res then return nil, tostring(err) end
|
||||
plugin.prices = res.prices
|
||||
return res
|
||||
end
|
||||
|
||||
-- 启动即加载：插件被内核加载时就自己读规则文件并编译 —— 内核不需要调用任何
|
||||
-- billing 专用接口，它只知道"加载插件"。
|
||||
plugin.rules_path = defaultRulesPath()
|
||||
do
|
||||
local loaded, lerr = readRulesFile(plugin.rules_path)
|
||||
if lerr then
|
||||
plugin.load_error = lerr
|
||||
elseif loaded ~= nil then
|
||||
plugin.rules = loaded
|
||||
end
|
||||
local res, err = applyRules(plugin.rules)
|
||||
if not res then plugin.load_error = tostring(err) end
|
||||
end
|
||||
|
||||
-- 插件 HTTP 分发把**整个请求表**交给 handler（method/path/body/headers…），
|
||||
-- body 是原始字符串；而直调导出（/call/...）拿到的就是参数表本身。这里统一
|
||||
-- 解出 JSON 载荷，避免"把请求信封当成规则写进 billing.json"（真实踩过：
|
||||
-- PUT 之后 billing.json 变成了整个请求的转储，规则全丢、价格表清空）。
|
||||
local function reqData(req)
|
||||
if type(req) ~= "table" then return {} end
|
||||
if type(req.json) == "table" then return req.json end
|
||||
if type(req.body) == "string" and req.body ~= "" then
|
||||
local ok, dec = pcall(json.decode, req.body)
|
||||
if ok and type(dec) == "table" then return dec end
|
||||
end
|
||||
if req.method == nil and req.path == nil then return req end
|
||||
return {}
|
||||
end
|
||||
|
||||
-- ---- 规则编辑器的 HTTP 面（挂在插件自己的 /api/plugins/http/billing/ 下）----
|
||||
-- 规则属于插件，写盘也在插件内完成；内核没有 billing 端点。
|
||||
function plugin.__http_rules_get(req)
|
||||
local out = { billing = plugin.rules, warnings = {} }
|
||||
local profile = activeProfile(plugin.rules or {})
|
||||
if profile then out.active = profile.id end
|
||||
local res = profile and compileRules(profile)
|
||||
if res and res.warnings then out.warnings = res.warnings end
|
||||
if plugin.load_error then out.error = plugin.load_error end
|
||||
return { status = 200, json = out }
|
||||
end
|
||||
|
||||
function plugin.__http_rules_put(req)
|
||||
local body = reqData(req)
|
||||
if type(body) ~= "table" then
|
||||
return { status = 400, json = { error = "请求体必须是 JSON 对象" } }
|
||||
end
|
||||
local rules = body.billing or body
|
||||
local ok, verr = validateRulesShape(rules)
|
||||
if not ok then return { status = 400, json = { error = verr } } end
|
||||
local res, cerr = applyRules(rules)
|
||||
if not res then return { status = 400, json = { error = tostring(cerr) } } end
|
||||
local wok, werr = writeRulesFile(plugin.rules_path, rules)
|
||||
if not wok then return { status = 500, json = { error = werr } } end
|
||||
plugin.rules = rules
|
||||
return { status = 200, json = { ok = true, billing = rules, warnings = res.warnings } }
|
||||
end
|
||||
|
||||
function plugin.__http_rules_add(req)
|
||||
local body = reqData(req) or {}
|
||||
local copy = json.decode(json.encode(plugin.rules or { active = "", profiles = {} }))
|
||||
local target = nil
|
||||
for _, p in ipairs(copy.profiles or {}) do
|
||||
if body.profile == nil or body.profile == "" or p.id == body.profile then target = p break end
|
||||
end
|
||||
if target == nil then
|
||||
return { status = 400, json = { error = "没有该方案: " .. tostring(body.profile) } }
|
||||
end
|
||||
local rule = body.rule
|
||||
if type(rule) ~= "table" then return { status = 400, json = { error = "缺少 rule" } } end
|
||||
local key = ruleSource(rule)
|
||||
if key == nil then return { status = 400, json = { error = "规则必须有 source" } } end
|
||||
target.rules = target.rules or {}
|
||||
local replaced = false
|
||||
for i, r in ipairs(target.rules) do
|
||||
if ruleSource(r) == key and trim(r.url) == trim(rule.url) then
|
||||
target.rules[i] = rule replaced = true break
|
||||
end
|
||||
end
|
||||
if not replaced then target.rules[#target.rules + 1] = rule end
|
||||
local ok, verr = validateRulesShape(copy)
|
||||
if not ok then return { status = 400, json = { error = verr } } end
|
||||
local res, cerr = applyRules(copy)
|
||||
if not res then return { status = 400, json = { error = tostring(cerr) } } end
|
||||
local wok, werr = writeRulesFile(plugin.rules_path, copy)
|
||||
if not wok then return { status = 500, json = { error = werr } } end
|
||||
plugin.rules = copy
|
||||
return { status = 200, json = { ok = true, billing = copy } }
|
||||
end
|
||||
|
||||
function plugin.__http_rules_delete(req)
|
||||
local body = reqData(req) or {}
|
||||
local copy = json.decode(json.encode(plugin.rules or { active = "", profiles = {} }))
|
||||
local target = nil
|
||||
for _, p in ipairs(copy.profiles or {}) do
|
||||
if body.profile == nil or body.profile == "" or p.id == body.profile then target = p break end
|
||||
end
|
||||
if target == nil then
|
||||
return { status = 400, json = { error = "没有该方案: " .. tostring(body.profile) } }
|
||||
end
|
||||
local out = {}
|
||||
local found = false
|
||||
for _, r in ipairs(target.rules or {}) do
|
||||
if ruleSource(r) == body.source or trim(r.url) == trim(body.url) then
|
||||
found = true
|
||||
else
|
||||
out[#out + 1] = r
|
||||
end
|
||||
end
|
||||
if not found then return { status = 404, json = { error = "没有该规则" } } end
|
||||
target.rules = out
|
||||
local ok, verr = validateRulesShape(copy)
|
||||
if not ok then return { status = 400, json = { error = verr } } end
|
||||
local res, cerr = applyRules(copy)
|
||||
if not res then return { status = 400, json = { error = tostring(cerr) } } end
|
||||
local wok, werr = writeRulesFile(plugin.rules_path, copy)
|
||||
if not wok then return { status = 500, json = { error = werr } } end
|
||||
plugin.rules = copy
|
||||
return { status = 200, json = { ok = true, billing = copy } }
|
||||
end
|
||||
|
||||
-- 直调导出（/api/plugins/billing/call/<fn>）：返回纯 JSON（不带 HTTP 状态壳），
|
||||
-- 便于运维/测试与插件自己的 HTTP 路由共用同一套实现。
|
||||
local function unwrap(res)
|
||||
if type(res) == "table" and res.json ~= nil then return res.json end
|
||||
return res
|
||||
end
|
||||
function plugin.rules_get(args) return unwrap(plugin.__http_rules_get({ json = args or {} })) end
|
||||
function plugin.rules_put(args) return unwrap(plugin.__http_rules_put({ json = args or {} })) end
|
||||
function plugin.rules_add(args) return unwrap(plugin.__http_rules_add({ json = args or {} })) end
|
||||
function plugin.rules_delete(args) return unwrap(plugin.__http_rules_delete({ json = args or {} })) end
|
||||
|
||||
-- 每条路由的 path 必须唯一：内核的路由冲突仲裁按**路径**（不看 method）判重，
|
||||
-- 同一路径挂 4 个 method 会被当成 4 个插件抢同一个路径。所以按操作拆路径。
|
||||
plugin.http = {
|
||||
{ path = "billing/rules", method = "GET", handler = "__http_rules_get",
|
||||
roles = { "admin" }, description = "计费规则读取" },
|
||||
{ path = "billing/rules/replace", method = "PUT", handler = "__http_rules_put",
|
||||
roles = { "admin" }, description = "替换整份计费规则（插件自己写盘）" },
|
||||
{ path = "billing/rules/add", method = "POST", handler = "__http_rules_add",
|
||||
roles = { "admin" }, description = "新增/替换一条计费规则" },
|
||||
{ path = "billing/rules/remove", method = "DELETE", handler = "__http_rules_delete",
|
||||
roles = { "admin" }, description = "删除一条计费规则" },
|
||||
}
|
||||
|
||||
-- 测试/运维直调导出（/api/plugins/billing/call/...）；不是内核接口。
|
||||
plugin.exports = { "rules_get", "rules_put", "rules_add", "rules_delete" }
|
||||
|
||||
-- Default prompt-cache discount. 0.1 = a cache read costs a tenth of a fresh
|
||||
-- token, which is what DeepSeek/Qwen/Kimi and most others charge. It can be
|
||||
-- overridden per price entry (prices.models.<m>.cache_discount) or globally by
|
||||
@ -189,7 +594,11 @@ local function priceFor(payload)
|
||||
-- look the same: an unpriced model silently costing 0 is the most dangerous
|
||||
-- failure mode a cost plugin has, because the bill still adds up and just
|
||||
-- quietly under-reports. It is counted separately and surfaced in the UI.
|
||||
out = {
|
||||
-- ★ local 是必须的：此前写的是全局 `out = {`，而 priceFor 每个请求都会调用，
|
||||
-- 于是这个表挂在 VM 级全局变量上跨请求残留/泄漏（GitCode issue #1 顺带发现）。
|
||||
-- 功能上因为每次整体重建而侥幸正确，但它是共享可变状态，任何提前 return 或
|
||||
-- 后续分支忘了重建都会让上一单的价格漏到下一单 —— 计费错误不会报错。
|
||||
local out = {
|
||||
prompt = d.prompt or 0, completion = d.completion or 0,
|
||||
per_request = 0, cache_discount = d.cache_discount, peak = d.peak,
|
||||
-- nil = bill reasoning at the completion rate, which is what every
|
||||
@ -236,6 +645,16 @@ local function priceFor(payload)
|
||||
if kp.cache_discount ~= nil then out.cache_discount = kp.cache_discount end
|
||||
if kp.peak ~= nil then out.peak = kp.peak end
|
||||
end
|
||||
|
||||
-- profile 级 default 策略（编译期写入 prices.default_mode）在这里落地：
|
||||
-- `free` = 运营声明"没有单独规则的源一律免费" ⇒ 计为已定价（不进 unpriced 统计）；
|
||||
-- 但显式 mode=unpriced 的源必须仍计入，否则"只统计不收费"的声明失效。
|
||||
if not out.priced and p.default_mode == "free" then
|
||||
local explicit = p.explicit_unpriced
|
||||
if not (type(explicit) == "table" and payload.source and explicit[payload.source]) then
|
||||
out.priced = true
|
||||
end
|
||||
end
|
||||
return out
|
||||
end
|
||||
|
||||
@ -1054,12 +1473,12 @@ plugin.ui = {
|
||||
})();
|
||||
|
||||
(function () {
|
||||
var API = "/api/plugins/billing/rules";
|
||||
var API = "/api/plugins/http/billing/rules";
|
||||
var STR = {
|
||||
en: {
|
||||
title: "Billing rules", profile: "Profile", newProfile: "New profile",
|
||||
addRule: "Add rule", delRule: "Delete", exportYaml: "Export YAML",
|
||||
url: "Source URL", mode: "Mode", prompt: "Prompt $/M",
|
||||
source: "Source", mode: "Mode", prompt: "Prompt $/M",
|
||||
completion: "Completion $/M", currency: "Currency",
|
||||
monthly: "Monthly fee", peakMult: "Peak multiplier",
|
||||
peakHours: "Peak hours (UTC)", peakDays: "Peak days (UTC 1=Mon)",
|
||||
@ -1071,11 +1490,11 @@ plugin.ui = {
|
||||
lcHint: "Long-context step: tokens PAST the threshold cost more. Leave all three blank for one flat rate.", reasoningHint: "Thinking tokens are billed as output unless you set a different rate here.", mFree: "free", mToken: "token", mSub: "subscription", mUnpriced: "unpriced",
|
||||
noProfile: "No profile yet — create one to start pricing.",
|
||||
noRules: "No rules in this profile",
|
||||
warnNoSource: "matches no configured source",
|
||||
warnNoSource: "no such source on this gateway",
|
||||
active: "ACTIVE", setActive: "Set active",
|
||||
saving: "saving…", saved: "Saved", rejected: "Rejected", save: "Save",
|
||||
newProfileId: "New profile id",
|
||||
needUrl: "a rule needs a URL", needPrices: "every model needs both prices",
|
||||
needUrl: "a rule needs a source name (or url '*' for a global model catalogue)", needPrices: "every model needs both prices",
|
||||
badPeakHours: "peak hours look like 1-4,6-10 (UTC, 0-23)",
|
||||
badPeakDays: "peak days are UTC weekday numbers 0-6",
|
||||
peakNeedsMult: "peak hours need a multiplier",
|
||||
@ -1085,7 +1504,7 @@ plugin.ui = {
|
||||
zh: {
|
||||
title: "计费规则", profile: "方案", newProfile: "新建方案",
|
||||
addRule: "新增规则", delRule: "删除", exportYaml: "导出 YAML",
|
||||
url: "源 URL", mode: "计费模式", prompt: "输入 $/M",
|
||||
source: "源（source 名）", mode: "计费模式", prompt: "输入 $/M",
|
||||
completion: "输出 $/M", currency: "币种",
|
||||
monthly: "月费", peakMult: "峰段倍数",
|
||||
peakHours: "峰段小时 (UTC)", peakDays: "峰段星期 (UTC 1=周一)",
|
||||
@ -1098,11 +1517,11 @@ plugin.ui = {
|
||||
mSub: "subscription 订阅", mUnpriced: "unpriced 只统计",
|
||||
noProfile: "还没有方案 —— 新建一个开始定价。",
|
||||
noRules: "该方案暂无规则",
|
||||
warnNoSource: "匹配不到已配置的源",
|
||||
warnNoSource: "网关上没有这个源",
|
||||
active: "生效中", setActive: "设为生效",
|
||||
saving: "保存中…", saved: "已保存", rejected: "被拒绝", save: "保存",
|
||||
newProfileId: "新方案 id",
|
||||
needUrl: "规则必须有 URL",
|
||||
needUrl: "规则必须有源名（或 url '*' 作全局模型价目）",
|
||||
needPrices: "每个模型都要填输入和输出价格",
|
||||
badPeakHours: "峰段小时应形如 1-4,6-10（UTC，0-23）",
|
||||
badPeakDays: "峰段星期应为 UTC 星期编号 0-6",
|
||||
@ -1120,8 +1539,9 @@ plugin.ui = {
|
||||
return { "&": "&", "<": "<", ">": ">", '"': """ }[c];
|
||||
});
|
||||
}
|
||||
var OP_PATH = { GET: "", PUT: "/replace", POST: "/add", DELETE: "/remove" };
|
||||
function call(method, body) {
|
||||
return fetch(API, {
|
||||
return fetch(API + (OP_PATH[method] || ""), {
|
||||
method: method,
|
||||
credentials: "same-origin",
|
||||
headers: body ? { "Content-Type": "application/json" } : undefined,
|
||||
@ -1220,7 +1640,8 @@ plugin.ui = {
|
||||
}
|
||||
|
||||
function ruleRow(rule, known, T) {
|
||||
var unknown = rule.url !== "*" && known.indexOf(rule.url) < 0;
|
||||
var src = rule.source || (rule.url !== "*" ? rule.url : "");
|
||||
var unknown = src !== "" && src !== "*" && known.indexOf(src) < 0;
|
||||
var opts = [["free", T.mFree], ["token", T.mToken], ["subscription", T.mSub], ["unpriced", T.mUnpriced]]
|
||||
.map(function (o) {
|
||||
return "<option value='" + o[0] + "'" + (rule.mode === o[0] ? " selected" : "") + ">" + esc(o[1]) + "</option>";
|
||||
@ -1230,8 +1651,8 @@ plugin.ui = {
|
||||
// 的固有宽度会把列撑开：实测表格 986px 挤在 954px 容器里，最右侧的
|
||||
// 「删除」列被切掉 32px，按钮点不到；r-cur/r-sub 这些只有 style='width:74px'
|
||||
// 的输入框实测只有 40px 可用，几乎不能输入。
|
||||
return "<tr class='br-rule' data-url='" + esc(rule.url || "") + "'>" +
|
||||
"<td style='min-width:200px'><input class='r-url' value='" + esc(rule.url || "") + "' list='br-urls' style='width:100%;box-sizing:border-box'>" +
|
||||
return "<tr class='br-rule' data-source='" + esc(src) + "'>" +
|
||||
"<td style='min-width:200px'><input class='r-src' value='" + esc(src) + "' list='br-srcs' placeholder='source 名，或 * = 全局模型价目' style='width:100%;box-sizing:border-box'>" +
|
||||
(unknown ? "<div style='color:#e0a33e;font-size:11px'>⚠ " + esc(T.warnNoSource) + "</div>" : "") +
|
||||
"</td>" +
|
||||
"<td style='width:88px'><select class='r-mode' style='width:100%;box-sizing:border-box;padding:2px;font-size:12px'>" + opts + "</select></td>" +
|
||||
@ -1248,10 +1669,11 @@ plugin.ui = {
|
||||
|
||||
// ---- collecting one rule back out of the DOM ----------------------------
|
||||
function collectRule(tr, T) {
|
||||
var url = tr.querySelector(".r-url").value.trim();
|
||||
if (!url) throw new Error(T.needUrl);
|
||||
var src = tr.querySelector(".r-src").value.trim();
|
||||
if (!src) throw new Error(T.needUrl);
|
||||
var mode = tr.querySelector(".r-mode").value;
|
||||
var out = { url: url, mode: mode };
|
||||
var out = { source: src, mode: mode };
|
||||
if (src === "*") { out.source = ""; out.url = "*"; }
|
||||
var cur = tr.querySelector(".r-cur").value.trim();
|
||||
if (cur) out.currency = cur;
|
||||
if (mode === "subscription") {
|
||||
@ -1401,7 +1823,8 @@ plugin.ui = {
|
||||
dsl.profiles.push({ id: id, rules: rules });
|
||||
});
|
||||
dsl.active = DATA && DATA.active ? DATA.active : (dsl.profiles[0] || {}).id;
|
||||
return "billing:\n" + yamlValue(dsl, 1) + "\n";
|
||||
// 规则现在是插件自持的 billing.json，导出就是同一个 JSON 结构。
|
||||
return JSON.stringify(dsl, null, 2) + "\n";
|
||||
}
|
||||
|
||||
// ---- render --------------------------------------------------------------
|
||||
@ -1419,7 +1842,7 @@ plugin.ui = {
|
||||
var dsl = (DATA && DATA.billing) || null;
|
||||
var profiles = (dsl && dsl.profiles) || [];
|
||||
var active = (DATA && DATA.active) || (profiles[0] && profiles[0].id) || "";
|
||||
var known = (DATA && DATA.urls) || [];
|
||||
var known = (DATA && DATA.sources) || [];
|
||||
|
||||
var warn = "";
|
||||
if (DATA && DATA.warnings && DATA.warnings.length) {
|
||||
@ -1435,7 +1858,7 @@ plugin.ui = {
|
||||
return;
|
||||
}
|
||||
|
||||
var html = warn + "<datalist id='br-urls'>" + dl + "</datalist>";
|
||||
var html = warn + "<datalist id='br-srcs'>" + dl + "</datalist>";
|
||||
profiles.forEach(function (p) {
|
||||
var rules = p.rules || [];
|
||||
html += "<div data-profile='" + esc(p.id) + "' style='margin:0 0 18px'>" +
|
||||
@ -1533,7 +1956,7 @@ plugin.ui = {
|
||||
}
|
||||
if (idx < 0) return;
|
||||
dsl.profiles[idx].rules = dsl.profiles[idx].rules.filter(function (r) {
|
||||
return r.url !== tr.dataset.url;
|
||||
return (r.source || r.url) !== tr.dataset.source;
|
||||
});
|
||||
save(dsl);
|
||||
};
|
||||
@ -1547,10 +1970,10 @@ plugin.ui = {
|
||||
if (dsl.profiles[i].id === b.dataset.pid) idx = i;
|
||||
}
|
||||
if (idx < 0) return;
|
||||
// Seed with the first configured URL: an empty url row would fail
|
||||
// Seed with the first configured source: an empty source row would fail
|
||||
// validation on save and the operator would not know why.
|
||||
var first = ((DATA && DATA.urls) || [])[0] || "";
|
||||
dsl.profiles[idx].rules.push({ url: first, mode: "free" });
|
||||
var first = ((DATA && DATA.sources) || [])[0] || "*";
|
||||
dsl.profiles[idx].rules.push({ source: first === "*" ? "" : first, url: first === "*" ? "*" : nil, mode: "free" });
|
||||
save(dsl);
|
||||
};
|
||||
});
|
||||
@ -1627,19 +2050,27 @@ plugin.ui = {
|
||||
p.innerHTML = "<button class='ghost small' id='br-dl'>" + esc(T.exportYaml) + " (.yaml)</button>";
|
||||
holder.appendChild(p);
|
||||
document.getElementById("br-dl").onclick = function () {
|
||||
var blob = new Blob([text], { type: "text/yaml" });
|
||||
var blob = new Blob([text], { type: "application/json" });
|
||||
var a = document.createElement("a");
|
||||
a.href = URL.createObjectURL(blob);
|
||||
a.download = "billing-rules.yaml";
|
||||
a.download = "billing.json";
|
||||
a.click();
|
||||
setTimeout(function () { URL.revokeObjectURL(a.href); }, 2000);
|
||||
};
|
||||
}
|
||||
|
||||
function load() {
|
||||
call("GET")
|
||||
.then(function (j) {
|
||||
DATA = { billing: j.billing || null, active: j.active, urls: j.urls || [], warnings: j.warnings || [] };
|
||||
Promise.all([
|
||||
call("GET"),
|
||||
// 源名列表来自网关自己的 /api/sources（浏览器带管理员会话），插件不需要
|
||||
// 为了它去和内核对接 —— 这正是"规则以源名为键"能做到零内核依赖的原因。
|
||||
fetch("/api/sources", { credentials: "same-origin" })
|
||||
.then(function (r) { return r.ok ? r.json() : { sources: [] }; })
|
||||
.catch(function () { return { sources: [] }; }),
|
||||
]).then(function (res) {
|
||||
var j = res[0], sj = res[1] || {};
|
||||
var names = (sj.sources || []).map(function (s) { return s.name; }).filter(Boolean);
|
||||
DATA = { billing: j.billing || null, active: j.active, sources: names, warnings: j.warnings || [] };
|
||||
render();
|
||||
})
|
||||
.catch(function (e) {
|
||||
|
||||
@ -1046,6 +1046,24 @@ local PLATFORMS = {
|
||||
base_url = "https://snap-access.cn-north-4.myhuaweicloud.com",
|
||||
-- 本插件**自己装的**适配器（见 installCodeartsAdapter）：华为云 AK/SK 签名
|
||||
adapter = "codearts",
|
||||
-- ★ 上游的"不可用"多数是**账号级限流**，约 1 分钟自愈，不是模型坏了：
|
||||
-- * 400 `TM.00001041: 并发会话数已达上限(3个)` —— 账户并发会话上限，
|
||||
-- 实测最后一个请求结束后约 44~60 秒释放；
|
||||
-- * 429 `TPM limit ... 500000 tokens per minute` —— 每分钟 token 配额。
|
||||
-- 内核原本**只在 429 上做冷却、不重试**，于是这类"等一会儿就好"的拒绝
|
||||
-- 直接冒到客户端（流式还表现为 `empty stream`，因为上游用 HTTP 200 +
|
||||
-- error 帧表达拒绝）。
|
||||
--
|
||||
-- 这里声明**声明式重试策略**（内核只按数据匹配，不认识 CodeArts）：
|
||||
-- 命中即在网关内部每 10s 重试一次，最多 18 次（≈3 分钟），客户端只是
|
||||
-- 多等几秒，不会看到错误。流式只在**握手阶段**重试（首帧尚未下发），
|
||||
-- 不会重复输出。
|
||||
retry_policy = {
|
||||
on_status = { 400, 429 },
|
||||
on_body_contains = { "TM.00001041", "TPM limit", "并发会话数已达上限" },
|
||||
delay = "10s",
|
||||
max_attempts = 18,
|
||||
},
|
||||
-- 签名平台：源注册要交**完整凭据 JSON**（适配器自己解 AK/SK 签名），
|
||||
-- 不是裸 access token。见 syncAccountSource 的注释。
|
||||
credential_shape = "full",
|
||||
@ -4627,6 +4645,9 @@ local function syncProviderSource(provider)
|
||||
if plat.endpoint then src.endpoint = plat.endpoint end
|
||||
if plat.endpoint_stream then src.endpoint_stream = plat.endpoint_stream end
|
||||
if plat.headers_extra then src.headers = plat.headers_extra end
|
||||
-- 声明式上游重试策略（见平台上的 retry_policy 注释）：内核只按 status +
|
||||
-- body 子串匹配，不认识任何厂商。
|
||||
if type(plat.retry_policy) == "table" then src.retry = plat.retry_policy end
|
||||
-- ★ 把**设备身份**随源带给适配器：有些平台的头不是简单 Bearer，
|
||||
-- 需要 uid / device_id / machine_id 派生（trae 的 X-Uid / X-Device-Id /
|
||||
-- X-Machine-Id）。适配器沙箱里**读不到凭据**，只能靠 meta 传。
|
||||
@ -10128,7 +10149,21 @@ end
|
||||
-- build_headers：内核按源注入 meta{url,method,body,api_key,...}。
|
||||
-- 凭据是完整 JSON（登录时存的就是它），从中解出 AK/SK/ST。
|
||||
function adapter.build_headers(meta)
|
||||
local h = { ["Content-Type"] = "application/json" }
|
||||
-- ★★ Connection: close 是**必须**的，不是优化。
|
||||
--
|
||||
-- CodeArts 后端按**账户**限制并发会话数（3 个），而它把"会话"绑定在
|
||||
-- HTTP 连接上：内核的 Go transport 会保留空闲连接（默认每 host 2 条）
|
||||
-- 用于 keep-alive，于是第 3~4 个请求就被判成"并发会话数已达上限(3个)"
|
||||
-- （TM.00001041），用户看到的是 codeart 连续不可用。
|
||||
--
|
||||
-- 实证（2026-10-20）：
|
||||
-- * 走网关连续 5 次 → #1/#2/#3 成功，#4/#5 全部 TM.00001041；
|
||||
-- * 同一凭据、每请求**新连接 + Connection: close** 连续 5 次 → 5/5 成功；
|
||||
-- * 之后连续流式也一样：3 次成功后开始 400。
|
||||
-- 即"会话"就是连接，关掉连接会话立刻释放。
|
||||
--
|
||||
-- 代价是每次多一次 TCP/TLS 握手（~百毫秒级），换"渠道真的能用"，值得。
|
||||
local h = { ["Content-Type"] = "application/json", ["Connection"] = "close" }
|
||||
local raw = tostring(meta.api_key or "")
|
||||
local cred = nil
|
||||
if raw:sub(1, 1) == "{" then
|
||||
@ -10146,6 +10181,19 @@ function adapter.build_headers(meta)
|
||||
return h
|
||||
end
|
||||
for k, v in pairs(signed) do h[k] = v end
|
||||
-- ★★ Session-Id 是**必须**的，不是可选项 —— 它决定"并发会话数"怎么算。
|
||||
--
|
||||
-- 实测（2026-10-20，对照参考实现 llm-adapter.ts:1168 `headers.set('Session-Id', …)`）：
|
||||
-- 不发送会话号时，后端把**每个请求**当成一个新会话；而会话在响应结束后还要
|
||||
-- 存活一段时间才释放，于是连续 3 个请求就把账户的「并发会话数上限(3个)」占满，
|
||||
-- 之后所有请求（含流式）稳定回 TM.00001041 / 200+空流（用户看到 empty stream）。
|
||||
-- 发一个**稳定**的会话号（同一账号内恒定），后端复用同一会话 ⇒ 不再累加。
|
||||
--
|
||||
-- 会话号取 AK 的 sha256 前 32 hex：稳定、可复现、不引入需要持久化的状态。
|
||||
local ak = tostring(cred.access_key_id or cred.access_key or "")
|
||||
if ak ~= "" then
|
||||
h["Session-Id"] = string.sub(sha256_hex(ak), 1, 32)
|
||||
end
|
||||
return h
|
||||
end
|
||||
|
||||
|
||||
@ -1156,7 +1156,7 @@ func (p *Provider) Chat(ctx context.Context, req *types.ChatRequest) (*types.Uni
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
raw, status, err := p.do(ctx, chatURL, body, hdrs)
|
||||
raw, status, err := p.doWithRetry(ctx, chatURL, body, hdrs)
|
||||
if err != nil {
|
||||
// a client disconnect or cancelled context is neither a success nor
|
||||
// a failure for scheduling purposes — only upstream errors count
|
||||
@ -1227,8 +1227,48 @@ func (p *Provider) ChatStream(ctx context.Context, req *types.ChatRequest) (<-ch
|
||||
}
|
||||
rc := make(chan respOrErr, 1)
|
||||
go func() {
|
||||
resp, err := p.doRawStream(ctx, streamURL, body, hdrs)
|
||||
rc <- respOrErr{resp, err}
|
||||
// Retry only the HANDSHAKE: a policy-matched rejection (e.g. a
|
||||
// per-account concurrency cap answered with a business 4xx) can clear
|
||||
// within seconds, and nothing has been sent to the client yet — the
|
||||
// kernel holds the first chunk back. Once a 200 arrives and streaming
|
||||
// starts, no retry happens (that would duplicate output).
|
||||
rp := p.cfg.Retry
|
||||
attempts := 1
|
||||
if rp != nil && rp.MaxAttempts > 1 {
|
||||
attempts = rp.MaxAttempts
|
||||
}
|
||||
for i := 0; i < attempts; i++ {
|
||||
resp, err := p.doRawStream(ctx, streamURL, body, hdrs)
|
||||
if err != nil {
|
||||
rc <- respOrErr{resp, err}
|
||||
return
|
||||
}
|
||||
if resp.StatusCode == 200 {
|
||||
rc <- respOrErr{resp, nil}
|
||||
return
|
||||
}
|
||||
// A rejection can only be classified after the body is read: the
|
||||
// policy matches on body substrings, and a source may use the same
|
||||
// status (400) for unrelated errors that must NOT be retried.
|
||||
raw, _ := io.ReadAll(resp.Body)
|
||||
resp.Body.Close()
|
||||
if !rp.ShouldRetry(resp.StatusCode, string(raw)) {
|
||||
// Not the retryable body: hand the rejection back verbatim so
|
||||
// the caller reports the real upstream error.
|
||||
resp2 := &http.Response{StatusCode: resp.StatusCode, Body: io.NopCloser(strings.NewReader(string(raw)))}
|
||||
rc <- respOrErr{resp2, nil}
|
||||
return
|
||||
}
|
||||
if i == attempts-1 {
|
||||
resp2 := &http.Response{StatusCode: resp.StatusCode, Body: io.NopCloser(strings.NewReader(string(raw)))}
|
||||
rc <- respOrErr{resp2, nil}
|
||||
return
|
||||
}
|
||||
if serr := sleepCtx(ctx, rp.DelayDuration()); serr != nil {
|
||||
rc <- respOrErr{nil, serr}
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
|
||||
inner := make(chan types.UnifiedChunk, 64)
|
||||
@ -1560,6 +1600,50 @@ func (p *Provider) do(ctx context.Context, url, body string, hdr http.Header) (s
|
||||
return string(raw), resp.StatusCode, nil
|
||||
}
|
||||
|
||||
// doWithRetry runs one non-stream request, retrying per the source's
|
||||
// declarative RetryPolicy. The policy is pure data (status list + body
|
||||
// substrings), so the kernel stays vendor-agnostic: a provider that answers a
|
||||
// per-account concurrency cap with a business 4xx can clear by itself while the
|
||||
// client just waits a few seconds instead of seeing an error.
|
||||
func (p *Provider) doWithRetry(ctx context.Context, url, body string, hdr http.Header) (string, int, error) {
|
||||
rp := p.cfg.Retry
|
||||
attempts := 1
|
||||
if rp != nil && rp.MaxAttempts > 1 {
|
||||
attempts = rp.MaxAttempts
|
||||
}
|
||||
var raw string
|
||||
var status int
|
||||
for i := 0; i < attempts; i++ {
|
||||
r, st, err := p.do(ctx, url, body, hdr)
|
||||
if err != nil {
|
||||
return "", 0, err
|
||||
}
|
||||
raw, status = r, st
|
||||
if status == 200 || !rp.ShouldRetry(status, raw) || i == attempts-1 {
|
||||
break
|
||||
}
|
||||
if serr := sleepCtx(ctx, rp.DelayDuration()); serr != nil {
|
||||
return "", 0, serr
|
||||
}
|
||||
}
|
||||
return raw, status, nil
|
||||
}
|
||||
|
||||
// sleepCtx sleeps for d unless the request context ends first.
|
||||
func sleepCtx(ctx context.Context, d time.Duration) error {
|
||||
if d <= 0 {
|
||||
return ctx.Err()
|
||||
}
|
||||
t := time.NewTimer(d)
|
||||
defer t.Stop()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-t.C:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func (p *Provider) doRaw(ctx context.Context, url, body string, hdr http.Header) (*http.Response, error) {
|
||||
httpReq, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader([]byte(body)))
|
||||
if err != nil {
|
||||
|
||||
131
internal/provider/retry_policy_test.go
Normal file
131
internal/provider/retry_policy_test.go
Normal file
@ -0,0 +1,131 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/types"
|
||||
)
|
||||
|
||||
// 上游重试是**内核的通用机制**，判据纯数据（status + body 子串），
|
||||
// 目的是让"账号级限流、等一会儿就好"这类拒绝不冒到客户端。
|
||||
//
|
||||
// 真实事故（2026-10-20）：CodeArts 账户并发会话上限 3 个，连续第 3~4 个请求
|
||||
// 回 400 TM.00001041，最后一个请求结束后约 44~60 秒才释放；流式更糟 ——
|
||||
// 上游用 HTTP 200 + error 帧表达拒绝，内核只能报 `empty stream`。
|
||||
// 参考实现的处理是每 10s 重试一次 chat 直到放行，本测试钉住同一语义。
|
||||
func TestChatRetriesDeclaredUpstreamRejection(t *testing.T) {
|
||||
var hits int32
|
||||
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
n := atomic.AddInt32(&hits, 1)
|
||||
if n <= 2 {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
fmt.Fprint(w, `{"error_code":"TM.00001041","error_msg":"并发会话数已达上限(3个)"}`)
|
||||
return
|
||||
}
|
||||
fmt.Fprint(w, `{"choices":[{"message":{"content":"hi"},"finish_reason":"stop"}]}`)
|
||||
}))
|
||||
defer up.Close()
|
||||
|
||||
s := src("codearts-like", up.URL, "openai", "m1")
|
||||
s.Retry = &config.RetryPolicy{
|
||||
OnStatus: []int{400},
|
||||
OnBodyContains: []string{"TM.00001041"},
|
||||
Delay: "1ms",
|
||||
MaxAttempts: 4,
|
||||
}
|
||||
p := newTestProvider(t, s)
|
||||
resp, err := p.Chat(context.Background(), &types.ChatRequest{
|
||||
Model: "m1",
|
||||
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("hi")}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("命中重试策略后应成功，实得错误: %v", err)
|
||||
}
|
||||
if resp.Content != "hi" {
|
||||
t.Fatalf("content = %q", resp.Content)
|
||||
}
|
||||
if got := atomic.LoadInt32(&hits); got != 3 {
|
||||
t.Fatalf("应恰好尝试 3 次（2 次失败 + 1 次成功），实得 %d", got)
|
||||
}
|
||||
}
|
||||
|
||||
// 同一状态码但 body 不匹配 ⇒ **不许重试**（400 也常用于真正的请求错误，
|
||||
// 无差别重试只会把错误延迟 3 分钟才暴露给用户）。
|
||||
func TestChatDoesNotRetryUnmatchedBody(t *testing.T) {
|
||||
var hits int32
|
||||
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
atomic.AddInt32(&hits, 1)
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
fmt.Fprint(w, `{"error_code":"TM.00001005","error_msg":"无正常可用套餐"}`)
|
||||
}))
|
||||
defer up.Close()
|
||||
|
||||
s := src("codearts-like", up.URL, "openai", "m1")
|
||||
s.Retry = &config.RetryPolicy{
|
||||
OnStatus: []int{400},
|
||||
OnBodyContains: []string{"TM.00001041"},
|
||||
Delay: "1ms",
|
||||
MaxAttempts: 5,
|
||||
}
|
||||
p := newTestProvider(t, s)
|
||||
_, err := p.Chat(context.Background(), &types.ChatRequest{
|
||||
Model: "m1",
|
||||
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("hi")}},
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("不可重试的 body 必须失败")
|
||||
}
|
||||
if got := atomic.LoadInt32(&hits); got != 1 {
|
||||
t.Fatalf("body 不匹配时只能尝试 1 次，实得 %d", got)
|
||||
}
|
||||
}
|
||||
|
||||
// 流式：只重试**握手**。命中策略 → 重试直到 200 才开始派发；一旦开始流式
|
||||
// 就不再重试（重复输出比报错更糟）。
|
||||
func TestChatStreamRetriesHandshakeOnly(t *testing.T) {
|
||||
var hits int32
|
||||
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
n := atomic.AddInt32(&hits, 1)
|
||||
if n == 1 {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
fmt.Fprint(w, `{"error_code":"TM.00001041","error_msg":"并发会话数已达上限(3个)"}`)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "text/event-stream")
|
||||
fmt.Fprint(w, "data: {\"choices\":[{\"delta\":{\"content\":\"hi\"},\"finish_reason\":\"stop\"}]}\n\n")
|
||||
fmt.Fprint(w, "data: [DONE]\n\n")
|
||||
}))
|
||||
defer up.Close()
|
||||
|
||||
s := src("codearts-like", up.URL, "openai", "m1")
|
||||
s.Retry = &config.RetryPolicy{
|
||||
OnStatus: []int{400},
|
||||
OnBodyContains: []string{"TM.00001041"},
|
||||
Delay: "1ms",
|
||||
MaxAttempts: 4,
|
||||
}
|
||||
p := newTestProvider(t, s)
|
||||
ch, err := p.ChatStream(context.Background(), &types.ChatRequest{
|
||||
Model: "m1",
|
||||
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("hi")}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("握手重试后应成功: %v", err)
|
||||
}
|
||||
var content string
|
||||
for ck := range ch {
|
||||
content += ck.Content
|
||||
}
|
||||
if content != "hi" {
|
||||
t.Fatalf("content = %q", content)
|
||||
}
|
||||
if got := atomic.LoadInt32(&hits); got != 2 {
|
||||
t.Fatalf("应握手重试 1 次（共 2 次请求），实得 %d", got)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user