Files
ModelRouter/internal/config/config.go
JianFeeeee 25970a2135 refactor(scheduler)!: 移除 priority 废弃字段 + 一通道一源 + discover 复用凭据
用户判据:「priority 直接移除」「即使带 key 也没有 /v1/responses」
「channel 插件的问题还没完全解决」「生图和聊天合并到一个导航项」

## 一、删除 priority(它从未参与跨源调度)

`config.Model.Priority` 号称"高者优先参与 AUTO 选源",**那是错的**:
AUTO 链由 `config.ModelScope.Tier` 驱动(scheduler.Chain),priority 只被
`bestChatModel/bestImageModel` 用于"本源内部该发哪个模型",与跨源调度无关。
把它当调度依据会让人以为拖个数字就能改顺序,改完毫无效果。

- 删 `config.Model.Priority` 字段与 `bestChatModel/bestImageModel` 的优先级比较
- 删 `seedAuto()`:**AUTO 链为空时网关应直接 503**,不能凭空播种用户没配过的档位
- 删 UI `buildLanes` 的 discovery 回退(按源 priority 自动铺开)与 sourceRows
- 清掉 config.yaml 里 140 行 priority

## 二、一通道 = 一个适配器 = 一个源(channels 插件)

旧设计「一账号一源」(`channels-<平台>-<12位hex>`) 有三个结构性后果:
1. 三个账号 ⇒ 三个源、三份同名模型,用户看到三行无法区分的东西
2. 谁先谁后**取决于注册顺序,不是余额**(rankAccounts 只用于展示,从未接进同步路径)
3. 换号要改源名 ⇒ 用户书签里的 `源:模型` 全部失效

新设计:
- 源名只跟通道绑定:`channels-<平台>`,换号 = 同名 add_source 覆盖 api_key
  (内核 AddSource 是 UpsertSourceInYAML + 内存替换),对客户端完全透明
- `pickAccountForSource()`:凭据可用 → 余额分档(先快作废后永久)→
  跳过已标记失效的号。没有 balance 声明的平台不据此排除(不因"不会查"判死)
- `request_end` 钩子:真实请求失败且是**终态**错误(需充值/封禁/凭据永久失效)
  时给当前账号打 termErr,下次选号跳过;成功则清标记。
  ⚠️ 只认终态 —— 429/超时/抖动都**不**标记(那会把好号一个个废掉)
- `refresh_creds` 每 30 分钟巡检后重新选号(兜底心跳,纯本地判定)
- 迁移:同步时自动清掉旧版按账号拆的源(按已知平台前缀匹配,不用贪婪 match)
- 顺带修:`refresh_creds` 漏认 `refresh_fn` ⇒ codearts 的 DPoP 续期永不触发

## 三、discover 支持用已存凭据(真实缺口)

刷新已有源的模型目录,此前必须让运维**再粘一次明文 key** —— 而库里是
`enc:v1:` 密文,WebUI 也不回显明文。结果就是"想刷新 → 得重新申请 key",
很多人干脆不刷,源的目录越放越旧(实例:某源 3 个生图模型从未登记)。

- `{"name":"oin"}` ⇒ 从已存源取 base_url + 解密凭据 + adapter
- 安全边界:已要求 admin;只能探测**本网关已配置的**源(不能拿任意 URL+key
  做探测,那就成了 SSRF/密钥外带);响应不回显 key

## 四、image_models 显式声明(补元数据缺失)

`discoveredModelKind` 只看上游 modality 字段,并刻意**不从模型名推断**
(错误的 kind 会毒死 AUTO 链)。但有些网关的 /v1/models 只返回
`{id,object,created,owned_by}` —— 零 modality,于是生图模型全落成 chat,
生图端点永远 `no image source configured`。

不解这个两难:默认仍完全按元数据(不猜),但允许 `image_models` **显式声明**。
写错的 id 明确报 400,不静默造一个不存在的模型。

## 五、UI:对话与生图合并为一个导航项

两者本是同一件事的两种输入(都选模型、都发请求、都读结果),只是 kind 与端点
不同。合并只发生在**导航层**:模型下拉仍按 kind 分组/禁选(把图片模型选进 chat
会请求到错误端点)。两个面板同时存在于 DOM,切换只改 display —— 来回切不会丢
已输入的文字、对话历史或已生成的图。

## 测试

- `TestUI_ImageModeIsReachableFromChatTab`:改为钉"合并后仍可达"(分段控件、
  切换函数、面板容器、端点),并断言**旧 tab 三件套必须消失**(否则 goTab 会找
  一个不存在的面板)
- 测试夹具显式建 AUTO 链(内核不再 seedAuto):默认一源一 tier 倒序
  (TestChatAUTO 验优先级);TestChatAutoChainTierFailover 自己覆盖成同 tier
  (验 tier 内轮转)—— 两者验不同机制,不能共用一个默认
- core 的一批断言从"按账号 id 找源"改为"按平台名找源"

gateway/config/provider 全绿;core 的 channels 用例全绿。
TestRealMutants 需工作树干净后才跑(其自带的变异残留防护)。

BREAKING CHANGE: config.Model.Priority 已删除。旧的 priority 字段会被 YAML
解码静默忽略(内核不校验未知字段),因此既有 config.yaml 仍可加载,但该字段
不再有任何作用 —— 调度顺序请通过 AUTO 链的 tier 表达。
2026-10-19 16:26:59 +08:00

706 lines
27 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

// Package config loads the gateway YAML configuration plus a runtime overlay
// (web UI edits) and resolves them into sources with per-model priority.
package config
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"strconv"
"strings"
"time"
"gopkg.in/yaml.v3"
)
// Config is the top-level gateway configuration.
type Config struct {
Path string `yaml:"-" json:"-"`
Listen string `yaml:"listen"`
GatewayKeys []string `yaml:"gateway_keys"`
DefaultModel string `yaml:"default_model"` // e.g. "AUTO" or a model id
AdapterDir string `yaml:"adapter_dir"`
PluginDir string `yaml:"plugin_dir,omitempty"` // Lua plugin directory (optional; hooks + WebUI extensions)
RuntimeFile string `yaml:"runtime_file"`
TLSCertFile string `yaml:"tls_cert_file,omitempty"` // PEM cert; when set together with tls_key_file, serve HTTPS
TLSKeyFile string `yaml:"tls_key_file,omitempty"` // PEM private key
PublicBaseURL string `yaml:"public_base_url,omitempty"` // external base for generated config snippets; default inferred from request
Sources []Source `yaml:"sources"`
Auto []ModelScope `yaml:"auto,omitempty"` // AUTO 调度链规则(WebUI 优先级页编辑,chat)
AutoImage []ModelScope `yaml:"auto_image,omitempty"` // AUTO 生图调度链规则(WebUI 优先级页·生图)
Keys []GWKey `yaml:"keys,omitempty"` // 网关密钥(WebUI 密钥页管理)
// BillingDSL declares per-URL pricing profiles for the billing plugin.
//
// Declarative, because hand-writing the prices JSON was error-prone in ways
// that looked successful: the first PUT went out without the required
// `prices` wrapper and silently REPLACED the accumulated totals with the
// price table; the second used a peak-window shape the plugin does not read,
// so peak traffic was billed at off-peak rates — no error, the bill just
// quietly halved. A schema-checked config file catches both at load.
BillingDSL *BillingDSL `yaml:"billing,omitempty"`
// PluginVisibility overrides the audience a plugin declared for itself,
// keyed by plugin name: "admin" hides a plugin (and its state) from user
// keys, "all" widens one that shipped admin-only, "user,admin" names roles
// explicitly. An absent name leaves the plugin's own declaration in force.
//
// This exists because a plugin author cannot know the operator's policy, and
// the operator should not have to edit a bundled .lua to change one — the
// next upgrade overwrites it.
PluginVisibility map[string]string `yaml:"plugin_visibility,omitempty" json:"plugin_visibility,omitempty"`
// box seals credentials (sources' api_key/headers, keys' key) at rest.
// In-memory values are always plaintext; only the bytes on disk are sealed.
// Wired by AttachSecretBox — Load leaves it nil so `-check` and tests stay
// filesystem-free.
box *SecretBox
}
// Defaults applied to any source (YAML or runtime) that leaves a field unset.
const (
DefaultSourceTimeout = 120 * time.Second
DefaultSourceQueueTimeout = 60 * time.Second
DefaultSourceConcurrency = 8
)
// Model is a single exposed model id bound to a source.
//
// ★ 曾经有 `Priority int` 字段,并号称"高者优先参与 AUTO 选源"。**那是错的**:
//
// AUTO 链由 `config.ModelScope.Tier` 驱动(见 scheduler.Chain),而 priority
// 只被 `bestChatModel/bestImageModel` 用于"本源码内部该发哪个模型",
// 与跨源调度毫无关系。把它当成调度依据会让人误以为改个数字就能改顺序,
// 改完发现毫无效果 —— 已彻底删除,避免再次误导。
type Model struct {
ID string `yaml:"id" json:"id"`
Kind string `yaml:"kind" json:"kind"` // "chat" (default) | "image"
Meta map[string]interface{} `yaml:"meta" json:"meta,omitempty"`
}
// Source describes a single upstream LLM provider.
type Source struct {
Name string `yaml:"name" json:"name"`
BaseURL string `yaml:"base_url" json:"base_url"`
APIKey string `yaml:"api_key" json:"api_key"`
APIKeyEnv string `yaml:"api_key_env,omitempty" json:"-"` // reference to an env var holding the key (overrides api_key)
Adapter string `yaml:"adapter" json:"adapter"`
Endpoint string `yaml:"endpoint" json:"endpoint,omitempty"` // chat endpoint override
// StreamEndpoint overrides the CHAT endpoint for streaming requests. Most
// upstreams stream on the same path; Gemini is the exception (same path,
// different verb). Declaring it here keeps that knowledge out of the kernel,
// which only compares opaque strings.
StreamEndpoint string `yaml:"endpoint_stream" json:"endpoint_stream,omitempty"`
ImageEndpoint string `yaml:"image_endpoint" json:"image_endpoint,omitempty"` // image endpoint override
Models []Model `yaml:"models" json:"models"`
Headers map[string]string `yaml:"headers" json:"headers,omitempty"`
// ProxyURL routes this source's HTTP(S) traffic through an explicit proxy
// (e.g. http://127.0.0.1:7890). Empty = direct connection. Needed for
// upstreams behind geo/IP blocks that only respond through a proxy.
ProxyURL string `yaml:"proxy_url" json:"proxy_url,omitempty"`
Meta map[string]interface{} `yaml:"meta" json:"meta,omitempty"`
Temperature float64 `yaml:"temperature" json:"temperature,omitempty"`
MaxTokens int `yaml:"max_tokens" json:"max_tokens,omitempty"`
Timeout time.Duration `yaml:"timeout" json:"-"`
MaxConcurrent int `yaml:"max_concurrent" json:"max_concurrent"`
QueueTimeout time.Duration `yaml:"queue_timeout" json:"-"`
RPM int `yaml:"rpm" json:"rpm,omitempty"` // optional requests-per-minute cap (0 = unlimited)
}
// Load reads and validates a config file. When the file does not exist yet a
// default config is generated at that path first (first-run bootstrap), so a
// fresh binary just works: `llmsproxy -config /path/to/config.yaml`.
func Load(path string) (*Config, error) {
if _, err := EnsureDefault(path); err != nil {
return nil, err
}
data, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var cfg Config
if err := yaml.Unmarshal(data, &cfg); err != nil {
return nil, fmt.Errorf("parse config: %w", err)
}
cfg.Path = path
if err := cfg.ApplyDefaults(); err != nil {
return nil, err
}
return &cfg, nil
}
// EnsureDefault creates a default config file at path when it does not exist
// yet and returns whether it was created. The repo ships no config file
// (config files carry real keys); the binary generates one per install with a
// fresh random admin key. An existing file is never touched.
func EnsureDefault(path string) (bool, error) {
if _, err := os.Stat(path); err == nil {
return false, nil
} else if !os.IsNotExist(err) {
return false, err
}
if err := writeDefaultConfig(path); err != nil {
return false, err
}
return true, nil
}
// writeDefaultConfig writes a minimal, safe-by-default config: loopback-only
// listen and a fresh random admin key. No upstream sources are preconfigured
// (the repo ships none — add them via WebUI or config.yaml). adapter_dir /
// runtime_file live next to the config file so the binary works regardless of
// the working directory it is started from.
func writeDefaultConfig(path string) error {
key, err := NewGatewayKey()
if err != nil {
return fmt.Errorf("generate gateway key: %w", err)
}
dir := filepath.Dir(path)
abs, err := filepath.Abs(dir)
if err != nil {
abs = dir
}
cfg := Config{
Listen: "127.0.0.1:8080",
GatewayKeys: []string{key},
DefaultModel: "AUTO",
AdapterDir: filepath.Join(abs, "adapters"),
PluginDir: filepath.Join(abs, "plugins"),
RuntimeFile: filepath.Join(abs, "runtime.json"),
}
out, err := yaml.Marshal(&cfg)
if err != nil {
return fmt.Errorf("marshal default config: %w", err)
}
// The config holds the plaintext admin key — restrict permissions.
if err := os.MkdirAll(abs, 0755); err != nil {
return fmt.Errorf("mkdir config dir: %w", err)
}
return os.WriteFile(path, out, 0600)
}
// RemoveSourceFromYAML deletes the named source entry from the config file so
// the delete is a real one (no tombstone needed). Uses yaml.Node to preserve
// the rest of the file's comments and formatting.
func RemoveSourceFromYAML(path, name string) error {
data, err := os.ReadFile(path)
if err != nil {
return err
}
var doc yaml.Node
if err := yaml.Unmarshal(data, &doc); err != nil {
return err
}
content := doc.Content
if len(content) == 0 {
return nil
}
root := content[0]
if root.Kind != yaml.MappingNode {
return nil
}
for i := 0; i+1 < len(root.Content); i += 2 {
key, val := root.Content[i], root.Content[i+1]
if key.Value != "sources" || val.Kind != yaml.SequenceNode {
continue
}
kept := val.Content[:0]
for _, item := range val.Content {
if item.Kind != yaml.MappingNode {
continue
}
found := false
for j := 0; j+1 < len(item.Content); j += 2 {
if item.Content[j].Value == "name" && item.Content[j+1].Value == name {
found = true
break
}
}
if !found {
kept = append(kept, item)
}
}
val.Content = kept
}
out, err := yaml.Marshal(&doc)
if err != nil {
return err
}
return os.WriteFile(path, out, 0644)
}
// UpsertSourceInYAML adds or updates a source entry in the YAML config file.
// Uses yaml.Node to preserve the rest of the file's comments and formatting.
// When box is non-nil the source's credentials are sealed before writing, so a
// newly added source never lands in the file as plaintext.
func UpsertSourceInYAML(path, name string, src Source, box *SecretBox) error {
if path == "" {
return fmt.Errorf("config path is empty")
}
if err := sealSource(&src, box); err != nil {
return err
}
data, err := os.ReadFile(path)
if err != nil {
return err
}
var doc yaml.Node
if err := yaml.Unmarshal(data, &doc); err != nil {
return err
}
content := doc.Content
if len(content) == 0 {
return nil
}
root := content[0]
if root.Kind != yaml.MappingNode {
return nil
}
// Build yaml.Node for the new source value
srcRaw, _ := yaml.Marshal(src)
var srcNode yaml.Node
yaml.Unmarshal(srcRaw, &srcNode)
for i := 0; i+1 < len(root.Content); i += 2 {
key, val := root.Content[i], root.Content[i+1]
if key.Value != "sources" || val.Kind != yaml.SequenceNode {
continue
}
replaced := false
for _, item := range val.Content {
if item.Kind != yaml.MappingNode {
continue
}
for j := 0; j+1 < len(item.Content); j += 2 {
if item.Content[j].Value == "name" && item.Content[j+1].Value == name {
// Replace the existing entry with the new source
item.Content = srcNode.Content[0].Content
replaced = true
break
}
}
if replaced {
break
}
}
if !replaced {
val.Content = append(val.Content, srcNode.Content[0])
}
break
}
out, err := yaml.Marshal(&doc)
if err != nil {
return err
}
return os.WriteFile(path, out, 0644)
}
// Save writes the current config back to the YAML file (preserving comments
// via yaml.Node round-trip when possible, or full marshaling as fallback).
// Credentials are sealed on the way out and the in-memory copy is restored to
// plaintext afterwards, so callers keep working with usable values.
func (c *Config) Save() error {
if c.Path == "" {
return fmt.Errorf("config path is empty")
}
if err := c.sealInPlace(c.box); err != nil {
return err
}
out, err := yaml.Marshal(c)
c.unsealAfterWrite(c.box)
if err != nil {
return fmt.Errorf("marshal config: %w", err)
}
return os.WriteFile(c.Path, out, 0644)
}
// ApplyDefaults sets missing values and validates the config.
func (c *Config) ApplyDefaults() error {
if c.Listen == "" {
c.Listen = ":8080"
}
if c.AdapterDir == "" {
c.AdapterDir = "adapters"
}
// PluginDir is deliberately NOT defaulted: with no plugins a missing
// directory would be created and seeded on every start for nothing. An
// operator opts in by setting it (or by dropping files into it).
if c.RuntimeFile == "" {
c.RuntimeFile = "runtime.json"
}
if c.DefaultModel == "" {
c.DefaultModel = "AUTO"
}
seen := map[string]bool{}
for i := range c.Sources {
s := &c.Sources[i]
if s.Name == "" {
return fmt.Errorf("config: sources[%d] missing name", i)
}
if s.BaseURL == "" {
return fmt.Errorf("config: source %s missing base_url", s.Name)
}
if s.Adapter == "" {
s.Adapter = "openai"
}
if s.Timeout == 0 {
s.Timeout = DefaultSourceTimeout
}
if s.QueueTimeout == 0 {
s.QueueTimeout = DefaultSourceQueueTimeout
}
if s.MaxConcurrent == 0 {
s.MaxConcurrent = DefaultSourceConcurrency
}
if seen[s.Name] {
return fmt.Errorf("config: duplicate source name %q", s.Name)
}
seen[s.Name] = true
for j := range s.Models {
m := &s.Models[j]
if m.ID == "" {
return fmt.Errorf("config: source %s has a model without id", s.Name)
}
// Allow same model ID on multiple sources — disambiguation is via
// "source-model" / "source:model" / "source/model" pinning and
// the AUTO chain slots carry explicit (model, source) pairs.
}
}
return nil
}
// RuntimeConfig is the legacy runtime file format (kept for migration only).
type RuntimeConfig struct {
Sources []Source `json:"sources,omitempty"`
SourceTemplates []SourceTemplate `json:"source_templates,omitempty"`
DeletedSources []string `json:"deleted_sources,omitempty"`
DeletedAdapters []string `json:"deleted_adapters,omitempty"`
PresetTemplates []string `json:"preset_templates,omitempty"` // preset names the user has seen (or deleted) — never re-seeded
Keys []GWKey `json:"keys,omitempty"`
Auto []ModelScope `json:"auto,omitempty"`
AutoImage []ModelScope `json:"auto_image,omitempty"`
}
// SourceTemplate stores reusable source configuration (everything except
// name and api_key) so the WebUI can spin up multiple key-bearing sources
// from one shared template.
type SourceTemplate struct {
Name string `json:"name"`
BaseURL string `json:"base_url"`
Adapter string `json:"adapter"`
Endpoint string `json:"endpoint,omitempty"`
StreamEndpoint string `json:"endpoint_stream,omitempty"`
ImageEndpoint string `json:"image_endpoint,omitempty"`
Models []Model `json:"models"`
Headers map[string]string `json:"headers,omitempty"`
// ProxyURL routes this source's HTTP(S) traffic through an explicit proxy.
ProxyURL string `json:"proxy_url,omitempty"`
Meta map[string]interface{} `json:"meta,omitempty"`
Temperature float64 `json:"temperature,omitempty"`
MaxTokens int `json:"max_tokens,omitempty"`
MaxConcurrent int `json:"max_concurrent,omitempty"`
RPM int `json:"rpm,omitempty"`
}
// GWKey is a gateway API key persisted in the config file. Role is "admin"
// (full management) or "user" (sees only its own key); Models is the allowed
// model scope with per-model token quota (0 = unlimited).
type GWKey struct {
Key string `yaml:"key" json:"key"`
Role string `yaml:"role" json:"role"`
Name string `yaml:"name,omitempty" json:"name,omitempty"`
// Auto is this key's own AUTO scheduling chain. It overrides the global
// cfg.Auto for requests authenticated with this key; nil (or an empty
// list) means "inherit the global chain", so keys created before per-key
// chains existed keep working untouched.
Auto []ModelScope `yaml:"auto,omitempty" json:"auto,omitempty"`
Models []ModelScope `yaml:"models,omitempty" json:"models,omitempty"`
Note string `yaml:"note,omitempty" json:"note,omitempty"`
CreatedAt int64 `yaml:"created_at,omitempty" json:"created_at,omitempty"`
Seed bool `yaml:"seed,omitempty" json:"seed,omitempty"` // true if migrated from config gateway_keys
}
// HasOwnAuto reports whether the key declares its own AUTO chain rather than
// inheriting the global one. An explicitly empty list still counts as "no own
// chain" so a user can clear an override back to the global default.
func (k GWKey) HasOwnAuto() bool { return len(k.Auto) > 0 }
// BillingDSL holds declarative per-URL pricing profiles for the billing plugin.
//
// Profiles are the "let the user pick" axis: the same upstream URL can be
// declared under multiple profiles, and the operator chooses which is active.
// Switching recomputes and re-injects the billing plugin's prices table.
type BillingDSL struct {
// Active is the id of the profile in effect. Empty => the first profile.
Active string `yaml:"active,omitempty" json:"active,omitempty"`
Profiles []BillingProfile `yaml:"profiles,omitempty" json:"profiles,omitempty"`
}
// BillingProfile is one named set of per-URL pricing rules.
type BillingProfile struct {
ID string `yaml:"id" json:"id"`
Label string `yaml:"label,omitempty" json:"label,omitempty"`
// Default applies when no rule matches: "free" or "unpriced".
Default string `yaml:"default,omitempty" json:"default,omitempty"`
Rules []BillingRule `yaml:"rules,omitempty" json:"rules,omitempty"`
}
// BillingRule declares the pricing for everything under one URL.
type BillingRule struct {
// URL matches a source's base_url. "*" = any unmatched URL.
URL string `yaml:"url" json:"url"`
// Mode is one of: free, token, subscription, unpriced.
Mode string `yaml:"mode" json:"mode"`
Currency string `yaml:"currency,omitempty" json:"currency,omitempty"`
// Subscription is the flat monthly cost (for mode=subscription).
Subscription float64 `yaml:"subscription,omitempty" json:"subscription,omitempty"`
// Models declares per-token prices when mode=token.
Models map[string]BillingToken `yaml:"models,omitempty" json:"models,omitempty"`
// Free is true when mode=free (kept for explicitness).
// Peak applies to ALL token-priced requests under this URL.
Peak *BillingPeak `yaml:"peak,omitempty" json:"peak,omitempty"`
}
// BillingToken is the per-model price.
type BillingToken struct {
Prompt string `yaml:"prompt" json:"prompt"` // USD per million tokens, e.g. "0.15"
Completion string `yaml:"completion" json:"completion"` // USD per million tokens
CacheDiscount *float64 `yaml:"cache_discount,omitempty" json:"cache_discount,omitempty"`
// Reasoning is the USD-per-million rate for thinking tokens, for the rare
// provider that charges them differently from ordinary output. Omit it and
// reasoning is billed at Completion — which is what OpenAI, DeepSeek and
// Anthropic all do, and what every source on this deployment reports.
//
// It is a rate for a SUBSET of completion tokens, never an extra charge: the
// billing plugin applies only the difference between this and Completion, so
// setting it to a multiple of Completion reprices the thinking share rather
// than billing it twice.
Reasoning string `yaml:"reasoning,omitempty" json:"reasoning,omitempty"`
// LongContext is the prompt-price step above a token threshold. Nil = one
// flat rate, which is what a rule means when it says nothing.
LongContext *BillingLongContext `yaml:"long_context,omitempty" json:"long_context,omitempty"`
}
// BillingPeak declares peak-hour pricing.
type BillingPeak struct {
Multiplier float64 `yaml:"multiplier" json:"multiplier"`
// Weekdays: 1=Mon..5=Fri (Lua os.date !%w, 0=Sun)
Weekdays []int `yaml:"weekdays,omitempty" json:"weekdays,omitempty"`
// Hours: each pair is [lo,hi] inclusive
Hours [][2]int `yaml:"hours,omitempty" json:"hours,omitempty"`
}
// BillingLongContext is a prompt-price step for long inputs.
//
// Every major provider charges the input tokens ABOVE a threshold at a higher
// rate — DeepSeek doubles past 128k, OpenAI and Anthropic step up as well. A
// flat prompt price therefore under-bills exactly the traffic an agent workload
// generates, and it under-bills it silently: the number is plausible, the bill
// adds up, and nothing says the rate was wrong.
//
// The step applies to the PROMPT only. Output and cache reads are billed at
// their own rates regardless of input length, which is how providers publish
// it — the higher rate is for the extra context you made the model read, not for
// what it wrote back.
type BillingLongContext struct {
// Above is the token count the step starts at. A request with exactly this
// many prompt tokens is still at the base rate; the step covers the tokens
// BEYOND it. Inclusive-below matches how the providers document it.
Above int `yaml:"above" json:"above"`
// Prompt is the USD-per-million rate for the tokens above Above.
Prompt string `yaml:"prompt" json:"prompt"`
// Multiplier is an alternative to Prompt: a factor applied to the base rate
// (DeepSeek's published step is exactly 2x). When both are set Prompt wins,
// and setting neither is a validation error rather than a silent no-op.
Multiplier float64 `yaml:"multiplier,omitempty" json:"multiplier,omitempty"`
}
// Validate rejects a BillingDSL that could not work as written.
func (b *BillingDSL) Validate() error {
if b == nil {
return nil
}
ids := map[string]bool{}
for i := range b.Profiles {
p := &b.Profiles[i]
if p.ID == "" {
return fmt.Errorf("billing profile [%d] has no id", i)
}
if ids[p.ID] {
return fmt.Errorf("billing profile id %q appears more than once", p.ID)
}
ids[p.ID] = true
if p.Default != "" && p.Default != "free" && p.Default != "unpriced" {
return fmt.Errorf("profile %q: default must be free or unpriced, got %q", p.ID, p.Default)
}
for j := range p.Rules {
r := &p.Rules[j]
if r.URL == "" {
return fmt.Errorf("profile %q rule [%d]: url is required", p.ID, j)
}
switch r.Mode {
case "free", "token", "subscription", "unpriced":
default:
return fmt.Errorf("profile %q rule [%d]: mode must be free/token/subscription/unpriced, got %q", p.ID, j, r.Mode)
}
if r.Mode == "token" {
for m, t := range r.Models {
if t.Prompt == "" || t.Completion == "" {
return fmt.Errorf("profile %q model %q: prompt and completion are required for token mode", p.ID, m)
}
if _, err := ParseFloatUSDPerM(t.Prompt); err != nil {
return fmt.Errorf("profile %q model %q prompt: %w", p.ID, m, err)
}
if _, err := ParseFloatUSDPerM(t.Completion); err != nil {
return fmt.Errorf("profile %q model %q completion: %w", p.ID, m, err)
}
}
}
}
}
// Active must resolve to a profile id.
if b.Active != "" && !ids[b.Active] {
return fmt.Errorf("billing.active = %q but no profile has that id", b.Active)
}
return nil
}
// Resolve returns the profile with the given id; empty id (or an id that does
// not exist) falls back to the FIRST profile. A config with profiles but no
// active marker should mean "use the one I wrote first" rather than "price
// nothing", which is what a nil here would silently do.
func (b *BillingDSL) Resolve(id string) *BillingProfile {
if b == nil || len(b.Profiles) == 0 {
return nil
}
if id != "" {
for i := range b.Profiles {
if b.Profiles[i].ID == id {
return &b.Profiles[i]
}
}
}
return &b.Profiles[0]
}
// parseFloatUSDPerM parses a USD-per-million price string. Returns the
// per-single-token rate the plugin expects (price / 1e6).
//
// Kept as a string in the DSL rather than a float: "0.15" survives round-trips
// through YAML editors, and a malformed value must fail VALIDATION (with the
// file and field named) rather than parse as 0 and under-bill silently — which
// is exactly the failure mode that motivated the DSL.
func ParseFloatUSDPerM(s string) (float64, error) {
f, err := strconv.ParseFloat(strings.TrimSpace(s), 64)
if err != nil {
return 0, fmt.Errorf("not a number: %q", s)
}
if f < 0 {
return 0, fmt.Errorf("negative price: %q", s)
}
return f / 1e6, nil
}
// KeyQuota is retained only to carry a scope entry's caps through the admin
// API. Quotas are per model, never per key: there is deliberately no key-wide
// total, so exhausting one model's budget never blocks the others.
type KeyQuota struct {
TokenQuota int64 `json:"token_quota"`
ReqQuota int64 `json:"req_quota"`
Period string `json:"period"`
Hours int64 `json:"hours"`
}
// NormalizeRole defaults an empty role to "user", so a key can never end up in
// a state where no role means "neither admin nor user".
func NormalizeRole(role string) string {
if role == "admin" {
return "admin"
}
return "user"
}
// Validate rejects a quota configuration that could not work as written. A
// period is only meaningful when at least one cap is set, and a cap of zero
// means "unlimited" rather than "deny everything", so those are the only two
// things worth rejecting.
func (q KeyQuota) Validate() error {
if q.TokenQuota < 0 {
return fmt.Errorf("token_quota must be >= 0 (0 = unlimited)")
}
if q.ReqQuota < 0 {
return fmt.Errorf("req_quota must be >= 0 (0 = unlimited)")
}
if q.Hours < 0 {
return fmt.Errorf("hours must be >= 0")
}
if q.TokenQuota > 0 || q.ReqQuota > 0 {
if err := ValidatePeriod(q.Period, q.Hours); err != nil {
return err
}
}
return nil
}
// ValidatePeriod accepts the quota period vocabulary: "" (never resets),
// "hour", "week", "month", or "nhour" with hours >= 1. An unknown period is
// rejected rather than silently treated as "never resets", which would turn a
// typo into an all-time quota — the opposite of what the operator typed.
func ValidatePeriod(period string, hours int64) error {
switch period {
case "", "hour", "week", "month":
return nil
case "nhour":
if hours < 1 {
return fmt.Errorf("period %q needs hours >= 1", period)
}
return nil
}
return fmt.Errorf("period must be one of \"\", hour, week, month, nhour (got %q)", period)
}
// ModelScope is one allowed model for a key, or one AUTO scheduling slot.
// Its TokenQuota and ReqQuota cap THAT entry only, independently of every
// other entry on the same key: a model that runs out of budget stops being
// served while the key's other models keep working. TokenQuota 0 / ReqQuota 0
// = unlimited. Period "" = never resets; "hour"/"week"/"month" are fixed
// windows; "nhour" uses Hours.
type ModelScope struct {
Model string `yaml:"model" json:"model"`
Source string `yaml:"source,omitempty" json:"source,omitempty"` // optional: pin to one upstream source; "" = any source
Tier int `yaml:"tier,omitempty" json:"tier,omitempty"`
TokenQuota int64 `yaml:"token_quota" json:"token_quota"`
ReqQuota int64 `yaml:"req_quota,omitempty" json:"req_quota,omitempty"`
Period string `yaml:"period,omitempty" json:"period,omitempty"`
Hours int64 `yaml:"hours,omitempty" json:"hours,omitempty"`
}
// UnmarshalJSON accepts both the legacy "model-id" string form and the
// {"model":"...","token_quota":N} object form so old runtime files keep
// loading.
func (m *ModelScope) UnmarshalJSON(b []byte) error {
var s string
if err := json.Unmarshal(b, &s); err == nil {
m.Model = s
return nil
}
var o struct {
Model string `json:"model"`
Source string `json:"source"`
Tier int `json:"tier"`
TokenQuota int64 `json:"token_quota"`
ReqQuota int64 `json:"req_quota"`
Period string `json:"period"`
Hours int64 `json:"hours"`
}
if err := json.Unmarshal(b, &o); err != nil {
return err
}
m.Model = o.Model
m.Source = o.Source
m.Tier = o.Tier
m.TokenQuota = o.TokenQuota
m.ReqQuota = o.ReqQuota
m.Period = o.Period
m.Hours = o.Hours
return nil
}