mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-11-12 20:27:23 +00:00
feat(billing): 长上下文阶梯定价(边际费率,与缓存折扣/峰段可叠加)
## 影响面实测 审计里 1000 条请求:**883 条(88.3%)prompt 超过 128k**,中位数 246849, p90 298179,最大 354251。这不是小众场景,是 agent 负载的主流形态。 按「模型 × 源」拆开:其中真正按 token 计价的只有 alittokenplan 的 10 次 (deepseek-v4.1-flash,约 350k prompt),其余走 $70 套餐(commandcode)或 免费源(localzen),记 0 是正确的,对阶梯无影响。 那 10 次当前少算 54.9%($0.3148 → $0.4877)。绝对值不大,但比例严重, 而且这个偏差只会随上下文变长而变大。 ## 边际而非混合平均 阶梯只对**超过阈值的部分**重新计价,不是把整段按高价算。理由是可叠加性: 缓存读取同时享受缓存折扣**和**长上下文阶梯价,这与供应商的计费方式一致。 混合平均必须二选一,选错就正好低估「缓存命中的长请求」——而这恰是缓存 存在的意义所在(本部署缓存命中率 88%)。 超出部分按缓存/新鲜的比例拆分,不重不漏。 ## DSL 两种写法 供应商两种发布方式都支持:绝对费率(OpenAI 报 \$3.00/M past 200k)和 倍数(DeepSeek 报 2x past 128k)。两者都缺必须报错 —— 一个没有费率的阶梯 会编译成一条「看起来处理了长上下文、实际没处理」的规则,静默少算且无迹可寻。 两者都给时以绝对费率为准,因为那是供应商真正报出的数字。 ## 判据 Lua 侧两条(边际性、与缓存折扣叠加)+ Go 侧一条(两种写法与拒绝规则)。 变异自证三次:整段按高价 / 缓存部分不叠加折扣 / 缓存部分不计阶梯, 分别命中 0.40 vs 0.272、0.165 vs 0.2325、0.09 vs 0.2325。 规则编辑器加了三个输入(阈值 + 绝对费率 + 倍数),三项都留空 = 无阶梯 —— 与「阶梯费率为 0」刻意区分,所以是三个独立输入而不是从 0 推断是否存在阶梯。 Co-Authored-By: ModelRouter <noreply@modelrouter.dev>
This commit is contained in:
@ -463,6 +463,9 @@ type BillingToken struct {
|
||||
// setting it to a multiple of Completion reprices the thinking share rather
|
||||
// than billing it twice.
|
||||
Reasoning string `yaml:"reasoning,omitempty" json:"reasoning,omitempty"`
|
||||
// LongContext is the prompt-price step above a token threshold. Nil = one
|
||||
// flat rate, which is what a rule means when it says nothing.
|
||||
LongContext *BillingLongContext `yaml:"long_context,omitempty" json:"long_context,omitempty"`
|
||||
}
|
||||
|
||||
// BillingPeak declares peak-hour pricing.
|
||||
@ -474,6 +477,31 @@ type BillingPeak struct {
|
||||
Hours [][2]int `yaml:"hours,omitempty" json:"hours,omitempty"`
|
||||
}
|
||||
|
||||
// BillingLongContext is a prompt-price step for long inputs.
|
||||
//
|
||||
// Every major provider charges the input tokens ABOVE a threshold at a higher
|
||||
// rate — DeepSeek doubles past 128k, OpenAI and Anthropic step up as well. A
|
||||
// flat prompt price therefore under-bills exactly the traffic an agent workload
|
||||
// generates, and it under-bills it silently: the number is plausible, the bill
|
||||
// adds up, and nothing says the rate was wrong.
|
||||
//
|
||||
// The step applies to the PROMPT only. Output and cache reads are billed at
|
||||
// their own rates regardless of input length, which is how providers publish
|
||||
// it — the higher rate is for the extra context you made the model read, not for
|
||||
// what it wrote back.
|
||||
type BillingLongContext struct {
|
||||
// Above is the token count the step starts at. A request with exactly this
|
||||
// many prompt tokens is still at the base rate; the step covers the tokens
|
||||
// BEYOND it. Inclusive-below matches how the providers document it.
|
||||
Above int `yaml:"above" json:"above"`
|
||||
// Prompt is the USD-per-million rate for the tokens above Above.
|
||||
Prompt string `yaml:"prompt" json:"prompt"`
|
||||
// Multiplier is an alternative to Prompt: a factor applied to the base rate
|
||||
// (DeepSeek's published step is exactly 2x). When both are set Prompt wins,
|
||||
// and setting neither is a validation error rather than a silent no-op.
|
||||
Multiplier float64 `yaml:"multiplier,omitempty" json:"multiplier,omitempty"`
|
||||
}
|
||||
|
||||
// Validate rejects a BillingDSL that could not work as written.
|
||||
func (b *BillingDSL) Validate() error {
|
||||
if b == nil {
|
||||
|
||||
Reference in New Issue
Block a user