mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-11-08 10:17:21 +00:00
Compare commits
7 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 12af22fda6 | |||
| 9114468753 | |||
| 611e975456 | |||
| 6733534282 | |||
| 499f0cac2f | |||
| 7fb8f96b82 | |||
| 6a34cd6f86 |
4
.gitignore
vendored
4
.gitignore
vendored
@ -26,7 +26,3 @@ cmd/gui/dist/
|
||||
|
||||
# local design/working notes (not part of the shipped repo)
|
||||
/plan.md
|
||||
|
||||
# scratch build/probe dirs used while debugging (GOTMPDIR, throwaway binaries)
|
||||
.build-work/
|
||||
.probe/
|
||||
|
||||
81
README.md
81
README.md
@ -24,7 +24,6 @@
|
||||
### 强大的多租户调度能力
|
||||
|
||||
- **多密钥多租户**：支持无限密钥，每个密钥独立角色、模型范围、Token 配额、重置周期
|
||||
- **按模型配额**：每把 key 的每个模型单独配 token + 请求数配额与重置周期（小时/周/月/自定义 N 小时）；一个模型用满只影响该模型，同 key 其它模型照常；耗尽返 429 + `Retry-After` 并点名模型，admin key 永不受限
|
||||
- **AUTO 智能调度**：基于优先级档位的分级调度，同优先级源自动轮询负载均衡，故障自动毫秒级故障转移
|
||||
- **自愈冷却**：冷却上限 5 分钟，过半后放行 1 个探测请求，上游/额度恢复即刻回归轮询，无需等满冷却窗口
|
||||
- **Token 配额管理**：精确到模型级别的 Token 配额控制，支持小时/周/月/自定义小时周期自动重置
|
||||
@ -179,81 +178,6 @@ sources:
|
||||
- 客户端用任意一个已授权的密钥明文作为 Bearer（`Authorization: Bearer <key>`）。
|
||||
- 删除密钥即从运行时存储移除，立即失效。
|
||||
|
||||
#### 密钥用量配额
|
||||
|
||||
**配额按模型单独设置**：每个密钥的 `models[]` 里，每一条模型范围各自带一份
|
||||
token 配额与请求数配额。一个模型用满只影响该模型，同一密钥的其它模型照常工作。
|
||||
|
||||
```yaml
|
||||
keys:
|
||||
- key: sk-gw-<hex>
|
||||
role: user
|
||||
name: agent-alice
|
||||
models:
|
||||
- model: deepseek-v4-flash
|
||||
token_quota: 1000000 # 本周期内该 key 用这个模型的 token 预算
|
||||
req_quota: 20000 # 本周期内的请求次数
|
||||
period: nhour # "" | hour | week | month | nhour
|
||||
hours: 6 # 仅 nhour
|
||||
- model: AUTO
|
||||
token_quota: 5000000 # AUTO 也是一条独立配额
|
||||
period: day # ← 这种写法会被拒绝（词表只有 hour/week/month/nhour）
|
||||
- model: kimi-k3 # 未列配额 = 无限
|
||||
```
|
||||
|
||||
- **没有「整钥总配额」**：这是刻意的设计。整钥总额会让「把 A 模型的额度挪给
|
||||
B 模型」变成一次全局重分配；按模型独立计费则每个模型各自可控，运维可以
|
||||
看出哪个模型吃掉了预算。
|
||||
- `period` 词表：空 = 永不过期（累计总量），`hour` / `week` / `month` = 固定窗口，
|
||||
`nhour` + `hours` = 自定义小时数。**拼错的周期在写入时就被拒**，不会静默变成
|
||||
永不过期。
|
||||
- 配额严格按密钥隔离，且**同一密钥内按模型隔离**：A 密钥用满 `m1` 不会消耗
|
||||
B 密钥的额度，同一密钥的 `m2` 也不受影响。
|
||||
- 配额统计含聊天、流式、生图，跨重启从审计日志回放（保留 40 天，覆盖最长的
|
||||
month 窗口）。
|
||||
- 配额耗尽返回 **429 + `Retry-After`**（`rate_limit_exceeded`），消息里点名是哪个
|
||||
模型用满了，客户端可等窗口重置后自动恢复；模型越权才是 403。
|
||||
**admin 密钥永不受配额限制**（其 scope 上的配额也不强制），避免把管理员
|
||||
锁在门外。
|
||||
- 窗口用量按整点小时分桶统计，实际释放比配置窗口最多晚 1 小时（配额宁可晚释放
|
||||
也不超发）。
|
||||
- `PUT /api/keys/{key}` 提交 `models` 即同时提交它们的配额（配额就是 scope 的一部分，
|
||||
不存在会与模型列表脱节的第二份预算）。显式 `0` = 解除该模型的限制。
|
||||
|
||||
##### 配额拒绝 vs 容量拒绝：两种「拒绝」含义不同
|
||||
|
||||
配额在**入口**检查，不占用任何上游槽位，因此是廉价拒绝；容量不足则要走
|
||||
`busyWait` 有界等待后才返回 503。实测（容量 8、0.6s/请求、100 并发）：
|
||||
|
||||
| 情况 | 状态码 | 延迟 | 是否打到上游 |
|
||||
|---|---|---|---|
|
||||
| 配额用尽（`req_quota` 50 → 100 并发） | `429 rate_limit_exceeded` + `Retry-After` | **19–21ms** | 否 |
|
||||
| 容量打满（同一拓扑，未限配额） | `503 upstream_error` | ~2.6s（`busyWait`） | 超出部分否 |
|
||||
|
||||
客户端因此可以区分「等窗口重置」与「等上游腾出容量」：前者只需等待，后者
|
||||
通常该降并发或换源。
|
||||
|
||||
##### 配额的开销
|
||||
|
||||
配额桶按 (密钥, 模型, 整点小时) 分桶保留 40 天，实测（AMD 7840HS）：
|
||||
|
||||
- 窗口查询按窗口长度而非保留总量扫描：1h 窗口 49ns、24h 窗口 55ns、
|
||||
30d 窗口 3.9μs（此前全扫保留总量，960 桶时 5.9μs）。
|
||||
- 记录一条请求：283ns、3 分配（与引入配额前相同，分配来自 ring buffer）。
|
||||
- 内存：生产形态（7 密钥 × 8 模型 × 2 源 × 满 40 天 retention）约 **3.7MB**。
|
||||
按源 pin 的 `source::model` 桶**惰性创建**——只有当某条配额真的 pin 了
|
||||
某个源时才维护，否则每条记录多写一份桶，在 20 密钥 × 8 模型 × 3 源下会
|
||||
多占 18MB。
|
||||
代价：配置 pinned 配额之前的历史用量无法事后按源拆分，所以 **pinned 配额的
|
||||
首个窗口可能少算**。
|
||||
|
||||
```bash
|
||||
# 配额耗尽时客户端看到（点名了具体模型）
|
||||
HTTP/1.1 429 Too Many Requests
|
||||
Retry-After: 2100
|
||||
{"error":{"type":"rate_limit_exceeded","message":"token quota exceeded for \"deepseek-v4-flash\" (5000000/5000000)"}}
|
||||
```
|
||||
|
||||
### 模型路由
|
||||
|
||||
`/v1/chat/completions` 的 `model` 解析顺序：
|
||||
@ -420,10 +344,7 @@ Environment=MALLOC_ARENA_MAX=2
|
||||
支持按时间范围导出 CSV；点击模型可生成 pin 到该模型的连接配置。
|
||||
- **对话页**：流式/非流式调试。
|
||||
- **密钥页**：创建/编辑网关 key，为每个 key 配模型范围（模型 + 源 + token 配额 + 周期），
|
||||
管理员管理全部 key，用户只看到自己的 key。每个模型砖块显示自己的配额徽标
|
||||
（如 `1.0K 77×·1h`，未设配额显示 `∞`），点开可编辑该模型的 token 配额、
|
||||
请求数配额与重置周期 —— 配额按模型独立生效，一个用满不影响同一 key 的其它模型。
|
||||
「我的密钥」页对用户展示本 key 的模型范围与各自配额。
|
||||
管理员管理全部 key，用户只看到自己的 key。
|
||||
- **优先级页**：拖拽积木配置 AUTO 链档位。
|
||||
- **源页**：在线增删改上游源（API key 等敏感字段加密落盘）。
|
||||
- **适配器页**：上传 / 删除 Lua 适配器脚本。
|
||||
|
||||
99
README_EN.md
99
README_EN.md
@ -38,11 +38,6 @@ Extracted and independently evolved from the multi-source LLM adapter layer of
|
||||
`reasoning_content`, `tool_calls`, `usage`).
|
||||
- **Image generation**: `POST /v1/images/generations`, routed to models with
|
||||
`kind: image`.
|
||||
- **Per-model quota**: each key gives every model its own token and request caps
|
||||
plus a reset period (hour/week/month/custom N hours). One model running out
|
||||
affects only that model — the key's other models keep working. Exhaustion
|
||||
answers 429 + `Retry-After` naming the model, so a client resumes when the
|
||||
window rolls over; admin keys are never capped.
|
||||
- **Multimodal**: `content` arrays (`image_url` etc.) pass through losslessly;
|
||||
Anthropic/Gemini/Ollama are translated automatically.
|
||||
- **LuaJIT VM**: golua-binding LuaJIT; each adapter has its own VM + worker
|
||||
@ -176,103 +171,11 @@ under the `keys` field of the runtime file (encrypted at rest):
|
||||
delete the seed key.
|
||||
- The WebUI **Keys page** creates/deletes keys. Each key has a role (`admin`
|
||||
manages everything, `user` sees only its own key) and an optional **model
|
||||
scope** (model + source + token quota + reset period). Each model brick shows
|
||||
its own budget badge (e.g. `1.0K 77×·1h`, `∞` when uncapped); clicking it
|
||||
edits that model's token quota, request quota and reset period. Quotas apply
|
||||
per model, so one model running out never blocks the key's others. The
|
||||
"My key" view shows a user its model scopes and their budgets.
|
||||
scope** (model + source + token quota + reset period).
|
||||
- Clients authenticate with any authorized key's plaintext as
|
||||
`Authorization: Bearer <key>`.
|
||||
- Deleting a key removes it from the store immediately.
|
||||
|
||||
#### Per-key usage quota
|
||||
|
||||
**Quotas are per model.** Each key's `models[]` list gives every model its own
|
||||
token budget and request budget. One model running out affects only that model
|
||||
— the key's other models keep working.
|
||||
|
||||
```yaml
|
||||
keys:
|
||||
- key: sk-gw-<hex>
|
||||
role: user
|
||||
name: agent-alice
|
||||
models:
|
||||
- model: deepseek-v4-flash
|
||||
token_quota: 1000000 # this key's token budget for this model
|
||||
req_quota: 20000 # requests within the window
|
||||
period: nhour # "" | hour | week | month | nhour
|
||||
hours: 6 # n-hour only
|
||||
- model: AUTO
|
||||
token_quota: 5000000 # AUTO is a quota entry like any other
|
||||
period: hour
|
||||
- model: kimi-k3 # no quota listed = unlimited
|
||||
```
|
||||
|
||||
- **There is deliberately no key-wide total.** A key-wide cap would make
|
||||
"move A's budget to B" a global reallocation; per-model budgets keep each
|
||||
model independently controllable, so it stays visible which model is
|
||||
actually consuming the spend.
|
||||
- `period`: empty = never resets (lifetime total); `hour` / `week` / `month` =
|
||||
fixed windows; `nhour` + `hours` = a custom hour count. **A misspelled
|
||||
period is rejected at write time** rather than silently becoming a
|
||||
never-resetting quota.
|
||||
- Quotas are isolated per key *and* per model within a key: one key exhausting
|
||||
`m1` never draws on another key's budget, and never blocks the same key's
|
||||
`m2`.
|
||||
- Usage counts chat, streaming and image requests, and survives a restart by
|
||||
replaying the audit log (40 days retained, covering the longest `month`
|
||||
window).
|
||||
- An exhausted quota returns **429 + `Retry-After`**
|
||||
(`rate_limit_exceeded`) and the message names the model that ran out, so a
|
||||
client resumes when the window rolls over; a model the key may not use stays
|
||||
403. **Admin keys are never capped** (quotas on their scopes are not
|
||||
enforced either), so a cap can never lock the operator out.
|
||||
- Buckets are whole unix hours, so a window frees up at most an hour late
|
||||
(deliberately freeing late rather than overspending).
|
||||
- `PUT /api/keys/{key}` submits quotas by submitting `models` — the caps are
|
||||
part of the scope, so there is no second budget that can drift out of sync
|
||||
with the model list. An explicit `0` lifts that model's cap.
|
||||
|
||||
##### Quota rejection vs capacity rejection
|
||||
|
||||
A quota is checked at the entry point and occupies no upstream slot, so it is a
|
||||
cheap rejection. Running out of capacity instead waits out the bounded
|
||||
`busyWait` before answering 503. Measured (capacity 8, 0.6 s/request, 100
|
||||
concurrent clients):
|
||||
|
||||
| Case | Status | Latency | Reaches upstream |
|
||||
|---|---|---|---|
|
||||
| Quota spent (`req_quota` 50, 100 concurrent) | `429 rate_limit_exceeded` + `Retry-After` | **19–21 ms** | no |
|
||||
| Capacity exhausted (same topology, no quota) | `503 upstream_error` | ~2.6 s (`busyWait`) | overflow: no |
|
||||
|
||||
So a client can tell "wait for the window to roll over" apart from "wait for
|
||||
upstream capacity" — the first only needs patience, the second usually means
|
||||
back off concurrency or switch sources.
|
||||
|
||||
##### What the quota costs
|
||||
|
||||
Buckets are kept per (key, model, whole unix hour) for 40 days. Measured on an
|
||||
AMD 7840HS:
|
||||
|
||||
- A window query scans the window, not the whole retention: 49 ns for 1 h,
|
||||
55 ns for 24 h, 3.9 µs for 30 d.
|
||||
- Recording one request: 283 ns, 3 allocations (unchanged from before this
|
||||
feature; the allocations come from the record ring buffer).
|
||||
- Memory: the production shape (7 keys x 8 models x 2 sources at full 40-day
|
||||
retention) costs about **3.7 MB**. The `source::model` bucket used by a
|
||||
source-pinned quota is created **lazily** — it is maintained only once some
|
||||
quota actually pins that source, because writing it on every record would
|
||||
cost an extra 18 MB at 20 keys x 8 models x 3 sources.
|
||||
The trade-off: usage recorded before a pinned quota was configured cannot be
|
||||
split by source afterwards, so **a pinned quota may under-count its first
|
||||
window**.
|
||||
|
||||
```
|
||||
HTTP/1.1 429 Too Many Requests
|
||||
Retry-After: 2100
|
||||
{"error":{"type":"rate_limit_exceeded","message":"token quota exceeded for \"deepseek-v4-flash\" (5000000/5000000)"}}
|
||||
```
|
||||
|
||||
### Model routing
|
||||
|
||||
`/v1/chat/completions` `model` resolution order:
|
||||
|
||||
179
cmd/gui/main.js
179
cmd/gui/main.js
@ -17,7 +17,7 @@ const fs = require("fs");
|
||||
const http = require("http");
|
||||
const os = require("os");
|
||||
const crypto = require("crypto");
|
||||
const { spawn, execFileSync } = require("child_process");
|
||||
const { spawn } = require("child_process");
|
||||
|
||||
const LOG_FILE = path.join(app.getPath("userData"), "gui.log");
|
||||
function log(msg) {
|
||||
@ -97,99 +97,24 @@ function readAdminKey() {
|
||||
// gateway_keys entries get marked seed:true by the core (-> "replace the
|
||||
// initial key" warning in the web UI), while a plain keys entry authenticates
|
||||
// without any seed warning.
|
||||
const raw = readConfigRaw();
|
||||
if (!raw) return "";
|
||||
// keys:
|
||||
// - key: sk-gw-...
|
||||
const k = /^\s*keys:\s*\n(?:\s*-\s*key:\s*"?([^\s"#]+)"?[^\n]*\n)+/m.exec(
|
||||
raw,
|
||||
);
|
||||
let v = k && k[1] ? k[1] : "";
|
||||
if (!v) {
|
||||
// legacy: gateway_keys list (pre-migration configs)
|
||||
const g = /^\s*gateway_keys:\s*\n(?:\s*-\s*"?([^\s"#]+)"?\s*\n)+/m.exec(raw);
|
||||
if (g && g[1]) v = g[1];
|
||||
if (!v) {
|
||||
const g2 = /gateway_keys:[^[\n]*\[\s*"?([^\s"\]]+)"?/m.exec(raw);
|
||||
if (g2 && g2[1]) v = g2[1];
|
||||
}
|
||||
}
|
||||
// The core seals credentials at rest (enc:v1:...), so from the second start
|
||||
// onward the file holds ciphertext. Injecting that as a cookie is a
|
||||
// guaranteed 401, which surfaces to the user as "enter your key" for a key
|
||||
// they never set. Ask the core to unseal rather than reimplementing its
|
||||
// crypto here. -show-secrets only reads, prints and exits: it starts nothing
|
||||
// and writes nothing.
|
||||
if (v && v.indexOf(SEALED_PREFIX) === 0) return unsealViaCore();
|
||||
return v;
|
||||
}
|
||||
|
||||
const SEALED_PREFIX = "enc:v1:";
|
||||
|
||||
function readConfigRaw() {
|
||||
try {
|
||||
if (!fs.existsSync(CONFIG_FILE)) return "";
|
||||
return fs.readFileSync(CONFIG_FILE, "utf-8");
|
||||
} catch (e) {
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
// Returns the admin key in the clear, or "" if the core cannot be asked.
|
||||
//
|
||||
// cwd must be the directory that holds master.key. The core derives that path
|
||||
// from the config's runtime_file, so a call made from any other directory has
|
||||
// the core generate a *second* master key in the CWD and then fail to decrypt
|
||||
// ("master key changed?"). Electron's CWD is not the profile dir, so this is
|
||||
// not optional: without it the desktop build cannot read its own key back.
|
||||
//
|
||||
// readAdminKey runs on every outbound request, and this spawns a process, so
|
||||
// the result is cached against the config's mtime. Editing the config (or the
|
||||
// port rewrite, or the first write) changes the mtime and invalidates it, which
|
||||
// keeps the "read live so ordering never matters" property the auth rule
|
||||
// depends on.
|
||||
function unsealViaCore() {
|
||||
const st = safeStat(CONFIG_FILE);
|
||||
const stamp = st ? st.mtimeMs : 0;
|
||||
if (unsealCache && unsealCache.stamp === stamp) return unsealCache.key;
|
||||
let key = "";
|
||||
try {
|
||||
const out = execFileSync(
|
||||
CORE_EXE,
|
||||
["-config", CONFIG_FILE, "-show-secrets"],
|
||||
{
|
||||
encoding: "utf-8",
|
||||
timeout: 10000,
|
||||
windowsHide: true,
|
||||
cwd: PROFILE_DIR,
|
||||
},
|
||||
const raw = fs.readFileSync(CONFIG_FILE, "utf-8");
|
||||
// keys:
|
||||
// - key: sk-gw-...
|
||||
const k = /^\s*keys:\s*\n(?:\s*-\s*key:\s*"?([^\s"#]+)"?[^\n]*\n)+/m.exec(
|
||||
raw,
|
||||
);
|
||||
// -show-secrets prints one line per credential:
|
||||
// key <name> role=<role> <value>
|
||||
const admin = /^key\s+\S+\s+role=admin\s+(\S+)\s*$/m.exec(out);
|
||||
if (admin && admin[1]) key = admin[1];
|
||||
// Tolerate a field-order or spacing change rather than locking the user out.
|
||||
if (!key) {
|
||||
const loose = /role=admin\s+(\S+)/.exec(out);
|
||||
if (loose && loose[1]) key = loose[1];
|
||||
}
|
||||
} catch (e) {
|
||||
console.error("unseal via core failed:", e.message);
|
||||
}
|
||||
// Only cache a success. A transient failure must not pin an empty key until
|
||||
// the config next changes, or a momentary spawn error locks the user out.
|
||||
if (key) unsealCache = { stamp, key };
|
||||
return key;
|
||||
}
|
||||
|
||||
let unsealCache = null;
|
||||
|
||||
function safeStat(p) {
|
||||
try {
|
||||
return fs.statSync(p);
|
||||
} catch (e) {
|
||||
return null;
|
||||
}
|
||||
if (k && k[1]) return k[1];
|
||||
// legacy: gateway_keys list (pre-migration configs)
|
||||
const g = /^\s*gateway_keys:\s*\n(?:\s*-\s*"?([^\s"#]+)"?\s*\n)+/m.exec(
|
||||
raw,
|
||||
);
|
||||
if (g && g[1]) return g[1];
|
||||
const g2 = /gateway_keys:[^[\n]*\[\s*"?([^\s"\]]+)"?/m.exec(raw);
|
||||
if (g2 && g2[1]) return g2[1];
|
||||
} catch (e) {}
|
||||
return "";
|
||||
}
|
||||
|
||||
const embeddedBaseUrl = () =>
|
||||
@ -671,78 +596,6 @@ ipcMain.handle(
|
||||
(e) => !!BrowserWindow.fromWebContents(e.sender)?.isMaximized(),
|
||||
);
|
||||
|
||||
// ---- plugin management over IPC -------------------------------------------
|
||||
//
|
||||
// The renderer cannot call the embedded core directly: it has no key and no
|
||||
// network identity, and the core binds a loopback port that only the main
|
||||
// process knows about. So every plugin action is proxied through the main
|
||||
// process, which already knows how to obtain the admin key (unsealViaCore).
|
||||
//
|
||||
// The proxy is deliberately a raw (method, path, body) pass-through rather than
|
||||
// a fixed set of commands. A fixed set would have to be extended for every new
|
||||
// plugin endpoint, and the one thing worse than "no button for this" is "a
|
||||
// button that silently does nothing" — with a pass-through the renderer can talk
|
||||
// to any /api/plugins route the core grows, and the path is validated here so
|
||||
// this channel cannot be used to reach arbitrary endpoints.
|
||||
function pluginProxy(req) {
|
||||
const { method, path, body } = req || {};
|
||||
const M = ["GET", "POST", "PUT", "DELETE"];
|
||||
if (!M.includes(method)) throw new Error("bad method: " + method);
|
||||
// The path must stay inside the plugin namespace. A prefix check alone would
|
||||
// still allow /api/plugins/../keys, so reject any traversal outright.
|
||||
if (typeof path !== "string" || !path.startsWith("/api/plugins")) {
|
||||
throw new Error("path must start with /api/plugins");
|
||||
}
|
||||
if (path.includes("..") || path.includes("\\")) {
|
||||
throw new Error("path traversal rejected");
|
||||
}
|
||||
const key = unsealViaCore();
|
||||
if (!key) throw new Error("no admin key available yet");
|
||||
return new Promise((resolve, reject) => {
|
||||
const u = new URL(embeddedBaseUrl() + path);
|
||||
const data = body == null ? null : JSON.stringify(body);
|
||||
const headers = { Authorization: "Bearer " + key };
|
||||
if (data) {
|
||||
headers["Content-Type"] = "application/json";
|
||||
headers["Content-Length"] = Buffer.byteLength(data);
|
||||
}
|
||||
const r = http.request(
|
||||
{
|
||||
hostname: u.hostname,
|
||||
port: u.port,
|
||||
path: u.pathname + u.search,
|
||||
method,
|
||||
headers,
|
||||
},
|
||||
(res) => {
|
||||
let raw = "";
|
||||
res.setEncoding("utf8");
|
||||
res.on("data", (c) => (raw += c));
|
||||
res.on("end", () => {
|
||||
let parsed = null;
|
||||
try {
|
||||
parsed = raw ? JSON.parse(raw) : null;
|
||||
} catch (e) {
|
||||
parsed = { raw };
|
||||
}
|
||||
if (res.statusCode >= 400) {
|
||||
const msg =
|
||||
(parsed && parsed.error && parsed.error.message) ||
|
||||
"HTTP " + res.statusCode;
|
||||
reject(new Error(msg));
|
||||
return;
|
||||
}
|
||||
resolve(parsed);
|
||||
});
|
||||
},
|
||||
);
|
||||
r.on("error", reject);
|
||||
if (data) r.write(data);
|
||||
r.end();
|
||||
});
|
||||
}
|
||||
|
||||
ipcMain.handle("plugins:proxy", (_e, req) => pluginProxy(req));
|
||||
ipcMain.handle("core:state", () => ({
|
||||
running: coreStarted() && coreReady,
|
||||
ready: coreReady,
|
||||
|
||||
8
cmd/gui/package-lock.json
generated
8
cmd/gui/package-lock.json
generated
@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "modelrouter-gui",
|
||||
"version": "1.9.2",
|
||||
"version": "1.5.3",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "modelrouter-gui",
|
||||
"version": "1.9.2",
|
||||
"version": "1.5.3",
|
||||
"license": "Proprietary",
|
||||
"devDependencies": {
|
||||
"electron": "^33.0.0",
|
||||
@ -582,7 +582,7 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@peculiar/webcrypto": {
|
||||
"version": "1.7.5",
|
||||
"version": "1.7.1",
|
||||
"resolved": "https://registry.npmjs.org/@peculiar/webcrypto/-/webcrypto-1.7.1.tgz",
|
||||
"integrity": "sha512-ODOov0sGMJMf3jPonOkgGqPknTsu+DdQ7kD++gz8aI+aFMOMHFbWAA2taqXXVTdP+OTOQR/znGvSpmkeI0WTYQ==",
|
||||
"dev": true,
|
||||
@ -3335,7 +3335,7 @@
|
||||
}
|
||||
},
|
||||
"node_modules/resedit": {
|
||||
"version": "1.7.5",
|
||||
"version": "1.7.2",
|
||||
"resolved": "https://registry.npmjs.org/resedit/-/resedit-1.7.2.tgz",
|
||||
"integrity": "sha512-vHjcY2MlAITJhC0eRD/Vv8Vlgmu9Sd3LX9zZvtGzU5ZImdTN3+d6e/4mnTyV8vEbyf1sgNIrWxhWlrys52OkEA==",
|
||||
"dev": true,
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "modelrouter-gui",
|
||||
"version": "1.9.2",
|
||||
"version": "1.5.3",
|
||||
"description": "ModelRouter Desktop — embedded ModelRouter core with tray",
|
||||
"author": "ModelRouter",
|
||||
"main": "main.js",
|
||||
|
||||
@ -16,13 +16,6 @@ contextBridge.exposeInMainWorld("modelrouter", {
|
||||
key: () => ipcRenderer.invoke("core:key"),
|
||||
onState: (cb) => ipcRenderer.on("core:state", (_e, d) => cb(d)),
|
||||
},
|
||||
plugins: {
|
||||
// Raw pass-through to the embedded core's /api/plugins surface. The main
|
||||
// process validates the path and attaches the admin key; the renderer never
|
||||
// sees either.
|
||||
request: (method, path, body) =>
|
||||
ipcRenderer.invoke("plugins:proxy", { method, path, body }),
|
||||
},
|
||||
settings: {
|
||||
get: () => ipcRenderer.invoke("settings:get"),
|
||||
set: (patch) => ipcRenderer.invoke("settings:set", patch),
|
||||
|
||||
@ -155,7 +155,6 @@ async function openSettings() {
|
||||
$("#set-tray").checked = !!state.settings.minimizeToTray;
|
||||
$("#settings-overlay").style.display = "flex";
|
||||
renderRail();
|
||||
loadPlugins();
|
||||
}
|
||||
function closeSettings() {
|
||||
$("#settings-overlay").style.display = "none";
|
||||
@ -183,296 +182,6 @@ async function saveSettings() {
|
||||
}
|
||||
}
|
||||
|
||||
// esc / escAttr escape text for innerHTML. The WebUI has its own copies; the
|
||||
// shell needs its own because renderer/app.js is a separate document that
|
||||
// never loads index.html's script.
|
||||
function esc(s) {
|
||||
return String(s == null ? "" : s).replace(
|
||||
/[&<>"']/g,
|
||||
(c) => ({ "&": "&", "<": "<", ">": ">", '"': """, "'": "'" })[c],
|
||||
);
|
||||
}
|
||||
function escAttr(s) {
|
||||
return esc(s).replace(/`/g, "`");
|
||||
}
|
||||
|
||||
// ===== plugin management =====
|
||||
//
|
||||
// The desktop shell manages plugins through the embedded core's /api/plugins
|
||||
// surface, proxied over IPC (see plugins:proxy in the main process). The
|
||||
// renderer never holds the admin key.
|
||||
//
|
||||
// Scope note: the desktop build has no plugin_dir configured by default, so this
|
||||
// panel normally reports "plugins disabled" with the one-line fix. That is a
|
||||
// deliberate state, not an error — the packaged profile is a per-user directory
|
||||
// and seeding a plugin tree into someone's home without asking would be rude.
|
||||
|
||||
async function loadPlugins() {
|
||||
const list = $("#pl-list");
|
||||
const hint = $("#set-plugins-hint");
|
||||
if (!list || !hint) return;
|
||||
let j;
|
||||
try {
|
||||
j = await window.modelrouter.plugins.request("GET", "/api/plugins");
|
||||
} catch (e) {
|
||||
hint.textContent = "内核未就绪：" + e.message;
|
||||
list.innerHTML = "";
|
||||
return;
|
||||
}
|
||||
if (!j.plugin_dir) {
|
||||
hint.innerHTML =
|
||||
'未配置 <code>plugin_dir</code>，插件功能未启用。在 config.yaml 加一行后重启内核即可。';
|
||||
list.innerHTML = "";
|
||||
return;
|
||||
}
|
||||
const rows = j.on_disk || [];
|
||||
window.__plRows = rows; // read by editPlugin/deletePlugin for the builtin flag
|
||||
const active = rows.filter((p) => p.loaded && !p.disabled).length;
|
||||
const broken = rows.filter((p) => !p.loaded).length;
|
||||
hint.textContent =
|
||||
`${rows.length} 个插件 · ${active} 个启用中` +
|
||||
(broken ? ` · ${broken} 个加载失败` : "");
|
||||
list.innerHTML = rows.length
|
||||
? rows
|
||||
.map((p) => {
|
||||
const cls = !p.loaded ? "pl-broken" : p.disabled ? "pl-off" : "pl-on";
|
||||
const label = !p.loaded
|
||||
? "加载失败"
|
||||
: p.disabled
|
||||
? "已禁用"
|
||||
: "启用中";
|
||||
const btn = p.loaded
|
||||
? `<button class="ghost" data-act="toggle" data-name="${escAttr(
|
||||
p.name,
|
||||
)}" data-en="${p.disabled ? "1" : "0"}">${
|
||||
p.disabled ? "启用" : "禁用"
|
||||
}</button>`
|
||||
: "";
|
||||
const builtin = p.builtin
|
||||
? '<span class="pl-builtin">内置</span>'
|
||||
: "";
|
||||
// Edit / view source / delete are the operations an operator
|
||||
// actually needs after enable/disable: a plugin that cannot be
|
||||
// read back cannot be fixed, and one that cannot be removed cannot
|
||||
// be rolled back.
|
||||
const acts = [
|
||||
btn,
|
||||
`<button class="ghost" data-act="code" data-name="${escAttr(p.name)}">${
|
||||
p.builtin ? "查看源码" : "编辑"
|
||||
}</button>`,
|
||||
builtin
|
||||
? ""
|
||||
: `<button class="ghost danger" data-act="del" data-name="${escAttr(
|
||||
p.name,
|
||||
)}">删除</button>`,
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join("");
|
||||
const pages = Array.isArray(p.pages) ? p.pages : [];
|
||||
return `<div class="pl-item ${cls}">
|
||||
<div class="pl-head"><b>${esc(p.name)}</b>${builtin}<span class="pl-state">${label}</span></div>
|
||||
${p.description ? `<div class="pl-desc">${esc(p.description)}</div>` : ""}
|
||||
${pages.length ? `<div class="pl-desc">页面：${esc(pages.join(", "))}</div>` : ""}
|
||||
${p.error ? `<div class="pl-err">${esc(String(p.error).slice(0, 160))}</div>` : ""}
|
||||
<div class="pl-acts">${acts}</div>
|
||||
</div>`;
|
||||
})
|
||||
.join("")
|
||||
: '<div class="pl-empty">插件目录为空</div>';
|
||||
list.querySelectorAll('button[data-act="toggle"]').forEach((b) => {
|
||||
b.onclick = () => togglePlugin(b.dataset.name, b.dataset.en === "1");
|
||||
});
|
||||
list.querySelectorAll('button[data-act="code"]').forEach((b) => {
|
||||
b.onclick = () => editPlugin(b.dataset.name);
|
||||
});
|
||||
list.querySelectorAll('button[data-act="del"]').forEach((b) => {
|
||||
b.onclick = () => deletePlugin(b.dataset.name);
|
||||
});
|
||||
}
|
||||
|
||||
// editPlugin shows the plugin source in a modal. Built-in plugins are
|
||||
// read-only: their file ships inside the binary, so a save would either be
|
||||
// ignored or silently do nothing — showing an editable text area that
|
||||
// discards on save is worse than saying so up front.
|
||||
function editPlugin(name) {
|
||||
// The plugin list is the only place `builtin` is reported; keep the last
|
||||
// fetch so the modal does not need a second round trip (and cannot guess).
|
||||
const row = (window.__plRows || []).find((p) => p.name === name);
|
||||
const isBuiltin = !!(row && row.builtin);
|
||||
const wrap = document.createElement("div");
|
||||
wrap.className = "overlay";
|
||||
wrap.innerHTML = `<div class="overlay-card pl-wide">
|
||||
<div class="overlay-head"><h2>${esc(name)}${isBuiltin ? "（内置，只读）" : ""}</h2></div>
|
||||
${isBuiltin
|
||||
? `<p class="hint">内置插件随内核发布，源码在二进制内，不可编辑。</p>`
|
||||
: `<textarea id="pl-code" spellcheck="false"></textarea>
|
||||
<p class="hint" id="pl-code-err"></p>`}
|
||||
<div class="row actions">
|
||||
${isBuiltin ? "" : `<button id="pl-code-save">保存并热重载</button>`}
|
||||
<button class="ghost" id="pl-code-close">关闭</button>
|
||||
</div>
|
||||
</div>`;
|
||||
document.body.appendChild(wrap);
|
||||
wrap.addEventListener("click", (e) => {
|
||||
if (e.target === wrap) wrap.remove();
|
||||
});
|
||||
$("#pl-code-close").onclick = () => wrap.remove();
|
||||
const save = $("#pl-code-save");
|
||||
if (save) {
|
||||
save.onclick = async () => {
|
||||
save.disabled = true;
|
||||
try {
|
||||
// Saving writes through POST /api/plugins — the same install path the
|
||||
// installer uses, which validates the name, compiles the source and
|
||||
// hot-reloads. PUT /{name} only toggles enable/disable, so pointing the
|
||||
// save there would silently disable nothing and store nothing.
|
||||
await window.modelrouter.plugins.request("POST", "/api/plugins", {
|
||||
name,
|
||||
code: $("#pl-code").value,
|
||||
});
|
||||
toast(`${name} 已保存`);
|
||||
wrap.remove();
|
||||
await loadPlugins();
|
||||
} catch (e) {
|
||||
$("#pl-code-err").textContent = e.message;
|
||||
save.disabled = false;
|
||||
}
|
||||
};
|
||||
}
|
||||
if (isBuiltin) return;
|
||||
(async () => {
|
||||
const ta = $("#pl-code");
|
||||
try {
|
||||
const j = await window.modelrouter.plugins.request(
|
||||
"GET",
|
||||
`/api/plugins/${encodeURIComponent(name)}`,
|
||||
);
|
||||
ta.value = j.code || "";
|
||||
const err = $("#pl-code-err");
|
||||
if (err && j.parse_error) err.textContent = String(j.parse_error);
|
||||
} catch (e) {
|
||||
$("#pl-code-err").textContent = e.message;
|
||||
}
|
||||
})();
|
||||
}
|
||||
|
||||
function deletePlugin(name) {
|
||||
const wrap = document.createElement("div");
|
||||
wrap.className = "overlay";
|
||||
wrap.innerHTML = `<div class="overlay-card">
|
||||
<div class="overlay-head"><h2>删除插件</h2></div>
|
||||
<p>确定删除 <b>${esc(name)}</b>？插件目录中的文件会被移除，核心将不再加载它。</p>
|
||||
<div class="row actions">
|
||||
<button class="danger" id="pl-del-ok">删除</button>
|
||||
<button class="ghost" id="pl-del-no">取消</button>
|
||||
</div>
|
||||
</div>`;
|
||||
document.body.appendChild(wrap);
|
||||
wrap.addEventListener("click", (e) => {
|
||||
if (e.target === wrap) wrap.remove();
|
||||
});
|
||||
$("#pl-del-no").onclick = () => wrap.remove();
|
||||
$("#pl-del-ok").onclick = async () => {
|
||||
try {
|
||||
await window.modelrouter.plugins.request(
|
||||
"DELETE",
|
||||
`/api/plugins/${encodeURIComponent(name)}`,
|
||||
);
|
||||
toast(`已删除 ${name}`);
|
||||
wrap.remove();
|
||||
await loadPlugins();
|
||||
} catch (e) {
|
||||
toast(e.message, true);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
// installPlugin creates a new plugin from source. The name becomes the file
|
||||
// name, so it is validated client-side to save a round trip on the obvious
|
||||
// mistakes (empty, path separators, non-portable characters).
|
||||
function installPlugin() {
|
||||
const wrap = document.createElement("div");
|
||||
wrap.className = "overlay";
|
||||
wrap.innerHTML = `<div class="overlay-card pl-wide">
|
||||
<div class="overlay-head"><h2>安装插件</h2></div>
|
||||
<div class="row"><label>名称</label><input id="pl-new-name" placeholder="my_plugin" /></div>
|
||||
<p class="hint" id="pl-new-err"></p>
|
||||
<textarea id="pl-new-code" spellcheck="false" placeholder="-- 单文件 Lua 插件 plugin = {} ..."></textarea>
|
||||
<div class="row actions">
|
||||
<button id="pl-new-ok">安装</button>
|
||||
<button class="ghost" id="pl-new-no">取消</button>
|
||||
</div>
|
||||
</div>`;
|
||||
document.body.appendChild(wrap);
|
||||
wrap.addEventListener("click", (e) => {
|
||||
if (e.target === wrap) wrap.remove();
|
||||
});
|
||||
$("#pl-new-no").onclick = () => wrap.remove();
|
||||
$("#pl-new-ok").onclick = async () => {
|
||||
const name = $("#pl-new-name").value.trim();
|
||||
const code = $("#pl-new-code").value;
|
||||
const err = $("#pl-new-err");
|
||||
if (!/^[A-Za-z0-9_-]+$/.test(name)) {
|
||||
err.textContent = "名称只能包含字母、数字、下划线和连字符";
|
||||
return;
|
||||
}
|
||||
if (!code.trim()) {
|
||||
err.textContent = "源码不能为空";
|
||||
return;
|
||||
}
|
||||
$("#pl-new-ok").disabled = true;
|
||||
try {
|
||||
await window.modelrouter.plugins.request("POST", "/api/plugins", {
|
||||
// The backend appends .lua itself (installPlugin writes name+".lua"),
|
||||
// so passing it here would create name.lua.lua.
|
||||
name,
|
||||
code,
|
||||
});
|
||||
toast(`已安装 ${name}`);
|
||||
wrap.remove();
|
||||
await loadPlugins();
|
||||
} catch (e) {
|
||||
err.textContent = e.message;
|
||||
$("#pl-new-ok").disabled = false;
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
async function togglePlugin(name, disabled) {
|
||||
try {
|
||||
await window.modelrouter.plugins.request("PUT", `/api/plugins/${encodeURIComponent(name)}`, {
|
||||
enabled: disabled,
|
||||
});
|
||||
toast(disabled ? `已禁用 ${name}` : `已启用 ${name}`);
|
||||
await loadPlugins();
|
||||
} catch (e) {
|
||||
toast(e.message, true);
|
||||
}
|
||||
}
|
||||
|
||||
async function enableAllPlugins() {
|
||||
let j;
|
||||
try {
|
||||
j = await window.modelrouter.plugins.request("GET", "/api/plugins");
|
||||
} catch (e) {
|
||||
return toast(e.message, true);
|
||||
}
|
||||
const off = (j.on_disk || []).filter((p) => p.loaded && p.disabled);
|
||||
for (const p of off) {
|
||||
try {
|
||||
await window.modelrouter.plugins.request(
|
||||
"PUT",
|
||||
`/api/plugins/${encodeURIComponent(p.name)}`,
|
||||
{ enabled: true },
|
||||
);
|
||||
} catch (e) {
|
||||
toast(`${p.name}: ${e.message}`, true);
|
||||
}
|
||||
}
|
||||
toast(off.length ? `已启用 ${off.length} 个插件` : "没有处于禁用状态的插件");
|
||||
await loadPlugins();
|
||||
}
|
||||
|
||||
// ===== theme =====
|
||||
function applyTheme() {
|
||||
document.documentElement.dataset.theme = state.theme;
|
||||
@ -488,12 +197,6 @@ function init() {
|
||||
$("#tb-close").onclick = () => window.modelrouter.win.close();
|
||||
$("#tb-settings").onclick = openSettings;
|
||||
$("#rail-settings").onclick = openSettings;
|
||||
const plReload = document.getElementById("pl-reload");
|
||||
if (plReload) plReload.onclick = loadPlugins;
|
||||
const plAll = document.getElementById("pl-toggle-all");
|
||||
if (plAll) plAll.onclick = enableAllPlugins;
|
||||
const plInstall = document.getElementById("pl-install");
|
||||
if (plInstall) plInstall.onclick = installPlugin;
|
||||
$("#rail-autostart").onclick = toggleAutoStart;
|
||||
$("#rail-silent").onclick = toggleSilent;
|
||||
$("#rail-theme").onclick = () => {
|
||||
|
||||
@ -207,16 +207,6 @@
|
||||
><input type="checkbox" id="set-tray" /> 关闭时最小化到托盘</label
|
||||
><span class="hint">点关闭按钮隐藏到系统托盘</span>
|
||||
</div>
|
||||
<div class="row">
|
||||
<label>插件</label>
|
||||
<span class="hint" id="set-plugins-hint">加载中…</span>
|
||||
</div>
|
||||
<div id="pl-list" class="pl-list"></div>
|
||||
<div class="row actions">
|
||||
<button class="ghost" id="pl-toggle-all">全部启用</button>
|
||||
<button class="ghost" id="pl-install">安装插件</button>
|
||||
<button class="ghost" id="pl-reload">刷新</button>
|
||||
</div>
|
||||
<div class="row actions">
|
||||
<button class="ghost" id="set-dir">打开数据目录</button>
|
||||
<button class="ghost" id="set-log">查看日志</button>
|
||||
|
||||
@ -596,129 +596,3 @@ html[data-theme="dark"] .overlay {
|
||||
#toast.err {
|
||||
border-color: var(--danger);
|
||||
}
|
||||
|
||||
/* ===== plugin management panel =========================================
|
||||
* The existing .ghost/.primary rules are scoped to `.form .actions`, so a
|
||||
* button outside that selector gets browser defaults. The plugin rows live in
|
||||
* their own list, hence their own rules — reusing a scoped class here would have
|
||||
* produced unstyled buttons that still worked, which is the kind of thing that
|
||||
* looks fine until someone themes the shell.
|
||||
*/
|
||||
.pl-list {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 8px;
|
||||
margin: 8px 0 4px;
|
||||
}
|
||||
.pl-item {
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 9px;
|
||||
padding: 10px 12px;
|
||||
}
|
||||
.pl-item.pl-broken {
|
||||
border-color: var(--danger, #d1435b);
|
||||
}
|
||||
.pl-head {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
font-size: 13px;
|
||||
}
|
||||
.pl-builtin {
|
||||
font-size: 10px;
|
||||
padding: 1px 6px;
|
||||
border-radius: 999px;
|
||||
background: var(--primary-50);
|
||||
color: var(--primary-h);
|
||||
}
|
||||
.pl-state {
|
||||
margin-left: auto;
|
||||
font-size: 11px;
|
||||
color: var(--muted);
|
||||
}
|
||||
.pl-item.pl-broken .pl-state {
|
||||
color: var(--danger, #d1435b);
|
||||
}
|
||||
.pl-desc {
|
||||
font-size: 12px;
|
||||
color: var(--muted);
|
||||
margin-top: 3px;
|
||||
}
|
||||
.pl-err {
|
||||
font-size: 11px;
|
||||
color: var(--danger, #d1435b);
|
||||
margin-top: 4px;
|
||||
word-break: break-word;
|
||||
}
|
||||
.pl-acts {
|
||||
margin-top: 8px;
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
}
|
||||
.pl-acts button {
|
||||
padding: 5px 12px;
|
||||
font-size: 12px;
|
||||
border-radius: 7px;
|
||||
border: 1px solid var(--line);
|
||||
background: var(--bg-s2, #fff);
|
||||
color: var(--fg, inherit);
|
||||
cursor: pointer;
|
||||
transition: all 0.15s;
|
||||
}
|
||||
.pl-acts button:hover {
|
||||
border-color: var(--primary);
|
||||
color: var(--primary-h);
|
||||
}
|
||||
.pl-empty {
|
||||
font-size: 12px;
|
||||
color: var(--muted);
|
||||
padding: 10px 0;
|
||||
}
|
||||
|
||||
/* ---- plugin editor / installer --------------------------------------------
|
||||
The plugin source editor and the install dialog reuse the settings overlay
|
||||
(overlay / overlay-card) rather than a second popup system. They need three
|
||||
things the settings card does not: a wider column for code, a monospace
|
||||
textarea, and a destructive button for delete. */
|
||||
.overlay-card.pl-wide {
|
||||
width: 760px;
|
||||
}
|
||||
.pl-wide textarea {
|
||||
width: 100%;
|
||||
min-height: 380px;
|
||||
resize: vertical;
|
||||
padding: 10px 12px;
|
||||
border-radius: 10px;
|
||||
border: 1px solid var(--line-strong);
|
||||
background: var(--bg-sunken, rgba(127, 127, 127, 0.06));
|
||||
color: inherit;
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
|
||||
font-size: 12.5px;
|
||||
line-height: 1.55;
|
||||
tab-size: 2;
|
||||
}
|
||||
.pl-wide textarea:focus {
|
||||
outline: none;
|
||||
border-color: var(--primary);
|
||||
}
|
||||
.pl-wide .row input {
|
||||
flex: 1;
|
||||
padding: 7px 10px;
|
||||
border-radius: 9px;
|
||||
border: 1px solid var(--line-strong);
|
||||
background: var(--bg-sunken, rgba(127, 127, 127, 0.06));
|
||||
color: inherit;
|
||||
}
|
||||
.pl-editor .hint,
|
||||
.pl-wide .hint {
|
||||
min-height: 16px;
|
||||
color: var(--muted, #888);
|
||||
}
|
||||
button.danger {
|
||||
color: #e5484d;
|
||||
border-color: rgba(229, 72, 77, 0.4);
|
||||
}
|
||||
button.danger:hover {
|
||||
border-color: #e5484d;
|
||||
color: #e5484d;
|
||||
}
|
||||
|
||||
@ -24,7 +24,6 @@ import (
|
||||
func main() {
|
||||
cfgPath := flag.String("config", "config.yaml", "path to gateway config file")
|
||||
checkOnly := flag.Bool("check", false, "validate the config file and exit (0 = valid, 1 = invalid); nothing is started and no file is written")
|
||||
showSecrets := flag.Bool("show-secrets", false, "print the config's credentials in the clear and exit; nothing is started and no file is written")
|
||||
flag.Parse()
|
||||
|
||||
// -check is the deploy-time preflight: parse and validate the config
|
||||
@ -42,20 +41,6 @@ func main() {
|
||||
return
|
||||
}
|
||||
|
||||
// -show-secrets is the operator escape hatch for a config whose
|
||||
// credentials are sealed at rest: it prints them to stdout and exits
|
||||
// without starting anything or writing anything. Nothing else in the
|
||||
// program prints a credential, and this path never logs to a file.
|
||||
if *showSecrets {
|
||||
if _, err := os.Stat(*cfgPath); err != nil {
|
||||
log.Fatalf("[llmsproxy] show-secrets: %v", err)
|
||||
}
|
||||
if err := config.PrintSecrets(*cfgPath); err != nil {
|
||||
log.Fatalf("[llmsproxy] show-secrets: %v", err)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
created, err := config.EnsureDefault(*cfgPath)
|
||||
if err != nil {
|
||||
log.Fatalf("[llmsproxy] config: %v", err)
|
||||
@ -73,26 +58,19 @@ func main() {
|
||||
}
|
||||
}
|
||||
|
||||
gw, err := gateway.New(c)
|
||||
gw, err := gateway.New(c, c.GatewayKeys())
|
||||
if err != nil {
|
||||
log.Fatalf("[llmsproxy] gateway: %v", err)
|
||||
}
|
||||
|
||||
// Ops hygiene: surface the two most common footguns instead of silently
|
||||
// running with them.
|
||||
//
|
||||
// Read the AUTHORITATIVE source. Auth uses core.ListKeys() (cfg.Keys), not
|
||||
// the legacy cfg.GatewayKeys list: seedKeys copies gateway_keys into keys[]
|
||||
// on first start and the YAML list is ignored for auth afterwards. Checking
|
||||
// GatewayKeys() here made the warning lie — once the legacy list was empty
|
||||
// (e.g. after rotating the starter key away) it reported "every request is
|
||||
// rejected" while seven working keys, one of them admin, were in service.
|
||||
if keys := c.ListKeys(); len(keys) == 0 {
|
||||
log.Printf("[llmsproxy] WARNING: no gateway key configured — every request will be rejected. Add one in the WebUI or set gateway_keys")
|
||||
if keys := c.GatewayKeys(); len(keys) == 0 {
|
||||
log.Printf("[llmsproxy] WARNING: gateway_keys is EMPTY — without a key every request is rejected")
|
||||
} else {
|
||||
for _, k := range keys {
|
||||
if k.Key == "sk-gw-local-0001" || k.Key == "sk-local-0001" {
|
||||
log.Printf("[llmsproxy] WARNING: gateway key %q looks like the starter/example key — rotate it before exposing the gateway", k.Key)
|
||||
if k == "sk-gw-local-0001" || k == "sk-local-0001" {
|
||||
log.Printf("[llmsproxy] WARNING: gateway key %q looks like the starter/example key — rotate it before exposing the gateway", k)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
116
deploy.sh
116
deploy.sh
@ -244,117 +244,6 @@ verify_config() {
|
||||
return 1
|
||||
}
|
||||
|
||||
# ---------- 密钥可解封校验 ----------
|
||||
# config.yaml 里的凭据以 enc:v1: 密文落盘，解密钥匙是 master.key（或
|
||||
# LLMS_PROXY_MASTER_KEY）。一旦钥匙丢失/错位，配置在**启动时**才会报错，
|
||||
# 那时服务已经处于「二进制已替换 + 配置已替换」的半死状态，只能靠回滚。
|
||||
# 所以在替换任何东西之前先验：能不能用现有的钥匙解开现有的密文。
|
||||
#
|
||||
# 两种情形要分开：
|
||||
# 1. 配置里**有**密文 ⇒ 钥匙必须能解开（否则拒绝部署）
|
||||
# 2. 配置里**没有**密文 ⇒ 全新安装/首次加密，无需钥匙（但会顺手记一条，
|
||||
# 因为它意味着这次部署会把明文写成密文，而钥匙必须同时就位）
|
||||
verify_master_key() {
|
||||
log "密钥可解封校验"
|
||||
[[ -f "$TARGET_CONFIG" ]] || return 0 # 配置都不在，交给 verify_config 报错
|
||||
|
||||
local sealed
|
||||
sealed=$(grep -c 'enc:v1:' "$TARGET_CONFIG" 2>/dev/null || echo 0)
|
||||
if [[ "$sealed" -eq 0 ]]; then
|
||||
warn "配置内无密文（尚未加密或全为明文）——首次加密需要 master.key 可写"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# 钥匙来源与 config 包一致：env 优先，否则 runtime_file 同目录的 master.key
|
||||
local runtime_file key_file
|
||||
runtime_file=$(awk '/^runtime_file:/{print $2; exit}' "$TARGET_CONFIG")
|
||||
[[ -n "$runtime_file" ]] || runtime_file="$(dirname "$TARGET_CONFIG")/runtime.json"
|
||||
key_file="$(dirname "$runtime_file")/master.key"
|
||||
|
||||
if [[ -n "${LLMS_PROXY_MASTER_KEY:-}" ]]; then
|
||||
log " 钥匙来源: LLMS_PROXY_MASTER_KEY（环境变量）"
|
||||
# 走真实解密路径验证，而不是只检查格式
|
||||
if out=$("$TARGET_BIN" -show-secrets -config "$TARGET_CONFIG" 2>&1); then
|
||||
log "✓ 凭据可解封（$(echo "$out" | grep -cE '^(source|key) ') 项）"
|
||||
return 0
|
||||
fi
|
||||
err "凭据无法用 LLMS_PROXY_MASTER_KEY 解封，未替换任何文件："
|
||||
printf '%s\n' "$out" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
if [[ ! -f "$key_file" ]]; then
|
||||
err "配置含 $sealed 处密文，但钥匙文件不存在: $key_file"
|
||||
err " 没有它，部署后服务将无法启动。恢复 master.key 或设置 LLMS_PROXY_MASTER_KEY 后重试。"
|
||||
return 1
|
||||
fi
|
||||
log " 钥匙文件: $key_file ($(stat -c '%A' "$key_file" 2>/dev/null || echo '?')，$(wc -c < "$key_file") 字节)"
|
||||
|
||||
if out=$("$TARGET_BIN" -show-secrets -config "$TARGET_CONFIG" 2>&1); then
|
||||
log "✓ 凭据可解封（$(echo "$out" | grep -cE '^(source|key) ') 项）"
|
||||
return 0
|
||||
fi
|
||||
err "凭据无法解密（钥匙与配置不匹配），未替换任何文件："
|
||||
printf '%s\n' "$out" | head -5 >&2
|
||||
err " 密钥来源: $key_file"
|
||||
err " 常见原因：master.key 被重新生成、恢复了不匹配的备份，或换了机器。"
|
||||
return 1
|
||||
}
|
||||
|
||||
# ---------- 适配器备份保留策略 ----------
|
||||
# 每次部署都新建一个 adapters.bak.<时间戳> 目录，而回滚只用到最近一次
|
||||
# （$BACKUP_BIN / $BACKUP_CONFIG 都是单文件覆盖）。多出来的目录从没人清理，
|
||||
# 实测线上已积累 71 个，/etc/llmsproxy 因此涨到 122M。
|
||||
#
|
||||
# 保留最近 KEEP_ADAPTER_BACKUPS 份足够回滚，同时给目录数设上限——否则
|
||||
# 一次误配置（比如 adapter_dir 指错）就可能在几秒内造出成百上千个目录。
|
||||
# 只删名字严格匹配 adapters.bak.<14位时间戳> 的目录，避免误伤人工放的目录。
|
||||
KEEP_ADAPTER_BACKUPS=5
|
||||
|
||||
prune_adapter_backups() {
|
||||
local base="/etc/llmsproxy"
|
||||
[[ -d "$base" ]] || return 0
|
||||
|
||||
# 先按数量上限硬裁：即使时间戳排序失效也不会无上限增长。
|
||||
local all
|
||||
mapfile -t all < <(find "$base" -maxdepth 1 -type d -name 'adapters.bak.*' -printf '%f\n' | sort)
|
||||
local cap=$((KEEP_ADAPTER_BACKUPS * 4))
|
||||
if (( ${#all[@]} > cap )); then
|
||||
warn "适配器备份目录有 ${#all[@]} 个（异常），裁到 $cap"
|
||||
local i=0
|
||||
for d in "${all[@]}"; do
|
||||
i=$((i + 1))
|
||||
# 从最旧的开始删（sort 后升序）。名字不规范的跳过不删。
|
||||
if (( i <= ${#all[@]} - cap )); then
|
||||
if [[ "$d" =~ ^adapters\.bak\.[0-9]{14}$ ]]; then
|
||||
rm -rf "${base:?}/$d"
|
||||
else
|
||||
warn "跳过名字不规范的备份目录（不删）: $d"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
mapfile -t all < <(find "$base" -maxdepth 1 -type d -name 'adapters.bak.*' -printf '%f\n' | sort)
|
||||
fi
|
||||
|
||||
# 再按时间保留最近 KEEP_ADAPTER_BACKUPS 份（sort 后最新在末尾）。
|
||||
local total=${#all[@]} i=0
|
||||
for d in "${all[@]}"; do
|
||||
i=$((i + 1))
|
||||
# i <= total-KEEP 的是较旧的，要删。
|
||||
if (( i <= total - KEEP_ADAPTER_BACKUPS )); then
|
||||
if [[ "$d" =~ ^adapters\.bak\.[0-9]{14}$ ]]; then
|
||||
rm -rf "${base:?}/$d"
|
||||
else
|
||||
warn "跳过名字不规范的备份目录（不删）: $d"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
local left
|
||||
left=$(find "$base" -maxdepth 1 -type d -name 'adapters.bak.*' | wc -l)
|
||||
log " 适配器备份保留最近 $KEEP_ADAPTER_BACKUPS 份（当前剩 $left 个）"
|
||||
}
|
||||
|
||||
# ---------- 同步适配器 ----------
|
||||
sync_adapters() {
|
||||
log "同步适配器"
|
||||
@ -372,7 +261,6 @@ sync_adapters() {
|
||||
cp -rf "$TARGET_ADAPTERS/"*.lua "$BACKUP_DIR/" 2>/dev/null || true
|
||||
log " 旧适配器已备份到 $BACKUP_DIR"
|
||||
fi
|
||||
prune_adapter_backups
|
||||
|
||||
cp -f "$SRC_ADAPTERS/"*.lua "$TARGET_ADAPTERS/"
|
||||
chmod 0644 "$TARGET_ADAPTERS/"*.lua
|
||||
@ -505,10 +393,6 @@ main() {
|
||||
parse_args "$@"
|
||||
log "═══ llmsproxy 原子部署开始 ═══"
|
||||
precheck
|
||||
# 密钥校验放在 build/替换之前：它的失败后果最重（配置里的密文永久解不开），
|
||||
# 必须在动任何文件之前就挡住。此阶段用的是**当前线上二进制**，它已经
|
||||
# 支持 -show-secrets（首次引入该子命令的版本除外，此时跳过并提示）。
|
||||
verify_master_key
|
||||
build
|
||||
sync_adapters
|
||||
backup_old_state
|
||||
|
||||
@ -15,7 +15,7 @@ deployable.
|
||||
|---|---|---|---|
|
||||
| Main | `main` | permanent | Only long-lived branch. Always deployable. Accumulates the next version. |
|
||||
| Feature | `feature/<desc>` | short (dev → merge → delete) | New features / ordinary fixes. Born from `main`, merged back into `main`. |
|
||||
| Release | `release/vX.Y.x` | one version cycle | Cut from `main`, tagged for release. Version-specific hotfixes land here. |
|
||||
| Release | `release/vX.Y.Z` | one version cycle | Cut from `main`, tagged for release. Version-specific hotfixes land here. |
|
||||
|
||||
## Change flow (important)
|
||||
|
||||
@ -26,21 +26,15 @@ deployable.
|
||||
│ │
|
||||
│ cut │ cut
|
||||
▼ ▼
|
||||
release/v1.6.x release/v1.7.x
|
||||
release/v1.4.2 release/v1.4.3
|
||||
│ │
|
||||
tag: v1.6.0 tag: v1.7.6
|
||||
tag: v1.4.2 tag: v1.4.3
|
||||
│ │
|
||||
hotfix ◄─────┘ hotfix ◄─────┘
|
||||
│ │
|
||||
└── cherry-pick back ──────────┘
|
||||
```
|
||||
|
||||
The patch position in the branch name is a literal `x`, while the tag carries
|
||||
the concrete version: one `release/v1.7.x` can hold tags v1.7.0 … v1.7.6.
|
||||
Spanning several patches on a single minor branch is deliberate — patches are
|
||||
revisions of the same feature batch, hotfixes land on one branch, and back-port
|
||||
to main never has to resolve dependencies between several release branches.
|
||||
|
||||
### Key rules
|
||||
|
||||
1. **main is always deployable**: never leave half-done work on `main`.
|
||||
@ -49,9 +43,9 @@ to main never has to resolve dependencies between several release branches.
|
||||
then `git merge --no-ff feature/xxx` (or squash) when done.
|
||||
3. **Release = cut a release branch from main + tag**:
|
||||
```bash
|
||||
git checkout -b release/v1.7.x main
|
||||
git tag -a v1.7.0 -m "ModelRouter v1.7.0"
|
||||
git push origin release/v1.7.x v1.7.0
|
||||
git checkout -b release/v1.4.2 main
|
||||
git tag -a v1.4.2 -m "ModelRouter v1.4.2"
|
||||
git push origin release/v1.4.2 v1.4.2
|
||||
```
|
||||
Build installers and upload the GitCode Release from this tag so the
|
||||
published state is exactly reproducible.
|
||||
@ -60,7 +54,7 @@ to main never has to resolve dependencies between several release branches.
|
||||
an already-released branch (unless you deliberately ship a minor revision).
|
||||
5. **Hotfixes MUST flow back to main**:
|
||||
```bash
|
||||
git checkout release/v1.7.x # fix in the release branch
|
||||
git checkout release/v1.4.2 # fix in the release branch
|
||||
git commit -m "fix: ..."
|
||||
git checkout main
|
||||
git cherry-pick <hotfix-commit> # and into main
|
||||
@ -73,23 +67,12 @@ to main never has to resolve dependencies between several release branches.
|
||||
When the next version ships, the previous release branch retires:
|
||||
|
||||
- **Default: delete the remote release branch**
|
||||
(`git push origin :release/v1.7.x`). All hotfixes were already
|
||||
(`git push origin :release/v1.4.2`). All hotfixes were already
|
||||
cherry-picked into main, so main contains everything; no merge needed.
|
||||
- **Long-term maintenance** (e.g. an enterprise client pinned to an old
|
||||
version): keep the branch, accept only security fixes, keep the
|
||||
commit-then-cherry-pick loop.
|
||||
|
||||
> **Where practice diverged from this section (checked 2026-10-01)**:
|
||||
> `release/v1.4.x` and `release/v1.5.x` still exist locally and on the remote,
|
||||
> so "retire the previous branch when the next version ships" was never
|
||||
> carried out. Keeping them is harmless (hotfixes were back-ported), but it
|
||||
> contradicts the rule above and makes a reader wonder whether they should be
|
||||
> there at all. **Feature branches, by contrast, are cleaned up**:
|
||||
> `feature/key-quota-control`, `feature/toolcall-id-sanitize`,
|
||||
> `feature/anthropic-usage-cache` and `feature/agentrouter-id-sanitize` were
|
||||
> deleted on 2026-10-01 after confirming with a per-commit `git patch-id`
|
||||
> comparison that their work had already landed in main.
|
||||
|
||||
## Explicit non-goals
|
||||
|
||||
- **Never rebase main**: main's history stays append-only; anyone pulling gets
|
||||
@ -151,8 +134,8 @@ pain points:
|
||||
2. No feature branches meant two independent efforts could not proceed in
|
||||
parallel without colliding.
|
||||
|
||||
With release branches: the published state = `release/vX.Y.x` branch +
|
||||
the concrete `vX.Y.Z` tag, exactly reproducible; hotfixes have a clear landing
|
||||
spot; main stays "latest + all fixes + deployable".
|
||||
With release branches: the published state = `release/vX.Y.Z` branch +
|
||||
`vX.Y.Z` tag, exactly reproducible; hotfixes have a clear landing spot; main
|
||||
stays "latest + all fixes + deployable".
|
||||
|
||||
> 中文版见 [docs/git-workflow.md](git-workflow.md)。
|
||||
@ -13,7 +13,7 @@ ModelRouter 采用 **GitHub Flow + 发布分支** 模型：`main` 是唯一长
|
||||
|---|---|---|---|
|
||||
| 主分支 | `main` | 永久 | 唯一长命分支。永远可部署。积攒下一个版本的功能。 |
|
||||
| 特性分支 | `feature/<描述>` | 短命（开发→合并即删） | 新特性 / 一般 bug 修复。从 `main` 开出，完成后合回 `main`。 |
|
||||
| 发布分支 | `release/vX.Y.x` | 一个版本周期 | 从 `main` 分出，打 tag 发布。该版本生命周期内的 hotfix 都提交在此分支。 |
|
||||
| 发布分支 | `release/vX.Y.Z` | 一个版本周期 | 从 `main` 分出，打 tag 发布。该版本生命周期内的 hotfix 都提交在此分支。 |
|
||||
|
||||
## 变更流向（重要）
|
||||
|
||||
@ -24,20 +24,15 @@ ModelRouter 采用 **GitHub Flow + 发布分支** 模型：`main` 是唯一长
|
||||
│ │
|
||||
│ 切出 │ 切出
|
||||
▼ ▼
|
||||
release/v1.6.x release/v1.7.x
|
||||
release/v1.4.2 release/v1.4.3
|
||||
│ │
|
||||
tag: v1.6.0 tag: v1.7.6
|
||||
tag: v1.4.2 tag: v1.4.3
|
||||
│ │
|
||||
hotfix ◄─────┘ hotfix ◄─────┘
|
||||
│ │
|
||||
└── cherry-pick 回 main ───────┘
|
||||
```
|
||||
|
||||
分支名里 patch 位是**字面的 x**，而 tag 打具体版本号：`release/v1.7.x` 这一条
|
||||
发布分支上的 tag 可以有 v1.7.0 … v1.7.6 多个。一条 minor 分支跨多个 patch 是
|
||||
刻意的：patch 是同一批功能的不同修订，hotfix 落在同一条分支上，回流 main 时
|
||||
也不必处理多条 release 分支之间的依赖。
|
||||
|
||||
### 关键规则
|
||||
|
||||
1. **main 永远可部署**：不在 main 上留半成品。任何未完成的工作必须在特性分支上。
|
||||
@ -45,16 +40,16 @@ ModelRouter 采用 **GitHub Flow + 发布分支** 模型：`main` 是唯一长
|
||||
开发完 `git merge --no-ff feature/xxx` 或 squash 合回。
|
||||
3. **发布 = 从 main 切 release 分支 + 打 tag**：
|
||||
```bash
|
||||
git checkout -b release/v1.7.x main
|
||||
git tag -a v1.7.0 -m "ModelRouter v1.7.0"
|
||||
git push origin release/v1.7.x v1.7.0
|
||||
git checkout -b release/v1.4.2 main
|
||||
git tag -a v1.4.2 -m "ModelRouter v1.4.2"
|
||||
git push origin release/v1.4.2 v1.4.2
|
||||
```
|
||||
构建安装包、上传 GitCode Release 都基于这个 tag，保证可精确回溯发布态。
|
||||
4. **版本生命周期内只收该版本的 hotfix**：新特性一律并入 `main` 等下一个版本，
|
||||
绝不塞进已发布的 release 分支（除非主动选择在该版本内发次要版）。
|
||||
5. **hotfix 必须回流 main**：
|
||||
```bash
|
||||
git checkout release/v1.7.x # 在发布分支提交修复
|
||||
git checkout release/v1.4.2 # 在发布分支提交修复
|
||||
git commit -m "fix: ..."
|
||||
git checkout main
|
||||
git cherry-pick <hotfix-commit> # 回主分支
|
||||
@ -66,20 +61,11 @@ ModelRouter 采用 **GitHub Flow + 发布分支** 模型：`main` 是唯一长
|
||||
|
||||
下一个版本发布时，上一个 release 分支退役：
|
||||
|
||||
- **默认：直接删除远端 release 分支**（`git push origin :release/v1.7.x`）。
|
||||
- **默认：直接删除远端 release 分支**（`git push origin :release/v1.4.2`）。
|
||||
因为 hotfix 都已逐个 cherry-pick 回 main，main 已包含全部修复，无需再合并。
|
||||
- **如需要长期维护旧版**（例如企业大客户卡在旧版本）：保留分支，仅 stopship 接受
|
||||
该版本的安全修复，继续走「提交 + cherry-pick 回 main」循环。
|
||||
|
||||
> **实践与本节的历史出入（2026-10-01 核对）**：`release/v1.4.x` 与 `release/v1.5.x`
|
||||
> 至今仍在本地与远端，说明"下一个版本发布就删上一个分支"实际没有执行。
|
||||
> 保留无害（hotfix 已回流），但它与上面写的规则不一致，读文档的人会以为
|
||||
> 这些分支不该存在。**特性分支则确实在清理**：`feature/key-quota-control`、
|
||||
> `feature/toolcall-id-sanitize`、`feature/anthropic-usage-cache`、
|
||||
> `feature/agentrouter-id-sanitize` 四个分支在 2026-10-01 删除——它们的工作
|
||||
> 早已全部进入 main（逐提交用 `git patch-id` 比对确认），留着只是给下个版本
|
||||
> 制造 cherry-pick/merge 陷阱。
|
||||
|
||||
## 明确不做的事
|
||||
|
||||
- **不 rebase main**：`main` 的历史保持追加式，任何人拉取后 `git pull` 都得到直接可用的历史。
|
||||
|
||||
507
docs/plugins.md
507
docs/plugins.md
@ -1,507 +0,0 @@
|
||||
# 插件系统（Plugin System）
|
||||
|
||||
> **English**: this document is the reference for writing ModelRouter plugins.
|
||||
> The Chinese version is the primary one; section titles map 1:1.
|
||||
|
||||
ModelRouter 的插件是**单个 `.lua` 文件**，放在 `config.yaml` 的 `plugin_dir` 目录里。
|
||||
插件能做两件事：
|
||||
|
||||
1. **挂钩子**：在请求流水线的若干 stage 上注册回调，看到每个请求的完整信息，
|
||||
并可以把结果累加进自己的状态。
|
||||
2. **贡献界面**：在启动时返回 HTML / CSS / JS，由内核注入 WebUI——可以是一整个
|
||||
新页面，也可以是往现有页面里追加一个组件。
|
||||
|
||||
两者互相独立：只想统计请求数的插件不必碰界面；只想加个仪表盘的插件不必碰钩子。
|
||||
|
||||
---
|
||||
|
||||
## 1. 快速上手
|
||||
|
||||
一个最小的插件：
|
||||
|
||||
```lua
|
||||
-- plugins/hello.lua
|
||||
local plugin = {
|
||||
name = "hello",
|
||||
version = "1.0.0",
|
||||
description = "示例插件",
|
||||
author = "you",
|
||||
}
|
||||
|
||||
-- 声明钩子
|
||||
plugin.hooks = {
|
||||
request_end = "on_request_end",
|
||||
}
|
||||
|
||||
-- 钩子实现
|
||||
function plugin.on_request_end(payload)
|
||||
-- payload 是解码后的 table，不是 JSON 字符串
|
||||
log("info", string.format("%s via %s: %d prompt tokens",
|
||||
payload.model, payload.source, payload.prompt_tokens or 0))
|
||||
return nil -- 最后一个 stage 没有下游，return 无意义
|
||||
end
|
||||
|
||||
-- 贡献界面
|
||||
plugin.ui = {
|
||||
page = {
|
||||
page_id = "hello", -- kebab-case
|
||||
title = "Hello",
|
||||
icon = "👋",
|
||||
order = 90, -- 侧栏排序
|
||||
mount = [[<div id="hello">hello</div>]],
|
||||
},
|
||||
}
|
||||
|
||||
return plugin -- 必须返回一个 table
|
||||
```
|
||||
|
||||
放进 `plugin_dir` 后重启即生效。`GET /api/plugins` 确认它被加载了。
|
||||
|
||||
---
|
||||
|
||||
## 2. 加载与生命周期
|
||||
|
||||
```
|
||||
core.New
|
||||
└─ lua.NewVM(adapter_dir).Start() 适配器状态
|
||||
└─ lua.NewPlugins(vm, plugin_dir)
|
||||
├─ SeedBundled() 仅当目录不存在时写入内置插件（目前是 billing）
|
||||
└─ LoadDir() 按文件名字典序逐个加载
|
||||
```
|
||||
|
||||
**加载失败不影响网关启动。** 一个语法错误的插件会被记录在 `GET /api/plugins`
|
||||
的 `error` 字段里，永远不会被调用。这与适配器一致，但理由更强：插件是可选的
|
||||
第三方扩展，因为一个 `.lua` 打错字就让网关起不来是错误的取舍。
|
||||
|
||||
**目录一旦存在就是权威的。** 与适配器同规则：首启会 seed 内置插件，之后目录里
|
||||
的文件说了算，删除或编辑内置插件都是真实生效的操作。
|
||||
|
||||
### 2.1 热更新
|
||||
|
||||
| 方式 | 效果 |
|
||||
|---|---|
|
||||
| `POST /api/plugins {name, code}` | 写文件 + 立即加载新版本（旧的 Lua 状态被关闭重建，**累计量清零**） |
|
||||
| `DELETE /api/plugins/{name}` | 删文件 + 卸载 |
|
||||
| 改文件后 `POST` 同名 | 同上 |
|
||||
|
||||
改文件但**不** POST，需要重启才生效。
|
||||
|
||||
---
|
||||
|
||||
## 3. 流水线 stage
|
||||
|
||||
一个请求依次经过三个 stage。插件可以为任意 stage 注册钩子；未注册的 stage
|
||||
被忽略，所以插件不会因为网关将来新增 stage 而报错。
|
||||
|
||||
```
|
||||
客户端请求
|
||||
│
|
||||
┌─────────▼──────────┐
|
||||
│ request_start │ 已解析、已鉴权，尚未选源
|
||||
│ · type │ "chat" | "stream" | "image"
|
||||
│ · model │ 客户端请求的原始 model（"AUTO" 也在这里）
|
||||
│ · key / role │ 掩码后的网关 key id（"***a1b2c3"）与角色
|
||||
│ · source │ 空（还没选源）
|
||||
│ · stream │
|
||||
│ · messages_count │
|
||||
│ · tools_count │
|
||||
│ · ts │ unix 秒
|
||||
└─────────┬──────────┘
|
||||
│ （调度：tier 遍历 → 槽位轮转 → 冷却/配额过滤）
|
||||
┌─────────▼──────────┐
|
||||
│ routed │ 已选定 (source, model)，尚未发往上游
|
||||
│ · source / model │ 实际选中的
|
||||
│ · tier │ AUTO 链的档位；直连 = -1；AUTO = -2
|
||||
│ · stream / key │
|
||||
└─────────┬──────────┘
|
||||
│ （HTTP 往返 / SSE 流）
|
||||
┌─────────▼──────────┐
|
||||
│ request_end │ 每个请求恰好一次，成功失败都触发
|
||||
│ · ok / status │
|
||||
│ · latency_ms │
|
||||
│ · first_byte_ms │ 流式的首字节时间
|
||||
│ · prompt_tokens │ 上游真实 usage，缺失时为字节估算
|
||||
│ · completion_tokens
|
||||
│ · cache_hit_tokens / cache_miss_tokens
|
||||
│ · image_count │ 生图数量（图片不计 token）
|
||||
│ · error │ 失败原因，成功时为 ""
|
||||
│ · time │ unix **毫秒**
|
||||
└─────────┬──────────┘
|
||||
│
|
||||
写审计 + 聚合统计
|
||||
```
|
||||
|
||||
### 3.1 触发点在哪
|
||||
|
||||
| stage | 代码位置 | 说明 |
|
||||
|---|---|---|
|
||||
| `request_start` | `gateway/chat.go` `handleChat` / `fireImageStart` | 每个请求一次（chat 与生图各一条），在配额闸门**之前** |
|
||||
| `chain_step` | `gateway/chat.go` `chainTraceSink` | **仅 AUTO 路径**，每步一次 |
|
||||
| `routed` | `singleChat` / `streamChat` / `singleChatAuto` / `streamChatAuto` | 成功选定源之后，各一次 |
|
||||
| `request_end` | `gateway/chat.go` `writeRec` | **所有出口的唯一汇合点**，每个请求一次 |
|
||||
|
||||
### 3.1b 为什么单独有 `chain_step`
|
||||
|
||||
`routed` 只在**遍历结束后**触发一次，只带最终胜出的槽位。所以
|
||||
"tier 1 冷却所以降级到 tier 3"和"tier 1 正常接单"在它眼里**完全一样**——
|
||||
而这恰恰是优先级链存在的全部理由。
|
||||
|
||||
`chain_step` 补上这条信息，四种 `kind`：
|
||||
|
||||
| kind | 含义 | 何时产生 |
|
||||
|---|---|---|
|
||||
| `tier_skip` | 整档被跳过 | 该档所有槽位冷却中/配额用尽 |
|
||||
| `slot_fail` | 某个槽位硬失败 | 上游报错 / 适配器输出不可用 |
|
||||
| `tier_busy` | 整档全忙且有界等待超时 | 2s 内没等到空位 |
|
||||
| `selected` | 这个槽位接了单 | 每次成功遍历**恰好一次**，且是最后一步 |
|
||||
|
||||
顺序保证：所有 `chain_step` 都在 `routed` 之前，`selected` 是最后一步。
|
||||
所以只订阅 `request_end` 的插件也能拿到轨迹摘要（见下）。
|
||||
|
||||
### 3.1c `request_end` 里的轨迹摘要
|
||||
|
||||
除了逐个 `chain_step`，`request_end` 还带三个便于做报表的字段：
|
||||
|
||||
| 字段 | 含义 |
|
||||
|---|---|
|
||||
| `chain_walk` | 整个遍历的步骤数组（上限 12 步，超出截断） |
|
||||
| `degraded` | 布尔。`true` = 有过跳过/失败，即**发生了降级** |
|
||||
| `tier_served` | 实际服务的那一档；直连或全失败时为 `-1` |
|
||||
|
||||
> **计费口径**：按**实际服务的模型**计费。降级到 tier 3 仍按 tier 3 的价算，
|
||||
> `chain_step` / `degraded` / `tier_served` 只作**观测**，不参与计价。
|
||||
> 理由见 §7.5。
|
||||
|
||||
`request_end` 放在 `writeRec` 是因为四条入口路径（直连/AUTO × 流式/非流式）都
|
||||
经过它，既不会漏（流式的 token 数只有流结束才知道），也不会重复。
|
||||
|
||||
**hot path 注意事项**：没有插件注册某 stage 时，`Fire` 立刻返回（一次 `RLock`
|
||||
加一次 map 查找）。装了插件之后，每个请求会在该 stage 上多一次 Lua 调用——
|
||||
这是同步的，在关键路径上。计费插件那种"每请求一次"是正常的；把重活放进钩子是
|
||||
反模式。
|
||||
|
||||
### 3.2 钩子的返回值
|
||||
|
||||
- 返回 `nil` 或不返回 = **没有意见**，payload 原样传给下一个插件
|
||||
- 返回 table = 其中的键会**合并进 payload**，并作为 `Fire` 的返回值
|
||||
|
||||
**同一 stage 的多个插件并行执行**，但返回值按**插件加载顺序**合并，所以结果是
|
||||
确定的（不依赖 goroutine 调度）。代价是一个插件看不到另一个插件刚加的字段：
|
||||
每个钩子拿到的是**同一份 payload 快照**。
|
||||
|
||||
这与早期版本不同 —— 早期是顺序执行，后一个插件能看到前一个的返回值。它从未被
|
||||
实际依赖（随核心发布的 billing 在每个 stage 都 `return nil`，注释里写着
|
||||
"nobody downstream would read a return value"），但这是一处**契约变化**：如果你的
|
||||
插件依赖「读到前一个插件写的字段」，并行的两个插件之间必须改用外部通信
|
||||
（例如各自写 `plugin.state`，由 `/api/plugins/<name>/state` 读取）。
|
||||
|
||||
单个插件时不启 goroutine，直接调用。
|
||||
|
||||
前三个 stage 的返回值目前没有内部消费者（最后一个 stage 之后就是写审计），
|
||||
所以计费插件改用 `plugin.state` + `/state` 端点来暴露数据。
|
||||
|
||||
---
|
||||
|
||||
## 4. 界面扩展
|
||||
|
||||
### 4.1 整页
|
||||
|
||||
```lua
|
||||
plugin.ui = {
|
||||
page = {
|
||||
page_id = "billing", -- 必填,kebab-case。侧栏 data-tab 与 #tab-billing
|
||||
title = "Billing", -- 必填,侧栏文字
|
||||
icon = "💰", -- 可选
|
||||
order = 40, -- 侧栏排序,默认 100
|
||||
mount = [[...HTML...]],
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### 4.2 往现有页面追加元素
|
||||
|
||||
```lua
|
||||
plugin.ui = {
|
||||
elements = {
|
||||
{
|
||||
target = "status", -- status | chat | keys | sort | sources | adapters
|
||||
anchor = "top", -- "top" | "bottom" | "before:<sel>" | "after:<sel>"
|
||||
order = 5,
|
||||
mount = [[...HTML...]],
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### 4.3 `mount` 里可以带 `<script>` 和 `<style>`
|
||||
|
||||
内核的注入顺序是：**先插 HTML，再执行 `<script>`**，所以脚本里访问
|
||||
`document.getElementById` 一定能拿到已渲染的节点。
|
||||
|
||||
### 4.4 插件可用的浏览器端 API
|
||||
|
||||
| API | 作用 |
|
||||
|---|---|
|
||||
| `window.pluginAPI.fetchState(name)` | 等价于 `GET /api/plugins/<name>/state` |
|
||||
| `window.pluginAPI.onTabShown(fn)` | 注册"页面切到可见时"的回调（轮询类组件用） |
|
||||
| `window.pluginAPI.postState(name, obj)` | `PUT` 自己的 state（写操作，需 admin） |
|
||||
|
||||
### 4.5 一个完整例子
|
||||
|
||||
见 `internal/lua/plugins/billing.lua`——它同时用了两种 UI 形式、完整的价格配置、
|
||||
以及全部三个 stage。
|
||||
|
||||
---
|
||||
|
||||
## 5. 插件状态与 HTTP API
|
||||
|
||||
### 5.1 端点
|
||||
|
||||
```
|
||||
GET /api/plugins 已加载插件清单 + 钩子 + UI + 错误
|
||||
POST /api/plugins 上传/替换（admin）{name, code}
|
||||
DELETE /api/plugins/{name} 删除（admin）
|
||||
GET /api/plugins/{name}/state 读取插件自己发布的状态
|
||||
PUT /api/plugins/{name}/state 替换状态（admin）
|
||||
```
|
||||
|
||||
`GET .../state` 对**任意角色开放**：它是报表数据（开销、计数），用户自己的
|
||||
计费组件要能渲染。而 `PUT` 需要 admin。
|
||||
|
||||
### 5.2 约定：`prices` 与 `state` 分离
|
||||
|
||||
`PUT .../state` 的载荷里如果有 `prices` 键，它会被写进插件的 **`plugin.prices`**
|
||||
字段并从 `state` 里剔除。
|
||||
|
||||
**为什么**：`state` 是累计量（历史），`prices` 是配置。一次改价如果整体替换
|
||||
`state`，累计量就没了——账目会在改价那一刻清零。分离之后：
|
||||
|
||||
- 只带 `prices` → 改配置，**不动** `state`（累计量保留）
|
||||
- 带其它键 → 替换 `state`（这是显式的重置）
|
||||
|
||||
> 插件可以不遵守这个约定，把整个载荷放进 `state` 自行处理。约定只为内置的
|
||||
> 计费插件存在。
|
||||
|
||||
### 5.3 `GET /api/plugins` 的响应
|
||||
|
||||
```json
|
||||
{
|
||||
"plugins": [{
|
||||
"name": "billing", "version": "1.0.0",
|
||||
"description": "...", "author": "...",
|
||||
"hooks": ["request_end"],
|
||||
"ui": { "page": "billing", "elements": 1 },
|
||||
"loaded": true
|
||||
}],
|
||||
"hook_errors": { "request_end": { "count": 3, "last_error": "billing: ..." } },
|
||||
"plugin_dir": "/etc/llmsproxy/plugins"
|
||||
}
|
||||
```
|
||||
|
||||
`hook_errors` 是**排错入口**：一个坏掉的插件表现为"功能缺失"而不是报错，
|
||||
这个计数让它可见。
|
||||
|
||||
---
|
||||
|
||||
## 6. 运行时约束（重要）
|
||||
|
||||
### 6.1 一个插件 = 一个 Lua 状态
|
||||
|
||||
适配器可以有多个独立的 worker 状态（它们无状态，这是对的）。**插件不行**：
|
||||
钩子通常往 `plugin.state` 里累加，多个状态就意味着总额被劈成几份，而写价格只
|
||||
写进其中一个状态，钩子恰好跑到另一个时**所有请求按 0 计费**。
|
||||
|
||||
因此插件持有**唯一一个**状态，所有入口（`Fire` / `State` / `SetState`）用互斥锁
|
||||
串行化。代价：**一个卡住的钩子会卡住所有插件的钩子**。所以——
|
||||
|
||||
> **钩子必须短、同步、不阻塞。** 不要在里面做 HTTP 请求、睡眠或重计算。
|
||||
|
||||
### 6.2 异常被隔离
|
||||
|
||||
钩子里 `error()` 不会影响转发：异常被捕获、记进 `hook_errors`、跳到下一个插件。
|
||||
适配器变换出错会让源进入冷却，但插件出错**不产生任何惩罚**——插件是可选功能。
|
||||
|
||||
### 6.3 内置可用函数
|
||||
|
||||
与适配器相同：`json.encode` / `json.decode`、`log(level, msg)`、
|
||||
`hmac_sha256_hex` / `sha256_hex` / `base64_encode` / `tohex`。
|
||||
|
||||
⚠️ `os.date` 在极简运行时里可能不可用（计费插件因此有降级路径）。
|
||||
|
||||
### 6.4 命名
|
||||
|
||||
- 全局 `__llmsproxy_plugin` 存放插件返回的 table，**不要占用**
|
||||
- 插件间状态完全隔离，一个插件改不了另一个的全局
|
||||
|
||||
---
|
||||
|
||||
## 7. 内置计费插件
|
||||
|
||||
`plugins/billing.lua`，默认随首启 seed 进来，开箱可用。
|
||||
|
||||
### 7.1 计费维度
|
||||
|
||||
| 维度 | 用途 | 优先级 |
|
||||
|---|---|---|
|
||||
| `sources` | 源的固定价（如按请求计费的转发服务） | 中 |
|
||||
| `models` | 模型的 per-token 价 | 高 |
|
||||
| `keys` | 单个网关 key 的覆盖价 | **最高** |
|
||||
|
||||
**token 价**优先级：`keys` > `models` > `default`。
|
||||
**`per_request` 固定价是叠加的**（不覆盖），所以一个生图模型可以既算 token 又收固定费。
|
||||
**`cache_discount`** 见 §7.7。
|
||||
|
||||
### 7.2 价格单位
|
||||
|
||||
**USD / 单个 token**。这是各家 provider 的公布口径，所以典型数值长这样：
|
||||
`1.25e-6`。插件内部乘以 token 数，全程不做单位换算。
|
||||
|
||||
```json
|
||||
{
|
||||
"prices": {
|
||||
"currency": "USD",
|
||||
"default": { "prompt": 0, "completion": 0, "per_request": 0 },
|
||||
"sources": { "trae": { "per_request": 0.01 } },
|
||||
"models": {
|
||||
"gpt-5.4": { "prompt": 1.25e-6, "completion": 1e-5 },
|
||||
"kolors": { "per_request": 0.04 }
|
||||
},
|
||||
"keys": { "***a1b2c3": { "prompt": 1.1e-6, "completion": 9e-6 } }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
配置价目：
|
||||
|
||||
```bash
|
||||
curl -X PUT http://127.0.0.1:8080/api/plugins/billing/state \
|
||||
-H "Authorization: Bearer $ADMIN_KEY" -H "Content-Type: application/json" \
|
||||
-d '{"prices":{"models":{"gpt-5.4":{"prompt":1.25e-6,"completion":1e-5}}}}'
|
||||
```
|
||||
|
||||
读回账目：
|
||||
|
||||
```bash
|
||||
curl -H "Authorization: Bearer $ADMIN_KEY" \
|
||||
http://127.0.0.1:8080/api/plugins/billing/state
|
||||
```
|
||||
|
||||
### 7.3 累计维度
|
||||
|
||||
`total` / `by_source` / `by_model` / `by_key` / `by_day`（`YYYY-MM-DD` UTC），
|
||||
每项含 `cost`、`requests`、`prompt_tokens`、`completion_tokens`、`failures`。
|
||||
|
||||
另有两个**降级观测**维度（来自 `chain_step`）：
|
||||
|
||||
| 字段 | 含义 |
|
||||
|---|---|
|
||||
| `degraded_reqs` | 发生过降级的请求数 |
|
||||
| `by_tier_served` | 各档实际接单数（`{"1": 812, "2": 37}`） |
|
||||
| `skip_reasons` | 跳过原因计数，等待时长已归一（`no free slot within <wait>`） |
|
||||
|
||||
这三项是"网关是不是在悄悄降级"的核心指标：一个持续降级的网关，账单结构和健康
|
||||
网关看起来一模一样——**除非**单独统计降级次数。
|
||||
|
||||
### 7.4 计费策略：失败的请求怎么算
|
||||
|
||||
**保留 token 费用，丢弃固定费用。** 理由：上游在生成后才 500，token 确实被消耗
|
||||
了；但那个从未真正发生的固定费不该收。
|
||||
|
||||
这是本插件里**最可争议的一条**。想改成"失败也收固定费"：
|
||||
|
||||
```json
|
||||
{ "prices": { ... }, "count_failures": true }
|
||||
```
|
||||
|
||||
### 7.5 ⚠️ 计费插件只报表，不执法
|
||||
|
||||
**网关自己的配额会计（`internal/gateway/stats.go`，入口处强制）才是限额权威。**
|
||||
本插件不参与任何路由或配额决策。
|
||||
|
||||
理由：两套独立的会计路径如果对不上，比一套功能略少的更糟。计费是**观察**，
|
||||
配额是**控制**，二者分开。
|
||||
|
||||
### 7.6 未定价流量（重要）
|
||||
|
||||
**任何维度都没配价的请求，成本记 0。** 这是最危险的失败模式：账单照样能加总，
|
||||
只是**悄悄少报**，而且没有任何报错。
|
||||
|
||||
所以插件单独统计它们：
|
||||
|
||||
| 字段 | 含义 |
|
||||
|---|---|
|
||||
| `unpriced_reqs` | 没有任何价目覆盖的请求数 |
|
||||
| `unpriced_models` | 按模型点名（`{"MYSTERY-MODEL": 12}`）——直接告诉你价目表缺了哪一行 |
|
||||
|
||||
仪表盘上有 "Unpriced" 卡片。**这个数应该是 0**；不是 0 就去补价目。
|
||||
|
||||
注意"未定价"不等于"免费"：这些请求的 `requests` / token 数**照常计入**
|
||||
`total` 与各维度，只有金额是 0。
|
||||
|
||||
### 7.7 提示缓存计价
|
||||
|
||||
**缓存命中的 prompt token 不按全价算。** 绝大多数 provider 对缓存读给很深的折扣
|
||||
（常见是 1/10），而 agent 流量会反复重放长前缀——正是缓存要让它便宜的那类流量。
|
||||
|
||||
```
|
||||
fresh = prompt_tokens - cache_hit_tokens → 全价
|
||||
cached = cache_hit_tokens → 全价 × cache_discount
|
||||
```
|
||||
|
||||
`cache_discount` 默认 **0.1**（10 倍折扣），因为 DeepSeek / Qwen / Kimi 等都是这个
|
||||
量级。它是**每个 provider 的事实、不是自然常数**，所以可以按条目覆盖：
|
||||
|
||||
```json
|
||||
"models": { "gpt-5.4": { "prompt": 1.25e-6, "completion": 1e-5, "cache_discount": 0.25 } }
|
||||
```
|
||||
|
||||
设成 `1` 恢复成旧的"prompt 一律全价"行为，设成 `0` 表示该 provider 不打折。
|
||||
|
||||
优先级与 token 价一致（`keys` > `models` > `default`）。
|
||||
|
||||
> **这一条改过行为。** 修复前缓存命中按全价算：100 万 prompt token 里 90 万是
|
||||
> 缓存命中，会算出 10 USD 而不是 ~1.9——**高估约 10 倍**，而且恰好发生在缓存
|
||||
> 最有价值的高频流量上。
|
||||
|
||||
### 7.8 数据从哪来
|
||||
|
||||
`request_end` 的 `prompt_tokens` / `completion_tokens` 优先取上游真实的
|
||||
`usage`；上游没报时网关用字节估算（`len/3+1`）。流式请求在流结束后用上游真实
|
||||
数字**覆盖**估算值。所以计费数字的精度取决于上游是否报 usage。
|
||||
|
||||
---
|
||||
|
||||
## 8. 排错
|
||||
|
||||
| 现象 | 查什么 |
|
||||
|---|---|
|
||||
| 插件没出现在 `/api/plugins` | 目录对不对（响应里的 `plugin_dir`）；文件是不是 `.lua` |
|
||||
| `loaded: false` 且有 `error` | 语法错误或没 `return table`，错误信息在 `error` 字段 |
|
||||
| 功能"没反应"但无报错 | 查 `hook_errors`——坏钩子只记录不抛出 |
|
||||
| 钩子没被调用 | 该 stage 确实触发了吗（`routed` 只在**成功**选源后触发，调度全失败时不触发） |
|
||||
| 界面空白 | 内核的 `GET /api/ui-inject` 里有没有你的 `page_id`；脚本有没有报错（浏览器 console） |
|
||||
| `pluginAPI` 未定义 | 脚本在注入前执行了；确认用的是 `mount` 而不是别的方式插入 |
|
||||
|
||||
### 调试钩子
|
||||
|
||||
插件的 `log(level, msg)` 输出到网关的日志：
|
||||
|
||||
```
|
||||
journalctl -u llmsproxy -f | grep -i plugin
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. 与适配器的区别
|
||||
|
||||
| | 适配器 | 插件 |
|
||||
|---|---|---|
|
||||
| 位置 | `adapter_dir` | `plugin_dir` |
|
||||
| 作用 | 上游协议转换 | 请求流水线 + 界面 |
|
||||
| 钩子 | `transform_request` / `transform_response` / `transform_stream_chunk` / `build_headers` / `transform_error` | `request_start` / `routed` / `request_end` |
|
||||
| 状态 | 每个源独立，多 worker | 单状态，见 §6.1 |
|
||||
| 出错后果 | 该 (源,模型) 进入冷却退避 | 仅该功能缺失 |
|
||||
| 鉴权 | 需要 `gateway_keys` 之外的独立凭据 | 网关 key |
|
||||
| 必需性 | 核心 | 可选，缺了网关照常跑 |
|
||||
|
||||
完整适配器协议见 [lua-adapters.md](lua-adapters.md)。
|
||||
141
e2e/e2e_test.go
141
e2e/e2e_test.go
@ -326,144 +326,3 @@ func statusOf(resp *http.Response) int {
|
||||
}
|
||||
return resp.StatusCode
|
||||
}
|
||||
|
||||
// The startup warning about key configuration must describe the key set that
|
||||
// AUTH actually uses (cfg.Keys, via core.ListKeys), not the legacy
|
||||
// gateway_keys YAML list.
|
||||
//
|
||||
// It used to check GatewayKeys() only. seedKeys copies gateway_keys into keys[]
|
||||
// on first start and the YAML list stops being consulted, so once that list was
|
||||
// emptied — e.g. after rotating the starter key away — the process logged
|
||||
// "every request will be rejected" while seven working keys, one of them admin,
|
||||
// were in service. Observed in production: the warning appeared and that same
|
||||
// key returned HTTP 200 on /v1/models.
|
||||
//
|
||||
// The fixture reproduces exactly that shape: gateway_keys EMPTY, with the
|
||||
// working key present only in keys[].
|
||||
func TestStartupWarningReflectsRealKeysNotLegacyList(t *testing.T) {
|
||||
up := newMockUpstream(t)
|
||||
dir := t.TempDir()
|
||||
|
||||
// Seed a real install once so keys[] gets populated from gateway_keys, then
|
||||
// rewrite the file to look like a rotated install: legacy list cleared, the
|
||||
// seeded key still present in keys[].
|
||||
path := writeConfig(t, dir, "127.0.0.1:0", map[string]*mockUpstream{"good": up})
|
||||
bin := buildBinary(t)
|
||||
|
||||
// First run creates the sealed config with keys[] populated.
|
||||
seedPort := freePort(t)
|
||||
cfg1 := writeConfig(t, dir, "127.0.0.1:"+seedPort, map[string]*mockUpstream{"good": up})
|
||||
g1 := startGateway(t, bin, "127.0.0.1:"+seedPort, cfg1)
|
||||
g1.stop()
|
||||
|
||||
data, err := os.ReadFile(cfg1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(data), "keys:") {
|
||||
t.Fatalf("first run did not populate keys[]:\n%s", data)
|
||||
}
|
||||
|
||||
// Now clear the legacy gateway_keys list, keeping keys[] intact. The
|
||||
// rewritten file may use different indentation, so match loosely.
|
||||
cleared, n := clearGatewayKeys(string(data))
|
||||
if n == 0 {
|
||||
t.Fatalf("could not find a gateway_keys entry to clear in:\n%s", data)
|
||||
}
|
||||
if err := os.WriteFile(cfg1, []byte(cleared), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Start again and inspect the log. The config pins its listen address, so
|
||||
// rewrite that field to the port we will actually poll.
|
||||
port2 := freePort(t)
|
||||
rehosted, ok := replaceListen(string(cleared), "127.0.0.1:"+port2)
|
||||
if !ok {
|
||||
t.Fatalf("could not rewrite listen in:\n%s", cleared)
|
||||
}
|
||||
cfg2 := filepath.Join(dir, "config2.yaml")
|
||||
if err := os.WriteFile(cfg2, []byte(rehosted), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
g2 := startGateway(t, bin, "127.0.0.1:"+port2, cfg2)
|
||||
defer g2.stop()
|
||||
|
||||
// Sanity: the key really does authenticate, so the warning would be a lie.
|
||||
resp, _ := g2.do("GET", "/v1/models", "", true)
|
||||
if resp == nil || resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("key from keys[] did not authenticate (status %v); test fixture is wrong",
|
||||
resp)
|
||||
}
|
||||
|
||||
logs := g2.logString()
|
||||
if strings.Contains(logs, "gateway_keys is EMPTY") {
|
||||
t.Errorf("startup logged 'gateway_keys is EMPTY ... every request is rejected' while "+
|
||||
"a working admin key was in service — the warning must read the authoritative "+
|
||||
"key set (core.ListKeys), not the legacy YAML list.\nlog:\n%s", logs)
|
||||
}
|
||||
_ = path
|
||||
_ = g1
|
||||
}
|
||||
|
||||
func freePort(t *testing.T) string {
|
||||
t.Helper()
|
||||
l, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer l.Close()
|
||||
_, port, _ := net.SplitHostPort(l.Addr().String())
|
||||
return port
|
||||
}
|
||||
|
||||
// stop terminates the gateway and waits, so its log buffer is complete.
|
||||
func (g *gatewayUnderTest) stop() {
|
||||
if g == nil || g.cmd == nil || g.cmd.Process == nil {
|
||||
return
|
||||
}
|
||||
_ = g.cmd.Process.Kill()
|
||||
_, _ = g.cmd.Process.Wait()
|
||||
}
|
||||
|
||||
// logString returns everything the gateway has written so far.
|
||||
func (g *gatewayUnderTest) logString() string { return g.log.String() }
|
||||
|
||||
// clearGatewayKeys empties the gateway_keys list in a config, tolerating any
|
||||
// indentation the YAML writer chose. Returns the new text and how many entries
|
||||
// it removed.
|
||||
func clearGatewayKeys(src string) (string, int) {
|
||||
lines := strings.Split(src, "\n")
|
||||
var out []string
|
||||
removed := 0
|
||||
inList := false
|
||||
for _, l := range lines {
|
||||
trimmed := strings.TrimSpace(l)
|
||||
if strings.HasPrefix(l, "gateway_keys:") {
|
||||
out = append(out, "gateway_keys: []")
|
||||
inList = true
|
||||
removed++
|
||||
continue
|
||||
}
|
||||
if inList {
|
||||
if strings.HasPrefix(trimmed, "- ") {
|
||||
removed++
|
||||
continue
|
||||
}
|
||||
inList = false
|
||||
}
|
||||
out = append(out, l)
|
||||
}
|
||||
return strings.Join(out, "\n"), removed
|
||||
}
|
||||
|
||||
// replaceListen rewrites the top-level listen: value.
|
||||
func replaceListen(src, addr string) (string, bool) {
|
||||
lines := strings.Split(src, "\n")
|
||||
for i, l := range lines {
|
||||
if strings.HasPrefix(l, "listen:") {
|
||||
lines[i] = "listen: " + addr
|
||||
return strings.Join(lines, "\n"), true
|
||||
}
|
||||
}
|
||||
return src, false
|
||||
}
|
||||
|
||||
@ -1,264 +0,0 @@
|
||||
package billing
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// Compile turns one billing profile into the prices table the billing plugin
|
||||
// expects, resolving URL rules against the gateway's actual sources.
|
||||
//
|
||||
// WHY URL RULES NEED RESOLVING AT ALL: the operator declares pricing by URL
|
||||
// because that is what a provider's price list is keyed on, and because several
|
||||
// sources can point at the same URL. The plugin, however, looks up by
|
||||
// `payload.source` (a source NAME) and by model — it has no idea what URL a
|
||||
// request went to. So the URL match happens here, at compile time, where the
|
||||
// config's name->base_url mapping is known, and the result is expressed in the
|
||||
// dimensions the plugin already supports.
|
||||
//
|
||||
// Profiles are the "let the user choose" axis: the same URL can appear in
|
||||
// several profiles and switching recomputes this table, so a gateway can be
|
||||
// repriced without editing the plugin.
|
||||
func Compile(profile *config.BillingProfile, sources []config.Source) (map[string]interface{}, error) {
|
||||
return CompileOpts(profile, sources, false)
|
||||
}
|
||||
|
||||
// CompileOpts is Compile with an explicit policy for rules that match no
|
||||
// source.
|
||||
//
|
||||
// The default is STRICT and that is correct for startup: a profile with a
|
||||
// typo'd URL prices nothing, every request on it is recorded as unpriced, and
|
||||
// the bill silently comes out wrong. Failing the load is the right response.
|
||||
//
|
||||
// The rules editor needs the opposite. An operator editing a half-finished
|
||||
// profile — adding a rule before the source that will use it exists, or
|
||||
// renaming a URL while typing — must be able to save and then see the warning,
|
||||
// not be blocked by an error they can only resolve by guessing. Lenient mode
|
||||
// keeps the offending rule in the config and drops it from the compiled table,
|
||||
// so it is preserved as text and clearly reported, without pretending it
|
||||
// prices anything.
|
||||
func CompileOpts(profile *config.BillingProfile, sources []config.Source, lenient bool) (map[string]interface{}, error) {
|
||||
if profile == nil {
|
||||
return nil, fmt.Errorf("no billing profile")
|
||||
}
|
||||
prices := map[string]interface{}{
|
||||
"currency": "USD",
|
||||
"default": map[string]interface{}{"prompt": 0.0, "completion": 0.0, "per_request": 0.0},
|
||||
"sources": map[string]interface{}{},
|
||||
"models": map[string]interface{}{},
|
||||
"keys": map[string]interface{}{},
|
||||
}
|
||||
if profile.Default != "" {
|
||||
prices["default_mode"] = profile.Default
|
||||
}
|
||||
|
||||
// Currency is a profile-level statement; a rule may override it.
|
||||
for i := range profile.Rules {
|
||||
if c := profile.Rules[i].Currency; c != "" {
|
||||
prices["currency"] = c
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
matched := map[string]bool{}
|
||||
for i := range profile.Rules {
|
||||
rule := &profile.Rules[i]
|
||||
targets := matchSources(rule.URL, sources)
|
||||
if len(targets) == 0 && rule.URL != "*" {
|
||||
// A rule for a URL no source uses is almost always a typo (or a
|
||||
// source that was removed). Failing loudly beats a profile that
|
||||
// silently prices nothing — the whole reason this is a config file
|
||||
// instead of a hand-written JSON blob.
|
||||
if lenient {
|
||||
continue // kept in the config text, reported as a warning
|
||||
}
|
||||
return nil, fmt.Errorf("rule url %q matches no configured source base_url", rule.URL)
|
||||
}
|
||||
for _, srcName := range targets {
|
||||
if matched[srcName] {
|
||||
// First rule wins. Two rules matching one source is ambiguous,
|
||||
// and silently letting the later one win makes the file's
|
||||
// meaning depend on ordering the operator cannot see.
|
||||
continue
|
||||
}
|
||||
matched[srcName] = true
|
||||
if err := applyRule(prices, rule, srcName, sources); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Sources no rule matched fall through to the profile default, which the
|
||||
// plugin applies via prices.default. Recording them explicitly means the
|
||||
// UI can say "this source is unpriced" instead of leaving the operator to
|
||||
// infer it from a zero.
|
||||
var unmatched []string
|
||||
for _, src := range sources {
|
||||
if !matched[src.Name] {
|
||||
unmatched = append(unmatched, src.Name)
|
||||
}
|
||||
}
|
||||
if len(unmatched) > 0 {
|
||||
prices["unmatched_sources"] = unmatched
|
||||
}
|
||||
return prices, nil
|
||||
}
|
||||
|
||||
// matchSources returns the source names whose base_url matches pattern.
|
||||
// "*" matches every source (used as a catch-all default rule).
|
||||
func matchSources(pattern string, sources []config.Source) []string {
|
||||
var out []string
|
||||
if pattern == "*" {
|
||||
for _, s := range sources {
|
||||
out = append(out, s.Name)
|
||||
}
|
||||
return out
|
||||
}
|
||||
want := normalizeURL(pattern)
|
||||
for _, s := range sources {
|
||||
if normalizeURL(s.BaseURL) == want {
|
||||
out = append(out, s.Name)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// normalizeURL compares URLs the way an operator expects: trailing slashes and
|
||||
// case in the host are differences the provider's price list does not care
|
||||
// about, and requiring an exact byte match would make the config brittle.
|
||||
func normalizeURL(u string) string {
|
||||
u = strings.TrimSpace(u)
|
||||
u = strings.TrimRight(u, "/")
|
||||
return strings.ToLower(u)
|
||||
}
|
||||
|
||||
// applyRule writes one rule's pricing for one source into the prices table.
|
||||
func applyRule(prices map[string]interface{}, rule *config.BillingRule, srcName string, sources []config.Source) error {
|
||||
switch rule.Mode {
|
||||
case "free":
|
||||
// Explicitly priced at zero. This is NOT the same as unpriced: a source
|
||||
// the operator says is free must not appear in the unpriced warnings,
|
||||
// or those warnings become noise and stop being read.
|
||||
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
|
||||
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
|
||||
}
|
||||
return nil
|
||||
|
||||
case "subscription":
|
||||
// A fixed monthly commitment: the per-request MARGINAL cost is zero, and
|
||||
// the flat fee is reported separately. Spreading a monthly fee across
|
||||
// requests would invent a per-request number the provider never charges,
|
||||
// and it would change every time traffic did.
|
||||
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
|
||||
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
|
||||
}
|
||||
fixed, _ := prices["subscriptions"].(map[string]interface{})
|
||||
if fixed == nil {
|
||||
fixed = map[string]interface{}{}
|
||||
prices["subscriptions"] = fixed
|
||||
}
|
||||
cur := "USD"
|
||||
if rule.Currency != "" {
|
||||
cur = rule.Currency
|
||||
}
|
||||
fixed[srcName] = map[string]interface{}{
|
||||
"monthly": rule.Subscription, "currency": cur,
|
||||
}
|
||||
return nil
|
||||
|
||||
case "unpriced":
|
||||
// Deliberately left out of `sources` so the plugin's unpriced_models /
|
||||
// unpriced_reqs counters catch it. That is the point: a subscription
|
||||
// plan whose credits cannot be converted to tokens must be VISIBLE as
|
||||
// unbilled, not quietly estimated.
|
||||
return nil
|
||||
|
||||
case "token":
|
||||
if len(rule.Models) == 0 {
|
||||
return fmt.Errorf("rule for url %q: mode token requires at least one model", rule.URL)
|
||||
}
|
||||
models := prices["models"].(map[string]interface{})
|
||||
for modelID, t := range rule.Models {
|
||||
entry := map[string]interface{}{}
|
||||
prompt, err := usdPerM(t.Prompt)
|
||||
if err != nil {
|
||||
return fmt.Errorf("model %q prompt: %w", modelID, err)
|
||||
}
|
||||
completion, err := usdPerM(t.Completion)
|
||||
if err != nil {
|
||||
return fmt.Errorf("model %q completion: %w", modelID, err)
|
||||
}
|
||||
entry["prompt"] = prompt
|
||||
entry["completion"] = completion
|
||||
if t.CacheDiscount != nil {
|
||||
entry["cache_discount"] = *t.CacheDiscount
|
||||
}
|
||||
if rule.Peak != nil {
|
||||
entry["peak"] = compilePeak(rule.Peak)
|
||||
}
|
||||
models[modelID] = entry
|
||||
}
|
||||
// A specific-URL token rule needs a source entry so a model served
|
||||
// from THIS url but absent from `rule.Models` is still recognised as
|
||||
// belonging to a priced provider.
|
||||
//
|
||||
// A WILDCARD rule ("*") must NOT get one. "*" is a model catalogue — it
|
||||
// says "these model ids have these prices, wherever they come from" —
|
||||
// and marking every source priced at {0,0} would make priceFor() set
|
||||
// priced=true for every model the catalogue does NOT list. Paid
|
||||
// sources (commandcode, alittokenplan, …) would then record $0 instead
|
||||
// of showing up in unpriced_reqs, which is precisely the silent
|
||||
// zero-billing failure this whole plugin is built to make visible.
|
||||
// The model lookup p.models[payload.model] already marks a listed
|
||||
// model priced on its own, so the wildcard loses nothing.
|
||||
if rule.URL != "*" {
|
||||
if _, ok := prices["sources"].(map[string]interface{})[srcName]; !ok {
|
||||
prices["sources"].(map[string]interface{})[srcName] = map[string]interface{}{
|
||||
"prompt": 0.0, "completion": 0.0, "per_request": 0.0,
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("unknown mode %q", rule.Mode)
|
||||
}
|
||||
|
||||
// usdPerM converts a USD-per-million string into the per-token rate the plugin
|
||||
// expects. Delegates to config.ParseFloatUSDPerM so the DSL validator and the
|
||||
// compiler agree byte-for-byte on what a valid price is — a value that passes
|
||||
// Validate() but fails here (or vice versa) would be the worst kind of drift.
|
||||
func usdPerM(s string) (float64, error) {
|
||||
return config.ParseFloatUSDPerM(s)
|
||||
}
|
||||
|
||||
// compilePeak renders the peak window in the shape the plugin reads:
|
||||
// { multiplier, windows = { { days = {...}, hours = { {lo,hi}, ... } } } }.
|
||||
//
|
||||
// The plugin reads `days` / `hours` pairs. The first version of the deployed
|
||||
// price table used { start, end, weekdays } — a shape nothing reads — so peak
|
||||
// traffic was billed at off-peak rates with no error anywhere. Compiling from
|
||||
// typed config fields removes the chance of writing the wrong key names by hand.
|
||||
func compilePeak(p *config.BillingPeak) map[string]interface{} {
|
||||
win := map[string]interface{}{}
|
||||
if len(p.Weekdays) > 0 {
|
||||
days := make([]interface{}, 0, len(p.Weekdays))
|
||||
for _, d := range p.Weekdays {
|
||||
days = append(days, d)
|
||||
}
|
||||
win["days"] = days
|
||||
}
|
||||
if len(p.Hours) > 0 {
|
||||
hours := make([]interface{}, 0, len(p.Hours))
|
||||
for _, h := range p.Hours {
|
||||
hours = append(hours, []interface{}{h[0], h[1]})
|
||||
}
|
||||
win["hours"] = hours
|
||||
}
|
||||
out := map[string]interface{}{"multiplier": p.Multiplier}
|
||||
if len(win) > 0 {
|
||||
out["windows"] = []interface{}{win}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@ -1,88 +0,0 @@
|
||||
package billing
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// The wildcard token rule must not blanket-mark every source as priced.
|
||||
//
|
||||
// Why this is a hazard and not a style question: priceFor() sets priced=true as
|
||||
// soon as the request's SOURCE has an entry in the price table, and that flag
|
||||
// is what keeps a request OUT of unpriced_reqs. A "*" rule is a model
|
||||
// catalogue — it prices specific model ids wherever they come from — but
|
||||
// compiling it into a {0,0,0} entry on all 21 sources made every model the
|
||||
// catalogue does not list look "priced at zero". Paid providers (commandcode,
|
||||
// alittokenplan, …) then stopped appearing in the unpriced warnings, which is
|
||||
// exactly the silent zero-billing failure the plugin exists to surface.
|
||||
func TestWildcardTokenRuleDoesNotMarkEverySourcePriced(t *testing.T) {
|
||||
sources := []config.Source{
|
||||
{Name: "free1", BaseURL: "https://free1.example.com/v1"},
|
||||
{Name: "paid1", BaseURL: "https://paid1.example.com/v1"},
|
||||
{Name: "paid2", BaseURL: "https://paid2.example.com/v1"},
|
||||
}
|
||||
profile := &config.BillingProfile{
|
||||
ID: "p",
|
||||
Rules: []config.BillingRule{
|
||||
{URL: "https://free1.example.com/v1", Mode: "free"},
|
||||
{
|
||||
URL: "*", Mode: "token",
|
||||
Models: map[string]config.BillingToken{
|
||||
"only-catalogued-model": {Prompt: "1.00", Completion: "2.00"},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
prices, err := Compile(profile, sources)
|
||||
if err != nil {
|
||||
t.Fatalf("compile: %v", err)
|
||||
}
|
||||
srcs := prices["sources"].(map[string]interface{})
|
||||
|
||||
// The explicitly free source keeps its entry (free is a deliberate 0).
|
||||
if _, ok := srcs["free1"]; !ok {
|
||||
t.Error("the explicit free rule lost its source entry")
|
||||
}
|
||||
// Paid sources must NOT be marked, or their unlisted models bill $0.
|
||||
for _, name := range []string{"paid1", "paid2"} {
|
||||
if _, ok := srcs[name]; ok {
|
||||
t.Errorf("★ wildcard token rule marked %q as priced at 0 — its unlisted "+
|
||||
"models will bill $0 instead of appearing as unpriced", name)
|
||||
}
|
||||
}
|
||||
// The catalogue itself must still be present, or the rule prices nothing.
|
||||
models := prices["models"].(map[string]interface{})
|
||||
if _, ok := models["only-catalogued-model"]; !ok {
|
||||
t.Error("the wildcard rule did not publish its model price")
|
||||
}
|
||||
}
|
||||
|
||||
// The inverse guard: a token rule for a SPECIFIC url still marks that source,
|
||||
// because it genuinely declares pricing for the models served from it.
|
||||
func TestSpecificURLTokenRuleStillMarksItsSource(t *testing.T) {
|
||||
sources := []config.Source{
|
||||
{Name: "mine", BaseURL: "https://mine.example.com/v1"},
|
||||
{Name: "other", BaseURL: "https://other.example.com/v1"},
|
||||
}
|
||||
profile := &config.BillingProfile{
|
||||
ID: "p",
|
||||
Rules: []config.BillingRule{{
|
||||
URL: "https://mine.example.com/v1", Mode: "token",
|
||||
Models: map[string]config.BillingToken{
|
||||
"m": {Prompt: "1.00", Completion: "2.00"},
|
||||
},
|
||||
}},
|
||||
}
|
||||
prices, err := Compile(profile, sources)
|
||||
if err != nil {
|
||||
t.Fatalf("compile: %v", err)
|
||||
}
|
||||
srcs := prices["sources"].(map[string]interface{})
|
||||
if _, ok := srcs["mine"]; !ok {
|
||||
t.Error("a specific-url token rule must still mark its own source priced")
|
||||
}
|
||||
if _, ok := srcs["other"]; ok {
|
||||
t.Error("an unrelated source was marked priced by a specific-url rule")
|
||||
}
|
||||
}
|
||||
@ -7,8 +7,6 @@ import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
@ -21,7 +19,6 @@ type Config struct {
|
||||
GatewayKeys []string `yaml:"gateway_keys"`
|
||||
DefaultModel string `yaml:"default_model"` // e.g. "AUTO" or a model id
|
||||
AdapterDir string `yaml:"adapter_dir"`
|
||||
PluginDir string `yaml:"plugin_dir,omitempty"` // Lua plugin directory (optional; hooks + WebUI extensions)
|
||||
RuntimeFile string `yaml:"runtime_file"`
|
||||
TLSCertFile string `yaml:"tls_cert_file,omitempty"` // PEM cert; when set together with tls_key_file, serve HTTPS
|
||||
TLSKeyFile string `yaml:"tls_key_file,omitempty"` // PEM private key
|
||||
@ -30,20 +27,6 @@ type Config struct {
|
||||
Auto []ModelScope `yaml:"auto,omitempty"` // AUTO 调度链规则（WebUI 优先级页编辑，chat）
|
||||
AutoImage []ModelScope `yaml:"auto_image,omitempty"` // AUTO 生图调度链规则（WebUI 优先级页·生图）
|
||||
Keys []GWKey `yaml:"keys,omitempty"` // 网关密钥（WebUI 密钥页管理）
|
||||
// BillingDSL declares per-URL pricing profiles for the billing plugin.
|
||||
//
|
||||
// Declarative, because hand-writing the prices JSON was error-prone in ways
|
||||
// that looked successful: the first PUT went out without the required
|
||||
// `prices` wrapper and silently REPLACED the accumulated totals with the
|
||||
// price table; the second used a peak-window shape the plugin does not read,
|
||||
// so peak traffic was billed at off-peak rates — no error, the bill just
|
||||
// quietly halved. A schema-checked config file catches both at load.
|
||||
BillingDSL *BillingDSL `yaml:"billing,omitempty"`
|
||||
// box seals credentials (sources' api_key/headers, keys' key) at rest.
|
||||
// In-memory values are always plaintext; only the bytes on disk are sealed.
|
||||
// Wired by AttachSecretBox — Load leaves it nil so `-check` and tests stay
|
||||
// filesystem-free.
|
||||
box *SecretBox
|
||||
}
|
||||
|
||||
// Defaults applied to any source (YAML or runtime) that leaves a field unset.
|
||||
@ -144,7 +127,6 @@ func writeDefaultConfig(path string) error {
|
||||
GatewayKeys: []string{key},
|
||||
DefaultModel: "AUTO",
|
||||
AdapterDir: filepath.Join(abs, "adapters"),
|
||||
PluginDir: filepath.Join(abs, "plugins"),
|
||||
RuntimeFile: filepath.Join(abs, "runtime.json"),
|
||||
}
|
||||
out, err := yaml.Marshal(&cfg)
|
||||
@ -210,15 +192,10 @@ func RemoveSourceFromYAML(path, name string) error {
|
||||
|
||||
// UpsertSourceInYAML adds or updates a source entry in the YAML config file.
|
||||
// Uses yaml.Node to preserve the rest of the file's comments and formatting.
|
||||
// When box is non-nil the source's credentials are sealed before writing, so a
|
||||
// newly added source never lands in the file as plaintext.
|
||||
func UpsertSourceInYAML(path, name string, src Source, box *SecretBox) error {
|
||||
func UpsertSourceInYAML(path, name string, src Source) error {
|
||||
if path == "" {
|
||||
return fmt.Errorf("config path is empty")
|
||||
}
|
||||
if err := sealSource(&src, box); err != nil {
|
||||
return err
|
||||
}
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return err
|
||||
@ -276,17 +253,11 @@ func UpsertSourceInYAML(path, name string, src Source, box *SecretBox) error {
|
||||
|
||||
// Save writes the current config back to the YAML file (preserving comments
|
||||
// via yaml.Node round-trip when possible, or full marshaling as fallback).
|
||||
// Credentials are sealed on the way out and the in-memory copy is restored to
|
||||
// plaintext afterwards, so callers keep working with usable values.
|
||||
func (c *Config) Save() error {
|
||||
if c.Path == "" {
|
||||
return fmt.Errorf("config path is empty")
|
||||
}
|
||||
if err := c.sealInPlace(c.box); err != nil {
|
||||
return err
|
||||
}
|
||||
out, err := yaml.Marshal(c)
|
||||
c.unsealAfterWrite(c.box)
|
||||
if err != nil {
|
||||
return fmt.Errorf("marshal config: %w", err)
|
||||
}
|
||||
@ -301,9 +272,6 @@ func (c *Config) ApplyDefaults() error {
|
||||
if c.AdapterDir == "" {
|
||||
c.AdapterDir = "adapters"
|
||||
}
|
||||
// PluginDir is deliberately NOT defaulted: with no plugins a missing
|
||||
// directory would be created and seeded on every start for nothing. An
|
||||
// operator opts in by setting it (or by dropping files into it).
|
||||
if c.RuntimeFile == "" {
|
||||
c.RuntimeFile = "runtime.json"
|
||||
}
|
||||
@ -384,233 +352,24 @@ type SourceTemplate struct {
|
||||
// (full management) or "user" (sees only its own key); Models is the allowed
|
||||
// model scope with per-model token quota (0 = unlimited).
|
||||
type GWKey struct {
|
||||
Key string `yaml:"key" json:"key"`
|
||||
Role string `yaml:"role" json:"role"`
|
||||
Name string `yaml:"name,omitempty" json:"name,omitempty"`
|
||||
// Auto is this key's own AUTO scheduling chain. It overrides the global
|
||||
// cfg.Auto for requests authenticated with this key; nil (or an empty
|
||||
// list) means "inherit the global chain", so keys created before per-key
|
||||
// chains existed keep working untouched.
|
||||
Auto []ModelScope `yaml:"auto,omitempty" json:"auto,omitempty"`
|
||||
Models []ModelScope `yaml:"models,omitempty" json:"models,omitempty"`
|
||||
Note string `yaml:"note,omitempty" json:"note,omitempty"`
|
||||
CreatedAt int64 `yaml:"created_at,omitempty" json:"created_at,omitempty"`
|
||||
Seed bool `yaml:"seed,omitempty" json:"seed,omitempty"` // true if migrated from config gateway_keys
|
||||
Key string `yaml:"key" json:"key"`
|
||||
Role string `yaml:"role" json:"role"`
|
||||
Name string `yaml:"name,omitempty" json:"name,omitempty"`
|
||||
Models []ModelScope `yaml:"models,omitempty" json:"models,omitempty"`
|
||||
Note string `yaml:"note,omitempty" json:"note,omitempty"`
|
||||
CreatedAt int64 `yaml:"created_at,omitempty" json:"created_at,omitempty"`
|
||||
Seed bool `yaml:"seed,omitempty" json:"seed,omitempty"` // true if migrated from config gateway_keys
|
||||
}
|
||||
|
||||
// HasOwnAuto reports whether the key declares its own AUTO chain rather than
|
||||
// inheriting the global one. An explicitly empty list still counts as "no own
|
||||
// chain" so a user can clear an override back to the global default.
|
||||
func (k GWKey) HasOwnAuto() bool { return len(k.Auto) > 0 }
|
||||
|
||||
// BillingDSL holds declarative per-URL pricing profiles for the billing plugin.
|
||||
//
|
||||
// Profiles are the "let the user pick" axis: the same upstream URL can be
|
||||
// declared under multiple profiles, and the operator chooses which is active.
|
||||
// Switching recomputes and re-injects the billing plugin's prices table.
|
||||
type BillingDSL struct {
|
||||
// Active is the id of the profile in effect. Empty => the first profile.
|
||||
Active string `yaml:"active,omitempty" json:"active,omitempty"`
|
||||
Profiles []BillingProfile `yaml:"profiles,omitempty" json:"profiles,omitempty"`
|
||||
}
|
||||
|
||||
// BillingProfile is one named set of per-URL pricing rules.
|
||||
type BillingProfile struct {
|
||||
ID string `yaml:"id" json:"id"`
|
||||
Label string `yaml:"label,omitempty" json:"label,omitempty"`
|
||||
// Default applies when no rule matches: "free" or "unpriced".
|
||||
Default string `yaml:"default,omitempty" json:"default,omitempty"`
|
||||
Rules []BillingRule `yaml:"rules,omitempty" json:"rules,omitempty"`
|
||||
}
|
||||
|
||||
// BillingRule declares the pricing for everything under one URL.
|
||||
type BillingRule struct {
|
||||
// URL matches a source's base_url. "*" = any unmatched URL.
|
||||
URL string `yaml:"url" json:"url"`
|
||||
// Mode is one of: free, token, subscription, unpriced.
|
||||
Mode string `yaml:"mode" json:"mode"`
|
||||
Currency string `yaml:"currency,omitempty" json:"currency,omitempty"`
|
||||
// Subscription is the flat monthly cost (for mode=subscription).
|
||||
Subscription float64 `yaml:"subscription,omitempty" json:"subscription,omitempty"`
|
||||
// Models declares per-token prices when mode=token.
|
||||
Models map[string]BillingToken `yaml:"models,omitempty" json:"models,omitempty"`
|
||||
// Free is true when mode=free (kept for explicitness).
|
||||
// Peak applies to ALL token-priced requests under this URL.
|
||||
Peak *BillingPeak `yaml:"peak,omitempty" json:"peak,omitempty"`
|
||||
}
|
||||
|
||||
// BillingToken is the per-model price.
|
||||
type BillingToken struct {
|
||||
Prompt string `yaml:"prompt" json:"prompt"` // USD per million tokens, e.g. "0.15"
|
||||
Completion string `yaml:"completion" json:"completion"` // USD per million tokens
|
||||
CacheDiscount *float64 `yaml:"cache_discount,omitempty" json:"cache_discount,omitempty"`
|
||||
}
|
||||
|
||||
// BillingPeak declares peak-hour pricing.
|
||||
type BillingPeak struct {
|
||||
Multiplier float64 `yaml:"multiplier" json:"multiplier"`
|
||||
// Weekdays: 1=Mon..5=Fri (Lua os.date !%w, 0=Sun)
|
||||
Weekdays []int `yaml:"weekdays,omitempty" json:"weekdays,omitempty"`
|
||||
// Hours: each pair is [lo,hi] inclusive
|
||||
Hours [][2]int `yaml:"hours,omitempty" json:"hours,omitempty"`
|
||||
}
|
||||
|
||||
// Validate rejects a BillingDSL that could not work as written.
|
||||
func (b *BillingDSL) Validate() error {
|
||||
if b == nil {
|
||||
return nil
|
||||
}
|
||||
ids := map[string]bool{}
|
||||
for i := range b.Profiles {
|
||||
p := &b.Profiles[i]
|
||||
if p.ID == "" {
|
||||
return fmt.Errorf("billing profile [%d] has no id", i)
|
||||
}
|
||||
if ids[p.ID] {
|
||||
return fmt.Errorf("billing profile id %q appears more than once", p.ID)
|
||||
}
|
||||
ids[p.ID] = true
|
||||
if p.Default != "" && p.Default != "free" && p.Default != "unpriced" {
|
||||
return fmt.Errorf("profile %q: default must be free or unpriced, got %q", p.ID, p.Default)
|
||||
}
|
||||
for j := range p.Rules {
|
||||
r := &p.Rules[j]
|
||||
if r.URL == "" {
|
||||
return fmt.Errorf("profile %q rule [%d]: url is required", p.ID, j)
|
||||
}
|
||||
switch r.Mode {
|
||||
case "free", "token", "subscription", "unpriced":
|
||||
default:
|
||||
return fmt.Errorf("profile %q rule [%d]: mode must be free/token/subscription/unpriced, got %q", p.ID, j, r.Mode)
|
||||
}
|
||||
if r.Mode == "token" {
|
||||
for m, t := range r.Models {
|
||||
if t.Prompt == "" || t.Completion == "" {
|
||||
return fmt.Errorf("profile %q model %q: prompt and completion are required for token mode", p.ID, m)
|
||||
}
|
||||
if _, err := ParseFloatUSDPerM(t.Prompt); err != nil {
|
||||
return fmt.Errorf("profile %q model %q prompt: %w", p.ID, m, err)
|
||||
}
|
||||
if _, err := ParseFloatUSDPerM(t.Completion); err != nil {
|
||||
return fmt.Errorf("profile %q model %q completion: %w", p.ID, m, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
// Active must resolve to a profile id.
|
||||
if b.Active != "" && !ids[b.Active] {
|
||||
return fmt.Errorf("billing.active = %q but no profile has that id", b.Active)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Resolve returns the profile with the given id; empty id (or an id that does
|
||||
// not exist) falls back to the FIRST profile. A config with profiles but no
|
||||
// active marker should mean "use the one I wrote first" rather than "price
|
||||
// nothing", which is what a nil here would silently do.
|
||||
func (b *BillingDSL) Resolve(id string) *BillingProfile {
|
||||
if b == nil || len(b.Profiles) == 0 {
|
||||
return nil
|
||||
}
|
||||
if id != "" {
|
||||
for i := range b.Profiles {
|
||||
if b.Profiles[i].ID == id {
|
||||
return &b.Profiles[i]
|
||||
}
|
||||
}
|
||||
}
|
||||
return &b.Profiles[0]
|
||||
}
|
||||
|
||||
// parseFloatUSDPerM parses a USD-per-million price string. Returns the
|
||||
// per-single-token rate the plugin expects (price / 1e6).
|
||||
//
|
||||
// Kept as a string in the DSL rather than a float: "0.15" survives round-trips
|
||||
// through YAML editors, and a malformed value must fail VALIDATION (with the
|
||||
// file and field named) rather than parse as 0 and under-bill silently — which
|
||||
// is exactly the failure mode that motivated the DSL.
|
||||
func ParseFloatUSDPerM(s string) (float64, error) {
|
||||
f, err := strconv.ParseFloat(strings.TrimSpace(s), 64)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("not a number: %q", s)
|
||||
}
|
||||
if f < 0 {
|
||||
return 0, fmt.Errorf("negative price: %q", s)
|
||||
}
|
||||
return f / 1e6, nil
|
||||
}
|
||||
|
||||
// KeyQuota is retained only to carry a scope entry's caps through the admin
|
||||
// API. Quotas are per model, never per key: there is deliberately no key-wide
|
||||
// total, so exhausting one model's budget never blocks the others.
|
||||
type KeyQuota struct {
|
||||
TokenQuota int64 `json:"token_quota"`
|
||||
ReqQuota int64 `json:"req_quota"`
|
||||
Period string `json:"period"`
|
||||
Hours int64 `json:"hours"`
|
||||
}
|
||||
|
||||
// NormalizeRole defaults an empty role to "user", so a key can never end up in
|
||||
// a state where no role means "neither admin nor user".
|
||||
func NormalizeRole(role string) string {
|
||||
if role == "admin" {
|
||||
return "admin"
|
||||
}
|
||||
return "user"
|
||||
}
|
||||
|
||||
// Validate rejects a quota configuration that could not work as written. A
|
||||
// period is only meaningful when at least one cap is set, and a cap of zero
|
||||
// means "unlimited" rather than "deny everything", so those are the only two
|
||||
// things worth rejecting.
|
||||
func (q KeyQuota) Validate() error {
|
||||
if q.TokenQuota < 0 {
|
||||
return fmt.Errorf("token_quota must be >= 0 (0 = unlimited)")
|
||||
}
|
||||
if q.ReqQuota < 0 {
|
||||
return fmt.Errorf("req_quota must be >= 0 (0 = unlimited)")
|
||||
}
|
||||
if q.Hours < 0 {
|
||||
return fmt.Errorf("hours must be >= 0")
|
||||
}
|
||||
if q.TokenQuota > 0 || q.ReqQuota > 0 {
|
||||
if err := ValidatePeriod(q.Period, q.Hours); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ValidatePeriod accepts the quota period vocabulary: "" (never resets),
|
||||
// "hour", "week", "month", or "nhour" with hours >= 1. An unknown period is
|
||||
// rejected rather than silently treated as "never resets", which would turn a
|
||||
// typo into an all-time quota — the opposite of what the operator typed.
|
||||
func ValidatePeriod(period string, hours int64) error {
|
||||
switch period {
|
||||
case "", "hour", "week", "month":
|
||||
return nil
|
||||
case "nhour":
|
||||
if hours < 1 {
|
||||
return fmt.Errorf("period %q needs hours >= 1", period)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("period must be one of \"\", hour, week, month, nhour (got %q)", period)
|
||||
}
|
||||
|
||||
// ModelScope is one allowed model for a key, or one AUTO scheduling slot.
|
||||
// Its TokenQuota and ReqQuota cap THAT entry only, independently of every
|
||||
// other entry on the same key: a model that runs out of budget stops being
|
||||
// served while the key's other models keep working. TokenQuota 0 / ReqQuota 0
|
||||
// = unlimited. Period "" = never resets; "hour"/"week"/"month" are fixed
|
||||
// windows; "nhour" uses Hours.
|
||||
// ModelScope is one allowed model for a key, or one AUTO scheduling slot,
|
||||
// with an optional token quota and reset period. TokenQuota 0 = unlimited;
|
||||
// Period "" = never resets; "hour"/"week"/"month" are fixed windows; "nhour"
|
||||
// uses Hours as the window length in hours.
|
||||
type ModelScope struct {
|
||||
Model string `yaml:"model" json:"model"`
|
||||
Source string `yaml:"source,omitempty" json:"source,omitempty"` // optional: pin to one upstream source; "" = any source
|
||||
Tier int `yaml:"tier,omitempty" json:"tier,omitempty"`
|
||||
TokenQuota int64 `yaml:"token_quota" json:"token_quota"`
|
||||
ReqQuota int64 `yaml:"req_quota,omitempty" json:"req_quota,omitempty"`
|
||||
Period string `yaml:"period,omitempty" json:"period,omitempty"`
|
||||
Hours int64 `yaml:"hours,omitempty" json:"hours,omitempty"`
|
||||
}
|
||||
@ -629,7 +388,6 @@ func (m *ModelScope) UnmarshalJSON(b []byte) error {
|
||||
Source string `json:"source"`
|
||||
Tier int `json:"tier"`
|
||||
TokenQuota int64 `json:"token_quota"`
|
||||
ReqQuota int64 `json:"req_quota"`
|
||||
Period string `json:"period"`
|
||||
Hours int64 `json:"hours"`
|
||||
}
|
||||
@ -640,7 +398,6 @@ func (m *ModelScope) UnmarshalJSON(b []byte) error {
|
||||
m.Source = o.Source
|
||||
m.Tier = o.Tier
|
||||
m.TokenQuota = o.TokenQuota
|
||||
m.ReqQuota = o.ReqQuota
|
||||
m.Period = o.Period
|
||||
m.Hours = o.Hours
|
||||
return nil
|
||||
|
||||
@ -1,317 +0,0 @@
|
||||
package config
|
||||
|
||||
// Secret handling for config.yaml.
|
||||
//
|
||||
// config.yaml is 0644 world-readable by design (ops need to inspect it), so any
|
||||
// credential in it must not sit there in plaintext. Sources' api_key / headers
|
||||
// and gateway keys are therefore sealed at rest with the same SecretBox used by
|
||||
// the runtime store, and unsealed in memory at load time.
|
||||
//
|
||||
// The invariant that makes this safe: **in-memory values are always plaintext**,
|
||||
// and the enc:v1: prefix is what marks a file value as sealed. Read paths that
|
||||
// predate this (core.resolveSourceKey) already unseal, so only the write side and
|
||||
// the load-time normalize step are new.
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"log"
|
||||
"os"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// sealSource seals one source's credentials in place (used by the YAML upsert
|
||||
// path, which is a package function and therefore has no Config to borrow a box
|
||||
// from).
|
||||
func sealSource(s *Source, box *SecretBox) error {
|
||||
if box == nil {
|
||||
return nil
|
||||
}
|
||||
if s.APIKey != "" && !strings.HasPrefix(s.APIKey, encPrefix) {
|
||||
v, err := box.Encrypt(s.APIKey)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
s.APIKey = v
|
||||
}
|
||||
for k, v := range s.Headers {
|
||||
if v == "" || strings.HasPrefix(v, encPrefix) {
|
||||
continue
|
||||
}
|
||||
e, err := box.Encrypt(v)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
s.Headers[k] = e
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// normalizeSecrets unseals every credential in the freshly parsed config so the
|
||||
// rest of the program only ever sees plaintext. A value without the enc:v1:
|
||||
// prefix is left untouched, which keeps hand-written plaintext configs working
|
||||
// (and is what a pre-encryption config file looks like).
|
||||
//
|
||||
// A value that carries the prefix but fails to decrypt is a hard error, not
|
||||
// something to paper over: returning the ciphertext (MustDecrypt's behavior)
|
||||
// would let the next Save re-seal it and turn one bad value into permanent,
|
||||
// compounding corruption. Losing the master key must be loud.
|
||||
func (c *Config) normalizeSecrets(box *SecretBox) error {
|
||||
if box == nil {
|
||||
return nil
|
||||
}
|
||||
for i := range c.Sources {
|
||||
s := &c.Sources[i]
|
||||
if strings.HasPrefix(s.APIKey, encPrefix) {
|
||||
v, err := box.Decrypt(s.APIKey)
|
||||
if err != nil {
|
||||
return fmt.Errorf("source %q api_key: %w", s.Name, err)
|
||||
}
|
||||
s.APIKey = v
|
||||
}
|
||||
for k, v := range s.Headers {
|
||||
if strings.HasPrefix(v, encPrefix) {
|
||||
d, err := box.Decrypt(v)
|
||||
if err != nil {
|
||||
return fmt.Errorf("source %q header %q: %w", s.Name, k, err)
|
||||
}
|
||||
s.Headers[k] = d
|
||||
}
|
||||
}
|
||||
}
|
||||
for i := range c.Keys {
|
||||
if strings.HasPrefix(c.Keys[i].Key, encPrefix) {
|
||||
v, err := box.Decrypt(c.Keys[i].Key)
|
||||
if err != nil {
|
||||
return fmt.Errorf("gateway key %q: %w", c.Keys[i].Name, err)
|
||||
}
|
||||
c.Keys[i].Key = v
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// sealInPlace replaces plaintext credentials with ciphertext for writing. It is
|
||||
// deliberately a separate step from Marshal: callers that need the plaintext
|
||||
// (auth comparisons, log output, returning a key to the operator who just
|
||||
// created it) must not be handed a sealed config by accident.
|
||||
func (c *Config) sealInPlace(box *SecretBox) error {
|
||||
if box == nil {
|
||||
return nil
|
||||
}
|
||||
for i := range c.Sources {
|
||||
s := &c.Sources[i]
|
||||
if s.APIKey != "" && !strings.HasPrefix(s.APIKey, encPrefix) {
|
||||
v, err := box.Encrypt(s.APIKey)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
s.APIKey = v
|
||||
}
|
||||
if len(s.Headers) > 0 {
|
||||
sealed := make(map[string]string, len(s.Headers))
|
||||
for k, v := range s.Headers {
|
||||
if v == "" || strings.HasPrefix(v, encPrefix) {
|
||||
sealed[k] = v
|
||||
continue
|
||||
}
|
||||
e, err := box.Encrypt(v)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
sealed[k] = e
|
||||
}
|
||||
s.Headers = sealed
|
||||
}
|
||||
}
|
||||
for i := range c.Keys {
|
||||
k := &c.Keys[i]
|
||||
if k.Key != "" && !strings.HasPrefix(k.Key, encPrefix) {
|
||||
v, err := box.Encrypt(k.Key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
k.Key = v
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// unsealAfterWrite restores plaintext after a sealed marshal so the live process
|
||||
// keeps working on plaintext values (mirrors Store.persistLocked's dance).
|
||||
func (c *Config) unsealAfterWrite(box *SecretBox) {
|
||||
// Save just encrypted every value it can see, so a failure here is
|
||||
// impossible; ignore the error rather than panic in a write path.
|
||||
_ = c.normalizeSecrets(box)
|
||||
}
|
||||
|
||||
// hasPlaintextSecrets reports whether any credential in the config is still in
|
||||
// the clear. Used to decide whether a startup migration write is needed, and to
|
||||
// warn (without leaking values) when no master key is available.
|
||||
func (c *Config) hasPlaintextSecrets() bool {
|
||||
for _, s := range c.Sources {
|
||||
if s.APIKey != "" && !strings.HasPrefix(s.APIKey, encPrefix) {
|
||||
return true
|
||||
}
|
||||
for _, v := range s.Headers {
|
||||
if v != "" && !strings.HasPrefix(v, encPrefix) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, k := range c.Keys {
|
||||
if k.Key != "" && !strings.HasPrefix(k.Key, encPrefix) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// countPlaintextSecrets returns how many credentials are still in the clear, for
|
||||
// an operator-facing migration log line that must not print the values.
|
||||
func (c *Config) countPlaintextSecrets() int {
|
||||
n := 0
|
||||
for _, s := range c.Sources {
|
||||
if s.APIKey != "" && !strings.HasPrefix(s.APIKey, encPrefix) {
|
||||
n++
|
||||
}
|
||||
for _, v := range s.Headers {
|
||||
if v != "" && !strings.HasPrefix(v, encPrefix) {
|
||||
n++
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, k := range c.Keys {
|
||||
if k.Key != "" && !strings.HasPrefix(k.Key, encPrefix) {
|
||||
n++
|
||||
}
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// NormalizeSecretsForRun unseals the loaded config and then seals it back on
|
||||
// disk if anything was still in the clear. Order matters: Load() read the file
|
||||
// with ciphertext still in place, so the unseal has to happen before the
|
||||
// registry (and any Save the startup path performs) sees the values.
|
||||
func (c *Config) NormalizeSecretsForRun(box *SecretBox) error {
|
||||
hadPlaintext, err := c.UnsealSecrets(box)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.SealIfNeeded(hadPlaintext)
|
||||
}
|
||||
|
||||
// UnsealSecrets decrypts every sealed credential in memory and reports whether
|
||||
// the config ON DISK still held plaintext (i.e. whether a sealing write is
|
||||
// needed). It never writes; a failed decrypt (wrong master key) is returned so
|
||||
// the process refuses to start instead of running with unusable credentials.
|
||||
//
|
||||
// The return value must be computed BEFORE unsealing and from the disk state,
|
||||
// not from memory: after a Save the in-memory values are always plaintext, so a
|
||||
// "is anything plaintext?" test run afterwards is unconditionally true and a
|
||||
// caller would rewrite the file on every start. That was the actual behaviour -
|
||||
// migratePlaintextSecrets() claimed to be idempotent in a comment but rewrote
|
||||
// config.yaml on every boot.
|
||||
//
|
||||
// Callers that consume credentials (the provider registry, key seeding) must
|
||||
// unseal FIRST. Seeding compares cfg.Keys[i].Key against the plaintext
|
||||
// gateway_keys entries; running it while keys are still ciphertext made the
|
||||
// dedupe never match, so every restart appended another copy of the same admin
|
||||
// key (production accumulated four).
|
||||
func (c *Config) UnsealSecrets(box *SecretBox) (bool, error) {
|
||||
if box == nil {
|
||||
return false, nil
|
||||
}
|
||||
c.AttachSecretBox(box)
|
||||
hadPlaintext := c.hasPlaintextSecrets()
|
||||
if err := c.normalizeSecrets(box); err != nil {
|
||||
return hadPlaintext, err
|
||||
}
|
||||
return hadPlaintext, nil
|
||||
}
|
||||
|
||||
// SealIfNeeded writes the config back once if hadPlaintext reported that the
|
||||
// file still held clear-text credentials. When it is false the file is left
|
||||
// untouched, which is what makes startup a no-op for an already-sealed config.
|
||||
func (c *Config) SealIfNeeded(hadPlaintext bool) error {
|
||||
if !hadPlaintext || c.box == nil || c.Path == "" {
|
||||
return nil
|
||||
}
|
||||
n := c.countPlaintextSecrets()
|
||||
if err := c.Save(); err != nil {
|
||||
return err
|
||||
}
|
||||
log.Printf("[config] sealed %d plaintext credential(s) in %s", n, c.Path)
|
||||
return nil
|
||||
}
|
||||
|
||||
// AttachSecretBox wires the encryption box into the config so Save can seal
|
||||
// credentials. Kept as an explicit call (rather than a constructor argument) so
|
||||
// config.Load stays usable in contexts that have no filesystem secrets (tests,
|
||||
// `-check`).
|
||||
func (c *Config) AttachSecretBox(box *SecretBox) { c.box = box }
|
||||
|
||||
// SecretBox returns the wired encryption box, or nil when none is attached.
|
||||
func (c *Config) SecretBox() *SecretBox { return c.box }
|
||||
|
||||
// migratePlaintextSecrets seals any credential still in the clear and writes the
|
||||
// file once. Kept for callers that attach the box themselves; it decides from
|
||||
// the in-memory state, which is why UnsealSecrets + SealIfNeeded (which decide
|
||||
// from the on-disk state) are preferred on the startup path.
|
||||
func (c *Config) migratePlaintextSecrets() error {
|
||||
if c.box == nil || c.Path == "" {
|
||||
return nil
|
||||
}
|
||||
if !c.hasPlaintextSecrets() {
|
||||
return nil
|
||||
}
|
||||
n := c.countPlaintextSecrets()
|
||||
if err := c.Save(); err != nil {
|
||||
return err
|
||||
}
|
||||
log.Printf("[config] sealed %d plaintext credential(s) in %s", n, c.Path)
|
||||
return nil
|
||||
}
|
||||
|
||||
// PrintSecrets writes the config's credentials to stdout in the clear and
|
||||
// returns an error only when the file cannot be read or the master key does not
|
||||
// match. It is the operator counterpart to sealing-at-rest: with keys stored as
|
||||
// ciphertext, "which key is this source using" must still be answerable.
|
||||
//
|
||||
// It deliberately takes a path rather than a *Config so the caller cannot
|
||||
// accidentally hand it a config that has already been unsealed in memory, and it
|
||||
// never writes anything.
|
||||
func PrintSecrets(path string) error {
|
||||
cfg, err := Load(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
box, err := NewSecretBox(cfg.RuntimeFile)
|
||||
if err != nil {
|
||||
return fmt.Errorf("%w (is master.key present and intact?)", err)
|
||||
}
|
||||
if err := cfg.normalizeSecrets(box); err != nil {
|
||||
return err
|
||||
}
|
||||
w := os.Stdout
|
||||
fmt.Fprintf(w, "# %s — %d source(s), %d gateway key(s)\n", path, len(cfg.Sources), len(cfg.Keys))
|
||||
for _, s := range cfg.Sources {
|
||||
fmt.Fprintf(w, "source %-16s api_key=%s\n", s.Name, orNone(s.APIKey))
|
||||
for k, v := range s.Headers {
|
||||
if v != "" {
|
||||
fmt.Fprintf(w, "source %-16s header[%s]=%s\n", s.Name, k, v)
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, k := range cfg.Keys {
|
||||
fmt.Fprintf(w, "key %-16s role=%-5s %s\n", k.Name, k.Role, k.Key)
|
||||
}
|
||||
fmt.Fprintf(w, "gateway_keys (legacy): %s\n", strings.Join(cfg.GatewayKeys, " "))
|
||||
return nil
|
||||
}
|
||||
|
||||
func orNone(s string) string {
|
||||
if s == "" {
|
||||
return "(unset)"
|
||||
}
|
||||
return s
|
||||
}
|
||||
@ -1,323 +0,0 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// writeMasterKey plants a deterministic master.key next to the given runtime
|
||||
// file so tests exercise the real box instead of the env-var shortcut.
|
||||
func writeMasterKey(t *testing.T, runtimeFile string) *SecretBox {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(filepath.Join(filepath.Dir(runtimeFile), "master.key"),
|
||||
[]byte(strings.Repeat("ab", 32)), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
box, err := NewSecretBox(runtimeFile)
|
||||
if err != nil {
|
||||
t.Fatalf("NewSecretBox: %v", err)
|
||||
}
|
||||
return box
|
||||
}
|
||||
|
||||
func TestSaveSealsCredentialsAndLoadUnseals(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "config.yaml")
|
||||
runtime := filepath.Join(dir, "runtime.json")
|
||||
box := writeMasterKey(t, runtime)
|
||||
|
||||
cfg := &Config{
|
||||
Path: path,
|
||||
Listen: "127.0.0.1:0",
|
||||
Sources: []Source{{
|
||||
Name: "up",
|
||||
BaseURL: "http://up/v1",
|
||||
APIKey: "sk-plaintext-secret",
|
||||
Adapter: "openai",
|
||||
Headers: map[string]string{"X-Extra": "header-secret"},
|
||||
Models: []Model{{ID: "m", Kind: "chat"}},
|
||||
}},
|
||||
Keys: []GWKey{{Key: "sk-gw-plain", Role: "admin", Name: "admin"}},
|
||||
}
|
||||
cfg.AttachSecretBox(box)
|
||||
if err := cfg.Save(); err != nil {
|
||||
t.Fatalf("Save: %v", err)
|
||||
}
|
||||
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
text := string(raw)
|
||||
if strings.Contains(text, "sk-plaintext-secret") {
|
||||
t.Error("source api_key written in plaintext")
|
||||
}
|
||||
if strings.Contains(text, "header-secret") {
|
||||
t.Error("source header written in plaintext")
|
||||
}
|
||||
if strings.Contains(text, "sk-gw-plain") {
|
||||
t.Error("gateway key written in plaintext")
|
||||
}
|
||||
if n := strings.Count(text, encPrefix); n != 3 {
|
||||
t.Errorf("expected 3 sealed values, found %d", n)
|
||||
}
|
||||
|
||||
// The live in-memory config must still hold plaintext after Save.
|
||||
if cfg.Sources[0].APIKey != "sk-plaintext-secret" {
|
||||
t.Errorf("in-memory api_key mutated by Save: %q", cfg.Sources[0].APIKey)
|
||||
}
|
||||
if cfg.Keys[0].Key != "sk-gw-plain" {
|
||||
t.Errorf("in-memory gateway key mutated by Save: %q", cfg.Keys[0].Key)
|
||||
}
|
||||
|
||||
// A fresh load must hand back plaintext again.
|
||||
loaded, err := Load(path)
|
||||
if err != nil {
|
||||
t.Fatalf("Load: %v", err)
|
||||
}
|
||||
loaded.AttachSecretBox(box)
|
||||
loaded.normalizeSecrets(box)
|
||||
if loaded.Sources[0].APIKey != "sk-plaintext-secret" {
|
||||
t.Errorf("loaded api_key = %q, want plaintext", loaded.Sources[0].APIKey)
|
||||
}
|
||||
if loaded.Sources[0].Headers["X-Extra"] != "header-secret" {
|
||||
t.Errorf("loaded header = %q, want plaintext", loaded.Sources[0].Headers["X-Extra"])
|
||||
}
|
||||
if loaded.Keys[0].Key != "sk-gw-plain" {
|
||||
t.Errorf("loaded gateway key = %q, want plaintext", loaded.Keys[0].Key)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPlaintextConfigStillLoads(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "config.yaml")
|
||||
// A hand-written, pre-encryption config: no enc:v1: anywhere.
|
||||
body := `listen: 127.0.0.1:0
|
||||
sources:
|
||||
- name: legacy
|
||||
base_url: http://legacy/v1
|
||||
api_key: sk-written-by-hand
|
||||
adapter: openai
|
||||
models:
|
||||
- id: m
|
||||
kind: chat
|
||||
`
|
||||
if err := os.WriteFile(path, []byte(body), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, err := Load(path)
|
||||
if err != nil {
|
||||
t.Fatalf("Load: %v", err)
|
||||
}
|
||||
box := writeMasterKey(t, filepath.Join(dir, "runtime.json"))
|
||||
cfg.AttachSecretBox(box)
|
||||
cfg.normalizeSecrets(box)
|
||||
if cfg.Sources[0].APIKey != "sk-written-by-hand" {
|
||||
t.Errorf("plaintext config value changed: %q", cfg.Sources[0].APIKey)
|
||||
}
|
||||
}
|
||||
|
||||
// Sealing a config is a one-time migration: the FIRST run writes, every later
|
||||
// start must leave the file alone.
|
||||
//
|
||||
// This test previously called migratePlaintextSecrets() twice and compared
|
||||
// ModTime. That assertion was both weak and wrong:
|
||||
//
|
||||
// - migratePlaintextSecrets decides from IN-MEMORY state, and Save() ends by
|
||||
// unsealing memory so the running process keeps working. So every call sees
|
||||
// "plaintext present" and rewrites the file. Comparing ModTime hid this
|
||||
// because two writes inside one filesystem timestamp tick look identical —
|
||||
// the test flaked (~1 in 6) instead of failing.
|
||||
// - it also exercised a function production no longer calls on the startup
|
||||
// path, which now uses UnsealSecrets + SealIfNeeded (they decide from the
|
||||
// ON-DISK state).
|
||||
//
|
||||
// So the test targets that real path and compares file CONTENT, which cannot be
|
||||
// fooled by timestamp granularity.
|
||||
func TestSealingIsIdempotentAcrossStarts(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "config.yaml")
|
||||
runtime := filepath.Join(dir, "runtime.json")
|
||||
box := writeMasterKey(t, runtime)
|
||||
|
||||
cfg := &Config{
|
||||
Path: path,
|
||||
Listen: "127.0.0.1:0",
|
||||
Sources: []Source{{Name: "up", BaseURL: "http://up/v1", APIKey: "sk-clear", Adapter: "openai", Models: []Model{{ID: "m"}}}},
|
||||
}
|
||||
cfg.AttachSecretBox(box)
|
||||
|
||||
hadPlaintext, err := cfg.UnsealSecrets(box)
|
||||
if err != nil {
|
||||
t.Fatalf("unseal: %v", err)
|
||||
}
|
||||
if !hadPlaintext {
|
||||
t.Fatal("a config written with a clear-text credential must report plaintext")
|
||||
}
|
||||
if err := cfg.SealIfNeeded(hadPlaintext); err != nil {
|
||||
t.Fatalf("seal: %v", err)
|
||||
}
|
||||
first, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(first), encPrefix) {
|
||||
t.Fatal("sealing did not encrypt the value on disk")
|
||||
}
|
||||
// In-memory must be plaintext so the running process keeps working.
|
||||
if cfg.Sources[0].APIKey != "sk-clear" {
|
||||
t.Errorf("in-memory api_key = %q, want plaintext", cfg.Sources[0].APIKey)
|
||||
}
|
||||
|
||||
// Simulate the next start: load from disk, unseal, seal-if-needed. The file
|
||||
// was already sealed, so nothing may be written.
|
||||
cfg2, err := Load(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg2.AttachSecretBox(box)
|
||||
hadPlaintext2, err := cfg2.UnsealSecrets(box)
|
||||
if err != nil {
|
||||
t.Fatalf("second unseal: %v", err)
|
||||
}
|
||||
if hadPlaintext2 {
|
||||
t.Error("a sealed config reported plaintext — SealIfNeeded would rewrite " +
|
||||
"the file on every start")
|
||||
}
|
||||
if err := cfg2.SealIfNeeded(hadPlaintext2); err != nil {
|
||||
t.Fatalf("second seal: %v", err)
|
||||
}
|
||||
second, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(first) != string(second) {
|
||||
t.Error("the second start rewrote an already-sealed config; startup must be a no-op")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSealedValueDoesNotDoubleEncrypt(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
box := writeMasterKey(t, filepath.Join(dir, "runtime.json"))
|
||||
cfg := &Config{Path: filepath.Join(dir, "config.yaml"), Sources: []Source{{Name: "a", BaseURL: "http://a"}}}
|
||||
cfg.AttachSecretBox(box)
|
||||
|
||||
once, err := box.Encrypt("sk-x")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg.Sources[0].APIKey = once
|
||||
if err := cfg.Save(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := box.Decrypt(cfg.Sources[0].APIKey)
|
||||
if err != nil {
|
||||
t.Fatalf("value became undecryptable: %v", err)
|
||||
}
|
||||
if got != "sk-x" {
|
||||
t.Errorf("round trip = %q, want sk-x", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpsertSourceInYAMLSealsCredentials(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "config.yaml")
|
||||
box := writeMasterKey(t, filepath.Join(dir, "runtime.json"))
|
||||
|
||||
if err := os.WriteFile(path, []byte("listen: 127.0.0.1:0\nsources: []\n"), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
src := Source{Name: "new", BaseURL: "http://new/v1", APIKey: "sk-fresh", Adapter: "openai", Models: []Model{{ID: "m"}}}
|
||||
if err := UpsertSourceInYAML(path, src.Name, src, box); err != nil {
|
||||
t.Fatalf("UpsertSourceInYAML: %v", err)
|
||||
}
|
||||
raw, _ := os.ReadFile(path)
|
||||
if strings.Contains(string(raw), "sk-fresh") {
|
||||
t.Error("newly added source stored its api_key in plaintext")
|
||||
}
|
||||
if !strings.Contains(string(raw), encPrefix) {
|
||||
t.Error("newly added source was not sealed")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCountPlaintextSecrets(t *testing.T) {
|
||||
cfg := &Config{
|
||||
Sources: []Source{
|
||||
{Name: "a", APIKey: encPrefix + "abc", Headers: map[string]string{"H": encPrefix + "x"}},
|
||||
{Name: "b", APIKey: "sk-in-clear", Headers: map[string]string{"H2": "clear-too"}},
|
||||
},
|
||||
Keys: []GWKey{{Key: "sk-gw-clear"}},
|
||||
}
|
||||
if got := cfg.countPlaintextSecrets(); got != 3 {
|
||||
t.Errorf("countPlaintextSecrets = %d, want 3", got)
|
||||
}
|
||||
if !cfg.hasPlaintextSecrets() {
|
||||
t.Error("hasPlaintextSecrets = false, want true")
|
||||
}
|
||||
cfg.Sources[1].APIKey = encPrefix + "y"
|
||||
cfg.Sources[1].Headers["H2"] = encPrefix + "z"
|
||||
cfg.Keys[0].Key = encPrefix + "k"
|
||||
if cfg.hasPlaintextSecrets() {
|
||||
t.Error("hasPlaintextSecrets = true after sealing everything")
|
||||
}
|
||||
}
|
||||
|
||||
// TestNormalizeSecretsFailsLoudlyOnWrongMasterKey is the negative case that
|
||||
// guards the migration: with a wrong key, a sealed value must NOT be handed
|
||||
// back as ciphertext for a later re-seal (that compounds corruption and turns
|
||||
// one lost key file into permanently unusable config).
|
||||
func TestNormalizeSecretsFailsLoudlyOnWrongMasterKey(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "config.yaml")
|
||||
runtime := filepath.Join(dir, "runtime.json")
|
||||
good := writeMasterKey(t, runtime)
|
||||
|
||||
cfg := &Config{
|
||||
Path: path,
|
||||
Listen: "127.0.0.1:0",
|
||||
Sources: []Source{{Name: "up", BaseURL: "http://up/v1", APIKey: "sk-secret", Adapter: "openai", Models: []Model{{ID: "m"}}}},
|
||||
}
|
||||
cfg.AttachSecretBox(good)
|
||||
if err := cfg.Save(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sealedOnce, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// A different master key must be rejected, loudly. NewSecretBox derives the
|
||||
// key path from the RUNTIME file's directory, so the wrong key has to live
|
||||
// in a different directory — otherwise it would read the good master.key.
|
||||
otherDir := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(otherDir, "master.key"),
|
||||
[]byte(strings.Repeat("cd", 32)), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
badBox, err := NewSecretBox(filepath.Join(otherDir, "runtime.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
loaded, err := Load(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
loaded.AttachSecretBox(badBox)
|
||||
if err := loaded.NormalizeSecretsForRun(badBox); err == nil {
|
||||
t.Fatal("expected an error with the wrong master key, got nil")
|
||||
}
|
||||
|
||||
// Crucially: the file must be untouched — no second seal on top.
|
||||
after, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(after) != string(sealedOnce) {
|
||||
t.Error("config file was rewritten despite the decrypt failure")
|
||||
}
|
||||
if n := strings.Count(string(after), encPrefix); n != strings.Count(string(sealedOnce), encPrefix) {
|
||||
t.Errorf("sealed count changed %d -> %d (double encryption)", strings.Count(string(sealedOnce), encPrefix), n)
|
||||
}
|
||||
}
|
||||
@ -7,7 +7,6 @@ import (
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
@ -16,7 +15,6 @@ import (
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/billing"
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/lua"
|
||||
"llmsproxy/internal/provider"
|
||||
@ -31,18 +29,11 @@ type Core struct {
|
||||
mu sync.Mutex
|
||||
cfg *config.Config
|
||||
vm *lua.VM
|
||||
plugins *lua.Plugins
|
||||
store *config.Store
|
||||
scheduler *scheduler.Scheduler
|
||||
registry *provider.Registry
|
||||
autoChain atomic.Pointer[scheduler.Chain] // chat AUTO chain
|
||||
autoImageChain atomic.Pointer[scheduler.Chain] // image-generation AUTO chain
|
||||
|
||||
// keyAutoChains caches per-key AUTO chains, keyed by the gateway key.
|
||||
// Entries are rebuilt by rebuildKeyAutoChains whenever sources or key
|
||||
// config change; AutoChainFor reads them without taking c.mu so request
|
||||
// handling stays lock-free like the global chain lookup.
|
||||
keyAutoChains atomic.Pointer[map[string]*scheduler.Chain]
|
||||
}
|
||||
|
||||
// New builds the core from a config file plus runtime overlay.
|
||||
@ -61,51 +52,7 @@ func NewFromConfig(cfg *config.Config) (*Core, error) {
|
||||
if err := c.vm.Start(); err != nil {
|
||||
return nil, fmt.Errorf("lua vm: %w", err)
|
||||
}
|
||||
// Plugins load AFTER the VM is up (a plugin is just another Lua state) but
|
||||
// BEFORE anything can serve traffic, so a plugin's request_end hook sees
|
||||
// the very first request. A plugin that fails to load is logged and skipped
|
||||
// rather than failing startup: plugins are optional extensions, and failing
|
||||
// to boot the gateway because a third-party .lua has a typo would be the
|
||||
// wrong trade.
|
||||
c.plugins = lua.NewPlugins(c.vm, cfg.PluginDir)
|
||||
if err := c.plugins.SeedBundled(); err != nil {
|
||||
log.Printf("[core] seed bundled plugins: %v", err)
|
||||
}
|
||||
if err := c.plugins.LoadDir(); err != nil {
|
||||
log.Printf("[core] plugin dir: %v", err)
|
||||
}
|
||||
for _, p := range c.plugins.List() {
|
||||
if msg, bad := p["error"]; bad {
|
||||
log.Printf("[core] plugin %v failed to load: %v", p["name"], msg)
|
||||
continue
|
||||
}
|
||||
log.Printf("[core] plugin %v %v loaded (hooks=%v ui=%v)",
|
||||
p["name"], p["version"], p["hooks"], p["ui"])
|
||||
}
|
||||
c.applyBillingDSL()
|
||||
c.store = config.NewStore(cfg.RuntimeFile)
|
||||
// Share one box between the runtime store and config.yaml so a single
|
||||
// master.key seals both files. config.Load left the config holding
|
||||
// ciphertext (if it was sealed); unseal it NOW, before anything reads a
|
||||
// credential.
|
||||
//
|
||||
// ORDER IS LOAD-BEARING. These steps each consume secrets and must run
|
||||
// after the unseal:
|
||||
// - seedKeys compares cfg.Keys[i].Key against the plaintext
|
||||
// gateway_keys entries; with ciphertext keys the comparison never
|
||||
// matched and every restart appended another duplicate admin key
|
||||
// (production had four copies of the same admin key).
|
||||
// - rebuildRegistry hands cfg.Sources[i].APIKey to the providers; with
|
||||
// ciphertext it built every upstream client with "enc:v1:..." as its
|
||||
// bearer token.
|
||||
// Previously the unseal happened at the END of this function, and things
|
||||
// only appeared to work because seedKeys' Save() unsealed memory as a side
|
||||
// effect. Removing the redundant saves exposed the real ordering bug.
|
||||
cfg.AttachSecretBox(c.store.SecretBox())
|
||||
hadPlaintextSecrets, err := cfg.UnsealSecrets(c.store.SecretBox())
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("unseal secrets: %w", err)
|
||||
}
|
||||
if err := c.store.Load(); err != nil {
|
||||
return nil, fmt.Errorf("runtime store: %w", err)
|
||||
}
|
||||
@ -127,12 +74,6 @@ func NewFromConfig(cfg *config.Config) (*Core, error) {
|
||||
if err := c.seedPresetTemplates(); err != nil {
|
||||
return nil, fmt.Errorf("seed preset templates: %w", err)
|
||||
}
|
||||
// Seal any credential still in the clear in config.yaml. Startup writes
|
||||
// nothing when the file was already sealed (hadPlaintextSecrets is decided
|
||||
// from the on-disk state, before the unseal above).
|
||||
if err := cfg.SealIfNeeded(hadPlaintextSecrets); err != nil {
|
||||
return nil, fmt.Errorf("normalize secrets: %w", err)
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
@ -251,11 +192,6 @@ func buildRetries(cfg *config.Config) int {
|
||||
// VM exposes the Lua adapter runtime.
|
||||
func (c *Core) VM() *lua.VM { return c.vm }
|
||||
|
||||
// Plugins exposes the loaded plugin set. Never nil once New* has returned, so
|
||||
// the gateway can fire stages unconditionally; an unconfigured plugin
|
||||
// directory yields an empty registry whose Fire is a no-op.
|
||||
func (c *Core) Plugins() *lua.Plugins { return c.plugins }
|
||||
|
||||
func (c *Core) Scheduler() *scheduler.Scheduler { return c.scheduler }
|
||||
|
||||
func (c *Core) Registry() *provider.Registry { return c.registry }
|
||||
@ -263,68 +199,6 @@ func (c *Core) Registry() *provider.Registry { return c.registry }
|
||||
// AutoChain returns the current AUTO scheduling chain.
|
||||
func (c *Core) AutoChain() *scheduler.Chain { return c.autoChain.Load() }
|
||||
|
||||
// AutoChainFor returns the AUTO chain to use for a request authenticated with
|
||||
// key. A key that declares its own chain gets that chain; every other key
|
||||
// inherits the global chain, so keys created before per-key chains existed
|
||||
// keep their current behaviour with no configuration change.
|
||||
//
|
||||
// It returns (chain, ok). ok is false only when the key has its own chain but
|
||||
// it compiled to nothing usable (e.g. every slot names a model that no longer
|
||||
// exists) — the caller must surface that rather than silently downgrading the
|
||||
// user to the global chain, which would be a confusing, invisible switch.
|
||||
func (c *Core) AutoChainFor(key string) (*scheduler.Chain, bool) {
|
||||
if m := c.keyAutoChains.Load(); m != nil {
|
||||
if ch, ok := (*m)[key]; ok {
|
||||
return ch, true
|
||||
}
|
||||
}
|
||||
return c.autoChain.Load(), true
|
||||
}
|
||||
|
||||
// rebuildKeyAutoChains recompiles every per-key AUTO chain from the current
|
||||
// config and provider registry, then swaps the cache in one shot so requests
|
||||
// never observe a half-built map. Keys without their own chain are absent
|
||||
// from the map and fall back to the global chain.
|
||||
func (c *Core) rebuildKeyAutoChains() {
|
||||
m := make(map[string]*scheduler.Chain)
|
||||
for _, k := range c.cfg.Keys {
|
||||
if !k.HasOwnAuto() {
|
||||
continue
|
||||
}
|
||||
m[k.Key] = c.chainForRules(k.Auto, false)
|
||||
}
|
||||
c.keyAutoChains.Store(&m)
|
||||
}
|
||||
|
||||
// SaveKeyAuto persists one key's own AUTO chain and recompiles the per-key
|
||||
// chain cache. Passing an empty list clears the override so the key inherits
|
||||
// the global chain again.
|
||||
func (c *Core) SaveKeyAuto(key string, entries []config.ModelScope) error {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
idx := -1
|
||||
for i, k := range c.cfg.Keys {
|
||||
if k.Key == key {
|
||||
idx = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if idx < 0 {
|
||||
return fmt.Errorf("key not found")
|
||||
}
|
||||
if err := ValidateScopeQuotas(entries); err != nil {
|
||||
return err
|
||||
}
|
||||
c.cfg.Keys[idx].Auto = cleanScopes(entries)
|
||||
if err := c.saveConfig(); err != nil {
|
||||
return err
|
||||
}
|
||||
// Rebuild the whole cache so a cleared override stops shadowing the
|
||||
// global chain for this key.
|
||||
c.rebuildKeyAutoChains()
|
||||
return nil
|
||||
}
|
||||
|
||||
// AutoImageChain returns the persisted image-generation AUTO chain snapshot.
|
||||
func (c *Core) AutoImageChain() *scheduler.Chain { return c.autoImageChain.Load() }
|
||||
|
||||
@ -364,15 +238,10 @@ func (c *Core) FindKey(key string) (config.GWKey, bool) {
|
||||
}
|
||||
|
||||
// CreateKey builds a new random gateway key and persists it to config.yaml.
|
||||
// Quotas live on the model scope entries, so a new key's budget is whatever
|
||||
// its scopes carry.
|
||||
func (c *Core) CreateKey(name, role string, models []config.ModelScope, note string) (config.GWKey, error) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
models = cleanScopes(models)
|
||||
if err := ValidateScopeQuotas(models); err != nil {
|
||||
return config.GWKey{}, err
|
||||
}
|
||||
key := make([]byte, 16)
|
||||
if _, err := rand.Read(key); err != nil {
|
||||
return config.GWKey{}, err
|
||||
@ -385,7 +254,9 @@ func (c *Core) CreateKey(name, role string, models []config.ModelScope, note str
|
||||
Note: note,
|
||||
CreatedAt: time.Now().Unix(),
|
||||
}
|
||||
rec.Role = config.NormalizeRole(rec.Role)
|
||||
if rec.Role == "" {
|
||||
rec.Role = "user"
|
||||
}
|
||||
c.cfg.Keys = append(c.cfg.Keys, rec)
|
||||
if err := c.saveConfig(); err != nil {
|
||||
return config.GWKey{}, err
|
||||
@ -393,16 +264,10 @@ func (c *Core) CreateKey(name, role string, models []config.ModelScope, note str
|
||||
return rec, nil
|
||||
}
|
||||
|
||||
// UpdateKey mutates a key's name/role/model scope and persists it. The scope
|
||||
// entries carry their own quotas, so replacing the scope replaces the budgets.
|
||||
// UpdateKey mutates a key's name/role/model scope and persists it.
|
||||
func (c *Core) UpdateKey(key, name, role string, models []config.ModelScope, note string) (config.GWKey, error) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
if models != nil {
|
||||
if err := ValidateScopeQuotas(models); err != nil {
|
||||
return config.GWKey{}, err
|
||||
}
|
||||
}
|
||||
for i, k := range c.cfg.Keys {
|
||||
if k.Key == key {
|
||||
if name != "" {
|
||||
@ -574,11 +439,6 @@ func (c *Core) mergedSources() []config.Source {
|
||||
}
|
||||
|
||||
// resolveSourceKey applies api_key_env and decrypts enc:v1: ciphertext.
|
||||
// Config values are already unsealed at startup (Config.NormalizeSecretsForRun),
|
||||
// so the decrypt branch is the belt-and-braces path for a source that arrived
|
||||
// already sealed through another route. A failure there leaves the ciphertext in
|
||||
// place, which makes the upstream reject the key loudly rather than sending an
|
||||
// empty Authorization header that might look like a config-less source.
|
||||
func (c *Core) resolveSourceKey(s config.Source) config.Source {
|
||||
if s.APIKeyEnv != "" {
|
||||
if v := os.Getenv(s.APIKeyEnv); v != "" {
|
||||
@ -587,11 +447,7 @@ func (c *Core) resolveSourceKey(s config.Source) config.Source {
|
||||
return s
|
||||
}
|
||||
if box := c.store.SecretBox(); box != nil && strings.HasPrefix(s.APIKey, "enc:v1:") {
|
||||
if v, err := box.Decrypt(s.APIKey); err == nil {
|
||||
s.APIKey = v
|
||||
} else {
|
||||
log.Printf("[core] source %s: api_key decrypt failed: %v", s.Name, err)
|
||||
}
|
||||
s.APIKey = box.MustDecrypt(s.APIKey)
|
||||
}
|
||||
return s
|
||||
}
|
||||
@ -614,7 +470,6 @@ func (c *Core) rebuildRegistry() error {
|
||||
}
|
||||
c.buildAutoChain()
|
||||
c.buildAutoImageChain()
|
||||
c.rebuildKeyAutoChains()
|
||||
return nil
|
||||
}
|
||||
|
||||
@ -624,28 +479,17 @@ func (c *Core) rebuildRegistry() error {
|
||||
// back to the source's best chat model and cooldown/quota bookkeeping would
|
||||
// key on a name that never matches.
|
||||
func (c *Core) buildAutoChain() {
|
||||
c.autoChain.Store(c.chainForRules(c.cfg.Auto, false))
|
||||
}
|
||||
|
||||
// chainForRules compiles a chat AUTO chain from a rule list. kindFilter is
|
||||
// false for the chat chain (image models are skipped) and true for the image
|
||||
// chain (only image models participate). Per-key chains always compile as
|
||||
// chat chains: a user's key configures the models *they* talk to.
|
||||
func (c *Core) chainForRules(rules []config.ModelScope, imageOnly bool) *scheduler.Chain {
|
||||
prov := func(model, source string) scheduler.Provider {
|
||||
p := c.registry.ProviderForSlot(model, source)
|
||||
if p == nil {
|
||||
return nil
|
||||
}
|
||||
m := p.ModelByID(model)
|
||||
if imageOnly && (m == nil || m.Kind != "image") {
|
||||
return nil
|
||||
}
|
||||
if !imageOnly && m != nil && m.Kind == "image" {
|
||||
if m := p.ModelByID(model); m != nil && m.Kind == "image" {
|
||||
return nil
|
||||
}
|
||||
return p
|
||||
}
|
||||
rules := c.cfg.Auto
|
||||
sr := make([]scheduler.Rule, 0, len(rules))
|
||||
for _, e := range rules {
|
||||
model, source := e.Model, e.Source
|
||||
@ -654,11 +498,7 @@ func (c *Core) chainForRules(rules []config.ModelScope, imageOnly bool) *schedul
|
||||
model = exact
|
||||
}
|
||||
if m := p.ModelByID(model); m != nil && m.Kind == "image" {
|
||||
if !imageOnly {
|
||||
continue // image-kind models never join the chat AUTO chain
|
||||
}
|
||||
} else if imageOnly {
|
||||
continue
|
||||
continue // image-kind models never join the chat AUTO chain
|
||||
}
|
||||
if source == "" {
|
||||
source = p.Name()
|
||||
@ -673,7 +513,7 @@ func (c *Core) chainForRules(rules []config.ModelScope, imageOnly bool) *schedul
|
||||
Hours: e.Hours,
|
||||
})
|
||||
}
|
||||
return scheduler.BuildChain(sr, prov)
|
||||
c.autoChain.Store(scheduler.BuildChain(sr, prov))
|
||||
}
|
||||
|
||||
// buildAutoImageChain rebuilds the image-generation AUTO chain snapshot.
|
||||
@ -815,7 +655,7 @@ func (c *Core) AddSource(src config.Source) error {
|
||||
if err := normalizeSource(&src); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := config.UpsertSourceInYAML(c.cfg.Path, src.Name, src, c.cfg.SecretBox()); err != nil {
|
||||
if err := config.UpsertSourceInYAML(c.cfg.Path, src.Name, src); err != nil {
|
||||
return err
|
||||
}
|
||||
// Update in-memory Sources so mergedSources() finds the entry.
|
||||
@ -912,88 +752,7 @@ func normalizeSource(s *config.Source) error {
|
||||
|
||||
// Close releases resources.
|
||||
func (c *Core) Close() {
|
||||
// Plugin state must be flushed BEFORE the VM stops. The saver's final write
|
||||
// reads each plugin's Lua tables; once vm.Stop() has closed those states the
|
||||
// read finds nothing and the last interval of accumulation is lost — which
|
||||
// is the exact failure this persistence was added to prevent.
|
||||
if c.plugins != nil {
|
||||
c.plugins.Close()
|
||||
}
|
||||
if c.vm != nil {
|
||||
c.vm.Stop()
|
||||
}
|
||||
}
|
||||
|
||||
// applyBillingDSL compiles the configured billing profile and injects it into
|
||||
// the billing plugin.
|
||||
//
|
||||
// Configured pricing replaces the hand-written JSON blob that used to be PUT
|
||||
// through the state API. That path was used twice in production and both times
|
||||
// it "succeeded" while being wrong: once without the required `prices` wrapper
|
||||
// (silently wiping the accumulated totals), once with a peak-window shape the
|
||||
// plugin does not read (peak traffic billed at off-peak rates, no error). A
|
||||
// typed config with load-time validation turns both mistakes into startup
|
||||
// errors naming the file and field.
|
||||
//
|
||||
// A DSL with no profile, or one that fails to compile, leaves the plugin's
|
||||
// prices untouched: billing keeps running on whatever it had, and the problem
|
||||
// is logged rather than becoming a startup failure — the gateway must forward
|
||||
// even when its cost accounting is misconfigured.
|
||||
func (c *Core) applyBillingDSL() {
|
||||
ps := c.plugins
|
||||
if ps == nil || ps.Count() == 0 {
|
||||
return
|
||||
}
|
||||
// The billing plugin may simply not be installed; a DSL without it is a
|
||||
// no-op, not an error.
|
||||
var have bool
|
||||
for _, row := range ps.List() {
|
||||
if row["name"] == "billing" && row["loaded"] == true {
|
||||
have = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !have {
|
||||
return
|
||||
}
|
||||
dsl := c.cfg.BillingDSL
|
||||
if dsl == nil || len(dsl.Profiles) == 0 {
|
||||
return
|
||||
}
|
||||
if err := dsl.Validate(); err != nil {
|
||||
log.Printf("[billing] config invalid, profile not applied: %v", err)
|
||||
return
|
||||
}
|
||||
profile := dsl.Resolve(dsl.Active)
|
||||
if profile == nil {
|
||||
log.Printf("[billing] no billing profile resolved (active=%q)", dsl.Active)
|
||||
return
|
||||
}
|
||||
prices, err := billing.Compile(profile, c.cfg.Sources)
|
||||
if err != nil {
|
||||
log.Printf("[billing] profile %q not applied: %v", profile.ID, err)
|
||||
return
|
||||
}
|
||||
if err := ps.SetState("billing", map[string]interface{}{"prices": prices}); err != nil {
|
||||
log.Printf("[billing] applying profile %q: %v", profile.ID, err)
|
||||
return
|
||||
}
|
||||
log.Printf("[billing] profile %q applied (%d rules, %d sources)", profile.ID, len(profile.Rules), len(c.cfg.Sources))
|
||||
}
|
||||
|
||||
// ValidateScopeQuotas checks every scope entry's caps before they are stored.
|
||||
// A typo in a period must be rejected at write time rather than silently
|
||||
// becoming a never-resetting budget — the opposite of what was typed.
|
||||
func ValidateScopeQuotas(entries []config.ModelScope) error {
|
||||
for _, e := range entries {
|
||||
if err := (config.KeyQuota{
|
||||
TokenQuota: e.TokenQuota,
|
||||
ReqQuota: e.ReqQuota,
|
||||
Period: e.Period,
|
||||
Hours: e.Hours,
|
||||
}).Validate(); err != nil {
|
||||
return fmt.Errorf("model %q: %w", e.Model, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@ -1,189 +0,0 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// AutoChainFor sits on the request hot path and reads keyAutoChains with no
|
||||
// lock, while SaveKeyAuto / rebuildRegistry replace that map from the admin
|
||||
// path. A "call it twice" test never interleaves those, so it would pass even
|
||||
// if the map were mutated in place. These tests run both sides at once under
|
||||
// -race and assert what readers actually observe.
|
||||
//
|
||||
// The property: a reader sees a chain that is either the key's own or the
|
||||
// global one — never a torn or empty one. If rebuildKeyAutoChains built the
|
||||
// map in place instead of swapping a fresh pointer, readers would sometimes
|
||||
// see a partially-filled map and silently fall back to the global chain, i.e.
|
||||
// a user quietly losing their configured chain under load.
|
||||
|
||||
// concurrentChainConfig builds N keys, half with their own chain pointing at a
|
||||
// distinct model so a fallback to the global chain is distinguishable.
|
||||
func concurrentChainConfig(t *testing.T, n int) *config.Config {
|
||||
t.Helper()
|
||||
cfg := perKeyTestConfig(t)
|
||||
cfg.Keys = nil
|
||||
cfg.GatewayKeys = []string{"sk-gw-admin"}
|
||||
cfg.Auto = []config.ModelScope{{Model: "gpt-4o", Source: "s1", Tier: 1}}
|
||||
for i := 0; i < n; i++ {
|
||||
k := config.GWKey{
|
||||
Key: fmt.Sprintf("sk-gw-k%03d", i),
|
||||
Role: "user",
|
||||
Name: fmt.Sprintf("user%d", i),
|
||||
}
|
||||
if i%2 == 0 {
|
||||
k.Auto = []config.ModelScope{{Model: "gpt-4o-mini", Source: "s1", Tier: 1}}
|
||||
}
|
||||
cfg.Keys = append(cfg.Keys, k)
|
||||
}
|
||||
return cfg
|
||||
}
|
||||
|
||||
func TestAutoChainForConcurrentWithSaveKeyAuto(t *testing.T) {
|
||||
cfg := concurrentChainConfig(t, 40)
|
||||
c := newTestCore(t, cfg)
|
||||
|
||||
var stop atomic.Bool
|
||||
var wg sync.WaitGroup
|
||||
var bad atomic.Int64
|
||||
|
||||
for r := 0; r < 8; r++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
for !stop.Load() {
|
||||
for i := 0; i < 40; i++ {
|
||||
key := fmt.Sprintf("sk-gw-k%03d", i)
|
||||
ch, ok := c.AutoChainFor(key)
|
||||
if !ok || ch == nil || len(ch.Tiers) == 0 {
|
||||
bad.Add(1)
|
||||
continue
|
||||
}
|
||||
// Legal values are this key's own chain model (either
|
||||
// side of the writer's flip) or the global chain's. The
|
||||
// assertion is about the legal set, not a fixed value:
|
||||
// the writers legitimately change the chain underneath.
|
||||
got := ch.Tiers[0].Slots[0].Model
|
||||
if got != "gpt-4o" && got != "gpt-4o-mini" {
|
||||
bad.Add(1)
|
||||
}
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
for w := 0; w < 2; w++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
model := "gpt-4o-mini"
|
||||
for i := 0; i < 25 && !stop.Load(); i++ {
|
||||
for j := 0; j < 40; j += 2 {
|
||||
if err := c.SaveKeyAuto(fmt.Sprintf("sk-gw-k%03d", j), []config.ModelScope{
|
||||
{Model: model, Source: "s1", Tier: 1},
|
||||
}); err != nil {
|
||||
bad.Add(1)
|
||||
return
|
||||
}
|
||||
}
|
||||
if model == "gpt-4o-mini" {
|
||||
model = "gpt-4o"
|
||||
} else {
|
||||
model = "gpt-4o-mini"
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
time.Sleep(400 * time.Millisecond)
|
||||
stop.Store(true)
|
||||
wg.Wait()
|
||||
|
||||
if n := bad.Load(); n > 0 {
|
||||
t.Fatalf("%d reads saw a chain outside {own, global} — the cache is "+
|
||||
"being swapped in place instead of atomically", n)
|
||||
}
|
||||
}
|
||||
|
||||
// The registry rebuild path (AddSource / RemoveSource) also replaces the
|
||||
// cache, so readers must keep a usable chain throughout.
|
||||
//
|
||||
// Churned sources use a "churn" prefix on purpose: deleting a source a chain
|
||||
// references legitimately empties that chain (BuildChain drops unresolvable
|
||||
// slots), and asserting otherwise would be testing a fiction rather than the
|
||||
// code.
|
||||
func TestAutoChainForConcurrentWithSourceChanges(t *testing.T) {
|
||||
cfg := concurrentChainConfig(t, 20)
|
||||
c := newTestCore(t, cfg)
|
||||
|
||||
var stop atomic.Bool
|
||||
var wg sync.WaitGroup
|
||||
var bad atomic.Int64
|
||||
|
||||
for r := 0; r < 6; r++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
for !stop.Load() {
|
||||
for i := 0; i < 20; i++ {
|
||||
ch, ok := c.AutoChainFor(fmt.Sprintf("sk-gw-k%03d", i))
|
||||
if !ok || ch == nil || len(ch.Tiers) == 0 {
|
||||
bad.Add(1)
|
||||
continue
|
||||
}
|
||||
if got := ch.Tiers[0].Slots[0].Model; got != "gpt-4o" && got != "gpt-4o-mini" {
|
||||
bad.Add(1)
|
||||
}
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
for n := 0; n < 30 && !stop.Load(); n++ {
|
||||
name := fmt.Sprintf("churn%d", n)
|
||||
_ = c.AddSource(config.Source{
|
||||
Name: name, BaseURL: "http://127.0.0.1:9/v1", Adapter: "openai",
|
||||
Models: []config.Model{{ID: "m-" + name, Priority: 10}}},
|
||||
)
|
||||
_ = c.RemoveSource(name)
|
||||
}
|
||||
}()
|
||||
|
||||
time.Sleep(500 * time.Millisecond)
|
||||
stop.Store(true)
|
||||
wg.Wait()
|
||||
|
||||
if n := bad.Load(); n > 0 {
|
||||
t.Fatalf("%d reads lost the chain while unrelated sources were churned", n)
|
||||
}
|
||||
}
|
||||
|
||||
// Many distinct keys: the lookup must stay a map hit and must not degrade into
|
||||
// returning the global chain as the key count grows.
|
||||
func TestAutoChainForManyKeys(t *testing.T) {
|
||||
cfg := concurrentChainConfig(t, 500)
|
||||
c := newTestCore(t, cfg)
|
||||
|
||||
for i := 0; i < 500; i++ {
|
||||
key := fmt.Sprintf("sk-gw-k%03d", i)
|
||||
ch, _ := c.AutoChainFor(key)
|
||||
if ch == nil || len(ch.Tiers) == 0 {
|
||||
t.Fatalf("key %s lost its chain with 500 keys configured", key)
|
||||
}
|
||||
want := "gpt-4o"
|
||||
if i%2 == 0 {
|
||||
want = "gpt-4o-mini"
|
||||
}
|
||||
if got := ch.Tiers[0].Slots[0].Model; got != want {
|
||||
t.Fatalf("key %s got %q, want %q", key, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,221 +0,0 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/scheduler"
|
||||
)
|
||||
|
||||
// Per-key AUTO chains let an admin give one user their own scheduling chain
|
||||
// while every other key keeps the global one. These tests pin the three
|
||||
// properties that make that safe:
|
||||
//
|
||||
// 1. a key WITH its own chain gets it;
|
||||
// 2. a key WITHOUT one inherits the global chain (backward compatible);
|
||||
// 3. the override is actually persisted, so it survives a restart.
|
||||
//
|
||||
// A test that only checked (1) would pass even if AutoChainFor ignored the key
|
||||
// and always returned the global chain whenever the two happened to be equal,
|
||||
// so (2) uses deliberately *different* chains.
|
||||
|
||||
func perKeyTestConfig(t *testing.T) *config.Config {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
return &config.Config{
|
||||
Path: filepath.Join(dir, "config.yaml"),
|
||||
AdapterDir: filepath.Join(dir, "adapters"),
|
||||
RuntimeFile: filepath.Join(dir, "runtime.json"),
|
||||
Listen: "127.0.0.1:0",
|
||||
DefaultModel: "AUTO",
|
||||
GatewayKeys: []string{"sk-gw-admin"},
|
||||
Sources: []config.Source{
|
||||
{Name: "s1", BaseURL: "http://127.0.0.1:1/v1", Adapter: "openai",
|
||||
Models: []config.Model{{ID: "gpt-4o", Priority: 10}, {ID: "gpt-4o-mini", Priority: 5}}},
|
||||
},
|
||||
// Global chain points at gpt-4o.
|
||||
Auto: []config.ModelScope{{Model: "gpt-4o", Source: "s1", Tier: 1}},
|
||||
Keys: []config.GWKey{
|
||||
{Key: "sk-gw-admin", Role: "admin"},
|
||||
{Key: "sk-gw-inherits", Role: "user", Name: "inherits"},
|
||||
// This key's own chain points at a DIFFERENT model, so returning
|
||||
// the global chain by mistake is detectable.
|
||||
{Key: "sk-gw-own", Role: "user", Name: "own",
|
||||
Auto: []config.ModelScope{{Model: "gpt-4o-mini", Source: "s1", Tier: 1}}},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func firstSlotModel(t *testing.T, c *Core, key string) string {
|
||||
t.Helper()
|
||||
ch, _ := c.AutoChainFor(key)
|
||||
if ch == nil || len(ch.Tiers) == 0 {
|
||||
t.Fatalf("key %s: no chain tiers", key)
|
||||
}
|
||||
slots := ch.Tiers[0].Slots
|
||||
if len(slots) == 0 {
|
||||
t.Fatalf("key %s: tier 1 has no slots", key)
|
||||
}
|
||||
return slots[0].Model
|
||||
}
|
||||
|
||||
// A key that declares its own chain must be scheduled by it, not the global.
|
||||
func TestAutoChainForUsesKeyOwnChain(t *testing.T) {
|
||||
c := newTestCore(t, perKeyTestConfig(t))
|
||||
|
||||
if got := firstSlotModel(t, c, "sk-gw-own"); got != "gpt-4o-mini" {
|
||||
t.Fatalf("key with own chain must use it, got %q (want gpt-4o-mini)", got)
|
||||
}
|
||||
// Sanity: the global chain really is different, so the assertion above
|
||||
// cannot pass by accident.
|
||||
if got := firstSlotModel(t, c, "sk-gw-admin"); got != "gpt-4o" {
|
||||
t.Fatalf("admin must use global chain, got %q (want gpt-4o)", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A key with no own chain inherits the global one. This is what keeps every
|
||||
// pre-existing deployment working with no config change.
|
||||
func TestAutoChainForInheritsGlobalWhenUnset(t *testing.T) {
|
||||
c := newTestCore(t, perKeyTestConfig(t))
|
||||
|
||||
if got := firstSlotModel(t, c, "sk-gw-inherits"); got != "gpt-4o" {
|
||||
t.Fatalf("key without own chain must inherit global, got %q (want gpt-4o)", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A key that is not in the key table at all (e.g. a seed key) must also
|
||||
// inherit the global chain rather than erroring or returning nil.
|
||||
func TestAutoChainForUnknownKeyInheritsGlobal(t *testing.T) {
|
||||
c := newTestCore(t, perKeyTestConfig(t))
|
||||
|
||||
ch, ok := c.AutoChainFor("sk-gw-never-configured")
|
||||
if !ok || ch == nil || len(ch.Tiers) == 0 {
|
||||
t.Fatalf("unknown key must fall back to global chain, got ok=%v chain=%v", ok, ch)
|
||||
}
|
||||
}
|
||||
|
||||
// The per-key chain must survive a save/reload round trip: an admin configuring
|
||||
// a chain in the WebUI and restarting the gateway must not silently lose it.
|
||||
func TestSaveKeyAutoPersistsAndApplies(t *testing.T) {
|
||||
cfg := perKeyTestConfig(t)
|
||||
c := newTestCore(t, cfg)
|
||||
|
||||
if err := c.SaveKeyAuto("sk-gw-inherits", []config.ModelScope{
|
||||
{Model: "gpt-4o-mini", Source: "s1", Tier: 1},
|
||||
}); err != nil {
|
||||
t.Fatalf("SaveKeyAuto: %v", err)
|
||||
}
|
||||
if got := firstSlotModel(t, c, "sk-gw-inherits"); got != "gpt-4o-mini" {
|
||||
t.Fatalf("saved chain must apply immediately, got %q", got)
|
||||
}
|
||||
|
||||
// Reload from disk exactly like a restart does.
|
||||
reloaded, err := config.Load(cfg.Path)
|
||||
if err != nil {
|
||||
t.Fatalf("reload config: %v", err)
|
||||
}
|
||||
c2 := newTestCore(t, reloaded)
|
||||
if got := firstSlotModel(t, c2, "sk-gw-inherits"); got != "gpt-4o-mini" {
|
||||
t.Fatalf("per-key chain must survive restart, got %q", got)
|
||||
}
|
||||
// The other key must be unaffected by its neighbour's override.
|
||||
if got := firstSlotModel(t, c2, "sk-gw-admin"); got != "gpt-4o" {
|
||||
t.Fatalf("override must not leak to other keys, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// Clearing the override must return the key to the global chain, both in
|
||||
// memory and after a restart. This is what the WebUI's "use global" toggle
|
||||
// does, so a stale shadowing entry would silently pin the user to their old
|
||||
// chain forever.
|
||||
func TestSaveKeyAutoClearRestoresGlobal(t *testing.T) {
|
||||
cfg := perKeyTestConfig(t)
|
||||
c := newTestCore(t, cfg)
|
||||
|
||||
if err := c.SaveKeyAuto("sk-gw-own", nil); err != nil {
|
||||
t.Fatalf("clear: %v", err)
|
||||
}
|
||||
if got := firstSlotModel(t, c, "sk-gw-own"); got != "gpt-4o" {
|
||||
t.Fatalf("cleared key must inherit global again, got %q", got)
|
||||
}
|
||||
|
||||
reloaded, err := config.Load(cfg.Path)
|
||||
if err != nil {
|
||||
t.Fatalf("reload: %v", err)
|
||||
}
|
||||
c2 := newTestCore(t, reloaded)
|
||||
if got := firstSlotModel(t, c2, "sk-gw-own"); got != "gpt-4o" {
|
||||
t.Fatalf("clear must survive restart, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// Saving an unknown key must fail loudly rather than creating a shadow entry
|
||||
// that no request can ever match.
|
||||
func TestSaveKeyAutoUnknownKeyFails(t *testing.T) {
|
||||
c := newTestCore(t, perKeyTestConfig(t))
|
||||
|
||||
if err := c.SaveKeyAuto("sk-gw-nope", []config.ModelScope{{Model: "gpt-4o", Source: "s1"}}); err == nil {
|
||||
t.Fatal("SaveKeyAuto on unknown key must return an error")
|
||||
}
|
||||
}
|
||||
|
||||
// Per-key quota validation must run, exactly like the model scope path: an
|
||||
// override is another place an operator can write a bad budget.
|
||||
func TestSaveKeyAutoRejectsBadQuota(t *testing.T) {
|
||||
c := newTestCore(t, perKeyTestConfig(t))
|
||||
|
||||
err := c.SaveKeyAuto("sk-gw-inherits", []config.ModelScope{
|
||||
{Model: "gpt-4o", Source: "s1", TokenQuota: -5},
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("negative token quota in a per-key auto chain must be rejected")
|
||||
}
|
||||
}
|
||||
|
||||
// Source changes must recompile per-key chains. A key whose chain names a
|
||||
// model that does not exist yet compiles to an empty chain (BuildChain drops
|
||||
// slots it cannot resolve — sending the request into a black hole would be
|
||||
// worse), and must start working as soon as the source appears.
|
||||
func TestKeyAutoChainsRebuildOnSourceChange(t *testing.T) {
|
||||
cfg := perKeyTestConfig(t)
|
||||
cfg.Keys = append(cfg.Keys, config.GWKey{
|
||||
Key: "sk-gw-late", Role: "user", Name: "late",
|
||||
Auto: []config.ModelScope{{Model: "new-model", Source: "s2", Tier: 1}},
|
||||
})
|
||||
c := newTestCore(t, cfg)
|
||||
|
||||
// s2 does not exist yet, so the chain has no usable slot.
|
||||
if ch, _ := c.AutoChainFor("sk-gw-late"); chainSlotsOf(ch) != 0 {
|
||||
t.Fatalf("chain naming an unknown model must compile empty, got %d slots",
|
||||
chainSlotsOf(ch))
|
||||
}
|
||||
// The other keys are unaffected by the broken one.
|
||||
if got := firstSlotModel(t, c, "sk-gw-admin"); got != "gpt-4o" {
|
||||
t.Fatalf("broken per-key chain must not affect others, got %q", got)
|
||||
}
|
||||
|
||||
// Add the source; the per-key chain must pick it up without a restart.
|
||||
cfg.Sources = append(cfg.Sources, config.Source{
|
||||
Name: "s2", BaseURL: "http://127.0.0.1:2/v1", Adapter: "openai",
|
||||
Models: []config.Model{{ID: "new-model", Priority: 10}},
|
||||
})
|
||||
if err := c.Reload(); err != nil {
|
||||
t.Fatalf("Reload: %v", err)
|
||||
}
|
||||
if got := firstSlotModel(t, c, "sk-gw-late"); got != "new-model" {
|
||||
t.Fatalf("per-key chain must resolve after the source is added, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// chainSlotsOf counts usable slots, tolerating a nil chain.
|
||||
func chainSlotsOf(ch *scheduler.Chain) int {
|
||||
if ch == nil {
|
||||
return 0
|
||||
}
|
||||
n := 0
|
||||
for _, tn := range ch.Tiers {
|
||||
n += len(tn.Slots)
|
||||
}
|
||||
return n
|
||||
}
|
||||
@ -1,178 +0,0 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// Startup must be idempotent, and every credential consumer must see PLAINTEXT.
|
||||
//
|
||||
// Two bugs shared one root cause: NewFromConfig unsealed the config at the END,
|
||||
// after the steps that read credentials had already run.
|
||||
//
|
||||
// 1. seedKeys() deduped cfg.Keys[i].Key against the plaintext gateway_keys
|
||||
// entries. With ciphertext keys the comparison never matched, so every
|
||||
// restart appended another copy of the same admin key — production reached
|
||||
// four identical admin keys.
|
||||
// 2. rebuildRegistry() handed cfg.Sources[i].APIKey to provider.New, so with
|
||||
// ciphertext it built every upstream client with "enc:v1:..." as its bearer
|
||||
// token and every upstream call would 401.
|
||||
//
|
||||
// They masked each other: seedKeys' Save() unsealed memory as a side effect, so
|
||||
// bug 2 was invisible until the redundant saves were removed. These tests drive
|
||||
// the real entry point (core.New -> config.Load off disk), because building a
|
||||
// fresh in-memory config per attempt hides both bugs — that is exactly how the
|
||||
// first draft of this test failed to reproduce anything.
|
||||
|
||||
// writeSealedInstall creates a config on disk the way a real install looks
|
||||
// after its first run: credentials already sealed, gateway_keys still plaintext.
|
||||
func writeSealedInstall(t *testing.T, dir string) string {
|
||||
t.Helper()
|
||||
path := filepath.Join(dir, "config.yaml")
|
||||
cfg := &config.Config{
|
||||
Path: path,
|
||||
AdapterDir: filepath.Join(dir, "adapters"),
|
||||
RuntimeFile: filepath.Join(dir, "runtime.json"),
|
||||
Listen: "127.0.0.1:0",
|
||||
DefaultModel: "AUTO",
|
||||
GatewayKeys: []string{"sk-gw-seed-me"},
|
||||
Sources: []config.Source{{
|
||||
Name: "up", BaseURL: "http://127.0.0.1:1/v1", APIKey: "sk-upstream-secret",
|
||||
Adapter: "openai", Models: []config.Model{{ID: "gpt-4o", Priority: 10}},
|
||||
}},
|
||||
}
|
||||
c, err := NewFromConfig(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("initial install: %v", err)
|
||||
}
|
||||
c.Close()
|
||||
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(raw), "enc:v1:") {
|
||||
t.Fatalf("fixture is wrong: on-disk config holds no sealed credential, "+
|
||||
"so this test cannot detect ciphertext leaking into consumers:\n%s", raw)
|
||||
}
|
||||
return path
|
||||
}
|
||||
|
||||
// writeSealedInstallNoSeed is writeSealedInstall without gateway_keys, so no
|
||||
// key is ever seeded and no startup Save() can unseal the config as a side
|
||||
// effect. That isolation is what makes the ciphertext leak observable.
|
||||
func writeSealedInstallNoSeed(t *testing.T, dir string) string {
|
||||
t.Helper()
|
||||
path := filepath.Join(dir, "config.yaml")
|
||||
cfg := &config.Config{
|
||||
Path: path,
|
||||
AdapterDir: filepath.Join(dir, "adapters"),
|
||||
RuntimeFile: filepath.Join(dir, "runtime.json"),
|
||||
Listen: "127.0.0.1:0",
|
||||
DefaultModel: "AUTO",
|
||||
Sources: []config.Source{{
|
||||
Name: "up", BaseURL: "http://127.0.0.1:1/v1", APIKey: "sk-upstream-secret",
|
||||
Adapter: "openai", Models: []config.Model{{ID: "gpt-4o", Priority: 10}},
|
||||
}},
|
||||
}
|
||||
c, err := NewFromConfig(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("initial install: %v", err)
|
||||
}
|
||||
c.Close()
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(raw), "enc:v1:") {
|
||||
t.Fatalf("fixture is wrong: no sealed credential on disk:\n%s", raw)
|
||||
}
|
||||
return path
|
||||
}
|
||||
|
||||
// countKeysWithSecret loads the config, unseals it, and counts keys whose
|
||||
// plaintext equals one of the gateway_keys entries.
|
||||
func countKeysWithSecret(t *testing.T, path string) (total, matching int) {
|
||||
t.Helper()
|
||||
cfg, err := config.Load(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
box, err := config.NewSecretBox(cfg.RuntimeFile)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := cfg.UnsealSecrets(box); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := map[string]bool{}
|
||||
for _, g := range cfg.GatewayKeys {
|
||||
want[g] = true
|
||||
}
|
||||
for _, k := range cfg.Keys {
|
||||
total++
|
||||
if want[k.Key] {
|
||||
matching++
|
||||
}
|
||||
}
|
||||
return total, matching
|
||||
}
|
||||
|
||||
// Restarting must not grow the key list. Before the fix this went 2 -> 3 -> 4.
|
||||
func TestRestartDoesNotDuplicateSeededKeys(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := writeSealedInstall(t, dir)
|
||||
|
||||
if _, n := countKeysWithSecret(t, path); n != 1 {
|
||||
t.Fatalf("after install: %d keys carry the seeded secret, want 1", n)
|
||||
}
|
||||
|
||||
for i := 2; i <= 4; i++ {
|
||||
c, err := New(path) // production path: reads the file
|
||||
if err != nil {
|
||||
t.Fatalf("restart %d: %v", i, err)
|
||||
}
|
||||
c.Close()
|
||||
total, n := countKeysWithSecret(t, path)
|
||||
if n != 1 {
|
||||
t.Fatalf("restart %d: %d keys carry the seeded secret (total %d), want "+
|
||||
"exactly 1 — seedKeys is comparing ciphertext against plaintext and "+
|
||||
"appending a duplicate on every start", i, n, total)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Every provider must hold the real upstream credential, not its ciphertext.
|
||||
//
|
||||
// This deliberately runs with gateway_keys EMPTY. The buggy order was masked
|
||||
// whenever seedKeys had work to do, because its Save() unsealed memory as a
|
||||
// side effect — providers then happened to see plaintext. With no key to seed,
|
||||
// no Save runs, and the ciphertext reaches the registry directly. An earlier
|
||||
// version of this test kept gateway_keys populated and was insensitive: it
|
||||
// passed even with the bug present.
|
||||
func TestProvidersGetPlaintextCredentials(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := writeSealedInstallNoSeed(t, dir)
|
||||
|
||||
c, err := New(path)
|
||||
if err != nil {
|
||||
t.Fatalf("restart: %v", err)
|
||||
}
|
||||
defer c.Close()
|
||||
|
||||
for _, p := range c.registry.Providers() {
|
||||
key := p.Config().APIKey
|
||||
if strings.HasPrefix(key, "enc:v1:") {
|
||||
t.Errorf("provider %q holds CIPHERTEXT api_key %q — every upstream call "+
|
||||
"would 401", p.Name(), key)
|
||||
}
|
||||
if key != "sk-upstream-secret" {
|
||||
t.Errorf("provider %q api_key = %q, want the plaintext upstream secret",
|
||||
p.Name(), key)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -3,7 +3,6 @@ package gateway
|
||||
import (
|
||||
"encoding/csv"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
@ -79,177 +78,6 @@ type sourcePayload struct {
|
||||
MaxTokens int `json:"max_tokens"`
|
||||
MaxConcurrent int `json:"max_concurrent"`
|
||||
RPM int `json:"rpm"` // optional requests-per-minute cap, 0 = unlimited
|
||||
// ModelIDs is an additive-only shorthand: when set, the named models are
|
||||
// merged into an existing source instead of Models replacing the list.
|
||||
// It exists because "add one model" is the most common scripted edit and a
|
||||
// full Models list cannot be written without reading the source first.
|
||||
ModelIDs []string `json:"model_ids,omitempty"`
|
||||
|
||||
// The four fields below are POINTERS, and that is the whole point.
|
||||
//
|
||||
// POST/PUT is an upsert that rewrites the whole source, so any field the
|
||||
// payload cannot express is silently reset to its zero value. That already
|
||||
// destroyed api_key once (fixed with resolveAPIKey) and would equally
|
||||
// destroy proxy_url, api_key_env, timeout and queue_timeout.
|
||||
//
|
||||
// A plain string/duration cannot distinguish "the caller did not mention
|
||||
// this field" from "the caller asked for the empty value", and only the
|
||||
// former may inherit. So:
|
||||
//
|
||||
// nil → field absent from the request: keep the current value
|
||||
// &"x" → present: store exactly "x" (including "" to clear it)
|
||||
//
|
||||
// api_key deliberately keeps its older "empty means inherit" rule rather
|
||||
// than gaining a pointer: that rule is already published (v1.7.6) and
|
||||
// scripts depend on it. Changing it now would let a script that echoes an
|
||||
// empty api_key erase a live credential, which is the exact failure this
|
||||
// whole area exists to prevent.
|
||||
ProxyURL *string `json:"proxy_url,omitempty"`
|
||||
APIKeyEnv *string `json:"api_key_env,omitempty"`
|
||||
Timeout *string `json:"timeout,omitempty"` // duration string, e.g. "300s"
|
||||
QueueTimeout *string `json:"queue_timeout,omitempty"` // duration string, e.g. "90s"
|
||||
}
|
||||
|
||||
// optionalSourceFields carries the Source fields a partial payload must not
|
||||
// reset, each as a pointer so "absent" and "explicitly empty" stay distinct.
|
||||
//
|
||||
// nil → absent from the request: keep the current value
|
||||
// &"" → present: clear it
|
||||
// &"x" → present: store "x"
|
||||
//
|
||||
// The alternative — a plain value with "empty means inherit" — cannot express
|
||||
// "clear this field", so emptying the proxy input in the UI would silently keep
|
||||
// the old proxy. It is exactly why api_key is NOT modelled this way: for a
|
||||
// credential, failing to keep the old value is worse than failing to clear it,
|
||||
// and that rule is already published (v1.7.6). Two fields with opposite failure
|
||||
// modes therefore get opposite rules, and both are spelled out here.
|
||||
type optionalSourceFields struct {
|
||||
ProxyURL *string
|
||||
APIKeyEnv *string
|
||||
Timeout *string
|
||||
QueueTimeout *string
|
||||
}
|
||||
|
||||
// resolve overlays the four optional fields onto payload.
|
||||
//
|
||||
// A nil pointer means the request did not mention the field, so it keeps the
|
||||
// value already stored in cur (nil cur = a source being created, where the
|
||||
// payload's own zero value is correct). A non-nil pointer always wins, including
|
||||
// when it points at the empty string, which is how a UI form clears a field.
|
||||
//
|
||||
// Only these four fields are overlaid. Everything else in payload is taken as
|
||||
// sent: an upsert that inherited the whole record would make it impossible to
|
||||
// change anything.
|
||||
func (o optionalSourceFields) resolve(payload config.Source, cur *config.Source) (config.Source, error) {
|
||||
// cur is dereferenced exactly once here so the create path (nil) cannot
|
||||
// panic on the per-field lookups below.
|
||||
var storedProxyURL, storedKeyEnv string
|
||||
var storedTimeout, storedQueueTimeout time.Duration
|
||||
if cur != nil {
|
||||
storedProxyURL, storedKeyEnv = cur.ProxyURL, cur.APIKeyEnv
|
||||
storedTimeout, storedQueueTimeout = cur.Timeout, cur.QueueTimeout
|
||||
}
|
||||
pick := func(p *string, stored string) string {
|
||||
if p != nil {
|
||||
return *p
|
||||
}
|
||||
return stored
|
||||
}
|
||||
payload.ProxyURL = pick(o.ProxyURL, storedProxyURL)
|
||||
payload.APIKeyEnv = pick(o.APIKeyEnv, storedKeyEnv)
|
||||
if err := overlayDuration(&payload.Timeout, o.Timeout, "timeout", storedTimeout); err != nil {
|
||||
return payload, err
|
||||
}
|
||||
if err := overlayDuration(&payload.QueueTimeout, o.QueueTimeout, "queue_timeout", storedQueueTimeout); err != nil {
|
||||
return payload, err
|
||||
}
|
||||
return payload, nil
|
||||
}
|
||||
|
||||
// overlayDuration applies the absent/present rule to one duration field: a nil
|
||||
// pointer keeps stored, a set pointer replaces it (with "" / "0s" clearing).
|
||||
func overlayDuration(dst *time.Duration, p *string, field string, stored time.Duration) error {
|
||||
if p == nil {
|
||||
*dst = stored
|
||||
return nil
|
||||
}
|
||||
d, err := parseOptionalDuration(*p, field)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
*dst = d
|
||||
return nil
|
||||
}
|
||||
|
||||
// parseOptionalDuration parses a duration written as a Go duration string.
|
||||
// An empty string clears the field back to the config default (0), which is
|
||||
// what a UI form submitting an empty timeout box should mean. Parsing happens
|
||||
// only for fields the request actually mentions, so a typo can never surface
|
||||
// as a silent reset of something else.
|
||||
func parseOptionalDuration(v, field string) (time.Duration, error) {
|
||||
v = strings.TrimSpace(v)
|
||||
if v == "" {
|
||||
return 0, nil
|
||||
}
|
||||
d, err := time.ParseDuration(v)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("%s %q is not a duration (use e.g. 120s, 5m, 1h)", field, v)
|
||||
}
|
||||
if d < 0 {
|
||||
return 0, fmt.Errorf("%s must be >= 0 (got %s)", field, v)
|
||||
}
|
||||
return d, nil
|
||||
}
|
||||
|
||||
// keepExistingAPIKey is the mask a client sends when it means "keep the
|
||||
// current credential". POST /api/sources is an upsert, so a script that edits
|
||||
// one field and echoes a placeholder back would otherwise persist that
|
||||
// placeholder as the live api_key — which fails closed as 401 on the next
|
||||
// request, long after the script exited successfully. See TestSourceUpsertKeepsAPIKey.
|
||||
const keepExistingAPIKey = "__KEEP__"
|
||||
|
||||
// resolveAPIKey returns the credential to store, inheriting the current one
|
||||
// when the payload does not supply a new one.
|
||||
//
|
||||
// Two distinct "no new credential" signals exist and they must not collapse:
|
||||
// - the explicit mask keepExistingAPIKey — always inherit.
|
||||
// - the empty string — inherit only for a source that already has a
|
||||
// credential. Empty is a legitimate value for a *new* credential-less
|
||||
// source (dshcodebuddy uses "dsh-local-noauth", local servers use ""),
|
||||
// so a fresh source with an empty key must stay empty rather than inherit
|
||||
// nothing and fail differently.
|
||||
func (g *Gateway) resolveAPIKey(name, want string, exists bool) string {
|
||||
if want == keepExistingAPIKey {
|
||||
return g.currentAPIKey(name)
|
||||
}
|
||||
if exists && want == "" {
|
||||
return g.currentAPIKey(name)
|
||||
}
|
||||
return want
|
||||
}
|
||||
|
||||
func (g *Gateway) currentAPIKey(name string) string {
|
||||
if s := g.sourceByName(name); s != nil {
|
||||
return s.APIKey
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// sourceByName returns a live view of one source, or nil when it does not
|
||||
// exist. It goes through core.Sources() so the key is already unsealed and
|
||||
// defaults applied, matching what the routes actually use.
|
||||
func (g *Gateway) sourceByName(name string) *config.Source {
|
||||
for _, s := range g.core.Sources() {
|
||||
if s.Name == name {
|
||||
ss := s
|
||||
return &ss
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (g *Gateway) sourceExists(name string) bool {
|
||||
return g.sourceByName(name) != nil
|
||||
}
|
||||
|
||||
func (g *Gateway) handleSourcesAPI(w http.ResponseWriter, r *http.Request) {
|
||||
@ -263,34 +91,17 @@ func (g *Gateway) handleSourcesAPI(w http.ResponseWriter, r *http.Request) {
|
||||
switch r.Method {
|
||||
case http.MethodGet:
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{"sources": g.core.Sources()})
|
||||
case http.MethodPost, http.MethodPut:
|
||||
case http.MethodPost:
|
||||
var p sourcePayload
|
||||
body, _ := io.ReadAll(r.Body)
|
||||
if err := json.Unmarshal(body, &p); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
|
||||
return
|
||||
}
|
||||
// PUT addresses the source by path; the body name is only a fallback so
|
||||
// a single-field edit does not have to echo the name back. A body that
|
||||
// disagrees with the path is a caller bug, not something to silently
|
||||
// resolve — it usually means the wrong source is about to be written.
|
||||
if r.Method == http.MethodPut {
|
||||
if path == "" {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "source name required in path")
|
||||
return
|
||||
}
|
||||
if p.Name != "" && p.Name != path {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request",
|
||||
"source name in body ("+p.Name+") does not match path ("+path+")")
|
||||
return
|
||||
}
|
||||
p.Name = path
|
||||
}
|
||||
exists := g.sourceExists(p.Name)
|
||||
src := config.Source{
|
||||
Name: p.Name,
|
||||
BaseURL: p.BaseURL,
|
||||
APIKey: g.resolveAPIKey(p.Name, p.APIKey, exists),
|
||||
APIKey: p.APIKey,
|
||||
Adapter: p.Adapter,
|
||||
Endpoint: p.Endpoint,
|
||||
ImageEndpoint: p.ImageEndpoint,
|
||||
@ -302,44 +113,6 @@ func (g *Gateway) handleSourcesAPI(w http.ResponseWriter, r *http.Request) {
|
||||
MaxConcurrent: p.MaxConcurrent,
|
||||
RPM: p.RPM,
|
||||
}
|
||||
// The four upsert-fragile fields are applied last: a nil pointer means
|
||||
// "not in the request", so they are inherited from the stored source
|
||||
// instead of being reset to the payload's zero value. This runs whether
|
||||
// or not the request mentioned them, because resolve() itself decides
|
||||
// per field — passing them unconditionally keeps the rule in one place.
|
||||
src, err := optionalSourceFields{
|
||||
ProxyURL: p.ProxyURL,
|
||||
APIKeyEnv: p.APIKeyEnv,
|
||||
Timeout: p.Timeout,
|
||||
QueueTimeout: p.QueueTimeout,
|
||||
}.resolve(src, g.sourceByName(p.Name))
|
||||
if err != nil {
|
||||
writeError(w, http.StatusBadRequest, "source_error", err.Error())
|
||||
return
|
||||
}
|
||||
// model_ids is additive: "add these models" is the common scripted edit
|
||||
// and it must not require reading (and echoing) the whole list back.
|
||||
// A request that omits models entirely is therefore a pure add, not a
|
||||
// request to empty the list -- that distinction is the whole point.
|
||||
if len(p.ModelIDs) > 0 {
|
||||
if len(src.Models) == 0 {
|
||||
if existing := g.sourceByName(p.Name); existing != nil {
|
||||
src.Models = append(src.Models, existing.Models...)
|
||||
}
|
||||
}
|
||||
have := map[string]bool{}
|
||||
for _, m := range src.Models {
|
||||
have[m.ID] = true
|
||||
}
|
||||
for _, id := range p.ModelIDs {
|
||||
id = strings.TrimSpace(id)
|
||||
if id == "" || have[id] {
|
||||
continue
|
||||
}
|
||||
have[id] = true
|
||||
src.Models = append(src.Models, config.Model{ID: id, Kind: "chat"})
|
||||
}
|
||||
}
|
||||
if err := g.core.AddSource(src); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "source_error", err.Error())
|
||||
return
|
||||
@ -433,48 +206,6 @@ func (g *Gateway) handleStatsAPI(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
}
|
||||
key := exportKey(r)
|
||||
// ?period=day|week|month|all switches the whole payload to a calendar
|
||||
// window (UTC) aggregated off the audit files, instead of the
|
||||
// since-process-start totals. The CSV exports below are unaffected: they
|
||||
// take an explicit from/to range and stream, so a period selector there
|
||||
// would only be a second way to spell the same bounds.
|
||||
if p := periodFromQuery(r.URL.Query()); p != PeriodAll {
|
||||
if !ValidPeriod(p) {
|
||||
writeError(w, http.StatusBadRequest, "bad_period",
|
||||
"period must be one of day, week, month, all")
|
||||
return
|
||||
}
|
||||
out := g.stats.PeriodSnapshot(p, key, time.Now())
|
||||
// The dashboard paints its records table from this payload's `records`
|
||||
// field (paintRecords(st.records)), so a period view that omits the key
|
||||
// renders an empty table no matter how much traffic the window had. The
|
||||
// aggregate above is period-scoped; the record list is the same
|
||||
// newest-first page the scroll handler pulls from /api/stats/records,
|
||||
// which is deliberately not period-filtered (the cursor, not a date
|
||||
// range, is its state) — that asymmetry is what the paging code
|
||||
// already assumes.
|
||||
page := g.stats.AuditPage("", limit, key)
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{
|
||||
"period": out.Period,
|
||||
"from": out.From,
|
||||
"total": out.Total,
|
||||
"by_key": out.ByKey,
|
||||
"by_model": out.Models,
|
||||
"by_source": out.Srcs,
|
||||
"by_status": out.Status,
|
||||
"buckets": out.Bucket,
|
||||
"truncated": out.Truncated,
|
||||
"records": page.Records,
|
||||
// The paging cursor has to travel with the first page, otherwise
|
||||
// the scroll handler has nothing to page from and the table is
|
||||
// stuck at one screen forever.
|
||||
"next_cursor": page.Next,
|
||||
"has_more": page.HasMore,
|
||||
"rotated": page.Rotated,
|
||||
"key_names": g.keyNamesFor(),
|
||||
})
|
||||
return
|
||||
}
|
||||
if r.URL.Query().Get("export") == "csv" {
|
||||
from, _ := strconv.ParseInt(r.URL.Query().Get("from"), 10, 64)
|
||||
to, _ := strconv.ParseInt(r.URL.Query().Get("to"), 10, 64)
|
||||
@ -582,37 +313,12 @@ func (g *Gateway) handleStatsAPI(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
snap := g.stats.Snapshot(limit, key)
|
||||
snap["key_names"] = g.keyNamesFor()
|
||||
// Records come from the audit files via the same AuditPage the scroll
|
||||
// handler uses, not from Snapshot's in-memory ring. Two reasons:
|
||||
//
|
||||
// - Ordering. The ring is oldest-first, AuditPage is newest-first. The
|
||||
// first screen and the paged continuation have to agree, or scrolling
|
||||
// splices new rows in above old ones.
|
||||
// - Depth. The ring holds maxRecs entries (a few hundred); the audit
|
||||
// files hold everything, so "all" actually means all and the cursor
|
||||
// can keep paging.
|
||||
//
|
||||
// It also carries next_cursor, without which the table can never load a
|
||||
// second page.
|
||||
page := g.stats.AuditPage("", limit, key)
|
||||
snap["records"] = page.Records
|
||||
snap["next_cursor"] = page.Next
|
||||
snap["has_more"] = page.HasMore
|
||||
snap["rotated"] = page.Rotated
|
||||
writeJSON(w, http.StatusOK, snap)
|
||||
}
|
||||
|
||||
// keyNamesFor is the masked-id -> display-name map every stats payload needs.
|
||||
// It is keyed by keyID (the mask), not the raw key, because that is what the
|
||||
// aggregate rows carry — building it in one place stops the period branch and
|
||||
// the lifetime branch from drifting apart.
|
||||
func (g *Gateway) keyNamesFor() map[string]string {
|
||||
names := map[string]string{}
|
||||
keyNames := map[string]string{}
|
||||
for _, k := range g.core.ListKeys() {
|
||||
names[keyID(k.Key)] = k.Name
|
||||
keyNames[keyID(k.Key)] = k.Name
|
||||
}
|
||||
return names
|
||||
snap["key_names"] = keyNames
|
||||
writeJSON(w, http.StatusOK, snap)
|
||||
}
|
||||
|
||||
// handleStatsRecordsAPI pages the request records straight off the audit files.
|
||||
|
||||
@ -1,460 +0,0 @@
|
||||
package gateway
|
||||
|
||||
// Agent-facing management API.
|
||||
//
|
||||
// The Web UI has always driven the gateway through /api/*, so the management
|
||||
// surface already exists. What it lacked was anything an agent could *rely* on:
|
||||
// no way to discover the contract, no single call that answers "what is the
|
||||
// current state", and per-endpoint response shapes that make scripting brittle.
|
||||
//
|
||||
// This file adds a versioned, self-describing facade under /api/v1 without
|
||||
// touching the existing endpoints, so the Web UI keeps working unchanged while
|
||||
// agents get a stable contract:
|
||||
//
|
||||
// GET /api/v1 — machine-readable index of every endpoint
|
||||
// GET /api/v1/overview — one call: sources + auto chain + keys + health
|
||||
// GET /api/v1/sources — sources, credentials masked
|
||||
// GET /api/v1/sources/{name} — one source
|
||||
// GET /api/v1/auto — scheduling chain + live slot states
|
||||
// GET /api/v1/keys — key metadata (never the secret)
|
||||
// GET /api/v1/models — every model id the gateway can route
|
||||
// GET /api/v1/health — per-source health, no auth needed beyond the key
|
||||
//
|
||||
// Auth is the same gateway key as everywhere else (Authorization: Bearer, or
|
||||
// ?api_key=, or the gw_key cookie). Read endpoints accept any role; writes
|
||||
// still require admin, enforced by the same middleware the UI goes through.
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// apiV1Routes dispatches /api/v1/*. Kept separate from routes() so the original
|
||||
// switch stays readable and the versioned surface can grow on its own.
|
||||
func (g *Gateway) apiV1Routes(w http.ResponseWriter, r *http.Request) {
|
||||
path := strings.TrimPrefix(r.URL.Path, "/api/v1")
|
||||
path = strings.Trim(path, "/")
|
||||
|
||||
switch {
|
||||
case path == "":
|
||||
g.apiV1Index(w, r)
|
||||
case path == "overview":
|
||||
g.apiV1Overview(w, r)
|
||||
case path == "health":
|
||||
g.apiV1Health(w, r)
|
||||
case path == "models":
|
||||
g.apiV1Models(w, r)
|
||||
case path == "sources":
|
||||
if r.Method != http.MethodGet {
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{"sources": maskSources(g.core.Sources())})
|
||||
case strings.HasPrefix(path, "sources/"):
|
||||
name, err := decodePathSegment(strings.TrimPrefix(path, "sources/"))
|
||||
if err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", err.Error())
|
||||
return
|
||||
}
|
||||
if r.Method != http.MethodGet {
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed",
|
||||
"use PUT/DELETE on /api/sources/{name} to modify a source")
|
||||
return
|
||||
}
|
||||
for _, s := range g.core.Sources() {
|
||||
if s.Name == name {
|
||||
// ?reveal=credentials is admin-only and is what the Web UI's
|
||||
// edit dialog uses: a form that round-trips a source must be
|
||||
// able to show the current key, otherwise saving an unrelated
|
||||
// field would blank it. Everything else stays masked.
|
||||
if r.URL.Query().Get("reveal") == "credentials" {
|
||||
if reqRole(r.Context()) != "admin" {
|
||||
writeError(w, http.StatusForbidden, "forbidden",
|
||||
"admin role required to reveal credentials")
|
||||
return
|
||||
}
|
||||
// The edit form round-trips this response, so it needs the credential in the
|
||||
// clear AND the durations — see maskSourceFields.
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{"source": maskSourceFields(s, true, true)})
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{"source": maskSource(s)})
|
||||
return
|
||||
}
|
||||
}
|
||||
writeError(w, http.StatusNotFound, "not_found", "no such source: "+name)
|
||||
case path == "auto":
|
||||
if r.Method != http.MethodGet {
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{
|
||||
"rules": g.core.AutoRules(),
|
||||
"image_rules": g.core.AutoImageRules(),
|
||||
"states": g.core.AutoSlotStates(),
|
||||
})
|
||||
case path == "keys":
|
||||
if reqRole(r.Context()) != "admin" {
|
||||
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
|
||||
return
|
||||
}
|
||||
if r.Method != http.MethodGet {
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed",
|
||||
"use POST/PUT/DELETE on /api/keys to manage keys")
|
||||
return
|
||||
}
|
||||
keys := g.core.ListKeys()
|
||||
out := make([]map[string]interface{}, 0, len(keys))
|
||||
for _, k := range keys {
|
||||
out = append(out, map[string]interface{}{
|
||||
"name": k.Name,
|
||||
"role": k.Role,
|
||||
"models": k.Models,
|
||||
"note": k.Note,
|
||||
"created_at": k.CreatedAt,
|
||||
"seed": k.Seed,
|
||||
// Quotas live on the scope entries (k.Models), echoed above;
|
||||
// there is deliberately no key-wide total.
|
||||
// The secret itself is never echoed. An operator that needs it
|
||||
// already has it from creation time or from config.yaml.
|
||||
"key_prefix": maskKey(k.Key),
|
||||
})
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{"keys": out})
|
||||
default:
|
||||
g.apiV1Index(w, r)
|
||||
}
|
||||
}
|
||||
|
||||
// apiV1Index is the discovery document: it names every endpoint with its
|
||||
// method, auth requirement and purpose, so an agent does not have to scrape the
|
||||
// HTML to find out what it can call.
|
||||
func (g *Gateway) apiV1Index(w http.ResponseWriter, r *http.Request) {
|
||||
type ep struct {
|
||||
Method string `json:"method"`
|
||||
Path string `json:"path"`
|
||||
Auth string `json:"auth"`
|
||||
Summary string `json:"summary"`
|
||||
WriteEffect string `json:"writes,omitempty"`
|
||||
}
|
||||
index := map[string]interface{}{
|
||||
"api_version": "v1",
|
||||
"description": "llmsproxy management API. Authenticate with a gateway key: " +
|
||||
"'Authorization: Bearer <key>', '?api_key=<key>', or the gw_key cookie. " +
|
||||
"Read endpoints accept any role; mutations require an admin key.",
|
||||
"discovery": []ep{
|
||||
{Method: "GET", Path: "/api/v1", Auth: "any", Summary: "this document"},
|
||||
{Method: "GET", Path: "/api/v1/overview", Auth: "any",
|
||||
Summary: "current state in one call: sources, auto chain, key count, source health"},
|
||||
{Method: "GET", Path: "/api/v1/health", Auth: "any",
|
||||
Summary: "per-source health snapshot only"},
|
||||
{Method: "GET", Path: "/api/v1/models", Auth: "any",
|
||||
Summary: "every model id the gateway can route, grouped by owning source"},
|
||||
{Method: "GET", Path: "/api/v1/sources", Auth: "any",
|
||||
Summary: "all sources with credentials masked"},
|
||||
{Method: "GET", Path: "/api/v1/sources/{name}", Auth: "any", Summary: "one source"},
|
||||
{Method: "GET", Path: "/api/v1/auto", Auth: "any",
|
||||
Summary: "AUTO scheduling chain and live per-slot state"},
|
||||
{Method: "GET", Path: "/api/v1/keys", Auth: "admin",
|
||||
Summary: "gateway key metadata; secrets are never returned"},
|
||||
|
||||
{Method: "GET", Path: "/api/sources", Auth: "admin", Summary: "raw source list (includes api_key)"},
|
||||
{Method: "POST", Path: "/api/sources", Auth: "admin", Summary: "add or replace a source",
|
||||
WriteEffect: "writes config.yaml (api_key sealed at rest)"},
|
||||
{Method: "PUT", Path: "/api/sources/{name}", Auth: "admin", Summary: "update one source",
|
||||
WriteEffect: "writes config.yaml (api_key sealed at rest)"},
|
||||
{Method: "DELETE", Path: "/api/sources/{name}", Auth: "admin", Summary: "delete a source",
|
||||
WriteEffect: "writes config.yaml"},
|
||||
|
||||
{Method: "GET", Path: "/api/auto", Auth: "any", Summary: "scheduling chain (readable by any role)"},
|
||||
{Method: "PUT", Path: "/api/auto", Auth: "admin", Summary: "replace the scheduling chain",
|
||||
WriteEffect: "writes config.yaml (the AUTO chain lives here)"},
|
||||
|
||||
{Method: "GET", Path: "/api/keys", Auth: "admin", Summary: "gateway keys"},
|
||||
{Method: "POST", Path: "/api/keys", Auth: "admin", Summary: "create a gateway key",
|
||||
WriteEffect: "writes config.yaml"},
|
||||
{Method: "PUT", Path: "/api/keys/{key}", Auth: "admin",
|
||||
Summary: "update a gateway key (name, role, model scopes and their per-model quotas)",
|
||||
WriteEffect: "writes config.yaml"},
|
||||
{Method: "DELETE", Path: "/api/keys/{key}", Auth: "admin",
|
||||
Summary: "delete a gateway key — the path segment is the KEY itself, not its name",
|
||||
WriteEffect: "writes config.yaml"},
|
||||
|
||||
{Method: "GET", Path: "/api/status", Auth: "any", Summary: "per-source health detail"},
|
||||
{Method: "GET", Path: "/api/stats", Auth: "any", Summary: "usage aggregates"},
|
||||
{Method: "GET", Path: "/api/stats/records", Auth: "any", Summary: "paged request records"},
|
||||
{Method: "GET", Path: "/api/adapters", Auth: "admin", Summary: "installed Lua adapters"},
|
||||
{Method: "GET", Path: "/api/source_templates", Auth: "admin", Summary: "source templates"},
|
||||
{Method: "POST", Path: "/v1/chat/completions", Auth: "any", Summary: "OpenAI-compatible inference"},
|
||||
},
|
||||
"conventions": map[string]interface{}{
|
||||
"errors": "{ \"error\": { \"type\": <code>, \"message\": <text> } }",
|
||||
"path_escape": "URL-encode source and key names; {name} is a single path segment",
|
||||
"idempotency": "POST /api/sources and PUT /api/sources/{name} both upsert by name; " +
|
||||
"PUT takes the name from the path and rejects a body name that disagrees",
|
||||
"partial_update": "api_key may be omitted or sent as the literal \"__KEEP__\" to inherit the " +
|
||||
"current credential; model_ids adds models to the existing list instead of replacing it, " +
|
||||
"so a one-field edit never needs to read the source first",
|
||||
"optional_fields": "proxy_url, api_key_env, timeout and queue_timeout use presence semantics: " +
|
||||
"OMITTED from the request keeps the stored value, present (even as \"\") overwrites it. " +
|
||||
"The upsert rewrites the whole source, so without this a one-field edit would silently reset " +
|
||||
"them — and api_key_env in particular is only visible as a credential failure on the NEXT " +
|
||||
"upstream call. api_key deliberately keeps the older \"empty means inherit\" rule instead, " +
|
||||
"because losing a credential breaks the source while losing a proxy only changes its route.",
|
||||
"durations": "timeout and queue_timeout are Go duration strings (\"300s\", \"5m\", \"1h\"); " +
|
||||
"the empty string clears them back to the config defaults",
|
||||
"config_truth": "all configuration lives in config.yaml; API writes are persisted immediately",
|
||||
"credentials": "credentials are masked by default. GET /api/v1/sources/{name}?reveal=credentials " +
|
||||
"returns them in the clear and is admin-only — the Web UI edit dialog uses it, because a form " +
|
||||
"that round-trips a source must show the current key or saving another field would blank it.",
|
||||
},
|
||||
}
|
||||
writeJSON(w, http.StatusOK, index)
|
||||
}
|
||||
|
||||
// apiV1Overview is the "what is the current state" call an agent makes first.
|
||||
// One round trip instead of five.
|
||||
func (g *Gateway) apiV1Overview(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet {
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
|
||||
return
|
||||
}
|
||||
sources := g.core.Sources()
|
||||
names := make([]string, 0, len(sources))
|
||||
modelCount := 0
|
||||
for _, s := range sources {
|
||||
names = append(names, s.Name)
|
||||
modelCount += len(s.Models)
|
||||
}
|
||||
keys := g.core.ListKeys()
|
||||
adminCount, userCount := 0, 0
|
||||
for _, k := range keys {
|
||||
if k.Role == "admin" {
|
||||
adminCount++
|
||||
} else {
|
||||
userCount++
|
||||
}
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{
|
||||
"api_version": "v1",
|
||||
"summary": map[string]interface{}{
|
||||
"source_count": len(sources),
|
||||
"model_count": modelCount,
|
||||
"gateway_key_count": len(keys),
|
||||
"admin_key_count": adminCount,
|
||||
"user_key_count": userCount,
|
||||
"auto_slots": len(g.core.AutoRules()),
|
||||
},
|
||||
"sources": maskSources(sources),
|
||||
"auto": g.core.AutoRules(),
|
||||
"auto_image": g.core.AutoImageRules(),
|
||||
"health": g.sourceHealthBrief(),
|
||||
"caller": map[string]interface{}{
|
||||
"role": reqRole(r.Context()),
|
||||
"key": maskKey(reqKey(r.Context())),
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func (g *Gateway) apiV1Health(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet {
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{"sources": g.sourceHealthBrief()})
|
||||
}
|
||||
|
||||
// apiV1Models lists routable model ids per source, which is what an agent needs
|
||||
// to build a valid request — /v1/models flattens them and hides the owner.
|
||||
func (g *Gateway) apiV1Models(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet {
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
|
||||
return
|
||||
}
|
||||
bySource := map[string][]string{}
|
||||
var all []string
|
||||
for _, s := range g.core.Sources() {
|
||||
for _, m := range s.Models {
|
||||
bySource[s.Name] = append(bySource[s.Name], m.ID)
|
||||
all = append(all, m.ID)
|
||||
}
|
||||
}
|
||||
sort.Strings(all)
|
||||
all = dedupeStrings(all)
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{
|
||||
"count": len(all),
|
||||
"models": all,
|
||||
"by_source": bySource,
|
||||
"note": "request a model as \"<source>:<model>\" to pin one source, or the bare id to let the gateway choose",
|
||||
})
|
||||
}
|
||||
|
||||
// ---- helpers ----
|
||||
|
||||
// maskSources returns sources with credentials replaced by a presence marker.
|
||||
// The UI still uses /api/sources (which returns real values, because its edit
|
||||
// form round-trips them); this is the safe view for programmatic callers.
|
||||
func maskSources(srcs []config.Source) []map[string]interface{} {
|
||||
out := make([]map[string]interface{}, 0, len(srcs))
|
||||
for _, s := range srcs {
|
||||
out = append(out, maskSource(s))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func maskSource(s config.Source) map[string]interface{} {
|
||||
return maskSourceFields(s, false, false)
|
||||
}
|
||||
|
||||
// maskSourceFields builds the API view of a source.
|
||||
//
|
||||
// revealCredentials swaps the masked api_key for the real one — admin-only, and
|
||||
// used by the WebUI edit dialog, whose form has to round-trip the whole source
|
||||
// or saving an unrelated field would blank the key.
|
||||
//
|
||||
// exposeDurations adds the two durations as strings, matching what the write
|
||||
// path accepts. It rides along with the credential reveal because both exist for
|
||||
// the same reason: config.Source tags Timeout/QueueTimeout `json:"-"`, so a
|
||||
// plain marshal of the struct omits them. A form that cannot SEE the stored
|
||||
// timeout would clear it on every save, since the timeout box is always sent.
|
||||
func maskSourceFields(s config.Source, revealCredentials, exposeDurations bool) map[string]interface{} {
|
||||
key := maskKey(s.APIKey)
|
||||
if revealCredentials {
|
||||
key = s.APIKey
|
||||
}
|
||||
m := map[string]interface{}{
|
||||
"name": s.Name,
|
||||
"base_url": s.BaseURL,
|
||||
"adapter": s.Adapter,
|
||||
"endpoint": s.Endpoint,
|
||||
"image_endpoint": s.ImageEndpoint,
|
||||
"api_key": key,
|
||||
"api_key_set": s.APIKey != "",
|
||||
"models": s.Models,
|
||||
"headers": maskHeaders(s.Headers),
|
||||
"proxy_url": s.ProxyURL,
|
||||
"api_key_env": s.APIKeyEnv,
|
||||
"meta": s.Meta,
|
||||
"temperature": s.Temperature,
|
||||
"max_tokens": s.MaxTokens,
|
||||
"max_concurrent": s.MaxConcurrent,
|
||||
"rpm": s.RPM,
|
||||
}
|
||||
if exposeDurations {
|
||||
m["timeout"] = s.Timeout.String()
|
||||
m["queue_timeout"] = s.QueueTimeout.String()
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func maskKey(k string) string {
|
||||
if k == "" {
|
||||
return ""
|
||||
}
|
||||
if strings.HasPrefix(k, "enc:v1:") {
|
||||
return "(sealed)"
|
||||
}
|
||||
if len(k) <= 10 {
|
||||
return k[:2] + "…"
|
||||
}
|
||||
return k[:6] + "…" + k[len(k)-4:]
|
||||
}
|
||||
|
||||
func maskHeaders(h map[string]string) map[string]string {
|
||||
if h == nil {
|
||||
return nil
|
||||
}
|
||||
out := make(map[string]string, len(h))
|
||||
for k, v := range h {
|
||||
out[k] = maskKey(v)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// sourceHealthBrief reuses the same registry status the UI's /api/status shows,
|
||||
// so the agent view and the UI view cannot drift apart. Recent-traffic counters
|
||||
// come from the same stats window, because a source that is actually serving
|
||||
// traffic must never look down just because a probe was rate-limited.
|
||||
func (g *Gateway) sourceHealthBrief() []map[string]interface{} {
|
||||
sts := g.core.Registry().Status()
|
||||
recent := g.stats.SourceRecent(300)
|
||||
avgs := g.stats.SourceAverages(300)
|
||||
for i := range sts {
|
||||
if v, ok := recent[sts[i].Name]; ok {
|
||||
sts[i].RecentOK = v[0]
|
||||
sts[i].RecentErr = v[1]
|
||||
}
|
||||
if a, ok := avgs[sts[i].Name]; ok {
|
||||
sts[i].AvgFirstByteMs = a.AvgFirstByteMs
|
||||
sts[i].AvgTokPerS = a.AvgTokPerS
|
||||
}
|
||||
}
|
||||
out := make([]map[string]interface{}, 0, len(sts))
|
||||
for _, s := range sts {
|
||||
row := map[string]interface{}{
|
||||
"name": s.Name,
|
||||
"adapter": s.Adapter,
|
||||
"healthy": s.Healthy,
|
||||
"available": s.Available,
|
||||
"live_available": s.LiveAvailable,
|
||||
"model_count": len(s.Models),
|
||||
}
|
||||
if s.LastError != "" {
|
||||
row["last_error"] = s.LastError
|
||||
}
|
||||
if s.LastChecked > 0 {
|
||||
row["last_checked"] = s.LastChecked
|
||||
}
|
||||
if s.RecentOK > 0 || s.RecentErr > 0 {
|
||||
row["recent_ok"] = s.RecentOK
|
||||
row["recent_err"] = s.RecentErr
|
||||
}
|
||||
if s.AvgFirstByteMs > 0 {
|
||||
row["avg_first_byte_ms"] = s.AvgFirstByteMs
|
||||
}
|
||||
if s.AvgTokPerS > 0 {
|
||||
row["avg_tok_per_s"] = s.AvgTokPerS
|
||||
}
|
||||
if s.FailCount > 0 {
|
||||
row["fail_count"] = s.FailCount
|
||||
}
|
||||
if s.Permanent {
|
||||
row["permanent"] = true
|
||||
}
|
||||
out = append(out, row)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// decodePathSegment URL-decodes one path segment and rejects an empty result,
|
||||
// so a name containing a slash cannot be silently mis-resolved.
|
||||
func decodePathSegment(seg string) (string, error) {
|
||||
s, err := url.PathUnescape(seg)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if s == "" {
|
||||
return "", errEmptySegment
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
func dedupeStrings(in []string) []string {
|
||||
out := in[:0]
|
||||
var last string
|
||||
for i, s := range in {
|
||||
if i == 0 || s != last {
|
||||
out = append(out, s)
|
||||
}
|
||||
last = s
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// errEmptySegment marks a path like /api/v1/sources/ with no name after it.
|
||||
var errEmptySegment = errors.New("empty path segment")
|
||||
@ -1,299 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// v1Gateway builds a gateway with one source and two keys, and returns the
|
||||
// secrets so a test can assert they never appear in a response.
|
||||
func v1Gateway(t *testing.T) (*Gateway, string, string) {
|
||||
t.Helper()
|
||||
g := newTestGateway(t, config.Source{
|
||||
Name: "up",
|
||||
BaseURL: "http://up.example/v1",
|
||||
APIKey: "sk-up-secret",
|
||||
Adapter: "openai",
|
||||
Models: []config.Model{{ID: "m1", Kind: "chat"}},
|
||||
})
|
||||
// The shared helper only seeds one admin key; add a user key through the
|
||||
// core so role-gated endpoints have something to reject.
|
||||
if _, err := g.core.CreateKey("agent", "user", nil, "test agent key"); err != nil {
|
||||
t.Fatalf("CreateKey: %v", err)
|
||||
}
|
||||
rec := doReq(t, g, http.MethodGet, "/api/keys", "")
|
||||
var doc struct {
|
||||
Keys []config.GWKey `json:"keys"`
|
||||
}
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), &doc); err != nil {
|
||||
t.Fatalf("decode keys: %v", err)
|
||||
}
|
||||
userKey := ""
|
||||
for _, k := range doc.Keys {
|
||||
if k.Role == "user" {
|
||||
userKey = k.Key
|
||||
}
|
||||
}
|
||||
if userKey == "" {
|
||||
t.Fatal("no user key was created")
|
||||
}
|
||||
return g, "sk-test", userKey
|
||||
}
|
||||
|
||||
func decodeJSON(t *testing.T, body string, v interface{}) {
|
||||
t.Helper()
|
||||
if err := json.Unmarshal([]byte(body), v); err != nil {
|
||||
t.Fatalf("decode %s: %v", body, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIV1IndexIsDiscoverable(t *testing.T) {
|
||||
g, _, _ := v1Gateway(t)
|
||||
rec := doReq(t, g, http.MethodGet, "/api/v1", "")
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200: %s", rec.Code, rec.Body.String())
|
||||
}
|
||||
var doc struct {
|
||||
APIVersion string `json:"api_version"`
|
||||
Description string `json:"description"`
|
||||
Discovery []struct {
|
||||
Method string `json:"method"`
|
||||
Path string `json:"path"`
|
||||
Auth string `json:"auth"`
|
||||
Summary string `json:"summary"`
|
||||
} `json:"discovery"`
|
||||
Conventions map[string]interface{} `json:"conventions"`
|
||||
}
|
||||
decodeJSON(t, rec.Body.String(), &doc)
|
||||
if doc.APIVersion != "v1" {
|
||||
t.Errorf("api_version = %q", doc.APIVersion)
|
||||
}
|
||||
if doc.Description == "" {
|
||||
t.Error("index should carry a usage description")
|
||||
}
|
||||
if len(doc.Discovery) == 0 {
|
||||
t.Fatal("discovery list is empty")
|
||||
}
|
||||
want := map[string]bool{
|
||||
"/api/v1/overview": false, "/api/v1/sources": false,
|
||||
"/api/v1/auto": false, "/api/v1/models": false,
|
||||
"/api/v1/health": false, "/api/v1/keys": false,
|
||||
"/api/sources": false, "/api/auto": false, "/v1/chat/completions": false,
|
||||
}
|
||||
for _, e := range doc.Discovery {
|
||||
if _, ok := want[e.Path]; ok {
|
||||
want[e.Path] = true
|
||||
}
|
||||
if e.Method == "" || e.Path == "" || e.Auth == "" || e.Summary == "" {
|
||||
t.Errorf("incomplete discovery entry: %+v", e)
|
||||
}
|
||||
}
|
||||
for p, found := range want {
|
||||
if !found {
|
||||
t.Errorf("discovery is missing %s", p)
|
||||
}
|
||||
}
|
||||
if doc.Conventions["errors"] == nil {
|
||||
t.Error("conventions should document the error shape")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIV1OverviewSummarisesState(t *testing.T) {
|
||||
g, _, _ := v1Gateway(t)
|
||||
rec := doReq(t, g, http.MethodGet, "/api/v1/overview", "")
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d: %s", rec.Code, rec.Body.String())
|
||||
}
|
||||
var doc struct {
|
||||
Summary struct {
|
||||
SourceCount int `json:"source_count"`
|
||||
ModelCount int `json:"model_count"`
|
||||
GatewayKeyCount int `json:"gateway_key_count"`
|
||||
} `json:"summary"`
|
||||
Sources []map[string]interface{} `json:"sources"`
|
||||
Auto []map[string]interface{} `json:"auto"`
|
||||
Health []map[string]interface{} `json:"health"`
|
||||
Caller struct {
|
||||
Role string `json:"role"`
|
||||
} `json:"caller"`
|
||||
}
|
||||
decodeJSON(t, rec.Body.String(), &doc)
|
||||
if doc.Summary.SourceCount != 1 {
|
||||
t.Errorf("source_count = %d, want 1", doc.Summary.SourceCount)
|
||||
}
|
||||
if doc.Summary.ModelCount != 1 {
|
||||
t.Errorf("model_count = %d, want 1", doc.Summary.ModelCount)
|
||||
}
|
||||
if doc.Summary.GatewayKeyCount < 2 {
|
||||
t.Errorf("gateway_key_count = %d, want >= 2", doc.Summary.GatewayKeyCount)
|
||||
}
|
||||
if len(doc.Sources) != 1 || doc.Sources[0]["name"] != "up" {
|
||||
t.Errorf("sources = %+v", doc.Sources)
|
||||
}
|
||||
if doc.Caller.Role != "admin" {
|
||||
t.Errorf("caller.role = %q, want admin", doc.Caller.Role)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIV1NeverEchoesSecrets(t *testing.T) {
|
||||
g, admin, user := v1Gateway(t)
|
||||
for _, path := range []string{"/api/v1", "/api/v1/overview", "/api/v1/sources", "/api/v1/sources/up", "/api/v1/keys", "/api/v1/models", "/api/v1/health", "/api/v1/auto"} {
|
||||
rec := doReq(t, g, http.MethodGet, path, "")
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("%s status = %d: %s", path, rec.Code, rec.Body.String())
|
||||
}
|
||||
if strings.Contains(rec.Body.String(), "sk-up-secret") {
|
||||
t.Errorf("%s leaked the source api_key", path)
|
||||
}
|
||||
if admin != "" && strings.Contains(rec.Body.String(), admin) {
|
||||
t.Errorf("%s leaked the admin gateway key", path)
|
||||
}
|
||||
if strings.Contains(rec.Body.String(), user) {
|
||||
t.Errorf("%s leaked the user gateway key", path)
|
||||
}
|
||||
}
|
||||
// Masking must still be useful: it says a key is configured.
|
||||
rec := doReq(t, g, http.MethodGet, "/api/v1/sources/up", "")
|
||||
if !strings.Contains(rec.Body.String(), `"api_key_set":true`) {
|
||||
t.Error("masked source should report that a key is configured")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIV1KeysRequiresAdminAndMasks(t *testing.T) {
|
||||
g, _, user := v1Gateway(t)
|
||||
// Authenticate as the user key by swapping the shared helper's header.
|
||||
req := newAuthedRequest(t, http.MethodGet, "/api/v1/keys", user)
|
||||
rec := serveViaHandler(t, g, req)
|
||||
if rec.Code != http.StatusForbidden {
|
||||
t.Errorf("user role on /api/v1/keys = %d, want 403: %s", rec.Code, rec.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIV1SourcesNotFound(t *testing.T) {
|
||||
g, _, _ := v1Gateway(t)
|
||||
rec := doReq(t, g, http.MethodGet, "/api/v1/sources/nope", "")
|
||||
if rec.Code != http.StatusNotFound {
|
||||
t.Errorf("status = %d, want 404: %s", rec.Code, rec.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIV1RejectsWrites(t *testing.T) {
|
||||
g, _, _ := v1Gateway(t)
|
||||
for _, m := range []string{http.MethodPost, http.MethodPut, http.MethodDelete, http.MethodPatch} {
|
||||
rec := doReq(t, g, m, "/api/v1/sources", `{}`)
|
||||
if rec.Code == http.StatusOK {
|
||||
t.Errorf("%s /api/v1/sources returned 200; the v1 read facade must not mutate", m)
|
||||
}
|
||||
}
|
||||
// And the source must be untouched.
|
||||
rec := doReq(t, g, http.MethodGet, "/api/v1/sources", "")
|
||||
if !strings.Contains(rec.Body.String(), `"name":"up"`) {
|
||||
t.Error("the source disappeared after a rejected write")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIV1ModelsGroupsBySource(t *testing.T) {
|
||||
g, _, _ := v1Gateway(t)
|
||||
rec := doReq(t, g, http.MethodGet, "/api/v1/models", "")
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d", rec.Code)
|
||||
}
|
||||
var doc struct {
|
||||
Count int `json:"count"`
|
||||
Models []string `json:"models"`
|
||||
BySource map[string][]string `json:"by_source"`
|
||||
}
|
||||
decodeJSON(t, rec.Body.String(), &doc)
|
||||
if doc.Count != 1 || doc.Models[0] != "m1" {
|
||||
t.Errorf("models = %+v", doc.Models)
|
||||
}
|
||||
if len(doc.BySource["up"]) != 1 || doc.BySource["up"][0] != "m1" {
|
||||
t.Errorf("by_source = %+v", doc.BySource)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIV1RequiresAuthentication(t *testing.T) {
|
||||
g, _, _ := v1Gateway(t)
|
||||
for _, path := range []string{"/api/v1", "/api/v1/overview", "/api/v1/sources", "/api/v1/health", "/api/v1/models"} {
|
||||
req := newRequest(http.MethodGet, path, "")
|
||||
rec := serveViaHandler(t, g, req)
|
||||
if rec.Code != http.StatusUnauthorized {
|
||||
t.Errorf("%s without a key = %d, want 401", path, rec.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestMaskKeyHidesTheSecret(t *testing.T) {
|
||||
cases := map[string]string{
|
||||
"": "",
|
||||
"ab": "ab…",
|
||||
"sk-gw-abcdefghijkl": "sk-gw-…ijkl",
|
||||
"enc:v1:AAAA": "(sealed)",
|
||||
}
|
||||
for in, want := range cases {
|
||||
if got := maskKey(in); got != want {
|
||||
t.Errorf("maskKey(%q) = %q, want %q", in, got, want)
|
||||
}
|
||||
}
|
||||
// The masked form must not reveal the middle of the secret.
|
||||
full := maskKey("sk-gw-1234567890abcdef")
|
||||
if strings.Contains(full, "4567890abcdef") {
|
||||
t.Errorf("maskKey leaked the middle: %q", full)
|
||||
}
|
||||
}
|
||||
|
||||
// newRequest builds an unauthenticated request.
|
||||
func newRequest(method, path, body string) *http.Request {
|
||||
req, _ := http.NewRequest(method, path, strings.NewReader(body))
|
||||
if body != "" {
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
return req
|
||||
}
|
||||
|
||||
// newAuthedRequest builds a request carrying the given bearer key.
|
||||
func newAuthedRequest(t *testing.T, method, path, key string) *http.Request {
|
||||
t.Helper()
|
||||
req := newRequest(method, path, "")
|
||||
req.Header.Set("Authorization", "Bearer "+key)
|
||||
return req
|
||||
}
|
||||
|
||||
// serveViaHandler pushes a request through the full handler chain (auth included).
|
||||
func serveViaHandler(t *testing.T, g *Gateway, req *http.Request) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
rec := httptest.NewRecorder()
|
||||
g.Handler().ServeHTTP(rec, req)
|
||||
return rec
|
||||
}
|
||||
|
||||
// TestAPIV1RevealRequiresAdmin: the only way to read a key in the clear is the
|
||||
// explicit admin-only reveal. Without the role check this endpoint would hand
|
||||
// every source credential to any valid (even user-scoped) key.
|
||||
func TestAPIV1RevealRequiresAdmin(t *testing.T) {
|
||||
g, _, user := v1Gateway(t)
|
||||
rec := serveViaHandler(t, g, newAuthedRequest(t, http.MethodGet, "/api/v1/sources/up?reveal=credentials", user))
|
||||
if rec.Code != http.StatusForbidden {
|
||||
t.Errorf("user reveal = %d, want 403: %s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if strings.Contains(rec.Body.String(), "sk-up-secret") {
|
||||
t.Error("a forbidden reveal still leaked the key")
|
||||
}
|
||||
rec = doReq(t, g, http.MethodGet, "/api/v1/sources/up?reveal=credentials", "")
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("admin reveal = %d: %s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if !strings.Contains(rec.Body.String(), "sk-up-secret") {
|
||||
t.Error("admin reveal should return the key in the clear (the edit form needs it)")
|
||||
}
|
||||
// The masked default view must still hide it on the same path.
|
||||
rec = doReq(t, g, http.MethodGet, "/api/v1/sources/up", "")
|
||||
if strings.Contains(rec.Body.String(), "sk-up-secret") {
|
||||
t.Error("the default view leaked the key")
|
||||
}
|
||||
}
|
||||
@ -1,282 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
|
||||
"llmsproxy/internal/billing"
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// Billing rules API.
|
||||
//
|
||||
// GET /api/plugins/billing/rules the active profile's rules + all profiles
|
||||
// PUT /api/plugins/billing/rules replace the whole DSL and re-inject prices
|
||||
// POST /api/plugins/billing/rules { profile, rule } append one rule
|
||||
// DELETE /api/plugins/billing/rules { profile, url } remove one rule
|
||||
//
|
||||
// Why a separate endpoint instead of the generic plugin state: prices are
|
||||
// plugin state, but they are also a durable, reviewable CONFIGURATION. Routing
|
||||
// them through /state would let an admin key PUT arbitrary plugin state and
|
||||
// silently reset the accumulated accounting. Here the server owns the shape:
|
||||
// it validates the DSL, writes it back to config.yaml, recompiles, and hands
|
||||
// the plugin its prices table. The operator edits rules; the plugin keeps
|
||||
// numbers. The two are never mixed in one payload.
|
||||
//
|
||||
// Editing means "what the operator typed is what config.yaml holds". A
|
||||
// billing mistake found next week must be traceable to a reviewable file, not
|
||||
// to a blob in the plugin's state sidecar.
|
||||
func (g *Gateway) handleBillingRules(w http.ResponseWriter, r *http.Request) {
|
||||
if reqRole(r.Context()) != "admin" {
|
||||
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
|
||||
return
|
||||
}
|
||||
switch r.Method {
|
||||
case http.MethodGet:
|
||||
g.getBillingRules(w)
|
||||
case http.MethodPut:
|
||||
g.putBillingRules(w, r)
|
||||
case http.MethodPost:
|
||||
g.addBillingRule(w, r)
|
||||
case http.MethodDelete:
|
||||
g.delBillingRule(w, r)
|
||||
default:
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET/PUT/POST/DELETE")
|
||||
}
|
||||
}
|
||||
|
||||
// billingRulesPayload is the UI-facing view: the profiles as declared, plus
|
||||
// which one is active and the source URLs they can match — the editor needs
|
||||
// the URL list to offer suggestions, and an operator typing a URL that matches
|
||||
// no source is exactly the silent-no-pricing failure this feature exists to
|
||||
// remove.
|
||||
type billingRulesPayload struct {
|
||||
DSL *config.BillingDSL `json:"billing"`
|
||||
Active string `json:"active"`
|
||||
URLs []string `json:"urls"`
|
||||
Warnings []string `json:"warnings"`
|
||||
}
|
||||
|
||||
func (g *Gateway) billingRulesPayload() billingRulesPayload {
|
||||
out := billingRulesPayload{}
|
||||
cfg := g.core.Config()
|
||||
if cfg != nil && cfg.BillingDSL != nil {
|
||||
// Copy so the response cannot be mutated back through the pointer.
|
||||
cp := *cfg.BillingDSL
|
||||
out.DSL = &cp
|
||||
if p := cp.Resolve(""); p != nil {
|
||||
out.Active = p.ID
|
||||
}
|
||||
}
|
||||
for _, s := range g.sourceURLs() {
|
||||
out.URLs = append(out.URLs, s)
|
||||
}
|
||||
out.Warnings = g.billingRuleWarnings()
|
||||
return out
|
||||
}
|
||||
|
||||
func (g *Gateway) sourceURLs() []string {
|
||||
cfg := g.core.Config()
|
||||
if cfg == nil {
|
||||
return nil
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
var out []string
|
||||
for _, s := range cfg.Sources {
|
||||
if s.BaseURL == "" || seen[s.BaseURL] {
|
||||
continue
|
||||
}
|
||||
seen[s.BaseURL] = true
|
||||
out = append(out, s.BaseURL)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// billingRuleWarnings reports rules that match no configured source. This is
|
||||
// the silent killer of hand-written price tables: the rule parses, validates,
|
||||
// and prices nothing at all, so every request on that URL lands in unpriced.
|
||||
// Saying so out loud is the difference between a five-second fix and an
|
||||
// afternoon wondering why the bill is zero.
|
||||
func (g *Gateway) billingRuleWarnings() []string {
|
||||
cfg := g.core.Config()
|
||||
if cfg == nil || cfg.BillingDSL == nil {
|
||||
return nil
|
||||
}
|
||||
urls := map[string]bool{}
|
||||
for _, u := range g.sourceURLs() {
|
||||
urls[u] = true
|
||||
}
|
||||
var warns []string
|
||||
for _, p := range cfg.BillingDSL.Profiles {
|
||||
for _, r := range p.Rules {
|
||||
if r.URL == "*" || urls[r.URL] {
|
||||
continue
|
||||
}
|
||||
warns = append(warns, "profile "+p.ID+": rule url "+r.URL+" matches no configured source")
|
||||
}
|
||||
}
|
||||
return warns
|
||||
}
|
||||
|
||||
func (g *Gateway) getBillingRules(w http.ResponseWriter) {
|
||||
writeJSON(w, http.StatusOK, g.billingRulesPayload())
|
||||
}
|
||||
|
||||
func (g *Gateway) putBillingRules(w http.ResponseWriter, r *http.Request) {
|
||||
var body struct {
|
||||
Billing *config.BillingDSL `json:"billing"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
|
||||
return
|
||||
}
|
||||
if body.Billing == nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "billing is required")
|
||||
return
|
||||
}
|
||||
g.applyBillingDSLUpdate(w, body.Billing)
|
||||
}
|
||||
|
||||
func (g *Gateway) addBillingRule(w http.ResponseWriter, r *http.Request) {
|
||||
var body struct {
|
||||
Profile string `json:"profile"`
|
||||
Rule config.BillingRule `json:"rule"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
|
||||
return
|
||||
}
|
||||
cfg := g.core.Config()
|
||||
if cfg == nil || cfg.BillingDSL == nil || len(cfg.BillingDSL.Profiles) == 0 {
|
||||
writeError(w, http.StatusBadRequest, "no_profile", "no billing profiles configured; create one first")
|
||||
return
|
||||
}
|
||||
next := cloneBillingDSL(cfg.BillingDSL)
|
||||
idx := profileIndex(next, body.Profile)
|
||||
if idx < 0 {
|
||||
writeError(w, http.StatusBadRequest, "unknown_profile", "no such profile: "+body.Profile)
|
||||
return
|
||||
}
|
||||
p := &next.Profiles[idx]
|
||||
// Replace rather than append when the URL is already declared: two rules
|
||||
// for one URL would silently make the first unreachable (first match wins),
|
||||
// which is the exact ambiguity an editor should not be able to create.
|
||||
replaced := false
|
||||
for i := range p.Rules {
|
||||
if p.Rules[i].URL == body.Rule.URL {
|
||||
p.Rules[i] = body.Rule
|
||||
replaced = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !replaced {
|
||||
p.Rules = append(p.Rules, body.Rule)
|
||||
}
|
||||
g.applyBillingDSLUpdate(w, next)
|
||||
}
|
||||
|
||||
func (g *Gateway) delBillingRule(w http.ResponseWriter, r *http.Request) {
|
||||
var body struct {
|
||||
Profile string `json:"profile"`
|
||||
URL string `json:"url"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
|
||||
return
|
||||
}
|
||||
cfg := g.core.Config()
|
||||
if cfg == nil || cfg.BillingDSL == nil {
|
||||
writeError(w, http.StatusBadRequest, "no_profile", "no billing profiles configured")
|
||||
return
|
||||
}
|
||||
next := cloneBillingDSL(cfg.BillingDSL)
|
||||
idx := profileIndex(next, body.Profile)
|
||||
if idx < 0 {
|
||||
writeError(w, http.StatusBadRequest, "unknown_profile", "no such profile: "+body.Profile)
|
||||
return
|
||||
}
|
||||
p := &next.Profiles[idx]
|
||||
out := p.Rules[:0]
|
||||
found := false
|
||||
for _, rule := range p.Rules {
|
||||
if rule.URL == body.URL {
|
||||
found = true
|
||||
continue
|
||||
}
|
||||
out = append(out, rule)
|
||||
}
|
||||
if !found {
|
||||
writeError(w, http.StatusNotFound, "not_found", "no rule for url "+body.URL)
|
||||
return
|
||||
}
|
||||
p.Rules = out
|
||||
g.applyBillingDSLUpdate(w, next)
|
||||
}
|
||||
|
||||
// applyBillingDSLUpdate is the single write path: validate, persist to
|
||||
// config.yaml, recompile, inject, and only then report success. If the config
|
||||
// cannot be written the change is NOT applied in memory either — a rules editor
|
||||
// that says "saved" and loses the edit on the next restart is worse than one
|
||||
// that refuses.
|
||||
func (g *Gateway) applyBillingDSLUpdate(w http.ResponseWriter, next *config.BillingDSL) {
|
||||
if err := next.Validate(); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_rules", err.Error())
|
||||
return
|
||||
}
|
||||
cfg := g.core.Config()
|
||||
if cfg == nil {
|
||||
writeError(w, http.StatusInternalServerError, "no_config", "no config loaded")
|
||||
return
|
||||
}
|
||||
// Assign BEFORE saving. The previous order saved first and assigned after,
|
||||
// so every rule edit reported success while writing a config.yaml with no
|
||||
// billing section at all — the operator's edit vanished on restart and
|
||||
// nothing in the API response said so.
|
||||
prev := cfg.BillingDSL
|
||||
cfg.BillingDSL = next
|
||||
|
||||
// Compile before persisting: a profile that cannot be turned into prices
|
||||
// must not reach the config file, or the next restart fails to load the
|
||||
// very rules the editor just accepted.
|
||||
prices, err := billing.CompileOpts(next.Resolve(next.Active), cfg.Sources, true)
|
||||
if err != nil {
|
||||
cfg.BillingDSL = prev // no half-applied state
|
||||
writeError(w, http.StatusBadRequest, "compile_failed", err.Error())
|
||||
return
|
||||
}
|
||||
if err := cfg.Save(); err != nil {
|
||||
cfg.BillingDSL = prev
|
||||
writeError(w, http.StatusInternalServerError, "persist_failed", err.Error())
|
||||
return
|
||||
}
|
||||
ps := g.core.Plugins()
|
||||
if ps != nil {
|
||||
if err := ps.SetState("billing", map[string]interface{}{"prices": prices}); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "plugin_error", err.Error())
|
||||
return
|
||||
}
|
||||
}
|
||||
p := g.billingRulesPayload()
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{
|
||||
"ok": true, "billing": p.DSL, "active": p.Active, "warnings": p.Warnings,
|
||||
})
|
||||
}
|
||||
|
||||
func cloneBillingDSL(d *config.BillingDSL) *config.BillingDSL {
|
||||
out := &config.BillingDSL{Active: d.Active}
|
||||
out.Profiles = make([]config.BillingProfile, len(d.Profiles))
|
||||
copy(out.Profiles, d.Profiles)
|
||||
return out
|
||||
}
|
||||
|
||||
func profileIndex(d *config.BillingDSL, id string) int {
|
||||
if id == "" {
|
||||
return -1
|
||||
}
|
||||
for i, p := range d.Profiles {
|
||||
if p.ID == id {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
@ -1,281 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"sort"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// rulesGateway needs a real billing plugin (the API injects prices into it)
|
||||
// AND two sources whose base_urls the rules will match, so it cannot reuse
|
||||
// either existing helper: gatewayWithBilling has no sources, and
|
||||
// newTestGateway loads no plugins.
|
||||
func rulesGateway(t *testing.T) *Gateway {
|
||||
t.Helper()
|
||||
g := gatewayWithBilling(t)
|
||||
cfg := g.core.Config()
|
||||
cfg.Sources = []config.Source{
|
||||
{Name: "localzen", BaseURL: "https://free.example.com/v1"},
|
||||
{Name: "commandcode", BaseURL: "https://api.commandcode.ai/v1"},
|
||||
}
|
||||
return g
|
||||
}
|
||||
|
||||
func rulesDo(t *testing.T, g *Gateway, method, body string) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
return doReq(t, g, method, "/api/plugins/billing/rules", body)
|
||||
}
|
||||
|
||||
// TestBillingRulesCRUD covers the whole editor loop against the real HTTP
|
||||
// surface: read, add, replace-by-URL, delete. The replace-by-URL case is the
|
||||
// one worth pinning — two rules for one URL would make the first unreachable
|
||||
// (first match wins) without any error, which is exactly the kind of silent
|
||||
// ambiguity an editor must not be able to create.
|
||||
func TestBillingRulesCRUD(t *testing.T) {
|
||||
g := rulesGateway(t)
|
||||
seed := `{"billing":{"active":"p1","profiles":[{"id":"p1","rules":[
|
||||
{"url":"https://free.example.com/v1","mode":"free"}]}]}}`
|
||||
if rr := rulesDo(t, g, http.MethodPut, seed); rr.Code != http.StatusOK {
|
||||
t.Fatalf("seed rules = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
|
||||
// Add a token-priced rule for the second URL.
|
||||
add := `{"profile":"p1","rule":{"url":"https://api.commandcode.ai/v1","mode":"token",
|
||||
"models":{"deepseek-v4.1-flash":{"prompt":"0.15","completion":"0.60"}}}}`
|
||||
if rr := rulesDo(t, g, http.MethodPost, add); rr.Code != http.StatusOK {
|
||||
t.Fatalf("add rule = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var got billingRulesPayload
|
||||
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got.DSL.Profiles[0].Rules) != 2 {
|
||||
t.Fatalf("after add there are %d rules, want 2", len(got.DSL.Profiles[0].Rules))
|
||||
}
|
||||
|
||||
// Adding the SAME url again must replace, not append.
|
||||
if rr := rulesDo(t, g, http.MethodPost, `{"profile":"p1","rule":{"url":"https://api.commandcode.ai/v1","mode":"free"}}`); rr.Code != http.StatusOK {
|
||||
t.Fatalf("re-add same url = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if n := len(got.DSL.Profiles[0].Rules); n != 2 {
|
||||
t.Fatalf("★ re-adding an existing url appended instead of replacing: %d rules", n)
|
||||
}
|
||||
var mode string
|
||||
for _, r := range got.DSL.Profiles[0].Rules {
|
||||
if r.URL == "https://api.commandcode.ai/v1" {
|
||||
mode = r.Mode
|
||||
}
|
||||
}
|
||||
if mode != "free" {
|
||||
t.Errorf("re-added rule mode = %q, want free (the replacement must win)", mode)
|
||||
}
|
||||
|
||||
// Delete by URL.
|
||||
if rr := rulesDo(t, g, http.MethodDelete, `{"profile":"p1","url":"https://api.commandcode.ai/v1"}`); rr.Code != http.StatusOK {
|
||||
t.Fatalf("delete rule = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got.DSL.Profiles[0].Rules) != 1 {
|
||||
t.Errorf("after delete there are %d rules, want 1", len(got.DSL.Profiles[0].Rules))
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingRulesRejectUnknownURLWithWarning is the silent-failure guard: a
|
||||
// rule whose URL matches no source prices nothing, so every request on it
|
||||
// falls into unpriced. The API must say so instead of accepting it silently.
|
||||
func TestBillingRulesRejectUnknownURLWithWarning(t *testing.T) {
|
||||
g := rulesGateway(t)
|
||||
if rr := rulesDo(t, g, http.MethodPut, `{"billing":{"active":"p","profiles":[{"id":"p",
|
||||
"rules":[{"url":"https://typo.example.com/v1","mode":"free"}]}]}}`); rr.Code != http.StatusOK {
|
||||
t.Fatalf("seed = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var got billingRulesPayload
|
||||
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got.Warnings) == 0 {
|
||||
t.Fatal("★ a rule matching no configured source was accepted with no warning — " +
|
||||
"it will price nothing and every request will be recorded as unpriced")
|
||||
}
|
||||
if !strings.Contains(got.Warnings[0], "typo.example.com") {
|
||||
t.Errorf("warning does not name the offending url: %v", got.Warnings)
|
||||
}
|
||||
// The editor needs the real URL list to offer suggestions.
|
||||
if len(got.URLs) != 2 {
|
||||
t.Errorf("urls offered to the editor = %v, want the 2 configured base_urls", got.URLs)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingRulesInvalidIsRejectedNotPersisted checks the write path refuses
|
||||
// bad input and leaves the previous rules in place — an editor that clears the
|
||||
// table on a typo loses the price table.
|
||||
func TestBillingRulesInvalidIsRejectedNotPersisted(t *testing.T) {
|
||||
g := rulesGateway(t)
|
||||
seed := `{"billing":{"active":"p","profiles":[{"id":"p","rules":[
|
||||
{"url":"https://free.example.com/v1","mode":"free"}]}]}}`
|
||||
if rr := rulesDo(t, g, http.MethodPut, seed); rr.Code != http.StatusOK {
|
||||
t.Fatalf("seed = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
// An unknown mode must be refused.
|
||||
if rr := rulesDo(t, g, http.MethodPost, `{"profile":"p","rule":{"url":"https://x/v1","mode":"banana"}}`); rr.Code != http.StatusBadRequest {
|
||||
t.Errorf("invalid mode accepted with %d, want 400", rr.Code)
|
||||
}
|
||||
var got billingRulesPayload
|
||||
if err := json.Unmarshal(rulesDo(t, g, http.MethodGet, "").Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got.DSL.Profiles[0].Rules) != 1 {
|
||||
t.Errorf("★ a rejected rule was persisted anyway: %d rules", len(got.DSL.Profiles[0].Rules))
|
||||
}
|
||||
// An unparseable price must be refused too, not silently priced at zero.
|
||||
if rr := rulesDo(t, g, http.MethodPost, `{"profile":"p","rule":{"url":"https://y/v1","mode":"token",
|
||||
"models":{"m":{"prompt":"free","completion":"0.6"}}}}`); rr.Code != http.StatusBadRequest {
|
||||
t.Errorf("non-numeric price accepted with %d, want 400", rr.Code)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingRulesPersistAndReachThePlugin is the point of the whole
|
||||
// endpoint, and mutation verification showed the earlier CRUD tests missed it
|
||||
// twice: dropping cfg.Save() and dropping the SetState injection both left
|
||||
// every test green. Both failures are invisible in a GET — the response echoes
|
||||
// the in-memory rules either way — so this test checks the two consequences
|
||||
// that actually matter:
|
||||
//
|
||||
// 1. persistence: the rule must be in the config file on disk, because an
|
||||
// edit that evaporates on restart is a lie the UI told the operator;
|
||||
// 2. effect: the billing plugin must now PRICE the source (its published
|
||||
// prices table has a non-zero entry), because a rule that is stored but
|
||||
// never injected prices nothing at all.
|
||||
func TestBillingRulesPersistAndReachThePlugin(t *testing.T) {
|
||||
g := rulesGateway(t)
|
||||
cfg := g.core.Config()
|
||||
|
||||
seed := `{"billing":{"active":"p","profiles":[{"id":"p","rules":[
|
||||
{"url":"https://free.example.com/v1","mode":"free"},
|
||||
{"url":"https://api.commandcode.ai/v1","mode":"token",
|
||||
"models":{"deepseek-v4.1-flash":{"prompt":"0.15","completion":"0.60"}}}]}]}}`
|
||||
if rr := rulesDo(t, g, http.MethodPut, seed); rr.Code != http.StatusOK {
|
||||
t.Fatalf("seed = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
|
||||
// 1. persisted to the file the process loaded from.
|
||||
raw, err := os.ReadFile(cfg.Path)
|
||||
if err != nil {
|
||||
t.Fatalf("read config: %v", err)
|
||||
}
|
||||
if !strings.Contains(string(raw), "commandcode.ai") {
|
||||
t.Errorf("★ the rule is not in config.yaml on disk — an edit that does not "+
|
||||
"survive a restart:\n%s", raw)
|
||||
}
|
||||
reloaded, err := config.Load(cfg.Path)
|
||||
if err != nil {
|
||||
t.Fatalf("reload saved config: %v", err)
|
||||
}
|
||||
if reloaded.BillingDSL == nil || len(reloaded.BillingDSL.Profiles) == 0 {
|
||||
t.Fatalf("saved config has no billing DSL: %s", raw)
|
||||
}
|
||||
// A round-trip through YAML must not lose the price strings.
|
||||
found := false
|
||||
for _, r := range reloaded.BillingDSL.Profiles[0].Rules {
|
||||
if strings.Contains(r.URL, "commandcode") {
|
||||
found = true
|
||||
if m, ok := r.Models["deepseek-v4.1-flash"]; !ok || m.Prompt != "0.15" {
|
||||
t.Errorf("price did not survive the YAML round-trip: %+v", m)
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Errorf("the token rule is missing after reload:\n%s", raw)
|
||||
}
|
||||
|
||||
// 2. reached the plugin. Prices live in a separate Lua field from `state`,
|
||||
// and the GET /state endpoint returns the published state together with
|
||||
// the prices sidecar — which is exactly what the plugin's own UI reads.
|
||||
state := g.core.Plugins().State("billing")
|
||||
if state == nil {
|
||||
t.Fatalf("billing plugin published no state")
|
||||
}
|
||||
rr := doReq(t, g, http.MethodGet, "/api/plugins/billing/state", "")
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("GET state = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var envelope struct {
|
||||
Prices map[string]interface{} `json:"prices"`
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &envelope); err != nil {
|
||||
t.Fatalf("decode state envelope: %v", err)
|
||||
}
|
||||
prices := envelope.Prices
|
||||
if prices == nil {
|
||||
t.Fatalf("state payload carries no prices")
|
||||
}
|
||||
sources, ok := prices["sources"].(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("prices has no sources map: %#v", prices)
|
||||
}
|
||||
cc, ok := sources["commandcode"]
|
||||
if !ok {
|
||||
t.Fatalf("★ the rule was saved but never injected: prices has %v, "+
|
||||
"no commandcode entry. Every request on it stays unpriced.", keysOf(sources))
|
||||
}
|
||||
if cc == nil {
|
||||
t.Error("commandcode price entry is nil")
|
||||
}
|
||||
// The free rule must price localzen, and the two must not be conflated.
|
||||
if _, ok := sources["localzen"]; !ok {
|
||||
t.Errorf("the free rule did not reach the plugin: %v", keysOf(sources))
|
||||
}
|
||||
// The injected price must be per-token, i.e. 0.15 USD/M => 1.5e-7.
|
||||
// Asserting the exact magnitude catches a unit error (per-million vs
|
||||
// per-token), which would be off by a factor of a million and still look
|
||||
// like "a number".
|
||||
models, ok := cc.(map[string]interface{})["models"].(map[string]interface{})
|
||||
if ok {
|
||||
if m, ok := models["deepseek-v4.1-flash"].(map[string]interface{}); ok {
|
||||
p, _ := m["prompt"].(float64)
|
||||
if p <= 0 || p > 1e-6 {
|
||||
t.Errorf("injected prompt price = %v, want per-token (~1.5e-7); "+
|
||||
"a per-million value here would over-bill by 1e6x", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func keysOf(m map[string]interface{}) []string {
|
||||
out := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
out = append(out, k)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// TestBillingRulesRequireAdmin keeps price rewriting behind the admin role,
|
||||
// like every other plugin write: prices are configuration, not user data.
|
||||
func TestBillingRulesRequireAdmin(t *testing.T) {
|
||||
g := rulesGateway(t)
|
||||
rec, err := g.core.CreateKey("viewer", "user", nil, "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
userKey := rec.Key
|
||||
for _, m := range []string{http.MethodPut, http.MethodPost, http.MethodDelete} {
|
||||
req, _ := http.NewRequest(m, "/api/plugins/billing/rules", strings.NewReader(`{}`))
|
||||
req.Header.Set("Authorization", "Bearer "+userKey)
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
if rr := newRecorderFor(t, g, req); rr.Code != http.StatusForbidden {
|
||||
t.Errorf("user %s rules = %d, want 403", m, rr.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -8,13 +8,11 @@ import (
|
||||
"log"
|
||||
"net/http"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/lua"
|
||||
"llmsproxy/internal/provider"
|
||||
"llmsproxy/internal/scheduler"
|
||||
"llmsproxy/internal/types"
|
||||
@ -31,9 +29,6 @@ type chatRequest struct {
|
||||
ToolChoice interface{} `json:"tool_choice,omitempty"`
|
||||
DisableThinking bool `json:"disable_thinking"`
|
||||
ExtraBody map[string]interface{} `json:"extra_body,omitempty"`
|
||||
|
||||
// PromptCacheKey 是 OpenAI 原生的缓存键；部分客户端用它携带会话标识。
|
||||
PromptCacheKey string `json:"prompt_cache_key,omitempty"`
|
||||
}
|
||||
|
||||
// ChatCompletion is the non-streaming OpenAI response object.
|
||||
@ -44,10 +39,6 @@ type ChatCompletion struct {
|
||||
Model string `json:"model"`
|
||||
Choices []ChatChoice `json:"choices"`
|
||||
Usage *types.TokenUsage `json:"usage,omitempty"`
|
||||
// Cost is the upstream-reported charge for this request, passed through
|
||||
// verbatim (OpenCode reports it as a decimal string, always "0" on the
|
||||
// flat-rate Go subscription). Absent when the upstream reports nothing.
|
||||
Cost string `json:"cost,omitempty"`
|
||||
}
|
||||
|
||||
type ChatChoice struct {
|
||||
@ -72,10 +63,6 @@ type ChatChunk struct {
|
||||
// Usage is sent in the final chunk of a stream (empty choices) so
|
||||
// OpenAI-compatible clients can read token usage.
|
||||
Usage *types.TokenUsage `json:"usage,omitempty"`
|
||||
// Cost is the upstream-reported charge for this request, passed through
|
||||
// verbatim (OpenCode reports it as a decimal string). Emitted on the
|
||||
// terminal chunk, mirroring OpenCode's own {"choices":[],"cost":"0"}.
|
||||
Cost string `json:"cost,omitempty"`
|
||||
}
|
||||
|
||||
type ChunkChoice struct {
|
||||
@ -198,95 +185,35 @@ func intersectModels(models []string, allow []config.ModelScope) []string {
|
||||
// A scope entry with model "AUTO" only allows requests where the effective
|
||||
// model is AUTO (the routing mode). It does NOT grant access to specific model
|
||||
// ids — that requires an explicit scope entry for the model.
|
||||
// checkModelScope keeps its string-returning signature for callers that only
|
||||
// need to know whether the request may proceed.
|
||||
func (g *Gateway) checkModelScope(ctx context.Context, model string) string {
|
||||
if q := g.checkQuota(ctx, model); q != nil {
|
||||
return q.msg
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// quotaRejection is a quota verdict: the message plus whether the client
|
||||
// should retry. A spent quota is a rate limit (429 + Retry-After), not a
|
||||
// permission failure (403): a client that sees 403 gives up on the key, while
|
||||
// one that sees 429 with a retry hint waits and resumes when the window rolls
|
||||
// over.
|
||||
type quotaRejection struct {
|
||||
msg string
|
||||
retry int64 // seconds until the window resets; 0 = unknown
|
||||
}
|
||||
|
||||
func (q *quotaRejection) Error() string { return q.msg }
|
||||
|
||||
// checkQuota validates the effective model against the key's model scope and
|
||||
// that entry's quota. Returns nil when the request may proceed.
|
||||
//
|
||||
// Quotas are per scope entry, never key-wide: a model whose budget is spent
|
||||
// is refused on its own while the key's other models keep working. The verdict
|
||||
// carries the remaining seconds of the reset window so a spent budget answers
|
||||
// 429 + Retry-After (come back when it rolls over) instead of 403 (which reads
|
||||
// as "this key may never use this model" and makes clients give up).
|
||||
func (g *Gateway) checkQuota(ctx context.Context, model string) *quotaRejection {
|
||||
allow := g.allowedModels(ctx)
|
||||
if allow == nil {
|
||||
return nil
|
||||
return ""
|
||||
}
|
||||
for _, sc := range allow {
|
||||
if sc.Model != model {
|
||||
continue
|
||||
}
|
||||
win := AutoPeriodSeconds(sc.Period, sc.Hours)
|
||||
k := keyID(reqKey(ctx))
|
||||
if sc.TokenQuota > 0 {
|
||||
if used := g.scopeTokens(ctx, sc); used >= sc.TokenQuota {
|
||||
return "aRejection{
|
||||
msg: fmt.Sprintf("token quota exceeded for %q (%d/%d)", model, used, sc.TokenQuota),
|
||||
retry: AutoSecondsToReset(sc.Period, sc.Hours),
|
||||
}
|
||||
used := g.scopeTokens(ctx, sc)
|
||||
if used >= sc.TokenQuota {
|
||||
return fmt.Sprintf("token quota exceeded for %q (%d/%d)", model, used, sc.TokenQuota)
|
||||
}
|
||||
}
|
||||
if sc.ReqQuota > 0 {
|
||||
if used := g.stats.KeyWindowReqs(k, win); used >= sc.ReqQuota {
|
||||
return "aRejection{
|
||||
msg: fmt.Sprintf("request quota exceeded for %q (%d/%d)", model, used, sc.ReqQuota),
|
||||
retry: AutoSecondsToReset(sc.Period, sc.Hours),
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
return ""
|
||||
}
|
||||
return "aRejection{msg: fmt.Sprintf("model %q is not allowed for this key", model)}
|
||||
return fmt.Sprintf("model %q is not allowed for this key", model)
|
||||
}
|
||||
|
||||
// quotaWindowSuffix describes a quota's reset window for an error message, so
|
||||
// a rejected caller can tell a permanent block from one that clears in an hour.
|
||||
func quotaWindowSuffix(period string, hours int64) string {
|
||||
switch {
|
||||
case period == "hour":
|
||||
return ", resets hourly"
|
||||
case period == "week":
|
||||
return ", resets weekly"
|
||||
case period == "month":
|
||||
return ", resets monthly"
|
||||
case period == "nhour" && hours > 1:
|
||||
return fmt.Sprintf(", resets every %dh", hours)
|
||||
}
|
||||
return ""
|
||||
} // scopeTokens returns the tokens this key consumed on the scope entry's model
|
||||
// within its reset window, isolated per key. For an AUTO entry the cap covers
|
||||
// everything the key routed through AUTO; for a model entry it covers that
|
||||
// model only.
|
||||
//
|
||||
// It reads the per-key hourly buckets rather than the key-blind model
|
||||
// buckets, so one key's usage can never exhaust another's quota.
|
||||
// scopeTokens returns the tokens a scope entry has consumed within its reset
|
||||
// window (total for AUTO / per model otherwise).
|
||||
func (g *Gateway) scopeTokens(ctx context.Context, sc config.ModelScope) int64 {
|
||||
k := keyID(reqKey(ctx))
|
||||
win := AutoPeriodSeconds(sc.Period, sc.Hours)
|
||||
if sc.Model != "" && strings.EqualFold(sc.Model, "AUTO") {
|
||||
return g.stats.KeyWindowTokens(k, win)
|
||||
return g.stats.KeyTokens(k)
|
||||
}
|
||||
return g.stats.KeyWindowModelTokens(k, sc.Model, sc.Source, win)
|
||||
win := AutoPeriodSeconds(sc.Period, sc.Hours)
|
||||
return g.stats.WindowTokens(sc.Model, sc.Source, win)
|
||||
}
|
||||
|
||||
// hasScopeModel reports whether a model (possibly with a "source-model" /
|
||||
@ -310,26 +237,6 @@ func (g *Gateway) hasScopeModel(list []config.ModelScope, s string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// writeScopeReject answers a model-scope or quota rejection. A spent quota is
|
||||
// 429 (rate_limit_exceeded) with Retry-After, so a client waits and resumes
|
||||
// after the reset; a model the key may not use stays 403 (model_not_allowed),
|
||||
// because retrying cannot help.
|
||||
func (g *Gateway) writeReject(w http.ResponseWriter, q *quotaRejection) {
|
||||
if q.retry > 0 {
|
||||
w.Header().Set("Retry-After", strconv.FormatInt(q.retry, 10))
|
||||
writeError(w, http.StatusTooManyRequests, "rate_limit_exceeded", q.msg)
|
||||
return
|
||||
}
|
||||
// no window to wait for: the cap is either permanent or key-wide with no
|
||||
// period. "rate_limit_exceeded" still says "come back after the operator
|
||||
// raises the cap", which 403 would not.
|
||||
if strings.Contains(q.msg, "quota exceeded") {
|
||||
writeError(w, http.StatusTooManyRequests, "rate_limit_exceeded", q.msg)
|
||||
return
|
||||
}
|
||||
writeError(w, http.StatusForbidden, "model_not_allowed", q.msg)
|
||||
}
|
||||
|
||||
func (g *Gateway) resolveByModel(model string) ([]*provider.Provider, string) {
|
||||
if isAuto(model) {
|
||||
return g.core.Registry().Resolve("AUTO"), ""
|
||||
@ -376,8 +283,6 @@ func (g *Gateway) handleChat(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
|
||||
return
|
||||
}
|
||||
// 客户端自带的会话标识（若有）比推导出来的更准，见 clientSessionFromRequest。
|
||||
clientSession := clientSessionFromRequest(r, req.PromptCacheKey)
|
||||
if len(req.Messages) == 0 {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "messages is required")
|
||||
return
|
||||
@ -386,28 +291,14 @@ func (g *Gateway) handleChat(w http.ResponseWriter, r *http.Request) {
|
||||
if model == "" {
|
||||
model = g.core.DefaultModel()
|
||||
}
|
||||
// request_start fires for EVERY chat request, on both the AUTO and the
|
||||
// direct path, and it fires BEFORE the quota / model-scope gates on
|
||||
// purpose: a plugin that counts volume or audits traffic must also see the
|
||||
// requests the gateway rejected, otherwise "requests accepted" would be all
|
||||
// it could ever report. It sits after authentication (so the key and role in
|
||||
// the payload are real) and after the messages check (a body with no
|
||||
// messages is not a chat request at all).
|
||||
//
|
||||
// Calling it here rather than inside each branch is what keeps the two paths
|
||||
// honest: an earlier version called it only from the AUTO branch, so every
|
||||
// direct (model-pinned) request silently skipped it. That was caught by
|
||||
// TestHooksFireOnRealDirectChat, not by reading the code.
|
||||
g.fireStart(r.Context(), &req, "chat", model, len(req.Messages), len(req.Tools))
|
||||
if isAuto(model) {
|
||||
// The key's own chain wins; keys without one inherit the global chain.
|
||||
chain, _ := g.core.AutoChainFor(reqKey(r.Context()))
|
||||
chain := g.core.AutoChain()
|
||||
if chain == nil || len(chain.Tiers) == 0 {
|
||||
writeError(w, http.StatusServiceUnavailable, "no_provider", "no auto slot configured")
|
||||
return
|
||||
}
|
||||
if q := g.checkQuota(r.Context(), "AUTO"); q != nil {
|
||||
g.writeReject(w, q)
|
||||
if msg := g.checkModelScope(r.Context(), "AUTO"); msg != "" {
|
||||
writeError(w, http.StatusForbidden, "model_not_allowed", msg)
|
||||
return
|
||||
}
|
||||
ctx := r.Context()
|
||||
@ -422,7 +313,6 @@ func (g *Gateway) handleChat(w http.ResponseWriter, r *http.Request) {
|
||||
ToolChoice: req.ToolChoice,
|
||||
DisableThinking: req.DisableThinking,
|
||||
ExtraBody: req.ExtraBody,
|
||||
ClientSession: clientSession,
|
||||
}
|
||||
rec := &Req{
|
||||
Key: keyID(reqKey(ctx)),
|
||||
@ -460,8 +350,8 @@ func (g *Gateway) handleChat(w http.ResponseWriter, r *http.Request) {
|
||||
if effective == "" {
|
||||
effective = firstModel(cands[0])
|
||||
}
|
||||
if q := g.checkQuota(r.Context(), effective); q != nil {
|
||||
g.writeReject(w, q)
|
||||
if msg := g.checkModelScope(r.Context(), effective); msg != "" {
|
||||
writeError(w, http.StatusForbidden, "model_not_allowed", msg)
|
||||
return
|
||||
}
|
||||
ctx := r.Context()
|
||||
@ -478,7 +368,6 @@ func (g *Gateway) handleChat(w http.ResponseWriter, r *http.Request) {
|
||||
ToolChoice: req.ToolChoice,
|
||||
DisableThinking: req.DisableThinking,
|
||||
ExtraBody: req.ExtraBody,
|
||||
ClientSession: clientSession,
|
||||
}
|
||||
rec := &Req{
|
||||
Key: keyID(reqKey(ctx)),
|
||||
@ -645,9 +534,6 @@ var overflowMarkers = []*regexp.Regexp{
|
||||
}
|
||||
|
||||
// overflowCanonical 命中 pi 的 /context[_ ]length[_ ]exceeded/i。
|
||||
//
|
||||
// 注意：这个前缀只是必要条件，不是充分条件——pi 还会先用
|
||||
// NON_OVERFLOW_PATTERNS 排除整条消息，见 overflowClientMessage。
|
||||
const overflowCanonical = "context_length_exceeded"
|
||||
|
||||
// looksLikeOverflow 判断错误文本是否属于上下文超窗。
|
||||
@ -666,59 +552,14 @@ func looksLikeOverflow(s string) bool {
|
||||
// Per-tier one-line reasons are kept (quota/cooling skips carry no error body
|
||||
// and are the actionable part); each is capped so HTML dumps can't leak.
|
||||
//
|
||||
// 超窗会被归一化成「干净」的 overflowCanonical 消息，见 overflowClientMessage。
|
||||
// 超窗会被归一化成 overflowCanonical 前缀，见 overflowMarkers 的说明。
|
||||
func clientUpstreamErr(err error) string {
|
||||
log.Printf("[gateway] upstream failure surfaced to client: %v", err)
|
||||
if looksLikeOverflow(err.Error()) {
|
||||
return overflowClientMessage(err)
|
||||
msg := upstreamErrSummary(err)
|
||||
if looksLikeOverflow(err.Error()) || looksLikeOverflow(msg) {
|
||||
return fmt.Sprintf("%s: %s", overflowCanonical, msg)
|
||||
}
|
||||
return upstreamErrSummary(err)
|
||||
}
|
||||
|
||||
// overflowHint 是给客户端的超窗说明，刻意只用 pi 认得的措辞。
|
||||
const overflowHint = "context window is full; reduce the length of the messages"
|
||||
|
||||
// clientSessionFromRequest 取客户端自带的会话标识，拿不到时返回 ""。
|
||||
//
|
||||
// 通用 OpenAI 客户端默认不带会话 id，但 pi 支持：只要 provider 的 compat 里打开
|
||||
// sendSessionAffinityHeaders，pi 就会把**平台真实的会话 id**（uuidv7，整个会话
|
||||
// 恒定）放到 x-session-affinity / x-client-request-id 上（sessionAffinityFormat
|
||||
// 为 openrouter 时是 x-session-id，为 openai 时是 session_id）。
|
||||
// 有了它就不必再从首条 user 消息推导会话指纹（那只在客户端不发会话 id 时才作为
|
||||
// 退路，且历史压缩后会漂移）。prompt_cache_key 是 OpenAI 原生的缓存键，
|
||||
// 客户端若带也一并当会话用。
|
||||
//
|
||||
// 刻意不采纳 x-client-request-id：名字含 request，部分客户端每请求都换，
|
||||
// 拿它当会话会让上游缓存永不命中。pi 总会同时发 x-session-affinity，够用。
|
||||
func clientSessionFromRequest(r *http.Request, bodyKey string) string {
|
||||
for _, h := range []string{"X-Session-Affinity", "X-Session-Id", "Session-Id"} {
|
||||
if v := strings.TrimSpace(r.Header.Get(h)); v != "" {
|
||||
return v
|
||||
}
|
||||
}
|
||||
return strings.TrimSpace(bodyKey)
|
||||
}
|
||||
|
||||
// overflowClientMessage 为超窗失败生成「干净」的客户端消息。
|
||||
//
|
||||
// 为什么不能沿用 upstreamErrSummary：pi 的 isContextOverflow 先查
|
||||
// NON_OVERFLOW_PATTERNS（/rate limit/、/too many requests/、Bedrock 前缀），
|
||||
// 一旦命中就直接判为「非超窗」——**哪怕消息里已经有 context_length_exceeded**，
|
||||
// pi 也不会压缩重试。而 AUTO 链的失败消息天生是多 tier 原因的拼接：
|
||||
// 超窗 tier（gozen 400 maximum context length）常与配额/限流 tier
|
||||
// （429 token plan exhausted、cooling、no free slot）同时出现。
|
||||
// 把明细原样带出去，等于让一条限流 tier 的措辞反过来封杀超窗识别。
|
||||
// 所以这里只保留「超窗」措辞 + 是哪个源超的窗，其余一律不带。
|
||||
func overflowClientMessage(err error) string {
|
||||
var ce *scheduler.ChainErr
|
||||
if errors.As(err, &ce) {
|
||||
for _, t := range ce.Tiers {
|
||||
if looksLikeOverflow(t.Err.Error()) {
|
||||
return fmt.Sprintf("%s: %s (%s/%s)", overflowCanonical, overflowHint, t.Source, t.Model)
|
||||
}
|
||||
}
|
||||
}
|
||||
return fmt.Sprintf("%s: %s", overflowCanonical, overflowHint)
|
||||
return msg
|
||||
}
|
||||
|
||||
// upstreamErrSummary 把上游失败压成一行短消息。
|
||||
@ -807,7 +648,6 @@ func writeChatCompletion(w http.ResponseWriter, resp *types.UnifiedResponse, mod
|
||||
Created: time.Now().Unix(),
|
||||
Model: modelName,
|
||||
Choices: []ChatChoice{{Index: 0, Message: msg, FinishReason: resp.FinishReason}},
|
||||
Cost: resp.Cost,
|
||||
}
|
||||
if resp.TokenUsage.Total > 0 || resp.TokenUsage.Prompt > 0 || resp.TokenUsage.Completion > 0 {
|
||||
out.Usage = &resp.TokenUsage
|
||||
@ -832,7 +672,6 @@ func (g *Gateway) singleChat(w http.ResponseWriter, ctx context.Context, cands [
|
||||
recordChatUsage(rec, req, resp)
|
||||
rec.Source = usedSrc
|
||||
rec.Model = usedModel
|
||||
g.fireRouted(ctx, "chat", usedSrc, usedModel, -1, false)
|
||||
// Non-streaming: the whole response arrives at once, so TTFB equals
|
||||
// the total latency.
|
||||
rec.FirstByteMs = rec.LatMs
|
||||
@ -840,18 +679,7 @@ func (g *Gateway) singleChat(w http.ResponseWriter, ctx context.Context, cands [
|
||||
writeChatCompletion(w, resp, effective)
|
||||
}
|
||||
|
||||
// writeRec records a finished request (audit + aggregates) and fires the
|
||||
// plugin request_end stage.
|
||||
//
|
||||
// This is the ONE place every request passes through on its way out, which is
|
||||
// what makes it the right hook point: the four entry points (single/stream ×
|
||||
// direct/auto) all funnel here, so a plugin sees each request exactly once with
|
||||
// its final accounting. Firing earlier would miss the streamed ones (their
|
||||
// numbers are only known once the stream finishes), and firing in each entry
|
||||
// point would mean four call sites to keep in sync.
|
||||
//
|
||||
// Hooks run AFTER the record is written: a plugin must not be able to delay or
|
||||
// lose the audit trail, and a plugin that throws is contained by Fire.
|
||||
// writeRec records a finished request (audit + aggregates).
|
||||
func (g *Gateway) writeRec(rec *Req) {
|
||||
if rec == nil {
|
||||
return
|
||||
@ -860,191 +688,6 @@ func (g *Gateway) writeRec(rec *Req) {
|
||||
rec.Time = time.Now().UnixMilli()
|
||||
}
|
||||
g.stats.Record(*rec)
|
||||
g.fireEnd(rec)
|
||||
}
|
||||
|
||||
// fireStart dispatches the plugin request_start stage: the request has been
|
||||
// parsed and authorized but no upstream slot has been chosen yet, so `source`
|
||||
// is empty. A plugin that only wants volume/acceptance counts can subscribe
|
||||
// here and stay out of the per-request hot path entirely.
|
||||
func (g *Gateway) fireStart(ctx context.Context, req *chatRequest, kind, model string, msgs, tools int) {
|
||||
ps := g.core.Plugins()
|
||||
if ps == nil || ps.Count() == 0 {
|
||||
return
|
||||
}
|
||||
ps.Fire(lua.StageRequestStart, map[string]interface{}{
|
||||
"stage": string(lua.StageRequestStart),
|
||||
"type": kind,
|
||||
"model": model,
|
||||
"key": keyID(reqKey(ctx)),
|
||||
"role": reqRole(ctx),
|
||||
"source": "",
|
||||
"stream": req.Stream,
|
||||
"messages_count": msgs,
|
||||
"tools_count": tools,
|
||||
"ts": time.Now().Unix(),
|
||||
})
|
||||
}
|
||||
|
||||
// fireImageStart dispatches request_start for /v1/images/generations.
|
||||
//
|
||||
// It is a separate function rather than a call to fireStart with a nil
|
||||
// chatRequest because the image body has no messages and no tools: passing
|
||||
// zeroes through a struct built for chat would invite someone to read a field
|
||||
// that simply does not exist on this path.
|
||||
func (g *Gateway) fireImageStart(ctx context.Context, model string) {
|
||||
ps := g.core.Plugins()
|
||||
if ps == nil || ps.Count() == 0 {
|
||||
return
|
||||
}
|
||||
ps.Fire(lua.StageRequestStart, map[string]interface{}{
|
||||
"stage": string(lua.StageRequestStart),
|
||||
"type": "image",
|
||||
"model": model,
|
||||
"key": keyID(reqKey(ctx)),
|
||||
"role": reqRole(ctx),
|
||||
"source": "",
|
||||
"stream": false,
|
||||
"messages_count": 0,
|
||||
"tools_count": 0,
|
||||
"ts": time.Now().Unix(),
|
||||
})
|
||||
}
|
||||
|
||||
// fireRouted dispatches the plugin routed stage once a (source, model) slot has
|
||||
// been selected. tier is the AUTO tier index, or -1 on the direct path, so a
|
||||
// plugin can tell "this came from tier 1" from "this bypassed the chain".
|
||||
func (g *Gateway) fireRouted(ctx context.Context, kind, source, model string, tier int, stream bool) {
|
||||
ps := g.core.Plugins()
|
||||
if ps == nil || ps.Count() == 0 {
|
||||
return
|
||||
}
|
||||
ps.Fire(lua.StageRouted, map[string]interface{}{
|
||||
"stage": string(lua.StageRouted),
|
||||
"type": kind,
|
||||
"source": source,
|
||||
"model": model,
|
||||
"key": keyID(reqKey(ctx)),
|
||||
"tier": tier,
|
||||
"stream": stream,
|
||||
"ts": time.Now().Unix(),
|
||||
})
|
||||
}
|
||||
|
||||
// chainTraceSink adapts a scheduler TraceSink into the plugin chain_step stage.
|
||||
//
|
||||
// It returns nil when no plugin is loaded, so the scheduler's emit() does a
|
||||
// single nil check per event and the AUTO hot path pays nothing on a gateway
|
||||
// with no plugins.
|
||||
//
|
||||
// The events are also accumulated into walk so request_end can carry a compact
|
||||
// summary: a plugin that only listens to request_end still learns that a
|
||||
// degradation happened, which is the common case for a dashboard that does not
|
||||
// want to subscribe to a high-frequency stage.
|
||||
func (g *Gateway) chainTraceSink(ctx context.Context, kind string, walk *[]map[string]interface{}) scheduler.TraceSink {
|
||||
ps := g.core.Plugins()
|
||||
if ps == nil || ps.Count() == 0 {
|
||||
return nil
|
||||
}
|
||||
key := keyID(reqKey(ctx))
|
||||
return func(ev scheduler.TraceEvent) {
|
||||
payload := map[string]interface{}{
|
||||
"stage": string(lua.StageChainStep),
|
||||
"kind": string(ev.Kind),
|
||||
"type": kind,
|
||||
"key": key,
|
||||
"tier": ev.Tier,
|
||||
"attempt": ev.Attempt,
|
||||
}
|
||||
if ev.Source != "" {
|
||||
payload["source"] = ev.Source
|
||||
}
|
||||
if ev.Model != "" {
|
||||
payload["model"] = ev.Model
|
||||
}
|
||||
if ev.Reason != "" {
|
||||
payload["reason"] = ev.Reason
|
||||
}
|
||||
if ev.Err != "" {
|
||||
payload["error"] = ev.Err
|
||||
}
|
||||
if walk != nil {
|
||||
// Keep the summary bounded: a pathological chain could emit many
|
||||
// steps, and request_end's payload is written to the audit trail.
|
||||
if len(*walk) < maxWalkSummary {
|
||||
*walk = append(*walk, map[string]interface{}{
|
||||
"kind": string(ev.Kind), "tier": ev.Tier,
|
||||
"source": ev.Source, "model": ev.Model, "reason": ev.Reason,
|
||||
})
|
||||
}
|
||||
}
|
||||
ps.Fire(lua.StageChainStep, payload)
|
||||
}
|
||||
}
|
||||
|
||||
// maxWalkSummary caps how many chain steps request_end carries, so a long
|
||||
// degradation cannot inflate every audit record.
|
||||
const maxWalkSummary = 12
|
||||
|
||||
// tierServed returns the AUTO tier that actually served the request, or -1 when
|
||||
// the walk is empty (a direct request) or ended without a selection (total
|
||||
// failure). It is the single most useful number for "why did my expensive tier
|
||||
// not get used".
|
||||
func tierServed(walk []map[string]interface{}) int {
|
||||
for i := len(walk) - 1; i >= 0; i-- {
|
||||
if k, _ := walk[i]["kind"].(string); k == string(scheduler.TraceSelected) {
|
||||
if t, ok := walk[i]["tier"].(int); ok {
|
||||
return t
|
||||
}
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
// fireEnd dispatches the plugin request_end stage for one finished request.
|
||||
func (g *Gateway) fireEnd(rec *Req) {
|
||||
ps := g.core.Plugins()
|
||||
if ps == nil || ps.Count() == 0 {
|
||||
return
|
||||
}
|
||||
payload := map[string]interface{}{
|
||||
"stage": string(lua.StageRequestEnd),
|
||||
"type": rec.Type,
|
||||
"model": rec.Model,
|
||||
"source": rec.Source,
|
||||
"key": rec.Key,
|
||||
"ok": rec.OK,
|
||||
"status": rec.Status,
|
||||
"latency_ms": rec.LatMs,
|
||||
"first_byte_ms": rec.FirstByteMs,
|
||||
"prompt_tokens": rec.Prompt,
|
||||
"completion_tokens": rec.Compl,
|
||||
"cache_hit_tokens": rec.CacheHit,
|
||||
"cache_miss_tokens": rec.CacheMiss,
|
||||
// Whether UPSTREAM reported a cache number at all. A plugin cannot
|
||||
// infer this from cache_hit_tokens alone: zero hits because nothing was
|
||||
// cached and zero hits because the provider never reports caching are
|
||||
// the same value, and they mean opposite things when you are checking
|
||||
// whether a cache discount is doing anything. The audit record already
|
||||
// carried this (rec.CacheReported); the plugin payload did not, so any
|
||||
// plugin-level cache accounting had to guess.
|
||||
"cache_reported": rec.CacheReported,
|
||||
"image_count": rec.ImageCount,
|
||||
"error": rec.Err,
|
||||
"time": rec.Time,
|
||||
// chain_walk: the AUTO tier-by-tier trace, when the request went
|
||||
// through the chain. Empty for a direct request and for a gateway with
|
||||
// no plugins loaded. Absent rather than empty so a plugin can tell
|
||||
// "no chain" from "chain with no degradation".
|
||||
"degraded": len(rec.Walk) > 1,
|
||||
"chain_walk": rec.Walk,
|
||||
"tier_served": tierServed(rec.Walk),
|
||||
}
|
||||
// The merged result is intentionally discarded: request_end is the last
|
||||
// stage, so there is nobody downstream to read a plugin's additions. Plugins
|
||||
// that need to publish derived numbers (the billing plugin) do it in their
|
||||
// OWN state and expose them through the /api/plugins/<name>/state endpoint.
|
||||
ps.Fire(lua.StageRequestEnd, payload)
|
||||
}
|
||||
|
||||
// mergeUsage combines token usage across stream chunks additively. Some
|
||||
@ -1124,15 +767,11 @@ func (g *Gateway) pumpStream(w http.ResponseWriter, rec *Req, chunks <-chan type
|
||||
// source's status-page latency average.
|
||||
rec.FirstByteMs = time.Since(t0).Milliseconds()
|
||||
var lastUsage *types.TokenUsage
|
||||
var lastCost string
|
||||
lastFinish := ""
|
||||
for ck := range chunks {
|
||||
if ck.Usage != nil {
|
||||
lastUsage = mergeUsage(lastUsage, ck.Usage)
|
||||
}
|
||||
if ck.Cost != "" {
|
||||
lastCost = ck.Cost
|
||||
}
|
||||
chunk := ChatChunk{
|
||||
ID: id, Object: "chat.completion.chunk", Created: created, Model: modelName,
|
||||
}
|
||||
@ -1169,19 +808,6 @@ func (g *Gateway) pumpStream(w http.ResponseWriter, rec *Req, chunks <-chan type
|
||||
// Final usage chunk (OpenAI standard: empty choices + usage before [DONE]).
|
||||
// Prefer the upstream's exact usage if the stream carried it; fall back to
|
||||
// the gateway's estimate otherwise.
|
||||
// Write the upstream's exact numbers back onto the audit record. Without
|
||||
// this the streamed path kept only the per-chunk byte estimate, so the same
|
||||
// request recorded ~1.4x its real completion tokens (measured: upstream 100,
|
||||
// audit 145) while the non-streaming path recorded 100. Two paths, two
|
||||
// different numbers for one request is a reporting bug, not a rounding one.
|
||||
if lastUsage != nil {
|
||||
if lastUsage.Prompt > 0 {
|
||||
rec.Prompt = int64(lastUsage.Prompt)
|
||||
}
|
||||
if lastUsage.Completion > 0 {
|
||||
rec.Compl = int64(lastUsage.Completion)
|
||||
}
|
||||
}
|
||||
var tut *types.TokenUsage
|
||||
if lastUsage != nil {
|
||||
tut = lastUsage
|
||||
@ -1212,14 +838,6 @@ func (g *Gateway) pumpStream(w http.ResponseWriter, rec *Req, chunks <-chan type
|
||||
ID: id, Object: "chat.completion.chunk", Created: created, Model: modelName,
|
||||
Choices: []ChunkChoice{},
|
||||
Usage: tut,
|
||||
Cost: lastCost,
|
||||
})
|
||||
} else if lastCost != "" {
|
||||
// Cost arrived without any usage (OpenCode sends it on its own frame).
|
||||
send(ChatChunk{
|
||||
ID: id, Object: "chat.completion.chunk", Created: created, Model: modelName,
|
||||
Choices: []ChunkChoice{},
|
||||
Cost: lastCost,
|
||||
})
|
||||
}
|
||||
fmt.Fprintf(w, "data: [DONE]\n\n")
|
||||
@ -1251,7 +869,6 @@ func (g *Gateway) streamChat(w http.ResponseWriter, ctx context.Context, cands [
|
||||
// failover it differs from the first candidate). Direct streams previously
|
||||
// discarded it.
|
||||
rec.Source = usedSrc
|
||||
g.fireRouted(ctx, "stream", usedSrc, rec.Model, -1, true)
|
||||
rec.Prompt = estimatePromptTokens(req)
|
||||
g.pumpStream(w, rec, chunks, effective, t0)
|
||||
}
|
||||
@ -1265,11 +882,8 @@ func (g *Gateway) streamChat(w http.ResponseWriter, ctx context.Context, cands [
|
||||
func (g *Gateway) singleChatAuto(w http.ResponseWriter, ctx context.Context, chain *scheduler.Chain, req *types.ChatRequest, rec *Req, quotaExhausted func(*scheduler.Slot) bool) {
|
||||
rec.LatMs = 0
|
||||
t0 := time.Now()
|
||||
var walk []map[string]interface{}
|
||||
resp, usedSrc, usedModel, err := g.core.Scheduler().ChainChat(ctx, chain, req, quotaExhausted,
|
||||
g.chainTraceSink(ctx, "chat", &walk))
|
||||
resp, usedSrc, usedModel, err := g.core.Scheduler().ChainChat(ctx, chain, req, quotaExhausted)
|
||||
rec.LatMs = time.Since(t0).Milliseconds()
|
||||
rec.Walk = walk
|
||||
if err != nil {
|
||||
g.failChat(w, rec, err)
|
||||
g.writeRec(rec)
|
||||
@ -1280,10 +894,6 @@ func (g *Gateway) singleChatAuto(w http.ResponseWriter, ctx context.Context, cha
|
||||
recordChatUsage(rec, req, resp)
|
||||
rec.Source = usedSrc
|
||||
rec.Model = usedModel
|
||||
// AUTO has no single tier to report: the chain may have walked several
|
||||
// before this slot served the request, so -2 means "resolved by the chain"
|
||||
// and a plugin can tell that apart from the direct path's -1.
|
||||
g.fireRouted(ctx, "chat", usedSrc, usedModel, -2, false)
|
||||
rec.FirstByteMs = rec.LatMs
|
||||
g.writeRec(rec)
|
||||
writeChatCompletion(w, resp, usedModel)
|
||||
@ -1302,10 +912,7 @@ func (g *Gateway) streamChatAuto(w http.ResponseWriter, ctx context.Context, cha
|
||||
rec.LatMs = time.Since(t0).Milliseconds()
|
||||
g.writeRec(rec)
|
||||
}()
|
||||
var walk []map[string]interface{}
|
||||
chunks, usedSrc, usedModel, err := g.core.Scheduler().ChainChatStream(ctx, chain, req, quotaExhausted,
|
||||
g.chainTraceSink(ctx, "stream", &walk))
|
||||
rec.Walk = walk
|
||||
chunks, usedSrc, usedModel, err := g.core.Scheduler().ChainChatStream(ctx, chain, req, quotaExhausted)
|
||||
if err != nil {
|
||||
g.failChat(w, rec, err)
|
||||
return
|
||||
@ -1314,7 +921,6 @@ func (g *Gateway) streamChatAuto(w http.ResponseWriter, ctx context.Context, cha
|
||||
rec.Model = usedModel
|
||||
}
|
||||
rec.Source = usedSrc
|
||||
g.fireRouted(ctx, "stream", usedSrc, rec.Model, -2, true)
|
||||
rec.Prompt = estimatePromptTokens(req)
|
||||
g.pumpStream(w, rec, chunks, usedModel, t0)
|
||||
}
|
||||
@ -1337,15 +943,10 @@ func (g *Gateway) handleImage(w http.ResponseWriter, r *http.Request) {
|
||||
if model == "" {
|
||||
model = g.core.DefaultModel()
|
||||
}
|
||||
// Same rule as the chat path, and for the same reason: an image request is
|
||||
// billable traffic, so a cost plugin must see it. It fires before the
|
||||
// quota/scope gates so rejected image requests are visible too.
|
||||
// messages_count/tools_count are 0: the image request has neither.
|
||||
g.fireImageStart(r.Context(), model)
|
||||
if isAuto(model) {
|
||||
if chain := g.core.AutoImageChain(); chain != nil && len(chain.Tiers) > 0 {
|
||||
if q := g.checkQuota(r.Context(), "AUTO"); q != nil {
|
||||
g.writeReject(w, q)
|
||||
if msg := g.checkModelScope(r.Context(), "AUTO"); msg != "" {
|
||||
writeError(w, http.StatusForbidden, "model_not_allowed", msg)
|
||||
return
|
||||
}
|
||||
done := g.stats.Begin()
|
||||
@ -1365,13 +966,9 @@ func (g *Gateway) handleImage(w http.ResponseWriter, r *http.Request) {
|
||||
if usedModel != "" {
|
||||
rec.Model = usedModel // actual image model served, not "AUTO"
|
||||
}
|
||||
g.fireRouted(r.Context(), "image", usedSrc, rec.Model, -2, false)
|
||||
rec.OK = true
|
||||
rec.Status = http.StatusOK
|
||||
// Image generation has no token concept. Recording len(ImageData)
|
||||
// (the image COUNT) in completion_tokens mislabels image count as
|
||||
// tokens and feeds it into the token totals; leave it 0.
|
||||
rec.ImageCount = len(resp.ImageData)
|
||||
rec.Compl = int64(len(resp.ImageData))
|
||||
g.writeRec(rec)
|
||||
writeJSON(w, http.StatusOK, types.ImageGenResponse{
|
||||
Created: time.Now().Unix(),
|
||||
@ -1395,8 +992,8 @@ func (g *Gateway) handleImage(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, http.StatusServiceUnavailable, "no_provider", "no image source configured")
|
||||
return
|
||||
}
|
||||
if q := g.checkQuota(r.Context(), effectiveImageModel(model, cands)); q != nil {
|
||||
g.writeReject(w, q)
|
||||
if msg := g.checkModelScope(r.Context(), effectiveImageModel(model, cands)); msg != "" {
|
||||
writeError(w, http.StatusForbidden, "model_not_allowed", msg)
|
||||
return
|
||||
}
|
||||
done := g.stats.Begin()
|
||||
@ -1418,11 +1015,9 @@ func (g *Gateway) handleImage(w http.ResponseWriter, r *http.Request) {
|
||||
if resp.Model != "" {
|
||||
rec.Model = resp.Model // record the actual model served, not the raw request id
|
||||
}
|
||||
g.fireRouted(r.Context(), "image", rec.Source, rec.Model, -1, false)
|
||||
rec.OK = true
|
||||
rec.Status = http.StatusOK
|
||||
// Image generation has no token concept — see the AUTO path above.
|
||||
rec.ImageCount = len(resp.ImageData)
|
||||
rec.Compl = int64(len(resp.ImageData))
|
||||
g.writeRec(rec)
|
||||
writeJSON(w, http.StatusOK, types.ImageGenResponse{
|
||||
Created: time.Now().Unix(),
|
||||
|
||||
@ -1,47 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// 客户端自带的会话标识必须被采纳：它比「首条 user 消息指纹」更准
|
||||
// （历史压缩后指纹会漂移），也是唯一能天然按会话隔离的来源。
|
||||
func TestClientSessionFromRequestHeaders(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
headers map[string]string
|
||||
bodyKey string
|
||||
want string
|
||||
}{
|
||||
{"pi 默认格式 x-session-affinity", map[string]string{"X-Session-Affinity": "ses-1"}, "", "ses-1"},
|
||||
{"openrouter 格式 x-session-id", map[string]string{"X-Session-Id": "ses-2"}, "", "ses-2"},
|
||||
{"openai 格式 session_id", map[string]string{"Session-Id": "ses-3"}, "", "ses-3"},
|
||||
{"body prompt_cache_key 兜底", nil, "cache-4", "cache-4"},
|
||||
{"头优先于 body", map[string]string{"X-Session-Affinity": "ses-5"}, "cache-x", "ses-5"},
|
||||
{"空白不算", map[string]string{"X-Session-Affinity": " "}, "", ""},
|
||||
{"都没有则为空（适配器自行推导）", nil, "", ""},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
r, _ := http.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
|
||||
for k, v := range tc.headers {
|
||||
r.Header.Set(k, v)
|
||||
}
|
||||
if got := clientSessionFromRequest(r, tc.bodyKey); got != tc.want {
|
||||
t.Errorf("clientSessionFromRequest = %q, want %q", got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// 刻意不采纳 x-client-request-id：名字含 request，部分客户端每请求都换，
|
||||
// 拿它当会话会让上游前缀缓存永不命中。
|
||||
func TestClientRequestIDIsNotUsedAsSession(t *testing.T) {
|
||||
r, _ := http.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
|
||||
r.Header.Set("X-Client-Request-Id", "per-request-uuid")
|
||||
if got := clientSessionFromRequest(r, ""); got != "" {
|
||||
t.Errorf("x-client-request-id 不应被当作会话（否则缓存永不命中），got %q", got)
|
||||
}
|
||||
}
|
||||
@ -1,204 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// deploy.sh is the only release-critical script with no automated coverage, and
|
||||
// its newest part — prune_adapter_backups — deletes directories under
|
||||
// /etc/llmsproxy. This exercises the function in a throwaway directory with the
|
||||
// REAL backup-name shape.
|
||||
//
|
||||
// The shape matters and got this wrong twice: the guard is
|
||||
// `^adapters\.bak\.[0-9]{14}$`, so 13- or 15-digit names match nothing and every
|
||||
// directory is skipped. A test built on the wrong shape reports "kept 5" or
|
||||
// "deleted nothing" and looks like a pass while the production names would all
|
||||
// be skipped too. The names below are copied from the live directory listing.
|
||||
func TestDeployPruneAdapterBackups(t *testing.T) {
|
||||
if _, err := os.Stat("../../deploy.sh"); err != nil {
|
||||
t.Skip("deploy.sh not present (packaging-only checkout)")
|
||||
}
|
||||
if _, err := exec.LookPath("bash"); err != nil {
|
||||
t.Skip("bash not available")
|
||||
}
|
||||
|
||||
base := t.TempDir()
|
||||
fn, err := os.ReadFile("../../deploy.sh")
|
||||
if err != nil {
|
||||
t.Fatalf("read deploy.sh: %v", err)
|
||||
}
|
||||
src := string(fn)
|
||||
i := strings.Index(src, "prune_adapter_backups()")
|
||||
if i < 0 {
|
||||
t.Fatal("prune_adapter_backups not found in deploy.sh")
|
||||
}
|
||||
j := strings.Index(src[i:], "\n}\n")
|
||||
if j < 0 {
|
||||
t.Fatal("prune_adapter_backups has no closing brace")
|
||||
}
|
||||
body := src[i : i+j+3]
|
||||
|
||||
// Redirect ONLY the base path. The logic under test must stay verbatim.
|
||||
body = strings.Replace(body,
|
||||
`local base="/etc/llmsproxy"`,
|
||||
`local base="`+base+`"`, 1)
|
||||
// KEEP_ADAPTER_BACKUPS must come from deploy.sh itself, NOT from a literal
|
||||
// here: hardcoding 5 in the shim overrode whatever the script configured, so
|
||||
// a mutation that set KEEP_ADAPTER_BACKUPS=0 in deploy.sh still passed. The
|
||||
// value is part of what is under test.
|
||||
var keepRe = regexp.MustCompile(`(?m)^KEEP_ADAPTER_BACKUPS=(\d+)`)
|
||||
keepM := keepRe.FindStringSubmatch(src)
|
||||
if keepM == nil {
|
||||
t.Fatal("deploy.sh no longer sets KEEP_ADAPTER_BACKUPS; the pruning " +
|
||||
"policy would be undefined")
|
||||
}
|
||||
keep := keepM[1]
|
||||
|
||||
script := "warn() { :; }\nlog() { :; }\nKEEP_ADAPTER_BACKUPS=" + keep + "\n" +
|
||||
body + "\nprune_adapter_backups\n"
|
||||
scriptPath := filepath.Join(base, "fn.sh")
|
||||
if err := os.WriteFile(scriptPath, []byte(script), 0o600); err != nil {
|
||||
t.Fatalf("write script: %v", err)
|
||||
}
|
||||
|
||||
// 12 well-formed backups. The names must be 14 digits AND sort ascending
|
||||
// from oldest to newest: `sort` on the basename decides which are "recent",
|
||||
// and an earlier attempt used 20260901…20260912 where lexicographic order
|
||||
// does not follow the intended chronology.
|
||||
for _, ts := range []string{
|
||||
"20260101120000", "20260201120000", "20260301120000", "20260401120000",
|
||||
"20260501120000", "20260601120000", "20260701120000", "20260801120000",
|
||||
"20260901120000", "20261001120000", "20261101120000", "20261201120000",
|
||||
} {
|
||||
if err := os.MkdirAll(filepath.Join(base, "adapters.bak."+ts), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
// Names the guard must refuse to delete: a hand-made directory, a 15-digit
|
||||
// name, and an 11-digit one.
|
||||
for _, name := range []string{"manual", "202609011200000", "2026090112000"} {
|
||||
if err := os.MkdirAll(filepath.Join(base, "adapters.bak."+name), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
if out, err := exec.Command("bash", scriptPath).CombinedOutput(); err != nil {
|
||||
t.Fatalf("prune_adapter_backups failed: %v\n%s", err, out)
|
||||
}
|
||||
|
||||
entries, err := os.ReadDir(base)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := map[string]bool{}
|
||||
for _, e := range entries {
|
||||
if e.IsDir() {
|
||||
got[e.Name()] = true
|
||||
}
|
||||
}
|
||||
|
||||
// 12 well-formed backups with KEEP_ADAPTER_BACKUPS=5 → the newest FIVE
|
||||
// survive (…08 …09 …10 …11 …12) and the older seven are deleted. Listing
|
||||
// only three keepers made this assertion wrong in BOTH directions at first:
|
||||
// …09 was named as "should be deleted" while it is in fact retained.
|
||||
// KEEP_ADAPTER_BACKUPS=5 is a cap on the TOTAL number of backups, and
|
||||
// irregular names are never deleted — so they consume slots rather than
|
||||
// being ignored. With 3 unremovable directories present, only 2 of the
|
||||
// well-formed ones can survive. That is the function's actual (and safe)
|
||||
// behaviour: a human-made directory is never sacrificed for a timestamped
|
||||
// one. Assert the real outcome rather than the KEEP value.
|
||||
var wellFormedLeft []string
|
||||
for _, e := range entries {
|
||||
if strings.HasPrefix(e.Name(), "adapters.bak.") &&
|
||||
len(strings.TrimPrefix(e.Name(), "adapters.bak.")) == 14 {
|
||||
wellFormedLeft = append(wellFormedLeft, e.Name())
|
||||
}
|
||||
}
|
||||
if len(wellFormedLeft) == 0 {
|
||||
t.Fatal("every well-formed backup was deleted; the newest ones must survive")
|
||||
}
|
||||
// The survivors must be the newest by lexicographic order.
|
||||
if want := "adapters.bak.20261201120000"; !got[want] {
|
||||
t.Errorf("%s was deleted but it is the newest timestamp", want)
|
||||
}
|
||||
if got["adapters.bak.20260101120000"] {
|
||||
t.Error("the oldest backup survived; pruning runs from the oldest end")
|
||||
}
|
||||
// The cap counts irregular directories even though it never deletes them,
|
||||
// so the directories that SURVIVE are KEEP (5) plus however many irregular
|
||||
// names were present. Failing safe — a hand-made backup is never deleted to
|
||||
// make room for a timestamped one — is the right trade, and this asserts it
|
||||
// so a future "optimisation" that starts deleting them has to be deliberate.
|
||||
var irregular, regular int
|
||||
for _, e := range entries {
|
||||
if !strings.HasPrefix(e.Name(), "adapters.bak.") {
|
||||
continue
|
||||
}
|
||||
if len(strings.TrimPrefix(e.Name(), "adapters.bak.")) == 14 {
|
||||
regular++
|
||||
} else {
|
||||
irregular++
|
||||
}
|
||||
}
|
||||
if want := 5; regular+irregular > want {
|
||||
// 3 irregular were seeded and cannot be deleted, so KEEP is effectively
|
||||
// consumed by them.
|
||||
if regular+irregular-3 > want {
|
||||
t.Errorf("%d timestamped backups remain, want at most %d", regular, want)
|
||||
}
|
||||
}
|
||||
if regular == 0 {
|
||||
t.Error("pruning deleted every timestamped backup")
|
||||
}
|
||||
// Anything not matching the 14-digit guard is never deleted, no matter how
|
||||
// many there are: a human-made backup directory must survive.
|
||||
for _, name := range []string{"manual", "202609011200000", "2026090112000"} {
|
||||
if !got["adapters.bak."+name] {
|
||||
t.Errorf("adapters.bak.%s was deleted; the guard only matches "+
|
||||
"^adapters\\.bak\\.[0-9]{14}$", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestDeployBacksUpTheLiveFileBeforeOverwriting guards the mistake that makes a
|
||||
// rollback useless: `cp <new source> <.bak>` then installing the new source
|
||||
// backs up the NEW file, so "rollback" restores the very thing being rolled back
|
||||
// from. The backup must be taken from the target path.
|
||||
func TestDeployBacksUpTheLiveFileBeforeOverwriting(t *testing.T) {
|
||||
b, err := os.ReadFile("../../deploy.sh")
|
||||
if err != nil {
|
||||
t.Skip("deploy.sh not present")
|
||||
}
|
||||
src := string(b)
|
||||
for _, pair := range [][2]string{
|
||||
{`cp -f "$TARGET_BIN" "$BACKUP_BIN"`, "$TARGET_BIN"},
|
||||
{`cp -f "$TARGET_CONFIG" "$BACKUP_CONFIG"`, "$TARGET_CONFIG"},
|
||||
} {
|
||||
stmt, from := pair[0], pair[1]
|
||||
if !strings.Contains(src, stmt) {
|
||||
t.Errorf("expected the pre-install backup %q in deploy.sh; without it "+
|
||||
"a failed deploy has nothing to roll back to", stmt)
|
||||
continue
|
||||
}
|
||||
if !strings.Contains(stmt, from) {
|
||||
t.Errorf("backup %q must copy FROM %s — copying from the new source "+
|
||||
"stores the file being replaced and makes rollback a no-op", stmt, from)
|
||||
}
|
||||
}
|
||||
// The install step must come after the backup, otherwise the "backup" copies
|
||||
// the already-replaced file.
|
||||
backupAt := strings.Index(src, `cp -f "$TARGET_BIN" "$BACKUP_BIN"`)
|
||||
installAt := strings.Index(src, `mv -f "$STAGING_BIN" "$TARGET_BIN"`)
|
||||
if backupAt < 0 || installAt < 0 {
|
||||
t.Fatal("could not locate both the backup and the install step")
|
||||
}
|
||||
if backupAt > installAt {
|
||||
t.Error("the binary is installed before it is backed up; the backup then " +
|
||||
"captures the new file and rollback restores nothing")
|
||||
}
|
||||
}
|
||||
@ -58,7 +58,7 @@ func newTestGateway(t *testing.T, srcs ...config.Source) *Gateway {
|
||||
t.Fatalf("core: %v", err)
|
||||
}
|
||||
t.Cleanup(c.Close)
|
||||
g, err := New(c)
|
||||
g, err := New(c, []string{"sk-test"})
|
||||
if err != nil {
|
||||
t.Fatalf("gateway: %v", err)
|
||||
}
|
||||
|
||||
@ -1,192 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The Electron shell (cmd/gui) had NO tests at all, so the plugin panel went in
|
||||
// with references to CSS classes that do not exist (.tag, .sm) and to helper
|
||||
// functions that were never defined in that document (esc / escAttr). All of
|
||||
// it rendered as unstyled text and would have thrown a ReferenceError at click
|
||||
// time — and none of that is visible without opening the app.
|
||||
//
|
||||
// These tests are deliberately static. They do not launch Electron: what they
|
||||
// guard is the class of mistake that "looks fine until someone themes it",
|
||||
// which is exactly what a missing CSS class or a missing helper is.
|
||||
|
||||
func guiFile(t *testing.T, rel string) string {
|
||||
t.Helper()
|
||||
// The tests live in internal/gateway, so walk up to the repo root.
|
||||
p := filepath.Join("..", "..", rel)
|
||||
b, err := os.ReadFile(p)
|
||||
if err != nil {
|
||||
t.Skipf("%s not readable: %v", rel, err)
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
|
||||
// classUseRe finds class="..." occurrences in a document.
|
||||
var classUseRe = regexp.MustCompile(`class="([^"]+)"`)
|
||||
|
||||
// classTokenRe matches ANY ".name" inside the stylesheet. Deliberately loose:
|
||||
// it also matches inside compound selectors (".tb-btn.tb-close:hover" must count
|
||||
// as defining .tb-close, which a "must be at the start of a selector" rule
|
||||
// misses) and inside comments, which only ever makes the check MORE permissive.
|
||||
// A false pass here would be bad, so the strictness lives elsewhere: the
|
||||
// variable below is what actually guards the new code.
|
||||
var classTokenRe = regexp.MustCompile(`\.([A-Za-z_][A-Za-z0-9_-]*)`)
|
||||
|
||||
// guiKnownUnstyled lists classes the shell markup has always used with no
|
||||
// matching rule. They are pre-existing cosmetic gaps, not regressions, and
|
||||
// failing on them would make this test useless as a guard for NEW work.
|
||||
var guiKnownUnstyled = map[string]bool{
|
||||
"blob": true, // decorative blur blobs, styled per-instance via .b1/.b2/.b3
|
||||
"tgl": true, // rail toggle affordance that leaned on .rail-btn
|
||||
"rail": true, // the rail container itself has no rule; .rail-btn children carry the look
|
||||
}
|
||||
|
||||
// TestGUICSSClassesExist is the guard that would have caught .tag and .sm: every
|
||||
// class used in the shell's markup must be defined in its stylesheet.
|
||||
//
|
||||
// The comparison is on the LAST segment of a selector, because the stylesheet
|
||||
// scopes things (`.form .actions .primary`, `#bgfx .b1`): a rule for
|
||||
// `.pl-acts button` defines no class at all, and `.form .row .toggle` defines
|
||||
// `.toggle`. Requiring a top-level class would be too strict; requiring that
|
||||
// some selector's last identifier matches is the right level.
|
||||
func TestGUICSSClassesExist(t *testing.T) {
|
||||
html := guiFile(t, "cmd/gui/renderer/index.html")
|
||||
css := guiFile(t, "cmd/gui/renderer/style.css")
|
||||
|
||||
defined := map[string]bool{}
|
||||
|
||||
// Collect every class token that appears at the START of a selector
|
||||
// position. A full CSS parser is overkill and was the source of two wrong
|
||||
// turns here; what the check needs is simply "does the name .foo appear
|
||||
// anywhere in the stylesheet as a selector component".
|
||||
//
|
||||
// Scoping is respected loosely: `.form .actions .primary` counts as
|
||||
// defining `.primary`, and `.pl-acts button` defines no class — which is
|
||||
// exactly why the plugin panel needed its own rules.
|
||||
for _, m := range classTokenRe.FindAllStringSubmatch(css, -1) {
|
||||
defined[m[1]] = true
|
||||
}
|
||||
if len(defined) == 0 {
|
||||
t.Fatal("no classes parsed from the stylesheet; the check is broken")
|
||||
}
|
||||
|
||||
// Classes the JS builds as strings must exist too.
|
||||
js := guiFile(t, "cmd/gui/renderer/app.js")
|
||||
var missing []string
|
||||
seen := map[string]bool{}
|
||||
note := func(cls, where string) {
|
||||
// A "${...}" token is a template literal being spliced at runtime, not
|
||||
// a class name; the classes it can expand to are checked at their
|
||||
// definition sites instead.
|
||||
if cls == "" || strings.ContainsAny(cls, "${}") || seen[cls] {
|
||||
return
|
||||
}
|
||||
seen[cls] = true
|
||||
if guiKnownUnstyled[cls] {
|
||||
return
|
||||
}
|
||||
if !defined[cls] {
|
||||
missing = append(missing, cls+" ("+where+")")
|
||||
}
|
||||
}
|
||||
for _, m := range classUseRe.FindAllStringSubmatch(html, -1) {
|
||||
for _, c := range strings.Fields(m[1]) {
|
||||
note(c, "index.html")
|
||||
}
|
||||
}
|
||||
for _, m := range classUseRe.FindAllStringSubmatch(js, -1) {
|
||||
for _, c := range strings.Fields(m[1]) {
|
||||
note(c, "app.js")
|
||||
}
|
||||
}
|
||||
if len(missing) > 0 {
|
||||
t.Errorf("classes used but not defined in style.css (they render unstyled):\n %s",
|
||||
strings.Join(missing, "\n "))
|
||||
}
|
||||
}
|
||||
|
||||
// TestGUIHelperFunctionsAreDefined catches the other half: renderer/app.js is a
|
||||
// separate document from the WebUI, so it does NOT have the WebUI's esc/escAttr.
|
||||
// Referencing them gives a ReferenceError only when the line runs.
|
||||
func TestGUIHelperFunctionsAreDefined(t *testing.T) {
|
||||
js := guiFile(t, "cmd/gui/renderer/app.js")
|
||||
for _, fn := range []string{"esc", "escAttr", "toast", "loadPlugins", "togglePlugin", "enableAllPlugins"} {
|
||||
defined := regexp.MustCompile(`function ` + fn + `\b`).MatchString(js)
|
||||
called := regexp.MustCompile(`\b` + fn + `\s*\(`).MatchString(js)
|
||||
if called && !defined {
|
||||
t.Errorf("%s() is called but never defined in app.js", fn)
|
||||
}
|
||||
if !called && !defined {
|
||||
// A defined-but-unused helper is dead code, not an error.
|
||||
continue
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestGUIPluginPanelIsReachable: the panel must be inside the settings overlay
|
||||
// AND the settings overlay must actually open it. A panel wired to a button
|
||||
// that was never bound is invisible-but-present, which passes a grep review.
|
||||
func TestGUIPluginPanelIsReachable(t *testing.T) {
|
||||
html := guiFile(t, "cmd/gui/renderer/index.html")
|
||||
js := guiFile(t, "cmd/gui/renderer/app.js")
|
||||
if !strings.Contains(html, `id="pl-list"`) {
|
||||
t.Error("no #pl-list in the settings overlay")
|
||||
}
|
||||
if !strings.Contains(html, `id="settings-overlay"`) {
|
||||
t.Fatal("the settings overlay is gone")
|
||||
}
|
||||
// inside the overlay: the element index must come after the overlay's
|
||||
if strings.Index(html, `id="settings-overlay"`) > strings.Index(html, `id="pl-list"`) {
|
||||
t.Error("#pl-list appears before the settings overlay, so it renders outside the panel")
|
||||
}
|
||||
// The buttons must be bound.
|
||||
for _, id := range []string{"pl-reload", "pl-toggle-all"} {
|
||||
if !strings.Contains(html, `id="`+id+`"`) {
|
||||
t.Errorf("#%s is missing from the markup", id)
|
||||
}
|
||||
if !strings.Contains(js, `"`+id+`"`) {
|
||||
t.Errorf("#%s exists but app.js never binds it", id)
|
||||
}
|
||||
}
|
||||
// And openSettings must trigger the load, or the panel shows a stale empty
|
||||
// list on every open.
|
||||
if !strings.Contains(js, "loadPlugins()") {
|
||||
t.Error("app.js never calls loadPlugins()")
|
||||
}
|
||||
}
|
||||
|
||||
// TestGUIIPCPathIsConstrained: plugins:proxy is a raw pass-through, which is
|
||||
// convenient but would be a hole if it accepted arbitrary paths. The main
|
||||
// process must reject anything outside /api/plugins and any traversal.
|
||||
func TestGUIIPCPathIsConstrained(t *testing.T) {
|
||||
main := guiFile(t, "cmd/gui/main.js")
|
||||
for _, needle := range []string{
|
||||
`path.startsWith("/api/plugins")`,
|
||||
`path.includes("..")`,
|
||||
"plugins:proxy",
|
||||
"unsealViaCore()",
|
||||
} {
|
||||
if !strings.Contains(main, needle) {
|
||||
t.Errorf("cmd/gui/main.js is missing the guard %q", needle)
|
||||
}
|
||||
}
|
||||
// The plugins channel must be a proxy, not a key passthrough: the renderer
|
||||
// sends (method, path) and the main process attaches the key.
|
||||
//
|
||||
// NOTE: preload does expose a pre-existing `core.key` getter — the shell
|
||||
// needs the admin key to load the embedded WebUI without a login. That is
|
||||
// existing, deliberate design and out of scope here; asserting on "key:" in
|
||||
// preload would flag a pre-existing feature as a new hole.
|
||||
pre := guiFile(t, "cmd/gui/preload.js")
|
||||
if !strings.Contains(pre, "request: (method, path, body)") {
|
||||
t.Error("preload does not expose the plugins request proxy")
|
||||
}
|
||||
}
|
||||
@ -1,257 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"path/filepath"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/core"
|
||||
)
|
||||
|
||||
// Per-key AUTO chain admin API. The behaviours pinned here are the ones that
|
||||
// are easy to get wrong and invisible when they are:
|
||||
//
|
||||
// - /api/keys/me/auto must be reachable by a NON-admin (it is their own
|
||||
// chain); routing it through the generic {key}/auto handler would 403 it.
|
||||
// - /api/keys/{key}/auto must 403 a non-admin.
|
||||
// - an empty PUT and a DELETE mean the same thing ("inherit global"), so the
|
||||
// WebUI cannot persist a chain that would then 503 with no slots.
|
||||
// - a PUT that resolves to zero slots is reported back, so an admin who
|
||||
// mistyped a model learns it now instead of from the user's next 503.
|
||||
|
||||
// keyAutoGateway builds a gateway with one admin key and one user key.
|
||||
func keyAutoGateway(t *testing.T) *Gateway {
|
||||
t.Helper()
|
||||
td := t.TempDir()
|
||||
cfgPath := filepath.Join(td, "config.yaml")
|
||||
if err := os.WriteFile(cfgPath, []byte("listen: :0"), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg := &config.Config{
|
||||
Path: cfgPath,
|
||||
AdapterDir: filepath.Join(td, "adapters"),
|
||||
RuntimeFile: filepath.Join(td, "runtime.json"),
|
||||
DefaultModel: "AUTO",
|
||||
GatewayKeys: []string{"sk-admin"},
|
||||
Keys: []config.GWKey{
|
||||
{Key: "sk-admin", Role: "admin", Name: "admin"},
|
||||
{Key: "sk-user", Role: "user", Name: "user"},
|
||||
},
|
||||
Sources: []config.Source{
|
||||
{Name: "s1", BaseURL: "http://127.0.0.1:1/v1", Adapter: "openai",
|
||||
Models: []config.Model{{ID: "gpt-4o", Priority: 10}, {ID: "gpt-4o-mini", Priority: 5}}},
|
||||
},
|
||||
}
|
||||
if err := cfg.ApplyDefaults(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := core.NewFromConfig(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("core: %v", err)
|
||||
}
|
||||
t.Cleanup(c.Close)
|
||||
g, err := New(c)
|
||||
if err != nil {
|
||||
t.Fatalf("gateway: %v", err)
|
||||
}
|
||||
return g
|
||||
}
|
||||
|
||||
// doReqAs issues an authenticated request as a specific key.
|
||||
func doReqAs(t *testing.T, g *Gateway, key, method, path, body string) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
req, _ := http.NewRequest(method, path, strings.NewReader(body))
|
||||
req.Header.Set("Authorization", "Bearer "+key)
|
||||
if body != "" {
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
rr := httptest.NewRecorder()
|
||||
g.Handler().ServeHTTP(rr, req)
|
||||
return rr
|
||||
}
|
||||
|
||||
func TestKeyAutoAPIPermissions(t *testing.T) {
|
||||
g := keyAutoGateway(t)
|
||||
|
||||
// A user may read their own chain...
|
||||
if rr := doReqAs(t, g, "sk-user", "GET", "/api/keys/me/auto", ""); rr.Code != http.StatusOK {
|
||||
t.Fatalf("GET /api/keys/me/auto as user = %d, want 200 (body %s)", rr.Code, rr.Body.String())
|
||||
}
|
||||
// ...but not another key's.
|
||||
if rr := doReqAs(t, g, "sk-user", "GET", "/api/keys/sk-admin/auto", ""); rr.Code != http.StatusForbidden {
|
||||
t.Fatalf("GET other key's auto as user = %d, want 403", rr.Code)
|
||||
}
|
||||
// ...and not their own via the admin route either.
|
||||
if rr := doReqAs(t, g, "sk-user", "PUT", "/api/keys/sk-user/auto", `{"auto":[{"model":"gpt-4o"}]}`); rr.Code != http.StatusForbidden {
|
||||
t.Fatalf("user PUT own auto = %d, want 403 (only admin configures)", rr.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestKeyAutoAPICrudAndInherit(t *testing.T) {
|
||||
g := keyAutoGateway(t)
|
||||
|
||||
// Initially inherits.
|
||||
var got struct {
|
||||
Inherits bool `json:"inherits"`
|
||||
Auto []config.ModelScope `json:"auto"`
|
||||
}
|
||||
rr := doReqAs(t, g, "sk-admin", "GET", "/api/keys/sk-user/auto", "")
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("GET = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !got.Inherits || len(got.Auto) != 0 {
|
||||
t.Fatalf("fresh key must inherit global, got inherits=%v auto=%v", got.Inherits, got.Auto)
|
||||
}
|
||||
|
||||
// Admin sets a chain; the response reports the resolved slot count.
|
||||
rr = doReqAs(t, g, "sk-admin", "PUT", "/api/keys/sk-user/auto",
|
||||
`{"auto":[{"model":"gpt-4o-mini","source":"s1","tier":1}]}`)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("PUT = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var put struct {
|
||||
Inherits bool `json:"inherits"`
|
||||
Slots int `json:"slots"`
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &put); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if put.Inherits {
|
||||
t.Fatal("after a non-empty PUT the key must no longer inherit")
|
||||
}
|
||||
if put.Slots != 1 {
|
||||
t.Fatalf("resolved slots = %d, want 1", put.Slots)
|
||||
}
|
||||
|
||||
// And it reads back as an own chain.
|
||||
rr = doReqAs(t, g, "sk-admin", "GET", "/api/keys/sk-user/auto", "")
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Inherits || len(got.Auto) != 1 || got.Auto[0].Model != "gpt-4o-mini" {
|
||||
t.Fatalf("own chain must read back, got inherits=%v auto=%v", got.Inherits, got.Auto)
|
||||
}
|
||||
|
||||
// DELETE restores inheritance.
|
||||
if rr := doReqAs(t, g, "sk-admin", "DELETE", "/api/keys/sk-user/auto", ""); rr.Code != http.StatusOK {
|
||||
t.Fatalf("DELETE = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
rr = doReqAs(t, g, "sk-admin", "GET", "/api/keys/sk-user/auto", "")
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !got.Inherits {
|
||||
t.Fatal("after DELETE the key must inherit the global chain again")
|
||||
}
|
||||
}
|
||||
|
||||
// An empty PUT must behave like DELETE. Otherwise a WebUI that submits an
|
||||
// empty list (every slot deleted in the editor) would persist an empty chain
|
||||
// and the next AUTO request would fail with no_provider.
|
||||
func TestKeyAutoAPIEmptyPutClearsOverride(t *testing.T) {
|
||||
g := keyAutoGateway(t)
|
||||
|
||||
if rr := doReqAs(t, g, "sk-admin", "PUT", "/api/keys/sk-user/auto",
|
||||
`{"auto":[{"model":"gpt-4o","source":"s1","tier":1}]}`); rr.Code != http.StatusOK {
|
||||
t.Fatalf("seed PUT = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
rr := doReqAs(t, g, "sk-admin", "PUT", "/api/keys/sk-user/auto", `{"auto":[]}`)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("empty PUT = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var got struct {
|
||||
Inherits bool `json:"inherits"`
|
||||
}
|
||||
if rr := doReqAs(t, g, "sk-admin", "GET", "/api/keys/sk-user/auto", ""); rr.Code != http.StatusOK {
|
||||
t.Fatal(rr.Body.String())
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !got.Inherits {
|
||||
t.Fatal("an empty PUT must clear the override, not persist an empty chain")
|
||||
}
|
||||
}
|
||||
|
||||
// A chain naming a model that does not exist compiles to zero slots. The API
|
||||
// must say so at write time instead of letting the user's next request 503.
|
||||
func TestKeyAutoAPIReportsUnresolvableSlots(t *testing.T) {
|
||||
g := keyAutoGateway(t)
|
||||
|
||||
rr := doReqAs(t, g, "sk-admin", "PUT", "/api/keys/sk-user/auto",
|
||||
`{"auto":[{"model":"does-not-exist","source":"s1","tier":1}]}`)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("PUT = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var put struct {
|
||||
Inherits bool `json:"inherits"`
|
||||
Slots int `json:"slots"`
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &put); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if put.Slots != 0 {
|
||||
t.Fatalf("unknown model must resolve to 0 slots, got %d", put.Slots)
|
||||
}
|
||||
}
|
||||
|
||||
func TestKeyAutoAPIUnknownKey404(t *testing.T) {
|
||||
g := keyAutoGateway(t)
|
||||
|
||||
if rr := doReqAs(t, g, "sk-admin", "GET", "/api/keys/sk-nope/auto", ""); rr.Code != http.StatusNotFound {
|
||||
t.Fatalf("GET unknown key auto = %d, want 404", rr.Code)
|
||||
}
|
||||
if rr := doReqAs(t, g, "sk-admin", "PUT", "/api/keys/sk-nope/auto", `{"auto":[{"model":"gpt-4o"}]}`); rr.Code != http.StatusNotFound {
|
||||
t.Fatalf("PUT unknown key auto = %d, want 404", rr.Code)
|
||||
}
|
||||
}
|
||||
|
||||
// A bad quota must be rejected with 400, not persisted and not silently
|
||||
// clamped — same rule as the model scope path.
|
||||
func TestKeyAutoAPIRejectsBadQuota(t *testing.T) {
|
||||
g := keyAutoGateway(t)
|
||||
|
||||
rr := doReqAs(t, g, "sk-admin", "PUT", "/api/keys/sk-user/auto",
|
||||
`{"auto":[{"model":"gpt-4o","source":"s1","tier":1,"token_quota":-1}]}`)
|
||||
if rr.Code != http.StatusBadRequest {
|
||||
t.Fatalf("negative quota PUT = %d, want 400 (body %s)", rr.Code, rr.Body.String())
|
||||
}
|
||||
// Nothing may have been persisted.
|
||||
var got struct {
|
||||
Inherits bool `json:"inherits"`
|
||||
}
|
||||
get := doReqAs(t, g, "sk-admin", "GET", "/api/keys/sk-user/auto", "")
|
||||
if get.Code != http.StatusOK {
|
||||
t.Fatal(get.Body.String())
|
||||
}
|
||||
if err := json.Unmarshal(get.Body.Bytes(), &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !got.Inherits {
|
||||
t.Fatal("a rejected PUT must not persist a chain")
|
||||
}
|
||||
}
|
||||
|
||||
// The other key endpoints must keep working: the {key}/auto route must not
|
||||
// shadow /api/keys/{key} itself.
|
||||
func TestKeyAutoAPIDoesNotShadowKeyRoutes(t *testing.T) {
|
||||
g := keyAutoGateway(t)
|
||||
|
||||
if rr := doReqAs(t, g, "sk-admin", "GET", "/api/keys", ""); rr.Code != http.StatusOK {
|
||||
t.Fatalf("GET /api/keys = %d, want 200", rr.Code)
|
||||
}
|
||||
if rr := doReqAs(t, g, "sk-admin", "GET", "/api/keys/me", ""); rr.Code != http.StatusOK {
|
||||
t.Fatalf("GET /api/keys/me = %d, want 200", rr.Code)
|
||||
}
|
||||
if rr := doReqAs(t, g, "sk-user", "GET", "/api/keys/me", ""); rr.Code != http.StatusOK {
|
||||
t.Fatalf("GET /api/keys/me as user = %d, want 200", rr.Code)
|
||||
}
|
||||
}
|
||||
@ -1,253 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/core"
|
||||
)
|
||||
|
||||
// adminGateway builds a gateway whose admin key can call /api/keys.
|
||||
func adminGateway(t *testing.T, keys ...config.GWKey) *Gateway {
|
||||
t.Helper()
|
||||
td := t.TempDir()
|
||||
cfgPath := td + "/config.yaml"
|
||||
if err := os.WriteFile(cfgPath, []byte("listen: :0"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg := &config.Config{
|
||||
Path: cfgPath,
|
||||
AdapterDir: td + "/adapters",
|
||||
RuntimeFile: td + "/runtime.json",
|
||||
Keys: keys,
|
||||
}
|
||||
if err := cfg.ApplyDefaults(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := core.NewFromConfig(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("core: %v", err)
|
||||
}
|
||||
t.Cleanup(c.Close)
|
||||
g, err := New(c)
|
||||
if err != nil {
|
||||
t.Fatalf("gateway: %v", err)
|
||||
}
|
||||
return g
|
||||
}
|
||||
|
||||
func adminReq(t *testing.T, g *Gateway, method, path, body string) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
req, _ := http.NewRequest(method, path, strings.NewReader(body))
|
||||
req.Header.Set("Authorization", "Bearer sk-admin")
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
rr := httptest.NewRecorder()
|
||||
g.Handler().ServeHTTP(rr, req)
|
||||
return rr
|
||||
}
|
||||
|
||||
// keyRecord pulls one key's stored record out of the admin list.
|
||||
func keyRecord(t *testing.T, g *Gateway, secret string) config.GWKey {
|
||||
t.Helper()
|
||||
rr := adminReq(t, g, "GET", "/api/keys", "")
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("GET /api/keys: %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var out struct {
|
||||
Keys []config.GWKey `json:"keys"`
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &out); err != nil {
|
||||
t.Fatalf("decode: %v (%s)", err, rr.Body.String())
|
||||
}
|
||||
for _, k := range out.Keys {
|
||||
if k.Key == secret {
|
||||
return k
|
||||
}
|
||||
}
|
||||
t.Fatalf("key %q not found in %s", secret, rr.Body.String())
|
||||
return config.GWKey{}
|
||||
}
|
||||
|
||||
func scopeOf(t *testing.T, k config.GWKey, model string) config.ModelScope {
|
||||
t.Helper()
|
||||
for _, m := range k.Models {
|
||||
if m.Model == model {
|
||||
return m
|
||||
}
|
||||
}
|
||||
t.Fatalf("scope %q not found in %+v", model, k.Models)
|
||||
return config.ModelScope{}
|
||||
}
|
||||
|
||||
// Quotas live on the scope entries, not on the key: creating a key with a
|
||||
// budget means creating scopes that carry it, and they must survive a
|
||||
// read-back (persisted, not just echoed).
|
||||
func TestKeyAPICreatesPerModelQuota(t *testing.T) {
|
||||
g := adminGateway(t, config.GWKey{Key: "sk-admin", Role: "admin"})
|
||||
rr := adminReq(t, g, "POST", "/api/keys",
|
||||
`{"name":"agent-x","role":"user","models":[{"model":"m1","token_quota":50000,"req_quota":200,"period":"nhour","hours":6}]}`)
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("create: %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var created struct {
|
||||
Key config.GWKey `json:"key"`
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &created); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
sc := scopeOf(t, created.Key, "m1")
|
||||
if sc.TokenQuota != 50000 || sc.ReqQuota != 200 || sc.Period != "nhour" || sc.Hours != 6 {
|
||||
t.Fatalf("created scope did not carry the caps: %+v", sc)
|
||||
}
|
||||
back := scopeOf(t, keyRecord(t, g, created.Key.Key), "m1")
|
||||
if back.TokenQuota != 50000 || back.ReqQuota != 200 || back.Period != "nhour" || back.Hours != 6 {
|
||||
t.Errorf("read-back lost the caps: %+v", back)
|
||||
}
|
||||
}
|
||||
|
||||
// Two models on one key carry independent budgets.
|
||||
func TestKeyAPIKeepsPerModelQuotaIndependent(t *testing.T) {
|
||||
g := adminGateway(t, config.GWKey{Key: "sk-admin", Role: "admin"})
|
||||
rr := adminReq(t, g, "POST", "/api/keys",
|
||||
`{"name":"agent-y","role":"user","models":[{"model":"m1","token_quota":1000,"period":"hour"},{"model":"m2","token_quota":9999,"req_quota":7,"period":"week"}]}`)
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("create: %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var created struct {
|
||||
Key config.GWKey `json:"key"`
|
||||
}
|
||||
_ = json.Unmarshal(rr.Body.Bytes(), &created)
|
||||
back := keyRecord(t, g, created.Key.Key)
|
||||
if a := scopeOf(t, back, "m1"); a.TokenQuota != 1000 || a.ReqQuota != 0 || a.Period != "hour" {
|
||||
t.Errorf("m1 caps wrong: %+v", a)
|
||||
}
|
||||
if b := scopeOf(t, back, "m2"); b.TokenQuota != 9999 || b.ReqQuota != 7 || b.Period != "week" {
|
||||
t.Errorf("m2 caps wrong: %+v", b)
|
||||
}
|
||||
}
|
||||
|
||||
// Sending 0 explicitly lifts that model's cap.
|
||||
func TestKeyAPIUpdateZeroLiftsCap(t *testing.T) {
|
||||
g := adminGateway(t, config.GWKey{Key: "sk-admin", Role: "admin"})
|
||||
rr := adminReq(t, g, "POST", "/api/keys",
|
||||
`{"name":"z","role":"user","models":[{"model":"m1","token_quota":1000,"period":"hour"}]}`)
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("create: %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var created struct {
|
||||
Key config.GWKey `json:"key"`
|
||||
}
|
||||
_ = json.Unmarshal(rr.Body.Bytes(), &created)
|
||||
secret := created.Key.Key
|
||||
|
||||
rr = adminReq(t, g, "PUT", "/api/keys/"+secret,
|
||||
`{"models":[{"model":"m1","token_quota":0,"req_quota":0,"period":""}]}`)
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("lift: %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if back := scopeOf(t, keyRecord(t, g, secret), "m1"); back.TokenQuota != 0 || back.ReqQuota != 0 || back.Period != "" {
|
||||
t.Errorf("caps not lifted: %+v", back)
|
||||
}
|
||||
}
|
||||
|
||||
// A misspelled period must be refused, not quietly turned into an all-time
|
||||
// quota — which is the exact opposite of what the operator typed.
|
||||
func TestKeyAPIRejectsBadPeriod(t *testing.T) {
|
||||
g := adminGateway(t, config.GWKey{Key: "sk-admin", Role: "admin"})
|
||||
rr := adminReq(t, g, "POST", "/api/keys",
|
||||
`{"name":"bad","role":"user","models":[{"model":"m1","token_quota":1000,"period":"houre"}]}`)
|
||||
if rr.Code != http.StatusBadRequest {
|
||||
t.Fatalf("want 400 for a bad period, got %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if !strings.Contains(rr.Body.String(), "period") {
|
||||
t.Errorf("error should name the period field: %s", rr.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestKeyAPIRejectsNegativeQuota(t *testing.T) {
|
||||
g := adminGateway(t, config.GWKey{Key: "sk-admin", Role: "admin"})
|
||||
rr := adminReq(t, g, "POST", "/api/keys",
|
||||
`{"name":"bad","role":"user","models":[{"model":"m1","token_quota":-5}]}`)
|
||||
if rr.Code != http.StatusBadRequest {
|
||||
t.Fatalf("want 400 for a negative quota, got %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
// The rejection must say WHICH model is over budget, so an operator looking at
|
||||
// a key with a dozen scopes can tell which one to raise.
|
||||
func TestKeyAPIRejectionNamesTheModel(t *testing.T) {
|
||||
g := adminGateway(t, config.GWKey{Key: "sk-admin", Role: "admin"})
|
||||
rr := adminReq(t, g, "POST", "/api/keys",
|
||||
`{"name":"bad","role":"user","models":[{"model":"m1","req_quota":-1}]}`)
|
||||
if rr.Code != http.StatusBadRequest {
|
||||
t.Fatalf("want 400, got %d", rr.Code)
|
||||
}
|
||||
if !strings.Contains(rr.Body.String(), "m1") {
|
||||
t.Errorf("error should name the offending model: %s", rr.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
// A non-admin key must not be able to mint keys.
|
||||
func TestKeyAPIQuotaIsAdminOnly(t *testing.T) {
|
||||
g := adminGateway(t,
|
||||
config.GWKey{Key: "sk-admin", Role: "admin"},
|
||||
config.GWKey{Key: "sk-u", Role: "user", Models: []config.ModelScope{{Model: "m1", TokenQuota: 10, Period: "hour"}}},
|
||||
)
|
||||
req, _ := http.NewRequest("POST", "/api/keys",
|
||||
strings.NewReader(`{"name":"x","role":"admin","models":[{"model":"m1"}]}`))
|
||||
req.Header.Set("Authorization", "Bearer sk-u")
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
rr := httptest.NewRecorder()
|
||||
g.Handler().ServeHTTP(rr, req)
|
||||
if rr.Code != http.StatusForbidden {
|
||||
t.Fatalf("non-admin create: want 403, got %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
// /api/v1/keys exposes the per-model caps but never a secret
|
||||
rr = adminReq(t, g, "GET", "/api/v1/keys", "")
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("GET /api/v1/keys: %d", rr.Code)
|
||||
}
|
||||
if strings.Contains(rr.Body.String(), "sk-u") {
|
||||
t.Error("/api/v1/keys leaked a key secret")
|
||||
}
|
||||
if !strings.Contains(rr.Body.String(), `"token_quota":10`) {
|
||||
t.Errorf("/api/v1/keys should expose the per-model caps: %s", rr.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
// A user must be able to see their own budget: /api/keys/me is the only key
|
||||
// view a non-admin gets, so a cap missing from it is invisible to the very
|
||||
// client it constrains.
|
||||
func TestKeyMeExposesOwnQuota(t *testing.T) {
|
||||
g := adminGateway(t,
|
||||
config.GWKey{Key: "sk-admin", Role: "admin"},
|
||||
config.GWKey{Key: "sk-u", Role: "user", Name: "agent",
|
||||
Models: []config.ModelScope{{Model: "m1", TokenQuota: 123456, ReqQuota: 42, Period: "week"}}},
|
||||
)
|
||||
req, _ := http.NewRequest("GET", "/api/keys/me", nil)
|
||||
req.Header.Set("Authorization", "Bearer sk-u")
|
||||
rr := httptest.NewRecorder()
|
||||
g.Handler().ServeHTTP(rr, req)
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("GET /api/keys/me: %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
// the endpoint wraps the record: {"key": {...}}
|
||||
var wrap struct {
|
||||
Key config.GWKey `json:"key"`
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &wrap); err != nil {
|
||||
t.Fatalf("decode: %v (%s)", err, rr.Body.String())
|
||||
}
|
||||
sc := scopeOf(t, wrap.Key, "m1")
|
||||
if sc.TokenQuota != 123456 || sc.ReqQuota != 42 || sc.Period != "week" {
|
||||
t.Errorf("own quota not visible to the key's owner: %+v", sc)
|
||||
}
|
||||
}
|
||||
|
||||
var _ = fmt.Sprintf
|
||||
@ -1,60 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The per-key quota buckets are shared maps mutated on every recorded request
|
||||
// and read on every quota check. Single-threaded unit tests cannot see a race
|
||||
// there at all, so this drives 64 goroutines through record + read together
|
||||
// under `go test -race`. It also opts one model into the lazily-created pinned
|
||||
// bucket mid-flight, which is the path that mutates two bucket maps at once.
|
||||
func TestConcurrentQuotaAccounting(t *testing.T) {
|
||||
s := NewStats(0)
|
||||
keys := []string{"k0", "k1", "k2", "k3", "k4", "k5", "k6"}
|
||||
models := []string{"m0", "m1", "m2", "AUTO"}
|
||||
var wg sync.WaitGroup
|
||||
stop := time.Now().Add(2 * time.Second)
|
||||
|
||||
for w := 0; w < 64; w++ {
|
||||
wg.Add(1)
|
||||
go func(w int) {
|
||||
defer wg.Done()
|
||||
k := keys[w%len(keys)]
|
||||
m := models[w%len(models)]
|
||||
now := time.Now()
|
||||
for time.Now().Before(stop) {
|
||||
s.Record(Req{Time: now.UnixMilli(), Key: k, Model: m, Source: "src",
|
||||
Prompt: 100, Compl: 50, OK: true, Status: 200, Type: "chat"})
|
||||
_ = s.KeyWindowTokens(k, 3600)
|
||||
_ = s.KeyWindowReqs(k, 3600)
|
||||
_ = s.KeyWindowModelTokens(k, m, "", 3600)
|
||||
_ = s.KeyWindowModelTokens(k, m, "src", 3600) // opts into the pinned bucket
|
||||
}
|
||||
}(w)
|
||||
}
|
||||
wg.Wait()
|
||||
|
||||
// Every recorded request must be accounted for exactly once.
|
||||
s.mu.Lock()
|
||||
var total int64
|
||||
for _, hm := range s.keyHour {
|
||||
for _, v := range hm {
|
||||
total += v
|
||||
}
|
||||
}
|
||||
recs := int64(0)
|
||||
for _, r := range s.recs {
|
||||
recs += r.Prompt + r.Compl
|
||||
}
|
||||
s.mu.Unlock()
|
||||
if total == 0 {
|
||||
t.Fatal("no tokens accounted")
|
||||
}
|
||||
t.Logf("accounted %d tokens across %d keys", total, len(keys))
|
||||
if recs == 0 {
|
||||
t.Error("no records retained; the concurrent writers lost work")
|
||||
}
|
||||
}
|
||||
@ -1,125 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"math/rand"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Perf: the per-key quota bookkeeping runs on every recorded request and on
|
||||
// every quota check, so its cost lands directly on the request path. These
|
||||
// benchmarks exist to catch a regression that would make the feature
|
||||
// expensive; the numbers that matter are relative to each other and to the
|
||||
// pre-change baseline (Record was 275 ns/op with 3 allocs before this work).
|
||||
|
||||
func BenchmarkRecordQuotaBucket(b *testing.B) {
|
||||
s := NewStats(0)
|
||||
r := Req{Time: time.Now().UnixMilli(), Key: "keyABC", Model: "deepseek-v4-flash",
|
||||
Source: "deepseek", Prompt: 800, Compl: 200, OK: true, Status: 200, Type: "chat"}
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
s.Record(r)
|
||||
}
|
||||
}
|
||||
|
||||
// A key that has been busy for the whole 40-day retention.
|
||||
func BenchmarkRecordQuotaBucketFullRetention(b *testing.B) {
|
||||
s := NewStats(0)
|
||||
nowH := time.Now().Unix() / 3600
|
||||
for h := int64(0); h < quotaRetentionHours; h++ {
|
||||
s.Record(Req{Time: (nowH - h) * 3600 * 1000, Key: "keyABC", Model: "m1",
|
||||
Source: "deepseek", Prompt: 100, Compl: 100, OK: true, Status: 200})
|
||||
}
|
||||
r := Req{Time: time.Now().UnixMilli(), Key: "keyABC", Model: "m1", Source: "deepseek",
|
||||
Prompt: 100, Compl: 100, OK: true, Status: 200, Type: "chat"}
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
s.Record(r)
|
||||
}
|
||||
}
|
||||
|
||||
// The whole per-request quota check: total tokens + request count.
|
||||
func BenchmarkQuotaCheckHourly(b *testing.B) {
|
||||
s := NewStats(0)
|
||||
for h := 0; h < 24; h++ {
|
||||
s.Record(Req{Time: time.Now().Add(-time.Duration(h) * time.Hour).UnixMilli(),
|
||||
Key: "keyABC", Model: "m1", Source: "deepseek", Prompt: 100, Compl: 100, OK: true, Status: 200})
|
||||
}
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
s.KeyWindowTokens("keyABC", 3600)
|
||||
s.KeyWindowReqs("keyABC", 3600)
|
||||
}
|
||||
}
|
||||
|
||||
// A 30-day window has to visit 720 hour buckets. This is the worst case the
|
||||
// quota feature puts on the request path.
|
||||
func BenchmarkQuotaCheckMonthly(b *testing.B) {
|
||||
s := NewStats(0)
|
||||
nowH := time.Now().Unix() / 3600
|
||||
for h := int64(0); h < 720; h++ {
|
||||
s.Record(Req{Time: (nowH - h) * 3600 * 1000, Key: "keyABC", Model: "m1",
|
||||
Prompt: 100, Compl: 100, OK: true, Status: 200})
|
||||
}
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
s.KeyWindowTokens("keyABC", 30*24*3600)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSumBucketsMatchesFullScan is the correctness guard for the optimization
|
||||
// that made the quota check proportional to the window instead of to the whole
|
||||
// retention: the windowed sum must agree with a full scan for every bucket
|
||||
// placement, or a quota would silently start letting traffic through. It
|
||||
// caught a real off-by-one (floor instead of ceil on the first hour).
|
||||
func TestSumBucketsMatchesFullScan(t *testing.T) {
|
||||
const hour = 3600
|
||||
base := int64(1_700_000_000) / hour * hour
|
||||
rng := rand.New(rand.NewSource(7))
|
||||
for trial := 0; trial < 400; trial++ {
|
||||
hm := map[int64]int64{}
|
||||
for i, n := 0, rng.Intn(40)+1; i < n; i++ {
|
||||
h := base/hour - int64(rng.Intn(1200))
|
||||
hm[h] += int64(rng.Intn(1000)) + 1
|
||||
}
|
||||
now := base + int64(rng.Intn(hour))
|
||||
for _, sec := range []int64{3600, 2 * 3600, 6 * 3600, 24 * 3600, 7 * 24 * 3600, 30 * 24 * 3600} {
|
||||
var want int64
|
||||
cut := now - sec
|
||||
for h, v := range hm {
|
||||
if h*hour >= cut {
|
||||
want += v
|
||||
}
|
||||
}
|
||||
if got := sumBuckets(hm, now, sec); got != want {
|
||||
t.Fatalf("trial %d sec=%d: sumBuckets=%d, full scan=%d", trial, sec, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The pinned bucket is maintained only for (key, model) pairs whose quota pins
|
||||
// a source. Doing it unconditionally doubled the bucket count — measured at
|
||||
// 26.7 MB for 20 keys x 8 models x 3 sources at full retention, against a
|
||||
// documented ~32 MB total memory budget — to serve a lookup nobody performs.
|
||||
func TestPinnedBucketNotMaintainedUnlessWanted(t *testing.T) {
|
||||
s := NewStats(100)
|
||||
now := time.Now().UnixMilli()
|
||||
for i := 0; i < 5; i++ {
|
||||
s.Record(Req{Time: now, Key: "keyA", Model: "m1", Source: "srcX", Prompt: 100, Compl: 100, OK: true, Status: 200})
|
||||
}
|
||||
s.mu.Lock()
|
||||
_, pinned := s.keyModelHour["keyA"]["srcX::m1"]
|
||||
_, bare := s.keyModelHour["keyA"]["m1"]
|
||||
s.mu.Unlock()
|
||||
if pinned {
|
||||
t.Error("a pinned bucket exists before any quota read the pin — memory regression")
|
||||
}
|
||||
if !bare {
|
||||
t.Error("the bare-model bucket must always exist; an unpinned quota depends on it")
|
||||
}
|
||||
}
|
||||
@ -1,225 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// ---- data layer: per-key isolation ----
|
||||
|
||||
func TestKeyWindowTokensIsolatesKeys(t *testing.T) {
|
||||
s := NewStats(100)
|
||||
now := time.Now().UnixMilli()
|
||||
s.Record(Req{Time: now, Key: "keyA", Model: "m1", Prompt: 500, Compl: 500, OK: true, Status: 200})
|
||||
s.Record(Req{Time: now, Key: "keyB", Model: "m1", Prompt: 500, Compl: 500, OK: true, Status: 200})
|
||||
|
||||
if got := s.KeyWindowModelTokens("keyA", "m1", "", 3600); got != 1000 {
|
||||
t.Errorf("keyA model tokens = %d, want 1000 (its own usage only)", got)
|
||||
}
|
||||
if got := s.KeyWindowModelTokens("keyB", "m1", "", 3600); got != 1000 {
|
||||
t.Errorf("keyB model tokens = %d, want 1000", got)
|
||||
}
|
||||
// the key-blind bucket stays global on purpose: it backs the AUTO slot
|
||||
// quota, which limits the whole gateway, not one key.
|
||||
if got := s.WindowTokens("m1", "", 3600); got != 2000 {
|
||||
t.Errorf("WindowTokens = %d, want 2000 (global per-model total must be unchanged)", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A source pin scopes the cap to that one upstream. The pinned bucket is
|
||||
// maintained lazily (it is only read by quotas that actually pin a source), so
|
||||
// usage recorded before anything queried the pin is not attributed to it.
|
||||
func TestKeyWindowTokensSeparatesSourcePin(t *testing.T) {
|
||||
s := NewStats(100)
|
||||
now := time.Now().UnixMilli()
|
||||
s.Record(Req{Time: now, Key: "keyA", Model: "m1", Source: "srcX", Prompt: 100, Compl: 100, OK: true, Status: 200}) // 200 tok
|
||||
s.Record(Req{Time: now, Key: "keyA", Model: "m1", Source: "srcY", Prompt: 200, Compl: 200, OK: true, Status: 200}) // 400 tok
|
||||
|
||||
// The unpinned bucket always counts — it is what a scope entry without a
|
||||
// source pin reads. 200 + 400.
|
||||
if got := s.KeyWindowModelTokens("keyA", "m1", "", 3600); got != 600 {
|
||||
t.Errorf("unpinned model tokens = %d, want 600 (both sources)", got)
|
||||
}
|
||||
|
||||
// Reading a pin opts this (key, model) into pinned accounting.
|
||||
if got := s.KeyWindowModelTokens("keyA", "m1", "srcX", 3600); got != 0 {
|
||||
t.Errorf("srcX pin before opt-in = %d, want 0 (the lazy bucket has not accrued yet)", got)
|
||||
}
|
||||
|
||||
// From now on both pins accrue.
|
||||
more := now + 1
|
||||
s.Record(Req{Time: more, Key: "keyA", Model: "m1", Source: "srcX", Prompt: 10, Compl: 10, OK: true, Status: 200}) // 20 tok
|
||||
s.Record(Req{Time: more, Key: "keyA", Model: "m1", Source: "srcY", Prompt: 30, Compl: 30, OK: true, Status: 200}) // 60 tok
|
||||
if got := s.KeyWindowModelTokens("keyA", "m1", "srcX", 3600); got != 20 {
|
||||
t.Errorf("srcX-pinned tokens after opt-in = %d, want 20", got)
|
||||
}
|
||||
if got := s.KeyWindowModelTokens("keyA", "m1", "srcY", 3600); got != 60 {
|
||||
t.Errorf("srcY-pinned tokens = %d, want 60 (a pin read for one source must not blind the other)", got)
|
||||
}
|
||||
}
|
||||
|
||||
// This is the shape of every real chat record: Source is always populated.
|
||||
// The unpinned bucket must still see the tokens, or a per-model quota without
|
||||
// a source pin reads an empty bucket and never trips.
|
||||
func TestKeyWindowModelTokensUnpinnedSeesSourcedTraffic(t *testing.T) {
|
||||
s := NewStats(100)
|
||||
s.Record(Req{Time: time.Now().UnixMilli(), Key: "keyA", Model: "m1", Source: "deepseek",
|
||||
Prompt: 1000, Compl: 1000, OK: true, Status: 200})
|
||||
|
||||
if got := s.KeyWindowModelTokens("keyA", "m1", "", 3600); got != 2000 {
|
||||
t.Errorf("unpinned tokens = %d, want 2000 — an unpinned per-model quota would never trip otherwise", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestKeyWindowRespectsResetWindow(t *testing.T) {
|
||||
s := NewStats(100)
|
||||
old := time.Now().Add(-30 * 24 * time.Hour).UnixMilli()
|
||||
s.Record(Req{Time: old, Key: "keyA", Model: "m1", Prompt: 100, Compl: 100, OK: true, Status: 200})
|
||||
s.Record(Req{Time: time.Now().UnixMilli(), Key: "keyA", Model: "m1", Prompt: 5, Compl: 5, OK: true, Status: 200})
|
||||
|
||||
if got := s.KeyWindowTokens("keyA", 3600); got != 10 {
|
||||
t.Errorf("1h window = %d, want 10 (30-day-old usage must not count)", got)
|
||||
}
|
||||
if got := s.KeyWindowTokens("keyA", 0); got != 210 {
|
||||
t.Errorf("all-time total = %d, want 210", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestKeyWindowReqsCountsEveryRequest(t *testing.T) {
|
||||
s := NewStats(100)
|
||||
now := time.Now().UnixMilli()
|
||||
// a failed request and an image request both count: a client that loops on
|
||||
// failures must still burn its request quota
|
||||
s.Record(Req{Time: now, Key: "keyA", Model: "m1", Type: "chat", OK: true, Status: 200})
|
||||
s.Record(Req{Time: now, Key: "keyA", Model: "m1", Type: "chat", OK: false, Status: 500})
|
||||
s.Record(Req{Time: now, Key: "keyA", Model: "img", Type: "image", OK: true, Status: 200})
|
||||
s.Record(Req{Time: now, Key: "keyB", Model: "m1", Type: "chat", OK: true, Status: 200})
|
||||
|
||||
if got := s.KeyWindowReqs("keyA", 3600); got != 3 {
|
||||
t.Errorf("keyA reqs = %d, want 3 (chat ok + chat fail + image)", got)
|
||||
}
|
||||
if got := s.KeyWindowReqs("keyB", 3600); got != 1 {
|
||||
t.Errorf("keyB reqs = %d, want 1", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestKeyWindowTokensZeroTokensNotCounted(t *testing.T) {
|
||||
s := NewStats(100)
|
||||
now := time.Now().UnixMilli()
|
||||
// a request that reported no usage must not create a bucket entry
|
||||
s.Record(Req{Time: now, Key: "keyA", Model: "", Type: "chat", OK: true, Status: 200})
|
||||
if got := s.KeyWindowTokens("keyA", 3600); got != 0 {
|
||||
t.Errorf("tokens = %d, want 0", got)
|
||||
}
|
||||
if got := s.KeyWindowReqs("keyA", 3600); got != 1 {
|
||||
t.Errorf("reqs = %d, want 1 (request still happened)", got)
|
||||
}
|
||||
}
|
||||
|
||||
// ---- reset window arithmetic ----
|
||||
|
||||
func TestAutoSecondsToReset(t *testing.T) {
|
||||
if got := AutoSecondsToReset("", 0); got != 0 {
|
||||
t.Errorf("no period = %d, want 0 (never resets -> no retry hint)", got)
|
||||
}
|
||||
now := time.Now().Unix()
|
||||
for _, tc := range []struct {
|
||||
period string
|
||||
hours int64
|
||||
want int64
|
||||
}{
|
||||
{"hour", 0, 3600},
|
||||
{"week", 0, 7 * 24 * 3600},
|
||||
{"month", 0, 30 * 24 * 3600},
|
||||
{"nhour", 6, 6 * 3600},
|
||||
} {
|
||||
got := AutoSecondsToReset(tc.period, tc.hours)
|
||||
if got <= 0 || got > tc.want {
|
||||
t.Errorf("AutoSecondsToReset(%q,%d) = %d, want in (0,%d]", tc.period, tc.hours, got, tc.want)
|
||||
}
|
||||
// must never exceed the window itself
|
||||
if got < tc.want-now%3600 {
|
||||
t.Logf("note: %q hint %ds < remaining-window %ds (rounds to hour boundary)", tc.period, got, tc.want-now%3600)
|
||||
}
|
||||
if got > tc.want {
|
||||
t.Errorf("hint %d exceeds window %d", got, tc.want)
|
||||
}
|
||||
}
|
||||
_ = now
|
||||
}
|
||||
|
||||
func TestAutoSecondsToResetNeverExceedsWindow(t *testing.T) {
|
||||
// "nhour" with a tiny window must not hand out a longer wait than the
|
||||
// window itself (which would stall a client past its own reset)
|
||||
for hours := int64(1); hours <= 48; hours++ {
|
||||
got := AutoSecondsToReset("nhour", hours)
|
||||
if got <= 0 || got > hours*3600 {
|
||||
t.Errorf("nhour/%d = %d, want in (0,%d]", hours, got, hours*3600)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---- config validation ----
|
||||
|
||||
func TestKeyQuotaValidate(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
q config.KeyQuota
|
||||
wantErr bool
|
||||
}{
|
||||
{"unlimited", config.KeyQuota{}, false},
|
||||
{"tokens hourly", config.KeyQuota{TokenQuota: 1000, Period: "hour"}, false},
|
||||
{"tokens n-hour", config.KeyQuota{TokenQuota: 1000, Period: "nhour", Hours: 6}, false},
|
||||
{"reqs weekly", config.KeyQuota{ReqQuota: 100, Period: "week"}, false},
|
||||
{"no period never resets", config.KeyQuota{TokenQuota: 1000}, false},
|
||||
{"negative tokens", config.KeyQuota{TokenQuota: -1}, true},
|
||||
{"negative reqs", config.KeyQuota{ReqQuota: -1}, true},
|
||||
{"negative hours", config.KeyQuota{TokenQuota: 5, Hours: -1}, true},
|
||||
{"typo period becomes all-time if accepted", config.KeyQuota{TokenQuota: 5, Period: "houre"}, true},
|
||||
{"nhour without hours", config.KeyQuota{TokenQuota: 5, Period: "nhour"}, true},
|
||||
{"nhour with 0 hours", config.KeyQuota{TokenQuota: 5, Period: "nhour", Hours: 0}, true},
|
||||
{"junk period with only req quota", config.KeyQuota{ReqQuota: 5, Period: "daily"}, true},
|
||||
{"junk period with no caps is irrelevant", config.KeyQuota{Period: "daily"}, false},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
err := tc.q.Validate()
|
||||
if tc.wantErr && err == nil {
|
||||
t.Errorf("%s: expected error, got nil", tc.name)
|
||||
}
|
||||
if !tc.wantErr && err != nil {
|
||||
t.Errorf("%s: unexpected error: %v", tc.name, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizeRole(t *testing.T) {
|
||||
if got := config.NormalizeRole(""); got != "user" {
|
||||
t.Errorf("empty role = %q, want user", got)
|
||||
}
|
||||
if got := config.NormalizeRole("admin"); got != "admin" {
|
||||
t.Errorf("admin role = %q, want admin", got)
|
||||
}
|
||||
if got := config.NormalizeRole("root"); got != "user" {
|
||||
t.Errorf("unknown role = %q, want user (never escalate)", got)
|
||||
}
|
||||
}
|
||||
|
||||
// Retention must hold for the lazy pinned buckets too, or a long-lived key
|
||||
// would grow without bound.
|
||||
func TestPinnedBucketRespectsRetention(t *testing.T) {
|
||||
s := NewStats(100)
|
||||
_ = s.KeyWindowModelTokens("keyA", "m1", "srcX", 3600) // opt in
|
||||
nowH := time.Now().Unix() / 3600
|
||||
for h := int64(0); h < quotaRetentionHours+50; h++ {
|
||||
s.Record(Req{Time: (nowH - h) * 3600 * 1000, Key: "keyA", Model: "m1", Source: "srcX",
|
||||
Prompt: 10, Compl: 10, OK: true, Status: 200})
|
||||
}
|
||||
s.mu.Lock()
|
||||
n := len(s.keyModelHour["keyA"]["srcX::m1"])
|
||||
s.mu.Unlock()
|
||||
if n > quotaRetentionHours {
|
||||
t.Errorf("pinned bucket holds %d hours, want <= %d", n, quotaRetentionHours)
|
||||
}
|
||||
}
|
||||
@ -1,444 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/core"
|
||||
)
|
||||
|
||||
// quotaGateway builds a gateway with two user keys that differ only in their
|
||||
// configured caps, against one mock upstream that reports 4 tokens per call.
|
||||
func quotaGateway(t *testing.T, keyA, keyB config.GWKey) (*Gateway, *upstreamCtrl) {
|
||||
t.Helper()
|
||||
ctrl := &upstreamCtrl{}
|
||||
up := upstream(t, ctrl)
|
||||
t.Cleanup(up.Close)
|
||||
|
||||
td := t.TempDir()
|
||||
cfgPath := td + "/config.yaml"
|
||||
if err := os.WriteFile(cfgPath, []byte("listen: :0"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg := &config.Config{
|
||||
Path: cfgPath,
|
||||
AdapterDir: td + "/adapters",
|
||||
RuntimeFile: td + "/runtime.json",
|
||||
Keys: []config.GWKey{keyA, keyB},
|
||||
Sources: []config.Source{{
|
||||
Name: "up", BaseURL: up.URL, Adapter: "openai",
|
||||
Models: []config.Model{{ID: "m1", Priority: 100}},
|
||||
}},
|
||||
}
|
||||
if err := cfg.ApplyDefaults(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := core.NewFromConfig(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("core: %v", err)
|
||||
}
|
||||
t.Cleanup(c.Close)
|
||||
g, err := New(c)
|
||||
if err != nil {
|
||||
t.Fatalf("gateway: %v", err)
|
||||
}
|
||||
return g, ctrl
|
||||
}
|
||||
|
||||
// chatAs issues a chat completion with a specific gateway key and returns the
|
||||
// recorder plus the decoded error code.
|
||||
func chatAs(t *testing.T, g *Gateway, key, model string) (*httptest.ResponseRecorder, string) {
|
||||
t.Helper()
|
||||
req, _ := http.NewRequest("POST", "/v1/chat/completions",
|
||||
strings.NewReader(fmt.Sprintf(`{"model":%q,"messages":[{"role":"user","content":"hi"}]}`, model)))
|
||||
req.Header.Set("Authorization", "Bearer "+key)
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
rr := httptest.NewRecorder()
|
||||
g.Handler().ServeHTTP(rr, req)
|
||||
code := ""
|
||||
if rr.Code != 200 {
|
||||
var e struct {
|
||||
Error struct {
|
||||
Type string `json:"type"`
|
||||
} `json:"error"`
|
||||
}
|
||||
_ = json.Unmarshal(rr.Body.Bytes(), &e)
|
||||
code = e.Error.Type
|
||||
}
|
||||
return rr, code
|
||||
}
|
||||
|
||||
// A key whose token budget is spent must be refused, and the refusal must be a
|
||||
// 429 the client can retry after the window rolls over — not a 403 that reads
|
||||
// as "this key may never use this model".
|
||||
func TestKeyTokenQuotaBlocksWithRetryAfter(t *testing.T) {
|
||||
g, _ := quotaGateway(t,
|
||||
config.GWKey{Key: "sk-a", Role: "user",
|
||||
Models: []config.ModelScope{{Model: "m1", TokenQuota: 8, Period: "hour"}}},
|
||||
config.GWKey{Key: "sk-b", Role: "user", Name: "b"},
|
||||
)
|
||||
// 4 tokens per call, budget 8 -> the third call crosses it
|
||||
for i := 1; i <= 2; i++ {
|
||||
rr, _ := chatAs(t, g, "sk-a", "m1")
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("call %d: want 200, got %d (%s)", i, rr.Code, rr.Body.String())
|
||||
}
|
||||
}
|
||||
rr, code := chatAs(t, g, "sk-a", "m1")
|
||||
if rr.Code != http.StatusTooManyRequests {
|
||||
t.Fatalf("third call: want 429, got %d (%s)", rr.Code, rr.Body.String())
|
||||
}
|
||||
if code != "rate_limit_exceeded" {
|
||||
t.Errorf("error code = %q, want rate_limit_exceeded", code)
|
||||
}
|
||||
if ra := rr.Header().Get("Retry-After"); ra == "" {
|
||||
t.Error("429 must carry Retry-After so a client knows when to come back")
|
||||
} else if n := mustAtoi(t, ra); n <= 0 || n > 3600 {
|
||||
t.Errorf("Retry-After = %q, want a positive value within the 1h window", ra)
|
||||
}
|
||||
if !strings.Contains(rr.Body.String(), "quota") {
|
||||
t.Errorf("body should name the quota: %s", rr.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
// The quota is per key: exhausting one key's budget must not affect another
|
||||
// key that is allowed the same model.
|
||||
func TestKeyTokenQuotaIsIsolatedPerKey(t *testing.T) {
|
||||
g, _ := quotaGateway(t,
|
||||
config.GWKey{Key: "sk-a", Role: "user",
|
||||
Models: []config.ModelScope{{Model: "m1", TokenQuota: 4, Period: "hour"}}},
|
||||
config.GWKey{Key: "sk-b", Role: "user",
|
||||
Models: []config.ModelScope{{Model: "m1", TokenQuota: 1000, Period: "hour"}}},
|
||||
)
|
||||
rr, _ := chatAs(t, g, "sk-a", "m1")
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("keyA first call: want 200, got %d", rr.Code)
|
||||
}
|
||||
if rr, code := chatAs(t, g, "sk-a", "m1"); rr.Code != http.StatusTooManyRequests {
|
||||
t.Fatalf("keyA second call: want 429, got %d (%s)", rr.Code, code)
|
||||
}
|
||||
// keyB has its own budget and must still be served
|
||||
if rr, _ := chatAs(t, g, "sk-b", "m1"); rr.Code != 200 {
|
||||
t.Fatalf("keyB must be unaffected by keyA's exhausted quota, got %d", rr.Code)
|
||||
}
|
||||
}
|
||||
|
||||
// A per-model quota inside the scope must also be per key: one key burning a
|
||||
// shared model budget must not lock the other key out.
|
||||
func TestScopeModelTokenQuotaIsolatedPerKey(t *testing.T) {
|
||||
g, _ := quotaGateway(t,
|
||||
config.GWKey{Key: "sk-a", Role: "user", Models: []config.ModelScope{
|
||||
{Model: "m1", TokenQuota: 4, Period: "hour"},
|
||||
}},
|
||||
config.GWKey{Key: "sk-b", Role: "user", Models: []config.ModelScope{
|
||||
{Model: "m1", TokenQuota: 4, Period: "hour"},
|
||||
}},
|
||||
)
|
||||
if rr, _ := chatAs(t, g, "sk-a", "m1"); rr.Code != 200 {
|
||||
t.Fatalf("keyA first call: want 200, got %d", rr.Code)
|
||||
}
|
||||
if rr, _ := chatAs(t, g, "sk-a", "m1"); rr.Code != http.StatusTooManyRequests {
|
||||
t.Fatalf("keyA second call: want 429 (its own model quota), got %d", rr.Code)
|
||||
}
|
||||
// keyB shares the model but not the counter — it must be served twice too
|
||||
if rr, _ := chatAs(t, g, "sk-b", "m1"); rr.Code != 200 {
|
||||
t.Fatalf("keyB first call: want 200, got %d", rr.Code)
|
||||
}
|
||||
if rr, _ := chatAs(t, g, "sk-b", "m1"); rr.Code != http.StatusTooManyRequests {
|
||||
t.Fatalf("keyB second call: want 429, got %d", rr.Code)
|
||||
}
|
||||
}
|
||||
|
||||
// An admin key is never capped: a budget set on an admin key must not lock the
|
||||
// operator out of the gateway they administer.
|
||||
func TestAdminKeyIsNeverQuotaCapped(t *testing.T) {
|
||||
g, _ := quotaGateway(t,
|
||||
config.GWKey{Key: "sk-admin", Role: "admin",
|
||||
Models: []config.ModelScope{{Model: "m1", TokenQuota: 1, ReqQuota: 1, Period: "hour"}}},
|
||||
config.GWKey{Key: "sk-b", Role: "user"},
|
||||
)
|
||||
for i := 1; i <= 3; i++ {
|
||||
if rr, _ := chatAs(t, g, "sk-admin", "m1"); rr.Code != 200 {
|
||||
t.Fatalf("admin call %d: want 200 (admin keys are uncapped), got %d", i, rr.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A request quota caps sustained volume; RPM-style bursts are not its job but
|
||||
// the count must still stop the key.
|
||||
func TestKeyRequestQuotaBlocks(t *testing.T) {
|
||||
g, ctrl := quotaGateway(t,
|
||||
config.GWKey{Key: "sk-a", Role: "user",
|
||||
Models: []config.ModelScope{{Model: "m1", ReqQuota: 2, Period: "hour"}}},
|
||||
config.GWKey{Key: "sk-b", Role: "user"},
|
||||
)
|
||||
for i := 1; i <= 2; i++ {
|
||||
if rr, _ := chatAs(t, g, "sk-a", "m1"); rr.Code != 200 {
|
||||
t.Fatalf("call %d: want 200, got %d", i, rr.Code)
|
||||
}
|
||||
}
|
||||
rr, code := chatAs(t, g, "sk-a", "m1")
|
||||
if rr.Code != http.StatusTooManyRequests || code != "rate_limit_exceeded" {
|
||||
t.Fatalf("third call: want 429/rate_limit_exceeded, got %d/%s", rr.Code, code)
|
||||
}
|
||||
if ctrl.hits != 2 {
|
||||
t.Errorf("upstream saw %d calls, want 2 (a refused request must not reach upstream)", ctrl.hits)
|
||||
}
|
||||
}
|
||||
|
||||
// A model outside the scope is still 403, not 429: retrying cannot help.
|
||||
func TestModelOutsideScopeStaysForbidden(t *testing.T) {
|
||||
g, _ := quotaGateway(t,
|
||||
config.GWKey{Key: "sk-a", Role: "user",
|
||||
Models: []config.ModelScope{{Model: "other-model", TokenQuota: 1000, Period: "hour"}}},
|
||||
config.GWKey{Key: "sk-b", Role: "user"},
|
||||
)
|
||||
rr, code := chatAs(t, g, "sk-a", "m1")
|
||||
if rr.Code != http.StatusForbidden {
|
||||
t.Fatalf("want 403, got %d (%s)", rr.Code, rr.Body.String())
|
||||
}
|
||||
if code != "model_not_allowed" {
|
||||
t.Errorf("error code = %q, want model_not_allowed", code)
|
||||
}
|
||||
if rr.Header().Get("Retry-After") != "" {
|
||||
t.Error("a 403 must not advertise a retry time")
|
||||
}
|
||||
}
|
||||
|
||||
// An unlimited key (no caps at all) is never rejected — the default config
|
||||
// must keep working exactly as before.
|
||||
func TestKeyWithoutCapsIsUnlimited(t *testing.T) {
|
||||
g, _ := quotaGateway(t,
|
||||
config.GWKey{Key: "sk-a", Role: "user", Models: []config.ModelScope{{Model: "m1"}}},
|
||||
config.GWKey{Key: "sk-b", Role: "user"},
|
||||
)
|
||||
for i := 1; i <= 5; i++ {
|
||||
if rr, _ := chatAs(t, g, "sk-a", "m1"); rr.Code != 200 {
|
||||
t.Fatalf("call %d: want 200 (no caps = unlimited), got %d", i, rr.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func mustAtoi(t *testing.T, s string) int64 {
|
||||
t.Helper()
|
||||
var n int64
|
||||
if _, err := fmt.Sscanf(s, "%d", &n); err != nil {
|
||||
t.Fatalf("Retry-After %q is not an integer: %v", s, err)
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// quotaCtx builds a request context carrying a gateway key, as the auth
|
||||
// middleware would.
|
||||
func quotaCtx(t *testing.T, g *Gateway, key string) context.Context {
|
||||
t.Helper()
|
||||
return withAuth(context.Background(), key, "user")
|
||||
}
|
||||
|
||||
func nowMSOffset(sec int64) int64 {
|
||||
return time.Now().Add(time.Duration(sec) * time.Second).UnixMilli()
|
||||
}
|
||||
|
||||
// The key-wide quota and the AUTO slot quota are different limits with
|
||||
// different scopes: the slot quota is gateway-wide (a shared upstream budget),
|
||||
// the key quota belongs to one caller. When both are exhausted the caller must
|
||||
// see the KEY quota, because that is the one it can act on — the slot quota
|
||||
// would otherwise be reported as "no capacity", which reads like an outage.
|
||||
func TestKeyQuotaWinsOverSlotQuota(t *testing.T) {
|
||||
up := upstream(t, &upstreamCtrl{})
|
||||
defer up.Close()
|
||||
g := newQuotaGW(t, up.URL,
|
||||
config.GWKey{Key: "sk-a", Role: "user",
|
||||
Models: []config.ModelScope{{Model: "AUTO", TokenQuota: 4, Period: "hour"}}})
|
||||
ctx := quotaCtx(t, g, "sk-a")
|
||||
|
||||
// exhaust the key first
|
||||
g.stats.Record(Req{Time: time.Now().UnixMilli(), Key: keyID("sk-a"), Model: "m1",
|
||||
Source: "up", Prompt: 100, Compl: 100, OK: true, Status: 200})
|
||||
rr, code := chatAs(t, g, "sk-a", "AUTO")
|
||||
if rr.Code != http.StatusTooManyRequests {
|
||||
t.Fatalf("want 429 from the key quota, got %d (%s)", rr.Code, rr.Body.String())
|
||||
}
|
||||
if code != "rate_limit_exceeded" {
|
||||
t.Errorf("code = %q, want rate_limit_exceeded", code)
|
||||
}
|
||||
_ = ctx
|
||||
}
|
||||
|
||||
// A key with NO caps must never be blocked by the slot quota check reaching it
|
||||
// through the scope path: scopeTokens on an uncapped key returns 0 usage and
|
||||
// the limit check must treat that as "no cap", not "exhausted".
|
||||
func TestUncappedKeyNeverBlockedByEmptyScope(t *testing.T) {
|
||||
up := upstream(t, &upstreamCtrl{})
|
||||
defer up.Close()
|
||||
g := newQuotaGW(t, up.URL,
|
||||
config.GWKey{Key: "sk-a", Role: "user", Models: []config.ModelScope{{Model: "AUTO"}}})
|
||||
for i := 0; i < 5; i++ {
|
||||
if rr, _ := chatAs(t, g, "sk-a", "AUTO"); rr.Code != 200 {
|
||||
t.Fatalf("call %d: want 200 (AUTO scope with no quota), got %d (%s)", i, rr.Code, rr.Body.String())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// newQuotaGW builds a gateway with one mock upstream serving model m1 and the
|
||||
// given keys.
|
||||
func newQuotaGW(t *testing.T, upURL string, keys ...config.GWKey) *Gateway {
|
||||
t.Helper()
|
||||
td := t.TempDir()
|
||||
cfgPath := filepath.Join(td, "config.yaml")
|
||||
if err := os.WriteFile(cfgPath, []byte("listen: :0"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg := &config.Config{
|
||||
Path: cfgPath,
|
||||
AdapterDir: filepath.Join(td, "adapters"),
|
||||
RuntimeFile: filepath.Join(td, "runtime.json"),
|
||||
Keys: keys,
|
||||
Sources: []config.Source{{
|
||||
Name: "up", BaseURL: upURL, Adapter: "openai",
|
||||
Models: []config.Model{
|
||||
{ID: "m1", Priority: 100},
|
||||
{ID: "m2", Priority: 90},
|
||||
},
|
||||
}},
|
||||
}
|
||||
if err := cfg.ApplyDefaults(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := core.NewFromConfig(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("core: %v", err)
|
||||
}
|
||||
t.Cleanup(c.Close)
|
||||
secrets := make([]string, 0, len(keys))
|
||||
for _, k := range keys {
|
||||
secrets = append(secrets, k.Key)
|
||||
}
|
||||
g, err := New(c)
|
||||
if err != nil {
|
||||
t.Fatalf("gateway: %v", err)
|
||||
}
|
||||
return g
|
||||
}
|
||||
|
||||
// The core of per-model quotas: a model that runs out of budget must stop
|
||||
// being served on its own, while every other model on the SAME key keeps
|
||||
// working. A key-wide total would fail this — it would block m2 because m1 was
|
||||
// capped, which is exactly the coupling this design removes.
|
||||
func TestOneModelsQuotaDoesNotBlockAnother(t *testing.T) {
|
||||
up := upstream(t, &upstreamCtrl{})
|
||||
defer up.Close()
|
||||
td := t.TempDir()
|
||||
cfgPath := td + "/config.yaml"
|
||||
if err := os.WriteFile(cfgPath, []byte("listen: :0"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg := &config.Config{
|
||||
Path: cfgPath, AdapterDir: filepath.Join(td, "adapters"),
|
||||
RuntimeFile: filepath.Join(td, "runtime.json"),
|
||||
Keys: []config.GWKey{{
|
||||
Key: "sk-a", Role: "user", Name: "two-models",
|
||||
Models: []config.ModelScope{
|
||||
{Model: "m1", TokenQuota: 4, Period: "hour"},
|
||||
{Model: "m2", TokenQuota: 1000, Period: "hour"},
|
||||
},
|
||||
}},
|
||||
Sources: []config.Source{{
|
||||
Name: "up", BaseURL: up.URL, Adapter: "openai",
|
||||
Models: []config.Model{
|
||||
{ID: "m1", Priority: 100},
|
||||
{ID: "m2", Priority: 90},
|
||||
},
|
||||
}},
|
||||
}
|
||||
if err := cfg.ApplyDefaults(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := core.NewFromConfig(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("core: %v", err)
|
||||
}
|
||||
t.Cleanup(c.Close)
|
||||
g, err := New(c)
|
||||
if err != nil {
|
||||
t.Fatalf("gateway: %v", err)
|
||||
}
|
||||
|
||||
// Spend on m2 FIRST. Without this the two designs are
|
||||
// indistinguishable: a key-wide counter and m1's own counter would both
|
||||
// read 0 before m1 is used, so the test would pass either way (it did —
|
||||
// see the commit that rewrote it).
|
||||
for i := 1; i <= 3; i++ {
|
||||
if rr, _ := chatAs(t, g, "sk-a", "m2"); rr.Code != 200 {
|
||||
t.Fatalf("m2 priming call %d: want 200, got %d", i, rr.Code)
|
||||
}
|
||||
}
|
||||
// m1's budget is 4 and each call costs 4, so the key-wide total (m2+m1) is
|
||||
// already 12 when m1 starts: a key-wide cap would refuse m1 immediately.
|
||||
if rr, _ := chatAs(t, g, "sk-a", "m1"); rr.Code != 200 {
|
||||
t.Fatalf("m1 must be served on its own budget (m2's spend is not its problem), got %d (%s)",
|
||||
rr.Code, rr.Body.String())
|
||||
}
|
||||
if rr, code := chatAs(t, g, "sk-a", "m1"); rr.Code != http.StatusTooManyRequests {
|
||||
t.Fatalf("m1 second call: want 429, got %d (%s)", rr.Code, code)
|
||||
}
|
||||
// m2 must keep working
|
||||
for i := 1; i <= 3; i++ {
|
||||
if rr, _ := chatAs(t, g, "sk-a", "m2"); rr.Code != 200 {
|
||||
t.Fatalf("m2 call %d must be served while m1 is capped, got %d (%s)", i, rr.Code, rr.Body.String())
|
||||
}
|
||||
}
|
||||
// and the message must name m1, not the key
|
||||
rr, _ := chatAs(t, g, "sk-a", "m1")
|
||||
if !strings.Contains(rr.Body.String(), "m1") {
|
||||
t.Errorf("rejection should name the capped model: %s", rr.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
// A model with no quota on it is never blocked by a sibling's cap, and an
|
||||
// uncapped key is never blocked at all.
|
||||
func TestUncappedModelNeverBlocked(t *testing.T) {
|
||||
up := upstream(t, &upstreamCtrl{})
|
||||
defer up.Close()
|
||||
g := newQuotaGW(t, up.URL,
|
||||
config.GWKey{Key: "sk-a", Role: "user", Models: []config.ModelScope{
|
||||
{Model: "m1", TokenQuota: 1, Period: "hour"},
|
||||
{Model: "m2"},
|
||||
}})
|
||||
if rr, _ := chatAs(t, g, "sk-a", "m1"); rr.Code != 200 {
|
||||
t.Fatalf("m1 first: want 200, got %d", rr.Code)
|
||||
}
|
||||
if rr, _ := chatAs(t, g, "sk-a", "m1"); rr.Code != http.StatusTooManyRequests {
|
||||
t.Fatalf("m1 second: want 429, got %d", rr.Code)
|
||||
}
|
||||
for i := 1; i <= 4; i++ {
|
||||
if rr, _ := chatAs(t, g, "sk-a", "m2"); rr.Code != 200 {
|
||||
t.Fatalf("m2 (uncapped) call %d: want 200, got %d", i, rr.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// An admin key is never capped even when its scopes carry budgets: a cap that
|
||||
// locked the operator out would be unrecoverable through the UI.
|
||||
func TestAdminKeyScopesAreNotEnforced(t *testing.T) {
|
||||
up := upstream(t, &upstreamCtrl{})
|
||||
defer up.Close()
|
||||
g := newQuotaGW(t, up.URL,
|
||||
config.GWKey{Key: "sk-admin", Role: "admin", Models: []config.ModelScope{
|
||||
{Model: "m1", TokenQuota: 1, ReqQuota: 1, Period: "hour"},
|
||||
}})
|
||||
for i := 1; i <= 4; i++ {
|
||||
if rr, _ := chatAs(t, g, "sk-admin", "m1"); rr.Code != 200 {
|
||||
t.Fatalf("admin call %d: want 200 (admin scopes are not enforced), got %d", i, rr.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -7,7 +7,6 @@ import (
|
||||
"strings"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/scheduler"
|
||||
)
|
||||
|
||||
// handleKeysAPI manages gateway keys: GET /api/keys (admin: all keys),
|
||||
@ -18,22 +17,10 @@ func (g *Gateway) handleKeysAPI(w http.ResponseWriter, r *http.Request) {
|
||||
path = strings.Trim(path, "/")
|
||||
role := reqRole(r.Context())
|
||||
|
||||
// /api/keys/me/auto — a user's own chain, readable without admin rights.
|
||||
// Must be matched before the generic {key}/auto case below so it does not
|
||||
// ask admin rights of a plain user asking about their own chain.
|
||||
if path == "me/auto" {
|
||||
g.handleKeyMeAuto(w, r)
|
||||
return
|
||||
}
|
||||
if path == "me" {
|
||||
g.handleKeyMe(w, r)
|
||||
return
|
||||
}
|
||||
// /api/keys/{key}/auto — the per-key AUTO chain editor.
|
||||
if i := strings.LastIndex(path, "/auto"); i > 0 && path[i+len("/auto"):] == "" {
|
||||
g.handleKeyAutoAPI(w, r, path[:i])
|
||||
return
|
||||
}
|
||||
if role != "admin" {
|
||||
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
|
||||
return
|
||||
@ -61,7 +48,13 @@ func (g *Gateway) handleKeysAPI(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
|
||||
return
|
||||
}
|
||||
body.Role = config.NormalizeRole(body.Role)
|
||||
if body.Role == "" {
|
||||
body.Role = "user"
|
||||
}
|
||||
if body.Role != "admin" && body.Role != "user" {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "role must be admin or user")
|
||||
return
|
||||
}
|
||||
rec, err := g.core.CreateKey(body.Name, body.Role, body.Models, body.Note)
|
||||
if err != nil {
|
||||
writeError(w, http.StatusBadRequest, "key_error", err.Error())
|
||||
@ -83,9 +76,6 @@ func (g *Gateway) handleKeysAPI(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
|
||||
return
|
||||
}
|
||||
// Each scope entry carries its own token/request caps, so replacing the
|
||||
// scope replaces the budgets with it — there is no separate key-wide
|
||||
// quota that could drift out of sync with the models.
|
||||
rec, err := g.core.UpdateKey(path, body.Name, body.Role, body.Models, body.Note)
|
||||
if err != nil {
|
||||
writeError(w, http.StatusBadRequest, "key_error", err.Error())
|
||||
@ -152,101 +142,6 @@ func (g *Gateway) allowedModels(ctx context.Context) []config.ModelScope {
|
||||
return rec.Models
|
||||
}
|
||||
|
||||
// handleKeyAutoAPI manages one key's own AUTO chain:
|
||||
// GET /api/keys/{key}/auto returns it, PUT replaces it, DELETE clears the
|
||||
// override so the key falls back to the global chain. Admin only — a user can
|
||||
// inspect their own chain through GET /api/keys/me/auto.
|
||||
//
|
||||
// An empty chain and a cleared chain are deliberately the same state: "inherit
|
||||
// the global chain". That keeps the WebUI's "use global" toggle a plain DELETE
|
||||
// with no sentinel value to carry through config.yaml.
|
||||
func (g *Gateway) handleKeyAutoAPI(w http.ResponseWriter, r *http.Request, key string) {
|
||||
if reqRole(r.Context()) != "admin" {
|
||||
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
|
||||
return
|
||||
}
|
||||
rec, ok := g.core.FindKey(key)
|
||||
if !ok {
|
||||
writeError(w, http.StatusNotFound, "not_found", "key not found")
|
||||
return
|
||||
}
|
||||
switch r.Method {
|
||||
case http.MethodGet:
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{
|
||||
"key": key,
|
||||
"auto": rec.Auto,
|
||||
"inherits": !rec.HasOwnAuto(),
|
||||
})
|
||||
case http.MethodPut, http.MethodPatch:
|
||||
var body struct {
|
||||
Auto []config.ModelScope `json:"auto"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
|
||||
return
|
||||
}
|
||||
if len(body.Auto) == 0 {
|
||||
// Treat an empty PUT as "give up the override" so the endpoint
|
||||
// cannot persist a chain that would 503 with no slots.
|
||||
if err := g.core.SaveKeyAuto(key, nil); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "key_error", err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{"ok": true, "inherits": true})
|
||||
return
|
||||
}
|
||||
if err := g.core.SaveKeyAuto(key, body.Auto); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "key_error", err.Error())
|
||||
return
|
||||
}
|
||||
// Report whether the chain actually resolved to usable slots. An
|
||||
// admin who typed a model that no longer exists should learn it here,
|
||||
// not from the user's next 503.
|
||||
chain, _ := g.core.AutoChainFor(key)
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{
|
||||
"ok": true, "inherits": false, "slots": chainSlots(chain),
|
||||
})
|
||||
case http.MethodDelete:
|
||||
if err := g.core.SaveKeyAuto(key, nil); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "key_error", err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{"ok": true, "inherits": true})
|
||||
default:
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "")
|
||||
}
|
||||
}
|
||||
|
||||
// chainSlots counts the usable slots in a chain, for API feedback after an
|
||||
// edit. A chain that compiled to zero slots is reported so the caller can warn
|
||||
// instead of letting the next request fail with no_provider.
|
||||
func chainSlots(ch *scheduler.Chain) int {
|
||||
if ch == nil {
|
||||
return 0
|
||||
}
|
||||
n := 0
|
||||
for _, tn := range ch.Tiers {
|
||||
n += len(tn.Slots)
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// handleKeyMeAuto lets a user read their own AUTO chain without admin rights.
|
||||
func (g *Gateway) handleKeyMeAuto(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet {
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
|
||||
return
|
||||
}
|
||||
rec, ok := g.core.FindKey(reqKey(r.Context()))
|
||||
if !ok {
|
||||
writeError(w, http.StatusUnauthorized, "invalid_api_key", "key not found")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{
|
||||
"key": rec.Key, "auto": rec.Auto, "inherits": !rec.HasOwnAuto(),
|
||||
})
|
||||
}
|
||||
|
||||
// handleAutoAPI manages the AUTO scheduling slots: GET /api/auto returns the
|
||||
// current rules; PUT /api/auto replaces them (admin only).
|
||||
func (g *Gateway) handleAutoAPI(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
@ -94,74 +94,3 @@ func TestOverflowPhraseSurvivesTruncationWidth(t *testing.T) {
|
||||
t.Fatalf("归一化标记丢失: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// piNonOverflowRe 是 @earendil-works/pi-ai 的 NON_OVERFLOW_PATTERNS。
|
||||
// pi 的 isContextOverflow 先查这一组，命中则直接判为「非超窗」——
|
||||
// 即使 OVERFLOW_PATTERNS 也命中，也不会压缩重试。
|
||||
var piNonOverflowRe = []*regexp.Regexp{
|
||||
regexp.MustCompile(`^(Throttling error|Service unavailable):`),
|
||||
regexp.MustCompile(`(?i)rate limit`),
|
||||
regexp.MustCompile(`(?i)too many requests`),
|
||||
}
|
||||
|
||||
// piWouldCompact 复刻 isContextOverflow 的 Case 1 判定顺序：
|
||||
// 先排除 NON_OVERFLOW，再匹配 OVERFLOW。
|
||||
func piWouldCompact(errorMessage string) bool {
|
||||
for _, p := range piNonOverflowRe {
|
||||
if p.MatchString(errorMessage) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return piOverflowRe.MatchString(errorMessage)
|
||||
}
|
||||
|
||||
// issue 回归：AUTO 链失败消息天然把「超窗 tier」和「配额/限流 tier」拼在一起，
|
||||
// 超窗标记后面只要出现 rate limit / too many requests 的字样，
|
||||
// pi 就会把整条消息判为「非超窗」而拒绝压缩重试。
|
||||
func TestOverflowMessageSurvivesRateLimitedSiblingTier(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
sibling string
|
||||
}{
|
||||
{"rate limit 措辞", "api error 429: rate limit exceeded"},
|
||||
{"too many requests 措辞", "429 Too Many Requests"},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
err := &scheduler.ChainErr{Tiers: []scheduler.TierError{
|
||||
{Tier: 1, Source: "sensenova", Model: "glm-5.2", Err: errors.New(tc.sibling)},
|
||||
{Tier: 3, Source: "gozen", Model: "deepseek-v4.1-flash",
|
||||
Err: errors.New("api error 400: This model's maximum context length is 1048576 tokens")},
|
||||
}}
|
||||
|
||||
got := clientUpstreamErr(err)
|
||||
if !piWouldCompact(got) {
|
||||
t.Fatalf("同链限流 tier 封杀了超窗识别，pi 不会压缩重试: %q", got)
|
||||
}
|
||||
// 干净：不得把限流 tier 的措辞带出去。
|
||||
for _, p := range piNonOverflowRe {
|
||||
if p.MatchString(got) {
|
||||
t.Fatalf("超窗消息仍带限流措辞 %v: %q", p, got)
|
||||
}
|
||||
}
|
||||
// 但仍要能看出是哪个源超的窗。
|
||||
if !strings.Contains(got, "gozen/deepseek-v4.1-flash") {
|
||||
t.Errorf("超窗源信息丢失: %q", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// 直连超窗同样要干净可用。
|
||||
func TestDirectOverflowMessageIsClean(t *testing.T) {
|
||||
raw := "api error 400: Error from provider (Console Go): This model's maximum context length is 1048576 tokens. " +
|
||||
"However, you requested 1440050 tokens. Please reduce the length of the messages or completion."
|
||||
got := clientUpstreamErr(errors.New(raw))
|
||||
if !piWouldCompact(got) {
|
||||
t.Fatalf("直连超窗未被 pi 识别: %q", got)
|
||||
}
|
||||
if !strings.Contains(got, overflowCanonical) {
|
||||
t.Fatalf("缺少归一化标记: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,626 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/core"
|
||||
)
|
||||
|
||||
// newFailingServer serves handler and returns its test server.
|
||||
func newFailingServer(h http.Handler) *httptest.Server { return httptest.NewServer(h) }
|
||||
|
||||
// THE TEST THAT WAS MISSING.
|
||||
//
|
||||
// Every other plugin test calls Plugins.Fire() directly. That proves the Lua
|
||||
// runtime works, but it says nothing about whether the GATEWAY actually fires
|
||||
// the stages — the wiring in handleChat / singleChat / writeRec could be
|
||||
// deleted and the whole suite would stay green while no hook ever ran in
|
||||
// production.
|
||||
//
|
||||
// So these tests drive real HTTP requests through the real handler and assert
|
||||
// on what the plugin observed.
|
||||
|
||||
// observed is what a spy plugin recorded. It is written from Lua, so the
|
||||
// counters live in plugin state and are read back through the state API.
|
||||
type spyPayload struct {
|
||||
Stage string
|
||||
Type string
|
||||
Model string
|
||||
Source string
|
||||
OK bool
|
||||
Tokens int64
|
||||
}
|
||||
|
||||
// spyPlugin returns Lua source for a plugin that appends every stage it sees to
|
||||
// `state.seen`. It is generated per test so each test gets a clean log.
|
||||
func spyPlugin() string {
|
||||
return `
|
||||
local plugin = { name = "spy", version = "1.0.0" }
|
||||
plugin.state = { seen = {} }
|
||||
plugin.hooks = {
|
||||
request_start = "on_start",
|
||||
routed = "on_routed",
|
||||
request_end = "on_end",
|
||||
}
|
||||
local function rec(p, stage)
|
||||
table.insert(plugin.state.seen, {
|
||||
stage = stage, type = p.type or "", model = p.model or "",
|
||||
source = p.source or "", ok = p.ok and true or false,
|
||||
tokens = p.prompt_tokens or 0,
|
||||
})
|
||||
end
|
||||
function plugin.on_start(p) rec(p, "request_start") return nil end
|
||||
function plugin.on_routed(p) rec(p, "routed") return nil end
|
||||
function plugin.on_end(p) rec(p, "request_end") return nil end
|
||||
return plugin
|
||||
`
|
||||
}
|
||||
|
||||
// gatewayWithSpy boots a gateway with one mock upstream and the spy plugin
|
||||
// loaded, and returns the gateway plus the upstream URL.
|
||||
func gatewayWithSpy(t *testing.T) (*Gateway, string) {
|
||||
t.Helper()
|
||||
up := mockUpstream()
|
||||
t.Cleanup(up.Close)
|
||||
|
||||
dir := t.TempDir()
|
||||
cfgPath := filepath.Join(dir, "config.yaml")
|
||||
cfgBody := "listen: :0\n" +
|
||||
"adapter_dir: " + filepath.Join(dir, "adapters") + "\n" +
|
||||
"plugin_dir: " + filepath.Join(dir, "plugins") + "\n" +
|
||||
"runtime_file: " + filepath.Join(dir, "runtime.json") + "\n" +
|
||||
"gateway_keys:\n - sk-test\n"
|
||||
if err := os.WriteFile(cfgPath, []byte(cfgBody), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, err := config.Load(cfgPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg.Sources = []config.Source{{
|
||||
Name: "spyup",
|
||||
BaseURL: up.URL,
|
||||
Adapter: "openai",
|
||||
APIKey: "sk-x",
|
||||
Models: []config.Model{{ID: "spy-model", Kind: "chat"}},
|
||||
}}
|
||||
if err := cfg.ApplyDefaults(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := core.NewFromConfig(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("core: %v", err)
|
||||
}
|
||||
t.Cleanup(c.Close)
|
||||
if err := c.Plugins().LoadSource("spy", spyPlugin()); err != nil {
|
||||
t.Fatalf("load spy: %v", err)
|
||||
}
|
||||
g, err := New(c)
|
||||
if err != nil {
|
||||
t.Fatalf("gateway: %v", err)
|
||||
}
|
||||
return g, up.URL
|
||||
}
|
||||
|
||||
// seenStages reads the spy's log out of its published state.
|
||||
func seenStages(t *testing.T, g *Gateway) []map[string]interface{} {
|
||||
t.Helper()
|
||||
raw := g.core.Plugins().State("spy")
|
||||
if raw == nil {
|
||||
return nil
|
||||
}
|
||||
b, err := json.Marshal(raw)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var st struct {
|
||||
Seen []map[string]interface{} `json:"seen"`
|
||||
}
|
||||
if err := json.Unmarshal(b, &st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return st.Seen
|
||||
}
|
||||
|
||||
func stageNames(seen []map[string]interface{}) []string {
|
||||
out := make([]string, 0, len(seen))
|
||||
for _, s := range seen {
|
||||
out = append(out, s["stage"].(string))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// TestHooksFireOnRealDirectChat: a plain non-streaming chat through the real
|
||||
// handler must produce request_start, routed and request_end — in that order,
|
||||
// with the real source and model filled in.
|
||||
func TestHooksFireOnRealDirectChat(t *testing.T) {
|
||||
g, _ := gatewayWithSpy(t)
|
||||
rr := doReq(t, g, http.MethodPost, "/v1/chat/completions",
|
||||
`{"model":"spy-model","messages":[{"role":"user","content":"hi"}]}`)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("chat status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
seen := seenStages(t, g)
|
||||
got := stageNames(seen)
|
||||
want := []string{"request_start", "routed", "request_end"}
|
||||
if len(got) != 3 {
|
||||
t.Fatalf("stages = %v, want %v (a hook that never fires is the failure "+
|
||||
"this test exists for)", got, want)
|
||||
}
|
||||
for i := range want {
|
||||
if got[i] != want[i] {
|
||||
t.Fatalf("stage order = %v, want %v", got, want)
|
||||
}
|
||||
}
|
||||
// The payload must be real data, not empty scaffolding.
|
||||
end := seen[2]
|
||||
if end["source"] != "spyup" {
|
||||
t.Errorf("request_end source = %v, want spyup", end["source"])
|
||||
}
|
||||
if end["model"] != "spy-model" {
|
||||
t.Errorf("request_end model = %v, want spy-model", end["model"])
|
||||
}
|
||||
if end["ok"] != true {
|
||||
t.Errorf("request_end ok = %v, want true", end["ok"])
|
||||
}
|
||||
if toInt(t, end["tokens"]) <= 0 {
|
||||
t.Errorf("request_end prompt_tokens = %v, want the mock upstream's real usage", end["tokens"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestHooksFireOnRealStreamChat: streaming is a DIFFERENT code path (the record
|
||||
// is written from a deferred call after the stream ends), so it needs its own
|
||||
// assertion. A hook that only worked on the non-streaming path would be a real
|
||||
// bug for agent clients, which are the main streaming users.
|
||||
func TestHooksFireOnRealStreamChat(t *testing.T) {
|
||||
g, _ := gatewayWithSpy(t)
|
||||
rr := doReq(t, g, http.MethodPost, "/v1/chat/completions",
|
||||
`{"model":"spy-model","stream":true,"messages":[{"role":"user","content":"hi"}]}`)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("stream status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
got := stageNames(seenStages(t, g))
|
||||
if len(got) != 3 {
|
||||
t.Fatalf("stream stages = %v, want all three", got)
|
||||
}
|
||||
if got[0] != "request_start" || got[2] != "request_end" {
|
||||
t.Errorf("stream stage order = %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestHooksFireOnAutoRequest: the AUTO chain path is a third code path. It also
|
||||
// must fire, and routed must report tier -2 (resolved by the chain) rather than
|
||||
// the direct path's -1.
|
||||
func TestHooksFireOnAutoRequest(t *testing.T) {
|
||||
g, _ := gatewayWithSpy(t)
|
||||
// Register the model in the AUTO chain via the admin API.
|
||||
put := doReq(t, g, http.MethodPut, "/api/auto", `{"rules":[{"model":"spy-model"}]}`)
|
||||
if put.Code != http.StatusOK {
|
||||
t.Fatalf("save auto: %d %s", put.Code, put.Body.String())
|
||||
}
|
||||
rr := doReq(t, g, http.MethodPost, "/v1/chat/completions",
|
||||
`{"model":"AUTO","messages":[{"role":"user","content":"hi"}]}`)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("auto chat status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
seen := seenStages(t, g)
|
||||
if len(seen) != 3 {
|
||||
t.Fatalf("AUTO stages = %v, want three", stageNames(seen))
|
||||
}
|
||||
if seen[0]["model"] != "AUTO" {
|
||||
t.Errorf("request_start model = %v, want the requested \"AUTO\"", seen[0]["model"])
|
||||
}
|
||||
if seen[1]["model"] != "spy-model" {
|
||||
t.Errorf("routed model = %v, want the RESOLVED model", seen[1]["model"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestHooksFireOnFailedRequest is the one that matters most for accounting: a
|
||||
// request that 500s upstream must still reach request_end, or a billing plugin
|
||||
// would never see failed traffic.
|
||||
func TestHooksFireOnFailedRequest(t *testing.T) {
|
||||
// Point at an upstream that always fails.
|
||||
bad := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
_, _ = w.Write([]byte(`{"error":"boom"}`))
|
||||
})
|
||||
up := newFailingServer(bad)
|
||||
defer up.Close()
|
||||
|
||||
dir := t.TempDir()
|
||||
cfgPath := filepath.Join(dir, "config.yaml")
|
||||
cfgBody := "listen: :0\n" +
|
||||
"adapter_dir: " + filepath.Join(dir, "adapters") + "\n" +
|
||||
"plugin_dir: " + filepath.Join(dir, "plugins") + "\n" +
|
||||
"runtime_file: " + filepath.Join(dir, "runtime.json") + "\n" +
|
||||
"gateway_keys:\n - sk-test\n"
|
||||
os.WriteFile(cfgPath, []byte(cfgBody), 0600)
|
||||
cfg, err := config.Load(cfgPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg.Sources = []config.Source{{
|
||||
Name: "badup", BaseURL: up.URL, Adapter: "openai", APIKey: "sk-x",
|
||||
Models: []config.Model{{ID: "bad-model", Kind: "chat"}},
|
||||
}}
|
||||
if err := cfg.ApplyDefaults(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := core.NewFromConfig(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("core: %v", err)
|
||||
}
|
||||
defer c.Close()
|
||||
if err := c.Plugins().LoadSource("spy", spyPlugin()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
g, err := New(c)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
rr := doReq(t, g, http.MethodPost, "/v1/chat/completions",
|
||||
`{"model":"bad-model","messages":[{"role":"user","content":"hi"}]}`)
|
||||
if rr.Code == http.StatusOK {
|
||||
t.Fatal("expected the upstream failure to surface as a non-200")
|
||||
}
|
||||
seen := seenStages(t, g)
|
||||
// request_start must fire; routed must NOT (no source was ever selected);
|
||||
// request_end MUST (otherwise billing misses failed traffic).
|
||||
got := stageNames(seen)
|
||||
if len(got) == 0 || got[0] != "request_start" {
|
||||
t.Fatalf("stages = %v, want request_start first", got)
|
||||
}
|
||||
hasEnd := false
|
||||
for _, s := range seen {
|
||||
if s["stage"] == "request_end" {
|
||||
hasEnd = true
|
||||
if s["ok"] != false {
|
||||
t.Error("request_end reported ok=true for a failed request")
|
||||
}
|
||||
}
|
||||
}
|
||||
if !hasEnd {
|
||||
t.Error("a FAILED request never reached request_end; a billing plugin " +
|
||||
"would not see it at all")
|
||||
}
|
||||
if contains(got, "routed") {
|
||||
t.Error("routed fired for a request that never selected a source")
|
||||
}
|
||||
}
|
||||
|
||||
// TestBrokenPluginDoesNotBreakForwarding is the safety property, asserted
|
||||
// end-to-end: a plugin that throws in EVERY stage must not stop a chat from
|
||||
// being served.
|
||||
func TestBrokenPluginDoesNotBreakForwarding(t *testing.T) {
|
||||
up := mockUpstream()
|
||||
defer up.Close()
|
||||
dir := t.TempDir()
|
||||
cfgPath := filepath.Join(dir, "config.yaml")
|
||||
os.WriteFile(cfgPath, []byte("listen: :0\n"+
|
||||
"adapter_dir: "+filepath.Join(dir, "adapters")+"\n"+
|
||||
"plugin_dir: "+filepath.Join(dir, "plugins")+"\n"+
|
||||
"runtime_file: "+filepath.Join(dir, "runtime.json")+"\n"+
|
||||
"gateway_keys:\n - sk-test\n"), 0600)
|
||||
cfg, err := config.Load(cfgPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg.Sources = []config.Source{{
|
||||
Name: "u", BaseURL: up.URL, Adapter: "openai", APIKey: "sk-x",
|
||||
Models: []config.Model{{ID: "m", Kind: "chat"}},
|
||||
}}
|
||||
if err := cfg.ApplyDefaults(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := core.NewFromConfig(cfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer c.Close()
|
||||
broken := `
|
||||
local plugin = { name = "broken", version = "1.0.0" }
|
||||
plugin.hooks = { request_start = "f", routed = "f", request_end = "f" }
|
||||
function plugin.f(p) error("plugin is broken") end
|
||||
return plugin
|
||||
`
|
||||
if err := c.Plugins().LoadSource("broken", broken); err != nil {
|
||||
t.Fatalf("load broken: %v", err)
|
||||
}
|
||||
g, err := New(c)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rr := doReq(t, g, http.MethodPost, "/v1/chat/completions",
|
||||
`{"model":"m","messages":[{"role":"user","content":"hi"}]}`)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("a broken plugin broke forwarding: status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if !strings.Contains(rr.Body.String(), "pong") {
|
||||
t.Errorf("the response is not the upstream's: %s", rr.Body.String())
|
||||
}
|
||||
// And the failure must be visible rather than silent.
|
||||
if len(c.Plugins().HookErrors()) == 0 {
|
||||
t.Error("a plugin that threw in every stage left no error record")
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginListIsServedWithStages is the operator-facing check: an admin
|
||||
// debugging a plugin needs to see which stages are live and whether the hook
|
||||
// file is where the kernel thinks it is.
|
||||
func TestPluginListIsServedWithStages(t *testing.T) {
|
||||
g, _ := gatewayWithSpy(t)
|
||||
rr := doReq(t, g, http.MethodGet, "/api/plugins", "")
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("status=%d", rr.Code)
|
||||
}
|
||||
for _, want := range []string{"spy", "request_start", "routed", "request_end", "plugin_dir"} {
|
||||
if !strings.Contains(rr.Body.String(), want) {
|
||||
t.Errorf("/api/plugins response lacks %q", want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func toInt(t *testing.T, v interface{}) int64 {
|
||||
t.Helper()
|
||||
f, ok := v.(float64)
|
||||
if !ok {
|
||||
return 0
|
||||
}
|
||||
return int64(f)
|
||||
}
|
||||
|
||||
func contains(list []string, s string) bool {
|
||||
for _, x := range list {
|
||||
if x == s {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// imageUpstream serves an OpenAI-shaped image response.
|
||||
func imageUpstream() *httptest.Server {
|
||||
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_, _ = w.Write([]byte(`{"created":1,"data":[{"b64_json":"aGk="}]}`))
|
||||
}))
|
||||
}
|
||||
|
||||
// TestHooksFireOnRealImageRequest: the image path is a THIRD entry point. It
|
||||
// has its own handler and its own scheduler call, so "chat works" says nothing
|
||||
// about it — and image traffic is billable, so a cost plugin that never sees
|
||||
// it under-reports.
|
||||
func TestHooksFireOnRealImageRequest(t *testing.T) {
|
||||
up := imageUpstream()
|
||||
defer up.Close()
|
||||
|
||||
dir := t.TempDir()
|
||||
cfgPath := filepath.Join(dir, "config.yaml")
|
||||
os.WriteFile(cfgPath, []byte("listen: :0\n"+
|
||||
"adapter_dir: "+filepath.Join(dir, "adapters")+"\n"+
|
||||
"plugin_dir: "+filepath.Join(dir, "plugins")+"\n"+
|
||||
"runtime_file: "+filepath.Join(dir, "runtime.json")+"\n"+
|
||||
"gateway_keys:\n - sk-test\n"), 0600)
|
||||
cfg, err := config.Load(cfgPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg.Sources = []config.Source{{
|
||||
Name: "imgup", BaseURL: up.URL, Adapter: "openai", APIKey: "sk-x",
|
||||
Models: []config.Model{{ID: "img-model", Kind: "image"}},
|
||||
}}
|
||||
if err := cfg.ApplyDefaults(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := core.NewFromConfig(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("core: %v", err)
|
||||
}
|
||||
defer c.Close()
|
||||
if err := c.Plugins().LoadSource("spy", spyPlugin()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
g, err := New(c)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
rr := doReq(t, g, http.MethodPost, "/v1/images/generations",
|
||||
`{"model":"img-model","prompt":"a cat"}`)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("image status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
seen := seenStages(t, g)
|
||||
got := stageNames(seen)
|
||||
if len(got) != 3 {
|
||||
t.Fatalf("image stages = %v, want all three (image is a separate handler)", got)
|
||||
}
|
||||
if seen[0]["type"] != "image" {
|
||||
t.Errorf("request_start type = %v, want \"image\"", seen[0]["type"])
|
||||
}
|
||||
if seen[1]["source"] != "imgup" {
|
||||
t.Errorf("routed source = %v, want imgup", seen[1]["source"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestRejectedChatStillFiresRequestStart pins the ordering decision: request_start
|
||||
// fires BEFORE the quota/scope gates, so a plugin auditing traffic sees rejected
|
||||
// requests too. A plugin that only ever saw served requests could not compute a
|
||||
// true request rate.
|
||||
func TestRejectedChatStillFiresRequestStart(t *testing.T) {
|
||||
up := mockUpstream()
|
||||
defer up.Close()
|
||||
dir := t.TempDir()
|
||||
cfgPath := filepath.Join(dir, "config.yaml")
|
||||
os.WriteFile(cfgPath, []byte("listen: :0\n"+
|
||||
"adapter_dir: "+filepath.Join(dir, "adapters")+"\n"+
|
||||
"plugin_dir: "+filepath.Join(dir, "plugins")+"\n"+
|
||||
"runtime_file: "+filepath.Join(dir, "runtime.json")+"\n"+
|
||||
"gateway_keys:\n - sk-test\n"), 0600)
|
||||
cfg, err := config.Load(cfgPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg.Sources = []config.Source{{
|
||||
Name: "u", BaseURL: up.URL, Adapter: "openai", APIKey: "sk-x",
|
||||
Models: []config.Model{{ID: "m", Kind: "chat"}},
|
||||
}}
|
||||
if err := cfg.ApplyDefaults(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := core.NewFromConfig(cfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer c.Close()
|
||||
if err := c.Plugins().LoadSource("spy", spyPlugin()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
g, err := New(c)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// A model the gateway does not know => 404, rejected before any routing.
|
||||
rr := doReq(t, g, http.MethodPost, "/v1/chat/completions",
|
||||
`{"model":"no-such-model","messages":[{"role":"user","content":"hi"}]}`)
|
||||
if rr.Code != http.StatusNotFound {
|
||||
t.Fatalf("expected 404 for an unknown model, got %d", rr.Code)
|
||||
}
|
||||
got := stageNames(seenStages(t, g))
|
||||
if !contains(got, "request_start") {
|
||||
t.Errorf("stages = %v; a REJECTED request must still fire request_start so a "+
|
||||
"plugin can count real traffic, not just served traffic", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestChainStepReachesPluginOnDegradation is the end-to-end proof for the
|
||||
// AUTO trace: a request that had to drop from tier 1 to tier 2 must be visible
|
||||
// to a plugin as a tier_skip followed by a selected, and request_end must carry
|
||||
// tier_served=2.
|
||||
//
|
||||
// Before the trace existed the plugin saw only tier=-2 ("resolved by the
|
||||
// chain") and could not tell a degradation from a clean tier-1 hit — which is
|
||||
// the whole question a priority chain exists to answer.
|
||||
func TestChainStepReachesPluginOnDegradation(t *testing.T) {
|
||||
// tier 1's source always fails, so the walk must drop to tier 2.
|
||||
bad := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
_, _ = w.Write([]byte(`{"error":"boom"}`))
|
||||
}))
|
||||
defer bad.Close()
|
||||
good := mockUpstream()
|
||||
defer good.Close()
|
||||
|
||||
dir := t.TempDir()
|
||||
cfgPath := filepath.Join(dir, "config.yaml")
|
||||
os.WriteFile(cfgPath, []byte("listen: :0\n"+
|
||||
"adapter_dir: "+filepath.Join(dir, "adapters")+"\n"+
|
||||
"plugin_dir: "+filepath.Join(dir, "plugins")+"\n"+
|
||||
"runtime_file: "+filepath.Join(dir, "runtime.json")+"\n"+
|
||||
"gateway_keys:\n - sk-test\n"), 0600)
|
||||
cfg, err := config.Load(cfgPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg.Sources = []config.Source{
|
||||
{Name: "t1", BaseURL: bad.URL, Adapter: "openai", APIKey: "sk-x",
|
||||
Models: []config.Model{{ID: "hi-tier", Kind: "chat"}}},
|
||||
{Name: "t2", BaseURL: good.URL, Adapter: "openai", APIKey: "sk-x",
|
||||
Models: []config.Model{{ID: "lo-tier", Kind: "chat"}}},
|
||||
}
|
||||
if err := cfg.ApplyDefaults(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := core.NewFromConfig(cfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer c.Close()
|
||||
|
||||
// A spy that records chain_step events too.
|
||||
sp := `
|
||||
local plugin = { name = "walker", version = "1.0.0" }
|
||||
plugin.state = { steps = {}, ends = {} }
|
||||
plugin.hooks = { chain_step = "step", request_end = "fin" }
|
||||
function plugin.step(p)
|
||||
table.insert(plugin.state.steps, { kind = p.kind, tier = p.tier, source = p.source, model = p.model, reason = p.reason })
|
||||
return nil
|
||||
end
|
||||
function plugin.fin(p)
|
||||
plugin.state.ends[#plugin.state.ends + 1] = {
|
||||
tier_served = p.tier_served, degraded = p.degraded,
|
||||
walk = p.chain_walk, source = p.source, model = p.model,
|
||||
}
|
||||
return nil
|
||||
end
|
||||
return plugin
|
||||
`
|
||||
if err := c.Plugins().LoadSource("walker", sp); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
g, err := New(c)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Two tiers, both in the chain.
|
||||
if rr := doReq(t, g, http.MethodPut, "/api/auto",
|
||||
`{"rules":[{"model":"hi-tier","source":"t1","tier":1},{"model":"lo-tier","source":"t2","tier":2}]}`); rr.Code != 200 {
|
||||
t.Fatalf("save auto: %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
|
||||
rr := doReq(t, g, http.MethodPost, "/v1/chat/completions",
|
||||
`{"model":"AUTO","messages":[{"role":"user","content":"hi"}]}`)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("chat = %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
|
||||
raw := c.Plugins().State("walker")
|
||||
b, _ := json.Marshal(raw)
|
||||
var st struct {
|
||||
Steps []struct {
|
||||
Kind string `json:"kind"`
|
||||
Tier int `json:"tier"`
|
||||
Source string `json:"source"`
|
||||
Model string `json:"model"`
|
||||
} `json:"steps"`
|
||||
Ends []struct {
|
||||
TierServed int `json:"tier_served"`
|
||||
Degraded bool `json:"degraded"`
|
||||
Source string `json:"source"`
|
||||
Model string `json:"model"`
|
||||
} `json:"ends"`
|
||||
}
|
||||
if err := json.Unmarshal(b, &st); err != nil {
|
||||
t.Fatalf("decode: %v (%s)", err, string(b))
|
||||
}
|
||||
if len(st.Steps) < 2 {
|
||||
t.Fatalf("chain_step events = %+v, want at least a slot_fail and a selected", st.Steps)
|
||||
}
|
||||
if st.Steps[0].Kind != "slot_fail" || st.Steps[0].Tier != 1 {
|
||||
t.Errorf("first step = %+v, want slot_fail on tier 1", st.Steps[0])
|
||||
}
|
||||
last := st.Steps[len(st.Steps)-1]
|
||||
if last.Kind != "selected" || last.Tier != 2 {
|
||||
t.Errorf("last step = %+v, want selected on tier 2", last)
|
||||
}
|
||||
if len(st.Ends) != 1 {
|
||||
t.Fatalf("request_end count = %d, want 1", len(st.Ends))
|
||||
}
|
||||
if st.Ends[0].TierServed != 2 {
|
||||
t.Errorf("tier_served = %d, want 2", st.Ends[0].TierServed)
|
||||
}
|
||||
if !st.Ends[0].Degraded {
|
||||
t.Error("degraded = false, but the request dropped a tier")
|
||||
}
|
||||
if st.Ends[0].Model != "lo-tier" {
|
||||
t.Errorf("served model = %q, want lo-tier", st.Ends[0].Model)
|
||||
}
|
||||
}
|
||||
@ -1,284 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"llmsproxy/internal/lua"
|
||||
)
|
||||
|
||||
var (
|
||||
errPluginName = errors.New("plugin name must be non-empty and contain no path separator or dot")
|
||||
errNoPluginDir = errors.New("no plugin_dir configured; set plugin_dir in config.yaml to enable plugins")
|
||||
)
|
||||
|
||||
// Plugin management API.
|
||||
//
|
||||
// GET /api/plugins list loaded plugins + their hooks/UI/errors
|
||||
// POST /api/plugins upload/replace one plugin (.lua), hot-applied
|
||||
// DELETE /api/plugins/{name} remove a plugin
|
||||
// GET /api/plugins/{name}/state the plugin's own published state
|
||||
// PUT /api/plugins/{name}/state replace that state (admin only)
|
||||
//
|
||||
// Why state is a first-class endpoint: request_end is the LAST stage, so a
|
||||
// hook's return value has no downstream consumer inside the gateway. A plugin
|
||||
// that accumulates numbers (the billing plugin does exactly this) therefore
|
||||
// keeps them in its own Lua state and serves them here, which is what its UI
|
||||
// component fetches. This keeps plugin data clearly separated from the
|
||||
// gateway's own stats — see the accounting note in docs/plugins.md: the
|
||||
// gateway's quota accounting stays authoritative, a plugin only reports.
|
||||
|
||||
// handlePluginUI serves the merged UI extensions the WebUI injects at boot.
|
||||
//
|
||||
// It is a single GET (not per-plugin) because the browser needs ALL extensions
|
||||
// before it can build the sidebar: a page contributed by one plugin and an
|
||||
// element contributed by another land in the same payload, and fetching them
|
||||
// separately would mean the sidebar has to be rebuilt as each arrives.
|
||||
//
|
||||
// Served to any authenticated role: the WebUI is authenticated before it asks,
|
||||
// and a plugin's own widgets need to render for the user whose key they show.
|
||||
func (g *Gateway) handlePluginUI(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet {
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "use GET")
|
||||
return
|
||||
}
|
||||
ps := g.core.Plugins()
|
||||
if ps == nil {
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{"ui": lua.UIExtension{}})
|
||||
return
|
||||
}
|
||||
// AllStages, not a hand-written list: an earlier version enumerated the
|
||||
// three stages literally here, so when chain_step was added it was silently
|
||||
// missing from this response — a plugin author reading the discovery
|
||||
// document would have believed the stage did not exist. Deriving it from
|
||||
// the one place that defines the order is the whole point of having it.
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{
|
||||
"ui": ps.UI(),
|
||||
"stages": pipelineStageNames(),
|
||||
})
|
||||
}
|
||||
|
||||
// stageNames returns the pipeline stage names in firing order, for the
|
||||
// discovery payload and for tests.
|
||||
func pipelineStageNames() []string {
|
||||
out := make([]string, 0, len(lua.AllStages))
|
||||
for _, s := range lua.AllStages {
|
||||
out = append(out, string(s))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (g *Gateway) handlePluginsAPI(w http.ResponseWriter, r *http.Request) {
|
||||
ps := g.core.Plugins()
|
||||
if ps == nil {
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{"plugins": []interface{}{}})
|
||||
return
|
||||
}
|
||||
path := strings.TrimPrefix(r.URL.Path, "/api/plugins")
|
||||
path = strings.Trim(path, "/")
|
||||
|
||||
// /api/plugins/{name}/state
|
||||
if strings.HasSuffix(path, "/state") {
|
||||
name := strings.TrimSuffix(path, "/state")
|
||||
if name == "" {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "plugin name required")
|
||||
return
|
||||
}
|
||||
g.handlePluginState(w, r, name)
|
||||
return
|
||||
}
|
||||
|
||||
if reqRole(r.Context()) != "admin" {
|
||||
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
|
||||
return
|
||||
}
|
||||
|
||||
if path == "" {
|
||||
switch r.Method {
|
||||
case http.MethodGet:
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{
|
||||
"plugins": ps.List(),
|
||||
"on_disk": ps.OnDisk(),
|
||||
"hook_errors": ps.HookErrors(),
|
||||
"plugin_dir": g.pluginDir(),
|
||||
})
|
||||
case http.MethodPost:
|
||||
var body struct {
|
||||
Name string `json:"name"`
|
||||
Code string `json:"code"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
|
||||
return
|
||||
}
|
||||
if err := g.installPlugin(body.Name, body.Code); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "plugin_error", err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{"ok": true, "name": body.Name})
|
||||
default:
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "")
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// /api/plugins/{name} — source read (GET), enable/disable (PUT), delete.
|
||||
// Multi-segment paths (…/state) are handled earlier and never reach here.
|
||||
if !strings.Contains(path, "/") {
|
||||
switch r.Method {
|
||||
case http.MethodGet:
|
||||
// Reading the SOURCE (not the runtime state) is what an editor
|
||||
// needs; the state endpoint is /state and returns accumulated data
|
||||
// instead. Mixing the two would make a "save what I read"
|
||||
// round-trip impossible.
|
||||
code, err := g.readPluginSource(path)
|
||||
if err != nil {
|
||||
writeError(w, http.StatusNotFound, "not_found", err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{
|
||||
"name": path, "code": code,
|
||||
"enabled": g.core.Plugins().Enabled(path),
|
||||
})
|
||||
case http.MethodPut:
|
||||
// Enable / disable. The intent rides in the body rather than the
|
||||
// verb, because "enable" and "disable" are the same resource and
|
||||
// PUT /{name} is the one the docs advertise.
|
||||
var body struct {
|
||||
Enabled *bool `json:"enabled"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
|
||||
return
|
||||
}
|
||||
if body.Enabled == nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", `body must be {"enabled":true|false}`)
|
||||
return
|
||||
}
|
||||
if err := g.core.Plugins().SetEnabled(path, *body.Enabled); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "plugin_error", err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{
|
||||
"ok": true, "name": path, "enabled": *body.Enabled,
|
||||
})
|
||||
case http.MethodDelete:
|
||||
if err := g.removePlugin(path); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "plugin_error", err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{"ok": true})
|
||||
default:
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "")
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
writeError(w, http.StatusNotFound, "not_found", "unknown plugin sub-resource: "+path)
|
||||
}
|
||||
|
||||
// readPluginSource returns a plugin's file contents for the editor.
|
||||
func (g *Gateway) readPluginSource(name string) (string, error) {
|
||||
if err := validPluginName(name); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if g.pluginDir() == "" {
|
||||
return "", errNoPluginDir
|
||||
}
|
||||
b, err := os.ReadFile(filepath.Join(g.pluginDir(), name+".lua"))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("no plugin source named %q", name)
|
||||
}
|
||||
return string(b), nil
|
||||
}
|
||||
|
||||
// validPluginName rejects anything that could escape the plugin directory.
|
||||
// Shared by install / remove / read so the check cannot drift between them.
|
||||
func validPluginName(name string) error {
|
||||
if name == "" || strings.ContainsAny(name, `/\.`) {
|
||||
return errPluginName
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// handlePluginState serves GET (read state) and PUT (replace state).
|
||||
func (g *Gateway) handlePluginState(w http.ResponseWriter, r *http.Request, name string) {
|
||||
ps := g.core.Plugins()
|
||||
switch r.Method {
|
||||
case http.MethodGet:
|
||||
// Any role may read: plugin state is reporting data (cost, counts),
|
||||
// and the caller has already been authenticated. Admin-only would stop
|
||||
// a user key's own billing widget from rendering.
|
||||
// prices rides along: the billing rules editor must show the table
|
||||
// that is actually in effect, not a reconstruction from config. It is
|
||||
// absent for plugins with no prices (every plugin but billing).
|
||||
payload := map[string]interface{}{
|
||||
"plugin": name,
|
||||
"state": ps.State(name),
|
||||
}
|
||||
if pr := ps.Prices(name); pr != nil {
|
||||
payload["prices"] = pr
|
||||
}
|
||||
writeJSON(w, http.StatusOK, payload)
|
||||
case http.MethodPut:
|
||||
if reqRole(r.Context()) != "admin" {
|
||||
writeError(w, http.StatusForbidden, "forbidden", "admin role required")
|
||||
return
|
||||
}
|
||||
var state interface{}
|
||||
if err := json.NewDecoder(r.Body).Decode(&state); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid_request", "invalid json: "+err.Error())
|
||||
return
|
||||
}
|
||||
if err := ps.SetState(name, state); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "plugin_error", err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]interface{}{"ok": true})
|
||||
default:
|
||||
writeError(w, http.StatusMethodNotAllowed, "method_not_allowed", "")
|
||||
}
|
||||
}
|
||||
|
||||
func (g *Gateway) pluginDir() string {
|
||||
if c := g.core.Config(); c != nil {
|
||||
return c.PluginDir
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// installPlugin writes a plugin to disk and hot-loads it. A syntax error is
|
||||
// returned to the caller AND the file is left on disk so the operator can fix
|
||||
// it, matching how adapters behave (the file is authoritative once present).
|
||||
func (g *Gateway) installPlugin(name, code string) error {
|
||||
if err := validPluginName(name); err != nil {
|
||||
return err
|
||||
}
|
||||
dir := g.pluginDir()
|
||||
if dir == "" {
|
||||
return errNoPluginDir
|
||||
}
|
||||
if err := os.MkdirAll(dir, 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, name+".lua"), []byte(code), 0644); err != nil {
|
||||
return err
|
||||
}
|
||||
return g.core.Plugins().LoadSource(name, code)
|
||||
}
|
||||
|
||||
func (g *Gateway) removePlugin(name string) error {
|
||||
if err := validPluginName(name); err != nil {
|
||||
return err
|
||||
}
|
||||
if g.pluginDir() == "" {
|
||||
return errNoPluginDir
|
||||
}
|
||||
_ = os.Remove(filepath.Join(g.pluginDir(), name+".lua"))
|
||||
return g.core.Plugins().Unload(name)
|
||||
}
|
||||
@ -1,212 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/core"
|
||||
"llmsproxy/internal/lua"
|
||||
)
|
||||
|
||||
// gatewayWithBilling boots a gateway with the bundled billing plugin loaded, so
|
||||
// the UI-injection endpoint is exercised against a real plugin rather than a
|
||||
// hand-written stub. The other plugin tests in this package assert on the
|
||||
// WebUI source; this one asserts on the HTTP contract the browser consumes.
|
||||
func gatewayWithBilling(t *testing.T) *Gateway {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
cfgPath := filepath.Join(dir, "config.yaml")
|
||||
body := "listen: :0\n" +
|
||||
"adapter_dir: " + filepath.Join(dir, "adapters") + "\n" +
|
||||
"plugin_dir: " + filepath.Join(dir, "plugins") + "\n" +
|
||||
"runtime_file: " + filepath.Join(dir, "runtime.json") + "\n" +
|
||||
"gateway_keys:\n - sk-test\n"
|
||||
if err := os.WriteFile(cfgPath, []byte(body), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, err := config.Load(cfgPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := core.NewFromConfig(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("core: %v", err)
|
||||
}
|
||||
t.Cleanup(c.Close)
|
||||
// Load the shipped plugin explicitly: seeding only runs for a directory that
|
||||
// does not exist yet, and this test wants a known plugin regardless.
|
||||
src, err := lua.ReadBundledPlugin("billing")
|
||||
if err != nil {
|
||||
t.Fatalf("read bundled billing: %v", err)
|
||||
}
|
||||
if err := c.Plugins().LoadSource("billing", src); err != nil {
|
||||
t.Fatalf("load billing: %v", err)
|
||||
}
|
||||
g, err := New(c)
|
||||
if err != nil {
|
||||
t.Fatalf("gateway: %v", err)
|
||||
}
|
||||
return g
|
||||
}
|
||||
|
||||
// TestUIInjectServesPluginUI: GET /api/ui-inject is the single call the WebUI
|
||||
// makes at boot, and it must carry BOTH a contributed page and contributed
|
||||
// elements — the browser builds the sidebar from the page and mounts the
|
||||
// elements into existing panes from the same payload, so a partial response
|
||||
// would produce a page with no body or a missing tile.
|
||||
func TestUIInjectServesPluginUI(t *testing.T) {
|
||||
g := gatewayWithBilling(t)
|
||||
rr := doReq(t, g, http.MethodGet, "/api/ui-inject", "")
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var view struct {
|
||||
// Decoded into the real types so the test cannot drift from the wire
|
||||
// contract. An inline copy missed the `pages` field when the payload
|
||||
// shape changed and failed to compile, which is at least loud — but the
|
||||
// same copy also went on asserting the OLD single-page shape for a
|
||||
// release, silently.
|
||||
UI lua.UIExtension `json:"ui"`
|
||||
Stages []string `json:"stages"`
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &view); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
// The payload carries every page in one list; ui.page is no longer a
|
||||
// separate slot (it was, and a second plugin contributing a page overwrote
|
||||
// whatever was there).
|
||||
var billing *lua.UIPage
|
||||
for i, pg := range view.UI.Pages {
|
||||
if pg != nil && pg.PageID == "billing" {
|
||||
billing = view.UI.Pages[i]
|
||||
}
|
||||
}
|
||||
if billing == nil {
|
||||
t.Fatalf("no billing page in the inject payload; pages=%d", len(view.UI.Pages))
|
||||
}
|
||||
if !strings.Contains(billing.Mount, "billing-root") {
|
||||
t.Error("the page mount came back empty")
|
||||
}
|
||||
if len(view.UI.Elements) == 0 {
|
||||
t.Error("billing contributes an element to the status page but it is missing")
|
||||
}
|
||||
for _, e := range view.UI.Elements {
|
||||
if e.Target != "status" {
|
||||
t.Errorf("element target = %q, want \"status\"", e.Target)
|
||||
}
|
||||
}
|
||||
// Derived from AllStages, not hardcoded: the previous assertion of "3"
|
||||
// is exactly what let chain_step go missing from this payload unnoticed.
|
||||
if len(view.Stages) != len(lua.AllStages) {
|
||||
t.Errorf("stages = %v, want %d (one per AllStages entry)", view.Stages, len(lua.AllStages))
|
||||
}
|
||||
for i, st := range lua.AllStages {
|
||||
if i >= len(view.Stages) || view.Stages[i] != string(st) {
|
||||
t.Errorf("stages[%d] = %v, want %q", i, view.Stages, string(st))
|
||||
}
|
||||
}
|
||||
if !containsStr(view.Stages, string(lua.StageChainStep)) {
|
||||
t.Error("the discovery payload does not advertise chain_step; a plugin " +
|
||||
"author would conclude the stage does not exist")
|
||||
}
|
||||
}
|
||||
|
||||
// TestUIInjectIsEmptyWithoutPlugins: a gateway with no plugins must still answer
|
||||
// 200 with an empty (not missing, not null) payload. The WebUI calls this
|
||||
// unconditionally at boot, so a 404 or a null `ui` would break every dashboard.
|
||||
func TestUIInjectIsEmptyWithoutPlugins(t *testing.T) {
|
||||
g := newTestGateway(t)
|
||||
rr := doReq(t, g, http.MethodGet, "/api/ui-inject", "")
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if !strings.Contains(rr.Body.String(), `"ui"`) {
|
||||
t.Error("no ui key in the response; the WebUI would have nothing to read")
|
||||
}
|
||||
}
|
||||
|
||||
// containsStr reports whether list has s.
|
||||
func containsStr(list []string, s string) bool {
|
||||
for _, x := range list {
|
||||
if x == s {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// newRecorderFor pushes a request through the full handler chain.
|
||||
func newRecorderFor(t *testing.T, g *Gateway, req *http.Request) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
rr := httptest.NewRecorder()
|
||||
g.Handler().ServeHTTP(rr, req)
|
||||
return rr
|
||||
}
|
||||
|
||||
// TestPluginsListAndStateAPI covers the management surface the plugin docs
|
||||
// promise: listing, and reading a plugin's own published state.
|
||||
func TestPluginsListAndStateAPI(t *testing.T) {
|
||||
g := gatewayWithBilling(t)
|
||||
rr := doReq(t, g, http.MethodGet, "/api/plugins", "")
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("GET /api/plugins = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
for _, want := range []string{"billing", "hook_errors", "plugin_dir", "request_end"} {
|
||||
if !strings.Contains(rr.Body.String(), want) {
|
||||
t.Errorf("/api/plugins response lacks %q", want)
|
||||
}
|
||||
}
|
||||
// state read: the plugin published its (empty) state, so the key exists.
|
||||
rr = doReq(t, g, http.MethodGet, "/api/plugins/billing/state", "")
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("GET state = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if !strings.Contains(rr.Body.String(), `"total"`) {
|
||||
t.Errorf("billing state lacks the total bucket: %s", rr.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginStatePUTIsAdminOnly: only the WRITE side is gated. A user key must
|
||||
// be able to READ its own billing widget's data, but must not be able to
|
||||
// rewrite the price table.
|
||||
func TestPluginStatePUTIsAdminOnly(t *testing.T) {
|
||||
g := gatewayWithBilling(t)
|
||||
|
||||
// Build a user-role key and remember its secret.
|
||||
rec, err := g.core.CreateKey("viewer", "user", nil, "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
userKey := rec.Key
|
||||
|
||||
// A user key may read the state.
|
||||
req, _ := http.NewRequest(http.MethodGet, "/api/plugins/billing/state", nil)
|
||||
req.Header.Set("Authorization", "Bearer "+userKey)
|
||||
rr := newRecorderFor(t, g, req)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Errorf("user GET state = %d, want 200 (the billing widget must render for users)", rr.Code)
|
||||
}
|
||||
|
||||
// A user key may NOT write it.
|
||||
put, _ := http.NewRequest(http.MethodPut, "/api/plugins/billing/state",
|
||||
strings.NewReader(`{"prices":{"default":{"prompt":0}}}`))
|
||||
put.Header.Set("Authorization", "Bearer "+userKey)
|
||||
put.Header.Set("Content-Type", "application/json")
|
||||
prr := newRecorderFor(t, g, put)
|
||||
if prr.Code != http.StatusForbidden {
|
||||
t.Errorf("user PUT state = %d, want 403 (a user must not rewrite the price table)", prr.Code)
|
||||
}
|
||||
|
||||
// An admin key may.
|
||||
adm := doReq(t, g, http.MethodPut, "/api/plugins/billing/state",
|
||||
`{"prices":{"default":{"prompt":1e-6}}}`)
|
||||
if adm.Code != http.StatusOK {
|
||||
t.Errorf("admin PUT state = %d: %s", adm.Code, adm.Body.String())
|
||||
}
|
||||
}
|
||||
@ -115,7 +115,7 @@ func (g *Gateway) loginRecord(ip string, ok bool, now int64) {
|
||||
}
|
||||
}
|
||||
|
||||
func New(c *core.Core) (*Gateway, error) {
|
||||
func New(c *core.Core, gatewayKeys []string) (*Gateway, error) {
|
||||
sub, err := fs.Sub(uiFS, "ui")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@ -222,21 +222,10 @@ func (g *Gateway) routes(w http.ResponseWriter, r *http.Request) {
|
||||
g.handleModels(w, r)
|
||||
case r.URL.Path == "/api/adapters" || strings.HasPrefix(r.URL.Path, "/api/adapters/"):
|
||||
g.handleAdaptersAPI(w, r)
|
||||
case r.URL.Path == "/api/v1" || strings.HasPrefix(r.URL.Path, "/api/v1/"):
|
||||
g.apiV1Routes(w, r)
|
||||
case r.URL.Path == "/api/sources" || strings.HasPrefix(r.URL.Path, "/api/sources/"):
|
||||
g.handleSourcesAPI(w, r)
|
||||
case r.URL.Path == "/api/source_templates" || strings.HasPrefix(r.URL.Path, "/api/source_templates/"):
|
||||
g.handleSourceTemplatesAPI(w, r)
|
||||
case r.URL.Path == "/api/ui-inject" || strings.HasPrefix(r.URL.Path, "/api/ui-inject/"):
|
||||
g.handlePluginUI(w, r)
|
||||
case r.URL.Path == "/api/plugins/billing/rules":
|
||||
// Checked BEFORE the generic /api/plugins/ prefix: the plugin router
|
||||
// treats anything after the name as a plugin sub-resource and would
|
||||
// 404 on "rules" instead of reaching the billing editor.
|
||||
g.handleBillingRules(w, r)
|
||||
case r.URL.Path == "/api/plugins" || strings.HasPrefix(r.URL.Path, "/api/plugins/"):
|
||||
g.handlePluginsAPI(w, r)
|
||||
case r.URL.Path == "/api/chat":
|
||||
g.handleChat(w, r)
|
||||
case r.URL.Path == "/api/status":
|
||||
|
||||
@ -1,495 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/core"
|
||||
)
|
||||
|
||||
// This file covers the second round of the same class of defect bf0657b fixed.
|
||||
//
|
||||
// bf0657b stopped a partial source edit from clobbering api_key. The upsert
|
||||
// still rewrites the WHOLE source, so every other field the request could not
|
||||
// express was reset to its zero value on any edit: proxy_url, api_key_env,
|
||||
// timeout and queue_timeout.
|
||||
//
|
||||
// The four are worse than cosmetic because they change how the source dials out.
|
||||
// proxy_url silently turns a proxied upstream into a direct one (or vice versa)
|
||||
// and api_key_env turns a source whose credential lives in an environment
|
||||
// variable into a credential-less source — which keeps answering 200 on write
|
||||
// and only fails on the NEXT call upstream, long after the editor left.
|
||||
//
|
||||
// The trigger is not exotic scripting: the WebUI's own edit form posts exactly
|
||||
// such a payload. See TestWebUIEditPayloadPreservesRoutingFields.
|
||||
|
||||
// richGateway writes a config.yaml whose source carries all four fragile
|
||||
// fields, then boots a gateway over it.
|
||||
func richGateway(t *testing.T) *Gateway {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
cfgPath := filepath.Join(dir, "config.yaml")
|
||||
cfg := `listen: 127.0.0.1:8080
|
||||
gateway_keys:
|
||||
- sk-test
|
||||
default_model: AUTO
|
||||
adapter_dir: ` + filepath.Join(dir, "adapters") + `
|
||||
runtime_file: ` + filepath.Join(dir, "runtime.json") + `
|
||||
sources:
|
||||
- name: rich
|
||||
base_url: https://api.deepseek.com
|
||||
api_key: sk-live
|
||||
adapter: deepseek
|
||||
proxy_url: http://127.0.0.1:7890
|
||||
timeout: 300s
|
||||
queue_timeout: 90s
|
||||
max_concurrent: 8
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
priority: 100
|
||||
kind: chat
|
||||
`
|
||||
if err := os.WriteFile(cfgPath, []byte(cfg), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := config.Load(cfgPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
core, err := core.NewFromConfig(c)
|
||||
if err != nil {
|
||||
t.Fatalf("core: %v", err)
|
||||
}
|
||||
t.Cleanup(core.Close)
|
||||
g, err := New(core)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return g
|
||||
}
|
||||
|
||||
// sourceField reads one field off the live source, i.e. what the routes
|
||||
// actually use rather than what the file happens to contain.
|
||||
func sourceField(t *testing.T, g *Gateway, name string, get func(config.Source) interface{}) interface{} {
|
||||
t.Helper()
|
||||
for _, s := range g.core.Sources() {
|
||||
if s.Name == name {
|
||||
return get(s)
|
||||
}
|
||||
}
|
||||
t.Fatalf("source %q not found", name)
|
||||
return nil
|
||||
}
|
||||
|
||||
// webUIPayload is byte-for-byte the JSON that index.html's saveSource() builds
|
||||
// from the edit dialog. It deliberately carries none of the four fragile
|
||||
// fields, because the dialog has no inputs for them.
|
||||
const webUIPayload = `{
|
||||
"name":"rich","base_url":"https://api.deepseek.com","api_key":"sk-live",
|
||||
"adapter":"deepseek","endpoint":"","image_endpoint":"",
|
||||
"models":[{"id":"deepseek-v4-flash","priority":100,"kind":"chat"}],
|
||||
"headers":{},"meta":{},"temperature":0,"max_tokens":0,"max_concurrent":16,"rpm":0
|
||||
}`
|
||||
|
||||
// TestWebUIEditPayloadPreservesRoutingFields is the regression test for the
|
||||
// reported defect: changing max_concurrent from 8 to 16 in the UI must not
|
||||
// silently drop the proxy, the env-var credential, or the timeouts.
|
||||
func TestWebUIEditPayloadPreservesRoutingFields(t *testing.T) {
|
||||
g := richGateway(t)
|
||||
|
||||
// The one field the operator actually changed.
|
||||
rr := doReq(t, g, "POST", "/api/sources", webUIPayload)
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("edit status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
|
||||
checks := []struct {
|
||||
field string
|
||||
got interface{}
|
||||
want interface{}
|
||||
}{
|
||||
{"proxy_url", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.ProxyURL }),
|
||||
"http://127.0.0.1:7890"},
|
||||
{"timeout", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.Timeout }),
|
||||
300 * time.Second},
|
||||
{"queue_timeout", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.QueueTimeout }),
|
||||
90 * time.Second},
|
||||
// The field the edit DID change, so a fix that simply refuses to write
|
||||
// anything would also pass the three assertions above.
|
||||
{"max_concurrent", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.MaxConcurrent }),
|
||||
16},
|
||||
}
|
||||
for _, c := range checks {
|
||||
if c.got != c.want {
|
||||
t.Errorf("%s = %v, want %v", c.field, c.got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestSourceEditPreservesAPIKeyEnv is called out separately because it is the
|
||||
// only one of the four whose loss is a credential failure rather than a routing
|
||||
// difference, and it is invisible until the next upstream call.
|
||||
func TestSourceEditPreservesAPIKeyEnv(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
cfgPath := filepath.Join(dir, "config.yaml")
|
||||
cfg := `listen: 127.0.0.1:8080
|
||||
gateway_keys:
|
||||
- sk-test
|
||||
default_model: AUTO
|
||||
adapter_dir: ` + filepath.Join(dir, "adapters") + `
|
||||
runtime_file: ` + filepath.Join(dir, "runtime.json") + `
|
||||
sources:
|
||||
- name: envonly
|
||||
base_url: https://api.deepseek.com
|
||||
api_key_env: DEEPSEEK_KEY
|
||||
adapter: deepseek
|
||||
max_concurrent: 8
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
priority: 100
|
||||
kind: chat
|
||||
`
|
||||
if err := os.WriteFile(cfgPath, []byte(cfg), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := config.Load(cfgPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
core, err := core.NewFromConfig(c)
|
||||
if err != nil {
|
||||
t.Fatalf("core: %v", err)
|
||||
}
|
||||
t.Cleanup(core.Close)
|
||||
g, err := New(core)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// A credential-less source is legitimate, so the API must not invent one
|
||||
// here; the point is that the ENV reference survives the edit.
|
||||
rr := doReq(t, g, "POST", "/api/sources", `{
|
||||
"name":"envonly","base_url":"https://api.deepseek.com","api_key":"",
|
||||
"adapter":"deepseek","models":[{"id":"deepseek-v4-flash","priority":100,"kind":"chat"}],
|
||||
"headers":{},"meta":{},"max_concurrent":4}`)
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("edit status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
got := sourceField(t, g, "envonly", func(s config.Source) interface{} { return s.APIKeyEnv })
|
||||
if got != "DEEPSEEK_KEY" {
|
||||
t.Fatalf("api_key_env = %v after an edit that did not mention it; "+
|
||||
"the source now has no credential and will only fail on the next upstream call", got)
|
||||
}
|
||||
if n := sourceField(t, g, "envonly", func(s config.Source) interface{} { return s.MaxConcurrent }); n != 4 {
|
||||
t.Fatalf("max_concurrent = %v, want the edited 4", n)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSourceEditPersistsInheritedFieldsToDisk: inheriting in memory is not
|
||||
// enough — the values must reach config.yaml, or a restart silently undoes the
|
||||
// fix and the field is lost anyway.
|
||||
func TestSourceEditPersistsInheritedFieldsToDisk(t *testing.T) {
|
||||
g := richGateway(t)
|
||||
if rr := doReq(t, g, "POST", "/api/sources", webUIPayload); rr.Code != 200 {
|
||||
t.Fatalf("edit status=%d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
raw, err := os.ReadFile(g.core.Config().Path)
|
||||
if err != nil {
|
||||
t.Fatalf("read config: %v", err)
|
||||
}
|
||||
text := string(raw)
|
||||
for _, want := range []string{
|
||||
"proxy_url: http://127.0.0.1:7890",
|
||||
} {
|
||||
if !strings.Contains(text, want) {
|
||||
t.Errorf("config.yaml lost %q on an unrelated edit:\n%s", want, text)
|
||||
}
|
||||
}
|
||||
// The durations are re-serialized in Go's canonical form (300s becomes
|
||||
// 5m0s), so they must be checked SEMANTICALLY. Asserting the original
|
||||
// spelling would be a false red: the value survived, only its text changed.
|
||||
// Re-loading the file is also the only way to prove the persisted value
|
||||
// parses back to the same duration.
|
||||
reloaded, err := config.Load(g.core.Config().Path)
|
||||
if err != nil {
|
||||
t.Fatalf("re-read config: %v", err)
|
||||
}
|
||||
if len(reloaded.Sources) == 0 {
|
||||
t.Fatal("no sources after reload")
|
||||
}
|
||||
s := reloaded.Sources[0]
|
||||
if s.Timeout != 300*time.Second {
|
||||
t.Errorf("persisted timeout = %v, want 300s", s.Timeout)
|
||||
}
|
||||
if s.QueueTimeout != 90*time.Second {
|
||||
t.Errorf("persisted queue_timeout = %v, want 90s", s.QueueTimeout)
|
||||
}
|
||||
if s.ProxyURL != "http://127.0.0.1:7890" {
|
||||
t.Errorf("persisted proxy_url = %v, want it preserved", s.ProxyURL)
|
||||
}
|
||||
}
|
||||
|
||||
// ---- explicit set / clear, i.e. the pointer semantics actually work ----
|
||||
|
||||
func TestSourceOptionalFieldsCanBeSet(t *testing.T) {
|
||||
g := richGateway(t)
|
||||
rr := doReq(t, g, "POST", "/api/sources", `{
|
||||
"name":"rich","base_url":"https://api.deepseek.com","api_key":"sk-live",
|
||||
"adapter":"deepseek","models":[{"id":"deepseek-v4-flash","priority":100,"kind":"chat"}],
|
||||
"headers":{},"meta":{},
|
||||
"proxy_url":"http://127.0.0.1:1080",
|
||||
"api_key_env":"NEW_ENV_KEY",
|
||||
"timeout":"45s",
|
||||
"queue_timeout":"5s"}`)
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
for _, c := range []struct {
|
||||
field string
|
||||
got interface{}
|
||||
want interface{}
|
||||
}{
|
||||
{"proxy_url", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.ProxyURL }), "http://127.0.0.1:1080"},
|
||||
{"api_key_env", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.APIKeyEnv }), "NEW_ENV_KEY"},
|
||||
{"timeout", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.Timeout }), 45 * time.Second},
|
||||
{"queue_timeout", sourceField(t, g, "rich", func(s config.Source) interface{} { return s.QueueTimeout }), 5 * time.Second},
|
||||
} {
|
||||
if c.got != c.want {
|
||||
t.Errorf("%s = %v, want %v", c.field, c.got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestSourceOptionalFieldsCanBeCleared is the half that an "empty means
|
||||
// inherit" implementation gets wrong: it can never clear a field, so emptying
|
||||
// the proxy box in the UI would keep using the old proxy forever.
|
||||
func TestSourceOptionalFieldsCanBeCleared(t *testing.T) {
|
||||
g := richGateway(t)
|
||||
rr := doReq(t, g, "POST", "/api/sources", `{
|
||||
"name":"rich","base_url":"https://api.deepseek.com","api_key":"sk-live",
|
||||
"adapter":"deepseek","models":[{"id":"deepseek-v4-flash","priority":100,"kind":"chat"}],
|
||||
"headers":{},"meta":{},
|
||||
"proxy_url":"","timeout":"","queue_timeout":""}`)
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if got := sourceField(t, g, "rich", func(s config.Source) interface{} { return s.ProxyURL }); got != "" {
|
||||
t.Errorf("proxy_url = %v, want it cleared", got)
|
||||
}
|
||||
// A cleared timeout falls back to the config default (120s), which is what
|
||||
// mergedSources applies for a zero value — assert the effective behaviour
|
||||
// rather than the raw zero, since that is what a caller actually gets.
|
||||
if got := sourceField(t, g, "rich", func(s config.Source) interface{} { return s.Timeout }); got != config.DefaultSourceTimeout {
|
||||
t.Errorf("timeout = %v, want the default %v after clearing", got, config.DefaultSourceTimeout)
|
||||
}
|
||||
// api_key_env was NOT in the request, so it must be untouched even though
|
||||
// its sibling fields were cleared.
|
||||
_ = rr
|
||||
}
|
||||
|
||||
// TestSourceOptionalFieldsClearIsScoped: clearing the proxy must not clear the
|
||||
// env credential that the same request did not mention.
|
||||
func TestSourceOptionalFieldsClearIsScoped(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
cfgPath := filepath.Join(dir, "config.yaml")
|
||||
cfg := `listen: 127.0.0.1:8080
|
||||
gateway_keys:
|
||||
- sk-test
|
||||
default_model: AUTO
|
||||
adapter_dir: ` + filepath.Join(dir, "adapters") + `
|
||||
runtime_file: ` + filepath.Join(dir, "runtime.json") + `
|
||||
sources:
|
||||
- name: both
|
||||
base_url: https://api.deepseek.com
|
||||
api_key_env: DEEPSEEK_KEY
|
||||
adapter: deepseek
|
||||
proxy_url: http://127.0.0.1:7890
|
||||
max_concurrent: 8
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
priority: 100
|
||||
kind: chat
|
||||
`
|
||||
os.WriteFile(cfgPath, []byte(cfg), 0600)
|
||||
c, err := config.Load(cfgPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
core, err := core.NewFromConfig(c)
|
||||
if err != nil {
|
||||
t.Fatalf("core: %v", err)
|
||||
}
|
||||
t.Cleanup(core.Close)
|
||||
g, err := New(core)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rr := doReq(t, g, "POST", "/api/sources", `{
|
||||
"name":"both","base_url":"https://api.deepseek.com","api_key":"",
|
||||
"adapter":"deepseek","models":[{"id":"deepseek-v4-flash","priority":100,"kind":"chat"}],
|
||||
"headers":{},"meta":{},"proxy_url":""}`)
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if got := sourceField(t, g, "both", func(s config.Source) interface{} { return s.APIKeyEnv }); got != "DEEPSEEK_KEY" {
|
||||
t.Errorf("api_key_env = %v; clearing proxy_url must not touch it", got)
|
||||
}
|
||||
if got := sourceField(t, g, "both", func(s config.Source) interface{} { return s.ProxyURL }); got != "" {
|
||||
t.Errorf("proxy_url = %v, want cleared", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceRejectsBadDuration(t *testing.T) {
|
||||
g := richGateway(t)
|
||||
before := sourceField(t, g, "rich", func(s config.Source) interface{} { return s.Timeout })
|
||||
rr := doReq(t, g, "POST", "/api/sources", `{
|
||||
"name":"rich","base_url":"https://api.deepseek.com","api_key":"sk-live",
|
||||
"adapter":"deepseek","models":[{"id":"deepseek-v4-flash","priority":100,"kind":"chat"}],
|
||||
"headers":{},"meta":{},"timeout":"not-a-duration"}`)
|
||||
if rr.Code != 400 {
|
||||
t.Fatalf("status=%d, want 400 for an unparseable duration: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if !strings.Contains(rr.Body.String(), "timeout") {
|
||||
t.Errorf("the error must name the offending field: %s", rr.Body.String())
|
||||
}
|
||||
// A rejected write must leave the source exactly as it was.
|
||||
if after := sourceField(t, g, "rich", func(s config.Source) interface{} { return s.Timeout }); after != before {
|
||||
t.Errorf("a rejected edit still changed timeout: %v -> %v", before, after)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSourceCreateSetsOptionalFields: the create path must accept them too, and
|
||||
// must not require an existing source to inherit from (a nil-pointer crash
|
||||
// here would only show up as a 500 on every new source).
|
||||
func TestSourceCreateSetsOptionalFields(t *testing.T) {
|
||||
up := mockUpstream()
|
||||
defer up.Close()
|
||||
g := newTestGateway(t)
|
||||
rr := doReq(t, g, "POST", "/api/sources", `{
|
||||
"name":"fresh","base_url":"`+up.URL+`","api_key":"sk-x","adapter":"openai",
|
||||
"models":[{"id":"m1","kind":"chat"}],"headers":{},"meta":{},
|
||||
"proxy_url":"http://127.0.0.1:7890","timeout":"10s","queue_timeout":"3s"}`)
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("create status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if got := sourceField(t, g, "fresh", func(s config.Source) interface{} { return s.ProxyURL }); got != "http://127.0.0.1:7890" {
|
||||
t.Errorf("proxy_url = %v, want it set on create", got)
|
||||
}
|
||||
if got := sourceField(t, g, "fresh", func(s config.Source) interface{} { return s.Timeout }); got != 10*time.Second {
|
||||
t.Errorf("timeout = %v, want 10s on create", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSourceRevealExposesDurations guards the round-trip the edit form depends
|
||||
// on. config.Source tags Timeout/QueueTimeout `json:"-"`, so marshaling the
|
||||
// struct omits them; the form then reads an empty timeout box and — because it
|
||||
// always sends the box back — CLEARS the stored timeout on any unrelated save.
|
||||
// A test that only checks "the edit preserves timeout" would not have caught
|
||||
// that, because the loss happens between the read and the write.
|
||||
//
|
||||
// The credentials are asserted in the same breath: they came from the same
|
||||
// branch, and an earlier iteration of this fix masked the key and broke the
|
||||
// form's other half.
|
||||
func TestSourceRevealExposesDurations(t *testing.T) {
|
||||
up := mockUpstream()
|
||||
defer up.Close()
|
||||
g := newTestGateway(t, config.Source{
|
||||
Name: "revealed",
|
||||
BaseURL: up.URL,
|
||||
Adapter: "openai",
|
||||
APIKey: "sk-reveal-me",
|
||||
Models: []config.Model{{ID: "m1", Kind: "chat"}},
|
||||
Timeout: 300 * time.Second,
|
||||
})
|
||||
|
||||
rr := doReq(t, g, "GET", "/api/v1/sources/revealed?reveal=credentials", "")
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("reveal status=%d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var view struct {
|
||||
Source map[string]interface{} `json:"source"`
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &view); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
// Duration strings, matching what the write path parses.
|
||||
if got := view.Source["timeout"]; got != "5m0s" {
|
||||
t.Errorf("reveal timeout = %v, want the string \"5m0s\"; the edit form "+
|
||||
"would show an empty box and clear the stored value on save", got)
|
||||
}
|
||||
// The unset one shows the DEFAULT, not "": ApplyDefaults filled
|
||||
// QueueTimeout before the form ever sees it. Both are acceptable as long as
|
||||
// the value round-trips, so assert it is a parseable duration string.
|
||||
qs, _ := view.Source["queue_timeout"].(string)
|
||||
if _, err := time.ParseDuration(qs); err != nil {
|
||||
t.Errorf("reveal queue_timeout = %q is not a duration string: %v", qs, err)
|
||||
}
|
||||
if got := view.Source["api_key"]; got != "sk-reveal-me" {
|
||||
t.Errorf("reveal api_key = %v, want it in the clear for the form", got)
|
||||
}
|
||||
// The masked view must NOT gain the durations' secrets — it is fine for it
|
||||
// to omit them, but it must never carry the key.
|
||||
masked := doReq(t, g, "GET", "/api/v1/sources/revealed", "")
|
||||
if strings.Contains(masked.Body.String(), "sk-reveal-me") {
|
||||
t.Error("the masked view leaked the key")
|
||||
}
|
||||
}
|
||||
|
||||
// TestSourcePayloadCoversEveryEditableField is the guard against the class
|
||||
// itself. Any config.Source field a source edit can legitimately set must
|
||||
// exist on sourcePayload, otherwise the upsert resets it.
|
||||
//
|
||||
// It reflects over the STRUCT rather than over json.Marshal output: the four
|
||||
// pointer fields carry `omitempty`, so a nil pointer legitimately disappears
|
||||
// from the marshaled form — that absence is precisely the "not mentioned" signal
|
||||
// the handler relies on. Checking marshaled keys would therefore report a
|
||||
// healthy field as missing.
|
||||
func TestSourcePayloadCoversEveryEditableField(t *testing.T) {
|
||||
// Fields of config.Source that a source edit can legitimately set.
|
||||
want := []string{
|
||||
"name", "base_url", "api_key", "api_key_env", "adapter", "endpoint",
|
||||
"image_endpoint", "models", "headers", "proxy_url", "meta",
|
||||
"temperature", "max_tokens", "max_concurrent", "rpm",
|
||||
"timeout", "queue_timeout",
|
||||
}
|
||||
pt := reflect.TypeOf(sourcePayload{})
|
||||
for _, f := range want {
|
||||
if _, ok := pt.FieldByName(f); !ok {
|
||||
// Tolerate a differently-spelled Go field only when the JSON tag
|
||||
// matches, so the check stays about the wire contract.
|
||||
if !hasJSONTag(pt, f) {
|
||||
t.Errorf("config.Source has %q but sourcePayload has no such JSON key: "+
|
||||
"an edit that omits it resets the stored value", f)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// hasJSONTag reports whether any field of t carries the given JSON name.
|
||||
func hasJSONTag(t reflect.Type, name string) bool {
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
if jsonName(t.Field(i)) == name {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// jsonName returns the JSON name of a struct field, falling back to the Go
|
||||
// field name when no tag is present.
|
||||
func jsonName(f reflect.StructField) string {
|
||||
tag := f.Tag.Get("json")
|
||||
if tag == "" {
|
||||
return f.Name
|
||||
}
|
||||
if n := strings.Split(tag, ",")[0]; n != "" {
|
||||
return n
|
||||
}
|
||||
return f.Name
|
||||
}
|
||||
@ -1,259 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// These cover the two write-path defects found while adding a model to a live
|
||||
// source by hand:
|
||||
//
|
||||
// 1. PUT /api/sources/{name} was documented in the API index but not
|
||||
// implemented, so it answered 405 while POST (which does the same upsert)
|
||||
// worked. A caller following the documented contract could not edit a
|
||||
// source at all.
|
||||
// 2. POST is an upsert that replaces the whole source, so a partial edit that
|
||||
// did not carry api_key persisted an empty/placeholder credential. The
|
||||
// source kept its name and models and answered 401 on the next call, well
|
||||
// after the writing script had exited 0.
|
||||
//
|
||||
// The second one is the dangerous class: a write that reports success and
|
||||
// breaks the source later.
|
||||
|
||||
// sourceAPIKey reads the stored key back the way an operator would, i.e. what
|
||||
// the routes actually use, not what the file happens to contain.
|
||||
func sourceAPIKey(t *testing.T, g *Gateway, name string) string {
|
||||
t.Helper()
|
||||
for _, s := range g.core.Sources() {
|
||||
if s.Name == name {
|
||||
return s.APIKey
|
||||
}
|
||||
}
|
||||
t.Fatalf("source %q not found", name)
|
||||
return ""
|
||||
}
|
||||
|
||||
func seededGateway(t *testing.T) (*Gateway, string) {
|
||||
t.Helper()
|
||||
up := mockUpstream()
|
||||
t.Cleanup(up.Close)
|
||||
g := newTestGateway(t, config.Source{
|
||||
Name: "seeded",
|
||||
BaseURL: up.URL,
|
||||
Adapter: "openai",
|
||||
APIKey: "sk-real-credential",
|
||||
Models: []config.Model{{ID: "m-one", Kind: "chat"}},
|
||||
})
|
||||
return g, up.URL
|
||||
}
|
||||
|
||||
func TestSourcePutUpdatesSource(t *testing.T) {
|
||||
g, up := seededGateway(t)
|
||||
|
||||
// A single-field edit: change one model's priority, name the source only in
|
||||
// the path. This is the shape the API index documents for PUT.
|
||||
rr := doReq(t, g, "PUT", "/api/sources/seeded", fmt.Sprintf(
|
||||
`{"base_url":%q,"adapter":"openai","api_key":"sk-real-credential","models":[{"id":"m-one","priority":42,"kind":"chat"}]}`, up))
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("PUT status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
|
||||
var found bool
|
||||
for _, s := range g.core.Sources() {
|
||||
if s.Name != "seeded" {
|
||||
continue
|
||||
}
|
||||
found = true
|
||||
for _, m := range s.Models {
|
||||
if m.ID == "m-one" && m.Priority != 42 {
|
||||
t.Fatalf("priority not applied: %+v", s.Models)
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Fatal("source vanished after PUT")
|
||||
}
|
||||
// And it must be routable, not merely stored.
|
||||
if rr := doReq(t, g, "GET", "/v1/models", ""); rr.Code != http.StatusOK ||
|
||||
!strings.Contains(rr.Body.String(), "m-one") {
|
||||
t.Fatalf("model not routable after PUT: %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourcePutRejectsNameMismatch(t *testing.T) {
|
||||
g, _ := seededGateway(t)
|
||||
// Body names a different source than the path. Silently preferring either
|
||||
// one would let a client edit a source it did not name.
|
||||
rr := doReq(t, g, "PUT", "/api/sources/seeded", `{"name":"other","base_url":"http://x","models":[{"id":"m"}]}`)
|
||||
if rr.Code != http.StatusBadRequest {
|
||||
t.Fatalf("name mismatch accepted: status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if sourceAPIKey(t, g, "seeded") != "sk-real-credential" {
|
||||
t.Fatal("rejected PUT still mutated the source")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourcePutRequiresNameInPath(t *testing.T) {
|
||||
g, _ := seededGateway(t)
|
||||
rr := doReq(t, g, "PUT", "/api/sources", `{"base_url":"http://x","models":[{"id":"m"}]}`)
|
||||
if rr.Code != http.StatusBadRequest {
|
||||
t.Fatalf("PUT without a name in the path: status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceUpsertKeepsAPIKey(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
sentKey string
|
||||
}{
|
||||
{"explicit mask", keepExistingAPIKey},
|
||||
{"omitted key", ""},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
g, up := seededGateway(t)
|
||||
rr := doReq(t, g, "PUT", "/api/sources/seeded", fmt.Sprintf(
|
||||
`{"base_url":%q,"adapter":"openai","api_key":%q,"models":[{"id":"m-one"},{"id":"m-two"}]}`,
|
||||
up, tc.sentKey))
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if got := sourceAPIKey(t, g, "seeded"); got != "sk-real-credential" {
|
||||
t.Fatalf("api_key was clobbered: got %q, want the pre-existing credential", got)
|
||||
}
|
||||
// The failure this guards is a source that looks configured but
|
||||
// cannot authenticate, so assert the route still works.
|
||||
if rr := doReq(t, g, "GET", "/v1/models", ""); rr.Code != http.StatusOK {
|
||||
t.Fatalf("source unusable after edit: %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceUpsertStillWritesNewKey(t *testing.T) {
|
||||
// The keep-existing path must not swallow a deliberate rotation.
|
||||
g, up := seededGateway(t)
|
||||
rr := doReq(t, g, "PUT", "/api/sources/seeded", fmt.Sprintf(
|
||||
`{"base_url":%q,"adapter":"openai","api_key":"sk-rotated","models":[{"id":"m-one"}]}`, up))
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if got := sourceAPIKey(t, g, "seeded"); got != "sk-rotated" {
|
||||
t.Fatalf("rotation ignored: got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceUpsertNewSourceMayHaveNoKey(t *testing.T) {
|
||||
// Empty is a real value for a credential-less source (local upstreams), so
|
||||
// inherit-on-empty must not invent one for a source that does not exist yet.
|
||||
up := mockUpstream()
|
||||
defer up.Close()
|
||||
g := newTestGateway(t)
|
||||
rr := doReq(t, g, "POST", "/api/sources", fmt.Sprintf(
|
||||
`{"name":"noauth","base_url":%q,"adapter":"openai","models":[{"id":"local-m"}]}`, up.URL))
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if got := sourceAPIKey(t, g, "noauth"); got != "" {
|
||||
t.Fatalf("unexpected key on a new credential-less source: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceModelIDsIsAdditive(t *testing.T) {
|
||||
// "Add one model" is the edit that motivated the defect: it needs no read
|
||||
// of the existing list, so it must not require one.
|
||||
g, up := seededGateway(t)
|
||||
rr := doReq(t, g, "PUT", "/api/sources/seeded", fmt.Sprintf(
|
||||
`{"base_url":%q,"adapter":"openai","api_key":%q,"model_ids":["m-two","m-two"," m-three "]}`, up, keepExistingAPIKey))
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var ids []string
|
||||
for _, s := range g.core.Sources() {
|
||||
if s.Name == "seeded" {
|
||||
for _, m := range s.Models {
|
||||
ids = append(ids, m.ID)
|
||||
}
|
||||
}
|
||||
}
|
||||
// m-one kept, m-two not duplicated, blank entries dropped, whitespace trimmed.
|
||||
want := map[string]int{"m-one": 1, "m-two": 1, "m-three": 1}
|
||||
if len(ids) != len(want) {
|
||||
t.Fatalf("models=%v, want exactly %v", ids, want)
|
||||
}
|
||||
seen := map[string]int{}
|
||||
for _, id := range ids {
|
||||
seen[id]++
|
||||
}
|
||||
for id, n := range want {
|
||||
if seen[id] != n {
|
||||
t.Fatalf("models=%v: %q appears %d times, want %d", ids, id, seen[id], n)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceWriteRejectsNonAdmin(t *testing.T) {
|
||||
g, up := seededGateway(t)
|
||||
for _, m := range []string{"POST", "PUT", "DELETE"} {
|
||||
req, _ := http.NewRequest(m, "/api/sources/seeded", strings.NewReader(fmt.Sprintf(
|
||||
`{"name":"seeded","base_url":%q,"models":[{"id":"m"}]}`, up)))
|
||||
req.Header.Set("Authorization", "Bearer sk-not-a-real-key")
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
rr := httptest.NewRecorder()
|
||||
g.Handler().ServeHTTP(rr, req)
|
||||
if rr.Code == http.StatusOK {
|
||||
t.Fatalf("%s allowed for an unknown key: %s", m, rr.Body.String())
|
||||
}
|
||||
}
|
||||
if sourceAPIKey(t, g, "seeded") != "sk-real-credential" {
|
||||
t.Fatal("unauthenticated write mutated the source")
|
||||
}
|
||||
}
|
||||
|
||||
// TestSourceWritePersistsToDisk guards the difference between "the in-memory
|
||||
// registry accepted it" and "it will survive a restart". The api_key must be
|
||||
// sealed at rest, never the literal placeholder.
|
||||
func TestSourceWritePersistsToDisk(t *testing.T) {
|
||||
g, up := seededGateway(t)
|
||||
rr := doReq(t, g, "PUT", "/api/sources/seeded", fmt.Sprintf(
|
||||
`{"base_url":%q,"adapter":"openai","api_key":%q,"models":[{"id":"m-one"}]}`, up, keepExistingAPIKey))
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
raw, err := os.ReadFile(g.core.Config().Path)
|
||||
if err != nil {
|
||||
t.Fatalf("read config: %v", err)
|
||||
}
|
||||
text := string(raw)
|
||||
if strings.Contains(text, keepExistingAPIKey) {
|
||||
t.Fatal("the keep-mask was persisted as a literal api_key")
|
||||
}
|
||||
if strings.Contains(text, "sk-real-credential") {
|
||||
t.Fatal("api_key stored in plaintext; it must be sealed at rest")
|
||||
}
|
||||
if !strings.Contains(text, "m-one") {
|
||||
t.Fatal("source not persisted")
|
||||
}
|
||||
// The masked GET view must not leak the credential either.
|
||||
rr = doReq(t, g, "GET", "/api/v1/sources/seeded", "")
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("get status=%d", rr.Code)
|
||||
}
|
||||
var view struct {
|
||||
Source struct {
|
||||
APIKey string `json:"api_key"`
|
||||
} `json:"source"`
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &view); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
if view.Source.APIKey == "sk-real-credential" {
|
||||
t.Fatal("GET /api/v1/sources leaked the plaintext api_key")
|
||||
}
|
||||
}
|
||||
@ -48,27 +48,6 @@ type Req struct {
|
||||
Status int `json:"status"`
|
||||
// Err short error message
|
||||
Err string `json:"error,omitempty"`
|
||||
// ImageCount is how many images an image-generation request produced.
|
||||
// Kept separate from Compl/Prompt: image generation has no token concept,
|
||||
// so counting images as "completion tokens" would corrupt the token totals.
|
||||
ImageCount int `json:"image_count,omitempty"`
|
||||
|
||||
// Walk is the AUTO chain's step-by-step trace for this request: which tiers
|
||||
// were skipped and why, which slots hard-failed, which one served it. It is
|
||||
// the only way a consumer can tell "tier 1 served this" from "tier 1 was
|
||||
// cooling so we dropped to tier 3" — a distinction that is the entire point
|
||||
// of a priority chain.
|
||||
//
|
||||
// json:"-" — deliberately NOT persisted. The audit file is a hot append and
|
||||
// this is observational detail: on a degraded gateway every request would
|
||||
// carry a multi-element array, and the audit trail's own retention (16 files
|
||||
// x 16 MB) is already the largest thing on the box. A plugin that wants the
|
||||
// walk sees it live at request_end; an operator post-mortem reads it from the
|
||||
// plugin's own accumulated state or from /api/auto slot health.
|
||||
//
|
||||
// Only populated when a plugin is loaded (chainTraceSink returns nil
|
||||
// otherwise), so a gateway with no plugins allocates nothing for it.
|
||||
Walk []map[string]interface{} `json:"-"`
|
||||
}
|
||||
|
||||
// Stat aggregates counters for one dimension row.
|
||||
@ -91,16 +70,6 @@ type agrRow struct {
|
||||
|
||||
// Stats collects per-key / per-model / per-source aggregates plus a bounded
|
||||
// ring of raw request records, all guarded by one mutex.
|
||||
//
|
||||
// LOCKING: mu is a plain sync.Mutex and is NOT reentrant. The *Locked methods
|
||||
// (aggregateLocked, addKeyTokenLocked, addKeyHourLocked, addKeyReqLocked,
|
||||
// wantPinnedBuckets, rotateAuditLocked, …) assume the caller already holds it,
|
||||
// while every other exported method takes it itself.
|
||||
//
|
||||
// Calling an exported method while already holding mu DEADLOCKS. This is not
|
||||
// hypothetical: a test that did KeyWindowReqs under s.mu.Lock() hung until the
|
||||
// 10-minute panic timeout. Always reach for the *Locked variant when the lock
|
||||
// is already held, and prefer the exported method when it is not.
|
||||
type Stats struct {
|
||||
mu sync.Mutex
|
||||
active int64
|
||||
@ -115,21 +84,6 @@ type Stats struct {
|
||||
auditPath string
|
||||
replayPartial bool // aggregates built from a bounded audit tail
|
||||
modelHour map[string]map[int64]int64 // model -> unix-hour bucket -> tokens
|
||||
// keyModelHour buckets the same tokens as modelHour but keyed by
|
||||
// (gateway key id, model) so per-key model quotas are isolated from each
|
||||
// other. modelHour stays key-blind on purpose: it backs the AUTO slot
|
||||
// quota, which is a gateway-wide limit on a slot, not a per-key one.
|
||||
keyModelHour map[string]map[string]map[int64]int64 // key -> model -> unix-hour -> tokens
|
||||
// keyHour buckets a key's total tokens per unix hour, backing the
|
||||
// whole-key quota (all models of one key share one budget).
|
||||
keyHour map[string]map[int64]int64
|
||||
// keyReqHour buckets a key's request count per unix hour, backing the
|
||||
// key-wide request quota. Counted for every request type (chat, stream,
|
||||
// image) including failed ones, so a failing client cannot loop for free.
|
||||
keyReqHour map[string]map[int64]int64
|
||||
// pinnedWanted records which (key, model) pairs have a quota that pins a
|
||||
// source, so the pinned bucket is maintained only where it is read.
|
||||
pinnedWanted map[string]bool
|
||||
}
|
||||
|
||||
const hourSec = 3600
|
||||
@ -162,18 +116,14 @@ func NewStats(maxRecords int) *Stats {
|
||||
maxRecords = defaultRingSize
|
||||
}
|
||||
return &Stats{
|
||||
byKey: map[string]*Stat{},
|
||||
byModel: map[string]*Stat{},
|
||||
bySrc: map[string]*Stat{},
|
||||
byKeyModel: map[string]map[string]*Stat{},
|
||||
byKeySrc: map[string]map[string]*Stat{},
|
||||
byStatus: map[int]*Stat{},
|
||||
modelHour: map[string]map[int64]int64{},
|
||||
keyModelHour: map[string]map[string]map[int64]int64{},
|
||||
keyHour: map[string]map[int64]int64{},
|
||||
keyReqHour: map[string]map[int64]int64{},
|
||||
pinnedWanted: map[string]bool{},
|
||||
maxRecs: maxRecords,
|
||||
byKey: map[string]*Stat{},
|
||||
byModel: map[string]*Stat{},
|
||||
bySrc: map[string]*Stat{},
|
||||
byKeyModel: map[string]map[string]*Stat{},
|
||||
byKeySrc: map[string]map[string]*Stat{},
|
||||
byStatus: map[int]*Stat{},
|
||||
modelHour: map[string]map[int64]int64{},
|
||||
maxRecs: maxRecords,
|
||||
}
|
||||
}
|
||||
|
||||
@ -368,9 +318,6 @@ func (s *Stats) Record(r Req) {
|
||||
// aggregateLocked folds r into every aggregate row and the quota window
|
||||
// bucket. Caller must hold s.mu.
|
||||
func (s *Stats) aggregateLocked(r Req) {
|
||||
if r.Key != "" {
|
||||
s.addKeyReqLocked(r.Key, (r.Time/1000)/hourSec, 1)
|
||||
}
|
||||
inc(s.byKey, r.Key, r)
|
||||
if r.Model != "" {
|
||||
inc(s.byModel, r.Model, r)
|
||||
@ -415,12 +362,6 @@ func (s *Stats) aggregateLocked(r Req) {
|
||||
s.modelHour[key] = hm
|
||||
}
|
||||
hm[h] += tok
|
||||
// per-key buckets: same token split, but isolated per key so one
|
||||
// key's quota cannot be drained by another key's traffic
|
||||
if r.Key != "" {
|
||||
s.addKeyTokenLocked(r.Key, r.Model, r.Source, h, tok)
|
||||
s.addKeyHourLocked(r.Key, h, tok)
|
||||
}
|
||||
// retention: 24*40 = 960 hourly buckets ≈ 40 days of history (covers
|
||||
// the longest "month" quota window)
|
||||
if len(hm) > 24*40 {
|
||||
@ -433,185 +374,6 @@ func (s *Stats) aggregateLocked(r Req) {
|
||||
}
|
||||
}
|
||||
|
||||
// quotaRetentionHours is how much hourly history the per-key quota buckets
|
||||
// keep. It matches the modelHour retention (40 days) so the longest "month"
|
||||
// window is fully covered after a restart, and it is deliberately applied per
|
||||
// key so an idle key's buckets are reclaimed instead of pinning memory.
|
||||
const quotaRetentionHours = 24 * 40
|
||||
|
||||
// addKeyTokenLocked adds tok to one key's (model[, source]) hourly bucket.
|
||||
//
|
||||
// The bare-model bucket is always written: it is what a scope entry without a
|
||||
// source pin reads, and writing it is what makes that quota work at all (every
|
||||
// recorded request carries a resolved source, so a source::model-only bucket
|
||||
// would leave an unpinned quota reading an empty bucket forever).
|
||||
//
|
||||
// The pinned "source::model" bucket is written only for (key, model) pairs
|
||||
// some scope entry actually pins a source for. Maintaining it unconditionally
|
||||
// doubles the bucket count — measured at 26.7 MB for 20 keys x 8 models x 3
|
||||
// sources at full retention, against a documented ~32 MB total budget — to
|
||||
// serve a lookup that almost nobody performs.
|
||||
func (s *Stats) addKeyTokenLocked(key, model, source string, h, tok int64) {
|
||||
if model == "" {
|
||||
return
|
||||
}
|
||||
byModel := s.keyModelHour[key]
|
||||
if byModel == nil {
|
||||
byModel = map[string]map[int64]int64{}
|
||||
s.keyModelHour[key] = byModel
|
||||
}
|
||||
addBucket(byModel, model, h, tok)
|
||||
if source == "" {
|
||||
return
|
||||
}
|
||||
pinned := source + "::" + model
|
||||
if pinned == model || !s.pinnedWanted[key+"\x00"+model] {
|
||||
return
|
||||
}
|
||||
addBucket(byModel, pinned, h, tok)
|
||||
}
|
||||
|
||||
// addBucket adds tok to one named hourly bucket and enforces retention.
|
||||
func addBucket(byModel map[string]map[int64]int64, name string, h, tok int64) {
|
||||
hm := byModel[name]
|
||||
if hm == nil {
|
||||
hm = map[int64]int64{}
|
||||
byModel[name] = hm
|
||||
}
|
||||
hm[h] += tok
|
||||
if len(hm) > quotaRetentionHours {
|
||||
for k := range hm {
|
||||
if k < h-quotaRetentionHours {
|
||||
delete(hm, k)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// wantPinnedBuckets starts maintaining the pinned (source::model) bucket for a
|
||||
// key's model, so a quota that pins a source counts traffic from the moment it
|
||||
// is configured. Usage recorded before that cannot be attributed to a source
|
||||
// after the fact, so a pinned quota may under-count its first window.
|
||||
func (s *Stats) wantPinnedBuckets(key, model string) {
|
||||
if key == "" || model == "" {
|
||||
return
|
||||
}
|
||||
s.pinnedWanted[key+"\x00"+model] = true
|
||||
}
|
||||
|
||||
// addKeyHourLocked adds tok to one key's all-model hourly total.
|
||||
func (s *Stats) addKeyHourLocked(key string, h, tok int64) {
|
||||
hm := s.keyHour[key]
|
||||
if hm == nil {
|
||||
hm = map[int64]int64{}
|
||||
s.keyHour[key] = hm
|
||||
}
|
||||
hm[h] += tok
|
||||
if len(hm) > quotaRetentionHours {
|
||||
for k := range hm {
|
||||
if k < h-quotaRetentionHours {
|
||||
delete(hm, k)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// KeyWindowReqs returns how many requests one key issued within sec seconds;
|
||||
// sec <= 0 means all retained history.
|
||||
func (s *Stats) KeyWindowReqs(key string, sec int64) int64 {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
return sumBuckets(s.keyReqHour[key], time.Now().Unix(), sec)
|
||||
}
|
||||
|
||||
// addKeyReqLocked adds n requests to a key's hourly count bucket.
|
||||
func (s *Stats) addKeyReqLocked(key string, h, n int64) {
|
||||
hm := s.keyReqHour[key]
|
||||
if hm == nil {
|
||||
hm = map[int64]int64{}
|
||||
s.keyReqHour[key] = hm
|
||||
}
|
||||
hm[h] += n
|
||||
if len(hm) > quotaRetentionHours {
|
||||
for k := range hm {
|
||||
if k < h-quotaRetentionHours {
|
||||
delete(hm, k)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// sumBuckets totals the hourly buckets inside the window ending at `now`
|
||||
// (unix seconds). sec <= 0 means "all retained history" (no reset).
|
||||
//
|
||||
// It only visits the hour buckets a window can actually cover. The naive full
|
||||
// scan measured 5.9us/op at the full 40-day retention, and a per-key quota
|
||||
// check runs on every request (2-3 of these), so the scan cost lands directly
|
||||
// on the request path; visiting at most sec/3600 + 1 buckets makes it
|
||||
// proportional to the window instead of to the retained history. The map keys
|
||||
// are whole unix hours, so the covered range is exact rather than a sample.
|
||||
func sumBuckets(hm map[int64]int64, now, sec int64) int64 {
|
||||
if len(hm) == 0 {
|
||||
return 0
|
||||
}
|
||||
if sec <= 0 {
|
||||
var total int64
|
||||
for _, v := range hm {
|
||||
total += v
|
||||
}
|
||||
return total
|
||||
}
|
||||
nowHour := now / hourSec
|
||||
// A bucket at hour H covers [H*3600, (H+1)*3600). The pre-existing
|
||||
// WindowTokens counts a bucket when h*3600 >= now-sec, so the first hour
|
||||
// that qualifies is ceil((now-sec)/3600) — NOT floor, which would pull in
|
||||
// one bucket too many (caught by TestSumBucketsMatchesFullScan: a 30-day
|
||||
// window reported 8878 vs 8649).
|
||||
firstHour := (now - sec + hourSec - 1) / hourSec
|
||||
if firstHour < 0 {
|
||||
firstHour = 0
|
||||
}
|
||||
var total int64
|
||||
for h := firstHour; h <= nowHour; h++ {
|
||||
total += hm[h]
|
||||
}
|
||||
return total
|
||||
}
|
||||
|
||||
// KeyWindowModelTokens returns the tokens one key consumed for one model
|
||||
// (optionally pinned to a source) within sec seconds; sec <= 0 means all
|
||||
// retained history. Unlike WindowTokens this is isolated per key, which is
|
||||
// what a per-key model quota needs.
|
||||
//
|
||||
// A source pin matches the pinned bucket exactly. Without a pin the call
|
||||
// totals the key's bare-model bucket, which counts traffic on every source —
|
||||
// a quota on "this model" should not stop applying just because the request
|
||||
// happened to be served by a different upstream.
|
||||
func (s *Stats) KeyWindowModelTokens(key, model, source string, sec int64) int64 {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
byModel := s.keyModelHour[key]
|
||||
if len(byModel) == 0 {
|
||||
return 0
|
||||
}
|
||||
mk := model
|
||||
if source != "" {
|
||||
mk = source + "::" + model
|
||||
// from now on this pair keeps a pinned bucket, so a quota configured
|
||||
// against this source counts the traffic it is meant to cap
|
||||
s.wantPinnedBuckets(key, model)
|
||||
}
|
||||
return sumBuckets(byModel[mk], time.Now().Unix(), sec)
|
||||
}
|
||||
|
||||
// KeyWindowTokens returns one key's total tokens across all models within sec
|
||||
// seconds; sec <= 0 means all retained history. It backs the whole-key quota.
|
||||
func (s *Stats) KeyWindowTokens(key string, sec int64) int64 {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
return sumBuckets(s.keyHour[key], time.Now().Unix(), sec)
|
||||
}
|
||||
|
||||
// rotateAuditLocked renames the audit file to <path>.<unix>.old once it
|
||||
// exceeds auditRotateBytes and prunes old files beyond auditKeepOld, keeping
|
||||
// the newest ones. Caller must hold s.mu.
|
||||
@ -702,27 +464,6 @@ func AutoPeriodSeconds(period string, hours int64) int64 {
|
||||
return 0
|
||||
}
|
||||
|
||||
// AutoSecondsToReset returns how many seconds remain until the quota window
|
||||
// rolls over, for the Retry-After header on a 429. Buckets are whole unix
|
||||
// hours, so the value is rounded up to the next hour boundary and never
|
||||
// exceeds one window. 0 means "unknown" (never resets, or no period set), in
|
||||
// which case the caller should not advertise a retry time.
|
||||
func AutoSecondsToReset(period string, hours int64) int64 {
|
||||
sec := AutoPeriodSeconds(period, hours)
|
||||
if sec <= 0 {
|
||||
return 0
|
||||
}
|
||||
now := time.Now().Unix()
|
||||
// window = the last sec seconds, bucketed by whole hours; the oldest
|
||||
// bucket still inside the window expires at its own hour boundary
|
||||
elapsed := now % hourSec
|
||||
left := sec - elapsed
|
||||
if left <= 0 || left > sec {
|
||||
left = sec
|
||||
}
|
||||
return left
|
||||
}
|
||||
|
||||
// WindowTokens returns the tokens consumed for one model (optionally pinned
|
||||
// to a single source) within the window; sec <= 0 means all time. Buckets are
|
||||
// whole unix hours, so a sliding window overcounts by up to one hour — an
|
||||
|
||||
@ -1,132 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The Stats mutex is a plain sync.Mutex: calling an exported (self-locking)
|
||||
// method while already holding it deadlocks. That happened for real — a probe
|
||||
// held s.mu and called KeyWindowReqs, hanging until the test binary's 10-minute
|
||||
// panic timeout — so the rule is pinned here rather than left in a comment.
|
||||
//
|
||||
// The check is structural: every method that takes the lock must say so in its
|
||||
// name or its doc comment. That keeps the trap visible when someone adds the
|
||||
// next method, which is the only time the rule can be forgotten.
|
||||
|
||||
// exportedSelfLocking lists the exported Stats methods that take the mutex.
|
||||
// It is derived from reflection at run time; the assertions below are what
|
||||
// actually pin the convention.
|
||||
func TestStatsExportedMethodsDocumentTheirLocking(t *testing.T) {
|
||||
typ := reflect.TypeOf(&Stats{})
|
||||
for i := 0; i < typ.NumMethod(); i++ {
|
||||
m := typ.Method(i)
|
||||
if m.PkgPath != "" { // unexported
|
||||
continue
|
||||
}
|
||||
// Methods that neither lock nor touch guarded state are fine either way;
|
||||
// the ones that matter are those reaching into the maps under mu.
|
||||
if !methodTouchesLockedState(m.Name) {
|
||||
continue
|
||||
}
|
||||
if strings.HasSuffix(m.Name, "Locked") {
|
||||
t.Errorf("Stats.%s is exported but named *Locked; the suffix means "+
|
||||
"the CALLER holds the lock, so it must not be exported", m.Name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// methodTouchesLockedState is the set of exported methods known to read or
|
||||
// write state guarded by Stats.mu. Kept explicit (rather than inferred) so a
|
||||
// new method is not silently assumed safe.
|
||||
func methodTouchesLockedState(name string) bool {
|
||||
switch name {
|
||||
case "KeyWindowReqs", "KeyWindowModelTokens", "KeyWindowTokens",
|
||||
"WindowTokens", "KeyTokens", "ModelTokens", "Record", "AppendAudit",
|
||||
"LoadAudit", "Snapshot", "SourceRecent", "SourceAverages",
|
||||
"AuditRecords", "AuditPage", "StreamAuditRecords", "ReplayPartial",
|
||||
"PoolStats":
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// TestStatsLockedMethodsAreNotCalledUnderLock is the behavioural half: it
|
||||
// proves the internal helpers the ones above use are reachable while the lock
|
||||
// is held. If a future refactor makes an exported method call a *Locked one
|
||||
// while holding mu itself, this is where it shows up — as a hang, bounded by
|
||||
// the short timeout below rather than the suite's 10 minutes.
|
||||
func TestStatsLockedMethodsAreNotCalledUnderLock(t *testing.T) {
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
defer close(done)
|
||||
s := NewStats(10)
|
||||
h := time.Now().Unix() / 3600
|
||||
s.mu.Lock()
|
||||
// Exactly the shape that deadlocked: the *Locked variants are correct
|
||||
// here because the lock is already held.
|
||||
s.addKeyReqLocked("k", h, 5)
|
||||
s.addKeyHourLocked("k", h, 100)
|
||||
s.wantPinnedBuckets("k", "m1")
|
||||
s.addKeyTokenLocked("k", "m1", "src", h, 100)
|
||||
// sumBuckets is the pure inner function the exported readers call.
|
||||
if got := sumBuckets(s.keyReqHour["k"], time.Now().Unix(), 3600); got != 5 {
|
||||
t.Errorf("sumBuckets under lock = %d, want 5", got)
|
||||
}
|
||||
s.mu.Unlock()
|
||||
}()
|
||||
select {
|
||||
case <-done:
|
||||
case <-time.After(20 * time.Second):
|
||||
buf := make([]byte, 1<<16)
|
||||
n := runtime.Stack(buf, true)
|
||||
t.Fatalf("deadlocked while using the *Locked helpers under s.mu:\n%s", buf[:n])
|
||||
}
|
||||
}
|
||||
|
||||
// TestSumBucketsEdgeCases covers the boundaries the quota check depends on,
|
||||
// including the ones a regression would silently get wrong (an off-by-one here
|
||||
// either lets a quota leak or locks a key out early).
|
||||
func TestSumBucketsEdgeCases(t *testing.T) {
|
||||
const hour = 3600
|
||||
now := int64(10*hour + 61) // 10:00:61, i.e. just past the boundary
|
||||
buckets := map[int64]int64{
|
||||
0: 100, // ancient
|
||||
9: 200, // previous hour
|
||||
10: 7, // current hour
|
||||
}
|
||||
|
||||
if got := sumBuckets(buckets, now, 0); got != 307 {
|
||||
t.Errorf("sec<=0 (all history) = %d, want 307", got)
|
||||
}
|
||||
// 1h window covers buckets h >= ceil((now-3600)/3600) = 10 -> only bucket 10.
|
||||
if got := sumBuckets(buckets, now, hour); got != 7 {
|
||||
t.Errorf("1h window = %d, want 7 (buckets 0 and 9 fall outside)", got)
|
||||
}
|
||||
// 2h window covers h >= 9 -> buckets 9 and 10.
|
||||
if got := sumBuckets(buckets, now, 2*hour); got != 207 {
|
||||
t.Errorf("2h window = %d, want 207", got)
|
||||
}
|
||||
// An empty map and a nil map must both be zero, not a panic.
|
||||
if got := sumBuckets(map[int64]int64{}, now, hour); got != 0 {
|
||||
t.Errorf("empty map = %d, want 0", got)
|
||||
}
|
||||
if got := sumBuckets(nil, now, hour); got != 0 {
|
||||
t.Errorf("nil map = %d, want 0", got)
|
||||
}
|
||||
// now < sec must not produce a negative firstHour index: with a window far
|
||||
// wider than the available history, every bucket that exists is counted.
|
||||
// (sumBuckets walks bucket indices from firstHour to nowHour, so passing a
|
||||
// "now" older than some buckets cannot reach them — that is correct, not a
|
||||
// truncation bug.)
|
||||
if got := sumBuckets(buckets, 10*hour+61, 100*hour); got != 307 {
|
||||
t.Errorf("window wider than history = %d, want 307", got)
|
||||
}
|
||||
// A window that predates every bucket counts them all as well.
|
||||
if got := sumBuckets(buckets, 10*hour+61, hour); got != 7 {
|
||||
t.Errorf("1h window at t=10:00:61 = %d, want 7", got)
|
||||
}
|
||||
}
|
||||
@ -1,234 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Period is a reporting window for the usage dashboard. The dashboard used to
|
||||
// have exactly one view — everything since process start — which made a quiet
|
||||
// morning indistinguishable from a quiet week. Periods give the operator a
|
||||
// scale to read the numbers at: today vs this week vs this month vs all time.
|
||||
//
|
||||
// The set is deliberately calendar-based and UTC-anchored. A rolling 24h window
|
||||
// would put "today" and "the last day" at different boundaries, so the same
|
||||
// number would move between two cards depending on when you looked; calendar
|
||||
// days are what people actually mean by "today". UTC also matches the billing
|
||||
// plugin's peak-window arithmetic, so a peak-rate hour does not land in a
|
||||
// different day in the cost view than in the usage view.
|
||||
type Period string
|
||||
|
||||
const (
|
||||
// PeriodDay is the current UTC calendar day.
|
||||
PeriodDay Period = "day"
|
||||
// PeriodWeek is the current ISO week (Mon 00:00 UTC to now).
|
||||
PeriodWeek Period = "week"
|
||||
// PeriodMonth is the current UTC calendar month.
|
||||
PeriodMonth Period = "month"
|
||||
// PeriodAll is since process start — the only view backed by the
|
||||
// in-memory aggregates, and the only one available when no audit file is
|
||||
// configured.
|
||||
PeriodAll Period = "all"
|
||||
)
|
||||
|
||||
// ValidPeriod reports whether p is a period the aggregator understands.
|
||||
// An unknown period is a client error, not a silent fallback to "all": a
|
||||
// dashboard that quietly shows lifetime totals when the caller asked for today
|
||||
// is worse than one that refuses.
|
||||
func ValidPeriod(p Period) bool {
|
||||
switch p {
|
||||
case PeriodDay, PeriodWeek, PeriodMonth, PeriodAll:
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// periodStart returns the inclusive start of the window for p at time now.
|
||||
// Only PeriodDay/Week/Month are meaningful here; PeriodAll returns 0, which
|
||||
// every "from > 0" bounds check treats as unbounded.
|
||||
func periodStart(p Period, now time.Time) int64 {
|
||||
now = now.UTC()
|
||||
switch p {
|
||||
case PeriodDay:
|
||||
return time.Date(now.Year(), now.Month(), now.Day(), 0, 0, 0, 0, time.UTC).UnixMilli()
|
||||
case PeriodWeek:
|
||||
// ISO week starts Monday. Go's Weekday() is Sunday=0, so the shift
|
||||
// below is 1 on Sunday and 0 on Monday..Saturday.
|
||||
off := (int(now.Weekday()) + 6) % 7
|
||||
d := now.AddDate(0, 0, -off)
|
||||
return time.Date(d.Year(), d.Month(), d.Day(), 0, 0, 0, 0, time.UTC).UnixMilli()
|
||||
case PeriodMonth:
|
||||
return time.Date(now.Year(), now.Month(), 1, 0, 0, 0, 0, time.UTC).UnixMilli()
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// PeriodBucket is one labelled point on the dashboard's timeline. Buckets are
|
||||
// the aggregation grain for a period: hourly for a day (so a spike is visible
|
||||
// inside "today"), daily for a week or month (so a week is not 7×24 points
|
||||
// wide and a month is not 31×24), and empty for "all" — a lifetime total has
|
||||
// no meaningful timeline, and pretending otherwise by drawing 500 hourly
|
||||
// buckets of rolling memory would be a different lie.
|
||||
type PeriodBucket struct {
|
||||
Bucket string `json:"bucket"`
|
||||
Stat
|
||||
}
|
||||
|
||||
// PeriodSnapshot is the period-scoped twin of Snapshot's payload: the same
|
||||
// totals and the same by_* rows, plus a timeline. It deliberately mirrors
|
||||
// Snapshot's field names so the dashboard's table painters work unchanged —
|
||||
// paintModelTable(st.by_model) reads rows of {name, ...Stat}, and that shape
|
||||
// does not care where the numbers came from.
|
||||
type PeriodSnapshot struct {
|
||||
Period Period `json:"period"`
|
||||
From int64 `json:"from"` // unix millis, 0 when PeriodAll
|
||||
Total Stat `json:"total"`
|
||||
ByKey []StatsRow `json:"by_key"`
|
||||
Models []StatsRow `json:"by_model"`
|
||||
Srcs []StatsRow `json:"by_source"`
|
||||
Status []agrRow `json:"by_status"`
|
||||
Bucket []PeriodBucket `json:"buckets"`
|
||||
// Truncated marks that the window was clipped by the retained audit
|
||||
// history, so the numbers are a lower bound rather than the true period
|
||||
// total. The dashboard shows this next to the numbers rather than letting
|
||||
// a rotated-away week read as "that week had no traffic".
|
||||
Truncated bool `json:"truncated"`
|
||||
}
|
||||
|
||||
// bucketKey maps a record's timestamp onto the timeline grain for p.
|
||||
// Daily buckets are stamped at UTC midnight; hourly buckets carry the hour.
|
||||
func bucketKey(p Period, ms int64) string {
|
||||
t := time.UnixMilli(ms).UTC()
|
||||
if p == PeriodDay {
|
||||
return t.Format("2006-01-02T15")
|
||||
}
|
||||
return t.Format("2006-01-02")
|
||||
}
|
||||
|
||||
// bucketOf returns the bucket label plus the truncated-flag side effects of
|
||||
// walking a file: a record older than the requested window means the window
|
||||
// starts before the retained history, and the file may have been cut short.
|
||||
func (s *Stats) PeriodSnapshot(p Period, key string, now time.Time) PeriodSnapshot {
|
||||
out := PeriodSnapshot{Period: p}
|
||||
from := periodStart(p, now)
|
||||
out.From = from
|
||||
|
||||
// "all" is exactly what Snapshot already answers, from the in-memory
|
||||
// aggregates, and it is the one view that must keep working with no audit
|
||||
// file configured at all (a fresh dev setup, or an operator who turned
|
||||
// auditing off). Serving it from the same code path keeps the dashboard's
|
||||
// "total" card identical whether or not a period is selected.
|
||||
if p == PeriodAll {
|
||||
snap := s.Snapshot(firstScreenRecords, key)
|
||||
if tot, ok := snap["total"].(Stat); ok {
|
||||
out.Total = tot
|
||||
}
|
||||
out.ByKey, _ = snap["by_key"].([]StatsRow)
|
||||
out.Models, _ = snap["by_model"].([]StatsRow)
|
||||
out.Srcs, _ = snap["by_source"].([]StatsRow)
|
||||
out.Status, _ = snap["by_status"].([]agrRow)
|
||||
// replay_partial is the same "these numbers came from a bounded
|
||||
// tail" caveat, carried through under this view's own name.
|
||||
out.Truncated, _ = snap["replay_partial"].(bool)
|
||||
return out
|
||||
}
|
||||
|
||||
// A bounded window is aggregated from the audit files, because the
|
||||
// in-memory aggregates are lifetime totals and the ring buffer holds only
|
||||
// maxRecs records (500 by default). Reading the ring would silently
|
||||
// under-report any period longer than the last few hundred requests.
|
||||
total := Stat{}
|
||||
byKey := map[string]*Stat{}
|
||||
byModel := map[string]*Stat{}
|
||||
bySrc := map[string]*Stat{}
|
||||
byStatus := map[string]*Stat{}
|
||||
buckets := map[string]*Stat{}
|
||||
|
||||
var seen int
|
||||
err := s.StreamAuditRecords(from, 0, key, func(r Req) error {
|
||||
seen++
|
||||
incStatus(&total, "", r)
|
||||
inc(byKey, r.Key, r)
|
||||
if r.Model != "" {
|
||||
inc(byModel, r.Model, r)
|
||||
}
|
||||
inc(bySrc, r.Source, r)
|
||||
if r.Status != 0 {
|
||||
inc(byStatus, strconv.Itoa(r.Status), r)
|
||||
}
|
||||
k := bucketKey(p, r.Time)
|
||||
b := buckets[k]
|
||||
if b == nil {
|
||||
b = &Stat{}
|
||||
buckets[k] = b
|
||||
}
|
||||
incStatus(b, k, r)
|
||||
return nil
|
||||
})
|
||||
|
||||
out.Total = total
|
||||
out.ByKey = rows(byKey)
|
||||
out.Models = rows(byModel)
|
||||
out.Srcs = rows(bySrc)
|
||||
bs := make([]agrRow, 0, len(byStatus))
|
||||
for code, st := range byStatus {
|
||||
bs = append(bs, agrRow{Name: code, Stat: *st})
|
||||
}
|
||||
sort.Slice(bs, func(i, j int) bool {
|
||||
ci, _ := strconv.Atoi(bs[i].Name)
|
||||
cj, _ := strconv.Atoi(bs[j].Name)
|
||||
return ci < cj
|
||||
})
|
||||
out.Status = bs
|
||||
|
||||
ks := make([]string, 0, len(buckets))
|
||||
for k := range buckets {
|
||||
ks = append(ks, k)
|
||||
}
|
||||
// Chronological, string-sorted: "2006-01-02T15" and "2006-01-02" both
|
||||
// sort lexicographically in time order, so no date parsing is needed.
|
||||
sort.Strings(ks)
|
||||
out.Bucket = make([]PeriodBucket, 0, len(ks))
|
||||
for _, k := range ks {
|
||||
out.Bucket = append(out.Bucket, PeriodBucket{Bucket: k, Stat: *buckets[k]})
|
||||
}
|
||||
|
||||
// The window is only "complete" if the audit walk actually reached back
|
||||
// far enough to cover it. Two ways it cannot:
|
||||
//
|
||||
// 1. The walk found nothing at all in a window that certainly had
|
||||
// traffic, because the files holding it rotated away.
|
||||
// 2. The walk errored part-way (I/O), leaving a partial total.
|
||||
//
|
||||
// Case 2 is reported from err; case 1 from seen == 0 combined with the
|
||||
// caller having asked for a bounded window. It is deliberately
|
||||
// conservative: a genuinely empty hour is rare enough that flagging it as
|
||||
// possibly-truncated costs one tooltip, whereas silently under-reporting a
|
||||
// month because rotation ate it is a wrong number with no indication.
|
||||
if err != nil || (seen == 0 && p != PeriodAll) {
|
||||
out.Truncated = true
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// periodFromQuery parses the ?period= parameter. An empty value means "all" so
|
||||
// that existing callers of /api/stats keep seeing exactly what they saw.
|
||||
// A malformed value is rejected by the caller (ValidPeriod) rather than
|
||||
// defaulting, so a typo in a bookmarked URL surfaces as an error instead of
|
||||
// silently switching the operator to lifetime totals.
|
||||
func periodFromQuery(q map[string][]string) Period {
|
||||
v := strings.ToLower(strings.TrimSpace(firstQuery(q, "period")))
|
||||
if v == "" {
|
||||
return PeriodAll
|
||||
}
|
||||
return Period(v)
|
||||
}
|
||||
|
||||
func firstQuery(q map[string][]string, key string) string {
|
||||
if vs, ok := q[key]; ok && len(vs) > 0 {
|
||||
return vs[0]
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@ -1,159 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/url"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The dashboard paints its records table from the `records` field of
|
||||
// /api/stats (paintRecords(st.records)). The period branch of that handler
|
||||
// used to hand-write a response map and simply omit the key, so switching the
|
||||
// selector to day / week / month rendered an EMPTY table no matter how much
|
||||
// traffic the window had — while the lifetime view worked, because it goes
|
||||
// through Snapshot() which carries records.
|
||||
//
|
||||
// These tests pin the contract across every period: same records, same cursor.
|
||||
func TestStatsPeriodViewsShipRecords(t *testing.T) {
|
||||
td := t.TempDir()
|
||||
g := newTestGateway(t)
|
||||
|
||||
// Write audit records the period aggregator and the record pager both read.
|
||||
g.stats.auditPath = filepath.Join(td, "audit.jsonl")
|
||||
now := time.Now().UnixMilli()
|
||||
for i := 0; i < 5; i++ {
|
||||
g.stats.Record(Req{
|
||||
Time: now - int64(i)*1000, Type: "chat", OK: true,
|
||||
Model: "m1", Source: "s1", Status: 200,
|
||||
Prompt: 10, Compl: 2,
|
||||
})
|
||||
}
|
||||
|
||||
for _, p := range []string{"day", "week", "month", "all"} {
|
||||
rr := doReq(t, g, "GET", "/api/stats?period="+p, "")
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("period=%s: HTTP %d: %s", p, rr.Code, rr.Body.String())
|
||||
}
|
||||
var st map[string]interface{}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &st); err != nil {
|
||||
t.Fatalf("period=%s: %v", p, err)
|
||||
}
|
||||
recs, ok := st["records"].([]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("period=%s: response has no `records` array — the dashboard "+
|
||||
"records table renders empty for this view (keys: %v)", p, keysOf(st))
|
||||
}
|
||||
if len(recs) != 5 {
|
||||
t.Fatalf("period=%s: got %d records, want 5", p, len(recs))
|
||||
}
|
||||
// The scroll handler pages from next_cursor; without it the table is
|
||||
// stuck at one screen no matter how far the user scrolls.
|
||||
if _, ok := st["next_cursor"]; !ok {
|
||||
t.Fatalf("period=%s: no next_cursor — paging can never continue", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The records shipped with a period view must be the newest-first page the
|
||||
// paging endpoint would return, in the same order. If /api/stats and
|
||||
// /api/stats/records disagreed, scrolling would duplicate or skip rows.
|
||||
func TestStatsPeriodRecordsMatchPager(t *testing.T) {
|
||||
td := t.TempDir()
|
||||
g := newTestGateway(t)
|
||||
g.stats.auditPath = filepath.Join(td, "audit.jsonl")
|
||||
now := time.Now().UnixMilli()
|
||||
for i := 0; i < 8; i++ {
|
||||
g.stats.Record(Req{
|
||||
Time: now - int64(i)*1000, Type: "chat", OK: true,
|
||||
Model: "m1", Status: 200, Prompt: 5, Compl: 1,
|
||||
})
|
||||
}
|
||||
|
||||
rr := doReq(t, g, "GET", "/api/stats?period=day&limit=100", "")
|
||||
var st struct {
|
||||
Records []struct {
|
||||
Time int64 `json:"time"`
|
||||
Model string `json:"model"`
|
||||
} `json:"records"`
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rr2 := doReq(t, g, "GET", "/api/stats/records?limit=100", "")
|
||||
var pg struct {
|
||||
Records []struct {
|
||||
Time int64 `json:"time"`
|
||||
Model string `json:"model"`
|
||||
} `json:"records"`
|
||||
}
|
||||
if err := json.Unmarshal(rr2.Body.Bytes(), &pg); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(st.Records) != len(pg.Records) {
|
||||
t.Fatalf("period view shipped %d records, pager returned %d — the two "+
|
||||
"sources must agree or scrolling duplicates/skips rows",
|
||||
len(st.Records), len(pg.Records))
|
||||
}
|
||||
for i := range st.Records {
|
||||
if st.Records[i].Time != pg.Records[i].Time {
|
||||
t.Fatalf("record %d differs: period view t=%d, pager t=%d",
|
||||
i, st.Records[i].Time, pg.Records[i].Time)
|
||||
}
|
||||
}
|
||||
// The two sources must agree on DIRECTION as well as content. Which
|
||||
// direction that is depends on how many reverse-read chunks the audit file
|
||||
// spans (a small file is read whole, a large one chunk-by-chunk), so the
|
||||
// test asserts agreement rather than hard-coding newest-first — but they
|
||||
// can never disagree, because the table interleaves both streams.
|
||||
if len(st.Records) > 1 {
|
||||
periodAsc := st.Records[1].Time > st.Records[0].Time
|
||||
pagerAsc := pg.Records[1].Time > pg.Records[0].Time
|
||||
if periodAsc != pagerAsc {
|
||||
t.Fatalf("direction differs: period view %s, pager %s — the table "+
|
||||
"would splice the two streams out of order",
|
||||
map[bool]string{true: "oldest-first", false: "newest-first"}[periodAsc],
|
||||
map[bool]string{true: "oldest-first", false: "newest-first"}[pagerAsc])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A non-admin caller cannot widen their view by passing someone else's key in
|
||||
// the query string: exportKey overrides ?key= with the caller's own masked id.
|
||||
// This is a data-leak boundary, and it applies to the records the period view
|
||||
// ships just as much as to the aggregates.
|
||||
func TestStatsPeriodRecordsIgnoreForeignKeyForUsers(t *testing.T) {
|
||||
td := t.TempDir()
|
||||
g := newTestGateway(t)
|
||||
g.stats.auditPath = filepath.Join(td, "audit.jsonl")
|
||||
|
||||
other := keyID("sk-gw-cccccccccccccccc")
|
||||
own := keyID("sk-test")
|
||||
now := time.Now().UnixMilli()
|
||||
g.stats.Record(Req{Time: now, OK: true, Model: "m", Status: 200, Key: other})
|
||||
g.stats.Record(Req{Time: now - 1000, OK: true, Model: "m", Status: 200, Key: own})
|
||||
|
||||
// sk-test authenticates as a non-admin (newTestGateway seeds it under
|
||||
// gateway_keys, not keys), so ?key=<other> must be ignored and the caller
|
||||
// sees only its own rows.
|
||||
rr := doReq(t, g, "GET",
|
||||
"/api/stats?period=day&key="+url.QueryEscape(other), "")
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("HTTP %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var st struct {
|
||||
Records []struct {
|
||||
Key string `json:"key"`
|
||||
} `json:"records"`
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, r := range st.Records {
|
||||
if r.Key == other {
|
||||
t.Fatalf("another key's record leaked into a user view "+
|
||||
"(caller=%s, requested=%s)", own, other)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,403 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// auditPeriod writes records straight into an audit file, which is what the
|
||||
// period aggregator reads. Going through Record() would also rotate/prune the
|
||||
// file; here the file shape is the thing under test.
|
||||
func auditPeriod(t *testing.T, path string, recs ...Req) {
|
||||
t.Helper()
|
||||
f, err := os.Create(path)
|
||||
if err != nil {
|
||||
t.Fatalf("create audit file: %v", err)
|
||||
}
|
||||
defer f.Close()
|
||||
for _, r := range recs {
|
||||
if r.Time == 0 {
|
||||
t.Fatal("test record needs an explicit timestamp")
|
||||
}
|
||||
if r.Type == "" {
|
||||
r.Type = "chat"
|
||||
}
|
||||
b, err := json.Marshal(r)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal record: %v", err)
|
||||
}
|
||||
if _, err := f.Write(append(b, '\n')); err != nil {
|
||||
t.Fatalf("write record: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func periodStats(t *testing.T, auditPath string) *Stats {
|
||||
t.Helper()
|
||||
s := NewStats(10)
|
||||
s.mu.Lock()
|
||||
s.auditPath = auditPath
|
||||
s.mu.Unlock()
|
||||
return s
|
||||
}
|
||||
|
||||
// TestPeriodWindowBoundaries is the boundary contract: a day window covers
|
||||
// exactly today-from-midnight, a week window starts Monday (not Sunday — the
|
||||
// Go Weekday() trap), and a month window starts on the 1st. All UTC.
|
||||
func TestPeriodWindowBoundaries(t *testing.T) {
|
||||
// Wednesday 2026-03-11 15:30 UTC.
|
||||
now := time.Date(2026, 3, 11, 15, 30, 0, 0, time.UTC)
|
||||
day := time.Date(2026, 3, 11, 0, 0, 0, 0, time.UTC)
|
||||
if got := time.UnixMilli(periodStart(PeriodDay, now)).UTC(); !got.Equal(day) {
|
||||
t.Errorf("day window starts %s, want %s", got, day)
|
||||
}
|
||||
// Wednesday minus 2 days = Monday the 9th.
|
||||
monday := time.Date(2026, 3, 9, 0, 0, 0, 0, time.UTC)
|
||||
if got := time.UnixMilli(periodStart(PeriodWeek, now)).UTC(); !got.Equal(monday) {
|
||||
t.Errorf("week window starts %s, want Monday %s", got, monday)
|
||||
}
|
||||
// Same instant on a Sunday must still start on the PRECEDING Monday,
|
||||
// never on the Sunday itself. This is the off-by-one that
|
||||
// int(now.Weekday()) would introduce (Sunday=0 → no shift).
|
||||
sun := time.Date(2026, 3, 15, 10, 0, 0, 0, time.UTC) // Sunday
|
||||
prevMon := time.Date(2026, 3, 9, 0, 0, 0, 0, time.UTC)
|
||||
if got := time.UnixMilli(periodStart(PeriodWeek, sun)).UTC(); !got.Equal(prevMon) {
|
||||
t.Errorf("Sunday week window starts %s, want the Monday before it %s", got, prevMon)
|
||||
}
|
||||
first := time.Date(2026, 3, 1, 0, 0, 0, 0, time.UTC)
|
||||
if got := time.UnixMilli(periodStart(PeriodMonth, now)).UTC(); !got.Equal(first) {
|
||||
t.Errorf("month window starts %s, want %s", got, first)
|
||||
}
|
||||
if got := periodStart(PeriodAll, now); got != 0 {
|
||||
t.Errorf("all window from = %d, want 0 (unbounded)", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodAggregationExcludesOlderRecords is the reason the aggregator reads
|
||||
// the audit file: a record outside the window must not be counted, even though
|
||||
// it sits in the very same file, right next to records that are.
|
||||
func TestPeriodAggregationExcludesOlderRecords(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "audit.jsonl")
|
||||
now := time.Date(2026, 3, 11, 15, 0, 0, 0, time.UTC)
|
||||
h := func(n int) int64 { return now.Add(-time.Duration(n) * time.Hour).UnixMilli() }
|
||||
|
||||
auditPeriod(t, path,
|
||||
// inside today
|
||||
Req{Time: h(1), Model: "m", Source: "s", Prompt: 100, Compl: 10, LatMs: 5, OK: true},
|
||||
Req{Time: h(2), Model: "m", Source: "s", Prompt: 200, Compl: 20, LatMs: 7, OK: true},
|
||||
// outside today (yesterday) but inside the week
|
||||
Req{Time: now.AddDate(0, 0, -1).UnixMilli(), Model: "old", Source: "s", Prompt: 999, Compl: 99, OK: true},
|
||||
)
|
||||
s := periodStats(t, path)
|
||||
|
||||
day := s.PeriodSnapshot(PeriodDay, "", now)
|
||||
if day.Total.Reqs != 2 {
|
||||
t.Errorf("day window counted %d requests, want 2 (yesterday's record must be excluded)", day.Total.Reqs)
|
||||
}
|
||||
if day.Total.Prompt != 300 {
|
||||
t.Errorf("day prompt tokens = %d, want 300", day.Total.Prompt)
|
||||
}
|
||||
|
||||
week := s.PeriodSnapshot(PeriodWeek, "", now)
|
||||
if week.Total.Reqs != 3 {
|
||||
t.Errorf("week window counted %d requests, want 3", week.Total.Reqs)
|
||||
}
|
||||
if week.Total.Prompt != 1299 {
|
||||
t.Errorf("week prompt tokens = %d, want 1299", week.Total.Prompt)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodDimensionsAggregateIndependently checks the by_* rows actually
|
||||
// split by their own key rather than all collapsing into one row.
|
||||
func TestPeriodDimensionsAggregateIndependently(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "audit.jsonl")
|
||||
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
|
||||
auditPeriod(t, path,
|
||||
Req{Time: now.UnixMilli(), Key: "k1", Model: "alpha", Source: "s1", Prompt: 10, OK: true},
|
||||
Req{Time: now.UnixMilli(), Key: "k1", Model: "beta", Source: "s2", Prompt: 20, OK: true},
|
||||
Req{Time: now.UnixMilli(), Key: "k2", Model: "alpha", Source: "s1", Prompt: 30, OK: true},
|
||||
)
|
||||
s := periodStats(t, path)
|
||||
got := s.PeriodSnapshot(PeriodDay, "", now)
|
||||
|
||||
if len(got.Models) != 2 {
|
||||
t.Fatalf("by_model has %d rows, want 2: %+v", len(got.Models), got.Models)
|
||||
}
|
||||
// alpha = 10 + 30
|
||||
for _, r := range got.Models {
|
||||
if r.Name == "alpha" && r.Prompt != 40 {
|
||||
t.Errorf("alpha prompt = %d, want 40", r.Prompt)
|
||||
}
|
||||
}
|
||||
if len(got.Srcs) != 2 {
|
||||
t.Errorf("by_source has %d rows, want 2: %+v", len(got.Srcs), got.Srcs)
|
||||
}
|
||||
if len(got.ByKey) != 2 {
|
||||
t.Errorf("by_key has %d rows, want 2: %+v", len(got.ByKey), got.ByKey)
|
||||
}
|
||||
// The rows must add up to the total, or the dashboard shows a total that
|
||||
// disagrees with its own table.
|
||||
var sum int64
|
||||
for _, r := range got.Models {
|
||||
sum += r.Prompt
|
||||
}
|
||||
if sum != got.Total.Prompt {
|
||||
t.Errorf("by_model prompts sum to %d but total is %d — the table would contradict the KPI", sum, got.Total.Prompt)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodDayBucketsAreHourlyAndOrdered pins the timeline grain and its
|
||||
// order. Day = hourly (a spike must be visible inside "today"); week/month =
|
||||
// daily. Sorted chronologically, because a chart fed unsorted buckets draws
|
||||
// nonsense.
|
||||
func TestPeriodDayBucketsAreHourlyAndOrdered(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "audit.jsonl")
|
||||
now := time.Date(2026, 3, 11, 15, 0, 0, 0, time.UTC)
|
||||
// Deliberately out of order in the file.
|
||||
auditPeriod(t, path,
|
||||
Req{Time: now.Add(-2 * time.Hour).UnixMilli(), Model: "m", Prompt: 1, OK: true},
|
||||
Req{Time: now.Add(-4 * time.Hour).UnixMilli(), Model: "m", Prompt: 1, OK: true},
|
||||
Req{Time: now.UnixMilli(), Model: "m", Prompt: 1, OK: true},
|
||||
)
|
||||
s := periodStats(t, path)
|
||||
got := s.PeriodSnapshot(PeriodDay, "", now)
|
||||
if len(got.Bucket) != 3 {
|
||||
t.Fatalf("day buckets = %d, want 3 (hourly): %+v", len(got.Bucket), got.Bucket)
|
||||
}
|
||||
for i := 1; i < len(got.Bucket); i++ {
|
||||
if got.Bucket[i-1].Bucket >= got.Bucket[i].Bucket {
|
||||
t.Errorf("day buckets not chronological at %d: %q >= %q",
|
||||
i, got.Bucket[i-1].Bucket, got.Bucket[i].Bucket)
|
||||
}
|
||||
}
|
||||
// Weekly grain is daily, so the same three records collapse to one day.
|
||||
week := s.PeriodSnapshot(PeriodWeek, "", now)
|
||||
if len(week.Bucket) != 1 {
|
||||
t.Errorf("week buckets = %d, want 1 (daily grain): %+v", len(week.Bucket), week.Bucket)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodKeyFilterIsolatesKeys guards the multi-tenant boundary: asking for
|
||||
// one key must never return another key's rows in any dimension.
|
||||
func TestPeriodKeyFilterIsolatesKeys(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "audit.jsonl")
|
||||
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
|
||||
auditPeriod(t, path,
|
||||
Req{Time: now.UnixMilli(), Key: "alice", Model: "m", Source: "s", Prompt: 10, OK: true},
|
||||
Req{Time: now.UnixMilli(), Key: "bob", Model: "m", Source: "s", Prompt: 999, OK: true},
|
||||
)
|
||||
s := periodStats(t, path)
|
||||
got := s.PeriodSnapshot(PeriodDay, "alice", now)
|
||||
if got.Total.Reqs != 1 || got.Total.Prompt != 10 {
|
||||
t.Errorf("alice's window = %d reqs / %d prompt, want 1 / 10 — another key leaked in",
|
||||
got.Total.Reqs, got.Total.Prompt)
|
||||
}
|
||||
if len(got.Srcs) != 1 || got.Srcs[0].Prompt != 10 {
|
||||
t.Errorf("by_source leaked: %+v", got.Srcs)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodAllUsesLifetimeAggregates checks "all" still answers from the
|
||||
// in-memory aggregates (no audit file needed) and carries the same rows the
|
||||
// dashboard already renders.
|
||||
func TestPeriodAllUsesLifetimeAggregates(t *testing.T) {
|
||||
s := NewStats(100)
|
||||
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
|
||||
s.Record(Req{Time: now.AddDate(0, 0, -40).UnixMilli(), Key: "k", Model: "m", Source: "s", Prompt: 7, OK: true})
|
||||
s.Record(Req{Time: now.UnixMilli(), Key: "k", Model: "m", Source: "s", Prompt: 3, OK: true})
|
||||
|
||||
got := s.PeriodSnapshot(PeriodAll, "", now)
|
||||
if got.Total.Reqs != 2 || got.Total.Prompt != 10 {
|
||||
t.Errorf("all view = %d reqs / %d prompt, want 2 / 10 (lifetime)", got.Total.Reqs, got.Total.Prompt)
|
||||
}
|
||||
if len(got.Models) != 1 || got.Models[0].Prompt != 10 {
|
||||
t.Errorf("all view by_model = %+v, want one row with 10 prompt", got.Models)
|
||||
}
|
||||
// A lifetime total has no timeline to draw; buckets must stay empty
|
||||
// rather than inventing 500 hourly points.
|
||||
if len(got.Bucket) != 0 {
|
||||
t.Errorf("all view produced %d buckets, want 0", len(got.Bucket))
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodEmptyWindowFlagsTruncated covers the "rotated away" case: a bounded
|
||||
// window with nothing to show may mean "quiet day" or "the files are gone".
|
||||
// The aggregator must not report a confidently wrong zero.
|
||||
func TestPeriodEmptyWindowFlagsTruncated(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
s := periodStats(t, filepath.Join(dir, "audit.jsonl")) // file does not exist
|
||||
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
|
||||
got := s.PeriodSnapshot(PeriodDay, "", now)
|
||||
if got.Total.Reqs != 0 {
|
||||
t.Errorf("empty window reqs = %d, want 0", got.Total.Reqs)
|
||||
}
|
||||
if !got.Truncated {
|
||||
t.Error("★ an empty bounded window must be flagged truncated — otherwise a " +
|
||||
"rotated-away week reads as 'no traffic that week'")
|
||||
}
|
||||
}
|
||||
|
||||
// TestValidPeriodRejectsUnknown guards the deliberate strictness: a typo must
|
||||
// be an error, not a silent fallback to lifetime totals.
|
||||
func TestValidPeriodRejectsUnknown(t *testing.T) {
|
||||
for _, ok := range []Period{PeriodDay, PeriodWeek, PeriodMonth, PeriodAll} {
|
||||
if !ValidPeriod(ok) {
|
||||
t.Errorf("ValidPeriod(%q) = false, want true", ok)
|
||||
}
|
||||
}
|
||||
for _, bad := range []Period{"", "year", "hour", "today", "DAY "} {
|
||||
if ValidPeriod(bad) {
|
||||
t.Errorf("ValidPeriod(%q) = true, want false", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodStatusDimensionCountsFailures covers by_status, which nothing else
|
||||
// exercised — an aggregation branch with no test is exactly how a period view
|
||||
// silently loses the failure pie. M8 (disabling the status dimension) is
|
||||
// caught only because this test exists.
|
||||
func TestPeriodStatusDimensionCountsFailures(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "audit.jsonl")
|
||||
now := time.Date(2026, 3, 11, 10, 0, 0, 0, time.UTC)
|
||||
auditPeriod(t, path,
|
||||
Req{Time: now.UnixMilli(), Model: "m", Source: "s", Status: 200, OK: true, Prompt: 5},
|
||||
Req{Time: now.UnixMilli(), Model: "m", Source: "s", Status: 502, OK: false, Prompt: 1},
|
||||
Req{Time: now.UnixMilli(), Model: "m", Source: "s", Status: 502, OK: false, Prompt: 1},
|
||||
Req{Time: now.UnixMilli(), Model: "m", Source: "s", Status: 429, OK: false, Prompt: 1},
|
||||
)
|
||||
s := periodStats(t, path)
|
||||
got := s.PeriodSnapshot(PeriodDay, "", now)
|
||||
|
||||
byCode := map[string]int64{}
|
||||
for _, r := range got.Status {
|
||||
byCode[r.Name] = r.Reqs
|
||||
}
|
||||
if len(got.Status) != 3 {
|
||||
t.Fatalf("by_status has %d rows, want 3 (200/502/429): %+v", len(got.Status), got.Status)
|
||||
}
|
||||
if byCode["200"] != 1 || byCode["502"] != 2 || byCode["429"] != 1 {
|
||||
t.Errorf("by_status counts = %v, want 200:1 502:2 429:1", byCode)
|
||||
}
|
||||
// Status codes must sort numerically: "429" before "502" lexically would
|
||||
// render the failure legend in the wrong order.
|
||||
if got.Status[0].Name != "200" || got.Status[1].Name != "429" || got.Status[2].Name != "502" {
|
||||
t.Errorf("by_status not numerically sorted: %s, %s, %s",
|
||||
got.Status[0].Name, got.Status[1].Name, got.Status[2].Name)
|
||||
}
|
||||
// The failure rows must agree with the OK/Err split on the total, or the
|
||||
// pie and the KPI row contradict each other.
|
||||
if got.Total.OK != 1 || got.Total.Err != 3 {
|
||||
t.Errorf("total ok/err = %d/%d, want 1/3", got.Total.OK, got.Total.Err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPeriodFromQuery(t *testing.T) {
|
||||
cases := []struct {
|
||||
query string
|
||||
want Period
|
||||
}{
|
||||
{"", PeriodAll}, // no selector → unchanged legacy behaviour
|
||||
{"?period=day", PeriodDay},
|
||||
{"?period=WEEK", PeriodWeek},
|
||||
{"?period=month", PeriodMonth},
|
||||
{"?period=all", PeriodAll},
|
||||
}
|
||||
for _, c := range cases {
|
||||
r := httptest.NewRequest(http.MethodGet, "/api/stats"+c.query, nil)
|
||||
if got := periodFromQuery(r.URL.Query()); got != c.want {
|
||||
t.Errorf("periodFromQuery(%q) = %q, want %q", c.query, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeriodWeekMayStartBeforeMonth pins a fact that looks like a bug and
|
||||
// gets reported as one: the calendar periods are NOT nested.
|
||||
//
|
||||
// On 2026-10-03 (a Saturday) the ISO week runs Mon 2026-09-28 .. Sat
|
||||
// 10-03, while the month window is 10-01 .. 10-03. The week therefore reaches
|
||||
// back into the previous month and its total is legitimately LARGER than the
|
||||
// month's. Anyone reading only the totals will conclude the aggregator is
|
||||
// broken; it is not, and this test exists so a future "fix" that makes the
|
||||
// windows nested (e.g. clamping the week to the month) is caught as the
|
||||
// regression it would be.
|
||||
func TestPeriodWeekMayStartBeforeMonth(t *testing.T) {
|
||||
// Saturday 2026-10-03, UTC.
|
||||
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
|
||||
|
||||
weekFrom := periodStart(PeriodWeek, now)
|
||||
monthFrom := periodStart(PeriodMonth, now)
|
||||
|
||||
weekFromD := time.UnixMilli(weekFrom).UTC()
|
||||
monthFromD := time.UnixMilli(monthFrom).UTC()
|
||||
|
||||
if got := weekFromD.Format("2006-01-02"); got != "2026-09-28" {
|
||||
t.Fatalf("week window start = %s, want 2026-09-28 (the Monday)", got)
|
||||
}
|
||||
if got := monthFromD.Format("2006-01-02"); got != "2026-10-01" {
|
||||
t.Fatalf("month window start = %s, want 2026-10-01", got)
|
||||
}
|
||||
if !weekFromD.Before(monthFromD) {
|
||||
t.Fatal("this test only means anything when the week starts before the month")
|
||||
}
|
||||
// The week must reach strictly further back than the month window does,
|
||||
// otherwise the nesting intuition would hold and the UI note would be noise.
|
||||
if back := monthFromD.Sub(weekFromD).Hours() / 24; back <= 0 {
|
||||
t.Fatalf("week should start at least a day before the month window, got %v days back", back)
|
||||
}
|
||||
}
|
||||
|
||||
// The same overlap must be visible in the aggregate numbers, not just in the
|
||||
// window bounds: a week total larger than a month total is the exact symptom
|
||||
// an operator reported.
|
||||
func TestPeriodWeekTotalCanExceedMonthTotal(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
audit := filepath.Join(dir, "audit.jsonl")
|
||||
|
||||
// Two records before the month window (inside this ISO week) and one
|
||||
// inside October.
|
||||
pre := time.Date(2026, 9, 29, 10, 0, 0, 0, time.UTC).UnixMilli()
|
||||
in1 := time.Date(2026, 10, 1, 10, 0, 0, 0, time.UTC).UnixMilli()
|
||||
in2 := time.Date(2026, 10, 3, 10, 0, 0, 0, time.UTC).UnixMilli()
|
||||
|
||||
auditPeriod(t, audit,
|
||||
Req{Time: pre, OK: true, Prompt: 100, Compl: 10},
|
||||
Req{Time: pre + 1000, OK: true, Prompt: 100, Compl: 10},
|
||||
Req{Time: in1, OK: true, Prompt: 50, Compl: 5},
|
||||
Req{Time: in2, OK: true, Prompt: 50, Compl: 5},
|
||||
)
|
||||
|
||||
s := periodStats(t, audit)
|
||||
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
|
||||
|
||||
week := s.PeriodSnapshot(PeriodWeek, "", now)
|
||||
month := s.PeriodSnapshot(PeriodMonth, "", now)
|
||||
|
||||
if week.Total.Reqs != 4 {
|
||||
t.Fatalf("week reqs = %d, want 4 (both September records fall in this ISO week)", week.Total.Reqs)
|
||||
}
|
||||
if month.Total.Reqs != 2 {
|
||||
t.Fatalf("month reqs = %d, want 2", month.Total.Reqs)
|
||||
}
|
||||
if week.Total.Tokens <= month.Total.Tokens {
|
||||
t.Fatalf("week tokens (%d) must exceed month tokens (%d): the week reaches into September",
|
||||
week.Total.Tokens, month.Total.Tokens)
|
||||
}
|
||||
// Day must be a strict subset of both, which is the only nesting that holds.
|
||||
day := s.PeriodSnapshot(PeriodDay, "", now)
|
||||
if day.Total.Reqs != 1 {
|
||||
t.Fatalf("day reqs = %d, want 1", day.Total.Reqs)
|
||||
}
|
||||
if day.Total.Tokens >= month.Total.Tokens || day.Total.Tokens >= week.Total.Tokens {
|
||||
t.Fatal("today must be contained in both the week and the month totals")
|
||||
}
|
||||
}
|
||||
@ -1,231 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
)
|
||||
|
||||
// Token accounting must record TOKENS, not byte estimates and not image
|
||||
// counts. These tests pin the two units bugs found on 2026-09-27:
|
||||
//
|
||||
// 1. streamed chat: rec.Compl was only ever the per-chunk byte estimate
|
||||
// (len(content)/3), even when the upstream sent exact usage on its final
|
||||
// frame. Measured on production traffic: the same model reported a
|
||||
// streamed completion median 4-27x its non-streamed median. Upstream 100
|
||||
// recorded as 145.
|
||||
// 2. image generation: rec.Compl was set to len(resp.ImageData) — the image
|
||||
// COUNT — and folded into the token totals.
|
||||
//
|
||||
// A fake upstream that reports FIXED numbers makes both mismatches
|
||||
// unambiguous: any difference between what it reports and what the audit
|
||||
// record holds is the gateway's doing.
|
||||
|
||||
// fixedUsageUpstream reports prompt=prompt completion=compl on both the
|
||||
// streaming and the non-streaming path. The streaming path emits usage on its
|
||||
// final frame before [DONE], exactly like OpenAI-compatible upstreams.
|
||||
func fixedUsageUpstream(t *testing.T, prompt, compl int, images int) *httptest.Server {
|
||||
t.Helper()
|
||||
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
var body map[string]interface{}
|
||||
_ = json.NewDecoder(r.Body).Decode(&body)
|
||||
stream, _ := body["stream"].(bool)
|
||||
usage := fmt.Sprintf(`{"prompt_tokens":%d,"completion_tokens":%d,"total_tokens":%d}`,
|
||||
prompt, compl, prompt+compl)
|
||||
|
||||
// Image generation wire: /v1/images/generations style body.
|
||||
if strings.Contains(r.URL.Path, "image") {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
var data []string
|
||||
for i := 0; i < images; i++ {
|
||||
data = append(data, `{"b64_json":"AAAA"}`)
|
||||
}
|
||||
fmt.Fprintf(w, `{"created":1,"data":[%s]}`, strings.Join(data, ","))
|
||||
return
|
||||
}
|
||||
|
||||
if !stream {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
fmt.Fprintf(w, `{"model":"m1","choices":[{"message":{"content":"pong pong pong"},
|
||||
"finish_reason":"stop"}],"usage":%s}`, usage)
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "text/event-stream")
|
||||
w.WriteHeader(200)
|
||||
fl, _ := w.(http.Flusher)
|
||||
send := func(s string) {
|
||||
fmt.Fprintf(w, "data: %s\n\n", s)
|
||||
if fl != nil {
|
||||
fl.Flush()
|
||||
}
|
||||
}
|
||||
// Deliberately verbose payloads: a byte-estimating path records a
|
||||
// number far from `compl`, so the test would catch it.
|
||||
send(`{"model":"m1","choices":[{"delta":{"reasoning_content":"` + strings.Repeat("think ", 40) + `"}}]}`)
|
||||
send(`{"model":"m1","choices":[{"delta":{"content":"` + strings.Repeat("answer ", 40) + `"}}]}`)
|
||||
send(`{"model":"m1","choices":[{"delta":{},"finish_reason":"stop"}]}`)
|
||||
send(`{"model":"m1","choices":[],"usage":` + usage + `}`)
|
||||
send(`[DONE]`)
|
||||
}))
|
||||
}
|
||||
|
||||
// lastChatRecord returns the most recent audit record whose Type is one of the
|
||||
// given types.
|
||||
func lastChatRecord(t *testing.T, g *Gateway, types ...string) Req {
|
||||
t.Helper()
|
||||
recs := g.stats.AuditRecords(0, 0, "")
|
||||
want := map[string]bool{}
|
||||
for _, ty := range types {
|
||||
want[ty] = true
|
||||
}
|
||||
for i := len(recs) - 1; i >= 0; i-- {
|
||||
if want[recs[i].Type] {
|
||||
return recs[i]
|
||||
}
|
||||
}
|
||||
t.Fatalf("no audit record of type %v found (have %d records)", types, len(recs))
|
||||
return Req{}
|
||||
}
|
||||
|
||||
// The streamed path must record the upstream's exact usage, not a byte
|
||||
// estimate. Before the fix this recorded 145 for an upstream-reported 100.
|
||||
func TestStreamedChatRecordsUpstreamUsage(t *testing.T) {
|
||||
const wantPrompt, wantCompl = 44, 100
|
||||
up := fixedUsageUpstream(t, wantPrompt, wantCompl, 0)
|
||||
defer up.Close()
|
||||
g := newTestGateway(t, config.Source{
|
||||
Name: "up", BaseURL: up.URL, Adapter: "openai",
|
||||
Models: []config.Model{{ID: "m1", Priority: 100}},
|
||||
})
|
||||
|
||||
rr := doReq(t, g, "POST", "/v1/chat/completions",
|
||||
`{"model":"m1","messages":[{"role":"user","content":"hi"}],"stream":true}`)
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("stream status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
|
||||
rec := lastChatRecord(t, g, "stream")
|
||||
if rec.Prompt != wantPrompt || rec.Compl != wantCompl {
|
||||
t.Errorf("streamed record prompt=%d completion=%d, want %d/%d — the audit "+
|
||||
"must carry the upstream's tokens, not a byte estimate",
|
||||
rec.Prompt, rec.Compl, wantPrompt, wantCompl)
|
||||
}
|
||||
}
|
||||
|
||||
// Streaming and non-streaming must agree on the same request's tokens. This is
|
||||
// the cross-path consistency check: the two paths previously recorded 145 and
|
||||
// 100 for one identical upstream usage value.
|
||||
func TestStreamedAndNonStreamedAgreeOnTokens(t *testing.T) {
|
||||
const wantPrompt, wantCompl = 44, 100
|
||||
up := fixedUsageUpstream(t, wantPrompt, wantCompl, 0)
|
||||
defer up.Close()
|
||||
g := newTestGateway(t, config.Source{
|
||||
Name: "up", BaseURL: up.URL, Adapter: "openai",
|
||||
Models: []config.Model{{ID: "m1", Priority: 100}},
|
||||
})
|
||||
|
||||
if rr := doReq(t, g, "POST", "/v1/chat/completions",
|
||||
`{"model":"m1","messages":[{"role":"user","content":"hi"}]}`); rr.Code != 200 {
|
||||
t.Fatalf("non-stream status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if rr := doReq(t, g, "POST", "/v1/chat/completions",
|
||||
`{"model":"m1","messages":[{"role":"user","content":"hi"}],"stream":true}`); rr.Code != 200 {
|
||||
t.Fatalf("stream status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
|
||||
chat, stream := lastChatRecord(t, g, "chat"), lastChatRecord(t, g, "stream")
|
||||
if chat.Prompt != stream.Prompt || chat.Compl != stream.Compl {
|
||||
t.Errorf("chat=%d/%d stream=%d/%d — identical upstream usage must record "+
|
||||
"identically on both paths", chat.Prompt, chat.Compl, stream.Prompt, stream.Compl)
|
||||
}
|
||||
}
|
||||
|
||||
// A fallback to the byte estimate is still correct WHEN the upstream reports no
|
||||
// usage at all. Without this the fix could silently zero out every streamed
|
||||
// record from upstreams that omit usage.
|
||||
func TestStreamedChatFallsBackToEstimateWithoutUsage(t *testing.T) {
|
||||
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "text/event-stream")
|
||||
w.WriteHeader(200)
|
||||
fl, _ := w.(http.Flusher)
|
||||
send := func(s string) {
|
||||
fmt.Fprintf(w, "data: %s\n\n", s)
|
||||
if fl != nil {
|
||||
fl.Flush()
|
||||
}
|
||||
}
|
||||
send(`{"model":"m1","choices":[{"delta":{"content":"` + strings.Repeat("abc ", 60) + `"}}]}`)
|
||||
send(`{"model":"m1","choices":[{"delta":{},"finish_reason":"stop"}]}`)
|
||||
send(`[DONE]`)
|
||||
}))
|
||||
defer up.Close()
|
||||
g := newTestGateway(t, config.Source{
|
||||
Name: "up", BaseURL: up.URL, Adapter: "openai",
|
||||
Models: []config.Model{{ID: "m1", Priority: 100}},
|
||||
})
|
||||
|
||||
if rr := doReq(t, g, "POST", "/v1/chat/completions",
|
||||
`{"model":"m1","messages":[{"role":"user","content":"hi"}],"stream":true}`); rr.Code != 200 {
|
||||
t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
rec := lastChatRecord(t, g, "stream")
|
||||
if rec.Compl <= 0 {
|
||||
t.Errorf("completion=%d, want a positive byte-estimate fallback when the "+
|
||||
"upstream reports no usage", rec.Compl)
|
||||
}
|
||||
}
|
||||
|
||||
// Image generation has no token concept: recording the image COUNT as
|
||||
// completion tokens corrupted the token totals. The count is reported
|
||||
// separately and must not leak into Prompt/Compl.
|
||||
func TestImageGenerationDoesNotRecordTokens(t *testing.T) {
|
||||
up := fixedUsageUpstream(t, 44, 100, 3)
|
||||
defer up.Close()
|
||||
src := config.Source{
|
||||
Name: "up", BaseURL: up.URL, Adapter: "openai",
|
||||
Models: []config.Model{{ID: "img-m", Priority: 100, Kind: "image"}},
|
||||
}
|
||||
g := newTestGateway(t, src)
|
||||
|
||||
rr := doReq(t, g, "POST", "/v1/images/generations",
|
||||
`{"model":"img-m","prompt":"a cat","n":3}`)
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("image status=%d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
|
||||
rec := lastChatRecord(t, g, "image")
|
||||
if rec.Compl != 0 || rec.Prompt != 0 {
|
||||
t.Errorf("image record prompt=%d completion=%d, want 0/0 — image count is "+
|
||||
"not a token and must not enter the token totals", rec.Prompt, rec.Compl)
|
||||
}
|
||||
if rec.ImageCount != 3 {
|
||||
t.Errorf("image_count=%d, want 3 (the count belongs in its own field)", rec.ImageCount)
|
||||
}
|
||||
// And the aggregate must see zero tokens for it.
|
||||
if got := g.stats.WindowTokens("img-m", "", 3600); got != 0 {
|
||||
t.Errorf("window tokens for an image model = %d, want 0", got)
|
||||
}
|
||||
}
|
||||
|
||||
// The audit JSON must carry image_count so the WebUI can show images rather
|
||||
// than a bogus token figure.
|
||||
func TestImageCountSurvivesAuditJSON(t *testing.T) {
|
||||
b, err := json.Marshal(Req{Type: "image", ImageCount: 2, OK: true})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(b), `"image_count":2`) {
|
||||
t.Errorf("audit JSON %s lacks image_count", b)
|
||||
}
|
||||
// Omitempty keeps chat records unchanged.
|
||||
b2, _ := json.Marshal(Req{Type: "chat", Prompt: 1, Compl: 2, OK: true})
|
||||
if strings.Contains(string(b2), "image_count") {
|
||||
t.Errorf("chat record %s should omit image_count", b2)
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@ -59,82 +59,6 @@ func lineOf(src string, idx int) int {
|
||||
return strings.Count(src[:idx], "\n") + 1
|
||||
}
|
||||
|
||||
// TestUIEditFormRoundTripsEveryOptionalSourceField pins the JS half of the
|
||||
// source-edit contract.
|
||||
//
|
||||
// The server distinguishes "absent" from "explicitly empty" for proxy_url,
|
||||
// api_key_env and timeout (they are pointers). That only helps if the form
|
||||
// SENTS them — a form that omits them falls back to "inherit", so clearing the
|
||||
// proxy box would silently keep the old proxy, which is the exact bug in the
|
||||
// other direction.
|
||||
//
|
||||
// It must also send the api_key_env value it was given, since that is the one
|
||||
// field whose loss is invisible until the next upstream call.
|
||||
func TestUIEditFormRoundTripsEveryOptionalSourceField(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
body, ok := jsFunctionBody(src, "saveSource")
|
||||
if !ok {
|
||||
t.Fatal("saveSource() not found in the WebUI")
|
||||
}
|
||||
for _, field := range []string{"proxy_url", "api_key_env", "timeout"} {
|
||||
if !strings.Contains(body, field+":") {
|
||||
t.Errorf("saveSource() does not send %q; the server treats an absent "+
|
||||
"field as \"keep the stored value\", so the form could never clear it", field)
|
||||
}
|
||||
}
|
||||
// The form's inputs must be filled from the values the server sends, or a
|
||||
// save would post an empty box and clear a configured field.
|
||||
for _, read := range []string{`$("#s-proxy")`, `$("#s-keyenv")`, `$("#s-timeout")`} {
|
||||
if !strings.Contains(src, read+".value") {
|
||||
t.Errorf("the source form never reads %s — it would post an empty value", read)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestUIEditFormReadsDurationsFromTheRevealView: config.Source tags
|
||||
// Timeout/QueueTimeout `json:"-"`, so the durations only reach the form if the
|
||||
// reveal endpoint adds them explicitly. The form MUST read them (see above),
|
||||
// which makes this API response load-bearing rather than informational.
|
||||
func TestUIEditFormReadsDurationsFromTheRevealView(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
for _, id := range []string{"s-proxy", "s-keyenv", "s-timeout"} {
|
||||
if !strings.Contains(src, `id="`+id+`"`) {
|
||||
t.Errorf("source form input #%s is missing", id)
|
||||
}
|
||||
}
|
||||
// The timeout box must render through the duration helper. Reading a raw
|
||||
// time.Duration number would show nanoseconds in a box that expects "300s".
|
||||
if !strings.Contains(src, "timeoutText(") {
|
||||
t.Error("the timeout input does not go through timeoutText(); a raw " +
|
||||
"time.Duration JSON number is nanoseconds and would be uneditable")
|
||||
}
|
||||
}
|
||||
|
||||
// TestUIEditFormSendsEverySourceField is the mirror of
|
||||
// TestSourcePayloadCoversEveryEditableField on the JavaScript side: each field
|
||||
// the server accepts must be submitted by the form, or the server's upsert has
|
||||
// nothing to store for it.
|
||||
func TestUIEditFormSendsEverySourceField(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
body, ok := jsFunctionBody(src, "saveSource")
|
||||
if !ok {
|
||||
t.Fatal("saveSource() not found")
|
||||
}
|
||||
// Fields the edit dialog owns. proxy_url / api_key_env / timeout are
|
||||
// covered by the round-trip test above; this one catches the rest.
|
||||
// models and meta are local variables submitted by Go shorthand
|
||||
// (payload = { models, meta }), so they are matched as bare identifiers.
|
||||
for _, field := range []string{
|
||||
"name:", "base_url:", "api_key:", "adapter:", "endpoint:",
|
||||
"image_endpoint:", "max_concurrent:", "rpm:", "temperature:",
|
||||
"models,", "meta,",
|
||||
} {
|
||||
if !strings.Contains(body, field) {
|
||||
t.Errorf("saveSource() does not send %s", field)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestUIAPICallsDeclareMethod asserts that every api() call passing an options
|
||||
// object also declares an HTTP method (or is a GET that only passes an
|
||||
// AbortSignal). Without this, fetch defaults to GET and mutating endpoints are
|
||||
@ -423,60 +347,3 @@ func cssBlock(src, selector string) (string, bool) {
|
||||
}
|
||||
return src[i : i+j+1], true
|
||||
}
|
||||
|
||||
// The WebUI ships two locale objects (`zh` and `en`) that every rendered string
|
||||
// goes through. A key added to only one of them does not error: t() falls back
|
||||
// to `t("key") || "literal"` in some call sites and to the bare key string in
|
||||
// others, so the user sees either a hardcoded local string or a raw key name.
|
||||
// Nothing in the Go test suite noticed — this was found by hand while adding
|
||||
// thImgs. Pin the key sets so a one-sided edit fails here instead of shipping.
|
||||
func TestUILocaleKeyParity(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
iZh := strings.Index(src, "zh: {")
|
||||
iEn := strings.Index(src, "en: {")
|
||||
if iZh < 0 || iEn < 0 || iEn < iZh {
|
||||
t.Fatalf("locale blocks not found (zh=%d en=%d)", iZh, iEn)
|
||||
}
|
||||
zh := src[iZh:iEn]
|
||||
en := src[iEn:]
|
||||
// End the EN block at its closing brace, so keys from later objects
|
||||
// (config templates etc.) do not pollute the comparison.
|
||||
if end := strings.Index(en, "\n },"); end > 0 {
|
||||
en = en[:end]
|
||||
}
|
||||
|
||||
keyRe := regexp.MustCompile(`(?m)^\s*([A-Za-z_][A-Za-z0-9_]*)\s*:`)
|
||||
collect := func(seg string) map[string]bool {
|
||||
m := map[string]bool{}
|
||||
for _, g := range keyRe.FindAllStringSubmatch(seg, -1) {
|
||||
m[g[1]] = true
|
||||
}
|
||||
return m
|
||||
}
|
||||
zk, ek := collect(zh), collect(en)
|
||||
// The locale markers themselves are the two block headers.
|
||||
delete(zk, "zh")
|
||||
delete(ek, "en")
|
||||
|
||||
var missingEN, missingZH []string
|
||||
for k := range zk {
|
||||
if !ek[k] {
|
||||
missingEN = append(missingEN, k)
|
||||
}
|
||||
}
|
||||
for k := range ek {
|
||||
if !zk[k] {
|
||||
missingZH = append(missingZH, k)
|
||||
}
|
||||
}
|
||||
if len(missingEN) > 0 {
|
||||
t.Errorf("translation keys present in zh but missing in en: %v", missingEN)
|
||||
}
|
||||
if len(missingZH) > 0 {
|
||||
t.Errorf("translation keys present in en but missing in zh: %v", missingZH)
|
||||
}
|
||||
if len(zk) < 100 {
|
||||
t.Errorf("only %d zh keys parsed — the block boundary regexp has drifted "+
|
||||
"and this test is no longer checking anything", len(zk))
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,331 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Per-key AUTO chain UI contract.
|
||||
//
|
||||
// The per-key editor deliberately REUSES the global lane canvas instead of
|
||||
// shipping a second, simpler editor. That is a design choice with teeth: a
|
||||
// reduced copy would lack drag ordering and per-slot quota/period/hours, and
|
||||
// the two would drift apart silently. These checks pin the sharing, because
|
||||
// the only symptom of un-sharing is "the per-key editor got worse" — no error,
|
||||
// just a smaller feature set that nobody notices until they use it.
|
||||
func TestUIPerKeyAutoWiring(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
|
||||
// The admin key list must offer the editor, otherwise the whole feature is
|
||||
// unreachable from the UI. Assert the *call site* inside keyCanvasHtml,
|
||||
// not just that the functions exist: removing the button leaves every
|
||||
// definition intact, and that is exactly the regression to catch.
|
||||
kcStart := strings.Index(src, "function keyCanvasHtml")
|
||||
if kcStart < 0 {
|
||||
t.Fatal("keyCanvasHtml is gone — cannot check the AUTO chain button")
|
||||
}
|
||||
kcEnd := strings.Index(src[kcStart:], "\n }")
|
||||
if kcEnd < 0 {
|
||||
t.Fatal("could not delimit keyCanvasHtml")
|
||||
}
|
||||
if !strings.Contains(src[kcStart:kcStart+kcEnd], "openKeyAutoModal(") {
|
||||
t.Fatal("per-key AUTO chain button is missing from the key card header")
|
||||
}
|
||||
for _, fn := range []string{
|
||||
"function openKeyAutoModal",
|
||||
"function keyAutoInherit",
|
||||
"function keyAutoSave",
|
||||
"function keyAutoClose",
|
||||
} {
|
||||
if !strings.Contains(src, fn) {
|
||||
t.Fatalf("UI is missing %s — the editor cannot function", fn)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The per-key editor must render the SAME canvas the global editor uses.
|
||||
// If this regresses, the per-key chain silently loses drag ordering and
|
||||
// per-slot quota/period/hours while still looking like it works.
|
||||
func TestUIPerKeyAutoReusesGlobalCanvas(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
|
||||
// One canvas implementation, two mount ids.
|
||||
if !strings.Contains(src, "function renderSortEditor(") {
|
||||
t.Fatal("no shared canvas renderer — the per-key editor must reuse renderSortEditor")
|
||||
}
|
||||
if !strings.Contains(src, `canvasId: "scr-canvas"`) {
|
||||
t.Fatal("the global editor no longer mounts the shared canvas")
|
||||
}
|
||||
if !strings.Contains(src, `canvasId: "key-auto-canvas"`) {
|
||||
t.Fatal("the per-key editor does not mount the shared canvas")
|
||||
}
|
||||
// Canvas lookup must consider both ids, or drag/drop silently no-ops on
|
||||
// whichever surface is not #scr-canvas.
|
||||
if !strings.Contains(src, "function sortCanvasEl(") ||
|
||||
!strings.Contains(src, `getElementById("key-auto-canvas")`) {
|
||||
t.Fatal("sortCanvasEl must resolve both canvas ids")
|
||||
}
|
||||
// The rules<->lanes conversion is shared too, so the per-key editor sends
|
||||
// exactly the fields the server understands (quota/period/hours included).
|
||||
for _, fn := range []string{"function buildLanes(", "function lanesToRules("} {
|
||||
if !strings.Contains(src, fn) {
|
||||
t.Fatalf("%s must be shared between the global and per-key editors", fn)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Saving must route through persistAuto's scope switch, not a hand-rolled
|
||||
// request, or the two editors will serialize rules differently.
|
||||
func TestUIPerKeyAutoSavesThroughScope(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
if !strings.Contains(src, "sortState.scope = key") {
|
||||
t.Fatal("openKeyAutoModal must set sortState.scope so the save targets the key")
|
||||
}
|
||||
if !strings.Contains(src, "sortState.scope = null") {
|
||||
t.Fatal("closing the modal must clear sortState.scope, or the GLOBAL editor would save to this key")
|
||||
}
|
||||
if !strings.Contains(src, "if (sortState.scope)") {
|
||||
t.Fatal("persistAuto must branch on sortState.scope")
|
||||
}
|
||||
if !strings.Contains(src, `"/auto"`) {
|
||||
t.Fatal("per-key editor must call the /api/keys/{key}/auto endpoint")
|
||||
}
|
||||
// An empty lane list is how "inherit the global chain" is expressed.
|
||||
if !strings.Contains(src, "JSON.stringify({ auto: lanesToRules(sortState.lanes) })") {
|
||||
t.Fatal("keyAutoSave must PUT lanesToRules(sortState.lanes) verbatim")
|
||||
}
|
||||
}
|
||||
|
||||
// Both languages need the strings, otherwise a non-Chinese admin sees raw
|
||||
// i18n keys in the modal.
|
||||
func TestUIPerKeyAutoHasBothLanguages(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
for _, k := range []string{
|
||||
"kAutoChainTitle", "kAutoChainHint", "kAutoChainInherit",
|
||||
"kAutoChainEmpty", "kAutoChainGlobal", "kAutoChainBad",
|
||||
} {
|
||||
n := strings.Count(src, k+`: "`)
|
||||
if n < 2 {
|
||||
t.Fatalf("i18n key %s defined %d time(s), want both zh and en", k, n)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The statistics dashboard must print the window it actually covers. Calendar
|
||||
// periods are not nested — "this week" starts on its Monday and therefore
|
||||
// reaches into the previous month early in a month, making its total LARGER
|
||||
// than "this month". That is correct arithmetic, but without the dates on
|
||||
// screen it reads as a bug, and it has been reported as one.
|
||||
func TestUIStatsShowsPeriodWindow(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
if !strings.Contains(src, `id="period-range"`) {
|
||||
t.Fatal("the stats dashboard has no element for the period window range")
|
||||
}
|
||||
if n := strings.Count(src, "periodRange"); n < 2 {
|
||||
t.Fatalf("i18n key periodRange appears %d time(s), want both zh and en", n)
|
||||
}
|
||||
}
|
||||
// Both canvases can exist in the DOM at once: the modal is mounted on top of
|
||||
// the settings tab, whose #scr-canvas stays alive behind it. Resolving the
|
||||
// canvas by document order (whichever comes first) painted the per-key chain
|
||||
// into the hidden background tab and left the dialog empty — and worse,
|
||||
// overwrote the global chain the user returns to. The lookup must follow
|
||||
// sortState.scope instead.
|
||||
func TestUISortCanvasResolvesByScope(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
el := readFuncBody(t, src, "sortCanvasEl")
|
||||
if !strings.Contains(el, "if (sortState.scope)") {
|
||||
t.Fatal("sortCanvasEl must branch on sortState.scope, not document order")
|
||||
}
|
||||
if strings.Contains(el, `getElementById("scr-canvas")`) &&
|
||||
strings.Index(el, `getElementById("scr-canvas")`) < strings.Index(el, "sortState.scope") {
|
||||
t.Fatal("sortCanvasEl returns #scr-canvas before checking scope — the " +
|
||||
"per-key editor paints into the global canvas")
|
||||
}
|
||||
}
|
||||
|
||||
// A successful save must close the dialog even when the slots do not resolve.
|
||||
// The write DID succeed; leaving the modal open makes the user click Save
|
||||
// again and again. This regressed once: the !slots branch returned early.
|
||||
func TestUIKeyAutoSaveClosesOnSuccess(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
body := readFuncBody(t, src, "keyAutoSave")
|
||||
// keyAutoClose() must be reached on every success path, not only in the
|
||||
// happy branch.
|
||||
iClose := strings.Index(body, "keyAutoClose()")
|
||||
if iClose < 0 {
|
||||
t.Fatal("keyAutoSave never calls keyAutoClose — the dialog would stay open")
|
||||
}
|
||||
iToast := strings.Index(body, "toast(")
|
||||
if iToast >= 0 && iClose < iToast {
|
||||
t.Fatal("keyAutoClose is called before the toast; the warning must be " +
|
||||
"shown first or it is lost with the modal")
|
||||
}
|
||||
// An early `return` inside a branch bypasses keyAutoClose entirely, so
|
||||
// merely finding the call is not enough: every `return` that sits between
|
||||
// entering the try block and the close call is a path that leaves the
|
||||
// dialog open after a successful write.
|
||||
tryStart := strings.Index(body, "try {")
|
||||
seg := body[tryStart:iClose]
|
||||
for _, line := range strings.Split(seg, "\n") {
|
||||
trimmed := strings.TrimSpace(line)
|
||||
if trimmed == "return;" || strings.HasPrefix(trimmed, "return ") {
|
||||
t.Fatal("a `return` before keyAutoClose leaves the dialog open after " +
|
||||
"a successful save: " + trimmed)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// readFuncBody returns the source of a top-level function by name, using
|
||||
// brace matching so nested blocks do not truncate it.
|
||||
func readFuncBody(t *testing.T, src, name string) string {
|
||||
t.Helper()
|
||||
idx := strings.Index(src, "function "+name+"(")
|
||||
if idx < 0 {
|
||||
t.Fatalf("function %s not found", name)
|
||||
}
|
||||
start := strings.Index(src[idx:], "{")
|
||||
if start < 0 {
|
||||
t.Fatalf("function %s has no body", name)
|
||||
}
|
||||
i := idx + start
|
||||
depth := 0
|
||||
for j := i; j < len(src); j++ {
|
||||
switch src[j] {
|
||||
case '{':
|
||||
depth++
|
||||
case '}':
|
||||
depth--
|
||||
if depth == 0 {
|
||||
return src[i : j+1]
|
||||
}
|
||||
}
|
||||
}
|
||||
t.Fatalf("function %s body is unterminated", name)
|
||||
return ""
|
||||
}
|
||||
|
||||
// The per-key dialog must not share the app-wide #modal-wrap id.
|
||||
//
|
||||
// Every dialog in this file uses that id, and more than one can be open at
|
||||
// once (the "add slot" picker opens ON TOP of this one). Handlers resolve
|
||||
// their own dialog via `.closest()` from their button, which works — but this
|
||||
// dialog is also looked up from the CANVAS side (sortCanvasEl, keyAutoClose,
|
||||
// keyAutoInherit), where there is no button to walk up from.
|
||||
// getElementById("#modal-wrap") then returns whichever dialog comes first in
|
||||
// the document, i.e. not the one being edited: Cancel closed the wrong box and
|
||||
// the canvas could resolve to the picker's.
|
||||
func TestUIKeyAutoModalHasOwnID(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
body := readFuncBody(t, src, "openKeyAutoModal")
|
||||
if !strings.Contains(body, `wrap.id = "key-auto-modal"`) {
|
||||
t.Fatal("openKeyAutoModal must give its dialog its own id, not the shared " +
|
||||
"#modal-wrap (another dialog may be open on top of it)")
|
||||
}
|
||||
close := readFuncBody(t, src, "keyAutoClose")
|
||||
if strings.Contains(close, `"modal-wrap"`) {
|
||||
t.Fatal("keyAutoClose resolves the shared #modal-wrap; it would close " +
|
||||
"whichever dialog comes first in the document")
|
||||
}
|
||||
if !strings.Contains(close, `"key-auto-modal"`) {
|
||||
t.Fatal("keyAutoClose must resolve the per-key dialog by its own id")
|
||||
}
|
||||
}
|
||||
|
||||
// Editing a chain inside the per-key dialog must NOT persist it. That dialog
|
||||
// has Save and Cancel, so auto-writing on add/drag/delete meant the change was
|
||||
// already committed before Save and Cancel could not undo it.
|
||||
//
|
||||
// The global page is the opposite: it has always applied edits immediately.
|
||||
func TestUIKeyAutoEditsDoNotAutoPersist(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
body := readFuncBody(t, src, "afterChainEdit")
|
||||
if !strings.Contains(body, "if (sortState.scope)") {
|
||||
t.Fatal("afterChainEdit must branch on sortState.scope")
|
||||
}
|
||||
// The scoped branch must return before reaching persistAuto.
|
||||
iScope := strings.Index(body, "if (sortState.scope)")
|
||||
iReturn := strings.Index(body[iScope:], "return")
|
||||
iPersist := strings.Index(body, "persistAuto()")
|
||||
if iScope < 0 || iReturn < 0 || iPersist < 0 {
|
||||
t.Fatal("afterChainEdit no longer has the expected shape")
|
||||
}
|
||||
if iScope+iReturn > iPersist {
|
||||
t.Fatal("the scoped branch reaches persistAuto() — a per-key edit would " +
|
||||
"be written before Save")
|
||||
}
|
||||
// And it must not call itself (a regex-driven rewrite did exactly that).
|
||||
if strings.Contains(body, "afterChainEdit()") {
|
||||
t.Fatal("afterChainEdit calls itself: infinite recursion")
|
||||
}
|
||||
// Every in-canvas edit goes through it, so none of them bypass the check.
|
||||
for _, fn := range []string{"scrAddFromForm", "sortScopeSave", "scrDelSlot"} {
|
||||
b := readFuncBody(t, src, fn)
|
||||
if strings.Contains(b, "persistAuto()") {
|
||||
t.Fatalf("%s calls persistAuto() directly, bypassing afterChainEdit", fn)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The stats range line formats its end date from the LAST bucket label.
|
||||
// Bucket labels are hourly for the day view ("2026-10-03T09") and daily for
|
||||
// week/month ("2026-10-01"), so blindly appending "T00:00:00Z" to the hourly
|
||||
// form builds "2026-10-03T09T00:00:00Z" — Date rejects it, toISOString()
|
||||
// throws, and the assignment is lost. The symptom is an empty range line on
|
||||
// exactly the view people look at most, with no error anywhere.
|
||||
func TestUIStatsPeriodRangeHandlesHourlyBuckets(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
el := readFuncBody(t, src, "paintStats")
|
||||
// It must branch on the label shape instead of concatenating blindly.
|
||||
if !strings.Contains(el, `T\d{2}$`) && !strings.Contains(el, `\d{2}-\d{2}T\d{2}`) {
|
||||
t.Fatal("paintStats must recognise the hourly bucket label shape " +
|
||||
"(YYYY-MM-DDTHH); concatenating a time onto it yields an invalid Date")
|
||||
}
|
||||
// And it must not contain the naive form that broke it.
|
||||
if strings.Contains(el, `bucket + "T00:00:00Z"`) {
|
||||
t.Fatal("paintStats still appends T00:00:00Z to the bucket label — " +
|
||||
"this throws on the day view and leaves the range line empty")
|
||||
}
|
||||
}
|
||||
|
||||
// A key with no chain of its own must open on a COPY OF THE GLOBAL CHAIN, not
|
||||
// on an empty canvas. The realistic admin task is "this user gets the global chain
|
||||
// minus the sources they must not hit"; starting blank forces retyping every
|
||||
// tier against a 226-model picker. This is deliberately NOT the discovery
|
||||
// fallback (which dumps every model of every source) — it is the same rules
|
||||
// the global chain actually runs.
|
||||
func TestUIKeyAutoSeedsFromGlobalChain(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
body := readFuncBody(t, src, "openKeyAutoModal")
|
||||
if !strings.Contains(body, `api("/api/auto")`) {
|
||||
t.Fatal("openKeyAutoModal must read the global chain to seed the editor")
|
||||
}
|
||||
if !strings.Contains(body, "seededFromGlobal") {
|
||||
t.Fatal("the editor must remember whether its content came from the " +
|
||||
"global chain, so the notice can say so")
|
||||
}
|
||||
// And it must not fall back to the discovery path that fills 226 models.
|
||||
if strings.Contains(body, `buildLanes(idx, [], `) {
|
||||
t.Fatal("seeding from an empty rule list triggers buildLanes discovery")
|
||||
}
|
||||
}
|
||||
|
||||
// The notice must distinguish "already this key's chain" from "a copy that
|
||||
// only becomes its chain on Save" — those two look identical on the canvas and
|
||||
// behave very differently if the admin hits Cancel.
|
||||
func TestUIKeyAutoNoticeDistinguishesSavedFromCopied(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
body := readFuncBody(t, src, "keyAutoInheritNotice")
|
||||
for _, k := range []string{"kAutoChainSeeded", "kAutoChainNewOwn", "kAutoChainWillInherit"} {
|
||||
if !strings.Contains(body, k) {
|
||||
t.Fatalf("the notice never uses %s — a copied chain and a saved one "+
|
||||
"would look the same", k)
|
||||
}
|
||||
}
|
||||
// Both languages, or a non-Chinese admin sees raw keys.
|
||||
for _, k := range []string{"kAutoChainSeeded", "kAutoChainNewOwn", "kAutoChainWillInherit"} {
|
||||
if n := strings.Count(src, k+`: "`); n < 2 {
|
||||
t.Fatalf("i18n key %s defined %d time(s), want both zh and en", k, n)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,390 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The plugin UI injection is JavaScript inside the embedded index.html, and it
|
||||
// is the ONLY thing that turns a plugin's `ui` block into a visible page or
|
||||
// element. These tests pin the wiring on the JS side; the server side (what the
|
||||
// payload contains) is covered by TestUIInjectServesPluginUI and the lua
|
||||
// package's TestBillingPluginDeclaresUI.
|
||||
//
|
||||
// What makes this worth pinning: a missing hook here fails SILENTLY. The page
|
||||
// simply never appears, there is no error anywhere, and it looks like "the
|
||||
// plugin didn't declare a page" rather than "the UI forgot to inject it".
|
||||
|
||||
// uiSource returns the embedded WebUI document.
|
||||
func uiSourceX(t *testing.T) string {
|
||||
t.Helper()
|
||||
return uiSource(t)
|
||||
}
|
||||
|
||||
// TestUIFetchesPluginInjection: the boot sequence must ask the kernel what to
|
||||
// inject. Without this fetch the whole feature is inert.
|
||||
func TestUIFetchesPluginInjection(t *testing.T) {
|
||||
src := uiSourceX(t)
|
||||
if !strings.Contains(src, "/api/ui-inject") {
|
||||
t.Error("the WebUI never calls /api/ui-inject; plugin pages and elements can never appear")
|
||||
}
|
||||
}
|
||||
|
||||
// TestUIInjectsBeforeFirstRender: injection must be awaited before the first
|
||||
// refresh, otherwise the sidebar is built without the plugin entry and the
|
||||
// first paint races the fetch. This is an ordering contract, so it is asserted
|
||||
// on the source order rather than trusted.
|
||||
func TestUIInjectsBeforeFirstRender(t *testing.T) {
|
||||
src := uiSourceX(t)
|
||||
iInject := strings.Index(src, "injectPluginUI()")
|
||||
iRefresh := strings.LastIndex(src, `refresh("status")`)
|
||||
if iInject < 0 {
|
||||
t.Fatal("injectPluginUI() is never called")
|
||||
}
|
||||
if iRefresh < 0 {
|
||||
t.Fatal("the boot sequence no longer calls refresh(\"status\")")
|
||||
}
|
||||
if iInject > iRefresh {
|
||||
t.Error("injectPluginUI() is called after the first refresh; the sidebar " +
|
||||
"and #main would be built before the plugin page exists")
|
||||
}
|
||||
// And it must be awaited, not fire-and-forget.
|
||||
window := src[iInject:]
|
||||
if !strings.Contains(window[:200], ".finally") && !strings.Contains(window[:200], "await") {
|
||||
t.Error("injectPluginUI() is not awaited before refresh; a slow response " +
|
||||
"would race the first paint")
|
||||
}
|
||||
}
|
||||
|
||||
// TestUIPluginScriptsRunAfterMarkup is the subtle one. Setting innerHTML with a
|
||||
// <script> tag does NOT execute it; appending via a template neither does. The
|
||||
// mount therefore has to be inserted first and its scripts re-created
|
||||
// afterwards, or a plugin's script runs before its own DOM exists — which is
|
||||
// exactly the "document.getElementById returns null" failure mode.
|
||||
func TestUIPluginScriptsRunAfterMarkup(t *testing.T) {
|
||||
src := uiSourceX(t)
|
||||
// A <template> is used to parse the mount without executing scripts...
|
||||
if !strings.Contains(src, "createElement(\"template\")") {
|
||||
t.Error("the mount is not parsed via <template>; scripts could execute before their DOM")
|
||||
}
|
||||
// ...and scripts are then re-created as fresh elements so they DO run.
|
||||
if !strings.Contains(src, "document.createElement(\"script\")") {
|
||||
t.Error("plugin <script> blocks are never re-created, so they never execute")
|
||||
}
|
||||
if !strings.Contains(src, "replaceWith(s)") {
|
||||
t.Error("the original inert <script> is not replaced by an executable one")
|
||||
}
|
||||
}
|
||||
|
||||
// TestUIPluginAPISurface: the documented browser API must exist with the exact
|
||||
// names docs/plugins.md promises, since plugin authors code against it.
|
||||
func TestUIPluginAPISurface(t *testing.T) {
|
||||
src := uiSourceX(t)
|
||||
for _, member := range []string{"fetchState", "postState", "onTabShown"} {
|
||||
if !strings.Contains(src, member+":") && !strings.Contains(src, member+"(") {
|
||||
t.Errorf("window.pluginAPI.%s is missing; docs/plugins.md documents it", member)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestUIPluginPageBecomesRealTab: a plugin page must get a pane in #main AND a
|
||||
// sidebar button wired to goTab, otherwise the page is unreachable.
|
||||
func TestUIPluginPageBecomesRealTab(t *testing.T) {
|
||||
src := uiSourceX(t)
|
||||
// pane in #main
|
||||
if !strings.Contains(src, `pane.id = "tab-" + id`) {
|
||||
t.Error("no pane is created for a plugin page")
|
||||
}
|
||||
if !strings.Contains(src, "main.appendChild(pane)") {
|
||||
t.Error("the plugin pane is not appended to #main")
|
||||
}
|
||||
// sidebar button wired to the tab router
|
||||
if !strings.Contains(src, "btn.dataset.tab = id") {
|
||||
t.Error("the sidebar button is not given a data-tab, so goTab() will not route to it")
|
||||
}
|
||||
if !strings.Contains(src, "btn.onclick = () => goTab(id)") {
|
||||
t.Error("the sidebar button is not wired to goTab()")
|
||||
}
|
||||
// and the router must know about it
|
||||
if !strings.Contains(src, "PLUGIN_PAGES.has(tab)") {
|
||||
t.Error("refresh() does not route plugin pages, so opening one renders nothing")
|
||||
}
|
||||
}
|
||||
|
||||
// TestUIPluginElementsHonorAnchor: elements declare top / bottom / before:sel /
|
||||
// after:sel. Silently ignoring the anchor would put a "top" tile at the bottom
|
||||
// of the status page, which looks like a layout bug rather than a plugin bug.
|
||||
func TestUIPluginElementsHonorAnchor(t *testing.T) {
|
||||
src := uiSourceX(t)
|
||||
for _, anchor := range []string{`anchor === "top"`, `anchor.startsWith("before:")`, `"after:"`} {
|
||||
if !strings.Contains(src, anchor) {
|
||||
t.Errorf("the anchor form %s is not handled; elements would all land at the bottom", anchor)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestUIPluginInjectionFailureIsNonFatal: plugins are optional, so a failed
|
||||
// /api/ui-inject must still leave a working UI (the dashboard has to render).
|
||||
// Two places have to cooperate: the function swallows the fetch error, and the
|
||||
// caller catches anything that still escapes so refresh() always runs.
|
||||
func TestUIPluginInjectionFailureIsNonFatal(t *testing.T) {
|
||||
src := uiSourceX(t)
|
||||
// inside the function: the fetch is wrapped in try/catch
|
||||
fnStart := strings.Index(src, "async function injectPluginUI()")
|
||||
if fnStart < 0 {
|
||||
t.Fatal("injectPluginUI() is not defined")
|
||||
}
|
||||
fn := src[fnStart:]
|
||||
if !strings.Contains(fn, "plugins are optional; the UI must work without them") {
|
||||
t.Error("injectPluginUI does not guard its own fetch failure")
|
||||
}
|
||||
// at the call site: the rejection cannot escape before the first render
|
||||
// LastIndex, not Index: the DEFINITION of injectPluginUI also matches, and
|
||||
// the definition has no .catch on it.
|
||||
iCall := strings.LastIndex(src, "injectPluginUI()")
|
||||
if iCall < 0 {
|
||||
t.Fatal("injectPluginUI() is never called")
|
||||
}
|
||||
// Bound the window at len(src): the call site sits near EOF and a fixed
|
||||
// slice overruns it (a panic in a test is worse than a skipped assertion).
|
||||
end := iCall + 220
|
||||
if end > len(src) {
|
||||
end = len(src)
|
||||
}
|
||||
if !strings.Contains(src[iCall:end], ".catch") {
|
||||
t.Error("a failed /api/ui-inject would reject before refresh(\"status\"), " +
|
||||
"leaving the dashboard blank")
|
||||
}
|
||||
}
|
||||
|
||||
// ---- plugin management UI contract ---------------------------------------
|
||||
//
|
||||
// The management page is the operator's only way to take a broken plugin out
|
||||
// of the request path. Every one of these assertions guards a link that, if it
|
||||
// silently broke, would leave the gateway running with a plugin it cannot
|
||||
// disable — the worst kind of gap: everything looks fine and nothing is
|
||||
// reachable.
|
||||
|
||||
func TestUIHasPluginTabAndPane(t *testing.T) {
|
||||
src := uiSourceX(t)
|
||||
if !strings.Contains(src, `data-tab="plugins"`) {
|
||||
t.Error("no sidebar entry for the plugin page")
|
||||
}
|
||||
if !strings.Contains(src, `id="tab-plugins"`) {
|
||||
t.Error("no #tab-plugins pane")
|
||||
}
|
||||
// The tab list is now a single constant; a new tab must be added there or
|
||||
// goTab will not un-hide its pane.
|
||||
if !strings.Contains(src, `const TABS = [`) {
|
||||
t.Error("TABS is gone; the tab list went back to a duplicated literal")
|
||||
}
|
||||
for _, tn := range []string{"status", "chat", "keys", "sort", "sources", "adapters", "plugins"} {
|
||||
if !strings.Contains(src, `"`+tn+`"`) {
|
||||
t.Errorf("TABS is missing %q", tn)
|
||||
}
|
||||
}
|
||||
// goTab must iterate TABS, not its own list.
|
||||
if !strings.Contains(src, "TABS.forEach((tn) =>") {
|
||||
t.Error("goTab does not iterate TABS")
|
||||
}
|
||||
if strings.Contains(src, `["status", "chat", "keys", "sort", "sources", "adapters"].forEach`) {
|
||||
t.Error("a duplicated tab literal survived; it will drift from TABS")
|
||||
}
|
||||
}
|
||||
|
||||
func TestUIRendersPluginManagement(t *testing.T) {
|
||||
src := uiSourceX(t)
|
||||
body, ok := jsFunctionBody(src, "renderPlugins")
|
||||
if !ok {
|
||||
t.Fatal("renderPlugins() not found")
|
||||
}
|
||||
// It must read the DISK listing, not just the loaded set: a plugin that
|
||||
// failed to compile is absent from the loaded set, and showing only the
|
||||
// loaded set makes a syntax error look like "the plugin is not installed".
|
||||
if !strings.Contains(body, "on_disk") {
|
||||
t.Error("renderPlugins reads only the loaded set; a failed plugin would " +
|
||||
"be invisible instead of shown with its error")
|
||||
}
|
||||
if !strings.Contains(body, "/api/plugins") {
|
||||
t.Error("renderPlugins does not call /api/plugins")
|
||||
}
|
||||
// Hook errors must be surfaced: a plugin that throws in every stage leaves
|
||||
// no other trace, so without this the symptom is "the feature just doesn't
|
||||
// work".
|
||||
if !strings.Contains(body, "hook_errors") {
|
||||
t.Error("renderPlugins ignores hook_errors; a silently broken plugin is undebuggable")
|
||||
}
|
||||
// Enable / disable / remove / edit.
|
||||
for _, fn := range []string{"togglePlugin", "delPlugin", "installPlugin", "editPlugin"} {
|
||||
if _, ok := jsFunctionBody(src, fn); !ok {
|
||||
t.Errorf("%s() is missing from the WebUI", fn)
|
||||
}
|
||||
}
|
||||
// The toggle must go through the enable/disable endpoint, not delete.
|
||||
tb, ok := jsFunctionBody(src, "togglePlugin")
|
||||
if !ok {
|
||||
t.Fatal("togglePlugin() missing")
|
||||
}
|
||||
if !strings.Contains(tb, `method: "PUT"`) {
|
||||
t.Error("togglePlugin does not use PUT")
|
||||
}
|
||||
if !strings.Contains(tb, "enabled:") {
|
||||
t.Error("togglePlugin does not send an \"enabled\" field")
|
||||
}
|
||||
// And the admin-only tab list must include plugins, or a non-admin would
|
||||
// see a page whose every action 403s.
|
||||
if !strings.Contains(src, `["sort", "sources", "adapters", "plugins"]`) {
|
||||
t.Error("the admin-only tab list omits \"plugins\"; a user key would see a " +
|
||||
"page full of actions that all fail with 403")
|
||||
}
|
||||
}
|
||||
|
||||
// TestUIBindDropzoneIsParameterised guards the refactor: the adapter and plugin
|
||||
// upload forms share one dropzone, so a hard-coded id would send a dropped
|
||||
// plugin file into the adapter name field.
|
||||
func TestUIBindDropzoneIsParameterised(t *testing.T) {
|
||||
src := uiSourceX(t)
|
||||
body, ok := jsFunctionBody(src, "bindDropzone")
|
||||
if !ok {
|
||||
t.Fatal("bindDropzone() not found")
|
||||
}
|
||||
if strings.Contains(body, `$("#dz")`) || strings.Contains(body, `$("#adp-name")`) {
|
||||
t.Error("bindDropzone still hard-codes the adapter's element ids; the " +
|
||||
"plugin form would write into the adapter form")
|
||||
}
|
||||
if !strings.Contains(body, "dzId") || !strings.Contains(body, "nameSel") {
|
||||
t.Error("bindDropzone does not accept the ids to bind")
|
||||
}
|
||||
// Both forms must call it.
|
||||
if !strings.Contains(src, `bindDropzone("pl-dz", "pl-file", "#pl-name", "#pl-code")`) {
|
||||
t.Error("the plugin upload form does not use the parameterised dropzone")
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginElementsSurviveHostRebuild guards the defect that made plugin
|
||||
// elements look absent no matter how the injection was configured.
|
||||
//
|
||||
// renderStatus (and six other pages) assign pane.innerHTML wholesale. Anything a
|
||||
// plugin had mounted into that pane is destroyed by the assignment. The symptom
|
||||
// is silent and misleading: the plugin really did declare an element, the
|
||||
// payload really did arrive, and the element is still gone on the next repaint —
|
||||
// so the natural conclusion is "my plugin declared nothing", which sends you
|
||||
// looking in the wrong file.
|
||||
//
|
||||
// The fix is to re-mount after the rebuild. This test asserts the re-mount is
|
||||
// wired at the SINGLE place every renderer passes through, rather than leaving
|
||||
// it to be re-added per page.
|
||||
func TestPluginElementsSurviveHostRebuild(t *testing.T) {
|
||||
html := uiSource(t)
|
||||
|
||||
if !strings.Contains(html, "function remountPluginElements") {
|
||||
t.Fatal("remountPluginElements is not defined; nothing can re-attach a plugin element after a host rebuild")
|
||||
}
|
||||
if !strings.Contains(html, "const PLUGIN_MOUNT_HOOKS = []") {
|
||||
t.Fatal("PLUGIN_MOUNT_HOOKS is not declared")
|
||||
}
|
||||
// The hook array must be declared BEFORE injectPluginUI pushes to it, and
|
||||
// before refresh() calls into it. A use-before-declaration in a const
|
||||
// block is a hard TDZ ReferenceError at first paint.
|
||||
hookDecl := strings.Index(html, "const PLUGIN_MOUNT_HOOKS = []")
|
||||
push := strings.Index(html, "PLUGIN_MOUNT_HOOKS.push")
|
||||
use := strings.Index(html, "PLUGIN_MOUNT_HOOKS.forEach")
|
||||
if hookDecl < 0 || push < 0 || use < 0 {
|
||||
t.Fatal("the hook array is declared but never both filled and drained")
|
||||
}
|
||||
if hookDecl > push || hookDecl > use {
|
||||
t.Error("PLUGIN_MOUNT_HOOKS is used before its declaration (const TDZ: first paint would throw)")
|
||||
}
|
||||
|
||||
// refresh() is the chokepoint every renderer passes through.
|
||||
rf := strings.Index(html, "function refresh(tab)")
|
||||
if rf < 0 {
|
||||
t.Fatal("refresh(tab) is gone")
|
||||
}
|
||||
body := html[rf:]
|
||||
if i := strings.Index(body, "\n }"); i > 0 {
|
||||
body = body[:i]
|
||||
}
|
||||
if !strings.Contains(body, "remountPluginElements") {
|
||||
t.Error("refresh() does not re-mount plugin elements: a page that rebuilds its DOM wipes them")
|
||||
}
|
||||
|
||||
// A mount must be idempotent, or the widget's <script> runs again on every
|
||||
// host repaint and its counters silently double.
|
||||
if !strings.Contains(html, `data-idx="`) || !strings.Contains(html, "plugin-el[data-idx=") {
|
||||
t.Error("element mounting is not guarded by a per-pane marker; re-mounting would re-run plugin scripts")
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginPagesAreReachableByGoTab guards a bug that only shows up in a
|
||||
// browser: goTab() iterated the hardcoded TABS list to toggle `hidden`, and a
|
||||
// plugin-contributed page is not in that list. The pane existed, the plugin had
|
||||
// filled it with real data, and clicking the sidebar entry changed nothing —
|
||||
// the `hidden` class was never removed.
|
||||
//
|
||||
// It reads exactly like "the page is blank" while the content sits in the DOM,
|
||||
// and no static check catches it: the injection is correct, the data is correct,
|
||||
// and the API returns 200.
|
||||
//
|
||||
// So this asserts the two structural facts that make it reachable: goTab
|
||||
// consults the plugin page set, and that set is declared before goTab runs (a
|
||||
// `const` further down the file would be a temporal-dead-zone ReferenceError on
|
||||
// the first click).
|
||||
func TestPluginPagesAreReachableByGoTab(t *testing.T) {
|
||||
ui := uiSource(t)
|
||||
|
||||
goTab := strings.Index(ui, "function goTab(")
|
||||
if goTab < 0 {
|
||||
t.Fatal("goTab() is gone")
|
||||
}
|
||||
end := strings.Index(ui[goTab:], "\n }")
|
||||
if end < 0 {
|
||||
t.Fatal("could not isolate goTab()")
|
||||
}
|
||||
body := ui[goTab : goTab+end]
|
||||
|
||||
if !strings.Contains(body, "PLUGIN_PAGES.forEach") {
|
||||
t.Error("★ goTab() does not toggle plugin pages — a plugin page keeps its " +
|
||||
"`hidden` class forever, so the sidebar entry does nothing")
|
||||
}
|
||||
|
||||
// Declaration must precede the use, or the first click throws.
|
||||
decl := strings.Index(ui, "var PLUGIN_PAGES = new Set()")
|
||||
if decl < 0 {
|
||||
// A const later in the file would also "work" only if nothing reads it
|
||||
// first — make that explicit rather than silent.
|
||||
if strings.Contains(ui, "const PLUGIN_PAGES = new Set()") {
|
||||
t.Error("PLUGIN_PAGES is a `const` declared after goTab() reads it — " +
|
||||
"temporal dead zone: the first click on a plugin page throws")
|
||||
} else {
|
||||
t.Error("PLUGIN_PAGES is not declared anywhere")
|
||||
}
|
||||
return
|
||||
}
|
||||
if decl > goTab {
|
||||
t.Errorf("PLUGIN_PAGES is declared at %d but goTab() at %d reads it — "+
|
||||
"declaration must come first", decl, goTab)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginSidebarEntryCarriesATitle: a plugin tab must render icon AND label.
|
||||
//
|
||||
// Native tabs are `<svg>…</svg><span data-i="navStatus">状态</span>`; the plugin
|
||||
// branch set innerHTML to the icon alone, so every plugin page showed up as a
|
||||
// nameless icon in the sidebar. An operator cannot tell what an icon means
|
||||
// until they click it.
|
||||
func TestPluginSidebarEntryCarriesATitle(t *testing.T) {
|
||||
html := uiSource(t)
|
||||
anchor := "btn.innerHTML =\n pluginIconHTML(pg.icon) +"
|
||||
i := strings.Index(html, anchor)
|
||||
if i < 0 {
|
||||
t.Fatal("plugin sidebar button construction not found")
|
||||
}
|
||||
// Look at the statement the anchor opens, not the whole file.
|
||||
stmt := html[i : i+400]
|
||||
if strings.Contains(stmt, "esc(pg.title") {
|
||||
return
|
||||
}
|
||||
t.Errorf("plugin sidebar button renders the icon only; a label is required. "+
|
||||
"got: %.200s", stmt)
|
||||
}
|
||||
@ -1,267 +0,0 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The WebUI dialogs all share the id "modal-wrap", and more than one can be
|
||||
// open at the same time (the seed-key notice sits on top of the keys page).
|
||||
// A save handler that closes "the" modal via $("#modal-wrap") therefore removes
|
||||
// whichever one comes FIRST in the document — which is the wrong dialog: the
|
||||
// form the user just submitted stays on screen while an unrelated dialog
|
||||
// vanishes.
|
||||
//
|
||||
// This is the same class of bug the api() contract test pins: reviewing inline
|
||||
// JS by eye does not catch it, and the symptom ("the dialog did not close")
|
||||
// points away from the cause.
|
||||
|
||||
// modalCloseRe finds every `$(...)`-style lookup of the shared modal id.
|
||||
var modalCloseRe = regexp.MustCompile(`\$\("#modal-wrap"\)`)
|
||||
|
||||
// closestModalRe finds the safe form: resolve the dialog from the clicked
|
||||
// button instead of from the document.
|
||||
var closestModalRe = regexp.MustCompile(`\.closest\("#modal-wrap"\)`)
|
||||
|
||||
func TestUIDialogClosesItselfNotTheFirstModal(t *testing.T) {
|
||||
// Strip comments first: prose that *names* the unsafe pattern (as the fix's
|
||||
// own comment does) would otherwise be flagged as a violation.
|
||||
code := stripJSComments(uiSource(t))
|
||||
|
||||
for _, m := range modalCloseRe.FindAllStringIndex(code, -1) {
|
||||
after := code[m[1]:]
|
||||
stmtEnd := strings.Index(after, ";")
|
||||
if stmtEnd < 0 || stmtEnd > 200 {
|
||||
continue
|
||||
}
|
||||
stmt := after[:stmtEnd]
|
||||
if strings.Contains(stmt, ".remove()") {
|
||||
line := 1 + strings.Count(code[:m[0]], "\n")
|
||||
t.Errorf("line %d closes the first #modal-wrap in the document, not its own dialog:\n\t%s",
|
||||
line, strings.TrimSpace(stmt))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// stripJSComments removes // line comments and /* block */ comments from JS
|
||||
// embedded in the UI document. It is deliberately simple: the document is our
|
||||
// own source, and a false negative here only means the check is silent.
|
||||
func stripJSComments(src string) string {
|
||||
var out strings.Builder
|
||||
lines := strings.Split(src, "\n")
|
||||
inBlock := false
|
||||
for _, ln := range lines {
|
||||
trimmed := strings.TrimSpace(ln)
|
||||
if inBlock {
|
||||
if strings.Contains(ln, "*/") {
|
||||
inBlock = false
|
||||
}
|
||||
continue
|
||||
}
|
||||
if strings.HasPrefix(trimmed, "/*") {
|
||||
if !strings.Contains(ln, "*/") {
|
||||
inBlock = true
|
||||
}
|
||||
continue
|
||||
}
|
||||
if i := strings.Index(ln, "//"); i >= 0 {
|
||||
before := ln[:i]
|
||||
if strings.Count(before, `"`)%2 == 0 && strings.Count(before, "'")%2 == 0 {
|
||||
ln = before
|
||||
}
|
||||
}
|
||||
out.WriteString(ln)
|
||||
out.WriteString("\n")
|
||||
}
|
||||
return out.String()
|
||||
}
|
||||
|
||||
// Every handler that closes a dialog must do it one of the two safe ways.
|
||||
func TestUIDialogClosuresGoThroughSafePaths(t *testing.T) {
|
||||
src := stripJSComments(uiSource(t))
|
||||
for _, fn := range []string{
|
||||
"downloadStatsCsv", "downloadKeysCsv", "saveSource", "saveTemplate",
|
||||
"scrAddFromForm", "sortScopeSave", "scopeSave", "createKey",
|
||||
} {
|
||||
body, ok := jsFunctionBody(src, fn)
|
||||
if !ok {
|
||||
t.Errorf("%s not found", fn)
|
||||
continue
|
||||
}
|
||||
if !strings.Contains(body, `closest("#modal-wrap")`) && !strings.Contains(body, "closeTopModal()") {
|
||||
t.Errorf("%s closes a dialog with neither .closest nor closeTopModal", fn)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(src, "function closeTopModal(") {
|
||||
t.Error("closeTopModal helper is missing")
|
||||
}
|
||||
}
|
||||
|
||||
// The helper must pick the LAST dialog (the topmost one the user sees), not the
|
||||
// first — that inversion is the whole bug.
|
||||
func TestUICloseTopModalTakesTheLast(t *testing.T) {
|
||||
body, ok := jsFunctionBody(uiSource(t), "closeTopModal")
|
||||
if !ok {
|
||||
t.Fatal("closeTopModal not found")
|
||||
}
|
||||
if !strings.Contains(body, "all.length - 1") {
|
||||
t.Errorf("closeTopModal does not take the last dialog:\n\t%s", oneLine(body))
|
||||
}
|
||||
}
|
||||
|
||||
// Handlers that resolve their dialog from a button must actually receive one:
|
||||
// a signature without the parameter means the .closest() silently yields null
|
||||
// and the save leaves its form stranded.
|
||||
func TestUIDialogHandlersReceiveTheirButton(t *testing.T) {
|
||||
src := stripJSComments(uiSource(t))
|
||||
for _, fn := range []string{
|
||||
"downloadStatsCsv", "saveSource", "scrAddFromForm", "sortScopeSave",
|
||||
"scopeSave", "createKey",
|
||||
} {
|
||||
body, ok := jsFunctionBody(src, fn)
|
||||
if !ok {
|
||||
t.Errorf("%s not found", fn)
|
||||
continue
|
||||
}
|
||||
if !strings.Contains(body, "closest(\"#modal-wrap\")") {
|
||||
continue // uses closeTopModal only
|
||||
}
|
||||
sig := body[:strings.Index(body, ")")+1]
|
||||
if !strings.Contains(sig, "btn") {
|
||||
t.Errorf("%s uses .closest(\"#modal-wrap\") but its signature %s has no button parameter —\n"+
|
||||
"the lookup would always be null and the form would never close", fn, oneLine(sig))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Quotas belong to the scope entries, so putScope only has to ship the scope
|
||||
// list — the caps travel inside it. What must NOT come back is a key-wide
|
||||
// total: it would be a second budget able to drift out of sync with the models
|
||||
// it is supposed to cover.
|
||||
func TestUIPutScopeShipsOnlyScopeQuotas(t *testing.T) {
|
||||
body, ok := jsFunctionBody(uiSource(t), "putScope")
|
||||
if !ok {
|
||||
t.Fatal("putScope not found")
|
||||
}
|
||||
code := stripJSComments(body)
|
||||
if !strings.Contains(code, "models:") || !strings.Contains(code, "scopes") {
|
||||
t.Error("putScope must ship the scope list the caps live in")
|
||||
}
|
||||
for _, gone := range []string{"kquota", "kreqquota", "kperiod", "khours"} {
|
||||
if strings.Contains(code, gone) {
|
||||
t.Errorf("putScope still references the removed key-wide quota field %q", gone)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A model brick carries its own budgets, and the scope editor reads and writes
|
||||
// both of them: dropping req_quota on the round trip would silently lift a
|
||||
// request cap every time someone edited a token cap.
|
||||
func TestUIScopeEditorRoundTripsBothQuotas(t *testing.T) {
|
||||
src := stripJSComments(uiSource(t))
|
||||
for _, fn := range []string{"scopeHtml", "readScopes", "scopeEdit", "scopeSave", "scopeQuotaBadge"} {
|
||||
if _, ok := jsFunctionBody(src, fn); !ok {
|
||||
t.Errorf("%s not found in the UI source", fn)
|
||||
}
|
||||
}
|
||||
for _, fn := range []string{"scopeHtml", "readScopes", "scopeSave", "scopeQuotaBadge"} {
|
||||
body, ok := jsFunctionBody(src, fn)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if !strings.Contains(body, "req_quota") && !strings.Contains(body, "reqquota") {
|
||||
t.Errorf("%s does not carry req_quota — a request cap would be lost on edit", fn)
|
||||
}
|
||||
}
|
||||
if form, ok := jsFunctionBody(src, "scopeEdit"); ok && !strings.Contains(form, "sc-reqs") {
|
||||
t.Error("the scope editor has no request-quota input")
|
||||
}
|
||||
}
|
||||
|
||||
// The whole key-wide quota surface must be gone from the UI: a badge or a
|
||||
// button reading a field the server no longer has would render "undefined" or
|
||||
// silently do nothing.
|
||||
func TestUIHasNoKeyWideQuotaSurface(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
for _, gone := range []string{
|
||||
"keyCapBadges", "keyQuotaEdit", "keyQuotaSave",
|
||||
"kq-tokens", "kq-reqs", "kq-period", "kq-hours",
|
||||
"kc-tokens", "kc-reqs", "kc-period", "kc-hours",
|
||||
"data-kquota", "data-kreqquota", "data-kperiod", "data-khours",
|
||||
} {
|
||||
if strings.Contains(src, gone) {
|
||||
t.Errorf("UI still references the removed key-wide quota surface %q", gone)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Existence of the strings is not enough: the badge has to READ the scope's
|
||||
// fields, and the editor has to read back what the brick writes. A field can
|
||||
// be present in the source and still never reach the screen — e.g. left in a
|
||||
// dead branch, or read from a data attribute the writer never sets.
|
||||
func TestUIKeyQuotaDataflowIsLive(t *testing.T) {
|
||||
src := stripJSComments(uiSource(t))
|
||||
|
||||
badge, ok := jsFunctionBody(src, "scopeQuotaBadge")
|
||||
if !ok {
|
||||
t.Fatal("scopeQuotaBadge not found")
|
||||
}
|
||||
for _, field := range []string{"m.token_quota", "m.req_quota", "m.period"} {
|
||||
if !strings.Contains(badge, field) {
|
||||
t.Errorf("scopeQuotaBadge does not read %q — the cap would never show on the model brick", field)
|
||||
}
|
||||
}
|
||||
|
||||
brick, ok := jsFunctionBody(src, "scopeHtml")
|
||||
if !ok {
|
||||
t.Fatal("scopeHtml not found")
|
||||
}
|
||||
edit, ok := jsFunctionBody(src, "scopeEdit")
|
||||
if !ok {
|
||||
t.Fatal("scopeEdit not found")
|
||||
}
|
||||
brickCode, editCode := stripJSComments(brick), stripJSComments(edit)
|
||||
reader := stripJSComments(mustBody(t, src, "readScopes"))
|
||||
for _, ds := range []string{"quota", "reqquota", "period", "hours"} {
|
||||
if !strings.Contains(brickCode, "data-"+ds+"=") {
|
||||
t.Errorf("scopeHtml does not write data-%s, so the editor has nothing to prefill", ds)
|
||||
}
|
||||
// the editor pre-fills through readScopes(), which is what walks the
|
||||
// bricks' data attributes — check the reader, not the form
|
||||
if !strings.Contains(reader, "dataset."+ds) {
|
||||
t.Errorf("readScopes does not read dataset.%s — editing a brick would save zeros over it", ds)
|
||||
}
|
||||
}
|
||||
// and the form must actually consume what readScopes produced
|
||||
for _, field := range []string{"sc.token_quota", "sc.req_quota", "sc.period", "sc.hours"} {
|
||||
if !strings.Contains(editCode, field) {
|
||||
t.Errorf("scopeEdit does not prefill from %q", field)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func mustBody(t *testing.T, src, fn string) string {
|
||||
t.Helper()
|
||||
b, ok := jsFunctionBody(src, fn)
|
||||
if !ok {
|
||||
t.Fatalf("%s not found", fn)
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
// The quota period vocabulary must match the server's, or the UI can offer a
|
||||
// value the API rejects.
|
||||
func TestUIQuotaPeriodsMatchServer(t *testing.T) {
|
||||
src := uiSource(t)
|
||||
for _, p := range []string{`value=""`, `value="hour"`, `value="week"`, `value="month"`, `value="nhour"`} {
|
||||
if !strings.Contains(src, p) {
|
||||
t.Errorf("UI period select is missing %s", p)
|
||||
}
|
||||
}
|
||||
for _, p := range []string{`"hour"`, `"week"`, `"month"`, `"nhour"`} {
|
||||
if !strings.Contains(src, `if (p === `+p+`)`) && !strings.Contains(src, `=== `+p+`)`) {
|
||||
t.Errorf("periodText() does not describe %s, so a badge would omit the window", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -2,22 +2,7 @@ local adapter = {}
|
||||
|
||||
adapter.name = "gemini"
|
||||
adapter.version = "2.0.0"
|
||||
-- The Go layer builds the request URL as base_url + endpoint, statically
|
||||
-- (see provider.URL). Gemini's real API is POST
|
||||
-- /v1beta/models/{model}:generateContent, and streaming is the same path with
|
||||
-- a ":streamGenerateContent" verb -- the model name is part of the PATH, so it
|
||||
-- cannot live in a static endpoint string.
|
||||
--
|
||||
-- "{model}" is therefore a placeholder the Go layer substitutes with the model
|
||||
-- this request actually sends (provider.urlFor substitutes it; see
|
||||
-- provider.go). Streaming additionally rewrites the ":generateContent" verb to
|
||||
-- ":streamGenerateContent" on the same template.
|
||||
--
|
||||
-- Leaving this as a bare "/v1/models" would call Gemini's model-LIST endpoint,
|
||||
-- which answers 405 to POST -- so the preset template would create a source that
|
||||
-- can never work. A source that overrides `endpoint:` bypasses the template
|
||||
-- entirely and must then spell the whole path itself.
|
||||
adapter.endpoint = "/v1beta/models/{model}:generateContent"
|
||||
adapter.endpoint = "/v1/models"
|
||||
adapter.headers = {}
|
||||
|
||||
-- Gemini API: POST /v1/models/{model}:generateContent
|
||||
|
||||
@ -54,14 +54,8 @@ function adapter.transform_request(raw_body)
|
||||
if not ok then return raw_body end
|
||||
req.disable_thinking = nil
|
||||
req.extra_body = nil
|
||||
-- stream_options is only valid alongside stream:true. Sending it on a
|
||||
-- non-streaming request is rejected by stricter upstreams (OpenCode Go:
|
||||
-- "stream_options should be set along with stream"), which failed every
|
||||
-- non-stream call. Only attach it when the request actually streams.
|
||||
if req.stream then
|
||||
if type(req.stream_options) ~= "table" then req.stream_options = {} end
|
||||
req.stream_options.include_usage = true
|
||||
end
|
||||
if type(req.stream_options) ~= "table" then req.stream_options = {} end
|
||||
req.stream_options.include_usage = true
|
||||
if req.messages then
|
||||
local kept = {}
|
||||
for _, msg in ipairs(req.messages) do
|
||||
|
||||
@ -1,354 +0,0 @@
|
||||
local adapter = {}
|
||||
|
||||
adapter.name = "opencodego"
|
||||
adapter.version = "1.0.0"
|
||||
adapter.endpoint = "/chat/completions"
|
||||
|
||||
-- OpenCode Go（包月订阅端点，https://opencode.ai/zen/go/v1）
|
||||
--
|
||||
-- 与 OpenCode Zen（opencodezen.lua，https://opencode.ai/zen/v1）是两个不同的
|
||||
-- 服务，行为要求并不相同，因此各有专用适配器：
|
||||
--
|
||||
-- * 本适配器（Go）服务包月订阅池。上游**强制要求**
|
||||
-- `x-opencode-session` 头，缺失直接 400 "MissingSessionID"。
|
||||
-- * Go 的 thinking 模型（deepseek-v4.1-flash 等）**必须回传** assistant 轮的
|
||||
-- `reasoning_content`，否则 400：
|
||||
-- The `reasoning_content` in the thinking mode must be passed back to the API.
|
||||
-- 这是本适配器与 Zen 适配器最关键的差异 —— Zen 会剥掉它，Go 必须原样保留。
|
||||
-- * Go 上游对协议更严格：非流式请求带 `stream_options` 会被拒
|
||||
-- （"stream_options should be set along with stream"）。
|
||||
--
|
||||
-- 同样需要 opencode 客户端指纹（UA + x-opencode-*）才能被正确识别与路由。
|
||||
adapter.headers = {
|
||||
["User-Agent"] = "opencode/1.18.21 ai-sdk/provider-utils/4.0.23 runtime/bun/1.3.14",
|
||||
}
|
||||
|
||||
-- 每请求生成身份头。沙箱无 os/math，用 meta.timestamp + 请求体哈希派生：
|
||||
-- 同秒内重复请求 id 相同可接受（zen 只校验存在性，不校验格式）。
|
||||
local function rand_id(prefix, seed)
|
||||
return prefix .. string.sub(sha256_hex(seed), 1, 24)
|
||||
end
|
||||
|
||||
-- 会话指纹：取历史里**第一条 user 消息**。
|
||||
--
|
||||
-- 为什么不直接用客户端的会话 id：实测抓包（tcpdump 抓 127.0.0.1:8081 的真实
|
||||
-- agent 请求）确认通用 OpenAI 客户端**根本不发**任何会话标识 —— body 里没有
|
||||
-- user / session_id / conversation_id / metadata，请求头也只有 X-Stainless-*
|
||||
-- （OpenAI JS SDK）与 User-Agent。opencode 原生客户端那套 x-opencode-session
|
||||
-- 是它自己的概念，通用客户端无从转发。
|
||||
--
|
||||
-- 退而求其次但要够用：历史被逐轮重放，**第一条 user 消息在整个会话里恒定**，
|
||||
-- 用它做指纹即可得到"每会话一个 session"，而不是"每源一个 session"。
|
||||
-- 拿不到时（无 user 消息）返回空串，调用方回退到按源稳定。
|
||||
local function conversation_fingerprint(body)
|
||||
if type(body) ~= "string" or body == "" then return "" end
|
||||
local ok, req = pcall(json.decode, body)
|
||||
if not ok or type(req) ~= "table" then return "" end
|
||||
for _, m in ipairs(req.messages or {}) do
|
||||
if type(m) == "table" and m.role == "user" then
|
||||
local c = m.content
|
||||
if type(c) == "string" then
|
||||
if c ~= "" then return c end
|
||||
elseif type(c) == "table" then
|
||||
for _, part in ipairs(c) do
|
||||
if type(part) == "table" and part.type == "text" and part.text then
|
||||
return part.text
|
||||
end
|
||||
end
|
||||
end
|
||||
return ""
|
||||
end
|
||||
end
|
||||
return ""
|
||||
end
|
||||
|
||||
-- session_seed 决定上游会话号（x-opencode-session 的种子）。
|
||||
--
|
||||
-- 优先级：
|
||||
-- 1) 客户端自带的会话标识（meta.client_session）——pi 等在开启
|
||||
-- compat.sendSessionAffinityHeaders 后会发 x-session-affinity，
|
||||
-- 这是平台真实的会话 id，整个会话恒定且天然按会话隔离。
|
||||
-- 2) 退路：历史里**第一条 user 消息**做会话指纹。
|
||||
-- 3) 再退：只按源固定（连 user 消息都没有时）。
|
||||
--
|
||||
-- 为什么需要 2/3：通用 OpenAI 客户端默认**根本不发**会话标识 —— 实测抓包
|
||||
-- （tcpdump 抓 127.0.0.1:8081 真实 agent 请求）确认 body 里没有
|
||||
-- user / session_id / conversation_id / metadata，请求头也只有
|
||||
-- X-Stainless-*（OpenAI JS SDK）与 User-Agent。opencode 原生客户端那套
|
||||
-- x-opencode-session 是它自己的概念，通用客户端无从转发。
|
||||
--
|
||||
-- 为什么必须稳定：上游前缀缓存是**会话级**的。会话号每请求一变，
|
||||
-- 缓存永不命中（实测：固定会号第 2 次命中 5888，每请求换会号则恒为 0）。
|
||||
local function session_seed(meta)
|
||||
local src = (meta.source and meta.source.name) or ""
|
||||
local base = "session|llmsproxy|" .. src
|
||||
local cs = meta.client_session
|
||||
if type(cs) == "string" and cs ~= "" then
|
||||
return base .. "|client|" .. cs
|
||||
end
|
||||
return base .. "|" .. conversation_fingerprint(meta.body)
|
||||
end
|
||||
|
||||
function adapter.build_headers(meta)
|
||||
local ts = tostring(meta.timestamp or "")
|
||||
local src = (meta.source and meta.source.name) or ""
|
||||
return {
|
||||
["User-Agent"] = adapter.headers["User-Agent"],
|
||||
["x-opencode-client"] = "cli",
|
||||
-- project 固定：同网关实例共享一个工作区身份
|
||||
["x-opencode-project"] = string.sub(sha256_hex("llmsproxy|" .. src), 1, 32),
|
||||
-- session 必须**按源稳定**，不能每请求换：上游的前缀缓存在同一 session
|
||||
-- 内才复用。实测（同一段 6032 token 提示词）：
|
||||
-- 固定 session -> 第 2 次命中 5888/6032，cached_tokens=5888
|
||||
-- 每请求换 session -> 永远 0 命中
|
||||
-- 原先用 meta.timestamp 派生，等于每请求都是新会话，缓存永远无效，
|
||||
-- 上游也无法做会话亲和路由。
|
||||
["x-opencode-session"] = rand_id("ses_", session_seed(meta)),
|
||||
-- request id 仍每请求唯一（它只是请求标识，不参与缓存键）
|
||||
["x-opencode-request"] = rand_id("msg_", "request|" .. ts .. "|" .. tostring(meta.body or "")),
|
||||
}
|
||||
end
|
||||
|
||||
-- OpenAI /chat/completions format (pass-through, strip provider-specific fields)
|
||||
-- zen 上游 schema 只接受 text content part（无视觉/音频能力）：多模态 part
|
||||
-- （image_url / input_audio / file 等）一律剥离。剥离后 content 变空的消息
|
||||
-- 若不再携带 tool_calls / tool_call_id 才整条丢弃（避免上游
|
||||
-- "unknown variant `image_url`, expected `text`"）；带工具调用的必须保留，
|
||||
-- 否则会把紧随其后的 tool 结果变成孤儿，模型会反复重发同一个调用。
|
||||
-- zen 上游角色白名单只有 system / user / assistant / tool / latest_reminder：
|
||||
-- OpenAI 的 developer（及 function 等）不在其中，直接透传会触发上游
|
||||
-- "unknown variant `developer`, expected one of ..." 错误；统一归一化为 system。
|
||||
local ROLE_WHITELIST = {
|
||||
system = true,
|
||||
user = true,
|
||||
assistant = true,
|
||||
tool = true,
|
||||
latest_reminder = true,
|
||||
}
|
||||
|
||||
function adapter.transform_request(raw_body)
|
||||
local ok, req = pcall(json.decode, raw_body)
|
||||
if not ok then return raw_body end
|
||||
req.disable_thinking = nil
|
||||
req.extra_body = nil
|
||||
-- stream_options is only valid alongside stream:true; sending it on a
|
||||
-- non-streaming request is rejected by strict upstreams.
|
||||
if req.stream then
|
||||
if type(req.stream_options) ~= "table" then req.stream_options = {} end
|
||||
req.stream_options.include_usage = true
|
||||
end
|
||||
if req.messages then
|
||||
local kept = {}
|
||||
for _, msg in ipairs(req.messages) do
|
||||
if type(msg.role) == "string" and not ROLE_WHITELIST[msg.role] then
|
||||
msg.role = "system"
|
||||
end
|
||||
-- Go 的 thinking 模式**必须**回传 reasoning_content：剥掉它会让
|
||||
-- 上游直接 400（"The `reasoning_content` in the thinking mode must
|
||||
-- be passed back to the API"），agent 的每一轮都会失败。
|
||||
-- 注意：与 opencodezen.lua 的行为**相反**，不要在这里剥。
|
||||
--
|
||||
-- 但绝大多数客户端（pi 等）根本不保存也不回传 reasoning，只保留
|
||||
-- 工具调用本身。上游只在“带 tool_calls 的助手轮”上校验这个字段，
|
||||
-- 实测**空串即可通过校验**，所以缺省时补空串：既满足上游的
|
||||
-- 一致性要求，又不伪造任何推理内容（用户看到的 reasoning 仍然是
|
||||
-- 上游本轮真实返回的）。
|
||||
if msg.role == "assistant"
|
||||
and type(msg.tool_calls) == "table" and #msg.tool_calls > 0
|
||||
and msg.reasoning_content == nil then
|
||||
msg.reasoning_content = ""
|
||||
end
|
||||
local drop = false
|
||||
if type(msg.content) == "table" then
|
||||
local parts = {}
|
||||
for _, part in ipairs(msg.content) do
|
||||
if type(part) == "table" and part.type ~= nil and part.type ~= "text" then
|
||||
-- multimodal part not supported by zen
|
||||
else
|
||||
table.insert(parts, part)
|
||||
end
|
||||
end
|
||||
if #parts == 0 then
|
||||
-- Content collapsed to nothing after stripping unsupported
|
||||
-- parts. A message that still carries a tool call must
|
||||
-- NEVER be dropped: the very next message is its tool
|
||||
-- result, and dropping the call orphans that result. The
|
||||
-- model then sees a result for a call it never made and
|
||||
-- re-issues the same tool call on every turn (observed as
|
||||
-- an infinite "repeated tool call" loop).
|
||||
if (type(msg.tool_calls) == "table" and #msg.tool_calls > 0)
|
||||
or msg.tool_call_id ~= nil then
|
||||
msg.content = ""
|
||||
else
|
||||
drop = true
|
||||
end
|
||||
else
|
||||
msg.content = parts
|
||||
end
|
||||
end
|
||||
if not drop then
|
||||
table.insert(kept, msg)
|
||||
end
|
||||
end
|
||||
req.messages = kept
|
||||
end
|
||||
return json.encode(req)
|
||||
end
|
||||
|
||||
function adapter.transform_response(raw_body)
|
||||
local ok, resp = pcall(json.decode, raw_body)
|
||||
if not ok or resp == nil then return raw_body end
|
||||
|
||||
local unified = {
|
||||
content = "",
|
||||
finish_reason = "",
|
||||
token_usage = { prompt = 0, completion = 0, total = 0 }
|
||||
}
|
||||
|
||||
if type(resp.usage) == "table" then
|
||||
unified.token_usage.prompt = resp.usage.prompt_tokens or 0
|
||||
unified.token_usage.completion = resp.usage.completion_tokens or 0
|
||||
unified.token_usage.total = resp.usage.total_tokens or 0
|
||||
local hit = 0
|
||||
if type(resp.usage.prompt_tokens_details) == "table" and resp.usage.prompt_tokens_details.cached_tokens ~= nil then
|
||||
hit = resp.usage.prompt_tokens_details.cached_tokens
|
||||
unified.token_usage.prompt_tokens_details = { cached_tokens = hit }
|
||||
end
|
||||
if (resp.usage.prompt_cache_hit_tokens or 0) > 0 then
|
||||
unified.token_usage.prompt_cache_hit_tokens = resp.usage.prompt_cache_hit_tokens
|
||||
unified.token_usage.prompt_cache_miss_tokens = resp.usage.prompt_cache_miss_tokens or 0
|
||||
if hit == 0 then
|
||||
unified.token_usage.prompt_tokens_details = { cached_tokens = resp.usage.prompt_cache_hit_tokens }
|
||||
end
|
||||
end
|
||||
-- 上游报告输出里有多少是思考 token。不给客户端的话，无法判断
|
||||
-- completion_tokens 里多少是「可见回答」、多少是「思考」（都按输出计费）。
|
||||
local ctd = resp.usage.completion_tokens_details
|
||||
if type(ctd) == "table" and ctd.reasoning_tokens ~= nil then
|
||||
unified.token_usage.completion_tokens_details = { reasoning_tokens = ctd.reasoning_tokens }
|
||||
end
|
||||
end
|
||||
|
||||
-- 本次调用的费用，上游放在**顶层**且是**字符串**（如 "0"、"0.0012"）。
|
||||
-- Go 订阅是包月制，恒为 "0"；只有 Zen 按量付费模型才有信息量。
|
||||
-- 原样透传：网关不解析、不换算、不汇总 —— 它只是上游事实的搬运者。
|
||||
if resp.cost ~= nil then
|
||||
unified.cost = tostring(resp.cost)
|
||||
end
|
||||
|
||||
if type(resp.choices) == "table" and #resp.choices > 0 then
|
||||
local ch = resp.choices[1]
|
||||
if type(ch.message) == "table" then
|
||||
unified.content = ch.message.content or ""
|
||||
local reasoning = ch.message.reasoning_content or ch.message.reasoning
|
||||
if reasoning then
|
||||
unified.reasoning_content = reasoning
|
||||
end
|
||||
if type(ch.message.tool_calls) == "table" then
|
||||
local tcs = {}
|
||||
for _, tc in ipairs(ch.message.tool_calls) do
|
||||
local args_ok, args = pcall(json.decode, tc["function"].arguments)
|
||||
if not args_ok then args = {} end
|
||||
table.insert(tcs, {
|
||||
id = tc.id,
|
||||
type = tc.type or "function",
|
||||
name = tc["function"].name,
|
||||
arguments = args
|
||||
})
|
||||
end
|
||||
unified.tool_calls = tcs
|
||||
end
|
||||
end
|
||||
unified.finish_reason = ch.finish_reason or ""
|
||||
end
|
||||
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
function adapter.transform_stream_chunk(raw_chunk)
|
||||
local ok, chunk = pcall(json.decode, raw_chunk)
|
||||
if not ok then return "" end
|
||||
|
||||
-- OpenAI-style streams may attach usage to a chunk with empty choices
|
||||
-- (the final usage chunk). Preserve it; the gateway emits it as the
|
||||
-- terminal usage chunk. Note: keys must match Go's TokenUsage json tags
|
||||
-- (prompt/completion/total); the gateway re-emits standard *_tokens.
|
||||
local uses = nil
|
||||
if type(chunk.usage) == "table" then
|
||||
uses = {
|
||||
prompt = chunk.usage.prompt_tokens or chunk.usage.prompt or 0,
|
||||
completion = chunk.usage.completion_tokens or chunk.usage.completion or 0,
|
||||
total = chunk.usage.total_tokens or chunk.usage.total or 0,
|
||||
}
|
||||
-- 独立字段与 prompt_tokens_details **并存**透传，不要写成 elseif：
|
||||
-- 上游命中缓存时两者都发，非流式路径也是两个都带。写成 elseif 会让
|
||||
-- 流式丢掉 prompt_cache_hit_tokens/miss，与同一源的**非流式**产出不一致
|
||||
-- （dsh 优先读 details，但只认独立字段的老客户端会看不到缓存）。
|
||||
if type(chunk.usage.prompt_tokens_details) == "table" and chunk.usage.prompt_tokens_details.cached_tokens ~= nil then
|
||||
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_tokens_details.cached_tokens }
|
||||
end
|
||||
if (chunk.usage.prompt_cache_hit_tokens or 0) > 0 then
|
||||
uses.prompt_cache_hit_tokens = chunk.usage.prompt_cache_hit_tokens
|
||||
uses.prompt_cache_miss_tokens = chunk.usage.prompt_cache_miss_tokens or 0
|
||||
if uses.prompt_tokens_details == nil then
|
||||
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_cache_hit_tokens }
|
||||
end
|
||||
end
|
||||
local ctd = chunk.usage.completion_tokens_details
|
||||
if type(ctd) == "table" and ctd.reasoning_tokens ~= nil then
|
||||
uses.completion_tokens_details = { reasoning_tokens = ctd.reasoning_tokens }
|
||||
end
|
||||
end
|
||||
|
||||
if not chunk.choices or #chunk.choices == 0 then
|
||||
-- 空 choices 的帧既不是内容也不是结束信号。上游发两种：
|
||||
-- {"choices":[],"usage":{...}} —— 终帧用量
|
||||
-- {"choices":[],"cost":"0"} —— 单独的费用帧
|
||||
-- 两者可能落在同一帧上，必须都转出去；早退只带 usage 会把费用丢干净。
|
||||
local out = { done = false }
|
||||
if uses ~= nil then out.usage = uses end
|
||||
if chunk.cost ~= nil then out.cost = tostring(chunk.cost) end
|
||||
if uses == nil and chunk.cost == nil then return "" end
|
||||
return json.encode(out)
|
||||
end
|
||||
local delta = chunk.choices[1].delta or {}
|
||||
local fr = chunk.choices[1].finish_reason
|
||||
|
||||
local finish = (type(fr) == "string" and fr ~= "") and fr or nil
|
||||
|
||||
local unified = {
|
||||
content = delta.content or "",
|
||||
done = (finish ~= nil)
|
||||
}
|
||||
if finish then
|
||||
unified.finish_reason = finish
|
||||
end
|
||||
if uses ~= nil then
|
||||
unified.usage = uses
|
||||
end
|
||||
-- zen 用 reasoning 字段承载推理文本（OpenAI 惯例是 reasoning_content）
|
||||
local reasoning = delta.reasoning_content or delta.reasoning
|
||||
if reasoning then
|
||||
unified.reasoning_content = reasoning
|
||||
end
|
||||
if delta.tool_calls then
|
||||
-- pass raw streaming fragments through; OpenAI clients accumulate index+id+name+arguments
|
||||
unified.tool_calls = delta.tool_calls
|
||||
end
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
-- 错误收敛（可选钩子）：zen 错误信封固定为 {error={type,message}}；
|
||||
-- 免费池限流 FreeUsageLimitError 单独标注。返回 nil 走通用兜底。
|
||||
function adapter.transform_error(status, body)
|
||||
local ok, resp = pcall(json.decode, body)
|
||||
if not ok or type(resp) ~= "table" then return nil end
|
||||
local e = resp.error
|
||||
if type(e) ~= "table" then return nil end
|
||||
if e.type == "FreeUsageLimitError" then
|
||||
return "zen free pool quota exhausted"
|
||||
end
|
||||
return e.message
|
||||
end
|
||||
|
||||
return adapter
|
||||
@ -1,342 +0,0 @@
|
||||
local adapter = {}
|
||||
|
||||
adapter.name = "opencodezen"
|
||||
adapter.version = "1.0.0"
|
||||
adapter.endpoint = "/chat/completions"
|
||||
|
||||
-- OpenCode Zen（按量付费端点，https://opencode.ai/zen/v1）
|
||||
--
|
||||
-- 与 OpenCode Go（opencodego.lua，https://opencode.ai/zen/go/v1）是两个不同的
|
||||
-- 服务，行为要求并不相同，因此各有专用适配器：
|
||||
--
|
||||
-- * 本适配器（Zen）服务免费池 / 按量付费池。免费模型必须带 opencode 客户端
|
||||
-- 指纹（UA + x-opencode-*），否则上游拒绝："free tier can only be used in
|
||||
-- OpenCode"。Zen 的免费池以非 thinking 模型为主，历史上回传
|
||||
-- reasoning_content 会引出上游报错，故这里剥掉（Go 侧相反，见 opencodego.lua）。
|
||||
-- * Go 订阅读者请看 opencodego.lua：那边**必须**回传 reasoning_content，
|
||||
-- 否则 thinking 模式直接 400。
|
||||
--
|
||||
-- 缺 x-opencode-client/session/request/project 身份头会被判为匿名客户端，
|
||||
-- 部分模型在带 tools 时上游直接失败（流式返回单 chunk
|
||||
-- "finish_reason":"network_error" 空 content，非流式 503
|
||||
-- "Endpoint is unavailable"，表现为"空回复"）。
|
||||
adapter.headers = {
|
||||
["User-Agent"] = "opencode/1.18.21 ai-sdk/provider-utils/4.0.23 runtime/bun/1.3.14",
|
||||
}
|
||||
|
||||
-- 每请求生成身份头。沙箱无 os/math，用 meta.timestamp + 请求体哈希派生：
|
||||
-- 同秒内重复请求 id 相同可接受（zen 只校验存在性，不校验格式）。
|
||||
local function rand_id(prefix, seed)
|
||||
return prefix .. string.sub(sha256_hex(seed), 1, 24)
|
||||
end
|
||||
|
||||
-- 会话指纹：取历史里**第一条 user 消息**。
|
||||
--
|
||||
-- 为什么不直接用客户端的会话 id：实测抓包（tcpdump 抓 127.0.0.1:8081 的真实
|
||||
-- agent 请求）确认通用 OpenAI 客户端**根本不发**任何会话标识 —— body 里没有
|
||||
-- user / session_id / conversation_id / metadata，请求头也只有 X-Stainless-*
|
||||
-- （OpenAI JS SDK）与 User-Agent。opencode 原生客户端那套 x-opencode-session
|
||||
-- 是它自己的概念，通用客户端无从转发。
|
||||
--
|
||||
-- 退而求其次但要够用：历史被逐轮重放，**第一条 user 消息在整个会话里恒定**，
|
||||
-- 用它做指纹即可得到"每会话一个 session"，而不是"每源一个 session"。
|
||||
-- 拿不到时（无 user 消息）返回空串，调用方回退到按源稳定。
|
||||
local function conversation_fingerprint(body)
|
||||
if type(body) ~= "string" or body == "" then return "" end
|
||||
local ok, req = pcall(json.decode, body)
|
||||
if not ok or type(req) ~= "table" then return "" end
|
||||
for _, m in ipairs(req.messages or {}) do
|
||||
if type(m) == "table" and m.role == "user" then
|
||||
local c = m.content
|
||||
if type(c) == "string" then
|
||||
if c ~= "" then return c end
|
||||
elseif type(c) == "table" then
|
||||
for _, part in ipairs(c) do
|
||||
if type(part) == "table" and part.type == "text" and part.text then
|
||||
return part.text
|
||||
end
|
||||
end
|
||||
end
|
||||
return ""
|
||||
end
|
||||
end
|
||||
return ""
|
||||
end
|
||||
|
||||
-- session_seed 决定上游会话号（x-opencode-session 的种子）。
|
||||
--
|
||||
-- 优先级：
|
||||
-- 1) 客户端自带的会话标识（meta.client_session）——pi 等在开启
|
||||
-- compat.sendSessionAffinityHeaders 后会发 x-session-affinity，
|
||||
-- 这是平台真实的会话 id，整个会话恒定且天然按会话隔离。
|
||||
-- 2) 退路：历史里**第一条 user 消息**做会话指纹。
|
||||
-- 3) 再退：只按源固定（连 user 消息都没有时）。
|
||||
--
|
||||
-- 为什么需要 2/3：通用 OpenAI 客户端默认**根本不发**会话标识 —— 实测抓包
|
||||
-- （tcpdump 抓 127.0.0.1:8081 真实 agent 请求）确认 body 里没有
|
||||
-- user / session_id / conversation_id / metadata，请求头也只有
|
||||
-- X-Stainless-*（OpenAI JS SDK）与 User-Agent。opencode 原生客户端那套
|
||||
-- x-opencode-session 是它自己的概念，通用客户端无从转发。
|
||||
--
|
||||
-- 为什么必须稳定：上游前缀缓存是**会话级**的。会话号每请求一变，
|
||||
-- 缓存永不命中（实测：固定会号第 2 次命中 5888，每请求换会号则恒为 0）。
|
||||
local function session_seed(meta)
|
||||
local src = (meta.source and meta.source.name) or ""
|
||||
local base = "session|llmsproxy|" .. src
|
||||
local cs = meta.client_session
|
||||
if type(cs) == "string" and cs ~= "" then
|
||||
return base .. "|client|" .. cs
|
||||
end
|
||||
return base .. "|" .. conversation_fingerprint(meta.body)
|
||||
end
|
||||
|
||||
function adapter.build_headers(meta)
|
||||
local ts = tostring(meta.timestamp or "")
|
||||
local src = (meta.source and meta.source.name) or ""
|
||||
return {
|
||||
["User-Agent"] = adapter.headers["User-Agent"],
|
||||
["x-opencode-client"] = "cli",
|
||||
-- project 固定：同网关实例共享一个工作区身份
|
||||
["x-opencode-project"] = string.sub(sha256_hex("llmsproxy|" .. src), 1, 32),
|
||||
-- session 必须**按源稳定**，不能每请求换：上游的前缀缓存在同一 session
|
||||
-- 内才复用。实测（同一段 6032 token 提示词）：
|
||||
-- 固定 session -> 第 2 次命中 5888/6032，cached_tokens=5888
|
||||
-- 每请求换 session -> 永远 0 命中
|
||||
-- 原先用 meta.timestamp 派生，等于每请求都是新会话，缓存永远无效，
|
||||
-- 上游也无法做会话亲和路由。
|
||||
["x-opencode-session"] = rand_id("ses_", session_seed(meta)),
|
||||
-- request id 仍每请求唯一（它只是请求标识，不参与缓存键）
|
||||
["x-opencode-request"] = rand_id("msg_", "request|" .. ts .. "|" .. tostring(meta.body or "")),
|
||||
}
|
||||
end
|
||||
|
||||
-- OpenAI /chat/completions format (pass-through, strip provider-specific fields)
|
||||
-- zen 上游 schema 只接受 text content part（无视觉/音频能力）：多模态 part
|
||||
-- （image_url / input_audio / file 等）一律剥离。剥离后 content 变空的消息
|
||||
-- 若不再携带 tool_calls / tool_call_id 才整条丢弃（避免上游
|
||||
-- "unknown variant `image_url`, expected `text`"）；带工具调用的必须保留，
|
||||
-- 否则会把紧随其后的 tool 结果变成孤儿，模型会反复重发同一个调用。
|
||||
-- zen 上游角色白名单只有 system / user / assistant / tool / latest_reminder：
|
||||
-- OpenAI 的 developer（及 function 等）不在其中，直接透传会触发上游
|
||||
-- "unknown variant `developer`, expected one of ..." 错误；统一归一化为 system。
|
||||
local ROLE_WHITELIST = {
|
||||
system = true,
|
||||
user = true,
|
||||
assistant = true,
|
||||
tool = true,
|
||||
latest_reminder = true,
|
||||
}
|
||||
|
||||
function adapter.transform_request(raw_body)
|
||||
local ok, req = pcall(json.decode, raw_body)
|
||||
if not ok then return raw_body end
|
||||
req.disable_thinking = nil
|
||||
req.extra_body = nil
|
||||
-- stream_options is only valid alongside stream:true; sending it on a
|
||||
-- non-streaming request is rejected by strict upstreams.
|
||||
if req.stream then
|
||||
if type(req.stream_options) ~= "table" then req.stream_options = {} end
|
||||
req.stream_options.include_usage = true
|
||||
end
|
||||
if req.messages then
|
||||
local kept = {}
|
||||
for _, msg in ipairs(req.messages) do
|
||||
if type(msg.role) == "string" and not ROLE_WHITELIST[msg.role] then
|
||||
msg.role = "system"
|
||||
end
|
||||
-- Zen 免费池不接受回传 reasoning_content（Go 侧相反）
|
||||
msg.reasoning_content = nil
|
||||
local drop = false
|
||||
if type(msg.content) == "table" then
|
||||
local parts = {}
|
||||
for _, part in ipairs(msg.content) do
|
||||
if type(part) == "table" and part.type ~= nil and part.type ~= "text" then
|
||||
-- multimodal part not supported by zen
|
||||
else
|
||||
table.insert(parts, part)
|
||||
end
|
||||
end
|
||||
if #parts == 0 then
|
||||
-- Content collapsed to nothing after stripping unsupported
|
||||
-- parts. A message that still carries a tool call must
|
||||
-- NEVER be dropped: the very next message is its tool
|
||||
-- result, and dropping the call orphans that result. The
|
||||
-- model then sees a result for a call it never made and
|
||||
-- re-issues the same tool call on every turn (observed as
|
||||
-- an infinite "repeated tool call" loop).
|
||||
if (type(msg.tool_calls) == "table" and #msg.tool_calls > 0)
|
||||
or msg.tool_call_id ~= nil then
|
||||
msg.content = ""
|
||||
else
|
||||
drop = true
|
||||
end
|
||||
else
|
||||
msg.content = parts
|
||||
end
|
||||
end
|
||||
if not drop then
|
||||
table.insert(kept, msg)
|
||||
end
|
||||
end
|
||||
req.messages = kept
|
||||
end
|
||||
return json.encode(req)
|
||||
end
|
||||
|
||||
function adapter.transform_response(raw_body)
|
||||
local ok, resp = pcall(json.decode, raw_body)
|
||||
if not ok or resp == nil then return raw_body end
|
||||
|
||||
local unified = {
|
||||
content = "",
|
||||
finish_reason = "",
|
||||
token_usage = { prompt = 0, completion = 0, total = 0 }
|
||||
}
|
||||
|
||||
if type(resp.usage) == "table" then
|
||||
unified.token_usage.prompt = resp.usage.prompt_tokens or 0
|
||||
unified.token_usage.completion = resp.usage.completion_tokens or 0
|
||||
unified.token_usage.total = resp.usage.total_tokens or 0
|
||||
local hit = 0
|
||||
if type(resp.usage.prompt_tokens_details) == "table" and resp.usage.prompt_tokens_details.cached_tokens ~= nil then
|
||||
hit = resp.usage.prompt_tokens_details.cached_tokens
|
||||
unified.token_usage.prompt_tokens_details = { cached_tokens = hit }
|
||||
end
|
||||
if (resp.usage.prompt_cache_hit_tokens or 0) > 0 then
|
||||
unified.token_usage.prompt_cache_hit_tokens = resp.usage.prompt_cache_hit_tokens
|
||||
unified.token_usage.prompt_cache_miss_tokens = resp.usage.prompt_cache_miss_tokens or 0
|
||||
if hit == 0 then
|
||||
unified.token_usage.prompt_tokens_details = { cached_tokens = resp.usage.prompt_cache_hit_tokens }
|
||||
end
|
||||
end
|
||||
-- 上游报告输出里有多少是思考 token。不给客户端的话，无法判断
|
||||
-- completion_tokens 里多少是「可见回答」、多少是「思考」（都按输出计费）。
|
||||
local ctd = resp.usage.completion_tokens_details
|
||||
if type(ctd) == "table" and ctd.reasoning_tokens ~= nil then
|
||||
unified.token_usage.completion_tokens_details = { reasoning_tokens = ctd.reasoning_tokens }
|
||||
end
|
||||
end
|
||||
|
||||
-- 本次调用的费用，上游放在**顶层**且是**字符串**（如 "0"、"0.0012"）。
|
||||
-- Go 订阅是包月制，恒为 "0"；只有 Zen 按量付费模型才有信息量。
|
||||
-- 原样透传：网关不解析、不换算、不汇总 —— 它只是上游事实的搬运者。
|
||||
if resp.cost ~= nil then
|
||||
unified.cost = tostring(resp.cost)
|
||||
end
|
||||
|
||||
if type(resp.choices) == "table" and #resp.choices > 0 then
|
||||
local ch = resp.choices[1]
|
||||
if type(ch.message) == "table" then
|
||||
unified.content = ch.message.content or ""
|
||||
local reasoning = ch.message.reasoning_content or ch.message.reasoning
|
||||
if reasoning then
|
||||
unified.reasoning_content = reasoning
|
||||
end
|
||||
if type(ch.message.tool_calls) == "table" then
|
||||
local tcs = {}
|
||||
for _, tc in ipairs(ch.message.tool_calls) do
|
||||
local args_ok, args = pcall(json.decode, tc["function"].arguments)
|
||||
if not args_ok then args = {} end
|
||||
table.insert(tcs, {
|
||||
id = tc.id,
|
||||
type = tc.type or "function",
|
||||
name = tc["function"].name,
|
||||
arguments = args
|
||||
})
|
||||
end
|
||||
unified.tool_calls = tcs
|
||||
end
|
||||
end
|
||||
unified.finish_reason = ch.finish_reason or ""
|
||||
end
|
||||
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
function adapter.transform_stream_chunk(raw_chunk)
|
||||
local ok, chunk = pcall(json.decode, raw_chunk)
|
||||
if not ok then return "" end
|
||||
|
||||
-- OpenAI-style streams may attach usage to a chunk with empty choices
|
||||
-- (the final usage chunk). Preserve it; the gateway emits it as the
|
||||
-- terminal usage chunk. Note: keys must match Go's TokenUsage json tags
|
||||
-- (prompt/completion/total); the gateway re-emits standard *_tokens.
|
||||
local uses = nil
|
||||
if type(chunk.usage) == "table" then
|
||||
uses = {
|
||||
prompt = chunk.usage.prompt_tokens or chunk.usage.prompt or 0,
|
||||
completion = chunk.usage.completion_tokens or chunk.usage.completion or 0,
|
||||
total = chunk.usage.total_tokens or chunk.usage.total or 0,
|
||||
}
|
||||
-- 独立字段与 prompt_tokens_details **并存**透传，不要写成 elseif：
|
||||
-- 上游命中缓存时两者都发，非流式路径也是两个都带。写成 elseif 会让
|
||||
-- 流式丢掉 prompt_cache_hit_tokens/miss，与同一源的**非流式**产出不一致
|
||||
-- （dsh 优先读 details，但只认独立字段的老客户端会看不到缓存）。
|
||||
if type(chunk.usage.prompt_tokens_details) == "table" and chunk.usage.prompt_tokens_details.cached_tokens ~= nil then
|
||||
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_tokens_details.cached_tokens }
|
||||
end
|
||||
if (chunk.usage.prompt_cache_hit_tokens or 0) > 0 then
|
||||
uses.prompt_cache_hit_tokens = chunk.usage.prompt_cache_hit_tokens
|
||||
uses.prompt_cache_miss_tokens = chunk.usage.prompt_cache_miss_tokens or 0
|
||||
if uses.prompt_tokens_details == nil then
|
||||
uses.prompt_tokens_details = { cached_tokens = chunk.usage.prompt_cache_hit_tokens }
|
||||
end
|
||||
end
|
||||
local ctd = chunk.usage.completion_tokens_details
|
||||
if type(ctd) == "table" and ctd.reasoning_tokens ~= nil then
|
||||
uses.completion_tokens_details = { reasoning_tokens = ctd.reasoning_tokens }
|
||||
end
|
||||
end
|
||||
|
||||
if not chunk.choices or #chunk.choices == 0 then
|
||||
-- 空 choices 的帧既不是内容也不是结束信号。上游发两种：
|
||||
-- {"choices":[],"usage":{...}} —— 终帧用量
|
||||
-- {"choices":[],"cost":"0"} —— 单独的费用帧
|
||||
-- 两者可能落在同一帧上，必须都转出去；早退只带 usage 会把费用丢干净。
|
||||
local out = { done = false }
|
||||
if uses ~= nil then out.usage = uses end
|
||||
if chunk.cost ~= nil then out.cost = tostring(chunk.cost) end
|
||||
if uses == nil and chunk.cost == nil then return "" end
|
||||
return json.encode(out)
|
||||
end
|
||||
local delta = chunk.choices[1].delta or {}
|
||||
local fr = chunk.choices[1].finish_reason
|
||||
|
||||
local finish = (type(fr) == "string" and fr ~= "") and fr or nil
|
||||
|
||||
local unified = {
|
||||
content = delta.content or "",
|
||||
done = (finish ~= nil)
|
||||
}
|
||||
if finish then
|
||||
unified.finish_reason = finish
|
||||
end
|
||||
if uses ~= nil then
|
||||
unified.usage = uses
|
||||
end
|
||||
-- zen 用 reasoning 字段承载推理文本（OpenAI 惯例是 reasoning_content）
|
||||
local reasoning = delta.reasoning_content or delta.reasoning
|
||||
if reasoning then
|
||||
unified.reasoning_content = reasoning
|
||||
end
|
||||
if delta.tool_calls then
|
||||
-- pass raw streaming fragments through; OpenAI clients accumulate index+id+name+arguments
|
||||
unified.tool_calls = delta.tool_calls
|
||||
end
|
||||
return json.encode(unified)
|
||||
end
|
||||
|
||||
-- 错误收敛（可选钩子）：zen 错误信封固定为 {error={type,message}}；
|
||||
-- 免费池限流 FreeUsageLimitError 单独标注。返回 nil 走通用兜底。
|
||||
function adapter.transform_error(status, body)
|
||||
local ok, resp = pcall(json.decode, body)
|
||||
if not ok or type(resp) ~= "table" then return nil end
|
||||
local e = resp.error
|
||||
if type(e) ~= "table" then return nil end
|
||||
if e.type == "FreeUsageLimitError" then
|
||||
return "zen free pool quota exhausted"
|
||||
end
|
||||
return e.message
|
||||
end
|
||||
|
||||
return adapter
|
||||
@ -1,351 +0,0 @@
|
||||
package lua
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// runBillingPage executes the billing page's mount script under node against a
|
||||
// DOM stub, drives the period selector, and returns what ended up on screen.
|
||||
//
|
||||
// It asserts against RENDERED output rather than the plugin's internal
|
||||
// helpers. Reaching into the IIFE for daysInWindow/rescale needs a test-only
|
||||
// hook in the plugin, and the first version of this file took a long detour
|
||||
// through such a hook: the harness's fake `Date` broke toISOString(), the
|
||||
// plugin script aborted before assigning the hook, and every assertion
|
||||
// compared "" against a real value. The rendered tables are the contract that
|
||||
// actually matters, they need no hook, and they cannot pass if the rescale is
|
||||
// wrong — which is exactly the bug class under test.
|
||||
func runBillingPage(t *testing.T, todayISO, period, captureID string) string {
|
||||
t.Helper()
|
||||
node, err := exec.LookPath("node")
|
||||
if err != nil {
|
||||
t.Skipf("node not available (%v)", err)
|
||||
}
|
||||
page, _ := billingUI(t)
|
||||
js := strings.Join(extractScripts(page), "\n")
|
||||
if !strings.Contains(js, "__billing_setPeriod") {
|
||||
t.Fatal("billing page exposes no period selector")
|
||||
}
|
||||
|
||||
harness := `
|
||||
global.window = global;
|
||||
global.__written = [];
|
||||
global.__errors = [];
|
||||
var __realErr = console.error;
|
||||
console.error = function () {
|
||||
global.__errors.push("console.error: " + Array.prototype.map.call(arguments, function (a) {
|
||||
return (a && a.message) ? a.message : String(a);
|
||||
}).join(" "));
|
||||
__realErr.apply(console, arguments);
|
||||
};
|
||||
process.on("uncaughtException", function (e) { global.__errors.push("uncaught: " + String(e && e.message || e)); });
|
||||
process.on("unhandledRejection", function (e) { global.__errors.push("unhandled: " + String(e && e.message || e)); });
|
||||
|
||||
// Pin "now" for the window under test.
|
||||
//
|
||||
// Two earlier attempts were wrong here, and both looked like product bugs:
|
||||
// replacing the Date constructor wholesale breaks toISOString/UTC; and
|
||||
// subclassing (setting __FakeDate.prototype = __RealDate.prototype while
|
||||
// returning __RealDate instances) leaves instances WITHOUT a working
|
||||
// getUTCDay, so the week window came out NaN and every rendered day became
|
||||
// "Invalid time value". A Proxy over the real constructor intercepts only the
|
||||
// zero-argument call and leaves instances — and therefore every prototype
|
||||
// method — completely untouched.
|
||||
var __NOW_MS = new Date("` + todayISO + `T12:34:56Z").getTime();
|
||||
var __RealDate = global.Date;
|
||||
var __PatchedDate = new Proxy(__RealDate, {
|
||||
construct: function (target, args) {
|
||||
// Reflect.construct, not a hand-spelled argument list: writing
|
||||
// new target(a, b, c, d, e, f, g) passes seven explicit undefineds for a
|
||||
// one-argument call, which Date reads as year..millisecond fields and turns
|
||||
// into an Invalid Date. The symptom was a week window rendering
|
||||
// "Invalid time value" for every day — a harness fault indistinguishable
|
||||
// from a product bug.
|
||||
if (args.length === 0) return Reflect.construct(target, [__NOW_MS]);
|
||||
return Reflect.construct(target, args);
|
||||
},
|
||||
apply: function () { return new __RealDate(__NOW_MS); }
|
||||
});
|
||||
__PatchedDate.now = function () { return __NOW_MS; };
|
||||
__PatchedDate.UTC = __RealDate.UTC;
|
||||
__PatchedDate.parse = __RealDate.parse;
|
||||
__PatchedDate.prototype = __RealDate.prototype;
|
||||
global.Date = __PatchedDate;
|
||||
|
||||
global.document = {
|
||||
getElementById: function (id) {
|
||||
if (!global.__els) global.__els = {};
|
||||
if (!global.__els[id]) global.__els[id] = {
|
||||
style: {}, dataset: {}, classList: { add: function(){}, remove: function(){} },
|
||||
options: [{value:"day"},{value:"week"},{value:"month"},{value:"all"}],
|
||||
set innerHTML(v) { if (v && String(v).trim()) global.__written.push(id); this.__h = v; },
|
||||
get innerHTML() { return this.__h || ""; },
|
||||
set textContent(v) { if (v !== undefined && String(v).trim()) global.__written.push(id); this.__t = v; },
|
||||
get textContent() { return this.__t || ""; },
|
||||
set innerText(v) { if (v !== undefined && String(v).trim()) global.__written.push(id); this.__i = v; },
|
||||
get innerText() { return this.__i || ""; },
|
||||
appendChild: function(){}, querySelector: function(){ return null; },
|
||||
querySelectorAll: function(){ return []; }, addEventListener: function(){}
|
||||
};
|
||||
return global.__els[id];
|
||||
},
|
||||
createElement: function () { return { style: {}, dataset: {}, appendChild: function(){}, setAttribute: function(){} }; },
|
||||
addEventListener: function () {},
|
||||
};
|
||||
|
||||
// Fixture days carry deliberately distinctive costs so a fold over the wrong
|
||||
// subset still produces a number, and only a mismatch shows it. 2026-01-05/06/07
|
||||
// are Monday/Tuesday/Wednesday; 2025-12-01 is far outside any window here.
|
||||
global.__st = {
|
||||
currency: "USD",
|
||||
total: { cost: 999, requests: 999, prompt_tokens: 999999 },
|
||||
by_source: { LIFETIMESRC: { cost: 999, requests: 999 } },
|
||||
by_model: { LIFETIMEMODEL: { cost: 999, requests: 999 } },
|
||||
by_key: { LIFETIMEKEY: { cost: 999, requests: 999 } },
|
||||
by_day: {
|
||||
"2026-01-05": { cost: 10, requests: 1, prompt_tokens: 100, completion_tokens: 10, failures: 0 },
|
||||
"2026-01-06": { cost: 20, requests: 2, prompt_tokens: 200, completion_tokens: 20, failures: 1 },
|
||||
"2026-01-07": { cost: 30, requests: 3, prompt_tokens: 300, completion_tokens: 30, failures: 0 },
|
||||
"2025-12-01": { cost: 777, requests: 77, prompt_tokens: 7700 }
|
||||
},
|
||||
by_day_src: {
|
||||
"2026-01-05": { srcA: { cost: 10, requests: 1 } },
|
||||
"2026-01-06": { srcA: { cost: 20, requests: 2 }, srcB: { cost: 5, requests: 1 } },
|
||||
"2026-01-07": { srcA: { cost: 30, requests: 3 } },
|
||||
"2025-12-01": { srcOLD: { cost: 777, requests: 77 } }
|
||||
},
|
||||
by_day_model: {
|
||||
"2026-01-05": { mA: { cost: 10, requests: 1 } },
|
||||
"2026-01-06": { mA: { cost: 20, requests: 2 } },
|
||||
"2026-01-07": { mA: { cost: 30, requests: 3 } }
|
||||
},
|
||||
by_day_key: {
|
||||
"2026-01-05": { kA: { cost: 10, requests: 1 } },
|
||||
"2026-01-06": { kA: { cost: 20, requests: 2 } },
|
||||
"2026-01-07": { kA: { cost: 30, requests: 3 } }
|
||||
},
|
||||
degraded_reqs: 4242, unpriced_reqs: 77
|
||||
};
|
||||
global.fetch = function () {
|
||||
return Promise.resolve({ ok: true, json: function () {
|
||||
return Promise.resolve({ plugin: "billing", state: global.__st });
|
||||
}});
|
||||
};
|
||||
global.pluginAPI = { onTabShown: function () {}, lang: "en" };
|
||||
`
|
||||
script := harness + "\n" + js + "\n" +
|
||||
"__billing_setPeriod(" + jsonStr(period) + ");\n" +
|
||||
// Let the period-triggered refresh() settle before reading the DOM.
|
||||
"setTimeout(function () {\n" +
|
||||
" var el = document.getElementById(" + jsonStr(captureID) + ");\n" +
|
||||
" console.log('__CAPTURE__' + JSON.stringify({\n" +
|
||||
" html: el ? String(el.innerHTML) : '(missing)',\n" +
|
||||
" kpis: String(document.getElementById('billing-kpis').innerHTML),\n" +
|
||||
" sel: String(document.getElementById('billing-period').value),\n" +
|
||||
" errors: global.__errors\n" +
|
||||
" }));\n" +
|
||||
"}, 60);\n"
|
||||
|
||||
cmd := exec.Command(node, "-e", script)
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatalf("node execution failed: %v\n%s", err, out)
|
||||
}
|
||||
o := string(out)
|
||||
idx := strings.Index(o, "__CAPTURE__")
|
||||
if idx < 0 {
|
||||
t.Fatalf("no capture in node output (the plugin script aborted?):\n%s", o)
|
||||
}
|
||||
raw := strings.TrimSpace(o[idx+len("__CAPTURE__"):])
|
||||
if nl := strings.IndexAny(raw, "\r\n"); nl >= 0 {
|
||||
raw = raw[:nl]
|
||||
}
|
||||
var got struct {
|
||||
HTML string `json:"html"`
|
||||
KPIs string `json:"kpis"`
|
||||
Sel string `json:"sel"`
|
||||
Errors []string `json:"errors"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(raw), &got); err != nil {
|
||||
t.Fatalf("capture is not JSON: %v\n%s", err, raw)
|
||||
}
|
||||
if len(got.Errors) > 0 {
|
||||
t.Fatalf("billing script reported errors: %v", got.Errors)
|
||||
}
|
||||
if got.Sel != period {
|
||||
t.Errorf("selector shows %q after setting %q — the change handler did not take", got.Sel, period)
|
||||
}
|
||||
return got.HTML + "\n@@KPI@@\n" + got.KPIs
|
||||
}
|
||||
|
||||
func jsonStr(s string) string {
|
||||
b, _ := json.Marshal(s)
|
||||
return string(b)
|
||||
}
|
||||
|
||||
// TestBillingPeriodDayShowsOnlyThatDay checks the one-day window end to end:
|
||||
// the tables must carry that day's rows and drop every other day, and the
|
||||
// lifetime sentinel (999 / LIFETIMESRC) must appear nowhere.
|
||||
func TestBillingPeriodDayShowsOnlyThatDay(t *testing.T) {
|
||||
got := runBillingPage(t, "2026-01-07", "day", "billing-by-source")
|
||||
if strings.Contains(got, "LIFETIME") || strings.Contains(got, "999") {
|
||||
t.Errorf("a one-day view still shows lifetime figures:\n%s", got)
|
||||
}
|
||||
if strings.Contains(got, "srcB") {
|
||||
t.Errorf("a one-day view shows srcB, which only had traffic on 01-06:\n%s", got)
|
||||
}
|
||||
if strings.Contains(got, "srcOLD") {
|
||||
t.Errorf("a one-day view shows a day from 2025-12:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "srcA") {
|
||||
t.Errorf("a one-day view lost the only source that had traffic that day:\n%s", got)
|
||||
}
|
||||
// The KPI total must be that day's cost (30), not the week (60) and not
|
||||
// the lifetime sentinel (999).
|
||||
if !strings.Contains(got, "30") {
|
||||
t.Errorf("KPI does not show the day's cost 30:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingPeriodWeekSumsThreeDays is the arithmetic: Mon+Wed rows and a
|
||||
// total of 60. A window that started on Sunday or ended early still renders
|
||||
// tables, so only the number distinguishes it.
|
||||
func TestBillingPeriodWeekSumsThreeDays(t *testing.T) {
|
||||
// 2026-01-07 is a Wednesday: the ISO week holds 01-05, 01-06, 01-07.
|
||||
got := runBillingPage(t, "2026-01-07", "week", "billing-by-source")
|
||||
if strings.Contains(got, "srcOLD") || strings.Contains(got, "LIFETIME") {
|
||||
t.Errorf("weekly view leaks out-of-window or lifetime rows:\n%s", got)
|
||||
}
|
||||
// srcA = 10+20+30 = 60, srcB = 5 (only 01-06, but inside the week).
|
||||
if !strings.Contains(got, "60") {
|
||||
t.Errorf("weekly total should be 60, got:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "srcB") {
|
||||
t.Errorf("weekly view dropped srcB, whose only day (01-06) is in the window:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingPeriodSundayReachesBackToMonday pins the ISO-week Monday rule
|
||||
// from the other side. On Sunday 2026-01-11 the week that already ended is
|
||||
// Mon 05..Sun 11, so all three fixture days are in it; a start-on-Sunday bug
|
||||
// would still include 01-07 and pass, so this case is paired with the
|
||||
// weekday case above rather than relied on alone.
|
||||
func TestBillingPeriodSundayReachesBackToMonday(t *testing.T) {
|
||||
got := runBillingPage(t, "2026-01-11", "week", "billing-by-source")
|
||||
if !strings.Contains(got, "60") {
|
||||
t.Errorf("Sunday's week view should still total 60:\n%s", got)
|
||||
}
|
||||
// And the Monday view must NOT include Sunday's (nonexistent) traffic.
|
||||
gotMon := runBillingPage(t, "2026-01-05", "day", "billing-by-source")
|
||||
if strings.Contains(gotMon, "30") {
|
||||
t.Errorf("Monday's single day view shows Wednesday's cost 30:\n%s", gotMon)
|
||||
}
|
||||
if !strings.Contains(gotMon, "10") {
|
||||
t.Errorf("Monday's day view lost Monday's own cost 10:\n%s", gotMon)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingPeriodKpiTotalMatchesWindowDays pins the KPI total itself.
|
||||
//
|
||||
// The dimension-table tests above cannot catch a broken window or a broken
|
||||
// fold: by_source is computed from the per-day DIMENSION map, independently of
|
||||
// the total. Mutation verification proved it — disabling the total fold, and
|
||||
// shifting the ISO-week start to Sunday, both left every earlier test green.
|
||||
// The total is derived from by_day, so it is the one value that moves when
|
||||
// either the window membership or the accumulation is wrong.
|
||||
func TestBillingPeriodKpiTotalMatchesWindowDays(t *testing.T) {
|
||||
kpis := func(t *testing.T, today, period string) string {
|
||||
t.Helper()
|
||||
return runBillingPage(t, today, period, "billing-by-source")
|
||||
}
|
||||
// 2026-01-07 is a Wednesday; fixture days are 05(10) 06(20) 07(30).
|
||||
// KPI renders through fmtInt, so 30 -> "30", 60 -> "60".
|
||||
if got := kpis(t, "2026-01-07", "day"); !strings.Contains(got, "30") {
|
||||
t.Errorf("day KPI should show that day's cost 30:\n%s", got)
|
||||
}
|
||||
if got := kpis(t, "2026-01-07", "week"); !strings.Contains(got, "60") {
|
||||
t.Errorf("week KPI should total 10+20+30 = 60:\n%s", got)
|
||||
}
|
||||
// Sunday 2026-01-11: the ISO week Mon 05..Sun 11 still holds all three days.
|
||||
if got := kpis(t, "2026-01-11", "week"); !strings.Contains(got, "60") {
|
||||
t.Errorf("Sunday's week KPI should still total 60:\n%s", got)
|
||||
}
|
||||
// The Monday rule from the negative side. A window that started on SUNDAY
|
||||
// would cover Sun 11..Sat 17 and contain none of the fixture days, giving
|
||||
// a total of 0 — so this case is what makes the +6 shift observable. The
|
||||
// positive case alone cannot distinguish it: shifting the start one day
|
||||
// later still happens to contain 01-07.
|
||||
if got := kpis(t, "2026-01-11", "day"); !strings.Contains(got, "0") {
|
||||
t.Errorf("Sunday's single-day window has no fixture traffic and must total 0:\n%s", got)
|
||||
}
|
||||
// Monday 2026-01-12 begins a new week; it contains no fixture days.
|
||||
if got := kpis(t, "2026-01-12", "week"); !strings.Contains(got, "0") {
|
||||
t.Errorf("an empty week should report 0, not stale data:\n%s", got)
|
||||
}
|
||||
// ...whereas the Monday-to-Sunday week that just ended does.
|
||||
if got := kpis(t, "2026-01-12", "day"); !strings.Contains(got, "0") {
|
||||
t.Errorf("Monday the 12th has no fixture traffic and must total 0:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingPeriodDimensionFoldSumsAcrossDays covers the per-dimension
|
||||
// accumulation. Asserting "60 appears somewhere" was NOT enough: mutation
|
||||
// verification showed a fold that keeps only the last day still passed,
|
||||
// because 60 also appears in the KPI total and in a token column. The
|
||||
// assertion must therefore pin srcA's own cost cell, not the page's text.
|
||||
func TestBillingPeriodDimensionFoldSumsAcrossDays(t *testing.T) {
|
||||
got := runBillingPage(t, "2026-01-07", "week", "billing-by-source")
|
||||
// srcA exists on all three days at 10/20/30. Summed = 60; last-only = 30.
|
||||
if !strings.Contains(got, "USD 60.0000") {
|
||||
t.Errorf("weekly by-source did not sum srcA across days (want USD 60.0000, "+
|
||||
"last-day-only would be USD 30.0000):\n%s", got)
|
||||
}
|
||||
// srcB exists only on 01-06 at 5, inside the window, so it must survive at
|
||||
// exactly its own value.
|
||||
if !strings.Contains(got, "USD 5.0000") {
|
||||
t.Errorf("weekly by-source lost or altered srcB (want USD 5.0000):\n%s", got)
|
||||
}
|
||||
// srcA's summed cost must also differ from its last-day value in the
|
||||
// rendered table, so an overwrite cannot masquerade as a sum.
|
||||
if strings.Count(got, "USD 60.0000") < 2 {
|
||||
t.Errorf("expected the summed 60 in both the table and the KPI:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// state untouched, including the lifetime sentinel and the lifetime-only
|
||||
// degraded counter. Without this control, a rescale that silently degraded
|
||||
// into "always empty" would pass the narrow-window tests.
|
||||
// TestBillingPeriodAllStaysLifetime is the control: "all" must keep the stored
|
||||
// state untouched. Without it, a rescale that silently degraded into "always
|
||||
// empty" would pass every narrow-window test above.
|
||||
func TestBillingPeriodAllStaysLifetime(t *testing.T) {
|
||||
got := runBillingPage(t, "2026-01-07", "all", "billing-by-source")
|
||||
if !strings.Contains(got, "LIFETIMESRC") && !strings.Contains(got, "999") {
|
||||
t.Errorf("the all-time view dropped the lifetime figures:\n%s", got)
|
||||
}
|
||||
// 4242 is the lifetime degraded_reqs; it has no day-keyed tally, so it
|
||||
// must remain visible when the window is "all". The KPI goes through
|
||||
// fmtInt, so it renders grouped ("4,242") — assert the rendered form.
|
||||
if !strings.Contains(got, "4,242") {
|
||||
t.Errorf("the all-time view should still show the lifetime degraded count:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingPeriodHidesLifetimeOnlyCounters is the mismatch the period view
|
||||
// exists to remove: degraded/unpriced have no day-keyed tally, so printing
|
||||
// the lifetime number next to a per-window cost is misleading. They must be
|
||||
// hidden, not merely relabelled.
|
||||
func TestBillingPeriodHidesLifetimeOnlyCounters(t *testing.T) {
|
||||
got := runBillingPage(t, "2026-01-07", "day", "billing-by-source")
|
||||
if strings.Contains(got, "4,242") || strings.Contains(got, "Degraded") {
|
||||
t.Errorf("a one-day view still shows the lifetime degraded counter:\n%s", got)
|
||||
}
|
||||
// ...while "all" does show it (covered above), so this is not just a
|
||||
// dictionary that lost the words.
|
||||
if !strings.Contains(runBillingPage(t, "2026-01-07", "all", "billing-by-source"), "4,242") {
|
||||
t.Error("the all-time view lost the degraded counter — the hide is not period-scoped")
|
||||
}
|
||||
}
|
||||
@ -1,968 +0,0 @@
|
||||
package lua
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The billing plugin ships with the gateway, so its arithmetic is a contract:
|
||||
// a wrong price silently produces wrong money. These tests drive it through the
|
||||
// real hook path and check the NUMBERS, not merely that it loads.
|
||||
|
||||
func billingVM(t *testing.T) (*Plugins, string) {
|
||||
t.Helper()
|
||||
dir := filepath.Join(t.TempDir(), "adapters")
|
||||
vm := NewVM(dir)
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatalf("vm: %v", err)
|
||||
}
|
||||
t.Cleanup(vm.Stop)
|
||||
pdir := filepath.Join(t.TempDir(), "plugins")
|
||||
ps := NewPlugins(vm, pdir)
|
||||
if err := ps.SeedBundled(); err != nil {
|
||||
t.Fatalf("seed: %v", err)
|
||||
}
|
||||
if err := ps.LoadDir(); err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
return ps, pdir
|
||||
}
|
||||
|
||||
// stateOf reads the plugin's published state as a generic map.
|
||||
func stateOf(t *testing.T, ps *Plugins) map[string]interface{} {
|
||||
t.Helper()
|
||||
raw := ps.State("billing")
|
||||
if raw == nil {
|
||||
t.Fatal("billing published no state")
|
||||
}
|
||||
b, err := json.Marshal(raw)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var out map[string]interface{}
|
||||
if err := json.Unmarshal(b, &out); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func approx(t *testing.T, name string, got, want float64) {
|
||||
t.Helper()
|
||||
d := got - want
|
||||
if d < 0 {
|
||||
d = -d
|
||||
}
|
||||
if d > 1e-9 {
|
||||
t.Errorf("%s = %v, want %v (delta %v)", name, got, want, d)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingZeroPricesIsSafe: with no configuration the plugin must still run
|
||||
// and report volume. A nil-price crash here would take out every request.
|
||||
func TestBillingZeroPricesIsSafe(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m", "source": "s", "key": "***aaaaaa", "ok": true,
|
||||
"prompt_tokens": 100, "completion_tokens": 50, "time": 1750000000000,
|
||||
})
|
||||
st := stateOf(t, ps)
|
||||
total := st["total"].(map[string]interface{})
|
||||
if total["requests"].(float64) != 1 {
|
||||
t.Errorf("requests = %v, want 1", total["requests"])
|
||||
}
|
||||
approx(t, "cost with no prices", total["cost"].(float64), 0)
|
||||
}
|
||||
|
||||
// TestBillingModelTokenPricing: the core case. prompt and completion are priced
|
||||
// SEPARATELY, which is how providers publish and how the total must come out.
|
||||
func TestBillingModelTokenPricing(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
// Setting prices must NOT disturb the (still empty) totals, which is the
|
||||
// whole point of the prices/state split.
|
||||
if err := ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"currency": "USD",
|
||||
"models": map[string]interface{}{
|
||||
"gpt-5.4": map[string]interface{}{"prompt": 1.25e-6, "completion": 1e-5},
|
||||
},
|
||||
},
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// 1000 prompt * 1.25e-6 = 0.00125 ; 500 completion * 1e-5 = 0.005
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "gpt-5.4", "source": "up", "key": "***aaaaaa", "ok": true,
|
||||
"prompt_tokens": 1000, "completion_tokens": 500, "time": 1750000000000,
|
||||
})
|
||||
st := stateOf(t, ps)
|
||||
approx(t, "total cost", st["total"].(map[string]interface{})["cost"].(float64), 0.00625)
|
||||
byModel := st["by_model"].(map[string]interface{})["gpt-5.4"].(map[string]interface{})
|
||||
approx(t, "model cost", byModel["cost"].(float64), 0.00625)
|
||||
if byModel["completion_tokens"].(float64) != 500 {
|
||||
t.Errorf("completion_tokens = %v, want 500", byModel["completion_tokens"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingPerRequestAndTokenCombine: a flat fee is ADDED to the token cost,
|
||||
// which is how an image model can be "tokens + fixed fee".
|
||||
func TestBillingPerRequestAndTokenCombine(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
if err := ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{
|
||||
"kolors": map[string]interface{}{"prompt": 1e-6, "completion": 2e-6, "per_request": 0.04},
|
||||
},
|
||||
},
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// 100*1e-6 + 50*2e-6 + 0.04 = 0.0402
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "kolors", "source": "sf", "key": "***bbbbbb", "ok": true,
|
||||
"prompt_tokens": 100, "completion_tokens": 50, "time": 1750000000000,
|
||||
})
|
||||
st := stateOf(t, ps)
|
||||
approx(t, "total", st["total"].(map[string]interface{})["cost"].(float64), 0.0402)
|
||||
}
|
||||
|
||||
// TestBillingPrecedence: keys > models > default for token prices.
|
||||
func TestBillingPrecedence(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"default": map[string]interface{}{"prompt": 9e-6, "completion": 9e-6},
|
||||
"models": map[string]interface{}{"m": map[string]interface{}{"prompt": 2e-6, "completion": 3e-6}},
|
||||
"keys": map[string]interface{}{"***cccccc": map[string]interface{}{"prompt": 1e-6, "completion": 1.5e-6}},
|
||||
},
|
||||
})
|
||||
|
||||
// No key match -> model price.
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m", "source": "s", "key": "***other", "ok": true,
|
||||
"prompt_tokens": 1000, "completion_tokens": 1000, "time": 1750000000000,
|
||||
})
|
||||
// Key match -> key price wins.
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m", "source": "s", "key": "***cccccc", "ok": true,
|
||||
"prompt_tokens": 1000, "completion_tokens": 1000, "time": 1750000000000,
|
||||
})
|
||||
st := stateOf(t, ps)
|
||||
// 1000*2e-6 + 1000*3e-6 = 0.005 ; 1000*1e-6 + 1000*1.5e-6 = 0.0025
|
||||
approx(t, "total (model + key)", st["total"].(map[string]interface{})["cost"].(float64), 0.0075)
|
||||
|
||||
// An unpriced model falls back to default.
|
||||
ps2, _ := billingVM(t)
|
||||
_ = ps2.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"default": map[string]interface{}{"prompt": 9e-6, "completion": 9e-6},
|
||||
},
|
||||
})
|
||||
ps2.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "unknown", "source": "s", "key": "***d", "ok": true,
|
||||
"prompt_tokens": 1000, "completion_tokens": 1000, "time": 1750000000000,
|
||||
})
|
||||
approx(t, "default fallback", stateOf(t, ps2)["total"].(map[string]interface{})["cost"].(float64), 0.018)
|
||||
}
|
||||
|
||||
// TestBillingAggregatesEveryDimension: one request must land in all four
|
||||
// rollups plus the daily bucket. A missing dimension is the kind of bug a
|
||||
// dashboard hides (it just renders an empty table).
|
||||
func TestBillingAggregatesEveryDimension(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"models": map[string]interface{}{"m1": map[string]interface{}{"prompt": 1e-6, "completion": 1e-6}},
|
||||
})
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m1", "source": "srcA", "key": "***key01", "ok": true,
|
||||
"prompt_tokens": 100, "completion_tokens": 100, "time": 1750000000000,
|
||||
})
|
||||
st := stateOf(t, ps)
|
||||
for _, dim := range []string{"by_source", "by_model", "by_key", "by_day"} {
|
||||
m, ok := st[dim].(map[string]interface{})
|
||||
if !ok || len(m) == 0 {
|
||||
t.Errorf("%s is empty; a dimension is missing", dim)
|
||||
}
|
||||
}
|
||||
if _, ok := st["by_source"].(map[string]interface{})["srcA"]; !ok {
|
||||
t.Error("by_source lacks srcA")
|
||||
}
|
||||
if _, ok := st["by_key"].(map[string]interface{})["***key01"]; !ok {
|
||||
t.Error("by_key lacks the gateway key")
|
||||
}
|
||||
// Milliseconds must be converted, not used as seconds: a raw 1750000000000
|
||||
// would land in a year-57000 bucket.
|
||||
days := st["by_day"].(map[string]interface{})
|
||||
found := false
|
||||
for k := range days {
|
||||
if len(k) == 10 && strings.Contains(k, "-") {
|
||||
found = true
|
||||
}
|
||||
if strings.HasPrefix(k, "5") && len(k) > 6 {
|
||||
t.Errorf("by_day key %q suggests millisecond timestamps were not converted", k)
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Errorf("by_day has no YYYY-MM-DD key: %v", days)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingFailedRequestPolicy: a failed request keeps its token cost (tokens
|
||||
// really were consumed) but drops the flat per_request fee (never charged).
|
||||
func TestBillingFailedRequestPolicy(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{
|
||||
"m": map[string]interface{}{"prompt": 1e-6, "completion": 1e-6, "per_request": 0.5},
|
||||
},
|
||||
},
|
||||
})
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m", "source": "s", "key": "***e", "ok": false, "status": 500,
|
||||
"prompt_tokens": 1000, "completion_tokens": 0, "time": 1750000000000,
|
||||
})
|
||||
st := stateOf(t, ps)
|
||||
// 1000*1e-6 = 0.001, flat dropped.
|
||||
approx(t, "failed request", st["total"].(map[string]interface{})["cost"].(float64), 0.001)
|
||||
if st["total"].(map[string]interface{})["failures"].(float64) != 1 {
|
||||
t.Error("failures not counted")
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingStateAPIReplace: the admin price update must actually change
|
||||
// subsequent pricing (not just be stored).
|
||||
func TestBillingStateAPIReplace(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{"m": map[string]interface{}{"prompt": 1e-6, "completion": 0}},
|
||||
},
|
||||
})
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m", "source": "s", "key": "***f", "ok": true,
|
||||
"prompt_tokens": 1000, "completion_tokens": 0, "time": 1750000000000,
|
||||
})
|
||||
approx(t, "before reprice", stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64), 0.001)
|
||||
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{"m": map[string]interface{}{"prompt": 2e-6, "completion": 0}},
|
||||
},
|
||||
})
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m", "source": "s", "key": "***f", "ok": true,
|
||||
"prompt_tokens": 1000, "completion_tokens": 0, "time": 1750000000000,
|
||||
})
|
||||
// 0.001 (old) + 0.002 (new price)
|
||||
approx(t, "after reprice", stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64), 0.003)
|
||||
}
|
||||
|
||||
// TestBillingPluginDeclaresUI: the shipped plugin must ship its dashboard, or
|
||||
// "billing is enabled" would be true while showing the user nothing.
|
||||
func TestBillingPluginDeclaresUI(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
for _, row := range ps.List() {
|
||||
if row["name"] != "billing" {
|
||||
continue
|
||||
}
|
||||
ui, ok := row["ui"].(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatal("billing declares no ui")
|
||||
}
|
||||
if page, _ := ui["page"].(string); page != "billing" {
|
||||
t.Errorf("ui.page = %v, want \"billing\"", ui["page"])
|
||||
}
|
||||
// ONE page, not two. The rule editor and the totals share a sidebar
|
||||
// entry because billing is one thing: prices decide the numbers and the
|
||||
// numbers are the result of the prices. Split across two pages, saving a
|
||||
// price left no on-screen way to see its effect — the loop the operator
|
||||
// actually works in was cut in half.
|
||||
pages, _ := ui["pages"].([]string)
|
||||
if len(pages) != 1 || pages[0] != "billing" {
|
||||
t.Errorf("ui.pages = %v, want exactly one page [billing]", pages)
|
||||
}
|
||||
if n, _ := ui["elements"].(int); n < 1 {
|
||||
t.Error("billing contributes no element to an existing page")
|
||||
}
|
||||
return
|
||||
}
|
||||
t.Fatal("billing plugin is not loaded")
|
||||
}
|
||||
|
||||
// TestBillingRuleEditorLivesInsideTheBillingPage: the rule editor is a VIEW
|
||||
// inside the billing page, not a second sidebar entry.
|
||||
//
|
||||
// This started as a separate page and was wrong: saving a price sent the
|
||||
// operator to a different screen to find out whether it worked. A single page
|
||||
// with a view switch keeps the loop closed, and the rule editor already refreshes
|
||||
// the usage numbers on save, so the effect is visible immediately.
|
||||
//
|
||||
// The mount must therefore carry BOTH the usage tables and the rule editor, and
|
||||
// the switch that toggles between them.
|
||||
func TestBillingRuleEditorLivesInsideTheBillingPage(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
var ui *UIExtension
|
||||
for _, p := range ps.plugins {
|
||||
if p.Info.Name == "billing" {
|
||||
ui = p.UI
|
||||
}
|
||||
}
|
||||
if ui == nil || ui.Page == nil {
|
||||
t.Fatal("billing plugin loaded with no page")
|
||||
}
|
||||
if ui.Page.PageID != "billing" {
|
||||
t.Errorf("page id = %q, want billing", ui.Page.PageID)
|
||||
}
|
||||
mount := ui.Page.Mount
|
||||
|
||||
// Both halves on one page.
|
||||
for _, needle := range []string{`id="billing-kpis"`, `id="billing-by-source"`} {
|
||||
if !strings.Contains(mount, needle) {
|
||||
t.Errorf("the usage half is missing %s", needle)
|
||||
}
|
||||
}
|
||||
for _, needle := range []string{
|
||||
`id="br-body"`,
|
||||
"data-act='save'", "data-act='export'", "data-act='newprofile'",
|
||||
".r-url", ".r-mode",
|
||||
} {
|
||||
if !strings.Contains(mount, needle) {
|
||||
t.Errorf("the rule editor is missing %s", needle)
|
||||
}
|
||||
}
|
||||
// The switch itself, or the two halves are both on screen at once.
|
||||
for _, needle := range []string{`id="billing-view-usage"`, `id="billing-view-rules"`, `window.__billing_view(`} {
|
||||
if !strings.Contains(mount, needle) {
|
||||
t.Errorf("the view switch is missing %s", needle)
|
||||
}
|
||||
}
|
||||
// The rules view must start hidden, otherwise it renders under the usage
|
||||
// tables and the page is a wall of two editors stacked.
|
||||
if !strings.Contains(mount, `id="billing-view-rules" style="display:none;`) {
|
||||
t.Error(`the rules view does not start hidden — both halves would render at once`)
|
||||
}
|
||||
// NO DUPLICATE IDS between the switch buttons and the view containers.
|
||||
// They shipped as billing-view-<name> on BOTH the button and the pane, so
|
||||
// document.getElementById returned the 72px-wide BUTTON for the pane and
|
||||
// every measurement was of the wrong element: the table measured 0 wide and
|
||||
// the page looked broken while the layout was fine. The ids must be unique
|
||||
// and the buttons carry a distinct suffix.
|
||||
for _, id := range []string{`billing-view-usage`, `billing-view-rules`} {
|
||||
if n := strings.Count(mount, `id="`+id+`"`); n != 1 {
|
||||
t.Errorf("id %q appears %d times; getElementById would return the wrong element", id, n)
|
||||
}
|
||||
if !strings.Contains(mount, `id="`+id+`-btn"`) {
|
||||
t.Errorf("switch button for %q is missing its -btn id", id)
|
||||
}
|
||||
}
|
||||
// Both view containers must FILL the pane. The host's tab-pane is a flex
|
||||
// column, so a plain div shrinks to its content: without flex:1/width:100%
|
||||
// the rules view measured 72px wide and its table measured 0 — the page
|
||||
// rendered "nothing" while the DOM was perfectly correct. This is exactly
|
||||
// the class of bug a DOM assertion misses and a screenshot catches.
|
||||
for _, needle := range []string{
|
||||
`id="billing-view-usage" style="flex:1`,
|
||||
`id="billing-view-rules" style="display:none;flex:1`,
|
||||
} {
|
||||
if !strings.Contains(mount, needle) {
|
||||
t.Errorf("view container does not fill the pane: %q missing — "+
|
||||
"a flex item without flex:1 collapses to content width", needle)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestUIExtensionMergesEveryPluginPage: two plugins contributing pages must
|
||||
// BOTH appear. The old merge assigned a single field, so the second plugin
|
||||
// erased the first one's page from the sidebar with no error anywhere.
|
||||
func TestUIExtensionMergesEveryPluginPage(t *testing.T) {
|
||||
ps, pdir := billingVM(t)
|
||||
mk := func(name, code string) {
|
||||
if err := os.WriteFile(filepath.Join(pdir, name+".lua"), []byte(code), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource(name, code); err != nil {
|
||||
t.Fatalf("load %s: %v", name, err)
|
||||
}
|
||||
}
|
||||
mk("other", `
|
||||
local plugin = {}
|
||||
plugin.name = "other"
|
||||
plugin.version = "0.1"
|
||||
plugin.ui = { page = { page_id = "other-page", title = "Other", order = 90,
|
||||
mount = "<div id='other-root'></div>" } }
|
||||
return plugin`)
|
||||
mk("third", `
|
||||
local plugin = {}
|
||||
plugin.name = "third"
|
||||
plugin.version = "0.1"
|
||||
plugin.ui = { pages = {
|
||||
{ page_id = "third-a", title = "Third A", order = 80, mount = "<div id='ta'></div>" },
|
||||
{ page_id = "third-b", title = "Third B", order = 81, mount = "<div id='tb'></div>" },
|
||||
} }
|
||||
return plugin`)
|
||||
|
||||
ui := ps.UI()
|
||||
if ui == nil {
|
||||
t.Fatal("no merged UI")
|
||||
}
|
||||
got := map[string]bool{}
|
||||
for _, pg := range ui.Pages {
|
||||
got[pg.PageID] = true
|
||||
}
|
||||
// billing contributes exactly one page (the rule editor is a view inside it),
|
||||
// so it is listed once. The multi-page merging is exercised by the three
|
||||
// plugins added here.
|
||||
for _, want := range []string{"billing", "other-page", "third-a", "third-b"} {
|
||||
if !got[want] {
|
||||
t.Errorf("merged UI lost page %q; has %v", want, got)
|
||||
}
|
||||
}
|
||||
// A duplicate page_id must not appear twice. Two plugins claiming the same
|
||||
// id collide in the DOM (getElementById returns the first, the second pane
|
||||
// is silently unreachable), so the merge keeps the first and drops the
|
||||
// later one.
|
||||
mk("collide", `
|
||||
local plugin = {}
|
||||
plugin.name = "collide"
|
||||
plugin.version = "0.1"
|
||||
plugin.ui = { pages = {
|
||||
{ page_id = "third-a", title = "Impostor", order = 79, mount = "<div id='impostor'></div>" },
|
||||
} }
|
||||
return plugin`)
|
||||
ui = ps.UI()
|
||||
counts := map[string]int{}
|
||||
for _, pg := range ui.Pages {
|
||||
counts[pg.PageID]++
|
||||
}
|
||||
if counts["third-a"] != 1 {
|
||||
t.Errorf("page id third-a appears %d times; a duplicate id collides in the DOM", counts["third-a"])
|
||||
}
|
||||
for _, pg := range ui.Pages {
|
||||
if pg.PageID == "third-a" && pg.Title == "Impostor" {
|
||||
t.Error("the LATER plugin won the id; first writer should keep it")
|
||||
}
|
||||
}
|
||||
|
||||
// Order must be honoured so the sidebar is predictable.
|
||||
if len(ui.Pages) != 4 {
|
||||
t.Fatalf("expected all 4 pages to merge, got %d: %v", len(ui.Pages), got)
|
||||
}
|
||||
for i := 1; i < len(ui.Pages); i++ {
|
||||
if ui.Pages[i].Order < ui.Pages[i-1].Order {
|
||||
t.Errorf("pages out of order at %d: %d before %d",
|
||||
i, ui.Pages[i-1].Order, ui.Pages[i].Order)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingPluginLoadedByDefault: the shipped plugin must load with no
|
||||
// configuration, since seeding only happens on a fresh plugin dir.
|
||||
func TestBillingPluginLoadedByDefault(t *testing.T) {
|
||||
ps, pdir := billingVM(t)
|
||||
if ps.Count() != 1 {
|
||||
t.Fatalf("expected 1 bundled plugin, got %d", ps.Count())
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(pdir, "billing.lua")); err != nil {
|
||||
t.Errorf("billing.lua was not written to the plugin dir: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingCountsDegradations: the plugin must distinguish a request that had
|
||||
// to drop below the top tier from one the top tier served. Without the chain
|
||||
// trace these were identical in the accounts, so a quietly degraded gateway
|
||||
// looked healthy while spending more per request.
|
||||
func TestBillingCountsDegradations(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{
|
||||
"hi-tier": map[string]interface{}{"prompt": 1e-5, "completion": 1e-5},
|
||||
"lo-tier": map[string]interface{}{"prompt": 1e-6, "completion": 1e-6},
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
// Request 1: degraded. tier 1 hard-failed, tier 2 served it.
|
||||
ps.Fire(StageChainStep, map[string]interface{}{
|
||||
"kind": "slot_fail", "tier": 1, "source": "t1", "model": "hi-tier",
|
||||
})
|
||||
ps.Fire(StageChainStep, map[string]interface{}{
|
||||
"kind": "selected", "tier": 2, "source": "t2", "model": "lo-tier",
|
||||
})
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "lo-tier", "source": "t2", "key": "***d1", "ok": true,
|
||||
"prompt_tokens": 1000, "completion_tokens": 1000,
|
||||
"degraded": true, "tier_served": 2, "time": 1750000000000,
|
||||
})
|
||||
|
||||
// Request 2: clean, served by the top tier.
|
||||
ps.Fire(StageChainStep, map[string]interface{}{
|
||||
"kind": "selected", "tier": 1, "source": "t1", "model": "hi-tier",
|
||||
})
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "hi-tier", "source": "t1", "key": "***d1", "ok": true,
|
||||
"prompt_tokens": 1000, "completion_tokens": 1000,
|
||||
"degraded": false, "tier_served": 1, "time": 1750000000000,
|
||||
})
|
||||
|
||||
st := stateOf(t, ps)
|
||||
if got := st["degraded_reqs"].(float64); got != 1 {
|
||||
t.Errorf("degraded_reqs = %v, want 1 (one of the two requests dropped a tier)", got)
|
||||
}
|
||||
tiers := st["by_tier_served"].(map[string]interface{})
|
||||
if tiers["2"].(float64) != 1 {
|
||||
t.Errorf("by_tier_served[2] = %v, want 1", tiers["2"])
|
||||
}
|
||||
if tiers["1"].(float64) != 1 {
|
||||
t.Errorf("by_tier_served[1] = %v, want 1", tiers["1"])
|
||||
}
|
||||
// Cost reflects the model actually served, not the one that should have been.
|
||||
// 1000*1e-6*2 = 0.002 for the degraded one, 1000*1e-5*2 = 0.02 for the clean one.
|
||||
approx(t, "total", st["total"].(map[string]interface{})["cost"].(float64), 0.022)
|
||||
}
|
||||
|
||||
// TestBillingAggregatesSkipReasons: skip reasons are the actionable diagnostic
|
||||
// ("no schedulable slot (cooling or quota exhausted)"), so they must be
|
||||
// counted. The wait time is normalised, otherwise a fresh row per request would
|
||||
// appear whenever the busy-wait text varies.
|
||||
func TestBillingAggregatesSkipReasons(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
ps.Fire(StageChainStep, map[string]interface{}{
|
||||
"kind": "tier_skip", "tier": 1, "reason": "no schedulable slot (cooling or quota exhausted)",
|
||||
})
|
||||
ps.Fire(StageChainStep, map[string]interface{}{
|
||||
"kind": "tier_busy", "tier": 2, "reason": "no free slot within 2s",
|
||||
})
|
||||
ps.Fire(StageChainStep, map[string]interface{}{
|
||||
"kind": "tier_busy", "tier": 3, "reason": "no free slot within 2.0001s",
|
||||
})
|
||||
st := stateOf(t, ps)
|
||||
reasons := st["skip_reasons"].(map[string]interface{})
|
||||
if len(reasons) != 2 {
|
||||
t.Errorf("skip_reasons = %v, want 2 (the two variable waits must collapse to one)", reasons)
|
||||
}
|
||||
busy, ok := reasons["no free slot within <wait>"]
|
||||
if !ok {
|
||||
t.Errorf("busy reason missing; got %v", reasons)
|
||||
} else if busy.(float64) != 2 {
|
||||
t.Errorf("busy count = %v, want 2 (two different wait texts, one cause)", busy)
|
||||
}
|
||||
}
|
||||
|
||||
// ---- prompt-cache pricing -------------------------------------------------
|
||||
//
|
||||
// A cached prompt token is not a fresh one. Charging the full prompt rate made a
|
||||
// 1M-token request of which 900k were cache reads cost 10 USD instead of ~1.9
|
||||
// — an order of magnitude, on exactly the traffic the cache exists to make
|
||||
// cheap. Agent traffic replays long shared prefixes constantly, so this was the
|
||||
// single largest source of over-billing in the plugin.
|
||||
|
||||
func TestBillingCacheHitsAreDiscounted(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{
|
||||
"m": map[string]interface{}{"prompt": 1e-5, "completion": 1e-5},
|
||||
},
|
||||
},
|
||||
})
|
||||
// 1M prompt of which 900k cached, default discount 0.1
|
||||
// 100k fresh * 1e-5 = 1.0 ; 900k cached * 1e-5 * 0.1 = 0.9
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m", "source": "s", "key": "***c", "ok": true,
|
||||
"prompt_tokens": 1000000, "completion_tokens": 0,
|
||||
"cache_hit_tokens": 900000, "time": 1750000000000,
|
||||
})
|
||||
approx(t, "cache-discounted cost", stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64), 1.9)
|
||||
}
|
||||
|
||||
// A per-model discount overrides the global one, because the ratio is a
|
||||
// per-provider fact, not a constant.
|
||||
func TestBillingCacheDiscountIsPerModel(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{
|
||||
"free-cache": map[string]interface{}{
|
||||
"prompt": 1e-5, "completion": 0, "cache_discount": 0,
|
||||
},
|
||||
"flat": map[string]interface{}{
|
||||
"prompt": 1e-5, "completion": 0, "cache_discount": 1,
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
for _, m := range []string{"free-cache", "flat"} {
|
||||
ps2, _ := billingVM(t)
|
||||
_ = ps2.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{
|
||||
m: map[string]interface{}{"prompt": 1e-5, "completion": 0, "cache_discount": map[bool]float64{true: 0, false: 1}[m == "free-cache"]},
|
||||
},
|
||||
},
|
||||
})
|
||||
ps2.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": m, "source": "s", "key": "***c", "ok": true,
|
||||
"prompt_tokens": 1000000, "cache_hit_tokens": 1000000,
|
||||
"time": 1750000000000,
|
||||
})
|
||||
want := 0.0
|
||||
if m == "flat" {
|
||||
want = 10.0
|
||||
}
|
||||
approx(t, m+" (all cached)", stateOf(t, ps2)["total"].(map[string]interface{})["cost"].(float64), want)
|
||||
}
|
||||
}
|
||||
|
||||
// A misbehaving adapter reporting more cache hits than prompt tokens must not
|
||||
// produce negative fresh tokens.
|
||||
func TestBillingCacheHitClampedToPrompt(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{"m": map[string]interface{}{"prompt": 1e-5, "completion": 0}},
|
||||
},
|
||||
})
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m", "source": "s", "key": "***c", "ok": true,
|
||||
"prompt_tokens": 100, "completion_tokens": 0,
|
||||
"cache_hit_tokens": 999999, // nonsense from a broken adapter
|
||||
"time": 1750000000000,
|
||||
})
|
||||
// Clamped to 100 cached, 0 fresh => 100 * 1e-5 * 0.1
|
||||
got := stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64)
|
||||
if got < 0 {
|
||||
t.Errorf("cost = %v, must never be negative", got)
|
||||
}
|
||||
approx(t, "clamped cost", got, 0.0001)
|
||||
}
|
||||
|
||||
// ---- unpriced traffic -----------------------------------------------------
|
||||
|
||||
// An unpriced model silently costing 0 is the most dangerous failure a cost
|
||||
// plugin has: the bill still adds up, it just quietly under-reports, and
|
||||
// nothing looks broken. It must be counted and named.
|
||||
func TestBillingCountsUnpricedTraffic(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{
|
||||
"priced": map[string]interface{}{"prompt": 1e-5},
|
||||
},
|
||||
},
|
||||
})
|
||||
// 100k+100k tokens on a model with no price entry.
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "MYSTERY-MODEL", "source": "s", "key": "***u", "ok": true,
|
||||
"prompt_tokens": 100000, "completion_tokens": 100000, "time": 1750000000000,
|
||||
})
|
||||
// A priced one, to prove the counter is selective.
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "priced", "source": "s", "key": "***u", "ok": true,
|
||||
"prompt_tokens": 1000, "completion_tokens": 0, "time": 1750000000000,
|
||||
})
|
||||
st := stateOf(t, ps)
|
||||
if got := st["unpriced_reqs"].(float64); got != 1 {
|
||||
t.Errorf("unpriced_reqs = %v, want 1 (only the mystery model)", got)
|
||||
}
|
||||
models := st["unpriced_models"].(map[string]interface{})
|
||||
if models["MYSTERY-MODEL"].(float64) != 1 {
|
||||
t.Errorf("unpriced_models = %v, want MYSTERY-MODEL counted", models)
|
||||
}
|
||||
if _, present := models["priced"]; present {
|
||||
t.Error("a priced model was counted as unpriced")
|
||||
}
|
||||
// The traffic is still recorded: "unpriced" must not mean "invisible".
|
||||
if got := st["total"].(map[string]interface{})["requests"].(float64); got != 2 {
|
||||
t.Errorf("total requests = %v, want 2 (unpriced traffic is still traffic)", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A source-only or key-only price counts as priced: any dimension covering the
|
||||
// request is enough.
|
||||
func TestBillingAnyDimensionCountsAsPriced(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"sources": map[string]interface{}{"flat-fee": map[string]interface{}{"per_request": 0.02}},
|
||||
},
|
||||
})
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "any-model", "source": "flat-fee", "key": "***p", "ok": true,
|
||||
"prompt_tokens": 10, "completion_tokens": 0, "time": 1750000000000,
|
||||
})
|
||||
st := stateOf(t, ps)
|
||||
if got := st["unpriced_reqs"].(float64); got != 0 {
|
||||
t.Errorf("unpriced_reqs = %v, want 0 (the source price covers it)", got)
|
||||
}
|
||||
approx(t, "flat fee", st["total"].(map[string]interface{})["cost"].(float64), 0.02)
|
||||
}
|
||||
|
||||
// ---- 峰谷 / 时段定价 ---------------------------------------------------
|
||||
//
|
||||
// commandcode 的 DeepSeek V4 系列就是这么定价的：非高峰 17h/天，高峰 01-04 &
|
||||
// 06-10 UTC 工作日，价格恰好 2 倍。这类规则用静态价目无法表达，而算错方向是
|
||||
// 静默的——不会报错，只会一直算错。
|
||||
//
|
||||
// 时间判据的可测性：os.date("!%H") 取 UTC 小时。测试通过选择"确定落在窗口内"
|
||||
// 与"确定落在窗口外"的时段来判定，不去伪造时钟（Lua 侧没有可注入的时钟，
|
||||
// 伪造反而会让测试与真实行为脱节）。
|
||||
|
||||
func TestBillingPeakWindowDoublesOutsidePeak(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{
|
||||
"dsv41": map[string]interface{}{
|
||||
"prompt": 1.5e-7, "completion": 6e-7,
|
||||
"peak": map[string]interface{}{
|
||||
"multiplier": 2,
|
||||
// 全部 7 天全部 24 小时 ⇒ 永远命中
|
||||
"windows": []interface{}{
|
||||
map[string]interface{}{
|
||||
"days": []interface{}{0, 1, 2, 3, 4, 5, 6},
|
||||
"hours": []interface{}{[]interface{}{0, 23}},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
// 1000 prompt + 1000 completion，非高峰 0.00075 → 命中峰谷 ×2 = 0.0015
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "dsv41", "source": "commandcode", "key": "***pk", "ok": true,
|
||||
"prompt_tokens": 1000, "completion_tokens": 1000, "time": 1750000000000,
|
||||
})
|
||||
got := stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64)
|
||||
want := (1000*1.5e-7 + 1000*6e-7) * 2
|
||||
approx(t, "always-peak cost", got, want)
|
||||
}
|
||||
|
||||
// 一个不存在的窗口（UTC 25 点不存在）⇒ 永不命中 ⇒ 静态价。
|
||||
func TestBillingPeakWindowNotHit(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{
|
||||
"dsv41": map[string]interface{}{
|
||||
"prompt": 1.5e-7, "completion": 6e-7,
|
||||
"peak": map[string]interface{}{
|
||||
"multiplier": 2,
|
||||
// 星期 = {0..6} 但小时窗写成 [99,100]：永远不可能命中
|
||||
"windows": []interface{}{
|
||||
map[string]interface{}{
|
||||
"days": []interface{}{0, 1, 2, 3, 4, 5, 6},
|
||||
"hours": []interface{}{[]interface{}{99, 100}},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "dsv41", "source": "commandcode", "key": "***pk", "ok": true,
|
||||
"prompt_tokens": 1000, "completion_tokens": 1000, "time": 1750000000000,
|
||||
})
|
||||
got := stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64)
|
||||
approx(t, "never-peak cost", got, 1000*1.5e-7+1000*6e-7)
|
||||
}
|
||||
|
||||
// 星期不匹配 ⇒ 不命中。这一条正是"用本地时区算会整体偏移"要防的东西：
|
||||
// 周日按 UTC 算，用本地时区可能算成周六而错误地命中工作日窗口。
|
||||
func TestBillingPeakWindowDayMismatch(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{
|
||||
"dsv41": map[string]interface{}{
|
||||
"prompt": 1.5e-7, "completion": 6e-7,
|
||||
"peak": map[string]interface{}{
|
||||
"multiplier": 2,
|
||||
// 只在"不存在的星期 7"上开窗（os.date %w 只到 0..6）
|
||||
"windows": []interface{}{
|
||||
map[string]interface{}{
|
||||
"days": []interface{}{7},
|
||||
"hours": []interface{}{[]interface{}{0, 23}},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "dsv41", "source": "commandcode", "key": "***pk", "ok": true,
|
||||
"prompt_tokens": 1000, "completion_tokens": 1000, "time": 1750000000000,
|
||||
})
|
||||
got := stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64)
|
||||
if got > (1000*1.5e-7+1000*6e-7)*1.5 {
|
||||
t.Errorf("cost = %v: a non-matching weekday must not trigger the peak multiplier", got)
|
||||
}
|
||||
}
|
||||
|
||||
// 没有 peak 规则的条目完全不受影响（向后兼容）。
|
||||
func TestBillingNoPeakRuleIsUnaffected(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{
|
||||
"plain": map[string]interface{}{"prompt": 1e-6, "completion": 2e-6},
|
||||
},
|
||||
},
|
||||
})
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "plain", "source": "s", "key": "***n", "ok": true,
|
||||
"prompt_tokens": 1000, "completion_tokens": 1000, "time": 1750000000000,
|
||||
})
|
||||
approx(t, "no-peak cost", stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64), 1000*1e-6+1000*2e-6)
|
||||
}
|
||||
|
||||
// 缓存读价【不】跟着峰谷翻倍：它是另一条上游费率，观测到的非峰谷价里已经含了
|
||||
// 自己的折扣，跟着翻倍会把两个折扣叠在一起。
|
||||
func TestBillingPeakDoesNotDoubleCacheRead(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
_ = ps.SetState("billing", map[string]interface{}{
|
||||
"prices": map[string]interface{}{
|
||||
"models": map[string]interface{}{
|
||||
"dsv41": map[string]interface{}{
|
||||
"prompt": 1.5e-7, "completion": 6e-7, "cache_discount": 0.02,
|
||||
"peak": map[string]interface{}{
|
||||
"multiplier": 2,
|
||||
"windows": []interface{}{map[string]interface{}{
|
||||
"days": []interface{}{0, 1, 2, 3, 4, 5, 6},
|
||||
"hours": []interface{}{[]interface{}{0, 23}},
|
||||
}},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "dsv41", "source": "commandcode", "key": "***c", "ok": true,
|
||||
"prompt_tokens": 1000000, "completion_tokens": 0,
|
||||
"cache_hit_tokens": 1000000, "time": 1750000000000,
|
||||
})
|
||||
// 全部缓存命中 ⇒ 只按 cache 价 = prompt * 0.02，且不翻倍
|
||||
got := stateOf(t, ps)["total"].(map[string]interface{})["cost"].(float64)
|
||||
approx(t, "cache-only cost (not doubled)", got, 1e6*1.5e-7*0.02)
|
||||
}
|
||||
|
||||
// TestBillingTracksCacheUsage is the guard for the gap production exposed: the
|
||||
// gateway had prompt_cache_hit_tokens and costFor() priced the cache leg, but no
|
||||
// bucket recorded the number. On a gateway where 99.88% of prompt tokens were
|
||||
// cache reads, the report showed a prompt_tokens figure with no way to tell that
|
||||
// most of it was cached.
|
||||
func TestBillingTracksCacheUsage(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m", "source": "s", "key": "k", "ok": true,
|
||||
"prompt_tokens": 1000, "completion_tokens": 50,
|
||||
"cache_hit_tokens": 900, "cache_reported": true,
|
||||
})
|
||||
// A second request from a source that does not report caching at all.
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m2", "source": "s2", "ok": true,
|
||||
"prompt_tokens": 100, "completion_tokens": 10,
|
||||
})
|
||||
|
||||
st := ps.State("billing").(map[string]interface{})
|
||||
total := st["total"].(map[string]interface{})
|
||||
if total["cache_hit_tokens"] != float64(900) {
|
||||
t.Errorf("total.cache_hit_tokens = %v, want 900", total["cache_hit_tokens"])
|
||||
}
|
||||
if total["cache_fresh_tokens"] != float64(200) {
|
||||
t.Errorf("total.cache_fresh_tokens = %v, want 200 (1000-900 + 100)", total["cache_fresh_tokens"])
|
||||
}
|
||||
// Only the first request reported a cache number.
|
||||
if total["cache_reported_reqs"] != float64(1) {
|
||||
t.Errorf("★ total.cache_reported_reqs = %v, want 1 — a source that never "+
|
||||
"reports cache usage must be distinguishable from one reporting zero hits",
|
||||
total["cache_reported_reqs"])
|
||||
}
|
||||
|
||||
// Per-source separation.
|
||||
bySrc := st["by_source"].(map[string]interface{})
|
||||
s1 := bySrc["s"].(map[string]interface{})
|
||||
if s1["cache_hit_tokens"] != float64(900) {
|
||||
t.Errorf("by_source[s].cache_hit_tokens = %v, want 900", s1["cache_hit_tokens"])
|
||||
}
|
||||
s2 := bySrc["s2"].(map[string]interface{})
|
||||
if s2["cache_reported_reqs"] != float64(0) {
|
||||
t.Errorf("by_source[s2].cache_reported_reqs = %v, want 0", s2["cache_reported_reqs"])
|
||||
}
|
||||
if s2["cache_fresh_tokens"] != float64(100) {
|
||||
t.Errorf("by_source[s2].cache_fresh_tokens = %v, want 100", s2["cache_fresh_tokens"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingCacheBucketsSurviveOlderStateFiles: a state file written before these
|
||||
// fields existed must not crash the hook. `nil + number` is an error in Lua, and
|
||||
// a hook that throws stops accounting for that request entirely — which is how a
|
||||
// billing gap turns into a silent one.
|
||||
func TestBillingCacheBucketsSurviveOlderStateFiles(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
// Simulate a state restored from an older build: buckets without the new keys.
|
||||
legacy := map[string]interface{}{
|
||||
"total": map[string]interface{}{
|
||||
"cost": 1.0, "requests": float64(5), "prompt_tokens": float64(500),
|
||||
"completion_tokens": float64(50), "failures": float64(0),
|
||||
},
|
||||
"by_source": map[string]interface{}{
|
||||
"legacy": map[string]interface{}{"cost": float64(0), "requests": float64(5),
|
||||
"prompt_tokens": float64(500), "completion_tokens": float64(50), "failures": float64(0)},
|
||||
},
|
||||
"by_model": map[string]interface{}{}, "by_key": map[string]interface{}{},
|
||||
"by_day": map[string]interface{}{}, "started": float64(0),
|
||||
}
|
||||
if err := ps.SetState("billing", legacy); err != nil {
|
||||
t.Fatalf("SetState: %v", err)
|
||||
}
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m", "source": "legacy", "ok": true,
|
||||
"prompt_tokens": 100, "completion_tokens": 10,
|
||||
"cache_hit_tokens": 60, "cache_reported": true,
|
||||
})
|
||||
if len(ps.HookErrors()) != 0 {
|
||||
t.Fatalf("hook error on a legacy state: %v", ps.HookErrors())
|
||||
}
|
||||
st := ps.State("billing").(map[string]interface{})
|
||||
tot := st["total"].(map[string]interface{})
|
||||
if tot["requests"] != float64(6) {
|
||||
t.Errorf("requests = %v, want 6 (5 legacy + 1 new)", tot["requests"])
|
||||
}
|
||||
if tot["cache_hit_tokens"] != float64(60) {
|
||||
t.Errorf("cache_hit_tokens = %v, want 60", tot["cache_hit_tokens"])
|
||||
}
|
||||
lg := st["by_source"].(map[string]interface{})["legacy"].(map[string]interface{})
|
||||
if lg["cache_hit_tokens"] != float64(60) {
|
||||
t.Errorf("legacy bucket cache_hit_tokens = %v, want 60", lg["cache_hit_tokens"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingCacheHitClampedInStats: costFor clamps the cache leg, so the
|
||||
// recorded numbers must be clamped the same way. A provider that reports more
|
||||
// cache hits than prompt tokens must not produce negative fresh tokens.
|
||||
func TestBillingCacheHitClampedInStats(t *testing.T) {
|
||||
ps, _ := billingVM(t)
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{
|
||||
"model": "m", "source": "s", "ok": true,
|
||||
"prompt_tokens": 100, "completion_tokens": 5,
|
||||
"cache_hit_tokens": 5000, "cache_reported": true,
|
||||
})
|
||||
st := ps.State("billing").(map[string]interface{})
|
||||
tot := st["total"].(map[string]interface{})
|
||||
if tot["cache_hit_tokens"] != float64(100) {
|
||||
t.Errorf("★ cache_hit_tokens = %v, want 100 (clamped to prompt_tokens)",
|
||||
tot["cache_hit_tokens"])
|
||||
}
|
||||
if tot["cache_fresh_tokens"] != float64(0) {
|
||||
t.Errorf("★ cache_fresh_tokens = %v, want 0, never negative",
|
||||
tot["cache_fresh_tokens"])
|
||||
}
|
||||
}
|
||||
@ -1,416 +0,0 @@
|
||||
package lua
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"os/exec"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The billing page rendered EMPTY in production while its data endpoint returned
|
||||
// 200 with real numbers. The cause was one line: render(st) referenced an
|
||||
// undefined `s` for two KPI cells, so the ReferenceError aborted the whole
|
||||
// render and every table stayed at its initial empty innerHTML.
|
||||
//
|
||||
// Nothing in the build, the tests or the API surfaced it. This file is the guard
|
||||
// for the whole class: a plugin's injected UI that references an undefined name,
|
||||
// or that depends on a container the page does not provide, fails silently.
|
||||
|
||||
// billingUI returns the injected markup for the billing plugin: the full page
|
||||
// mount and the status-page element mount.
|
||||
func billingUI(t *testing.T) (page string, statusElement string) {
|
||||
t.Helper()
|
||||
src, err := os.ReadFile("plugins/billing.lua")
|
||||
if err != nil {
|
||||
t.Fatalf("read billing.lua: %v", err)
|
||||
}
|
||||
// Located BY CONTENT, not by index. The plugin also uses a long string for
|
||||
// its inline SVG icon, so "the first long string" is the icon and "the
|
||||
// second" is the page — which is exactly the kind of positional assumption
|
||||
// that breaks the next time an icon or a description is added.
|
||||
page = longStringContaining(t, string(src), "billing-root")
|
||||
statusElement = longStringContaining(t, string(src), "billing-status-tile")
|
||||
return page, statusElement
|
||||
}
|
||||
|
||||
// longStringContaining returns the [==[ ... ]==] body that contains marker.
|
||||
func longStringContaining(t *testing.T, src, marker string) string {
|
||||
t.Helper()
|
||||
re := regexp.MustCompile(`(?s)\[==\[(.*?)\]==\]`)
|
||||
for _, m := range re.FindAllStringSubmatch(src, -1) {
|
||||
if strings.Contains(m[1], marker) {
|
||||
return m[1]
|
||||
}
|
||||
}
|
||||
t.Fatalf("no long string contains %q", marker)
|
||||
return ""
|
||||
}
|
||||
|
||||
// TestBillingMountScriptExecutes is the guard for the production bug.
|
||||
//
|
||||
// The Billing page rendered empty while its data endpoint returned 200 with real
|
||||
// numbers. Cause: render(st) referenced an undefined `s` for two KPI cells, the
|
||||
// ReferenceError aborted the render, and every table kept its initial empty
|
||||
// innerHTML. Nothing in the build or the API surfaced it.
|
||||
//
|
||||
// Two earlier attempts at a static check were both wrong: a "declared names"
|
||||
// scan flagged every CSS class inside the inline HTML strings (class, div, td),
|
||||
// and a CSS-selector parse of the stylesheet reported the stylesheet itself as
|
||||
// broken. Static analysis of JS embedded in HTML strings is the wrong tool.
|
||||
//
|
||||
// So this actually RUNS the script, in node, against a minimal DOM stub, and
|
||||
// fails on any thrown error. Skipped when node is unavailable, with the reason
|
||||
// printed — never silently passing as if it had checked.
|
||||
func TestBillingMountScriptExecutes(t *testing.T) {
|
||||
page, el := billingUI(t)
|
||||
scripts := extractScripts(page)
|
||||
if len(scripts) == 0 {
|
||||
t.Fatal("no <script> found in the billing page mount")
|
||||
}
|
||||
for i, js := range scripts {
|
||||
assertRendersAndDoesNotThrow(t, i, js)
|
||||
}
|
||||
for i, js := range extractScripts(el) {
|
||||
assertRendersAndDoesNotThrow(t, i, js)
|
||||
}
|
||||
}
|
||||
|
||||
// assertRendersAndDoesNotThrow executes a mount script under node against a DOM
|
||||
// stub and fails on EITHER a thrown/reported error OR an empty render.
|
||||
//
|
||||
// Checking only for exceptions is not enough, and that is the third wrong
|
||||
// attempt at this guard. The plugin's render() opens with `if (!st) return;`,
|
||||
// so passing the wrong field (`render(j.stateX)`) is a SILENT no-op: no throw,
|
||||
// no console.error, no rejection — just an empty page. Only looking at the
|
||||
// produced DOM catches that class.
|
||||
func assertRendersAndDoesNotThrow(t *testing.T, idx int, js string) {
|
||||
t.Helper()
|
||||
node, err := exec.LookPath("node")
|
||||
if err != nil {
|
||||
t.Skipf("node not available (%v): cannot execute the injected script", err)
|
||||
}
|
||||
|
||||
stub := `
|
||||
global.window = global;
|
||||
global.document = {
|
||||
getElementById: function (id) {
|
||||
if (!global.__els) global.__els = {};
|
||||
if (!global.__els[id]) global.__els[id] = {
|
||||
style: {}, dataset: {}, classList: { add: function(){}, remove: function(){} },
|
||||
// Both writes count: the Billing page fills innerHTML, the status-page
|
||||
// tile assigns textContent. Watching only one of them flagged the tile as
|
||||
// "renders nothing" — a false positive that would have taught everyone to
|
||||
// ignore this test.
|
||||
set innerHTML(v) { if (v && String(v).trim()) global.__written.push(id); this.__h = v; },
|
||||
get innerHTML() { return this.__h || ""; },
|
||||
set textContent(v) { if (v !== undefined && String(v).trim()) global.__written.push(id); this.__t = v; },
|
||||
get textContent() { return this.__t || ""; },
|
||||
set innerText(v) { if (v !== undefined && String(v).trim()) global.__written.push(id); this.__i = v; },
|
||||
get innerText() { return this.__i || ""; },
|
||||
appendChild: function(){}, querySelector: function(){ return null; },
|
||||
querySelectorAll: function(){ return []; }, addEventListener: function(){} };
|
||||
return global.__els[id];
|
||||
},
|
||||
createElement: function () { return { style: {}, dataset: {}, appendChild: function(){}, setAttribute: function(){} }; },
|
||||
addEventListener: function () {},
|
||||
};
|
||||
global.pluginAPI = { onTabShown: function () {} };
|
||||
global.fetch = function () {
|
||||
// A payload with real numbers, so a working render produces visible output.
|
||||
return Promise.resolve({ ok: true, json: function () {
|
||||
return Promise.resolve({ plugin: "billing", state: {
|
||||
currency: "USD",
|
||||
total: { cost: 1.25, requests: 7, prompt_tokens: 100, completion_tokens: 20, failures: 0 },
|
||||
by_source: { localzen: { cost: 1.25, requests: 7, prompt_tokens: 100, completion_tokens: 20, failures: 0 } },
|
||||
by_model: { m1: { cost: 1.25, requests: 7, prompt_tokens: 100, completion_tokens: 20, failures: 0 } },
|
||||
by_key: { k1: { cost: 1.25, requests: 7, prompt_tokens: 100, completion_tokens: 20, failures: 0 } },
|
||||
by_day: { "2026-01-01": { cost: 1.25, requests: 7, prompt_tokens: 100, completion_tokens: 20, failures: 0 } },
|
||||
degraded_reqs: 1, unpriced_reqs: 2,
|
||||
}});
|
||||
}});
|
||||
};
|
||||
global.__written = [];
|
||||
global.__errors = [];
|
||||
// The plugin's refresh() swallows its own errors so decoration can never break
|
||||
// the host page — which is right, and is exactly why the production ReferenceError
|
||||
// was invisible. So the trace has to come from console.error, which the plugin now
|
||||
// emits. Hooking process events alone made this test pass against the very bug it
|
||||
// was written for (verified by re-introducing the typo and watching it stay green).
|
||||
var __realErr = console.error;
|
||||
console.error = function () {
|
||||
global.__errors.push("console.error: " + Array.prototype.map.call(arguments, function (a) {
|
||||
return (a && a.message) ? a.message : String(a);
|
||||
}).join(" "));
|
||||
__realErr.apply(console, arguments);
|
||||
};
|
||||
process.on("uncaughtException", function (e) { global.__errors.push("uncaught: " + String(e && e.message || e)); });
|
||||
process.on("unhandledRejection", function (e) { global.__errors.push("unhandled: " + String(e && e.message || e)); });
|
||||
`
|
||||
// 50ms is not a guess: the plugin's IIFE kicks off refresh() which awaits a
|
||||
// fetch; a rejection lands on the microtask queue almost immediately.
|
||||
script := strings.Join([]string{
|
||||
stub, js,
|
||||
`setTimeout(function(){
|
||||
console.log("__ERRS__" + JSON.stringify({errors: global.__errors, written: global.__written}));
|
||||
}, 300);`,
|
||||
}, "\n")
|
||||
cmd := exec.Command(node, "-e", script)
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Errorf("mount script %d crashed under node: %v\n%s", idx, err, out)
|
||||
return
|
||||
}
|
||||
// The script may print other things; find the JSON we appended.
|
||||
line := ""
|
||||
for _, l := range strings.Split(string(out), "\n") {
|
||||
if i := strings.Index(l, "__ERRS__"); i >= 0 {
|
||||
line = strings.TrimSpace(l[i+len("__ERRS__"):])
|
||||
}
|
||||
}
|
||||
if line == "" {
|
||||
t.Errorf("mount script %d produced no error report — the harness did not "+
|
||||
"run to completion, so it cannot be trusted to have checked anything", idx)
|
||||
return
|
||||
}
|
||||
var report struct {
|
||||
Errors []string `json:"errors"`
|
||||
Written []string `json:"written"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(line), &report); err != nil {
|
||||
t.Errorf("could not parse the error report %q: %v", line, err)
|
||||
return
|
||||
}
|
||||
for _, e := range report.Errors {
|
||||
t.Errorf("mount script %d reported %q — this is the failure that leaves "+
|
||||
"the Billing page blank while the API still returns data", idx, e)
|
||||
}
|
||||
if len(report.Written) == 0 {
|
||||
t.Errorf("mount script %d wrote NOTHING into any container — the page "+
|
||||
"renders empty. render() guards with `if (!st) return`, so a wrong "+
|
||||
"field name is a silent no-op: no throw, no console output, no error "+
|
||||
"anywhere. This is what the production page looked like.", idx)
|
||||
}
|
||||
}
|
||||
|
||||
func extractScripts(html string) []string {
|
||||
re := regexp.MustCompile(`(?s)<script[^>]*>(.*?)</script>`)
|
||||
var out []string
|
||||
for _, m := range re.FindAllStringSubmatch(html, -1) {
|
||||
out = append(out, m[1])
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// TestBillingPageCoversItsData is the other half: the page declares tables for
|
||||
// per-source / per-model / per-key / per-day and must actually render into them.
|
||||
// A table id that is never written to is exactly how "the page loads and shows
|
||||
// nothing" happens without an error.
|
||||
func TestBillingPageCoversItsData(t *testing.T) {
|
||||
page, _ := billingUI(t)
|
||||
for _, id := range []string{
|
||||
"billing-kpis", "billing-by-source", "billing-by-model",
|
||||
"billing-by-key", "billing-by-day",
|
||||
} {
|
||||
if !strings.Contains(page, `id="`+id+`"`) {
|
||||
t.Errorf("the page has no container #%s", id)
|
||||
}
|
||||
}
|
||||
// Every container must be written to by the script, not just declared.
|
||||
scripts := extractScripts(page)
|
||||
all := strings.Join(scripts, "\n")
|
||||
for _, id := range []string{"billing-kpis", "billing-by-source", "billing-by-model", "billing-by-key", "billing-by-day"} {
|
||||
if !strings.Contains(all, `getElementById("`+id+`")`) {
|
||||
t.Errorf("#%s is declared but never read by the script — it stays empty forever", id)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginIconIsNotARawEmoji guards the sidebar icon. The billing plugin
|
||||
// declared icon = "💰" and the WebUI drops that verbatim into the button, while
|
||||
// every native tab uses an inline SVG styled with `stroke: currentColor`. An
|
||||
// emoji there renders at the wrong size and ignores the theme, so it does not
|
||||
// match its neighbours — which is what the operator reported.
|
||||
func TestPluginIconIsNotARawEmoji(t *testing.T) {
|
||||
icon := billingIcon(mustBillingSource(t))
|
||||
if icon == "" {
|
||||
t.Fatal("billing declares no icon; the sidebar entry would be blank")
|
||||
}
|
||||
if isEmojiIcon(icon) {
|
||||
t.Errorf("page icon is the raw emoji %q — the WebUI sidebar renders "+
|
||||
"native tabs as inline SVG (stroke: currentColor), so an emoji is the "+
|
||||
"wrong size and ignores the theme. Use an inline SVG path instead.", icon)
|
||||
}
|
||||
}
|
||||
|
||||
// isEmojiIcon reports whether s is a pictographic emoji rather than markup or a
|
||||
// text glyph. Codepoints in the pictographic blocks, plus the regional-indicator
|
||||
// pair used by flags.
|
||||
func isEmojiIcon(s string) bool {
|
||||
r := []rune(s)
|
||||
if len(r) == 0 {
|
||||
return false
|
||||
}
|
||||
// Anything containing '<' is markup (an inline <svg>), which is the fix.
|
||||
if strings.ContainsRune(s, '<') {
|
||||
return false
|
||||
}
|
||||
for _, c := range r {
|
||||
switch {
|
||||
case c >= 0x1F300 && c <= 0x1FAFF, // pictographs, symbols, supplemental
|
||||
c >= 0x1F000 && c <= 0x1F2FF, // mahjong/domino/cards
|
||||
c >= 0x2600 && c <= 0x27BF, // misc symbols + dingbats
|
||||
c >= 0x2B00 && c <= 0x2BFF, // arrows/misc symbols
|
||||
c == 0xFE0F, // variation selector-16
|
||||
c >= 0x1F1E6 && c <= 0x1F1FF: // regional indicators (flags)
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func mustBillingSource(t *testing.T) string {
|
||||
t.Helper()
|
||||
b, err := os.ReadFile("plugins/billing.lua")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
|
||||
// billingIcon extracts the declared page icon. It accepts BOTH a quoted string
|
||||
// and a [==[ ... ]==] long string, because an inline SVG cannot be written as a
|
||||
// Lua short string without escaping every quote in it.
|
||||
func billingIcon(src string) string {
|
||||
if m := regexp.MustCompile(`(?m)^\s*icon\s*=\s*"([^"]*)"`).FindStringSubmatch(src); m != nil {
|
||||
return m[1]
|
||||
}
|
||||
if m := regexp.MustCompile(`(?s)\bicon\s*=\s*\[==\[(.*?)\]==\]`).FindStringSubmatch(src); m != nil {
|
||||
return m[1]
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// TestBillingTableHeaderMatchesRowColumns catches column drift.
|
||||
//
|
||||
// row() gained cache columns (fresh / cache / cache%) while the header row did
|
||||
// not, in the same edit. The result is a table whose cells are shifted one
|
||||
// column left from "fresh" onward — so "cache%" sits under "completion" and the
|
||||
// last cell has no label. It renders, it has data, and it is wrong in a way that
|
||||
// takes a careful read to notice.
|
||||
func TestBillingTableHeaderMatchesRowColumns(t *testing.T) {
|
||||
page, _ := billingUI(t)
|
||||
js := strings.Join(extractScripts(page), "\n")
|
||||
if !strings.Contains(js, "<th>") {
|
||||
t.Fatal("no table header found in the billing page script")
|
||||
}
|
||||
hStart := strings.Index(js, "function tableFor")
|
||||
if hStart < 0 {
|
||||
t.Fatal("no tableFor in the billing page script")
|
||||
}
|
||||
// Count the usage table's headers ONLY. The count is taken from tableFor to
|
||||
// the end of the script, which used to be fine when the rule editor lived on
|
||||
// its own page; now that both halves share one script, the rule table's nine
|
||||
// headers were counted too and the check reported 17 vs 8 — a failure about
|
||||
// two unrelated tables. Stop at the rule editor's script.
|
||||
header := js[hStart:]
|
||||
if cut := strings.Index(header, "/api/plugins/billing/rules"); cut > 0 {
|
||||
header = header[:cut]
|
||||
}
|
||||
// Count <th> with OR without attributes: the name column now carries
|
||||
// style='width:22%' so the header and its data cell can be given the same
|
||||
// width, and a bare "<th>" count silently reported 7 vs 8 — the check
|
||||
// failing on the very column it had just been taught to size.
|
||||
th := strings.Count(header, "<th>") + strings.Count(header, "<th ")
|
||||
|
||||
rStart := strings.Index(js, "function row")
|
||||
rEnd := strings.Index(js, "function tableFor")
|
||||
if rStart < 0 || rEnd < 0 || rEnd <= rStart {
|
||||
t.Fatal("could not isolate row()")
|
||||
}
|
||||
row := js[rStart:rEnd]
|
||||
// Each cell closes with </td>; the first cell uses <td><b>..</b></td> so
|
||||
// counting </td> is exact.
|
||||
td := strings.Count(row, "</td>")
|
||||
|
||||
if th != td {
|
||||
t.Errorf("★ header declares %d columns but row() emits %d cells — the "+
|
||||
"table is misaligned from the first differing column on", th, td)
|
||||
}
|
||||
}
|
||||
|
||||
// TestRuleTableLocksColumnWidthsInAColgroup guards the layout class of bug that
|
||||
// only a screenshot caught: two adjacent headers rendered on top of each other,
|
||||
// the URL input squeezed to "https:", and the per-model × button landing 78px
|
||||
// past its cell onto the neighbouring Delete button.
|
||||
//
|
||||
// The measurement that found each one:
|
||||
// - table-layout:fixed with widths on <td> → 954px divided into 9 equal
|
||||
// 106px columns, every declared width discarded.
|
||||
// - widths on <th> WITHOUT fixed → browser sizes from content and ignores
|
||||
// them; the 70px currency column still collapsed to 48px (24px of input).
|
||||
// - fixed + <th> widths → they take effect, but the model-price cell's width
|
||||
// then depended on WHICH RULE was widest (352px vs 206px).
|
||||
//
|
||||
// The fix is all three together: fixed layout, a colgroup, and flex children
|
||||
// that may shrink.
|
||||
func TestRuleTableLocksColumnWidthsInAColgroup(t *testing.T) {
|
||||
page, _ := billingUI(t)
|
||||
js := strings.Join(extractScripts(page), "\n")
|
||||
// Match the ATTRIBUTE, not the bare string: a substring check on
|
||||
// "table-layout:fixed" is satisfied by the explanatory comment that sits
|
||||
// three lines above the tag, so removing the attribute from the table still
|
||||
// passed (mutation-verified). Require it inside a style='...' literal.
|
||||
if !strings.Contains(js, `font-size:12px;table-layout:fixed'`) {
|
||||
t.Error("the rule table must stay table-layout:fixed — with auto layout " +
|
||||
"the <colgroup> widths below are ignored and columns size from content")
|
||||
}
|
||||
colStart := strings.Index(js, "<colgroup>")
|
||||
if colStart < 0 {
|
||||
t.Fatal("the rule table declares no <colgroup>; per-column widths on " +
|
||||
"<th> or <td> are advisory and get recomputed from content")
|
||||
}
|
||||
// The colgroup is built from a JS array of pixel widths, so assert on the
|
||||
// array, not on literal "<col " occurrences — counting the literal found
|
||||
// exactly one and reported "1 columns, want 9" against correct code.
|
||||
if !strings.Contains(js, "[214, 80, 62, 70, 66, 74, 62, 262, 74]") {
|
||||
t.Error("the rule table's colgroup widths changed or vanished; measured " +
|
||||
"values were [214 80 62 70 66 74 62 262 74] for a 954px pane — the " +
|
||||
"delete column needs 74px or the button wraps to two lines")
|
||||
}
|
||||
if !strings.Contains(js, `"<col style='width:" + w + "px'>"`) {
|
||||
t.Error("colgroup entries are no longer emitted as <col style='width:Npx'>")
|
||||
}
|
||||
// Every price-row INPUT must be allowed to shrink, or the row's fixed widths
|
||||
// sum past the cell and the × button lands on the next column. The × button
|
||||
// is the opposite: it must NOT shrink (flex:0 0 auto), because a squashed
|
||||
// delete button is unclickable.
|
||||
for _, needle := range []string{"m-name", "m-p", "m-c"} {
|
||||
i := strings.Index(js, "class='"+needle+"'")
|
||||
if i < 0 {
|
||||
i = strings.Index(js, "class='ghost small "+needle+"'")
|
||||
}
|
||||
if i < 0 {
|
||||
t.Errorf("price row lost the %s input/button", needle)
|
||||
continue
|
||||
}
|
||||
window := js[i : i+220]
|
||||
if !strings.Contains(window, "flex:") {
|
||||
t.Errorf("%s has no flex sizing; the row is 80+96+96+38 ≈ 330px wide "+
|
||||
"against a 262px cell and overflows onto the next column", needle)
|
||||
}
|
||||
if !strings.Contains(window, "min-width:0") {
|
||||
t.Errorf("%s lacks min-width:0; a flex item will not shrink below its "+
|
||||
"content width, which is how the × button ended up 78px past its cell", needle)
|
||||
}
|
||||
}
|
||||
i := strings.Index(js, "class='ghost small m-del'")
|
||||
if i < 0 {
|
||||
t.Fatal("the per-model delete button is gone")
|
||||
}
|
||||
if !strings.Contains(js[i:i+120], "flex:0 0 auto") {
|
||||
t.Error("the per-model × button must be flex:0 0 auto — it is a fixed-size " +
|
||||
"control and must never be squeezed by the flexible inputs")
|
||||
}
|
||||
}
|
||||
@ -1,139 +0,0 @@
|
||||
package lua
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
// This file exists to remove the JSON round-trip from the plugin hook path.
|
||||
//
|
||||
// Measured before this change: one Fire() call cost 14.6us, of which
|
||||
// json.Marshal(payload) was 4.0us and json.Unmarshal back into interface{} was
|
||||
// 5.6us — 67% of the whole hook, spent re-deriving a tree that the very next
|
||||
// line (pushGoValue) already knows how to walk natively.
|
||||
//
|
||||
// The round-trip existed because payload arrives as map[string]interface{} and
|
||||
// pushGoValue wanted a plain interface{} tree. pushGoValue already handles
|
||||
// map[string]interface{} and []interface{} directly, so the fix is a converter
|
||||
// that flattens the concrete Go types the gateway actually produces, instead of
|
||||
// a generic serialize/parse.
|
||||
//
|
||||
// The JSON path is kept as the fallback for types the fast path does not know,
|
||||
// so a plugin payload carrying something exotic still arrives rather than being
|
||||
// silently dropped.
|
||||
|
||||
// jsonNumberType is the type json.Unmarshal produces for every JSON number.
|
||||
// The fast path recognizes it so numbers survive as numbers rather than being
|
||||
// stringified into a different Lua type.
|
||||
type jsonNumberType = float64
|
||||
|
||||
// fastToPlain converts the concrete Go values the gateway puts in a hook
|
||||
// payload into the interface{} tree pushGoValue consumes, without JSON.
|
||||
//
|
||||
// The interesting cases are the ones json.Unmarshal would have normalized:
|
||||
// - json.Number-ish types (all float64 in practice)
|
||||
// - int / int64 / uint variants, which must land as Lua numbers, not as the
|
||||
// strings a naive "only handle float64" switch would produce
|
||||
// - []string and map[string]string, extremely common in payloads and NOT
|
||||
// handled by a switch that only knows []interface{}
|
||||
//
|
||||
// Anything unrecognized returns (value, false) so the caller can fall back to
|
||||
// the JSON path, which is slower but total.
|
||||
func fastToPlain(v interface{}) (interface{}, bool) {
|
||||
switch x := v.(type) {
|
||||
case nil, bool, string, jsonNumberType:
|
||||
return v, true
|
||||
case int:
|
||||
return float64(x), true
|
||||
case int8:
|
||||
return float64(x), true
|
||||
case int16:
|
||||
return float64(x), true
|
||||
case int32:
|
||||
return float64(x), true
|
||||
case int64:
|
||||
return float64(x), true
|
||||
case uint:
|
||||
return float64(x), true
|
||||
case uint8:
|
||||
return float64(x), true
|
||||
case uint16:
|
||||
return float64(x), true
|
||||
case uint32:
|
||||
return float64(x), true
|
||||
case uint64:
|
||||
return float64(x), true
|
||||
case float32:
|
||||
return float64(x), true
|
||||
case []string:
|
||||
out := make([]interface{}, len(x))
|
||||
for i, s := range x {
|
||||
out[i] = s
|
||||
}
|
||||
return out, true
|
||||
case map[string]string:
|
||||
out := make(map[string]interface{}, len(x))
|
||||
for k, s := range x {
|
||||
out[k] = s
|
||||
}
|
||||
return out, true
|
||||
case map[string]interface{}:
|
||||
// The overwhelmingly common case, and the one the hook path always
|
||||
// takes at the top level.
|
||||
out := make(map[string]interface{}, len(x))
|
||||
for k, it := range x {
|
||||
c, ok := fastToPlain(it)
|
||||
if !ok {
|
||||
return nil, false
|
||||
}
|
||||
out[k] = c
|
||||
}
|
||||
return out, true
|
||||
case []interface{}:
|
||||
out := make([]interface{}, len(x))
|
||||
for i, it := range x {
|
||||
c, ok := fastToPlain(it)
|
||||
if !ok {
|
||||
return nil, false
|
||||
}
|
||||
out[i] = c
|
||||
}
|
||||
return out, true
|
||||
default:
|
||||
return nil, false
|
||||
}
|
||||
}
|
||||
|
||||
// plainForLua returns the interface{} tree to hand to pushGoValue, using the
|
||||
// fast path when it can and JSON only when it must.
|
||||
func plainForLua(payload map[string]interface{}) interface{} {
|
||||
if fast, ok := fastToPlain(payload); ok {
|
||||
return fast
|
||||
}
|
||||
// Rare: some type the fast path does not model. Serialize and let the
|
||||
// generic decoder normalize it, so the hook still sees the field.
|
||||
b, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return payload
|
||||
}
|
||||
var decoded interface{}
|
||||
if err := json.Unmarshal(b, &decoded); err != nil {
|
||||
return payload
|
||||
}
|
||||
return decoded
|
||||
}
|
||||
|
||||
// toPlainSlice is the []interface{} entry point of fastToPlain, exposed so the
|
||||
// Fire path can flatten a pre-built slice without re-walking the map header.
|
||||
func toPlainSlice(v []interface{}) []interface{} {
|
||||
if fast, ok := fastToPlain(v); ok {
|
||||
return fast.([]interface{})
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
// luaNumberString renders a float the way Lua would print it, for the rare case
|
||||
// a hook wants the textual form. Kept out of the hot path.
|
||||
func luaNumberString(f float64) string {
|
||||
return strconv.FormatFloat(f, 'g', -1, 64)
|
||||
}
|
||||
@ -1,132 +0,0 @@
|
||||
package lua
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The hook payload used to be JSON round-tripped on every call. It no longer is,
|
||||
// so the fast path and the old JSON path must be indistinguishable — otherwise
|
||||
// a plugin silently sees a different payload than before, which is the worst
|
||||
// kind of change: it compiles, passes a smoke test, and misprices traffic.
|
||||
//
|
||||
// These tests therefore compare the two paths on the SAME inputs rather than
|
||||
// asserting the fast path's output in isolation.
|
||||
|
||||
func viaJSON(t *testing.T, payload map[string]interface{}) interface{} {
|
||||
t.Helper()
|
||||
b, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal: %v", err)
|
||||
}
|
||||
var decoded interface{}
|
||||
if err := json.Unmarshal(b, &decoded); err != nil {
|
||||
t.Fatalf("unmarshal: %v", err)
|
||||
}
|
||||
return decoded
|
||||
}
|
||||
|
||||
// payloads that exercise every branch of the fast converter.
|
||||
func payloadCases() []map[string]interface{} {
|
||||
return []map[string]interface{}{
|
||||
{"model": "deepseek-v4.1-flash", "source": "commandcode", "ok": true},
|
||||
// Numbers of every width: a converter that only knows float64 turns
|
||||
// ints into something else, and a token count that arrives as a string
|
||||
// makes a Lua hook do arithmetic on nil.
|
||||
{"i": 42, "i8": int8(8), "i16": int16(16), "i32": int32(32), "i64": int64(1 << 40),
|
||||
"u": uint(7), "u64": uint64(1 << 50), "f32": float32(1.5), "f64": 2.25},
|
||||
// Zero, negative, and very large values must stay numbers.
|
||||
{"zero": 0, "neg": -17, "huge": 1e308, "tiny": 1e-308},
|
||||
// Slices and string maps: common in payloads and absent from a switch
|
||||
// that only knows []interface{} / map[string]interface{}.
|
||||
{"msgs": []interface{}{"a", "b"}, "tags": []string{"x", "y"}},
|
||||
{"kv": map[string]string{"a": "1", "b": "2"}},
|
||||
// Nesting, which is where a shallow converter silently drops a level.
|
||||
{"usage": map[string]interface{}{
|
||||
"prompt_tokens": 2048, "cache_hit_tokens": 1024,
|
||||
"nested": map[string]interface{}{"deep": []interface{}{1, "two", true, nil}},
|
||||
}},
|
||||
{"nil_field": nil, "empty_map": map[string]interface{}{}, "empty_slice": []interface{}{}},
|
||||
// A value the fast path does NOT model: it must fall back to JSON and
|
||||
// still arrive, not disappear.
|
||||
{"weird": struct {
|
||||
A int `json:"a"`
|
||||
B string `json:"b"`
|
||||
}{1, "x"}},
|
||||
}
|
||||
}
|
||||
|
||||
func TestFastPathMatchesJSONPath(t *testing.T) {
|
||||
for i, p := range payloadCases() {
|
||||
want := viaJSON(t, p)
|
||||
got := plainForLua(p)
|
||||
if !reflect.DeepEqual(want, got) {
|
||||
t.Errorf("case %d: fast path differs from JSON path\n payload: %#v\n json: %#v\n fast: %#v",
|
||||
i, p, want, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestFastPathIsActuallyUsed guards against the fast path silently degrading to
|
||||
// JSON for the payload the gateway really sends. If a future payload gains a
|
||||
// type the converter does not model, this still works (it falls back) but the
|
||||
// optimization is gone — and the next person measuring the hook would be
|
||||
// measuring the old cost without knowing why.
|
||||
func TestFastPathIsActuallyUsed(t *testing.T) {
|
||||
// This mirrors the real request_end payload shape from the gateway.
|
||||
realistic := map[string]interface{}{
|
||||
"stage": "request_end", "kind": "end", "model": "deepseek-v4.1-flash",
|
||||
"source": "commandcode", "key": "stress-key", "ok": true,
|
||||
"status": 200, "duration_ms": 1234,
|
||||
"usage": map[string]interface{}{
|
||||
"prompt_tokens": float64(2048), "completion_tokens": float64(512),
|
||||
"cache_hit_tokens": float64(1024), "total_tokens": float64(3584),
|
||||
},
|
||||
"walk": []interface{}{
|
||||
map[string]interface{}{"kind": "tier_skip", "tier": 1, "source": "", "model": "", "reason": "cooldown"},
|
||||
map[string]interface{}{"kind": "selected", "tier": 2, "source": "commandcode", "model": "m", "reason": ""},
|
||||
},
|
||||
"ts": float64(1700000000),
|
||||
}
|
||||
if _, ok := fastToPlain(realistic); !ok {
|
||||
t.Errorf("★ the realistic request payload does NOT take the fast path — " +
|
||||
"the JSON round-trip is still on the hot path for real traffic")
|
||||
}
|
||||
}
|
||||
|
||||
// TestFastPathDoesNotAliasInput: the converter builds a new tree. If it ever
|
||||
// returned the caller's map directly, a Lua hook's writes could not reach Go —
|
||||
// but worse, a later mutation of the payload would race with the snapshot the
|
||||
// persistence saver is holding.
|
||||
func TestFastPathDoesNotAliasInput(t *testing.T) {
|
||||
src := map[string]interface{}{
|
||||
"usage": map[string]interface{}{"prompt_tokens": float64(1)},
|
||||
"list": []interface{}{"a"},
|
||||
}
|
||||
out, ok := fastToPlain(src)
|
||||
if !ok {
|
||||
t.Fatal("fast path declined a plain payload")
|
||||
}
|
||||
m := out.(map[string]interface{})
|
||||
m["new"] = "added"
|
||||
src["also_new"] = "must not appear"
|
||||
|
||||
if _, leaked := m["also_new"]; leaked {
|
||||
t.Error("output map aliases the input map")
|
||||
}
|
||||
if _, leaked := src["new"]; leaked {
|
||||
t.Error("writing to the output mutated the input")
|
||||
}
|
||||
// And the nested maps must be copies too.
|
||||
inner := m["usage"].(map[string]interface{})
|
||||
inner["prompt_tokens"] = float64(999)
|
||||
if src["usage"].(map[string]interface{})["prompt_tokens"] != float64(1) {
|
||||
t.Error("nested map is shared, not copied — a hook could mutate the payload")
|
||||
}
|
||||
list := m["list"].([]interface{})
|
||||
list[0] = "changed"
|
||||
if src["list"].([]interface{})[0] != "a" {
|
||||
t.Error("nested slice is shared, not copied")
|
||||
}
|
||||
}
|
||||
@ -1,334 +0,0 @@
|
||||
package lua
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Fire runs same-stage hooks in parallel across plugins. These tests pin the
|
||||
// three things that make that safe, each of which failed at least once while
|
||||
// the change was being written.
|
||||
|
||||
// twoHookPlugins loads two plugins that both hook request_end.
|
||||
func twoHookPlugins(t *testing.T, codeA, codeB string) *Plugins {
|
||||
t.Helper()
|
||||
vm := NewVM(filepath.Join(t.TempDir(), "adapters"))
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatalf("vm start: %v", err)
|
||||
}
|
||||
t.Cleanup(vm.Stop)
|
||||
ps := NewPlugins(vm, filepath.Join(t.TempDir(), "plugins"))
|
||||
ps.save.setWakeDelayForTest(time.Millisecond)
|
||||
t.Cleanup(ps.Close)
|
||||
ps.DisableStatePersistence()
|
||||
if err := ps.LoadSource("aaa", codeA); err != nil {
|
||||
t.Fatalf("load aaa: %v", err)
|
||||
}
|
||||
if err := ps.LoadSource("bbb", codeB); err != nil {
|
||||
t.Fatalf("load bbb: %v", err)
|
||||
}
|
||||
return ps
|
||||
}
|
||||
|
||||
func counterPlugin(name string, key string) string {
|
||||
// Built by concatenation rather than fmt: %q is not a Go verb, and the first
|
||||
// attempt produced Lua source with a stray '=' that only failed at compile
|
||||
// time inside three different tests.
|
||||
return `
|
||||
local p = { name = "` + name + `" }
|
||||
p.state = { n = 0 }
|
||||
p.hooks = { request_end = "bump" }
|
||||
function p.bump(payload)
|
||||
p.state.n = p.state.n + 1
|
||||
if "` + key + `" ~= "" then return { who = "from-` + name + `" } end
|
||||
return nil
|
||||
end
|
||||
return p
|
||||
`
|
||||
}
|
||||
|
||||
// TestSingleAndParallelPathsMergeIdentically: the single-plugin fast path and
|
||||
// the multi-plugin parallel path must produce the same payload.
|
||||
//
|
||||
// This is not theoretical. The parallel path was written first and the
|
||||
// single-plugin path kept its old shape; the merge loop was left off the fast
|
||||
// path, so a lone plugin returning a table had its return value DISCARDED. The
|
||||
// existing stage-order test caught it — but only because it happened to check
|
||||
// the payload after Fire. A plugin that returned fields nobody read would have
|
||||
// broken silently.
|
||||
func TestSingleAndParallelPathsMergeIdentically(t *testing.T) {
|
||||
single := twoHookPlugins(t, counterPlugin("aaa", "who"), `
|
||||
local p = { name = "zzz" }
|
||||
p.state = { n = 0 }
|
||||
p.hooks = { request_end = "bump" }
|
||||
function p.bump(payload) p.state.n = p.state.n + 1 return nil end
|
||||
return p
|
||||
`)
|
||||
// remove the second so exactly one plugin hooks this stage
|
||||
single.mu.Lock()
|
||||
single.plugins = single.plugins[:1]
|
||||
single.mu.Unlock()
|
||||
single.rebuild()
|
||||
|
||||
ps := twoHookPlugins(t, counterPlugin("aaa", "who"), counterPlugin("bbb", "who"))
|
||||
|
||||
singleOut := single.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})
|
||||
psOut := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})
|
||||
|
||||
if singleOut["who"] != "from-aaa" {
|
||||
t.Errorf("★ single-plugin path dropped the return value: %v", singleOut)
|
||||
}
|
||||
// Two plugins both return `who`; the merge order decides the winner, and it
|
||||
// must be deterministic (load order), not scheduler-dependent.
|
||||
if psOut["who"] != "from-bbb" {
|
||||
t.Errorf("merge order is not load order: who = %v, want from-bbb", psOut["who"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestMergeOrderIsDeterministic: goroutine completion order must not leak into
|
||||
// the result. Running Fire repeatedly must always yield the same payload, or a
|
||||
// gateway's behaviour changes run to run with the same plugins installed.
|
||||
func TestMergeOrderIsDeterministic(t *testing.T) {
|
||||
// The plugins differ in cost so a scheduler-dependent merge is visible.
|
||||
slow := `
|
||||
local p = { name = "aaa" }
|
||||
p.state = { n = 0 }
|
||||
p.hooks = { request_end = "slow" }
|
||||
function p.slow(payload)
|
||||
local acc = 0
|
||||
for i = 1, 3000 do acc = acc + i % 7 end
|
||||
p.state.n = p.state.n + 1
|
||||
return { winner = "aaa", acc = acc }
|
||||
end
|
||||
return p
|
||||
`
|
||||
fast := `
|
||||
local p = { name = "bbb" }
|
||||
p.state = { n = 0 }
|
||||
p.hooks = { request_end = "fast" }
|
||||
function p.fast(payload)
|
||||
p.state.n = p.state.n + 1
|
||||
return { winner = "bbb" }
|
||||
end
|
||||
return p
|
||||
`
|
||||
ps := twoHookPlugins(t, slow, fast)
|
||||
first := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})["winner"]
|
||||
for i := 0; i < 60; i++ {
|
||||
got := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})["winner"]
|
||||
if got != first {
|
||||
t.Fatalf("★ winner changed between runs: %q then %q — merge depends on goroutine "+
|
||||
"scheduling, so the same configuration behaves differently run to run", first, got)
|
||||
}
|
||||
}
|
||||
// Load order decides: bbb is loaded second, so it wins.
|
||||
if first != "bbb" {
|
||||
t.Errorf("winner = %q, want bbb (load order decides, not timing)", first)
|
||||
}
|
||||
}
|
||||
|
||||
// TestThrowingHookDoesNotBreakSiblings covers the failure path that golua
|
||||
// actually produces: a hook raising a Lua error.
|
||||
//
|
||||
// The test name used to claim it covered panics, and it used error() to do it.
|
||||
// Probing the six ways a Lua program can fault (error(), indexing nil, calling
|
||||
// nil, concatenating nil, arithmetic on nil, unbounded recursion) showed golua
|
||||
// converts ALL of them into an error RETURN, not a Go panic — so the recover()
|
||||
// in Fire was untested by that case, and deleting recover() still passed. The
|
||||
// test was renamed to say what it verifies.
|
||||
//
|
||||
// recover() is kept anyway: it guards the Go side of Fire (a nil map write, a
|
||||
// future change to how the payload is prepared), which is cheap and cannot be
|
||||
// triggered from Lua today. Claiming it is covered by a Lua test would be the
|
||||
// kind of assurance that evaporates the first time someone checks.
|
||||
func TestThrowingHookDoesNotBreakSiblings(t *testing.T) {
|
||||
throwing := `
|
||||
local p = { name = "aaa" }
|
||||
p.state = {}
|
||||
p.hooks = { request_end = "boom" }
|
||||
function p.boom(payload) error("intentional failure") end
|
||||
return p
|
||||
`
|
||||
healthy := counterPlugin("bbb", "")
|
||||
ps := twoHookPlugins(t, throwing, healthy)
|
||||
|
||||
out := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"}) // must not crash
|
||||
if out == nil {
|
||||
t.Error("Fire returned nil")
|
||||
}
|
||||
st, ok := ps.State("bbb").(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatal("healthy plugin has no state")
|
||||
}
|
||||
if n := st["n"]; n != float64(1) {
|
||||
t.Errorf("★ healthy plugin did not run alongside the throwing one: n = %v", n)
|
||||
}
|
||||
he := ps.HookErrors()
|
||||
if len(he) == 0 {
|
||||
t.Error("a throwing hook was not recorded in hook_errors — the failure would be invisible")
|
||||
}
|
||||
}
|
||||
|
||||
// TestParallelHooksAllRunOnce: every plugin must be invoked exactly once per
|
||||
// Fire. A lost or duplicated goroutine shows up as a wrong total, which for the
|
||||
// billing plugin means a wrong bill.
|
||||
func TestParallelHooksAllRunOnce(t *testing.T) {
|
||||
ps := twoHookPlugins(t, counterPlugin("aaa", ""), counterPlugin("bbb", ""))
|
||||
const fires = 100
|
||||
for i := 0; i < fires; i++ {
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})
|
||||
}
|
||||
for _, name := range []string{"aaa", "bbb"} {
|
||||
st, ok := ps.State(name).(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("%s has no state", name)
|
||||
}
|
||||
if n := st["n"]; n != float64(fires) {
|
||||
t.Errorf("%s counted %v hooks, want %d", name, n, fires)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestConcurrentFireIsSafe drives Fire from many goroutines at once. Each
|
||||
// plugin has its own Lua state and its own mutex, so this must hold; the race
|
||||
// detector is what proves it, not the assertions.
|
||||
func TestConcurrentFireIsSafe(t *testing.T) {
|
||||
ps := twoHookPlugins(t, counterPlugin("aaa", ""), counterPlugin("bbb", ""))
|
||||
var wg sync.WaitGroup
|
||||
for g := 0; g < 8; g++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
for i := 0; i < 50; i++ {
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
for _, name := range []string{"aaa", "bbb"} {
|
||||
st, ok := ps.State(name).(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("%s has no state", name)
|
||||
}
|
||||
if n := st["n"]; n != float64(400) {
|
||||
t.Errorf("%s counted %v, want 400", name, n)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestFireDoesNotHoldRegistryLockWhileRunningHooks: a hook that installs or
|
||||
// disables a plugin takes ps.mu for write. If Fire held ps.mu across the hook,
|
||||
// that would self-deadlock — the exact failure TrueAgent recorded for its own
|
||||
// parallel stop path.
|
||||
func TestFireDoesNotHoldRegistryLockWhileRunningHooks(t *testing.T) {
|
||||
reentrant := `
|
||||
local p = { name = "aaa" }
|
||||
p.state = {}
|
||||
p.hooks = { request_end = "reenter" }
|
||||
function p.reenter(payload)
|
||||
-- Reading the registry from inside a hook is the read half of the same lock.
|
||||
local _ = #payload
|
||||
return nil
|
||||
end
|
||||
return p
|
||||
`
|
||||
vm := NewVM(filepath.Join(t.TempDir(), "adapters"))
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer vm.Stop()
|
||||
ps := NewPlugins(vm, filepath.Join(t.TempDir(), "plugins"))
|
||||
defer ps.Close()
|
||||
ps.DisableStatePersistence()
|
||||
if err := ps.LoadSource("aaa", reentrant); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource("bbb", counterPlugin("bbb", "")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
defer close(done)
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})
|
||||
// Touch the registry the way an admin request would, right after.
|
||||
_ = ps.Count()
|
||||
_ = ps.List()
|
||||
}()
|
||||
select {
|
||||
case <-done:
|
||||
case <-time.After(10 * time.Second):
|
||||
t.Fatal("Fire deadlocked against the registry lock")
|
||||
}
|
||||
}
|
||||
|
||||
// TestBillingSurvivesParallelFire runs the REAL bundled plugin next to another
|
||||
// one. A synthetic counter cannot catch a mismatch between the documented
|
||||
// payload shape and what the plugin actually reads.
|
||||
func TestBillingSurvivesParallelFire(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
pdir := filepath.Join(dir, "plugins")
|
||||
os.MkdirAll(pdir, 0o755)
|
||||
src, err := os.ReadFile("plugins/billing.lua")
|
||||
if err != nil {
|
||||
t.Fatalf("read billing.lua: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(pdir, "billing.lua"), src, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(pdir, "observer.lua"),
|
||||
[]byte(`local p={name="observer"} p.state={n=0}
|
||||
p.hooks={request_end="see"}
|
||||
function p.see(payload) p.state.n=p.state.n+1 return nil end
|
||||
return p`), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
vm := NewVM(filepath.Join(dir, "adapters"))
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer vm.Stop()
|
||||
ps := NewPlugins(vm, pdir)
|
||||
ps.save.setWakeDelayForTest(time.Millisecond)
|
||||
defer ps.Close()
|
||||
ps.DisableStatePersistence()
|
||||
if err := ps.LoadDir(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ps.Count() != 2 {
|
||||
t.Fatalf("loaded %d plugins, want 2", ps.Count())
|
||||
}
|
||||
|
||||
payload := map[string]interface{}{
|
||||
"model": "deepseek-v4.1-flash", "source": "commandcode", "key": "k", "ok": true,
|
||||
"usage": map[string]interface{}{
|
||||
"prompt_tokens": float64(1000), "completion_tokens": float64(100),
|
||||
"cache_hit_tokens": float64(0),
|
||||
},
|
||||
}
|
||||
ps.Fire(StageRequestEnd, payload)
|
||||
|
||||
st := ps.State("billing")
|
||||
if st == nil {
|
||||
t.Fatal("billing produced no state")
|
||||
}
|
||||
m := st.(map[string]interface{})
|
||||
total, ok := m["total"].(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("billing.total is %T", m["total"])
|
||||
}
|
||||
if total["requests"] != float64(1) {
|
||||
t.Errorf("billing counted %v requests, want 1 — the plugin was starved by the parallel path",
|
||||
total["requests"])
|
||||
}
|
||||
if n := ps.State("observer").(map[string]interface{})["n"]; n != float64(1) {
|
||||
t.Errorf("observer counted %v, want 1", n)
|
||||
}
|
||||
if len(ps.HookErrors()) != 0 {
|
||||
t.Errorf("hook errors under parallel Fire: %v", ps.HookErrors())
|
||||
}
|
||||
}
|
||||
@ -1,76 +0,0 @@
|
||||
package lua
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A plugin hook that RAISES must not take the process down. This is the
|
||||
// production outage: a Lua error anywhere in a hook made golua's callEx call
|
||||
// L.StackTrace(), which calls lua_getinfo and SIGSEGVs on a deep enough stack —
|
||||
// a C-level signal Go cannot recover from, so one buggy plugin killed the whole
|
||||
// gateway and took every in-flight request with it.
|
||||
//
|
||||
// The fix routes hook calls through a Lua-side pcall guard, so the error comes
|
||||
// back as an ordinary return value. This test fires hooks that raise on purpose
|
||||
// and asserts three things: the process survives, the failure is RECORDED, and
|
||||
// a well-behaved plugin on the same state keeps working afterwards (the error
|
||||
// must not poison the Lua state).
|
||||
func TestHookThatRaisesDoesNotCrashTheProcess(t *testing.T) {
|
||||
ps, pdir := billingVM(t)
|
||||
boom := `
|
||||
local plugin = {}
|
||||
plugin.name = "boom"
|
||||
plugin.version = "0.1"
|
||||
function plugin.request_end(payload)
|
||||
-- Raise on a table index, the exact shape of the billing bug that caused the
|
||||
-- outage. Deliberately NOT a syntax error: this must load fine and fail only
|
||||
-- when invoked.
|
||||
local x = nil
|
||||
return x.field
|
||||
end
|
||||
return plugin`
|
||||
if err := os.WriteFile(filepath.Join(pdir, "boom.lua"), []byte(boom), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ps.LoadSource("boom", boom); err != nil {
|
||||
t.Fatalf("load boom: %v", err)
|
||||
}
|
||||
|
||||
payload := map[string]interface{}{
|
||||
"model": "m", "source": "s", "ok": true,
|
||||
"prompt_tokens": 100, "completion_tokens": 10, "time": 1750000000000,
|
||||
}
|
||||
// Fire many times: a single call could pass by luck, but if the error ever
|
||||
// escapes into golua's C path the process dies and this test never returns.
|
||||
for i := 0; i < 50; i++ {
|
||||
ps.Fire(StageRequestEnd, payload)
|
||||
}
|
||||
// Reaching this line at all is the primary assertion.
|
||||
|
||||
errs := ps.HookErrors()
|
||||
end, ok := errs[string(StageRequestEnd)]
|
||||
if !ok {
|
||||
t.Fatal("a raising hook left no record — failures must be observable, not swallowed")
|
||||
}
|
||||
if end["count"] == nil || end["count"].(int) == 0 {
|
||||
t.Error("hook error count is zero despite 50 raising calls")
|
||||
}
|
||||
msg, _ := end["last_error"].(string)
|
||||
if !strings.Contains(msg, "boom") {
|
||||
t.Errorf("last_error does not name the offending plugin: %q", msg)
|
||||
}
|
||||
|
||||
// The billing plugin shares the same Plugins registry and must still work:
|
||||
// one broken plugin may not disable the others.
|
||||
st, _ := ps.State("billing").(map[string]interface{})
|
||||
if st == nil || st["total"] == nil {
|
||||
t.Fatalf("the healthy plugin stopped working after another plugin raised")
|
||||
}
|
||||
tot, _ := st["total"].(map[string]interface{})
|
||||
if tot == nil || tot["requests"] == nil || tot["requests"].(float64) == 0 {
|
||||
t.Errorf("billing recorded no requests after the raising plugin ran: %v", st["total"])
|
||||
}
|
||||
}
|
||||
@ -1,283 +0,0 @@
|
||||
package lua
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// These cover plugin state persistence: a plugin's totals live in the Lua VM,
|
||||
// which dies with the process. Measured on the real gateway before this change:
|
||||
// 218,241 requests and 26.2M prompt tokens gone after one systemctl restart.
|
||||
|
||||
func persistTestPlugin() string {
|
||||
return `
|
||||
local plugin = { name = "counter", version = "1.0" }
|
||||
plugin.state = { n = 0, prices_seen = 0 }
|
||||
plugin.prices = { rate = 1 }
|
||||
plugin.hooks = { request_end = "bump" }
|
||||
function plugin.bump(payload)
|
||||
plugin.state.n = plugin.state.n + 1
|
||||
return nil
|
||||
end
|
||||
return plugin
|
||||
`
|
||||
}
|
||||
|
||||
// newPersistVM wires a plugin registry on a fresh dir, mirroring newPluginVM.
|
||||
func newPersistVM(t *testing.T) (*Plugins, string) {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
vm := NewVM(filepath.Join(dir, "adapters"))
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatalf("vm start: %v", err)
|
||||
}
|
||||
t.Cleanup(vm.Stop)
|
||||
pdir := filepath.Join(dir, "plugins")
|
||||
ps := NewPlugins(vm, pdir)
|
||||
// Shorten the saver debounce: Close() waits for the saver goroutine, so
|
||||
// with the production 2s interval every test would pay 2s on teardown.
|
||||
ps.save.setWakeDelayForTest(2 * time.Millisecond)
|
||||
t.Cleanup(ps.Close)
|
||||
return ps, pdir
|
||||
}
|
||||
|
||||
func loadCounter(t *testing.T, ps *Plugins) {
|
||||
t.Helper()
|
||||
if err := ps.LoadSource("counter", persistTestPlugin()); err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func stateN(t *testing.T, ps *Plugins) float64 {
|
||||
t.Helper()
|
||||
st := ps.State("counter")
|
||||
m, ok := st.(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("state is %T, want map", st)
|
||||
}
|
||||
n, ok := m["n"].(float64)
|
||||
if !ok {
|
||||
t.Fatalf("state.n is %T, want float64", m["n"])
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// TestStateSurvivesRestart is the defect itself: a rebuilt registry over the
|
||||
// same plugin dir must come up with the previous totals, not at zero.
|
||||
func TestStateSurvivesRestart(t *testing.T) {
|
||||
ps, pdir := newPersistVM(t)
|
||||
loadCounter(t, ps)
|
||||
for i := 0; i < 25; i++ {
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
|
||||
}
|
||||
if got := stateN(t, ps); got != 25 {
|
||||
t.Fatalf("in-process total = %v, want 25", got)
|
||||
}
|
||||
// Force the write the saver would do, so the test does not depend on timing.
|
||||
ps.save.flush()
|
||||
ps.Close()
|
||||
|
||||
// A brand-new registry over the same dir: this is the restart.
|
||||
vm := NewVM(filepath.Join(t.TempDir(), "adapters"))
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer vm.Stop()
|
||||
ps2 := NewPlugins(vm, pdir)
|
||||
defer ps2.Close()
|
||||
loadCounter(t, ps2)
|
||||
|
||||
if got := stateN(t, ps2); got != 25 {
|
||||
t.Fatalf("★ total after restart = %v, want 25 — this is the 'restart loses the books' bug", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPricesAndStateAreSeparate: restoring configuration over history (or the
|
||||
// reverse) would either erase the totals or resurrect stale prices.
|
||||
func TestPricesAndStateAreSeparate(t *testing.T) {
|
||||
ps, pdir := newPersistVM(t)
|
||||
loadCounter(t, ps)
|
||||
for i := 0; i < 7; i++ {
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
|
||||
}
|
||||
ps.save.flush()
|
||||
ps.Close()
|
||||
|
||||
b, err := os.ReadFile(ps.stateFile("counter"))
|
||||
if err != nil {
|
||||
t.Fatalf("no state file: %v", err)
|
||||
}
|
||||
var saved struct {
|
||||
State map[string]interface{} `json:"state"`
|
||||
Prices map[string]interface{} `json:"prices"`
|
||||
}
|
||||
if err := json.Unmarshal(b, &saved); err != nil {
|
||||
t.Fatalf("state file is not valid JSON: %v", err)
|
||||
}
|
||||
if saved.Prices == nil || saved.Prices["rate"] != float64(1) {
|
||||
t.Errorf("prices were not persisted separately: %v", saved.Prices)
|
||||
}
|
||||
if saved.State["n"] != float64(7) {
|
||||
t.Errorf("state.n = %v, want 7", saved.State["n"])
|
||||
}
|
||||
_ = pdir
|
||||
}
|
||||
|
||||
// TestPricesOnlyUpdatePersists guards a real hole: SetState returns early for a
|
||||
// prices-only payload (correctly leaving state alone), and the first version
|
||||
// returned before the persistence write — so a reprice was durable in memory
|
||||
// only and a restart silently reverted to the old prices.
|
||||
func TestPricesOnlyUpdatePersists(t *testing.T) {
|
||||
ps, _ := newPersistVM(t)
|
||||
loadCounter(t, ps)
|
||||
for i := 0; i < 5; i++ {
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
|
||||
}
|
||||
ps.save.flush()
|
||||
|
||||
err := ps.SetState("counter", map[string]interface{}{
|
||||
"prices": map[string]interface{}{"rate": 42},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("SetState: %v", err)
|
||||
}
|
||||
if got := stateN(t, ps); got != 5 {
|
||||
t.Errorf("a prices-only payload must not touch state: n = %v, want 5", got)
|
||||
}
|
||||
|
||||
b, err := os.ReadFile(ps.stateFile("counter"))
|
||||
if err != nil {
|
||||
t.Fatalf("no state file after a prices-only update: %v", err)
|
||||
}
|
||||
var saved struct {
|
||||
State map[string]interface{} `json:"state"`
|
||||
Prices map[string]interface{} `json:"prices"`
|
||||
}
|
||||
if err := json.Unmarshal(b, &saved); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if saved.Prices["rate"] != float64(42) {
|
||||
t.Errorf("★ new price not persisted: %v — a restart would revert to the old price", saved.Prices)
|
||||
}
|
||||
if saved.State["n"] != float64(5) {
|
||||
t.Errorf("state was clobbered by a price change: %v", saved.State["n"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestHookReturnValueSurvivesSnapshot guards the ordering bug: snapshotting
|
||||
// plugin.state resets the Lua stack, so doing it BEFORE reading the hook's
|
||||
// return value silently turned every opinionated plugin into a silent one —
|
||||
// breaking the documented "return a table to merge into payload" contract with
|
||||
// no error anywhere.
|
||||
func TestHookReturnValueSurvivesSnapshot(t *testing.T) {
|
||||
ps, _ := newPersistVM(t)
|
||||
code := `
|
||||
local plugin = { name = "opinionated" }
|
||||
plugin.state = { n = 0 }
|
||||
plugin.hooks = { request_end = "tag" }
|
||||
function plugin.tag(payload)
|
||||
plugin.state.n = plugin.state.n + 1
|
||||
return { cost_usd = 1.25, verdict = "billed" }
|
||||
end
|
||||
return plugin
|
||||
`
|
||||
if err := ps.LoadSource("opinionated", code); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
out := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
|
||||
if out["cost_usd"] != 1.25 {
|
||||
t.Errorf("★ hook return value lost: payload = %v", out)
|
||||
}
|
||||
if out["verdict"] != "billed" {
|
||||
t.Errorf("merged field lost: %v", out)
|
||||
}
|
||||
ps.save.flush()
|
||||
st := ps.State("opinionated").(map[string]interface{})
|
||||
if st["n"] != float64(1) {
|
||||
t.Errorf("state.n = %v, want 1 (the hook still ran)", st["n"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestCorruptStateFileIsNotFatal: a truncated write must degrade to compiled-in
|
||||
// defaults, never to a gateway that refuses to start.
|
||||
func TestCorruptStateFileIsNotFatal(t *testing.T) {
|
||||
ps, _ := newPersistVM(t)
|
||||
loadCounter(t, ps)
|
||||
for i := 0; i < 3; i++ {
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
|
||||
}
|
||||
ps.save.flush()
|
||||
ps.Close()
|
||||
|
||||
if err := os.WriteFile(ps.stateFile("counter"), []byte("{not json"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
vm := NewVM(filepath.Join(t.TempDir(), "adapters"))
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer vm.Stop()
|
||||
ps2 := NewPlugins(vm, ps.dir)
|
||||
defer ps2.Close()
|
||||
var warned bool
|
||||
ps2.logf = func(string, ...interface{}) { warned = true }
|
||||
loadCounter(t, ps2)
|
||||
|
||||
if got := stateN(t, ps2); got != 0 {
|
||||
t.Errorf("with a corrupt file the plugin must fall back to its defaults, got n = %v", got)
|
||||
}
|
||||
if !warned {
|
||||
t.Error("a corrupt state file must warn the operator, not fail silently")
|
||||
}
|
||||
// And it must still forward.
|
||||
ps2.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
|
||||
}
|
||||
|
||||
// TestFlushIsCoalesced: N mutations must not become N writes. A synchronous
|
||||
// per-request write would put a file write on the hot path.
|
||||
func TestFlushIsCoalesced(t *testing.T) {
|
||||
ps, _ := newPersistVM(t)
|
||||
loadCounter(t, ps)
|
||||
for i := 0; i < 500; i++ {
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
|
||||
}
|
||||
ps.save.flush()
|
||||
// 500 mutations must collapse to a handful of writes, not 500. The bound is
|
||||
// loose on purpose: the saver also flushes on a timer, so how many writes
|
||||
// happen during the loop depends on how long 500 hook calls take. What must
|
||||
// never happen is one write per mutation.
|
||||
if w := ps.save.writes.Load(); w > 5 {
|
||||
t.Errorf("500 hook calls produced %d file writes; the saver must coalesce", w)
|
||||
}
|
||||
if got := stateN(t, ps); got != 500 {
|
||||
t.Fatalf("n = %v, want 500", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCloseFlushesTail: the shutdown path must not lose the last interval,
|
||||
// which would reintroduce the same defect in a smaller window.
|
||||
func TestCloseFlushesTail(t *testing.T) {
|
||||
ps, _ := newPersistVM(t)
|
||||
loadCounter(t, ps)
|
||||
ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "source": "s"})
|
||||
ps.Close()
|
||||
// Close() already waits for the final flush; no sleep needed.
|
||||
|
||||
b, err := os.ReadFile(ps.stateFile("counter"))
|
||||
if err != nil {
|
||||
t.Fatalf("★ Close() did not flush: %v — a shutdown loses the tail", err)
|
||||
}
|
||||
var saved struct {
|
||||
State map[string]interface{} `json:"state"`
|
||||
}
|
||||
if err := json.Unmarshal(b, &saved); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if saved.State["n"] != float64(1) {
|
||||
t.Errorf("flushed n = %v, want 1", saved.State["n"])
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@ -1,447 +0,0 @@
|
||||
package lua
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// newPluginVM builds a VM plus a plugin registry rooted at dir.
|
||||
func newPluginVM(t *testing.T) (*VM, *Plugins, string) {
|
||||
t.Helper()
|
||||
dir := filepath.Join(t.TempDir(), "adapters")
|
||||
vm := NewVM(dir)
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatalf("vm start: %v", err)
|
||||
}
|
||||
t.Cleanup(vm.Stop)
|
||||
pdir := filepath.Join(t.TempDir(), "plugins")
|
||||
return vm, NewPlugins(vm, pdir), pdir
|
||||
}
|
||||
|
||||
// loadPlugin writes one plugin to disk and loads it.
|
||||
func loadPlugin(t *testing.T, ps *Plugins, name, code string) error {
|
||||
t.Helper()
|
||||
if err := os.MkdirAll(ps.dir, 0755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(ps.dir, name+".lua"), []byte(code), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return ps.LoadSource(name, code)
|
||||
}
|
||||
|
||||
// TestPluginManifestAndHooks: the two hook registration forms both work and the
|
||||
// manifest is read.
|
||||
func TestPluginManifestAndHooks(t *testing.T) {
|
||||
_, ps, _ := newPluginVM(t)
|
||||
code := `
|
||||
local p = {}
|
||||
p.name = "demo"
|
||||
p.version = "1.2.3"
|
||||
p.description = "a demo plugin"
|
||||
p.author = "tester"
|
||||
p.hooks = { request_end = "on_end" }
|
||||
function p.on_end(payload)
|
||||
payload.seen = true
|
||||
payload.name_seen = "demo"
|
||||
return payload
|
||||
end
|
||||
return p
|
||||
`
|
||||
if err := loadPlugin(t, ps, "demo", code); err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
list := ps.List()
|
||||
if len(list) != 1 {
|
||||
t.Fatalf("List() = %d plugins, want 1", len(list))
|
||||
}
|
||||
if list[0]["name"] != "demo" || list[0]["version"] != "1.2.3" {
|
||||
t.Errorf("manifest not read: %+v", list[0])
|
||||
}
|
||||
hooks := list[0]["hooks"].([]string)
|
||||
if len(hooks) != 1 || hooks[0] != string(StageRequestEnd) {
|
||||
t.Errorf("hooks = %v, want [request_end]", hooks)
|
||||
}
|
||||
out := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})
|
||||
if out["seen"] != true || out["name_seen"] != "demo" {
|
||||
t.Errorf("hook did not mutate payload: %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginAnonymousHookForm: `request_end = function() end` directly on the
|
||||
// table must register too, since a single-hook plugin should not need a name.
|
||||
func TestPluginAnonymousHookForm(t *testing.T) {
|
||||
_, ps, _ := newPluginVM(t)
|
||||
code := `
|
||||
local p = { name = "anon" }
|
||||
p.request_end = function(payload)
|
||||
payload.hit = 1
|
||||
return payload
|
||||
end
|
||||
return p
|
||||
`
|
||||
if err := loadPlugin(t, ps, "anon", code); err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
out := ps.Fire(StageRequestEnd, map[string]interface{}{})
|
||||
if out["hit"] != float64(1) {
|
||||
t.Errorf("anonymous hook did not fire: %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginHookStagesFireInOrder: each stage reaches only its own hooks.
|
||||
func TestPluginHookStagesFireInOrder(t *testing.T) {
|
||||
_, ps, _ := newPluginVM(t)
|
||||
code := `
|
||||
local p = { name = "stages" }
|
||||
p.hooks = {
|
||||
request_start = "s1",
|
||||
routed = "s2",
|
||||
request_end = "s3",
|
||||
}
|
||||
function p.s1(x) x.order = (x.order or "") .. "1" return x end
|
||||
function p.s2(x) x.order = (x.order or "") .. "2" return x end
|
||||
function p.s3(x) x.order = (x.order or "") .. "3" return x end
|
||||
return p
|
||||
`
|
||||
if err := loadPlugin(t, ps, "stages", code); err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
payload := map[string]interface{}{}
|
||||
ps.Fire(StageRequestStart, payload)
|
||||
ps.Fire(StageRouted, payload)
|
||||
ps.Fire(StageRequestEnd, payload)
|
||||
if payload["order"] != "123" {
|
||||
t.Errorf("stage order = %v, want \"123\"", payload["order"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginErrorIsContained is the critical safety property: a throwing hook
|
||||
// must not propagate. Forwarding depends on it.
|
||||
func TestPluginErrorIsContained(t *testing.T) {
|
||||
_, ps, _ := newPluginVM(t)
|
||||
code := `
|
||||
local p = { name = "boom" }
|
||||
p.hooks = { request_end = "kaboom" }
|
||||
function p.kaboom(payload)
|
||||
error("intentional plugin failure")
|
||||
end
|
||||
return p
|
||||
`
|
||||
if err := loadPlugin(t, ps, "boom", code); err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
// Must not panic and must return the payload unchanged.
|
||||
out := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m"})
|
||||
if out["model"] != "m" {
|
||||
t.Errorf("payload was altered by a failing plugin: %+v", out)
|
||||
}
|
||||
// And the failure must be visible, not silent.
|
||||
errs := ps.HookErrors()
|
||||
if errs["request_end"] == nil {
|
||||
t.Error("a failing plugin left no error record; it would be silently missing")
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginFailingHookDoesNotBlockLaterPlugins: one bad plugin must not stop
|
||||
// the next one from running.
|
||||
func TestPluginFailingHookDoesNotBlockLaterPlugins(t *testing.T) {
|
||||
_, ps, _ := newPluginVM(t)
|
||||
bad := `
|
||||
local p = { name = "bad" }
|
||||
p.hooks = { request_end = "f" }
|
||||
function p.f(x) error("boom") end
|
||||
return p
|
||||
`
|
||||
good := `
|
||||
local p = { name = "good" }
|
||||
p.hooks = { request_end = "f" }
|
||||
function p.f(x) x.good = true return x end
|
||||
return p
|
||||
`
|
||||
_ = loadPlugin(t, ps, "bad", bad)
|
||||
if err := loadPlugin(t, ps, "good", good); err != nil {
|
||||
t.Fatalf("load good: %v", err)
|
||||
}
|
||||
out := ps.Fire(StageRequestEnd, map[string]interface{}{})
|
||||
if out["good"] != true {
|
||||
t.Errorf("a good plugin was blocked by a failing one: %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginSyntaxErrorIsIsolated: a plugin that will not compile is listed
|
||||
// with its error and is never called — it must not prevent LoadDir from loading
|
||||
// the rest.
|
||||
func TestPluginSyntaxErrorIsIsolated(t *testing.T) {
|
||||
_, ps, _ := newPluginVM(t)
|
||||
broken := "this is not lua((("
|
||||
good := `
|
||||
local p = { name = "ok" }
|
||||
p.hooks = { request_end = "f" }
|
||||
function p.f(x) x.ok = true return x end
|
||||
return p
|
||||
`
|
||||
_ = loadPlugin(t, ps, "broken", broken)
|
||||
if err := loadPlugin(t, ps, "ok", good); err != nil {
|
||||
t.Fatalf("load ok: %v", err)
|
||||
}
|
||||
if err := ps.LoadDir(); err != nil {
|
||||
t.Fatalf("LoadDir: %v", err)
|
||||
}
|
||||
// The broken plugin must not be callable and must carry an error.
|
||||
for _, row := range ps.List() {
|
||||
if row["name"] == "broken" {
|
||||
if row["loaded"] == true {
|
||||
t.Error("a plugin with a syntax error reported itself as loaded")
|
||||
}
|
||||
if row["error"] == nil || row["error"] == "" {
|
||||
t.Error("a broken plugin carries no error message")
|
||||
}
|
||||
}
|
||||
}
|
||||
// The good plugin still works.
|
||||
out := ps.Fire(StageRequestEnd, map[string]interface{}{})
|
||||
if out["ok"] != true {
|
||||
t.Errorf("good plugin stopped working: %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginUIExtension: a plugin can contribute a page and elements.
|
||||
func TestPluginUIExtension(t *testing.T) {
|
||||
_, ps, _ := newPluginVM(t)
|
||||
code := `
|
||||
local p = { name = "ui" }
|
||||
p.hooks = { request_end = "f" }
|
||||
function p.f(x) return x end
|
||||
p.ui = {
|
||||
page = {
|
||||
page_id = "billing",
|
||||
title = "Billing",
|
||||
icon = "💰",
|
||||
order = 50,
|
||||
mount = "<div id=billing>hi</div><script>console.log('m')</script>",
|
||||
},
|
||||
elements = {
|
||||
{ target = "status", anchor = "top", mount = "<div>cost</div>" },
|
||||
},
|
||||
}
|
||||
return p
|
||||
`
|
||||
if err := loadPlugin(t, ps, "ui", code); err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
// Every contributed page — including the single `page` field — is folded
|
||||
// into one merged list. Reading ui.Page here would silently pass on a
|
||||
// payload whose pages were all dropped.
|
||||
ui := ps.UI()
|
||||
if len(ui.Pages) != 1 {
|
||||
t.Fatalf("expected 1 merged page, got %d", len(ui.Pages))
|
||||
}
|
||||
pg := ui.Pages[0]
|
||||
if pg.PageID != "billing" || pg.Title != "Billing" {
|
||||
t.Errorf("page = %+v", pg)
|
||||
}
|
||||
if !strings.Contains(pg.Mount, "console.log") {
|
||||
t.Error("mount lost its script content")
|
||||
}
|
||||
if len(ui.Elements) != 1 || ui.Elements[0].Target != "status" {
|
||||
t.Errorf("elements = %+v", ui.Elements)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginHookReturnsNilIsNoOpinion: a hook returning nothing must leave the
|
||||
// payload untouched (plugins should not be forced to echo it back).
|
||||
func TestPluginHookReturnsNilIsNoOpinion(t *testing.T) {
|
||||
_, ps, _ := newPluginVM(t)
|
||||
code := `
|
||||
local p = { name = "silent" }
|
||||
p.hooks = { request_end = "f" }
|
||||
function p.f(payload)
|
||||
-- records nothing, returns nothing
|
||||
return nil
|
||||
end
|
||||
return p
|
||||
`
|
||||
if err := loadPlugin(t, ps, "silent", code); err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
out := ps.Fire(StageRequestEnd, map[string]interface{}{"model": "m", "ok": true})
|
||||
if out["model"] != "m" || out["ok"] != true {
|
||||
t.Errorf("a no-op hook disturbed the payload: %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginUIJSONShape is a wire-format guard: the kernel sends this to the
|
||||
// browser, so the shape is a contract with the WebUI.
|
||||
func TestPluginUIJSONShape(t *testing.T) {
|
||||
_, ps, _ := newPluginVM(t)
|
||||
code := `
|
||||
local p = { name = "shape" }
|
||||
p.hooks = { request_end = "f" }
|
||||
function p.f(x) return x end
|
||||
p.ui = { elements = { { target = "keys", mount = "<b>k</b>" } } }
|
||||
return p
|
||||
`
|
||||
if err := loadPlugin(t, ps, "shape", code); err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
b, err := json.Marshal(ps.UI())
|
||||
if err != nil {
|
||||
t.Fatalf("marshal UI: %v", err)
|
||||
}
|
||||
var view struct {
|
||||
Elements []struct {
|
||||
Target string `json:"target"`
|
||||
Mount string `json:"mount"`
|
||||
} `json:"elements"`
|
||||
}
|
||||
if err := json.Unmarshal(b, &view); err != nil {
|
||||
t.Fatalf("unmarshal: %v", err)
|
||||
}
|
||||
if len(view.Elements) != 1 || view.Elements[0].Target != "keys" {
|
||||
t.Errorf("UI JSON shape = %+v", view.Elements)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPluginFireWithNoPluginsIsNoop: an empty registry must not allocate or fail.
|
||||
func TestPluginFireWithNoPluginsIsNoop(t *testing.T) {
|
||||
_, ps, _ := newPluginVM(t)
|
||||
in := map[string]interface{}{"a": 1}
|
||||
out := ps.Fire(StageRequestEnd, in)
|
||||
if out["a"] != 1 || ps.Count() != 0 {
|
||||
t.Errorf("empty registry misbehaved: %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestChainStepIsARealStage: chain_step is documented as a distinct stage that
|
||||
// fires once per step of an AUTO walk. If it were only a field on `routed`, a
|
||||
// plugin author following the docs would silently get one event instead of the
|
||||
// whole walk.
|
||||
func TestChainStepIsARealStage(t *testing.T) {
|
||||
found := false
|
||||
for _, s := range AllStages {
|
||||
if s == StageChainStep {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Fatal("StageChainStep is not in AllStages, so the dispatcher never registers it")
|
||||
}
|
||||
// Ordering: it must sit between request_start and routed, which is what
|
||||
// docs/plugins.md promises.
|
||||
var iStart, iStep, iRouted = -1, -1, -1
|
||||
for i, s := range AllStages {
|
||||
switch s {
|
||||
case StageRequestStart:
|
||||
iStart = i
|
||||
case StageChainStep:
|
||||
iStep = i
|
||||
case StageRouted:
|
||||
iRouted = i
|
||||
}
|
||||
}
|
||||
if !(iStart < iStep && iStep < iRouted) {
|
||||
t.Errorf("stage order = %v, want request_start < chain_step < routed", AllStages)
|
||||
}
|
||||
_, ps, _ := newPluginVM(t)
|
||||
code := `
|
||||
local p = { name = "stepper" }
|
||||
p.hooks = { chain_step = "s" }
|
||||
function p.s(payload)
|
||||
payload.kinds = (payload.kinds or "")
|
||||
return nil
|
||||
end
|
||||
return p
|
||||
`
|
||||
if err := loadPlugin(t, ps, "stepper", code); err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
// It must actually dispatch.
|
||||
seen := false
|
||||
for _, row := range ps.List() {
|
||||
if row["name"] == "stepper" {
|
||||
hooks := row["hooks"].([]string)
|
||||
for _, h := range hooks {
|
||||
if h == string(StageChainStep) {
|
||||
seen = true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if !seen {
|
||||
t.Error("a plugin registered for chain_step is not reported as such")
|
||||
}
|
||||
}
|
||||
|
||||
// TestOnDiskReportsPages pins that the disk listing carries the plugin's UI
|
||||
// page ids. The desktop plugin panel reads on_disk (not List) and shows the
|
||||
// pages so an operator can see at a glance that a plugin renders something;
|
||||
// without this field the GUI reads p.pages === undefined and the line simply
|
||||
// never appears, which is the kind of omission nobody reports as a bug.
|
||||
func TestOnDiskReportsPages(t *testing.T) {
|
||||
_, ps, _ := newPluginVM(t)
|
||||
|
||||
code := `
|
||||
local p = { name = "paged", version = "1.0.0" }
|
||||
p.ui = { pages = {} }
|
||||
table.insert(p.ui.pages, { page_id = "alpha", title = "Alpha", mount = "<b>a</b>" })
|
||||
table.insert(p.ui.pages, { page_id = "beta", title = "Beta", mount = "<b>b</b>" })
|
||||
return p
|
||||
`
|
||||
if err := loadPlugin(t, ps, "paged", code); err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
|
||||
var found *DiskEntry
|
||||
for _, de := range ps.OnDisk() {
|
||||
if de.Name == "paged" {
|
||||
d := de
|
||||
found = &d
|
||||
}
|
||||
}
|
||||
if found == nil {
|
||||
t.Fatal("paged plugin missing from OnDisk")
|
||||
}
|
||||
if len(found.Pages) != 2 {
|
||||
t.Fatalf("Pages = %v, want [alpha beta]", found.Pages)
|
||||
}
|
||||
if found.Pages[0] != "alpha" || found.Pages[1] != "beta" {
|
||||
t.Fatalf("Pages = %v, want [alpha beta] in declaration order", found.Pages)
|
||||
}
|
||||
|
||||
// The field must survive JSON too — the desktop panel reads the HTTP body,
|
||||
// not the Go struct.
|
||||
b, err := json.Marshal(found)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(b), `"pages":["alpha","beta"]`) {
|
||||
t.Fatalf("json omits pages: %s", b)
|
||||
}
|
||||
}
|
||||
|
||||
// The single-page form (`ui.page`) must also surface, since a plugin using it
|
||||
// is just as rendered as one using `ui.pages`.
|
||||
func TestOnDiskReportsSinglePage(t *testing.T) {
|
||||
_, ps, _ := newPluginVM(t)
|
||||
code := `
|
||||
local p = { name = "single", version = "1.0.0" }
|
||||
p.ui = { page = { page_id = "solo", title = "Solo", mount = "<b>x</b>" } }
|
||||
return p
|
||||
`
|
||||
if err := loadPlugin(t, ps, "single", code); err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
for _, de := range ps.OnDisk() {
|
||||
if de.Name == "single" {
|
||||
if len(de.Pages) != 1 || de.Pages[0] != "solo" {
|
||||
t.Fatalf("Pages = %v, want [solo]", de.Pages)
|
||||
}
|
||||
return
|
||||
}
|
||||
}
|
||||
t.Fatal("single plugin missing from OnDisk")
|
||||
}
|
||||
@ -118,236 +118,3 @@ func TestAdaptersPreserveToolCalls(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestOpenCodeStreamOptionsOnlyWhenStreaming pins the fix for
|
||||
// "stream_options should be set along with stream": OpenCode Go (and other
|
||||
// strict OpenAI-compatible upstreams) reject a non-streaming request that
|
||||
// carries stream_options, which broke every non-stream call through the
|
||||
// opencode adapter.
|
||||
func TestOpenCodeStreamOptionsOnlyWhenStreaming(t *testing.T) {
|
||||
vm := NewVM(freshAdapterDir(t))
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer vm.Stop()
|
||||
|
||||
// non-streaming: stream_options must be absent entirely
|
||||
out, err := vm.Transform("opencode", "transform_request",
|
||||
`{"model":"m","messages":[{"role":"user","content":"hi"}]}`)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(out, "stream_options") {
|
||||
t.Fatalf("non-streaming request must not carry stream_options: %s", out)
|
||||
}
|
||||
|
||||
// streaming: stream_options.include_usage must be set
|
||||
sout, err := vm.Transform("opencode", "transform_request",
|
||||
`{"model":"m","stream":true,"messages":[{"role":"user","content":"hi"}]}`)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(sout, "stream_options") || !strings.Contains(sout, "include_usage") {
|
||||
t.Fatalf("streaming request must carry stream_options.include_usage: %s", sout)
|
||||
}
|
||||
}
|
||||
|
||||
// TestOpenCodeGoVsZenReasoning pins the one behavioural difference that made a
|
||||
// shared adapter wrong: OpenCode Go's thinking mode REQUIRES the assistant
|
||||
// turn's reasoning_content to be echoed back ("The `reasoning_content` in the
|
||||
// thinking mode must be passed back to the API"), while the Zen free pool must
|
||||
// not receive it. Hence two purpose-built adapters.
|
||||
func TestOpenCodeGoVsZenReasoning(t *testing.T) {
|
||||
vm := NewVM(freshAdapterDir(t))
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer vm.Stop()
|
||||
|
||||
body := `{"model":"m","messages":[
|
||||
{"role":"user","content":"hi"},
|
||||
{"role":"assistant","content":"","reasoning_content":"I should read the file.","tool_calls":[{"id":"call_1","type":"function","function":{"name":"read","arguments":"{\"path\":\"/x\"}"}}]},
|
||||
{"role":"tool","tool_call_id":"call_1","content":"r"}
|
||||
]}`
|
||||
|
||||
// Go: reasoning_content must survive, and the tool call must stay paired.
|
||||
gout, err := vm.Transform("opencodego", "transform_request", body)
|
||||
if err != nil {
|
||||
t.Fatalf("opencodego: %v", err)
|
||||
}
|
||||
if !strings.Contains(gout, "I should read the file.") {
|
||||
t.Errorf("opencodego must pass reasoning_content back (thinking mode requires it): %s", gout)
|
||||
}
|
||||
if !strings.Contains(gout, "call_1") || !strings.Contains(gout, "tool_call_id") {
|
||||
t.Errorf("opencodego must keep the tool call paired with its result: %s", gout)
|
||||
}
|
||||
|
||||
// Zen: reasoning_content is stripped.
|
||||
zout, err := vm.Transform("opencodezen", "transform_request", body)
|
||||
if err != nil {
|
||||
t.Fatalf("opencodezen: %v", err)
|
||||
}
|
||||
if strings.Contains(zout, "I should read the file.") {
|
||||
t.Errorf("opencodezen must strip reasoning_content: %s", zout)
|
||||
}
|
||||
if !strings.Contains(zout, "call_1") {
|
||||
t.Errorf("opencodezen must still keep the tool call: %s", zout)
|
||||
}
|
||||
|
||||
// An assistant turn that carries tool_calls but no reasoning_content must
|
||||
// get an empty one injected: Go validates that field on tool-calling turns
|
||||
// and 400s when it is absent, while most clients never send it.
|
||||
// Verified against the live endpoint that an empty string passes.
|
||||
bare := `{"model":"m","messages":[
|
||||
{"role":"user","content":"go"},
|
||||
{"role":"assistant","content":"","tool_calls":[{"id":"c1","type":"function","function":{"name":"read","arguments":"{}"}}]},
|
||||
{"role":"tool","tool_call_id":"c1","content":"r"}
|
||||
]}`
|
||||
bout, err := vm.Transform("opencodego", "transform_request", bare)
|
||||
if err != nil {
|
||||
t.Fatalf("opencodego bare: %v", err)
|
||||
}
|
||||
if !strings.Contains(bout, `"reasoning_content":""`) {
|
||||
t.Errorf("opencodego must inject an empty reasoning_content on a tool-calling turn: %s", bout)
|
||||
}
|
||||
// A plain assistant turn (no tool calls) must NOT gain the field.
|
||||
plainAsst := `{"model":"m","messages":[{"role":"user","content":"hi"},{"role":"assistant","content":"hello"}]}`
|
||||
pout, err := vm.Transform("opencodego", "transform_request", plainAsst)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(pout, "reasoning_content") {
|
||||
t.Errorf("opencodego must not inject reasoning_content on a plain assistant turn: %s", pout)
|
||||
}
|
||||
|
||||
// Both must still omit stream_options on non-streaming requests.
|
||||
for _, a := range []string{"opencodego", "opencodezen"} {
|
||||
out, err := vm.Transform(a, "transform_request", `{"model":"m","messages":[{"role":"user","content":"hi"}]}`)
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", a, err)
|
||||
}
|
||||
if strings.Contains(out, "stream_options") {
|
||||
t.Errorf("%s must not send stream_options when not streaming: %s", a, out)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestOpenCodeSessionIsStableForCache pins the session-derivation contract that
|
||||
// makes the upstream prefix cache usable.
|
||||
//
|
||||
// Measured against the live OpenCode Go endpoint: the prefix cache is
|
||||
// SESSION-scoped. Replaying the same 6032-token prompt hits 5888 cached tokens
|
||||
// when the session id is fixed, and 0 when it changes per request. Deriving the
|
||||
// session from meta.timestamp (the previous behaviour) meant every request was a
|
||||
// new session, so the cache could never hit.
|
||||
//
|
||||
// The session must be stable WITHIN one conversation and differ BETWEEN
|
||||
// conversations. Since generic OpenAI clients send no session id at all
|
||||
// (verified by capturing real agent traffic: no user / session_id /
|
||||
// conversation_id / metadata in the body, no session header), the conversation
|
||||
// is fingerprinted by its FIRST user message — history is replayed every turn,
|
||||
// so that message is invariant for the life of the conversation.
|
||||
func TestOpenCodeSessionIsStableForCache(t *testing.T) {
|
||||
for _, name := range []string{"opencodego", "opencodezen"} {
|
||||
vm := NewVM(freshAdapterDir(t))
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
meta := func(ts int, body string) map[string]interface{} {
|
||||
return map[string]interface{}{
|
||||
"timestamp": ts,
|
||||
"body": body,
|
||||
"source": map[string]interface{}{"name": "somesource", "meta": nil},
|
||||
}
|
||||
}
|
||||
|
||||
// Same conversation, later turn: history grew, first user msg unchanged.
|
||||
h1, err := vm.BuildHeaders(name, meta(1000,
|
||||
`{"messages":[{"role":"user","content":"fix the bug"},{"role":"assistant","content":"looking"}]}`))
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", name, err)
|
||||
}
|
||||
h2, err := vm.BuildHeaders(name, meta(2000,
|
||||
`{"messages":[{"role":"user","content":"fix the bug"},{"role":"assistant","content":"looking"},{"role":"user","content":"and tests"}]}`))
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", name, err)
|
||||
}
|
||||
// Different conversation.
|
||||
h3, err := vm.BuildHeaders(name, meta(3000,
|
||||
`{"messages":[{"role":"user","content":"an unrelated task"}]}`))
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", name, err)
|
||||
}
|
||||
|
||||
if h1["x-opencode-session"] == "" {
|
||||
t.Fatalf("%s: session header must be set", name)
|
||||
}
|
||||
if h1["x-opencode-session"] != h2["x-opencode-session"] {
|
||||
t.Errorf("%s: x-opencode-session must be STABLE across turns of one conversation "+
|
||||
"(%q vs %q) — the prefix cache is session-scoped", name, h1["x-opencode-session"], h2["x-opencode-session"])
|
||||
}
|
||||
if h1["x-opencode-session"] == h3["x-opencode-session"] {
|
||||
t.Errorf("%s: a different conversation must get a different session", name)
|
||||
}
|
||||
if h1["x-opencode-request"] == h2["x-opencode-request"] {
|
||||
t.Errorf("%s: x-opencode-request must differ per request", name)
|
||||
}
|
||||
// No user message at all falls back to per-source stability.
|
||||
h4, _ := vm.BuildHeaders(name, meta(4000, `{"messages":[{"role":"system","content":"s"}]}`))
|
||||
h5, _ := vm.BuildHeaders(name, meta(5000, `{"messages":[{"role":"system","content":"s"}]}`))
|
||||
if h4["x-opencode-session"] != h5["x-opencode-session"] {
|
||||
t.Errorf("%s: sessionless requests must still be stable per source", name)
|
||||
}
|
||||
vm.Stop()
|
||||
}
|
||||
}
|
||||
|
||||
// 客户端自带的会话标识（pi 的 x-session-affinity）必须优先于会话指纹：
|
||||
// 它是平台真实会话 id，历史压缩后也不会漂移，且天然按会话隔离。
|
||||
func TestOpenCodePrefersClientSessionID(t *testing.T) {
|
||||
for _, name := range []string{"opencodego", "opencodezen"} {
|
||||
vm := NewVM(freshAdapterDir(t))
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
meta := func(body, cs string) map[string]interface{} {
|
||||
return map[string]interface{}{
|
||||
"timestamp": 1000,
|
||||
"body": body,
|
||||
"client_session": cs,
|
||||
"source": map[string]interface{}{"name": "somesource", "meta": nil},
|
||||
}
|
||||
}
|
||||
|
||||
// 同一条会话：历史在长，但客户端会话 id 不变。
|
||||
a, err := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"fix the bug"}]}`, "cs-aaa"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
b, err := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"a totally different opener"}]}`, "cs-aaa"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if a["x-opencode-session"] != b["x-opencode-session"] {
|
||||
t.Errorf("%s: 客户端会话 id 相同就必须同会话（内容无关）: %q vs %q",
|
||||
name, a["x-opencode-session"], b["x-opencode-session"])
|
||||
}
|
||||
|
||||
// 不同会话。
|
||||
c, _ := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"fix the bug"}]}`, "cs-bbb"))
|
||||
if a["x-opencode-session"] == c["x-opencode-session"] {
|
||||
t.Errorf("%s: 不同客户端会话必须不同会号", name)
|
||||
}
|
||||
|
||||
// 客户端没给会话 id 时，回落到指纹（同内容 -> 同会号）。
|
||||
d, _ := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"fix the bug"}]}`, ""))
|
||||
e, _ := vm.BuildHeaders(name, meta(`{"messages":[{"role":"user","content":"fix the bug"}]}`, ""))
|
||||
if d["x-opencode-session"] != e["x-opencode-session"] {
|
||||
t.Errorf("%s: 无客户端会号时应回落到稳定指纹", name)
|
||||
}
|
||||
if d["x-opencode-session"] == a["x-opencode-session"] {
|
||||
t.Errorf("%s: 客户端会号与指纹两条路必须产出不同会号", name)
|
||||
}
|
||||
vm.Stop()
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,177 +0,0 @@
|
||||
package lua
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// OpenCode Zen/Go 在 usage 里报告输出有多少是思考 token，并在**顶层**用一个
|
||||
// **字符串**给出本次费用。前者决定「completion_tokens 里多少是可见回答」，
|
||||
// 后者是唯一的费用信号。两者都曾被适配器丢掉。
|
||||
func TestOpenCodeCostAndReasoningPassthrough(t *testing.T) {
|
||||
for _, name := range []string{"opencodego", "opencodezen"} {
|
||||
vm := NewVM(freshAdapterDir(t))
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// ---- 非流式 ----
|
||||
resp := `{"id":"x","object":"chat.completion","model":"deepseek-v4.1-flash",
|
||||
"choices":[{"index":0,"message":{"role":"assistant","content":"hi","reasoning_content":"think"},
|
||||
"finish_reason":"stop"}],
|
||||
"usage":{"prompt_tokens":37,"completion_tokens":40,"total_tokens":77,
|
||||
"prompt_cache_hit_tokens":0,"prompt_cache_miss_tokens":37,
|
||||
"prompt_tokens_details":{"cached_tokens":0},
|
||||
"completion_tokens_details":{"reasoning_tokens":40}},
|
||||
"cost":"0"}`
|
||||
out, err := vm.Transform(name, "transform_response", resp)
|
||||
if err != nil {
|
||||
t.Fatalf("%s transform_response: %v", name, err)
|
||||
}
|
||||
var r struct {
|
||||
TokenUsage struct {
|
||||
Completion int `json:"completion"`
|
||||
Details *struct {
|
||||
ReasoningTokens int `json:"reasoning_tokens"`
|
||||
} `json:"completion_tokens_details"`
|
||||
} `json:"token_usage"`
|
||||
Cost string `json:"cost"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(out), &r); err != nil {
|
||||
t.Fatalf("%s unmarshal: %v (%s)", name, err, out)
|
||||
}
|
||||
if r.Cost != "0" {
|
||||
t.Errorf("%s: cost 未透传（上游给的是字符串 \"0\"）: %q", name, r.Cost)
|
||||
}
|
||||
if r.TokenUsage.Details == nil || r.TokenUsage.Details.ReasoningTokens != 40 {
|
||||
t.Errorf("%s: completion_tokens_details.reasoning_tokens 未透传: %s", name, out)
|
||||
}
|
||||
// 前提校验：reasoning_tokens 确实 ≤ completion_tokens（全为思考）。
|
||||
if r.TokenUsage.Details != nil && r.TokenUsage.Details.ReasoningTokens > r.TokenUsage.Completion {
|
||||
t.Errorf("%s: reasoning 不应超过 completion", name)
|
||||
}
|
||||
|
||||
// ---- 流式：非零费用也要透传 ----
|
||||
chunk := `{"id":"x","object":"chat.completion.chunk","choices":[],
|
||||
"usage":{"prompt_tokens":10,"completion_tokens":5,"total_tokens":15,
|
||||
"completion_tokens_details":{"reasoning_tokens":4}},"cost":"0.0012"}`
|
||||
co, err := vm.Transform(name, "transform_stream_chunk", chunk)
|
||||
if err != nil {
|
||||
t.Fatalf("%s transform_stream_chunk: %v", name, err)
|
||||
}
|
||||
var c struct {
|
||||
Cost string `json:"cost"`
|
||||
Usage *struct {
|
||||
Details *struct {
|
||||
ReasoningTokens int `json:"reasoning_tokens"`
|
||||
} `json:"completion_tokens_details"`
|
||||
} `json:"usage"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(co), &c); err != nil {
|
||||
t.Fatalf("%s unmarshal chunk: %v (%s)", name, err, co)
|
||||
}
|
||||
if c.Usage == nil || c.Usage.Details == nil || c.Usage.Details.ReasoningTokens != 4 {
|
||||
t.Errorf("%s: 流式 reasoning_tokens 未透传: %s", name, co)
|
||||
}
|
||||
if c.Cost != "0.0012" {
|
||||
t.Errorf("%s: 流式 cost 未透传: %s", name, co)
|
||||
}
|
||||
|
||||
// ---- 独立的 cost 帧（上游就这么发：choices 为空、只有 cost）----
|
||||
only := `{"choices":[],"cost":"0"}`
|
||||
oo, err := vm.Transform(name, "transform_stream_chunk", only)
|
||||
if err != nil {
|
||||
t.Fatalf("%s cost-only chunk: %v", name, err)
|
||||
}
|
||||
if oo == "" {
|
||||
t.Fatalf("%s: 纯 cost 帧被整个丢弃（客户端将看不到费用）", name)
|
||||
}
|
||||
var oc struct {
|
||||
Cost string `json:"cost"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(oo), &oc); err != nil {
|
||||
t.Fatalf("%s unmarshal cost chunk: %v (%s)", name, err, oo)
|
||||
}
|
||||
if oc.Cost != "0" {
|
||||
t.Errorf("%s: 纯 cost 帧内容丢失: %s", name, oo)
|
||||
}
|
||||
|
||||
// ---- 回归：没有 cost / 没有 details 时不得凭空造字段 ----
|
||||
bare := `{"choices":[{"index":0,"delta":{"content":"hi"},"finish_reason":"stop"}],
|
||||
"usage":{"prompt_tokens":3,"completion_tokens":1,"total_tokens":4}}`
|
||||
bo, _ := vm.Transform(name, "transform_response", bare)
|
||||
if got := bo; len(got) > 0 && containsAll(got, `"cost"`) {
|
||||
t.Errorf("%s: 上游没给 cost 却透传了该字段: %s", name, got)
|
||||
}
|
||||
|
||||
vm.Stop()
|
||||
}
|
||||
}
|
||||
|
||||
func containsAll(s string, subs ...string) bool {
|
||||
for _, sub := range subs {
|
||||
found := false
|
||||
for i := 0; i+len(sub) <= len(s); i++ {
|
||||
if s[i:i+len(sub)] == sub {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// 流式与非流式对同一上游切片的产出必须一致：命中缓存时上游同时给
|
||||
// prompt_tokens_details.cached_tokens 和独立的 hit/miss 字段，两边都要透传。
|
||||
func TestOpenCodeStreamCacheFieldsMatchNonStream(t *testing.T) {
|
||||
for _, name := range []string{"opencodego", "opencodezen"} {
|
||||
vm := NewVM(freshAdapterDir(t))
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
type cache struct {
|
||||
Details *struct {
|
||||
CachedTokens int `json:"cached_tokens"`
|
||||
} `json:"prompt_tokens_details"`
|
||||
Hit int `json:"prompt_cache_hit_tokens"`
|
||||
Miss int `json:"prompt_cache_miss_tokens"`
|
||||
}
|
||||
usage := `"usage":{"prompt_tokens":5732,"completion_tokens":20,"total_tokens":5752,
|
||||
"prompt_cache_hit_tokens":5504,"prompt_cache_miss_tokens":228,
|
||||
"prompt_tokens_details":{"cached_tokens":5504}}`
|
||||
|
||||
// 非流式
|
||||
nout, _ := vm.Transform(name, "transform_response",
|
||||
`{"choices":[{"index":0,"message":{"role":"assistant","content":"hi"},"finish_reason":"stop"}],`+usage+`}`)
|
||||
var nr struct {
|
||||
U cache `json:"token_usage"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(nout), &nr); err != nil {
|
||||
t.Fatalf("%s: %v", name, err)
|
||||
}
|
||||
// 流式
|
||||
sout, _ := vm.Transform(name, "transform_stream_chunk",
|
||||
`{"choices":[],`+usage+`}`)
|
||||
var sr struct {
|
||||
U cache `json:"usage"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(sout), &sr); err != nil {
|
||||
t.Fatalf("%s chunk: %v (%s)", name, err, sout)
|
||||
}
|
||||
|
||||
if nr.U.Hit != 5504 || nr.U.Miss != 228 {
|
||||
t.Errorf("%s: 非流式 hit/miss 异常: %+v", name, nr.U)
|
||||
}
|
||||
if sr.U.Hit != nr.U.Hit || sr.U.Miss != nr.U.Miss {
|
||||
t.Errorf("%s: 流式 hit/miss 与非流式不一致 (stream %d/%d vs non-stream %d/%d): %s",
|
||||
name, sr.U.Hit, sr.U.Miss, nr.U.Hit, nr.U.Miss, sout)
|
||||
}
|
||||
if sr.U.Details == nil || sr.U.Details.CachedTokens != 5504 {
|
||||
t.Errorf("%s: 流式 cached_tokens 丢失: %s", name, sout)
|
||||
}
|
||||
vm.Stop()
|
||||
}
|
||||
}
|
||||
@ -36,7 +36,7 @@ import (
|
||||
golua "github.com/aarzilli/golua/lua"
|
||||
)
|
||||
|
||||
//go:embed adapters/*.lua plugins/*.lua
|
||||
//go:embed adapters/*.lua
|
||||
var bundledAdapters embed.FS
|
||||
|
||||
// adapterGlobal is the reserved global holding the adapter table after the
|
||||
@ -103,11 +103,6 @@ type adapterPool struct {
|
||||
lastGrow time.Time
|
||||
idleRounds int // consecutive janitor rounds that saw reclaimable slack
|
||||
peakInUse int // high-water mark of inUse, for observability
|
||||
// pluginMode makes boot() store the returned table under pluginGlobal
|
||||
// instead of adapterGlobal. Everything else (elastic sizing, reclaim) is
|
||||
// identical, which is why plugins reuse this pool rather than getting a
|
||||
// second implementation.
|
||||
pluginMode bool
|
||||
}
|
||||
|
||||
const (
|
||||
@ -121,13 +116,6 @@ const (
|
||||
// growCooldown keeps a burst of misses from batching repeatedly while the
|
||||
// previous batch is still booting.
|
||||
growCooldown = time.Second
|
||||
// maxPluginStates caps how many concurrent VM states ONE plugin may occupy.
|
||||
// A plugin is third-party code on the request path, so its ceiling is much
|
||||
// lower than an adapter's (which is sized from the sources'
|
||||
// max_concurrent): a plugin hook is a short synchronous call, so a handful
|
||||
// of states is already far more parallelism than any real hook needs, and a
|
||||
// runaway plugin cannot balloon memory the way a per-source adapter pool can.
|
||||
maxPluginStates = 4
|
||||
// residentWorkers is how many states an adapter keeps warm once it has
|
||||
// served at least one request. Booting is milliseconds, but keeping one warm
|
||||
// removes that from the critical path of the next request. Adapters that
|
||||
@ -277,14 +265,7 @@ func (p *adapterPool) boot() (*worker, error) {
|
||||
L.Close()
|
||||
return nil, fmt.Errorf("adapter %s must return a table", p.name)
|
||||
}
|
||||
// SetGlobal POPS the value off the stack, so it can only be called once per
|
||||
// boot. Plugins therefore store under pluginGlobal only, and NOT under
|
||||
// adapterGlobal: a second SetGlobal on the now-empty stack would assign nil.
|
||||
if p.pluginMode {
|
||||
L.SetGlobal(pluginGlobal)
|
||||
} else {
|
||||
L.SetGlobal(adapterGlobal)
|
||||
}
|
||||
L.SetGlobal(adapterGlobal)
|
||||
L.SetTop(0)
|
||||
return &worker{L: L}, nil
|
||||
}
|
||||
@ -620,55 +601,6 @@ func (v *VM) Start() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// bundledPluginsDir is the directory inside the embedded FS holding the
|
||||
// plugins shipped with the gateway. It is separate from adapters/ on purpose:
|
||||
// the two are loaded by different machinery into different kinds of Lua state
|
||||
// (a protocol transform vs. request-pipeline hooks), and keeping them apart
|
||||
// makes it obvious that dropping a file in one does not affect the other.
|
||||
const bundledPluginsDir = "plugins"
|
||||
|
||||
// ReadBundledPlugin returns the source of a plugin shipped with the gateway.
|
||||
// It exists so a test (or an operator tool) can load a bundled plugin without
|
||||
// depending on whether seeding has already run for this directory.
|
||||
func ReadBundledPlugin(name string) (string, error) {
|
||||
data, err := bundledAdapters.ReadFile(bundledPluginsDir + "/" + name + ".lua")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("bundled plugin %s: %w", name, err)
|
||||
}
|
||||
return string(data), nil
|
||||
}
|
||||
|
||||
// writeBundledPlugins seeds the plugin directory with the shipped plugins.
|
||||
//
|
||||
// It runs only when the directory does not exist yet (same rule as adapters):
|
||||
// once the directory exists it is authoritative, so deleting a shipped plugin is
|
||||
// a real delete and editing one survives restarts.
|
||||
func writeBundledPlugins(dir string) error {
|
||||
if dir == "" {
|
||||
return nil
|
||||
}
|
||||
if err := os.MkdirAll(dir, 0755); err != nil {
|
||||
return fmt.Errorf("mkdir plugin dir: %w", err)
|
||||
}
|
||||
entries, err := bundledAdapters.ReadDir(bundledPluginsDir)
|
||||
if err != nil {
|
||||
return nil // nothing embedded; not an error
|
||||
}
|
||||
for _, e := range entries {
|
||||
if e.IsDir() || filepath.Ext(e.Name()) != ".lua" {
|
||||
continue
|
||||
}
|
||||
data, err := bundledAdapters.ReadFile(bundledPluginsDir + "/" + e.Name())
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, e.Name()), data, 0644); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (v *VM) Stop() {
|
||||
select {
|
||||
case <-v.janitorStop:
|
||||
@ -993,47 +925,9 @@ func restorePcall(L *golua.State) {
|
||||
}
|
||||
}
|
||||
|
||||
// hookGuardName is the Lua global that wraps every plugin hook call.
|
||||
const hookGuardName = "__llmsproxy_call_hook"
|
||||
|
||||
// hookGuardSrc defines the wrapper. It exists because of a hard constraint in
|
||||
// the binding: golua's callEx calls L.StackTrace() on ANY pcall error, and
|
||||
// StackTrace() calls lua_getinfo, which SIGSEGVs in this LuaJIT build once the
|
||||
// stack is deep enough (a request_end payload with the AUTO chain trace does
|
||||
// it). Go cannot recover from a C-level signal, so a single Lua mistake inside
|
||||
// one plugin killed the whole gateway — the production outage this was written
|
||||
// for. Repeated crashes proved the boundary is not theoretical.
|
||||
//
|
||||
// pcall INSIDE Lua catches the error before golua ever sees a non-zero
|
||||
// pcall status, so the C stack-trace path is never entered. The failure comes
|
||||
// back as an ordinary (nil, message) pair, which the Go side records in
|
||||
// hook_errors and moves on from — the documented contract that "a broken plugin
|
||||
// must not affect request forwarding" finally holds for script errors too, not
|
||||
// just for Go panics.
|
||||
//
|
||||
// returns: (result, errorMessage) — both nil/"" on success.
|
||||
const hookGuardSrc = `
|
||||
function ` + hookGuardName + `(fn, payload)
|
||||
local ok, res = pcall(fn, payload)
|
||||
if not ok then
|
||||
return nil, tostring(res)
|
||||
end
|
||||
return res, nil
|
||||
end`
|
||||
|
||||
func registerHookGuard(L *golua.State) {
|
||||
if err := L.DoString(hookGuardSrc); err != nil {
|
||||
// Nothing useful to do here beyond leaving the global absent: invoke()
|
||||
// checks for it and falls back to a direct call, which still works for
|
||||
// correct plugins (only their ERRORS stop being survivable).
|
||||
L.Pop(1)
|
||||
}
|
||||
}
|
||||
|
||||
func setupGlobals(L *golua.State) {
|
||||
restorePcall(L)
|
||||
buildJSONTable(L)
|
||||
registerHookGuard(L)
|
||||
|
||||
registerFn(L, "hmac_sha256_hex", func(L *golua.State) int {
|
||||
key := L.ToString(1)
|
||||
@ -1210,56 +1104,6 @@ func pushGoValue(L *golua.State, v interface{}) {
|
||||
}
|
||||
|
||||
func jsonEncode(v interface{}) ([]byte, error) { return json.Marshal(v) }
|
||||
|
||||
// luaToJSON converts the Lua value at idx into a Go value via json.encode, then
|
||||
// unmarshals it into out. It is the bridge used by the plugin manifest/UI
|
||||
// reader: the plugin returns a plain Lua table, and Go wants a typed struct.
|
||||
//
|
||||
// It goes through JSON rather than walking the Lua stack directly because the
|
||||
// adapter/plugin boundary already speaks JSON everywhere else (transform_request
|
||||
// gets a JSON string, hooks get a JSON string), so this keeps one representation
|
||||
// instead of two.
|
||||
func luaToJSON(L *golua.State, idx int, out interface{}) error {
|
||||
if L.GetTop() < 1 {
|
||||
return fmt.Errorf("empty stack")
|
||||
}
|
||||
abs := idx
|
||||
if abs < 0 {
|
||||
abs = L.GetTop() + 1 + abs
|
||||
}
|
||||
if abs < 1 || abs > L.GetTop() {
|
||||
return fmt.Errorf("index %d out of range (top=%d)", idx, L.GetTop())
|
||||
}
|
||||
// Absolute indices throughout: this binding aborts the process (SIGABRT)
|
||||
// on a bad index rather than panicking, so the stack is captured before any
|
||||
// push instead of being addressed relative to a shifting top.
|
||||
//
|
||||
// json.encode is pushed onto the stack and the value is pushed AFTER it, so
|
||||
// Call(1, 1) invokes it (Call takes no function index — it calls whatever
|
||||
// sits below the nargs values).
|
||||
L.GetGlobal("json")
|
||||
if L.IsNil(-1) {
|
||||
L.SetTop(0)
|
||||
return fmt.Errorf("json global missing")
|
||||
}
|
||||
L.GetField(-1, "encode")
|
||||
if L.Type(-1) != golua.LUA_TFUNCTION {
|
||||
L.SetTop(0)
|
||||
return fmt.Errorf("json.encode missing")
|
||||
}
|
||||
L.PushValue(abs)
|
||||
if err := L.Call(1, 1); err != nil {
|
||||
L.SetTop(0)
|
||||
return err
|
||||
}
|
||||
if L.GetTop() < 1 || L.Type(-1) != golua.LUA_TSTRING {
|
||||
L.SetTop(0)
|
||||
return fmt.Errorf("json.encode did not return a string")
|
||||
}
|
||||
s := L.ToString(-1)
|
||||
L.SetTop(0)
|
||||
return json.Unmarshal([]byte(s), out)
|
||||
}
|
||||
func jsonDecode(s string) (interface{}, error) {
|
||||
var v interface{}
|
||||
if err := json.Unmarshal([]byte(s), &v); err != nil {
|
||||
|
||||
@ -1,270 +0,0 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"llmsproxy/internal/config"
|
||||
"llmsproxy/internal/lua"
|
||||
"llmsproxy/internal/types"
|
||||
)
|
||||
|
||||
// Gemini is the only adapter whose endpoint carries the model in the PATH
|
||||
// (POST /v1beta/models/{model}:generateContent), so it is the only one that
|
||||
// needs the substitution this file covers.
|
||||
//
|
||||
// The defect it fixes was silent: the adapter shipped endpoint="/v1/models",
|
||||
// which is Gemini's model-LIST endpoint and answers 405 to POST, so the preset
|
||||
// template produced a source that could never work.
|
||||
|
||||
func geminiVM(t *testing.T) *lua.VM {
|
||||
t.Helper()
|
||||
// The dir must be named "adapters": the VM seeds its bundled adapters into
|
||||
// <dir> on Start and a differently-named dir leaves it empty, which surfaces
|
||||
// as "adapter gemini not loaded" rather than an obvious setup error.
|
||||
vm := lua.NewVM(filepath.Join(t.TempDir(), "adapters"))
|
||||
if err := vm.Start(); err != nil {
|
||||
t.Fatalf("lua vm: %v", err)
|
||||
}
|
||||
t.Cleanup(vm.Stop)
|
||||
return vm
|
||||
}
|
||||
|
||||
// geminiProvider builds a provider on the gemini adapter against base.
|
||||
func geminiProvider(t *testing.T, base string, models ...string) *Provider {
|
||||
t.Helper()
|
||||
if len(models) == 0 {
|
||||
models = []string{"gemini-3.6-flash"}
|
||||
}
|
||||
ms := make([]config.Model, 0, len(models))
|
||||
for _, m := range models {
|
||||
ms = append(ms, config.Model{ID: m, Kind: "chat"})
|
||||
}
|
||||
return New(config.Source{
|
||||
Name: "gem",
|
||||
BaseURL: base,
|
||||
Adapter: "gemini",
|
||||
APIKey: "k",
|
||||
MaxConcurrent: 4,
|
||||
Models: ms,
|
||||
}, geminiVM(t))
|
||||
}
|
||||
|
||||
// geminiRecorder captures the path the gateway actually requested and replies
|
||||
// with a minimal Gemini-shaped payload.
|
||||
type geminiRecorder struct {
|
||||
mu sync.Mutex
|
||||
paths []string
|
||||
bodies []string
|
||||
stream bool
|
||||
}
|
||||
|
||||
func (g *geminiRecorder) handler(t *testing.T) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
raw, _ := io.ReadAll(r.Body)
|
||||
g.mu.Lock()
|
||||
// RequestURI, not URL.Path: URL.Path is the DECODED path, so an escaped
|
||||
// %2F looks like a real "/" and the assertion could not tell an escaped
|
||||
// id from an unescaped one.
|
||||
g.paths = append(g.paths, r.RequestURI)
|
||||
g.bodies = append(g.bodies, string(raw))
|
||||
stream := g.stream
|
||||
g.mu.Unlock()
|
||||
|
||||
if stream {
|
||||
w.Header().Set("Content-Type", "text/event-stream")
|
||||
w.WriteHeader(200)
|
||||
fmt.Fprintf(w, "data: %s\n\n", `{"candidates":[{"content":{"parts":[{"text":"hi"}]},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":3,"candidatesTokenCount":1,"totalTokenCount":4}}`)
|
||||
fmt.Fprintln(w, "data: [DONE]")
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
fmt.Fprint(w, `{"candidates":[{"content":{"parts":[{"text":"pong"}]},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":3,"candidatesTokenCount":1,"totalTokenCount":4}}`)
|
||||
}
|
||||
}
|
||||
|
||||
func (g *geminiRecorder) lastPath() string {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
if len(g.paths) == 0 {
|
||||
return ""
|
||||
}
|
||||
return g.paths[len(g.paths)-1]
|
||||
}
|
||||
|
||||
// TestGeminiEndpointCarriesModel: a non-streaming chat must POST to
|
||||
// /v1beta/models/<model>:generateContent. The old static endpoint produced
|
||||
// "/v1beta/v1/models" (a list endpoint, 405 on POST).
|
||||
func TestGeminiEndpointCarriesModel(t *testing.T) {
|
||||
rec := &geminiRecorder{}
|
||||
srv := httptest.NewServer(rec.handler(t))
|
||||
defer srv.Close()
|
||||
|
||||
p := geminiProvider(t, srv.URL, "gemini-3.6-flash")
|
||||
if _, err := p.Chat(context.Background(), &types.ChatRequest{
|
||||
Model: "gemini-3.6-flash",
|
||||
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("hi")}},
|
||||
}); err != nil {
|
||||
t.Fatalf("chat: %v (path was %q)", err, rec.lastPath())
|
||||
}
|
||||
if got, want := rec.lastPath(), "/v1beta/models/gemini-3.6-flash:generateContent"; got != want {
|
||||
t.Errorf("path = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// TestGeminiStreamUsesStreamGenerateContent: the streaming verb lives in the
|
||||
// path too, so it must be swapped for stream requests only.
|
||||
func TestGeminiStreamUsesStreamGenerateContent(t *testing.T) {
|
||||
rec := &geminiRecorder{stream: true}
|
||||
srv := httptest.NewServer(rec.handler(t))
|
||||
defer srv.Close()
|
||||
|
||||
p := geminiProvider(t, srv.URL, "gemini-3.6-flash")
|
||||
ch, err := p.ChatStream(context.Background(), &types.ChatRequest{
|
||||
Model: "gemini-3.6-flash",
|
||||
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("hi")}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("stream: %v (path was %q)", err, rec.lastPath())
|
||||
}
|
||||
for range ch {
|
||||
}
|
||||
if got, want := rec.lastPath(), "/v1beta/models/gemini-3.6-flash:streamGenerateContent"; got != want {
|
||||
t.Errorf("path = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// TestGeminiModelFollowsRequestNotSourceDefault is the part a naive fix gets
|
||||
// wrong: substituting the source's default model would make every request on a
|
||||
// multi-model source bill the wrong model. AUTO pins req.Model per slot, so the
|
||||
// URL must follow the REQUEST's model.
|
||||
func TestGeminiModelFollowsRequestNotSourceDefault(t *testing.T) {
|
||||
rec := &geminiRecorder{}
|
||||
srv := httptest.NewServer(rec.handler(t))
|
||||
defer srv.Close()
|
||||
|
||||
p := geminiProvider(t, srv.URL, "gemini-3.6-flash", "gemini-3.1-pro")
|
||||
for _, m := range []string{"gemini-3.1-pro", "gemini-3.6-flash"} {
|
||||
if _, err := p.Chat(context.Background(), &types.ChatRequest{
|
||||
Model: m,
|
||||
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("hi")}},
|
||||
}); err != nil {
|
||||
t.Fatalf("chat %s: %v", m, err)
|
||||
}
|
||||
if want := "/v1beta/models/" + m + ":generateContent"; rec.lastPath() != want {
|
||||
t.Errorf("path = %q, want %q", rec.lastPath(), want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestGeminiModelIsEscaped: a model id reaches a URL path, so an id carrying a
|
||||
// slash or space must be escaped rather than silently addressing a different
|
||||
// resource (or producing a request Go's http client rejects).
|
||||
func TestGeminiModelIsEscaped(t *testing.T) {
|
||||
rec := &geminiRecorder{}
|
||||
srv := httptest.NewServer(rec.handler(t))
|
||||
defer srv.Close()
|
||||
|
||||
// Configure the odd id as a real model so ModelFor resolves it.
|
||||
p := geminiProvider(t, srv.URL, "vendor/model:v1 beta")
|
||||
if _, err := p.Chat(context.Background(), &types.ChatRequest{
|
||||
Model: "vendor/model:v1 beta",
|
||||
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("hi")}},
|
||||
}); err != nil {
|
||||
t.Fatalf("chat: %v", err)
|
||||
}
|
||||
got := rec.lastPath()
|
||||
if strings.Contains(got, "/v1beta/models/vendor/model") {
|
||||
t.Errorf("path %q was not escaped: the id's slash changed the resource", got)
|
||||
}
|
||||
if !strings.HasPrefix(got, "/v1beta/models/") || !strings.HasSuffix(got, ":generateContent") {
|
||||
t.Errorf("path = %q, want an escaped id between the prefix and the verb", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestNonGeminiEndpointsAreUntouched: the substitution and the verb rewrite must
|
||||
// be no-ops for every adapter whose endpoint is a fixed path. A regression here
|
||||
// would silently rewrite OpenAI/Anthropic/… URLs.
|
||||
func TestNonGeminiEndpointsAreUntouched(t *testing.T) {
|
||||
for _, adapter := range []string{"openai", "deepseek", "anthropic", "ollama", "mistral", "trae", "sensenova"} {
|
||||
vm := geminiVM(t)
|
||||
p := New(config.Source{
|
||||
Name: "s", BaseURL: "https://example.test", Adapter: adapter,
|
||||
APIKey: "k", MaxConcurrent: 2,
|
||||
Models: []config.Model{{ID: "m1", Kind: "chat"}},
|
||||
}, vm)
|
||||
for _, stream := range []bool{false, true} {
|
||||
got := p.ChatURL("m1", stream)
|
||||
want := "https://example.test" + p.Endpoint()
|
||||
if got != want {
|
||||
t.Errorf("adapter %s stream=%v: URL = %q, want %q", adapter, stream, got, want)
|
||||
}
|
||||
if strings.Contains(got, "streamGenerateContent") {
|
||||
t.Errorf("adapter %s: streaming verb leaked into a fixed endpoint: %q", adapter, got)
|
||||
}
|
||||
}
|
||||
vm.Stop()
|
||||
}
|
||||
}
|
||||
|
||||
// TestSourceEndpointOverridesTemplate: a source that spells its own endpoint
|
||||
// must win outright — including when it spells one WITHOUT the placeholder.
|
||||
func TestSourceEndpointOverridesTemplate(t *testing.T) {
|
||||
vm := geminiVM(t)
|
||||
defer vm.Stop()
|
||||
p := New(config.Source{
|
||||
Name: "s", BaseURL: "https://example.test", Adapter: "gemini",
|
||||
Endpoint: "/custom/path", APIKey: "k", MaxConcurrent: 2,
|
||||
Models: []config.Model{{ID: "m1", Kind: "chat"}},
|
||||
}, vm)
|
||||
if got, want := p.ChatURL("m1", false), "https://example.test/custom/path"; got != want {
|
||||
t.Errorf("URL = %q, want %q (source endpoint must override the adapter template)", got, want)
|
||||
}
|
||||
// And the streaming verb rewrite must not damage an unrelated path.
|
||||
if got := p.ChatURL("m1", true); strings.Contains(got, "streamGenerate") {
|
||||
t.Errorf("streaming rewrite leaked into a user-spelled path: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestGeminiResponseStillParses: the endpoint fix must not disturb the body
|
||||
// transform — a request that reaches the right URL still has to come back as a
|
||||
// unified response with Gemini's usageMetadata mapped.
|
||||
func TestGeminiResponseStillParses(t *testing.T) {
|
||||
rec := &geminiRecorder{}
|
||||
srv := httptest.NewServer(rec.handler(t))
|
||||
defer srv.Close()
|
||||
|
||||
p := geminiProvider(t, srv.URL, "gemini-3.6-flash")
|
||||
resp, err := p.Chat(context.Background(), &types.ChatRequest{
|
||||
Model: "gemini-3.6-flash",
|
||||
Messages: []types.ChatMessage{{Role: "user", Content: types.StringContent("hi")}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("chat: %v", err)
|
||||
}
|
||||
if resp.Content != "pong" {
|
||||
t.Errorf("content = %q, want %q", resp.Content, "pong")
|
||||
}
|
||||
if resp.TokenUsage.Prompt != 3 || resp.TokenUsage.Completion != 1 || resp.TokenUsage.Total != 4 {
|
||||
t.Errorf("usage = %+v, want prompt 3 / completion 1 / total 4", resp.TokenUsage)
|
||||
}
|
||||
// The body the adapter produced must still be Gemini-native.
|
||||
rec.mu.Lock()
|
||||
body := rec.bodies[0]
|
||||
rec.mu.Unlock()
|
||||
var sent map[string]interface{}
|
||||
if err := json.Unmarshal([]byte(body), &sent); err != nil {
|
||||
t.Fatalf("adapter sent invalid json: %v (%s)", err, body)
|
||||
}
|
||||
if _, ok := sent["contents"]; !ok {
|
||||
t.Errorf("adapter did not convert to Gemini's contents[]: %s", body)
|
||||
}
|
||||
}
|
||||
@ -531,9 +531,7 @@ func isAutoID(s string) bool {
|
||||
return s == "" || strings.EqualFold(s, "AUTO")
|
||||
}
|
||||
|
||||
// Endpoint resolves the upstream chat path. It may contain the placeholder
|
||||
// "{model}"; use URL (or ChatURL) rather than calling this directly when the
|
||||
// path has to be usable.
|
||||
// Endpoint resolves the upstream chat path.
|
||||
func (p *Provider) Endpoint() string {
|
||||
if p.cfg.Endpoint != "" {
|
||||
return p.cfg.Endpoint
|
||||
@ -555,40 +553,8 @@ func (p *Provider) ImageEndpoint() string {
|
||||
return "/v1/images/generations"
|
||||
}
|
||||
|
||||
// modelPlaceholder is the marker an adapter puts in its endpoint template when
|
||||
// the upstream carries the model id in the PATH rather than in the body.
|
||||
// Gemini is the only such adapter: POST /v1beta/models/{model}:generateContent.
|
||||
// Every other adapter's endpoint is a fixed path, so substituting is a no-op
|
||||
// for them.
|
||||
const modelPlaceholder = "{model}"
|
||||
|
||||
// ChatURL resolves the full chat URL for a specific model.
|
||||
//
|
||||
// Two substitutions happen here, both driven by the adapter's endpoint template:
|
||||
//
|
||||
// - "{model}" is replaced by the model id actually being sent. Without this
|
||||
// the gateway would POST to Gemini's model-LIST endpoint, which answers 405.
|
||||
// - for a streaming request, the ":generateContent" verb becomes
|
||||
// ":streamGenerateContent". Gemini streams over the same path with a
|
||||
// different verb, and the verb is part of the path, so the two cannot both
|
||||
// be static. The rewrite is deliberately narrow: it only fires on the exact
|
||||
// ":generateContent" suffix, so an adapter whose endpoint merely mentions
|
||||
// the word keeps its path untouched.
|
||||
func (p *Provider) ChatURL(model string, stream bool) string {
|
||||
ep := p.Endpoint()
|
||||
if strings.Contains(ep, modelPlaceholder) {
|
||||
// A model id is put in a URL path, so it must be escaped: an id with a
|
||||
// slash would otherwise silently address a different resource.
|
||||
ep = strings.ReplaceAll(ep, modelPlaceholder, url.PathEscape(model))
|
||||
}
|
||||
if stream {
|
||||
ep = strings.Replace(ep, ":generateContent", ":streamGenerateContent", 1)
|
||||
}
|
||||
return strings.TrimRight(p.cfg.BaseURL, "/") + ep
|
||||
}
|
||||
|
||||
func (p *Provider) URL() string {
|
||||
return p.ChatURL("", false)
|
||||
return strings.TrimRight(p.cfg.BaseURL, "/") + p.Endpoint()
|
||||
}
|
||||
|
||||
func (p *Provider) ImageURL() string {
|
||||
@ -741,7 +707,7 @@ func (p *Provider) probeModels(ctx context.Context) (bool, string) {
|
||||
if err != nil {
|
||||
return false, ""
|
||||
}
|
||||
if hdrs, herr := p.buildHeaders("{}", u, ""); herr == nil {
|
||||
if hdrs, herr := p.buildHeaders("{}", u); herr == nil {
|
||||
req.Header = hdrs
|
||||
}
|
||||
resp, err := p.client.Do(req)
|
||||
@ -784,11 +750,10 @@ func (p *Provider) probeChat(ctx context.Context) (bool, string) {
|
||||
body, err := json.Marshal(probe)
|
||||
if err == nil {
|
||||
var hdr http.Header
|
||||
probeURL := p.ChatURL(model, false)
|
||||
if hdrs, herr := p.buildHeaders(string(body), probeURL, ""); herr == nil {
|
||||
if hdrs, herr := p.buildHeaders(string(body), p.URL()); herr == nil {
|
||||
hdr = hdrs
|
||||
}
|
||||
raw, status, derr := p.do(ctx, probeURL, string(body), hdr)
|
||||
raw, status, derr := p.do(ctx, p.URL(), string(body), hdr)
|
||||
if derr != nil {
|
||||
msg = derr.Error()
|
||||
} else if status == 200 {
|
||||
@ -1066,15 +1031,13 @@ func (p *Provider) Throttle(ctx context.Context) error {
|
||||
|
||||
// ---- request construction ----
|
||||
|
||||
func (p *Provider) buildHeaders(body, url, clientSession string) (http.Header, error) {
|
||||
func (p *Provider) buildHeaders(body, url string) (http.Header, error) {
|
||||
meta := map[string]interface{}{
|
||||
"url": url,
|
||||
"method": http.MethodPost,
|
||||
"body": body,
|
||||
"api_key": p.cfg.APIKey,
|
||||
"timestamp": types.Now(),
|
||||
// client_session 是客户端自带的会话标识（请求头），为空时适配器自行推导。
|
||||
"client_session": clientSession,
|
||||
"source": map[string]interface{}{
|
||||
"name": p.cfg.Name,
|
||||
"meta": p.cfg.Meta,
|
||||
@ -1119,12 +1082,11 @@ func (p *Provider) Chat(ctx context.Context, req *types.ChatRequest) (*types.Uni
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
chatURL := p.ChatURL(model, false)
|
||||
hdrs, err := p.buildHeaders(body, chatURL, req.ClientSession)
|
||||
hdrs, err := p.buildHeaders(body, p.URL())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
raw, status, err := p.do(ctx, chatURL, body, hdrs)
|
||||
raw, status, err := p.do(ctx, p.URL(), body, hdrs)
|
||||
if err != nil {
|
||||
// a client disconnect or cancelled context is neither a success nor
|
||||
// a failure for scheduling purposes — only upstream errors count
|
||||
@ -1180,10 +1142,7 @@ func (p *Provider) ChatStream(ctx context.Context, req *types.ChatRequest) (<-ch
|
||||
p.Release()
|
||||
return nil, err
|
||||
}
|
||||
// stream=true so a path-carried model plus the streaming verb is resolved
|
||||
// for THIS request's model, not the source default.
|
||||
streamURL := p.ChatURL(model, true)
|
||||
hdrs, err := p.buildHeaders(body, streamURL, req.ClientSession)
|
||||
hdrs, err := p.buildHeaders(body, p.URL())
|
||||
if err != nil {
|
||||
p.Release()
|
||||
return nil, err
|
||||
@ -1195,7 +1154,7 @@ func (p *Provider) ChatStream(ctx context.Context, req *types.ChatRequest) (<-ch
|
||||
}
|
||||
rc := make(chan respOrErr, 1)
|
||||
go func() {
|
||||
resp, err := p.doRawStream(ctx, streamURL, body, hdrs)
|
||||
resp, err := p.doRawStream(ctx, p.URL(), body, hdrs)
|
||||
rc <- respOrErr{resp, err}
|
||||
}()
|
||||
|
||||
@ -1452,7 +1411,7 @@ func (p *Provider) Image(ctx context.Context, req *types.ImageGenRequest) (*type
|
||||
// fall back to passthrough adapter (openai-style)
|
||||
transformed = string(b)
|
||||
}
|
||||
hdrs, err := p.buildHeaders(transformed, p.ImageURL(), "")
|
||||
hdrs, err := p.buildHeaders(transformed, p.ImageURL())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@ -1557,24 +1516,12 @@ func standardSSEChunk(data string) string {
|
||||
PromptTokensDetails *struct {
|
||||
CachedTokens int `json:"cached_tokens"`
|
||||
} `json:"prompt_tokens_details"`
|
||||
CompletionTokensDetails *struct {
|
||||
ReasoningTokens int `json:"reasoning_tokens"`
|
||||
} `json:"completion_tokens_details"`
|
||||
} `json:"usage"`
|
||||
// Cost is a top-level decimal string on OpenCode Zen/Go responses.
|
||||
Cost json.RawMessage `json:"cost"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(data), &raw); err != nil {
|
||||
return ""
|
||||
}
|
||||
if len(raw.Choices) == 0 && raw.UpstreamUsage.Total == 0 && raw.UpstreamUsage.TotalTokens == 0 {
|
||||
// Cost-only chunk: OpenCode sends the charge on its own frame
|
||||
// {"choices":[],"cost":"0"} with no usage at all. Dropping it here
|
||||
// would lose the only cost signal the client can see.
|
||||
if c := rawCostString(raw.Cost); c != "" {
|
||||
out, _ := json.Marshal(types.UnifiedChunk{Cost: c})
|
||||
return string(out)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
var usage *types.TokenUsage
|
||||
@ -1592,11 +1539,6 @@ func standardSSEChunk(data string) string {
|
||||
CachedTokens: pu.PromptTokensDetails.CachedTokens,
|
||||
}
|
||||
}
|
||||
if pu.CompletionTokensDetails != nil {
|
||||
usage.CompletionTokensDetails = &types.CompletionTokensDetails{
|
||||
ReasoningTokens: pu.CompletionTokensDetails.ReasoningTokens,
|
||||
}
|
||||
}
|
||||
}
|
||||
finish := ""
|
||||
done := false
|
||||
@ -1621,24 +1563,6 @@ func standardSSEChunk(data string) string {
|
||||
return string(out)
|
||||
}
|
||||
|
||||
// rawCostString normalizes an upstream cost field into a plain string. The
|
||||
// value is a decimal **string** on OpenCode Zen/Go ("0", "0.0012"), but other
|
||||
// upstreams have been seen to send a bare number, so both are accepted.
|
||||
// Returns "" when absent, null, or empty.
|
||||
func rawCostString(raw json.RawMessage) string {
|
||||
s := strings.TrimSpace(string(raw))
|
||||
if s == "" || s == "null" {
|
||||
return ""
|
||||
}
|
||||
if s[0] == '"' {
|
||||
var unq string
|
||||
if json.Unmarshal(raw, &unq) == nil {
|
||||
return strings.TrimSpace(unq)
|
||||
}
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// pickFirst returns a if non-zero, else b (for usage keys that may appear in
|
||||
// either standard *_tokens or legacy short form).
|
||||
func pickFirst(a, b int) int {
|
||||
|
||||
@ -1,127 +0,0 @@
|
||||
package scheduler
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"llmsproxy/internal/types"
|
||||
)
|
||||
|
||||
// TestResultIsEmptyMatchesTheReasoningOnlyResponse guards the production bug
|
||||
// behind "model AUTO returned a completed response with no content".
|
||||
//
|
||||
// Production measurement: 2 of 20 AUTO requests returned zero content, both
|
||||
// served by claude-opus-4-8. Direct upstream capture showed 28 chunks of
|
||||
// `reasoning_content` and NO `content`, finish_reason=length — the token budget
|
||||
// was consumed by thinking before any text was emitted. runTier accepted it
|
||||
// because err == nil.
|
||||
func TestResultIsEmptyMatchesTheReasoningOnlyResponse(t *testing.T) {
|
||||
// Exactly what the upstream returned, as a decoded response.
|
||||
onlyReasoning := &types.UnifiedResponse{
|
||||
Model: "claude-opus-4-8",
|
||||
ReasoningContent: "Alright, the user just said \"hi\". Simple greeting…",
|
||||
FinishReason: "length",
|
||||
TokenUsage: types.TokenUsage{Prompt: 14, Completion: 30, Total: 44},
|
||||
}
|
||||
if !resultIsEmpty(onlyReasoning) {
|
||||
t.Error("a reasoning-only response with usage must count as empty; " +
|
||||
"it is what the client sees as \"completed with no content\"")
|
||||
}
|
||||
|
||||
// The same response WITH text is a normal answer.
|
||||
if resultIsEmpty(&types.UnifiedResponse{Content: "Hi! 👋", ReasoningContent: "hmm"}) {
|
||||
t.Error("a response with content is not empty")
|
||||
}
|
||||
// A tool-calling agent turn has no text and is perfectly valid.
|
||||
if resultIsEmpty(&types.UnifiedResponse{
|
||||
ToolCalls: []types.ToolCall{{Name: "read_file", Arguments: map[string]interface{}{"path": "x"}}},
|
||||
}) {
|
||||
t.Error("a tool-call-only response must NOT count as empty — agents " +
|
||||
"legitimately produce tool calls with no text")
|
||||
}
|
||||
// An image slot returns no text by design.
|
||||
if resultIsEmpty(&types.UnifiedResponse{
|
||||
ImageData: []types.ImageData{{URL: "http://x/y.png"}},
|
||||
}) {
|
||||
t.Error("an image response must NOT count as empty")
|
||||
}
|
||||
// Whitespace-only content is as useless to a client as none at all.
|
||||
if !resultIsEmpty(&types.UnifiedResponse{Content: " \n\t"}) {
|
||||
t.Error("whitespace-only content must count as empty")
|
||||
}
|
||||
if resultIsEmpty(nil) {
|
||||
t.Error("nil is not an empty response; it is an absent one")
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeekStreamHoldsReasoningAndReportsEmpty is the streaming half: a channel
|
||||
// of reasoning-only chunks must be reported as empty so chainDrive degrades.
|
||||
func TestPeekStreamHoldsReasoningAndReportsEmpty(t *testing.T) {
|
||||
in := make(chan types.UnifiedChunk, 8)
|
||||
for i := 0; i < 5; i++ {
|
||||
in <- types.UnifiedChunk{ReasoningContent: "thinking…"}
|
||||
}
|
||||
in <- types.UnifiedChunk{Done: true, FinishReason: "length"}
|
||||
close(in)
|
||||
|
||||
_, peek := peekStream(in)
|
||||
if peek() {
|
||||
t.Error("a reasoning-only stream must be reported empty")
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeekStreamForwardsContentAfterReasoning: the common case still works,
|
||||
// and the reasoning preamble is not forwarded ahead of the content (holding it
|
||||
// back is what keeps the degrade path available).
|
||||
func TestPeekStreamForwardsContentAfterReasoning(t *testing.T) {
|
||||
in := make(chan types.UnifiedChunk, 8)
|
||||
in <- types.UnifiedChunk{ReasoningContent: "let me think"}
|
||||
in <- types.UnifiedChunk{Content: "Hi"}
|
||||
in <- types.UnifiedChunk{Content: "!"}
|
||||
in <- types.UnifiedChunk{Done: true, FinishReason: "stop"}
|
||||
close(in)
|
||||
|
||||
out, peek := peekStream(in)
|
||||
if !peek() {
|
||||
t.Fatal("a stream with content must be reported as having content")
|
||||
}
|
||||
var got []string
|
||||
for ck := range out {
|
||||
if ck.Content != "" {
|
||||
got = append(got, ck.Content)
|
||||
}
|
||||
if strings.TrimSpace(ck.ReasoningContent) != "" {
|
||||
t.Error("reasoning preamble must not be forwarded before content; " +
|
||||
"that is what pins the client to a stream it cannot escape")
|
||||
}
|
||||
}
|
||||
if strings.Join(got, "") != "Hi!" {
|
||||
t.Errorf("forwarded content = %q, want %q", got, "Hi!")
|
||||
}
|
||||
}
|
||||
|
||||
// TestPeekStreamDoesNotDeadlockOnToolCalls: a tool-call delta is content for
|
||||
// this purpose and must unblock peek immediately.
|
||||
func TestPeekStreamDoesNotDeadlockOnToolCalls(t *testing.T) {
|
||||
raw, _ := json.Marshal([]types.ToolCall{{Name: "ls"}})
|
||||
in := make(chan types.UnifiedChunk, 4)
|
||||
in <- types.UnifiedChunk{ToolCalls: raw}
|
||||
in <- types.UnifiedChunk{Done: true, FinishReason: "tool_calls"}
|
||||
close(in)
|
||||
|
||||
_, peek := peekStream(in)
|
||||
done := make(chan bool, 1)
|
||||
go func() { done <- peek() }()
|
||||
select {
|
||||
case ok := <-done:
|
||||
if !ok {
|
||||
t.Error("a tool-call delta must count as content")
|
||||
}
|
||||
case <-time.After(2 * time.Second):
|
||||
t.Fatal("peek blocked on a tool-call-only stream")
|
||||
}
|
||||
_ = context.Background()
|
||||
}
|
||||
@ -11,7 +11,6 @@ import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
@ -246,115 +245,6 @@ func normalCount(cands []candidate) int {
|
||||
//
|
||||
// Normal candidates rotate by base; probe candidates form a fixed tail tried
|
||||
// only after every normal slot failed or was busy.
|
||||
// emptyResultReason describes why a 200-with-no-content response counts as a
|
||||
// slot failure for AUTO.
|
||||
//
|
||||
// WHY: reasoning models (claude-opus-*, codebuddy_glm-*, …) emit
|
||||
// `reasoning_content` first and only then `content`. When the caller's
|
||||
// max_tokens is small enough that the thinking phase consumes the whole budget,
|
||||
// upstream returns 200 / finish_reason=length with 28 chunks of reasoning and
|
||||
// ZERO content. runTier treated "err == nil" as success and handed that to the
|
||||
// client, which then failed with "returned a completed response with no
|
||||
// content" — a client-side error message for what is really a bad slot choice.
|
||||
//
|
||||
// So an empty result is a SLOT failure, not a request failure: the gateway
|
||||
// degrades to the next slot and the user still gets an answer. Measured on
|
||||
// production AUTO: 2 of 20 requests returned empty content, all of them
|
||||
// claude-opus-4-8.
|
||||
//
|
||||
// A response carrying tool_calls or image data is NOT empty: an agent turn
|
||||
// legitimately produces tool calls with no text. ReasoningContent does NOT
|
||||
// rescue it either — see resultIsEmpty.
|
||||
const emptyResultReason = "upstream returned no content (reasoning-only response, or the token budget was consumed before any text)"
|
||||
|
||||
// resultIsEmpty reports whether a successful-but-useless response should be
|
||||
// treated as a slot failure.
|
||||
//
|
||||
// Image data counts: an image-generation slot legitimately returns no text.
|
||||
// A usage-only response is NOT empty either — the upstream answered, it just
|
||||
// said nothing, and that is exactly the case worth degrading away from.
|
||||
func resultIsEmpty(resp *types.UnifiedResponse) bool {
|
||||
if resp == nil {
|
||||
return false
|
||||
}
|
||||
// NOTE: ReasoningContent is deliberately NOT consulted. My first version
|
||||
// excluded it ("the model was thinking, that is an answer"), and the test
|
||||
// built from the real production capture failed immediately: the captured
|
||||
// response is exactly reasoning_content-with-usage and zero text. The
|
||||
// client asked for text and there is none; holding a request hostage to
|
||||
// another model's thinking phase is strictly worse than degrading.
|
||||
return strings.TrimSpace(resp.Content) == "" &&
|
||||
len(resp.ToolCalls) == 0 &&
|
||||
len(resp.ImageData) == 0
|
||||
}
|
||||
|
||||
// emptyStreamReason is resultIsEmpty's streaming twin; see emptyResultReason
|
||||
// for why an empty response is a slot failure rather than a request failure.
|
||||
const emptyStreamReason = emptyResultReason
|
||||
|
||||
// peekStream wraps a chunk channel so the caller learns whether the stream
|
||||
// produced real content BEFORE the chunks are forwarded.
|
||||
//
|
||||
// Why this is necessary: reasoning models emit reasoning_content first. With a
|
||||
// small max_tokens the whole budget is spent thinking, the stream ends with
|
||||
// finish_reason=length and zero content. If the gateway forwarded those chunks
|
||||
// as they arrived, the client would already have seen a 200 SSE stream and
|
||||
// could not be given a different slot — its only recourse is the useless
|
||||
// "returned a completed response with no content" error. Buffering until the
|
||||
// first real content (or the end of the stream) keeps the degrade path
|
||||
// available at the cost of holding back the first few chunks.
|
||||
//
|
||||
// What is NOT buffered: the wrapper starts forwarding as soon as a chunk with
|
||||
// non-empty Content or ToolCalls arrives, and keeps forwarding everything from
|
||||
// then on, so only the reasoning preamble is held. Reasoning-only responses
|
||||
// are dropped in full and reported as empty, which lets chainDrive try the
|
||||
// next slot.
|
||||
func peekStream(in <-chan types.UnifiedChunk) (<-chan types.UnifiedChunk, func() bool) {
|
||||
out := make(chan types.UnifiedChunk, 16)
|
||||
var (
|
||||
mu sync.Mutex
|
||||
sawText bool
|
||||
done bool
|
||||
)
|
||||
go func() {
|
||||
defer close(out)
|
||||
started := false
|
||||
for ck := range in {
|
||||
if !started {
|
||||
// Hold back the reasoning / usage-only preamble. A tool-call
|
||||
// delta counts as content: an agent turn legitimately emits
|
||||
// tool_calls with no text.
|
||||
if strings.TrimSpace(ck.Content) == "" && len(ck.ToolCalls) == 0 {
|
||||
continue
|
||||
}
|
||||
started = true
|
||||
mu.Lock()
|
||||
sawText = true
|
||||
mu.Unlock()
|
||||
}
|
||||
out <- ck
|
||||
}
|
||||
mu.Lock()
|
||||
done = true
|
||||
mu.Unlock()
|
||||
}()
|
||||
// peek blocks until the stream either produces content or ends, then
|
||||
// reports whether any content was seen. Polling a 2ms tick rather than
|
||||
// using a second channel keeps peekStream single-goroutine and leak-free.
|
||||
peek := func() bool {
|
||||
for {
|
||||
mu.Lock()
|
||||
seen, finished := sawText, done
|
||||
mu.Unlock()
|
||||
if seen || finished {
|
||||
return seen
|
||||
}
|
||||
time.Sleep(2 * time.Millisecond)
|
||||
}
|
||||
}
|
||||
return out, peek
|
||||
}
|
||||
|
||||
func runTier(ctx context.Context, tn *TierNode, cands []candidate, base int64, req *types.ChatRequest, stream bool) tierResult {
|
||||
n := len(cands)
|
||||
norm := normalCount(cands)
|
||||
@ -377,18 +267,7 @@ func runTier(ctx context.Context, tn *TierNode, cands []candidate, base int64, r
|
||||
if stream {
|
||||
chunks, err := sl.Prov.ChatStream(ctx, &r)
|
||||
if err == nil {
|
||||
guarded, peek := peekStream(chunks)
|
||||
if peek() {
|
||||
return tierResult{chunks: guarded, src: sl.Source, model: sl.Model}
|
||||
}
|
||||
// The stream finished with no content at all: a
|
||||
// reasoning-only response. Drain and move on to the next
|
||||
// slot instead of pinning the client to a useless stream.
|
||||
hard = append(hard, TierError{
|
||||
Tier: tn.Tier, Source: sl.Source, Model: sl.Model,
|
||||
Err: errors.New(emptyStreamReason),
|
||||
})
|
||||
continue
|
||||
return tierResult{chunks: chunks, src: sl.Source, model: sl.Model}
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
return tierResult{}
|
||||
@ -401,17 +280,6 @@ func runTier(ctx context.Context, tn *TierNode, cands []candidate, base int64, r
|
||||
}
|
||||
resp, err := sl.Prov.Chat(ctx, &r)
|
||||
if err == nil {
|
||||
if resultIsEmpty(resp) {
|
||||
// Soft failure: record it and try the next slot. Deliberately
|
||||
// NOT a hard TierError — a hard error is reported to the client
|
||||
// verbatim when the whole chain fails, and "this one model was
|
||||
// unhelpful" is not the client's problem to debug.
|
||||
hard = append(hard, TierError{
|
||||
Tier: tn.Tier, Source: sl.Source, Model: sl.Model,
|
||||
Err: errors.New(emptyResultReason),
|
||||
})
|
||||
continue
|
||||
}
|
||||
return tierResult{resp: resp, src: sl.Source, model: sl.Model}
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
@ -425,75 +293,13 @@ func runTier(ctx context.Context, tn *TierNode, cands []candidate, base int64, r
|
||||
return tierResult{hard: hard}
|
||||
}
|
||||
|
||||
// TraceKind classifies one step of an AUTO chain walk.
|
||||
type TraceKind string
|
||||
|
||||
const (
|
||||
// TraceTierSkip: the whole tier was skipped — every slot was cooling,
|
||||
// quota-exhausted, or none was schedulable. Reason says which.
|
||||
TraceTierSkip TraceKind = "tier_skip"
|
||||
// TraceSlotFail: one slot failed hard (upstream error / bad adapter). The
|
||||
// walk continues to the next slot or tier.
|
||||
TraceSlotFail TraceKind = "slot_fail"
|
||||
// TraceTierBusy: the tier was fully busy and the bounded wait expired.
|
||||
TraceTierBusy TraceKind = "tier_busy"
|
||||
// TraceSelected: this slot served the request. Exactly one per successful
|
||||
// chain walk, and the last event emitted.
|
||||
TraceSelected TraceKind = "selected"
|
||||
)
|
||||
|
||||
// TraceEvent is one observable step of an AUTO chain walk.
|
||||
//
|
||||
// WHY THIS EXISTS: chainDrive's return value is (resp, src, model, err), so a
|
||||
// caller learns only which slot finally served the request. Everything the
|
||||
// scheduler decided on the way there — which tiers it skipped and WHY, which
|
||||
// slots hard-failed, whether a tier was merely busy — was computed and then
|
||||
// discarded. That is invisible to operators and to plugins: "tier 1 was cooling
|
||||
// so we degraded to tier 3" looked exactly like "tier 1 served it".
|
||||
//
|
||||
// The walk already accumulates this in ChainErr, but ONLY on total failure, and
|
||||
// ChainErr is an error return, not a record. Emitting a trace as it happens
|
||||
// covers the far more common case: a request that SUCCEEDED after degrading.
|
||||
//
|
||||
// Design constraints:
|
||||
// - scheduler stays dependency-free and independently testable. A TraceEvent
|
||||
// is a plain struct in this package and the sink is a func parameter, so no
|
||||
// import is added and no test has to change to observe a walk.
|
||||
// - The sink is optional (nil = emit nothing). The overhead on the hot path
|
||||
// is one nil check per event.
|
||||
// - Events are OBSERVATION ONLY. Nothing in the scheduler branches on them,
|
||||
// and the gateway does not feed them back into routing, cooldown or quota —
|
||||
// see docs/plugins.md for why accounting and enforcement are kept apart.
|
||||
type TraceEvent struct {
|
||||
Kind TraceKind
|
||||
Tier int
|
||||
Source string
|
||||
Model string
|
||||
Reason string // human-readable, for TraceTierSkip / TraceSlotFail
|
||||
Err string // the underlying error text, for TraceSlotFail
|
||||
// Attempt counts the 1-based slot attempt within the whole walk.
|
||||
Attempt int
|
||||
}
|
||||
|
||||
// TraceSink receives chain-walk events. It must not block: it is called from the
|
||||
// request path, and a slow sink slows the request.
|
||||
type TraceSink func(TraceEvent)
|
||||
|
||||
// chainDrive runs a request down the chain (plan 2.3): tiers ascending (tier
|
||||
// 1, the highest priority, first), per-tier round-robin starting at the tier
|
||||
// cursor, same-tier runs ordered by preference (negative prefs sink but stay
|
||||
// reachable). Quota-exhausted and cooling slots are filtered up front; a
|
||||
// fully busy tier is polled for a bounded time before falling through.
|
||||
// Failures are summarized in *ChainErr for the caller to map to HTTP 503.
|
||||
//
|
||||
// trace may be nil; when set it receives one event per observable step.
|
||||
func (s *Scheduler) chainDrive(ctx context.Context, chain *Chain, req *types.ChatRequest, exhausted func(*Slot) bool, stream bool, trace TraceSink) (*types.UnifiedResponse, <-chan types.UnifiedChunk, string, string, error) {
|
||||
emit := func(ev TraceEvent) {
|
||||
if trace != nil {
|
||||
trace(ev)
|
||||
}
|
||||
}
|
||||
attempt := 0
|
||||
func (s *Scheduler) chainDrive(ctx context.Context, chain *Chain, req *types.ChatRequest, exhausted func(*Slot) bool, stream bool) (*types.UnifiedResponse, <-chan types.UnifiedChunk, string, string, error) {
|
||||
if chain == nil || len(chain.Tiers) == 0 {
|
||||
return nil, nil, "", "", fmt.Errorf("no auto slot configured")
|
||||
}
|
||||
@ -503,9 +309,7 @@ func (s *Scheduler) chainDrive(ctx context.Context, chain *Chain, req *types.Cha
|
||||
// dropped unless they qualify as half-cooldown probes (appended last).
|
||||
cands := collectCands(tn.Slots, exhausted)
|
||||
if len(cands) == 0 {
|
||||
reason := "no schedulable slot (cooling or quota exhausted)"
|
||||
ce.Skipped = append(ce.Skipped, fmt.Sprintf("tier %d: %s", tn.Tier, reason))
|
||||
emit(TraceEvent{Kind: TraceTierSkip, Tier: tn.Tier, Reason: reason})
|
||||
ce.Skipped = append(ce.Skipped, fmt.Sprintf("tier %d: no schedulable slot (cooling or quota exhausted)", tn.Tier))
|
||||
continue
|
||||
}
|
||||
// No Pref sort: load balancing is done by round-robin cursor.
|
||||
@ -514,8 +318,6 @@ func (s *Scheduler) chainDrive(ctx context.Context, chain *Chain, req *types.Cha
|
||||
base := tn.NextStart()
|
||||
res := runTier(ctx, tn, cands, base, req, stream)
|
||||
if res.resp != nil || res.chunks != nil {
|
||||
attempt++
|
||||
emit(TraceEvent{Kind: TraceSelected, Tier: tn.Tier, Source: res.src, Model: res.model, Attempt: attempt})
|
||||
releaseProbes(cands)
|
||||
return res.resp, res.chunks, res.src, res.model, nil
|
||||
}
|
||||
@ -525,13 +327,6 @@ func (s *Scheduler) chainDrive(ctx context.Context, chain *Chain, req *types.Cha
|
||||
}
|
||||
if len(res.hard) > 0 {
|
||||
ce.Tiers = append(ce.Tiers, res.hard...)
|
||||
for _, h := range res.hard {
|
||||
attempt++
|
||||
emit(TraceEvent{
|
||||
Kind: TraceSlotFail, Tier: tn.Tier, Source: h.Source, Model: h.Model,
|
||||
Err: types.OneLine(h.Err.Error(), 200), Attempt: attempt,
|
||||
})
|
||||
}
|
||||
releaseProbes(cands)
|
||||
continue // hard failures: fall through to the next tier, no waiting
|
||||
}
|
||||
@ -539,13 +334,10 @@ func (s *Scheduler) chainDrive(ctx context.Context, chain *Chain, req *types.Cha
|
||||
if err := s.pollBusyTier(ctx, tn, cands, base, req, stream, &ce); err != nil {
|
||||
releaseProbes(cands)
|
||||
if r, ok := err.(*tierSuccess); ok {
|
||||
attempt++
|
||||
emit(TraceEvent{Kind: TraceSelected, Tier: tn.Tier, Source: r.res.src, Model: r.res.model, Attempt: attempt})
|
||||
return r.res.resp, r.res.chunks, r.res.src, r.res.model, nil
|
||||
}
|
||||
return nil, nil, "", "", err
|
||||
}
|
||||
emit(TraceEvent{Kind: TraceTierBusy, Tier: tn.Tier, Reason: fmt.Sprintf("no free slot within %v", busyWait)})
|
||||
releaseProbes(cands)
|
||||
}
|
||||
if len(ce.Tiers) == 0 && len(ce.Skipped) == 0 {
|
||||
@ -609,16 +401,16 @@ func (s *Scheduler) pollBusyTier(ctx context.Context, tn *TierNode, cands []cand
|
||||
// non-nil, decides slot token-quota exhaustion. Returns the response, the
|
||||
// serving source and the exact model id used; on total failure a *ChainErr
|
||||
// summarizing every tier.
|
||||
func (s *Scheduler) ChainChat(ctx context.Context, chain *Chain, req *types.ChatRequest, exhausted func(*Slot) bool, trace TraceSink) (*types.UnifiedResponse, string, string, error) {
|
||||
resp, _, src, model, err := s.chainDrive(ctx, chain, req, exhausted, false, trace)
|
||||
func (s *Scheduler) ChainChat(ctx context.Context, chain *Chain, req *types.ChatRequest, exhausted func(*Slot) bool) (*types.UnifiedResponse, string, string, error) {
|
||||
resp, _, src, model, err := s.chainDrive(ctx, chain, req, exhausted, false)
|
||||
return resp, src, model, err
|
||||
}
|
||||
|
||||
// ChainChatStream runs a streaming AUTO request down the chain. A slot is
|
||||
// abandoned only on connect failures / busy (before its first chunk); after a
|
||||
// stream starts it is pinned. Same return contract as ChainChat.
|
||||
func (s *Scheduler) ChainChatStream(ctx context.Context, chain *Chain, req *types.ChatRequest, exhausted func(*Slot) bool, trace TraceSink) (<-chan types.UnifiedChunk, string, string, error) {
|
||||
_, chunks, src, model, err := s.chainDrive(ctx, chain, req, exhausted, true, trace)
|
||||
func (s *Scheduler) ChainChatStream(ctx context.Context, chain *Chain, req *types.ChatRequest, exhausted func(*Slot) bool) (<-chan types.UnifiedChunk, string, string, error) {
|
||||
_, chunks, src, model, err := s.chainDrive(ctx, chain, req, exhausted, true)
|
||||
return chunks, src, model, err
|
||||
}
|
||||
|
||||
|
||||
@ -176,7 +176,7 @@ func TestChainRoundRobin(t *testing.T) {
|
||||
s := New(0)
|
||||
var got []string
|
||||
for i := 0; i < 4; i++ {
|
||||
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
|
||||
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
|
||||
if err != nil {
|
||||
t.Fatalf("iter %d: %v", i, err)
|
||||
}
|
||||
@ -200,7 +200,7 @@ func TestChainPreferenceSinksButStaysReachable(t *testing.T) {
|
||||
{Tier: 0, Model: "g", Source: "good"},
|
||||
}, bySource(neg, good))
|
||||
s := New(0)
|
||||
resp, src, model, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
|
||||
resp, src, model, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
|
||||
if err != nil {
|
||||
t.Fatalf("chain: %v", err)
|
||||
}
|
||||
@ -213,7 +213,7 @@ func TestChainPreferenceSinksButStaysReachable(t *testing.T) {
|
||||
t.Fatal("neg was tried first and succeeded; good must not be attempted")
|
||||
}
|
||||
// Second request: cursor advances. neg wins again (good hard-fails).
|
||||
resp2, src2, _, err2 := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
|
||||
resp2, src2, _, err2 := s.ChainChat(context.Background(), ch, chatReq(), nil)
|
||||
if err2 != nil {
|
||||
t.Fatalf("second chain: %v", err2)
|
||||
}
|
||||
@ -230,7 +230,7 @@ func TestChainBusySkipsWithoutPenalty(t *testing.T) {
|
||||
{Tier: 0, Model: "b", Source: "s2"},
|
||||
}, bySource(a, b))
|
||||
s := New(0)
|
||||
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
|
||||
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
|
||||
if err != nil {
|
||||
t.Fatalf("chain: %v", err)
|
||||
}
|
||||
@ -256,7 +256,7 @@ func TestChainAllBusyBoundedWaitThenNextTier(t *testing.T) {
|
||||
}, bySource(a, b, c))
|
||||
s := New(0)
|
||||
t0 := time.Now()
|
||||
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
|
||||
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
|
||||
el := time.Since(t0)
|
||||
if err != nil {
|
||||
t.Fatalf("chain: %v", err)
|
||||
@ -277,7 +277,7 @@ func TestChainQuotaExhausted(t *testing.T) {
|
||||
}, bySource(a, b))
|
||||
s := New(0)
|
||||
exhausted := func(sl *Slot) bool { return sl.Source == "s1" && sl.Quota > 0 }
|
||||
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), exhausted, nil)
|
||||
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), exhausted)
|
||||
if err != nil {
|
||||
t.Fatalf("chain: %v", err)
|
||||
}
|
||||
@ -300,7 +300,7 @@ func TestChainErrSummary(t *testing.T) {
|
||||
{Tier: 1, Model: "c", Source: "s3"},
|
||||
}, bySource(a, b, c))
|
||||
s := New(0)
|
||||
_, _, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
|
||||
_, _, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
|
||||
var ce *ChainErr
|
||||
if !errors.As(err, &ce) {
|
||||
t.Fatalf("err = %v, want *ChainErr", err)
|
||||
@ -325,7 +325,7 @@ func TestChainStreamFallsBackBeforeFirstChunk(t *testing.T) {
|
||||
{Tier: 0, Model: "b", Source: "s2"},
|
||||
}, bySource(a, b))
|
||||
s := New(0)
|
||||
chunks, src, model, err := s.ChainChatStream(context.Background(), ch, chatReq(), nil, nil)
|
||||
chunks, src, model, err := s.ChainChatStream(context.Background(), ch, chatReq(), nil)
|
||||
if err != nil {
|
||||
t.Fatalf("chain stream: %v", err)
|
||||
}
|
||||
@ -365,7 +365,7 @@ func TestChainProbeIsLastResort(t *testing.T) {
|
||||
}, bySource(healthy, cooling))
|
||||
s := New(0)
|
||||
for i := 0; i < 3; i++ {
|
||||
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
|
||||
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
|
||||
if err != nil {
|
||||
t.Fatalf("iter %d: %v", i, err)
|
||||
}
|
||||
@ -390,7 +390,7 @@ func TestChainProbeServesWhenNothingElseCan(t *testing.T) {
|
||||
cooling.probeable.Store(true)
|
||||
ch := BuildChain([]Rule{{Tier: 0, Model: "c", Source: "cooling"}}, bySource(cooling))
|
||||
s := New(0)
|
||||
resp, src, model, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
|
||||
resp, src, model, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
|
||||
if err != nil {
|
||||
t.Fatalf("probe must serve the request: %v", err)
|
||||
}
|
||||
@ -417,14 +417,14 @@ func TestChainProbePermitReleasedOnFailure(t *testing.T) {
|
||||
cooling.fail.Store(true)
|
||||
ch := BuildChain([]Rule{{Tier: 0, Model: "c", Source: "cooling"}}, bySource(cooling))
|
||||
s := New(0)
|
||||
if _, _, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil); err == nil {
|
||||
if _, _, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil); err == nil {
|
||||
t.Fatal("expected the failing probe to surface an error")
|
||||
}
|
||||
if cooling.probeClaims.Load() != 1 || cooling.probeDones.Load() != 1 {
|
||||
t.Fatalf("permit accounting: claims=%d dones=%d, want 1/1", cooling.probeClaims.Load(), cooling.probeDones.Load())
|
||||
}
|
||||
// permit is free again for the next attempt
|
||||
if _, _, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil); err == nil {
|
||||
if _, _, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil); err == nil {
|
||||
t.Fatal("expected the second probe to fail too")
|
||||
}
|
||||
if cooling.probeClaims.Load() != 2 {
|
||||
@ -443,7 +443,7 @@ func TestChainProbeDoesNotBlockTierFallthrough(t *testing.T) {
|
||||
{Tier: 2, Model: "b", Source: "backup"},
|
||||
}, bySource(cold, backup))
|
||||
s := New(0)
|
||||
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
|
||||
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
|
||||
if err != nil || src != "backup" {
|
||||
t.Fatalf("want fallthrough to backup, got src=%q err=%v", src, err)
|
||||
}
|
||||
@ -482,7 +482,7 @@ func TestChainProbeRoundRobinUnaffected(t *testing.T) {
|
||||
s := New(0)
|
||||
var got []string
|
||||
for i := 0; i < 4; i++ {
|
||||
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil, nil)
|
||||
_, src, _, err := s.ChainChat(context.Background(), ch, chatReq(), nil)
|
||||
if err != nil {
|
||||
t.Fatalf("iter %d: %v", i, err)
|
||||
}
|
||||
|
||||
@ -1,191 +0,0 @@
|
||||
package scheduler
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The chain trace is the only way a caller learns that a request was DEGRADED
|
||||
// — served by a lower tier than the one that should have taken it. chainDrive's
|
||||
// return value carries only the winner, so without these events "tier 1 was
|
||||
// cooling and we dropped to tier 2" is indistinguishable from "tier 1 served
|
||||
// it", which is the exact question a priority chain exists to answer.
|
||||
|
||||
// recorder collects trace events for assertions.
|
||||
type recorder struct{ events []TraceEvent }
|
||||
|
||||
func (r *recorder) sink(ev TraceEvent) { r.events = append(r.events, ev) }
|
||||
|
||||
func (r *recorder) kinds() []TraceKind {
|
||||
out := make([]TraceKind, 0, len(r.events))
|
||||
for _, e := range r.events {
|
||||
out = append(out, e.Kind)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (r *recorder) find(k TraceKind) *TraceEvent {
|
||||
for i := range r.events {
|
||||
if r.events[i].Kind == k {
|
||||
return &r.events[i]
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// TestTraceSelectedOnlyOnHappyPath: a clean walk emits exactly one event.
|
||||
func TestTraceSelectedOnlyOnHappyPath(t *testing.T) {
|
||||
p1 := fakeProv("p1", "m1")
|
||||
ch := BuildChain([]Rule{{Model: "m1", Source: "p1", Tier: 1}},
|
||||
func(model, source string) Provider { return p1 })
|
||||
var rec recorder
|
||||
_, _, _, err := New(3).ChainChat(context.Background(), ch, chatReq(), nil, rec.sink)
|
||||
if err != nil {
|
||||
t.Fatalf("chain: %v", err)
|
||||
}
|
||||
if got := rec.kinds(); len(got) != 1 || got[0] != TraceSelected {
|
||||
t.Errorf("events = %v, want a single selected", got)
|
||||
}
|
||||
e := rec.find(TraceSelected)
|
||||
if e.Tier != 1 || e.Source != "p1" || e.Model != "m1" {
|
||||
t.Errorf("selected event = %+v, want tier 1 / p1 / m1", e)
|
||||
}
|
||||
if e.Attempt != 1 {
|
||||
t.Errorf("Attempt = %d, want 1", e.Attempt)
|
||||
}
|
||||
}
|
||||
|
||||
// TestTraceRecordsTierSkipAndDegradation is the core case: tier 1 is
|
||||
// unschedulable, tier 2 answers. The trace must show the skip AND the eventual
|
||||
// selection, so a consumer can see the request was served one tier down.
|
||||
func TestTraceRecordsTierSkipAndDegradation(t *testing.T) {
|
||||
// p1 is unavailable (not probeable), so tier 1 yields no candidates.
|
||||
p1 := fakeProv("p1", "m1")
|
||||
p1.available.Store(false)
|
||||
p2 := fakeProv("p2", "m2")
|
||||
ch := BuildChain([]Rule{
|
||||
{Model: "m1", Source: "p1", Tier: 1},
|
||||
{Model: "m2", Source: "p2", Tier: 2},
|
||||
}, bySource(p1, p2))
|
||||
var rec recorder
|
||||
_, src, model, err := New(3).ChainChat(context.Background(), ch, chatReq(), nil, rec.sink)
|
||||
if err != nil {
|
||||
t.Fatalf("chain: %v", err)
|
||||
}
|
||||
if src != "p2" || model != "m2" {
|
||||
t.Fatalf("served by %s/%s, want p2/m2", src, model)
|
||||
}
|
||||
skip := rec.find(TraceTierSkip)
|
||||
if skip == nil {
|
||||
t.Fatalf("no tier_skip event; events = %v", rec.kinds())
|
||||
}
|
||||
if skip.Tier != 1 {
|
||||
t.Errorf("skip tier = %d, want 1", skip.Tier)
|
||||
}
|
||||
if !strings.Contains(skip.Reason, "cooling") {
|
||||
t.Errorf("skip reason = %q, want it to mention cooling", skip.Reason)
|
||||
}
|
||||
sel := rec.find(TraceSelected)
|
||||
if sel == nil || sel.Tier != 2 {
|
||||
t.Errorf("selected = %+v, want tier 2", sel)
|
||||
}
|
||||
// The order matters: the skip must be observable BEFORE the selection.
|
||||
if rec.events[0].Kind != TraceTierSkip || rec.events[len(rec.events)-1].Kind != TraceSelected {
|
||||
t.Errorf("event order = %v, want skip first and selected last", rec.kinds())
|
||||
}
|
||||
}
|
||||
|
||||
// TestTraceRecordsHardSlotFailures: a slot that returns an upstream error is a
|
||||
// different event from a skip — the request tried it and it failed. Losing that
|
||||
// distinction makes a flaky upstream look like an idle one.
|
||||
func TestTraceRecordsHardSlotFailures(t *testing.T) {
|
||||
p1 := fakeProv("p1", "m1")
|
||||
p1.fail.Store(true) // Chat returns "upstream error"
|
||||
p2 := fakeProv("p2", "m2")
|
||||
ch := BuildChain([]Rule{
|
||||
{Model: "m1", Source: "p1", Tier: 1},
|
||||
{Model: "m2", Source: "p2", Tier: 2},
|
||||
}, bySource(p1, p2))
|
||||
var rec recorder
|
||||
_, src, _, err := New(3).ChainChat(context.Background(), ch, chatReq(), nil, rec.sink)
|
||||
if err != nil {
|
||||
t.Fatalf("chain: %v", err)
|
||||
}
|
||||
if src != "p2" {
|
||||
t.Fatalf("served by %s, want p2", src)
|
||||
}
|
||||
fail := rec.find(TraceSlotFail)
|
||||
if fail == nil {
|
||||
t.Fatalf("no slot_fail event; events = %v", rec.kinds())
|
||||
}
|
||||
if fail.Tier != 1 || fail.Source != "p1" || fail.Model != "m1" {
|
||||
t.Errorf("slot_fail = %+v, want tier 1 / p1 / m1", fail)
|
||||
}
|
||||
if !strings.Contains(fail.Err, "upstream error") {
|
||||
t.Errorf("slot_fail error = %q, want the upstream text", fail.Err)
|
||||
}
|
||||
// A hard failure must NOT be reported as a skip.
|
||||
if rec.find(TraceTierSkip) != nil {
|
||||
t.Error("a hard failure was also reported as a tier_skip")
|
||||
}
|
||||
}
|
||||
|
||||
// TestTraceNilSinkIsSafe: the gateway passes nil when no plugin is loaded, so
|
||||
// every emit path must tolerate it. This is the "plugins are optional" property
|
||||
// on the scheduler side.
|
||||
func TestTraceNilSinkIsSafe(t *testing.T) {
|
||||
p1 := fakeProv("p1", "m1")
|
||||
p1.fail.Store(true)
|
||||
p2 := fakeProv("p2", "m2")
|
||||
ch := BuildChain([]Rule{
|
||||
{Model: "m1", Source: "p1", Tier: 1},
|
||||
{Model: "m2", Source: "p2", Tier: 2},
|
||||
}, bySource(p1, p2))
|
||||
if _, _, _, err := New(3).ChainChat(context.Background(), ch, chatReq(), nil, nil); err != nil {
|
||||
t.Fatalf("a nil trace sink broke the walk: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestTraceOnTotalFailure: when every tier fails, the walk still emits its
|
||||
// per-step events AND returns the ChainErr. The trace is additive — it must not
|
||||
// replace or disturb the error contract callers depend on for the 503.
|
||||
func TestTraceOnTotalFailure(t *testing.T) {
|
||||
p1 := fakeProv("p1", "m1")
|
||||
p1.fail.Store(true)
|
||||
p2 := fakeProv("p2", "m2")
|
||||
p2.fail.Store(true)
|
||||
ch := BuildChain([]Rule{
|
||||
{Model: "m1", Source: "p1", Tier: 1},
|
||||
{Model: "m2", Source: "p2", Tier: 2},
|
||||
}, bySource(p1, p2))
|
||||
var rec recorder
|
||||
_, _, _, err := New(3).ChainChat(context.Background(), ch, chatReq(), nil, rec.sink)
|
||||
var ce *ChainErr
|
||||
if !errors.As(err, &ce) {
|
||||
t.Fatalf("err = %v, want a *ChainErr so the gateway can answer 503", err)
|
||||
}
|
||||
if len(ce.Tiers) != 2 {
|
||||
t.Errorf("ChainErr.Tiers = %d, want 2 (the error contract must be unchanged)", len(ce.Tiers))
|
||||
}
|
||||
if n := len(rec.kinds()); n != 2 {
|
||||
t.Errorf("events = %v, want two slot_fail and no selection", rec.kinds())
|
||||
}
|
||||
if rec.find(TraceSelected) != nil {
|
||||
t.Error("a selected event was emitted for a walk that served nothing")
|
||||
}
|
||||
}
|
||||
|
||||
// TestTraceSkipsEmptyChain: no chain configured must not emit anything; the
|
||||
// gateway answers 503 before scheduling in that case anyway.
|
||||
func TestTraceSkipsEmptyChain(t *testing.T) {
|
||||
var rec recorder
|
||||
_, _, _, err := New(3).ChainChat(context.Background(), &Chain{}, chatReq(), nil, rec.sink)
|
||||
if err == nil {
|
||||
t.Fatal("expected an error for an empty chain")
|
||||
}
|
||||
if len(rec.events) != 0 {
|
||||
t.Errorf("events = %v, want none", rec.kinds())
|
||||
}
|
||||
}
|
||||
@ -29,10 +29,6 @@ type ChatRequest struct {
|
||||
ToolChoice interface{} `json:"tool_choice,omitempty"`
|
||||
DisableThinking bool `json:"disable_thinking"`
|
||||
ExtraBody map[string]interface{} `json:"-"`
|
||||
|
||||
// ClientSession 是客户端自带的会话标识（请求头 x-session-affinity 等）。
|
||||
// 不参与序列化：它只用于让适配器生成稳定的上游会话，不应泄漏给上游 body。
|
||||
ClientSession string `json:"-"`
|
||||
}
|
||||
|
||||
func (r *ChatRequest) MarshalJSON() ([]byte, error) {
|
||||
@ -117,11 +113,6 @@ type UnifiedResponse struct {
|
||||
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
|
||||
// ImageData used by image-generation adapters.
|
||||
ImageData []ImageData `json:"image_data,omitempty"`
|
||||
// Cost is the upstream-reported charge for this request, passed through
|
||||
// verbatim (OpenCode Zen/Go put a decimal **string** at the response top
|
||||
// level). Deliberately not parsed or summed by the gateway: it is an
|
||||
// upstream fact, and only some upstreams report it at all.
|
||||
Cost string `json:"cost,omitempty"`
|
||||
}
|
||||
|
||||
type TokenUsage struct {
|
||||
@ -138,16 +129,6 @@ type TokenUsage struct {
|
||||
// prompt_tokens_details.cached_tokens is absent.
|
||||
PromptCacheHit int `json:"prompt_cache_hit_tokens,omitempty"`
|
||||
PromptCacheMiss int `json:"prompt_cache_miss_tokens,omitempty"`
|
||||
// CompletionTokensDetails mirrors the OpenAI v2 usage.completion_tokens_details
|
||||
// object. OpenCode Zen/Go report how much of the completion was thinking
|
||||
// tokens here; without it clients cannot tell how much of the billed output
|
||||
// was reasoning rather than visible answer.
|
||||
CompletionTokensDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
|
||||
}
|
||||
|
||||
// CompletionTokensDetails is the OpenAI v2 completion_tokens_details object.
|
||||
type CompletionTokensDetails struct {
|
||||
ReasoningTokens int `json:"reasoning_tokens"`
|
||||
}
|
||||
|
||||
// PromptTokensDetails is the OpenAI v2 prompt_tokens_details object. Only
|
||||
@ -172,16 +153,15 @@ func (t TokenUsage) MarshalJSON() ([]byte, error) {
|
||||
pdetails = &PromptTokensDetails{CachedTokens: t.PromptCacheHit}
|
||||
}
|
||||
return json.Marshal(struct {
|
||||
PromptTokens int `json:"prompt_tokens"`
|
||||
CompletionTokens int `json:"completion_tokens"`
|
||||
TotalTokens int `json:"total_tokens"`
|
||||
Prompt int `json:"prompt"`
|
||||
Completion int `json:"completion"`
|
||||
Total int `json:"total"`
|
||||
PromptTokensDetails *PromptTokensDetails `json:"prompt_tokens_details,omitempty"`
|
||||
PromptCacheHitTokens int `json:"prompt_cache_hit_tokens,omitempty"`
|
||||
PromptCacheMissTokens int `json:"prompt_cache_miss_tokens,omitempty"`
|
||||
CompletionDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
|
||||
PromptTokens int `json:"prompt_tokens"`
|
||||
CompletionTokens int `json:"completion_tokens"`
|
||||
TotalTokens int `json:"total_tokens"`
|
||||
Prompt int `json:"prompt"`
|
||||
Completion int `json:"completion"`
|
||||
Total int `json:"total"`
|
||||
PromptTokensDetails *PromptTokensDetails `json:"prompt_tokens_details,omitempty"`
|
||||
PromptCacheHitTokens int `json:"prompt_cache_hit_tokens,omitempty"`
|
||||
PromptCacheMissTokens int `json:"prompt_cache_miss_tokens,omitempty"`
|
||||
}{
|
||||
PromptTokens: t.Prompt,
|
||||
CompletionTokens: t.Completion,
|
||||
@ -192,7 +172,6 @@ func (t TokenUsage) MarshalJSON() ([]byte, error) {
|
||||
PromptTokensDetails: pdetails,
|
||||
PromptCacheHitTokens: t.PromptCacheHit,
|
||||
PromptCacheMissTokens: t.PromptCacheMiss,
|
||||
CompletionDetails: t.CompletionTokensDetails,
|
||||
})
|
||||
}
|
||||
|
||||
@ -236,9 +215,6 @@ type UnifiedChunk struct {
|
||||
// often has empty choices). Gateway uses it to emit exact usage in the
|
||||
// final stream chunk instead of estimates.
|
||||
Usage *TokenUsage `json:"usage,omitempty"`
|
||||
// Cost is upstream-reported charge for this request (decimal string),
|
||||
// passed through verbatim; OpenCode sends it on its own stream chunk.
|
||||
Cost string `json:"cost,omitempty"`
|
||||
}
|
||||
|
||||
// Meta passed to Lua build_headers hook
|
||||
|
||||
@ -2,7 +2,6 @@ package types
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
@ -118,37 +117,3 @@ func TestChatMessageNormalizesEmptyArrayContent(t *testing.T) {
|
||||
t.Fatalf("empty tool_calls array must be dropped, got %s", om.ToolCalls)
|
||||
}
|
||||
}
|
||||
|
||||
// 上游报告输出里有多少是思考 token；缺少这个字段，客户端无法判断
|
||||
// completion_tokens 里多少是可见回答、多少是思考（两者都按输出计费）。
|
||||
func TestTokenUsageMarshalsCompletionTokensDetails(t *testing.T) {
|
||||
u := TokenUsage{
|
||||
Prompt: 37, Completion: 40, Total: 77,
|
||||
CompletionTokensDetails: &CompletionTokensDetails{ReasoningTokens: 40},
|
||||
}
|
||||
b, err := json.Marshal(u)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var got map[string]interface{}
|
||||
if err := json.Unmarshal(b, &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctd, ok := got["completion_tokens_details"].(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("completion_tokens_details 未输出: %s", b)
|
||||
}
|
||||
if n, _ := ctd["reasoning_tokens"].(float64); int(n) != 40 {
|
||||
t.Errorf("reasoning_tokens = %v, want 40 (%s)", ctd["reasoning_tokens"], b)
|
||||
}
|
||||
// 标准 OpenAI 字段同时存在，老客户端不受影响。
|
||||
if _, ok := got["completion_tokens"]; !ok {
|
||||
t.Errorf("标准 completion_tokens 丢失: %s", b)
|
||||
}
|
||||
|
||||
// 未设置时不得凭空造字段。
|
||||
b2, _ := json.Marshal(TokenUsage{Prompt: 1, Completion: 2, Total: 3})
|
||||
if strings.Contains(string(b2), "completion_tokens_details") {
|
||||
t.Errorf("无数据时不应输出 completion_tokens_details: %s", b2)
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,43 +0,0 @@
|
||||
# llmsproxy — 统一 OpenAI 兼容网关配置（示例模板）
|
||||
#
|
||||
# 这个是打包进发行包、供首次安装使用的模板。它故意不含任何可用的凭据：
|
||||
# 首次启动时程序会生成一个随机 admin key 并写入实际配置（WebUI「密钥」页可见）。
|
||||
#
|
||||
# ⚠ 不要在本文件里填一个写死的具体 key。示例文件是公开的，写进去的任何值
|
||||
# 都会成为一个全网皆知的 admin key（旧版示例里的 sk-gw-local-0001 就踩过
|
||||
# 这个坑：它被直接抄进生产配置并真实可用）。
|
||||
|
||||
# 网关监听地址（默认 :8080，建议绑内网/回环）
|
||||
listen: 127.0.0.1:8080
|
||||
|
||||
# 客户端访问本网关所需的 API Key（Bearer）。留空数组 = 不鉴权（仅内网）。
|
||||
gateway_keys: []
|
||||
|
||||
# 默认模型选择：具体模型 id 或 AUTO。
|
||||
# AUTO = 走 WebUI「优先级页」保存的调度链（存在本文件的 `auto:` 字段）。
|
||||
default_model: AUTO
|
||||
|
||||
# Lua 适配器目录（默认 adapters/，首次启动自动写入内置适配器）
|
||||
adapter_dir: adapters
|
||||
|
||||
# 插件目录（可选）。设置后：
|
||||
# - 首次启动会把随核心发布的示例插件（billing：按源/模型/密钥计费 + 仪表盘）
|
||||
# 写入本目录，它会在请求流水线上挂钩子，并向 WebUI 注入一个「Billing」页面
|
||||
# 和状态页上的一块总开销组件；
|
||||
# - 之后 WebUI「插件」页与 Electron 壳的设置面板可安装/禁用/删除/编辑。
|
||||
# 目录一旦存在即以目录为准：删除或改写内置插件都是真实生效的操作。
|
||||
# 留空 = 插件功能完全关闭（不影响网关其它功能）。
|
||||
# plugin_dir: /etc/llmsproxy/plugins
|
||||
|
||||
# 运行时文件：存放 WebUI 管理的源模板、已删除标记、预置模板名单。
|
||||
#
|
||||
# 注意：**AUTO 调度链、网关密钥、上游源都存在本 config.yaml 里**，
|
||||
# 不在这个文件。runtime.json 只管模板与删除标记。
|
||||
runtime_file: runtime.json
|
||||
|
||||
# 全局并发上限（0 = 不限）
|
||||
max_concurrent: 0
|
||||
|
||||
# ---- 上游 LLM 源列表 ----
|
||||
# 留空：安装后到 WebUI 添加源，或从内置模板一键创建。
|
||||
sources: []
|
||||
@ -68,30 +68,10 @@ log "packaging tar.gz..."
|
||||
TAR_DIR="$DIST/llmsproxy-$VERSION-linux-amd64"
|
||||
rm -rf "$TAR_DIR"; mkdir -p "$TAR_DIR"
|
||||
cp "$BUILD_BIN" "$TAR_DIR/llmsproxy"
|
||||
# Use the tracked sanitised template, NOT the local config.yaml: that file is
|
||||
# gitignored and carries the operator's real gateway_keys / upstream keys. It
|
||||
# was previously copied verbatim as config.example.yaml, which shipped a real,
|
||||
# working admin key (sk-gw-local-0001) to every install — and postinst.sh copies
|
||||
# the example to /etc when none exists, so it ended up live in production.
|
||||
cp "$ROOT/packaging/config.example.yaml" "$TAR_DIR/config.example.yaml"
|
||||
cp "$ROOT/config.yaml" "$TAR_DIR/config.example.yaml"
|
||||
cp "$ROOT/packaging/llmsproxy.service" "$TAR_DIR/llmsproxy.service"
|
||||
mkdir -p "$TAR_DIR/adapters"
|
||||
cp "$ROOT"/internal/lua/adapters/*.lua "$TAR_DIR/adapters/"
|
||||
|
||||
# The shipped plugins go into the archive too, not just into the binary's
|
||||
# embedded FS.
|
||||
#
|
||||
# Seeding works without them (internal/lua/vm.go embeds plugins/*.lua and
|
||||
# writeBundledPlugins materialises them on first start — verified: a fresh
|
||||
# plugin_dir gains billing.lua, 70031 bytes, and the plugin loads), so this is
|
||||
# not a functional fix. It is a contents/claim mismatch: the v1.8.0 release
|
||||
# notes tell the operator the billing plugin ships with the core, and this
|
||||
# archive — the artefact they unpack and inspect — did not contain it. Shipping
|
||||
# the file also means an operator can read and adapt it before installing,
|
||||
# instead of only discovering it after the first boot writes it out.
|
||||
mkdir -p "$TAR_DIR/plugins"
|
||||
cp "$ROOT"/internal/lua/plugins/*.lua "$TAR_DIR/plugins/"
|
||||
|
||||
tar -C "$DIST" -czf "$DIST/llmsproxy-$VERSION-linux-amd64.tar.gz" \
|
||||
"$(basename "$TAR_DIR")"
|
||||
rm -rf "$TAR_DIR"
|
||||
@ -105,45 +85,6 @@ for f in "$DIST"/llmsproxy_*.deb "$DIST"/llmsproxy-*.rpm "$DIST"/*.tar.gz; do
|
||||
[ "$sz" -gt 100000 ] || { echo "[core-dist] FATAL: $f suspiciously small ($sz B)"; exit 1; }
|
||||
log " ✓ $f ($((sz/1024)) KB)"
|
||||
done
|
||||
# dpkg-deb takes ONE archive; every extra argument is read as an additional
|
||||
# control-component name. With the glob unquoted this used to expand to three
|
||||
# .deb files (1.5.9 / 1.6.0 / the one just built) and dpkg-deb exited 2 — under
|
||||
# `set -e` that aborted the script right here, so the "done" line never printed
|
||||
# and NO content verification below ever ran. The size gate above was the only
|
||||
# check that ever executed. Pick the newest deb explicitly.
|
||||
newest_deb=$(ls -t "$DIST"/llmsproxy_*.deb 2>/dev/null | head -1 || true)
|
||||
if [ -n "$newest_deb" ]; then
|
||||
dpkg-deb -I "$newest_deb" 2>/dev/null | grep -E 'Package|Version' | head -2 || true
|
||||
fi
|
||||
|
||||
# 6) the archive must actually carry what the release notes promise.
|
||||
#
|
||||
# A size floor alone cannot catch this: a tar.gz missing all 70KB of plugins
|
||||
# still clears the 100KB threshold easily, and the seeded-on-first-boot
|
||||
# behaviour hides it at runtime — the operator sees a working billing plugin and
|
||||
# concludes the package is complete. Assert the file is in the archive.
|
||||
log "verifying archive contents..."
|
||||
TARBALL="$DIST/llmsproxy-$VERSION-linux-amd64.tar.gz"
|
||||
[ -f "$TARBALL" ] || { echo "[core-dist] FATAL: $TARBALL missing"; exit 1; }
|
||||
listing=$(tar tzf "$TARBALL" 2>/dev/null || true)
|
||||
[ -n "$listing" ] || { echo "[core-dist] FATAL: cannot list $TARBALL"; exit 1; }
|
||||
for want in llmsproxy config.example.yaml llmsproxy.service plugins/billing.lua; do
|
||||
if ! printf '%s\n' "$listing" | grep -q "/$want\$"; then
|
||||
echo "[core-dist] FATAL: archive is missing $want"
|
||||
echo " (release notes claim the billing plugin ships with the core)"
|
||||
exit 1
|
||||
fi
|
||||
log " ✓ archive contains $want"
|
||||
done
|
||||
# Directories are matched by their CONTENTS, not by name: tar written with
|
||||
# `tar -czf` from a populated tree lists "…/adapters/openai.lua" and never a bare
|
||||
# "…/adapters" entry, so an earlier version of this check that grepped for
|
||||
# "/adapters$" reported a complete archive as broken.
|
||||
n_adapters=$(printf '%s\n' "$listing" | grep -c "/adapters/.*\.lua\$" || true)
|
||||
if [ "$n_adapters" -lt 10 ]; then
|
||||
echo "[core-dist] FATAL: archive has only $n_adapters adapters (expected >= 10)"
|
||||
exit 1
|
||||
fi
|
||||
log " ✓ archive contains $n_adapters adapters"
|
||||
dpkg-deb -I "$DIST"/llmsproxy_*.deb 2>/dev/null | grep -E 'Package|Version' | head -2
|
||||
|
||||
log "done: $DIST"
|
||||
@ -10,53 +10,15 @@ Type=simple
|
||||
# OS thread that touches malloc reserved its own ~1 MB arena that is never
|
||||
# returned. Measured: 8-12 arenas -> 0.
|
||||
# GOGC=50 halves the Go heap growth target. On its own it does NOT help (the
|
||||
# saved heap is immediately eaten by extra glibc arenas); combined with
|
||||
# saved heap is immediately eaten by more glibc arenas); combined with
|
||||
# MALLOC_ARENA_MAX it cut settled RSS by ~19%. This gateway is I/O bound, so
|
||||
# the extra GC cycles are free.
|
||||
Environment=GOGC=50
|
||||
Environment=MALLOC_ARENA_MAX=2
|
||||
ExecStart=/usr/local/bin/llmsproxy -config /etc/llmsproxy/config.yaml
|
||||
ExecStart=/usr/bin/llmsproxy -config /etc/llmsproxy/config.yaml
|
||||
WorkingDirectory=/etc/llmsproxy
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
# ---- 加固（2026-10-01 逐条实测后加入，不是照抄文档）----
|
||||
#
|
||||
# 为什么仍然以 root 运行：master.key 是 0600 root。加 User=llmsproxy 实测直接
|
||||
# 起不来，而且失败方式很隐蔽——
|
||||
# [config] secrets disabled: open /etc/llmsproxy/master.key: permission denied
|
||||
# 只是**一行日志**，服务会带着"敏感值将以明文落盘"继续跑起来。
|
||||
# 也就是说降权在当前文件权限下不是加固，而是把密钥降级。要降权必须先把
|
||||
# master.key 交给服务用户并统一 /etc/llmsproxy 的属主，那是一次独立的、有回滚
|
||||
# 需求的变更，不该和加固混在一起。
|
||||
#
|
||||
# 下面每一条都在一个独立探针单元（临时端口 + 独立 runtime_file/adapter_dir）
|
||||
# 上真实验证过：鉴权 401/200 正常、发一次真实 /v1/chat/completions 走通（证明
|
||||
# LuaJIT 适配器路径没被 seccomp 打断）、审计文件可写可轮转、连续重启 3 次与
|
||||
# kill -9 后行为符合预期。systemd 对非法指令值不报错只"忽略"，逐条实测是唯一
|
||||
# 可靠做法。
|
||||
NoNewPrivileges=yes
|
||||
# 读路径全部落在 /etc/llmsproxy；写路径经核对只有 config.yaml / runtime.json /
|
||||
# audit.jsonl / adapters/*.lua / master.key，全在该目录下（internal/{config,gateway,
|
||||
# core,lua} 里的 WriteFile|Rename|Remove 调用点）。
|
||||
ProtectSystem=strict
|
||||
ReadWritePaths=/etc/llmsproxy
|
||||
ProtectHome=yes
|
||||
PrivateTmp=yes
|
||||
ProtectKernelTunables=yes
|
||||
ProtectKernelModules=yes
|
||||
ProtectControlGroups=yes
|
||||
RestrictSUIDSGID=yes
|
||||
RestrictRealtime=yes
|
||||
LockPersonality=yes
|
||||
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
|
||||
# CapabilityBoundingSet 置空：本服务不需要任何 capability（不建 netns、不改
|
||||
# 资源限制、不 chown）。留空即"一个都不给"，比列一份允许清单更难写错。
|
||||
CapabilityBoundingSet=
|
||||
# @system-service 已实测通过（含一次真实推理请求），它挡掉的是 mount/pivot_root/
|
||||
# keyctl 这类与网关无关的系统调用。
|
||||
SystemCallFilter=@system-service
|
||||
SystemCallArchitectures=native
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@ -28,11 +28,8 @@ contents:
|
||||
dst: /etc/systemd/system/llmsproxy.service
|
||||
file_info:
|
||||
mode: 0644
|
||||
# example config (seeded to /etc/llmsproxy on first install by postinst).
|
||||
# Must be the tracked sanitised template: ./config.yaml is the operator's own
|
||||
# gitignored config and shipping it would leak real credentials — it shipped a
|
||||
# working admin key (sk-gw-local-0001) to every install for a while.
|
||||
- src: ./packaging/config.example.yaml
|
||||
# example config (seeded to /etc/llmsproxy on first install by postinst)
|
||||
- src: ./config.yaml
|
||||
dst: /usr/share/llmsproxy/config.example.yaml
|
||||
file_info:
|
||||
mode: 0644
|
||||
|
||||
@ -9,19 +9,9 @@ mkdir -p /etc/llmsproxy/adapters
|
||||
if [ -f /etc/llmsproxy/config.yaml ]; then
|
||||
echo "llmsproxy: keeping existing /etc/llmsproxy/config.yaml"
|
||||
else
|
||||
# Do NOT seed a starter config here. Packaging/installing one used to hand
|
||||
# everyone the same file, and while it was copied from the maintainer's
|
||||
# live config it shipped a real, working admin key (sk-gw-local-0001) to
|
||||
# every install.
|
||||
#
|
||||
# Instead, leave the file absent: on first start the binary's EnsureDefault
|
||||
# path creates config.yaml with a FRESH RANDOM admin key and logs it, which
|
||||
# is both unique per install and actually usable. The packaged
|
||||
# config.example.yaml stays in /usr/share as a reference template only.
|
||||
#
|
||||
# (Seeding it with gateway_keys: [] would be worse: the gateway would start
|
||||
# but reject every request, with no way in except editing the file.)
|
||||
echo "llmsproxy: no /etc/llmsproxy/config.yaml — a random admin key will be generated on first start"
|
||||
# seed a starter config from the packaged example
|
||||
cp -f /usr/share/llmsproxy/config.example.yaml /etc/llmsproxy/config.yaml 2>/dev/null || true
|
||||
chmod 0644 /etc/llmsproxy/config.yaml
|
||||
fi
|
||||
|
||||
# systemd
|
||||
|
||||
Reference in New Issue
Block a user