Two defects on the admin source write path, both found while adding a model
to a live source by hand.
PUT /api/sources/{name} was advertised in the API index but never
implemented — handleSourcesAPI only switched on GET/POST/DELETE, so the
documented update verb answered 405 while the POST upsert behind it worked.
POST is an upsert that replaces the whole source, so a partial edit that did
not carry api_key persisted an empty or placeholder credential. The source
kept its name, base_url and models, the write returned 200, and the source
then answered 401 on the next request — long after the writing script exited
0. The WebUI had been routing around this by loading the real key through
?reveal=credentials; any script or partial update went straight into it.
- implement PUT, taking the name from the path and rejecting a body name
that disagrees rather than silently resolving to one of them
- inherit the stored credential when api_key is omitted or sent as the
literal "__KEEP__"; an explicit new key still rotates
- an empty api_key on a source that does not exist yet stays empty, since
credential-less local upstreams are legitimate
- add model_ids, an additive shorthand, so "add these models" never has to
read and echo the existing list back
- align the API index with the implementation
The model_ids merge had a first cut that dropped the existing list when the
request carried no models field; TestSourceModelIDsIsAdditive caught it.
Verified by mutation: removing PUT turns three tests red, flattening
resolveAPIKey into a pass-through turns TestSourceUpsertKeepsAPIKey red
on both subtests, and making model_ids replace instead of merge turns
TestSourceModelIDsIsAdditive red.
A key with scope=[AUTO] could previously:
1. request ANY concrete model id directly (hasScopeModel/checkModelScope
treated AUTO as a wildcard)
2. see the full 56-model list on /v1/models (intersectModels considered
AUTO as grant-everything)
AUTO now only authorizes the AUTO routing mode. Direct requests to a
specific model require an explicit scope entry.
Also carries agentrouter.lua WAF fingerprint headers (Origin/Referer/
X-Requested-With) already staged on this branch.
Tests: TestHasScopeModelWithSourcePrefix updated; full suite green.
(cherry picked from commit 7fb8f96b82)
Adds a fourth accent alongside sakura/ocean/violet. Unlike those it is not just a
different hue: the coloured themes are glass surfaces (translucent cards with
backdrop-filter) floating over an animated gradient-mesh background, so setting
--card:#ffffff there still renders as a tinted grey. Mono therefore also switches
off the translucency and hides the blobs, so #ffffff is actually #ffffff and
#000000 is actually #000000, with greys carrying the hierarchy that hue carries
elsewhere. A side effect worth having: no backdrop-filter and no animated blobs
makes it the cheapest theme to render, which helps on weak GPUs and over remote
desktops.
Both light and dark variable blocks are defined, so the existing light/dark
toggle drives it with no extra wiring: light -> white, dark -> black.
Also fixes a latent theme bug found while checking contrast on black: the active
chart's grid baseline assigned the literal string "var(--line)" to
ctx.strokeStyle. Canvas 2D does not resolve CSS custom properties, so that was an
invalid colour the browser ignored, leaving the previous fillStyle (black) — an
invisible baseline on every dark theme. Colours used on a canvas now go through a
cssVar() helper.
Tests: TestUIThemeMatrix asserts every accent defines BOTH a light and a dark
block plus a picker button (a half-defined theme shows up as unreadable text, not
as an error); TestUIMonoThemeIsFlat pins the opaque surfaces and disabled blobs;
TestUICanvasColorsResolveVars fails if any ctx.strokeStyle/fillStyle is handed a
raw var().
Unrelated packaging fix in the same commit: dist:linux only built deb+AppImage
while build.linux.target listed rpm too, so `make gui-dist` silently skipped the
rpm that release builds are expected to produce. Makefile/README wording updated
to match.
The gateway still built Stats with NewStats(10000), a leftover from when the
ring WAS the source of the dashboard's numbers. With aggregates now computed
from the full audit history, a 10000-entry ring only means 10000 resident Req
structs — the live instance came up at 64 MB instead of ~40 MB, putting back
most of the memory the on-demand paging work removed.
The ring's only jobs are the status page's 5-minute SourceRecent /
SourceAverages windows and the dashboard's first screen, both of which fit in
defaultRingSize (500). Older records are paged from disk.
TestGatewayRingStaysSmall pins it so the constant cannot drift back up.
Two problems reported after the on-demand log work landed.
1. Dashboard totals were wrong. LoadAudit only replayed the last 4 MB of the
audit file, so requests/tokens/per-key rows reflected a window instead of all
time — a regression in reported numbers, not just in presentation.
The aggregates are now built by streaming EVERY audit file (oldest first, so
the hourly quota buckets keep their intended trailing window) and keeping
nothing per record: aggregate maps are keyed by key/model/source, so their
size is bounded by cardinality. Measured on the production host: 29 MB /
221k lines / 37k requests in ~260 ms at startup.
What stays bounded is the RAW-record ring: a fixed-size reqRing keeps only the
newest maxRecs records, so the ~25 MB that used to be spent appending every
record into a slice is still saved. auditReplayBytes is gone, and
replayPartial now means "an audit file could not be read", which is the only
remaining way for the totals to be incomplete.
2. Scrolling to the bottom stopped loading more records. Two independent causes:
* paintRecords rebuilt the entire table on every 5s poll whenever the row
count did not exceed the first screen — the "is a paged view live?" test
compared row counts and matched exactly on the first refresh — wiping loaded
pages and resetting scroll position.
* IntersectionObserver only fires on TRANSITIONS. With a short list, or after a
page whose rows all duplicated the first screen, the sentinel stayed visible
and never fired again.
paintRecords now builds once (recsState.built) and later polls PREPEND only
genuinely new rows; attachRecsObserver adds a scroll-position fallback;
fillRecordsViewport loads until the list actually overflows; and
loadMoreRecords chains (bounded) when a page yields no new rows, since the
first fetch necessarily overlaps the first screen.
TestUIRecordsPagingWiring pins all four mechanisms structurally, since none of
them is reachable from Go. Test names/comments referring to bounded replay are
updated to describe the bounded RING instead, and both READMEs now state that
totals come from the full history while records are paged.
Deleting a gateway key from the admin UI did nothing and reported
"use GET /api/keys": delKey() called api() with an empty options object, so
fetch defaulted to GET and the request landed in the GET branch of
handleKeysAPI. delAdapter() had the identical bug and reported
"adapter code not exposed; edit in UI".
This is the third instance of the same mistake — ab20f1b fixed delSource and
delTemplate, missing these two — so it is now pinned by tests instead of by
review:
* TestUIAPICallsDeclareMethod walks every api() call in the embedded
index.html and fails if one passes an options object without a method
(an AbortSignal-only read is allowed, being a deliberate GET).
* TestUIDeleteHelpersUseDelete / TestUIMutatingHelpersUseWriteMethods pin the
verb of each removal and write helper by name.
* TestKeyDeleteRoundTrip covers create -> DELETE -> gone -> second DELETE is a
clean 404, and TestCannotDeleteOwnKey keeps the lockout guard.
The 404 bodies for GET /api/keys/{key} and GET /api/adapters/{name} now name the
verb to use ("DELETE /api/keys/{key} to remove"), because that message is what a
mis-methoded client actually shows its user; "use GET /api/keys" read as though
the caller had done nothing wrong.
delSource's indentation, broken by ab20f1b, is also straightened out.
Front-end half of the on-demand log loading plus the observability for the two
new scheduling mechanisms.
Records table:
* the first screen comes from the dashboard poll, and an IntersectionObserver
sentinel below the last row pulls the next page from /api/stats/records as
the user scrolls.
* rendered rows are capped at RECS_MAX_DOM=1000 (oldest rendered rows are
dropped) so a long scroll cannot grow the DOM without bound, with a "back
to newest" button to reset cheaply.
* releaseRecords() drops the buffer, disconnects the observer and aborts the
in-flight fetch (AbortController) on tab switch, on key-filter change and
on pagehide/beforeunload — leaving the page releases everything at once.
* the poll no longer rebuilds the table once extra pages are loaded, so the
5s refresh cannot throw away scrolled history.
* when the server reports replay_partial, the filter row states that the
aggregates cover the recent audit tail and points at CSV for full history.
Adapters page shows each pool as created/max plus in_use/idle and the live
grow/shrink steps, with the sizing rule in the hover text.
Priority page health tags now distinguish cooling / probe-ready / probing
instead of a flat "cooling", and the tooltip spells out when the window opened,
when the single probe is allowed through and when the slot clears completely.
core.AutoSlotState carries cooldown_from / probe_after / probing / probeable to
feed this.
Startup RSS on this deployment was 56 MB with a 29 MB audit log and ~10 MB
without one: LoadAudit() json-unmarshalled the ENTIRE file into the aggregates
and kept a 10000-entry ring of raw records. Two more paths had the same shape —
AuditRecords() materialized a whole export window into a []Req before sorting
it, and a dashboard poll serialized the full ring so the browser could render
300 rows of it.
The audit file is now the source of truth and memory only holds the live
window:
* LoadAudit replays only the last auditReplayBytes (4 MB) and drops the
truncated first line; the ring default drops 10000 -> 500, which still
covers both of its consumers (the status page's 5-minute SourceRecent /
SourceAverages windows and the first screen of the records table).
replayPartial is exported so the UI can say the totals cover a window
rather than all time. Token-quota accounting is unaffected: it reads the
modelHour buckets, not the ring (pinned by a test).
* AuditPage(cursor, limit, key) pages records straight off disk, reading the
newest file backwards in 64 KB chunks and returning as soon as the page is
full. The cursor is "<file>:<offset>" and walks into rotated .old files;
a cursor whose file rotated away reports rotated=true so the client can
reset instead of silently skipping records. No state is cached between
requests and the file handle is closed before responding, so "release when
the user leaves the page" is guaranteed by never retaining anything.
* StreamAuditRecords(from,to,key,fn) replaces the accumulate-then-sort export
path; the CSV handler writes rows as they are read and flushes every 1000,
and a write error (client gone) aborts the walk. Export memory is O(1)
regardless of the window. AuditRecords is kept as a test-only wrapper.
* Snapshot ships one screen (firstScreenRecords=100) by default; aggregates
are untouched.
* Audit rotation 64 MB x 10 -> 16 MB x 16: same 256 MB total budget, but a
smaller newest file keeps the first reverse page cheap.
New route: GET /api/stats/records?before=&limit=&key= (non-admins are pinned to
their own key by exportKey). /api/status additionally reports adapter_pools for
admins.
Measured with production's 29 MB audit copied to the test instance: startup RSS
19.0 MB (was 56 MB); scrolling 10 pages (1000 records) +0.7 MB; exporting the
full history (36441 rows / 4.4 MB CSV) +0.1 MB with no residual growth.
gofmt -l reported 13 files with misaligned struct tags / stale formatting.
This commit contains ONLY formatting: no behaviour change, no logic touched.
Files that also carry real changes in this series are formatted by their own
commits.
delSource and delTemplate called api() without a method, so fetch
defaulted to GET; the DELETE handlers never ran and the UI silently
left the item in place.
A template stores every source field except name and api_key, so
operators spin up N key-bearing sources from one shared skeleton
instead of duplicating the whole source block N times.
- config: SourceTemplate type + RuntimeConfig.SourceTemplates stored
in runtime.json alongside runtime sources
- store: UpsertTemplate / ListTemplates / RemoveTemplate
- core: Templates / SaveTemplate / RemoveTemplate
- gateway: GET/POST/DELETE /api/source_templates
- webui: source list gains a Templates button opening a manager with
per-template edit/delete; the add-source dialog gains 'from
template' (event-delegated picker) and 'as template' (card modal)
buttons in its header; z-index fixed so the template editor layers
above the manager
Image models previously could not be scheduled through a priority
chain: the chat AUTO chain explicitly skips image-kind slots, and
AUTO image requests fell back to unordered registry discovery.
- config: add auto_image rules (auto_image yaml / image_rules json);
legacy auto rules keep their meaning as the chat chain
- core: buildAutoImageChain mirrors buildAutoChain with inverted kind
filter (image-only); SaveAutoImageRules + AutoImageRules/AutoImageChain
- scheduler: ChainImage walks the chain tier-by-tier with round-robin
and preference ordering, skipping cooling slots
- gateway: handleImage AUTO now runs down AutoImageChain when one is
configured (falls back to legacy discovery otherwise) and records
the actual served model; handleAutoAPI GET returns image_rules and
PUT accepts image_rules independently of rules
- webui: priority page gains a chat/image toggle editing two
independent lane sets; add-slot picker filters by active kind;
persistAuto writes only the active chain's field
The priority page and key-scope pickers skipped kind=image models,
so image sources (e.g. Kwai-Kolors/Kolors) could not be placed in
the AUTO chain or granted per-key. The backend already supports it:
handleImage resolves AUTO through the chain then filters with
imageOnly, while chat requests are protected by chatOnly, so image
slots never receive chat traffic.
Image models now appear in the priority canvas, the add-slot picker
(labelled ' (image)'), and the key scope dialog.
handleImage recorded the raw request model id, so AUTO image
generations showed up as model=AUTO in the request records and
by-model aggregates instead of the image model actually served
(e.g. Kwai-Kolors/Kolors).
UnifiedResponse gains an optional Model field; Provider.Image fills
it with the resolved id (AUTO resolves to the source's best image
model), and handleImage prefers it when writing the audit record.
6 call sites used the fetch default GET while sending a request body,
which browsers reject outright ('Request with GET/HEAD method cannot
have body'). Affected flows: create key, save source, save auto rules,
upload adapter, update key model scope, and /api/chat streaming.
All now send the method their backend handlers require (POST or PUT)
with an explicit Content-Type.
Prerequisite for removing the nginx global-auth layer in front of the
gateway: llmsproxy must defend its own login endpoint.
Design:
- per-IP failure counter: 5 consecutive failures trigger an exponential
lockout (30s base, doubling per extra burst, capped at 30min); 15min of
quiet forgives the counter
- global budget: max 100 failures/minute across all IPs so a distributed
spray cannot outrun per-IP windows
- locked-out and over-budget attempts get the SAME 'invalid gateway api
key' 401 as normal failures — no oracle to probe lockout state, no info
leak on key validity timing
- successful login clears the IP's counter entirely
- clientIP(): prefers X-Real-IP (trusted nginx proxy), falls back to
RemoteAddr host
Tests: lockout engages at threshold with identical replies, valid keys
rejected while locked, other IPs unaffected, success resets counters,
X-Real-IP extraction.
Live testing across the zen pool showed models report
prompt_tokens_details.cached_tokens even when the hit count is 0 (e.g.
nemotron-3-ultra-free returns cached_tokens:0, audio_tokens:0,
cache_write_tokens:0). The previous >0 guard dropped those objects, so a
cache-enabled upstream looked identical to one without cache support.
- types: PromptTokensDetails.CachedTokens always emitted (drop inner
omitempty) so clients see cached_tokens:0 explicitly; dsh reads it as
a 0% hit instead of 'no data'
- adapters (9): forward prompt_tokens_details whenever the upstream
provides it (presence check instead of >0)
- Req: add cache_reported flag set when usage carried cache accounting;
WebUI shows an amber 0% tag for reported-but-missed rows and keeps
the em-dash only for sources that never report cache data
- Req: add CacheHit and CacheMiss fields (carrying upstream cache
accounting from either prompt_tokens_details.cached_tokens or legacy
prompt_cache_hit_tokens)
- recordChatUsage (non-streaming): copy cache fields from resp.TokenUsage
- pumpStream (streaming): write lastUsage cache fields back onto rec at
stream end, so streaming requests carry cache data too
- CSV export: add first_byte_ms, cache_hit_tokens, cache_miss_tokens
columns alongside the existing latency/prompt/completion
- WebUI request-records table: add a Cache column showing hit% per row
(green/amber tag with tooltip hit/miss breakdown; em-dash when the
upstream reported no cache data)
- Req: add FirstByteMs field (ms to first byte, tracked for streaming)
- Stat: add FirstByteSum for aggregation
- SourceAverages(): new method computing per-source avg TTFB and tokens/s
from the in-memory ring (300s window)
- SourceStatus: add AvgFirstByteMs and AvgTokPerS fields
- pumpStream: record FirstByteMs after first SSE chunk sent to client
- singleChat/singleChatAuto: set FirstByteMs = LatMs (non-streaming)
- handleStatusAPI: populate the new SourceStatus fields from SourceAverages()
- WebUI source table: two new columns showing TTFB (s) and Tokens/s
dsh displays cache-hit %, but llmsproxy dropped every upstream's cache
fields — deepseek prompt_cache_hit_tokens, OpenAI prompt_tokens_details.
cached_tokens, anthropic cache_read_input_tokens, gemini cachedContentTokenCount.
Changes:
- TokenUsage: add PromptTokensDetails (with CachedTokens) + PromptCacheHit/Miss
- MarshalJSON: emit prompt_tokens_details.cached_tokens (OpenAI v2 standard)
and prompt_cache_hit/miss_tokens (DeepSeek legacy) — dsh reads the former
first, falls back to the latter
- mergeUsage: preserve cache fields across stream chunks
- standardSSEChunk: parse the upstream raw prompt_tokens_details too
- deepseek.lua: forward prompt_cache_hit/miss_tokens + create
prompt_tokens_details from them
- openai.lua: forward prompt_tokens_details.cached_tokens and legacy
prompt_cache_hit/miss_tokens; normalize legacy hits into the standard
object so dsh sees them regardless of upstream format
- anthropic.lua: map cache_read_input_tokens → prompt_tokens_details
- gemini.lua: map cachedContentTokenCount → prompt_tokens_details
WebUI (internal/gateway/ui):
- BUG: the export modal's custom-range button called
downloadStatsCsvFromForm() which was never defined — clicking it threw a
ReferenceError and nothing downloaded. Implement it: reads #exp-from /
#exp-to date inputs and forwards to downloadStatsCsv.
- BUG-adjacent: clearChat() existed but was reachable from no control —
add a Clear button to the chat composer so conversation reset is actually
possible (+ cClear i18n zh/en).
- remove byte-identical duplicate html[data-theme=dark] CSS block (15 lines)
- remove 8 dead CSS rules (.keys-grid .m-model-row .scope-add/.scope-box/
.scope-chips .scr-blocks .tag-warn .twrap) and the never-consumed
--accent custom property
- remove 3 dead JS functions (activeTab/findSlots/scopeUncomb; lastTab decl kept)
- remove 24 dead i18n keys x zh/en (~55 lines) — legacy of the replaced
key-scope editor, matching the removed .scope-* styles
GUI (cmd/gui/main.js):
- BUG: stopCore() set app.isQuitting=true and nothing reset it — after using
tray 'stop core', closing the window quit the whole app instead of hiding
to tray, and core crash auto-restart stayed disabled. isQuitting now only
flips in restartCore (scoped) and before-quit.
Verified: go vet/test green; node --check on all three GUI js files and the
WebUI inline script.
- extract csvHeaders() (Content-Type + Content-Disposition) shared by both
export branches
- extract exportKey(): the identical admin/user key-filter logic existed
twice (JSON path + keys-csv); now all three call sites share one function
- inline the nine single-use intermediate variables in the keys-csv loop
singleChat / singleChatAuto / streamChat / streamChatAuto shared ~260 near-
identical lines (diff after stripping comments was empty). Extract three
shared bodies and shrink all four entry points to thin dispatchers:
- failChat: error → record + writeError, shared ChainErr extraction
(errors.As returns false for direct-path errors, so the two paths stay
equivalent without a branch)
- recordChatUsage: exact upstream numbers win, byte estimates fill gaps
- writeChatCompletion: unified ChatCompletion rendering (model name is the
only direct/auto difference, passed in)
- pumpStream: the full SSE pump (preamble, delta loop, terminal finish +
usage chunk, [DONE]) — shared by both streaming entries
Behavior change (pinned by TestDirectStreamFailoverAuditSource): direct
streams now pin rec.Source to the source that actually served the stream
after a failover, instead of discarding it (_, usedModel). The audit row
previously recorded the first candidate, which was wrong on failover.
chat.go: 982 → 896 lines (−86).
provider.oneLineStr and gateway.oneLine had byte-identical bodies (flatten
whitespace + cap length). Move the single implementation into the types
package, which both layers already depend on, and delete both locals.
- provider.truncate: zero callers (oneLineStr is the used superset)
- gateway.normalizeModel: zero callers
- config.resolvedAPIKey: zero callers (core.resolveSourceKey is the live equivalent)
- Stats.Records: zero callers (CSV export uses AuditRecords)
- store.containsString: zero callers
- Config.MaxConcurrent: global inflight-cap field never read; per-source
MaxConcurrent is what actually drives semaphores. Legacy configs carrying a
top-level max_concurrent key still load (yaml.v3 ignores unknown fields —
verified by test).
- gui renderer esc(): zero callers; renderer uses textContent, and the embedded
WebUI has its own esc()
statusRecorder (the access-log wrapper) did not implement http.Flusher, so
w.(http.Flusher) inside streamChat/streamChatAuto returned nil and every SSE
chunk stayed buffered until the response ended. Add Flush() that delegates to
the underlying writer when it supports flushing.
Regression test: TestStatusRecorderFlusher pins the interface assertion and
the forwarding path.
WindowTokens buckets are whole unix hours, so a fixture built from raw
offsets like "1 minute ago" landed in the previous hour bucket whenever
the suite ran in the first minute of an hour — the 1h-window assertion
then saw 0 tokens and failed. Anchor rows to hour boundaries instead
(H-2 / H-1 / current bucket) and assert the exact current-bucket value;
the suite is now deterministic for any run time.
Clients saw the full per-tier chain error including upstream HTML WAF
pages and JSON quota payloads. The response now carries one capped
one-line reason per tier (quota/cooling skips preserved); full detail
remains in rec.Err / stats API and is logged server-side.
The gateway hardcoded "stop" on every terminating stream chunk, so
tool-call rounds reported finish_reason=stop and length caps were
invisible to clients. UnifiedChunk now carries finish_reason; adapters
emit it (with empty-string finish reasons like sensenova treated as
non-terminal), standardSSEChunk passes it through for un-adapted
upstreams, [DONE] no longer emits a duplicate reason-less done chunk,
and both streaming paths emit the real reason with "stop" as fallback.
Also vendor sensenova/agentrouter adapters into the repo: they were
WebUI-only uploads and a deploy sync silently removed them while live
AUTO-chain slots still referenced them.
The prior usage-passthrough fix only covered openai/opencode; the same
empty-choices+usage drop bug remained in the 5 sibling OpenAI-compatible
adapters, and non-OpenAI providers (anthropic/gemini/ollama) never surfaced
streaming usage at all.
- deepseek/github/groq/kimicode/mistral: preserve usage on empty-choices
chunks and attach it to normal chunks (same pattern as openai.lua)
- anthropic: emit usage from message_start (prompt) and message_delta
(completion); gateway merges split usage additively
- gemini: read usageMetadata in the stream path
- ollama: fix non-streaming key (usage -> token_usage, matches
UnifiedResponse json tag) and read prompt_eval_count/eval_count;
surface counts from the done stream chunk
- gateway: mergeUsage combines usage across chunks (non-zero fields win,
total recomputed from prompt+completion) so split usage doesn't lose
the prompt half; single-chunk case (OpenAI) preserved exactly
- usage-only chunks: done=false (no redundant terminal stop), matching
the Go fallback standardSSEChunk
Streaming responses now carry the upstream's real token usage instead of
gateway estimates:
- UnifiedChunk gains an optional Usage field; adapters (opencode, openai)
extract usage from upstream stream chunks (including the final chunk with
empty choices) and pass it through.
- standardSSEChunk preserves usage for passthrough adapters.
- Gateway emits the exact usage in the final stream chunk when available,
falling back to estimates only when the upstream provided none.
Non-streaming usage was already fixed to emit OpenAI-standard keys.
