- Req: add CacheHit and CacheMiss fields (carrying upstream cache
accounting from either prompt_tokens_details.cached_tokens or legacy
prompt_cache_hit_tokens)
- recordChatUsage (non-streaming): copy cache fields from resp.TokenUsage
- pumpStream (streaming): write lastUsage cache fields back onto rec at
stream end, so streaming requests carry cache data too
- CSV export: add first_byte_ms, cache_hit_tokens, cache_miss_tokens
columns alongside the existing latency/prompt/completion
- WebUI request-records table: add a Cache column showing hit% per row
(green/amber tag with tooltip hit/miss breakdown; em-dash when the
upstream reported no cache data)
- Req: add FirstByteMs field (ms to first byte, tracked for streaming)
- Stat: add FirstByteSum for aggregation
- SourceAverages(): new method computing per-source avg TTFB and tokens/s
from the in-memory ring (300s window)
- SourceStatus: add AvgFirstByteMs and AvgTokPerS fields
- pumpStream: record FirstByteMs after first SSE chunk sent to client
- singleChat/singleChatAuto: set FirstByteMs = LatMs (non-streaming)
- handleStatusAPI: populate the new SourceStatus fields from SourceAverages()
- WebUI source table: two new columns showing TTFB (s) and Tokens/s
dsh displays cache-hit %, but llmsproxy dropped every upstream's cache
fields — deepseek prompt_cache_hit_tokens, OpenAI prompt_tokens_details.
cached_tokens, anthropic cache_read_input_tokens, gemini cachedContentTokenCount.
Changes:
- TokenUsage: add PromptTokensDetails (with CachedTokens) + PromptCacheHit/Miss
- MarshalJSON: emit prompt_tokens_details.cached_tokens (OpenAI v2 standard)
and prompt_cache_hit/miss_tokens (DeepSeek legacy) — dsh reads the former
first, falls back to the latter
- mergeUsage: preserve cache fields across stream chunks
- standardSSEChunk: parse the upstream raw prompt_tokens_details too
- deepseek.lua: forward prompt_cache_hit/miss_tokens + create
prompt_tokens_details from them
- openai.lua: forward prompt_tokens_details.cached_tokens and legacy
prompt_cache_hit/miss_tokens; normalize legacy hits into the standard
object so dsh sees them regardless of upstream format
- anthropic.lua: map cache_read_input_tokens → prompt_tokens_details
- gemini.lua: map cachedContentTokenCount → prompt_tokens_details
WebUI (internal/gateway/ui):
- BUG: the export modal's custom-range button called
downloadStatsCsvFromForm() which was never defined — clicking it threw a
ReferenceError and nothing downloaded. Implement it: reads #exp-from /
#exp-to date inputs and forwards to downloadStatsCsv.
- BUG-adjacent: clearChat() existed but was reachable from no control —
add a Clear button to the chat composer so conversation reset is actually
possible (+ cClear i18n zh/en).
- remove byte-identical duplicate html[data-theme=dark] CSS block (15 lines)
- remove 8 dead CSS rules (.keys-grid .m-model-row .scope-add/.scope-box/
.scope-chips .scr-blocks .tag-warn .twrap) and the never-consumed
--accent custom property
- remove 3 dead JS functions (activeTab/findSlots/scopeUncomb; lastTab decl kept)
- remove 24 dead i18n keys x zh/en (~55 lines) — legacy of the replaced
key-scope editor, matching the removed .scope-* styles
GUI (cmd/gui/main.js):
- BUG: stopCore() set app.isQuitting=true and nothing reset it — after using
tray 'stop core', closing the window quit the whole app instead of hiding
to tray, and core crash auto-restart stayed disabled. isQuitting now only
flips in restartCore (scoped) and before-quit.
Verified: go vet/test green; node --check on all three GUI js files and the
WebUI inline script.
- extract csvHeaders() (Content-Type + Content-Disposition) shared by both
export branches
- extract exportKey(): the identical admin/user key-filter logic existed
twice (JSON path + keys-csv); now all three call sites share one function
- inline the nine single-use intermediate variables in the keys-csv loop
singleChat / singleChatAuto / streamChat / streamChatAuto shared ~260 near-
identical lines (diff after stripping comments was empty). Extract three
shared bodies and shrink all four entry points to thin dispatchers:
- failChat: error → record + writeError, shared ChainErr extraction
(errors.As returns false for direct-path errors, so the two paths stay
equivalent without a branch)
- recordChatUsage: exact upstream numbers win, byte estimates fill gaps
- writeChatCompletion: unified ChatCompletion rendering (model name is the
only direct/auto difference, passed in)
- pumpStream: the full SSE pump (preamble, delta loop, terminal finish +
usage chunk, [DONE]) — shared by both streaming entries
Behavior change (pinned by TestDirectStreamFailoverAuditSource): direct
streams now pin rec.Source to the source that actually served the stream
after a failover, instead of discarding it (_, usedModel). The audit row
previously recorded the first candidate, which was wrong on failover.
chat.go: 982 → 896 lines (−86).
provider.oneLineStr and gateway.oneLine had byte-identical bodies (flatten
whitespace + cap length). Move the single implementation into the types
package, which both layers already depend on, and delete both locals.
- provider.truncate: zero callers (oneLineStr is the used superset)
- gateway.normalizeModel: zero callers
- config.resolvedAPIKey: zero callers (core.resolveSourceKey is the live equivalent)
- Stats.Records: zero callers (CSV export uses AuditRecords)
- store.containsString: zero callers
- Config.MaxConcurrent: global inflight-cap field never read; per-source
MaxConcurrent is what actually drives semaphores. Legacy configs carrying a
top-level max_concurrent key still load (yaml.v3 ignores unknown fields —
verified by test).
- gui renderer esc(): zero callers; renderer uses textContent, and the embedded
WebUI has its own esc()
statusRecorder (the access-log wrapper) did not implement http.Flusher, so
w.(http.Flusher) inside streamChat/streamChatAuto returned nil and every SSE
chunk stayed buffered until the response ended. Add Flush() that delegates to
the underlying writer when it supports flushing.
Regression test: TestStatusRecorderFlusher pins the interface assertion and
the forwarding path.
WindowTokens buckets are whole unix hours, so a fixture built from raw
offsets like "1 minute ago" landed in the previous hour bucket whenever
the suite ran in the first minute of an hour — the 1h-window assertion
then saw 0 tokens and failed. Anchor rows to hour boundaries instead
(H-2 / H-1 / current bucket) and assert the exact current-bucket value;
the suite is now deterministic for any run time.
Clients saw the full per-tier chain error including upstream HTML WAF
pages and JSON quota payloads. The response now carries one capped
one-line reason per tier (quota/cooling skips preserved); full detail
remains in rec.Err / stats API and is logged server-side.
The gateway hardcoded "stop" on every terminating stream chunk, so
tool-call rounds reported finish_reason=stop and length caps were
invisible to clients. UnifiedChunk now carries finish_reason; adapters
emit it (with empty-string finish reasons like sensenova treated as
non-terminal), standardSSEChunk passes it through for un-adapted
upstreams, [DONE] no longer emits a duplicate reason-less done chunk,
and both streaming paths emit the real reason with "stop" as fallback.
Also vendor sensenova/agentrouter adapters into the repo: they were
WebUI-only uploads and a deploy sync silently removed them while live
AUTO-chain slots still referenced them.
The prior usage-passthrough fix only covered openai/opencode; the same
empty-choices+usage drop bug remained in the 5 sibling OpenAI-compatible
adapters, and non-OpenAI providers (anthropic/gemini/ollama) never surfaced
streaming usage at all.
- deepseek/github/groq/kimicode/mistral: preserve usage on empty-choices
chunks and attach it to normal chunks (same pattern as openai.lua)
- anthropic: emit usage from message_start (prompt) and message_delta
(completion); gateway merges split usage additively
- gemini: read usageMetadata in the stream path
- ollama: fix non-streaming key (usage -> token_usage, matches
UnifiedResponse json tag) and read prompt_eval_count/eval_count;
surface counts from the done stream chunk
- gateway: mergeUsage combines usage across chunks (non-zero fields win,
total recomputed from prompt+completion) so split usage doesn't lose
the prompt half; single-chunk case (OpenAI) preserved exactly
- usage-only chunks: done=false (no redundant terminal stop), matching
the Go fallback standardSSEChunk
Streaming responses now carry the upstream's real token usage instead of
gateway estimates:
- UnifiedChunk gains an optional Usage field; adapters (opencode, openai)
extract usage from upstream stream chunks (including the final chunk with
empty choices) and pass it through.
- standardSSEChunk preserves usage for passthrough adapters.
- Gateway emits the exact usage in the final stream chunk when available,
falling back to estimates only when the upstream provided none.
Non-streaming usage was already fixed to emit OpenAI-standard keys.
- TokenUsage.MarshalJSON now emits both standard (prompt_tokens,
completion_tokens, total_tokens) and legacy (prompt, completion, total)
keys, so OpenAI-compatible clients (DSH, DevEco Code, etc.) can read
token usage.
- ChatChunk gains an optional Usage field; stream responses now send a
final usage chunk (empty choices) before [DONE].
Refs: usage not visible in clients because the gateway serialized only the
internal short keys and never emitted a streaming usage chunk.
- api/status sources now carry recent_ok/recent_err (last 300s real gateway
requests via Stats.SourceRecent) so a source actually serving traffic is
never shown as down just because probe /models got rate-limited
- WebUI source status column repaints every 5s (no more frozen-at-first-
render) with a manual refresh button; shows success rate + probe + cooldown
- tray menu status rows were enabled:false (GTK fixed light-grey, invisible
on light themes) — now enabled with no-op click and nativeTheme listener
rebuilds the menu on dark/light switches
- ignore local ops scripts (scripts/, machine-specific)
- adapters/opencode.lua: opencode.ai zen free pool adapter — sends the
opencode client User-Agent (zen fingerprints clients by UA; non-official
clients hit FreeUsageLimitError); pairs with api_key: public
- config: no config file ships in the repo; first run generates a default
config at the -config path with a random admin key, loopback listen and a
keyless zen source (config.EnsureDefault); remove config.example.yaml
- lua: seed bundled adapters from the embedded FS instead of a hardcoded
name list
- ui: widen model kind select (chat was clipped to 'cha')
- phase 5 bugfixes: stats ms/s bucket mixing, cleanScopes nil, ctx.Err
guards, direct-path ModelAvailable, empty stream body failure,
bestImageModel rewrite, transform failure recording, Core.mu, timer,
effective model for tool-calls
- Move all config (auto rules, keys) from runtime.json to config.yaml
- Store now only holds runtime sources (WebUI-created)
- Add OpenRouter free models to config.yaml
- One-time migration from legacy runtime.json on startup
- Fix gateway tests for new config structure
- Update core.go with migrateFromRuntime, saveConfig, seedKeys/seedAuto
- Remove SaveKey/KeyByValue/AutoRules from Store
- Add Config.Save() with YAML marshaling
- Update WebUI admin keys visibility (show all keys including admin)
- Bump binary to 11MB with luajit
The sidebar's backdrop-filter: blur() made the popup sidebar itself
unreadable on ≤900px screens. Disabled the blur in the responsive
media query; now sidebar has solid glass background and the
full-screen blur only applies to the backdrop behind it.
renderStatus() re-injected the entire status tab (#tab-status.innerHTML) + all charts + tables on every visit, so each tab switch back to home stuttered. Now the page is built once (pane.dataset.built) and revisits only reload live data via paintStats() + restart the 3s poll. Static parts (source table, model chips, connection box) are not rebuilt. Deployed (bak .bak.20260811y), server active.
- .kpi fixed height 206px was too short for the tokens card (hdr + big value + two-line in/out + chart + legend) -> text overflowed; bumped to 252px with legend capped at max-height/scroll
- tokens chart: stack shows top-4 slices + lumped 'other', total uses the FULL model set (true shares, was relative to top-4); legend lists every model that has non-zero tokens (was hardcoded top-4, so small/zero slices were dropped -> 'only two models' visible)
- add kOther zh/en
- deployed (bak .bak.20260811x), server active
Previously the skeleton was a fixed 186px but the real cards had different natural heights (status pie 110 + legend, tokens two-line sub + legend, etc.), so replacing the skeleton caused a size jump. Now every .kpi is a fixed 206px flex column; .k-chart absorbs the remaining space (flex:1) and the canvas absolutely fills it (draw fns read container height, not a hardcoded 86/110). Skeleton .ksk is exactly 206px with the same layout, so first paint and real cards are identical in size. Deployed (bak .bak.20260811w), server active.
- KPI skeleton (.ksk) min-height now matches the real card (186px) with a title/value/chart bar layout so first paint doesn't jump
- global entry animations: cards fade+slide-up (fadeUp), tab panes fade+rise (tabIn), chat empty state fades in
- goTab restarts the tabIn animation on switch (reset animation + forced reflow)
- table-row animation intentionally NOT added (those re-render every 3s poll and would flicker)
- deployed (bak .bak.20260811v), server active
The five KPI cards are built only after /api/stats returns; before that #kpi-row was empty, so the top of the dashboard appeared blank then jumped in. Now the row starts with a shimmer skeleton (5 placeholder cards with a CSS gradient animation) that paintStats replaces with the real cards on first data. Deployed (bak .bak.20260811u), server active.
paintStats() rewrote #kpi-row innerHTML on every 3s refresh, destroying/recreating all five cards + canvases each time -> each poll flashed. Now the card structure is built once (dataset.built guard) and subsequent polls only update the value text (data-kpi attrs) and redraw the canvases. Preserves colored success rate and two-line in/out tokens. Deployed (bak .bak.20260811t), server active.
ensureProbe() called Registry.ProbeAll() synchronously on every status request when >30s had passed since the last probe, blocking the response on the slowest upstream (zen ~1s, frank ~1.9s, total ~1.5s) — this stalled WebUI page loads and tab switches. Now the probe runs in a background goroutine (15s budget); status returns in ~7ms and the 3s stats poll picks up updated reachability next tick. Deployed (bak .bak.20260811s), server active.
- chat tab model select: list each model grouped by source as 'source · model' with value 'source:model' (the unambiguous pin syntax — all 18 real models contain '-' but none contain ':'), so same-named models on different sources are distinguishable in tests; falls back to flat model list for non-admin (sources not exposed)
- tokens card: remove on-chart text, add color-swatch legend (#tb-tokens-legend) mapping bar color -> model; in/out tokens on separate lines (.k-sub.k2)
- deployed (bak .bak.20260811r), server active
The center 'hole' was faked by overpainting a small circle with var(--card), which exposed a yellow/warm card background blob and a visible seam. Now each slice is a real ring sector (outer+inner arc, reverse) so the center is genuinely transparent and matches the card. Deployed (bak .bak.20260811p), server active.
- remove the total-req number drawn in the pie center
- carve a center hole (donut) with the card background for a cleaner look; total still shown in the card header caption and legend
- deployed (bak .bak.20260811o), server active
- drawActiveChart: rolling request-rate buffer (window.qpsSeries) sampled ~every 2s from records timestamps, drawn as an area+fine; header shows current active requests + 'reqs / 3s' subtitle; refreshes on the existing 3s paintStats poll
- status pie: remove the hard slice outline (no stroke), clean seamless slices
- active card now consistent with others: big value + subtitle + chart
- deployed (bak .bak.20260811n), server active
- rewrite drawStatusChart to a Canvas pie (no per-slice labels), center shows total requests
- status card: title + total caption + tall pie area (110px) + color legend row below (swatch + code + count), each item has title tooltip with ok/err
- store status swatch palette and reuse across pie/legend for color consistency
- deployed (bak .bak.20260811m), server active
Redo the KPI row properly: five identical cards each with title row, large metric value, subtitle, and an 86px canvas chart. Replaces the previous gray-box canvas placeholders (which dropped the numbers — a regression).
- Active requests: line sparkline (real-time refresh) + live count
- Total requests: 7-day vertical bar chart + current total
- Tokens: stacked bar of per-model token share + big total
- Avg latency: line sparkline (real-time refresh) + avg ms
- Status codes: stacked bar by code + chip subtitle (code/reqs/percent)
All charts drawn with plain Canvas 2D (sparkline/dailyBars helpers), no third-party lib. DPR-aware for crispness. Deployed (bak .bak.20260811l), server active.
- move status distribution out of the full-width standalone card into the top KPI row as a 5th small block (aligned with the four KPI boxes)
- swap stroke-dasharray circles for SVG path arcs inside a rotated <g>: keeps each slice crisp and prevents the thick stroke clipping against the viewBox edge
- compact 108px donut + wrap-able status chip legend (hover title shows code/reqs/percent)
- drop redundant dashStatus subtitle; simplify title
- deployed to /usr/local/bin/llmsproxy (bak .bak.20260811j), server active
- paintStatusTable: replace plain table with SVG ring (conic slices per status code, center total, clickable legend with title=value/reqs/percent/ok/err)
- paintModelTable: percentage now = model share of total requests (was relative-to-max, so the top model always showed 100%/97%); bar stays max-relative
- plan.md: record P4-9 (visualization) and P4-10 (claude 400 'extra usage' = Anthropic account quota, not fingerprint; qijiar+openai serves claude fine, no adapter disguise needed)
- deployed to /usr/local/bin/llmsproxy (bak .bak.20260811i), server active
- api.go: handleSourcesAPI now requires admin role (GET leaks upstream api_keys, POST/DELETE mutate routing)
- chat.go: hasScopeModel made a Gateway method that strips source-model/:// prefix strictly via Registry.EffectiveModel (only when the prefix names a real source serving the bare model) so dash-bearing ids like deepseek-v4-flash-free are never corrupted; +TestHasScopeModelWithSourcePrefix
- config.go: DefaultSourceTimeout/QueueTimeout/Concurrency constants shared by YAML ApplyDefaults and runtime sources
- core.go: mergedSources applies the same defaults to runtime sources (JSON never persisted timeout fields); a dead upstream can no longer hold a concurrency slot forever
- provider.go: split non-streaming client{Timeout} vs stream client{} sharing a Transport with ResponseHeaderTimeout, so long SSE bodies are not cut by client.Timeout; ChatStream uses doRawStream
- plan.md: mark P4-4/5/6 done, record P4-7/8 (tier-order, audit export, UI key view, zen upstream diagnosis)
- online verified: user key -> /api/sources 403 (GET+POST), admin 200, AUTO stream/non-stream healthy
- scheduler: BuildChain sorts tiers ascending so tier 1 (highest priority) is tried first; previously descending inverted the chain (P10-1..P10-5)
- stats/api: handleStatsAPI limit→20000; CSV reads full audit via new Stats.AuditRecords (includes *.old rotation); key_names mapped by keyID() masked key; non-admin filter and keys-csv use masked keys
- server: ring buffer NewStats(10000)
- ui: dash-row card scroll area moved to table container (fix overflow below card); key view toggle (re-click same key returns to global) + kpi-exit affordance; rec-exit span
- chat: chain-failure record now surfaces first failed tier; AUTO comment sync
