Live testing across the zen pool showed models report
prompt_tokens_details.cached_tokens even when the hit count is 0 (e.g.
nemotron-3-ultra-free returns cached_tokens:0, audio_tokens:0,
cache_write_tokens:0). The previous >0 guard dropped those objects, so a
cache-enabled upstream looked identical to one without cache support.
- types: PromptTokensDetails.CachedTokens always emitted (drop inner
omitempty) so clients see cached_tokens:0 explicitly; dsh reads it as
a 0% hit instead of 'no data'
- adapters (9): forward prompt_tokens_details whenever the upstream
provides it (presence check instead of >0)
- Req: add cache_reported flag set when usage carried cache accounting;
WebUI shows an amber 0% tag for reported-but-missed rows and keeps
the em-dash only for sources that never report cache data
Live testing proved both sensenova and zen DO return cache fields:
- zen laguna-s-2.1-free: usage.prompt_tokens_details.cached_tokens = 32
(real hit), plus cache_write_tokens/audio_tokens
- sensenova glm-5.2: prompt_tokens_details.cached_tokens present (0 on
short prompts)
The previous round only patched deepseek/openai/anthropic/gemini.lua;
sensenova/opencode (localzen!) and the other adapters still dropped them.
- sensenova/opencode/groq/mistral/github/kimicode: stream + response
cache passthrough (same pattern as openai.lua)
- agentrouter: response passthrough + NEW stream usage forwarding (it
previously dropped the terminal usage-only chunk entirely)
- ollama skipped intentionally: its native API has no cache fields
Verified end-to-end through the gateway: localzen/laguna-s-2.1-free now
returns prompt_tokens_details.cached_tokens=32 to clients, and the request
record carries cache_hit_tokens (both chat and stream paths).
- Req: add CacheHit and CacheMiss fields (carrying upstream cache
accounting from either prompt_tokens_details.cached_tokens or legacy
prompt_cache_hit_tokens)
- recordChatUsage (non-streaming): copy cache fields from resp.TokenUsage
- pumpStream (streaming): write lastUsage cache fields back onto rec at
stream end, so streaming requests carry cache data too
- CSV export: add first_byte_ms, cache_hit_tokens, cache_miss_tokens
columns alongside the existing latency/prompt/completion
- WebUI request-records table: add a Cache column showing hit% per row
(green/amber tag with tooltip hit/miss breakdown; em-dash when the
upstream reported no cache data)
- Req: add FirstByteMs field (ms to first byte, tracked for streaming)
- Stat: add FirstByteSum for aggregation
- SourceAverages(): new method computing per-source avg TTFB and tokens/s
from the in-memory ring (300s window)
- SourceStatus: add AvgFirstByteMs and AvgTokPerS fields
- pumpStream: record FirstByteMs after first SSE chunk sent to client
- singleChat/singleChatAuto: set FirstByteMs = LatMs (non-streaming)
- handleStatusAPI: populate the new SourceStatus fields from SourceAverages()
- WebUI source table: two new columns showing TTFB (s) and Tokens/s
When only the legacy DeepSeek fields (prompt_cache_hit_tokens) are set but
the OpenAI-standard prompt_tokens_details is nil, MarshalJSON now auto-
generates the nested object. This ensures dsh (which reads the standard
format first) sees cache hit data regardless of which adapter format the
upstream uses.
dsh displays cache-hit %, but llmsproxy dropped every upstream's cache
fields — deepseek prompt_cache_hit_tokens, OpenAI prompt_tokens_details.
cached_tokens, anthropic cache_read_input_tokens, gemini cachedContentTokenCount.
Changes:
- TokenUsage: add PromptTokensDetails (with CachedTokens) + PromptCacheHit/Miss
- MarshalJSON: emit prompt_tokens_details.cached_tokens (OpenAI v2 standard)
and prompt_cache_hit/miss_tokens (DeepSeek legacy) — dsh reads the former
first, falls back to the latter
- mergeUsage: preserve cache fields across stream chunks
- standardSSEChunk: parse the upstream raw prompt_tokens_details too
- deepseek.lua: forward prompt_cache_hit/miss_tokens + create
prompt_tokens_details from them
- openai.lua: forward prompt_tokens_details.cached_tokens and legacy
prompt_cache_hit/miss_tokens; normalize legacy hits into the standard
object so dsh sees them regardless of upstream format
- anthropic.lua: map cache_read_input_tokens → prompt_tokens_details
- gemini.lua: map cachedContentTokenCount → prompt_tokens_details
WebUI (internal/gateway/ui):
- BUG: the export modal's custom-range button called
downloadStatsCsvFromForm() which was never defined — clicking it threw a
ReferenceError and nothing downloaded. Implement it: reads #exp-from /
#exp-to date inputs and forwards to downloadStatsCsv.
- BUG-adjacent: clearChat() existed but was reachable from no control —
add a Clear button to the chat composer so conversation reset is actually
possible (+ cClear i18n zh/en).
- remove byte-identical duplicate html[data-theme=dark] CSS block (15 lines)
- remove 8 dead CSS rules (.keys-grid .m-model-row .scope-add/.scope-box/
.scope-chips .scr-blocks .tag-warn .twrap) and the never-consumed
--accent custom property
- remove 3 dead JS functions (activeTab/findSlots/scopeUncomb; lastTab decl kept)
- remove 24 dead i18n keys x zh/en (~55 lines) — legacy of the replaced
key-scope editor, matching the removed .scope-* styles
GUI (cmd/gui/main.js):
- BUG: stopCore() set app.isQuitting=true and nothing reset it — after using
tray 'stop core', closing the window quit the whole app instead of hiding
to tray, and core crash auto-restart stayed disabled. isQuitting now only
flips in restartCore (scoped) and before-quit.
Verified: go vet/test green; node --check on all three GUI js files and the
WebUI inline script.
- extract csvHeaders() (Content-Type + Content-Disposition) shared by both
export branches
- extract exportKey(): the identical admin/user key-filter logic existed
twice (JSON path + keys-csv); now all three call sites share one function
- inline the nine single-use intermediate variables in the keys-csv loop
singleChat / singleChatAuto / streamChat / streamChatAuto shared ~260 near-
identical lines (diff after stripping comments was empty). Extract three
shared bodies and shrink all four entry points to thin dispatchers:
- failChat: error → record + writeError, shared ChainErr extraction
(errors.As returns false for direct-path errors, so the two paths stay
equivalent without a branch)
- recordChatUsage: exact upstream numbers win, byte estimates fill gaps
- writeChatCompletion: unified ChatCompletion rendering (model name is the
only direct/auto difference, passed in)
- pumpStream: the full SSE pump (preamble, delta loop, terminal finish +
usage chunk, [DONE]) — shared by both streaming entries
Behavior change (pinned by TestDirectStreamFailoverAuditSource): direct
streams now pin rec.Source to the source that actually served the stream
after a failover, instead of discarding it (_, usedModel). The audit row
previously recorded the first candidate, which was wrong on failover.
chat.go: 982 → 896 lines (−86).
provider.oneLineStr and gateway.oneLine had byte-identical bodies (flatten
whitespace + cap length). Move the single implementation into the types
package, which both layers already depend on, and delete both locals.
- provider.truncate: zero callers (oneLineStr is the used superset)
- gateway.normalizeModel: zero callers
- config.resolvedAPIKey: zero callers (core.resolveSourceKey is the live equivalent)
- Stats.Records: zero callers (CSV export uses AuditRecords)
- store.containsString: zero callers
- Config.MaxConcurrent: global inflight-cap field never read; per-source
MaxConcurrent is what actually drives semaphores. Legacy configs carrying a
top-level max_concurrent key still load (yaml.v3 ignores unknown fields —
verified by test).
- gui renderer esc(): zero callers; renderer uses textContent, and the embedded
WebUI has its own esc()
statusRecorder (the access-log wrapper) did not implement http.Flusher, so
w.(http.Flusher) inside streamChat/streamChatAuto returned nil and every SSE
chunk stayed buffered until the response ended. Add Flush() that delegates to
the underlying writer when it supports flushing.
Regression test: TestStatusRecorderFlusher pins the interface assertion and
the forwarding path.
Every upstream formats errors differently, which is adapter territory:
the protocol gains an optional transform_error(status, body) hook and all
built-in adapters implement their own envelope parsing (zen free-pool
labels, anthropic/gemini/ollama/mistral shapes, sensenova quota notes,
agentrouter WAF pages). The core keeps a single uniform fallback: when no
hook yields a reason clients get "api error <status>: unknown error" and
the raw body goes to server logs only.
api error strings embedded raw upstream response bodies, so JSON quota
payloads and WAF HTML pages leaked through to clients (and through the
per-tier chain summary). shortAPIError extracts the envelope reason
(error.message / message / msg), collapses HTML blocklist pages to a
marker, and caps everything at one line.
WindowTokens buckets are whole unix hours, so a fixture built from raw
offsets like "1 minute ago" landed in the previous hour bucket whenever
the suite ran in the first minute of an hour — the 1h-window assertion
then saw 0 tokens and failed. Anchor rows to hour boundaries instead
(H-2 / H-1 / current bucket) and assert the exact current-bucket value;
the suite is now deterministic for any run time.
Clients saw the full per-tier chain error including upstream HTML WAF
pages and JSON quota payloads. The response now carries one capped
one-line reason per tier (quota/cooling skips preserved); full detail
remains in rec.Err / stats API and is logged server-side.
Some upstreams (zen free pool) answer HTTP 200 with a single-chunk stream
whose only payload is finish_reason:"network_error" and an empty delta.
ChatStream used to hand that channel up as a success, so clients received
a laundered empty reply (agents ended their turn mid-loop) and the broken
slot kept its scheduling preference. ChatStream now holds back the first
chunk: an error-only done chunk fails the candidate before any byte
reaches the gateway, letting the scheduler fall through to the next
source. Standard finish reasons (including instant-empty "stop") are never
classified as errors.
The gateway hardcoded "stop" on every terminating stream chunk, so
tool-call rounds reported finish_reason=stop and length caps were
invisible to clients. UnifiedChunk now carries finish_reason; adapters
emit it (with empty-string finish reasons like sensenova treated as
non-terminal), standardSSEChunk passes it through for un-adapted
upstreams, [DONE] no longer emits a duplicate reason-less done chunk,
and both streaming paths emit the real reason with "stop" as fallback.
Also vendor sensenova/agentrouter adapters into the repo: they were
WebUI-only uploads and a deploy sync silently removed them while live
AUTO-chain slots still referenced them.
zen fingerprints clients via UA + x-opencode-client/session/request/project
headers; requests without them are routed as anonymous and fail with a
single-chunk network_error stream (tools+stream) or 503 Endpoint is
unavailable (non-stream), which the adapter laundered into empty-but-valid
replies. Derive per-request session/request ids from build_headers meta
(sandbox has no os/math), bump UA to the real client format, map zen
reasoning field to reasoning_content, and set stream_options.include_usage.
The prior usage-passthrough fix only covered openai/opencode; the same
empty-choices+usage drop bug remained in the 5 sibling OpenAI-compatible
adapters, and non-OpenAI providers (anthropic/gemini/ollama) never surfaced
streaming usage at all.
- deepseek/github/groq/kimicode/mistral: preserve usage on empty-choices
chunks and attach it to normal chunks (same pattern as openai.lua)
- anthropic: emit usage from message_start (prompt) and message_delta
(completion); gateway merges split usage additively
- gemini: read usageMetadata in the stream path
- ollama: fix non-streaming key (usage -> token_usage, matches
UnifiedResponse json tag) and read prompt_eval_count/eval_count;
surface counts from the done stream chunk
- gateway: mergeUsage combines usage across chunks (non-zero fields win,
total recomputed from prompt+completion) so split usage doesn't lose
the prompt half; single-chunk case (OpenAI) preserved exactly
- usage-only chunks: done=false (no redundant terminal stop), matching
the Go fallback standardSSEChunk
Streaming responses now carry the upstream's real token usage instead of
gateway estimates:
- UnifiedChunk gains an optional Usage field; adapters (opencode, openai)
extract usage from upstream stream chunks (including the final chunk with
empty choices) and pass it through.
- standardSSEChunk preserves usage for passthrough adapters.
- Gateway emits the exact usage in the final stream chunk when available,
falling back to estimates only when the upstream provided none.
Non-streaming usage was already fixed to emit OpenAI-standard keys.
- TokenUsage.MarshalJSON now emits both standard (prompt_tokens,
completion_tokens, total_tokens) and legacy (prompt, completion, total)
keys, so OpenAI-compatible clients (DSH, DevEco Code, etc.) can read
token usage.
- ChatChunk gains an optional Usage field; stream responses now send a
final usage chunk (empty choices) before [DONE].
Refs: usage not visible in clients because the gateway serialized only the
internal short keys and never emitted a streaming usage chunk.
Root cause of the packaged GUI crashing with SIGSEGV inside ld.so on user
machines (segfault at fixed +0x1ff36, undefined symbols nspr_use_zone_allocator
/ localtime64): the build host had LD_LIBRARY_PATH polluted by a third-party
runtime (/opt/cangjie), which electron-builder baked into the produced binary's
dependency resolution. Clean machines without that library then fail in the
dynamic loader before any app code runs.
- Makefile: every gui-* pack target now runs under `env -u LD_LIBRARY_PATH`
- main.js: app.disableHardwareAcceleration() before ready (avoids the common
Chromium GPU-process SIGSEGV on hybrid-GPU/Wayland Linux hosts)
- main.js: window icon reads from process.resourcesPath (real file, not asar)
— asar-path icons are a known GTK segfault source on Linux
- api/status sources now carry recent_ok/recent_err (last 300s real gateway
requests via Stats.SourceRecent) so a source actually serving traffic is
never shown as down just because probe /models got rate-limited
- WebUI source status column repaints every 5s (no more frozen-at-first-
render) with a manual refresh button; shows success rate + probe + cooldown
- tray menu status rows were enabled:false (GTK fixed light-grey, invisible
on light themes) — now enabled with no-op click and nativeTheme listener
rebuilds the menu on dark/light switches
- ignore local ops scripts (scripts/, machine-specific)
Window/taskbar icon never showed because the packaged .desktop had
StartupWMClass=ModelRouter while the live window WM_CLASS is the lowercase
app class 'modelrouter-gui' — no match, so GNOME fell back to the default
Electron icon.
- main.js: set BrowserWindow icon asset + backgroundColor (window paints
correctly with frameless titlebar and provides the taskbar's default icon)
- package.json: build.linux.desktop.entry.StartupWMClass now equals the real
WM_CLASS so packaged installs match too
- local integration: user-level .desktop installed at
~/.local/share/applications/modelrouter-gui.desktop (points at the unpacked
binary) so GNOME resolves the icon immediately
opencode.ai/zen is a free-quota pool for the official opencode app, keyed by
User-Agent fingerprint. Auto-provisioning it in the default config effectively
impersonated the official client to bypass FreeUsageLimitError — a ToS risk.
- default config (Go + GUI embed) now ships NO sources; users add their own
- README/README_EN: remove opencode adapter from built-in list and the zen
free-pool paragraph
- adapter script and its unit tests stay (explicit opt-in only)
- coreReady now only flips after the embedded core actually answers HTTP
(fixes blank-screen race where the iframe loaded before the listener)
- renderer probes reachability, retries frame load with exponential backoff,
and recovers via onerror instead of giving up forever
- keep per-user theme in localStorage
- embedded profile no longer preconfigures a zen source (desktop users add
their own upstreams on the sources page)
- adapters/opencode.lua: opencode.ai zen free pool adapter — sends the
opencode client User-Agent (zen fingerprints clients by UA; non-official
clients hit FreeUsageLimitError); pairs with api_key: public
- config: no config file ships in the repo; first run generates a default
config at the -config path with a random admin key, loopback listen and a
keyless zen source (config.EnsureDefault); remove config.example.yaml
- lua: seed bundled adapters from the embedded FS instead of a hardcoded
name list
- ui: widen model kind select (chat was clipped to 'cha')
- phase 5 bugfixes: stats ms/s bucket mixing, cleanScopes nil, ctx.Err
guards, direct-path ModelAvailable, empty stream body failure,
bestImageModel rewrite, transform failure recording, Core.mu, timer,
effective model for tool-calls
- Move all config (auto rules, keys) from runtime.json to config.yaml
- Store now only holds runtime sources (WebUI-created)
- Add OpenRouter free models to config.yaml
- One-time migration from legacy runtime.json on startup
- Fix gateway tests for new config structure
- Update core.go with migrateFromRuntime, saveConfig, seedKeys/seedAuto
- Remove SaveKey/KeyByValue/AutoRules from Store
- Add Config.Save() with YAML marshaling
- Update WebUI admin keys visibility (show all keys including admin)
- Bump binary to 11MB with luajit
The sidebar's backdrop-filter: blur() made the popup sidebar itself
unreadable on ≤900px screens. Disabled the blur in the responsive
media query; now sidebar has solid glass background and the
full-screen blur only applies to the backdrop behind it.
renderStatus() re-injected the entire status tab (#tab-status.innerHTML) + all charts + tables on every visit, so each tab switch back to home stuttered. Now the page is built once (pane.dataset.built) and revisits only reload live data via paintStats() + restart the 3s poll. Static parts (source table, model chips, connection box) are not rebuilt. Deployed (bak .bak.20260811y), server active.
- .kpi fixed height 206px was too short for the tokens card (hdr + big value + two-line in/out + chart + legend) -> text overflowed; bumped to 252px with legend capped at max-height/scroll
- tokens chart: stack shows top-4 slices + lumped 'other', total uses the FULL model set (true shares, was relative to top-4); legend lists every model that has non-zero tokens (was hardcoded top-4, so small/zero slices were dropped -> 'only two models' visible)
- add kOther zh/en
- deployed (bak .bak.20260811x), server active
Previously the skeleton was a fixed 186px but the real cards had different natural heights (status pie 110 + legend, tokens two-line sub + legend, etc.), so replacing the skeleton caused a size jump. Now every .kpi is a fixed 206px flex column; .k-chart absorbs the remaining space (flex:1) and the canvas absolutely fills it (draw fns read container height, not a hardcoded 86/110). Skeleton .ksk is exactly 206px with the same layout, so first paint and real cards are identical in size. Deployed (bak .bak.20260811w), server active.
- KPI skeleton (.ksk) min-height now matches the real card (186px) with a title/value/chart bar layout so first paint doesn't jump
- global entry animations: cards fade+slide-up (fadeUp), tab panes fade+rise (tabIn), chat empty state fades in
- goTab restarts the tabIn animation on switch (reset animation + forced reflow)
- table-row animation intentionally NOT added (those re-render every 3s poll and would flicker)
- deployed (bak .bak.20260811v), server active
The five KPI cards are built only after /api/stats returns; before that #kpi-row was empty, so the top of the dashboard appeared blank then jumped in. Now the row starts with a shimmer skeleton (5 placeholder cards with a CSS gradient animation) that paintStats replaces with the real cards on first data. Deployed (bak .bak.20260811u), server active.
