Startup RSS on this deployment was 56 MB with a 29 MB audit log and ~10 MB
without one: LoadAudit() json-unmarshalled the ENTIRE file into the aggregates
and kept a 10000-entry ring of raw records. Two more paths had the same shape —
AuditRecords() materialized a whole export window into a []Req before sorting
it, and a dashboard poll serialized the full ring so the browser could render
300 rows of it.
The audit file is now the source of truth and memory only holds the live
window:
* LoadAudit replays only the last auditReplayBytes (4 MB) and drops the
truncated first line; the ring default drops 10000 -> 500, which still
covers both of its consumers (the status page's 5-minute SourceRecent /
SourceAverages windows and the first screen of the records table).
replayPartial is exported so the UI can say the totals cover a window
rather than all time. Token-quota accounting is unaffected: it reads the
modelHour buckets, not the ring (pinned by a test).
* AuditPage(cursor, limit, key) pages records straight off disk, reading the
newest file backwards in 64 KB chunks and returning as soon as the page is
full. The cursor is "<file>:<offset>" and walks into rotated .old files;
a cursor whose file rotated away reports rotated=true so the client can
reset instead of silently skipping records. No state is cached between
requests and the file handle is closed before responding, so "release when
the user leaves the page" is guaranteed by never retaining anything.
* StreamAuditRecords(from,to,key,fn) replaces the accumulate-then-sort export
path; the CSV handler writes rows as they are read and flushes every 1000,
and a write error (client gone) aborts the walk. Export memory is O(1)
regardless of the window. AuditRecords is kept as a test-only wrapper.
* Snapshot ships one screen (firstScreenRecords=100) by default; aggregates
are untouched.
* Audit rotation 64 MB x 10 -> 16 MB x 16: same 256 MB total budget, but a
smaller newest file keeps the first reverse page cheap.
New route: GET /api/stats/records?before=&limit=&key= (non-admins are pinned to
their own key by exportKey). /api/status additionally reports adapter_pools for
admins.
Measured with production's 29 MB audit copied to the test instance: startup RSS
19.0 MB (was 56 MB); scrolling 10 pages (1000 records) +0.7 MB; exporting the
full history (36441 rows / 4.4 MB CSV) +0.1 MB with no residual growth.
Live testing across the zen pool showed models report
prompt_tokens_details.cached_tokens even when the hit count is 0 (e.g.
nemotron-3-ultra-free returns cached_tokens:0, audio_tokens:0,
cache_write_tokens:0). The previous >0 guard dropped those objects, so a
cache-enabled upstream looked identical to one without cache support.
- types: PromptTokensDetails.CachedTokens always emitted (drop inner
omitempty) so clients see cached_tokens:0 explicitly; dsh reads it as
a 0% hit instead of 'no data'
- adapters (9): forward prompt_tokens_details whenever the upstream
provides it (presence check instead of >0)
- Req: add cache_reported flag set when usage carried cache accounting;
WebUI shows an amber 0% tag for reported-but-missed rows and keeps
the em-dash only for sources that never report cache data
- Req: add CacheHit and CacheMiss fields (carrying upstream cache
accounting from either prompt_tokens_details.cached_tokens or legacy
prompt_cache_hit_tokens)
- recordChatUsage (non-streaming): copy cache fields from resp.TokenUsage
- pumpStream (streaming): write lastUsage cache fields back onto rec at
stream end, so streaming requests carry cache data too
- CSV export: add first_byte_ms, cache_hit_tokens, cache_miss_tokens
columns alongside the existing latency/prompt/completion
- WebUI request-records table: add a Cache column showing hit% per row
(green/amber tag with tooltip hit/miss breakdown; em-dash when the
upstream reported no cache data)
- Req: add FirstByteMs field (ms to first byte, tracked for streaming)
- Stat: add FirstByteSum for aggregation
- SourceAverages(): new method computing per-source avg TTFB and tokens/s
from the in-memory ring (300s window)
- SourceStatus: add AvgFirstByteMs and AvgTokPerS fields
- pumpStream: record FirstByteMs after first SSE chunk sent to client
- singleChat/singleChatAuto: set FirstByteMs = LatMs (non-streaming)
- handleStatusAPI: populate the new SourceStatus fields from SourceAverages()
- WebUI source table: two new columns showing TTFB (s) and Tokens/s
- provider.truncate: zero callers (oneLineStr is the used superset)
- gateway.normalizeModel: zero callers
- config.resolvedAPIKey: zero callers (core.resolveSourceKey is the live equivalent)
- Stats.Records: zero callers (CSV export uses AuditRecords)
- store.containsString: zero callers
- Config.MaxConcurrent: global inflight-cap field never read; per-source
MaxConcurrent is what actually drives semaphores. Legacy configs carrying a
top-level max_concurrent key still load (yaml.v3 ignores unknown fields —
verified by test).
- gui renderer esc(): zero callers; renderer uses textContent, and the embedded
WebUI has its own esc()
- api/status sources now carry recent_ok/recent_err (last 300s real gateway
requests via Stats.SourceRecent) so a source actually serving traffic is
never shown as down just because probe /models got rate-limited
- WebUI source status column repaints every 5s (no more frozen-at-first-
render) with a manual refresh button; shows success rate + probe + cooldown
- tray menu status rows were enabled:false (GTK fixed light-grey, invisible
on light themes) — now enabled with no-op click and nativeTheme listener
rebuilds the menu on dark/light switches
- ignore local ops scripts (scripts/, machine-specific)
- adapters/opencode.lua: opencode.ai zen free pool adapter — sends the
opencode client User-Agent (zen fingerprints clients by UA; non-official
clients hit FreeUsageLimitError); pairs with api_key: public
- config: no config file ships in the repo; first run generates a default
config at the -config path with a random admin key, loopback listen and a
keyless zen source (config.EnsureDefault); remove config.example.yaml
- lua: seed bundled adapters from the embedded FS instead of a hardcoded
name list
- ui: widen model kind select (chat was clipped to 'cha')
- phase 5 bugfixes: stats ms/s bucket mixing, cleanScopes nil, ctx.Err
guards, direct-path ModelAvailable, empty stream body failure,
bestImageModel rewrite, transform failure recording, Core.mu, timer,
effective model for tool-calls
- scheduler: BuildChain sorts tiers ascending so tier 1 (highest priority) is tried first; previously descending inverted the chain (P10-1..P10-5)
- stats/api: handleStatsAPI limit→20000; CSV reads full audit via new Stats.AuditRecords (includes *.old rotation); key_names mapped by keyID() masked key; non-admin filter and keys-csv use masked keys
- server: ring buffer NewStats(10000)
- ui: dash-row card scroll area moved to table container (fix overflow below card); key view toggle (re-click same key returns to global) + kpi-exit affordance; rec-exit span
- chat: chain-failure record now surfaces first failed tier; AUTO comment sync
