Two problems reported after the on-demand log work landed.
1. Dashboard totals were wrong. LoadAudit only replayed the last 4 MB of the
audit file, so requests/tokens/per-key rows reflected a window instead of all
time — a regression in reported numbers, not just in presentation.
The aggregates are now built by streaming EVERY audit file (oldest first, so
the hourly quota buckets keep their intended trailing window) and keeping
nothing per record: aggregate maps are keyed by key/model/source, so their
size is bounded by cardinality. Measured on the production host: 29 MB /
221k lines / 37k requests in ~260 ms at startup.
What stays bounded is the RAW-record ring: a fixed-size reqRing keeps only the
newest maxRecs records, so the ~25 MB that used to be spent appending every
record into a slice is still saved. auditReplayBytes is gone, and
replayPartial now means "an audit file could not be read", which is the only
remaining way for the totals to be incomplete.
2. Scrolling to the bottom stopped loading more records. Two independent causes:
* paintRecords rebuilt the entire table on every 5s poll whenever the row
count did not exceed the first screen — the "is a paged view live?" test
compared row counts and matched exactly on the first refresh — wiping loaded
pages and resetting scroll position.
* IntersectionObserver only fires on TRANSITIONS. With a short list, or after a
page whose rows all duplicated the first screen, the sentinel stayed visible
and never fired again.
paintRecords now builds once (recsState.built) and later polls PREPEND only
genuinely new rows; attachRecsObserver adds a scroll-position fallback;
fillRecordsViewport loads until the list actually overflows; and
loadMoreRecords chains (bounded) when a page yields no new rows, since the
first fetch necessarily overlaps the first screen.
TestUIRecordsPagingWiring pins all four mechanisms structurally, since none of
them is reachable from Go. Test names/comments referring to bounded replay are
updated to describe the bounded RING instead, and both READMEs now state that
totals come from the full history while records are paged.
Startup RSS on this deployment was 56 MB with a 29 MB audit log and ~10 MB
without one: LoadAudit() json-unmarshalled the ENTIRE file into the aggregates
and kept a 10000-entry ring of raw records. Two more paths had the same shape —
AuditRecords() materialized a whole export window into a []Req before sorting
it, and a dashboard poll serialized the full ring so the browser could render
300 rows of it.
The audit file is now the source of truth and memory only holds the live
window:
* LoadAudit replays only the last auditReplayBytes (4 MB) and drops the
truncated first line; the ring default drops 10000 -> 500, which still
covers both of its consumers (the status page's 5-minute SourceRecent /
SourceAverages windows and the first screen of the records table).
replayPartial is exported so the UI can say the totals cover a window
rather than all time. Token-quota accounting is unaffected: it reads the
modelHour buckets, not the ring (pinned by a test).
* AuditPage(cursor, limit, key) pages records straight off disk, reading the
newest file backwards in 64 KB chunks and returning as soon as the page is
full. The cursor is "<file>:<offset>" and walks into rotated .old files;
a cursor whose file rotated away reports rotated=true so the client can
reset instead of silently skipping records. No state is cached between
requests and the file handle is closed before responding, so "release when
the user leaves the page" is guaranteed by never retaining anything.
* StreamAuditRecords(from,to,key,fn) replaces the accumulate-then-sort export
path; the CSV handler writes rows as they are read and flushes every 1000,
and a write error (client gone) aborts the walk. Export memory is O(1)
regardless of the window. AuditRecords is kept as a test-only wrapper.
* Snapshot ships one screen (firstScreenRecords=100) by default; aggregates
are untouched.
* Audit rotation 64 MB x 10 -> 16 MB x 16: same 256 MB total budget, but a
smaller newest file keeps the first reverse page cheap.
New route: GET /api/stats/records?before=&limit=&key= (non-admins are pinned to
their own key by exportKey). /api/status additionally reports adapter_pools for
admins.
Measured with production's 29 MB audit copied to the test instance: startup RSS
19.0 MB (was 56 MB); scrolling 10 pages (1000 records) +0.7 MB; exporting the
full history (36441 rows / 4.4 MB CSV) +0.1 MB with no residual growth.
Live testing across the zen pool showed models report
prompt_tokens_details.cached_tokens even when the hit count is 0 (e.g.
nemotron-3-ultra-free returns cached_tokens:0, audio_tokens:0,
cache_write_tokens:0). The previous >0 guard dropped those objects, so a
cache-enabled upstream looked identical to one without cache support.
- types: PromptTokensDetails.CachedTokens always emitted (drop inner
omitempty) so clients see cached_tokens:0 explicitly; dsh reads it as
a 0% hit instead of 'no data'
- adapters (9): forward prompt_tokens_details whenever the upstream
provides it (presence check instead of >0)
- Req: add cache_reported flag set when usage carried cache accounting;
WebUI shows an amber 0% tag for reported-but-missed rows and keeps
the em-dash only for sources that never report cache data
- Req: add CacheHit and CacheMiss fields (carrying upstream cache
accounting from either prompt_tokens_details.cached_tokens or legacy
prompt_cache_hit_tokens)
- recordChatUsage (non-streaming): copy cache fields from resp.TokenUsage
- pumpStream (streaming): write lastUsage cache fields back onto rec at
stream end, so streaming requests carry cache data too
- CSV export: add first_byte_ms, cache_hit_tokens, cache_miss_tokens
columns alongside the existing latency/prompt/completion
- WebUI request-records table: add a Cache column showing hit% per row
(green/amber tag with tooltip hit/miss breakdown; em-dash when the
upstream reported no cache data)
- Req: add FirstByteMs field (ms to first byte, tracked for streaming)
- Stat: add FirstByteSum for aggregation
- SourceAverages(): new method computing per-source avg TTFB and tokens/s
from the in-memory ring (300s window)
- SourceStatus: add AvgFirstByteMs and AvgTokPerS fields
- pumpStream: record FirstByteMs after first SSE chunk sent to client
- singleChat/singleChatAuto: set FirstByteMs = LatMs (non-streaming)
- handleStatusAPI: populate the new SourceStatus fields from SourceAverages()
- WebUI source table: two new columns showing TTFB (s) and Tokens/s
- provider.truncate: zero callers (oneLineStr is the used superset)
- gateway.normalizeModel: zero callers
- config.resolvedAPIKey: zero callers (core.resolveSourceKey is the live equivalent)
- Stats.Records: zero callers (CSV export uses AuditRecords)
- store.containsString: zero callers
- Config.MaxConcurrent: global inflight-cap field never read; per-source
MaxConcurrent is what actually drives semaphores. Legacy configs carrying a
top-level max_concurrent key still load (yaml.v3 ignores unknown fields —
verified by test).
- gui renderer esc(): zero callers; renderer uses textContent, and the embedded
WebUI has its own esc()
- api/status sources now carry recent_ok/recent_err (last 300s real gateway
requests via Stats.SourceRecent) so a source actually serving traffic is
never shown as down just because probe /models got rate-limited
- WebUI source status column repaints every 5s (no more frozen-at-first-
render) with a manual refresh button; shows success rate + probe + cooldown
- tray menu status rows were enabled:false (GTK fixed light-grey, invisible
on light themes) — now enabled with no-op click and nativeTheme listener
rebuilds the menu on dark/light switches
- ignore local ops scripts (scripts/, machine-specific)
- adapters/opencode.lua: opencode.ai zen free pool adapter — sends the
opencode client User-Agent (zen fingerprints clients by UA; non-official
clients hit FreeUsageLimitError); pairs with api_key: public
- config: no config file ships in the repo; first run generates a default
config at the -config path with a random admin key, loopback listen and a
keyless zen source (config.EnsureDefault); remove config.example.yaml
- lua: seed bundled adapters from the embedded FS instead of a hardcoded
name list
- ui: widen model kind select (chat was clipped to 'cha')
- phase 5 bugfixes: stats ms/s bucket mixing, cleanScopes nil, ctx.Err
guards, direct-path ModelAvailable, empty stream body failure,
bestImageModel rewrite, transform failure recording, Core.mu, timer,
effective model for tool-calls
- scheduler: BuildChain sorts tiers ascending so tier 1 (highest priority) is tried first; previously descending inverted the chain (P10-1..P10-5)
- stats/api: handleStatsAPI limit→20000; CSV reads full audit via new Stats.AuditRecords (includes *.old rotation); key_names mapped by keyID() masked key; non-admin filter and keys-csv use masked keys
- server: ring buffer NewStats(10000)
- ui: dash-row card scroll area moved to table container (fix overflow below card); key view toggle (re-click same key returns to global) + kpi-exit affordance; rec-exit span
- chat: chain-failure record now surfaces first failed tier; AUTO comment sync
