Two problems reported after the on-demand log work landed.
1. Dashboard totals were wrong. LoadAudit only replayed the last 4 MB of the
audit file, so requests/tokens/per-key rows reflected a window instead of all
time — a regression in reported numbers, not just in presentation.
The aggregates are now built by streaming EVERY audit file (oldest first, so
the hourly quota buckets keep their intended trailing window) and keeping
nothing per record: aggregate maps are keyed by key/model/source, so their
size is bounded by cardinality. Measured on the production host: 29 MB /
221k lines / 37k requests in ~260 ms at startup.
What stays bounded is the RAW-record ring: a fixed-size reqRing keeps only the
newest maxRecs records, so the ~25 MB that used to be spent appending every
record into a slice is still saved. auditReplayBytes is gone, and
replayPartial now means "an audit file could not be read", which is the only
remaining way for the totals to be incomplete.
2. Scrolling to the bottom stopped loading more records. Two independent causes:
* paintRecords rebuilt the entire table on every 5s poll whenever the row
count did not exceed the first screen — the "is a paged view live?" test
compared row counts and matched exactly on the first refresh — wiping loaded
pages and resetting scroll position.
* IntersectionObserver only fires on TRANSITIONS. With a short list, or after a
page whose rows all duplicated the first screen, the sentinel stayed visible
and never fired again.
paintRecords now builds once (recsState.built) and later polls PREPEND only
genuinely new rows; attachRecsObserver adds a scroll-position fallback;
fillRecordsViewport loads until the list actually overflows; and
loadMoreRecords chains (bounded) when a page yields no new rows, since the
first fetch necessarily overlaps the first screen.
TestUIRecordsPagingWiring pins all four mechanisms structurally, since none of
them is reachable from Go. Test names/comments referring to bounded replay are
updated to describe the bounded RING instead, and both READMEs now state that
totals come from the full history while records are paged.
Startup RSS on this deployment was 56 MB with a 29 MB audit log and ~10 MB
without one: LoadAudit() json-unmarshalled the ENTIRE file into the aggregates
and kept a 10000-entry ring of raw records. Two more paths had the same shape —
AuditRecords() materialized a whole export window into a []Req before sorting
it, and a dashboard poll serialized the full ring so the browser could render
300 rows of it.
The audit file is now the source of truth and memory only holds the live
window:
* LoadAudit replays only the last auditReplayBytes (4 MB) and drops the
truncated first line; the ring default drops 10000 -> 500, which still
covers both of its consumers (the status page's 5-minute SourceRecent /
SourceAverages windows and the first screen of the records table).
replayPartial is exported so the UI can say the totals cover a window
rather than all time. Token-quota accounting is unaffected: it reads the
modelHour buckets, not the ring (pinned by a test).
* AuditPage(cursor, limit, key) pages records straight off disk, reading the
newest file backwards in 64 KB chunks and returning as soon as the page is
full. The cursor is "<file>:<offset>" and walks into rotated .old files;
a cursor whose file rotated away reports rotated=true so the client can
reset instead of silently skipping records. No state is cached between
requests and the file handle is closed before responding, so "release when
the user leaves the page" is guaranteed by never retaining anything.
* StreamAuditRecords(from,to,key,fn) replaces the accumulate-then-sort export
path; the CSV handler writes rows as they are read and flushes every 1000,
and a write error (client gone) aborts the walk. Export memory is O(1)
regardless of the window. AuditRecords is kept as a test-only wrapper.
* Snapshot ships one screen (firstScreenRecords=100) by default; aggregates
are untouched.
* Audit rotation 64 MB x 10 -> 16 MB x 16: same 256 MB total budget, but a
smaller newest file keeps the first reverse page cheap.
New route: GET /api/stats/records?before=&limit=&key= (non-admins are pinned to
their own key by exportKey). /api/status additionally reports adapter_pools for
admins.
Measured with production's 29 MB audit copied to the test instance: startup RSS
19.0 MB (was 56 MB); scrolling 10 pages (1000 records) +0.7 MB; exporting the
full history (36441 rows / 4.4 MB CSV) +0.1 MB with no residual growth.
WindowTokens buckets are whole unix hours, so a fixture built from raw
offsets like "1 minute ago" landed in the previous hour bucket whenever
the suite ran in the first minute of an hour — the 1h-window assertion
then saw 0 tokens and failed. Anchor rows to hour boundaries instead
(H-2 / H-1 / current bucket) and assert the exact current-bucket value;
the suite is now deterministic for any run time.
- adapters/opencode.lua: opencode.ai zen free pool adapter — sends the
opencode client User-Agent (zen fingerprints clients by UA; non-official
clients hit FreeUsageLimitError); pairs with api_key: public
- config: no config file ships in the repo; first run generates a default
config at the -config path with a random admin key, loopback listen and a
keyless zen source (config.EnsureDefault); remove config.example.yaml
- lua: seed bundled adapters from the embedded FS instead of a hardcoded
name list
- ui: widen model kind select (chat was clipped to 'cha')
- phase 5 bugfixes: stats ms/s bucket mixing, cleanScopes nil, ctx.Err
guards, direct-path ModelAvailable, empty stream body failure,
bestImageModel rewrite, transform failure recording, Core.mu, timer,
effective model for tool-calls
