fix(gateway): aggregate the full audit history, repair record paging

Two problems reported after the on-demand log work landed.

1. Dashboard totals were wrong. LoadAudit only replayed the last 4 MB of the
   audit file, so requests/tokens/per-key rows reflected a window instead of all
   time — a regression in reported numbers, not just in presentation.

   The aggregates are now built by streaming EVERY audit file (oldest first, so
   the hourly quota buckets keep their intended trailing window) and keeping
   nothing per record: aggregate maps are keyed by key/model/source, so their
   size is bounded by cardinality. Measured on the production host: 29 MB /
   221k lines / 37k requests in ~260 ms at startup.

   What stays bounded is the RAW-record ring: a fixed-size reqRing keeps only the
   newest maxRecs records, so the ~25 MB that used to be spent appending every
   record into a slice is still saved. auditReplayBytes is gone, and
   replayPartial now means "an audit file could not be read", which is the only
   remaining way for the totals to be incomplete.

2. Scrolling to the bottom stopped loading more records. Two independent causes:

   * paintRecords rebuilt the entire table on every 5s poll whenever the row
     count did not exceed the first screen — the "is a paged view live?" test
     compared row counts and matched exactly on the first refresh — wiping loaded
     pages and resetting scroll position.
   * IntersectionObserver only fires on TRANSITIONS. With a short list, or after a
     page whose rows all duplicated the first screen, the sentinel stayed visible
     and never fired again.

   paintRecords now builds once (recsState.built) and later polls PREPEND only
   genuinely new rows; attachRecsObserver adds a scroll-position fallback;
   fillRecordsViewport loads until the list actually overflows; and
   loadMoreRecords chains (bounded) when a page yields no new rows, since the
   first fetch necessarily overlaps the first screen.

TestUIRecordsPagingWiring pins all four mechanisms structurally, since none of
them is reachable from Go. Test names/comments referring to bounded replay are
updated to describe the bounded RING instead, and both READMEs now state that
totals come from the full history while records are paged.
This commit is contained in:
JianFeeeee
2026-08-30 09:29:42 +08:00
parent 2378bc00ba
commit 3ddae41f0c
6 changed files with 409 additions and 143 deletions

View File

@ -49,9 +49,11 @@ Extracted and independently evolved from the multi-source LLM adapter layer of
the shrink step follows the live connection count
(`clamp(ceil(slack/(1+in_use)), 1, slack)`), so an idle pool collapses in one
round while a busy one gives up a single state at a time.
- **On-demand request logs**: the audit log is never held in memory — the
dashboard loads one screen, scrolling pages the rest straight off disk, CSV
export streams in O(1) memory, and leaving the page releases everything.
- **On-demand request logs**: dashboard totals are computed from the **full**
audit history (streamed once at startup and released — 29 MB / 221k lines in
~260 ms), while the raw records are never held in memory: the dashboard loads
one screen, scrolling pages the rest straight off disk, CSV export streams in
O(1) memory, and leaving the page releases everything.
- **Self-healing cooldown**: cooldown is capped at 5 minutes and, past the
window's midpoint, exactly one probe request is allowed through; a recovered
upstream (or a reset quota) returns to full rotation on that probe instead of
@ -285,8 +287,9 @@ health state), not with uptime. Measured on this machine (Linux x86_64, 12 cores
| **16 sources / 13 adapters / 59 models (this host)** | ~28 MB | **~37-42 MB** |
> For reference, the same production config used **~105 MB** before this round of
> work. The reduction comes from three places: the audit log is no longer replayed
> in full (~25 MB), Lua state pools no longer grow monotonically, and the two
> work. The reduction comes from three places: raw audit records are no longer kept
> resident (~25 MB — the aggregates still scan the full history, but the scan
> releases as it goes), Lua state pools no longer grow monotonically, and the two
> runtime knobs below.
Breakdown of the production instance (per-region, from `/proc/<pid>/smaps`):