mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-09-20 00:48:00 +00:00
813de19bd0
Production oscillated between ~46 MB and ~56 MB RSS with the openai pool cycling 1 -> 10..12 -> 1 states every couple of minutes, while peak_in_use never went above 2. Cause: the batch prewarm sized itself purely on the adapter's ceiling. With max_concurrent summing to 76, growStep is 8, so any two overlapping requests warmed 8 states — 6 more than anything was waiting for. A minute later the janitor correctly reclaimed the surplus, the next pair of overlapping requests warmed 8 again, and the pool churned boot/discard forever. The elasticity was working; the growth signal was simply wrong. Prewarm is now bounded by BOTH limits: the ceiling still caps the step, but the batch never exceeds p.waiting, the number of goroutines actually blocked on the pool. Overlapping-but-not-queued traffic (the common case) creates exactly the states it uses; a genuinely queued burst still ramps in one jump. TestContentionBatchPrewarms is rewritten to queue real waiters instead of relying on the ceiling to imply demand, and TestNoPrewarmWithoutWaiters pins the production shape: two overlapping requests against a 76-wide adapter must create exactly 2 states.