mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-09-19 16:39:15 +00:00
25d8bd8632
Production after the first deploy showed the openai pool stuck at 9 states with peak_in_use=1: a startup burst grew it, and then it never shrank again. The shrink grace counter was reset by every CHECKOUT, so on a gateway that always has a request in flight the counter never reached shrinkGraceRounds and the burst's leftover states were pinned indefinitely — the same monotonic-growth behaviour this series set out to remove, just with an extra step. Grace is now reset by GROWTH (a miss that had to boot a state), which is the actual signal that capacity is short. Ordinary sequential traffic no longer defers reclaim, while two consecutive quiet-ish rounds are still required so a gap between two bursts does not tear the pool down. TestCheckoutResetsGrace asserted the old behaviour and is replaced by TestGrowthResetsGrace (sequential traffic must NOT defer, growth must) plus TestBurstLeftoverIsReclaimedUnderSteadyTraffic, which reproduces the production shape: 9 concurrent holds, then one request per janitor round, and the pool must still fall back to the resident floor.