mirror of
https://gitcode.com/JianFeeeee/ModelRouter.git
synced 2026-09-19 16:39:15 +00:00
39b48e556e
Follow-on to the session-stability fix. "Per source" already made the prefix cache hit, but it puts every conversation into one upstream session. Using the client's own session id is not possible: capturing real agent traffic (tcpdump on 127.0.0.1:8081) shows generic OpenAI clients send NO session identifier at all — no user / session_id / conversation_id / metadata in the body, and no session header (only X-Stainless-* plus User-Agent: pi). The x-opencode-session the Go endpoint asks for is an OpenCode native-client concept that a generic client cannot forward. Since history is replayed every turn, the FIRST user message is invariant for the life of a conversation, so it is used as the conversation fingerprint. The session becomes stable within a conversation and distinct across conversations; requests with no user message fall back to per-source stability. Measured through the gateway (same 5.7k-token prompt): 2nd call cached_tokens=5504, and an unrelated conversation gets its own session. Test: TestOpenCodeSessionIsStableForCache covers same-conversation stability, cross-conversation separation, per-request request ids and the sessionless fallback.