jianf/ModelRouter
1
0
Fork 0
mirror of https://gitcode.com/JianFeeeee/ModelRouter.git synced 2026-09-22 09:58:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(opencode): make x-opencode-session stable so the upstream prefix cache can hit

Browse Source 
The opencode adapters derived x-opencode-session from meta.timestamp, i.e. a
brand new session on every request. The upstream prefix cache is
session-scoped, so no request could ever hit it, and the cache fields the
endpoint does report (prompt_tokens_details.cached_tokens,
prompt_cache_hit_tokens/prompt_cache_miss_tokens) always came back 0/absent.

Measured against the live endpoint, same 6032-token prompt:

  fixed session id   -> 2nd call: hit 5888, miss 144
  rotating session id -> every call: hit 0, miss 6032

Fix: derive the session from the source name (stable), matching how
x-opencode-project is already derived. x-opencode-request stays unique per
request — it is only a request identifier, not part of the cache key.
Applied to both opencodego and opencodezen.

Through the gateway the same prompt now reports, on the 2nd call:
  details={'cached_tokens': 5888} hit=5888 miss=144      (non-streaming)
  prompt_tokens_details={'cached_tokens': 5888}          (streaming)

Test: TestOpenCodeSessionIsStableForCache asserts the session is stable
across requests for one source while the request id differs.
This commit is contained in:
JianFeeeee
2026-09-11 15:42:22 +08:00
parent 1f7d17feac
commit 791d198f47
3 changed files with 70 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

50
internal/lua/toolcall_preservation_test.go
View File

@ -231,3 +231,53 @@ func TestOpenCodeGoVsZenReasoning(t *testing.T) {
}
}
}
// TestOpenCodeSessionIsStableForCache pins that the opencode client header set
// keeps x-opencode-session STABLE per source while x-opencode-request stays
// unique per call.
//
// Measured against the live OpenCode Go endpoint: the upstream prefix cache is
// session-scoped. Replaying the same 6032-token prompt hits 5888 cached tokens
// when the session id is fixed, and 0 when it changes per request. Deriving the
// session from meta.timestamp (the previous behaviour) made every request a new
// session, so the cache could never hit.
func TestOpenCodeSessionIsStableForCache(t *testing.T) {
for _, name := range []string{"opencodego", "opencodezen"} {
vm := NewVM(freshAdapterDir(t))
if err := vm.Start(); err != nil {
t.Fatal(err)
}
metaA := map[string]interface{}{
"timestamp": 1000,
"body": `{"messages":[{"role":"user","content":"one"}]}`,
"source": map[string]interface{}{"name": "somesource", "meta": nil},
}
metaB := map[string]interface{}{
"timestamp": 2000, // different second, different body
"body": `{"messages":[{"role":"user","content":"two"}]}`,
"source": map[string]interface{}{"name": "somesource", "meta": nil},
}
h1, err := vm.BuildHeaders(name, metaA)
if err != nil {
t.Fatalf("%s headers A: %v", name, err)
}
h2, err := vm.BuildHeaders(name, metaB)
if err != nil {
t.Fatalf("%s headers B: %v", name, err)
}
if h1["x-opencode-session"] == "" {
t.Fatalf("%s: session header must be set", name)
}
if h1["x-opencode-session"] != h2["x-opencode-session"] {
t.Errorf("%s: x-opencode-session must be stable across requests (%q vs %q) — "+
"the upstream prefix cache is session-scoped, a rotating session kills every cache hit",
name, h1["x-opencode-session"], h2["x-opencode-session"])
}
if h1["x-opencode-request"] == h2["x-opencode-request"] {
t.Errorf("%s: x-opencode-request must differ per request", name)
}
vm.Stop()
}
}