Commit Graph

3 Commits

Author SHA1 Message Date
c524831616 fix(deploy): roll back config together with the binary, and preflight it before restart
The 446-restart-loop incident: adding a headers block to a source without
removing the source's existing `headers: {}` produced a duplicate YAML key.
The process exited on startup, healthcheck failed, and rollback restored only
the binary — so the old binary kept parsing the same broken config and the
service span in systemd's restart loop. Config was treated as out of scope
for deployment; it is not.

Three changes close the loop:

1. cmd/llmsproxy: new `-check` flag validates a config (parse +
   ApplyDefaults) and exits, without starting the Lua VM, touching
   runtime.json, or binding a port — safe to run against a live service.
   Unlike normal startup it does NOT create a default config, so a missing
   file is an error.

2. deploy.sh `--config <file>`: stage a config for deployment, atomically
   renamed into place with the same copy->rename(2) technique as the binary.
   Omitted means the live config is left alone.

3. Ordering: config replacement and preflight both run BEFORE
   restart_service, so an invalid config is caught while the service is still
   healthy and never triggers a restart. rollback() now restores binary AND
   config (only when this run replaced it, so concurrent WebUI edits survive),
   then re-runs -check before restarting — refusing to restart into a config
   that still fails, instead of trading one restart storm for another.

Also: the sha256-unchanged early exit now only fires when there is no pending
config, otherwise `--config` would be silently dropped.

Verified on the live deployment:
- reproduced the exact duplicate-key config: preflight caught it, PID
  unchanged (zero interruption), binary and config both rolled back, gateway
  still answering 200
- valid config: replaced, service restarted, new value live
- no --config: binary-only deploy unaffected
- go test -tags luajit ./... passes
2026-09-05 09:46:43 +08:00
dev
18c2385c61 feat: add per-source TTFB and tokens/s metrics to status page
- Req: add FirstByteMs field (ms to first byte, tracked for streaming)
- Stat: add FirstByteSum for aggregation
- SourceAverages(): new method computing per-source avg TTFB and tokens/s
  from the in-memory ring (300s window)
- SourceStatus: add AvgFirstByteMs and AvgTokPerS fields
- pumpStream: record FirstByteMs after first SSE chunk sent to client
- singleChat/singleChatAuto: set FirstByteMs = LatMs (non-streaming)
- handleStatusAPI: populate the new SourceStatus fields from SourceAverages()
- WebUI source table: two new columns showing TTFB (s) and Tokens/s
2026-08-25 08:24:08 +08:00
690f55c6f1 feat: proactive rate limiting (RPM) + 429 short cooldown for sources
- config.go: Source add RPM field (requests-per-minute cap, 0=unlimited)
- provider.go: RecordRateLimit() — 429 uses fixed 30s cooldown, not exponential
- provider.go: Throttle() — token-bucket proactive rate limiter, spaces requests
  at 60s/RPM interval, respects context cancellation
- provider.go: ReportStatus() — 429 -> RecordRateLimit, 5xx -> RecordFailure
- provider.go: Chat/ChatStream — wire Throttle after TryAcquire
- api.go: sourcePayload + RPM, buildSource passes RPM through
- ui/index.html: add RPM input field in source editor, bilingual i18n labels
- deploy.sh: backup old binary + rollback on healthcheck failure
- provider_test.go: TestModelStateRateLimitShortCooldown, TestThrottleSpacingAndCancel
- config.yaml: sensenova rpm: 12
2026-08-24 02:00:25 +08:00