docs: replace invented memory figures with measured ones, ship the tuning knobs

The README claimed "~15 MB RSS" and, after the log-loading work, "~10 MB idle /
~19 MB with a 29 MB audit log". Those were TEST-INSTANCE numbers: one mock source
and one adapter. The real production config on this host (16 sources, 13
adapters, 59 models) sits at ~37-42 MB, and sat at ~105 MB before this series.
Quoting the single-source figure as the headline was misleading.

Both READMEs now state that memory scales with the number of configured sources
rather than with uptime, give a three-row measurement table (1 source / 1 source
with a 29 MB audit history / the 16-source production instance), and break the
production RSS down per region (Go heap, thread stacks + LuaJIT, mapped binary,
Go reservations, shared libs) so an operator can tell which part their own
deployment will grow.

Two runtime knobs are documented and now shipped by default in the desktop
build's core spawn (cmd/gui/main.js, overridable by exporting either variable):

  * MALLOC_ARENA_MAX=2 — LuaJIT allocates through cgo into glibc malloc, and
    glibc keeps up to 8*nproc per-thread arenas of ~1 MB that are never returned.
    Measured 8-15 arenas (7-12 MB) -> 0.
  * GOGC=50 — halves the Go heap target. Documented explicitly as useless ALONE
    (measured 20.3 -> 21.5 MB, i.e. worse, because the saved heap is eaten by
    more glibc arenas); only the pair cuts settled RSS, by ~19%.

Also corrects the binary size (8-12 MB, ~8 MB after the deploy script's -s -w)
and adds the elastic-pool / on-demand-log / self-healing-cooldown bullets that
README.md already had to README_EN.md.
This commit is contained in:
JianFeeeee
2026-08-30 09:09:21 +08:00
parent 882288f67f
commit 2378bc00ba
3 changed files with 130 additions and 6 deletions

View File

@ -15,8 +15,8 @@
### 极致轻量
- **单二进制**:编译后约 10MB零运行时依赖仅依赖系统 libc部署即用
- **低内存占用**空闲 ~10MB RSS带 29MB 历史审计日志启动仅 ~19MB满载并发 100+ 请求时峰值 < 100MB
- **单二进制**:编译后 8~12MB`-s -w` strip 后约 8MB,零运行时依赖(仅依赖系统 libc部署即用
- **低内存占用**与源数量相关,非与运行时长相关——单源 ~10MB、16 源生产实例 ~40MB[实测分解与调优](#内存占用实测与调优)
- **日志按需加载**审计日志不常驻内存——默认只加载首屏下滚自动分页CSV 导出流式写出O(1) 内存),离页即释放
- **零运行时依赖**:纯 Go + LuaJIT 静态链接,无需安装 Python/Node/Java 等运行时
- **启动极快**:冷启动 < 200ms热重载配置 < 10ms
@ -284,6 +284,52 @@ WebUI 上的"新增/编辑源"、"上传 Lua 适配器"、"改 AUTO 优先级链
模板不会自己变成源——它只是「配方」。真正承担流量的是从模板创建出来的、带 key
的具体源。
### 内存占用(实测与调优)
内存不是一个定数:它随**配置的源数量**增长(每个源一个 `http.Transport` 与连接池、
一组健康状态与运行时长无关。本机实测Linux x86_6412 核):
| 部署形态 | 启动 RSS | 稳态 RSS |
|---|---|---|
| 1 源 / 1 适配器(最小配置) | ~4 MB | ~10 MB |
| 1 源 + 29 MB 历史审计日志 | ~19 MB | ~20 MB |
| **16 源 / 13 适配器 / 59 模型(本机生产)** | ~28 MB | **~3742 MB** |
> 历史参考:本项优化前同一生产配置为 **~105 MB**。降幅来自三处:审计日志不再全量回放
> (约 25 MB、Lua 状态池不再单调增长、以及下面两个运行时开关。
内存构成(生产实例分段测量,`/proc/<pid>/smaps`
| 区域 | RSS | 说明 |
|---|---|---|
| Go 堆 | ~14 MB | provider/registry/scheduler 结构 + 连接池缓冲 |
| 其他匹名(线程栈 / LuaJIT chunk / runtime | ~12 MB | 与线程数、已加载适配器数相关 |
| 二进制 text+rodata | ~8 MB | 映射的可执行文件页(只读、可被内核回收) |
| Go runtime 预留 | ~4 MB | VSZ 上看到的 2 GB+ 是地址空间预留,不占物理内存 |
| 共享库 | ~3 MB | libc / libluajit / libm |
**两个推荐的部署开关**(只适用于环境变量,无需改代码):
```ini
# /etc/systemd/system/llmsproxy.service
Environment=GOGC=50
Environment=MALLOC_ARENA_MAX=2
```
- `MALLOC_ARENA_MAX=2`LuaJIT 的分配走 cgo → glibc mallocglibc 默认允许 `8×nproc`
个 per-thread arena每个碰到 malloc 的 OS 线程会占用一个(各约 1 MB且**不归还给系统**)。
实测从 815 个 arena约 712 MB降到 **0**
- `GOGC=50`:把 Go 堆增长目标减半。**单独使用无效**(省下的堆会立即被更多 glibc
arena 吃掉,实测 20.3 → 21.5 MB 反而变大),必须与 `MALLOC_ARENA_MAX` 配合,
两者同时开启才降 **~19%**。网关是 I/O 密集型(本机 9 小时仅消耗 1min10s CPU
多出的 GC 周期与它的空闲 CPU 相比可忽略。
> 桌面版Electron已在 `cmd/gui/main.js` 里默认为内嵌核心注入这两个开关;手动 export 同名
> 环境变量可覆盖。如果你用自己的 systemd unit / 容器,建议照上面加上。
可选:`GOMEMLIMIT=48MiB` 作软上限,实测再省 ~1 MB代价是逐近上限时 GC 转激进,
温和场景不必开。
### disable_thinking
请求体带 `"disable_thinking": true`网关透传给各适配器DeepSeek 适配器将其
@ -369,7 +415,7 @@ return {
| 场景 | 推荐 | 理由 |
| ------ | ------ | ------ |
| 服务器 / 内网网关 / 无人值守常驻 | **Headless**(单二进制) | 极轻量(~10MB、~15MB RSS),零依赖单进程,直接跑在 systemd/任意容器里,远程管理 |
| 服务器 / 内网网关 / 无人值守常驻 | **Headless**(单二进制) | 极轻量(二进制 ~8MBRSS 随源数量,单源 ~10MB、16 源 ~40MB见[内存占用](#内存占用实测与调优)),零依赖单进程,直接跑在 systemd/任意容器里,远程管理 |
| 个人桌面日常使用 / 多设备内网共享 | **Desktop**GUI | 免登录内嵌 WebUI、系统托盘一键启停、开机自启、静默后台适合不懂命令行的使用者 |
| Windows 桌面 | **Desktop** | 纯后端在 Windows 上需自行注册服务GUI 提供原生托盘/自启体验 |
| CI 一键出三平台安装包 | **Desktop 打包脚本** | `make gui-*` 系列出 deb/AppImage/NSIS可进发行流水线 |

View File

@ -11,7 +11,7 @@ KimiCode…) behind a single endpoint. Pick a specific model or use **AUTO** mod
which routes to the best healthy upstream by configured priority.
> **Lightweight, no recompile to add sources**: the gateway is a single Go
> binary (~10 MB, zero runtime dependencies). Adding or switching an upstream is
> binary (8-12 MB, ~8 MB stripped; zero runtime dependencies). Adding or switching an upstream is
> just a `sources` entry in `config.yaml` (or via the WebUI) or a `.lua`
> adapter — **no Go changes, no recompile**. Changes submitted through the
> WebUI take effect immediately (hot reload); editing `config.yaml` or a `.lua`
@ -41,7 +41,21 @@ Extracted and independently evolved from the multi-source LLM adapter layer of
- **Multimodal**: `content` arrays (`image_url` etc.) pass through losslessly;
Anthropic/Gemini/Ollama are translated automatically.
- **LuaJIT VM**: golua-binding LuaJIT; each adapter has its own VM + worker
pool for safe concurrency.
pool for safe concurrency. Pools are **elastic**: the sum of `max_concurrent`
over an adapter's sources is a ceiling, not a preallocation — states are booted
on demand and reclaimed when demand drops, so an idle gateway holds close to
zero Lua states. The grow step follows the adapter's max concurrency
(`clamp(ceil(max/8), 1, 8)`, and never exceeds the number of queued callers);
the shrink step follows the live connection count
(`clamp(ceil(slack/(1+in_use)), 1, slack)`), so an idle pool collapses in one
round while a busy one gives up a single state at a time.
- **On-demand request logs**: the audit log is never held in memory — the
dashboard loads one screen, scrolling pages the rest straight off disk, CSV
export streams in O(1) memory, and leaving the page releases everything.
- **Self-healing cooldown**: cooldown is capped at 5 minutes and, past the
window's midpoint, exactly one probe request is allowed through; a recovered
upstream (or a reset quota) returns to full rotation on that probe instead of
waiting out the window.
- **disable_thinking**: `disable_thinking: true` toggles reasoning per-request.
- **Lua adapter protocol**: each source mounts a `.lua` adapter with
`transform_request` / `transform_response` / `transform_stream_chunk` — all
@ -258,6 +272,59 @@ independently).
A template is just a recipe — it doesn't become a source by itself. Only the
key-bearing sources created from it carry real traffic.
### Memory footprint (measured and tuned)
Memory is not a constant: it scales with the **number of configured sources**
(each source owns an `http.Transport` with its connection pool plus per-model
health state), not with uptime. Measured on this machine (Linux x86_64, 12 cores):
| Deployment shape | Startup RSS | Settled RSS |
|---|---|---|
| 1 source / 1 adapter (minimal) | ~4 MB | ~10 MB |
| 1 source + a 29 MB audit history | ~19 MB | ~20 MB |
| **16 sources / 13 adapters / 59 models (this host)** | ~28 MB | **~37-42 MB** |
> For reference, the same production config used **~105 MB** before this round of
> work. The reduction comes from three places: the audit log is no longer replayed
> in full (~25 MB), Lua state pools no longer grow monotonically, and the two
> runtime knobs below.
Breakdown of the production instance (per-region, from `/proc/<pid>/smaps`):
| Region | RSS | Notes |
|---|---|---|
| Go heap | ~14 MB | provider/registry/scheduler state + connection-pool buffers |
| Other anonymous (thread stacks / LuaJIT chunks / runtime) | ~12 MB | scales with thread count and loaded adapters |
| Binary text+rodata | ~8 MB | mapped executable pages (read-only, reclaimable by the kernel) |
| Go runtime reservations | ~4 MB | the 2 GB+ you see in VSZ is address space, not physical memory |
| Shared libraries | ~3 MB | libc / libluajit / libm |
**Two recommended deployment knobs** (environment only, no code change):
```ini
# /etc/systemd/system/llmsproxy.service
Environment=GOGC=50
Environment=MALLOC_ARENA_MAX=2
```
- `MALLOC_ARENA_MAX=2`: LuaJIT allocates through cgo into glibc malloc, and glibc
allows up to `8 x nproc` per-thread arenas. Every OS thread that touches malloc
claims one (~1 MB each) and **never returns it to the OS**. Measured: 8-15
arenas (7-12 MB) down to **0**.
- `GOGC=50`: halves the Go heap growth target. **It does nothing on its own**
the heap it saves is immediately consumed by additional glibc arenas (measured
20.3 -> 21.5 MB, i.e. slightly worse) — so it must be paired with
`MALLOC_ARENA_MAX`. Together they cut settled RSS by **~19%**. The gateway is
I/O bound (1min10s of CPU per 9 hours here), so the extra GC cycles are free.
> The desktop build (Electron) already injects both when spawning the embedded
> core (`cmd/gui/main.js`); exporting either variable yourself overrides it. If you
> write your own systemd unit or container spec, add them there.
Optional: `GOMEMLIMIT=48MiB` as a soft ceiling saves roughly 1 MB more, at the
cost of GC turning aggressive as the limit approaches. Not worth it for a relaxed
deployment.
### disable_thinking
With `"disable_thinking": true` in the request body, the gateway passes it to
@ -354,7 +421,7 @@ Release installers ship in two flavors for different audiences:
| Scenario | Pick | Why |
| ---- | ---- | ---- |
| Server / intranet gateway / unattended long-running | **Headless** (single binary) | ~10 MB, ~15 MB RSS, zero-dep single process — drop it into systemd or any container, remote admin |
| Server / intranet gateway / unattended long-running | **Headless** (single binary) | ~8 MB binary; RSS scales with the number of sources (~10 MB for one, ~40 MB for 16 — see [Memory footprint](#memory-footprint-measured-and-tuned)), zero-dep single process — drop it into systemd or any container, remote admin |
| Personal desktop daily use / multi-device intranet sharing | **Desktop** (GUI) | no-login embedded WebUI, tray one-click, autostart, silent background — for non-CLI users |
| Windows desktop | **Desktop** | plain backend needs manual service registration; GUI ships native tray/autostart |
| CI one-shot 3-platform installers | **Desktop packaging scripts** | `make gui-*` emits deb / AppImage / NSIS, drops straight into a release pipeline |

View File

@ -198,6 +198,17 @@ function startCore() {
cwd: PROFILE_DIR,
env: Object.assign({}, process.env, {
LLMS_PROXY_GUI: "1",
// Memory tuning, measured on a 16-source deployment (see README 内存占用):
// MALLOC_ARENA_MAX caps glibc per-thread malloc arenas. LuaJIT allocates
// through cgo -> glibc malloc, and glibc defaults to 8*nproc arenas, so
// every OS thread touching malloc reserves its own ~1 MB arena that is
// never returned to the OS.
// GOGC halves the Go heap growth target. On its own it does not help (the
// saved heap is immediately eaten by extra glibc arenas); only the pair
// lowers settled RSS. The core is I/O bound, so the extra GC is free.
// Both are overridable: a user who exports them keeps their own values.
GOGC: process.env.GOGC || "50",
MALLOC_ARENA_MAX: process.env.MALLOC_ARENA_MAX || "2",
}),
stdio: ["ignore", "pipe", "pipe"],
});