From a77cbdab26956ef6ac9cdca544e5fb9ec1f3bb2a Mon Sep 17 00:00:00 2001 From: Entrpi Date: Mon, 29 Jun 2026 10:14:40 +1000 Subject: [PATCH] docs(readme): link dgx-spark-serving-mode for freeing unified memory --- README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/README.md b/README.md index 76e7656..596ad81 100644 --- a/README.md +++ b/README.md @@ -148,6 +148,14 @@ streams queue rather than fail) or a smaller `--ctx`. > [`scripts/monitor.sh`](scripts/monitor.sh) (OOM auto-kill guard) the first time > at any new `gpu-mem` or `ctx`. +> **Seeing a smaller KV pool / less free memory than the numbers above?** The +> Spark's 128 GB is *unified* — the GNOME desktop and maintenance daemons can hold +> **~10–15 GiB** that would otherwise go to the KV cache. Drop to a headless +> `multi-user.target` with +> [**Entrpi/dgx-spark-serving-mode**](https://github.com/Entrpi/dgx-spark-serving-mode) +> (`sudo serving-mode serve`) to hand that memory back to the model — it directly +> raises the usable pool and context length. + ### Startup time Time to READY is **~3 min** (vs ~12 min on the default loader). `--load-format