TBNilles/qwen3.5-122B-A10B-on-spark
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(readme): link dgx-spark-serving-mode for freeing unified memory

Browse Source
This commit is contained in:
Entrpi
2026-06-29 10:14:40 +10:00
parent 41d92cb6f9
commit a77cbdab26
1 changed files with 8 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+8
View File
@@ -148,6 +148,14 @@ streams queue rather than fail) or a smaller `--ctx`.
> [`scripts/monitor.sh`](scripts/monitor.sh) (OOM auto-kill guard) the first time > [`scripts/monitor.sh`](scripts/monitor.sh) (OOM auto-kill guard) the first time
> at any new `gpu-mem` or `ctx`. > at any new `gpu-mem` or `ctx`.
> **Seeing a smaller KV pool / less free memory than the numbers above?** The
> Spark's 128 GB is *unified* — the GNOME desktop and maintenance daemons can hold
> **~10–15 GiB** that would otherwise go to the KV cache. Drop to a headless
> `multi-user.target` with
> [**Entrpi/dgx-spark-serving-mode**](https://github.com/Entrpi/dgx-spark-serving-mode)
> (`sudo serving-mode serve`) to hand that memory back to the model — it directly
> raises the usable pool and context length.
### Startup time ### Startup time
Time to READY is **~3 min** (vs ~12 min on the default loader). `--load-format Time to READY is **~3 min** (vs ~12 min on the default loader). `--load-format