55c0c1702f
fix(install): wire --port into the container and clear stale compile cache on nspec change
...
- pass -e PORT to docker run so --port reaches serve.sh/mtp_serve.sh; previously PORT
was only used by the host-side readiness poll, so a non-default --port left the
server on 8000 and the health check never succeeded
- clear torch_compile_cache when the profile+nspec signature changes, avoiding vLLM
startup crashes against a stale compile cache after changing speculative depth
Both reported on the NVIDIA DGX Spark forum thread.
2026-06-27 16:35:57 +10:00
d3688cc833
load speed: persist compile cache at $HF_HOME/.vllm_cache (was wiped each container boot, -42s/boot); propagate --load-format to mtp_serve; document startup time
2026-06-24 16:32:02 +10:00
a0b661b016
make dense the default profile (validated download-only at 0.82/seqs3); README: per-workload concurrency table + dense/dflash pool tradeoff (376k vs 456k)
2026-06-24 15:40:44 +10:00
17bad6c39b
dense profile: download prebuilt hybrid checkpoint (HYBRID_REPO) instead of requiring --build-hybrid; serve as HF repo id (no /model mount); --build-hybrid kept as opt-in
2026-06-24 15:07:57 +10:00
cadfa61184
install.sh: fix false-negative GPU-access probe (--entrypoint true)
2026-06-24 14:44:29 +10:00
df8981d54c
VALIDATED defaults: gpu-mem 0.82 + seqs 3 (0.88+ over-subscribes/swaps); README corrected with measured KV pool (457k tok, 1.74x) + concurrency curve
2026-06-24 14:30:56 +10:00
dabc866507
tune for 1 orchestrator + up to 3 subagents: gpu-mem 0.88 (~15GB reserve), seqs=4, ctx 262144; document use case + tradeoffs
2026-06-24 13:31:36 +10:00
6797a9f6dd
default max-num-seqs=4 (>=2 with headroom; ~5 fit full native ctx each)
2026-06-24 13:21:52 +10:00
7dd6a74d38
tune defaults for max context + KV depth: gpu-mem 0.89 (reserve ~14GB), ctx 262144, max-num-seqs 1, decouple max-batched-tokens (8192)
2026-06-24 13:18:32 +10:00
70c91f94a7
docs: GB10 memory is 128 GB (119 GiB), not 128 GiB
2026-06-24 13:10:36 +10:00
884945ada4
install.sh: fix set-e exit from bare 'return' in build_hybrid guard
2026-06-24 13:06:03 +10:00
60bf1b7b02
qwen3.5-122B-A10B on DGX Spark: vLLM + DFlash + dense-bandwidth stack, one-shot installer
2026-06-24 13:02:35 +10:00
Entrpi