a77cbdab26
docs(readme): link dgx-spark-serving-mode for freeing unified memory
main
Entrpi
2026-06-29 10:14:40 +10:00
41d92cb6f9
feat(serve): enable prefix caching with DFlash (on by default)
Entrpi
2026-06-28 21:29:40 +10:00
3f05a87cdf
fix(serve): let empty TOOL_PARSER/REASONING_PARSER disable the parser
Entrpi
2026-06-27 17:29:27 +10:00
55c0c1702f
fix(install): wire --port into the container and clear stale compile cache on nspec change
Entrpi
2026-06-27 16:35:57 +10:00
72badaf39a
serve: enable OpenAI auto tool-calling (qwen3_xml) + reasoning split by default
Entrpi
2026-06-24 23:08:31 +10:00
2cb5033fb8
kv: recover dense pool 376k->427k (+13%) at the same 0.82 headroom
Entrpi
2026-06-24 17:41:07 +10:00
1d0f2e1af3
docs: fix Reproducing — --start defaults to dense now; dflash is the explicit opt-in
ent
2026-06-24 16:46:22 +10:00
4e5c9552fe
docs: refactor README so dense is unambiguously the default and the hosted hybrid (bleysg/...int4-fp8-hybrid) is the primary/chosen checkpoint (title link, model bullet, quick start, related-work)
ent
2026-06-24 16:45:57 +10:00
d3688cc833
load speed: persist compile cache at $HF_HOME/.vllm_cache (was wiped each container boot, -42s/boot); propagate --load-format to mtp_serve; document startup time
ent
2026-06-24 16:32:02 +10:00
bdb57d776d
load speed: --load-format fastsafetensors (read straight to device, no mmap) — weight load 463s -> 32s (14.5x), time-to-READY ~12min -> ~3min on GB10. Falls back to nogds automatically.
ent
2026-06-24 16:30:33 +10:00
9b7be16bee
int8 lm-head: output bf16 (match stock lm-head dtype) instead of fp32 — halves the logits buffer that was inflating vLLM's profiled activation reserve and shrinking the dense KV pool
ent
2026-06-24 15:53:58 +10:00
a0b661b016
make dense the default profile (validated download-only at 0.82/seqs3); README: per-workload concurrency table + dense/dflash pool tradeoff (376k vs 456k)
ent
2026-06-24 15:40:44 +10:00
efd80e8fe7
docs: dense profile downloads the prebuilt hybrid (no build step)
ent
2026-06-24 15:08:13 +10:00
17bad6c39b
dense profile: download prebuilt hybrid checkpoint (HYBRID_REPO) instead of requiring --build-hybrid; serve as HF repo id (no /model mount); --build-hybrid kept as opt-in
ent
2026-06-24 15:07:57 +10:00
314ae39000
docs: refactor README to a consistent neutral technical-documentation voice (drop second-person/session-narrative); add conc_workloads.py
ent
2026-06-24 14:54:40 +10:00
df8981d54c
VALIDATED defaults: gpu-mem 0.82 + seqs 3 (0.88+ over-subscribes/swaps); README corrected with measured KV pool (457k tok, 1.74x) + concurrency curve
ent
2026-06-24 14:30:56 +10:00
dabc866507
tune for 1 orchestrator + up to 3 subagents: gpu-mem 0.88 (~15GB reserve), seqs=4, ctx 262144; document use case + tradeoffs
ent
2026-06-24 13:31:36 +10:00
6797a9f6dd
default max-num-seqs=4 (>=2 with headroom; ~5 fit full native ctx each)
ent
2026-06-24 13:21:52 +10:00
7dd6a74d38
tune defaults for max context + KV depth: gpu-mem 0.89 (reserve ~14GB), ctx 262144, max-num-seqs 1, decouple max-batched-tokens (8192)
ent
2026-06-24 13:18:32 +10:00
70c91f94a7
docs: GB10 memory is 128 GB (119 GiB), not 128 GiB
ent
2026-06-24 13:10:36 +10:00
dc9d40a1c4
docs: credit albond's recipe as the foundation + head-to-head comparison (59.0 vs 51.58 e2e, same method)
ent
2026-06-24 13:08:52 +10:00
884945ada4
install.sh: fix set-e exit from bare 'return' in build_hybrid guard
ent
2026-06-24 13:06:03 +10:00
60bf1b7b02
qwen3.5-122B-A10B on DGX Spark: vLLM + DFlash + dense-bandwidth stack, one-shot installer
ent
2026-06-24 13:02:35 +10:00
Entrpi