 ent
|
bdb57d776d
|
load speed: --load-format fastsafetensors (read straight to device, no mmap) — weight load 463s -> 32s (14.5x), time-to-READY ~12min -> ~3min on GB10. Falls back to nogds automatically.
|
2026-06-24 16:30:33 +10:00 |
|
|
ent
|
9b7be16bee
|
int8 lm-head: output bf16 (match stock lm-head dtype) instead of fp32 — halves the logits buffer that was inflating vLLM's profiled activation reserve and shrinking the dense KV pool
|
2026-06-24 15:53:58 +10:00 |
|
|
ent
|
df8981d54c
|
VALIDATED defaults: gpu-mem 0.82 + seqs 3 (0.88+ over-subscribes/swaps); README corrected with measured KV pool (457k tok, 1.74x) + concurrency curve
|
2026-06-24 14:30:56 +10:00 |
|
|
ent
|
dabc866507
|
tune for 1 orchestrator + up to 3 subagents: gpu-mem 0.88 (~15GB reserve), seqs=4, ctx 262144; document use case + tradeoffs
|
2026-06-24 13:31:36 +10:00 |
|
|
ent
|
6797a9f6dd
|
default max-num-seqs=4 (>=2 with headroom; ~5 fit full native ctx each)
|
2026-06-24 13:21:52 +10:00 |
|
|
ent
|
7dd6a74d38
|
tune defaults for max context + KV depth: gpu-mem 0.89 (reserve ~14GB), ctx 262144, max-num-seqs 1, decouple max-batched-tokens (8192)
|
2026-06-24 13:18:32 +10:00 |
|
|
ent
|
60bf1b7b02
|
qwen3.5-122B-A10B on DGX Spark: vLLM + DFlash + dense-bandwidth stack, one-shot installer
|
2026-06-24 13:02:35 +10:00 |
|