 ent
|
d3688cc833
|
load speed: persist compile cache at $HF_HOME/.vllm_cache (was wiped each container boot, -42s/boot); propagate --load-format to mtp_serve; document startup time
|
2026-06-24 16:32:02 +10:00 |
|
|
ent
|
df8981d54c
|
VALIDATED defaults: gpu-mem 0.82 + seqs 3 (0.88+ over-subscribes/swaps); README corrected with measured KV pool (457k tok, 1.74x) + concurrency curve
|
2026-06-24 14:30:56 +10:00 |
|
|
ent
|
dabc866507
|
tune for 1 orchestrator + up to 3 subagents: gpu-mem 0.88 (~15GB reserve), seqs=4, ctx 262144; document use case + tradeoffs
|
2026-06-24 13:31:36 +10:00 |
|
|
ent
|
6797a9f6dd
|
default max-num-seqs=4 (>=2 with headroom; ~5 fit full native ctx each)
|
2026-06-24 13:21:52 +10:00 |
|
|
ent
|
7dd6a74d38
|
tune defaults for max context + KV depth: gpu-mem 0.89 (reserve ~14GB), ctx 262144, max-num-seqs 1, decouple max-batched-tokens (8192)
|
2026-06-24 13:18:32 +10:00 |
|
|
ent
|
60bf1b7b02
|
qwen3.5-122B-A10B on DGX Spark: vLLM + DFlash + dense-bandwidth stack, one-shot installer
|
2026-06-24 13:02:35 +10:00 |
|