ComfyUI caches the last model when RAM is plentiful (unified memory), so
memory doesn't drop after switching models even though models are being
swapped, not accumulated. Add a sidebar "Free GPU memory" button that
proxies ComfyUI's POST /free (unload_models + free_memory) via a new
/api/comfyui/free endpoint (COMFYUI_URL env). Verified it releases ~7GB.
README documents this plus the --disable-smart-memory auto-unload option.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fp16 VAE produces NaNs on many SD1.5/SDXL checkpoints, yielding black
images (often mistaken for NSFW censorship - there is no NSFW filter in
the generator). Switch COMFYUI_FLAGS from --fp16-vae to --fp32-vae in
.env.example and document the fix in the README. Verified end-to-end:
a test generation now produces a real image (mean ~86) instead of black.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a checkbox dropdown to filter the CivitAI catalog by one or more base
models (SD 1.5, SDXL, Pony, Illustrious, Flux, etc.), mapped to the API's
`baseModels` array param. Backend search accepts comma-separated
base_models; button shows the selected count with a Clear action.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CivitAI Early Access versions require purchased access and otherwise fail
with 401. Surface version `availability` from the API as an `early_access`
flag (per card and per version), show an amber "EARLY ACCESS" tag on the
card, label such entries in the version dropdown, and warn before
attempting to download one.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Gallery: DELETE /api/gallery/all removes every image under output/;
"Delete all" button with in-app confirm and a deleted/failed count.
- Downloads: surface a clear, actionable message when CivitAI/HuggingFace
returns 401/403 (model requires login/early-access, or the key/token
lacks access) instead of a bare error, both at resolve time and during
the download stream.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Browsers suppress native confirm() dialogs after repeated use (the
"prevent this page from creating additional dialogs" checkbox), which
silently broke deletes. Add a promise-based in-app confirmation modal
and use it for gallery photo and installed-model deletes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ComfyUIMini's built-in gallery is view-only. Inject a "Manage Photos"
link into its sidebar (via the shared head.ejs partial at build time, so
no fork) that points to the Model Manager's delete-capable Gallery. The
URL is built client-side from the browser host; the port is baked from
the MODEL_MANAGER_PORT build arg (default 8189).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Gallery view: grid of generated photos from ComfyUI's output/, full-size
lightbox, and permanent delete (with confirm). Paginated ("Load more").
- Backend: GET /api/gallery, GET /gallery/file (path-guarded image serve),
DELETE /api/gallery (path-guarded; clear error on permission denial).
- Mount ./output read-write into model-manager so the gallery can delete.
- Device-routing landing at /start: phones -> ComfyUIMini, desktops ->
the Gallery; ?force=mobile|desktop overrides. Ports come from the new
/api/ui-config (COMFYUI_PORT / COMFYUIMINI_PORT env).
- Responsive tweaks so the gallery is usable if opened directly on a phone.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Downloads previously landed in models/ owned by root because the
container ran as root. Add `user: "${PUID:-1000}:${PGID:-1000}"` to the
model-manager service and PUID/PGID to .env.example so downloaded models
are owned by the host user. Defaults to 1000:1000.
Note: existing root-owned files under models/ and sparkyui-data/ must be
chowned once (e.g. via a one-off root container) when upgrading.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- New "Browse CivitAI" view: thumbnail grid with search, type/sort/period
filters and NSFW toggle; click a model card to download it (per-card
version picker for multi-version models). Cursor + page based "Load more".
- Backend: /api/civitai/search and /api/civitai/download endpoints; new
civitai_search() catalog helper.
- Fix 401 on paste: recognize the civitai.red mirror (and any civitai.*
host), normalize API calls to civitai.com, and always resolve the
model-version so type + filename are auto-detected for every CivitAI URL.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The mounted patches/model_management.py and patches/utils.py were authored
against an older ComfyUI, but COMFYUI_REF=master clones the latest. Upstream
added the DynamicVRAM/AIMDO system, and main.py now calls
model_management.get_all_torch_devices() (13 functions were missing in total),
causing comfyui to crash-loop on startup with AttributeError.
Regenerated both patches from the current master files and re-applied the
documented Sparky edits on top so they stay API-compatible:
- model_management.py: unified-memory detection, NORMAL_VRAM retention,
95% weight ratio, intermediate_device()->cuda, soft_empty_cache skip
- utils.py: copy=False tensor load on unified memory
comfyui now starts cleanly with DynamicVRAM enabled and the Sparky
unified-memory path active.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New FastAPI container (port 8189) to download and manage models:
- Installed Models, Add/Download (CivitAI/HuggingFace/direct URL), Settings views
- Persistent SQLite storage for API keys and download history (./sparkyui-data)
- Downloads land in ./models, auto-sorted into ComfyUI's standard subfolders
- Default COMFYUI_HOST_PATH and SPARKYUI_DATA_PATH to the project root
- Wire docker-compose service, env defaults, gitignore, README docs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Major unified memory optimization changes:
1. model_management.py: HIGH_VRAM → NORMAL_VRAM
- GB10 unified memory: offloading to CPU doesn't save physical RAM
(same pool), but NORMAL_VRAM allows per-layer partial loading when
memory is tight instead of all-or-nothing OOM
- text_encoder_offload_device() and vae_offload_device() now return
CPU (allows ComfyUI to offload unused models)
- intermediate_device() still returns GPU (VAE outputs must stay in
CUDA allocator for honest memory tracking)
- User can force HIGH_VRAM with --highvram if models fit
2. utils.py: copy=True → copy=False for tensor.to(device)
- On GB10 unified memory, copy=True creates a full duplicate in both
CPU and CUDA allocators simultaneously (ComfyUI issue #10896)
- copy=False makes .to(device) a zero-copy device label change since
both allocators draw from the same physical LPDDR5X
- Halves model loading memory usage when --disable-mmap is set
3. Removed --disable-dynamic-vram from ComfyUI flags
- Was preventing AIMDO (comfy_aimdo) from initializing
- AIMDO now activates: VBAR-based page-level VRAM management at 32MB
granularity instead of blunt .to(cpu) copies
- Falls back to NORMAL_VRAM per-layer loading if AIMDO has issues
4. Added CUDA_CACHE_MAXSIZE=4294967296 (4GB kernel cache)
- PTX→SASS kernel caching for sm_121 (GB10 Blackwell)
- 3x speedup on subsequent runs reported by DGX Spark community
5. System: vm.swappiness reduced from 60 to 1
- Swap thrashing on unified memory causes silent system freezes
- Near-zero swappiness ensures clean OOM kills instead
On Grace-Blackwell (GB10), CPU and GPU share the same physical RAM.
intermediate_device() was returning 'cpu', which means ComfyUI allocates
output buffers (like VAE decode) through the CPU allocator on the same
physical memory pool it thinks is free VRAM. This causes:
1. Memory accounting mismatch — ComfyUI thinks intermediates are 'over
there' on CPU and overestimates available VRAM
2. Unnecessary .to(device) copies through separate allocator heaps
3. Heap fragmentation across the unified memory pool
Now matches text_encoder_offload_device() and vae_offload_device() which
already return get_torch_device() on UNIFIED_MEMORY.
intermediate_device() controls where large output tensors (decoded video
frames) are accumulated. On unified memory, cpu and cuda:0 share the same
physical RAM, but the CUDA allocator has different fragmentation behavior.
With intermediate_device=cuda:0, LTX video VAE decode hung because
tiled_scale_multidim allocates the full output tensor on cuda:0 upfront,
and the CUDA allocator can't efficiently reclaim space during tiled
decode. Reverting to cpu fixes the hang.
vae_offload_device() and text_encoder_offload_device() remain cuda:0
since those model-loading paths benefit from GPU allocation.
- Add "What's Included" section listing ComfyUI, ComfyUI-Manager,
SageAttention, and PyTorch versions
- Update clone URL to actual GitHub repo (ecarmen16/SparkyUI)
- ComfyUI-Manager is auto-installed on first container run
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Docker-based ComfyUI setup for NVIDIA DGX Spark ARM64 + sm_121:
- CUDA 13.0.2 base (required for compute_121 support)
- PyTorch 2.9.1+cu130 ARM64 wheels
- SageAttention compiled with TORCH_CUDA_ARCH_LIST="12.1"
- Triton/torch.compile disabled (no sm_121 support yet)
- ComfyUI-Manager auto-installed at runtime
- Configurable model/data paths via .env
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
TBNilles
Evan Carmen