Long Prompts on a 16GB M4: Memory, Swap, and First Response
Qwen3-8B on a 16GB M4: nine Metal runs measure first response, RSS, KV memory, compression and swap. A separate 16K scout hit a 120-second cutoff.
VRAM LAB / SECTION
Questions, conditions, results, and their limits.
Qwen3-8B on a 16GB M4: nine Metal runs measure first response, RSS, KV memory, compression and swap. A separate 16K scout hit a 120-second cutoff.
Measured Hub downloads, Arrow files, import order, saved datasets, and cleanup on WSL2. Why HF_DATASETS_CACHE and cache_dir leave other files behind.
On an RTX 5060 Ti, llama.cpp's default CUDA build sent mixed q8_0/q4_0 Flash Attention to CPU; one build flag restored CUDA and an 86x PP gap.
All three PyTorch 2.13 wheels saw my RTX 5060 Ti, but cu126 mixed working dot/GEMM paths with missing native CUDA kernels.
I tested four reasons torch.cuda.is_available() returns False on WSL2: a CUDA 13 driver mismatch, CPU-only wheel, hidden GPU, and wrong Python.
On WSL 2.7.3, compressed and sparse VHDX files booted but --resize failed with 0xc03a001a; clearing the attribute restored 9/9 retries.
I tested HF_HOME, HF_HUB_CACHE, symlinks, and /mnt/c on WSL2, then measured cleanup behavior with the current hf cache commands.
Deleted 20 GB inside WSL but ext4.vhdx stayed huge? Measured on WSL 2.7.3: fstrim no longer needed, sparse VHD gated as unsafe, diskpart rejects sparse files.