Running large language models (LLMs) like DeepSeek-R1, Llama-4, or Qwen-2.5 locally on consumer GPUs often hits a frustrating wall: RuntimeError: CUDA error: out of memory or sudden drops from 65 tokens/sec down to a painful 4 tokens/sec. In most cases, your GPU actually has enough raw VRAM for the model weights, but silent allocation spikes in the KV (Key-Value) Cache, unoptimized context windows, or WSL2 memory leaks push usage past physical limits.

⚡ Before tweaking configs: Calculate the exact VRAM required for your model weights, quantization level, and context window using our free Interactive AI Model VRAM Calculator on NextByte Tech →

1. Why Ollama and vLLM Crash With CUDA Out of Memory

When you load a 7B, 14B, or 32B parameter model, total GPU memory consumption is governed by three separate components:

  • Model Weights: Fixed size determined by parameter count and quantization (e.g., Q4_K_M vs FP16).
  • KV Cache (Context Buffer): Scales linearly or quadratically with context length (num_ctx) and parallel request slots.
  • Compute & Activation Overhead: Temporary CUDA kernels and FlashAttention buffers (typically 0.8 GB to 2.2 GB).

If your weights + KV cache exceed physical VRAM by even 150 MB on Windows 11, the NVIDIA driver spills shared memory over PCIe into system DDR5 RAM, causing generation speeds to plummet by 85% or triggering an immediate OOM crash on Linux.

2. VRAM Usage Breakdown by Model Size & Quantization

Model Size FP16 Weights Q8_0 Quant Q4_K_M Quant 8K Context KV Cache (FP16 / Q8) Recommended Minimum GPU VRAM
8B Parameters 16.2 GB 8.6 GB 4.9 GB 1.2 GB / 0.6 GB 8 GB (RTX 4060 / RX 7600)
14B Parameters 28.4 GB 15.1 GB 8.9 GB 2.1 GB / 1.1 GB 12 GB (RTX 4070 / RTX 3060 12GB)
32B Parameters 64.5 GB 34.2 GB 19.8 GB 4.4 GB / 2.2 GB 24 GB (RTX 3090 / RTX 4090)
70B Parameters 140.0 GB 74.5 GB 42.5 GB 8.8 GB / 4.4 GB 48 GB (Dual RTX 3090 / Cloud A6000)

3. Fix #1: Enable Flash Attention & Quantize the KV Cache in Ollama

By default, Ollama allocates the KV cache in 16-bit floating point (FP16). Switching KV cache storage to q8_0 or q4_0 cuts context memory usage by 50% to 75% with virtually zero perplexity loss when paired with Flash Attention.

On Linux (systemd service), edit your Ollama environment configuration:

🚀
Interactive Tool

RAM Speed & True Latency Calculator

Calculate exact real response times in nanoseconds (ns) and dual-channel bandwidth for AMD Ryzen & Intel PC builds.

Calculate Latency →
Terminal / Script
sudo systemctl edit ollama.service # Add these lines under [Service]: [Service] Environment="OLLAMA_FLASH_ATTENTION=1" Environment="OLLAMA_KV_CACHE_TYPE=q8_0" Environment="OLLAMA_NUM_PARALLEL=1" Environment="OLLAMA_MAX_LOADED_MODELS=1"

Reload and restart the service:

Terminal / Script
sudo systemctl daemon-reload && sudo systemctl restart ollama

Setting OLLAMA_NUM_PARALLEL=1 is critical for single-user workstations. Without it, Ollama pre-allocates 4 parallel context slots, multiplying your KV cache footprint by 4x automatically.

4. Fix #2: Cap Context Window (`num_ctx`) & Prevent System RAM Spillover

Many third-party WebUIs (Open WebUI, AnythingLLM, or IDE coding extensions like Continue and Roo Code) request a 32,768 or 65,536 token context window. On a 32B model, a 32K context window demands over 12 GB of VRAM for the KV cache alone. Lock a custom Modelfile with a predictable context boundary:

Terminal / Script
FROM deepseek-r1:32b-q4_K_M PARAMETER num_ctx 8192 PARAMETER num_batch 512 PARAMETER num_gpu 99

Build and run the tuned profile:

Terminal / Script
ollama create deepseek-32b-tuned -f ./Modelfile ollama run deepseek-32b-tuned

5. Fix #3: Tune vLLM `--gpu-memory-utilization` and `--max-model-len`

When deploying production endpoints with vLLM, the engine pre-allocates 90% of total GPU VRAM on startup (`--gpu-memory-utilization 0.90`). If your display server, browser, or background Docker container is using 1.5 GB of VRAM, vLLM will crash during CUDA graph capture.

Terminal / Script
python3 -m vllm.entrypoints.openai.api_server \ --model Qwen/Qwen2.5-Coder-14B-Instruct-AWQ \ --quantization awq \ --dtype half \ --max-model-len 8192 \ --gpu-memory-utilization 0.85 \ --kv-cache-dtype fp8 \ --enforce-eager
  • --enforce-eager: Disables heavy CUDA graph pre-allocation, saving 1.2 GB to 2.5 GB of VRAM on 12 GB and 16 GB cards.
  • --max-model-len 8192: Prevents vLLM from trying to reserve KV blocks for the model's native 128K maximum length.

6. Fix #4: Disable NVIDIA Sysmem Fallback on Windows 11 & WSL2

On Windows 11, NVIDIA Driver 560+ automatically overflows full VRAM into system RAM instead of throwing an error, dropping speeds from 50 tok/s to 2 tok/s. Open NVIDIA Control Panel → Manage 3D Settings → CUDA - Sysmem Fallback Policy and set it to Prefer No Sysmem Fallback. In WSL2, create %UserProfile%\.wslconfig to release zombie cached memory:

Terminal / Script
[wsl2] memory=24GB swap=8GB autoMemoryReclaim=gradual sparseVhd=true

Apply these four optimizations and your local inference stack will maintain peak memory bandwidth without unexpected CUDA OOM terminations.

About the Author: Binod Bhatt is a systems developer and hardware researcher at NextByte Tech, specializing in Linux kernel benchmarks, local AI deployments, and interactive hardware optimization tools.