Can Your GPU Run It? VRAM Compatibility Checker for 80+ LLMs

How to read these compatibility tables

The tables below show which models fit on each GPU at Q4_K_M quantization with 4K context length. Green means the model fits comfortably with headroom. Tight means it technically fits but leaves less than 2 GB free for longer contexts. Offload means it exceeds the GPU VRAM and requires partial CPU offloading, which reduces speed significantly. For the exact VRAM numbers behind these compatibility ratings, see our complete VRAM reference table.

8 GB GPUs: RTX 4060, RTX 3060 Ti, RX 7600

8 GB is the entry point for local LLM inference. You can run 7B models comfortably and 8B models at Q4, which covers capable models like Qwen 3 8B and Llama 3.1 8B. The 2026 generation of MoE models also helps here — Gemma 4 E2B loads 5.1B total but only activates 2B, giving you excellent quality per GB. If you are buying a GPU specifically for LLMs, 8 GB should be considered the absolute minimum.

12 GB GPUs: RTX 3060 12 GB, RTX 4070

The RTX 3060 12 GB is the best value entry point for local LLMs. For around $180 used, you get enough VRAM to run 14B models at Q4 comfortably. This is the tier where local LLMs start feeling genuinely useful for coding, writing, and analysis tasks. The 2026 MoE models also shine here — gpt-oss 20B MoE fits with room to spare despite packing flagship-class knowledge.

16 GB GPUs: RTX 4070 Ti Super, RTX 5080, RX 9070 XT

16 GB is the tier where MoE models really start to flex. gpt-oss 20B MoE fits at Q4 with 4 GB headroom despite 20B total parameters. You can also run dense 14B models at Q5 for near-native quality. This is also where AMD GPUs become competitive — the RX 9070 XT and RX 7800 XT both offer 16 GB at lower prices than NVIDIA equivalents, though with software ecosystem tradeoffs covered in our AMD vs NVIDIA comparison.

24 GB GPUs: RTX 4090, RTX 3090, RX 7900 XTX

24 GB is the sweet spot for single-GPU local LLM inference. This is the tier where Qwen 3 32B and Gemma 4 31B at Q4 run with comfortable context headroom. MoE models like Qwen 3 30B-A3B fit easily with lots of room for long context. Both the RTX 4090 and the used RTX 3090 sit here — see our 3090 vs midrange comparison for why the used 3090 at $450 is often the smarter buy for LLM workloads.

The standout here is Qwen 3 32B at Q4_K_M. It fits in 19 GB, runs fast on a single GPU, and benchmarks close to GPT-4 class output for many tasks. If you have a 24 GB GPU, this should be your daily driver model. The MoE models (Qwen 3 30B-A3B, Gemma 4 26B-A4B) are even more compelling if you prioritize inference speed — they activate only 3-4B parameters per token.

32 GB GPUs: RTX 5090

The RTX 5090 is the first consumer GPU to break the 24 GB barrier. The extra 8 GB opens the door to running 70B models at Q3 (which fits in ~30 GB) and running 32B models at Q5 or Q8 with full context headroom. For the full breakdown of whether the premium over an RTX 4090 is worth it for LLM workloads, see our RTX 5090 vs 4090 comparison.

AMD GPU compatibility notes

AMD GPUs with ROCm support can run most LLMs via Ollama and llama.cpp. The RX 7900 XTX (24 GB) has the same VRAM capacity as an RTX 4090, so the model compatibility tables above apply identically. However, there are ecosystem differences to be aware of when choosing between AMD and NVIDIA for LLM workloads.

• -ROCm vs CUDA: Most inference frameworks support both, but CUDA gets updates first and has better documentation. See our AMD GPU guide for the current state of ROCm support.
• -Flash Attention: AMD supports Flash Attention on RDNA3 GPUs via ROCm, but performance is typically 10-20% behind NVIDIA's implementation.
• -Multi-GPU: AMD's multi-GPU support for LLM inference is less mature than NVIDIA's NCCL-based tensor parallelism. If you plan to run dual GPUs, NVIDIA is the safer choice.
• -vLLM support: vLLM supports AMD GPUs but may lag behind CUDA versions by weeks for new features. llama.cpp and Ollama have more parity between AMD and NVIDIA.

Check your exact GPU-model compatibility

The tables above cover the most common GPU-model combinations at Q4_K_M. But your specific setup may use different quantization levels, context lengths, or less common models. The check your exact setup — enter your GPU, pick your model, set quantization and context length, and get a VRAM breakdown with a pass/fail verdict for your hardware.

Related Guides