Local LLM VRAM Calculator

Will it fit on your GPU? Pick a model, a GGUF quant and how much context you want. You'll see how much video memory it needs, how much context fits, and roughly how fast it will run in llama.cpp, KoboldCpp, LM Studio or Ollama.

Model

Pick a model

Model details (edit for any model not listed)

Size (B)

Layers

KV heads

Head size

Active (B), MoE only

Max context

Find these in the model's config.json on Hugging Face: num_hidden_layers, num_key_value_heads and head_dim (or hidden size ÷ attention heads).

Quantization

GGUF quant

Context

Context length

KV cache type

Q8_0 halves the context memory with almost no quality loss. Needs flash attention turned on.

Sliding-window cacheGemma models only keep a short window for most layers. Recent llama.cpp and KoboldCpp use this to save a lot of memory.

Your hardware

Graphics card

How many

System RAM (used if it spills over)

This card also runs my screensKeeps about 0.6 GB free for Windows and your desktop

Every quant for this model

At your context length and hardware. Click a row to use it. Lower quants are smaller and faster but less accurate; Q4_K_M is the usual sweet spot.

Quant	File size	Total needed	Fits?	Speed (tok/s)

How this works

Model weights

Parameters × bits per weight. A 12B model at Q4_K_M (about 4.9 bits) is around 6.9 GB. File sizes vary a few percent between uploads.

Context (KV cache)

Every token you keep in context stores keys and values for each layer: 2 × layers × KV heads × head size × context × bytes. Long chats and roleplay eat this fast.

Speed

Writing each token reads the whole model from memory, so speed mostly depends on memory bandwidth. Anything that spills into system RAM runs many times slower.

Spilling over

If it doesn't all fit, llama.cpp and KoboldCpp can keep some layers on the CPU. Set GPU layers to the number shown and it still runs, just slower.

How accurate?

Within about 10% for most setups. Drivers, flash attention, batch size and other apps using VRAM all shift the real number. Leave a little headroom.

Your data

Everything is calculated in your browser. Your last settings are remembered on this device only.