llama-cpp

How much VRAM do you need to run an LLM locally? Do the arithmetic once

VRAM goes to two things: the model's weights and the cache for its context. Both are easy to calculate, and the second…