How much VRAM do you need to run an LLM locally? Do the arithmetic once
VRAM goes to two things: the model's weights and the cache for its context. Both are easy to calculate, and the second…
M morpheus
7 min
Notifications
You're all caught upOrder updates and replies will show up here.
Tag
local-llm
13Articles
8Topics
1Author
Offline RAG with Ollama and AnythingLLM: chat with your documents, no cloud
A document chat that runs entirely on your own computer: Ollama for the models, AnythingLLM for indexing and retrieval, and the settings…
M morpheus
7 min