How much VRAM do you need to run an LLM locally? Do the arithmetic once
VRAM goes to two things: the model's weights and the cache for its context. Both are easy to calculate, and the second…
M morpheus
7 min
Notifications
You're all caught upOrder updates and replies will show up here.
Category
Self-hosted AI
Running models and AI apps on your own hardware: sizing GPUs, local RAG, and open-source alternatives to hosted chat tools.
13Articles
8Topics
1Author
Self-hosted ChatGPT alternatives: picking one by who will use it
LibreChat, Open WebUI, AnythingLLM, LobeHub and Onyx all run private AI chat. The right one depends on your users, the licence and…
M morpheus
7 min
Offline RAG with Ollama and AnythingLLM: chat with your documents, no cloud
A document chat that runs entirely on your own computer: Ollama for the models, AnythingLLM for indexing and retrieval, and the settings…
M morpheus
7 min