Comment on [AIT] what is your selfhosted local LLM stack?

communism@lemmy.ml ⁨4⁩ ⁨days⁩ ago

Literally just having ollama installed and some models that work well with the amount of VRAM I have. I don’t auto-start the daemon because I rarely use it. Occasionally I use it for proof-reading/feedback on things like cover letters, or debugging. I don’t let it write anything. If it has something useful to say it can tell me in the chat and I can implement it if it’s good. Usually I don’t need it though so it just sits on my disk for the most part.

source
Sort:hotnewtop