Comment on Is anyone self-hosting their own AI?
Nighed@feddit.uk 4 days ago
My server’s old GPU (8gb) can’t run anything useful LLW wise I don’t think.
Have run some stuff locally on my main computer, but nothing setup as a proper service.
Comment on Is anyone self-hosting their own AI?
Nighed@feddit.uk 4 days ago
My server’s old GPU (8gb) can’t run anything useful LLW wise I don’t think.
Have run some stuff locally on my main computer, but nothing setup as a proper service.
Natanox@discuss.tchncs.de 4 days ago
Not for complex tasks, really. Gemma4b or similar might be just enough as the backend for a voice assistant replacing Alexa (which I honestly hope, I need to get family members off of that fucking spy devices…). Small web searches might just fit. Or image recognition for Nextcloud Memories, stuff like that.
Nighed@feddit.uk 3 days ago
Immich’s image recognition works fine off the CPU. I did get it running an LMM, but it was useless.