Comment on Is anyone self-hosting their own AI?
anamethatisnt@sopuli.xyz 4 weeks agoGemma 4 12b unified as a q6 gguf fits with about 24k context inside your vram. koboldcpp+sillytavern works well on the software side.
Comment on Is anyone self-hosting their own AI?
anamethatisnt@sopuli.xyz 4 weeks agoGemma 4 12b unified as a q6 gguf fits with about 24k context inside your vram. koboldcpp+sillytavern works well on the software side.
laz@lemmy.dbzer0.com 4 weeks ago
Sc117