1070ti and some hexacore something cpu that’s alright with around 24gb ddr4, I think one of the slower speeds. I also got a faster PC with another llama.cpp instance and I vibe coded a quick and dirty proxy which I can use to switch between the different backends. It’s genuinely useful because I got code editors but also my Open-WebUI using it.
Sounds elaborate but generally it’s all pretty primitive.
hoshikarakitaridia@lemmy.world 2 weeks ago
1070ti and some hexacore something cpu that’s alright with around 24gb ddr4, I think one of the slower speeds. I also got a faster PC with another llama.cpp instance and I vibe coded a quick and dirty proxy which I can use to switch between the different backends. It’s genuinely useful because I got code editors but also my Open-WebUI using it.
Sounds elaborate but generally it’s all pretty primitive.