Comment on Is anyone self-hosting their own AI?

PumpkinEscobar@lemmy.world ⁨1⁩ ⁨week⁩ ago

Qwen 3.8 flash next on a 4090, using strata instead of llama.cpp, gives me about 45-55 tokens/sec. Mainly used for coding tasks. The model size is perfect. If the qwen 4.0 models can be a bit less wordy in reasoning but the same size and quality they’ll be really strong.

source
Sort:hotnewtop