Comment on Is anyone self-hosting their own AI?
twilightwolf90@lemmy.world 6 days agoHow well does it handle concurrency? I could see this as a vast improvement on llamacpp, but I was already speccing out vLLM.
Comment on Is anyone self-hosting their own AI?
twilightwolf90@lemmy.world 6 days agoHow well does it handle concurrency? I could see this as a vast improvement on llamacpp, but I was already speccing out vLLM.