Comment on Is anyone self-hosting their own AI?
JasonDJ@lemmy.zip 4 days ago
What a timely comment.
I recently gave my kid a gaming PC for his birthday…which I used as an excuse to upgrade my GPU from a 1660 Super to a 5060ti 16GB
Combined with a 5600x3d and 32gb of system memory.
I just started messing with local LLMs yesterday. “Getting started” is rather easy. pacman -S lm-studio (I use arch, btw), launch it, download a model that your hardware supports, and start talking.
Right now I’m working on getting it to talk to vscode.
ShadowZone@lemmy.world 3 days ago
I used Cline as my VSCode chat plugin for models in LMStudio. Worked pretty well but I have moved my local LLM stack to Hermes in the meantime.
JasonDJ@lemmy.zip 3 days ago
I tried using zoo with qwen3.6 and 2.5 but it failed horrendously. My card could only handle like a 16k context window with those models, and the prompt alone was like 11k.
Tried opencode and that was fast…but can’t get it to interact with system yet. More playing to do.