Comment on AI PCs flow into distribution pipeline, but who wants them?
rumba@lemmy.zip 5 weeks ago
I certainly don’t wan to run windows on it :)
I’ve been running llama keep my telemetry out of the hands of Microsoft/Google/"open"AI. I’m kind of shocked how much I can do locally with a half assed video card, and offline model and a hacked up copy of searxng.
PhilipTheBucket@ponder.cat 5 weeks ago
To me, that’s the killer flaw of these things.
It would be great if they were designed from the ground up to be good machines for running models, say with a GPU that had a copious amount of memory that didn’t cost $1,500 for an add-on. Unfortunately, to do that they’d have to create something from nothing, so instead they’ve added something that is worse than most GPUs, added some dumb software which is designed to pair with the ultimate result of disappointing people, and called it a day.
rumba@lemmy.zip 5 weeks ago
You can get a lot done currently with ARC. The mobile ARC versions share system memory, So if you get a mini PC with ARC and upgrade it to 96GB, you can share system ram with the GPU and load decently large models. They’re a little slow it not being vram and all, but still useful (and cheap)
www.youtube.com/watch?v=xyKEQjUzfAk
I have it running on a zenbook duo with 32GB so I can’t load the 70B models, but I works shockingly well.