Open Menu
AllLocalCommunitiesAbout
lotide
AllLocalCommunitiesAbout
Login

Forget megabucks Nvidia GPUs, apparently all you need to run an LLM is a Pentium II CPU from 1997

⁨15⁩ ⁨likes⁩

Submitted ⁨⁨10⁩ ⁨months⁩ ago⁩ by ⁨cm0002@lemmy.world⁩ to ⁨technology@lemmy.zip⁩

https://www.pcgamer.com/software/ai/forget-megabucks-nvidia-gpus-apparently-all-you-need-to-run-an-llm-is-a-pentium-ii-cpu-from-1997/

source

Comments

Sort:hotnewtop
  • AmbiguousProps@lemmy.today ⁨10⁩ ⁨months⁩ ago

    The other catch is that the model is tiny, which will cause hallucinations to skyrocket.

    source
    • projectmoon@forum.agnos.is ⁨10⁩ ⁨months⁩ ago

      @ambiguousprops@lemmy.today still an impressive feat, though. Surprised that anything resembling modern software even runs on a system that old now. Probably requires an ancient kernel version and other things. Is there anywhere where they detail this process more specifically? (I only had time to skim the article)

      source
      • AmbiguousProps@lemmy.today ⁨10⁩ ⁨months⁩ ago

        The GitHub has more details. github.com/exo-explore/llama98.c

        source