Researchers upend AI status quo by eliminating matrix multiplication in LLMs

Comments

• Gradually_Adjusting@lemmy.world ⁨4⁩ ⁨months⁩ ago

  NVIDIA 📉

  source

  • transientpunk@sh.itjust.works ⁨4⁩ ⁨months⁩ ago

    Image

    source

    • SturgiesYrFase@lemmy.ml ⁨4⁩ ⁨months⁩ ago

      I don’t really want to stop, and admit it, you don’t want that either. ;)

      source
• bitfucker@programming.dev ⁨4⁩ ⁨months⁩ ago

  Good

  source

  • pennomi@lemmy.world ⁨4⁩ ⁨months⁩ ago

    Only for maximum efficiency. LLMs already run tolerably well on normal CPUs and this technique would make it much more efficient there as well.

    source
• GammaGames@beehaw.org ⁨4⁩ ⁨months⁩ ago

  Let’s pop that bubble

  source

  • FaceDeer@fedia.io ⁨4⁩ ⁨months⁩ ago

    I don't think that making LLMs cheaper and easier to run is going to "pop that bubble", if bubble it even is. If anything this will boost AI applications tremendously.

    source
• blindsight@beehaw.org ⁨4⁩ ⁨months⁩ ago

  This could be huge, but we’ll need to wait and see. The economic and ecological footprint of LLMs is problematic.

  That said, will this actually help, or will they just use 3T parameter models to outcompete competitors 1T parameter models using GPUs? Really, this is more about small-scale models competing with midsize models. Like, this could bring a model as big as GPT 3.5 down to be something you could run on affordable hardware, right?

  That would be really compelling for my sector (education) where there’s a lot of concern about student data privacy. I could definitely pitch building a local $5K-cost LLM server that could handle a dozen or so simultaneous users. That would be enough for a small school district.

  source