Endmaker
@Endmaker@ani.social
- Comment on My new laptop chip has an 'AI' processor in it, and it's a complete waste of space 1 week ago:
Someone with the expertise should correct me if I am wrong; it’s been 4-5 years since I learnt about NPUs during my internship so I am very rusty:
You don’t even need a GPU if all you want to do is to run (i.e. perform inference using) a neural network (abbreviating it to NN). Just a CPU would do if the NN is sufficiently lightweight. The GPU is only needed to speed up the training of NNs.
The thing is, the CPU is a general-purpose processor, so it won’t be able run the NN optimally / as efficiently as possible. Imagine you want to do something that requires the NN and as a result, you can’t do anything else on your phone / laptop (it won’t be problem for desktops with GPUs though).
Where NPU really shines is when there are performance constraints on the model: when it has to be fast, lightweight and memory efficient. Use cases include mobile computing and IoT.
In fact, there’s news about live translation on Apple AirPod. I think this may be the perfect scenario for using NPUs - ideally housed within the earphones directly but if not, within a phone.
Disclaimer: I am only familiar with NPUs in the context of “old-school” convolutional neural networks (boy, tech moves so quickly). I am not familiar with NPUs for transformers - and LLMs by extension - but I won’t be surprised if NPUs have been adapted to work with them.
- Comment on My new laptop chip has an 'AI' processor in it, and it's a complete waste of space 1 week ago:
I’m pretty sure most video games require AI already. I have difficulty naming even one that doesn’t use AI.
Neural nets, on the other hand - I find it hard to imagine one running locally without impacting performance.
- Comment on I just found out 'Age of War' is on steam 4 months ago:
Having Glorious Morning as the background music is perfect