markovs_gun@lemmy.world 5 days ago
Yeah the “nobody understands why” is an absurd statement when it’s pretty obvious, they’re training it on itself and keep trying to “improve” it with shit that’s making it worse. Plus, if it’s using reinforcement learning based on interactions with the general public (learning based on user responses to and ratings of bot responses) the more “like” the general public the LLM will become, aka stupid. Furthermore, I have a personal theory that LLM power users who aren’t programmers are dumber than average or at the very least less creative, and this will also skew results if user responses are given a heavy weight.
kadup@lemmy.world 5 days ago
It’s also just a language model. People have trouble internalising what this means, because it sounds smarter than it actually is.
ChatGPT does not reason in the same way you think it does, even when they offer those little reasoning windows that show the “thought process”.
It’s still only predicting the next likely word based on the previous word. It can do that many times and feed in extra words to direct it one way or another, but that’s very different from understanding a topic and reasoning within it.
So as you keep pushing the model to learn more and more, you start getting many artifacts because it’s not actually learning these concepts - it’s just getting more data to infer “what’s the most likely word X that would follow words Z, Y and A?”
markovs_gun@lemmy.world 5 days ago
You’re missing a crucial detail- the discussions of LLMs as just being probability machines ignores the fact that when we’re talking about “what is the most likely next word?” The answer to that question isn’t merely just “What, in the training corpus, was the most likely word to follow in this instance?” But rather there is a “finger on the scale” so to speak in favor of certain types of responses, and this is frequently updated. You cannot have a useful LLM without this because it will talk as if it is a human with a sense of self, display blatant prejudice just because it’s common, and say creepy things (see the Microsoft “Sydney” fiasco) because the humans who wrote the sources in the training corpus do have a sense of self, do hold prejudices, and express thoughts and feelings that are inappropriate coming from a chatbot. When done intentionally and carefully, this creates a much more useful product, but when done poorly it potentially makes things worse. It seems that at least part of that weighting is based on user interaction, which is what I was talking about with models getting dumber the more they interact with the general public.
Furthermore, the newest versions of ChatGPT attempt to include “reasoning” as an actual feature of the response. I’ve played around with them and they are definitely a lot better at logic and math problems than older models but not necessarily less prone to “hallucinations” when it comes to factual information. I haven’t read a whole lot about how the “reasoning” works because I have been a lot more interested in non-LLM methods lately but it is intended to combat the issue you described. Personally I am not convinced this will fix much of anything in its current strategy but it’s certainly interesting to see.
kadup@lemmy.world 5 days ago
You’re not wrong, but it’s still a language model, which is not the entirety of how intelligence and reasoning works. There are clear limitations that do not arise only from people toying around with ChatGPT, but are known for decades of theoretical understanding of what language is.