> LLMs are ‘just a next-token predictor’. The only problem with this phrasing is the implication of the use of the word ‘just’.
Well said. People confuse a simple task definition (token continuation), with a simple task, and simple solutions.
But there is no limit to how complex the patterns in data for a continuation task can be.
And there is no limit to the complexity and power that a solution must implement to perform that task.
So these models are not "just" token continuation engines. They are human conversation models, capturing much of the complexity of human thought and knowledge contained in the example conversations.
--
Another factor I don't see given enough weight. The models are currently operating with severe restrictions compared to humans:
1. only given an architecture allowing a limited number of steps (before incoherence).
2. forced to continually output tokens without any "hidden" steps for contemplation, not even initial contemplative steps.
3. required to perform all reasoning within their limited in-line memory, as apposed to being able to use any side-memory such as (the digital version of) note pads or white board.
4. required to generate one answer, whole and coherent, without option to post edit.
5. required to be aware and familiar with a vast cross section of human knowledge, far beyond any single person's fluency, and mix that knowledge sensibly. Languages, subjects, modes of thought, communication and roles.
6. Limited modes of information, i.e. text. (A limit which is now being removed for many models.)
Within those incredible restrictions, they are already vastly better at "reasoning" than a human being being asked to perform the same task with the same restrictions.
--
So at this point, they reason better than us, within those limits. None of us, even experts in an area being tested, would avoid making numerous simple mistakes with those restrictions.
And outside those limitations, they just don't operate yet - so our reasoning is mixed, but often supreme. I say mixed, because no human has familiarity with so much information, or fluency in so many modes of communication, as these models have now - even given a day or two to respond.
Writing a supreme court brief on the implications to a mathematical theorem to the economic impact of some political event in five languages, in normal speech, song, pig latin, Dr. Seuss prose, and as a James Bond story respectively, all in the perspective of a version of the US run by a German federal system in an alternate world where World War II came down differently, isn't something any human being could do as well, no matter how imperfect current models responses to that request might be.
So we are already in territory where they are vastly better at reasoning than us, within clear constraints. And in some cases, better than us outside those constraints, even if those cases might be contrived.
--
That is all verifiable, even obvious.
As far as opinion, I don't see any indication that as those limitations are overcome they won't continue to vastly exceed us - within whatever limitations still exist.
Our brains are amazing. But its worth noting we are not able to scale in terms of training intensity, data quantity, computational power, precise and unwavering memory cells, etc. And our cells have enormous demands on them that transistors don't: maintaining their own structure, metabolism, fighting off chemical and biological adversaries, etc.
Models also train single mindedly. Human learning is always within our biological context - we continually subdivide our attention in real time, in order to remain situationally aware of threats, needs and our social environment. And our brains enforce time limits on our best performance, due to the real time need to remetabolize basic means of operation (neurochemicals), and expectations of limited energy resources.
And models amortize the training of just one model instance, across the inference of any number of copies of that model. They never get tired. Efficiency savings vs. humans that are unprecedented.
I find most pessimistic dismissive arguments are based on criticism against some imagined standard of perfection, instead of a standard of what humans are really capable of when performing the same task with the same limitations.
Humans are notoriously unreliable.
(Some criticisms are valid: such as models current greater propensity to confabulate what they don't know, vs. humans having more awareness of what we remember vs. when we unconsciously fill in the blanks of our memories - but humans do that too. Ask any detective who takes witness testimony.)
> 2. forced to continually output tokens without...
I reference one paper that explores this. It lets a model output identical ellipsis tokens that are then discarded before the final output. It worked pretty well and is interesting.
There is another restriction that I thought about, and think I mentioned a bit in the article which is the grammatical structure. All model outputs must be grammatically sound so you would expect the first few and last few layers to be dedicated to decoding and encoding any thoughts into grammatically correct language. This restricts the number of layers working with free-form thoughts.
> I find most pessimistic dismissive arguments are based on criticism against some imagined standard of perfection...
We are in agreement there. They have their imperfections and issues, we have ours. Our natures are fundamentally different so this should be expected.
Well said. People confuse a simple task definition (token continuation), with a simple task, and simple solutions.
But there is no limit to how complex the patterns in data for a continuation task can be.
And there is no limit to the complexity and power that a solution must implement to perform that task.
So these models are not "just" token continuation engines. They are human conversation models, capturing much of the complexity of human thought and knowledge contained in the example conversations.
--
Another factor I don't see given enough weight. The models are currently operating with severe restrictions compared to humans:
1. only given an architecture allowing a limited number of steps (before incoherence).
2. forced to continually output tokens without any "hidden" steps for contemplation, not even initial contemplative steps.
3. required to perform all reasoning within their limited in-line memory, as apposed to being able to use any side-memory such as (the digital version of) note pads or white board.
4. required to generate one answer, whole and coherent, without option to post edit.
5. required to be aware and familiar with a vast cross section of human knowledge, far beyond any single person's fluency, and mix that knowledge sensibly. Languages, subjects, modes of thought, communication and roles.
6. Limited modes of information, i.e. text. (A limit which is now being removed for many models.)
Within those incredible restrictions, they are already vastly better at "reasoning" than a human being being asked to perform the same task with the same restrictions.
--
So at this point, they reason better than us, within those limits. None of us, even experts in an area being tested, would avoid making numerous simple mistakes with those restrictions.
And outside those limitations, they just don't operate yet - so our reasoning is mixed, but often supreme. I say mixed, because no human has familiarity with so much information, or fluency in so many modes of communication, as these models have now - even given a day or two to respond.
Writing a supreme court brief on the implications to a mathematical theorem to the economic impact of some political event in five languages, in normal speech, song, pig latin, Dr. Seuss prose, and as a James Bond story respectively, all in the perspective of a version of the US run by a German federal system in an alternate world where World War II came down differently, isn't something any human being could do as well, no matter how imperfect current models responses to that request might be.
So we are already in territory where they are vastly better at reasoning than us, within clear constraints. And in some cases, better than us outside those constraints, even if those cases might be contrived.
--
That is all verifiable, even obvious.
As far as opinion, I don't see any indication that as those limitations are overcome they won't continue to vastly exceed us - within whatever limitations still exist.
Our brains are amazing. But its worth noting we are not able to scale in terms of training intensity, data quantity, computational power, precise and unwavering memory cells, etc. And our cells have enormous demands on them that transistors don't: maintaining their own structure, metabolism, fighting off chemical and biological adversaries, etc.
Models also train single mindedly. Human learning is always within our biological context - we continually subdivide our attention in real time, in order to remain situationally aware of threats, needs and our social environment. And our brains enforce time limits on our best performance, due to the real time need to remetabolize basic means of operation (neurochemicals), and expectations of limited energy resources.
And models amortize the training of just one model instance, across the inference of any number of copies of that model. They never get tired. Efficiency savings vs. humans that are unprecedented.
I find most pessimistic dismissive arguments are based on criticism against some imagined standard of perfection, instead of a standard of what humans are really capable of when performing the same task with the same limitations.
Humans are notoriously unreliable.
(Some criticisms are valid: such as models current greater propensity to confabulate what they don't know, vs. humans having more awareness of what we remember vs. when we unconsciously fill in the blanks of our memories - but humans do that too. Ask any detective who takes witness testimony.)