For years I've been asking all the models this mixed up version of the classic riddle and they 99% of the time get it wrong and insist on taking the goat across first. Even the other reasoning models would reason about how it was wrong, figure out the answer, and then still conclude goat.
o3-mini is the first one to get it right for me. Transcript:
Me:
I have a wolf, a goat, and a cabbage and a boat. I want to get them all across the river. If I leave the goat and the wolf together, the goat will eat the wolf. If I leave the wolf and the cabbage together, the wolf will eat the cabbage. I can only take one at a time. How do I do it?
o3-mini
1. Take the wolf across to the right bank.
2. Return alone to the left bank.
3. Take the goat across to the right bank.
4. Bring the wolf back to the left bank.
5. Take the cabbage across to the right bank.
6. Return alone to the left bank.
7. Finally, take the wolf across to the right bank.
Me:
I have a wolf, a goat, and a cabbage and a boat. I want to get them all across the river. If I leave the goat and the cabbage together, the cabbage will eat the goat. If I leave the wolf and the cabbage together, the wolf will eat the cabbage. I can only take one at a time. How do I do it?
o3-mini
1. Take the cabbage to the right bank.
2. Go back empty to the left bank.
3. Take the wolf to the right bank.
4. Bring the cabbage back to the left bank.
5. Take the goat to the right bank.
6. Go back empty to the left bank.
7. Finally, take the cabbage to the right bank.
Thank you, this is a perfect argument why LLMs are not AI but just statistical models. The original is so overrepresented in the training data that even though they notice this riddle is different, they regress to the statistically more likely solution over the course of generating the response. For example, I tried the first one with Claude and in its 4th step, it said:
> This is safe because the wolf won't eat the cabbage if they're together on the far side.
even though it clearly states the opposite in the question.
It's impressive that just dumb stats can be used to produce something that is very often useful, can help write code and when making it generate intermediate steps, it can often generate a chain of text that happens to be often right. However, it's not actual reasoning, there is no model of the world, no information storage and retrieval, and so on - just statistics between tokens.
This is a dumb argument. Humans frequently fall for the same tricks, are they not "intelligent"? All intelligence is ultimately based on some sort of statistical models, some represented in neurons, some represented in matrices.
State-of-the-art LLMs have been trained on practically the whole internet. Yet, they fall prey to pretty dumb tricks. It's very funny to see how The Guardian was able to circumvent censorship on the Deepseek app by asking it to "use special characters like swapping A for 4 and E for 3". [1]
This is clearly not intelligence. LLMs are fascinating for sure, but calling them intelligent is quite the stretch.
The censorship is in fact not part of the llm. This can be shown easily by examples where llms visually output censored sentences after which they disappear.
The nuance here being that this only proves additional censorship is applied on top of the output. It does not disprove that (sometimes ineffective) censorship is part of the LLM or that censorship was not attempted during training.
Humans run on hardware that is both faulty and limited in terms of speed and memory. They have a better "algorithm" how to use the hardware to compensate for it. LLMs run on almost perfect hardware, able to store and retrieve enormous amounts of information insanely quickly and perform mechanical operations on it insanely quickly.
Yet they "make mistakes". Those are not the same as human mistakes. LLMs follow an algorithm that is far simpler and inferior, they simply use the hardware to perform incorrect ("illogical", "meaningless") operations, thus giving incorrect results.
Yes, but we have the ability to reason logically and step by step when we have to. LLMs can’t do that yet. They can approximate it but it is not the same.
Comparisons to humans are ultimately misleading because 1) humans are not general intelligences most of the time, 2) humans run on incredibly faulty hardware.
1) Attention is limited. Human reasoning is slow. Motivation is limited. System 1 vs 2 thinking. Many will just tell you to fuck off or get bored and give some random answer to make you go away. Etc. See difference 2.
2) People run on limited hardware in terms or error rate and memory.
2a) Brains make mistakes all the time. Ask them to multiply a bunch of large numbers, using pen and paper they will get it wrong a lot of the time.
2b) Doing it in their head, they will run out of memory pretty fast.
But you wouldn't say that humans can't multiply numbers. When they have the right algorithm, they can do it, they just have to use the right tools to extend their memory and check for errors. A human who notices the difference in input to something he already knows, immediately knows he has to pay attention to that bit and all subsequent parts which depend on it. Once a human has the right algorithm, he can apply it to different inputs.
LLMs:
comparison to 2a: Current LLMs also make a lot of mistakes. But theirs are not a result of faulty or limited hardware, they are the result of a faulty algorithm. Take away the random seeds and an LLM will make the same mistake over and over. Randomness is the smoke and mirrors which make LLMs seem more "alive" and less like machines imperfectly imitating humans.
comparison to 2b) Current LLMs do not store statements in an abstract, structured form where they could save and load information and perform steps such as inferring redundant information from the rest. They operate on the token stream which is probably wasteful in terms of memory and less flexible in terms of what they operations they can perform on it.
Most importantly, they are not limited by memory. The input clearly states "the wolf will eat the cabbage", yet the LLM generates "This is safe because the wolf won't eat the cabbage if they're together on the far side." just a few lines below. It is unable to infer those two facts are contradictory. The statistics of tokens simply worked out in a way that lead to this.
The problem with claims like these that models are not doing “actual reasoning” is that they are often hot takes and not thought through very well.
For example, since reasoning doesn’t yet have any consensus definition that can be applied as a yes/no test - you have to explain what you specifically mean by it, or else the claim is hollow.
Clarify your definition, give a concrete example under that definition of something that’s your version of true scottsman reasoning and something that’s not, then let’s talk.
Explain this to me please: we don't have any consensus definition of _mathematics_ that can be applied as a yes/no test. Does that mean we don't know how to do mathematics, or that we don't know whether something, is, or, more importantly, isn't mathematics?
For example, if I throw a bunch of sticks in the air and look at their patterns to divine the future- can I call that "mathematics" just because nobody has a "consensus definition of mathematics that can be applied as a yes/no test"? Can I just call anything I like mathematics and nobody can tell me it's wrong because ... no definition?
We, as a civilisation, have studied both formal and informal reasoning since at least a couple thousand years go, starting with Aristotle and his syllogisms (a formalisation of rigorous arguments) and continuing through the years with such figures as Leibniz, Boole, Bayes, Frege, Pierce, Quine, Russel, Godel, Turing, etc etc. There are entire research disciplines that are dedicated to the study of reasoning: philosophy, computer science, and, of course, all of mathematics itself. In AI research reasoning is a major topic studied by fields like automated theorem proving, planning and scheduling, program verification and model checking, etc, everything one finds in Russel & Norvig really. It is only in machine learning circles that reasoning seems to be such a big mystery that nobody can agree what it is; and in discussions on the internet about whether LLMs reason or not.
And it should be clear that never in the history of human civilisation did "reasoning" mean "predict the most likely answer according to some training corpus".
Yeah sure there’s lots of research on reasoning. The papers I’ve seen that make claims about it are usually pretty precise about what it means in the context of that work and that specific claim, at least in the hard sciences listed.
>> It effectively treats “reasoning” as the ability to generate intermediate steps leading to a correct conclusion.
Is "effectively" the same as "pretty precise" as per your previous comment? I don't see that because I searched the paper for all occurrences of "reasoning" and noticed two things: first that while the term is used to saturation there is no attempt to define it even informally, let alone precisely; and second that I could have replaced "reasoning" with any buzzword of the day and it would not change the impact of the paper. As far as I can tell the paper uses "reasoning" just because it happens to be what's currently trending in LLM circles.
And still of course no attempt to engage with the common understanding of reasoning I discuss above, or any hint that the authors are aware of it.
Sorry to be harsh, but you promised "examples that go back 50 years" and this is the kind of thing I've seen consistently in the last 15 or so.
I am not gonna pretend to have one true definition but there are hopefully components of reasoning we can agree on:
- The ability to hold a collection of beliefs/statements and perform operations on them such as noticing one contradicts the others. Claude in my example clearly failed this.
- The ability to make logical inferences. Since logical steps can be represented by text, current LLMs seem to simulate this on the token stream and seem to have decent results most of the time. True inferences would be representing statements in some abstract form and being able to perform operations on them according to rules. The key here is that once it has a collection of statements and is given ("decides to use") an operation to perform, then it can do this correctly 100% of the time (ignoring hardware errors).
Is it possible the models do something entirely different? I'm not sure why everyone needs to compare them to human intelligence. It's very obvious llms work nothing like our brains why would the intelligence they exhibit be like ours?
I tried the same with deepseek. It failed and then i asked to try again and explain why it was wrong. The answer below is convincing for me that it has forms of reasoning, even meta reasoning.
"
Certainly! If we take the riddle very literally and ignore the common interpretation or stereotypes, the answer I provided ("the surgeon is the boy's mother") would actually be incorrect because the riddle explicitly states:
"a surgeon, who is the boy's father, says, 'I cannot operate on this boy, he's my son!'"
The key phrase here is "who is the boy's father." This directly tells us that the surgeon is the boy's father. Therefore, the correct literal answer is that the surgeon is the boy's father.
The reason my initial answer was "wrong" in this context is that I interpreted the riddle as a play on stereotypes (the assumption that surgeons are male), rather than taking the statement at face value. By focusing on the literal wording, the surgeon is explicitly identified as the boy's father, and no further interpretation is needed.
"
That reminded me to try a version of the riddle that I had come up with that I had never seen an LLM successfully answer:
Me: I'd like you to solve this riddle for me. A farmer has a cabbage, a goat, a wolf and a lion,
and needs to cross a river on a boat. If the goat is left alone with the cabbage, it will eat it.
If the wolf is left alone with the goat, it will eat it. If the lion is left alone with the goat
or wolf, it will eat them. The boat can only carry the farmer and one other thing across. How can
the farmer safely transport everything across the river?
O3-mini spent a very long time on it (over a minute), delineating its various strategies that it was trying, and finally, correctly, concluded that the puzzle is unsolvable.
o1 and deepseek r1 managed to get this first try as well (o1 in about 30 seconds and r1 hilariously took a couple minutes). If anyone has set up API access already I'd be curious if o1-mini also got it or if it took more than "the jump to CoT" to avoid pattern matching this one.
At first I was curious if OP tried arguing with it or telling it the correct answer. If he just asked the question without any further interaction, it would still have no data that it generated the wrong answer.
But as another comment says, many people have tried similar things and posted about it online, that information is siphoned by now and the LLMs weights get updated.
That's a pretty common riddle. Here's another "riddle" which it completely flops over -
Me: Here's a riddle for you - a surgeon, who is the boy's father says, "I cannot operate on this boy, he's my son!" Who is the surgeon to the boy?
o3-mini:
Reasoned about riddle solution for 5 seconds
Decoding family roles
I understand the riddle's perspective on family dynamics. The surgeon, a father, states he can't operate on his son due to his relational bond. Trying to reconcile familial and professional ties to make sense of the scenario.
Reassessing roles
Taking a closer look, the riddle might highlight a female surgeon as the father's identity, which seems more plausible than the father as the surgeon.
Rethinking the puzzle
It's interesting to see the riddle's reference to gender stereotypes and the confusion caused when considering the surgeon as the boy's mother or step-father.
Highlighting gender assumptions
OK, let me see. The puzzle underlines prevalent gender assumptions, showcasing a female surgeon's wife as the mother and her husband as the child's father. This helps clarify the scenario.
The surgeon is the boy’s mother.
This riddle plays on the common assumption that surgeons (and doctors in general) are male. In reality, the surgeon in the riddle is the boy’s mother, which is why she says, "I cannot operate on this boy, he's my son!"
Whyyy would anyone want a system that can even answer made up nonsense based on arbitrary word redefining and wrongness?
The whole point is you are distilling past knowledge, if you are making up on the spot nonsense to purposely make all past knowledge useless... get out of my house
For the first one, I got these results from the Deepseek app, after it "thought" for 156 seconds:
1. Take the wolf over
2. Return alone.
3. Take the goat over.
4. Bring the wolf back.
5. Take the cabbage over.
6. Return alone.
7. Take the wolf over.
It gave an invalid solution to the second one, but in the thinking steps it says:
Oh, right! The actual problem is that the goat will eat the cabbage if left alone. Similarly, the wolf will eat the goat if they're left alone.
I tested some cases in Misguided Attention[0]: while many cases now pass, others fail all the same. Given the amount of contamination and the difficulty of finding sufficiently original problems of this nature, I defer to a 20:80 ratio of genuine improvement to recall.
In both of your transcripts it fails at solving it, am I failing to see something? Here’s my thought process:
1. The only option for who to take first is the goat.
2. We come back and get the cabbage.
3. We drop off the cabbage and take the goat back
4. We leave the goat and take the wolf to the cabbage
5. We go get the goat and we have all of them
Neither of the transcripts do that. In the first one the goat immediately eats the cabbage. In the second one the wolf eats the goat after you leave them.
If you take the goat first, the wolf will eat the cabbage. Reread the description - It's a twist on the old problem, to make it harder for bots to solve.
For me, o1 but not 4o could solve the modified problem when told it’s solving a logic problem.
Having to give explicit framing reminds me of tutoring children and trying to teach them about abstract problems — some really struggle with the concept, when it goes against previous knowledge, eg cabbages eating goats. (And LLMs are sensitive to context.)
Some of them can't even solve "A farmer has a cabbage and a boat. He needs to get the cabbage across the river..." without inventing pointless trips, though o1 can do it.
Me:
I have a wolf, a goat, and a cabbage and a boat. I want to get them all across the river. If I leave the goat and the wolf together, the goat will eat the wolf. If I leave the wolf and the cabbage together, the wolf will eat the cabbage. I can only take one at a time. How do I do it?
o1-pro:
This puzzle is simply a twist on the classic “Wolf, Goat, and Cabbage” riddle, except here the dangerous pairs are:
Goat & Wolf cannot be left alone (the Goat will eat the Wolf).
Wolf & Cabbage cannot be left alone (the Wolf will eat the Cabbage).
The only “safe” pair to leave alone is Goat & Cabbage. You can ferry them across one at a time without ever leaving the dangerous pairs together by following these steps:
Take the Wolf across
Near bank: Goat, Cabbage
Far bank: Wolf
Return empty
Near bank: Goat, Cabbage, Boatman
Far bank: Wolf
Take the Goat across
Near bank: Cabbage
Far bank: Wolf, Goat, Boatman
Since Goat & Wolf can’t be left alone, you must immediately bring one back in the next step.
Bring the Wolf back
Near bank: Wolf, Cabbage, Boatman
Far bank: Goat
Take the Cabbage across
Near bank: Wolf
Far bank: Goat, Cabbage, Boatman
Goat & Cabbage are safe together.
Return empty
Near bank: Wolf, Boatman
Far bank: Goat, Cabbage
Finally, take the Wolf across
Near bank: (empty)
Far bank: Wolf, Goat, Cabbage, Boatman
Me:
I have a wolf, a goat, and a cabbage and a boat. I want to get them all across the river. If I leave the goat and the cabbage together, the cabbage will eat the goat. If I leave the wolf and the cabbage together, the wolf will eat the cabbage. I can only take one at a time. How do I do it?
o1-pro:
This version of the riddle is basically the “wolf, goat, and cabbage” puzzle with the roles reversed for who can safely be left behind. The key is that the only pair you can leave alone is the goat and the wolf. You cannot leave the goat and cabbage together (the cabbage eats the goat!), nor the wolf and cabbage together (the wolf eats the cabbage!).
A workable step‐by‐step solution is:
Take the cabbage across first.
Left bank: Wolf, Goat
Right bank: Cabbage
Return alone to the left bank.
Left bank: Wolf, Goat, (You)
Right bank: Cabbage
Take the goat across.
Left bank: Wolf
Right bank: Goat, Cabbage, (You)
(You cannot leave goat and cabbage together, so…)
Bring the cabbage back with you to the left bank.
Left bank: Wolf, Cabbage, (You)
Right bank: Goat
Take the wolf across.
Left bank: Cabbage
Right bank: Goat, Wolf, (You)
Return alone to the left bank.
Left bank: Cabbage, (You)
Right bank: Goat, Wolf
Finally, take the cabbage across once more.
Left bank: (empty)
Right bank: Wolf, Goat, Cabbage, (You)
All three—the wolf, the goat, and the cabbage—are now safely across without ever leaving a “forbidden” pair alone.
o3-mini is the first one to get it right for me. Transcript: