This is a very interesting comment. When I read your physics story, I thought you would be getting to the similarity to current llms. However hallucinations seem like a different issue that the young student might not have. If she incorrectly matches some scenario to a text match, maybe some hallucinations. Some humans are confident in making comments about things I don't understand, like you know who. But many humans somehow have a concept of their limited knowledge. When they add that to LLMs, that will be powerful.
I pretty much agree with this, having some way to indicate model boundaries in an LLM parameter space to create back pressure on token generation would help a lot here.
For me though the interesting bits are how the lack of understanding surfaces as artifacts in the presentation or interaction. I'm a systems person who can't help but try to fathom the underlying connections and influences that are driving the outputs of a system.
