From chess to Go to Atari games, AI research has always been about games. You are unintentionally positioning yourself as more sophisticated than a huge number of AI luminaries, including Nobel and Turing winners.

Games are heavily used in RL research.

I understand that. Doing games in real time is just a performance problem that can be solved with more compute or inane optimizations. It’s not interesting research.

I think you are trivializing the field of RL research. Games are not a solved problem. Doing that efficiently in real-time is even more difficult and is highly relevant to real world applications.

Right, we have not even solved games in the non-real time environment, so why bother adding additional constraints like “on real hardware in real time”. This is exactly like Tesla trying to switch from LIDAR to cameras before self driving is even solved. It’s avoiding the real harder challenge and going off on inane tangents. John is essentially bike shedding.

In this case, John is going off on this inane tangent because of his prior experience with hardware and video games instead of challenging himself to solve the actual hard and open problems.

I’m going to predict how this plays out for the inevitable screenshot in one to two years. John picks some existing RL algo and optimizes it to run in real time on real hardware. While he’s doing this the field moves on to better and new algorithms and architectures. John finally achieves his goal and posts a vid of some (now ancient) RL algo playing some Atari game in real time. Everyone says “neat” and moves on. John gets to feel validated yet all his work is completely useless.

False dichotomy. It's not "avoiding the real harder challenge". It's solving a different problem and it is extremely relevant to real world applications. These are actual hard and open problems to be solved.

Here's a heuristic that somebody gave me a while ago: using the word "just" in the way you did is a signal that you don't understand the topic.

John's document covers why he's doing what he's doing:

> Fundamentally, I believe in the importance of learning from a stream of interactive experience, as humans and animals do, which is quite different from the throw-everything-in-a-blender approach of pretraining an LLM. The blender approach can still be world-changingly valuable, but there are plenty of people advancing the state of the art there.

He thinks interacting with the real world and learning as you go isn't getting enough attention and might take us farther than the LLM approach. So he's applying these ideas to a subject that he's an expert in. You don't seem to find this approach interesting but John does (and I do too, for the record).

Everybody dismissing him might be right. Those keeping score know that Carmack's batting average isn't one thousand. But those people also know Carmack has the resources to work on pretty much whatever he wants to work on. I'm happy he's still working hard at something and sharing his work.

for systems that learn in real-time, is there a way for the humans to know/understand how/why the system came to the conclusion it has? there are examples of where humans ran experiments that came to a conclusion for the wrong reasons. if an AI system thinks it knows the answer for the wrong reason, wouldn't that then poison its reasoning later as well? can an AI system learn its reasoning is wrong and then update it when provided better evidence? that seems to be something a vast majority of humans cannot do.

Not sure what gave you that impression. It's definitely not what I was trying to say.

> Everybody dismissing him might be right. Those keeping score know that Carmack's batting average isn't one thousand. But those people also know Carmack has the resources to work on pretty much whatever he wants to work on.

To me this reads as "this is very far-fetched but he's got the money so golly for him"