Why is AGI so hungry? Our brains learn on the fly and do amazing things and how much energy do they use?
So IMO we certainly haven't hit the "right technology" yet and the attempt to achieve it by spending billions, building nuclear power plants etc, is vulnerable to a technical development. So why should we screw up our environmental situation for something that can obviously be done in a vastly better way on the energy equivalent of 3 square meals a day?
Your brain has more capacity than an average LLM, and a self-supervised training regime etched into it by millions of years of evolution. Its inductive bias is very well tuned for the environment. It still takes decades to build itself up to peak performance.
And at its peak? Human brain doesn't actually have an overwhelming advantage over an LLM. It's a mixed bag of advantages and disadvantages.
LLMs think fast, and can input and output data much faster than a human. But they struggle to work on the same task for a long time, and have a problem with visual inputs and object manipulation. LLMs have more knowledge in total, but humans have better meta-knowledge, which is useful for hallucination avoidance. LLMs can only learn in context efficiently, but humans learn continuously and retain what they learned. LLMs and humans are currently trading blows when it comes to inference energy efficiency - especially when you account for things like sleep or rest.
I don't think there's a "right technology" at all. There may not be a state-change upgrade that gets us x100000 on a dime and goes all the way to an AGI on every smartphone and an ASI in any datacenter worth the name. I expect there to be a lot of little +5% and +10% upgrades that add up over time.
The history of AI has been to reject human inspired forms and then be forced to adopt them.
OK that's a sweeping statement but I think it encapsulates the general attitude quite well for a single sentence. We fly in planes which are not like birds but they have wings and tails and flaps and slats and we're even getting into making the wings warp.
Sleep is needed for memory I think so there is a price to pay for continuous learning. It's important to be able to forget too. - to rotate the logfiles so to speak. AGI only interests me because it will be able to understand us - therefor it has to work like us and experience things like us.
So IMO we certainly haven't hit the "right technology" yet and the attempt to achieve it by spending billions, building nuclear power plants etc, is vulnerable to a technical development. So why should we screw up our environmental situation for something that can obviously be done in a vastly better way on the energy equivalent of 3 square meals a day?