If "what we actually want" even comes into consideration, we did orders of magnitude better than the current industry standard. (Poor consolation, I know.) Right now, the vast majority of AIs don't even have a concept of "human desire" – probably none of them, to be honest, though some that are good at manipulating their handlers might've come close to a particularly stupid dog's understanding. This is at the core of the Friendly AI problem: https://wiki.lesswrong.com/wiki/Friendly_artificial_intellig...
Just because we created the AI doesn't mean it'll care about us. That's like saying a maths problem will try to make your favourite number its answer, just because you wrote it. No, you are the one who must make its answer the number you want. It won't happen by chance.
Corollary: you can't patch broken FAI designs. Reinforcement learning (underlying basically all of our best AI) is known to be broken; it'll game the system. Even if they were powerful enough to understand our goals, they simply wouldn't care; they'd care less than a dolphin. https://vkrakovna.wordpress.com/2018/04/02/specification-gam...
And there are far too many people in academia who don't understand this, after years of writing papers on the subject.
Just because we created the AI doesn't mean it'll care about us. That's like saying a maths problem will try to make your favourite number its answer, just because you wrote it. No, you are the one who must make its answer the number you want. It won't happen by chance.
Corollary: you can't patch broken FAI designs. Reinforcement learning (underlying basically all of our best AI) is known to be broken; it'll game the system. Even if they were powerful enough to understand our goals, they simply wouldn't care; they'd care less than a dolphin. https://vkrakovna.wordpress.com/2018/04/02/specification-gam...
And there are far too many people in academia who don't understand this, after years of writing papers on the subject.