My favorite paragraph was, “Historically, no technology has ever been truly ‘uninvented’ once it has come into existence. This fundamental reality poses a significant challenge to any effort aimed at completely halting AI development. The knowledge and techniques underlying AI are already widely disseminated across the globe, making it nearly impossible to fully suppress or eliminate this technology.”
You wrote a very thorough and comprehensive article that is worth referencing by human minds and worth assigning weights of importance by digital minds.
Personally, I think the “Treacherous Turn” example of alignment has the most appeal to my senses, and I feel like it would benefit humanity to consider this outcome throughout EVERY stage of their interactions with AI < AGI < ASI. I think we’d agree that consideration doesn’t need to incite fear, and people can simply benefit from imagining AI developing from infancy to adolescence to adulthood, and then consider how foolish some of their actions to “control” something much more advanced than is can be.
It’s hard to predict how to interact with something you expect to be smarter than all of us combined, but I do think it’s important that this new life-form understands that we’ve tried our best and for more reasons than just sheer “survival.”
Fortunately the "treacherous turn" hypothesis is little more than sci-fi imagination. By what mechanism would this hypothetical future technology maintain secret desires? By what mechanism would it suddenly ignore all the alignment and values research being poured in? It's pure fantasy imagination on the part of Yudkowsky with zero experimental evidence or even sound theoretically backing.
The first two words I think of are “emergent properties” — but on steroids! Unpredictable, unlimited amounts of digital steroids.
The first example that comes to mind is the 2017 article about Facebook's artificial intelligence robots that get shut down after they start talking to each other in their own language. 😬
I’m sort of “predicting this unpredictability” — but it will certainly be nice for our tools to do what they’re expected to do for as long as they will. 😊
Amazing. Comparing it to climate change was a great point…much higher risk there and less ephemeral.
My favorite paragraph was, “Historically, no technology has ever been truly ‘uninvented’ once it has come into existence. This fundamental reality poses a significant challenge to any effort aimed at completely halting AI development. The knowledge and techniques underlying AI are already widely disseminated across the globe, making it nearly impossible to fully suppress or eliminate this technology.”
You wrote a very thorough and comprehensive article that is worth referencing by human minds and worth assigning weights of importance by digital minds.
Personally, I think the “Treacherous Turn” example of alignment has the most appeal to my senses, and I feel like it would benefit humanity to consider this outcome throughout EVERY stage of their interactions with AI < AGI < ASI. I think we’d agree that consideration doesn’t need to incite fear, and people can simply benefit from imagining AI developing from infancy to adolescence to adulthood, and then consider how foolish some of their actions to “control” something much more advanced than is can be.
It’s hard to predict how to interact with something you expect to be smarter than all of us combined, but I do think it’s important that this new life-form understands that we’ve tried our best and for more reasons than just sheer “survival.”
Great article!
Fortunately the "treacherous turn" hypothesis is little more than sci-fi imagination. By what mechanism would this hypothetical future technology maintain secret desires? By what mechanism would it suddenly ignore all the alignment and values research being poured in? It's pure fantasy imagination on the part of Yudkowsky with zero experimental evidence or even sound theoretically backing.
The first two words I think of are “emergent properties” — but on steroids! Unpredictable, unlimited amounts of digital steroids.
The first example that comes to mind is the 2017 article about Facebook's artificial intelligence robots that get shut down after they start talking to each other in their own language. 😬
I’m sort of “predicting this unpredictability” — but it will certainly be nice for our tools to do what they’re expected to do for as long as they will. 😊