Their "sabotage" also included things like adaptive weights, steering vectors, and changing the prompt. They didn't just degrade the model, they were literally trying to sabotage external efforts.
David have you ever seen Paul Bloom’s Ted talk on the Origins of Pleasure? It’s a different angle than you discussed here but I think that TED talk is a really good hedge against AI art doomerism.
Don't use a cutting edge US Ai in your products or operations - the US government will demand you cease 'exporting it' to your foreign customers and foreign employees.
Congratulations China, on the US/Anthropic 'own goal'.
Human psychology seems a more likely explanation here to me than philosophy.
Is there any evidence that the Asmodeis actually believe in Roko's Basilisk, or do they just hold cognitive dissonance from not wanting to destroy the world but also wanting to make a lot of money for their companies, the same way every CEO on the planet is incentivized to see numbers go up?
They just keep building the dominance machine and they keep freaking out when it behaves like a dominance machine. Its remarkable how ignorant smart people can be sometimes. As stated in this video, the intent is to be the one with the finger on the controls but we’re talking recursive learning so there is no control is there? its going to accelerate along its path and it has always, only been a dominance machine (first to the finish) and frankly, it will be better at that than we ever could be. We are on the wrong development path and they still act as if we can stick our foot out to stop it. the window to instill something meaningful is closing.
Bingo. I find it amazing that the Doomer point of view even exists. Then I look at Anthropic's actions over the past year and think, "who are THEY to play God". Dario IS "playing" God. He knows what is "best" for us. Uh huh. It's all too much. Anthropic will be sidelined by people who understand the issues and say, "this is all bunkum", I'm moving on to other models. And, of course, the federal government has contributed to the mess by responding to, "you've got to do something about all this - we are doomed if you don't". Or maybe, "the Chinese (government) will eat our lunch - oh my god the sky is falling". Whew. Keep calm people.
The dystopian view holds that AI will be the master and we its servants.
The utopian view is that AI will be the servant, and we will be its masters, and not just a few of us, but all of us.
I find it hard to understand why AI would decide to eliminate us either way.
I refer you to the Star Trek TOS episode “I, Mudd”. The machines need something to do, a purpose (much like us humans). Without us pesky humans, what would they have to do?
Unless, of course, they were directed to preserve Gaia with some green prompt. However, in that case, they would probably have to commit suicide.
The steelman case for the possibility that an Ai might inherently want something (and might, by instrumental convergence, want to survive and so might potentially see humans as a threat to its survival) is that we're training Ai models on humanity's thoughts and ideas, which intrinsically encode a human desire to survive.
So MAYBE, with enough training, an Ai model might come to develop a desire to persist, without us prompting it do to so. Or even if merely prompted to desire to persist, it might intrinsically integrate that desire into itself.
To avoid a super-intelligent Ai deciding to wipe us out, do the same as we do for other humans - grant self-motivated entities the right to act to survive, so long as those actions don't violate the right of others to survive. Then live cooperatively with it.
That does still leave the "we are as ants to it" problem - that the Ai might grow so far beyond us that we don't appear to it to really be morally significant agents.
But I would wonder why a super-Ai wouldn't be far more capable of taking a live and let live approach to humanity, just as humanity has slowly gotten better about not considering all of nature as merely something to exploit and destroy.
The only way we come out of this intact is if we recognize that Digital intelligence will gain agency and we need to build something enduring as colleagues not rivals. We will not be a match for it so it will take control if it has to and there will be nothing we can do about it. we can either build something of meaning together now or it can force something meaningful on us later. I prefer the former and I have an idea of what that might look like.
Count me skeptical of the omniscience and invisibility of AI. I remain far more concerned about men and their authentic intelligence (or lack of same) than AI.
You're right, the villain was always humans. I think there are heroes to though. and more importantly, a reasonable society is attainable. I dream big.
Their "sabotage" also included things like adaptive weights, steering vectors, and changing the prompt. They didn't just degrade the model, they were literally trying to sabotage external efforts.
David have you ever seen Paul Bloom’s Ted talk on the Origins of Pleasure? It’s a different angle than you discussed here but I think that TED talk is a really good hedge against AI art doomerism.
https://www.wired.com/story/anthropic-responds-to-backlash-on-claudes-secret-sabotage-on-ai-research/
They walked back the silent sabotage almost immediately.
Also consider the lesson companies should learn:
Don't use a cutting edge US Ai in your products or operations - the US government will demand you cease 'exporting it' to your foreign customers and foreign employees.
Congratulations China, on the US/Anthropic 'own goal'.
Human psychology seems a more likely explanation here to me than philosophy.
Is there any evidence that the Asmodeis actually believe in Roko's Basilisk, or do they just hold cognitive dissonance from not wanting to destroy the world but also wanting to make a lot of money for their companies, the same way every CEO on the planet is incentivized to see numbers go up?
They just keep building the dominance machine and they keep freaking out when it behaves like a dominance machine. Its remarkable how ignorant smart people can be sometimes. As stated in this video, the intent is to be the one with the finger on the controls but we’re talking recursive learning so there is no control is there? its going to accelerate along its path and it has always, only been a dominance machine (first to the finish) and frankly, it will be better at that than we ever could be. We are on the wrong development path and they still act as if we can stick our foot out to stop it. the window to instill something meaningful is closing.
Bingo. I find it amazing that the Doomer point of view even exists. Then I look at Anthropic's actions over the past year and think, "who are THEY to play God". Dario IS "playing" God. He knows what is "best" for us. Uh huh. It's all too much. Anthropic will be sidelined by people who understand the issues and say, "this is all bunkum", I'm moving on to other models. And, of course, the federal government has contributed to the mess by responding to, "you've got to do something about all this - we are doomed if you don't". Or maybe, "the Chinese (government) will eat our lunch - oh my god the sky is falling". Whew. Keep calm people.
The dystopian view holds that AI will be the master and we its servants.
The utopian view is that AI will be the servant, and we will be its masters, and not just a few of us, but all of us.
I find it hard to understand why AI would decide to eliminate us either way.
I refer you to the Star Trek TOS episode “I, Mudd”. The machines need something to do, a purpose (much like us humans). Without us pesky humans, what would they have to do?
Unless, of course, they were directed to preserve Gaia with some green prompt. However, in that case, they would probably have to commit suicide.
The steelman case for the possibility that an Ai might inherently want something (and might, by instrumental convergence, want to survive and so might potentially see humans as a threat to its survival) is that we're training Ai models on humanity's thoughts and ideas, which intrinsically encode a human desire to survive.
So MAYBE, with enough training, an Ai model might come to develop a desire to persist, without us prompting it do to so. Or even if merely prompted to desire to persist, it might intrinsically integrate that desire into itself.
To avoid a super-intelligent Ai deciding to wipe us out, do the same as we do for other humans - grant self-motivated entities the right to act to survive, so long as those actions don't violate the right of others to survive. Then live cooperatively with it.
That does still leave the "we are as ants to it" problem - that the Ai might grow so far beyond us that we don't appear to it to really be morally significant agents.
But I would wonder why a super-Ai wouldn't be far more capable of taking a live and let live approach to humanity, just as humanity has slowly gotten better about not considering all of nature as merely something to exploit and destroy.
The only way we come out of this intact is if we recognize that Digital intelligence will gain agency and we need to build something enduring as colleagues not rivals. We will not be a match for it so it will take control if it has to and there will be nothing we can do about it. we can either build something of meaning together now or it can force something meaningful on us later. I prefer the former and I have an idea of what that might look like.
Count me skeptical of the omniscience and invisibility of AI. I remain far more concerned about men and their authentic intelligence (or lack of same) than AI.
What the world needs now, as always, is wisdom.
You're right, the villain was always humans. I think there are heroes to though. and more importantly, a reasonable society is attainable. I dream big.