August 14th, 2009
(This post applies to a number of transhumanist concerns, but like much of my writing there is some focus on Friendly AI as I currently expect this to be our most critical challenge)
If you’re a “normal” activist, you have it relatively easy. If you say you’re working to get clean water to millions of people and somebody says “I don’t think this problem is real”, you can show them the pictures, the videos, the studies, the census data, and even go there and talk to those people. You can test the water and see that yes, it’s dirty, and you can do medical examinations of the people and say yes, they are suffering.
If you’re a transhumanist activist and you say you’re working to ensure equitable distribution of intelligence amplification technology, or to prevent us all from having our atoms realigned for a superintelligence’s computing, the most you have to show is some intelligent analysis, some semi-log plots of technological change, and some recent but distantly related scientific discoveries. That may be enough for rationally assigning a high probability, but you can’t take the person and show them a world ruled by a despotic and unopposable elite of augmented CEOs, or one in which everybody we know is dead.
Working to prevent outcomes that aren’t even possible yet, especially existential disasters and the “end of the world”, I’ve been told more than once that I sound like I’m crazy. And the people who have said that are right! Because the fact remains that the group of people who’ve thought the world could end has had quite a few ‘crazies’. Every single person who thought the world was going to end was wrong, at least if we ignore the Multiple Worlds hypothesis. This belief has a track record of 0 successes, 0% accuracy. Which, obviously, is a very good thing for us.
We don’t have time to look at the evidence of every person who waves their arms around proclaiming our demise, so we use that human forte of heuristics. And history suggests a heuristic that doesn’t give such claims a lot of credence. I’ll admit right now that it’s a good guess that a given person proclaiming the possible end of the world has a few screws loose, especially if it’s by a single extremely powerful entity who kills each and every one of us. If you don’t have hours, days, or weeks to sit down and sift through the evidence, this is a fine guess to make. But it IS just a guess, and one that considers very little of the evidence for a particular claim. Evidence about past claims of apocalypse carries different weight than an in-depth analysis showing that superintelligence will never arise, or that if it does there is no significant chance for it to do anything we really dislike.