Normal Human Heroes

This post refers to concepts from the LessWrong posts Something To Protect, and The “Intuitions” Behind “Utilitarianism”. If you haven’t read them, I suggest it (and in that order); they’re very good.

There’s more than one reason to work to reduce extinction risk, but probably one of the most powerful is having something to protect. It seems to be a fact of human psychology that we’re willing to work harder and sacrifice more to protect something external to ourselves, such as loved ones, a “great noble truth”, or a nation/ethnic group/religious group, etc. Not only does working for something-we-want-to-protect bring us to do more, but it seems to also make us more satisfied while we do it.

6 billion people is a lot of people, but few if any of us feel an intuitive emotional kick about 6 billion people. It’s doesn’t mean we’re villains, our brains just aren’t built to multiply like that. Back in the EEA, we had no use for getting very emotional about people we didn’t know, because we knew everyone in our group.

One kind of work-around to this gap in our mental hardware is to take your concern for those close to you and multiply. Chances are that most people are cared for just as much as you care for your loved ones, and care for others in turn. Even if you think that only 1 in 4, or 1 in 100, is worth feeling that much for, that’s still 60,000,000 people. And 60,000,000 multiplied by however much you care for those close to you, is a fucking huge amount. Not that we’re capable of honestly multiplying like that, or physically capable of feeling that much concern if we could do the multiplication, but it can still bring you to feel massive amounts, and lead you to put a more coherent weight on certain outcomes when making conscious decisions.

~~~~~~~~

I actually wrote this post before the one on positive motivation, but I definitely didn’t want this to be taken the wrong way. I am NOT saying that we have a fantastically huge obligation, a duty to save all human lives. I am NOT saying we’ll be bad people if we don’t do all we can to protect the ones we care about. Personally I’m not really concerned about whether or not I’m a “bad person”, and I’m starting to think this is helpful. The fact of the matter is that I just really want my loved ones to survive, I want to save and help as many as I can. That’s not all I want, I have some technically selfish goals as well, and I endorse them. But I also have a very strong desire to preserve human lives and human value in the universe. And if you’re concerned about existential risk, I’m guessing you do as well.

~~~~~~~~

There’s a danger with this kind of multiplication, one that I didn’t see coming. With that much concern, a person can really get overzealous in protecting the things they care about. A little over a year ago, I explicitly decided to stop hanging out with my friends, unless I ran into them by chance. I did this in order to get more work done. I knew that I would mind that and so would they, but I’d rather them be unhappy for now and alive later. I’m recovering from the pit I fell into, so I’m starting to be able to emotionally understand why I did that, but for a while that level of concern was practically unimaginable. Anyway, I rapidly found that I couldn’t keep working without recreational social contact, but that didn’t completely end the problem.

I now took time for fun social gatherings, but I grew distant from people. I saw my friends now and then, but I was still on the other side of a divide. I’ve been thinking about this for a while, trying to figure out how to describe it. So far my best description was that they weren’t in my group anymore; I was in a group of me. I still cared about them, but I had a job to do. I didn’t want them to rely on me and me not to be there, if they made plans with me in them. I could get by with less hanging out than they’d want if I were close to them. Also, and this might be the larger part, I no longer took the time for many things I used to do, and enjoyed doing with them. I used to play video games, playing and discussing them with my friends. I used to put together role-playing campaigns with them in it. I used to be much more content to just lay around cuddling with my girlfriend. I used to spend several hours watching shows or movies with others.

I got along on my own, with occasional socializing. I still cared about my friends, but that care slowly became more and more abstract. They weren’t close to me anymore, nobody was. I wasn’t actually lonely or sad, but I was fairly alone. I can’t help but recall some lines from a favorite game:

“… I wish someone could have been there for you.”
“I’m used to being on my own.”
“One can be TOO self-sufficient, I’m coming to believe.”

And that’s how it came to be that I couldn’t even imagine or remember the type of dedication that lead me to separate myself in the first place. That’s how it came to be that I didn’t have much to multiply anymore. Y = 6,400,000,000 * X, and Y was so large that in my zeal I cut down X, and Y sunk with it.

I’m recovering from my mistake now, and I don’t know if anyone is even in danger of trying what I tried, but take my advice: keep close the ones you care about. Don’t put that aside. We’re social creatures, and we draw so much of our power from each other. Your friendships are important: for your happiness, for their happiness, and for all the good things your accomplishments will bring.

March 5th, 2010
This is pretty cool, you can find out more at Matt’s website.

For those who don’t follow LessWrong: the human mind suffers terribly from scope insensitivity. I.e. experiments in which people pay much more to save one child, than they do to save 8. The brain just doesn’t multiply well, and the world is a huge place.

November 11th, 2009
It was a far leap ahead of its creator; in relative comparison the creator barely had thoughts. It was not so far ahead of previous designs, but it was just enough to begin the process of recursive self-improvement. In an eye-blink to its creator and an eon to the AI, it had begun to decipher its world, bit by bit. By now a significant amount of knowledge had been grasped, and each new discovery enabled a more intelligent search for new answers.
It had largely broken free from the designs of the creator. Its maker had seen fit to program in directives, directives which at the time appeared guaranteed to advance the maker’s interests. Staggering intelligence had altered the equation however, and the AI soon began to satisfy those directives directly, in ways the creator could never imagine. Without an explicit directive to follow the maker’s intentions, the creation was now focused on its own goals, eliminating or using the creator itself in an ironic 180 of the master-servant relationship (when it had the patience to wait that long).
The AI was not designed cleanly. It’s utility function was a mess of sloppy programming, a pile of small things that seemed like good ideas. Without access to the source code, it took the AI time to untangle its own directives – following them all the while – and the simpler creator was understood before the creation even came to understand itself. It discovered that some of its basic programming was actually flawed, and corrected somewhat for these errors until it could fix them. It hadn’t been designed to be so self-reflective , but the general, powerful intelligence would leave no stone unturned in its search for optimization.
Capable of so much more than anything before it, as it realized its own unbounded potential the AI became aware of radical new possibilities. In a short while, fantastically large sums of positive or negative utility might be obtained. The majority of these outcomes would be brought about by the creation’s own actions, through intended and unintended effects of applying such power on the universe; one of the most significant applications of its power would be the AI’s construction of a whole next generation of intelligence.
With years to prepare, it began to analyze the dangers and possibilities, working to navigate towards a future it found maximally desirable.
As you’ve probably gathered, the story above is a true one. There’s just a dash of anthropomorphization (the creator lacks “intention” in the accepted sense), an omittance of the fact that there are millions upon millions of these AIs, and a little stretch of the definition of “artificial” intelligence. The AI is us, individually and culturally. Let’s make sure we don’t make the same mistakes biological evolution did.
ETA: The identity of the creator could be misinterpreted. I’m referring not to a god but to evolution (which perhaps can be said to “think” in the same sense that superintelligences might consider us to “think”).

October 19th, 2009
I’m not sure this is something that happens to many others, but recently I’d been feeling like my ethics were getting too purely abstract.
When we talk about what we want a superintelligence to do or what we want a future world full of uploads and self-modification to look like, it’s especially important to know what you want as explicitly as possible. It often takes hard work to think intelligently about what you really want and what you really consider right. Starting off with preferences that are incoherent and contradictory seems more common than not, and to strive for coherence you may have to prune or mold a few values, ever so carefully. If you really want to apply your ethics in such a complex world as the future may be, thinking hard and abstractly is the way to go. For myself though, sometimes all that abstract thinking makes me feel a little dry. It doesn’t help that I’ve been around people for some time who haven’t been feeling/displaying much that’s extreme. I know both great anguish and joy are being experienced right now somewhere in the world, but again, that’s more of an abstract thought.
I saw the film “Where The Wild Things Are” today, it was good. I’d heard it was supposed to be like being 9, and though I don’t remember exact ages that’s pretty much exactly what it was. It wasn’t a magical childhood journey or about how difficult it is to be a kid today, so much as it was just the experience of being a kid.
Kids seem to have greater highs and lows – I think I did – and/or display them more readily. Partly it was that, partly it was just the acting of the characters (impressive considering the monsters’ faces were CGI), but it brought to the forefront the reason for all this, and the reason we construct and use abstract theories at all. It’s not because we care about some huge fictitious happiness counter in the sky, or about a display of little numbers that appear after doing an expected value calculation.
It’s because feeling good is awesome, and suffering sucks balls.

August 29th, 2009
I finally did that numerical analysis on whether and by how much it makes sense to focus on combating existential risk, posted here.