Posts Tagged ‘Existential Risk’

Metropolis

Tuesday, January 26th, 2010

metropolis-de-osamu-tezuka

This is a really great film, one of my favorites. It’s based loosely on the German classic, using a comic by the creator of Astroboy (after he died and couldn’t stop them), and written by the creator of Akira. The basic plotline is that of two detectives foreign to the city, investigating a rogue scientist. They meet up with his creation as political turmoil erupts in the city. It’s generally fun to watch throughout and the ending is fantastic. In particular relevance to this blog, it displays both the folly of anthropomorphizing AIs, and the existential disaster they can cause. It’s not like a lesswronger wrote it, but in comparison to most such films it’s excellent.

Don’t Just Sit There Caring

Friday, January 22nd, 2010

For my senior honor’s thesis I’m researching the implicit assumptions underlying the use of generally misguided statistics in genetic studies. Some time ago I read through the recent book ”How Doctor’s Think”. It didn’t turn out to be very useful for my project, but it was written well and interesting. There’s a few patient stories, one detailing a misdiagnosis of an adopted Vietnamese infant thought to have an immune deficiency. A section at the end of it bothered me:

“Shira received her morning feeding through the tube, and then Rachel went to the end of the hospital corridor to a pay phone. She called one of her closest friends from her congregation and told her the news.
“It’s so wonderful,” her friend exclaimed. But then there was a long silence.
Rachel wondered what was wrong.
“Turn on your TV.”
Rachel stood frozen in the room and felt as if her heart, so full of joy, were being torn. At the moment she celebrated Shira’s restored life, thousands were likely dead in the attack on the World Trade Center. How can I rejoice when God’s creatures are dying?

Forty-five days after Rachel and Shira went to the Children’s Hospital ER, mother and daughter left for home. It was Friday, just hours before the onset of the Sabbath. When Rachel turned the key and entered her apartment in Brookline, she could smell the meal left by friends. Two candles stood ready to be lit, two fresh challahs ready to be savored. Rachel held Shira after lighting the candles. The soft glow of the flames played off her daughter’s face. It was the day of rest and of peace, the day when all woes were meant to cease, the day that Rachel had not truly had for more than six weeks.
At each step, Rachel had not been sure whether she would find the strength she needed to endure, and the courage to question. Silently, she again thanked God for creating all human beings with such remarkable reservoirs of resilience. She thought how the Sabbath was the time when these reservoirs were refilled. She prayed that during this first Sabbath after 9/11 her country would find the strength and courage to defend itself and to care, with a full heart, for the families who had lost loved ones.”

As most readers know, not that many people really died in September 11th. A little under 3,000, which is the world death toll from all causes every half hour (h/t to Vladimir for the correction) .So don’t just sit there caring about some tragedy that’s already occurred, work to stop the tragedies that are occurring constantly all over our planet, and the extremely large disasters that havea very good chance of happening, in this century. An existential disaster would be equivalent to more than two million 9/11 events, in terms of human death. If you feel sorrow for those we havelost, use that to save all those we will lose.

In my research on existential disasters in general, I also read “Never Saw It Coming.” This book was one of the most foolish books I’ve ever read. Karen Cerulo takes the observation that there is a cultural asymmetry in focusing more on the good than the bad, and then applies this perspective far more widely than appropriate. She often ignores contradictory evidence and any concern for the actual objectives at hand; I get the impression she had an alright idea and just really wanted it to be a great idea. It’s a kind of comic example of confirmation bias. There are a few tidbits and points that are worthwhile, but almost nothing applicable to existential risk.

There was a good quotation in there though, which relates to the tendency of people to apply a lot more effort to mourning than to saving lives.

People who are in decision-making positions are not mentally preconditioned to think in terms of what happened. So that’s what I mean about a failure of imagination. The evidence comes in, but your mental reactions are not geared to thinking in these kinds of terms. When a guy calls from a flight school and says they could take a 747 with fuel and plow it into a building and that’s a bomb, you hear it but you say, “Ah, that’s a wacko idea.” You don’t say, “Holy Jesus, that’s what we’ve got to worry about.”…[You] fail to imagine what the danger is. [You] fail to understand the world we live in and the nature of the enemy.”   -Journalist Hedrick Smith

My Most Motivating Thought

Tuesday, January 12th, 2010

I’ve built up a strong habit of daydreaming, something that I’m working to reduce. I often find myself daydreaming about extremely unlikely situations. Sometimes they’re unlikely but common on a larger scale, for example being burglarized while you’re in the house. Sometimes they are patently ridiculous, such as being caught with a loved one in a sheep stampede (do they even do that?) while holding on to an old wooden fence which is slowly falling over, or resting inside a pile of sharp metal shavings while being attacked by someone with a sledgehammer. Sometimes they’re at least common in movies, along the lines of Jason Bourne or James Bond.

In these daydreams, the general train of thought is “How could I survive that?”, often paired with the objective of protecting friends and family. Being burglarized my chances are good, if I don’t try to be a hero. In an action movie situation my chances are moderate, if I already have the skills of a hero. For the ridiculous ones, it varies wildly. Underlying all this though, are movie expectations. In a movie the hero goes along, is hit with varied obstacles, and manages to pull through by being the best and keeping their head. But is this really how the world tends to work?

I’m increasingly aware that it is not, even if you have a lot of time to plan out your response under conditions of minimal stress. For one thing, your underlying proficiencies, automatic responses and reflexes may not do what you want. Perhaps it’s a good idea to fire a gun, but how many times have you fired a gun before? You’re highly unlikely to be a good shot at first. Maybe a fancy maneuver would be nice, but have you trained as an acrobat, gymnast, or martial artist? If not, pulling it off even in a safe environment is a dubious proposition. And even if you’ve got all those skills, who’s to say that’s enough? If you’re on an open field, walking along unsuspecting while a sniper has you in the sights of a high powered rifle, there is no stunning victory, no champaign room. You just die. In war, not every one who is smart and skilled survives, luck plays a huge factor. Nobody is good at surviving a mortar. In nature, if something like a cold snap or simple misfortune has resulted in malnourishment, and some predator that’s faster than you got close enough without you detecting it, it’s over already. “Nature” would say ’sorry’ if it cared, which it doesn’t.

The real world has no proclivity to providing you only with obstacles you can survive, even in a hypothetical sense. The real world plays by no rules but the rules of physics, and those are as brutal and unforgiving as landing on the solid granite rocks they make up. If we reach the point at which we’re able to create artificial general intelligence and haven’t put in the work to understand how to make it friendly, too bad. It’s lights out, with no excuses and no second chances.

The lesson of all this is to start acting now. If you care about the survival of yourself and your loved ones, if you have any desire to see 80 or 8,000, then don’t wait until the situation has grown impossible to break out your best self! The world yet allows you free time, so use it! Do all you can to ensure that we avoid the “no-possible-win” scenarios, and give ourselves at least a moderate chance of surviving the ones we can’t avoid. If you go to take the final for a class in a new subject and you’ve never studied, you’re going to fail, no ifs, ands, or buts. The test is coming, so get studying now and change those odds!

(Note: it is possible to take this too far. From my own experience, I’ve found that I require social contact to stay productive, and I might also require some occasions to kick back and stop optimizing my time for a few hours. In addition, fun is important long term, and a highly productive presingularity life can be a fun one. But be honest with yourself, which in this sort of affair is usually a nontrivial task. If you’re anywhere close to average, it’s very unlikely that you’re already optimizing your productivity, or the density of your fun and relaxation. I’m still working at it myself. )

Some additional quick reasons to reduce existential risk

Wednesday, December 30th, 2009

Steven Kaas at Black Belt Bayesian posted a great list of “sound-bite” reasons to reduce existential risk. Check it out, they’re both amusing and relevant.

2010 Singularity Research Challenge

Tuesday, December 29th, 2009

For the readers I don’t share, check out this short piece by Michael Anissimov on the Singularity Institute, their work, and a recap of why they’re a great place to donate.

War Is Over

Friday, December 25th, 2009

I have really positive associations with Christmas, and as an atheist but a cultural Christian I still really enjoy it. I self-perceived most of my youth as a fight against boredom, and with little income, Christmas really helped out. It was also one of the only 3 or so times a year I got to see my “fun” grandparents.

These days I have the money to buy the things I really want, so the gifts aren’t such a big thing anymore, but it’s still a lot of fun. Listening to a song that came on (posted below), a different kind of fantasy struck me. Being focused on existential risk throughout the year, I have it occasionally. There’s a lot of beauty that’s possible, and enjoyments that are worlds beyond what we can imagine, but sometimes, I fantasize just about safety. I think about a world where we’ve done it, succeeded, with no more existential risk, no more involuntary death, no more significant suffering. The enhancements, the superhuman happiness, the vastly greater intelligence, the radically enhanced communication, I can wait for. We could take our time, the future shining and unblemished by danger or fear, each of us knowing we’d be around as long as we want to be, able to take ourselves as far as we want, as fast as we want. We’d be able to relax so fully, enjoying each other’s company as we approach a welcoming horizon.

Sarah McLachlan singing John Lennon’s “Happy Christmas (War is Over)” You might want to listen without watching the video, which while nice, dissipates and changes the tone of the song a little.

P.S. If you’re wondering what I’m talking about in technical terms, I’m referring to the rise of  a benevolent singleton, protecting our present and future while allowing the self-determination of personal choice. Something we might get if SIAI or their allies eventually succeed.

Back On The Horse

Friday, December 18th, 2009

Two months ago I experienced some burnout and had to scale back my efforts at existential risk reduction. I was optimizing every single bit of my time towards that end, and for a possible variety of reasons I couldn’t sustain it. That was acceptable, as long as I knew I couldn’t do it. When it comes to saving billions of lives – as well as both our future and our past – I can’t accept applying anything other than my best effort. If that meant taking some more time for relaxation, so it was.

Even so, this went again my grain in a little. We are entering the single most pivotal time in the history of our civilization, species, and Earth, and as long as I can do something about it, it seems a little silly to sit around playing video games. In addition I’ve never been good at balancing really entertaining games and work, beyond getting homework in on time. Months ago I made a policy against playing any new video games, and I came back to decide that again; these days I’m most satisfied if I keep myself grounded in this world. In addition, any sort of role model I have would be fighting tooth and nail to protect the beauty of our present and the potential of our future. I could accept doing less if I had to, but it felt very off.

For about a month now I’ve been back at spending more and more time and effort on “saving the world”. I’m happier because of it, and I think I’ve got things figured out this time, I’m a little surprised I didn’t see it before. When I was going through that burnout, all the things I most wanted to do were social things. As I pointed out even then, I was doing fine until I got into a situation where my optimizations didn’t take  me on their own into proximity with others. Now I’m back to optimizing all my time towards existential risk reduction, with the one exception of occasional social events. I think I actually had tried this earlier, but by that point I was already coming down and didn’t have the time to see it work.

What’s more, I’m now an aspiring rationalist, and despite some fair assumptions by others, I did not really consider myself one before. A few readers may not fully realize what I’m talking about, which is kind of a pity. I’d suggest reading LessWrong, but even doing that I didn’t “get it” for months. Reading some good sci-fi also helped. For about a year I had been working at being maximally productive with my actions, but only recently am I working to be maximally productive with my thoughts. This is where the very significant gains lie.

If there ever was a time for us to become heroes, it is now, and it’s satisfying for me to be more directly back on that path.

You Said You Wanted Hugs

Sunday, November 29th, 2009

PBF115-Hug_Bot

PBF115-Hug_Bot2

Why Friendly AI research is critical: even if you give an AI nice sounding directives, it can be hard to know how far such an alien mind will take something.  We take for granted all the other beliefs going around in our heads, such as that a hug shouldn’t be that strong, partly because we aren’t powerful enough to take things that far. The often discussed situation is that of just directing an AI to make people happy. What counts as a person? What counts as happy? What are acceptable ways to make people happy? You don’t want the AI to disassemble you into a hundred smaller “humans” and make them happy, or worse yet a bunch of microscopic pictures of happy people. You also don’t want it to put everyone into a drugged stupor. Designing a superintelligence is analogous to having a wish granting genie, but one of those annoying literal types for which almost every wish is a very bad thing.

“Accelerando” Review

Saturday, November 21st, 2009

(This review contains only negligible spoilers).

Accelerando(1stEd)

I just finished reading Accelerando by Charles Stross. It’s a masterpiece of hard science fiction, and I highly recommend it.

Vernor Vinge coined the term Singularity when he observed that no author could write realistically about smarter-than-human intelligences. Literature allows you to realize characters who are stronger or more outgoing, but the intelligence of their plans is limited by your ability to think of them. Superintelligences must be kept off screen or in some infant stage. Considering the likelihood of superhuman intelligence in our relatively near future, this makes writing really hard sci-fi a difficult endeavor. Vinge blurbs on the back cover of the book “There is an intrinsic unknowability about the technological singularity. Most writers leave it safely offstage or invent reasons why it doesn’t happen”, which applies to his own work as well. In comparison Stross’s Accelerando dives in headfirst, with mainstream (post)human civilization becoming essentially incomprehensible by the middle of the book.

Of course any real superintelligences still have to be kept off screen, and so the story follows characters who for one reason or another have been left behind by the tsunami of increasing intelligence. This creates the interesting effect that despite inhabitating lives stranger (and more probable) than those in the vast majority of sci-fi, the characters’s situations manage to feel very backwater. It’s as if you were following a family of Amish throughout the industrial and information revolutions, but more significant.

At a few points I grew dissapointed as it seemed Stross’s literary ambitions may have overcame him, with superintelligences mysteriously on our level, but by the end it all makes fair sense through one route or the other. I would have enjoyed those sections more if I knew that to begin with. There were also a small handful of differences between my own best estimates and the world of Accelerando. The Singularity is a tad on the slow side, and there always remain a number of independent, comparably powerful entities. This is probably my largest contention (it seems more likely for a single superintelligence to achieve dominance at some point), but there might not be much of a story if this were otherwise. There’s also no significant mention of emotional engineering, ala David Pearce’s Hedonistic Imperative or Nick Bostrom’s Letter From Utopia. A more sociological than technological point, but people are pretty nonchalant about creating and deleting copies of themselves. I care more about expectation of experience than identity, but as a preferential utilitarian I can get along just fine with those who think otherwise, as long as they don’t force that choice on others.

When I first heard about this book, the take-away message was the great density of concepts. The book is packed with advanced technological proposals, internet subculture references, unusual vocabulary, economics, neuroscience, history…it goes on. However Accelerando is much more readable than this would suggest, and most of the references are tangential, perfect understanding not required. The few times a concept is critical he takes a moment to explain it, and those interested can hunt down referenes on the net (try doing that 10 or 15 years ago). I’m admittedly not bleeding-edge (cutting maybe?) on speculative technology, but to the limits of my knowledge all the ideas are presented in a sober, best-guess fashion. To load the book with so many ideas takes quite an intellect or a great deal of work, and most likely both. Stross took 5 years to write this and has an impressive background, with degrees in pharmacy and computer science, and those who’ve known (biblical sense) WoW might be interested to learn he came up with death knights, back in the day.

The best and favorite thing I can say about this book is that it is mind boggling. A common criticism of sci-fi is that it takes one idea and places it in an otherwise changed world. Accelerando is just the opposite, which includes just about every feasible proposal and then mixes them in with additional ideas about their interaction. This allows for very unique and interesting turns of plot, and the book tends to put the reader in a constant state of future shock, continuing for 400 pages, even while in the relative backwater I mentioned above. The density of information and references adds to this effect nicely. The human mind suffers from the conjunction fallacy, and we’re more likely to put belief in speculation that is more specific, such as the setting of a book. Despite this I think Accelerando is excellent for improving our sense of the future, by reminding us that the future isn’t going to be one or a few new ideas, it’s going to be a great number of them, all interacting and creating ever newer ones. There are three meanings to the Singularity, one of which is that without intelligence enhancement, really understanding the world is something you’d have to entrust to others. It’s one thing to read about that kind of future but another to catch a glimpse of it, and that’s something that Accelerando provides.

Vinge also calls Accelerando “the most sustained and unflinching look into radical optimism I’ve seen”. While our own future could be much better, this really is a pretty optimistic book, which I like as I’m generally an optimist myself. It also presents some very possible dangers and threats. The future could be better than we’re physically capable of imagining, but there are thousands of ways it could go badly and it’d be worth it (understatement) to prevent those outcomes. The future may be incomprehensible, but for now we’re still in control, and still the most intelligent life on this planet. Let’s make the most of that, because it’s not going to last.

I don’t want to leave the stage now

Tuesday, November 17th, 2009

Birds, Emiliana Torrini

Let’s stay awake and listen to the dark.
Before the birds, before they all wake up.
It’s the ending of a play and soon begins another.
Hear the leaves applaud the wind.

See the sun come rising and white-wings start to fly.
Like strings of pearls in the firey sky.
I don’t want to close my eyes, don’t want to leave the stage now,
As the leaves applaud our stay.

Lend me yours wings and teach me how to fly.
Show me when it rains, the place you go to hide.
And the curtains draw again and bow – another day ends.
The leaves applaud the wind.