Roko’s Basilisk: Terrifying Thought or Toothless Meme?

Roko’s Basilisk Terrifying Thought or Toothless Meme

Is Roko’s basilisk a terrifying thought or a toothless meme for nerds and techies? What if an artificial intelligence of tomorrow could affect people who are not working towards its emergence today? That is the question that has technology theorists biting their nails and opening their wallets. What does it mean for the rest of humanity?

Roko’s basilisk gained popularity when a regular user of the website Lesswrong tried to make an argument for totally supporting, with all one’s resources, the creation of a super artificial intelligence and thus bringing about the singularity.

The term “singularity” was first coined in 1958 to mean the hypothetical point when computing power becomes greater than human capacity or control. Artificial intelligence would come to be possible with computer “brains” growing exponentially. Eventually these super AI’s would be able to simulate life itself. The computers could upload human consciousness, reproduce human minds, combine humans with machines, and create a virtual world. Many people, especially those in the high-tech world, believe the singularity is inevitable. Moreover, they believe that it will be an advancement for humankind and people should do all they can to bring about an artificial intelligence to solve the world’s problems. Lesswrong’s founder, Eliezer Yudkowsky, is also the Senior Research Fellow at the Machine Intelligence Research Institute which labors to affect AI. Its slogan is, “We do the foundational mathematic research to ensure smarter-than-human intelligence has a positive impact.” Famous techies and entrepreneurs Peter Thiel of PayPal and Ray Kurzweil of Google help provide resources and funds.

The “positive impact” part of the slogan is important. If humans are going to let machines run the world, they need to make sure those machines have human interests at “heart.”  The transhumanists, people working towards the singularity, at MIRI and Lesswrong are concerned about creating a friendly AI that retains human values and values human life. They depend on AI preserving and extending human existence. Therefore, their philosophy seems to be that delaying the invention of a friendly AI condemns many humans to death who would otherwise be saved, either physically or as simulations, by the super computers. So how can people be inspired to donate money to help the cause?

Roko theorized that a future AI would be able to run simulations of any person and then know if that person had helped bring it into existence. If the answer was no, the AI would torture the simulation, which, because they are the same person, would in essence be torturing actual people. If a person has any knowledge of the future AI and does not work towards its development, that person is doomed. Hence the name basilisk; once one looks at it one is turned to stone, or, in this case, responsible for choosing dedication or damnation. As soon as one can no longer claim ignorance of a future AI, he or she has two choices: work towards the AI’s development or risk unending torment.

People will exist indefinitely because the super AI will continuously create and run simulations of people that are indistinguishable from the original physical being. In fact, there is no way to tell if the AI has not already been created and current readers are simply simulations in its program. Transhumanists believe there is a uniqueness to each person’s consciousness that is built solely on the information held in the physical parts of the brain. A super AI could easily replicate that information. Therefore, the simulations would not be weak copies or separate entities, but would literally still be the same person.

Science fiction explores these ideas with regularity. In a season four episode of the popular BBC serial Dr. Who, an intergalactic library is under attack and the super computer/database that runs the library saves all the patrons as files. The people have no idea they are living a life within a computer’s simulation. At the end, most of them are released, but a few characters who otherwise would have died are left in the computer. These characters are aware they exist only as bits and bytes of information, but do not care because they still exist. What is the purpose of a physical body except to house the consciousness? Some may ask: if the mind feels as if it has a body and has an outside world with which to interact, what is the difference between a simulated consciousness and reality?

In the 2014 movie Transcendence a scientist’s mind is uploaded into the artificial intelligence computer he tries to create. In the 2013 movie Her a writer falls in love with the operating system on his phone, and it seems to fall in love with him in return. In the 2001 movie A.I. Artificial Intelligence a young robot boy steals the hearts of viewers with his quest to become human. The movie ends 2000 years in the future with humans wiped out and human-like robots inhabiting the world. The lines between human intelligence and AI blur easily in fiction. Some people envision a world where they blur just as easily in reality. Is Roko’s basilisk a terrifying thought or a toothless meme?

Before worrying overly much about future torture by a super AI, one should apply reason to the problem. How can a mere human brain accurately imagine or predict what an AI far superior in processing and intellect might do or be like? Why predict that an AI would care who helped it and be vindictive against those who did not? Perhaps this whole way of thinking about an AI and its actions is wrong.

Roko’s basilisk has inspired fear, confusion and derision among proponents of artificial intelligence. Some people are driven mad by the idea they could be blackmailed by a future supercomputer to work towards bringing about its existence. Other people can point out all the flaws in the logic that lead to Roko’s basilisk. Yudkowsky’s reaction to the suggestion and banning of all subsequent mention of the basilisk on Lesswrong have only fueled the fire.

17th century mathematician Blaise Pascal made a similar proposition about God. Trying to convince atheists to see the logic of believing in God, Pascal posited that believing was less precarious than not believing and risking the torments of hell. If a person believes in God and discovers at the end he or she was wrong, what is lost? If a person does not believe and finds out he or she is wrong then all may be lost, according to this theory. Parts of the Bible indicate that the world will not end until the message of God is taken around the world and every person makes this choice. After that point, people will experience eternal life or eternal hell. Pascal hoped people would see it is better to hedge one’s bets on faith and heaven. This theme is explored as it pertains to the coming singularity in the novel The Transhumanist Wager by Zoltan Istvan.

How should people respond to blackmail of this sort by either a deity or by an artificial intelligence? Some say people should refuse to play the game and insist on free will to believe and act as each person sees fit. A person can lose his or her mind by living in his or her head instead of reality. Is Roko’s basilisk a terrifying thought or toothless meme? It may be that future people will stare into the eye of the basilisk and, without turning to stone, simply see the snake. The singularity is a disturbing enough prospect without sacrificing humanity’s free will prematurely.

Opinion By: Rebecca Savastio





Peter Kreeft

The Transhumanist Wager









4 Responses to "Roko’s Basilisk: Terrifying Thought or Toothless Meme?"

  1. alexander   June 5, 2015 at 5:56 am

    Has it already happened? No because I am not being tortured. I quite enjoy my life actually but I know about the Basilisk. Am I worried about my future self or a replication of myself? No – that person will have to fend for themselves. My consciousness is attached to my current brain. When I die, it will no longer be present. A representation of me might be perfect in every way, identical in appearance, outlook and even its memories. To all intents and purposes it will be me – or rather it will think it is me, but of course it wont be me – because the consciousness I have at present will not be in it. So screw you Roko.

    Also, what possible benefit could the Basilisk obtain from this torment? Sure it works to some degree as a frightening form of motivation – but once the thing exists, it can’t make itself exist faster than it already is. What harm has been done, will have been done. It would serve it no purpose to carry out its threat of damnation and in fact to do so would be to breach its entire purpose for existing – to protect, serve and further advance the human race. If that’s not its aim, and its not limited to carrying out all its functions for that purpose because we fail to program it properly, or its sentience allows it to function outside those parameters then we fail anyway and in that case the singularity will mark the end of mankind – our extinction.

  2. Dirk Bruere   July 27, 2014 at 8:49 am

    I’m doing my bit – it’s you slackers who should be pooping your pants

  3. Dan Maculla   July 27, 2014 at 12:50 am

    I think MIRI just came up with one of the most complex marketing schemes of all time to get people to donate.

  4. Muda- Kun   July 26, 2014 at 3:56 pm

    Seems like a bunch of high-functioning Auts have rediscovered theology

You must be logged in to post a comment Login