The Precipice: Existential Risk and the Future of Humanity

Philosophy is, generally, considered a very dry subject, and one only fit for academia with people arguing about very fine details of history, ethics and morals. Our young people rarely learn about the consequences of making minor changes to personal, group or country wide thoughts. Increasingly, as adults, we don't have the time to 'think' a lot about the consequences of our life styles, our political decisions or what we allow our elected representatives to do in our names for - and to us. Toby Ord is an extraordinary scientist, and philosopher; he is also a relatively young person who has, on our behalf, done an extraordinary amount of work and research to make complex ideas accessible to most of us, regardless of age, education or culture. Increasingly his analysis is being sought by governments, corporations and Think Tanks pondering the consequences facing Humanity, Planet Earth, Communities everywhere, Families and each of us, as 8 billion people and corporations introduce technology quicker and quicker - all while our climate changes are happening with unpredictable aggregate effect. The Precipice is a 400+ resource that is accessible, readable and compelling; it's a book that each of us can read slowly and it has useful ideas and information that may help all of us, and our children, lead more peaceful and productive lives. On the subject of the AI technology, and how it may move us, help us or control us, he has this to say (p. 146-147): For the most damaging people in history have not been the strongest. Hitler, Stalin and Genghis Khan achieved absolute control over large parts of the world using words to convince millions of people to win the requisite physical contests. So long as an AI system can entice or coerce people to do its physical bidding, it wouldn't need robots at all......The most realistic scenarios may involve subtle and non-human behaviors which we can neither predict, nor truly grasp. And these behaviors may be aimed at weak points in our civilization to which we are currently blind... The AI section is interestingly shedding much needed light on how complex commercial systems already are bending entire countries to their will by controlling and monitoring our media, purchase habits and cell phone use. Mr. Ord has shown us that philosophy may need to be re-introduced into our schools and Book Clubs, as a matter of survival of the species.

Should appeal to fans of big-picture books like 'Sapiens,' 'Enlightenment Now,' or 'Guns, Germs, and Steel.' Somehow the book is simultaneously beautifully written, intellectually rigorous, and deeply researched, and is both "the one book to read" on its topic (existential risk) and filled with lots of novel content, even for someone like me who has been following this research area closely for many years. It also manages to be hopeful and inspiring despite its grim subject matter. The book makes compelling arguments for several important conclusions, including: - Mitigating existential risk should be a global priority according to many different value systems. - Fortunately, natural risks such as asteroids are extremely unlikely to be existential in the course of a century. - Anthropogenic risks such as climate change and nuclear weapons are much more worrying, but their odds of being a truly existential catastrophe in the next century probably vary by orders of magnitude. - Some of the most valuable work for reducing existential risk is probably not directed at a specific existential risk, but at various "risk factors" such as great powers war, which may not be existential risks themselves but increase the danger coming from other sources of existential risk. I've been waiting for this book for years, and it does not disappoint. In fact, I've read it front to back 3 times already. Highly recommended. (Full disclosure: I work for a funder of Toby's work but I work on other areas and haven't been involved in those grants.)

Existential risk… It is not a popular subject. I suspect that most of us don’t factor it in thinking of our own lives. Mired in day-to-day bustling, we live as if tomorrow is always to come. We rarely contemplate the impact and importance of our actions, let alone the dynamics of our societies. Yet, Toby Ord invites us to take a perspective even further from the mundane - a perspective of the whole humanity itself, in a timescale stretching to infinity. Through such lenses, the author argues, it becomes evident that our species is risking an unacceptable chance of irrecoverable collapse or even extinction. Those catastrophic outcomes are symbolised by the eponymous precipice, which humanity must avoid on its path toward a better future. Toby believes that our times are special, because only recently (in historic terms) humans possessed technologies which allow us to destroy ourselves. The increase of power is not met with heightened wisdom - we are still narrow-minded creatures, risking nuclear war over national squabbles, racing in unregulated biotechnology and AI research, polluting our planet in the name of next quarter’s profits. It is not moral to jeopardize the lives of untold future generations, just because we are too short-sighted to notice that we owe them the chance. Recklessness may prevent us from realising the true potential of humanity, “probably the only complex and significant part of the universe”. If we manage to pull through, future historians will look back at our period as the pivotal age of humanity embracing the long-term, wise course. This book hopes to lay foundation for such a change, by identifying existential risks and proposing methods to tackle them. It seems that the highest risk is man-made, so there is an expectation that countries with better aligned governments should spell unimpeded progress and prosperity to all. I understand why this book was written and what it tries to accomplish. I admire the research and fact-checking. More than a few times I have ruminated over the book’s passages. However, I am afraid that the overall tone is not very persuasive. It lacks compelling stories which could inspire future efforts, leans heavily toward statistics, sometimes reading like a project proposal for management of a risk portfolio. Nevertheless, it is worth a try, as it offers unusual perspective and intriguing insights. My last two cents - in (tongue in cheek) defense of short-termism. Toby Ord urges to coordinate in order to increase the chance of humanity fulfilling its destiny. He defends it as moral. But, wouldn’t it be an even more astounding feat if our civilisation thrived without such efforts? We are a product of “let’s promote what works now” evolution, and our societies emerged from collective actions of usually self-centered individuals. Perhaps it is our way, and the beauty comes from our flaws. Do we really have to optimise and max out?

Since the particles of infinity have memory and retain experience, and of the potential of doing the right thing through love, it’s possible to simultaneously bind without capturing and support without limiting. Therefore, infinite potential has a constant, which is love, that can ignite a quality and a quantity of these particles.

This is a very hands-on approach to the doomsday scenarios. It discusses our moral obligation to keep humanity thriving and actually goes over the probability of these rhings happening.

A great read! Insightful and interesting!

Los tiempos actuales deben entenderse y solo ver noticias no ayuda. Este libro nos hace entender los tiempos que vivimos y hacia dónde vamos.

人類の存続に関わる existential risk について著者なりの筋に沿って整理してあるが、話が拡散しすぎていもる。その基本構造は概ね次のようになる。すなわち、人類の存在価値は計り知れないがその真価を顕すまで存続できるとは限らない。存続を脅かすリスクには自然に起因するものと人類自身に起因するものがあるが、今後数世紀の間は後者のリスクが高くこれを乗り越えられなければ存続はおぼつかない。一方、前者については人類が長期にわたり存続できれば必ずや解決策を見出すであろう。よって、今が正念場（precipice）というわけだ。 前半は論理的に分かりやすく異論はないが、気候変動では滅亡にまでは至らないとしてAIの暴走を当面の最大の危機と見積もるのは本書の特徴かもしれない。全体的に Steven Pinker 的な楽観論とMax Tegmark的な時空の広がりがあるように思う。違う言い方をすればカント的な人間理性への信頼か？ また、世代間倫理的な観点が強く出ている。人類の極めて長期の存続は文句なしに善でありかつ可能であるとされ、ここ数百年の危機を乗り切ったならば人類が銀河のスケールで広がっていくこともありえる (原文 canやmayではなくmight )という。逆に文明の進歩を止めればやがて来る破局的自然災害（隕石 etc）などで滅ぶのでそれはできない。著者は科学や技術を否定して「自然に帰れ」などと言ってるのではない。 ここまでくると、科学ではなく哲学とも言い難くある種の「思想」と思った方がいいが、物理法則で禁じられないことは人間が愚かにも自滅しなければ実現の可能性はゼロではない、というスタンスのようだ。ただ、銀河の話はともかく、人類の（比較的）長期の存続という観点が当たり前になるくらい人間が進化してもいいと評者も考えるし、具体的なことを考えるきっかけとして一度は遥か未来、銀河の果てまで思いを馳せてみるのも悪くない。