Our Final Invention: Artificial Intelligence and the End of the Human Era

to know why not to be slave of technology everyone must read. to ensure our ie. of human countinuance in the far far future humans must preserve universal love and be prepared to take on machines to show we are ultimate masters.these are the thoughts arose in my mind. A must read book of a kind.

I have been reading many books on artificial intelligence and its expected impact on humans and society. Few books are balanced; most veer between unquestioning optimism or dark pessimism. I am bemused and unsure of what will happen to humanity or society. James Barrat's book, "Our Final Invention," veers towards the pessimistic end of the spectrum, and there are moments when you – the reader – may feel waves of depression washing over you. Considering the increase in levels of inequality in all countries, I feel the digital divide between countries and within countries will increase. Moreover, I fear most people do not understand how AI has overtaken our lives. AI and its associated technologies have become an integral part of humanity, and most of us cannot imagine a life without these new technologies. James Barrat discusses how AI can go wrong: incorrect coding. This AI system becomes self-improving and self-generating, one we will not fathom as the technology approaches the 'singularity' as defined by Ray Kurzweil. On one side stand eternal optimists like Ray Kurzweil, who believe humans and super-intelligent machines will coexist harmoniously for centuries. On the other side stand pessimists who believe these super-intelligent machines will not need us. Will these machines develop ways to ensure we cannot switch them off? James Barrat believes in the possibility that this scenario will unfold. He's held conversations with most AI practitioners and has incorporated their varying views in the book, allowing him to present his perspective after taking everyone's perspective into account. This approach sets the book apart from others, as he reduces the temptation to live in an echo chamber. We should note that he was an optimist but is now a slight pessimist. The book reads well; he's provided many links for those keen to explore the subject further. The e-book version has links to the sources embedded in the text. Even though the book is – now – old, it is worth reading. You will get an alternative perspective compared to that of most optimists and business leaders.

An eye opener After waiting for almost 1 month for the book I m utterly happy to read it. Just gives a experience how wonderfully James Barratt had presented the senarior where humanity is walking towards. Really thrilling and where humanity is heading towards if we don't think of our next generation for humanity

More about the risk of AGI and ASI. I expected positive views about the new features and encourage to know more.

Seems bit overestimated but neva know any can end up ri8 core predictions seems like rational n helpful in predicting future

Must read book ! To imagine that AI could be man's biggest threat !

Too abstract.

Discovery of fire. Invention of agriculture. Printing press. Steam engine. Aviation and mass transport. Mechanization of production. Internet. If these are some of the milestones on the road of human civilization, Barrat believes that we may – well, still may – have reached the end of the road. As the wise has said, the day man’s creation becomes smarter than the creator that is the beginning of the end for Man. Is the day near? Barrat believes it is. Darkly, he predicts that, “Social media might turn out to be its (AI’s) incubator, its distribution system, or both.” Is it really the stuff of science fiction? Fear of the unknown that has haunted the human imagination since the dawn of history? May not be. When your Amazon account declares “…based on your browsing history, Amazon recommends …,” it may be affinity analysis algorithm at work, but more importantly it is a machine that is directing and influencing our choices. When Cambridge Analytica “steals” data of five million (i.e. 5,000,000!) Facebook users to influence national elections, this much is clear that we, as a race, have entered a new era. An era which countless generations of our ancestors had never seen. Is the march of machine unstoppable? It is. The only choice we have is to choose wisely. Machines will not be our slaves for long. But let make them our friends, not enemies. Because the consequences of the later will be horrible. But are we capable of choosing wisely? Given what we have reduced our planet to, the chances are pretty dim. Barrat’s book should serve as a call to action. An important book.

The author uses very convincing arguments. There is no doubt that we are in front of a very scary scenario.

Pour tout ceux à qui le terme SINGULARITÉ ou HUMANITÉ 2.0 dit quelque chose, la lecture de ce livre est indispensable pour mieux appréhender toutes les implications et conséquences qu'amènera l'émergence inéluctable d'ici quelques petites décennies d'une AI "Articial Intelligence" : l'auteur à juste titre exprime de façon très bien argumentée et documentée toutes ses craintes que cette AI ne soit pas aussi amicale et docile qu'on veut bien nous le faire croire si ceux qui la créeront ne prennent beaucoup plus de précautions qu'ils ne semblent vouloir en prendre actuellement dans sa création

I read this book in 2013 when it was first published. It is now near the end of 2024, 12 years later. Back in 2013, you rarely read about AI (artificial intelligence), AGI (artificial general intelligence) or ASI (artificial super intelligence); now, I see mention of them in the press and other media almost daily. Barrat's book attempts two things: (1) to convince the reader that artificial intelligence is here today and growing --- and its growth is accelerating, and (2) to argue that humanity MUST develop ways to instill AI with some type of morality or ethics, so that, even though its intelligence will surpass that of humanity, it will in some sense respect its creators and not turn on us. In the first effort, Barrat certainly succeeds --- the past 12 years have proved that. But, based on what I have been hearing and reading since ChatGPT hit the internet two years ago, except for a few voices crying out in the wilderness, humanity is making little if any progress on the second item --- perhaps that task is close to impossible? Barrat defines AGI as a level of intelligence roughly equal to that of human beings. He defines ASI as a level of intelligence greater than that. He then argues that AI will soon be able to both replicate itself and increase its intelligence --- and do so more and more rapidly. In 2024, I repeatedly read that AI will reach AGI within the next 3 to 5 years --- then, how long will it be before AGI learns to improve itself? Think of intelligence measured by points on a continuum (like a number line from high school math). AGI (modern day human-level intelligence) is a fixed point on that continuum. But at what point, either somewhat smaller than AGI or somewhat larger than AGI, will AI, of its own accord, begin to move to higher and higher points on the continuum (which is what Barrat means by AI improving itself)? We have no way of knowing, but Barrat argues convincingly that this phenomenon WILL occur, and most of the book is devoted to this argument. Digression: Our universe contains billions and billions of planets, and, I suspect, many with life, and, many of those with intelligent life. Won't a substantial number of them have gone through the AGI - ASI process? Is there no evidence of this that we can detect with our telescopes? In a universe populated with ASI's, why haven't we heard anything? Are we one of the first civilizations to develop artificial intelligence? Barrat doesn't open this Pandora's box, but I suspect he was tempted to (see pp. 90 - 92). To the curious reader: Look through the other 5-star reviews. Most of them bring up similar, valid points. Barrat has written an intelligent, highly readable book that is also, frankly, pretty alarming. And it is not dated at all --- it reads as though it was written yesterday. It is well worth reading now and in the foreseeable future. (added in May 2025): Much of what Barrat predicts is happening. Some things are occurring or about to occur that move beyond his predictions. The curious person might read "Situational Awareness" (by Leopold Aschenbrenner), AI 2027, or Ray Kurzweil's latest effort. Floating in space without a tether might be preferable to what is coming. Added Aug 10, 2025: With the recent release(s) of ChatGPT (up to version 5.0 now), AI can, by any reasonable measure, pass the Turing Test. Many folks regularly use ChatGPT, and it is truly stunning. Barrat mentions various individuals in OUR FINAL INVENTION, such as I.J. Good and Eliezer Yudkowsky, who have been deeply worried about AI evolving from AGI to ASI. Yudkowsky has written a new book, IF ANYONE BUILDS IT, EVERYONE DIES, that is due to come out next month. In some sense, it may serve as a sequel or extension to Barrat's book. ..... .....

大変衝撃的な内容で、今まで人類を滅ぼすものは、人類自体か、何らかの実態を持った生物だけだと考えてきましたが、この本を読むと、どうやら、それは間違いのようで、機械（人工知能）やウイルス等の情報もターミネータになりうるようである。 また、従来、とかく、感情的にならず、冷静に対応すれば紛争を回避可能と考えがちであるが、人工知能も地味魍魎が跳梁跋扈するネットジャングルに放り出されると感情的になることを考慮すると、感情的対応は、意外と、生き馬の目を抜き社会では、長期的視点に立った冷静な対応であると考えざるを得ないのである。 さらに、本書には、有害ソフトのほとんどが、紹興市周辺由来であること、米国とイスラエルがイランのウラン濃縮設備破壊のために送り込んだモンスター級のコンピュータウイルスが世界に拡散したことが紹介されており、その後が懸念されるところである。

"Our Final Invention: Artificial Intelligence and the End of the Human Era" by James Barrat is an excellent book offering essential warnings and explanations about AI.