Bostrom’s style can probably best be described as achingly clear – if a little dry – and level-headed to a fault. As Bostrom puts it: “Before the prospect of an intelligence explosion, we humans are like small children playing with a bomb … We have little idea when the detonation will occur, though if we hold the device to our ear we can hear a faint ticking sound.”īesides its rallying call, one of the most impressive features of Superintelligence is the absence of hyperbole.

He recommends we grapple now with how to develop “good” AI, how to spot a “bad” one, and urges the adoption of best practices by all AI researchers worldwide. Given that Bostrom considers the emergence of an SAI to be more or less inevitable, that means we had best not delay. And it can probably think it through more rapidly and formulate counter-countermeasures faster than we can respond.Īs such, we must act to resolve the issues of how to control an SAI well before such an entity springs into being. After all, if we can think of a countermeasure, so can the SAI.

Bostrom also thinks the various countermeasures we might use, such as stunting the SAI, or “boxing” it in by confining it to a walled garden remote from the rest of the world’s computers, are all unlikely to work. This could lead to what Bostrom calls the “treacherous turn” by an SAI, which is likely to take us by surprise, and happen far too late for us to do anything about it. Furthermore, it is likely to be better than we are at strategic thinking, and thus at getting what it wants. No matter what form an SAI might take – and Bostrom describes a few likely alternatives – and no matter what its values might be (even if its penchant happens to be to calculate pi out to a googolplex decimal points), it is likely to have interests that rub up against ours – such as a desire for resources or energy. This is because superintelligence is serious business. If we are greeted by a superintelligent artificial intelligence (SAI) within this century, as Oxford University philosopher Nick Bostrom suggests is highly likely, we may look upon this book either as an alarm that helped avert disaster or as a prescient insight into the makings of our doom. Indeed, perhaps even the most significant book of your lifetime. Superintelligence might be the most important book you’ll read this year. Superintelligence: Paths, Dangers, Strategies

Nick bostrom superintelligence review