Not fiction, this one, it’s Bostrom’s actual academic treatment of what happens if we build an artificial intelligence smarter than us, and I read it mostly because it kept coming up in other things and I wanted the primary source instead of the pop-science summary. The opening fable sets the whole tone: a flock of sparrows decides raising an owl chick would be useful, since a full-grown owl could guard the nest and do the heavy lifting, and only a couple of skeptical sparrows stop to ask how, exactly, anyone plans to control something that powerful once it’s grown. They keep working on the control problem. The flock, mostly, goes looking for an owl egg anyway.
The thesis is straightforward and genuinely unsettling once it sinks in: if we build a system capable of improving its own intelligence, the gap between “roughly human-level” and “so far beyond human comprehension that we’re the equivalent of ants to it” might close far faster than intuition suggests, and whatever goal we’ve coded in by that point is the goal we’re stuck with, because a system that smart won’t let us patch it afterward. Bostrom’s actual line, “we humans are like small children playing with a bomb,” gets quoted constantly for good reason, it’s the clearest sentence in a book that otherwise buries its best ideas under a lot of taxonomic scaffolding.
And that’s my real complaint, shared by a wide swath of the people who’ve read this: it is dry in a very specific, joyless way. Not difficult exactly, but exhausting, page after page of hypothetical-upon-hypothetical, oracle AIs versus genie AIs versus sovereign AIs, control methods stacked against control methods, until the book starts to feel like a policy handbook wearing a science fiction novel’s premise. One reader put it as reading like “a blank paint-by-numbers publication, where there are no numbers,” which is unkind but not wrong about the texture of it.
What kept me going anyway is the toddlers-with-detonators image near the end, a room full of small children each holding a button that could end everything, and the grim conclusion that someone in that room is eventually going to press it just to see what happens. That’s the book in miniature: a genuinely serious, well-reasoned argument that humanity is bad at exactly the kind of caution this problem requires, delivered in prose that makes you work far harder for the payoff than the ideas themselves demand. There’s a real strand of readers, and I don’t fully disagree, who think Bostrom’s confident that a superintelligence would be better-equipped to resolve deep philosophical questions than we are, without doing much to actually back that particular claim up.
Three stars, and it pains me a little to land there because the underlying argument is one I now can’t stop thinking about, especially the whole-brain-emulation chapter, which raises a question about suffering multiplied across parallel copies that the book itself doesn’t fully sit with. But I can’t in good conscience call a book fun that had me checking how many pages were left every twenty minutes. Read it if the subject actually keeps you up at night. Otherwise, get the argument secondhand and spend your patience elsewhere.