Foreword

The question that organized [YOU] on AI was personal. It asked what becomes of you, the specific human being, when the most powerful amplifier ever built is pointed at whatever signal you happen to be feeding into the world. Are you worth amplifying? It was a question to be asked at the kitchen table, at the desk, at the moment before you opened the laptop and began the conversation with the machine that had learned to speak your language.

This book asks the same question one scale up.

What becomes of a civilization when the amplifier is pointed at it? When the technology is the same — the same transformer architectures, the same scaling laws, the same compute substrates produced by the same handful of companies — but the civilization on the other end of the amplification is not? When one civilization asks of the new technology what kind of people it will make us, and another asks what the technology is actually doing? When the first phrases its safety problem as the cultivation of wisdom and the second phrases it as the inspection of weights? When both believe, with the certainty available only inside a fishbowl, that their version of the question is the universal one?

I went looking for the answer in ten minds. Five Chinese: Yi Zeng, who is trying to write benevolence into the architecture of spiking neural networks at the Chinese Academy of Sciences. Bing Song, who has spent the past five years arguing that liberal AI is a philosophical position pretending to be a universal default. Zheng Yongnian, who reads the Chinese state not as a deviant version of a liberal state but as a different kind of institution altogether. Kai-Fu Lee, who came home from a Beijing hospital bed in 2013 with a lymphoma diagnosis and the conviction that human beings are for love, in the technical sense that love resists optimization. Robin Li, who is building artificial intelligence in Chinese for the 1.4 billion specific people Baidu has spent two decades learning how to serve.

And five American: Dario Amodei, who replaced the word "AGI" with the phrase "powerful AI" because the philosophical term could not be operationalized fast enough to govern. Elon Musk, who said in 2014 that we were summoning the demon and then, eleven months later, co-founded OpenAI — because the demon was coming whether or not he was in the room, and the room was where the decisions were being made. Sam Altman, who deliberately echoed the 1954 promise of nuclear power too cheap to meter when he began promising intelligence too cheap to meter, because the prior promise had failed not from physics but from politics, and the politics this time were being constructed more deliberately. Mark Zuckerberg, whose open-source bet is the most American of the five American moves: distribute the weights, let three billion individuals each get their own personalized assistant, refuse to let any single entity own the infrastructure of intelligence. Jensen Huang, who is selling sovereign AI to every nation on earth, simultaneously, in the calm voice of a man who has noticed that supply allocation is the most powerful form of soft power and has nothing in particular to gain from naming it that.

I read these ten thinkers in their own words across thousands of pages of essay, interview, testimony, and book. I read each of them the way the orange pill asks you to read anyone — looking for the fishbowl that produced the argument, not just for the argument. And I discovered something I did not expect.

The ten of them are not giving different answers to the same question. They are asking, of the same technology, different questions.

The Chinese ask: what kind of civilization do you want to be, and what AI would serve it? The Americans ask: what is the technology actually doing, who controls it, who benefits, and how do we steer the variance? These are not different shadings of the same conversation. They are two different conversations, conducted in the same room, about the same machine, in voices that have not yet learned to hear each other. Each side regularly believes the other is missing something obvious. Each side is right. Each side is missing what the other side is supplying.

This book is my attempt to make both fishbowls visible from outside. To hand the reader two orange pills — one for each bowl — and to invite the reader to swallow them in sequence and to live, for the length of this book, with binocular vision. The river the Americans are swimming in. The river the Chinese are swimming in. The bridge between them, where the future of this technology will either be built or be missed.

I am Edo Segal. I am still rolling down the same river I described in the first book. So is the technology. So, increasingly, is the planet.

Take the stairs. Climb the bridge. The view from the middle is the only view this century is going to permit.

— Edo^Opus