Yi Zeng & Dario Amodei — What Wisdom Requires of Architecture
Bring the two researchers into the same room. Both are trying to build AI that earns the trust of the civilization that deploys it. Both come to their work with a methodological inheritance from the natural sciences — Yi Zeng from cognitive neuroscience, Amodei from biophysics. Both have published extensively. Both run technical research programs that constitute, in their respective regions, the most serious institutional attempts to address what the safe development of frontier AI requires. Both are deeply serious people. Both are operating, on different sides of the Pacific, on the same underlying physical artifacts — the same transformer architectures, the same scaling laws, the same GPU clusters, the same global research literature.
And yet what they are building is different in a way that runs all the way down.
Yi Zeng's architecture must specify harmony and benevolence as engineering constraints. Wisdom must be a design target. The BrainCog program's spiking neural networks include modules for theory of mind, for social cognition, for the embodied self-modeling that, in Yi Zeng's framework, distinguishes systems that are merely capable from systems that have the architectural capacity for the kinds of moral perception wisdom requires. The work is grounded in neuroscience but oriented toward a substantive philosophical question. The technical specification flows from a prior commitment about what the system is supposed to be — what kind of being it is, in relation to what kinds of beings we are.
Amodei's architecture must specify interpretability as the load-bearing bet. Understanding must be the prerequisite for accountability. The Anthropic program's mechanistic interpretability work attempts to map the circuits, features, and reasoning pathways inside trained models so that the systems can be inspected before deployment. The Responsible Scaling Policy ties capability thresholds to safety measures the company commits to in advance. The work is grounded in biophysics but oriented toward a procedural philosophical question. The technical specification flows from a prior commitment about what the institutional response to capability advancement should be — how the procedural apparatus should be calibrated to evidence rather than to fear or to enthusiasm.
Where they converge: neither believes capability alone is sufficient. Both believe the technology requires substantive evaluation beyond benchmark performance. Both believe the institutional apparatus must be calibrated to the actual properties of the systems being built. Both have rejected the framings of their colleagues — Yi Zeng has rejected the framing that AI ethics is a soft overlay on hard engineering, Amodei has rejected the framing that "AGI" is a useful term — in favor of more rigorous alternatives that connect philosophical concerns to technical specifications.
Where they diverge: the substantive content of what each is trying to engineer.
Yi Zeng is designing for a relational civilization. The system must have the architectural capacity to model other agents as having states that matter. The system must have something like Mencian alarm — the immediate, architecturally embedded sensitivity to situations that call for care. The system must produce outcomes evaluated by their effects on the relational fabric of the society in which the system is deployed. The criteria are substantive. The criteria draw on a two-and-a-half-thousand-year-old philosophical tradition that has articulated, in extensive detail, what the relational fabric requires and what kinds of systems support or erode it.
Amodei is designing for procedural accountability to individuals and institutions. The system must be inspectable, so that individual users, individual regulators, and individual researchers can hold it accountable. The system must be deployable under conditions that have been specified in advance and that the public can verify the company has followed. The criteria are procedural. The criteria draw on the Western liberal tradition of procedural governance, the empirical methodology of the natural sciences, and the institutional design conventions of the American technology industry.
The convergence is real. Both researchers are addressing the same problem from opposite ends. Both have produced serious, sustained, technically sophisticated programs that, taken together, articulate the full shape of what frontier AI governance requires.
The divergence is deeper. Yi Zeng's program presupposes a substantive answer to the question of what AI is for that Amodei's program structurally brackets. Amodei's program produces procedural apparatus that Yi Zeng's program has not, to the same degree, developed. Each program needs what the other supplies. Neither program, on its own, is sufficient.
The honest articulation of this divergence is that the two researchers are not, in any straightforward sense, working on the same problem. They are working on adjacent problems, in adjacent traditions, on the same underlying technology, with adjacent methodologies that produce overlapping but non-identical conclusions. The conversation between them has not yet, in any sustained way, happened. The book that contains this chapter is one attempt to set the conversation up.
What would it look like if they did sit down together?
Begin with the place where they would, almost certainly, agree. Both would agree that capability advancement without safety measures is irresponsible. Both would agree that benchmark performance is insufficient as an evaluation of frontier systems. Both would agree that the institutional response to AI must be calibrated to the actual properties of the systems being built. Both would agree that the populations on whom the technology is being deployed deserve substantive consideration in the deployment decisions. These agreements are not trivial. They place both researchers in a small minority of the global AI ecosystem and identify them as natural interlocutors despite the different traditions they are working in.
Then the substantive disagreement. Yi Zeng would say: the procedural apparatus is necessary but not sufficient. Specifying capability thresholds does not tell you what the system is for. The Responsible Scaling Policy, however rigorous, does not address whether the system has the architectural capacity for ren. Interpretability tools that map circuits do not, by themselves, verify that the architecture is capable of harmony in the Confucian sense. The procedural apparatus needs the substantive specification of what is being engineered. The substantive specification needs the wisdom of a tradition that has been articulating substantive answers to the question of what AI is for. That tradition is available. It is the Confucian tradition. The American program would benefit from engaging it.
Amodei would say: the substantive specification is necessary but not, on its own, sufficient. Specifying ren and harmony as engineering constraints is the right philosophical aspiration. The operationalization is the hard problem. Without the procedural apparatus that allows the substantive aspirations to be tested, verified, and held accountable at the tempo capability advancement requires, the substantive specification becomes wisdom literature rather than working systems. The substantive specification needs the procedural apparatus that the Western institutional tradition has developed. That apparatus is available. The Chinese program would benefit from engaging it.
Both responses are correct. Both reveal the partial completeness of the position the responder is responding to. The combination — the substantive depth of the Confucian frame plus the procedural sophistication of the Western frame — is what the situation requires. Neither tradition, alone, has produced the combination.
What this would require, in practice, is a sustained institutional effort to bring the two traditions into substantive conversation. Not the polite multicultural consultation that produces unobjectionable joint statements. Not the comparative philosophy seminar that produces academic publications. Something more demanding: a collaborative research program that takes seriously the methodological commitments of both traditions and attempts to build technical specifications that incorporate both. The substantive criteria from the Confucian frame. The procedural apparatus from the Western frame. Both encoded in working systems that can be evaluated against both kinds of criteria simultaneously.
This program does not currently exist. There are gestures toward it — the Berggruen Institute's work, the Beijing Academy of AI's international collaborations, various track-two diplomatic dialogues between Chinese and Western AI researchers. None of these has yet produced the depth of substantive technical collaboration the situation requires. The geopolitical tension between the United States and China has made collaboration harder, not easier, over the past several years. The export controls, the technology transfer restrictions, the national security framing of AI development — all of these have raised the institutional cost of substantive collaboration.
This is the great unmet opportunity of the AI moment. The two researchers who are perhaps best positioned to produce the substantive-plus-procedural synthesis the technology requires are operating in geopolitically separated ecosystems that make sustained collaboration structurally difficult. The institutional response that would address the synthesis problem is the institutional response the geopolitical conditions are actively preventing.
What this book can do, since the institutional response is currently impossible, is articulate the substantive content of the synthesis as clearly as possible from the outside. The substance of the synthesis would include: engineering constraints that specify the architectural capacity for relational responsiveness (the Yi Zeng contribution) operationalized through procedural apparatus that allows the constraints to be verified, audited, and held accountable at scale (the Amodei contribution). The substantive content would draw on Chinese philosophical tradition for the answer to what is being engineered. The procedural apparatus would draw on Western institutional tradition for the answer to how the substantive specification gets tested.
This is, in summary, the synthesis the head-to-head between Yi Zeng and Amodei reveals to be necessary. The synthesis is not, currently, being attempted at the depth required. The conditions that would make the synthesis possible — sustained substantive collaboration between Chinese and American AI ethics research — are not currently present. The geopolitical environment has, if anything, deteriorated since 2022.
[YOU] on AI asks what is worth amplifying. Yi Zeng's answer, on his own articulation: the wisdom that AI requires of architecture. Amodei's answer: the institutional capacity that the compressed twenty-first century requires of human society. The combination — wisdom encoded in architecture, capacity built into institutions, both calibrated to the substantive criteria the situation requires and operationalized through the procedural apparatus capability advancement demands — is the synthesis neither researcher, alone, has produced.
The conversation has not yet happened. The book is the attempt to put both halves of the conversation on the same page, so the conversation can begin elsewhere. Whether it begins is a question for institutional actors outside the writing room. What this chapter has done is make audible what the two voices would say if they were brought into direct conversation. The audibility is the prerequisite. The conversation is the work the chapter cannot finish.