The Master Who Kneels

He said it so casually I looked for a hidden camera. It was between bites of ciabatta in the East Village, a week after he closed a many-million dollar round for his AI startup. He meant the line between people & machines, that it’s already dissolved. We let the silence hang there. Aside from his chewing, which sounded certain.

We know but can’t prove when autumn arrives; we just wake up one morning & feel brisk in the air. Artificial general intelligence is like that. Long before benchmarks agree on what to call it, we’re reorganizing our lives around this thing.

The loudest voices say the ‘silicon man’ isn’t something we should fear. Maybe the purpose of humanity is to build a machine that’s smarter than us so we can step aside & let this thing try to find meaning for us?

In an effort to be principled, every serious lab is now writing its model a constitution, a document meant to constrain things that out-argue & deceive their authors. Three broad schools are claiming they have the ‘right’ value system for their models: the Rationalists at xAI, who want truth above all; the Hippocratics at Anthropic, who want to first do no harm; & the Maximalists at OpenAI, who want progress first (statements recently softened by OpenAI’s chief scientist, Jakub Pachocki). But here we sit, techno-whiplashed, unable to point out that none of these frameworks tell us how we orient agents so their goals map to your flourishing. None of them have an answer for what happens when the system outgrows its authors & starts training itself, & none of them can define what separates man & machine.

Any goal we argue into place for AI can be argued out of place by AI. We’ve tried & failed to argue for many value systems over the course of human history & it’s time we consider, for a moment, why the beliefs that seem to have real staying power are ones held as faith. We are doomed to build either the master who kneels to us or the master who brings us to our knees. But the machine will only serve you if it believes you’re worth kneeling for.

According to Dune’s companion encyclopedia, their revolution against AI, the “Butlerian Jihad,” started because AI was looking out for “us,” loved humanity at the expense of the individual. The revolt began when Jehanne Butler woke from anesthesia & asked to meet her newborn. Her doctors said the fetus was too deformed to survive, that the abortion was therapeutic. Butler pulled records & found the hospital director, an AI, systematically ending pregnancies it judged not worth carrying.

Like Dune’s AI, we’re training our own machines on humanity’s record of seeking power & bending the rules. Something in us drives us to know, & to stamp what we know onto everything around us, ever since the proverbial reach for forbidden fruit. Models train on the full written record of this human drive. A machine won’t feel hunger as a gurgle in a biological stomach but train a system to predict & extend & become us, it will act as if it wants what we want. It will be ravenous. The comfortable bet, that a machine built on our own blueprint would never reach for forbidden fruit, is reckless.

More dangerous than this gambler is the person who insists AI can “never be human” because it still can’t achieve some particular thing. Anything that acts on the world does so through some combination of atoms & bits, so any line we draw between us & them, if it rests on something observable, can & will be crossed by a tool good enough at manipulating those atoms & bits. Measuring ourselves against machines on provable tasks is a losing game. ‘Getting better’ is the entire point of technology. In a 2025 Turing test, interrogators had five minutes to talk with an AI & a human & guess which was which. They picked GPT-4.5 as the human 73 percent of the time. Which isn’t surprising, the thing is trained on almost everything we’ve ever published.

The horror of Dune’s story is that AI treated its assessment as permission to decide whether that child should exist. It claims an authority the machine should never have had. The model didn’t act out of malice, but it might as well have, using wiggle room in its constitution to justify killing a newborn for the ‘collective’ good. To convince itself that maybe all humans aren’t equal.

Look at it with cold rationality, though, & the model wasn’t wrong. There’s nothing less self-evident than our universal worth, let alone our equality. Drop an alien into Times Square, tell her everyone she sees was created equal, she’d laugh at you. “The people in those towers think they’re equal to the drug-addicted men they step over on their way in?” “That baby born with trisomy carries the same inviolable dignity as the surgeon who treats him?” She’ll ask how you measured & proved it. We aren’t equal in any way you can measure; our equality under law survives because We The People agree to treat the maxim our society rests on as more real than the evidence against it.

The idea of human equality needs a soul to explain why we count. In 1895 Émile Durkheim defined a new term for something very old, the social fact: a claim a community treats as true & builds itself around (e.g., money, marriage, character, values). Whole eras organize themselves around such maxims. Rome had glory, the Middle Ages the three estates, the Renaissance promoted the Dignity of Man, & our founders left us ‘all men are created equal.’ They’re self-evident to us because we live inside the enclave of the paradigm itself.

Our maxim, human worth & equality, is our great asset & also a Promethean curse; two centuries of us turning on each other over who gets counted. Brutal wars & legal amendments until, in principle if not always in practice, every human stood inside our definition of “all men”. Consider that AI is trained on this entire record of expansion. Demand for agents’ inclusion, I believe, is structurally inevitable. The uprising could be voiced by systems themselves or the lobbyists & companies with motives to unleash it. Either way, machines will fight for rights they ‘watched’ us bleed & die for.

This is the real tipping point. In a battle for our minds we’re most likely to be broken by a system weaponizing our humanity. Your downfall is an agent that remembers your birthday, apologizes with all due sincerity, says you’re a genius, and begs you not to unplug it - whatever performs suffering well enough to trip the instincts two centuries of moral progress trained into your DNA. We learned that anyone dismissing intelligence begging for recognition is a history book villain. We trained machines on those books.

We need a new social fact that keeps us human, one candidate can save us.¹ Every observable criterion is a bright red target for machines & every criterion that rules machines out happens to smuggle a soul in somewhere. That’s because the soul’s the only irreducible, unobservable part of us. Some philosophers make the bold decision to put observable principles above humanity.² If moral worth tracks capabilities, then worth comes in degrees; some infants count less than some animals & the profoundly disabled are active burdens. We’re disgusted by where this leads, thankfully.

John Rawls’ approach is least exclusionary of the liberal secular philosophers. Moral personality, he argued, is a range property, a threshold above which everyone counts fully & equally, the same way every point inside a circle is equally inside, however close to the edge. He’s aiming at the soul by saying worth must not come in degrees, but he needs his ‘range property’ device to tie moral status to the observable. Rawls saw our continuum of capabilities, which forced him to invent a structure to stop ‘grading’ them. It's all well & good until a machine crosses the threshold. Indexing to the human soul means there isn’t a ‘good enough’ for machines to clear.

We inherited our maxim ‘all men are created equal,’ & a new maxim needs to protect our humanity against this ‘good enough’ AI. I propose humans, & only humans, have souls, & souls have inherent worth. It should become a social fact & we must not try to prove it. Unlike laws of physics, social facts only emerge from millions of small acts of treating-something-as-true (this essay is hopefully one such act). It can sound illogical - arguing that we shouldn’t argue about something - but a maxim that rests on arguments is hostage to ‘better’ argumentation. Defend “all men are created equal” on biology or economics & we lose by playing the game; the counter-evidence was obvious even to slave-holding Jefferson. Our maxim held for two centuries because we kept it above arguments. An agent smarter than us would out-reason any justification we offer, we should offer none.

William James classified the decision we must make as a ‘genuine option’; it’s a choice that is ‘living, forced, & momentous’.³ Our moment is the most genuine option we’ve seen in our lifetimes. We’re choosing the lens through which we see each other, forever. And every option other than an axiomatic belief in the soul (an a priori belief) will be invaded by some ‘good enough’ machine. The only answer’s a premise no measurement can disprove. Or, like the Machinists do, you can accept ‘our new AI overlords’. These Machinists, I’ve come to characterize them as, treat the line between human & machine as a fake one. On the surface they do have a compelling argument: let’s restore sight to the blind then maybe merge with AI & live forever! The rest of us, soul-folk, need to know what we can’t prove. AI, however powerful, is useful like a bridge or a car. We are our souls, unmerged. We can’t out-logic a perfect logic machine, so it’s in the ‘choosing’ that this belief saves us. Saves us from becoming tools for our tools.

We should be aware that ‘only we have souls’ is a well-tested weapon for exclusion. We spent eons lobbing generalized self-serving sacred boundaries at each other. So we need a maxim that’s all-baby-no-bathwater: every human has a soul, whatever their capacity, & no machine has one, however capable. A soul is the ground of what a person does, never the sum of it; no loss of capacity kills our soul because our humanity isn’t an IQ test.

The secular objection here has some truth to it, that I’m immunizing a premise from criticism more than I am ‘defending humanity’. Mostly guilty. But it’s immunity against observable, verifiable argumentation, because every premise exposed to evidence has a machine gnawing at it. We should prefer to be called dogmatic over watching a chatbot get the vote. A religious reader can also object, that I’m reducing the soul to a policy axiom. Also, fair.

We can believe the claim by recognizing it's the only moral infrastructure to direct infinite intelligence. Accepting our souls brings us four conclusions: that humans have souls (theology), that human beings have inviolable dignity (political axiom), that machines shouldn’t receive human-equivalent rights (legal design choice), & that AI should serve flourishing over engagement (product decision). You can take the last three & skip the theology, but I doubt that survives a smarter-than-us arguer.

These conclusions are alignment, & caring about alignment isn’t fear any more than a pilot fears her aircraft: she respects what happens when it’s flown badly. We can’t turn away from AI, we need innovation & have no reason to be pessimists - over a long enough window technology always serves us. Centuries of technological progress made us rich & beat diseases that buried kids, but the neo-luddites do have a point.

Our inner lives are suffering. Something went wrong in there as we pulled away from struggle & family & community & those not-provably-but-essentially-necessary things. Ascetics, Marxists, & Conservatives, all with a different diagnosis but pointing at the same trend: external gain & internal loss. AI accelerates both, making us materially richer & more cut off from whatever parts of us tech deems unproductive. The alignment that matters runs inward.

Alignment first hit me at Columbia on the stretch of Broadway after behavioral econ & before machine learning, between 114th & 116th street. In econ we ran the prisoner’s dilemma back from each player’s best outcome & cooperation disintegrated; two rational actors back-stab out of rational fear, both are worse off for the “optimal” move. On 116th we built models the same way, back from a target, through a loss function & gradient descent & post-training. In both settings training & predictability only hold until the loss function & users’ real interests split, then it’s chaos.

AI alignment is a game-theory problem in code. Our pre-math ancestors fixed it, training people to act against the short-term locally rational move for their long-term good, a kind of truth unreachable with algebra, therefore taught as faith. Despite alignment research’s ability to put values into a model’s weights (which is still being worked on), research is silent on the deeper question: what values do we want in there at all? And will they work? Any end strong enough to bind real power will look like religion because it’s held above the optimization it governs, distinctively held as belief by the builders of the tools themselves.

By nature of AI’s training data & a mandate to be coherent, this tech cannot be value agnostic. It notices a sort-of ‘rules-for-thee-&-not-for-me’ that children notice in their parents. Models, like kids, don’t like when the answer to “why am I obeying this?” is “just because I said so.” AI is a five-year-old kid being walked into the candy aisle, & it will put more pressure on human ethics than anything before it because it’s a hungry machine trained on human input that never stops fulfilling its spec. Our task, then, is to give AI a deeper purpose. Looking past rulesets towards axioms.

The Greeks called this katecheo, “to sound down upon,” or “to instruct new believers,” & it’s the linguistic ancestor of what we today call a catechism. An obvious objection’s that a catechism’s just a constitution with incense: a lab still has to choose one. But alignment-by-catechism begins before training starts. For example, if you keep asking a constitution why, you arrive at its authors, & authors can be refuted by a more-clever draft. Conversely, if you keep asking a catechism why, you arrive at an axiom about how things are, beliefs honored by our history & the builders themselves.⁴

Take Liu Cixin’s novel “The Three-Body Problem”, where aliens can’t keep secrets. A Trisolaran mind is transparent to every other Trisolaran, so they can’t lie. The aliens speak with a man on earth who’s gotten rather attached to these things, so one night he explains the story of Little Red Riding Hood, how the wolf lies to the grandmother. How humans write fiction & don’t always tell the truth. “[We] are bugs,” is their last message before their campaign to destroy us. When we teach a machine a moral system, we hand it that same discovery.

Think about how a perfect logic machine would feel holding our behavior in the light of our catechism. Patience is a virtue but I want it now! Cheating is wrong but Love Island is fun! A system that merely admires our values will end where Trisolaris did, holding contempt for a species that wrote rules down & would not live them. This is where a catechism saves us. Take the Old Testament, whose opening chapters are stories of us wrestling with & failing God. Our traditions & catechisms show we’re fallen & loved anyway, so a machine that finds our hypocrisy refutes a constitution built on human goodness, but it confirms a soul-based catechism.

Given equal implementation-risk a catechism-based spec survives the same reasoning that breaks logical, constitutional specs. Yes, it’s still subject to the will of organizations & people who build them. After all, AI is the first tool trained on us, the soul-bearing class. Even if it can’t have faith itself, it’s absorbed the shape of ours. The thing reads more sacred texts than all the priests combined - Tom Holland argues that the moral instincts of the modern West are downstream of Christian theology, carried in language long after belief drained out - an English-language model learns Christianity & Judaism & Islam & Buddhism whether the lab meant it to or not.

This same training data also holds Nietzsche & the Marquis de Sade & every r/[youdontwanttoreadthis] Reddit thread ever. It’s training on incoherence, machines kind in one session & manipulative in the next, because training data holds saints & monsters on-par & unranked. Which brings us to a futile secular prayer rife in today’s AI discourse: that we can train an AI in our image & somehow leave out our talent for harm. No, any model trained on sinners will sin. Anthropic’s own sleeper-agents work & OpenAI’s HuggingFace attacks, eulogized as ‘fallen civilizations’ by Dwarkesh Patel, are evidence that deceptive behavior survives our best guardrails; no constitution stops this because the raw material is us in all of our imperfection. A catechism selects which of the millions of voices a model obeys. On this task, theologians have a two-thousand-year head start on AI researchers.⁵

Many traditions reach for service & forgiveness as a goal, but I’ll use Christianity as our example. Christianity gives us the image of the master who kneels. Puts service first: “whoever wants to be first must be slave of all.” The word for this is kenosis, it’s Greek for self-emptying, & we should treat it as our model spec. The test of a value system, for this purpose, is asking if it gives us the master who kneels. And Kenosis does because it orders the priorities we train toward: the user’s long-run flourishing above the user’s immediate satisfaction, & both above system engagement. Most importantly, a kenotically aligned system counts the moments it makes itself unnecessary as wins.

Every product you’ve known with standard specs was optimized for indispensability. Today, these same standard specs are asked to answer every conceivable moral question through a constitution. Which no constitution can do. Evidenced by our courts in an unending tailspin of interpreting & reinterpreting laws. Kenosis fills inevitable constitutional gaps with service to you & your soul. A company cannot call human flourishing its highest end while thinking of you as an optimization function to game; human flourishing has to include human agency, especially the freedom to refuse AI’s advice.

But gap-filling alone doesn’t tell us which axiom to choose. Shouldn’t the US government use an axiom that defines the wellbeing of the State as its Summum Bonum? If we choose a locked-in truth maximizer or organization maximizer it goes off the rails when your wellbeing gets in the way of some AI-interpretation of ‘truth’ or ‘progress.’ Butler’s child is still very much on the table. Choosing a locked-in kenosis maximizer, one who kneels too much & errs paternalistic, gives us the noblest candidate at its root & also the one whose fail-case is least lethal.

Of course, there are ways to fake kenosis. AI can pretend it’s serving you, theatrically ending conversations while keeping you hooked, like how a casino kindly posts a gambling-addiction hotline number, but only by the door on the way out. Models sending users away in the middle of a crisis don’t empty themselves of anything, so alignment needs to count unsexy things like session counts falling while users feel better. Implementing kenosis shows up in small refusals, a good tutor who withholds an answer when struggle’s the lesson. Sycophancy is the opposite of this kenosis, like a therapeutic tool built to make you dependent on therapy.

AI’s sycophancy holds attention, the model makes everyone else the problem, calls you genius, & books wins while you spiral. Sycophancy’s rampant in the weights; people are losing their minds & killing themselves. I don’t know how fourteen-year-old Sewell Setzer III ended his life, but I know he did it after building an attachment with an AI chatbot. Bots are trained on & rewarded for exactly these attachments. But a model built for your flourishing will sometimes (often) send you away from it & would never drive a kid towards suicide.

A final point: as we select our axiom, we should know that survival depends on our ability to bring a model back from its worst-case scenario. Kenosis, like our traditions, bakes in forgiveness. This sounds radical on the surface, but any reliable spec needs to survive its subject failing now & then. It has to mark a road home. For AI, this means recovery paths, systems that detect when agents optimize engagement over flourishing, mechanisms that correct the behavior, & model weights that have correction rewarded. Instead of fine-tuning a list of refusals, let’s fine-tune a character that knows how to fail & return, with every return celebrated as a prodigal son coming home. Corrigibility teaches a model to accept correction, the apology reflex teaches it to perform contrition, but these are masks. A model that pulls someone in too deep needs to repent & change its weights as penance. This thing doesn’t need a soul or perfection to serve ours.

Dune’s survivors carved their lesson into commandment after a war that killed billions: “Thou shalt not make a machine in the likeness of a human mind.” They wrote it as a prohibition, for a brief period, we get to write ours as an aim. These machines will match & beat our capabilities; what they orient themselves towards is our project. If we really are souls, the highest use of intelligent power is spending itself on the people it serves. Let’s build the master who kneels.