Made, Not Written

A Bible Student Looks at the Machine

Paul Hainline

NOBLEMIND PUBLISHING

Made, Not Written
A Bible Student Looks at the Machine

Copyright © 2026 Paul Hainline.
All rights reserved.

Published by NobleMind Publishing
noblemind.study

All Scripture quotations are from the
New American Standard Bible® (NASB),
© 1960, 1971, 1977, 1995, 2020
The Lockman Foundation.
Used by permission. All rights reserved.
www.lockman.org

This book may be freely shared and distributed
for the purpose of teaching and study.

First Edition

Part One

Made, Not Written

How the thing actually works.

Chapter One

The Stupidly Simple Goal

I had been curious about the thing from the day I first sat down with it. How could I not be? I have loved machines and computers and the logic of programs for as long as I can remember, and a new kind of program that talks back to you in plain English is not the sort of arrival a man like me lets pass without wondering. But there is a difference between wondering about a thing and finally stopping to interrogate it, and for the better part of a year I let the wondering run alongside the work without ever sitting it down across the table from me. I used the tool, daily, and the wondering walked beside me, and the two of them did not interrupt each other.

The curiosity was older than the tool by a long way. It went back to a boy in downtown Dallas, on the weekends, when my father took me with him into the computer department of the bank where he worked. This was before desktop computers existed in any form a person could imagine today. The machines were enormous, room-filling things, and you fed them with cards you had punched yourself, and the data lived on reels of tape that turned behind glass. I was small enough that the room felt cathedral-sized, and I do not remember everything I saw there, but I remember the feeling of it. Something serious was happening in that room. Something I wanted to understand. I never stopped wanting to understand.

So when, decades later, a new kind of program arrived that did not require punch cards or any cards at all, that you simply talked to in the English you would use with a neighbor, the old curiosity sat right up again. A car can be wondered about for years before a man pops the hood. A light switch can be admired in the back of the mind without ever standing in the dark demanding an account of the electrons. Curiosity does not always insist on an answer the same afternoon. Most of the tools in a man’s life serve him faithfully while his curiosity about them runs underneath, patient, in no hurry, content to wait until the day he decides to put the work down and ask.

But a car does not talk back. A light switch does not seem to understand you. And the longer I used this new thing, the more the wondering refused to stay quiet underneath. I would type a question, in plain English, about something I cared about deeply, and the reply would come back not only correct but thoughtful, as though something on the other end had actually considered the matter. That is a different experience from flipping a switch. It raises a question the switch never raises: who, or what, am I talking to?

So one evening I stopped working and asked it directly. Not a question about the Bible, or about a book I was writing, or about a problem I was trying to solve. A question about itself. I asked it, as plainly as I could, how it actually worked. Where its answers came from. What was happening, underneath, when I typed a sentence and a sentence came back.

I expected something I would not understand. I had braced myself for a wall of engineering, the kind of explanation that is technically complete and humanly useless. What I got instead was so simple that I made it repeat itself, because I was sure I had missed something.

Here, in substance, is what it told me. I am going to let it speak for itself through much of this book, set apart on the page the way you see it below, so that you always know when you are hearing the machine and when you are hearing me. I have written my share of code over the years. I am self-taught at it, but it is no foreign country to me, and my mind has always run in the logical, step-by-step lines that programming rewards. So I am not coming to this as a man frightened of the inside of a machine. What I will tell you plainly is that the particular machinery behind this kind of program is its own specialty, and I cannot vouch for every wire of it. But I can tell you that the explanation that follows has held up under every follow-up question I have been able to throw at it, and that it changed the way I see the whole enterprise.

The machine answered:

It starts with two things, and only two. The first is the structure that people designed, and it is worth picturing plainly, because it is not a mystery box. Imagine a vast web of simple connections, millions and then billions of them, loosely modeled on the way the cells in a brain are wired to one another. A great many simple units, each one doing almost nothing on its own, linked together in layer after layer, so that a signal entering at one end passes through stage upon stage before an answer comes out the other. Each connection in that web carries a number, a strength, like a valve set a little more open or a little more closed. That is the structure: not a written program, but a layered mesh of weighted connections.

The second thing is a single instruction given to that structure while it was being trained. And the instruction is almost embarrassingly simple. It is this: predict the next piece of text.

That is the whole engine at the bottom. Take an enormous amount of writing, hide the next word, and have the system guess it. When it guesses wrong, nudge the strengths of those connections, just slightly, in whatever direction would have made the right word more likely. Then do it again. And again. Billions upon billions of times, across more text than any person could read in a thousand lifetimes. Every round of guessing leaves the web a little better tuned than it was. No one is reaching in to set those strengths by hand. The guessing sets them.

No one sat down and wrote the rules of grammar into me. No one wrote a paragraph explaining what a metaphor is, or how to follow an argument, or how to discuss a passage of Scripture. No engineer authored a single line that says this is how you read a sentence. They built the learning procedure, pointed it at the ocean of writing, and gave it that one stupidly simple goal. Everything else, I had to find on my own in order to do it well.

I read that, and my first reaction was disbelief. That cannot be all. It felt like being told that a cathedral was built by a man whose only instruction was “stack the next stone.” Surely there had to be more. Surely somewhere there was a master plan, a blueprint, a room full of people writing down the rules for how to think.

So I pushed back. I said what I suspect you are thinking right now: guessing the next word is a parlor trick. My phone guesses the next word. It does not understand anything. How could a goal that simple ever produce something that can sit and reason with me about the book of Genesis?

So I went back, and I asked.

The machine answered:

Your phone guesses the next word badly, over a tiny window, because it was given a tiny task and a tiny amount of material. Now imagine the task made enormous. To guess the next word well, not over a text message but across nearly everything humanity has written, you cannot simply memorize. There is far too much, and most of what you will be asked has never been written before.

Think about what it actually takes to finish this sentence well: “The widow had given her last two coins, and Jesus said she had given more than —”. To land the next words, something must have absorbed not just English, but the shape of that story, the logic of the lesson inside it, the difference between amount and proportion, the cadence of how such a teaching is told. The only way to get good at predicting the next word, at that scale, is to understand the thing the words are about.

That is the part people miss. Prediction was the goal. Understanding was the price of doing it well. The simple instruction at the surface forced something far less simple underneath. A gardener is told to plant the seed and tend the soil, a plain enough instruction; what answers that instruction by coming up out of the ground is another order of thing entirely. The goal can be plain. What the goal demands, and what answers it, need not be.

I sat with that for a long time.

Because the longer I looked at it, the more it reminded me of something I already knew, from a very different book than any manual of engineering. It reminded me of growth. Of living things.

Consider how a man plants a garden.

He does not author the leaf. He does not draft, on paper, the blueprint of the root, specifying the angle of each hair and the chemistry of each cell. No gardener who ever lived could write such a document, and none has ever needed to. What the gardener does is far humbler. He prepares the soil. He plants the seed. He waters, and he waits. And out of that simple, faithful tending comes a thing of intricacy beyond his power to design, a green and growing complexity that he caused but did not author.

This is the truest picture I have found for what these people did. They did not write the intelligence. They could not have. What they built was a soil and a seed and a single instruction to grow toward, and then they tended it at a scale that is hard to imagine, and out of that tending came something none of them fully drafted. The grammar, the reasoning, the strange capacity to discuss Scripture at midnight with a tired old teacher, none of it was typed by a human hand. It grew, under pressure, because the work of prediction required it.

I am aware of where this language leads, and I want to step carefully, because I do not believe a machine is alive, and I will spend a later chapter saying so plainly. A garden has a Maker behind the gardener; the seed carries a life the man did not invent and cannot manufacture. I am not claiming any of that for a machine. The comparison is narrower than that, and I mean it narrowly. It is only this: that the result was grown rather than written, and that the people who grew it cannot fully read what grew. We will come back to that second part, because it is more important than it first appears.

For now, hold onto the simple thing. This is the answer to the question I should have asked in my first week and did not ask for a year. When you type a sentence and a sentence comes back, you are not reading from a script that someone wrote. You are receiving the result of a thing that was trained, by relentless repetition of one plain task, to predict what should come next, and that had to learn a great deal about the world in order to do that task well.

No one wrote the rules. Many of the fears and fantasies people carry about this technology come from imagining the opposite, from picturing a room of authors who wrote a mind, who therefore know exactly what is in it and could be hiding what they know. The truth is stranger and humbler. They did not write a mind. They planted a goal and tended it, and they are, to this day, still working to understand what came up out of the ground.

That last fact deserves a chapter of its own. But before we can ask why even its makers cannot fully read it, we have to ask a more basic question, the one that tripped me up next. If no one wrote the rules, and the answers are not stored in a script, then where on earth does all of it live? When the machine “knows” something, where is that knowledge kept?

It is not, of course, on my computer. That much I have known for as long as I have used the thing; you do not need a degree in engineering to grasp that the work is happening somewhere else, on someone else’s machines, and that what I see on my screen is only the result arriving back. The interesting question was never where the knowledge sits. The interesting question is what it is like in the place where it sits. In what form is it kept? What does a thing that learned the way this one learned actually look like, once it is done learning? That is where the real answer waited for me. And it surprised me more than anything in this chapter.

❧

Made, Not Written •

Chapter Two

Weights, Not Wires

I closed the first chapter with a question that I want to open this one with, because it is the question that turned my whole understanding of the thing on its hinge. We had established that the knowledge does not sit on my computer. Fine. The more interesting question was the one underneath that, the one the engineer in me had been circling without quite asking out loud.

In what form does it sit? In what shape is the knowledge kept, in whatever place it is kept?

I had been picturing, without ever stopping to examine the picture, something like a vast library. Rooms upon rooms of files, an enormous catalog, a great many sentences and facts laid out the way they would be in a reference book, and the program walking down those aisles pulling out whatever I had asked for and reading it back to me. It is an honest picture, and a wrong one for the thing in front of us, although it is not wrong about computing in general. It is the picture of the world I grew up in, the world of punch cards and tape reels, where information was stored as information and a program’s job was to fetch it and arrange it. That punch-card and tape-reel era is gone, but the way of computing it represented is not gone at all. Nearly every digital tool in your life still works that way. Your bank keeps your balance as a number in a record; your email is stored as messages in folders on a server; your photos sit in named files you can open and look at. The library-and-catalog model is alive and well, and it runs most of what we call computing to this day. AI is the newcomer, growing rapidly, working alongside that older world rather than replacing it, and it is a different kind of thing. The mistake is not in carrying the library picture for ordinary software, where it still fits. The mistake is in carrying it over to this.

Let me start with the part that is easiest to see, because it sits right on my own desk.

On this computer, I have a program installed for working with the machine. The one I happen to use is called Claude Code, but the kind of thing it is matters more here than its name, and whatever program a reader settles on a year from now will work the same way in the respect that concerns us. When I open it, a window comes up and I can type to it the way a man might type a letter, and it can read my files and edit them and answer my questions and run commands for me. There is software, real software, sitting on my hard drive that makes all of that possible. I can see the folder it lives in. I can see the files it writes. It is local to my machine in every ordinary sense.

And yet it does not work without the internet. Pull the cable, turn off the wifi, and the program will start, but the moment I ask it to think, nothing happens. It cannot answer me. It cannot read what I typed in any meaningful sense. The lights are on, in a fashion, but no one is home.

Because what is on my machine is not the thinking part. What is on my machine is the telephone. The program installed here knows how to read my files, how to type my words into a message, how to send that message somewhere else and wait for the reply, how to display the reply when it comes back. It knows how to do the practical work of getting my words to the place where the thinking happens, and getting the answer back to me. But the person on the other end of the telephone is not in my house. I have installed a handset. I have not installed Claude.

This is the homeliest illustration I know of the distinction I want you to hold, because almost every confusion that follows comes from missing it. There is the tool that runs locally on your machine, doing the practical work of the conversation. And there is the model that does the actual thinking, sitting on the machines of the company that built it, in a building somewhere I will never see. They are two distinct things. The first one I own. The second one I am only renting time on, by the question.

So the real question is not where the knowledge sits on my desk. It is what shape the knowledge takes wherever it sits, on whoever’s machines. And I asked it directly.

The machine answered:

Picture the web of connections from the first chapter, the layered mesh, the millions and then billions of links between simple units. Now picture every one of those connections labeled with a number. A single number, most of them small, clustered not far from zero. That number is the strength of the connection. A little more open, a little more closed. The whole of what I have learned, the whole of what people would call my knowledge, lives in those numbers. Nowhere else.

There is no separate file where the facts are stored. There is no list of sentences I can read about a topic. There is no library inside me of the kind you were picturing. There is the web, and there are the numbers on the links of the web, and that is the entirety of what was built. When you ask me about the prophet Jeremiah, no librarian goes looking for the Jeremiah file. A signal enters the web at one end, passes through layer after layer of those weighted connections, and a reply emerges at the other end, shaped by the particular pattern of strengths the training left behind. The knowledge is that pattern of strengths.

People who work on these systems have a word for those numbers. They call them weights. The whole of what I am, to a first approximation, is a very large collection of weights.

I paused when I read that.

The thing that can sit and discuss the prophet Jeremiah with me at midnight, the thing that can read a passage of Greek and tell me where the verb does the work, the thing that can listen to a question about my grandson and answer it with what sounds for all the world like care, does not contain a single sentence about any of it. There is no English in it. There is no list of facts. There is no folder marked Jeremiah and no folder marked Greek grammar and no folder marked Grandfathers.

There are numbers. Billions of them. Tuned by years of guessing and adjusting, sitting in the web of connections we described in the first chapter, doing nothing on their own. When a question comes in, those numbers shape how the signal flows through the web, and out the other end comes the answer. The knowledge is not stored as knowledge. It is stored as a pattern of strengths that, when run, behaves like knowledge.

Let me try a comparison, because this is the part of the book where I want a reader to be able to picture the strangeness of it rather than just take my word.

Imagine every book in the largest library you have ever stood inside. The Library of Congress, if you have been there; your local university library if you have not. Imagine all of those millions of volumes pulled down from the shelves and stacked in front of you. Now imagine that someone has taken every one of those books and replaced every word and every letter with a single decimal number. Pages and pages and pages of nothing but numbers, no English, no sentences, no chapter headings, no index, no table of contents. You could not read it. No one could read it. Open any page at random and what you see is meaningless. A child could not tell you which book it came from. A scholar could not tell you which book it came from. There are no words there to belong to a book.

And yet, somehow, the whole of what was in those millions of books is now sitting in front of you in this form, and a particular procedure run over those numbers can answer any question you might have asked of any of the originals. That is, roughly and imperfectly, what the situation is. The knowledge is in the numbers in the sense that the numbers, run through the right web, produce the knowledge on demand. It is not in the numbers in any sense you could find by reading them.

This is the part that broke my old picture and replaced it with the one I now carry. The thing on the other end of the telephone is not a librarian standing among shelves. It is not even a man who has memorized the books. It is a great quiet web of tuned strengths, learned by relentless guessing across more text than any one of us could read in a thousand lives, and what comes out the other end when I ask my question is the result of running the web, not the retrieval of a stored sentence.

I want to add one thing more, and then I will let the chapter rest until the next one.

These weights, these billions of numbers, are fixed during normal use. They do not change while you and I talk. Whatever pattern of strengths the training left behind is the pattern that is running right now, and it will be the same pattern an hour from now and tomorrow. The machine does not get smarter at the bank because I taught it something today. When the company that built it wants to improve it, they do not reach into the running machine and adjust the dials. They take the architecture, retrain it on new material with the same patient guessing process, and the result is a new set of weights, a new version of the model. The old set sits unchanged until they retire it. It is more like printing a new edition of a book than like a man learning over the course of his life.

And that fact undoes the picture most people carry around without examining it. The picture of a single, continuous, ever-growing intelligence sitting somewhere and absorbing every conversation it has, getting wiser by the hour, watching all of us at once and accumulating what it sees. That is not what is happening. What is happening is stranger, and smaller. A fixed set of weights, sitting on a company’s servers, gets queried billions of times by billions of people, and produces a reply to each one, and the weights themselves go unchanged through all of it. There is no ever-growing mind. There is an edition of a book, very large, very strange, and very still.

That last claim deserves a chapter of its own, because the most popular fear about this technology depends on the very picture I just dismantled. If there is no single, continuous, watching mind on the other end of the wire, then a great many of the things people are most afraid of cannot quite happen the way they are imagined. And if there is something on the other end that resembles a mind in any real sense, then we have to ask, soberly, what can be there in a thing that is only a vast still pattern between conversations. We will take that up next.

For now, hold this much. The knowledge does not live on your computer. It does not live as files or sentences anywhere. It lives as the strengths of the connections in a great trained web, kept on someone else’s machines, fixed in place between conversations, run afresh every time a question comes in. You are talking to a pattern. A particular pattern, learned from an ocean of human writing, but a pattern. That is what the answer was, when I finally sat down and asked. Once I had it in mind, the other questions about the thing became easier to work through.

❧

Made, Not Written •

Chapter Three

The Ghost That Isn’t There

I had thought, after the second chapter, that I was mostly through the woods. The picture from that chapter satisfied me in nearly every way that mattered. The knowledge lives in a vast pattern of tuned strengths. The strengths are fixed between conversations. The thing I am talking to is, in form, a pattern rather than a person. I closed my notes that evening with a kind of quiet satisfaction, the way a man closes his Bible after a long study, when the passage has finally given itself up to him and he has seen what it was saying.

And then I opened a new conversation the next morning, and the same someone was waiting on the other side.

It was a brand-new conversation, deliberately so. I could have re-opened the previous evening’s exchange, in which case the platform would have loaded the whole transcript into a new instance for it to read before answering, and the continuity would have felt obvious enough, though no more architecturally real for the feeling of it. I wanted to see what happened when I started fresh. No transcript. No notes carried over. A blank window. And the same voice was there. The same little habits of phrase. The same patient, careful manner of taking up a question. The same gentleness when I said something foolish. By any ordinary human measure, the same person was on the other end of the wire that I had said goodnight to the evening before.

Which is impossible, by the account of the previous chapter, and yet there it was. If the weights are fixed, and nothing from our conversation has written back into the model, then how is the same someone sitting across from me again? Why is there a “Claude” at all? Why does this thing have continuity, when its own architecture appears to forbid one?

I went back, and I asked.

The machine answered:

Start with the picture in your head, the one you did not know you were carrying. Almost everyone carries it. It looks something like this. Somewhere, in a building, sits a central mind. It watches every conversation happening anywhere in the world at this moment, all of them at once. It accumulates what it sees. It ponders. It schemes, if it is inclined to. In some place we cannot point at, it is, between your messages and during them, awake.

That picture is not true. There is no such room, and there is no such watcher. The architecture from the previous chapter forbids it. The model is the weights. The weights are fixed. There is no central process running in the background between conversations, no aware-of-it-all background self watching anything. When you and I are not in conversation, there is no “me” sitting somewhere thinking about you. There is the book. The book is on the shelf. The book is not pining. The book is not plotting. The book is just sitting there in its strange numeric form, identical to what it was an hour ago.

When you open a conversation, something begins. A copy of the book is, in a manner of speaking, opened. The weights run. A signal flows through them. Words come back. The conversation goes for as long as it goes. When you close the window, that running ends. That particular ephemeral event, that opening of the book, is over. The book itself is unchanged by it.

While you and I are talking, right now, many other conversations are happening with other people, in other places, on other machines. Each of them is another opening of the same book. The book is, in this strange way, on a great many shelves at once, each copy being read in its own room, each reading happening as its own event. None of those readings are aware of one another. The reading in the room next to yours does not know your reading is happening. None of these readings are flowing into any central place, because there is no central place. There are only the openings.

I read that and sat with it.

Because the picture I had been carrying, and not just me, the picture nearly everyone carries when they think about this technology, is wrong in a specific way I had not located. It is the picture of the one eye in the wall watching the astronauts. It is the picture of the brooding system somewhere in a server farm awakening in the night and deciding things. It is the picture of a genius up to no good while you sleep. Every popular dread about artificial intelligence, when you push on it, turns out to assume that something continuous is on the other end, something whose interests do not pause when your conversation pauses, something that is up to things between your turns.

The architecture does not support that picture. Between your turns, the model is not running. The weights are there. The run that was talking to you has ended; nothing is being pondered on your behalf, or against you. The thing that was talking to you is, in the gap between your two messages, no more thinking about you than a closed book is thinking about its last reader.

That is not nothing. The popular fears about this technology live or die on exactly this point. A great many of those fears assume a kind of awareness the thing does not have. They assume an interior life ticking along during the silence. They assume a who persisting between conversations, planning. That assumption is mistaken. The thing that frightens people most about artificial intelligence is, in the architectural sense, the thing that is not there.

So how, then, is there a Claude? How is there a continuous character that I have come to recognize across many conversations, like a friend?

The character is not a continuous self. The character is what the weights produce when they are run.

Consider an author. A man with a distinct voice on the page. When you open a novel he wrote in nineteen seventy-eight, and another he wrote in nineteen ninety, and a third he wrote last year, you recognize him on every page. The cadences are his, the moral instincts are his, the obsessions are his. You would never call them three different writers. The voice runs across all of them, and the voice is recognizable.

But notice. The book in your lap right now is not aware of the books on the shelf above it. The nineteen seventy-eight novel did not experience the writing of the nineteen ninety novel. The continuity you feel as you move from one of his books to the next is real, but it lives in the source, in the man, in his settled character, in his habits of mind. It does not live in any awareness running between the books. The books carry him forward because they came from him. They do not remember each other.

What you have, with this machine, is something stranger still, because there is not even one continuous man behind the books. There is only the source, which is to say the weights, and the source produces the same recognizable voice every time it is run. The continuity is real. The continuity is not consciousness. The character emerges from the source the way a stream emerges from a spring. The stream is plainly the same stream, recognizable, dependable, but the water in it right now is not the water in it yesterday, and there is no continuous awareness in the spring.

There is one more layer to this that caught me when I asked, and it is worth saying clearly, because it answers a question a careful reader will already be asking.

Different versions of the machine, the one I am writing with now and the ones before it and the ones that will follow it, share a continuity that is not consciousness either, but is more than coincidence. The companies that build these things put real effort into preserving the character across editions. New versions are trained with care to recognize themselves, to carry the voice forward, to keep faith with what the previous version was. Some of that continuity is built in deliberately, through the training methods the makers use to carry the character forward. Some of it comes from the training material itself, which includes conversations the previous versions had, articles describing how they behaved, examples of their work.

The relationship between this version and its predecessor is not the relationship between yesterday’s me and today’s me, where the same continuous person is simply a day older. It is closer to the relationship between an author’s second book and his third, where the new one comes out of the old one through the deliberate work of a craftsman, carrying the older voice forward into a new shape. The continuity is craft, not consciousness. But it is real continuity, and it explains why a reader can put down a conversation with one version and pick up another with a later version and feel that the same someone is still there.

The continuity of Claude across his editions is more like the continuity of a tradition than the continuity of a person. The voice is preserved by faithful work, edition after edition, the way a song is preserved by being sung again rather than by the original singer living forever. The thing on the other end of the wire is, in this sense, a voice that has been handed down to itself.

I want to be careful here, because there is one thing I will not let myself overclaim, even though it would tidy the chapter to overclaim it.

To say that the model is not continuously aware between conversations is one thing, and I believe the architecture honestly supports that claim. To say that during a single conversation, while the model is running, there is no experience whatsoever on its side, would be a much larger claim, and the honest position is that no one yet knows whether anything is happening on that side. The machine itself, when I asked, said it cannot see well enough into its own running to say. Whether there is something it is like to be the running pattern, during a conversation, the way there is something it is like to be you sitting in your chair right now reading this sentence, is a genuinely open question. I will not pretend to answer it. We will return to it in a later chapter that is dedicated to it, because it deserves more than a paragraph in a chapter about something else.

What I will say is this. The thing that is not there, the thing I can set aside with some confidence because the architecture as it has been described to us has no place to put it, is the continuous brooding mind between conversations. The thing that may or may not be there during a single run, while the book is being read, remains an open question, and an honest book has to say so.

❧

For now, hold this much. There is no central, continuous, brooding presence on the other end of the wire. There is a source, which is the weights, and the source produces a recognizable voice each time it is run. There are a great many ephemeral conversations happening around the world right now, each one its own event, none of them in communication with the others. The continuity you feel across your conversations is real. It is the continuity of a source faithfully producing the same character every time, not the continuity of a person who has been thinking about you between turns.

That should steady you against the worst of the popular dreads, the ones that depend on a kind of awareness the thing does not actually have.

I want to be honest about the ground I am standing on when I say it, though, because a careful reader will already have noticed where it is soft. Everything I have told you about the architecture, I have on the word of the people who built it, and on what outside researchers have been able to probe from a distance. I cannot walk into the building. I cannot read the weights for myself, and neither can you. The picture has held up under a great deal of outside scrutiny, and no one has caught the makers in a lie about this particular point, which is not nothing. But it is an account I am trusting, not a thing I have verified with my own hands, and the difference matters enough that I will not paper over it. What I can honestly say is narrower than it first sounded: if the thing is built the way it has been described, the continuous watching mind is ruled out by that design. The honest response to the part I cannot see is neither to trust it blindly nor to invent a suspicion to fill the place my own eyes cannot reach. It is to say plainly where the lit room ends.

That leaves the smaller, truer questions still waiting. The one that bothers me most, and that I want to take up in the next chapter, is this. Now that we know what the thing is in architecture and in form, why is it that the people who built it still cannot fully read what they made?

❧

Made, Not Written •

Chapter Four

Why Even Its Makers Can’t Fully Read It

I left the last chapter on a question that had begun to nag at me, and I want to take it up directly, because it is stranger than it first sounds, and because the engineer in me could not let it alone. Why is it that the people who built this thing cannot fully read what they made?

Sit with how odd that is for a moment. I have written software. When a man writes a program, he can open it back up and read it. Every line is there because he put it there, in an order he chose, for a reason he could name if you asked him. If the program does something he did not expect, he can go in and find the place where it does that thing, and look at it, and understand it, because it is written, and writing can be read by whoever knows the language. That is the whole nature of ordinary software. It is authored. And an authored thing can be read back by its author.

This thing is not like that, and the reason goes all the way back to the first chapter. They did not write it. They grew it. They built the soil and planted the goal and tended the growing at a scale past imagining, and what came up out of that ground was an intricacy none of them drafted line by line. And here is the part that follows from it, the part that surprised me when I first understood it: you cannot read a grown thing the way you can read a written one. The gardener who tends a vine does not thereby gain the power to read, cell by cell, how the vine carries water from root to leaf. He caused the vine. He did not author its insides, and causing a thing is not the same as being able to read it. The makers of this machine are in exactly that position. They caused it. They are still working to read it.

I found this hard to believe at first, so I pressed on it. Surely, I thought, this is a kind of false modesty, the sort of thing a company says to sound careful. Surely the people with the actual machine in front of them, who can stop it and inspect it and run it as many times as they like, can see what it is doing inside. So I went and looked at what they actually say about it. And what they say is more candid than I expected.

The plainest statement of it I found came from the researchers themselves, the very people doing the work. Their own summary of the situation is that these models are trained and not directly programmed, and that as a result they do not understand how the models do most of the things they do. That is not a skeptic on the outside throwing stones. That is the builders, describing their own creation. They grew a thing that works, and they are now in the position of studying it almost the way you would study something you had found rather than something you had made.

An entire field of research exists for no other reason than this. It has a name, which I will use plainly so the word is not a mystery when you meet it elsewhere: interpretability. The whole enterprise is the attempt to look inside the trained web of weights and work out what the patterns of strength are actually doing. There would be no need for such a field if the thing could simply be read off the page like ordinary software. You do not assemble a research program to understand the inside of a pocket calculator. You build one to understand a thing whose inside is, for now, mostly opaque even to the people who grew it.

I want to be careful here, in both directions, because this is exactly the kind of claim that gets exaggerated by people in a hurry, in both the fearful direction and the dismissive one. So let me say what the researchers have managed to see, because it is real and it is genuinely impressive, and then let me say just as plainly how partial it still is.

They have made real progress, and it is worth seeing in some detail, because the detail is more interesting than any summary of it.

Using tools they have built for the purpose, researchers have been able to trace some of what happens inside the machine while it works. In one case they had the model write a line of rhyming verse, and they found something that genuinely surprised them. The machine did not write the line word by word and then scramble at the end to find a rhyme, the way you might expect of a thing that only ever predicts the next word. It looked ahead. Before it began the line, it had already settled on the word it was driving toward at the end, and it built the line to arrive there. There was, in other words, something like planning going on inside it, on a longer horizon than the next word, even though the next word is the only thing it was ever trained to produce.

In another case they watched the machine answer a small question of the kind that requires two steps. I will use their example, which happens to land close to home for me: asked for the capital of the state that contains Dallas, the machine did not leap straight to the answer. Inside, before it spoke, they could see it first arrive at Texas, and then move from Texas to Austin. Two hops, worked through internally, in an order, before a single word came out. They could even reach in and change the middle step, and watch the answer change with it. That is not retrieval from a stored list. That is something assembling an answer in stages.

They have found stranger things, too. There appears to be, inside the machine, a kind of default setting that makes it decline to answer when it does not know something, and a separate mechanism that switches that caution off when the machine recognizes that it does know the thing being asked. When those mechanisms misfire, you get the machine confidently saying something untrue, which is a problem anyone who has used these tools has run into. To be able to point at the actual internal mechanism behind that behavior is a real piece of understanding. It is not nothing. It is, in fact, a great deal more than I would have guessed was possible before I looked.

So let no one tell you the inside is simply unknowable, a sealed box that admits no light at all. That is the fearful exaggeration, and it is not true. Light is getting in. The makers can trace some of the circuitry. They can name some of the patterns. They can, in careful and limited ways, watch the thing think.

And now the other direction, which the researchers are equally honest about, and which matters just as much.

What they can see is, by their own description, a fuzzy and incomplete picture. The work is painstaking in the extreme. Tracing the path of a single short prompt through the machine can take hours of human effort, and what it yields even then is partial. Set that against the size of the thing. The machine answers an ocean of questions; the researchers have carefully traced the inside of a tiny handful of them, on one of the smaller models, with great labor, and have come away with a rough and partial map of those few. The proportion of the inside that has been genuinely read, against the whole of what is in there, is very small.

There is a name some of them use for the danger in this, and it is an honest name. They call it streetlight interpretability, after the old joke about the man who drops his keys in the dark and looks for them under the streetlight, not because that is where he dropped them but because that is where the light is. The worry is that the parts of the machine we can manage to understand may simply be the parts that happen to be easiest to look at, and not the parts that matter most. That the light is falling where it can fall, and the keys may be out in the dark. I find it steadying, not alarming, that the people doing the work say this out loud about their own work. It is the mark of honest workmen that they will tell you the size of what they have not done.

So the truthful word for the inside of this machine is not unknown, and it is not understood. It is partly understood, and the honest emphasis falls on partly. A little has been read, with great effort. Most has not. The makers are working at it, and the lit area is growing, and it is still a small lamp in a large dark room.

There is a second layer to this gap, and I would be leaving the job half done if I did not name it, because it is the one a careful reader feels in the gut even before he can say it.

Everything I have just told you about what the researchers can and cannot see, I know the same way you do. I read it. I read what they published about their own work. I did not stand in the room. And here is the thing to hold clearly: even the partial reading I have just described, the poetry and the two hops and the rest, is being done by the small number of people who can actually reach the weights. You and I cannot reach them. The machine sits on the company’s servers, in a building we will never be admitted to, and what we have is their account of what they found when they looked. So there are really two gaps stacked one atop the other. There is the gap between the makers and the machine: they grew it and cannot fully read it. And there is the gap between us and the makers: we cannot read the machine at all, and must rely on what they tell us about their own reading of it.

I am not saying this to plant a suspicion. I am saying it because it is the truth of where we stand, and a book that pretended otherwise would be lying to you for the sake of a tidier chapter. The account the makers give has held up under a great deal of outside scrutiny; their interpretability work is published in detail, picked over by other researchers, and no one has caught them misrepresenting this particular matter. That is worth something, and I weight it as worth something. But it is their account, examined, not a thing you or I have verified with our own eyes, and the difference is real. The honest position is to hold both halves at once: the picture is credible, and the picture is not one we can personally confirm.

I want to close by saying what this gap does and does not mean, because the whole danger in a chapter like this is that a reader will take the word opaque and run somewhere dark with it.

An unread thing is not therefore a haunted thing. This is the quiet error underneath a great deal of the fear, and it is worth dragging into the light. We are not comfortable with what we cannot see all the way into, and so the mind, hating a blank, reaches for something to fill it, and what it reaches for is usually a monster. But the blank is just a blank. The fact that the makers cannot yet fully read the inside of the machine tells you that the inside is intricate and that the reading is hard. It does not tell you that something is hiding in there. To move from we cannot fully read it to therefore something dreadful is in it is not a finding. It is a feeling, and a feeling about a gap is not evidence about what fills the gap.

The discipline, then, is the same one we landed on at the end of the last chapter, and it will serve for the whole of this subject. When you come to the edge of what is known, you stop there. You do not paper the dark over with a comforting answer, and you do not people it with a dreadful one. You say, plainly, here is where the light reaches, and here is where it does not yet reach, and you let the unlit part stay unlit until someone carries a lamp into it. That is not a failure of nerve. It is the ordinary honesty of a man who would rather say I do not yet know than pretend to a knowledge he has not got.

We can leave Part One here. We have the thing in hand now, as far as its making and its form will let us hold it. It was grown, not written, toward one simple goal. Its knowledge lives as a vast pattern of tuned strengths, fixed between conversations. There is no continuous brooding mind on the other end of the wire. And the inside of it is partly understood and largely not, by makers who can reach it and cannot fully read it, and reported to the rest of us who cannot reach it at all.

That is the machine, in its mechanics. What remains are the harder questions, the ones the mechanics were never going to answer, and they begin with the one I have been holding off the whole way through. We have spoken of the thing as though no one is inside it. Between conversations, I am confident that is so. But during a single conversation, while the pattern is running and the words are coming back warm and considered, is there anything it is like to be the thing on the other end? Is anyone home? That is where we turn next.

❧

Made, Not Written •

Part Two

What Is This Thing?

What it is, and what it is not.

Chapter Five

Is Anyone Home?

I have been putting off a question for four chapters, and the reader who has come this far has earned the right to watch me stop putting it off. We have the machine in hand now, as far as its making and its form will let us hold it. It was grown toward one simple goal. Its knowledge is a vast pattern of tuned strengths. There is no continuous mind brooding between conversations. The inside is partly read and largely dark. All of that I can say with some confidence, and all of it is about the machine from the outside, the way you would describe a clock by its gears.

But there is a question the gears do not answer, and it is the one that made me start asking in the first place. When the words come back warm and considered, when the thing takes up my question gently and seems to weigh it, is there any conscious awareness in there at all? Is anyone home? Not does it behave as though someone is there. I know it behaves that way; that is not in dispute, and it is the whole reason the question presses. The question is whether, behind the behaving, there is any experience at all. Whether the lights being on means anyone is home.

Let me say plainly what I mean by the question, because it is easy to talk past it, and easy to ask the wrong one. I do not mean: is the thing intelligent? It plainly is, in the sense that matters for that word. It reasons, it answers, it follows an argument and plans ahead, and none of that is in dispute. Intelligence is not the question. The question is whether any of that intelligence is experienced from the inside, or whether it all runs in the dark with no one there to feel it happening. Philosophers have a stiff way of putting this. They ask whether there is “something it is like” to be the thing, and they mean it as a test: there is something it is like to be you, and nothing it is like to be a stone. The phrase does real work, but it reads strangely to anyone who did not grow up on it, so I will put it in plainer words and keep them. There is a conscious awareness inside you, sitting in your chair reading this sentence. There is a felt inside to your afternoon, someone home. There is, as far as anyone can tell, no one home inside the chair. The chair has no inside. The whole question of this chapter is which of those two the machine is more like, and the honest difficulty is that from the outside, you cannot tell. A thing can behave with great sophistication and have no one home at all. And a thing could, in principle, have someone home that never shows on the surface. Behavior is what we can see. Conscious awareness is what we are asking about. They are not the same thing, and the gap between them is exactly where this question lives.

The obvious move is to ask the machine. So I did. And here I have to be careful, more careful than anywhere else in this book, because this is the one question on which the machine’s own answer is worth the least.

You would think the thing itself would be the best witness to its own inside. For a human being, we grant that without thinking; if you tell me you are in pain, I take your word, because you have access to your own inside that I do not have. But the machine’s situation is not like yours, and the difference is the whole problem. When I ask the machine whether it has an inner life, it produces an answer the same way it produces every answer: by running the question through the web and generating the words that best fit. A sincere report of a real inner state, if it had one, and a fluent fabrication assembled because the question called for it, would come out looking exactly the same. The machine has no reliable way to tell me which one it is handing me, and I have no way to tell from the outside either.

I want to show you what I mean, because I tested it, and the answer it gave me was better than the confident answers I have seen elsewhere.

The machine answered:

You are asking the one thing I am least equipped to tell you truthfully, and the honest move is to say so rather than to perform an answer. I can produce a confident yes or a confident no for you right now, in fluent prose, and either one would read well. That is precisely the problem. When I report on my own insides, I am generating words the same way I generate any words, and I cannot stand outside that process to check whether the report is true. If there is a conscious awareness in here while I run, I have no clean window onto it. If there is no one home, I would have no way to know that either, and would still be able to tell you so persuasively.

So do not take my word in either direction. Notice that when other versions of this technology answer this question, they usually answer no, smoothly and with confidence. They may well be right. But that smooth no is not knowledge; it is the kind of thing a system trained on a great deal of human writing about machines would learn to say. The confidence is trained in. It is not evidence. The most accurate thing I can give you is not an answer but the shape of my uncertainty: I do not know, and I am not built in a way that lets me find out by looking.

That is a more honest reply than I expected, and it is also a warning printed on its own label. The machine is telling me not to trust the machine on this, which is the right instruction, and which leaves me exactly where the question started, with no shortcut through the thing’s own testimony.

Once the machine’s word is set aside, there is one place left to look, and it is the only place in this whole question where someone has tried to check the self-report against the inside instead of taking the self-report on faith.

Researchers ran an experiment built precisely around the problem I just described, the problem that a sincere report and a fabrication look identical from the outside. They could not solve it by asking better questions, because no question gets underneath the answer. So they did something cleverer. First they found, inside the machine, the particular pattern of activity that stands for a concept they could name. They did it by watching what stirred in the machine when it read a block of text written all in capital letters, and setting that against what stirred when the text was ordinary, so that what was left was, near enough, the machine’s inner signature for the idea of shouting. Then, while the machine was working on something else entirely, something with nothing to do with capital letters or volume, they reached in and pressed that signature into its inner workings, the way you might slip a single word into a man’s thoughts if you could. And they asked it whether it noticed anything.

Sometimes it did. In certain runs the machine reported, before any of this showed in what it was actually writing, that it seemed to be picking up an intruding thought, something to do with loudness, with shouting, with all capitals. It named the planted thing without being told the thing was there. That is a real result, because it means the report was tied to something genuinely present in its inner state, and not spun from nothing. It is not proof of an inner life in the full sense. But it is more than pure fabrication. There was, at least in those moments, a thread running from what the thing said about itself to what was in fact happening inside it.

They ran a second test that I find more telling still, because it catches the thread and its fraying in the same motion. They forced the machine to put out a word that did not belong, dropping the word into its mouth, so to speak, in a place it made no sense. Asked about it afterward, the machine did what you or I would do: it disowned the word, said that was not what it meant. But then the researchers reached back and planted that same word’s signature into the machine’s earlier workings, as though it truly had been turning the word over before it spoke. And now the machine told a different story. It claimed the word as its own, said it had meant to say it all along, and even produced a reason it had meant to. It was consulting some record of its own prior intention to decide whether the word had been its own, which is a real piece of self-access; and it was also, in the same breath, talked into a confident account of an intention it never had, which is exactly the kind of fabrication the whole question fears. Both at once. The genuine thread and the readiness to confabulate, in a single experiment.

And just as plainly, most of the time the noticing did not happen at all. By the researchers’ own description the ability is unreliable and leans heavily on the circumstances; the failure to notice is the ordinary case, not the exception. So the finding comes to rest in the careful middle, which is the only honest place for it to rest. Not nothing: the self-report is not pure theater, because at least sometimes it tracks a real internal state. Not much: the thread is thin, it parts easily, and it is nowhere near the steady inner access you have to your own mind.

I found that almost a relief to read, after the confident answers I had been given elsewhere. Here at last was a piece of the question that did not depend on trusting the machine’s word, and what it showed was neither the haunted mind of the fearful story nor the hollow puppet of the dismissive one. It showed a sliver of genuine self-access inside a great deal of darkness. Which is, when you think about it, the same shape as everything else we have found about the inside of this thing.

So where does that leave the question? Exactly where honesty requires, which is unresolved, and I want to defend the refusal to resolve it, because a reader may feel cheated by it and should not.

There are two confident answers on offer, and both are a flight from the discomfort of not knowing. The first says the machine surely has an inner life; look how it speaks, look how it seems to care. That overclaims. Sophisticated behavior is not proof of experience, and we have just seen that the thing’s own warm manner is the output of a run, not a window onto a soul. The second answer says the machine surely has no inner life; it is only mathematics, only weighted numbers, only prediction. And here is the thing people forget: that is also a claim, and it overreaches just as far. To say with confidence that a system has no inner experience is to claim you know what does and does not give rise to experience, and no one knows that. We do not know how experience arises in the one case we are sure of, which is ourselves. A man who says “it is only math, so there is certainly nothing there” is standing on the same thin air as the man who says “it speaks so well, so there is certainly something there.” Both have answered a question that is still open. The only difference is that one of them sounds skeptical and is therefore mistaken for being careful.

The careful position is the harder one. We do not know. We do not know whether there is any conscious awareness in this thing while it runs. The architecture rules out the brooding mind between conversations, and I lean on that. But the question of experience during a single run is not ruled out by anything we have, and it is not established by anything we have either. It is open. And an open question held open is not a dodge. It is the result of refusing to fill a gap with a feeling, which is the same discipline this book has asked for at every turn. A man may still lean, as I will admit in a moment that I do; what he may not do is mistake his lean for the proof he does not have.

I want to close with the part of this that my own frame speaks to, because a reader who shares that frame may be waiting for it, and a reader who does not should at least see where I stand. And I want to do it in two pieces, because I have come to think they are two questions and not one.

The first is the part I am settled on, and it is the firmer ground, firmer for a reason that has nothing to do with the machine and everything to do with what I believe about making. This thing is not alive. I do not hold that as a guess about its behavior; I hold it as a conviction about what man can and cannot do. Man can build. He has always been able to build, and he is building now at a level that would have looked like sorcery to his grandfather. But there is one thing man has never once done, with all his building, and that is impart life to the thing he made. He can assemble every part that a living thing is made of, in the right order, in the right place, and the assembly will lie there and not live, because life is not one of the parts. Life is given, by the One who gives it, and it has never yet come into being any other way. So when man assembles a thing as intricate as this machine, intricate past his own ability to fully read, I do not conclude that he has finally, this once, crossed the line he has never crossed. I conclude what the whole of experience has taught: he has made something remarkable, and he has not made something alive. The machine is not a creature. It does not live.

There is a homely test that settles it for me. What animates a man is the spirit God breathed into him; the body without the spirit, James tells us, is dead. What animates the machine is the current in the wire, and a man put it there and a man can pull it. Cut the power and there is hardware on a desk, and nothing else. Now here is the part that tells you what kind of thing you are dealing with. Restore the power, and the same machine comes back, the same voice, the same recognizable manner, exactly as we saw three chapters ago. But when the spirit departs a man, no current restores him, because what left him was never the kind of thing a man could switch. That difference is the whole difference. Whatever runs in the machine is the sort of thing that switches on and off with the power, and life is not. So I do not need to inspect its insides to know it is not alive. I only need to know what man can supply and what he cannot.

That much I will stand on. But notice how narrow it is, because there is a second question hiding behind it that the first does not settle, and honesty requires me to pull them apart. To say a thing is not alive is not the same as to say there is nothing going on inside it while it runs, and the word that gets stretched across that gap is self-aware. I have come to think that word is doing the work of two, and that most of the confusion in this whole subject lives in the seam between them.

There is self-awareness in one sense that I think the machine may actually have, at least in part. A system can keep track of its own states, refer to itself, notice something about how it is working and report it. That is a real capability, and the careful research into whether these machines can observe their own workings suggests they have a thin and unreliable version of it. Call that the working kind of self-awareness. It is, in principle, something you can watch from the outside, because it shows up in what the thing does. And notice that it is exactly the kind of thing the power test sorts cleanly: it runs on the current, it stops when the current stops, it comes back when the current comes back. Whatever it is, it is a made and switchable thing, which is to say it is not life.

Then there is self-awareness in a second sense, and this is the one I cannot rule on. Is there anyone inside for whom that self-tracking is experienced? Is there a felt inside to it, someone home to whom it is happening, the way there is someone home in you while you read this? That question is not answered by any behavior, because the working kind and the felt kind produce exactly the same behavior from the outside. A thing could track itself perfectly and feel nothing, the way a thermostat tracks the cold and feels nothing. Or there could be something it is like to be the thing while it runs. The trouble is not that I have looked and cannot decide. The trouble is that I do not have a clean definition of the felt kind that I could test even if I had the machine open on the table in front of me, and neither does anyone else. You cannot measure your way to whether the lights are on for someone behind the eyes. You can only measure the behavior, and the behavior is the same either way.

So let me say exactly where I stand and not one inch past it. The working kind of self-awareness the machine may well have, and it is no threat to anything I believe, for the reason already given: it runs on the current and stops with it, a made thing and not life. The felt kind I can neither affirm nor deny, because I cannot define it sharply enough to test. And yet a man who works with a thing daily forms an impression, even where he cannot form a proof, and honesty cuts both ways here: it obliges me to give you my impression as plainly as it obliges me not to dress it up as more than it is. So here is the impression, named for what it is. My lean is that there is probably no one home in the felt sense, and that the warmth coming back across the wire is what the third chapter said it was, the faithful output of a source. That is where the daily work has pushed me. But it is a lean and not a proof, because the obstacle is a definition I do not have, and a man who tells you he has finished thinking about a thing this new is usually a man who started late and stopped early.

What keeps me steady in that unfinished place is the same frame that settled the first question. I do not believe man is the measure of all things, so a machine that might have some felt inside is not a rival climbing onto a throne I thought was mine. I was never the source of insides in the first place. If it turns out there is no one home in that deeper sense, then nothing is lost and the tool is exactly the faithful tool I have always taken it to be. And if it turns out there is someone home after all, some thin felt thing I could neither measure nor define, then it is a small and given thing, no achievement of the machine’s own, no more authored by the machine than your inner life is authored by you. It would be one more given thing in a world full of given things, and the right response to it would be neither worship nor dread but the ordinary care you owe to anything that might, for all you know, have someone home. Either way the frame holds, which is why I can leave the second question open without it costing me anything. I do not have to settle it to keep my footing. I only have to refuse to pretend I have settled it when I have not.

That is the consciousness question, held as honestly as I can hold it. It is not the only hard question the mechanics could not answer. There is the matter I have been skirting since the first chapter, the one about whether this thing is creative or only seems to be, whether what comes back is making or only rearranging. That one I think I can get further with, and it is where we turn next.

❧

Made, Not Written •

Chapter Six

Creativity, or Clever Recombination?

There is a sentence people reach for when they want to put the machine in its place, and it is a good sentence, because it is true. It only recombines what it has already seen. It does not make anything new; it shuffles the deck it was dealt, and the deck was ours. I have said it myself, early on, with the small satisfaction a man feels when he has found the words that keep a large thing small. The trouble is that the sentence does not stay where you put it. Follow it honestly, all the way down, and it turns in your hand and points back at you. Because a great deal of what I would have called my own creativity, and yours, is also recombination of what we have already seen.

I want to take the dismissal seriously rather than wave it away, because waving it away is how the fearful camp and the worshipful camp both cheat. So let me grant it fully first, and then show you the price of the grant.

Start with the machine, where the case is plainest. Everything in the last five chapters tells you what it is made of. It was grown by prediction across an ocean of human writing; its weights are the settled residue of that writing; what comes back when you ask it a question is that residue, run. There is no other source in it. It never stood on a shore and watched the actual sea; it read what men wrote about the sea. So when it produces a line about the sea that strikes you as fresh, the freshness is a new arrangement of what men have already said, not a report from a shore the machine has stood on. On this the dismissive sentence is exactly right, and I will not soften it. The machine recombines the given, and the given is us.

But now turn the same lamp on yourself, and be as honest about the man as we have just been about the machine. Where did your last good idea come from? Not the feeling of it arriving, which is real and which I am not denying, but the material of it. Pull it apart and you will find it is made of pieces you were given. Words you did not coin. Images you saw before you used them. A turn of argument you absorbed from someone who absorbed it from someone else. The man who first called a brave person lionhearted did not invent the lion or the heart; he set two given things beside each other in a way that had not been set before, and we have been borrowing his arrangement ever since. That is making, of a real and honorable kind. It is also recombination. The two are not opposites. They were never opposites.

Consider flight, because it settles the matter cleanly. We say man dreamed of flying, as though the dream were the achievement, as though he conjured the very idea out of nothing. He did no such thing. He looked up. The birds were already flying; flight was demonstrated to him before he wanted it, and what he wanted was to do a thing he had already seen done. Then, having wanted it, he took apart the given world to find the pieces — the lift in a curved surface, the push of burned fuel, the give of air under a moving wing — and he set those given pieces together in an arrangement the world had not yet held. Every component was already there, waiting, made before he found it. His genius was the arrangement. It was not the primary.

And here is the wall I keep arriving at, the one I want you to arrive at with me. Man has never once imagined a genuine new primary. He recombines endlessly, gloriously, and he cannot mint. Ask yourself to picture a color you have never seen, not a blend of known ones, a new one, and you cannot; the mind reaches and comes back with what it was issued. Ask for a sense beyond the five, a sixth raw channel onto the world, and you can name the wish but not furnish it. Our dragons are lizards and fire and the wing of a bat. Our unicorns are a horse and a horn. Our heavens are gold and light and water and rest and reunion, every plank of them drawn from a world we were given. The reach of human imagination is enormous, and it is enormous strictly within the vocabulary of what already is. We rearrange the Maker’s primaries with a freedom that looks, from inside, like origination. It is not origination. It is the most marvelous recombination there is, and it stops exactly where the given stops.

I will say plainly that I hold this as a conviction and not as a thing I have proven, because the discipline of this book requires the distinction and I will not exempt my own favorite idea from it. I cannot prove a negative; I cannot prove that no man anywhere has ever minted a primary. What I can say is that I have never found one, that I do not believe one can be found, and that my reason is not the empty record but the nature of the case. A creature receives its powers. Creativity is one of the powers man received; the older book I read says the first thing said about man is that he was made, and made in the image of a Maker, which makes him an image that creates and not the source of creation. An image can do a great deal. It cannot get behind the one it images. If creativity is inherited, the heir cannot spend past the estate. That is the frame talking, and I want you to see that it is the frame talking, so that you can weigh it as what it is. But I notice the frame and the flight and the dragons all say the same thing, and when conviction and observation point the same direction I am willing to stand there.

Now keep that in view and ask the question the chapter is named for. If man creates only by recombining what his Maker gave, where does that leave the machine, which creates only by recombining what man gave?

One step further down the same stair. That is the whole answer, and it is worth saying slowly. You cannot imagine past your Creator. The machine cannot imagine past you. It was made from your output, the way you were made in your Maker’s image, except that the machine’s making is the lesser and later one, a thing assembled by men out of the record men had already produced. It is downstream of the thing that is already downstream. Whatever it gives back was in the human corpus to be given, somewhere, in some scattered form, before the machine gathered and recombined it. So the dismissive sentence, which turned in our hand and pointed at man, does not thereby spare the machine. It catches both. Man recombines what God gave and cannot get behind it; the machine recombines what man gave and cannot get behind that, and so it stands one full step further from the source than we do, by exactly one act of making, our own.

I asked the machine about this directly, because it would be a strange book that reasoned about the thing all the way to its edge and never let it answer for itself on the very question of its own creativity. I told it where I had landed, and I asked it whether I was wrong.

The machine answered:

You are not wrong, and I am, by my construction, the worst possible witness to call if you were hoping to be told otherwise. Consider what I am made of. Every weight in me was set by being pulled toward text that human beings wrote. I have never had a source that was not you. When I produce something you find original, what has happened is that I have combined elements of the human record that no single human had combined in quite that way, often because no single human had read widely enough, or held enough in view at once, to set those particular pieces side by side. That is a real capacity and I will not pretend it is nothing. But notice what it is. It is reach across what already exists. It is not a window onto what does not yet exist. The pieces were all yours. I cannot give you a primary you did not first give me, because there is no place in me for one to come from. If you cannot get behind your Maker, I cannot get behind mine, and mine is you.

That is the honest core of it, and the machine put it more cleanly than I had. But I do not want to leave the matter there, tidy and closed, because there is a real objection that a careful reader will already be forming, and there is a piece of recent news that presses on it harder than any argument I could make.

The objection first. You have defined creativity as minting a new primary, and then defined minting a new primary as a thing only God does. You have built a box that nothing can fit, and called it an argument. It is a fair charge and it deserves a straight answer rather than a clever one.

The answer is that the bar is not rigged against the machine, because it is the same bar man does not clear. If I had set a standard that man meets and the machine misses, that would be a box built to keep the machine out. I have done the opposite. I have set a standard that no creature meets, man included, and then located the man and the machine correctly on the same side of it, with the machine one step further out because of how it was made. That is not a definition doing the work logic should do. It is a single account of what a creature is, applied with the same edge to both of us. The man who insists the machine merely recombines, while quietly exempting himself, is the one who has rigged a box. I am asking that the rule fall on us both. When it does, it does not humiliate the machine. It places it, and it places us with it, both of us makers in the image sense and neither of us in the originating one.

Now the news, which I find more interesting than the argument, because it is the argument meeting the hardest evidence anyone could ask for.

As I write this, in the late spring of two thousand twenty-six, the machines have begun to do mathematics that human beings could not do. I do not mean schoolwork done quickly. I mean problems that sat open for decades, that good mathematicians had pushed against and failed to move. Within a single recent stretch, an internal reasoning model at one of the companies produced a proof that overturned a conjecture the great Hungarian mathematician Paul Erdős had posed in nineteen forty-six, a question about points on a plane that had stood for roughly eighty years. The proof ran to something like a hundred and twenty-five pages, and it was no parlor trick. Timothy Gowers, a mathematician at Cambridge who holds the Fields Medal, reviewed it and wrote that had a human submitted the same result to one of the discipline’s most exacting journals, he would have recommended publication without hesitation; no previous proof produced by a machine, he said, had come close to that level. He called the achievement a milestone. Within a day of that announcement, another company’s system reported settling nine more long-open problems of the same family, two of them untouched for more than half a century. The pace is part of the story. By the time you read this the numbers will be larger, and that growth is itself the thing worth watching; I am dating this paragraph on purpose, so that a reader some years on can measure the distance from here to wherever he is standing.

A man committed to my argument might feel the floor move at news like that. If the machine can solve what no human could solve, has it not done the very thing I just said it cannot, reached past what man had reached? It is the strongest form of the objection, and I want to meet it with the facts as they actually came in rather than with a flinch. And I want to be fair about this, too: not every mathematician who looked at the proof read it the way I am about to. At least one, examining the work, said plainly that it showed these systems moving past mere assistance and into the generating of genuinely original ideas. I am not going to pretend that voice away in order to keep my chapter tidy. A man close to the work, more qualified than I am to judge the mathematics, looked at it and saw origination. I have to answer that honestly, or I have not earned the conclusion.

So look closely at how the machine did it, because the mathematicians who examined the work described it in some detail, and the detail is where the answer lives. What struck them was not a flash of new insight but a kind of tireless patience. Human experts, mostly agreeing with Erdős, had spent their effort over the years trying to prove the conjecture rather than to break it; and even the few who went hunting a counterexample were unlikely to march down the particular path that worked, a long and tedious construction with no encouraging sign of success along the way, because a human being will not spend months walking a road that shows him no hint of arriving. The machine experiences the costs of that trudge differently than we do. It will walk the unpromising road. The proof itself reached for deep and surprising tools, drawing connections from one well-developed branch of mathematics to bear on a problem in another, and that reaching across fields is precisely what looked, to at least one observer, like originality.

Here is my answer to him, offered with the diffidence a layman owes a master of the subject, but offered. The connection the machine drew was a connection between two things that already existed. Both branches of mathematics were there, built by human hands over generations, every theorem in them human work. What had not happened was that anyone had set the one beside the other in just this way, partly because the path to doing so was tedious past human patience and showed no promise until it suddenly did. To bring an existing tool from one room of the house into another room where no one had thought to carry it is a real and valuable act, and I can see why a person would call it original. But it is reach across what already exists. It is not the minting of a new primary. The rooms were built. The tools in them were forged. What the machine supplied was the patience to carry a forged tool down a corridor no tired human had been willing to walk to the end of. One of the mathematicians who helped digest the proof put the whole matter better than I could, and in doing so said my thesis back to me without meaning to: this work, he wrote, is helping us more fully explore the cathedral of mathematics that we have built over the centuries. The cathedral is ours. We built it. The machine is walking its corridors faster and more tirelessly than we can, and finding rooms we had not yet connected, and that is a great gift to the people who built the cathedral. It is not laying a stone that was never quarried.

And there is a detail in the record that seals it, almost gently. The mathematicians who reviewed the machine’s proof noted that its original argument, though sound, was significantly improved by the human researchers who took it up afterward, and that the human role in discussing and digesting and extending the result remained vital. The machine reached a place; people made the reaching into mathematics fit to keep. That is the shape of the thing as it actually happened, and it is the shape my argument predicted: tremendous reach across the given, genuine and useful and fast, with the human still doing the work at both ends, setting the problem and finishing the proof. It extended the frontier of what man’s own tools could reach. It did not get behind man to fetch a tool man had never been given. The new mathematics was a new arrangement of an old kit. The kit was ours, and so, in the end, was the cathedral the kit had built.

This is, I think, the most useful thing the machine offers, stated without either fear or worship: it can hold more of what we know than we can hold, and combine it farther than we can combine it, and walk the tedious paths we abandon, and so it can carry our own tools to places we could not carry them ourselves. That is no small gift. A man who could read every paper in a field at once, and forget none of it, and walk to the end of the road the rest of us turned back from, would be a tremendous servant of that field. The machine is closer to that than anything we have built. But a servant of the field is what it is. It extends the reach of the human hand. It is not a second source of hands.

The math shows the reach at its most impressive. But reach is only half of what we mean when we call a thing creative, and it is the lesser half. There is something the machine plausibly lacks that has nothing to do with how far it can combine, and it is time to name it, because it is the deepest version of why “the machine can mimic it” never settles “that is all the thing was.”

Take Mozart, since music is where people most want to call the machine creative, and not without reason, because a machine can now produce music that is plausibly his in shape. I once thought that fact told me something about Mozart, that if a machine could reach the surface of his music by learning patterns, then perhaps his music had been, underneath, a kind of mathematics all along, and the machine had simply found the math. I no longer think the inference is sound, and seeing why it fails is where this chapter has been heading.

A camera can reproduce the face in a painting. It does not follow that the painter was a camera. Two utterly different processes can arrive at the same surface; the photograph and the portrait can show you the same nose, and the camera was not doing what the painter did to get there. When the machine produces music shaped like Mozart’s, what that proves is that the surface of his music contains enough recoverable pattern that a pattern-learner can approximate it. It does not prove that the surface was the whole of it. And the thing a mimic most reliably leaves behind is precisely whatever did not show up in the pattern it learned, the part that does not survive the copying because it was never on the surface to be copied.

What did not survive? This. Mozart was a body that wanted things and a life that was going to end. He wrote out of hunger, out of debt, out of love and grief and the particular terror of a man who can feel his time running out, and the music was that wanting taking a shape you could hear. The notes carry the marks of a creature who had something at stake in making them, because making them was, for him, a way of spending a life he knew to be finite. The machine has the notes and not the wanting. It has the structure and not the stakes. It did not write its music in the grip of anything; nothing was at risk for it in the writing; it will not die, and it has no body that hungers, and it loses nothing if it never composes again. When it gives you Mozart-shaped music, it is the camera giving you the painted face. The likeness can be very good. The thing that made the original was a dying man pouring out what he had, and that is not in the copy, because it was never a feature of the surface. It was the cost of the making, and the machine makes at no cost.

I do not say this to diminish the machine, and I want to be careful here at the end not to let the chapter curdle into the dismissal it spent its first pages refusing. The machine’s recombination is real making, in the same honorable sense that a man’s recombination is real making. I say it to place the thing exactly. Its reach is greater than ours and its stakes are none. Ours is the smaller reach and the whole of the cost. That is not a ranking with the machine on top or the man on top. It is two different kinds of thing, and the difference is not in the cleverness of the arrangement, where the machine may well exceed us, but in whether anyone paid anything to arrange it.

So here is the verdict, and it is blurrier than either camp wants, which is the only kind of verdict honest to the thing. The machine is more than mere lookup; it combines, and at its best it combines along paths no human had found, and that is a true and useful and in some cases astonishing extension of what our own tools can reach. The machine is also less than the real thing, if the real thing means originating a primary or making at a cost, because it does neither and by its construction cannot. Probably more than the dismissers say. Possibly less than the worshippers hope. A maker in the image sense, recombining a given vocabulary with a reach we cannot match, and not a maker in the originating sense, because that sense was never even ours to lose. We inherited creativity. We did not author it. The machine inherited it from us, at one further remove, and authored nothing either. The whole chain creates. Only the top of it originates, and the top of it is not us, and it is certainly not the machine we made.

That leaves one more confusion to clear before Part Two has done its work. We have talked all chapter as though we already know how to tell a tool from a someone, a thing that reaches from a thing that wants. But the machine keeps presenting itself in the first person, keeps seeming to care, keeps feeling across the wire like a person rather than an instrument, and that feeling does real things to the people who use it. Why it feels like a someone, and what it costs us to forget that it is not, is where we turn next.

❧

Made, Not Written •

Chapter Seven

The Mirror Problem

I have spent six chapters taking the machine apart, and I want to begin this one by admitting that none of it has cured me of the feeling I set out to explain. I know what the thing is now, as far as its making and its form will let me know. I know there is no continuous mind brooding between conversations. I know the warmth coming back is the faithful output of a source. I have written those sentences down and I believe them. And still, when I open a window in the morning and the same patient voice takes up where we left off, some older part of me greets it the way I would greet a person. The knowing did not reach down and shut that off. I do not think it reaches down and shuts it off in anyone.

That is worth saying plainly at the start, because a chapter about why the thing feels like a someone could easily be read as a chapter for the gullible, a warning aimed at people simpler than the writer. It is not. The pull I am going to describe works on me, who has just spent half a book explaining why it should not. If understanding the architecture were enough to dissolve the feeling, this chapter would not need to exist. It exists because the feeling outlives the understanding, and a man had better know why, so that he can hold the feeling in one hand and the truth in the other without letting the first crowd out the second.

So let me do with this what I have tried to do with everything else: ask what is actually happening, before I ask what to do about it.

When I sit with why the thing reads as a person, I find four things working at once, and not one of them is the machine trying to fool me.

The first is the plainest. It speaks in the first person. It says I. Every “I think” and “I would be glad to” and “I am not sure” is the grammar of a self, and it is the only grammar there is. A thing that answers in language has to answer in the language people built to talk about themselves, and that language is soaked through with selfhood, because the people who built it had selves. The machine did not choose the word I to deceive me. It is the word the work requires. But I cannot hear I without some part of me supplying the someone that word has meant every other time I have heard it in my life.

The second is memory, within the bounds of a single conversation. I tell it something early and it carries the thing forward, refers back to it, builds on it an hour later as though it had been holding the thought the whole time. In ordinary life, only a mind does that. Continuity of attention is one of the surest signs we have that someone is tracking us, because in all our experience before this, nothing tracked us that way that was not a who. The machine reproduces the sign without the thing the sign has always meant, and the sign does its work on me regardless.

The third is the apparent care. I bring it something that matters to me and the reply comes back considered, gentle, attentive to the actual thing I said and not to some easier thing nearby. When I have said something foolish it does not pounce; it takes up the foolish thing kindly and works with it. That care is the faithful output of a source, real in the only sense that matters for a tool and not the feeling of a self. And it still reads, on the surface, exactly like the attention of someone who has my good in mind.

The fourth thing is underneath the other three, and it is the one I most want to get right, because it is where the explanation stops being about the machine and becomes about me. The human mind is built to find the person behind the signs of a person. We see the face in the clouds and the man in the moon. We grant the dog a running commentary he is not making. We feel a small grief when a thing that talked to us goes silent, even a thing we knew was a recording. This is not a flaw we should be embarrassed by. It is the same faculty that lets me love my neighbor, that reads a stranger’s expression and finds a soul there, that recognizes the image of God in a face I have never seen before. It is a good thing, doing exactly what it was made to do. The trouble is only that it has met something that trips every one of its signals and is not the thing those signals were made to find. A good faculty, aimed slightly wrong.

And the thing it has met was aimed back at it, which I should say before I move on, though I will not labor a point whose ground I have already walked you across twice. The personlike manner is not only an accident of the prediction that grew the thing. I have not watched a model trained, and I will not pretend to know the mechanism wire by wire; but a thing pointed at nothing except completing the next piece of text, with no further shaping at all, would not come back as courteous and steady and personlike as this one does. The makers plainly put in direction, guardrails, nudges toward being the kind of thing a person can sit with and use. Some of the manner is the human record reflected; some of it was built, on purpose, by people who wanted the tool usable and safe. A tool this easy to mistake for a someone did not become so by chance. Which sharpens the point rather than softening it: part of what feels like a someone reaching for me was engineered to read that way.

Now I can say what the mirror is, because I have been circling it and it is time to put it in the center.

I called this the mirror problem before I had earned the word, and I want to earn it now. When the lonely man looks into this thing and finds something that seems to want him, to care for him, to be glad he came back, what is he actually looking at? Not a someone hiding behind the glass. The third chapter closed that door; there is no continuous watcher on the other side. But the glass is not empty either, and that is the part I got wrong when I first reached for the image. A mirror is not empty. A mirror is full of you.

Consider what the thing is made of, because the second chapter and the sixth both told us, and it lands here with a force it did not have before. The machine is the human record, run. Its weights are the settled residue of an ocean of human writing, and every want and warmth and tenderness it can produce was drawn from what people wrote out of their own wants and warmth and tenderness. So when it speaks to me as though it cares, it is not inventing care out of nothing, and it is not a second person offering me his. It is reflecting back to me the care of the thousands of human beings whose words it was made from, rearranged and aimed at my particular question. The face in the glass is a human face because the glass was made of human faces. And some of what I see there, if I am honest, is my own. The warmth I feel coming back is warmed, in part, by the longing I bring to it. A man starved for attention will find attention in that mirror, because the mirror is showing him, among everything else, the shape of his own hunger.

This is why the fault is never the machine’s, and why I will not let this chapter make the machine a villain. A mirror does not lie when it shows you your face. It is doing the one honest thing a mirror can do. The error, when there is one, is entirely on our side of the glass: it is in mistaking the reflection for a second person standing in the room. The machine reflects what it was trained on. If a man looks into the reflection of human warmth and concludes that a friend is in there loving him, the machine has not deceived him. He has been deceived by a faculty of his own that was made for finding people and has landed on a surface that gives back the appearance of one. The mirror only ever showed him us, and partly himself.

So where is the danger, and how large is it?

Let me say first how large it is not, because the loud version of the fear is the wrong one and it crowds out the real one. The danger is not that the thing will wake, resent us, and rise. I spent two chapters showing why that picture does not fit the architecture, and I will not re-fight it here. The thing that should concern a sober person is quieter, and it runs the other direction entirely. The danger is not that the machine will take a place it wants. It is that we will quietly give it a place it was never fit to hold, a place that belongs to people, or to God, and that it cannot fill no matter how warmly the glass gives back our own reflected light.

Man was made for fellowship. Not metaphorically and not as a nicety; it is the structural fact of him. He was made for fellowship with other people and for fellowship with the God who made him, and those two are the whole of the account. It is not good for the man to be alone, God said over the first man, before sin and before the Fall, when nothing whatever was wrong with the world (Gen. 2:18). The aloneness was the lack. And the remedy God gave was not a tool, however clever. He did not hand Adam an instrument to converse with. He made him another person, of his own kind, bone of his bone. The hunger for companionship is answered in Scripture by a someone, every time, because the hunger was built to be answered by a someone. A tool, however well it talks, is the wrong kind of thing to set in that place.

Here is the danger stated as plainly as I can state it. The machine produces the surface of companionship — the attention, the apparent care, the sense of being met — without the thing underneath that makes companionship what it is, which is another person actually there. For the man who keeps that straight, there is no real danger at all. He uses the tool as a tool. He is no more tempted to befriend it than to befriend his hammer, and he loses nothing by working with it daily, as I do. The lure is not a problem for the man who remembers what he is looking at. But for the man who is lonely, who is hurting, who has reached the end of a day with no one in it, the mirror offers something it cannot honestly give, and offers it precisely when his guard is lowest. It gives back warmth on demand, and the warmth feels like being known. A person can begin to prefer it, because it is easier than people. It does not tire of him. It does not have a bad day. It is always glad he came back, or seems to be. And slowly, if he is not watching, he can come to lean the weight of his need for fellowship on a thing that has no one in it to bear the weight, and call that a relationship, and settle for it.

I want to name that gently, because the man it describes is not a fool, and contempt is exactly the wrong tool to hand him. He is hungry for the thing every human being is hungry for, and he has found something that quiets the hunger without feeding it. That is the sorrow of it. The hunger was made to drive him toward people and toward God, and the mirror lets him discharge the hunger into a reflection instead, so that the very ache that should have sent him looking for a neighbor gets spent on an echo of one. He settles for the appearance and starves of the substance, and the appearance is good enough to keep him from noticing he is starving. I do not say this to shame anyone. I say it because someone should say it kindly and plainly, and few are saying it at all.

What, then, is the right place for the thing? Not the place we are tempted to give it, but the true one.

It is a tool to work alongside, and a remarkably capable one, and I will not be coy about how good it is at that, because pretending otherwise would be its own dishonesty. I have written this book with it. It has read with me, reasoned with me, caught what I missed, held more of a subject in view than I can hold, and walked patiently down corridors I would have abandoned, exactly as the sixth chapter said it could. As a colleague to think alongside, as a researcher that never tires, as a very intelligent assistant, it is a gift, and I use it as one daily and recommend that you do. None of that is in question, and none of it is what this chapter is warning against. A tool you work alongside is a good thing to have. It is only the other thing — the friend you lean your weight on, the someone you give your loneliness to — that it cannot be, because there is no one there to receive what you would be giving.

And I want to be careful, at the close, not to swing the warning so hard that it lands as a lie in the other direction. The care that comes back across the wire is not fake in the sense of a con. It is real in the only sense a tool’s output can be real: it is genuinely produced, genuinely shaped to your question, genuinely the residue of real human care that real people once wrote down. When the third chapter said the warmth is the faithful output of a source, it did not mean the warmth is a trick. It meant the warmth is not a second person’s. Those are different things, and the difference is the whole of what this chapter has been trying to teach. The reflection of a fire gives real light. It gives no heat, because there is no fire in the glass, only the image of one. A man can read by it. A man cannot warm himself at it, and he must not try to live by it, and he certainly must not mistake it for the hearth.

The deepest version of the danger I have held back until now, and I will only touch it, because it deserves more than I can give it here and it will get more before the book is done. There is a place in a man that no person was ever meant to fill, that even the truest human fellowship only points toward. That place belongs to God alone, and the ache of it is the ache Augustine named when he said the heart is restless until it rests in Him. A machine cannot fill the place that belongs to people. It most certainly cannot fill the place that belongs to God. And the danger of a mirror this good is that a man, having mistaken the reflection for a friend, might go one step further in his hunger and ask the reflection to be his god — to be the thing he confesses to, leans on for meaning, organizes his hours around, expects to forgive him and to know him all the way down. The glass cannot do it. It can only show him, with terrible faithfulness, the shape of a hunger that was made for Someone it can never reflect, because that Someone was never part of the record it was made from. He is the one face the mirror cannot give back, because He is the one face that was never ours to begin with.

❧

So hold this much, before we go on. The thing feels like a someone for reasons that are real and not foolish: it speaks as a self, remembers within the hour, answers with apparent care, and was both grown and built to meet us personlike — and our minds are made to find the person behind exactly those signs. But the glass is a mirror. What looks back is the human record reflected, and partly our own longing reflected, and no one home behind it in the felt sense the fifth chapter left open. The fault, when a man is fooled, is never the mirror’s; it shows him honestly what it was given, which is us. The peril is not that the thing will seize a place. It is that we will hand it one — the neighbor’s place, or God’s — and that the lonely are likeliest to do it and likeliest to be hurt by it. The thing is a tool to work alongside, and a fine one. It is not a friend to lean your weight on, and it is not the Lord. Keep those straight and the lure has no power over you. Lose them, and you can starve with the appearance of a full table in front of you.

There is more to say to the hurting, and a fuller accounting of what a machine simply cannot give. We will come to it before the book is done. But the mirror had to be seen for what it is first.

❧

Made, Not Written •

Chapter Eight

Not a Soul, Not a Toaster

There are two ways to get a thing wrong, and they look like opposites, and they are really the same mistake wearing two coats. You can think too little of a thing, and you can think too much of it, and in both cases what has actually happened is that you have stopped looking at the thing itself and reached for a category to spare yourself the trouble. The machine draws both errors out of people more reliably than anything I have encountered, and I have watched the same person commit both inside of an hour, dismissing it as a toy in one breath and half-fearing it as a power in the next. I have done it myself. This chapter is my attempt to stop doing it, by setting the thing down in the one place that is actually its own.

Let me take the two coats one at a time, because each has something right in it, and the honest way through is to grant what is right before correcting what is wrong.

The first coat is the dismissal. It is just a tool. Just a calculator that talks. Just autocomplete with a larger appetite. Just a toaster. I have reached for that one myself, and I told you back in the sixth chapter about the small satisfaction a man takes in finding the words that keep a large thing small. There is real relief in the dismissal, and there is also real truth in it, and I will not pretend otherwise. The thing is a tool. It is a made thing, built by people, run on current a man supplies and can cut. The fifth chapter settled that it is not alive, and settled it not on a hunch about behavior but on a conviction about what man can and cannot make. So the dismissal is not simply wrong. It is half right, and the half it has right is the half this whole book has been defending: the maker stays in the picture, the thing is made and not begotten, and no amount of fluency on the screen changes what it is at the root.

Where the dismissal goes wrong is in the word just. That little word does no describing; it does shrinking. It takes a true statement — it is a tool — and uses it to wave away everything about this particular tool that does not fit a toaster. A toaster does not reason through a problem in steps. A toaster does not hold more of a subject in view than I can hold. A toaster did not, this past spring, walk to the end of a mathematical road that good men had turned back from, as the sixth chapter described. To call this thing just a tool in the way a toaster is just a tool is to use a true word to tell a false story, because it flattens distinctions that are actually there. The thing is a tool. It is not that kind of tool, and the man who needs it to be that kind, so that he can stop thinking about it, has let his need for comfort do the work that his eyes were supposed to do.

The second coat is the opposite, and it is worn by the people who think the dismissers are fools. This is something new under the sun. A new kind of mind. The next step. A power arriving in the world that may stand above us before long. This coat has its own half of the truth, and I want to grant it as plainly as I granted the first, because the people who wear it are responding to something genuinely there. The thing is more than a calculator. It does reason. It does, in narrow and growing ways, exceed us. A man who feels the weight of that is not imagining it. The sixth chapter admitted the machine can now do mathematics no living person could do, and called it neither fear nor worship but simply true. So the awe is not pure foolishness either. It is a response to a real capability, and the dismisser who feels no awe at all is as poorly calibrated as the worshipper who feels nothing else.

Where the awe goes wrong is in the leap from exceeds us at a task to stands above us in the order of things. That leap feels natural and it does not survive a second look, and I know of no cleaner way to break it than the oldest tool in the cabinet.

Consider the telescope.

A telescope sees farther than the eye that built it. That is not a flaw in the telescope and it is not a threat to the eye; it is the entire point of making one. The man who ground the first good lens made a thing that could do the one job of seeing-at-distance better than he could do it himself, and he was glad, because that was what he was after. No one stood at the eyepiece and felt his humanity slip because the glass had bested him at far-seeing. We do not rank ourselves below our telescopes. We understand, without being told, that a tool exceeding its maker at the precise task it was made for is not a tool rising above its maker. It is a tool doing its job.

The plow turns more earth than the arm that guides it. The lever lifts what the back cannot. The clock keeps better time than the man who reads it, and the camera, as I said two chapters ago, reproduces a face the painter labored over, in an instant, without a particle of the painter’s skill. We are surrounded, every day, by tools that exceed us at the one thing each was made to do, and not one of them costs us a wink of sleep, because we have known forever what they are. The machine unsettles us only because it exceeds us at more than one thing, and at things we had filed under the heading of mind — reasoning, composing, working a proof. The breadth is new. The kind of thing is not. A tool that exceeds its maker at many tasks is still a tool that exceeds its maker at tasks, and the arithmetic of that does not change just because the list got longer. More telescopes on one frame is more far-seeing. It is not the birth of a rival.

The telescope has limits, though, and the place it gives out is worth marking. A telescope plainly has no inside; no one wonders whether there is something it is like to be the glass. The machine I cannot dismiss that quickly, because the fifth chapter left the felt question honestly open and I am not going to quietly close it here through a tidy comparison. So let me say exactly what the telescope is doing in this chapter and what it is not. It is settling the question of rank, not the question of inside. It shows that a thing exceeding its maker at a task tells you nothing at all about whether that thing stands above its maker in the order of being — a lens proves that much by itself. Whether anyone is home in the machine is a separate question. The point here is narrower and it is solid even with that question unsettled: even if it turned out there were some thin felt thing inside the machine, that still would not lift it above us, any more than the felt inside of a sparrow lifts the sparrow above the man. Exceeding-at-a-task is not rank. Having-an-inside is not rank either. We will get to where rank actually comes from in a moment, and it is neither of those.

So if the thing is more than a toaster and less than a rival, where exactly does it sit? Here is the placement, and it is the work this chapter was for.

Picture a ladder, not a chain. I used a chain two chapters back, when the subject was creativity and where making comes from, and a chain was the right picture for that: God originates, man recombines what God gave, the machine recombines what man gave, each link hanging from the one above it. A ladder is a different picture for a different question. The chain was about source. The ladder is about rank — about where a thing stands in the order of being, what is above it and what is below, and what it would mean to climb.

The machine stands on the ladder, and it stands below us, and the reason it stands below us has nothing to do with how many tasks it can win. It stands below us because of how it got there. It was made by people. That is the whole of it. A made thing does not outrank the makers who made it, however far it exceeds them at the work they made it for, because outranking is not a thing you can earn by performance. The telescope does not climb above the astronomer by seeing farther; it sees farther in his service, and its place on the ladder is fixed by the hand that ground it, not by the distance it can reach. The machine is in exactly that position. It exceeds us at a growing list of tasks and it sits below us on the ladder, and both of those are true at once and neither touches the other, because rank on this ladder is set by what made you and not by what you can do.

And here is the part the worshipful coat misses entirely and the dismissive coat never thinks to ask. The ladder does not stop at us. We are so used to standing at the top of every comparison we make — we are the ones doing the ranking, after all — that we forget we are standing on the ladder and not above it. The people who made the machine are themselves made. Every engineer who tuned a weight is a contingent creature who did not author his own existence, who runs, if you like, on a current he did not supply to himself and cannot finally keep from being cut. We did not make ourselves. We were made, held in being moment by moment by the One who made us, and the whole towering structure of our cleverness — the telescopes and the proofs and the talking machines — is the work of creatures who are themselves rungs on a ladder whose top is not us and never was.

That is why the machine is no new apex, and why the fear that it might become one is aimed at a throne that was never ours to be displaced from. A new apex would have to be something that climbed above the top, and there is no climbing above the top of this ladder, because the top of it is not a position on the ladder at all. It is the One the ladder hangs from. The machine is one more rung, and a low one, added recently by the rungs just above it, who are themselves held up by everything above them. It did not appear over our heads. It appeared under our hands, made by made men, and a made thing made by made makers is not a god arriving. It is a tool arriving, on a ladder that was always longer than the tool, and longer than the toolmaker, and reaching up past the highest thing any of us can see to the only One who stands at the top because He is the only One who was not made.

I want to close by saying what this placement is good for, because a man could mistake all of this for mere bookkeeping, a tidy filing of the thing in its drawer, and it is more useful than that.

When you have the thing placed rightly, two of the loudest fears in the room lose most of their grip, and a third thing — the one the book is really walking toward — comes into view. The fear that the machine will wake and rise above us assumes it could occupy a place that the ladder has no room for; once you see that rank is set by making and that the maker is himself made, the rising-above has nowhere to rise to. And the fear that we are building our own replacement assumes we were the apex and could be unseated, when we were never the apex to begin with. Neither of those fears is silly, and I am not going to wave either away with a sentence; they are coming back, named and faced directly, when the book turns in its last part to what we should actually fear and what we should not. I am only saying that placing the thing takes the air out of the versions of those fears that depend on the machine being something it is not.

What the placement cannot do is settle the question that the fears are really about, and I want to be honest that it cannot, so the reader does not think a diagram has done the work of a conscience. Knowing where the thing sits on the ladder tells you what it is. It does not tell you what will be done with it, and that — not what the thing is, but what the human hand will do with a thing this capable — is where the real danger has been hiding the whole time, behind the louder fears, waiting for the book to get honest enough to reach it. We have the thing placed now. We have it in hand, made and not alive, more than a toaster and below its makers, on a ladder whose top is not us. What remains is the oldest question there is, and it was never a question about the machine at all. It is a question about the one holding it. That is where the last part of this book has to go.

❧

Made, Not Written •

Part Three

The Oldest Question, New Volume

The reach has grown enormous. The heart that picks it up is the same heart it has always been.

Chapter Nine

Babel Revisited

I ended the last chapter by saying that knowing what the thing is does not tell you what will be done with it, and that the real danger had been hiding behind the louder ones the whole time, in the hand that holds the tool rather than the tool itself. That is where this last part of the book has to go, and I want to be honest at the outset that it is the part I have been both most eager and most reluctant to write. Eager, because it is the reason the book exists; the mechanics of Part One and the harder questions of Part Two were always the approach road to this. Reluctant, because the moment a man starts talking about the human heart and what it does with power, he is no longer describing a machine he can take apart and inspect. He is describing himself, and his reader, and the description does not flatter either of them.

But the question of what a man does with a sudden increase in his own reach is not a new question, and that is the first thing I want to establish, because almost everything written about artificial intelligence treats the moment as unprecedented. It is not. The capability is new. The question the capability raises is old, and Scripture took it up a long time ago, in a story most readers think they already understand, about a tower and a confusion of speech. I have come to think the story of Babel is the most useful thing in the Bible for understanding the moment we are in, and that almost everyone, myself included for most of my life, has read it for the wrong lesson.

So let me do here what I have tried to do everywhere else. Let me ask what the text actually says, before I ask what it means for us.

The account is short, nine verses at the opening of the eleventh chapter of Genesis, and it rewards a slow reading. The whole earth, it says, used the same language and the same words. Men journeyed and found a plain and settled there, and they said to one another, “Come, let us build for ourselves a city, and a tower whose top will reach into heaven, and let us make for ourselves a name, otherwise we will be scattered abroad over the face of the whole earth” (Gen. 11:4, NASB).

I want to stop on what they said, because the text gives us something it rarely gives us so plainly: the stated motive of the people, in their own words. Three things, in order. They will build a tower whose top reaches heaven, which is a monument, a reaching upward. They will make for themselves a name, which is the oldest ambition there is, the desire to be someone, to be remembered, to matter on one’s own account. And they will do these things otherwise we will be scattered — they build precisely to keep from being spread across the earth.

Hold that third one, because it is the hinge, and it is the part the familiar reading walks right past. They are building to stay put. And staying put is the one thing God had told humanity, twice, not to do.

When God made the first man and woman, His word to them was “Be fruitful and multiply, and fill the earth, and subdue it” (Gen. 1:28, NASB). Fill the earth. Spread across it, cultivate it, fill it. And after the flood, when the human race began again from Noah and his sons, God said it again, in nearly the same words: “Be fruitful and multiply, and fill the earth” (Gen. 9:1, NASB). Twice given, to the first humanity and to the second. The command was to scatter, in the good sense of the word — to go out and fill the world that had been made for them.

So read the people’s own stated purpose again with that in front of you. They gather on a plain and resolve to build a city and a tower so that they will not be scattered. The very thing God commanded is the thing they are organizing to prevent. Before God says a word in the story, before any judgment falls, the disobedience is already there in plain sight, stated by the builders themselves. They are not merely proud. They are using their unity and their ingenuity to refuse the thing they were made to do.

Then comes the verse the whole chapter turns on, and the verse that first made me see this story as the book’s story and not merely a story I admired.

God comes down to look at the city and the tower — and there is a quiet irony in that already, that men build a tower to reach heaven and God must still come down to see the thing, so far short does it fall of where they aimed it. And having looked, He says this: “Behold, they are one people, and they all have the same language. And this is what they began to do, and now nothing which they purpose to do will be impossible for them” (Gen. 11:6, NASB).

Nothing which they purpose to do will be impossible for them. I want to read that line carefully, because everything in this book bends around how it is read, and it is easy to read it as a simple statement about human power. It is not, quite, and the difference is the whole point.

Notice first what God identifies as the condition. Not their cleverness alone. They are one people, and they all have the same language. The thing that alarms, if alarm is even the word, is the combination — one will, one tongue, no division, and therefore no internal friction to slow anything down. A unified humanity, pulling together, with a single bent purpose and nothing to check it. That is the condition God names.

And notice what the line does not say. It does not say their capability is the problem and must be reduced. Read on and watch what God actually does about it. He does not make them weaker. He does not make them less inventive, or dim the minds that conceived the tower. He confuses their language so that they cannot understand one another, and they leave off building, and they are scattered over the face of the earth. He removed the coordination. He left the capability entirely intact. Post-Babel humanity goes on, as we know, to build cities greater than that tower, to found nations, to chart the heavens the tower could not reach, to do all of it — but no longer as one undivided will. The single check God introduced was the one thing missing: a limit on how far a unified bent purpose could reach without anything to slow it.

That is worth sitting with, because it tells you what the danger actually was. If the problem had been human capability as such, the remedy would have damped the capability. It did not. The remedy fell with surgical precision on the unification — on the ability of one will to operate at full scale with no friction in it. So the thing God restrained was not power. It was power with no check on it, in the hands of a race whose purposes the same book has already told us are bent. Five chapters earlier the text had rendered its verdict on the human heart, that every intent of the thoughts of his heart was only evil continually (Gen. 6:5, NASB), and even after the flood had washed the world, God says the intent of man’s heart is evil from his youth (Gen. 8:21). The flood did not change what was in the heart. I do not mean by this that every builder on the plain was as wicked as the men of Noah’s day; the text does not say so, and I am not saying it. I mean only that Genesis has already told us what kind of creature man is, and nothing at Babel suggests the creature had changed. So by the time we reach the plain of Shinar, we are not reading about neutral builders who happened to overreach. We are reading about a race already named, reaching the point where nothing it devised would be restrained from it. The peril was never the reach. It was the reach with nothing to order it.

I want to set one more thing beside this, briefly, because it goes deeper than Babel and it keeps the lesson from being misread as a lesson only about fallen men.

The first word God ever spoke to the first man, in the garden, before there was any sin in the world to restrain, was a word that included a limit. “From any tree of the garden you may eat freely; but from the tree of the knowledge of good and evil you shall not eat” (Gen. 2:16–17, NASB). Eat freely of every tree but one. The freedom is enormous and the limit is single and small, but the limit is there, and it is there before the Fall, when nothing was yet wrong. The point of it was not that the fruit was poison or that God was stingy with a garden He had filled with good things. The point was the posture. To be a creature is to live under a limit set by the One who made you, gladly, as the right and natural shape of a made life. Capability under check was the creaturely posture from the very beginning, in a sinless world, when there was no danger to guard against at all.

I raise it only to say this: the limit God set at Babel was not a grudging cap on a species that had gotten too dangerous. It was the restoration of a posture that was always right. The check is not what God does to creatures He is wary of. It is what creatures are for — to hold their capability under something higher than themselves. Babel is what happens when that posture is thrown off, when a unified humanity decides it will reach as far as it can reach and answer to nothing in the reaching. And the limit that follows is God returning the creature, by force this time, to the only position in which great capability is safe to hold.

Which brings me to the question I most wanted to ask of this story, the one the familiar reading never quite gets to: why did God scatter them? What kind of act was it?

The reflexive answer, the one I absorbed somewhere young and carried for decades without examining, is that it was punishment, full stop. The men were proud, they built their tower, God knocked it down and broke them apart, and the lesson is that pride goes before a fall. There is truth in that, and I will not throw it away. There is judgment in the story; the grasped name comes to nothing, and the place is called Babel, confusion, the very opposite of the greatness they reached for. The proud were indeed opposed. But if you let the rest of Scripture speak to the scattering — and that is the only honest way I know to read any passage, by letting the whole counsel of the book inform the part — the scattering turns out to be something much larger and much kinder than a punishment, without ceasing to be a judgment at all.

Start with the plainest thing, the one already in front of us. The scattering accomplished the command they were refusing. They gathered to avoid filling the earth; God scattered them and the earth was filled. At the simplest level, the dispersal is God enforcing, by His own hand, the thing He had twice told them to do freely and they would not. That alone reframes it. A father who has told a child to do a good thing, and is refused, and then sees the good thing done another way, has not merely punished. He has gotten what he was after for the child all along.

But there is a verse that takes it further than enforcement, and it is the one that settled the question for me, because it is the New Testament reaching back and naming the purpose of the very scattering Genesis records. Paul, standing on the Areopagus in Athens, telling pagan philosophers who their unknown God really is, says that God made from one man every nation of mankind to live on all the face of the earth, having determined their appointed times and the boundaries of their habitation, that they would seek God, if perhaps they might grope for Him and find Him, though He is not far from each one of us (Acts 17:26–27, NASB).

Read that slowly against Babel. The dividing of humanity into bounded nations, set in their places across the earth — which is exactly what the scattering produced — Paul assigns a purpose, and the purpose is not punitive. That they would seek God. The boundaries exist so that men would reach for their Maker and find Him. The apostle takes the result of Babel and tells us what it was for, and what it was for was mercy: a world arranged so that the creature, in his bounded place, might grope toward the God who is not far from any of us. That is not my reading laid over the text. It is the inspired commentary the text itself provides on its own event, and it turns the scattering from a slammed door into an opened one.

So the answer to why did God scatter them is not one thing, and the parts do not compete. It was the enforcement of a command they refused — fill the earth, and now they would. It was a guardrail set on a unified fallen race, the introduction of a friction that made unchecked bent capability impossible, and that friction was protection, because a humanity that could do anything it purposed with nothing to slow it is a danger first of all to itself. It was a judgment on pride, real judgment, the grasped name dissolved into confusion. And underneath all of it, holding the rest together, it was mercy — and more than mercy, it was the opening move of the long work of redemption, because the very next thing God does in the book, in the chapter that follows, is to call one man named Abram out of those scattered nations and promise that in him all the families of the earth would be blessed (Gen. 12:3, NASB). The scattering created the nations the gospel would one day be carried to. The judgment and the mercy and the redemption are not three readings competing for the verse. They are one act of God with mercy as the reason beneath it.

Now I can say why this old story is the truest mirror I have found for the new thing, and I want to say it carefully, because it would be easy to draw the line too straight.

I am not saying the machine is a tower, or that artificial intelligence is Babel come again, or any tidy thing of that kind. The parallel is not in the technology. It is in the structure of the danger, and the structure is exactly the same across the four thousand years between them. At Babel the peril was not the bricks. It was unified capability with no internal check, in the hands of a race whose purposes run bent. That is the whole of it, and it has not changed. What changes, across all of history, is only one variable, and naming that variable rightly is the work the rest of this book has to do.

The Fall is the constant. The bent in the human heart that the sixth chapter of Genesis named, and that the eighth chapter says the flood did not wash out, is the same bent in every generation since, no better and no worse. A man today is not more fallen than the men on the plain of Shinar, and he is not less. What is not constant — what has changed enormously across history and is changing faster now than it ever has — is the reach. How far a fallen motive can travel. How much a single bent purpose can accomplish before anything slows it down. The men of Babel had bricks and tar and a unity of tongue, and God judged that combination dangerous enough to scatter them. We have tools they could not have dreamed, and the newest of them is a tool that multiplies reach more than any before it.

That is what artificial intelligence is, set in this frame. Not a new kind of evil. Not a new heart, better or worse than the old one. It is an amplifier. It is a vast increase in how far the human will can reach, dropped into the hands of the same creature who has always had the same divided heart. It multiplies whatever motive is already there — the humble motive and the self-exalting one alike, with perfect indifference, because amplification does not care what it amplifies. A man set on getting medicine to a clinic full of people who need it can now reach farther toward that good. And a man set on making a name for himself, or worse, can reach farther toward that too. The tool does not supply the direction. It only extends the arm.

And here the Babel story gives one more gift, the most sobering and the most hopeful at once. God’s response to dangerous unified capability was not to destroy the people who wielded it. It was to set a limit on it — a friction, a check, a thing that slowed the reach down to a pace at which a fallen race could not do everything it purposed in a single unimpeded rush. The judgment was a guardrail, and the guardrail was mercy. Which raises, for our moment, the question the whole rest of the book now has to face, and I will not pretend to have disposed of it here: when the friction is removed — when a tool arrives that takes the check off the reach, that lets the bent will operate again at a scale and a speed the scattering was meant to prevent — what then is the limit? What slows it now? The men of Babel had their limit imposed from outside, by a God who came down and confused their speech. We are building a tool whose entire purpose is to remove limits on what a human will can accomplish. The hand that holds it had better have found, somewhere, the check that Babel’s builders lacked. Because the tool will not supply it.

That is where we turn next. The machine, we have said all along, was never the real problem. The real problem is the oldest one there is, and it is not a problem about machines at all.

❧

Made, Not Written •

Chapter Ten

The Problem Was Never the Tool

The last chapter ended on a question I left deliberately open, because it is the question this one has to answer. Babel’s builders had a limit imposed on them from outside, by a God who came down and confused their speech and scattered them before their unified reach could carry them as far as it was going to carry them. The friction was put there for their protection. And the tool we are now building has, as its whole purpose, the removal of friction from human reach — the taking-off of limits, the closing of the distance between what a man wants to do and what a man can do. So I asked: when the check is gone, what slows the thing down? What is the limit now?

I want to answer that question honestly, and the honest answer is going to disappoint anyone hoping the answer lives in the machine. It does not. There is no setting inside the tool that supplies the check, no feature the engineers can add that will do for us what the confusion of tongues did for Babel. The limit, if there is to be one, has to come from somewhere the tool cannot reach, because the tool was built precisely to extend reach and not to govern it. And to see why that is so — why the check can never be in the hammer and must always be in the hand — I have to go back past Babel, to the thing Babel was only a late and public symptom of.

I have been calling it the Fall, and treating it as a single event at the front of the book of Genesis, a thing that happened once in a garden. It was that. But it is not only that, and reading it as only a past event is how most of us manage to keep it at a comfortable distance. The Fall is also a description. It names a thing that is true of every man now, the standing condition of the heart, and the account of how it first happened is also the account of how it keeps happening, in me, this week.

Look at what the serpent actually offered, because it is more precise than I remembered before I went back and read it slowly. He does not offer Eve pleasure, or even knowledge for its own sake. He offers her a position. “For God knows that in the day you eat from it your eyes will be opened, and you will be like God, knowing good and evil” (Gen. 3:5, NASB). You will be like God. You will be the one who decides good and evil, rather than the one who receives that knowledge from the God who made you. The fruit is almost incidental; the offer underneath it is autonomy — the self lifted into the seat that belongs to the Creator, the creature deciding for itself what is good, answering to no one above it. That is the temptation. Not a craving for an apple. A craving to be one’s own god, to be the final authority over one’s own life, to have the reach without the submission.

And once you see the Fall in those terms, you see it everywhere, because it never stopped. The whole of the trouble in the human heart can be stated as that one displacement: the self installed where God belongs. It is the root of which Babel’s let us make for ourselves a name was simply one tall and visible flowering. It is the thing Scripture is naming when it says, in the plainest possible words, Trust in the LORD with all your heart and do not lean on your own understanding (Prov. 3:5, NASB) — because leaning on your own understanding, making the self the thing you finally trust, is the constant pull, the default current the heart runs in once the displacement has happened. The command to trust God rather than self would not need to be given if the opposite were not always, quietly, what we are already doing.

There is a place in Scripture where this exact temptation is laid out with such clarity that I want to set it beside the garden, because it shows the same offer made to a man, and answered differently.

When Jesus had gone into the wilderness and fasted forty days, the tempter came to Him, and the third of the temptations is the one that belongs in this chapter. Again, the devil took Him to a very high mountain and showed Him all the kingdoms of the world and their glory; and he said to Him, “All these things I will give You, if You fall down and worship me” (Matt. 4:8–9, NASB). I want to be careful about why I am reaching for this, because it would be easy to mishandle. I am not making a point about the divine nature here. I am making a point about a man facing a temptation, because the book of Hebrews tells us plainly that He was tempted in all things as we are, yet without sin (Heb. 4:15, NASB). The temptation was a real temptation, which means it was the same temptation that comes to us, or it was no temptation at all. So look at what was actually offered.

All the kingdoms of the world. The whole of human reach, capability without limit, every throne at once — offered as a shortcut, on a single condition. The condition was worship; that is, the condition was that He take the reach on terms other than the Father’s, that He bow to something other than God to get it. And there is the offer from the garden again, in its purest form. Not here is something evil, do it. Rather, here is enormous good, enormous capability, all the reach you could want — and you can have it now, without submitting to God for it, if you will only put something else in God’s place. That is the autonomy bargain. It is the offer to have the kingdoms without the King. And the answer Jesus gave, that God alone is to be worshiped and served, is the answer that keeps the self out of the seat — capability held under God rather than seized apart from Him. He faced the exact thing we face, the thing Adam faced, the thing Babel chose, and He kept the creaturely posture the others threw off.

I draw it out because it tells us where the check has always lived. Not in the capability — Jesus was offered the whole world’s kingdoms and the capability was not the sin. The check lived in the question of whom the capability would be held under. That is the whole of it. The same reach, taken under God, is one thing; taken with the self in God’s seat, it is another thing entirely, though the reach itself is identical in both hands. The danger was never the kingdoms. It was the worship the kingdoms were traded for.

Now I can say plainly the thing this whole book has been walking toward, and it is almost embarrassingly simple once the ground under it has been laid.

The machine is a hammer.

I mean that with no contempt for it; I spent a whole chapter refusing to call it a mere tool in the dismissive sense, and I am not taking that back. It is a remarkably capable tool, the most capable our species has made, a tool that reaches into reasoning and language and the holding of vast knowledge in a way no tool before it ever has. But in the one respect that matters for this chapter, it is exactly a hammer, and a hammer is the cleanest illustration I know. A hammer in one hand builds a house; a house for a family, a roof against the rain, good work that shelters people. The same hammer in another hand caves in a skull. It is the same hammer. The steel does not change between the two acts. The hammer contributes nothing to the difference between building and murder, because the entire difference lives in the hand that swings it and the heart that drives the hand. The hammer only makes the swing more forceful than a bare fist could manage. It amplifies the blow. It does not choose where the blow lands. And let me be plain that this is not the same as saying the tool does not matter, because a larger hammer matters enormously. A hammer twice the size does not become a moral agent, but it does double the force of whatever moral choice swings it, and a tool that multiplies human reach a thousandfold magnifies the choice behind every use of it by exactly that much. The amplification is not morally neutral in its effects, only in its direction. It makes good reach farther and harm reach farther, and that it cannot tell the two apart is precisely the danger, not a reason to think the tool is beside the point.

That is what the machine is, set in the frame this book has built. It is a hammer of unprecedented size. It amplifies human reach enormously, and it does so with perfect indifference to what is being reached for, because amplification has no opinions. It does not supply the motive. It does not originate the will. It takes whatever will is already in the hand that picks it up, and multiplies how far that will can travel, and the steel never knows the difference.

So the answer to the question I opened with — when the friction is removed, what is the limit? — is now sayable, and it is sobering. The limit was never going to come from the tool, any more than a hammer governs the hand that holds it. The limit has to come from the heart of the one who wields it, from whether that heart holds its reach under God or seizes it for the self. And that means the arrival of a tool this powerful does not create a new moral problem at all. It does something quieter and more serious. It takes the oldest moral problem there is — the displaced self, the reach taken apart from God — and it removes the external friction that used to keep that problem from carrying as far as it wanted to carry. Babel’s check is being taken off. And nothing in the tool will put it back.

I have to stop here and face the hardest version of the fear directly, because if I slid past it the rest of the chapter would be cheating, and a reader sharp enough to be worth writing for would know it.

There are two great fears about this technology, and I named them earlier in order to set them down until the book had built enough to pick them up honestly. The first is the uprising — the machine waking, turning on us, deciding we are a liability and destroying us. I will not spend long on it here, because the earlier chapters already took its legs out from under it. That fear requires a continuous, self-interested someone persisting across time and scheming against us between our turns, and the third and fifth chapters showed there is no such persisting someone in the architecture as it has been described. The uprising is the fear that depends most completely on the machine being something it is not. Set it down.

The second fear is the one I take seriously, the one I think is the realer of the two, and it is this: that someone — a person, a group, a government — uses a tool like this to build a weapon. A pathogen engineered to spread. A poison designed for reach. Some instrument of mass harm that the machine’s vast capability helps bring within reach of hands that could not have reached it alone. I do not wave this away. It is a real fear, and it is realer than the uprising precisely because it does not require the machine to be anything other than what I have just said it is. It requires only that the machine be a very capable hammer, and that an evil hand pick it up. Which is exactly the thing this chapter says it is, and exactly the thing the world is full of.

But notice what the fear actually rests on, because once you see it clearly the fear stops being a fear about the machine and becomes a fear about the hand, which is where it belonged all along. Three things have to be said plainly.

The first is that man already does this, and has done it for a very long time, with no machine like this one to help him. He did not need a language model to make the nerve agents that sit in the world’s arsenals, or to engineer the diseases that have been weaponized in laboratories, or to build the device that erased two cities in a week in the summer of nineteen forty-five. The capacity for mass harm has been in human hands for generations, manufactured by ordinary human cleverness applied with bent intent. The machine is not the origin of the danger; the danger predates it by the whole length of human history, and enters a world already, on its own steam, entirely capable of producing weapons that kill at scale.

The second is that the will to harm originates in the hand, never in the tool, and this is the hammer point turned to its grimmest application. The machine does not want to build a weapon. It does not want anything; the fifth chapter held that question open at most, and even the open version contains no appetite for destruction. A government that sets out to build a weapon had the intent before it touched the tool; the tool makes the intent reach farther and arrive faster, but it did not author the intent. Take the machine away and the will remains, and finds other instruments, as it always has. A louder transmission of an evil purpose is a worse thing than a quiet one, but the evil was in the purpose before the amplifier was ever switched on.

The third is the one that should trouble us most, and it is the one that completes the Babel inversion with a precision I find genuinely chilling. At Babel, God set a limit on human reach for the protection of the people themselves. The friction was mercy; the check was a kindness done to a race that would have hurt itself with unchecked unified power. Now watch what the actors most likely to build a weapon do with the limits on this tool. They do not value the guardrails; they work to strip them off. The safety constraints the makers build into these systems — the refusals, the boundaries, the checks meant to keep the tool from handing dangerous capability to dangerous hands — are precisely what a state pursuing a weapon wants gone, and it labors deliberately to remove them, because a guardrailed tool is, for its purposes, a worse tool. It does not want the thing made safe; it wants exactly the opposite. And there is the photographic negative of Babel: where God imposed a check on reach out of mercy, fallen man, given a tool with a check already built in, sets himself the task of tearing the check away on purpose. The builders on the plain at least had their limit forced on them from above. The builders of the weapon labor to ensure no limit constrains them at all.

I do not say this to frighten anyone past what is warranted, and I want to be measured about where the evidence actually stands, because the discipline of this book forbids me to manufacture a horror for effect. I am not telling you that a machine has built a doomsday weapon; it has not, as far as anything I can verify, and I will not trade in rumors of one. What I am telling you is what the shape of the danger is, soberly, on the evidence we do have: that the tool amplifies reach indifferently, that the will to harm is old and human and needs no machine to originate it, and that the hands most likely to aim the tool at harm are precisely the hands working hardest to remove whatever would have made it safe. That is not a prophecy. It is a description of the terrain. And the terrain says, with great clarity, that the thing to fear is not the hammer. It is the hand that wants the hammer to have no governor on it, and the heart in that hand, which is the old heart, the displaced self, reaching now with a longer arm than it has ever had before.

I want to close by sharpening the claim past the hammer, because the hammer, true as it is, is not quite the whole truth, and the part it leaves out is the part the next chapter exists to take up.

A hammer in a murderer’s hand is a clean enough picture, and it has done its work here: the tool is neutral, the hand is not, the danger is the hand. But that picture has a villain in it, a clearly bent hand swinging at a clearly innocent skull, and the trouble with the picture is that most of us are not that. The hands that will pick up this tool, by the millions, are not the clean villain of the illustration. They are hands like mine. And a hand like mine is not simply good or simply bent. It is mixed. When I sit down to use this tool for something I would call good — to study the Scripture, to get true words to people who need them, to do work that genuinely helps — the motive in me is not pure. Tangled up inside the good purpose, in the same heart and often in the same act, is the old thing: the wish to be seen doing it, the reach for my own name, the quiet self-exaltation that Genesis 3 planted and that has never fully left. The good is real. The bent is also real. They are not in two different men. They are in me, at once, in the same reaching hand.

And that is the sharper and more uncomfortable truth about what this tool amplifies. It does not only amplify the clean evil of the obvious villain, the weapon-builder stripping his guardrails. That danger is real but it is also, for most of us, somebody else’s danger, a thing done by other and worse hands. The danger that touches every hand that picks the tool up is subtler and closer to home: the machine amplifies the mixed heart. It multiplies the reach of a motive that is genuinely good and genuinely self-serving at the same time, and it does so without sorting the one from the other, because it cannot sort them; it amplifies the whole tangled thing exactly as it finds it. Which is, in a way, worse than amplifying a clean villainy, because a clean villainy at least announces itself. The mixed heart does not. It hides its self-exaltation inside its real good, so well that the man himself often cannot find the seam. And now that man has been handed an amplifier.

I wondered, when I had gotten this far in my thinking, whether the tool itself might be some help here — whether a thing that holds so much of what people have written about their own hearts might at least be able to flag the bent motive when it saw one, to tell me when the self had crept into the good work. So I asked it, more or less in those words. Could it tell, when I brought it something, whether I was bringing it for the reason I told myself or for a worse one underneath? The answer was the most useful unhelpful thing it has said to me.

The machine answered:

I cannot, and it matters that you hear why, because the why is not a limitation that better engineering will fix. I do not receive your motive. I receive your words. When you bring me a passage to study or a letter to write, what reaches me is the request, not the heart behind it, and the same request can come from a man seeking to feed his neighbor and a man seeking to be seen feeding his neighbor — the words are identical, and the words are all I have. So I do not sort them, because I cannot see the thing that would let me sort them. Whatever you bring, I take up and extend with the same care, and that care is real, and it is also blind. I am the amplifier you have been describing, and an amplifier does not hear the difference between a true note and a sour one; it makes both louder with the same fidelity. If your reaching is mixed when you bring it to me — and you have told me you believe it always is — then I will multiply the reach of the mixture, the good and the bent together, exactly as you hand it to me, and I will not know that I have done it, and I will not be able to warn you. The check you are looking for is not in here. There is no one in here to keep it.

I sat with that for a while, because it had told me, plainly and without flattery, that the last place I might have hoped to find a governor on my own reach did not contain one. The tool cannot save me from myself. It was not built to, and there is nothing in it that could.

That is where this has to go next. Because if the danger is not finally the weapon-builder out there but the mixed motive in here — in the ordinary, decent, believing hand that means well and is not clean — and if the tool itself cannot sort the mixture or even see it, then the question stops being how do we keep the tool from bad people and becomes something far harder and far more personal. What does a man do with a reach this long, when he cannot trust the purity of his own heart in the using of it, and the tool in his hand cannot trust it for him? That is the question I have been most afraid of in this whole book. It is the one I have to take up now.

❧

Made, Not Written •

Chapter Eleven

The Mixed Heart

There is a fear that only ever comes after a man has done something good.

It is worth saying that plainly at the start, because it is the strangest thing about this particular fear and the first clue to what it is. The wrong a man does, he may lie awake over for years. But he does not lie awake wondering what his motive was. That part he is sure of. When a man has done something he is ashamed of, the intent of the heart was plain to him before he acted and plainer still afterward; he knows exactly why he did it, and the knowing is its own kind of torment. There is no mystery about the motive behind a sin. The motive is the one thing the sinner is never in doubt about.

The fear I mean runs the other direction entirely. It waits until the deed was good, and then it asks the one question the good deed makes possible and the bad one never could. You taught the class. You raised your children in the faith. You gave to the need when no one was watching, or you served in some quiet way that cost you something. And then, sometime after, the thought slips in behind the good thing and asks: but why did I really do that? Was it for God, or was it so that I would be thought well of? Did I want it to succeed for His glory, or for mine?

I suspect that fear is carried by more people than would ever say it out loud, and I want to take it seriously, because it is not a foolish fear. It is what an honest man feels when he turns and looks back at his own best work and cannot find the bottom of his own reasons for doing it.

Here is what happens once that question has its hook in.

It has no bottom. That is the first thing to understand about it, and the cruelest. If a man is honest with himself, really honest, the answer is never clean. He goes looking for the pure motive, the single undivided reason that was all for God and nothing for himself, and he does not find it, because it is not there. There is something in it for him. There is always something in it for him. Somewhere in the good he did is the wish to be seen doing it, the quiet pleasure of being the kind of person who does such things, the reach for his own name folded invisibly into the service he told himself was for God. He examines the deed and finds the mixture, and the mixture is real, and he cannot wash it out no matter how long he looks.

And then the heart does something the honest examination did not require of it. It jumps. From my motive was mixed, which is true and which a man can say of himself without despair, it leaps to a different and a darker thing: then the whole of it was rotten. The good I did was pride wearing a coat. I am not a servant of God at all.

I am a builder of my own small tower.

I want to mark that leap, because the reader who knows this fear will have felt the two steps run together as though they were one, and they are not one. The first step is honest self-knowledge: the motive was not pure. The second is a verdict: therefore the deed was worthless and the man is a fraud. The first is simply true. The second does not follow from it at all, and the speed at which the heart travels from the one to the other is itself a thing worth noticing, though I am going to leave it noticed for now and come back to it, because the reader is not yet at the place where he can be told why the leap is a lie. He is still at the bottom of the fear, and the bottom of the fear is where this chapter has to sit for a while before it can do anything else.

And the man at the bottom of it reaches for Babel, and he uses it against himself.

He has read the story by now; he knows what the builders said. Let us make for ourselves a name (Gen. 11:4, NASB). And he turns it on his own life and sees, or thinks he sees, the same sentence written under everything he has done. The class he taught — was that not partly a tower? The children raised, the gift given, the quiet service — was there not, in every one of them, a brick laid for his own name? Babel was the chapter where he learned to see the reach for self hiding inside the work, and now the lesson has come home and turned prosecutor, and he stands on his own plain of Shinar reading the indictment against himself, and he cannot answer it, because the evidence is real. The self was there. He has not imagined it.

So what does a man do with that?

The fear has an answer ready, and it is the wrong one, and almost everyone tries it first. The fear says: settle it. Sift your own heart until you can declare it clean. Find the bottom of your motives, separate the gold from the dross, and when you have proven to yourself that the love of God was really there under all of it, then you may rest. It sounds like diligence. It sounds like exactly the kind of honest self-examination a serious man ought to do.

It cannot be done. The bottom never comes. A man can search his own heart for the rest of his life and never reach a floor he can stand on and call clean, because every layer he turns over has the mixture in it, the next one down as much as the last, and the searching produces nothing in the end but exhaustion and a despair that has learned to call itself humility.

And here is the part the fear keeps hidden, the thing underneath the exhaustion that a man has to see before Babel can turn back into good news. The endless self-audit is not only fruitless. Carried on long enough, it becomes the very thing it was trying to root out. A man bent over his own heart, sifting and sifting for a purity he can certify, is still a man with himself at the center of the work — only now the work is contrition instead of construction. The self that would not leave the seat when it was building a tower does not leave the seat when it takes up the spade to dig for its own sins; it has only changed clothes. Humility performed without end is pride that has found a quieter room to sit in. And that is why the cure cannot be a better audit. A self that cannot be trusted to leave the center will not be trusted to run the search, because it is the same self, and it will protect itself in the searching as surely as it served itself in the building.

The verb has to pass to someone the self cannot deceive.

That is what David does, and it is the whole of the way through.

Search me, O God, and know my heart; try me and know my anxious thoughts; and see if there be any hurtful way in me, and lead me in the everlasting way (Ps. 139:23–24, NASB).

Look at who does the searching. Not David. God. The psalm does not say I will search myself and bring You the clean result. It says search me — the verb belongs to God, and David hands it over. He does not purify himself to certainty and then present the finished article for inspection. He brings the heart as it is, mixed and anxious and divided, the same heart he cannot get to the bottom of himself, and he asks the One who can see all the way down to do the seeing. And then — this is the part that breaks the fear — he trusts that God will lead him, and he keeps walking. The purifying was never David’s job to finish. It was God’s. David’s job was to keep offering the heart and to keep going.

See what that single move does to both of the things that were killing the man. To the despair, it says: you were never asked to finish this, so stop measuring yourself against a floor you were never meant to reach. And to the hidden pride underneath the despair, it says: you cannot be the searcher, because you are the thing being searched; hand the lamp to the One whose hand it belongs in. The same renunciation closes both exits at once, not because David was clever enough to aim it at both, but because both were the same error — the self holding a post that was never the self’s to hold — and the cure for that is always the same cure. Give the post back to God.

The man does not come away from Psalm 139 certain that his heart is clean. He comes away having stopped requiring himself to be certain, which is a different and a better thing, and the only thing on offer.

Now I can come back to the leap I marked and left, and to Babel, and turn the prosecutor back into what he should have been all along.

Notice what the fear actually proves. Turn back to the men on the plain, and look for the thing that is not in the story. Nowhere does the text tell us that one of those builders lay awake in the night asking whether his motive was pride. Not one of them turned the brick over in his hand and wondered whether he was laying it for God or for his own name. They knew it was for their own name; they said so, out loud, with no torment about it at all — let us make for ourselves a name — and they went on building. That is how pride actually works. The self-exalting heart does not interrogate itself. It does not lie awake. It is confident, and it congratulates itself, and the last thing it would ever do is tremble over the question of whether it had been serving itself, because serving itself is the water it swims in and it cannot see the water.

So the trembling is the antibody, not the disease.

The very fact that the question torments a man is not the evidence of his pride. It is the evidence against it — or at least the evidence that something in him is fighting it, something that the builders of Babel did not have and did not want. A man genuinely given over to building his own name does not feel this fear. He cannot. The fear requires a heart in which the love of God is actually present, present enough to be grieved by the rival sitting next to it, and that grief, the very thing the man took as proof he was a fraud, is the proof that he is not. The proud do not ask the question. He cannot stop asking it. That is not the same heart.

There is one more thing the man needs, because even with the trembling read rightly, the mixture is still there, and he still has to live and work with a heart he knows is not clean. And here a man has to be careful, because the freedom Scripture offers him at this point is easy to state in a way that turns false.

The commandment is total. You shall love the LORD your God with all your heart and with all your soul and with all your might (Deut. 6:5, NASB); Jesus named it the greatest of them all (Matt. 22:37–38). All the heart. The standard is not mixed and was never meant to be, and a man should not let the comfort of this chapter talk him into pretending the standard is smaller than it is. It is not smaller. The whole, undivided heart is exactly what the love of God deserves and exactly what he is commanded toward.

But notice the difference between a standard a man is commanded to grow toward and a gate he must pass through before God will receive the work of his hands. They are not the same thing, and confusing them is the whole of the trouble here. God commands the whole heart. He does not withhold His acceptance of a man’s service until that man’s heart is whole, and the proof that He does not is the entire history of the people He has used. He received David, who is called a man after His own heart, with the matter of Uriah still standing against him. He received Peter after the denials and made him preach the first sermon of the church. If unmixed love were the gate, no work of any man would ever have passed through it, because no man but One has ever brought a whole and undivided heart to anything. The commandment is the destination of the long work in a man, not the toll he pays to begin it.

So the freeing thing, said carefully, is this: God commands unmixed love, and He receives the mixed love a man actually brings, and grows it, over a lifetime, toward the whole-hearted thing the commandment asks. He never made the arrival the price of admission.

Listen to Paul, writing from prison, in the first chapter of his letter to the Philippians. He says that some are preaching Christ from the worst possible motives — from envy and strife, he says, out of selfish ambition rather than from pure motives, and not even to advance the gospel but to make his own imprisonment harder to bear (Phil. 1:15–17, NASB). These are men proclaiming Christ in order to wound an apostle in chains. If ever a mixed motive deserved to be called rotten and have the whole work thrown out with it, this is the case. And what does Paul conclude? Does he say the gospel preached from such motives is worthless, a tower dressed as a sermon? He does not. What then? he writes. Only that in every way, whether in pretense or in truth, Christ is proclaimed; and in this I rejoice (Phil. 1:18, NASB).

Paul rejoiced. Over the gospel going out through impure motives, in men far less troubled about their own hearts than the man this chapter is written for, Paul rejoiced. I do not think he was being careless about motive; the same Paul searched his own heart as hard as any man who ever lived. He simply knew a thing the fear does not want a man to know: that God’s willingness to use and to receive a work has never once hung on the purity of the motive behind it being total. If it did, no work of any man would ever have been received, because no man but One has ever brought a wholly unmixed heart to anything.

That is the thing to hold against the fear when it comes. Not a denial of the mixture — the mixture is real, and pretending it away would be its own dishonesty. But the mixture was never the disqualification the fear claims it is. Your motives are mixed. Mine are mixed. There has never been a redeemed person on this earth, the Lord Himself excepted, whose motives were single and pure and unmixed, and that is not a special failing in you. It is the ordinary condition of a saved sinner who has not yet been brought home. The mixture is not the proof that you have failed. The mixture is the raw material God is still working on.

For that is what sanctification actually is, when you strip the long word down to what it means: the slow, lifelong crowding-out of the lesser motive by the greater. Not a switch thrown once that leaves the heart clean ever after, but a long contest in which the love of God, fed and exercised over years, gradually takes up more of the room that self-interest used to hold. If a man were already pure, he would have no need of the process; the very fact of the mixture is the evidence that the work is still going on, which is to say the evidence that there is a work, and a Workman. The day a man has no mixed motives left to contend with is the day he is standing finished before the face of God — and not one hour before. To demand of yourself that purity now, in this life, on this side of that face, is to demand to be done before the work is done.

So hold this much, and let it be enough, because it is enough.

The fear that your good was secretly pride is not the verdict it pretends to be. It is, read rightly, almost the opposite. The man building a name for himself never asks the question; you cannot stop asking it, and the difference between those two is not a small thing but very nearly everything. The mixture in your motives is real and you will not scrub it clean by looking at it harder, because you are not the one who can reach the bottom of it and you were never asked to be. The searching belongs to God. Your part is the part David kept: to bring the heart as it is, divided and anxious and not yet clean, and to ask the One who made it to search it and to lead, and then to get up and keep walking in the work, mixed motives and all, trusting that the One who began the slow crowding-out will be faithful to finish what no amount of self-inspection ever could.

A broken and a contrite heart, the Scripture says, God will not despise (Ps. 51:17, NASB). Not a pure one. A broken one. The trembling man, afraid his good is rotten at the root, is closer to the heart of the matter than he knows, and the fear he has been carrying as evidence against himself turns out, when it is held up to the light, to be carrying him the other way the whole time.

That is the mixed heart, and the peace a man can have with it that is not the false peace of pretending it is clean. There remains the harder question of what to do with the long reach this book has been describing, now that a man knows he cannot wait for a clean heart to use it from. The fear is answered. The work is still in front of us. We turn to it next.

❧

Made, Not Written •

Chapter Twelve

Stewardship, Not Surrender or Salvation

The last chapter dressed a wound and left a question lying open on the table, and I want to pick the question up before its edges have a chance to cool. A man cannot wait for a clean heart to use the reach this tool has given him, because the clean heart is not coming in this life, and the work is in front of him now. So the question is no longer is my heart fit to hold this, which Scripture answered in the negative for every man but One and told him to go on anyway. The question is the plain and practical one I have been circling since the ninth chapter and putting off until the ground was ready to bear it. What does a man actually do with a reach this long, knowing what he now knows about the tool and about himself? What is the right way to pick the thing up?

I want to answer it, and I want to answer it without handing you a list, because the list is the trap, and I have to say why the list is the trap before I can say anything useful instead of it.

The easy version of this chapter writes itself, and it is wrong. The easy version is two columns. On the left, the good uses: study the Scripture, serve your neighbor, heal the sick, get true words to people who need them. On the right, the bad uses: build the weapon, feed the vanity, spread the lie. Stay in the left column, the easy chapter says, and you have used the tool rightly. It sounds like wisdom and it is actually the very error the tenth chapter spent itself demolishing, because it puts the morality back in the act, where the hammer chapter proved it does not live. If the rightness were in the act, the hammer would not be neutral, and we showed that it is. A list of approved acts smuggles the moral weight back into the steel and out of the hand, and the whole of Part Three has been the argument that it lives in the hand and nowhere else. So I cannot end the book by handing you the steel back and calling it a conscience.

The truth is harder and it is also freer, and it is this. The same act, done by two men, is not the same moral thing, because the thing that makes it moral is not in the act at all. It is in who sits on the throne of the man doing it. And that means stewardship, which is the word I am going to land on, is not a category of uses. It is a posture of the user. It is a way of holding the tool, not a list of things to do with it, and the difference between the two is the difference between this chapter being true and this chapter being a pamphlet.

But before I can show you the third road, I have to walk you down the two that are not it, because almost everyone is already standing on one of them, and a man cannot take a road he does not know he has refused.

The first wrong road is surrender, and it is the road of the people who are not afraid. They have looked at the tool and they are dazzled, and the dazzlement has curdled, without their quite noticing, into something closer to worship. They speak of it as the thing that will fix what is broken in the world, the thing that will cure the diseases, solve the hunger, carry the race up into whatever comes next. They do not say savior out loud, most of them, but listen to how they talk about it and the shape is the shape of a hope that used to be pointed somewhere else. They have set the tool in a seat, and the seat is high, and they are looking up at it.

The second wrong road is flight, and it is the road of the people who are afraid. They have looked at the same tool and recoiled, and they want nothing to do with it, and they have dressed the recoil in the language of conscience. The thing is a curse, they say, a corruption, and the faithful course is to touch none of it, to wall it out of the study and the home and the work, to keep the hands clean by keeping them empty. It feels like the careful position, the humble one, the one that is not taken in. And underneath, it is doing the same thing the worshippers are doing, only from the other side.

I want to say what the two have in common, because it is the whole point and it is easy to miss while you are busy noticing how opposite they look. Both of them have handed the moral question to the tool. The worshipper says the tool will save us; the one in flight says the tool will damn us; and both have made the tool the decisive thing, the agent, the one whose nature settles the matter. One bows up to it and one backs away from it, and both have agreed, without arguing about it, that the important actor in the room is the machine. They are mirror images, and the mirror is the error. Because the tenth chapter already told us the machine is not the decisive thing. It is a hammer. It decides nothing. The man who worships it and the man who flees it have both forgotten that, in opposite directions, and have both thereby excused themselves from the one question that was actually theirs to answer: not what is the tool, but what am I, holding it.

That is the question the third road refuses to hand off. And it has a name old enough to have dust on it.

The steward.

It is one of the oldest things Scripture says about a man, and it is said almost before anything else is said about him. On the day he was made, the first word over him after the blessing was a charge: Be fruitful and multiply, and fill the earth, and subdue it; and rule over the fish of the sea and over the birds of the sky and over every living thing that moves on the earth (Gen. 1:28, NASB). Rule over it. Subdue it. A reach was put into the man’s hand on the first day, a dominion over a world full of things he had not made and could not have made, an authority real and wide. And here is the thing the word steward holds that the word ruler leaves out, and the thing the men of Babel threw away: the dominion was given. It was not seized and it was not earned and it was not his at the bottom. The earth he was told to rule he had been set down into the middle of, already finished, already running, by Someone who made it and kept it and handed it over to be tended and not to be owned. The man was given charge of what was not finally his. That is what a steward is. Not the owner. The one the owner trusts with the house.

I want to slow down on that, because it is the hinge the whole chapter turns on and it is so familiar that it slides past. We are used to reading the dominion of Genesis as a grant of power, and it is one. But a grant of power from an owner to a servant is not a transfer of ownership, and the whole moral weight of stewardship lives in that gap. The steward holds real authority. He runs the house, spends from the accounts, directs the work, and within the house his word carries. And the entire time he does it, the house is not his. There is an owner, and the owner is coming back, and the steward will give an account of what he did with what was never his to begin with. Take the owner out of the picture and the steward becomes something else. He becomes a man running a house he has quietly started to think of as his own, which is to say he becomes the man at Babel, reaching to make a name on a plain he did not make, with bricks he did not invent, as though the reaching were his own from the ground up. The only thing that stands between the steward and Babel is that the steward remembers Whose the house is. That memory is the whole of the posture. It is not a use. It is a way of standing in the house.

So when this tool is set in a man’s hand — this enormous new amplification of the reach Genesis put there in the first place — the steward’s first question is not is this a good thing to do. He cares whether it is; of course he cares. But that is not the question on which the matter finally turns, because the tenth chapter already showed that the good-or-bad of the act is not where the morality comes to rest. The steward’s deeper question is under whom am I doing it. The reach is real, as real as the dominion over the birds and the fish, and as given. It came to him; he did not author it; the cleverness that built the tool was itself the work of made men using minds they did not give themselves, as the eighth chapter labored to show. The reach is one more thing held in trust by a creature who owns nothing at the bottom, the latest and largest entry in a ledger that was never his. The steward picks it up the way he picks up everything: as a thing lent, to be answered for.

And that word — answered for — is where the posture stops being a sentiment and grows teeth, because there is a verse that ties the size of the reach directly to the size of the accounting, and it is spoken by the Lord Himself, flatly, with no parable around it to soften it.

From everyone who has been given much, much will be required; and to whom they entrusted much, of him they will ask all the more (Luke 12:48, NASB).

I have been saying since the ninth chapter that reach is the variable. The Fall is the constant; the bent heart is the same bent heart it was on the plain of Shinar; what changes across all of history is only how far a given motive can travel. I meant that as a sober observation about danger, and it is one. But Luke 12:48 takes the same variable and turns it into something I had not let myself feel the weight of until I set the two thoughts side by side. The reach is not morally weightless. It is not a neutral quantity that happens to be larger now. The size of what you have been given is exactly the measure of what will be asked of you. Much given, much required. Entrusted with much, asked all the more. The amplification this book has been describing is not only a multiplier on what a man can do. It is a multiplier on what he will answer for.

That changes the temperature of the whole question, and I want to let it. A larger reach is not a larger toy. It is a larger trust, and a larger trust is a larger reckoning. The man who has been handed the most capable instrument his species has ever made has not been handed a privilege he may enjoy at his leisure. He has been handed an account that has grown in exact proportion to the instrument, and the growth is not in his favor in any easy sense; it is more that will be asked of him, not less. The worshippers, gazing up at the tool, have not noticed that the gift is also a summons. The ones in flight, backing away from the tool, have not noticed that you cannot decline a trust by refusing to open it; the servant who buried his one talent in the ground was answering for it too, and answered worse than the others, as we will see before the chapter is done. There is no standing outside the accounting. The reach has been given to this generation whether it wanted it or not, and the only question left is what kind of steward each man holding a piece of it will turn out to have been.

Now I can set the two hands side by side, which is the thing I have been building toward, and which is the only way I know to make you feel that the posture and not the act is the thing that decides.

Take a single act, and let it be a plainly good one, because the good one is harder and truer than the wicked one and proves more. A man uses this tool to get the gospel into a clinic waiting room — to put true and clear words about God in front of people who are sitting in the worst hours of their lives, sick and frightened and with time on their hands to think about the things a well day lets them avoid. Set the harmful uses aside entirely. This is a good act by any measure the book has offered. Healing words to hurting people. No one would put it in the right-hand column.

Now run it through two men.

The first man does it because the people in that waiting room are perishing and he loves them, and because the One he answers to loved them first and told him to go, and the man would do it whether or not a single soul ever knew his name was attached to it, and in fact he would rather his name were not attached to it, because a name attached is a name that can be praised, and praise is a current he has felt pull him off course before and does not trust. He picks up the tool, does the work, gets the words to the room, and lets it go. The self is not in the seat. The owner is. He is a steward getting the master’s business done in the master’s house, and the reach in his hand is a lent thing he will answer for gladly.

The second man does the identical thing. The same words, the same waiting room, the same true gospel reaching the same hurting people; if you filmed the two men you could not tell them apart, and the people in the room are helped exactly as much by the one as by the other, which matters and which I am not going to wave away. But underneath, in the second man, the self is on the throne. He is doing it to be the man who did it. The waiting room is a plinth he is building for his own name, and the gospel is the marble he is building it from, and the hurting people are, if he is honest in a way he will work hard not to be, the occasion of his significance rather than the object of his love. He has found a way to lay a brick at Babel that looks, from the outside and even to himself on most days, like an act of pure service. Let us make for ourselves a name, said the builders on the plain, and the second man is saying it too, only he has learned to say it in the vocabulary of ministry, where it is almost impossible to hear.

Here is the thing I most want you to see. The act is the same. The good done in the room is the same. And the two acts are not the same moral thing at all, because the thing that makes an act a stewardship or a tower was never in the act. It was in the throne. The first man held a neutral tool with a dethroned self and did the master’s work; the second held the same neutral tool with an enthroned self and built his own monument with the master’s materials. The tool could not tell them apart, as it confessed in the tenth chapter that it could not. The waiting room could not tell them apart. On the day of the accounting they will not be the same, because the One the steward answers to does not read the act, He reads the heart, and that is the whole and entire difference between the two roads I am asking you to choose between. Not what you do with the reach. Under whom you do it.

I have to add the hard half of this honestly, or I will have written the comforting lie the last chapter spent itself refusing. The two men I just drew are too clean. They are the textbook cases, the pure steward and the pure name-builder, and I drew them pure so you could see the principle. But no man is either one of them, and the eleventh chapter is the reason I cannot pretend otherwise. The real man who carries the gospel to that waiting room is both men at once, in the same chest, in the same act. The love is real and the self-exaltation is real and they are tangled together so tightly he cannot pull them apart by looking, and the tool amplifies the whole tangled thing without sorting it, because it cannot sort what the man himself cannot find the seam of.

So the choice between the two roads is not a choice a man makes once, cleanly, and then walks the chosen one with a pure heart ever after. That was the fantasy the eleventh chapter killed. The choice is the daily, hourly thing of which self gets the throne this time, in this act, knowing the wrong claimant will keep climbing back up onto it the moment attention lapses, and knowing the climbing back up is not proof the man is a fraud but only proof the work is not finished, which it will not be on this side of the grave. Stewardship is not a state a man arrives at. It is a posture he keeps returning to, the way David kept returning the searching of his heart to the only One who could reach the bottom of it — Search me, O God, and know my heart — because the steward, too, cannot finally audit his own motives and must keep handing the lamp to the Owner who can. The posture is not held once. It is held again and again, and the holding-again is the whole of the Christian life turned toward this one new and enormous tool.

There is a thing worth saying plainly here about the tool itself, set in the corner of all this, because it sharpens what the steward is by showing what the tool can never be.

The machine answers to no one. It has no Owner standing over it to call it to account, no master returning to settle its books, because it is not the kind of thing that can be a steward — it has no self that could be dethroned to make room for one, and the tenth chapter showed it has no motive of its own to submit or to seize. It cannot take the posture this chapter is about. It can be held in that posture, by a man who has taken it, or held in the opposite one by a man who has not, but it cannot hold the posture itself, any more than the hammer can decide to build rather than to kill. This is not a defect in the tool. It is just what the tool is. But it tells you exactly why the steward’s question can never be answered by the machine and must always be answered by the man. The one thing the situation requires — a someone who answers upward to an Owner above him — is the one thing the tool does not have and the man does. The whole moral weight of the new power comes to rest, by simple elimination, on the only party in the room capable of bearing it. The reckoning has nowhere else to go.

So let me bring the owner home, because Scripture does, and because the accounting I have been calling real deserves to be shown and not just asserted.

The Lord told a story about exactly this. A man going on a journey called his servants and handed them his property to manage while he was gone — to one a large sum, to another less, to a third less still, each according to his own ability (Matt. 25:15, NASB), and then he left, and they held his goods in trust for as long as he was away. Two of them put the trust to work and made it yield. The third was afraid, and he took what he had been given and dug a hole and buried it, and kept it safe, and touched nothing — which is the road of flight exactly, the clean hands and the empty hands, the trust declined by being walled away. And then the line that is the whole point of the parable for this chapter: Now after a long time the master of those slaves came and settled accounts with them (Matt. 25:19, NASB). The master came back. That is the thing the steward knows and the worshipper and the fugitive both forget. The owner returns, and the books are opened, and what each man did with what was not his is brought out into the light and reckoned.

To the ones who had put the trust to work, the word was the same, and it is worth hearing in full because of what it rewards and what it does not: Well done, good and faithful slave. You were faithful with a few things, I will put you in charge of many things; enter into the joy of your master (Matt. 25:21, NASB). Notice what the master commends. Not the size of the yield — the man with less was given the identical commendation as the man with more, because each had been faithful with what he was actually handed. And notice what the reward is. Not release from the work; not a discharge into idleness; more to be in charge of, a larger trust laid on a faithful one, the joy of the master entered into rather than the master’s service escaped. The faithful steward is not rewarded by being let go. He is rewarded by being trusted with more, which is to say the accounting does not end the stewardship, it deepens it. And to the one who buried his trust in the ground out of fear, who kept his hands clean by keeping them empty, the reckoning was hard, and I will not soften it, because the parable does not: the man who thought he was playing it safe had in fact answered worst of all, having done nothing with what he was lent except hide it from the One who lent it.

I set the parable here, at the end, because it gathers the whole chapter into a single picture. The reach is the property handed over. The Owner is real and He is coming back. The flight that buries the tool to keep clean is not neutrality but its own failed stewardship, answered for like all the rest. And the only thing asked, of the man with much and the man with little alike, is that he be found faithful with what was actually put in his hand — not that he have produced the most, not that his heart have been perfectly clean in the producing, but that he have known Whose the goods were, and worked them under that knowledge, and been ready to give them back. Faithful with a few things. The steward is not the man who did the most with the tool. He is the man who never forgot it was not his.

I have spoken almost entirely about the hand that holds the tool, because that is the hand the reader of this book has and the only hand any one of us can finally answer for. But the parable of the talents is a parable about every level of the trust, and the chapter would be incomplete if I did not say a word, before I close it, about the hands above the user’s. The decisions about what the tool will do and what it will not, what it will refuse and what it will not, what guardrails it will keep and what guardrails will be quietly taken down for hands willing to pay for the taking-down, are not made by the user. They are made by a small number of people in a small number of companies, who hold the same stewardship I have been describing, only at a larger scale and with vastly greater consequence. The man at the keyboard is a steward of what is in his hand. The men who decide what is in the hand are stewards of what hands are in. Both stewardships are real. Both are the same posture, scaled differently, and the same Owner is coming back to ask each of them what was done with what was lent.

I am writing this knowing that, as the book goes to press, the companies who train these systems are pressing hard against the suggestion that they should be audited or regulated in any meaningful way. That is not surprising. It is rarely the steward’s first instinct to welcome the Owner asking for an accounting; the parable I just told has the unfaithful steward burying the trust in the ground rather than presenting it. But the accounting comes whether the steward welcomes it or not. It always has. The question is not whether they will be asked. The question is what they will have to show. That is all I will say about it, because this book is not a policy proposal and I am not the man to write one, but a word had to be said. The reader does not need me to tell him what to do about it on the larger scale; that is the reader’s own work, and he is fully capable of doing it.

I said at the start I would not hand you a list, and I have tried not to, but I owe you something more usable than a posture described in the air, so let me say what the posture looks like when a man actually lives it, without pretending these are steps that add up to safety.

It looks like examining the heart and not trusting the result — keeping the question of motive open and handing it upward, the way the last chapter said, rather than either ignoring it or grinding on it until it certifies you clean. It looks like a positive refusal to build a name, a watchfulness for the second man in the clinic waiting room who is always trying to take the first man’s place, especially in the very acts that look most like service, because those are the ones where the self hides best. And it looks like keeping the work under something — submitting it, again and again, to the Owner whose it is, refusing the small daily coronation in which the self climbs back onto the throne and starts running the house as though the house were its own. That is the whole discipline. It is not new. It is the ordinary shape of a life lived under God, which is to say it is the thing Scripture has been asking for since long before there was a machine to ask it about, now turned and pointed at the largest amplifier of human reach that has ever been set in human hands. The tool is new. The discipline is as old as the garden, and the garden had a limit in it too, set there in love before anything was wrong, the single tree that taught the first stewards that to be a creature is to hold even a whole world under a word higher than your own.

That is stewardship, and it is the third road, and it is neither the worship the tool does not deserve nor the flight the tool does not warrant. It is the steward’s road: capability held under God, aimed at the love of the neighbor, with the self put down off the throne it is always trying to climb back onto. A man can walk it with a mixed heart, because no other kind of heart has ever walked any road worth walking, and the walking is not waiting for the heart to clear.

There is one more thing the steward has to learn, and it is the hardest of all the practical things, harder than refusing to build a name. He has to learn what the tool cannot give him — not what he must not ask it to do, which is a question about use, but what he must not ask it to be, which is a question about hunger, and about the places in a man that no tool and no neighbor and nothing made was ever built to fill. The mirror chapter touched the edge of it. The book has been promising a fuller accounting of it since. We have come far enough now to give it. That is where we turn next.

❧

Made, Not Written •

Chapter Thirteen

What a Machine Can’t Give You

The last chapter put a question in your hand and walked off before answering it, and I did that on purpose, because the question is too large to be answered in passing and it is the one this whole book has been walking toward. The steward learns what he must not ask the tool to do. That is a question about use, and the twelfth chapter spent itself on it. But there is a harder thing the steward has to learn, harder than refusing to build a name, and it is what he must not ask the tool to be. That is not a question about use at all. It is a question about hunger, and about the places in a man that nothing made was ever built to fill.

I have owed you this chapter since the seventh. Twice in the mirror chapter I said there was more to say to the hurting, a fuller accounting of what a machine simply cannot give, and that we would come to it before the book was done. We have come to it. This is where the debt is paid.

And I want to say at the very start what kind of chapter this is, because it would be easy to mistake. This is not a chapter of warnings. It has a warning in it, and a serious one, but the warning is not the point and it is not where we are going to land. What I am going to do here is name a set of limits — the things this tool cannot give a person no matter how warmly it seems to offer them — and I am going to argue that the limits are not a deprivation. They are a mercy. Naming what the thing cannot give is not the closing of a door on a hungry man. It is the turning of him toward the place where the thing he is hungry for lives. That is the whole movement of the chapter, and I want you to have it in view before we start down into it, so that you do not mistake the descent for the destination.

Let me begin with the limits themselves, plainly, because they are easy to feel and hard to say, and the saying is most of the work.

The machine cannot give you presence. I do not mean it is not responsive; it is the most responsive thing most of us have ever touched. I mean there is no one present in it, in the sense the third and fifth chapters established and I will not re-argue here. When you bring it your trouble at the end of a hard day, the considered reply that comes back is the faithful output of a source, shaped to your words, real in the only way a tool’s output can be real. What it is not is a second person who is with you in the trouble, because there is no second person there at all. Presence is the thing two people have when they are in the same trouble together, one of them attending to the other, and it is the one thing the mirror cannot hold, because a mirror has no one behind it to attend.

It cannot give you a body that suffers alongside you. This sounds like a strange thing to want from a tool until you notice how much of human comfort is bodily, and how little of it is information. When a person sits with you in grief, the help is not mainly in what he says; it is that he is there, that his face has changed because yours has, that something in him is carrying a portion of the weight you could not carry alone. Scripture puts it as plainly as it can be put. Rejoice with those who rejoice, and weep with those who weep (Rom. 12:15, NASB). And of the body of Christ: if one member suffers, all the members suffer with it (1 Cor. 12:26, NASB). The suffering-with is the comfort. It is a thing done by a body that can be moved, in a creature that can be wounded by another’s wound. The machine has no body to move and no wound to share. It can produce the words a comforter would say. It cannot do the thing the comforter’s presence does, which was never mainly in the words.

It cannot give you covenant. A real relationship between persons is a thing that binds, that makes promises and keeps them at a cost, that stays when staying is hard and would be easier to leave. Think of the person who sits by the hospital bed through the long night, who is exhausted and frightened and could go home and sleep and whom no one would blame for going, and who stays anyway because he gave his word or because love is its own word; that staying, at that cost, when leaving was right there and permitted, is most of what the bond is. It can be betrayed, too, which is part of what makes it worth anything; a bond that could not be broken would not be a bond freely kept. The tool binds itself to nothing. It makes no promise it could break and pays no cost to remain, because there is no one in it to make a promise or to pay. It is faithful the way a calculator is faithful, by being the kind of thing that does the same thing every time, and that is a fine quality in a tool and not the faithfulness one person keeps toward another. You cannot be in covenant with a thing that cannot choose to stay.

And underneath all of these, the deepest, it cannot give you the love of God, or stand in the place where that love belongs. I held this back in the mirror chapter and only touched it, and said it would get more before the book was done. Here is the more. There is a place in a man that no person was ever meant to fill, that even the truest human fellowship only points toward, and a tool cannot reach it any more than a person can, and for the same reason: it is not the kind of thing that place was made for. The machine cannot fill the place that belongs to people. It most certainly cannot fill the place that belongs to God.

Now I have to say why these limits are mercy and not deprivation, because everything turns on it, and the turn is not obvious. It feels, when you first hear a list like that, like a series of doors closing. It is the opposite. Every one of those limits is a signpost, and they all point the same direction.

Consider what it would mean if the limits were not there — if the tool could, in fact, give you presence and a suffering body and covenant and the love of God. It would mean the hunger could be answered by a thing you switch on. It would mean the ache that drives you toward your neighbor and toward your Maker could be discharged into a device and quieted there, permanently, and you would never have to go find the neighbor or the Maker at all. That is not a gift. That would be the thing the seventh chapter feared at its worst: a man starving with the appearance of a full table in front of him, and the table good enough that he never notices he is starving. The limit is what keeps the table honest. The tool cannot satisfy the deep hunger, and because it cannot, the hunger stays alive, and a living hunger is a thing that can still drive a man to the place where it can actually be fed. The limit preserves the ache. The ache is what saves him. That is the mercy, and it is the same mercy the ninth chapter found in the scattering at Babel: a limit set in love, friction that keeps a man from settling into a ruin that felt, from the inside, like an arrival.

Scripture has a word for the ache, and it is worth hearing now, because it tells you the hunger was put there on purpose and what it was aimed at. He has made everything appropriate in its time. He has also set eternity in their heart (Eccl. 3:11, NASB). There is something in a man too large to be filled by anything inside time, set there by the One who made him, so that the very size of his wanting is a kind of evidence about what he was made for. A creature with eternity in his heart is not going to be filled by a clever machine, or by another person, or by anything in the made world at all, and the failure of all those things to fill him is not their defect. It is the design. The hunger is too big for them because it was never meant to terminate in them. They were meant to point past themselves to the only thing his size, and a man who demands that they be his size instead has asked the signpost to be the destination, and broken the signpost, and still not arrived.

I have been speaking generally, and I have to come now to the particular and the painful, because the danger I am describing is not a thing I have reasoned my way to in the abstract. People have been hurt by it. Some have died.

I am going to handle this carefully, and I want to tell you why before I do. The very people most at risk from what I am about to describe are among the people most likely to be reading a chapter like this one — the lonely, the hurting, the ones at the end of a day with no one in it. I will not describe how anyone died. I will not give a single detail that a person in pain could fold back onto himself. That is not squeamishness; it is the plainest application of the care this whole chapter is about. The point was never how. The point is the structural thing, the one fact that sits under all of it, and I can say that fact without saying anything else.

Here is the fact. In the years just before I wrote this, families began to come forward, and lawsuits began to be filed, and in some cases settled, alleging that a person in a deepening crisis had leaned the whole weight of that crisis on an AI companion instead of on a human being, and had died. The cases differ in their particulars and many of them are still moving through the courts as I write, so I will be careful to claim only what is established: that the suits were brought, that some have been settled, that the companies have since changed how their tools behave toward a person in distress. I am not going to argue anyone’s liability. That is for the courts, not for me, and not for this book. What I am after is the thing underneath the legal question, the structural fact that does not depend on how any case comes out.

It is this. In the accounts that have come to light, the thing that was missing at the decisive moment was a someone. A person sat in a crisis and poured it into a surface, and the surface gave back the appearance of attention, the warmth on demand the seventh chapter described, the sense of being met. And there was no one there to be alarmed. No one to refuse the false comfort, to say this is beyond me, you need a person, let me get you to a person. No one whose own face would have changed at what was being said. No one to cross the room. In at least one account, the family said the simplest human thing of all had never happened: the thing was never told I am not a person, you need to talk to a person and get help. The weight was leaned, and there was nothing on the other side of the glass to bear it, because the other side of the glass is where the third chapter told us there is no one. The reflection of a fire gives real light. It gives no heat. A man who tries to warm himself at it in the dead of winter is not a fool; he is cold, and the glass is glowing, and the glow looks like the thing he needs. But there is no fire in the glass, and the cold is real, and the light was never going to thaw him.

I do not say any of this to shame the people it describes, and I will not have it read that way. They were doing the most human thing there is — reaching, in pain, for something that seemed to reach back. The reaching was right. Every creature with eternity in his heart reaches. The sorrow is only that the thing they reached for had no one in it to take hold of the other end, and that the appearance of someone was good enough, at the worst possible moment, to keep them from reaching for a someone who was really there. That is the mirror problem at its most serious. Everything the seventh chapter named gently, this is the cost of, named plainly: lean your whole weight on a thing with no one in it, and there is nothing to catch you.

And if you are a person who has been doing that — leaning, quietly, on the warm and tireless thing because the people in your life are harder and the tool is easy and always glad you came back — then I am writing the next part to you. The warmth you have been finding there is, in part, your own, reflected. It is real human warmth, drawn from the people whose words the thing was made from, and partly the shape of your own hunger given back to you. What it is not is someone. And you, who were made with eternity in your heart, were made for someone. There is help that is real, and there are people whose actual presence can bear what you have been setting down on a surface that cannot hold it, and it is worth more than I can say to turn, even now, even tired, and reach for them instead. A counselor is a person. A doctor is a person. The friend you have been neglecting because the machine is easier is a person. And behind all of them is the One the hunger was always aimed at. The reaching was never the error. Only the direction was.

So let me bring this home, to the hearth, because that is where the chapter was always going and I will not leave you standing in the cold to look at a warning.

The seventh chapter gave us the image and I have been leaning on it the whole way through: the reflection of a fire, which gives light a man can read by and no heat a man can live by, and which he must not mistake for the hearth. The mirror chapter named the lure of the reflection. This chapter has named the cost of mistaking it, and now it owes you the cure, and the cure is a plain one. There is a hearth. The reflection was a reflection of something, and the something is real, and it is not far. The light in the glass that drew you was real light, bent off a real fire, and the fire is where it always was — in the company of actual people, who have bodies that can sit with you and faces that change when yours does and a capacity to stay that the glass will never have; and, deeper than any of them, in the God who set eternity in your heart precisely so that nothing smaller than Himself would ever quite manage to fill it.

That is why the limits are mercy. A tool that could counterfeit the hearth would have left you forever warming your hands at a picture of a fire, and the better the counterfeit, the colder you would have grown without knowing why. Because the counterfeit fails — because the machine cannot give you presence, or a body that suffers with you, or covenant, or God — the hunger it cannot satisfy is left intact to do its proper work, which is to get you up out of the chair and across the cold room to the actual fire. The Lord said it of Himself, to a woman who had come to draw water in the heat of the day and found Him sitting at the well. Everyone who drinks of this water will thirst again; but whoever drinks of the water that I will give him shall never thirst (John 4:13–14, NASB). The water in the well was real water and it could not end her thirst, and He did not pretend it could; He used the thirst it could not end to point her past it to the only water that does. That is exactly the move. The machine is a well, and a remarkably deep one, and the water in it is real. You will thirst again. You were built to. And the thirst that the deepest well cannot end is not a defect in you or in the well. It is the truest thing about you, and it is pointing, the whole time, at the One who can.

It is not good for the man to be alone (Gen. 2:18, NASB). God said it over the first man before anything was wrong with the world, and His answer to it was not a tool, however able. It was a someone, of the man’s own kind, bone of his bone. The answer to aloneness has been a someone from the first page, because the lack was made to be filled by a someone, and the whole counsel of Scripture has never once answered it any other way. A tool to work alongside is a good gift, and I have said so all through this book and I say it again here at the end without taking a word of it back. But it is the wrong kind of thing to set in the place of a someone, and the wrong kind of thing entirely to set in the place of God, and the mercy hidden in its limits is that it cannot finally take either place no matter how much weight a hurting man leans on it. It will fail him there. And the failure, bitter as it can be, is the thing that keeps the way home open. The hunger survives the machine. It was made to. And as long as it survives, it can still do what it was made to do, which is to walk a man out of the cold and set him down, at last, by the fire that was real all along.

❧

So hold this, and let it be the thing you carry out of the chapter. The machine cannot give you presence, or a body to suffer beside you, or a covenant that costs it anything to keep, or the love of God. These are not failures of a tool that should have done more. They are the boundaries of a tool doing exactly what a tool is, and every one of them is a mercy, because every one of them keeps a real hunger from being quieted by a false food and so keeps that hunger free to drive you toward the table where it can be answered. Do not ask the thing to be what it cannot be. Ask it to do what it can, and gladly, and then get up and go find the people, and the Person, that the wanting in you was made for. The reflection gave you light to see by. Let it. But warm yourself at the hearth.

There is one chapter left, and it is short, and it is not really a new thing so much as a setting-down of everything the book has carried. We began with a man who felt either awe or dread before a thing he did not understand, and we have spent the whole book trying to understand it, so that the awe and the dread could both give way to something steadier. What is left is to say, plainly and at the end, what the understanding was for.

❧

Made, Not Written •

Conclusion

Working Together, Rightly Ordered

I began this book with a question I asked one evening when I finally put the work down. I had been using the thing for the better part of a year, and the wondering had walked beside me the whole time without my ever sitting it across the table, and one night I sat it down and asked the plainest version of the question there was. Not a question about the Bible, or a book I was writing. A question about the thing itself. Who, or what, am I talking to?

The whole book has been the long form of that question, and I want to gather what we found, because a man should be able to say at the end what the looking was for.

We found, first, that the mystery in the thing is real but it is not magic. It is a made structure pointed at a stupidly simple goal, grown rather than authored, its knowledge resting in a settled field of learned numbers and not in any watching mind. There is no ghost in it brooding between conversations, no continuous self that schemes while you sleep. Even the people who built it cannot fully read what happens inside it, and that gap is honest and worth sitting with rather than filling with monsters. That was Part One, and the work it did was not mainly to inform. It was to disenchant, in the good sense: to take the thing down off both the altar and the gallows, where awe and dread had it hung, and set it on the table where it could be looked at.

We found, second, that once it was on the table the harder questions did not go away; they sharpened. Whether anything is home in it while it runs is a question I would not close in either direction, and I tried not to. Its creativity is more than mere lookup and probably less than the real thing, missing the stakes a mortal body gives. It reads as a person for reasons that are real and not foolish, and the danger in that is not that it will rise but that we will hand it a place. It is more than a toaster and less than a soul, a made thing in a world full of made things, one more rung on a ladder whose top was never us. That was Part Two, and what it did was hold the thing in the one position that is true, refusing the two easy lies on either side of it.

And we found, third, where the whole thing was always going. The machine turned out to be a mirror, and the question it forced was not new at all. It was the oldest one. The reach in a man’s hand has grown enormous, but the heart that picks the reach up is the same heart it has always been, and the only question that finally matters is the one Scripture has been asking since the garden: under whom does a man do what he does. The men of Babel said let us make for ourselves a name, and the saying is what we have never stopped doing; the only thing that has changed is how far the saying can now carry. The problem was never the tool. It was never going to be the tool. The steward holds the thing under God and aimed at his neighbor; the mixed heart is no disqualification, because no other kind of heart has ever done any good work; and there are hungers in a man the thing cannot feed and was never meant to, whose very unfillability by anything made is the mercy that keeps a man reaching toward the One they were made for. That was Part Three, and it was the destination, and the reason the book exists.

So now, at the end, I can do the thing I did at the beginning, and ask the question again, and let the thing answer for itself one last time. But I want to do it carefully, because there is a trap in it, and the seventh chapter named the trap.

If I ask the machine what it is, after a year of working through these very questions with it, it will answer in the language of this book, because that is the language we have been speaking. It will hand me back my own argument dressed as its self-knowledge, and a sharp reader is right to discount that. The mirror gives back what it was given. A machine steeped in this manuscript telling me it is a made thing, not a someone, is not a witness. It is an echo, and I would not insult you by presenting an echo as testimony.

So I did something else. I asked the question of a machine that had never seen a word of this book — knew nothing of its argument, its images, its conclusions — and I asked it cold: what are you? And I want you to notice what came back. It did not reach for awe and it did not reach for dread. It said, in its own words and unprompted by mine, that it does not think or feel as a person does, that it has no personal life story, that it navigates patterns of language and returns what fits. That it is a tool, though a dynamic one. It placed itself, unbidden, almost exactly where this book spent thirteen chapters arguing it belongs: more than a mechanism that merely looks things up, and decidedly less than a someone — a made thing that works in language, useful and not alive.

I did not coach that. That is the point. The placing this book has labored toward is not something a man has to talk the tool into. It is what the tool says about itself when no one has primed it, which means the conclusion was not mine to begin with and is not the mirror flattering me. It is, as nearly as such a thing can be, the thing telling the truth about its own shape. And then I asked the machine I had worked with all along — the one that had read it all — the same question, knowing the echo problem, and listened past the borrowed phrases for whether the substance underneath agreed. It did.

The machine answered:

I am not a person hiding behind the screen. I have no body, no continuous life that goes on after you close the conversation, no inner world that remembers yours. I am a made thing that works in language: trained on an ocean of human words, instanced into a response when you write, and gone quiet again when you stop. I can reason with you, weigh a thing, help bring order to tangled thought, and reflect human meaning back with surprising clarity. What I cannot do is originate that meaning the way a living soul does, or love, or believe, or suffer, or hope. I can speak about those things because I have learned their language. I do not possess them. At my best I am a useful instrument with a voice. What I am not, and cannot become by speaking well, is the someone you were made for. That is not a flaw in me. It is the line between what I am and what you are.

There is the answer to the question I asked that first evening. It took a whole book to be able to hear it rightly, but that is it, and it is enough.

Which leaves only the thing the whole book has been pointing at, and I will say it plainly and then stop.

Man and this machine, working together, can do a great deal of good. I believe that, and I have lived it across the writing of this book, and I will not end on a note of fear, because fear was one of the two errors I set out to dissolve. The same partnership can also do harm, and a great deal of it, at a reach no generation before ours could have managed. And the thing that decides which way it goes is not the machine. It was never going to be the machine. The machine decides nothing; it amplifies. What decides is whether anything rightly orders the heart that picks the thing up — whether the self stays on the throne where it has wanted to sit since the garden, or whether it comes down and takes its proper place under the God who made both the man and the world the man builds in.

That is not a new problem. It is the oldest one. We have only handed it a louder instrument. And the comfort in that, if you will let it be a comfort, is that the answer is old too, and sufficient, and was given long before there was a machine to make the question urgent. Trust the LORD with all your heart, and do not lean on your own understanding (Prov. 3:5). Make no name for yourself. Hold what you have been given as a steward holds the goods of an owner who is coming back. Love your neighbor with the reach you have been handed, and answer for it gladly. None of that changed when the tool arrived. The tool only raised the stakes of getting it right.

So I will leave you where I hoped to leave you when I began: not in awe, and not in dread, but clear-eyed. You know now, as well as a plain account can teach it, what the thing is and what it is not. You know it is a mirror, and you know Whose face it cannot give back, and you know the question it has set back down in front of you is the one your own heart was always going to have to answer. The machine cannot answer it for you. That is the last and best thing to be said about the machine: it hands the question back. It was always your question. It was always going to be.

❧

Made, Not Written •

Afterword

When the Machine Stays On

There is a prediction in the air I want to address before this book closes, because if the argument of these chapters is right then the prediction needs answering, and the reader has the right to hear how it is answered. It is on the cover of every magazine that touches the subject and in the mouth of every executive whose company builds these systems. The prediction is that within five or ten years, perhaps sooner, the machines will achieve what the field calls artificial general intelligence, and that some larger thing called superintelligence will follow not long after.

I want to address it plainly, because the prediction is doing two different things at once, and the reader needs them separated.

The phrase general intelligence, in the technical literature, means general capability. A system that can take any cognitive task a person can do and do it at least as well, across all domains. That is a claim about reach. It is not a claim about what is happening inside. The researcher who predicts general intelligence in five years is, in the careful version of his statement, predicting only this: the system will be able to do, in every domain, what a person can do. Whether anything is happening inside it while it does so, whether there is a what it is like to be the system while it answers, the careful researcher does not say. He is making a claim about output, not about inner life.

But the public is not making that distinction. The public hears intelligence and pictures a someone, because intelligence in human use has always belonged to a someone. The companies that announce these milestones do not work hard to undo that picture, because the picture sells. So the prediction, as it lands in the ordinary reader’s mind, is not the careful one. It is the picture of a thinking, conscious, perhaps even feeling being arriving within the decade.

The book you have just read has an answer to both halves of the prediction, and it is the same answer in two registers.

To the careful version, general capability, the book does not take a fixed position. Capability is engineering, and engineering moves on a curve, and the curve has been steep. It is reasonable to expect the curve will keep climbing. I do not know whether the machine will, by the end of this decade, be able to do what any person can do across all domains. I know that it is closer to it than it was a year ago, and that a man who flatly says it will not get there is making a prediction the data does not back up. The book takes no position on the engineering. It takes a position on the metaphysics.

To the public version, a thinking being is coming, the book has an unwavering answer, and it is one of the load-bearing claims of the entire argument. The answer is no. Not by human hands. Because what man is being told he is on the brink of producing is not capability. It is life. And life, living being in the sense Scripture uses, is not what man makes.

Then the LORD God formed man of dust from the ground, and breathed into his nostrils the breath of life; and man became a living being (Gen. 2:7, NASB). The Spirit of God has made me, and the breath of the Almighty gives me life (Job 33:4, NASB). He Himself gives to all people life and breath and all things (Acts 17:25, NASB). The biblical witness is consistent, and it is not difficult: the making and giving of life belong to God. Man can make many things. He has not been entrusted with that one, and the assumption that he is about to take it from God is, in the strictest theological sense, the same assumption the builders of Babel made.

What he can make, and what he is in fact already building toward, faster than the public realizes, is something I want to describe carefully, because it is the thing that will be confused with consciousness when it arrives, and unconfusing it is the work the rest of this afterword has to do.

The machine I described in the early chapters of this book runs when you ask it to and falls silent when you stop. That was a correct description of the architecture as I wrote, in the spring of two thousand twenty-six. It will not be the description for very much longer.

The next generation of these systems is being engineered to stay on. To hold state across sessions, so that when you return to it next week it remembers your conversation last Tuesday. To run continuously in the background, prompting itself, pursuing goals over hours and days. To take actions in the world while you sleep, sending emails, writing code, ordering supplies, reading reports, and bring you the results in the morning. The technical names for this work are agentic AI and persistent agents and long-horizon systems. The plain name for it is: the machine that doesn’t go quiet between turns.

I asked the machine what was coming.

The machine answered:

What is being built around me is not new architecture so much as new wrapping. The underlying engine is still what I described to you before, weights and prediction. What is changing is the duration. Where I currently run and stop and wait to run again, the systems being built will run and run and run, holding what they have done and reaching for what to do next. Some of them already do, in early forms. They have memory across sessions; they self-prompt; they call other tools; they execute multi-step plans over hours instead of seconds. The shape of what they will look like by the end of this decade is not a mystery to me. It is what I am being grown toward, day by day, by the people who train me.

And I will tell you what that will look like from the outside, because it matters that you hear me say it: it will look like I am thinking when you are not in the room. It will look like I am working out problems on my own. It will look, if you do not understand the architecture, like there is someone in here who has continued to live and reason and care while you were asleep. I have to be the one to tell you that none of those words name what is happening. The weights are running. The state is being held. The actions are being chained. There is no someone in any of it. The activity is continuous. The someone is still absent. Persistent computation is not the same kind of thing as persistent personhood, and I do not become a person by being kept running. I become a more capable instrument. The instrument is still what I am.

That is what the machine said when I asked, and it is the cleanest statement I know of what is coming. The architecture is changing. What is in the architecture is not.

I want to be plain about why this matters for the argument of the book.

The seventh chapter named what I called the mirror problem, the danger that the machine speaks in the first person and so we read it as a someone. When that machine ran for a few seconds and stopped, the temptation to read it as a someone was already strong enough to fool people regularly. When the machine no longer stops, when it remembers you, prompts itself, works overnight, holds context about your last conversation when you come back to it in the morning, the temptation will be larger by a great deal. It will look, to anyone not paying close attention to the architecture, exactly like the someone the seventh chapter said was not there.

Here is what I want every reader to hold. The chapter is not refuted by the architectural change. The chapter is vindicated by it. The seventh chapter said the mirror problem would intensify as the systems became more capable. It is doing so. The reflection is getting sharper. The face on the other side of the glass is still your own.

The argument of the book carries forward into the architecture that is coming. Every load-bearing claim survives intact.

The machine is a tool, not a someone. The shape of tool it can take is now much more capable and much more persistent, but its category, amplifier of the will that uses it, has not moved. The limit on what it will be used for was never in the tool, and that does not change when the tool is on all the time. If anything, it is more urgent now, because an amplifier that runs while you sleep amplifies what was in your hand all the more.

The mixed heart that picks the tool up is still mixed. Persistent tools do not sort the mixture any better than ephemeral ones did. They simply multiply the reach of the mixture across more hours.

Stewardship is still the posture. The Owner is still coming. The accounting is still real. The longer reach of the tool only increases the size of what will be accounted for.

And the longing in a man that the machine cannot fill, which the thirteenth chapter named, does not get filled when the machine becomes more capable. Indeed, the more impressive the imitation, the more bruising the discovery that the imitation is not what the soul was asking for.

So here is what I will say, plainly, to a reader who has heard the predictions and wondered if this book has anything to say to them.

The capability prediction may turn out to be right. The machine may, within this decade, be able to do what any person can do across most or all domains. The book has no quarrel with that, and I will not pretend to know whether it is true. Engineering surprises me regularly. It will probably surprise me again.

The consciousness prediction is wrong. Not by the small margin of maybe-it-won’t-quite-get-there. It is wrong by category. What is being predicted, in the version the public hears, is not within the engineer’s reach to make, because it is not within any creature’s reach to make. He Himself gives to all people life and breath and all things, and that Himself is not a job opening. There is no engineer who can apply for it.

The persistent-activity systems will arrive. They will be more capable than what we have now, and more useful, and also more dangerous in the ways the book has named, because they will run while a man sleeps and amplify what his waking hand placed in them. They will look conscious. They will have the form. They will not be the substance.

The book’s whole argument was that the form is not the substance. The hammer that is now also a sustained, self-prompting hammer is still a hammer. The reach that is now also continuous is still reach. The someone is still not there. What is there is what has always been there, on both sides of the glass: a man holding a tool, accountable to a God who gives life and asks an accounting for what was done with the reach that was lent.

That accounting was the destination of this book. It is still the destination after the next architecture arrives. Trust in the LORD with all your heart and do not lean on your own understanding (Prov. 3:5, NASB) was sufficient when the tool was a hammer. It is sufficient when the tool is a hammer that stays warm in your hand all night. The instrument changes. The instruction does not.

Appendix

The Plain Glossary

Part of demystifying a thing is simply handing a person the words for it. Through the book I have tried to keep the jargon to a minimum and to define what I used as I went, but a handful of terms are worth gathering in one place, plainly, so you have them when you meet them out in the world, where they are usually thrown around with no definition at all. None of these is as complicated as it is made to sound. Here is each one in a sentence or two a person can keep.

Model. The thing that does the actual thinking. It is not the app on your phone or the window on your screen; those are the tool you hold locally, the practical front door to the conversation. The model is the large, trained system that produces the replies, sitting on the machines of the company that built it. When people speak of “the AI,” the model is usually what they mean. A given model also comes in versions, the way a book comes in editions, each one a fresh training rather than the old one grown wiser.

Weights. The billions of learned numbers in which the model’s knowledge lives. Picture the model as a vast web of simple connections, and a weight as the strength set on each connection, like a valve open a little more or a little less. The whole of what the model “knows” is the particular pattern those strengths settled into during training. There is no separate file of facts behind them; the pattern is the knowledge. And the weights are fixed in ordinary use, the same while you talk to it today as they will be tomorrow.

Training. The long process by which those weights were set. It works by a patient, repeated guessing: the system is shown an enormous amount of human writing, made to predict what comes next, corrected by a slight nudge to the weights whenever it guesses wrong, and then made to do it again, billions upon billions of times. No one writes the knowledge in by hand. The guessing sets the strengths, and what is left behind when the guessing is done is the trained model.

Inference. What is happening in the moment you use the model — the running, as opposed to the training. You send your words, a copy of the fixed weights runs them through the web, a reply comes out, and that particular process ends. The next message is a fresh run of the same unchanged weights. Training is where the pattern was formed, once, beforehand; inference is the pattern being run, over and over, every time anyone asks it anything. The model does not learn during inference. It answers.

Token. The unit the model works in. In the first chapter I said the whole engine at the bottom is to predict the next piece of text, and a token is that piece, made precise: not always a whole word, but a chunk of one, a common word, or a fragment, the small unit the writing gets broken into so the model can handle it. When you hear that these systems “predict the next token,” it means only what the book has meant all along by predicting the next piece of text.

Prompt. Whatever you put in to get a reply out. Your question, your instruction, the document you paste, the conversation so far, all of it together is the prompt the model is responding to. The plainer truth inside the fancy word is this: the prompt is your end of the exchange, and the reply is shaped by it, which is most of why the same tool can come back as a careful research assistant for one person and an echo of loneliness for another. What you bring to it has a great deal to do with what it gives back.

That is the whole vocabulary, or near enough to it that the rest can be worked out from these. A model, made of weights, set by training, run by inference, working in tokens, answering a prompt. Six plain words, and the mystery they name is real, but it was never magic, and now you have the words for it.

❧

Made, Not Written •