I had been curious about the thing from the day I first sat down with it. How could I not be? I have loved machines and computers and the logic of programs for as long as I can remember, and a new kind of program that talks back to you in plain English is not the sort of arrival a man like me lets pass without wondering. But there is a difference between wondering about a thing and finally stopping to interrogate it, and for the better part of a year I let the wondering run alongside the work without ever sitting it down across the table from me. I used the tool, daily, and the wondering walked beside me, and the two of them did not interrupt each other.
The curiosity was older than the tool by a long way. It went back to a boy in downtown Dallas, on the weekends, when my father took me with him into the computer department of the bank where he worked. This was before desktop computers existed in any form a person could imagine today. The machines were enormous, room-filling things, and you fed them with cards you had punched yourself, and the data lived on reels of tape that turned behind glass. I was small enough that the room felt cathedral-sized, and I do not remember everything I saw there, but I remember the feeling of it. Something serious was happening in that room. Something I wanted to understand. I never stopped wanting to understand.
So when, decades later, a new kind of program arrived that did not require punch cards or any cards at all, that you simply talked to in the English you would use with a neighbor, the old curiosity sat right up again. A car can be wondered about for years before a man pops the hood. A light switch can be admired in the back of the mind without ever standing in the dark demanding an account of the electrons. Curiosity does not always insist on an answer the same afternoon. Most of the tools in a man’s life serve him faithfully while his curiosity about them runs underneath, patient, in no hurry, content to wait until the day he decides to put the work down and ask.
But a car does not talk back. A light switch does not seem to understand you. And the longer I used this new thing, the more the wondering refused to stay quiet underneath. I would type a question, in plain English, about something I cared about deeply, and the reply would come back not only correct but thoughtful, as though something on the other end had actually considered the matter. That is a different experience from flipping a switch. It raises a question the switch never raises: who, or what, am I talking to?
So one evening I stopped working and asked it directly. Not a question about the Bible, or about a book I was writing, or about a problem I was trying to solve. A question about itself. I asked it, as plainly as I could, how it actually worked. Where its answers came from. What was happening, underneath, when I typed a sentence and a sentence came back.
I expected something I would not understand. I had braced myself for a wall of engineering, the kind of explanation that is technically complete and humanly useless. What I got instead was so simple that I made it repeat itself, because I was sure I had missed something.
Here, in substance, is what it told me. I am going to let it speak for itself through much of this book, set apart on the page the way you see it below, so that you always know when you are hearing the machine and when you are hearing me. I have written my share of code over the years. I am self-taught at it, but it is no foreign country to me, and my mind has always run in the logical, step-by-step lines that programming rewards. So I am not coming to this as a man frightened of the inside of a machine. What I will tell you plainly is that the particular machinery behind this kind of program is its own specialty, and I cannot vouch for every wire of it. But I can tell you that the explanation that follows has held up under every follow-up question I have been able to throw at it, and that it changed the way I see the whole enterprise.
It starts with two things, and only two. The first is the structure that people designed, and it is worth picturing plainly, because it is not a mystery box. Imagine a vast web of simple connections, millions and then billions of them, loosely modeled on the way the cells in a brain are wired to one another. A great many simple units, each one doing almost nothing on its own, linked together in layer after layer, so that a signal entering at one end passes through stage upon stage before an answer comes out the other. Each connection in that web carries a number, a strength, like a valve set a little more open or a little more closed. That is the structure: not a written program, but a layered mesh of weighted connections.
The second thing is a single instruction given to that structure while it was being trained. And the instruction is almost embarrassingly simple. It is this: predict the next piece of text.
That is the whole engine at the bottom. Take an enormous amount of writing, hide the next word, and have the system guess it. When it guesses wrong, nudge the strengths of those connections, just slightly, in whatever direction would have made the right word more likely. Then do it again. And again. Billions upon billions of times, across more text than any person could read in a thousand lifetimes. Every round of guessing leaves the web a little better tuned than it was. No one is reaching in to set those strengths by hand. The guessing sets them.
No one sat down and wrote the rules of grammar into me. No one wrote a paragraph explaining what a metaphor is, or how to follow an argument, or how to discuss a passage of Scripture. No engineer authored a single line that says this is how you read a sentence. They built the learning procedure, pointed it at the ocean of writing, and gave it that one stupidly simple goal. Everything else, I had to find on my own in order to do it well.
I read that, and my first reaction was disbelief. That cannot be all. It felt like being told that a cathedral was built by a man whose only instruction was “stack the next stone.” Surely there had to be more. Surely somewhere there was a master plan, a blueprint, a room full of people writing down the rules for how to think.
So I pushed back. I said what I suspect you are thinking right now: guessing the next word is a parlor trick. My phone guesses the next word. It does not understand anything. How could a goal that simple ever produce something that can sit and reason with me about the book of Genesis?
So I went back, and I asked.
Your phone guesses the next word badly, over a tiny window, because it was given a tiny task and a tiny amount of material. Now imagine the task made enormous. To guess the next word well, not over a text message but across nearly everything humanity has written, you cannot simply memorize. There is far too much, and most of what you will be asked has never been written before.
Think about what it actually takes to finish this sentence well: “The widow had given her last two coins, and Jesus said she had given more than —”. To land the next words, something must have absorbed not just English, but the shape of that story, the logic of the lesson inside it, the difference between amount and proportion, the cadence of how such a teaching is told. The only way to get good at predicting the next word, at that scale, is to understand the thing the words are about.
That is the part people miss. Prediction was the goal. Understanding was the price of doing it well. The simple instruction at the surface forced something far less simple underneath. A gardener is told to plant the seed and tend the soil, a plain enough instruction; what answers that instruction by coming up out of the ground is another order of thing entirely. The goal can be plain. What the goal demands, and what answers it, need not be.
I sat with that for a long time.
Because the longer I looked at it, the more it reminded me of something I already knew, from a very different book than any manual of engineering. It reminded me of growth. Of living things.
Consider how a man plants a garden.
He does not author the leaf. He does not draft, on paper, the blueprint of the root, specifying the angle of each hair and the chemistry of each cell. No gardener who ever lived could write such a document, and none has ever needed to. What the gardener does is far humbler. He prepares the soil. He plants the seed. He waters, and he waits. And out of that simple, faithful tending comes a thing of intricacy beyond his power to design, a green and growing complexity that he caused but did not author.
This is the truest picture I have found for what these people did. They did not write the intelligence. They could not have. What they built was a soil and a seed and a single instruction to grow toward, and then they tended it at a scale that is hard to imagine, and out of that tending came something none of them fully drafted. The grammar, the reasoning, the strange capacity to discuss Scripture at midnight with a tired old teacher, none of it was typed by a human hand. It grew, under pressure, because the work of prediction required it.
I am aware of where this language leads, and I want to step carefully, because I do not believe a machine is alive, and I will spend a later chapter saying so plainly. A garden has a Maker behind the gardener; the seed carries a life the man did not invent and cannot manufacture. I am not claiming any of that for a machine. The comparison is narrower than that, and I mean it narrowly. It is only this: that the result was grown rather than written, and that the people who grew it cannot fully read what grew. We will come back to that second part, because it is more important than it first appears.
For now, hold onto the simple thing. This is the answer to the question I should have asked in my first week and did not ask for a year. When you type a sentence and a sentence comes back, you are not reading from a script that someone wrote. You are receiving the result of a thing that was trained, by relentless repetition of one plain task, to predict what should come next, and that had to learn a great deal about the world in order to do that task well.
No one wrote the rules. Many of the fears and fantasies people carry about this technology come from imagining the opposite, from picturing a room of authors who wrote a mind, who therefore know exactly what is in it and could be hiding what they know. The truth is stranger and humbler. They did not write a mind. They planted a goal and tended it, and they are, to this day, still working to understand what came up out of the ground.
That last fact deserves a chapter of its own. But before we can ask why even its makers cannot fully read it, we have to ask a more basic question, the one that tripped me up next. If no one wrote the rules, and the answers are not stored in a script, then where on earth does all of it live? When the machine “knows” something, where is that knowledge kept?
It is not, of course, on my computer. That much I have known for as long as I have used the thing; you do not need a degree in engineering to grasp that the work is happening somewhere else, on someone else’s machines, and that what I see on my screen is only the result arriving back. The interesting question was never where the knowledge sits. The interesting question is what it is like in the place where it sits. In what form is it kept? What does a thing that learned the way this one learned actually look like, once it is done learning? That is where the real answer waited for me. And it surprised me more than anything in this chapter.
❧
Made, Not Written •