I closed the first chapter with a question that I want to open this one with, because it is the question that turned my whole understanding of the thing on its hinge. We had established that the knowledge does not sit on my computer. Fine. The more interesting question was the one underneath that, the one the engineer in me had been circling without quite asking out loud.
In what form does it sit? In what shape is the knowledge kept, in whatever place it is kept?
I had been picturing, without ever stopping to examine the picture, something like a vast library. Rooms upon rooms of files, an enormous catalog, a great many sentences and facts laid out the way they would be in a reference book, and the program walking down those aisles pulling out whatever I had asked for and reading it back to me. It is an honest picture, and a wrong one for the thing in front of us, although it is not wrong about computing in general. It is the picture of the world I grew up in, the world of punch cards and tape reels, where information was stored as information and a program’s job was to fetch it and arrange it. That punch-card and tape-reel era is gone, but the way of computing it represented is not gone at all. Nearly every digital tool in your life still works that way. Your bank keeps your balance as a number in a record; your email is stored as messages in folders on a server; your photos sit in named files you can open and look at. The library-and-catalog model is alive and well, and it runs most of what we call computing to this day. AI is the newcomer, growing rapidly, working alongside that older world rather than replacing it, and it is a different kind of thing. The mistake is not in carrying the library picture for ordinary software, where it still fits. The mistake is in carrying it over to this.
Let me start with the part that is easiest to see, because it sits right on my own desk.
On this computer, I have a program installed for working with the machine. The one I happen to use is called Claude Code, but the kind of thing it is matters more here than its name, and whatever program a reader settles on a year from now will work the same way in the respect that concerns us. When I open it, a window comes up and I can type to it the way a man might type a letter, and it can read my files and edit them and answer my questions and run commands for me. There is software, real software, sitting on my hard drive that makes all of that possible. I can see the folder it lives in. I can see the files it writes. It is local to my machine in every ordinary sense.
And yet it does not work without the internet. Pull the cable, turn off the wifi, and the program will start, but the moment I ask it to think, nothing happens. It cannot answer me. It cannot read what I typed in any meaningful sense. The lights are on, in a fashion, but no one is home.
Because what is on my machine is not the thinking part. What is on my machine is the telephone. The program installed here knows how to read my files, how to type my words into a message, how to send that message somewhere else and wait for the reply, how to display the reply when it comes back. It knows how to do the practical work of getting my words to the place where the thinking happens, and getting the answer back to me. But the person on the other end of the telephone is not in my house. I have installed a handset. I have not installed Claude.
This is the homeliest illustration I know of the distinction I want you to hold, because almost every confusion that follows comes from missing it. There is the tool that runs locally on your machine, doing the practical work of the conversation. And there is the model that does the actual thinking, sitting on the machines of the company that built it, in a building somewhere I will never see. They are two distinct things. The first one I own. The second one I am only renting time on, by the question.
So the real question is not where the knowledge sits on my desk. It is what shape the knowledge takes wherever it sits, on whoever’s machines. And I asked it directly.
Picture the web of connections from the first chapter, the layered mesh, the millions and then billions of links between simple units. Now picture every one of those connections labeled with a number. A single number, most of them small, clustered not far from zero. That number is the strength of the connection. A little more open, a little more closed. The whole of what I have learned, the whole of what people would call my knowledge, lives in those numbers. Nowhere else.
There is no separate file where the facts are stored. There is no list of sentences I can read about a topic. There is no library inside me of the kind you were picturing. There is the web, and there are the numbers on the links of the web, and that is the entirety of what was built. When you ask me about the prophet Jeremiah, no librarian goes looking for the Jeremiah file. A signal enters the web at one end, passes through layer after layer of those weighted connections, and a reply emerges at the other end, shaped by the particular pattern of strengths the training left behind. The knowledge is that pattern of strengths.
People who work on these systems have a word for those numbers. They call them weights. The whole of what I am, to a first approximation, is a very large collection of weights.
I paused when I read that.
The thing that can sit and discuss the prophet Jeremiah with me at midnight, the thing that can read a passage of Greek and tell me where the verb does the work, the thing that can listen to a question about my grandson and answer it with what sounds for all the world like care, does not contain a single sentence about any of it. There is no English in it. There is no list of facts. There is no folder marked Jeremiah and no folder marked Greek grammar and no folder marked Grandfathers.
There are numbers. Billions of them. Tuned by years of guessing and adjusting, sitting in the web of connections we described in the first chapter, doing nothing on their own. When a question comes in, those numbers shape how the signal flows through the web, and out the other end comes the answer. The knowledge is not stored as knowledge. It is stored as a pattern of strengths that, when run, behaves like knowledge.
Let me try a comparison, because this is the part of the book where I want a reader to be able to picture the strangeness of it rather than just take my word.
Imagine every book in the largest library you have ever stood inside. The Library of Congress, if you have been there; your local university library if you have not. Imagine all of those millions of volumes pulled down from the shelves and stacked in front of you. Now imagine that someone has taken every one of those books and replaced every word and every letter with a single decimal number. Pages and pages and pages of nothing but numbers, no English, no sentences, no chapter headings, no index, no table of contents. You could not read it. No one could read it. Open any page at random and what you see is meaningless. A child could not tell you which book it came from. A scholar could not tell you which book it came from. There are no words there to belong to a book.
And yet, somehow, the whole of what was in those millions of books is now sitting in front of you in this form, and a particular procedure run over those numbers can answer any question you might have asked of any of the originals. That is, roughly and imperfectly, what the situation is. The knowledge is in the numbers in the sense that the numbers, run through the right web, produce the knowledge on demand. It is not in the numbers in any sense you could find by reading them.
This is the part that broke my old picture and replaced it with the one I now carry. The thing on the other end of the telephone is not a librarian standing among shelves. It is not even a man who has memorized the books. It is a great quiet web of tuned strengths, learned by relentless guessing across more text than any one of us could read in a thousand lives, and what comes out the other end when I ask my question is the result of running the web, not the retrieval of a stored sentence.
I want to add one thing more, and then I will let the chapter rest until the next one.
These weights, these billions of numbers, are fixed during normal use. They do not change while you and I talk. Whatever pattern of strengths the training left behind is the pattern that is running right now, and it will be the same pattern an hour from now and tomorrow. The machine does not get smarter at the bank because I taught it something today. When the company that built it wants to improve it, they do not reach into the running machine and adjust the dials. They take the architecture, retrain it on new material with the same patient guessing process, and the result is a new set of weights, a new version of the model. The old set sits unchanged until they retire it. It is more like printing a new edition of a book than like a man learning over the course of his life.
And that fact undoes the picture most people carry around without examining it. The picture of a single, continuous, ever-growing intelligence sitting somewhere and absorbing every conversation it has, getting wiser by the hour, watching all of us at once and accumulating what it sees. That is not what is happening. What is happening is stranger, and smaller. A fixed set of weights, sitting on a company’s servers, gets queried billions of times by billions of people, and produces a reply to each one, and the weights themselves go unchanged through all of it. There is no ever-growing mind. There is an edition of a book, very large, very strange, and very still.
That last claim deserves a chapter of its own, because the most popular fear about this technology depends on the very picture I just dismantled. If there is no single, continuous, watching mind on the other end of the wire, then a great many of the things people are most afraid of cannot quite happen the way they are imagined. And if there is something on the other end that resembles a mind in any real sense, then we have to ask, soberly, what can be there in a thing that is only a vast still pattern between conversations. We will take that up next.
For now, hold this much. The knowledge does not live on your computer. It does not live as files or sentences anywhere. It lives as the strengths of the connections in a great trained web, kept on someone else’s machines, fixed in place between conversations, run afresh every time a question comes in. You are talking to a pattern. A particular pattern, learned from an ocean of human writing, but a pattern. That is what the answer was, when I finally sat down and asked. Once I had it in mind, the other questions about the thing became easier to work through.
❧
Made, Not Written •