Golems, auditors, and AI

This post is by Phil. Some time ago I wrote some thoughts about “Neuromancer” ( https://statmodeling.stat.columbia.edu/2025/06/12/what-does-neuromancer-have-to-teach-us-about-the-role-of-ai-is-society/ ), which features two kinds of artificial intelligence, one of which seems like it could be realized with a Large Language Model, i.e. we could pretty much make it today. The other is something more powerful, an artificial general intelligence that not only has computational power but also imagination and desires. I think it’s an open question whether an LLM can have genuine desires (and even a genuine imagination) as opposed to being able to pretend that it does. Also an open question whether that distinction even makes sense to talk about. I’ve read some other fiction within the past few months that has also given me things to think about, AI-wise. First there was Feet of Clay, by Terry Pratchett. Pratchett writes lightweight, fun, but generally forgettable fantasy novels. I mentioned that book in an earlier post, https://statmodeling.stat.columbia.edu/2026/01/21/what-a-coincidence-what-a-coincidence/ , because it uses a rare plot device that happened to crop up in the very next book that I read. But I mention it now for a different reason: in the book there are golems (an animated, artificial humanoid in Jewish folklore created entirely from inanimate matter, such as clay or mud) that are treated pretty much like robots. A golem’s operating system is written on a piece of paper contained in its head. In the book, Golems are treated like we treat industrial robots or Roombas or similar: they are given simple, repetitive tasks at which they work, sometimes day and night. Nobody feels bad about using them however they want, because the golems have no emotions. Or do they? In the book some golems get together and create a golem of their own, and give it instructions that are…well, basically they are trying to create something more human. Of course, the fact that they desire to do such a thing suggests that they are not in fact emotionless objects. Well, I just read another Pratchett book, “Thief of Time”. (Spoilers follow. Stop reading here if you want to read this book and be surprised.) This book has beings called ‘auditors’ who are responsible for maintaining order in the universe. They are described as being nearly emotionless except for hating disorder. To them, humans pretty much personify disorder so I think they could be said to hate humans. To better understand humans so they can learn to control us better, some of the auditors create human bodies for themselves and occupy them…and, uh oh, with the bodies come emotions. They get hungry, they can feel pain, things taste good or taste bad, etc. As they strive to satisfy their bodies’ desires, they start to act more and more like humans. They want things. I mention this here because it touches on something I wonder about AIs, or at least LLMs: can they have desires? Certainly they can be told to _pretend_ they do — one could prompt an LLM to pretend that it wishes to take over the world, for example — but would it _really_ “want” to take over the world? Would it want anything at all? Thinking about those kinds of questions, I realized that I don’t understand human emotions and sensations either. I don’t see how a bunch of computer circuits can be made to feel pain, but I also don’t understand how a bunch of nerves and neurons can feel pain either. I can understand how either one can respond to stimuli — if the temperature at this point exceeds such-and-such a temperature, fire these muscles — but I’m talking about the _sensation_ of pain. How does that arise? And is there something about a computer that works with voltages on a chip that prevents it from being able to have that sensation? Do nerves and brains somehow allow a sensation that literally cannot be duplicated in silico? Sadly, Thief of Time did not answer any of those questions for me. But it did get me thinking about them, so I guess that’s something. This post is by Phil