Nomic: Playing Self-Modifying Games with LLMs | Jim Cowie

"Six months ago, though, getting ChatGPT to simulate an interesting game of Nomic was a frustrating exercise. I had to supply the full set of rules as part of the initial prompt. Despite my best attempts, the LLM would simply forget what rules were in effect at any given time. It would fail to consistently apply existing rules, let alone new rules that the players might enact. Its arithmetic was dodgy. The new rules that the simulated users proposed were kind of vague and uninteresting. With careful prompting, it could be forced to laboriously review history, recite rules anew with each term, and tick the boxes to make sure the accumulating sets of new rules were consistently applied.

In the end … I was pretty convinced that whatever “this” was .. human players of Nomic faced no real competition.

January 2025 update

I put the project to bed, and didn’t think about it again for six months. On my bus ride into Cambridge today, though, thinking (for no particular reason) about the fragility of the rule of law and failures in rulemaking systems, I wondered: has the o1 model come far enough to make it interesting to revisit the challenge?..."