ChatGPT and history of maths misconceptions

You know how loads of things in maths are named for the wrong person? In 1996, a fun quiz appeared in The Mathematical Gazette based on history of maths misconceptions. It contained a series of questions where the obvious answer is not correct, such as “Who discovered Cramer’s rule?”, “Did Pascal discover the Pascal triangle?” and “Who first published Simpson’s rule?”

I was looking for a demo to show my students that generative AI programs are not producing accurate knowledge when I thought of this quiz. I put its questions to ChatGPT to see how it did. The point of the exercise is that these systems just parrot back words from their training data without any concept of truth, so if the training data is full of misconceptions, so too will be the responses. But these are misconceptions from the 1990s, so how much influence will they have on the responses?

You can see how ChatGPT did when I gave it the quiz in a short, free to read, open access paper in the British Journal for the History of Mathematics: Generative AI and accuracy in the history of mathematics.