"Neutrino Evidence Revisited (AI Debates)" | Is Mozart's K297b authentic?

Language Log 2024-11-13

[This is a guest post by Conal Boyce]

Recently I watched a video posted by Alexander Unzicker, a no-nonsense physicist who often criticizes Big Science (along the same lines as Sabine Hossenfelder — my hero). But in this case (link below) I was surprised to see Unzicker play back a conversation between himself and ChatGPT, on the subject of the original discovery of neutrinos — where the onslaught of background noise demands very strict screening procedures and care not to show "confirmation bias" (because one wants so badly to be the first one to actually detect a neutrino, thirty years after Pauli predicted them). It is a LONG conversation, between Unzicker and ChatGPT, perfectly coherent and informative, one that I found very pleasant to listen to (he uses the audio option: female voice interleaved with his voice).
 
[VHM note: This conversation between Unzicker and GPT is absolutely astonishing.  Despite the dense technicality of the subject, GPT understands well what he is saying and replies accordingly and naturally.]

Who knew ChatGPT could be like THAT? Five or six months ago, I had given up on it because of the glib "hallucinations" it often gave in lieu of genuine answers, vowing never to pay attention to that "dying fad" again. 
 
Given that Unzicker of all people was taking ChatGPT so seriously, I chided myself for having given up so easily on it, and decided to revisit it. Mozart's Sinfonia Concertante K297b has a very odd history. Like legions of others, music critic Alfred Einstein never had the slightest doubt that it was authentic Mozart. But somewhere in the 1970s the professional musicologists got into the act and spread the word that the piece was "inauthentic." The obvious question that a normal person would ask next is: Well, then, who the fuck was the Big Unknown Genius who did compose it, if not Mozart? But musicologists don't think that way. They just continued to build stronger and stronger "courtroom cases" for demonstrating that it couldn't possibly be Mozart.
 
So, to test ChatGPT just now, I thought it would be interesting to see where it stands on that long drawn-out K297b authenticity issue which in my mind has never been properly aired among people with common sense. 
 
This was my test query to ChatGPT: "Authenticity of Mozart Sinfonia Concertante K297b?"
 
Its response was to ramble on about an entirely different piece, the Sinfonia Concertante K364 (and K364b) for Violin and Viola soloists, not the one I specified, with 'K297b', which is for three or four wind soloists. I pointed out the error. In a flash came the all-too-familiar response (from when I used to actually have a paid ChatGPT account):
 
"I apologize for the error. You are correct, K297b is for winds and orchestra, not for violin and viola."
 
Eventually, it did finally answer my question, too. I was gratified to see that it sided with me (and with Alfred Einstein and commonsense). It asserted that K297b is genuine Mozart, it's just that parts of it are fragmentary, and it has a very messy performance and publication history blah-blah-blah.
 
Thus, the extremely weird non-"mind" of ChatGPT. It knows this immense bunch-o-stuff, but in its initial response, it may stumble in ridiculous ways that bear on precision-of-data ('K297b' versus 'K364' in this case), the very thing over which computers are generally thought to have perfect mastery!! Very strange.
 

Selected readings