"Age against the machine"

According to Roy Dayan et al., "Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis", BMJ Christmas 2024:

To evaluate the cognitive abilities of the leading large language models and identify their susceptibility to cognitive impairment, using the Montreal Cognitive Assessment (MoCA) and additional tests. […]

With the exception of ChatGPT 4o, almost all large language models subjected to the MoCA test showed signs of mild cognitive impairment. Moreover, as in humans, age is a key determinant of cognitive decline: “older” chatbots, like older patients, tend to perform worse on the MoCA test. These findings challenge the assumption that artificial intelligence will soon replace human doctors, as the cognitive impairment evident in leading chatbots may affect their reliability in medical diagnostics and undermine patients’ confidence.

Figure 1:

If you're not familiar with the BMJ's Christmas tradition, your brow may be furrowed.

Believe it or not, this article was picked up as (apparently unironic) clickbait by quite a few mass-media outlets, e.g. here, here, here.

See also "'Tis the season", 12/21/2007. And "How to turn one out of five into three out of four", 1/19/2016, where it took me a while to catch on…