AI plus MRI yields the ability to recognize what the mind is hearing

Enlarge (credit: Getty Images)

We have various ways of seeing what the brain is up to, from low-resolution electrodes that track waves of activity that ripple across the brain, to implanted electrodes that can follow the activity of individual cells. Combined with a detailed knowledge of which regions of the brain are involved in specific processes, we've been able to do remarkable things, such as using functional MRI (fMRI) to determine what letter a person was looking at or an implant to control a robotic arm

But today, researchers announced a new bit of mind reading that's impressive in its scope. By combining fMRI brain imaging with a system that's somewhat like the predictive text of cell phones, they've worked out the gist of the sentences a person is hearing in near real time. While the system doesn't get the exact words right and makes a fair number of mistakes, it's also flexible enough that it can reconstruct an imaginary monologue that goes on entirely within someone's head.

Making functional MRI functional

Functional MRI is a way of seeing what parts of the brain have been active. By tuning the sensitivity of the imaging to pick up differences in the flow of blood, it's possible to identify areas within the brain that are replenishing their energy after having processed some information. It has been extremely useful for understanding how the brain operates, but it also has some significant limitations.