Neuroscientists decode song from brain recordings, revealing areas dealing with rhythm and vocals

August 2023
University of California – Berkley

As the chords of Pink Floyd’s “Another Brick in the Wall, Part 1,” filled the surgery suite, neuroscientists at Albany Medical Center recorded the activity of electrodes placed on the brains of patients undergoing epilepsy surgery.

The goal? To capture the electrical activity of brain regions tuned to attributes of the music — tone, rhythm, harmony and words — to see if they could reconstruct what the patient was hearing. More than a decade later, after detailed analysis of data from 29 such patients by neuroscientists at the University of California, Berkeley, the answer is clearly yes.

For the new study, Bellier reanalyzed brain recordings obtained in 2012 and 2013 as patients were played an approximately 3-minute segment of the Pink Floyd song, which is from the 1979 album The Wall. He hoped to go beyond previous studies to actually reconstruct music phrases through regression-based decoding models.

The phrase “All in all it was just a brick in the wall” comes through recognizably in the reconstructed song, its rhythms intact, and the words muddy, but decipherable. This is the first time researchers have reconstructed a recognizable song from brain recordings.

The reconstruction shows the feasibility of recording and translating brain waves to capture the musical elements of speech, as well as the syllables. In humans, these musical elements, called prosody — rhythm,
stress, accent and intonation — carry meaning that the words alone do not convey.

The researchers also confirmed that the right side of the brain is more attuned to music than the left side. “Language is more left brain. Music is more distributed, with a bias toward right,” Knight said.

Reading your mind? Not yet.

Because these intracranial electroencephalography (iEEG) recordings can be made only from the surface of the brain — as close as you can get to the auditory centers — no one will be eavesdropping on the songs in your head anytime soon.

“Noninvasive techniques are just not accurate enough today. Let’s hope, for patients, that in the future we could, from just electrodes placed outside on the skull, read activity from deeper regions of the brain with a good signal quality. But we are far from there,” Bellier said.

But for people who have trouble communicating, whether because of stroke or paralysis, such recordings from electrodes on the brain surface could help reproduce the musicality of speech that’s missing from today’s robot-like reconstructions.

Knight is embarking on new research to understand the brain circuits that allow some people with aphasia due to stroke or brain damage to communicate by singing when they cannot otherwise find the words to express themselves.