How does the brain process language—using sounds to form words, and assigning words their meaning? Recording the activity of single neurons in the language region of patients undergoing deep brain stimulation, researchers from Harvard Medical School, Massachusetts General Hospital, and MIT have been able for the first time to observe how semantic information is processed and represented by these cells during language comprehension. They’ve discovered that the brain uses contextual clues to decipher meaning, implying that understanding words and sentences is a dynamic, interpretive process.
Using tiny electrodes with radio antennas, they recorded neural activity from about 300 cells as participants (receiving treatment for conditions such as Parkinson’s and essential tremor dystonia) listened to stories read aloud. By tracking these neurons’ activity during natural speech processing, the scientists observed that individual cells responded selectively to specific word meanings, and reliably distinguished words from nonwords. And they found that the neurons’ activities were dynamic, reflecting the words’ meanings based on their specific sentence contexts (“He picked the rose” versus “He finally rose,” for example), rather than responding to the words as fixed memory representations. They were even able to show how the activity of ensembles of these cells could be used to accurately predict the broad semantic categories of related words (such as “rat” and “mouse”) as they were heard in real time during speech.
Each participant in the study listened to about a thousand words. “Looking across individuals,” associate professor of neurosurgery Ziv Williams explains, “people have neurons that respond uniquely to words associated with food, or with emotions,” for example. In all, the responses of individual neurons to specific words clustered into nine categories: actions (walked, ran), emotional states (happy, sad), objects (hat, broom), foods (carrots, cake), animals (bunny, horse), nature (rain, sun), people and family (son, sister), names (such as George and Hannah), and spatiotemporal relationships (up, down, and behind).
Some cells responded only to words belonging to one category. Even fewer responded to two. The clusters themselves aren’t perfectly delineated, Williams says. Consider “bird,” “nest,” “egg,” and “barn.” Even though nests and barns aren’t animals, they’re associated with that category. Some neurons, he and his colleagues found, “responded a little bit more broadly to anything that related to animals (or food or people), meaning that they’re a little bit more like generalists. Others were finely tuned and would only respond to very specific meanings.”
Williams explains that this research is essentially listening in on individual nodes within a much larger network for decoding language. “Functional imaging studies have previously shown that semantic information is likely represented very broadly within the brain,” he says: lots of places show activity during language processing. “You can think of the brain as a little bit like an artificial neural network,” he continues. “All the nodes and all the neurons are in some way connected to all other neurons in the brain. There are some areas that are more specialized for certain functions, and certain neurons that are more closely connected than others. When you hear the word ‘dog,’ for example, it goes through your auditory cortex, but is then distributed fairly widely.” As a first step to probe how the process of language comprehension works, he and his colleagues wanted to know how information is represented across single neurons, the most basic units for encoding information in the brain.
In the first set of experiments, they studied neuronal responses to “very structured eight-word sentences” (“Eva placed a secret message in a bottle,” for instance) that were semantically diverse and allowed them to formally sample responses to a broad set of meanings. They used two different recording techniques and checked for a consistent response across the participants to ensure that the findings would be robust and generalizable. But they also probed deeper, asking whether a neuron that appears to respond only to action words would still react to such prompts under different conditions, topics, and scenarios—as when presented with an excerpt about the life and times of Elvis Presley. “The answer to that,” says Williams, “broadly, is ‘yes.’”
Perhaps the most interesting finding was that neurons tracked the meaning of words based on their context, even when two words sounded exactly the same: “My son jumped into the canal,” versus “The sun set into the ocean.” Neurons that track the family and relationships category respond differently to the first instance than to the second.
What that demonstrates, says Williams, is that “language is not a passive process. We’re actively trying to infer and predict what others are trying to say. If I say, ‘The dog chased the…’, you’ll probably infer that I’m going to say ‘cat.’” Further, when the researchers looked at different contexts in which the next word was either predictable or less predictable, they found that neurons modulated their activity based on how predictable the words were. Explains Williams, “We could actually predict the category of the word, based on the activity of a neuron. That suggests that these neurons are integrating information from the sentence to encode the correct meaning of the words that are being heard.”
The implication is that language is not “a simple memory-retrieval type process” in which meaning is associated with a stored “mirror representation” of a word, he explains. Instead, “Language…involves constantly integrating information from the phrases and sentences that you’re hearing to form a meaning or representation in real time.”
Now Williams wants to know if the word for “apple” in another language will trigger the same neurons in people who are bilingual. And what about an image of an apple? Will the neural responses be similar in another domain, he wonders? “We’re just in the very early days of this research. We’re still asking basic questions about how the brain processes language—this complex, unique human ability.”
