How Does the Brain Process Language?

Our society is built on shared ideas that go from one person to another via language that is understood. How do our brains enable us to understand a stream of spoken words? John T. Hale, Linguistics, and Wenming Luh, technical director of the Cornell MRI Facility, are addressing this grand challenge in neuroscience by mathematically modeling the language-understanding process. These models allow investigators to ask the question: which process-model best accounts for the signals from a particular brain region at a particular moment in time?

Hale is conducting the studies with collaborators at the University of Michigan and two teams in France, which are funded by the French National Research Agency. The experiments involve tracking the signals in the brain when French- and English-speaking people listen to versions of the same published book in their native language. By comparing across models and languages, the project’s goal is to differentiate between aspects of the understanding process that are language-specific and aspects that might be common to all humans.

Bringing together computational linguists and cognitive neuroscientists, the project pursues two specific questions: What aspects of sentence structure determine our expectations for upcoming words? What is the detailed balance between memorization and composition in natural language? Researchers are using electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI) to examine participants’ neural responses to the spoken recitation of the established literary work. The project paves the way for work with individuals who have trouble using language, such as those with autism spectrum disorder. It could also lead to computer systems that use language in a brain-inspired way.

Cornell Researchers

Funding Received

$570 Thousand spanning 3 years