Studies have shown that computer analysis of a person’s speech reflecting motor control problems can be used to remotely assess deficits in comprehending language, cognition and decision-making abilities. Dr. Philip Lieberman is using these findings to develop a system that monitors speech patterns to predict mental abilities. Lieberman is studying speech and behavior patterns of climbers ascending Mount Everest, where life-threatening stress and neurologic impairment is similar to what may be encountered in space. He is also using laboratory-based studies of task-induced stress. His work is integrated with other NSBRI projects aimed at systems that monitor an astronaut’s ability to perform. The techniques being developed also have general applications such as in the diagnosis and treatment of parkinsons disease and alzheimers dementia. One critical application will be on NASA’s projected manned mission to Mars.
Overview
Speech Monitoring of Cognitive Deficits and Stress
Principal Investigator:
Philip Lieberman, Ph.D.
Organization:
Brown University
Technical Summary
Our procedures are based on recent insight on how the brain works. Complex behaviors, such as walking, talking, comprehending the meaning of a sentence, or deciding what you should do when circumstance changes, involve linked activity in different parts of the brain. The subcortical basal ganglia are structures of the brain that support "circuits" (akin to electrical pathways) connecting different regions of the brain. Independent studies show that circuits involving the basal ganglia regulate motor control, cognition, emotional responses and some aspects of a person's personality. Damage to the basal ganglia, which are sensitive to both radiation and oxygen deprivation (hypoxia), thus can degrade these aspects of behavior. We have confirmed that acoustic measures quantifying slow speech motor control can be used to monitor cognitive impairment induced by hypoxic and cosmic ray-induced insult to the brain, as well as degraded cognitive performance resulting from task difficulty, sleep deprivation and other stressors. Other acoustic metrics can identify sleep deprivation and stress derived from perceived poor performance. We were moving toward an operational system and developed a prototype computer algorithm that automatically measures speech rates in low-noise environments.
Our project had two complementary components:
- Our Everest Space-Analog studies provide the foundation for an operational system that uses acoustic measures of a persons speech to detect cognitive deficits resulting from exposure to radiation in space or on the moon, as well as hypoxia in space walks.
- Our cooperative laboratory study with the NSBRI project directed by Dr. David Dinges at the University of Pennsylvania School of Medicine has yielded a procedure that uses these acoustic measures and additional ones to detect impaired cognitive performance resulting from the stress of task difficulty, as well as psychosocial stressors and sleep deprivation. We developed prototype computer algorithms that automatically derive relevant acoustic measures from running speech.
Independent NSBRI research confirms that the basal ganglia are sensitive to radiation, and they also are susceptible to oxygen deprivation (hypoxic insult) in climbers breathing thin air at extreme altitudes. Thus, we can use climbers ascending Everest as models for the some of the effects of radiation on crews in space. The research is ethical because subjects willfully expose themselves to the dangers of climbing Everest. Climbing Everest entails ascending to a series of high camps. At each camp with progressively lower oxygen, our climbersubjects perform sentence comprehension tests, the Wisconsin Card Sorting Test (WCST) and mental arithmetic tests that simulate operational tasks encountered in spaceflight. WCST performance translates to the ability to change plans when circumstances change. At higher altitudes, error rates on the WCST and arithmetic tests tend to increase, and it takes longer to comprehend the meanings of sentences. We used the Brown Lab Interactive Speech System (interactive speech analysis computer algorithms developed at Brown University) to derive acoustic speech measures that reflect slower motor control. These acoustic metrics track cognitive dysfunction. Our procedures detect lower sentence comprehension or degraded WCST performance 91 percent of the time. A system that used measures of speech rate to monitor these cognitive deficits would have had a 3 percent miss rate and 6 percent "false alarm rate (decisions that do not reflect with impairment).
Task difficulty, stress and sleep deprivation
In the Dinges laboratory, subjects had to perform easy and difficult mental arithmetic tasks. Subjects also performed these tasks after sleep deprivation and with a psychosocial stressor (being informed that their performance was deficient). Measures of slower speech tracked higher error rates and fewer solutions as they performed the more difficult task. Acoustic measures that reflect laryngeal activity identified those subjects who were sleep deprived or subjected to the psychosocial stressor.
Earthbound applications
Our objective acoustic analyses provide direction for focused intervention for children having verbal apraxia (speech motor sequencing difficulties). Our procedures have been used to evaluate new procedures for the treatment of Parkinson's disease, which involves basal ganglia degeneration. The effects of task difficulty, sleep deprivation and other stressors could be voice monitored in applications ranging from enhancing computer-implemented instruction to safely driving a truck.
Earth Applications
Our projects techniques have already been used to assess the efficacy of new surgical procedures for the treatment of Parkinsons disease. They may also provide instruments that can detect memory loss in the early stages of Alzheimer's disease. Such early detection would permit clinicians to take maximal advantage of therapies that can delay or even arrest further decline. Our techniques may have application to the diagnosis, assessment and treatment of other human pathologies stemming from impaired basal ganglia function in neural circuits regulating speech production, cognition and personality. These include not only neurodegenerative diseases, but also the results of acute insult. For example, hypoxia during birth can lead to verbal apraxia in children; a syndrome where speech motor and orofacial motor control is degraded and which can result in cognitive and linguistic deficits. Our computer-implemented speech analysis techniques identify specific deficits that are not evident by listening to the children; therapy can then be directed toward the remediation of these problems. We also can identify the specific cognitive deficits accompanying many instances of verbal apraxia, again directing treatment. Our research can be useful in identifying genetic and environmental factors underlying the condition. We are extending this line of enquiry to autism, working with specialists in pediatric neurology.
Another potential application is in assessing the truthfulness of verbal statements. Dissimulation generally involves greater cognitive effort, activating more brain structures than would be the case for truthful statements. This yields slower speech which we can readily detect.