Psychological and physical responses to stress could jeopardize the success of a space mission, and early detection of stress responses is the first step toward preventing this. Dr. David Dinges is observing patients’ visual responses to stressful conditions to create an optical recognition computer program that reliably detects the early presence of stress. The program will then be tested on the same patients during stressful activities to evaluate if the program can accurately measure early stress responses.
Overview
Optical Computer Recognition of Behavioral Stress
Principal Investigator:
David F. Dinges, Ph.D.
Organization:
University of Pennsylvania
Technical Summary
The goal of this project is to develop and test an optically based computer recognition algorithm of the face to reliably detect the presence of stress during performance demands. As manned space flights of increasingly longer durations are being planned, there is evidence from U.S. and Russian space missions that astronauts and cosmonauts have experienced operational stressors that adversely affected subjective well-being, physiology, and performance capability. In order to provide countermeasures for stressor-induced impairments in astronauts, objective, unobtrusive measures of the presence of stress reactions are needed. This project seeks to achieve such a measure through collaboration between laboratories with expertise in the evaluation of behavioral and physiological responses under stressful and non-stressful conditions (Prof. D. Dinges, Department of Psychiatry, Unit for Experimental Psychiatry, The University of Pennsylvania), and in optical computer recognition of human subjects' facial expressions and gestures (Prof. D. Metaxas, Computer Sciences, Rutgers University).
Astronauts aboard extended-duration space missions will endure the harsh space environment and the effects of various stressors (e.g., microgravity, perceived risks, work requirements, habitability constraints, radiation, restricted communication with Earth) to a much greater degree than have been experienced previously. Maintaining individual neurobehavioral functioning of astronauts will be vital to assuring mission success. However, in order to provide countermeasures for stressor-induced, physical and functional impairments in astronauts, objective measures of the presence of heightened stress reactions are needed. The earlier that stress reactions (regardless of their operational, psychosocial, or neurobiological source) can be detected, the greater the probability that an appropriate countermeasure strategy can be implemented (e.g., rest, pharmacology, behavior).
In the absence of objective detection of developing stress reactions, it is unlikely that countermeasures or stress impairment of astronauts can be managed. Many techniques for monitoring stress reactivity in space flight are impractical (e.g., cortisol measurement), unreliable (e.g., self-report), or obtrusive. However, unobtrusive, continuous video monitoring of the human face during neurobehavioral tasks offers a potential solution to these problems. Consequently, this project provides the first scientific test of the use of optically based computer recognition of the face to unobtrusively and reliably detect the presence of stress during laboratory performance demands.
The computer-based optical recognition system builds on the research of Prof. Metaxas by utilizing automatic optical tracking of human subjects' anatomical and motoric changes in facial expressions during non-verbal performance tests. Video input to the system was provided from experiments performed in the laboratory of Prof. Dinges in which healthy adults (males and females of different ethnic backgrounds) were exposed to behavioral stressors to increase the likelihood of developing a sensitive algorithm. The aim of the protocol is to experimentally establish whether an optical computer recognition algorithm based on facial expression can be developed that can objectively, independently and reliably discriminate when subjects are undergoing behavioral stressors, and whether a high degree of accurate categorization can be achieved for both male and female subjects for both younger (22-32 years) and older (33-45 years) subjects and for subjects of different ethnic backgrounds. Further, exploratory and heuristic analyses evaluate the effects of behavioral stressors on physiological responses of cortisol secretion and heart rate, on psychological responses of self-report ratings of stress and mood and on neurobehavioral performance responses; and explore the extent to which the magnitude of the stress response as assessed by these measures relates to the accuracy of the optically based computer recognition algorithm of the face.
A single-blind, repeated-measures controlled trial was used to achieve these aims and to provide the data required to test the hypothesis that an objective, unobtrusive, optically based computer recognition algorithm of the face can be developed to reliably detect the presence of high stress (and of low stress) during performance. A total of 60 healthy adults were studied in the Unit for Experimental Psychiatry laboratory (Dr. Dinges) during three sessions: Iscreening session; IItraining session for development of the optical computer recognition algorithm; and IIIprospective test session of the predictive utility of the optical recognition algorithm to discriminate high versus low stressed states associated with behavioral stressors.
Stress reactions were tracked during both control (low stress) and high stress conditions in sessions II and III, by measurement of salivary cortisol, heart rate, subjective mood/stress responses, and neurobehavioral performance. Videos of subjects faces in the low and high stress conditions of session II were used by the Vision Analysis and Simulation Technologies laboratory to develop a predictive optical algorithm that was tested blind to stressor level (i.e., high vs. low) in the behavioral stressor conditions of session III. The experiment is designed to test the hypothesis that an optical computer recognition algorithm can be used to discriminate when subjects are undergoing behavioral stressors as defined by established stress-related changes in cortisol secretion, heart rate, subjective reports, and performance.
Earth Applications
The study focuses on the ability of an unobtrusive, automated optical technology to detect psychological distress (and the need for countermeasures for it) during operational performance. The knowledge gained through this study has the potential to identify an objective, unobtrusive, automated method for the recognition, monitoring, and management of the risks of neurobehavioral dysfunction due to work-related stress in spaceflight and in many Earth-based safety-sensitive occupations, such as transportation workers (e.g., truck drivers, train conductors, airline pilots); operators in safety-sensitive industries (e.g., power plant control rooms); and military personnel.
This project's funding ended in 2004