“Not only could the app be used to learn more about childhood autism, it could possibly reveal signs of post-traumatic stress disorder (PTSD) and mild traumatic brain injury (TBI) in warfighters—conditions that often have subtle symptoms and are difficult to diagnose,” said Predrag Neskovic, from the US Office of Naval Research (ONR)’s Mathematical Data Science program.
The app, called “Autism and Beyond,” was developed by researchers and software developers at Duke University and the Duke Medical Center as part of a Bass Connections project team comprising researchers from different disciplinary backgrounds. “Autism & Beyond” has children complete a series of questionnaires and watch short videos designed to make them smile, laugh, and be surprised.
Parents or caregivers use a smartphone’s user-facing “selfie” camera to record children’s facial movements for evaluation by doctors, researchers and software. The app’s core technical component is a complex mathematical algorithm that automatically maps key landmarks on children’s faces and assesses emotional responses based on movements of facial muscles.
“We analyze the video to track position and movement of the head and face, including the lips, eyes and nose—all of which indicate emotions,” said Guillermo Sapiro, a professor at Duke University, who developed the algorithm.
“For example, while watching stimuli like a funny video, does the child smile, look towards the caregiver or ask the caregiver to view the video as well? We study all of that. Lack of emotion and social sharing are possible characteristics of childhood autism,” said Sapiro.