NEWS

MENTAL HEALTH,TECHNOLOGY

Google Glass could help children with autism socialize with others
The high-tech accessory is teaching kids to read facial expressions, a pilot trial suggests

BY MARIA TEMMING 5:00AM, AUGUST 2, 2018

Google Glass may have failed as a high-tech fashion trend, but it’s showing promise as a tool to help
children with autism better navigate social situations.

A new smartphone app that pairs with a Google Glass headset uses facial recognition software to give
the wearer real-time updates on which emotions people are expressing. In a pilot trial, described online
August 2 in npj Digital Medicine, 14 children with autism spectrum disorder used this program at home
for an average of just over 10 weeks. After treatment, the kids showed improved social skills, including
increased eye contact and ability to decode facial expressions.

After her 9-year-old son, Alex, participated in the study, Donji Cullenbine described the Google Glass
therapy as “remarkable.” She noticed within a few weeks that Alex was meeting her eyes more often —
a behavior change that’s stuck since treatment ended, she says. And Alex enjoyed using the Google
Glass app. Cullenbine recalls her son telling her excitedly, “Mommy, I can read minds.”

Unlike most children, who naturally learn to read facial expressions by interacting with family and
friends, children with autism often have to hone these skills through behavioral therapy. That typically
involves a therapist leading the child through structured activities, like exercises with flash cards that
depict faces wearing different expressions. But therapists are so few and far between that a child
diagnosed with autism can spend 18 months on a waiting list before starting treatment.

Dennis Wall, a biomedical data scientist specializing in pediatrics at Stanford University, and colleagues
built the new Google Glass program to offer children with autism at-home, on-demand behavioral
therapy. The headset’s camera records the faces of people in the child’s field of view and feeds that
information to a smartphone app. The app, trained on hundreds of thousands of face photos, is
designed to recognize eight core expressions: happiness, sadness, anger, disgust, surprise, fear,
contempt and calm. When the app recognizes an expression of one of these feelings, it sends the
information to the Google Glass wearer — either by naming the emotion through the headset speaker
or by displaying an emoticon on a small screen in the corner of the right spectacle frame.

In the pilot trial, children ages 3 to 17 with autism used this Google Glass program around their families
for at least three 20-minute sessions per week. Before and after treatment, parents completed
questionnaires that rated their children’s social skills. On this Social Responsiveness Scale, scores below
60 fall within the “normal” range, whereas scores 60 to 65, 65 to 75, and above 75 indicate mild,
moderate and severe autism, respectively. Over the course of treatment, the children’s average score
dropped from 80.07 to 72.93.
Eleven of the 14 children also completed an emotion recognition exam at the start and end of
treatment. In this test, an examiner acted out each of the eight core emotions five times, and the child
guessed which emotion the examiner was expressing. Before therapy, kids got an average 28.45 out of
40 questions right; afterward, they averaged 38 correct responses.

While these results are encouraging, the study did not include a control group of children who didn’t
receive the treatment. So it’s not entirely clear whether the Google Glass program was the only reason
that children showed improved social skills over the course of treatment, says Ned Sahin, a
neuroscientist at Brain Power, a company that develops wearable technology to help people with
autism, in Boston. A randomized controlled trial, where children are randomly assigned to receive
treatment or not, could provide further insight into the therapy’s effects, Sahin says.

Wall and his team are currently working on one such experiment with 74 children ages 6 to 12. If the
Google Glass therapy performs well in future clinical trials and is cleared for widespread use, it could be
a powerful learning aid for many children with autism — which affects an estimated 1 in 59 children in
the United States, according to the U.S. Centers for Disease Control and Prevention.

Already, Cullenbine expects that Alex will have better relationships with people, “and that’s life
changing.”

Citations

J. Daniels et al. Exploratory study examining the at-home feasibility of a wearable tool for social-
affective learning in children with autism. npj Digital Medicine. Published online August 2, 2018.
doi:10.1038/s41746-018-0035-3.

Further Reading

L. Sanders. Scientists seek early signs of autism. Science News. Vol. 191, April 29, 2017, p. 10.

L. Sanders. Training for parents may lessen some autism symptoms in kids. Science News Online,
October 25, 2016.

https://www.sciencenews.org/article/google-glass-could-help-children-autism-socialize-others?tgt=nr