[This article by David Roberts first appeared on the Kennesaw State University website, republished with permission. Photos by Darnell Wilburn]

Joy Li, assistant professor of game design and development in the College of Computing and Software Engineering, has always been interested in using advanced gaming tools in a serious manner.

Her current research involves helping children who are on the autism spectrum and have an attention deficit by introducing them to adaptive virtual reality (VR) systems. The project is funded by a four-year National Science Foundation (NSF) grant totaling more than $170,000.

Li’s research aims to improve the learning ability of children with autism spectrum disorder (ASD) and provide novel design guidelines for future virtual reality systems used in ASD therapy.

“I heard about the research topic from a visiting scholar when I was working toward my Ph.D., and I happened to know a few people with Asperger syndrome, so I decided to do my Ph.D. dissertation using VR to help people with ASD,” said Li, who received her Ph.D. in 2018 from the University of Louisville. “And this project was first brought up by a former KSU faculty member, Chao Mei, whose research interest was similar, so we collaborated a lot. In this project, we use a VR headset that has built-in eye-tracking sensors to help detect distracting objects during a lecture given by a virtual teacher, then we identify and train the attention skills.”

In the study, Li first exposes participants to a virtual environment and asks them to complete a task while simultaneously detecting their attention via eye-tracking data. The training system is set to identify any distractions and then manipulate the objects in a systematic way in that virtual environment. 

The ability to introduce and remove objects is called staging an “impossible experience,” which is imperative to the project.

“It means the things you normally don’t see in real life. In another word, fantasy,” Li said. “In this project, for example, we detect major distractions during the virtual teacher lecturing, then in the training session, we remove the distraction objects one by one in order, and then put them back with different strategies to train kids not to look at them.”

The study involves about 50 participants, which includes 25 typically developed participants and 25 children with ASD, each in grades K-12 with medium to high levels of ASD.

To identify the children with ASD, Li said she worked with Cumberland Academy, a private school for children with autism in Atlanta, to recruit kids who are interested in virtual reality and matched that number with typically developed young people.

In addition to the eye-tracking sensor in the headset that participants wear, Li also collects data through pre- and post-trial questionnaires, and a medical-level wristband called the Empatica E4.

“We try to use four strategies to train their attention skills: Positive/negative reinforcement, positive/negative punishment,” Li said. “They are visualized in the VR scene and expected to trigger different reactions that will affect the learning effectiveness.”

The Centers for Disease Control and Prevention’s Autism and Developmental Disabilities Monitoring Network estimates 1 in 44 children has been identified with ASD, a sharp uptick from 1 in 150 in 2000.

According to Children and Adults with Attention-Deficit/Hyperactivity Disorder (CHADD), more than half of all individuals who have been diagnosed with ASD also have signs of ADHD.

Li has an extensive background in using virtual environments as learning modules to simulate real-world situations.

In a partnership with Augusta University’s Medical College of Georgia, Li used the presence and immersion of virtual reality to simulate Parkinson’s disease, thereby enhancing medical students’ understanding of the disease. In the simulation, participants must perform routine tasks while their hands tremble similar to someone with Parkinson’s disease, which demonstrates the difficulties associated with the disease.

Recently, she collaborated with assistant professor of software engineering Yan Huang on a project that used virtual reality to teach middle school children the basics of cybersecurity. The project featured six education modules, including ransomware, hacking ethics, cipher/decipher, digital footprints, authentications, and phishing attempts. 

Li also serves as director of KSU’s Realities Lab, which provides students with an environment to work with the latest technology in a virtual space.

Li is nearing completion of her current project and said she will process the results through data analytics methods and seek publications to share the results.

“From the data, we noticed there were some surprising results that we did not anticipate,” Li said. “But you know, science is not going forward if everything is working as expected. So, the interesting findings will provide new insights in the field and may lead to more research projects and grant opportunities.”