Two game-like, computerized devices are seeking FDA approval to make the diagnosis and treatment of ADHD—one of the most common childhoodmental health concerns—a digital phenomenon.

For diagnosis, a virtual reality tool known as the Virtual Classroom was submitted to the FDA by technology company Cognitive Leap in late 2018. If granted approval, it will be the first fully immersive, VR program available to help doctors diagnose ADHD. On the treatment side, startup Akili Interactive recently submitted AKL-T01, a video game specifically designed to improve attention in children with ADHD. It would be the first video game that’s available only by prescription, like a traditional pharmacological intervention.

Both Cognitive Leap and Akili contend that their approaches will change the way ADHD is recognized and managed. Both have spent years collecting data to bolster their claims of effectiveness. But FDA approval may be just the first of several real-world barriers to overcome in launching new treatment modalities.

Digital Diagnosis

Commercial virtual reality regularly transports users to fictional worlds; Cognitive Leap’s VR Classroom generates a more realistic environment. After putting on the headset, a child finds himself somewhere he knows well: sitting at a desk in a classroom at school. Cars drive by outside; paper airplanes glide; classmates sneeze.

The Classroom offers more than immersive graphics, however. It measures attention and inhibition via a neuropsychological assessment called a continuous performance test (CPT), explains neuroscientist Crystal Goh, Cognitive Leap’s chief scientific officer. Children are instructed to press a button when certain letters appear on the virtual chalkboard; they’re scored based on correct hits, incorrect hits, and reaction speed. During the 13-minute task, the headset clocks how often they look toward the digital distractions, and additional sensors on the child’s body track fidgeting. Both behaviors factor into the final score.

The environment has been designed to measure multiple ADHD-related symptoms at once, and the maker claims it is more realistic and engaging than monotonous two-dimensional CPTs, while remaining highly controlled. “The child believes that she is in a classroom,” Goh says. “Because she’s using her body, her sensory motor cortex is engaged in the same way as if she were really experiencing that setting. By purposefully manipulating the distractions, we know exactly at what point the child is reacting. We’re not looking just at the traditional CPT score.”

After the task is complete, a report outlines how the child’s self-control, focus, hyperactivity, and other impairments differ from those of a typically developing peer. It doesn’t provide a definitive yes or no on whether a child has ADHD, Goh says; rather, the data, by pointing out specific attentional strengths and weaknesses, can be used to determine the probability that a diagnosis is warranted.

Why VR?
The Classroom aims to bring objectivity to a notoriously subjective diagnostic process, says Cognitive Leap CEO Jack Chen. Chen founded the company to bring treatment options to China, where attentional problems, like many mental health matters, are undertreated and widely stigmatized. He then decided that the United States has its own need for improved diagnostic tools.

“The diagnosis of ADHD or attention issues takes a huge amount of training,” he says. “Even with training, there’s a well-known lack of objectivity.” Since symptoms can be intermittent and hard to quantify, current best practices rely on standardized rating scales filled out by parents and teachers to assess inattention and impulsivity in various domains. The scales require a significant time investment, and because they’re based on memory and subjective interpretations, ratings are subject to human error.

The lack of objectivity and the limited technology available to clinicians inspired psychologist Albert Rizzo, now director of medical virtual reality at the University of Southern California and an advisor to Cognitive Leap, to build a primitive version of the Classroom in 1999. “We started to test it with kids who had ADHD and kids who were definitively neurotypical, based on good neuropsychological evaluations,” he says. “We saw striking differences in performance—[particularly] when distractions were added in.” Though CPTs have been used to measure attention (and to diagnose related disorders) for decades, the Classroom incorporates additional measures of distractibility and hyperactivity, Rizzo says.

He claims he has replicated his original findings in dozens of studies since. As he researched, VR tools improved in quality and decreased in cost, he adds. “The dream that I’ve had [for decades] is now doable.”

Once Cognitive Leap partnered with Rizzo to commercialize the Classroom and seek FDA approval, they initiated a series of trials to test the tool’s efficacy against traditional diagnostic methods and to build a bank of normative data.

Across multiple locations, the company has collected data from approximately 1,500 children between the ages of 5 and 12—500 with ADHD and 1,000 without any psychiatric or behavioral disorders, as determined by standardized neuropsychological batteries. According to Goh, the Classroom’s ability to identify impaired levels of attention and hyperactivity is comparable to traditional assessment procedures that can take hours or days. “As our data pool increases, we can increase the accuracy,” she adds.

Do People Want A New Diagnostic Tool?
Cognitive Leap views VR as a game-changer, but it isn’t the first company that’s tried to upgrade assessment. Over the years, “there have been a number of attempts to bring in more objective diagnostic tools,” says David Rabiner, a psychologist who researches ADHD at Duke University. He points to simpler CPTs, as well as a quantitative EEG measure that uses brainwave patterns to assess attention levels. Although several of these secured FDA clearance, “none of them have been adopted in any meaningful way,” he says.

Rizzo and Goh contend that parents apprehend the Classroom’s value once their child tries it. The bigger hurdle may be clinicians, who are short on time and sometimes ill-trained in ADHD, Rabiner says. Even if the FDA clears the Classroom, he predicts that Cognitive Leap will have “a hell of a hard time” persuading doctors to adopt it, in large part because no major medical associations recommend the use of such “objective” diagnostic measures.

And rating scales, despite their faults, remain the gold standard because ADHD often manifests as academic and social challenges that extend beyond symptomatology. “Certainly parents crave something more ‘objective’ than what a teacher has to say,” Rabiner says, which may explain why they’re attracted to high-tech tests. “But if you don’t know what’s really happening in the classroom, then you’re in the dark.”

Rizzo bats down fears—not unique to VR—that any new tech will render psychologists obsolete. “I see this as a more precise, controlled way to assess function,” he says. “It’s not going to eliminate everything else. There’s always going to have to be a human in the loop who can use this data—not just as an assessment but as a way to come up with a better treatment plan.”

Technological Treatment
In the eyes of many who study ADHD, the “best” treatment plans—at least since Ritalin’s introduction in 1960—often involve medication. But shortly before Akili Interactive was founded in 2011, cofounder and CEO Eddie Martucci observed “a pullback from the traditional pharma players and less research going into new therapeutics.” ADHD patients began to push back against medication as the dominant approach to treatment, he says.
There was research at this time into video games that helped rather than harmed the brain—particularly a program called NeuroRacer, designed by neuroscientist Adam Gazzaley of the University of California, San Francisco to strengthen cognitive control. That program and others inspired Martucci and Akili cofounders to seek the first “digital treatment” for ADHD. AKL-T01 is their flagship product; similar games in the pipeline could one day target symptoms of autism, depression, and multiple sclerosis.

Designed in collaboration with Gazzaley—now Akili’s chief scientific advisor—the game’s true objective is to hone focus by forcing players to juggle multiple goals while tuning out deliberately deployed distractions. In many ways, Martucci notes, it’s been designed to “feel familiar” to a child and to play much like a commercial video game, with rewards, music, and high-quality graphics. Players move rapidly through different worlds and react to stimuli flashed on the screen, Gazzaley says, requiring them to switch quickly between tasks and ignore extraneous information. “This goal-directed attentional control uses the prefrontal cortex and its networks with the rest of the brain,” he says. EEG data taken from older adults and children with sensory processing disorder indicate that playing AKL-T01 may increase midline frontal theta activity, a measure of prefrontal cortical engagement that has been associated with improved attention.

Akili tested AKL-T01 in a proof-of-concept study, published in PLOS One in January 2018. Forty children with ADHD and 40 without played the game five days a week, 30 to 45 minutes at a time, for 28 days. At the end of the study, those with ADHD showed significant improvements on the Test of Variables of Attention, or TOVA, an FDA-cleared CPT that measures attention and inhibition; children whose initial scores were poorest showed the largest gains, while children without ADHD saw no notable changes.

But whether the improved CPT scores translate to better performance at home or school—of paramount interest to most parents of children with ADHD—wasn’t firmly established. And ultimately, FDA approval hangs largely on a different study, a clinical trial known as Software Treatment for Actively Reducing Severity of ADHD (STARS for short). The randomized controlled trial of the game’s efficacy studied nearly 350 children who had been diagnosed with ADHD. Participants were randomly assigned to play AKL-T01 or a similar-looking control game at the same frequency as the proof-of-concept study (the “dose,” 25 minutes a day, was slightly decreased).

As in the proof-of-concept study, the game was shown to have statistically significant effects on attention and inhibitory control as measured by TOVA scores. But participants in the STARS trial were also assessed for real-world symptom changes using rating scales—and though scores on the scales did improve for the AKL-T01 group, the improvement was the same for control subjects.

Akili doesn’t see the lack of differentiated scores on rating scales as cause for concern. “The real-world transfer that I expect to see—like how well a child deploys attention when reading a book, for example—isn’t really sampled in those outcome measures,” Gazzaley says. “[Rating scales] are a lot more responsive to hyperactivity changes, rather than to the actual attentional abilities of a child.” He argues that larger studies will highlight real-world metrics more reflective of the type of cognitive improvements the treatment is aimed at.

Potential Pitfalls
Martucci believes FDA approval will distinguish AKL-T01 from other “brain-training” programs claiming to improve attention, memory, or overall cognition. But not everyone is convinced such a distinction exists—at least, not without more evidence or full release of the data gathered so far. (The complete results of the STARS study will eventually be published in a peer-reviewed journal, Martucci says; to avoid looking as if they’re attempting to influence the FDA’s ongoing review, Martucci and other Akili spokespeople decline to discuss much of its data publicly at this time.)
Dan Simons, a psychologist at the University of Illinois who assesses brain-training claims, points out that Akili’s marketing materials highlight just two peer-reviewed papers: the proof-of-concept study and another published in Nature in 2013

The latter found that older adults between the ages of 60 and 85 who played NeuroRacer saw improvements in memory and focus and began to show brainwave patterns similar to control subjects in their 20s, who hadn’t experienced age-related cognitive decline. Though the paper triggered widespread news coverage, it was criticized by some in the field (including Simons) for small samples, imperfect control conditions, and oversold results.

Simons, who authored a large, damning review of brain-training research in 2016, says that generalization across populations is a common trap in the literature. Martucci counters that “the types of benefits that were shown in those older adults mapped closely to the core deficits in ADHD.”

But because both the STARS and the proof-of-concept studies relied on the Test of Variables of Attention, a computerized continuous performance test, they may struggle to provide the same real-world data as do rating scales, Rabiner says. “It’s always easier to find improvements on highly controlled measures like a CPT,” he explains—and in fact, some researchers hypothesize that the score of anyone taking the TOVA more than once will improve. “But what does that actually mean in terms of kids’ functioning?”

The game’s effects on real-world performance haven’t yet been demonstrated, Simons argues—and until they are, it may be premature to promote AKL-T01 as a treatment. “The game isn’t likely to cause direct harm,” he observes. “But it presents an opportunity cost.” It could be a waste of patients’ time, energy, and money.

Martucci acknowledges that families dealing with ADHD have been burned before by tools that promised improved focus but didn’t deliver. If FDA approval is granted, “It’s going to be incumbent on us to be very clear about what we can promise,” he says. “We believe we’ll be a potential treatment for some patients. For [others], we won’t be the right option—and that’s OK.”

Akili may have an advantage over Cognitive Leap—it fits more closely with clinicians’ established habits. Says Rabiner, “They’re trying to get doctors to do something they’re used to doing: writing a prescription.”

“Psychologists are traditionally conservative about tech,” Rizzo observes—which means both Cognitive Leap and Akili face an uphill battle. Still, he believes, initial skepticism about VR has started to fade, which gives him hope that clinicians are open to change. “My vision is to drag psychology kicking and screaming into the 21st century, for its own good,” he says. “There’s certainly more research that needs to be done. But if you think this technology doesn’t have value or that it’s going to go away, you’re going to be left behind.”