Assessing the Utility of Electronic Measures as a Proxy for Cognitive Ability

Large-scale genomic studies have identified over 100 genes associated with autism spectrum disorder (ASD); however, important phenotypic variables are captured inconsistently. In many cases, the resources required for comprehensive characterization hinder the feasibility of collecting critical information, such as intellectual ability. Thus, electronic collection of important phenotypes would greatly facilitate large-scale data collection efforts. This study assessed the utility of two electronic assessments as a proxy of cognitive ability relative to clinician-administered cognitive assessments. Ninety-two participants completed the study, including individuals with ASD (probands, n = 19), parents of probands (n = 46), and siblings without ASD (n = 27). Participants were administered the electronic-Peabody Picture Vocabulary Test, Fourth Edition (e-PPVT-4), an electronic visual reasoning (VR) test, and a clinician-administered Wechsler Abbreviated Scales of Intelligence, Second Edition (WASI-II). Probands also completed a full, in-person, cognitive assessment and Vineland Adaptive Behavior Scales, 2nd Edition. Correlations between scores on electronic and clinician-administered measures were examined. Classification accuracy of individual scores based on 95% confidence intervals and score range (below average, average, above average) were also assessed. Moderate to strong correlations were identified between both electronic measures and the clinician-administered WASI-II (ρ = 0.606–0.712). Mean difference between standard scores ranged from 10.7 to 14.8 for the cohort. Classification accuracy based on WASI-II 95% confidence interval was consistently low (27.5%–47.3%). Classification accuracy by score range (below average, average, above average) was variable, ranging from 33% to 86% for probands. All participants unable to complete the electronic assessments met DSM-5 criteria for intellectual disability. e-PPVT-4 and VR scores were strongly correlated with scores on the WASI-II full-scale IQ (ρ = 0.630, 0.712), indicating utility of these measures at the group level in large-scale genomic studies. However, the poor precision of measurement across both measures suggests that the e-PPVT-4 and VR are not useful alternatives to in-person testing for the purpose of clinical assessment of an individual’s IQ score.

Lay Summary

Large-scale studies designed to identify genes associated with autism have been successful in identifying over 100 genes. However, important clinical information about participants with autism and their family members is often missed—including cognitive functioning. Cognitive testing requires in-person administration by a trained clinician and therefore can be burdensome and often reduces feasibility of diverse samples. Here, we assessed whether electronic assessments could take the place of in-person cognitive testing. We found that at the group level, for large-scale studies, electronic measures added valuable information; however, they were not accurate enough to be used on an individual level (i.e., to offer feedback about an individual’s predicted IQ score).