Pieri, L., Tosi, G., & Romano, D. (2023). Virtual reality technology in neuropsychological testing: A systematic review. Journal of Neuropsychology. https://doi.org/10.1111/jnp.12304
Key Points
The paper presents a systematic review of the use of virtual reality (VR) technology for neuropsychological assessment. The key findings are:
- VR can improve ecological validity of neuropsychological tests by simulating realistic environments and tasks. However, few tools have robust evidence of clinical utility so far.
- Most VR assessment tools are still in the early research stages without normative data or evidence of psychometric properties. Only 3 tools seem ready for clinical use.
- VR tests have been developed across cognitive domains like memory, attention, executive functions, spatial navigation, and activities of daily living. But language functions are relatively less studied.
- The main advantages of VR tests include experimental control, multisensory stimulation, objective scoring, and the potential to motivate test takers.
- Challenges include the risk of cybersickness, high development costs, and the need for technological skills. Suitability and accessibility for patients with significant functional difficulties are also a concern.
The paper makes a case for VR tests as a complementary evaluation methodology alongside traditional tests. But, it also highlights the need for further evidence of ecological validity, clinical utility, and accessibility before widespread implementation.
Rationale
Prior research has questioned the ecological validity of paper-and-pencil neuropsychological tests in relating test performance to real-world functioning (Chaytor & Schmitter-Edgecombe, 2003).
VR technology offers greater immersion and control to simulate realistic environments and tasks. This could help address concerns over ecological validity.
Further, VR setups allow the collection of objective digital measures like reaction times and motion data that may inform assessment.
However, despite increasing research on VR assessment tools over the past two decades, few studies have presented complete psychometric evidence or normative data to support clinical adoption (Climent et al., 2021; Iriarte et al., 2016). The systematic review addresses this research gap.
Method
The authors searched three databases – PubMed, Web of Science, and PsycINFO using the terms “virtual reality” and “neuropsychological assessment.”
They included peer-reviewed English articles from 2000-2021 describing VR paradigms for neuropsychological testing.
Case studies, reviews, rehabilitation protocols, and non-VR tests were excluded.
Data extracted included sample details, cognitive domain, test availability, normative data, and technological specifications. Tests were classified as immersive or non-immersive.
Sample
The reviewed studies had varying sample sizes ranging from single cases to over a thousand healthy and clinical participants. The tools showing adequate normative data had sample sizes between 243 to 1469.
Statistical measures
Both descriptive and inferential statistics were employed by studies developing and validating VR tests.
Correlational analyses were frequently done to demonstrate convergent validity with traditional tests.
Between groups comparisons helped establish discriminant validity and sensitivity.
Results
The review process resulted in 287 eligible studies, with 134 published since 2016, highlighting increasing research attention on immersive VR tests.
However, only 3 tools provided robust psychometric profiles and normative data for clinical consideration – Nesplora AULA, Aquarium, and Systemic Lisbon Battery. These tools assessed attention and memory.
Spatial navigation, executive functions, attention, and memory were the most researched domains. However, language functions were relatively less studied through VR tests.
Both simulated adaptations of traditional tests and virtual equivalents of real-life tasks were designed across domains.
Insight and Depth
The systematic review offers useful insights into the emerging landscape of VR-based neuropsychological assessment.
It highlights the spectrum of cognitive domains where digitally rendered simulations offer value.
At the same time, it points to the significant research-to-clinical translation gap despite two decades of preliminary feasibility studies.
Strengths
- The review methodology is a key strength, with a thorough search strategy across databases and systematic inclusion approach guided by PRISMA guidelines.
- Both immersive and non-immersive tools are covered to represent the state of the art.
- The discussion section effectively relates findings to issues of accessibility, suitability, and validity that have significant clinical implications.
Limitations
- The review is limited to studies published in English which leaves out research in other languages.
- Further, the cognitive domains classified are based on author descriptions, which may sometimes be ambiguous considering the complexity of neuropsychological tasks.
- Reporting positives could have biased included studies.
These limitations can be mitigated in future reviews by including multiple languages and a panel classification of cognitive domains assessed.
Implications
The insights on VR’s advantages over traditional tests can guide the development of more engaging, sensitive, and ecologically valid tools.
But the persisting issues of cybersickness susceptibility and access to advanced VR systems imply that traditional tests will continue retaining value in assessment.
Rather than a substitution, VR tests can complement traditional batteries pending stronger psychometric evidence.
This also highlights the need for greater interdisciplinary collaboration between neuropsychology researchers, human-computer interaction experts, and VR developers to create clinically useful solutions.
Conclusions
The review comprehensively captures the possibilities and challenges of incorporating VR technology in neuropsychological assessment.
VR-based testing paradigms are starting to address concerns over artificial sterile environments of traditional tests.
However, most proposed tools are still far from clinical reality. Converging efforts to conduct robust validation studies on promising tools can help build the 3.0 wave of assessment.
Overall, there is an exciting scope for VR tests to become a mainstay of comprehensive neuropsychological evaluations in the near future. But more problems need to be solved before they can translate from benches to bedsides.
References
Chaytor, N., & Schmitter-Edgecombe, M. (2003). The ecological validity of neuropsychological tests: A review of the literature on everyday cognitive skills. Neuropsychology Review, 13, 181–197. https://doi.org/10.1023/B:NERV.0000009483.91468.f
Climent, G., Rodríguez, C., García, T., Areces, D., Mejías, M., Aierbe, A., Moreno, M., Cueto, E., Castellá, J., & Feli González, M. (2021). New virtual reality tool (Nesplora Aquarium) for assessing attention and working memory in adults: A normative study. Applied Neuropsychology:Adult, 6, 1–13. https://doi.org/10.1080/23279095.2019.1646745
Iriarte, Y., Diaz-Orueta, U., Cueto, E., Irazustabarrena, P., Banterla, F., & Climent, G. (2016). AULA—Advanced virtual reality tool for the assessment of attention: Normative study in Spain. Journal of Attention Disorders, 20(6), 542–568. https://doi.org/10.1177/1087054712465335
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Systematic reviews, 10(1), 1–11. https://doi.org/10.1136/bmj.n71
Learning check
- How can virtual reality technology concretely help improve ecological validity and address the limitations of paper-and-pencil tests? What examples does the review present across cognitive domains?
- If VR-based tests correlate weakly with traditional tests, does it necessarily mean they are not valid? What is the argument made in the paper regarding this?
- What key feasibility issues remain before VR tests can translate from research to clinical practice? What steps can test developers take to address these gaps?
- What disabilities or impairments could make VR-based assessments difficult or unsuitable? How can VR environments be designed to maximize accessibility?
- Considering costs, skills, and risks involved, do you think VR tests can fully replace traditional tests instead of complementing them? What factors did the review highlight regarding this?
