Purpose
This study aimed to evaluate the effects of health interventions using virtual reality (VR) on improving knowledge, attitudes, and skills; and inducing behavioral change among adolescents.
Methods
This study is a systematic review and meta-analysis following PRISMA guidelines. We searched Cochrane, MEDLINE, Embase, CINAHL, Scopus, Web of Science, and Korean databases between database inception and April 10, 2021. Based on heterogeneity, a random- or fixed-effects model was used, as appropriate, to calculate effect sizes in terms of the standardized mean difference (SMD) and odds ratio (OR). Studies were selected if they verified the effects of health education using VR on adolescents; there was an appropriate control group; and if the effects of education were reported in terms of changes in knowledge, attitudes, skills, or behaviors.
Results
This analysis included six studies (n = 1,086). The intervention groups showed greater responses in knowledge and attitudes (SMD = 0.57, 95% confidence interval (CI) [0.12 to 1.02]), skills related to health behavior (SMD = -0.45, 95% CI [-0.71 to -0.19]), and behavioral change after 12 months (OR = 2.36, 95% CI [1.03 to 5.41]).
Conclusion
The results confirm the effectiveness of health interventions using virtual reality (VR). Although the analysis include a small number of studies, a case can be made for health interventions using VR to be utilized as educational methods and strategies to prevent risky behaviors among adolescents.
This study aimed to evaluate the effects of health interventions using virtual reality (VR) on improving knowledge, attitudes, and skills; and inducing behavioral change among adolescents.
This study is a systematic review and meta-analysis following PRISMA guidelines. We searched Cochrane, MEDLINE, Embase, CINAHL, Scopus, Web of Science, and Korean databases between database inception and April 10, 2021. Based on heterogeneity, a random- or fixed-effects model was used, as appropriate, to calculate effect sizes in terms of the standardized mean difference (SMD) and odds ratio (OR). Studies were selected if they verified the effects of health education using VR on adolescents; there was an appropriate control group; and if the effects of education were reported in terms of changes in knowledge, attitudes, skills, or behaviors.
This analysis included six studies (n = 1,086). The intervention groups showed greater responses in knowledge and attitudes (SMD = 0.57, 95% confidence interval (CI) [0.12 to 1.02]), skills related to health behavior (SMD = -0.45, 95% CI [-0.71 to -0.19]), and behavioral change after 12 months (OR = 2.36, 95% CI [1.03 to 5.41]).
The results confirm the effectiveness of health interventions using virtual reality (VR). Although the analysis include a small number of studies, a case can be made for health interventions using VR to be utilized as educational methods and strategies to prevent risky behaviors among adolescents.
Adolescence is an impressionable and important stage of life that lays the foundation for good growth and development. Adolescents learn desirable health patterns in diet, physical activity, peer relationships, and stress management; however, they are also vulnerable to risky behaviors and consequences, such as drinking, smoking, drug usage, unsafe sex, and pregnancy [1, 2]. In a study of 148,839 adolescents, the prevalence of a healthy lifestyle was found to decrease as age increased; as adolescents became older, they engaged in less physical activity, consumed fewer fruits and vegetables daily, spent more time on screen-based activities, and showed a decreased abstinence from alcohol and tobacco consumption [3]. A stratified sample of 54,948 middle and high school students was also found to have more unhealthy lifestyle factors with increasing age: 37.5% of male and 29.1% of female students in Korea had experience consuming alcohol, and their smoking rates were 6.0% and 2.7%, respectively. Furthermore, 23.6%~27.0% of them reported consuming fast food more than thrice a week, and they spent 3.8 hours per weekday and 5.9 hours on weekends sitting unrelated to study [4]. As experiences during adolescence shape feeling, thinking, and decision-making until adulthood, it is critical to teach adolescents to promote health throughout their lifespan.
One of the factors with the strongest influence on adolescent lifestyles is utilization of the Internet and online communication via smartphones. Adolescents today are considered to be “digital natives,” meaning that they cannot remember a time when digital media were not all around them [5]. The development of information and communications technology (ICT) has provided unhindered access to a variety of multimedia content and changed adolescents’ lives. One of the main technologies in this regard is the smartphone. Recent studies have found that 92.0% and 93.5% of Korean middle and high school students, respectively, own smartphones [6] and use them primarily to consume media content for interpersonal communication via social network service (SNS) [7, 8, 9]. Undoubtedly, rapid changes in the media landscape have altered marketing patterns to suit adolescents. Initiation of tobacco and alcohol consumption, overweight and obesity, and sexual behavior are known to be influenced by the media [9].
Nonetheless, technology plays a role in influencing healthy decision making and motivating the use of health-promoting interventions, a notable example of which is virtual reality (VR). VR is a non-, semi-, or fully immersive computer simulation of a real or virtual place, in which the user becomes part of a created environment, performs tasks, or interacts with virtual objects and avatars in real time [10]. Various health education programs utilizing VR have recently been conducted among adolescent populations on pedestrian safety for accident prevention [11], exercise training [12], sexual risk behavior intervention [13], and suicide prevention [14]. As adolescents are familiar with ICT-based environments [15], VR simulations that provide a special presence and sensation of immersion may be more effective than existing strategies [16] to make adolescents interested in health issues and eventually induce them to engage in healthy behaviors.
Despite these trials, research has yet to establish whether VR-based health intervention programs are effective for adolescents. Some researchers still express concerns about the use of VR training in medical education as it may be viewed as less effective than traditional means of learning [17]. Few studies have considered the effects of VR on adolescent health. Overall, evidence for the effectiveness of health education using virtual reality is unclear. This study aimed to evaluate the effects of VR-based health interventions on adolescents. Specifically, it aimed to systematically review and meta-analyze the existing literature on the effects of VR-based health education on changes in knowledge, attitudes, skills, and behaviors and to compare the effect sizes of the included studies.
This systematic review and meta-analysis was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guideline [18].
Searches were performed in the MEDLINE, Embase, Cochrane Central Register of Controlled Trials (CENTRAL), CINAHL, Scopus, Web of Science, and Korean databases (KoreaMed, KMBASE, RISS, KISS, and Science ON-NDSL) between database inception and April 10, 2021. Search terms related to VR interventions and adolescents were used, including combinations of Medical Subject Heading terms, Emtree headings, and free-text words. The combined keywords and text words for adolescents, teenagers, virtual reality, and interventions were used. The full search strategy in MEDLINE was (((adolescent* OR teen* OR youth*) OR ((junior OR high OR middle) adj3 school student*)) AND ((virtual realit* OR VR OR VRET) AND (program* OR education* OR exposure* OR treatment* OR therap* OR intervention))). Additional studies were performed by screening the reference lists.
Studies were selected according to the population, intervention, comparison, and outcome (PICO) framework. Eligible studies (1) included adolescents without diseases, (2) evaluated health education using VR methods, (3) compared the intervention with usual care, other care, or no treatment, (4) reported outcomes of interest (e.g., knowledge, attitudes, skills, and behavioral changes), and (5) were published in English or Korean. The literature included in the final analysis was decided by three researchers based on the inclusion and exclusion criteria, and conflicting opinions were reconciled through discussions. The study designs were limited to randomized controlled trials (RCTs) and non-RCTs. Because VR-based interventions have mainly been performed since 2000, only papers published after 2000 were considered. Data screening was performed from April 10, 2021, to April 25, 2021.
Data collection and extraction took place from May 10, 2021, to May 30, 2021. We extracted data from the studies according to the Cochrane recommendations [19]. In each included study, the background (authors, publication year, country), information on participants (number of participants, eligibility), information on intervention (content, duration), comparators, and outcomes (knowledge, attitude, skills, behavioral changes) were extracted. Data extraction was conducted by three authors and discrepancies were resolved through discussion. The authors were contacted via email when outcome-related data were missing or unclear.
The internal validity of the included RCTs was assessed using the Cochrane Collaboration’s Risk of Bias 2.0 (RoB 2.0), which was based on the randomization process, bias in the intended interventions, missing outcome data, outcome measures, and selection of reported outcomes [20]. The risk of bias for the items overall was marked as low, with some concern, or high. For non-RCTs, the risk of bias was assessed using the Risk of Bias in Non-Randomized Studies of Interventions (ROBINS-I) tool. These include bias due to confounding factors, bias in the selection of participants in the study, bias in the classification of interventions, bias due to deviations from intended interventions, bias due to missing data, bias in measurement of outcomes, and bias in selection of the reported results [21]. The risk of bias for each item was marked as low, moderate, serious, or critical. The three authors evaluated the risk of bias and reconciled conflicting opinions through discussion.
The effects of the health interventions using VR were analyzed using RevMan ver. 5.3.3 (the Cochrane Collaboration, 2014. Nordic Cochrane Centre, Copenhagen, Denmark) and the Comprehensive Meta-Analysis version 2.2. As a measure of the effect size, the standardized mean difference (SMD) was calculated based on the scores of the intervention and control groups. The odds ratio (OR) was estimated for binary data. All tests used 95% confidence intervals (CIs) with a statistical significance level of p < .05.
A fixed- or random-effects model was selected to calculate the pooled effect measures for the included studies. If homogeneity was confirmed, the analysis was performed using a fixed-effects model, which assumes that the effect size of the population is the same and that the difference in the effect size of each study is due to sampling error. If heterogeneity was identified, a random effects model was used. The Mantel-Haenszel method and general inverse variance estimation method were applied to synthesize the results. The heterogeneity of the studies was tested using the Higgins I2 statistic, and I2 values of 25.0%, 50.0%, and 75.0% indicated low, moderate, and high heterogeneity, respectively. To further explore heterogeneity, pre-specified subgroup analyses were conducted according to outcome variables or program content. Although it was not possible to quantitatively assess publication bias with the small number of studies included in the meta-analysis (n < 10), we evaluated potential bias using funnel plots and Egger linear regression values for each outcome. A sensitivity analysis was performed to assess the robustness of the conclusions of the meta-analysis. Some parts violate the assumption of independence when calculating the overall effect size. However, because of the limited number of documents included in the analysis, the loss of information was minimized using the effect size as the unit of analysis.
The requirement for informed consent was waived by the Ethics Committee of Yonsei University Bioethics Review Committee (IRB No. CR321319) because there was no sensitive information, and the survey was anonymously conducted.
A literature search yielded 2,015 citations. After duplicates were removed, there were 1,347 references. During screening, 1,100 studies were excluded based on the selection criteria (351 after title screening and 749 after abstract screening). Of the remaining 247 studies, 241 were excluded based on the inclusion and exclusion criteria. Therefore, six studies [22, 23, 24, 25, 26, 27] met the selection criteria and were included in the final analysis (Figure 1).
Figure 1
Flowchart of the process of study selection for the systematic review and meta-analysis.
Evidence of health interventions using VR has been summarized in six studies. The major characteristics of the six studies [22, 23, 24, 25, 26, 27] are summarized in Table 1. Four RCTs [22, 24, 25, 27] and two non-RCTs [23, 26] with 1,086 participants were included (Table 1). The mean age of the subjects in the individual studies ranged from 12.9 to 16.0 years old. The target population recruited in the studies was predominantly healthy teenagers (e.g., never smokers), although one study included participants who had smoked at least one cigarette within 30 days. The publication years of the identified studies ranged from 2003 to 2020. Three studies were published in the United States [24, 25, 27]; the other countries included in the study were Korea, the Netherlands, and Sweden [22, 23, 26].
Table 1
Main Characteristics of Included Studies
In terms of content, interventions using VR mainly include smoking cessation [25, 27], sexual health education [24, 26], and cardiopulmonary resuscitation (CPR) [22, 23]. The duration of the interventions ranged from a day to six months, with half- to two-hour session and one to seven sessions. Knowledge was measured in three studies [22, 24, 26], attitudes in two [24, 25], skills in three studies [22, 23, 24], and behavioral changes in one [27]. The results of the knowledge and attitude scores are presented as mean values and standard deviations. Skills were evaluated by comparing the incorrect and correct answers with those of the control group. The results of the behavioral changes are presented using the improvement rate.
Four studies [22, 24, 25, 27] compared the outcomes of a VR exposure intervention group with those of other treatments or conventional methods (two-dimensional videos, role-playing, or online counseling). In one study [26], the outcomes of VR teaching were compared with those of no treatment. One study [23] compared the outcomes of a combination of VR and conventional interventions with those of the conventional intervention alone. Studies on behavioral changes were analyzed according to the follow-up period, and the data were analyzed by considering each outcome index as a separate study.
Figure 2 shows the results of the risk of bias assessment for the studies included in the analysis. Two RCTs [22, 24] were considered to have low risk of bias, while the other RCTs [25, 27] had some concerns regarding bias (Figure 2A). One nonrandomized study [23] was deemed to have low risk of bias, and another [26] was deemed to have moderate risk of bias. Overall, most studies were evaluated to raise some concerns in at least one domain, but were not at a high risk of bias.
Figure 2
Assessment of risk of bias in the included studies. (A) Randomized controlled trials (RCTs), (B) non-RCTs.
The effects of using VR in health education on adolescents in terms of knowledge and attitudes (n = 7), skills (n = 5), and behavioral changes (n = 1) are shown in Figure 3.
Figure 3
Forest plot of the meta-analysis for virtual reality interventions to prevent health problems in adolescents. Each study is identified by the first author. The individual effect sizes are identified as the SMD with lower and upper limits (95% CIs). The overall summary effect size of the meta-analysis is noted as a diamond on the bottom line. (A) Knowledge and attitudes, (B) skills (1: CPR training, 2: sexual health education program), (C) behavioral change.
SD = Standard deviation; SMD = Standardized mean difference; CPR = Cardiopulmonary resuscitation; CI = Confidence interval; OR = Odds ratio.
Summary estimates from the meta-analysis of knowledge and attitudes in four studies [22, 23, 24, 26] showed a statistically significant difference between the intervention and control groups (SMD = 0.57; 95% CI 0.12 to 1.02). However, heterogeneity was high (I2 = 84.0) (Figure 3A).
A subgroup analysis of each indicator showed that knowledge was reported in three studies [22, 23, 24]. A statistically significant difference was found between the intervention and control groups in terms of knowledge (SMD = 0.54; 95% CI 0.08 to 1.00; p = .020, I2 = 61.0). According to one study, a non-significant effect was found for attitudes (SMD = 0.06; 95% CI -0.37 to 0.49; p = .790) [24]. Knowledge and attitudes showed a positive effect (SMD = 1.10; 95% CI 0.12 to 1.02; p < .001) in one study [26] (Figure 3A).
The overall pooled estimate (SMD = -0.45; 95% CI -0.71 to -0.19; p = .001) from three studies [22, 23, 24] for skills (measured in terms of errors) showed a significant difference between the intervention and control groups. However, heterogeneity among the studies was high (I2 = 81.5).
As a result of the subgroup analysis according to educational content, a statistically significant difference was found for the outcomes of CPR training (SMD = -1.62; 95% CI -2.23 to -1.02; p < .001; I2 = 0) in one study [24]. However, a sexual health education program did not show a positive effect (SMD = -0.19; 95% CI -0.48 to 0.09; p > .05), with somewhat heterogeneous results (Figure. 3B).
One study [27] reported behavioral changes (smoking cessation) in a VR-exposure group. The effect estimate showed improvement in the abstinence rate in the intervention group (OR = 2.36; 95% CI 1.03 to 5.41; p < .05).
Publication bias was evaluated using a funnel plot and the results were unbiased and evenly distributed within the triangles (Figure 4). There was also no possibility of publication bias in the Egger’s test (p < .05). Therefore, the risk of publication bias was considered to be low.
Figure 4
Funnel plot of the effects of virtual reality interventions. (A) Knowledge and attitudes, (B) skills.
The pooled SMD was 0.45 to 0.69 for knowledge and attitudes and -0.41 to -0.89 for skills related to the prevention of health-related problems. These results were broadly concordant with those of the primary analysis. Excluding individual studies from the sensitivity analysis did not affect the initial effect size estimates. These results were broadly consistent with those of the primary analyses.
Despite the relatively small number of studies included, this meta-analysis generated meaningful results. Furthermore, as far as we know, this is the first meta-analysis that evaluated the effectiveness of health interventions using VR for healthy adolescents. VR-based interventions were found to be more effective than interventions that did not use this technology.
Education is a strong factor in adolescent health, and new technologies offer the possibility of a healthier generation of adolescents. As ICT-based models have been adopted for health education, VR, which was initially used for military training [28] is now being applied to various health issues. Thus, this meta-analysis was conducted to identify the effectiveness of VR-based interventions for adolescents and to identify future avenues for their expansion in health education.
Among the six studies analyzed, three were recently published RCTs (since 2017), implying a current trend in VR utilization, as well as future expansion to broader situations with advances in simulation technology. This study focused on the outcome measures of knowledge, attitudes, skills, and behavioral change when evaluating the effectiveness of VR-based interventions because healthcare providers need these attributes to foster engagement with adolescents [9]. Despite the limited number of included studies, interventions using VR for sex education, smoking, and resuscitation were effective in increasing knowledge in these content areas. This finding, which is consistent with previous studies [29, 30], suggests that vivid VR situations activate learners’ interests and motivation and enhance their cognitive ability to recall the health information to which they are exposed [31].
Skill is a prerequisite for fostering behavioral changes; it encompasses not only direct skills related to educational content but also learners’ perceived self-efficacy [32]. Two studies in this analysis indicated that VR was significantly effective in providing CPR training by immersing learners in the relevant procedures. VR-based education for skills training has been applied to the areas of laparoscopic surgery in medicine, training for scrub nurses in operating rooms [33], and tracheostomy care [34], and its effectiveness has been demonstrated. A systematic review also indicated that the use of VR has equaled or surpassed traditional skill acquisition methods [35]. Thus, VR-based education is a reliable modality for enhancing learners’ skills.
An intervention on sex education measured skills but showed an insignificant effect [24]. As adolescents have already been exposed to extensive sexual content including pornography in the cyber-world [9], VR interventions seem to have a limited ability to influence their skills in the domain of sexuality, even though short-term application. However, virtual sex education programs radically change the dynamics of sexual knowledge, unlike parental or religious norms [36], and can be customized to reflect cultural, social, and other user preferences [37]. Recent studies have also recommended tailored guidance and advice on sex education using virtual simulations because of their enhanced capability and effectiveness through consistent interactions with experts [38]. Moreover, repeated application of VR scenarios [39] is advantageous for improving sexual health-related skills.
Health education aims to promote healthy behaviors and ultimately help sustain desirable changes. However, smoking cessation education often fails to demonstrate lasting effects. The relapse rate of smoking among 184 adolescents was 6.8%, which was higher than that among older individuals [40]. A previous meta-analysis also reported relapse rates ranging from 5.0% to 17.0% one year after smoking cessation [41]. Meanwhile, VR interventions could be used repeatedly and as a booster, not only to induce immediate behavioral changes but also to sustain these changes. The parallel application of VR experiences through avatars and real-time counseling through cyber chatrooms are also known to have significant effects [10, 23, 25, 27]. In fact, the additional application of a booster program was effective in maintaining smoking cessation among high school students [42]. In addition to the effects of adding counseling services to virtual environments [43], the group in Woodruff’s study [27] that received an intervention program combining VR with a real-time motivational interview with a smoking cessation counselor showed a statistically significantly higher smoking cessation rate (OR = 2.36; 95% CI 1.03 to 5.41) than the group that received conventional education. Furthermore, a meta-analysis demonstrated that real-time chatting services have significant effects on customer behavioral intentions and quality improvement [44]. Thus, the application of VR with additional or hybrid strategies is thought to enhance the effects of health education, particularly for adolescents.
VR is being utilized as a notable educational strategy in various fields. In medical education, VR-based health interventions conducted among adolescents are mostly about illness and symptom management, such as attention deficit hyperactivity disorder [45], internet gaming disorder [46], and dressing changes in patients with burns [47]. However, during adolescence, the desire for novelty increases along with the curiosity and courage to experiment with new and often unhealthy behaviors [48]. In this sense, VR-based health interventions are particularly useful because they involve content that could be dangerous to apply directly to the human body or that would be difficult to simulate in reality, such as suicide prevention [14], disaster situations [49], or claustrophobia [50]. Overall, VR-based health interventions are considered a good strategy, given the advantages of presence, immersion, and repeatability [16]. The positive effects of VR-based education are linked to the interest of adolescents as members of Generation Z in the gamification of education and prompt feedback on their results, which encourages immersion in the program [51].
Nonetheless, more research is needed on the effects of VR on the outcomes of interest at different levels of immersion and interaction. Most studies that evaluated the outcomes in our review reported incomplete or incomparable outcome data, thus primary studies focusing on these outcomes are needed. Finally, as most of the included studies reported post-intervention scores rather than changes in scores for outcomes, there is a need to standardize the methods to report the most meaningful and accurate data for outcomes.
This study has several limitations. Potential bias was assessed using funnel plots and Egger’s linear regression values, but a small number of studies were included in the meta-analysis.
Nursing education needs to adopt the current changes in VR technology and media to meet the needs of students and achieve the best learning outcomes. Furthermore, as health educators, nurses must identify and implement strategies and methods to strengthen the effectiveness of health education for both patient and public health.
This systematic review and meta-analysis included six studies and attempted to identify the effects of VR programs on adolescents. Studies have also shown that health education using VR is effective for adolescents. In particular, VR programs can be useful in improving health promotion–related knowledge, skills, and behavioral changes. Therefore, health interventions using VR can be utilized as educational methods and strategies to prevent risky behaviors in adolescents.
CONFLICTS OF INTEREST:Kim, Gaeun has been the Editorial Board Member of JKAN since 2022 but has no role in the review process. Except for that, no potential conflict of interest relevant to this article was reported.
FUNDING:This study received financial support from a National Research Foundation of Korea grant funded by the Korean government (2021R1A2C1003083).
AUTHOR CONTRIBUTIONS:
Conceptualization or/and Methodology: Park SM & Kim G.
Data curation or/and Analysis: Chung CW & Kim G.
Funding acquisition: Park SM.
Investigation: Chung CW.
Project administration or/and Supervision: Park SM.
Resources or/and Software: Kim G.
Validation: Park SM.
Visualization: Kim G.
Writing original draft or/and review & editing: Park SM & Chung CW & Kim G.
None.
Please contact the corresponding author for data availability.