Mobile Apps for Sexual and Reproductive Health Education: a Systematic Review and Quality Assessment

The aim of this study was to present the current state of research on mobile health apps for sexual and reproductive health (SRH) education. Apps were analysed based on contents (by using the World Health Organization’s SRH framework), features, intended audiences and quality of evidence (by applying the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach). Taking German sexuality education apps as an example, the rapid development in the field of SRH apps over the last 3 years has been revealed by comparing the quality of apps available in 2019 with apps from 2022. SRH apps allow health information to be disseminated quickly, at low thresholds and in a practical and cost-effective manner. Moreover, they allow for anonymous usage independently of time and place. In the absence of network coverage, offline use is also possible. Previous research focused on individual SRH aspects (e.g. human immunodeficiency virus (HIV), contraception). However, some studies were designed to cover a broader range of SRH topics, but identified only a few relevant apps. To improve SRH, it would be helpful if the apps would be of high-quality design and be made up of relevant content. Furthermore, they should be tailored to the target group and have been tested in real-life settings. A total of 50 SRH apps with sufficiently high-quality ratings were included. The apps cover a variety of SRH topics, but they often lack field-based evaluation. The effectiveness of SRH apps has not yet been sufficiently studied in a scientific manner. Only 9 apps were deemed to be adequate for a moderate GRADE level. Despite this grading, the study nevertheless shows that there are several apps that could potentially promote SRH.


Introduction
To facilitate interactions between physicians and patients, the use of a health app such as Babyscripts might be helpful. Besides providing pregnancy information the Babyscripts app also allows pregnant persons to communicate with their doctors. Furthermore, the application is linked to remote monitoring devices. If any deviating parameters are detected, the app assists by alerting nurses and patients [1 •, 2, 3].
Sexual minorities can also benefit from health apps that are tailored to meet their needs [4]. One example is the myPEEPS app. With interactive components such as gamifications (e.g. quizzes), scenarios as well as role plays, it aims to provide same-sex attracted young males with helpful 1 3 knowledge and skills to reduce their sexual risk behaviour and the risk of them contracting a sexual transmitted infection (STI) [5][6][7][8].
This study focuses on health apps as a specific type of mobile health (mHealth) interventions. mHealth interventions, i.e. interventions delivered via mobile devices, can successfully improve people's health [9]. Compared to conventional interventions, people benefit from mHealth by being enabled to quickly access information at low thresholds and independently of time and place as well as at low cost [9][10][11][12][13][14]. mHealth interventions can also meet the need for anonymity, which is especially crucial when it comes to sexuality [4,14,15].
Health apps can be used to disseminate health information effectively, practically and cost-effectively via mobile phones or tablets [10][11][12][13]16]. Another advantage of native apps in comparison to other mHealth interventions is the possible offline use in the absence of network coverage [4]. In addition to common text message interventions [9], a variety of interactive features can be implemented in apps to positively influence people's health.
This article is based on the current World Health Organization (WHO) definition of sexual health [17]. Therefore, apps that focus on aspects such as sexual well-being are being additionally considered to more conventional topics of sexual health like prevention of human immunodeficiency virus (HIV) or sexual risk behaviours. Sexual and reproductive health [18,19] are closely intertwined [20]. Hence, this study also covers apps that focus mainly on reproductive aspects. For analysing sexual and reproductive health (SRH) aspects, a WHO-framework that distinguishes eight domains of SRH and four socio-structural aspects is used [20].
To promote SRH, apps can provide fundamental knowledge through playful elements (e.g. quizzes) [21][22][23]. Self-assessment and tracking options for symptoms, the menstrual cycle or fertility biomarkers (e.g. basal body temperature, cervical fluid) are other important features of SRH apps [22,24,25]. Users can often get their questions answered through the app (FAQs, chatbots, interactive components) [21,26]. Furthermore, an important element of some apps is the references to offline help services or in-app contact options to experts [21,24].

Aim of the Review
Current app research often focuses on individual SRH aspects (e.g. HIV, pregnancy prevention) [24,27,28]. Other studies already include a wider range of SRH topics but only identify a few relevant apps [29, 30, 31•]. The aim of this study was to present the most researched SRH apps and evaluate their characteristics and outcomes. To identify the apps and the evidence available about them, two types of publications were used: literature reviews (i.e. papers summarising research on mHealth solutions) and app reviews (i.e. papers presenting apps available on common app platforms). Taking German sex education apps as an example, the rapid development in the field of SRH apps is shown.

Methods
The study was conducted in two steps: (1) a systematic literature search to identify SRH apps known in the scientific community followed by (2) an analysis of the selected SRH apps in terms of contents, features, intended audiences and quality of evidence.

Literature Search
The following databases were searched: Pubmed, Embase, PsycInfo, PSYNDEX, PsycArticle, CINAHL, Web of Science, Cochrane Reviews and Cochrane Trials. For this purpose, studies were searched that contained an SRH component (e.g. sexual health, sexual well-being, reproductive health, reproductive well-being) and an app component (e.g. app, application, mobile phone) in title or abstract. The study presents systematic reviews and meta-analysis limited to publications of the last 5 years due to the rapid change in availability of apps [10,21,32,33]. Studies were included if they (a) focused on or addressed SRH components, (b) included smartphone or tablet apps and (c) were written in English or German. On the other side, studies were excluded if they (a) focused on other health-related topics (e.g. mental health) or (b) included other mHealth interventions (e.g. SMS interventions) without mentioning apps or (c) interventions via other apps (e.g. Social Media Platforms). After duplicates removed, 315 studies were screened by title and abstract (for details on study identification, see Appendix 1). By screening all reference lists of included studies, 8 additional studies were identified. Finally, 25 systematic reviews were included in the analysis. In some of the included literature reviews, study protocols of randomised controlled trials (RCT) or articles with ongoing RCTs were mentioned [7,[34][35][36]. In these cases, an additional search for published studies on particular apps was conducted [37][38][39][40][41][42][43][44][45][46].

App Analysis
Apps that achieved at least acceptable quality ratings or were recommended by the study authors were included (for details, see Appendix 2). Apps were analysed with focus on contents, features, intended audiences and quality of evidence. In order to compare findings across studies, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Approach proposed by the GRADE Working Group was used to assess whether the quality of evidence for each app was (a) high, (b) moderate, (c) low or (d) very low [47].

Identification of SRH Apps
No apps were included if studies (a) addressed digital interventions, but did not contain apps [30,54] or (b) did not name individual apps [28,33]. Furthermore, if (c) study authors did not recommend any app [58] or (d) no quality assessment was conducted in the study [24] apps were also not included (see Appendix 2 for information on the studies and the selection criteria for the apps included in the analysis).

SRH Apps
In terms of contents, technical aspects (features), intended audiences as well as quality of evidence 50 apps were analysed (see Table 1 for details and short summaries of relevant findings on each app).

Contents of SRH Apps
Nine apps focused on (a) comprehensive education and information, 3 on (b) gender-based violence prevention, support and care, 9 on (c) prevention and control of HIV and other STIs and only one on (d) sexual function and psychosexual counselling. Covered by 12 apps was (e) antenatal, intrapartum and postnatal care. There were 9 apps on the topic (f) contraception counselling and provision and 20 on (g) fertility care (including trackingapps). No app focussing on (h) safe abortion care could be found. Four apps addressed cultural and social norms around sexuality, among these one was noted to be transformative on gender relations [55 ••]. The remaining socialstructural factors may have been approached indirectly (e.g. gender inequalities can be addressed in gender-based violence prevention apps), but they were not the main app contents or explicit research subjects. Figure 1 shows the distribution of contents according to GRADE levels and intended audience groups.

Features of SRH Apps
Many apps (n = 31) offer different data collection options for users e.g. self-tracking menstruation, managing symptoms by taking notes or in some cases collecting data through other devices via Bluetooth [2,3,60]. Educational information was also provided in many (n = 19) apps. Reminders or push notifications were a feature which 16 apps contained. A few apps featured audio (n = 5) or video (n = 4) content and used gamification (n = 4) or interactive components (n = 4). Some also enabled communication with experts via app (n = 9), provided an FAQ section (n = 6), supported decision-making (n = 6) or had a community section for social support (e.g. forum, n = 4). Sharing data between client and healthcare workers was possible in 3 apps. Only 2 apps mentioned offline care services.

Intended Audiences of SRH Apps
The included apps addressed four target groups: (a) women, (b) children, adolescents and young adults, (c) healthcare workers and/or patients and (d) men. They are presented in Table 1, which Annotations: is structured accordingly to these four groups. For each audience group, one app is described in more detail as an example. For women, 25 apps, including 7 with a focus on pregnant and postpartum persons, were identified. Plan A Birth Control is a tablet app available in English. It informs women about possible contraception options and their effectiveness while they are waiting for a face-to-face consultation with a physican. Tested in a real-life setting, the app was shown to save time without compromising the overall quality of the counselling [61].
For children, adolescents and young adults 16 apps were identified. One of them is the app My Sex Doctor [59]. The smartphone app is available in English as well. It is an app that was designed with the aim to provide comprehensive sex education for adolescents covering a variety of SRH topics. Contents are presented in a Q&A format and include short text explanations on various subjects. There is also a dictionary-function that helps clarifying meanings of SRHrelated terms. Adolescents can further check for symptoms that may indicate STIs. There are two versions of the app available (one being suitable for adolescents aged 12 to 16 and one for those that are older). Kalke et al. (2018) rated the app with an overall score of 29 out of possible 38 [59].
Ten apps were designed for healthcare workers and/or patients. Those are mostly apps that assist in data collection and decision-making processes of healthcare workers and support the interaction between professionals and their patients via in-app education or data sharing. OpenSRP supports local health workers through a smartphone app. It offers them educational information, enables data collection, supports decision-making and prompts appointment reminders as well as warn signals. Communication and coordination between different healthcare workers are also possible. The 3 apps aimed at men specialised on MSM. One App is the smartphone app HealthMindr available in English. The app contained tools which help with self-assessment [5]. It recommends prevention strategies to the users and assists them in ordering condoms and HIV self-tests [5]. The app also reminds people about prevention services, condom use, HIV testing and screening [5]. Sullivan et al. (2017) Fig. 1 Number of apps that covered the WHO SRH topics: (a) comprehensive education and information, (b) genderbased violence prevention, support and care, (c) prevention and control of HIV and other STIs, (d) sexual function and psychosexual counselling, (e) antenatal, intrapartum and postnatal care, (f) contraception counselling and provision, (g) fertility care, (h) safe abortion care; Social-structural factors: (i) cultural and social norms around sexuality, (ii) gender and socioeconomic inequalities, (iii) human rights, (iv) laws, policies, regulations and strategies [20]. AYA, adolescents and young adults. Upper chart distinguishes between GRADE Levels of Evidence (possible: very low, low, moderate, high). The lower chart between intended audience groups. Apps aimed at young MSM are presented in the MSM group, those aimed at young women are presented in the Children & Adolescents and young adults group found HealthMindr acceptable for MSM and considered its usability to be above average [62]. Figure 2 shows a selection of SRH apps for different GRADE levels. Following the GRADE approach, almost half of the apps (n = 23) were rated as very low regarding quality of evidence. This mostly (n = 19) included apps that were rated in app reviews according to their quality, without any information on further research [1 •, 23, 26, 32, 59]. Some articles described the development of apps without testing them with the target group or with an insufficient number of participants [63][64][65]. A usability approach was mentioned in one study [66].
Following the GRADE approach, no app obtained a high quality of evidence rating. This would require several highquality studies indicating no further research is needed to be sure about the estimated effect.

Case Study: Availability and Development of SRH Apps in Germany
In order to keep track of the current apps, it is essential to monitor the apps availability at regular intervals. Building on a previous work from 2019 [23] in 2022, an update for the German app stores was conducted. Developments in the field of sexual education apps for children and adolescents are presented in order to illustrate the dynamics of the app market. Three years after the original study (December 2022), the app stores (Apple App Store, Google Play) were searched again with the same inclusion and exclusion  [92]. To allow a comparison, Table 2 presents the new as well as the original ratings.
Four sexual education apps were additionally identified. One of the apps is designed for children (Das bin ich!) and three of the apps are designed for adolescents (KNOWBODY, Sexuell Gesund, Klar und Einfach). Two of the apps from 2019 were no longer available in the app stores. Four of the 8 sexual education apps achieved an overall (total) rating of good (4)

Discussion
This article presents the current state of research on SRH apps. In the following section, the contents, typical features, intended audiences and the quality of evidence of SRH apps are discussed. Challenges in investigating SRH apps are addressed as well.

Contents of SRH Apps
Among the apps identified in the study, there were many focusing on comprehensive education and fertility care, which includes menstrual tracking apps. However, most apps with these contents achieved very low GRADE ratings. Nine apps on prevention and control of HIV and other STIs (mostly for MSM, adolescents and young adults) with at least a low quality of evidence were identified. Another eight apps on contraception counselling and provision and eight on fertility care (usually for women and healthcare workers) also achieved a low or, in some cases, a moderate GRADE level. There were five apps (mostly aimed at healthcare workers) that supported antenatal, intrapartum and postnatal care with at least a low GRADE level. Three apps included contents on gender-based violence prevention, support and care and only a single app aimed at promoting sexual function and psychosexual counselling. All of these were rated with a low GRADE level or higher. Even though some apps might include abortion as a topic no SRH-app could be identified that focused on abortion care. As an SRH issue with a high mortality, health apps dealing with said topic could be a potential aid when it comes to the decision-making process or to reducing the stigma still attached to abortion [93,94].   Social-structural factors were rarely the primary aim of the apps although they are highly relevant for the way we deal with SRH [20]. Even if they are taken into account for example through the selection of the intedend audiences (e.g. MSM), there is still a need for research that explicitly investigates the possibility of shaping context factors using app-based technology in order to promote SRH [20]. Three low or moderate quality apps addressed cultural and social norms around sexuality, and one was noted to be transformative on gender relations [55••]. Other social-structural factors may have been addressed indirectly. For instance, gender inequalities can be addressed in gender-based violence prevention apps. However, they were neither the main contents of the apps nor mentioned explicitly in the studies.

Features of SRH Apps
More than half of the apps (62%) enabled data collection, mostly the possibility to self-track menstruation or to manage symptoms. Providing educational information was also a fundemental part of 38% of the apps. Reminders or push notifications were used in 32% of the apps. Audio or video content, as well as gamification or interactive components, was mentioned less frequently overall. Other aspects like the possibility of communicating with experts or having typical questions answered inapp were only available in some of the apps. The same applies to the support of decision-making processes, peer discussions in forums and data sharing and the linkage to offline care services. However, the small percentages of those features can be explained by the high number of menstrual tracking apps. Depending on the target group, various features were used differently.

Intended Audiences of SRH Apps
Four audience groups were addressed by the apps. The vast majority of which, particularly menstrual tracking apps, were aiming at women including pregnant and postpartum persons. There were also a lot of apps for children, adolescents and young adults containing various educational insights. Those apps mostly provided information using audio or video content, gamification or other interactive components. Some apps were designed to support interactions between healthcare workers and patients and a few apps aimed at MSM for HIV prevention and management. Those SRH apps typically enabled data collection for users, which in turn can support keeping track of current health statuses as well as in-app communication with experts. A few apps also allowed sharing data between clients and professionals. There were no apps for men in general, which does not come as a surprise given that only one article targeted them [28]. This can be explained by numerous fertility care studies, which often tend to focus on women. However, men might benefit from digital health interventions when it comes to their own SRH and may be involved in preconception care as well [95,96] and therefore a more balanced proportion in research would be desirable. The same applies for LGBTQIA + groups, who could benefit from well-designed and relevant health apps [4], as besides apps for MSM, no further apps were mentioned.

Evaluation of SRH Apps
Various studies note that the app market is developing rapidly and the availability of health apps is constantly changing [7,13,23,24]. Keeping track of currently available and relevant apps is almost impossible, especially when it comes to a field as broad as SRH. Authors of app reviews are trying to overcome this issue by systemising the current market [1•, 5, 23-26, 28, 32, 33, 58, 59]. However, they often lack the tools to recommend high-quality and relevant apps for practice [5,24,25,58]. In addition, studies in which quality assessment scales are used [23,26,58] or those with articles that date back to 2014 [1 •, 66] covered apps that may no longer be available or only in modified versions. In order to keep an overview of the current apps, it is therefore essential to view the apps at regular intervals and with the shortest possible delay in the publication process. When it comes to German sexual education apps, in the last 3 years, a positive development in the availability and quality has been observed. Two out of 6 apps could no longer be downloaded. Apps in other languages and with other SRH-focuses should also be regularly reviewed and evaluated. For practical purposes, databases that contain currently relevant apps should be made more apparent in order to facilitate better awareness of the wider audience regarding the apps available. Otherwise, it can be difficult for intended users to identify adequate apps [97].
Besides the overwhelming number, the efficacy of SRH apps has not yet been sufficiently studied in a scientific manner. Figure 2 shows a selection of apps sorted by GRADE level. Individual apps were deemed to be rated with at least a moderate GRADE level in the present study. The apps aimed at women are AKUD, ClearBlue, myPlan, Plan A Birth Control and Pulse. Those targeting adolescents and young adults are myPEEPS, PlayForward, Pulse and Tumaini and those for MSM are myPeeps and WeChat. Apps for women showed the potential to improve SRH by decreasing menstrual pain (AKUD), increasing of the conceiving likelihood (Clear-Blue), increases in knowledge and skills due to intimate partner violence (myPlan) as well as knowledge gain on contraception options (Plan A Birth Control). Adolescents and young adults benefitted from apps by improving their health knowledge (Playforward, Pulse, Tumaini), attitudes (Play-Forward, Tumaini) and self-efficacy (Tumaini) as well as by decreasing their reported sexual risk behaviour (Pulse). Apps for MSM reduced sexual risk behaviour (mePEEPS, WeChat) and enhanced self-testing behaviour (WeChat).
Apps from app reviews usually showed very low quality of evidence, as in the framework of the GRADE approach, those studies are not considered sufficient enough. It is possible that among the apps for which no pilot or RCT study was found, there are some that may have a positive impact on SRH. Some of the apps showed content expert ratings indicating that they are of good quality and therefore have the potential to improve people's SRH [1•, 23, 32, 59]. Thus, apps such as iMamma could support people during their pregnancy, My Sex Doctor could provide helpful information for adolescents and OpenSRP could support healthcare workers in their decision-making processes when no well-tested alternative is available.
In order to have a positive impact on SRH, it would be useful if apps would not only be of high-quality design and have relevant content, but they should also be tailored to the intended target group [4]. Especially younger audiences can be skeptical about app use [98]. In addition, it is useful to embed the apps in educational or healthcare settings so that users are not left on their own when using them. A good start is the possibility to communicate with experts via the app, as provided in some apps. However, a better approach would be to integrate the app in a real-life environment such as school lessons. A good example was provided by Sridhar et al. (2015), who examined an educational app (Plan a Birth Control) in the intended setting (waiting room) [61]. The authors found that the app can be used to prepare women for meeting with a doctor, saving time and increasing the overall quality of the consultation [61].

Limitations
Comparability between studies is limited. Often study-specific scales and categories are developed to assess quality, function or content of the apps [1•, 24, 25, 32, 33, 59]. In single studies, scientifically sound scales for quality ratings were used, which, however, are not tailored to SRH or the intended audience group [23,26,58]. One study explicitly focused on developing a framework for assessing general health apps for adolescents that could be helpful in studies for this specific target group [50••]. Harmonisation would be helpful in order to be able to compare studies and adequately reflect the current state of scientific knowledge. Thus, the selection of apps in this study can only be comprehensive to a limited extent. It is possible that among the apps, for which no pilot or RCT study was found, there are some that can impact SRH positively. This is especially true for articles that did not conduct any quality rating [5,24,25]. This may result in relevant apps not being included in the study. Additionally, to promote SRH, different outcomes have been measured across different studies, making it difficult to draw firm conclusions about the effectiveness of apps overall. Hamornisation here would be helpful as well. In order to address this issue in a concise manner, study aims and findings for each app are noted in Table 1.
For five apps, new articles were searched manually via the app name as well as the names of the authors of the original studies. Additional papers could be identified in every case indicating that there are more current studies for some of the other apps as well. These are not included in the present article, as it would have exceeded the scope of this study. Due to geographical location and language capabilities, it was not possible to verify for all apps whether they were still available or whether they exceeded the contents and functions reported in the studies. It is possible that some of the analysed apps are no longer available due to the fast changing market of health apps [10,21,32,33].
For app assessment, the GRADE approach proposed by the GRADE Working Group was followed to distinguish whether the Quality of Evidence was (a) high, (b) moderate, (c) low or (d) very low [47]. Ratings were only conducted by a single scientist (MM) and not double checked to enhance the validity. Expert opinion is not a category of quality of evidence due to GRADE [47]. Nevertheless, apps that have been rated with relevant scales by experts were included, even if the intended audience itself was not involved (see Appendix 2 for more details on the selection of apps).

Conclusions
In this study, SRH apps that are known among the scientific community were examined. Many apps cover a variety of SRH topics, but they often lack high-quality assessment. In order to improve SRH, it would be helpful if apps were of high-quality design, contained relevant evidence-based content, would be tailored to the intended audience and tested in real-life settings [4,16,92,99]. But this does not necessarily mean that less researched apps cannot positively influence health. Even some apps with a very low level of evidence showed relevant contents and promising features to support people in managing their SRH.
Finally, a common issue in the field of SRH interventions emerged again in the present study [100,101]. Outcomes that are used for effectiveness measurements are not necessarily all factors that lead to good SRH according to our present understanding. Negative factors such as limiting risk behaviour and preventing diseases as well as knowledge gain are still the primary focus. However, there were also studies which set a good example by noting other outcomes. Among these are enhanced feelings of confidence and resilience [88], well-informed SRH decisions [88] and the ability to initiate SRH discussions [72]. In addition to traditional outcomes, there should be a greater emphasis on examining positive-psychological and competence-oriented variables.