Abstract
Sexual and reproductive health interventions’ effectiveness should be evaluated using a valid instrument. This study aimed to validate a questionnaire on sexual and reproductive health in adolescents and young adults from São Tomé and Príncipe who voluntarily enrolled in a vocational school in Portugal to complete their 12th school year and obtain a professional card.
A questionnaire consisting of perception and knowledge sections was adapted from previous questionnaires. We followed the steps of psychometric analysis. Firstly, we assessed face validity and content validity. We used factorial analysis to validate the perceptions section (which included Likert-type questions). For the knowledge section (which consisted of multiple-choice questions), we used the key check, discrimination index, and difficulty index. We evaluated internal consistency through Cronbach’s alpha for the perceptions section and the Kuder-Richardson score for the knowledge section.
Out of 105 invited students, a total of ninety students were enrolled in this study, of whom 88 completed most of the questions. The exploratory analysis showed that most students agreed with the right to refuse intercourse. However, while approximately 23% disagreed that consensual sex among adult women or men is always wrong, around the same proportion agreed. A considerable number of students recognized condoms and pills as effective methods of contraception. However, most students were unfamiliar with other methods. Most knowledge questions showed acceptable difficulty levels, and the discrimination index varied among questions. The knowledge questions demonstrated good consistency levels. This study enabled us to develop an appropriate instrument for evaluating the effectiveness of public health interventions and identifying specific knowledge gaps in migrant populations from low-income countries. This will help prioritise topics to be addressed in sexual health education sessions.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
It is well-known that school-based sex education can prevent unwanted pregnancies, and promote positive sexual health at the individual, family, community, or health system level [1,2,3]. Most adolescents do not feel comfortable discussing sexuality issues with their parents [4]. Moreover, they spend most of their time at school, which is why schools play a vital role in conveying information and demystifying false beliefs related to sexual and reproductive health.
A substantial part of the world’s youth lives in developing countries. However, little is known about their knowledge and understanding of sexual health [5,6,7]. Adolescent pregnancy is a significant public health problem in São Tomé and Príncipe [8]. It has one of the highest adolescent pregnancy rates in Sub-Saharan Africa, reaching around 27% [9]. Teenage pregnancy can have adverse consequences for the mother, such as obstetric complications, early school dropout, low levels of education, less skilled jobs, and a higher probability of depending on social allowances, exacerbating the low socioeconomic status of people from developing countries [10].
The Sustainable Development Goals established by the United Nations emphasize the significance of advancing the eradication and treatment of sexually transmitted infections (STIs). STIs are among the most prevalent diseases globally, with Sub-Saharan Africa having the highest prevalence [11]. Adolescents have some of the highest incidence rates of STIs [12]. Studies have shown the effectiveness of school-based interventions in preventing STIs among young people [13, 14].
A pilot study is a type of feasibility study performed in preparation for a major study [15]. Conducting a pilot study provides information on whether the survey instrument is appropriate or too complex for the participants. A well-conducted pilot study can benefit other projects that use similar methods and tools. Well-designed and well-performed pilot studies can provide information about the research process and, in some cases, insights into the expected results [7]. A few studies report on the results of questionnaire validation in sexual and reproductive health among migrants from low-income countries.
SWITCH ON is a vocational academy for students who have completed their 9th year. One of its schools is in Viana do Castelo, a city in northern Portugal, where there are a total of 270 students, most of them from São Tomé and Príncipe, a Portuguese-speaking African country. These students are fundamental in the current scenario of the country’s systemic shortage of workforce in the construction and energy sector, which has a marked tendency to intensify over the next decade, forcing the country to resort to migrants.
This study aims to validate an adapted questionnaire about sexual and reproductive health for a population of adolescents and young adults from São Tomé and Príncipe. The secondary objectives were to analyse factors associated with adolescents’ perceptions and knowledge related to sexual and reproductive health and to identify gaps to prioritize topics to be addressed in a public health intervention.
Methods
This study was designed as part of a field study. We selected five courses, comprising a total of 105 students, who were having classes in the school between December 2022 and February 2023, the period of the pre-intervention test, health education sessions, and post-intervention test. We performed a cluster-randomized sampling, randomly allocating each course to one of three groups: (A) health education sessions (two courses), (B) written information delivery (two courses), and (C) control group without intervention (one course).
The inclusion criteria were being a student between the ages of 15 and 26, coming from a Portuguese-speaking country, residing in Viana do Castelo, and undergoing training at the SWITCH ON Vocational Training Academy. Students who did not provide informed consent or who, for some reason, were not present in the classroom when completing the questionnaires were excluded.
A pre-intervention pilot questionnaire on perceptions and knowledge related to sexual and reproductive health was administered to all participants. It consisted of questions from three previous questionnaires: (1) sociodemographic questions and questions from the perceptions section of an internationally validated standard questionnaire [24], (2) questions about knowledge of sexual and reproductive health from previous studies conducted in Portugal [16, 17] and validated for the Portuguese population [17]. Our questionnaire mainly comprises closed-ended questions.
The questionnaire is divided into four sections: (1) presentation of the study, and consent to complete and process data from the questionnaire, (2) sociodemographic questions, (3) perceptions about sexuality (Likert-style questions), and (4) knowledge of sexual and reproductive health (multiple-choice questions (MCQ), with one or more correct answers). Sociodemographic variables were selected because they are potential determinants of students’ knowledge of sexual and reproductive health and could be analysed as potential confounders. The questionnaire was administered by the main investigator and self-completed by students in the classroom using Microsoft Forms®.
To assess face validity, the method of spoken reflection was used. We first delivered the questionnaire to six randomly selected adolescents, who were representative of the desired sample. Then, we conducted face-to-face interviews to assess the items’ difficulty, relevance, and ambiguity. Based on their feedback, we made the necessary modifications [16].
The content validity was assessed qualitatively by three experts who ascertained whether the questions were relevant, appropriate, and representative of the construct being examined [18]. The items were subjected to exploratory analysis, and the respective results are summarized in the Results section. Knowledge questions that were not answered were considered incorrect.
Construct validity was evaluated separately for the perceptions and knowledge sections using different methodologies. We used exploratory factor analysis (principal component analysis with varimax rotation) for perception questions. The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett’s Test of Sphericity were used to assess the suitability of the sample for factor analysis [19]. The knowledge section includes dichotomous items that were not subjected to factor analysis. Instead, we performed a key check, item discrimination analysis, and item difficulty analysis, following the methodology proposed by Considine et al. [20]. The key check determines whether the correct answer to the MCQ is correct and verifies that there is only one answer that can be considered valid [20]. The questionnaire was filled out by three medical doctors, and questions whose answers raised doubts were confirmed by a senior doctor who is an expert in sexual and reproductive health [18]. Item discrimination analysis examines how each MCQ is related to the overall test score and is computed using a biserial point correlation [20, 21].
The most common formula used to measure internal consistency is ‘coefficient alpha’ [20] or ‘Cronbach’s alpha score’. This was used to analyse the internal consistency of the perceptions section. Kuder-Richardson (KR-20) is a special form of the coefficient alpha formula, used for dichotomous data, such as answers that are scored as correct or incorrect in the context of MCQs [22]. It is interpreted like the Cronbach alpha scores: the questionnaire is considered consistent if the score is higher than 0.7. We used the KR-20 score to analyse the internal consistency of the knowledge section. Those students with knowledge scores above the mean were categorized as having good knowledge, while those with knowledge scores below the mean were categorized as having poor knowledge. We searched for associations between the sociodemographic variables, and the knowledge test score (as a continuous and categorical variable).
The data were analysed using R software, version 4.1.3 for RStudio Build 461, Vienna, Austria, and IBM SPSS Statistics for Windows, version 27.0, Armonk, USA.
The study protocol was approved by the Ethics Committee for Health of the Local Health Unit of Alto Minho, with reference number 78/2022.
Out of 105 invited students, a total of 90 students were enrolled in this study, of whom 88 completed most of the questions. Their characteristics can be found in Table 1. The median age was 21 (SEM: 0.303), 70% were female, and 38% had received education on sexual and reproductive health in the last two years.
Results
Sociodemographic Variables
Among the female students, 12 (20%) reported at least one previous pregnancy occurring between 16 and 21 years of age. When asked if they were willing to have a baby when they got pregnant, five (42%) answered ‘I did not want to get pregnant at that point’, and only one (8%) answered ‘I really to get pregnant at that time’.
Perceptions
Exploratory Analysis
Answers to questions on perceptions regarding sexual and reproductive health can be found in Table 2. Most of the students agreed or strongly agreed that women (85%), and men (79%) have the right to refuse sexual intercourse if they do not want to engage in it. However, while 23% disagreed that consensual sex among adult women or men is always wrong, around the same proportion either agreed or had a neutral opinion (36–40%). Thirty-six per cent agreed with the right to abortion, and 64% believed that the couple should make this decision together.
Construct Validity and Internal Consistency Analysis
Based on Bartlett’s test, the correlation matrix significantly differed from the identity matrix, and the Kaiser-Meyer-Olkin measure confirmed its adequacy for principal component analysis. We performed a principal component analysis with varimax rotation, which revealed that the first principal component explained 24% of the variance and the second component accounted for 17% of the initial variance. According to the criterion of eigenvalue greater than 1, we retained five principal components, as shown in Fig. 1, which explained 75% of the total variance.
The questions ‘A woman has the right to say ‘no’ to sexual intercourse if she does not want to do it’, ‘It is acceptable for a woman to have sex before marriage’, ‘Having pleasurable sex is important to a woman’s sex life and overall well-being’, and ’Men naturally have greater sexual needs than women’ were considered redundant and removed. The calculated coefficient alpha was 0.34. It was not possible to conduct a confirmatory factor analysis using the usual method, due to our small sample size.
Knowledge
Exploratory Analysis
The questions on knowledge and their corresponding answers can be found in Table 3. The mean score for knowledge was 10.93 (± 2.47) out of 20 points (54%). Most students (89%) accurately defined basic reproductive concepts, such as puberty, and recognised condoms as a contraceptive agent (91%). However, 88% of the students were unfamiliar with the vaginal ring, and 79% were not familiar with the intrauterine device. Moreover, some misconceptions were observed regarding condom use, as 88% believed that condoms should be put on only at the time of ejaculation. Knowledge gaps were also identified regarding sexually transmitted infections (STIs), as 88% and 83% of participants did not recognise gonorrhoea and hepatitis B as STIs respectively. Additionally, the students lacked insight into what to do in case a condom breaks: 91% of participants did not know that emergency contraception can be used up to five days after unprotected intercourse, 84% did not know that emergency contraception is to be used on occasion, and 71% did not know that, in case of a broken condom, the woman should not douche but instead wash and urinate as soon as possible.
Construct Validity and Internal Consistency Analysis
As observed in Table 4, the item discrimination indices ranged between − 0.14 and 0.46, with an average of 0.19. The difficulty of the knowledge items varied from 0.11 to 0.94, with an average of 0.55. The KR20 score for the knowledge section was 0.73.
Discussion
We observed that unwanted pregnancies, including adolescent pregnancy, were a concern in this student population. Additionally, most students believed that the abortion decision should be made by the couple, despite Portuguese legislation stating that the woman had the final say. A considerable proportion of students considered same-sex intercourse to always be wrong. It would have been important to collect information about the students’ religion, as it had been associated with homophobic attitudes and behaviours in previous studies [23].
A low overall knowledge score was found in this study, similar to other diagnostic tests administered to students [24]. This may be related to informal sources of information, such as their peers, who are often equally uninformed.
HIV/AIDS was widely known as an STI among students. However, most of the students did not know about some of the HIV transmission routes, such as men who have sex with men or vertical transmission. Additionally, most students did not recognise gonorrhoea, syphilis, HPV, and hepatitis B as STIs, which could lead them to not give importance to the symptoms or choose not to use condoms if they found that their partner was not HIV positive.
Regarding STI prevention knowledge, most students knew that condoms reduced the risk of contracting STIs, which was consistent with other studies [25,26,27]. However, we observed that many newer contraceptive methods, such as contraceptive rings, intrauterine devices/systems, and subdermal implants, which are available for free in the Portuguese NHS, were not known to these students.
Unlike other similar studies [25], the variables ‘birth sex’, ‘gender’, ‘age’, ‘age group’, ‘previous sex education (last two years)’ and ‘education of legal guardians’ were not predictors of the score on the knowledge test. This may be related to an insufficient sample size that did not allow us to demonstrate any of these associations.
The instrument consisted of four sections, of which two were thoroughly analysed in this study. The perceptions section showed low internal consistency. Low-reliability coefficients may indicate that the test is too short or that the multiple-choice questions have little in common. Therefore, they can be used as a set of questions but not as a scale for a singular construct. During the diagnostic test, some students reported that the questionnaire was too long, so we decided to remove this section. Nevertheless, since the main objective of our study was to assess the effectiveness of a public health intervention in increasing knowledge of sexual and reproductive health, this change could increase the acceptability of the questionnaire.
On the other hand, the knowledge section demonstrated acceptable internal consistency. Furthermore, most of the items showed adequate item difficulty. However, 27 items showed an unacceptable discrimination index (below 0.25). As a result, in the final questionnaire, we removed all items with discrimination indexes below 0.20 and between 0.20 and 0.25 with either too low (< 0.15) or too high (> 0.85) difficulty index.
Our study had some limitations, mainly due to its observational nature. The sample size was considered low, which might have reduced the statistical power and the generalizability of the results. A systematic review of articles available in Scopus recommended a sample size of 500–600 for questionnaire validation studies involving students [28]. Moreover, a proportion of students were not evaluated because they were out on internships. These students are mostly in their second year of the course, are probably older and may have different perceptions and/or knowledge regarding sexual and reproductive health. Furthermore, this study did not include vocational school dropouts who might have faced more challenges and risks related to sexual and reproductive health than the students who completed their education.
Public Health Implications/Practice Implications
The poor knowledge scores on sexual and reproductive health shown in the questionnaire support the urgent need for health education interventions for these students. Additionally, it allowed the investigators to specifically identify the main knowledge gaps of the population in the study so that the topics to address in the intervention are prioritised. On the other hand, the validation of the pilot questionnaire contributed to improving its definitive version, with more valid and less ambiguous questions. In summary, the questionnaire can help educators, healthcare providers and policymakers understand the unique sexual and reproductive health needs of this population and develop targeted interventions to improve their overall health and well-being.
References
Underhill K, Operario D, Montgomery P. Systematic review of abstinence-plus HIV prevention programs in high-income countries. PLoS Med. 2007; 4(9):e275. doi:https://doi.org/10.1371/journal.pmed.0040275
Aarons SJ, Jenkins RR, Raine TR, et al. Postponing sexual intercourse among urban junior high school students-a randomized controlled evaluation. J Adolesc Health. 2000; 27(4):236–47. doi:https://doi.org/10.1016/s1054-139x(00)00102-6
Coyle K, Basen-Engquist K, Kirby D, et al. Safer choices: Reducing teen pregnancy, HIV, and STDs. Public Health Rep. 2001; 116 Suppl 1(Suppl 1):82–93. doi:https://doi.org/10.1093/phr/116.S1.82
Ashcraft AM, Murray PJ. Talking to parents about adolescent sexuality. Pediatr Clin North Am. 2017; 64(2):305–20. doi:https://doi.org/10.1016/j.pcl.2016.11.002
Frost JJ, Forrest JD. Understanding the impact of effective teenage pregnancy prevention programs. Fam Plann Perspect. 1995; 27(5):188–95.
Hughes J, McCauley AP. Improving the fit: Adolescents’ needs and future programs for sexual and reproductive health in developing countries. Stud Fam Plann. 1998; 29(2):233–45.
Acharya DR, Thomas M, Cann R. Validation of a questionnaire to measure sexual health knowledge and understanding (sexual health questionnaire) in nepalese secondary school: A psychometric process. J Educ Health Promot. 2016; 5:18. doi:https://doi.org/10.4103/2277-9531.184560
Vasconcelos A, Bandeira N, Sousa S, Machado MC, Pereira F. Adolescent pregnancy in Sao Tome and Principe: Are there different obstetric and perinatal outcomes? BMC Pregnancy and Childbirth. 2022; 22(1):453. doi:https://doi.org/10.1186/s12884-022-04779-9
UNICEF. Inquérito de Indicadores Múltiplos, São Tome e Principe, MICS-STP. 2014.
Poudel S, Upadhaya N, Khatri RB, Ghimire PR. Trends and factors associated with pregnancies among adolescent women in Nepal: Pooled analysis of Nepal demographic and health surveys (2006, 2011 and 2016). PLoS One. 2018; 13(8):e0202107. doi:https://doi.org/10.1371/journal.pone.0202107
Furuoka F, Hoque MZ, Jacob RI, Ziegenhain P. An analysis of the development assistance for health (DAH) allocations for STD control in Africa. Health Economics, Policy and Law. 2020; 15(4):458–76. doi:https://doi.org/10.1017/S1744133119000197
Upchurch DM, Mason WM, Kusunoki Y, Kriechbaum MJ. Social and behavioral determinants of self-reported STD among adolescents. Perspect Sex Reprod Health. 2004; 36(6):276–87. doi:https://doi.org/10.1363/psrh.36.276.04
Jemmott JB, 3rd, Jemmott LS, O’Leary A, et al. School-based randomized controlled trial of an HIV/STD risk-reduction intervention for south african adolescents. Arch Pediatr Adolesc Med. 2010; 164(10):923–9. doi:https://doi.org/10.1001/archpediatrics.2010.176
Rohrbach LA, Donatello RA, Moulton BD, Afifi AA, Meyer KI, De Rosa CJ. Effectiveness evaluation of it’s your game: Keep it real, a Middle School HIV/Sexually transmitted Infection/Pregnancy Prevention Program. J Adolesc Health. 2019; 64(3):382–9. doi:https://doi.org/10.1016/j.jadohealth.2018.09.021
Polit DF, Hungler BP. Essentials of nursing research. 1993.
Costa S. Conhecimentos, atitudes e crenças face à sexualidade e educação sexual de adolescentes do 8º e 10º ano de escolaridade. Porto: Escola Superior de Enfermagem do Porto; 2015.
Dias A, Pereira I. Prevenção da Gravidez na Adolescência: Educação Sexual em Contexto Escolar. Nursing. 2010; (259).
Advances in questionnaire Design, development, evaluation and testing. Beatty PC, Collins D, Kaye L, Padilla J-L, Willis GB, Wilmot A, editors. Hoboken, NJ, US: John Wiley & Sons, Inc.; 2020.
Fonseca MJ, Costa P, Lencastre L, Tavares F. A statistical approach to quantitative data validation focused on the assessment of students’ perceptions about biotechnology. Springerplus. 2013; 2:496. doi:https://doi.org/10.1186/2193-1801-2-496
Considine J, Botti M, Thomas S. Design, format, validity and reliability of multiple choice questions for use in nursing research and education. Collegian. 2005; 12(1):19–24. doi:https://doi.org/10.1016/s1322-7696(08)60478-3
Kaplan RM, Saccuzzo DP. Psychological testing: Principles, applications, and issues: Cengage Learning; 2017.
Gravetter FJ, Forzano L-AB. Research methods for the behavioral sciences: Cengage learning; 2018.
Oyarce-Vildósola O, Rodríguez-Fernández A, Maury-Sintjago E. Association between Homophobia and Sociodemographic characteristics in Health Workers in Southern Chile. Int J Environ Res Public Health. 2022; 19(21). doi:https://doi.org/10.3390/ijerph192113749
Mba CI, Obi SN, Ozumba BC. The impact of health education on reproductive health knowledge among adolescents in a rural nigerian community. J Obstet Gynaecol. 2007; 27(5):513–7. doi:https://doi.org/10.1080/01443610701478991
Folasayo AT, Oluwasegun AJ, Samsudin S, Saudi SN, Osman M, Hamat RA. Assessing the knowledge level, Attitudes, Risky Behaviors and Preventive Practices on sexually transmitted Diseases among University students as Future Healthcare Providers in the Central Zone of Malaysia: A cross-sectional study. Int J Environ Res Public Health. 2017; 14(2). doi:https://doi.org/10.3390/ijerph14020159
Awang H, Wong LP, Jani R, Low WY. Knowledge of sexually transmitted diseases and sexual behaviours among malaysian male youths. J Biosoc Sci. 2014; 46(2):214–24. doi:https://doi.org/10.1017/s0021932013000114
Sekirime WK, Tamale J, Lule JC, Wabwire-Mangen F. Knowledge, attitude and practice about sexually transmitted diseases among university students in Kampala. Afr Health Sci. 2001; 1(1):16–22.
White M. Sample size in quantitative instrument validation studies: A systematic review of articles published in Scopus, 2021. Heliyon. 2022; 8(12):e12223. doi:https://doi.org/10.1016/j.heliyon.2022.e12223
Acknowledgements
We would like to thank Dr Humberto Faria from the Local Health Unit of Alto Minho for his support in the design of the questionnaire and content validation.
Funding
Open access funding provided by FCT|FCCN (b-on).
Author information
Authors and Affiliations
Contributions
Conceptualization, C.B., L.S. and R.C.; methodology, C.B., L.S., R.C.; writing—original draft preparation, C.B., M.N., J.D., L.R.; writing—review and editing, all authors; supervision, L.S and R.C. All authors have read and agreed to the published version of the manuscript.
Corresponding author
Ethics declarations
Ethics Statement
The Ethics Committee of the Local Health Unit of Alto Minho (ULSAM) approved the protocol of this study with reference number 78/2022. The ethical principles of human medical research contained in the Declaration of Helsinki and national legislation were respected. The participants gave informed consent and the data collected was anonymized, guaranteeing the necessary confidentiality of the information collected. In addition, the principal investigator and her supervisor are subjected to medical confidentiality according to the Code of Ethics of the Portuguese Medical Association.
Statements and Declarations
The authors declare that they have no known competing commercial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Brazete, C., Caveiro, D., Neto, M.L. et al. Validation of a Questionnaire on Sexual and Reproductive Health Among Immigrant Vocational Education Students in Portugal from São Tomé and Príncipe. J Community Health 48, 847–856 (2023). https://doi.org/10.1007/s10900-023-01230-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10900-023-01230-8