Introduction

Infertility is a chronic disease commonly accompanied by psychological symptomatology1. The rate of infertile women who meet the criteria for a psychiatric diagnosis may reach 40%2, with depressive3,4 and anxiety disorders5 being the most prevalent within clinical samples. Women are almost unanimously found to present higher levels of mental health problems connected to infertility than men6. If distress reaches mental disorder levels, it also seems to hinder the success of assisted reproduction techniques (ART)7.

Psychosocial interventions increase psychological well-being and the likelihood of pregnancy in infertility patients8,9,10,11. However, there is a large disparity between the number of patients reporting infertility distress and the number of those seeking psychological support12. Furthermore, patients’ mental health status deteriorates with unsuccessful attempts. Therefore, it is necessary that patients (1) be informed about their mental status compared to normative data and (2) should be checked at various time points during infertility treatment with properly validated mental health instruments.

Several psychometric tools exist to assess adjustment to infertility, the methodological trend moving from the use of generic tools, also applicable to infertile patients, to instruments developed specifically for this population. The European Society of Human Reproduction and Embryology guideline13 lists 12 such tools, 5 generic and 7 specific. Out of the ones assessing mental health and not, say, perceived quality of care, experience of patient-centeredness, or fertility status awareness, the most comprehensive and widely used instrument is the FertiQoL14,15, assessing patient needs in three domains: behavioral, emotional, and social-relational. The one recommended for screening purposes is the SCREENIVF16, a combination of both generic and specific items, also assessing three domains (emotional, social-relational and cognitive), and quick to complete. Out of the existing fertility-related questionnaires, only the FertiQoL and the SCREENIVF have been validated in Hungarian17,18.

The SCREENIVF consists of five subscales measuring anxiety, depression, social support, cognitions of helplessness and lack of infertility acceptance, each based on a reflective model, that is, one in which all items are a manifestation of the same underlying construct19. The tool has been validated in Portuguese20, Dutch21, and Turkish22. Cutoff values are not uniform in these studies, and there are considerable deviations in the rates of patients found to be at risk. A Hungarian validation study was performed on a small sample (N = 60), showing good reliability and model fit but low specificity18. Therefore, further validation of the Hungarian version seems justified.

The aim of the present study was to reinvestigate the psychometric characteristics and screening capacities of the Hungarian SCREENIVF on a larger sample of Hungarian women in assisted reproduction. We hypothesized that (1) the psychometric properties of the SCREENIVF can remain convincing and (2) the SCREENIVF is able to screen out ART patients to be further explored for mood disturbances if cutoffs for the Depression subscale and the SCREENIVF Risk Factor scale are tested against real data and applied accordingly.

Results

Study participants were in their mid-30s (Table 1), almost all in legally sanctioned heterosexual relationships, qualified, employed and financially secure. The mean duration of, mainly primary, infertility was 3.3 years. Etiology was varied, with one-third unexplained or (yet) unknown. The existence of other chronic diseases was not typical, but being overweight was rather frequent.

Table 1 Sociodemographic and medical characteristics of the sample.
Full size table

Reliability and validity

Reliability tests resulted in good to excellent Cronbach’s alpha values (Table 2).

Table 2 Descriptive statistics of SCREENIVF subscales with Cronbach’s alpha values.
Full size table

All item-subscale correlations were higher than 0.40, indicating that the items contribute sufficiently to their subscales, except for one depression item, referring to suicidal ideation (item 7, 0.381; Supplementary Table S1). Substantial correlations were found between the SCREENIVF subscales (Table 3), with the highest between Depression and Anxiety, and the lowest between Helplessness and Social Support.

Table 3 Spearman’s rank correlation coefficients between SCREENIVF subscales, risk factors and cognate measures.
Full size table

CFA with DWLS indicated optimal model fit (χ2 = 630.866, p < 0.001, RMSEA = 0.018 [CI90 = 0.013–0.023], CFI = 0.998, TLI = 0.997, SRMR = 0.042). Standardized factor loadings ranged from 0.4 to 0.9, confirm to the majority of empirical research studies23, except for, again, the Depression item on suicidal ideation, which had a weak factor loading (0.37; Fig. 1). CFA without the suicide-related Depression item, however, showed worse model fit than with it (χ2 = 615.692, p < 0.001, RMSEA = 0.020 [CI90 = 0.015–0.025], CFI = 0.997, TLI = 0.997, SRMR = 0.043).

Figure 1
figure 1

Standardized regression weights of factor loading. A anxiety subscale item, D depression subscale item, S social support subscale item, HC helplessness cognitions subscale item, AC acceptance cognitions subscale item.

Full size image

As hypothesized, all correlations between SCREENIVF subscales and cognate measures were significant, and most of them were fairly strong (Table 3), the highest when the subscale and the external tool measured the same construct, e.g. Depression and the BDI, Anxiety and the STAI-S, Social Support and the MSPSS. The Helplessness subscale related best to the Core FertiQoL, and the Acceptance subscale with FertiQol Emotional. The SCREENIVF Risk Factors total score showed the strongest relationship with the Core FertiQoL, but was also related well to the BDI.

Cutoffs, sensitivity and specificity

Based on BDI results, 50.1% (N = 323) of our population was unaffected by depressive symptoms, 30.4% (N = 196) displayed mild symptoms, and 19.5% (N = 126) displayed moderate to severe, that is, clinically relevant symptomatology. The various cutoff values and corresponding ‘at risk’ sample rates in the original study, the five validation studies and the present study are listed in Table 4. All overall models of SCREENIVF risk categories (‘at risk’ vs ‘not at risk’) as predictors of BDI categories (‘case’ vs ‘no-case’ and ‘no-to-mild’ vs ‘moderate-to-severe depression’) were statistically significant when compared to the null model, irrespective of Depression subscale cutoff point.

Table 4 Cutoffs and rates of female population found to be ’at risk’ in SCREENIVF validation studies.
Full size table

The best equation results were found in two cases: first, when a 3/4 cutoff on the SCREENIVF Depression subscale and a 0/1 cutoff on the SCREENIVF Risk Factor scale were applied, a procedure correctly predicting 84.5% of BDI depression cases vs no-cases (χ2(1) = 220.246, p < 0.001, Nagelkerke R2 = 0.386, Youden index corresponding to the cutoff: J = 69.0). Second, when a 6/7 cutoff on the SCREENIVF Depression subscale and a 1/2 cutoff on the SCREENIVF Risk Factors scale were applied, a procedure correctly predicting 87.4% of BDI moderate-to-severe vs no-to-mild depression cases (χ2(1) = 220.608, p < 0.001, Nagelkerke R2 = 0.462, Youden index corresponding to the cutoff: J = 66.4; Supplementary Table S2).

Two-step cluster analysis of the SCREENIVF Risk Factors scale confirmed a natural demarcation between scores 1 and 2 (Supplementary Fig. S1). ROC analysis showed the best result when the SCREENIVF Risk Factors scale was used with a 4/5 cutoff on the Depression subscale, with BDI no-to-mild vs moderate-to-severe depression applied as a state variable (AUC = 0.918, CI [0.895, 0.940]). ROC curves indicated that the SCREENIVF had a better diagnostic value when used to discriminate between no-to-mild and moderate-to-severe BDI cases of depression (AUCs between 0.647 and 0.898) than when distinguishing cases from no-cases (AUCs between 0.603 and 0.795). When ROC curves for all individual and total SCREENIVF risk factors were compared, the AUC for the total factor (0.898) was superior to that of all subfactors, including Depression (0.848; Supplementary Fig. S2).

Discussion

The aim of this study was to revalidate the Hungarian version of the SCREENIVF on a fairly large sample and to test its ability to grasp emotional maladjustment with various cutoffs on the Depression subscale and the total Risk Factors scale. The SCREENIVF proved to be a valid and reliable instrument to detect emotional maladjustment among Hungarian women undergoing ART, and was able to discriminate between scores possibly indicative of different levels of depression as measured by the BDI.

Our respondents were typical of ART patients in terms of help-seeking age (mid-30s) in developed countries25, where the age of family formation is constantly increasing26. Additionally, our sample displayed fairly high socioeconomic status, a common finding among patients in ART27. The homogeneity of our sample’s marital status reflects Hungarian legislation, namely, that ART is allowed only for married or officially cohabitating opposite-sex couples and single women who cannot have children otherwise.

The unfortunate coincidence of the study timelines with the COVID-19 pandemic posed considerable challenges. Participant enrolment practically stopped during the first wave, when only life-saving operations were permitted. As an exception, however, immediately after the first wave, the Hungarian health regulations allowed for assisted reproductive interventions to be performed. Even so, with the threat of treatment cancellations due to patients’ virus infection, partners not being allowed to accompany women to examinations and interventions, and the general existential and financial concerns, COVID-19 most probably influenced the stress of infertility28. Unfortunately, our original, preregistered study protocol did not allow us to investigate the added effect of the pandemic on the mental wellbeing of the participants.

Online data collection may have introduced sampling bias due to low response rate, possibly caused by overlooking the announcement, interpreting it as junk mail, or completion interrupted by technical or personal difficulties29. Our response rate (22.9%) was lower than average proportions found in meta-analyses (34–36%)30,31. This was not attributable to a lack of internet access, since we were able, with permission, to extract from the medical database email addresses for all patients, who thus had equal opportunities to fill in the questionnaire, with illegitimate participation also minimized. Low response rates do not necessarily indicate large nonresponse error, i.e. that nonrespondents would have provided different answers than actual respondents32.

All of our Cronbach’s alpha values fall within the intervals found in previous studies. The Depression subscale also fits in the tendency of being the least consistent, with the suicidality item not contributing well to its reliability. Even so, the subscale proved more reliable than in Irmak Vural and colleagues’ study22, where four items had to be left out. Despite the poor factor loading of the suicidality item, we do not recommend its removal, since (1) model fit indices did not improve with its removal, and (2) the item may warn of suicide risk, signaling an urgent need for further exploration. This is of utmost importance in Hungary, which is customarily among the first five highest suicide-rate states in Europe, e.g. second highest in 2020 (https://ec.europa.eu/eurostat/web/products-eurostat-news/w/edn-20230908-3).

Although two of the previous validation studies of the SCREENIVF20,21 used parcel- rather than item-based CFA, here, DWLS estimation yielded excellent results. DWLS is specifically designed for categorical (e.g., binary or ordinal) data in which neither the normality assumption nor the continuity property is plausible23, especially for higher sample sizes33. Therefore, the hypothesis of the five latent variables underlying the SCREENIVF seems to be confirmed.

Previous literature on the SCREENIVF draws attention to probable transcultural differences in cutoffs20 and to a need for ROC analyses in establishing them21. We decided to test cutoff points in relation to a ‘gold standard’ measure, the BDI, because depression tools are commonly used as criterion measures of psychological maladjustment in infertility34. We found that the SCREENIVF is able to detect possible depression, reaching the best sensitivity with a 3/4 cutoff on the Depression subscale and 0/1 on the Risk Factors scale. Additionally, it is able to identify presumable moderate-to-severe cases of depression, showing the best specificity with a 6/7 cutoff on the Depression subscale and 1/2 on the Risk Factors scale. When the goal is a good trade-off between sensitivity and specificity, the SCREENIVF Risk Factors scale is best used with a 4/5 cutoff on the Depression subscale, to separate no-to-mild cases from those to be further evaluated for moderate-to-severe depression.

Therefore, if one wants to use the SCREENIVF to identify the highest number of possible cases of psychological disorders, a lower cutoff value is needed to maximize sensitivity. If, however, the SCREENIVF is intended to find individuals most likely already affected by depression, a higher cutoff value is needed to maximize specificity. Our analysis showed that the SCREENIVF serves both goals fairly well as a stand-alone tool able to signal different levels of emotional maladjustment, depending on the purposes of the clinician.

Ockhuijsen et al.21 found that the SCREENIVF did not do well at prognosticating whether ‘at risk’ patients would actually develop psychological problems during treatment. We found that the SCREENIVF has high screening accuracy at the time of administration. Thus, we believe that the SCREENIVF can safely be used for screening out ‘afflicted’ rather than ‘at-risk-for-later-maladjustment’ patients, who can then be referred to a full diagnostic evaluation. Additionally, given that the ROC curve for the SCREENIVF Risk Factors total scale outperformed that of the Depression subscale, it seems that there is more to the emotional disturbance in infertility than just depressed mood: anxiety, helplessness, lack of social support and unacceptance of infertility all add their own shades to the clinical aspect.

ROC analysis can be combined with utility-based decisions to determine optimal cutoff points35, in this case, a desirable balance between the number of patients found vulnerable and the financial costs of delivering psychosocial support. In Hungary, where there is a scarcity of mental health professionals in state-subsidized facilities, it is essential that we single out patients most in need of psychological services by applying higher thresholds on SCREENIVF scales. The proportion (22.8%) of the possibly afflicted population spotted in this sample is realistic in terms of healthcare possibilities on the Eastern European scene, while also comparable to international prevalence data.

A great strength of our study is the use of four cognate measures, all validated in Hungarian, to test the convergent validity of the SCREENIVF, two of them measuring general psychological distress, and two assessing infertility-related issues. Additionally, arbitrariness was ruled out by relying on statistical methods that reveal natural groupings in real-life data. We worked with a relatively sizable sample from a large fertility center in the capital that also serves IVF patients from the rest of the country.

The study has some limitations. Including only women made it impossible to validate the SCREENIVF in men or at the couple level. The cross-sectional design did not allow for testing the predictive validity of the SCREENIVF. There was only one measurement point, so neither test–retest reliability nor responsiveness was assessed. We did not collect data on ART treatment type, so we could not test either the tool’s discriminative validity or the possibly changing status of the patients in different stages of ART. Form X of the STAI was used here, a version still predominantly administered in Hungary, a circumstance that, however, may make international comparisons problematic. Online sampling resulted in a lower than average response rate, which may decrease generalizability. Finally, the overlap of our study timelines with the COVID-19 pandemic may be a source of bias, since our design made it impossible to discern infertility stress per se from the distress caused by COVID-specific effects. Future studies are warranted to overcome the above shortcomings.

Conclusions

Our findings demonstrated the SCREENIVF to be a valid and reliable measure with high accuracy in predicting possible depression, which gives it significant clinical value in assessing the psychological status of Hungarian women pursuing ART. Given its fair screening precision and goal-adjusted flexibility, we highly recommend its administration in routine fertility care to promote mental health and thus raise the probability of treatment adherence and success.

Methods

Study participants

The study was nested in the recruitment and eligibility phase of a randomized controlled trial on the effects of a psychosocial intervention on women’s well-being and ART outcomes (Clinical Trials.gov: NCT04151485). The study was approved by the Semmelweis University Regional and Institutional Committee of Science and Research Ethics, Budapest (reference number: 83/2019) and was carried out in accordance with the tenets of the Declaration of Helsinki. The research is reported in conformity with the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) reporting guideline for studies on measurement properties of patient reported outcome measures36. The inclusion criteria were as follows: (1) female sex; (2) reproductive age (18 to 45 years); (3) fluency in Hungarian; and (4) failure to achieve pregnancy after 12 or more months of regular unprotected sexual intercourse37. Women treated in the Assisted Reproduction Center of the Department of Obstetrics and Gynecology of Semmelweis University, Budapest, between December 2019 and February 2023 (n = 2830) were consecutively contacted via email directing patients to a web survey. Sample size was determined so as to fall in between those in previous validation studies of the SCREENIVF38. Eleven addresses were erroneous; thus, 2819 emails were successfully sent. Participation was voluntary and anonymous and based on informed consent after learning about the purpose and data management of the research. The overall response rate was 22.9%, resulting in a sample of n = 647 responders. Questionnaires were completed online, designed in a way that missing data were not allowed. Two men were found to have provided data, which were removed from analysis. The final sample consisted of N = 645 participants (Fig. 2).

Figure 2
figure 2

Recruitment flow chart.

Full size image

Measurement instruments

SCREENIVF

The 34-item questionnaire16 consists of five subscales measuring five risk factors: (1) anxiety (10 items originating from the Spielberger State-Trait Anxiety Scale39); (2) depression (a 7-item version of the Beck Depression Inventory for screening among primary care patients24); (3) perceived social support (5 items, derived from the Inventory of Social Involvement40); and infertility-related cognitions of (4) helplessness (6 items) and (5) acceptance (6 items), all from the Illness Cognition Questionnaire for IVF patients41,42. Each subscale preserves the original question and response format of its parent questionnaire. Cutoffs on the subscales differ from culture to culture. Dichotomous scores are given on each risk factor: 1 if the patient’s score falls in the critical range on the respective subscale, and 0 if not. Adding up all subscale scores yields the final Risk Factors score ranging between 0 (no risk factor) and 5 (5 risk factors). Subjects are considered to be at risk when scoring above 0. Cronbach’s alpha values in our sample ranged between 0.81 and 0.93.

Spielberger State-Trait Anxiety Inventory (STAI)

The Spielberger State-Trait Anxiety Inventory39,43 assesses anxiety on two 20-item scales: the State Scale (STAI-S), measuring transient states of subjective fear, tension and vegetative excitement, and the Trait Scale (STAI-T), capturing a more stable tendency of an individual to become anxious. All questions are answered on 4-point Likert scales. The results on both scales can range from 20 to 80, where a higher score indicates greater levels of anxiety. Form X, freely available and most widely administered in research and practice in Hungary, was used in this study. Cronbach’s alpha values in our sample were 0.95 and 0.92, respectively.

Beck Depression Inventory (BDI)

The Beck Depression Inventory44,45 contains 21 items with 4-point Likert scale responses about symptoms of depression, such as pessimism, lack of satisfaction, and guilt. Results can range from 0 to 63. Conventional cutoff scores on the BDI yield the following categories: normal range (0–9 points), mild (10–19 points), moderate (20–29 points), and severe depression (30–63 points). The nonclinical/clinical cutoff of 18/19 routinely applied in Hungarian studies46 is almost identical to the 19/20 cutoff suggested by Beck and associates44. In the present sample, the questionnaire yielded a Cronbach’s alpha score of 0.90.

Multidimensional Scale of Perceived Social Support (MSPSS)

The Multidimensional Scale of Perceived Social Support47,48 consists of 12 items across 3 subscales, assessing the individual’s subjective social support from family, friends, and significant others on 8-point Likert scales. The higher the mean on a factor or altogether, the stronger the support perceived by the responder. Cronbach’s alpha value in our sample was 0.87.

Fertility Quality of Life scale (FertiQoL)

The FertiQoL14,17 is a 36-item instrument for the assessment of the fertility-specific quality of life (QoL) of individuals. The tool contains 2 general items, a core and an optional treatment section. Core FertiQoL is composed of four subscales: Emotional; Mind–body; Relational; and Social. Treatment FertiQoL comprises two subscales: Environment and Treatment tolerability. Response formats follow 5-point Likert scales. All scale scores range between 0 and 100, with higher scores indicative of better QoL. In the present study, only the Core FertiQoL was used. Internal reliability was 0.90 for the module and ranged between 0.75 and 0.86 for the subscales.

Sociodemographic information such as age, marital status, number of existing children, residence, education, employment status, personal perception of financial situation, and health information such as height, weight, duration and etiology of infertility were gathered.

Translation and adaptation

After obtaining permission via email from the developer of the original questionnaire, adaptation and translation were performed on version English 2.0 of the SCREENIVF following international recommendations49,50. Anxiety and depression subscale items were identified within the extant valid translations of the STAI and the BDI. For the remaining three subscales, the following steps were taken: forward translation by two translators fluent in both languages; consensus Hungarian version by a psychologist and a linguist; backward translation by two bilingual linguists; final version agreed upon by an expert committee of leading psychologists proficient in English and familiar with the research topic; and finally, pilot testing for comprehensibility, face validity and cultural appropriateness with 20 ART patients.

Descriptive statistics

Statistical analyses were performed with IBM SPSS for Windows, v20.051, and the lavaan R package52. All scale data were tested for normality of distribution with Shapiro‒Wilk tests.

Reliability and validity

Internal consistency was measured with Cronbach’s alpha computed on the basis of the factor loadings of each item in a subscale. Reliability is acceptable when Cronbach’s alpha is higher than.70, and values > 0.80 are commonly considered good53.

To test concurrent validity, Spearman rank coefficients were computed to assess correlations between SCREENIVF subscales (not normally distributed) and their corresponding criterion measures. To avoid overlap, items used in the SCREENIVF were removed from the BDI and the STAI, and correlation tests were run with the bulk of remaining items only.

As for construct validity, since the SCREENIVF is a compilation of preexisting, unidimensional scales, and its underlying factor structure has already been identified in earlier studies, exploratory factor analysis is no longer necessary54. To verify the postulated underlying latent constructs in the Hungarian version, confirmatory factor analysis (CFA) was performed. Instead of the maximum likelihood (ML) method, which assumes that the observed indicators follow a continuous and multivariate normal distribution, diagonally weighted least squares (DWLS) was used, a method suggested to be superior to ML when ordinal data are analyzed23. All five factors were allowed to correlate. No cross-loadings or correlated errors were allowed. The goodness of fit of the model was evaluated by χ2 tests, the standardized root mean square residual (SRMR), root mean square error of approximation (RMSEA), and the comparative fit index (CFI). A model is considered to show good fit with SRMR/RMSEA < 0.08, and CFI/TLI > 0.90/0.9555,56,57.

Cutoffs, sensitivity and specificity

A review of cutoffs applied in the original16 and the validation studies of the SCREENIVF18,20,21,22 was performed to inform thresholds for use in the present study. First, cutoff points for all Hungarian SCREENIVF subscales were calculated on the basis of sample means plus/minus standard deviations (M ± SD), as adequate. In the case of the Depression subscale, two other cut-points were also tested: 3/4, suggested by the developers of the SCREENIVF following Beck and colleagues24, and 4/5, a demarcation point resulting from a two-step cluster analysis that reveals natural groupings in a dataset. A similar cluster analysis was also performed on the SCREENIVF Risk Factors scale results.

Sensitivity and specificity measurements were performed for depression, the most commonly used indicator of psychological maladjustment in infertility, which, unlike anxiety, has internationally accepted scale cutoffs as reference points. Sensitivity here refers to the ability of the SCREENIVF to correctly identify patients falling into the ‘clinically depressed’ category on the BDI, while specificity refers to its ability to correctly identify patients without a mood disorder. Binary logistic regression was used to test the power of the SCREENIVF with different cut points to predict BDI depression categories. Receiver operating characteristic (ROC) curves were calculated for all SCREENIVF subscales and the Risk Factors scale, with BDI scores serving as a reference. ROC curves are a plot of false positives against true positives, where the closer the area under the curve (AUC) of a test is to 1.0, the better it is in terms of sensitivity and specificity58. AUC is heuristically interpreted as small (0.5 < AUC ≤ 0.7), moderate (0.7 < AUC ≤ 0.9), or high (0.9 < AUC ≤ 1)59. Youden Indexes of different cut-points were computed60. For all analyses, p < 0.05 was considered an indication of significance.