According to national surveys from Australia, Norway, and the Netherlands, between 17.0 and 37.3% of children up to 18 years old live with a parent with a mental illness (Children of Parents with a Mental Illness; COPMI) (Goossens and Van der Zanden 2012; Maybery et al. 2009; Torvik and Rognmo 2011). Mostly qualitative studies showed that COPMI often experience feelings of guilt and shame regarding their parent with a mental illness (e.g., Bosch et al. 2017; Gladstone et al. 2011; Mordoch and Hall 2008). For instance, some children feel guilty and responsible for causing family problems (Gladstone et al. 2011). Adolescent COPMI may feel ashamed as a result of their awareness of the stigma concerning mental illness: they believe that parental mental illness should be concealed (Fjone et al. 2009; Mordoch and Hall 2008). Feelings of guilt and shame have been associated with experiencing more problems, such as depression or anxiety (Ferguson et al. 1999; Harder et al. 1992; Tangney and Fischer 1995). Therefore, measuring guilt and shame in adolescent COPMI is important.

The concepts of guilt and shame are often used interchangeably, because they are elicited by similar types of situations (Tangney 1992; Tangney et al. 1994). However, guilt and shame are different types of emotions (Lewis 1971). The crucial distinction lies in the fact that guilt pertains to a negative evaluation of a specific behavior (‘I did that wrong’), whereas shame is concerned with a negative evaluation of the global self (‘I did that wrong’). For this reason, when feeling guilty, people feel remorse and regret, and wish they had behaved differently (Tangney and Tracy 2012; Tracy and Robins 2004). When people feel ashamed, they feel worthless and inferior, leading to a desire to disappear or escape (Tangney and Tracy 2012; Tracy and Robins 2004).

Moreover, feelings of guilt and shame are useful as they regulate social life and interpersonal behavior: these feelings prompt people to behave in a social and morally appropriate way (Tangney and Tracy 2012). When feelings of guilt and shame are expressed intensely, frequently and inappropriately, they can become maladaptive (Barrett 1998; Harder et al. 1992; Schore 1996). In that case, feelings of guilt and shame may increase the risk of experiencing problems, such as depression or anxiety (Ferguson, et al. 1999; Harder et al. 1992; Tangney and Fischer 1995). Although it is quite common for children and adolescents to feel ashamed of their parent or to feel guilty (Schalkwijk 2014), COPMI may experience more frequent feelings of guilt and shame about their parent with a mental illness, than children and adolescents who do not have a parent with a mental illness. Therefore, COPMI might be at an increased risk to experience internalizing and externalizing problems, due to increased levels of guilt and shame (Van Loon 2015).

To prevent the development of internalizing and externalizing problems in COPMI, preventive interventions are offered on a large scale for COPMI in the Netherlands. COPMI can participate in peer support groups, where they can meet, talk and receive psycho-education (Van Santvoort et al. 2013). One aim of these programs is to reduce negative thoughts and emotions, such as guilt and shame, regarding parental mental illness. So far, there are no instruments available to measure if this aim of reducing negative feelings—particularly guilt and shame—is accomplished.

Several questionnaires have been developed to measure guilt and shame. However, these questionnaires have some limitations with regard to the measurement of guilt and shame in adolescent COPMI. First, many existing questionnaires measure proneness to guilt and shame, such as the Personal Feelings Questionnaire (PFQ-2; Harder and Zalma 1990). However, to study feelings of guilt and shame and its effect on outcomes, it is important to measure actual feelings of guilt and shame, instead of proneness to these feelings. Moreover, actual feelings of guilt and shame have been associated with increasing risk of developing problems (i.e., Ferguson et al. 1999). Therefore, we want to target these actual feelings instead of proneness to feelings of guilt and shame.

In addition, many guilt and shame questionnaires include descriptions of everyday-life situations in which people are likely to experience guilt and shame (e.g., Test of Self-Conscious Affect; Tangney et al. 2000). These descriptions of situations do not target COPMI, who may experience different or more specific feelings of guilt and shame. For instance, adolescents who do not have a parent with a mental illness may feel ashamed of their parents’ clothing or appearance, whereas adolescent COPMI may feel ashamed of their parents’ behavior related to their mental illness. Therefore, questionnaires that include situations in which people may feel guilty or ashamed are not useful for COPMI. In addition, studies about feelings of guilt and shame that have targeted COPMI are mostly qualitative studies (e.g., Bosch et al. 2017; Mordoch and Hall 2008) or measure these feelings very broadly with one item, such as ‘Sometimes children feel ashamed of their parent with a mental illness. Do you experience this?’ (e.g., Van Santvoort et al. 2013). Finally, most COPMI studies have focused on school-age children (e.g., Van Santvoort et al. 2013), whereas the likelihood of developing internalizing and externalizing problems tends to increase during adolescence (e.g., Graber and Sontag 2009; Farrington 2009).

All in all, research and practitioners could benefit from an instrument that measures guilt and shame in COPMI. Both researchers and practitioners may use such a questionnaire to establish feelings of guilt and shame in adolescent COPMI. Practitioners also could use it, for instance, to examine whether feelings of guilt and shame have changed after (preventive) interventions such as peer support groups.

The present study describes the development of the ‘Guilt and Shame Questionnaire—for Adolescents of Parents with a Mental Illness (GSQ-APMI)’, which is specifically designed for adolescent COPMI aged 12–21 years. The psychometric properties (i.e., validity and reliability) will be tested. We developed the GSQ-APMI using a mixed-methods approach which included three phases. In phase 1, ‘Operationalization and Item Development’, we worked together with COPMI experts to operationalize the concepts of guilt and shame and developed items based on the State Shame and Guilt Scale (SSGS; Marschall et al. 1994). Phase 2, ‘Pilot Testing’, tested the validity and reliability of the first-draft questionnaire and we adjusted the first draft based on these results. During phase 3, ‘Psychometric Properties’, we assessed the validity and reliability of the final draft of the GSQ-APMI in a new sample.

Methods

Participants

Phase 1: operationalization and item development

In phase 1, COPMI experts worked together to operationalize the concepts of guilt and shame. No participants were involved in this phase.

Phase 2: pilot testing

During this phase, Dutch families were recruited from participants in a previous longitudinal study of adolescent COPMI (Van Loon et al. 2013). For a detailed description of the sampling and study procedure, see Van Loon et al. (2013). From this previous study, families were selected that met the following inclusion criteria: families included at least one biological parent, and children had to live with their parent with mental health problems and had to be between the ages of 12 and 21. A total of 90 families fulfilled the inclusion criteria and were approached via telephone, and 67 families agreed to participate. From these 67 families, we included the 60 families in which at least one biological parent still experienced mental health problems. From these 60 families, we included 79 adolescents who fulfilled the inclusion criteria and agreed to participate. Also, these adolescents were aware of their parent’s psychological problems. Not all parents in this sample have informed their adolescent children about their mental health problems. Therefore, only adolescents who answered ‘yes’ on the question about whether their parent(s) experience psychological problems were asked to fill out the GSQ-APMI. Table 1 describes characteristics of both parents and adolescents in samples from phase two and phase three.

Table 1 Characteristics of adolescents and parents in phases 2 and 3
Full size table

In addition, we included a control group of 108 high school students (Table 1) who filled out the questionnaire about both parents.

Phase 3: psychometric properties

We tested the psychometric properties of version B of the questionnaire in a new group of adolescent COPMI. We approached group leaders of adolescent COPMI support groups at different mental health institutions. To be able to participate, adolescents had to be between ages 12 and 21. A total of 55 COPMI were included; 47 of the adolescent COPMI met the inclusion criteria (Table 1). Eight COPMI did not met the criteria, because they were older than 21.

To test discriminant validity, we also included a control group from a master’s thesis project (Maas, 2016). This control group consisted of 112 high school students without a parent with mental health problems (aged 12-18 years) who filled out the questionnaire about both parents (Table 1).

Procedures

Phase 1: operationalization and item development

The items of the first draft (version A) of the GSQ-APMI were based on the State Shame and Guilt Scale (SSGS; Marschall et al. 1994), using the same formulations to measure feelings of guilt and shame, such as ‘I feel like apologizing’ and ‘I want to sink into the floor and disappear’. We collaborated with two COPMI experts to determine in which situations adolescent COPMI may feel guilty or ashamed. We identified different situations such as when their parent with mental health problems has a bad day or when they are in public with this parent. This resulted in 21 items, which are listed in Table 2. An example is ‘If I am in public with my parent with problems, I want to sink into the floor and disappear’. Adolescents had to answer how often they experienced feelings of guilt or shame, with answers ranging from 1 (never) to 5 (always). The items were originally developed in Dutch. The items were translated to English and back-translated to Dutch by three independent readers, to ensure the English translations were in line with the original Dutch items.

Table 2 Factor loadings for principal component analysis with oblimin rotation of the guilt and shame items (translated from Dutch)
Full size table

Phase 2: pilot testing

First, parents and the participating adolescents filled out a consent form. Next, adolescents filled out the GSQ-APMI (version A), which was send to them via mail. To determine test-retest reliability, adolescents were asked to complete the questionnaire again one month later. The questionnaire for the control group was adjusted so that the items did not include ‘my parent with problems’, but only included ‘my parent’.

Our aim was to shorten the questionnaire, thereby making it more concise and easier to fill out. Based on the results of the pilot testing, we shortened and adjusted the GSQ-APMI (version B; Table 3).

Table 3 Factor loadings for principal component analysis with oblimin rotation of the guilt and shame items of the adjusted GSQ-APMI
Full size table

Phase 3: psychometric properties

Adolescent COPMI who participated in the support groups were asked at the end of a meeting by the group leader or by a trained research assistant if they wanted to participate in the present study. Adolescents received a letter with information about the study and could indicate whether or not they wanted to participate. If they wanted to participate, they filled out a consent form. Parents of adolescents who were younger than 16 years received a letter explaining the study, and they could indicate within one week if they did not want their child to participate (i.e., passive consent). After giving consent, adolescents filled out the questionnaire about both parents: first, about their parent with mental health problems and second, about their other parent with or without mental health problems. Finally, adolescents who agreed to complete the questionnaire again were asked to do it again approximately six weeks later, to determine test-retest reliability. The group leader or research assistant handed out the questionnaire and gathered the completed questionnaires.

In addition to the paper and pencil questionnaire, we also created an online version of the questionnaire. This could be used if there was not enough time to fill out the paper and pencil questionnaire during the support group meetings. If this was the case, adolescents still had to fill out a paper and pencil version of the consent form and their e-mail address. They were then sent an e-mail with a link to the online version of the questionnaire.

The questionnaire for the control group was adjusted so that the items did not include ‘my parent with problems’, but only included ‘my parent’. Again, a passive consent procedure was used.

Measures

In phase 2 ‘pilot testing’ participants filled out version A of the questionnaire, and in phase 3 ‘psychometric properties’ participants filled out version B of the questionnaire.

Data Analyses

Phase 2: pilot testing

To examine whether the GSQ-APMI measured two different constructs (i.e., guilt and shame), we conducted an exploratory factor analysis using oblimin rotations. Oblimin rotations were used because we assumed that the two components, guilt and shame, were correlated. As an extraction method, we used principal axis factoring. To assess face validity, we asked a COPMI expert with over 20 years of experience in working with COPMI about the questionnaire. Also, 18 adolescent COPMI were asked their opinion about the questionnaire in interviews. Next, we assessed convergent validity by correlating the results of the scales of the newly developed questionnaire with items about feelings of guilt and shame from the study of Van Santvoort et al. (2013). In that study, one item assessed feelings of guilt: ‘Children sometimes think that they are the cause of their parent’s problems. Do you ever think so?’ and one item assessed feelings of shame: ‘Sometimes children are ashamed of their father or mother with problems. Are you ever ashamed?’ (Van Santvoort et al. 2013). These items were answered on a 5-point scale (ranging from 1 never to 5 always). To assess discriminant validity, a control group of adolescents with a parent without mental health problems (N = 108; Table 1) filled out the questionnaires, to determine with t-tests if adolescent COPMI and adolescents in the control group differed in their sum scores on the developed scales.

We determined internal consistency by calculating Cronbach’s alpha for both the guilt and shame scales. Next, we determined test-retest reliability. In total, 47 adolescents agreed to fill out the questionnaire again one month later. Pearson correlations were calculated between scores of the guilt and shame questionnaire at time one, and time two, one month later.

Although higher ratios are generally better, Hatcher (1994, p. 73) recommends a minimum subject to item ratio of at least 5:1 in exploratory factor analysis. In phase 2 this resulted in a minimum of 105 participants (5 times 21 items). Therefore, the exploratory factor analysis was conducted with the COPMI and control group combined (N = 187).

Phase 3: psychometric properties

Consistent with phase 2, we examined whether version B of the questionnaire measured two different constructs (i.e., guilt and shame) by conducting an exploratory factor analysis using oblimin rotation. As an extraction method, we used principal axis factoring. To assess face validity, adolescents could give their opinion about the questionnaire at the end of the questionnaire, and were asked whether or not the questions captured situations in which they may feel ashamed or guilty. We also asked four different COPMI experts their opinion about the questionnaire. Next, convergent validity was assessed by correlating the results of the scales of the newly developed questionnaire with items about feelings of guilt and shame from the study of Van Santvoort et al. (2013). Finally, we could determine discriminant validity because adolescents filled out the questionnaire about their parent with and without mental health problems. Sum scores on the two subscales were compared to determine if adolescents differed in how much they experience feelings of guilt and shame about their parent with or without mental health problems. Sum scores on each subscale also were compared between the adolescent COPMI and the control group, to determine if adolescent COPMI and adolescents in the control group differed in how much they experience feelings of guilt and shame.

We determined internal consistency by calculating Cronbach’s alpha for both the guilt and shame scale. Next, we determined test-retest reliability. Adolescents (N = 10) who indicated they wanted to fill out the questionnaire again received the questionnaire again after approximately six weeks and filled it out again. Due to the small sample size, Spearman correlations were calculated between scores of the guilt and shame questionnaires at time one and time two.

Similar to phase 2, the minimum sample size was calculated by using an subject to item ratio of 5:1 for the exploratory factor analysis. For phase 3, this resulted in a required sample size of at least 60 participants (5 times 12 items). Therefore, the exploratory factor analysis was conducted with the COPMI and control group combined (N = 159).

Results

Phase 1: Operationalization and Item Development

The goal of Phase 1 was to develop the first version of the GSQ-APMI. Based on the SSGS scale and the experts’ knowledge, we first developed nine items to assess feelings of guilt and 12 items to assess feelings of shame (Table 2). The developed items for guilt and shame were combined in one questionnaire, where items about guilt where alternated with items about shame. In total, version A of the GSQ-APMI included 21 items on a 5-point scale with answers ranging from 1 (never) to 5 (always).

Phase 2: Pilot Testing

The goal of Phase 2 was examining the validity and reliability of version A of the GSQ-APMI, including 21 items. An exploratory factor analysis, using oblimin rotations, revealed a clear two component structure, with the components being guilt and shame (Table 2). However, one item (item 4) developed to measure shame loaded higher on the guilt component than the shame component. Moreover, three items (5, 11, 19) developed to measure shame, did not discriminate between the constructs (difference of 0.40 or less). Thus, these items may not adequately measure feelings of shame. These items (4, 5, 11, 19) were removed from the questionnaire.

Validity

Regarding the validity of the questionnaire, the developed questionnaire seemed to support face validity. A COPMI expert confirmed that the items did reflect feelings of guilt and shame in adolescent COPMI. Also, 18 COPMI were asked their opinion about the questionnaire. In general, they mentioned that the situations described in the items reflected situations in which they mostly felt ashamed of their parent or guilty. However, the adolescent COPMI did mention that they could not imagine feeling ‘humiliated’ (items 11, 14, 19). Therefore, these items were removed. The questionnaire also supported convergent validity: the sum score of feelings of guilt was strongly and positively associated with the guilt item from the previous study (r = 0.69, p < 0.01) (Van Santvoort et al. 2013). The sum score of feelings of shame was also positively and strongly associated with the shame item from the previous study (r = 0.83, p < 0.01) (Van Santvoort et al. 2013). These correlations show that the developed items and the items from the previous study measured the same constructs, namely feelings of guilt and shame. To assess discriminant validity, a control group of adolescents (N = 108) filled out the questionnaires, to determine if adolescent COPMI and adolescents in the control group differed in experienced feelings of guilt and shame. T-tests indicated no differences in feelings of guilt (t(188) = −1.61, p = 0.110) between adolescent COPMI (M = 15.13, SD = 6.91) and adolescents in the control group (M = 16.62, SD = 5.88), and feelings of shame (t(188) = −1.08, p = 0.282) between adolescent COPMI (M = 19.21, SD = 8.04) and adolescents in the control group (M = 20.34, SD = 6.45). These results implied that adolescent COPMI and adolescents in the control group did not differ in their experiences of guilt or shame and this did not support discriminant validity.

Reliability

Internal consistency was determined by calculating Cronbach’s alpha for the guilt (α = 0.94) and shame (α = 0.93) scales. These values indicated high reliability for both scales. Test-retest reliability was determined by sending the developed questionnaire again to a subgroup of 47 adolescents, approximately four weeks (M = 4.27, SD = 1.18), after these adolescents filled out the first questionnaire. Test-retest correlations were high for both guilt (r = 0.82, p < 0.001) and shame (r = 0.89, p < 0.001) (Dancey and Reidy 2017).

GSQ-APMI version B

Based on the outcomes of the psychometric properties and factor analysis of GSQ-APMI version A in Phase 2, we adjusted and improved the questionnaire. We removed items that did not discriminate between the two constructs (items 4, 5, 11, 19) or that included ‘humiliated’ (items 11, 14, 19). In addition, our aim was to develop a short questionnaire in order to make the questionnaire feasible to use in both research and clinical practice. Therefore, we also removed items with lower factor loadings or that were quite similar to each other (items 1, 2, 10, 18). This resulted in a total of 12 items with six items per scale (i.e., guilt and shame). Version B of the GSQ-APMI is presented in Table 3.

Phase 3: Psychometric Properties

The goal of Phase 3 was examining the validity and reliability of version B of the GSQ-APMI, including 12 items. An exploratory factor analysis, using oblimin rotations, revealed a two-component structure, with the components “Feelings of guilt” and “Feelings of shame” (Table 3). However, the factor loadings of items 3, 7 and 12 showed a difference of less than 0.40 between the two constructs, meaning that these items did not adequately discriminate between the two constructs. Items 3 and 12 were removed from the questionnaire. We aimed at five items per construct. Items 3 and 7 were aimed at measuring guilt, whereas item 12 was aimed at measuring shame. In order to result in five items measuring guilt, we had to choose between items 3 and 7: item 3 only showed a difference of 0.08 between the two constructs and therefore was removed, resulting in version C (Appendix). In addition, item 7 corresponds with qualitative data showing that children feel guilty and responsible for causing parental problems (e.g., Gladstone et al. 2011).

Validity

Version C of the GSQ-APMI also seemed to have face validity. Four COPMI experts thought that this set of items reflected important feelings of guilt and shame in adolescent COPMI. Also, adolescents who gave their opinion at the end of the questionnaire mentioned that the situations that were described mostly reflected situations in which they felt guilty or ashamed of their parent.

Version C also supported convergent validity. The sum score of feelings of guilt was positively associated with the guilt item from the previous study of Van Santvoort and colleagues (2013) (r = 0.83, p < 0.01). The sum score of feelings of shame was also positively associated with the shame item from that previous study (r = 0.84, p < 0.01). These correlations show that the developed items and the items from the previous study measured the same constructs (i.e., guilt and shame).

To assess discriminant validity, adolescents filled out the questionnaire about their parent with and without mental health problems, to determine if adolescent COPMI differed in how much they experience feelings of guilt and shame about their parent with or without mental health problems. T-tests indicated (t(26) = 1.10, p = 0.283) no differences in feelings of guilt about the parent with mental health problems (M = 12.13, SD = 5.54) and the parent without mental health problems (M = 11.29, SD = 5.98). These results implied that adolescent COPMI did not differ in their experiences of guilt about their parent with and without mental health problems. Therefore, this did not support discriminant validity for the guilt component. However, T-tests did indicate (t(26) = 2.90, p < 0.05) differences in feelings of shame about parents with mental health problems (M = 14.49, SD = 6.45) and parents without mental health problems (M = 9.58, SD = 5.58). Adolescent COPMI felt more ashamed of their parent with mental health problems than their parent without mental health problems. Therefore, this did support discriminant validity for the feelings of shame component.

We also assessed discriminant validity by comparing the sum scores for guilt and shame of a control group (N = 112) and the adolescent COPMI. T-tests indicated differences in feelings of guilt (t(73.52) = 4.82, p < 0.001) between adolescent COPMI (M = 12.66, SD= 5.48) and adolescents in the control group (M = 8.24, SD = 3.32), and differences in feelings of shame (t(56.95) = 4.97, p < 0.001) between adolescent COPMI (M = 12.72, SD= 6.31) and adolescents in the control group (M = 7.89, SD = 2.34): adolescent COPMI experienced significantly more guilt and shame feelings than adolescents without a parent with mental health problems. Therefore, these results supported discriminant validity.

Reliability

Reliability was determined by calculating Cronbach’s alpha for the guilt (α = 0.89) and shame (α = 0.93) scales. These values indicated high reliability for both scales. Test-retest reliability was determined by sending the developed questionnaire again to 10 adolescents, after these adolescents filled out the first questionnaire. Adolescents filled out the questionnaire again after approximately six weeks (M = 6.50, SD = 2.75). High test-retest correlations were found for guilt (r = 0.69, p < 0.001) and shame (r = 0.71, p < 0.05) (Dancey and Reidy 2017).

Discussion

The purpose of the present study was to develop a valid and reliable questionnaire to measure feelings of guilt and shame in adolescent COPMI (Children Of Parents with a Mental Illness). Guilt and shame could increase the risk of developing problems in COPMI. In peer support groups, COPMI can learn to deal with negative feelings related to parental mental illness, such as guilt and shame. However, existing questionnaires that measure feelings of guilt and shame have limitations, such as measuring proneness to feelings of guilt and shame instead of measuring actual feelings of guilt and shame. Also, these questionnaires do not measure situations that are specific for adolescent COPMI.

The results of the present study indicate that we developed a valid and reliable questionnaire: the Guilt and Shame Questionnaire—for Adolescents of Parents with a Mental Illness (GSQ-APMI). The final questionnaire includes 10 items, of which five items were developed to measure shame and five items to measure guilt (Appendix). Specifically, the GSQ-APMI results in one score reflecting feelings of guilt and one score reflecting feelings of shame (i.e., higher scores reflect higher feelings of guilt or shame). A total score is not applicable. Factor analytic and reliability criteria provided evidence for a 2-factor structure (i.e., guilt and shame) and internal consistency in Phase 2 (pilot testing) and Phase 3 (examining psychometric properties) of the study. Correlations with items from another study (Van Santvoort et al. 2013) confirmed the convergent validity of the GSQ-APMI. The comparison of sum scores of adolescents with and without parents with mental health problems provided good evidence for discriminant validity in Phase 3 (psychometric properties). In phase 2 the results did not seem to support discriminant validity: adolescent COPMI and adolescents in the control group did not differ in their experiences of guilt or shame. Although this might indicate that the developed questionnaire was not sensitive enough to discriminate between adolescents with and without parents with mental health problems, it may also be explained by the fact that adolescent COPMI and adolescents in the control group may not have been comparable. That is, adolescents in the control group included ages between 14 and 18 (M = 15.26, SD = 0.98), whereas adolescent COPMI ranged in ages between 12 and 21 (M= 16.52, SD = 2.90). Younger adolescents might feel more ashamed and more guilty than older adolescents; the latter group might better understand and sympathize with their parents, and therefore feel less ashamed or guilty.

Qualitative research already showed that adolescent COPMI experience feelings of guilt and shame (e.g., Bosch et al. 2017; Mordoch and Hall 2008). The GSQ-APMI offers both researchers and practitioners the opportunity to quantify these feelings of guilt and shame in adolescent COPMI. A questionnaire specifically developed for adolescent COPMI to measure feelings of guilt and shame did not exist yet. While developing this, we relied on the input of our target population: adolescent COPMI and practitioners working with adolescent COPMI. By taking into account their opinions and feedback about the questionnaire, we tried to capture their experiences and thereby enhancing the validity of the GSQ-APMI. The items that are included in the final version C were perceived relevant according to the target population itself. Also, two different adolescent COPMI samples (i.e., one community sample and one clinical sample) were included in this study, thereby trying to capture a wide range of experiences of feelings of guilt and shame of these adolescents. The GSQ-APMI specifically focuses on adolescent COPMI and their feelings of guilt and shame.

Limitations and Future Research Directions

The present study has several limitations. One limitation includes the assessment of test-retest reliability in Phase 3. The participants in this phase participated in a COPMI group program, which could have helped in decreasing feelings of guilt and shame. Therefore, their feelings of guilt and shame might not be stable over time and this might influence the results for test-retest reliability. Nevertheless, we found high test-retest correlations. In addition, the sample size of adolescent COPMI in phase 3 was relatively small. In phase 3, a sample of COPMI was included that participated in support groups. Gathering data in these groups entailed more difficulties than in the COPMI sample in phase 2 (where we could better control the recruitment and data collection). Despite this smaller sample size, we did find significant results in Phase 3. Another limitation entails that we could not examine if the type of mental illness might influence the results. If the groups would be divided into different types of mental illnesses, the sample size would be too small to detect an effect. Finally, the developmental age range is quite large (12 to 21 years old), due to pragmatic reasons. That is, this age range corresponded with the ages of the Dutch COPMI in the peer support groups.

Future research could examine whether this questionnaire is also applicable for COPMI younger than 12 years. It could be that these children experience feelings of guilt and shame more frequently or in different situations, which the GSQ-APMI did not capture. For example, one study showed that negative cognitions, such as guilt and shame, decrease when COPMI are well informed about their parents’ illness (Van Santvoort et al. 2014b). Possibly, adolescents have a higher degree of mental health literacy compared to younger children, and are thus better informed about their parents’ illness and therefore understand it better (Van Santvoort et al. 2014b). Therefore, adolescent COPMI may experience feelings of guilt and shame less or in different situations than younger COPMI. Moreover, in future research the age range could be adjusted so that it is more in line with a specific developmental stage (e.g., early, middle or late adolescence). However, there is no standard age range for defining adolescence. For instance, it has recently been suggested that “a definition of 10–24 years corresponds more closely to adolescent growth and popular understandings of this life phase” (Sawyer et al. 2018). In addition, future research could also examine whether the English version of the GSQ-APMI is valid and reliable for assessing feelings of guilt and shame in English-speaking adolescent COPMI. Finally, the final version C of the questionnaire, could be tested again in a larger COPMI sample to further validate the two factor structure of the questionnaire by conducting a confirmatory factor analysis and to examine if the results differ for different types of mental illnesses.

All in all, the present study showed that we developed a valid and reliable questionnaire. Particularly, researchers could use this questionnaire to assess feelings of guilt and shame of adolescent COPMI and use this in their research. Also, this questionnaire could be used to examine the effectiveness of (preventive) interventions, such as COPMI groups, in reducing feelings of guilt and shame. One of the aims of these preventive support groups is to decrease negative cognitions related to their parent’s illness, such as guilt, shame, and loneliness (Van Santvoort et al. 2014a). Specifically, the GSQ-APMI could be used to examine whether negative cognitions (i.e., guilt and shame) have decreased after such support groups.