In this phase, we 1) present information on the development and factor structure of the ASRS, 2) demonstrate internal and test-retest reliability and 3) model the relationship between trait ASR and global self-esteem using factor analysis and structural equation model (SEM) techniques.
We aimed to develop a unidimensional trait ASR scale since we were interested in capturing the global dimension of (character-based) appraisal self-respect (i.e., global self-appraisal as having a respectworthy honourable character). A focus on the global dimension (as opposed to the individual underpinning character traits) is theoretically justified because it is generally agreed that ASR makes most sense as a holistic concept (see Kristjansson, 2007). We aimed to develop a measure for use in the general adult population that was brief for ease of administration in basic and applied research contexts.
As a specific self-evaluation as having respectworthy honourable character traits, ASR is narrower in bandwidth than global self-esteem (see Clucas, 2019). In line with multidimensional models of self-esteem (see Marsh & Craven, 2006), we expected ASR to be distinct from, yet strongly related to global self-esteem. This was also expected based on prior research on state self-respect and ASR (Clucas, 2019; Kumashiro et al., 2002). Like other domain-specific self-evaluations, ASR was expected to contribute to global self-esteem though a bottom-up process, and also to be influenced by global self-esteem in a top-down fashion (see Rosenberg et al., 1995). Indeed, feelings of self-worth developed early in life support the use of self-enhancing strategies or biases to promote and protect feelings of self-worth in people with high self-esteem, such as engaging in selective social comparison processes, taking credit for success and excusing failure, and minimising self-descriptiveness of undesirable traits (Brown et al., 2001). High global self-esteem can, therefore, lead to inflated and non-realistic self-views, particularly with respect to global domain self-assessments (Baumeister et al., 2003), which are also more likely to be inflated by overall feelings of positivity (vs. negativity) associated with high self-esteem. Self-reported ASR is, therefore, likely to reflect global self-esteem in addition to grounded or “realistic” self-perception as having respectworthy honourable character traits.
Demographic sample details on age, gender and country of residence can be found in Table 1. Sample 1 (pilot study sample) was used to provide additional support for honourable character traits being perceived as respectworthy prior to the development of the ASR item pool and consisted of 80 university students recruited face-to-face across university campus contacts. Sample 2 was used to explore the factor structure of the ASRS (exploratory factor analysis (EFA) sample) and consisted of 219 university students and acquaintances of student researchers,Footnote 2 recruited through the psychology department online research participation system (RPS) as well as via face-to-face contacts on and off campus. Sample 3 was used to confirm the factor structure of the ASRS (confirmatory factor analysis (CFA) 1 sample) and consisted of a different sample of 371 university students recruited through RPS. Sample 4 (CFA 2 sample) was used to provide additional support for the ASRS’s factor structure in a more diverse general population sample of 731 non-student adults from Western English-speaking countries recruited through Prolific Academic, a crowdsourcing research platform structure (paid between £0.90 and £2.50 for their participation depending on questionnaire length – rate of around £6/h). Sample 4 was also used to model the relationship between ASR and global self-esteem. Seven participants were excluded from Sample 3 because they did not show engagement with the questions based on their response to negatively worded items (see Online Resource Appendix D) and 29 participants (4%) were excluded from Sample 4 because they failed the attention check on the questionnaire (see Online Resource Appendix D for supporting literature).
A subgroup of 255 participants from Sample 4 (Sample 4B) was invited to complete the ASR scale three months later to provide test-retest reliability information on the ASR scale. Test-retest reliability data was obtained from 196 participants (77% follow-up rate) of which 180 passed the attention check.
We aimed to obtain a pool of items from which to construct a unidimensional measure of trait ASR. Prior to generating the item pool based on theorising and existing research on respect, self-respect and honour, we carried out a pilot study to provide additional support for honourable character traits being perceived as respectworthy. We aimed to demonstrate that honourable character traits are perceived to be more closely related to self-respect than the more general construct of self-esteem, using vignettes, and gain a fuller picture of characteristics more closely associated with self-respect than with self-esteem through an inductive analysis of answers to open-ended questions.
Sample 1 completed a study in two parts. In Part 1, they completed two open-ended questions asking them about the personal attributes/characteristics that contribute to their self-esteem and self-respect. An inductive thematic analysis was performed to look for patterns in the open-ended responses, followed by McNemar tests to compare frequencies of mention of each characteristic for self-respect and self-esteem. In Part 2, participants were presented with ten scenarios describing a person as having or lacking an honourable character trait postulated or shown in the literature to be related to self-respect: Adherence to morals, criminal behaviour, dignified behaviour, magnanimity, moral courage, personal care (taking care of one’s appearance), respecting the environment and hard work - or self-esteem: Confidence in skills/abilities and competence (in sports) (see Online Resource Appendix B for supporting literature). Participants rated the scenarios on the extent to which they agreed that the person described had (a) high/low self-respect and (b) high/low self-esteem on 5-point Likert scales. Self-respect and self-esteem scenario ratings were compared using paired samples t-tests.
In accordance with prior literature (Clucas, 2019; Kristjansson, 2007; Kumashiro et al., 2002; Luchies et al., 2010), analysis of the open-ended questions showed participants listed honourable character traits when describing characteristics contributing to their self-respect, namely morals (standards and being law-abiding), strength of character (discipline and willpower), and dignity, which were mentioned significantly more frequently for self-respect than for self-esteem, supporting honourable character traits as being experienced as important respectworthy qualities. Similarly, paired samples t-tests supported the portrayed character traits of adherence to morals, criminal behaviour, not having dignified behaviour, magnanimity, moral courage, not showing personal care and not respecting the environment as influencing how respectworthy a person is by showing these traits to be more strongly related to self-respect than to self-esteem in scenario ratings (see Online Resource Appendix B for a fuller description of the results).
In contrast, the identified characteristics of attractiveness, confidence, positive attitude towards self and emotional state (e.g., anxiety and well-being) were mentioned significantly more frequently for self-esteem than for self-respect, and competence and confidence in skills/abilities were more strongly related to self-esteem in scenario ratings, reflecting the contribution of physical, emotional and performance self-esteem domains and the definition of global self-esteem as a person’s attitude towards the self as a whole (Marsh & Craven, 2006). These findings support self-respect as being a distinct and more specific self-evaluation than global self-esteem in line with theorising and prior research (Clucas, 2019).
Subsequently, we used existing theorising and research on respect, self-respect and honour (Clucas, 2019; Cross et al., 2014; Dillon, 2010; Kristjansson, 2007; Kumashiro et al., 2002; Luchies et al., 2010; Frei & Shaver, 2002; Prestwich & Lalljee, 2009; Uskul et al., 2012), complemented by the results of the pilot study, to develop items to capture self-perception as having key respectworthy honourable character traits, notably: Adherence to morals/standards, strength of character (including moral courage) and having dignified behaviour which in combination support trait ASR (see Kristjansson, 2007). We aimed to capture self-appraisal as having honourable character merits, as opposed to recognition of inherent human worth (Dillon, 2010; Renger, 2018). We started with 24 items but agreed on a pool of 18 items for further testing that most directly captured the concept of ASR (see Online Resource Appendix C for a list of the items). The items had simple phrasing and face as well as content validity, enhanced by having an expert in the field from outside the research team comment on the items. Both positively and negatively worded items were included to reduce acquiescence bias. Three of the items were taken directly from Kumashiro et al.’s preliminary short ASR measure: “I give in too easily to others’ wishes or requests” (RC), “I should treat myself better than I do” (RC) and “I have a lot of respect for myself”. The items were scored using 7-point Likert scales (1 = strongly disagree, 2 = disagree, 3 = slightly disagree, 4 = neither agree nor disagree, 5 = slightly agree, 6 = agree, 7 = strongly agree).
To examine the ASRS’s relationship with self-esteem, Sample 4 also completed the ten item Rosenberg (1965) Self-esteem Scale (RSES) (α = .92).
To explore the factor structure of the 18 initial ASR items, an EFA was conducted in Sample 2 using Robust Maximum Likelihood Extraction (RML) in FACTOR 10.8.04 (Lorenzo-Seva & Ferrando, 2013). Two CFAs were carried out in Mplus version 6 using Maximum Likelihood Robust estimation in Samples 3 and 4 to confirm the unidimensional factor structure of the final ASR scale. Model fit was evaluated using various indices: Tucker–Lewis Index (TLI) and Comparative Fit Index (CFI) values ≥ .90 and Root Mean Square Error of Approximation (RMSEA) values < .08 indicate acceptable model fit whilst TLI and CFI values ≥ .95, RMSEA values ≤ .06 and Standardized Root Mean Square Residual (SRMR) values ≤ .05 indicate good model fit (Byrne, 2013). For the EFA and CFA analyses, data were first checked for extreme univariate and multivariate outliers. Unidimensionality assessment I-ECV (Item Explained Common Variance) and ECV (Explained Common Variance) indices were obtained for all three samples from EFAs using RML in FACTOR.
To examine the test-retest reliability of the ASRS scale, an Intraclass Correlation Coefficient (ICC) was used. To examine the relationship of ASR with global self-esteem, an EFA of the 7 ASRS items and 10 RSES items was initially conducted in FACTOR using Robust Diagonally Weighted Least Squares (RDWLS) with polychoric correlations (since the RSES used a shorter 4-point rating scale) and Promin rotation in Sample 4. This was followed by full bifactor and correlated ASR self-esteem bifactor CFA analyses in Mplus 6 using Diagonally Weighted Least Squares (WLSMV) estimation to model ASR as a distinct subdomain of self-esteem and then estimate the correlation between the ASR factor and the general self-esteem factor.
Results and Discussion
Exploratory Factor Analysis (EFA)
The EFA identified one main factor based on parallel analysis (eigenvalue of 5.46, explained variance = 30.3%) in line with our theorising. Eight extreme outliers were removed prior to carrying out the EFA. With the exception of three items with very low factor loadings, factor loadings ranged between .30 and .82 (see Online Resource Appendix C). We then selected items most closely associated with the main factor, with factor loadings above .55 (Comery & Lee, 1992). After redundancy concerns were considered, we were left with seven items (see Table 2 for the list). Most of the items that were removed because of factor loadings below .55 were negatively worded, which is consistent with research showing that introducing negatively worded items can create a method artefact linked to differences in response style to positively and negatively worded items that compromises the unidimensionality of scales such as self-esteem and self-concept measures (Marsh, 1996). Moreover, participants often find it harder to respond to negatively worded items, increasing the risk of confusion and inaccurate responses (Marsh, 1996) (see Online Resource Appendix C for additional analyses supporting this explanation, and Online Resource Appendix D for other methods that we used to check for acquiescence bias or careless responding).
The factor with the retained seven items explained 51% of the variance in the items (eigenvalue of 3.58) (see Table 2 for factor loadings). The 7-item ASR scale (ASRS) correlated significantly with the deleted items’ average composite score, r(207) = .58, p < .001. Although fewer in number, the items still provided a good representation of the key character traits as described earlier, underpinning ASR.Footnote 3 The items were also broad rather than referring to specific experiences, making them more widely applicable.
Inspection of unidimensionality assessment item-level indices showed all seven items to satisfactorily represent the principal latent dimension (i.e., global dimension of appraisal self-respect) with I-ECV values over .77, see Table 2); I-ECV values represent the proportion of common variance explained by the first principal factor at the item level. An overall scale ECV (proportion of all common variance in the ASRS items explained by the first principal factor) of .90 (above the threshold of .70) also supported the final 7-item scale to be essentially unidimensional despite some heterogeneity in item content (Ferrando & Navarro-González, 2018). The factor was also strong and well-defined and, therefore, likely replicable with a Generalized G-H Index value of .87 (> .80, see Ferrando & Navarro-González, 2018).
Inter-item correlations for the seven items ranged from .28 to .63 (M = .43). Corrected item-total correlations ranged from .49 to .75 (α = .84, Omega total = .89, Omega H = .77Footnote 4). Descriptive statistics for the ASR items in the EFA and CFA samples can be found in Table 2.
Twelve extreme univariate and/or multivariate outliers were removed for CFA 1 in Sample 3 and 16 for CFA 2 in Sample 4. CFA 1 in Sample 3 confirmed the unidimensionality of the 7-item ASRS, showing that the one-factor model fit the data well with fit indexes within acceptable to good ranges: χ2(14, N = 359) = 36.11, p = .001, RMSEA = .067, TLI = .94, CF I = .96, SRMR = .04 (see Table 2 for factor loadings). Corrected item-total correlations ranged from .46 to .71 (α = .83, Omega total = .88, Omega H = .75). CFA 2 in Sample 4 also showed the one-factor model to fit the data reasonably well: χ2(14, N = 715) = 64.39, p < .001, RMSEA = .070, TLI = .94, CF I = .96, SRMR = .03, but the fit improved after correlating the residuals for the items “I take pride living according to my moral code” and “I will stick to my principles even if asked to do otherwise” as recommended by the modification indices (this suggestion makes theoretical sense since the two items both directly assess adherence to principles): χ2(13, N = 715) = 40.60, p < .001, RMSEA = .056, TLI = .97, CF I = .98, SRMR = .03 (see Table 2 for factor loadings). Corrected item-total correlations ranged from .58 to .71 (α = .87, Omega total = .90, Omega H = .82). Returning to CFA sample 1, we found the residuals for the two items to also be significantly correlated (p = .016), with the model fit also improving after allowing for the residuals to correlate (χ2(13, N = 359) = 30.32, p = .004, RMSEA = .062, TLI = .95, CFI = .97, SRMR = .03).
In both CFA samples, I-ECV values were all above .72 (CFA sample 1) or .76 (CFA sample 2, see Table 2) and the overall ECV was high (see Table 2), supporting the ASRS to be essentially unidimensional. The factor was also strong and well-defined (see Table 2 for G-H values). Samples 5/7 provided further evidence of a unidimensional factor structure for the ASRSFootnote 5 (see Online Resource Appendix G).
The ASRS showed acceptable test-retest reliability over a three-month period with an ICC of .76 [.69, .82] (N = 176). This supports the measure as tapping a relatively stable trait.
Relationship between ASR and Global Self-Esteem
The manifest correlation between the ASRS and RSES in Sample 4 was .58. In accordance with our predictions, an EFA of the 7 ASRS items and 10 RSES items in FACTOR supported ASR and global self-esteem as being distinct. Indeed, parallel analysis identified two factors, although an acceptable fit was only achieved with a 3-factor model (RMSEA = .072, CI95% = [.05, .08], vs. RMSEA (2-factors) = .088, CI95% = [.07, .10] and RMSEA (1-factor) = .135, CI95% = [.12, .15]): An ASR factor, and the RSES items divided into positive and negative self-esteem factors.
Subsequently, a CFA analysis was carried out to model ASR as a distinct subdomain of self-esteem by fitting a full bifactor model to the ASRS and RSES items with the ASRS items specified to load on the general self-esteem (g) factor as well as a third (ASR) grouping factor alongside the positive and negative self-esteem method factors. This model was a good fit to the data χ2(102, N = 686) = 384.58, p < .001, RMSEA = .064, TLI = .97, CF I = .98). Factor loadings on the ASR factor were all significant and ranged from .25 to .74, indicating notable amounts of unique variance for the ASR items over the g factor, supporting ASR as a distinctive subdomain of self-esteem (Gomez et al., 2015). See Fig. 1 in Online Resource Appendix F for a pictorial representation of the model with factor loadings. A similar bifactor model but with ASR and positive self-esteem items specified to load on the same grouping factor provided a worse fit: χ2(102, N = 686) = 589.34, p < .001, RMSEA = .083, TLI = .96, CFI = .97.
Lastly, we estimated a correlation of .59 between the ASR factor and the general self-esteem factor in a correlated ASR self-esteem bifactor CFA model in which a bifactor model was fitted just to the RSES items to account for the two positive and negative self-esteem method factors (see Hyland et al., 2014). This also supported the constructs as strongly related but distinct, with 65% of the variance being unshared. The model was a good fit, χ2(107, N = 686) = 484.81, p < .001, RMSEA = .072, TLI = .97, CFI = .97, after allowing the global self-respect item to cross-load on the general self-esteem factor as indicated by a large modification index (see Online Resource Appendix F for factor loadings).
The global self-respect item “I have a lot of respect for myself” also had the smaller loading of .25 on the ASR factor in the full bifactor model while the other ASR items had factor loadings over .50. This item was included to tap perceived respectworthiness of oneself as an honourable person, but as a more global assessment as worthy of respect, may not have been expected to load as highly on the ASR grouping factor. In particular, as a global evaluative item, it is likely to be heavily influenced by global affective feelings of self-worth in addition to grounding in honourable character (Baumeister et al., 2003). Other research has documented face valid global ratings of overall appearance and performance to similarly load more marginally on the appearance and performance domain specific grouping factors in a bifactor analysis (see Clucas, 2019). Supporting the ASRS as measuring a self-evaluation of respectworthiness is the stronger correlation of the total score rating for the six other items with this global item (r = .59) than with global self-esteem (r = .49), William’s t(671) = 4.22, p < .001. In addition, this global item had a high I-ECV value of .94 in the ASRS unidimensionality analysis for this sample, which was not paralleled by the more global RSES item “On the whole I am satisfied by myself” that showed an I-ECV of .46 when added to the ASRS unidimensionality analysis in this sample.
We provide additional evidence for ASR being distinct from global self-esteem in the next two phases by showing different association patterns for ASR and global self-esteem with other constructs. Furthermore, we demonstrate that the ASRS is primarily a measure of “realistic” self-evaluation as being a respectworthy honourable person, as opposed to simply reflecting self-esteem, by showing theoretically expected association patterns with moral self, principledness and prosocial variables predicted to be specifically related to ASR, that were not substantially influenced by adjustment for self-esteem.