1 Introduction

Self-efficacy theory (Bandura, 1997) has provided a crucial framework for understanding job search beliefs and effort for 30-odd years (see Kim, Kim, & Lee, 2019; Saks, Zikic, & Koen, 2015). Job search self-efficacy is defined as the confidence in one’s ability to successfully search for jobs and to gain employment, and has long been found to be the most proximal determinant of employment among job seekers (Eden & Aviram, 1993; Kanfer & Hulin, 1985; Kanfer, Wanberg, & Kantrowitz, 2001; Saks & Ashforth, 1999). Despite this, the measurement of job search self-efficacy has been somewhat diverse with various researchers developing and using their own measure to assess the construct (see Caplan, Vinokur, Price, & van Ryn, 1989; Ellis & Taylor, 1983; Wanberg, Zhang, & Diehn, 2010; van Hoye, Van Hooft, Stremersch, & Lievens, 2019). To address the proliferation of job search self-efficacy measures in the job search literature in order to offer opportunities for the comparison of research results across studies, Saks et al. (2015) organised the various measures and integrated them into a single measure, the Job-Search Self-Efficacy (JSSE) scale. The JSSE scale is a 20-item measure, composed of two dimensions that tap important aspects of the job search beliefs and outcomes. Saks et al. (2015) named the two dimensions (a) Job-search self-efficacy behaviour (JSSE-B) and (b) Job-search self-efficacy outcomes (JSSE-O). The initial validity information reported by the authors for the JSSE is promising.

Yet, to date, the psychometric properties of the JSSE have been based only on classical test theory (CTT) approaches. The CTT approach has inherent weaknesses, which limit its capability to provide full diagnostic information on the functioning of the JSSE. Most notably, the CTT approach cannot evaluate the 5-point Likert rating scale of the JSSE (see Kean, Bisson, Brodke, Biber, & Gross, 2017; Petrillo, Cano, McLeod, & Coon, 2015). Consequently, it remains unknown whether respondents use all of the response categories of the JSSE or not. Given job search self-efficacy’s importance in the job search literature, there is a clear need for further psychometric evaluation of the JSSE with robust psychometric techniques from modern test theory.

Moreover, there is little information about the JSSE’s ability to achieve measurement equivalence across gender because Saks et al. (2015) did not examine it. Thus, we do not know whether male and female respondents would interpret the scale items and the latent factor in the same way. This important information is needed because some previous work has shown that there are gender differences in job search activities, in general, and job search self-efficacy, in particular (Cifre, Vera, Sánchez-Cardona, & de Cuyper, 2018; Llinares-Insa, González-Navarro, Córdoba-Iñesta, & Zacarés-González, 2018; Papyrina, Strebel, & Robertson, 2020). For example, Eriksson and Lagerström (2012) found that, compared with men, women job seekers searched for jobs in areas closer to where they lived and that they received fewer firm contacts. They concluded that because the labour market is not gender-neutral, gender differences should be considered central in job search research. One advantage to gain from establishing the measurement equivalence of the JSSE across gender is that it will make it possible for future studies to meaningfully compare groups or individuals over time (see Davidov, Meuleman, Cieciuch, Schmidt, & Billiet, 2014; Jones, 2019; Putnick & Bornstein, 2016).

The aim of the current study was to provide further validity evidence for the JSSE using Rasch measurement (Rasch, 1960), which is one of a family of item response models. Rasch analysis has the capability to transform ordinal level Likert scale data into an interval level data using the raw score-to-logit transformation (Andrich & Marais, 2019; Bond & Fox, 2015; Boone, Staver, & Yale, 2014Footnote 1). The linear Rasch measures allow for the differences between the response categories to be interpretable (Andrich, 1978, 2011; Boone & Noltemeyer, 2017; Harwell & Gatti, 2001; Medvedev et al., 2019). Another aim was to examine if the JSSE latent construct and items have the same meaningful structure for men and women. To achieve the research aims, this study used data from recent university graduates. This is because recent university graduates generally search for jobs to enable them to use their newly acquired knowledge and skills (i.e. school-to-work transition). Young graduates seeking jobs, following the completion of compulsory schooling, may find the process challenging and would require personal agency in the form of self-efficacy beliefs to persevere, devote more efforts, and to perceive control over the job search process.

2 Method

2.1 Participants

Data for the current analysis are from the School-to-Work-Transition (SWoT) study. The SWoT study aims at validating selected job search and career interest measures. A total of 480 recent university graduates in the Greater Accra Region of Ghana who were making the school-to-work transition were invited to participate in the SWoT study. Some individuals did not return their completed questionnaire whereas others abstained for various reasons. The final sample (N = 429) comprised 50.1% men and 49.9% women with a mean age of 24.11 years (SD = 2.80). Participants completed a questionnaire battery on job search and career interest together with consent forms. Inclusion criteria were being an unemployed recent university graduate looking for a job or serving under the one-year mandatory national service programme, and a willingness to participate. The study was approved by the Institutional Review Board of the University of Ghana (Ref#: ECH116/19-20).

2.2 Measure

Job search self-efficacy was assessed with the 20-item JSSE scale developed by Saks et al. (2015). The JSSE is a self-report measure (see Table 1), composed of two dimensions consisting of 10 items each. The two dimensions are Job-search self-efficacy behaviour (JSSE-B; 10 items) and Job-search self-efficacy outcomes (JSSE-O; 10 items). The JSSE asks respondents about their confidence/beliefs to successfully engage in job search behaviours and to receive favourable job search outcomes. All of the 20 items are rated on a 5-point Likert scale from 1 (not at all confident) to 5 (totally confident). Saks et al. (2015) reported Cronbach’s alpha reliability of 0.89 and 0.96 for the JSSE-B and JSSE-O, respectively, in a sample of 487 job seekers. Pilot work led to a slight modification of two phrases “cold calls” and “sales pitch” in two of the items used in the original study. In the current study, “cold calls” was modified to read “unsolicited calls” (item#4) and “sales pitch” was re-worded to read “persuasive talk” (item#6). In addition to the JSSE, study participants provided biographical data on gender and age (years).

Table 1 Response category functioning of the Job-Search Self-Efficacy (JSSE) scale (N = 429)
Full size table

2.3 Statistical Analysis

The statistical analysis was undertaken in WINSTEPS® (v4.4.2; Linacre, 2019), an item response theory based Rasch analysis software. To evaluate how well the JSSE data fit the Rasch model, the Rasch-Andrich rating scale model (RSM) was used (Andrich & Marais, 2019). The RSM was used because the JSSE consists of polytomous items that are rated on the same Likert response scale. RSM analysis was conducted separately for each of the two dimensions of the JSSE (i.e. JSSE-B and JSSE-O). Item difficulty, person and item reliabilities, person and separation indices, point-measure correlation, response category functioning and threshold ordering, construct unidimensionality, and differential item functioning were all calculated in WINSTEPS®. Data-Rasch model fit was evaluated using the information-weighted sensitive fit (infit) and outlier-sensitive fit (outfit) mean-squared (MNSQ) cut-off values greater than 0.5 but less than 1.5, as recommended by Aryadoust, Ng, and Sayama (2021) and by Linacre (2020). Infit MNSQ and outfit MNSQ statistic greater than 0.5 but less than 1.5 indicate good data-model fit (Boone et al., 2014).

Rasch models assume that items in a measure are unidimensional and locally independent (Edwards et al., 2018; Fan & Bond, 2019; Linacre, 2009). To test the assumption of unidimensionality and local independency, the principal components analysis of residuals (PCA-R) test in WINSTEPS is used. The PCA-R looks for patterns in the data (i.e. the correlation matrix of the residuals) to find the component/dimension that accounts for the biggest variance in the residuals (Chou & Wang, 2010; Smith & Miao, 1994; Wright, 1994, 2000). The raw variance explained by the measures in the data is referred to as the “Rasch dimension/component” (Boone & Staver, 2020). Any remaining raw unexplained variance in the data is referred to as the “secondary dimension/component” or “contrast”. By default, WINSTEPS produces up to five secondary dimensions/contrasts in the data (see Boone & Staver, 2020). To satisfy the unidimensionality assumption in Rasch measurement, the unexplained variance in the first contrast/first secondary dimension should have an eigenvalue less than 2.00 (see Linacre, 1998a, 1998b; Raîche, 2005; Smith, 2002).

Overall, the PCA-R computation processes are similar to the traditional dimension reduction PCA test used in common factor analysis, except that the PCA-R results are interpreted differently because they indicate contrasts between opposing components rather than loadings on a factor. Note also that WINSTEPS computes a PCA using the residuals instead of the original observations. For more information on the WINSTEPS data analysis procedures, see Linacre (2020). Local independence is an additional indicator of construct unidimensionality. Locally independent items suggest that the answer to one item on the measure is independent of the answer to another item (De Bruin & Henn, 2013; De Bruin, Hill, Henn, & Muller, 2013; Hattie, 2015; Wilson, 1988). In other words, a wrong or correct response to an item should not bring about a wrong or correct response to another item on the measure.

Construct unidimensionality was evaluated through the principal components analysis of the standardised residuals (PCA-R). Each subscale was considered unidimensional if the Rasch dimension explained more than 40% of the total variance (Cordier et al., 2017; Linacre, 2020) and the eigenvalue of the first contrast/secondary dimension was less than 2.00 (see Linacre, 1998a, 1998b; Raîche, 2005; Smith, 2002). Additionally, residual correlation values between item pairs were inspected to determine local independence (Edwards et al., 2018; Testa et al., 2019). Items were considered locally independent of each other if the relationship between any two item pairs was (r < 0.30; Lambert et al., 2013; Røe et al., 2014; Zhong et al., 2014). Average measures and thresholds were calculated to assess the response category functioning. Response category functioning was considered acceptable if the Andrich thresholds were correctly ordered (i.e. progressed monotonically; Hansen & Kjaersgaard, 2020), supplemented by well-ordered category probability curves and fit statistics less than 2.00. Interpreted much like Cronbach’s alpha, person and item reliabilities typically range from 0 to 1 with estimates >0.8 considered evidence of good internal consistency reliability (Bond & Fox, 2015; Boone & Noltemeyer, 2017).

Separation indices provide important, additional information about a test’s functioning relative to internal construct validity. Separation indices range typically from 0 to infinity, and higher indices are indicative of better test functioning (Boone, 2016; Boone et al., 2014). Person separation index shows how well scale items distinguish participants into different ability levels on the trait. A person separation index >2.00 is deemed desirable. Relatedly, item separation demonstrates how well the study participants can separate the items along the trait construct (Bond & Fox, 2015). An item separation index >2.00 reflects the ability of the test to distinguish, at least, two-person ability levels (e.g. low versus high) along the construct (Chang, Wang, Tang, Cheng, & Lin, 2014). Uniform differential item functioning (DIF; Henn & Morgan, 2019) analysis was performed across gender to investigate whether the JSSE items were interpreted and used in the same way by men and women. When both DIF contrast is <.50 and significance level of Mantel χ2 statistic is (p > .05), they indicate lack of noticeable DIF across groups.

3 Results

3.1 Response Category Functioning

The JSSE uses a 5-point Likert rating scale from 1(not at all confident) to 5 (totally confident) to assess job search self-efficacy beliefs. In this study, the average measures progressed monotonically with each of the five response categories (see Table 1). Fit statistics of the response categories were excellent, with the infit MNSQ statistic ranging from 0.91 to 1.27 and the outfit MNSQ statistic ranging from 0.90 to 1.21. The fit statistics provided further evidence that the items on the JSSE were correctly ordered relative to their difficulty. Inspection of the category probability curves for each dimension revealed that each response category exhibited a clear-cut peak on the underlying construct with four distinct Andrich thresholds. The probability curves indicated that each response category was the most probable of being endorsed or agreed with along the trait continuum. Figures 1 and 2 report the test information function/curve, which plots information in the data relative to each measure or score on the scale.

Fig. 1
figure 1

Test information function for the JSSE-B. Note. The test information function/curve displays information for the JSSE-B along the latent trait continuum. The curve shows that the JSSE-B yields most information between approximately −2.0 and +1.5 logits (see rotated rectangle). Approximately 82% of the person measures fell within the rotated rectangle area, which indicates that the JSSE-B yields sensitive measures for most of the participants. JSSE-B =job-search self-efficacy behaviour

Full size image
Fig. 2
figure 2

Test information function for the JSSE-O. Note. The test information function/curve displays information for the JSSE-O along the latent trait continuum. The curve shows that the JSSE-O yields most information between approximately −2.0 and +2.0 logits (see rotated rectangle). Approximately 85% of the person measures fell within the rotated rectangle area, which indicates that the JSSE-O yields sensitive measures for most of the participants. JSSE-O = job search self-efficacy outcomes

Full size image

3.2 Data-Model Fit

The fit of the JSSE data to the Rasch model was evaluated using the infit and outfit MNSQ goodness-of-fit statistics. Results showed that the JSSE data fitted the Rasch model very well (see Table 2). As can be seen in Table 2, the infit MNSQ statistic for the 10-item JSSE-B ranged from 0.90 to 1.40 whereas the outfit MNSQ statistic ranged from 0.90 to 1.41. Both the infit and outfit MNSQ values were well within the acceptable cut-off criteria of >0.5 and <1.5 recommended by Linacre (2020) and by Aryadoust et al. (2021). In addition, the point-measure correlation (PTMEA) estimates were strong and in the positive direction from (r = .62) to (r = .72). Similarly, for the 10-item JSSE-O, the infit MNSQ values were from 0.87 to 1.17 and the outfit MNSQ values were from 0.87 to 1.14. The PTMEA estimates ranged from (r = .67) to (r = .75).

Table 2 Item difficulty and fit statistics of the 20-item Job-Search Self-Efficacy (JSSE) scale (N = 429)
Full size table

3.3 Unidimensionality and Local Independence

Principal components analysis of the residuals showed that the Rasch dimension explained 45.8% of the total raw variance in the JSSE-B data. The eigenvalue of the first component (i.e. first contrast) of the raw unexplained variance was 1.83 and satisfied the recommended cut-off criterion of eigenvalues below 2.00. Thus, the JSSE-B data demonstrated unidimensionality with very little possibility of a secondary dimension emerging from the data. Inspection of the item pair residual correlations revealed that the JSSE-B’s items were locally independent of each other because the largest correlation value was (r = .16; between item#1 and item#2). For the JSSE-O data, the Rasch dimension explained 48.3% of the total raw variance and the eigenvalue of the first component of the raw unexplained variance was 1.84. This eigenvalue confirmed the unidimensionality of the JSSE-O and revealed that the unexplained raw variance was substantively too random to constitute a secondary dimension. Similarly, inspection of the item pair residual correlations showed that the JSSE-O items were locally independent of each other. The largest item pair correlation value for the JSSE-O items was (r = .20; between item#11 and item#13).

3.4 Reliability and Separation Indices

Rasch reliability analysis showed that both the JSSE-B and JSSE-O had good internal consistency reliability estimates and separation indices. The person reliability estimate of the JSSE-B was 0.87 (person separation index = 2.58) whereas the item reliability was 0.83 (item separation index = 2.18). For the JSSE-O, the person reliability was 0.88 [person separation index = 2.70]) and the item reliability was 0.82 [item separation index = 2.16]).

3.5 Differential Item Functioning

Uniform DIF analysis across gender (male vs. female) using the recommended (a) DIF contrast cut-off value >0.5 logits and (b) Mantel χ2 statistic with p < 0.05 indicated that no noticeable DIF was present in the data across the two groups for both JSSE-B and JSSE-O (see Table 3). Note that for DIF to be noticeable both Mantel χ2 statistical significance (p < 0.05) and DIF magnitude (>0.5 logits) should be established contemporaneously. The current results demonstrated that men and women did not differ in their interpretation and use of JSSE items.

Table 3 Differential item functioning across gender of the Job-Search Self-Efficacy (JSSE) scale (N = 429)
Full size table

4 Discussion

In this study, we evaluated the psychometric properties of the JSSE scale (see Saks et al., 2015), a widely used instrument in job search research (see Kim et al., 2019), in a sample of recent university graduates. Rasch analysis, an item response theory approach, was used to perform the psychometric evaluation. The use of Rasch analysis enhanced a detailed diagnostic assessment of the functioning of the JSSE. Various psychometric properties were examined, including response category functioning, construct unidimensionality, person and item reliability, data-model fit, and measurement invariance across gender. Overall, Rasch analysis provided sound evidence that the JSSE is a reliable measure for assessing job search self-efficacy beliefs and outcomes in job seeker populations. In the paragraphs that follow, we offer a detailed discussion of the results.

The 5-point Likert rating scale of the JSSE displayed a stable structure and satisfied the monotonicity assumptions inherent in Rasch measurement. The expectation of the Rasch measurement is that for each scale item, the response category should be properly ordered relative to the item’s difficulty (Hagquist & Andrich, 2017; Khine, 2020). In order words, the Rasch measurement expects that respondents should demonstrate a lower challenge of agreeing with or endorsing the response category “not at all confident” than the response category “only a little confident” and that successive response categories should be endorsed following this response pattern (Boone et al., 2014; Linacre, 2020). In this study, the response categories functioned in this expected order as indicated by the average measures and Andrich thresholds (see Table 1), suggesting that there was no disordering of the category thresholds and that there was no need for rescoring the responses. That is, respondents in the current study made good use of all of the five response options in endorsing the items.

Moreover, the JSSE-B and JSSE-O demonstrated unidimensionality as expected by the Rasch measurement theory. For both dimensions, the unexplained variance of the first contrast or first secondary dimension had an eigenvalue less than 2.00, suggesting that no secondary dimension was probable in the data. In other words, for each dimension of the JSSE, the unexplained variance of the first contrast/first secondary dimension displayed only one-item strength (i.e. had only one item), which falls short of the requirement that, at least, the strength of three items (Boone & Staver, 2020; Bravini, Giordano, Sartorio, Ferriero, & Vercelli, 2017; Linacre, 2020) is needed for a secondary dimension to be considered meaningful and acceptable. In addition, all of the items on the JSSE fitted the Rasch model well, suggesting that all items tapped their respective latent constructs. These results reflected good targeting between item difficulty and person ability. Besides, the item difficulty calibrations ranged from −0.17 to 0.33 logits for the JSSE-B and from −0.34 to 0.26 logits for the JSSE-O. Person and item reliabilities for the two dimensions were found to be good (JSSE-B: 0.87, 0.83; JSSE-O: 0.88, 0.82). That is, the results showed that the JSSE was able to locate items and persons along the trait continuum with sufficient precision. The reliability estimates obtained in this study are consistent with the requirement of the Rasch measurement (Boone & Noltemeyer, 2017).

Relatedly, the separation indices of the JSSE were good. For example, the person separation index of the JSSE-B was 2.58 while item separation index was 2.18. For the JSSE-O, person separation index was 2.70 and the item separation index was 2.16. These results indicated that (a) the sample size used in this study was large enough to establish the item difficulty hierarchy of the JSSE, and (b) the JSSE was sensitive enough to classify respondents into, at least, two ability levels (i.e. high job search self-efficacy versus low job search self-efficacy). These psychometric properties provide useful information about the utility of a measure such as the JSSE (see Bragstad et al., 2020).

Furthermore, there was no noticeable DIF in the interpretation of the items by male and female graduates in this study. The absence of noticeable DIF (>0.5 logits) in the JSSE indicates that the items demonstrated consistency in meaning across the two groups. This finding suggests that the JSSE is sufficiently robust enough to be used to detect differences in job search self-efficacy mean scores across groups or to meaningfully compare the job search self-efficacy mean score of individuals over time. Measurement equivalence is considered an important property of good measures (Davidov et al., 2014; Jones, 2019) because it provides opportunities not only  to meaningfully compare groups but also to assess a measure’s responsiveness.

5 Limitations

This study has some limitations. First, job search self-efficacy was assessed using a self-report measure. Self-report data can be compromised by threats such as common method bias and social desirability. Second, although the sample size used in this study was found to be adequate by the Rasch model, a larger sample size may have enhanced the precision and fit of the data to the Rasch model. Third, participants in this study were recruited using convenience sampling techniques. Thus, generalization of the results ought to be done with caution. Nonetheless, this study provided evidence to the effect that, in its current form, the 20-item JSSE scale fulfilled the requirements of the Rasch measurement. Fourth, DIF was not examined across age, ethnicity, and academic disciplinary background (e.g. social science versus humanities or physical sciences) in the present study. This limitation may be considered as an important idea for future research on the JSSE.

6 Conclusion

Job search self-efficacy (JSSE) scale is a popular measure in the job search literature. However, previous psychometric work on the measure relied only on classical test theory. This study used Rasch analysis, an item response theory approach, to provide useful information on the reliability and internal validity of the measure. Overall, the results showed that the JSSE has sound psychometric properties and can be used for job search research among recent university graduates making the school-to-work transition.