Introduction

Loneliness is defined as an unpleasant feeling arising from a discrepancy between desired and perceived levels of social connectedness (Perlman & Peplau, 1981). Loneliness can be classified into social loneliness, characterized by the lack of social networks or sense of community, and emotional loneliness, characterized by the absence of a personal, intimate relationship (Perlman & Peplau, 1981). Although inconclusive, loneliness tends to be more common in older adults, patients with chronic diseases, with rates ranging from 21 to 34%, men than women, and vary across different cultures (Lasgaard et al., 2016; Rokach, 2018; Yang & Victor, 2011). However, more conclusive research has shown significant differences between married or having a partner than unmarried and singles (Park et al., 2019; Rokach, 2018; D Russell et al., 1980; Stephan et al., 1988).

In the general population loneliness is a major public health issue associated with poor mental and physical health outcomes (Gerst-Emerson & Jayawardhana, 2015). It has also been linked to poor structural social relationships, including being single and living alone as well as social problems such as alcoholism and suicide, which increase mortality risk (Rico-Uribe et al., 2018; Rokach, 2019; D Russell et al., 1980).

Given the above negative impact of loneliness on mental, physical, and social health, it is important that it be assessed using valid and reliable instruments. The University of California and Los Angeles Loneliness scale (UCLA-LS) is the most used scale for assessing loneliness in the general population. It was developed by Russell and colleagues in 1978 to assess social and emotional dimensions of loneliness (Dan Russell et al., 1978). It has become a key clinical and research questionnaire, and reports of its application have come from several countries/societies. The original scale consisted of 20 negatively worded self-reported questions designed to identify feelings of loneliness in college students (Dan Russell et al., 1978). To reduce possible systematic response bias, it was revised in 1980 to include a mix of ten negatively and ten positively worded items (D Russell et al., 1980). It was revised again (version 3) in 1996 (Russell, 1996) to create a simplified response format for easier use in broader groups of respondents, including older adults, and to be administered by mail, surveys and personal interviews. Participants respond to each item on a 1–4 Likert scale, from “never” to “always” The scale has a possible total score of 20 to 80 points, with a higher score indicating greater loneliness (Russell, 1996).

Several short versions of the UCLA-LS, named by the number of the included questions: UCLA−16 (Faustino et al., 2019), UCLA-11 (Lee & Cagle, 2017), UCLA-10 (Knight et al., 1988; Russell, 1996), UCLA−8 (Hays & DiMatteo, 1987), UCLA-7 (Oshagan & Allen, 1992), UCLA-6 (Neto, 2014; Niu et al., 2018), UCAL-5 (Lamm & Stephan, 1987), UCLA-4 (D Russell et al., 1980), and UCLA- 3 (Hughes et al., 2004) were developed to reduce questionnaire length and increase the response rate in clinical practice and longitudinal studies. The psychometric properties of the 20-item and all short forms of the UCLA-LS have been reported by several studies, and one systematic review (Vassar & Crosby, 2008) summarized the internal consistency reliability coefficients (alpha range, 0.71–0.95) of the UCLA-20 and concluded that the UCLA-20 was reliable for use in the general population (Vassar & Crosby, 2008). However, Vassar & Crosby, 2008 did not assess or report validity results and test-retest reliability of the UCLA-20 and the short versions of the UCLA-LS.

In this systematic review, we aimed to assess the validity (structural, construct, criterion, cross-cultural), reliability (internal consistency, test-retest), and the cross-cultural adaption of the 20-item and the short versions of the UCLA-LS. Such information will assist clinicians, especially those involved in social psychology, to select the most appropriate version of the UCLA-LS to be used in their population.

Methods

The systematic review was conducted according to the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) guidelines for systematic reviews (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). The cross-cultural adaptation was assessed using the Guidelines for the Process of Cross-Cultural Adaptation of Self-report Measures (Beaton et al., 2000). The manuscript was reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Page et al., 2021) The protocol registration number on PROSPERO is CRD42021241610.

Inclusion and Exclusion Criteria

Inclusion Criteria

  1. (1)

    Studies describing the development and assessing the psychometric properties (reliability, validity, responsiveness) of the UCLA-LS in adults.

  2. (2)

    Studies published in a peer-reviewed journal from inception until March 2021,

  3. (3)

    Studies that report the process of culturally adapting or translating the UCLA-LS into another language and evaluating its psychometric properties.

Exclusion Criteria

  1. (1)

    Studies that validated the UCLA-LS in children and adolescents,

  2. (2)

    Interventional studies that used UCLA-LS as an outcome measure,

  3. (3)

    Non-psychometric studies of the UCLA-LS such as qualitative studies,

  4. (4)

    Conference publications, abstracts, posters, editorials, and commentaries of the UCLA-LS.

Search Methods for Identification of Studies

We conducted electronic searches in PubMed, EMBASE, Scopus, and PsycINFO databases from inception until March 2021, using the following keywords: Reliability OR validity OR responsiveness OR psychometric properties OR measurement properties OR translation OR cross-cultural adaptation AND UCLA loneliness scale OR UCLA-LS OR University of California, Los Angeles Loneliness Scale. In addition, each of the selected studies’ reference lists was manually searched to identify further studies. An example of the search strategy conducted on PubMed is reported in Online Resource 1.

Study Selection

Two authors (SA and RH) independently carried out systematic electronic searches in each of the four databases. The same authors independently performed the screening of the titles and abstracts of the identified articles. Studies deemed to be relevant to the review were retrieved and assessed for eligibility. Disagreements between the two authors were resolved by discussion until consensus was reached.

Data Extraction

Two authors (SA and AO) performed the data extraction. The extracted data from individual studies included the author, year, country or society, UCLA-LS characteristics, sample size, age, sex/gender, recall period, original language and the available translations, and the psychometric properties of the UCLA-LS (validity, reliability, responsiveness). Furthermore, data on the translation and cross-cultural adaptation process were also extracted to examine each design.

Assessing Methodological Quality

The methodological quality of each selected study was independently evaluated by two authors (SA and AO) using the COSMIN Risk of Bias (RoB) checklist (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). This checklist distinguishes three main domains: reliability, validity, and responsiveness and seven sub-domains: content validity, structural validity, hypothesis testing for construct validity, cross-cultural validity, criterion validity, internal consistency, reliability, measurement error, and responsiveness.

The quality of the individual studies was rated on a 4-point scale as very good, adequate, doubtful, or inadequate, and the overall rating of the quality of each study was based on the lowest rating of any standard (the worst score count principle) (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). The result of each psychometric property was assessed against the updated criteria for good psychometric properties and was rated as sufficient (+), insufficient (−), or indeterminate (?). A detailed description of the quality criteria and the rating system is published (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018).

The results of the psychometric properties were qualitatively summarized and compared against the criteria for good psychometric properties to rate the overall summary result of the scale as sufficient (+), insufficient (−), indeterminate (?), or inconsistent (±). In case of inconsistent results, the overall rating was based on only high-quality studies and the quality of evidence was downgraded for inconsistency (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018).

Assessing Cross-Cultural Adaptation

The translation methodology of the included studies was categorized according to the Guidelines for the Process of Cross-Cultural Adaptation of Self-report Measures (Beaton et al., 2000). These guidelines indicate that an optimal (accurate) adaptation must include five processes: an initial translation, synthesis of translations, back translations, reviews by the expert committee and finally, the pre-test of the instrument. A detailed description of the cross-cultural adaptation process is published (Beaton et al., 2000).

Hypothesis Testing for Construct Validity

To interpreting the results of hypothesis testing and assessing the quality of evidence for construct validity, the review team formulated a set of hypotheses about the expected relationships (direction and magnitude) of the UCLA-LS and other instruments and variables. These hypotheses were based on the literature and the clinical experiences of the review team. For the convergent and divergent validity, the review team expected strong correlations (r ≥ 0.50) between the UCLA-LS and other loneliness scales (similar construct), moderate correlations (0.30 ≤ r < 0.50) between the UCLA-LS and related constructs such as positive and negative affect, weak or no correlations (< 0.30) for unrelated constructs such as demographics. For discriminative validity, the review team expected non-significant differences between groups except for marital status. These hypotheses are summarized in Online Resource 2. Lastly, the term criterion validity was used only when a short version of the UCLA-LS was compared to the 20-item version, as recommended by COSMIN (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018).

Assessing Overall Quality of Evidence

The overall quality of the evidence was graded as high, moderate, low, or very low, using the modified Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach, which considers methodological quality of studies, inconsistency, imprecision, and indirectness to provide a rating of evidence quality. A detailed description of the evidence quality rating is published (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018).

The quality of evidence was downgraded by one, two or three levels based on the risk of bias, imprecision, and results inconsistency as shown in Online Resource 3 (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018).

Results

Study Selection

The database and reference list search yielded 575. After removing duplicates, 464 citations were screened, leaving 256 studies for full-text screen for eligibility. Of these, 81 studies were considered eligible (Adams et al., 2004; Allen & Oshagan, 1995; Anderson & Malikiosi-Loizos, 1992; Anderssen et al., 2020; Arimoto & Tadaka, 2019; Auné et al., 2019; Austin, 1983; Balandin et al., 2006; Boffo et al., 2012; Britton & Conner, 2007; Caballer et al., 2020; Chalise et al., 2007, 2010; Chang et al., 2017; Chou, 2005; Çinar & Toker, 2019; Cole et al., 2015; Doğan et al., 2011; Durak & Senol-Durak, 2010; Dussault et al., 2009; Elemo et al., 2020; Elphinstone, 2018; Ergün et al., 2020; Faustino et al., 2019; Gan et al., 2015; Gillespie et al., 2021; Grey et al., 2020; Hackett et al., 2012, 2019; Hagger & Riley, 2019; Hansen et al., 2012; Hartshorne, 1993; Hawkley et al., 2020; Hays & DiMatteo, 1987; Heisel et al., 2016; Hojat, 1982; Hughes et al., 2004; Igarashi, 2019; Igbokwe et al., 2020; Jakobsen et al., 2020; O. Kim, 1999a, 1999b; Y. Kim et al., 2020; Knight et al., 1988; Kong et al., 2015; Kurina et al., 2011; Lamm & Stephan, 1987; Lee & Cagle, 2017; Lukács et al., 2019; Matthews-Ewald & Zullig, 2013; McWhirter, 1990; Nazzal et al., 2018; Neto, 2014; Niu et al., 2018; Oshagan & Allen, 1992; Park et al., 2019; Penning et al., 2014; Pretorius, 1993; Robinson-Whelen et al., 2016; Russell, 1996; Russell et al., 1980; Dan Russell et al., 1978; Saito et al., 2019; Sancho et al., 2020; Shi et al., 2016; Shorey et al., 2019; Solano, 1980; Stancliffe et al., 2014; Stephan et al., 1988; Steptoe et al., 2004; Sun & Chang, 2021; Velotti et al., 2021; Wang et al., 2018; Wang et al., 2001; Wilson et al., 1992; Wittenborn et al., 2020; Wongpakaran et al., 2020; Wu & Yao, 2008; Yu et al., 2020; Zakahi & Duran, 1982; Zarei et al., 2016). The PRISMA flow diagram of the study selection process with reasons for excluding studies is shown in Fig. 1.

Fig. 1
figure 1

PRISMA flow diagram showing study selection process

Full size image

Characteristics of Included Studies

The 81 eligible studies were conducted between 1978 and 2021 and included 104,387 participants aged 18–97 years. The study sample size ranged from 20 to 50,054 participants. A summary description of the included studies is presented in Table 1 per version of the UCLA-LS, and Online Resource 4 per study.

Table 1 Characteristics of UCLA-LS per Version
Full size table

Description of UCLA Loneliness Scale

Three versions of the 20-item (original, revised, and version 3) and nine short versions, UCLA-16, UCLA-11, UCLA-10, UCLA-8, UCLA-7, UCLA-6, UCLA-5, UCLA-4, and UCLA-3 of the UCLA-LS were identified in the 81 included studies. The characteristics and assessment of the UCLA-10 and UCLA-6 were divided into sub-versions because each study included a different set of items. A detailed description of each scale is presented in Table 1 and Online Resource 4.

Methodological Quality and Psychometric Properties of the Studies

A qualitative synthesis was used to report the results on the psychometric properties of the included studies. Results of the included studies were summarized based on the assessed psychometric measures (validity and reliability) and the study quality ratings. The UCLA-LS development process was rated as inadequate due to the lack of information on the comprehensiveness and comprehensibility of the instrument, as well as the lack of content validity studies.

Validity of Long and Short Versions of UCLA-LS

Results on validity assessment are presented in Online Resource 5, and the overall quality assessment is summarized in Table 2. The structural validity was assessed for all UCLA-LS versions except the original UCLA-20 (Dan Russell et al., 1978; Solano, 1980) and UCLA-5 (Lamm & Stephan, 1987). Based on the confirmatory factor (CFA) and Rasch analyses, high-quality evidence of sufficient structural validity was available for the UCLA-11(Lee & Cagle, 2017) and UCLA-10 (all sub-versions) (Elphinstone, 2018; Knight et al., 1988; Russell, 1996) on a multidimensional scale (two-factor model) and the UCLA-7 (Allen & Oshagan, 1995; Elphinstone, 2018; Oshagan & Allen, 1992), UCLA-6 (Nazzal et al., 2018; Neto, 2014; Wongpakaran et al., 2020; Wu & Yao, 2008), and UCLA-4 (Elphinstone, 2018) on a unidimensional scale (one-factor model). Low to moderate-quality evidence of sufficient structural validity was available for the Revised UCLA-20 (Çinar & Toker, 2019; Dussault et al., 2009; Hartshorne, 1993; Park et al., 2019; Penning et al., 2014) and version 3 UCLA-20 (Boffo et al., 2012; Durak & Senol-Durak, 2010; Elphinstone, 2018; Russell, 1996; Sancho et al., 2020; Zarei et al., 2016) on a multidimensional scale (three-factor model) and for the UCLA-8 (Doğan et al., 2011; Elphinstone, 2018; Wu & Yao, 2008) on a unidimensional scale (one-factor model). Low to moderate-quality evidence of insufficient structural validity was available for the UCLA-6 (Niu et al., 2018), UCLA-3 (Chalise et al., 2007, 2010; Jakobsen et al., 2020; Matthews-Ewald & Zullig, 2013), and UCLA-16 (Faustino et al., 2019) as indicated by the Rasch and exploratory factor analyses (EFA) (Online Resource 5).

Table 2 Validity results summary and evidence quality of UCLA-LS versions
Full size table

Construct validity was assessed for all UCLA-LS versions except UCLA-11. Moderate quality evidence of sufficient convergent validity was available for the original UCLA-20 (Dan Russell et al., 1978; Solano, 1980), UCLA-16 (Faustino et al., 2019), and UCLA-6 (Nazzal et al., 2018; Neto, 2014). Moderate quality evidence of sufficient discriminative validity was available for the original UCLA-20 (Dan Russell et al., 1978; Solano, 1980), UCLA-16 (Faustino et al., 2019), UCLA-10 (Knight et al., 1988; D. W. Russell, 1996), UCLA-8 (Hays & DiMatteo, 1987; Wilson et al., 1992), UCLA-4 (Hays & DiMatteo, 1987; Wilson et al., 1992), and UCLA-3 (Chalise et al., 2007; Hawkley et al., 2020; Igarashi, 2019; Matthews-Ewald & Zullig, 2013; Robinson-Whelen et al., 2016). Low quality evidence of sufficient criterion validity was available for the revised UCLA-20 (Anderson & Malikiosi-Loizos, 1992), UCLA-8 (Doğan et al., 2011; Hays & DiMatteo, 1987; Wilson et al., 1992), UCLA-6 (Nazzal et al., 2018; Neto, 2014), and UCLA-3 (Arimoto & Tadaka, 2019; Hughes et al., 2004; Igarashi, 2019), and lastly, high quality evidence of sufficient cross-cultural validity was available for the UCLA-6 (Wongpakaran et al., 2020) (Table 2, Online Resource 5).

Reliability of Long and Short Versions of UCLA-LS

The internal consistency (Cronbach’s Alpha) was assessed for all UCLA-LS versions. Results revealed sufficient internal consistency of all versions except for the original UCLA-20 (Ergün et al., 2020; Gillespie et al., 2021; Dan Russell et al., 1978; Solano, 1980) and UCLA-5 (Lamm & Stephan, 1987) due to lack of studies on structural validity. The quality of evidence ranged from high (UCLA-7 (Elphinstone, 2018; Oshagan & Allen, 1992) and UCLA-6 (Elemo et al., 2020; Nazzal et al., 2018; Neto, 2014)) to very low. Test re-test reliability was assessed for the revised UCLA-20 (Hartshorne, 1993; Hojat, 1982; Stephan et al., 1988), version 3 UCLA-20 (Balandin et al., 2006; Britton & Conner, 2007; Russell, 1996; Zarei et al., 2016), and UCLA-3 (Cole et al., 2015). Moderate quality evidence of sufficient reliability was available for version 3 UCLA-20 (Balandin et al., 2006; Britton & Conner, 2007; Russell, 1996; Zarei et al., 2016) with excellent Interclass Correlation Coefficient (ICC) values of 0.76–0.93. A summary of results for internal consistency and test re-test reliability is presented in Online Resource 6, and the overall quality evidence is summarized in Table 3.

Table 3 Reliability results summary and evidence quality of UCLA-LS versions
Full size table

Cross-Cultural Adaption of UCLA-LS

Table 4 displays the ratings of the translation and cross-cultural adaptations of 12 studies across ten countries and societies including Iran, Greece, Korea, Taiwan, Nepal, Turkey, Italy, Japan, Spain, and Thailand. None of these studies followed 100% of the recommended cross-cultural adaptation guidelines when performing the initial translation, synthesis, back-translation, and expert committee review. However, two studies (Sancho et al., 2020; Zarei et al., 2016) followed the guidelines for the initial translation of version 3 UCLA-LS, and one study (Zarei et al., 2016) included an expert committee to review version 3 UCLA-LS (Table 4).

Table 4 Cross-cultural adaptations of UCLA-LS into different languages
Full size table

Discussion

Selecting the most appropriate instrument to assess loneliness and to evaluate the outcome of approaches to its management is essential in clinical research and practice. Information on the methodological quality of such instruments helps inform which outcome measures have the most robust psychometric properties. In this systematic review, we evaluated the psychometric properties and cross-cultural adaptation of the 20-item and all short forms of the UCLA-LS. We synthesized the current evidence from 81 studies and identified three versions of the 20-item and nine short versions of the UCLA-LS. Validity and reliability studies were supportive of the use of the UCLA-LS for assessing loneliness in adults.

Content validity is especially important as it reflects how items are representative of the construct to be measured (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). However, the lack of studies on the scale developmental process and content validity prevented an assessment of content validity. The internal structure of the outcome measure was determined from its structural validity, internal consistency, and cross-cultural validity. To assess the structural validity of a scale, CFA and EFA are the commonly used statistical methods. However, CFA is preferred over the EFA because it tests whether measures of a construct are consistent with a priori hypotheses and confirm the factor structure of a set of observed variables (model fit). Besides, the internal consistency assesses the degree of the interrelatedness among the items on a scale while the cross-cultural validity examines whether respondents from different groups respond similarly to a particular item (measurement invariance) (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). Based on these parameters, high to moderate-quality evidence supported sufficient internal structure of the short versions UCLA-10, UCLA-8, UCLA-7, UCLA-6, and UCLA-4, while low-quality evidence supported sufficient internal structure of the long versions (Revised UCLA-20 and version 3 UCLA-20) of the UCLA-LS, to encourage the use of these versions in loneliness-related research and clinical practice.

Good test re-test reliability reflects the internal validity of the outcome measure and ensures that the repeated measurements are stable over time (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). There were limited and indeterminate reports on test re-test reliability because of unclear test conditions, lack of ICC values, and inadequate measurement intervals between the two administrations of the scale. However, only version 3 UCLA-20 demonstrated moderate-quality evidence of sufficient test re-test reliability with excellent ICC values.

Sufficient construct validity is necessary to ensure that the intended theoretical construct and results are consistent with the hypotheses (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). Although we noted low- to moderate-quality evidence of sufficient construct validity for some of the scales, the quality of evidence was reduced due to the risk of bias and inconsistency of study results.

Many of the UCLA-LS versions such as the revised UCLA-20, version 3 UCLA-20, UCLA-8, and UCLA-3 were translated into multiple languages (e.g., Chinese, Turkish, Persian, Spanish, and Thai) and culturally adapted across several countries and societies (e.g., Greece, Italy, Japan, Taiwan, Korea, and Nepal). Additionally, the psychometric properties of the translated UCLA-LS versions were evaluated and reported by several studies. However, there was insufficient information about their cross-cultural adaptation because many of these studies were not available in English (Çinar & Toker, 2019; Lamm & Stephan, 1987; Nazzal et al., 2018; Stephan et al., 1988).

The UCLA-LS has been validated for use in individuals with different clinical conditions. For instance, the validity and reliability of version 3 UCLA-20 and UCLA-3 were assessed in patients with cerebral palsy (Balandin et al., 2006), opioid dependence (Britton & Conner, 2007), physical and mental disabilities (Stancliffe et al., 2014), stroke and brain tumors (Hagger & Riley, 2019). It has also been assessed among those with obesity, hypertension and sleep apnea (Chalise et al., 2007; Kurina et al., 2011). Besides, the internal consistency of the original UCLA-20, revised UCLA-20, UCLA-10 and UCLA-8 were assessed in patients with COVID-19 (Gillespie et al., 2021), diabetes (Hackett et al., 2019), hypertension and cardiovascular diseases (Gan et al., 2015), and AIDS (Hansen et al., 2012). However, the insufficient information in these studies precluded a robust conclusion about the suitability of these versions for use in the corresponding clinical populations. Studies among those with chronic medical conditions such as chronic lung disease are lacking despite the high prevalence of loneliness among such patients (Reijnders et al., 2018). 

Despite the large number of studies on physical, mental, and social interventions on loneliness (Ma et al., 2020), information on responsiveness has been consistently lacking. Such information is valuable when tracking change with time (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018).

Review Limitations

A limitation of this review is the exclusion of 17 non-English articles, which might have yielded important additional information regarding scale development, content validity, cross-adaptation, and translation. Direct comparisons were limited by the many sub-versions of the UCLA-10 and UCLA-6, thereby preventing clear conclusions regarding their psychometric properties. We were aware of the small risk of publication bias by excluding conference posters and abstracts. However, the results of this systematic review were qualitatively summarized. As a result, no meta-analysis was conducted, and the possibility of publication bias was not assessed.

Conclusion

This is the first systematic review that provides a detailed assessment of the psychometric properties of the 20-item UCLA-LS and its subsequent abbreviated versions, spanning multiple languages, cultures, and social circumstances. High-quality evidence supports sufficient internal structure of the UCLAs: 10, 7, 6, and 4, suggesting that these short versions would best serve those interested in loneliness-related research and clinical practice. Future studies are required to assess the responsiveness, cross-cultural and criterion validity of the UCLA-LS among a broad variety of clinical and social populations. This information will further increase our understanding of the psychometric properties of this outcome measure.