Background

The COVID-19 pandemic has emerged as the most pressing public health and economic challenge of our time. As of June 2022, there have been more than 6 million recorded COVID-19-related deaths, with a global economic recession that has surpassed any economic downturn since World War 2. A public health crisis of this magnitude is bound to impact the psychological well-being and health-related quality of life (HRQL) of the masses. Previous research has shown that pandemics result in higher levels of psychological distress, including higher rates of suicide attempts and suicides [1,2,3]. Many individuals who were previously not considered vulnerable or predisposed to mental illness may experience increased stress levels due to loss of employment, heightened caregiver responsibilities, illness or death of a loved one due to COVID-19, constant media messaging, or distrust of governing bodies [4,5,6]. In isolation or combined, these factors may result in the onset of new mental health disorders or worsening of pre-existing ones. There is an urgent need to understand the impact of the pandemic and related societal changes on the psychological well-being and overall HRQL of the general population.

Patient-reported outcome measures (PROMs) are questionnaires that assess health status from the patient's perspective [7]. Within healthcare delivery and research, PROMs have been used to understand the outcomes and cost-effectiveness of treatment interventions to improve how healthcare is planned, organized, and delivered. In public health, validated PROMs can be used to assess the psychological well-being and overall health and well-being of the general population during the COVID-19 pandemic. The PROM data when combined with social determinants of health information can be used to target resources and interventions to population subgroups that are most vulnerable to psychological distress. Previous studies have used PROMs to assess the health and well-being of the general population during the early pandemic in countries including China [8,9,10], the United Kingdom [11, 12], Italy [13], Spain [14], and Brazil [15].

The primary objective of this study was to assess the psychological well-being of the US general population during the COVID-19 pandemic using validated PROMs in the early pandemic (i.e., May 2020). A secondary objective was to examine the relationship between key sociodemographic and clinical variables and psychological well-being.

Methods

Ethics

The study was approved by the research ethics board of Mass General Brigham, Boston, Massachusetts (IRB Protocol#: 2020P001440).

Study design and participants

For this cross-sectional study, participants were recruited through Prolific (Prolific Academic Ltd, Oxford; https://www.prolific.co). Prolific is an online crowdsourcing platform that was established for subject recruitment for research studies. It has a user-friendly interface and includes a minimum payment per unit of time required to complete the study-related task. The participant pool in the Prolific has been shown to be more honest, internationally diverse and less exposed to common research tasks compared to other platforms (e.g., MTurk) [16,17,18]. A non-representative sample of adult (18 years or older) members of the general public residing in the US at the time of survey administration (i.e., May 2020), who were able to read and write English and did not have cognitive limitations that impacted online survey participation, were included. Eligible participants received an invitation to participate with a brief description of the study objectives and procedures via Prolific's internal email system. Interested participants were asked to click on a link that directed them to a detailed study information sheet. Participants could choose to continue with the survey or ignore the email. Consent was implied if the participant decided to complete the study.

Participants completed an online Research Electronic Data Capture (REDCap) survey hosted at BWH. The survey included questions about participants' sociodemographic characteristics (e.g., age, gender, employment (pre-and post-pandemic)), health status (e.g., smoking status, other pre-existing health conditions), and questions related to their attitudes towards and symptoms of COVID-19. A set of questions (n = 8) derived from a literature review of measures of financial toxicity (e.g., COST [19]) and social determinants of health (SDOH; e.g., Mass General Brigham SDOH questionnaire [20]) that ask about the financial status on a 4-point Likert scale (strongly agree, agree, disagree, strongly disagree) were also included. Participants were asked if they avoided visiting a healthcare service during the pandemic or had a scheduled surgery or a cancelled medical procedure. Female participants were also asked if they were pregnant. Finally, all participants completed three short PROMs, namely the PROMIS-10 Global Health, Generalized Anxiety Disorder Scale-7 (GAD-7), and Patient Health Questionnaire-9 (PHQ-9). The survey took approximately ten minutes to complete, and the participants received monetary compensation (pre-set through Prolific) for their time.

PROMs administered

PROMIS-10 Global Health

The adult PROMIS-10 Global Health (v1.2) short form is a 10-item questionnaire that measures general health and functioning (i.e., overall physical health, mental health, social health, pain, fatigue, and overall perceived quality of life). Evidence suggests that PROMIS-10 Global Health is reliable, valid, and responsive [21,22,23,24]. The questionnaire is designed to be applicable across various health conditions. The items in the adult PROMS-10 Global Health are scored on a five-point Likert scale [25, 26] to produce a Global Physical Health and Global Mental Health T-score. High scores reflect more of the concept being measured, i.e., a higher Mental Health score corresponds to better mental health. PROMIS scores have a mean of 50 and a standard deviation of 10 in the US general population. More recently, the cut points or thresholds for the PROMIS-10 Physical Health score categories of excellent, very good, good, fair, and poor have been established [27].

Generalized Anxiety Disorder Scale-7 (GAD-7)

The GAD-7 is a 7-item self-report scale developed as a screening tool and severity indicator for generalized anxiety disorder [28]. The items on GAD-7 ask respondents how bothered they are by several anxiety-related problems, over the last two weeks, on a four-point Likert scale (not at all, several days, over half the days, nearly every day). Individual item scores are summed to provide a total score that ranges from 0 to 21, with higher scores indicating more severe generalized anxiety disorder symptoms. Scores of 5, 10, and 15 represent cutpoints for mild, moderate, and severe anxiety, respectively. The psychometric properties of reliability, criterion validity, and construct validity of the GAD-7 have been evaluated in the general population [29], primary care [30], and psychiatric samples [31, 32]. Previous literature suggests that approximately 5% of the general population have GAD-7 scores of 10 or greater, and approximately 1% have GAD-7 scores of 15 or greater [28].

Patient Health Questionnaire-9 (PHQ-9)

The PHQ-9 is a nine-item questionnaire designed as a screening tool for depression in primary care and other healthcare settings. Each question asks the frequency that a patient has experienced a particular depressive symptom in the past two weeks on a four-point Likert scale (not at all, several days, more than half the days, nearly every day). The responses on the items are totalled and range from 0 to 27, with higher scores indicating more severe depression. Scores of 5, 10, 15, and 20 represent cutpoints for mild, moderate, moderately severe and severe depression, respectively. The PHQ-9 has been widely validated and has shown good psychometric properties against other established and commonly used PROMs for depression assessment, such as the Beck Depression Inventory (BDI) and the Hamilton Depression Rating Scale (HDRS) [33]. The PHQ-9 is shorter (9 questions) as compared to BDI and HDRS which have 21 and 17 questions respectively and is recommended by the American Academy of Family Physicians as a tool for depression screening. Previous research has established that women tend to score 3.1 points higher than men and that 18.1% of the population will have mild depression (PHQ score, 5–9), and 8.1% will have moderate to severe depression (PHQ score, 10–27) [30, 34].

Statistical analysis

All statistical analysis was performed using IBM SPSS Statistics for Mac Version 26.0 (IBM Corp. Released 2019. Armonk, NY: IBM Corp) and R software, version 4.0.2 [35]. A sample size of 2000 participants was set a priori and was based on the budgetary constraints of the study. No sample size calculation was performed. However, due to a glitch in the system, 2023 participants responded to the survey. Data from all 2023 participants were included in the analysis. Means and standard deviations were used to summarize the PROM data by the demographic and clinical characteristics of the participants. Categorical variables were summarized using frequencies and percentages, and contingency tables were used to assess the distribution of the PROM scores by demographic and clinical variables. A one-way analysis of variance was conducted to compare the 8 financial questions with the 4 PROM scores.

A set of key demographic and clinical factors were selected based on clinical reasoning and the literature [36,37,38,39,40,41,42,43,44,45,46,47,48,49] and their relationship was examined with the 4 PROM scores (i.e., PROMIS-10 Global Mental Health T-score, PROMIS-10 Global Physical Health T-score, GAD-7 score, and PHQ-9 score) with a multivariable linear regression analysis. The variables included in the regression were age, gender, ethnicity, BMI, education attainment, marital status, living arrangement, income and insurance status, prior psychiatric illness, and the belief that COVID-19 is deadlier than flu. For the regression analysis, the BMI variable was recoded as normal or overweight (BMI < 25–29.99) and obese (BMI ≥ 30), ethnicity was recoded as White and Others, marital was recoded as married or living common-law and Others, the living arrangement was recoded as live alone or live with someone, and the highest level of education was recoded as low (i.e., some or completed high school or some college, trade or university diploma) and high (completed college, trade or university diploma and post-graduate degree). The income variable was recoded to ≤ $49,999 and ≥ $50,000 and the insurance variable was recoded to uninsured and insured. The statistical significance was set at p value < 0.05.

Results

The mean age of the 2023 participants was 31.92 years (SD, 11.57; range 18 to 82 years), and the mean body mass index (BMI) was 25.58 kg/m2 (SD, 6.13, range 10.38 to 55.22). Majority of the participants were White (n = 1,309, 65%), young adults (n = 1,589, 79%), single or never been married (n = 1,188, 59%), had completed college, trade or university (n = 823, 41%), and reported no change in employment due to COVID-19 (n = 1,382 m 68%). Further, most of the participants reported no pre-existing mental (n = 1,428, 71%), cardiorespiratory (n = 1,847, 91%), cancer (n = 1,992, 99%), diabetes (n = 1,968, 97%), blood-related (n = 1,841, 91%), autoimmune (1,911, 95%) illness. A total of 13 (< 1%) participants reported that they had tested positive for the COVID-19 and 17 (< 1%) participants reported living with someone who had tested positive for the COVID-19. Most participants believed COVID-19 virus was more deadly than flu (n = 1,574, 78%) and said that they were planning to receive the vaccine when one becomes available (n = 1,340, 66%).

The mean scores for PROMIS-10 Global Physical and Mental Health, GAD-7 and PHQ-9 by the demographic and clinical characteristics of the participants are provided in Table 1. The mean PROMIS-10 Global Physical Health T-scores for most of the patient characteristics were found to be close to the reference value for the US general population of 50; however, the mean Mental Health T-scores were much lower (i.e., worse mental health compared to the general population) across several patient characteristics. This deviation from the reference value of 50 for the Mental Health T-score was much more pronounced for participants who chose “other” as gender, were Indigenous, lived alone, had some high school education, had a change in the type of employment, reported less than USD 15,000 annual income in the previous year, were uninsured, had a pre-existing chronic or mental health condition, and had their scheduled surgery cancelled due to the pandemic. For the GAD-7 and PHQ-9, participants who identified as the “other” gender or Indigenous, had lost their job or were unable to work, were uninsured, and had their surgery cancelled reported higher levels of anxiety and depression, respectively.

Table 1 Mean and standard deviation scores for PROMIS-10 physical health, mental health, GAD-7 and PHQ-9 by participant demographic and clinical characteristics
Full size table

The mean scores and distribution by the 3 PROMs based on the responses to the question on self-reported financial status are shown in Table 2. Most participants in the sample reported being able to meet their monthly expenses (86%, n = 1,736); however, a substantial number of participants had to cut down on expenses (64%, n = 1,296), were not happy with their current financial situation (54%, n = 1,089), and reported worrying about their future financial status (72%, n = 1,457). Similar to the demographic and clinical variables, the mean value for the PROMIS-10 Global Physical Health was close to the reference value of 50, whereas larger variability was noted for the PROMIS-10 Global Mental health scores for the financial questions. A one-way analysis of variance analysis revealed that there was a statistically significant difference in the mean PROM scores and all 8 questions (p < 0.001). Post-hoc analysis using Tukey’s HSD test for multiple comparisons found that the mean value of the 4 PROM scores was significantly different except between response levels disagree and strongly disagree for questions asking about the ability to meet monthly expenses, money saved for essentials, feeling financially stressed, and feeling worried about future financial status; between strongly agree and agree for questions asking about the current financial situation, having to cut down on expenses and the need to borrow money; and between strongly agree, agree and disagree for the question asking about borrowing money from a financial institution.

Table 2 Mean and standard deviation scores for PROMIS-10 physical health, mental health, GAD-7 and PHQ-9 by answers to questions about financial status
Full size table

Table 3 show the results of the regression analysis. We found that younger adults scored worse on the PROMIS-10 Mental health score, GAD-7 and PHQ-9, and the difference between younger and older adults was significant. The younger adults had higher scores on the Global 10 Physical health score, indicating better physical well-being. Individuals who identified as females and chose others as their gender response had worse psychological being compared to men. Women on average scored 3 and 3.5 points higher on the GAD-7 and PHQ-9 respectively. Similarly, individuals who had BMI ≥ 30 had worse outcomes on all 4 PROM scores. Participants from non-White ethnic backgrounds scored worse compared to White participants on PROMIS-10 Physical and Mental scores but not with GAD-7 or PHQ-9. Lower income (i.e., ≤ $49,999), being uninsured, and having a history of prior psychiatric illness were associated with statistically significant worse psychological well-being as compared to income ≥ $50,000, being insured and having no history of prior psychiatric illness. Participants who were separated, divorced, or widowed scored had worse Global 10 Mental Health and PHQ-9 scores compared to individuals who were married or common-law, whereas living arrangement was not associated with any of the 4 PROM scores. The belief that COVID-19 was deadlier than flu was associated with worse outcomes on all 4 PROMs.

Table 3 Regression analyses results
Full size table

Discussion

In this cross-sectional survey of the US general population, we found that valid, reliable, and generic PROMs can be used to assess psychological well-being and health-related quality of life during the COVID-19 pandemic. Individuals who were female or other gender, younger, non-White, obese, not married or living common-law, uninsured, diagnosed with psychiatric illness, and earned less than $49,999 annual income in the prior year were at a higher risk of poor psychological well-being.

Recent literature on the psychological well-being of populations during the pandemic corroborates our findings [38,39,40, 42, 50, 51]. Females have been found to have worse psychological well-being compared to males, possibly due to a couple of reasons. First, females are more likely to work in industries, such as retail, that were more negatively affected by the pandemic. Further, women may be working "second shifts" as primary caregivers to children, elderly or infected family members. We also found that young to middle-aged adults experienced higher psychological distress compared to their older counterparts, which may be associated in part with job loss and financial uncertainty. Further, this group is more likely to consist of digitally literate individuals and hence, are subject to distress related to media messaging in the context of COVID-19 [5, 6, 52, 53]. Lastly, our findings align with previous research that suggests poor economic status, lower education level, and being uninsured may cause individuals to develop new mental illness(es), especially depression, during the pandemic [5, 54,55,56].

We found that there was a statistically significant difference in the mean scores of the 4 PROMs and the 8 questions asking about financial status. A post hoc analysis for multiple comparisons revealed that the levels at which this difference occurred were not consistent. This is an interesting but also a non-informative finding of this study. The financial status questions were included to get a sense of the participant’s financial status and the source of distress related to aspects of financial status (e.g., being able to meet monthly expenses, borrowing from financial institutions); and were derived from pre-existing measures of financial toxicity and social determinants of health. The response levels were determined by the study team. As the intention was not to develop a new scale to assess financial status during the pandemic, an a priori evaluation of the questions and the construct measured was not completed (e.g., exploratory factor analysis). A plausible cause for the inconsistency noted in post hoc group comparisons is that the response levels for the financial questions were unable to adequately discriminate between the participants. Subsequently, we chose to use the most used indicator of annual income in our regression model and not the responses to these individual questions or a make-do summed score. However, these results underline the need for assessing the financial status during the pandemic (especially, in relation to psychological well-being) using validated measures.

A post hoc descriptive comparison of published reference values for the PROMs from pre-pandemic literature with our study showed a higher prevalence of anxiety and depression in the general population during the pandemic than the pre-pandemic levels. The T-scores for the PROMIS-10 Global Physical and Mental Health scales were found to be lower than the reference values during the pandemic for all population subgroups by gender and age; however, for physical health, the T-scores were slightly better than the reference values for females and individuals between the ages of 45–64 years (Table 4, Fig. 1). For the GAD-7, 25.2% (n = 510) of the participants reported scores of 10 or higher, and 9.3% (n = 189) reported scores of 15 or higher. On PHQ-9, women scored 1.73 points higher than men, with 29.2% (n = 590) reporting mild depression and 31.1% (n = 630) reporting moderate to severe depression. However, these results are merely conjecture due to small sample sizes in some of the categories of reference groups and the unrepresentativeness of our sample. Future studies should explore the shift in reference values during the pandemic and how long the response shifts last post-pandemic.

Table 4 Normative values for the US general population for the PROMIS Global Health measure
Full size table
Fig. 1
figure 1

PROMIS global physical and mental health T-scores—reference scores versus scores during the pandemic

Full size image

The study results have important implications. First, this study demonstrates that PROMs can be used to screen and monitor psychological well-being during a pandemic. When linked with other relevant public health data, such as sociodemographic information, healthcare usage, and morbidity outcomes by zip code, PROMs can play a critical role in achieving effective and efficient healthcare delivery by targeting the health resource allocation to the most vulnerable subgroups of the population. Additionally, PROMs can be used to assess the quality of life impact of "long COVID-19 [57, 58]” during hospitalization to enhance the management of patients, support discharge pathways during recovery and survivorship, and determine unmet needs [59, 60]. Finally, this study highlights the importance of asking about stressors (e.g., change in employment status, front-line worker) in the context of COVID-19 when conducting assessments of psychological well-being.

Our study has some important limitations. An online crowdsourcing platform was used for recruitment purposes, which may have resulted in the exclusion of population subgroups that are digitally illiterate, have no or inadequate access to technology and the internet, and non-English speaking. Further, we did not pre-set recruitment quotas for gender, ethnicity or age groups, resulting in a non-representative sample of the US population. To put this in perspective, according to the recent US Census data (https://www.census.gov/quickfacts/fact/table/US/PST045221), 16.5% of the adults are 65 years and older, 32.9% have college or higher level of education, 50.8% identify as females and 76.3% are White. The study sample included 1.5% of adults that were 65 years and older, 61.6% had college or higher level of education, 52.3% were females and 64.7% were White. The under-representation of these subgroups is an important limitation since the negative psychological well-being and quality of life impact of the pandemic may be exacerbated in these groups due to pre-existing vulnerabilities. Lastly, we did not ask participants who chose “other” as their gender to specify their gender identity. However, considering that the data included only 21 participants, we do not believe it impacted the results or their interpretation substantially.

To conclude, while physical distancing measures and stay-at-home orders represent essential public health strategies for curbing the spread of the COVID-19 pandemic, they may be a severe threat to the psychological well-being of the general population. Using PROMs to assess psychological well-being and quality of life to monitor, plan and deliver healthcare resources should be an essential part of the COVID-19 response.