Abstract
Purpose
There is a lack of well-validated self-report measures to assess public speaking anxiety. This study explored the psychometric properties of the Public Speaking Anxiety Scale (PSAS).
Methods
Seventy-two adults completed the PSAS as part of the baseline screening procedure of a randomized controlled trial. Rasch analysis was used to assess the scale’s response category functioning, precision, targeting, unidimensionality, and differential item functioning. Construct validity was assessed using classical test theory methods.
Results
While thresholds were ordered and no systematic bias in responses for age, gender, or screen failure was found, the PSAS demonstrated evidence of multidimensionality (variance by first factor = 39.7%, eigenvalue of first contrast = 2.76). Multidimensionality was resolved after splitting the scale into two discrete subscales: Emotional and Physiological. Three misfitting items (i.e. item 5 from Emotional, items 6 and 14 from Physiological) were removed. Scale precision and targeting remained suboptimal after subscale split and removal of misfitting items (PSI = 1.41, PR = 0.67 for Emotional; PSI = 1.49, PR = 0.69 for Physiological).
Conclusion
The PSAS demonstrated adequate convergent validity. Psychometric properties of the PSAS after Rasch-guided modifications were overall promising. Further studies are needed to confirm our results.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Research data are not shared.
Abbreviations
- SAD:
-
Social anxiety disorder
- DSM-5:
-
Diagnostic and statistical manual of mental disorders 5th ed.
- LSAS:
-
Liebowitz social anxiety scale
- PSAS:
-
Public speaking anxiety scale
- CTT:
-
Classical test theory
- IRT:
-
Item response theory
- BDI-II:
-
Beck depression inventory 2nd Ed.
- PSI:
-
Person separation index
- PR:
-
Person reliability
- DIF:
-
Differential item functioning
References
Anderson, P.L., Zimand, E., Hodges, L.F., Rothbaum, B.O.: Cognitive behavioral therapy for public-speaking anxiety using virtual reality for exposure. Depress. Anxiety 22(3), 156–158 (2005). https://doi.org/10.1002/da.20090
Bartholomay, E.M., Houlihan, D.D.: Public speaking anxiety scale: preliminary psychometric data and scale validation. Personality Individ. Differ. 94, 211–215 (2016). https://doi.org/10.1016/j.paid.2016.01.026
Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Manual for the Beck Depression Inventory-II. Pearson.
Blöte, A.W., Kint, M.J.W., Miers, A.C., Westenberg, P.M.: The relation between public speaking anxiety and social anxiety: A review. J. Anxiety Disord. 23(3), 305–313 (2009). https://doi.org/10.1016/j.janxdis.2008.11.007
Bögels, S.M., Alden, L.E., Beidel, D.C., Clark, L., Pine, D., Stein, M.B., Voncken, M.J.: Social anxiety disorder: Questions and answers for the DSM-V. Depress. Anxiety 27(2), 168–189 (2010). https://doi.org/10.1002/da.20670
Bond, T. G. & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Lawrence Erlbaum Associates Publishers.
Bruce, S.E., Yonkers, K.A., Otto, M.W., Eisen, J.L., Weisberg, R.B., Pagano, M., Keller, M.B.: Influence of psychiatric comorbidity on recovery and recurrence in generalized anxiety disorder, social phobia, and panic disorder: A 12-year prospective study. Am. J. Psychiatry 162(6), 1179–1187 (2005). https://doi.org/10.1176/appi.ajp.162.6.1179
Dinno, A. (2018). paran: Horn's Test of Principal Components/Factors (Version 1.5.2.). Retrieved from https://CRAN.R-project.org/package=paran
Dueñas, L.J., Restrepo-Castro, J.C., Becerra Garcia, A.M.: Reliability and factorial analysis of the public speaking anxiety scale in Spanish (PSAS-S). Psychol. Neurosci. 11(1), 50–57 (2018). https://doi.org/10.1037/pne0000126
Fehm, L., Beesdo, K., Jacobi, F., Fiedler, A.: Social anxiety disorder above and below the diagnostic threshold: Prevalence, comorbidity and impairment in the general population. Soc. Psychiatry Psychiatr. Epidemiol. 43(4), 257–265 (2008). https://doi.org/10.1007/s00127-007-0299-4
Fenwick, E.K., Ong, P.G., Sabanayagam, C., Rees, G., Xie, J., Holloway, E., Cheng, C.Y., Wong, T.Y., Lim, B., Tan, P.C., Lamoureux, E.L.: Assessment of the psychometric properties of the Chinese Impact of Vision Impairment questionnaire in a population-based study: findings from the Singapore Chinese Eye Study. Qual. Life Res. 25(4), 871–880 (2016). https://doi.org/10.1007/s11136-015-1141-1
Fresco, D.M., Coles, M.E., Heimberg, R.G., Liebowitz, M.R., Hami, S., Stein, M.B., Goetz, D.: The Liebowitz Social Anxiety Scale: a comparison of the psychometric properties of self-report and clinician-administered formats. Psychol. Med. 31(6), 1025–1035 (2001). https://doi.org/10.1017/S0033291701004056
Furmark, T., Tillfors, M., Stattin, H., Ekselius, L., Fredrikson, M.: Social phobia subtypes in the general population revealed by cluster analysis. Psychol. Med. 30(6), 1335–1344 (2000). https://doi.org/10.1017/S0033291799002615
Glorfeld, L.W.: An improvement on Horn’s parallel analysis methodology for selecting the correct number of factors to retain. Educ. Psychol. Measur. 55(3), 377–393 (1995). https://doi.org/10.1177/0013164495055003002
Heeren, A., Jones, P.J., McNally, R.J.: Mapping network connectivity among symptoms of social anxiety and comorbid depression in people with social anxiety disorder. J. Affect. Disord. 228, 75–82 (2018). https://doi.org/10.1016/j.jad.2017.12.003
Heinrichs, N., Rapee, R.M., Alden, L.A., Bögels, S., Hofmann, S.G., Jak, Oh., Sakano, Y.: Cultural differences in perceived social norms and social anxiety. Behav. Res. Ther. 44(8), 1187–1197 (2006). https://doi.org/10.1016/j.brat.2005.09.006
Hofmann, S.G., Anu Asnaani, M.A., Hinton, D.E.: Cultural aspects in social anxiety and social anxiety disorder. Depress. Anxiety 27(12), 1117–1127 (2010). https://doi.org/10.1002/da.20759
Horn, J.L.: A rationale and test for the number of factors in factor analysis. Psychometrika 30(2), 179–185 (1965). https://doi.org/10.1007/BF02289447
Kessler, R.C., Stang, P., Wittchen, H.U., Stein, M., Walters, E.E.: Lifetime co-morbidities between social phobia and mood disorders in the US national comorbidity survey. Psychol. Med. 29(3), 555–567 (1999). https://doi.org/10.1017/S0033291799008375
Kessler, R.C., Chiu, W.T., Demler, O., Walters, E.E.: Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the national comorbidity survey replication. Arch. Gen. Psychiatry 62(6), 617–627 (2005). https://doi.org/10.1001/archpsyc.62.6.617
Kopalle, P.K., Lehmann, D.R.: Alpha inflation? the impact of eliminating scale items on cronbach’s alpha. Organ. Behav. Hum. Decis. Process. 70(3), 189–197 (1997). https://doi.org/10.1006/obhd.1997.2702
Lamoureux, E., Pesudovs, K.: Vision-specific quality-of-life research: a need to improve the quality. Am. J. Ophthalmol. 151(2), 195–197 (2011). https://doi.org/10.1016/j.ajo.2010.09.020
Lang, P.J.: The application of psychophysiological methods to the study of psychotherapy ad behavior modification. In: Bergin, A.E., Garfield, S.L. (eds.) Handbook of psychotherapy and behavior change, pp. 75–125. Wiley (1971)
Li, Z.: A power formula for the Mantel-Haenszel test for differential item functioning. Appl. Psychol. Meas. 39(5), 373–388 (2015). https://doi.org/10.1177/0146621614568805
Liebowitz, M.R.: Social phobia. Mod. Probl. Pharmacopsychiatry 22, 141–173 (1987). https://doi.org/10.1159/000414022
Lin, X.B., Lee, T.S., Cheung, Y.B., Ling, J., Poon, S.H., Lim, L., Zhang, H.H., Chin, Z.Y., Wang, C.C., Krishnan, R., Guan, C.: Exposure therapy with personalized real-time arousal detection and feedback to alleviate social anxiety symptoms in an analogue adult sample: pilot proof-of-concept randomized controlled trial. JMIR Mental Health 6(6), e13869 (2019). https://doi.org/10.2196/13869
Linacre, J.M.: What do infit and outfit, mean-square and standardized mean. Rasch Measure. Trans. 16(2), 878 (2002)
Linacre, J. M. (2005). A user’s guide to Winsteps ministeps rasch-model computer programs program manual. Winsteps.com.
Lindner, P., Miloff, A., Fagernäs, S., Andersen, J., Sigeman, M., Andersson, G., Carlbring, P.: Therapist-led and self-led one-session virtual reality exposure therapy for public speaking anxiety with consumer hardware and software: A randomized controlled trial. J. Anxiety Disord. 61, 45–54 (2019). https://doi.org/10.1016/j.janxdis.2018.07.003
Manson, S.M.: Culture and DSM-IV: Implications for the diagnosis of mood and anxiety disorders. In: Mezzich, J.E., Kleinman, A., Fabrega, H., Parron, D.L. (eds.) Culture and psychiatric diagnosis: A DSM-IV perspective, pp. 99–113. American Psychiatric Association (1996)
Mennin, D.S., Fresco, D.M., Heimberg, R.G., Schneier, F.R., Davies, S.O., Liebowitz, M.R.: Screening for social anxiety disorder in the clinical setting: using the Liebowitz social anxiety scale. J. Anxiety Disord. 16(6), 661–673 (2002). https://doi.org/10.1016/S0887-6185(02)00134-2
Pesudovs, K.: Item banking: A generational change in patient-reported outcome measurement. Optom. vis. Sci. 87(4), 285–293 (2010). https://doi.org/10.1097/OPX.0b013e3181d408d7
Pesudovs, K., Burr, J.M., Harley, C., Elliott, D.B.: The development, assessment, and selection of questionnaires. Optom. vis. Sci. 84(8), 663–674 (2007). https://doi.org/10.1097/OPX.0b013e318141fe75
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Association.
Rytwinski, N.K., Fresco, D.M., Heimberg, R.G., Coles, M.E., Liebowitz, M.R., Cissell, S., Hofmann, S.G.: Screening for social anxiety disorder with the self-report version of the liebowitz social anxiety scale. Depress. Anxiety 26(1), 34–38 (2009). https://doi.org/10.1002/da.20503
Stein, D. J., Lim, C. C. W., Roest, A. M., de Jonge, P., Aguilar-Gaxiola, S., Al-Hamzawi, A., WHO World Mental Health Survey Collaborators. (2017). The cross-national epidemiology of social anxiety disorder: Data from the world mental health survey initiative. BMC Medicine, 15(1), 1-21. doi: https://doi.org/10.1186/s12916-017-0889-2
Thissen, D., Nelson, L., Rosa, K., & McLeod, L. D. (2001). Item response theory for items scored in more than two categories. In Test scoring (pp. 141–186). Lawrence Erlbaum Associates Publishers.
Väänänen, J., Fröjd, S., Ranta, K., Marttunen, M., Helminen, M., Kaltiala-Heino, R.: Relationship between social phobia and depression differs between boys and girls in mid-adolescence. J. Affect. Disord. 133(1), 97–104 (2011). https://doi.org/10.1016/j.jad.2011.03.036
World Medical Association. (2013). World medical association Declaration of Helsinki: Ethical principles for medical research involving human subjects. JAMA, 310(20), 2191-2194. doi: https://doi.org/10.1001/jama.2013.281053
Acknowledgements
This research was part of a pilot, proof-of-concept, randomized controlled trial that was supported by the National Medical Research Council [NMRC/CTGIITE/0002/2015] granted to Tih-Shih Lee. The authors would like to thank all participants who contributed their data.
Funding
This research was part of a pilot, proof-of-concept, randomized controlled trial that was supported by the National Medical Research Council [NMRC/CTGIITE/0002/2015] granted to Tih-Shih Lee.
Author information
Authors and Affiliations
Contributions
XB was involved in conceptualization, data curation, formal analyses, methodology, and manuscript writing. REK contributed to formal analyses, methodology, and manuscript writing. TS acquired funding and resources. E advised on methodology. TS and E reviewed and supervised manuscript writing. XB, TS, and SH contributed to the investigation process. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
There is no conflict of interest to declare.
Ethical approval
This study was approved by the Institutional Review Board of the National University of Singapore (Reference Code: B-14–098).
Consent to participate
Written informed consent was obtained prior to study participation.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lin, X.B., Lee, TS., Man, R.E.K. et al. Rasch analysis reveals multidimensionality in the public speaking anxiety scale. Health Serv Outcomes Res Method 22, 332–348 (2022). https://doi.org/10.1007/s10742-021-00265-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10742-021-00265-5