Abstract
This chapter approaches surveys as powerful research tool, with reference to a large study into the issue of doctoral researchers’ mental health. The authors Dr. Cassie Hazell and Dr. Clio Berry guide the readers through the different stages of the study, starting with the process of identifying the constructs you are interested in and the questionnaires you will use to measure these. This chapter expands on how surveys can be used to collect quantitative, qualitative and mixed methods data—including the use of control groups, outcome measure forms as well as closed and open questions. This chapter explores recruitment strategy, reflecting on different strategies that can be used to improve participants recruitment and retention. It additionally describes key considerations for making your online survey ready for distribution, and for producing and enacting an appropriate analysis plan that addresses your research questions whilst reflecting on the different stories that you can tell with your data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Allen, M. S., Iliescu, D., & Greiff, S. (2022). Single item measures in psychological science. European Journal of Psychological Assessment, 38(1), 1–5. https://doi.org/10.1027/1015-5759/A000699
Antoun, C., Zhang, C., Conrad, F. G., & Schober, M. F. (2016). Comparisons of online recruitment strategies for convenience samples: Craigslist, Google AdWords, Facebook, and Amazon Mechanical Turk. Field Methods, 28(3), 231–246. https://doi.org/10.1177/1525822X15603149
Ball, H. L. (2019). Conducting online surveys. Journal of Human Lactation, 35(3), 413–417. https://doi.org/10.1177/0890334419848734
Beck, A. T., & Bredemeier, K. (2016). A unified model of depression: Integrating clinical. Cognitive, Biological, and Evolutionary Perspectives, 4(4), 596–619. https://doi.org/10.1177/2167702616628523
Berry, C., Niven, J. E., Chapman, L., Valeix, S., Roberts, P., & Hazell, C. M. (2021a). A mixed mixed-methods investigation of mental health stigma, absenteeism and presenteeism among UK postgraduate researchers. Studies in Graduate and Postdoctoral Education, 12(1), 145–170.
Berry, C., Niven, J. E., & Hazell, C. M. (2021b). Personal, social and relational predictors of UK postgraduate researcher mental health problems. BJPsych Open, 7(6), e205. https://doi.org/10.1192/BJO.2021.1041
Blair, E., & Zinkhan, G. M. (2006). Nonresponse and generalizability in academic research. Journal of the Academy of Marketing Science, 34(1), 4–7. https://doi.org/10.1177/0092070305283778
Blumenberg, C., Menezes, A. M. B., Gonçalves, H., Assunção, M. C. F., Wehrmeister, F. C., & Barros, A. J. D. (2019). How different online recruitment methods impact on recruitment rates for the web-based coortesnaweb project: A randomised trial. BMC Medical Research Methodology, 19(1), 1–9. https://doi.org/10.1186/s12874-019-0767-z
Braun, V., & Clarke, V. (2013). Successful qualitative research: A practical guide for beginners. Sage. http://eprints.uwe.ac.uk/21156/3/SQR
Brick, J. M., & Kalton, G. (1996). Handling missing data in survey research. Statistical Methods in Medical Research, 5(3), 215–238. https://doi.org/10.1177/096228029600500302
Burghardt, J., & Bodansky, A. N. (2021). Why psychology needs to stop striving for novelty and how to move towards theory-driven research. Frontiers in Psychology, 12, 67. https://doi.org/10.3389/FPSYG.2021.609802/BIBTEX
Callegaro, M., Jon Cohen, G., Elizabeth Dean, S., Harwood, P., & Josh Pasek, T. (2014). Mobile technologies for conducting, augmenting and potentially replacing surveys: Report of the AAPOR task force on emerging technologies in public opinion research. www.aapor.org
Christensen, T., Riis, A. H., Hatch, E. E., Wise, L. A., Nielsen, M. G., Rothman, K. J., Sørensen, H. T., & Mikkelsen, E. M. (2017). Costs and efficiency of online and offline recruitment methods: A web-based cohort study. Journal of Medical Internet Research, 19(3), e58. https://doi.org/10.2196/jmir.6716
Connelly, L. M. (2022). Understanding research. Measurement instrument validity. MEDSURG Nursing, 31(1), 64–65. https://web.p.ebscohost.com/abstract?direct=true&profile=ehost&scope=site&authtype=crawler&jrnl=10920811&AN=155300022&h=evWO%2FnjPmz%2Bv8Exx5QpuPWg8W3nx6fdF%2BL%2FVS0%2FxZDpvUXrqNb8PYBANvyp5G1W5HjjEuCEdy4CnpH7LFm%2Fx5A%3D%3D&crl=c&resultNs=AdminWebAuth&resultLocal=ErrCrlNotAuth&crlhashurl=login.aspx%3Fdirect%3Dtrue%26profile%3Dehost%26scope%3Dsite%26authtype%3Dcrawler%26jrnl%3D10920811%26AN%3D155300022
Cornette, M. M., Deroon-Cassini, T. A., Fosco, G. M., Holloway, R. L., Clark, D. C., & Joiner, T. E. (2009). Application of an interpersonal-psychological model of suicidal behavior to physicians and medical trainees. Archives of Suicide Research, 13(1), 1–14. https://doi.org/10.1080/13811110802571801
Danermark, B., & Ekström, M. (2002). Explaining society: Critical realism in the social sciences. Routledge.
Danermark, B., Ekström, M., & Karlsson, J. C. (2002). Explaining society: Critical realism in the social sciences. Routledge.
Deutskens, E., de Ruyter, K., Wetzels, M., & Oosterveld, P. (2004). Response rate and response quality of internet-based surveys: An experimental study. Marketing Letters, 15(1), 21–36.
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method. Wiley.
Doyle, L., Brady, A. M., & Byrne, G. (2009). An overview of mixed methods research. Journal of Research in Nursing, 14(2), 175–185. https://doi.org/10.1177/1744987108093962
Draper, H., Wilson, S., Flanagan, S., & Ives, J. (2009). Offering payments, reimbursement and incentives to patients and family doctors to encourage participation in research. Family Practice, 26(3), 231–238. https://doi.org/10.1093/fampra/cmp011
Draucker, C. B., Martsolf, D. S., & Poole, C. (2009). Developing distress protocols for research on sensitive topics. Archives of Psychiatric Nursing, 23(5), 343–350. https://doi.org/10.1016/j.apnu.2008.10.008
Duffy, M. E. (1987). Methodological triangulation: A vehicle for merging quantitative and qualitative research methods. Image: The Journal of Nursing Scholarship, 19(3), 130–133. https://doi.org/10.1111/J.1547-5069.1987.TB00609.X
Evans, J. R., & Mathur, A. (2018). The value of online surveys: A look back and a look ahead. Internet Research, 28(4), 854–887. https://doi.org/10.1108/IntR-03-2018-0089
Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving integration in mixed methods designs - principles and practices. Health Services Research, 48(6 PART2), 2134–2156. https://doi.org/10.1111/1475-6773.12117
Field, A. (2013). Discovering statistics using IBM SPSS. SAGE.
Fletcher, A. J. (2017). Applying critical realism in qualitative research: Methodology meets method. International Journal of Social Research Methodology, 20(2), 181–194. https://doi.org/10.1080/13645579.2016.1144401
Fowler, D., Hodgekins, J., Berry, C., Clarke, T., Palmier-Claus, J., Sacadura, C., Graham, A., Lowen, C., Steele, A., Pugh, K., Fraser, S., & French, P. (2019). Social recovery therapy: A treatment manual. Psychosis, 11, 261–272.
Galesic, M. (2006). Dropouts on the web: Influence of changes in respondents’ interest and perceived burden during the web survey. Journal of Official Statistics, 22(2), 313–328.
Graham, J. W., Cumsille, P. E., & Shevock, A. E. (2013). Methods for handling missing data. In J. A. Schinka, W. F. Velicer, & I. B. Weiner (Eds.), Handbook of psychology: Research methods in psychology. Wiley.
Grand, A., Davies, G., Holliman, R., & Adams, A. (2015). Mapping public engagement with research in a UK university. PLoS One, 10(4), 1–19. https://doi.org/10.1371/journal.pone.0121874
Grant, R. W., & Sugarman, J. (2004). Ethics in human subjects research: Do incentives matter? The Journal of Medicine and Philosophy: A Forum for Bioethics and Philosophy of Medicine, 29(6), 717–738. https://doi.org/10.1080/03605310490883046
Greenlaw, C., & Brown-Welty, S. (2009). A comparison of web-based and paper-based survey methods: Testing assumptions of survey mode and response cost. Evaluation Review, 33(5), 464–480. https://doi.org/10.1177/0193841X09340214
Grimes, D. A., & Schulz, K. F. (2002). Bias and causal associations in observational research. The Lancet, 359(9302), 248–252. https://doi.org/10.1016/S0140-6736(02)07451-2
Hawkley, L. C., & Cacioppo, J. T. (2010). Loneliness matters: A theoretical and empirical review of consequences and mechanisms. Annals of Behavioral Medicine: A Publication of the Society of Behavioral Medicine, 40(2), 218–227. https://doi.org/10.1007/S12160-010-9210-8
Hazell, C. M., Chapman, L., Valeix, S. F., Roberts, P., Niven, J. E., & Berry, C. (2020). Understanding the mental health of doctoral researchers: A mixed methods systematic review with meta-analysis and meta-synthesis. Systematic Reviews, 9(1), 197. https://doi.org/10.1186/s13643-020-01443-1
Hazell, C. M., Niven, J. E., Chapman, L., Roberts, P. E., Cartwright-Hatton, S., Valeix, S., & Berry, C. (2021). Nationwide assessment of the mental health of UK doctoral researchers. Humanities and Social Sciences Communications, 8(1), 8–16. https://doi.org/10.1057/s41599-021-00983-8
Hlatshwako, T. G., Shah, S. J., Kosana, P., Adebayo, E., Hendriks, J., Larsson, E. C., Hensel, D. J., Erausquin, J. T., Marks, M., Michielsen, K., Saltis, H., Francis, J. M., Wouters, E., & Tucker, J. D. (2021). Online health survey research during COVID-19. The Lancet Digital Health, 3(2), e76–e77. https://doi.org/10.1016/S2589-7500(21)00002-9/ATTACHMENT/DACDFE51-523B-4EFB-AC39-E798FA533B66/MMC1.PDF
Hoerger, M. (2010). Participant dropout as a function of survey length in internet-mediated university studies: Implications for study design and voluntary participation in psychological research. Cyberpsychology, Behavior and Social Networking, 13(6), 697–701.
Hood, K., Robling, M., Ingledew, D., Gillespie, D., Greene, G., Ivins, R., Russell, I., Sayers, A., Shaw, C., & Williams, J. (2012). Mode of data elicitation, acquisition and response to surveys: A systematic review. Health Technology Assessment, 16(27), 1–161. https://doi.org/10.3310/hta16270
Jager, J., Putnick, D. L., & Bornstein, M. H. (2017). More than just convenient: The scientific merits of homogeneous convenience samples. Monographs of the Society for Research in Child Development, 82(2), 13–30. https://doi.org/10.1111/mono.12296
Jenkins, P. E., Ducker, I., Gooding, R., James, M., & Rutter-Eley, E. (2021). Anxiety and depression in a sample of UK college students: A study of prevalence, comorbidity, and quality of life. Journal of American College Health, 69(8), 813–819. https://doi.org/10.1080/07448481.2019.1709474
Jetten, J., Haslam, C., & Haslam, A. S. (Eds.). (2012). The social cure: Identity, health and well-being. Psychology Press. https://books.google.co.uk/books?hl=en&lr=&id=EeB4AgAAQBAJ&oi=fnd&pg=PR3&dq=The+social+cure:+Identity,+health+and+well-being.&ots=OZ8Ny-jXCU&sig=w8YpkrvAj79W3nW_Yfgomh2onc0#v=onepage&q=The
Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). The Guildford Press. https://doi.org/10.1080/10705511.2012.687667
Knapp, P., Mitchell, N., Raynor, D. K., Silcock, J., Parkinson, B., Holt, J., Birks, Y., & Gilbody, S. (2011). Can we improve recruitment to trials and informed consent by improving participant information sheets? - a nested RCT. Trials, 12(S1), 1–2. https://doi.org/10.1186/1745-6215-12-s1-a123
Kroenke, K., Spitzer, R. L., Williams, J. B. W., & Löwe, B. (2010). The patient health questionnaire somatic, anxiety, and depressive symptom scales: A systematic review. General Hospital Psychiatry, 32(4), 345–359. https://doi.org/10.1016/j.genhosppsych.2010.03.006
Leonard, N. R., Lester, P., Rotheram-Borus, M. J., Mattes, K., Gwadz, M., & Ferns, B. (2003). Successful recruitment and retention of participants in longitudinal behavioral research. AIDS Education and Prevention, 15(3), 269–281. https://doi.org/10.1521/aeap.15.4.269.23827
Lindsay, J. (2005). Getting the numbers: The unacknowledged work in recruiting for survey research. Field Methods, 17(1), 119–128. https://doi.org/10.1177/1525822X04271028
McEvoy, P., & Richards, D. (2003). Critical realism: A way forward for evaluation research in nursing? Journal of Advanced Nursing, 43(4), 411–420. https://doi.org/10.1046/j.1365-2648.2003.02730.x
MHFA England. (2020). Mental health statistics. MHFA England.
Miller, C. A., Guidry, J. P. D., Dahman, B., & Thomson, M. D. (2020). A tale of two diverse qualtrics samples: Information for online survey researchers. Cancer Epidemiology Biomarkers and Prevention, 29(4), 731–735. https://doi.org/10.1158/1055-9965.EPI-19-0846/70558/AM/A-TALE-OF-TWO-DIVERSE-QUALTRICS-SAMPLES
Mirzaei, A., Carter, S. R., Patanwala, A. E., & Schneider, C. R. (2022). Missing data in surveys: Key concepts, approaches, and applications. Research in Social and Administrative Pharmacy, 18(2), 2308–2316. https://doi.org/10.1016/j.sapharm.2021.03.009
Nolte, M. T., Shauver, M. J., & Chung, K. C. (2015). Analysis of four recruitment methods for obtaining normative data through a web-based questionnaire: A pilot study. The Hand, 10(3), 529–534. https://doi.org/10.1007/s11552-014-9730-y
O’Brien, K. K., Solomon, P., Worthington, C., Ibáñez-Carrasco, F., Baxter, L., Nixon, S. A., Baltzer-Turje, R., Robinson, G., & Zack, E. (2014). Considerations for conducting web-based survey research with people living with human immunodeficiency virus using a community-based participatory approach. Journal of Medical Internet Research, 16(3), e3064. https://doi.org/10.2196/JMIR.3064
O’Cathain, A., Murphy, E., & Nicholl, J. (2010). Three techniques for integrating data in mixed methods studies. BMJ (Online), 341(7783), 1147–1150. https://doi.org/10.1136/bmj.c4587
Oberauer, K., & Lewandowsky, S. (2019). Addressing the theory crisis in psychology. Psychonomic Bulletin and Review, 26(5), 1596–1618. https://doi.org/10.3758/S13423-019-01645-2/FIGURES/6
Olsen, W. K., Haralambos, M., & Holborn, M. (2004). Triangulation in social research: Qualitative and quantitative methods can really be mixed. In W. K. Olsen, M. Haralambos, & M. Holborn (Eds.), Developments in sociology (pp. 1–30). Causeway Press.
Papageorgiou, G., Grant, S. W., Takkenberg, J. J. M., & Mokhles, M. M. (2018). Statistical primer: How to deal with missing data in scientific research? Interactive Cardiovascular and Thoracic Surgery, 27(2), 153–158. https://doi.org/10.1093/icvts/ivy102
Peugh, J. L., & Enders, C. K. (2004). Missing data in educational research: A review of reporting practices and suggestions for improvement. Review of Educational Research, 74(4), 525–556. https://doi.org/10.3102/00346543074004525
Pozzar, R., Hammer, M. J., Underhill-Blazey, M., Wright, A. A., Tulsky, J. A., Hong, F., Gundersen, D. A., & Berry, D. L. (2020). Threats of bots and other bad actors to data quality following research participant recruitment through social media: Cross-sectional questionnaire. Journal of Medical Internet Research, 22(10). https://doi.org/10.2196/23021
Roberts, C. (2016). Response styles in surveys: Understanding their causes and mitigating their impact on data quality. In C. Wolf (Ed.), The SAGE handbook of survey methodology (pp. 579–596). Sage. https://doi.org/10.4135/9781473957893.N36
Roberts, C., Gilbert, E., Allum, N., & Eisner, L. (2019). Research Synthesis Satisficing in surveys: A systematic review of the literature. Public Opinion Quarterly, 83(3), 598–626. https://doi.org/10.1093/POQ/NFZ035
Smith, G. (2008). Does gender influence online survey participation? A record-linkage analysis of university faculty online survey response behavior. In ERIC Document Reproduction Service No. ED 501717 (Vol. 501717).
Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097. https://doi.org/10.1001/archinte.166.10.1092
Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53. https://doi.org/10.5116/IJME.4DFB.8DFD
The British Psychological Society. (2021). Code of ethics and conduct. The British Psychological Society, 21, 134. https://doi.org/10.1016/s0315-5463(88)70754-3
Tishler, C. L., & Bartholomae, S. (2002). The recruitment of normal healthy volunteers: A review of the literature on the use of financial incentives. Journal of Clinical Pharmacology, 42(4), 365–375. https://doi.org/10.1177/00912700222011409
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge University Press. https://books.google.co.uk/books?hl=en&lr=&id=bjVYdyXXT3oC&oi=fnd&pg=PR11&dq=Tourangeau+R,+Rips+LJ,+Rasinski+K.+The+psychology+of+survey+response.+Cambridge:+Cambridge+University+Press%3B+2000.&ots=Z-YhOL8CBV&sig=zAD7y0LC8ZXK6vNvv13Vr8BlpVE#v=onepage&q=To
Tremblay, M. C., Dutta, K., & Vandermeer, D. (2010). Using data mining techniques to discover bias patterns in missing data. Journal of Data and Information Quality, 2(1), 1. https://doi.org/10.1145/1805286.1805288
UCAS. (2021). Starting the conversation: UCAS report on student mental health. UCAS.
van Teijlingen, E., & Hundley, V. (2001). The importance of pilot studies. Social Research Update, 35. https://aura.abdn.ac.uk/handle/2164/157
Watts, G. (2020). COVID-19 and the digital divide in the UK. The Lancet Digital Health, 2(8), e395–e396. https://doi.org/10.1016/S2589-7500(20)30169-2
World Health Organisation (WHO). (2021). Suicide. WHO.
Wright, K. B. (2005). Researching internet-based populations: Advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services. Journal of Computer-Mediated Communication, 10(3). https://doi.org/10.1111/J.1083-6101.2005.TB00259.X/4614509
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Hazell, C., Berry, C. (2022). Dilemmas and Decisions in Quantitative-Driven Online Survey Research into Researchers’ Mental Health and Support. In: Bager-Charleson, S., McBeath, A. (eds) Supporting Research in Counselling and Psychotherapy . Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-031-13942-0_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-13942-0_7
Published:
Publisher Name: Palgrave Macmillan, Cham
Print ISBN: 978-3-031-13941-3
Online ISBN: 978-3-031-13942-0
eBook Packages: Behavioral Science and PsychologyBehavioral Science and Psychology (R0)