Abstract
This paper aims to provide a comprehensive overview of piloting approaches, systemize piloting processes, and underline good practices in large scale longitudinal birth cohort studies. We identified 54 eligible studies, and by applying predefined inclusion criteria, we included 10 studies in the systematic review. The main findings are that the longitudinal birth cohort surveys most often include all three types of pilots in every sweep—pre-test, instruments pilot and dress rehearsal. Sampling procedures and sample sizes in pilots vary a lot depending on the type and aims of a pilot, and from sweep to sweep. The reporting on piloting and its key findings is regularly present in longitudinal birth cohort studies, especially for instruments pilots and dress rehearsals, while it is less common for pre-tests. Based on the findings, the recommendations for other researchers are to adopt the same terms for different types of pilots—pre-pilots, instruments pilots and dress rehearsals, to extend and uniform reports on piloting, to report information about pilots’ response and attrition rates, and to dedicate a separate chapter to piloting in survey general technical report.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Altman, D. G., Bland, J. M.: Absence of evidence is not evidence of absence. BMJ 311–485 (1995)
Arain, M., Campbell, M.J., Cooper, C.L., Lancaster, G.A.: What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Med Res Methodol 10(67), 1–4 (2010)
Baker, T.L.: Doing social research, 2nd edn. McGraw-Hill Inc, New York (1994)
Calitz, M. G.: Pilot study (chapter 5). https://uir.unisa.ac.za/bitstream/handle/10500/1648/06chapter5.pdf (2009). Accessed 26 November 2019
Charlesworth, G., Burnell, K., Hoe, J., Orrell, M., Russell, I.: Acceptance checklist for clinical effectiveness pilot trials: a systematic approach. BMC Med Res Methodol 13(78), 1–7 (2013)
De Vaus, D.A.: Surveys in Social Research, 3rd edn. UCL Press, London (1993)
Deng, Y., Hillygus, S.D., Reiter, J.P., Si, Y., Zheng, S.: Handling attrition in longitudinal studies: the case for refreshment samples. Stat Sci 28(2), 238–256 (2013)
Ekinsmyth, C.: The British longitudinal birth cohort studies: Their utility for the study of health and place. Health Place 2(1), 15–26 (1996)
Hertzog, M.A.: Considerations in determining sample size for pilot studies. Res Nursing Health 31(2), 180–191 (2008)
Hinds, P.S., Gattuso, J.: From pilot work to a major study in cancer nursing. Res Cancer Nursing 14(3), 132–135 (1991)
Lancaster, G.A., Dodd, S., Williamson, P.R.: Design and analysis of pilot studies: Recommendations for good practice. J Evaluat Clin Pract 10(2), 307–312 (2004)
Lee, E.C., Whitehead, A.L., Jacques, R.M., Julious, S.A.: The statistical interpretation of pilot trials: should significance thresholds be reconsidered? BMC Med Res Methodol 14(41), 1–8 (2014)
Leon, A.C., Davis, L.L., Kraemer, H.C.: The role and interpretation of pilot studies in clinical research. J Psychiatric Res 45(5), 626–629 (2011)
Lindquist, R.A.: Don’t forget the pilot work! Heart Lung 20(1), 91–92 (1991)
Martin, J., Bynner, J., Kalton, G., Boyle, P., Goldstein, H., Gayle, V., Parsons, S., Piesse, A.: Strategic review of panel and cohort studies. Longview, London (2006)
Moher, D., Schulz, K.F., Altman, D.: The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials. J Am Med Assoc 285(15), 1987–1991 (2001)
Moore, C.G., Carter, R.E., Nietert, P.J., Stewart, P.W.: Recommendations for planning pilot studies in clinical and translational sciences. Clin Translational Sci 4(5), 332–337 (2011)
Morrison, J., Clement, T., Nestel, D., Brown, J.: “Underdiscussed, underused and Underreported”: Pilot work in team-based qualitative research. Qual Res J 16(4), 314–330 (2016)
Muoio, R., Wolcott, L., Seigel, H.: A win-win situation: The pilot program. J Continuing Edu Nurs 26(5), 230–233 (1995)
Murray, A., McCrory, C., Williams, J.: Report on the Pre-pilot, Pilot and Dress Rehearsal Exercises for Wave 2 of the Infant Cohort at Age Three Years (Growing Up in Ireland). Department of Children and Youth Affairs, Dublin. https://www.growingup.ie/pubs/BKMNEXT312.pdf (2014). Accessed 10 September 2019
Peat, J., Mellis, C., Williams, K., Xuan, W.: Health Science Research: A Handbook of Quantitative Methods. Sage, London (2002)
Polit, D.F., Beck, C.T.: Nursing research: Principles and methods, 7th edn. Lippincott, Philadelphia (2004)
Polit, D.F., Beck, C.T., Hungler, B.P.: Essentials of Nursing Research: Methods, Appraisal and Utilization, 5th edn. Lippincott, Philadelphia (2001)
Porta, M.: A Dictionary of epidemiology. Oxford University Press, Oxford (2008)
Prescott, P.A., Soeken, K.L.: The potential uses of pilot work. Nurs Res 38(1), 60–62 (1989)
Tashakkori, A., Teddlie, C.: Mixed Methodology: Combining Qualitative & Quantitative Approaches. Sage, CA, USA (1998)
Thabane, L., Ma, J., Chu, R., Cheng, J., Ismaila, A., Rios, L.P., Robson, R., Thabane, M., Giangregorio, L., Goldsmith, C.H.: A tutorial on pilot studies: The what, why and how. BMC Med Res Methodol 10(1), 1–10 (2010)
van Teijlingen, E.R., Hundley, V.: The importance of pilot studies. Soc Res Update 35, 1–4 (2001)
van Teijlingen, E.R., Rennie, A.-M., Hundley, V., Graham, W.: The importance of conducting and reporting pilot studies: the example of the Scottish Births Survey. J Adv Nurs 34, 289–295 (2001)
Funding
This research was done within European Cohort Development Project (ECDP) supported by the European Union's Horizon 2020 research and innovation programme under grant agreement No 777449.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no conflict of interest.
Availability of data and material (data transparency)
Not applicable.
Code availability (software application or custom code)
Not applicable.
Appendix
Rights and permissions
About this article
Cite this article
Blažev, M., Babarović, T. & Serracant, P. Characteristics of piloting longitudinal birth cohort surveys: a systematic review. Qual Quant 55, 1047–1069 (2021). https://doi.org/10.1007/s11135-020-01042-1
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11135-020-01042-1