Skip to main content
SpringerLink
Log in

Empirical research consolidation: a generic overview and a classification scheme for methods

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

I define research consolidation as comparing and combining (amalgamating or, at least, juxtaposing) results or other output from multiple previously conducted research activities, relevant to a goal or topic of interest. The concept, not limited to empirical research or any specific method, is similar to and subsumes what is often referred to as research synthesis or research integration; I explain the concept and my rationale for the new term. After introducing the broader concept, I focus on consolidation of empirical research. As background for this, I offer a brief introduction to empirical research. Then I provide a generic overview of empirical research consolidation, abstracted from several of its specific methods, such as quantitative meta-analysis and qualitative meta-synthesis; this sort of overview—method agnostic and capturing the commonality—is not readily available in the literature. At the core of the paper, I propose a scheme to classify methods for empirical research consolidation followed by a review and classification of selected methods, illustrating the scheme. I also discuss and clarify related terminology. The classification scheme differentiates between two major attributes of methods—consolidation technique and type of content to be consolidated—that may reflect the positivist or constructivist research paradigm. Consolidation techniques could be aggregative, interpretive, or a combination thereof. Content consolidated, from empirical research reports, could be empirical or not and could be from quantitative research, qualitative research, other kinds of empirical research, or a combination thereof.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alford, M.L., Mendes, E.: Scholarly research process: investigating the effects of link type and directionality. In: Proceedings of the 20th ACM Conference on Hypertext and Hypermedia, HT’09, June 29–July 1, 2009, Torino, Italy, pp. 99–108 (2009)

  • Bartneck, C.: Using the metaphysics of quality to define design science. In: Proceedings of the 4th International Conference on Design Science Research in Information Systems and Technology (DESRIST 2009), May 7–8, 2009, Malvern, PA, USA (2009)

  • Baskerville, R., Pries-Heje, J., Venable, J.: Soft design science methodology. Design science research in information systems and technologies. In: Proceedings of the 4th International Conference on Design Science Research in Information Systems and Technology [DESRIST ‘09], Malvern, Pennsylvania, USA, May 7–8, 2009. SESSION: Design research: methods. Article No. 9 (2009)

  • Bassey M.: Creating education through research. Brit. Educ. Res. J. 18(1), 3–16 (1992)

    Article  Google Scholar 

  • Bayazit N.: Investigating design: a review of forty years of design research. Des. Issues 20(1), 16–29 (2004)

    Article  Google Scholar 

  • Biolchini, J., Mian, P.G., Natali, A.C.C., Travassos, G.H.: Systematic review in software engineering. Technical Report ES 679/05, Systems Engineering and Computer Science Department, COPPE/UFRJ. Rio de Janeiro, Brazil, May 2005. http://www.cin.ufpe.br/~in1037/leitura/systematicReviewSE-COPPE.pdf. (2005). Accessed September 2010

  • Blagosklonny, M.V., Pardee, A.B.: Conceptual biology: unearthing the gems. Nature 416, 373 (28 March 2002) http://www.nature.com/nature/journal/v416/n6879/full/416373a.html (2002). Accessed September 2010

  • Bondas T., Hall E.O.C.: Challenges in approaching metasynthesis research. Qual. Health Res. 17(1), 113–121 (2007)

    Article  Google Scholar 

  • Bryman A.: The debate about quantitative and qualitative research: a question of method or epistemology?. Brit. J. Sociol. 35(1), 75–92 (1984)

    Article  Google Scholar 

  • Burger, M.J.C.: Towards a framework for the elicitation of dilemmas. Qual. Quant. 42, 541–562 (2008). Published online: 8 August 2007 (2008)

    Google Scholar 

  • Bushman B.J., Cooper H.M.: Effects of alcohol on human aggression: an integrative research review. Psychol. Bull. 107(3), 341–354 (1990)

    Article  Google Scholar 

  • Chalmers I., Haynes B.: Systematic reviews: reporting, updating, and correcting systematic reviews of the effects of health care. BMJ 309, 862–865 (1994)

    Article  Google Scholar 

  • Chen C., Rada R.: Interacting with hypertext: a meta-analysis of experimental studies. Hum.-Comput. Interact. 11(2), 125–156 (1996)

    Article  Google Scholar 

  • Creswell J.W.: Research Design. Qualitative & Quantitative Approaches. Sage Publications, Thousand Oaks, CA (1994)

    Google Scholar 

  • Creswell J.W., Garrett A.L.: The ‘movement’ of mixed methods research and the role of educators. South Afr. J. Educ. 28, 321–333 (2008)

    Google Scholar 

  • Cross N.: Designerly ways of knowing: design discipline versus design science. Des. Issues 17(3), 49–55 (2001)

    Article  Google Scholar 

  • Davison R.M., Martinsons M.G., Kock N.: Principles of canonical action research. Inform. Syst. J. 14, 65–86 (2004)

    Article  Google Scholar 

  • Dixon-Woods, M., Agarwal, S., Young, B., Jones, D., Sutton, A.: Integrative approaches to qualitative and quantitative evidence. Health Development Agency, London. Published on http://www.hda-online.org.uk/documents/integrative_approaches.pdf. (2004). Accessed August 2010

  • Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A.: Synthesising qualitative and quantitative evidence: a review of possible methods. J. Health Serv. Res. Policy 10(1), 45–53 (2005)

    Article  Google Scholar 

  • Dowell J., Hudson H.: A qualitative study of medication-taking behaviour in primary care. Fam. Pract. 14(5), 369–375 (1997)

    Article  Google Scholar 

  • Ellis M.V.: Conducting and reporting integrative research reviews: accumulating scientific knowledge. Couns. Educ. Superv. 30(3), 0011–0035 (1991)

    Google Scholar 

  • Finfgeld D.L.: Metasynthesis: the state of the art—so far. Qual. Health Res. 13(7), 893–904 (2003)

    Article  Google Scholar 

  • Firestone W.A.: Meaning in method: the rhetoric of quantitative and qualitative research. Educ. Res. 16(7), 16–21 (1987)

    Google Scholar 

  • Galliers R.D., Land F.F.: Choosing appropriate information systems research methodologies. Commun. ACM 30(11), 900–902 (1987)

    Article  Google Scholar 

  • Gallupe R.B., Tan F.B.: A research manifesto for global information management. J. Glob. Inform. Manage. 7(3), 5–18 (1999)

    Google Scholar 

  • Gambling T., Brown P., Hogg P.: Research in our practice—a requirement not an option: discussion paper. Radiography 9, 71–76 (2003)

    Article  Google Scholar 

  • Gemperle, F., DiSalvo, C., Forlizzi, J., Yonkers, W.: The Hug: a new form for communication. Designing For User Experiences. In: Proceedings of the 2003 conference on Designing for user experiences, San Francisco, California, SESSION: DUX in practice II, pp. 1–4 (2003)

  • Glahn D.C., Ragland J.D., Abramoff A., Barrett J., Laird A.R., Bearden C.E., Velligan D.I.: Beyond hypofrontality: a quantitative meta-analysis of functional neuroimaging studies of working memory in schizophrenia. Hum. Brain Mapp. 25(1), 60–69 (2005)

    Article  Google Scholar 

  • Glasmeier A.K., Farrigan T.: Understanding community forestry: a qualitative meta-study of the concept, the process, and its potential for poverty alleviation in the united states case. Geogr. J. 171(1), 56–69 (2005)

    Article  Google Scholar 

  • Glass G.V.: Meta-analysis: the quantitative synthesis of research findings. In: Green, J.L., Camilli, G., Elmore, P.B. (eds) Handbook of Complementary Methods in Education Research, Chapter 25, Lawrence Erlbaum Associates, Mahwah, NJ (2006)

    Google Scholar 

  • Glass R.L., Vessey I., Ramesh V.: Research in software engineering: an analysis of the literature. Inform. Software Technol. 44(8), 491–506 (2002)

    Article  Google Scholar 

  • Guba E.G., Lincoln Y.S.: Competing paradigms in qualitative research. In: Denzin, N.K., Lincoln, Y.S. (eds) Handbook of Qualitative Research, pp. 105–117. Sage, Thousand Oaks, CA (1994)

    Google Scholar 

  • Hara, N., Bonk, C.J., Angeli, C.: Content analysis of online discussion in an applied educational psychology course. CRLT Technical Report No. 2-98, The Center for Research on Learning and Technology, Indiana University, Bloomington, IN, November 20, 1998. http://users.cdli.ca/bmann/0_ARTICLES/Web_Bonk.pdf (1998). Accessed December 2010

  • Harrison L.L.: Pulling it all together: the importance of integrative research reviews and meta-analyses in nursing. Guest editorial. J. Adv. Nurs 24(2), 224–225 (1996)

    Article  Google Scholar 

  • Head, A.J.: Beyond Google: how do students conduct academic research? First Monday 12(8), August 2007, http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1998/1873 (2007). Accessed November 1, 2010

  • Held I.M.: The gap between simulation and understanding in climate modeling. Bull. Am. Meteorol. Soc 86(11), 1609–1614 (2005)

    Article  Google Scholar 

  • Higgins, J., Green, S. (eds.): Cochrane Handbook for Systematic Reviews of Interventions, Version 5.0.2 [updated September 2009]. The Cochrane Collaboration (2009). Available from http://www.cochrane-handbook.org.

  • Holz, H.J., Applin, A., Haberman, B., Joyce, D., Purchase, H., Reed, C.: Research methods in computing: what are they, and how should we teach them? Annual Joint Conference Integrating Technology into Computer Science Education, Working group reports on Innovation and technology in computer science education (ITiCSE), ITiCSE-2006 working group reports, Bologna, Italy, 2006, pp. 96–114 (2006)

  • Howe K., Eisenhart M.: Standards for qualitative (and quantitative) research: a prolegomenon. Educ. Res. 19(4), 2–9 (1990)

    Google Scholar 

  • Inui T.S.: The virtue of qualitative and quantitative research. Ann. Intern. Med. 125(9), 770–771 (1996)

    Google Scholar 

  • Jackson G.B.: Methods for integrative reviews. Rev. Educ. Res. 50(3), 438–460 (1980)

    Google Scholar 

  • Jager, J., Kooy, A., Lehert, P., Wulffelé, M., Kolk, J., Bets, D., Verburg, J., Donker, A., Stehouwer, C.: Long term treatment with metformin in patients with type 2 diabetes and risk of vitamin B-12 deficiency: randomised placebo controlled trial. BMJ 2010 340, c2181 (2010)

  • Jarvinen, P.H.: Research questions guiding selection of an appropriate research method. ECIS [European Conference on Information Systems] 2000 Proceedings, 3–5 July 2000, paper 26, pp. 124–131. http://aisel.aisnet.org/ecis2000/26. (2000)

  • Järvinen P.: Action research is similar to design science. Qual. Quant. 41(1), 37–54 (2007)

    Article  Google Scholar 

  • Jensen L.A., Allen M.N.: Meta-synthesis of qualitative findings. Qual. Health Res. 6(4), 553–560 (1996)

    Article  Google Scholar 

  • Kassarjian H.H.: Content analysis in consumer research. J. Consum. Res. 4(1), 8–18 (1977)

    Article  Google Scholar 

  • Kitchenham B.A., Pfleeger S.L., Pickard L.M., Jones P.W., Hoaglin D.C., El Emam K., Rosenberg J.: Preliminary guidelines for empirical research in software engineering. IEEE Trans. Software Eng. 28(8), 721–734 (2002)

    Article  Google Scholar 

  • Knafl K.A., Howard M.J.: Interpreting and reporting qualitative research. Res. Nurs. Health 7(1), 17–24 (1984)

    Article  Google Scholar 

  • Kulik, J.A.: An analysis of the research on ability grouping: historical and contemporary perspectives. Ability Grouping: Research-Based Decision Making Series. National Research Center on the Gifted and Talented. Report No. NRC-G/T-9204, February 1992 (1992)

  • Leuzinger-Bohleber, M., Fischmann, T.: What is conceptual research in psychoanalysis? Int. J. Psycho-Anal. 87(Part 5), 1355–1386 (2006)

  • March S.T., Smith G.F.: Design and natural science research on information technology. Decis. Support Syst. 15(4), 251–266 (1995)

    Article  Google Scholar 

  • McCormick J., Rodney P., Varcoe C.: Reinterpretations across studies: an approach to meta-analysis. Qual. Health Res. 13(7), 933–944 (2003)

    Article  Google Scholar 

  • Merriam-Webster’s 11th Collegiate Dictionary and Thesaurus. (Electronic edition, version 4.0.) Merriam-Webster Inc. (2008)

  • Morrison J., George J.F.: Exploring the software engineering component in MIS research. Commun. ACM 38(7), 80–91 (1995)

    Article  Google Scholar 

  • Noblit, G.W., Hare, R.D.: Meta-ethnography: Synthesizing qualitative studies. Qualitative Research Methods Series, vol. 11. Sage Publications, London (1988)

  • Onwuegbuzie, A.J.: Effect sizes in qualitative research. Annual Meeting of the Association for the Advancement of Educational Research (AAER), Ponte Vedra, FL, November 2000 (2000)

  • Paterson B.L., Thorne S.E., Canam C., Jillings C.: Meta-study of Qualitative Health Research: A Practical Guide to Meta-analysis and Metasynthesis. Sage, Thousand Oaks, CA (1995)

    Google Scholar 

  • Paterson, B.L., Thorne, S.E., Canam, C., Jillings, C. (eds.): Meta-Study of Qualitative Health Research: A Practical Guide to Meta-analysis and Meta-Synthesis (Methods in Nursing Research). Sage Publications, Thousand Oaks, CA (2001)

  • Ponterotto J.G.: Qualitative research in counseling psychology: a primer on research paradigms and philosophy of science. J. Counsel. Psychol. 52(2), 126–136 (2005)

    Article  Google Scholar 

  • Roux G., Dingley C., Bush H.: Inner strength in women: Metasynthesis of qualitative findings in theory development. J. Theory Constr. Testing Lisle 6(1), 86–93 (2002)

    Google Scholar 

  • Rudner, L.M., Schafer, W.D.: How to write a scholarly research report. Pract. Assess. Res. Evaluation 6(13), http://pareonline.net/getvn.asp?v=6&n=13 (1999). Accessed August 2010

  • Russell, C.L.: An overview of the integrative research review. Progress in Transplantation, March 2005, http://findarticles.com/p/articles/mi_qa4117/is_200503/ai_n13476203/. (2005). Accessed December 22, 2010

  • Sale J.E.M., Lohfeld L.H., Brazil K.: Revisiting the quantitative-qualitative debate: implications for mixed-methods research. Qual. Quant. 36(1), 43–53 (2002)

    Article  Google Scholar 

  • Sambunjak D., Straus S., Marusic A.: A systematic review of qualitative research on the meaning and characteristics of mentoring in academic medicine. J. Gen. Intern. Med. 25(1), 72–78 (2010)

    Article  Google Scholar 

  • Sandelowski M.: Whatever happened to qualitative description?. Res. Nurs. Health 23, 334–340 (2000)

    Article  Google Scholar 

  • Sandelowski M., Barroso J.: Writing the proposal for a qualitative research methodology project. Qual. Health Res. 13(6), 781–820 (2003a)

    Article  Google Scholar 

  • Sandelowski M., Barroso J.: Classifying the findings in qualitative studies. Qual. Health Res. 13(7), 905–923 (2003b)

    Article  Google Scholar 

  • Sandelowski M., Docherty S., Emden C.: Qualitative metasynthesis: issues and techniques. Res. Nurs. Health 20, 365–371 (1997)

    Article  Google Scholar 

  • Sandelowski M., Voils C.I., Barroso J.: Defining and designing mixed research synthesis studies. Res. Schools 13(1), 29–40 (2006)

    Google Scholar 

  • Segal, J., Grinyer, A., Sharp, H.: The type of evidence produced by empirical software engineers. In: Proceedings of the 2005 International Conference on Software Engineering, Session: Realising Evidence-Based Software Engineering (REBSE), St. Louis, Missouri, pp. 1–4 (2005)

  • Shelton J.H.: Handbook for Technical Writing. NTC Business Books, Chicago (1994)

    Google Scholar 

  • Sjøberg D., Dyboa T., Jørgensen M.: The future of empirical methods in software engineering research. In: Briand, L., Wolf, A. (eds) Future of Software Engineering, IEEE-CS Press, Los Alamitos, CA (2007)

    Google Scholar 

  • Suri, H.: A critique of contemporary methods of research synthesis. Post-Script, vol. 1, 1, September 2000, Faculty of Education, University of Melbourne, Australia. Available at http://www.edfac.unimelb.edu.au/research/resources/student_res/postscriptfiles/vol1/vol1_1_suri.pdf. (2000). Accessed September 2010

  • Sutton R.I., Staw B.M.: What theory is Not. Admin. Sci. Quart. 40(3), 371–384 (1995)

    Article  Google Scholar 

  • Thorne S.: Data analysis in qualitative research. Evidence Based Nurs. 3, 68–70 (2000)

    Article  Google Scholar 

  • Thorne S., Jensen L., Kearney M., Noblit G., Sandelowski M.: Qualitative metasynthesis: reflections on methodological orientation and ideological agenda. Qual. Health Res. 14(10), 1342–1365 (2004)

    Article  Google Scholar 

  • Venable, J.R.: The role of theory and theorising in design science research. In: Proceedings of the 1st International Conference on Design Science in Information Systems and Technology (DESRIST 2006), February 24–25, 2006, Claremont, CA (2006)

  • Voils C., Sandelowski M., Barroso J., Hasselblad V.: Making sense of qualitative and quantitative findings in mixed research synthesis studies. Field Methods 20(1), 3–25 (2008)

    Article  Google Scholar 

  • Whittemore R., Knafl K.: The integrative review: updated methodology. J. Adv. Nurs. 52(5), 546–553 (2005)

    Article  Google Scholar 

  • Williams A.: Perspectives on spirituality at the end of life: a meta-summary. Palliative Supportive Care 4(4), 407–417 (2006)

    Google Scholar 

  • Wolf F.M.: Meta-Analysis: Quantitative Methods for Research Synthesis. Sage Publications, Inc., Newbury Park, CA (1986)

    Google Scholar 

  • Zuber-Skerritt O.: Action learning and action research: paradigm, praxis and programs. In: Sankara, S., Dick, B., Passfield, R. (eds) Effective Change Management through Action Research and Action Learning: Concepts, Perspectives, Processes and Applications, Chapter 1., pp. 1–20. Southern Cross University Press, Lismore (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Lockheed Martin Corporation, 3333 Pilot Knob Road, Mail Stop U2 G28, Eagan, MN, 55121, USA

    Nuzhat Haneef

Authors
  1. Nuzhat Haneef
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nuzhat Haneef.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Haneef, N. Empirical research consolidation: a generic overview and a classification scheme for methods. Qual Quant 47, 383–410 (2013). https://doi.org/10.1007/s11135-011-9524-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-011-9524-z

Keywords

Search

Navigation