Quality & Quantity

, Volume 45, Issue 6, pp 1253–1271 | Cite as

Assessing legitimation in mixed research: a new framework

  • Anthony J. Onwuegbuzie
  • R. Burke Johnson
  • Kathleen M. T. Collins
Article

Abstract

In this article, we have merged or intersected two typologies: Greene’s (Res Sch 13(1):93–98, 2006) four-domain typology for developing a methodological or research paradigm in the social and behavioral sciences and Onwuegbuzie and Johnson’s (Res Sch 13(1):48–63, 2006) nine-component typology for assessing mixed research legitimation. We argue that merging or interconnecting these typologies present a framework for assessing legitimation in mixed research. Specifically, we demonstrate how the nine types of legitimation map onto Greene’s (Res Sch 13(1):93–98, 2006) four methodological domains and illustrate how legitimation in mixed research, rather than being viewed as a procedure that occurs at a specific step of the mixed research process, is better conceptualized as a continuous iterative, interactive, and dynamic process. Additionally, in presenting this framework, we hope to reduce misperceptions that some researchers have voiced about mixed research.

Keywords

Research paradigm Assessing legitimation Mixed research Dynamic process 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bhaskar R.: A Realist Theory of Science. Harvester Press, Sussex (1978)Google Scholar
  2. Bracht G.H., Glass G.V.: The external validity of experiments. Am. Edu. Res. J. 5, 437–474 (1968)Google Scholar
  3. Brandom, R.B. (eds): Rorty and his Critics. Blackwell, Malden, MA (2000)Google Scholar
  4. Campbell D.T.: Factors relevant to the validity of experiments in social settings. Psychol. Bull. 54, 297–312 (1957)CrossRefGoogle Scholar
  5. Campbell D.T., Stanley J.C.: Experimental and Quasi-experimental Designs for Research. Rand McNally, Chicago (1963)Google Scholar
  6. Collins, K.M.T.: Advanced sampling designs in mixed research: Current practices and emerging trends in the social and behavioral sciences. In: Tashakkori, A., Teddlie, C. (eds.) The Handbook of Mixed Methods in Social and Behavioral Research, 2nd edn. Sage, Thousand Oaks, CA (in press)Google Scholar
  7. Collins K.M.T., Onwuegbuzie A.J., Sutton I.L.: A model incorporating the rationale and purpose for conducting mixed methods research in special education and beyond. Learn. Disabil. A Contemp. J. 4, 67–100 (2006)Google Scholar
  8. Cook T.D., Campbell D.T.: Quasi-experimentation: Design and Analysis Issues for Field Settings. Rand McNally, Chicago (1979)Google Scholar
  9. Curtis S., Gesler W., Smith G., Washburn S.: Approaches to sampling and case selection in qualitative research: examples in the geography of health. Soc. Sci. Med. 50, 1001–1014 (2000)CrossRefGoogle Scholar
  10. Dellinger A.: Validity and the review of the literature. Res. Sch. 12(2), 41–54 (2005)Google Scholar
  11. Eisenhart M.A., Howe K.R.: Validity in educational research. In: LeCompte, M.D., Millroy, W.L., Preissle, J. (eds) The Handbook of Qualitative Research in Education, pp. 643–680. Academic Press, San Diego, CA (1992)Google Scholar
  12. Ely M., Anzul M., Friedman T., Garner D., Steinmetz A.C.: Doing Qualitative Research: Circles within Circles. Falmer, New York (1991)Google Scholar
  13. Etheredge, C.: Living in guild wars: a cultural analysis of the discourse, dance and evolution of an MMOG phenomenon. Unpublished master’s thesis, University of South Alabama, Mobile, Alabama (2006)Google Scholar
  14. Glaser B.G., Strauss A.L.: The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine De Gruyter, New York (1967)Google Scholar
  15. Greene J.C.: Toward a methodology of mixed methods social inquiry. Res. Sch. 13(1), 93–98 (2006)Google Scholar
  16. Greene J.C.: Is mixed methods social inquiry a distinctive methodology?. J. Mix. Method Res. 2, 7–22 (2008)CrossRefGoogle Scholar
  17. Greene J.C., Caracelli V.J.: Defining and describing the paradigm issue in mixed-method evaluation. In: Greene, J.C., Varacelli, J.C. (eds) Advances in Mixed-Method Evaluation: The Challenges and Benefits of Integrating Diverse Paradigms. New Directions for Evaluation, vol. 74, pp. 5–17. Jossey-Bass, San Francisco, CA (1997)Google Scholar
  18. Greene J.C., Caracelli V.J.: Making paradigmatic sense of mixed methods practice. In: Tashakkori, A., Teddlie, C. (eds) Handbook of Mixed Methods in Social and Behavioral Research, pp. 91–110. Sage, Thousand Oaks, CA (2003)Google Scholar
  19. Greene J.C., Caracelli V.J., Graham W.F.: Toward a conceptual framework for mixed-method evaluation designs. Edu. Eval. Policy Anal. 11, 255–274 (1989)Google Scholar
  20. Greene J.C., Benjamin L., Goodyear L.: The merits of mixing methods in evaluation. Evaluation 7(1), 25–44 (2001)CrossRefGoogle Scholar
  21. Guba E.G., Lincoln Y.S.: Fourth Generation Evaluation. Sage, Newbury Park, CA (1989)Google Scholar
  22. Guba E.G., Lincoln Y.S. : Paradigmatic controversies, contradictions, and emerging confluences. In: Denzin, N.K., Lincoln, Y.S. (eds) The Sage Handbook of Qualitative Research, 3rd edn, pp. 191–215. Sage, Thousand Oaks, CA (2005)Google Scholar
  23. Hammersley M.: What’s Wrong with Ethnography. Routledge, London (1992)Google Scholar
  24. Howe K.R.: Against the quantitative-qualitative incompatibility thesis or dogmas die hard. Edu. Res. 17(8), 10–16 (1988)Google Scholar
  25. Huck S.W., Sandler H.M.: Rival Hypotheses: Alternative Interpretations of Data Based Conclusions. Harper Collins, New York (1979)Google Scholar
  26. Johnson R.B.: Examining the validity structure of qualitative research. Education 118, 282–292 (1999)Google Scholar
  27. Johnson R.B.: Editorial: living with tensions. J. Mix. Method Res. 2, 203–207 (2008)CrossRefGoogle Scholar
  28. Johnson R.B.: Toward a more inclusive scientific research in education. Edu. Res. 38, 449–457 (2009)CrossRefGoogle Scholar
  29. Johnson R.B., Christensen L.B.: Educational Research: Quantitative, Qualitative, and Mixed Approaches. Sage, Thousand Oaks, CA (2008)Google Scholar
  30. Johnson R.B., Turner L.A.: Data collection strategies in mixed methods research. In: Tashakkori, A., Teddlie, C. (eds) Handbook of Mixed Methods in Social and Behavioral Research, pp. 297–319. Sage, Thousand Oaks, CA (2003)Google Scholar
  31. Johnson R.B., Onwuegbuzie A.J.: Mixed methods research: a research paradigm whose time has come. Edu. Res. 33(7), 14–26 (2004)CrossRefGoogle Scholar
  32. Johnson R.B., Onwuegbuzie A.J., Turner L.A.: Toward a definition of mixed methods research. J. Mix. Method Res. 1(2), 1–22 (2007)Google Scholar
  33. Kuhn T. : Postscript—1969. In: Kuhn, T. (eds) The Structure of Scientific Revolutions, 2nd edn, pp. 174–210. University of Chicago Press, Chicago (1970)Google Scholar
  34. Kuhn T.: The Essential Tension: Selected Studies in Scientific Tradition and Change. University of Chicago Press, Chicago (1974)Google Scholar
  35. Kuhn T.: The structure of scientific revolutions, 3rd edn. University of Chicago Press, Chicago (1996) (Original work published 1962)Google Scholar
  36. Kuhn T.: The Road Since Structure. University of Chicago Press, Chicago (2000)Google Scholar
  37. Kvale S.: The social construction of validity. Qual. Inq. 1, 19–40 (1995)CrossRefGoogle Scholar
  38. Lather P.: Issues of validity in openly ideological research: between a rock and a soft place. Interchange 17, 63–84 (1986)CrossRefGoogle Scholar
  39. Lather P.: Fertile obsession: validity after poststructuralism. Soc. Q. 34, 673–693 (1993)CrossRefGoogle Scholar
  40. Li S., Marquart J.M., Zercher C.: Conceptual issues and analytical strategies in mixed-method studies of preschool inclusion. J. Early Interv. 23, 116–132 (2000)CrossRefGoogle Scholar
  41. Lincoln Y.S.: Emerging criteria for quality in qualitative and interpretive research. Qual. Inq. 1, 275–289 (1995)CrossRefGoogle Scholar
  42. Madey D.L.: Some benefits of integrating qualitative and quantitative methods in program evaluation, with some illustrations. Edu. Eval. Policy Anal. 4, 223–236 (1982)Google Scholar
  43. Maxcy S.J. : Pragmatic threads in mixed methods research in the social sciences: The search for multiple modes of inquiry and the end of the philosophy of formalism. In: Tashakkori, A., Teddlie, C. (eds) Handbook of Mixed Methods in Social and Behavioral Research, pp. 51–89. Sage, Thousand Oaks, CA (2003)Google Scholar
  44. Maxwell J.A.: Understanding and validity in qualitative research. Harv. Edu. Rev. 62, 279–299 (1992)Google Scholar
  45. Maxwell J.A.: Causal explanation, qualitative research, and scientific inquiry in education. Edu. Res. 33(2), 3–11 (2004a)CrossRefGoogle Scholar
  46. Maxwell, J.A.: Realism as a stance for mixed methods research. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA, April 2004bGoogle Scholar
  47. McLean, J.E.: Foreword: From qualitative to quantitative and half way back. Res. Sch. 13(1), iii–iv (2006)Google Scholar
  48. McMillan, J.H.: Examining categories of rival hypotheses for educational research. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA, April 2000Google Scholar
  49. Mertens D. : Mixed methods and the politics of human research: the transformative-emancipatory perspective. In: Tashakkori, A., Teddlie, C. (eds) Handbook of Mixed Methods in Social and Behavioral Research, pp. 135–164. Sage, Thousand Oaks, CA (2003)Google Scholar
  50. Messick S.: Validity. In: Linn, R.L. (eds) Educational Measurement, 3rd edn, pp. 13–103. Macmillan, Old Tappan, N.J (1989)Google Scholar
  51. Messick S.: Validity of psychological assessment: validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. Am. Psychol. 50, 741–749 (1995)CrossRefGoogle Scholar
  52. Miles M.B., Huberman A.M.: Drawing valid meaning from qualitative data: toward a shared craft. Edu. Res. 13, 20–30 (1984)Google Scholar
  53. Morgan D.L.: Paradigms lost and pragmatism regained: methodological implications of combining qualitative and quantitative methods. J. Mix. Method Res. 1(1), 48–76 (2007)CrossRefGoogle Scholar
  54. Newman I., Ridenour C.S., Newman C., DeMarco G.M.P.: A typology of research purposes and its relationship to mixed methods. In: Tashakkori, A., Teddlie, C. (eds) Handbook of Mixed Methods in Social and Behavioral Research, pp. 167–188. Sage, Thousand Oaks, CA (2003)Google Scholar
  55. Onwuegbuzie A.J.: Effect sizes in qualitative research: a prolegomenon. Qual. Quant. Int. J. Methodol. 37, 393–409 (2003)CrossRefGoogle Scholar
  56. Onwuegbuzie A.J. : Mixed methods research in sociology and beyond. In: Ritzer, G. (eds) The Blackwell Encyclopedia of Sociology, vol. VI, pp. 2978–2981. Blackwell Publishers Ltd, Oxford (2007)Google Scholar
  57. Onwuegbuzie A.J., Johnson R.B.: The validity issue in mixed research. Res. Sch. 13(1), 48–63 (2006)Google Scholar
  58. Onwuegbuzie A.J., Leech N.L.: On becoming a pragmatist researcher: the importance of combining quantitative and qualitative research methodologies. Int. J. Soc. Res. Methodol. Theory Pract. 8, 375–387 (2005)CrossRefGoogle Scholar
  59. Onwuegbuzie, A.J., Leech, N.L.: Linking research questions to mixed methods data analysis procedures. Qual. Rep. 11, 474–498. http://www.nova.edu/ssss/QR/QR11-3/onwuegbuzie.pdf (2006) Accessed 29 Nov 2006
  60. Onwuegbuzie A.J., Leech N.L.: A call for qualitative power analyses. Qual. Quant. Int. J. Methodol. 41, 105–121 (2007)CrossRefGoogle Scholar
  61. Onwuegbuzie A.J., Witcher A.E., Collins K.M.T., Filer J.D., Wiedmaier C.D., Moore C.W.: Students’ perceptions of characteristics of effective college teachers: A validity study of a teaching evaluation form using a mixed methods analysis. Am. Edu. Res. J. 44, 113–160 (2007)CrossRefGoogle Scholar
  62. Onwuegbuzie A.J., Daniel L.G., Collins K.M.T.: Problems associated with student teaching evaluations. Qual. Quant. Int. J. Methodol. 43, 197–209 (2009)CrossRefGoogle Scholar
  63. Patton, M.Q.: Foreword: Trends and issues as context. Res. Sch. 13(1), i–ii (2006)Google Scholar
  64. Phillips D.C.: Validity in qualitative research: why the worry about warrant will not wane. Edu. Urban Soc. 20, 9–24 (1987)CrossRefGoogle Scholar
  65. Putnam H.: The Collapse of the Fact/Value Dichotomy and Other Essays. Harvard University Press, Cambridge, MA (2002)Google Scholar
  66. Reichardt C.S., Rallis S.F.: Qualitative and quantitative inquiries are not incompatible: a call for a new partnership. In: Reichardt, C.S., Rallis, S.F. (eds) The Qualitative–Quantitative Debate: New Perspectives, pp. 85–91. Jossey-Bass, San Francisco, CA (1994)Google Scholar
  67. Rescher N.: Realistic Pragmatism: An Introduction to Pragmatic Philosophy. State University of New York Press, Albany, NY (2000)Google Scholar
  68. Sandelowski M.: Real qualitative researchers don’t count: the use of numbers in qualitative research. Res. Nurs. Health 24, 230–240 (2001)CrossRefGoogle Scholar
  69. Sandelowski M. : Tables or tableaux? The challenges of writing and reading mixed methods studies. In: Tashakkori, A., Teddlie, C. (eds) Handbook of Mixed Methods in Social and Behavioral Research, pp. 321–350. Sage, Thousand Oaks, CA (2003)Google Scholar
  70. Sandelowski, M., Voils, C.I., Barroso, J.: Defining and designing mixed research synthesis studies. Res. Sch. 13(1), (2006)Google Scholar
  71. Smith M.L., Glass G.V.: Research and evaluation in education and the social sciences. Prentice Hall, Englewood Cliffs, NJ (1987)Google Scholar
  72. Tashakkori A., Teddlie C.: Mixed Methodology: Combining Qualitative and Quantitative Approaches. Applied Social Research Methods Series, vol. 46. Sage, Thousand Oaks, CA (1998)Google Scholar
  73. Tashakkori A., Teddlie C.: The past and future of mixed methods research: from data triangulation to mixed model designs. In: Tashakkori, A., Teddlie, C. (eds) Handbook of Mixed Methods in Social and Behavioral Research, pp. 671–701. Sage, Thousand Oaks, CA (2003a)Google Scholar
  74. Tashakkori, A., Teddlie, C. (eds): Handbook of Mixed Methods in Social and Behavioral Research. Sage, Thousand Oaks, CA (2003b)Google Scholar
  75. Tashakkori, A., Teddlie, C.: Validity issues in mixed methods research: calling for an integrative framework. Paper presented at the annual meeting of the American educational research association. San Francisco, CA, April, 2006Google Scholar
  76. Teddlie C., Stringfield S.: Schools Make a Difference: Lessons Learned from a 10-year Study of School Effects. Teachers College Press, New York (1993)Google Scholar
  77. Teddlie C., Tashakkori A.: Major issues and controversies in the issue of mixed methods in the social and behavioral sciences. In: Tashakkori, A., Teddlie, C. (eds) Handbook of Mixed Methods in Social and Behavioral Research, pp. 3–50. Sage, Thousand Oaks, CA (2003)Google Scholar
  78. Willems E.P., Raush , H. L.: Naturalistic Viewpoints in Psychological Research. Holt, Rinehart, and Winston, New York (1969)Google Scholar
  79. Williams D.D.: Naturalistic evaluation: potential conflicts between evaluation standards and criteria for conducting naturalistic inquiry. Edu. Eval. Policy Anal. 8, 87–99 (1986)Google Scholar
  80. Yanchar S.C., Williams D.D.: Reconsidering the compatibility thesis and eclecticism: five proposed guidelines for methods use. Edu. Res. 35(9), 3–12 (2006)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2009

Authors and Affiliations

  • Anthony J. Onwuegbuzie
    • 1
  • R. Burke Johnson
    • 2
  • Kathleen M. T. Collins
    • 3
  1. 1.Department of Educational Leadership and CounselingSam Houston State UniversityHuntsvilleUSA
  2. 2.College of EducationUniversity of South AlabamaMobileUSA
  3. 3.Department of Curriculum and InstructionUniversity of Arkansas at FayettevilleFayettevilleUSA

Personalised recommendations