Is Quantitative Research Ethical? Tools for Ethically Practicing, Evaluating, and Using Quantitative Research

Abstract

This editorial offers new ways to ethically practice, evaluate, and use quantitative research (QR). Our central claim is that ready-made formulas for QR, including ‘best practices’ and common notions of ‘validity’ or ‘objectivity,’ are often divorced from the ethical and practical implications of doing, evaluating, and using QR for specific purposes. To focus on these implications, we critique common theoretical foundations for QR and then recommend approaches to QR that are ‘built for purpose,’ by which we mean designed to ethically address specific problems or situations on terms that are contextually relevant. For this, we propose a new tool for evaluating the quality of QR, which we call ‘relational validity.’ Studies, including their methods and results, are relationally valid when they ethically connect researchers’ purposes with the way that QR is oriented and the ways that it is done—including the concepts and units of analysis invoked, as well as what its ‘methods’ imply more generally. This new way of doing QR can provide the liberty required to address serious worldly problems on terms that are both practical and ethically informed in relation to the problems themselves rather than the confines of existing QR logics and practices.

This is a preview of subscription content, log in to check access.

References

  1. Abrahamson, E., Berkowitz, H., & Dumez, H. (2016). A more relevant approach to relevance in management studies: An essay on performativity. Academy of Management Review, 41, 367–381.

    Article  Google Scholar 

  2. American Psychological Association. (2009). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association.

    Google Scholar 

  3. Bettis, R. A., Ethiraj, S., Gambardella, A., Helfat, C., & Mitchell, W. (2016). Creating repeatable cumulative knowledge in strategic management. Strategic Management Journal, 37(2), 257–261.

    Article  Google Scholar 

  4. *Buchholz, R. A., & Rosenthal, S. B. (2008). The unholy alliance of business and science. Journal of Business Ethics, 78(1), 199–206.

    Article  Google Scholar 

  5. Campbell, D. T. (1957). Factors relevant to the validity of experiments in social settings. Psychological Bulletin, 54, 297–312.

    Article  Google Scholar 

  6. Campbell, D. T. (1991). Methods for the experimenting society. Evaluation Practice, 12(3), 223–260.

    Article  Google Scholar 

  7. Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research on teaching. In N. L. Gage (Ed.), Handbook of research on teaching (pp. 171–246). Chicago: Rand McNally.

    Google Scholar 

  8. Cartwright, N. (1993). In defence of this worldly’causality: Comments on van Fraassen’s laws and symmetry. Philosophy and Phenomenological Research, 53(2), 423–429.

    Article  Google Scholar 

  9. Cartwright, N. (2004). Causation: One word, many things. Philosophy of Science, 71(5), 805–819.

    Article  Google Scholar 

  10. Cartwright, N. (2006). Well-ordered science: Evidence for use. Philosophy of Science, 73(5), 981–990.

    Article  Google Scholar 

  11. Cartwright, N. (2007). Hunting causes and using them: Approaches in philosophy and economics. Cambridge: Cambridge University Press.

    Google Scholar 

  12. *Collison, D., Cross, S., Ferguson, J., Power, D., & Stevenson, L. (2012). Legal determinants of external finance revisited: The inverse relationship between investor protection and societal well-being. Journal of Business Ethics, 108(3), 393–410.

    Article  Google Scholar 

  13. Cunliffe, A. L. (2003). Reflexive inquiry in organizational research: Questions and possibilities. Human Relations, 56, 983–1003.

    Article  Google Scholar 

  14. Daston, L. (1995). The moral economy of science. Osiris, 10, 2–24.

    Article  Google Scholar 

  15. Daston, L. (2005). Scientific error and the ethos of belief. Social Research, 72, 1–28.

    Google Scholar 

  16. Davies, W. (2017, January 19). How statistics lost their power—And why we should fear what comes next. The Guardian. Retrieved from https://www.theguardian.com/politics/2017/jan/19/crisis-of-statistics-big-data-democracy.

  17. Davis, M. S. (1971). That’s interesting! Towards a phenomenology of sociology and a sociology of phenomenology. Philosophy of the Social Sciences, 1(4), 309–344.

    Article  Google Scholar 

  18. Deetz, S. (1996). Describing differences in approaches to organization science: Rethinking Burrell and Morgan and their legacy. Organization Science, 7, 191–207.

    Article  Google Scholar 

  19. Dewey, J. (1929). The quest for certainty. New York: Minton, Balch, & Co.

    Google Scholar 

  20. Dunn, W. N. (1982). Reforms as arguments. Knowledge, 3(3), 293–326.

    Google Scholar 

  21. Erturk, I., Froud, J., Johal, S., Leaver, A., & Williams, K. (2013). (How) do devices matter in finance? Journal of Cultural Economy, 6(3), 336–352.

    Article  Google Scholar 

  22. Ezzamel, M., & Willmott, H. (2014). Registering ‘the ethical’ in organization theory formation: Towards the disclosure of an ‘invisible force’. Organization Studies, 35, 1013–1039.

    Article  Google Scholar 

  23. Falleti, T. G., & Lynch, J. F. (2009). Context and causal mechanisms in political analysis. Comparative Political Studies, 42(9), 1143–1166.

    Article  Google Scholar 

  24. Farjoun, M., Ansell, C., & Boin, A. (2015). Pragmatism in organization studies: Meeting the challenges of a dynamic and complex world. Organization Science, 26(6), 1787–1804.

    Article  Google Scholar 

  25. Feldman, M. S., & Orlikowski, W. J. (2011). Theorizing practice and practicing theory. Organization science.

  26. Freeman, R. E. (2002). Toward a new vision for management research: A commentary on “Organizational researcher values, ethical responsibility, and the committed-to-participant research perspective”. Journal of Management Inquiry, 11(2), 186–189.

    Article  Google Scholar 

  27. Gabbay, D. M., Hartmann, S., & Woods, J. (2011). Handbook of the history of logic: Inductive logic (Vol. 10). Oxford: Elsevier.

    Google Scholar 

  28. Gelman, A. (2015). The connection between varying treatment effects and the crisis of unreplicable research a Bayesian perspective. Journal of Management, 41, 632–643.

    Article  Google Scholar 

  29. Gigerenzer, G., & Marewski, J. N. (2015). Surrogate science the idol of a universal method for scientific inference. Journal of Management, 41, 421–440.

    Article  Google Scholar 

  30. Gigerenzer, G., Swijtink, Z. G., Porter, T. M., Daston, L., Beatty, J., & Krüger, L. (1989). The empire of chance: How probability changed science and everyday life. Cambridge: Cambridge University Press.

    Google Scholar 

  31. *Greenwood, M. (2016). Approving or improving research ethics in management journals. Journal of Business Ethics, 137, 1–14.

    Article  Google Scholar 

  32. Hacking, I. (1990). The taming of chance. Cambridge: Cambridge University Press.

    Google Scholar 

  33. Hacking, I. (1992a). Statistical language, statistical truth and statistical reason: The self-authentification of a style of scientific reasoning. In E. McMullin (Ed.), The social dimensions of science (Vol. 3, pp. 130–157). Notre Dame: University of Notre Dame Press.

    Google Scholar 

  34. Hacking, I. (1992b). The self-vindication of the laboratory sciences. In A. Pickering (Ed.), Science as practice and culture (pp. 29–64). Chicago: Chicago Unviersity Press.

    Google Scholar 

  35. Hacking, I. (1999). The social construction of what?. Cambridge: Harvard University Press.

    Google Scholar 

  36. Hacking, I. (2001). An introduction to probability and inductive logic. Cambridge: Cambridge University Press.

    Google Scholar 

  37. Hacking, I. (2002). Historical Ontology. Cambridge: Harvard University Press.

    Google Scholar 

  38. Hacking, I. (2006). The emergence of probability: A philosophical study of early ideas about probability, induction and statistical inference. Cambridge: Cambridge University Press.

    Google Scholar 

  39. Hakala, J., & Ylijoki, O.-H. (2001). Research for whom? Research orientations in three academic cultures. Organization, 8(2), 373–380.

    Article  Google Scholar 

  40. Hardy, C., & Clegg, S. (1997). Relativity without relativism: Reflexivity in post-paradigm organization studies. British Journal of Management, 8, 5–17.

    Article  Google Scholar 

  41. Hardy, C., Phillips, N., & Clegg, S. (2001). Reflexivity in organization and management theory: A study of the production of the research “subject”. Human Relations, 54, 531–560.

    Article  Google Scholar 

  42. *Hill, R. P. (2002). Stalking the poverty consumer a retrospective examination of modern ethical dilemmas. Journal of Business Ethics, 37(2), 209–219.

    Article  Google Scholar 

  43. *Holland, D., & Albrecht, C. (2013). The worldwide academic field of business ethics: Scholars’ perceptions of the most important issues. Journal of Business Ethics, 117(4), 777–788.

    Article  Google Scholar 

  44. Howie, D. (2002). Interpreting probability: Controversies and developments in the early twentieth century. Cambridge: Cambridge University Press.

    Google Scholar 

  45. Huhtala, M., Feldt, T., Lämsä, A. M., Mauno, S., & Kinnunen, U. (2011). Does the ethical culture of organisations promote managers’ occupational well-being? Investigating indirect links via ethical strain. Journal of Business Ethics, 101(2), 231–247.

    Article  Google Scholar 

  46. Jeanes, E. (2016). Are we ethical? Approaches to ethics in management and organisation research. Organization. doi:10.1177/1350508416656930.

  47. *Kaptein, M., & Schwartz, M. S. (2008). The effectiveness of business codes: A critical examination of existing studies and the development of an integrated research model. Journal of Business Ethics, 77(2), 111–127.

    Article  Google Scholar 

  48. *Keeble, J. J., Topiol, S., & Berkeley, S. (2003). Using indicators to measure sustainability performance at a corporate and project level. Journal of Business Ethics, 44(2), 149–158.

    Article  Google Scholar 

  49. *Kerssens-van Drongelen, I. C., & Fisscher, O. A. (2003). Ethical dilemmas in performance measurement. Journal of Business Ethics, 45(1), 51–63.

    Article  Google Scholar 

  50. *Knox, S., & Gruar, C. (2007). The application of stakeholder theory to relationship marketing strategy development in a non-profit organization. Journal of Business Ethics, 75(2), 115–135.

    Article  Google Scholar 

  51. Kozlowski, S. W. J., & Klein, K. J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 3–90). San Francisco: Jossey-Bass.

    Google Scholar 

  52. Latour, B., & Woolgar, S. (1986). Laboratory life: The construction of scientific facts. Beverly Hills: Sage.

    Google Scholar 

  53. Law, J. (2009). Seeing like a survey. Cultural Sociology, 3(2), 239–256.

    Article  Google Scholar 

  54. MacKenzie, D. A., Muniesa, F., & Siu, L. (2007). Do economists make markets? On the performativity of economics. Princeton: Princeton University Press.

    Google Scholar 

  55. Martela, F. (2015). Fallible inquiry with ethical ends-in-view: A pragmatist philosophy of science for organizational research. Organization Studies, 36, 537–563.

    Article  Google Scholar 

  56. *Michalos, A. C. (1988). Editorial. Journal of Business Ethics, 1, 1.

    Article  Google Scholar 

  57. Misangyi, V. F., Greckhamer, T., Furnari, S., Fiss, P. C., Crilly, D., & Aguilera, R. (2017). Embracing causal complexity the emergence of a neo-configurational perspective. Journal of Management, 43(1), 255–282.

    Article  Google Scholar 

  58. Morgan, G. (2006). Images of organization. Thousand Oaks: Sage.

    Google Scholar 

  59. OED Online. Oxford University Press, (June 2016). Retrieved June 10, 2016, from http://www.oxforddictionaries.com/definition/english/orient.

  60. *Orlitzky, M., Louche, C., Gond, J. P., & Chapple, W. (2015). Unpacking the drivers of corporate social performance: A multilevel, multistakeholder, and multimethod analysis. Journal of Business Ethics. doi:10.1007/s10551-015-2822-y.

  61. *Painter-Morland, M. (2011). Rethinking responsible agency in corporations: Perspectives from Deleuze and Guattari. Journal of Business Ethics, 101(1), 83–95.

    Article  Google Scholar 

  62. Panter, A. T., & Sterba, S. K. (Eds.). (2011). Handbook of ethics in quantitative methodology. New York: Routledge.

    Google Scholar 

  63. Parkhurst, J. O., & Abeysinghe, S. (2016). What constitutes “good” evidence for public health and social policy-making? From hierarchies to appropriateness. Social Epistemology, 30(5–6), 665–679.

    Article  Google Scholar 

  64. Pashler, H., & Wagenmakers, E. J. (2012). Editors’ introduction to the special section on replicability in psychological science a crisis of confidence? Perspectives on Psychological Science, 7(6), 528–530.

    Article  Google Scholar 

  65. Pedhazur, E. J., & Schmelkin, L. P. (2013). Measurement, design, and analysis: An integrated approach. Washington, DC: Psychology Press.

    Google Scholar 

  66. *Prado, A. M., & Woodside, A. G. (2015). Deepening understanding of certification adoption and non-adoption of international-supplier ethical standards. Journal of Business Ethics, 132(1), 105–125.

    Article  Google Scholar 

  67. *Ralston, D. A., Egri, C. P., Furrer, O., Kuo, M. H., Li, Y., Wangenheim, F., et al. (2014). Societal-level versus individual-level predictions of ethical behavior: A 48-society study of collectivism and individualism. Journal of Business Ethics, 122(2), 283–306.

    Google Scholar 

  68. *Rathner, S. (2013). The influence of primary study characteristics on the performance differential between socially responsible and conventional investment funds: A meta-analysis. Journal of Business Ethics, 118(2), 349–363.

    Article  Google Scholar 

  69. Rorty, R. (2009). Philosophy and the mirror of nature. Princeton: Princeton University Press.

    Google Scholar 

  70. Rose, N. (1985). The psychological complex. London: Routledge Kegan.

    Google Scholar 

  71. *Rousseau, D. M., Manning, J., & Denyer, D. (2008). Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. Academy of Management Annals, 2(1), 475–515.

    Article  Google Scholar 

  72. Russell, J., Greenhalgh, T., Byrne, E., & McDonnell, J. (2008). Recognizing rhetoric in health care policy analysis. Journal of Health Services Research and Policy, 13, 40–46.

    Article  Google Scholar 

  73. Schön, D. A. (1992). The theory of inquiry: Dewey’s legacy to education. Curriculum Inquiry, 22(2), 119–139.

    Article  Google Scholar 

  74. Scott, J. C. (1998). Seeing like a state: How certain schemes to improve the human condition have failed. New Haven: Yale University Press.

    Google Scholar 

  75. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. New York: Wadsworth Cengage learning.

    Google Scholar 

  76. Shapin, S., & Schaffer, S. (1985). Leviathan and the air pump: Hobbes, Boyle and the experimental life. Princeton: Princeton University Press.

    Google Scholar 

  77. Singleton, V., & Law, J. (2013). Devices as rituals: Notes on enacting resistance. Journal of Cultural Economy, 6(3), 259–277.

    Article  Google Scholar 

  78. *Soares, C. (2003). Corporate versus individual moral responsibility. Journal of Business Ethics, 46(2), 143–150.

    Article  Google Scholar 

  79. Stone, D. A. (1989). Causal stories and the formation of policy agendas. Political Science Quarterly, 104(2), 281–300.

    Article  Google Scholar 

  80. Tuck, E., & McKenzie, M. (2015). Relational validity and the “where” of inquiry: Place and land in qualitative research. Qualitative Inquiry, 21(7), 633–638.

    Article  Google Scholar 

  81. Turker, D. (2009). Measuring corporate social responsibility: A scale development study. Journal of business ethics, 85(4), 411–427.

    Article  Google Scholar 

  82. Wasserman, L. (2013). All of statistics: A concise course in statistical inference. New York: Springer.

    Google Scholar 

  83. Werhane, P. H., & Freeman, R. E. (1999). Business ethics: The state of the art. International Journal of Management Reviews, 1(1), 1–16.

    Article  Google Scholar 

  84. Wicks, A. C., & Freeman, R. E. (1998). Organizational studies and the new pragmatism: Positivism, anti-positivism, and the search for ethics. Organization Science, 9, 123–140.

    Article  Google Scholar 

  85. Wooldridge, J. M. (2010). Econometric analysis of cross section and panel data. Cambridge: MIT press.

    Google Scholar 

  86. Young, I. M. (2011). Justice and the politics of difference. Princeton: Princeton University Press.

    Google Scholar 

  87. Zyphur, M. J., Pierides, D. C., & Roffe, J. (2016a). Measurement and statistics in ‘organization science’: Philosophical, sociological, and historical perspectives. In R. Mir, H. Willmott, & M. Greenwood (Eds.), The Routledge companion to philosophy in organization studies (pp. 474–482). Abingdon: Routledge.

    Google Scholar 

  88. Zyphur, M. J., Zammuto, R. F., & Zhang, Z. (2016b). Multilevel latent polynomial regression for modeling (in) congruence across organizational groups: The case of organizational culture research. Organizational Research Methods, 19(1), 53–79.

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by Australian Research Council’s Future Fellowship scheme (project FT140100629).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Michael J. Zyphur.

Appendix

Appendix

Typical regression methods minimize the residual variance of outcome variables by predicting the mean (or statistical ‘expectation’) of an outcome. This can be shown by a simple regression model as follows:

$$y_{i} = a + \beta x_{i} + e_{i}$$

wherein \(y_{i}\) is an outcome for some unit i, \(a\) is a regression intercept, \(\beta\) is a slope linking a predictor \(x_{i}\) to the outcome, and \(e_{i}\) is a residual. Typical regression assumptions pertain to \(e\) because this is parameterized as a random variable for estimation and inference, typically with a normal distribution such that:

$$e\sim\,N\left( {0,\;\sigma^{2} } \right),$$

wherein the residual variable has zero mean and variance \(\sigma^{2}\).

However, if the outcome variable y is parameterized using the regression equation, the prediction of the outcome enters as the variable’s average. Specifically:

$$y\sim\,N\left( {a + \beta x,\;\sigma^{2} } \right)$$

wherein all terms are as before, but the focus on the average of the outcome \(y\) at each level of the predictor \(x\) is clarified by showing how what is predicted are average levels of the outcomes \(y\) at different values of the predictor \(x\).

The implication is that most regression methods implicitly assume that predicting averages are what is of greatest interest to researchers. With a focus on reducing errors in inference, the best way to do this probabilistically is to predict averages, but this is only true to the extent that a single numerical prediction of an assumedly homogenous group is desired based on the group’s average standing along an outcome \(y\) at a specific value of a predictor \(x\). However, whether or not (and to what extent) averages may be relevant for a specific purpose and research orientation is typically left unclarified in QR, and we propose that this should be examined on a case-by-case basis with an eye to the ethics this or other QR practices.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zyphur, M.J., Pierides, D.C. Is Quantitative Research Ethical? Tools for Ethically Practicing, Evaluating, and Using Quantitative Research. J Bus Ethics 143, 1–16 (2017). https://doi.org/10.1007/s10551-017-3549-8

Download citation

Keywords

  • Quantitative research
  • Quantitative methods
  • Statistics
  • Probability
  • Regression
  • Research design
  • Data analysis
  • Inductive inference