Skip to main content
SpringerLink
Log in

Control variable use and reporting in operations management: a systematic literature review and revisit

  • Published:
Management Review Quarterly Aims and scope Submit manuscript

Abstract

The purpose of this paper is to identify (1) to what extent do researchers follow standards on the use and reporting of control variables? (2) what are the main weaknesses presented in current as they relate to control variable inclusion/exclusion? And (3) what are the recommendations based on previous standards when it comes to the selection and inclusion of control variables? We do this through the lens of Operations Management literature (inclusive of Supply Chain Management, Information Systems and Knowledge Management) to control for field specific practices and apply management theory as an initial lens to examine best practices. An extensive systematic literature review and content analysis is conducted on literature spanning 2010–2020. Control variable analyses are conducted and organized into interdisciplinary domains and DVs, providing researchers with insights in use of specific control variables from a micro-level perspective. Next, we identify strengths and weaknesses in current control variable use among a ten-year span from a macro perspective. We also provide trends across time on control variable use and inclusion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

The data set analyzed in the study are available from the corresponding author on reasonable request.

Notes

  1. OM: operations management, SCM: supply chain management, CV: control variable, DV: dependent variable, IV: independent variable, IS/KM: information systems and knowledge management.

  2. Some articles were published that referenced a biography of a particular author and/or an author who had recently passed away.

  3. To analyze the standards as set forth by control variable literature, reporting methods needed to include statistical data analysis. Most articles referenced archival data analyses and survey data analyses. As a result, our results are heavily biased toward these methods and should not be utilized to assess laboratory observational experimental design. However, we did not exclude survey-based experimental designs. Nevertheless, this design did not represent a large enough sample in our data to be analyzed separately.

  4. Control variable standard practices by method are provided in Table 8 in the Appendix.

  5. Research methods keywords included specific method keywords that can be utilized broadly. These keywords included “survey design,” “case study,” “action research,” etc. Theory categorization was similar with specific theories referenced in the keywords utilized. Examples included “resource-based view,” “systems theory,” “agency theory,” etc.

  6. While not within the confines of the article to distinguish between operations management and supply chain management, given the various similarities, interdependencies, and more established corporate reputation for OM (e.g., chief operations officer overseeing supply chain management), we denoted a delineation particularly from the standpoint of supply chain management referencing external activities (e.g., supplier relationship management, procurement/purchasing, transportation) versus operations management referencing internal activities (e.g. production, manufacturing, internal operational control, and monitor).

  7. Based on sample size calculation, the difference between samples testing for equivalence (with α = 5%, group number = 11), the minimum n per group = 11. Assuming a non-normal distribution, n per group = 13 (Lehmann 2012).

  8. Firm performance refers to dependent variables not offering specific identification of performance (e.g., “organizational performance,” “performance”) but rather as an aggregate construct theoretically specified as a composite of dimensions, and at times, those dimensions are not specified. While it is not within the confines of this article to debate the use of “firm performance” as a variable, one should note the prevalence of the performance variable in OM literature as well as current perspectives regarding its use as an aggregate measure (e.g., Miller et al. 2012).

  9. True to the OM literature, most control variables were firm level (e.g., firm age, financial performance, market performance, etc.), and few were demographic oriented (e.g., top management gender, age, etc.). This is understandable given that in most cases, the unit of analysis stays within a manufacturing organization. However, there was a lack of insight into interorganizational relationships with performance-dependent variables. With the growing importance of SCM, the inclusion of such control variables in OM and IS/KM domains may be necessary for assuming theoretical justification.

  10. Only control variables that were common among dependent variables were leveraged for statistical comparison tests.

References

  • Aguilar MS (2020) Replication studies in mathematics education: what kind of questions would be productive to explore? Int J Sci Math Educ 18(1):37–50. https://doi.org/10.1007/s10763-020-10069-7

    Article  Google Scholar 

  • Atinc G, Simmering MJ, Kroll MJ (2012) Control variable use and reporting in macro and micromanagement research. Organ Res Methods 15(1):57–74. https://doi.org/10.1177/1094428110397773

    Article  Google Scholar 

  • Becker TE (2005) Potential problems in the statistical control of variables in organizational research: a qualitative analysis with recommendations. Organ Res Methods 8(3):274–289. https://doi.org/10.1177/1094428105278021

    Article  Google Scholar 

  • Becker TE, Atinc G, Breaugh JA, Carlson KD, Edwards JR, Spector PE (2016) Statistical control in correlational studies: 10 essential recommendations for organizational researchers. J Organ Behav 37(2):157–167. https://doi.org/10.1002/job.2053

    Article  Google Scholar 

  • Bernerth JB, Aguinis H (2016) A critical review and best-practice recommendations for control variable usage. Pers Psychol 69(1):229–283. https://doi.org/10.1111/peps.12103

    Article  Google Scholar 

  • Bernerth JB, Cole MS, Taylor EC, Walker HJ (2018) Control variables in leadership research: a qualitative and quantitative review. J Manag 44(1):131–160. https://doi.org/10.1177/0149206317690586

    Article  Google Scholar 

  • Block J, Brändle L (2022) Looking back to look ahead: MRQ’s past five years of evidence-based management research in numbers. Manag Rev Q 72(4):917–926. https://doi.org/10.1007/s11301-022-00303-7

    Article  Google Scholar 

  • Block JH, Fisch C, Kanwal N, Lorenzen S, Schulze A (2022) Replication studies in top management journals: an empirical investigation of prevalence, types, outcomes, and impact. Manag Rev Q. https://doi.org/10.1007/s11301-022-00269-6

    Article  Google Scholar 

  • Bonett DG (2012) Replication-extension studies. Curr Dir Psychol Sci 21(6):409–412. https://doi.org/10.1177/0963721412459512

    Article  Google Scholar 

  • Carlson KD, Wu J (2012) The illusion of statistical control: control variable practice in management research. Organ Res Methods 15(3):413–435. https://doi.org/10.1177/1094428111428817

    Article  Google Scholar 

  • Cuervo-Cazurra A, Andersson U, Brannen MY, Nielsen BB, Rebecca Reuber A (2016) From the editors: can I trust your findings? Ruling out alternative explanations in international business research. J Int Bus Stud 47:881–897

    Article  Google Scholar 

  • Curry A (2019) Across the great divide: a literature review of management accounting and operations management at the shop floor. Manag Rev Q 69(1):75–119. https://doi.org/10.1007/s11301-018-0147-5

    Article  Google Scholar 

  • DiMaggio PJ, Powell WW (1983) The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 147–160

  • Durach CF, Kembro J, Wieland A (2017) A new paradigm for systematic literature reviews in supply chain management. J Supply Chain Manag 53(4):67–85

    Article  Google Scholar 

  • Edwards JR (2008) To prosper, organizational psychology should… overcome methodological barriers to progress. J Organ Behav Int J Ind Occup Organ Psychol Behav 29(4):469–491

    Google Scholar 

  • Fisher M, Olivares M, Staats BR (2020) Why empirical research is good for operations management, and what is good empirical operations management? Manuf Serv Oper Manag 22(1):170–178

    Article  Google Scholar 

  • Fraser H, Barnett A, Parker TH, Fidler F (2020) The role of replication studies in ecology. Ecol Evol 10(12):5197–5207. https://doi.org/10.1002/ece3.6330

    Article  Google Scholar 

  • Greenland S, Pearl J (2011) Adjustments and their consequences—collapsibility analysis using graphical models. Int Stat Rev 79(3):401–426

    Article  Google Scholar 

  • Hallgren KA (2012) Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol 8(1):23

    Article  Google Scholar 

  • Hansen C, Steinmetz H, Block J (2022) How to conduct a meta-analysis in eight steps: a practical guide. Manag Rev Q 72(1):1–19. https://doi.org/10.1007/s11301-021-00247-4

    Article  Google Scholar 

  • Helmuth CA, Craighead CW, Connelly BL, Collier DY, Hanna JB (2015) Supply chain management research: key elements of study design and statistical testing. J Oper Manag 36:178–186

    Article  Google Scholar 

  • Hill TD, Davis AP, Roos JM, French MT (2020) Limitations of fixed-effects models for panel data. Sociol Perspect 63(3):357–369

    Article  Google Scholar 

  • Hochrein S, Glock CH, Bogaschewsky R, Heider M (2015) Literature reviews in supply chain management: a tertiary study. Manag Rev Q 65(4):239–280. https://doi.org/10.1007/s11301-015-0113-4

    Article  Google Scholar 

  • Hsieh HF, Shannon SE (2005) Three approaches to qualitative content analysis. Qual Health Res 15(9):1277–1288

    Article  Google Scholar 

  • Janis I (1965) The problem of validating content analysis. The content analysis reader, 358–375

  • Johns G (2006) The essential impact of context on organizational behavior. Acad Manag Rev 31(2):386–408. https://doi.org/10.5465/amr.2006.20208687

    Article  Google Scholar 

  • Krippendorf K (2004) Content analysis: an introduction to its methodology, 2nd edn. Sage, Thousand Oaks, CA

    Google Scholar 

  • Lavenberg SS, Welch PD (1981) A perspective on the use of control variables to increase the efficiency of Monte Carlo simulations. Manag Sci 27(3):322–335

    Article  Google Scholar 

  • Lehmann EL (2012) “Student” and small-sample theory. Selected works of EL Lehmann, 1001–1008

  • Light RJ (1971) Measures of response agreement for qualitative data: some generalizations and alternatives. Psychol Bull 76(5):365

    Article  Google Scholar 

  • Li M (2021) Uses and abuses of statistical control variables: Ruling out or creating alternative explanations? J Bus Res 126:472–488

    Article  Google Scholar 

  • Mayring P (2008) Qualitative inhaltanalyse—Grundlagen und Techniken (qualitative content analysis). Beltz Verlag, Weinheim

    Google Scholar 

  • Miller J, Davis-Sramek B, Fugate BS, Pagell M, Flynn BB (2021) Editorial commentary: addressing confusion in the diffusion of archival data research. J Supply Chain Manag 57(3):130–146

    Article  Google Scholar 

  • Miller CC, Washburn NT, Glick WH (2012) PERSPECTIVE—The Myth of firm performance. Organ Sci 24(3):948–964

    Article  Google Scholar 

  • Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group* T (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 151(4):264–269

    Article  Google Scholar 

  • Nielsen BB, Raswant A (2018) The selection, use, and reporting of control variables in international business research: a review and recommendations. J World Bus 53(6):958–968

    Article  Google Scholar 

  • Ramos J (2003) Using TF-IDF to determine word relevance in document queries (Vol. 242). Proceedings of the first instructional conference on machine learning

  • Scott WR, Meyer JW (1994) Institutional environments and organizations: structural complexity and individualism. Sage, New York, NY

    Google Scholar 

  • Seuring S, Gold S (2012) Conducting content-analysis based literature reviews in supply chain management. Supply Chain Manag Int J 17(5):544–555

    Article  Google Scholar 

  • Shaw JD (2017) Advantages of starting with theory. Acad Manag J 60(3):819–822

    Article  Google Scholar 

  • Spector PE, Brannick MT (2011) Methodological urban legends: the misuse of statistical control variables. Organ Res Methods 14(2):287–305. https://doi.org/10.1177/1094428110369842

    Article  Google Scholar 

  • Spector PE, Zapf D, Chen PY, Frese M (2000) Why negative affectivity should not be controlled in job stress research: don’t throw out the baby with the bath water. J Organ Behav 21(1):79–95

    Article  Google Scholar 

  • Steinmetz H, Block J (2022) Meta-analytic structural equation modeling (MASEM): new tricks of the trade. Manag Rev Q 72(3):605–626. https://doi.org/10.1007/s11301-022-00293-6

    Article  Google Scholar 

  • Stremersch S, Gonzalez J, Valenti A, Villanueva J (2023) The value of context-specific studies for marketing. J Acad Mark Sci 51(1):50–65. https://doi.org/10.1007/s11747-022-00872-9

    Article  Google Scholar 

  • Tabachnick BG, Fidell LS (2000) Computer-assisted research design and analysis. Allyn & Bacon Inc, Boston, MA

    Google Scholar 

  • Terwiesch C, Olivares M, Staats BR, Gaur V (2019) OM forum—A review of empirical operations management over the last two decades. Manuf Serv Oper Manag 22(4):656–668. https://doi.org/10.1287/msom.2018.0755

    Article  Google Scholar 

  • Van Exel J, De Graaf G (2005) Q methodology: a sneak preview. https://www.betterevaluation.org/sites/default/files/vanExel.pdf (Accessed 15 April 2023)

Download references

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. ADVANCE/CSG, ISEG, Universidade de Lisboa, Rua Miguel Lupi, 20, 1200-725, Lisbon, Portugal

    Carla Curado & Mírian Oliveira

  2. Escola de Negócios, Pontifícia Universidade Católica do Rio Grande do Sul, Av. Ipiranga, 6681- Prédio 50, Porto Alegre, RS, 90619-900, Brazil

    Mírian Oliveira

  3. University of Rhode Island, Kingston, RI, 02881, USA

    Dara G. Schniederjans

  4. Graduate School of Management, University of the West of Santa Catarina, Chapecó, SC, 89813-000, Brazil

    Eduardo Kunzel Teixeira

Authors
  1. Carla Curado
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mírian Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dara G. Schniederjans
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eduardo Kunzel Teixeira
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design equally. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Dara G. Schniederjans.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Table 8.

Table 8 Control variable standards by method
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Curado, C., Oliveira, M., Schniederjans, D.G. et al. Control variable use and reporting in operations management: a systematic literature review and revisit. Manag Rev Q (2023). https://doi.org/10.1007/s11301-023-00348-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11301-023-00348-2

Keywords

JEL Classification

Search

Navigation