Abstract
The rise of scientific psychology has been tied closely to the adoption of experimental techniques in the study of human cognition and behavior. Experiments provide an elegant and rigorous tool to test mechanistic theories, and they do so by distilling a theoretically relevant idea into an observable measure as part of a behavioral task – a process commonly dubbed as operationalization. I argue that modern experimental psychology is plagued by a continued trend toward overlooking the critical role of such operationalization. This neglect manifests in two seemingly opposing ways, either in research that jumps to inadequate conclusions by playing down the role of tasks in generating the research findings or, conversely, in research that is mainly focused on examining tasks (“paradigms”) for their own sake rather than testing theoretical ideas. This state of affairs can be rectified by re-embracing the power of explicit, theory-driven operationalization as I discuss in general terms as well as applied to the exemplary case of research on rule-violation behavior. Shifting focus back to how a theoretical idea is operationalized in a given research program will help render experimental research in psychology methodologically sound while at the same time being able to provide generalizable findings.
Author Note
The outline of this chapter was drafted during a research stay at the Center for Interdisciplinary Research, Bielefeld, and this chapter would not have come to life without their generous support. I thank Leonhard Höhnel for contributing his beautiful artwork, Felicitas Muth for her critical comments on an earlier version of this argument, Robert “How2Rulebreaker” Wirth for his continued obsession with research on rule-violation behavior, and Davood Gozli for taking this work at least as serious as we do.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This claim might warrant additional evidence in the form of exemplary citations (quite a few of which readily come to mind) or a quantitative corpus analysis of a larger body of the published literature. I opted not to include any specific references because this chapter does not intend to point fingers at particular articles or journals. Instead, the assessment is meant to describe my subjective perception of the field.
- 2.
The vocabulary employed in this sentence as well as the methods described in the context of the following studies may suggest a theoretical relation to “action dynamics” accounts (McKinstry et al., 2008). This resemblance is coincidental, however, and this work was not performed with such a theoretical perspective in mind.
- 3.
Instructing participants to break a similarly instructed rule may seem somewhat unorthodox, because rule-breaking then becomes nested in a meta-rule of either following or violating an instructed stimulus-response mapping (Gozli, 2017). This is especially the case if both instructions emerge from the same source, e.g., from the same experimenter as in the case of our experiments. We still opted to do so because relying on free choices between rule-following and rule-breaking is plagued by a general reluctance to opt for rule violations so that it is difficult to find a control condition which comes with a similar experience for one or the other response. Crucially, even this artificial situation conforms to the minimal definition of rule-breaking as behavior that does not align with a rule. Whether participants do construe it the same way is a different question, of course (Gozli, 2019). This concern would be especially relevant if the results of the free choice condition had not replicated for instructed violations. Observing a similar pattern of results for instructed violations, however, seems to validate the experimental design.
- 4.
I borrow the sculpting metaphor from an inspiring presentation by Thomas Schenk (personal communication) in the summer term 2014 on the two visual stream models. Like many other influential theories and models in psychology, this field of research started with an overly general claim on the neurophysiological and functional separation of two pathways for processing visual information (Milner & Goodale, 2006). Follow-up work then continued to show exceptions from this simple model, thus carving an elaborate understanding of visual processing in the context of conscious perception and action control alike (Schenk et al., 2011).
References
Asch, S. E. (1951). Effects of group pressure upon the modification and distortion of judgments. In H. Guetzkow (Ed.), Groups, leadership and men (pp. 177–190). Carnegie Press.
Balzer, W., Moulines, C. U., & Sneed, J. D. (1987). An architectonic for science: The structuralist program. Reidel.
Becker, G. S. (1968). Crime and punishment: An economic approach. Journal of Political Economy, 76(2), 169–217. https://doi.org/10.1007/978-1-349-62853-7_2
Bereby-Meyer, Y., & Shalvi, S. (2015). Deliberate honesty. Current Opinion in Psychology, 6, 195–198. https://doi.org/10.1016/j.copsyc.2015.09.004
Bermeitinger, C. (2012). Paradigmen der Kognitiven Psychologie: Affektive Reize I [The paradigms of cognitive psychology: Affective stimuli I]. Uni-Edition.
Billig, M. (2013). Learn to write badly: How to succeed in the social sciences. Cambridge, UK: Cambridge University Press.
Blass, T. (1999). The Milgram paradigm after 35 years: Some things we now know about obedience to authority. Journal of Applied Social Psychology, 29(5), 955–978. https://doi.org/10.1111/j.1559-1816.1999.tb00134.x
Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychological Review, 108, 624–652. https://doi.org/10.1037/0033-295X.108.3.624
Chen, S., Hui, N., Bond, M., Sit, A., Wong, S., Chow, V., et al. (2006). Reexamining personal, social, and cultural influences on compliance behavior in the United States, Poland, and Hong Kong. The Journal of Social Psychology, 146, 223–244. https://doi.org/10.3200/SOCP.146.2.223-244
Cialdini, R. B., & Goldstein, N. J. (2004). Social influence: Compliance and conformity. Annual Review of Psychology, 55, 591–621. https://doi.org/10.1146/annurev.psych.55.090902.142015
Dignath, D., Kiesel, A., & Eder, A. B. (2014). Flexible conflict management: Conflict avoidance and conflict adjustment in reactive cognitive control. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(4), 975. https://doi.org/10.1037/xlm0000089
Dreisbach, G. (2012). Mechanisms of cognitive control: The functional role of task rules. Current Directions in Psychological Science, 21(4), 227–231.
Elms, A. C., & Milgram, S. (1966). Personality characteristics associated with obedience and defiance toward authoritative command. Journal of Experimental Research in Personality, 1, 282–289.
Eriksen, B. A., & Eriksen, C. W. (1974). Effects of noise letters upon the identification of a target letter in a nonsearch task. Perception & Psychophysics, 16, 143–149. https://doi.org/10.3758/BF03203267
Fischbacher, U., & Föllmi-Heusi, F. (2013). Lies in disguise—An experimental study on cheating. Journal of the European Economic Association, 11, 525–547. https://doi.org/10.1111/jeea.12014
Foerster, A., Pfister, R., Schmidts, C., Dignath, D., & Kunde, W. (2013). Honesty saves time (and justifications). Frontiers in Psychology, 4, 473. https://doi.org/10.3389/fpsyg.2013.00473
Frings, C., Schneider, K. K., & Fox, E. (2015). The negative priming paradigm: An update and implications for selective attention. Psychonomic Bulletin & Review, 22(6), 1577–1597. https://doi.org/10.3758/s13423-015-0841-4
Garner, W. R., Hake, H. W., & Eriksen, C. W. (1956). Operationism and the concept of perception. Psychological Review, 63(3), 149–159. https://doi.org/10.1037/h0042992
Gneezy, U. (2005). Deception: The role of consequences. American Economic Review, 95(1), 384–394. https://doi.org/10.1257/0002828053828662
Gottfredson, M. R., & Hirschi, T. (1990). A general theory of crime. Stanford University Press.
Gozli, D. G. (2017). Behaviour versus performance: The veiled commitment of experimental psychology. Theory & Psychology, 27, 741–758. https://doi.org/10.1177/0959354317728130
Gozli, D. G. (2019). Experimental psychology and human agency. Springer. https://doi.org/10.1007/978-3-030-20422-8
Gozli, D. G., & Deng, W. (2018). Building blocks of psychology: On remaking the unkept promises of early schools. Integrative Psychological and Behavioral Science, 52, 1–24. https://doi.org/10.1007/s12124-017-9405-7
Grace, R. C. (2001). On the failure of operationism. Theory & Psychology, 11(1), 5–33. https://doi.org/10.1177/0959354301111001
Hackman, J. R. (1969). Toward understanding the role of tasks in behavioral research. Acta Psychologica, 31, 97–128. https://doi.org/10.1016/0001-6918(69)90073-0
Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2–3), 61–83. https://doi.org/10.1017/S0140525X0999152X
Hilbig, B. E., & Hessler, C. M. (2013). What lies beneath: How the distance between truth and lie drives dishonesty. Journal of Experimental Social Psychology, 49(2), 263–266. https://doi.org/10.1016/j.jesp.2012.11.010
Hilbig, B. E., & Thielmann, I. (2017). Does everyone have a price? On the role of payoff magnitude for ethical decision making. Cognition, 163, 15–25. https://doi.org/10.1016/j.cognition.2017.02.011
Hommel, B., Proctor, R. W., & Vu, K. P. L. (2004). A feature-integration account of sequential effects in the Simon task. Psychological Research, 68(1), 1–17. https://doi.org/10.1007/s00426-003-0132-y
Imhof, M. F., & Rüsseler, J. (2019). Performance monitoring and correct response significance in conscientious individuals. Frontiers in Human Neuroscience, 13, 239. https://doi.org/10.3389/fnhum.2019.00239
Jusyte, A., Pfister, R., Mayer, S. V., Schwarz, K. A., Wirth, R., Kunde, W., & Schönenberg, M. (2017). Smooth criminal: Convicted rule-breakers show reduced cognitive conflict during deliberate rule violations. Psychological Research, 81(5), 939–946. https://doi.org/10.1007/s00426-016-0798-6
Kiesel, A., Steinhauser, M., Wendt, M., Falkenstein, M., Jost, K., Philipp, A. M., & Koch, I. (2010). Control and interference in task switching–a review. Psychological Bulletin, 136(5), 849–874. https://doi.org/10.1037/a0019842
Kunde, W., Kiesel, A., & Hoffmann, J. (2003). Conscious control over the content of unconscious cognition. Cognition, 88(2), 223–242. https://doi.org/10.1016/S0010-0277(03)00023-4
McKinstry, C., Dale, R., & Spivey, M. J. (2008). Action dynamics reveal parallel competition in decision making. Psychological Science, 19(1), 22–24. https://doi.org/10.1111/j.1467-9280.2008.02041.x
Meiran, N., Pereg, M., Kessler, Y., Cole, M. W., & Braver, T. S. (2014). The power of instructions: Proactive configuration of stimulus–response translation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(3), 768. https://doi.org/10.1037/xlm0000063
Meiser, T. (2011). Much pain, little gain? Paradigm-specific models and methods in experimental psychology. Perspectives on Psychological Science, 6(2), 183–191. https://doi.org/10.1177/1745691611400241
Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67(4), 371–378. https://doi.org/10.1037/h0040525
Milgram, S. (1974). Obedience to authority. Harper & Row.
Milner, D., & Goodale, M. (2006). The visual brain in action (2nd Ed.). OUP.
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49–100. https://doi.org/10.1006/cogp.1999.0734
Pashler, H., & Johnston, J. C. (1989). Chronometric evidence for central postponement in temporally overlapping tasks. The Quarterly Journal of Experimental Psychology, 41(1), 19–45. https://doi.org/10.1080/14640748908402351
Pfister, R., Wirth, R., Schwarz, K., Steinhauser, M., & Kunde, W. (2016). Burdens of non-conformity: Motor execution reveals cognitive conflict during deliberate rule violations. Cognition, 147, 93–99. https://doi.org/10.1016/j.cognition.2015.11.009
Pfister, R., Wirth, R., Weller, L., Foerster, A., & Schwarz, K. A. (2019). Taking shortcuts: Cognitive conflict during motivated rule-breaking. Journal of Economic Psychology, 71, 138–147. https://doi.org/10.1016/j.joep.2018.06.005
Popper, K. R. (1935/2005). Logik der Forschung [The logic of scientific discovery]. Mohr Siebeck.
Popper, K. R. (1972). Objective knowledge: An evolutionary approach. Clarendon.
Popper, K. R. (1988). The open universe: An argument for indeterminism (Vol. II). Routledge.
Pratt, T. C., & Cullen, F. T. (2000). The empirical status of Gottfredson and Hirschi’s general theory of crime: A meta-analysis. Criminology, 38(3), 931–964. https://doi.org/10.1111/j.1745-9125.2000.tb00911.x
Reason, J. (1990). Human error. Cambridge University Press.
Schenk, T., Franz, V., & Bruno, N. (2011). Vision-for-perception and vision-for-action: Which model is compatible with the available psychophysical and neuropsychological data? Vision Research, 51(8), 812–818. https://doi.org/10.1016/j.visres.2011.02.003
Shalvi, S., Eldar, O., & Bereby-Meyer, Y. (2012). Honesty requires time (and lack of justifications). Psychological Science, 23, 1264–1270. https://doi.org/10.1177/0956797612443835
Simon, J. R. (1990). The effects of an irrelevant directional cue on human information processing. In R. W. Proctor & T. G. Reeve (Eds.), Stimulus–response compatibility. An integrated perspective (pp. 31–86). Elsevier. https://doi.org/10.1016/S0166-4115(08)61218-2
Smedslund, J. (1997). The structure of psychological common sense. Mahwah, NJ: Lawrence Erlbaum.
Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18(6), 643–662. https://doi.org/10.1037/h0054651
Teo, T. (2018). Outline of theoretical psychology: Critical investigations. London, UK: Palgrave Macmillan.
Ting, C. (2020). The feedback loop of rule-breaking: Experimental evidence. The Social Science Journal, 1–14. https://doi.org/10.1016/j.soscij.2018.11.004
Ting, C. (this volume). The role of social context in experimental studies on dishonesty.
Ting, C., & Fitzgerald, R. (2020). The work to make an experiment work. International Journal of Social Research Methodology, 23(3), 329–345. https://doi.org/10.1080/13645579.2019.1694621
Valsiner, J. (2017). From methodology to methods in human psychology. Springer.
Waszak, F., Pfister, R., & Kiesel, A. (2013). Top-down vs. bottom-up: When instructions overcome automatic retrieval. Psychological Research, 77(5), 611–617. https://doi.org/10.1007/s00426-012-0459-3
Wenke, D., Gaschler, R., Nattkemper, D., & Frensch, P. A. (2009). Strategic influences on implementing instructions for future actions. Psychological Research, 73(4), 587–601. https://doi.org/10.1007/s00426-009-0239-x
Westermann, R. (1987). Wissenschaftstheoretische Grundlagen der experimentellen Psychologie [Epistemological foundations of experimental psychology]. In G. Lüer (Ed.), Allgemeine Experimentelle Psychologie (pp. 4–42). Fischer.
Westermann, R. (2017). Methoden psychologischer Forschung und evaluation [Methods for psychological research and evaluation] (Chapter 7). Kohlhammer.
Wirth, R., Pfister, R., Foerster, A., Huestegge, L., & Kunde, W. (2016). Pushing the rules: Effects and aftereffects of deliberate rule violations. Psychological Research, 80(5), 838–852. https://doi.org/10.1007/s00426-015-0690-9
Wirth, R., Foerster, A., Herbort, O., Kunde, W., & Pfister, R. (2018). This is how to be a rule breaker. Advances in Cognitive Psychology, 14(1), 21–37. https://doi.org/10.5709/acp-0235-2
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Pfister, R. (2022). Operationalization and Generalization in Experimental Psychology: A Plea for Bold Claims. In: Gozli, D., Valsiner, J. (eds) Experimental Psychology. Theory and History in the Human and Social Sciences. Springer, Cham. https://doi.org/10.1007/978-3-031-17053-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-17053-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17052-2
Online ISBN: 978-3-031-17053-9
eBook Packages: Behavioral Science and PsychologyBehavioral Science and Psychology (R0)