Skip to main content
SpringerLink
Log in

In Skinner's Early Footsteps: Analyzing Verbal Behavior in Large Published Corpora

  • Theoretical Article
  • Published:
The Psychological Record Aims and scope Submit manuscript

Abstract

Using data from large databases of words that listeners have rated in terms of how positively or negatively they respond to them, we present a mode of analysis that reveals regularities across several languages. Specifically, a normalized measure of bias in emotional valence shows that the degree to which positively-valenced words predominate in a language is a reliable function of absolute valence intensity. This measure supports the comparison of corpora in which valence has been measured in different ways and can serve as a baseline for evaluating the role of interactive variables. We also suggest a way to apply the measure to characterize the verbal behavior of individuals. Throughout we show how the general topic is amenable to the functional perspective that behavior analysts apply to verbal behavior, which creates an opportunity for behavior analysts to contribute to interdisciplinary inquiries into verbal phenomena.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. The rare exceptions often have been impactful. Consider the landmark study of early language development by Hart and Risley (1995), which has been cited more than 5000 times (Google Scholar search conducted April 6, 2016), and about which Suskind (2015) has asserted, "Hart and Risley's insight into the role of early language exposure in a child's ultimate academic achievement was an incredible step forward in social thought" (p. 28). For other interesting verbal investigations, see Moerk (1990) and McDowell and Caron (2010).

  2. The precepts of this literature could prompt lengthy theoretical debate, including about the ontological status of emotions (Skinner, 1953, called them epiphenomial), the utility of words as a unit of analysis (Skinner, 1957, suggested that verbal topographies can be misleading), and the value of verbal-report data (Baer, Wolf, & Risley, 1968, called them unreliable). Regarding such matters, we assert for present purposes only that "epiphenominal" implies a type of response that bears explanation (Skinner, 1953, 1957); that measurement in research always relies on behavior topography to some extent (e.g., Johnston & Pennypacker, 1980); and that verbal reports are one of the few means available to estimate the occurrence of important private events (Wolf, 1978).

  3. Thus, the procedure uses verbal responses to estimate the emotional effect that words "spoken" by others typically have on listeners. While acknowledging the challenges of using self-reports to measure internal events (e.g., Critchfield & Epting, 1998; Critchfield, Tucker, & Vuchinich, 1998), we challenge the reader to come up with a better procedure. Skeptics about the validity of emotion ratings should note that the overall emotional quality of spoken and written communication, when measured through these ratings, correlates with a variety of practical listener actions that might be anticipated based on positive or negative moods (Floh, Koller, & Zauner, 2013; Guerini, Pepe, & Lepri, 2012; Norman, Avolio, & Luthans, 2010; Petty, Schumann, Richman, & Strathman, 1993).

  4. In such procedures, the rating-scale anchors "happy" and "unhappy" apparently are not intended to be taken strictly literally but rather, through research experience, have been shown to be useful in promoting reports of general pleasantness and unpleasantness of emotional responding. This generic character of valence ratings is illustrated in a study by Warriner, Kuperman, and Brysbaert (2013), who created a large corpus of emotion ratings of English words by instructing raters to think of "happy" as "pleased, satisfied, contented, hopeful" and "unhappy" as "annoyed, unsatisfied, melancholic, despaired, or bored" (p. 1193). Dodds, et al. (2015) provided no such elaboration, but their ratings correlate strongly with those of Warriner et al. (see Warriner & Kuperman, 2015), suggesting that the simple anchors "happy" and "unhappy" indeed suffice to measure general valence.

  5. To appreciate the general character of valence, consider the Dodds et al. (2015) mean ratings for some words that register as pleasant (laughter = 8.5, free = 7.96, sisters = 6.62), essentially neutral (desk = 5.26, tooth = 4.84), and unpleasant (weeping = 3.02, stupid = 2.68, terrorist = 1.30).

  6. There is, however, some evidence that raters over-use items presented on the left side of a horizontally-arrayed scale (Friedman & Amoo, 1999). In the scale of Dodds et al. (2015), response options progressed from negative (left) to positive (right), so their study may actually have underestimated the magnitude of positivity dominance.

  7. We arrived at this conclusion by assigning valence ratings from the Dodds et al. (2015) corpus to 88 emotional adjectives for which Ridgeway et al. (1985, Table 2) provided cross-sectional data on child usage. No valence rating was available for 33 additional words, and two others were dropped from consideration because of multiple connotations that might elicit distinctly different emotional responses. A word was considered to be "in use" if at least 50 % of the children reportedly used it by age 6.

  8. Warriner & Kuperman (2015, Figure 2) showed previously that words with extreme valences, both positive and negative, tend to generate strong listener arousal, but the tendency is more pronounced for extremely negative words. Our Fig. 4 indicates how this outcome is expressed in the context of a log valence-ratio function.

References

  • Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91–97. doi:10.1901/jaba.1968.1-91.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bailey, J. S. (1991). Marketing behavior analysis requires different talk. Journal of Applied Behavior Analysis, 24, 445–448. doi:10.1901/jaba.1991.24-445.

    Article  PubMed  PubMed Central  Google Scholar 

  • Baron, A., & Derenne, A. (2000). Quantitative summaries of single-subject studies: What do group comparisons tell us about individual performances? The Behavior Analyst, 23(1), 101–106.

    PubMed  PubMed Central  Google Scholar 

  • Baumeister, R. F., Bratslavsky, E., Finkenauer, C., & Vohs, K. D. (2001). Bad is stronger than good. Review of General Psychology, 5(4), 323–370.

    Article  Google Scholar 

  • Becirevic, A., Critchfield, T. S., & Reed, D. D. (2016). On the social acceptability of behavior-analytic terms: Crowdsourced comparisons of lay and technical language. The Behavior Analyst. doi:10.1007/s40614-016-0067-4. Advance online publication.

    Google Scholar 

  • Bolanowski, S. J., Jr., & Gescheider, G. A. (1991). Ratio scaling of psychological magnitude: Essays in honor of S.S. Stevens. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Boucher, J., & Osgood, C. E. (1969). The Pollyanna hypothesis. Journal of Verbal Learning and Verbal Behavior, 8, 1–8.

    Article  Google Scholar 

  • Bradley, M.M., & Lang, P.J. (1999). Affective norms for English words (ANEW): Instruction manual and affective ratings. Technical Report C-1, The Center for Research in Psychophysiology, University of Florida.

  • Cook, S. W., & Skinner, B. F. (1939). Some factors influencing the distribution of associated words. The Psychological Record, 3, 178–84.

    Google Scholar 

  • Critchfield, T.S. (2010). Crucial issues in the applied analysis of verbal behavior: A review of Crucial conversations: Tools for talking when the stakes are high. The Analysis of Verbal Behavior, 133-145.

  • Critchfield, T. S. (2011). Translational contributions of the experimental analysis of behavior. The Behavior Analyst, 34, 3–17.

    PubMed  PubMed Central  Google Scholar 

  • Critchfield, T.S., Becirevic, A., & Reed, D.D. (2016a). Normative emotional responses to behavior analysis jargon: How not to use words to win friends and influence people. Behavior Analysis in Practice. In press.

  • Critchfield, T.S., Becirevic, A., & Reed, D.D. (2016b). On the social validity of behavior analytic communication: A call for research and a description of one method. Analysis of Verbal Behavior. In press

  • Critchfield, T. S., & Epting, L. K. (1998). The trouble with babies and the value of bath water: Complexities in the use of verbal reports as data. Analysis of Verbal Behavior, 15, 65–74.

    PubMed  PubMed Central  Google Scholar 

  • Critchfield, T. S., Haley, R., Sabo, B., Colbert, J., & Macropoulis, G. (2003). A half century of scalloping in the work habits of the United States Congress. Journal of Applied Behavior Analysis, 36, 465–486.

    Article  PubMed  PubMed Central  Google Scholar 

  • Critchfield, T. S., Reed, D. D., & Jarmolowicz, D. P. (2014). Historically low productivity by the United Stated Congress: Snapshot of a reinforcement-contingency system in transition. The Psychological Record, 65, 161–176.

    Article  Google Scholar 

  • Critchfield, T. S., & Stilling, S. T. (2014). Explanatory flexibility of the matching law: Complex situational bias effects in football play selection. The Psychological Record, 64, 371–380.

    Article  Google Scholar 

  • Critchfield, T. S., & Stilling, S. T. (2015). A matching law analysis of risk tolerance and gain-loss framing in football play selection. Behavior Analysis: Research and Practice, 15, 112–121.

    Google Scholar 

  • Critchfield, T. S., Tucker, J. A., & Vuchinich, R. E. (1998). Self-report methods. In K. A. Lattal & M. Perone (Eds.), Handbook of methods for the experimental analysis of human behavior (pp. 435–470). New York: Plenum.

    Chapter  Google Scholar 

  • Curry, B., Foxall, G. R., & Sigurdsson, V. (2010). On the tautology of the matching law in consumer behavior. Behavioural Processes, 84, 390–399.

    Article  PubMed  Google Scholar 

  • Davison, M., & McCarthy, D. (1988). The matching law: A research review. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Dodds, P. S., Clark, E. M., Desu, S., Frank, M. R., Reagan, A. J., Williams, J. R., … & Megerdoomian, K. (2015). Human language reveals a universal positivity bias. Proceedings of the National Academy of Sciences, 112, 2389-2394. doi: 10.1073/pnas.1411678112

  • Doughty, A. H., Holloway, C., Shields, M. C., & Kennedy, L. E. (2012). Marketing behavior analysis requires (really) different talk: A critique of Kohn (2005) and a(nother) call to arms. Behavior and Social Issues, 21, 115–134. doi:10.5210/bsi.v21i0.3914.

    Article  Google Scholar 

  • Dymond, S., & Rehfeldt, R. A. (2000). Understanding complex behavior: The transformation of stimulus function. The Behavior Analyst, 23, 239–254.

    PubMed  PubMed Central  Google Scholar 

  • Epting, L. K., & Critchfield, T. S. (2006). Self-editing: On the relation between behavioral and psycholinguistic approaches. The Behavior Analyst, 29(2), 211–234.

    Google Scholar 

  • Floh, A., Koller, M., & Zauner, A. (2013). Taking a deeper look at online reviews: Asymmetric effect of valence intensity on shopping behavior. Journal of Marketing Management, 29, 646–670. doi:10.1080/0267257x.2013.776620.

    Article  Google Scholar 

  • Foxx, R. M. (1996). Translating the covenant: The behavior analyst as ambassador and translator. The Behavior Analyst, 19, 147–161.

    PubMed  PubMed Central  Google Scholar 

  • Friedman, H. H., & Amoo, T. (1999). Rating the rating scales. Journal of Marketing Management, 9, 114–123.

    Google Scholar 

  • Guerini, M., Pepe, A., & Lepri, B. (2012). Do linguistic style and readability of scientific abstracts affect their virality? Download from http://arxiv.org/pdf/1203.4238.pdf.

  • Hart, B., & Risley, T. R. (1995). Meaningful differences in the everyday experience of young American children. Baltimore: Brookes.

    Google Scholar 

  • Johnston, J. M., & Pennypacker, H. S. (1980). Strategies and tactics of human behavior research. Nillsdale, NJ: Erlbaum.

    Google Scholar 

  • Kloumann, I. M., Danforth, C. M., Harris, K. D., Bliss, C. A., & Dodds, P. S. (2012). Positivity of the English language. PLOS One, 7, e29484. doi:10.1371/journal.pone.0029484.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kuperman, V., Estes, Z., Brysbaert, M., & Warriner, A. B. (2014). Emotion and language: Valence and arousal affect word recognition. Journal of Experimental Psychology: General, 143, 1065–1081.

    Article  Google Scholar 

  • Li, Y., & Yu, D. (2015). Development of emotion word comprehension in Chinese children from 2 to 13 years old: Relationships with valence and empathy. PLoS ONE, 10(12), e0143712. doi:10.1371/journal.pone.0143712.

    Article  PubMed  PubMed Central  Google Scholar 

  • Lindsley, O. R. (1991). From technical jargon to plain English for application. Journal of Applied Behavior Analysis, 24, 449–458. doi:10.1901/jaba.1991.24-449.

    Article  PubMed  PubMed Central  Google Scholar 

  • Mace, F. C., & Critchfield, T. S. (2010). Translational research in behavior analysis: Historical traditions and imperative for the future. Journal of the Experimental Analysis of Behavior, 93, 293–312.

    Article  PubMed  PubMed Central  Google Scholar 

  • Mace, F. C., Lalli, J. S., Shea, M. C., & Nevin, J. A. (1992). Behavioral momentum in college basketball. Journal of Applied Behavior Analysis, 25, 657–663.

    Article  PubMed  PubMed Central  Google Scholar 

  • Maurice, C. (1993). Let me hear your voice: a family's triumph over autism. New York: Random House.

    Google Scholar 

  • McDowell, J. J., & Caron, M. L. (2010). Bias and undermatching in delinquent boys' verbal behavior as a function of their level of deviance. Journal of the Experimental Analysis of Behavior, 93, 471–483.

    Article  PubMed  PubMed Central  Google Scholar 

  • Moerk, E. L. (1990). Three-term contingency patterns in mother-child verbal interactions during first-language acquisition. Journal of the Experimental Analysis of Behavior, 54, 293–305.

    Article  PubMed  PubMed Central  Google Scholar 

  • Norman, S. M., Avolio, B. J., & Luthans, F. (2010). The impact of positivity and transparency on trust in leaders and their perceived effectiveness. The Leadership Quarterly, 21, 350–364. doi:10.1016/j.leaqua.2010.03.002.

    Article  Google Scholar 

  • Paolacci, G., & Chandler, J. (2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23, 184–188. doi:10.1177/0963721414531598.

    Article  Google Scholar 

  • Petty, R. E., Schumann, D. W., Richman, S. S., & Strathman, A. J. (1993). Positive mood and persuasion: Different roles for affect under high- and low-elaboration conditions. Journal of Personality and Social Psychology, 64, 5–20. doi:10.1037/0022-3514.64.1.5.

    Article  Google Scholar 

  • Poling, A. (2010). Looking to the future: Will behavior analysis survive and prosper? The Behavior Analyst, 33, 7–17.

    PubMed  PubMed Central  Google Scholar 

  • Reed, D. D., Critchfield, T. S., & Martens, B. K. (2006). Operant choice in elite sport competition: The matching law and play-calling in professional football. Journal of Applied Behavior Analysis., 39, 281–297.

    Article  PubMed  PubMed Central  Google Scholar 

  • Reed, D. D., Partington, S. W., Kaplan, B. A., Roma, P. G., & Hursh, S. R. (2013). Behavioral economic analysis of demand for fuel in North America. Journal of Applied Behavior Analysis, 46, 651–655.

    Article  PubMed  Google Scholar 

  • Ridgeway, D., Waters, E., & Kuczaj, S. A., II. (1985). Acquisition of emotion-descriptive language: Receptive and productive vocabulary norms for ages 18 months to 6 years. Developmental Psychology, 21, 901–908.

    Article  Google Scholar 

  • Riegel, M., Wierzba, M., Wypych, M., Zurawski, L., Jednorog, K., Grabowska, A., & Marchewka, A. (2015). Nencki Affective Word List (NAWL): cultural adaptation of the Berlin Affective Word List-Reloaded (BAWL-R) for Polish. Behavior Research Methods, 47, 1222–1236. doi:10.3758/s13428-014-0552-1.

    Article  PubMed  PubMed Central  Google Scholar 

  • Rozin, P., & Royzman, E. B. (2001). Negativity bias, negativity dominance, and contagion. Personality and Social Psychology Review, 5, 296–320.

    Article  Google Scholar 

  • Sautter, R. A., & LeBlanc, L. A. (2006). Empirical applications of Skinner's analysis of verbal behavior with humans. The Analysis of Verbal Behavior, 22, 35–48.

    PubMed  PubMed Central  Google Scholar 

  • Skinner, B. F. (1937). The distribution of associated words. The Psychological Record, 1, 71–76.

    Google Scholar 

  • Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. New York: Appleton-Century.

    Google Scholar 

  • Skinner, B. F. (1953). Science and human behavior. New York, NY: Macmillan.

    Google Scholar 

  • Skinner, B. F. (1957). Verbal behavior. New York, NY: Appleton-Century-Crofts.

    Book  Google Scholar 

  • Stadthagen-Gonzalez, H., Imbault, C., Perez Sanchez, M.A., & Brysbaert, M. (2016). Norms of valence and arousal for 14,031 Spanish words. Behavior Research Methods. Online prepublication; downloaded from http://link.springer.com/article/10.3758%2Fs13428-015-0700-2. doi: 10.3758/s13428-015-0700-2

  • Staats, A. W., & Staats, C. K. (1958). Attitudes established by classical conditioning. Journal of Abnormal and Social Psychology, 57, 37.

    Article  Google Scholar 

  • Suskind, D. (2015). Thirty million words: Building a child's brain. New York: Dutton.

    Google Scholar 

  • Vo, M. L.-H., Conrad, M., Kuckinke, L., Urton, K., Hofmann, M. J., & Jacobs, A. M. (2009). The Berlin Affective Word List Reloaded (BAWL-R). Behavior Research Methods, 41, 534–538.

    Article  PubMed  Google Scholar 

  • Warriner, A. B., & Kuperman, V. (2015). Affective biases in English are bi-dimensional. Cognition and Emotion, 29, 1147–1167.

    Article  PubMed  Google Scholar 

  • Warriner, A. B., Kuperman, V., & Brysbaert, M. (2013). Norms of valence, arousal, and dominance for 13,915 English lemmas. Behavior Research Methods, 45, 1–17. doi:10.3758/s13428-012-0314-x.

    Article  Google Scholar 

  • Weisberg, P., & Waldrop, P. B. (1972). Fixed-interval work habits of Congress. Journal of Applied Behavior Analysis, 5, 93–97.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203–214.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Illinois State University, Normal, IL, 61761, USA

    Thomas S. Critchfield

  2. University of Kansas, Lawrence, KS, USA

    Amel Becirevic & Derek D. Reed

Authors
  1. Thomas S. Critchfield
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amel Becirevic
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Derek D. Reed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas S. Critchfield.

Ethics declarations

Funding

This work was not supported by any source of external funding.

Conflict of Interest

The authors declare no conflicts of interest relevant to this work.

Ethical Approval

Institutional Review Board oversight is not relevant to this work because it employed archival data sets that are in the public domain.

Informed Consent

Informed consent is not relevant to this work because it employed archival data sets that are in the public domain.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Critchfield, T.S., Becirevic, A. & Reed, D.D. In Skinner's Early Footsteps: Analyzing Verbal Behavior in Large Published Corpora. Psychol Rec 66, 639–647 (2016). https://doi.org/10.1007/s40732-016-0197-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40732-016-0197-9

Keywords

Search

Navigation