The idea that we were going to have to do research to prove that reinforcement works you know, that didn't seem plausible anymore, [it was] like gravity." (Jack Michael, reflecting on the 1950s founding of applied behavior analysis; quoted in Rutherford, 2009, p. 59)

Every science demands faith in fundamental principles. Also required, of course, is the careful elaboration of those principles. In the quotation above, Jack Michael’s commitment to behavior principles is appealing -- but also audacious. Is reinforcement really “like gravity?” As every high school student knows, integral to physics is an understanding of how gravity is modulated by a variety of specific factors. Indeed, physics succeeds as a discipline partly because it has a good handle on the conditions under which gravity manifests, and to what degree. Of present interest is the extent to which Michael’s gravity/reinforcement analogy portrays what can be called reinforcer theory, which addresses the question of when a given stimulus will function as a reinforcer.Footnote 1 As with many stories involving behavior science, the present one begins with B.F. Skinner, whose early work established the normative approach to understanding reinforcers.

A Defining Concept

In a single sentence in "Two types of conditioned reflex and a pseudo type," Skinner (1935) first introduced the now-familiar theme that consequences are defined by their effects on behavior:

Stimuli may be divided into two classes, positively and negatively conditioning, according to whether they produce an increase or decrease when used as reinforcement" (unpaginated electronic reprint; see https://psychcentral.com/classics/Skinner/Twotypes/twotypes.htm).

The same point, scarcely better developed, appeared briefly in Behavior of Organisms. After indicating that reinforcement is evident in “changes in strength” of a response, Skinner (1938) noted that:

A reinforcing stimulus is defined... by its power to produce the resulting change. There is no circularity about this; some stimuli are found to produce the change, others not, and they are classified as reinforcing and non-reinforcing accordingly. (p. 62)

Skinner (1953) later elaborated:

The only way to tell whether or not a given event is reinforcing to a given organism under given conditions is to make a direct test. We observe the frequency of a selected response, then make an event contingent upon it and observe any change in frequency. If there is a change, we classify the event as reinforcing to the organism under the existing conditions. There is nothing circular about classifying events in terms of their effects; the criterion is both empirical and objective. It would be circular, however, if we then went on to assert that a given event strengthens an operant because it is reinforcing. (pp. 72-73)

In essence, what Skinner asserted is that reinforcers are what reinforcers do, and one can only make this determination after the fact, once an event has followed behavior and the behavior has measurably accelerated. This is, of course, a statement of what, not of why (and in science “why” really means “when”). Thus, a theory of reinforcers would provide a framework for predicting the range of conditions under which any given stimulus functions as a reinforcer. Skinner never sought such a framework.

As Skinner’s appeals to circularity attest, he was concerned that his account would be perceived as a traditional theory (explanation) when in fact he advanced a means of labeling certain kinds of functional relations. Here it is important to remember that Skinner was much influenced by advocates of pragmatic, inductive, descriptive approaches such as Mach and Wittgenstein (Day, 1969; Marr, 2003; Moxley, 2005), and his account of reinforcers was in keeping with those influences.Footnote 2 As Skinner (1938) wrote in, and of, Behavior of Organisms:

So far as the scientific method is concerned, the system... may be characterized as follows. It is positivistic [in the Machian sense]. It confines itself to description rather than explanation. Its concepts are defined in terms of immediate observations and are not given local or physiological properties.... Terms... are used merely to bring together groups of observations, to state uniformities, and to express properties of behavior which transcend single instances. They are not hypotheses, in the sense of things to be proved or disproved, but convenient representations of things already known. As to hypotheses, the system does not require them -- at least in the usual sense. (p. 44).

Thus, Skinner saw general behavior theory (which subsumes everything that matters in the study of behavioral processes) as systematizing observationally-validated functional relations (Skinner, 1950). In eschewing the development of a formal reinforcer theory he (perhaps) remained true to this conception.

Benefits of the Descriptive Approach

Skinner’s descriptive approach set the stage for positive developments in behavior science. First and foremost, a science needs facts, and to assemble facts one needs standardized operations to employ in investigation. When Skinner (1935, 1938) developed his pragmatic perspective on reinforcers, facts were in short supply, and it is never wise to build theory faster than the data sets for which it is supposed to account. In the 1930s excessive ruminating about the “why” of reinforcement was not just premature but also a potential distraction from the hard work of accumulating facts.

Skinner (e.g., 1938; 1953; 1972) knew that when external influences over behavior are poorly understood, people tend to flirt with essentialism or speculate about unmeasured internal states. His descriptive approach therefore served as a hedge against the development, within behavior analysis, of theories in which reinforcers are things with transituational behavior-strengthening properties (see Meehl, 1950) or in which reinforcers results from to vaguely-defined needs or drives (McDougall, 1923; Hull, 1943). Arguably, what has been accomplished in the last eight decades of research based on Skinner’s functional account of reinforcers compares favorably to the track record of those types of reinforcer theories.

Second, from fairly early on, Skinner anticipated that a functional definition would prove useful in cutting through the ineffectual approaches to everyday behavior that humans are so prone to generating. He correctly inferred that identifying effective reinforcers would be a recurring challenge in field settings:

A survey of the events which reinforce a given individual is often required in the practical application of operant conditioning. In every field in which human behavior figures prominently -- education, government, the family, the clinic, industry, art, literature, and so on -- we are constantly changing probabilities of response by arranging reinforcing consequences.... The information [provided by a reinforcer survey] may be valuable if it is for any reason necessary to reinforce or extinguish our subject's behavior. (Skinner, 1953, pp. 73-74)

Skinner maintained that, in the field as in the laboratory, the only reliable guide to what functions as a reinforcer is the behavior that it does or does not control. It should therefore be clear that a profound conceptual debt is owed Skinner for the later development of analog reinforcer assessment procedures and functional analysis methodology, both of which have had a major impact on the growth and proliferation of applied behavior analysis (e.g., Dixon, Vogel, & Tarbox, 2013; Tullis et al., 2011). In this way, reinforcers are what reinforcers do has proven to be a most useful heuristic during the first half century of applied behavior analysis.

Why Not a Theory of Reinforcers?

Skinner’s descriptive approach to reinforcers emphasizes functional relations, and nothing in nature or science precludes conditional functional relations (e.g., Marr, 2003). Indeed, all functional relations, even those we call gravity, are conditional. If a reinforcer is a stimulus that strengthens behavior, one can remain strictly Skinnerian in asking about the conditions under which that relation holds to greater or lesser degrees.

Why this question has not received more attention in the science that Skinner set into motion is unclear, but it was Skinner himself who pointed out that the behavior of scientists follows the same general laws as any other kind of behavior. In inductive science, “some people are lucky” (Skinner, 1956, p. 225), and Skinner’s functional approach to reinforcers quickly led to the identification of a few types of outcomes for a few species (e.g., mixed grain for pigeons) that functioned robustly as reinforcers under a few circumstances (e.g., food deprivation). Because "some ways of doing research are easier than others" (Skinner, 1956, p. 224), those few events, species, and conditions became standardized in the practices of most operant laboratories. Thereafter, certain aspects of behavior theory evolved at breakneck pace as more and more was learned about what happens assuming that a given stimulus functions as a reinforcer. And because it is fruitful "when you run into something interesting [to] drop everything else and study it" (Skinner, 1956, p. 223), such early achievements may have overshadowed the need to pursue a predictive reinforcer theory. In short, perhaps reinforcer theories have received little attention, not because they are a bad idea, but rather because the behavior of inductive scientists has been shaped in other directions.

And so, almost 80 years after the publication of Behavior of Organisms, there is no generally accepted account of when a stimulus will function as a reinforcer. Although there have been encouraging efforts to create predictive frameworks, some of these accounts are not yet well developed, and most have received limited attention outside of a small group of theorist-investigators. There has been very little comparative research assessing the relative validity of competing reinforcer theories. Ultimately, the irony of the Jack Michael quote that began this essay is that although Skinner-defined reinforcers share with gravity the property of ubiquity (both are evident everywhere), in a more sophisticated sense they could not be more different. Gravity can be predicted. Reinforcers, not so much.

The irony is magnified when considering the applied realm that has profited so much from Skinner’s approach. On the one hand, post-hoc identification of reinforcers is the basis for reinforcer assessment procedures that help to make successful interventions possible (Tullis et al., 2011). On the other hand, imagine the benefits if a predictive framework freed clinicians from the need to conduct reinforcer assessments in the first place, allowing extra time and effort to be directed into life-altering interventions.

In order to advance the contemporary discussion of reinforcer theory, The Behavior Analyst presents a target article by Killeen and Jacobs (2016 and this issue) plus commentaries by a panel of distinguished scientists. In taking up the problem of when a stimulus can be expected to function as a reinforcer, the target article’s most important contribution may be its most general: the potential to disrupt deeply entrenched thinking about reinforcers by asking us to view them in ways that, to many readers, will be unfamiliar. For example, Killeen and Jacobs suggest that a predictive account might be achieved by attending to certain kinds of behavioral dispositions and “internal states.” Perhaps not all readers will embrace their proposal that such variables can be reconciled with the philosophy of radical behaviorism, but we predict that the process of carefully considering the underlying arguments will yield valuable food for thought.

The commentaries provide a valuable case in point. Authors were asked, not to critique Killeen and Jacobs (2016) per se, but to draw on their diverse areas of expertise to enrich the context in which a reader might evaluate the target article’s arguments. In various ways they clarify the context in which a theory of behavior strengthening, as per Skinner's (1938) early conception of reinforcement, must be developed. The final commentary, by Shahan, turns the problem of reinforcer theory on its head by questioning what it means to speak of strengthening in the first place, and whether doing so really constitutes a logically-defensible standard for defining reinforcers.

Frames of reference brought to bear by the target article and commentaries include, but are not limited to: disequilibrium theory, motivating operations, stimulus relations and relational frame theory, habituation, emotions, ecological theory, an evolutionary theory, embodied cognition, behavioral momentum theory, selectionism, and information theory. To our knowledge, no other contemporary source brings together, for consumption by a general audience, such a diverse set of views on reinforcer theory. The end product is not exactly a coherent whole, but it identifies a variety of threads that must somehow be woven neatly together before we can think about reinforcers in the same way that we think about gravity.