Comparing Reinforcement Values of Facial Expressions: An Eye-Tracking Study

The Psychological Record volume 69, pages393–400(2019)Cite this article

Abstract

Reinforcement values of facial expressions for visual fixation remain underdeveloped in extant literature, though numerous studies have investigated the function of facial expressions as antecedents of gaze behavior. This study used a concurrent gaze-contingent reinforcement procedure to compare reinforcement values of positive and negative facial expressions for gaze fixation. In total, 7 of the 20 typical adults fixated more frequently on the side that was followed by positive facial expressions. The results suggest that a novel application of eye tracking technology could allow for the evaluation and comparison of reinforcement values of social visual stimuli.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

References

  1. Anderson, B. A., & Yantis, S. (2013). Persistence of value-driven attentional capture. Journal of Experimental Psychology: Human Perception and Performance, 39, 6–9.

    PubMed  Google Scholar 

  2. Anderson, B. A., Laurent, P. A., & Yantis, S. (2013). Reward prediction bias attentional selection. Frontiers in Human Neuroscience, 7, 262.

    PubMed  PubMed Central  Google Scholar 

  3. Averbeck, B. B., & Duchaine, B. (2009). Integration of social and utilitarian factors in decision making. Emotion, 9, 599–608.

    Article  PubMed  Google Scholar 

  4. Behrens, T. E. J., Hunt, L. T., Woolrich, M. W., & Rushworth, M. F. (2008). Associative learning of social value. Nature, 456, 245–249.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Blair, R. J. R. (2003). Facial expressions, their communicatory functions and neuro-cognitive substrates. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 358, 561–572.

    Article  PubMed  Google Scholar 

  6. Call, N. A., Shillingsburg, M. A., Bowen, C. N., Reavis, A. R., & Findley, A. J. (2013). Direct assessment of preferences for social interactions in children with autism. Journal of Applied Behavior Analysis, 46, 821–826.

    Article  Google Scholar 

  7. Cannella-Malone, H. I., Sabielny, L., & Tullis, C. A. (2015). Using eye gaze to identify reinforcers for individuals with severe multiple disabilities. Journal of Applied Behavior Analysis, 48, 680–684.

    Article  PubMed  Google Scholar 

  8. Constantino, J. N., Kennon-McGill, S., Weichselbaum, C., Marrus, N., Haider, A., Glowinski, A. L., et al. (2017). Infant viewing of social scenes is under genetic control and is atypical in autism. Nature, 547, 340–344.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Dube, W. V., Dickson, C. A., Balsamo, L. M., O’Donnell, K. L., Tomanari, G. Y., Farren, K. M., . . . McIlvane, W. J. (2010). Observing behavior and atypically restricted stimulus control. Journal of the Experimental Analysis of Behavior, 94, 297–313.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Eisenbarth, H., & Alpers, G. W. (2011). Happy mouth and sad eyes: Scanning emotional facial expressions. Emotion, 11, 860–865.

    Article  PubMed  Google Scholar 

  11. Falck-Ytter, T., Bölte, S., & Gredebäck, G. (2013). Eye tracking in early autism research. Journal of neurodevelopmental disorders, 5(1), 28. https://doi.org/10.1186/1866-1955-5-28.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Fantz, R. L. (1965). Visual perception from birth as shown by pattern selectivity. Annals of the New York Academy of Sciences, 118, 793–814.

    Article  PubMed  Google Scholar 

  13. Feinman, S., & Lewis, M. (1983). Social referencing at ten months: A second-order effect on infants' responses to strangers. Child Development, 54, 878–887.

    PubMed  Google Scholar 

  14. Ferster, C. B., & Skinner, B. F. (1957). Schedules of reinforcement. New York, NY: Appleton-Century-Crofts.

    Google Scholar 

  15. Fisher, W., Piazza, C. C., Bowman, L. G., Hagopian, L. P., Owens, J. C., & Slevin, I. (1992). A comparison of two approaches for identifying reinforcers for persons with severe and profound disabilities. Journal of Applied Behavior Analysis, 25, 491–498.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Fleming, C. V., Wheeler, G. M., Cannella-Malone, H. I., Basbagill, A. R., Chung, Y. C., & Day, K. G. (2010). An evaluation of the use of eye gaze to measure preference of individuals with severe physical and developmental disabilities. Developmental Neurorehabilitation, 13, 266–275.

    Article  PubMed  Google Scholar 

  17. Frank, M. C., Vul, E., & Johnson, S. P. (2009). Development of infants’ attention to faces during first year. Cognition, 110, 160–170.

    Article  PubMed  Google Scholar 

  18. Furl, N., Gallagher, S., & Averbeck, B. B. (2012). A selective emotional decision-making bias elicited by facial expressions. PLoS ONE, 7, e33461.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Gliga, T., Elsabbagh, M., Andravizou, A., & Johnson, M. (2009). Faces attract infants’ attention in complex displays. Infancy, 14, 550–562.

    Article  Google Scholar 

  20. Heerey, E. A. (2014). Learning from social rewards predicts individual differences in self-reported social ability. Journal of Experimental Psychology: General, 143, 332–339.

    Article  Google Scholar 

  21. Hernstein, R. J. (1961). Relative and absolute strength of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4, 267–272.

    Article  Google Scholar 

  22. Jones, W., & Klin, A. (2013). Attention to eyes is present but in decline in 2–6-month-old infants later diagnosed with autism. Nature, 504, 427–431.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Kelly, M. A., Roscoe, E. M., Hanley, G. P., & Schlichenmeyer, K. (2014). Evaluation of assessment methods for identifying social reinforcers. Journal of Applied Behavior Analysis, 47, 113–135.

    Article  PubMed  Google Scholar 

  24. Klin, A., Jones, W., Schultz, R., Volkmar, F., & Cohen, D. (2002). Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry, 59, 809–816.

    Article  Google Scholar 

  25. Kringelbach, M., & Rolls, E. T. (2003). Neural correlates of rapid reversal learning in a simple model of human social interaction. NeuroImage, 20, 1371–1383.

    Article  PubMed  Google Scholar 

  26. LaBarbera, J. D., Izard, C. E., Vietze, P., & Parisi, S. A. (1976). Four- and six-month-old infants’ visual responses to joy, anger, and neutral expressions. Child Development, 47, 535–538.

    Article  PubMed  Google Scholar 

  27. Laidlaw, K. E. W., Foulsham, T., Kuhn, G., & Kingstone, A. (2011). Potential social interactions are important to social attention. Proceedings of the National Academy of Sciences of the United States of America, 108, 5548–5553.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Martin, T. L., Yu, C. T., Martin, G. L., & Fazzio, D. (2006). On choice, preference, and preference for choice. Behavior Analyst Today, 7, 234–241.

    Article  PubMed  Google Scholar 

  29. Matsuda, S., Minagawa, Y., & Yamamoto, J. (2015). Gaze behavior of children with ASD toward pictures of facial expressions. Autism Research & Treatment, 2015. https://doi.org/10.1115/2015/617190.

  30. Mazur, J. E. (1991). Choice. In H. Iverson & K. A. Lattal (Eds.), Experimental analysis of behavior (pp. 219–250). New York, NY: Elsevier.

    Google Scholar 

  31. Paeye, C., & Madelain, L. (2011). Reinforcing saccadic amplitude variability. Journal of the Experimental Analysis of Behavior, 95, 149–162.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Paeye, C., Schütz, A. C., & Gegenfurther, K. R. (2016). Visual reinforcement shapes eye movements in visual search. Journal of Vision, 16, 15.

    Article  PubMed  Google Scholar 

  33. Pelaez, M., Virues-Ortega, J., & Gewirtz, J. L. (2012). Acquisition of social referencing via discrimination training in infants. Journal of Applied Behavior Analysis, 45, 23–36.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Perez, W. F., Endemann, P., Pessôa, C. V. B. B., & Tomanari, G. Y. (2015). Assessing stimulus control in a discrimination task with compound stimuli: Evaluating testing procedures and tracking eye fixations. The Psychological Record, 65, 83–88.

    Article  Google Scholar 

  35. Pierce, K., Marinero, S., Hazin, R., McKenna, B., Barnes, C. C., & Malige, A. (2016). Eye tracking reveals abnormal visual preference for geometric images as an early biomarker of an autism spectrum disorder subtype associated with increased symptom severity. Biological Psychiatry, 79, 657–666.

    Article  PubMed  Google Scholar 

  36. Pourtois, G., Grandjean, D., Sander, D., & Vuilleumier, P. (2004). Electrophysiological correlates of rapid spatial orienting towards fearful faces. Cerebral Cortex, 14, 619–633.

    Article  PubMed  Google Scholar 

  37. Rosenberger, P. B. (1973). Concurrent schedule control of human visual target fixations. Journal of the Experimental Analysis of Behavior, 20, 411–416.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Schrandt, J. A., Townsend, D. B., & Poulson, C. L. (2009). Teaching empathy skills to children with autism. Journal of Applied Behavior Analysis, 42, 17–32.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Schroeder, S. R., & Holland, J. G. (1968a). Operant control of eye movements during human vigilance. Science, 161, 292–293.

    Article  PubMed  Google Scholar 

  40. Schroeder, S. R., & Holland, J. G. (1968b). Operant control of eye movements. Journal of Applied Behavior Analysis, 1, 161–166.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Schroeder, S. R., & Holland, J. G. (1969). Reinforcement of eye movement with concurrent schedules. Journal of the Experimental Analysis of Behavior, 12, 897–903.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Steingrimsdottir, H. S., & Arntzen, E. (2016). Eye movements during conditional discrimination training. The Psychological Record, 66, 201–212.

    Article  Google Scholar 

  43. Teller, D. Y. (1979). The forced-choice preferential looking procedure: A psychophysical technique for use with human infants. Infant Behavior & Development, 2, 135–153.

    Article  Google Scholar 

  44. Vernetti, A., Senju, A., Charman, T., Johnson, M. H., Gliga, T., & Team, B. A. S. I. S. (2018). Stimulating interaction: Using gaze-contingent eye-tracking to measure the reward value of social signals in toddlers with and without autism. Developmental Cognitive Neuroscience, 29, 21–29.

    Article  PubMed  Google Scholar 

  45. Vernetti, A., Smith, T. J., & Senju, A. (2017). Gaze-contingent reinforcement learning reveals incentive value of social signals in young children adults. Proceedings of the Royal Society of London B: Biological Sciences, 284, 20162747.

    Article  Google Scholar 

  46. Wang, Q., Bolhuis, J., Rothkopf, C. A., Kolling, T., Knoph, M., & Triesch, J. (2012). Infants in control: Rapid anticipation of action outcomes in a gaze-contingent paradigm. PLoS ONE, 7, e30884.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Windsor, J., Piché, L. M., & Locke, P. A. (1994). Preference testing: A comparison of two presentation methods. Research in Developmental Disabilities, 15, 439–455.

    Article  PubMed  Google Scholar 

  48. de Wit, T. C. J., Falck-Ytter, T., & von Hofsten, C. (2008). Young children with autism spectrum disorder look differently at positive versus negative emotional faces. Research in Autism Spectrum Disorders, 2, 651–659.

    Article  Google Scholar 

  49. Zahn-Waxler, C., Robinson, J. L., & Emde, R. N. (1992). The development of empathy in twins. Developmental Psychology, 28, 1038–1047.

    Article  Google Scholar 

Download references

Funding

This study was funded by Grant-in-Aid for Scientific Research on Innovative Areas No. 18H04182 “Construction of the Face-Body Studies in Transcultural Conditions” to S. M., and the CREST research project on Social Imaging (JPMJCR14E2), Japan Science and Technology Agency to J. Y.

Author information

Affiliations

  1. Faculty of Human Sciences, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, 305-8573, Japan

    Soichiro Matsuda

  2. Department of Psychology, Keio University, Tokyo, Japan

    Takahide Omori & Junichi Yamamoto

  3. Department of Psychology and Kinney Cefignter for Autism Education and Support, Saint Joseph’s University, Philadelphia, PA, USA

    Joseph P. McCleery

  4. Center for Autism Research, Children’s Hospital of Philadelphia, Philadelphia, PA, USA

    Joseph P. McCleery

Authors
  1. Soichiro Matsuda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Takahide Omori
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joseph P. McCleery
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Junichi Yamamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Soichiro Matsuda.

Ethics declarations

Conflict of Interest

All authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table 3 Mean (SD) number of reinforcements obtained for each model
Full size table

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Matsuda, S., Omori, T., McCleery, J.P. et al. Comparing Reinforcement Values of Facial Expressions: An Eye-Tracking Study. Psychol Rec 69, 393–400 (2019). https://doi.org/10.1007/s40732-019-00330-z

Download citation

Keywords

  • Gaze fixation
  • Social reinforcers
  • Facial expressions
  • Eye tracking
  • Smile