Abstract
As a part of ongoing perception, the human cognitive system segments others’ activities into discrete episodes (event segmentation). Although prior research has shown that this process is likely related to changes in an actor’s actions and goals, it has not yet been determined whether untrained observers can reliably identify action and goal changes as naturalistic activities unfold, or whether the changes they identify are tied to visual features of the activity (e.g., the beginnings and ends of object interactions). This study addressed these questions by examining untrained participants’ identification of action changes, goal changes, and event boundaries while watching videos of everyday activities that were presented in both first-person and third-person perspectives. We found that untrained observers can identify goal changes and action changes consistently, and these changes are not explained by visual change and the onsets or offsets of contact with objects. Moreover, the action and goal changes identified by untrained observers were associated with event boundaries, even after accounting for objective visual features of the videos. These findings suggest that people can identify action and goal changes consistently and with high agreement, that they do so by using sensory information flexibly, and that the action and goal changes they identify may contribute to event segmentation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bach, P., Nicholson, T., & Hudson, M. (2014). The affordance-matching hypothesis: How objects guide action understanding and prediction. Frontiers in Human Neuroscience, 8. https://doi.org/10.3389/fnhum.2014.00254
Bach, P., & Schenke, K. C. (2017). Predictive social perception: Towards a unifying framework from action observation to person knowledge. Social and Personality Psychology Compass, 11(7), e12312. https://doi.org/10.1111/spc3.12312
Bailey, H. R., Kurby, C. A., Giovannetti, T., & Zacks, J. M. (2013). Action perception predicts action performance. Neuropsychologia, 51(11), 2294–2304. https://doi.org/10.1016/j.neuropsychologia.2013.06.022
Baker, C. L., Saxe, R., & Tenenbaum, J. B. (2009). Action understanding as inverse planning. Cognition, 113(3), 329–349. https://doi.org/10.1016/j.cognition.2009.07.005
Baldassano, C., Chen, J., Zadbood, A., Pillow, J. W., Hasson, U., & Norman, K. A. (2017). Discovering Event Structure in Continuous Narrative Perception and Memory. Neuron, 95(3), 709-721.e5. https://doi.org/10.1016/j.neuron.2017.06.041
Baldwin, D., Andersson, A., Saffran, J., & Meyer, M. (2008). Segmenting dynamic human action via statistical structure. Cognition, 106(3), 1382–1407. https://doi.org/10.1016/j.cognition.2007.07.005
Baldwin, D., Baird, J. A., Saylor, M. M., & Clark, M. A. (2001). Infants parse dynamic action. Child Development, 72(3), 708–717. https://doi.org/10.1111/1467-8624.00310
Barrett, L. F., & Satpute, A. B. (2013). Large-scale brain networks in affective and social neuroscience: Towards an integrative functional architecture of the brain. Current Opinion in Neurobiology, 23(3), 361–372. https://doi.org/10.1016/j.conb.2012.12.012
Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59(1), 617–645. https://doi.org/10.1146/annurev.psych.59.103006.093639
Bläsing, B. E. (2015). Segmentation of dance movement: Effects of expertise, visual familiarity, motor experience and music. Frontiers in Psychology, 5. https://doi.org/10.3389/fpsyg.2014.01500
Blakemore, S.-J., & Decety, J. (2001). From the perception of action to the understanding of intention. Nature Reviews Neuroscience, 2(8), 561–567. https://doi.org/10.1038/35086023
Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10(4), 433–436. https://doi.org/10.1163/156856897X00357
Buchsbaum, D., Griffiths, T. L., Plunkett, D., Gopnik, A., & Baldwin, D. (2015). Inferring action structure and causal relationships in continuous sequences of human action. Cognitive Psychology, 76, 30–77. https://doi.org/10.1016/j.cogpsych.2014.10.001
Catmur, C. (2015). Understanding intentions from actions: Direct perception, inference, and the roles of mirror and mentalizing systems. Consciousness and Cognition, 36, 426–433. https://doi.org/10.1016/j.concog.2015.03.012
Cutting, J. E. (2014). Event segmentation and seven types of narrative discontinuity in popular movies. Acta Psychologica, 149, 69–77. https://doi.org/10.1016/j.actpsy.2014.03.003
Decroix, J., Roger, C., & Kalénine, S. (2020). Neural dynamics of grip and goal integration during the processing of others’ actions with objects: An ERP study. Scientific Reports, 10(1), 5065. https://doi.org/10.1038/s41598-020-61963-7
El-Sourani, N., Wurm, M. F., Trempler, I., Fink, G. R., & Schubotz, R. I. (2018). Making sense of objects lying around: How contextual objects shape brain activity during action observation. NeuroImage, 167, 429–437. https://doi.org/10.1016/j.neuroimage.2017.11.047
Ezzyat, Y., & Davachi, L. (2011). What Constitutes an Episode in Episodic Memory? Psychological Science, 22(2), 243–252. https://doi.org/10.1177/0956797610393742
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146
Fogassi, L., Ferrari, P. F., Gesierich, B., Rozzi, S., Chersi, F., & Rizzolatti, G. (2005). Parietal Lobe: From Action Organization to Intention Understanding. Science, 308(5722), 662–667. https://doi.org/10.1126/science.1106138
Friend, M., & Pace, A. (2011). Beyond event segmentation: Spatial- and social-cognitive processes in verb-to-action mapping. Developmental Psychology, 47(3), 867–876. https://doi.org/10.1037/a0021107
Gallese, V., & Goldman, A. (1998). Mirror neurons and the simulation theory of mind-reading. Trends in Cognitive Sciences, 2(12), 493–501. https://doi.org/10.1016/S1364-6613(98)01262-5
Hafri, A., Papafragou, A., & Trueswell, J. C. (2013). Getting the gist of events: Recognition of two-participant actions from brief displays. Journal of Experimental Psychology: General, 142(3), 880–905. https://doi.org/10.1037/a0030045
Hamilton, A. F. D. C., & Grafton, S. (2007). The motor hierarchy: From kinematics to goals and intentions. In P. Haggard, Y. Rosetti, & M. Kawato (Eds.), Attention and Performance (xxii ed.). Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199231447.001.0001
Hard, B. M., Recchia, G., & Tversky, B. (2011). The shape of action. Journal of Experimental Psychology: General, 140(4), 586–604. https://doi.org/10.1037/a0024310
Hard, B. M., Tversky, B., & Lang, D. S. (2006). Making sense of abstract events: Building event schemas. Memory & cognition, 34(6), 1221–1235.
Heyes, C., & Catmur, C. (2022). What Happened to Mirror Neurons? Perspectives on Psychological Science, 17(1), 153–168. https://doi.org/10.1177/1745691621990638
Holm, S. (1979). A Simple Sequentially Rejective Multiple Test Procedure. Scandinavian Journal of Statistics, 6(2), 65–70.
Hudson, M., Nicholson, T., Ellis, R., & Bach, P. (2016). I see what you say: Prior knowledge of other’s goals automatically biases the perception of their actions. Cognition, 146, 245–250. https://doi.org/10.1016/j.cognition.2015.09.021
Kopatich, R. D., Feller, D. P., Kurby, C. A., & Magliano, J. P. (2019). The role of character goals and changes in body position in the processing of events in visual narratives. Cognitive Research: Principles and Implications, 4(1), 22. https://doi.org/10.1186/s41235-019-0176-1
Kosie, J. E., & Baldwin, D. A. (2018). Tuning to the Task at Hand: Processing Goals Shape Adults’ Attention to Unfolding Activity. In CogSci.
Koul, A., Cavallo, A., Ansuini, C., & Becchio, C. (2016). Doing It Your Way: How Individual Movement Styles Affect Action Prediction. PLOS ONE, 11(10), e0165297. https://doi.org/10.1371/journal.pone.0165297
Kuperberg, G. R. (2021). Tea With Milk? A Hierarchical Generative Framework of Sequential Event Comprehension. Topics in Cognitive Science, 13(1), 256–298. https://doi.org/10.1111/tops.12518
Kurby, C. A., & Zacks, J. M. (2008). Segmentation in the perception and memory of events. Trends in Cognitive Sciences, 12(2), 72–79. https://doi.org/10.1016/j.tics.2007.11.004
Kurby, C. A., & Zacks, J. M. (2011). Age differences in the perception of hierarchical structure in events. Memory & Cognition, 39(1), 75–91. https://doi.org/10.3758/s13421-010-0027-2
Kurby, C. A., & Zacks, J. M. (2022). Priming of movie content is modulated by event boundaries. Journal of Experimental Psychology: Learning, Memory, and Cognition, 48(11), 1559–1570. https://doi.org/10.1037/xlm0001085
Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). lmerTest Package: Tests in linear mixed effects models. Journal of Statistical Software, 82(13). https://doi.org/10.18637/jss.v082.i13
Lenth, R. V. (2023). emmeans: Estimated Marginal Means, aka Least-Squares Means (Version 1.9.0) [Computer software]. https://CRAN.R-project.org/package=emmeans
Levine, D., Hirsh-Pasek, K., Pace, A., & MichnickGolinkoff, R. (2017). A goal bias in action: The boundaries adults perceive in events align with sites of actor intent. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43(6), 916–927. https://doi.org/10.1037/xlm0000364
Libby, L. K., Shaeffer, E. M., & Eibach, R. P. (2009). Seeing meaning in action: A bidirectional link between visual perspective and action identification level. Journal of Experimental Psychology: General, 138(4), 503–516. https://doi.org/10.1037/a0016795
Magliano, J. P., Todaro, S., Millis, K., Wiemer-Hastings, K., Kim, H. J., & McNamara, D. S. (2005). Changes in Reading Strategies as a Function of Reading Training: A Comparison of Live and Computerized Training. Journal of Educational Computing Research, 32(2), 185–208. https://doi.org/10.2190/1LN8-7BQE-8TN0-M91L
Mukamel, R., Ekstrom, A. D., Kaplan, J., Iacoboni, M., & Fried, I. (2010). Single-Neuron Responses in Humans during Execution and Observation of Actions. Current Biology, 20(8), 750–756. https://doi.org/10.1016/j.cub.2010.02.045
Naish, K. R., Reader, A. T., Houston-Price, C., Bremner, A. J., & Holmes, N. P. (2013). To eat or not to eat? Kinematics and muscle activity of reach-to-grasp movements are influenced by the action goal, but observers do not detect these differences. Experimental Brain Research, 225(2), 261–275. https://doi.org/10.1007/s00221-012-3367-2
Nakagawa, S., & Schielzeth, H. (2013). A general and simple method for obtaining R 2 from generalized linear mixed-effects models. Methods in Ecology and Evolution, 4(2), 133–142. https://doi.org/10.1111/j.2041-210x.2012.00261.x
Newberry, K. M., Feller, D. P., & Bailey, H. R. (2021). Influences of domain knowledge on segmentation and memory. Memory & Cognition, 49(4), 660–674. https://doi.org/10.3758/s13421-020-01118-1
Newtson, D. (1980). An Interactionist Perspective on Social Knowing. Personality and Social Psychology Bulletin, 6(4), 520–531. https://doi.org/10.1177/014616728064004
Newtson, D., Engquist, G., & Bois, J. (1977). The objective basis of behavior units. Journal of Personality and Social Psychology, 35(12), 847–862.
Oberpriller, J., De Souza Leite, M., & Pichler, M. (2022). Fixed or random? On the reliability of mixed‐effects models for a small number of levels in grouping variables. Ecology and Evolution, 12(7). https://doi.org/10.1002/ece3.9062
Olofson, E. L., & Baldwin, D. (2011). Infants recognize similar goals across dissimilar actions involving object manipulation. Cognition, 118(2), 258–264. https://doi.org/10.1016/j.cognition.2010.11.012
R Core Team. (2021). R: A Language and Environment for Statistical Computing [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org/
Richmond, L. L., & Zacks, J. M. (2017). Constructing Experience: Event Models from Perception to Action. Trends in Cognitive Sciences, 21(12), 962–980. https://doi.org/10.1016/j.tics.2017.08.005
Sasmita, K., & Swallow, K. M. (2022). Measuring event segmentation: An investigation into the stability of event boundary agreement across groups. Behavior Research Methods, 55(1), 428–447. https://doi.org/10.3758/s13428-022-01832-5
Smith, M. A., & Anderson, B. D. (2004). A Window on Reality? Journal of Clinical Oncology, 22(8), 1360–1362. https://doi.org/10.1200/JCO.2004.01.946
Speer, N. K., Swallow, K. M., & Zacks, J. M. (2003). Activation of human motion processing areas during event perception. Cognitive, Affective, & Behavioral Neuroscience, 3(4), 335–345. https://doi.org/10.3758/CABN.3.4.335
Speer, N. K., & Zacks, J. M. (2005). Temporal changes as event boundaries: Processing and memory consequences of narrative time shifts☆. Journal of Memory and Language, 53(1), 125–140. https://doi.org/10.1016/j.jml.2005.02.009
Spunt, R. P., Falk, E. B., & Lieberman, M. D. (2010). Dissociable Neural Systems Support Retrieval of How and Why Action Knowledge. Psychological Science, 21(11), 1593–1598. https://doi.org/10.1177/0956797610386618
Swallow, K. M., Kemp, J. T., & Candan Simsek, A. (2018). The role of perspective in event segmentation. Cognition, 177, 249–262. https://doi.org/10.1016/j.cognition.2018.04.019
Swallow, K. M., & Wang, Q. (2020). Culture influences how people divide continuous sensory experience into events. Cognition, 205, 104450. https://doi.org/10.1016/j.cognition.2020.104450
Swallow, K. M., Zacks, J. M., & Abrams, R. A. (2009). Event boundaries in perception affect memory encoding and updating. Journal of Experimental Psychology: General, 138(2), 236–257. https://doi.org/10.1037/a0015631
Ullman, T., Baker, C., Macindoe, O., Evans, O., Goodman, N., & Tenenbaum, J. (2009). Help or Hinder: Bayesian Models of Social Goal Inference. Advances in Neural Information Processing Systems, 22. https://proceedings.neurips.cc/paper_files/paper/2009/hash/52292e0c763fd027c6eba6b8f494d2eb-Abstract.html
Vallacher, R. R., & Wegner, D. M. (1987). What do people think they’re doing? Action identification and human behavior. Psychological Review, 94(1), 3–15. https://doi.org/10.1037/0033-295X.94.1.3
Vallacher, R. R., & Wegner, D. M. (1989). Levels of personal agency: Individual variation in action identification. Journal of Personality and Social Psychology, 57(4), 660–671. https://doi.org/10.1037/0022-3514.57.4.660
Woodward, A. L., & Sommerville, J. A. (2000). Twelve-Month-Old Infants Interpret Action in Context. Psychological Science, 11(1), 73–77. https://doi.org/10.1111/1467-9280.00218
Woodworth, R. S. (1899). Accuracy of voluntary movement. The Psychological Review: Monograph Supplements, 3(3), i–114. https://doi.org/10.1037/h0092992
Wurm, M. F., & Lingnau, A. (2015). Decoding Actions at Different Levels of Abstraction. The Journal of Neuroscience, 35(20), 7727–7735. https://doi.org/10.1523/JNEUROSCI.0188-15.2015
Zacks, J. M. (2004). Using movement and intentions to understand simple events. Cognitive Science, 28(6), 979–1008. https://doi.org/10.1207/s15516709cog2806_5
Zacks, J. M., & Tversky, B. (2001). Event structure in perception and conception. Psychological Bulletin, 127(1), 3–21. https://doi.org/10.1037/0033-2909.127.1.3
Zacks, J. M., Braver, T. S., Sheridan, M. A., Donaldson, D. I., Snyder, A. Z., Ollinger, J. M., Buckner, R. L., & Raichle, M. E. (2001). Human brain activity time-locked to perceptual event boundaries. Nature Neuroscience, 4(6), 651–655. https://doi.org/10.1038/88486
Zacks, J. M., Tversky, B., & Iyer, G. (2001). Perceiving, remembering, and communicating structure in events. Journal of Experimental Psychology: General, 130(1), 29–58. https://doi.org/10.1037/0096-3445.130.1.29
Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S., & Reynolds, J. R. (2007). Event perception: A mind-brain perspective. Psychological Bulletin, 133(2), 273–293. https://doi.org/10.1037/0033-2909.133.2.273
Zacks, J. M., Speer, N. K., & Reynolds, J. R. (2009). Segmentation in reading and film comprehension. Journal of Experimental Psychology: General, 138(2), 307–327. https://doi.org/10.1037/a0015305
Zacks, J. M., Speer, Nicole K., Swallow, Khena M., & Maley, Corey J. (2010). The brain’s cutting-room floor: Segmentation of narrative cinema. Frontiers in Human Neuroscience, 4. https://doi.org/10.3389/fnhum.2010.00168
Zacks, J. M., Kurby, C. A., Eisenberg, M. L., & Haroutunian, N. (2011). Prediction Error Associated with the Perceptual Segmentation of Naturalistic Events. Journal of Cognitive Neuroscience, 23(12), 4057–4066. https://doi.org/10.1162/jocn_a_00078
Zheng, Y., Zacks, J. M., & Markson, L. (2020). The development of event perception and memory. Cognitive Development, 54, 100848. https://doi.org/10.1016/j.cogdev.2020.100848
Ziaeetabar, F., Pomp, J., Pfeiffer, S., El-Sourani, N., Schubotz, R. I., Tamosiunaite, M., & Wörgötter, F. (2020). Using enriched semantic event chains to model human action prediction based on (minimal) spatial information. PLOS ONE, 15(12), e0243829. https://doi.org/10.1371/journal.pone.0243829
Acknowledgements
The authors would like to thank Michael Goldstein for his input on an earlier version of this manuscript and Joel Cruz for help with data collection. This work was funded by the Cornell University College of Arts and Sciences.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Practices Statement
The data and materials for this experiment will be made available upon reasonable request (participants did not consent to having their data made publicly available). Statistical analyses were performed using publicly available tools, including programs available here: https://github.com/ksasmita/esMethods. The experiment was not pre-registered.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Su, X., Swallow, K.M. People can reliably detect action changes and goal changes during naturalistic perception. Mem Cogn (2024). https://doi.org/10.3758/s13421-024-01525-8
Accepted:
Published:
DOI: https://doi.org/10.3758/s13421-024-01525-8