Online Aggregated-Event Representation for Multiple Event Detection in Videos

  • Molefe Vicky Mleya
  • Weiqi Li
  • Jiayu Liang
  • Kunliang Liu
  • Yunkuan Sun
  • Guanghao Jin
  • Jianming WangEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11888)


Event detection is used to locate the frames corresponding to events of interest in given videos. Real-world videos contain multiple events of interest, and they are rarely segmented. Existing online methods can only detect segments containing single event instances, and this is not suitable for processing videos with several event instances. There are multiple event detection methods, but they are all relatively inefficient and offline methods. To handle the online detection of several events, we propose a novel framework with three modules that are: the event proposal generation, aggregated-event representation, and refined detection modules. The first module can locate time intervals that are likely to contain target events, termed as proposals. The second module can aggregate all events before the current time to form a temporal context that will be used to generate initial detection results of multiple events. The refined detection module finally refines the results based on event proposals and object detection. The proposed method achieves a detection accuracy of 24.88% on a multi-event dataset - Charades, which is higher than state-of-the-art methods.


Multiple event detection Online event detection Aggregated-event representation 


  1. 1.
    Buch, S., Escorcia, V., Shen, C., Ghanem, B., Carlos Niebles, J.: SST: single-stream temporal action proposals. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017Google Scholar
  2. 2.
    Carreira, J., Zisserman, A.: Quo vadis, action recognition? A new model and the kinetics dataset. CoRR abs/1705.07750 (2017).
  3. 3.
    Du, T., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3D convolutional networks. In: IEEE International Conference on Computer Vision, pp. 4489–4497 (2015)Google Scholar
  4. 4.
    Gao, J., Yang, Z., Nevatia, R.: Cascaded boundary regression for temporal action detection. CoRR abs/1705.01180 (2017).
  5. 5.
    De Geest, R., Gavves, E., Ghodrati, A., Li, Z., Snoek, C., Tuytelaars, T.: Online action detection. CoRR abs/1604.06506 (2016).
  6. 6.
    Ghosh, P., Yao, Y., Davis, L.S., Divakaran, A.: Stacked spatio-temporal graph convolutional networks for action segmentation. CoRR abs/1811.10575 (2018).
  7. 7.
    Lin, T., et al.: Microsoft COCO: common objects in context. CoRR abs/1405.0312 (2014).
  8. 8.
    Piergiovanni, A.J., Ryoo, M.S.: Activity detection with latent sub-event hierarchy learning. CoRR abs/1803.06316 (2018).
  9. 9.
    Piergiovanni, A., Ryoo, M.S.: Learning latent super-events to detect multiple activities in videos. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018Google Scholar
  10. 10.
    Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. CoRR abs/1506.01497 (2015).
  11. 11.
    Shou, Z., et al.: Online detection of action start in untrimmed, streaming videos. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 551–568. Springer, Cham (2018). Scholar
  12. 12.
    Sigurdsson, G.A., Divvala, S., Farhadi, A., Gupta, A.: Asynchronous temporal fields for action recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017Google Scholar
  13. 13.
    Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. CoRR abs/1406.2199 (2014).
  14. 14.
    Tang, K., Yao, B., Fei-Fei, L., Koller, D.: Combining the right features for complex event recognition. In: The IEEE International Conference on Computer Vision (ICCV), December 2013Google Scholar
  15. 15.
    Xu, H., Das, A., Saenko, K.: R-C3D: region convolutional 3D network for temporal activity detection. CoRR abs/1703.07814 (2017).
  16. 16.
    Yeung, S., Russakovsky, O., Ning, J., Andriluka, M., Mori, G., Li, F.F.: Every moment counts: dense detailed labeling of actions in complex videos. Int. J. Comput. Vis. 126(2–4), 1–15 (2018)MathSciNetGoogle Scholar
  17. 17.
    Zhao, Y., Xiong, Y., Wang, L., Wu, Z., Lin, D., Tang, X.: Temporal action detection with structured segment networks. CoRR abs/1704.06228 (2017).

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Molefe Vicky Mleya
    • 1
  • Weiqi Li
    • 1
  • Jiayu Liang
    • 1
  • Kunliang Liu
    • 1
  • Yunkuan Sun
    • 2
  • Guanghao Jin
    • 1
  • Jianming Wang
    • 1
    Email author
  1. 1.School of Computer Science and TechnologyTianjin Polytechnic UniversityTianjinChina
  2. 2.School of Electronics and Information EngineeringTianjin Polytechnic UniversityTianjinChina

Personalised recommendations