Visual Non-verbal Social Cues Data Modeling

  • Mahmoud QodseyaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11158)


Although many methods have been developed in social signal processing (SSP) field during the last decade, several issues related to data management and scalability are still emerging. As the existing visual non-verbal behavior analysis (VNBA) systems are task-oriented, they do not have comprehensive data models, and they are biased towards particular data acquisition procedures, social cues and analysis methods. In this paper, we propose a data model for the visual non-verbal cues. The proposed model is privacy-preserving in the sense that it grants decoupling social cues extraction phase from analysis one. Furthermore, this decoupling allows to evaluate and perform different combinations of extraction and analysis methods. Apart from the decoupling, our model can facilitate heterogeneous data fusion from different modalities since it facilitates the retrieval of any combination of different modalities and provides deep insight into the relationships among the VNBA systems components.


Social signal processing VNBA systems Metadata modeling Visual non-verbal cues 


  1. 1.
    Akhtar, Z., Falk, T.: Visual nonverbal behavior analysis: the path forward. In: IEEE MultiMedia (2017)Google Scholar
  2. 2.
    Vinciarelli, A., et al.: Bridging the gap between social animal and unsocial machine: a survey of social signal processing. IEEE Trans. Affect. Comput. 3, 69–87 (2012)CrossRefGoogle Scholar
  3. 3.
    Cristani, M., Raghavendra, R., Del Bue, A., Murino, V.: Human behavior analysis in video surveillance: a social signal processing perspective. Neurocomputing 100, 86–97 (2013)CrossRefGoogle Scholar
  4. 4.
    Salah, A.A., Pantic, M., Vinciarelli, A.: Recent developments in social signal processing. In: 2011 IEEE International Conference on Systems, Man, and Cybernetics, pp. 380–385, October 2011Google Scholar
  5. 5.
    Kukhun, D.A., Codreanu, D., Manzat, A.-M., Sedes, F.: Towards a pervasive access control within video surveillance systems. In: Cuzzocrea, A., Kittl, C., Simos, D.E., Weippl, E., Xu, L. (eds.) CD-ARES 2013. LNCS, vol. 8127, pp. 289–303. Springer, Heidelberg (2013). Scholar
  6. 6.
    Baltrusaitis, T., Zadeh, A., Lim, Y.C., Morency, L.P.: Openface 2.0: facial behavior analysis toolkit. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 59–66 (2018)Google Scholar
  7. 7.
    Redmon, J., Farhadi, A.: Yolov3: An incremental improvement. arXiv (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Institut de Recherche en Informatique de Toulouse (IRIT), Université de ToulouseToulouseFrance

Personalised recommendations