Detection of Clothes Change Fusing Color, Texture, Edge and Depth Information

  • Dimitrios SgouropoulosEmail author
  • Theodoros Giannakopoulos
  • Giorgos Siantikos
  • Evaggelos Spyrou
  • Stavros Perantonis
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 554)


Changing clothes is a basic activity of daily living (ADL) which may be used as a measurement of the functional status of e.g. an elderly person, or a person with certain disabilities. In this paper we propose a methodology for the detection of when a human has changed clothes. Our non-contact unobtrusive monitoring system is built upon the Microsoft Kinect depth camera. It uses the OpenNI SDK to detect a human skeleton and extract the upper and lower clothes’ visual features. Color, texture and edge descriptors are then extracted and fused. We evaluate our system on a publicly available set of real recordings for several users and under various illumination conditions. Our results show that our system is able to successfully detect when a user changes clothes, thus to assess the quality of the corresponding ADL.


Clothes’ change ADL Kinect OpenNI Data fusion 



The research leading to these results has received funding from the European Union’s Seventh Framework Programme (FP7/2007-2013) under grant agreement no 288532. For more details, please see


  1. 1.
    Microsoft kinect sensor (2011). Accessed 1 April 2013
  2. 2.
    Cushen, G.A., Nixon, M.S.: Real-time semantic clothing segmentation. In: Bebis, G., et al. (eds.) ISVC 2012, Part I. LNCS, vol. 7431, pp. 272–281. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  3. 3.
    Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006) CrossRefGoogle Scholar
  4. 4.
    Bossard, L., Dantone, M., Leistner, C., Wengert, C., Quack, T., Van Gool, L.: Apparel classification with style. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012, Part IV. LNCS, vol. 7727, pp. 321–335. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  5. 5.
    Chen, H., Gallagher, A., Girod, B.: Describing clothing by semantic attributes. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 609–623. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  6. 6.
    Collin, C., Wade, D.: The barthel adl index: a standard measure of physical disability? Disabil. Rehabil. 10(2), 64–67 (1988)Google Scholar
  7. 7.
    Collin, C., Wade, D., Davies, S., Horne, V.: The barthel adl index: a reliability study. Disabil. Rehabil. 10(2), 61–63 (1988)Google Scholar
  8. 8.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2005, CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)Google Scholar
  9. 9.
    Fleury, A., Vacher, M., Noury, N.: SVM-based multimodal classification of activities of daily living in health smart homes: sensors, algorithms, and first experimental results. IEEE Trans. Inf. Technol. Biomed. 14(2), 274–283 (2010)CrossRefGoogle Scholar
  10. 10.
    Kalantidis, Y., Kennedy, L., Li, L.J.: Getting the look: clothing recognition and segmentation for automatic product suggestions in everyday photos. In: Proceedings of the 3rd Conference on International Conference on Multimedia Retrieval, pp. 105–112. ACM (2013)Google Scholar
  11. 11.
    Liu, S., Feng, J., Song, Z., Zhang, T., Lu, H., Xu, C., Yan, S.: Hi, magic closet, tell me what to wear! In: Proceedings of the 20th International Conference on Multimedia, pp. 619–628. ACM (2012)Google Scholar
  12. 12.
    Liu, S., Song, Z., Liu, G., Xu, C., Lu, H., Yan, S.: Street-to-shop: cross-scenario clothing retrieval via parts alignment and auxiliary set. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3330–3337. IEEE (2012)Google Scholar
  13. 13.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  14. 14.
    Maitin-Shepard, J., Cusumano-Towner, M., Lei, J., Abbeel, P.: Cloth grasp point detection based on multiple-view geometric cues with application to robotic towel folding. In: 2010 IEEE International Conference on Robotics and Automation (ICRA), pp. 2308–2315. IEEE (2010)Google Scholar
  15. 15.
    Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recogn. 29(1), 51–59 (1996)CrossRefGoogle Scholar
  16. 16.
    Ramisa, A., Alenya, G., Moreno-Noguer, F., Torras, C.: Using depth and appearance features for informed robot grasping of highly wrinkled clothes. In: International Conference on Robotics and Automation, pp. 1703–1708. IEEE (2012)Google Scholar
  17. 17.
    Self-maintenance, P.: Assessment of older people: self-maintaining and instrumental activities of daily living (1969)Google Scholar
  18. 18.
    Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., Moore, R.: Real-time human pose recognition in parts from single depth images. Commun. ACM 56(1), 116–124 (2013)CrossRefGoogle Scholar
  19. 19.
    Spyrou, E., Le Borgne, H., Mailis, T., Cooke, E., Avrithis, Y., O’Connor, N.E.: Fusing MPEG-7 visual descriptors for image classification. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 847–852. Springer, Heidelberg (2005) Google Scholar
  20. 20.
    Stikic, M., Huynh, T., Laerhoven, K.V., Schiele, B.: ADL recognition based on the combination of RFID and accelerometer sensing. In: Pervasive Computing Technologies for Healthcare, 2008, pp. 258–263. IEEE (2008)Google Scholar
  21. 21.
    Wang, X., Han, T.X., Yan, S.: An HOG-LBP human detector with partial occlusion handling. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 32–39. IEEE (2009)Google Scholar
  22. 22.
    Willimon, B., Birchfleld, S., Walker, I.: Classification of clothing using interactive perception. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 1862–1868. IEEE (2011)Google Scholar
  23. 23.
    Willimon, B., Walker, I., Birchfield, S.: A new approach to clothing classification using mid-level layers. In: Proceedings of the International Conference on Robotics and Automation (ICRA) (2013)Google Scholar
  24. 24.
    Xia, L., Chen, C.C., Aggarwal, J.: Human detection using depth information by kinect. In: 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 15–22. IEEE (2011)Google Scholar
  25. 25.
    Zhang, Z.: Microsoft kinect sensor and its effect. IEEE Multimedia 19(2), 4–10 (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Dimitrios Sgouropoulos
    • 1
    Email author
  • Theodoros Giannakopoulos
    • 1
  • Giorgos Siantikos
    • 1
  • Evaggelos Spyrou
    • 1
  • Stavros Perantonis
    • 1
  1. 1.Computational Intelligence Laboratory (CIL), Institute of Informatics and TelecommunicationsNational Center for Scientific Research–DEMOKRITOSAgia Paraskevi, AthensGreece

Personalised recommendations