Multimedia Systems

, Volume 16, Issue 4–5, pp 293–307 | Cite as

Asynchronous reflections: theory and practice in the design of multimedia mirror systems

Regular Paper


In this paper, we present a theoretical framing of the functions of a mirror by breaking the synchrony between the state of a reference object and its reflection. This framing provides a new conceptualization of the uses of reflections for various applications. We describe the fundamental technical components of such systems and illustrate the technical challenges in two different forms of electronic mirror systems for apparel shopping. The first example, the Responsive Mirror, is an intelligent video capture and access system for clothes shopping in physical stores that provides personalized asynchronous reflections of clothing items through an implicitly controlled human–computer interface. The Responsive Mirror employs computer vision and machine learning techniques to interpret the visual cues of the shopper’s behavior from cameras to then display two different reflections of the shopper on digital displays: (1) the shopper in previously worn clothing with matching pose and orientation and (2) other people in similar and dissimilar shirts with matching pose and orientation. The second example system is a Countertop Responsive Mirror that differs from the first in that the images do not respond to the real-time movement of the shopper but to frames in a recorded video so that the motion of the shopper in the different recordings are matched non-sequentially. These instantiations of the mirror systems in fitting room and jewelry shopping scenarios are described, focusing on the system architecture and the intelligent computer vision components. The effectiveness of Responsive Mirror is demonstrated by the user study. The paper contributes a conceptualization of reflection and examples of systems illustrating new applications in multimedia systems that break traditional reflective synchronies.


Pervasive computing Intelligent user interface Multimedia system Asynchronous reflection Personalized media content Computer vision Machine learning Responsive Mirror Apparel shopping 


  1. 1.
    Altman, I.: The environment and social behavior: privacy, personal space, territory and crowding. Brooks/Cole Publishing Company, Monterey (1975)Google Scholar
  2. 2.
    Anguelov, D., Lee K., Gokturk, S.B., Sumengen, B.: Contextual identity recognition in personal photo albums. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–7 (2007)Google Scholar
  3. 3.
    Begole, B., Matsumoto, T., Zhang, W., Liu, J.: Responsive mirror: fitting information for fitting rooms. In: Proceedings of Workshop on Ambient Persuasion at Computer/Human Interaction (CHI) Conference (2008)Google Scholar
  4. 4.
    Begole, B., Matsumoto, T., Zhang, W., Yee, N., Liu, J., Chu, M.: Designed to fit: challenges of interaction design for clothes fitting room technologies. In: HCI International, LNCS, vol. 5613, pp. 448–457. Springer, Berlin (2009)Google Scholar
  5. 5.
    Brown, J.: Prada gets personal. BusinessWeek. McGraw-Hill, USA (2002)Google Scholar
  6. 6.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern. Anal. Mach. Intell. 8(6), 679–698 (1986)CrossRefGoogle Scholar
  7. 7.
    Christopher, J.C.B.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Disc. 2, 121–167 (1998)CrossRefGoogle Scholar
  8. 8.
    Chu, M., Dalal, B., Walendowski, A., Begole, B.: Countertop responsive mirror: supporting physical retail shopping for sellers, buyers and companions. Submitted to Computer/Human Interaction (CHI) Conference (2010)Google Scholar
  9. 9.
    Davis, F.: Fashion, Culture and Identity. University Of Chicago Press, USA (1994)Google Scholar
  10. 10.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, New York (2001). ISBN: 0-471-05669-3MATHGoogle Scholar
  11. 11.
    Fujinami, K., Kawsar, F., Nakajima, T.: AwareMirror: a personalized display using a mirror. In: Proceedings of Pervasive, pp. 315–332 (2005)Google Scholar
  12. 12.
    Gallagher, A.C., Chen, T.: Using context to recognize people in consumer images. IPSJ Trans. Comput. Vis. Appl. 1, 115–126 (2009)CrossRefGoogle Scholar
  13. 13.
    Haritaoglu, I., Flickner, M.: Attentive billboards. In: Proceedings of International Conference on Image Analysis and Processing (ICIAP), pp. 162–167 (2001)Google Scholar
  14. 14.
    Harris, C., Stephens, M.: A combined corner and edge detector. In: Proceedings of the 4th Alvey Vision Conference, pp. 147–151 (1988)Google Scholar
  15. 15.
    Iwabuchi, E., Nakagawa, M., Siio, I.: Smart makeup mirror: computer-augmented mirror to aid makeup application. HCI International in LNCS, vol. 5613, pp. 495–503. Springer, Berlin (2009)Google Scholar
  16. 16.
    Kienzle, W., Bakir, G., Franz, M., Scholkopf, B.: Face detection – efficient and rank deficient. In: Proceedings of Advances in Neural Information Processing Systems (NIPS), pp. 673–680 (2005)Google Scholar
  17. 17.
    Lee, D., Park, J., Lee, M., Hahn, M.: Personalized magic mirror: interactive mirror based on user behavior. In: Proceedings of 5th International Conference on Smart Homes and Health Telematics (ICOST), Nara, Japan, pp. 162–169 (2007)Google Scholar
  18. 18.
    Lepetit, V., Fua, P.: Monocular model-based 3D tracking of rigid objects: a survey. Foundations and Trends®. Comput. Graphics Vis. 1(1) (2005)Google Scholar
  19. 19. Last accessed 5 January 2010
  20. 20.
    Lindsay, G.: Prada’s High-Tech Misstep. Business 2.0, 1 (2004)
  21. 21.
    Nanda, S.: Virtual Mirrors. Reuters (2007).
  22. 22.
    RockYou: Last accessed 5 January 2010
  23. 23.
    Roussel, N., Evans, H., Hansen, H.: Proximity as an interface for video communication. Multimedia IEEE 11(3), 12–16 (2004)CrossRefGoogle Scholar
  24. 24.
    Rozin, D.: Wooden mirror. IEEE Spectr. 38, 3 (2001)Google Scholar
  25. 25.
    Sivic, J., Zitnick, C.L., Szeliski, R.: Finding people in repeated shot of the same scene. In: Proceedings of 16th British Machine Vision Conference (BMVC), pp. 909–918 (2006)Google Scholar
  26. 26.
    Song, Y., Leung, T.: Context-aided human recognition – clustering. In: Proceedings of 9th European Conference on Computer Vision (ECCV) 3954, pp. 382–395 (2006)Google Scholar
  27. 27.
    Tran, Q., Mynatt, E.: Cook’s collage: two exploratory designs. In: Proceedings of Computer/Human Interaction (CHI) Conference (2002)Google Scholar
  28. 28.
    Truong, K., Abowd, G., Brotherton, J.: Who, what, when, where, how: design issues of capture & access applications. In: Proceedings of International Conference on Ubiquitous Computing (Ubicomp), pp. 209–224. ACM Press, New York (2001)Google Scholar
  29. 29.
    Zhang, L., Chen, L., Li, M., Zhang, H.: Automated annotation of human faces in family albums. In: Proceedings of ACM Multimedia (MM), pp. 355–358 (2003)Google Scholar
  30. 30.
    Zhang, W., Matsumoto, T., Liu, J., Chu, M., Begole, B.: An intelligent fitting room using multi-camera perception. In: Proceedings of International Conference on Intelligent User Interfaces (IUI), pp. 60–69 (2008)Google Scholar
  31. 31.
    Zhang, W., Begole, B., Chu, M., Liu, J., Yee, N.: Real-time clothes comparison based on multi-view vision. In: Proceedings of ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC), 7–11 Sep (2008)Google Scholar
  32. 32.
    Zhao, M., Teo, Y.W., Liu, S., Chua, T., Jain, R.: Automatic person annotation of family photo album. In: Proceedings of ACM International Conference on Image and Video Retrieval (CIVR), pp. 163–172 (2006)Google Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  1. 1.Like.comSan MateoUSA
  2. 2.Palo Alto Research Center, Inc. (PARC)Palo AltoUSA

Personalised recommendations