Person-Centric Multimedia: How Research Inspirations from Designing Solutions for Individual Users Benefits the Broader Population

  • Sethuraman Panchanathan
  • Ramin Tadayon
  • Hemanth VenkateswaraEmail author
  • Troy McDaniel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11010)


While Human-Centered Multimedia Computing (HCMC) improves upon traditional multimedia computing paradigms by accounting for the differences among populations of humans, inter-personal differences between, and intra-personal differences within, populations have created the need for a new paradigm which is sensitive to the needs of a specific user, task and environment. The paradigm of Person-Centered Multimedia Computing (PCMC) addresses this challenge by focusing the design of a system on a single user and challenge, shifting the focus to the individual. It is proposed that this paradigm can then extend the applicability of multimedia technology from the individual user to the broader population through the application of adaptation and integration. These concepts are discussed within the context of disability, where variations among individuals are particularly prevalent. Examples in domain adaptation and autonomous rehabilitative training are presented as proofs-of-concept to illustrate this process within PCMC.


Person-centric computing Coadaptive design Human-computer interaction Domain adaptation 



The authors thank Arizona State University and National Science Foundation for their funding support. This material is partially based upon work supported by the National Science Foundation under Grant Nos. 1069125 and 1116360.


  1. 1.
    Afergan, D., et al.: Dynamic difficulty using brain metrics of workload. In: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, pp. 3797–3806. ACM (2014)Google Scholar
  2. 2.
    Bala, S., McDaniel, T., Panchanathan, S.: Visual-to-tactile mapping of facial movements for enriched social interactions. In: 2014 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE), pp. 82–87. IEEE (2014)Google Scholar
  3. 3.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15(6), 1373–1396 (2003)CrossRefGoogle Scholar
  4. 4.
    Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. In: BMVC (2014)Google Scholar
  5. 5.
    Chen, J.: Flow in games (and everything else). Commun. ACM 50(4), 31–34 (2007)CrossRefGoogle Scholar
  6. 6.
    Cooley, M.: Human-centered design. In: Information Design, pp. 59–81 (2000)Google Scholar
  7. 7.
    Craig, S.D., D’Mello, S., Witherspoon, A., Graesser, A.: Emote aloud during learning with autotutor: applying the facial action coding system to cognitive-affective states during learning. Cogn. Emot. 22(5), 777–788 (2008)CrossRefGoogle Scholar
  8. 8.
    Ekman, P., Rosenberg, E.L.: What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS). Oxford University Press, Oxford (1997)Google Scholar
  9. 9.
    Fernandez-Cervantes, V., Stroulia, E., Oliva, L.E., Gonzalez, F., Castillo, C.: Serious games: rehabilitation fuzzy grammar for exercise and therapy compliance. In: 2015 IEEE Games Entertainment Media Conference (GEM), pp. 1–8. IEEE (2015)Google Scholar
  10. 10.
    Fernando, B., Habrard, A., Sebban, M., Tuytelaars, T.: Unsupervised visual domain adaptation using subspace alignment. In: CVPR, pp. 2960–2967 (2013)Google Scholar
  11. 11.
    Fugl-Meyer, A.R., Jääskö, L., Leyman, I., Olsson, S., Steglind, S.: The post-stroke hemiplegic patient. 1. A method for evaluation of physical performance. Scand. J. Rehabil. Med. 7(1), 13–31 (1975)Google Scholar
  12. 12.
    Gallina, P., Bellotto, N., Di Luca, M.: Progressive co-adaptation in human-machine interaction. In: 2015 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO), vol. 2, pp. 362–368. IEEE (2015)Google Scholar
  13. 13.
    Gong, B., Shi, Y., Sha, F., Grauman, K.: Geodesic flow kernel for unsupervised domain adaptation. In: IEEE CVPR (2012)Google Scholar
  14. 14.
    Gretton, A., Smola, A., Huang, J., Schmittfull, M., Borgwardt, K., Schölkopf, B.: Covariate shift by kernel mean matching. In: Dataset Shift in Machine Learning, vol. 3, no. 4, p. 5 (2009)Google Scholar
  15. 15.
    Hunicke, R.: The case for dynamic difficulty adjustment in games. In: Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, pp. 429–433. ACM (2005)Google Scholar
  16. 16.
    Jaimes, A., Sebe, N., Gatica-Perez, D.: Human-centered computing: a multimedia perspective. In: Proceedings of the 14th ACM International Conference on Multimedia, pp. 855–864. ACM (2006)Google Scholar
  17. 17.
    Jewitt, C., Bezemer, J., O’Halloran, K.: Introducing Multimodality. Routledge, Abingdon (2016)Google Scholar
  18. 18.
    Jorritsma, W., Cnossen, F., van Ooijen, P.M.: Adaptive support for user interface customization: a study in radiology. Int. J. Hum.-Comput. Stud. 77, 1–9 (2015)CrossRefGoogle Scholar
  19. 19.
    Long, M., Wang, J., Ding, G., Sun, J., Yu, P.: Transfer joint matching for unsupervised domain adaptation. In: CVPR, pp. 1410–1417 (2014)Google Scholar
  20. 20.
    Long, M., Wang, J., Ding, G., Sun, J., Yu, P.S.: Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2200–2207 (2013)Google Scholar
  21. 21.
    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: CVPR, pp. 94–101. IEEE (2010)Google Scholar
  22. 22.
    Mislevy, R.J., Haertel, G., Riconscente, M., Rutstein, D.W., Ziker, C.: Evidence-centered assessment design. In: Mislevy, R.J., Haertel, G., Riconscente, M., Rutstein, D.W. (eds.) Assessing Model-Based Reasoning using Evidence- Centered Design. SS, pp. 19–24. Springer, Cham (2017). Scholar
  23. 23.
    Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22(2), 199–210 (2011)CrossRefGoogle Scholar
  24. 24.
    Panchanathan, S., Chakraborty, S., McDaniel, T., Tadayon, R.: Person-centered multimedia computing: a new paradigm inspired by assistive and rehabilitative applications. IEEE MultiMedia 23(3), 12–19 (2016)CrossRefGoogle Scholar
  25. 25.
    Pantic, M., Valstar, M., Rademaker, R., Maat, L.: Web-based database for facial expression analysis. In: ICME. IEEE (2005)Google Scholar
  26. 26.
    Rauschecker, J.P.: Compensatory plasticity and sensory substitution in the cerebral cortex. Trends Neurosci. 18(1), 36–43 (1995)CrossRefGoogle Scholar
  27. 27.
    Shaughnessy, M., Resnick, B.M., Macko, R.F.: Testing a model of post-stroke exercise behavior. Rehabil. Nurs. 31(1), 15–21 (2006)CrossRefGoogle Scholar
  28. 28.
    Shute, V.J., Kim, Y.J.: Formative and stealth assessment. In: Spector, J.M., Merrill, M.D., Elen, J., Bishop, M.J. (eds.) Handbook of Research on Educational Communications and Technology, pp. 311–321. Springer, New York (2014). Scholar
  29. 29.
    Smith, D., et al.: Remedial therapy after stroke: a randomised controlled trial. Br. Med. J. (Clin. Res. Ed.) 282(6263), 517–520 (1981)CrossRefGoogle Scholar
  30. 30.
    Stephanidis, C.: User interfaces for all: new perspectives into human-computer interaction. In: User Interfaces for All-Concepts, Methods, and Tools, vol. 1, pp. 3–17 (2001)Google Scholar
  31. 31.
    Sun, B., Feng, J., Saenko, K.: Return of frustratingly easy domain adaptation. In: ICCV, TASK-CV (2015)Google Scholar
  32. 32.
    Tadayon, R.: A person-centric design framework for at-home motor learning in serious games. Ph.D. thesis, Arizona State University (2017)Google Scholar
  33. 33.
    Tadayon, R., et al.: Interactive motor learning with the autonomous training assistant: a case study. In: Kurosu, M. (ed.) HCI 2015. LNCS, vol. 9170, pp. 495–506. Springer, Cham (2015). Scholar
  34. 34.
    Venkateswara, H., Chakraborty, S., McDaniel, T., Panchanathan, S.: Model selection with nonlinear embedding for unsupervised domain adaptation. In: KnowPros Workshop - Proceedings of the AAAI Conference on Artificial Intelligence (2017)Google Scholar
  35. 35.
    Vygotsky, L.: Zone of proximal development. In: Mind in Society: The Development of Higher Psychological Processes, vol. 5291, p. 157 (1987)Google Scholar
  36. 36.
    Wolf, S.L., Catlin, P.A., Ellis, M., Archer, A.L., Morgan, B., Piacentino, A.: Assessing wolf motor function test as outcome measure for research in patients after stroke. Stroke 32(7), 1635–1639 (2001)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Sethuraman Panchanathan
    • 1
  • Ramin Tadayon
    • 1
  • Hemanth Venkateswara
    • 1
    Email author
  • Troy McDaniel
    • 1
  1. 1.Center for Cognitive Ubiquitous Computing (CUbiC)Arizona State UniversityTempeUSA

Personalised recommendations