Lucida: Enhancing the Creation of Photography Through Semantic, Sympathetic, Augmented, Voice Agent Interaction

  • Brad WrobleskiEmail author
  • Alexander Ivanov
  • Eric Eidelberg
  • Katayoon Etemad
  • Denis Gadbois
  • Christian Jacob
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10903)


We present a dynamic framework for the integration of Machine Learning (ML), Augmented Reality (AR), Affective Computing (AC), Natural Language Processing (NLP) and Computer Vision (CV) to make possible, the development of a mobile, sympathetic, ambient (virtual), augmented intelligence (Agent). For this study we developed a prototype agent to assist photographers to enhance the learning and creation of photography. Learning the art of photography is complicated by the technical complexity of the camera, the limitations of the user to see photographically and the lack of real time instruction and emotive support. The study looked at the interaction patterns between human student and instructor, the disparity between human vision and the camera, and the potential of an ambient agent to assist students in learning. The study measured the efficacy of the agent and its ability to transmute human-to-Human method of instruction to human-to-Agent interaction. This study illuminates the effectiveness of Agent based instruction. We demonstrate that a mobile, semantic, sympathetic, augmented intelligence, ambient agent can ameliorate learning photography metering in real time, ‘on location’. We show that the integration of specific technologies and design produces an effective architecture for the creation of augmented agent-based instruction.


Affective Computing Augmented Reality Agent Machine Learning Natural Language Processing Computer Vision Voice interaction Instructional technology 



The research team would like to thank Rick Young, Jill Hockaday, Rob Leedham and John Brosz for their assistance with this research.


  1. 1.
    Adams, A.: The Camera. The New Ansel Adams Photography Series. New York Graphic Society, Boston (1980)Google Scholar
  2. 2.
    Affectiva: Metrics (2018).
  3. 3.
    Albrecht, U.V., Folta-Schoofs, K., Behrends, M., Von Jan, U.: Effects of mobile augmented reality learning compared to textbook learning on medical students: randomized controlled pilot study. J. Med. Internet Res. 15(8), e182 (2013)CrossRefGoogle Scholar
  4. 4.
    Cambridge in Colour: Cameras vs the human eye (2018).
  5. 5.
    De Paoli, S.: Not all the bots are created equal: the ordering turing test for the labeling of bots in MMORPGs. Soc. Media Soc. 3(4), 1–13 (2017). Scholar
  6. 6.
    Ekman, P.: What scientists who study emotion agree about. Perspect. Psychol. Sci. 11(1), 31–34 (2016)CrossRefGoogle Scholar
  7. 7.
    Freeman, M., et al.: The DSLR Field Guide. CRC Press, ‎Boca Raton (2013)Google Scholar
  8. 8.
    Goodfellow, I., Bengio, Y., Courville, A., Bengio, Y.: Deep Learning, vol. 1. MIT Press, Cambridge (2016)zbMATHGoogle Scholar
  9. 9.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep con- volutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  10. 10.
    Machiraju, S., Modi, R.: Natural language processing. In: Developing Bots with Microsoft Bots Framework, pp. 203–232. Apress, Berkeley (2018). Scholar
  11. 11.
    Magdin, M., Prikler, F., Hamzaoui, Y., Amnai, M., Choukri, A., Fakhri, Y., Sam-durkar, A.S., Kamble, S.D., Thakur, N.V., Patharkar, A.S., et al.: Real time facial expression recognition using webcam and SDK affectiva. Int. J. Interact. Multimed. Artif. Intell. 4 (2018, in press)Google Scholar
  12. 12.
    McCann, J.J., Rizzi, A.: The Ansel Adams zone system. In: The Art and Science of HDR Imaging, pp. 59–68 (2011)Google Scholar
  13. 13.
    McDuff, D., El Kaliouby, R., Senechal, T., Amr, M., Cohn, J.F., Picard, R.: Affectiva-mit facial expression dataset (AM-FED): naturalistic and spontaneous facial expressions collected “in-the-wild”. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 881–888. IEEE (2013)Google Scholar
  14. 14.
    Condé Nast: Machine learning at Condé Nast, part 2: Handbag brand and color detection (2018).
  15. 15.
    Google Cloud Platform: Cloud speech API (2018).
  16. 16.
    Google Cloud Platform: Cloud vision API (2018).
  17. 17.
    Qualtrics:, Provo, UT, USA (2013)
  18. 18.
    Russell, J.A., Fernández-Dols, J.M.: The Psychology of Facial Expression. Cambridge University Press, New York (1997)CrossRefGoogle Scholar
  19. 19.
    Shevat, A.: Designing Bots: Creating Conversational Experiences. O’Reilly Media Inc., Sebastopol (2017)Google Scholar
  20. 20.
    Skorka, O., Joseph, D.: Toward a digital camera to rival the human eye. J. Electron. Imaging 20(3), 033009-1–033009-18 (2011)CrossRefGoogle Scholar
  21. 21.
    Open Source: Tensorflow (2018).
  22. 22.
    Tzanavari, A.: Affective, interactive and cognitive methods for e-learning design: creating an optimal education experience. IGI Global, Hershey (2010)CrossRefGoogle Scholar
  23. 23.
    Unity: Vuforia (2018).
  24. 24.
    Watson, I.: Build on the AI platform for business (2018).
  25. 25.
    Yan, Z., Zhang, H., Wang, B., Paris, S., Yu, Y.: Automatic photo adjustment using deep neural networks. ACM Trans. Graph. (TOG) 35(2), 11 (2016)CrossRefGoogle Scholar
  26. 26.
    Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2921–2929. IEEE (2016)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Brad Wrobleski
    • 1
    Email author
  • Alexander Ivanov
    • 1
  • Eric Eidelberg
    • 1
  • Katayoon Etemad
    • 1
  • Denis Gadbois
    • 1
  • Christian Jacob
    • 1
  1. 1.Computational Media Design and Department of Computer ScienceUniversity of CalgaryCalgaryCanada

Personalised recommendations