“Mirror, mirror on my search...”: Data-Driven Reflection and Experimentation with Search Behaviour

  • Angela FesslEmail author
  • Aitor Apaolaza
  • Ann Gledson
  • Viktoria Pammer-Schindler
  • Markel Vigo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11722)


Searching on the web is a key activity for working and learning purposes. In this work, we aimed to motivate users to reflect on their search behaviour, and to experiment with different search functionalities. We implemented a widget that logs user interactions within a search platform, mirrors back search behaviours to users, and prompts users to reflect about it. We carried out two studies to evaluate the impact of such widget on search behaviour: in Study 1 (N = 76), participants received screenshots of the widget including reflection prompts while in Study 2 (N = 15), a maximum of 10 search tasks were conducted by participants over a period of two weeks on a search platform that contained the widget. Study 1 shows that reflection prompts induce meaningful insights about search behaviour. Study 2 suggests that, when using a novel search platform for the first time, those participants who had the widget prioritised search behaviours over time. The incorporation of the widget into the search platform after users had become familiar with it, however, was not observed to impact search behaviour. While the potential to support un-learning of routines could not be shown, the two studies suggest the widget’s usability, perceived usefulness, potential to induce reflection and potential to impact search behaviour.


Search behaviour Reflective learning Activity log data analysis 



The project “MOVING - TraininG towards a society of data-saVvy inforMation prOfessionals to enable open leadership iNnovation” is funded under the Horizon 2020 of the European Commission (project number 693092). The Know-Center is funded within the Austrian COMET Program - Competence Centers for Excellent Technologies - under the auspices of the Austrian Federal Ministry of Transport, Innovation and Technology, the Austrian Federal Ministry of Economy, Family and Youth and by the State of Styria. COMET is managed by the Austrian Research Promotion Agency FFG.


  1. 1.
    Apaolaza, A., et al.: MOVING Project, Deliverable 1.4: Final implementation of user studies and evaluation (2019)Google Scholar
  2. 2.
    Asgari, E., Mofrad, M.R.: Continuous distributed representation of biological sequences for deep proteomics and genomics. PLoS ONE 10(11), e0141287 (2015)CrossRefGoogle Scholar
  3. 3.
    Aula, A., Nordhausen, K.: Modeling successful performance in web searching. J. Am. Soc. Inf. Sci. Technol. 57(12), 1678–1693 (2006)CrossRefGoogle Scholar
  4. 4.
    Bateman, S., Teevan, J., White, R.W.: The search dashboard: how reflection and comparison impact search behavior. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1785–1794. ACM (2012)Google Scholar
  5. 5.
    Boud, D., Keogh, R., Walker, D.: Promoting reflection in learning: a model. In: Reflection: Turning Experience into Learning, pp. 18–40. Routledge Falmer, New York (1985)CrossRefGoogle Scholar
  6. 6.
    Carroll, J.M., Rosson, M.B.: Paradox of the active user. In: Interfacing Thought: Cognitive Aspects of Human-computer Interaction, pp. 80–111. MIT Press, Cambridge (1987)Google Scholar
  7. 7.
    Edwards, S.L., Bruce, C.S.: Panning for gold: Understanding students’ information searching experiences. In: Transforming IT Education: Promoting a Culture of Excellence, pp. 351–369 (2006)Google Scholar
  8. 8.
    Evans, B.M., Chi, E.H.: An elaborated model of social search. Inf. Process. Manage. 46(6), 656–678 (2010)CrossRefGoogle Scholar
  9. 9.
    Fessl, A., Wesiak, G., Rivera-Pelayo, V., Feyertag, S., Pammer, V.: In-app reflection guidance: lessons learned across four field trials at the workplace. IEEE Trans. Learn. Technol. 10(4), 488–501 (2017)CrossRefGoogle Scholar
  10. 10.
    Ifenthaler, D.: Determining the effectiveness of prompts for self-regulated learning in problem-solving scenarios. Ed. Technol. Soc. 15(1), 38–52 (2012)Google Scholar
  11. 11.
    Kirkpatrick, D.L., Kirkpatrick, J.D.: Evaluating Training Programs: The Four Levels, 3rd edn. Berrett-Koehler Publishers, San Francisco (2006)Google Scholar
  12. 12.
    Kocielnik, R., Avrahami, D., Marlow, J., Lu, D., Hsieh, G.: Designing for workplace reflection: a chat and voice-based conversational agent. In: Proceedings of the 2018 Designing Interactive Systems Conference, pp. 881–894. ACM (2018)Google Scholar
  13. 13.
    Kocielnik, R., Xiao, L., Avrahami, D., Hsieh, G.: Reflection companion: a conversational system for engaging users in reflection on physical activity. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2(2), 70:1–70:26 (2018)CrossRefGoogle Scholar
  14. 14.
    Lagun, D., Lalmas, M.: Understanding user attention and engagement in online news reading. In: Proceedings of the Ninth ACM International Conference on Web Search and Data Mining, pp. 113–122. ACM (2016)Google Scholar
  15. 15.
    Lalmas, M., O’Brien, H., Yom-Tov, E.: Measuring user engagement. Synth. Lect. Inf. Concepts Retrieval Serv. 6(4), 1–132 (2014)Google Scholar
  16. 16.
    Malacria, S., Scarr, J., Cockburn, A., Gutwin, C., Grossman, T.: Skillometers: reflective widgets that motivate and help users to improve performance. In: Proceedings of the 26th ACM Symposium on User Interface Software and Technology, pp. 321–330. ACM (2013)Google Scholar
  17. 17.
    Pammer, V., Bratic, M., Feyertag, S., Faltin, N.: The value of self-tracking and the added value of coaching in the case of improving time management. In: Conole, G., Klobučar, T., Rensing, C., Konert, J., Lavoué, É. (eds.) EC-TEL 2015. LNCS, vol. 9307, pp. 467–472. Springer, Cham (2015). Scholar
  18. 18.
    Pammer, V., Krogstie, B., Prilla, M.: Let’s talk about reflection at work. Int. J. Technol. Enhanced Learn. (IJTEL) 9(2/3), 151–168 (2017)CrossRefGoogle Scholar
  19. 19.
    Park, S.Y., et al.: An analysis of the technology acceptance model in understanding university students’ behavioral intention to use e-learning. Educ. Technol. Soc. 12(3), 150–162 (2009)Google Scholar
  20. 20.
    Perrone, V.: Librarians and the nature of expertise. In: Proceedings LIANZA Conference 2004, LIANZA (2004)Google Scholar
  21. 21.
    Prilla, M., Renner, B.: Supporting collaborative reflection at work: a comparative case analysis. In: Proceedings of the 18th International Conference on Supporting Group Work, pp. 182–193. ACM (2014)Google Scholar
  22. 22.
    Sciascio, C.D., Sabol, V., Veas, E.: Supporting exploratory search with a visual user-driven approach. ACM Trans. Interact. Intell. Syst. 7(4), 18 (2017)CrossRefGoogle Scholar
  23. 23.
    Tucker, V.M.: The expert searcher’s experience of information. In: Information experience: Approaches to Theory and Practice, pp. 239–255. Emerald Group Publishing Limited (2014)Google Scholar
  24. 24.
    Vagliano, I., et al.: Open innovation in the big data era with the moving platform. IEEE MultiMedia 25(3), 8–21 (2018)CrossRefGoogle Scholar
  25. 25.
    Xia, Y., Cambria, E., Hussain, A., Zhao, H.: Word polarity disambiguation using bayesian model and opinion-level features. Cogn. Comput. 7(3), 369–380 (2015)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Angela Fessl
    • 1
    Email author
  • Aitor Apaolaza
    • 2
  • Ann Gledson
    • 2
  • Viktoria Pammer-Schindler
    • 1
    • 3
  • Markel Vigo
    • 2
  1. 1.Know-Center GmbHGrazAustria
  2. 2.School of Computer ScienceUniversity of ManchesterManchesterUK
  3. 3.Institute for Interactive Systems and Data ScienceGraz University of TechnologyGrazAustria

Personalised recommendations