Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Sentient destination prediction

  • 40 Accesses


In addition to context awareness and proactive behaviour, personalization constitutes one of the most important factors to consider when building digital assistance systems. In the field of location prediction, personal information can lead to a significant improvement of the predictive performance of the respective models. However, most of the existing approaches handle this type of information separate from the actual location and movement data, making them incapable of taking the entire existing underlying dynamics into account. In the presented work, inspired by the Minsky’s frame system theory in the 1970s, we evaluate an adapted ontological construct, which we call context-specific cognitive frame (CSCF), in order to capture the entire experience of a user at a given moment. Moreover, we use CSCFs to associate situations that arise when visiting a certain location with the respective context information as well as the emotions and the personality of the user. We show that our method can be used to provide a flexible, more accurate and therefore more personalized user experience using the example of location prediction.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12


  1. 1.

    http://ntcir-lifelog.computing.dcu.ie/ and https://www.imageclef.org/2018/lifelog.

  2. 2.


  3. 3.



  1. Adali, S., Golbeck, J.: Predicting personality with social behavior: a comparative study. Soc. Netw. Anal. Min. 4, 159 (2014)

  2. Adomavicius, G., Tuzhilin, A.: Context-Aware Recommender Systems, pp. 217–253. Springer, Boston (2011)

  3. Ajzen, I.: The theory of planned behavior. Organ. Behav. Hum. Decis. Process. 50(2), 179–211 (1991)

  4. Al Ridhawi, I., Aloqaily, M., Karmouch, A., Agoulmine, N.: A location-aware user tracking and prediction system. In: Information Infrastructure Symposium, GIIS’09, pp. 1–8. IEEE Global (2009)

  5. Ashbrook, D., Starner, T.: Using GPS to learn significant locations and predict movement across multiple users. Personal Ubiquitous Comput. 7(5), 275–286 (2003)

  6. Bollen, J., Mao, H., Zeng, X.: Twitter mood predicts the stock market. J. Comput. Sci. 2(1), 1–8 (2011)

  7. Borkenau, P.: To predict some of the people more of the time. In: Craik, K.H., Hogan, R., Wolfe, R.N. (eds.) Fifty Years of Personality Psychology, pp. 237–249. Springer, Boston (1993)

  8. Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)

  9. Broekens, J., Brinkman, W.P.: Affectbutton: towards a standard for dynamic affective user feedback. In: 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, ACII 2009, pp. 1–8. IEEE (2009)

  10. Buss, A.R.: The Trait-Situation Controversy and the Concept of Interaction, vol. 3, pp. 196–201. Sage, Thousand Oaks (1977)

  11. Celli, F., Bruni, E., Lepri, B.: Automatic personality and interaction style recognition from facebook profile pictures. In: Proceedings of the 22nd ACM International Conference on Multimedia, MM ’14, pp. 1101–1104. ACM, New York (2014)

  12. Chauhan, A., Toshniwal, D., Tejwani, R.: Predicting future place of visit using user’s personality profile. In: 2016 International Conference on Computational Techniques in Information and Communication Technologies (ICCTICT), pp. 427–432 (2016)

  13. Chauhan, A., Kummamuru, K., Toshniwal, D.: Prediction of places of visit using tweets. Knowl. Inf. Syst. 50(1), 145–166 (2017)

  14. Cho, S.B.: Exploiting machine learning techniques for location recognition and prediction with smartphone logs. Neurocomputing 176, 98–106 (2016)

  15. Cook, R., Kay, J.: The justified user model: a viewable, explained user model. In: Proceedings of the Fourth International Conference on User Modeling, pp. 145–150 (1994)

  16. Dagar, D., Hudait, A., Tripathy, H.K., Das, M.N.: Automatic emotion detection model from facial expression. In: 2016 International Conference on Advanced Communication Control and Computing Technologies (ICACCCT), pp. 77–85 (2016)

  17. Dey, A.K.: Understanding and using context. Personal Ubiquitous Comput. 5(1), 4–7 (2001)

  18. Dourish, P.: What we talk about when we talk about context. Personal Ubiquitous Comput. 8(1), 19–30 (2004)

  19. Eagle, N., Pentland, A.S.: Reality mining: sensing complex social systems. Personal Ubiquitous Comput. 10(4), 255–268 (2006)

  20. Ezhilarasi, R., Minu, R.: Automatic emotion recognition and classification. Procedia Eng. 38, 21–26 (2012)

  21. Fleeson, W.: Toward a Structure-and Process-Integrated View of Personality: Traits as Density Distributions of States, vol. 80, p. 1011. American Psychological Association (2001)

  22. Foursquare: Foursquare venue categories. https://developer.foursquare.com/docs/resources/categories. Accessed 2018-04-06

  23. Gao, H., Tang, J., Liu, H.: Mobile location prediction in spatio-temporal context. In: Nokia Mobile Data Challenge Workshop, vol. 41, pp. 1–4 (2012)

  24. Goldberg, L.R.: The structure of phenotypic personality traits. Am. Psychol. 48(1), 26 (1993)

  25. Google: tensorflow. https://www.tensorflow.org. Accessed 2018-04-06

  26. Gruber, T.R.: Toward principles for the design of ontologies used for knowledge sharing. Int. J. Hum. Comput. Stud. 43(5–6), 907–928 (1995)

  27. Guarino, N., Oberle, D., Staab, S.: What is an ontology? In: Staab, S., Studer, R. (eds.) Handbook on Ontologies, pp. 1–17. Springer, Berlin (2009)

  28. Hainš, V.V., Lovrenčić, S., Kirinić, V.: Personality model representation using ontology. DAAAM Int. Sci. B. 2007(7), 423 (2007)

  29. Heckmann, D., Schwartz, T., Brandherm, B., Schmitz, M., von Wilamowitz-Moellendorff, M.: Gumo—the general user model ontology. In: Ardissono, L., Brna, P., Mitrovic, A. (eds.) User Modeling 2005, pp. 428–432. Springer, Berlin (2005)

  30. Hochreiter, S.: The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int. J. Uncertain. Fuzziness Knowl. Based Syst. 6(02), 107–116 (1998)

  31. Jaccard, J.J.: Predicting Social Behavior from Personality Traits, vol. 7, pp. 358–367. Elsevier, Amsterdam (1974)

  32. Juanjuan, Z., Junjie, C.: The research on user modeling for personalized affective computing. In: 2008 International Conference on Computer Science and Software Engineering, vol. 1, pp. 459–462. IEEE (2008)

  33. Kahneman, D., Krueger, A.B., Schkade, D.A., Schwarz, N., Stone, A.A.: A survey method for characterizing daily life experience: the day reconstruction method. Science 306(5702), 1776–1780 (2004)

  34. Kalimeri, K., Lepri, B., Pianesi, F.: Going beyond traits: multimodal classification of personality states in the wild. In: Proceedings of the 15th ACM on International Conference on Multimodal Interaction, pp. 27–34. ACM (2013)

  35. Karatzoglou, A.: Evolutionary optimization on artificial neural networks for predicting the user’s future semantic location. In: Macintyre, J., Iliadis, L., Maglogiannis, I., Jayne, C. (eds.) Engineering Applications of Neural Networks, pp. 379–390. Springer, Cham (2019)

  36. Karatzoglou, A.: Multi–channel convolutional neural networks for handling multi-dimensional semantic trajectories and predicting future semantic locations. In: Tserpes, K., Renso, C., Matwin, S. (eds.) Multiple-Aspect Analysis of Semantic Trajectories, pp. 117–132. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-38081-6_9

  37. Karatzoglou, A., Beigl, M.: Enhancing the affective sensitivity of location based services using situation-person-dependent semantic similarity. In: The Eleventh International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies (UBICOMM 2017), pp. 95–100. IARIA (2017a)

  38. Karatzoglou, A., Beigl, M.: Applying situation–person-driven semantic similarity on location-specific cognitive frames for improving location prediction. In: 8th International Conference on Knowledge Engineering and Semantic Web (KESW 2017), pp. 4–5. KESW (2017b)

  39. Karatzoglou, A., Beigl, M.: Semantic-enhanced learning (SEL) on artificial neural networks using the example of semantic location prediction. In: Proceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM (2019)

  40. Karatzoglou, A., Lamp, S.C., Beigl, M.: Matrix factorization on semantic trajectories for predicting future semantic locations. In: 13th IEEE International Conference on Wireless and Mobile Computing, Networking and Communications, pp. 1–7. WiMob 2017, Rome, 9–11 Oct (2017a)

  41. Karatzoglou, A., Sentürk, H., Jablonski, A., Beigl, M.: Applying artificial neural networks on two-layer semantic trajectories for predicting the next semantic location. In: 26th International Conference on Artificial Neural Networks and Machine Learning—ICANN 2017, Proceedings, Part II, pp. 233–241. Alghero, 11–14 Sept (2017b)

  42. Karatzoglou, A., Jablonski, A., Beigl, M.: A Seq2Seq learning approach for modeling semantic trajectories and predicting the next location. In: Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pp. 528–531. ACM (2018a)

  43. Karatzoglou, A., Koehler, D., Beigl, M.: Purpose-of-visit-driven semantic similarity analysis on semantic trajectories for enhancing the future location prediction (to come). In: 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), pp. 0–0. IEEE (2018b)

  44. Karatzoglou, A., Szarvas, M., Beigl, M.: Towards an affective semantic trajectory generator (ASTG). In: 14th IEEE International Conference on Wireless and Mobile Computing, Networking and Communications, pp. 1–10. WiMob 2018, Limassol, 15–17 Oct (2018c)

  45. Karatzoglou, A., Schnell, N., Beigl, M.: A convolutional neural network approach for modeling semantic trajectories and predicting future locations. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds.) Artificial Neural Networks and Machine Learning—ICANN 2018, pp. 61–72. Springer, Cham (2018d)

  46. Karatzoglou, A., Schnell, N., Beigl, M.: Applying depthwise separable and multi-channel convolutional neural networks of varied kernel size on semantic trajectories. Neural Comput. Appl. (2019). https://doi.org/10.1007/s00521-019-04603-0

  47. Kassin, S.: Psychology. Prentice Hall, Upper Saddle River (2004)

  48. Kay, J.: The um toolkit for cooperative user modelling. User Model. User-Adapt. Interact. 4(3), 149–196 (1994)

  49. Kim, S., Li, F., Lebanon, G., Essa, I.: Beyond sentiment: the manifold of human emotions. In: Artificial Intelligence and Statistics, pp. 360–369 (2013)

  50. Kim, S., Song, H.Y., Koo, H.J.: Probabilistically predicting location of human with psychological factors. Proceedings of ECCS (2012)

  51. Kim, H., Song, H.Y.: Formulating human mobility model in a form of continuous time Markov chain. Procedia Comput. Sci. 10, 389–396 (2012)

  52. Kim, S.Y., Song, H.Y.: Predicting human location based on human personality. In: Balandin, S., Andreev, S., Koucheryavy, Y. (eds.) International Conference on Next Generation Wired/Wireless Networking, pp. 70–81. Springer, Cham (2014)

  53. Kim, D.Y., Song, H.Y.: Method of predicting human mobility patterns using deep learning. Neurocomputing 280, 56–64 (2018)

  54. Kim, S.Y., Koo, H.J., Song, H.Y.: A study on influence of human personality to location selection. J. Ambient Intell. Humaniz. Comput. 7(2), 267–285 (2016)

  55. Kim, S.Y., Koo, H.J., Song, H.Y.: A study on estimation of human personality from location visiting preference. J. Ambient Intell. Humaniz. Comput. 9, 1–14 (2017)

  56. Körner, A., Geyer, M., Roth, M., Drapeau, M., Schmutzer, G., Albani, C., Schumann, S., Brähler, E.: Persönlichkeitsdiagnostik mit dem neo-fünf-faktoren-inventar: die 30-item-kurzversion (neo-ffi-30). PPmP-Psychotherapie\(\cdot \) Psychosomatik\(\cdot \) Medizinische Psychologie 58(06), 238–245 (2008)

  57. Lee, W.N., Shah, N., Sundlass, K., Musen, M.: Comparison of ontology-based semantic-similarity measures, vol. 2008, pp. 384–388 (2008)

  58. Lee, S., Lee, K.C., Cho, H.: A dynamic bayesian network approach to location prediction in ubiquitous computing environments. In: Papasratorn, B., Lavangnananda, K., Chutimaskul, W., Vanijja, V. (eds.) Advances in Information Technology, pp. 73–82. Springer, Berlin (2010)

  59. Levandowsky, M., Winter, D.: Distance between sets. Nature 234(5323), 34–35 (1971)

  60. Lupton, D.: The Quantified Self. Wiley, Hoboken (2016)

  61. Mairesse, F., Walker, M.A., Mehl, M.R., Moore, R.K.: Using linguistic cues for the automatic recognition of personality in conversation and text. J. Artif. Intell. Res. 30, 457–500 (2007)

  62. Martinho, C., Machado, I., Paiva, A.: A Cognitive Approach to Affective User Modeling, pp. 64–75. Springer, Berlin (2000)

  63. McNair, D.M., Lorr, M., Droppleman, L.F.: Profile of mood states: manual. Edits (1992)

  64. Meng, L., Huang, R., Gu, J.: A review of semantic similarity measures in wordnet. Int. J. Hybrid Inf. Technol. 6, 1–12 (2013)

  65. Minsky, M.: A framework for representing knowledge. Technical Report, Cambridge (1974)

  66. Nasoz, F., Lisetti, C.L.: Affective user modeling for adaptive intelligent user interfaces. In: Jacko, J.A. (ed.) International Conference on Human–Computer Interaction, pp. 421–430. Springer, Berlin (2007)

  67. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

  68. Pentland, A., Liu, A.: Modeling and prediction of human behavior. Neural comput. 11(1), 229–242 (1999)

  69. Protege: Protege ontology editor. https://protege.stanford.edu/. Accessed 2018-04-06

  70. Rentfrow, P.J., Gosling, S.D., Jokela, M., Stillwell, D.J., Kosinski, M., Potter, J.: Divided we stand: three psychological regions of the united states and their political, economic, social, and health correlates. J. Personal. Soc. Psychol. 105(6), 996 (2013)

  71. Ruder, S.: An overview of gradient descent optimization algorithms. arXiv:1609.04747 (2016)

  72. Sacharin, V., Schlegel, K., Scherer, K.R.: Geneva emotion wheel rating study (2012)

  73. Samaan, N., Karmouch, A.: A mobility prediction architecture based on contextual knowledge and spatial conceptual maps. IEEE Trans. Mob. Comput. 4(6), 537–551 (2005)

  74. Siegert, I., Böck, R., Vlasenko, B., Philippou-Hübner, D., Wendemuth, A.: Appropriate emotional labelling of non-acted speech using basic emotions, geneva emotion wheel and self assessment manikins. In: 2011 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2011)

  75. Song, H.Y., Kang, H.B.: Analysis of relationship between personality and favorite places with Poisson regression analysis. In: ITM Web of Conferences, vol. 16, p. 02001. EDP Sciences (2018)

  76. Spaccapietra, S., Parent, C., Damiani, M.L., de Macedo, J.A., Porto, F., Vangenot, C.: A conceptual view on trajectories. Data Knowl. Eng. 65(1), 126–146 (2008). Including special section: privacy aspects of data mining workshop (2006)—five invited and extended papers

  77. Staiano, J., Lepri, B., Aharony, N., Pianesi, F., Sebe, N., Pentland, A.: Friends don’t lie: inferring personality traits from social network structure. In: Proceedings of the 2012 ACM Conference on Ubiquitous Computing, pp. 321–330. ACM, New York (2012)

  78. Swan, M.: The quantified self: fundamental disruption in big data science and biological discovery. Big Data 1(2), 85–99 (2013). PMID: 27442063

  79. Tversky, A.: Features of similarity. Psychol. Rev. 84, 327–352 (1977)

  80. Vogt, T., André, E.: Improving automatic emotion recognition from speech via gender differentiaion. In: LREC, pp. 1123–1126 (2006)

  81. Welty, C., McGuinness, D.L., Smith, M.K.: Owl web ontology language guide. W3C recommendation, W3C (2004). http://www.w3.org/TR/2004/REC-owl-guide-20040210

  82. Yao, D., Zhang, C., Huang, J., Bi, J.: Serm: a recurrent model for next location prediction in semantic trajectories. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp. 2411–2414. ACM (2017)

  83. Ying, J.J.C., Lee, W.C., Weng, T.C., Tseng, V.S.: Semantic trajectory mining for location prediction. In: Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM, pp. 34–43 (2011)

  84. Zheng, Y., Zhang, L., Ma, Z., Xie, X., Ma, W.Y.: Recommending friends and locations based on individual location history. ACM Trans. Web 5, 5 (2011)

Download references


The authors would like to thank all the diligently annotating participants of the user study.

Author information

Correspondence to Antonios Karatzoglou.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Karatzoglou, A., Ebbing, J., Ostheimer, P. et al. Sentient destination prediction. User Model User-Adap Inter (2020). https://doi.org/10.1007/s11257-020-09257-5

Download citation


  • Personalization
  • Context-specific cognitive frames
  • Location-specific cognitive frames
  • Ontology design patterns
  • Context awareness
  • Personality
  • Mental states
  • Emotions