Skip to main content
SpringerLink
Log in

SLOG: smartphone-based semantic lifelogging framework for digital prosthetic memory development

  • Original Paper
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

The advancements in sensors, processing, storage and networking technologies have turned smartphone into de facto lifelogging device. Realizing the lifelogging potential of smartphone, researchers have postulated with smartphone-based lifelogging systems with varying capabilities of capturing and storing information about peoples’ daily life activities, behaviors, interactions, locations, environments, etc., into their lifelog archives. However, the systems are lacking with several capturing, storing, retrieving and design issues, importantly not organizing the lifelog information with the same semantics as they exist in the real world and encoded in human episodic memory. The semantic organization of lifelog information will develop a true digital prosthetic memory to solve problems in different real-world domains including memory augmentation. In this paper, we have presented the architecture and implementation of Semantic LifelOGging (SLOG) framework for smartphone-based semantic lifelogging using Semantic Web technologies. The SLOG framework incorporates smartphone sensors and processing capabilities, and Semantic Web technologies to develop a semantically enriched digital prosthetic memory on smartphone. The semantic lifelog data model is developed by extending SmartOntoSensor ontology with relevant concepts and properties to represent, store and relate lifelog and contextual information. The semantic lifelog data model will provide contextual semantics to organize, annotate and relate lifelog information like human episodic memory and provide enough contextual cues to recall lifelog information like associative recall in human memory. The proof-of-concept applications are developed to show practicality of the SLOG framework. The quantitative and empirical evaluations have shown the SLOG framework and system has improved features and is acceptable in usability and effectiveness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Data availability

All of the data and material used in this research work are available.

Code availability

The tools used in this research work are available.

Notes

  1. https://www.pewresearch.org/methods/u-s-survey-research/questionnaire-design/

References

  1. Gurrin C, Smeaton AF, Doherty AR (2014) LifeLogging: personal big data. Found Trends Inf Retr 8(1):1–125

    Article  Google Scholar 

  2. Ali S (2018) Exploiting sensor data semantics for smartphone-based lifelogging: Towards the development of digital prosthetic memory on smartphone. PhD Dissertation, University of Peshawar, Pakistan

  3. Chen Y, Jones GJF (2010) Augmenting human memory using personal lifelogs. In Proceedings of the 1st Augmented Human International Conference. ACM, pp. 1–9

  4. Hodges S, Williams L, Berr E, Izadi S, Srinivasan J, Butler A, Smyth G, Kapur N, Wood K (2006) SenseCam: A retrospective memory aid. In Proceedings of the 8th international conference on Ubiquitous Computing. Springer, pp. 177–193

  5. Aizenbud-Reshef N, Belinsky E, Jacovi M, Laufer D, Soroka V (2008) Pensieve: augmenting human memory. In Proceedings of the CHI '08 Extended Abstracts on Human Factors in Computing Systems. ACM, pp 3231–3236. https://doi.org/10.1145/1358628.1358836

  6. Chennuru SK, Chen P-W, Zhu J, Zhang JY (2010) Mobile Lifelogger - recording, indexing, and understanding a mobile user's life. In Proceedings of the International Conference on Mobile Computing, Applications, and Services. Springer, pp 263–281

  7. Klapperich H, Uhde A, Hassenzahl M (2020) Designing everyday automation with well-being in mind. Pers Ubiquit Comput 24(6):763–779

    Article  Google Scholar 

  8. Belimpasakis P, Roimela K, Yu Y (2009) Experience Explorer: a life-logging platform based on mobile context collection. In Proceedings of the Third International Conference on Next Generation Mobile Applications, Services and Technologies - NGMAST '09. IEEE, pp 77–82. https://doi.org/10.1109/ngmast.2009.49

  9. Albatal R, Gurrin C, Zhou J, Yang Y, Carthy D, Li N (2013) SenseSeer: mobile-cloud-based lifelogging framework. In Proceedings of the IEEE International Symposium on Technology and Society (ISTAS). IEEE, pp 27–29

  10. Mohamed EST (2012) Designing and evaluating a user interface for continues embedded lifelogging based on physical context. PhD Dissertation, Newcastle University, England

  11. Karkkainen T, Vaittinen T, Vaananen-Vainio-Mattila K (2010) I don’t mind being logged, but want to remain in control: a field study of mobile activity and context logging. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, pp. 163–172

  12. Bush V (1996) As we may think interactions 3(2):35–46. https://doi.org/10.1145/227181.227186

    Article  Google Scholar 

  13. Oren E (2006) An overview of information management and knowledge work studies: lessons for the semantic desktop. In Proceedings of the 5th International Conference on Semantic Desktop and Social Semantic Collaboration- SemDesk’06. ACM, pp. 14–24

  14. Dodge M, Kitchin R (2007) ‘Outlines of a world coming into existence’: pervasive computing and the ethics of forgetting. Environ Plann B Plann Des 34(3):431–445

    Article  Google Scholar 

  15. Kalnikaite V, Sellen A, Whittaker S, Kirk D (2010) Now let me see where i was: understanding how lifelogs mediate memory. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, pp. 2045–2054

  16. Ali S, Khusro S, Khan A, Khan I, Solheria SF (2019) An insight of smartphone-based lifelogging research: Issues, challenges, and research opportunities. Proceedings of the Pakistan Academy of Sci: A Physical and Computational Sci 56(3):1–16

    MathSciNet  Google Scholar 

  17. Sellen A, Whittaker S (2010) Beyond total capture: A constructive critique of lifelogging. Commun ACM 53(5):70–77. https://doi.org/10.1145/1735223.1735243

    Article  Google Scholar 

  18. Rawassizadeh R (2012) A holistic multi-purpose life logging framework. PhD Dissertation, University of Vienna, Austria

  19. Siewiorek D, Smailagic A, Furukawa J, Krause A, Moraveji N, Reiger K, Shaffer J, Wong FL (2003) SenSay: A context-aware mobile phone. In Proceedings of the 7th IEEE International Symposium on Wearable Computers. IEEE, pp. 248–253

  20. Shah M, Mears B, Chakrabarti C, Spanias A (2012) Lifelogging: Archival and retrieval of continuously recorded audio using wearable devices. In Proceedings of the IEEE International Conference on Emerging Signal Processing Applications (ESPA). IEEE, pp 99–102

  21. Wolf K, Schmidt A, Bexheti A, Langheinrich M (2014) Lifelogging: you’re wearing a camera? IEEE Pervasive Comput 13(3):8–12

    Article  Google Scholar 

  22. Gurrin C, Johansen H, Sødring T, Johansen D (2014) Digital chronofiles of life experience. In Proceedings of the International Conference on Social Informatics. Springer, pp 264–273

  23. Rawassizadeh R, Tomitsch M, Wac K, Tjoa AM (2013) UbiqLog: a generic mobile phone-based life-log framework. Pers Ubiquit Comput 17(4):621–637. https://doi.org/10.1007/s00779-012-0511-8

    Article  Google Scholar 

  24. Vemuri S, Schmandt C, Bender W (2006) iRemember: a personal, long-term memory prosthesis. In Proceedings of the 3rd ACM Workshop on Continous Archival and Retrieval of Persoanl Experiences CARPE' 06. ACM, pp 65–74

  25. Ali S, Khusro S, Rauf A, Mahfooz S (2014) Sensors and mobile phones: Evolution and state-of-the-art. Pak J Sci 66(4):386–400

    Google Scholar 

  26. Ali S, Khusro S, Khan AA, Hassan L (2014) A survey of mobile phones context-awareness using sensing computing research. J Engineering and Applied Sci 33(4):75–93

    Google Scholar 

  27. Myka A (2005) Nokia lifeblog -- towards a truly personal multimedia information system. In Proceedings of the Workshop on Mobile Data Banken and Information Systemes - MDBIS'05. pp 21–30

  28. Ahmed M, Hoang HH, Karim MS, Khusro S, Lanzenberger M, Latif K, Michlmayr E, Mustofa K, Nguyen H, Rauber A (2004) 'SemanticLIFE'-a framework for managing information of a human lifetime. In Proceedings of the Sixth International Conference on Information Integration and Web Based Applications & Services. iiWAS2004, pp. 725–734

  29. Froehlich J, Chen MY, Consolvo S, Harrison B, Landay JA (2007) MyExperience: A system for in situ tracing and capturing of user feedback on mobile phones. In Proceedings of the 5th International Conference on Mobile Systems, Applications and Services. ACM, pp. 57–70

  30. Packer HS, Smith A, Lewis P (2012) MemoryBook: generating narratives from lifelogs. In Proceedings of the 2nd Workshop on Narrative and Hypertext. ACM, pp 7–12

  31. Memon MA, Bhatti S, Mahoto NA (2016) A Digital Diary: Remembering the past using the present context. Mehran University Research J Engineering Technol 35(2):275–286

    Google Scholar 

  32. Sangkeun L, Gihyun G, Inbeom H, Sang-goo L (2010) LifeLogOn: A practical lifelog system for building and exploiting lifelog ontology. In Proceeding of the IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (SUTC). IEEE, pp 367–373. https://doi.org/10.1109/sutc.2010.33

  33. Zafeiropoulos A, Konstantinou N, Arkoulis S, Spanos DE, Mitrou N A (2008) Semantic-based architecture for sensor data fusion. In Proceedings of the Second Internation Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies. IEEE, pp 116–121. https://doi.org/10.1109/ubicomm.2008.67

  34. Tuffield MM, Loizou A, Dupplaw D (2006) The Semantic Logger: Supporting service building from personal context. In Proceedings of the 3rd ACM Workshop on Continuous Archival and Retrival of Personal Experences - CARPE '06. ACM, pp. 55–64

  35. Song I, Haam D, Kim H, Kim MH (2010) OntoLMS: An ontology-based lifelog management system. In Proceedings of the 12th International Asia-Pacific Web Conference. IEEE, pp 341–343

  36. Ali S, Khusro S (2016) POEM: practical ontology engineering model for semantic web ontologies. Cogent Engineering 3(1):1193959. https://doi.org/10.1080/23311916.2016.1193959

    Article  Google Scholar 

  37. Cohen G, Conway MA (2008) Memory in the real world. Psychology Press, Hove East Sussex, United Kingdom

    Google Scholar 

  38. Tulving E, Murray D (1985) Elements of episodic memory. Can Psychol 26(3):235–238

    Article  Google Scholar 

  39. Tulving E, Thomson DM (1973) Encoding specificity and retrieval processes in episodic memory. Psychol Rev 80(5):359–380

    Article  Google Scholar 

  40. Ali S, Khusro S, Ullah I, Khan A, Khan I (2017) SmartOntoSensor: ontology for semantic interpretation of smartphone sensors data for context-aware applications. Journal of Sensors 17(2017):1–26

    Article  Google Scholar 

  41. Preuveneers D, den Bergh JV, Wagelaar D, Georges A, Rigole P, Clerckx T, Berbers Y, Coninx K, Jonckers V, Bosschere KD (2004) Towards an extensible context ontology for ambient intelligence. In Proceedings of the Second European Symposium on Ambient Intelligence. Springer, pp. 148–159

  42. Kim I-J, Ahn SC, Kim H-G (2007) Personalized life log media system in ubiquitous environment. In: Ubiquitous Convergence Technology. Springer, pp 20–29

  43. Weiser M (1994) The world is not a desktop. interactions 1 (1):7–8

  44. Indulska J, Sutton P (2003) Location management in pervasive systems. In Proceedings of the Australasian Information Security Workshop Conference on ACSW Frontiers. Australian Computer Society Inc, pp 143–151

  45. Ali S, Khusro S (2016) Mobile phone sensing: a new application paradigm. Indian J Sci Technol 9(19):1–42

    Article  Google Scholar 

  46. Wang P (2011) Semantic interpretation of events in lifelogging. PhD Dissertation, Dublin City University, Ireland

  47. Smeaton AF, Over P, Kraaij W (2009) High-level feature detection from video in TRECVid: A 5-year retrospective of achievements. Multimedia Content Analysis, Signals and Communication:151–174

  48. Arndt N (2011) TriplePlace: a flexible triple store for Android with six indices. https://core.ac.uk/reader/226099420#page=15. Accessed 20/03/2020 2020

  49. Möller K, Handschuh S (2007) Towards a light-weight semantic desktop. In Proceedings of the Semantic Desktop Design Workshop (SemDeskDesign 2007) at ESWC2007.

  50. Bell G (2001) A personal digital store. Commun ACM 44(1):86–91

    Article  Google Scholar 

  51. Allen AL (2008) Dredging up the past: Lifelogging, memory, and surveillance. The University of Chicago Law Review 75(1):47–74

    Google Scholar 

  52. Strahilevitz LJ (2005) A social networks theory of privacy. The University of Chicago Law Review:919–988

  53. Langheinrich M (2001) Privacy by design—principles of privacy-aware ubiquitous systems. In Proceedings of the International conference on Ubiquitous Computing. Springer, pp 273–291

  54. Khan I, Khusro S, Ali S, Din AU (2016) Daily life activities on smartphones and their effect on battery life for better personal information management. Proceedings of the Pakistan Academy of Sci: A Physical and Computational Sci 53(1):61–74

    Google Scholar 

  55. Khan I, Ali S, Khusro S (2019) Smartphone-based lifelogging: an investigation of data volume generation strength of smartphone sensors. In, Cham, 2019. Simulation tools and techniques. Springer International Publishing, pp 63–73

  56. Brooke J (1996) SUS: a “quick and dirty” usability. Usability evaluation in indmstry:189

  57. Mazumdar S, Petrelli D, Ciravegna F (2014) Exploring user and system requirements of linked data visualization through a visual dashboard approach. Semantic Web 5(3):203–220

    Article  Google Scholar 

  58. Bangor A, Kortum P, Miller J (2009) Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud 4(3):114–123

    Google Scholar 

  59. Zeis C, Regassa H, Shah A, Ahmadain A (2001) Goodness-of-fit test for rating scale data: applying the minimum chi-sqaure method. J Econ Soc Meas 27(1–2):25–39

    Article  Google Scholar 

Download references

Author information

Author notes

  1. Shaukat Ali, Shah Khusro, Akif Khan & Iftikhar Alam

    Present address: Department of Computer Science, University of Peshawar, Peshawar, 25120, Pakistan

  2. Salman Faiz Solehria

    Present address: Department of Computer Science, Sarhad University of Information Technology, Peshawar, 25000, Pakistan

  3. Inayat Khan

    Present address: Department of Computer Science, University of Engineering and Technology Mardan, Mardan, 23200, Pakistan

Authors and Affiliations

    Authors
    1. Shaukat Ali
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Shah Khusro
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Akif Khan
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Inayat Khan
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Iftikhar Alam
      View author publications

      You can also search for this author in PubMed Google Scholar

    6. Salman Faiz Solehria
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Shaukat Ali.

    Ethics declarations

    Conflict of interest

    The authors declare no competing interests.

    Additional information

    Publisher's note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Rights and permissions

    Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Ali, S., Khusro, S., Khan, A. et al. SLOG: smartphone-based semantic lifelogging framework for digital prosthetic memory development. Pers Ubiquit Comput 27, 2071–2102 (2023). https://doi.org/10.1007/s00779-022-01701-0

    Download citation

    • Received:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s00779-022-01701-0

    Keywords

    Search

    Navigation