Skip to main content
SpringerLink
Log in

Myscéal: a deeper analysis of an interactive lifelog search engine

  • 1228: Advances in Lifelog Retrieval at the Lifelog Search Challenge 2021
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

There is a growing number of lifelogging retrieval systems that have been introduced in several lifelogging workshops and sessions. Across all systems at the LSC, which is an annual international challenge about lifelogging retrieval, our Myscéal is currently considered as the state-of-the-art. In this paper, we describe the system in detail and show how it has been upgraded through time since firstly introduced in 2020. In addition, we analyse Myscéal performance not only in the three lifelog retrieval competitions it participated in but also with additional user experiments. The result shows that the fast searching time of Myscéal is the system’s most important feature that helps it get some significant advantages in competitions. On the other hand, the findings from user experiments indicate that Myscéal still needs some improvements for novice users who are unfamiliar with how to interact with the system. Moreover, the user study plays a vital role in the development of Myscéal as many updates of this system came from the feedback of the participating users. We also demonstrate the efficacy of Myscéal as a lifelog retrieval system to help the lifeloggers, who capture their daily life in images, recall memorable moments in their massive lifelog archives.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/

  2. https://cloud.google.com/vision

  3. Here, we also consider an OCR text as a visual object with its bounding box.

  4. https://developers.google.com/maps/documentation/geocoding/requests-reverse-geocoding

  5. https://www.imageclef.org/system/files/ImageCLEF2020-test-topics.pdf

References

  1. Alam N, Graham Y, Gurrin C (2021) Memento: a prototype lifelog search engine for LSC’21. In: Proceedings of the 4th annual on lifelog search challenge. LSC ’21, Association for Computing Machinery, New York, pp 53–58

  2. Alateeq A, Roantree M, Gurrin C (2021) Voxento 2.0: a prototype voice-controlled interactive search engine for lifelogs. In: Proceedings of the 4th annual on lifelog search challenge. LSC ’21, Association for Computing Machinery, New York, pp 65–70

  3. Anderson P, He X, Buehler C, Teney D, Johnson M, Gould S, Zhang L (2018) Bottom-up and top-down attention for image captioning and visual question answering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6077–6086

  4. Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp 801–818

  5. Dang Nguyen DT, Piras L, Riegler M, Boato G, Zhou L, Gurrin C (2017) Overview of imageclef lifelog 2017: lifelog retrieval and summarization

  6. de Oliveira Barra G, Cartas Ayala A, Bolaños M, Dimiccoli M, Giró Nieto X, Radeva P (2016) Lemore: a lifelog engine for moments retrieval at the ntcir-lifelog lsat task. In: Proceedings of the 12th NTCIR conference on evaluation of information access technologies

  7. Duane A, Gurrin C, Huerst W (2018) Virtual reality lifelog explorer: lifelog search challenge at acm icmr 2018. In: Proceedings of the 2018 ACM workshop on the lifelog search challenge. LSC’18, pp 20–23

  8. Gurrin C, Smeaton AF, Doherty AR et al (2014) Lifelogging: personal big data. Found and Trends®; Inf Retrieval 8(1):1–125

    Article  Google Scholar 

  9. Gurrin C, Joho H, Hopfgartner F, Zhou L, Albatal R (2016) Overview of ntcir-12 lifelog task. NTCIR

  10. Gurrin C, Schoeffmann K, Joho H, Leibetseder A, Zhou L, Duane A, Nguyen D, Tien D, Riegler M, Piras L et al (2019) Comparing approaches to interactive lifelog search at the lifelog search challenge (LSC 2018). ITE Trans Media Technol Appl 7(2):46–59

    Google Scholar 

  11. Kovalčík G, Škrhak V, Souček T, Lokoč J (2020) Viret tool with advanced visual browsing and feedback. In: Proceedings of the 3rd annual workshop on lifelog search challenge. LSC ’20, Association for Computing Machinery, New York, pp 63–66

  12. Krishna R, Zhu Y, Groth O, Johnson J, Hata K, Kravitz J, Chen S, Kalantidis Y, Li L-J, Shamma DA et al (2017) Visual genome: connecting language and vision using crowdsourced dense image annotations. Int J Comput Vision 123(1):32–73

    Article  MathSciNet  Google Scholar 

  13. Le N-K, Nguyen D-H, Nguyen V-T, Tran M-T (2019) Lifelog moment retrieval with advanced semantic extraction and flexible moment visualization for exploration. In: CLEF (working notes)

  14. Leibetseder A, Schoeffmann K (2020) Lifexplore at the lifelog search challenge 2020. In: Proceedings of the 3rd annual workshop on lifelog search challenge. LSC ’20, Association for Computing Machinery, New York, pp 37–42

  15. Lokoč J, Mejzlík F, Veselỳ P, Souček T (2021) Enhanced somhunter for known-item search in lifelog data. In: Proceedings of the 4th annual on lifelog search challenge. LSC’21, pp 71–73

  16. Lowe DG (1999) Object recognition from local scale-invariant features. In: Proceedings of the 7th IEEE international conference on computer vision, vol 2, pp 1150–11572

  17. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60(2):91–110

    Article  Google Scholar 

  18. Luo H, Wei H, Lai LL (2011) Creating efficient visual codebook ensembles for object categorization. IEEE Trans Syst Man Cybern Part A Syst Humans 41(2):238–253

    Article  Google Scholar 

  19. Mejzlík F, Veselý P, Kratochvíl M, Souček T, Lokoč J (2020) Somhunter for lifelog search. In: Proceedings of the 3rd annual workshop on lifelog search challenge. LSC ’20, Association for Computing Machinery, New York, pp 73–75

  20. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119

  21. Nguyen T-N, Le T-K, Ninh V-T, Tran M-T, Thanh Binh N, Healy G, Caputo A, Gurrin C (2021) Lifeseeker 3.0: an interactive lifelog search engine for LSC’21. In: Proceedings of the 4th annual on lifelog search challenge, pp 41–46

  22. Nguyen M-D, Nguyen BT, Gurrin C (2021) Graph-based indexing and retrieval of lifelog data. In: International conference on multimedia modeling. MMM’21. Springer, pp 256–267

  23. Oram P (2001) Wordnet: an electronic lexical database. christiane fellbaum (ed.). Cambridge, ma: Mit press, 1998. pp 423. Appl Psycholinguist 22(1):131–134

    Article  Google Scholar 

  24. Rossetto L, Gasser R, Heller S, Amiri Parian M, Schuldt H (2019) Retrieval of structured and unstructured data with vitrivr. In: Proceedings of the ACM workshop on lifelog search challenge. LSC’19, pp 27–31

  25. Rossetto L, Baumgartner M, Gasser R, Heitz L, Wang R, Bernstein A (2021) Exploring graph-querying approaches in lifegraph. In: Proceedings of the 4th annual on lifelog search challenge. LSC ’21, Association for Computing Machinery, New York, pp 7–10

  26. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  27. Tran L-D, Nguyen M-D, Binh NT, Lee H, Gurrin C (2020) Myscéal: an experimental interactive lifelog retrieval system for LSC’20. In: Proceedings of the 3rd annual workshop on lifelog search challenge, pp 23–28

  28. Tran L-D, Nguyen M-D, Nguyen BT, Gurrin C (2020) An experiment in interactive retrieval for the lifelog moment retrieval task at imageCLEFlifelog2020. In: CLEF (working notes), p 12

  29. Tran L-D, Nguyen M-D, Thanh Binh N, Lee H, Gurrin C (2021) Myscéal 2.0: a revised experimental interactive lifelog retrieval system for LSC’21. In: Proceedings of the 4th annual on lifelog search challenge. LSC’21, pp 11–16

  30. Tran L-D, Nguyen M-D, Thanh Binh N, Lee H, Gurrin C (2022) E-myscéal: embedding-based interactive lifelog retrieval system for LSC’22. In: Proceedings of the 5th annual lifelog search challenge

  31. Trang-Trung H-P, Le H-A, Tran M-T (2020) Lifelog moment retrieval with self-attention based joint embedding model. In: CLEF (working notes)

Download references

Funding

This publication has emanated from research supported in party by research grants from Science Foundation Ireland Centre for Research Training under grant numbers 18/CRT/6223 and 18/CRT/6224.

Author information

Author notes

    Authors and Affiliations

    1. Dublin City University, Dublin, Ireland

      Ly-Duyen Tran, Manh-Duy Nguyen & Liting Zhou

    2. AISIA Research Laboratory, Ho Chi Minh, Vietnam

      Binh T. Nguyen

    3. VNU HCM - University of Science, Ho Chi Minh, Vietnam

      Binh T. Nguyen

    Authors
    1. Ly-Duyen Tran
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Manh-Duy Nguyen
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Binh T. Nguyen
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Liting Zhou
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Binh T. Nguyen.

    Ethics declarations

    Conflict of Interests

    The authors have no competing interests to declare that are relevant to the content of this article.

    Additional information

    Publisher’s note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Ly-Duyen Tran and Manh-Duy Nguyen are contributed equally to this work.

    Rights and permissions

    Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Tran, LD., Nguyen, MD., Nguyen, B.T. et al. Myscéal: a deeper analysis of an interactive lifelog search engine. Multimed Tools Appl 82, 37789–37806 (2023). https://doi.org/10.1007/s11042-023-15078-6

    Download citation

    • Received:

    • Revised:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s11042-023-15078-6

    Keywords

    Search

    Navigation