Abstract
Instead of occurring independently, semantic concepts pairs tend to co-occur within a single image and it is intuitive that concept detection accuracy for visual concepts can be enhanced if concept correlation can be leveraged in some way. In everyday concept detection for visual lifelogging using wearable cameras to automatically record everyday activities, the captured images usually have a diversity of concepts which challenges the performance of concept detection. In this paper a semantically smoothed refinement algorithm is proposed using concept correlations which exploit topic-related concept relationships, modeled externally in a user experiment rather than extracted from training data. Results for initial concept detection are factorized based on semantic smoothness and adjusted in compliance with the extracted concept correlations. Refinement performance is demonstrated in experiments to show the effectiveness of our algorithm and the extracted correlations.
This work was part-funded by 973 Program under Grant No. 2011CB302206, National Natural Science Foundation of China under Grant No. 61272231, 61472204, 61502264, Beijing Key Laboratory of Networked Multimedia and by Science Foundation Ireland under grant SFI/12/RC/2289.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aly, R., Hiemstra, D., de Jong, F., Apers, P.: Simulating the future of concept-based video retrieval under improved detector performance. Multimed. Tools Appl. 60(1), 1–29 (2011)
Gurrin, C., Smeaton, A., Doherty, A.: Lifelogging: personal big data. Found. Trends Inf. Retr. 8(1), 1–127 (2014)
Jiang, W., Chang, S.F., Loui, A.: Context-based concept fusion with boosted conditional random fields. In: ICASSP 2007, vol. 1, pp. I-949–I-952, April 2007
Jiang, Y.G., Dai, Q., Wang, J., Ngo, C.W., Xue, X., Chang, S.F.: Fast semantic diffusion for large-scale context-based image and video annotation. IEEE Trans. Image Proces. 21(6), 3080–3091 (2012)
Jin, Y., Khan, L., Wang, L., Awad, M.: Image annotations by combining multiple evidence & WordNet. In: ACM MULTIMEDIA 2005, pp. 706–715 (2005)
Lakoff, G.: Women, fire, and dangerous things. University of Chicago Press, April 1990
Lee, D., Seung, H.: Learning the parts of objects by nonnegative matrix factorization. Nature 401, 788–791 (1999)
Lee, H., Smeaton, A., O’Connor, N., Jones, G., Blighe, M., Byrne, D., Doherty, A., Gurrin, C.: Constructing a SenseCam visual diary as a media process. Multimed. Syst. 14(6), 341–349 (2008)
Pan, R., Zhou, Y., Cao, B., Liu, N.N., Lukose, R., Scholz, M., Yang, Q.: One-class collaborative filtering. In: ICDM 2008, pp. 502–511. IEEE (2008)
Qi, G.J., Hua, X.S., Rui, Y., Tang, J., Mei, T., Zhang, H.J.: Correlative multi-label video annotation. In: ACM MULTIMEDIA 2007, pp. 17–26 (2007)
Smeaton, A.F., Over, P., Kraaij, W.: Evaluation campaigns and trecvid. In: Proceedings of the ACM International Workshop on Multimedia Information Retrieval, pp. 321–330. ACM (2006)
Smeaton, A.F., Over, P., Kraaij, W.: High-level feature detection from video in TRECVid: a 5-year retrospective of achievements. In: Divakaran, A. (ed.) Multimedia Content Analysis, Theory and Applications, pp. 1–24. Springer, Boston (2009). http://dx.doi.org/10.1007/978-0-387-76569-3_6
Steve, H., Emma, B., Ken, W.: Sensecam: a wearable camera that stimulates and rehabilitates autobiographical memory. Memory 19(7), 685–96 (2011)
Wang, C., Jing, F., Zhang, L., Zhang, H.J.: Image annotation refinement using random walk with restarts. In: ACM MULTIMEDIA 2006, pp. 647–650 (2006)
Wang, P., Smeaton, A.: Using visual lifelogs to automatically characterise everyday activities. Inf. Sci. 230, 147–161 (2013)
Wang, P., Smeaton, A.F., Gurrin, C.: Factorizing time-aware multi-way tensors for enhancing semantic wearable sensing. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015. LNCS, vol. 8935, pp. 571–582. Springer, Switzerland (2015). doi:10.1007/978-3-319-14445-0_49
Wang, P., Smeaton, A.F.: Semantics-based selection of everyday concepts in visual lifelogging. Int. J. Multimed. Inf. Retr. 1(2), 87–101 (2012)
Wang, P., Sun, L., Yang, S., Smeaton, A.F.: Towards training-free refinement for semantic indexing of visual media. In: Tian, Q., Sebe, N., Qi, G.-J., Huet, B., Hong, R., Liu, X. (eds.) MMM 2016. LNCS, vol. 9516, pp. 251–263. Springer, Heidelberg (2016). doi:10.1007/978-3-319-27671-7_21
Wu, Y., Tseng, B., Smith, J.: Ontology-based multi-classification learning for video concept detection. In: ICME 2004, vol. 2, pp. 1003–1006 (2004)
Xue, X., Zhang, W., Zhang, J., Wu, B., Fan, J., Lu, Y.: Correlative multi-label multi-instance image annotation. In: ICCV 2011, pp. 651–658 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Wang, P., Sun, L., Yang, S., Smeaton, A.F. (2016). Semantically Smoothed Refinement for Everyday Concept Indexing. In: Chen, E., Gong, Y., Tie, Y. (eds) Advances in Multimedia Information Processing - PCM 2016. PCM 2016. Lecture Notes in Computer Science(), vol 9916. Springer, Cham. https://doi.org/10.1007/978-3-319-48890-5_31
Download citation
DOI: https://doi.org/10.1007/978-3-319-48890-5_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-48889-9
Online ISBN: 978-3-319-48890-5
eBook Packages: Computer ScienceComputer Science (R0)