Skip to main content
SpringerLink
Log in

MultiMICS: a contextual multifaceted intelligent multimedia information fusion paradigm

  • S.I.: Multifaceted Intelligent Computing Systems (MICS)
  • Published:
Innovations in Systems and Software Engineering Aims and scope Submit manuscript

Abstract

Accomplishment in various aspects of life is associated with the balanced intelligence of human beings. Intelligence can be classified into several types, each of them has an influence on one’s individual and activity-related outcomes. Depending on the previous literature on intelligent systems, this article intends to trace the evolution of diverging types of multifaceted and multiple computational intelligence in the context of multimedia information framework; right from cognitive intelligence to naturalistic intelligence. We have illustrated a systematic schema of how the multiple and mutually exclusive intelligence categories affect the structural information processing phenomena. We have depicted a set of contextual case studies over the multimedia information patterns, such as images, linguistics, and music to enhance the capability of learning and representation. Thereby, distinctive characteristics of intelligence suggestively influence the entire multimedia information fusion schema in terms of multimedia pattern analysis, information retrieval, and recommendations. We have incorporated three diverge and semantic case studies for demonstrating the multifaceted intelligent information fusion on the multimedia contents such as (a) facial recognition system for auto-generated response management, (b) speech recognition module for response management and interactive system demonstration, and (c) music processing schema for illustrating pervasive music teaching–learning framework. We have evaluated the performance metrics for each of the three demonstrated case studies. The appraised outcome shows that our projected multimedia module-based information processing paradigm provides efficient system manifestation and is effectively capable of signifying multifaceted intelligent computing systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Thambu N, Prayitno HJ, Zakaria GAN (2021) Incorporating active learning into moral education to develop multiple intelligences: a qualitative approach. Indones J Learn Adv Educ (IJOLAE) 3(1):17–29. https://doi.org/10.23917/ijolae.v3i1.10064

    Article  Google Scholar 

  2. Cichocki A, Kuleshov AP (2021) Future trends for Human-AI collaboration: a comprehensive taxonomy of AI/AGI using multiple intelligences and learning styles. Comput Intell Neurosci. https://doi.org/10.1155/2021/8893795

    Article  Google Scholar 

  3. Xhomara N, Shkembi F (2020) The influence of multiple intelligences on learning styles in teaching and learning. J Appl Tech Educ Sci 10(1):19–48. https://doi.org/10.24368/jates.v10i1.148

    Article  Google Scholar 

  4. MacFarlane A, Missaoui S, Frankowska-Takhari S (2020) On machine learning and knowledge organization in multimedia information retrieval. KO KNOWLEDGE ORGANIZATION 47(1):45–55. https://doi.org/10.5771/0943-7444-2020-1-45

    Article  Google Scholar 

  5. Wang C, Fang T, Gu Y (2020) Learning performance and behavioral patterns of online collaborative learning: Impact of cognitive load and affordances of different multimedia. Comput Educ 143:103683. https://doi.org/10.1016/j.compedu.2019.103683

    Article  Google Scholar 

  6. Li N, Martin A, Estival R (2021) Heterogeneous information fusion: combination of multiple supervised and unsupervised classification methods based on belief functions. Inf Sci 544:238–265. https://doi.org/10.1016/j.ins.2020.07.039

    Article  MathSciNet  MATH  Google Scholar 

  7. Ren J, Hussain A, Han J, Jia X (2019) Cognitive modelling and learning for multimedia mining and understanding. Cogn Comput 11(6):761–762. https://doi.org/10.1007/s12559-019-09684-6

    Article  Google Scholar 

  8. Javed U, Shaukat K, Hameed IA, Iqbal F, Alam TM, Luo S (2021) A review of content-based and context-based recommendation systems. International Journal of Emerging Technologies in Learning (iJET), 16(3): 274–306. https://www.learntechlib.org/p/219036/

  9. Wang S, Zhang L, Yu M, Wang Y, Ma Z, Zhao Y (2021) Attribute-aware multi-task recommendation. J Supercomput 77(5):4419–4437. https://doi.org/10.1007/s11227-020-03440-6

    Article  Google Scholar 

  10. Zhu W, Wang X, Gao W (2020) Multimedia intelligence: When multimedia meets artificial intelligence. IEEE Trans Multimedia 22(7):1823–1835. https://doi.org/10.1109/TMM.2020.2969791

    Article  Google Scholar 

  11. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501. https://doi.org/10.1016/j.neucom.2005.12.126

    Article  Google Scholar 

  12. Xiao D, Li B, Mao Y (2017) A multiple hidden layers extreme learning machine method and its application. Math Probl Eng. https://doi.org/10.1155/2017/4670187

    Article  Google Scholar 

  13. Fan Q, Niu L, Kang Q (2020) Regression and multiclass classification using sparse extreme learning machine via smoothing group L 1/2 regularizer. IEEE Access 8:191482–191494. https://doi.org/10.1109/access.2020.3031647

    Article  Google Scholar 

  14. Sarkar SD, Ajitha Shenoy KB (2020) Face recognition using artificial neural network and feature extraction. In: 2020 7th International conference on signal processing and integrated networks, SPIN 2020, pp 417–422. https://doi.org/10.1109/SPIN48934.2020.9071378.

  15. Sarhan S, Nasr AA, Shams MY (2020) Multipose face recognition-based combined adaptive deep learning vector quantization. Comput Intell Neurosci. https://doi.org/10.1155/2020/8821868

    Article  Google Scholar 

  16. Zangeneh E, Rahmati M, Mohsenzadeh Y (2020) Low resolution face recognition using a two-branch deep convolutional neural network architecture. Expert Syst Appl 139:112854. https://doi.org/10.1016/j.eswa.2019.112854

    Article  Google Scholar 

  17. Agrawal A, Mittal N (2020) Using CNN for facial expression recognition: a study of the effects of kernel size and number of filters on accuracy. Vis Comput 36:405–412. https://doi.org/10.1007/s00371-019-01630-9

    Article  Google Scholar 

  18. Maity S, Roy S, De De (2021) Upasthiti: A Feature Learning-inspired Remote Attendance Management System, In: 2nd International Conference on Advanced Computing and Applications (ICACA-2021), Submission id: 183.

  19. Tao F, Busso C (2020) End-to-end audiovisual speech recognition system with multitask learning. IEEE Trans Multimedia 23:1–11. https://doi.org/10.1109/TMM.2020.2975922

    Article  Google Scholar 

  20. Jiang J, Wang HH (2021) Application intelligent search and recommendation system based on speech recognition technology. Int J Speech Technol 24(1):23–30. https://doi.org/10.1007/s10772-020-09703-0

    Article  Google Scholar 

  21. Anand PB, Nath R (2020) Content‐Based Recommender Systems. Recommender System with Machine Learning and Artificial Intelligence: Practical Tools and Applications in Medical, Agricultural and Other Industries pp 165–195. https://doi.org/10.1002/9781119711582.ch9

  22. Roy P, Roy S, De D (2020) TMIR: transient length extraction strategy for ANN-inspired musical instrument recognition. In: 2020 IEEE international women in engineering (WIE) conference on electrical and computer engineering (WIECON-ECE), IEEE, pp 267–271. https://doi.org/10.1109/WIECON-ECE52138.2020.9398035

  23. Beheshti A, Yakhchi S, Mousaeirad S, Ghafari SM, Goluguri SR, Edrisi MA (2020) Towards cognitive recommender systems. Algorithms 13(8):176. https://doi.org/10.3390/a13080176

    Article  Google Scholar 

  24. Roy S, Sarkar D, Hati S, De D (2018) Internet of music things: an edge computing paradigm for opportunistic crowdsensing. J Supercomput 74(11):6069–6101. https://doi.org/10.1007/s11227-018-2511-6

    Article  Google Scholar 

  25. Roy S, Sarkar D, De D (2020) Entropy-aware ambient IoT analytics on humanized music information fusion. J Ambient Intell Humaniz Comput 11(1):151–171. https://doi.org/10.1007/s12652-019-01261-x

    Article  Google Scholar 

  26. Roy S, Sarkar D, De D (2021) DewMusic: crowdsourcing-based internet of music things in dew computing paradigm. J Ambient Intell Human Comput 12:2103–2119. https://doi.org/10.1007/s12652-020-02309-z

    Article  Google Scholar 

  27. Wen X (2021) Using deep learning approach and IoT architecture to build the intelligent music recommendation system. Soft Comput 25(4):3087–3096. https://doi.org/10.1007/s00500-020-05364-y

    Article  Google Scholar 

  28. Roy S, Biswas M, De D (2020) iMusic: a session-sensitive clustered classical music recommender system using contextual representation learning. Multimedia Tools Appl 79:24119–24155. https://doi.org/10.1007/s11042-020-09126-8

    Article  Google Scholar 

  29. Magron P, Févotte C (2021) Leveraging the structure of musical preference in content-aware music recommendation. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp 581–585. IEEE. https://doi.org/10.1109/ICASSP39728.2021.9414194

  30. Roy S, Chakrabarty S, De D (2017) Time-based raga recommendation and information retrieval of musical patterns in Indian classical music using neural networks. IAES Int J Artif Intell 6(1):33–48. https://doi.org/10.11591/ijai.v6.i1.pp33-48

    Article  Google Scholar 

  31. Roy S, Mukherjee A, De D (2021) OrangeMusic: An orange computing-inspired recommender framework in internet of music things. Internet Technol Lett. https://doi.org/10.1002/itl2.331

    Article  Google Scholar 

  32. Andrade RM, Junior BRA, Oliveira PAM, Maia ME, Viana W, Nogueira TP (2021) Multifaceted infrastructure for self-adaptive IoT systems. Inf Softw Technol 132:106505. https://doi.org/10.1016/j.infsof.2020.106505

    Article  Google Scholar 

  33. Shah SH (2021) a review on matrix factorization techniques used for an intelligent recommender system. Turkish J Comput Math Educ (TURCOMAT) 12(7):1812–1823. https://doi.org/10.17762/turcomat.v12i7.3069

    Article  Google Scholar 

  34. Ramírez J, Flores MJ (2020) Machine learning for music genre: multifaceted review and experimentation with audioset. J Intell Inf Syst 55(3):469–499. https://doi.org/10.1007/s10844-019-00582-9

    Article  Google Scholar 

  35. Abbas A, Zhang L, Khan SU (2015) A survey on context-aware recommender systems based on computational intelligence techniques. Computing 97(7):667–690. https://doi.org/10.1007/s00607-015-0448-7

    Article  MathSciNet  Google Scholar 

  36. Ren Y, Ren Y, Li G, Zhou W (2011) Automatic generation of recommendations from data: a multifaceted survey. Deakin University, School of Information Technology

  37. Mali M, Mishra DS, Vijayalaxmi M (2020) Multifaceted recommender systems methods: a review. J Stat Manag Syst 23(2):349–361. https://doi.org/10.1080/09720510.2020.1736318

    Article  Google Scholar 

  38. Ren F, Dong Y, Wang W (2019) Emotion recognition based on physiological signals using brain asymmetry index and echo state network. Neural Comput Appl 31(9):4491–4501. https://doi.org/10.1007/s00521-018-3664-1

    Article  Google Scholar 

  39. Steiner P, Stone S, Birkholz P, Jalalvand A (2021) Multipitch tracking in music signals using echo state networks. In: 2020 28th European Signal Processing Conference (EUSIPCO), IEEE, pp 126–130. https://doi.org/10.23919/Eusipco47968.2020.9287638

  40. Roy S, Maity S, De D (2021) Data for: Upasthiti: A Feature Learning-inspired Remote Attendance Management System. Mendeley Data, V2. https://doi.org/10.17632/gcjh52j2j2.2

  41. Roy S, Maity S, De D (2021) Data for: MultiMICS. Mendeley Data, V1. https://doi.org/10.17632/k68gd2jwsw.1

Download references

Acknowledgements

The authors are grateful to the University Grants Commission (UGC), Govt. of India, for sanctioning research fellowship under which this article has been completed. Authors are also grateful to the Department of Science and Technology (DST) for sanctioning a research Projects and TEQIP-III, MAKAUT, WB, India.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Centre for Mobile Cloud Computing (CMCC), Maulana Abul Kalam Azad University of Technology, West Bengal (formerly known as, West Bengal University of Technology), BF-142, Sector-I, Salt Lake City, Kolkata, West Bengal, 700064, India

    Samarjit Roy, Satanu Maity & Debashis De

  2. University of Western Australia, 35 Stirling Hwy, Crawley, WA, 6009, Australia

    Debashis De

Authors
  1. Samarjit Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Satanu Maity
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Debashis De
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Samarjit Roy.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Roy, S., Maity, S. & De, D. MultiMICS: a contextual multifaceted intelligent multimedia information fusion paradigm. Innovations Syst Softw Eng (2022). https://doi.org/10.1007/s11334-022-00438-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11334-022-00438-6

Keywords

Search

Navigation