Abstract
The task of predicting a movie genre from its poster can be very challenging owing to the high variability of movie posters. A novel approach for the generation of a multi-label movie genre prediction from its poster using neural networks that employ knowledge transfer learning has been proposed in this paper. This approach works on two fronts; one is aimed at creating a large, diverse and balanced dataset for movie genre prediction. The second front involves reframing the problem to simpler single-label multi-class classification and generating a multi-label multi-class prediction on a given movie poster as input. The experimental evaluation suggests that our approach generates a remarkable accuracy which is a result of a larger, evenly distributed dataset, simplifying the problem to a single-label multi-class classification problem and because of the use of knowledge transfer learning to extract higher-level feature.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and material
All relevant data and material are presented in the main paper.
References
Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow IJ, Harp A, Irving G, Isard M, Jia Y, Józefowicz R, Kaiser L, Kudlur M, Levenberg J, Mane D, Monga R, Moore S, Murray DG, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker PA, Vanhoucke V, Vasudevan V, Viégas FB, Vinyals O, Warden P, Wattenberg, Wicke M, Yu Y, Zheng X (2016) Tensor flow: large-scale machine learning on heterogeneous distributed systems, pp 1–19
Chu W, Guo H (2017) Movie genre classification based on poster images with deep neural networks. In: MUSA2, proceedings of the workshop on multimodal understanding of social, affective and subjective attributes. ACM, New York, NY, pp 39–45
Hui-Yu H, Weir-Sheng S, Wen-Hsing H (2008) A film classifier based on low-level visual features. J Multimed. https://doi.org/10.4304/jmm.3.3.26-33
Huang H, Shih W, Hsu WH (2008) Film classification based on low-level visual effect features. J Electron Imaging 17(2):023011
Huang YF, Wang SH (2012) Movie genre classification using SVM with audio and video features. In: Huang R, Ghorbani AA, Pasi G, Yamaguchi T, Yen NY, Jin B (eds) Active media technology. AMT 2012. Lecture Notes in Computer Science, 7669. Springer, Berlin, pp 1–10
Ivasic-Kos M, Pobar M, Mikec L (2014) Movie posters classification into genres based on low-level features. In: 37th international convention on information and communication technology, electronics and microelectronics (MIPRO), IEEE, pp 1198–1203
Jha K, Doshi A, Patel P, Shah M (2019) A comprehensive review on automation in agriculture using artificial intelligence. Artif Intell Agric 2:1–12
Kakkad V, Patel M, Shah M (2019) Biometric authentication and image encryption for image security in cloud framework. Multiscale Multidiscip Model Exp Des. https://doi.org/10.1007/s41939-019-00049-y
Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Part of: Advances in neural information processing systems. NIPS 2012, vol 25, pp 1–9
Krupel B (2016) Judging a movie by its poster using deep learning. Stanford CS221, pp 1–5
Lu J, Behbood V, Hao P, Zuo H, Xue S, Zhang G (2015) Transfer learning using computational intelligence: a survey. Knowl-Based Syst 80:14–23
Nebauer C (1998) Evaluation of convolutional neural networks for visual recognition. IEEE Trans Neural Netw 9(4):685–696
Pandya R, Nadiadwala S, Shah R, Shah M (2019) Buildout of methodology for meticulous diagnosis of K-complex in EEG for aiding the detection of Alzheimer’s by artificial intelligence. Augment Human Res. https://doi.org/10.1007/s41133-019-0021-6
Rasheed Z, Shah M (2002) Movie genre classification by exploiting audio-visual features of previews. In: Object recognition supported by user interaction for service robots, vol 2, pp 1086–1089
Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. In: IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, HI, pp 6517–6525
Shah G, Shah A, Shah M (2019) Panacea of challenges in real-world application of big data analytics in healthcare sector. Data Inf Manag. https://doi.org/10.1007/s42488-019-00010-1
Tsoumakas G, Vlahavas I (2007) Random k-labelsets: an ensemble method for multilabel classification. In: Kok JN, Koronacki J, Mantaras RL, Matwin S, Mladenič D, Skowron A (eds) Machine learning: ECML 2007. ECML 2007. Lecture Notes in Computer Science, vol 4701. Springer, Berlin, pp 406–417
Wehrmann J, Barros RC (2017) Movie genre classification: a multi-label approach based on convolutions through time. Appl Soft Comput 61:973–982
Wehrmann J, Mattjie A, Barros RC (2018) Order embeddings and character-level convolutions for multimodal alignment. Pattern Recognit Lett 102:15–22
Zhang ML, Zhou ZH (2007) Ml-knn: a lazy learning approach to multi-label learning. Pattern Recognit 40(7):2038–2048
Zhou B, Lapedriza A, Xiao J, Torralba A, Oliva A (2014) Learning deep features for scene recognition using places database. In: Advances in neural information processing systems, pp 487–495
Zhou H, Hermans T, Karandikar AV, Rehg JM (2010) Movie genre classification via scene categorization. In: Proceedings of the international conference on multimedia, pp 747–750
Acknowledgements
The authors are grateful to Indus University and School of Technology, Pandit Deendayal Petroleum University for the permission to publish this research.
Funding
Not Applicable.
Author information
Authors and Affiliations
Contributions
All the authors make substantial contribution in this manuscript. KK, YP, and MS participated in drafting the manuscript. KK and YP wrote the main manuscript, and all the authors discussed the results and implication on the manuscript at all stages.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no competing interests.
Consent for publication
Not applicable.
Ethics approval and consent to participate
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kundalia, K., Patel, Y. & Shah, M. Multi-label Movie Genre Detection from a Movie Poster Using Knowledge Transfer Learning. Augment Hum Res 5, 11 (2020). https://doi.org/10.1007/s41133-019-0029-y
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s41133-019-0029-y