Skip to main content
SpringerLink
Log in
Book cover

Iberian Conference on Pattern Recognition and Image Analysis

IbPRIA 2022: Pattern Recognition and Image Analysis pp 602–615Cite as

Film Shot Type Classification Based on Camera Movement Styles

  • 1494 Accesses

Part of the Lecture Notes in Computer Science book series (LNCS,volume 13256)

Abstract

Visual information contains the most important characteri-stics of a movie regarding the related content and filming techniques. Especially the way the camera moves to capture the scene is vital to define the director’s aesthetics. However, most of the machine learning tasks existing in the literature treat the movie as shallow content, rather than as an artistic work, and therefore focus on detecting objects and faces, recognizing activities and extracting plot-related topics. On the other hand, cinematography is closely connected to the choice of different ways to handle the camera, and thus camera movements include information that is useful in order to analyse the artistic style of a movie. In this work we present an original, publicly available (https://github.com/magcil/movie_shot_classification_dataset) dataset for film shot type classification that is associated with the distinction across 10 types of camera movements that cover the vast majority of types of shots in real movies. In addition, two different methods are evaluated on the new dataset, one static that is based on feature statistics across frames, and one sequential that tries to predict the target class based on the input frame sequence using LSTMs. Based on the evaluation process it is inferred that the sequential method is more suited for modeling the camera movements.

Keywords

  • Shot classification
  • Camera movement classification
  • Movie analysis

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://github.com/tyiannak/multimodal_movie_analysis.

  2. 2.

    https://github.com/theopsall/video_annotator.

  3. 3.

    https://github.com/tyiannak/multimodal_movie_analysis.

  4. 4.

    https://github.com/magcil/movie_shot_classification_dataset.

References

  1. Bak, H.Y., Park, S.B.: Comparative study of movie shot classification based on semantic segmentation. Appl. Sci. 10(10), 3390 (2020). https://doi.org/10.3390/app10103390, https://www.mdpi.com/2076-3417/10/10/3390

  2. Baraldi, L., Grana, C., Cucchiara, R.: Shot and scene detection via hierarchical clustering for re-using broadcast video. In: Azzopardi, G., Petkov, N. (eds.) CAIP 2015. LNCS, vol. 9256, pp. 801–811. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23192-1_67

    CrossRef  Google Scholar 

  3. Benini, S., Svanera, M., Adami, N., Leonardi, R., Kovács, A.B.: Shot scale distribution in art films. Multimedia Tools Appl. 75(23), 16499–16527 (2016). https://doi.org/10.1007/s11042-016-3339-9

    CrossRef  Google Scholar 

  4. Bhattacharya, S., Mehran, R., Sukthankar, R., Shah, M.: Classification of cinematographic shots using lie algebra and its application to complex event recognition. IEEE Trans. Multimedia 16(3), 686–696 (2014)

    CrossRef  Google Scholar 

  5. Bougiatiotis, K., Giannakopoulos, T.: Enhanced movie content similarity based on textual, auditory and visual information. Expert Syst. Appl. 96, 86–102 (2018)

    CrossRef  Google Scholar 

  6. Braudy, L.: Film: an international history of the medium. Film Q. (ARCHIVE) 48(3), 59 (1995)

    CrossRef  Google Scholar 

  7. Canini, L., Benini, S., Leonardi, R.: Classifying cinematographic shot types. Multimedia Tools Appl. 62(1), 51–73 (2013)

    CrossRef  Google Scholar 

  8. Choi, S.M., Ko, S.K., Han, Y.S.: A movie recommendation algorithm based on genre correlations. Expert Syst. Appl. 39(9), 8079–8085 (2012)

    CrossRef  Google Scholar 

  9. Diao, Q., Qiu, M., Wu, C.Y., Smola, A.J., Jiang, J., Wang, C.: Jointly modeling aspects, ratings and sentiments for movie recommendation (JMARS). In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 193–202 (2014)

    Google Scholar 

  10. Ertugrul, A.M., Karagoz, P.: Movie genre classification from plot summaries using bidirectional LSTM. In: 2018 IEEE 12th International Conference on Semantic Computing (ICSC), pp. 248–251. IEEE (2018)

    Google Scholar 

  11. Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: a search space odyssey. IEEE Trans. Neural Networks Learn. Syst. 28(10), 2222–2232 (2016)

    CrossRef  MathSciNet  Google Scholar 

  12. Haq, I.U., Muhammad, K., Hussain, T., Kwon, S., Sodanil, M., Baik, S.W., Lee, M.Y.: Movie scene segmentation using object detection and set theory. Int. J. Distrib. Sens. Networks 15(6), 1550147719845277 (2019)

    Google Scholar 

  13. Hasan, M.A., Xu, M., He, X., Xu, C.: CAMHID: camera motion histogram descriptor and its application to cinematographic shot classification. IEEE Trans. Circ. Syst. Video Technol. 24(10), 1682–1695 (2014). https://doi.org/10.1109/TCSVT.2014.2345933

    CrossRef  Google Scholar 

  14. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  15. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456. PMLR (2015)

    Google Scholar 

  16. Lekakos, G., Caravelas, P.: A hybrid approach for movie recommendation. Multimedia Tools Appl. 36(1), 55–70 (2008)

    CrossRef  Google Scholar 

  17. Li, K., Li, S., Oh, S., Fu, Y.: Videography-based unconstrained video analysis. IEEE Trans. Image Process. 26(5), 2261–2273 (2017). https://doi.org/10.1109/TIP.2017.2678800

    CrossRef  MathSciNet  MATH  Google Scholar 

  18. Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    CrossRef  Google Scholar 

  19. Lucas, B.D., Kanade, T., et al.: An iterative image registration technique with an application to stereo vision. Vancouver, British Columbia (1981)

    Google Scholar 

  20. Minhas, R.A., Javed, A., Irtaza, A., Mahmood, M.T., Joo, Y.B.: Shot classification of field sports videos using AlexNet convolutional neural network. Appl. Sci. 9(3), 483 (2019)

    CrossRef  Google Scholar 

  21. Park, S.C., Lee, H.S., Lee, S.W.: Qualitative estimation of camera motion parameters from the linear composition of optical flow. Pattern Recogn. 37(4), 767–779 (2004)

    CrossRef  Google Scholar 

  22. Psallidas, T., Koromilas, P., Giannakopoulos, T., Spyrou, E.: Multimodal summarization of user-generated videos. Appl. Sci. 11(11), 5260 (2021). https://doi.org/10.3390/app11115260, https://www.mdpi.com/2076-3417/11/11/5260

  23. Rao, A., et al.: A unified framework for shot type classification based on subject centric lens. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12356, pp. 17–34. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58621-8_2

    CrossRef  Google Scholar 

  24. Rasheed, Z., Shah, M.: Scene detection in Hollywood movies and tv shows. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Proceedings, vol. 2, p. II-343. IEEE (2003)

    Google Scholar 

  25. Sang, J., Xu, C.: Character-based movie summarization. In: Proceedings of the 18th ACM International Conference on Multimedia, pp. 855–858 (2010)

    Google Scholar 

  26. Savardi, M., Signoroni, A., Migliorati, P., Benini, S.: Shot scale analysis in movies by convolutional neural networks. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 2620–2624. IEEE (2018)

    Google Scholar 

  27. Simões, G.S., Wehrmann, J., Barros, R.C., Ruiz, D.D.: Movie genre classification with convolutional neural networks. In: 2016 International Joint Conference on Neural Networks (IJCNN), pp. 259–266. IEEE (2016)

    Google Scholar 

  28. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  29. Subramaniyaswamy, V., Logesh, R., Chandrashekhar, M., Challa, A., Vijayakumar, V.: A personalised movie recommendation system based on collaborative filtering. Int. J. High Perform. Comput. Networking 10(1–2), 54–63 (2017)

    CrossRef  Google Scholar 

  30. Tsai, C.M., Kang, L.W., Lin, C.W., Lin, W.: Scene-based movie summarization via role-community networks. IEEE Trans. Circ. Syst. Video Technol. 23(11), 1927–1940 (2013)

    CrossRef  Google Scholar 

  31. Ul Haq, I., Ullah, A., Muhammad, K., Lee, M.Y., Baik, S.W.: Personalized movie summarization using deep CNN-assisted facial expression recognition. In: Complexity 2019 (2019)

    Google Scholar 

  32. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 1, p. I-I. IEEE (2001)

    Google Scholar 

  33. Wang, H.L., Cheong, L.F.: Taxonomy of directing semantics for film shot classification. IEEE Trans. Circ. Syst. Video Technol. 19(10), 1529–1542 (2009). https://doi.org/10.1109/TCSVT.2009.2022705

    CrossRef  Google Scholar 

  34. Zhou, H., Hermans, T., Karandikar, A.V., Rehg, J.M.: Movie genre classification via scene categorization. In: Proceedings of the 18th ACM International Conference on Multimedia, pp. 747–750 (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Thessaly, Volos, Greece

    Antonia Petrogianni

  2. National Center for Scientific Research - Demokritos, Athens, Greece

    Panagiotis Koromilas & Theodoros Giannakopoulos

Authors
  1. Antonia Petrogianni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Panagiotis Koromilas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Theodoros Giannakopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Panagiotis Koromilas .

Editor information

Editors and Affiliations

  1. University of Aveiro, Aveiro, Portugal

    Armando J. Pinho

  2. University of Aveiro, Aveiro, Portugal

    Dr. Petia Georgieva

  3. University of Porto, Porto, Portugal

    Luís F. Teixeira

  4. Universitat Politècnica de València, Valencia, Spain

    Joan Andreu Sánchez

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Petrogianni, A., Koromilas, P., Giannakopoulos, T. (2022). Film Shot Type Classification Based on Camera Movement Styles. In: Pinho, A.J., Georgieva, P., Teixeira, L.F., Sánchez, J.A. (eds) Pattern Recognition and Image Analysis. IbPRIA 2022. Lecture Notes in Computer Science, vol 13256. Springer, Cham. https://doi.org/10.1007/978-3-031-04881-4_48

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-04881-4_48

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-04880-7

  • Online ISBN: 978-3-031-04881-4

  • eBook Packages: Computer ScienceComputer Science (R0)

search

Navigation