Towards Cognitive and Perceptive Video Systems

  • Toygar Akgun
  • Charles Attwood
  • Andrea Cavallaro
  • Christian Fabre
  • Fabio PoiesiEmail author
  • Piotr Szczuko


In this chapter we cover research and development issues related to smart cameras. We discuss challenges, new technologies and algorithms, applications and the evaluation of today’s technologies. We will cover problems related to software, hardware, communication, embedded and distributed systems, multi-modal sensors, privacy and security. We also discuss future trends and market expectations from the customer’s point of view.


Graphic Processing Unit Market Expectation Smart Camera Memory Consistency Model Globally Asynchronous Locally Synchronous 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work has been partially funded by the Artemis JU and partially by TÜBİTAK—The Scientific and Technological Research Council of Turkey (Toygar Akgun), the UK Technology Strategy Board (Charles Attwood, Andrea Cavallaro, Fabio Poiesi), French Ministère de l’économie, du redressement productif et du numérique (Christian Fabre) and Polish National Centre for Research and Development (Piotr Szczuko) as part of the COPCAMS project ( under Grant Agreement number 332913.


  1. 1.
    Dollar P, Wojek C, Schiele B, Perona P (2012) Pedestrian detection: an evaluation of the state of the art. IEEE Trans Pattern Anal Mach Intell 34(4):743–761CrossRefGoogle Scholar
  2. 2.
    Koblar V, Filipic B (2013) Designing a quality-control procedure for commutator manufacturing. In: Proceedings of multiconference information society, Ljubljana, Slovenia, Oct 2013, pp 55–58Google Scholar
  3. 3.
    Stalder S, Grabner H, Van Gool L (2010) Cascaded confidence filtering for improved tracking-by-detection. In: Proceedings of European conference on computer vision, Crete, Greece, Sept 2010, pp 369–382Google Scholar
  4. 4.
    Abdelkader MF, Chellappa R, Zheng Q, Chan AL (2006) Integrated motion detection and tracking for visual surveillance. In: Proceedings of the computer visionsystems, New York, pp 28–34Google Scholar
  5. 5.
    Kiryati N, Raviv TR, Ivanchenko Y, Rochel S (2008) Real-time abnormal motion detection in surveillance video. In: Proceedings of international conference on patternrecognition, Tampa, Dec 2008, pp 1–4Google Scholar
  6. 6.
    Bhargava M, Chen CC, Ryoo MS, Aggarwal JK (2009) Detection of object abandonment using temporal logic. Mach Vis Appl 20(5):271–281CrossRefGoogle Scholar
  7. 7.
    Smith K, Quelhas P, Gatica-Perez D (2006) Detecting abandoned luggage items in a public space. In: Proceedings of computer vision and pattern recognition, workshop on performance evaluation of tracking and surveillance, New York, June 2006, pp 75–82Google Scholar
  8. 8.
    Szwoch G, Dalka P, Czyzewski A (2010) A framework for automatic detection of abandoned luggage in airport terminal. In: Tsihrintzis GA, Damiani E, Virvou M, Howlett RJ, Jain LC (eds) Smart innovation, systems and technologies, intelligent interactive multimedia systems and services. Springer, Heidelberg, pp 13–22CrossRefGoogle Scholar
  9. 9.
    Chen X, Yang SX (2013) A practical solution for ripe tomato recognition and localization. J Real-Time Image Proc 8(1):35–51CrossRefGoogle Scholar
  10. 10.
    What is GPU accelerated computing? (2014) Last accessed June 2014
  11. 11.
  12. 12.
    Melpignano D et al (2012) Platform 2012, a many-core computing accelerator for embedded SoCs: performance evaluation of visual analytics applications. In: Proceedings of design automation conference, San Francisco, June 2012, pp 1137–1142Google Scholar
  13. 13.
    Wilson R et al (2014) A 460mhz at 397mv, 2.6ghz at 1.3v, 32b vliw dsp, embedding fmax tracking. In: Proceedings of solid-state circuits conference digest of technical papers, San Francisco, Feb 2014, pp 452–453Google Scholar
  14. 14.
    Meincke T et al (1999) Globally asynchronous locally synchronous architecture for large high-performance ASICs. In: Proceedings of circuits and systems, Orlando, July 1999, pp 512–515Google Scholar
  15. 15.
    Smart camera NI (2014) Last accessed June 2014
  16. 16.
    Belbachir AN, Nabil A (eds) (2010) Smart cameras. Springer, New YorkGoogle Scholar
  17. 17.
    MALI OpenCL SDK (2014) Last accessed June 2014
  18. 18.
    Open Computing Language (2014) Last accessed June 2014
  19. 19.
    Gaster B, Howes L, Kaeli DR, Mistry P, Schaa D (eds) (2011) Heterogeneous computing with OpenCL. Elsevier, AmsterdamGoogle Scholar
  20. 20.
    Munshi A, Gaster B, Mattson TG, Fung J, Ginsburg D (eds) (2011) OpenCL programming guide. Addison-Wesley Professional, New DelhiGoogle Scholar
  21. 21.
    Scarpino M (ed) (2011) OpenCL in action: how to accelerate graphics and computations. Manning Publications, WalthamGoogle Scholar
  22. 22.
    Jaja J (ed) (1992) Introduction to parallel algorithms. Addison-Wesley Professional, ReadingzbMATHGoogle Scholar
  23. 23.
    Kirk DB, Hwu WW (eds) (2012) Programming massively parallel processors: a hands-on approach. Morgan Kaufmann, San FranciscoGoogle Scholar
  24. 24.
    Roosta SH (ed) (2000) Parallel processing and parallel algorithms: theory and computation. Springer, New YorkGoogle Scholar
  25. 25.
    Fabre C et al (2013) PRO3D, programming for future 3D manycore architectures: project interim status. Formal Methods Compon Objects Lect Notes Comput Sci 7542:277–293Google Scholar
  26. 26.
    A parallel computing platform and programming model (2014) Last accessed June 2014
  27. 27.
    A specification for parallel programming (2014) Last accessed June 2014
  28. 28.
    Lepley T, Paulin P, Flamand E (2013) A novel compilation approach for image processing graphs on a many-core platform with explicitly managed memory. In: Proceedings of compilers, architecture and synthesis for embedded systems, Montreal, pp 1–10Google Scholar
  29. 29.
    Llopard I, Cohen A, Fabre C, Hili N (2014) A parallel action language for embedded applications and its compilation flow. In: Proceedings of software and compilers for embedded systems, St. Goar, June 2014, pp 118–127Google Scholar
  30. 30.
    Guler P, Emeksiz D, Temizel A, Teke M, Temizel T (2013) Real-time multi-camera video analytics system on GPU. J Real-Time Image Process 8(4):389–401CrossRefGoogle Scholar
  31. 31.
    SanMiguel JC, Micheloni C, Shoop K, Foresti GL, Cavallaro A (2014) Self-reconfigurable smart camera networks. IEEE Comput 47(5):67–73CrossRefGoogle Scholar
  32. 32.
    Camera Link Standard (2014) Last accessed June 2014
  33. 33.
    Open Network Video Interface Forum (2014) Last accessed June 2014
  34. 34.
    The Physical Security Interoperability Alliance (2014) Last accessed June 2014
  35. 35.
    OmniCast (2014) Last accessed June 2014
  36. 36.
    Corsi C (2014) Infrared: a key technology for security systems. In: Baldini F et al (eds) Sensors, lecture notes in electrical engineering, vol 162. Springer, Heidelberg, pp 37–42Google Scholar
  37. 37.
    Houben Q, Czyz J, Tocino D, Debeir O, Warzee N (2009) Feature-based stereo vision using smart cameras for traffic surveillance. In: Fritz M, Schiele B, Piater JH (eds) Computer vision systems. Springer, Heidelberg, pp 144–153CrossRefGoogle Scholar
  38. 38.
    Wang Y, Kato J (2012) Integrated pedestrian detection and localization using stereo cameras. In: Hansen J, Boyraz P, Takeda K, Abut H (eds) Signal processing for in-vehicle systems and safety. Springer, Heidelberg, pp 229–238CrossRefGoogle Scholar
  39. 39.
    Mittal A, Davis LS (2002) M2Tracker: a multi-view approach to segmenting and tracking people in a cluttered scene using region-based stereo. In: Proceedings of European conference on computer vision, Copenhagen, May 2002, pp 18–22Google Scholar
  40. 40.
    Szwoch G, Dalka P, Czyzewski A (2013) Spatial calibration of a dual PTZ-fixed camera system for tracking moving objects in video. J Imaging Sci Technol 57(2):1–10CrossRefGoogle Scholar
  41. 41.
    Haering N, Venetianer PL, Lipton A (2008) The evolution of video surveillance: an overview. Mach Vis Appl 19(5–6):279–290CrossRefzbMATHGoogle Scholar
  42. 42.
    Lopatka K, Kotus J, Czyzewski A (2011) Application of vector sensors to acoustic surveillance of a public interior space. Arch Acoust 36(4):851–860CrossRefGoogle Scholar
  43. 43.
    Arguedas VF, Zhang Q, Izquierdo E (2014) Multimodal fusion in surveillance applications. In: Ionescu B, Benois-Pineau J, Piatrik T, Qunot G (eds) Fusion in computer vision. Springer, Heidelberg, pp 161–184CrossRefGoogle Scholar
  44. 44.
    Kotus J (2010) Application of passive acoustic radar to automatic localization, tracking and classification of sound sources. In: Proceedings of information technology, Gdansk, June 2010, pp 67–70Google Scholar
  45. 45.
    Kotus J, Lopatka K, Czyzewski A (2014) Detection and localization of selected acoustic events in acoustic field for smart surveillance applications. Multimed Tools Appl 68(1):5–21CrossRefGoogle Scholar
  46. 46.
    Cisco Gunshot Location Surveillance (2014) Last accessed June 2014
  47. 47.
    SST, Shotspotter Flex (2014) Last accessed June 2014
  48. 48.
    Lopatka K, Kotus J, Czyzewski A (2014) Evaluation of sound event detection, classification and localization in the presence of background noise for acoustic surveillance of hazardous situations. In: Andrzej D, Andrzej C (eds) Multimedia communications, services and security communications in computer and information science, vol 429. Springer, pp 96–110Google Scholar
  49. 49.
    Microflown (2014) Last accessed June 2014
  50. 50.
    Wind J, de Bree H-E, Buye X (2010) 3D sound source localization and sound mapping using a PU sensor array. In: Proceedings of AIAA/CEAS aeroacoustics, Stockholm, June 2010Google Scholar
  51. 51.
    Kellermann W (2008) Beamforming for speech and audio signals. In: Havelock D, Kuwano S, Vorlander M (eds) Handbook of signal processing in acoustics, vol 691–702. Springer, HeidelbergGoogle Scholar
  52. 52.
    Cichowski J, Czyzewski A (2011) Reversible video stream anonymization for video surveillance systems based on pixels relocation and watermarking. In: Proceedings of international conference on computer vision workshops, Barcelona, Nov 2011, pp 1971–1977Google Scholar
  53. 53.
    Helten F, Fischer B (2004) Reactive attention: video surveillance in Berlin shopping malls. Surveill Soc 2(2/3):323–345Google Scholar
  54. 54.
    Cavallaro A (2007) Privacy in video surveillance. IEEE Signal Process Mag 24(2):166–168CrossRefMathSciNetGoogle Scholar
  55. 55.
    Dalka P (2012) Multi-camera vehicle tracking using local image features and neural networks. In: Andrzej D, Andrzej C (eds) Multimedia communications, services and security. Communications in computer and information science, vol 287. Springer, Heidelberg, pp 58–67Google Scholar
  56. 56.
    Hamdoun O, Moutarde F, Stanciulescu B, Steux B (2008) Person re-identification in multi-camera system by signature based on interest point descriptors collected on short video sequences. In: Proceedings of distributed smart cameras, Stanford, Sept 2008, pp 1–6Google Scholar
  57. 57.
    D’Arminio P et al (2012) Technologies for granting balance between security and privacy in video-surveillance. In: Proceedings of intelligence and security informatics, Arlington, VA, Aug. 2012, pp 278–283Google Scholar
  58. 58.
    Dalka P, Bratoszewski P (2013) Visual detection of people movement rules violation in crowded indoor scenes. In: Andrzej D, Andrzej C (eds) Multimedia communications, services and security. Communications in computer and information science, vol 368. Springer, Berlin, pp 48–58Google Scholar
  59. 59.
    Szczuko P (2014) Augmented reality for privacy-sensitive visual monitoring. In: Dziech A, Czyzewski A (eds) Multimedia communications, services and security. Communications in computer and information science, vol 429. Springer, Switzerland, pp 229–241Google Scholar
  60. 60.
    Kato Z, Zerubia J (2012) Markov random fields in image segmentation. Found Trends Sig Process 5(1–2):1–155Google Scholar
  61. 61.
  62. 62.

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Toygar Akgun
    • 1
  • Charles Attwood
    • 2
  • Andrea Cavallaro
    • 3
  • Christian Fabre
    • 4
    • 5
  • Fabio Poiesi
    • 3
    Email author
  • Piotr Szczuko
    • 6
  1. 1.ASELSAN Elektronik Sanayi Ve Ticaret A.S.Yenimahalle AnkaraTurkey
  2. 2.THALES UK Limited, Research and Technology, Worton DriveReadingUK
  3. 3.Queen Mary University of LondonLondonUK
  4. 4.CEA, LETI, MINATEC CampusGrenobleFrance
  5. 5.Université Grenoble AlpesGrenobleFrance
  6. 6.Faculty of Electronics, Telecommunications and InformaticsGdansk University of TechnologyGdanskPoland

Personalised recommendations