Security Journal

, Volume 28, Issue 3, pp 272–289 | Cite as

The future of video analytics for surveillance and its ethical implications

  • Andrew A AdamsEmail author
  • James M Ferryman
Original Article


The current state of the art and direction of research in computer vision aimed at automating the analysis of CCTV images is presented. This includes low level identification of objects within the field of view of cameras, following those objects over time and between cameras, and the interpretation of those objects’ appearance and movements with respect to models of behaviour (and therefore intentions inferred). The potential ethical problems (and some potential opportunities) such developments may pose if and when deployed in the real world are presented, and suggestions made as to the necessary new regulations which will be needed if such systems are not to further enhance the power of the surveillers against the surveilled.


surveillance CCTV video analytics ethics regulation computer vision 



This work was supported by the UK's EPSRC (EP/G069808/1) and Japan's JSPS (Kakenhi (B) 24330127). This work was supported by the European Union project ARENA (FP7-SEC-2010-1: 261658). However, this paper does not necessarily represent the opinion of the European Community, and the European Community is not responsible for any use which may be made of its contents.


  1. Adams, A.A. (2007) Regulating CCTV. Pages 3–14 of: Ethicomp 2007: Proceedings of the Ninth International Conference. Tokyo, Japan: Meiji University.Google Scholar
  2. Adams, A.A., Murata, K. and Orito, Y. (2010) The development of Japanese data protection. Policy and Internet 2 (2): 93–124.CrossRefGoogle Scholar
  3. Annesley, J., Colombo, A., Orwell, J. and Velastin, S.A. (2007) A profile of MPEG-7 for visual surveillance. In: Proceedings of the Fourth IEEE Advanced Video and Signal-Based Surveillance (AVSS), London, IEEE Computer Society: Online, doi:10.1109/AVSS.2007.4425358, pp. 482–487.Google Scholar
  4. BBC. (2003) (February). Pensioner freed after FBI bungle,
  5. BBC. (2006) (January). Peeping tom CCTV workers jailed,
  6. BBC. (2012) High-def CCTV cameras risk backlash, warns UK watchdog, 3 October,
  7. Bazzani, L., Cristani, M., Perina, M. and Murino, V. (2012) Multiple-shot person re-identification by chromatic and epitomic analyses. Pattern Recognition Letters 33 (7): 898–903.CrossRefGoogle Scholar
  8. Ben Shitrit, H., Berclaz, J., Fleuret, F. and Fua, P. (2011) Tracking multiple people under global appearance constraints. In: D.N. Metaxas, L. Quan, A. Sanfeliu and L.J. Van Gool (eds.) 2011 IEEE International Conference On, Computer Vision (ICCV), Barcelona. IEEE Computer Society, Online, doi:10.1109/ICCV.2011.6126235, pp. 137–144.Google Scholar
  9. Berger, A.M. (2000) Privacy mode for acquisition cameras and camcorders. US Patent 6067399, Sony Corporation, 23 May.Google Scholar
  10. Brin, D. (1998) The Transparent Society. Jackson, TN: Perseus.Google Scholar
  11. Cezar, J., Jacques, S. and Musse, S.R. (2010) Crowd analysis using computer vision techniques. IEEE Signal Processing Magazine 27 (September): 66–77.Google Scholar
  12. Coetzer, B., van der Merwe, J. and Josephs, B. (2011) Information management and video analytics: The future of intelligent video surveillance. In: W. Lin (ed.) Video Surveillance. Rijeka, Croatia: InTech, pp. 3–16.Google Scholar
  13. Coudert, F., Gemo, M., Beslay, L. and Andritsos, F. (2011) Pervasive monitoring: Appreciating citizen's surveillance as digital evidence in legal proceedings. Imaging for Crime Detection and Prevention (2011) (ICDP 2011), 4th International Conference On, London. IEEE Computer Society, Online, doi:10.1049/ic.2011.0130, pp. 1–6.Google Scholar
  14. Dee, H. and Hogg, D. (2004) Detecting inexplicable behaviour. In: A. Hoppe, S. Barman and T. Ellis (eds). Proceedings of the British Machine Vision Conference, The British Machine Vision Association, Kingston, London. British Machine Vision Association (BMVA), pp. 73–81.Google Scholar
  15. Dee, H.M. and Hogg, D.C. (2009) Navigational strategies in behaviour modelling. Artificial Intelligence 173 (2): 329–342.Google Scholar
  16. Dee, H.M. and Velastin, S.A (2008) How close are we to solving the problem of automated visual surveillance? Machine Vision and Applications: 19 (5–6): 329–343.CrossRefGoogle Scholar
  17. Dick, A.R. and Brooks, M.J. (2003) Issues in automated visual surveillance. In: C. Sun, H. Talbot, S. Ourselin and T . Adriaansen (eds.). Proceedings 7th International Conference on Digital Image Computing: Techniques and Applications, DICTA'03. Macquarie University, Sydney, Australia. CSIRO Publishing, pp. 195–204, 16–12 December.Google Scholar
  18. Dollár, P., Wojek, C., Schiele, B. and Perona, P. (2012) Pedestrian detection: An evaluation of the state of the art. Pattern Analysis and Machine Intelligence (PAMI) 34 (4): 743–761.CrossRefGoogle Scholar
  19. Donald, C. (2005) How many monitors should a CCTV operator view? CCTV Image. Spring, p. 35.Google Scholar
  20. Dufaux, F. and Ebrahimi, T. (2008) Scrambling for privacy protection in video surveillance systems. IEEE Transactions on Circuits and Systems for Video Technology 18 (8): 1168–1174.CrossRefGoogle Scholar
  21. Evans, M., Boyle, J.N. and Ferryman, J. (2012) Vehicle classification using evolutionary forests. In: P.L. Carmona, J.S. Sánchez and A.L.N. Fred (eds.) Proceedings of the International Conference on Pattern Recognition Applications and Methods, Vilamoura Portugal. SciTe Press, pp. 387–393.Google Scholar
  22. Florence, J. (2006) Making the no fly list fly: A due process model for terrorist watchlists. Yale Law Journal 115 (8): 2148–2181.CrossRefGoogle Scholar
  23. Gong, S., Loy, C.C. and Xiang, T. (2011) Security and Surveillance. New York: Springer.CrossRefGoogle Scholar
  24. Goold, B.J. (2002) Privacy rights and public spaces: CCTV and the problem of the ‘Unobservable observer’. Criminal Justice Ethics 21 (1): 21–27.CrossRefGoogle Scholar
  25. Helbing, D. and Molnar, P. (1995) Social force model for pedestrian dynamics. Physical Review E 51 (5): 4282–4286.CrossRefGoogle Scholar
  26. Hempel, L. and Töpfer, E. (2004) CCTV in Europe (Final Report). Technical Report, Urban Eye Project: HPSE-CT2001-00094.Google Scholar
  27. Hu, W., Tan, T., Wang, L. and Maybank, S (2004) A survey on visual surveillance of object motions and behaviour. IEEE Transactions on Systems, Man, and Cybernetics 34 (3): 334–252.CrossRefGoogle Scholar
  28. Independent Police Complaints Commission (UK). (2007) Stockwell One: Investigation into the shooting of Jean Charles de Menezes at Stockwell Underground Station on 22 July 2005, 8 November.Google Scholar
  29. Introna, L.D. (2000) Workplace surveillance, privacy and distributive justice. ACM SIGCAS Computers and Society 30 (4): 33–39.CrossRefGoogle Scholar
  30. Introna, L.D. and Wood, D. (2004) Picturing algorithmic surveillance: The politics of facial recognition systems. Surveillance and Society 2 (2/3): 177–198.Google Scholar
  31. Jia, K. and Gong, S. (2008) Generalized face super-resolution. IEEE Transactions on Image Processing 17 (6): 873–886.CrossRefGoogle Scholar
  32. Ko, T. (2011) Video surveillance. InTech. Chap. A survey on behaviour analysis in video surveillance applications.
  33. Lee, S.-W., Park, J. and Lee, S.-W. (2006) Low resolution face recognition based on support vector data description. Pattern Recognition 39 (9): 1809–1812.CrossRefGoogle Scholar
  34. Lewis, P. (2009) Ian Tomlinson death: Guardian video reveals police attack on man who died at G20 protest. The Guardian, 7 April.
  35. Liu, C., Shum, H.-Y. and Freeman, W.T. (2007) Face hallucination: Theory and practice. International Journal of Computer Vision 75 (1): 115–134.CrossRefGoogle Scholar
  36. Lyall, S. (2009) Critics assail British Police for harsh tactics during the G-20 Summit Meeting. New York Times, 30 May.
  37. (2011) Global video surveillance market, applications and management services forecasts (2010–2015), 1 January,
  38. Mehran, R., Moore, E.R. and Shah, M. (2010) A streakline representation of flow in crowded scenes. In: European Conference on Computer Vision (ECCV), Berlin: Springer, 5–11 September. Heraklion, Greece.Google Scholar
  39. Moussaid, M., Perozo, N., Garnier, S., Helbing, D. and Theraulaz, G. (2010) The walking behaviour of pedestrian social groups and its impact on crowd dynamics. PLoS ONE 5 (4): e10047.Google Scholar
  40. Norris, C. and Armstrong, G. (1999) The Maximum Surveillance Society: The Rise of CCTV. Oxford, UK: Berg.Google Scholar
  41. Orrite, C., Martinez, F., Herrero, E., Ragheb, H. and Velastin, S. (2008) Independent viewpoint silhouette-based human action modelling and recognition. In: Proceedings of the 1st International Workshop on Machine Learning for Vision-based Motion Analysis in Conjunction with the European Conference on Computer Vision (ECCV), 17 October. Marseille, France.Google Scholar
  42. Park, J.-S. and Lee, S.-W. (2008) An example-based face hallucination method for single-frame, low-resolution facial images. IEEE Transactions on Image Processing 17 (10): 1806–1816.CrossRefGoogle Scholar
  43. Patino, L., Bremond, F., Evans, M., Shahrokni, A. and Ferryman, J. (2010) Video activity and reporting with incremental unsupervised learning. In: Proceedings Seventh IEEE Advanced Video and Signal-Based Surveillance (AVSS), Sophia Antipolis, France. IEEE Computer Society: Online doi: 10.1109/AVSS.2010.74, pp. 511–518.Google Scholar
  44. PETS. (2006) Proceedings of the IEEE 2006 Ninth International Workshop on Performance Evaluation of Tracking and Surveillance. 18 June. Scholar
  45. PETS. (2007) Proceedings of the IEEE 2007 Tenth International Workshop on Performance Evaluation of Tracking and Surveillance. 14 October.
  46. PETS. (2009) Proceedings of the IEEE 2009 Twelfth International Workshop on Performance Evaluation of Tracking and Surveillance. 25 June. Scholar
  47. Pham, Q.-C., Gond, L., Begard, J., Allezard, N. and Sayd, P. (2007) Real time posture analysis in a crowd using thermal imaging. In: Proceedings Computer Vision and Pattern Recognition (CVPR’07), Minneapolis. IEEE Computer Society: Online doi: 10.1109/CVPR.2007.383496, pp. 1–8.Google Scholar
  48. Ragheb, H., Velastin, S., Remagnino, P. and Ellis, T. (2008) ViHASi: Virtual human action silhouette data for the evaluation of silhouette-based action recognition methods. In: Proceedings of the ACM International Workshop on Vision Networks for Behaviour Analysis in conjunction with the ACM Multimedia (MM), 31 October. Vancouver, Canada. Piscataway, NJ: IEE.Google Scholar
  49. Reeves, B. and Nass, C. (2002) The Media Equation, 2nd edn. Stanford, CA: CSLI.Google Scholar
  50. Rowlands, M. (2011) Statewatch analysis – Six months on: An update on the UK coalition government's commitment to civil liberties, 21 January,
  51. Senior, A. et al (2005) Enabling video privacy through computer vision. IEEE Security and Privacy 3 (3): 50–57.CrossRefGoogle Scholar
  52. Sochman, J. and Hogg, D.C. (2011) Who knows who – Inverting the social force model for finding groups. In: Proceedings IEEE International Workshop on Socially Intelligent Surveillance and Monitoring (SISM), Barcelona, Spain. IEEE Computer Society: Online, doi: 10.1109/1CCVW.2011.6130338, pp. 830–837.Google Scholar
  53. Spindler, T., Wartmann, C., Hovestadt, L., Roth, D., Van Gool, L. and Steffen, A. (2008) Privacy in video surveilled spaces. Computer Security 16 (2): 199–222.CrossRefGoogle Scholar
  54. The Economist. (2012) (28 April). Video surveillance: I spy with my big eye,
  55. Tickner, A.H. and Poulton, E.C. (1973) Monitoring up to 16 synthetic television pictures showing a great deal of movement. Ergonomics 16 (4): 381–401.CrossRefGoogle Scholar
  56. Tweed, D. and Ferryman, J. (2008) Enhancing change detection in low-quality surveillance video with markov random fields. In: Proceedings of the 1st ACM International Workshop on Vision Networks for Behaviour Analysis (VNBA’08) (in conjunction with the ACM Multimedia Conference). 31 October. Vancouver, Canada. New York: ACM.Google Scholar
  57. Valera, M., Velastin, S., Ellis, A-L. and Ferryman, J. (2011) Communication mechanisms and middleware for distributed video surveillance. IEEE Transactions on Circuits and Systems for Video Technology 21 (12): 1795–1809.CrossRefGoogle Scholar
  58. Welsh, B.P. and Farrington, D.P. (2009) Public area CCTV and crime prevention: An updated systematic review and meta-analysis. Justice Quarterly 26 (4): 716–745.CrossRefGoogle Scholar
  59. Yilmaz, A., Javed, O. and Shah, M. (2006) Object tracking: A survey. ACM Computing Surveys 38 (4): 1–45.CrossRefGoogle Scholar
  60. Zaidenberg, S., Boulay, B. and Bremond, F. (2012) A generic framework for video understanding applied to group behaviour recognition. In: Proceedings of the IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance (AVSS 2012), Beijing, China. IEEE Computer Society: Online, doi: 10.1109/AVSS.2012.1, pp. 126–142.Google Scholar
  61. Zhan, B., Monekosso, D.N., Remagnino, P., Velastin, S.A. and Xu, L.Q. (2008) Crowd analysis: A survey. Machine Vision and Applications 19 (5–6): 345–357.CrossRefGoogle Scholar

Copyright information

© Palgrave Macmillan, a division of Macmillan Publishers Ltd 2013

Authors and Affiliations

  1. 1.Centre for Business Information Ethics, Meiji UniversityTokyo-ToJapan
  2. 2.Computational Vision Group, School of Systems Engineering, University of ReadingReadingUK

Personalised recommendations