A Multi-agent Architecture Based on the BDI Model for Data Fusion in Visual Sensor Networks

  • Federico CastanedoEmail author
  • Jesús García
  • Miguel A. Patricio
  • José M. Molina


The newest surveillance applications is attempting more complex tasks such as the analysis of the behavior of individuals and crowds. These complex tasks may use a distributed visual sensor network in order to gain coverage and exploit the inherent redundancy of the overlapped field of views. This article, presents a Multi-agent architecture based on the Belief-Desire-Intention (BDI) model for processing the information and fusing the data in a distributed visual sensor network. Instead of exchanging raw images between the agents involved in the visual network, local signal processing is performed and only the key observed features are shared. After a registration or calibration phase, the proposed architecture performs tracking, data fusion and coordination. Using the proposed Multi-agent architecture, we focus on the means of fusing the estimated positions on the ground plane from different agents which are applied to the same object. This fusion process is used for two different purposes: (1) to obtain a continuity in the tracking along the field of view of the cameras involved in the distributed network, (2) to improve the quality of the tracking by means of data fusion techniques, and by discarding non reliable sensors. Experimental results on two different scenarios show that the designed architecture can successfully track an object even when occlusions or sensor’s errors take place. The sensor’s errors are reduced by exploiting the inherent redundancy of a visual sensor network with overlapped field of views.


Distributed visual sensor networks Multi-agent Systems Data fusion 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Moeslund, T., Hilton, A., Krüger, V.: A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104(2–3), 90–126 (2006)CrossRefGoogle Scholar
  2. 2.
    Patricio, M.A., Carbo, J., Perez, O., Garcia, J., Molina, J.M.: Multi-agent framework in visual sensor networks. EURASIP J. Appl. Signal Process. 2007, Article ID 98639, 21 (2007)Google Scholar
  3. 3.
    Wooldridge, M., Jennings, N.: Intelligent agents: theory and practice. Knowl. Eng. Rev. 10(2), 115–152 (1995)CrossRefGoogle Scholar
  4. 4.
    Berge-Cherfaoui, V., Vachon, B.: A multi-agent approach of the multi-sensor fusion. In: Fifth International Conference on Robots in Unstructured Environments, pp. 1264–1259 (1991)Google Scholar
  5. 5.
    Multimedia Object Descriptions Extraction from Surveillance Tapes. Accessed 18 September 2009.
  6. 6.
    FIPA, Fipa communicative act library specification. Accessed 18 September 2009.
  7. 7.
    Botelho, L., Lopes, R., Sequeira, M., Almeida, P., Martins, S.: Inter-agent communication in a FIPA compliant intelligent distributed dynamic-information system. In: Proc. 5th International Conference on Information Systems Analysis and Synthesis (ISAS99) (1999)Google Scholar
  8. 8.
    Graf, T., Knoll, A.: A Multi-Agent System Architecture for Distributed Computer Vision. Int. J. Artif. Intell. Tools 9(2), 305–319 (2000)CrossRefGoogle Scholar
  9. 9.
    Remagnino, P., Shihab, A., Jones, G.: Distributed intelligence for multi-camera visual surveillance. Pattern Recogn. 37(4), 675–689 (2004)CrossRefGoogle Scholar
  10. 10.
    Abreu, B., Botelho, L., Cavallaro, A., Douxchamps, D., Ebrahimi, T., Figueiredo, P., Macq, B., Mory, B., Nunes, L., Orri, J. et al.: Video-based multi-agent traffic surveillance system. In: Intelligent Vehicles Symposium. Proceedings of the IEEE, pp. 457–462 (2000)Google Scholar
  11. 11.
    Orwell, J., Massey, S., Remagnino, P., Greenhill, D., Jones, G.A.: A multi-agent framework for visual surveillance. In: Proceedings of the 10th International Conference on Image Analysis and Processing, p. 1104. IEEE Computer Society, Washington, USA (1999)CrossRefGoogle Scholar
  12. 12.
    Matsuyama, T., Hiura, S., Wada, T., Murase, K., Toshioka, A.: Dynamic memory: architecture for real time integration of visual perception, camera action, and network communication. In: IEEE Conference on Computer Vision and Pattern Recognition. Proceedings, vol. 2 (2000)Google Scholar
  13. 13.
    Ukita, N.: Real-time cooperative multi-target tracking by dense communication among active vision agents. In: IEEE/WIC/ACM International Conference on Intelligent Agent Technology, pp. 664–671 (2005)Google Scholar
  14. 14.
    Bakhtari, A., Mackay, M., Benhabib, B.: Active-Vision for the Autonomous Surveillance of Dynamic, Multi-Object Environments. J. Intell. Robot. Syst. 54(4), 567–593 (2009)CrossRefGoogle Scholar
  15. 15.
    Marchesotti, L., Piva, S., Regazzoni, C.: An agent-based approach for tracking people in indoor complex environments. In: 12th International Conference on Image Analysis and Processing, 2003, pp. 99–102 (2003)Google Scholar
  16. 16.
    Nguyen, N.T., Venkatesh, S., West, G., Bui, H.H.: Multiple camera coordination in a surveillance system. Acta. Automatica Sinica 29(3), 408–421 (2003)Google Scholar
  17. 17.
    Garcia, J., Molina, J.M., Besada, J.A., Portillo, J.I.: A multitarget tracking video system based on fuzzy and neuro-fuzzy techniques. EURASIP J. Appl. Signal Process. 2005, 2341–2358 (2005)zbMATHCrossRefGoogle Scholar
  18. 18.
    Patricio, M.A., Castanedo, F., Berlanga, A., Perez, O., Garcia, J., Molina, J.M.: Computational intelligence in visual sensor networks: improving video processing systems. In: Computational Intelligence in Multimedia Processing: Recent Advances Series: Studies in Computational Intelligence, vol. 96. ISBN: 978-3-540-76826-5 (2008)Google Scholar
  19. 19.
    Patricio, M.A., Garcia, J., Berlanga, A., Molina, J.M.: Solving video-association problem with explicit evaluation of hypothesis using EDAs. In: Proceedings of the 2008 Congress on Evolutionary Computation, CEC (2008)Google Scholar
  20. 20.
    Sanchez, A.M., Patricio, M.A., Garcia, J., Molina, J.M.: Video tracking improvement using context-based information. In: 10th International Conference on Information Fusion (2007)Google Scholar
  21. 21.
    Tittle, J.S., Todd, J.T.: Perception of three-dimensional structure. In: Arbib, M.A. (ed.) The handbook of brain theory and neural networks, pp. 715–718. MIT Press, Cambridge, MA (1998)Google Scholar
  22. 22.
    Quevedo, A., Maravall, D.: Wheelchair tracking system in Hospital’s Corridors by means of video-surveillance systems. Technical report. Artificial Intelligence Department, Polytechnic University of Madrid (2003)Google Scholar
  23. 23.
    Castanedo, F., Patricio, M.A., Garcia, J., Molina, J.M.: Robust data fusion in a visual sensor multi-agent architecture. In: 10th International Conference on Information Fusion (2007)Google Scholar
  24. 24.
    Castanedo, F., Garcia, J., Patricio, M.A., Molina, J.M: Analysis of distributed fusion alternatives in coordinated vision agents. In: 11th International Conference on Information Fusion (2008)Google Scholar
  25. 25.
    Tsai, R.: A versatile camera calibration technique for high accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Autom. 3(4), 323–344 (1987)CrossRefGoogle Scholar
  26. 26.
    Heikkila, J.: Geometric camera calibration using circular control points. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1066–1077 (2000)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)CrossRefGoogle Scholar
  28. 28.
    Kettnaker, V., Zabih, R.: Bayesian multi-camera surveillance. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 253–259 (1999)Google Scholar
  29. 29.
    FIPA, ACL Message Structure Specification. Accessed 18 September 2009.
  30. 30.
    Rao, A.S., Georgeff, M.P.: BDI-agents: from theory to practice. In: Proceedings of the First International Conference on Multiagent Systems, San Francisco (1995)Google Scholar
  31. 31.
    Bratman, M.: Intentions, Plans and Practical Reasoning. Harvard University Press, Cambridge, Massachusetts (1987)Google Scholar
  32. 32.
    Dennett, D.: The Intentional Stance. Bradford Books (1987)Google Scholar
  33. 33.
    Jo, C., Chen, G., Choi, J.: A new approach to the BDI agent-based modeling. In: Proceedings of the 2004 ACM symposium on Applied Computing, pp. 1541–1545, 13–17 March 2004Google Scholar
  34. 34.
    Georgeff, M., Pell, B., Pollack, M., Tambe, M., Wooldridge, M.: The belief-desire-intention model of agency. In: Proceedings of the 5th International Workshop on Intelligent Agents V: Agent Theories, Architectures, and Languages (ATAL-98), vol. 1555, pp. 1–10 (1998)Google Scholar
  35. 35.
    Georgeff, M., Lansky, A.: Reactive reasoning and planning. In: Proceedings of the Sixth National Conference on Artificial Intelligence (AAAI-87), pp. 677–682 (1987)Google Scholar
  36. 36.
    Ingrand, F., Georgeff, M., Rao, A.: An architecture for real-time reasoning and system control. In: Expert, IEEE [see also IEEE Intelligent Systems and Their Applications], vol. 7, no. 6, pp. 34–44 (1992)Google Scholar
  37. 37.
    Busetta, P., Ronnquist, R., Hodgson, A., Lucas, A.: JACK intelligent agents-components for intelligent agents in java. AgentLink Newsletter 2, 2–5 (1999)Google Scholar
  38. 38.
    Huber, M.: JAM: a BDI-theoretic mobile agent architecture. In: Proceedings of the third annual conference on Autonomous Agents, pp. 236–243 (1999)Google Scholar
  39. 39.
    Kinny, D., Phillip, R.: Building composite applications with goal-directed (TM) agent technology. AgentLink News. 16, 6–8 (2004)Google Scholar
  40. 40.
    Pokahr, A., Braubach, L., Lamersdorf, W.: Jadex: a BDI reasoning engine. In: Multi-Agent Programming: Languages, Platforms and ApplicationsSpringer, Berlin (2005)Google Scholar
  41. 41.
    Pokahr, A., Braubach, L., Lamersdorf, W.: Jadex: implementing a BDI infrastructure for jade agents. In: Search of Innovation (Special Issue on JADE), vol. 3, pp. 76–85 (2003)Google Scholar
  42. 42.
    OpenCV, Accessed 18 September 2009.
  43. 43.
    JNI, Java Native Method Invocation Specification v1.1, (1997)Google Scholar
  44. 44.
    Mills, D.: Internet time synchronization: the network time protocol. IEEE Trans. Commun. 39(10), 1482–1493 (1991)CrossRefGoogle Scholar
  45. 45.
    Bratman, M.E., Israel, D.J., Pollack, M.E.: Plans and resource-bounded practical reasoning. Comput. Intell. 4(3), 349–355 (1988)CrossRefGoogle Scholar
  46. 46.
    Apidis, Accessed 18 September 2009.
  47. 47.
    Weng, J., Cohen, P., Herniou, M.: Camera calibration with distortion models and accuracy evaluation. IEEE Trans. Pattern Anal. Mach. Intell. 14(10), 965–980 (1992)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  • Federico Castanedo
    • 1
    Email author
  • Jesús García
    • 1
  • Miguel A. Patricio
    • 1
  • José M. Molina
    • 1
  1. 1.Applied Artificial Intelligence Group, Department of Computer ScienceUniversity Carlos III of MadridMadridSpain

Personalised recommendations