Report on NSF/ARO/ONR Workshop on Distributed Camera Networks: Research Challenges and Future Directions

Abstract

Large-scale video networks are becoming increasingly important for a wide range of critical applications. The development of automated techniques for aggregating and interpreting information from multiple video streams in large-scale networks in real-life scenarios is very challenging. Research in video sensor networks is highly interdisciplinary and requires expertise from a variety of fields. The goal of this effort was to organize a two-day nationally recognized workshop in the domain of camera networks that brings together leading researchers from academia, industry and the government. The workshop was held at the University of California at Riverside on May 11–12, 2009. The workshop was attended by 75 participants. The workshop was sponsored by the US National Science Foundation, US Army Research Office and US Office of Naval Research. The workshop addressed critical interdisciplinary challenges at the intersection of large-scale video camera networks and distributed sensing, processing, communication and control; distributed video understanding; embedded real-time systems; graphics and simulation; and education. The recommendations of the workshop are summarized in the following order of topics:

• A. Video Processing and Video Understanding

• B. Simulation, Graphics, Cognition and Video Networks

• C. Wireless Video Sensor Networks, Communications and Control

• D. Distributed Embedded Cameras and Real Time Video Analysis

• E. Applications

• F. Educational Opportunities and Curriculum Development

We include the workshop report in this book with the hope that it will catalyze a research agenda and interdisciplinary collaborations in large-scale camera networks—an emerging field with significant and exciting scientific and technological challenges, which will set the stage for potentially transformative breakthroughs.

The workshop website is http://videonetworks2009.cs.ucr.edu/.

Appendices

Appendix 1: List of Attendees in Alphabetical Order

Note: All the abstracts by participants are available on the workshop website: http://videonetworks2009.cs.ucr.edu/.

J.K. Aggarwal, The University of Texas at Austin

Abstract: Sensor Networks and Recognition of Human Activities

Hamid Aghajan, Stanford University

Abstract: Vision: Ideas for Enabling Ambient Intelligence and Serving Social Networks

Narendra Ahuja, UIUC

Abstract: Image Representation for Image Understanding

Ian Akyildiz, Georgia Tech

Abstract: Research Challenges for Wireless Multimedia Sensor Networks

Rick Allan, BBN

Kevin Almeroth, UC-Santa Barbara

Norman Badler, UPenn

Abstract: Simulating a Functional Populace

Ruzena Bajcsy, UCBerkeley

Abstract: Active Perception: Past, Present, and Future

Azer Bestavros, Boston University

Abstract: Virtualization and Programming Support for Video Sensor Networks

Bir Bhanu, UCR

Abstract: VideoWeb: Design of a Wireless Camera Network for Real-time Monitoring of Activities

Paul Brewer, ObjectVideo

Abstract: Maritime Persistent Surveillance

Mark Campbell, Cornell University

Abstract: Sensing, Cooperation and Control in Networked Systems

Rama Chellappa, UMD

Abstract: Exploiting Sparsity, Geometry and Statistics for Video Processing and Understanding Using a Camera Network

Jie Chen, University of California

Abstract: Best Achievable Tracking Performance Under Limited and Constrained Information Feedback

Hui Cheng, Sarnoff Corporation

Abstract: Real-Time Entity Tracking Using Wide Area Surveillance Videos

Bojan Cukic, West Virginia University

Abstract: Fault Tolerant Sensor Networks for Border Activity Detection

Liyi Dai, US Army Research Office

Raju Damarla, US Army Research Laboratory

Larry Davis, UMD

Abstract: Event Modeling and Recognition in Camera Networks

Jim Davis, Ohio State University

Abstract: Control, Registration, and Exploitation of Video Sensor Networks

Sudhir Dixit, Center for Internet Excellence

Abstract: Challenges of Wireless Communication in Video Sensing

Chuck Dyer, University of Wisconsin-Madison

Abstract: Tracking in Camera Networks Using Joint Projective Invariants

Jay Farrell, U of CA Riverside

John Fisher, MIT CSAIL

Abstract: Information-driven Inference Under Resource Constraints

Arun Hampapur, IBM

Abstract: Media Analytics Research at IBM

Mary Ann Harrison, WVHTC Foundation

Abstract: Tactical Analysis of Video Imagery

Parag Havaldar, SONY Pictures, Imageworks

Abstract: Performance Driven Character Animation on a Real Production Set Using Multiple Cameras

Glenn Healey, UC Irvine

Abstract: Processing Multispectral/Hyperspectral Video

Tom Henderson, University of Utah

Abstract: Cognitive Video Sensor Networks

Jenq-Neng Hwang, University of Washington, Seattle

Abstract: Intelligent Surveillance and Event Understanding in Distributed Embedded Camera Networks of a Large Scale Community

Ted Isaacson, ONR/MTCSC

Ioannis Kakadiaris, University of Houston

Abstract: Face Recognition: Your Face is Your Password

Behzad Kamgar-Parsi, ONR

Aggelos Katsaggelos, Northwestern University

Abstract: Content-based Scalable Video Streaming in Video Sensor Networks

Iouri Kompaniets, Physical Optics Corporation

Abstract: Advanced Hyperspectral Zoom Optics Sensor

Rick Kremer, Logostech

Martin Kruger, Office of Naval Research

Vinod Kulathumani, West Virginia University

Abstract: Collaborative Face Recognition Using Network of Smart Cameras

Mohan Kumar, The University of Texas at Arlington

Abstract: Collaborative Virtual Observation (CoVO) in Dynamic Environments

Aditi Majumder, UCI

Abstract: Ubiquitous Displays via a Distributed Framework of Projector-Camera Systems

Bogdan Matei, Sarnoff Corporation

Abstract: Multi-Camera Tracking and Data Association

Shean T. McMahon, Physical Optics Corporation

Abstract: Advanced Hyperspectral Zoom Optics System

Gerard Medioni, University of Southern California

Abstract: Distributed Vision: Algorithms, Scalability, Forensics

Sharad Mehrotra, UCI

Abstract: SATWARE: A Semantic Enhanced Middleware and Database System for Sentient Spaces

Scott Midkiff, National Science Foundation

Vassilios Morellas, UMN

Abstract: Linking Video Information Across Cameras

Thinh Nguyen, Oregon State University

Abstract: Towards Building a Robust Video Sensor Network

Antonio Ortega, University of Southern California

Abstract: Challenges in Practical Distributed Video Compression

Robert Pless, Washington University in St. Louis

Abstract: Passive Vision: Observing the World While Sitting Still

T.V. Prabhakar, Indian Institute of Science

Abstract: An Energy Efficient Distributed Algorithm for Image Slicing in WSNs

Ranjit Pradhan, Physical Optics Corporation R.R. Venkatesha Prasad, Delft University of Technology

Abstract: An Energy Efficient Distributed Algorithm for Image Slicing in Wireless Sensor Networks

Faisal Qureshi, UOIT

Abstract: Proactive Camera Control for Collaborative Sensing

V. Ramesh, Siemens

Abstract: Robust Video Understanding Systems—Review and Challenges

Raghuveer Rao, Army Research Laboratory

Chinya Ravishankar, UC Riverside

Ray Rimey, Lockheed Martin

Abstract: Recognizing Activity Structures in Massive Numbers of Simple Events Over Large Areas

Amit Roy Chowdhury, University of California, Riverside

Abstract: Video Understanding in Distributed Active Camera Networks

Maya Rubeiz, Office of Naval Research

Venkatesh Saligrama, Boston University

Abstract: Video Analytics in Multicamera Networks

Wes Snyder, NCSU

Abstract: Shape Recognition Based on Accumulators

Stefano Soatto, UCLA

Abstract: From Shannon to Gibson: Actionable Information in Video

Bi Song, UCR

Abstract: Scene Analysis, Control and Communication in Distributed Camera Networks

Tanveer Syeda-Mahmood, IBM Almaden Research Center

Demetri Terzopoulos, University of California, Los Angeles

Satish Tripathi, University at Buffalo, SUNY

Mohan Trivedi, University of California at San Diego

Abstract: Distributed Video Networks in Intelligent Vehicles and Transportation Systems

Yanghai Tsin, Siemens Corporate Research

Peter Tu, GE

Abstract: Video Analytics from Homeland Protection to Theft Prevention and Crowd Analysis

Pramod Varshney, Syracuse University

Abstract: Sensor Selection and Information Fusion in Sensor Networks

Rene Vidal, Johns Hopkins University

Abstract: Distributed Scene Understanding

Wayne Wolf, Georgia Tech

Abstract: Distributed Smart Cameras

Ming-Hsuan Yang, University of California at Merced

Abstract: Visual Tracking with Online Multiple Instance Learning

Allen Y. Yang, UCB

Abstract: Multiple-View Object Recognition in Band-Limited Distributed Camera Networks

Appendix 2: Groups and Group Leaders

Group 1: Video Processing and Video Understanding

Group Co-Leaders: Rama Chellappa (UMD), Gerard Medioni (USC)

Participants:

J.K. Aggarwal also group 2

Narendra Ahuja also group 6

Robert Pless

Vinod Kulathumani

Amit Roy Chowdhury also group 3

Stefano Soatto also group 2

Rene Vidal

Ming-Hsuan Yang

Bir Bhanu

Jim Davis also group 2

Larry Davis also group 6

V. Ramesh also group 4

Wes Snyder also group 4

Bi Song

Yanghai Tsin

Allen Y. Yang also group 4

Group 2: Simulation, Graphics, Cognition and Video Networks

Group Co-Leaders: Demetri Terzopoulos (UCLA), Chuck Dyer (U. Wisconsin)

Participants:

Ruzena Bajcsy also group 6

Jim Davis also group 1

Ioannis Kakadiaris

Aditi Majumder

Faisal Qureshi

Avideh Zakhor also group 6

J.K. Aggarwal

Norman Badler

Parag Havaldar also group 5

Tom Henderson also group 6

Stefano Soatto also group 1

Group 3: Wireless Video Sensor Networks, Communications and Control

Group Co-Leaders: Antonio Ortega (USC), Venkatesh Saligrama (BU)

Participants:

Ian Akyildiz

Jie Chen

Mohan Kumar

Thinh Nguyen

Pramod Varshney also group 6

Mark Campbell

Raju Damarla

Jay Farrell

John Fisher also group 6

Aggelos Katsaggelos also group 6

Amit Roy Chowdhury also group 1

Satish Tripathi

Group 4: Distributed Embedded Cameras and Real Time Video Analysis

Group Co-Leaders: Hamid Aghajan (Stanford), Paul Brewer (Object Video)

Participants:

Jenq-Neng Hwang also group 6

Sharad Mehrotra also group 6

Ray Rimey also group 5

Wes Snyder also group 1

Mohan Trivedi also group 6

Wayne Wolf (phone/internet)

Allen Y. Yang also group 1

Rick Allan

Azer Bestavros

Rick Kremer

V. Ramesh also group 1

Raghuveer Rao

Group 5: Applications

Group Co-Leaders: Glenn Healey (UCI), Bogdan Matei (Sarnoff)

Participants:

Bojan Cukic

Mary Ann Harrison

Iouri Kompaniets

R. Prasad/T. Prabhakar

Parag Havaldar also group 2

Ted Isaacson

Shean T. McMahon

Ranjit Pradhan

Ray Rimey also group 4

Peter Tu also group 6

Group 6: Educational Opportunities and Curriculum Development

Group Co-Leaders: Sharad Mehrotra (UCI), Jenq-Neng Hwang (UW)

Participants:

Narendra Ahuja also group 1

Ruzena Bajcsy also group 2

Larry Davis also group 1

John Fisher also group 3

Tom Henderson also group 2

Aggelos Katsaggelos also group 3

Mohan Trivedi also group 4

Peter Tu also groups 5

Pramod Varshney also group 3

Avideh Zakhor also group 2

Appendix 3: Talks with Titles and Presenters

Plenary Presentations

Past, Present and Future of Active Vision, Ruzena Bajcsy (UCB)

Past, Present and Future of Motion Analysis, J.K. Aggarwal (UT Austin)

Event Modeling and Recognition in Camera Networks, Larry Davis (UMD)

Robust Video Understanding Systems—Review and Challenges, V. Ramesh (Siemens)

Simulating a Functional Populace, Norm Badler (UPenn)

Information-Driven Inference under Resource Constraints, John Fisher (MIT)

Research Challenges for Wireless Multimedia Sensor Networks, Ian Akyildiz (GTech)

Content-based Scalable Video Streaming in Video Sensor Networks, Aggelos Katsaggelos (Northwestern)

Sensing, Cooperation and Control in Networked Systems, Mark Campbell (Cornell)

snBench: Virtualization and Programming Support for Video Sensor Networks, Azer Bestavros (BU)

Video Analytics from Homeland Protection to Theft Prevention and Crowd Analysis, Peter Tu (GE)

VideoWeb: A Network of Wireless Video Cameras, Bir Bhanu (UCR)

Education Opportunities: Computer Science Perspective, Thomas Henderson (UU)

Education Opportunities: Electrical Engineering Perspective, Narendra Ahuja (UIUC)

ONR Programs, Martin Kruger (ONR)

NSF Opportunities, Scott Midkiff (NSF)

ARO Opportunities, Raghuveer Rao (ARO)

Group Presentations to the Entire Workshop and Discussions

Group 1: Video Processing and Video Understanding

Group 2: Simulation, Graphics, Cognition and Video Networks

Group 3: Wireless Video Sensor Networks, Communications and Control

Group 4: Distributed Embedded Cameras and Real Time Video Analysis

Group 5: Applications

Group 6: Educational Opportunities and Curriculum Development

Recommendation by the Groups for Research Directions and Educational Opportunities/Curriculum Development

Group Presentations and Discussions

Group 1: Video Processing and Video Understanding

Exploiting Sparsity, Geometry and Statistics for Video Processing and Understanding Using a Camera Network, Rama Chellappa (UMD)

Distributed Vision: Algorithms, Scalability, Forensics, Gerard Medioni (USC)

Presentations by other group participants (see abstracts on the website)

Group 2: Simulation, Graphics, Cognition and Video Networks

Virtual Vision: A Simulation Framework for Camera Sensor Networks Research, Demetri Terzopoulos (UCLA)

Tracking in Camera Networks Using Joint Projective Invariants, Chuck Dyer (U. Wisconsin)

Presentations by other group participants (see abstracts on the website)

Group 3: Wireless Video Sensor Networks, Communications and Control

Video Analytics in Multicamera Networks, Venkatesh Saligrama (BU)

Challenges in Practical Distributed Video Compression, Antonio Ortega (USC)

Presentations by other group participants (see abstracts on the website)

Group 4: Distributed Embedded Cameras and Real Time Video Analysis

Vision: Ideas for Enabling Ambient Intelligence and Serving Social Networks, Hamid Aghajan (Stanford)

Maritime Persistent Surveillance, Paul Brewer (Object Video)

Presentations by other group participants (see abstracts on the website)

Group 5: Applications

Processing Multispectral/Hyperspectral Video, Glenn Healey (UCI)

Multi-Camera Tracking and Data Association, Bogdan Matei (Sarnoff)

Presentations by other group participants (see abstracts on the website)

Group 6: Educational Opportunities and Curriculum Development

Educational Opportunities and Curriculum Development, Sharad Mehrotra (UCI), Jenq-Neng Hwang (UW)

Presentations by other group participants (see abstracts on the website)