Crowd-aware Thresholded Loss for Object Detection in Wide Area Motion Imagery

Abstract

Detecting objects in Wide Area Motion Imagery (WAMI), an essential task for many practical applications, is particularly challenging in crowded scenes, such as areas with heavy traffic, since pixel resolutions of objects and ground sampling distance are highly compromised, and different factors disrupt visual signals. To address this challenge, we design a framework that combines preprocessing operations and deep detectors. To train deep networks for detection in WAMI for improved performance in especially crowded areas, we propose a novel crowd-aware thresholded loss (CATLoss) function. Moreover, we introduce a hard sampling mining method to strengthen the discriminative ability of the proposed solution. Additionally, we extend prior networks used in the literature using novel spatio-temporal cascaded architectures to incorporate more contextual information without introducing additional parameters. Overall, our approach is causal, more generalizable, and more robust even in reduced spatial sizes. On the WPAFB-2009 dataset, we show that our solution performs better than or on par with state-of-the-art without introducing any computational complexity during inference. The code and trained models will be released at (https://github.com/poyrazhatipoglu/CATLoss).

Appendix

1.1 A.1 Visual Outputs

See Fig. 11.

Fig. 11

A frame sample taken demonstrating AOI 04 and processing outputs. Foreground pixel regions are marked with the highest intensity value on a black-white image, as shown in (b). Target regions are shown with green rectangles on top of the ground truth points in (c) and (d). The red dots in (c) and (d) are placed to point out patches including detected objects and the centre of the objects after the localization operation, respectively. As we mark the centre of any patch encompassing any part of the targets, multiple red dots are observed in close proximity to each other in (c). However, unlike the layout shown in (c), only the estimated centres of the targets are indicated in (d)

1.2 A.2 Haversine Distance

The distance traveled between two consecutive frames and the speeds of targets can be calculated using the latitude ($\varphi$), and longitude ($\lambda$) coordinates of targets, and the time elapsed between two consecutive frames. To calculate the approximate distances between two points (($\varphi _1$, $\lambda _1$), ($\varphi _2$, $\lambda _2$)) on Earth’s surface, Haversine distance (Van Brummelen 2012) formulated in Eq. (9) is used.

$$\begin{aligned} \small d = 2r \arcsin \left(\sqrt{\sin ^2{\left(\frac{\varphi _2 -\varphi _1}{2}\right)}+ \cos {\varphi _1} \cos {\varphi _2} \sin ^2{\left(\frac{\lambda _2 -\lambda _1}{2}\right)}}\right) \end{aligned}$$

(9)

where r is the approximate Earth’s radius.