Robust Adversarial Defence: Use of Auto-inpainting

Abstract

Adversarial patch attacks have become a primary concern in recent years as they pose a significant threat to the security and reliability of deep neural networks. Modifying benign images by introducing adversarial patches comprising localized adversarial pixels alters the salient features of the image resulting in misclassification. The novelty of our approach is in the use of image inpainting technique as an adversarial defence for rectifying the patch region. Adversarial patch is automatically localized using Fast Score Class Activation Map and superseded by inpainting using Fast Marching Method which efficiently propagates pixel information from the surrounding areas into the patch region. This approach ensures original image’s structural integrity while simultaneously inpainting the adversarial pixels. Moreover, at the time of the attack it is not expected to have prior knowledge about the patch. Therefore, we propose our novel adversarial defence technique in a black-box setting assuming no knowledge about the patch location, shape or its size. Furthermore, we do not rely on re-training our victim model on adversarial examples, indicating its potential usefulness for real-world applications. Our experimental results show that the proposed approach achieves accuracy up to 76.37% on ImageNet100 despite the adversarial patch attack amounting to a considerable improvement of 76.28% points. Moreover, on benign images our approach gives decent accuracy of $81.11\%$ thereby suggesting that our defence pipeline is applicable irrespective of whether the input image is adversarial or clean.

S. Sharma and R. Joshi—Co-first authors.