Deep neural networks have achieved impressive results in many complex applications, including classification tasks for image and speech recognition, pattern analysis or perception in self-driving vehicles. However, it has been observed that even highly trained networks are very vulnerable to adversarial perturbations. Adding minimal changes to inputs that are correctly classified can lead to wrong predictions, raising serious security and safety concerns. Existing techniques for checking robustness against such perturbations only consider searching locally around a few individual inputs, providing limited guarantees. We propose DeepSafe, a novel approach for automatically assessing the overall robustness of a neural network. DeepSafe applies clustering over known labeled data and leverages off-the-shelf constraint solvers to automatically identify and check safe regions in which the network is robust, i.e. all the inputs in the region are guaranteed to be classified correctly. We also introduce the concept of targeted robustness, which ensures that the neural network is guaranteed not to misclassify inputs within a region to a specific target (adversarial) label. We evaluate DeepSafe on a neural network implementation of a controller for the next-generation Airborne Collision Avoidance System for unmanned aircraft (ACAS Xu) and for the well known MNIST network. For these networks, DeepSafe identified many regions which were safe, and also found adversarial perturbations of interest.
|Title of host publication
|Automated Technology for Verification and Analysis - 16th International Symposium, ATVA 2018, Proceedings
|Chao Wang, Shuvendu K. Lahiri
|Number of pages
|Published - 2018
|16th International Symposium on Automated Technology for Verification and Analysis, ATVA 2018 - Los Angeles, United States
Duration: 7 Oct 2018 → 10 Oct 2018
|Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
|16th International Symposium on Automated Technology for Verification and Analysis, ATVA 2018
|7/10/18 → 10/10/18
Bibliographical notePublisher Copyright:
© 2018, Springer Nature Switzerland AG.