每周一篇 Paper 赏析：基于latent样本的对抗训练技术

2022年7月27日20:15:36评论100 views字数 2260阅读7分32秒阅读模式

在经历了考试周和众多DDL之后，本周让我们来看一篇关于adversarial training的文章，假期也不要忘记学习哦！

该文标题为：Adversarial Training and Provable Defenses: Bridging the Gap。发表于ICLR (Oral) 2020，来自ETH的SRILAB (www.sri.inf.ethz.ch)，主要思想是使用验证的相关技术来提升对抗样本训练的鲁棒性，提出了一种新的对抗训练方法convex layerwise adversarial training (COLT)。

We propose a new method to train neural networks based on a novel combination of adversarial training and provable defenses. The key idea is to model training as a procedure which includes both, the verifier and the adversary. In every iteration, the verifier aims to certify the network using convex relaxation while the adversary tries to find inputs inside that convex relaxation which cause verification to fail. We experimentally show that this training method is promising and achieves the best of both worlds – it produces a state-of-the-art neural network with certified robustness of 58.1% and accuracy of 78.8% on the challenging CIFAR-10 dataset with a 2/255 L-inf perturbation. This significantly improves over the currently known best results of 53.9% certified robustness and 68.3% accuracy.

背景

为了提升模型的鲁棒性，常见的一种方法是adversarial training，利用对抗样本对网络进行再训练，该方法的实用性已经得到了充分的实验论证，有点是准确率高，但无法提供理论保证。与其相对的一种技术为provable defense，该方法利用验证的思想，可以确保训练出的模型对部分样本具备local robustness，但是缺点是准确率较低。在该文中，作者考虑将两者的训练过程进行一定程度的融合。

We show that it is possible to train more accurate and provably robust neural networks using the same convex relaxations as those used in existing, state-of-the-art provable defense methods, but with a new, different optimization procedure inspired by adversarial training.

关键思想

本文的关键思想来自验证过程的松弛。如上图所示，神经网络真实的鲁棒边界如虚线所示（虚线内可以找到许多对抗样本，当虚线内无法找到对抗样本时，认为对于输入(x) 具有$epsilon$-鲁棒性），但在神经网络验证过程中，为了保证sound并提高运算效率，经常使用relaxation技术，在这种情况下，边界会变为实线所描述的convex形状。