GAT: Guided Adversarial Training with Pareto-optimal Auxiliary Tasks

Salah Ghamizi; Jingfeng Zhang; Maxime Cordy; Mike Papadakis; Masashi Sugiyama; Yves Le Traon

GAT: Guided Adversarial Training with Pareto-optimal Auxiliary Tasks

Salah Ghamizi, Jingfeng Zhang^*, Maxime Cordy, Mike Papadakis, Masashi Sugiyama, Yves Le Traon

^*Corresponding author for this work

Research output: Contribution to journal › Conference article › peer-review

2 Citations (Scopus)

Abstract

While leveraging additional training data is well established to improve adversarial robustness, it incurs the unavoidable cost of data collection and the heavy computation to train models. To mitigate the costs, we propose Guided Adversarial Training (GAT), a novel adversarial training technique that exploits auxiliary tasks under a limited set of training data. Our approach extends single-task models into multi-task models during the min-max optimization of adversarial training, and drives the loss optimization with a regularization of the gradient curvature across multiple tasks. Experimentally, GAT increases the robust AUC of CheXpert medical imaging dataset from 50% to 83% and On CIFAR-10, GAT outperforms eight state-of-the-art adversarial training and achieves 56.21% robust accuracy with Resnet-50. Overall, we demonstrate that guided multi-task learning is an actionable and promising avenue to push further the boundaries of model robustness.

Original language	English
Pages (from-to)	11255-11282
Number of pages	28
Journal	Proceedings of Machine Learning Research
Volume	202
Publication status	Published - 2023
Externally published	Yes
Event	40th International Conference on Machine Learning, ICML 2023 - Honolulu, United States Duration: 23 Jul 2023 → 29 Jul 2023

Cite this

@article{64ffb03e41824dc9995f47d13d7798fa,

title = "GAT: Guided Adversarial Training with Pareto-optimal Auxiliary Tasks",

abstract = "While leveraging additional training data is well established to improve adversarial robustness, it incurs the unavoidable cost of data collection and the heavy computation to train models. To mitigate the costs, we propose Guided Adversarial Training (GAT), a novel adversarial training technique that exploits auxiliary tasks under a limited set of training data. Our approach extends single-task models into multi-task models during the min-max optimization of adversarial training, and drives the loss optimization with a regularization of the gradient curvature across multiple tasks. Experimentally, GAT increases the robust AUC of CheXpert medical imaging dataset from 50% to 83% and On CIFAR-10, GAT outperforms eight state-of-the-art adversarial training and achieves 56.21% robust accuracy with Resnet-50. Overall, we demonstrate that guided multi-task learning is an actionable and promising avenue to push further the boundaries of model robustness.",

author = "Salah Ghamizi and Jingfeng Zhang and Maxime Cordy and Mike Papadakis and Masashi Sugiyama and {Le Traon}, Yves",

note = "Publisher Copyright: {\textcopyright} 2023 Proceedings of Machine Learning Research. All rights reserved.; 40th International Conference on Machine Learning, ICML 2023 ; Conference date: 23-07-2023 Through 29-07-2023",

year = "2023",

language = "English",

volume = "202",

pages = "11255--11282",

journal = "Proceedings of Machine Learning Research",

issn = "2640-3498",

publisher = "ML Research Press",

}

TY - JOUR

T1 - GAT

T2 - 40th International Conference on Machine Learning, ICML 2023

AU - Ghamizi, Salah

AU - Zhang, Jingfeng

AU - Cordy, Maxime

AU - Papadakis, Mike

AU - Sugiyama, Masashi

AU - Le Traon, Yves

PY - 2023

Y1 - 2023

N2 - While leveraging additional training data is well established to improve adversarial robustness, it incurs the unavoidable cost of data collection and the heavy computation to train models. To mitigate the costs, we propose Guided Adversarial Training (GAT), a novel adversarial training technique that exploits auxiliary tasks under a limited set of training data. Our approach extends single-task models into multi-task models during the min-max optimization of adversarial training, and drives the loss optimization with a regularization of the gradient curvature across multiple tasks. Experimentally, GAT increases the robust AUC of CheXpert medical imaging dataset from 50% to 83% and On CIFAR-10, GAT outperforms eight state-of-the-art adversarial training and achieves 56.21% robust accuracy with Resnet-50. Overall, we demonstrate that guided multi-task learning is an actionable and promising avenue to push further the boundaries of model robustness.

AB - While leveraging additional training data is well established to improve adversarial robustness, it incurs the unavoidable cost of data collection and the heavy computation to train models. To mitigate the costs, we propose Guided Adversarial Training (GAT), a novel adversarial training technique that exploits auxiliary tasks under a limited set of training data. Our approach extends single-task models into multi-task models during the min-max optimization of adversarial training, and drives the loss optimization with a regularization of the gradient curvature across multiple tasks. Experimentally, GAT increases the robust AUC of CheXpert medical imaging dataset from 50% to 83% and On CIFAR-10, GAT outperforms eight state-of-the-art adversarial training and achieves 56.21% robust accuracy with Resnet-50. Overall, we demonstrate that guided multi-task learning is an actionable and promising avenue to push further the boundaries of model robustness.

UR - http://www.scopus.com/inward/record.url?scp=85174407236&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85174407236

SN - 2640-3498

VL - 202

SP - 11255

EP - 11282

JO - Proceedings of Machine Learning Research

JF - Proceedings of Machine Learning Research

Y2 - 23 July 2023 through 29 July 2023

ER -

GAT: Guided Adversarial Training with Pareto-optimal Auxiliary Tasks

Abstract

Other files and links

Fingerprint

Cite this