Reducing Adversarial Vulnerability through Adaptive Training Batch Size

Ken Sasongko, Ken Sasongko and Adila Alfa Krisnadhi, Adila Alfa Krisnadhi and Mohamad Ivan Fanany, Mohamad Ivan Fanany (2021) Reducing Adversarial Vulnerability through Adaptive Training Batch Size. Jurnal Ilmu Komputer dan Informasi, 14 (1). pp. 27-37. ISSN e-ISSN 2502-9274

Text Reducing Adversarial Vulnerability through Adaptive Training Batch Size.pdf - Other Download (702kB)

Abstract

Neural networks possess an ability to generalize well to data distribution, to an extent that they are capable
of fitting to a randomly labeled data. But they are also known to be extremely sensitive to adversarial
examples. Batch Normalization (BatchNorm), very commonly part of deep learning architecture, has been
found to increase adversarial vulnerability. Fixup Initialization (Fixup Init) has been shown as an alternative
to BatchNorm, which can considerably strengthen the networks against adversarial examples. This robustness
can be improved further by employing smaller batch size in training. The latter, however, comes with a
tradeoff in the form of a significant increase of training time (up to ten times longer when reducing batch
size from the default 128 to 8 for ResNet-56). In this paper, we propose a workaround to this problem by
starting the training with a small batch size and gradually increase it to larger ones during training. We
empirically show that our proposal can still improve adversarial robustness (up to 5.73%) of ResNet-56 with
Fixup Init and default batch size of 128. At the same time, our proposal keeps the training time considerably
shorter (only 4 times longer, instead of 10 times).

Item Type:	Article
Uncontrolled Keywords:	adversarial examples, batch normalization, fixup initialization, batch size variation
Subjects:	T Technology > T Technology (General)
Divisions:	Faculty of Engineering, Science and Mathematics > School of Electronics and Computer Science
Depositing User:	Mrs Ni Made Yunia Dwi Savitri
Date Deposited:	17 Nov 2022 01:35
Last Modified:	17 Nov 2022 01:35
URI:	http://eprints.triatmamulya.ac.id/id/eprint/1836

Actions (login required)

View Item