Why adversarial reprogramming works, when it fails, and how to tell the difference

Zheng, Yang;Feng, Xiaoyi;Xia, Zhaoqiang;Jiang, Xiaoyue;Demontis, Ambra

;Pintor, Maura;Biggio, Battista;Roli, Fabio

Last

2023-01-01

---

Abstract

Adversarial reprogramming allows repurposing a machine-learning model to perform a different task. For example, a model trained to recognize animals can be reprogrammed to recognize digits by embedding an adversarial program in the digit images provided as input. Recent work has shown that adversarial reprogramming may not only be used to abuse machine-learning models provided as a service, but also beneficially, to improve transfer learning when training data is scarce. However, the factors affecting its success are still largely unexplained. In this work, we develop a first-order linear model of adversarial reprogramming to show that its success inherently depends on the size of the average input gradient, which grows when input gradients are more aligned, and when inputs have higher dimensionality. The results of our experimental analysis, involving fourteen distinct reprogramming tasks, show that the above factors are correlated with the success and the failure of adversarial reprogramming.