An Efficient Adversarial Learning Strategy for Constructing Robust Classification Boundaries

  • Wei Liu
  • Sanjay Chawla
  • James Bailey
  • Christopher Leckie
  • Kotagiri Ramamohanarao
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7691)

Abstract

Traditional classification methods assume that the training and the test data arise from the same underlying distribution. However in some adversarial settings, the test set can be deliberately constructed in order to increase the error rates of a classifier. A prominent example is email spam where words are transformed to avoid word-based features embedded in a spam filter. Recent research has modeled interactions between a data miner and an adversary as a sequential Stackelberg game, and solved its Nash equilibrium to build classifiers that are more robust to subsequent manipulations on training data sets. However in this paper we argue that the iterative algorithm used in the Stackelberg game, which solves an optimization problem at each step of play, is sufficient but not necessary for achieving Nash equilibria in classification problems. Instead, we propose a method that transforms singular vectors of a training data matrix to simulate manipulations by an adversary, and from that perspective a Nash equilibrium can be obtained by solving a novel optimization problem only once. We show that compared with the iterative algorithm used in recent literature, our one-step game significantly reduces computing time while still being able to produce good Nash equilibria results.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Dalvi, N., Domingos, P., Mausam, Sanghai, S., Verma, D.: Adversarial classification. In: Proc. of KDD 2004, pp. 99–108 (2004)Google Scholar
  2. 2.
    Lowd, D., Meek, C.: Adversarial learning. In: KDD 2005, pp. 641–647 (2005)Google Scholar
  3. 3.
    Globerson, A., Roweis, S.: Nightmare at test time: robust learning by feature deletion. In: Proc. of ICML 2006, pp. 353–360 (2006)Google Scholar
  4. 4.
    Kołcz, A., Teo, C.: Feature weighting for improved classifier robustness. In: CEAS 2009: Sixth Conference on Email and Anti-Spam (2009)Google Scholar
  5. 5.
    Kantarcioglu, M., Xi, B., Clifton, C.: Classifier evaluation and attribute selection against active adversaries. Data Min. Knowl. Discov. 22(1), 291–335 (2011)MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Liu, W., Chawla, S.: A game theoretical model for adversarial learning. In: Proceedings of the 2009 IEEE International Conference on Data Mining Workshops, pp. 25–30 (2009)Google Scholar
  7. 7.
    Liu, W., Chawla, S.: Mining Adversarial Patterns via Regularized Loss Minimization. Machine Learning 81(1), 69–83 (2010)CrossRefGoogle Scholar
  8. 8.
    Brückner, M., Scheffer, T.: Stackelberg games for adversarial prediction problems. In: Proc. of KDD 2011, pp. 547–555 (2011)Google Scholar
  9. 9.
    Fudenberg, D., Tirole, J.: Game Theory, 1st edn. The MIT Press (1991)Google Scholar
  10. 10.
    Fortuna, J., Capson, D.: Improved support vector classification using PCA and ICA feature space modification. Pattern Recognition 37(6), 1117–1129 (2004)MATHCrossRefGoogle Scholar
  11. 11.
    Selvan, S., Ramakrishnan, S.: SVD-based modeling for image texture classification using wavelet transformation. IEEE Transactions on Image Processing 16(11), 2688–2696 (2007)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Byrd, R., Schnabel, R., Shultz, G.: Approximate solution of the trust region problem by minimization over two-dimensional subspaces. Mathematical Programming 40(1), 247–263 (1988)MathSciNetMATHCrossRefGoogle Scholar
  13. 13.
    Moré, J., Sorensen, D.: Computing a trust region step. SIAM Journal on Scientific and Statistical Computing 4, 553 (1983)MATHCrossRefGoogle Scholar
  14. 14.
    Branch, M., Coleman, T., Li, Y.: A subspace, interior, and conjugate gradient method for large-scale bound-constrained minimization problems. SIAM Journal on Scientific Computing 21(1), 1–23 (2000)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Delany, S.J., Cunningham, P., Tsymbal, A., Coyle, L.: Tracking concept drift in spam filtering. Knowledge-Based Systems 18(4-5), 187–195 (2005)CrossRefGoogle Scholar
  16. 16.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)MATHGoogle Scholar
  17. 17.
    Hastie, T., Tibshirani, R., Friedman, J.: The elements of statistical learning (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Wei Liu
    • 1
  • Sanjay Chawla
    • 2
  • James Bailey
    • 1
  • Christopher Leckie
    • 1
  • Kotagiri Ramamohanarao
    • 1
  1. 1.Dept of Computing and Information SystemsThe University of MelbourneAustralia
  2. 2.School of Information TechnologiesThe University of SydneyAustralia

Personalised recommendations