Low-Rank Outlier Detection

  • Sheng Li
  • Ming Shao
  • Yun Fu


In this chapter, we present a novel low-rank outlier detection approach, which incorporates a low-rank constraint into the support vector data description (SVDD) model. Different from the traditional SVDD, our approach learns multiple hyper-spheres to fit the normal data. The low-rank constraint helps us group the complicated dataset into several clusters dynamically. We present both primal and dual solutions to solve this problem, and provide the detailed strategy of outlier detection. Moreover, the kernel-trick used in SVDD becomes unnecessary in our approach, which implies that the training time and memory space could be substantially reduced. The performance of our approach, along with other related methods, was evaluated using three image databases. Results show our approach outperforms other methods in most scenarios.


Low-rank constraint Hyper-spheres Support vector data description Outlier detection 



This research is supported in part by the NSF CNS award 1314484, Office of Naval Research award N00014-12-1-1028, Air Force Office of Scientific Research award FA9550-12-1-0201, and U.S. Army Research Office grant W911NF-13-1-0160.


  1. 1.
    F.R. Bach, Consistency of trace norm minimization. J. Mach. Learn. Res. 9, 1019–1048 (2008)MathSciNetzbMATHGoogle Scholar
  2. 2.
    A. Banerjee, P. Burlina, R. Meth, Fast hyperspectral anomaly detection via svdd, in ICIP, vol 4, pp. 101–104 (2007)Google Scholar
  3. 3.
    D. Barbará, P. Chen, Using the fractal dimension to cluster datasets, in KDD, pp. 260–264 (2000)Google Scholar
  4. 4.
    V. Barnett, T. Lewis, Outliers in Statistical Data (Wiley, New York, 1994)zbMATHGoogle Scholar
  5. 5.
    S.D. Bay, M. Schwabacher, Mining distance-based outliers in near linear time with randomization and a simple pruning rule, in KDD, pp. 29–38 (2003)Google Scholar
  6. 6.
    D. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods (Athena Scientific, Belmont, 1982)zbMATHGoogle Scholar
  7. 7.
    M. Breitenbach, G.Z. Grudic, Clustering through ranking on manifolds, in ICML, pp. 73–80 (2005)Google Scholar
  8. 8.
    J.F. Cai, E.J. Candes, Z.W. Shen, A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    E.J. Candès, X.D. Li, Y. Ma, J. Wright, Robust principal component analysis? J. ACM 58(3), 11 (2011)MathSciNetCrossRefGoogle Scholar
  10. 10.
    M. Cha, J.-S. Kim, J.-G. Baek, Density weighted support vector data description. Expert Syst. Appl. 41(7), 3343–3350 (2014)CrossRefGoogle Scholar
  11. 11.
    M. Elahi, K. Li, W. Nisar, X. Lv, H. Wang, Efficient clustering-based outlier detection algorithm for dynamic data stream, in FSKD, vol. 5, pp. 298–304 (2008)Google Scholar
  12. 12.
    P.-Y. Hao, Y.-H. Lin, A new multi-class support vector machine with multi-sphere in the feature space, in IEA/AIE, pp. 756–765 (2007)Google Scholar
  13. 13.
    D.M. Hawkins, Constrained Optimization and Lagrange Multiplier Methods (Athena Scientific, Belmont, 1982)Google Scholar
  14. 14.
    S. Hawkins, H. He, G. J. Williams, R.A. Baxter, Outlier detection using replicator neural networks, in DaWaK, pp. 170–180 (2002)Google Scholar
  15. 15.
    K.A. Heller, K.M. Svore, A.D. Keromytis, S.J. Stolfo, One class support vector machines for detecting anomalous windows registry accesses, in The Workshop on Data Mining for Computer Security (2003)Google Scholar
  16. 16.
    W. Jin, A.K.H. Tung, J. Han, Mining top-n local outliers in large databases, in KDD, pp. 293–298 (2001)Google Scholar
  17. 17.
    A. Koufakou, M. Georgiopoulos, A fast outlier detection strategy for distributed high-dimensional data sets with mixed attributes. Data Min. Knowl. Discov. 20(2), 259–289 (2010)MathSciNetCrossRefGoogle Scholar
  18. 18.
    T. Le, D. Tran, W. Ma, D. Sharma, A theoretical framework for multi-sphere support vector data description, in ICONIP, vol. 2, pp. 132–142 (2010)Google Scholar
  19. 19.
    T. Le, D. Tran, P. Nguyen, W. Ma, D. Sharma, Proximity multi-sphere support vector clustering. Neural Comput. Appl. 22(7–8), 1309–1319 (2013)CrossRefGoogle Scholar
  20. 20.
    L. Li, S. Li, Y. Fu, Discriminative dictionary learning with low-rank regularization for face recognition, in FG, pp. 1–6 (2013)Google Scholar
  21. 21.
    S. Li, Y. Fu, Low-rank coding with b-matching constraint for semi-supervised classification, in IJCAI, pp. 1472–1478 (2013)Google Scholar
  22. 22.
    S. Li, Y. Fu, Robust subspace discovery through supervised low-rank constraints, in SDM (2014)Google Scholar
  23. 23.
    S. Li, M. Shao, Y. Fu, Locality linear fitting one-class svm with low-rank constraints for outlier detection, in International Joint Conference on Neural Networks (IJCNN) (2014)Google Scholar
  24. 24.
    S. Li, I.W. Tsang, Maximum margin/volume outlier detection, in ICTAI, pp. 385–392 (2011)Google Scholar
  25. 25.
    Z.C. Lin, M.M. Chen, L.Q. Wu, Y. Ma, The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices (Technique Report, UIUC, 2009)Google Scholar
  26. 26.
    B. Liu, Y. Xiao, L. Cao, Z. Hao, F. Deng, Svdd-based outlier detection on uncertain data. Knowl. Inf. Syst. 34(3), 597–618 (2013)CrossRefGoogle Scholar
  27. 27.
    G.C. Liu, Z.C. Lin, Y. Yu, Robust subspace segmentation by low-rank representation, in ICML, pp. 663–670 (2010)Google Scholar
  28. 28.
    G.C. Liu, S.C. Yan, Latent low-rank representation for subspace segmentation and feature extraction, in ICCV (2011)Google Scholar
  29. 29.
    H.M. Lukashevich, S. Nowak, P. Dunker, Using one-class svm outliers detection for verification of collaboratively tagged image training sets, in IEEE International Conference on Multimedia and Expo (ICME), pp. 682–685 (2009)Google Scholar
  30. 30.
    E.J. Pauwels, O. Ambekar, One class classification for anomaly detection: support vector data description revisited, in ICDM, pp. 25–39 (2011)Google Scholar
  31. 31.
    P.J. Rousseeuw, A.M. Leroy, Robust Regression and Outlier Detection (Wiley, New York, 1987)CrossRefzbMATHGoogle Scholar
  32. 32.
    B. Schölkopf, J.C. Platt, J. Shawe-Taylor, A.J. Smola, R.C. Williamson, Estimating the support of a high-dimensional distribution. Neural Comput. 13(7), 1443–1471 (2001)CrossRefzbMATHGoogle Scholar
  33. 33.
    S. Shekhar, C.T. Lu, P. Zhang, Detecting graph-based spatial outliers: algorithms and applications (a summary of results), in KDD, pp. 371–376 (2001)Google Scholar
  34. 34.
    T. Sim, S. Baker, M. Bsat, The cmu pose, illumination, and expression database. IEEE Trans. Pattern Anal. Mach. Intell. 25(12), 1615–1618 (2003)CrossRefGoogle Scholar
  35. 35.
    D.M.J. Tax, R.P.W. Duin, Support vector data description. Mach. Learn. 54(1), 45–66 (2004)CrossRefzbMATHGoogle Scholar
  36. 36.
    G.J. Williams, R.A. Baxter, H. He, S. Hawkins, L. Gu, A comparative study of rnn for outlier detection in data mining, in ICDM, pp. 709–712 (2002)Google Scholar
  37. 37.
    Y. Xiao, B. Liu, L. Cao, X. Wu, C. Zhang, Z. Hao, F. Yang, J. Cao, Multi-sphere support vector data description for outliers detection on multi-distribution data, in ICDM Workshops, pp. 82–87 (2009)Google Scholar
  38. 38.
    B. Zhang, L. Zhang, D. Zhang, L. Shen, Directional binary code with application to polyu near-infrared face database. Pattern Recogn. Lett. 31(14), 2337–2344 (2010)CrossRefGoogle Scholar
  39. 39.
    Y. Zhang, Z. Jiang, L.S. Davis, Learning structured low-rank representations for image classification, in CVPR, pp. 676–683 (2013)Google Scholar
  40. 40.
    X. Zhou, C. Yang, W. Yu, Automatic mitral leaflet tracking in echocardiography by outlier detection in the low-rank representation, in CVPR, pp. 972–979 (2012)Google Scholar
  41. 41.
    F. Zhu, N. Ye, W. Yu, S. Xu, G. Li, Boundary detection and sample reduction for one-class support vector machines. Neurocomputing 123, 166–173 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringNortheastern UniversityBostonUSA
  2. 2.Department of Electrical and Computer Engineering and College of Computer and Information ScienceNortheastern UniversityBostonUSA

Personalised recommendations