Recovering Missing Values from Corrupted Spatio-Temporal Sensory Data via Robust Low-Rank Tensor Completion

  • Wenjie RuanEmail author
  • Peipei Xu
  • Quan Z. Sheng
  • Nickolas J. G. Falkner
  • Xue Li
  • Wei Emma Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10177)


With the booming of the Internet of Things, tremendous amount of sensors have been installed in different geographic locations, generating massive sensory data with both time-stamps and geo-tags. Such type of data usually have shown complex spatio-temporal correlation and are easily missing in practice due to communication failure or data corruption. In this paper, we aim to tackle the challenge – how to accurately and efficiently recover the missing values for corrupted spatio-temporal sensory data. Specifically, we first formulate such sensor data as a high-dimensional tensor that can naturally preserve sensors’ both geographical and time information, thus we call spatio-temporal Tensor. Then we model the sensor data recovery as a low-rank robust tensor completion problem by exploiting its latent low-rank structure and sparse noise property. To solve this optimization problem, we design a highly efficient optimization method that combines the alternating direction method of multipliers and accelerated proximal gradient to minimize the tensor’s convex surrogate and noise’s \(\ell _1\)-norm. In addition to testing our method by a synthetic dataset, we also use passive RFID (radio-frequency identification) sensors to build a real-world sensor-array testbed, which generates overall 115,200 sensor readings for model evaluation. The experimental results demonstrate the accuracy and robustness of our approach.


Receive Signal Strength Indicator Sensor Reading Matrix Completion Tucker Decomposition Recovery Accuracy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Acar, E., Dunlavy, D.M., Kolda, T.G., Mørup, M.: Scalable tensor factorizations for incomplete data. Chemometr. Intell. Lab. Syst. 106(1), 41–56 (2011)CrossRefGoogle Scholar
  2. 2.
    Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Cai, J.F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM (JACM) 58(3), 11 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717–772 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Chen, Y., Jalali, A., Sanghavi, S., Caramanis, C.: Low-rank matrix recovery from errors and erasures. IEEE Trans. Info. Theor. 59(7), 4324–4337 (2013)CrossRefGoogle Scholar
  7. 7.
    Chen, Y., Xu, H., Caramanis, C., Sanghavi, S.: Robust matrix completion with corrupted columns. arXiv preprint (2011). arXiv:1102.2254
  8. 8.
    Da Silva, C., Herrmann, F.J.: Hierarchical tucker tensor optimization-applications to tensor completion. In: Proceedings of 10th International Conference on Sampling Theory and Applications (2013)Google Scholar
  9. 9.
    Da Xu, L., He, W., Li, S.: Internet of things in industries: a survey. IEEE Trans. Industr. Inf. 10(4), 2233–2243 (2014)CrossRefGoogle Scholar
  10. 10.
    Gandy, S., Recht, B., Yamada, I.: Tensor completion and low-n-rank tensor recovery via convex optimization. Inverse Prob. 27(2), 025010 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Goldfarb, D., Qin, Z.: Robust low-rank tensor recovery: models and algorithms. SIAM J. Matrix Anal. Appl. 35(1), 225–253 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Grewal, M.S.: Kalman Filtering. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  13. 13.
    Hazan, T., Polak, S., Shashua, A.: Sparse image coding using a 3D non-negative tensor factorization. In: Tenth IEEE International Conference on Computer Vision, ICCV 2005, vol. 1, pp. 50–57. IEEE (2005)Google Scholar
  14. 14.
    Hsieh, H.P., Lin, S.D., Zheng, Y.: Inferring air quality for station location recommendation based on urban big data. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 437–446 (2015)Google Scholar
  15. 15.
    Ji, H., Huang, S., Shen, Z., Xu, Y.: Robust video restoration by joint sparse and low rank matrix approximation. SIAM J. Imaging Sci. 4(4), 1122–1142 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Kreimer, N., Stanton, A., Sacchi, M.D.: Tensor completion based on nuclear norm minimization for 5D seismic data reconstruction. Geophysics 78(6), V273–V284 (2013)CrossRefGoogle Scholar
  18. 18.
    Kressner, D., Steinlechner, M., Vandereycken, B.: Low-rank tensor completion by Riemannian optimization. BIT Numer. Math. 54(2), 447–468 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Lawrance, A., Lewis, P.: An exponential moving-average sequence and point process. J. Appl. Probab. 14, 98–113 (1977)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Lin, Z., Chen, M., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint (2010). arXiv:1009.5055
  21. 21.
    Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 2114–2121. IEEE (2009)Google Scholar
  22. 22.
    Norcia, A.M., Clarke, M., Tyler, C.W.: Digital filtering and robust regression techniques for estimating sensory thresholds from the evoked potential. IEEE Eng. Med. Biol. Mag. 4(4), 26–32 (1985)CrossRefGoogle Scholar
  23. 23.
    Ruan, W., Sheng, Q.Z., Yao, L., Gu, T., Ruta, M., Shangguan, L.: Device-free indoor localization and tracking through human-object interactions. In: 2016 IEEE 17th International Symposium on A World of Wireless, Mobile and Multimedia Networks (WoWMoM), pp. 1–9. IEEE (2016)Google Scholar
  24. 24.
    Ruan, W., Xu, P., Sheng, Q.Z., Tran, N.K., Falkner, N.J., Li, X., Zhang, W.E.: When sensor meets tensor: filling missing sensor values through a tensor approach. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 2025–2028. ACM (2016)Google Scholar
  25. 25.
    Ruan, W., Yao, L., Sheng, Q.Z., Falkner, N., Li, X., Gu, T.: TagFall: towards unobstructive fine-grained fall detection based on UHF passive RFID tags. In: Proceedings of the 12th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services, pp. 140–149 (2015)Google Scholar
  26. 26.
    Sakurai, Y., Matsubara, Y., Faloutsos, C.: Mining and forecasting of big time-series data. In: Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, pp. 919–922. ACM (2015)Google Scholar
  27. 27.
    Signoretto, M., De Lathauwer, L., Suykens, J.A.: Nuclear norms for tensors and their use for convex multilinear estimation. Linear Algebra Appl. 43 (2010)Google Scholar
  28. 28.
    Sun, J., Tao, D., Faloutsos, C.: Beyond streams and graphs: dynamic tensor analysis. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 374–383. ACM (2006)Google Scholar
  29. 29.
    Toh, K.C., Yun, S.: An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems. Pac. J. Optim. 6(615–640), 15 (2010)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Tomioka, R., Hayashi, K., Kashima, H.: Estimation of low-rank tensors via convex optimization. arXiv preprint (2010). arXiv:1010.0789
  31. 31.
    Wright, J., Ganesh, A., Rao, S., Peng, Y., Ma, Y.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. In: Advances in Neural Information Processing Systems, pp. 2080–2088 (2009)Google Scholar
  32. 32.
    Xiuwen Yi, Y., Zheng, J.: ST-MVL: filling missing values in geo-sensory time series data. In: IJCAI 2016 (2016)Google Scholar
  33. 33.
    Yao, L., Ruan, W., Sheng, Q.Z., Li, X., Falkner, N.J.: Exploring tag-free RFID-based passive localization and tracking via learning-based probabilistic approaches. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 1799–1802. ACM (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Wenjie Ruan
    • 1
    Email author
  • Peipei Xu
    • 1
    • 2
  • Quan Z. Sheng
    • 3
  • Nickolas J. G. Falkner
    • 1
  • Xue Li
    • 4
  • Wei Emma Zhang
    • 1
  1. 1.School of Computer ScienceThe University of AdelaideAdelaideAustralia
  2. 2.School of Electronic EngineeringUESTCChengduChina
  3. 3.Department of ComputingMacquarie UniversitySydneyAustralia
  4. 4.School of ITEEUniversity of QueenslandBrisbaneAustralia

Personalised recommendations