Skip to main content
SpringerLink
Log in
Book cover

Vertex-Frequency Analysis of Graph Signals pp 207–222Cite as

Wavelets on Graphs via Deep Learning

  • Chapter
  • First Online:

Part of the book series: Signals and Communication Technology ((SCT))

Abstract

An increasing number of applications require processing of signals defined on weighted graphs. While wavelets provide a flexible tool for signal processing in the classical setting of regular domains, the existing graph wavelet constructions are less flexible—they are guided solely by the structure of the underlying graph and do not take directly into consideration the particular class of signals to be processed. This chapter introduces a machine learning framework for constructing graph wavelets that can sparsely represent a given class of signals. Our construction uses the lifting scheme, and is based on the observation that the recurrent nature of the lifting scheme gives rise to a structure resembling a deep auto-encoder network. Particular properties that the resulting wavelets must satisfy determine the training objective and the structure of the involved neural networks. The training is unsupervised, and is conducted similarly to the greedy pre-training of a stack of auto-encoders. After training is completed, we obtain a linear wavelet transform that can be applied to any graph signal in time and memory linear in the size of the graph. Improved sparsity of our wavelet transform for the test signals is confirmed via experiments both on synthetic and real data.

This work was done at Computer science department, Stanford University.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Mark Schmidt, http://www.di.ens.fr/~mschmidt/Software/minFunc.html.

  2. 2.

    National Climatic Data Center, http://www.ftp://ftp.ncdc.noaa.gov/pub/data/gsod/2012/.

References

  1. D.I. Shuman, S.K. Narang, P. Frossard, A. Ortega, P. Vandergheynst, The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process. Mag. 30(3), 83–98 (2013)

    Article  Google Scholar 

  2. M. Crovella, E.D. Kolaczyk, Graph wavelets for spatial traffic analysis, in INFOCOM (2003)

    Google Scholar 

  3. A.D. Szlam, M. Maggioni, R.R. Coifman, J.C. Bremer, Diffusion-driven multiscale analysis on manifolds and graphs: top-down and bottom-up constructions, in SPIE (2005), p. 5914

    Google Scholar 

  4. M. Gavish, B. Nadler, R.R. Coifman, Multiscale wavelets on trees, graphs and high dimensional data: theory and applications to semi supervised learning, in ICML (2010), pp. 367–374

    Google Scholar 

  5. R.R. Coifman, M. Maggioni, Diffusion wavelets. Appl. Comput. Harmon. Anal. 21(1), 53–94 (2006). https://doi.org/10.1016/j.acha.2006.04.004

  6. D.K. Hammond, P. Vandergheynst, R. Gribonval, Wavelets on graphs via spectral graph theory. Appl. Comput. Harmon. Anal. 30(2), 129–150 (2011). https://doi.org/10.1016/j.acha.2010.04.005

  7. I. Ram, M. Elad, I. Cohen, Generalized tree-based wavelet transform. IEEE Trans. Signal Process. 59(9), 4199–4209 (2011)

    Article  MathSciNet  Google Scholar 

  8. R.M. Rustamov, Average interpolating wavelets on point clouds and graphs. CoRR (2011). arXiv:1110.2227

  9. S.K. Narang, A. Ortega, Multi-dimensional separable critically sampled wavelet filterbanks on arbitrary graphs, in ICASSP (2012), pp. 3501–3504

    Google Scholar 

  10. M.N. Do, Y.M. Lu, Multidimensional filter banks and multiscale geometric representations. Found. Trends Signal Process. 5(3), 157–264 (2012)

    Article  Google Scholar 

  11. G.E. Hinton, S. Osindero, Y.W. Teh, A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006). https://doi.org/10.1162/neco.2006.18.7.1527

    Article  MathSciNet  MATH  Google Scholar 

  12. G.E. Hinton, R. Salakhutdinov, Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006). https://doi.org/10.1126/science.1127647

    Article  MathSciNet  MATH  Google Scholar 

  13. Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle, Greedy layer-wise training of deep networks, in Adv. Neural Inf. Process. Syst. 19, ed. by B. Schölkopf, J. Platt, T. Hoffman (MIT Press, MA, 2007), pp. 153–160

    Google Scholar 

  14. M. Ranzato, C. Poultney, S. Chopra, Y. LeCun, Efficient learning of sparse representations with an energy-based model, in Adv. Neural Inf. Process. Syst. 19, ed. by B. Schölkopf, J. Platt, T. Hoffman (MIT Press, MA, 2007), pp. 1137–1144

    Google Scholar 

  15. R. Rustamov, L.J. Guibas, Wavelets on graphs via deep learning, in Advances in Neural Information Processing Systems 26, ed. by C.J.C. Burges, L. Bottou, M. Welling, Z. Ghahramani, K.Q. Weinberger (Curran Associates, Inc., 2013), pp. 998–1006. http://papers.nips.cc/paper/5046-wavelets-on-graphs-via-deep-learning.pdf

  16. F. Scarselli, M. Gori, A.C. Tsoi, M. Hagenbuchner, G. Monfardini, The graph neural network model. Trans. Neural Netw. 20(1), 61–80 (2009). https://doi.org/10.1109/TNN.2008.2005605

  17. M.M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, P. Vandergheynst, Geometric deep learning: going beyond euclidean data. IEEE Signal Process. Mag. 34(4), 18–42 (2017). https://doi.org/10.1109/MSP.2017.2693418

  18. S. Mallat, A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way, 3rd edn. (Academic Press, 2008)

    Google Scholar 

  19. W. Sweldens, The lifting scheme: a construction of second generation wavelets. SIAM J. Math. Anal. 29(2), 511–546 (1998)

    Article  MathSciNet  Google Scholar 

  20. I. Daubechies, W. Sweldens, Factoring wavelet transforms into lifting steps. J. Fourier Anal. Appl. 4(3), 245–267 (1998)

    Article  MathSciNet  Google Scholar 

  21. R.L. Claypoole, G. Davis, W. Sweldens, R.G. Baraniuk, Nonlinear wavelet transforms for image coding via lifting. IEEE Trans. Image Process. 12(12), 1449–1459 (2003)

    Article  MathSciNet  Google Scholar 

  22. D.P. Kingma, J. Ba, Adam: A method for stochastic optimization (2015)

    Google Scholar 

  23. M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G.S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, X. Zheng, TensorFlow: Large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/. Software available from tensorflow.org

  24. M. Belkin, P. Niyogi, Semi-supervised learning on riemannian manifolds. Mach. Learn. 56(1–3), 209–239 (2004)

    Article  Google Scholar 

  25. A.Y. Ng, M.I. Jordan, Y. Weiss, On spectral clustering: analysis and an algorithm, in NIPS (2001), pp. 849–856

    Google Scholar 

  26. R.R. Coifman, S. Lafon, Diffusion maps. Applied and Computational Harmonic Analysis 21(1), 5–30 (2006). https://doi.org/10.1016/j.acha.2006.04.006

  27. X. Zhu, Z. Ghahramani, J.D. Lafferty, Semi-supervised learning using gaussian fields and harmonic functions, in ICML (2003), pp. 912–919

    Google Scholar 

  28. A. Georghiades, P. Belhumeur, D. Kriegman, From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 643–660 (2001)

    Article  Google Scholar 

  29. X. Zhang, X. Dong, P. Frossard, Learning of structured graph dictionaries, in ICASSP (2012), pp. 3373–3376. https://doi.org/10.1109/ICASSP.2012.6288639

  30. J. Bruna, S. Mallat, Invariant scattering convolution networks. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1872–1886 (2013)

    Article  Google Scholar 

Download references

Acknowledgements

We thank Jonathan Huang for discussions and especially for his advice regarding the experimental section. The authors acknowledge the support of NSF grants FODAVA 808515 and DMS 1228304, AFOSR grant FA9550-12-1-0372, ONR grant N00014-13-1-0341, a Google research award, and the Max Plack Center for Visual Computing and Communications.

Author information

Authors and Affiliations

  1. Data Science and AI Research, AT&T Labs Research, 1 AT&T Way, Bedminster, NJ, 07931, USA

    Raif M. Rustamov

  2. Computer Science Department, Stanford University, 353 Serra Mall, Stanford, CA, 94305, USA

    Leonidas J. Guibas

Authors
  1. Raif M. Rustamov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leonidas J. Guibas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Raif M. Rustamov .

Editor information

Editors and Affiliations

  1. Faculty of Electrical Engineering, University of Montenegro, Podgorica, Montenegro

    Ljubiša Stanković

  2. Department of Electrical and Computer Engineering, Swanson School of Engineering, University of Pittsburgh, Pittsburgh, PA, USA

    Ervin Sejdić

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Rustamov, R.M., Guibas, L.J. (2019). Wavelets on Graphs via Deep Learning. In: Stanković, L., Sejdić, E. (eds) Vertex-Frequency Analysis of Graph Signals. Signals and Communication Technology. Springer, Cham. https://doi.org/10.1007/978-3-030-03574-7_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-03574-7_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-03573-0

  • Online ISBN: 978-3-030-03574-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation