Skip to main content
  • Textbook
  • © 2022

Geometry of Deep Learning

A Signal Processing Perspective

Authors:

  • Covers recent developments in deep learning and a wide spectrum of issues, with exercise problems for students

  • Employs unified mathematical approaches with illustrative graphics to present various techniques and their results

  • Closes the gap between the purely mathematical and implementation-oriented treatments of deep learning

Part of the book series: Mathematics in Industry (MATHINDUSTRY, volume 37)

Buying options

eBook USD 69.99
Price excludes VAT (USA)
  • ISBN: 978-981-16-6046-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Hardcover Book USD 89.99
Price excludes VAT (USA)

This is a preview of subscription content, access via your institution.

Table of contents (15 chapters)

  1. Front Matter

    Pages i-xvi
  2. Basic Tools for Machine Learning

    1. Front Matter

      Pages 1-1
    2. Mathematical Preliminaries

      • Jong Chul Ye
      Pages 3-28
    3. Linear and Kernel Classifiers

      • Jong Chul Ye
      Pages 29-44
    4. Linear, Logistic, and Kernel Regression

      • Jong Chul Ye
      Pages 45-59
  3. Building Blocks of Deep Learning

    1. Front Matter

      Pages 77-77
    2. Biological Neural Networks

      • Jong Chul Ye
      Pages 79-90
    3. Convolutional Neural Networks

      • Jong Chul Ye
      Pages 113-134
    4. Graph Neural Networks

      • Jong Chul Ye
      Pages 135-154
    5. Normalization and Attention

      • Jong Chul Ye
      Pages 155-191
  4. Advanced Topics in Deep Learning

    1. Front Matter

      Pages 193-193
    2. Geometry of Deep Neural Networks

      • Jong Chul Ye
      Pages 195-226
    3. Deep Learning Optimization

      • Jong Chul Ye
      Pages 227-242
    4. Generalization Capability of Deep Learning

      • Jong Chul Ye
      Pages 243-266
    5. Generative Models and Unsupervised Learning

      • Jong Chul Ye
      Pages 267-313
    6. Summary and Outlook

      • Jong Chul Ye
      Pages 315-316
    7. Bibliography

      • Jong Chul Ye
      Pages 317-325
  5. Back Matter

    Pages 327-330

About this book

The focus of this book is on providing students with insights into geometry that can help them understand deep learning from a unified perspective. Rather than describing deep learning as an implementation technique, as is usually the case in many existing deep learning books, here, deep learning is explained as an ultimate form of signal processing techniques that can be imagined. 

To support this claim, an overview of classical kernel machine learning approaches is presented, and their advantages and limitations are explained. Following a detailed explanation of the basic building blocks of deep neural networks from a biological and algorithmic point of view, the latest tools such as attention, normalization, Transformer, BERT, GPT-3, and others are described. Here, too, the focus is on the fact that in these heuristic approaches, there is an important, beautiful geometric structure behind the intuition that enables a systematic understanding. A unified geometric analysis to understand the working mechanism of deep learning from high-dimensional geometry is offered. Then, different forms of generative models like GAN, VAE, normalizing flows, optimal transport, and so on are described from a unified geometric perspective, showing that they actually come from statistical distance-minimization problems.

Because this book contains up-to-date information from both a practical and theoretical point of view, it can be used as an advanced deep learning textbook in universities or as a reference source for researchers interested in acquiring the latest deep learning algorithms and their underlying principles. In addition, the book has been prepared for a codeshare course for both engineering and mathematics students, thus much of the content is interdisciplinary and will appeal to students from both disciplines.


Keywords

  • Deep learning
  • Mathematical principle of deep learning
  • Geometric understanding of deep neural network
  • Review of state-of-the art deep learning methods
  • Optimal transport

Authors and Affiliations

  • Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea (Republic of)

    Jong Chul Ye

About the author

The author is currently a full Professor at Korea Advanced Institute of Science and Technology (KAIST). Also he has been a Fellow of IEEE since January 2020. 

Bibliographic Information

Buying options

eBook USD 69.99
Price excludes VAT (USA)
  • ISBN: 978-981-16-6046-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Hardcover Book USD 89.99
Price excludes VAT (USA)