Skip to main content
  • Textbook
  • © 2020

Linear Algebra and Optimization for Machine Learning

A Textbook

  • First textbook to provide an integrated treatment of linear algebra and optimization with a special focus on machine learning issues

  • Includes many examples to simplify exposition and facilitate in learning semantically

  • Complemented by examples and exercises throughout the book. A solution manual for the exercises at the end of each chapter is available to teaching instructors

  • Includes supplementary material: sn.pub/extras

Buying options

eBook USD 39.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-40344-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book USD 49.99
Price excludes VAT (USA)
Hardcover Book USD 69.99
Price excludes VAT (USA)

This is a preview of subscription content, access via your institution.

Table of contents (11 chapters)

  1. Front Matter

    Pages I-XXI
  2. Linear Transformations and Linear Systems

    • Charu C. Aggarwal
    Pages 41-95
  3. Eigenvectors and Diagonalizable Matrices

    • Charu C. Aggarwal
    Pages 97-139
  4. Optimization Basics: A Machine Learning View

    • Charu C. Aggarwal
    Pages 141-203
  5. Advanced Optimization Solutions

    • Charu C. Aggarwal
    Pages 205-253
  6. Constrained Optimization and Duality

    • Charu C. Aggarwal
    Pages 255-297
  7. Singular Value Decomposition

    • Charu C. Aggarwal
    Pages 299-337
  8. Matrix Factorization

    • Charu C. Aggarwal
    Pages 339-378
  9. The Linear Algebra of Similarity

    • Charu C. Aggarwal
    Pages 379-410
  10. The Linear Algebra of Graphs

    • Charu C. Aggarwal
    Pages 411-446
  11. Optimization in Computational Graphs

    • Charu C. Aggarwal
    Pages 447-482
  12. Back Matter

    Pages 483-495

About this book

This textbook introduces linear algebra and optimization in the context of machine learning. Examples and exercises are provided throughout the book. A solution manual for the exercises at the end of each chapter is available to teaching instructors. This textbook targets graduate level students and professors in computer science, mathematics and data science. Advanced undergraduate students can also use this textbook. The chapters for this textbook are organized as follows: 

1. Linear algebra and its applications: The chapters focus on the basics of linear algebra together with their common applications to singular value decomposition, matrix factorization, similarity matrices (kernel methods), and graph analysis. Numerous machine learning applications have been used as examples, such as spectral clustering, kernel-based classification, and outlier detection. The tight integration of linear algebra methods with examples from machine learning differentiates this book from generic volumes on linear algebra. The focus is clearly on the most relevant aspects of linear algebra for machine learning and to teach readers how to apply these concepts.

2. Optimization and its applications: Much of machine learning is posed as an optimization problem in which we try to maximize the accuracy of regression and classification models. The “parent problem” of optimization-centric machine learning is least-squares regression. Interestingly, this problem arises in both linear algebra and optimization, and is one of the key connecting problems of the two fields.  Least-squares regression is also the starting point for support vector machines, logistic regression, and recommender systems. Furthermore, the methods for dimensionality reduction and matrix factorization also require the development of optimization methods. A general view of optimization in computational graphs is discussed together with its applications to back propagation in neural networks. 

A frequent challenge faced by beginners in machine learning is the extensive background required in linear algebra and optimization. One problem is that the existing linear algebra and optimization courses are not specific to machine learning; therefore, one would typically have to complete more course material than is necessary to pick up machine learning. Furthermore, certain types of ideas and tricks from optimization and linear algebra recur more frequently in machine learning than other application-centric settings. Therefore, there is significant value in developing a view of linear algebra and optimization that is better suited to the specific perspective of machine learning.

Keywords

  • Linear Algebra
  • Optimization
  • Machine Learning
  • Deep Learning
  • Neural Networks
  • Dynamic Programming
  • Support Vector Machines
  • Linear Regression
  • Matrix Algebra
  • Numerical Algebra
  • Gradient Descent
  • matrix theory

Reviews

“Based on the topics covered and the excellent presentation, I would recommend Aggarwal's book over these other books for an advanced undergraduate or beginning graduate course on mathematics for data science.” (Brian Borchers, MAA Reviews, March 28, 2021)

“This book should be of interest to graduate students in engineering, applied mathematics, and other fields requiring an understanding of the mathematical underpinnings of machine learning.” (IEEE Control Systems Magazine, Vol. 40 (6), December, 2020)

Authors and Affiliations

  • IBM T.J. Watson Research Center, Yorktown Heights, USA

    Charu C. Aggarwal

About the author

Charu C. Aggarwal is a Distinguished Research Staff Member (DRSM) at the IBM T. J. Watson Research Center in Yorktown Heights, New York. He completed his undergraduate degree in Computer Science from the Indian Institute of Technology at Kanpur in 1993 and his Ph.D. in Operations Research from the Massachusetts Institute of Technology in 1996. He has published more than 400 papers in refereed conferences and journals and has applied for or been granted more than 80 patents. He is author or editor of 19 books, including textbooks on data mining, neural networks, machine learning (for text), recommender systems, and outlier analysis. Because of the commercial value of his patents, he has thrice been designated a Master Inventor at IBM. He has received several internal and external awards, including the EDBT Test-of-Time Award (2014), the IEEE ICDM Research Contributions Award (2015), and the ACM SIGKDD Innovation Award (2019). He has served as editor-in-chief of the ACM SIGKDD Explorations, and is currently serving as an editor-in-chief of the ACM Transactions on Knowledge Discovery from Data. He is a fellow of the SIAM, ACM, and the IEEE, for “contributions to knowledge discovery and data mining algorithms.”


          

Bibliographic Information

  • Book Title: Linear Algebra and Optimization for Machine Learning

  • Book Subtitle: A Textbook

  • Authors: Charu C. Aggarwal

  • DOI: https://doi.org/10.1007/978-3-030-40344-7

  • Publisher: Springer Cham

  • eBook Packages: Computer Science, Computer Science (R0)

  • Copyright Information: Springer Nature Switzerland AG 2020

  • Hardcover ISBN: 978-3-030-40343-0

  • Softcover ISBN: 978-3-030-40346-1

  • eBook ISBN: 978-3-030-40344-7

  • Edition Number: 1

  • Number of Pages: XXI, 495

  • Number of Illustrations: 67 b/w illustrations, 26 illustrations in colour

  • Topics: Machine Learning, Linear Algebra, Computer Communication Networks

Buying options

eBook USD 39.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-40344-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book USD 49.99
Price excludes VAT (USA)
Hardcover Book USD 69.99
Price excludes VAT (USA)