The Moore–Penrose inverse: a hundred years on a frontline of physics research

Abstract

The Moore–Penrose inverse celebrated its 100th birthday in 2020, as the notion standing behind the term was first defined by Eliakim Hastings Moore in 1920 (Bull Am Math Soc 26:394–395, 1920). Its rediscovery by Sir Roger Penrose in 1955 (Proc Camb Philos Soc 51:406–413, 1955) can be considered as a caesura, after which the inverse attracted the attention it deserves and has henceforth been exploited in various research branches of applied origin. The paper contemplates the role, which the Moore–Penrose inverse plays in research within physics and related areas at present. An overview of the up-to-date literature leads to the conclusion that the inverse “grows” along with the development of physics and permanently (maybe even more demonstrably now than ever before) serves as a powerful and versatile tool to cope with the current research problems.

1 Introduction

It is commonly accepted that the history of the term “Moore–Penrose inverse” dates back to 1920, when an American mathematician Eliakim Hastings Moore (1862–1932) published the paper Moore (1920). The article–providing the first definition of the notion, which is at present known as the Moore–Penrose inverse–did not kindle any particular attention of the scientific community. Thirty five years later, not aware of the work by Moore, a British physicist and mathematician Sir Roger Penrose (1931– ) in the paper Penrose (1955) provided an equivalent (though differently formulated) definition of the same concept. The fact that Moore and Penrose defined actually the same notion was soon recognized by Richard Rado¹ (1906–1989) in Rado (1956). An attempt to identify reasons why no noticeable reaction was kindled by the paper Moore (1920), whereas the response caused by the article Penrose (1955) was notable, was made in Ben-Israel (2002); see also [Ben-Israel and Greville 2003, “Appendix A”]. In both these sources one finds the following judgement on the way the Moore’s definition was formulated and, simultaneously, an explanation way it was generally overlooked: “...it was much too idiosyncratic and used unnecessarily complicated notation, making it illegible for all but very dedicated readers.”. An earlier article Ben-Israel (1986) contains a thorough discussion over the inverse from the perspective of the work of Penrose.

It is noteworthy that the lifespans of the two scientist whose names are now attributed to the focal object of the present article overlapped by less than 17 months. Eliakim Hastings Moore² was an important figure of American mathematics at the end of 19th and beginning of 20th centuries, known for his contributions to abstract algebra, algebraic geometry, analysis, geometry, integral equations, linear algebra, and number theory. He founded the Chicago branch of the American Mathematical Society and served as the Society’s sixth President (1901–1902). He also brought to life the Transactions of the AMS, a journal which has been continuously published since 1900. In 2002 AMS honored him by establishing a prize—E.H. Moore Research Article Prize—awarded every three years for an outstanding research article to have appeared in one of the AMS primary research journals. In 1955 Roger Penrose was a student at The College of St John the Evangelist in the University of Cambridge. His subsequent most distinguished scientific contributions concern general relativity and cosmology. Penrose has received several prizes and awards for his achievements, including the Nobel Prize in Physics 2020 “for the discovery that black hole formation is a robust prediction of the general theory of relativity”. Interestingly, according to the Mathematics Genealogy Project Moore and Penrose had similar number of Ph.D. students, with 31 attributed to Moore and 33 to Penrose. However, the database lists 26217 descendants of Moore and 203 of Penrose.

Since the work Penrose (1955) was published, the Moore–Penrose inverse focuses attention of researchers representing a variety of—often seemingly distant—science branches. Efforts of the researchers concentrate on both, purely theoretical investigations aimed at acquiring possibly deep knowledge on the features of this concept, its generalizations, and properties, and on explorations of the possibilities it offers to solve real problems emerging in a wide range of research areas. In the present article, after a brief introduction in the next section, we provide a concise review of the up-to-date literature, shading a spotlight at the role the Moore–Penrose inverse plays in physics and related research areas 100 years after it was first defined.

2 Definitions of the Moore–Penrose inverse of a matrix

In the literature referring to the notion of the Moore–Penrose inverse the authors most often recall its definition as it was formulated in Penrose (1955). The definition, based on four matrix equations known now as the Penrose conditions, is recalled below. The superscript $^*$ stands therein for the conjugate transpose (Hermitian transpose) of a matrix argument.

Definition of the Moore–Penrose inverse according to Penrose (1955). Let $\mathbf{A}$ be an $m \times n$ complex matrix. Then the Moore–Penrose inverse of $\mathbf{A}$ is the matrix $\mathbf{A}^\dagger$ satisfying the following (Penrose) conditions:

$$\begin{aligned}&\mathbf{AA}^\dagger \mathbf{A} =\mathbf{A}, \quad \mathbf{A}^\dagger \mathbf{AA}^\dagger = \mathbf{A}^\dagger , \nonumber \\&\mathbf{AA}^\dagger = {(\mathbf{AA}^\dagger )}^*, \mathbf{A}^\dagger \mathbf{A} = {(\mathbf{A}^\dagger \mathbf{A})}^*. \end{aligned}$$

(1)

The system of matrix Eqs. (1) is consistent, which means that every matrix $\mathbf{A}$ has its Moore–Penrose inverse $\mathbf{A}^\dagger$. Furthermore, the inverse is unique, i.e., for every matrix $\mathbf{A}$ there exists exactly one $\mathbf{A}^\dagger$, which satisfies Eqs. (1); if $\mathbf{A}$ is square and nonsingular, then $\mathbf{A}^\dagger = \mathbf{A}^{-1}$. Example 1 provided in “Appendix A” gives an insight on how the set of matrices satisfying the Penrose conditions gets narrower as subsequent Penrose conditions are imposed, reaching one element set $\{\mathbf{A}^\dagger \}$ when all four conditions are fulfilled.

In the definition of the inverse formulated in Moore (1920) (called by the author the “general reciprocal”) rather awkward notation was used. Below the definition is restated using more modern notation.

Definition of the Moore–Penrose inverse according to Moore (1920). Let $\mathbf{A}$ be an $m \times n$ complex matrix. Then the Moore–Penrose inverse of $\mathbf{A}$ is the matrix $\mathbf{A}^\dagger$ satisfying the following conditions:

$$\begin{aligned} \mathbf{AA}^\dagger = \mathbf{P}_\mathbf{A}, \quad \mathbf{A}^\dagger \mathbf{A} = \mathbf{P}_{\mathbf{A}^*}, \end{aligned}$$

where $\mathbf{P}_\mathbf{A}$ and $\mathbf{P}_{\mathbf{A}^*}$ denote the orthogonal projectors onto column spaces (ranges) of $\mathbf{A}$ and $\mathbf{A}^*$, respectively.

An advantage of the Penrose definition of the inverse is that it enables to specify various (not unique) generalized inverses, which satisfy certain subsets of the four Penrose conditions. Some of such inverses proved to arise naturally and play distinguished roles in a variety of considerations, and occur in the literature under their own names. For example, the inverses satisfying: the first Penrose condition are known as inner, the second condition as outer³, the first two conditions as reflexive, the first three conditions as normalized, the first and third condition as least squares, whereas the first and fourth as minimum norm. On the other hand, a valuable feature of the Moore’s version of the definition is that it constitutes a direct bridge between algebraic notions (to which matrices belong) and geometric notions (to which projections belong). This link proved to be an extremely useful property of the Moore–Penrose inverse extensively utilized in the literature.

Fig. 1

Number of papers published, in whose titles or abstracts occurs the phrase “Moore–Penrose inverse” according to Elsevier Scopus (as of October 2020)

A discussion on algebraic properties of the Moore–Penrose inverse is beyond the scope of the present article. An interested reader is referred to quite a rich literature on the subject published over the years, for instance to one of the classical monographs: Ben-Israel and Greville (2003), Campbell and Meyer (2009) or [Rao and Mitra⁴ (1971)]⁵. A subjective selection of features of the inverse from the perspective of its links with physics was presented in Barata and Hussein (2012), whereas an original glance at the notion was put up in Horn (2018).

3 Applications of the Moore–Penrose inverse in physics: literature overview

The number of published works, in which the notion of the Moore–Penrose inverse is utilized is steadily increasing, which is well reflected by Fig. 1. The graph concerns both, the publications of purely theoretical nature as well as those concerned with applications, but the same increasing tendency characterizes also each of these subsets.

Probably the best known application of the Moore–Penrose matrix inverse is concerned with the least squares method utilized in most research areas, in which mathematical methods are used; see Penrose (1956), where such an applicability of the inverse was indicated. As the principles of the least squares method are usually credited to Carl Friedrich Gauß (1777–1855)⁶, the Moore–Penrose inverse can be viewed as a liaison between the mathematical tool whose roots reach back to the end of 18th century (i.e., the times before linear algebra as such existed) and its present applications in modern (rather remote) research areas of applied origin. The role of the Moore–Penrose inverse in the least squares method is briefly outlined in Appendix B. Least squares, similarly as linear programming, can be considered to be a subclass of convex optimization; a reader interested in taking a glance at the least squares methods from a perspective a of convex optimization is advised to look into [Boyd and Vandenberghe 2004, Chapter 4].

An everlasting interest in the least squares method is well reflected by a immutable stream of articles, in which the method is utilized and most often plays a relevant role. Regarding recent research within broadly understood physics, exemplary applications of the method can be encountered in the following articles: Chou et al. (2018) proposing a framework for privacy preserving compressive analysis (which exploits formulae for the Moore–Penrose inverse of columnwise partitioned matrices), Gaylord and Kilby (2004) specifying a procedure of measuring optical transmittance of photonic crystals, Huang et al. (2006) introducing the extreme learning machine algorithm, Le Bigot et al. (2008) presenting high-precision energy level calculations in atomic hydrogen and deuterium, Ordones et al. (2019) deriving frequency transfer function formalism for phase-shifting algorithms, Sahoo and Ganguly (2015) optimizing the linear Glauber model to analyse kinetic properties of an arbitrary Ising system, Stanimirović et al. (2013) introducing a computational method of the digital image restoration, Wang et al. (1993) identifying sources of neuronal activity within the brain from measurements of the extracranial magnetic field, Wang and Zhang (2012) deriving an online linear discriminant analysis algorithm (which exploits formulae for the Moore–Penrose inverse of modified matrices), and White et al. (2014) elaborating a method for computing the initial post-buckling response of variable-stiffness cylindrical panels (even though the least squares method was not explicitly mentioned in the paper, we conclude it was exploited from remarks on pp. 141 and 143 stating that the systems of equations solved were overdetermined).

The usefulness of the Moore–Penrose inverse in the least squares method is hard to overestimate, but it would be a misfortune to overlook other advantages the notion offers to the researchers of diverse background. In what follows selected publications from a rich set of articles demonstrating applications of the Moore–Penrose inverse in physics and related branches are briefly discussed.

Udwadia and Kalaba (1992) considered equations of motions of a classical dynamical system with constraints of different nature (holonomic and non-holonomic, scleronomic and reonomic, catastatyc and acatastatyc) by means of an approach (original at the time) based on the Moore–Penrose matrix inverse. The main result of the paper, which the authors entitled “new fundamental principle of Lagrangian mechanics” [Udwadia and Kalaba 1992, p. 409] reads:

The motion of a discrete dynamical system subject to constraints evolves in such a way that the deviations of its accelerations from those it would have if there were no constraints, are directly proportional to the extent to which the accelerations corresponding to its unconstrained motion do not satisfy the constraints; the matrix of proportionality is the Moore–Penrose inverse of the constraints matrix...

Another thought-provoking statement in Udwadia and Kalaba (1992) is formulated at the end of the paper:

Little did Moore and Penrose realize at the time, that their invention of generalized inverses would play such a fundamental role in Nature’s design; for it is these seemingly abstract generalized inverses, that provide the key to understanding the complex interactions between impressed forces and the constraints.

The article Udwadia and Kalaba (1992) inspired other researchers to exploit the formalism referring to the Moore–Penrose inverse to deal with the problems concerned with dynamics of classical physical systems, which resulted in a number of papers belonging to this stream of considerations. Summary of the work carried out by Udwadia and Kalaba is provided in an article Udwadia and Kalaba (2002) as well as in a monograph Udwadia and Kalaba (2008). Subsequent papers belonging to this stream include inter alia: Bajodah et al. (2005), Cariñena and Fernández-Núñez (2006, 2010), Lee et al. (2009), Marques et al. (2017), and Udwadia and Phohomsiri (2006, 2007).

Further papers demonstrating applications of the Moore–Penrose matrix inverse within the scope of theoretical physics include, among other, articles: Beylkin et al. (2008), where the formulae for the inverse of modified matrices were exploited in a Green’s function iteration algorithm introduced to solve the time-independent, multiparticle Schrödinger equation, He et al. (2012), which introduced phase-entanglement and phase-squeezing criteria for two bosonic fields that are robust against a number of fluctuations using the inverse to normalize the particle number operator, Huang and Li (2020), where formulae for the resistance distance and Kirchhoff index of a linear hexagonal (cylinder) chain were derived by means of the inverses of Laplacian matrices, Kametaka et al. (2015), where the inverse of singular discrete Laplacian was used to solve difference equations to estimate a maximal deviation of a carbon atom from the steady state in C60 fullerene buckyball, Kirkland (2015) dealing with a quantum state transfer in a quantum walk on a graph, with the inverse used to derive expressions for the first and second partial derivatives of the fidelity of the transfer with respect to a weight of an edge, Kougioumtzoglou et al. (2017), where an inverse based frequency response function was introduced to generalize frequency domain random vibration solution methodologies to account for linear and nonlinear structural systems with singular matrices, Lian et al. (2019), where the inverse was exploited for calculating charge density distribution through Hartree potential to disclose the physical mechanism of electrostatic potential anomaly in 2D Janus transition metal dichalcogenides, McCartin (2009) reexpressing the Rayleigh–Schrödinger perturbation theory procedure in terms of the inverse, Meister et al. (2014), where the inverse was used to formulate an optimal control algorithm with a control subspace defined by a superposition of arbitrary waveforms, Pignier et al. (2017), where a model of an aeroacoustic sound source was created based on compressible flow simulations, with the inverse used to compute the sound source strengths, Ranjan and Zhang (2013) exploring the geometry of complex networks in terms of an Euclidean embedding represented by the inverse of its graph Laplacian, Yang et al. (2018), where the inverse was used to solve an equilibrium equation originating in an empirical mode decomposition method combining the static and dynamic information for structural damage detection, and Yang et al. (2020), where an expression for the inverse of Laplacian matrices of two connected weighted graphs was established and utilized to derive a recursion formula for the resistance distance.

A numerous set of papers reveals applications of the Moore–Penrose matrix inverse to analyse experimental data, often with the use of statistical methods and numerical calculations. This was the case in the following articles: Anand et al. (2009) dealing with an optimization of the signal to noise ratio for NMR data, Bedini et al. (2005) concerned with a separation of correlated astrophysical sources from measured signals, Saha and Aluri (2016) and Saha et al. (2008), where the already mentioned formulae for the Moore–Penrose inverse of modified matrices were exploited in an elaboration of the cosmic microwave background data.

Fig. 2

Localization error with and without using the Moore–Penrose inverse. Reproduced from [Nara T. and Ito W. 2014. Moore-Penrose generalized inverse of the gradient tensor in Euler’s equation for locating a magnetic dipole. J. Appl. Phys. 115: no. 17E504, Fig. 4], with the permission of AIP Publishing. https://aip.scitation.org/doi/10.1063/1.4861675

An exemplary result confirming advantages resulting from the utilization of the Moore–Penrose inverse in the data analysis is provided on Fig. 2. The plot was drawn from Nara and Ito (2014), where the problem of a magnetic dipole localization was considered. A neodymium magnet was moved along the coordinate denoted by x in an experimental setup designed to measure the magnetic field generated by the dipole. The experimental data was afterwards used to determine the localization of the dipole by solving a system of linear equations, the so-called Euler’s equations. The system was solved independently with and without using the Moore–Penrose inverse of a $3 \times 3$ coefficient matrix (the theory indicates that for the considered orientation of the dipole the coefficient matrix should be singular). The outcomes of the investigations are presented on Fig. 2 and the findings were summarized by the authors as follows [Nara and Ito 2014, p. 3]:

Clearly, one finds that localization becomes more accurate and stable when using the generalized [Moore–Penrose] inverse than when using a simple inverse.

The Moore–Penrose matrix inverse is beneficially exploited also in several other, often interdisciplinary, research areas, such as robotics, inverse problems, digital image restoration, diffuse optical imaging, and neural networks. Applications of the inverse in robotics deal first of all with mechanics. A number of illustrative examples were discussed in an overview article Doty et al. (1993), which is devoted to the utilization of generalized inverses in robotics. Further examples of the applications along with extensive lists of references were provided in a paper Zhao and Gao (2009) and a monograph Sciavicco and Siciliano (2000).

One of the papers, in which an inverse problem was considered, namely Nara and Ito (2014), was already recalled, but also other articles from among those listed above fall into this category. Further relevant references in this topic include a paper Potthast and beim Graben (2009), where the Moore–Penrose inverse was used to solve the so-called Amari equation in order to construct synaptic weight kernels yielding a prescribed neural field dynamics. An extensive review of inverse problems mostly in geophysics, but also with several examples originating from different branches of physics, some of which involve generalized inverses of matrices, was provided in Snieder and Trampert (1999).

Regarding the digital image restoration, besides an already mentioned article Stanimirović et al. (2013), it is worth pointing out a paper Chountasis et al. (2009), where the Moore–Penrose inverse based restoration algorithm was introduced and applied to restore an X-ray image either blurred or with a salt and pepper noise. Applications of the Moore–Penrose inverse in the image restoration were discussed also in a book Stanimirović (2018), which contains an overview of the algorithms used to determine generalized inverses. In this light, it is worth pointing out a paper Soleimani et al. (2015), not mentioned in Stanimirović (2018), in which iterative methods for computing the Moore–Penrose inverse in balancing chemical equations were considered.

Diffuse optical imaging (both, tomography and spectroscopy) also faces problems, in which the Moore–Penrose inverse proves to be helpful. They concern an image reconstruction from data dealing with a propagation of photons traveling through scattering/absorbing medium (the task which ranges from a design and optimization of experimental setups (instruments) to data analysis). Exemplary articles, in which issues of the kind are addressed with the help of the Moore–Penrose inverse include, e.g., Blaney et al. (2020a, 2020b) and Shihab Uddin et al. (2017).

A paper concerned with neural networks, i.e., Huang et al. (2006), was already mentioned above to point out its link with the least squares method. Additional relevant references in this area include papers Guerra and Coelho (2008), focused on elaboration of learning algorithms based on particle swarm optimization, Yin et al. (2017), dealing with a trajectory tracking control of a marine surface vessel, and Zhou et al. (2016), where a traffic matrix estimation was considered.

The problems concerned with the computations of the Moore–Penrose inverse are beyond the scope of the present article. Nevertheless, due to their relevance, few further remarks and important references are provided, so that an interested reader knows where to look for additional information. Let us first mention that popular calculation packages, like Matlab or Mathematica, offer built-in functions to calculate the inverse of symbolic and numerical matrices. The implementation of those functions is based on the singular value decomposition⁷ (SVD). This seems to be a well justified method, as when SVD of a matrix is known, its Moore–Penrose inverse is obtained straightforwardly, by taking an inverse of a diagonal matrix having nonzero singular values of the matrix on its diagonal. However, the SVD based algorithms, even though accurate, require relatively large amount of computational resource; for further details see e.g., Chen and Wang (2011), Jhurani and Demkowicz (2012), or Katsikis and Pappas (2008). Thus, other computation algorithms were proposed; for discussion of their features and their comparison see Stanimirović (2018). Another fact worth mentioning in the context of the computations of the Moore–Penrose inverse is its discontinuity, which is reflected by an observation that for $x \in \mathbb {R}$, $\lim \nolimits _{x \rightarrow 0^+} x^\dagger {=} \infty$, $\lim \nolimits _{x \rightarrow 0^-} x^\dagger {=} -\infty$, and $0^\dagger = 0$. It seems that the discontinuity issues did not have an impact on the applications of the inverse in any of the papers discussed in the present article, but they should definitely be borne in mind when the computations of the inverse are performed.

4 Summary

In Moore (1920) there is no mentioning of possible applications of the “generalized reciprocal”, whereas the definition of the “generalized inverse” in Penrose (1955) arose from the role it plays in solutions of certain matrix equations. The usefulness of the inverse in the least squares method was recognized in Penrose (1956). Likely, the first applications of the Moore–Penrose inverse to deal with a physical phenomenon concerned electrical engineering and were reported in the sixties of the previous century; for details see [Campbell and Meyer 2009, Chapter 5] or [Rao and Mitra 1971, Chapter 10]. Over the years, the applicability of the inverse clearly reached far above those initial ideas, but in most cases these applications are still concerned with a solvability of various matrix equations (differential, difference, integral, integro-differential). However, it also happens that the inverse occurs naturally in some derivations, without being intentionally recalled. This was the case, for example, in a paper Rançon and Balog (2019), which deals with an existence of an effective action in statistical field theory (the Legendre transform of the cumulant generating function) in presence of non-linear local constraints; in the Abstract of the article it was remarked:

...we naturally obtain that the second derivative of the effective action is the Moore–Penrose pseudo-inverse of the correlation function.

Most of the papers recalled above were published in the 21st century, that is at least eighty years after the Moore–Penrose inverse was first defined. In each of them the inverse plays an important role, in some it was indicated that the usage of the inverse is the factor, which not only distinguishes the paper from earlier works on the same topic, but which in fact enabled to distinctly improve former results. An overall conclusion originating from these articles is that the applications of the Moore–Penrose inverse in physics equally well concern theoretical as experimental investigations. Furthermore, also the spatial scale of the phenomena is not a distinctive factor, as applications of the inverse range from subatomic to astronomical scale phenomena. Another fact worth mentioning in this context is that the original Moore–Penrose inverse was over the years generalized and/or extended to various mathematical settings⁸, such as operator algebras, C*-algebras, rings or tensors, with some of these successors of the matrix ancestor introduced exclusively as a response to the problems emerging in physics. An explicit illustration of such an application driven development was provided in a paper Cao et al. (2016), in which the Moore–Penrose inverse was generalized to the cases of symmetric tensors on Lorentz manifolds and utilized to solve the equations of motion occurring in the theory of massive gravity.

There is no doubt that further seminal contributions to the research involving the Moore–Penrose inverse should be anticipated, which will still, on the one hand, reinforce the theoretical fundamentals of the concept and, on the other hand, broaden the spectrum of its possible applications in physics and related science branches. Standing for a hundred years on the frontline of physics research, the Centenarian—a fruit of an indisputable mathematical intuition of Eliakim Hastings Moore and Sir Roger Penrose—shows no sign of an exhaustion!

Appendices

Appendix A

The example below gives an insight into how the set of matrices satisfying the Penrose conditions (1) narrows down when subsequent conditions are imposed. As expected, only when all four conditions are fulfilled, the set reduces to the one element set $\{\mathbf{A}^\dagger \}$.

Example 1

Let $\mathbf{A}$ and $\mathbf{B}$ be $2 \times 2$ matrices of the forms

$$\begin{aligned} \mathbf{A} = \begin{pmatrix} 0 &{}\quad 1 \\ 0 &{}\quad 1 \end{pmatrix}\quad \mathrm {and} \quad \mathbf{B} = \begin{pmatrix} a &{}\quad b \\ c &{}\quad d \end{pmatrix}, \end{aligned}$$

(A1)

where a, b, c, d are complex numbers. Then $\mathbf{B}$ is:

• (a) an inner inverse of $\mathbf{A}$ (i.e., satisfies $\mathbf{A}\mathbf{B}{} \mathbf{A} = \mathbf{A}$) if and only if $c + d = 1$;

• (b) a reflexive inverse of $\mathbf{A}$ (i.e., satisfies $\mathbf{A}\mathbf{B}{} \mathbf{A} = \mathbf{A}$ and $\mathbf{B}{} \mathbf{A}{} \mathbf{B} = \mathbf{B}$) if and only if $c + d = 1$ and $a = c(a + b)$;

• (c) a normalized inverse of $\mathbf{A}$ (i.e., satisfies $\mathbf{A}\mathbf{B}{} \mathbf{A} = \mathbf{A}$, $\mathbf{B}{} \mathbf{A}{} \mathbf{B} = \mathbf{B}$, and $(\mathbf{A}{} \mathbf{B})^*= \mathbf{A}{} \mathbf{B}$) if and only if $a = b$, $c = \frac{1}{2}$, and $d = \frac{1}{2}$;

• (d) the Moore–Penrose inverse of $\mathbf{A}$ (i.e., satisfies $\mathbf{A}{} \mathbf{B}{} \mathbf{A} = \mathbf{A}$, $\mathbf{B}{} \mathbf{A}{} \mathbf{B} = \mathbf{B}$, $(\mathbf{A}{} \mathbf{B})^*= \mathbf{A}{} \mathbf{B}$, and $(\mathbf{B}{} \mathbf{A})^*= \mathbf{B}{} \mathbf{A}$) if and only if $a = 0$, $b = 0$, $c = \frac{1}{2}$, and $d = \frac{1}{2}$. Thus,

  $$\begin{aligned} \mathbf{A}^\dagger = \begin{pmatrix} 0 &{}\quad 0 \\ \frac{1}{2} &{}\quad \frac{1}{2} \end{pmatrix}. \end{aligned}$$

Appendix B

In what follows we outline a role which the Penrose conditions (1) play in the solvability of the system of m simultaneous linear equations with n unknowns, the problem whose various versions can be encountered in most (if not all) research areas utilizing mathematical methods. Further information on the topic can be accessed, for instance, in any of the monographs: [Ben-Israel and Greville 2003, Chapter 3], [Campbell and Meyer 2009, Chapter 2], [Rao and Mitra 1971, Chapters 2 and 3] or Wang et al. (2018).

Let us consider the system of linear equations

$$\begin{aligned} \mathbf{Ax} = \mathbf{b}, \end{aligned}$$

(B1)

where complex $m \times n$ matrix $\mathbf{A}$ and $m \times 1$ vector $\mathbf{b}$ are known, whereas $n \times 1$ vector $\mathbf{x}$ is unknown. Taking into account the number of possible solutions, we can distinguish three situations, which might be characterized by rank of the matrix $(\mathbf{A} : \mathbf{b})$, obtained by extending the matrix $\mathbf{A}$ by a column vector $\mathbf{b}$, namely:

• (a) The system (B1) is consistent and has exactly one solution, which happens if and only if $\mathrm {rank}(\mathbf{A} : \mathbf{b}) = \mathrm {rank}(\mathbf{A}) = n$.

• (b) The system (B1) is consistent and has infinitely many solutions, which happens if and only if $\mathrm {rank}(\mathbf{A} : \mathbf{b}) = \mathrm {rank}(\mathbf{A})$ and $\mathrm {rank}(\mathbf{A}) < n$. (In such a case, it is often desired to determine those vectors $\mathbf{x}$ in the set of all solutions, which have minimal Euclidean norm, the so-called minimum norm solutions.)

• (c) The system (B1) is inconsistent and has no solutions, which happens if and only if $\mathrm {rank}(\mathbf{A} : \mathbf{b}) = \mathrm {rank}(\mathbf{A}) + 1$. (In such a case, there are no vectors $\mathbf{x}$ for which $\mathbf{Ax} - \mathbf{b} = \mathbf{0}$, but one may determine those vectors $\mathbf{x}$, which minimize the Euclidean norm of the difference $\mathbf{Ax} - \mathbf{b}$, the so-called least squares solutions.)

The generalized inverses specified by subsets of the four Penrose conditions (1) provide a handy tool to deal with each of the situations specified above. This claim is based on the following facts:

• (a) The vector $\mathbf{x} = \mathbf{Bb}$ is a solution to the system (B1) for every vector $\mathbf{b}$, for which (B1) is consistent if and only if $\mathbf{B}$ is an inner inverse of $\mathbf{A}$;

• (b) The vector $\mathbf{x} = \mathbf{Bb}$ is a minimum norm solution to the system (B1) for every vector $\mathbf{b}$, for which (B1) is consistent if and only if $\mathbf{B}$ is a minimum norm inverse of $\mathbf{A}$ (i.e., satisfies $\mathbf{A}{} \mathbf{B}{} \mathbf{A} = \mathbf{A}$ and $(\mathbf{B}{} \mathbf{A})^*= \mathbf{B}{} \mathbf{A}$);

• (c) The vector $\mathbf{x} = \mathbf{Bb}$ is a least squares solution to the system (B1) if and only if $\mathbf{B}$ is a least squares inverse of $\mathbf{A}$ (i.e., satisfies $\mathbf{A}{} \mathbf{B}{} \mathbf{A} = \mathbf{A}$ and $(\mathbf{A}{} \mathbf{B})^*= \mathbf{A}{} \mathbf{B}$);

• (d) The vector $\mathbf{x} = \mathbf{Bb}$ is the minimum norm least squares solution to the system (B1) if and only if $\mathbf{B}$ is the Moore–Penrose inverse of $\mathbf{A}$.

In what follows we provide a numerical example demonstrating how the theory works in practice.

Example 2

Consider the system of linear equations of the form (B1), where $\mathbf{A}$ is as specified in (A1) and $\mathbf{b}$ is a nonzero vector of the form $\mathbf{b} = \begin{pmatrix} \alpha \\ \beta \end{pmatrix}$, with complex numbers $\alpha$ and $\beta$. Let us distinguish two disjoint situations, namely when $\alpha = \beta$ and $\alpha \ne \beta$. In the former of them, the system (B1) is consistent and has infinitely many solutions. Then $\mathbf{x} = \mathbf{Bb}$, with $\mathbf{B}$ as defined in (A1), is a solution to (B1) if and only if $\mathbf{B}$ is an inner inverse of $\mathbf{A}$, i.e.,

$$\begin{aligned} \mathbf{B} = \begin{pmatrix} a &{}\quad b \\ c &{}\quad 1 - c \end{pmatrix}, \end{aligned}$$

(B2)

in which case $\mathbf{x} = \alpha \begin{pmatrix} e \\ 1 \end{pmatrix}$ for any complex number e. Among those solutions, $\mathbf{x} = \mathbf{Bb}$ has the least Euclidean norm if and only if $\mathbf{B}$ is a minimum norm inverse of $\mathbf{A}$, i.e.,

$$\begin{aligned} \mathbf{B} = \begin{pmatrix} a &{}\quad -a \\ c &{}\quad 1 - c \end{pmatrix}, \end{aligned}$$

(B3)

in which case $\mathbf{x} = \alpha \begin{pmatrix} 0 \\ 1 \end{pmatrix}$. Clearly, the Euclidean norm of $\mathbf{x}$ then equals $| \alpha |$.

In the latter situation, when $\alpha \ne \beta$, the system (B1) is inconsistent. In this case, $\mathbf{x} = \mathbf{Bb}$ is a least squares solution to (B1) if and only if $\mathbf{B}$ is a least squares inverse of $\mathbf{A}$, i.e.,

$$\begin{aligned} \mathbf{B} = \begin{pmatrix} a &{}\quad b \\ \frac{1}{2} &{}\quad \frac{1}{2} \end{pmatrix}, \end{aligned}$$

(B4)

in which case $\mathbf{x} = \begin{pmatrix} a \alpha + b \beta \\ \frac{1}{2} (\alpha + \beta ) \end{pmatrix}$. For such an $\mathbf{x}$, the Euclidean norm of the difference $\mathbf{Ax} - \mathbf{b}$ equals $\frac{1}{\sqrt{2}} | \alpha - \beta |$.

A conclusive observation is that the vector

$$\begin{aligned} \mathbf{x} = \mathbf{A}^\dagger \mathbf{b} = \begin{pmatrix} 0 \\ \frac{1}{2}(\alpha + \beta ) \end{pmatrix} \end{aligned}$$

constitutes a minimum norm least squares solution to the system (B1).

It is noteworthy that the unique Moore–Penrose inverse can be utilized to represent classes of various not necessarily unique generalized inverses. For example, the set of all inner generalized inverses of an $m \times n$ matrix $\mathbf{A}$ can be represented as

$$\begin{aligned}&\{ \mathbf{A}^\dagger + (\mathbf{I}_n - \mathbf{A}^\dagger \mathbf{A}) \mathbf{V} + \mathbf{W}(\mathbf{I}_m - \mathbf{A}\mathbf{A}^\dagger ),\ \hbox {for\,arbitrary}\nonumber \\&\ n \times m\ \mathrm {matrices}\ \mathbf{V}, \mathbf{W} \},\nonumber \\ \end{aligned}$$

(B5)

the set of all minimum norm generalized inverses of $\mathbf{A}$ as

$$\begin{aligned} \{ \mathbf{A}^\dagger + \mathbf{W}(\mathbf{I}_m - \mathbf{A}{} \mathbf{A}^\dagger ),\ \mathrm {for\,arbitrary}\ n \times m\ \mathrm {matrix}\ \mathbf{W} \},\nonumber \\ \end{aligned}$$

(B6)

whereas the set of all least squares generalized inverses of $\mathbf{A}$ as

$$\begin{aligned} \{ \mathbf{A}^\dagger + (\mathbf{I}_n - \mathbf{A}^\dagger \mathbf{A}) \mathbf{V},\ \mathrm {for\,\, arbitrary}\ n \times m\ \mathrm {matrix}\ \mathbf{V} \}.\nonumber \\ \end{aligned}$$

(B7)

It can be verified that general representations provided in (B5)–(B7) entail expressions (B2)–(B4), respectively.