Farkas’ Lemma in the bilinear setting and evaluation functionals

We prove the following Farkas’ Lemma for simultaneously diagonalizable bilinear forms: If A1,…,Ak\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_1,\ldots ,A_k$$\end{document}, and B:Rn×Rn→R\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B:\mathbb {R}^n \times \mathbb {R}^n \rightarrow \mathbb {R}$$\end{document} are bilinear forms, then one—and only one—of the following holds: B=a1A1+⋯+akAk,\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B=a_1 A_1 + \cdots + a_k A_k,$$\end{document} with non-negative ai's\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a_i\text {'s}$$\end{document}, there exists (x, y) for which A1(x,y)≥0,…,Ak(x,y)≥0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_1(x,y) \ge 0 , \ldots , A_k(x,y) \ge 0$$\end{document} and B(x,y)<0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B(x,y) < 0$$\end{document}. We study evaluation maps over the space of bilinear forms and consequently construct examples in which Farkas’ Lemma fails in the bilinear setting. B=a1A1+⋯+akAk,\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B=a_1 A_1 + \cdots + a_k A_k,$$\end{document} with non-negative ai's\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a_i\text {'s}$$\end{document}, there exists (x, y) for which A1(x,y)≥0,…,Ak(x,y)≥0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_1(x,y) \ge 0 , \ldots , A_k(x,y) \ge 0$$\end{document} and B(x,y)<0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B(x,y) < 0$$\end{document}.


Introduction
There are many results relating the zero sets of linear or multilinear forms and their linear dependence. For example, it is well-known that if f 1 , . . . , f k and g are linear forms such Richard then g = a 1 f 1 + · · · + a k f k for some scalars a 1 , ..., a k .
Also, if then g = a 1 f 1 + · · · + a k f k , with a i ≥ 0 for all i. This is known as Farkas' Lemma [4]. Note that condition (2) above is stronger than condition (1), as can be checked by replacing x with −x.
Published in 1902, Farkas' Lemma leads to linear programming duality and has proved to be a central result in the development of linear and non-linear mathematical optimization. According to MathSciNet, over 250 papers cite Farkas' Lemma (one such contains the particularly beautiful statement by J. Franklin [7] that "I hope to convince you that every mathematician should know the Farkas theorem and should know how to use it"). One good, fairly recent source for such papers can be found among the opening articles in [5]. However, very few of these papers deal with non-linear versions of the lemma which is our interest here. In the non-linear setting, a short article describing some open problems involving polynomial matrices is contained in the work of A. A. ten Dam and J. W. Nieuwenhuis [2].
In the multilinear setting, positive results are known only in the case of two m-linear forms A and B as follows: In addition, the following weak form of Farkas' Lemma is given in [3]: On the other hand, in [1] the authors give an example of symmetric bilinear forms A 1 , A 2 and B such that and in [3] the author shows (non-symmetric) bilinear forms A 1 , A 2 and B such that Farkas' Lemma also fails for 2-homogeneous polynomials. Consider P : R 2 → R and Q : R 2 → R given by P(x, y) = x 2 and Q(x, y) = y 2 . Then P(x, y) ≥ 0 implies Q(x, y) ≥ 0, since both are always non-negative, but P and Q are independent.
In view of these examples, there appears to be little room for positive results. However in Sect. 1, below, we give a version of Farkas' Lemma for simultaneously diagonalizable bilinear forms (see Theorem 1) as well as a result analogous to (1) above (see Theorem 2).
We note that a similar, but weaker, result: (where C D means C − D is positive semidefinite), is known and can be obtained through the S-procedure as in [6]. We require, however, equality in (3) for our version of the Farkas' Lemma. Moreover, in Sect. 2, we take a closer look at the reason for the lack of positive results. Farkas' Lemma may be viewed as an application of the Hahn-Banach Theorem (unavailable to Farkas in 1902): indeed, given linear forms f 1 , . . . , f k , their positive cone is a convex set, so if g / ∈ F then it can be separated from F by a linear functional on R n * . But all linear functionals on R n * are given by "evaluation maps". Thus there is a point x ∈ R n such that and in particular, f 1 (x) ≥ 0, . . . , f k (x) ≥ 0 and g(x) < 0. This is the crucial point for the lack of Farkas type results in the multilinear (and other) settings: few linear functionals are "evaluation maps". We study evaluation maps on spaces of real-valued bilinear forms on R n (n ≥ 2) (see Propositions 3 and 4), and this enables us to produce new examples (see Examples 1 and 2) of bilinear forms A 1 , A 2 and B such that with non-negative a i 's.

Farkas' Lemma for simultaneously diagonalizable bilinear forms
In this section, A 1 , . . . , A k , and B will be bilinear forms on R n ×R n . We will use the notation B x (y) = B(x, y) (and similarly for the bilinear forms A i ). Note that a bilinear form A(x, y) can be written as We shall prove Farkas' Lemma in the following form: Proof Clearly ⇒) is trivial, so we must see ⇐).
Note that for any x ∈ R n , we have so that by the linear Farkas' Lemma there are a 1 (x), . . . , a k (x) ≥ 0 such that The A i 's are diagonalized simultaneously by a matrix U whose columns form a basis of eigenvectors {u 1 , . . . , u n }. Set and We have v i , u j = δ i j , and for each j = 1, . . . , k, Note that if r = s, then for all j, A j (v r , u s ) = 0. Therefore from ( * ) we deduce that B(v r , u s ) = 0 for r = s as well. Thus the same basis diagonalizes B. We have Consider 1 1 = (1, . . . , 1), and z = (z 1 , . . . , z n ). Then for each j = 1, . . . , k, Now, for r = 1, . . . , n, set z = e r . We have Thus Therefore B = a 1 A 1 + · · · + a k A k , with all a i ≥ 0, and the proof is complete.

Evaluation maps over spaces of bilinear forms
As we have mentioned above, linear forms on R n * identify with elements of R n , and thus are all evaluation maps: γ → γ (x). This is far from the case in most function spaces. can be separated by the measure δ x − δ y for any 0 ≤ x = y ≤ 1, although they cannot be separated by any evaluation map δ x .
We now characterize the evaluation maps on the space of bilinear forms L 2 (R n ) and on the space of symmetric bilinear forms L 2 s (R n ) in order to construct examples of positive cones F = {a 1 A 1 + · · · + a n A n : a i ≥ 0} and B / ∈ F which cannot be separated by evaluation maps. We consider L 2 (R n ) as R n×n , the space of n × n matrices with inner product: where A = (a i j ) and B = (b i j ). For a matrix A, we denote by tr(A) the trace of A and by rk(A) the rank of A. Any ϕ ∈ L 2 (R n ) * can be represented as ϕ(A) = tr(AN t ) for some n × n matrix N . As we will see below, in the symmetric setting, any ϕ ∈ L 2 s (R n ) * can be represented as ϕ(A) = tr(AS), with a symmetric matrix S. We say that ϕ is an evaluation map if there are x and y in R n such that ϕ(A) = A(x, y) = x t Ay.
We characterize the evaluation maps in L 2 (R n ) * and L 2 s (R n ) * as matrices N and S as follows.

Proposition 3 ϕ ∈ L 2 (R n ) * is an evaluation map if and only if ϕ(A) = tr(AN t ), where N has rank less than or equal to one.
Proof x t Ay = i j a i j x i y j , so set n t i j = x i y j for the entries of N t . Thus for some y ∈ R n , and so n t i j = x i y j , and N produces an evaluation map.
We now discuss the symmetric setting. We note that we can associate L 2 s (R n ) with the space R n×n s of symmetric n × n matrices. Also, any continuous linear functional ϕ : tr(M N t ), we can replace N by a symmetric matrix as follows.
For each x, y ∈ R n , we will identify an evaluation functional E(x, y) on R n×n We can associate E(x, y) to the symmetric n × n matrix Observe that if a matrix N ∈ R n×n s represents an evaluation map, then so does the matrix U NU t , where U ∈ O(n) (the orthogonal group in R n×n ). To see this, if N = E(x, y) is an evaluation map, so that E(x, y)(M) = tr(M N t ), then for any such U , where the second equality holds since N is symmetric. Therefore, if N is a symmetric n × n matrix, we can find an orthogonal matrix U so that U NU t has a diagonal representation. In particular, for any such diagonal matrix [E(x, y)] = ⎡ ⎢ ⎢ ⎣ x 1 y 1 0 · · · 0 0 x 2 y 2 · · · 0 0 · · · x i y i · · · 0 0 · · · x n y n ⎤ ⎥ ⎥ ⎦ In fact, at most two of the diagonal entries must be non-zero. Indeed, suppose that there are three non-zero entries, λ i = x i y i , λ j = x j y j , and λ k = x k y k . Since the off-diagonal entries are all 0, it follows that As a consequence, Since none of the entries is 0, it follows that all three of λ i , λ j , and λ k are of opposite sign, a clear impossibility. In addition, the argument shows that if there are just two non-zero diagonal entries, say with i = 1 and j = 2, then λ 1 λ 2 < 0.

Proposition 4 The square matrix S ∈ R n×n s represents an evaluation map if and only if S is one of the following:
(i) S = 0, or (ii) S has only one non-zero eigenvalue, or (iii) S has exactly two non-zero eigenvalues which are of opposite sign.
We now construct counterexamples to Farkas' Lemma for both cases, the four dimensional L 2 (R 2 ) and the three dimensional L 2 s (R 2 ). Note that according to our characterizations, evaluation functionals over L 2 (R 2 ) are of the form ϕ(A) = A, N where det N = 0, while evaluation functionals over L 2 s (R 2 ) are of the form ϕ(A) = A, S where det S ≤ 0. We begin with counterexamples in L 2 (R 2 ).

Example 1 Let
[I ] ⊥ ⊂ L 2 (R 2 ) be the orthogonal complement of the line spanned by the identity matrix, and {A 1 , ϕ is given by the matrix N : ϕ(X ) = X , N . We may suppose N has norm one. We will show that if ε > 0 is chosen to be sufficiently small, then ϕ cannot be an evaluation map. We have The first inequality follows from the fact that the right part is equal to the left part plus the double products of A i , N A j , N , which are terms greater than or equal to zero since 0 ≤ ϕ(X ) for all X ∈ F . Also, . Now by the parallelogram law we have which can be made smaller than one for small ε because 3 − 2 √ 2 < 0.18. Thus, for such ε, I − N is smaller than one, and N is invertible. By Proposition 3, ϕ cannot be an evaluation map.
For the symmetric case we have the following. I where ε > 0 will be chosen later. Clearly B is not in F ≡ {a 1 A 1 + a 2 A 2 : a i ≥ 0}. Let ϕ be such that ϕ(B) < 0 ≤ ϕ(X ) for all X ∈ F . ϕ is given by the symmetric matrix S, which we may suppose has norm one: ϕ(X ) = X , S . Picking a suitable small enough ε as we have done in the previous example, we have I −S < 1. But then the determinant of S must be positive: if det S ≤ 0, the line segment joining I and S would, by the intermediate value theorem, contain a matrix X with det X = 0, at a distance smaller than one from the identity. By Proposition 4, ϕ cannot be an evaluation map.
Note that no basis of [I ] ⊥ can be diagonalized simultaneously. Otherwise, all matrices would be simultaneously diagonalizable.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.