On the Tightness of Semidefinite Relaxations for Rotation Estimation

Why is it that semidefinite relaxations have been so successful in numerous applications in computer vision and robotics for solving non-convex optimization problems involving rotations? In studying the empirical performance we note that there are few failure cases reported in the literature, in particular for estimation problems with a single rotation, motivating us to gain further theoretical understanding. A general framework based on tools from algebraic geometry is introduced for analyzing the power of semidefinite relaxations of problems with quadratic objective functions and rotational constraints. Applications include registration, hand-eye calibration and rotation averaging. We characterize the extreme points, and show that there exist failure cases for which the relaxation is not tight, even in the case of a single rotation. We also show that some problem classes are always tight given an appropriate parametrization. Our theoretical findings are accompanied with numerical simulations, providing further evidence and understanding of the results.


Introduction
Optimization over the special orthogonal group of the orthogonal matrices with determinant one occurs in many geometric vision problems where rigidity of a model needs to be preserved under transformations. While the objective functions are often simple least squares costs, constraining a matrix to be a rotation requires a number of quadratic equality constraints on the elements which makes the problem non-convex. On the other hand, since both the objective and the constraints are quadratic the Lagrange dual function can be computed (in closed form) and therefore optimization of the dual problem can be considered. It turns out that this is a linear semidefinite program (SDP), which can be reliably solved with standard solvers in polynomial time.
Recent studies [9,18,28,12,14,32] have observed that in many practical applications the lower bound provided by the dual problem is often the same as the optimal value . In almost 50% of the cases, an extreme point with rank greater than one is obtained, and hence the SDP relaxations are not tight. See Section 7.3 for details.
of the primal one. In such cases duality offers a way of solving the original non-convex problem using a tight convex relaxation. For different problem classes, the prevalence of problem instances with tight relaxations varies, as illustrated by the synthetically generated instances in Figure 1. Furthermore, finding conditions that can be rigorously proven to be sufficient for a tight relaxation and strong duality for a given problem class remains a challenging research area. In this paper we focus on the converse question: For what problem classes can we find objective functions that give a non-zero duality gap and a nontight relaxation? We use tools from algebraic geometry for analyzing when this happens in the case of general quadratic objective functions over rotational constraints.
In particular we consider the three most commonly occurring parametrizations that can be realized with quadratic constraints, namely 3D-rotations represented either by matrices from SO(3) or by unit norm quaternions, and planar rotations represented by matrices from SO (2).
We consider the dual of the dual wherein all quadratic terms of the primal problem are replaced by linear terms over a set of 'lifted' variables subject to semidefinite constraints. The quadratic objective functions are then replaced by linear functions in the new variables which are known to attain their optimum in extreme points of the feasible set. By studying the extreme points of this relaxation we show that the situation is not as favorable as one might expect from the literature: Even for applications with relatively few rotations, we prove the existence of extreme points with rank strictly greater than one, and objective functions which are minimized at such points. The larger rank then prevents us from extracting a solution to the primal problem from the 'lifted' variables, and shows that there is a duality gap.
Our main contributions are: • We present a novel analysis of the duality properties for quadratic minimization problems subject to rotational constraints based on algebraic geometry.
• We characterize for several applications when the standard SDP relaxation is tight and when it is not. For instance, we give counterexamples for the registration problem with SO(3)-parametrization, showing that the SDP relaxation is not always tight since its solution may be an extreme point of rank 6. Similarly, we generate counterexamples which show that averaging of four planar rotations is not necessarily tight.
• We show that the registration problem and the handeye calibration problem with SO(2)-parametrization or quaternion parametrization are always guaranteed to produce tight SDP-relaxations.

Related work
It is well-known that finding the optimal rigid transformation that registers two point clouds can be done in closed form [27]. This is a key subroutine used in many different applications, for example, in the ICP algorithm [2]. However, registering other geometric primitives is a much harder problem. In [34,35], a branch-and-bound approach is proposed for finding the 3D rigid transformation for corresponding points, lines and planes. The same problem is solved in [9] by first eliminating the translation and then using SDP relaxations for estimating the rotation. Empirically, it was noted that the relaxations were always tight, but no theoretical analysis was given. The problem of registering multiple point clouds was solved using SDP relaxations and Lagrangian duality in [14]. The problem was further studied in [28] where it was shown that for low noise regimes, the SDP relaxation is always tight. In robotics, SDP relaxations for estimating rigid transformations in simultaneous localization and mapping (SLAM) have been explored in a number of recent papers [12,11,8,32,19]. Again, the empirical performance is generally good, the optimal solutions can be efficiently computed [5], and the relaxations are shown to be tight for bounded noise levels. Non-tight counterexamples are also provided. In computer vision, there are many structure-from-motion (SfM) pipelines that rely on solving the so-called rotation averaging problem, see [23,20,13,24,1,40,18,17]. One of the first approaches to use convex relaxations and duality in this setting was [20] where it was empirically observed that the relaxations are tight. A theoretical analysis and proof that for low noise regimes, SDP relaxations are indeed tight (no duality gap) have been derived in [15,36,18] for the problem of rotation averaging. The recent paper [17] explores this analysis to develop an efficient algorithm with optimality guarantees.
Estimating the pose of a camera also involves optimization over the special orthogonal group, [41,38,33,10]. Approaches based on minimal solvers and Gröbner bases are often used. Alternatively, we show that the camera pose problem can be solved with SDP relaxations and convex optimization. Another classical problem that involves rotations is the hand-eye calibration problem [26]. In a recent paper [21], an SDP relaxation is proposed, again, with seemingly good empirical performance.
There are several previous works with similar aims as ours, but for more general problem settings. For a general, geometric overview of the problem at hand, we recommend [37] where orbitopes are studied. An orbitope is the convex hull of an orbit of a compact group acting linearly on a vector space. The dual of the dual formulation that we study corresponds to the first order relaxation of the moment-SOS hierarchy [25], pioneered by Lasserre [30]. The approach of Lasserre has previously been applied to multiview geometry [29], but without any tightness guarantees. In [15], SDP relaxations for quadratically constrained quadratic programs (QCQP) are analyzed. Given that the SDP relaxation correctly solves a problem under noiseless observations (which is the case for the problems that we analyze), conditions are given which guarantee strong duality in the low noise regime. The size of this neighborhood is however not explicitly given. Further, a geometric interpretation of the relaxation is provided in [16]. We base our framework on the mathematical results proved in [4] where a deep connection is established between, on one hand, algebraic varieties of minimal degree and on the other hand, the study of non-negativity and its relation with sums of squares (SOS).

Contents of the paper
In the next section, we present our general problem formulation. In Section 3, applications are presented and formulated on the standard form, of which the SDP relaxation is given in Section 4. In Sections 5-7 we relate our problem to duality and analyze it using results from algebraic geometry. Our main result, a complete classification of SDP tightness for our example applications, is presented in Section 8.

Problem formulation
The class of problems that we are interested in analyzing are problems involving rotations, parametrized either by (i) p × p matrices belonging to the special orthogonal group, denoted SO(p), where p = 2 or p = 3 for planar rotations and 3D-rotations, respectively.
In addition we require that the objective function is quadratic in the variables of the chosen parametrization.
where each R i ∈ SO(p) with p = 2 or p = 3 and let vec (R) denote the column-wise stacked vector of the p × pn matrix R. Now let M be a real, symmetric (p 2 n + 1) × (p 2 n + 1) matrix, then we would like to solve the following non-convex optimization problem Alternatively we model 3D-rotations with unit quaternions, q = [q 1 , . . . q n ], and consider the similar formulation Note that not every problem with 3D-rotations may be straightforward to model on both of the formulations (1) and (2), and furthermore, their residual errors have different interpretation and therefore the formulations are not equivalent. Also, the set of quaternions forms a double covering of the set of rotations meaning that q and −q represent the same rotations [24]. Both of these problem formulations can be put in the following standard form The l quadratic equations r T A i r = 0 enforce the rotational constraints and r T e = 1 with e = [0, . . . , 0, 1] T forces the final element of r to be one. The rest of the paper will be devoted to this standard form and we will analyze it in detail.

Applications
There are several application problems that can be modelled and solved using the above formulation. Often, one would like to solve for one or several rigid transformations (a rotation and a translation). However, in many cases, one can directly eliminate the translation, and concentrate on the more difficult part of finding the rotations. Next we give several examples of rotation problems appearing in the literature. Example 1. Registration with point-to-point, point-toline and point-to-plane correspondences can be written as in (3). The residuals are all of the form where v i is a unit directional vector and y i is a point on the line. Similarly, if x i lies on a plane then set P i = n T i , where n i is a unit normal and y i is a point on the plane.
Minimizing over t gives the closed-form solution which is linear in vec (R). Inserting this into the objective function thus gives an expression which is quadratic in vec (R) and therefore can be reshaped into (3).
In [27], it is shown that the registration problem with point-to-point correspondences can be formulated as a quadratic optimization problem in the quaternion representation. We are not aware of any quadratic quaternion formulation for the point-to-line and point-to-plane cases. Example 2. Resectioning is the problem of recovering the position and orientation of a camera given 2D to 3D correspondences. Geometrically this can be done by aligning the viewing rays from the camera with the 3D points. This reduces the problem to a special case of point-to-line registration where all of the lines intersect in the camera center. Example 3. Hand-eye calibration is the problem of determining the transformation between a sensor (often a camera) and a robot hand on which the sensor is mounted. Given rotation measurements U i and V i , i = 1, . . . , m relative to a global frame, for the sensor and the robot hand, respectively, the objective is to find the relative rotation R between them by solving the following optimization problem: min An alternative formulation using quaternions can also be employed. If the unit quaternions u, v ∈ Q represent the rotations U, V ∈ SO(3) then the quaternion representing the composition U V can be written Q(u)v where Therefore the optimization problem can also be used and turned into the standard form in order to solve hand-eye calibration. Note however that due to the double covering the signs of u i and v i have to be selected consistently in order for q to give a low objective value.
Example 4. Rotation averaging aims to determine a set of absolute orientations R i , i = 1, ..., n from a number of measured relative rotations R ij by minimizing Since (11) Letting M = blkdiag (M 0 ⊗ I 3 , 0) now gives an optimization problem of the form (3). I 3 is a 3 × 3 identity matrix and the blkdiag (·) operation constructs a blockdiagonal matrix.
Similar to the hand-eye calibration problem, rotation averaging can be formulated with quaternions [22] using the objective function i =j Q(r i )r ij −r j 2 , which after simplifications yields an expression similar to (11) and hence can be put in the standard form (3).
Example 5. Point set averaging is the problem of registering a number of point sets, measured in different coordinate systems, to an unknown average model. If X i , i = 1, ..., n are 3 × n matrices containing measurements of corresponding 3D points, we want to find a 3×n matrix Y , rotations R i and translations t i such that where 1 is a column vector with all ones, is minimized. Since the variables Y and t i , i = 1, ..., n are unconstrained, they can be solved for as a function of the rotations R i , i = 1, ..., n. Assuming that the centroid of the points in X i is the origin for all i, back-substitution allows us to write the problem solely in terms of the rotations as where Letting M = blkdiag (M 0 ⊗ I 3 , 0) now gives an optimization problem of the form (3).
In the above examples we only considered the case of 3D-rotations. Note however that all of these problems also have meaningful versions in the plane for which parametrization using SO(2) yields the same type of problem.

SDP relaxation
Let us now derive the standard convex SDP relaxation for our standard form (3). Consider the objective function, which can be rewritten using trace notation as where X = rr T . Note that rank (X) = 1 and X 0. For SO (3), the condition that R i belongs to the special orthogonal group can be expressed by quadratic constraints in the entries of R i , for instance R T i R i − I = 0. Similarly, that the cross product of the first and second rows should equal the third, which ensures det R i = 1, is also a quadratic constraint. Consequently, the same constraints can be expressed by linear equations in the entries of X in the form tr (A i X) = 0. It can be checked that there are 20 linearly independent such constraints for each R i .
The corresponding program for SO (2) is similar to that of SO(3) but it only requires one constraint per rotation.

We represent a rotation by
Hence, for n rotations, only 2n variables are needed in the vector r and the unit length constraint becomes linear in the entries of X. Similarly for quaternions Q, the unit length constraint for each q i can be written as a linear constraint.
If we ignore the non-convex constraint that rank (X) = 1, then we get a semidefinite program problem over X: The objective function is linear in X subject to linear equality constraints and a positive semidefinite constraint, X 0. This leads to the following convex relaxation: Note that as we have relaxed (ignored) rank (X) = 1, the optimal value will be a lower bound on the original nonconvex problem (3). Further, if the optimal solution X * has rank one, then we say that the relaxation is tight and the globally optimal solution is obtained.

Duality and sums of squares
Consider again the objective function in (3) and the Langrangian dual problem of (15) where (γ, λ) are the dual variables. By construction this problem gives the same objective value as (15) and therefore a lower bound on the original (3). We are interested in knowing when this lower bound is tight. Let I be the ideal of the polynomials defining the constraint set, that is, a polynomial p is in I when (17) where v and v i are any polynomials. The variety V (I) consists of the feasible points {r | p(r) = 0 ∀p ∈ I}. Let R 2 denote the set of quadratic polynomials modulo I, that is, two polynomials f, g ∈ R 2 are considered equal if f − g ∈ I.
The question of tightness between the original problem (3) and the relaxation (15) and its dual (16) is related to the two convex, closed cones and Note that the cones are defined to be dependent on the constraint set of (3), and not on the actual form of the objective function r T M r. As any quadratic polynomial f in Σ is a sum of linear squares on the feasible set V (I) and hence non-negative, it follows that Σ ⊆ P .
Consider again our original problem in (3), written as It follows that r T (M − η * ee T )r ∈ P . If γ * is the optimal value of (16) with dual variables λ * , then the matrix M − i λ * i A i −γ * ee T is positive semidefinite and we can factor it into a sum of rank-1 matrices j a j a T j . Therefore, Now, r T ( i λ * i A i ) r lies in I and we can conclude that the quadratic polynomial r T (M − γ * ee T )r belongs to Σ when (γ * , λ * ) is the solution to (16).
In view of the above discussion it is clear that the convex formulations (15) and (16) can only give the same objective value as (3) when r T M r − η * is a sum of squares, where η * is the optimal value of (3). The question we are interested in answering is hence when is it possible to find an SOS for this non-negative quadratic form? If the cones are not equal, that is, Σ P , then there may exist objective functions for which the relaxation is not tight. We shall investigate this further in a constructive manner. First, we need some more tools from algebraic geometry. See also the book [3] for a general introduction. For any variety V , deg V ≥ codim V + 1. A variety is called minimal if it is non-degenerate (that is, not contained in a hyperplane) and deg V = codim V + 1. Similarly, it is called almost minimal when deg V = codim V + 2. Considering the degrees and co-dimensions of the varieties previously listed, Table 1 summarizes their characterization as minimal, almost minimal, and not minimal, for the cases n = 1, n = 2, and n > 2.

The extreme points of the SDP relaxation
In this section, we investigate further the convex cone of non-negative polynomials and that of SOS polynomials over the rotational varieties, SO(3), SO(2) and Q. The goal is to find out when the SDP relaxation is tight and to characterize all possible extreme points for the relaxation.
The following general result is proved in [4].

Lemma 1. (Blekherman et al. [4]) Let V be a real irreducible non-degenerate variety such that its subset of real points is Zariski dense. Every real quadratic form that is non-negative on V is a sum of squares of linear forms if and only if V is a variety of minimal degree.
An illustration of the result is given in Figure 2. We will now apply it to our varieties of interest 1 .

Minimal varieties
Among the varieties in Table 1, we know that only SO (2) and Q are minimal. In the remaining cases, the convex cones P and Σ defined in (18) and (19) are therefore strictly different, i.e. Σ P . The proof of Lemma 1 Σ P \ Σ Figure 2: Illustration of the closed convex cones Σ and P defined in (18) and (19). If the variety V (I) is minimal then Σ = P , otherwise Σ P .
is constructive and it allows us to generate objective functions that are non-negative, but not sums of squares, and thereby the SDP relaxations will not be tight for these optimization problems. However, our example applications have objective functions of a special form and it remains to see if there are such objectives which are not sums of squares. We will return to this question in Section 8. We can conclude that any hand-eye calibration problem defined over SO(2) or Q will always have tight SDP relaxations.

Almost minimal varieties
In the case when V is almost minimal, that is, when V is either SO(3), SO(2) × SO(2) or Q × Q (Table 1), we will still have Σ P , but the gap between the cones will be smaller. Furthermore, for problems in P \ Σ, the extreme points of the corresponding SDP relaxation can be characterized based on the theory in [4]. An immediate reformulation of Proposition 3.5 for our purposes gives the following corollary.
Corollary 1. Assume that the variety V is almost minimal and arithmetically Cohen-Macaulay. Then, the extreme points X * of the SDP relaxation in (15) either have rank (X * ) = 1 or rank (X * ) = codim V .
All of the varieties we study are smooth, and therefore arithmetically Cohen-Macaulay.
If rank (X * ) = 1 then the corresponding objective function r T M r − η * (where η * is the optimal objective value) is a sum of squares, and as shown previously can be retrieved by solving the SDP. If rank (X * ) > 1 and X * cannot be decomposed into rank-1 extreme points, then the corresponding objective function r T M r − η * is not a sum of squares. For almost minimal arithmetically Cohen-Macaulay varieties, such extreme points X * must be of rank (X * ) = codim V and there are no other possibilities.
To summarize, if when minimizing a given problem over an almost minimal variety V we obtain the extreme point X * which has rank (X * ) = 1, then we have indeed computed the globally optimal solution, but if it turns out that rank (X * ) = codim V , then the relaxation is not tight, and we do not even have a feasible solution to the original problem, just a lower bound on the optimal value.

Prevalence of non-tight problem instances
In Figure 1, we presented the results of two sets of synthetic experiments, illustrating the significance of almost minimal varieties.
In the first set of experiments, the domain is the almost minimal variety V = SO(3) and the entries of the objective function, encoded by the 10 × 10 symmetric matrix M in (3), were randomly drawn from a uniform distribution from [ -1, 1 ]. In all 1000 examples, we obtained a rank-1, globally optimal solution for the SDP relaxation, even though the variety is not minimal. This shows that the rank-6 extreme points predicted by Corollary 1 are rare in practice among the random objective functions considered. It is however possible to produce such nontight examples and we shall return to this question later.
In the second set of experiments, the optimization took place over V = SO(3) × SO (3), which is not almost minimal. The entries of the 19×19 symmetric matrix M were generated in the same way via a uniform distribution. In this case, the relaxation works poorer and various ranks are obtained for its solutions.
Remark. For neither minimal nor almost minimal varieties, the non-negative cone P becomes significantly larger than the SOS cone Σ. Non-tight SDP relaxations will be more prevalent, and various ranks will be observed for the solutions to these non-tight relaxations. A rank-1 solution will however always provide a solution to the primal problem.

Tightness of our example applications
The theoretical results in the previous section apply to general quadratic objective functions. For actual applications, the objective functions will be structured. For instance, consider the hand-eye calibration problem in Example 3. There are only purely quadratic terms of the rotation variables in the objective, and no linear ones. Hence, the last row and the last column of the matrix M will be zero. In this section, we analyze structured objective functions corresponding to different problem classes. We also relate our new results to previous ones in the literature. In Table 2 we present a complete classification of SDP tightness for our example applications. In accordance with Table 1, applications for the minimal varieties SO (2) and Q are always tight -this is a known result, as there is a single quadratic constraint (see e.g. Boyd and Vandenberghe [6]). For the almost minimal varieties, we generate rare non-tight problem instances, and for the non-minimal cases we conclude that tightness can only be guaranteed Non-tight instances found Non-tight instances found Low noise [36,15,18] Low noise [14,28]

SO(2)
Always tight Always tight Low noise [42] Low noise [14] Q Always tight Always tight Low noise [15] Not applicable Table 2: Tightness of SDP relaxations for various applications and parametrizations. Colors follow Table 1, illustrating whether the domain is minimal, almost minimal, or neither. The main new results are for the almost minimal cases, for which we have generated rare non-tight counterexamples. For the low noise cases, tightness can only be guaranteed in the low noise regime. We conclude that only the problem classes over minimal varieties come with tightness guarantees.
in the low noise regime, supported by previous works and empirically demonstrated by us.
Noise-free case. All of our considered example applications have objective functions of the form r T M r = r T U T U r = U r 2 for some matrix U . If the optimal value η * = min r∈V (I) r T M r is equal to zero (which corresponds to the noise-free case) then r T (M − η * ee T )r = r T M r ∈ P , where P is the cone of non-negative quadratic forms in (18). Further, since M has non-negative eigenvalues, M 0 and we can factor it into a sum of rank-1 matrices M = j a j a T j . It follows that r T M r = j (a T j r) 2 is SOS. This is a well-known result and it has further been studied in [15] where it is shown that for low noise levels (η * close to zero), the non-negative polynomial r T M r − η * is a sum of squares as well.

Registration and resectioning
The formulations of these two applications over the domain SO(3) are given in Examples 1 and 2, respectively. As the variety SO(3) is almost minimal (Table 1), one may wonder if there are actual problem instances that lead to non-tight relaxations and extreme points of rank-6 (Corollary 1)? In [4], there is a procedure for finding polynomials that are non-negative, but not sums of squares. However, this will in general not result in objective functions originating from registration or resectioning problems. The objective function for this type of problem is of the form r T M r = r T U T U r = U r 2 where M = U T U with U of size m × 10 and m is the number of correspondences of type point-to-point, point-to-line or point-to-plane. That in particular means that M has nonnegative eigenvalues and M 0, and there are some additional requirements as well.
In Section A in the appendix, we show how to modify the procedure of Blekherman et al. [4] in order to achieve such objective functions. For every non-tight problem instance generated with Procedure 1 described in Section A, we get a rank-6 solution X * as predicted by Corollary 1 and consequently no feasible solution is obtained. Hence, there exist indeed problem instances that are non-tight, but they are rare in practice. See also the first column of Table 2.
Relation to the empirical results of Briales and Gonzalez-Jimenez [9].
Extensive experiments using the SDP relaxation in (15) for registration over SO(3) are performed in [9], but not a single instance with a nontight relaxation among their real or synthetic experiments is found. This is consistent with our experiments, presented in Figure 1(a), where we have done an empirical analysis of SDP tightness over SO(3) for quadratic objective functions with random entries. The counterexamples are indeed rare in practice for this almost minimal variety.

Hand-eye calibration
As previously mentioned, the objective function for handeye calibration contains only purely quadratic terms of the rotation variables and no linear ones. Hence, the last row and the last column of the 10 × 10 symmetric matrix M will be zero. We tested the same procedure as for registration (see Procedure 1 in Section A in the appendix) in order to generate non-tight counterexamples with the structure of a hand-eye calibration objective. We succeeded in obtaining problem instances for averaging 8 rotation matrices that yielded objective functions with non-tight relaxations, see the second column of Table 2 for a summary. Again, all of these optimization problems attain their minima at rank-6 extreme points, in accordance with Corollary 1.

Rotation averaging
In [18], Eriksson et al. proved that the SDP relaxation for problems involving three rotations with SO(3)parametrization is always tight. This result trivially extends to SO(2). Further, for instances with more than three cameras, it is shown that in the low noise regime the SDP relaxation is tight. Low noise results applicable to SO(3) as well as SO(2) have also been presented by Rosen et al. [36], although SO(2) is parametrized by all matrix elements in their case. Fredriksson and Olsson [20] parametrize the rotation averaging problem with quaternions Q and in all the reported experiments, the SDP relaxation was always numerically found to be tight. For SO(2), Zhong and Boumal [42] proved the existence of an upper bound on the noise level for which the SDP relaxation is tight, however no explicit estimates were given 2 .
Here we present results for the case of four rotations in SO(2) (a three rotation problem is always tight [18]). Figure 3 shows the average rank of the computed SDP solution X * . The M 0 matrix in (11) was generated by sampling the relative rotation angles from N (0, σ 2 ), σ ∈  [0, 1] radians. For each noise level σ, we ran the problem 10, 000 times and plotted the average obtained rank of the lifted variables X * . The observed ranks were 1 or 2. Similar to our results, Fan et al. [19] find instances of the 2D SLAM problem with non-tight relaxations 2 , and Carlone et al. [12] present analogous results for 3D pose-graph optimization (PGO).
Relation to Mangelson et al. [32]. The planar posegraph problem with i =j is studied in [32]. Here, additional relative translation estimates t ij are present, but τ = 0 reduces the problem to rotation averaging. A 'proof' of strong duality for the Sparse-BSOS relaxation [31,39] is presented. This would imply that rotation-averaging can be solved exactly in polynomial time but that the SDP relaxation (15) still gives a duality gap. While such a weakness is entirely plausible we note that the presented proof in [32] is in fact flawed as the domain, SO(2) n using unit norm constraints on the diagonals of the rotation matrices, is incorrectly claimed to be SOS-convex (see [3] for a definition). It is not even a convex domain.
While the lack of a proof does not exclude the possibility that Sparse-BSOS is exact, our counterexamples in Figure 3 show that this is only possible if Sparse-BSOS is stronger than the SDP relaxation (15). A detailed comparison of these two formulations would reveal if this is the case. Such an in depth analysis is however beyond the scope of this paper.

Point set averaging
In previous work by Chaudhury et al. and Iglesias et al. [14,28], it has been shown that SDP relaxations are tight in the low noise regime for registering multiple point clouds, while in the high noise regime non-tight instances arise. Here we reproduce similar results, registering an artificial point set over four frames. 100 points are sampled from N (0, 1), after which one direction is squeezed with a factor 1/100, causing higher prevalence of non-tight instances. Gaussian noise, sampled from N (0, σ 2 ), was added to each point. Figure 4 shows the average rank over 10.000 problem instances of the computed SDP solution X * , for each noise level σ. σ (rel. to signal) Avg. rank

Conclusions
We have presented a framework for analyzing the power of SDP relaxations for optimization over rotational constraints. The key to our analysis has been to investigate the two convex cones of non-negative and sum-of-squares polynomials, and to establish a connection between them and the tightness of an SDP relaxation. We have shown that certain parametrizations lead to tight SDP relaxations and others do not. For our applications which have structured objective functions, we have generated non-tight counterexamples to settle the question of whether the relaxation is always tight or not.
An interesting avenue for future research is to develop algorithms that can recover a good solution from a nontight relaxation for practitioners of SDP relaxations. This was recently done for the rotation averaging problem [17]. Another interesting direction is to explore the existence of noise bounds for which the registration and hand-eye calibration problems over SO(3) are guaranteed to be tight.
A Generating non-tight leastsquares problems Using Procedure 1, one can generate non-negative polynomials that are not sums of squares, and consequently such objective functions minimized over the variety will result in non-tight relaxations. The procedure is adapted from Procedure 3.3 in [4] (for general non-minimal varieties), and presented here specifically for the SO(3)-case. We will generate explicit counterexamples for the handeye calibration problem and the registration problem that do not have tight SDP relaxations.
Procedure 1 Non-negative polynomials that are not sums of squares, p ∈ P \ Σ.
This procedure gives an objective function r T M r which is not an SOS and for which the optimal solution is a rank-6 extreme point. Recall that r is a 10-vector consisting of 9 variables and the constant term 1 and M is a 10 × 10 symmetric matrix. 2. Choose a quadratic form r T M 0 r that (i) vanishes to order at least two at each of the 6 selected intersection points and (ii) does not belong to the subspace generated by the forms (h T i r)(h T j r) for i, j ∈ {0, 1, 2, 3}.
3. For every sufficiently small δ, the quadratic form is non-negative on SO(3) but not a sum of squares.
There are a few modifications of the procedure that are required so that the objective functions do originate from a specific example application. For least-squares problems (such as hand-eye and registration), we know that M 0. This is not automatically fulfilled via Procedure 1, but we can change Step 2 to account for this. More specifically, one can choose a quadratic form M 0 which fulfills the two prescribed conditions (i) and (ii), and at the same time maximizes the minimum eigenvalue of M 0 . This can be cast as a convex optimization problem. If an M 0 0 is found, then M 0 will follow. Further, for hand-eye calibration and registration, there are linear relationships between the elements of M that should be satisfied for any problem instance, which can be accounted for when maximizing the minimum eigenvalue of M 0 .
To show that such an M corresponds to an actual problem instance, we look for, in the case of hand-eye calibration, pairs of rotation matrices (U i , V i ), i = 1, . . . , m such that the objective function in (6) gives the correct matrix M . Note that we do not need to find rotation matrices that map exactly to M , since the set P of quadratic forms that are non-negative is closed and thereby quadratic forms close to r T M r will also be in P , but not in Σ (cf. Figure 2). In practice, we solve the following minimization problem with gradient descent: where is the matrix obtained from formula (7) using (U i , V i ), i = 1, . . . , m and M is the matrix obtained from Procedure 1. Empirically, we have found that for m = 8, one can find such rotation matrices which result in a non-tight objective function.
In the case of registration, we have empirically found non-tight problem instances with 5 point-line correspondences where the residuals are of the form (4). Similarly to the case of hand-eye calibration in (21), we explicitly optimize for point-line correspondences that produce an objective function which is close to the given matrix M .

Remark.
Although Procedure 1 in general is guaranteed to find a polynomial p ∈ P \ Σ, it is not evident a priori that there exist such polynomials with the M 0 constraint. Nevertheless, in practice we are able to find many such counterexamples, with non-tight relaxations.