Geometry-based 3D face morphology analysis: soft-tissue landmark formalization

Abstract

The face is perhaps the most important human anatomical part, and its study is very important in many fields, such as the medical one and the identification one. Technical literature presents many works on this topic involving bi-dimensional solutions. Even if these solutions are able to provide interesting results, they are strongly subjected to images distortion. Thanks to the significant improvements obtained in the 3D scanner domain (photogrammetry for instance), today it is possible to replace the 2D images with more precise and complete 3D models (triangulated points clouds). Working on three-dimensional data, in fact, it is possible to obtain a more complete set of information about the face morphology. At present, even if it is possible to find interesting papers on this field, there is the lack of a complete protocol for converting the big amount of data coming from the three-dimensional point clouds in a reliable set of facial data, which could be employed for recognition and medical tasks. Starting from some anatomical human face concepts, it has been possible to understand that some soft-tissue landmarks could be the right data set for supporting many processes working on three-dimensional models. So, working in the Differential Geometry domain, through the Coefficients of the Fundamental Forms, the Principal Curvatures, Mean and Gaussian Curvatures and also with the derivatives and the Shape and Curvedness Indexes, the study has proposed a structured methodology for soft-tissue landmark formalization in order to provide a methodology for their automatic identification. The proposed methodology and its sensitivity have been tested with the involvement of a series of subjects acquired in different scenarios.

Appendix 1 – Coefficients of the fundamental forms

Since a patch can be written as an n-tuple of functions

$$ x\left( {u,v} \right) = \left( {{x_1}\left( {u,v} \right),...,{x_n}\left( {u,v} \right)} \right), $$

the partial derivative of x with respect to u can be defined by

$$ {x_u} = \left( {\frac{{\partial {x_1}}}{{\partial u}},...,\frac{{\partial {x_n}}}{{\partial u}}} \right). $$

The other partial derivatives are defined similarly.

It is possible to measure distances on a surface. In Euclidean space ℜⁿ, if $\underline p = \left( {{p_1},...,{p_n}} \right)$ and $\underline q = \left( {{q_1},...,{q_n}} \right)$ are points in ℜⁿ, then the distance s from $\underline p$ to $\underline q$ is given by

$$ {s^2} = {\left( {{p_1} - {q_1}} \right)^2} + ... + {\left( {{p_n} - {q_n}} \right)^2}. $$

Because a general surface is curved, distance on it is not the same as in Euclidean space; in particular, the form above is in general false however the coordinates are interpreted. To describe how to measure distance on a surface, the mathematically imprecise concept of an “infinitesimal” is necessary. The infinitesimal version of that for n = 2 for a surface is

$$ d{s^2} = Ed{u^2} + 2Fdudv + Gd{v^2}, $$

called First Fundamental Form, or Riemann Metric. This is the classical notation for a metric on a surface. E, F, G are functions U → ℜ such that:

$$ \begin{array}{*{20}{c}} \hfill {E = {{{\left\| {{{x}_{u}}} \right\|}}^{2}},} \\ \hfill {F = \left\langle {{{x}_{u}},{{x}_{v}}} \right\rangle ,} \\ \hfill {G = {{{\left\| {{{x}_{v}}} \right\|}}^{2}},} \\ \end{array} $$

and they are called Coefficients of the First Fundamental Form. These coefficients are given by inner products of the partial derivatives of the surface. Therefore, the First Fundamental Form is merely the expression of how the surface inherits the natural inner product of ℜ³. Geometrically, the first fundamental form allows to make measurements on the surface (lengths of curves, angles of tangent vectors, areas of regions) without referring back to the ambient space ℜ³ where the surface lies [9].

To introduce the Second Fundamental Form, the definitions of Gauss map must be given. For an injective patch x: U → ℜⁿ the unit normal vector field or surface normal N is defined by

$$ N\left( {u,v} \right) = \frac{{{x_u} \times {x_v}}}{{\left| {{x_u} \times {x_v}} \right|}}\left( {u,v} \right) $$

at those points $\left( {u,v} \right) \in U$ at which ${x_u} \times {x_v}$ does not vanish [14]. The Gauss Map is the map which assigns to each point p on a surface the point on the unit sphere ${S^2}(1) \subset {\Re^3}$ that is parallel to the unit normal N(p), or N _p.

Let x: U → ℜⁿ be a regular patch. Then

$$ \begin{array}{*{20}{c}} {e = - \left\langle {{{N}_{u}},{{x}_{u}}} \right\rangle = \left\langle {N,{{x}_{{uu}}}} \right\rangle ,} \\ {f = - \left\langle {{{N}_{v}},{{x}_{u}}} \right\rangle = \left\langle {N,{{x}_{{uv}}}} \right\rangle = \left\langle {N,{{x}_{{vu}}}} \right\rangle = - \left\langle {{{N}_{u}},{{x}_{v}}} \right\rangle ,} \\ {g = - \left\langle {{{N}_{v}},{{x}_{v}}} \right\rangle = \left\langle {N,{{x}_{{vv}}}} \right\rangle } \\ \end{array} $$

are called the Coefficients of the Second Fundamental Form of x, and $ed{u^2} + 2fdudv + gd{v^2}$ is the Second Fundamental Form of the patch x.

Usually a surface is given as the graph of a differentiable function z = h(x, y), where (x, y) belong to an open set U → ℜ². It is, therefore, convenient to have close at hand formulas for the relevant concepts in this case. To obtain such formulas let us parametrize the surface by

$$ \matrix{ {x\left( {u,v} \right) = \left( {u,v,h\left( {u,v} \right)} \right),} \hfill & {\left( {u,v} \right) \in U,} \hfill \\ }<!end array> $$

where u = x, v = y. A simple computation shows that

$$ \begin{array}{*{20}{c}} \hfill {{{x}_{u}} = \left( {1,0,{{h}_{u}}} \right),} \\ \hfill {{{x}_{v}} = \left( {0,1,{{h}_{v}}} \right),} \\ \hfill {{{x}_{{uu}}} = \left( {0,0,{{h}_{{uu}}}} \right),} \\ \hfill {{{x}_{{uv}}} = \left( {0,0,{{h}_{{uv}}}} \right),} \\ \hfill {{{x}_{{vv}}} = \left( {0,0,{{h}_{{vv}}}} \right).} \\ \end{array} $$

Thus

$$ N\left( {x,y} \right) = \frac{{\left( { - {h_x}, - {h_y},1} \right)}}{{\sqrt {{1 + {h_x}^2 + {h_y}^2}} }} $$

is a unit normal field on the surface, and the Coefficients of the Second Fundamental Form in this orientation are given by

$$ \begin{array}{*{20}{c}} {e = \frac{{ - {{h}_{{xx}}}}}{{\sqrt {{1 + {{h}_{x}}^{2} + {{h}_{y}}^{2}}} }},} \\ {f = \frac{{ - {{h}_{{xy}}}}}{{\sqrt {{1 + {{h}_{x}}^{2} + {{h}_{y}}^{2}}} }},} \\ {g = \frac{{ - {{h}_{{yy}}}}}{{\sqrt {{1 + {{h}_{x}}^{2} + {{h}_{y}}^{2}}} }}.} \\ \end{array} $$

From the expressions above, any needed formula can be easily computed. For instance, the Coefficients of the First Fundamental Form are obtained:

$$ \begin{array}{*{20}{c}} \hfill {E = 1 + {{h}_{x}}^{2},} \\ \hfill {F = {{h}_{x}}{{h}_{y}},} \\ \hfill {G = 1 + h_{y}^{2}.} \\ \end{array} $$