Influence of involuntary facial expressions on reproducibility of 3D stereophotogrammetry in children with and without complete unilateral cleft lip and palate from 3 to 18 months of age

To assess the influence of involuntary facial expressions on 3D facial stereophotogrammetry reproducibility in children with and without unilateral cleft lip, alveolus and palate (UCLP) aged 3–18 months. Three to eight 3D facial images per time point were acquired within 10 min of 31 children with UCLP and 50 controls at 3, 12 and 18 months of age. 3D mapping of two 3D facial images per subject per age was performed. Distance kits of the full face and nasolabial area were calculated. In the total subject pool, mean variation between two 3D facial images ranged from 0.38–0.88 mm. There were no significant differences within groups for the various ages. Variation between controls and UCLP subjects did not differ significantly. Variation was higher in the nasolabial area than in the full face. The influence of involuntary facial expressions on the estimation of facial growth should not be underestimated, especially in the nasolabial region of UCLP subjects aged 3 months. To improve 3D facial imaging reliability, image capturing should be performed by a trained photographer following a meticulous image capturing protocol, including thorough review after capture. Facial 3D stereophotogrammetry is a useful tool for monitoring facial growth longitudinally in young children with facial deformities, as no radiation is involved and image capture is easy and fast. It can be performed reliably in children with and without UCLP aged 3–18 months by an experienced photographer utilising a meticulous image capturing protocol.


Introduction
Maxillofacial surgeons and orthodontists have adopted 3D stereophotogrammetry of the face for diagnostic and treatment planning purposes, as well as evaluation of treatment outcomes, for example in children with orofacial clefts and adults who have undergone orthognathic surgery [1][2][3][4]. The rise of 3D facial imaging coincides with the advantages the technique offers compared to conventional direct and indirect anthropometry with the use of facial casts, and with technical advancements that simplify image acquisition. 3D stereophotogrammetry acquires complex data relating to shape and size. The technique is fast and safe and facilitates data storage for longitudinal analysis [5,6]. Furthermore, its accuracy and reproducibility have been demonstrated in several studies [7][8][9][10].
Difficulties that clinicians and researchers have encountered include a lack of normative 3D facial data and a commonly accepted method for superimposition of 3D facial images over time. Furthermore, the variation in facial expression within and across individual subjects remains a source of reproducibility errors. Subjects need to maintain a neutral facial expression in order for 3D facial images to be reproducible [11]. In adults, variation of the face at rest is submillimetre, and 3D facial imaging via stereophotogrammetry has proven to be reproducible, although variation associated with certain landmarks can be up to 3 mm [12,13]. In contrast, however, young children and babies are not capable of cooperating with instructions to maintain a neutral facial expression during 3D image capture. Operator patience and distraction of the child's attention are essential in order to obtain 3D facial images with a neutral facial expression. A recent pilot study involving ten 4-month-old infants with unrepaired cleft lip and palate concluded that changes in facial expression represented a substantial source of measurement error, with deformation of the face generally increasing from the forehead to the chin [14]. The aim of the present study was to assess the influence of involuntary facial expressions on the reproducibility of 3D facial stereophotogrammetry in children with and without unilateral cleft lip, alveolus and palate (UCLP) at ages 3, 12 and 18 months.

Ethical approval and informed consent
The study protocol was approved by the medical ethical commission of the institution in which the study was conducted (CMO 2007/163) and was performed in accordance with the ethical standards prescribed in the 1964 Declaration of Helsinki and its subsequent amendments. All subjects' parents provided written informed consent prior to their inclusion in the study.

Subjects
The UCLP group consisted of children with non-syndromic complete UCLP, and healthy Caucasian children comprised the control group. The subjects in this study are part of a large prospective longitudinal 3D study into facial growth from the ages of 3 months to 6 years. Patients with UCLP were collected within the first month after birth at the Cleft Palate Craniofacial Unit of the Radboud University Medical Centre of Nijmegen, in the Netherlands, between September 2008 and December 2011. The inclusion criteria were nonsyndromic complete UCLP and an age ≤ 3 months at the time of study entry. Only infants born at term (38+ weeks) to parents who were both Caucasian were included. Exclusion criteria were congenital malformations other than UCLP, and soft tissue bands. A total of 31 patients with UCLP were enrolled in the study.
The control group was recruited before the age of 3 months at the Maternity Clinic of the Radboud University Nijmegen Medical Centre and Regional Health Services (GGD Gelderland-Zuid) between April 2007 and September 2010. Inclusion criteria for the controls were born at term (38+ weeks) and both parents Caucasian. Exclusion criteria for the controls were occurrence of oral clefts in first-, secondor third-degree relatives. Fifty controls were enrolled in the study.

3D image acquisition
Image acquisition was performed in a designated 3D imaging room with no windows, under a consistent amount of ambient lighting. The 3dMDcranial System (3dMD Ltd., Atlanta, GA, USA) used was set up permanently and was calibrated daily for correct image acquisition. The 3D camera setup consisted of five pods, each equipped with three digital cameras and a flash. 3dMD software was used to construct virtual 3D facial images derived from the 2D images acquired by the 15 digital cameras. 3D images of the subjects' faces were acquired at 3, 12 and 18 months of age at the 3D Lab (Nijmegen, Netherlands) by trained photographers. The duration of image capture was 1.5 ms. On each occasion, three to eight 3D images were obtained within 10 min, depending on the subject's cooperation. Image quality of 3D photographs was quickly visually assessed immediately after acquisition by the photographer for completeness of 3D image data and a neutral facial expression using 3dMDpatient V4.0 software. 3D image capture was considered successful if two high-quality 3D images of the face at rest were collected on each occasion, in order to create a pair for 3D mapping.

Selection of eligible 3D images
One observer (S.B.) selected all high-quality 3D images of the face at rest from the entire sample, for each UCLP subject and each control subject at ages 3, 12 and 18 months. Inclusion criteria were (1) a neutral facial expression with the eyes open and the lips lightly opposed without straining; (2) a natural head position; (3) no data holes in the facial region medial to the ears, caudal to the hairline and cranial to menton; and (4) correct 3D image construction. Two of the 3D images captured at each occasion were selected by one observer (S.B.) based on visual similarity, in order to create a pair for 3D mapping. To assess the reliability of the selection procedure, subsets of 99 images were selected by two observers (S.B. and R.N.) independently of each other.

3D image processing
Excess data were removed from the included 3D images via the 'clean up model' function in the 3dMDpatient 4.0 software, and additional cropping of data below the neck. The resulting 3D images were exported from the 3dMDpatient 4.0 software as wavefront object (.obj) files, and imported into Maxilim version 2.3.0.3 (Nobel-Biocare, Mechelen, Belgium) for further analysis. See Fig. 1 for examples of 3D images of a control subject and a UCLP subject at ages 3, 12 and 18 months.
The cephalometry plug-in tool in the Maxilim software was used to create boundary planes of the nasolabial region in five steps [15,16]. In short, the 3D facial image was positioned in natural head position by (1) indicating right and left exocanthion and (2) rotation of the axis through the right and left exocanthion until the canthion-superaural line was horizontal, seen from a lateral view. Second, a pupil reconstructed point was located at the intersection of the midline of the nose and the bipupilar line derived from a frontal view. Third, the reference frame was derived from horizontal (y = 0), median (x = 0) and vertical (z = 0) planes. Fourth, the following landmarks were located: endocanthion right, endocanthion left, cheilion right, cheilion left, lower lip (middle lowest point of lip). Lastly, three additional planes were created and the nasolabial region was defined (Fig. 2): endocanthion-cheilion left plane and endocanthion-cheilion right plane were perpendicular to the vertical plane and defined the left and right borders of the nasolabial area. The lower lip plane was parallel to the horizontal plane, and these planes defined the lower and upper borders of the nasolabial area, respectively.

Image analysis
The 3D patient-plug-in tool in the Maxilim software was used to map pairs of 3D images. Mapping utilises a mathematical approach to minimise the distances between two superimposed 3D images, also known as best-fit registration or the iterative closest point algorithm. First, initial alignment of the two 3D images was performed by indicating the left and right exocanthion, tip of the nose and pogonion. Second, the surface for registration was selected manually by indicating the face and excluding the hair, ears and neck directly below the border of the mandible and menton in the pair of 3D images. Third, final alignment was computed and intersurface distance was calculated to create a distance kit of the face (Fig. 3). Lastly, a second distance kit was calculated of the nasolabial area only. Distance kit numerical data were exported to a text file. Analysis of the distance kits was performed via Matlab software (MathWorks, MA, USA). Distance kits were used to determine the variation of the 3D facial surface as a whole, as well as the nasolabial surface in particular, as this is the region of interest in UCLP subjects. In this way, variation in 3D facial surface represents the error of the 3D imaging method due to involuntary facial expression.

Statistical analysis
Cohen's kappa statistics were used to assess the inter-rater agreement for the process of image selection per subject. For patients and controls (full face and nasolabial area) the absolute mean variation, 50th percentile (p50), 90th percentile (p90) and 95th percentile (p95) of the intersurface distances were calculated as measures of the mean error and the distribution of variance. To assess the influence of age, one-way analysis of variance was used. t tests were applied to assess differences in intersurface distances between patients and controls, and to assess differences between full face and nasolabial areas. A 95% confidence interval was applied when interpreting the reliability of the results. Boxplots were generated to represent descriptive statistics of the mean intersurface distances for each age and group.

Sample
During the recruitment period, 143 children with clefts were registered at the Cleft Palate Craniofacial Unit within 3 months after birth. A total of 31 patients with UCLP were enrolled in the study. Reasons for exclusion were having a syndrome (n = 29), a condition other than complete UCLP (n = 76), non-Caucasian parentage (n = 1) and lack of informed consent (n = 6).

Reliability of image selection
Inter-rater agreement for image selection was very good, with a kappa value of 0.90. Table 1 shows the results of the selection process. In total, 510 3D images of controls and 183 3D images of UCLP subjects were available for the selection process. After the selection process, 197 images were included from the control group and 103 were included from the UCLP group. From these included 3D images, 56 pairs of 3D images were selected from the control group and 34 pairs were selected from the UCLP group.

Assessment of variation in pairs of selected 3D images: control vs. UCLP subjects
Mean variation in pairs of selected 3D images ranged from 0.38-0.66 mm in the control subjects and 0.40-0.88 mm in the UCLP subjects. p50 variation in pairs of selected 3D images ranged from 0.28-0.51 mm in the control subjects and 0.26-0.51 mm in the UCLP subjects. p90 variation in pairs of selected 3D images ranged from 0.77-1.23 mm in the control subjects and 0.85-1.40 mm in the UCLP subjects. p95 variation in pairs of selected 3D images ranged from 1.01-1.71 mm in the control subjects and 1.14-3.00 mm in the UCLP subjects. No statistically significant differences in variation of the full face were detected between control subjects and UCLP subjects. However, for the nasolabial area, there was a statistically significant difference between controls and UCLP subjects in the p95 variation at age 3 months (Tables 2 and 3).
Assessment of variation in pairs of selected 3D images: the full face vs. the nasolabial area Mean variation in pairs of selected 3D images in control subjects ranged from 0.38-0.41 mm for the full face and 0.57-0.66 mm for the nasolabial area. p50 variation in pairs of selected 3D images in control subjects ranged from 0.28-0.30 mm for the full face and 0.41-0.51 mm for the nasolabial region. p90 variation in pairs of selected 3D images in control subjects ranged from 0.77-0.84 mm for the full face and 0.98-1.23 mm for the nasolabial region. p95 variation in pairs of selected 3D images in control subjects ranged from 1.01-1.09 mm for the full face and 1.34-1.71 mm for the nasolabial region. There were statistically significant differences in the mean, p50, p90 and p95 variation of the full face compared to the nasolabial area at all ages in the control subjects, with the exception of p90 and p95 at the age of 12 months (Tables 2 and  3). Mean variation in pairs of selected 3D images in UCLP subjects ranged from 0.40-0.50 mm for the full face and 0.69-0.88 mm for the nasolabial area. p50 variation in pairs of selected 3D images in UCLP subjects ranged from 0.26-0.35 mm for the full face and 0.45-0.51 mm for the nasolabial region. p90 variation in pairs of selected 3D images in UCLP subjects ranged from 0.85-1.05 mm for the full face and 1.20-1.40 mm for the nasolabial region. p95 variation in pairs of selected 3D images in UCLP subjects ranged from 1.14-1.38 mm for the full face and 1.74-3.00 mm for the nasolabial region. There were statistically significant differences in the mean, p50, p90 and p95 variation of the full face compared to the nasolabial area at 3 months of age (Tables 2 and 3).

Assessment of variation in pairs of selected 3D images at ages 3, 12 and 18 months
Descriptive statistics for the absolute mean variation, p50, p90 and p95 for controls and UCLP subjects of the face and nasolabial area in each age group are shown in Table 2. Mean variation in pairs of selected 3D images ranged from 0.41-0.88 mm at age 3 months, 0.38-0.71 mm at age 12 months and 0.39-0.69 mm at age 18 months. p50 variation in pairs of selected 3D images ranged from 0.30- Graphical presentation of the data Figure 4 shows boxplots representing the mean variation in pairs of selected 3D images of the face and the nasolabial area b-k Significant differences in mean, p50, p90 and p95 variation of the full face compared to the nasolabial area at all ages in controls, with the exception of p90 and p95 at age 12 months l-o Significant differences in mean, p50, p90 and p95 variation of the full face compared to the nasolabial area at 3 months of age in UCLP subjects Table 3 p values and 95% confidence intervals for each statistically significant difference indicated in Table 2 Significant difference closer to each other compared to the boxplots representing the variation of the nasolabial area. This indicates that there was a tendency towards more variation in the nasolabial area, with the largest variation in the nasolabial area in UCLP subjects at 3 months of age. The least variation was apparent in the full face images of control subjects. Variation of the mean, 50th, 90th and 95th percentiles is presented as the standard deviation (sd) in Table 2. The sd gradually increases from p50 to p95, with the exception that the mean and sd of p95 for the nasiolabial area in UCLP patients at age 3 months exhibits a large increase to 3.00 (sd 2.47).

Mean variation
To our knowledge, to date, only Hermann et al. [14] have addressed variation due to 3D facial expression, in a pilot study in infants, although stereophotogrammetry has often been applied in previous studies with non-compliant infants and less compliant children. In Hermann et al. [14], alignment of 3D facial images of 10 infants (mean age 3.9 months, seven with UCLP and three with bilateral cleft lip, alveolus and palate) was performed on the forehead via an iterative closest point algorithm, and the results were compared with data derived from 10 adults and a static mannequin head. In that study, deformation generally increased moving downward from the forehead to the chin, and mean variation was 0 mm at the forehead, 1.1 mm at the nose, 1.7 mm at the cleft region, 3.1 mm for the lower lip and 3.5 mm for the chin. In the current study, mean variation was 0.88 mm at the nasolabial region in UCLP subjects at the age of 3 months, and mean variation was 0.66 for the full face. The value of 0.88 mm is half of the value of 1.7 mm that Hermann et al. [14] reported at the cleft region. This difference may be due to differences in consistency of the image capture procedure, i.e. the experience of the photographer. Experience of the physician in case of direct anthropometry has been demonstrated to be a factor of influence as well, as it is likely to be with any other technique [17]. Other potentially contributory factors are that the two studies used different methods for the alignment of 3D facial images, and there were differences in the size and homogeneity of the samples. In conjunction with the selected methods of alignment, it can be expected that the error in a particular facial area is related to its distance to the area of reference, i.e. mapping on the forehead will lead to a larger error in the area of the chin. On the other hand, it is reasonable to assume that aligning on the full facial surface will average variation around various facial regions, i.e. regression to the mean. In the adult control sample in Hermann et al. [14], there was a similar increase in deformation moving downward from the forehead to the chin; however, the values were a factor of 10-20 times less (0.09 mm at the forehead to 0.25 mm at the chin) than those derived from the infant sample. These values are Fig. 4 Boxplots of the mean variation (mm) in the control group and the UCLP group at ages 3, 12 and 18 months for the whole face and the nasolabial area suggested to be clinically acceptable and indicate that adults are sufficiently able to present a neutral facial expression. In a static mannequin, however, head deformation did not increase moving downward from the forehead to the chin (0.032 mm at the forehead to 0.027 mm at the chin) [14] indicating that alignment on the forehead is a perfectly reliable method in a static mannequin. Notably, it also indicates that even in compliant adults, minor deformation in the surface of the forehead exists.
Variation of the face at rest using 3D stereophotogrammetry in adults has been described by Maal et al. [12] and Lubbers et al. [13]. Maal et al. [12] used surface-based registration of 25 3D images of the same adult individual taken with and without wax bite in maximal occlusion at T 0 and repeated the process after 6 weeks. They reported a mean variation of 0.25 mm and no large differences between different time points when a wax bite was used. There were small variations in anatomical regions of the face, the smallest variation being in the nose and forehead and the largest variation in the mouth and eyes. In the current study, the mean variations of the face were 0.40 mm in the control sample and 0.44 mm in the UCLP sample, which are larger values than that reported by Maal et al. [12]. This is to be expected, due to the non-compliant nature of infants with regard to maintaining a neutral facial expression during 3D image acquisition. However, the differences at 3, 12 and 18 months of age were small in the current study, and not clinically relevant. Lubbers et al. [13] examined calliper distances of labelled landmarks in two adult subjects and reported a mean global error of 0.41 mm, of which technical error accounted for 0.09 mm and involuntary facial movements accounted for 0.32 mm. This is comparable to our findings in controls and UCLP subjects at the ages of 3, 12 and 18 months.
To estimate the clinical significance of error due to changing facial expressions, it is relevant to compare this error with the system error and with normative data of craniofacial growth. The accuracy of the 3dMD system compared to direct facial anthropometry for linear measurements reportedly ranges from 0.21 to 0.94 mm [9,10,18,19], and for landmark identification it ranges from 0.49 to 0.83 mm [11,20,21]. The mean r value representing the correlation between linear measurements derived from the 3dMD and direct anthropometry was reportedly 0.88 in one study, indicating a very high correlation [22]. The error due to involuntary facial expressions in the current study ranged from 0.38 to 0.50 mm, and actually includes the system error itself. Therefore, the influence of involuntary facial expressions seems to be clinically insignificant. However, when summing up all potential factors like system error, landmark identification error and error due to involuntary facial expression a clinically significant error is possible.
Furthermore, studies that produced normative data of craniofacial growth from 0 until 2 years of age in North American Caucasians measured with direct anthropometry have shown that linear cranial growth was 11 to 20 mm in width and 21 to 27 mm in height; linear facial growth was 5 to 9 mm in width and 10 to 17 mm in height; linear growth of the nose was − 0.2 to 1.7 mm in width, 7.2 to 8 mm in height and 0.6 to 1.8 mm in length; and linear growth of the lips and mouth was 4.4 to 5.2 mm in width and 0.6 to 4.1 mm in height [23]. Compared to these increments for linear measurements due to craniofacial growth, the error due to involuntary facial expressions in the current study (0.38 to 0.50 mm) seems to be clinically insignificant except for measurements of the width and length of the nose and height of the lips and mouth. Future studies on longitudinal 3D facial growth in young children should take the mean variation in facial expression into consideration in relation to the mean variation due to facial growth.

Assessment of variation in pairs of selected 3D images at ages 3, 12 and 18 months
The mean, p50, p90 and p95 of the variation in the face and nasolabial area of controls and UCLP subjects did not differ significantly in the current study. Age was not related to variation in facial expression in children with or without UCLP at the ages of 3, 12 and 18 months. The explanation for this is that a carefully instructed photographer can capture multiple 3D facial images of an infant aged between 3 and 18 months with the same level of neutral facial expression. Results reported by Hermann et al. [14] indicate that the ability of a photographer to recognise a neutral facial expression is one of many factors related to variations in 3D images. In the current study, 43% of all available 3D facial images of controls and UCLP subjects were included (Table 1). There was a tendency towards a higher percentage of included 3D facial images in subjects at 3 months of age (58%) and a tendency towards a lower percentage of included 3D facial images in subjects at 12 months of age (32%) and 18 months of age (38%). An explanation for the higher percentage of included 3D facial images of subjects aged 3 months might be that they are less aware of the process of image capturing, and therefor unconsciously more cooperative or easier to district than subjects aged 12 and 18 months. We recommend a thorough review by the photographer after 5-8 3D images of the same child have been captured, in order to maximise the number of high-quality 3D images with a neutral facial expression. Furthermore, the quality of the process of selecting eligible 3D images has a strong influence on outcome. To achieve a good selection of eligible 3D images of the same child, two observers were calibrated for the independent selection of eligible 3D images. The inter-observer agreement of 0.90 indicates that selection of high-quality 3D images can be reliably performed. This is a crucial factor for both reliability of results within a single study and for comparison of results derived from different studies.

Assessment of variation in pairs of selected 3D images: controls vs. UCLP subjects
In general, no statistically significant difference between controls and UCLP subjects was found, indicating that the presence of an unrepaired or repaired cleft had little influence on the mean variation in facial expression. Therefore, 3D imaging seems to be a robust imaging technique. An exception to this was the p95 of the nasolabial area. In UCLP subjects aged 3 months, involuntary facial expressions significantly influenced variation in the nasolabial area of unrepaired UCLP faces. Excluding the point cloud of the face around the nasolabial area seems to eliminate the regression to the mean for the outliers. Mean variation in the nasolabial area at age 3 months in UCLP patients was highest of all, at 0.88 mm. It was still well below 1 mm, and however, compared to the estimations of facial growth in normative data, this seems to be clinically acceptable [23].
Assessment of variation in pairs of selected 3D images: the whole face vs. the nasolabial area Statistically significant differences were found in the mean, p50, p90 and p95 variation of the face compared to the nasolabial area for all ages in controls and at age 3 months in UCLP subjects. As expected, variation was larger in the nasolabial area than in the full face due to movement of the lips and mandible. In the total subject pool (including controls and UCLP subjects), the p95 of the mean variation of the face ranged from 1.01 to 1.38 mm, whereas the p95 of the mean variation of the nasolabial area in UCLP at age 3 months was up to 3.00 mm with a sd of 2.47 mm, indicating that in this specific region, reliability was questionable. Conversely, mean variation in the nasolabial area was 0.88 mm at most, which is well below 1 mm and therefore clinically acceptable. In general, the upper quartile was below a mean variation of 1 mm. Exceptions to this were the nasolabial region in UCLP subjects at age 3 and 18 months and the nasolabial region of controls at age 12 months. Also, outliers in the higher mean variation were present indicating an unsteady character of the nasolabial region due to variation in involuntary facial expression.

Conclusions
Caution should be exercised when interpreting 3D facial imaging data derived from young children in order to estimate treatment outcomes or growth, especially when the data include 3D facial images of the nasolabial region of UCLP subjects aged 3 months. Mean variation was well below 1 mm in both the controls and the UCLP subjects at all ages investigated in the current study, for both the face and the nasolabial area, which is clinically acceptable. When estimating treatment outcomes or facial growth, the influence of involuntary facial expressions should not be underestimated. There is a need for suitably experienced photographers, meticulous 3D image capturing and evaluation protocols, to obtain a high-quality 3D facial image database. Furthermore, a rigid selection procedure is required to identify 3D facial images with a neutral facial expression.
Funding This work was supported by a grant from the Dutch Technology Foundation to STW (grant no. 10315).

Compliance with ethical standards
Conflict of interest The authors declare that they have no conflict of interest.
Ethical approval All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards (ABR no.: NL 17934.091.07).
Informed consent Informed consent was obtained from all individual participants included in the study.
Open Access This article is distributed under the terms of the Creative Comm ons Attribution 4.0 International License (http:// creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.