Background

Appraisal of tooth movement, through digital superimpositions is critical in contemporary orthodontic protocols. Careful evaluation and quantification of superimpositions, enable professionals to understand capabilities and limitations of appliances and mechanics employed therein [1,2,3,4,5].

Digital intraoral models derived either indirectly from model scans or directly by intraoral scans are the first step in obtaining a detailed 3D representation of the dentition, on which planning, tracking, measurements, as well as simulations are performed [6,7,8,9]. Tooth movement can be quantified by registering serial 3D models acquired at different time points where they can be combined in the same spatial coordinate system [10]. Various techniques and software packages have been used for 3D digital registration of virtual models as well as for tooth movement measurements, so as to quantify treatment effects between time lines.

Software packages differ in the registration methods they offer, in the method of measuring 3D tooth movements, in costs, as well as in the time and complexity to perform a specific task [11, 12]. Accurate registration of serial models on stable and identifiable structures is mandatory [13].

Available software packages for model registration use a combination of computer-based algorithm and operator data-input. Software packages work by combining large amounts of data with fast, iterative processing and intelligent algorithms, allowing them to learn automatically from patterns or features in the data [14, 15]. Recently, registration software packages have been marketed which operate solely based on algorithms with minimal operator input [16].

Algorithms available for 3D registration include 1) Surface-based (registration based on surface area to bring the two preoperative and postoperative models to fit to each other) e.g. Geomagic software [17], 2) Landmark-based (software-assisted best-fit registration of arbitrary selected anatomical points/specified area) e.g. OrthoAnalyzer software [18], or 3) Information theory and mathematical algorithm technique-based (software-assisted superimposition of the registered structures) e.g. Compare software [19]. Landmarks and regions of interest used for registration should fulfill the basic requirement of being stable during growth and unchanged by bone modeling associated with orthodontic tooth movement especially during space closure after extractions [13]. Up till now, there is no consensus in the literature regarding the techniques to superimpose serial 3D intraoral digital models [3].

Several limitations exist in current literature comparing different registration techniques [3]. Most of them [20, 21] assessed the reliability of the superimposition technique in comparison to cephalometric radiographs which themselves are unreliable. Studies using implants as standard reference [22, 23] did not evaluate precisely the stability of the implants in the first place. In addition, most studies [11, 24] have compared their tooth movement outcomes with a doubtful self-assumed standard reference point/plane, which has not been validated. Studies have used different software packages that register digital models and assess 3D tooth movements to quantify treatment effects, but validation of the package itself remains untested. This study aims to answer this knowledge gap.

The aim of this study was to evaluate the accuracy of three different 3D digital model registration software packages that measure the amount of tooth movement, using predetermined amount of simulated tooth movement on a 3D Digital Setup, as a reference standard.

The null hypothesis was that there is no agreement between the predetermined tooth movement generated by the Digital Setup and the different digital model registration software packages in measuring the amount of linear tooth movements evaluated.

Materials and methods

Study design

This diagnostic accuracy and agreement study followed a modification of the Guidelines for Reporting Reliability and Agreement Studies (GRRAS) where each software package was considered as a rater [25]. IRB approval was obtained at the Faculty of Dentistry, Alexandria University (IRB: 00010556-IORG: 0008839), and informed consents sought from the subjects whose scans were used as a study material. Access to the original scans was limited to the principal investigator. All potentially identifiable patient information were removed from the scans. Scans were stored in one computer terminal connected to a protected server. The minimal sample size was calculated based on previous studies that evaluated the reliability of newly developed software calculating 3D tooth movements [12, 26]. Based on the results, a sample size of 20 scans was deemed enough to conduct this agreement study [27], with minimum accepted reliability ρ0 = 0.6 and maximum expected reliability ρ1 = 0.9, k = 3, where k corresponds to the number of tested software packages. The statistical significance alpha was set at 0.01 to account for multiple comparisons and a statistical power, 1 − β = 0.9. The minimum calculated sample size was 18, increased to 20 to account for defective scans.

Sample collection

The sample consisted of full arch pretreatment maxillary and mandibular intraoral digital scans of actual adult patients undergoing CAT. All scans were randomly selected from the records of a single orthodontic office in Mumbai, India with more than 15 years of experience with CAT. A random number list of 20 was generated using Microsoft Excel from the total number of scans available in the office archive. The scanner used was a TRIOS 3-D intraoral scanner (3Shape, Copenhagen, Denmark). The scan data was then exported in STL format file extension and the files were imported into the three studied software packages and analyzed in the Department of Orthodontics, Alexandria University. The study group comprised scans of 20 patients with a Little’s irregularity index that ranged from 4 to 6 mm. All teeth in both arches were evaluated for 3D tooth movements except for third molars. The inclusion criteria for the scans were (1) Adult subjects treated with CAT who received treatment in both arches, (2) Scans had to be complete and of acceptable quality with a full complement of teeth except for the third permanent molars. Scans were excluded if (1) Treatment involved extraction of permanent teeth, (2) Teeth had surface anomalies or if (3) Scans had soft-tissue lesions covering the palate or the mucogingival junction (MGJ) of the mandibular arch, (4) Scans that had any previous history of orthodontic /orthopedic treatment.

All the scans that met the eligibility criteria were given an identification number. All digital scans were de-identified by an independent investigator, and imported into the three different tooth measuring software programs for the principal investigator to evaluate (Fig. 1).

Fig. 1
figure 1

Research flowchart

Full size image

Procedure

Digital Setup

Full arch maxillary and mandibular pretreatment scans (T1) were imported to OrthoAnalyzer software (3Shape Ortho System, Copenhagen, Denmark). Scan preparations were deemed necessary for all maxillary and mandibular pretreatment scans. Models were trimmed and oriented into three planes. Virtual Digital Setups were done by using virtual segmentation techniques. After defining the mesial and distal edges of each tooth, the software automatically creates a cut spline for each tooth that can be manually adjusted if needed. The original occlusal plane and vertical planes were determined and used as a reference. The long axis of each tooth was determined and teeth were moved virtually to their desired ideal position in the arch. All linear tooth movements were visualized and quantified in three directions (buccolingual BL, mesiodistal MD, occluso-gingival OG). Tooth movement measurements of this Digital Setup (DS) were tabulated for all teeth and used as reference for measuring accuracy of the three different software packages. The DS were exported as STL model files and termed (T2) (Fig. 2).

Fig. 2
figure 2

Segmentation of teeth and virtual tooth movements during Digital Setup generation

Full size image

T1 and T2 models were imported as STL files to the tooth measuring software programs, for registration and 3D linear measurements. The three studied software packages were:

  1. 1.

    Geomagic (G)-(Geomagic U.S., Research Triangle Park, NC) using landmark based method followed by regional global surface closest point registration algorithm (ICP) [17].

  2. 2.

    OrthoAnalyzer (O)-(3Shape Ortho System, Copenhagen, Denmark) using surface 3-point method of registration [18].

  3. 3.

    eModel 9.0 “Compare” (C)-(Geodigm Corporation, Chanhassen, MN) using automatic surface to surface closest point registration algorithm (ICP) [19].

After importing all STL files into the different software packages, the following steps were conducted before measurements were made: 1. Registration, 2. Coordinate system generation, 3. Measurement of tooth movement

  1. 1.

    Registration of the initial model and the Digital Setup using the three software packages

Geomagic software (G): [17] Registration was done in two steps (Fig. 3).

  1. 1.

    Landmark based registration: The program was instructed to carry out an initial superimposition based on 3 points on the medial ends of third and second rugae areas in the maxillary arch and 3 points on the mucogingival junction (MGJ) between first premolar and second premolar, second premolar and first molar, first molar and second molar.

  2. 2.

    Global and fine regional surface registration: The program made fine-tuned automatic adjustments to the spatial position of two models based on all points and changes the coordinates of one object to match the other using a best-fit surface algorithm.

Fig. 3
figure 3

Registration of a maxillary models, b mandibular models, c corresponding heat maps by Geomagic software. Three points on the medial ends of third and second rugae areas in the maxillary arch and three points on the mucogingival junction (MGJ) between first premolar and second premolar, second premolar and first molar, first molar and second molar were selected as reference landmarks, followed by a global and fine regional surface registration

Full size image

OrthoAnalyzer software (O): [18] (Fig. 4)

Fig. 4
figure 4

Registration of a maxillary models, b mandibular models with corresponding heat maps by OrthoAnalyzer software. In the maxillary arch, three points on the medial ends of third and second rugae areas, plus an area of the palate limited anteriorly by the medial 2/3 of the third rugae and laterally by two lines parallel to the mid-palatal suture were used as the landmark. In the mandibular arch, three points on the MGJ between first premolar and second premolar, second premolar and first molar, first molar and second molar, plus an area 1 mm above and below the selected points on the MGJ

Full size image

Registration was done using surface 3-point method which involved selecting three landmarks on each of the corresponding models followed by painting an area of known stability to be used for surface-based registration. In the maxillary arch, 3 points on the medial ends of third and second rugae areas, plus an area of the palate limited anteriorly by the medial 2/3 of the third rugae and laterally by two lines parallel to the mid-palatal suture were used as the landmark. In the mandibular arch, 3 points on the MGJ between first premolar and second premolar, second premolar and first molar, first molar and second molar, plus an area 1 mm above and below the selected points on the MGJ.


Compare software (C): [19] Registration was done in three steps (Fig. 5).

  1. 1-

    Model trimming and segmentation of individual teeth: To ensure that the analysis was based solely on tooth-surface features, interproximal papillae and the model base apical to the gingival margin were removed from both the T2 and T1 models. T2 models were then segmented to isolate each tooth as a separate object for superimposition on unsegmented T1 models.

  2. 2-

    Global initial alignment: The dental arches were initially aligned globally, by three-points based on the mesial-buccal cusps of the first molars and the mesial-incisal point of the right central incisor in each arch that were used as matching points for initial registration. Alignment was based on a predefined fit region, which consisted of the occlusal surfaces of the first molars and premolars, tips of canines, and incisal edges of lateral and central incisors. This region was chosen to represent the average occlusal plane for each arch. This initial registration was then refined by 30 iterations of a closest-point algorithm to achieve best fit of the occlusal surfaces.

  3. 3-

    Best fit surface registration: The software then automatically superimposed individual teeth from the segmented T2 models on the corresponding teeth in the unsegmented T1 models using best-fit surface-based registration.

Fig. 5
figure 5

Registration of maxillary models and mandibular models by Compare software. a Global alignment of maxillary and mandibular T2 (orange) over T1 model (white). b Superimposition of individual segmented teeth from T2 (green) over the unsegmented T1 model (white). c Placement of coordinates at the center of resistance of each tooth

Full size image

  1. 2.

    Generation of 3D coordinate system and orthogonal planes for tooth movement measurements

After registration, a three-dimensional (3D) coordinate system along the 3 principal axes was generated for tooth movement measurements. According to the software used, either model (Geomagic, OrthoAnalyzer) or tooth (Compare) global reference frames were generated. Model global reference frames are defined as a coordinate system of three mutually perpendicular, intersecting axes (x = sagittal/ anteroposterior, y = vertical/occluso-gingival, and z = transverse/mediolateral). The “x-axis” is defined as the intersection of sagittal and occlusal planes, the “y-axis” as the intersection of the sagittal and coronal planes and the “z-axis” as the intersection of the coronal and occlusal planes [28]. Orthogonal planes defined by the 3 principal axes were also generated.

The 3D planes of space are the transverse plane (occlusal plane) (XZ) which is defined as the plane passing through cusps of bilateral maxillary first molars, second premolars and first premolars. Midsagittal plane (XY) which is defined as the plane perpendicular to the transverse plane and passing through the median palatine raphe, and finally, the frontal plane (coronal) (YZ) which is the plane perpendicular to both the transverse and midsagittal planes. The midsagittal plane was constructed on the mid-palatal suture of the maxilla and then subsequently transferred to the mandible.

For Geomagic, one global model reference frame with the three mutually perpendicular intersecting axes (X, Y, Z) and orthogonal planes was constructed to measure all tooth movements (Composite Model Coordinates). On the other hand, for OrthoAnalyzer, each tooth required the generation of its own spatial model reference frame with the corresponding axes, and planes generated individually to measure tooth movements (Repeated Model Coordinates). However, for Compare, a local tooth reference frame that the software automatically generates, defining the principal local coordinate tooth axes was generated (Automated Tooth Coordinates). The axes were x = mesiodistal (red), y = buccolingual (green), and z = occluso-gingival (blue). Once the axes were placed on the T1 model, the software automatically generated analogous axes for each corresponding tooth in the T2 model.

  1. 3.

    3D tooth movement measurements

After all digital models (T1 and T2) were oriented in the same coordinate system via registration, it was possible to evaluate how the tooth positions changed.


Measurements made were:

Linear measurements

The change in the translation of each tooth between pretreatment models (T1) and their digital setups (T2) was measured in mm. These were [28]:

  1. a

    Buccolingual (BL) translation.

  2. b

    Mesiodistal (MD) translation.

  3. c

    Occlusogingival (OG) translation.

The measured linear changes were recorded in Excel (Microsoft Excel: 2016 Microsoft Corporation) for comparisons with similar measurements taken from the three studied software packages.

Intra and inter-examiner reliability

Initially, one researcher (SA) performed all registrations of pretreatment scans with their Digital Setups, reference landmarks and axes identification, modification of local coordinates, as well as all tooth movement measurements. The same and another calibrated investigator (NV) repeated the measurements on 5 randomly selected scan sets 4 weeks later to test reliability. All measures were pooled to give a summary estimate to calculate Intraclass Correlation Coefficients for intra-examiner and inter-examiner reliability.

Statistical analysis of the data

Statistical analysis was carried out using IBM SPSS software package version 20.0. (Armonk, NY: IBM Corp). Data from individual teeth were pooled to provide an overall estimate of the amount of tooth movement in each degree of freedom and summarized as mean and standard deviation. Two-way fixed-rater single-measure Intraclass Correlation Coefficients (ICC) of absolute agreement were calculated between the pooled amount of tooth movement in each degree of freedom measured by each software package and the amount of tooth movement from the digital setup (reference standard). Overall agreement between the three software packages were similarly calculated. Based on the 95% confident interval of the ICC estimate, values less than 0.5, between 0.5 and 0.75, between 0.75 and 0.9, and greater than 0.90 are indicative of poor, moderate, good, and excellent reliability, respectively based on “A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research” by Koo and Li [29]. Statistical significance of the obtained results was expressed at p ≤ 0.01 to account for multiple comparisons.

Results

Intra-examiner reliability for Geomagic, OrthoAnalyzer and Compare software were 0.941, 0.899, and 0.978 respectively. As for inter-examiner reliability, they were 0.926, 0.798, and 0.944 respectively.

Table 1 shows the descriptive statistics (mean and standard deviation) for the maxillary and mandibular teeth with respect to the three linear movements for the DS and the three tested software (Geomagic, OrthoAnalyzer and Compare). Agreement between each package and the reference standard are presented as ICC in Table 2.

Table 1 Amount of linear tooth movement measured in mm as determined by each software package
Full size table
Table 2 Intraclass Correlation Coefficients for different movements among the three software packages, in comparison to the digital setup
Full size table

BL linear movement showed the highest agreements in all three software packages (Overall ICC G: 0.928, O: 0.676, C: 0.922), followed by comparable values for agreements for the MD (Overall ICC G: 0.914, O: 0.624, C: 0.912) and OG movements (Overall ICC G: 0.911, O: 0.615, C: 0.911).

For Geomagic software, the maxillary ICC for all linear measurements showed excellent agreements with the DS (> 0.90), while the mandibular ICCs for all measurements showed good agreements (0.75–0.90). In OrthoAnalyzer software, all maxillary and mandibular linear ICCs showed only moderate values for agreements (0.50–< 0.75). As for Compare software, all maxillary and mandibular linear measurements showed excellent agreements with the DS (> 0.90).

Discussion

Quantifying the amount of orthodontic tooth movement in digital orthodontics is performed by registration software packages, which are dependent on the registration algorithm used [10, 30]. Scholarly literature lacks studies testing accuracy of software packages to compare treatment effects and efficacy [5, 8, 10, 19,20,21, 31,32,33,34,35]. The present study calculated the amount of tooth movement in the three linear directions for the entire maxillary and mandibular dentitions including second permanent molars.

Since tooth movements on the Digital Setup were performed by the principal investigator, the true value for translation (type, direction and amount) for each tooth could be used as a reference [13, 20, 36]. The reliability and validity of using Digital Setup generated by OrthoAnalyzer software was previously studied and it was concluded that it is as effective and accurate as manual setups and represents an efficient tool for diagnosis and appliance fabrication, that can be reliably reproduced [37, 38]. The present study used reference landmarks and area on the rugae for registration of the maxillary digital models in two of the three software packages (Geomagic, OrthoAnalyzer) that required reference structures. The selected landmarks have been documented previously in several studies to be considered as stable landmarks for maxillary digital model registration [11, 22, 23, 39]. In the mandibular arch, landmarks on the MGJ were used both in Geomagic and OrthoAnalyzer softwares. This was based on the findings by Ioshida et al. [40], who concluded that MGJ can be used as a reference area of good stability for mandibular superimpositions. However, the Compare software did not require the identification of a reference point or area for either arch outside the dentition.

The Compare software showed higher mandibular Intraclass Correlation Coefficients (ICCs) for all movements in comparison to their maxillary equivalents. In addition, mandibular ICCs were always greater for Compare software than the other two software. The Compare software does not require anatomical landmark or surface selection before registration but rather depends on the automatic superimposition tool of the software after initial global alignment. Mandibular ICCs were consistently lower than their maxillary equivalents for all linear movements in the two software (Geomagic and Ortho Analyzer) that required either landmark or surface selection for the registration. While Compare removes the interproximal papillae and model base apical to the gingival margin to ensure that the registration is based solely on tooth-surface features. This suggests that the mandibular superimposition using the MGJ landmarks with an area around it as a reference is less accurate than the maxillary superimposition using the rugae area. Only two studies have attempted to study stable landmarks and areas to be used for accurate and reliable mandibular superimposition [24, 40]. On the other hand, numerous studies have endorsed the accuracy of maxillary reference points and areas to be used for maxillary digital superimposition [11,12,13, 21, 22, 26, 39, 41,42,43].

The current study employed registration techniques as mandated by the algorithms employed by the various software packages [16]. The registration technique for the Geomagic and Compare software used a best-fit method [20, 22, 31, 42]. This technique of ‘fine matching’ uses thousands of reference points instead of a few landmarks/area and is based on ‘Iterative Closest Point Algorithms’ (ICP) [44]. The effect of outliers is decreased while accuracy markedly improves. Although OrthoAnalyzer uses a surface-based registration technique, it doesn’t use an algorithm that iterates to improve the overall quality of registration, unlike the ICP employed in the Geomagic and Compare software. This could explain the higher ICC values for Geomagic and Compare software compared to OrthoAnalyzer software.

The lowest ICC values for all measured linear movements with OrthoAnalyzer software could be attributed to a crucial factor for digital superimpositions which is to generate an accurate and reproducible coordinate system. Geomagic software requires only one global coordinate system, Compare software generates automated computations for placement of local coordinate systems at each tooth’s approximate center of resistance [10, 19, 33], while OrthoAnalyzer software required creation of multiple customized global coordinates for each tooth. An assumption can be made that the coordinate system employed by OrthoAnalyzer will be more accurate because it is customized for each tooth according to its location in the dental arch. However, this assumption was not true as it induced more operator error each time a plane was constructed for measurements.

Linear measurements

In the present study, Geomagic and OrthoAnalyzer software packages used surface landmarks, since it was not possible to place a centroid on T1 and T2 and measure the differential positions using these software packages. While Compare software used automatic derived coordinates at the centroid. Current literature describes tooth movement in digital superimpositions based on the movement of different landmarks on a tooth. Some authors used cusp tips and incisal edges [12, 42], like what we used in the present study. Although in theory it is reliable to locate a landmark on a cusp tip, its displacement represents only the displacement of that landmark and not the displacement of the whole tooth, therefore, not differentiating between tipping and bodily movement [23, 31, 42]. Studies with landmarks averaged to a centroid were able to describe the translational movements of teeth but not report rotational changes [41, 45]. Additionally, Sandler used a shell to calculate molar movement keeping the buccal and palatal outlines similar between T1 and T2, to get rid of any gingival effects on the posttreatment model (e.g., gingival irritation, overgrowth, or hyperplasia), which would affect the determination of the center of mass (Co M) of the crown [32]. The difference in landmark location between the studied software packages could clarify the excellent agreement for all linear movements measured by Compare software in both arches in comparison to the other software packages. Though both of Geomagic and OrthoAnalyzer software packages used surface landmarks, the ICC values of Geomagic demonstrated far better agreements. This could be attributed to the fact that in Geomagic, a true 3D distance was measured between T1 and T2 landmarks compared to a constructed plane in OrthoAnalyzer. Additionally, the brush size that was used to locate landmarks in Geomagic  software is much smaller in size than the one used in OrthoAnalyzer software, rendering point location with greater precision.

Buccolingual movements showed slightly higher agreements in the three tested software packages in comparison to other linear measurements, with fairly close ICC values between the MD and OG movements. This was in agreement with Choi et al. [42], who reported that the three linear measurements did not differ significantly upon comparing plaster and digital models.

Based on the findings of the current study, Compare software package showed the greatest advantage in terms of accuracy compared to the other software packages.

Conclusions

  1. 1.

    Compare and Geomagic software packages consistently showed maximum accuracy in measuring the amount of tooth movement in the maxillary arch compared to the reference standard.

  2. 2.

    Compare software showed highest agreements in the mandibular arch.

  3. 3.

    None of the studied software packages showed poor agreement with the DS across all tooth movement measurements in both arches.

  4. 4.

    Buccolingual displacement was the movement recorded with highest accuracy and agreement between software packages.

Limitations of the present study

  1. 1.

    All measurements were based on the morphology of the clinical crown due to the absence of roots in intraoral scans, therefore the tooth centroid could not be defined. The linear measurements calculated cannot determine the type of tooth movement whether translation, or rotation or a combination of both.

  2. 2.

    Using the digital setup as a reference standard maximize the chance of agreement with the registration software packages since the adjacent soft tissues are not altered. Accounting for the tissue changes concomitant with orthodontic tooth movement, the accuracy of the registration software packages will likely be lower than the reported values.