Study design
This diagnostic accuracy and agreement study followed a modification of the Guidelines for Reporting Reliability and Agreement Studies (GRRAS) where each software package was considered as a rater [25]. IRB approval was obtained at the Faculty of Dentistry, Alexandria University (IRB: 00010556-IORG: 0008839), and informed consents sought from the subjects whose scans were used as a study material. Access to the original scans was limited to the principal investigator. All potentially identifiable patient information were removed from the scans. Scans were stored in one computer terminal connected to a protected server. The minimal sample size was calculated based on previous studies that evaluated the reliability of newly developed software calculating 3D tooth movements [12, 26]. Based on the results, a sample size of 20 scans was deemed enough to conduct this agreement study [27], with minimum accepted reliability ρ0 = 0.6 and maximum expected reliability ρ1 = 0.9, k = 3, where k corresponds to the number of tested software packages. The statistical significance alpha was set at 0.01 to account for multiple comparisons and a statistical power, 1 − β = 0.9. The minimum calculated sample size was 18, increased to 20 to account for defective scans.
Sample collection
The sample consisted of full arch pretreatment maxillary and mandibular intraoral digital scans of actual adult patients undergoing CAT. All scans were randomly selected from the records of a single orthodontic office in Mumbai, India with more than 15 years of experience with CAT. A random number list of 20 was generated using Microsoft Excel from the total number of scans available in the office archive. The scanner used was a TRIOS 3-D intraoral scanner (3Shape, Copenhagen, Denmark). The scan data was then exported in STL format file extension and the files were imported into the three studied software packages and analyzed in the Department of Orthodontics, Alexandria University. The study group comprised scans of 20 patients with a Little’s irregularity index that ranged from 4 to 6 mm. All teeth in both arches were evaluated for 3D tooth movements except for third molars. The inclusion criteria for the scans were (1) Adult subjects treated with CAT who received treatment in both arches, (2) Scans had to be complete and of acceptable quality with a full complement of teeth except for the third permanent molars. Scans were excluded if (1) Treatment involved extraction of permanent teeth, (2) Teeth had surface anomalies or if (3) Scans had soft-tissue lesions covering the palate or the mucogingival junction (MGJ) of the mandibular arch, (4) Scans that had any previous history of orthodontic /orthopedic treatment.
All the scans that met the eligibility criteria were given an identification number. All digital scans were de-identified by an independent investigator, and imported into the three different tooth measuring software programs for the principal investigator to evaluate (Fig. 1).
Procedure
Digital Setup
Full arch maxillary and mandibular pretreatment scans (T1) were imported to OrthoAnalyzer software (3Shape Ortho System, Copenhagen, Denmark). Scan preparations were deemed necessary for all maxillary and mandibular pretreatment scans. Models were trimmed and oriented into three planes. Virtual Digital Setups were done by using virtual segmentation techniques. After defining the mesial and distal edges of each tooth, the software automatically creates a cut spline for each tooth that can be manually adjusted if needed. The original occlusal plane and vertical planes were determined and used as a reference. The long axis of each tooth was determined and teeth were moved virtually to their desired ideal position in the arch. All linear tooth movements were visualized and quantified in three directions (buccolingual BL, mesiodistal MD, occluso-gingival OG). Tooth movement measurements of this Digital Setup (DS) were tabulated for all teeth and used as reference for measuring accuracy of the three different software packages. The DS were exported as STL model files and termed (T2) (Fig. 2).
T1 and T2 models were imported as STL files to the tooth measuring software programs, for registration and 3D linear measurements. The three studied software packages were:
-
1.
Geomagic (G)-(Geomagic U.S., Research Triangle Park, NC) using landmark based method followed by regional global surface closest point registration algorithm (ICP) [17].
-
2.
OrthoAnalyzer (O)-(3Shape Ortho System, Copenhagen, Denmark) using surface 3-point method of registration [18].
-
3.
eModel 9.0 “Compare” (C)-(Geodigm Corporation, Chanhassen, MN) using automatic surface to surface closest point registration algorithm (ICP) [19].
After importing all STL files into the different software packages, the following steps were conducted before measurements were made: 1. Registration, 2. Coordinate system generation, 3. Measurement of tooth movement
-
1.
Registration of the initial model and the Digital Setup using the three software packages
Geomagic software (G): [17] Registration was done in two steps (Fig. 3).
-
1.
Landmark based registration: The program was instructed to carry out an initial superimposition based on 3 points on the medial ends of third and second rugae areas in the maxillary arch and 3 points on the mucogingival junction (MGJ) between first premolar and second premolar, second premolar and first molar, first molar and second molar.
-
2.
Global and fine regional surface registration: The program made fine-tuned automatic adjustments to the spatial position of two models based on all points and changes the coordinates of one object to match the other using a best-fit surface algorithm.
OrthoAnalyzer software (O): [18] (Fig. 4)
Registration was done using surface 3-point method which involved selecting three landmarks on each of the corresponding models followed by painting an area of known stability to be used for surface-based registration. In the maxillary arch, 3 points on the medial ends of third and second rugae areas, plus an area of the palate limited anteriorly by the medial 2/3 of the third rugae and laterally by two lines parallel to the mid-palatal suture were used as the landmark. In the mandibular arch, 3 points on the MGJ between first premolar and second premolar, second premolar and first molar, first molar and second molar, plus an area 1 mm above and below the selected points on the MGJ.
Compare software (C): [19] Registration was done in three steps (Fig. 5).
-
1-
Model trimming and segmentation of individual teeth: To ensure that the analysis was based solely on tooth-surface features, interproximal papillae and the model base apical to the gingival margin were removed from both the T2 and T1 models. T2 models were then segmented to isolate each tooth as a separate object for superimposition on unsegmented T1 models.
-
2-
Global initial alignment: The dental arches were initially aligned globally, by three-points based on the mesial-buccal cusps of the first molars and the mesial-incisal point of the right central incisor in each arch that were used as matching points for initial registration. Alignment was based on a predefined fit region, which consisted of the occlusal surfaces of the first molars and premolars, tips of canines, and incisal edges of lateral and central incisors. This region was chosen to represent the average occlusal plane for each arch. This initial registration was then refined by 30 iterations of a closest-point algorithm to achieve best fit of the occlusal surfaces.
-
3-
Best fit surface registration: The software then automatically superimposed individual teeth from the segmented T2 models on the corresponding teeth in the unsegmented T1 models using best-fit surface-based registration.
-
2.
Generation of 3D coordinate system and orthogonal planes for tooth movement measurements
After registration, a three-dimensional (3D) coordinate system along the 3 principal axes was generated for tooth movement measurements. According to the software used, either model (Geomagic, OrthoAnalyzer) or tooth (Compare) global reference frames were generated. Model global reference frames are defined as a coordinate system of three mutually perpendicular, intersecting axes (x = sagittal/ anteroposterior, y = vertical/occluso-gingival, and z = transverse/mediolateral). The “x-axis” is defined as the intersection of sagittal and occlusal planes, the “y-axis” as the intersection of the sagittal and coronal planes and the “z-axis” as the intersection of the coronal and occlusal planes [28]. Orthogonal planes defined by the 3 principal axes were also generated.
The 3D planes of space are the transverse plane (occlusal plane) (XZ) which is defined as the plane passing through cusps of bilateral maxillary first molars, second premolars and first premolars. Midsagittal plane (XY) which is defined as the plane perpendicular to the transverse plane and passing through the median palatine raphe, and finally, the frontal plane (coronal) (YZ) which is the plane perpendicular to both the transverse and midsagittal planes. The midsagittal plane was constructed on the mid-palatal suture of the maxilla and then subsequently transferred to the mandible.
For Geomagic, one global model reference frame with the three mutually perpendicular intersecting axes (X, Y, Z) and orthogonal planes was constructed to measure all tooth movements (Composite Model Coordinates). On the other hand, for OrthoAnalyzer, each tooth required the generation of its own spatial model reference frame with the corresponding axes, and planes generated individually to measure tooth movements (Repeated Model Coordinates). However, for Compare, a local tooth reference frame that the software automatically generates, defining the principal local coordinate tooth axes was generated (Automated Tooth Coordinates). The axes were x = mesiodistal (red), y = buccolingual (green), and z = occluso-gingival (blue). Once the axes were placed on the T1 model, the software automatically generated analogous axes for each corresponding tooth in the T2 model.
-
3.
3D tooth movement measurements
After all digital models (T1 and T2) were oriented in the same coordinate system via registration, it was possible to evaluate how the tooth positions changed.
Measurements made were:
Linear measurements
The change in the translation of each tooth between pretreatment models (T1) and their digital setups (T2) was measured in mm. These were [28]:
-
a
Buccolingual (BL) translation.
-
b
Mesiodistal (MD) translation.
-
c
Occlusogingival (OG) translation.
The measured linear changes were recorded in Excel (Microsoft Excel: 2016 Microsoft Corporation) for comparisons with similar measurements taken from the three studied software packages.
Intra and inter-examiner reliability
Initially, one researcher (SA) performed all registrations of pretreatment scans with their Digital Setups, reference landmarks and axes identification, modification of local coordinates, as well as all tooth movement measurements. The same and another calibrated investigator (NV) repeated the measurements on 5 randomly selected scan sets 4 weeks later to test reliability. All measures were pooled to give a summary estimate to calculate Intraclass Correlation Coefficients for intra-examiner and inter-examiner reliability.
Statistical analysis of the data
Statistical analysis was carried out using IBM SPSS software package version 20.0. (Armonk, NY: IBM Corp). Data from individual teeth were pooled to provide an overall estimate of the amount of tooth movement in each degree of freedom and summarized as mean and standard deviation. Two-way fixed-rater single-measure Intraclass Correlation Coefficients (ICC) of absolute agreement were calculated between the pooled amount of tooth movement in each degree of freedom measured by each software package and the amount of tooth movement from the digital setup (reference standard). Overall agreement between the three software packages were similarly calculated. Based on the 95% confident interval of the ICC estimate, values less than 0.5, between 0.5 and 0.75, between 0.75 and 0.9, and greater than 0.90 are indicative of poor, moderate, good, and excellent reliability, respectively based on “A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research” by Koo and Li [29]. Statistical significance of the obtained results was expressed at p ≤ 0.01 to account for multiple comparisons.