Abstract
Purpose
The present study compared manual and automated measurement of Cobb angle in idiopathic scoliosis based on deep learning keypoint detection technology.
Methods
A total of 181 anterior–posterior spinal X-rays were included in this study, including 165 cases of idiopathic scoliosis and 16 normal adult cases without scoliosis. We labeled all images and randomly chose 145 as the training set and 36 as the test set. Two state-of-the-art deep learning object detection models based on convolutional neural networks were used in sequence to segment each vertebra and locate the vertebral corners. Cobb angles measured from the output of the models were compared to manual measurements performed by orthopedic experts.
Results
The mean Cobb angle in test cases was 27.4° ± 19.2° (range 0.00–91.00°) with manual measurements and 26.4° ± 18.9° (range 0.00–88.00°) with automated measurements. The automated method needed 4.45 s on average to measure each radiograph. The intra-class correlation coefficient (ICC) for the reliability of the automated measurement of the Cobb angle was 0.994. The Pearson correlation coefficient and mean absolute error between automated positioning and expert annotation were 0.990 and 2.2° ± 2.0°, respectively. The analytical result for the Spearman rank-order correlation was 0.984 (p < 0.001).
Conclusion
The automated measurement results agreed with the experts’ annotation and had a high degree of reliability when the Cobb angle did not exceed 90° and could locate multiple curves in the same scoliosis case simultaneously in a short period of time. Our results need to be verified in more cases in the future.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Introduction
Different methods of measuring the Cobb angle include manual measurement and computer-assisted semi-automated measurements. The most widely used manual method of measurement requires extensive technical experience and is prone to intra- and inter-observer errors of 2.8–8° [1, 2]. Although semi-automated measurement methods can improve work efficiency, clinicians still need to select the end vertebrae and determine the positions of endplates. Therefore, it is difficult for inexperienced observers to measure Cobb angles accurately. Besides, inevitable manual operations increase the work burden [3].
Deep learning technology can be used to optimize diagnosis, prognosis, and outcome prediction in spinal diseases [4,5,6,7,8]. However, the use of deep learning in the field of scoliosis measurement is still immature. Previous studies have performed deep learning measurements based on mild scoliosis cases or chest X-ray images, or non-deep learning methods with redundant pixel-level segmentation of images, requiring excessive manual assistance [9,10,11]. Therefore, the measurement methods in these studies may not apply to severe scoliosis. Most studies have only measured the main scoliosis curve, but the upper and lower curves adjacent to the main curve are also crucial for making treatment decisions.
To address these problems and compare manual and automated measurement of Cobb angle in idiopathic scoliosis, we chose anterior–posterior (AP) spine X-ray images as the research object and proposed an automated measurement system based on deep learning object detection and keypoint detection technology. The system consists of three parts: isolated vertebra detection, vertebral corner localization, and Cobb angle calculation. The Cobb angle measurement based on the four corners of the vertebra can avoid the influence of irregular or unclear contours during measurement.
Materials and methods
Experimental materials
The inclusion criteria for images were as follows: patients with idiopathic scoliosis (IS) or normal adults without scoliosis, no other previous spinal disease or surgery, and a full-length AP spinal X-ray image should include the entire spine without interference blocking the vertebrae. 181 AP spinal X-ray images were included in this study, including 165 IS cases (Cobb angle range: 10–93°) and 16 normal adults without scoliosis (Cobb angle range 0–10°). We used the image annotation tool ‘Labelme’ to label the four corners of each vertebra from C7 to L5. We used 145 X-ray images (2610 vertebrae) for training and 36 X-ray images (648 vertebrae) for testing.
Proposed methods
Our automated measurement method consisted of three steps (Fig. 1). First, the C7-L5 vertebrae were detected and cropped from the X-ray images. The resolution of the X-ray images was zoomed out from 1750 × 3064 to 768 × 1152 pixels to locate each vertebra efficiently with global information. The vertebral regions were cropped from the original X-ray images to facilitate the predictions made by the deep learning detection model. Second, the four vertebral corners were located in the cropped regions. The regions were zoomed in to 512 × 512 pixels before being fed into the deep learning corner detection model to precisely locate the four corners. Third, the Cobb angles were measured automatically using the connecting line between the upper or lower corners, which, in clinical practice, is equivalent to the upper or lower endplate.
C7-L5 vertebrae localization
To locate the region of interest of the spine, the zoomed-out X-ray images were fed into the CenterNet-based [12] model named CenterNet1 to approximately locate the C7-L5 vertebral regions. CenterNet is a state-of-the-art deep learning object detection technology that uses keypoint estimation to find center points and regresses by measuring the size of the borders. After the vertebrae were located, their sequence was determined by the longitudinal height. The lowest detection result was L5 in each X-ray image, and the other vertebrae included L4-C7 from bottom to top. To ensure that four corners were included in the vertebral region, the widths and heights of the bounding boxes were enlarged 1.5-times. Subsequently, each vertebral region was cropped from the high-resolution X-ray images and zoomed in to 512 × 512 pixels. (Fig. 2).
Vertebral corner localization
Next, we used keypoint detection technology to locate the four corners in each vertebral region. The proposed vertebral corner detection model was another CenterNet-based model named CenterNet2 consisting of two parts: the backbone and the output heads (Fig. 3). The backbone used the pertaining DLA-34 [13]. The vertebral corner detection model included three output heads, which were used to estimate the center heatmap, corner locations, and corner offset. Corner locations were the relative locations of the corners compared to the center. The corner offset was the discretization error of the corners.
The input of the vertebral corner detection model was the cropped vertebrae images described above. During the deep learning system training, supervision consisted of center heatmap loss, corner location loss, and corner offset loss. The center heatmap loss used the focal loss, whereas the other losses used the L1-norm loss. The left-top, right-top, left-bottom, and right-bottom corners of each vertebra were determined according to the vertical and horizontal coordinates after the vertebral corner detection.
Cobb angle measurement
The Cobb angle is defined as the angle between the upper endplate of the upper end vertebra and the lower endplate of the lower end vertebra. This study used the line connecting the left-top corner and the right-top corner as the upper endplate, and the line connecting the left-bottom corner and the right-bottom corner as the lower endplate for each vertebra (Fig. 4).
To obtain the Cobb angle, each angle composed by different vertebrae is calculated recursively by changing the lower and upper end vertebra from top to bottom (Fig. 5a). After the Cobb angle of the main curve is detected, Cobb angles in its upper and lower curves will also be measured in a similar way. A spinal X-ray is given as an example to show the detection result (Fig. 5b).
Finally, descriptive statistics were calculated for all patient characteristics as means and standard deviations for each group (expert annotated and deep learning located). To measure the reliability of the Cobb angles calculated by the proposed deep learning method, the intra-class correlation coefficient (ICC) was utilized, whereas the Pearson correlation coefficient, the mean absolute error and Spearman rank-order correlation were used to explore the relationships between deep learning results and the experts’ annotation as appropriate. The level of significance was set at p < 0.05. The data were analyzed using Python (rpy2 3.3.6) and calling R language.
Results
The GPU-accelerated deep learning system measured 36 test set images simultaneously, and the average time required for each image was 4.45 s. The automated system was set up to mark the name of each vertebra and its four corners, then to measure the Cobb angle of the main curve and display any adjacent curve > 10° above and below it (Table 1). We selected four test images for demonstration, the most severe one had a Cobb angle of 88° for the main curve, and its upper and lower curves were 30 and 49°, respectively (Fig. 6).
The mean Cobb angles of the two methods and correlation coefficients are given in Table 2. The statistical analysis indicated that the automated measurements agree with the experts’ annotation.
The Spearman rank-order correlation of 0.984 (p < 0.001) indicated that the Cobb angle results obtained using the deep learning-located corners and those obtained using the expert-annotated corners highly correlated.
Besides, since deep learning models always give the identical output on the identical input, intra-rater variability of the models involved in this paper always remains zero.
Discussion
Cobb angle is the most common way to quantify the severity of scoliosis. Currently, manual measurement is widely used. However, inexperienced observers often lead to improper localization of end vertebrae and endplates, resulting in inevitable random errors. The larger the curve, the more variable the measurement accuracy [14, 15]. In addition, manual measurement is time-consuming [15]. To address these problems, some researchers have explored methods of automatically or semi-automatically measuring the Cobb angle [9,10,11, 16,17,18,19]. Compared to manual measurement and semi-automated measurement, automated measurement can reduce differences in the selection of end vertebrae and errors caused by manually drawing endplate lines, thereby achieving high-precision in the measurements.
To reduce artificial errors, many previous studies have proposed various semi-automated methods of measuring the Cobb angle, which could minimize manual participation in the measurement process [11, 16,17,18]. In 2002, Chockalingam et al. [16] developed a semi-automated computerized technique to measure the Cobb angle. Subjectivity arises when the observer determines the upper and lower end vertebra of the curve on the spinal image, and when the medial and lateral edges of the vertebra are defined. The mean analysis time for each radiograph was 2.3 min. Although research on semi-automated measurement has deepened, and measurement time and accuracy are improving, the process of manual participation cannot be avoided. For example, a system developed by Zhang et al. [11] could achieve an accuracy similar to that of experienced physicians, and the average computing time was less than half a minute, but it still required manual selection and labeling of the end vertebrae.
The use of deep learning technology in identifying and measuring spinal X-ray images is still in its early stages, and there were significant limitations in previous studies [9, 10, 19]. The automated systems developed by Pan et al. [9] measured Cobb angles in 248 chest X-rays with mild scoliosis (6.6–48.3°; average 14.87 ± 5.57°), whereas the ICC between the system and the physician’s manual measurement was 0.854 and the mean absolute difference was 3.32°. In other words, the sensitivity (89.59%) and specificity (70.37%) of this system for identifying scoliosis were unsatisfactory. Horng et al. [10] only tested their automated measurement system in a Cobb angle range of 0–20.1°. The ICC and Pearson correlation coefficient were within the range of 0.936–0.971, and the Spearman rank-order correlation between the expert manual measurements and the system measurements was 0.891. This system exhibited high accuracy, but it was not verified with a Cobb angle > 20.1°. Zhang et al. [19] also used the vertebral corner keypoint detection method to achieve automated measurement of the Cobb angle. However, they did not use statistical methods to compare the automated method’s consistency against the manual method. Furthermore, the range of Cobb angles that can be measured effectively was not mentioned.
To solve these problems, we proposed a new scoliosis measurement method based on deep learning. Our system can achieve automated measurement of Cobb angles through the vertebral positioning based on deep learning object detection technology and vertebral corner positioning based on deep learning keypoint detection technology. Our study included various Cobb angles in the training set (145 X-ray images), ranging from 0 to 93°. We used the trained system to automatically identify and measure Cobb angles on 36 AP spine X-ray images that were not included in the training set. There were no significant differences in the measurements of 36 scoliosis cases with different severity by both the system and orthopedic experts. The trained deep learning system could satisfactorily recognize curves up to almost 90°. Compared to previous studies, our system has obvious advantages in its scope of application and degree of automation. The vertebral positioning based on deep learning object detection technology and vertebral corner positioning based on deep learning keypoint detection technology allowed faster measurement of Cobb angles by our system, which is more robust and finer-grained (Fig. 7). On the one hand, our automated system can identify each vertebra and obtain Cobb angles rapidly (average 4.45 s per image), avoiding human assistance at each step. On the other hand, this system can accurately measure severe Cobb angles up to almost 90° and locate multiple curves in the same scoliosis case simultaneously.
Our study has some limitations. First, our experiments found that the measurements made by our system greatly deviated from manual measurements when the curve was greater than 90°. The reason may be that, in severe scoliosis, the vertebrae are tilted and rotated severely, and the vertebrae may overlap in X-ray images, especially in the thoracic spine, where the ribs interfere with the image. Our system cannot accurately segment the vertebrae and identify the four corners of the vertebrae in these severe cases. Second, our system has difficulty identifying the four corners of the L5 vertebra. Compared to other lumbar vertebrae, L5 is usually structurally different and more inclined in the sagittal plane. Even an experienced orthopedist may not be able to identify the four corners of L5 precisely. We measured the angles between the upper and lower endplates of L1-L5 and the horizontal line on sagittal X-ray images from 36 test sets (Fig. 8). The results show that L5 is the most inclined vertebra in the sagittal plane of the lumbar spine, which leads to projection of the upper and lower endplates of the L5 vertebra closer to a “plane” rather than a “line” on the coronal plane. The “planes” of the upper and lower endplates are closer to each other, or even overlap, which makes it difficult to identify the four corners of the L5 vertebra on the coronal plane. Hopefully, future studies can overcome these problems and achieve more accurate measurements.
Conclusion
We developed an automated measurement method for Cobb angle in idiopathic scoliosis based on deep learning keypoint detection and compared it to manual measurement. The automated measurement results agreed with the experts’ annotation and had a high degree of reliability when the Cobb angle did not exceed 90°. The technique could also locate multiple curves in the same scoliosis case simultaneously in a short period of time. However, the number of images in our test set was small, the results of this experimental study need to be verified in more cases in the future.
References
Kuklo TR, Potter BK, Polly DW, Jr., O’Brien MF, Schroeder TM, Lenke LG (2005) Reliability analysis for manual adolescent idiopathic scoliosis measurements. Spine (Phila Pa 1976) 30:444–454. https://doi.org/10.1097/01.brs.0000153702.99342.9c
Morrissy RT, Goldsmith GS, Hall EC, Kehl D, Cowie GH (1990) Measurement of the Cobb angle on radiographs of patients who have scoliosis. Evaluation of intrinsic error. J Bone Jt Surg Am 72:320–327
Safari A, Parsaei H, Zamani A, Pourabbas B (2019) A Semi-automatic algorithm for estimating cobb angle. J Biomed Phys Eng 9:317–326. https://doi.org/10.31661/jbpe.v9i3Jun.730
Azimi P, Yazdanian T, Benzel EC, Aghaei HN, Azhari S, Sadeghi S, Montazeri A (2020) A review on the use of artificial intelligence in spinal diseases. Asian Spine J. https://doi.org/10.31616/asj.2020.0147
Lessmann N, van Ginneken B, de Jong PA, Išgum I (2019) Iterative fully convolutional neural networks for automatic vertebra segmentation and identification. Med Image Anal 53:142–155. https://doi.org/10.1016/j.media.2019.02.005
Al Arif S, Knapp K, Slabaugh G (2018) Fully automatic cervical vertebrae segmentation framework for X-ray images. Comput Methods Progr Biomed 157:95–111. https://doi.org/10.1016/j.cmpb.2018.01.006
Azimi P, Mohammadi HR, Benzel EC, Shahzadi S, Azhari S, Montazeri A (2015) Artificial neural networks in neurosurgery. J Neurol Neurosurg Psychiatry 86:251–256. https://doi.org/10.1136/jnnp-2014-307807
Bien N, Rajpurkar P, Ball RL, Irvin J, Park A, Jones E, Bereket M, Patel BN, Yeom KW, Shpanskaya K, Halabi S, Zucker E, Fanton G, Amanatullah DF, Beaulieu CF, Riley GM, Stewart RJ, Blankenberg FG, Larson DB, Jones RH, Langlotz CP, Ng AY, Lungren MP (2018) Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet. PLoS Med 15:e1002699. https://doi.org/10.1371/journal.pmed.1002699
Pan Y, Chen Q, Chen T, Wang H, Zhu X, Fang Z, Lu Y (2019) Evaluation of a computer-aided method for measuring the Cobb angle on chest X-rays. Eur Spine J 28:3035–3043. https://doi.org/10.1007/s00586-019-06115-w
Horng MH, Kuok CP, Fu MJ, Lin CJ, Sun YN (2019) Cobb angle measurement of spine from X-ray images using convolutional neural network. Comput Math Methods Med 2019:6357171. https://doi.org/10.1155/2019/6357171
Zhang J, Lou E, Le LH, Hill DL, Raso JV, Wang Y (2009) Automatic Cobb measurement of scoliosis based on fuzzy Hough Transform with vertebral shape prior. J Digit Imaging 22:463–472. https://doi.org/10.1007/s10278-008-9127-y
Zhou X, Wang D, Krähenbühl P (2019) Objects as points. In. p.
Yu F, Wang D, Shelhamer E, Darrell T (2018) Deep layer aggregation. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, pp 2403–2412
Facanha-Filho FA, Winter RB, Lonstein JE, Koop S, Novacheck T, L’Heureux EA Jr, Noren CA (2001) Measurement accuracy in congenital scoliosis. J Bone Jt Surg Am 83:42–45. https://doi.org/10.2106/00004623-200101000-00006
Elfiky T, Patil N, Shawky M, Siam A, Ragab R, Allam Y (2020) Oxford Cobbometer versus computer assisted-software for measurement of Cobb angle in adolescent idiopathic scoliosis. Neurospine 17:304–311. https://doi.org/10.14245/ns.1938260.130
Chockalingam N, Dangerfield PH, Giakas G, Cochrane T, Dorgan JC (2002) Computer-assisted Cobb measurement of scoliosis. Eur Spine J 11:353–357. https://doi.org/10.1007/s00586-002-0386-x
Stokes IA, Aronsson DD (2006) Computer-assisted algorithms improve reliability of King classification and Cobb angle measurement of scoliosis. Spine (Phila Pa 1976) 31:665–670. https://doi.org/10.1097/01.brs.0000203708.49972.ab
Zhang J, Li H, Lv L, Shi X, Guo F, Zhang Y (2015) A Computer-aided method for improving the reliability of Lenke classification for scoliosis. J Healthc Eng 6:145–157. https://doi.org/10.1260/2040-2295.6.2.145
Zhang K, Xu N, Yang G, Wu J, Fu X (2019) An automated Cobb angle estimation method using convolutional neural network with area limitation. In. Springer International Publishing, Cham, pp 775–783
Funding
This research was funded by Beijing Chaoyang District Science and Technology Plan Project, grant number: CYSF2109.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All authors declare that they have no conflict of interest.
Ethical approval
The X-ray images were obtained from Beijing Chaoyang Hospital, and no other personal information was retained for any patient during this study. Our research was approved by the Ethics Committee of Beijing Chaoyang Hospital.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Sun, Y., Xing, Y., Zhao, Z. et al. Comparison of manual versus automated measurement of Cobb angle in idiopathic scoliosis based on a deep learning keypoint detection technology. Eur Spine J 31, 1969–1978 (2022). https://doi.org/10.1007/s00586-021-07025-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00586-021-07025-6