Deep Learning–Assisted Identification of Femoroacetabular Impingement (FAI) on Routine Pelvic Radiographs

Abstract

To use a novel deep learning system to localize the hip joints and detect findings of cam-type femoroacetabular impingement (FAI). A retrospective search of hip/pelvis radiographs obtained in patients to evaluate for FAI yielded 3050 total studies. Each hip was classified separately by the original interpreting radiologist in the following manner: 724 hips had severe cam-type FAI morphology, 962 moderate cam-type FAI morphology, 846 mild cam-type FAI morphology, and 518 hips were normal. The anteroposterior (AP) view from each study was anonymized and extracted. After localization of the hip joints by a novel convolutional neural network (CNN) based on the focal loss principle, a second CNN classified the images of the hip as cam positive, or no FAI. Accuracy was 74% for diagnosing normal vs. abnormal cam-type FAI morphology, with aggregate sensitivity and specificity of 0.821 and 0.669, respectively, at the chosen operating point. The aggregate AUC was 0.736. A deep learning system can be applied to detect FAI-related changes on single view pelvic radiographs. Deep learning is useful for quickly identifying and categorizing pathology on imaging, which may aid the interpreting radiologist.

Appendix

The localization network implemented a 40-layer custom-built neural network, trained from random initializations. The design of this network was based on principles borrowed from three well-established architectures, namely, ResNet, a product of Microsoft, Inception by Google, and RetinaNet from Facebook. Each of these architectures contributed unique elements to the customization of the neural network, aimed at optimal functionality and efficiency.

The localization network was trained using inputs consisting of full-resolution original anterior–posterior (AP) view radiographs of the hip. These inputs were fed to an algorithm that was responsible for generating what are known as anchor boxes. Anchor boxes, a concept adopted from the RetinaNet architecture, are essentially potential bounding boxes of various sizes and aspect ratios that could potentially encapsulate a target object in an image—in this case, the hip joint.

Three distinct sizes for these anchor boxes were selected, ranging from 122 to 128 pixels. In terms of aspect ratios, two different variants were chosen, varying from 0.80 to 1.10. Each anchor box was strategically offset by 8 pixels to ensure a comprehensive search space over the entire unscaled input image.

Anchors were then classified into positive and negative classes, with positive anchor boxes signifying the presence of a hip joint. The classification was based on the degree of overlap between the anchor box and the ground truth bounding box (the region that truly encapsulates the hip joint in the image) using a metric known as intersection over union (IoU). Anchor boxes with an IoU greater than 0.6 were classified as positive, while those with an IoU less than 0.25 were deemed negative. Anchor boxes with IoU values in the range of 0.25 to 0.6 were excluded from the training set to maintain a clean separation between the positive and negative classes.

Intersection over Union (IoU) is a metric used to measure the accuracy of object detection models. It calculates the amount of overlap between two bounding boxes: a predicted bounding box and a ground truth bounding box. The greater the region of overlap, the greater the IoU.

To calculate the IoU between the predicted and the ground truth bounding boxes, you first take the intersecting area between the two corresponding bounding boxes for the same object. You subsequently divide the union of the boxes, which is the total area covered by both the boxes. IoU is a number from 0 to 1 that specifies the amount of overlap between the predicted and ground truth bounding box.

Over the course of the study, over 30 million anchor boxes were generated in the bounding box dataset, of which just over 30 thousand were identified as positive for the presence of a hip joint. Considering the large class imbalance, the study incorporated certain strategies during the training of the localization network. Positive boxes were oversampled to make up 1% of the network inputs while training, addressing the imbalanced class distribution to an extent. Further, a modified focal loss function, as outlined in the RetinaNet paper, was utilized. This function forced higher training gradients on misclassified positive cases, ensuring that the model pays more attention to these instances, thereby enhancing the performance of the localization network. The localization network head which ultimately classified the boxes was a standard Inception-ResNet hybrid network trained from random initializations.

Inputs to the classifier network were the five bounding boxes, determined by the preceding localization network, that exhibited the highest object scores, suggesting these areas had the highest likelihood of containing the hip joint.

The structure of this classifier convolutional neural network (CNN) was designed to mirror that of the previously described localization network. However, the classifier network was initialized with the trained weights of the localization network, which essentially means it started off with pre-existing knowledge from the localization process.

To ensure robustness and prevent overfitting, runtime regularization techniques were applied to both the localization and classifier networks. One such technique is L2 regularization, which penalizes large weights in the network model by adding a term proportional to the square of the magnitude of weights to the loss function. This discourages overly complex models and aids in generalization. Another technique used was dropout, a method that temporarily deactivates a random subset of neurons (units) during training to prevent them from adapting too much to the training data, further promoting model generalization.

The classification network relied on a weighted cross-entropy loss function for two to four classes, used to compute the error between the network’s predictions and the true labels. This function played a crucial role in updating the model weights during backpropagation and consequent iterations.

The raw outputs of the classification network were two numerical values, interpreted as un-normalized logarithmic odds (log-odds) for each class. To make these log-odds interpretable, they were normalized using the softmax function, a common method in machine learning that transforms log-odds into probabilities. The final predictions for each patient were then determined by aggregating the softmax scores after adjusting them for the object score.

The classifier network’s hyperparameters were meticulously fine-tuned based on its performance on a validation set, comprising 20% of the total sample. To achieve an unbiased estimate of the model’s performance, the dataset was divided into training (70%), validation (20%), and testing (10%) subsets. The testing set was sequestered, or kept separate, during the training and validation process, ensuring an unbiased assessment of the model’s final performance.

Once the optimal hyperparameters were identified using the validation set, the network was subjected to a single run on the sequestered test set, and its performance was recorded. The primary performance metric chosen for this network was the aggregated area under the receiver-operating curve (AUC). AUC measures the quality of a classifier and indicates how well the model can distinguish between the classes across various threshold settings. An AUC closer to 1 is desirable as it signifies a higher true-positive rate and a lower false-positive rate.