Evaluating Machine Learning Tools to Predict VTE after Ankle Fracture

Venous thromboembolism (VTE) is a potentially fatal complication associated with orthopaedic surgery, often requiring the use of prophylaxis. A study presented at the AAOS 2023 Annual Meeting utilized a machine-learning model to identify potential risk factors associated with VTE following ankle fracture surgery in the hopes of developing a patient-specific predictive model to aid in decision making regarding the usefulness of VTE prophylactic measures. Lance Silverman, MD, FAAOS, AAOS Now Editorial Board member, spoke with the study’s presenting author, Nour Nassour, MD, about the results of the study and potential applications in practice.

“This project was initiated due to concerns about VTE within the ankle fracture population. We aimed to create a tool to standardize or develop a method to predict the risk of VTE,” said Dr. Nassour, who is a research fellow at Massachusetts General Hospital. “We incorporated artificial intelligence (AI) into this decision-support tool to create something clinically useful.”

The researchers utilized retrospective data from 16,421 patients with ankle fractures and identified 239 with confirmed VTE within 180 days of sustaining a fracture. Data from the patients with confirmed VTE were combined with 937 patients without VTE (controls) to reach a 1:4 case-to-control ratio for statistical analysis. The total population (n = 1,176) was further subdivided according to administration of chemoprophylaxis.

The predictive algorithm dataset included more than 110 factors and variables with a prevalence of at least 1 percent in the total population, including patient demographics, medical history, surgical history, fracture characteristics, treatment, medications, and laboratory values. Three algorithms were used for predictive analysis: backward logistic regression, decision-tree classifier, and machine learning. The researchers also used conventional statistics (chi-squared and t-test) to compare case and control groups. Odds ratios (ORs) were calculated for significant parameters.

Dr. Nassour noted that the association between risk factors and VTE was “very patient specific” and that each predictive algorithm selected between nine to 14 variables associated with VTE; these variables were utilized to develop a “limited variables” predictive model. The predictive value of the limited variables model was compared with an “all variables” model, which included all 110 identified factors.

Factors associated with VTE
The statistical analysis identified several factors positively associated with incidence of VTE: motor vehicle accident (OR = 4.48, P = 0.001), surgical treatment of fracture (OR = 1.97, P = 0.01), and longer hospital stay (9.30 days in VTE cohort versus 3.8 days in controls; P = 0.001). The use of warfarin as prophylaxis was associated with greater incidence of VTE (OR = 2.42, P = 0.01). Patients taking warfarin who experienced VTE were those with a “high-risk” profile for thromboembolic events, such as those with past VTE or cancer, the authors noted.

Statin use was negatively correlated with incidence of VTE (OR = 0.52, P = 0.01). After age-matching, there were no correlations found between VTE and the following factors: fracture type (open or closed), smoking status, gender, polytrauma, hyperlipidemia, osteoporosis, cardiovascular disease, and hypertension.

Predictive model efficacy
The predictive models were utilized in both the chemoprophylaxis and no-prophylaxis subgroups. Using the limited-variables dataset, the machine-learning model (random forest classifier) demonstrated the best predictive ability for VTE incidence in patients who did not receive prophylaxis, with an accuracy of 83.1 percent and an area under the random operator curve (AUROC) of 0.859. For the patients who did receive prophylaxis, the machine-learning model (gradient boosting classifier) predicted symptomatic VTE with an accuracy of 81 percent and an AUROC of 0.789.

The all-variables model demonstrated an AUROC ranging from 0.71 to 0.88 across both populations. The authors noted that the limited-variables models were slightly inferior to the all-variables models, adding, “In order to develop a [tool that is] adequate, clinically useful, and specific to each patient, the best route to go is use a model that will integrate all the variables to produce a specific and personalized prediction score to each patient seen in clinic.”

According to Dr. Nassour, “One of the things we would hope to do in this paper is showcase how important a patient-specific AI decision tool is and how important they would be in clinical assessment. There are some general factors [associated with VTE] that we all know, but we want also to think about these other patient-specific factors that might be a factor for one patient and not the other.”

Looking forward
Based on the results of this study, Dr. Nassour stated, “Truly, there is a lot of potential in AI decision support to be a decision aid for clinicians” regarding ankle fracture treatment. To expand on these findings, the researchers hope to increase the amount of data used, “to improve on the generalizability of our models,” Dr. Nassour said. “Future work will incorporate bigger data, like [a bigger] population and a more granular data set in order to [improve] the validity of our models.”

The ultimate goal would be the development of a “all-variables” type of model that could be integrated into the electronic health record system to aid clinicians in making decisions regarding prevention of VTE, such as whether or not to utilize chemoprophylaxis.

“That would be the dream,” Dr. Nassour said. “For now, we really want to see how we can better these models in order to make them clinically useful.”

Dr. Nassour’s coauthors of “Can Venous Thromboembolism Be Predicted after Ankle Fractures? A Machine Learning Analysis” are Bardiya Akhbari, PhD; Noopur Ranganathan; David Shin, MD; Christopher W. DiGiovanni, MD, FAAOS; Joseph H. Schwab, MD, FAAOS; Hamid Ghaednia, PhD; Soheil Ashkani Esfahani, MD; and Daniel Guss, MD, MBA, FAAOS.

Rebecca Araujo is the managing editor of AAOS Now. She can be reached at raraujo@aaos.org.