Cross-validation of a machine learning algorithm that determines anterior cruciate ligament rehabilitation status and evaluation of its ability to predict future injury

Abstract

Classification algorithms determine the similarity of an observation to defined classes, e.g., injured or healthy athletes, and can highlight treatment targets or assess progress of a treatment. The primary aim was to cross-validate a previously developed classification algorithm using a different sample, while a secondary aim was to examine its ability to predict future ACL injuries. The examined outcome measure was ‘healthy-limb’ class membership probability, which was compared between a cohort of athletes without previous or future (No Injury) previous (PACL) and future ACL injury (FACL). The No Injury group had significantly higher probabilities than the PACL (p < 0.001; medium effect) and FACL group (p ≤ 0.045; small effect). The ability to predict group membership was poor for the PACL (area under curve [AUC]; 0.61<AUC<0.62) and FACL group (0.57<AUC<0.59). The ACL injury incidence proportion was highest in athletes with probabilities below 0.20 (9.4%; +2.7% to baseline), while athletes with probabilities above 0.80 had an incidence proportion of 4.1% (−2.6%). While findings that a low probability might represent an increase in injury risk on a group level, it is not sensitive enough for injury screening to predict a future injury on the individual level.