Automated Machine Learning to Predict Anatomical Outcomes in Pneumatic Retinopexy

Author Type(s)

Faculty

Document Type

Abstract

Publication Date

6-2023

Journal Title

Investigative Ophthalmology & Visual Science

Department

Ophthalmology

Abstract

Purpose : Pneumatic retinopexy (PR) is a minimally invasive procedure for retinal detachment (RD) repair with a variable anatomic success rate. However, no definitive measures exist to reliably predict the anatomical success of the procedure. Machine learning (ML) models have been previously developed by computer scientists to predict outcomes of various medical procedures. We sought to evaluate the feasibility of implementing fully automated ML algorithms developed by medical professionals without coding experience and evaluate the models’ discriminative performance for predicting success in PR.

Methods : This is a retrospective multicenter study of 539 consecutive patients with primary RD (between 2002 and 2022). Medical professionals without coding background used two autoML platforms, MATLAB Classification Learner App and Google Cloud AutoML Vertex AI, to develop models. We used single-procedure anatomic success as the outcome of interest and included patients’ baseline clinical characteristics in the models. Additional ML models were developed by computer scientists in Python (v3.8) to evaluate the accuracy and area under ROC curves (AUC). The ML experts were free to adopt any data pre-processing steps (e.g. augmentation or imputation) and ML algorithms.

Results : The dataset was divided into a training set (n=483) and a test set (n=56). We used this training set (Dataset 1) to train the MATLAB model. Given Google Cloud AutoML minimum data threshold (n>1000), the training set was tripled to make Dataset prime1 (n=1449) with a similar 2:1 success-to-failure ratio. Additional imputed and augmented datasets were generated in Python: Dataset 2 (n=660) and Dataset 3 (n=1313) with 1:1 success:failure ratio. The autoML models showed test accuracy of 53.6%, AUC=0.87 (MATLAB) and 57.4%, AUC=0.61 (Google autoML) on the imbalanced datasets (1 and prime1, respectively). Use of pre-processed datasets improved accuracy of autoML models to 80.4% (AUC=0.87, MATLAB) and 60.7% (AUC=0.69, Google AutoML), which were comparable to the models developed by the ML experts (accuracy 86%, AUC=0.86).

Conclusions : AutoML platforms have great potential in predicting procedure outcomes and can be used by clinicians without prior coding knowledge. However, limitations exist especially if datasets contain missing variables or are highly imbalanced. Proper data pre-processing, including augmentation techniques, can improve usability of autoML tools.

Link to Full Text

Share

COinS