Producción CyT

Class imbalance on medical image classification: towards better evaluation practices for discrimination and calibration performance

Artículo

Autoría:

Mosquera, Candelaria ; Ferrer, Luciana ; Milone, Diego H. ; Luna, Daniel ; FERRANTE, ENZO

Fecha:

2024

Editorial y Lugar de Edición:

Springer

Revista:

European Radiology Springer

Resumen *

PurposeThis work aims to assess standard evaluation practices used by the research community for evaluating medical imaging classifiers, with a specific focus on the implications of class imbalance. The analysis is performed on chest X-rays as a case study and encompasses a comprehensive model performance definition, considering both discriminative capabilities and model calibration.Materials and methodsWe conduct a concise literature review to examine prevailing scientific practices used when evaluating X-ray classifiers. Then, we perform a systematic experiment on two major chest X-ray datasets to showcase a didactic example of the behavior of several performance metrics under different class ratios and highlight how widely adopted metrics can conceal performance in the minority class.ResultsOur literature study confirms that: (1) even when dealing with highly imbalanced datasets, the community tends to use metrics that are dominated by the majority class; and (2) it is still uncommon to include calibration studies for chest X-ray classifiers, albeit its importance in the context of healthcare. Moreover, our systematic experiments confirm that current evaluation practices may not reflect model performance in real clinical scenarios and suggest complementary metrics to better reflect the performance of the system in such scenarios.ConclusionOur analysis underscores the need for enhanced evaluation practices, particularly in the context of class-imbalanced chest X-ray classifiers. We recommend the inclusion of complementary metrics such as the area under the precision-recall curve (AUC-PR), adjusted AUC-PR, and balanced Brier score, to offer a more accurate depiction of system performance in real clinical scenarios, considering metrics that reflect both, discrimination and calibration performance. Información suministrada por el agente en SIGEVA

Palabras Clave

computer assisted diagnosisdeep learningclass imbalance