Beyond Accuracy: What Clinical Machine Learning Actually Requires

In many machine learning communities, performance metrics dominate evaluation. In healthcare, that mindset is incomplete. A model with strong AUC or F1-score can still be unsafe, unusable, or irrelevant in practice. Having transitioned from 12 years of pharmacy practice into public health and precision medicine data science, I’ve observed several recurring pitfalls in clinical ML projects. Here are five that matter most: Temporal Leakage Using data that would not be available at prediction time. Example: Including discharge notes to predict readmission. Solution: Strict event indexing and temporal slicing. Healthcare data is sequential. Respect the timeline. Ignoring Calibration Discrimination measures ranking ability. Calibration measures probability accuracy. In clinical decision-making, poorly calibrated risk estimates can distort thresholds and lead to overtreatment or undertreatment. Calibration curves and recalibration methods are not optional. Treating Missing Data as Random Mis