Comprehensive Diabetes Risk Prediction Using BRFSS Data: Performance, Explainability, Fairness, and Calibration

Abstract

This study aims to develop and evaluate machine learning models for diabetes risk prediction using a comprehensive approach that considers performance, interpretability, fairness, and calibration aspects. The research employs several classification algorithms, including Logistic Regression, Random Forest, XGBoost, and CatBoost, using the BRFSS dataset. The models are evaluated using multiple metrics, including Accuracy, Balanced Accuracy, Precision, Recall, F1-Score, ROC-AUC, Precision-Recall AUC (PR-AUC), Matthews Correlation Coefficient (MCC), and Brier Score. Explainability analysis is conducted using SHAP to understand feature contributions, while fairness and calibration analyses are performed to assess model reliability and bias across demographic groups. The results show that CatBoost achieves the best overall performance, with the highest ROC-AUC and Recall, as well as the lowest Brier Score, indicating better predictive capability and calibration. Explainability analysis reveals that GenHlth, BMI, and Age are the most influential features, while fairness analysis indicates potential disparities across certain age groups. Furthermore, ablation and misclassification analyses highlight key features and areas for model improvement. Overall, this study demonstrates that integrating performance evaluation with explainability and fairness analysis can produce more reliable and interpretable predictive models for healthcare applications.