Variational inference for responsible logistic regression

Abstract

Machine Learning is increasingly used in high-stakes applications such as health- care or finance, where interpretability, transparency and fairness are critical. These demands have led to growing interest in models that can meet such requirements. Logistic regression remains a popular choice in these contexts due to its versatility and interpretability. While sparsity has been explored extensively and fairness only recently, few works have considered both simultaneously. In this work, we present a novel approach to logistic regression based on Mean-Field Variational Inference (MFVI), which integrates both sparsity and fairness into a unified Bayesian frame- work. We do so by modifying the KL divergence term in the variational objective with a penalization that encourages fair solutions, while also incorporating a sparsity- inducing prior that allows for feature selection. We further adapt the inference process to include group sparsity, enabling the selection or elimination of grouped features such as dummy variables derived from categorical inputs. The numerical results show a triple trade-off between accuracy, fairness and interpretability which may significantly improve the user’s capability to properly balance the three criteria.