Canada

Artificial Intelligence (AI) systems for skin-disease classification can perform unevenly across skin tones, risking wider disparities in dermatological care. Using Fitzpatrick17k, we trained a Swin Transformer classifier and observed a 26.4-percentage-point accuracy gap across Fitzpatrick skin-type groups. This gap was non-monotonic: accuracy was lowest for the lightest group (41.9%) and highest for a mid-tone group (68.3%), not following a simple lighter-versus-darker pattern. We then tested whether dermatology-specific transfer learning improves accuracy, fairness, and cross-dataset generalization. We standardized five public datasets (Fitzpatrick17k, DermaCon-IN, Derm1M, PASSION, DDI) to a unified eight-class taxonomy and evaluated staged transfer learning, large-scale pretraining on Derm1M, and concept-bottleneck models. Cross-dataset fine-tuning slightly improved accuracy but widened the skin-tone gap from 26.4% to 29.9%. Derm1M pretraining raised Top-5 accuracy and macro AUC by 1.6 and 7.0 percentage points but did not improve Top-1 accuracy or reduce the skin-tone gap, and external validation on PASSION and DDI showed marked performance declines, indicating limited robustness under dataset shift. Concept-bottleneck models matched baseline accuracy while adding interpretability. Persistent inflammatory-versus-infectious confusion and frequent high-confidence errors remained. These results show that taxonomy standardization and domain-specific pretraining, though necessary for combining fragmented datasets, are not sufficient to close fairness gaps; explicit fairness- and imbalance-aware methods with multi-dataset training are the needed next steps.

Equitable Health Technologies: Overcoming Biases and False Inferences in AI Diagnostics and Drug Discovery

Skin-Tone Fairness in Dermatology AI: Limits of Pretraining and Harmonization

dermatology ai

dataset harmonization

skin-tone fairness

concept bottleneck models

transfer learning

As AI transforms healthcare, ensuring reliability is critical. However, systems learning from existing data often reflect gaps or inconsistencies, leading to false inferences, diagnostic errors, and unequal performance across patient groups.

Open Data is pivotal for transparency and accountability. Biases emerging from unrepresentative datasets or flawed assumptions can perpetuate health disparities.

Leveraging public data on genomics, clinical trials, and social determinants allows researchers to audit and mitigate these risks effectively.

We task undergraduate and graduate students to harness data science to:

* Detect, reduce, and prevent bias in diagnostics and drug discovery;

* Develop data-driven tools that increase fairness and inclusivity;

* Innovate bias-aware model evaluation techniques to build trustworthy AI systems.

As AI transforms healthcare, ensuring reliability is critical. However, systems learning from existing data often reflect gaps or inconsistencies, leading to false inferences, diagnostic errors, and unequal performance across patient groups.

technical paper

Protein–protein interactions (PPIs) are a cornerstone of disease mechanisms, drug synthesis, and therapeutic discovery. Protein-protein binding affinity is a quantitative descriptor of PPIs which poses an important computational challenge. Structure prediction systems such as AlphaFold2, AlphaFold3, and ColabFold generate ranked candidate structures, while structure-based predictors such as ProAffinity-GNN estimate the binding affinity directly from a complex structure. In the common screening setting, where measured affinities are not yet available, we ask how stable the downstream estimates are across ranked predictions and prediction methods. For each complex, we score the top five structures per method and compare estimates using rank-wise, method-wise, and agreement statistics. We assess the pipeline's internal consistency and sensitivity. Estimates are highly stable to rank-choice but markedly more sensitive to the upstream prediction method. This inter-method effect is driven almost entirely by AlphaFold2, which produces systematically lower estimates, while AlphaFold3 and ColabFold agree closely with no detectable difference. The AF2 inputs are assembled from independently folded chains rather than jointly folded multimers, which we argue is the most likely driver. Estimates from different prediction methods should not be mixed, while scoring one representative rank may reduce downstream inference.

Rank and Method Variance in ProAffinity-GNN Binding Affinity Estimates from AlphaFold Variants

Antimicrobial peptides (AMPs) are short polypeptides that naturally exist in multicellular organisms as part of the immune defence system against microbial infections. Due to their broad-spectrum inhibitory effects on microbes, including bacteria, they pose a challenge for microbes to develop resistance to, making them an alternative to antimicrobial molecules used today and thereby mitigating antimicrobial resistance. Recent studies have investigated various Artificial Intelligence (AI) models to predict and generate AMPs for use against microbial infections. Among these approaches, shallow and deep learning models are proposed. However, one of the biggest issues with using generative AI methods is the increase in false positives, often stemming from systematic bias in data sampling during machine learning model training, which eventually fails when tested experimentally. This highlights the necessity of bias-aware AMP identification. Consequently, scientists should use more accurate performance metrics to evaluate models. Several approaches have been proposed to reduce false-positive AMP predictions, including benchmarking and reproducibility standards that promote fair evaluation and comparison across models. Furthermore, advanced models such as DeepAMPNet allow the integration of sequence, structural, and evolutionary information, enabling a more comprehensive analysis of AMPs. To improve the reliability of future models, representative datasets, more robust external validation, and experimental feedback should be incorporated. Overall, AMPs are an integral part of the antibiotic discovery pipeline. AI-driven approaches have significant potential to revolutionize AMP discovery; however, their effectiveness depends on proper optimization and validation.

Bias-Aware AMP Discovery: Mitigating False Positives in Generative Antimicrobial Models

Artificial intelligence (AI) screening tools are increasingly integrated into clinical decision-making. However, these tools are trained on structured clinical data that omit subgroup level disparities, risking inequitable performance across heterogeneous patient subpopulations. Polycystic ovarian syndrome (PCOS), characterized by menstrual irregularity and overlapping metabolic, endocrine, and inflammatory pathways, affects approximately 8-13% of reproductive-aged women. This study examined whether incorporating metabolic, inflammatory, and endocrine markers improves screening accuracy and subgroup equity in AI-based PCOS screening tools. Using the National Health and Nutrition Examination Survey (NHANES) 2017-2018 cycle, ten datasets spanning reproductive health, demographic, and metabolic/hormonal variables were merged and limited to reproductive-aged women (18-45 years old). Rotterdam criteria were approximated using menstrual irregularity, elevated testosterone, and elevated AMH/LH as a substitute for the lacking ultrasound-confirmed polycystic ovary morphology in NHANES. Two binary logistic regression models were developed: Model 1 used hormonal and menstrual features only, while Model 2 added metabolic and inflammatory features. Model 2 performed substantially better than Model 1, with lower AIC (984.59 to 279.73) and residual deviance (966.58 to 233.73). Across both models, menstrual regularity, testosterone, and luteinizing hormone significantly predicted PCOS risk (p < 0.001). Gene expression data (GEO accession GSE114419) comparing ovarian granulosa cell transcriptomes between PCOS patients and controls (n = 3) were analyzed to find 231 transcripts with fold-change greater than 2, providing biological context for hormonal markers used in the Rotterdam criteria. The method was translated into a publicly accessible online risk calculator to assess PCOS risk.

Uncovering Metabolic/Inflammatory Bias in AI Screening Models for Polycystic Ovarian Syndrome

Diabetes affects an ever-growing number of patients, and disparities in how they are diagnosed, treated and managed contribute to unequal outcomes regarding their health. In this study, demographic and socio-economic factors are examined to assess their effects on glycemic control, diabetes diagnosis, and access to care among individuals of similar A1c levels. These analyses were achieved by using data from the National Health and Nutrition Examination Survey (NHANES). First, among participants diagnosed with diabetes, ordinary least squares regression revealed that insurance status and education were positively correlated with glycemic control. Second, among the full sample, a logistic regression indicated that Mexican Americans and Non-Hispanic Black individuals had a markedly higher likelihood of being undiagnosed diabetics. Finally, 93.5% of participants displaying poor glycemic control reported having a usual place of care; however, the few lacking access were not evenly distributed. This study highlights the importance of addressing social factors that influence diabetes prevention and management.

Clinical Needs Fail to Equalise Access in Diabetes Care

Cardiovascular disease (CVD) is the leading cause of death worldwide. CVD risk can be predicted by traditional risk factor scoring systems. However, these models are limited by unequal performance across demographic groups. While alternative AI-based risk models are increasingly used for CVD risk prediction, it remains to be seen if they amplify these disparities. This study trained a XGBoost machine learning classifier on the National Health and Nutrition Examination Survey (NHANES) 2017–2018, a nationally representative sample of 9,254 American adults, to classify prevalent CVD. We evaluated model performance across sex, race/ethnicity, income, and age subgroups and benchmarked results against the Framingham Risk Score, a tool used for cardiovascular risk prediction. The model achieved strong overall discrimination with a ROC-AUC of 0.911 and a PR-AUC of 0.405, outperforming the Framingham Risk Score on all primary metrics. However, subgroup analysis revealed meaningful performance disparities. Female participants had a false negative rate of 0.629 compared to 0.421 in male participants, meaning nearly two thirds of women with CVD were missed. Low-income participants showed the lowest sensitivity at 0.386. Among Non-Hispanic Black participants, XGBoost offered no improvement over the Framingham Risk Score. These findings demonstrate that strong average performance does not guarantee fair performance across demographic groups. AI models trained on data reflecting historical inequities in healthcare access and diagnosis may risk encoding and perpetuating those same inequities. Systematic subgroup evaluation should be a minimum standard for any CVD classification model considered for clinical deployment.

Performance disparities across demographic subgroups in XGBoost-based classification of prevalent cardiovascular disease

Cancer survival prediction models are increasingly able to predict cancer survival using large sets of clinical and molecular data that often rely on the patient’s chronological age. However, chronological age alone may neglect important variables between patient subgroups, which could be captured through epigenetic age, a DNA methylation-based estimate of biological aging. This study investigated whether epigenetic age improves cancer survival prediction and subgroup equity for 2,470 patients across four cancer types: breast invasive carcinoma (BRCA), colon adenocarcinoma (COAD), lung adenocarcinoma (LUAD), and liver hepatocellular carcinoma (LIHC). TCGA methylation and clinical datasets were used to compute epigenetic age with the Horvath Clock, a DNA-methylation-based age estimator. Epigenetic age acceleration variables were then generated and used to compare three random forest models: Model A, which used chronological age and standard variables; Model B, which used epigenetic age and epigenetic age acceleration (EAA) variables; and Model C, which combined chronological and epigenetic age features. Performance and subgroup fairness of the models were evaluated by AUC, sensitivity, specificity, net reclassification improvement, and statistical tests. Breast cancer and lung cancer datasets showed significant racial differences in biological aging patterns. In lung cancer, Black patients showed a 13.39-year larger tumor-normal biological aging gap than White patients. Black breast cancer patients benefited more from the combined model than White patients, with higher net reclassification improvement. The above results imply that using epigenetic age in survival prediction models for cancers can improve fairness-related performance without compromising predictive performance.

Two ages, One patient: epigenetic age and equity in cancer survival prediction

Medical image anonymization is widely used to protect patient privacy, yet its impact on the fairness of artificial intelligence (AI) systems remains poorly understood. This study evaluated whether common anonymization techniques introduce demographic disparities in chest X-ray classification. A DenseNet-121 classifier was fine-tuned on a 1,622-image subset of the CheXpert Plus dataset to detect five thoracic findings: atelectasis, cardiomegaly, consolidation, edema, and pleural effusion. Three training regimes were evaluated using original images, Gaussian-blurred images (7×7 kernel), and border-cropped images (15% border removal). Fairness was assessed using the Equal Accuracy Gap and Equal Opportunity Gap across sex, age group, race, and ethnicity. Both anonymization methods increased fairness disparities relative to the baseline model. Cropping produced the greatest deterioration in equal-opportunity fairness, increasing the mean Equal Opportunity Gap from 0.315 to 0.504, with the largest effects observed across age and ethnicity groups. Blurring produced the greatest increase in equal-accuracy disparities, increasing the mean Equal Accuracy Gap from 0.119 to 0.268, driven primarily by age-group differences. Although race-based results were influenced by very small subgroup sample sizes, the overall pattern was consistent across demographic axes. These findings suggest that privacy-preserving image transformations can unintentionally worsen demographic disparities in medical AI systems. Consequently, anonymization techniques should be evaluated not only for their effectiveness in protecting patient privacy but also for their impact on fairness across patient populations.

Effects of Medical Image Anonymization on Fairness in Chest X-ray Classification

Missing data can create significant problems in healthcare datasets. It can affect interpretations made from the datasets as well as the fairness and performance of models trained on the datasets. While there are approaches that can help correct missing values in a dataset, it can be difficult to identify the mechanism of missingness. In this paper, we design a tool that uses Little’s MCAR test and a shadow matrix correlation analysis to identify whether missingness is consistent with the Missing Completely at Random (MCAR), Missing at Random (MAR), and Missing Not at Random (MNAR) mechanisms. Little’s MCAR test was used as the initial screening tool, and the shadow matrix correlation analysis was used to test for relationships between missingness and the observed variables in the dataset. In order to evaluate the framework, synthetic MCAR, MAR, and MNAR scenarios were introduced into a Diet Recommendations dataset, then the tool was applied to the dataset to confirm its ability to detect missingness. The tool was able to successfully flag simulated MCAR and MAR patterns. Due to its nature, MNAR cannot be definitively proven as it depends on unobserved data. However, with a combination of graphical outputs from our tool and relevant domain knowledge, MNAR patterns may be detectable.

Developing a diagnostic tool for assessing missingness mechanisms in healthcare datasets

Artificial Intelligence (AI) is a tool that is increasingly being used in dermatology. However, systems are not always producing the same results in differing skin tones. Inequitable performance between groups can worsen health inequities. This study analyzes whether AI diagnostic perfor- mances differ between Fitzpatrick skin type groups for classifying malignant and non-malignant skin conditions using the deep-learning model MobileNetV2. The data set used was Fitzpatrick17k, which includes 16,577 pictures of skin conditions, labeled by diagnosis and Fitzpatrick skin tone classification (I-VI). Performance was measured through accuracy, precision, recall, and subgroup comparison between skin tone groups. The Fitzpatrick17k dataset was found to be unbalanced with a 2.2 times higher rates of lighter skin tone images (Fitzpatrick I-III) than darker skin tone images (Fitzpatrick IV-VI). Sensitivity analysis across multiple thresholds showed that recall rates were also consistently lower in IV-VI images than in I-III images. These results suggest that the imbalance of data sets can contribute to unfair diagnostic performance and emphasize the need for more representative dermatology datasets.

Assessment of Diagnostic Performance Across Fitzpatrick Skin

Artificial intelligence is increasingly used to predict new drug treatments by learning from large biomedical databases. If those databases describe some diseases far better than others, AI predictions may inherit that imbalance while appearing equally reliable for all. We tested whether this occurs for neglected tropical diseases (NTDs), a group of conditions that affect over a billion people in low-income regions yet receive minimal research investment. Using Open Targets, a widely used drug–disease evidence platform, we trained a knowledge graph model on 759,217 evidence links and audited its predictions across 96 diseases, introducing the clinical translation ratio: the share of a disease's evidence drawn from clinically actionable sources rather than basic research. NTDs had a 42-fold lower clinical translation ratio than other diseases (0.5% versus 20.4%, p = 3.49 × 10⁻⁸). The gap held across four evidence-classification schemes, persisted against matched infectious diseases, and widened over three database releases. Twelve of nineteen NTDs had no clinically actionable evidence in any release. Most concerning, the model did not account for this scarcity. It was equally confident in its NTD predictions despite near-zero supporting evidence, and its top NTD predictions were three times less likely to correspond to known treatments. The model inherits the evidence gap and hides it behind uniform confidence. To address this, we built an open evidence-equity auditing framework and dashboard that reveals when AI confidence rests on evidence poverty. Before AI can accelerate drug discovery, it must first show whose evidence it is missing.

Downloads

Next from Equitable Health Technologies: Overcoming Biases and False Inferences in AI Diagnostics and Drug Discovery

Rank and Method Variance in ProAffinity-GNN Binding Affinity Estimates from AlphaFold Variants

Similar lecture

Assessment of Diagnostic Performance Across Fitzpatrick Skin

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from Equitable Health Technologies: Overcoming Biases and False Inferences in AI Diagnostics and Drug Discovery

Rank and Method Variance in ProAffinity-GNN Binding Affinity Estimates from AlphaFold Variants

Similar lecture

Assessment of Diagnostic Performance Across Fitzpatrick Skin

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads