dataset-shift

Frouros: an open-source Python library for drift detection in machine learning systems.

Code for 'Automatic dataset shift identification to support root cause analysis of AI performance drift', MICCAI 2025

A flexible and powerful Python library for dataset shift analysis and characterization, providing supervised and unsupervised evaluation of temporal and multi-source data shifts, visualization tools, and statistical insights for data integrity and model performance monitoring

Code for "Distance Matters for Improving Performance Estimation Under Covariate Shift", ICCV Workshop on Uncertainty Quantification 2023, Roschewitz & Glocker.

Check and correct a clinical risk model's calibration when the patient mix changes. AUC can stay at 0.89 while predicted risk is 39% too high; calshift measures it and fixes what one number can.

Code for the paper "Where are we with calibration under dataset shift in image classification?"

statistical tests for drift detection and dataset shift

Cross-dataset chest X-ray reliability study of semantic shift, calibration, and high-confidence errors.

Reproducible disease-classification study: data audit, nested model comparison, confusion matrices, calibration, thresholds, and hospital transfer.

Code and results for "Stable Discrimination Can Hide Reliability Failures in AI Decision Support Under Distribution Shift and Changing Target Definitions". Computers 15(9), 560 (2026). DOI: 10.3390/computers15090560

Analysis code and aggregate results for a three-database study (MIMIC-IV, eICU-CRD, ALOTT) of cross-institution transport loss in a sepsis-associated acute kidney injury prediction model

Reliability-first cardiovascular AI with patient-disjoint HPO, selective prediction, calibration stress tests, and cross-dataset transportability auditing.

Calibration, seed stability, and external validation for diabetic retinopathy under dataset shift.

Code behind a calibration audit of the public China-Fundus-CIMT benchmark: the model ranks patients well (AUC 0.89) but predicts 39% too much risk in a differently mixed group.

WaX: explainable Wasserstein distances. Attributes a Wasserstein distance to instances, features and subspaces (Naumann, Kauffmann and Montavon, TPAMI 2026).

Label-shift vs concept-drift experiments and a lightweight logit-offset adapter for prevalence shift in binary classification.

Beyond Random Tile Splits: Source-Held-Out Evaluation and Multi-Stream Fusion for Colorectal Histology. Held-out-source evaluation of color, traditional CV, CNN, and foundation-model (Phikon-v2, ConvNeXt-Tiny) pipelines on Kather-5k colorectal histology. Macro F1 drops from 0.98 to 0.82 when whole source images are held out.

Genome-anchored calibration of a continuous phenotype across cohorts, with uncertainty-propagated probabilistic labels

Shift type, not magnitude, determines ML failure modes under deployment shift — a cross-domain audit protocol and benchmark

Clinical laboratory AI assurance: cross-site benchmarking, population shift detection, CLIA/CAP method verification and medical-director reapproval on federated aggregates