Fold-stratified cross-validation for unbiased and privacy-preserving federated learning - Centre de Recherche en Cancérologie et Immunologie Nantes Angers Accéder directement au contenu
Article Dans Une Revue Journal of the American Medical Informatics Association Année : 2020

Fold-stratified cross-validation for unbiased and privacy-preserving federated learning

Résumé

Abstract Objective We introduce fold-stratified cross-validation, a validation methodology that is compatible with privacy-preserving federated learning and that prevents data leakage caused by duplicates of electronic health records (EHRs). Materials and Methods Fold-stratified cross-validation complements cross-validation with an initial stratification of EHRs in folds containing patients with similar characteristics, thus ensuring that duplicates of a record are jointly present either in training or in validation folds. Monte Carlo simulations are performed to investigate the properties of fold-stratified cross-validation in the case of a model data analysis using both synthetic data and MIMIC-III (Medical Information Mart for Intensive Care-III) medical records. Results In situations in which duplicated EHRs could induce overoptimistic estimations of accuracy, applying fold-stratified cross-validation prevented this bias, while not requiring full deduplication. However, a pessimistic bias might appear if the covariate used for the stratification was strongly associated with the outcome. Discussion Although fold-stratified cross-validation presents low computational overhead, to be efficient it requires the preliminary identification of a covariate that is both shared by duplicated records and weakly associated with the outcome. When available, the hash of a personal identifier or a patient’s date of birth provides such a covariate. On the contrary, pseudonymization interferes with fold-stratified cross-validation, as it may break the equality of the stratifying covariate among duplicates. Conclusion Fold-stratified cross-validation is an easy-to-implement methodology that prevents data leakage when a model is trained on distributed EHRs that contain duplicates, while preserving privacy.
Fichier principal
Vignette du fichier
2001.08090 (907.31 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03579136 , version 1 (08-01-2024)

Identifiants

Citer

Romain Bey, Romain Goussault, François Grolleau, Mehdi Benchoufi, Raphaël Porcher. Fold-stratified cross-validation for unbiased and privacy-preserving federated learning. Journal of the American Medical Informatics Association, 2020, 27 (8), pp.1244-1251. ⟨10.1093/jamia/ocaa096⟩. ⟨hal-03579136⟩
51 Consultations
10 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More