Abstract

High reported accuracies on small mammographic datasets are frequently inflated by data leakage, most often from augmentation applied before the train/test split. Changing only the split order on whole-image Mini-MIAS moves a deep-feature plus Support Vector Machine (SVM) pipeline from 0.976 accuracy to 0.513, which is chance. The predictive signal survives at the lesion-patch scale. Our contribution is protocol-level rather than architectural: a reproducible leakage audit; the first quantification and repair of the reported patient overlap in the CBIS-DDSM official split, which we trace to its mass and calcification partitions being drawn independently (31 patients, 8.5% of test lesions); an architecture sweep that tests rather than assumes the performance ceiling; and a label-free procedure for transferring the decision threshold to a new institution. A lightweight EfficientNet patch pipeline trained on CBIS-DDSM under a strict patient-wise protocol obtains ROC-AUC 0.745 (95% CI 0.703-0.794); without retraining it attains 0.734 (0.640-0.826) on MIAS and 0.877 (0.804-0.946) on INbreast, and a DeLong test cannot distinguish the MIAS result from the internal one (p=0.82). Results are stable over ten seeds (sigma

Key words: Breast lesion classification, Data leakage, Transfer learning, External validation, Mammography