This paper proposes a new learning methodology involving deep features and two-way metric learning for large, extremely imbalanced face datasets where the number of minority classes and the imbalance ratio are both very high. The problem arises because the faces of some celebrities, being more popular, are readily available in social media and the internet, while the faces of some relatively lesser-known personalities are fewer in number. Resampling being impractical in this scenario, we propose metric learning as the tool for mitigating the class-imbalance problem prior to the classification stage. To reduce the computational overhead associated with metric learning, we separately conduct weakly supervised metric learning with majority and minority class subsets, a process that we call two-way metric learning. Transformation matrices learnt from the majority and minority subsets are used to transform the entire input space twice. The test sample in the transformed space is assigned the class of its nearest neighbor in the training set of the twice-transformed input space. Deep features derived from the state-of-the-art pre-trained deep network VGG-Face form the input space, and the aggregate cosine similarity measure is used to find the closest neighbor in the training set of the twice-transformed input space. Experiments on the benchmark LFW face database having 1680 classes of celebrity faces prove that the proposed methodology is more effective than existing methods for the classification of large, extremely imbalanced face datasets. The classification accuracies of the minority classes are especially found to be boosted which is a rare accomplishment among existing methods for imbalanced learning in deep frameworks.
Key words: Face Recognition, Metric Learning, VGG-Face, Deep Learning, Imbalanced Learning, Extremely imbalanced dataset
|
