DESIGN AND DEVELOPMENT OF DEEP NEURAL NETWORKS ARCHITECTURES FOR IMBALANCED DATASETS IN COMPUTER VISION

Abstract

This thesis involves the design and development of deep learning techniques to address the class imbalance problem in the computer vision domain. The core idea behind this study is to examine various approaches which would help to combat the impact of biases towards the majority class which could leave the minority classes undetected, that overall might lead to misleading performance results. Very limited empirical study is found in this research area of deep learning while dealing with class imbalance. Several challenges are involved while dealing with imbalanced datasets due to the unequal distribution of samples corresponding to each class present in the dataset, including biased and lower performances in minority classes. Hence it is very challenging to deal with such imbalanced problems, especially in the case of the multi class imbalanced datasets. Different evaluation parameters also need to be considered for evaluating the overall performance of the model. Throughout the study, we have tried to bring the changes at the data level and the algorithm level by designing and developing novel deep learning techniques to deal with the imbalanced data to solve computer vision problems. For validating our approach, we have used various challenging binary and multi-class imbalanced datasets including Graz-02 dataset, TF-Flowers dataset, BreakHis dataset, Breast-Histopathological-Images dataset, Kaggle Diabetic Retinopathy dataset, DDR Dataset, Indian Diabetic Retinopathy Image (IDRiD) dataset and Intel MobileODT Cervical Cancer Screening dataset. We present insights into the design and implementation of deep learning models with imbalanced datasets of various scales. In support of the same, we have conducted a detailed curated set of experiments on the available benchmark datasets. A detailed comprehensive experimental analysis is conducted on the datasets, comparing our results with the state-of-the-art methods in the field. Our contributions are summarized below, highlighting our key findings and innovations. We have performed a thorough analysis of multiple state-of-the-art pre-trained networks across various tasks, including classification, object detection, and segmentation on varied size datasets (small, medium, and large). These tasks were evaluated on diverse applications, such as diabetic retinopathy, breast cancer, cervical cancer, and more. Additionally, we proposed efficient machine learning classifiers, such as Chi² SVM, Quasi SVM and weighted SVM, to address the challenges posed by imbalanced datasets. These classifiers aim to mitigate the impact of class imbalance and improve overall performance. A novel model using visual codebook generation obtained from ResNet-50 deep features along with the Chi² SVM classifier is proposed to effectively tackle the class imbalance problem that arises while dealing with multi-class image datasets. Another contribution consists of exploring the effect of data augmentation on the overall performance of the deep learning models. The effect of data augmentation approaches was seen after applying (i) Traditional affine transformation (shifted, zoomed in/out, rotated, flipped, distorted, cropping, rescaling or shaded with hue, etc.) and (ii) Generative Adversarial Nets (GANs) to generate synthetic samples from the original images which makes the models more robust and also helps in resolving the class imbalance issue. We have proposed a novel learning framework in collaboration with the Deep Convolutional Generative Adversarial network (DCGAN). The DCGAN is used in the initial phase for data augmentation of the minority class only with the modified less computationally challenging VGG16 deep network architecture. The significance of adding batch normalization layers is discussed as it helps to mitigate the effect of covariance shift. Additionally, emphasis is given to hyperparameters, and fine-tuning also plays a crucial role in the overall model performance. Major contribution is the development of novel deep learning architecture VGGIN-NET which adapts to class imbalance in both binary and multi class datasets.