ADVANCING VISUAL NARRATION THROUGH TRANSFORMERS

Abstract

Image captioning has experienced significant advancements over the past decade, transitioning from traditional semantic approaches to sophisticated neural network models. This thesis explores the enhancement of image captions using a novel architecture combining MobileNet and Transformers. Traditional models, such as the Encoder-Decoder framework with CNN-RNN architectures, have laid the groundwork for image description generation. However, these models often face limitations in capturing complex image contexts and generating coherent, contextually rich descriptions. The proposed model leverages MobileNet for efficient feature extraction and Transformers for superior sequence generation, addressing the limitations of earlier methods. By incorporating attention mechanisms, the model enhances the understanding of intricate image details, resulting in more accurate and descriptive captions. Experimental results demonstrate a 12.5% improvement in captioning performance compared to standard methods, showcasing the potential of this approach. This work contributes to the ongoing innovation in image captioning, paving the way for more advanced techniques and applications in the field.