DEVELOPMENT OF FRAMEWORK FOR IMAGE MANIPULATION DETECTION

Abstract

In an era where digital images are extensively disseminated and manipulated, the authenticity of the visual content has become progressively vulnerable to manipulation. Digital images are now a crucial source of information in social media, thanks to advancements in technology and the Internet. Modern futuristic image editing software and tools make it simple to tweak a digital image without leaving any visible clues. The widespread use of digital images in news and legal proceedings has raised worries about their validity, integrity, and reliability. Manipulated or tampered photos can mislead the public, harm a person's reputation or business, influence political opinions, or impact criminal investigations. Conventional image manipulation techniques include copy-move, splicing and inpainting, whereas recent developments in image manipulation include synthetically generated images such as deepfake. Passive image manipulation detection and localization of manipulated regions within an image remains challenging. The thesis is structured into comprehensive chapters, beginning with foundational aspects, moving through specific and multiple manipulation detection methodologies and culminating in a robust solution for recent advancements in manipulations such as deepfake detection. The thesis laid the groundwork by introducing the fundamentals of image manipulation detection, including image manipulation categorization, basic terminologies, application of image manipulation, the challenges of image manipulation detection and the classification of forgery detection techniques. This foundational knowledge provided context for understanding the scope and complexity of the problem. Furthermore, motivation and problem statement, performance metrics and thesis organization are discussed. The thesis comprehensively reviews existing state-of-the-art (SOTA) methods employed for image manipulation detection. Various methods are reviewed, including traditional handcrafted, machine learning and deep learning-based methods for image manipulation detection. This review also examines the v limitations of the existing techniques and identifies the research gaps, leading to the formulation of research objectives. The thesis provides a targeted approach for specific types of manipulation detection, such as offline signature forgery detection (OfSFD) and copy-move forgery detection (CMFD). The thesis developed a robust and efficient method for writer- independent offline signature forgery detection (WIOfSFD). The technique presents a formulation that uses the pre-trained model to direct the feature learning process and uses the Siamese neural network (SNN) to distinguish between genuine and forged signatures. Also, a residual-based convolutional neural network has been developed for CMFD. The thesis introduces two methodologies, namely MDLFormer and LFRViT, for detecting multiple forgeries using a single framework. MDLFormer used multi- modal data to exploit various inconsistencies present in a manipulated image, global context-based swin transformer (GCST) encoder to enhance the model's ability to aggregate, refine, and focus on critical global discrepancies between various patches and feature pyramid network (FPN) based decoder for manipulation detection and localization. In contrast, LFRViT uses a Laplacian filter residual (LFR) based vision transformer (ViT) for multiple forgery detection. The thesis also presented a hybrid learning-based approach consisting of kernel principal component analysis (KPCA) for deepfake face manipulation detection. The method uses the EfficientNetV2-L model for the feature extraction topped up with KPCA for feature dimensionality reduction to have an effective and fast feature learning process. The method is robust to various facial manipulation techniques such as identity swap, expression swap, attribute-based manipulation, and entirely synthesized faces. Experimental results validate the method’s effectiveness and demonstrate its potential as a reliable tool for detecting synthetic manipulations, which are becoming more common in digital forensics. Finally, this thesis work is concluded and the future scope of image manipulation detection is discussed.