MEDICAL IMAGE ANALYSIS OF WIRELESS CAPSULE ENDOSCOPY DATA

Abstract

Gastrointestinal (GI) diseases, often diagnosed through endoscopy, constitute a significant global health burden. The manual inspection of endoscopy data, particularly in colonoscopies and video capsule endoscopy (VCE) is time-intensive and prone to oversight. Automating abnormality detection and cleanliness assessment through medical image analysis (MIA) and artificial intelligence (AI) promises to revolutionize this process, offering quicker and more precise diagnostics. Such assessments may help to enhance patient outcomes by developing sophisticated algorithms capable of detecting abnormalities and assessing cleanliness in real-time. By streamlining endoscopy evaluations, such automatic assessments may help in addressing critical healthcare needs, facilitating earlier detection, and intervention for GI diseases, ultimately improving patient care and reducing the burden on healthcare systems. Currently, the diagnostic yield of colonoscopy and VCE in a real-time clinical setting has been investigated in several Indian and abroad medical studies but MIA and AI based studies to perform automatic abnormality detection and cleanliness assessment in endoscopy are rare. The absence of high quality, multi-labelled, and medically validated AI datasets is a major reason behind the less no. of studies being conducted. Lack of AI datasets hinder a transparent comparison between the performance of existing automated systems in this field with the up-coming systems. Most of the automated systems have been designed for less no. of endoscopy frames which are unavailable for public research use. The existing datasets mostly contain binary class labels such as ‘abnormal’ or ‘normal/healthy’ and ‘clean’ or ‘unclean/dirty’ and ‘adequate’ or ‘inadequate’ and do not provide information related to mucosal visual quality, presence of impairments, artefacts, medical scores and distortions etc. Multi-label classification is an emerging and presently less explored area. It xi has the potential to address several tasks such as the automatic cleanliness scoring in endoscopy. The assessment of cleanliness in endoscopy is crucial for ensuring optimal visualization and, consequently, accurate diagnosis. Cleanliness metrics play a pivotal role in maintaining the quality of endoscopy examinations, allowing healthcare professionals to make informed decisions based on clear and unobstructed images. They play an even more crucial role in VCE as it is non-invasive in nature and lacks therapeutic capabilities. Owing to the above-discussed research gaps, this research focuses on two tasks namely automatic detection of abnormality in polyp and non polyp frame in colonoscopy frames and automatic assessment of cleanliness in VCE. The first task focused on developing an explainable, end-to-end and robust architecture for automatic colorectal polyp diagnosis using colonoscopy polyp and non-polyp frames. The developed architecture consisted of a novel, fine-tuned feature extracting module, followed by polyp and non-polyp frame identification and a window-based polyp detection system. To show the robustness of the developed architecture, a new test set was developed and evaluated. After the analysis, it was released on Zenodo, an open-source platform for research purposes. It is called the gastrointestinal atlas-colon polyp dataset. It consisted of seven patient videos obtained from open-source, copyright free web sources. Explainable and evaluation methods like class activation mapping, feature mapping, occlusion testing, hyper-parameter tuning ablation experiments, and separate, sequential, and non-sequential frame-based test set analysis have been used to show the efficacy of the proposed architecture. The developed architecture has been compared with the existing state-of-the-art methodologies in this field. Additionally, architecture has been compared with a transfer learning architecture as well. The second task focused on development of methodology to automatically assess the cleanliness in VCE video frames. First, the process of scoring VCE frames has been automated as per existing KOrea CanaDA (KODA) scoring system. The process is an easy-to-use mobile application called AI-KODA. AI-KODA Score is a flutter-based application which can be downloaded on a mobile. The application first trains a gastroenterologist how to use KODA. After a simple training, the xii gastroenterologist can upload VCE video frames on the application and score them. After successful scoring, a report is generated for the overall score. The scores are also collected in real-time and saved for the development of a frame level, high-quality, and multi-labelled dataset for automatic multi-label classification of clean v/s dirty VCE video frames. The developed dataset has been subjective to medical verification with the help of three experienced gastroenterologists. Based on the common consensus by the three gastroenterologists, a common dataset comprising of 2173 with eight distinct labels of KODA has been developed. A comprehensive evaluation, interpretation, benchmarking of the generated dataset has been done using ten machine learning models and eight transfer learning algorithms on google Collaboratory and a supercomputer named, NVIDIA RTX A5000 workstation. The developed dataset and its methodology are first-of-its-kind. The proposed methodologies for these two tasks are scalable, robust, work in real-time, and are explainable in nature. The comprehensive analysis followed for each of the tasks shows its promising future in the gastroenterology department. Both the methodologies help in reducing the time and effort of the gastroenterologist in timely detection of polyps in colonoscopy and cleanliness assessment in VCE.