MEDICAL DIAGNOSIS AUTOMATION USING LLM-POWERED MULTI-AGENT SYSTEMS

Abstract

The rapid advancements in Artificial Intelligence (AI), particularly Large Language Models (LLMs), have ushered in transformative opportunities in medical imaging diag nostics. However, the complexity and volume of medical imaging data, coupled with spe cialist shortages and the inherent opacity of AI systems, present significant challenges to timely, accurate, and trustworthy diagnosis. This thesis addresses these challenges by de veloping a comprehensive, AI-powered platform that automates medical image analysis, enhances explainability, fosters collaborative diagnosis, and enables interactive semantic querying of past cases. The proposed system integrates state-of-the-art multimodal LLMs (such as GPT-4 Vision) within a modular multi-agent architecture designed to mimic real-world clinical workflows. It supports diverse medical image formats—including DICOM, NIfTI, JPEG, and PNG—and applies advanced preprocessing pipelines for robust input handling. The AI engine generates detailed, structured diagnostic reports with clinically relevant find ings, differential diagnoses, and patient-friendly explanations. To overcome the black-box nature of AI, the platform incorporates explainable AI (XAI) techniques that generate saliency heatmaps highlighting image regions influential in diagnosis, thereby increasing clinician trust and transparency. Further, a multi-agent chat system simulates multidisciplinary collaboration among virtual specialist agents (e.g., radiologists, pulmonologists, cardiologists) and human users, facilitating dynamic clini cal discussion and consensus building. Complementing this, a retrieval-augmented gen eration (RAG) based question-answering module empowers users to pose context-aware queries regarding historical diagnostic reports, supporting evidence-backed, interactive decision support. Implemented as a user-friendly, web-based application using Streamlit, the platform also integrates automated medical literature retrieval from PubMed and clinical trial databases, enriching reports with current scientific knowledge. Evaluation on public datasets—including NIH ChestX-ray14 and brain MRI collections—demonstrates diagnostic accuracy ex ceeding 90%, clinically meaningful explainability, effective multi-agent collaboration, and high usability ratings from domain experts. This thesis contributes a scalable, extensible framework that bridges AI advancements iii and clinical practice, aiming to democratize diagnostic expertise, reduce radiologist work load, and improve patient outcomes. The work concludes with discussions on ethical considerations, system limitations, and future directions involving clinical integration, multimodal data fusion, on-device AI inference, and multilingual support.