A CONTEXT-AWARE MEDICAL SUPPORT SYSTEM USING RETRIEVAL-AUGMENTED GENERATION WITH LANGCHAIN, HUGGING FACE, MISTRAL LLM, AND FAISS

Abstract

An important problem in healthcare use cases is the inclination of conventional AI language models to provide realistic but factually incorrect or hallucinated answers. This limitation has been made apparent by increasing the need for credible medical infor mation. Incorrect claims about facts and lack of citations to credible sources are ef fects of employing only parametric data in standard approaches such as GPT-3.5 and other fine-tuned LLMs. In complex medical domains specifically, existing instantiations of retrieval-augmented generation (RAG) systems are plagued by a trade-off between re trieval precision, semantic coherence, and response suitability, even though RAG systems were initially suggested as the solution to these issues. This work constructed a RAG based medical AI assistant to assist with these obstacles. To provide answers in a timely manner, it utilizes FAISS for the retrieval of vectors and Mistral-7B-Instruct-v0.3. The AI has also been trained to provide responses only that are supported by valid medical sources, according to The Gale Encyclopedia of Medicine. The quantitative metrics that were used for our system’s evaluation were BERTScore (Precision: 0.8334, Recall: 0.8119, F1-Score: 0.8225), Answer Relevancy (0.9221), and Faithfulness (1.0). Its performance was significantly superior to that of general-purpose models like GPT-3.5-Turbo (Faithfulness: 0.89) and LLaMA-2-70B-Chat (Faithfulness: 0.95). While being still therapeutically relevant and with high semantic coherence, our RAG model effectively limits hallucinations, as per the outcome. The observation that issues nonetheless occur when handling immensely complicated or unclear questions still supports the need for better reasoning methods. This paper sheds light on the potential of RAG systems that are tailored to specific areas of healthcare. Offers a reliable way of accessing correct medical records and gives way to innovation in dynamic knowledge fusion and multi-hop retrieval. The findings favor the applica tion of specialist AI assistants in areas such as healthcare and academia where accuracy, transparency, and reliability are essential.