Medical-RAG-System

A locally deployed Retrieval-Augmented Generation (RAG) system for medical case analysis, designed to retrieve clinically relevant evidence from medical documents and generate grounded responses. The project compares TF-IDF, all-mpnet-base-v2, and Bio_ClinicalBERT retrievers, supports local answer generation, and evaluates performance with Precision@k, Recall@k, BLEU, and ROUGE-L.

Overview

This project is built as a full medical RAG workflow rather than a simple chatbot wrapper. It supports:

• ingestion of medical .txt and .pdf documents
• chunking and indexing of clinical passages
• retriever comparison across baseline and embedding-based methods
• grounded answer generation from retrieved evidence
• evaluation on bundled and external medical QA benchmarks

Features

Privacy-Centric Deployment: Runs locally for privacy-sensitive medical document analysis.

Retriever Comparison: Compares TF-IDF, all-mpnet-base-v2, and Bio_ClinicalBERT on the same benchmark pipeline.

Grounded Medical Answers: Generates responses from retrieved evidence passages instead of unconstrained free-form output.

Quantified Evaluation: Measures retrieval and answer quality using Precision@k, Recall@k, BLEU, and ROUGE-L.

Interactive Interface: Streamlit app for document upload, evidence inspection, querying, and evaluation.

System Overview

flowchart LR
    A["Upload Medical Documents"] --> B["Parse PDF / TXT Files"]
    B --> C["Preprocess and Chunk Text"]
    C --> D["Embedding / Indexing Layer"]
    Q["Clinical Query"] --> E["Retriever Search"]
    D --> E
    E --> F["Top-k Relevant Passages"]
    F --> G["Prompt Builder"]
    G --> H["Generation Layer"]
    H --> I["Grounded Medical Answer"]
    I --> J["Evaluation Dashboard"]

Loading

The pipeline works as follows:

1. Upload medical case documents in .txt or .pdf format.
2. Parse and preprocess the text from each document.
3. Chunk documents into short evidence passages.
4. Build lexical or embedding-based retrieval indexes.
5. Retrieve top-k passages for a clinical query.
6. Build a grounded prompt from retrieved evidence.
7. Generate a response using the selected generator.
8. Evaluate retrieval and answer quality on benchmark queries.

Technical Stack

Programming Language: Python 3.10+

Libraries

• PyTorch
• Hugging Face Transformers
• SentenceTransformers
• scikit-learn
• PyPDF2
• pandas
• NumPy
• Streamlit
• Plotly

Retrievers

• TF-IDF
• all-mpnet-base-v2
• Bio_ClinicalBERT

Generators

• extractive-fallback
• optional Hugging Face generator such as google/flan-t5-base
• extensible to local domain models such as gemma-2-2b or Medical-Llama3-8B

Evaluation

• Retrieval: Precision@k, Recall@k
• Generation: BLEU, ROUGE-L

Dataset Strategy

Corpus

• PubMedQA abstracts or medically sourced passage exports

Evaluation

• MedQuAD for medically grounded question-answer references
• HealthSearchQA for realistic consumer-style health questions

The app is wired to automatically use local exports placed in:

data/external/pubmedqa_corpus.jsonl
data/external/medquad_eval.csv
data/external/healthsearchqa_eval.csv

Equivalent .json or .csv variants are also supported for the corpus, and .json / .jsonl are supported for evaluation files.

How to Run

git clone https://github.com/aadyasingh55/Medical-RAG-System.git
cd Medical-RAG-System
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
streamlit run app.py

Results

Evaluation on the bundled benchmark compared TF-IDF, all-mpnet-base-v2, and Bio_ClinicalBERT over 9 indexed chunks and 3 medical queries.

Retriever Comparison

Retriever	Precision@3	Recall@3	BLEU	ROUGE-L	Precision uplift vs TF-IDF
TF-IDF	0.667	1.000	0.372	0.522	0.00%
all-mpnet-base-v2	0.833	1.000	0.238	0.335	+24.89%
Bio_ClinicalBERT	0.444	1.000	0.372	0.522	-33.43%

Key Result

• all-mpnet-base-v2 improved Precision@3 from 66.7% to 83.3%, a 24.9% uplift over the TF-IDF baseline while maintaining 100% Recall@3.
• TF-IDF retained stronger generation overlap on this small benchmark with BLEU 0.372 and ROUGE-L 0.522.

Retrieval performance

Generation quality

Notes

• The project is structured to support a public dataset stack of PubMedQA for retrieval and MedQuAD + HealthSearchQA for evaluation without changing the app architecture.
• Domain-specific retrievers and generators are loaded lazily and may require local model availability.
• If a heavyweight model is unavailable, the app falls back to a baseline retrieval and extractive generation path.
• This is an educational and portfolio system, not a diagnostic tool.