This project involves fine-tuning a Huggingface transformer model to perform span-based question answering. Given a context and a question, the model is trained to predict the start and end tokens of the answer from the context.
This project was completed as part of an Explainable AI course.
The goal is to fine-tune a pre-trained transformer model to accurately predict answer spans from the input context and question.
-
Model:
DistilBertForQuestionAnswering- A lightweight version of BERT used for question-answering tasks.
-
Tokenizer:
DistilBertTokenizerFast- Efficiently encodes the input context and question pairs into token sequences suitable for the model.
The following hyperparameters were used for fine-tuning:
- Batch Size: 8
- Learning Rate: 5e-5
- Optimizer: AdamW