MTSI-BERT is a BERT based joint model for dialogue session classification. It was developed during my master degree thesis at LINKS Foundation under the supervision of @giusepperizzo.
Why
Session
The Architecture
How to use
Hyperparamters
Dataset
Results
Dependencies
MTSI-BERT goal is to extract information from the session of a multi-turn dialogue. It was developed as a joint model having three main tasks:
- End of session detection (EOS)
- Action classification for the session: corresponds to insert/fetch operations on a knowledge-base to fullfill the user goal for the session
- Intent classification for the session
A session is a contiguous ordered sequence of QA pairs in a multi-turn conversation. MTSI-BERT takes as input a triplet of QAQ to understand the existing relation between the previous QA pair and the current Q of the user. In this way it is able to detect the end-of-session.
To train the model:
python train.py
It will save the model dictionary into the folder:
savings/<TIMESTAMP>
and the plot of the loss into:
plots/
To test the model:
python test.py
Remember to set the path of the saved model to load in the args of the method:
def test(load_checkpoint_path):
| Parameter | Value |
|---|---|
| Mini-batch | 16 |
| BERT lr | 5e-5 |
| NN lr | 1e-3 |
| Weight decay | 0.1 |
| Milestones | 5, 10, 15, 20, 30, 40, 50, 75 |
| Gamma | 0.5 |
| Model | Precision | Recall | F1 |
|---|---|---|---|
| MTSI-BERT | 0.9915 ± 0.0003 | 0.9962 ± 0.0008 | 0.9938 ± 0.0005 |
| Reference | 0.9558 ± 0.0016 | 0.9659 ± 0.0003 | 0.9638 ± 0.0006 |
| Model | Precision | Recall | F1 |
|---|---|---|---|
| MTSI-BERT | 1.00 | 1.00 | 1.00 |
| Reference | 0.9980 | 0.9895 | 0.9937 |
| Model | Precision | Recall | F1 |
|---|---|---|---|
| MTSI-BERT | 1.00 | 1.00 | 1.00 |
| Reference | 1.00 | 1.00 | 1.00 |
- Python 3.7.4
- Transformer package by Hugging Face
- spaCy
paper: url
If you use this work please cite
@inproceedings{senese2020mtsi,
title={MTSI-BERT: A Session-aware Knowledge-based Conversational Agent},
author={Senese, Matteo Antonio and Rizzo, Giuseppe and Dragoni, Mauro and Morisio, Maurizio},
booktitle={Proceedings of The 12th Language Resources and Evaluation Conference},
pages={717--725},
year={2020}
}