README.md

---
language:
- en
license: apache-2.0
tags:
- dialogue state tracking
- task-oriented dialog

---

# roberta-base-trippy-dst-multiwoz21

This is a TripPy model trained on [MultiWOZ 2.1](https://github.com/budzianowski/multiwoz) for use in [ConvLab-3](https://github.com/ConvLab/ConvLab-3).
This model predicts informable slots, requestable slots, general actions and domain indicator slots.
Expected joint goal accuracy for MultiWOZ 2.1 is in the range of 55-56\%.

For information about TripPy DST, refer to [TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking](https://aclanthology.org/2020.sigdial-1.4/).

The training and evaluation code is available at the official [TripPy repository](https://gitlab.cs.uni-duesseldorf.de/general/dsml/trippy-public).

## Training procedure

The model was trained on MultiWOZ 2.1 data via supervised learning using the [TripPy codebase](https://gitlab.cs.uni-duesseldorf.de/general/dsml/trippy-public).
MultiWOZ 2.1 data was loaded via ConvLab-3's unified data format dataloader.
The pre-trained encoder is [RoBERTa](https://huggingface.co/docs/transformers/model_doc/roberta) (base).
Fine-tuning the encoder and training the DST specific classification heads was conducted for 10 epochs.

### Training hyperparameters

```
python3 run_dst.py \
  --task_name="unified" \
  --model_type="roberta" \
  --model_name_or_path="roberta-base" \
  --dataset_config=dataset_config/unified_multiwoz21.json \
  --do_lower_case \
  --learning_rate=1e-4 \
  --num_train_epochs=10 \
  --max_seq_length=180 \
  --per_gpu_train_batch_size=24 \
  --per_gpu_eval_batch_size=32 \
  --output_dir=results \
  --save_epochs=2 \
  --eval_all_checkpoints \
  --warmup_proportion=0.1 \
  --adam_epsilon=1e-6 \
  --weight_decay=0.01 \
  --fp16 \
  --do_train \
  --predict_type=dummy \
  --seed=42
```