This directory uses the mbr package to reproduce an experiment from the paper It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk (Bertsch et al., 2023).
- Task: Summarization
- Language: English
- Model: facebook/bart-large-cnn (Lewis et al., 2020)
- MBR metric: ROUGE-1 (Lin, 2004)
- Number of samples: 30
- Number of references: 30
- Sampling approach: sampling with temperature 0.5
- Reference sampling approach: sampling with temperature 1.0
- Test set: CNN/DailyMail (Nallapati et al., 2016)
- Evaluation metric: ROUGE-1
- Baselines: greedy decoding, beam search
| Paper | Reproduction |
|---|---|
 |
 |