| ID | Benchmark (bleu) | Output | OPUS-MT | bleu | external | bleu | Diff |
|---|
| 0 | flores101-devtest | compare | eng-fra/opus...2022-03-09 | 52.2 | facebook/nllb-200-3.3B | 48 | 4.2 |
| 1 | flores200-devtest | compare | eng-fra/opus...2022-03-09 | 52.2 | facebook/nllb-200-3.3B | 47.8 | 4.4 |
| 2 | multi30k_test_2016_flickr | compare | eng-fra/opus...2022-03-09 | 52.4 | facebook/nllb-200-3.3B | 51.3 | 1.1 |
| 3 | multi30k_test_2017_flickr | compare | eng-fra/opus...2022-03-09 | 52.8 | facebook/nll...illed-1.3B | 51.1 | 1.7 |
| 4 | multi30k_test_2017_mscoco | compare | gmw-deu+eng+...2024-05-30 | 55.7 | facebook/nll...illed-1.3B | 52.3 | 3.4 |
| 5 | multi30k_test_2018_flickr | compare | eng-fra/opus...2022-03-09 | 43.7 | facebook/nll...illed-1.3B | 44.2 | -0.5 |
| 6 | newsdiscusstest2015 | compare | eng-fra/opus...2022-03-09 | 40.3 | facebook/nll...illed-1.3B | 38.8 | 1.5 |
| 7 | newssyscomb2009 | compare | eng-fra/opus...2022-03-09 | 30.7 | facebook/nllb-200-3.3B | 32.6 | -1.9 |
| 8 | newstest2008 | compare | eng-fra/opus...2022-03-09 | 27.6 | facebook/nllb-200-1.3B | 28.3 | -0.7 |
| 9 | newstest2009 | compare | eng-fra/opus...2022-03-09 | 30 | facebook/nllb-200-3.3B | 30.9 | -0.9 |
| 10 | newstest2010 | compare | eng-fra/opus...2022-03-09 | 33.5 | facebook/nll...illed-1.3B | 33.9 | -0.4 |
| 11 | newstest2011 | compare | eng-fra/opus...2022-03-09 | 35 | facebook/nllb-200-3.3B | 35.8 | -0.8 |
| 12 | newstest2012 | compare | eng-fra/opus...2022-03-09 | 32.8 | facebook/nllb-200-3.3B | 33.4 | -0.6 |
| 13 | newstest2013 | compare | eng-fra/opus...2022-03-09 | 34.6 | facebook/nll...illed-1.3B | 35.2 | -0.6 |
| 14 | newstest2014 | compare | eng-fra/opus...2022-03-09 | 41.9 | facebook/nll...illed-1.3B | 41.6 | 0.3 |
| 15 | ntrex128 | compare | eng-fra/opus...2022-03-09 | 37.3 | facebook/nll...illed-1.3B | 34.6 | 2.7 |
| 16 | tatoeba-test-v2020-07-28 | compare | eng-fra/opus...2022-03-09 | 51.7 | facebook/nll...illed-1.3B | 50 | 1.7 |
| 17 | tatoeba-test-v2021-03-30 | compare | eng-fra/opus...2022-03-09 | 52.5 | facebook/nll...illed-1.3B | 50.7 | 1.8 |
| 18 | tatoeba-test-v2021-08-07 | compare | eng-fra/opus...2022-03-09 | 53.2 | facebook/nllb-200-3.3B | 51.3 | 1.9 |
| 19 | tico19-test | compare | eng-fra/opus...2022-03-09 | 40.6 | facebook/nll...illed-1.3B | 40.5 | 0.1 |
| | average | | 42.5 | | 41.6 | 0.9 |
|---|