| ID | Benchmark (bleu) | Output | OPUS-MT | bleu | external | bleu | Diff |
|---|---|---|---|---|---|---|---|
| 0 | flores101-devtest | compare | eng-ita/opus...2022-03-13 | 29.6 | facebook/nllb-200-3.3B | 29.4 | 0.2 |
| 1 | flores200-devtest | compare | deu+eng+fra+...2024-05-30 | 30.6 | facebook/nllb-200-3.3B | 29.4 | 1.2 |
| 2 | newssyscomb2009 | compare | gem-fra+ita+...2024-08-17 | 31.5 | facebook/nll...illed-1.3B | 31.6 | -0.1 |
| 3 | newstest2009 | compare | deu+eng+fra+...2024-05-30 | 31.9 | facebook/nll...illed-1.3B | 32.5 | -0.6 |
| 4 | ntrex128 | compare | deu+eng+fra+...2024-05-30 | 36.8 | facebook/nll...illed-1.3B | 36 | 0.8 |
| 5 | tatoeba-test-v2020-07-28 | compare | eng-ita/opus...2022-03-13 | 50.6 | facebook/nllb-200-3.3B | 51.8 | -1.2 |
| 6 | tatoeba-test-v2021-03-30 | compare | eng-ita/opus...2022-03-13 | 52.1 | facebook/nllb-200-3.3B | 53.1 | -1.0 |
| 7 | tatoeba-test-v2021-08-07 | compare | eng-ita/opus...2022-03-13 | 53.9 | facebook/nllb-200-3.3B | 54.3 | -0.4 |
| average | 39.6 | 39.8 | -0.1 |