| ID | Benchmark (bleu) | Output | OPUS-MT | bleu | external | bleu | Diff |
|---|---|---|---|---|---|---|---|
| 0 | flores101-devtest | compare | aav-aav/opus...2021-09-30 | 4.9 | facebook/m2m100_1.2B | 4.1 | 0.8 |
| 1 | flores200-devtest | compare | aav-aav/opus...2021-09-30 | 4.9 | facebook/m2m100_1.2B | 4.1 | 0.8 |
| 2 | newstest2020 | compare | deu+eng+fra+...2024-05-30 | 0.9 | facebook/nllb-200-3.3B | 1.3 | -0.4 |
| 3 | ntrex128 | compare | aav-aav/opus...2021-09-30 | 5.6 | facebook/nllb-200-3.3B | 2.8 | 2.8 |
| 4 | tatoeba-test-v2020-07-28 | compare | aav-aav/opus...2021-09-30 | 0.6 | facebook/nllb-200-3.3B | 1.7 | -1.1 |
| 5 | tatoeba-test-v2021-03-30 | compare | aav-aav/opus...2021-09-30 | 0.6 | facebook/m2m100_1.2B | 1.2 | -0.6 |
| 6 | tatoeba-test-v2021-08-07 | compare | aav-aav/opus...2021-09-30 | 0.6 | facebook/nllb-200-3.3B | 2 | -1.4 |
| 7 | tico19-test | compare | aav-aav/opus...2021-09-30 | 14.6 | facebook/m2m100_1.2B | 8.6 | 6.0 |
| 8 | wikipedia.devtest.km-en | compare | aav-aav/opus...2021-09-30 | 0.3 | facebook/m2m100_1.2B | 0.3 | 0.0 |
| average | 3.7 | 2.9 | 0.8 |