| ID | Benchmark (bleu) | Output | OPUS-MT | bleu | external | bleu | Diff |
|---|---|---|---|---|---|---|---|
| 0 | flores101-devtest | compare | eng-por/opus...2022-03-13 | 50.4 | facebook/nllb-200-3.3B | 47.2 | 3.2 |
| 1 | flores200-devtest | compare | gem-fra+ita+...2024-08-17 | 50.7 | facebook/nllb-200-3.3B | 46.9 | 3.8 |
| 2 | ntrex128 | compare | deu+eng+fra+...2024-05-30 | 34.2 | facebook/nll...illed-1.3B | 36.2 | -2.0 |
| 3 | tatoeba-test-v2020-07-28 | compare | gem-fra+ita+...2024-08-17 | 53.5 | facebook/m2m100_1.2B | 49 | 4.5 |
| 4 | tatoeba-test-v2021-03-30 | compare | gem-fra+ita+...2024-08-17 | 54 | facebook/m2m100_1.2B | 49.3 | 4.7 |
| 5 | tatoeba-test-v2021-08-07 | compare | deu+eng+fra+...2024-05-30 | 54.1 | facebook/m2m100_1.2B | 49.4 | 4.7 |
| 6 | tico19-test | compare | deu+eng+fra+...2024-05-30 | 50.3 | facebook/nllb-200-3.3B | 50.4 | -0.1 |
| average | 49.6 | 46.9 | 2.7 |