| ID | Benchmark (bleu) | Output | OPUS-MT | bleu | external | bleu | Diff |
|---|---|---|---|---|---|---|---|
| 0 | flores101-devtest | compare | itc-itc/opus...2022-08-10 | 37.7 | facebook/m2m100_1.2B | 36.9 | 0.8 |
| 1 | flores200-devtest | compare | itc-deu+eng+...2024-05-30 | 38 | facebook/m2m100_1.2B | 36.9 | 1.1 |
| 2 | ntrex128 | compare | roa-deu+eng+...2024-05-30 | 29.4 | facebook/nll...illed-1.3B | 30.8 | -1.4 |
| 3 | tatoeba-test-v2020-07-28 | compare | itc-fra+ita+...2024-08-17 | 59.4 | facebook/m2m100_1.2B | 54.4 | 5.0 |
| 4 | tatoeba-test-v2021-03-30 | compare | itc-fra+ita+...2024-08-17 | 59.1 | facebook/m2m100_1.2B | 54.8 | 4.3 |
| 5 | tatoeba-test-v2021-08-07 | compare | itc-fra+ita+...2024-08-17 | 59.2 | facebook/m2m100_1.2B | 54.9 | 4.3 |
| average | 47.1 | 44.8 | 2.4 |