| ID | Benchmark (bleu) | Output | OPUS-MT | bleu | external | bleu | Diff |
|---|---|---|---|---|---|---|---|
| 0 | flores101-devtest | compare | bul-jpn/opus-2021-02-18 | 12.2 | facebook/m2m100_1.2B | 24.2 | -12.0 |
| 1 | flores200-devtest | compare | bul-jpn/opus-2021-02-18 | 12.2 | facebook/m2m100_1.2B | 24.2 | -12.0 |
| 2 | ntrex128 | compare | bul-jpn/opus-2021-02-18 | 0.1 | facebook/nllb-200-1.3B | 0.5 | -0.4 |
| 3 | tatoeba-test-v2020-07-28 | compare | bul-jpn/opus-2021-02-18 | 18.1 | facebook/m2m100_418M | 19.6 | -1.5 |
| 4 | tatoeba-test-v2021-03-30 | compare | bul-jpn/opus-2021-02-18 | 18.3 | facebook/m2m100_418M | 19.7 | -1.4 |
| 5 | tatoeba-test-v2021-08-07 | compare | bul-jpn/opus-2021-02-18 | 18.3 | facebook/m2m100_418M | 19.5 | -1.2 |
| average | 13.2 | 17.9 | -4.7 |