Popular Neural Machine Translation model training uses strategies like backtranslation to improve BLEU scores, requiring large amounts of additional data and training. We introduce a class of conditional generative-discriminative hybrid losses that we use to finetune a trained machine translation model. Through a combination of targeted finetuning objectives and intuitive re-use of the training data the model has failed to adequately learn from, we improve the model performance of both a sentence-level and a simple contextual model without using any additional data. We target the improvement of pronoun translations through our finetuning and evaluate our models on a pronoun benchmark testset. Our sentence-level model shows a 0.5 BLEU improvement on both the WMT14 and the IWSLT13 De-En testsets, while our simple contextual model achieves the best results, improving from 31.81 to 32 BLEU on WMT14 De-En testset, and from 32.10 to 33.13 on the IWSLT13 De-En testset, with corresponding improvements in pronoun translation.
Pronoun-Targeted Finetuning for NMT with Hybrid Losses
Prathyusha Jwalapuram, Shafiq Joty, and Youlin Shen. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP'20) , pages 2267–2279, 2020.
PDF Abstract BibTex Slides