How to Optimize Your Machine Translation System

In research land we're happy to say, oh, the metric I care about is blue. But if you're actually deploying a production machine translation system, do you really want to super fine tune for one of them when what you really care about is user experience or something like that? There was a recent paper by folks at Salesforce, they used Rouge instead of blue, but they optimized this directly and found that it was much worse than same multitasking with the language modeling even though they got higher Rouge scores. Yeah, the output just looks incomprehensible so why should you do this?" "I think there's also a little bit of a disconnect between research land and real land," he says.