Machine Translationness: a Concept for Machine Translation Evaluation and Detection (2016)

Author: Joaquim Moré López

Supervisor: Salvador Climent Roca

Machine translationness (MTness) is the linguistic phenomena that make machine translations distinguishable from human translations. This thesis intends to present MTness as a research object and suggests an MT evaluation method based on determining whether the translation is machine-like instead of determining its human-likeness as in evaluation current approaches. Therefore we present an evaluation method that assesses machine translations according to what they are (translations produced by a machine) and not to what they resemble (human translations).

The method rates the MTness of a translation with a metric, the MTS (Machine Translationness Score). The MTS calculation is in accordance with the results of an experimental study on machine translation perception by common people. MTS proved to correlate well with human ratings on translation quality. Besides, our approach allows the performance of cheap evaluations since expensive resources (e.g. reference translations, training corpora) are not needed.

Machine translationness ratings can be applied for other uses beyond machine translation evaluation. The MTS metric can be an important indicator to prevent the consequences of the massive use of MT, such as plagiarism and other forms of cheating, or the detection of unsupervised MT documents published on the Web.