Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Correlations of perceived post-editing effort with measurements of actual effort

Moorkens, Joss orcid logoORCID: 0000-0003-0766-0071, O'Brien, Sharon orcid logoORCID: 0000-0003-4864-5986, da Silva, Igor A.L., de Lima Fonseca, Norma B. orcid logoORCID: 0000-0002-0207-4789 and Alves, Fabio (2015) Correlations of perceived post-editing effort with measurements of actual effort. Machine Translation, 29 (3). pp. 267-284. ISSN 0922-6567

Abstract
Human rating of predicted post-editing effort is a common activity and has been used to train confidence estimation models. However, the correlation between human ratings and actual post-editing effort is under-measured. Moreover, the impact of presenting effort indicators in a post-editing user interface on actual post-editing effort has hardly been researched. In this study, ratings of perceived post-editing effort are tested for correlations with actual temporal, technical and cognitive post-editing effort. In addition, the impact on post-editing effort of the presentation of post-editing effort indicators in the user interface is also tested. The language pair involved in this study is English-Brazilian Portuguese. Our findings, based on a small sample, suggest that there is little agreement between raters for predicted post-editing effort and that the correlations between actual post-editing effort and predicted effort are only moderate, and thus an inefficient basis for MT confidence estimation. Moreover, the presentation of post-editing effort indicators in the user interface appears not to impact on actual post-editing effort.
Metadata
Item Type:Article (Published)
Refereed:Yes
Uncontrolled Keywords:Post-editing; post-editing effort; eye-tracking; confidence estimation; confidence indicators; machine translation; user evaluation
Subjects:Computer Science > Machine translating
Humanities > Translating and interpreting
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Humanities and Social Science > School of Applied Language and Intercultural Studies
Research Initiatives and Centres > Centre for Translation and Textual Studies (CTTS)
Research Initiatives and Centres > ADAPT
Publisher:Springer
Official URL:http://dx.doi.org/10.1007/s10590-015-9175-2
Copyright Information:© 2015 Springer
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:CNGL (Science Foundation Ireland Grant 12/CE/I2267), Research Brazil Ireland, FALCON Project (falcon-project.eu), funded by the European Commission through the Seventh Framework Programme (FP7) Grant Agreement No. 610879
ID Code:24714
Deposited On:25 Jun 2020 10:44 by Joss Moorkens . Last Modified 25 Jun 2020 10:44
Documents

Full text available as:

[thumbnail of Moorkensetal2015_MT.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
967kB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record