Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Document-level machine translation evaluation project: methodology, effort and inter-annotator agreement

Castilho, Sheila orcid logoORCID: 0000-0002-8416-6555 (2020) Document-level machine translation evaluation project: methodology, effort and inter-annotator agreement. In: 22nd Annual Conference of the European Association for Machine Translation, 3-5 Nov 2020, Lisbon, Portugal (Online). ISBN 978-989-33-0589-8

Abstract
Recently, document-level (doc-level) human evaluation of machine translation (MT) has raised interest in the community after a few attempts have disproved claims of “human parity” (Toral et al., 2018; Laubli et al., 2018). However, lit- ¨ tle is still known about best practices regarding doc-level human evaluation. This project aims to identify methodologies to better cope with i) the current state-of-theart (SOTA) human metrics, ii) a possible complexity when assigning a single score to a text consisted of ‘good’ and ‘bad’ sentences, iii) a possible tiredness bias in doc-level set-ups, and iv) the difference in inter-annotator agreement (IAA) between sentence and doc-level set-ups.
Metadata
Item Type:Conference or Workshop Item (Paper)
Event Type:Conference
Refereed:Yes
Uncontrolled Keywords:machine translation evaluation; human evaluation of machine translation
Subjects:Computer Science > Machine translating
Humanities > Translating and interpreting
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Research Initiatives and Centres > ADAPT
Published in: Proceedings of the 22nd Annual Conference of the European Association for Machine Translation. . EAMT. ISBN 978-989-33-0589-8
Publisher:EAMT
Official URL:https://eamt2020.inesc-id.pt/proceedings-eamt2020....
Copyright Information:© 2020 The Authors. CC-BY-ND
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:EAMT under its programme “2019 Sponsorship of Activities, Science Foundation Ireland Research Centres Programme (Grant 13/RC/2106), European Regional Development Fund
ID Code:24647
Deposited On:18 Jun 2020 11:58 by Sheila Castilho . Last Modified 18 Jun 2020 12:09
Documents

Full text available as:

[thumbnail of proceedings-eamt2020-475-476.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
178kB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record