Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Monte Carlo modelling of confidence intervals in translation quality evaluation (TQE) and post-editing dstance (PED) measurement

Alekseeva, Alexandra orcid logoORCID: 0000-0002-7990-4592, Gladkoff, Serge, Sorokina, Irina and Han, Lifeng orcid logoORCID: 0000-0002-3221-2185 (2021) Monte Carlo modelling of confidence intervals in translation quality evaluation (TQE) and post-editing dstance (PED) measurement. In: Metrics 2021: Workshop on Informetric and Scientometric Research (SIG-MET), 23-24 Oct 2021, Online.

Abstract
From both human translators (HT) and machine translation (MT) researchers' point of view, translation quality evaluation (TQE) is an essential task. This is especially the case, when language service providers (LSPs) face huge amount of request frequently from their clients and users to acquire high-quality translations. While automatic translation quality assessment (TQA) metrics and quality estimation (QE) tools are widely available and easy to access, human assessment from professional translators (HAP) are often chosen as the golden standard \cite{han-etal-2021-TQA}. One challenge that comes to this point is this: \textit{to avoid the overall text quality checking from both cost and efficiency perspectives, how to choose the confidence sample size of the translated text, so as to properly estimate the overall text or document translation quality}? This work carries out such an motivated research to correctly estimate the confidence intervals \cite{Brown_etal2001Interval} regarding the sample size of translated text, e.g. the amount of words or sentences, that needs to be taken into account for confident evaluation of overall translation quality. The methodology we applied for this work is from Bernoulli Statistical Distribution Modelling (BSDM) and Monte Carlo Sampling Analysis (MCSA).
Metadata
Item Type:Conference or Workshop Item (Poster)
Event Type:Workshop
Refereed:Yes
Uncontrolled Keywords:Translation Quality Evaluation; Quality Estimation; Post-editing Distance; Confidence Intervals; Monte Carlo Modeling; Bernoulli Statistics
Subjects:Computer Science > Algorithms
Computer Science > Artificial intelligence
Computer Science > Computational linguistics
Computer Science > Computer software
Computer Science > Information technology
Mathematics > Probabilities
Mathematics > Stochastic analysis
Mathematics > Statistics
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Research Initiatives and Centres > ADAPT
Copyright Information:© 2021 The Authors
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:ADAPT
ID Code:26281
Deposited On:21 Sep 2021 09:48 by Lifeng Han . Last Modified 21 Sep 2021 09:48
Documents

Full text available as:

[thumbnail of Metrics21_Alekseeva_Poster.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
237kB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record