Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

The future of online testing and assessment: question quality in MOOCs

Costello, Eamon orcid logoORCID: 0000-0002-2775-6006, Holland, Jane and Kirwan, Colette (2018) The future of online testing and assessment: question quality in MOOCs. International Journal of Educational Technology in Higher Education, 15 (42).

Abstract
If MOOCs are to play a role in the future of higher education it is imperative that we critically examine how they are currently functioning. In particular, questions persist about the role MOOCs will play in the future of formal accredited learning. As the focus turns from informal and free to formal, accredited and paid, greater scrutiny will be brought to bear on the quality of the courses themselves. Although there have been some empirical studies into the quality of MOOCs, a notable gap exists in that such research has not examined Multiple Choice Questions (MCQs) which are a key component of much MOOC assessment and testing. Previous research suggests that flawed MCQ items may compromise the reliability and validity of these assessments, potentially leading to inconsistent outcomes for students. This study was hence designed to examine MCQ quality in MOOCs. 204 MCQs were analysed, from a selection of 18 MOOCs, sampling the domains of computing, social science and health sciences. Over 50% of MCQs (112) contained at least one item flaw; 57 MCQs contained multiple flaws. A large proportion of MOOC MCQs violated itemwriting guidelines, which is comparable with previous studies examining the prevalence of flaws in assessments in more traditional educational contexts. The problem of low quality MCQs can be ameliorated by appropriate faculty training and pre- and post-test quality checks. These activities are essential if MOOCs are to become a force that can enable enhanced and improved pedagogies in the future of higher education, instead of simply proceeding to replicate existing poor practices at scale.
Metadata
Item Type:Article (Published)
Refereed:Yes
Uncontrolled Keywords:MOOCs; Educational futures; Tests Quality; Multiple choice questions; Assessment
Subjects:Social Sciences > Education
DCU Faculties and Centres:DCU Faculties and Schools > NIDL (National Institute for Digital Learning)
Publisher:Springer Open
Official URL:https://doi.org/10.1186/s41239-018-0124-z
Copyright Information:© The Author(s) 2018
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
ID Code:22780
Deposited On:07 Nov 2018 16:21 by Thomas Murtagh . Last Modified 07 Dec 2018 16:19
Documents

Full text available as:

[thumbnail of s41239-018-0124-z.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
680kB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record