Costello, Eamon ORCID: 0000-0002-2775-6006, Holland, Jane and Kirwan, Colette (2018) The future of online testing and assessment: question quality in MOOCs. International Journal of Educational Technology in Higher Education, 15 (42).
Abstract
If MOOCs are to play a role in the future of higher education it is imperative that we
critically examine how they are currently functioning. In particular, questions persist
about the role MOOCs will play in the future of formal accredited learning. As the
focus turns from informal and free to formal, accredited and paid, greater scrutiny
will be brought to bear on the quality of the courses themselves. Although there
have been some empirical studies into the quality of MOOCs, a notable gap exists in
that such research has not examined Multiple Choice Questions (MCQs) which are a
key component of much MOOC assessment and testing. Previous research suggests
that flawed MCQ items may compromise the reliability and validity of these
assessments, potentially leading to inconsistent outcomes for students. This study
was hence designed to examine MCQ quality in MOOCs. 204 MCQs were analysed,
from a selection of 18 MOOCs, sampling the domains of computing, social science
and health sciences. Over 50% of MCQs (112) contained at least one item flaw; 57
MCQs contained multiple flaws. A large proportion of MOOC MCQs violated itemwriting
guidelines, which is comparable with previous studies examining the
prevalence of flaws in assessments in more traditional educational contexts. The
problem of low quality MCQs can be ameliorated by appropriate faculty training and
pre- and post-test quality checks. These activities are essential if MOOCs are to
become a force that can enable enhanced and improved pedagogies in the future
of higher education, instead of simply proceeding to replicate existing poor practices
at scale.
Metadata
Item Type: | Article (Published) |
---|---|
Refereed: | Yes |
Uncontrolled Keywords: | MOOCs; Educational futures; Tests Quality; Multiple choice questions; Assessment |
Subjects: | Social Sciences > Education |
DCU Faculties and Centres: | DCU Faculties and Schools > NIDL (National Institute for Digital Learning) |
Publisher: | Springer Open |
Official URL: | https://doi.org/10.1186/s41239-018-0124-z |
Copyright Information: | © The Author(s) 2018 |
Use License: | This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License |
ID Code: | 22780 |
Deposited On: | 07 Nov 2018 16:21 by Thomas Murtagh . Last Modified 07 Dec 2018 16:19 |
Documents
Full text available as:
Preview |
PDF
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
680kB |
Downloads
Downloads
Downloads per month over past year
Archive Staff Only: edit this record