Addressing the shortcomings of traditional multiple-choice tests: subset selection without mark deductions

Journal article


Otoyo, L and Bush, M (2018). Addressing the shortcomings of traditional multiple-choice tests: subset selection without mark deductions. Practical Assessment, Research and Evaluation. 23 (1).
AuthorsOtoyo, L and Bush, M
Abstract

This article presents the results of an empirical study of “subset selection” tests, which are a generalisation of traditional multiple-choice tests in which test takers are able to express partial knowledge. Similar previous studies have mostly been supportive of subset selection, but the deduction of marks for incorrect responses has been a cause for concern. For the present study, a novel marking scheme based on Akeroyd’s “dual response system” was used instead. In Akeroyd’s system, which assumes that every question has four answer options, test takers are able to split their single 100% bet on one answer option into two 50% bets by selecting two options, or into four 25% bets by selecting no options. To achieve full subset selection, this idea was extended so that test takers could also split their 100% bet equally between three options.

The results indicate increased test reliability (in the sense of measurement consistency), and also increased satisfaction on the part of the test takers. Furthermore, since the novel marking scheme does not in principle lead to either inflated or deflated marks, this makes it easy for educators who currently use traditional multiple-choice tests to switch to using subset selection tests.

Year2018
JournalPractical Assessment, Research and Evaluation
Journal citation23 (1)
PublisherPractical Assessment, Research & Evaluation
ISSN1531-7714
Publication dates
Print21 Dec 2018
Publication process dates
Deposited28 Nov 2018
Accepted21 Nov 2018
Accepted author manuscript
License
File Access Level
Open
Permalink -

https://openresearch.lsbu.ac.uk/item/86830

Download files


Accepted author manuscript
Addressing the shortcomings of m-c tests.pdf
License: CC BY 4.0
File access level: Open

  • 41
    total views
  • 26
    total downloads
  • 1
    views this month
  • 0
    downloads this month

Export as

Related outputs

Reducing the need for guesswork in multiple-choice tests
Bush, M (2014). Reducing the need for guesswork in multiple-choice tests. Assessment and Evaluation in Higher Education. 40 (2), pp. 218 - 231. https://doi.org/10.1080/02602938.2014.902192
A multiple choice test that rewards partial knowledge
Bush, M (2001). A multiple choice test that rewards partial knowledge. Journal of Further and Higher Education. 25 (2), pp. 157 - 163. https://doi.org/10.1080/03098770120050828
Quality assurance of multiple-choice tests
Bush, M (2006). Quality assurance of multiple-choice tests. Quality Assurance in Education. 14 (4), pp. 398 - 404. https://doi.org/10.1108/09684880610703974
Numerically-analysed Multiwavelet Transform computations: multidimensional compression case studies
Alramahi, N, Bush, M and Swash, R (2017). Numerically-analysed Multiwavelet Transform computations: multidimensional compression case studies. International Conference on Communication, Management and Information Technology. Warsaw, Poland 03 - 05 Apr 2017
Novel 3D Stereoscopic Film Compression using Multiwavelet
Alramahi, N, Bush, M and Swash, R (2017). Novel 3D Stereoscopic Film Compression using Multiwavelet. International Conference on Communication, Management and Information Technology. Warsaw, Poland 03 - 05 Apr 2017
Coding and Decoding Stereoscopic Film Compression by Multiwavelet Transform
Alramahi, N, Bush, M and Swash, R (2017). Coding and Decoding Stereoscopic Film Compression by Multiwavelet Transform. International Conference on Communication, Management and Information Technology. Warsaw, Poland 03 - 05 Apr 2017
Size Optimization of 3D Stereoscopic Film Frames
Alramahi, N and Bush, M (2017). Size Optimization of 3D Stereoscopic Film Frames. International Conference on Communication, Management and Information Technology. Warsaw, Poland 03 - 05 Apr 2017