Addressing the shortcomings of traditional multiple-choice tests: subset selection without mark deductions

Journal article


Otoyo, L and Bush, M (2018). Addressing the shortcomings of traditional multiple-choice tests: subset selection without mark deductions. Practical Assessment, Research and Evaluation. 23 (1).
AuthorsOtoyo, L and Bush, M
Abstract

This article presents the results of an empirical study of “subset selection” tests, which are a generalisation of traditional multiple-choice tests in which test takers are able to express partial knowledge. Similar previous studies have mostly been supportive of subset selection, but the deduction of marks for incorrect responses has been a cause for concern. For the present study, a novel marking scheme based on Akeroyd’s “dual response system” was used instead. In Akeroyd’s system, which assumes that every question has four answer options, test takers are able to split their single 100% bet on one answer option into two 50% bets by selecting two options, or into four 25% bets by selecting no options. To achieve full subset selection, this idea was extended so that test takers could also split their 100% bet equally between three options.

The results indicate increased test reliability (in the sense of measurement consistency), and also increased satisfaction on the part of the test takers. Furthermore, since the novel marking scheme does not in principle lead to either inflated or deflated marks, this makes it easy for educators who currently use traditional multiple-choice tests to switch to using subset selection tests.

Year2018
JournalPractical Assessment, Research and Evaluation
Journal citation23 (1)
PublisherPractical Assessment, Research & Evaluation
ISSN1531-7714
Publication dates
Print21 Dec 2018
Publication process dates
Deposited28 Nov 2018
Accepted21 Nov 2018
Accepted author manuscript
License
CC BY 4.0
File Access Level
Open
Permalink -

https://openresearch.lsbu.ac.uk/item/86830

  • 23
    total views
  • 16
    total downloads
  • 0
    views this month
  • 0
    downloads this month

Related outputs

Numerically-analysed Multiwavelet Transform computations: multidimensional compression case studies
Alramahi, N, Bush, M and Swash, R (2017). Numerically-analysed Multiwavelet Transform computations: multidimensional compression case studies. International Conference on Communication, Management and Information Technology. Warsaw, Poland 03 - 05 Apr 2017 London South Bank University.
Novel 3D Stereoscopic Film Compression using Multiwavelet
Alramahi, N, Bush, M and Swash, R (2017). Novel 3D Stereoscopic Film Compression using Multiwavelet. International Conference on Communication, Management and Information Technology. Warsaw, Poland 03 - 05 Apr 2017 London South Bank University.
Coding and Decoding Stereoscopic Film Compression by Multiwavelet Transform
Alramahi, N, Bush, M and Swash, R (2017). Coding and Decoding Stereoscopic Film Compression by Multiwavelet Transform. International Conference on Communication, Management and Information Technology. Warsaw, Poland 03 - 05 Apr 2017 London South Bank University.
Size Optimization of 3D Stereoscopic Film Frames
Alramahi, N and Bush, M (2017). Size Optimization of 3D Stereoscopic Film Frames. International Conference on Communication, Management and Information Technology. Warsaw, Poland 03 - 05 Apr 2017 London South Bank University.
Reducing the need for guesswork in multiple-choice tests
Bush, M (2014). Reducing the need for guesswork in multiple-choice tests. Assessment and Evaluation in Higher Education. 40 (2), pp. 218 - 231.
A multiple choice test that rewards partial knowledge
Bush, M (2001). A multiple choice test that rewards partial knowledge. Journal of Further and Higher Education. 25 (2), pp. 157 - 163.
Quality assurance of multiple-choice tests
Bush, M (2006). Quality assurance of multiple-choice tests. Quality Assurance in Education. 14 (4), pp. 398 - 404.