Pick-N multiple choice-exams: a comparison of scoring algorithms

Bauer, Daniel; Holzer, Matthias; Kopp, Veronika; Fischer, Martin R (2011). Pick-N multiple choice-exams: a comparison of scoring algorithms. Advances in health sciences education, 16(2), pp. 211-221. Springer 10.1007/s10459-010-9256-1

[img] Text
2011_Bauer_AHSE_PickN.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (199kB) | Request a copy

To compare different scoring algorithms for Pick-N multiple correct answer multiple-choice (MC) exams regarding test reliability, student performance, total item discrimination and item difficulty. Data from six 3rd year medical students' end of term exams in internal medicine from 2005 to 2008 at Munich University were analysed (1,255 students, 180 Pick-N items in total). Scoring Algorithms: Each question scored a maximum of one point. We compared: (a) Dichotomous scoring (DS): One point if all true and no wrong answers were chosen. (b) Partial credit algorithm 1 (PS(50)): One point for 100% true answers; 0.5 points for 50% or more true answers; zero points for less than 50% true answers. No point deduction for wrong choices. (c) Partial credit algorithm 2 (PS(1/m)): A fraction of one point depending on the total number of true answers was given for each correct answer identified. No point deduction for wrong choices. Application of partial crediting resulted in psychometric results superior to dichotomous scoring (DS). Algorithms examined resulted in similar psychometric data with PS(50) only slightly exceeding PS(1/m) in higher coefficients of reliability. The Pick-N MC format and its scoring using the PS(50) and PS(1/m) algorithms are suited for undergraduate medical examinations. Partial knowledge should be awarded in Pick-N MC exams.

Item Type:

Journal Article (Original Article)

Division/Institute:

04 Faculty of Medicine > Medical Education > Institute for Medical Education
04 Faculty of Medicine > Medical Education > Institute for Medical Education > Education and Media Unit (AUM)

UniBE Contributor:

Bauer, Daniel

Subjects:

600 Technology > 610 Medicine & health

ISSN:

1382-4996

Publisher:

Springer

Language:

English

Submitter:

Daniel Bauer

Date Deposited:

22 Aug 2016 14:10

Last Modified:

02 Feb 2018 09:30

Publisher DOI:

10.1007/s10459-010-9256-1

PubMed ID:

21038082

Web of Science ID:

000288997500006

BORIS DOI:

10.7892/boris.86180

URI:

https://boris.unibe.ch/id/eprint/86180

Actions (login required)

Edit item Edit item
Provide Feedback