Relating item analyses of multiple-choice questions (MCQ) to the test score and pass/fail determination
Refereed conference paper presented and published in conference proceedings


Full Text

Other information
AbstractBackground: This study examined the item analysis of 131 MCQ tests used in medicine year 1 to 3 over eight years. The aims were to determine how the various parameters were related and in addition, to the overall test performance, thus forming the basis to target improvements in item writing and selection.
Method: Over 6000 MCQ from 6-8 system-based courses in each medical year were analysed for their difficulty level, discriminating power and distractor efficiency. These parameters were further related to the mean and standard deviation of the test scores, and the failure rates at 50%.
Results: The mean test score was given by averaging the difficulty level of test items. Higher distractor efficiency of
test items was associated with significantly higher discriminating power and difficulty level. The standard deviation (SD) of test scores showed a highly significant correlation to the discriminating power of test items, with the latter dropping off at the high and low ends of difficulty level. Item analysis provides information on the quality of test items. It remains to be determined whether these parameters would show significant changes with repeated use. Preliminary study showed no consistent variations in the distractor efficiency, difficulty level and selection percentage of different options when the test items were re-used. For the test score, the mean is determined by the difficulty level of test items while the standard deviation (SD) is related to the discriminating power. These are important determinants of the failure rate irrespective of whether the pass mark is fixed or established by standard setting (e.g. referenced against the best performing students as in the Cohen method).
Conclusion: The item analysis would provide the necessary information for the identification of poor quality test items
to be eliminated or reconstructed. It also allows a good mix of MCQ to be selected for producing more consistent
and better quality test papers.
Acceptance Date27/04/2018
All Author(s) ListAU Chak Leung Simon
Name of ConferenceAMEE 2018 (An International Association for Medical Education) Annual Conference
Start Date of Conference25/08/2018
End Date of Conference29/08/2018
Place of ConferenceBasel
Country/Region of ConferenceSwitzerland
Year2018
Month8
LanguagesEnglish-United Kingdom

Last updated on 2020-04-05 at 11:28