Online Assessment: What is the best question type? How many options should students choose from?
This is very much a moot point. The extended matching questions (EMQs) originally espoused by Case and Swanson, are gradually being replaced within the MSD, and more widely in the medical sciences, by standard multiple choice questions (MCQs). This is in response to various papers suggesting that, while increasing the number of answers from which students choose can increase the difficulty of questions, there is very little loss in discrimination with significantly fewer options e.g Swanson et al., 2006.
The effect of decreasing the number of choices per item while lengthening the test proportionately is to increase the efficiency of the test for high-level examinees and to decrease its efficiency for low-level examinees
- Lord, F. M. (1977). Optimal number of choices per item—A comparison of four approaches. Journal of Educational Measurement, 14, 33– 38.
In fact, in a meta-analysis of previous research, Rodriguez suggests that in many cases, choosing between three options is most efficient as it minimises time taken to write questions, and for students to answer them, allowing a better coverage of the subject matter. The argument is that, in many questions, only two of the incorrect answer options are really distractors anyway, any additional options are eliminated immediately, effectively leaving a three-option MCQ. The same argument suggests that, although a five option MCQ might appear to give only a 20% likelihood of getting the right answer by chance, as long as students have sufficient ability and aren’t under time pressure - so are not guessing completely randomly, if there are only three plausible answers, the likelihood is actually 33%.
This may be an over-simplification, with its assumptions about the ability of examiners to write plausible distractors and the abilities of students to spot and discount the implausible, but it does suggest that the number of answer options should be chosen to match the question/subject matter/ability of the author/ability of the students, rather than arbitrarily deciding on a fixed value.
In practice in the MSD, five option MCQs now form the great majority of summative assessment questions while EMQs, multiple response questions (tick boxes), and MCQs with a different number of options make up the remainder.
If you only have 5 minutes: 10 Tips for Writing Great Multiple Choice Questions
At Oxford, there is also a regular seminar run by the Medical School on question writing.