Online Assessment: What is the best question type? How many options should students choose from?
This is very much a moot point. The extended matching questions (EMQs) originally espoused by Case and Swanson, are gradually being replaced within the MSD, and more widely in the medical sciences, by standard multiple choice questions (MCQs). This is in response to various papers suggesting that, while increasing the number of answers from which students choose can increase the difficulty of questions, there is very little loss in discrimination with significantly fewer options e.g Swanson et al., 2006.
In fact, in a meta-analysis of previous research, Rodriguez suggests that in many cases, choosing between three options is most efficient as it minimises time taken to write questions, and for students to answer them, allowing a better coverage of the subject matter. The argument is that, in many questions, only two of the incorrect answer options are really distractors anyway, any additional options are eliminated immediately, effectively leaving a three-option MCQ. The same argument suggests that, although a five option MCQ might appear to give only a 20% likelihood of getting the right answer by chance, as long as students aren’t under time pressure and guessing randomly, if there are only three plausible answers, the likelihood is actually 33%.
This may be an over-simplification, with its assumption about the ability of examiners to write plausible distractors, but it does suggest that the number of answer options should be chosen to match the question/subject matter/ability of the author rather than arbitrarily deciding on a fixed value.
In practice in the MSD, five option MCQs now form the great majority of summative assessment questions while EMQs, multiple response questions (tick boxes), and MCQs with a different number of options make up the remainder.
At Oxford, there is also a regular seminar run by the Medical School on question writing.