Skip to main content

Table 2 Prevalence of item flaws in 204 MCQs from 18 MOOCs

From: The future of online testing and assessment: question quality in MOOCs

Item Flaw Number detected % of MCQs
Outmoded item formats
 True / false format 22 10.8%
 Overly complex, or K-type, questions 17 8.3%
 Fill in the blank 4 2.0%
Question ambiguity or obscurity
 Gratuitous information in stem 0 0.0%
 Ambiguous or unclear information 19 9.3%
 Unfocussed stem 11 5.4%
 Absolute terms 30 14.7%
 Vague frequency terms 5 2.5%
 Negatively worded stem 21 10.3%
Structural or logical flaws
 Problem is in the options, not in the stem 6 2.9%
 Logical cues in stem & correct option 5 2.5%
 Word repeats in stem & correct answer 8 3.9%
 Longest option is correct See inline text
 Implausible distractors 15 7.4%
 More than one, or no correct answer 18 8.8%
 Use of “all of the above” 2 1.0%
 Use of “none of the above” 2 1.0%
 Option order / position of the correct option See inline text
 Convergence cues 9 4.4%