Table 2 Prevalence of item flaws in 204 MCQs from 18 MOOCs
From: The future of online testing and assessment: question quality in MOOCs
Item Flaw | Number detected | % of MCQs |
|---|---|---|
Outmoded item formats | ||
True / false format | 22 | 10.8% |
Overly complex, or K-type, questions | 17 | 8.3% |
Fill in the blank | 4 | 2.0% |
Question ambiguity or obscurity | ||
Gratuitous information in stem | 0 | 0.0% |
Ambiguous or unclear information | 19 | 9.3% |
Unfocussed stem | 11 | 5.4% |
Absolute terms | 30 | 14.7% |
Vague frequency terms | 5 | 2.5% |
Negatively worded stem | 21 | 10.3% |
Structural or logical flaws | ||
Problem is in the options, not in the stem | 6 | 2.9% |
Logical cues in stem & correct option | 5 | 2.5% |
Word repeats in stem & correct answer | 8 | 3.9% |
Longest option is correct | See inline text | |
Implausible distractors | 15 | 7.4% |
More than one, or no correct answer | 18 | 8.8% |
Use of “all of the above” | 2 | 1.0% |
Use of “none of the above” | 2 | 1.0% |
Option order / position of the correct option | See inline text | |
Convergence cues | 9 | 4.4% |