Multiple Choice Questions: Best Practices in Item Construction

Step One: Before Test Construction Begins

  • Define learning objectives to be assessed by the test
  • Construct a 2X2 matrix to track learning objectives and multiple choice questions (see example of Test Construction Matrix).
  • Plan for a minimum of two to three questions for each learning objective.  The more questions the more reliable the information about student performance is likely to be.

Step Two: Prepare Item Stem.

Always end the item stem with a question.  Do not ask a student to find an answer that completes an unfinished sentence.

  • Be succinct by providing only those details that pertain directly to the question being asked.
  • Clearly state the problem or question. Make sure that differing interpretations of the same question are not likely.
  • Use positive wording.   Avoid questions like “Which of the following treatments for condition X is likely not to be helpful in alleviating symptom Y?”

Step Three: Construct Alternatives.

  • Make all alternatives roughly similar in length.
  • Make sure there is correct grammar and spelling of each alternative choice.
  • Make sure there is only one correct answer.  Pilot test among  a few colleagues to ensure this.
  • Avoid extremes like “never, always, and only.”
  • Avoid “all of the above” as an alternative.
  • Make sure alternatives are mutually exclusive of one another.
  • Do not adhere to a strict rule on number of alternatives.  Sometimes three or four are enough because coming up with more results in silly or non-plausible distractors.  However, number of alternatives is positively correlated with difficulty level of test. If you want to encourage students in a relatively low stakes testing situation, make it easier by providing fewer alternatives from which to choose.  If you want to minimize the possibility of guessing in a high stakes situation, use five or even more alternatives if possible.

Step Four:  Avoid these mistakes.

  • Don’t give the answer to a question in another question.
  • Make sure to distribute keyed responses over all possible answer choices, i.e, strive to have approximately equal numbers of a’s, b’s, c’s and d’s as the keyed response .
  • Try not to use distractors that are clearly not plausible, e.g, Bill Clinton as a distractor to the question “Who was the first president of the United States?”

Step Five:   Follow up after test is administered and scored

  • Use item analysis to revise and weed out bad questions.