Step One: Before Test Construction Begins
- Define learning objectives to be assessed by the test
- Construct a 2X2 matrix to track learning objectives and multiple choice questions (see example of Test Construction Matrix).
- Plan for a minimum of two to three questions for each learning objective. The more questions the more reliable the information about student performance is likely to be.
Step Two: Prepare Item Stem.
Always end the item stem with a question. Do not ask a student to find an answer that completes an unfinished sentence.
- Be succinct by providing only those details that pertain directly to the question being asked.
- Clearly state the problem or question. Make sure that differing interpretations of the same question are not likely.
- Use positive wording. Avoid questions like “Which of the following treatments for condition X is likely not to be helpful in alleviating symptom Y?”
Step Three: Construct Alternatives.
- Make all alternatives roughly similar in length.
- Make sure there is correct grammar and spelling of each alternative choice.
- Make sure there is only one correct answer. Pilot test among a few colleagues to ensure this.
- Avoid extremes like “never, always, and only.”
- Avoid “all of the above” as an alternative.
- Make sure alternatives are mutually exclusive of one another.
- Do not adhere to a strict rule on number of alternatives. Sometimes three or four are enough because coming up with more results in silly or non-plausible distractors. However, number of alternatives is positively correlated with difficulty level of test. If you want to encourage students in a relatively low stakes testing situation, make it easier by providing fewer alternatives from which to choose. If you want to minimize the possibility of guessing in a high stakes situation, use five or even more alternatives if possible.
Step Four: Avoid these mistakes.
- Don’t give the answer to a question in another question.
- Make sure to distribute keyed responses over all possible answer choices, i.e, strive to have approximately equal numbers of a’s, b’s, c’s and d’s as the keyed response .
- Try not to use distractors that are clearly not plausible, e.g, Bill Clinton as a distractor to the question “Who was the first president of the United States?”
Step Five: Follow up after test is administered and scored
- Use item analysis to revise and weed out bad questions.