Recommended Practice Before Test Construction Begins
- Define learning objectives to be assessed by the test
- Construct a 2X2 matrix to track learning objectives and multiple choice questions (see example of Test Construction Matrix).
- Plan for a minimum of two to three questions for each learning objective. The more questions the more reliable the information about student performance is likely to be.
Guidelines for Preparation of Item Stems
- Always end the item stem with a question. Do not ask a student to find an answer that completes an unfinished sentence.
- Be succinct by providing only those details that pertain directly to the question being asked.
- Clearly state the problem or question. Make sure that differing interpretations of the same question are not likely.
- Use positive wording. Avoid questions like “Which of the following treatments for condition X is likely not to be helpful in alleviating symptom Y?”
Guidelines for Construction of Alternatives
- Make all alternatives roughly similar in length.
- Make sure there is correct grammar and spelling of each alternative choice.
- Make sure there is only one correct answer. Pilot test among a few colleagues to ensure this.
- Avoid extremes like “never, always, and only.”
- Avoid “all of the above” as an alternative.
- Make sure alternatives are mutually exclusive of one another.
- Do not adhere to a strict rule on number of alternatives. Sometimes three or four are enough because coming up with more results in silly or non-plausible distractors. However, number of alternatives is positively correlated with difficulty level of test. If you want to encourage students in a relatively low stakes testing situation, make it easier by providing fewer alternatives from which to choose. If you want to minimize the possibility of guessing in a high stakes situation, use five or even more alternatives if possible.
Mistakes to Avoid
- Don’t give the answer to a question in another question.
- Make sure to distribute keyed responses over all possible answer choices, i.e, strive to have approximately equal numbers of a’s, b’s, c’s and d’s as the keyed response .
- Try not to use distractors that are clearly not plausible, e.g, Bill Clinton as a distractor to the question “Who was the first president of the United States?”
Recommended Practice After Test is Administered and Scored
- Use item analysis to revise and weed out bad questions.