Construction of multiple choice tests

One important consideration when using multiple-choice tests concerns the size of the student group. For large numbers, it is important to keep the correction time under control. The construction of ‘good’ multiple-choice questions, however, is not an easy task. In this tip, we examine the points that should be considered in the construction and formulation of a multiple-choice question. The following questions can help you to arrive at a well formulate multiple-choice question:

Formulation

• Is the formulation of the question concise and neutral, without any unnecessary information? Unnecessary information generates unnecessary confusion.
   
• Are negatives (e.g. ‘not’ or ‘no’) emphasised, and does the question contain double negatives? For example, do not ask: In which cases is it incorrect to claim that the government does not need to carry out a motion of the House of Representatives?
   
• Does the selected formulation in the root of the question contain an unintended clue to the right answer? For example:

The statement ‘Plus fait douceur que violence’ comes from the writer:
a) Homer
b) Goethe
c) Hermans
d) La Fontaine

Problem: the French phrase in the root of the question points towards Answer d.

• Does the question contain words like ‘always’ or ‘never’? Such words often provide unintended clues to the right answer: ‘always’ is usually wrong.
   
• Is the construction of the question overly complicated? The question should not assess the extent to which students understand complex sentence structures, unless this is the teacher’s explicit intention.
   
• Opinions entail subjectivity. They appear primarily in sentences with verbs like ‘should’, ‘must’ and ‘ought’. Scholars within a given discipline are unlikely to be in complete agreement about the correct alternative. A neutral formulation should therefore be used, or a context should be stated (e.g. ‘according to Freud …’).
   
• Vague references in the root of the question or in one of the alternatives (e.g. many, sometimes, good, effective, important, large, small) specify boundaries that are too broad and therefore open to debate. Boundaries should therefore be stated in such terms as: ‘between 10% and 20%’, ‘in the majority of cases’ or ‘in more than 10%’). State a connection (e.g. by including a comparison): ‘… are determined more by … than they are by …’.

Options

• Avoid options like ‘none of the above’ or ‘all of the above’. Errors often occur in such options, particularly if the ‘summary alternative’ is the correct answer. The other alternatives are thus not 100% correct or incorrect.
   
• State the options positively (avoid negations) whenever possible.
   
• The suggested incorrect alternatives should represent errors that are frequently made by students. For example, incorrect answers to an open-ended question can subsequently be used as distractors for a multiple-choice question.
   
• Are all of the alternatives of the same length? If not, the longest alternative is often the correct one: this increases the likelihood of guessing the correct answer.
   
• Do the alternatives correspond well to the root of the question? If not, this reduces the legibility of the question and, in many cases, the only grammatically correct alternative proves to be the right answer.
   
• Do the alternatives resemble each other too closely, and do they overlap?

For example: A water buffalo is a:

a) mammal
b) cannon
c) sailing vessel
d) bovine

Problem: a) and d) overlap.
 

• Are all alternatives likely? If not, a question with four alternatives can easily become a question with three possibilities. This can increase the likelihood of guessing the correct answer.
   
• Assign a logical order to the alternatives (e.g. alphabetical or according to numerical size). This automatically results in a random distribution of the correct answers across the alternatives.

General characteristics of the test

• Are there enough questions to exclude the possibility of guessing the correct answer as much as possible?
   
• Check the level of difficulty. Try to arrive at a balanced test with a 60:40 ratio between relatively easy and difficult questions.
   
• Prevent time pressure: provide more than the estimated amount of time needed. Example: If you estimate that students will need two hours, allow 2.5–3 hours of testing time.
  ​

Want to know more?

Brame, C., (2013). Writing good multiple choice test questions. Retrieved from Vanderbilt University website (accessed August 23rd, 2019).

Burton, S., Sudweeks, R.,  Merrill, P. & Wood, B. (1991). How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty. Retrieved from Brigham Young University website.

Clay, B. (2001). Is This a Is This a Trick Question? Question? Question? A Short Guide to Writing Effective Test Questions. Retrieved from Kansas State University website.

Scully, Darina (2017). Constructing Multiple-Choice Items to Measure Higher-Order Thinking. Practical Assessment, Research & Evaluation, 22(4).