The quiz tool in Canvas is fantastic for creating multiple choice quizzes. There are currently two quiz tools available within Canvas, we currently recommend using the Classic quiz engine for graded assessment rather than the New quiz tool due to a lack of parity in features.

Multiple choice quizzes can be used to measure a wide range of knowledge, skills and competencies with your students and can be used to measure higher order skills such as problem solving and evaluation. Due to their nature they can also be an efficient manner of assessing students (as well as students being able to check their own understanding and identify their knowledge gaps) and interpreting results as a large number of question types within the Canvas quiz engines can be automatically marked by the system, removing the need for manual marking.

To define some terminology that is used within multiple choice quizzes, quizzes are composed of what are known as ‘items’, with each item being made up of a Stem and various potential answers the term Stem refers to the question itself, including any context or other content that may be related to the student within the question, then you have potential answers and correct answers, with the term ‘distractors’ referring to quiz questions that are incorrect. Below are some ideas incorporating best practices around designing your multiple choice questions.

Structure your items around learning outcomes

It’s important to attempt to align your quiz items with appropriate learning objectives, considering what it is that students are trying to achieve by taking the quiz, i.e. is it simply to assess their knowledge retention and basic recall, checking their knowledge comprehension or their ability to evaluate certain concepts or are you trying to achieve something else entirely? Having a number of multiple choice items that all focus on the same learning outcome can increase the reliability of assessment. The possibility for students to tackle a number of items in a quicker fashion than say an essay allows you to assess a broader range of your module material so use this to your advantage.

Avoid trick items 

Avoid the use of tricks such as ambiguous phrasing, double negatives, multiple options that look very similar or information unrelated to the required skills or knowledge necessary to answer the question. If trick items are used either accidentally or intentionally students are not assessed on their knowledge but rather on their ability to work out the ‘trick’ of the question. Students also report an unfavourable  view of these types of questions.

For a further exploration of what defines a trick question please see ‘An Empirical Study on the Nature of Trick Test Questions.’¹

Keep language and vocabulary simple

Keep the language of the question as simple as possible.  Simplified language reduces the influence of reading ability allowing you to better assess something else from your students (unless the purpose of the assessment is reading ability. Avoid excessive wording or any irrelevant information, keeping Stems short and succinct. Try to word Stems positively rather than negatively and if you do need to use negative language such as ‘not’ or ‘no’ then emphasise them in bold text to help students read the question correctly.

All these tips are useful to ensure your questions are as accessible as possible and that you are not putting students with reading accessibility needs or those for who English is a second language at a disadvantage².

Write good distractor answers 

When writing distractor answers focus on those that will sound plausible based on common errors or misconceptions in their understanding. Avoid making your distractions  sound far-fetched or dubious. It’s also a good idea to ensure all your distractor answers are of a similar length and language.  

Research seems to suggest that having three possible answers, one correct answer and two distractors, is best in most cases³. Writing more distractors can make a quiz harder for students and should be encouraged if possible, but the jump in difficulty from two distractors to three or four appears to be small and it can be hard to write more than two good distractor answers for a question, quality over quantity might therefore be recommended. 

Support and resources

In conclusion I hope some of the recommendations here will help you to write more effective multiple choice questions. If you want to start creating quizzes the Canvas guides below will help, but if you are teaching at Sussex and you’d like help or guidance with creating multiple choice questions then please do get in touch with TEL at TEL@sussex.ac.uk

Some Canvas guides on quizzes

• What quiz types can I create in a course?
• What options can I set in a quiz?
• How do I create a quiz with individual questions?
• How do I create a quiz with a question group to randomize quiz questions?
• How do I create a Multiple Choice quiz question?
• How do I create a Multiple Answers quiz question?
• How do I create a question bank in a course?
• How do I view student results in a quiz?

References

1. Roberts, Dennis M. “An Empirical Study on the Nature of Trick Test Questions.” Journal of Educational Measurement 30, no. 4 (1993): 331–44. http://www.jstor.org/stable/1435229.
2. Abedi, Jamal, Carolyn Huie Hofstetter, and Carol Lord. “Assessment Accommodations for English Language Learners: Implications for Policy-Based Empirical Research.” Review of Educational Research 74, no. 1 (2004): 1–28. http://www.jstor.org/stable/3516059.
3. Kilgour, J.M., Tayyaba, S. An investigation into the optimal number of distractors in single-best answer exams. Adv in Health Sci Educ 21, 571–585 (2016). https://doi.org/10.1007/s10459-015-9652-7