{"id":727,"date":"2024-03-05T11:59:06","date_gmt":"2024-03-05T11:59:06","guid":{"rendered":"https:\/\/blogs.sussex.ac.uk\/learning-matters\/?p=727"},"modified":"2026-03-26T10:19:13","modified_gmt":"2026-03-26T10:19:13","slug":"strategies-for-making-quantitative-assessments-more-ai-resilient","status":"publish","type":"post","link":"https:\/\/blogs.sussex.ac.uk\/learning-matters\/2024\/03\/05\/strategies-for-making-quantitative-assessments-more-ai-resilient\/","title":{"rendered":"Strategies for making quantitative assessments more AI resilient"},"content":{"rendered":"\n<div class=\"wp-block-image\"><figure class=\"alignleft size-large is-resized\"><img loading=\"lazy\" src=\"https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Portrait-819x1024.jpg\" alt=\"\" class=\"wp-image-733\" width=\"353\" height=\"442\" srcset=\"https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Portrait-819x1024.jpg 819w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Portrait-240x300.jpg 240w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Portrait-768x960.jpg 768w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Portrait-1229x1536.jpg 1229w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Portrait-1639x2048.jpg 1639w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Portrait-100x125.jpg 100w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Portrait-150x187.jpg 150w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Portrait-200x250.jpg 200w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Portrait-300x375.jpg 300w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Portrait-450x562.jpg 450w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Portrait-600x750.jpg 600w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Portrait-900x1125.jpg 900w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Portrait-scaled.jpg 2048w\" sizes=\"(max-width: 353px) 100vw, 353px\" \/><figcaption>Dr Myrna Hennequin<\/figcaption><\/figure><\/div>\n\n\n\n<p>In this case study, Dr Myrna Hennequin, Lecturer in Economics, shares her strategies for making online quantitative assessments more AI resilient.\u00a0<\/p>\n\n\n\n<p><strong>What I did&nbsp;<\/strong>&nbsp;<\/p>\n\n\n\n<p>I redesigned the online assessments for the module Quantitative Methods for Business and Economics: Maths (QMBE A). This introductory maths module for Foundation Year students is assessed by several Canvas quizzes. My goal was to create computer-markable exams that are varied, balanced and more resilient against academic misconduct through AI or collusion.&nbsp;<\/p>\n\n\n\n<p><strong>Why I did it<\/strong><strong>&nbsp;<\/strong><strong>&nbsp;<\/strong>&nbsp;<\/p>\n\n\n\n<p>Creating effective online assessments for quantitative modules poses several challenges. With the rapid rise of AI tools such as ChatGPT, I was concerned about cheating through the use of AI. At the same time, timetabling issues implied that the window during which the in-semester tests were available had to be extended from one hour to four hours, increasing the opportunity for collusion.&nbsp;<\/p>\n\n\n\n<p>In response to these issues, I aimed to make the quizzes more resilient. As the current free version of ChatGPT (GPT-3.5) does not allow for uploading images,\u00a0I ensured that each quiz included one or more questions involving graphs. <\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"alignright size-large is-resized\"><img loading=\"lazy\" src=\"https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Example_question.png\" alt=\"\" class=\"wp-image-734\" width=\"303\" height=\"547\" srcset=\"https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Example_question.png 491w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Example_question-166x300.png 166w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Example_question-100x181.png 100w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Example_question-150x271.png 150w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Example_question-200x361.png 200w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Example_question-300x542.png 300w, https:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2024\/03\/Example_question-450x813.png 450w\" sizes=\"(max-width: 303px) 100vw, 303px\" \/><\/figure><\/div>\n\n\n\n<p>I also took advantage of Canvas\u2019s built-in maths editor, which puts formulas in LaTeX format. It is currently not possible to directly copy these formulas into ChatGPT, which deters students from quickly copying an entire question including formulas. <\/p>\n\n\n\n<p>Another strategy I use is randomisation. Canvas allows for randomising questions as a whole by using question groups or setting up numerical questions with random numbers using formula questions. To prevent collusion, I ensured that nearly all quiz questions were randomised in some way. <\/p>\n\n\n\n<p>Lastly, I created a varied and balanced exam by using different question types. Even within the MCQ assessment mode, there are many ways to ask questions beyond the standard multiple-choice question consisting of a prompt and a set of answer options. I make use of a range of question types available in Canvas quizzes:&nbsp;<\/p>\n\n\n\n<ul><li>\u2018Formula questions\u2019 are numerical questions with randomised numbers.&nbsp;<\/li><li>\u2018Fill in multiple blanks\u2019 can be used to let students construct a step-by-step solution to a problem.&nbsp;<\/li><li>\u2018Multiple answers\u2019 are multiple-choice questions where there might be more than one correct answer.&nbsp;<\/li><li>\u2018Multiple dropdowns\u2019 are essentially fill-in-the-blank questions with predetermined answer options.&nbsp;<\/li><li>\u2018Matching\u2019 questions ask students to match a series of statements to a fixed set of answer options.&nbsp;<\/li><\/ul>\n\n\n\n<p>Combining various question types makes it possible to test different aspects of learning and get students to demonstrate depth of knowledge in a computer-markable format.&nbsp;<\/p>\n\n\n\n<p><strong>Challenges<\/strong>&nbsp;<\/p>\n\n\n\n<p>Designing robust quizzes and AI-proofing questions requires time and creativity. It can be time consuming to plan new approaches and to test how well ChatGPT responds to each type of question. However, the <a rel=\"noreferrer noopener\" href=\"https:\/\/community.canvaslms.com\/t5\/Instructor-Guide\/tkb-p\/Instructor\" target=\"_blank\">Canvas instructor guide<\/a> proved very useful as it introduced me to the different possibilities within Canvas quizzes.\u00a0<\/p>\n\n\n\n<p><strong>Impact and student feedback<\/strong>&nbsp;<\/p>\n\n\n\n<p>I always provide practice quizzes containing the same type of questions as the official assessments, so that students know what to expect and can prepare in advance. Students are happy with this combination of summative and formative assessments: on this year\u2019s mid-module evaluation, 21 students (out of 62 respondents) mentioned the practice quizzes and\/or the official online tests as a positive point in their written comments.&nbsp;<\/p>\n\n\n\n<p>In terms of results, the distribution of marks for the assessments this year was very much in line with previous years. This suggests that the marks were not inflated by the use of AI tools or other misconduct.&nbsp;<\/p>\n\n\n\n<p><strong>Future plans<\/strong>&nbsp;<\/p>\n\n\n\n<p>While my approach makes the quizzes more robust for the moment, AI tools are rapidly evolving. I would expect AI to get better at dealing with images (e.g. graphs) and mathematical formulas. This will be an additional challenge for the future.&nbsp;<\/p>\n\n\n\n<p><strong>Top 3 Tips<\/strong>&nbsp;<\/p>\n\n\n\n<ol><li>Think creatively: can you ask your quiz questions in a different way? Consider making use of the various question types available in Canvas quizzes.&nbsp;<\/li><\/ol>\n\n\n\n<ol start=\"2\"><li>Include images (e.g. graphs) to make it harder to answer questions using AI (*). Use Canvas\u2019s built-in maths editor for any formulas so that they cannot directly be copied. &nbsp;<\/li><\/ol>\n\n\n\n<ol start=\"3\"><li>Use randomisation to deter collusion. You can randomise questions as a whole by using question groups, or set up numerical questions with random numbers using formula questions.&nbsp;<\/li><\/ol>\n\n\n\n<p>(*) NB: Don\u2019t forget that some students may have visual impairments and require screen readers. You could try to add alt text to the image in such a way that the quiz question can still be answered, but without giving away too much information.&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this case study, Dr Myrna Hennequin, Lecturer in Economics, shares her strategies for making online quantitative assessments more AI resilient.\u00a0 What I did&nbsp;&nbsp; I redesigned the online assessments for the module Quantitative Methods for Business and Economics: Maths (QMBE<span class=\"ellipsis\">&hellip;<\/span><\/p>\n<div class=\"read-more\"><a href=\"https:\/\/blogs.sussex.ac.uk\/learning-matters\/2024\/03\/05\/strategies-for-making-quantitative-assessments-more-ai-resilient\/\">Read more &#8250;<\/a><\/div>\n<p><!-- end of .read-more --><\/p>\n","protected":false},"author":402,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"spay_email":""},"categories":[71758],"tags":[123707,202913,123700],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blogs.sussex.ac.uk\/learning-matters\/wp-json\/wp\/v2\/posts\/727"}],"collection":[{"href":"https:\/\/blogs.sussex.ac.uk\/learning-matters\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.sussex.ac.uk\/learning-matters\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.sussex.ac.uk\/learning-matters\/wp-json\/wp\/v2\/users\/402"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.sussex.ac.uk\/learning-matters\/wp-json\/wp\/v2\/comments?post=727"}],"version-history":[{"count":3,"href":"https:\/\/blogs.sussex.ac.uk\/learning-matters\/wp-json\/wp\/v2\/posts\/727\/revisions"}],"predecessor-version":[{"id":735,"href":"https:\/\/blogs.sussex.ac.uk\/learning-matters\/wp-json\/wp\/v2\/posts\/727\/revisions\/735"}],"wp:attachment":[{"href":"https:\/\/blogs.sussex.ac.uk\/learning-matters\/wp-json\/wp\/v2\/media?parent=727"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.sussex.ac.uk\/learning-matters\/wp-json\/wp\/v2\/categories?post=727"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.sussex.ac.uk\/learning-matters\/wp-json\/wp\/v2\/tags?post=727"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}