![\"Photograph](\"http:\/\/blogs.sussex.ac.uk\/learning-matters\/files\/2021\/02\/Junko-Winch.jpg\")
Module Evaluation Questionnaires (MEQs) are an important source of student feedback on teaching and learning. They are also often relied upon as evidence cases for promotion for tutors. However, in their current form they suffer from low response rates reducing their usefulness and validity. Local practices have grown to address the need for feedback, but they are inconsistent year on year or across the University. Existing research on teaching evaluations indicates that there are a source of bias and suggests careful design of MEQs.<\/p>\n\n\n\n
The MEQ project<\/h2>\n\n\n\n
The MEQ project was undertaken to inform the University of Sussex’s policy and practice. The output was presented to the University\u2019s Surveys Group for the strategic direction of the University.<\/p>\n\n\n\n
Method<\/h2>\n\n\n\nResearch Question 1: Purpose of MEQs<\/strong>
Method:<\/strong> Literature Review<\/h3>\n\n\n\nA literature search was conducted using a total of 79 journal publications from the fields of psychology, education, business management, and sociology. The publication dates ranged from 1978 to 2020. The keywords used in the search were student evaluation of teaching (SETs)<\/em>, validity<\/em>, assessment<\/em>, and evaluation<\/em>. All selected journals underwent a content analysis.<\/p>\n\n\n\n
\n\n\n\nResearch Question 2: Analysis of MEQ\u2019s Seven Statements<\/strong>
Method:<\/strong> Qualitative<\/h3>\n\n\n\nThis stage involved a qualitative analysis of the seven core statements in the Module Evaluation Questionnaire (MEQ). The analysis focused on the following aspects:<\/p>\n\n\n\n
- The purpose of the University MEQ<\/li>
- Identified weaknesses<\/li>
- Suggestions for improvement<\/li><\/ol>\n\n\n\n
Literature review<\/h2>\n\n\n\n
The literature review revealed tutors’ and students’ biases related to MEQs. However, bias is a source of unreliability, which also threatens validity. Validity and reliability are defined in various terms, but for the purpose of this report, validity is defined as \u201cthe general term most often used by researchers to judge quality or merit\u201d (Gliner et al., 2009<\/a>, 102) and reliability as \u201cconsistency with which we measure something\u201d (Robson, 2002<\/a>, 101).\u00a0 <\/p>\n\n\n\nTutors related biases<\/h3>\n\n\n\n
The literature highlighted that tutor related biases must be taken into account:<\/p>\n\n\n\n
- Students who have learned more in class will receive higher grades and will naturally rate the professor more highly because of the knowledge they have gained on the course (Patrick, 2011<\/a>, 242).<\/li>
- Tutors who give higher grades receive better evaluation (Carrell and West, 2010<\/a>).<\/li><\/ul>\n\n\n\n
Student related biases<\/h3>\n\n\n\n
Student biases may arise from a wide range of factors, including the weather, time of day, personality traits, gender, racial stereotypes, the tutor\u2019s physical attractiveness, student anxiety, and more.<\/p>\n\n\n\n
The findings and recommendations<\/h2>\n\n\n\n1. The purpose of MEQs<\/h3>\n\n\n\n
MEQs have three purposes: institutional, teaching and academic promotion. To help to reduce the bias effects outlined in the literature, full MEQs and other teaching related data should be provided to promotion panels to avoid the cherry picking of comments or data by applicants. For example, quantitative data such as class average attendance rate, average, minimum and maximum marks as well as qualitative response analysis would help build a more accurate overall picture of the class.<\/p>\n\n\n\n
2. Analysis of MEQs<\/h3>\n\n\n\n
Students\u2019 biases mentioned in the literature may present difficulty in relying on MEQs as sole instrument. Furthermore, the current MEQ statements may confuse students due their contents and wording. <\/p>\n\n\n\n
Following points are suggested:<\/p>\n\n\n\n
- Purpose and goal of the questionnaire should be clearly stated. The purpose of the stakeholders should be taken into account when designing the MEQs to ensure that the intended MEQ purpose is achieved.<\/li>
- Some statements ask two questions in one statement. However, some students may not necessarily answer both questions, which affect validity.<\/li>
- Consideration should be given to the words such as \u2018satisfied\u2019 which might have different connotations depending on cultures and individuals.<\/li><\/ul>\n\n\n\n
Recommendations<\/h3>\n\n\n\n
Carefully developed MEQs have potential to offer valuable insights to all stakeholders. The primary recommendation is to undertake a staff-student partnership to agree the purpose of the MEQs and co-design a revised instrument that meets the stated purpose.<\/p>\n\n\n\n
Reflections<\/h2>\n\n\n\n
I have engaged this project as my Continual Professional Development and appreciate that it has given me various opportunities. For example, I was given an opportunity to write this blog. Furthermore, giving a presentation to the University Surveys Group reminded me of my doctorate viva, as the University Survey Group included Pro Vice Chancellor for Education and Students, Associate Dean of the Business School and the Deputy Pro Vice Chancellor of Student Experience. When answering questions from the University Survey Group, I learned how difficult it is to meet the needs of different perspectives and cultures. For example, I was asked a question from a quality assurance perspective, which was unexpected as I wrote this report from a teaching staff perspective. The University Survey Group also included Students\u2019 Experience team, which also made me consider another perspective involving MEQs. Furthermore, working with my colleague from the Business School made me realise the departmental\/academic discipline\u2019s cultural differences from where I am affiliated (School of Media, Arts and Humanities). Looking back, this was a very valuable experience for me and hopefully the institution.<\/p>\n\n\n\n
References<\/h2>\n\n\n\n
Carrell, S.E. and West, J.E. (2010) \u2018Does professor quality matter? Evidence from random assignment of students to professors\u2019, Journal of Political Economy<\/em>, 118, pp.409\u2013432. https:\/\/doi.org\/10.1086\/653808<\/a>
Gliner, J.A., Morgan, G.A. and Leech, N.L. (2009) Research methods in applied settings: An integrated approach to design and analysis<\/em>. New York: Routledge.<\/p>\n\n\n\nPatrick, C.L. (2011) \u2018Student evaluations of teaching: effects of the Big Five personality traits, grades and the validity hypothesis\u2019, Assessment & Evaluation in Higher Education<\/em>, 36(2), pp.239\u2013249. https:\/\/doi.org\/10.1080\/02602930903308258<\/a><\/p>\n\n\n\nRobson, C. (2002) Real world research<\/em>. 2nd edn. Oxford: Blackwell.<\/p>\n","protected":false},"excerpt":{"rendered":"by Junko Winch Module Evaluation Questionnaires (MEQs) are an important source of student feedback on teaching and learning. They are also often relied upon as evidence cases for promotion for tutors. However, in their current form they suffer from low…<\/span><\/p>\n
Method:<\/strong> Literature Review<\/h3>\n\n\n\n
A literature search was conducted using a total of 79 journal publications from the fields of psychology, education, business management, and sociology. The publication dates ranged from 1978 to 2020. The keywords used in the search were student evaluation of teaching (SETs)<\/em>, validity<\/em>, assessment<\/em>, and evaluation<\/em>. All selected journals underwent a content analysis.<\/p>\n\n\n\n This stage involved a qualitative analysis of the seven core statements in the Module Evaluation Questionnaire (MEQ). The analysis focused on the following aspects:<\/p>\n\n\n\n The literature review revealed tutors’ and students’ biases related to MEQs. However, bias is a source of unreliability, which also threatens validity. Validity and reliability are defined in various terms, but for the purpose of this report, validity is defined as \u201cthe general term most often used by researchers to judge quality or merit\u201d (Gliner et al., 2009<\/a>, 102) and reliability as \u201cconsistency with which we measure something\u201d (Robson, 2002<\/a>, 101).\u00a0 <\/p>\n\n\n\n The literature highlighted that tutor related biases must be taken into account:<\/p>\n\n\n\n Student biases may arise from a wide range of factors, including the weather, time of day, personality traits, gender, racial stereotypes, the tutor\u2019s physical attractiveness, student anxiety, and more.<\/p>\n\n\n\n MEQs have three purposes: institutional, teaching and academic promotion. To help to reduce the bias effects outlined in the literature, full MEQs and other teaching related data should be provided to promotion panels to avoid the cherry picking of comments or data by applicants. For example, quantitative data such as class average attendance rate, average, minimum and maximum marks as well as qualitative response analysis would help build a more accurate overall picture of the class.<\/p>\n\n\n\n Students\u2019 biases mentioned in the literature may present difficulty in relying on MEQs as sole instrument. Furthermore, the current MEQ statements may confuse students due their contents and wording. <\/p>\n\n\n\n Following points are suggested:<\/p>\n\n\n\n Carefully developed MEQs have potential to offer valuable insights to all stakeholders. The primary recommendation is to undertake a staff-student partnership to agree the purpose of the MEQs and co-design a revised instrument that meets the stated purpose.<\/p>\n\n\n\n I have engaged this project as my Continual Professional Development and appreciate that it has given me various opportunities. For example, I was given an opportunity to write this blog. Furthermore, giving a presentation to the University Surveys Group reminded me of my doctorate viva, as the University Survey Group included Pro Vice Chancellor for Education and Students, Associate Dean of the Business School and the Deputy Pro Vice Chancellor of Student Experience. When answering questions from the University Survey Group, I learned how difficult it is to meet the needs of different perspectives and cultures. For example, I was asked a question from a quality assurance perspective, which was unexpected as I wrote this report from a teaching staff perspective. The University Survey Group also included Students\u2019 Experience team, which also made me consider another perspective involving MEQs. Furthermore, working with my colleague from the Business School made me realise the departmental\/academic discipline\u2019s cultural differences from where I am affiliated (School of Media, Arts and Humanities). Looking back, this was a very valuable experience for me and hopefully the institution.<\/p>\n\n\n\n Carrell, S.E. and West, J.E. (2010) \u2018Does professor quality matter? Evidence from random assignment of students to professors\u2019, Journal of Political Economy<\/em>, 118, pp.409\u2013432. https:\/\/doi.org\/10.1086\/653808<\/a> Patrick, C.L. (2011) \u2018Student evaluations of teaching: effects of the Big Five personality traits, grades and the validity hypothesis\u2019, Assessment & Evaluation in Higher Education<\/em>, 36(2), pp.239\u2013249. https:\/\/doi.org\/10.1080\/02602930903308258<\/a><\/p>\n\n\n\n Robson, C. (2002) Real world research<\/em>. 2nd edn. Oxford: Blackwell.<\/p>\n","protected":false},"excerpt":{"rendered":" by Junko Winch Module Evaluation Questionnaires (MEQs) are an important source of student feedback on teaching and learning. They are also often relied upon as evidence cases for promotion for tutors. However, in their current form they suffer from low…<\/span><\/p>\n
\n\n\n\nResearch Question 2: Analysis of MEQ\u2019s Seven Statements<\/strong>
Method:<\/strong> Qualitative<\/h3>\n\n\n\nLiterature review<\/h2>\n\n\n\n
Tutors related biases<\/h3>\n\n\n\n
Student related biases<\/h3>\n\n\n\n
The findings and recommendations<\/h2>\n\n\n\n
1. The purpose of MEQs<\/h3>\n\n\n\n
2. Analysis of MEQs<\/h3>\n\n\n\n
Recommendations<\/h3>\n\n\n\n
Reflections<\/h2>\n\n\n\n
References<\/h2>\n\n\n\n
Gliner, J.A., Morgan, G.A. and Leech, N.L. (2009) Research methods in applied settings: An integrated approach to design and analysis<\/em>. New York: Routledge.<\/p>\n\n\n\n