Farai is Associate Dean (Education & Students) in the University of Sussex Business School and Professor in the Department of Economics. She has several years’ higher education teaching experience in statistics, development economics and other applied economics topics. She has also worked for several international development agencies in the past.

There is increased emphasis in the UK higher education sector on measuring the impact of the education provided by universities to students on acquiring the knowledge, skills, and other competencies outlined in degree programmes. In 2023 the Office for Students asked education providers to present what ‘educational gains’ they intend their students to achieve, what support they offer students to achieve them, and what evidence they have that students are succeeding in achieving these. While the Teaching Excellence Framework (TEF) focuses on measures of continuation, completion and progression, educational gain also encompasses areas such as knowledge, skills, personal development and work readiness. However, the definition of educational gain is quite open-ended and leaves room for providers to conceptualize and articulate their interpretation of it in practice.

Accreditation is an important process for Business Schools globally. As part of AACSB (Association to Advance Collegiate Schools of Business) accreditation Business Schools must demonstrate they have a systematic process of Assurance of Learning (AoL). Assurance of learning (AoL) is about demonstrating, through assessment processes, that students achieve learning expectations for the programs in which they participate. It involves the use of robust, systematic and sustainable assessment processes designed to improve the learning of students. It is about process improvement and can also be the key driver of curriculum change.

Curriculum alignment

One of the early steps in the Assurance of Learning (AoL) process is curriculum alignment, where school learning/competency goals and course[1] learning objectives/outcomes are mapped on the curriculum. The focus here is on the common learning experience of students enrolled on the course. Curriculum alignment is important as the mission of the school (that is, what the school does) must align with the education the school offers. Hence, it is crucial to ensure that the school’s competency goals (e.g., sustainability, responsible leadership, collaboration) are reflected in the curriculum in a manner that allows students to develop that skill, knowledge, or attitude.

Step 1: Conceptual

An important initial step of the AoL process is articulating the overall competency goals of the school. For example, the University of Sussex Business School (USBS) has five competency goals which stem from its mission statement, namely:

1. Demonstrate appropriate discipline-specific knowledge using relevant methods and technologies
2. Work effectively in a team
3. Be responsible students and citizens
4. Communicate effectively with different audiences
5. Demonstrate the ability to work independently and apply critical thinking skills to develop innovative solutions

We expect the education we offer our students to enable them to acquire the above competencies by the time they graduate.

Step 2: Operational statements

The USBS offers many courses (i.e., degree programmes). Each course has its own learning outcomes/objectives. As part of AoL course learning objectives are mapped to the school competencies outlined in step 1 making explicit the curriculum alignment, that is, the relationship between the Business School’s overall goals and objectives and what is offered to students at the course level. A course typically has additional bespoke learning objectives that may not necessarily map one-to-one with school competencies which serve to differentiate the differences across courses.  

Step 3: 1st Measurement (opening the loop)

The next step of AoL is measuring whether we have delivered on our course learning objectives, that is, have our students acquired the course/degree level competencies we promised to deliver? Thus, an important aspect of AoL is to have a benchmark of what constitutes a course cohort having satisfactorily met the learning outcomes. For example, 80% of students achieving a pass mark on a course learning outcome at first attempt can be considered satisfactory performance. ‘Exceeding’ and ‘meeting’ the learning outcome are also differentiated. The former refers to a distinction mark while the latter refers to any pass mark below distinction. The benchmark for satisfactory performance does not have to be generic across courses. What matters is the rationale behind the benchmark.

Direct measures

Assessments are a key indicator of the extent to which students have learned what we taught them. To determine course cohort performance on a course learning outcome select core module assessments are mapped to each course learning objective. In some cases, there may be capstone assessments offered at the course level which align with specific course learning outcomes. In addition, either formative or summative assessments could be used to measure learning outcome performance, or a combination of the two. Where there is no suitable core module, optional modules that are representative of the course cohort can be used instead.

In practice a whole assessment component can be a suitable direct measure.  For example, an essay assessment can be a suitable measure for the learning outcome “Demonstrate an advanced understanding of management information systems using a range of concepts, theories and technologies”. However, in other cases, only certain aspects of the essay may be relevant to ascribe to a particular learning outcome. In this case, the marker must distinguish within the marking what parts of the assessment relate to the course learning outcome. For example, a dissertation may be a suitable assessment to measure the following learning outcome: “Understand the role of ethics in the evolution of innovation, change and contemporary issues in management”. However, it may not be appropriate to ascribe the whole dissertation mark to this learning outcome. Rather, only a component of it (e.g., 10 marks out of 100) may be related to that learning outcome. This implies the marking rubric must capture that component. Moreover, a student failing the overall dissertation may have not failed the course learning outcome as they may have achieved a pass mark for the component associated with the course learning outcome, and vice versa. It is important to understand that the performance here is in relation to the course learning outcome which may be attached to a whole module, an assessment component, or an assessment sub-component. Also, a big part of using assessments is aggregating the data at course level for assessments shared across different courses. This is to ensure cohort performance is captured separately for each course.

Indirect measures

The extent to which course learning outcomes have been met can also be measured using qualitative metrics (e.g., employer surveys, NSS scores, module evaluation questionnaires, alumni surveys, advisory board focus group discussions, etc.), referred to as indirect measures. Indirect measures can be a useful complement to direct measures and if used appropriately can provide valuable explanations to the findings from the quantitative results. Caution would need to be taken to ensure any sampling procedure for collecting qualitative data yields a representative sample.

This round of data collection is referred to as “opening the loop” in AoL language.

Step 4: Using the data to improve student learning

Where the target has not been met, it can be investigated why this is the case and what the right ‘intervention’ to achieve improvement is. There are two types of improvement: (i) process improvement (e.g., improving how to teach/assess, when to teach/assess; ‘systems’, etc.), and (ii) curriculum improvement (i.e., improving the syllabus, content, skills, knowledge, and competencies taught).

AoL is about being improvements oriented rather than compliance oriented. It is not a data collection project but a data usage project. That is, how can the data we have collected be used to help improve the learning of students. How can the data be acted on?

If students have not developed the competencies faculty thought they would have from the curriculum taught, faculty can reflect and develop learning experiences that can be used to improve student performance on the learning goals. It may take some trial and error, but over time students can improve their skills in specific competencies because of thoughtful, data-driven curriculum development and management. The objective of AoL is to assure student learning.

Step 5: 2^nd Measurement: closing the loop

This step involves a second data collection exercise akin to that described in step 3, e.g., one year later. From this second round of data collection, by looking at the outcomes it can be determined whether the interventions in step 4 achieved the desired effect. That way the loop will have been closed by having two data points (measurement before and after intervention). “Closing the loop” does not imply improvement in performance has been achieved. It is simply about having two data points to compare.

In the case where no intervention was required when the loop was opened, we can assess whether this is still the case at the point of closing the loop. This process repeats over time in that closing the loop is akin to opening the loop for the next period (see illustration below).

Continuous improvement process

Using the approach discussed above, we can continuously monitor our courses and the extent to which course learning outcomes are being met and how effective interventions are. We can also learn what we are doing ‘well’ and find out how and why this is. The data can also help us identify  whether there is a need to review learning outcomes to make them more ‘challenging’. For example, if we are continuously exceeding targets on the same learning outcome, we may need to adjust the benchmark. Or, we may decide to revise the learning outcome to offer a new competency to our students given we now have consistent satisfactory performance on the existing competency.

The AoL process is a continuous improvement process. The gap between opening a loop and closing a loop can be anything up to a six year gap. In the USBS we have adopted a one year gap for now, until we gain enough traction to widen the gap. Eventually this becomes a ‘self-driving’ process, enabling us to manage our curriculum effectively. The objective is to have a culture around what our students are learning, how we improve that learning, and how we work together to make that happen.

Effective AoL should lead to an improvement in student learning and raise the quality of graduates. AoL can also go a long way in addressing the intensifying pressure to develop data-driven responses to public demands for justification of investment in higher education.

Lastly, it is important to note that most educators already undertake the AoL exercise as part of their responsibilities in teaching and assessing students, making improvements based on past performance, and reflecting on current practice to inform future teaching and assessment. The AoL framework enables these processes to be captured in a more systematic, robust and sustainable manner. It also provides a holistic view at the course level and facilitates continuous curriculum and process management improvement.

---

[1] In this article, course refers to a degree programme.