Benchmarking

Benchmarking is the process of verifying standards against peers or external authoritative criteria.  Courses are benchmarked against (1) similar courses at peer institutions, (2) disciplinary standards and (3) professional or accreditation standards.

Whose standards?

Standards for university courses are a shared responsibility between the institutions that deliver courses and the discipline communities that develop disciplinary knowledge and skills.

Institutions are responsible for the quality of learning and teaching in their courses.  Disciplines are responsible for creating the body of knowledge and ways of thinking and doing that define a discipline. Every course design needs both!

Benchmarking methods

Introduction of the Higher Education Standards Framework requirement for external referencing has prompted several new benchmarking approaches in addition to long-standing accreditation processes. Prof Kerri-Lee Krause and Prof Liz Deane recently published a review of recent projects which benchmark or calibrate assessment tasks between institutions.

1. Professions that require registration for practice usually have well-developed disciplinary standards, and processes to ensure regular benchmarking through accreditation. Some science disciplines have formal accreditation processes (Royal Australian Chemical Institute (RACI)) but most do not.
2. Benchmarking between institutions across multiple disciplines has been established amongst sub-groupings of Australian Universities (Go8 Quality Verification Scheme, IRU Academic Calibration Project) and as a broader sector initiative funded through the Office for Learning and Teaching (Peer Review of Assessment Network).
3. Discipline standards have been developed through a series of projects funded through the ALTC/OLT. Some disciplines have established national schemes to support cross-institutional benchmarking: Achievement Matters (Accounting).  Discipline-level moderation is in development for benchmarking in Chemistry.

Rigorous benchmarking with peer reviewers is a significant investment.  The UK peer review scheme run by the UK Quality Assurance Agency (QAA) uses visiting external peer reviewers and is an interesting international comparison.  The UK HE quality system is now under review by the UK Government.

Majors or courses?

The Bachelor of Science and other generalist degrees must manage benchmarking across a diverse range of (sub)disciplines. This challenge varies greatly between disciplines: biology and biomedical sciences span many professional associations, whereas chemistry and physics coalesce around one major body for each.

Institutions may elect to benchmark at the level of a major (sub-discipline) or a whole degree (eg Bachelor of Science). The ACDS has created a comparison of Science TLOs and disciplines that lines up outcomes from each sub-disciplinary statement against the parent statements from the Science TLOs to assist with translation between majors and courses.

Reviewing

Research into review and moderation demonstrates the potential for variation and inconsistency between reviewers, even where they are considered discipline experts. Moderation of samples of real student work by groups of potential reviewers allows calibration of academic judgement.  Prof Royce Sadler argues that this is essential as assessing student achievement cannot be reduced simply to a scale of grades or a marking rubric.

Calibration of reviewers was explored successfully by the Achievement Matters project for Accounting standards. It is an example for other disciplines in building expertise for reviewers that also builds discipline understanding.

Training for reviewers for benchmarking in Australia is not consistent in institutions and disciplines and often depends on project funding. The roll-out of the HE Standards suggests we need to do more!