Results of the Quality Evaluation rounds

The Tertiary Education Commission (TEC) reports on the results of PBRF Quality Evaluations as part of its commitment to encouraging and rewarding excellent research in the tertiary education sector.

Interactive charts for all Quality Evaluation rounds

The final results of all four previous Quality Evaluation rounds (2003, 2006, 2012, and 2018) are available via the following two interactive charts. The charts allow tertiary education organisations (TEOs) and other stakeholders to see changes over time.

• Quality Evaluation results by TEO and subject area
• Quality Evaluation researcher demographics: results by subject area and demographic details (gender, ethnicity, age and employment status)

Things to note about the charts:

• The data is based on PBRF-eligible staff whose Evidence Portfolios were awarded a funded Quality Category (A, B, C, C(NE)). Figures are shown as full-time equivalent (FTE) weighted numbers and represented to two decimal places.
• The Pacific Research Panel was introduced for the 2018 Quality Evaluation, so there is no comparative data from previous rounds.
• The new and emerging researcher status was introduced in 2006, so there is no comparative data from the 2003 round.

Results for all four Quality Evaluation rounds by funded Quality Category, panel, subject area and nominated academic unit, as well as sector-wide demographics by subject area, are available via Ngā Kete. These are only available to TEOs and require an ESL-enabled log-in.

Infographics

The infographic below shows TEO participation across the four Quality Evaluation rounds (2003, 2006, 2012, and 2018).

• Quality Evaluation participation across four rounds (PDF 137 KB)

Quality Evaluation 2018

The final 2018 Quality Evaluation results were published in October 2019 and provided in the following online formats:

• Improving Research Quality: The results of the PBRF 2018 Quality Evaluation (PDF 1.1 MB)
  The report presents the results of the PBRF 2018 Quality Evaluation and provides analysis and comparisons with previous Quality Evaluation rounds. It provides an overview of the results, the average quality scores, the funding allocations and the outcomes of the complaints process
• background reports and infographics
• interactive charts for all Quality Evaluation rounds
• data visualisations (discontinued).

Background reports

Alongside Improving Research Quality: The results of the PBRF 2018 Quality Evaluation, there are two background reports:

• Report of the Moderation Panel and Peer-Review Panels (PDF 1.6 MB)
  Overview of the process each panel undertook through the assessment, analysis of the results within a panel context and panel recommendations.
• TEC Project Report (PDF 2 MB)
  Outline of the process the TEC undertook to implement the PBRF 2018 Quality Evaluation.

Infographics

The infographics below summarise each subsector’s participation in the PBRF 2018 Quality Evaluation.

• Institutes of technology and polytechnic sector infographic (PDF 98 KB)
• Private training establishment sector infographic (PDF 98 KB)
• University sector infographic (PDF 104 KB)
• Wānanga sector infographic (PDF 98 KB)

Infographic showing TEO participation across the four Quality Evaluation rounds (2003, 2006, 2012 and 2018):

• Quality Evaluation participation across four rounds (PDF 137 KB)

Audit Reports

The PBRF Quality Evaluation 2018 was supported by a comprehensive compliance audit programme. The objective of the audit programme was to provide assurance to the TEC that the guidelines for the 2018 Quality Evaluation were consistently and correctly applied by all participating tertiary education organisations. The audit process had two phases: process assurance and data evaluation.

KPMG was appointed as the auditor for the 2018 Quality Evaluation following a competitive procurement process. 

• KPMG PBRF 2018 Quality Evaluation Audit Report (PDF 479 KB) – March 2019
• PBRF Summary Report: Process Assurance (PDF 642 KB) – February 2018
• Audit Methodology for PBRF 2018 Quality Evaluation (PDF 923 KB)

FAQs

Frequently asked questions (FAQs) about the 2018 Quality Evaluation are available here:

• PBRF 2018 Quality Evaluation FAQs (PDF 374 KB)

Quality Evaluation 2003, 2006 and 2012

The Quality Evaluation 2012 final report was published in October 2013. The final report incorporates changes made based on the results of the complaints process, administrative corrections, and the presentation of information as requested by the sector. An overview of the results of the complaints process was included (Appendix 1). Indicative funding for 2013 PBRF allocations was also updated (Table 9.1).

• Evaluating Research Excellence ­– the 2012 Assessment (PDF 6.4 MB)
• Supplement to Appendices A and B (2003, 2006 and 2012 Quality Evaluations – TEOs by subject area results and nominated academic unit results (XLS 411 KB)
• Supplement to Appendix A (Part 1) – 2012 TEO, panel, and subject area results (XLS 50 KB)
• Supplement to Appendix A (Part 2) – 2012 nominated academic unit results (XLS 348 KB)
• Supplement to Appendix B (Part 1) – 2006 and 2003 TEO, panel, and subject area results (XLS 284 KB)
• Supplement to Appendix B (Part 2) – 2006 and 2003 nominated academic unit results (XLS 242 KB)
• Supplement to Appendix C – Contextual comparators for 2003, 2006 and 2012 Quality Evaluations (XLS 133 KB)

Peer-review panel and expert advisory group reports

Reports were developed by each of the 12 peer review panels and two expert advisory groups following the completion of the Quality Evaluation 2012. The reports provide information on their assessments and recommendations for the TEC.

• PBRF 2012 All Peer-Review Panel and Expert Advisory Group Reports (PDF 2.5 MB)

Quality Evaluation 2006

Most of the historical resources and publications relating to earlier PBRF cycles have been archived and are available on request. You can download the following documents.

• 2006 PBRF Evaluating Research Excellence – the 2006 Assessment (PDF 2.3 MB)
• PBRF TEO Results – All TEOs (XLS 43 KB)
• PBRF Panel Results – All Panels (XLS 37 KB)
• PBRF Subject-Area Results – All Subject Areas (XLS 50 KB)
• Peer Review Panels for the 2006 PBRF Quality Evaluation (PDF 92 KB)
• 2006 PBRF Guidelines (PDF 2.1 MB) – These guidelines were prepared to help participants in the 2006 processes for the PBRF.

Quality Evaluation 2003

• 2003 PBRF Evaluating Research Excellence: the 2003 assessment (PDF 2.4 MB)

Reviews of the PBRF – previous reports

When the PBRF was introduced in 2002, a three-phase evaluation strategy was also put in place. 

• Phase one started in August 2003 and focused on early indicators of the impacts of the design and implementation of the PBRF and the results of the 2003 Quality Evaluation.
• Phase two started in February 2008 and was an independent strategic review of the positive effects and unintended consequences of the PBRF on the sector. Its main focus was on how well the PBRF was accomplishing its primary goal, to encourage and reward research excellence in the tertiary education sector, and included information from the results of the Quality Evaluation 2006.

Phase three was a review by the Ministry of Education. It sought to build on the existing performance of the PBRF to identify ways in which it could be improved and included information from the implementation and results of the 2012 Quality Evaluation.

Here are the evaluation reports:

• Phase 1 Evaluation of the implementation of the PBRF and the conduct of the 2003 Quality Evaluation (PDF 4.7 MB)
• Final Report – Independent Review of the PBRF (PDF 788 KB)
• TEC's brief summary of the report (PDF 16 KB)

The Quality Evaluation 2018 was reviewed by an independent panel in 2019–2020. See the Ministry of Education's website for more information on the review, including the panel’s final report, and Cabinet’s decisions on changes to the PBRF.