The culture change of the REF 2028

Many aspects of the proposed new look to the UK’s Research Excellence Framework (REF) are welcome and to be applauded. As a veteran of the REF and its predecessors going back to the early noughties, I can say that the research assessment process has evolved almost beyond recognition.

Particularly welcome is evolution of the focus away from the selection of individuals with a certain number of ‘outputs’ to the unit – I remember the stress and distress of the days when staff were informed formally whether they were submitted or not, and the fear of the impact this would have on careers and livelihoods.  When this was initially relaxed in REF 2014, an elaborate and somewhat intrusive process was introduced whereby staff could be exempted from submitting four outputs if they could provide evidence of personal circumstances that prevented them from producing four outputs within the REF census period. Whilst further clarification on aspects of the new proposals is needed, the emphasis on the unit rather than the individual is a step in the right direction in my view.

I also welcome the increased emphasis on the research environment and culture, and I look forward to seeing the outcome of the commissioned work to develop appropriate indicators, and of the consultation that has been launched into this aspect of the new system. UKRIO will be responding to this consultation and encourage others to do so as well. It was a surprise and disappointment that such an important change was not included in the initial consultation.

I have two niggling worries. The first is that it will be very difficult to develop appropriate and comprehensive indicators of research culture in the rich and varied research environments that we are fortunate to have in the UK (to declare an interest, I work in a small, specialist institution as well as for UKRIO). Work has been carried out on this in the past, for example ‘Indicators of Research Integrity – An Initial Exploration of the landscape, opportunities and challenges’, which highlights some of the challenges ahead in the development of a comprehensive set of indicators that will work well for all institutions and disciplines.

The second is that in the process of attempting to assess research culture, something will change. Institutions are doing great work in this area and the enhancing research culture funding that has been available for the last couple of years in parts of the UK has supported this work. Quite apart from the nuts and bolts of how the assessment will be achieved, I believe there is a danger that the assessment process itself could lead to a change in mindset, from being focussed solely on developing and gradually improving your research culture to one where there is an awareness that the work being done will be judged and measured and a rating put on it, and that will impact the REF outcome for the institution and the funding resulting from it.

That could change what people do and how they do it. It could bring in an element of competition that by-and-large has not been present before, resulting in less sharing of good practice and resources. This is not to say that work on research integrity and research culture should not be subject to review and impact measurement, but there is a difference between self-evaluation and improvement, and external assessment that impacts on future reputation and funding. Such incentives can have mixed effects, as shown by a landscape study that UKRIO carried out with partner organisations.

An immutable feature of the REF that will not change is its ability to have unintended consequences and to impact institutional behaviour in ways that are not foreseen when the rules are drawn up. I hope that a version of the ‘Observer Effect’ does not occur following the introduction of a formal assessment of research culture.

A healthy research culture safeguards and enhances quality and ethical standards, and thereby helps ensure trust in research. I and my colleagues at UKRIO look forward to responding to the consultation on this key aspect of the REF and using our unique experience and perspective to inform the process of its development.

Please note that since this note was published, the REF has been rescheduled to 2029.

Written by Nicola Sainsbury, Research Integrity Manager at UKRIO and Research Manager at Rose Bruford College of Theatre and Performance.

Published 17th of October, 2023.