The week before Christmas saw Universities across the UK in a state of heightened anticipation. Santa, in the form of the REF results was due, and for a surprisingly large number of Universities it will continue to be the gift that keeps on giving, depending on which league table you want to believe.

For the rest of the world the REF (Research Excellence Framework) may be often viewed as an arcane, academic (in the pejorative sense of the word) absurdity. But we all love a league table, so designing a game for the supposedly brightest minds in the country to play with is irresistible. And when Universities ‘game’ these rules, then we’ll change them for next time and keep on playing.

Should this matter to the ‘real world’? Well yes for a couple of big reasons:

First, people pay attention to these league tables, without really knowing on what basis they are created. So the league table on which Universities scored the highest for their research, calculated as GPA (all submitted staff submit 4 papers each, these are then ranked by a panel of experts on a scale of 1-4, with 4 being world-leading, and a measure of research environment and research impact was also made on the same scoring system) is seen as the badge of research excellence.

So let’s take Cardiff University, which rose the GPA table to 6th from 22nd last time round (2008). Brilliant, great success. But let’s pause. This time round Cardiff chose to submit c700 staff to the REF, whereas last time they submitted c1000. The maximum number of staff they could have submitted was substantially higher. They’ve done nothing wrong by being very selective; they wanted to be at the top of this table, but doing so meant deciding to exclude many of their staff. Other Universities submitted even lower proportions in some cases.

What therefore do you think this league table is really showing? An alternative table using ‘research intensity’ (i.e. accounting for this proportionality/selectivity) shows Cardiff not 6th but 50th in the UK. Which league table is the better reflection of research? And which league table do you think Cardiff will be using in their promotional materials?

This is not to pick on Cardiff, as other Universities tied to play the same and different versions of the game, but perhaps did it less successfully or dramatically. All Universities, quite rightly, tried to manipulate their actions to get what they saw as the best result – and the system is designed to allow this.

Secondly, this time round Universities had to consider ‘impact’; not in an academic sense but on the real world. So impact case studies had to be submitted showing how research had been used by outside users such as companies or governments. A new task to ‘game’ and one that caused some angst for Universities, but seemed to have positive effects on some Universities and their positions. And so there’s another table on research impact rankings, so more choices for Universities to choose their ranking from and even more Universities claiming to be in the top 10 in the UK!

So what has this to do with retailing and Stirling? Stirling, as the University, did pretty well, rising on all tables and measures to be in the mid-40s in the UK. The Stirling Management School broke into the top 25 in the UK in a strong performance. Below that, at for example the subject level (e.g. retail) there is no disaggregation. On impact the Stirling Management School was 14th overall in the UK. That is a powerful endorsement of our impact on the world, by any measure.

One of the impact case studies we submitted was on the work of the retail group covering “retailing, retail planning and and town centres” in Scotland. It seems to have scored well and obviously personally I am pleased, not just in an impact case study sense but in reflecting on the work we’ve done over the last decade or so (and some of it reported as it happened in this blog).

Now, who knows what the impact assessment will be like next time? It changed things this time round so the rules will likely be amended and the ‘losers’ will be seeking to work out how they lost. But asking Universities and academics about their impact is likely to be here to stay. Thus for companies, organisations, charities, governments etc. there’s probably been no better time to engage with Universities, as the academics (and their bosses, the research managers) will be desperate to show that they are useful and have an impact beyond the sometimes barely-read academic journal article.

For the retail group at Stirling this external face, engaging with outside bodies, applying out research and writing for multiple audiences is ‘old hat’, but we’d welcome new partners in our research. Please get in touch to learn what we do and how we might work together. Other Universities are of course available – but depending on which league table you use, are nowhere near as good as Stirling!