Monthly Archives: May 2014

Summary Analysis of the Research Impact Network Event

Research Impact Network Event – Summary Analysis Part 1

This is our summary analysis of the inaugural Research Impact Network event, which took place at the University of Warwick on 26th February 2014.

The event was a valuable opportunity for reflection on experiences of supporting, co-ordinating or managing REF impact case studies and to identify key learning points to take forward to the next REF or any future impact reporting requirements. It also provided an opportunity to network with colleagues in similar roles in other HEIs and share experiences. I think some of us benefited from this therapeutic approach.

The inaugural Research Impact Network event generated a huge amount of discussion and captured a wealth of experience.

The inaugural Research Impact Network event generated a huge amount of discussion and captured a wealth of experience.

During the event, a huge amount of discussion was generated, and apologies that we haven’t been able to publish anything before now. We wanted to ensure that we did justice to the excellent contributions of all those who attended, and it has taken some time to make sense of all the material and shape it into a form which we can do something with, laying foundations for future activities and events and creating a provisional framework of resources for the future. For the last few months (in addition to all of the stuff that I do as part of my job) I have been treading water amidst a sea of flip chart paper and post-it notes, and this is what I’ve come up with so far.

Telling the story of REF through objects 1: from a crochet hook to a bolt gun.

Telling the story of REF through objects 1: from a crochet hook to a bolt gun.

Telling the story of REF through objects 2: chaos theory to Strictly Come Impact (how would Len score us.....obviously not with his customary 'seven!')

Telling the story of REF through objects 2: chaos theory to Strictly Come Impact (how would Len score us…..obviously not with his customary ‘seven!’)

This is the first of two reports which will be based on the following themes:

1) Where have we been and where are we now – our REF experiences and the challenges that we face?

2) Where do we need to go from here: The Future/The Manifesto and suggestions for future events and activities.

This report is on the first of these topics. The full version is available here: Research Impact Network Event – Analysis and Outcomes

For the sake of brevity and clarity, the discussion responses have been ‘themed’ into different areas of activity.  There are numerous ways to cut this, but by and large we wanted to create some coherent topics around the types of challenges that our particular community faces. (Inevitably there may be some overlap, or possibly even repetition). These could potentially form the basis for future, more detailed discussions around each of these topics or alternatively form the basis for an action plan for engaging with others within our ‘impact eco-system’ (of which, more anon).

Some of our challenges: the tip of the iceberg.......

Some of our challenges: the tip of the iceberg…….

Even more challenges.....

Even more challenges…..

Yet further challenges......

Yet further challenges……

WHERE HAVE WE BEEN and WHERE ARE WE NOW?

Our first session involved us all talking about the objects that we had brought to reflect our own experiences of working with research impact in the context of the REF.  Although possibly a slightly new type of concept for some, it had the benefit of bringing to the fore some of the emotionally laden, frustrating, infuriating or otherwise aspects of the REF in a graphic and often humorous way, enabling us to tap into elements of the experience which are often closed off to ‘colder’ discussions focussing purely on process and procedure. This is not just ‘touchy-feely’ stuff, it can illuminate, for institutions and funders, the things that frustrate us the most and often require disproportionate amounts of brain-power and energy to deal with.  If, collectively, we can begin to have a dialogue and work together on some of these issues, we could potentially work towards helping each other to achieve our goals.

We return to the practical and emotional challenges later

What links badgers, magnifying glasses, tape measures, clothes pegs.......

What links badgers, magnifying glasses, tape measures, clothes pegs…….

.....shoe polish, battenberg, a duck, a bolt gun, a crochet hook and another clothes peg....?

…..shoe polish, battenberg, a duck, a bolt gun, a crochet hook and another clothes peg….?

A: they all symbolise our perceptions of the REF Impact ‘experience’.

So, what are the challenges we face?

a) The business of ‘doing’ impact

Definitions and understanding impact

The potential for variable interpretations has its advantages in enabling us to be flexible about what counts as ‘impact’ (of the non-academic variety). Yet at the same time the variations of impact requirements for different funders and stakeholders creates confusion and a lack of coherence within our own institutions about what impact is and what we should be looking for or supporting. Our view is that stakeholders and funders might be able to work together more closely on this in the future, with a view to converging or helping us to find a form of language which conveys a clearer, more consistent definition of ‘impact’.

Identifying impact

Our challenges at institutional level involve selectivity about which projects or research activities we can support.  We simply don’t have the resources to track and support everything, but we would probably want a big enough sample of work to draw on the next time it comes to drafting impact case studies. A major challenge for Research Managers is the way in which we keep track of research and its impact. We were very much at the mercy of institutional memory to support the gathering of information about underpinning research.  If case studies feature as the main mode of reporting for the next REF exercise, the expectations might be so much higher given that we now know a bit about what is expected. How do research managers and academics keep appropriate records of their research and of their impact? What do ‘good’ case studies look like?

Crafting and constructing

We have to understand the original (‘underpinning’) research and how this relates to the subsequent impact (particularly where impact was more conceptual than just producing ‘widgets’ or return on investment), creating a coherent narrative in a form which is accessible to a general audience.  The two challenges which were mentioned time and time again were around evidence gathering and storytelling.

Processual Issues (as in understanding the process of impact, and the implications that this has for the way we work).

  • Understanding and education colleagues about some of the more technical aspects of impact: attribution, time-lag, duration, deadweight, displacement etc. and the implications that each of these have for tracking research and recording impact.
  • Supporting engagement with research, whilst appreciating that it does not constitute ‘impact’
  • Ensuring that researchers who were not involved with the REF impact case studies for the 2014 exercise understand how to establish an audit trail for the future
  • Linking relevant research outputs (as indicators of the quality of the underpinning research) to particular instances of impact; understanding the quality thresholds and appreciating that research quality and impactful research are not necessarily mutually exclusive.
  • ‘Monitoring’ (or tracking if you prefer) impact: gathering and storing evidence in its many varied forms
  • Maintaining productive relationships with external collaborators (‘users’) to provide testimonials as evidence of research impact
  • Understanding the ‘endpoint’: when to stop looking for relevant evidence.

The theme running through all of this is ownership. Where does this sit; with academic colleagues, with departments or with the central university administration? There is a need for robust, integrated approaches to managing these tasks within our institutions.  Roles and responsibilities need to be clear from the outset and there needs to be effective co-ordination between university departments, between central university administrators and faculties and between administrators and academics.

b) Practical and Emotional Challenges

We should also recognise the emotional investment that we and other colleagues in our institutions which is inevitably associated with the time and effort required for the task. It was often highly frustrating, for example when colleagues did not read guidance and whose contributions were therefore ineligible for inclusion. Some of us experienced it as a very inefficient process, with potential for too much wasted effort. We often had to deal with very stressful situations, providing reassurance and support during anxious times. The Impact Template (3A) was often met with much vexation and exasperation as there is no clear sense of what a good one looks like.

c) The Impact ‘Ecosystem’

Culture/expertise/community

Embedding impact is primarily about introducing the right structures and processes now, providing good support for academic colleagues who want to engage with the impact agenda and developing greater awareness and ‘impact literacy’ for others. Crucially, it may be that impact does not sit within one particular department in the institution, and it is likely to bring together multiple functions including marketing, communication, knowledge transfer, business development, training etc.  Whilst responsibilities and activities are likely to be distributed across the institution, some central co-ordination and management is essential to ensure that a coherent strategy is being implemented across the institution. Culture change is more about finding ways to work with academic colleagues in order to maximise impact and gaining their buy-in, both practically and ideologically.Culture change is a longer term project, admittedly, but one which needs to be kick-started now in terms of embedding relevant support structures within our institutions for engagement and impact.

Institutional Support/Constraints

Our institutional structures were obviously not designed with the REF impact case studies, or with Research Impact in mind. Whilst some put in place support specifically for the REF submission exercise, and others are continuing to provide infrastructure and support to impact and engagement activities, it was clear that organisational structures and ways of working could either enable or frustrate (mainly frustrate, actually) support for engagement and impact.

Therefore each institution’s senior management needs to maintain momentum by introducing impact strategies and practical measures such as systems/processes for capturing impact (with a note of caution that a push to operationalize before a well-formed strategy is in place will be potentially damaging, confusing and inefficient). Resourcing (including time and workload allocation) needs to be sufficient to enable support structures and methods for capturing impact to be established.

Just recognising that impact is something which needs to be done, and putting in place adequate support for it might not be enough. Institutions also need to be thinking about how to incentivise, recognise and reward efforts in support of public engagement and impact, and ensure buy-in by academic colleagues (introducing ‘impact champions’ for example).

Relationship Management

With good strategies and support structures, research managers should be in a good position to play a pivotal role in preparing institutions for the future demands of the impact agenda. Making this happen; bringing about organisational or culture change, will inevitably require the skillful management of relationships across different areas of the institution.

Relationship management isn’t just confined to the way we work with academic colleagues.  It is also fundamental to building productive relationships with our external partners, whether they are (potential) collaborators, funders or audiences for the research.

Coming up in part 2: The next part of the report focuses on how we can turn these challenges into practical actions, and what the Research Impact Network can do to help.

The full version of part 1 of the report can be downloaded here: Research Impact Network Event – Analysis and Outcomes

Advertisements