Defining and Measuring Impact - Professor Andy Neely Deputy Director, AIM Research

Page created by Rachel Bauer
 
CONTINUE READING
Defining and Measuring Impact - Professor Andy Neely Deputy Director, AIM Research
Defining and Measuring Impact

Professor Andy Neely
Deputy Director, AIM Research
Three questions…

1. What is impact?

2. Why does impact matter?

3. How can impact be measured?

       And then to a more important question:
            how can impact be enabled?
What is impact?
Why does impact matter?

The James Ladyman position…

“Those of us who oppose the research council’s emphasis on impact do so
   because we do not believe its policy is in the best interest of taxpayers.
   Accusing us of wishing to promote our own interests as researchers is a
   cheap shot… The impact agenda is distorting research priorities
   and distracting academics from their core activity, which is to
   produce scholarship for the consumption of their peers, who are
   usually the only people equipped to understand it and interested
   enough in it to bother trying”.

                                          Ladyman, J. (2009) letter in the Times Higher Education Supplement
The UK budget deficit!

                         Source: http://www.statistics.gov.uk/
Three questions…

1. What is impact?

2. Why does impact matter?

3. How can impact be measured?
Measurement is a mess…

                      S                              T                     T
      ∑QASASjs x   Wjs +   ∑QATASjL x             WjL +    ∑QATASkL x   WkL
       js                       jL                         kL
 A=
                                     S        T        T
                       ∑Wjs + ∑WjL+ ∑WkL
                           js            jL       kL

      “Often we measure the wrong things, in the
      wrong ways and frequently we measure too
                       much”
What is AIM?

•   AIM: The UK’s Research Initiative on Management.

•   Budget of £35+ million invested by the Economic and Social Research Council and the
    Engineering and Physical Sciences Research Council.

•   Used to fund over 250 Fellows and Scholars – all leading academics in their fields…

•   Working in cooperation with leading international academics and specialists as well as
    UK policy makers and business leaders…

•   Undertaking a wide range of collaborative research projects…

•   Disseminating ideas and shared learning through publications, reports, workshops and
    events…

•   Fostering new ways of working more effectively with managers and policy makers…

•   To enhance UK competitiveness and policy…
AIM’s mission and objectives

Mission – to significantly increase the contribution of and future
capacity for world class UK research on management.
Objectives
1. Conduct research that will identify actions to enhance the UK’s international
    competitiveness.

2.   Raise the scientific quality and international standing of UK research on
     international competitiveness.

3.   Expand the size and capacity of the active research base for UK research on
     management.

4.   Develop the engagement of that capacity with world class research outside the
     UK and with practitioners as co-producers of knowledge about management
     and other users of research within the UK.
Three questions…

1. What is impact?

2. Why does impact matter?

3. How can impact be measured?

       And then to a more important question:
            how can impact be enabled?
Who enables impact?
           Institutional context that enables research and impact

                          Research centre/group

                                 Individual
                                  research
                                   activity
Goal:                                                                          International
International                                                                Thought Leaders
Recognition for                                                              in Organisational
Thought                                                                        Performance
Leadership

Target                                  Press                        Practitioners             Policy                        Academics
Community:                                                                                     Makers

Output:                                                            CBP Web Page
Value                          Radio                Training /               Books /                                                 Top Journals
proposition                  Newspapers                                                                  Academic Conferences
                                                   Conferences /          Magazine Articles
                              Television            Education

                                                                                                               Thought-Leading Research:
                                                                                                                     • Best Practice
                                                                                                               •Theory Building & Testing
                                                                                                                   •Research Agendas
Activity / Internal                                                                                              •Tools and Techniques
Processes:
                                                  Policy
                                                                            Industrial
                                                  Makers
                           Relationships                                   Collaboration        Academic
                        In Press Community                    PMA                              Collaboration
How value is                                           CBP Community
created and sustained
                                                                                                                                 Generate Funding
                                                           Building Relationships

 Capabilities:
 Intangible value                   Best People                               Culture / Fun Place to Work /                    Technology & Data
 drivers e.g. people,                                                      Mutual Support / Knowledge Sharing                        Bases
 systems, climate and
 culture                                                                              CBP Team
Reflections on impact
•   Impact is not simply defined – its not just economic impact.

•   We need to think about the routes to impact:
    •  Patents and spin-outs are a minority activity…
    •  Other routes - e.g. press and media, books and practitioner articles are
       important…
    •  Don’t ignore consultancy – a hidden market…
    •  The most undervalued route to impact - teaching…

•   We need to understand what is involved in having impact and then create the
    infrastructure to help:
    •   Explore the academic value chain and the institutional infrastructure…
    •   Bear in mind an academic’s intellectual curiosity – many faculty move on to the
        next problem once they have “solved” the current one...

    Questions for research directors – [i] do you do enough to support the
    multiple routes to impact – especially the creation of teaching
    materials; [ii] which activities in the academic value chain should you
    support?
Reflections on evaluation

•   The dominant form of evaluation (as I perceive it) – publications,
    publications, publications…

•   HEFCE proposals for REF – the implication of measuring
    citations…

•   We need counter-balancing measures and incentives (but
    remember we are dealing with a global system)…

•   We should consider the hidden routes to impact in our evaluation
    (including promotion) models.
    Question for research directors – do you do enough in evaluation to
    counter balance the system’s natural focus on scholarly publication?
You can also read