What Works? evidence for decision makers - 25th November 2014
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
What Works? evidence for decision makers 25th November 2014
To find out ‘What Works’…
...visit the What Works
Centre websites and find The National Institute for
Health and Care Excellence The Education Endowment Foundation
out more about how you www.nice.org.uk http://educationendowmentfoundation.org.uk
can use our services to
help you get better value
for money.
The Early Intervention Foundation The College of Policing
http://www.eif.org.uk http://www.college.police.uk
The What Works Centre for Wellbeing
http://whatworkswellbeing.org
The What Works Centre for
Local Economic Growth The Centre for Ageing Better
http://whatworksgrowth.org http://www.centreforageingbetter.com
What Works Scotland The Public Policy Institute for Wales
http://whatworksscotland.ac.uk http://ppiw.org.uk
What Works? |3 Contents 5 A Message of Support 6 Introduction: Why ‘What Works’ Matters 8 What Works: Some of our findings… 9 What Works: Health and Social Care 13 What Works: Education 18 What Works: Early Intervention 22 What Works: Crime Reduction 25 What Works: Local Economic Growth 30 What Works: A Growing Network 32 What Works: What Next 34 Annex A: The What Works Centres
4| What Works?
What Works? |5
A Message of Support
In March 2013 we launched As we continue to improve and reform education, early intervention, crime and
public services at a time when finances local growth. We believe that in the coming
the What Works Network, are tight, it is more important than ever years, as the Centres mature, the Network
a world leading initiative that taxpayers’ money is spent in the most will deliver a step-change in the way that
designed to embed robust effective way. Together, the What Works local government, national government,
Centres cover policy areas with public and front line practitioners make decisions
evidence at the heart of local spending of over £200 billion, and are about how to deliver public services.
and national policy-making. helping to ensure that policy decisions
and professional practice are based We want that process to begin now, and so
upon robust evidence of what works we welcome this first report of collected
to deliver value for money and better findings. We urge commissioners and
outcomes for all. practitioners to draw on the growing body
of outputs from the Centres; and we urge
Five of the six initial Centres have now policy-makers to help the Centres find
been in action for a year or more, and last out what works by robustly evaluating
month saw the launch of the new What the impact of their policies. We are
Works Centre for Wellbeing. What Works convinced that in a decade we will
Scotland and the Public Policy Institute wonder how we ever did without the
for Wales have also joined the Network What Works Centres; let’s not wait
as associate members. until then to start putting their findings
to work.
This report brings together a selection of
early findings from the Centres: on health,
Rt Hon Oliver Letwin MP Rt Hon Danny Alexander MP
Minister for Government Policy Chief Secretary to the Treasury6| What Works?
Introduction: Why ‘What Works’ Matters
By Dr David Halpern, When your doctor prescribes a medicine, Generate
What Works National Adviser you have good grounds to trust that it will At the heart of the What Works mission is
be effective: drugs are tested for safety the generation and collation of evidence.
The ultimate goal of the and effectiveness before they come to The What Works Centres systematically
market, and the National Institute for assess and synthesise the evidence on
What Works Network is Health and Care Excellence (NICE) makes what works within their field of expertise.
to support better public evidence-based recommendations for Their independent status underpins
practitioners about which treatments their work, ensuring that commissioners,
services. The Network, should be available. But when you drop practitioners and the public can trust their
and Centres that compose your child off at school, or turn to the conclusions. Where gaps in the evidence
police to keep you safe, the evidence base are identified, some of the Centres also
it, are designed to do standing behind the education and crime have the resources to generate new
this by ensuring that the interventions being used has until recently evidence through trials and evaluations,
been much weaker. such as the 93 evaluations that have been
best evidence of ‘what commissioned since 2011 by the Education
works’ is available to the Loosely based on the model of NICE, Endowment Foundation.
founded in 1999, a series of
people who actually make independent ‘What Works’ institutions Transmit
the decisions; not only have been created since 2010. Generating and collating the evidence is of
government ministers and These Centres are dedicated to the no use if it never reaches the commissioners
generation, transmission and adoption of and professionals who need it. All the
council leaders, but also evidence. This report highlights their work What Works Centres are led or staffed by
doctors, headteachers, police by bringing together for the first time a leading experts, but rather than scholarly
selection of findings from the Centres’ articles, their outputs are designed to
chiefs, children’s services work to date. be widely accessible, fully public and
professionals and many more.What Works? |7
easily understood. Each of the Centres clinical practice1. The Centres are therefore The role of the What Works Centres is to
has developed, or is developing, easy committed not only to publishing expert offer the best expert assessment available.
to understand tools and guidance that guidance, but to supporting commissioners It is for the professional or commissioner
summarise the evidence and provide clear and practitioners to utilise it. They are to make the final judgment on what to
and practical advice for professionals and consulting with and meeting users, do. Commissioners and practitioners will
commissioners. they are training professionals and always need to consider additional factors,
they are working directly with Local such as public sentiment and local context;
These incorporate clear assessments Authorities and other decision making now they will also be able to weigh the
about which specific interventions are bodies to help them put evidence impartial, expert view on ‘what works’ into
effective, and which are not, and measures into action. This report provides these judgments.
of how confident the Centres are about some examples of these collaborations
these judgments, based on a systematic to date.
approach to assessing the strength of the Dr David Halpern, What Works National Adviser
evidence. Crucially, all the Centres also Danielle Mason, What Works Team
aspire to incorporate cost-effectiveness 1
Balas EA, Boren SA. Managing clinical knowledge for Laura Baynton, What Works Team
health care improvement. In: Bemmel J, McCray AT, editors.
into their advice and guidance. Yearbook of Medical Informatics 2000: Patient-Centered
Louise Moore, What Works Team
Systems. Stuttgart, Germany: Schattauer Verlagsgesellschaft
Adopt mbH; 2000:65-70.
Even in medicine, where robust empirical
studies have shown one treatment to be
more effective than another, it can take
many years for best practice to be adopted
across the profession: Balas and Boren
estimated that it took an average 17 years
for empirical results in medicine to feed into8| What Works?
Title Works: Some of our findings…
What
Crime Education Local Growth Health Early Intervention
‘Hot spot’ policing Peer tutoring Whilst they have More lives would be The Family Nurse
- patrolling in small approaches, where intrinsic social value, saved or improved if Partnership programme
areas where crime has learners work in small the local economic people with acute heart has been shown to be
been concentrated groups to provide each impacts of major failure were routinely effective in the US for
- reduces crime and other with explicit sporting and cultural treated by specialist improving children’s
does not simply move teaching support, have, projects tend not to heart failure teams. health and development,
it round the corner. on average, a high be large and are more with the benefits
impact on attainment often zero. outweighing the costs
at a low cost. by around four to one.
These are just a small selection of the Centres’ findings to date. Visit the websites listed on the second page for further findings.What Works? |9
What Works: Health and Social Care
The National Institute for NICE guidance
Originally set up to consider clinical and prevention with anticoagulant
Health and Care Excellence interventions, NICE’s remit has expanded; treatment. http://www.nice.org.uk/
(NICE) was founded in 1999. in 2006 to include public health, and in guidance/CG180
It has gained a worldwide 2012 social care. Over 1,000 pieces of NICE
guidance have been published in total, all • Providing people who inject drugs with
reputation for its pioneering available on the NICE website (www.nice. needles, syringes and other injecting
use of clinical and cost- org.uk). Some recent examples of NICE equipment reduces injection risks such
findings are provided below: as blood-borne viruses and bacterial
effectiveness methodologies infections.
to produce authoritative www.nice.org.uk/guidance/PH52
advice and guidelines. • More lives would be saved or improved
Indeed, the What Works if people with acute heart failure
were routinely treated by specialist NICE also maintain a record of practices
Network was first envisaged heart failure teams. www.nice.org.uk/ that they have categorised as not cost-
as a “NICE for social policy” guidance/CG187 effective at: http://www.nice.org.uk/
savingsAndProductivity/collection?page=
and the cost-effectiveness • Novel oral anticoagulants (NOACs) 1&pageSize=2000&type=Do%20not%20
aspect of the What Works should be offered, where appropriate, do&published=&impact=&filter. Some
instead of aspirin and as an alternative examples are:
remit was largely modelled to warfarin, to prevent stroke risk in
on the NICE technology patients with atrial fibrillation. NICE
appraisal. estimates 7,000 strokes and 2,000 • Do not routinely offer social skills
premature deaths could be avoided training (as a specific intervention)
each year with effective detection to people with psychosis or
schizophrenia.10| What Works?
As part of the systematic review of the cost-effectiveness is a central aspect of
• Do not offer acupuncture for the relevant evidence which underpins each TA, the appraisal. Figure 1 shows estimates of
management of osteoarthritis. the comparable cost-effectiveness of each the cost per QALY gained for technologies
technology is assessed by estimating the appraised by NICE between 2007 and Dec
• Do not use a pharmacological cost of the technology per ‘quality-adjusted 2013 (TA114 to TA301).
intervention to aid sleep in children and life year’ (QALY) gained as a result of its
young people unless sleep problems use2. In general, interventions costing less The graph illustrates the crucial role
are having a negative impact on them than £20,000 per QALY are considered that cost-effectiveness plays in the
and their family or carers. by NICE to be cost-effective. Interventions recommendations: all those technologies
costing between £20,000 and £30,000 per with a cost per QALY gained of £20,000
NICE technology appraisals QALY may be considered cost-effective if or below were recommended, whilst the
The purpose of NICE technology appraisals certain conditions are satisfied. NICE does majority of those with a cost per QALY
(TAs) is to appraise the health benefits and not usually recommend an intervention if it gained of £30,000 or more were not.
the costs of medical technologies such costs more than £30,000 per QALY (other
as medicines and surgical procedures. than for certain end-of-life treatments) Cost-effectiveness and public
The TAs inform NICE’s guidelines on which unless a strong case can be made that it is health interventions
technologies should be made available an effective use of NHS resources. It is also possible to assess cost-
on the National Health Service (NHS). effectiveness for public health interventions
Because the NHS has finite resources, NICE Cost-effectiveness is not the sole measure such as smoking cessation and exercise
guidelines need to consider not only the used to decide whether a technology will programmes, and NICE does this wherever
effectiveness of different treatments, but be recommended; also of relevance are possible in its Public Health Guidelines.
their cost-effectiveness: using more cost- factors such as the benefits of innovation, A 2011 study by Owen et al3 analysed
effective treatments means the NHS can the impact on other NHS objectives, and cost-effectiveness estimates for 200
achieve more with the same resources. other social value judgements. However, interventions. The authors conclude that
2
A quality-adjusted life year (QALY) is a measure of the state of health of a person or group in which the benefits, in terms of length of life, are adjusted to reflect the quality of life. One QALY is
equal to 1 year of life in perfect health. QALYs are calculated by estimating the years of life remaining for a patient following a particular treatment or intervention and weighting each year with
a quality of life score (on a zero to 1 scale).What Works? |11
Figure 1: Most credible cost per QALY gained for technologies appraised
by NICE between 2007 and Dec 2013 (TA114 to TA301)
Notes for Figure 1
Graph produced by NICE. The graph indicates
the decisions made by the Appraisal Committee,
chronologically since 2007, and the respective cost
per QALY gained. The graph includes the following
simplifications: Cost
• Where the Committee stated a range of credible
cost per QALY gained, the midpoint estimate is
shown.
• Where the Committee considered that the most
credible cost per QALY gained presented was
between £20,000 and £30,000, this is shown as
£30,000 per QALY gained.
• Where the technology dominated (i.e. was cheaper
and more effective than) the comparator, this is
shown as £0.
• On the occasions where the most credible cost per
QALY gained presented was above £150,000, this
is shown as £150,000 per QALY gained.
• Abbreviations: EoL, end of life; QALY, quality-
adjusted life year. Technology appraisals in chronological order12| What Works?
the majority of public health interventions However, evidence on the cost-
considered were highly cost-effective effectiveness of public health interventions
when assessed against the NICE thresholds is less widely collected than for clinical
discussed above. Indeed, the analysis interventions: better evidence on public
identified 30 interventions which were health interventions could help to support
actually cost saving. This means that more cost-effective allocation of health
they were not only more effective than resources in the future.
the comparator, but also cheaper.
These included:
• a number of smoking cessation
interventions;
• a number of interventions to manage
long term sickness and incapacity for
work; and
• a number of interventions to prevent
harmful drinking in people with alcohol
use disorders.
3
“The cost-effectiveness of public health interventions” in the Journal of Public Health, 2011. Owen L, Morgan A, Fischer A, Ellis S, Hoy A, Kelly MP.What Works? |13
What Works: Education
The Education Endowment Since its launch, the EEF has commissioned
93 evaluations and committed £52 million are trained to support pupils in
Foundation was launched of funding to innovative and scalable evidence-based and well-structured
in 2011 by lead charity The projects. In total, EEF projects are interventions.
Sutton Trust, in partnership working in 4,500 schools and reaching
630,000 pupils. • Helping pupils struggling with literacy
with Impetus Trust, with a at the start of secondary school is
founding grant from the These are some of the EEF’s latest findings, extremely challenging, and it is highly
covering both their own evaluations and unlikely that a single intervention
Department for Education. analysis of other published work: will be sufficient to help them catch
Its aim is to raise the up with their peers. However, some
approaches are more effective than
attainment of children • Peer tutoring approaches, where others. In a recent EEF trial, pupils who
facing disadvantage by learners work in small groups to went on school trips were then taught
provide each other with explicit a structured approach to improving
generating and synthesising teaching support, have, on average, their writing using the trip as a source
evidence about educational a high impact on attainment for of inspiration. The pupils who received
low cost. this intervention made an average
innovations, and encouraging of nine months additional progress
schools, government, • Research to date has suggested that compared to the control group.
students in a class with a teaching
charities, and others to apply assistant do not, on average, perform • Small group tuition can be a cost-
evidence and adopt those better than those in a class with only effective alternative to one-to-one
innovations found to a teacher. However, EEF trials have tuition as a way to provide intensive
shown that teaching assistants can support for struggling pupils.
be effective. have a positive impact if they This is true despite the fact that small
group tuition is on average, slightly14| What Works?
Figure 2: Selection from the EEF/Sutton Trust Teaching and Learning Toolkit
Toolkit i Toolkit info Pupil Premium Calculator Latest Updates Downloads
About the Toolkit
Arts participation + 2
months
Using the Toolkit
Pupil Premium Calculator Aspiration interventions 0
months
Videos and Case Studies
Evidence Briefs Behaviour interventions + 4
months
Toolkit Filter Block scheduling 0
months
Sort By
A-Z Collaborative learning + 5
months
4
Average impact i
Digital technology +
months
Cost i Early years intervention + 6
months
Extending schooltime + 2
months
Evidence i
Feedback + 8
months
Categories i
Primary
Homework (Primary) + 1
month
SecondaryWhat Works? |15
an online tool which allows teachers and The Toolkit has quickly become a popular
less effective than one-to-one tuition, school leaders to compare the estimated resource for school leadership teams
because it is also much less expensive. impact and cost of different types of looking for guidance on how to spend their
educational intervention. Like NICE resources effectively. In a recent survey
• Rewarding pupils’ effort with technology appraisals, the Toolkit uses a by the Sutton Trust, 45% of school leaders
financial incentives does not lead to common measure of impact - the average said they used the Toolkit to inform their
a significant improvement in GCSE additional months progress a pupil would spending decisions, up from 36% in 2013.
results, according to a randomised be expected to make in one year if they Among secondary school leaders the
controlled trial involving 10,000 pupils experienced the intervention - to enable proportion rises to 54%.
across England. comparison between different types of
intervention.
• Repeating a year is an expensive
intervention and has consistently been By summarising this data, along with details
found to have a negative impact on on implementation and applicability, the
attainment4. Toolkit provides guidance for teachers
and schools on how to best use their
resources, particularly their Pupil
The Teaching and Learning Toolkit Premium allocation.
The Teaching and Learning Toolkit –
produced in collaboration by the EEF, the Figure 2 shows a section from the Toolkit
Sutton Trust and Durham University – is a and Figure 3 uses the data from the Toolkit
groundbreaking example of how complex to illustrate the average cost and impact
and often inaccessible research findings of interventions. School leaders are also
can be presented in a useful way to encouraged to consider the context of
practitioners. their schools and their existing provision
as part of an evidence-informed decision
4
More detail on all the interventions listed can be found on
The Toolkit synthesises over 10,000 pieces making process. the EEF website: http://educationendowmentfoundation.org.
of quantitative educational research into uk/toolkit/about-the-toolkit/16| What Works?
Figure 3: Approximate cost and effect size for 34 education interventions As well as helping school leaders decide
which approaches to try, the Toolkit aims to
support decisions about implementation.
9 Interventions with relatively high
For example, research has shown that
impact for relatively low cost schools’ current deployment of Teaching
8 Meta-Cognition and Self-Regulation Assistants is not, on average, having an
Feedback
impact on pupils’ progress. However,
Average additional months of progress achieved
7 following promising findings from recent
EEF evaluations, the Toolkit now suggests
6 Peer Tutoring specific ways in which Teaching Assistants
Early Years Intervention
Collaborative Learning
Homework (Secondary) can be deployed to have a higher impact.
Reading Comprehension Strategies
5 Mastery Learning One to One Tuition The Toolkit also contains more detailed
Oral Language Interventions
information on the impact of interventions
Digital Technology
4 Phonics
Social and
Behaviour Interventions
Small Group Tuition on particular sub-groups, such as children
Emotional Learning
from low-income backgrounds.
Reducing class sizes
3 Outdoor Adventure Learning
Parental Involvement
Extending School Time
Source: The Sutton Trust – EEF Teaching and Learning
Learning Styles
2 Summer Schools Toolkit and Technical Appendices.
Arts Participation Sports Participation
Individualised Instruction
Estimated cost per pupil per year is based on a class
1 Homework (Primary) Mentoring Teaching Assistants size of 25.
Performance Pay Aspiration Interventions Text highlighted in bold signifies interventions for
Physical Environment
0
Block Scheduling 200
which the evidence on effectiveness is extensive or
400 600 800 1000 1200 1400 1600
School Uniform very extensive according to the Toolkit definitions.
-1 Setting or Streaming
Cost estimates in the graph are approximate. If you
Out of range
Average months progress = -4. Cost = £6000 wish to use the Toolkit cost estimates to support
-2 Estimated cost per pupil, per year (£) Repeating a Year decision making you should refer to the full Toolkit on
the EEF website for more detailed cost information.What Works? |17 “I have used [the Toolkit] countless times when talking to staff, parents and governors and it has... created a culture shift within the school so that staff became much more aware of educational research and started to question approaches to education and their impact.” Kate Atkins, Headteacher at Rosendale Primary School
18| What Works?
What Works: Early Intervention
The Early Intervention The EIF Guidebook Figure 4: The search page of
Foundation (EIF) was The EIF Guidebook (http://guidebook. the EIF guidebook
eif.org.uk/) is an innovative new resource
established in July 2013 as which currently summarises key features
an independent charity with of 50 Early Intervention (EI) programmes
available in the UK (see Figure 4).
a remit to: It provides information on: programme
aims, the nature of the intervention and,
• Assess the evidence on where available, costs and benefits.
which interventions work
Robust cost-benefit analysis is available
and their relative value for some EI programmes, mainly those
for money. implemented in the US; important
findings include:
• Advise government, local
• The HighScope Perry Preschool Program
councils and agencies, has a benefit-cost ratio of between
charities and investors 7:1 and 12:15, mainly through improved
employment and earnings, and reduced
on what works for crime and welfare dependence6.
whom, when.
• Chicago Child-Parent Centers delivered
• Advocate for early substantial benefits by the time
participants were age 26, mainly through
intervention to key increased earnings and reduced
decision makers. criminal justice and child welfare costs.
For preschool participants, the benefit-
cost ratio was almost 11:17.What Works? |19
“[We will implement]
• Follow-up studies of Family Nurse The EIF Pioneering Places
Partnership (now implemented in the The EIF is using the data from the a shift... toward an
UK) suggest that where it has been Guidebook and other new tools to evidence-based
implemented in the US, the benefit-cost support its 20 Pioneering Places, which
ratio achieved is between 3:1 and 5:1. are undertaking radical system change portfolio of support…
This is achieved mainly through reduced
welfare costs8 .
to improve overall effectiveness. It
has provided the areas with tailored
where cost benefit
assessments of the strength of evidence analysis can be
For the vast majority of EI programmes for the interventions they are using.
however, detailed cost and benefit analysis Based on returns from 13 EIF places aligned. Using the
has never been captured and there is
only general evidence on effectiveness.
it found that: evidence of the
Often the relevant information is available • Nearly half (47%) of the interventions Early Intervention
but has not yet been collated, analysed
and assessed. The EIF supports local
being delivered had little or no evidence
in an established clearing house.
Foundation
commissioners in understanding the local Guidebook we will
costs and benefits of different approaches, • 3% of the interventions being
and their implementation requirements. delivered have been proven to build on our service
In future versions of the Guidebook these
factors will also be included as means of
be ineffective.
portfolio and bring in
rating programmes, and providers will This work helps places to focus new programmes”.
have to supply cost information for their resources on services that are more likely
programme in order for it to be included to be effective and thus provide better
in the Guidebook. value for money.
Croydon Local Authority,
Transformation Challenge
Award submission20| What Works?
The EIF has also created a fund in Years and a review on Social and
partnership with the Economic and Social Emotional Learning in School and cost. There are emerging and promising
Research Council (ESRC) which encourages Youth Settings in train. Key findings approaches being tested and requiring
places to develop a partnership with a of what works in these areas are wider rollout, as discussed in EIF’s
university in order to produce robust presented below: recent evidence review. http://www.eif.
evaluation plans. This has led to the funding org.uk/publications/early-intervention-
of high quality programme evaluations in-domestic-violence-and-abuse-full-
in three Early Intervention Places: What works for tackling domestic report/
violence and abuse:
• ‘Step Up’, an early response model A narrowly specified ‘Duluth’ model (a As ever prevention is better than cure.
to improve child protection outcomes common programme for perpetrators There are some promising programmes
in cases of domestic violence in of domestic violence and abuse, the which aim to change attitudes and
Blackpool. content of which is based solely on behaviour among young people in
concepts of gender politics) has been relation to domestic violence and abuse.
• Functional Family Therapy in found by Randomised Controlled ‘Safe Dates’, a school-based approach,
Croydon for Troubled Families and those Trials in the USA to have no effect on is a relatively inexpensive example (an
at risk of involvement in youth crime. reoffending rates. It is also relatively indicative analysis suggests that it might
expensive (analysis indicates costs of typically cost around $17 per student)
• Baby Express, a month-by-month around $1,400 per participant)9. which would benefit from further testing
magazine for new parents in of effectiveness.
Greater Manchester. The overall cost to UK society of
domestic violence and abuse stands at
EIF Reviews over £15.7bn (and of course the human
Along with the Guidebook, EIF has costs are very great). Reoffending rates
published one new systematic review on are in the order of 40% and so the
Domestic Violence and Abuse with another benefits of effective interventions for
shortly to be published on the Early perpetrators would readily repay theirWhat Works? |21
What works to enhance the majority of behaviour programmes
development of language, surveyed are group-based, there are
communication and social and more group-based programmes with
emotional skills through parent and evidence of effectiveness overall.
child interaction (for 0-5 year-olds): Those offered at the targeted or
specialist level tend to have more
• Programmes work best if they robust evidence than those offered
are tailored to level of need and at the universal level.
specific ages, for example infancy,
toddlerhood, and pre-school. • Among the programmes that aim
to improve attachment, the most
• Of the programmes surveyed, those effective ones tend to feature the
that aim to improve behaviour tend following methods:
to have stronger more established
evidence of effectiveness compared • helping mothers understand their
to those that aim to support the own attachment histories. 5
That is, an overall return of between 7 and 12 dollars
for every dollar invested, once cost savings are taken
attachment relationship or children’s into account.
cognitive development. • coaching parents through the 6
Heckman, J., Moon, S., Pinto, R., Savelyev, P., Yavitz, A.
(2010), “The rate of return to the HighScope Perry Preschool
use of videotapes of parent/child Program”, Journal of Public Economics, Vol. 94, Issues 1–2,
• Among the programmes that aim interaction. pp. 114–128. http://www.sciencedirect.com/science/article/pii/
S0047272709001418
to improve behaviour, home visiting 7
Reynolds, A., Temple, J., White, B., Ou, S.-R., Robertson, D.
or individual therapy interventions • Many of these attachment (2011), “Age 26 cost-benefit analysis of the child-parent center
have the strongest evidence of programmes are suitable for early education program”, Child Development, Vol. 82, Issue
1, pp. 379–404. http://onlinelibrary.wiley.com/doi/10.1111/j.1467-
effectiveness, compared to group- delivery by health visitors and 8624.2010.01563.x/abstract
based ones for which the strength of clinical psychologists. 8
FNP Evidence Summary Leaflet, 2011, London:
Department of Health.
evidence is more variable. However as 9
See http://www.wsipp.wa.gov/BenefitCost/Program/86.22| What Works?
What Works: Crime Reduction
The College of Policing was The College’s What Works programme
• Simply putting more bobbies on the was set up in September 2013 to map the
established in 2013 with a beat has not been found to reduce crime reduction research evidence beyond
mission to identify, develop crime, unless they are carefully policing, and to get this evidence used in
and promote practice based targeted, or if police flood an area practice. The College and the ESRC have
because of a short-term risk. co-funded a consortium led by University
on evidence. The College College London to identify and label all the
has already produced • To reduce crime without displacing existing reviews of crime reduction research
it, officers need to be targeted on evidence, so the results can be viewed in
a series of ‘what works’ ‘hot spots’, which means they patrol one place online.
briefings to highlight what in small areas where crime has been
concentrated, or to work with a local The Online Crime Reduction Toolkit
is known about effective community to analyse and solve The What Works Centre online tool will
policing. The College’s problems (known as neighbourhood be launched in January to give easy
or community policing). access to the evidence on crime reduction
findings include: interventions. Over 300 research reviews
• Everyday police behaviour is have been identified and will be added
important: treating people fairly and to the online tool over time, with CCTV,
with respect in every encounter, street lighting and prison visits to deter
whether they are a victim or a young offenders (sometimes referred to as
suspect, means they are more likely Scared Straight programmes) first on the
to obey the law in the future. list to be presented. Users will be able to
weigh up evidence on the impact, cost and
You can see the full briefings at: www. implementation of interventions and use
college.police.uk this as part of their decision making.
An innovative element of this particularWhat Works? |23
toolkit is that it will also summarise Figure 5: Online Crime Reduction Toolkit
the evidence on how and in which
circumstances each intervention works,
enabling practitioners to ensure that they
Crime Reduction Toolkit
have captured the key elements of what
About the toolkit Intervention Impact on How it Where it How to What it
makes them work in a given operational crime works works do it costs
Our effect scale
context. Figure 5 below provides an Effect Mechanism Moderator Implementation Economic cost
illustration of what the tool will look like.
Closed circuit television
The new tool will be useful in highlighting Keys
programmes that are unlikely to deliver Quality of evidence
Improved street lighting in
value for money. For example, Scared public places
Straight style programmes, which take at-
risk young people on prison visits with the Filters
Sample 1
aim of reducing the risk of them offending Impact on crime
(select a range using the markers below)
in future. The evidence suggests that these
programmes are not effective, and in some Sample 2
cases participants may be more likely to
offend after the intervention.
Sample 5
Violent Crime Off On
The What Works Centre will also carry
out new evidence reviews to fill existing Property Crime Off On
Sample 6
evidence gaps, on topics including knife Offender based Off On
crime prevention, criminal justice responses Off On
Victim Based
to domestic abuse, and the use of alley Sample 7
Location Based Off On
gates to prevent crime.24| What Works?
“[I am] delighted Outreach programme What Works master classes: In Essex,
The What Works Centre also has an master classes prompted the senior team
that Essex Police, ambitious programme of engagement and to test the use of predictive policing
under the leadership outreach which aims to raise awareness techniques to tackle burglary, and to
and skill levels among police staff and other allocate their body worn video cameras
of Chief Constable stakeholders. to a randomly selected group of officers,
Stephen Kavanagh Professional standards: Research evidence
and compare their results with those of
officers without cameras. In this way, the
and Chief Supt is being built into the professional training, force and the College delivered a rigorous,
guidance, selection and promotion low cost test of the impact of body worn
Carl O’Malley, have processes in policing. At the same time, video on criminal justice outcomes in
implemented an police skills in reviewing and using evidence
are being developed. A pilot programme
domestic abuse. Results suggested use of
video helped increase the proportion of
innovative, science- called ‘Evidence Base Camp’ allowed 60 detections that ended with a charge.
based, professional police officers and staff to learn about
searching for, sifting and assessing research
policing operation.” while carrying out rapid reviews of evidence
on ‘hot’ topics, which they used to brief
national decision makers.
Innovation fund: The College is helping
forces and academics to create new
PCC Nick Alston research evidence. Its Innovation Fund
awarded £600,000 of seed funding to 16
joint force/academic bids to stimulate new
collaborative research at a regional level.What Works? |25
What Works: Local Economic Growth
The What Works Centre for The approach is three-fold: The Centre places a premium on making
its products straightforward and accessible.
Local Economic Growth was 1. Evidence reviews: review the existing Figure 6 taken from the Centre’s website
founded in October 2013. evidence base relating to economic describes the process of the evidence
The overall aim of the Centre development policy areas, drawing out reviews.
findings that are backed by systematic,
is to significantly improve the rigorous evaluation. Some of the headline results from the
use of evidence in the design reviews are summarised below.
2. Capacity building: work with policy-
and delivery of policies for makers and delivery partners to build
local economic growth and their capacity to incorporate measures of mployment training http://
E
policy impact into their programmes at whatworksgrowth.org/policy-area/
employment, leading to the earliest stage. employment-training/
more effective policies and
3. Demonstration projects: design • For basic skills or interventions aimed
policy-making. demonstration projects in partnership at raising general employability for
with Local Enterprise Partnerships individuals, shorter programmes
(LEPs) and Local Authorities to address (below six months, and probably
particular gaps in the evidence base. below four months) have a larger,
stronger effect on participants’
Evidence reviews employment than longer programmes.
Since October 2013, the Centre has Shorter programmes are also likely to
published four evidence reviews looking be cheaper which suggests that they
at the impact on local economic growth should be more cost-effective than
of employment training, business advice, longer programmes.
major sporting and cultural events and
access to finance programmes.26| What Works? Figure 6: Evidence Review Methodology for the What Works Centre for Local Economic Growth
What Works? |27
• In-firm/on the job training to develop options for properly they might improve wellbeing (the
programmes outperform classroom- evaluating relative cost-effectiveness What Works Centre for Wellbeing was
based training programmes. Employer of these different approaches as part launched in October).
co-design and activities that closely of the development of Growth Hubs.
mirror actual jobs appear to be key
design elements that could increase Major sporting and cultural events Access to finance
cost-effectiveness. http://whatworksgrowth.org/ http://whatworksgrowth.org/
policy-area/sport-and-culture/ policy-area/access-to-finance/
Business advice http://what-
worksgrowth.org/policy-area/ • The overall measurable effects of • While most access to finance
business-support/ major sport and culture projects on programmes appear to improve
a local economy tend not to be large access to finance (e.g. increase
• Business advice programmes show and are more often zero. credit availability), there is much
consistently better results for firm weaker evidence that this leads
productivity and output than they • Any wage and income effects of to improved firm performance.
do for firm employment. projects are usually small and limited This makes it much harder to assess
to the immediate locality or particular whether access to finance is a cost-
• Business advice programmes that use types of workers. effective way of improving the wider
a ‘managed brokerage’ approach seem economic outcomes (e.g. productivity,
to perform better than those that use • Given the significant cost of most employment) that policy-makers
a light touch delivery model. However, major sport and culture projects they care about.
such managed brokerage models are are unlikely to be cost-effective in
also more expensive, making it difficult terms of increasing local economic
to assess relative cost-effectiveness. growth, although sport and culture
The Centre is working with a number have intrinsic value to people aside
of Local Enterprise Partnerships from economic benefits, for example,28| What Works?
Capacity building innovative economic model which allows
The Centre has presented its work to comparison of expected programme
over 300 policy-makers, officers and cost-effectiveness.
delivery partners in its first year, and is
constantly expanding its network through: Demonstration projects
the dedicated supporters on its User The Centre is working on a number of
Panel; collaboration with partners such as projects intended to provide better
the Local Government Association; and evidence on programme cost-effectiveness
government-led projects such as the Public and to compare the cost-effectiveness
Services Transformation Network. of different programme design elements.
These demonstrator projects are being
The Centre is rolling out a series of co-created with a local partner (typically
workshops aimed at helping LEPs and a Local Authority or LEP) with the Centre
Local Authorities improve the use of providing advice on design, implementation
evidence in the design and delivery of and evaluation. Projects in development
policies. The sessions focus on the use include randomising elements of post Work
of cost-benefit analysis to compare Programme support, testing different forms
expected programme cost-effectiveness. of start-up and business support across a
They also highlight the importance of number of Growth Hubs, and randomising
building evaluation into programme design entry into a technology startup accelerator.
to allow for the assessment of actual cost-
effectiveness. The sessions are delivered by
New Economy Manchester (NEM), a LEP
that has significant expertise in this area.
The Centre is also working with New
Economy Manchester to embed findings
from the evidence reviews into the NEM’sWhat Works? |29 “Being clear about what works in terms of achieving economic growth is... something that all public service leaders should be interested in... the focus on the evidence base, and on rigorous evaluation ...is certainly the right way to develop our thinking and our approach. Particularly in an environment in which every penny counts.” Joanna Killian, Chief Executive of Essex County Council and Member of User Panel
30| What Works?
What Works: A Growing Network
The What Works Network The What Works Centre for Wellbeing help identify the changes needed to ensure
The What Works Centre for Wellbeing we all understand what we need to do to
is growing. On October 29th (www.whatworkswellbeing.org) was age better.
the What Works Centre for launched on October 29th, with the
Wellbeing was launched, support of 17 founding partners including There is clear evidence that inequalities in
Public Health England, the ESRC, our society have a huge impact on how we
and the Centre for Ageing government departments, the Office for experience ageing. There is also evidence
Better is currently under National Statistics, the Local Government that public services provided to people as
Association and the BIG Lottery Fund. they age tend to be reactive rather than
development. What Works The Centre is being set up by a preventative. The holistic needs of older
Scotland and the Public development group of the founding people are often ignored when developing
partners, chaired by Lord Gus O’Donnell. products or services. The goal of the
Policy Institute for Wales The wellbeing field has many pioneering Centre is to help break this cycle: to identify
recently joined the Network leaders and practitioners keen to share initiatives and behavioural changes that
their work, learn from others, and build can help people experience the
as associate members. the evidence base into a meaningful, opportunities and manage the challenges
reliable, easy to navigate source. of ageing. This will help individuals to feel
The Centre will develop a strong and that they are ageing well, help to prevent
credible evidence base which will support statutory services (which will always be
them to be able to focus their efforts needed) from becoming overwhelmed,
towards those interventions that will have and benefit the whole of society. This will
the biggest impact. be done by:
The Centre for Ageing Better • Developing the evidence base about
The Big Lottery Fund has previously what works to support ageing better.
announced its intention to fund up to
£50 million in the Centre for Ageing Better. • Funding projects that show promise to
The Centre will invest this in projects that make a difference and helping projectsWhat Works? |31
with proven effectiveness to operate at involved in the design and delivery of Public Policy Institute for Wales
greater scale. public services to: The Public Policy Institute for Wales (PPIW,
www.ppiw.org.uk) works directly with
• Working with the people, organisations, • Learn what is and what isn’t working in Welsh Ministers to improve policy-making
business and statutory bodies that can their local area. and delivery by providing them with access
best bring about an agenda for change. to authoritative independent analysis and
• Encourage collaborative learning with a expert advice. The Institute is funded by
• Joining up initiatives across the sectors range of local authority, business, public the Welsh Government and ESRC but is
to make sustainable and enduring sector and community partners. operationally independent of government.
change. The PPIW provides advice across the
• Better understand what effective full range of the Welsh Government’s
The Centre should be up and running in policy interventions and effective competencies by collaborating with
early 2015. services look like. researchers and policy experts from across
the UK and beyond to deliver a rolling
What Works Scotland • Promote the use of evidence in planning programme of work which is agreed with
What Works Scotland (www. and service delivery. the First Minister. In addition to undertaking
whatworksscotland.ac.uk) is a new initiative work on specific issues which are relevant
to improve the way local areas in Scotland • Help organisations get the skills and to each Minister’s portfolio, the Institute is
use evidence to make decisions about knowledge they need to use and developing research on three ‘cross-cutting’
public service development and reform. interpret evidence. themes which are relevant to the Welsh
The initiative brings together the Government as a whole and to the work of
Universities of Glasgow and Edinburgh, • Create case studies for wider sharing several of the other What Works Centres:
and other academics across Scotland, with and sustainability. managing demand for public services;
a wide range of local partners. What Works tackling poverty; and promoting effective
Scotland will work with specific local governance and service delivery.
Community Planning Partnerships32| What Works?
What Works: What Next?
The What Works Centres It is inevitable that in some areas the • Wider and deeper coverage: the release
findings will cause controversy. The long of a series of reports by the Centres
are bringing a rich seam
history of trialling and testing in medicine, and the steady expansion of the range
of practical empiricism to now taken for granted, has thrown up of interventions covered by their
policy and practice. It is hard many surprises, challenging established toolkits and guidebooks, developed
practices along the way. But this long road around the needs and questions of their
work, often involving trawling has also delivered many benefits – not least respective professional and commissioner
through thousands of studies longer, healthier lives – and in the process communities.
has transformed medicine from an art into
to piece together what is, the science we recognise today. • Training and skills: an expansion of
and is not, known about the support to, and skills of, the policy
This is a journey that is only just beginning community in government to ensure
what works. in many areas of policy and professional that civil servants have the skills to
practice. Very often, policy and practice use, and to expand, the evidence base.
have been guided more by tradition and This is not something that can be left
history – “this is how we’ve always done it” – to the analytical community alone,
than by the systematic study and testing of but needs to be in the toolkit of every
what works. civil servant.
In the coming year, the What Works • The internationalising of the agenda: the
Network and the National Adviser, UK’s What Works Centres have already
supported by the What Works Team, will begun to attract attention from across
aim to increase the profile and reach of the world. Most countries are asking
the Centres and the broader What Works similar questions about how best to
approach. Their work will include: teach children, boost local growth,What Works? |33 or reduce crime. The evidence the Centres draw on is international. Better coordination across countries could lead to the outputs being international too, and the costs shared. The UK public sector spent £674bn in 2012/13 alone. If the What Works Centres can help professionals and commissioners spend even a fraction of this money more effectively, the impact will be enormous. It is an ambitious programme, but one that the Centres are determined to deliver.
34| What Works?
Annex A: The What Works Centres
What Works Centre Est. Policy Area Status Funders
The National Institute 1999 Health and social care Operationally Department of Health
for Health and Care independent Non-
Excellence departmental Public Body
of the Department of
Health.
Education Endowment 2011 Educational attainment Founded by parent Department for Education
Foundation charities, the Sutton Trust
and Impetus-PEF, and
funded by a DfE grant.
Early Intervention July 2013 Early intervention Independent charity. ESRC and Government
Foundation Departments
What Works Centre for Sep 2013 Crime Hosted by the College of College of Policing and
Crime Reduction Policing. ESRC
What Works Centre for Oct 2013 Local economic growth Collaboration between ESRC and Government
Local Economic Growth the London School of Departments
Economics, Centre for
Cities and Arup.
What Works Centre for Oct 2014 Well-being Currently hosted by ESRC, Government
Well-being Public Health England. Departments and
agencies, potential
charitable funders
Centre for Ageing Better TBC Ageing In development. BIG Lottery Fund
What Works Scotland and The Public Policy Institute for Wales have also joined the What Works Network as associate members.Twitter: @whatworksuk Email: whatworks@cabinet-office.gsi.gov.uk Website: www.gov.uk/what-works-network
You can also read
