INSIGHTS INTO THE NATIONAL PAYMENT BY RESULTS (PBR) SCHEME - STUDY CARRIED OUT BY NCB AND NFER ON BEHALF OF C4EO FOR THE CHILDREN'S IMPROVEMENT BOARD

Insights into the National Payment
     by Results (PbR) Scheme
   Study carried out by NCB and NFER
   on behalf of C4EO for the Children’s
          Improvement Board

Aims of the study
    Explore if proposed measures for the national
    PbR scheme:
•   are clearly linked to desired outcomes/evidence
    of link between measures and outcomes
•   can be objectively measured at a suitable level
    (geographically and temporarily)
•   are attributable to the LA
•   are robust and do not create perverse incentives
•   economically coherent and clearly articulate
    mechanisms by which the PbR scheme is
    expected to deliver improvement

About the study
• Review of national data sources and the
  theoretical model
• Review of wave one LAs’ bids and workplans
• Interviews with wave one LAs – thank you for
  your help with these
• Trial LAs to comment on emerging findings to
  inform interpretation and conclusions –
  workshop later today

Five domains covered by the 20
    proposed national measures
1. Contact with families/families in greatest
   need
2. Child development and school readiness
3. Family health and wellbeing
4. Parenting aspirations, self-esteem and skills
5. Cross-cutting

Contact with families
Activities/outputs              Outcomes
• % of families with under 5s   • % of families in greatest
  registered with children’s      need who have sustained
  centres                         contact with children’s
                                  centres
                                • % of families in greatest
                                  need receiving sustained
                                  outreach and family
                                  support through children’s
                                  centres

Linked to policy objectives
          and measurable?
• Universal reach a defining feature of children’s
  centres and key to ensure non-stigmatising
  service, but ‘registration’ needs refining
• Focus on families in greatest need crucial but
  national definition required
• Notion of sustained contact/outreach/family
  support with families in greatest need
  important but national (outcome based)
  definition required

Attributable and robust?
• All measures dependent on effective
  partnership working and data sharing
  protocols but attribution to LA seems
  appropriate
• All measures need refining to be robust and
  avoid ‘tick box’ exercises

Child development and
             school readiness
Activities/outputs              Outcomes
• % of families receiving a     • % of families who receive
  summary of child’s              appropriate support where
  development at 24-36            additional needs identified
  months                          (e.g. through 24-36 mths
• Take-up of 2 year olds free     summary)
  entitlement                   • Early Years Foundation
                                  Stage (EYFS)

Linked to policy objectives
          and measurable?
• Development summary per se not linked to
  child outcomes, actions from summary more
  clearly linked to outcomes + need to clarify
  what summary would be based on
• Take-up of 2YO entitlement linked to child
  outcomes if delivered in good quality settings,
  suitable measure when programme scaled up
• EYFS linked to child outcomes, suitable
  measure

Attributable and robust?
• Development summary and follow-up
  actions: attribution to LA and robustness will
  depend on what final measures are
• 2YO entitlement: attribution to LA
  appropriate, measure robust (if linked to
  quality of provision) - but double payment?
• EYFS: attribution to LA appropriate, some
  concern about manipulation by schools

Family health and well-being
Activities/outputs                  Outcomes
• % of families in greatest   •   Breastfeeding at 6-8 weeks after
  need accessing ante-            birth
  natal support through       •   Incidence of low birth weight of
  children’s centres              full term live births
• % of children in greatest   •    % of parents with post-natal
  need accessing evidence         depression (or self-reported
  based healthy                   emotional wellbeing)
  eating/lifestyle support    •   Prevalence of healthy weight at
  through children’s              age 4-5
  centres                     •   Economic wellbeing – possible
                                  measures: child poverty and free
                                  school meals (FSM)

Linked to policy objectives
             and measurable?
• All linked to improved outcomes for children but not all
  measured in a way suitable for a national PbR scheme
• LA level data available for: breastfeeding, low birth
  weight, weight at age 4-5, economic well being
• Developing measures and collecting objective data on
  ante-natal support and post-natal
  depression/emotional wellbeing would require
  considerable resources
• Evidence based healthy eating/lifestyle support
  programme would require a national accreditation
  scheme

Attributable and robust?
• Ante-natal support, breastfeeding, low birth weight
  and post-natal depression: very dependent on health
  services - LA may be penalised/rewarded for
  poor/good performance of health services
• Healthy eating and healthy weight: attribution less
  problematic as LAs take responsibility for public health
• Economic wellbeing: very dependent on local
  economy and non LA-agencies (e.g. JC+)
• Fairly robust to perverse incentives if link between
  outcomes and payments is suitably designed, but could
  create incentives to manipulate figures

Parenting aspirations, self-esteem
               and skills
Activities/outputs            Outcomes
• % of families in greatest   • Parents self-reported
  need completing evidence      aspirations and self-esteem
  based parenting             • Levels of parental
  programmes through            language/literacy/
  children’s centres            numeracy

Linked to policy objectives and
              measurable?
• Parenting programmes with demonstrable
  impact: clearly linked to improved child
  outcomes but would require a national
  accreditation scheme
• Parents’ self-esteem and skills: link with child
  outcomes but likely to be difficult and
  expensive to collect relevant data

Attributable and robust?
• Take-up of parenting programmes: can
  attribute to LA, robustness will depend on
  payment linked to recruitment/engagement
  versus outcomes
• Parents’ self-esteem and skills: attribution to
  LA difficult, risk of perverse incentives would
  depend on measures used

Cross cutting measures

• % of (outstanding/good) Ofsted inspections
• Levels of volunteering in children’s centres
• Levels of parental satisfaction with children’s
  centre services

Linked to policy objectives
          and measurable?
• Ofsted inspections: quality of children’s
  services linked to child outcomes and data
  available at LA level– but possible challenge
  around frequency of inspections
• Volunteering: very difficult to measure
  reliably and to link to child outcomes
• Parental satisfaction: linked to child
  outcomes, but difficult to measure in a
  reliable and consistent way

Attributable and robust?
• Ofsted inspections: can attribute to LA and
  provide independent, objective and broad
  assessment
• Volunteering in children’s centres: can
  attribute to LA but could become a ‘tick box’
  exercise unless measure very sophisticated
• Parental satisfaction: can attribute to LA but
  measure subject to manipulation

‘Most promising’ measures
• Early Years Foundation Stage – need to
  consider effect of changes in assessment +
  potential manipulation by schools
• Take-up of 2 year olds entitlement – when
  scaled up and if places in good quality settings
• Healthy weight – when LAs take responsibility
  for public health
• Ofsted inspections – frequency issue

Data available at LA level but
   attribution to LA problematic
• Breastfeeding prevalence
• Low birth weight
• Economic wellbeing (child poverty and free
  school meals)

Measures requiring considerable
 refinement but attributable to LA
• % of families registered with children’s centres
• % of families in greatest need with sustained
  contact/outreach/family support
• % of families with 24-36 child development
  summary & additional support when needed
• % of families in greatest need on a evidence
  based healthy eating/parenting programme
• Levels of volunteering in and parental
  satisfaction with children’s centres

Measures requiring considerable
         refinement and
  attribution to LA problematic
• % of families in greatest need accessing anti-
  natal support though children’s centres
• % of parents with post-natal depression
• Parents aspirations and self-esteem
• Levels of parental language/literacy/
  numeracy

Next steps
• Workshop at 3.30pm with research team to
  discuss the early findings
• Draft report circulated for comments to trial
  authorities in mid November 2011