HANDS' SMH EVALUATION - SUMMARY FEEDBACK AND RECOMMENDATIONS 2014 EVALUATORS: TRACEY CURWEN, PHD & GLEN SHARPE, EDD
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Hands’ SMH Evaluation
Summary Feedback and Recommendations
2014
Evaluators: Tracey Curwen, PhD & Glen Sharpe, EdD
Acknowledgements: Erin Brock, Nathalie Ouelette, Michelle DermenjianGoals
Provide research evidence re SMH programs
Identify most effective Hands’ SMH service
Provide recommendations based on evidence
Literature research and Hands’ SMH outcomes
Curwen & Sharpe, 2014Summary
Overall, Hands’ SMH is beneficial
On average, clients improve, similar to literature
No differences among service models: mobile,
group, or classroom effectiveness
SMH services overall appear to be directed at few
children with moderate to severe concerns (*cf SNAP)
Most effective with younger population and those
who have greater difficulties
Focus on this population
with clear protocols of identifying mental health need
using most cost effective method
Curwen & Sharpe, 2014Process
1) Literature review
2) Program information
3) Program outcome data
Curwen & Sharpe, 20141) Literature Review
2 models
School-based SMH services (SBMH)
Provided by school personnel
Expanded SMH services (ESMH)
combines traditional SBMH community-based mental health
programs/multidisciplinary care
Focus on ESMH - model used by Hands
Curwen & Sharpe, 20141) Literature review continued
1st Review of Literature
Most samples did not meet “clinical cut-off” criteria
Significant decrease/improvement
moved from “no concern” to a lower score in the “no
concern” range
Others in clinical range
Significant decrease
Remain in clinical range
Not meeting criteria for program efficacy
Curwen & Sharpe, 20141) Literature review continued
Refocused review
Why programs are effective or ineffective
Program Components & Characteristics
Dosage and duration
Service administrator
Fidelity (implementation consistency)
Theory/framework/focus
Delivery
School characteristics
Intervention characteristics
Curwen & Sharpe, 20143) Hands’ SMH Outcomes
Data provided by Hands
Last 3 years
Data included:
Age
Gender
BCFPI at intake to agency
CAFAS pre- and post- SMH
Curwen & Sharpe, 20143) Hands’ SMH Outcomes
Participants
N = 543 children and youth
70% male
Age: M = 10.53 (SD = 3.27)
Range 4 – 18 years
4-5 = 4% 6-12 = 67% 13-18 = 29%
All children/youth had at least 1 SMH program
16 different SMH services
32% multiple services concurrent
37% services following SMH
35% services prior to SMH
Only 1st SMH considered
Curwen & Sharpe, 20141st SMH Participants
90
80
70
60
Percentage
50
40
30
20
10
0
4-5 6-12 13-18
Age Group
Male Female
Curwen & Sharpe, 2014First SMH Service
25.6%
15.8%
48.9% 5.9%
1.3
0.9
0.7
0.2 0.4
0.2
Mobile Group Consult
Classroom Student Specific Consultation Summer School Program
Monitoring Transitions Follow-up
SIS Intense Intervention
Curwen & Sharpe, 2014Outcomes
Of 543 SMH cases
232 CAFAS at pre- & post-1st SMH
All 232 combined:
Significant Improvement No improvement
Total Score Community
School/Work Substance Use
Home Thinking
Behaviour Towards Others Caregiver – material needs
Moods/emotions Caregiver - social support
Self-Harm
p < .01
Curwen & Sharpe, 2014Recall: Criteria for Evaluation
1) Investigate clinically significant change
initially met or exceeded clinically elevated cutoffs
improved to a non-clinical range post-treatment
Investigated actual scores
2) The change must be considered reliable
level of change must be sufficient enough to rule out random
fluctuations or measurement error
Used a stringent p value
CAFAS concern levels:
0 = none,
10 = minimal,
20 = moderate,
30 = severe
Curwen & Sharpe, 2014Pre-post CAFAS Scores
moderate
20
18
16
14
CAFAS Score
12
10 minimal
8
6
4
2
0
Pre-SMH Post- SMH Curwen & Sharpe, 2014BCFPI Mean Scores
80
Clinically concerning
70
60 Sub-clinical
50
T-scores
40
30
20
10
0
Mobile Group Classroom Consult
Internalizing** Externalizing** Total Problems**
Curwen & Sharpe, 2014Moderate – Severe Sample
Selected only those in the moderate to severe
range in any of the following:
Home
Behaviour towards others
Moods/emotion
n = 185
mobile (61%), group (24.3%), classroom (10.8%), and
consultation (3.8%)
Consultation dropped: only 8 participants
n = 177
Curwen & Sharpe, 2014Change scores by SMH service
10
9
Mean Change Score
8
7
6
5
4
3
2
1
0
Mobile (n = 113) Group (n = 45) Classroom (n = 20)
MANCOVA with age and pre-scores covaried
Curwen & Sharpe, 2014Single or Concurrent Hands’ Services
Only SMH Concurrent SMH+
CAFAS scale (n = 110) (n = 68)
School/Work 6.3 (8.2) 6.2 (10.8)
Home 4.8 (8.3) 3.5 (11.3)
Behaviour Towards
9.6 (10.9) 7.1 (10.1)
Others**
Moods/Emotions** 6.2 (9.1) 3.4 (6.9)
Note: pre-SMH CAFAS scores and age at SMH service were covaried
**p < .01
Curwen & Sharpe, 2014Summary
Many empirical investigation samples are NOT in
the clinical range pre-SMH
Similar profiles for Hands’ SMH clients
Many clients did not have pre-SMH assessment
measures completed
Unclear WHY referred to SMH
Does not fit clinical level of MH need
Presenting issue is unclear and thus its EBP
How is the service to be received determined?
Curwen & Sharpe, 2014Summary continued
Age is important to outcomes
Pre-SMH scores are important to outcomes
No difference in outcomes between mobile,
group, or classroom at exit
For those with at least one elevated Mental Health
score
Level of functioning 3-6 months later not assessed
Curwen & Sharpe, 2014General Recommendations
Identify those most in need of mental health
services
Concerns in clinical or sub-clinical range
Services and intervention based on empirical evidence
Conduct assessments including:
Protocol linking specific need (e.g., depression) and
service most likely to benefit (e.g. mobile)
Parent, teacher, and child
Evaluate changes seen by each
Curwen & Sharpe, 2014General Recommendations continued
Focus services on
Younger age groups
With more concerning functioning scores
Implement the most cost effective service among
mobile, group, classroom
Ensure clients meet eligibility criteria
Or consider refocusing the purpose of SMH services
Do not provide too many services at once
Clear assessment should allow for more focused
service/fewer services at one time
Keep the number of goals manageable *
Curwen & Sharpe, 2014General Recommendations continued
Consider new working relationship with school
personnel
Many children were not clinically concerning
May be a classroom or teacher-student issue
How SMH providers may work with/assist teachers differently
Curwen & Sharpe, 2014Limitations and Recommendations
Many children did not have an outcome measure
standard protocol to make SMH decisions
Re-consider data collection system and measures
Unclear how specific service is deemed necessary
Why group vs mobile vs consult?
Why multiple services?
Curwen & Sharpe, 2014Service System Partnership
School Boards and Hands
You and we are doing what the literature reveals
Our story is not unique-but not as effective as it could be
Students/Children and Youth
All can be referred to Hands for a variety of services
Priority is to link clinical levels of need with appropriate system
response (level of need, intervention type, location, capacity)
SMH Services System Partnership
Provide most cost effective, ethical, evidence based service at
appropriate levels in our collectiveWhich Student Should Be Priorized?
School
Pediatrician
CAS
Severe Problem Identification
…or probation, or family who’s environment is disruptedOpportunities
Increased inclusion of teacher in assessment and
treatment planning process for children and youth
Generalization of skills (from therapy) into classroom
With intensive intervention modelling and coaching
Increasing capacity of classroom milieu to support
child and not solely seeking child change
ongoing mental health issues (1:5) can be reflected in
chronic behavioural challenges whose intensity
intervention can moderate, but not necessarily eliminatedMaximizing Service System Capacity
75 schools and limited SMH staffing
We, collectively, need to do something different in
the system with collective mental health resources
Current Capacity
Hands serves over 2000 files across CYMH programs for
students who attend schools which should be priorized for
school based mental health servicesMaximizing Service System Capacity
WHAT SYSTEM DO WE WANT FOR OUR
CHILDREN AND YOUTH?
Partnering to provide equitable services
for children and youthYou can also read
