Kindergarten Readiness at Home - Idaho Legislature
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Kindergarten Readiness at Home
Introduction
Waterford Upstart is an in-home, cost effective approach to kindergarten readiness that helps families and
children develop skills needed for school success.
• The program has been piloted in 29 states and Navajo Nation and is currently being implemented through
public funding in 13 states.
Federally-Funded Programs
WA 1. Idaho
MT ND ME 2. Montana
VT
OR
ID
MN
NH 3. North Dakota
MA
SD WI NY 4. South Dakota
WY MI RI
IA PA NJ
CT 5. Wyoming
NV NE OH
UT IL IN DE
CA CO WV
VA
MD Publicly-Funded State Programs
KS MO KY
NN
TN
NC 1. Indiana
AZ OK
NM AR SC 2. Mississippi
MS AL GA 3. New Hampshire
AK TX LA 4. New Mexico
FL
5. Ohio
HI 6. South Carolina
7. Utah
8. Wisconsin
Slide 2
About Waterford Upstart
Waterford Upstart is an in-home kindergarten readiness program that was created by the Utah State
Legislature in 2009. Under the program:
• Preschool-age children use Waterford Upstart in the home for 15 minutes a day, five days a week.
• Children receive a personalized curriculum in reading, math and science.
• Each family receives coaching to ensure strong parental engagement. Waterford Upstart forms a
partnership with parents to ensure all children obtain the education necessary for success in school.
• Computers and Internet service are provided as part of the program if the home does not have it.
• Waterford Upstart is aligned to Idaho Early Learning Guidelines; Head Start Early Learning Outcomes
Framework; CASEL Social and Emotional Learning Competencies; and endorsed by the Council of
Administrators of Special Education.
Slide 3
Federal Education Innovation and Research (EIR) Expansion Grant
Waterford.org was awarded a federal EIR expansion grant in late 2018 that is being used to provide Waterford
Upstart in Great Plains states that do not provide significant funding for early childhood education, including
Idaho, Montana, North Dakota, South Dakota, and Wyoming.
• Waterford.org convened a TASK Force with education
stakeholders across the state – including representatives from
child development centers, Local Educational Agencies, and the
Idaho State Department of Education – to introduce Waterford
Upstart and seek guidance on program implementation.
• Under the grant, Waterford.org has provided Waterford Upstart to
just over 1,000 children across Idaho over the span of two years
with the goal of at least 200 children during the 2019-2020 school
year (Cohort 1) and the goal of at least 400 children during the
2020-2021 school year (Cohort 2).
Slide 4
Idaho EIR Upstart Pilot – Cohort 1 (2019-2020)
• Approximately 250 preschool-age children participated in Boundary
Cohort 1 of the Idaho EIR Upstart pilot. Bonner
Kootenai
Cohort 1
o 91.27% rural participants Benewah Shoshone
UPSTART
Latah
Counties
Clearwater
o 8.73% non-rural participants Nez
Perce
Lewis
Idaho
• Families began in-person trainings and initial student
assessments in August 2019; the pilot concluded with online Adams
Lemhi
Valley
final student assessments and virtual graduations (because of Washington
the COVID-19 pandemic) for families in June 2020.
Custer Clark
Fremont
Payette Boise
Gem
Jefferson Madison
Canyon Butte Teton
Ada Camas Blaine Bonneville
• Statewide, children from 72 different cities/towns across 31
Elmore
Bingham
Gooding Lincoln
counties used Waterford Upstart in the home under Cohort 1 Owyhee
Jerome Minidoka Power
Bannock
Caribou
Twin Falls
of the Idaho EIR Upstart pilot.
Bear
Cassia Lake
Oneida
Franklin
Slide 5
Average Weekly Usage – Cohort 1 (2019-2020)
ID Average Weekly Usage - Early Reading Program
The following chart shows Idaho participants’ 120
average reading usage on a weekly basis, as shown
by the blue line. The green line indicates the 100
recommended usage criterion of 75 minutes per 80
week.
60
• For 39 out of the 40 weeks, average weekly 40
usage exceeded the recommended usage 20
requirement of 75 minutes per week, indicating
0
significant parental engagement under the Idaho
AUG-26-2019
AUG-12-2019
AUG-19-2019
DEC-02-2019
DEC-09-2019
DEC-16-2019
DEC-23-2019
DEC-30-2019
JAN-06-2020
JAN-13-2020
JAN-20-2020
JAN-27-2020
FEB-03-2020
FEB-10-2020
FEB-17-2020
FEB-24-2020
APR-06-2020
APR-13-2020
APR-20-2020
APR-27-2020
NOV-04-2019
NOV-11-2019
NOV-18-2019
NOV-25-2019
MAR-30-2020
MAY-04-2020
MAY-11-2020
OCT-21-2019
SEP-02-2019
SEP-09-2019
SEP-16-2019
SEP-23-2019
SEP-30-2019
OCT-07-2019
OCT-14-2019
OCT-28-2019
MAR-02-2020
MAR-09-2020
MAR-16-2020
MAR-23-2020
EIR Upstart pilot.
Slide 6
Early Reading Program Outcomes – Cohort 1 (2019-2020)
Waterford Assessment of Core Skills (WACS) is a computerized adaptive test of early literacy for students in Pre-Kindergarten
through second grade with initial content validity established against state and national standards. The following chart shows
average results for WACS pretest and posttest scores, grouped by subtests for the program.
Note that Waterford.org was not able to proctor in-person final assessments because of COVID-19; accordingly, these
results only include students who scored within one standard deviation of their predicted overall end of year WACS scores.
• Children’s outcomes averaged at the ID WACS Reading Scores Grouped by Subtests
Pretest Posttest
“Kindergarten Intermediate” level, where
3000
most children score mid-point during the
Kindergarten year. 2667
• 93.24% of children scored at “Kindergarten 2333
Beginning” or above, demonstrating strong 2000
school readiness outcomes.
1667
• Children from low-income families (under 1333
100% of the federal poverty level)
outperformed their more affluent peers. 1000
Letter Letter Sound Initial Sound Blending Listening Vocabulary
Recognition Comprehension
Slide 7
Early Reading Program Outcomes – Cohort 1 (2019-2020)
The following table provides results on additional assessed skills. It shows average final WACS subtest grade
equivalents for the program.
Grade Equivalent of
Measured Skill
Final WACS Score
Letter Recognition Preschool Advanced*
Letter Sound Kindergarten Intermediate
Initial Sound Kindergarten Intermediate
Blending Kindergarten Advanced
Listening Comprehension Kindergarten Intermediate
Non Words Kindergarten Advanced
Sight Words Kindergarten Advanced
Real Words Kindergarten Intermediate
Reading Comprehension Kindergarten Intermediate
Vocabulary Kindergarten Advanced
* Note that “Kindergarten Beginning” is highest possible achievement level for the “letter recognition” skill.
Children’s outcomes averaged eight points below the highest level for this measured skill.
Slide 8
Parent Satisfaction – Cohort 1 (2019-2020)
Parents/guardians participating in the Idaho EIR Upstart pilot completed a comprehensive survey to provide
feedback related to their, and their child’s, experience in the program.
% Agree Survey Question
89% YES I like that the program expects our family to create a daily routine for using it.
95% YES The program helped prepare my child for Kindergarten.
81% YES The program prepared me to help my child in school next year.
87% YES My child is now better able to work independently.
85% YES The amount of contact from the coach (Family Education Liaison) was adequate.
95% YES Did the coach give you what you needed to effectively support your child?
99% YES The coach was knowledgeable about the program and software.
94% YES In general, did your child enjoy using the Early Reading Program?
96% YES Do you think the Early Reading Program was age appropriate?
97% YES Do you feel the weekly Early Reading Program usage requirements were appropriate?
98% YES Would you recommend Waterford UPSTART to a friend or colleague?
Slide 9
Parent Satisfaction – Cohort 1 (2019-2020)
Additional comments:
• I am so impressed by this program! My daughter went from learning letters and letter sounds to reading simple books in
just a few short months! I feel very confident sending her to Kindergarten this fall. We were part of a pilot program and I
am hoping and praying this program comes to Idaho permanently so that my other kids can also participate!
• I absolutely love this program! I would like to enroll my other son when he's able to!
• Our coach was very friendly and encouraging. It was nice to get off the phone with her and then tell my son what a
great job he was doing!
• We enjoyed reading the books on the program site because our library was closed due to Covid-19.
• We love this program! We hope my daughter can be enrolled in the future!
• The coach was always so nice and when I missed the calls, my daughter loved listening to the voicemails.
• I think it is a great program and easy for my child to follow and learn, I felt she learned a lot from the program.
Slide 10Idaho EIR Upstart Pilot – Cohort 2 (2020-2021)
• Approximately 770 preschool-age children participated in
Cohort 2 of the Idaho EIR Upstart pilot.
Cohort 2
o 58.80% rural participants UPSTART
Counties
o 41.20% non-rural participants
• Because of the COVID-19 pandemic, families began virtual
trainings and initial student assessments in August 2020; the
pilot will conclude with final student assessments and in-
person or virtual graduations (as appropriate) for families in
June 2021.
• Statewide, children from 107 different municipalities across 35
counties used Waterford Upstart in the home under Cohort 2
of the Idaho EIR Upstart pilot.
Slide 11Average Weekly Usage – Cohort 2 (2020-2021)
ID Average Weekly Usage - Early Reading Program
The following chart shows Idaho participants’
100
average reading usage on a weekly basis, as shown 90
by the blue line. The green line indicates the 80
recommended usage criterion of 75 minutes per 70
week. 60
50
* Usage generally
40 dips during holidays
• For 11 out of the 13 weeks, average weekly 30
(Thanksgiving).
usage exceeded the recommended usage 20
requirement of 75 minutes per week, indicating 10
significant parental engagement under the Idaho 0
OCT-05-2020
OCT-12-2020
OCT-19-2020
OCT-26-2020
NOV-02-2020
NOV-09-2020
NOV-16-2020
NOV-23-2020
SEP-14-2020
AUG-31-2020
SEP-07-2020
SEP-21-2020
SEP-28-2020
EIR Upstart pilot.
Slide 12Early Reading Program Strand Scores – Cohort 2 (2020-2021)
Reading strand scores are scores for subskills as a percentage of 100. A score of 80 or higher within the
Waterford Early Reading Program represents mastery, which is the goal of the cognitively based Waterford
Upstart program. The sequencer, within the software, individualizes instruction to provide remediation until a
child achieves mastery.
ID Average Strand Scores - Early Reading Program
100
• The following shows the average score, 90 85.58 87.83 90.03
organized by strand, for all students who scored 80
64.82*
on a given strand for reading. The red line 70
60
represents the expected average score, 80. 50
Through the first 13 weeks of the Idaho EIR 40
30
Upstart pilot, the average score for three of the 20
four reading strands was above 80. 10
0
Phonics Phonological Comprehension and Language Concepts
Awareness Vocabulary
* Note: Phonological awareness is an individual’s understanding of the sound structure of words (e.g., what rhymes with cat?). As such, it
is a difficult skill to learn, aligned closely to age, with significant growth seen even in month variations. While Waterford Upstart helps
students work on this skill through the curriculum and offline family activities, there is generally lower performance on this skill strand.
Slide 13Early Reading Program Average Overall Scores – Cohort 2 (2020-2021)
The following chart depicts the average overall score per student for the Waterford Early Reading Program.
The orange line emphasizes the expected average overall score, 80, which represents “mastery.” Each blue bar
represents the score of an individual student. This chart highlights the achievement level of students.
• 81.52% of students had an overall reading score of 80 or higher; the average overall score was 84.25
through the first 13 weeks of the Idaho EIR Upstart pilot.
FigureID
5:Average Overall
Idaho Avg. Scores
Overall – Early- Early
Scores Reading Program
Reading Program
100
90
80
70
60
50
40
30
20
10
0
Slide 14Questions
Slide 15Additional Background
Slide 16Adaptive and Sequenced Curriculum
• Waterford Upstart delivers individualized instruction in reading, math, and science that adapts to each
child’s learning trajectory.
o 2,500+ lessons o 360 digital books o More than 450 instructional hours
o 7,000+ activities o 330 animated songs o Progress reports for parents
• Waterford Early Learning instructional strands for literacy are aligned with the five essential components
of reading identified by the National Reading Panel.
Slide 17Waterford Assessment of Core Skills (WACS)
• WACS is a computerized adaptive assessment of early literacy for students in pre-kindergarten
through second grade.
o Initial content validity for WACS was established against state and national standards for 11
subtests (letter recognition, blending, reading comprehension, etc.).
• To establish concurrent validity and predictive validity, student performance on WACS was compared to
performance on five commonly-used standardized tests also measuring early reading skills:
o Dynamic Indicators of Basic Early Literacy Skills (DIBELS),
o Idaho Reading Indicator (IRI),
o Iowa Tests of Basic Skills (ITBS),
o Stanford Achievement Test Series (SAT 10), and
o Texas Primary Reading Indicator (TPRI).
Slide 18Parent Engagement
• Waterford Upstart empowers parents in the education of their young children through a parent
engagement curriculum developed over the program’s 11 years of experience working with families. The
program is designed to support small, incremental changes in behavior, with growth mindset activities that
provide clear examples of ideas for parents and caregivers that are easy to implement.
o Parent engagement activities are reinforced by live coaching from
the Family Education Liaison through weekly emails and personalized
calls with parents to support program participation and fidelity,
troubleshoot learning routines, and celebrate progress.
o While the curriculum is in English, families are supported in English
and Spanish, with translation services offered for additional
languages, as needed.
o In addition, Mentor (Waterford Upstart’s parent support app)
provides weekly messages that invite, support, and guide parents to
engage with their child’s development to produce stronger impacts
on parental engagement and child outcomes.
Slide 19Program Overview
1. Waterford Upstart Registration
▪ Pre-registration through the Waterford Upstart
website or by phone
▪ Program Registration
o Participant information, including a variety of
demographic information
o Usage and willingness to commit
o Agreements, including data-sharing
o Qualify for computer and Internet 2. Parent Training and Student Assessment
▪ Initial WACS assessment is given
▪ Conduct parent training:
o Build value in early education
o Software overview
o The all important 15/5 and establishing a routine
o Participant agreement
▪ Distribute computer and Internet
Slide 20Program Overview
3. Usage Management & Parent Engagement
Average Weekly Usage - Early Reading Program
▪ Internet and technical support 140
120
100
▪ Welcome call 80
60
40
▪ Weekly e-mails and texts 20
0
JUL-02-2018
JUL-09-2018
JUL-16-2018
JUL-23-2018
JUL-30-2018
APR-23-2018
APR-02-2018
APR-09-2018
APR-16-2018
APR-30-2018
AUG-06-2018
AUG-13-2018
AUG-20-2018
AUG-27-2018
MAY-07-2018
MAY-14-2018
MAY-21-2018
MAY-28-2018
MAR-12-2018
MAR-19-2018
MAR-26-2018
JUN-04-2018
JUN-11-2018
JUN-18-2018
JUN-25-2018
▪ Usage contests and activities
▪ Weekly usage review and
motivational support
▪ Monthly medals
4. Graduation and Final Assessment
▪ Administer the final WACS assessment
▪ Distribute a Parent Satisfaction Survey
▪ Honor the accomplishments of all our participants
Slide 21Annual Evaluations (Utah Statute)
Brigance Growth Rate Comparison
Third party evaluations show Waterford 2011
2012
Upstart children have significantly 2013
higher learning gains than children 2014
2015
from control groups – as measured over 2016
a number of years by the Brigance 2017
2018
Inventory of Early Development and the 0 10 20 30 40 50 60 70 80
Bader Reading and Language Inventory. UPSTART Control
Bader Growth Rate Comparison
2011
Note: Total potential scores were lower beginning 2012
in 2016 because the scale for the Brigance 2013
Inventory of Early Development changed. 2014
2015
Note: Total potential scores were lower in 2017 2016
and 2018 because the scale for the Bader 2017
Reading and Language Inventory changed. 2018
0 2 4 6 8 10 12
UPSTART Control
Slide 22Longitudinal Results
In 2016, the Utah State Board of Education conducted a longitudinal evaluation of UPSTART outcomes in Utah that
noted:
• Upstart students continually outperform state averages compared to control group on state standardized
tests in grades first through fourth (the highest-grade participants had achieved at the time of the study).
• Significantly, these gains were consistent across all subgroups, including special education, minority, low-
income, and English learner populations.
Slide 23Longitudinal Results
The longitudinal evaluation also noted:
• “The Upstart program shows continued success at helping preschool age children develop literacy skills
and prepare for school.”
• “These outcomes would have specific
benefits to at-risk children, whose
families struggle with poverty and
other issues, and often lack the
resources to help their children
develop the literacy skills needed to
succeed in school.”
• “The strong program effects support
wide-scale implementation across at-
risk preschool populations.”
Slide 24Randomized Controlled Trial (RCT) Effect Sizes
Investing in Innovation (i3) Validation Grant
U.S. Department of Education
UPSTART Randomized Controlled Trial (RCT) Effect Sizes
p > 0.5
0.45
0.40
0.42
0.35 0.38
0.30
0.25
0.20
0.15
0.10
0.05
0.00
Brigance Literacy Brigance Math
Effect sizes of 0.25 standard deviations or larger are substantively important
What Works Clearinghouse, Institute of Education Services (IES)
Slide 25You can also read
