Improving Middle School Student Engagement Through Career-Relevant Instruction in the Core Curriculum

Page created by Lisa Barber
The Journal of Educational Research, 106:27–38, 2013
          C Taylor & Francis Group, LLC
ISSN: 0022-0671 print / 1940-0675 online

    Improving Middle School Student
  Engagement Through Career-Relevant
   Instruction in the Core Curriculum
                                             DENNIS K. ORTHNER           PATRICK AKOS
                        University of North Carolina at Chapel Hill      RODERICK A. ROSE
                                                                         University of North Carolina at Chapel Hill
                                     HINCKLEY JONES-SANPEI
                                              Brigham Young University

                                                                         to be responsive to the social and learning environment
  ABSTRACT. The authors assessed the effect of career-
  relevant instruction on school valuing and engagement of               of the classroom and the school (Fredricks, Blumenfeld, &
  middle school students in a southern U.S. school district.             Paris, 2004), making it potentially malleable to pedagogical
  Previous research and theory indicate students learn best              reforms. Thus, school engagement is an early and predic-
  when new knowledge is provided within the context of in-               tive indicator of how much attention the student is giving
  formation students consider to be of value. The data come              to academic work that may be influenced by reforms at the
  from a school-based randomized trial of the CareerStart in-
  tervention that was introduced in 7 of 14 middle schools, and          classroom and school level.
  include the initial 3 years of data for 3,493 students. The au-           One of the many teaching strategies highlighted in school
  thors examined the effect of the CareerStart intervention and          reform efforts involves augmenting the relevance of the
  student-reported career-relevant instruction on psychosocial           curriculum so students can establish a link between the
  measures of school engagement and school valuing. After                content they are learning and either their environment or
  controlling for previous school engagement, demographic, so-
  cioeconomic, and academic factors, the analysis confirms that          their expectations for their future (Orthner, 2007). In the
  students in the treatment schools reported significantly higher        present study we evaluated the effectiveness of one such ap-
  levels of school valuing than students in the control schools,         proach, a teacher-focused, school capacity–building strategy
  and students reporting greater career-relevant instruction in-         called CareerStart. This was implemented by teachers in
  dicated significantly higher levels of school engagement and           middle school core subject courses including mathematics,
                                                                         language arts, social studies, and science. The purpose of
  Keywords: career orientation, intervention, middle schools,            this research was to determine the extent to which the Ca-
  student engagement, teacher instruction                                reerStart treatment and career-relevant instruction (CRI)
                                                                         influenced middle school students’ psychosocial school

F        or students to perform well in school, they must
         believe that their focus on education will pay divi-
         dends for them now or in their future. This focus or
school engagement has important consequences. Research
                                                                         School Engagement

                                                                            Presently, significant attention is given to promoting
has confirmed that students who are engaged in their edu-                school engagement to improve student attention and in-
cation, consider school as a valuable experience, and want               crease student retention. Although many of these efforts
to participate in school activities are more likely to demon-            are targeted at the high school level under the rubric of
strate high academic achievement (e.g., Klem & Connell,                  dropout prevention, outcomes including school failure and
2004; Wang & Holcombe, 2010) and are less likely to drop                 dropping out of school can be viewed as the culmination
out of school (Finn & Rock, 1997). In particular, recent re-             of a long-term process of disengagement that begins before
search has found that middle school performance and school               high school (Alexander, Entwisle, & Horsey, 1997), and of-
engagement were critical interim school outcomes and im-                 ten accelerates during the middle school years. Longitudinal
portant predictors of high school graduation (Blafanz, Fox,              research has indicated that many students deemed as being
Bridgeland, & McNaught, 2009). Students who consider
their education as relevant and preparing them to achieve
                                                                           Address correspondence to Dennis K. Orthner, Schools of Social Work
future goals are also more likely to perform well in school and          and Education, University of North Carolina at Chapel Hill, Chapel
graduate (Perry, 2008). Student engagement has been shown                Hill, NC 27599-3550, USA. (E-mail:
28                                                                                        The Journal of Educational Research

vulnerable to leaving school early were already disengaged       where I will work when I’m grown up” (p. 228), indicating
from their education before high school; thus, these students    a readiness for career exploration.
begin dropping out before innovative high school programs           Research has indicated that student career exploration
have an opportunity to make a difference (Orthner, Cook,         can have a positive effect on school engagement. Studies
Rose, & Randolph, 2002). Research on school engagement           of school-to-work programs offered in high schools have
during the middle school years (i.e., beginning of Grade 6       consistently found that students in these programs exhibit
to the end of Grade 8) has consistently found that students’     higher psychosocial school engagement and lower rates of
behavioral and psychosocial engagement declined precipi-         dropout than students not participating in career exploration
tously over this period (Orthner et al., 2010; Woolley &         (Castellano, Stringfield, & Stone, 2003; Plank, 2001). In a
Bowen, 2007). This pattern of decline suggests that a high       longitudinal study of low-income ninth-grade students, re-
proportion of students arrive in high school with diminished     searchers reported that students with higher levels of career
expectations for what their education can provide. There-        planning sustained higher levels of school valuing and feel-
fore, it is not surprising that only 75.6% of the nation’s       ings of belonging to their school (Kenny, Blustein, Hasse,
youth complete high school within the normal 4-year pe-          Jackson, & Perry, 2006). Similarly, Perry, Liu, and Pabian
riod, including those that graduate early (U.S. Department       (2010) examined the role of career-planning support during
of Education, 2010b).                                            the middle school and high school years on student school
   Student psychosocial engagement and valuing, the types        engagement and school grades among low-income students.
of engagement reviewed in this investigation, have demon-        Perry and colleagues found that students with greater ex-
strated that they are linked to student academic progress        posure to career-relevant planning were significantly more
and school success. Studies using measures of emotional          likely to be engaged in their education, and in turn, school
and psychosocial engagement have found positive relation-        engagement predicted higher grades.
ships to academic test performance (Borman & Overman,               Modern expectancy-value theories of learning are based
2004), student grade point averages (Gonzalez & Padilla,         on the observation that learning occurs most effectively
1997), and better student attendance at school (Finn &           in the context of what the student believes will help him
Rock, 1997; Klem & Connell, 2004). Further, school en-           or her achieve personal goals (Eccles et al., 1993). When
gagement research has indicated that students who regard         the information given or the tasks assigned are not clearly
their education as relevant and purposive in preparing them      connected to what the student perceives as being of value
to achieve future goals are much more likely to perform well     to his or her personal goals, short-term learning may oc-
in school and stay in school to graduate (Perry, 2008). En-      cur but the information is less likely to be retained or in-
hanced school engagement not only reduces risks of poor          tegrated into the student’s cognitive framework (Wigfield
outcomes, such as living in poverty, receiving public as-        & Eccles, 2002). An important corollary of this cognitive
sistance, and having poor health outcomes (Blafanz et al.,       framework is the work on possible-selves theory (Markus &
2009), but also increases the likelihood of students continu-    Nurius, 1986). This motivation theory provides some clues
ing to college or making a successful transition into a job or   as to why the career- and job-related relevance of teacher-
career (Fredricks et al., 2004).                                 provided instruction may be an important consideration in
                                                                 promoting student engagement, especially during the for-
Role of Career Relevance and Instruction                         mative years of early adolescence (Markus & Nurius, 1986).
                                                                 The possible selves theory proposes that youth are actively
   Although researchers have given little attention to ca-       engaged in developing concepts of their future selves that
reer development in elementary and middle schools, some          can potentially motivate them to learn and behave in ways
research has shown that tentative college plans and career       that are consistent with those projections of future selves
preferences begin emerging in elementary school (Trice &         (Oyserman, 2008). Because early adolescents are still form-
King, 1991). Students begin to develop identities during         ing these projections, accurate cues to future possible selves
their early adolescence, during which time they are influ-       may be quite motivating and serve as a schema around which
enced by early school experiences such as identification with    adolescents are able to collect and aggregate new informa-
workers, gender stereotypes, race, class, and social valua-      tion that applies to future-oriented self-concepts. Research
tion (Gottfredson, 1981; Trice, Hughes, Odom, Woods, &           has supported this assumption and demonstrated that stu-
McClellan, 1995) that can influence later career develop-        dent test performance, attendance, and retention were as-
ment. Even though middle school students are not typically       sociated with crystallization of the student’s possible self
ready to commit to career choices, these students experience     (Oyserman, 2008).
the careers of their parents or community and are expected          A further stream of research and theory from the litera-
to make choices in their middle and high school curricula        ture on school transition provided support not only for the
that can impact future career and educational opportunities      attention to engagement, but also the middle school tim-
(Akos, Lambie, Milsom, & Gilbert, 2007; Akos, Shoffner, &        ing of the CareerStart intervention. Research has suggested
Ellis, 2007). Research by Orthner et al. (2010) reported that    that when students transitioned from elementary to mid-
80% of the sampled students entering Grade 6 “think about        dle school, many early adolescents experienced academic
The Journal of Educational Research                                                                                           29

declines (Alspaugh, 1998) and increased distress (Ander-          of their instruction. For example, CareerStart provides
man, Maehr, & Midgley, 1999). Recommendations from                short, high-quality, and easy-to-teach lessons that each core
several authors (Akos & Galassi, 2004; Eccles et al., 1993)       teacher in Grades 6–8 can use to illustrate priority con-
have included creating an appropriate person–environment          cepts that are part of the state-mandated standard course
fit that supports students’ emerging autonomy and increased       of study. CareerStart lessons were prepared by experienced
capacity to think about their futures. CareerStart provides       teachers who had been recommended by their curriculum
an opportunity for students to explore career possibilities       coordinators. The lessons were then peer reviewed, and then
through structured supports offered in the classroom envi-        further reviewed by curriculum specialists prior to publish-
ronment. This may not only help to buffer transition de-          ing. Ten example lessons were prepared for each of the four
clines, but suggests an optimal time to impact these devel-       core courses in each middle school grade. Each of the 10
opmental pathways.                                                lessons was designed for a 1- or 2-day student experience.
                                                                  CareerStart lessons and support materials are available on-
CareerStart Intervention                                          line and can be easily accessed via the web at LEARN NC
                                                                  (, an instructional website widely used by
   CareerStart is a teacher-focused, schoolwide capacity-         teachers in the state. Teachers in the targeted schools were
building strategy that attempts to positively influence the       asked to use the CareerStart lessons and to continue de-
educational and workforce trajectory for all students, but es-    veloping and incorporating career-relevant illustrations in
pecially those at elevated risk for school failure. The overall   other topics they teach.
goal of CareerStart has been to develop, implement, and              In addition to the lessons, CareerStart teachers received
evaluate a strategy for future career orientation, designed       bimonthly updates through CareerStart Dispatch e-mail
to improve middle school student engagement, academic             newsletters, which kept teachers informed of new career con-
performance, and career exploration. If these middle school       nections to the core content as well as providing examples
goals are achieved, it is expected that students exposed to       of the ways other teachers have augmented class instruc-
CareerStart instruction and support will be better prepared       tion to include career relevance. Also, at the beginning of
for high school courses and will stay in school to graduate.      the academic year, teachers were provided opportunities for
   CareerStart helps teachers in core middle school courses       training and coaching on how to best develop career con-
(i.e., mathematics, science, language arts, and social studies)   nections in other aspects of their teaching. Lead teachers,
illustrate the value of learning state-required course content    curriculum coordinators, and school principals also received
by incorporating career examples drawn from industries rep-       updates on lessons learned from CareerStart, which in-
resented in the labor markets in which the schools reside.        cluded strategies for promoting CareerStart activities in their
Students in classrooms with operating CareerStart princi-         schools.
ples should be able to answer the questions asked most often:        The CareerStart strategy also involved parents, caregivers,
“Who uses this information in the real world?” or “When will      and other school professionals in the instructional approach.
I ever really use this information when I leave school?” Al-      One third of the lessons included a parent-engagement ac-
though middle school philosophy (Bishop & Pflaum, 2005)           tivity in which a student is asked to interview or to plan
already recommends relevant curriculum, CareerStart builds        an activity with their parents or caregivers. These activities
on these efforts by providing contemporary career illustra-       typically engaged students and parents in talking about ca-
tions and encouraging wider use of these teaching methods.        reer issues or the types of information and skills that adults
   The CareerStart strategy mainstreams career exploration        use in their own jobs. In addition, other school personnel are
into the overall curriculum and culture of the middle school.     often involved in career exploration activities or as support
Teachers are not asked to teach new core content; they use        for CareerStart lessons. School librarians were encouraged to
CareerStart lessons to teach required content with illustra-      have U.S. Department of Labor or other career information
tive examples from career fields in which this knowledge is       resources available to teachers and students. School coun-
used. For example, students learning to calculate volume in       selors were available to teachers and students to help in early
mathematics learn the relevance of that knowledge to heat-        career exploration and providing information on high school
ing and air conditioning technicians, equipment operators         courses, community college programs, or university oppor-
at a utility, and to applications in manufacturing and de-        tunities. School social workers could help students identify
sign processes. Language arts and mathematics content are         community resources that can be accessed to obtain career
applied in the exploration of business or office management       mentoring experiences or to receive guidance in career op-
activities, the development of business plans, or finance and     portunities available to youth and adults in the community,
marketing careers. The jobs that are illustrated in the Ca-       especially when parents or caregivers are not able to help
reerStart lessons range from those that require only a high       make those links.
school diploma and advanced technical training to careers            Although students in middle school may have different
that require college or postgraduate degrees.                     college or career aspirations, it should be noted that Ca-
   CareerStart provides critical tools that help teachers im-     reerStart does not track students into particular career voca-
plement their instruction in ways that improve the relevance      tions or educational trajectories. CareerStart is a universal
30                                                                                         The Journal of Educational Research

intervention that aims to expand all students’ visions of        CRI was conducted as an observational, rather than experi-
the future career and educational opportunities available        mental study.
to them. Instead of fostering potential inequity, integrat-
ing workforce examples in the core curriculum allows each        Sample
student to engage in career exploration and understand the
relevance of coursework at a fraction of the time and finan-        The sample included a 3-year longitudinal cohort of 3,649
cial costs of after-school or other targeted programs.           students who began Grade 6 in the 2006–2007 school year.
   As part of the evaluation of CareerStart, different lev-      The sample was 52% male, 47% Asian or Caucasian, and
els of fidelity to the CareerStart program were noted. Some      53% African American, Hispanic, or Native American. Pre-
teachers in the treatment schools did not use the Career-        vious research indicates that Asian students are more similar
Start lessons and some teachers in the control schools used      to Caucasian students with respect to risk of dropout and
career examples in their classrooms. Also, some teachers         school failure than they are to African American, Hispanic,
transferred between treatment and control schools. There-        or Native American students (U.S. Department of Educa-
fore, in addition to the treatment and control indicator, we     tion, 2010a). Of the total sample, 56% of students received
used a student-reported measure of CRI in the evaluation.        either free or reduced-price lunches at any time during the
                                                                 study period, 39% came from single-parent homes, and 19%
                                                                 were academically gifted as designated by the district. Ad-
                                                                 ministrative data on these students were collected annually,
   The evaluation of this intervention centers around the        and student survey data were collected during the spring of
questions of whether CareerStart and CRI are likely to in-       each of the 3 years of middle school (i.e., Grade 6–8). See
crease the psychosocial school engagement of middle school       Table 1 for a sample description with a comparison between
students, as measured by school valuing and school engage-       students in treatment and control schools. Although there
ment.                                                            were no significant differences between treatment and con-
                                                                 trol schools on gender, poverty, single-parent status, or on
Research Question 1: Do eighth-grade students after 3 years of   the two measures of psychosocial engagement, more students
  receiving the CareerStart treatment report higher valuing      in the treatment schools were African American or Hispanic
  and engagement relative to students in schools receiving       and academically gifted. Note that these differences did not
  standard instruction?                                          take into account nesting within schools. Analysis of base-
Research Question 2: Do eighth-grade students whose teach-       line data taking the multilevel data into account found no
  ers provided during the three middle school years more         significant differences between the treatment and control
  CRI report higher valuing and engagement relative to           schools for gender, race/ethnicity, single-parent household
  students in schools receiving less CRI?                        status, academic giftedness, or percentage of students on free
                                                                 or reduced-price lunch. In addition, our analyses found no
Evaluation Design and Data Sources                               differences between treatment and control schools with re-
                                                                 spect to standardized test scores in reading and mathematics
   Using a stratified randomization procedure, 14 middle         (obtained from elementary school) or psychosocial measures
schools in a single North Carolina school district were ran-     (administered at the beginning of Grade 6). None of the
domly assigned to either the treatment (n = 7) or the con-       normalized differences (i.e., effect sizes) was larger than 0.10
trol condition (n = 7). Of the 14 schools, six were Eq-          (Imbens & Wooldridge, 2009).
uity Plus schools, a district categorization based on Title 1
status. Three Equity Plus schools were randomly assigned         Measures
through a random number generation procedure to the treat-
ment condition. Similarly, four of the eight non–Equity Plus        Dependent variables: School psychosocial engagement outcome
schools were randomly assigned to the treatment condi-           measures. Psychosocial student engagement among eighth-
tion. Thus seven schools in the treatment condition, three       grade students was measured using two standardized scales:
Equity Plus schools and four non–Equity Plus schools, re-        School Valuing and School Engagement. The distribution
ceived the CareerStart intervention, whereas curricula of the    of student responses for these outcome measures were highly
seven schools in the control condition included the standard     skewed and not distributed normally because the majority
course of study only. Teachers in the control schools did not    of the students reported high valuing and high engagement.
receive the lessons, newsletters, and other supports received    Therefore, these variables violated the normal distribution
by teachers in the treatment schools; however, teachers in       assumption, raising questions about the appropriateness of
the control schools were not prohibited from using career-       linear modeling. Further, linear models assume metric invari-
relevant illustrations in their lessons. Therefore, the second   ance. For example, the difference on the scale score between
research question on CRI was included because randomizing        1 and 2 is the same as the difference between 4 and 5, which
schools or students to treatment conditions did not ensure       is arguably not the case for the outcomes of interest. There-
treatment exclusivity. The portion of the study regarding        fore, a logical threshold was established for each outcome
The Journal of Educational Research                                                                                                             31

   TABLE 1. Descriptive Statistics of Student Variables with Significance Tests

                                                        Treatment (n = 1,976)                   Control (n = 1,655)

   Variable                                         n         %        M         SD        N        %       M         SD       χ2         t

   Female (m = 0, f = 1)                          926       47.5                          786      48.1                       0.13
   Minority (Asian–Caucasian = 0,                1,122      57.2                          780      47.6                      33.20
     African American–Hispanic = 1)
   Poverty (nFRL = 0, FRL = 1)                   1,136      57.5                          899      54.3                       3.67†
   Special education (no = 0, yes = 1)            452       22.9                          351      21.2                       1.45
   Academically gifted (no = 0, yes = 1)          365       18.5                          192      11.6                      32.74
   Single parent (no = 0, yes = 1)                657       34.0                          517      32.1                       1.42
   Valuing, Grade 6a                                                  3.55      0.90                       3.50       0.96              –1.57
   School engagement, Grade 6a                                        4.14      0.63                       4.14       0.67               0.05

   Note. FRL = receiving free or reduced-price lunch; nFRL = not receiving free or reduced-price lunch.
   aAfter 1 year of treatment.
   †p < .10. ∗ ∗ ∗ p < .001

measure—above and below the mean response—and logistic                            Independent variables. Two variables were entered as inde-
regressions were used to test for the associations between                     pendent measures of interest, with each intended to answer
CareerStart treatment, CRI, and the outcome variables, al-                     one of the research questions. The first was a school-level
though controlling for student demographic characteristics,                    variable that identified each school’s status in the random
previously measured outcomes, and school characteristics.                      assignment to the treatment or control condition. The sec-
   The seven-item School Valuing subscale was derived from                     ond variable was a teaching team-level variable. In core
the Student Identification with School measure (Voelkl,                        courses, CRI was delivered by teachers working in teams
1996). This subscale reflects a student’s belief that school                   composed of two to four teachers. The second variable was
is important and provides an opportunity to learn useful                       a student-reported measure of each teaching team’s compli-
information that he or she will find helpful or useful in                      ance with the treatment assignment, which was represented
the future. The School Valuing scale includes items such as                    by a level or dosage of CRI. The level of CRI was calculated
“school is important, school and what I learn there is useful,                 from survey questions administered to students at the end of
school and what I learn there will be useful in getting a                      each school year regarding the extent to which they agreed
job, dropping out would be a mistake,” and “school is not                      with the statement, “the teacher often used career exam-
a waste of time.” The mean of the seven items was used                         ples from jobs and careers” in the classroom instruction. The
for the Valuing scale. If three or more items were missing                     same question was asked for each of the four core curricu-
then the scale score was missing. The threshold used for the                   lum classes: language arts, mathematics, science, and social
binary outcome variable was the mean. Overall, 58% of the                      studies. The students could choose from response options on
students reported at or above average on the Valuing scale.                    a 5-point Likert-type scale ranging from 1 (strongly disagree)
For the study sample, the scale had a Cronbach’s alpha value                   to 5 (strongly agree). The modal response for all students
of .79, and a range of 1–5.                                                    was “agree,” with 34%–42% (depending on core subject)
   The School Success Profile School Engagement subscale                       of students either agreeing or strongly agreeing that their
(Bowen, Rose, & Bowen, 2005) measures a student’s positive                     teachers used career and job examples. The CRI measure in-
anticipation of attending school, using three items that as-                   cluded in the model was the eighth-grade student–reported
sess the extent of the student’s excitement of being in school                 CRI aggregated at the teacher team level. In addition, the
and looking forward to learning at school. The School En-                      student-level deviation from the aggregated teacher team
gagement scale includes items such as “school is fun and                       measure was included as a control for measurement error.
exciting,” “I look forward to going to school,” and “I look
forward to learning new things.” The mean of the three                            Covariates. A series of covariates were included in the
items was used for the School Engagement scale. If more                        models to control for the effects of student preassignment
than one item was missing, then the scale score was missing.                   and school effects that may have remained after randomiza-
The threshold used for the binary outcome variable was the                     tion, and to control for effects that, if unaddressed, might
mean. Overall, 58% of the students reported positive antici-                   have inadvertently confounded the estimation of the treat-
pation at or above average on the School Engagement scale.                     ment assignment and dosage effects. Student-level covari-
For this study sample, the scale had a Cronbach’s alpha value                  ates included indicators of minority status, gender, and an
of .80 and a range of 1–5.                                                     indicator of free or reduced-price lunch status that was used
32                                                                                            The Journal of Educational Research

as a proxy measure of family socioeconomic status. Other         almost no missing individual values left to substantively in-
student-level variables were tested but omitted from the fi-     fluence parameter estimates. Given the minimal difference
nal models because sensitivity testing indicated they were       between imputation/deletion and complete case analysis, we
not statistically significant and did not substantively alter    proceeded with complete case analysis.
the treatment effect; these included indicators of academic
giftedness, special education, and students living with sin-     Analysis
gle parents. In addition, the models included measures of           Descriptive analysis. Descriptive methods were used to ex-
student deviation from the aggregated teacher team CRI           amine the changes over time in school valuing and engage-
dosage measure to control for variation between students in      ment as well as their relationship with CRI. The decline in
perception of CRI.                                               middle school psychosocial engagement as well as the corre-
   School-level covariates were treated differently in the       lation between CRI and psychosocial engagement measures
CareerStart treatment and the CRI models. The treatment          were illustrated using descriptive methods.
models included random intercepts for schools. In addition
to these random effects, the CareerStart treatment model in-
cluded two school-level measures. First, an indicator variable      Multilevel modeling. The data were multilevel because (a)
of Equity Plus status was included in the model. Schools were    the units of random assignment (school) and CRI (teacher
designated as Equity Plus schools by the district based on the   team) were not the same as the unit of analysis (student),
percent of students using free or reduced-price lunch and        and (b) because of the multistage sampling procedure used.
other need measures. Equity Plus schools were provided ad-       The multilevel structure cannot be ignored without threats
ditional resources for smaller classes and additional student    to the validity of statistical conclusions in the form of under-
services. Second, an aggregated measure of school poverty        estimated standard errors for school- or team-level variables.
(the percentage of students receiving free or reduced-price      Further, because the outcomes were binary, nonlinear mod-
lunch) was included in the CareerStart treatment model. For      eling was an appropriate choice. Therefore, we used hierar-
the CRI models, school indicator variables were included to      chical generalized linear modeling (HGLM; Raudenbush &
control for fixed effects at the school level.                   Bryk, 2002) to estimate nonlinear models with appropriately
                                                                 adjusted standard errors. With binary outcomes, the depen-
   Missing data. We conducted an analysis of the missing         dent variable was assumed to have a binomial distribution,
data to determine the extent of missingness in the data and      and a logit function was used to link the probability (Y) of
how to handle the missing values. There were missing values      high engagement at the end of Grade 8 to a linear model.
throughout the data, including on student variables such as         The pseudo-interclass correlation (ICC) measures recom-
female (number missing = 51), minority (number missing =         mended by Snijders and Boskers (1999) for multilevel logis-
31), and single-parent status (number missing = 107). The        tic models ranged from 0.02 to 0.07 for the four models. As
dependent variables were missing in only those cases for         long as the pseudo-ICC is not zero, then it is sufficient to
which more than half of the items were missing; in cases in      have an effect on the standard error and a multilevel model is
which fewer than half of the items were missing, we used a       appropriate. The design effect size, which incorporates both
process by which only valid responses were counted (Schafer      the ICC and the sample sizes, ranges from 3.45 to 16.17 for
& Graham, 2002). Nevertheless, 21.62% of the sample was          the four models (see results tables). A design effect size of 2
missing data for school valuing, and 21.59% was missing data     or higher suggests that multilevel modeling should be used
for school engagement. Although student-reported CRI data        (Maas & Hox, 2005).
were missing for 15.81% of the sample, we assumed that the          The percent of variance explained in the models is calcu-
aggregation of student reports to the teacher level for the      lated for Level 2, as there is no error in the models at Level
independent variable in the CRI models minimized the bias        1. The variance, which ranges from −42.1% to 55.6% for
that nonrandom missing values may have caused.                   the four models is reported in the results tables. The nega-
   Prior to estimating the models, we examined the distri-       tive percent change for the School Engagement/CareerStart
bution of missingness (i.e., whether the missing values were     Treatment model is consistent with the finding that the
missing completely at random [MCAR]) using the Little            treatment was not significant in that model.
(1988) test for MCAR. At the individual level, the findings
                                                                    CareerStart treatment model. The independent variable
of the Little test suggested that the missing values were not
                                                                 for CareerStart treatment was entered into a two-level
MCAR; therefore, the resulting parameter bias could be re-
                                                                 (school/student) multilevel model.
duced using a strategy such as multiple imputation (Schafer,
1997). However, after following the recommendation that
                                                                   Level 1 (student):
imputed values of the dependent variable should be deleted
after imputation to improve efficiency of the parameter esti-                  
mates (von Hippel, 2007), we found that the benefits from            ln            = β0 j + β1 j femalei j + β2 j minorityi j
                                                                          1 − y ij
using multiple imputation would be minimal. After list-
wise deletion of the imputed dependent variable, there were              + β3 j frli j + β4 j value6i j + β5 j value7i j
The Journal of Educational Research                                                                                                                33

  Level 2 (school):                                                    models were run using SAS 9.2 PROC GLIMMIX (SAS
                                                                       Institute, Cary, NC).
   β0 j = γ00 + γ01 EquityPlus j + γ02 Sch poverty j
              + γ03 CSTreatment j + u 0 j                              Results
   β1 j = γ10 , β2 j = γ20 , β3 j = γ30 , β4 j = γ40 , β5 j = γ50      Student Engagement During Middle School

   In the Level 1 model, minority represents minority stu-                Consistent with findings from other studies (Woolley &
dents, including American Indian, African American, and                Bowen, 2007), we found that measures of psychosocial en-
Hispanic students; frl represents socioeconomic status (i.e.,          gagement declined over the middle school years (Table 2).
students receiving free or reduced-cost lunch); value6 repre-          For example, the mean score on student-reported engage-
sents the Grade 6 valuing score; and value7 represents the             ment declined from 3.53 to 3.22 between the spring of Grade
Grade 7 valuing score. In the Level 2 model, EquityPlus rep-           6 and the spring of Grade 8. Similarly, the mean score of
resents whether the school was assigned EquityPlus status,             student-reported valuing declined from 4.14 to 3.95 over
sch poverty represents the percentage of students at the               the same period. Because the students had a year of middle
school receiving free or reduced-cost lunch, and CSTreat-              school prior to the first engagement measure, this decline
ment represents the school’s status as randomly assigned to            includes just 2 years of middle school (in the study location,
either the treatment or the control condition. The combined            middle school comprised Grades 6, 7, and 8). A similar anal-
model represents an effort to obtain unbiased estimates of             ysis with an earlier cohort that included a pretest measure
the CareerStart treatment effect (γ 03 ).                              administered before the Grade 6 year found a greater decline
                                                                       over the full 3 years of middle school.
                                                                          Table 3 shows the cumulative effect of CRI on psychoso-
   CRI model. The independent variable for CRI was en-
                                                                       cial engagement in Grade 8 based on teacher use of career
tered in a two-level (teacher team, student) multilevel
                                                                       examples in the four core classes over the 3 years of middle
model. Variation at the school level was controlled us-
                                                                       school (range 0–12). For example, only 34% of students who
ing school fixed effects (dummy variables for 13 of the 14
                                                                       reported hearing no career examples in any of their 12 core
                                                                       middle school classes also reported high school engagement.
                                                                       However, 83% of students who reported hearing career ex-
  Level 1 (student):
                                                                       amples in 11 or 12 of their core classes during middle school
                                                                     also reported high levels of school engagement. In addition,
               y                                                       although not shown in the table, even after controlling for
     ln                   = β0 j + β1 j femalei j + β2 j minorityi j
              1−y    ij                                                gender, race, socioeconomic status, single-parent household
                                                                       status, academic giftedness, and previous engagement mea-
          + β3 j frli j + β4 j value6i j + β5 j value7i j
                                                                       sures, the increase in school valuing and engagement with
          + β6 j CRIvari j + 13school indicators                       additional CRI persisted. In other words, we found that stu-
                                                                       dents whose teachers often provided career examples were
  Level 2 (teacher team):                                              more likely to maintain high levels of school valuing and
                                                                       engagement over time.
   β0 j = γ00 + γ01 teamCRI j + u 0 j
   β1 j = γ10 , β2 j = γ20 , β3 j = γ30 ,
   β4 j = γ40 , β5 j = γ50 , β6 j = γ60
                                                                          TABLE 2. Psychosocial Engagement During Middle
                                                                          School Years
   In the Level 1 model, CRIvar represents the student de-
viation from the mean teacher team CRI, and teamCRI rep-                                              Fall   Spring Spring Spring
resents the mean teacher team CRI. The combined model                                                Grade 6 Grade 6 Grade 7 Grade 8
represents an effort to obtain unbiased estimates of the CRI
                                                                          Cohort 1
dosage when the coefficient of interest is γ 01 , the effect of             School valuing             4.24       4.19       4.10       3.97a
the CRI dosage.                                                             School engagement          3.80       3.57       3.42       3.25a
   In the next section, the odds ratios of the multilevel lo-             Cohort 2
gistic models are reported and discussed for each student                   School valuing             N/A        4.14       4.03       3.95a
engagement outcome. The odds ratios for the CareerStart                     School engagement          N/A        3.53       3.32       3.22a
treatment model compare the odds of the outcome under                     Note. N/A = not applicable.
the CareerStart treatment to the odds under the control                   aSignificantly smaller than first measurement either fall or spring of

condition. The odds ratios for the CRI model represent the                Grade 6 at p < .001.
odds of the outcome for each unit increase in CRI. Both
34                                                                                                          The Journal of Educational Research

                                                                                 Not surprisingly, these data indicated that students who re-
     TABLE 3. Percent of Students with High Engagement                           ported higher levels of school valuing in their Grade 6 and
     and Valuing by Student-Reported CRI                                         Grade 7 surveys also reported higher levels of school valuing
                                                                                 in the Grade 8 survey.
     Number of core classes student                          School                 Although the CareerStart treatment effect was positive
     reported hearing career examples             School     engage-
     (Grades 6–8)                                 valuing     ment               for school engagement, as measured by the School Success
                                                                                 Profile Engagement scale (odds ratio = 1.15), this finding was
     0                                             78%         34%               not statistically significant. As was true for school valuing,
     1 or 2                                        85%         48%               levels for school engagement were linked to the student’s
     3 or 4                                        87%         44%
                                                                                 ethnicity and their own previous levels of engagement, but
     5 or 6                                        91%         54%
     7 or 8                                        94%         64%               no other control factors contributed to the model for en-
     9 or 10                                       98%         67%               gagement. It is interesting to note that poverty status, which
     11 or 12                                      97%         83%               was measured by either students using free and reduced-price
                                                                                 lunch programs or the school’s percentage of students using
     Note. CRI = career-relevant instruction.
                                                                                 these lunch programs did not contribute to either model.

                                                                                 CRI Model
CareerStart Treatment Model
                                                                                    Table 5 presents the results of the multilevel logistic
   Table 4 presents the results of the multilevel logistic model                 model for student-reported CRI. The model found that an
for the CareerStart treatment effect. The model found that                       additional unit of teacher team CRI was associated with stu-
students in the CareerStart treatment schools were 41%                           dents being 31% more likely to report above-average levels of
more likely to report above-average levels of school valu-                       school valuing and 59% more likely to report above-average
ing as compared with students in control schools (odds ratio                     levels of school engagement than students with lower levels
= 1.41), indicating a significant positive effect for Career-                    of teacher team CRI. Because school valuing and engage-
Start. In addition, the analysis indicated that boys reported                    ment levels can be linked to such factors as the students’
significantly lower valuing scores than girls, and Asian and                     ethnicity, gender, poverty, as well as each student’s own pre-
Caucasian students reported significantly lower valuing than                     vious levels of engagement, the analysis controlled for these
American Indian, African American, and Hispanic students.                        factors.
The analysis also controlled for student-reported valuing lev-                      In summary, boys reported significantly lower school valu-
els measured in the spring of their Grade 6 and Grade 7 years.                   ing scores than girls, consistent with other research (Voelkl,

     TABLE 4. Multilevel Logistic Model Odds Ratios for CareerStart Treatment Effect

                                                                                     Valuing                              School engagement

     Intercept                                                                          —                                        —
     Student, female (m = 0, f = 1)                                                   1.74                                      1.16
                                                                                          ∗∗∗                                      ∗∗∗
     Student, minority (Asian–Caucasian = 0,                                          1.93                                     1.68
       African American–Hispanic = 1)
     Student, poverty (nFRL = 0, FRL = 1)                                              0.86                                     1.10
                                                                                          ∗∗∗                                      ∗∗∗
     Student, Grade 6 outcome measure                                                 1.63                                     1.54
                                                                                          ∗∗∗                                      ∗∗∗
     Student, Grade 7 outcome measure                                                 4.24                                     2.46
     School, EquityPlus (0 = not EquityPlus,                                           0.56                                     1.07
       1 = EquityPlus)
     School, percent of students using FRL                                             1.76                                       1.31
     CareerStart treatment                                                            1.41                                        1.15
     Random effect (school)                                                            0.03                                      0.03
     n                                                                               2,270.0                                   2,270.0
     −2 log likelihood                                                               10662.8                                   10500.3
     Generalized χ 2                                                                  2439.1                                    2280.6
     Design effect size                                                                6.71                                      16.17
     Variance explained                                                               55.6%                                    –42.1%

     Note. FRL = receiving free or reduced-price lunch; nFRL = not receiving free or reduced-price lunch.
     ∗         ∗∗       ∗∗∗
       p < .05. p < .01. p < .001.
The Journal of Educational Research                                                                                                                35

   TABLE 5. Multilevel Logistic Model Odds Ratios for CRI

                                                                                   Valuing                                     School engagement

   Intercept                                                                          —                                                 —
   Student, female (m = 0, f = 1)                                                   1.77                                               1.15
                                                                                        ∗∗∗                                               ∗∗∗
   Student, minority (Asian–Caucasian = 0,                                          1.92                                              1.73
     African American–Hispanic = 1)
   Student, poverty (nFRL = 0, FRL = 1)                                              0.80†                                             0.99
                                                                                        ∗∗∗                                               ∗∗∗
   Student, Grade 6 outcome measure                                                 1.63                                              1.54
                                                                                        ∗∗∗                                               ∗∗∗
   Student, Grade 7 outcome measure                                                 3.99                                              2.47
                                                                                        ∗∗∗                                               ∗∗∗
   Student, variation from Team CRI                                                 1.16                                              1.16
   School indicator variables (13)
                                                                                         ∗∗                                                  ∗∗∗
   Team, Grade 8 CRI                                                                1.31                                             1.59
   Random effect (teacher team)                                                      0.06                                              0.09
   n                                                                               2,222.0                                           2,221.0
   −2 log likelihood                                                               10705.0                                           10498.0
   Generalized χ 2                                                                  2432.9                                            2188.0
   Design effect size                                                                3.45                                              6.68
   Variance explained                                                               36.5%                                             24.3%

   Note. CRI = career-relevant instruction; FRL = receiving free or reduced-price lunch; nFRL = not receiving free or reduced-price lunch.
   †p < .10. ∗ p < .05. ∗∗ p < .01. ∗ ∗ ∗ p < .001.

1996). Asian and Caucasian students also reported signifi-                     additional career examples in core classes increased school
cantly lower levels of school valuing and engagement than                      valuing and school engagement.
did American Indian, African American, and Hispanic stu-                          These findings are consistent with previous research sug-
dents. These findings are consistent with previous research                    gesting that student career exploration can have positive ef-
indicating that school psychosocial engagement may have                        fects on school engagement (Castellano et al., 2003; Kenny
a more significant role in the lives of minority students                      et al., 2006). The data also support the theoretical propo-
(Wilson, 1996). The analysis controlled for student-reported                   sitions of possible-selves theory (Markus & Nurius, 1986),
school valuing and engagement levels that were measured                        which proposes that students will attend to their learning
in the spring of their Grade 6 and Grade 7 years. These data                   and view school as having more value when class content is
indicated that students who reported higher levels of school                   provided in the context of information that the student val-
valuing and engagement during Grades 6 and 7 also reported                     ues or considers relevant for future benefits and choices. The
higher levels of school valuing and engagement in Grade 8.                     potential contribution of content and course relevancy has
School fixed-effects were included in the model by means of                    been proposed by the Gates Foundation (Vander Ark, 2005)
indicator variables (not shown).                                               and other education reformers, and the present research sup-
                                                                               ports this direction of education curricular reform.
Discussion                                                                        One of the benefits of CareerStart may be its simplicity
                                                                               in implementation. Because teachers were not asked to add
   The findings from this longitudinal study of middle school                  new substantive content to their already demanding course
students indicated that the students in the schools randomly                   schedules, the intervention appears to have been readily ac-
selected to implement CareerStart were significantly more                      cepted by the teachers. In addition, teachers may have been
likely to value their education than students in the control                   receptive to CareerStart because the lessons were prepared
schools, as measured by a standardized scale that assessed                     by peer teachers and all the materials needed to carry out the
whether students viewed school as important and providing                      lessons were available online, which aided teachers in their
useful information and knowledge that will help them in                        instructional planning. The teacher training and coaching
the future. This finding indicates that the CareerStart pro-                   was kept as simple as possible, and much of that support
gram is a promising approach to teacher pedagogy and leads                     was provided by the lead teachers or curriculum coordina-
to an important type of psychosocial engagement among                          tors in the schools themselves. This use of a school’s existing
students. An additional analysis found that students who                       infrastructure helped make CareerStart a school-based in-
reported higher levels of CRI from their core teachers over                    tervention with a broad array of instructional staff engaged
the 3 years of middle school, regardless of whether they were                  in the effort.
in a treatment or control school, also reported higher levels                     Although the treatment resulted in a positive effect for
of school valuing and school engagement, indicating that                       both psychosocial measures of engagement, only our measure
36                                                                                           The Journal of Educational Research

of school valuing (Voelkl, 1996) was significant. The non-         given the nonnormal distribution of the responses to the out-
significant findings for the School Success Profile measure        come measures, dichotomous outcome variables were used
of school engagement (Bowen et al., 2005) suggests that the        in the models.
CareerStart intervention may promote in students a more
positive set of beliefs that participating in school is impor-     Implications for Teachers and Schools
tant and will yield longer-term benefits (i.e., valuing) but
not necessarily result in students seeing school in the more          One benefit of the CareerStart approach to middle school
immediate sense as being fun and something to look forward         instruction is that the program is relatively easy to imple-
to each day (i.e., engaging). Perhaps the career connections       ment and support. Many teachers already routinely illustrate
that students are making in the classroom are increasing the       their teaching with examples from the students’ environ-
overall relevance of the school experience (Orthner, 2007),        ment to help students recognize the relevance of the lesson
a goal of the CareerStart program, but not having as much          content. Such teachers understand that contemporary rele-
impact on the day-to-day excitement that students may be           vance helps them connect better to their students’ interests
hoping for from their school experiences.                          and attention. However, the benefits of students recognizing
   On a positive note, however, higher levels of CRI, as re-       the relevance of content to their futures are not as well un-
ported by students, resulted in significantly higher effects       derstood; therefore, teachers may need help in understand-
for both measures of psychosocial engagement. There was            ing the value that students place on connecting their middle
a strong and significant association between the number of         school instruction to future education and career possibili-
core teachers students reported as providing more frequent         ties. The findings reported here indicate that the CareerStart
career examples and higher levels of school valuing and            approach and CRI does benefit students’ attention to their
engagement. This confirmed our expectations and may indi-          schooling, thus promoting the instructional environment
cate that when students themselves experience career rele-         that school teachers and principals want to nourish.
vance in instruction, and when this is repeated in class after        CRI can be implemented by simply encouraging teachers
class, and grade after grade, the connection to both short-        to meet and share examples of jobs and careers in which
and long-term benefits of being in school are realized. Thus,      the content of their courses has application. We have found
CareerStart, as a program, promoted CRI and student school         inviting parents and employers into the classrooms to talk
valuing but the more of this instruction that occurred, and        about how they use mathematics, science, and language con-
the more teachers who used this pedagogy over the 3 mid-           cepts can be helpful to students in realizing the relevance of
dle school years, the more likely that the benefits were seen      the content that is explored and taught in their classes. Some
by students in the short term (engagement) and long term           middle schools have even offered job fairs where employers
(valuing).                                                         are invited to talk about the jobs available in their businesses
                                                                   or industries and how important the employees’ education is
Limitations                                                        to being successful in those jobs. One school had a checklist
                                                                   for students that asked employers what kinds of reading and
   Two primary limitations of this analysis may have diluted       writing were required of their employees, as well as the spe-
the treatment effect, even though the reported treatment           cific use of mathematics, science, or civic knowledge their
effect was significant for one of the two engagement mea-          employees needed to demonstrate. This depth of information
sures. First, the low effective sample size of 14 schools in the   and employer feedback can help engage students in an ex-
cluster-randomized evaluation design might have limited the        ploration of their future education and career opportunities
treatment effect. Including a larger number of schools would       that will also help students connect their present learning to
have increased the robustness of this study. Second, uneven        future career thinking. Moreover, as suggested by our find-
treatment fidelity may also have diluted the treatment effect      ings, it is this hope—that an investment in education will
due to low implementation in some treatment schools and            pay dividends in the future—that serves both to improve
CRI by some teachers in control schools.                           student engagement and to create the value students place
   Other study limitations include that the CareerStart pro-       on being in and staying in school.
gram was evaluated in a single district, native to the study
district, and in part developed by teachers in the study dis-      Conclusion
trict. In addition, some students moved between treatment
and control schools. To account for this mobility, the anal-          The evaluation of teachers’ use of CRI in middle school
ysis used a decision rule that called for all students who         classrooms yielded a variety of promising findings. The re-
changed treatment conditions to be assigned to the treat-          sults confirm that students who report being exposed to more
ment group. However, this rule may have further diluted            career examples as part of their core subject instruction are
the results if, as a group, mobile students have lower school      more likely to report higher levels of both school valuing
engagement than other students. Finally, the use of dichoto-       and school engagement. Students in CareerStart treatment
mous outcome measures may be considered a limitation in            schools also reported significantly higher levels of school
that a certain amount of variance was eliminated. However,         valuing than students in the control schools. These findings
The Journal of Educational Research                                                                                                                           37

support the importance of the relevance component of in-                         Kenny, M. E., Blustein, D. L., Haase, R. A., Jackson, J., & Perry, J.
struction that has been increasingly promoted by the Gates                          C. (2006). Setting the stage: Career development and the student
                                                                                    engagement process. Journal of Counseling Psychology, 53, 272–279.
Foundation and others as a potential asset for transform-                           doi:10.1037/0022–0167.53.2.272
ing public education. In this analysis, we confirmed through                     Klem, A. M., & Connell, J. P. (2004). Relationships matter: Linking teacher
an evidence-based approach that teaching methods or ped-                            support to student engagement and achievement. Journal of School Health,
                                                                                    74, 262–274. doi:10.1111/j.1746–1561.2004.tb08283.x
agogy can target students’ motivation to learn, offering a                       Little, R. J. A. (1988). A test of missing completely at random for multivari-
practical approach to education reform that takes advantage                         ate data with missing values. Journal of the American Statistical Association,
of and augments the capacity of our present instructional en-                      83, 1198–1202. doi:10.2307/2290157
                                                                                 Maas, C. J. M., & Hox, J. J. (2005). Sufficient sample sizes for multilevel
vironment in the middle grades. More analyses are planned                          modeling. Methodology, 1, 86–89. doi:10.1027/1614–2241.1.3.86
to strengthen our understanding of how this instructional                        Markus, H. R., & Nurius, P. (1986). Possible selves. American Psychologist,
strategy benefits selected student groups.                                         41, 954–969. doi:10.1037/0003–066X.41.9.954
                                                                                 Orthner, D. K. (2007). Public schools: Building capacity for hope and
                                                                                   opportunity. In J. Edwards, M. Crain, & A. L. Kalleberg (Eds.). Ending
                         ACKNOWLEDGMENT                                            poverty: How to restore the American dream (pp. 218–229). New York, NY:
                                                                                   New Press.
  An earlier version of this study was presented at the American Education       Orthner, D. K., Akos, P., Rose, R., Jones-Sanpei, H., Mercado, M., &
Research Association meeting in Denver, Colorado, on May 2, 2010.                  Woolley, M. (2010). CareerStart: A middle school student engage-
                                                                                   ment and academic achievement program. Children and Schools, 32, 223–
                              REFERENCES                                           234.
                                                                                 Orthner, D. K., Cook, P., Rose, R., & Randolph, K. A. (2002). Welfare
Akos, P., & Galassi, J. (2004). Middle and high school transitions as              reform, poverty, and children’s performance in school: Challenges for
  viewed by students, parents, and teachers. Professional School Counsel-          the school community. Children and Schools, 24, 105–121.
  ing, 7, 212–221.                                                               Oyserman, D. (2008). Possible selves: Identity-based motivation and school
Akos, P., Lambie, G., Milsom, A., & Gilbert, K. (2007). Early adoles-              success. In H. Marsh, R. Craven, & D., McInerney (Eds.), Self-processes,
  cents’ aspirations and academic tracking: An exploratory investigation.          learning and enabling human potential dynamic new approaches (Vol. 3, pp.
  Professional School Counseling, 11(1), 57–64.                                    269–288). Charlotte, NC: Information Age Press.
Akos, P., Shoffner, M., & Ellis, M. (2007). Mathematics placement and the        Perry, J. C. (2008). School engagement among urban youth of color: Crite-
  transition to middle school. Professional School Counseling, 10, 238–244.        rion pattern effects of vocational exploration and racial identity. Journal
Alexander, K. L., Entwisle, D. R., & Horsey, C. S. (1997). From first grade        of Career Development, 34, 397–422. doi:10.1177/0894845308316293
  forward: Early foundations of high school dropout. Sociology of Education,     Perry, J. C., Liu, X., & Pabian, Y. (2010). School engagement as a mediator
  70, 87–107. doi:10.2307/2673158                                                  of academic performance among urban youth: The role of career prepara-
Alspaugh, J. W. (1998). Achievement loss associated with the transition to         tion, parental career support, and teacher support. Counseling Psychologist,
  middle school and high school. The Journal of Educational Research, 92,          38, 269–295. doi:10.1177/0011000009349272
  20–25. doi:10.1080/00220679809597572                                           Plank, S. (2001). Career and technical education in the balance: High school
Anderman, E. M., Maehr, M. L., & Midgley, C. (1999). Declining motiva-             persistence, academic achievement and post-secondary destinations. St. Paul,
  tion after the transition to middle school: Schools can make a difference.       MN: National Research Center for Career and Technical Education.
  Journal of Research and Development in Education, 32, 131–147.                 Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Appli-
Bishop, P. A., & Pflaum, S. W. (2005). Student perceptions of action,              cations and data analysis methods (Vol. 1, 2nd ed.). Thousand Oaks, CA:
  relevance and pace. Middle School Journal, 36, 4–12.                             Sage.
Blafanz, R., Fox, J., Bridgeland, J., & McNaught, M. (2009). Grad nation:        Schafer, J. L. (1997). Analysis of incomplete multivariate data. London, UK:
  A guidebook to help communities’ tackle the dropout crisis. Washington,          Chapman and Hall.
  DC: America’s Promise Alliance. Retrieved from http://www.everyday-            Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of                                               the state of the art. Psychological Methods, 7, 147–177. doi:10.1037/
Borman, G. D., & Overman, L. T. (2004). Academic resilience in mathe-               1082–989X.7.2.147
  matics among poor and minority students. The Elementary School Journal,        Snijders, T. A. B.. & Boskers, R. J. (1999). Multilevel analysis: An introduction
  104, 177–195. doi:10.1086/499748.                                                 to basic and advanced multilevel modeling. Newbury Park, CA: Sage.
Bowen, G. L., Rose, R. A., & Bowen, N. A. (2005). The reliability and validity   Trice, A. D., Hughes, M. A., Odom, C., Woods, K., & McClellan, N. C.
  of the School Success Profile. Philadelphia, PA: Xlibris Press.                   (1995). The origins of children’s career aspirations: IV. Testing hypothe-
Castellano, M., Stringfield, S., & Stone, J. R. (2003). Secondary career            ses from four theories. Career Development Quarterly, 43, 307–322.
  and technical education and comprehensive school reform: Implications          Trice, A. D., & King, R. (1991). Stability of kindergarten children’s career
  for research and practice. Review of Educational Research, 73, 231–272.           aspirations. Psychological Reports, 68, 1378. doi:10.2466/PRO.68.4.1378-
  doi:10.3102/00346543073002231                                                     1378
Eccles, J. S., Wigfield, A., Midgley, C., Reuman, D., Mac Iver, D., &            U.S. Department of Education. (2010a). Table 6. Status dropout rates
  Feldlaufer, J. (1993). Negative effects of traditional middle schools             and number and distribution of dropouts of 16 through 24-year-olds, by
  on students’ motivation. Elementary School Journal, 93, 553–574.                  selected background characteristics: October 2008. Trends in High School
  doi:10.1086/461740                                                                Dropout and Completion Rates in the United States: 1972–2008. Retrieved
Finn, J. D., & Rock, D. A. (1997). Academic success among students                  from 06.asp
  at-risk for school failure. Journal of Applied Psychology, 82, 221–234.        U.S. Department of Education. (2010b). Table 110. High school grad-
  doi:10.1037/0021–9010.82.2.221                                                    uates, by sex and control of school: Selected years, 1869–70 through
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School en-              2019–20. Digest of Education Statistics. Retrieved from
  gagement: Potential of the concept, state of the evidence. Review of              programs/digest/d10/tables/dt10 110.asp
  Educational Research, 74, 59–109. doi:10.3102/00346543074001059                U.S. Department of Education (2010b). “Table 6. Status dropout rates
Gonzalez, R., & Padilla, A. M. (1997). The academic resilience of Mexican-          and number and distribution of dropouts of 16 through 24-year-olds, by
  American high school students. Hispanic Journal of Behavioral Sciences,           selected background characteristics: October 2008.” Trends in High School
  19, 301–317. doi:10.1177/07399863970193004                                        Dropout and Completion Rates in the United States: 1972–2008. Retrieved
Gottfredson, L. S. (1981). Circumscription and compromise: A develop-               from 06.asp
  mental theory of occupational aspirations. Journal of Counseling Psychol-      Vander Ark, T. (2005). Every student a college ready graduate. Seattle, WA:
  ogy Monograph, 28, 545–579. doi:10.1037/0022–0167.28.6.545                        Bill & Melinda Gates Foundation.
Imbens, G. W., & Wooldridge (2009). Recent developments in the econo-            Voelkl, K. E. (1996). Measuring students’ identification with school.
  metrics of program evaluation. Journal of Economic Literature, 47, 5–86.          Educational and Psychological Measurement, 56, 760–770. doi:10.1177/
  doi:10.1257/jel.47.1.5                                                            0013164496056005003
You can also read