Missing the Mark: Students Gain Little from Mandating Extra Math and Science Courses
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
ACT Research & Policy
Policy Brief
Missing the Mark: Students Gain Little
from Mandating Extra Math and Science
August 2014 Courses
Richard Buddin and Michelle Croft
Introduction some districts had similar requirements in place
Evidence suggests that For several decades, policymakers have embraced already.4
changing course requirements the goal of preparing students for college and Using student-level data for nine Illinois high
alone may not be enough to careers, particularly for careers in the area of school graduation classes, this report examines
prepare students for college mathematics and science. The recent emphasis the relationship between high school graduation
and career. on these STEM (science, technology, engineering, requirements and student outcomes and
and mathematics) subjects is due to the growth assesses how changes in math and science
of STEM occupations and the perceived shortage requirements affected student outcomes. In
of qualified workers to fill these positions.1 There August 2005, Illinois enacted Public Act 94-0676
is a concern that many students do not currently aimed at increasing the rigor of high school by
have the level of STEM capabilities necessary for mandating more stringent high school graduation
high-skill STEM professions such as engineering requirements. Prior to the reform, most districts
or even for low-skill STEM positions in fields such allowed high school students to graduate with
as manufacturing.2 only two years of math (60%) and fewer (15%)
To better prepare students for these STEM required only one year of science.5 The law set a
careers, many states have increased the state minimum graduation requirement of three
mandated minimum course requirements in math years of math—including Algebra I and Geometry
and science over the past several years. Twenty- content—and two years of science.6 The law
seven states required at least one more year of was phased in over a four-year period. Each year
math instruction for the high school graduation from 2005 to 2008, entering freshman were
class of 2013 than for the class of 2006. 3 responsible for completing more coursework,
Similarly, 19 states required an additional year of with the full reform package applicable for
science for 2013 high school graduates compared the graduating class of 2013. The mandatory
to 2006 graduates. In 2013, 42 states required state math requirement rose from two years for
students to take at least three years of math, and graduation classes through 2008 to three years
37 states required at least three years of science. for the 2009 graduation class. Two years of
These policies aim to expose more students to science were required by the state for the 2011
sophisticated math and science concepts, and graduation compared to a one-year requirement
the extra rigor is intended to improve student for earlier classes.
outcomes in college or the workforce. Although This study used ACT,7 National Student
these state minimum requirements are binding on Clearinghouse,8 and Illinois State Board of
all students, in practice they only affect a minority Education survey9 data to examine the effects
of students since most college-bound students of the reform for public school students,10
have traditionally exceeded these minimums, and
www.act.org/research-policy
research.policy@act.org for more information.
© 2014by ACT, Inc. All rights reserved. ACT® is a registered trademark of ACT, Inc., in the USA and other countries. 1762ACT Research & Policy Missing the Mark: Students Gain Little from Mandating Extra Math and Science Courses
particularly students in the bottom half of Figure 1. Trends in math course taking by district type, class rank, and high school
their graduating class. We compared trends graduation year, 2005–2013
in course taking, achievement, and college
100
enrollment for districts that were affected
Percent of Students Taking 3 Years of Math
Top half, treated
by the new statewide requirements (treated
Top half, untreated
districts) relative to other districts that 90
already required the mandated levels of math
Bottom half, treated
and science coursework prior to the law’s 80
Bottom half, untreated
enactment (untreated districts). The new
requirements should expose more students 70
to an advanced curriculum often targeted
for college-motivated students. Specifically, 60
the study addresses the following three
questions: Higher Math Requirement
50
1. Have the new graduation requirements 2005 2006 2007 2008 2009 2010 2011 2012 2013
increased math and science course Graduation Year
taking?
2. Has student achievement improved in
math and science because of the policy at Risk, which recommended that students high-achieving graduates were already
change? graduating from high school should take taking three or more years of math prior
3. Have college enrollment trends been four years of English language arts; three to the graduation requirement reform
affected by the new law? years each of mathematics, science, and (Figure 1). This was the same whether
social studies; and a half-year of computer the student was in a treated or untreated
For each research question, we provide
science.11 A number of states increased their district. Interestingly, neither the treated nor
a brief overview and examine the law’s
graduation requirements in response to A untreated districts reached 100% compliance
effectiveness by comparing trends for treated
Nation at Risk, but few did so to the report’s with the requirements. It is possible that the
districts (i.e., those affected by the new law)
recommended minimums. 12
requirements are not enforced in practice
versus untreated districts (i.e., those that had
or are satisfied with credits for repeating
met or exceeded the requirements of the Past research examining the relationship
courses or for foundational or business math
2005 statute prior to its enactment). In each between graduation requirements and
courses.
case, we compared trends for students in the course taking has found that graduation
top and bottom halves of their graduation requirements affect only a few students Approximately two-thirds of low-ranking
classes. A priori, we expected that the because most students would have taken students in both the treated and untreated
law would have a larger effect on lower- the courses regardless of the requirements.13 districts were taking at least three years of
performing students since these students For those few students who would not math in 2005. By 2013, math course taking
typically took fewer math and science voluntarily take the courses, some studies increased to nearly 80% for the low-ranking
courses. If the law was effective at improving found an increase in coursework associated students. The increase in course taking is
student outcomes, then we should observe with graduation requirements for certain encouraging; however, the increase in course
that course taking, achievement, and college subgroups of students such as low-achieving taking occurred whether students were in
enrollments would improve at a faster pace in students.14 the treated or untreated districts. Although
the treated than in the untreated districts. more students were taking at least three
Math more pure math courses than before, the
Course Taking In Illinois the math requirement was raised trend in districts affected by the higher math
Interest in student course taking dates back to three years of math courses. Consistent graduation requirement differed little, if at
thirty years with the publication of A Nation with prior research, about 90% of the all, from the trend in unaffected districts. It
2ACT Research & Policy Missing the Mark: Students Gain Little from Mandating Extra Math and Science Courses
is possible that the enactment of the state Figure 2. Trends in science course taking by district type, class rank, and high school
law encouraged the untreated districts to graduation year, 2005–2013
enforce their preexisting policy, particularly 100
Percent of Students Taking 2 Years of Science
with low-ranking students, but we are unable Top half, untreated
to attribute the rise in math course taking 95 Top half, treated
Bottom half, untreated
directly to the law.
90
Bottom half, treated
Science
85
The law had little effect on science course
taking for high-ranking students, but it did
80
seem to improve course taking for low-
ranking students. In the treated districts, 75
78% of low-ranking students took two years
Higher Science Requirement
of science in 2005 as compared with 88% 70
in 2013. This 10-percentage-point increase 2005 2006 2007 2008 2009 2010 2011 2012 2013
compares favorably to a 5-percentage-point Graduation Year
increase for low-ranking students in the
untreated districts (Figure 2). The science
Figure 3. Trends in ACT Mathematics Test scores by district type, class rank, and high school
results suggest that the new law is closing graduation year, 2005–2013
a portion of the science-course-taking gap
24
between untreated and treated districts for
low-ranked students. 23 Top half, treated
ACT Mathematics Test Score
22
Achievement Top half, untreated
Lawmakers hope that exposure to more 21
advanced coursework in math and science 20
will result in higher student achievement.
19
Generally, students who complete core
coursework have much higher achievement 18 Bottom half, treated
scores than other students. However,
15 Bottom half, untreated
17
students completing a core coursework Higher Math Requirement
16
voluntarily as preparation for college are 2005 2006 2007 2008 2009 2010 2011 2012 2013
likely different from students who are Graduation Year
mandated to take the courses. The mandated
students may be less motivated or prepared
for advanced coursework and may not districts for either high- or low-ranked the overall increased math course load that
perform as well as students enrolled in those students (Figure 3). ACT math scores was the trend in all districts over this period.
courses on a voluntary basis. increased by 0.7 points for low-ranking The higher scores could also be related
students in untreated districts compared to to other changes in education policy and
Math 0.6 points for low-ranking students in treated practices in the state or unmeasured changes
Math achievement was unaffected by the districts. in the student composition over this period.
law. Mathematics Test scores on the ACT®
Since the gains were not significantly
college readiness assessment did rise Science
different in treated and untreated districts,
by small fractions of a point for all four Similar to math, the change in science
these test score gains may not be tied to the
groups from 2005 to 2013, but there was requirements did not seem to have an effect
new mandated math course requirements.
not a statistically significant difference in on science achievement. ACT Science Test
Instead, the slightly higher scores may reflect
the trends between treated and untreated scores were comparable across treated
3ACT Research & Policy Missing the Mark: Students Gain Little from Mandating Extra Math and Science Courses
and untreated districts in 2005, and they Figure 4. Trends in ACT Science Test scores by district type, class rank, and high school
remained comparable over the period, graduation year, 2005–2013
increasing overall by about 0.7 points 23
(Figure 4). For low-ranking students, the Top half, untreated
trend was statistically smaller for the treated 22 Top half, treated
group than for the untreated group. Science
ACT Science Test Score
21
scores still rose for both subgroups in the
treated districts, but treated districts’ science 20
scores rose at a slower pace than untreated
19
districts’.
18 Bottom half, untreated
This evidence suggests that the two-year
Bottom half, treated
science requirement had little effect on 17
science achievement. If effective, the overall Higher Science Requirement
trend should have been steeper in districts 16
2005 2006 2007 2008 2009 2010 2011 2012 2013
affected by the requirement. Effective reform
Graduation Year
should also have especially increased ACT
Science Test scores for the low-ranking
students who had traditionally taken fewer Figure 5. Trends in immediate college enrollment related to math graduation requirements by
science courses. Achievement growth was district type, class rank, and high school graduation year, 2005–2013
actually lower for low-ranking students in
Percent of Students Immediately Enrolling in College
the treated districts than in the untreated 80
Top half, treated
districts.
75
Top half, untreated
College Enrollment 70
Research shows that students who 65
voluntarily take more rigorous high school
60
coursework are more likely to enroll and
persist in college. Radunzel and Noble, 55
using data from 24,850 ACT-tested high 50 Bottom half, treated
school graduates, found that students who
45 Bottom half, untreated
completed a core curriculum were more Higher Math Requirement
likely to enroll in college and ultimately be 40
2005 2006 2007 2008 2009 2010 2011 2012
successful in obtaining a college degree. 16
Graduation Year
Similarly, Adelman found a relationship
between the highest level of high school
mathematics completed and bachelor’s
degree completion.17 These studies did not
take into account whether the course taking Math the enrollment rate for low-ranking students
was voluntary or mandatory; thus, the findings Although math course taking and rose 2 percentage points in treated districts
may not translate to increased college achievement scores did not appear to be (this trend is insignificantly different from
enrollment under the mandated requirement. affected by the law, there seemed to be zero) and 1 percentage point in the untreated
New requirements effectively target students an effect on college enrollment. College districts. For high-ranking students, the
that may have weaker preparation and enrollment rose faster in districts affected by enrollment rate rose 4 percentage points in
motivation for college rather than the group the higher math requirement than in other treated districts and 2 percentage points in
currently choosing more advanced math and districts (Figure 5). Between 2005 and 2012 untreated districts.
science courses. (2013 enrollment data is not yet available),
4ACT Research & Policy Missing the Mark: Students Gain Little from Mandating Extra Math and Science Courses
Science Figure 6. Trends in immediate college enrollment related to science graduation requirements
by district type, class rank, and high school graduation year, 2005–2013
Unlike in math, the science reform had no
positive effect on college enrollment
Percent of Students Immediately Enrolling in College
(Figure 6). For high-ranking students, college 80
Top half, untreated
enrollments rose 4 percentage points in 75 Top half, treated
treated districts over the eight-year period
70
versus only 2 percentage points in untreated
65
districts. For low-ranking students, enrollment
rates rose 2 percentage points in untreated 60
districts and fell 1 percentage point in treated 55
districts. The evidence shows that college 50
Bottom half, untreated
enrollments grew at slower rates in districts
45
Bottom half, treated
affected by the higher science requirement
40
than in other districts. Higher Science Requirement
35
2005 2006 2007 2008 2009 2010 2011 2012
Discussion
Graduation Year
State policymakers may be unrealistic in
expecting that raising math and science
graduation requirements alone through
state policy will improve student outcomes.
Despite the introduction of higher graduation motivated students who often have weaker “the level of content mastery by the median
requirements in math and science, there preparation for advanced coursework. For students receiving credit for a course with
was little effect on student course taking, students to succeed in these courses, they a given title declines over time.”19 Thus, as
achievement, or college enrollment. From must possess the necessary prerequisite more students are required to take Algebra II,
2005 to 2013, students took more math skills to then take advantage of the advanced for example, the content covered becomes
courses, but the increase was the same for material. Efforts should be focused on early lessened or easier so as to accommodate the
all students, regardless whether the students preparation to ensure that students have various skill levels of incoming students in
were in districts newly subjected to higher better skills by the time they reach high the course.
graduation requirements or were in districts school. For students already in high school,
Overall, as states try to increase the
already requiring the courses. There was also targeted remediation efforts are necessary,
preparedness of students for college and
no effect on math achievement scores, but and the remediation efforts may need to be
career, course requirements alone may
there was an increase in college enrollment. differentiated by ability level, distinguishing
not be a sufficient mechanism for change.
Science course taking did increase more for students with very weak ability from those
Exposing students to advanced material
students in treated than in untreated districts, only slightly behind.18
is an important first step, but we must
but this disproportionate increase did not
Another difficulty is ensuring that the courses recognize that better preparation, better
translate into high science achievement
remain challenging and best prepare instruction, better student commitment, better
scores or an increase in college enrollment.
students for college and career. An parental support, and a host of other factors
The research related to high school unintended side effect of increasing high are needed for students to master these
graduation requirements illustrates that school graduation requirements through a advanced skills. New research is needed
requiring certain courses alone will do little mandate is that students may be completing to identify what policies and practices will
to change student learning and behavior. more coursework that is less advanced. As allow a broader cross section of students to
The shortcoming may reflect that the laws schools expand course offerings, there may embrace and learn advanced concepts.
primarily affect lower-ability and less- be “course credit inflation” which is where
5ACT Research & Policy Missing the Mark: Students Gain Little from Mandating Extra Math and Science Courses
Notes Requirements: Classes 2006 through 2011 was omitted due to its size: there were 27,000
(Denver, CO: Education Commission of the freshmen in CPS compared to 3,000 in the
1 Economics and Statistics Administration, STEM:
States, 2006). The requirements for the 2013 next largest district. The nine-year time span
Good Jobs Now and for the Future (Washington,
graduation class are based on author review of provides information on cohorts before the
DC: US Department of Commerce, 2011),
state-by-state legislation. policy change, during the phase-in period, and
http://www.esa.doc.gov/sites/default/files/
after the implementation of the law.
reports/documents/stemfinalyjuly14_1.pdf; 4 Bradford Chaney, Kenneth Burgdorf, and Nadir
National Science Board, Preparing the Next Atash, “Influencing Achievement through High ACT test data consists of 818,611 ACT
Generation of STEM Innovators: Identifying and School Graduation Requirements,” Educational Mathematics Test and ACT Science Test scores
Developing Our Nation’s Capital (Alexandria, Evaluation and Policy Analysis 19, no. 3 (1997): from the high school graduation classes of
VA: National Science Foundation, 2010), http:// 229–244. doi: 10.3102/01623737019003229 2005 through 2013. Of these, about 24%
www.nsf.gov/nsb/publications/2010/nsb1033. claimed they were in the lowest half of their
5 As mentioned previously, college-bound graduation class. About 10% of students did
pdf.
students would complete more courses than not report information on math or science
2 National Science Board, National Action the district minimum. courses taken.
Plan for Addressing the Critical Needs of the
6 This study focuses on the math and science 8 Data on college enrollment was provided to
U.S. Science, Technology, Engineering, and
aspects of the reform. The law also increased ACT by the National Student Clearinghouse
Mathematics Education System (Alexandria,
the language arts requirement from three (NSC), which has enrollment information for
VA: National Science Foundation, 2007);
to four years along with a two-year writing over 96% of all students in public and private
Anthony P. Carnevale, Nicole Smith, and
intensive course requirement that might US postsecondary institutions. The college
Michelle Melton, STEM (Washington, DC:
overlap with other coursework. Students were enrollment sample is 643,935. This smaller
Center on Education and the Workforce,
also required to complete two years of social sample reflects enrollments through 2012
Georgetown University, 2011). International
studies coursework that included United States (2013 enrollments are not yet available from
assessments of math and science scores are
history or American government as well as an NSC). We were unable to match about 8% of
revealing. The United States scores below
elective. students with NSC records (primarily due to
the international average in mathematics and
close to the average in science on the PISA, 7 The primary data source for this analysis is missing Social Security Numbers).
an international assessment of 15-year-olds. high school graduation information collected as 9 Information on public school district graduation
OECD, Programme for International Student part of ACT statewide testing. All public school requirements was drawn from a 2005 Illinois
Assessment (PISA) Results from PISA 2012: students in Illinois participate in ACT testing State Board of Education Survey which
United States (Paris, France: OECD, 2013), in the spring of their junior year. This includes the authors received through a Freedom of
http://www.oecd.org/unitedstates/PISA- subtests in English, math, reading, and science. Information Act request. The survey asked
2012-results-US.pdf. Likewise, on the National Scores are on a scale from 1 to 36. As part whether districts met the overall requirements,
Assessment of Education Progress (NAEP), of the test administration, students identify the requirements in each subject area (English,
only 35% of eighth-graders were proficient college preparatory courses (i.e., Algebra I math, and science), and if students in the
or advanced in math in 2013 and only 32% but not general math) that they either have districts take specific math courses proposed
demonstrated proficiency in science in 2011. taken or plan to complete prior to graduating as part of the math requirement. About 87% of
National Center for Education Statistics, from high school. Students also provide districts responded to the survey. Based on the
National Assessment of Educational Progress detailed demographic and family background survey results, districts were categorized into
Data Explorer (accessed December 13, information. Multiple imputation techniques treated and untreated districts for each subject
2013), http://nces.ed.gov/nationsreportcard/ were used to fill missing data on student area (note that a district could be categorized
naepdata/. The PISA and NAEP results are demographics and background (see Roderick as a treated district in science but not as a
consistent with ACT test-takers. Forty-four J. A. Little, and Donald B. Rubin, Statistical treated district in math). Treated districts were
percent of ACT-tested 2013 high school Analysis with Missing Data. 2nd ed. [Hoboken, those that were affected by the law. Untreated
graduates met the ACT College Readiness NJ: Wiley, 2002]). More detail can be found in districts were those that had implemented the
Benchmark in mathematics, and 36% met the Richard Buddin and Michelle Croft, “Do Stricter same or more rigorous graduation requirements
science College Readiness Benchmark. ACT, High School Graduation Requirements Improve prior to the law’s enactment.
The Condition of College & Career Readiness College Readiness?” ACT Working Paper
2013 (Iowa City, IA: ACT, 2013). Series WP-2014-1 (Iowa City, IA: ACT, 2014), 10 ACT test records include students enrolled
https://www.act.org/research/papers/pdf/wp- in private schools, but these students are
3 The graduation requirements for the 2006
2014-1.pdf. omitted from our analysis for two reasons. First,
graduation class were drawn from two studies
private school students in the state take the
by Kyle Zinth, Mathematics Graduation The analysis further relies on information for
ACT on a voluntary basis. This group may be
Requirements: Classes 2006 through 2011 the 2005 through 2013 public high school
unrepresentative of private school students in
(Denver, CO: Education Commission of graduation classes for all districts except for
the state overall, as college enrollment is likely
the States, 2006) and Science Graduation the Chicago Public School (CPS) district. CPS
6ACT Research & Policy Missing the Mark: Students Gain Little from Mandating Extra Math and Science Courses
the primary motivation of private school test 14 For instance, Clune and White reviewed 15 Lyle E. Jones, Ernest C. Davenport, Jr., Aloha
takers. Second, the state school board survey student transcripts in four states (California, Bryson, Tanja Bekhuis, and Rebecca Zwick,
was restricted to public school districts, so Florida, Missouri, and Pennsylvania) at “Mathematics and Science Test Scores as
we have no information about private school three points of time (two before and one Related to Courses Taken in High School
graduation requirements in the baseline year of after the implementation of new graduation and Other Factors,” Journal of Educational
2005. requirements) to examine the effect of Measurement 23, no. 1 (1986): 197–208, doi:
changing graduation requirements on student 10.3102/01623737014001002; ACT, Raising
11 David P. Gardner et al., A Nation at Risk: The
course taking. The study found a slight the Bar: A Baseline for College and Career
Imperative for Educational Reform (Washington,
increase in core course taking for schools Readiness in Our Nation’s High School Core
DC: National Commission on Excellence in
serving predominately low-achieving students. Courses. (Iowa City, IA: ACT, 2012).
Education, 1983).
Likewise, Goodman found large increases
16 Justine Randunzel and Julie Noble, Tracking
12 William H. Clune, Paula White, and Janice in completed math coursework for African
2003 ACT-Tested High School Graduates:
Patterson, The Implementation and Effects of Americans, particularly for African American
College Readiness, Enrollment, and Long-Term
High School Graduation Requirements: First males, when using the High School and
Success (Iowa City, IA: ACT, 2012).
Steps Toward Curricular Reform, Research Beyond Survey for the classes of 1982, 1987,
Report Series RR-011 (New Brunswick, NJ: 1991, and 1994. William H. Clune and Paula 17 Clifford Adelman, The Toolbox Revisited:
Center for Policy Research in Education, White, “Education Reform in the Trenches: Paths to Degree Completion from High
1989); Elliott A. Medrich, Cynthia L. Brown, Increased Academic Course Taking in High School Through College (Washington, DC: US
Robin R. Henke, and Lisa Ross, Overview and Schools with Lower Achieving Students in Department of Education, 2006).
Inventory of State Requirements for School States with Higher Graduation Requirements,”
18 Takoko Nomi and Elaine Allenworth, “‘Double-
Coursework and Attendance, Report No. NCES Educational Evaluation and Policy Analysis 14,
Dose’ Algebra as an Alternative Strategy to
92-663, (Washington, DC: National Center for no. 1 (1992): 2–20; Joshua Goodman (2009).
Remediation: Effects on Students’ Academic
Education Statistics, 1992). The Labor of Division: Returns to Compulsory
Outcomes,” Journal of Research on Educational
Math Coursework (Cambridge, MA: HKS
13 For instance, a study using NAEP and 1990 Effectiveness 2, no. 2 (2009): 111–148.
Faculty Research Working Paper Series, 2009),
high school transcript data found that the
http://www.hks.harvard.edu/fs/jgoodma1/ 19 Chrys Dougherty, Lynn Mellor, and Shuling Jian,
majority of students completed more courses
papers/mathcourses.pdf. Orange Juice or Orange Drink? Ensuring That
than required. Chaney, Burgdorf, and Atash,
“Advanced Courses” Live Up to Their Labels,
“Influencing Achievement.”
NCEA Policy Brief No. 1 (Austin, TX: National
Center for Educational Accountability, 2006),
http://files.eric.ed.gov/fulltext/ED519415.pdf.
7You can also read
