California Assessment of Student Performance and Progress (CAASPP) Accountability and Reporting - Ekaterina P. Forrester Sara Shore
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
California Assessment of Student
Performance and Progress (CAASPP)
Accountability and Reporting
Ekaterina P. Forrester
Sara Shore
Final Scoring • For each student, the responses from the PT and CAT portions are merged for final scoring. • Resulting ability estimates are based on the specific test questions that a student answered, not the total number of items answered correctly. • Higher ability estimates are associated with test takers who correctly answer difficult and more discriminating items. • Lower ability estimates are associated with test takers who correctly answer easier and less discriminating items. • Two students will have the same ability estimate if they have the same set of test questions with the same responses. • It is possible for students to have the same ability estimate through different response patterns
Final Scoring:
Contribution of CAT and PT Sections
Number of Items defined by Test Blueprints
ELA/Literacy Mathematics
Grade CAT PT CAT PT
3–5 38–41 5–6 31–34 2–6
6–8 37–42 5–6 30–34 2–6
11 39–41 5–6 33–36 2–6
Final Scoring: Contribution of CAT and
PT Sections (cont.)
• Based on the test blueprint, the CAT section is emphasized
because there are more CAT items/points than PT items/points.
• Claims with more items/points are emphasized.
• Mathematics: Concepts and Procedures Problem
Solving/Modeling and Data Analysis Communicating Reasoning
• ELA: Reading Writing Speaking/Listening Research
• Groups of items that are more difficult and discriminating will
have a larger contribution on final scores.
• Therefore there is no specific weight associated with either PT
or CAT Sections
CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results
Smarter Balanced Scale Score Ranges by Grade Level
Grade Subject Min Max Subject Min Max
3 ELA 2114 2623 Mathematics 2189 2621
4 ELA 2131 2663 Mathematics 2204 2659
5 ELA 2201 2701 Mathematics 2219 2700
6 ELA 2210 2724 Mathematics 2235 2748
7 ELA 2258 2745 Mathematics 2250 2778
8 ELA 2288 2769 Mathematics 2265 2802
11 ELA 2299 2795 Mathematics 2280 2862
CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results
Achievement Levels
• Achievement level classifications based on overall scores
• Level 1—Standard Not Met
• Level 2—Standard Nearly Met
• Level 3—Standard Met (College and Career Readiness)
• Level 4—Standard Exceeded
CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results
Overall Scores – 2 Areas, 4 Levels
English Language Arts and Mathematics
Exceeded the Standard
Met the Standard
Nearly Met the Standard
Has Not Met the Standard
CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results
Achievement Levels by Grade CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results
Smarter Balanced Scale Score Ranges for ELA/Literacy
Level 1 Level 2 Level 3 Level 4
Standard Standard Standard Standard
Grade NOT Met Nearly Met Met Exceeded
3 2114–2366 2367–2431 2432–2489 2490–2623
4 2131–2415 2416–2472 2473–2532 2533–2663
5 2201–2441 2442–2501 2502–2581 2582–2701
6 2210–2456 2457–2530 2531–2617 2618–2724
7 2258–2478 2479–2551 2552–2648 2649–2745
8 2288–2486 2487–2566 2567–2667 2668–2769
11 2299–2492 2493–2582 2583–2681 2682–2795
CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results
Achievement Levels by Grade CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results
Smarter Balanced Scale Score Ranges
for Mathematics
Level 1 Level 2 Level 3 Level 4
Standard Standard Standard Standard
Grade NOT Met Nearly Met Met Exceeded
3 2189–2380 2381–2435 2436–2500 2501–2621
4 2204–2410 2411–2484 2485–2548 2549–2659
5 2219–2454 2455–2527 2528–2578 2579–2700
6 2235–2472 2473–2551 2552–2609 2610–2748
7 2250–2483 2484–2566 2567–2634 2635–2778
8 2265–2503 2504–2585 2586–2652 2653–2802
11 2280–2542 2543–2627 2628–2717 2718–2862
CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment ResultsMeasurement Precision: Error Bands
• For each scale score estimated for a student, there is
measurement error associated with each score.
• An error band is a useful tool that describes the measurement error
associated with a reported scale score.
• The error bands used to construct interval estimates were based
on one standard error of measurement.
• If the same test is given to student multiple times, about 68 percent of the time, the student
will score within this band.
CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment ResultsElements of the Student Score Report
Front Page Back Page
5
1
2
3
6 7
4
8Elements of the Student Score Report
Front Page
3Elements of the Student Score Report
Front Page
4Achievement Levels for Claims
• Achievement Levels for claims are very similar to subscores.
They provide supplemental information regarding a student’s
strengths or weaknesses.
• No achievement level setting occurred for claims.
• Only three achievement levels for claims were developed since
there are fewer items within each claim.
• Achievement levels for claims are based on the distance a
student’s performance on the claim is from the Level 3 proficiency
cut.
• A student must complete all items within a claim to receive an
estimate of his or performance on a claim.
CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment ResultsAdditional Scores – 3 levels
•English Language Arts
Above Standard
Above Standard
1.Reading
2.Writing
3.Speaking & Listening
4.Research/Inquiry
At/Near Standard
At/Near Standard
•Mathematics
1.Concepts & Procedures
2.Problem Solving & 4. Modeling & Data
Below Standard
Analysis
Below Standard
3.Communicating ReasoningElements of the Student Score Report
Back Page
5Elements of the Student Score Report
Back Page
6Elements of the Student Score Report
Back Page
7Elements of the Student Score Report:
Science Grades 5, 8, & 10 only
Back Page
8Elements of the Student Score Report:
Early Assessment Program Grade 11 only
Back Page
8ELA Percent of Students in Each Achivement Level
SBAC 2015, CST 2003, 2013
100 4
6 8 7
11 9
13 12 12
18
90 22
19
17 30
17
19
80 19
20 25
25 26
70
33
31
32
60 34
28 30 42
27 40
22 30
50
35
40
33
30 34
26
45
42 41 42
20 40
36 36
34
27
10
15 16
13
0
SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013
Grade 3 Grade 4 Grade 5 Grade 6
Below Basic Basic Proficient Advanced
23ELA Percent of Students in Each Achivement Level
SBAC 2015, CST 2003, 2013
100
4 3
5 6
8
12 13
18
90 16 23
18
20
28
80
30
30 29
70
34
36
60 35 33
29
50 25 34
30
34
40
31
30
33
45 45
43
20
37
33
28
25
10
17
11
0
SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013
Grade 7 Grade 8 Grade 11
Below Basic Basic Proficient Advanced
24Mathematics Percent of Students in Each Achivement Level
SBAC 2015, CST 2003, 2013
100
5
6
8 8
11 10
12
14
20
90
19 28
31 20
19 38 15
13
80 23
24
25
70
30
27
25
60 32
33
31
29 28
39
50 30
26 30
40
29
20
30 23
50 50
20
43
20 40 41
35 34 35
21 22
10 18
13
0
SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013
Grade 3 Grade 4 Grade 5 Grade 6
Below Basic Basic Proficient Advanced
25Mathematics Percent of Students in Each Achivement Level
SBAC 2015, CST 2003, 2013
100 1 1
3 3 3
9 10
8 8
12
17
90
17 17
14 26
16
80
27
31
29
70
27
35
34 29
60 30
33
50
30
40
28
64
30 58
53
48
45 46
20 38
30
10 20
0
SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013
Grade 7 Grade 8 Grade 11
Below Basic Basic Proficient Advanced
26SBAC Claims and Sample
Items
New Ways of Assessing Learning
2728
KDS’ SBAC Claims, Targets, Standards Alignment
http://www.keydatasys.com/
29Finding Claim, Target, DOK, Standards Tables Google: Claims Targets Standards Riverside or go to KeyDataSys.com under Educational Resources
Assessment Claims for ELA
• Overall ELA Claim (Grades 3-8)
• “Students can demonstrate progress toward college
and career readiness in English language arts and
literacy.”
• Overall ELA Claim (High School)
• “Students can demonstrate college and career
readiness in English language arts and literacy.”
http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/09/Smarter-Balanced-ELA-Literacy-
Claims.pdf
31Content and Reporting Claims for ELA
• Claim #1 Reading – Students can read closely and analytically
to comprehend a range of increasingly complex literary and
informational texts.
• Claim #2 Writing – Students can produce effective and well-
grounded writing for a range of purposes and audiences.
• Claim #3 Speaking/Listening – Students can employ effective
speaking and listening skills for a range of purposes and
audiences.
• Claim #4 Research– Students engage in research to investigate
topics, and to analyze, integrate, and present information.
http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/09/Smarter-Balanced-ELA-
Literacy-Claims.pdf
32Claim 1: Reading
• Literacy instruction that centers on careful
examination of texts
• Reading closely and drawing evidence from the text to
support inferences and judgments made
• Variety of text types
• Students should be able to
• Locate key details and summarize central ideas
• Use textual evidence to analyze and support judgments
made about the ideas presented
• Refer to details and examples in support of claims,
inferences, and conclusionsClaim 1: Sample SBAC Items
36
Claim 2: Writing
• Writing activities and assignments
• Designed with genuine purposes and audiences in mind
• from teachers and other students to community groups, local or
national officials, commercial interests, students’ friends and
relatives, and other potential readers
• Students should be able to
• Work with – compose, revise, and/or edit - a variety of shorter
and longer literary and informational texts for different purposes
• Narrative, about real or imaginary experiences or events
• Informational/explanatory texts, writing opinions/arguments
about a topic
• Opinions/arguments in response to texts read (either fiction or
nonfiction)Claim 2: Sample SBAC Items
Claim 3: Listening and Speaking
• Communicate effectively – demonstrating active listening,
interpersonal communication, and the ability to integrate
oral/visual/graphic information.
• Children’s listening comprehension outpaces reading
comprehension until the middle school years (grades 6–8)
• Sticht and James (1984)
• Students should be able to
• Listen to/view a variety of non-print texts, such as following
directions or procedures in a simulation or hands-on task, or
view demonstrations, lectures, media messages, speeches,
etc.
• Respond to comprehension- and integration/analysis–type
questions (similar to the selected response and open response
questions described for reading Claim #1).Claim 3: Sample SBAC Items
Claim 4: Research/Inquiry
• Filter information, evaluate the credibility of
sources, detect and challenge the underlying
assumptions, and make thoughtful decisions
based on their analysis of what is relevant to the
topic, issue, or problem being explored.
• What students should be able to do
• Apply literacy skills across content areas -
history/social studies, science, and technical
subjects, as well as the language artsClaim 4: Sample SBAC Items
Assessment Claims for Math
• Overall math Claim (Grades 3-8)
• “Students can demonstrate progress toward college
and career readiness in mathematics.”
• Overall math Claim (High School)
• “Students can demonstrate college and career
readiness in mathematics.”
http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/09/Smarter-Balanced-Mathematics-
Claims.pdf
47Content and Reporting Claims for Math
• Claim #1 Concepts & Procedures – Students can explain and
apply mathematical concepts and interpret and carry out
mathematical procedure with precision and fluency.
• Claim #2 Problem Solving– Students can solve a range of
complex and well-posed problems in pure and applied
mathematics, making productive use of knowledge and problem
solving strategies and Claim #4 Modeling and Data Analysis
–Students can analyze complex, real-world scenarios and can
construct and use mathematical models to interpret and solve
problems.
• Claim #3 Communicating Reasoning – Students can clearly
and precisely construct viable arguments to support their own
reasoning and critique the reasoning of others.
http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/09/Smarter-Balanced-
Mathematics-Claims.pdf 48Math Claim 1: Concepts and Procedures
• Math instruction that is centered on particular
mathematical skills or concepts
• Explain and carry out mathematical skills fluently
• Student should be able to
• Understand how concepts relate to one another and look
for patterns (SMP, 7)
• Know why math procedures work the way they do (SMP,
8)
• Fluently (accurately and efficiently) do math (SMP, 6)
• Attend to precision (SMP, 6)
49Claim 1: Sample SBAC Items
5051
52
Math Claim 2 & 4: Problem Solving and
Modeling and Data Analysis
•Claim 2
• Number sentences and word problems
•Students should be able to
• Come up with a solution to a problem that
is not evident (SMP, 2)
• Construct their own solution pathway
• Preserver in solving the problem (SMP, 1)
53Math Claim 2 & 4: Problem Solving and
Modeling and Data Analysis
• Claim 4
• Bridges the gap from “school math” to “real world” math
• Students are able to
• Do math when they’re not told what math to do
• Make assumptions and simplifications (SMP, 2)
• Select appropriate information
• Know what data are missing
• Model with mathematics (SMP, 4)
54Claim 2: Sample SBAC Items
5556
57
58
Math Claim 3: Communicating
Reasoning
• Instruction is based on allowing students to communicate
their reasoning in math and critic/evaluate the reasoning
of others
• Student are able to
• Construct and present a clear, logical and convincing
argument
• Provide justification and explanation, or counter-example
• Identify flaws or gaps in reasoning
• Distinguish correct logic or reasoning
59Claim 3: Sample SBAC Items
6061
62
63
SBAC Claim Level Analysis
SBAC 2015 ELA Claim Level Analysis
100
11 10 12 10 11 11 12 10
14 13 13 14 14 15
17
90 21
80
70
39 43 44
44 44 46
50 50
60 44
61 54 61
65 66
60
54
50
40
30
47 46 45
43 42 42
20 39
38 37
32
29 28
25 25 24 25
10
0
Reading Writing Listening and Research/Inquiry Reading Writing Listening and Research/Inquiry Reading Writing Listening and Research/Inquiry Reading Writing Listening and Research/Inquiry
Speaking Speaking Speaking Speaking
Grade 3 Grade 4 Grade 5 Grade 6
Below At or Near Above
64SBAC Claim Level Analysis
SBAC 2015 ELA Claim Level Analysis
100
8 7
14 11 13 12
14
90 18 18 20 18
21
80
70
44 49
45
60 64 64
46 57 49
64
55 53
50 58
40
30
43 41
20 40
36
33
28 29 30
27 27
24
10 21
0
Reading Writing Listening and Speaking Research/Inquiry Reading Writing Listening and Speaking Research/Inquiry Reading Writing Listening and Speaking Research/Inquiry
Grade 7 Grade 8 Grade 11
Below At or Near Above
65SBAC Claim Level Analysis
SBAC 2015 Mathematics Claim Level Analysis
100
11 11 11 9 10
12 12 16 12 14
17 15
90
80
26 36 30
70
33 41 43 46
42
36 45 49
60 55
50
40
30 59
56
52 53
47 46 48 46
43 45
20 41
33
10
0
Concepts and Problem Solving/Data Communicating Concepts and Problem Solving/Data Communicating Concepts and Problem Solving/Data Communicating Concepts and Problem Solving/Data Communicating
Procedures Analysis Reasoning Procedures Analysis Reasoning Procedures Analysis Reasoning Procedures Analysis Reasoning
Grade 3 Grade 4 Grade 5 Grade 6
Below At or Near Above
66SBAC Claim Level Analysis
SBAC 2015 Mathematics Claim Level Analysis
100
9 10 10 10 9 8 8
15 13
90
80
37
70 30 43 33
51 53
52 56
60
69
50
40
30
55 54 54
48
20 38 39 39
36
10 21
0
Concepts and Procedures Problem Solving/Data Analysis Communicating Reasoning Concepts and Procedures Problem Solving/Data Analysis Communicating Reasoning Concepts and Procedures Problem Solving/Data Analysis Communicating Reasoning
Grade 7 Grade 8 Grade 11
Below At or Near Above
6768
Additional Resources For
Communicating Scores
http://www.cde.ca.gov/ta/tg/ca/communicationskit.asp
69Communications Toolkit
• Short documents, in English and Spanish: key topics such as “Creating a Computer
Adaptive Test” or “Accessibility and Accommodations: Addressing the Needs of all
Students”
• Links to key sites such as the California PTA
• Brief videos, in English and Spanish: key topics such as What are the Smarter
Balanced Assessments? and Ready. Set. Succeed: Preparing Our Kids for College
and Career
http://www.cde.ca.gov/ta/tg/ca/communicationskit.asp
70Communications Toolkit (Cont.)
• Sample parent and guardian letter to accompany the Individual Student Report
• Reading Your Student Report, in multiple languages, to help parents and guardians read
and interpret the Individual Student Report
• Documents that include released questions that exemplify items in the Smarter
Balanced assessments to help parents/guardians understand the achievement levels
• Short video to help parents/guardians understand the Individual Student Report
http://www.cde.ca.gov/ta/tg/ca/communicationskit.asp
• Understanding your child’s score report
https://www.youtube.com/watch?v=FQi4qlOCrmk&index=1&list=PLgIRGe0-
q7SZtZI9b50s4TLpy1_LPf9tP
71Ekaterina@KeyDataSys.com
Sara@KeyDataSys.com
72You can also read
