The Utility of CBM Written Language Indices: An Investigation of Production-Dependent, Production-Independent, and Accurate-Production Scores

 
School Psychology Review,
                                                                          2005, Volume 34, No. 1, pp. 27-44

      The Utility of CBM Written Language Indices: An
     Investigation of Production-Dependent, Production-
        Independent, and Accurate-Production Scores

                                  Jennifer Jewell
                     Woodford County Special Education Association

                                   Christine Kerres Malecki
                                   Northern Illinois University

           Abstract. This study examined the utility of three categories of CBM written lan-
           guage indices including production-dependent indices (Total Words Written, Words
           Spelled Correctly, and Correct Writing Sequences), production-independent indi-
           ces (Percentage of Words Spelled Correctly and Percentage of Correct Writing
           Sequences), and an accurate-production indicator (Correct Minus Incorrect Writ-
           ing Sequences) and was designed to answer three research questions. First, how
           do these categories of CBM written language scores relate to criterion measures,
           thus providing evidence for their valid use in assessing written language? Second,
           how do the three categories of CBM written language scores compare to one an-
           other across grade levels? Finally, are there gender differences in CBM writing
           scores? Predictions were tested using a sample of 203 second-, fourth-, and sixth-
           grade students from an Illinois school district. Results indicated grade level differ-
           ences in how measures of written language related to students’ scores on a pub-
           lished standardized achievement test, their Language Arts grade, and an analytic
           rating. Specifically, with older students, the production-independent and accurate-
           production measures were more related to standardized achievement scores, an
           analytic rating, and classroom grades than measures of writing fluency. Implica-
           tions were made regarding the appropriateness of using each type of CBM written
           language index for different age levels, gender, and assessment purposes.

      Curriculum-based measurement (CBM)                  (Shinn, 1995) to gather information for instruc-
is an established procedure used to directly              tional decision making (Deno, Fuchs, Marston,
measure student performance in the general                & Shin, 2001; Fuchs, Fuchs, & Hamlett, 1990),
education curriculum. CBM can be used to                  eligibility decisions (Fuchs & Fuchs, 1997;
create norms, measure students’ achievement,              Shinn, Ysseldyke, Deno, & Tindal, 1982),
and monitor progress in the academic areas of             progress monitoring (Deno et al., 2001;
reading, written expression, spelling, and math-          Marston, Fuchs, & Deno, 1986), and other al-
ematics. An increasing number of educators                ternative uses (Deno et al., 2001; Elliott &
and support personnel are turning toward the              Fuchs, 1997; Espin et al., 2000). The current
use of CBM within a problem-solving process               study defines CBM (focusing on written lan-

Correspondence concerning this article should be addressed to Christine Malecki, PhD, Northern Illinois
University, Department of Psychology, DeKalb, IL 60115; E-mail: cmalecki@niu.edu.
Copyright 2005 by the National Association of School Psychologists, ISSN 0279-6015

                                                     27
School Psychology Review, 2005, Volume 34, No. 1

     guage) as a standardized, short-duration flu-         within the context of the phrase to a native
     ency measure of students’ writing skills gath-        speaker of the English language” (p. 11). CWS
     ered in the context of Shinn’s (1995) problem-        not only addresses fluency, as in Total Words
     solving model.                                        Written, or a single skill, as in Words Spelled
                                                           Correctly, but many components of written
     Research on the Reliability and Validity              expression, such as grammar, capitalization,
     of CBM Writing Measures                               punctuation, and spelling. Videen et al. (1982)
            The reliability and validity evidence for      found that a high percentage of interrater agree-
     uses of CBM in reading and mathematics is             ment was attained when scoring for CWS, and
     well-documented (Fuchs & Fuchs, 1997;                 that the average number of CWS in students’
     Marston, 1989; Tindal & Parker, 1989). How-           writing samples more than doubled from third
     ever, there is a need for more research on the        to sixth grade. The strongest correlations were
     reliability and validity of CBM in the area of        found between CWS and Words Spelled Cor-
     written language. Previous research has exam-         rectly (r = .92), Total Words Written (r = .91),
     ined the reliability and validity evidence for        holistic ratings (r = .85), and the raw score to-
     three types of CBM writing indices; produc-           tal on the Test of Written Language (TOWL; r
     tion-dependent indices, production-indepen-           = .69). This study provided evidence for the
     dent indices, and an accurate-production index.       reliability and validity of counting the number
     Research investigating the reliability and va-        of Correct Writing Sequences to assess student
     lidity of these three types of CBM writing in-        proficiency in written expression.
     dices is summarized below.                                  Production-independent (accuracy)
            Production-dependent (fluency) in-             indices. Production-independent indices such
     dices. Production-dependent indices are mea-          as Percentage of Words Spelled Correctly are
     sures of writing fluency because they depend          “percentage indices” and measures of writing
     upon how much a student writes. Deno, Mirkin,         accuracy because they are independent of the
     and Marston (1980) and Deno, Marston, and             length of the writing sample. Tindal and Parker
     Mirkin (1982) found that Total Words Writ-            (1989) examined a number of writing and spell-
     ten, Words Spelled Correctly, Correct Let-            ing indices; however, only three indices repre-
     ter Sequences, and Mature Words were all              sent production-independent (accuracy) mea-
     significantly correlated with written lan-            sures because the scores are independent of the
     guage criterion measures such as the Test             length of the writing sample. Percentage of
     of Written Language (Hammill & Larsen,                Legible Words, Percentage of Words Correctly
     1978) and the Stanford Achievement Test               Spelled, and Percentage of Words Correctly
     (Madden, Gardner, Rudman, Karlsen, &                  Sequenced communicate the level of accuracy
     Merwin, 1978; r = .41 to .88). These studies          of the writing rather than just the amount of,
     provided evidence that these scoring measures         or fluency of the writing.
     were closely related to other commonly ac-                  The results of the Tindal and Parker
     cepted outcome measures. Additionally,                (1989) study indicated that the three produc-
     Marston and Deno (1981) found strong evi-             tion-dependent indices were only weakly cor-
     dence for the reliability of Total Words Writ-        related to teachers’ holistic ratings of middle
     ten, Words Spelled Correctly, and Correct Let-        school students’ writing (rs = .10, .24, and .31,
     ter Sequences. Specifically, the test-retest re-      respectively). The two measures of Correct
     liability coefficients, parallel-test correlations,   Writing Sequences and the Percentage of Leg-
     split-half reliability coefficients, and interrater   ible Words were moderately related to holistic
     correlations were all moderate to high (r = .57       ratings (rs = .45 and .42, respectively). The two
     to .99). Videen, Deno, and Marston (1982)             remaining production-independent measures
     were the first to examine “Correct Writing            (Percentage of Words Correctly Spelled and
     Sequences” (CWS) defined as “two adjacent,            Percentage of Words Correctly Sequenced)
     correctly spelled words that are acceptable           were very strongly correlated to the holistic

28
CBM Written Language

ratings (rs = .73 and .75), providing evidence      municating results and making comparisons
for their valid use in evaluating students’ writ-   between students on the accuracy of writing,
ing skills.                                         production-independent indices (percentage
       Accurate-production indices. The ac-         scores) may be most understandable to teach-
curate-production index (Correct Minus Incor-       ers, parents, and the students themselves. How-
rect Writing Sequences) is a measure of both        ever, the production-independent indices do not
writing fluency and accuracy. A recent study        communicate any information about the
by Espin and colleagues (2000) revealed a rela-     amount of writing a student has produced. Fi-
tively new indicator for use in CBM written         nally, research has examined the issue of us-
language assessment: Correct Minus Incorrect        ing various CBM written language scoring in-
Writing Sequences (CMIWS). Using a sample           dices for making eligibility decisions (e.g.,
of sixth and seventh grade students, they found     Parker, Tindal, & Hasbrouck, 1991; Watkinson
internal consistency coefficients ranging from      & Lee, 1992). The results of the Watkinson and
.72 to .78 for CMIWS. As evidence for valid-        Lee (1992) and Parker et al. (1991) studies
ity, Espin et al. compared CMIWS scores to          support the use of accuracy-based (production-
teachers’ ratings of student writing proficiency    independent) curriculum-based writing mea-
and found moderately strong correlations when       sures to distinguish between students in regu-
examining 3-minute writing samples on story         lar and special education, particularly for sec-
writing (r = .66) and descriptive writing           ondary level students.
samples (r = .67). Correlations between the                A second consideration when determin-
CMIWS and students’ district writing test           ing what index is most valid for a particular
scores were identical (r = .69) when looking        use may be the age of the students. Deno and
at descriptive and story writing CBM samples.       colleagues (1980, 1982) and Videen et al.
These results suggest that the CMIWS index          (1982) found results indicating that production-
may be a valid score to use, particularly with      dependent indices are reliable and valid mea-
middle school students.                             sures in assessing elementary school children’s
                                                    writing skills. Tindal and Parker (1989),
Summary of Research Findings                        Watkinson and Lee (1992), and Parker et al.
                                                    (1991) found that measures of writing accu-
       Some limitations are evident in the afore-   racy (production-independent) were more ap-
mentioned studies such as only one or two cri-      propriate measures of older students’ writing
terion measures being used, and those mea-          skills (e.g., sixth through eighth grades) than
sures represented either direct or indirect writ-   measures of writing fluency.
ing measures (Marston, 1989; Moran, 1987;                  Another factor that may need to be taken
Stiggins, 1982) and not both. The research pre-     into consideration when deciding upon which
sented above also indicates that there is evi-      CBM scoring index to use is the student’s gen-
dence for the reliability of all three categories   der. Within the CBM reading literature, re-
of CBM written language scores. Furthermore,        search results have been mixed as to whether
there is adequate validity evidence for the de-     gender contributes to differences in students’
signed purposes of the scores. The task that        performance. A study by Kranzler, Miller, and
remains is to determine for what specific use(s)    Jordan (1999) suggested that fifth-grade stu-
and for whom each index is suitable. In prac-       dents’ reading performance differed signifi-
tice, each type of scoring index may be more        cantly between boys and girls. However, con-
appropriate to use for certain assessment pur-      tradictory evidence was presented by Knoff
poses, or with students of certain age and gen-     and Dean (1994), in which first-grade students’
der. For example, during individual student         scores showed significant gender differences,
progress monitoring, production-dependent           but older students’ scores did not. Similar stud-
measures, or the relatively new accurate-pro-       ies have not been conducted in the CBM area
duction measure (i.e., CMIWS) may be best           of written expression. Given the implications
for communicating student progress. For com-        that these findings might have regarding nor-

                                                                                                        29
School Psychology Review, 2005, Volume 34, No. 1

     mative data and decision making for boys and        pendent scores (Total Words Written, Words
     girls in the area of written language, further      Spelled Correctly, and Correct Writing Se-
     research in this area is needed.                    quences) based on the work of Tindal and
                                                         Parker (1989) and Espin et al. (2000). Finally,
     Research Questions and Predictions                  it was predicted that gender differences would
            The present study attempts to add to the     be found on some of the CBM measures with
     literature on CBM in the area of written lan-       girls outperforming boys (Knoff & Dean, 1994;
     guage. Specifically, the present study is an in-    Kranzler et al., 1999; Malecki & Jewell, 2003).
     vestigation of the utility of three categories of                       Method
     CBM written language indices including pro-
     duction-dependent indices (Total Words Writ-        Participants
     ten, Words Spelled Correctly, Correct Writing
     Sequences), production-independent indices                 The participants in this study were 203
     (Percentage of Words Spelled Correctly and          second- (n = 87), fourth- (n = 59), and sixth-
     Percentage of Correct Writing Sequences), and       grade (n = 57) students from three schools in
     an accurate-production indicator (Correct           one rural northern Illinois school district. The
     Minus Incorrect Writing Sequences). This in-        schools were recruited by contacting the prin-
     vestigation extends the literature by compar-       cipals and giving them a brief description of
     ing students’ CBM scores to both direct (ana-       the study’s procedure. Participation by the stu-
     lytical scoring) and indirect criterion measures    dents was voluntary and required parental con-
     of writing ability (standardized achievement        sent.
     test, and Language Arts grades) and by inves-              The three schools consisted of an early
     tigating students’ scores from three grade lev-     elementary, an elementary, and a middle
     els. The present study was designed to answer       school. The sample comprised 44% males (n
     the following three research questions: (a) How     = 90) and 56% females (n = 113). In addition,
     do these categories of CBM written language         6.4% of the sample was enrolled in special
     scores relate to criterion measures, thus pro-      education (n = 13). Of the 13 students in spe-
     viding evidence for validity in assessing writ-     cial education, 8 were male and 5 were female,
     ten language? (b) How do the three categories       5 were receiving services for a learning dis-
     of CBM written language scores compare to           ability in reading, 3 for a learning disability in
     one another across grade levels? and (c) What       writing, 4 for a learning disability in math, and
     gender differences are there in the three cat-      6 were receiving speech/language services. The
     egories of CBM scores?                              racial make-up of the participating school dis-
            It was predicted that adequate validity      trict was 94% Caucasian, 5% Hispanic, and
     evidence would be obtained for all six CBM          1% other (African American or Asian). Within
     scoring indices in the form of moderate to          the school district, 11% of the families were
     strong correlations between scores on curricu-      classified as low income and 89% as non-low
     lum-based measures, the Tindal and Hasbrouck        income. Furthermore, 97% of the district was
     (1991) Analytic Scoring System (THASS), the         English speaking and 3% had limited English
     Stanford Achievement Test (SAT; Harcourt            proficiency. Racial and SES data were not
     Brace Educational Measurement, 1997), and           available for the participating students.
     students’ Language Arts classroom grades. It        Measures
     was also predicted that scores on the two pro-
     duction-independent measures (Percentage of                The measures included in this study were
     Words Spelled Correctly and Percentage of           a curriculum-based measure of written expres-
     Correct Writing Sequences) and the accurate-        sion, an analytic scoring system, the Stanford
     production measure (Correct Minus Incorrect         Achievement Test (SAT; Harcourt Brace Edu-
     Writing Sequences) would more closely relate        cational Measurement, 1997), and students’
     to scores on the SAT, THASS, and the students’      classroom Language Arts grades from the fall
     Language Arts grades than the production-de-        semester.

30
CBM Written Language

      Curriculum-based measures (CBM)                      Stanford Achievement Test (SAT;
in written expression. One 3-minute CBM             Harcourt Brace Educational Measure-
writing probe was collected from participat-        ment, 1997). The SAT measures school
ing students. The students were given a lined       achievement for students in Kindergarten
sheet of paper with a story starter typed at        through Grade 12. The SAT is a paper-and-
the top. Per typical CBM writing probe pro-         pencil test in multiple-choice format. The stan-
cedures the students were given 1-minute            dardized battery consists of 11 different
to think about the story starter and 3 min-         subtests including: Total Reading, Word Study
utes to write. The writing samples were             Skills, Word Reading, Reading Comprehen-
scored using the following six procedures:          sion, Total Math, Problem Solving, Math Pro-
Total Words Written (TWW), Words Spelled            cedures, Language, Spelling, Environment, and
Correctly (WSC), Correct Writing Se-                Listening.
quences (CWS), Percentage of Words                         For the present study, there were two
Spelled Correctly (%WSC), Percentage of             particular subtests of interest within the SAT,
Correct Writing Sequences (%CWS), and               Language and Spelling. The Language
Correct Minus Incorrect Writing Sequences           subtest measures proficiency in mechanics
(CMIWS). Table 1 provides the definitions of        and expression. This subtest also looks at
each procedure.                                     content, organization, and sentence structure.
       Tindal and Hasbrouck analytic scor-          The Spelling subtest requires students to iden-
ing system (THASS). An analytic scoring             tify the correct spelling of words. All analyses
system described by Tindal and Hasbrouck            including these two subtests used their National
(1991) was used as an external criterion mea-       Normal Curve Equivalent (NCE) scores based
sure of students’ writing abilities. Using this     on grade level.
system, students’ writing samples are scored               Language Arts grade. Students’ quar-
on three dimensions of writing: story-idea, or-     terly report card grades for Language Arts were
ganization-cohesion, and conventions-mechan-        recorded for the fourth- and sixth-grade stu-
ics. On each dimension, a scoring rubric is used    dents. The second-grade students did not re-
to rate students’ writing samples on a scale        ceive letter grades and therefore were excluded
from 1 to 5, with a “5” representing the high-      from any analyses using letter grades. The let-
est quality. According to Tindal and Hasbrouck      ter grades from the students’ report cards were
(1991), story-idea represents the “extent to        converted to a 4-point scale. Students’ grades
which the composition has a cogent plot and a
                                                    included plusses and minuses; however, any
theme that is unique and captures the reader’s
                                                    “A” (A+, A, or A-) was converted to a 4.0, any
interest” (p. 238). Organization-cohesion ad-
                                                    “B” to a 3.0, and so on. In order to best reflect
dresses the “degree to which the story has an
                                                    students’ grades during the time the CBM
overall structure and progresses systematically,
                                                    samples were collected, the first and second
with well-defined transitions” (p. 238). Finally,
                                                    quarter grades were averaged to form a “fall
conventions-mechanics focuses on “syntax,
sentence structure, grammar, spelling, and          semester” grade.
handwriting” (p. 238). Specific guidelines          Procedure
for rating a writing sample on these three
dimensions using a 5-point scale can be                    Each school was sent copies of the cur-
found in Tindal and Hasbrouck (1991). This          riculum-based writing probes to be distributed
particular rating system was chosen for use         to the participants. The teachers in the students’
in the present study because of its similar-        primary classrooms administered the curricu-
ity to state-wide standardized writing assess-      lum-based measures. Verbally and in writing,
ment techniques. Interrater reliability coef-       the teachers were provided with specific oral
ficients for the three writing components           and written directions for proper administra-
within the THASS have ranged from .73 to .79        tion of the writing probes. First, the teachers
(Tindal & Parker, 1991).                            handed out a single sheet of lined paper to each

                                                                                                         31
32
                                                                       Table 1
                                                  Scoring Procedures, Definitions, and Computations

     Index                                                Definition                                                          Computation

     Production-Dependent Indices

         Total Words Written                TWW           A count of the number of words written. A word is                   TWW
                                                          defined as any letter or group of letters separated by
                                                          a space, even if the word is misspelled or is a
                                                          nonsense word.
         Words Spelled Correctly            WSC           A count of the number of words that are spelled                     WSC
                                                          correctly. A word is spelled correctly if it can stand
                                                           alone as a word in the English language.
         Correct Writing Sequences          CWS           A count of the correct writing sequences found in the
                                                                                                                                                                         School Psychology Review, 2005, Volume 34, No. 1

                                                                                                                              CWS or
                                                          sample. A correct writing sequence is defined as two                Possible Writing Sequences minus
                                                          adjacent writing units (i.e., word-word or word-punctuation)        Incorrect Writing Sequences
                                                          that are acceptable within the context of what is written.
                                                          Correct writing sequences take into account correct spelling,
                                                          grammar, punctuation, capitalization, syntax, and semantics.
         Production-Dependent Composite     PDC           Average of students’ raw scores on the measures of TWW, .           (TWW + WSC + CWS) / 3
                                                          WSC, and CWS
     Production-Independent Indices

         Percentage of Words Spelled        %WSC          The percentage of words in the sample that are spelled correctly.   WSC/TWW
         Correctly
         Percentage of Correct Writing      %CWS          The percentage of correct writing sequences in the sample.          CWS/Possible Writing Sequences
         Sequences
         Production-Independent Composite   PIC           Average of students’ scores on the measures of %WSC                 (%WSC + %CWS) / 2
         and %CWS.
                                                                                                                                                   (Table 1 continues)
(Table 1 continued)

     Index                                         Definition                                                        Computation

     Accurate Production Index

         Correct Minus Incorrect Writing   CMIWS   This measure subtracts the number of incorrect writing            CWS–Incorrect Writing Sequences
         Sequences                                 sequences in the sample from the total number of correct          or CWS– (Possible Writing
                                                   writing sequences. The number of incorrect writing sequences       Sequences– CWS)
                                                   was calculated by subtracting the number of correct writing
                                                   sequences in the sample from the total number of possible
                                                   writing sequences.

     Criterion Measure in Current Study

        THASSS                                     Writing samples are scored on three dimensions; story-idea,       Story-idea + Organization-Cohesion
                                                   organization-cohesion, and conventions-mechanics, each on         + Conventions-Mechanics
                                                   a 5-point scale. A single THASS score was calculated by
                                                   summing students’ scores for story-idea, organization-cohesion,
                                                   and conventions-mechanics.
                                                                                                                                                          CBM Written Language

33
School Psychology Review, 2005, Volume 34, No. 1

     participant that had the story starter, “Write a    ing indices. The coefficients were extremely
     story about your talking dog.” typed at the top.    high (all above .98) for Total Words Written,
     Next, the teacher read aloud the following di-      Words Spelled Correctly, and Correct Writing
     rections:                                           Sequences. Once it was known that interrater
        You are going to write a story. First I will     reliability was more than adequate, the group
        read the story starter at the top of your page   of raters scored the writing samples for the
        and you will write a story about it. You will    present study. Regarding the THASS, two re-
        have one minute to think about what you will
        write, and three minutes to write your story.    searchers scored the writing samples (one-third
        Remember to do your best work. If you don’t      of the samples were scored by both raters). The
        know how to spell a word, you should guess.      interrater reliability coefficients between the
        Are there any questions? (Pause – respond        two raters ranged from .78 to .88 on the three
        to questions.) Put your pencils down and lis-
        ten. For the next minute, think about… (in-      subscale scores and was .88 for the Total score.
        sert story starter and let the students think    For scores that did not match, a consensus score
        for one minute). Now please begin writing.       was determined by the two raters and that score
        (Students write for three minutes.) Please       was used in the final database.
        stop and put your pencils down.
            After administration, the writing samples    Analyses
     for each student were masked (any identifying
     information was removed) and scored on the                 To test predictions, analyses included: (a)
     five CBM scoring indices and the THASS scor-        conducting MANOVAs to investigate grade
     ing system.                                         level and gender differences in scores; (b) com-
            All students took the Stanford Achieve-      puting correlations between scores on the Lan-
     ment Test in the same month as the curricu-         guage and Spelling subtests of the SAT, stu-
     lum-based measure. This test was given over         dents’ Language Arts grades, scores on the
     the course of 6 school days. Testing times          Tindal and Hasbrouck (1991) Analytic Scor-
     ranged from 50 to 80 minutes on those days.         ing System (THASS), and scores on the cur-
     The test was administered by the classroom          riculum-based measures TWW, WSC, CWS,
     teachers and took place in the students’ regu-      CMIWS, %WSC, %CWS, PDC, and PIC; and
     lar classrooms. The students marked their           (c) conducting regression analyses in which
     answers to the multiple-choice questions in         students’ scores on the PDC, PIC, and CMIWS
     test booklets. The testing materials were then      and students’ grade level were used as predic-
     sent back to the publisher to be scored by com-     tors of students’ scores on the analytic scoring
     puter.                                              system (THASS). Overall, due to the number
            One graduate and seven undergraduate         of analyses conducted and a desire to reduce
     students were taught the proper procedure for       Type I error, a more conservative critical value
     scoring the writing samples using the measures      (p < .01) was set as the criterion for signifi-
     described above. This group of students then        cance for all analyses.
     practiced scoring on 20 writing samples per                             Results
     grade for Grades 1-8 (160 samples). Follow-
     ing their scoring, interrater reliabilities were           The predictions investigated in this study
     calculated for the scoring indices of Total         were tested using a series of MANOVAs, cor-
     Words Written, Words Spelled Correctly, and         relational, and regression analyses. The de-
     Correct Writing Sequences. These indices were       scriptive results including the students’ perfor-
     based on scores obtained from scoring entire        mance on the six curriculum-based measures
     writing passages. Reliability coefficients were     (TWW, WSC, CWS, CMIWS, %WSC, and
     not calculated for Correct Minus Incorrect          %CWS), the CBM composites (production-
     Writing Sequences, Percentage of Words              dependent and production-independent), the
     Spelled Correctly, or Percentage of Correct         SAT Language and Spelling subtest scores, fall
     Writing Sequences because each of these scor-       semester Language Arts grades, and Total
     ing indices is imbedded in the other three scor-    THASS scores are presented in Table 2.

34
Table 2
      Descriptive Data on CBM Scores by Grade and Gender, and Stanford Achievement Test Scores, Language Arts Grades,
                                        and Analytic Scoring System Scores by Grade

                                           Grade 2            Grade 4            Grade 6            Boys               Girls

                                           M    (SD) N        M     (SD) N       M      (SD) N      M      (SD) N      M       (SD) N

     Total Words Written                   24.60 (9.16) 87    39.14 (11.38) 59   43.67 (12.96) 57   30.90 (12.83) 90   36.79 (14.11) 113

     Words Spelled Correctly               20.83 (8.61) 87    36.80 (11.32) 59   41.86 (12.56) 57   28.10 (12.77) 90   33.98 (14.66) 113

     Correct Writing Sequences             15.61 (9.02) 87    34.56 (12.30) 59   40.79 (13.80) 57   25.48 (14.21) 90   30.35 (17.04) 113

     Production-Dependent Composite        20.34 (8.46) 87    36.83 (11.40) 59   42.11 (12.62) 57   28.16 (13.02) 90   33.71 (14.91) 113

     % Words Spelled Correctly             84.25 (10.20) 86   93.66 (5.24) 59    95.90 (4.32) 56    89.83 (9.21) 88    90.60 (9.26) 113

     % Correct Writing Sequences           59.59 (19.78) 86   79.60 (13.23) 59   85.69 (14.33) 56   72.61 (19.11) 88   70.56 (22.56) 113

     Production-Independent Composite      70.42 (41.15) 86   86.63 (8.83) 59    90.79 (8.77) 56    81.22 (13.63) 88   80.58 (15.37) 113

     Correct Min. Incorrect Writing Seq.   6.62 (13.03) 87    29.98 (15.44) 59   37.91 (18.57) 57   20.06 (17.76) 90   23.90 (22.70) 113

     SAT Language subtest                  58.91 (18.44) 86   59.66 (17.38) 58   51.79 (20.13) 55

     SAT Spelling subtest                  56.10 (16.04) 86   52.67 (18.60) 58   48.57 (21.60) 55

     Fall Language Arts Grade              na                 3.37 (.65) 58      3.13 (.90) 55

     T & H Analytic Scoring System         5.76 (1.81) 86     8.02 (1.86) 59     8.79 (2.19) 56
                                                                                                                                           CBM Written Language

35
School Psychology Review, 2005, Volume 34, No. 1

     Preliminary Analyses                                 level. Thus, results indicated that how much
                                                          students wrote was not significantly related to
            The CBM data were examined for po-            their writing accuracy. Furthermore, at the old-
     tential differences in scores for students in sec-   est grade level (sixth grade), the number of
     ond, fourth, and sixth grades and gender. A 2        words students spelled correctly was not sig-
     (gender) x 3 (grade level) MANOVA was run            nificantly related to the production-indepen-
     to conduct these exploratory analyses. The six       dent measures. This result suggests that with
     CBM scoring indices served as the dependent          older students, the spelling measure is not re-
     variables. There was no significant interaction      lated to measures of writing accuracy. The
     between grade and gender, Wilks’s lambda =           CMIWS scores, however, do relate well with
     .946, p > .05. However, main effects were            both the production-dependent and the produc-
     found for gender [Wilks’s lambda = .870, F           tion-independent indices, indicating that in fact
     (5, 191) = 5.70, p < .001] and grade [Wilks’s        the CMIWS score is tapping aspects of both
     lambda = .435, F (10, 382) = 19.69, p < .001].       fluency and quality.
     Results of follow-up univariate analyses for the
     gender main effect indicated that boys’ and          Relationships Between CBM Scores,
     girls’ scores differed significantly on the pro-     Published Standardized Scores, Grades,
     duction-dependent measures, Fs (1, 195) =            and the THASS
     16.05, 17.96, 10.30, respectively, ps < .01, but
     not on the production-independent or accurate-             Correlational analyses between students’
     production indices. For all of the production-       scores on the curriculum-based scoring indi-
     dependent measures, girls had significantly          ces and composites (TWW, WSC, CWS, PDC,
     higher scores than boys. Follow-up univariate        %WSC, %CWS, PIC, and CMIWS), subtest
     analyses on grade level revealed that students       scores on the SAT, Language Arts grades, and
     in different grades had significantly different      Total THASS scores were conducted (see Table
     scores on all six of the CBM scoring indices,        4). When testing for significant differences
     Fs (2, 195) = 69.61, 90.27, 104.26, 46.41,           between correlations, Williams’s (1959)
     60.49, and 82.66, respectively, ps < .001. Fol-      method of testing two dependent correlations
     low-up Scheffe contrasts found significant dif-      was used.
     ferences between students at all three grade                SAT Language subtest. In the second-
     levels (second, fourth, and sixth) within the six    and fourth-grade samples, positive correlations
     CBM scoring indices, with two exceptions. No         were found between the SAT Language subtest
     significant difference was found between             scores and most of the CBM measures (r = .34
     fourth- and sixth-grade students on the two          to .67, p < .01). Non-significant correlations
     production-independent measures (%WSC and            with the SAT Language score were with TWW
     %CWS). Because significant differences were          in both Grades 2 and 4, and WSC in Grade 4.
     found between grade levels using production-         Correlations between the SAT Language score
     dependent, accurate-production, and produc-          and the production-independent and accurate-
     tion-independent CBM scoring indices in the          production scores were typically stronger than
     majority of circumstances, correlational analy-      with the production-dependent scores (see
     ses were run at each grade level and with the        Table 4). In addition, when examining the
     total sample.                                        PDC, PIC, and CMIWS scores, the correlations
            To investigate the interrelationships         between the SAT Language score and the PIC
     among the six CBM written language indices,          and CMIWS were higher than the correlation
     correlational analyses were conducted (see           with the PDC at both levels (Grades 2 and 4).
     Table 3). Most of the scores were highly re-         Using Williams’s (1959) approach to test for
     lated to one another with a few notable excep-       differences between dependent correlations, it
     tions. The production-independent measures           was found that in fourth grade, both the PIC and
     (%WSC and %CWS) were not significantly               CMIWS scores were more significantly related
     related to Total Words Written at any grade          to SAT Language than the PDC (ps < .01).

36
CBM Written Language

                                       Table 3
                     Intercorrelations Among the CBM Indices

                           TWW          WSC        CWS        CMIWS       %WSC          %CWS

TWW
Grade 2                                 .95**      .73**      .30**      .12            .09
Grade 4                                 .99**      .89**      .68**      .24            .23
Grade 6                                 .99**      .82**      .52**     -.03           -.12

WSC
Grade 2                                            .86**      .53**      .41**          .33**
Grade 4                                            .93**      .76**      .40**          .36**
Grade 6                                            .88**      .62**      .16            .02

CWS
Grade 2                                                       .88**      .59**          .72**
Grade 4                                                       .94**      .53**          .63**
Grade 6                                                       .92**      .40**          .50**

CMIWS
Grade 2                                                                  .73**          .92**
Grade 4                                                                  .67**          .83**
Grade 6                                                                  .59**          .78**

%WSC
Grade 2                                                                                 .76**
Grade 4                                                                                 .79**
Grade 6                                                                                 .68**

** p < .01.

       For the Grade 6 sample, no significant     the two production-independent measures
correlations between the SAT Language scores      (%WSC and %CWS) and the SAT Spelling
and the production-dependent CBM scores           subtest scores were positive and significant (r
(TWW, WSC, and CWS) were found. How-              = .43 to .56, p < .01). The correlations between
ever, positive, significant correlations were     SAT Spelling and CMIWS were also positive
found between the SAT Language scores and         and significant at these two levels (rs = .49, p
the production-independent and accurate-pro-      < .01). In addition, there was a significant re-
duction CBM scores (%WSC, %CWS, and               lationship between CWS and the SAT Spell-
CMIWS) at r = .47, .52, and .41 (p < .01) (see    ing subtest at both grade levels (r = .37 and
Table 4). Furthermore, the correlations between   .39, p < .01) (see Table 4). In addition, the
the SAT Language score and the PIC and            correlations between SAT Spelling and the two
CMIWS were higher than the correlation with       CBM composites (PDC and PIC) and CMIWS
the PDC. Using Williams’s (1959) approach         were examined. The PIC and CMIWS scores
to test for differences between the dependent     were more strongly related to SAT Spelling
correlations, the PIC and CMIWS were found        than the PDC scores at both second and fourth
to have significantly higher correlations with    grades. At both grade levels, CMIWS had sig-
SAT Language than the PDC (ps < .001).            nificantly higher correlations with SAT Spell-
                                                  ing than the PDC (ps < .01). In second grade,
     SAT Spelling subtest. Examining the          the PIC had significantly higher correlations
Grade 2 and 4 samples, correlations between       with SAT Spelling than PDC (p < .01).

                                                                                                      37
School Psychology Review, 2005, Volume 34, No. 1

                                                      Table 4

            Correlations Between Scores on Curriculum-Based Measures, the
               Stanford Achievement Test, Fall Language Arts Grade, and
                            Analytic Scoring System Scores

                               TWW         WSC       CWS      PDC      %WSC      %CWS        PIC        CMIWS

     SAT Language

     Grade 2                      .24     38**       .57**     .42**     .46**    .59**      .58**        .62**

     Grade 4                      .22      .29       .46**     .34**     .50**    .67**      .65**        .57**

     Grade 6                     -.14      -.05      .23       .03       .47**    .52**      .54**        .41**

     SAT Spelling

     Grade 2                      .03      .17       .37**     .20       .43**    .51**      .51**        .49**

     Grade 4                      .19       .25      .39**     .29       .45**    .56**      .55**        .49**

     Grade 6                      .02       .13      .33       .17       .52**    .44**      .48**        .44**

     Language Arts Grade

     Grade 2                                          —not available—

     Grade 4                      .45**     .51**    .59**     .53**     .53**    .58**      .60**        .61**

     Grade 6                      .12       .20      .30       .22       .45**    .29        .35          .36**

     Total THASS Score

     Grade 2                      .36**    .45**     .58**     .49**     .34**    .47**      .46**        .54**

     Grade 4                      .44**     .49**    .55**     .51**     .35**    .40**      .41**        .56**

     Grade 6                      .16       .24      .46**     .31       .39**    .49**      .50**        .56**

     Note. PDC = production-dependent CBM composite score; PIC = production-independent CBM composite score; CMIWS
     = correct minus incorrect writing sequences.
     ** p < .01.

            In the sixth-grade sample, only the              ences between dependent correlations, it was
     production-independent and accurate-produc-             found that CMIWS was significantly more re-
     tion CBM indices (%WSC, %CWS, and                       lated to SAT Spelling than PDC (p < .01).
     CMIWS) were positively and significantly re-                   Language Arts grades. Within the
     lated to the SAT Spelling subtest (r = .52, .44,        fourth-grade sample, all correlations between
     and .44, p < .01). The PIC and CMIWS scores             the CBM scores and students’ fall semester
     were also more strongly related to SAT Spell-           Language Arts grade were significant (r = .45
     ing than the PDC score at this level. Using             to .61, p < .01). The strongest correlations with
     Williams’s (1959) approach to test for differ-          the Language Arts grade were found with

38
CBM Written Language

CMIWS, PIC, %CWS, and CWS (see Table                counted for in the writing criterion by students’
4). Furthermore, the correlations between stu-      scores on the CBM indices ranged from 12%
dents’ Language Arts grade and PIC and              to 45%.
CMIWS were higher than the correlation with                Regression analyses. Simultaneous re-
PDC, but no significant differences were re-        gression analyses were conducted to determine
vealed.                                             which type of curriculum-based measure (pro-
      The sixth-grade sample had only two sig-      duction-dependent, production-independent, or
nificant correlations between students’ Lan-        accurate-production) was most related to stu-
guage Arts grade and the CBM indices.               dents’ scores on the THASS. Students’ scores
CMIWS was significant (r = .36, p < .01), as        on the variables PDC, PIC, and CMIWS and
was %WSC (r = .45, p < .01). In addition, stu-      their grade level were entered simultaneously
dents’ Language Arts grades were more               as predictors in a series of four regression
strongly related to the PIC and CMIWS than          analyses that included students’ scores on the
the PDC at this level, but these differences were   THASS (Total, story-idea, organization-cohe-
not significant.                                    sion, and conventions-mechanics) as depen-
       Tindal and Hasbrouck analytic scor-          dent variables. The three writing components
ing system. At both grade levels, all of the        of the THASS had intercorrelations ranging
CBM scoring indices and composites were sig-        from .73 to .88. The four predictor variables
nificantly related to students’ scores on the       collectively accounted for 53% of the variance
THASS (r = .34 to .58, p < .01). At the sec-        in students’ Total THASS scores (p < .01).
ond-grade level, the production-dependent,          CMIWS was a significant predictor (β = .48, p
production-independent, and accurate-produc-        < .01). A total of 35% of the variance in stu-
tion measures were all similarly related to the     dents’ story-idea scores was accounted for by
THASS. At the fourth-grade level, the produc-       the four predictor variables (p < .01), with the
tion-dependent measures were more strongly          PDC being a significant predictor (β = .43, p <
related to the THASS than the production-in-        .01). Analyses conducted with students’ orga-
dependent measures, but not the accurate-pro-       nization-cohesion scores indicated that the
duction index. Further examining the PDC,           PDC, PIC, CMIWS, and students’ grade level
PIC, and CMIWS scores, CMIWS was the                accounted for 30% of the variance (p < .01),
most strongly related to the THASS, and PIC         but none of the individual variables were sig-
was the least related. No significant differences   nificant predictors. The four predictor variables
were found.                                         accounted for 62% of the variance in students’
       Examining the Grade 6 sample, the only       scores on conventions-mechanics (p < .01).
production-dependent scoring index signifi-         CMIWS (β = .77, p < .001) and students’ grade
cantly related to the THASS was CWS (r =            level (β = .22, p < .01) were both significant
.46, p < .01). Both production-independent          predictors.
measures (%WSC and %CWS) were signifi-
                                                                      Discussion
cantly related to the THASS (r = .39 and .49, p
< .01). Furthermore, the accurate-production               The results of the preliminary analyses
index had the strongest relationship to the         indicated grade level and gender differences
THASS (r = .56, p < .01). A similar pattern         in students’ performance on the CBM written
was seen for the PDC, PIC, and CMIWS                expression scoring indices. Regarding gender,
scores. Using Williams’s (1959) approach,           girls significantly outperformed boys on all of
CMIWS was found to be significantly more            the writing fluency measures, but there were
related to the THASS than the PDC at this           no gender differences on the production-inde-
grade level (p < .01).                              pendent or accurate-production indices. Thus,
       Overall, the significant correlations be-    girls at all grade levels tended to write more
tween the CBM indices and all other writing         and produce more correctly spelled words and
criterion used in the present study ranged from     correct writing sequences than boys. However,
.34 to .67. Thus, the amount of variance ac-        boys and girls performed similarly on measures

                                                                                                        39
School Psychology Review, 2005, Volume 34, No. 1

     of writing accuracy, indicating that even though     rate-production) related to external writing cri-
     boys may be less fluent, they are equally ac-        teria. The present study found that as grade
     curate in their writing. These new findings          level increased, fewer of the CBM scoring in-
     should be considered carefully by users of           dices were significantly correlated with the
     CBM writing data, particularly when only flu-        criterion measures of SAT subtest scores, lan-
     ency indices are being utilized. Because girls       guage grades, and scores on the THASS. Spe-
     are producing more writing, they will have an        cifically, with the early elementary-aged chil-
     advantage if being compared to boys on flu-          dren (second grade), most of the production-
     ency measures. Thus, perhaps separate norms          dependent, production-independent, and accu-
     should be created for boys and girls.                rate-production measures were significantly
             When examining grade level, second,          related to the writing criteria. However, when
     fourth, and sixth graders’ scores were signifi-      examining older students’ (sixth grade) scores,
     cantly different from one another on all CBM         in all but one case only the production-inde-
     indices with the exception of fourth and sixth       pendent and accurate-production writing mea-
     graders not having significantly different           sures were significantly related to the other
     scores on the production-independent indices         criteria. The number of correct writing se-
     (%WSC and %CWS). This is further validity            quences continued to be significantly related
     evidence for these measures in that one would        to the criterion measures in sixth grade, but
     expect the scores to change as students’ skills      the other more “pure” fluency measures (e.g.,
     matured, particularly between second and             number of words written and words spelled
     fourth grades. Additionally, results revealed        correctly) were not significantly related to the
     that the pure fluency measure (number of             criterion measures at the older grade level.
     words written) is not related to measures of         These findings suggest that as students hit the
     writing accuracy at any grade level. The             late elementary/middle school grades, teach-
     CMIWS scores, however, do seem to be tap-            ers and school psychologists should demon-
     ping aspects of both writing fluency and accu-       strate caution in using CBM to make high
     racy, as evidenced by significant correlations       stakes educational decisions or to influence
     with the production-dependent and production-        instruction if only TWW or WSC are being
     independent scoring indices. Malecki and             used. It appears that examining only the quan-
     Jewell (2003) also recently found that CMIWS         tity of these students’ writing is not assessing
     was significantly related to measures of both        the skills being measured by the criterion mea-
     writing fluency and accuracy. Thus, as consis-       sures. Rather, CWS, %WSC, %CWS, and
     tent with best practices, educators need to be       CMIWS appear to be the more valid indices at
     clear on the purpose of their assessments be-        the sixth grade level.
     fore choosing which CBM writing indicators                  A second major finding of the present
     are appropriate. Fluency and accuracy scores         study was that with few exceptions, a stronger
     will be tapping different abilities, particularly    relationship was found between production-
     at older grade levels. If one wanted a broad         independent scoring indices and the writing
     score that accounted for both fluency and ac-        criteria and between CMIWS and the criteria
     curacy, perhaps the CMIWS score would be             than with production-dependent scoring indi-
     an appropriate choice.                               ces at all grade levels. This finding suggests
             Unlike earlier work performed by Deno        that at all grade levels, measures of writing
     et al. (1980) and Deno et al. (1982), the present    accuracy (production-independent and accu-
     study did not find consistent significant rela-      rate-production) may be more strongly related
     tionships between the production-dependent           to students’ performance on other types of writ-
     scoring indices and the other criterion measures     ing criteria than measures of writing fluency.
     for the sixth-grade students. Instead, the present   This result is consistent with the work of Tindal
     study detected grade level differences with how      and Parker (1989), which revealed that pro-
     the three types of CBM scores (production-           duction-independent scoring indices were more
     dependent, production-independent, and accu-         closely related to teachers’ holistic ratings than

40
CBM Written Language

production-dependent scoring indices. The             similarity to state-wide writing assessments
present study extends the work of Tindal and          currently being used. The current study found
Parker by including both direct and indirect          that CMIWS was a significant predictor of stu-
types of writing criteria, by using three types       dents’ Total THASS and conventions-mechan-
of CBM writing indices, and by finding a grade        ics scores, suggesting that a measure that uti-
level trend in how CBM scores relate to the           lizes the combination of writing fluency and
writing criteria. The results of the present study    accuracy best relates to students’ overall writ-
indicate that at the older grade levels especially,   ing score and their usage of mechanics. This
the production-dependent measures may not             result is further evidence for the validity of the
be significantly related to students’ perfor-         CMIWS score. In addition, students’ grade
mance on other criteria.                              level was also a significant predictor of their
       This grade level trend in how measures         score on the conventions-mechanics compo-
of writing fluency versus measures of writing         nent of the THASS. This finding indicates that
accuracy relate to students’ scores on other          the grade level of the students affected their
writing criteria suggests differences in the va-      performance in writing mechanics. In other
lidity of the three types of CBM scores at dif-       words, as children advance through school,
ferent grade levels. For example, it appears that     their skills in writing conventions and mechan-
the pure measures of writing fluency (number          ics also progress. The same is true for students’
of words written and number of words spelled          scores on the various CBM indices.
correctly) are not appropriate to use at older               It should be noted that although the cor-
grade levels. On the other hand, at younger           relations among the CBM production-indepen-
grade levels, these fluency measures are related      dent and accurate-production indices for all
to writing criteria such as students’ Language        grade levels were significantly related to the
Arts grade and their scores on an analytic scor-      national standardized achievement test scores
ing system. Therefore, when working with stu-         as well as to the THASS scores, the correla-
dents in older grades, using production-inde-         tions ranged from .43 to .67 for the SAT scores,
pendent and accurate-production measures              and from .34 to .56 for the THASSS scores.
would provide information that is more closely        These correlations are moderate to strong, how-
tied to other writing criteria, such as academic      ever, they are not overwhelming. In addition,
achievement measures, than measures of writ-          the correlations are similar in strength between
ing fluency. The importance of using measures         the CBM indices and SAT scores versus the
of writing accuracy with older students could         THASS scores, meaning that no conclusions
mirror the increased emphasis that both stu-          can be drawn about the merits of the SAT ver-
dents and teachers place on writing quality as        sus the THASS as related to CBM indices.
students progress through school. Younger stu-        Overall, the results of the current study add to
dents are still working on becoming fluent            our understanding of CBM writing indices, but
writers, whereas older students begin to pay          help to emphasize that we should always
more attention to the quality of their writing.       choose the assessment that most closely relates
Thus, measures of writing quality are more            to the skills needing to be assessed and that is
accurate in their assessment of older students’       most valid for our intended purpose. These
writing abilities.                                    decisions should be made on a student-by-stu-
       The results of the present study indicated     dent, and purpose-by-purpose basis.
that collectively, the three types of curriculum-     Limitations of the Study
based measures (production-dependent, pro-
duction-independent, and accurate-produc-                   A limitation of the present study was that
tion), were significantly related to the four         data regarding procedural integrity were not
THASS scores (total, story-idea, organization-        collected. Teachers were given explicit direc-
cohesion, and conventions-mechanics). This            tions on how to administer the CBM writing
finding is important because the THASS was            probes to their class in both verbal and written
selected as a criterion measure because of its        formats. However, after the administration of

                                                                                                           41
School Psychology Review, 2005, Volume 34, No. 1

     the CBM probes, teachers were not asked if                 In conclusion, examining the relation-
     the proper procedure was followed. After ex-        ship between students’ scores on CBM, a pub-
     amining the writing samples generated by the        lished standardized achievement test, class-
     students, it appeared as if the standardized pro-   room grades, and an analytic scoring system
     cedure had been followed by all teachers, but       revealed consistently stronger relationships
     this can not be confirmed without procedural        between production-independent and accurate-
     integrity information.                              production CBM scoring measures with the
            A second limitation was that none of the     other writing criteria than with production-
     writing samples used in the present study were      dependent CBM scoring measures. With stu-
     scored by more than one rater. The scorers          dents in older grades, it was primarily the
     underwent an intensive training process and         production-independent and accurate-produc-
     practiced scoring on 160 “example” writing          tion measures that were significantly related
     samples from various grade levels. However,         to the criterion measures. With the early el-
     only one person scored each writing sample          ementary students, all three types of CBM
     used in the present study and therefore             scores were typically related to the criterion
     interrater reliability was not calculated for the   measures. These results illustrated a grade level
     actual study’s writing samples.                     trend in how measures of writing fluency ver-
                                                         sus measures of writing accuracy related to
     Future Research Directions
                                                         other indicators of students’ writing perfor-
            The three types of curriculum-based          mance.
     measures investigated in the present study (pro-           Finally, the results from this study em-
     duction-dependent, production-independent,          phasize the importance of making informed
     and accurate-production) have been shown to         decisions about what types of assessment mea-
     have adequate evidence for their reliable and       sures to use when making educational deci-
     valid use under typical circumstances (Deno         sions for students. The present study, along
     et al., 1980; Espin et al., 2000; Marston &         with previous literature (e.g., Espin et al., 2000;
     Deno, 1981; Tindal & Parker, 1989; Videen et        Tindal & Parker, 1989), suggests that the age,
     al., 1982). The present study was the first to      skill level, and gender of the student, the pur-
     examine these three types of CBM indices si-        pose of the testing (e.g., progress monitoring,
     multaneously and found a grade level trend in       making comparisons across students, com-
     how these indices relate to external writing        municating results with parents, eligibility
     criteria. Future research needs to elaborate        decisions), and the goal(s) of the assessment
     upon the grade level trend that exists within       are all factors that must be taken into ac-
     the area of CBM writing to further delineate        count. If a teacher wants to monitor the in-
     which type of CBM index is most appropriate         crease in the amount of writing a student can
     to use with different ages of students. Another     produce regardless of accuracy, Total Words
     possible direction for future research is to ex-    Written would be suitable. If a teacher wants
     plore the gender differences found in the cur-      to examine only the accuracy of writing in
     rent study between boys’ and girls’ writing flu-    terms of grammar, punctuation, spelling, or
     ency abilities. If this distinction between the     syntax, the percentage indices may be ap-
     genders is replicated, research could further       propriate, especially with older students.
     examine the practical implications of this dif-     Furthermore, the new indicator, Correct Mi-
     ference. Furthermore, research on the utility       nus Incorrect Writing Sequences, may serve
     of CBM writing measures for a variety of aca-       as an appropriate measure of students’ overall
     demic purposes, such as progress monitoring         writing proficiency that has shown convinc-
     and eligibility decisions, needs to be conducted.   ing reliability and validity evidence. Further
     When research examines the application of           research can continue to add to such recom-
     CBM written expression scoring indices, dif-        mendations and make educators’ decision mak-
     ferent grade levels should be used to further       ing in choosing appropriate CBM writing in-
     investigate possible developmental trends.          dices more explicit.

42
CBM Written Language

                     References                                   Based Measurement: Assessing Special Children (pp.
                                                                  18-78). New York: Guilford Press.
Deno, S. L., Fuchs, L.S., Marston, D., & Shin, J. (2001).     Marston, D., & Deno, S. (1981). The reliability of simple,
   Using curriculum-based measurement to establish                direct measures of written expression (Rep. No. 50).
   growth standards for students with learning disabili-          Minneapolis: University of Minnesota, Institute for
   ties. School Psychology Review, 30, 507-524.                   Research on Learning Disabilities.
Deno, S. L., Marston, D., & Mirkin, P. (1982). Valid mea-     Marston, D., Fuchs, L. S., & Deno, S. L. (1986). Measur-
   surement procedures for continuous evaluation of writ-         ing pupil progress: A comparison of standardized
   ten expression. Exceptional Children, 48, 368-371.             achievement tests and curriculum-related measures.
Deno, S. L., Mirkin, P. K., & Marston, D. (1980). Re-             Diagnostique, 11, 77-90.
   lationships among simple measures of written ex-           Moran, M. R. (1987). Options for written language as-
   pression and performance on standardized achieve-              sessment. Focus on Exceptional Children, 19, 1-10.
   ment tests (Rep. No. 22). Minneapolis: University          Parker, R., Tindal, G., & Hasbrouck, J. (1991). Count-
   of Minnesota, Institute for Research on Learning               able indices of writing quality: Their suitability for
   Disabilities.                                                  screening-eligibility decisions. Exceptionality, 2,
Elliott, S. N., & Fuchs, L. S. (1997). The utility of             1-17.
   curriculum-based measurement and performance               Shinn, M. R. (1995). Best practices in curriculum-
   assessment as alternatives to traditional intelligence         based measurement and its use in a problem-solv-
   and achievement tests. School Psychology Review,               ing model. In A. Thomas & J. Grimes (Eds.), Best
   26, 224-233.                                                   practices in school psychology—III (pp. 547–567).
Espin, C., Shin, J., Deno, S.L., Skare, S., Robinson, S., &       Washington, DC: National Association of School
   Benner, B. (2000). Identifying indicators of written           Psychologists.
   expression proficiency for middle school students.         Shinn, M. R., Ysseldyke, J., Deno, S., & Tindal, G. (1982).
   Journal of Special Education, 34, 140-153.                     A comparison of psychometric and functional differ-
Fuchs, L. S., & Fuchs, D. (1997). Use of curriculum-based         ences between students labeled learning disabled and
   measurement in identifying students with disabilities.         low achieving (Rep. No. 71). Minneapolis: University
   Focus on Exceptional Children, 30(3), 1-16.                    of Minnesota, Institute for Research on Learning Dis-
Fuchs, L. S., Fuchs, D., & Hamlett, C. L. (1990). Curricu-        abilities.
   lum-based measurement: A standardized, long-term           Stiggins, R. J. (1982). A comparison of direct and indirect
   goal approach to monitoring student progress. Aca-             writing assessment methods. Research in the Teach-
   demic Therapy, 25, 615-632.                                    ing of English, 16, 101-114.
Hammill, D. D., & Larsen, S. C. (1978). The Test of Writ-     Tindal, G., & Hasbrouck, J. (1991). Analyzing student
   ten Language. Austin, TX: Pro-Ed.                              writing to develop instructional strategies. Learning
Harcourt Brace Educational Measurement. (1997).                   Disabilities Research and Practice, 6, 237-245.
   Stanford Achievement Test, Ninth Edition. San Anto-        Tindal, G., & Parker, R. (1989). Assessment of written
   nio, TX: Harcourt Brace Educational Measurement.               expression for students in compensatory and special
Knoff, H. M., & Dean, K. R. (1994). Curriculum-based              education programs. The Journal of Special Educa-
   measurement of at-risk students’ reading skills: A pre-        tion, 23, 169-183.
   liminary investigation of bias. Psychological Reports,     Tindal, G., & Parker, R. (1991). Identifying measures for
   75, 1355-1360.                                                 evaluating written expression. Learning Disabilities
Kranzler, J. H., Miller, M. D., & Jordan, L. (1999). An           Research and Practice, 6, 211-218.
   examination of racial/ethnic and gender bias on cur-       Videen, J., Deno, S., & Marston, D. (1982). Correct word
   riculum-based measurement of reading. School Psy-              sequences: A valid indicator of proficiency in written
   chology Quarterly, 14, 327-342.                                expression (Rep. No. 84). Minneapolis: University of
Madden, R., Gardner, E. F., Rudman, H. C., Karlsen, B.,           Minnesota, Institute for Research on Learning Disabili-
   & Merwin, J. C. (1978). Stanford Achievement Test.             ties.
   New York: Harcourt Brace Jovanovich.                       Watkinson, J. T., & Lee, S. W. (1992). Curriculum-based
Malecki, C. K., & Jewell, J. (2003). Developmental, gen-          measures of written expression for learning-disabled
   der, and practical considerations in scoring curriculum-       and nondisabled students. Psychology in the Schools,
   based measurement writing probes. Psychology in the            29, 184-191.
   Schools, 40, 379-390.                                      Williams, E. J. (1959). The comparison of regression vari-
Marston, D. B. (1989). A curriculum-based measurement             ables. Journal of the Royal Statistical Society (Series
   approach to assessing academic performance: What it            B), 21, 396-399.
   is and why do it. In M. R. Shinn (Ed.), Curriculum-

         Jennifer Jewell received her PhD in Psychology (School specialization) at Northern Illi-
         nois University in DeKalb. She is a school psychologist with Woodford County Special
         Education Association in Illinois and is an adjunct professor in the Psychology Depart-
         ment at Bradley University in Peoria, IL. Her primary research area is curriculum-based
         measurement in written language.

                                                                                                                            43
You can also read