A Comparison of Chat Room Productivity: Robert Sanders 59

 
Robert Sanders                                                                            59

 A Comparison of Chat Room Productivity:
      In-class Versus Out-of-class
                                 ROBERT SANDERS
                             Portland State University

ABSTRACT
    Language production of Spanish students using synchronous CMC (chat) dur-
    ing class time was compared with that of students meeting outside of class. The
    study included about 100 participants enrolled in 10 sections of a first-quarter
    Spanish course at a US university. Students in the control group spent 30 minutes
    of their weekly class time in computer labs completing a chat room assignment.
    Students in the experimental group planned their own schedules for meeting 30
    minutes each week, outside of class time, in chat rooms. Transcripts of the chat
    sessions were analyzed for duration, turns, words, vocabulary, socially appropri-
    ate comments, and comments off task. Production was greater when students met
    with their own work groups outside of class. The implication of the study is that
    greater student collaboration and responsibility result in greater production while
    conserving class time and technology resources.

KEYWORDS
Chat, Computer-mediated Communication, Synchronous CMC, Distributed Education

INTRODUCTION
Several researchers have addressed the effectiveness of out-of-class email journ-
aling in comparison with in-class, paper dialogue journaling (see González-Bue-
no and Pérez, 2000; Pérez-Sotelo & González-Bueno, 2003) and with in-class
chat discussions (see Pérez, 2003). This study compares in-class and out-of-class
production in WebCT chat rooms by students enrolled in an introductory Spanish
course at a US university. The purpose of the study is to evaluate out-of-class,
electronic contact as an alternative to some face-to-face contact. Out-of-class par-
ticipation could provide significant cost savings through distributed resource uti-
lization. The university in question is struggling to provide students basic Spanish
instruction at a time when enrollment growth far outstrips available resources.
Its introductory Spanish program focuses on oral proficiency and uses class time
primarily for communication in the target language. Prior to implementation of
chat discussions, existing research was reviewed for indications of proper activity
design.

CALICO Journal, 24 (1), p-p 59-76.                            © 2006 CALICO Journal
60                                                 CALICO Journal, Vol. 24, No. 1
REVIEW OF LITERATURE
Benefits of Computer-mediated Communication
Communicative interaction and production have been repeatedly addressed in
SLA research on computer-mediated communication (CMC). Many studies (e.g.
Beauvois, 1992, 1997; Biesenbach-Lucas & Weasenforth, 2001; Blake, 2000;
Kern, 1995; Sotillo, 2000; Warschauer, 1996) approach communication through
measures of formal features that transcend oral and written modalities, such as t-
units, clauses, cohesive devices, turns, text length, words, authority, collaboration,
participation, equality, and opportunity. These approaches have proven effective
for analysis of online L2 communication and have been greatly facilitated by the
transcription features of most CMC software. Hypotheses about output (Swain,
1985), learner collaboration (Bayer, 1990), and learning communities (Meskill,
1999; Wenger, 1998) are frequently cited to explicate the role of communication
in SLA. These hypotheses correlate increased contact, production, and practice to
increased acquisition.
   Case studies have found that text-based CMC is effective for increasing student
production by facilitating greater student participation (Beauvois, 1992; Biesen-
bach-Lucas & Weasenforth, 2001; Chun, 1994; Kelm, 1992; Kern, 1995; Kiesler,
Siegel, & McGuire, 1984; Warschauer, 1996), greater distribution of participation
among students (Beauvois, 1992; Kern, 1995; Warschauer, 1996), greater partici-
pation by “shy” students (Beauvois, 1992; Chun, 1994; Kelm, 1992), transfer of
communicative authority to students (Beauvois, 1992; Chun, 1994; Darhower,
2002; Kern, 1995), reduced anxiety (Chun, 1994; Kern, 1995), reduced burden
on working memory (Payne & Whitney, 2002), development of extended discus-
sion more attuned to student motivation and ability (Weasenforth, Biesenbach-
Lucas, & Meloni, 2002), increased attention to classmates (Chun, 1994), and col-
laboration among students in the development of learner-learner support networks
(Darhower, 2002; Feenberg, 1999; Kern, 1995). Again, the results of CMC are
increased interaction and communication.
   Chun (1994) had suggested that the discursive cohesiveness and levels of in-
teraction observed in synchronous CMC (SCMC) might transfer to oral conver-
sation. In fact, CMC has been proven effective for increasing oral achievement
and proficiency. In a research study of 83 students in fourth-semester French,
Beauvois (1998) found a significant difference in achievement on oral exams of
+11% for students who had discussed assigned reading texts (the subject of oral
exams) only via SCMC. In a study of 20 ESL students, Warschauer (1996) report-
ed increased English proficiency scores in correlation to increased participation
in CMC. Payne and Whitney (2002), in a study of 58 students in a third-semester
Spanish course, found that SCMC improved performance in oral production inter-
views more than did in-class discussion. In a third-semester German course (Ittzes
Abrams, 2003), in-class SCMC (chat) sessions led to significantly better oral per-
formance in large group discussions than did typical course activities (reading,
exercises, and interviews). The study included 33 control subjects and 32 subjects
using SCMC. Improved oral performance was measured in c-units, or phrases
Robert Sanders                                                                   61
with communicative value. The students who completed all of the chat assign-
ments produced an average of 58% more words during oral testing and 53% more
c-units. While there was no significant effect on lexical richness, lexical density,
or syntactic complexity, Ittzes Abrams emphasized the improvement in fluency
among learners who could “access the necessary lexical items with greater facility
and speed” (p. 164).

Differences Between Synchronous and Asynchronous CMC
Each modality of text-based CMC appears to offer different advantages over oral
discussion and writing. Sotillo (2000) has addressed the differences in L2 lan-
guage production in SCMC and asynchronous CMC (ACMC). Drawing upon
Wei’s (2000) description of subsystems in lexical structure, Sotillo characterized
interaction in SCMC as fluid and interpersonal, in which students were “operat-
ing at the lexical-conceptual structural level, which conflates universally avail-
able semantic and pragmatic information” (Sotillo, p. 105), whereas interaction
in ACMC was more presentational in that students used more text with greater
complexity, indicative of “operating at the predicative-argument structural level”
(p. 106). In layman’s terms, transcripts of SCMC look like those of dynamic oral
conversations, while transcripts of ACMC are similar to formal reports or written
responses. Biesenbach-Lucas and Weasenforth (2001) performed an ESL study
that also suggests two distinct discursive styles in CMC. When the ESL students
used word processors in class they produced long documents with extensive con-
textualization, whereas they assumed more context when using email in class.
Electronic mail was not designed for and cannot fully facilitate SCMC, but simul-
taneous use by students meeting together in a class room constitutes an arguably
synchronous modality.
   The different styles evidenced in SCMC and ACMC correspond to Halliday’s
(1993) basic distinction between communicative interaction and interpretation of
experience. As the semiotic system of the child develops into that of an adult,
these perspectives of meaning become associated with speaking and writing.
Speaking corresponds to the dynamic aspect of reality, “reality as process” in
Halliday’s words, while writing foregrounds “reality as object” (p. 111). Halliday
points out that spoken language emphasizes verbs and adjectives, whereas in writ-
ing “processes and properties are construed as nouns” (p. 111). This construal in-
volves the complex subordination characteristic of ACMC that is generally absent
in SCMC.
   Synchronous electronic communication elicits an interactive style set upon
information exchange and social cohesion, whereas asynchronous electronic
communication elicits formal analysis and synthesis of information. Darhower’s
(2002) sociocultural study of electronic L2 communication found student nego-
tiation of intersubjectivity, topics, and social cohesion to be emerging features of
SCMC interaction. Knowledge construction was not a salient feature.
   The interactive and reflective functions evidenced by production in CMC are
also indicated by the effects of synchronous and asynchronous CMC on oral com-
munication. In a comparison of synchronous and asynchronous electronic dis-
62                                                 CALICO Journal, Vol. 24, No. 1
course with face-to-face activities, Ittzes Abrams (2003) found that where SCMC
facilitated significant production gains in follow-up oral discussions, ACMC did
not. In fact, the ACMC (electronic bulletin board) treatment resulted in the low-
est oral production of all when compared to SCMC and face-to-face preparation.
In the follow-up discussion, students who had used the electronic bulletin board
produced an average of 93% fewer words and 43% fewer c-units than the control
group, despite the fact that they used more “subordinate, relative and infinitive
clauses during their computer-mediated communication” (p.163).

Cost Effectiveness and Distributed CMC
Studies of the cost effectiveness of integrating technology in education are few
and far between (see Atkin, 1970; Fletcher, 1995; Hawley, Fletcher, & Piele, 1986;
Knight, 2000; Levin, 1984). Cost has not been a frequent variable in research on
L2 curricula and is surprisingly missing in research on technology in SLA. Sala-
berry’s (2001) retrospective study of Modern Language Journal articles on tech-
nology exposes a history of mostly unsubstantiated expectations of greater learn-
ing outcomes and cost effectiveness. Salaberry concludes that much skepticism is
in order and recommends that “efficient use of human and material resources” (p.
51) become an important criterion for evaluating L2 technology. Cost effective-
ness has, on occasion, appeared as an organizing theme in literature on second
language curricula (see Garrett, 1991; Maxwell & Garrett, 2002; Meredith, 1983).
The focus on effectiveness in studies of CALL may be justified given that robust
gains in learning efficiency can come from improved competency outcomes and
faster program completion,1 but the cost effectiveness of learner-centered CALL
has not yet been systematically addressed.
   Egbert, Paulus, and Nakamichi (2002) identify limited technology resources as
an obstacle to the effectiveness of CALL in the classroom and suggest that course
designs incorporating technology should include ways to teach around this barrier
(p. 122). One suggestion would be to make greater use of the resources students
possess as well as those provided by the institution for the general student popula-
tion. CMC activities that only involve students enrolled in the course (e.g. those
that do not involve foreign partners) could be performed out-of-class, while in-
class activities could focus on real speaking, cultural activities, face-to-face test-
ing, and physical practice of L2 non-verbal communication. Since CMC is place
and time independent (Warschauer, 1997), one should not expect that engaging in
CMC in class is necessarily more effective than engaging in it out of class.
   González-Bueno and Pérez (2000) found that student production in out-of-class
email dialogue journals (ACMC) exceeded that of in-class paper-and-pencil dia-
logue journaling. They did not find any significant lexical or grammatical advan-
tage in the electronic medium and suggested that dialogue journals and compo-
sitions, regardless of the medium, be complemented with form-focused writing
activities. Pérez-Sotelo and González-Bueno (2003) revisited the question in a
more controlled study in which the paper-and-pencil dialogue journaling was also
performed out-of-class. They found no significant differences between the elec-
tronic and the paper-and-pencil method with respect to lexical accuracy and the
Robert Sanders                                                                    63
amount of language produced, but significantly greater grammatical accuracy for
the control group. They recommend that the electronic journals not be used where
grammatical accuracy is a principal goal but, instead, that it be exploited for other
benefits. For example, Chun (1994) focused on “sociolinguistic and interactive
competence” in evaluating the benefits of SCMC (p. 18). Interestingly, Pérez-So-
telo and González-Bueno’s research study was instrumental in moving the open-
ended writing assignment out of class.
   In a study on vocabulary growth among first-semester Spanish students using
synchronous and asynchronous CMC, Pérez (2003) found no significant differ-
ence between the effectiveness of instructor supervised in-class chat sessions and
email journals completed at home, despite the fact that the instructor supplied new
vocabulary to the students during the chat sessions. Pérez also contends that the
subjects enjoyed writing email journals instead of attending class one day each
week.
   Despite the potential for distributed resource utilization, most case studies of
SCMC employ designs in which the activity is performed in classrooms or com-
puter labs by an entire class, simultaneously, with the instructor present. Not only
is it overly expensive to use CMC during traditional class meetings, it may be an
underutilization of the teacher’s core competence. While most L2 students tend
to value instructor presence very highly while using CMC (see Darhower, 2002;
Pérez, 2003; Stepp-Greany, 2002), this may be a reflection of the general desire
for personal guidance in most language study tasks. Students overwhelmingly
value contact with a teacher as one of the most important and effective compo-
nents of L2 study, but instructor assistance during CMC usually takes the form of
technical aid, task explanation, and vocabulary and grammar clarification. Most
of these needs can probably be met through student training, clear and simple ex-
pectations, illustrative examples (see Weasenforth et al., 2002, p. 63), reference to
the appropriate chapter of the textbook, and student-student collaboration. Facili-
tation and guidance by the instructor is by no means limited to his or her presence
during or at the physical interface with CMC; integration, implementation, and
monitoring of CMC in the curriculum, along with abundant feedback, bring forth
much more effectively and efficiently the instructor’s role of facilitator, trainer,
motivator, and collaborator.

RESEARCH QUESTION
Is student productivity greater when SCMC is used in class, or when it is used
out of class by students working collaboratively to schedule and complete their
assignments?

METHOD
Procedure
The differences between synchronous and asynchronous CMC discourse revealed
by existing research suggest general guidelines for their implementation. Lee
(2002) concludes that “CMC is an effective way for learners to negotiate both
64                                                 CALICO Journal, Vol. 24, No. 1
meaning and form and reinforce their communication skills” (p. 20).2 Lee empha-
sized the use of chat for completing communicative tasks that required personal
information exchange (focus on meaning), student selection of form (linguistic
and sociolinguistic competence), and that were related to a specific chapter of the
course text (grammar and vocabulary reference).
   In the study described here, students were required to exchange personal infor-
mation in groups and later report the information individually in formal composi-
tions. The weekly topics were chosen from the corresponding textbook chapter.
The control group completed chat room assignments during class time, under in-
structor supervision, in self-selected groups, using a WebCT chat room (SCMC).
In-class participation is normally assessed based on instructor perception, and
in the control group instructors used a 10-point scoring guide (see the scoring
guide in the Appendix). Transcripts were not used to grade the discussions be-
cause doing so would have duplicated the instructors’ supervision of the activity.
The following class day, control students turned in paper compositions reporting
the information they had learned about their partners. Teachers responded to the
compositions with written comments and a grade. Grades for the chat sessions
were reported using participation sheets.
   Collaboration was harnessed in the experimental treatment by requiring stu-
dents to form their own work groups and schedule their own chat sessions outside
of class time. Most of them used their home computers. They used WebCT chat
rooms for their discussions (SCMC) and posted their compositions in a WebCT
electronic bulletin board (ACMC). Students had until the end of the week to com-
plete the chat and composition assignments. Their chat discussions were graded
the following week using transcripts and following the same 10-point scoring
guide as in the control group. Grading participation with transcripts in the experi-
mental sections took roughly the same amount of time (30 minutes) as immediate
supervision of the control group. In the electronic bulletin board, students were
able to see the compositions of other students. The teachers replied to the content
of each composition with follow-up questions and reported grades for chat and
compositions to students using a participation sheet.
   The instructors did not participate in the activity with the students, although the
instructors for the control group were available to help students. Instructors from
both groups intervened in class with direct instruction (e.g., presentation, drills,
and/or communicative activities) whenever grammatical errors common to the
majority of students were detected.
   Using synchronous communication to exchange interesting personal informa-
tion that students will later synthesize and report gives the interaction a specific
purpose, thus motivating students to stay on task. It also gives the instructor a
meaningful context for evaluation in that incomplete reports are often the result of
insufficient information exchange in SCMC. Developing the same topics in both
CMC modalities allows for practice of form and vocabulary in both interpersonal
and presentational modes. In this context students often use transcripts of their
chat to help them recall the information and to monitor and correct form.
   It should be noted that this study is part of a greater research project investigat-
Robert Sanders                                                                   65
ing the effects of reduced seat time. The experimental treatment included reduced
seat time as well as lower maximum section size. The maximum allowable en-
rollment was 30 students per section in the control group and 25 per section in
the experimental group. The control group spent 200 minutes per week in class
(including 30 minutes in the computer lab completing the chat activity) while the
experimental group spent 130 minutes per week in class (not including the out-of-
class chat activity). Both groups completed a common set of weekly online gram-
mar activities and quizzes, listening comprehension quizzes, and reading compre-
hension quizzes. Additionally, the experimental group had one online vocabulary
activity and quiz per week and one more online listening comprehension activity
and quiz per week than the control group. The time to complete these activities
was estimated at 40 minutes. In the control group, these additional assignments
were administered as textbook-based in-class activities.

Data Analysis
In the first week of classes, students were surveyed to determine their prior experi-
ence studying Spanish. Transcripts from weeks seven and eight were chosen for
analysis because at that point the students and instructors were accustomed to per-
forming the activity. The topic for week seven was a typical day in the student’s
life, and week eight’s was about favorite places. The transcripts were analyzed for
minutes of activity, turns, Spanish words, correctly spelled Spanish words (both
accounting for accents and not accounting for accents), non-Spanish words, com-
ments off task, and socially appropriate comments. Proper nouns, unintelligible
words, numerals, and special chat symbols (emoticons) were not counted. Punc-
tuation and capitalization were excluded. The discussions were not analyzed for
grammatical quality. All results were recorded individually for each student.
   Non-Spanish words were typically English but sometimes included French or
Italian words. All instances of non-Spanish words were tallied (including repeti-
tions). “Socially appropriate comments” or questions were typically greetings,
leave takings, and related inquiries. Darhower (2002) found that these comments
were a significant feature of SCMC and greatly contributed to social cohesive-
ness. Drawing on Chun’s (1994) recognition of “minimal sociolinguistic compe-
tence” (p. 28), Meskill’s (1999) concept of communities of learners, and Wenger’s
(1998) theory of communities of practice, Darhower (2002) characterized socially
appropriate comments as part of “behaviors that led to the construction of an on-
line discourse community of Spanish learners” (Darhower, p. 265).
   Four Spanish word counts were employed: (a) all instances of recognizable
Spanish words, (b) all instances of correctly spelled Spanish words including ac-
cents and diacritical marks, (c) all instances of correctly spelled Spanish words
excluding accents and diacritical marks, and (d) correctly spelled, distinct Spanish
vocabulary words, excluding accents and diacritical marks. The first three Span-
ish word counts and the other features measured in the transcripts were tallied by
a graduate research assistant who was not affiliated with the course. The fourth
count—that of distinct Spanish vocabulary words—was calculated by the author
using word lists generated by Concordance, a software program developed by
66                                                CALICO Journal, Vol. 24, No. 1
R. J. C. Watt. All distinct vocabulary words in the lists that appeared to be cor-
rectly spelled Spanish words were tallied. To simplify the counting, accents and
diacritical marks were excluded, and all misspelled words were omitted. None of
the word counts is taken alone as a definitive indicator of quality, but together they
suggest something about quantity and quality of production at the word level.
  The following fragment, taken from an experimental group discussion in week
eight, can be used to demonstrate the analysis. There were in fact three partici-
pants in this discussion, but the fragment is short and only includes two partici-
pants.
  Student 1: Mi bar favorita es el Backroom.
  Student 1: Vamos a bailar el viernes
  Student 2: Mi bar favorito es el “Pub at the en of the Universe”.
  Student 1: ¿Donde es Pub at the end of the universe?
  Student 2: ¿Cuál es su balie favorito?
This sample does not include any non-Spanish words or socially appropriate com-
ments. Because the assigned topic was favorite places, none of the comments
were considered to be “off task.” Proper nouns are excluded. The first word count
tallies 12 instances of recognizable Spanish words for the first student and 10 for
the second. The word balie is recognized as baile even though it is misspelled.
The second count shows 11 instances of correctly spelled Spanish words for the
first student and 9 for the second. The word donde is misspelled because of the
lack of the diacritical mark. The word balie is discounted for misspelling, and
favorita is accepted as a correctly spelled word even though it does not agree with
the noun. (As stated above, this analysis does not concern grammatical accuracy.)
The third count, which excludes accents and diacritical marks, shows 12 instances
of correctly spelled Spanish words for the first student and 9 for the second. The
fourth count—that of distinct and apparently properly spelled Spanish vocabulary
words—shows 10 and 7 words, respectively. Student 1 repeated es and el. In the
word list, donde appears to be properly spelled. Student 2 repeated favorito and
es.

Participants
In week seven, 46 students from the control group completed the activity and con-
sented to having their transcripts used for research. In week eight, 42 completed
the activity and consented to the case study. The students had an average 2.0
years prior study of Spanish. Completion and consent for the experimental group
included 54 students in week seven and 57 students in week eight.3 They had an
average of 1.9 years prior study of Spanish. The average size of the experimental
chat groups was 3.0 students in week seven and 3.2 students in week eight. In the
control treatment, the average sizes were 3.6 and 3.4 students per chat group each
week, respectively.
  The control and experimental group instructors were not the same. The control
students were enrolled in four sections of Spanish 101 taught by three different
Robert Sanders                                                                              67
nonnative instructors. They had an average 1.7 years prior experience teaching
Spanish as a foreign language. The experimental treatment was given to six sec-
tions taught by three instructors and the course coordinator (author). The instruc-
tors had an average 1.7 years prior experience teaching Spanish, and one was a
native speaker. The course coordinator had 8 years prior experience. Because the
study was part of a greater investigation of reduced seat time instruction involving
15 sections and 12 teachers, the instructor variable could not be isolated. Although
the conditions for testing were not pristine, they reflect the type of practical imper-
fection experienced in many large foreign language programs.

RESULTS
All results were tested for significant differences using independent samples t
tests, with a two-tailed significance value of .05 as a standard. Care should be
taken in examining the results of the t tests since there were multiple dependent
variables, but in most comparisons the p value was well above or well below .05.
The students in the experimental group had significantly better results in both
weeks with respect to total minutes of activity (+45%), turns (+107%), instances
of Spanish words (+103%), instances of correctly spelled Spanish words when
accounting for accents (+91%), instances of correctly spelled Spanish words
when not accounting for accents (+93%), socially appropriate comments (+74%),
and original instances of Spanish vocabulary words (+60%; p < .01 in all com-
parisons). The total number of non-Spanish words was significantly higher for
students in the experimental group (+239%; p < .01 in both comparisons). The
results are summarized in Table 1.
Table 1
Mean Productivity
                                     Week 7 (n = 46, 54)          Week 8 (n = 42, 57)

                                  Cont.     Exp.           p   Cont.      Exp.          p

 Minutes                          23.67     33.09     .000*    21.74     32.93     .000*

 Turns                            17.65     33.56     .000*    16.41     36.77     .000*

 Spanish words                    75.57    144.91     .000*    70.12    150.58     .000*

 Correctly spelled, w/ accents    52.02     92.74     .000*    44.86     91.95     .000*

 Correctly spelled, w/o accents   59.33    105.59     .000*    50.67    106.33     .000*

 Spanish vocabulary               40.85     62.39     .000*    37.05     62.16     .000*

 Non-Spanish words                 2.52      9.07     .001*     3.07       9.88    .000*

 Social comments                   3.41      7.04     .000*     5.07       7.74    .001*

 Comments off task                 0.00      0.06     .359      0.00       0.12    .188
* p < .05
68                                                CALICO Journal, Vol. 24, No. 1
Social Interaction and Focus on Task
On the whole, the experimental group engaged electronically in more social com-
munication. This might be expected because these students were not in immediate
contact with each other, whereas students in the control group were in each other’s
presence in the computer laboratory during regularly scheduled class time.
   Both groups appear to have been similarly attentive to the assigned task. The
results for comments off task did not reveal any significant differences. The off-
task comments were in fact made by only a few students: one student with three
off-task comments in week seven, three students with a total of seven off-task
comments in week eight. These results differed greatly from those found by Dar-
hower (2002) in a study of sociocultural aspects of SCMC. He discovered that
when students used pseudonyms and interacted anonymously, 15% to 48% of all
chat discussion was off task, with interlocutors usually leaving the assigned topic
as soon as the instructor left the chat room. The difference in outcomes between
these two studies suggests that assigning participation grades and follow-up ac-
tivities (reflective reports) is effective for keeping students on task. But it should
be noted that Darhower’s study also identified student interest and topic relevance
as prevalent determinants in student negotiation of new topics, indicating that to
stray from assigned tasks may actually be beneficial for L2 production (see also
Weasenforth et al., 2002).

Spelling Accuracy
The control and experimental groups had a similar degree of spelling accuracy.
This is suggested by comparing each student’s ratio of correctly spelled Spanish
words (all instances) to his or her recognizable Spanish words (all instances). The
ratios indicate the percentage of Spanish words that are correctly spelled (see
Table 2).
Table 2
Percentage of Spanish Words Spelled Correctly
                                Week 7 (n = 46, 54)         Week 8 (n = 42, 57)

                              Cont.     Exp.       p      Cont.      Exp.      p

 Including accents            68%       64%      .132      64%      62%      .414

 Excluding accents            79%       73%      .043*     73%      72%      .700
* p < .05
  When excluding accents, the control group had an accuracy ratio 6% higher
than that of the experimental group in week seven and 1% higher in week eight.
When accounting for accents and diacritical marks, the control group had ratios
4% higher in week seven and 2% higher in week eight. Only one of the four com-
parisons showed a significant difference (t test).
Robert Sanders                                                                             69
Production and Time on Task
Time on task probably affected the other dependent variables. The total number of
minutes active was different for each student and significantly different between
the two groups. Some insight can be gained by adjusting for this difference. The
results for each student were divided by his or her number of turns, therefore
producing a time-independent description of student production, and by his or her
number of minutes active, therefore indicating the students’ rates of production.
Although these calculations manipulate the original data, independent samples t
tests of the means for each student in the two groups resulted in p values that were,
for the most part, well above or well below .05.
   Dividing each student’s production by his or her number of turns suggests only
one consistent difference between the two groups: the experimental treatment is
susceptible to greater use of non-Spanish words per turn. Results are summarized
in Table 3.
Table 3
Mean Productivity per Turn
                                     Week 7 (n = 46, 54)         Week 8 (n = 42, 57)

                                  Cont.     Exp.           p   Cont.     Exp.          p

 Spanish words                    3.99      4.44       .149    4.19      4.20      .980

 Correctly spelled, w/ accents    2.69      2.86       .438    2.69      2.64      .843

 Correctly spelled, w/o accents   3.08      3.26       .487    3.05      3.04      .948

 Spanish vocabulary               2.37      2.05       .124    2.45      1.86      .002*

 Non-Spanish words                0.12      0.26       .011*   0.19      0.25      .200

 Social comments                  0.22      0.23       .794    0.40      0.22      .001*

 Comments off task                0.00      0.00       .359    0.00      0.00      .141

* p < .05

  Independent samples t tests of the per-turn averages revealed significant differ-
ences in only 3 of 14 comparisons: the experimental group used more non-Span-
ish words in week seven (+113%, p = .011), had fewer original instances of Span-
ish vocabulary words in week eight (-24%, p = .002), and made fewer socially
appropriate comments in week eight (-44%, p = .001). For the most part, dividing
total student production by the number of turns produces mixed results.
  The averages produced by dividing each student’s results by his or her minutes
of activity are much more suggestive of differences between the groups, and in-
dependent samples t tests of these results indicate significant differences in 11 of
the 16 possible comparisons. Results are summarized in Table 4.
70                                                         CALICO Journal, Vol. 24, No. 1
Table 4
Mean Productivity per Minute
                                     Week 7 (n = 46, 54)            Week 8 (n = 42, 57)

                                  Cont.     Exp.           p      Cont.     Exp.          p

 Turns                            0.75      1.08       .001*      0.81      1.17      .000*

 Spanish words                    3.03      4.58       .001*      3.44      4.69      .009*

 Correctly spelled, w/ accents    2.08      2.91       .006*      2.21      2.87      .047*

 Correctly spelled, w/o accents   2.37      3.33       .005*      2.51      3.34      .025*

 Spanish vocabulary               1.69      2.00       .088       1.89      1.96      .727

 Non-Spanish words                0.10      0.31       .002*      0.15      0.32      .006*

 Social comments                  0.15      0.23       .001*      0.28      0.25      .441

 Comments off task                0.00      0.00       .359       0.00      0.00      .180

* p < .05

   In both weeks, students in the experimental group had better results with re-
spect to the number of turns per minute (+44%); instances of Spanish words per
minute (+43%); and instances of correctly spelled Spanish words per minute,
both when accounting for accents (+35%) and when not accounting for accents
(+37%). These differences were significant (p < .05 in all comparisons). The ex-
perimental group also had more original instances of Spanish words per minute in
both weeks (+11%), but the differences were not significant. Dividing the number
of social comments per minute of activity produced mixed results. Students in the
experimental group had poorer results with regard to the number of non-Spanish
words used per minute (+152%), with an average of .3 per minute compared with
.1 for the control group (p < .05 in comparisons for both weeks). On the whole,
the experimental group outperformed the control group in 11 of the 16 “per-min-
ute” calculations. These results indicate greater production for the experimental
group.

Cost Effectiveness
The use of class time and university resources was considerably lower for the
experimental group. Computer labs were reserved for the control group, whereas
students in the experimental group used general access labs and home computers
to complete the assignment. For the experimental group, the chat room activ-
ity extended student-student communication beyond the limits of class time and
space.

DISCUSSION
Time on task was the most salient difference between the treatments. In the con-
trol group, grading was based on instructor perception of participation, as is typi-
Robert Sanders                                                                   71
cal of traditional classroom management. Ironically, students may have perceived
the instructor to be responsible for their level of engagement. In the experimental
group, grading was done with transcripts that recorded time on task, and, more
important, the students alone were responsible for their level of activity. The fact
that students in the experimental group averaged 33.1 and 32.9 minutes actively
participating in weeks seven and eight—exceeding the requirement by 10%—
suggests that, given more responsibility and accountability for their participation,
students will spend more time on task. This suggestion was confirmed the follow-
ing year when out-of-class chat assignments were required in all sections of Span-
ish 101 at the university. The required length for the activity was reduced to 20
minutes, but this did not affect time on task. The average time in discussion with
other students (not simply logged in) was 29 minutes in week seven and 31 min-
utes in week eight, exceeding the requirement by an average of 50%. Apparently,
students are comfortable with the medium, interested in the topics, motivated to
communicate with each other, and they collaborate responsibly.
  An obvious explanation for the increased rate of productivity and use of non-
Spanish words in the experimental group would be that students code switched
more often to bridge interlanguage gaps. After all, students in the experimental
group used, on average, 152% more non-Spanish words per minute. But the ratios
of non-Spanish words (all instances) to Spanish words (all instances) for each
group appear similar. The control group had ratios of 3% and 5% in weeks seven
and eight, respectively, compared to the experimental group’s 6% and 7% (see
Table 5).
Table 5
Ratio of Non-Spanish Words to All Instances of Spanish Words
     Week 7 (n = 46, 54)              Week 8 (n = 42, 57)
  Cont.      Exp.         p         Cont.      Exp.        p
   .03        .06      .019*        .05         .07      .136
p < .05
  These findings seem to confirm other reports that code switching in SCMC is
minimal (see Beauvois, 1992, 1997; Darhower, 2002; see also Kelm, 1992). It is
unclear whether the increase in non-Spanish words was a cause or effect of faster
production. Additionally, many of the English words used by both groups were
introduced by students asking their partners for the Spanish equivalent. The ex-
perimental group may have used this technique more often because the instructor
and other students were not physically present to supply and clarify vocabulary.
Some researchers contend that peer advising on vocabulary is beneficial for SLA
(see Darhower, 2002; Kötter, 2003).

CONCLUSION
The experimental treatment was a more effective instructional model. Altogether,
the experimental group faired better in 23 out of 32 instances in which significant
72                                                      CALICO Journal, Vol. 24, No. 1
differences were found. Student collaboration outside of class resulted in more
student participation and production. The quality of production appears—at the
word level—to be similar. The main difference is quantity of production. Existing
research suggests that the increase in contact and practice should result in greater
acquisition.

Limitations of the Study and Suggestions for Further Research
This study did not include a grammatical analysis of the discourse produced.
A comparison of grammatical accuracy and syntactic complexity is warranted.
Analysis of student improvement during the course of a term is also needed.
While Ittzes Abrams (2003) reminds us that CMC is now an important mode of
communication in its own right, the results of out-of-class SCMC on oral perfor-
mance should be compared to those of in-class SCMC. Of greater need still is a
longitudinal study of the effects of CMC on learning and acquisition after several
years of use and on the best points within L2 curricula to utilize CMC.

NOTES
1
    See Fletcher’s (1995) study of computer-assisted flight training.
2
    On negotiation in CMC, see Smith (2000).
3
  Three students in the experimental group were excluded from the study for failure to
comply with the requirements of the assignment. In week seven, one of these students
decided to write English and used 115 non-Spanish words during the session. The remain-
ing students in the experimental group used an average of 9 non-Spanish words in the as-
signment. In week eight, two students were excluded from the analysis because they used
348 and 233 non-Spanish (mainly English) words, respectively. The other students in the
experimental group used an average of 10 non-Spanish words during the activity.

REFERENCES
Atkin, M. C. (1970). Evaluating the cost-effectiveness of instructional programs. In M. C.
         Wittrock & D. E. Wiley (Eds.), The evaluation of instruction: Issues and prob-
         lems (pp. 221-238). New York: Holt, Rinehart and Winston.
Bayer, A. (1990). Collaborative-apprenticeship learning: Language and thinking across
         the curriculum, K-12. Mountainview, CA: Mayfield.
Biesenbach-Lucas, S., & Weasenforth, D. (2001). E-mail and word processing in the
         ESL classroom: How the medium affects the message. Language Learning &
         Technology, 5 (1), 135-165. Retrieved April 18, 2006, from http://llt.msu.edu/
         vol5num1/weasenforth/default.html
Blake, R. (2000). Computer mediated communication: A window on L2 Spanish interlan-
          guage. Language Learning & Technology, 4 (1), 120-136. Retrieved April 18,
          2006, from http://llt.msu.edu/vol4num1/blake
Robert Sanders                                                                          73
Beauvois, M. H. (1992). Computer-assisted classroom discussion in the foreign language
         classroom: Conversation in slow motion. Foreign Language Annals, 25 (5), 455-
         464.
Beauvois, M. H. (1997). Computer-mediated communication: Technology for improv-
         ing speaking and writing. In M. D. Bush (Ed.), Technology-enhanced language
         learning (pp. 165-184). Lincolnwood, IL: National Textbook Company.
Beauvois, M. H. (1998). Write to speak: The effects of electronic communication on the
         oral achievement of fourth semester French students. In J. A. Muykens (Ed.),
         New ways of learning and teaching: Focus on technology and foreign language
         education (pp. 165-183). Boston: Heinle & Heinle.
Chun, D. (1994). Using computer networking to facilitate the acquisition of interactive
         competence. System, 22 (1), 17-31.
Darhower, M. (2002). Instructional features of synchronous computer-mediated communi-
        cation in the intermediate L2 class: A sociocultural case study. CALICO Journal,
        19 (2), 249-278.
Egbert, J., Paulus, T. M., & Nakamichi, M. (2002). The impact of CALL instruction on
          classroom computer use: A foundation for rethinking technology in teacher edu-
          cation. Language Learning & Technology, 6 (3), 108-126. Retrieved April 18,
          2006, from http://llt.msu.edu/vol6num3/egbert
Feenberg, A. (1999, Winter). Distance learning: Promise or threat? National CrossTalk, 7
         (1). Retrieved April 18, 2006, from http://www.highereducation.org/crosstalk/
         ct0199/opinion0199feenberg.shtml
Fletcher, J. D. (1995). [Computer-assisted military training]. In A. Melmed (Ed.), The
          costs and effectiveness of educational technology: Proceedings of a workshop
          (RAND, November 1995). Retrieved July 11, 2002, from http://www.ed.gov/
          Technology/Plan/RAND/Costs/cover/html
Garrett, N. (1989). The synergism of technology and theory in classroom second language
          acquisition research. In J. E. Alatis (Ed.), Georgetown University Round Table
          on Languages and Linguistics (pp. 288-294). Washington, DC: Georgetown Uni-
          versity Press.
Garrett, N. (1991). Technology in the service of language learners: Trends and issues. Mod-
          ern Language Journal, 75 (1), 74-101.
González-Bueno, M., & Pérez, L. C. (2000). Electronic mail in foreign language writing:
         A study of grammatical and lexical accuracy, and quantity of language. Foreign
         Language Annals, 33 (2), 189-198.
Halliday, M. A. K. (1993). Towards a language-based theory of learning. Linguistics and
          Education, 5 (1), 93-116.
Hawley, D. E., Fletcher, J. D., & Piele, P. K. (1986). Costs, effects, and utility of micro-
         computer-assisted instruction (Tech. Rep. No. 1). Eugene, OR: Center for Ad-
         vanced Technology in Education.
Ittzes Abrams, Z. (2003). The effect of synchronous and asynchronous CMC on oral per-
          formance in German. Modern Language Journal, 87 (2), 157-167.
Kelm, O. R. (1992). The use of synchronous computer networks in second language in-
         struction: A preliminary report. Foreign Language Annals, 25 (5), 441-454.
74                                                    CALICO Journal, Vol. 24, No. 1
Kern, R. G. (1995). Restructuring classroom interaction with networked computers: Ef-
         fects on quantity and characteristics of language production. Modern Language
         Journal, 79 (4), 457-476.
Kiesler, S., Siegel, J. & McGuire, T. W. (1984). Social psychological aspects of computer-
          mediated communication. American Psychologist, 39 (10), 1123-1134.
Knight, Susan. (2000). Expanding enrollment while maintaining quality: A Spanish 101
         experimental model. Hispania, 83 (4), 855-863.
Kötter, M. (2003). Negotiation of meaning and code switching in online tandems. Lan-
         guage Learning & Technology, 7 (2), 145-172. Retrieved April 18, 2006, from
         http://llt.msu.edu/vol7num2/kotter
Lee, L. (2002). Enhancing learners’ communication skills through synchronous electronic
          interaction and task-based instruction. Foreign Language Annals, 35 (1), 16-24.
Levin, H. M. (1984). Costs and cost-effectiveness of computer assisted instruction (Project
          Rep. No. 84-A21). Palo Alto, CA: Stanford University, School of Education.
Maxwell, D., & Garrett, N. (2002). The challenge to language learning in higher education.
         Change, 34 (3), 22-28.
Mehan, H. (1985). The structure of classroom discourse. In T. A. Van Dijk (Ed.), Handbook
        of discourse analysis (pp. 119-131). London: Academia.
Meredith, R. (1983). Materials and equipment: The next generation. Modern Language
         Journal, 67 (4), 424-430.
Meskill, C. (1999). Computers as tools for sociocollaborative language learning. In K.
         Cameron (Ed.), Computer-assisted language learning (CALL): Media, design
         and applications (pp. 144-152). Exton, PA: Swets and Zeitlinger.
Payne, J. S., & Whitney, P. J. (2002). Developing L2 oral proficiency through synchro-
          nous CMC: Output, working memory, and interlanguage development. CALICO
          Journal, 20 (1), 7-32.
Pérez, L. C. (2003). Foreign language productivity in synchronous versus asynchronous
          Computer-mediated communication. CALICO Journal, 21 (1), 89-104.
Pérez-Sotelo, L., & González-Bueno, M. (2003). Idea: Electronic writing in L2: Accuracy
         vs. other outcomes. Hispania, 86 (4),869-873.
Salaberry, M. R. (2001). The use of technology for second language learning and teaching:
          A retrospective. Modern Language Journal, 85 (1), 39-56.
Smith, B. (2000). Computer-mediated negotiated interaction: An expanded model. Modern
          Language Journal, 87 (1), 38-57.
Sotillo, S. M. (2000). Discourse functions and syntactic complexity in synchronous and
           asynchronous communication. Language Learning & Technology, 4 (1), 82-119.
           Retrieved April 18, 2006, from http://llt.msu.edu/vol4num1/sotillo
Stepp-Greany, J. (2002). Student perceptions on language learning in a technological envi-
         ronment: Implications for the new millennium. Language Learning & Technol-
         ogy, 6 (1), 165-180. Retrieved April 18, 2006, from http://llt.msu.edu/vol6num1/
         STEPPGREANY/default.html
Swain, M. (1985). Communicative competence: Some roles of comprehensible input and
        comprehensible output in its development. In S. Gass & C. Madden (Eds.), Input
        in second language acquisition (pp. 235-253). Rowley, MA: Newbury House.
Robert Sanders                                                                       75
Warschauer, M. (1996). Comparing face to face and electronic discussion in the second
        language classroom. CALICO Journal, 13 (2 & 3), 7-26.
Warschauer, M. (1997). Computer-mediated collaborative learning: Theory and practice.
        Modern Language Journal, 81 (4), 470-481.
Weasenforth, D., Biesenbach-Lucas, S., & Meloni, C. (2002). Realizing constructivist
        objectives through collaborative technologies: Threaded discussions. Language
        Learning and Technology, 6 (3), 58-86. Retrieved April 18, 2006, from http://llt.
        msu.edu/vol6num3/weasenforth
Wei, L. (2000). Unequal election of morphemes in adult second language. Applied Linguis-
          tics, 21 (1), 106-140.
Wenger, E. (1998). Communities of practice: Learning, meaning and identity. New York:
         Cambridge University Press.

APPENDIX
Participation Scoring Guide
8-10 = The student
  a. communicates very well in discussion contexts
  b. never uses English in discussion, pair or group activities, or when asking
     questions
  c. responds fully, voluntarily elaborates on answers, and elicits discussion
     from partners by asking pertinent questions
7 = The student
  a. communicates insufficiently in some discussion contexts
  b. frequently uses English, with and without instructor permission
  c. gives short answers, has difficulty elaborating when requested
1-6 = The student
  a. is unable to communicate in most or all discussion contexts, is tardy or
     leaves early
  b. speaks mostly English in class
  c. makes mostly errors with regard to materials covered
  d. responds to contextual questions with isolated words or silence
0 = The student is absent, regardless of cause.

ACKNOWLEDGMENT
The author would like to acknowledge the assistance of Michelle Jurado, the
graduate research assistant who tallied from transcripts Spanish words, correctly
spelled Spanish words (including and excluding accents), non-Spanish words,
comments off task and socially appropriate comments. This work could not have
been performed out of context and was essential for the study.
76                                              CALICO Journal, Vol. 24, No. 1
AUTHOR’S BIODATA
Robert Sanders is Assistant Professor of Spanish in the Department of Foreign
Languages and Literatures at Portland State university. He received his Ph.D.
from the University of Arizona. His research interests include distributed use of
technology for the enhancement of face-to-face instruction and learning opportu-
nity, and the redefining of language teaching.

AUTHOR’S ADDRESS
Robert Sanders
Portland State University
Department of Foreign Languages and Literatures
FLL - NH 393
PO Box 751
Portland, Oregon 97207
Phone: 503 725 5296
Fax:     503 7255276
Email: rsanders@pdx.edu
You can also read