Integrating the Personal Software Process (PSP) across the Undergraduate Curriculum

 
NEXT SLIDES
Integrating the Personal Software Process (PSP) across the Undergraduate Curriculum

                                      Massood Towhidnejad, Thomas Hilburn
                                          Computer Science Department
                                      Embry-Riddle Aeronautical University
                                            Daytona Beach, FL 32114
                                         [towhid, hilburn]@db.erau.edu
                                 http://erau.db.erau.edu/~[towhid, hilburn]

                      Abstract                             overcome this has been the introduction of project
                                                           courses where students are grouped into teams and
This paper summaries some Embry-Riddle                     assigned a "realistic" (but toy-sized by industry
University activities on the incorporation of Watts        standards) problem to solve. This has not proven to
Humphrey’s Personal Software Process (PSP)                 be effective for several reasons: 1) students are
throughout the undergraduate computer science              expected to apply methods and techniques of
curriculum. This paper includes a description of the       software development to a "realistic" problem while
project’s objectives, a discussion of the activities       they are learning these topics; 2) very little
engaged in by students, an explanation of how the          instruction on planning, teamwork, and intergroup
activities were integrated into the curriculum, a          coordination is given; 3) faculty, in most part, lack
description of the data collected and an assessment        knowledge of practical aspects of software
of the impact and value of the project. Finally ideas      development and project management; and 4) an
about modification and extension of the project’s          academic semester is too short a time period to learn
concepts are presented.                                    and practice all the life cycle issues. In addition,
                                                           many topics critical to software companies are not
Introduction                                               even discussed, often because the university faculty
                                                           members are simply unaware of industry’s primary
The existence of a gap between industry software           issues. Some of these topics are quality control,
engineering needs and academic software                    change management, configuration control, software
engineering education has been widely reported             requirements and specifications, and software sizing,
[1,2,8]. Industry requires software engineers but          estimating, and planning [8].
Universities are educating computer scientists [2].
A recent Software Engineering Institute report             It is almost impossible to design a program that will
succinctly states the problem: "Students are not           address all of the above issues. However, we believe
prepared to make the jump between computer                 that by modification of the existing traditional
science or software engineering in school to               computer science programs, we will be able to get
software engineering in the professional world."           closer to graduating students with qualifications that
                                                           are needed currently in industry. The computer
One reason for the gap between what students learn         science faculty at Embry-Riddle University have
in Universities and the skills required by the             developed a computer science program that addresses
software industry is that few existing academic            some of the issues. This program is centered around
programs address the competencies expected of              an aviation application domain and emphasizes good
incoming software engineers. These are                     software engineering practices. Some of the key
programming in the large, ability to work in teams,        elements of the program are: 1) software
knowledge of process, quality and the ability to           engineering concepts introduced intp the first year
estimate time and labor costs. Computer science            courses (CS1 and CS2) and expanded on in
education, where most software engineering                 subsequent years; 2) team software development is
education occurs, too often focuses on individual          introduced in the junior year, and emphasized
contributions rather than managed group efforts            through at least seven courses. A detailed discussion
involving these competencies. Group efforts,               on the software engineering aspects of this program
however, are the norm in the software industry [8].        is contained in reference [2].
Current curricula do not allow students to develop
real software, but rather focus on what can be called      In the following sections, we will discuss one specific
"toy projects in toy situations" [1]. One approach to      experiment that has been used to implement the
incorporation of software engineering practices          In the 1995-1996 academic year we decided to
across the curriculum.                                   introduce software process concepts and activities
                                                         into our CS1 and CS2 courses. We used an early
                                                         draft version of Watts Humphrey's Introduction to
The DQW/PSP Experiment                                   the Personal Software Process [5] to teach the
                                                         material. The book is written for students new to
In recent years there has been a great deal of           computer science and software engineering. Its
emphasis on the assessment and improvement of the        narrative style, examples and exercises have been
software development process. The Capability             tailored to first year computer science majors. There
Maturity Model (CMM) developed at the Software           is a major emphasis on time management, and the
Engineering Institute is used by many organizations      PSP material on estimation and design has been
as a basis for the formulation and the maintenance of    simplified. Also, there are no prescribed
an effective development process [1,7]. In addition      programming assignments for the PSP activities, so
to our undergraduate program in Computer Science,        the material can be used with a variety of different
we offer a Master of Software Engineering (MSE)          programming assignment profiles.
program that concentrates on the development of
effective processes for software development [6].        We called this project "Doing Quality Work"
                                                         (DQW). (Throughout this paper we refer to this
The Personal Software process (PSP) was created by       project as DQW/PSP.) Our goal was to emphasize to
Watts Humphrey [4] to address the need for               the students, from the very beginning, how important
individual software engineers to acquire a               quality was in their work. More specifically our
disciplined and effective approach to writing            objectives were:
programs. The philosophy behind the PSP is that an
organization’s ability to build large-scale software             Provide students with learning experiences
systems is dependent upon the ability of its                      that will help them to understand the
individual software engineers to develop high quality             importance of a disciplined approach in the
small-scale programs in a disciplined, effective                  development of software;
manner. The PSP is designed to help engineers
organize and plan their work, track their                        Engage students in activities in which they
performance, manage software defects, and analyze                 use the elements of a defined personal
and improve their personal process. The PSP is                    software process;
currently being taught in a number of industrial
organizations and has been offered at the senior                 Provide a foundation for further
undergraduate level and the graduate level in various             development/improvement of the student's
academic programs.                                                personal software process and its use in
                                                                  individual and team projects.
The basic PSP content (as described in Humphrey’s
book [4]) consists of ten programming assignments        We started with time management and beginning
and four reports. Students start with a simple           planning activities within CS1. We felt that a
process that includes estimating and recording their     beginning college student could best assimilate and
effort for the different process phases. As students     benefit from the time management activities.
progress through the assignments, they learn about,      However, although all students entering our CS1
and use, more detailed and mature processes. The         course had some programming experience, we did
PSP relies heavily on data collection, mathematical      not feel they were yet ready for a formally defined
estimation techniques, formal reviews (design and        software development process. At this stage we felt
code), and the use of analysis to improve process        they neither understood nor appreciated the problems
effectiveness.                                           in modern software development. Nor did they have
                                                         any real comprehension about the role of a software
We are using the PSP as the basis for the first course   engineer.
in our MSE program. The PSP serves as an
introduction to software engineering and as a            In CS2 we formally introduced PSP activities.
foundation for process activities that are introduced    Students carried out project planning for each
in subsequent courses.                                   project. This included using historical data to
                                                         estimate size, effort. and quality (defect projection).
Actual data for each project was collected and             example, it was hard for them to decide on whether
recorded on a summary sheet. We felt students at           the time spent coding after a compile need to be
this level (after finishing CS1 and their first semester   recorded in coding, compile, testing or even design
in college) had the maturity and were ready for a          stage.
more disciplined way to develop their programs. We
also felt that delaying the introduction of PSP for a      As might be expected, freshman students do not
further semester (or year) would allow sloppy and          relish this sort of intensive record keeping. A student
undisciplined programming practices to become              opinion survey, taken about nine weeks into the
more entrenched and make such practices much               course in the fall of 1995, indicated resentment about
more difficult to change.                                  including this material in CS1. Many students
                                                           voiced concern about sacrificing time needed to study
                                                           programming material in order to perform
Material covered in “Computer Science 1" (CS1)             DQW/PSP work.

In CS1, topics in time management, scheduling, and         An analysis of the comments on the CS1- Fall 1995
planning were combined with the normal material            survey form show the following reasons for the
(introduction of computer science and fundamentals         negative attitude toward DQW:
of programming) [3]. Students were required to
develop and/or use several types of documents: time                The way that DQW was introduced into
logs, task schedules and plan/summary documents.                    CS1
After the introductory work was completed, students
began each week by estimating the effort (time)                    Age and lack of maturity of students
necessary to complete their CS1 tasks. Throughout
the week they recorded the time it actually took to                Lack of commitment of some students to
complete those tasks, and, at the end of the week,                  computer science/software engineering as a
they summarized the time spent on the week’s                        career (some are not majoring in CS and
activities and compared their estimated times with                  some CS majors have the normal freshman
their actual times. For their programming tasks                     doubts about their long-term goals)
students recorded their estimated and actual data for
both effort (time in minutes) and program size (lines              The effort required for DQW work versus
of code).                                                           the apparent reward

During our first offering, students were also asked to     As a result of this analysis, changes were made in the
keep an accurate time log which represented the time       organization and delivery of CS1.
they spent on CS1 tasks. This time included
interruptions down to the minute. The DQW/PSP              The main change in the delivery technique was to try
material was distributed rather uniformly throughout       to make the DQW/PSP material as transparent as
the semester. On the average, one chapter of the text      possible in the overall delivery of the CS1 subject
was covered every one and a half weeks. About              matter. We also decided to cover much of the time
every three weeks, a quiz on DQW/PSP material              management/planning material in the first three
was given. Starting in the early part of the semester,     weeks of the semester. This allowed students to
students were asked to keep track of nine separate         start collecting data at an earlier point. This
tasks: class attendance, reading Ada material,             concentration of most of the DQW material in the
preparation for a quiz or test, reading and                first part of the course eliminated the regular
preparation for DQW material, and programming.             interruption of normal course material (with
The programming task was divided into five                 scheduled lectures) throughout the semester, while
subtasks: requirements, design, coding, compile, and       still allowing for selective re-enforcement of DQW
test. In the latter part of the semester we simplified     concepts, when necessary. The second change, was
the programming task, by requiring all programming         to relax some of the requirements for time tracking.
activities to be combined into one activity. This was      Students were no longer required to keep track of
based on the fact that, at this time, a majority of        small interruptions, and only asked to estimate
students had a problem distinguishing between the          interruptions of five minutes or more.
time spent on each stage of the life cycle. For
                                                           DQW assignments consisted of time management
exercises where students were asked to keep track of      spring semester of 1996. The DQW/PSP efforts were
the time they spent on the following four activities:     concentrated on software defect management and
                                                          used the second half of Humphrey’s introductory text
        CLASS             the time spent in class        [5]. A simplified version of PSP was combined with
                                                          the usual CS2 topics (introduction to data structures
        DQW               the time spent on filling      and algorithms) [3]. Students were asked to estimate
                           out forms and performing       and record data about effort in each of six phases of a
                           necessary calculations.        program development process: planning, design,
                                                          code, compile, test, and postmortem. As the course
        PROGRAMMING                                      progressed, students were asked to identify and
                   the time spent on a                    record their defects. They cited the phases in which
                   programming assignment                 defects were injected and in which they were
                                                          removed. A primary focus of this effort was to
        READING           the time spent preparing       encourage thorough and productive code reviews.
                           for class, including class,
                           quiz and exam
                           preparation                    During the Spring 1996 semester students were
                                                          assigned seven programming projects. Each
There were a total of eleven weeks of DQW/PSP             assignment involved PSP work (maintaining timing
assignments. Each weekly assignment consisted of          logs, defect logs, summary data forms) that counted
two sub assignments: a planned/estimated time that a      for 10 to 20 percent of their assignment grade. Each
student would spend on any of the four activities         new PSP concept was introduced in a fifteen to
during a week, and the actual time the student spent      twenty minute class discussion. Whenever and
on each of the four activities. Chart 1 shows the         wherever possible the PSP concepts were related to
average estimation error (for the 36 students) for the    the normal course material. A reading assignment
CLASS, DQW, PROGRAMMING and READING                       was made prior to the introduction of each new PSP
tasks for weeks 2 through 11. The estimation error is     concept and any new forms were made available (in
computed using the below formula:                         text format) in a public directory on our computer
                                                          network. New concepts were introduced every other
                                                          assignment, so students had a couple of assignments
         100 * (actual - estimate) / estimate             to get used to a new concept. Defect data was
                                                          collected for programs 3-7, and a formal code review
The error for PROGRAMMING and READING                     was required for programs 5-7.
indicate more irregularity in this data than for the
DQW and CLASS data. This can be attributed to             An interesting and beneficial side effect of this
the variation in assignment difficulty and course         experiment has been the large supply of data that is
topics, and the fact that students are learning new       made available to the teacher. The data can provoke
material for each assignment. On the other hand, the      questions about the effectiveness and productivity of
CLASS and DQW data estimates show little error.           student programmers and the quality of their work.
This can be attributed to the fact that these tasks       The data then provide the basis for a detailed
have less variation from week to week and, hence,         analysis of such questions.
are easy to predict.
                                                          As an example, consider Charts 2 and 3. Chart 2
Notice that in the last week of CLASS data the error      shows the defects/kLOC (normalized defects) for all
is relatively large ( -33% = 100 * (120 - 180) / 180 ).   students for program 4. Chart 3 shows the
There was a scheduled study day (no class) that week      defects/kLOC for all students for program 6. For
and a close look at individual data showed the            program 4 most students had 100 or more
students were not aware of the study day when their       defects/kLOC, while for program 6 most students
estimates were made.                                      had fewer than 50 defects/kLOC. Why such a
                                                          dramatic change? One explanation might be that
                                                          formal code reviews began in program 5 and that by
Material covered in “Computer Science 2" (CS2)            program 6 the code review was so effective that
                                                          program quality (by one measure) was dramatically
DQW/PSP activities were introduced into CS2 in the        improved. However, a closer look at the data reveals
some other possible factors for this improvement;           throughout the CS curriculum.
type of program or quality of testing .
                                                        5   DQW/PSP data provides teachers with
                                                            valuable information for analyzing student
DQW/PSP Experiment Results and Expansion                    effectiveness in completing course work -
                                                            especially programming projects.
Based on the objective and subject evaluations of the
DQW/PSP activities, in the 1995-1996 academic
year, we do not feel that DQW/PSP objectives have
been fully achieved. However, we have made some
meaningful progress. We think that with additional
effort, and some changes in our approach, we believe
we are getting closer to achieving our objectives.

We are also working on a world wide web project
involving automated data entry by the students
involved in DQW/PSP activities. This should reduce
the burden on students and make it easier to collect
and analyze data.

Students are required to perform the full PSP
activities in two additional courses: Data Structures
and Algorithms and Computer Organization II. We
have introduced some aspects of PSP in second third
and the fourth year of our undergraduate curriculum.
For example, the students are required to follow the
time management activities in all of their team
project courses. Of course the team project
introduces a new challenges in relation to the PSP
experiment, since PSP’s foundation is based on the
individual effort. Currently, our graduate program
involve, with cooperation of Watts Humphry, an
experiment called the Team Software Process (TSP).
Soon, we hope to be able to use the result of this
experiment in our undergraduate applied computing
classes.

The following list summarizes the conclusions of our
DQW/PSP work:

1       Humphrey’s introductory text [5] provides
        an excellent foundation and motivation for
        teaching DQW/PSP concepts.

2       There is a need for simplification and
        automation of forms, logs and record
        keeping.

3       We must continue to adjust and modify our
        teaching techniques to accommodate and
        integrate DQW/PSP concepts into CS1 and
        CS2.

4       DQW/PSP concepts needs to be integrated
References

1      Dion, Raymond, "Process Improvement and
       the Corporate Balance Sheet", IEEE
       Software, July 1993.

2      Hilburn, T., Hirmanpour, I. and Kornecki,
       A., "The Integration of Software
       Engineering into a Computer Science
       Curriculum", Lecture Notes in Computer
       Science: Software Engineering Education,
       Proceedings of the Eight SEI Conference
       on Software Engineering Education, March
       1995.

3      Hilburn, T. and Salimi, A., Ada, From Top
       to Bottom, Embry-Riddle Aeronautical
       University, August 1995..

4      Humphrey, Watts S., A Discipline for
       Software Engineering, Addison-Wesley,
       Reading, MA, 1995.

5      Humphrey, Watts S., An Introduction to the
       Personal Software Process, Addison-
       Wesley, Reading, MA, 1997

6      Khajenoori, S., "Process-Oriented Software
       Education", IEEE Software, November
       1994, pp. 99-101.

7      Paulk, M., et. al., "Capability Maturity
       Model, Version 1.1", IEEE Software, July
       1993, pp 18-27.

8      Tucker, A. M., "Computing Curricula
       1991", Communications of the ACM, 34(6),
       June 1991, pp. 68-84.
Chart 1: Average Effort Estimation Error
             36 students - CS 115 - Spring 1996
150

 50

                                                  Class

                                                  DQW

 -50                                              Programming

                                                  Reading

-150
                       Weeks 2-11
You can also read
NEXT SLIDES ... Cancel