Structuring Communication Processes and Enhancing Public Discourse: The Delphi Method Revisited
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Structuring Communication Processes and
Enhancing Public Discourse: The Delphi
Method Revisited
Fahri Yetim and Murray Turoff
Information Systems Department, College of Computing Sciences,
New Jersey Institute of Technology, Newark, NJ 07102- 1982, USA
fahri.yetim@njit.edu; Turoff@adm.njit.edu
Abstract
Increased advances in communication technology have enabled computer-
mediated communication among geographically dispersed individuals, and
created opportunities for them to participate in socio-political decisions at
the local, national and global level. Consequently, the structuring of
communication processes for facilitating discourse among larger groups of
individuals becomes a challenge. Delphi is one of the most used methods for
structuring group communication among experts. This paper theoretically
evaluates the usability of Delphi for discourses among larger groups from
the perspective of various public discourse criteria, which are endorsed by
public discourse models in modern democracies. Our assessment of Delphi
suggests that Delphi is in principle open for the realization of several
discourse conventions in practice. The paper also discusses the value of the
implementation of Delphi-like structures in a Social Decision Support
System (SDSS) and provides some suggestions for future research.
The copyright of this paper belongs to the paper’s authors. Permission to copy without fee
all or part of this material is granted provided that the copies are not made or distributed
for direct commercial advantage.
Proceedings of the 9th International Working Conference on the Language-Action
Perspective on Communication Modelling (LAP 2004)
Rutgers University, The State University of New Jersey, New Brunswick, NJ, USA, June
2-3, 2004
(M. Aakhus, M. Lind, eds.)
www.scils.rutgers.edu/lap04/lap04.htm
235F Yetim, M Turoff
1 Introduction
Increased advances in communication technology have enabled computer-
mediated communication among large numbers of geographically dispersed
individuals, groups, virtual teams, etc., and created opportunities for them to
collaborate as well as to participate in socio-political decisions at the local,
national and global level [Hiltz and Turoff, 1993]. Different types of computer-
mediated communication and cooperation support systems aim to support the
investigation of complex topics about which many diverse and opposing views are
held [e.g., Yetim, 2000; Turoff et al., 2002; Heng and de Moor, 2003]. Supporting
discourses at a local as well as global level among large numbers of individuals
raises several challenges. One challenge is the coordination of the communication
because of the parallel and non-linear contributions of dispersed members.
Another challenge arises with respect to the issue of what model of discourse
should be ideally supported by such discourse support systems, given cultural and
political differences in assessment of what counts as a ‘good’ discourse. More
specifically: What communication convention or ‘discourse culture’ should be
established for making such systems universally usable and allowing ‘true’
discourse [Yetim, 1998a/2004b]. For example: Should ideas be presented with or
without reason or justification? Should participants relate their ideas or arguments
to that of others? Etc.
Facilitation of communication is a common practice in asynchronous group
communication systems. Delphi is one of the most used methods for structuring a
group communication process so that the process is effective in allowing a large
group to deal with a complex problem [Linstone and Turoff, 1975]. It has been
applied and successfully used in various application areas to structure
communication and facilitate decision making among geographically dispersed
experts.
The main objective of this paper is to evaluate the Delphi method from the
perspective of various discourse criteria in order to understand what conventions
or ‘discourse cultures’ are embedded in the Delphi method. We will use the
criteria of public discourse models in modern democracies [Ferree et al., 2002].
Different political traditions endorse and emphasize specific normative criteria for
a good democratic public discourse. For example, some traditions value
deliberation and dialogical form of communication, whereas others worry about its
negative effects on discourse by referring to cultural values (e.g., harmony, face-
saving, etc.). In addition, we will discuss the added value through the extension of
Delphi’s features in a Social Decision Support Systems (SDSS) environment
[Turoff et al., 2002].
Facilitating discourse is one of the main objectives of the research within the
Language-Action Perspective (LAP). Both the structuring of discourse among
larger groups and the criteria of different public discourse models have not been
236 The Language-Action Perspective on Communication Modelling 2004Structuring Communication Processes and Enhancing Public Discourse
considered in the LAP. We hope that our theoretical evaluation of Delphi method
through the lens of public discourse criteria can provide some orientations and be
useful for the realization of future discursive communication processes.
This paper is organized as follows: We will briefly introduce the Delphi method
in section 2, and provide a summary of the discourse criteria of different public
discourse models in section 3. Then, we will use these criteria to assess the Delphi
method in section 4. In addition, we will discuss the value of the implementation
of Delphi-like structures in SDSS in section 5, and finally, conclude with some
suggestions for future Delphi implementations.
2 The Delphi Method
The Delphi technique was developed by Norman Dalkey and his associates
[Dalkey and Helmer, 1951], and has been utilized as a method to obtain judgments
or opinions on a particular topic from geographically-dispersed groups of experts.
Linstone and Turoff [1975] defined the Delphi as “a method for structuring a
group communication process so that the process is effective in allowing a group
of individuals, as a whole, to deal with a complex problem.” (p.3, emphasis
added).
Delphi usually undergoes four distinct phases [Linstone and Turoff, 1975]:
(1) Exploration of the subject under discussion, wherein each individual
contributes additional information he feels is pertinent to the issue.
(2) Reaching an understanding of how the group views the issue (i.e.,
where the members agree or disagree about the issues with respect to
voting scales like importance, desirability, and/or feasibility).
(3) Exploration of disagreements (if there is a significant disagreement), in
order to bring out the underlying reasons for the differences and
possibly to evaluate them.
(4) Final evaluation, which occurs when all previously gathered
information has been initially analyzed and the evaluations have been
fed back for consideration.
In Delphi literature, a difference is made between paper and pencil Delphis and
the way it can be done by using asynchronous group communication systems.
Computer-implementations of the Delphi method provide additional support and
facilitations for group communication and reduce process losses [Turoff and Hiltz,
1995]. Although there are different implementations of Delphi as well as many
different views on what are the "appropriate," and/or "useful" procedures for
accomplishing the various specific aspects of Delphi, the following characteristics
are regarded as common in most of all Delphi implementations [Cho, 2004].
(1) Facilitation: There is a facilitator (or team for a large group) who plans
the Delphi procedure, designs and makes survey questionnaires, and
sends them to the participants. He/she gathers the obvious items so that
The Language-Action Perspective on Communication Modelling 2004 237F Yetim, M Turoff
the contributors can add the items not commonly known and collects the
list of items contributed by the participants. He/she also analyzes and
organizes them and distributes a summarized report from the responses
to the participants. In addition to human facilitators, automated
facilitation can also be used in a computer-based Delphi.
(2) Nominal Idea Generation: In the first round, participants are asked to list
their initial ideas or opinions (traditionally, this is done by using a
questionnaire form).
(3) Group Feedback: After collecting participants’ responses, the facilitator
analyzes those responses, makes a summarized report, and then sends the
report to the participants along with the subsequent questionnaire.
(4) Iteration: This process of collecting ideas and providing feedback is
repeated in a number of rounds in order to give participants the group
view from the previous rounds and the opportunity to revise their
opinions. This process of “response-feedback-change” is iterated until a
certain level of consensus is reached or the changes in an individual’s
response have been stabilized to a certain level.
(5) Controlled Discussion: In Delphi, there are no direct discussions among
participants, which is prohibited to reduce the potential process losses.
Instead, participants indirectly interact with each other through a series
of questionnaires, i.e., through adding additional ideas, responding to
each other’s ideas, or updating their original judgments based on the
group views of the problem. In most cases, voting is used to give each
participant an opportunity to compare his/her own view with the group
view.
(6) Anonymity: Contributions are conducted anonymously in order to
prohibit undesirable social pressures.
(7) Asynchronous Communication Medium: An asynchronous
communication medium is used for gathering ideas or opinions from
participants as well as for distributing them in form of a summarized
report.
The Delphi method has been widely applied in various subject areas such as
business, education, management, health care, real estate, engineering,
environment, social science, policy evaluation, tourism and transportation [Gupta
and Clark, 1996; Rowe and Wright, 1999]. The Delphi consists of many different
implementation options, such as different types of feedback (e.g. statistical
distribution, ranks, weights, reasons, etc.), different number of rounds and
different types of tasks (e.g., hypothetical events, forecasting, idea generation,
problem-solving, policy, etc.). In addition, Delphi applications differ with respect
to their goals: They usually aim either at obtaining consensus and stability among
experts or at the clarification of divergent views and elucidation of arguments.
238 The Language-Action Perspective on Communication Modelling 2004Structuring Communication Processes and Enhancing Public Discourse Despite some criticism of the reliability of the Delphi method [e.g. Sackman, 1975], the widespread use of this method indicates that it has survived this criticism. Empirical findings prove that qualitative comments and reasons of their judgments are a more effective form of feedback in Delphi than quantitative (e.g., statistical) feedback alone [Rowe and Wright, 1999]. Thus, the benefit of Delphi technique may come from qualitative comments reflecting insights of group members, combined with quantitative judgments. The advantage of Delphi has been especially illustrated in situations where the problem is either ill-defined, so complex that it cannot profit from precise analytic technique or there is no single method or data for its solution [Linstone and Turoff, 1975]. In such situations, there is a need to obtain views and judgments from geographically dispersed knowledgeable people. The diversity of knowledge and expertise that is needed to address many issues requires a group size (e.g. 20 to 100) that can never function in a simultaneous face-to-face meeting. These are typical situations for public discourses in which diverse views and interests compete. The relevant issue is whether Delphi may also be appropriate for public discourses. In order to review Delphi method from the perspective of public discourses, let us first have a look at some criteria of public discourses. 3 Models of Public Discourse in Modern Democracies Contemporary theories of public sphere focus on the role of public communication in facilitating or hindering decision-making processes in democratic societies. In the following, we provide a brief overview of what the priority concerns of democratic theories are with respect to criteria of a good public discourse. Ferree et al. [2002] provides a review of four traditions of democratic theory (i.e., Representative Liberal, Participatory Liberal, Discursive, and Constructionist) and a summary of their criteria for a good democratic public discourse (Table 1). The criteria are grouped into four broad categories: who should participate, what should be the (ideal) content of their contributions to public discourse, how ideas should be presented (style of speech preferred), and what are the desirable outcomes of the discourse process (i.e. the relationship between discourse and decision-making that is sought or feared). Each tradition places its emphasis on a different question. The priority concern of the tradition is presented in italics. With respect to the question of who should participate, the representative liberal theory values the participation of experts and representatives of the citizens, whereas all other traditions value the inclusion of the citizens. Although there are no a priori restrictions on the content, major disagreements are on the issue of whether ideas should be expressed in a process that functions as a free marketplace of ideas or in a process that empowers participants; should it be a deliberative process among citizens that aims at a general will formation or a dialog across differences that recognize and maintain differences? The Language-Action Perspective on Communication Modelling 2004 239
F Yetim, M Turoff
There are also major differences on the issue of how to present ideas. Whereas
representative liberals consider civility and emotional detachment as a proper form
of communication, others see civility as way to discipline persons and ideas into
existing categories. Discursive tradition emphasizes more on dialog under mutual
respect. Constructionist theory does not reject dialog but is wary that it can de-
legitimize narratives of personal experience and other forms of communication
and thus exclude some cultural groups.
Finally, concerning the outcome of discourse, representative liberal tradition
regards continuing debate after decision making as a waste of resources and argues
in favor of a closure, whereas other traditions are more concerned with avoiding
premature, non-consensus, or exclusionary closure.
Theory types Criteria for a good democratic public discourse
Who participates In what sort of How ideas Outcome of
process should be relation
presented between
discourse and
decision-
making
Representative Representatives; Free market- Detachment; Closure
liberal place of ideas;
Expertise Civility
Transparency
Participatory Popular Range of Avoidance of
liberal inclusion Empowerment styles imposed
closure
Discursive Popular Deliberative Dialog; Avoidance of
inclusion premature,
Mutual
non-
respect;
consensus-
Civility based closure
Constructionist Popular Empowerment; Narrative Avoidance of
inclusion creativity exclusionary
Recognition
closure
Table 1: Criteria for public discourses (adapted from [Ferree et al., 2002, p. 316])
Even though this brief sketch cannot claim to do justice to the arguments of
each tradition, it provides an overview of major controversies concerning what
counts a good public discourse. We will consider some specific arguments of the
traditions in more detail while reviewing the Delphi.
240 The Language-Action Perspective on Communication Modelling 2004Structuring Communication Processes and Enhancing Public Discourse 4 Reviewing Delphi through the Lens of Public Discourse Criteria As mentioned before, the essence of the Delphi is the structuring of group communication. The relevant issue is whether the features and procedures of the Delphi do (implicitly or explicitly) favor some of the discourse criteria. Or is the Delphi method in principle open to any of them so that it is only a matter of how Delphi is implemented in practice? When reviewing the Delphi method in this section, our reasoning will mainly be theoretical and speculative, relying on the basic assumptions and arguments of different discourse traditions as well as on the ‘implementation-independent’ features of the Delphi. We will also refer to some empirical insights to support our reasoning. (1) Who participates? With respect to the issue of who participates, Delphi method is primarily designed for and used by experts. Thus, most Delphi implementations value expertise, which is one of the criteria of representative tradition. Delphi for experts is not contradictory to the criteria of discursive tradition that values popular inclusion, but also accepts the fact that some decisions, due to their complexity, are normally made by experts. Habermas [1996], one of the main figures in this tradition, points out that it is important that there is a feedback mechanism from decisions made by the experts to the opinions building of all those affected, when important normative questions are at stake. In contrast, Delphi for experts is inconsistent with the views of other traditions: Participatory liberal tradition rejects or is ambivalent about criteria such as expertise, whereas constructionist tradition is critical to such categorizations and suspects expertise is a way of managing discourse to maintain existing relations of dominance and subordination [Ferree et al., 2002]. However, it should be mentioned that some Delphi implementations include knowledgeable individuals rather than just experts. In a Delphi on poverty, one would include some poor people as well as various experts who study poor people and other groups such as administrators of programs for poor people. Nevertheless, the relevant issue poses itself with respect to the group size: Can Delphi method be applied to facilitate communication among a larger population, which is valued by three traditions mentioned? Delphi-related empirical research on this issue has not been carried out yet. (2) In what sort of process? Concerning the content of the discourse, Delphi method allows a process which functions as a free marketplace of ideas, i.e. inclusion of a variety of beliefs. On the other hand, Delphi does not favor the criteria of transparency of the discourse. The Language-Action Perspective on Communication Modelling 2004 241
F Yetim, M Turoff The liberal tradition favors the transparency, that is, the visibility of what experts or representatives are doing so that they can be held accountable, whereas other traditions do not reject this but emphasize its insufficiency. In Delphi, the source of arguments are not visible, in order to avoid psychological effects in gathering ideas and opinions. The anonymity allows people to change their mind about ideas and votes later without being embarrassed that they have backed down or changed their view [Turoff and Hiltz, 1995]. Thus, through the anonymity feature, Delphi realizes to some degree a ‘free speech situation’ (J. Habermas), which is regarded as a precondition for conducting rational discourses. In addition, the anonymity may encourage empowerment of the participants by improving and widening the range of ideas being considered by decision makers, which is explicitly or implicitly valued by some traditions. Deliberativeness, as favored by the discursive tradition, is valued by the Delphi as well. Yet, when assessing the deliberativeness enabled by Delphi processes from the perspective of discursive tradition, the deliberativeness of traditional Delphi processes is somewhat limited since there are only indirect interactions between participants and contributions are less dialogical, as we will discuss below. In contrast, computer-implementations of Delphi improve the deliberativeness to some extent by allowing continuous discussions and by providing dynamic feedback trough the dynamic voting facility. It also allows voting to be useful to focus the discussion around disagreements evident from the voting. As a result, people may change their vote at any time to reflect the influence of the discussion. How about the criteria of recognition of difference? Constructionist tradition regards the continuing recognition of difference as equally important, and argues in favor of a dialog across difference rather than transformation into a general will [Ferree et al., 2002]. Thus, the maintaining of difference may be more facilitated by those types of Delphi that mainly aim at understanding the different viewpoints rather than at achieving a final consensus (e.g. the Policy Delphi; [Turoff, 1970]). (3) How ideas are presented? Regarding the issue of how the ideas are presented, there are no explicit criteria such as emotional detachment (a rejection of the expression of emotion) and civility as the proper form of communication. In practice, a range of communication styles including slogans and polemic speech acts or symbols as a style of expression may be used. However, in those Delphi implementations that use human facilitators, the facilitators may play a role in filtering or reformulating some expressions and thus (intentionally or unintentionally) restrict their use. According to the participatory liberal tradition, these kinds of styles play an important mobilizing role and promote empowerment. 242 The Language-Action Perspective on Communication Modelling 2004
Structuring Communication Processes and Enhancing Public Discourse
As mentioned before, dialog between participants takes place in an indirect way
in Delphi implementations. In contrast, dialog is valued by the discursive tradition
and regarded in this tradition as a process in which one provides fully developed
arguments for one’s own position and responds to the arguments of others. Even
though the participants are expected to provide arguments and reasons for their
positions, the lack of explicit reference of arguments or positions to one another
may be viewed as a restriction of deliberativeness in Delphi implementations. The
avoidance of a direct dialog between participants in the sense of a conversational
turn-taking is based on the reason that it reduces the potential process losses.
On the other hand, the lack of a dialogical form of communication in Delphi
leaves open space for other forms of communication such as using narratives of
personal experience and other more culturally preferred forms of communication
in the ‘life world.’ This is explicitly valued in the constructionist tradition.
According to this tradition, legitimating the language of the life world in discourse
privileges the experiential knowledge of ordinary citizens and contributes to their
empowerment [Ferree et al., 2002].
Who participates In what sort of How ideas should Outcome of relation
process be presented between discourse
and decision-
making
Most Free market-place No explicit
of ideas; restrictions of
No explicit criteria
implementations Anonymity rather communication
for the moment of
style (emotional
than transparency is closure;
detachment, and
include experts favored;
civility), yet, the Consensus is not
Empowerment filtering by the goal of all
or possible due to moderators Delphi
anonymity; possible; implementations;
knowledgeable
Deliberativeness is Less dialogical, i.e. Voting is mostly
valued, but prevents the used for the final
individuals; somehow limited decision (e.g.,
development of
due to indirect conversational outcome is when
No experiments form of dialog; turn-taking; the vote changes
with large Recognition of But, leaving open have stopped)
population difference (e.g., space for other
Policy Delphi for forms of
understanding of communication
diverse views) (e.g., narratives)
Table 2: Summary of the review
The Language-Action Perspective on Communication Modelling 2004 243F Yetim, M Turoff
(4) The Outcome
Finally, on the outcome question, there are no clear criteria available for the
moment when consensus has been reached in a Delphi study. Some studies suggest
that most changes occur in the transition from the first to the second round, and
that four rounds suffice to reach consensus [Erffmeyer et al., 1986]. However, a
reduction of rounds is possible in a computer-implementation of Delphi [Turoff et
al., 2004]. On the other hand, consensus on a single definition is not the goal of all
Delphi types, at least not in the initial stages, but rather, the eliciting of many
diverse points of view and potential aspects of the problem [Turoff, 1970;
Zolingen and Klaassen, 2003].
In most Delphi applications, voting is used for the final decisions. Outcome in a
continues Delphi process is when the vote changes have stopped. Noting when
vote changes have stopped is one way of determining that an outcome has
occurred. One does not attempt to force consensus. The computer can actually give
signals of the status of the process by pointing out which items no longer have
changes occurring and which ones do have changes that are occurring. There is
much research in the area of setting up an appropriate measuring instrument for the
status of a discourse and the computer opens the opportunity to design and
experiment with such instruments. Several scaling methods have been used to
provide feedback to the participants about results of human discourse [Turoff and
Hiltz 1995].
To summarize, our evaluation of the Delphi method through the lens of public
discourse criteria suggest that Delphi is in principle open to the implementation of
several criteria. Table 2 provides a summary of our evaluation. Those criteria
shown in italics are either not empirically tested (i.e., popular inclusion) or not
well satisfied by the Delphi (i.e., deliberativeness and dialog).
5 A Computer-Based Delphi Implementation: Its Value and
Limits
We have so far restricted our reflections to the general characteristics of most
Delphi implementations. In this section, we will describe the value added and the
lessons learned by the implementation of Delphi-like structures in a software
system at the New Jersey Institute of Technology. The system is called Social
Decision Support System (SDSS) [Turoff et al., 2002], and is designed to allow
large groups of people (e.g. hundreds) to address complex issues such as the
relative value of any group of related items (e.g. tasks, goals, budget allocations,
etc.). This system offers features that go beyond both the traditional paper and
pencil Delphi and some computer-implementations that replicate traditional Delphi
(e.g., use questionnaire for feedback). The SDSS has been used in field study
244 The Language-Action Perspective on Communication Modelling 2004Structuring Communication Processes and Enhancing Public Discourse
mode in different graduate courses to scaffold complex discussions in
asynchronous learning networks [Wang et al., 2002; Turoff et al., 2004].
The relevant issues that we want to deal with are: What are the values added
through the features of the SDSS? And to what extend does SDSS come closer to
the priority concern of the discursive tradition (i.e., dialog and deliberativeness)?
(1) First, SDSS provides online space for participants and changes the way of
doing Delphi exercises. Instead of providing feedback by responding to a
questionnaire, everyone can focus on what they want to consider at the time and
the computer dynamically organizes the contributions.
Figure 1 shows the user interface of the system. Participants can contribute by
entering (root) items that they feel worth discussion and by providing comments
on the items as well as by suggesting modifications of the (root) items, if
necessary. They can vote on items and be continuously informed about the current
state of the contributions and the voting results.
Figure 1: Main interaction menu [Li, 2003; Wang, 2003]
(2) Second, in contrast to traditional Delphi, the continuous nature of the SDSS
comes closer to the deliberativeness criteria of the discursive tradition. It allows
participants to continuously react to the arguments of others and to add further
arguments during a session.
Figure 2 shows the discussion structure with examples of contributions. It
includes an item and several replies that express positions in support (pro) or
against (con) the relative importance of the item or comments on under what given
conditions is more or less meaningful (neutral).
Although the fundamental idea of Delphi is that the best argument should win,
arguments did not have a central role in the previous Delphi approaches. Such a
critique is also expressed by Tapio [2002]. In order to get more in-depth
arguments, the second round in Tapio’s study was conducted by interviews,
whereby the interviewee was systematically asked to comment on the arguments
presented. The role of the moderator was to state contra-arguments in order to get
more in-depth arguments from the respondents. The methodological concern to
make the interviews rational, well-argued discussions seems to be successful.
However, such interviews can only be done within a small group. In contrast, the
online environment of SDSS is designed to facilitate argumentative discussions of
larger groups.
The Language-Action Perspective on Communication Modelling 2004 245F Yetim, M Turoff
Nevertheless, from the perspective of discursive tradition, the deliberativeness
of the discussions in SDSS remains to some extent limited due to its
conversational structure. SDSS does not support threaded discussions. This means
that participants link their positions to the corresponding (root) item and not to the
comments of others, even though their arguments refer to that of others. The
rationale for this design choice is that the threaded discussion as a mode of
organizing collaborative discourse may lead to process losses by too many levels
of responses. Delphi tends to use categories of response types (e.g. pro, con and
neutral) to a major item rather than multiple levels. Contributions may become
disorganized and confusing when large groups work on a complex issue over a
period of time. As the dialog is the central criteria for the discursive tradition, a
threaded discussion may better meet the criteria of deliberativeness than the way it
is done by the current version of the SDSS. In section 6, we will reflect on the
possible integration of these different paradigms.
ITEM
Audit trails should be
implemented within the
organization The audit trails
Pro Neutral Con
This type of logging can It is a nice thing to have,
prove to be very useful but this can be a huge
I am neutral on this,
because as Ron said this is
Figure 2: The structure of discussions
(3) Third, in contrast to traditional Delphi, one of the most interesting aspects of
SDSS is that the voting process is continuous. The dynamic voting is a different
property in the on-line environment. It clearly establishes a feedback mechanism.
For example, the use of Thurstone’s scaling method by Wang [2003] on the
246 The Language-Action Perspective on Communication Modelling 2004Structuring Communication Processes and Enhancing Public Discourse
ranking of the most important things learned in a course gives a collaborative
group interval scale and allows everyone to see graphically the degree of similarity
or dissimilarity in the ranks of individual items. This is done dynamically during
the process so vote changes move items in the scale.
The vote guides the discussion to where the disagreements or uncertainties exist
(polarized or evenly distributed votes). Individuals are allowed to change their
vote at any time so as to maximize the opportunity for the material being supplied
to influence the judgments of the other participants. Discussion causes people to
change their mind and their vote so that the process is continuous.
Furthermore, not everyone is required to vote on every item. Given that one
expects participants with many different backgrounds, not everyone may feel able
to make a judgment about all the choices. They may also feel they wish to wait
until they learn more about some of the options, before voting. This introduces a
new form of uncertainty or "measure of confidence" in the voting result, which is
not related to the statistical variance of those who have voted. It is the uncertainty
due to the individuals who have not yet voted on that item [Turoff et al., 2002].
In a case study [Turoff et al., 2004], the task was to examine, discuss, and rate
different preventive measures for Information Systems Security. Figure 3 presents
the output of the final voting results for the top 7 items. This gives only the title of
the item and definitions appear in the text of the items. In this application, the
following voting scale is used: CI=Critically Important, VI=Very Important,
I=Important, SI=Slightly Important, UI=Unimportant, and NJ=No Judgment. The
Weighted Average (WA) is also calculated.
In this case study, no one could see the voting result until they had voted. They
were advised not to vote on an item if they had no confidence in their choice. This
is a typical Delphi instruction even when done with expert groups.
Figure 3: Voting results for first 7 Items [Turoff et al., 2004]
(4) Finally, compared to traditional Delphi, there is a major improvement in the
rounds of Delphi processes. SDSS is originally designed for supporting discussion
The Language-Action Perspective on Communication Modelling 2004 247F Yetim, M Turoff
processes without moderators [Turoff et al., 2002]. Since discussions are
continuous, one may not speak of rounds. However, SDSS allows actors to group
activities into rounds. For example, by using the system in this way and instructing
participants not to vote before comments submitted in a given time period, the
SDSS system allowed to cut three rounds to only two rounds and to carry out the
equivalent of a three round Delphi process in three weeks.
The first round includes activities such as:
(a) Proposing new items for the list of items.
(b) Proposing alternative wording of an existing item and voting yes or no to
accept an alternative wording.
(c) Commenting on any single item with replies that are classified as Pro, Con,
or Neutral.
The second round starts with voting and includes activities such as:
(a) Voting on the relative value of all the items using the chosen voting method.
(b) Continuing to discuss with pro, con, and neutral comments using the current
voting results to focus the discussion.
(c) Changing one’s vote as motivated by the insights gained from the ongoing
discussion.
In sum, SDSS realizes a completely automated Delphi Method process for a
group to conduct a highly structured group process in a shorter time period when
compared to traditional Delphi and also with a less coordinating effort.
6 Conclusions and suggestions for future work
In this paper, we have assessed the Delphi method by using several public
discourse criteria. The theoretical evaluation of the Delphi showed that it is
possible in practice to implement Delphi according to the requirements of the most
criteria, and that the online environments such as SDSS provide additional features
(e.g., voting) to improve Delphi processes. We hope that our discussions can
encourage researchers and designers of discourse support systems to continue to
exploit the potential of Delphi with alternative implementations.
We recognized that there is a connection of Delphi to the Habermas’s [1984]
discourse theory. Yet, this connection is not explored in the previous Delphi
publications. There is a tradeoff between realizing dialogical discussions as
preferred by the discursive tradition, and the way of discussions in SDSS
environment. In the SDSS structure, there is only a one level discussion thread
where the response items are of three distinct types. The design of SDSS is based
on the assumption that threaded discussions in online environments would lead to
process losses. On the other hand, a possible solution for facilitating more
dialogical form of discussions with less process losses may be achieved through a
human facilitator that can help to summarize the arguments and reduce the long
list of threads. In this context, different types of discourses proposed by Habermas
248 The Language-Action Perspective on Communication Modelling 2004Structuring Communication Processes and Enhancing Public Discourse
[1984, 1996] can also be integrated to allow the sorting of issues according to their
validity claims (e.g., theoretical discourse for the truth claim and effectiveness,
practical discourses for the normative rightness, etc.). These discourses may
improve reflective practice [Yetim, 2004b]. More experiments and field trials are
needed to investigate these rather critical design choices.
We should additionally note that our reflections on the value of SDSS are partly
based on the results of a case study with a group of 20 students. Although the field
trial demonstrated that Delphi-like structures in SDSS can support large groups
engaged in complex discussions, the public discourse should, however, allow
thousands of participants. An experiment with larger number of participants would
bring more insights on the limits and advantages of the structures embedded in the
SDSS system. In fact, many of the tools that discourse analysis commonly
employs to understand a finished discourse can be redesigned to be dynamic
measuring instruments providing guidance to the participants to better understand
what they have and should accomplish in the discourse.
Moreover, it is also an important matter for this system to allow participants to
arrive at a consensus on the meta-variables such as the thresholds for such things
as acceptance or replacement.
Finally, there is a need to tailor the structures of Delphi not only to the nature of
the application and the objective, but also to the group and particularly to the
cultural preferences of participants [Yetim, 1998/2004a]. For example, with
respect to the anonymity feature of computer-mediated communication, some
studies show that this feature may have culturally different influences in
persuasive argumentation [e.g., El-Shinnawy and Vinze, 1997]. In online Delphis
it is possible to use pennames which provides the benefits of anonymity but allows
each participant to maintain a persona that allows the totality of their views to be
represented.
All in all, with the ability to implement the Delphi in online environments as a
continuous asynchronous process, researchers and designers have more
opportunities for implementing and testing varied structures of Delphi and for
comparing them with alternative group communication structures.
References
[Cho, 2004] I.H. Cho. The effect of Delphi structure on small and medium-sized
asynchronous groups, Ph. D. Dissertation, January 2004, New Jersey
Institute of Technology, Information Systems Department.
http://www.library.njit.edu/etd/index.cfm
[Dalkey and Helmer, 1951] N. Dalkey, O. Helmer. The Use of Experts for the
Estimation of Bombing Requirements – A project Delphi Experiment,
Rand Report RM-7272-PR, November 1951.
The Language-Action Perspective on Communication Modelling 2004 249F Yetim, M Turoff
[El-Shinnawy and Vinze, 1997] M. El-Shinnawy, A.S. Vinze. Technology, Culture
and Persuasiveness: a study of choice-shifts in group settings. Human-
Computer Studies 47 (1997), 473-496
[Erffmeyer et al., 1986] R.C. Erffmeyer, E. Erffmeyer, and I.M. Lane. The Delphi
Technique: An empirical evaluation of the optimal number of rounds.
Group Organ. Stud. 11 (1986), 120-132
[Ferree et al. 2002] M. M. Ferree, W. A. Gamson; J. Gerhards, D. Rucht. Four
models of the public sphere in modern democracies. Theory and Society
31, 289-324.
[Habermas, 1996] J. Habermas. Between Facts and Norms. Cambridge, MA, MIT
Press.
[Habermas, 1984] J. Habermas. The Theory of Communicative Action: Reason and
The Rationalization of Society. Boston, MA, Beacon Press (Vol. I)
[Heng and Moor, 2003]. M.S. Heng, de Moor. From Habermas’s communicative
theory to practice on the internet. Information Systems Journal, 13(2003),
331-352
[Hiltz and Turoff, 1993]. S.R. Hiltz, M. Turoff. The Network Nation, Revised
Edition, MIT Press.
[Li, 2003] Z. Li Design and Evaluation of a Voting Tool in a Collaborative
Environment, Ph.D. Dissertation, 2003, New Jersey Institute of
Technology, Information Systems Department.
http://www.library.njit.edu/etd/index.cfm
[Linstone and Turoff, 1975] H. Linstone; M. Turoff (editors). The Delphi Method:
Techniques and Applications, Addison Wesley Advanced Book Program,
1975. (Online version can be accessed via http://is.njit.edu/turoff.
[Tapio, 2002] P. Tapio. Disaggregating policy Delphi Using cluster analysis as a
tool for systematic scenario formation. Journal of Technological
Forecasting and Social Change, 70(2002), 83-101.
[Turoff et al. 2004] M. Turoff, Z. Li, Y. Wang, H. Cho, and S.R. Hiltz, S. R.
eLearning Enhancement through the Delphi Method. OZCHI Meeting
Australia, November 2004.
[Turoff et al. 2002] M. Turoff, S.R. Hiltz, H. Cho, Z. Li, and Y. Wang. Social
Decision Support Systems (SDSS). Proceedings of the 35th Annual
Hawaii International Conference on System Sciences (HICSS) 2002.
[Turoff and Hiltz, 1995] M. Turoff, S.R. Hiltz. Computer Based Delphi Processes.
In: Adler, M. and Ziglio, E. (Eds), Gazing Into the Oracle: The Delphi
250 The Language-Action Perspective on Communication Modelling 2004Structuring Communication Processes and Enhancing Public Discourse
Method and Its Application to Social Policy and Public Health, London,
Kingsley Publishers, pp. 55-88.
[Turoff, 1970] M. Turoff. The Policy Delphi. Journal of Technological
Forecasting and Social Change, (2:2).
[Van Zolingen and Klaassen (2003]. S.J. Van Zolingen, C.A. Klaassen. Selection
processes in a Delphi study about key qualifications in Senior Secondary
Vocational Education. Journal of Technological Forecasting and Social
Change, 70(2003), 317-340.
[Wang, 2003]. Y. Wang. Design and Evaluation a List Gathering Tool in a Web-
based Collaborative Environment, Ph.D. Dissertation, 2003, New Jersey
Institute, Information Systems Department.
http://www.library.njit.edu/etd/index.cfm
[Wang et al. 2003] Y. Wang, Z. Li, M. Turoff, and S.R. Hiltz. Using a Social
Decision Support System Toolkit to Evaluate Achieved Course Objectives.
Proceedings AMCIS 2003.
[Yetim, 2004a] F. Yetim. Universal Actability: Towards an integral Understanding
of Universal Usability, (Intercultural) Communication Competence, and
Information Systems Actability. In: Proceedings of Language-Action
Perspective on Communication Modeling (LAP 2004), Rutgers, New
Brunswick, NJ (USA), 2-3 June 2004.
[Yetim, 2004b] F. Yetim. A Meta-Communication Model for Reflective Practice
in Information Systems Development: A Discursive-Ethical Approach.
Submitted to Information and Organization.
[Yetim, 2000] F. Yetim. Providing Informational Support for Argumentation: The
ISA Project. In: U. Reimer (ed.): Proceedings of the Third International
Conference on Practical Aspects of Knowledge Management (PAKM
2000), Basel/Switzerland, 30-31 October 2000.
[Yetim, 1998] F. Yetim. Cultural Aspects of Group Support Systems. In:
Zimmermann, H.; Schramm, V. (eds.): Knowledge Management and
Communication Systems (Proceedings of 6th international Symposium on
Information Science - ISI'98, 4 -7, Nov. 1998, Prague). Konstanz:
Universitätsverlag, 344 -357.
The Language-Action Perspective on Communication Modelling 2004 251F Yetim, M Turoff 252 The Language-Action Perspective on Communication Modelling 2004
You can also read
