How to Measure Congruence: Comparing Three Measures in Central America - Revista Española de ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
doi:10.5477/cis/reis.169.63
How to Measure Congruence: Comparing
Three Measures in Central America
Cómo medir la congruencia: comparando
tres medidas en América Central
Annabella España-Nájera and María del Mar Martínez Rosón
Key words Abstract
Latin America Recent literature on congruence has focused on the development of
• Congruence indicators that improve its measurement. In this methodological article,
• Ideology we contribute to this literature by comparing the strengths and limitations
• Measurement of three of these new measures —Golder and Stramski (2010), Lupu et al.
• Representation (2017) and our own measure— and compare their results when applied to
Central American cases to measure ideological and issue congruence.
This set of cases are ideal for the comparison because the preferences of
citizens and representatives do not follow a normal distribution. The
comparison highlights the implications that methodological choice has on
the study of congruence and representation and the need to develop
stronger methodological discussions in this literature.
Palabras clave Resumen
América Latina La literatura reciente sobre congruencia se ha centrado en el desarrollo
• Congruencia de indicadores que mejoren su medición. En este artículo metodológico
• Ideología contribuimos a esta literatura comparando las fortalezas y limitaciones
• Medición de tres de estas nuevas medidas —Golder y Stramski (2010), Lupu et al.
• Representación (2017) y nuestra medida— así como sus resultados una vez que se
aplican a los casos de América Central para medir la congruencia
ideológica y en políticas públicas. Estos casos son adecuados para la
comparación, ya que las preferencias de los ciudadanos y de los
representantes no siguen una distribución normal. La comparación
evidencia las implicaciones que tienen las elecciones metodológicas en
el estudio de la congruencia y la representación, así como la necesidad
de desarrollar discusiones metodológicas más sólidas en esta literatura.
Citation
España-Nájera, Annabella and Martínez Rosón, María del Mar (2020). “How to Measure Congruence:
Comparing Three Measures in Central America”. Revista Española de Investigaciones Sociológicas,
169: 63-84. (http://dx.doi.org/10.5477/cis/reis.169.63)
Annabella España-Nájera: California State University | aespanajera@csufresno.edu
María del Mar Martínez Rosón: Universidad de Salamanca | roson@usal.es
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-8464 How to Measure Congruence: Comparing Three Measures in Central America
Introduction of this growing literature is found in the work
on Latin America (Buquet and Selios, 2017;
Scholars1 of representation frequently use Luna and Zechmeister, 2005; Lupu and War-
the concept of congruence to examine the ner, 2017; Otero Felipe and Rodríguez
degree to which the preferences of represen- Zepeda, 2010; Rodríguez Zepeda, 2017;
tatives mirror citizens’ preferences. Studies Siavelis, 2009).
using ideological and programmatic congru-
In these newer analyses of congruence,
ence to evaluate the quality of representation scholars have contributed to methodological
in democracies are based on the principle of developments on the topic, including intro-
responsible party government. That is, the ducing new measures. The proliferation of
understanding that democratic representa- measures, however, has its drawbacks. Spe-
tion means “that citizens’ issue preferences cifically, we argue that by using various mea-
should correspond to the position or behavior sures of congruence, the accumulation of
of their representatives” (Powell, 2004: 274). comparative knowledge can be hindered.
This literature has introduced important de- Also, the proliferation of measures has made
bates to our understanding of representation it challenging to have a substantive discus-
(Achen, 1978; Budge and McDonald, 2007; sion of congruence and its relationship to
Dalton, 1985; Golder and Stramski, 2010; representation. With studies using different
Huber and Powell, 1994; Lupu et al., 2017; measures of congruence, scholars have
Powell, 2009; Warwick, 2016) by addressing struggled to consider the theoretical implica-
two significant questions. What is the best tion of the measures and their varying results.
way to measure congruence and what fac- This article seeks to address this challenge.
tors lead to higher levels of congruence? Contributing to the growing literature on con-
Historically, studies of congruence have gruence in Latin America and other newer
focused on advanced industrial democracies democracies, it undertakes an evaluation
(Andeweg, 2011; Blais and Bodet, 2006; Dal- and comparison of three such measures. The
ton, 1985; Ezrow, 2007; Golder and Stramski, goal is to consider the advantages and dis-
2010; Huber and Powell, 1994; Önnudóttir, advantages of each measure on the study of
2014; Powell, 2009; Reher, 2018; Thomas- congruence.
sen, 1999). Recently, with the increasing The three measures that we selected share
availability of new reliable sources of elite one important similarity while also having cer-
and public opinion data from third wave de- tain differences that make for a useful com-
mocracies, scholars have been more likely to parison. All three measures operationalize
use similar approaches (Blais and Bodet, congruence as a many-to-many relationship.
2006; Freire and Belchior, 2013; Real-Dato, That is, the goal of the three measures is to
2018; Tsatsanis et al., 2014). A good example analyze the extent to which the “collective
body of representatives reflects the ideologi-
cal preferences of the citizens” (Golder and
1 We wish to thank Fabrice Lehoucq, Lars G. Svåsand, Stramski, 2010:10). This important similarity
Michelle Taylor-Robinson, the participants in the Work-
shop of the Research Group of Elites and Political Parties makes them comparable. In other words, we
y Partidos Políticos (GREP, for its initials in Spanish) in should expect similar results from them. One
the Institute Social and Political Sciences in Barcelona difference between the measures is how they
(ICPS, for its initials in Spanish) and the participants in
the Seminar of Public Opinion in Latin America of the calculate the similarity between the prefer-
Institute of Ibero-America of the University of Salaman- ences of citizens and their representatives.
ca for their comments to the previous versions of the This difference makes for an interesting com-
manuscript. We would also like to thank the Anonymous
reviewers of the journal for their insightful comments and parison that allows us to consider the implica-
suggestions which have greatly improved this work. tions of adopting any one of these measures.
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-84Annabella España-Nájera and María del Mar Martínez Rosón 65
Two of the measures assess the correspon- ence equally. This is critical when dealing
dence between preferences using distribution with cases in which the preferences of citi-
functions. The first, proposed by Golder and zens and representatives do not have normal
Stramski (2010), calculates commutative dis- distributions. Second, our comparison high-
tribution functions. The second, developed by lights the importance of carefully considering
us, estimates the overlap between preferenc- the implications of different measures on our
es utilizing probability distribution functions. results. Scholars should carefully consider
The last measure, presented by Lupu et al. how the measures will affect their results so
(2017), takes a very different approach. In- as to begin accumulating knowledge on con-
stead of using distribution functions to calcu- gruence in third-wave democracies. Finally,
late congruence, this approach estimates the our empirical findings on Central America
distance between generalized histograms to suggest potential future directions for re-
develop a multi-dimensional measure. search that consider the theoretical relation-
To compare the three measures, we ex- ship between congruence and democracy in
amine both ideological and issue congru- a more thorough manner.
ence in Central America. Elite level data from
the Latin American Parliamentary Elites sur-
vey (PELA, for its initials in Spanish)2 and Conceptualizing congruence
public opinion data from the AmericasBa-
Congruence measures the degree to which a
rometer survey3 were used. To calculate
correspondence exists between the prefer-
ideological congruence, 44 surveys were
ence of citizens and those of their elected
matched. Twenty-two of these were from
representatives. While this is a straightfor-
PELA and 22 were from the AmericasBarom-
ward concept, its means of operationaliza-
eter survey, covering fifteen years, between
tion is not. Despite the considerable attention
1999 and 2014. To measure issue congru-
received by congruence in the literature, no
ence, twelve surveys were matched, six from
consensus exists as to the best way to com-
PELA and six from AmericasBarometer,
pare the preferences of citizens and their rep-
spanning a two-year period, from 2010 to
resentatives5. Major differences in this litera-
20124.
ture include how the principal-agent
The evaluation and comparison of the relationship should be conceptualized, how
three measures allows us to make three rec- to best capture the correspondence between
ommendations. First, we argue that there are citizens/voters and elected officials (parties,
significant advantages to the measure of government or legislators), and what data
congruence that we have developed, since it should be used to measure congruence6. As
treats every point on the scale (whether we Golder and Stramski (2010: 95) clearly illus-
use ideology or issues) to measure congru- trate, these methodological choices have
2 We wish to thank the Latin American Parliamentary
5 For a detailed discussion of the congruence measures,
Elites Project (University of Salamanca) and its principal in particular, how they have been applied to older de-
supporters (the Spanish Ministry of Economy and Com- mocracies, see Andeweg (2011), Golder and Stramski
petitiveness through research project, CSO2015- (2010), and Powell (2009).
64773-R) for access to data.
6 Whether a measure is absolute or relative is another
3 We wish to thank the Latin American Public Opinion way by which measure may differ (Golder and Stramski,
Project (LAPOP) and its principal supporters (The United 2010: 91-98). Relative measures normalize congruence,
States Agency for International Development, the Inter- with regards to the dispersion of citizen preferences and
American Development Bank, and Vanderbilt University) provide a metric-free measure of congruence, while abso-
for making the data available. lute differences are not metric-free (Golder and Stramski,
4 For a full list of the matching surveys, see Appendix I. 2010: 95).
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-8466 How to Measure Congruence: Comparing Three Measures in Central America
important implications for any conclusions position of two distributions, i.e., one corre-
that may be drawn with regards to represen- sponding to citizens’ opinions and the other
tation. In this section, we review three ways corresponding to those of the representa-
in which congruence measures can differ, tives. Measures relying on distance may in-
before turning our attention to a comparison clude one or more dimensions. For instance,
of three congruence measures. in a one-to-one relationship, congruence be-
One of the first decisions to be made tween citizens and representatives could be
when measuring congruence is how to con- calculated as a difference between the ideo-
ceptualize the principal-agent relationship. logical position of the median/mean voter and
That is, whether to focus on one citizen/rep- the median/mean position of legislators (Bu-
resentative or on many citizens/representa- quet and Selios, 2017). In this case, only one
tives. As Golder and Stramski (2010: 91-92) dimension is included. Using this approach,
explain, scholars should choose between a researchers are able to measure the distance
one-to-one relationship (one citizen and one between several dimensions concurrently. Al-
representative), a many-to-one relationship ternatively, measures using the superimposi-
(many citizens and one representative; for tion of distribution preferences of citizens and
instance, to study representatives and their representatives estimate percentage points.
districts), or a many-to-many relationship These points are then used to calculate the
(many citizens and many representatives; for part that both citizens and representatives
instance, to study citizens and their legisla- share (or overlap) in order to determine the
tures or citizens and political parties). On the degree to which preferences are congruent
one hand, because one-to-one and many-to- or, alternatively, to calculate the differences
one measures are based on the position of a (or gap) between the distributions, that is, the
incongruent area.
single citizen and/or legislative, e.g., the me-
dian voter or the median legislator, these Just as there are debates as to how con-
measures are incapable of capturing the en- gruence should be conceptualized, scholars
tire distribution of preferences of citizens or have used very different types of data to
politicians7. Many-to-many measurements, measure congruence. Although establishing
on other hand, have the advantage of allow- citizens’ preferences is relatively straight for-
ing us to capture the entire distribution of ward, different strategies have been adopted
preferences for both citizens and politicians8. to establish the preferences of representa-
tives, parties, and/or governments. Studies
How the correspondence between prefer-
have used expert surveys (Huber and Pow-
ences is captured is determined by how the
ell, 1994; McElroy and Benoit, 2007), roll call
principal-agent relationship is conceptual-
votes (Weissberg, 1978), public opinion data
ized. The literature points to at least two pos-
(Freire, 2008; Golder and Stramski, 2010;
sibilities. The first one are measures based on
van der Meer et al., 2009), as well as data
the distance between two or more points,
from the Manifesto Project (Ezrow, 2007;
while the second one measures the superim-
Benoit and Laver, 2006; McDonald et al.,
2004). Despite these differences, amongst
those using survey data, consensus exists
7 For examples of studies using a one-to-one measure
see Buquet and Selios (2017) and Luna and Zechmeister
that we should attempt to use identical (or at
(2005). Examples of studies that adopt a many-to-one least, very similar) questions to capture the
measure are found in the works of Golder and Stramski citizen and representative preferences, and
(2010) and Otero Felipe and Rodríguez Zepeda (2010).
that these questions should come from sur-
8 For examples by scholars that measure a many-to-
many relationship, see Andeweg (2011), Lupu et al. veys that are completed during comparable
(2017), and Real-Dato (2018). time periods.
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-84Annabella España-Nájera and María del Mar Martínez Rosón 67
Decisions as to how congruence is opera- measures, the one by Golder and Stramski
tionalized, whether to measure congruence (2010) and the one that we developed, as-
or incongruence or the distance between two sess the correspondence between prefer-
or more points or the superimposition of dis- ences using distribution functions. The for-
tributions, should be carefully considered mer estimates the differences between
since these decisions may directly affect the preferences using commutative distribution
results (Achen, 1978; Golder and Stramski, functions, while the latter calculates the over-
2010). In the next section, we carefully con- lap of preferences using probability distribu-
sidered and compare three measures along tion functions. The last measure, presented
these lines. We look at how these decisions by Lupu et al. (2017), takes a distinct ap-
affect the results in order to offer some rec- proach. In order to determine the correspon-
ommendations for future studies. dence between citizen and legislator prefer-
ences, it considers the distance between
generalized histograms.
Comparing the three measures First, we examine Golder and Stramski’s
of congruence (2010) measure. Then, we compare it to our
measure and consider the similarities and the
The development of new measures is com-
differences between them. Finally, we pres-
mon in the congruence literature, both in
ent an analysis of Lupu et al.’s (2017) ap-
studies of older and newer democracies. We
proach and a discussion of how it contrasts
argue that while the conceptualization of new
to our own.
measures offers important contributions to
our understanding of congruence, without To compare the three measures, we use
careful evaluations and comparisons of these data from Central America. Although the
measures, it is difficult to acquire knowledge. countries of this region are not always includ-
Moreover, without this methodological dis- ed in congruence studies, their differences
cussion, it is increasingly difficult for scholars and similarities allow us to evaluate the three
to determine the implications of each mea- measures of congruence. The six cases in-
sure on their results and on our understand- clude polarized systems with large parties on
ing of congruence. To increase our under- both the left- and right-hand side of the ideo-
standing of the concept of congruence and logical spectrum, and systems in which
its relationship to representation in both catch-all-parties dominate. These variations
older and newer democracies, it is important allow us to examine how the measures han-
to consider the theoretical implication of dle the preferences of citizens or representa-
varying results. To do so, we must evaluate tives that are far from a normal distribution.
and compare measures. This is the goal of
this study. Cumulative Distribution Functions:
Golder and Stramski’s Measure
We selected three measures for compari-
son, based on their similarities and differ- In their discussion of different ways in which
ences. The three measures operationalize the agent-principal relationship may be op-
congruence as a many-to-many relationship. erationalized, Golder and Stramski’s (2010)
That is, they seek to capture the extent to argue that many-to-many measures are the
which the preferences of the legislature re- best option for analyzing congruence. They
flect the preferences of the citizens. The posit that if we are interested in substantive
measures differ in how they calculate the representation and its relationship with “lev-
correspondence between the preferences of els of democratic legitimacy and responsive-
citizens and their representatives. Two of the ness, satisfaction with democracy, political
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-8468 How to Measure Congruence: Comparing Three Measures in Central America
FIGURE 1. Measuring Congruence with CDF
100
90
80
70
60
50
40
30
20
10
0
1 2 3 4 5 6 7 8 9 10
Public Opinion Representatives
participation, or personal efficacy and trust in that it calculates the level of incongruence
the political process” (2010: 95), then the re- between two preference distributions.
lationship must be conceptualized as many- To illustrate what the operationalization of
to-many. To capture this relationship, the ideological congruence captures when using
authors develop a new measure of congru- cumulative distributions to measure the dif-
ence9. Here, our interest lies in evaluating ferences between the two groups, we pres-
their many-to-many measure. ent an example in Figure 1. It shows a large
Golder and Stramski’s (2010) measure of gap between the distribution lines. This
congruence captures the many-to-many rela- means that, in this case, there are low levels
tionship by comparing the cumulative distri- of ideological congruence between citizens
bution functions (CDFs) of citizens and their and their legislators.
representatives. For instance, with the left- While Golder and Stramski (2010) argu-
right ideological scale, widely used in the ment that we should seek to capture the whole
congruence literature, the area between the
distribution of preferences is a useful one, we
CDFs will be zero when the distributions of
propose two problems in their approach. First,
citizens’ and representatives’ preferences
because their formula is based on a cumula-
are identical. In these cases, it may be de-
tive distribution function, it does not treat each
clared that perfect ideological congruence
point on the scale equally. For instance, when
exists; that is, that the citizens’ preferences
calculating ideological congruence, by captur-
match those of their representatives. Alterna-
ing the cumulative difference between legisla-
tively, when large gaps exist between the two
tors and citizens at each point on the 1 to
distributions, the area between the CDFs will
10-point scale, the last category must total
grow. In these cases, there is less congru-
hundred percent and the cumulative differ-
ence. Another way of viewing this measure is
ence between citizens and representatives
must be zero. Depending on the distribution of
preferences, this can cause problems.
9 Due to data limitations, in their article the authors are
unable to apply their new measure and instead, they use This is illustrated in Figure 2, using the
a many-to-one measure, calculating the incongruence same data from the previous example (Figure
in the distribution of preferences between citizens (many)
and their placement of political parties (mean placement 1). Here, we can see that the gap between the
for each party, one). distribution of preferences for citizens and
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-84Annabella España-Nájera and María del Mar Martínez Rosón 69
FIGURE 2. Measuring Congruence with PDF
40
35
30
25
20
15
10
5
0
1 2 3 4 5 6 7 8 9 10
Public Opinon Representatives
representatives on the extreme right-hand Nicaragua, and therefore, the measure will
side of the scale (point ten) is large. However, not accurately calculate the congruence.
in Figure 1 we see that the cumulative differ- Lower congruence levels than those that ac-
ence between them is actually zero. The same tually exist will be measured when the distri-
problem arises in different points of the scale. bution of preferences leans to the left10.
For example, in point nine, the difference be- We argue that CDFs functions are not an
tween citizens and representatives is small ideal approach to operationalizing congru-
(2.8, see Figure 2), but the cumulative differ- ence based on these two measurement
ence between them is 25.9, slightly more than problems that we have identified. To correct
one quarter (see Figure 1). This means that for these two problems, a new measure of
the cumulative function impacts the measure congruence has been created. Our approach
of congruence differently, depending on the relies on Golder and Stramski’s (2010) for-
point of the scale. In other words, the formula mula, but with two slight modifications. In the
does not treat each point of the scale equally. next section, these changes are described
This may be especially problematic when and the two measures are directly compared
dealing with non-normal distributions. using an example.
The second limitation of calculating con-
gruence based on CDFs is derived from the Probability Distribution Functions:
first. When measuring ideological congru- España-Nájera and Martínez Rosón
ence, where the left is assigned to the lower Measure
ends of the scale, the difference between
citizens and legislators on the left-hand side As we have described, our measure of con-
of the ideological scale produce lower levels gruence also operationalizes the concept as
of congruence than the differences on the a many-to-many relationship to compare the
right-hand side. The consequences of this on
the results is that in systems having strong
leftist parties, we expect to see a significant 10 If the ideological scale assigned the lower end of the
scale to the right, it would be seen that the right-hand
distribution of preferences at those points of side of the ideological scale produces smaller levels of
the scale, as with the case of El Salvador or congruence than the differences on the left-hand side.
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-8470 How to Measure Congruence: Comparing Three Measures in Central America
preferences of citizens and their legislators in Figure 3. Points six to ten account for
using the entire distribution of preferences. 58.6% of the congruence score. This exam-
To avoid the two problems found in Golder ple illustrates that, although the two mea-
and Stramski’s (2010) formula, we first adopt sures are similar in some respects, the results
a simple, but effective change. Congruence are in fact quite different. The implications for
is calculated with the distribution function for congruence studies are significant, depend-
each point of the scale to compare the rela- ing on the real distribution of preferences and
tive frequencies for representatives and citi- how much they resembled, or not, a normal
zens. So, instead of using CDFs, we com- distribution. We argue that by treating each
pare probability distribution functions (PDFs). point on the scale equally, our measure does
By counting each point on the scale equally, a better job of capturing congruence without
we avoid an incorrect weight being placed on the need to worry about systems in which dis-
the leftist parties, ignoring potential differ- tributions of preferences may not be normal.
ences on the right-hand side of the scale. The second modification made to Golder
In Figure 3, we illustrate the implications of and Stramski’s measure (2010) consisted of
using the two different measures, one based adding up the common percentage, that is,
on CDFs and the other based on PDFs, using the overlap (or congruence), instead of the
the same data from our previous example (see difference (or incongruence) between repre-
Figures 1 and 2). For ease of comparison, we sentatives and citizens. This makes results
measure the gap (or incongruence) between interpretation much more intuitive (Andeweg,
representatives’ and citizens’ preferences for 2011:43)12. The following equation depicts
both the CDF and PDF calculations11. The re- the formula that we propose as a more ac-
sults appear in Table 1. It may be seen that the curate measure of congruence:
final value for congruence is 178.6 when using Congruence = ∑ min {fx (z) , fy (z)} (1)
a CDF measure. Alternatively, when using a
PDF measure, the value is 75.3. Figure 3 il- where fx and fy are the probability distribution
lustrates the difference between measures by functions of the citizen and representative
visually showing how each point on the scale preferences, respectively.
contributes to the final CDF or PDF score for The new measure ranges from zero, which
the same data. With the latter, we can see that signifies no overlap between the citizen and
the lack of congruence from points one to five elected official preferences, to one hundred,
is 47.2%. This is the percentage of the final implying perfect congruence between the two
congruence score that is contributed by these groups. In Figure 4, a visualization of the areas
points, one to five. Points six to ten contribute that are captured with this adapted measure
the remaining 52.8% of the total measure of of congruence is presented for all six Central
congruence for this example. American countries in 2004.
When using PDFs to measure congru-
ence, we see that the same section (points Multi-dimensional Measure:
one to five) account for 41.4% of the total Lupu, Selios and Warner’s Measure
score, a lower and more accurate percentage
Compared to the other two measures, Lupu
of the score when examining the distribution
et al. (2017) take a different approach to
operationalizing congruence. While these
11 In the measure that we are proposing, we calculate
the level of congruence, that is, the common area be-
tween citizens and legislators (as described below), as 12 Lupu and Warner (2017) also utilize this same ap-
opposed to incongruence. proach.
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-84Annabella España-Nájera and María del Mar Martínez Rosón 71
TABLE 1. Differences between CDFs and PDFs Congruence Measures
Differences between
Ideological Legislators’ Citizens’ Cumulative
Representatives and
Position Distribution Distribution Differences
Citizens
1 15 9.8 5.2 5.2
2 18.8 2.4 16.4 21.6
3 3.8 3.1 0.7 22.3
4 2.5 3 0.5 21.8
5 7.5 15.9 8.4 13.4
6 10 9.6 0.4 13.8
7 18.8 6.7 12.1 25.9
8 11.3 8.4 2.9 28.7
9 5 7.8 2.8 25.9
10 7.5 33.4 25.9 0.0
Total 100 100 75.3 178.6
Source: Developed by the authors.
authors are also interested in capturing a the authors compare the generalized histo-
many-to-many relationship, they do so with grams of representatives and citizens on a
a multidimensional measure that uses the number of issues, computing the distance
Earth Mover’s Distance (EMD). EMD, they between the functions with an algorithm.
explain, “computes the minimum ‘work’ re- They argue that this approach is preferable
quired to transform two distributions so that since it avoids many of the limitations arising
they are identical” (2017: 96). In other words, in previous measures of congruence.
FIGURE 3. Measuring Congruence: Alternative Methods
35
30 28,7
28.7
25,9
25.9 25,9
25.9 25,9
25.9
25 22,3
22.3
21,6
21.6 21,8
21.8
20
16,4
16.4
15 13,4
13.4 13,8
13.8
12,1
12.1
10 8,4
8.4
5,2 5.2
5.2 5,2
5 2,9
2.9 2,8
2.8
0,7
0.7 0,5
0.5 0,4
0.4 00
0
1 2 3 4 5 6 7 8 9 10
PDFs differences CDFs differences
Source: Developed by the authors.
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-8472 How to Measure Congruence: Comparing Three Measures in Central America FIGURE 4. Ideological Congruence in Central America, 2004 Source: Developed by the authors with data from PELA and LAPOP. According to Lupu and his colleagues, grams—eliminating the need for binning” the first of these limitations is that when (ibid.). While Lupu et al. (2017) make an im- overlap between distribution preferences is portant observation on the limits of using used to measure congruence, data is being the overlap between preferences, it is only binned into histograms, but the within-bin relevant for continuous variables, not ordi- variation is eliminated from the calculations nal ones. Ideology and policy positions, (2017: 96). The EMD, they suggest, is a which most scholars use to capture congru- more accurate measure since “it works with ence, are measured with limited scales (i.e. variable-size signatures—generalized histo- 1-10, 0-10 or 1-7), and these are always Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-84
Annabella España-Nájera and María del Mar Martínez Rosón 73
discrete variables. Since these scales are Moreover, it may be argued that the main
not continuous, Lupu et al.’s (2017) call for reason why researchers focus on the one-
caution does not apply to measures such as dimensional measures of congruence,
Golder and Stramski’s (2010) or our own, whether an ideology or an issue, is due to
when these measures are applied to ordinal data limitations. For Latin America, and other
variables. When this is the case, both mea- regions, few opportunities exist to study is-
sures do not work with histograms that sue congruence since there are few instanc-
group different values in the same “bin”, but es when both public opinion and elite sur-
instead, with bar charts in which each fre- veys include the same set of issue-related
quency represents one discrete position on questions13. For example, the PELA and
the scale. So, bar charts do not include the AmericasBarometer surveys, two of the most
within-bin variation that concerns Lupu et frequently used surveys for the region, in-
al. (2017). clude an identical series of issue-related
According to Lupu et al., the second questions from only 2010 to 2012. In this
problem with distribution measures is that series, legislators and citizens were asked
they are limited to examining a single dimen- –during a comparable period of time– what
sion, be it ideology (the most common) or an they consider to be the ideal role of the state
issue (2017: 96). In their approach, the EMD across various areas. Beyond this coordi-
can operationalize congruence using multi- nated effort, which lasted only two years, few
dimensions, that is, ideology and any num- questions in the two surveys are identical,
ber of issues. In their empirical work, for in- especially if we consider the possibility that
stance, seven variables are used to calculate not all matching questions may be relevant
congruence in Latin America. The authors for studying congruence. The limitations of
argue that this gives their approach an ad- the data become even more problematic if
vantage, since it makes it possible to exam- the goal is to study congruence cross-re-
ine numerous issues or policies using a sin- gionally.
gle summary statistic of congruence. The lack of data, therefore, makes the ap-
While we agree that there are advantages plication of a multi-dimensional measure ap-
to a multi-dimensional approach, the limita- proach, such as the one recommended by
tions of the overlap approach may in fact be Lupu et al. (2017), difficult to implement. In
less restricting than it has been suggested. their own work on Latin America, which like
First, while using the overlap between distri- ours, relies on the PELA and AmericasBa-
butions to measure congruence means using rometer data, the authors are constrained by
only one dimension at a time, scholars can the availability of matching questions. They
combine single results to create an aggre- are limited to using the left-right ideological
gate measure. We propose that this is a scale and the set of five questions on the role
strength, as opposed to a limitation. For in- of the state that most studies of congruence
stance, it allows us to analyze each dimen- in Latin America use. Lupu et al. (2017) de-
sion on its own, and to identify cases where part from other studies, including ours, in
significant levels of ideological congruence that they also include a question on same-
co-exist with low levels of issue congruence sex marriage in their analysis. Our last con-
(Freire and Belchior, 2013). Distribution ap- cern with Lupu et al.’s (2017) multi-dimen-
proaches also permit the identification of
which single issues have the strongest rela-
tionship to the left-right dimension and, in 13 Scholars can use different data sources, for instance
as Golder and Stramski (2010) did in their application.
general, allow for the testing of the measure- They used public opinion surveys to map party prefe-
ment’s validity (Belchior and Freire, 2013). rences, but these approaches have other challenges.
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-8474 How to Measure Congruence: Comparing Three Measures in Central America
sional measure is that it is not clear from their consistently have lower levels of congru-
discussion whether there are theoretical ence15. It is also seen that among these cas-
problems with creating a multi-dimensional es, it is not always the institutionalized party
measure of congruence that includes diver- systems (e.g., El Salvador) in which the high-
gent questions having potentially unequal er levels of congruence appear, a common
relevance to citizens and/or representatives finding in the literature (e.g., Luna and Zech-
(Thomassen, 1999). meister, 2005; Otero Felipe and Rodríguez
Zepeda, 2010)16.
A second noteworthy finding from Table 2
Applying the three measures is that, despite the within-region variation,
of congruence to Central there are high levels of congruence overall in
America Central America. Andeweg (2011), using the
same measure that we used, calculates lev-
In this section, we continue to compare and
els of ideological congruence for the Nether-
evaluate the three measures by applying
lands that are comparable to the values
them to the Central American countries. We
found in Central America. For instance, con-
first present our own results using the prob- gruence in the Netherlands ranges from 55.1
ability distribution functions (PDFs) to calcu- in 1977-79 to 89.3 in 2006 (2011:44-46).
late the overlap between preferences. We
In order to compare the results of our
then compare these results to those calcu-
PDF measure to Golder and Stramski’s
lated using the cumulative distribution func-
measure (2010), which uses CDFs, we cal-
tions (CDFs) approach developed by Golder
culate this measure for Central America. The
and Stramski (2010). The PDF and CDF mea-
goal of this comparison is to evaluate the
sures are compared using ideological con-
usefulness of each approach by applying
gruence. Finally, we compare our results for
them to the same set of cases. As described
issue congruence to Lupu et al.’s (2017) re-
sults from their multi-dimensional measure14.
Table 2 presents the values for ideological 15 In studies of congruence in older democracies, the
congruence using PDFs. The first important focus has traditionally been on measuring the relation-
observation is that high levels of within-re- ship between congruence and representation, by deter-
mining which factors improve congruence. See, for ex-
gion variation are found. The highest level of ample, Blais and Bodet (2006), Dalton (1985), Erzow
congruence across cases and time is found (2007), Powell (2009), and Golder and Stramski (2010).
One exception is Andeweg (2011) who explores the re-
for Guatemala in 2004 (84.3). On the con- lationship between congruence and democracy, finding
trary, the lowest level is found for Panama in an intriguing relationship in the Netherlands. There, it is
2010-12 (59.9). We find much higher congru- found that while congruence has increased substan-
tially since the 1970s, trust in democracy has decreased.
ence levels in Guatemala and Honduras than In Central America, no significant relationship between
in other countries. These results are surpris- congruence and citizens’ satisfaction with democracy,
ing, since these two countries, wich consis- support for democracy, or citizens’ preference for de-
mocracy is found (correlate analysis not reported here).
tently rank as problematic democracies, 16 Contrary to these studies, in our own preliminary
have the highest levels of congruence. Alter- analysis of the correlates of congruence, which is not
natively, Costa Rica, which has the highest reported here, we do not find a significant relationship
between party system institutionalization (PSI) and con-
quality of democracy, and El Salvador, a mid- gruence (Rodríguez Zepeda (2017), in his analysis of 15
range country in terms of democracy, tend to Latin American countries, also fails to find a relationship).
We suggest that future work on this topic should extend
beyond PSI, to also examine the relationship between
type of parties in the system (for instance, ideological
14 For a complete list of survey questions used in this versus catch-all parties) and congruence. For an exam-
study, see Appendix II. ple of such an analysis, see Belchior and Freire (2013).
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-84Annabella España-Nájera and María del Mar Martínez Rosón 75
TABLE 2. Ideological Congruence in Central America and Panama (España-Nájera and Martínez Rosón Measure)
2004-2014
Country 2004 2006-2008 2010-2012 2012-2014
(Mean)
Guatemala 84.3 74.2 67.8 - 75.4
Honduras 83.3 73.1 71.3 65.7 73.4
Panama 83.6 84.1 59.9 60.3 72
Nicaragua 74.2 73.3 65.8 - 71.1
Costa Rica 62.8 71.8 71.1 71.6 69.3
El Salvador 62.5 71.4 61.1 66.1 65.3
Regional Average 75.1 74.7 66.2 65.9 70.5
Note: Larger numbers represent higher congruence.
Source: Authors’ calculations using data from PELA and AmericasBarometer, various years.
previously, Golder and Stramski use CDFs treat all points of the scales equally, we are
to measure the lack of overlap between confident that our measure provides a more
preferences. This means that for this meas accurate view of congruence for Nicaragua
ure, higher values represent less congru- and similar cases.
ence (see Table 3). Using this approach, we Next, we examine Lupu et al. (2017) multi-
find the highest congruence value in Pana- dimensional measure by comparing it to our
ma, at 51.5 in 2004 and the lowest value in measure for issue congruence. To analyze
El Salvador, at 178.6, for the same year. Al- issue congruence, we rely on five questions
though it is difficult to make a direct com- that were included in both the PELA and
parison of the measures, we can see that on AmericasBarometer surveys between 2010
average Guatemala, Honduras and Panama and 2012. These questions asked respon-
rank the highest for both measures. dents for their opinion on the proper role of
The order of the other three countries the state for five issues: the ownership of
does vary between the two measures. This companies and industries, the well-being of
may not be surprising given that some of individuals, job creation, the reduction of in-
these cases have higher levels of polarization come inequality, and the health care sys-
in their distribution of preferences, as well as tem17. Lupu et al. rely on these same ques-
a substantial percentage of preferences on tions but since theirs is a multi-dimensional
the left-hand side of the scale. If we take a measure, their values also include ideology
closer look at the Nicaraguan case, for ex- and a question on same-sex marriage18.
ample, we find a significant distribution of
preferences on the left-hand side of the ideo-
logical scale (see Figure 4). Considering the 17For a list of survey questions used to calculate issue
previously mentioned methodological prob- congruence see Appendix II.
lems found with Golder and Stramski’s mea- 18 Lupu et al. (2017) include five of the six Central Amer-
sure (2010), it is not surprising that in cases ican countries that we use in our analysis. To maximize
comparability, in the following section we also remove
such as Nicaragua, the two measures do not Panama from our sample. Similarly, in their own work,
match. Given that their measure does not Lupu and his co-authors use the five issue positions that
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-8476 How to Measure Congruence: Comparing Three Measures in Central America
TABLE 3. Ideological Incongruence in Central America and Panama (Golder and Stramski’s Measure)
2004-2014
Country 2004 2006-2008 2010-2012 2012-2014
(Mean)
Guatemala 57.7 59.1 90 - 68.9
Honduras 71.4 91 79.7 76.7 79.7
Panama 51.5 136 65.8 97.5 87.7
Costa Rica 157.3 89.8 91.4 97.6 109
El Salvador 178.6 81.7 97.9 78.8 109.3
Nicaragua 141 109.9 138.4 - 129.8
Regional Average 109.6 94.6 93.9 87.7 96.4
Note: Larger numbers represent lower congruence.
Source: Authors’ calculations using data from PELA and AmericasBarometer, various years.
Table 4 presents the results for issue con- To compare our measure with that of
gruence using the PDF measure. We can see Lupu et al. (2017), we focus on ranking the
that on average, there are high levels of issue countries according to the values19. We com-
congruence across the countries of the re- pare the averages for the five public policies
gion. Guatemala at 79.2 and El Salvador at using our measure to Lupu et al. measure,
78.5 have the highest averages. In compari- using only the same five public policies. Ta-
son, Costa Rica at 69.7 and Panama at 66.8 ble 5 presents these results. As discussed
have the lowest averages for issue congru- above, Lupu et al. make a compelling case
ence. for the use of a multi-dimensional measure to
A closer look at Table 4 also reveals that study congruence. When comparing their re-
income inequality has the highest level of sult to our own, however, we find a very sim-
policy congruence for the region, with a value ilar rank order of the countries. Guatemala
of 84. That is, for the citizens of Central has the highest level of congruence, followed
America and their legislators, we find the by El Salvador and then Honduras. For the
highest level of agreement for the state’s role two countries with the lowest rankings, Cos-
in reducing income inequality. Table 4 also ta Rica and Nicaragua, the order does
shows that the issue having the lowest level change. With our calculation, Costa Rica has
of agreement is the responsibility of the state the lowest level of issue congruence while for
or the market to create jobs. It is worth noting Lupu et al., Nicaragua has the lowest value.
that some variation exists for this question. Although our comparison is limited by the
Almost no agreement on this issue is seen in small number of cases, the degree of simi-
Nicaragua (38.2), while relatively high levels larities in the results from the two approaches
are found in Honduras (75.3). is noteworthy. When comparing the measures,
we are comparing, as well as a question on same-sex 19 Ideally, we would also examine the correlation be-
marriage and ideology. For the comparison, the last two tween measures, however, the small N makes this is an
are left out. unreliable approach.
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-84Annabella España-Nájera and María del Mar Martínez Rosón 77
TABLE 4. Policy Congruence in Central America and Panama
Ownership
Responsible Responsible Reduction of Responsible
of Compa-
Country for Well- for Creating Income for Health Mean
nies and
Being Jobs Inequality Care
Industries
Guatemala 68 80.2 70.3 88.9 88.6 79.2
El Salvador 78.3 89.2 63.3 80.5 81.3 78.5
Honduras 61.9 76.4 75.3 85 70.5 73.2
Nicaragua 76.7 76.3 38.2 77.6 83.6 70.5
Costa Rica 61.7 76.9 45 89.6 75.2 69.7
Panama 54.5 51.2 68 82.1 78.2 66.8
Regional Average 66.9 75 60 84 79.6 71.7
Note: Larger numbers represent lower congruence.
Source: Authors’ calculations using data from PELA and AmericasBarometer, various years.
we find that not much information is lost concentration of preferences on that side of
when relying on the probability distribution the scale. We expect the same issues to arise
functions. Moreover, given that the data de- in other Latin American countries that share
mands are greater for the multi-dimensional a similar distribution of preferences. A num-
measure, in practical terms, more advantag- ber of similarities are found between our
es may be found when using the one-dimen- measure and the multi-dimensional measure
sional measure, as compared to the alterna- proposed by Lupu et al. (2017). In our com-
tive approach. parison of these two measures, the PDF ap-
When comparing our measure of congru- proach is comparable. Little information is
ence to Golder and Stramski’s (2010) for lost when adopting it and given the limita-
Central America, important differences are tions of the other approach, it seems more
found, especially in cases having a strong appropriate to use our measure to study con-
gruence.
leftist preference (i.e., Nicaragua). These may
be due to the two problems arising in Golder
and Stramski’s formula, as described previ-
ously. First, the cumulative function impacts
Conclusion and discussion
the measure of congruence differently at dif- For some time now, scholars of older democ-
ferent points on the scale and second, that racies have used congruence to study repre-
differences between citizens and legislators sentation (e.g., Achen, 1978; Budge and Mc-
on the left-hand side of the ideological scale Donald, 2007; Dalton, 1985; Golder and
produce smaller levels of congruence than Stramski, 2010; Huber and Powell, 1994;
the differences produced on the right-hand Lupu et al., 2017; Powell, 2009; Warwick,
side. Both of these problems directly affect 2016). Increasingly, those interested in newer
the application of the measurement in Cen- democracies have relied on this same ap-
tral America, since El Salvador and Nicara- proach as data from these countries be-
gua have high levels of polarization and comes increasingly available. We have seen,
strong leftist parties, thereby affecting the for instance, a significant growth in the use of
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-8478 How to Measure Congruence: Comparing Three Measures in Central America
TABLE 5. Comparing issue congruence in Central America across measures
Lupu et al.’s España-Nájera and Martínez Rosón
Measure Measure
Guatemala Guatemala
El Salvador El Salvador
Honduras Honduras
Costa Rica Nicaragua
Nicaragua Costa Rica
Source: Authors’ calculations and Lupu et al. (2017).
congruence to study representation in Latin important advantages. By treating each dis-
America (e.g. Luna and Zechmeister, 2005; tinct point on a scale independently, it avoids
Siavelis, 2009; Kitschelt et al., 2010; Otero the miscalculation of congruence in the case
Felipe and Rodríguez Zepeda, 2010; Buquet of distributions of preferences that do not
and Selios, 2017; Lupu et al., 2017; Boas and follow a normal distribution. In addition to
Smith, 2019). While we believe that these this substantive effect, our measure is more
analyses contribute to our understanding of intuitive for interpretation. As compared to
congruence and representation in the region, Lupu et al.’s (2017) approach, while clearly
our goal was to contribute to this growing lit- there are some advantages to their approach,
erature by comparing three congruence mea- data limitations create a serious challenge to
sures. We suggest that such comparisons the applicability of this more complex meas-
are necessary. Without them, it may be diffi- ure. Beyond this real concern, however, we
cult to understand variations in study results also argue that the similarities between our
and what they tell us about the region. distribution approach and the distance
Our comparison of the congruence meas- measure has the advantages of revealing
ures led us to propose our own approach. how the issue or ideological preferences of
While doing so clearly adds to a growing field citizens and representatives, mapped out for
of available congruence measures, we be- each dimension, make our measure more
lieve that ours offers a number of advantages appropriate in most congruence studies.
over others. We propose this measure as one As shown in our comparison with data
that other scholars can use to study congru- from Central America, the application of dif-
ence with a high degree of confidence20. We ferent measures of congruence have implica-
highlight these advantages in our compari- tions for results. While there were some sim-
son with two other commonly used meas- ilarities across measures, it is clear that the
ures. Specifically, when compared to Golder approach that is used affects the conclusions
and Stramski’s (2010) measure, ours has two reached on congruence. In the growing liter-
ature on congruence in Latin America, few
consistent findings exist on the relationship
20 We encourage those interested in studying congru-
between congruence and possible correlates
ence to carefully examine the distribution of preferences
in order to consider how the measure that they adopt of representation (for instance, see Luna and
may affect their results. Zechmeister, 2005; Kitschelt et al., 2010;
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-84Annabella España-Nájera and María del Mar Martínez Rosón 79
Otero Felipe and Rodríguez Zepeda, 2014; is revealing about representation and the
Lupu et al., 2017). Such diverging results are health and quality of a democracy (see, for
difficult to explain. The first step in doing so, example, Andeweg, 2011). In Latin America,
we suggest, is to take a closer look at the for instance, it appears that higher quality
approach used to measure congruence and democracies (for instance, Costa Rica, Chile,
how it impacts the results. and Uruguay) tend to underperform in terms
Although it is not the focus of our study, of congruence. We would argue that a broad-
our empirical results lead us to some poten- er research agenda which explores these
tial future research lines. Generally speaking, results, as well as the relationship between
the three measures presented high levels of congruence and other dimensions of repre-
ideological and issue congruence in Central sentation, would enrich the literature. A bet-
America, although with some interesting and ter understanding of how congruence, repre-
unexpected patterns of within-region varia- sentation and democracy interrelate,
tion. While we do not explore these surpris- especially as we extend our analysis to third
ing findings in this article, a further step wave and newer democracies, will help us
would be to examine the correlates of con- accumulate knowledge and a better under-
gruence in this region, as other scholars have standing of these cases.
done for different sets of Latin American
cases (Luna and Zechmeister, 2005; Kitschelt
et al., 2010; Otero Felipe and Rodríguez
Bibliography
Zepeda, 2014; Lupu et al., 2017). Thus far, Achen, Christopher H. (1978). “Measuring Represen-
little consensus exists as to the influence of tation”. American Journal of Political Science,
these factors, for instance, trust in institu- 22(3): 475–510.
tions, party system institutionalization, mea- AmericasBarometer by the Latin American Public
sures of responsiveness, or economic fac- Opinion Project (LAPOP). Nashville: Vanderbilt
tors, on congruence21. University. Available at: www.LapopSurveys.org.
While the literature on congruence has Andeweg, Rudy B. (2011). “Approaching Perfect
Policy Congruence: Measurement, Development,
tended to focus on methodological ques-
and Relevance for Political Representation”. In:
tions surrounding the concept, as is the case Rosema, Martin; Denters, Bas and Aarts, Kees
in this article, to further our understanding of (eds.). How Democracy Works: Political Repre-
congruence we suggest that future works sentation and Policy Congruence in Modern So-
expand beyond these discussions to include cieties. Amsterdam: Amsterdam University Press.
the careful theorizing as to what congruence Belchior, Ana Maria and Freire, André (2013). “Is
Party Type Relevant to an Explanation of Policy
Congruence? Catchall versus Ideological Parties
21 Our preliminary findings on the correlates of congru- in the Portuguese Case”. International Political
ence in Central America reveal certain contradictory Science Review, 34(3): 273–288.
results with those found mainly for South American coun-
Benoit, Kenneth and Laver, Michael (2006). Party Pol-
tries (not reported here). While the small N in this study
makes these observations more exploratory than final, icy in Modern Democracies. London: Routledge.
it is noteworthy that we have found no relationship be- Blais, André and Bodet, Marc A. (2006). “Does Pro-
tween congruence and trust in elections or political par-
portional Representation Foster Closer Congru-
ties, unlike Luna and Zechmeister (2005) who find a
strong negative relationship between perceptions of ence between Citizens and Policymakers?”.
fraud in the electoral system and congruence. We do Comparative Political Studies, 39(1): 1243–1262.
find significant negative relationships between trust in
Boas, Taylor C. and Smith, Amy E. (2019). “Looks
parliament, trust in government, and congruence. This
means that, at least in Central America, as congruence Like Me, Thinks Like Me? Descriptive Represen-
decreases, citizens’ trust in government and parliament tation and Opinion Congruence in Brazil”. Latin
increases. American Research Review, 54(3).
Reis. Rev.Esp.Investig.Sociol. ISSN-L: 0210-5233. Nº 169, January - March 2020, pp. 63-84You can also read
