Methodology Is Destiny: The Effect of Survey Prompts on Reported Levels of Giving and Volunteering

 
Methodology Is Destiny   Rooney et al.
                         10.1177/0899764004269312

                                                    Methodology Is Destiny:
                                                    The Effect of Survey Prompts on
                                                    Reported Levels of Giving and Volunteering

                                                    Patrick Rooney
                                                    Kathryn Steinberg
                                                    Indiana University and
                                                    Indiana University–Purdue University at Indianapolis
                                                    Paul G. Schervish
                                                    Boston College and Indiana University

                                                       This article extends earlier methodological tests of giving and volunteering in Indiana to
                                                       a large (N = 4,200) cross-sectional sample collected in the United States in the fall of
                                                       2001. The authors find that the results are consistent with those found in the earlier
                                                       analyses, namely, that longer, more detailed prompts led respondents to recall giving and
                                                       volunteering at higher incidence rates (proportion donating at all or volunteering at all)
                                                       and at higher levels (dollars given or hours volunteered) than when compared to survey
                                                       methodologies with fewer prompts.

                                                       Keywords: giving; volunteering; measurement; methodology

                                                    This article extends earlier work conducted by the Center on Philanthropy at
                                                    Indiana University on the implications of different survey methodologies for
                                                    levels of reported giving and volunteering. In our earlier research, we studied
                                                    the methodological implications of question design for a sample of house-
                                                    holds in Indiana (Rooney, Steinberg, & Schervish, 2001). Here we report the

                                                    Note: The authors would like to thank Frank Walker and Walker Information for their expertise
                                                    and assistance with the data gathering and Ronald Kessler and Elizabeth Martin for their help in
                                                    reflecting on directions for further methodological inquiry. We are especially grateful to William
                                                    Chin and Kiyoko Kamimura for their help with the econometrics. We also thank Jeff Small for his
                                                    research assistance. Finally, we would like to thank the referees for several good suggestions and
                                                    insights that significantly enhanced this article. Address correspondence to Patrick Rooney or
                                                    Kathryn Steinberg at Center on Philanthropy at Indiana University, 550 W. North Street, Suite 301,
                                                    Indianapolis, IN 46202; phone: 317-278-8909; fax: 317-684-8900; e-mail: rooney@iupui.edu,
                                                    ksteinbe@iupui.edu

                                                    Nonprofit and Voluntary Sector Quarterly, vol. 33, no. 4, December 2004 628-654
                                                    DOI: 10.1177/0899764004269312
                                                    © 2004 Association for Research on Nonprofit Organizations and Voluntary Action

                                                    628

                                                                                Downloaded from nvs.sagepub.com by guest on February 27, 2015
Methodology Is Destiny                                                                 629

findings from a parallel methodological study from a large (N = 4,200), cross-
sectional, national sample collected in the fall of 2001. The results from this
national study are consistent with those from the earlier Indiana study,
namely, that longer, more detailed prompts when compared to less detailed
prompts result in respondents reporting higher participation rates and greater
amounts of charitable giving and volunteering.
   Rooney et al. (2001) and Steinberg, Rooney, and Chin (2002) show that there
are marked differences in reported giving and volunteering among several
sets of randomly selected Indiana residents. Those who were asked the most
detailed sets of questions about their giving patterns reported that they gave,
on average, $1,000 more per household than respondents given a short survey,
after controlling for differences in age, race, income, education, tax status, and
marital status. Likewise, respondents who responded to longer, more detailed
questionnaires about volunteering reported close to 200 more volunteer hours
per year than those who responded to a short set of questions. These results
were consistent whether using ordinary least squares (OLS), Tobit, or two-
stage OLS (Heckman adjustment) regression techniques. Having established
the methodological principle with Indiana—that more detailed questioning
leads to higher reported incidence and amounts of giving and volunteering—
the next question was whether the principle would hold true for the new and
independent data composing a larger representative random sample of the
U.S. population.

                                    METHODOLOGY

   In testing whether the methodological principle held true for the U.S. sam-
ple, we compared and contrasted the findings from five different survey mod-
ules. The AREA module replicated questions on giving and volunteering from
Independent Sector (e.g., 1999) surveys that derive an overall level of giving
and volunteering for a respondent’s household by totaling the amounts the
respondent gives to particular areas of concern such as education, religion,
health care, and so forth. The METHOD module replicated the measurement
items from a survey conducted by Statistics Canada (Hall, McKeown, &
Roberts, 2001), which included several prompts based on “method” of fund-
raising contact such as direct mail, special events, and the like, or types of vol-
unteering such as raking leaves, serving food, and so on. The Panel Study of
Income Dynamics (PSID) survey module, designed by the Center on Philan-
thropy and the University of Michigan’s PSID staff (for a longitudinal study
called the Center on Philanthropy Panel Study or COPPS), contained several
prompts based on key areas of giving and a screening question for whether
someone volunteered followed by a question about the amount of volunteer-
ing in the human services subsector. The fourth question set was the SHORT
module, which aside from the standard demographic questions contained
only the following questions: “Did you give last year?” “If so, how much?”

                       Downloaded from nvs.sagepub.com by guest on February 27, 2015
630                                                                                   Rooney et al.

“Did you volunteer last year?” “If so, how much?” Finally, we fielded the
METHOD/AREA module (based in part on O’Neill & Roberts, 2000) that
obtains information on giving by prompting first by method of fund-raising
contact, then by subsector of giving for each possible combination. For in-
formation on volunteering, this module simply relied on the relevant ques-
tions from the PSID module.
   To improve the response and completion rate for the METHOD/AREA
module, which in some cases could take as long as 90 minutes, we offered 75%
of the respondents the inducement of a long-distance calling card. The
remaining 25% were not offered this inducement so that we could obtain
information about whether such documents significantly affected response
and completion rates. Our previous research (Rooney et al., 2001) indicated
that inducements were particularly effective at increasing response rates in
that module. To learn whether such inducements were equally needed or
helpful for the other four modules, we offered the same incentive to approxi-
mately 100 respondents taking those other modules. Table 1 summarizes the
modules by the types of giving and volunteering prompts, the number of
questions, and inducements. The appendix provides a more detailed descrip-
tion and examples of the prompts for each module.
   The telephone interviews were conducted by Walker Information, head-
quartered in Indianapolis. Callers used random digit dialing of households to
obtain separate samples of at least 800 respondents for each module. Any
given household participated in only one survey module.
   Approximately two thirds of the way through the data gathering, the Sep-
tember 11 attack on America occurred. We suspended the telephone inter-
views for a month immediately following the attacks. After 1 month, we
resumed the telephone calls but added six tragedy-related questions to the
beginning of each module asking about donations of money, blood, or other
necessities, and/or volunteer time to the September 11 victims and to related
causes. Because calling for AREA and SHORT modules had been completed
prior to September 11, we added 100 additional respondents to these modules
to ensure that each module had some respondents from both before and after
the tragedy. In a separate analysis, Steinberg and Rooney (2002) investigate
the correlates of the tragedy-related giving and whether there is any evidence
that September 11 giving affected giving to other causes.
   It should be noted that in each module, we asked respondents about house-
hold giving. With the exception of the survey of giving in Canada (Hall et al.,
2001), this is the unit of inquiry that virtually all researchers have used when
collecting data on giving. Household giving is of interest because it is more
comparable to data reported on tax returns and therefore could, in theory, be
verified with Internal Revenue Service (IRS) data. The household is also the
unit that these surveys use to gather information on income and other socio-
economic demographics.
   In calculating an annual amount of volunteer service, we asked respon-
dents to designate how many hours per month they volunteered, and multi-

                      Downloaded from nvs.sagepub.com by guest on February 27, 2015
Table 1. Modules—America Gives

                                                                                                Demographic    Tragedy      VERY
                                                                                                 Questions    Questions    SHORT            PSID              AREA             METHOD           METHOD/AREA

                                                                      Sample size: total          4,200        1,304         900            800                900                 800                800
                                                                        Pre-9/11                  2,896           —          800            512                800                 576                208
                                                                        Post-9/11                 1,304        1,304         100            288                100                 224                592
                                                                      No. of questions: total        16            6           6             46                121                 170                458
                                                                        Giving                       —            —            4             38                 42                  47                454
                                                                        Volunteering                 —            —            2              4                 79                 123                  4

                                                                      Types of prompts:
                                                                        Giving                       —        Yes/no,     Yes/no,     Prompt by sub-     Prompt by sub-      Prompt by method Prompt by method
                                                                                                                amount      amount      sector of con-     sector of con-      of contact; 6     of contact, then
                                                                                                                            (formal     tribution          tribution; 3        prompts for in-   by subsector; 7
                                                                                                                            only)       (formal only)      prompts for         formal giving     prompts for in-
                                                                                                                                                           informal giv-                         formal giving
                                                                                                                                                           ing, 1 for polit-
                                                                                                                                                           ical donations
                                                                        Volunteering                 —        Yes/no,     Yes/no,     Yes/no, then by    Prompt formal       Prompt formal and Yes/no, then by 1
                                                                                                                amount      amount      1 subsector        volunteering by     informal volun-   subsector
                                                                                                                            (formal     (formal            subsector, 1        teering by me-    (formal only)
                                                                                                                            only)       only)              general prompt      thod of contact

Downloaded from nvs.sagepub.com by guest on February 27, 2015
                                                                                                                                                           for informal
                                                                                                                                                           volunteering

                                                                      Inducements:
                                                                        None: total               3,202          —           805            698                805                 694                200
                                                                        Calling card: total         998          —            95            102                 95                 106                600
                                                                          Pre-9/11                  501          —            95             56                 95                  47                208
                                                                          Post-9/11                 497          —             0             46                  0                  59                392

                                                                      Note: PSID = Panel Study of Income Dynamics.

                                                                631
632                                                                                   Rooney et al.

plied that figure by 12 to annualize it. Because previous studies (Hall, 2001;
Independent Sector, 1999) found that the reliability of reporting on volunteer-
ing by others in the household is quite low, we inquired only about the volun-
teering done by the individual respondents.
   In this article, our focus is on the measurement of formal giving by house-
holds and volunteering by individuals. Formal giving is the contributions of
money and/or goods to organizations officially designated as charitable enti-
ties (i.e., gifts that would qualify as a tax-deductible charitable contribution).
Formal volunteering is the contribution of time to such charitable entities.
Three of our modules (AREA, METHOD, and METHOD/AREA) include
questions on informal giving—that is, contributions that are not collected
through a particular organization or group. In addition, the AREA and
METHOD modules query informal volunteering (volunteering on one’s own
rather than through an organization). We will not emphasize the results of
informal giving and volunteering in this article except to raise the method-
ological question of whether measurement of giving and volunteering should
focus only on formal gifts or also include informal donations of time, money,
and goods. We also measure differences across demographic groups: age,
income, race, educational attainment, gender, number of children, and num-
ber of children in college.
   In analyzing the findings, we first used one-way ANOVAs in order to test
whether the samples receiving each module were comparable demographi-
cally and in their reported giving and volunteering. We then carried out sev-
eral multivariate analyses (as suggested by O’Neill, 2001) to discern whether
differences among the modules in mean levels of giving and volunteering can
be explained simply by variations in sample characteristics or are due to the
characteristics of the different modules. To do so, we explain donations of time
and money in regression frameworks by including a set of dummy variables
for the five modules, along with the demographic variables (listed above). If
there are pure module effects, they will show up as significant coefficients for
the module dummy variables.
   Unfortunately, the error terms in these regressions do not obey the classical
assumptions that justify the exclusive use of OLS regressions. Donations can-
not be negative, so the error term has a truncated distribution. In addition, giv-
ing and volunteering data appear to have a nonnormal (heteroskedastic) error
structure (e.g., Bradley, Holden, & McClelland, 1999; Rooney et al., 2001;
Steinberg et al., 2002). Under these circumstances, OLS is biased and inconsis-
tent. Tobit regression models, on the other hand, do not generate negative pre-
dicted donations. Unfortunately, Tobits are not robust to nonnormal
(heteroskedastic) errors. An additional problem with Tobit models is that they
enforce a proportionality between a variable’s effect on the probability of giv-
ing and the size of the donation for those who give. Another approach, a two-
stage Heckman model, solves this latter problem but is not robust to non-
normality. Because there is no commonly accepted ideal remedy for all of
these problems, we conducted four different approaches (Tobit, Heckman

                      Downloaded from nvs.sagepub.com by guest on February 27, 2015
Methodology Is Destiny                                                                633

two-stage, OLS on the full sample, and OLS on positive donors only) in the
hope that a consistent picture would emerge. We also ran probits to assess the
effects of the various survey modules on the probability of donating or volun-
teering at all. To simplify the presentation here, we focus our discussion on the
Tobits and probits. A complete set of results, including the OLS and Heckman
two-stage models, is available from the authors upon request.
   Our regression models use levels rather than logarithms for the continuous
variables, for several reasons. First, it significantly simplifies the interpreta-
tion of the results, which is helpful for a broad interdisciplinary readership.
Second, it does not make any difference to the fit. Third, given the nature of
the type of analysis, the large number of zeros (i.e., nondonors and/or non-
volunteers) seems to lend itself to using level data with Tobit analyses. Finally,
many of our main variables of interest are dichotomous in nature, so a log-log
model would not have any impact on these results.
   The data set used in this article includes five modules, each with samples of
800 or 900 respondents. Each module includes questions about giving and
volunteering. Hence, we have 4,200 respondents from the five modules that
address both the time and treasure components of personal philanthropy.

                                           RESULTS

   Table 2 presents a comparison of demographic characteristics of each sam-
ple. We discuss briefly only those sample characteristics that are statistically
significantly different from the combined or total sample. Compared to the
total sample means and proportions, the SHORT module contained signifi-
cantly more Blacks (10.4% vs. 8.1% overall). The PSID had more couples
(64.3% vs. 61.5% overall). The AREA module had slightly fewer households
with incomes in excess of $120,000 (5.2% vs. 5.8%). The METHOD module had
fewer couples (58.5% vs. 61.5%), a larger percentage with incomes of $40,000
or less (47.9% vs. 42.4% overall), a smaller percentage of high incomes (5.1%
vs. 5.8%), and a concomitant smaller percentage who itemized their deduc-
tions (45.9% vs. 49.8%). The METHOD/AREA module did not differ from the
overall sample in any significant manner. In general, there are relatively few
significant differences between the samples and these differences are rela-
tively small, especially given the large sample sizes. Furthermore, these differ-
ences can be controlled for statistically in our regressions.
   An interesting difficulty that we ran into was that there were small but sta-
tistically significant differences across modules in the proportion of respon-
dents reporting their income (χ2 = 9.504, p = .05). People were least likely to
report their income in the AREA module (78.4%) and most likely to report
income in the AREA/METHOD module (83.9%) (see Table 3). This is consis-
tent with results from the earlier study done in Indiana (Rooney et al., 2001).
Regressions with a variable for “income missing” were run to see if this
affected our main results. This variable was small and insignificant and did

                      Downloaded from nvs.sagepub.com by guest on February 27, 2015
634                                                                                                   Rooney et al.

                   Table 2. Demographics of Total Sample by Module

                                  Total         VERY                                                     METHOD/
                                 Sample        SHORT           PSID        AREA            METHOD         AREA

Sample size                      4,200         900             800          900             800            800
Female (%)                          59.3        57.6            60           61              58.8           59.3
Couples (%)                         61.5        60.9            64.3*        61.4            58.5*          62.3
White (%)                           81.1        79.4            82.3         81              81.4           81.4
Black (%)                            8.1        10.4*            7.1          7.6             7.5            7.5
Hispanic (%)                         4.7         4.3             4.4          5.3             5.2            4.1
Asian (%)                            2.1         2               2            2.4             1.8            2
Other minority (%)                   4.1         3.8             4.1          3.7             4.2            5
Age:
   Mean                              45.3        45.23           45.34       45.43           45.47          44.78
   Median                            44          44              44          44              44             44
   Minimum                           18          18              18          18              18             18
   Maximum                           93          91              87          92              90             93
Education (%):
   ≤ High school diploma             30.6        32.1            31.7        29              31.2           28.9
   Some college                      36.8        34.9            37          38.5            36.9           36.8
   Bachelor’s degree                 18.2        18.7            17.4        16.4            18             20.4
   Graduate/professional
     school                          14.4        14.3            13.8        16.1            13.9           13.9
   Joint tests                                                                                **

Income (%):
  $0-$40,000                         42.4        39.6            39.2        42.6            47.9**         42.9
  $40,000-$80,000                    37.3        40.7            40.1        35.3            34.1           36.2
  $80,000+                           20.3        19.7            20.7        22.1            18             20.9
   Total reporting income            81.2**      82.4            80.4        78.4            81.1**         83.9
   Joint tests
% with income > $120,000              5.8         6.1             6.3         5.2**           5.1**          6.5
% who itemized deductions            49.8        52.3            50.1        52.5            45.9**         47.4
% of itemizers with donations        78.7        78.3            79.8        77.5            80.7           77.7
% with itemized gifts                34.3        36.1            35.1        35.7            31.9           32.3

Note: PSID = Panel Study of Income Dynamics.
*p < .10. **p < .05.

not have any material impact on the variables of interest, so it is not reported in
detail here (copies of these results are available from the authors).
   In this next subsection, we will present the results for giving; we will
address volunteering in a separate subsection. We start each of these two sub-
sections by delineating the major differences and similarities between the var-
ious sampling frames or modules. Then we present regression analyses,
which allow us to test for differences in the methodologies while controlling
for other factors such as income and educational attainment.

BIVARIATE RESULTS ON GIVING

  In Table 3, we compare the simple means of formal giving for each of the
modules, and we see some interesting similarities and differences. The mean

                           Downloaded from nvs.sagepub.com by guest on February 27, 2015
Table 3. ANOVA Tests on Total Formal Giving (Excluding Outliers > $100,000)

                                                                                                                                       No. of
                                                                                                                                       Giving            %
                                                                                                                        a, b
                                                                      Independent Variable    Mean         N     F (or t)       p     Questions Median Donors    Independent Variable    Mean        n        F      p

                                                                      Module                                                                                    Income
                                                                        VERY SHORT           1,044.50      871    10.514       .001       4     200     78.0      $0-$20K                 336.74     535   65.018 .000
                                                                        PSID                 1,407.23      800     0.346       .729      38     400     72.4      $20K-$40K               779.31     904
                                                                        AREA                 1,275.04      898    –0.804       .422      42     200     70.0      $40K-$60K             1,366.80     748
                                                                        METHOD               1,141.78      798    –2.235       .026      47     286     92.0      $60K-$80K             1,632.06     522
                                                                        METHOD/AREA          2,041.89      797     4.036       .000     454     610     94.7      $80K-$100K            2,347.57     295
                                                                        Total (pooled)       1,373.45    4,164    10.514       .000             305     81.1      $100K-$120K           2,421.25     151
                                                                      Gender                                                                                      $120K+                5,029.91     241
                                                                        Male                 1,581.17    1,693    10.225       .001                               Total                 1,480.95   3,396
                                                                        Female               1,231.14    2,471                                                  Income (3 groups)
                                                                        Total                1,373.45    4,164                                                    Low ($0-$40K)           614.77   1,439   139.957 .000
                                                                      Marital (2 groups)                                                                          Medium ($40K-$80K)    1,475.83   1,270
                                                                        Singles                940.63    1,594    41.300       .000                               High ($80K+)          3,304.73     687
                                                                        Couples              1,651.53    2,546                                                    Total                 1,480.95   3,396
                                                                        Total                1,377.82    4,140                                                  Who decides donations
                                                                      Race (4 groups)                                                                             Self                  1,229.53   2,486    6.292 .000
                                                                        Whites               1,476.12    3,320      4.503      .004                               Spouse/partner        1,279.33     375

Downloaded from nvs.sagepub.com by guest on February 27, 2015
                                                                        Blacks                 997.69      331                                                    Self and spouse       1,841.74   1,077
                                                                        Hispanics              947.36      191                                                    Other family member   1,020.91     132
                                                                        Other minority         944.72      255                                                    Other nonfamily       1,527.81      16
                                                                        Total                1,379.74    4,097                                                    Total                 1,389.90   4,086
                                                                      Race (2 groups)                                                                           City size
                                                                        Whites               1,476.12    3,320    13.474       .000                               Rural                 1,188.48   1,493    5.159 .006
                                                                        Other minority         967.93      777                                                    Suburban              1,592.55   1,626
                                                                        Total                1,379.74    4,097                                                    Urban                 1,404.50     932
                                                                                                                                                                  Total                 1,400.37   4,051

                                                                635
                                                                                                                                                                                                             (continued)
636
                                                                                                                                            Table 3. (continued)

                                                                                                                                         No. of
                                                                                                                                         Giving            %
                                                                                                                          a, b
                                                                      Independent Variable       Mean        N     F (or t)       p     Questions Median Donors     Independent Variable      Mean        n      F     p

                                                                      Education (2 groups)                                                                         Region
                                                                        High school or less      636.32    1,270    85.210       .000                                New England             1,396.29     206   0.710 .683
                                                                        Postsecondary          1,708.10    2,874                                                     Mid-Atlantic            1,188.15     523
                                                                        Total                  1,379.64    4,144                                                     So. Atlantic            1,262.64     664
                                                                      Education (4 groups)                                                                           East No. Central        1,384.59     749
                                                                        High school or less      636.32    1,270    61.869       .000                                East So. Central        1,312.22     269
                                                                        Some college           1,235.36    1,524                                                     West No. Central        1,317.96     403
                                                                        Bachelor’s             1,746.98      753                                                     West So. Central        1,554.57     407
                                                                        Graduate/              2,865.86      597                                                     Mountain                1,673.82     265
                                                                           professional                                                                              Pacific                 1,436.84     678
                                                                           school                                                                                    Total                   1,373.45   4,164
                                                                        Total                  1,379.64    4,144                                                   Flag 9/11
                                                                      Religious attendance                                                                           Before 9/11             1,268.91   2,866   8.343 .004
                                                                        Not at all               741.86      847    47.399       .000                                After 9/11              1,604.29   1,298
                                                                        A few times/year         705.73      854                                                     Total                   1,373.45   4,164
                                                                        1-2 times/month        1,003.37      601

Downloaded from nvs.sagepub.com by guest on February 27, 2015
                                                                        Every week             1,906.70    1,420
                                                                        > once a week          2,988.09      387
                                                                        Total                  1,386.71    4,109

                                                                      Note: PSID = Panel Study of Income Dynamics.
                                                                      a. One-way ANOVAs (F tests) were used to test the hypothesis that several means were equal.
                                                                      b. t tests were used to test the hypothesis that individual model means were different from the total (pooled) mean.
Methodology Is Destiny                                                                637

of the METHOD module ($1,142) and the METHOD/AREA mean of $2,042
were significantly higher than the total sample mean. The SHORT mean of
$1,045 was significantly lower than that of the total sample. In addition, there
is a very high significant correlation between the number of questions asked
about giving and the mean amount reported per module (r2 = .952, p = .013).
These results are prima facie evidence that the number of prompts and the
length of the survey do matter in collecting data about individual philan-
thropy. However, as we discussed above, there were some statistically signifi-
cant differences in incomes across the various subsamples when compared to
the overall sample. Hence, we will need to further refine this analysis with
multivariate analyses.
   Table 3 also shows the median amounts donated and the percentage who
reported making any donations for each module, along with mean amounts
for various demographic groups. There is a high correlation between the num-
ber of questions asked about giving and the median reported donation per
module (r2 = .893, p = .041). The METHOD/AREA median of $610 was the
highest, and this is the longest module. The correlation between number of
questions and percentage reporting donations is also positive but is not statis-
tically significant (r2 = .665, p = .221). Nevertheless, the longest module still
yielded the highest rate of giving among respondents: 95% made donations in
the METHOD/AREA module. These results are consistent with the work of
Wilhelm (2002), who found that higher percentages of people reported dona-
tions with methods surveys (e.g., method of fund-raising contact), but also
that methods surveys primarily helped people recall small donations.
   Based on these important indicators of giving (mean and median dollars
given and the percentage who gave anything), it does seem that a greater
number of detailed prompts do stimulate greater recall. Of course, there is a
danger that respondents report gifts that they did not actually give in an effort
to conform to a perceived set of positive social expectations and/or to please
or impress the interviewer. Although this is a problem in any type of survey
research, it may be exacerbated by repeated questions about giving, which
may convey the message to the respondent that it is expected or “normal” to
give. Similarly, respondents may be embarrassed or bored if they repeatedly
report no giving in surveys that ask about giving by many different areas
and/or many different methods of contacts. These concerns are not readily
resolved, but they do indicate the need to verify survey results through inde-
pendent data sources whenever feasible.
   However, Havens and Schervish’s (2001) diary study, which used weekly
prompting, found that 100% of their sample made donations to charities dur-
ing the course of a year. In addition, they tested for the effects of repeated
prompting by conducting the surveys during a 13th month. They found in
surveying respondents during two consecutive Januaries that reported giving
was not higher in the second January (the 13th month) than in the first January,
which supports the hypothesis that extensive and repeated questioning does
not inflate the responses of participants. They found that even after 4 or more

                      Downloaded from nvs.sagepub.com by guest on February 27, 2015
638                                                                                    Rooney et al.

months of weekly questioning, respondents were remembering and recount-
ing charitable giving they had been carrying out all along but had forgotten to
mention or thought was not charitable giving. They conclude that their close
look at charitable giving suggests, although does not prove, that having fewer
prompts is more likely to lead to an understatement rather than an exagger-
ation of charitable giving.
   It is worth noting that the use of incentives did not make a significant differ-
ence in the reported amount of total formal giving (F[1, 4169] = 0.012, p = .914).
This is consistent with the previous results in Indiana (Rooney et al., 2001),
which found that inducement amounts, although positively related to
response rate, did not make a significant difference in the reported amount of
donations to charitable organizations.

MULTIVARIATE RESULTS ON GIVING

   Now let us turn to the multivariate results in which we used OLS, Tobit,
Heckman two-stage, and probit regression models to examine the marginal
effects of the independent variables and to test the effects of differences in the
modules more formally. Table 4 presents the main results of our Tobit and
probit regression analyses (summarized in Table 5). These models assume that
the various survey modules do not alter the coefficients of the other variables
but that they shift the regression line up or down depending on the impact a
survey module has on reported giving. Rooney, Mesch, Chin, and Steinberg
(2003) found significant interaction effects between both race and gender and
the various survey methodologies, so this must be treated as a simplifying
assumption.1
   Looking at the demographic variables in the regression models (i.e., age,
race, gender, education, income, number of children, and number of children
in college), we find nothing surprising or out of line with other studies. For
example, the Tobit analysis indicates that respondents with some college edu-
cation (or more) report household donations of $611 more than respondents
with a high school education or less (the excluded category). Middle-income
households ($40,000-$80,000) gave $699 more, and high-income households
($80,000+) gave $1,781 more than low-income households, holding every-
thing else constant. The OLS and Heckman two-stage models produced simi-
lar results, so a consistent picture emerges across all models: Giving increases
significantly with increased income and education.
   Unlike the Indiana Gives study, America Gives found that race and age
were significant in explaining differences in giving across the various mod-
ules. These differences may be due to differences in sample sizes (885 vs.
4,200), as the coefficients in both studies are of similar magnitudes. The mar-
ginal effect of age is small and is weakly significant (at best), so we think it can
be safely ignored. The differences by race and gender require more attention
than can be given in this article and are the focus of another paper (Rooney
et al., 2003). America Gives also asked about the number of children in the

                       Downloaded from nvs.sagepub.com by guest on February 27, 2015
Methodology Is Destiny                                                                                        639

                                  Table 4. Regressions on Giving

                                            Tobit                  Probability            Probit        Probability
Variable                               Marginal Impact               Value            Marginal Impact     Value

Constant                               –1,374.31220                   .000***           –.489047          .034**
PSID                                      164.96357                   .213              –.182103          .026**
AREA                                      127.04729                   .315              –.094559          .224
METHOD                                    377.11714                   .003***            .787051          .000***
METHOD/AREA                               954.33797                   .000***            .954134          .000***
Age                                       –15.44716                   .299               .022430          .023**
    2
Age                                         0.40164                   .008***           –.00016           .112
Male                                       39.99896                   .631               .225343          .000***
White                                     288.40536                   .006***            .223871          .001***
Some college                              610.95103                   .000***            .430579          .000***
Medium income ($40K-$80K)                 698.85896                   .000***            .417821          .000***
High income ($80K+)                     1,781.34590                   .000***            .661742          .000***
No. children                              127.97065                   .000***            .045082          .084*
No. children in college                   352.04912                   .000***            .016450          .811
Post 9/11 interview                        72.17975                   .472               .185087          .015**
Sigma                                   2281.232107                   .000***

n                                             3,355                                           3,355
           2
Adjusted R                                                                                     .138
Log likelihood                             –27,698.2                                        –1,230.34

Note: PSID = Panel Study of Income Dynamics. Probit marginal impacts estimate the marginal
probability of donating due to changes in each variable. Tobit marginal impacts estimate the
change in donation amount due to changes in each variable. All coefficients for categorical vari-
ables are relative to the values of the excluded category for that variable (females, minorities, high
school or less, income $0-40K, pre-9/11 interview, VERY SHORT module). Statistical significance
is determined for the coefficients on the latent index for donations, with respect to the latent indi-
cator variable. Table with standard errors or t scores is available on request from the authors.
*p < .10. **p < .05. ***p < .01.

                                   Table 5. Summary for Giving

Variable                                      Expected Sign                            Tobit              Probit

PSID                                                 +                                 +                   –**
AREA                                                 +                                 +                   –
METHOD                                               +                                 +***                +***
METHOD/AREA                                          +                                 +***                +***
Age                                                 N/A                                –                   +**
    2
Age                                                 N/A                                +***                –
Male                                                 +                                 +                   –***
White                                                +                                 +***                +***
Some college                                         +                                 +***                +***
Medium income ($40K-$80K)                            +                                 +***                +***
High income ($80K+)                                  +                                 +***                +***
No. of children                                      –                                 +***                +*
No. of children in college                           –                                 +***                +
Post-9/11 interview                                  0                                 +                   +**

Note: PSID = Panel Study of Income Dynamics. N/A = not applicable; 0 = no effect or unknown
effect.
*p < .10. **p < .05. ***p < .01.

                            Downloaded from nvs.sagepub.com by guest on February 27, 2015
640                                                                                    Rooney et al.

family and the number of children in college (we assumed that these would
both reduce the capacity for giving, as they would increase the internal famil-
ial needs). Perhaps surprisingly, we found both the number of children and
the number of children in college to be positive and significant explanatory
variables.
   Of more interest to this methodological study is the analysis of the dummy
variables for different modules. A rank ordering of the module coefficients in
each regression model presents an interesting picture. In the Tobit model, the
coefficient for the other modules is positive relative to the SHORT module
(which was the excluded category). The AREA module is insignificantly dif-
ferent from the SHORT module, but the METHOD module attains signifi-
cance and predicts $377 more in donations relative to the SHORT module,
holding everything else constant. The coefficients on the PSID module suggest
that these households donated approximately $165 more, on average, than
those answering the SHORT module, holding everything else constant, but
this difference was not significant. Of even more interest is the fact that the
METHOD/AREA module, which is the longest one, leads to reported giving
of $954 more in the Tobit relative to the SHORT module (again holding every-
thing else constant). (The results from the OLS and Heckman two-stage mod-
els were generally consistent with the Tobit results.) This is a crucially impor-
tant result, because it is almost identical to the main result of the Indiana Gives
survey, which had a much smaller sample. Because the results from two very
different samples at two different times were nearly identical, this is a strong
indicator of the validity of the methodological result. These results suggest
that the length and the number of prompts in the surveys do matter in terms of
gathering data on dollars donated by households.
   To examine whether various modules have an impact on the probability of
whether an individual or household donated at all, we look at the probit re-
sults. These results indicate that holding everything else constant, the coeffi-
cients for all modules behave the way we might expect: Longer, more detailed
prompts also increase the probability of recalling having made any donations
in the prior year. Controlling for all of the relevant demographic characteris-
tics, the probability of reporting any donations is highest with the METHOD/
AREA module (95% more likely than those replying to the SHORT module),
which had the most prompts for formal giving. Similarly, respondents to the
METHOD module, which had the second most giving prompts, were 79%
more likely to report making any donations than those replying to the SHORT
module. The only surprises in the probit analysis are that the AREA module is
not significantly different from the SHORT module and that respondents to
the PSID module were 18% less likely to recall making any donations than
were those with the SHORT—despite the fact that the PSID had many more
prompts than does the SHORT module (38 vs. 4 prompts). Taken together, the
results from the probit and Tobit once again point to the conclusion that sur-
vey length and number of detailed prompts do matter in estimating house-
hold giving.

                       Downloaded from nvs.sagepub.com by guest on February 27, 2015
Methodology Is Destiny                                                                641

BIVARIATE RESULTS ON VOLUNTEERING

   In Table 6, we compare the means, medians, and percentages of formal vol-
unteering for each of the modules and the means for various demographic
groups. (Note that this table represents the entire sample, excluding outliers of
more than 2,000 hours per year [i.e., full-time volunteers]. We dropped the
outliers because they were not evenly distributed across the modules. There-
fore, a random event might skew the results of our testing of prompting and
recall.) The means for the SHORT, PSID, and METHOD/AREA modules were
significantly lower, and the mean for the METHOD module was significantly
higher than the total sample mean. The means for the three shortest volunteer-
ing modules were clustered between 55 and 67 hours per year (PSID = 65 vol-
unteer hours per year; SHORT = 55 hours, and the METHOD/AREA = 67
hours; all three had median values of 0). The fact that the PSID and the
METHOD/AREA modules used the exact same questions for volunteering
and had almost identical mean values reinforces the notion that methodology
matters. This suggests that the results are not due to sample sizes, or to the
experience of callers, but that identical questions themselves produced very
similar results across two large samples.
   It is worth noting that the highest mean value (164) comes from the
METHOD module, which includes the highest number of questions measur-
ing volunteering. The module with the second most prompts for volunteering
was the AREA module, which had the second highest mean value (98 hours
per year). There is a very high and significant correlation between the number
of questions asked about volunteering and the mean reported volunteer hours
per module (r2 = .96, p = .01). In addition, the percentage of respondents report-
ing that they do any volunteer work is much higher for the longest module
than the shorter ones. Collectively, these results offer prima facie substantia-
tion to our thesis that the number of prompts related to volunteering does
matter in collecting data about individual philanthropy.
   As with reported giving, the use of incentives did not make a significant
difference in the reported amount of total formal volunteering. This is consis-
tent with the previous results in Indiana (Steinberg et al., 2002), which found
that inducement amounts did not make a significant difference in the amount
of formal volunteering.

MULTIVARIATE RESULTS ON VOLUNTEERING

   Next, we examine the multivariate results in which we used Tobit and
probit regression models to test the marginal effects of the independent vari-
ables and to assess the impact of differences in the modules more formally.2
(The probits determine the marginal probability of an individual volunteering
at all given various characteristics.) Table 7 presents the main results of our
regression analyses, summarized in Table 8. (Results from OLS and Heckman
two-stage models are also available from the authors upon request.) Begin-

                      Downloaded from nvs.sagepub.com by guest on February 27, 2015
642
                                                                                                      Table 6. ANOVA Tests on Formal Volunteering (Excluding Outliers > 2,000 Hours)

                                                                                                                                    No. of
                                                                                                                                 Volunteerng          %
                                                                                                                       a, b
                                                                      Independent Variable   Mean         N     F (or t)       p Questions Median Volunteers     Independent Variable    Mean      n      F       p

                                                                      Module                                                                                   Income
                                                                        VERY SHORT            54.54       893    –6.588       .000     2      0    42.4          $0-$20K                 56.70     553   3.714   .001
                                                                        PSID                  64.95       800    –4.325       .000     4      0    45.6          $20K-$40K               96.00     902
                                                                        AREA                  97.95       892     1.249       .212    79      0    39.4          $40K-$60K               84.18     741
                                                                        METHOD               163.85       780     6.014       .000   123      6    54.5          $60K-$80K               89.92     516
                                                                        METHOD/AREA           66.63       800    –3.890       .000     4      0    41.6          $80K-$100K             118.01     293
                                                                        Total (pooled)        88.63     4,165    33.058       .000            0    44.6          $100K-$120K            104.68     151
                                                                      Gender                                                                                     $120K+                 118.98     245
                                                                        Male                  82.89     1,697     1.902       .168                               Total                   90.25   3,381
                                                                        Female                92.58     2,468                                                  Income (3 groups)
                                                                        Total                 88.63     4,165                                                    Low ($0-$40K)           81.40   1,435   5.643   .004
                                                                      Marital (2 groups)                                                                         Medium ($40K-$80K)      86.54   1,257
                                                                        Singles               82.45     1,595     2.065       .151                               High ($80K+)           115.43     689
                                                                        Couples               92.69     2,544                                                    Total                   90.25   3,381
                                                                        Total                 88.74     4,139                                                  Who decides
                                                                      Race (4 groups)                                                                            Self                   N/A      N/A     N/A     N/A

Downloaded from nvs.sagepub.com by guest on February 27, 2015
                                                                        Whites                90.67     3,318     0.385       .764                               Spouse/partner         N/A      N/A
                                                                        Blacks                81.75       330                                                    Self and spouse        N/A      N/A
                                                                        Hispanics             88.62       192                                                    Other family member    N/A      N/A
                                                                        Other minority        77.90       254                                                    Other nonfamily        N/A      N/A
                                                                        Total                 89.06     4,094                                                    Total                  N/A      N/A
                                                                      Race (2 groups)                                                                          City size
                                                                        Whites                90.67     3,318     0.902       .342                               Rural                   82.05   1,496   2.222   .109
                                                                        Other minority        82.19       776                                                    Suburban                89.71   1,626
                                                                        Total                 89.06     4,094                                                    Urban                  101.74     930
                                                                                                                                                                 Total                   89.64   4,052
Education (2 groups)                                                                          Region
                                                                        High school or less      59.04     1,272   33.113    .000                                     New England             81.13     205   1.490   .155
                                                                        Postsecondary           102.13     2,872                                                      Mid-Atlantic            85.61     518
                                                                        Total                    88.90     4,144                                                      So. Atlantic            75.33     668
                                                                      Education (4 groups)                                                                            East No. Central        80.59     748
                                                                        High school or less      59.04     1,272   21.066    .000                                     East So. Central        79.50     271
                                                                        Some college             81.41     1,524                                                      West No. Central        97.15     402
                                                                        Bachelor’s              118.99       751                                                      Mountain                87.15     264
                                                                        Graduate/               133.79       597                                                      West So. Central        93.82     409
                                                                           professional                                                                               Pacific                111.15     680
                                                                           school                                                                                     Total                   88.63   4,165
                                                                        Total                    88.90     4,144                                                    Flag 9/11
                                                                      Religious attendance                                                                            Before 9/11            86.27    2,871 15.819    .000
                                                                        Not at all               48.79       844   21.830    .000                                     After 9/11             93.87    1,294
                                                                        A few times/year         68.91       854                                                      Total                  88.63    4,165
                                                                        1-2 times/month          74.79       603
                                                                        Every week              111.76     1,419
                                                                        > once a week           153.84       389
                                                                        Total                    88.48     4,109

                                                                      Note: PSID = Panel Study of Income Dynamics. N/A = not applicable.
                                                                      a. One-way ANOVAs (F tests) were used to test the hypothesis that several means were equal.
                                                                      b. t tests were used to test the hypothesis that individual model means were different from the total (pooled) mean.

Downloaded from nvs.sagepub.com by guest on February 27, 2015
                                                                643
644                                                                                                 Rooney et al.

                             Table 7. Regressions on Volunteering

                                            Tobit                  Probability            Probit        Probability
Variable                               Marginal Impact               Value            Marginal Impact     Value

PSID                                        19.05337                  .318               .072925          .305
AREA                                        43.17837                  .019**            –.066648          .331
METHOD                                     148.56956                  .000***            .302158          .000***
METHOD/AREA                                –26.23220                  .219              –.148902          .057*
Age                                          3.07166                  .164               .00771           .347
    2
Age                                         –0.01714                  .449              –.00004           .590
Male                                       –26.65009                  .028**            –.150414          .001***
Married                                     –5.479759                 .686               .00930           .855
White                                       27.2916                   .076*              .142134          .013**
Some college                               101.7348                   .000***            .433981          .000***
Medium income ($40K-$80K)                   23.39421                  .112               .169616          .002***
High income ($80K+)                         62.79343                  .000***            .309819          .000***
No. of children                             26.03154                  .000***            .105829          .000***
No. of children in college                   4.95296                  .716               .030234          .588
Post-9/11 interview                         53.02648                  .000***            .198842          .000***
Sigma                                      295.6933                   .000***

n                                             3,333                                         3,333
           2
Adjusted R                                                                                  .068
Log likelihood                             –1,2042.2

Note: PSID = Panel Study of Income Dynamics. Probit marginal impacts estimate the marginal
probability of donating due to changes in each variable. Tobit marginal impacts estimate the
change in donation amount due to changes in each variable. All coefficients for categorical vari-
ables are relative to the values of the excluded category for that variable (females, minorities, high
school or less, income $0-40K, pre-9/11 interview, VERY SHORT module). Statistical significance
is determined for the coefficients on the latent index for donations, with respect to the latent indi-
cator variable. Table with standard errors or t scores is available on request from the authors.
*p < .10. **p < .05. ***p < .01.

                               Table 8. Summary for Volunteering

Variable                                      Expected Sign                            Tobit              Probit

PSID                                                 0                                 +                   +
AREA                                                 +                                 +**                 –
METHOD                                               +                                 +***                +***
METHOD/AREA                                          0                                 –                   –*
Age                                                 N/A                                +                   +
    2
Age                                                 N/A                                –                   –
Male                                                 –                                 –**                 –***
Married                                              0                                 –                   +
White                                                0                                 +*                  +**
Some college                                         +                                 +***                +***
Medium income ($40K-$80K)                            +                                 +                   +***
High income ($80K+)                                  +                                 +***                +***
No. children                                         +                                 +***                +***
No. children in college                              –                                 +                   +
Post-9/11 interview                                  0                                 +***                +***

Note: PSID = Panel Study of Income Dynamics; N/A= not applicable; 0 = no effect or unknown effect.
*p < .10. **p < .05. ***p < .01.

                            Downloaded from nvs.sagepub.com by guest on February 27, 2015
Methodology Is Destiny                                                                 645

ning with the probits, we find that age and marital status do not seem to mat-
ter; men are 15% less likely to volunteer, Whites are 14% more likely to volun-
teer than non-Whites, those with some college education (or more) are 43%
more likely to volunteer than those with a high school education or less, and
there is a large increase in the probability of volunteering at all as one moves
up the income ladder.
   As would be expected, because of the parental involvement with activities
associated with school-aged children (e.g., coaching Little League, assisting in
the classrooms, etc.), having children is associated with an 11% increase in the
probability of volunteering, but having children in college has no significant
effect. Having children in college may necessitate working more hours at a
current job or taking a second job—either of which would reduce time avail-
able for volunteering. The gender and race issues will be addressed in other
work, as will the post-9/11 effect, but we should mention that there is a 20%
increase in the probability of having volunteered in the past year among those
interviewed in the aftermath of the attack on America.
   Looking at the differences in the number of volunteering hours reported
using Tobit, we find that males volunteer almost 27 fewer hours than women,
Whites volunteer just over 27 hours more than non-Whites (but this result
only approaches significance with p = .076), some college is associated with an
increase of almost 102 hours relative to those with a high school degree (or
less), high-income earners volunteer almost 63 hours more than low-income
earners, and those with more children volunteer 26 hours more. Those inter-
viewed following the attack on America reported volunteering 53 hours more
than those before the attack, holding everything else constant.
   With respect to the impact of the differences in survey methodologies on
hours volunteered, the Tobit results are very consistent: When compared to
the SHORT module (the omitted variable), the METHOD module, which has
the most prompts about formal volunteering, is associated with 149 (Tobit)
more volunteer hours in the previous year. The AREA module, which has the
second-highest number of prompts for formal volunteering, also has a large
and positive effect on the number of hours volunteered (43 using Tobit). Not
surprisingly, the PSID and the METHOD/AREA, which have only four
prompts compared to the two in the SHORT module, do not yield statis-
tically significant differences in the number of hours volunteered. (OLS and
Heckman two-stage results were generally consistent with the Tobits.)

THE RELATIONSHIP BETWEEN VOLUNTEERING AND GIVING

   In addition to looking at giving or volunteering alone, we also examined
the relationship between giving and volunteering. Previous research has
found that volunteers are more likely than nonvolunteers to make charitable
contributions (Hall et al., 2001; Independent Sector, 1999; Jalandoni & Hume,
2001; Steinberg et al., 2002). Likewise in this study, chi-square tests for differ-
ences in the proportion between donors and nondonors among volunteers

                       Downloaded from nvs.sagepub.com by guest on February 27, 2015
646                                                                                                   Rooney et al.

          Table 9. Relationship Between Giving and Volunteering Across Modules

                              Total           VERY                                                         METHOD/
                             Sample          SHORT              PSID            AREA          METHOD        AREA

n                         4,200             900              800               900            800          800
No. of questions                             21                62              138            186          473
Volunteers (%)
  No donations               11.1            10.1              17.8             21.2            4.4          2.7
  Donations                  88.9            89.9              82.2             78.8           95.6         97.3
Nonvolunteers (%)
  No donations               25.2            30.9              35.9             35.6           12.4          7.1
  Donations                  74.8            69.1              64.1             64.4           87.6         92.9
 2
χ                           133.981***       52.724***         32.355***        21.025***      17.271***     7.441**
Pearson correlations
  No. of questions
     and % of
     volunteers
     who donated                   .615
  No. of questions
     and % of non-
     volunteers
     who donated                   .824*

*p < .10. **p < .05. ***p < .01.

and nonvolunteers found that volunteers were much more likely to be donors
than nonvolunteers in the total sample, as well as in each of the modules (see
Table 9). In fact, across all of the modules, nonvolunteers were between 2 and 3
times more likely to be nondonors than were those who had volunteered.

                                             CONCLUSIONS

   Our results have shown that whether using simple means or multivariate
analyses, the longer and more detailed the module, the more likely a house-
hold was to report making a charitable contribution of time or treasure and the
higher the average level of its reported giving and volunteering. These results
persist even after controlling for differences in age, educational attainment,
income, household status, race, number of children, number of children in col-
lege, and gender. We view these results, coupled with similar findings from
our Indiana Gives comparison of survey methodology (Rooney et al., 2001) as
further confirmation of Schervish and Havens’s (1998)

     methodological proposition that the more carefully a survey samples the
     full range of households by income, interviews the knowledgeable deci-
     sion makers in a household, furthers respondent recall, contracts trained

                              Downloaded from nvs.sagepub.com by guest on February 27, 2015
Methodology Is Destiny                                                                647

  interviewers who ask complex questions patiently and accurately, and
  otherwise use rigorous field practices, the greater the amount of re-
  ported charitable giving. (p. 241)

   As we look to the future, one additional research question is to investigate
what kinds of giving people tend to remember and forget without being
prompted and whether the short or medium-length surveys capture most of
the larger donations, so that users of those methodologies could be more con-
fident about what they were likely to capture and to miss.
   There is also need for further research on other important methodological
questions. For example, within households, what are the key factors that de-
termine giving within a joint household: Are decisions usually made jointly?
If not, who decides? Does the gender of the decision maker affect where the
money goes once there is a decision to donate? Given that we cannot match
survey samples with IRS records, is there a way that we can verify or validate
our survey research? What, if any, are the true effects of race and gender when
a complex array of other background characteristics, motivational determi-
nants, and decision-making dynamics are taken into account?
   This discussion on survey methods raises a more general methodological
question that scholars need to test in the field: How is the measurement error
affected when surveys ask about individual versus household philanthropy?
The key considerations are greater knowledge and recall of one’s own philan-
thropy, but many respondents may recall how much they listed as itemized
deductions, which would typically be a joint total. (Even married persons fil-
ing separately would be likely to pool their contributions and list them under
one person, as this would minimize their combined tax burden.)
   The innovations in survey methodology we have introduced for the study
of giving and volunteering suggest an array of additional research issues to be
pursued in the future. Among the issues that methodologists have addressed
in regard to other topics, but have not been taken up in regard to giving and
volunteering, include the impact of extended prompts, the kind of interview
method in which the cues are embedded, and the potential for extended
prompting to induce socially desirable responses. Much other research con-
curs with our findings that using prompts (or cues) and cueing mechanisms to
aid information retrieval does, in fact, improve respondent recall and reduces
response bias and measurement error (Belli, 1998; DeMaio & Rothgeb, 1996;
Moore, Stinson, & Welniak, 1999; Shum & Rips, 1999). This literature provides
empirical support for the fact that improved cueing produces more valid re-
sponses. At the same time, it suggests additional cueing techniques that merit
testing in regard to giving and volunteering.
   Recently, Belli, Shay, and Stafford (2001) have demonstrated the potential
advantages of event-history calendars with a flexible mode of interviewing
that uses cues as part of a narrative form of inquiry rather than simply as addi-
tional close-ended questions. Introducing and evaluating different kinds of

                      Downloaded from nvs.sagepub.com by guest on February 27, 2015
648                                                                                   Rooney et al.

cues and the event-history approach to assist recall needs to be complemented
by additional work on the long-recognized problem of social desirability
(DeMaio, 1984). This is especially important for longer surveys such as ours in
which the use of prompts requires most respondents to frequently, if not re-
peatedly, answer in the negative about whether and how much they give and
volunteer. Although there is some evidence that survey respondents tend to
tell the “truth” when they know the answers, there is always the possibility of
some memory error (Bradburn & Sudman, 1988). Furthermore, Wentland and
Smith (1993) report that based on a meta-analysis of survey responses to ques-
tions for which there are actual numbers available for comparisons to the sur-
vey replies, the group averages tend to be fairly accurate. They report that
although individual replies may underestimate or overestimate the actual
results, these differences tend to net out in the group averages. They also sug-
gest that inaccuracies due to socially desirable responses may not be a prob-
lem in this type of research.
   Ideally, we would be able to match survey responses with IRS records for
those who itemize their charitable deductions and compare. However, even
this would not prove the accuracy of the survey replies because for different
reasons, some tax filers may overstate their deductions and others may under-
state them. As an experiment, we tried this approach with a small sample in a
pretest for an upcoming study. Unfortunately, we found very few who both
had their IRS 1040 Schedule A’s available and were willing to retrieve them
and share the results with us. Perhaps others might be able to develop a
research design that might test this method with greater success.
   Finally, all the issues and directions for research we have addressed in this
article need to be taken up as well in regard to the panoply of informal giv-
ing and volunteering. It is clear from closer examination of remittances by
immigrants and the daily practices of all groups, regardless of ethnicity or
income, there is a dramatic amount of financial and in-kind assistance of indi-
viduals both in and outside of the immediate family. Equally substantial is the
generous network of unpaid assistance that people carry out amidst their
daily round to meet the needs of others. The same methodological princi-
ple that calls for careful and detailed investigation of formal giving and vol-
unteering applies to these realms of informal caring behavior equally, if not
more so.
   Our results, in conjunction with our findings from the smaller Indiana
Gives study, strongly support the principle that when it comes to estimating
charitable giving and/or volunteering, methodology is destiny. The more
individuals are prompted in detail, the more likely they are to recall having
given any gifts or to have volunteered at all, and the more they report their
total giving and their hours of volunteer time to be. Although there is no
incontrovertible evidence to establish this conclusion, the growing prepon-
derance of evidence makes it thoroughly persuasive.

                      Downloaded from nvs.sagepub.com by guest on February 27, 2015
Methodology Is Destiny                                                                   649

                                   Appendix
                           Examples of Module Prompts

VERY SHORT Module:

   Giving (yes/no, amount):

  In the past 12 months, have you made any financial contributions to a charitable or
nonprofit organization or group?

   (If yes): How much?

   Volunteering: (yes/no, amount):

   In the past 12 months, have you done any volunteering for an organization or
group?

   (If yes): How many hours total?

PSID Module:

   Giving (Prompt by subsector of contribution, formal only):

   During the year 2000 did you or anyone in your household make donations

   • For religious purposes or spiritual development;
   • To an organization that served a combination of purposes (United Way, United
     Jewish Appeal, Catholic Charities, local community foundation);
   • To organizations that help people in need of food, shelter, or other basic necessities;
   • To health care or medical research organizations;
   • Toward educational purposes (not including tuition);
   • To other organizations: youth and family services; arts, culture, and ethnic aware-
     ness; improving neighborhoods or communities; environment; international aid
     or world peace?

   (If yes to any of the above): asked dollar amount first for each, provided categories if
respondent did not know or refused.

   Volunteering (yes/no, then prompt by one subsector, formal only):

   During the year 2000, did you yourself do any volunteer work through organiza-
tions that totaled 10 hours or more?

   (If yes): asked total hours first, provided categories if respondent did not know or
refused.
   (If yes): How many hours of that volunteer time was spent helping people in need of
food, shelter, or other basic necessities?

                         Downloaded from nvs.sagepub.com by guest on February 27, 2015
650                                                                                      Rooney et al.

AREA Module:

   Giving (Prompt by subsector of contribution, informal and political donations):

  In which, if any, of the following fields have you and the members of your family or
household contributed some money or other property in the past year?

   •   Health
   •   Education
   •   Religious organizations
   •   Human services
   •   Etc.

   (If yes to any of the above): asked dollar amount for each.

   During the past 12 months, did you or the members of your family or household
give money, food, or clothing to the homeless or street people, a needy neighbor, or an-
other needy person?

   (If yes): How much?

   Did you provide any financial assistance to or for relatives (including children and
parents) who do not live with you?

   (If yes): How much?

   How much, if anything, have you and the members of your family contributed to
political organizations in the past year?

   Volunteering (Prompt formal volunteering by subsector, one prompt for informal):

   In which, if any, of the following areas have you done some volunteer work in the
past 12 months?

   •   Health
   •   Education
   •   Religious organizations
   •   Human services
   •   Etc.
   •   Informal—alone—not for pay; Other

   (If yes to any of the above): asked number of hours in past month and past week.

METHOD Module:

   Giving (Prompt by method of contact [formal], several prompts for informal):

  In the past 12 months, have you and your household made a donation to charitable
and nonprofit organizations by

                         Downloaded from nvs.sagepub.com by guest on February 27, 2015
Methodology Is Destiny                                                                   651

   •   Responding to a request through the mail;
   •   Paying to attend a charity event;
   •   Using payroll deductions;
   •   Sponsoring someone in an event such as a walk-a-thon;
   •   Etc.?

   (If yes to any of the above): asked dollar amount for each.

  In the past 12 months, did you give any money directly to the homeless or street
people?

   (If yes): How much?

   Did you give any money to relatives, including children and parents, who do not
live with you?

   (If yes): How much?

   Did you give any money to other individuals not already mentioned?

   (If yes): How much?

   Volunteering (Prompt formal and informal volunteering by method of contact):

   In the past year, did you do any of the following volunteer activities through a group
or organization:

   •   Canvassing, campaigning, or fundraising;
   •   Serve on a board or committee;
   •   Organize or supervise events;
   •   Provide counseling or health care;
   •   Etc.?

  (If yes to any of the above): asked about months and weeks volunteered, regularity,
hours per week and month, extra hours.

   In the past 12 months, did you help anyone on your own, not through a particular
organization, by

   •   Cooking or cleaning
   •   Shopping, driving
   •   Teaching or coaching
   •   Etc.?

METHOD/AREA Module:

   Giving (Prompt by method of contact, then by subsector for formal giving; several
prompts for informal giving):

                         Downloaded from nvs.sagepub.com by guest on February 27, 2015
You can also read
NEXT SLIDES ... Cancel