Differential Grading Standards and University Funding: Evidence from Italy
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Di¤erential Grading Standards and University Funding:
Evidence from Italy
Manuel Baguesy Mauro Sylos Labiniz Natalia Zinovyevax
February 11, 2008
Abstract
This paper documents that grades vary signi…cantly across Italian public universities and de-
grees. We provide evidence suggesting that these di¤erences re‡ect the heterogeneity of grading
standards. A straightforward implication of this result is that university funding schemes based
on students’academic performance do not necessary favour universities that generate higher value
added. We test this for the case of the Italian funds allocation system, which rewards universities
according to the number of exams passed by their students. We …nd that university departments
that rank higher according to this indicator actually tend to be signi…cantly worse in terms of
their graduates’performance in the labour market.
JEL Classi…cation: I2, J31, J64.
Keywords: Higher Education, Grading Standards.
We would like to thank Stijn Kelchtermans, Reinhilde Veugelers and participants at seminars at the Université Louis
Pasteur, Sant’Anna School of Advance Studies, 2nd EIASM workshop on Process of Reform of University Systems, XXII
Jornadas de Economia Industrial, University of Alicante, the V Brucchi Luchino workshop and the CESifo Workshop
on Innovation and Higher Education for their comments on an earlier draft of this paper. The empirical analysis of this
paper would not have been possible without the data and the help provided by ISTAT (the Italian Statistical O¢ ce).
The econometric analysis was carried out at the ADELE Laboratory in Rome.
y Universidad Carlos III
z IMT Lucca Institute for Advanced Studies
x BETA at University of Strasbourg and FEDEA.
11 Introduction
In a number of European countries - including Italy, Spain and France - university grading standards are
presumed to be similar across institutions. This presumption justi…es the legal value that is typically
given to university titles and explains why public funding of universities is increasingly related to the
number of diplomas or grade points assigned by universities.
This paper empirically investigates the existence of di¤erences in grades and grading standards
across Italian universities. It exploits three editions of a survey run on a representative sample of
Italian graduates. The survey contains information about graduates’ academic and labour market
performance, as well as a large set of individual characteristics, including high school grade, province
of origin and various measures of family background. Conditional on this extensive set of controls, we
…nd that grades vary considerably across universities and disciplines. Evidence from graduates’labour
market performance and post-university external professional quali…cation exams (”abilitazione pro-
fessionale”) suggests that these variations in grades do actually re‡ect di¤erences in grading standards
and not true changes in students’quality. Indeed, we …nd a signi…cant negative correlation between
departments’average grades and the labour market outcomes of their graduates, i.e. graduating from
a high grading department leads to a higher unemployment probability and lower wages. As well,
graduates from departments with high average grades do not have higher chances to get professional
quali…cations in external examinations.
A straightforward policy implication follows from the above results. Policy makers should be
very cautious about using students’ academic performance as a proxy for university value added.
If, as shown in this paper, grading standards vary signi…cantly across departments and universities,
rewarding universities with high graduating rates may lead to undesirable consequences.
We test the relevance of this hypothesis using the main output variable that the Italian government
takes into account in order to …nance universities: the number of full-time equivalent students (FTE),
which measures the number of exams that students have passed in a given year. Consistently with our
predictions, we …nd that graduates from universities with a relatively higher number of FTE perform
signi…cantly worse in the labour market and do not obtain better results in professional quali…cation
exams. The evidence thus suggests that a …nancing scheme which is meant to reward those universities
that produce higher value added is, instead, favouring those universities with lower standards.
The rest of the paper is organized as follows. Section 2 presents the existent literature on grading
standards. Section 3 describes the data and the main variables use in the empirical part. Section 4
presents the empirical analysis and discusses policy implications. Finally, Section 5 summarizes the
main results and provides the conclusions.
2 Background
The issue of educational standards has been widely discussed in the economics literature both from
a theoretical and an empirical perspective. Grading standards may vary over time and across higher
education institutions for a number of reasons. Standards may adjust to the quality of students (Strenta
and Elliott, 1987). As well, professors may in‡ate grades to escape negative evaluations by students,
whose opinions matter for tenure and promotion decisions (Nelson and Lynch, 1984; Siegfried and Fels,
1979). Some departments may also increase their grades to …ll poorly attended courses that might
otherwise be canceled (Dickson, 1984; Staples, 1998). In addition, Freeman (1999) argues that given
the institutional constraints that prevent, within each university, a system of ‡exible money pricing for
those courses with di¤erent expected earnings, instructors and departments may act strategically to
manage enrolment by adjusting the time and the e¤ort cost of achieving a given grade. More generally,
Costrell (1994) notes that if institutions choose grading standards in a decentralized way a free rider
problem may arise, as high standards might not be fully appropriated by each institution. De Paola
and Scoppa (2007) point out that, in a decentralized setting, educational standards might be also
in‡uenced by the existence of labour market distorsions.
2An extensive empirical literature has documented the existence of variations in grades over time
across American universities and colleges. In particular, there has been, at least since the 1960’s,
an increase in the grades issued by American universities, coupled with the perception of a deteri-
oration in academic standards (Kolevzon, 1980; Sabot and Wakeman-Linn, 1991; Anglin and Meng,
2000). As well, there exists a line of studies which provide evidence on divergence in grades across
di¤erent disciplines (Sabot and Wakeman-Linn, 1991; Freeman, 1999; Dickson, 1984). For instance,
Sabot and Wakeman-Linn (1991) report average grades received by students in several disciplines in
eight American colleges and universities, …nding a clear division of colleges into high and low grading
departments.
Di¤erences in grades are also observed in Europe. A report on the development of exam grades
in Germany …nds that the average grades vary widely across universities (Wissenschaftsrat, 2004).
Several authors also observe that in the UK degree results vary according to institution. For example,
Chapman (1997) studies the degree results from 1973 to 1993 for eight disciplines and …nds a clear
tendency for certain universities to award consistently higher percentages of top degrees in all disciplines
with respect to the corresponding national average. As far as Italy is concerned, Boero et al. (2001)
report that grades tend to vary signi…cantly across degrees and regions.
Unfortunately, in most of these studies it is di¢ cult to disentangle whether the observed di¤erences
in grades re‡ect di¤erent quali…cations and performance or, conversely, di¤erences in teaching and
assessment practices. As Boero et al. (2001) put it, whether the observed di¤erences ”indicate use
of di¤erential standards across the di¤erent institutions or genuine institutional di¤erences in value-
added cannot be identi…ed from the data.” (Boero et al., 2001) (p.27). However, assessing whether
the observed heterogeneity in grades stems form di¤erent grading standards or from di¤erences in
graduates’true performance might be important for a number of reasons. Both in Europe and in the
US variations in grading standards might be problematic in the presence of informational asymmetries
about the quality of graduates and/or institutions. Most importantly, in many European countries
the institutional design of higher education typically requires the homogeneity of grading standards
across institutions. This assumption explains why titles have a legal value and are legally required for
many occupations and, as well, why several countries, such as Italy and Denmark, have adopted output
funding schemes based on the number of diplomas or grade points each higher education institutions
delivers.
3 Data
We investigate the potential existence of di¤erences in grading standards across Italian universities
and …elds of studies using a very detailed dataset concerning Italian university graduates which allows
to observe their socioeconomic background, high school grades, university performance and, …nally,
their outcomes in the labour market and in professional quali…cation exams.
More speci…cally, our main data is drawn from three distinct but almost identical surveys named
Indagine Inserimento Professionale Laureati (Survey on University-to-Work Transition) run in 1998,
2001, and 2004 on individuals that graduated in 1995, 1998, and 2001 respectively.1
The target samples consist of 25716 individuals in 1998, 36373 individuals in 2001, and 38470
individuals in 2004. They represent respectively the 25%, 28.1%, and 24.7% of the total population of
university graduates in Italian universities. The response rates were of 64.7%, 53.3%, and 67.6% for
a total of 17326, 20844 and 26006 respondents. In all three years the sample is strati…ed according
to sex, university and obtained degree and in the analysis below all estimations are performed using
strati…cation weights. We exclude from the sample graduates from physical education studies and from
the so-called “laurea primo livello”, since they were surveyed only in the 2001 edition (501 and 475
observations respectively).
1 Di¤erences may stem from the di¤erent interviewing technologies used in the surveys: in 1998 ISTAT mailed paper-
based questionnaires, while in 2001 and 2004 graduates were …rst contacted by mail and then questions were asked
following the so-called C.A.T.I. (Computer Assisted Telephone Interview) technique.
3As other European continental countries, Italy has a system of open admission into public univer-
sities: most departments are obliged to admit every applicant, without being allowed to set up any
entry restrictions. This system is common to all public universities and all disciplines except medi-
cine, veterinary and architecture (84% of our sample). Only 7% of graduates of our sample attended
a private university. For a number of reasons, grading standards are likely to be di¤erent in those
universities and …elds of study that can select their students; thus, we further restrict our sample to
those departments that cannot select students. This reduces the total size of the sample to 61844
observations.
The surveys provide information on (i) individual characteristics that are predetermined with re-
spect to college choices and outcomes, (ii) college-related individual indicators and (iii) labour market
outcomes. The …rst set of variables includes information on the individual socio-demographic back-
ground such as gender, nationality, number of siblings, province of residence before college enrolment,
parents’education and employment when respondent was around 14 years old, the situation of mili-
tary service obligations before attending university and self-reported high school curricula - high school
grade and type of school attended. The second includes university-related indicators: the type of de-
gree and university attended, educational outcomes— i.e. …nal grade obtained and the number of years
spent for the completion of the degree2 — and additional information such as occupational status dur-
ing studies, changes in the degree followed, attainment of an other degree and whether the respondent
originated from a town or province other than the one where her university was located. O¢ cial grades
range from 66 points and a maximum of 110 e lode. Third, the survey collects self-reported information
about a number of occupational outcomes three years after graduation. Among others, it is possible
to observe whether the graduate is employed, whether the job requires a university degree, her wage
and several indexes of job satisfaction. Table 1 depicts descriptive statistics for the key individual
variables.3
In addition to the individual information, we use data on several college characteristics. Fields
of study are aggregated in twelve di¤erent disciplines.4 Table 2 displays descriptive statistics at the
department level on the share of Full Time Equivalent Students (FTE) - the main measure used by
the Ministry for distribution of ordinary …nancial funds across universities - and ordinary …nancial
funds themselves.5 Finally, we also consider a number of demographic and economic indicators at the
provincial level such as GDP, total population and unemployment.
4 Empirical Analysis
To begin with, we investigate whether grades vary signi…cantly across disciplines and universities.
Then, we analyze whether the potential existence of di¤erences in grades across institutions stems
from di¤erences in grading standards or, rather, it re‡ects genuine di¤erences in institutional value
added. Finally, we analyze how the existence of di¤erential grading standards a¤ects the funding of
Italian universities.
2 In Italy the …nal grade is calculated as the sum of the grades obtained by the graduate during her courses plus the
grade received for the so-called degree dissertation (tesi di laurea). Any student whose …nal grade is higher than 110
obtains what is known as “110 e lode”. For simplicity, in the empirical analysis reported bellow the potential existence
of grades above 110 has been disregarded. The results obtained using a tobit regression, available upon request, are very
similar to the ones reported here. Also note than in the Italian education system in the analyzed period students were
not constrained either in time or in the number of trials taken for passing exams.
3 The unemployment rate for graduates in our sample is 14.7 percent. It is consistent with the OECD 2003 data
suggesting that 13.6 percent of Italian graduates aged 25-29 not being in education are unemployed. Italian graduates
experience disadvantage in terms of early performance in the labour market as the overall unemployment rate among
individuals aged 25-29 is 10.4 percent (OECD, Education at a Glance 2005)
4 The aggregated disciplines are Agriculture, Architecture, Chemistry and Pharmacy, Economics and Statistics, En-
gineering, Law, Literature, Medicine and Surgery, Pedagogy, Political and Sociological Studies, Sciences, Veterinary. In
what follows the term department stands for the corresponding disciplinary unit within a particular university.
5 Information on the number of FTEs comes from the Osservatorio per la valutazione del sistema universitario (1998).
See Perotti (2005) for detailed information on how the number of FTEs a¤ects universities’funding.
44.1 Grades
The grades obtained by a university graduate are likely to be related to a number of personal charac-
teristics including parental background and pre-university ability. We estimate the following model:
G i = X i + Df + Du + t + "itf u ; (1)
where Gi is a measure of the academic results obtained by individual i and Xi is a set of individual
characteristics, as described in Table 1, including dummies for the province where the individual lived
before attending university and gdp and unemployment rates at the provincial level. Df and Du are
the sets of dummies corresponding, respectively, to the …eld of study (or discipline) and university.
The time dummy controls general changes across time. Finally, the error term "itf u captures any
remaining factor a¤ecting academic performance.
Column 1 of Table 3 shows the results of an OLS estimation of equation (1) where the dependent
variable is the …nal aggregate grade obtained by the individual during her studies. In addition to
individual predetermined characteristics, the regression also controls for the number of extra years
taken to graduate.6 The e¤ect of individual characteristics is largely consistent with those obtained
by previous studies.7 We also observe that grades are positively correlated with unemployment rates.
This is consistent with the work of Dornbusch et al. (2000) and Di Pietro (2006), who point out
that local labor market conditions may in‡uence students’decisions. Lower unemployment rate may
encourage a number of students to devote less e¤ort to university in order to take advantage of the
improved labor market conditions.
Moreover, grades tend to vary to a large extent both across universities and across faculties.8 Figure
1 shows the set of estimated university dummy coe¢ cients, i.e. the component of an individual’s grade
that is statistically explained by her attendance to a given institution, conditional on her observable
characteristics, discipline, geographical origin and the time she took to graduate. Universities are
ordered from left to right according to their o¢ cial university code, lower codes corresponding in general
to northern locations and bigger codes’to southern ones. The positive slope suggests that, as one moves
across universities from the north to the south of Italy, grades - conditional on individual’s observable
characteristics - tend to increase. Similarly, Figure 2 shows how grades vary across disciplines. This
…gure suggests that there are notable di¤erences in the size of discipline …xed-e¤ects on grades with
Engineering, Economics and Statistics, Chemistry and Pharmacy, and Law being among the lowest
grading and Agriculture, Literature, Pedagogy and Architecture among the highest grading disciplines.
The second column of Table 3 displays the results of the above model when we use as dependent
variable the number of extra years taken to graduate. The previous …ndings are largely con…rmed.
Results concerning the variation of university and discipline dummy coe¢ cients in this case are quali-
tatively very similar to the ones of Figures 1 and 2 and are available upon request.
Two important caveats apply to the above estimations. First, note that the estimation builds
on the information provided by individuals with similar characteristics, including geographic origin,
but who decide to attend di¤erent departments. This strategy provides consistent estimates as long
as these individuals do not di¤er signi…cantly in their unobservable characteristics. Second, another
important concern regards the endogeneity of the sample. In fact, we observe only graduates, but not
drop-outs.9
6 The di¢ culty of each particular program could be described in two ways: as the time that is necessary in order to
complete a program and obtain a certain grade or as the …nal grade that an individual will obtain if she takes a given
period time to graduate.
7 See, for instance, Boero et al. (2001) who studies the determinants of academic success using the ISTAT survey
corresponding to year 1998.
8 The inclusion in equation (1) of university and discipline …xed a¤ects signi…cantly the explanatory of the model.
Including university dummies increases the R-square from 21.68 percent to 28.54 percent. The subsequent inclusion of
the discipline …xed e¤ects raises R-square to 39.82 percent.
9 This shortcoming generates two problems. First, the factors that a¤ect the grades obtained by those students that
do not manage to graduate could di¤er from the factors a¤ecting the grades obtained by graduates. A key assumption
is, therefore, that the grades obtained by graduates consistently re‡ect, conditional on observables, the grades obtained
54.2 Di¤erences in Quality or Di¤erences in Grading Standards?
The above results show that grades, conditional on graduates’predetermined characteristics, tend to
vary greatly across universities and …elds. In principle, these di¤erences could be due either to the value
added by universities or to their grading standards. To investigate these alternative explanations we use
two additional proxies of quality. First, we exploit the indicators detecting graduates’labour market
performance. If higher grades re‡ect higher value added, graduates from high grading departments
should perform better in the labour market. Second, we use the outcomes of external professional
qualifying exams. In Italy, they are compulsory for a number of professional occupations. If higher
grades re‡ect higher quality, graduates from high grading institutions should display higher passing
rates.
4.2.1 Labour Market Outcomes
Graduates’labour market performance Li is likely to be a¤ected by a number of socioeconomic char-
acteristics Xi , by their …eld of study Df and by the university attended Du : Equation (2) analyzes
this relationship:
Li = t + Xi + Df + Du + "itf u ; (2)
Table 3 presents the estimation results of this model when labour market performance is measured,
three years after graduation, by the employment status (column 3), the wage (column 4) and the
probability of …nding a job which requires a university degree (column 4) of those individuals who are
in the labour force.10 If, on the one hand, female perform better in terms of grades, on the other,
they exhibit a worse performance in the labour market. Similarly, foreign graduates are not able to
transform their higher academic performance into better labour market outcomes.
In addition to personal characteristics, the institution attended is a key predictor of future labour
market performance. Figure 3 depicts the estimates of universities’…xed e¤ects on wages conditional on
the individuals’observable characteristics, their geographical origin and discipline. Again, universities
are ordered from left to right according to their o¢ cial ISTAT code number, which increases as we
move from the north to the south of Italy. Thus, the negative slope observed in Figure 3 suggests
that northern universities’ graduates tend to earn higher wages than southern universities’ ones. A
similar pattern is observed if we restrict our analysis to graduates who …nished their studies on time.
Including the region of actual residence does not a¤ect the pattern observed in the histogram. Thus,
our results are not driven by unobserved labour market conditions. The picture is similar if we use
as dependent variable graduates’employment status: given two students with similar socioeconomic
backgrounds and geographical origins, those who graduate from a northern university are more likely to
be successful in the labour market than those who graduates from a southern university, even if they end
up working in the same region. This result is consistent with previous studies that also …nd a premium
for graduating in the North (Brunello and Cappellari, 2005; Pozzoli, 2006; Makovec, 2007). Moreover,
we observe signi…cant di¤erences across disciplines in terms of wages. In particular, conditional on
graduating in the same university, high school grades, individual background and province of origin,
graduates in Engineering, Economics and Statistics, Chemistry and Pharmacy and Medicine are likely
to have higher wage with respect to graduates in Veterinary, Literature, Law and Pedagogy (Figure
4).
by those students that dropped out before graduation. Second, a more subtle problem is related to the fact that the
very same unobservable characteristics – i.e. talent or grading standards– that a¤ect grades do also a¤ect selection into
the sample, this is, graduation. This makes the usual selection-based-on-observables assumption likely to fail. Still, the
nature of the problem allows us to make some predictions about the direction of the bias, at least among the cohort
of students that graduate on time. Any factor that generates an increase in grades would presumably increase the size
of this cohort. The new sample would include individuals which are, conditional on observables, relatively worse in
unobservables. This suggests that the e¤ect of factors that generate an increase in grades will tend to be underestimated
or, in other words, that the estimated coe¢ cients will tend to be a lower bound of their true value.
1 0 Results are essentially unchanged if we consider instead the whole population of graduates, including also those
graduates that do not look for a job.
6As shown in Figures 1 and Figure 3, while grades tend to be higher in southern universities, labour
market outcomes tend to be better for those that graduate in the North. With the exception of Law
departments11 , the same pattern generally holds at the discipline level: high grading disciplines tend
to provide lower labour market opportunities for their student. This descriptive evidence suggests that
there exists a negative correlation between departments’ grades and their graduates’ labour market
outcomes, both across universities and across …elds of study. Bellow we formally test this statistical
relationship.
First, we estimate an equation, in which -as in equation (1)- we analyze the determinants of
grades, but we substitute the discipline and university dummies with a set of dummies speci…c to each
university department separately for 1995, 1998 and 2001 Dtd :
Gi = Xi + Dtd + t + "itd : (3)
Second, using the department dummies coe¢ cients (b), we decompose individuals’ grades into
two components: (1) btd Dtd ; re‡ecting the (conditional) average grade obtained by individuals that
graduated within the same cohort and department and (2) the relative grade obtained by the individual,
calculated as a di¤erence between the actual grade and the estimated grade conditional on personal
e i = Gi btd Dtd ]. Third, we study how these components a¤ect labour market
characteristics [G
performance measures:
Li = t
e i + btd + "itd ;
+ Xi + G (4)
Table 4 presents the estimation results of equation (4) using three di¤erent measures of graduates’
labour market performance: the probability of being employed (columns 1 & 2), the probability of
…nding a job that requires a university degree (columns 3 & 4) and the expected wage (columns 5 &
6). Conditional on their observable personal characteristics, the number of years spent in university
and the discipline chosen, students that obtain higher grades relative to their classmates are more
likely to be employed three years after graduation and, if employed, tend to earn a signi…cantly higher
wage. However, the department’s average grade has the opposite e¤ect. Students that graduated from
universities where average grades were higher are signi…cantly less likely to be employed (column 1)
and, if employed, they are not more likely to have a job that requires a degree (column 3) and do not
tend to earn more (column 5). Results remain essentially the same if we include among the controls
graduates’class size or the region of graduates’residence when being interviewed. In columns 2, 4 and
6 we compare individuals who graduated in the same university but who had enrolled into di¤erent
…elds. We …nd that those individuals who obtained their degree in departments with relatively higher
average grades are signi…cantly less likely to …nd a job which requires being a graduate (column 4)
and actually tend to earn signi…cantly less (column 6). As in the previous analysis, controlling for the
region of current residence does not have a signi…cant e¤ect on the estimates.
The above results may help to rationalize the puzzling correlation that arises when we compare
the academic performance of Italian graduates with their performance in the labour market. A simple
descriptive analysis of the data provided by the ISTAT surveys on year 1995, 1998 and 2001 graduates
reveals that those individuals that had obtained higher grades in university do not obtain higher wages
later on (see Table 5, columns 1, 2 & 3).12 In the last edition of the survey, it turns out that grades
are negatively correlated with earnings: graduates who obtained lower grades tend to earn relatively
more. Of course, as our above results suggested, this negative relationship is driven by the di¤erent
grading standards that departments apply. As expected, once we take into account the university and
the department from which an individual has graduated the expected positive relationship between
grades and salary is re-established (though, signi…cant only at 11%).
1 1 Law is a quite particular case. Note that in Italy graduates in Law must spend at least two years as interns before
taking professional quali…cation exams and becoming lawyers.
1 2 Boero et al. (2001) already point out that the grades of 1998 graduates show no correlation with their wages
74.2.2 Professional Quali…cation Exams
An additional way to test whether higher grades re‡ect higher quality or simply di¤erent standards is to
exploit the outcomes of post-university professional quali…cation exams (”abilitazione professionale”).
These exams are granted by o¢ cial professional organizations and are meant to certify that a given
graduate holds a minimal set of competencies for a given profession. They are not compulsory but are
required in order to perform legally a number of professions. The set of professions for which an exam
is required includes Architects, Chemists, Accountants, Physicians, Psychologists or Engineers.13
The ISTAT survey allows to observe whether a given graduate has passed the corresponding external
quali…cation exam within three years of her graduation. A potential source of bias of this measure might
arise from the fact that we only observe whether individuals succeeded in the professional quali…cation
exam, but not whether they took it and failed. This problem is likely to be bigger in those disciplines
where graduates have other professional possibilities that do not require an o¢ cial quali…cation.
As it is shown in Table 1, about half of respondents have passed an external quali…cation exam
after graduation. However, the distribution of this percentage across …elds is not homogenous14 : the
probability that a graduate pass the quali…cation exam ranges from 0 to 40 percent in 66 percent of
courses, from 40 to 60 in 4 percent of courses and from 60 to 100 percent in 30 percent of courses.
In other words, there exist a big group of courses in which more than 60 percent of graduates do not
ever pass the exam, another group of courses in which more than 60 percent of graduates pass the
exam and very few courses that could not be attributed either to the …rst or to the second group.
In order to minimize the problem of self-selection described above we restrict the analysis to those
occupations where graduates have a very limited scope for professional possibilities unless they pass
the external quali…cation exam. In what follows only the latter group of courses, namely, the one
in which more than 60 percent of graduates passed the exam (mainly Engineering and Chemistry
courses), is considered.
Column 6 of Table 3 shows the relationship between individual characteristics and the probability
of success in quali…cation exams. As expected, success in this exam is closely related to graduates’
quality, as measured by high school grades and other socioeconomic characteristics.
In Table 6 we analyze the relationship between university grades and performance in external qual-
i…cation exams. We …nd that conditional on the department and university attended, those graduates
that obtained relatively better grades than their classmates in university are signi…cantly more likely
to pass the quali…cation exams. Then we investigate whether the (conditional) average grade of all
individuals that graduated within the same cohort and department btd Dtd , as de…ned in the previous
subsection, has a similar positive e¤ect on graduates’performance in professional quali…cation exams
Ai , estimating the following regression:
Ai = t
e i + btd + "itd :
+ Xi + G (5)
As shown in column 2 of Table 6, while we still …nd that within each department better students
are more likely to succeed in professional quali…cation exams, in general graduates from departments
with higher average grades tend to be less successful in professional quali…cation exams. Given that
in these …elds the lack of success in external exams is associated with signi…cantly lower employment
rates and with signi…cantly lower probabilities of …nding a job which requires a degree, our results
suggest that the variations in the department-average component of grades are not likely to re‡ect
better quality.15
1 3 For a complete list of Italian professional organizations and details of respective exams see
http://it.wikipedia.org/wiki/Albo_professionale.
1 4 Degree course de…nes graduates’ specialization within a certain discipline. Each disciplinary …eld on average o¤ers
around 10 di¤erent degree courses.
1 5 Those graduates who passed the professional exam have a probability of …nding a job that matches the knowledge
acquired in university which is 11 percentage points higher than the rest of individuals in the sample. Note also that if
individuals’ unobserved ability in university performance was positively correlated to individuals’ unobserved ability in
professional quali…cation exams, the estimated coe¢ cient must be considered as an upper bound of its true value.
84.3 Di¤erential Grading Standards and the Funding of Italian Universities
Before 1993 the Italian national ministry of education was in charge of …xing the total amount of
funds, their shares across public universities and their allocation across disciplines. Its decisions were
largely made on historical bases and were sometimes a¤ected by distinct deals with single institutions
and faculties within institutions. In 1993 a reform was approved allowing each university to become
an autonomous entity with its own budget to be allocated across distinct disciplines (law n.537/1993).
Moreover, discretion was replaced by a complex set of rules, which in the short run left about 90 per
cent of the big bulk of public funding to be assigned on historical basis and the rest to be allocated
via an equalization component (EC). The latter is supposed to progressively substitute the former.
The EC objective is two fold: …rst, to reduce public funding disparities across universities and across
disciplines and, second, to incentivate quality. On the incentives side, the EC seeks to reward the
quality of teaching linking funding to the number of exams passed by enrolled students. Technically,
the funds depend positively on the share of full-time equivalent students (FTE), which is de…ned as
the ratio between the number of exams that students passed and the number of exams that students
should have taken. See Perotti (2002) for details.
In principle, the measure of quality based on the share of FTE students might be subject to at
least two problems. First, it fails to take into account the initial quality of students. Universities that
attract students of better quality will tend to perform relatively better even if they fail to provide
better education. Second, in the absence of quality assurance mechanisms, the FTE might capture
both the students true quality and the easiness (or grading standards) of a given institution.16 In
fact, the evidence provided in the previous section suggests that the relationship between the average
grades issued by di¤erent universities and the performance of their graduates in the labour market
or in quali…cation exams is, if any, negative. A straightforward implication of this result is that
…nancing universities based on their self-evaluated academic performance does not necessary reward
those universities that generate a higher value.
Table 7 shows the relationship between graduates’labour market outcomes and the share FTE stu-
dents in the department where they graduated, conditional on graduates’socioeconomic background
and pre-university measures of quality. While the number of FTE students is meant to proxy the
quality of a department, we …nd a strong and signi…cant negative relationship between this measure
and graduates’ labour market outcomes, as measured by occupation rates (column 1) and obtaining
a job which requires a university degree (column 2). We also …nd no signi…cant relationship what-
soever between the share of FTE students and graduates’ wages (column 3) or their performance in
professional quali…cation exams and (column 4).
To sum up, FTE fails to capture quality of institutions, at least as measured by graduates’perfor-
mance in the labour market and in professional quali…cation exams.
5 Conclusion
In recent years a number of European Countries, including Italy, have adopted output funding schemes
based on the number of diplomas or grade points each institution delivers. One of the preconditions
for such systems to be e¤ective in providing quality enhancing incentives is ensuring homogeneity of
educational quality and grading standards across institutions. Otherwise, as noted by Jacobs and Van
der Ploeg (2005), this practice might undermine incentives to improve educational quality, as in most
cases the quantity rather than the quality of output is rewarded due to the di¢ culties in measuring
the later.
In this paper we analyze grading standards across Italian universities and disciplines. More specif-
ically, we study the performance of several cohorts of Italian graduates in the labour market and in
1 6 In 1996 and 1999 two distinct kinds of evaluating committees were established to preserve quality: a National
Committee (Comitato Nazionale per la Valutazione del Sistema Universitario) and several Internal Committees (Nuclei
di Valutazione Interna). However, as convincingly argued by Perotti (2002), their objectives are too vague and they
turned out to be to be largely ine¤ective.
9external quali…cation exams and analyze how it relates to their performance in university. We …nd
that, conditional on a large set of individuals’ observable characteristics that includes geographical
origin, high school grade, and socioeconomic background, graduates from high grading departments
tend to perform signi…cantly worse in the labour market. Moreover, graduates from high grading uni-
versities are less likely to succeed in external qualifying exams that are required for many professional
activities. These results suggest that the signi…cant variations in grades that can be observed in Italy
across disciplines and universities re‡ect to a large extent di¤erences in grading standards.
In line with this evidence, we also …nd that the output measure of university quality that has been
adopted by the Italian Ministry of Education to allocate funds across universities - i.e. the number of
full-time equivalent students (FTE) de…ned as the ratio between the number of exams that students
passed and the total number of exams that they should have taken - is negatively correlated with
graduates’labour market outcomes.
This …nding rises concerns on the e¤ectiveness of such funding mechanisms. In the light of this
evidence, the implementation of quality ensuring mechanisms - such as a system of external examiners
as in the UK - should be seriously considered as a necessary complement to any output funding scheme.
Additionally, given that obtaining objective evaluations of external examiners might be itself problem-
atic and costly, a more radical policy option may involve fostering reputation e¤ects in the market for
higher education. This goal may be approached in di¤erent ways, for instance, by allowing universities
to select their students and, simultaneously, promoting student mobility, by letting universities set
tuition fees and introducing e¢ cient student loan systems.17
References
Anglin, P. M. and R. Meng (2000). "Evidence on Grades and Grade In‡ation in Ontario’s Universities".
Canadian Public Policy, XXVI(3), 361-368.
Boero, G., A. McKnight, R. Naylor and J. Smith (2001). "Graduates and the Graduate Labour Market:
Evidence from the UK and Italy". Lavoro e Relazioni Industriali, 131-172, 2001.
Brunello, G. and L. Cappellari (2005). "The Labour Market E¤ects of Alma Mater: Evidence from
Italy". IZA Discussion Paper, No. 1562. Forthcoming in Economics of Education Review.
Chapman, K. (1997). "Degrees of Di¤erence: Variability of Degree Results in UK Universities". Higher
Education, 33(2), pp. 137-153.
Costrell, R. M. (1994). "A Simple Model of Educational Standards". The American Economic Review,
84(4), pp. 956-971.
De Paola, M. and V. Scoppa (2007). "Returns to Skills, Incentives to Study and Optimal Educational
Standards". Journal of Economics, 92(3), 229-262.
Dickson, V.A. (1984). "An Economic Model of Faculty Grading Practices". Journal of Economic
Education, 6 (Summer),197-203.
Di Pietro, G. (2006). "Regional labour market conditions and university dropout rates: Evidence from
Italy". Regional Studies: The Journal of the Regional Studies Association, 40(6), 617–630.
Dornbush, R., A. Gentilini and F. Giavazzi (2000). "Italian Labour Force Participation: disguised
unemployment on campus". Paper presented at the Annual Meeting of the European Economic
Association, Bolzano.
Freeman, D.G. (1999). "Grade Divergence as a Market Outcome". Journal of Economic Education,
30(4), 344-351.
1 7 See Mas-Colell (2003) and Mas-Colell (2004) for a thorough discussion on reforms that might foster competition and
reputation e¤ects in the European higher education space.
10Jacobs, B. and F. Van der Ploeg (2005). "Guide to Reform of Higher Education: A European Per-
spective". Economic Policy, 535-592.
Kolevzon, M.S. (1981). "Grade In‡ation in Higher Education: A Comparative Study". Research in
Higher Education, 15(3), 195-211.
Makovec, M. (2007). "Does it Pay to Study Far from Home? Explaining the Returns to Geographic
Mobility of Italian College Graduates". Universitat de Alicante, mimeo.
Mas-Colell, A. (2003). "The European Space of Higher Education: Incentive and Governance Issues."
Rivista di Politica Economica, November-Dicember, 9-27.
Mas-Colell, A. (2004). "The European Higher Education Space: Funding Issues". Third MEMS Lec-
ture, Master’s Program in Economics and Management Science, Humboldt University, Berlin, 30
April 2004.
Nelson, J. P. and K. A. Lynch (1984). "Grade In‡ation, Real Income, Simultaneity, and Teaching
Evaluations". Journal of Economic Education, Winter, 21-37.
OECD (2005). Education at a Glance. Paris, Organization for Economic Co-operation and Develop-
ment.
Osservatorio per la valutazione del sistema universitario (1998). "L’evoluzione della domanda di for-
mazione universitaria: studenti, laureati e studenti equivalenti". Rome, Italy.
Perotti, R. (2002). "The Italian University System: Rules vs. Incentives". Paper presented at the …rst
conference on Monitoring Italy, ISAE, Rome, January.
Pozzoli, D. (2006). "The Transition to Work for Italian University Graduates: Determinants of the
Time to Obtain the First Job". Milan Catholic University, mimeo.
Sabot, R. and J. Wakeman-Linn (1991). "Grade In‡ation and Course Choice". Journal of Economic
Perspectives, 5(1), 159-170.
Siegfried, J. J. and R. Fels (1979). "Research on Teaching College Economics: A Survey". Journal of
Economic Literature, September, 923-969.
Staples, B. (1998). "Why Colleges Shower Their Students With A’s". New York Times, March 8.
Strenta, A. C. and R. Elliott (1987). "Di¤erential Grading Standards Revisited". Journal of Educa-
tional Measurement, 24(4), 281-291.
Wissenschaftsrat (2004). Empfehlungen zur Reform des Hochschulzugangs [On-line].
http://www.wissenschaftsrat.de/texte/5920-04.pdf.
11Table 1: Descriptive Statistics - Individual Characteristics
Mean Min Max
1. Predetermined Individual Characteristics
Gender (Share of females) 0.532 0 1
Age 27.587 21 75
When an individual was 14 years old his father was:
- working 0.960 0 1
- looking for a job 0.004 0 1
- a pensioner 0.017 0 1
- other 0.019 0 1
When an individual was 14 years old his mother was:
- working 0.494 0 1
- looking for a job 0.004 0 1
- a pensioner 0.020 0 1
- other 0.482 0 1
When an individual was 14 years old his father’s
highest educational title was:
- elementary license or none 0.190 0 1
- secondary education license 0.236 0 1
- higher education diploma 0.340 0 1
- university degree 0.226 0 1
- no answer 0.008 0 1
When an individual was 14 years old his mother’s
highest educational title was:
- elementary license or none 0.250 0 1
- secondary education license 0.259 0 1
- higher education diploma 0.350 0 1
- university degree 0.135 0 1
- no answer 0.006 0 1
Father’s sector of work
- agriculture 0.050 0 1
- industry 0.260 0 1
- services 0.672 0 1
- no answer 0.018 0 1
Number of siblings 1.313 0 4
Nationality:
- Italian 0.991 0 1
- European Union 0.006 0 1
- Extra-communitarian 0.003 0 1
Type of high school:
- scienti…c lyceum 0.413 0 1
- classic lyceum 0.193 0 1
- technical industrial institute 0.062 0 1
- technical institute for geometers 0.034 0 1
- technical commercial institute 0.128 0 1
- other type of technical institute 0.030 0 1
- teachers school or institute 0.062 0 1
- language lyceum 0.036 0 1
- professional institute 0.029 0 1
- art lyceum or institute 0.013 0 1
High school grade 49.085 36 60
Military service obligations:
- exempt 0.219 0 1
- before university 0.039 0 1
- other 0.742 0 1
2. College-related individual characteristics
Median number of extra years taken to graduate af-
ter the end of the o¢ cial program duration (4 stands 2 0 4
for 4 and more years)
University grade 103.628 66 110
Moved from other course 0.107 0 1
Second degree 0.014 0 1
Studied in the region of birth 0.793 0 1
12Table 1: (continued)
Mean Min Max
Studied in the province of birth 0.519 0 1
Studied in the town of birth 0.412 0 1
Moved from own town to study 0.300 0 1
3. Graduates’Post-Graduation Performance
Passed profession quali…cation exam 0.452 0 1
In the labour force 0.843 0 1
Employed if in the labour force 0.853 0 1
Employed in a job for ful…lling of which the obtained
0.644 0 1
university degree is necessary if in the labour force
Wage 1135.786 77.468 10000
Notes :The number of observations is 61844. (*) In this case the median value is reported instead of the mean. Value 4
means that 4 or more extra years to graduate have been employed. (**) The number of observations with non-missing
wage is 37552.
Table 2: Descriptive Statistics - Department Characteristics
Year Mean Std. Dev. Min Max
Full Time Equivalent (FTE) Students (%) 1995 46.394 12.748 13.118 94.608
University ordinary …nancial funds( ) 1995, 1998, 2001 188.982 186.856 11.3 1186.1
Professor per student ratio ( ) 1996, 1999 0.093 0.101 0.004 1.429
Notes : In 2001 there were 410 di¤erent departments. (*) In this case the statistics are reported at the university level
in billions of lire. Note that the ordinary …nancial funds are only available for public universities. (**) This is the ratio
of the number of professors to the total number of non-delayed students.
13Table 3: Individual Characteristics and Performance in University, Labour Market and External Professional Quali…cation Exams
(1) (2) (3) (4) (5) (6)
Extra Years in Employment Employment
University grade Log Wage Quali…cation ex-
University probability with knowledge
ams
match
OLS OLS Probit OLS Probit Probit
1. Pre-determined individual characteristics
Female 0.757*** (0.081) -0.067*** (0.015) -0.047*** (0.005) -0.128*** (0.007) -0.068*** (0.008) 0.001 (0.006)
Age -0.169*** (0.011) 0.160*** (0.004) 0.002*** (0.001) 0.012*** (0.001) -0.002** (0.001) -0.007*** (0.001)
Father was:
- working Benchmark Benchmark Benchmark Benchmark Benchmark Benchmark
- looking for a job -0.060 (0.496) -0.229** (0.101) -0.013 (0.025) -0.019 (0.052) -0.002 (0.048) -0.011 (0.034)
- a pensioner 0.308 (0.217) -0.007 (0.045) -0.011 (0.014) -0.034 (0.021) -0.026 (0.021) -0.006 (0.018)
- other 0.329 (0.272) 0.087 (0.057) 0.003 (0.015) -0.076*** (0.030) 0.033 (0.024) -0.004 (0.023)
Mother was:
- working Benchmark Benchmark Benchmark Benchmark Benchmark Benchmark
- looking for a job 0.575 (0.376) 0.195** (0.087) -0.007 (0.024) -0.216*** (0.064) -0.021 (0.049) 0.003 (0.031)
- a pensioner 0.150 (0.215) -0.047 (0.042) -0.016 (0.017) -0.019 (0.017) -0.023 (0.022) 0.005 (0.016)
- other -0.118* (0.064) 0.001 (0.013) -0.004 (0.004) -0.005 (0.007) 0.000 (0.006) -0.010** (0.005)
Father education:
14
- elementary license or none Benchmark Benchmark Benchmark Benchmark Benchmark Benchmark
- secondary education license -0.136 (0.094) -0.016 (0.020) -0.003 (0.006) 0.022*** (0.008) 0.017* (0.009) 0.001 (0.007)
- higher education diploma -0.193* (0.100) 0.004 (0.021) 0.000 (0.006) 0.033*** (0.009) 0.028*** (0.010) 0.010 (0.007)
- university degree -0.204* (0.120) -0.048* (0.025) -0.013* (0.008) 0.035** (0.011) 0.037*** (0.012) 0.018** (0.008)
Mother education:
- elementary license or none Benchmark Benchmark Benchmark Benchmark Benchmark Benchmark
- secondary education license -0.216** (0.088) -0.036* (0.018) 0.012** (0.005) 0.020* (0.007) 0.012 (0.009) 0.000 (0.007)
- higher education diploma -0.420*** (0.098) -0.087*** (0.021) 0.014** (0.006) 0.030** (0.009) 0.021** (0.010) -0.003 (0.007)
- university degree -0.260** (0.131) -0.204*** (0.028) 0.019** (0.007) 0.028*** (0.013) 0.036*** (0.013) 0.015 (0.009)
Father’s sector of work
- agriculture Benchmark Benchmark Benchmark Benchmark Benchmark Benchmark
- industry 0.375*** (0.140) 0.016 (0.030) 0.024*** (0.008) 0.004 (0.013) 0.021 (0.013) 0.005 (0.010)
- services 0.504*** (0.134) 0.039 (0.029) 0.020*** (0.008) -0.016 (0.013) 0.012 (0.013) 0.000 (0.010)
- other 0.831** (0.347) 0.026 (0.079) -0.004 (0.021) 0.071*** (0.033) -0.053 (0.037) 0.043* (0.018)
Number of siblings 0.085** (0.033) 0.012* (0.007) 0.007*** (0.002) 0.005* (0.003) 0.005 (0.003) -0.000 (0.003)
Nationality:
- Italian Benchmark Benchmark Benchmark Benchmark Benchmark Benchmark
- European Union 0.989 (0.819) 0.055 (0.147) 0.066* (0.024) 0.094 (0.080) 0.162** (0.055) 0.027 (0.043)
- Extra-communitarian 1.786*** (0.688) 0.054 (0.129) 0.058 (0.057) 0.069 (0.070) 0.124* (0.069) 0.001 (0.047)
Type of high school:
- scienti…c lyceum Benchmark Benchmark Benchmark Benchmark Benchmark Benchmark
- classic lyceum 0.409*** (0.080) 0.034** (0.017) -0.025*** (0.005) -0.030*** (0.009) -0.015* (0.009) -0.005 (0.007)
- technical industrial institute -1.062*** (0.133) -0.021 (0.026) 0.030*** (0.008) 0.019*** (0.009) -0.016 (0.012) 0.014** (0.007)Table 3: (continued)
(1) (2) (3) (4) (5) (6)
Extra Years in Employment Employment
University grade Log Wage Quali…cation ex-
University probability with knowledge
ams
match
- technical institute for geometers -1.458*** (0.167) -0.020 (0.034) 0.009 (0.010) -0.035** (0.014) 0.008 (0.016) 0.026*** (0.008)
- technical commercial institute -1.544*** (0.102) 0.050** (0.020) -0.005 (0.006) -0.009 (0.008) -0.033*** (0.010) -0.001 (0.011)
- other type of technical institute -1.516*** (0.172) 0.039 (0.035) 0.004 (0.011) 0.020 (0.013) -0.008 (0.017) 0.021** (0.009)
- teachers school or institute -0.882*** (0.122) 0.221*** (0.030) 0.003 (0.007) 0.027*** (0.011) 0.022* (0.013) 0.017 (0.011)
- language lyceum -1.181*** (0.141) 0.198*** (0.038) -0.004 (0.010) 0.011 (0.013) -0.052*** (0.015) -0.004 (0.019)
- professional institute -2.223*** (0.181) 0.034 (0.041) -0.004 (0.011) -0.023 (0.019) -0.021 (0.018) 0.002 (0.013)
- art lyceum or institute -1.524*** (0.237) 0.221*** (0.045) -0.029* (0.016) -0.070*** (0.022) -0.052** (0.024) 0.001 (0.013)
- other -1.092*** (0.416) -0.039 (0.094) 0.006 (0.024) -0.013 (0.045) 0.080** (0.038) 0.027 (0.031)
High school grade 0.303*** (0.004) -0.016*** (0.001) 0.002*** (0.000) 0.005*** (0.000) 0.003*** (0.000) 0.001* (0.000)
Military service obligations:
- exempt -0.077 (0.093) 0.008 (0.017) 0.002 (0.006) 0.023*** (0.008) 0.009 (0.009) -0.019*** (0.006)
- before university 0.085** (0.033) -0.691*** (0.044) 0.047*** (0.009) 0.092*** (0.013) 0.049*** (0.017) -0.024* (0.015)
2. College-related individual characteristics
Moved from other course 0.021 (0.095) -0.320*** (0.024) 0.012** (0.006) 0.017*** (0.008) -0.001 (0.010) 0.006 (0.008)
Second degree 1.184*** (0.420) -1.151*** (0.137) 0.047 (0.047) 0.049 (0.098) 0.127 (0.079) -0.079 (0.049)
Studied in the region of birth 0.262*** (0.100) 0.147*** (0.021) 0.004 (0.006) -0.035*** (0.008) -0.027*** (0.010) 0.025*** (0.008)
Studied in the town of birth 0.687*** (0.078) -0.050*** (0.016) 0.004 (0.005) 0.020*** (0.007) -0.009 (0.008) -0.038** (0.006)
15
Moved from own town to study 0.020 (0.073) 0.056*** (0.015) 0.004 (0.004) 0.009 (0.007) 0.020*** (0.007) 0.006 (0.009)
3. Province of birth characteristics, two years before graduation
GDP*(10) 0.064 (0.082) -0.013 (0.017) -0.002 (0.005) -0.011 (0.008) 0.020** (0.009) 0.018*** (0.001)
Unemployment 0.054*** (0.019) 0.010*** (0.004) -0.001 (0.001) 0.011*** (0.002) 0.007*** (0.002) 0.002 (0.002)
Population*(10,000) -0.003 (0.003) 0.004 (0.006) 0.001 (0.002) 0.005* (0.003) 0.003 (0.003) -0.005** (0.002)
4. Other Dummies and controls
Province of origin Yes Yes Yes Yes Yes Yes
Course …xed-e¤ect Yes
Discipline …xed-e¤ect Yes Yes Yes Yes Yes
University …xed-e¤ect Yes Yes Yes Yes Yes Yes
Extra years taken to graduate Yes
University grade -0.033*** (0.001)
(Pseudo) R squared 0.403 0.361 0.157 0.226 0.0811 0.1726
Number of observations 61844 61844 52532 37552 49103 26344
Notes : signi…cant at 10%; signi…cant at 5%; signi…cant at 1%. For probit regressions marginal e¤ects at mean values are reported. Standard errors in
parentheses.Table 4: The E¤ect of Grades on Labour Market Outcomes
(1) (2) (3) (4) (5) (6)
Employment Employment Knowledge Knowledge Log Wage Log Wage
match match
Probit Probit Probit Probit OLS OLS
Individual relative grade 0.003* 0.003 0.010*** 0.010*** 0.003*** 0.004***
(0.002) (0.002) (0.002) (0.001) (0.001) (0.001)
Department …xed e¤ect on grade -0.020*** 0.005 0.006 –0.014*** 0.001 -0.009***
(0.006) (0.004) (0.004) (0.003) (0.001) (0.001)
Controls:
Year of graduation Yes Yes Yes Yes Yes Yes
Extra years taken to graduate Yes Yes Yes Yes Yes Yes
Individual characteristicsz Yes Yes Yes Yes Yes Yes
Province of origin*(High school grade) Yes Yes Yes Yes Yes Yes
Province of origin characteristicsz Yes Yes Yes Yes Yes Yes
Discipline dummies Yes Yes Yes
University dummies Yes Yes Yes
Observations 42819 42819 40051 40051 31040 31040
(Pseudo) R square 0.1614 0.1431 0.0780 0.0684 0.2335 0.2081
Notes : signi…cant at 10%; signi…cant at 5%; signi…cant at 1%. For probit regressions marginal e¤ects at mean values are reported. Standard errors in
16
parentheses. Students from private universities and departments with constrained admission are excluded. z Variables listed in Table 3 are included among the
regressors.You can also read
