Universal Dependencies and Quantitative Typological Trends. A Case Study on Word Order

Page created by Glenn Weber
 
CONTINUE READING
Universal Dependencies and Quantitative Typological Trends. A Case Study on Word Order
Universal Dependencies and Quantitative Typological Trends.
                                  A Case Study on Word Order

                Chiara Alzetta+ , Felice Dell’Orletta* , Simonetta Montemagni* , Giulia Venturi*
                                     +
                                       Università degli Studi di Genova, Via Balbi 5, Genova, Italy
          *
              Istituto di Linguistica Computazionale “Antonio Zampolli” (ILC–CNR) – ItaliaNLP Lab, www.italianlp.it
                                                      via G. Moruzzi, 1- Pisa, Italy
                    chiara.alzetta@edu.unige.it, {felice.dellorletta,simonetta.montemagni,giulia.venturi}@ilc.cnr.it
                                                                  Abstract
The paper presents a new methodology aimed at acquiring typological evidence from “gold” treebanks for different languages. In
particular, it investigates whether and to what extent algorithms developed for assessing the plausibility of automatically produced
syntactic annotations could contribute to shed light on key issues of the linguistic typological literature. It reports the first and promising
results of a case study focusing on word order patterns carried out on three different languages (English, Italian and Spanish).

Keywords: Linguistic Knowledge Extraction, Dependency Treebanks, Linguistic Typology

                       1.     Introduction                                ferent languages can significantly contribute to model lin-
The interaction between linguistics and computational lin-                guistic variation within and across languages. Word order
guistics has a long history dating back to the 60’s. In                   variation represents a widely investigated topic of the typo-
Kučera (1982), it is explicitly stated that “computational               logical literature whose recent developments include fine-
linguistics provides important potential tools for the test-              grained studies based on a wide range of features and their
ing of theoretical linguistic constructs and of their power               frequency distributions typically acquired from annotated
to predict actual language use”. This still appears to rep-               corpora (O’Horan et al., 2016). To mention only a few, see
resent a key objective, as claimed e.g. by Martin Kay in                  e.g. Gulordava and Merlo (2015a), Gulordava and Merlo
his ACL Lifetime Award speech in 2005 (Kay, 2005), or by                  (2015b) Futrell et al. (2015), Merlo (2016).
the more recent papers gathered in the Special Issue of the               More recently, such an approach to typological studies has
journal “Linguistic Issues in Language Technology” (LiLT)                 also been prompted by the availability of multi–lingual
focusing on the relationship between language technology                  treebanks such as those designed and constructed within
and linguistic insights (Baldwin and Kordoni, 2011). Af-                  the Universal Dependencies project2 for over 50 languages.
ter more than 40 years from the first pioneering studies, the             Universal Dependencies (UD) is a framework for cross–
growing availability of linguistic resources such as anno-                linguistically consistent treebank annotation aiming to cap-
tated corpora for many languages combined with the in-                    ture similarities as well as idiosyncrasies among typolog-
creasing reliability of Natural Language Processing (NLP)                 ically different languages (e.g., morphologically rich lan-
methods and tools enables the acquisition of quantitative                 guages, pro–drop languages, and languages featuring clitic
evidence ranging across different levels of linguistic de-                doubling). The goal in developing UD was not only to sup-
scription which can significantly contribute to the study of              port comparative evaluation and cross–lingual parsing but
open issues of the theoretical linguistic literature.                     also to enable comparative linguistic studies (Nivre, 2015).
This holds particularly true for the area of typological stud-            Within this area of research, the paper reports the results of
ies which can benefit a lot from this synergy, making it                  preliminary experiments aimed at acquiring quantitative ty-
possible to acquire quantitative evidence shedding light on               pological evidence from a selection of the UD treebanks for
how, why and to what extent languages vary with respect to                different languages. In particular, it focuses on the widely
key features covering major areas of language structure. By               investigated topic of word order, with the specific aim of
exploring collections of linguistically annotated corpora for             reconstructing word order patterns within and across lan-
different languages, complex and articulated frequency dis-               guages, for what concerns both the linear order of words
tributions of language constructions can be extracted. In-                and its degree of flexibility (giving rise to a wide typol-
formation acquired from available corpora can significantly               ogy of languages going from fixed-order to free word or-
enrich typological descriptions of languages such as the                  der languages). For the acquired word order patterns, the
World Atlas of Language Structures (WALS) (M. S. Dryer                    study also aims at investigating the factors underlying the
and M. Haspelmath, 2013) 1 , the most commonly-used and                   preference for one or the other order, both intra- and cross-
broadest database of structural (phonological, grammatical,               linguistically.
lexical) properties of languages whose data are based on                  To pursue this goal, we decided to test whether existing al-
primary sources such as grammars, dictionaries and scien-                 gorithms for assessing the plausibility of automatically pro-
tific papers. But impact and role of this information type                duced syntactic annotations could be used to acquire use-
cannot be limited to the descriptive level. Typological evi-              ful quantitative typological evidence. In fact, the result of
dence inferred from linguistically annotated corpora for dif-             these algorithms is typically driven by linguistic properties

   1                                                                         2
       Available online at http://wals.info                                      http://universaldependencies.org/

                                                                   4540
characterizing the language being processed: by comparing
the results achieved against different languages, it is possi-
ble to acquire information concerning typological similar-
ities and differences. This kind of algorithms operate at
the level of either the whole syntactic tree (cfr. for exam-
ple Dell’Orletta et al. (2011) and Reichart and Rappoport
(2009)), or individual dependencies (see, among others,
Dell’Orletta et al. (2013) and Che et al. (2014)). Given the
focus of this study on specific constructions, we selected         Figure 1: Distribution of right– vs left–headed non–
the class of algorithms operating at the level of individual       pronominal nsubj relations in the three UD treebanks.
dependencies, and in particular on those ranking dependen-
cies by decreasing plausibility of annotation. These algo-
rithms, originally meant to discern reliable from unreliable
annotations within the automatic output of parsers, have
also been applied to manually revised (i.e. “gold”) linguis-
tic annotations with the final aim of identifying annotation
inconsistencies, and thus for also detecting annotation er-
rors (Alzetta et al., 2018). Tusa et al. (2016) represent
the first attempt to exploit the plausibility score returned
by this class of algorithms to acquire linguistic evidence,
i.e. to infer the prototypicality degree of specific linguis-      Figure 2: Distribution of right– vs left–headed amod rela-
tic constructions. The experiment was carried out against          tions in the three UD treebanks.
the Italian Universal Dependency Treebank (IUDT) (Bosco
et al., 2013) with promising results: the plausibility-based
ranking of dependencies corresponding to specific syntac-
tic constructions turned out to closely reflect their linguistic   der patterns, which may not always coincide. According
“markedness” degree.                                               to WALS, no dominant Subject-Verb order exists for both
In what follows, we focus on word pattern variation across         Italian and Spanish (M. S. Dryer, 2013b), whereas Subject-
three different languages, English, Italian and Spanish.           Verb is considered a productive word order for both lan-
This goal is pursued by applying a plausibility assessment         guages in SSWL. If Adjective-Noun is a productive order
algorithm against the UD treebanks available for these lan-        for all the three languages according to SSWL, in WALS
guages. Achieved results have been compared with the               the Noun-Adjective order is considered dominant in Italian
threefold aim of: i) reconstructing the frequency distribu-        and Spanish while the reverse order is reported for English
tions of different word order patterns, with particular atten-     (M. S. Dryer, 2013a).
tion to specific constructions (Subject-Verb and Adjective-        Consider now the picture emerging from actual usage as
Noun); ii) assessing similarities and differences across lan-      attested in the English, Italian and Spanish UD treebanks.
guages; and iii) identifying and weighting the factors un-         Figures 1 and 2 report, for the three treebanks, the per-
derlying the different word order patterns identified.             centage distribution of different word orders involving non
                                                                   pronominal Subjects and Verbs (corresponding to the nsubj
         2.    Background and Motivation                           dependency relation), and Adjectives and Nouns (amod),
Starting from Greenberg (1963), word order has been used           respectively. Note that Left and Right in the figures re-
to set up a typology of languages based on the notion that         fer to the position of the dependent (subject or adjectival
“certain languages tend consistently to put modifying or           modifier) with respect to its syntactic head. As it can be
limiting elements before those modified or limited, while          noted, all languages turned out to prefer the Subject-Verb
others just as consistently do the opposite”. Within this          order, but with significant differences: namely, the Italian
area of research, the relative ordering of constituents at the     and Spanish treebanks are characterized by a much higher
clausal level (e.g. verb and subject) as well as at the phrasal    percentage of left–headed subjects than English (i.e. 30%
level (e.g. noun and modifying adjective) has been widely          in Italian, 18% in Spanish and 8% in English). For what
investigated in the typological literature. Nowadays, the          concerns adjectival modifiers, Figure 2 shows that whereas
outcome of these studies has been collected in publicly-           for English the Adjective-Noun order is highly preferred
accessible typological databases. Table 1 reports - for the        (though not the only possible one) in Italian and Spanish
three languages considered in our study - the different or-        the reverse order is rather preferred, i.e. with the adjective
derings of lexical (i.e. non pronominal) Subject and Verb,         occurring on the right side of the head.
and of Adjective and Noun as resulting from the World At-          Corpus-based evidence helps quantifying attested word or-
las of Language Structures (or WALS) and the Syntactic             der patterns across languages, thus leading to a more articu-
Structures of the World’s Languages (or SSWL) (SSWL,               lated picture of word order variation. Registered order vari-
2009) databases. It can be noted that the two provide a            ants, however, do not appear to be evenly distributed, both
slightly different picture for the three languages, maybe fol-     intra- and cross-linguistically. Some are used more often
lowing from the fact that whereas the former records the           and show less grammatical or stylistic restrictions than oth-
“dominant” order the latter testifies “productive” word or-        ers. Let us consider, for example, the relative ordering of

                                                               4541
WALS
                                          Word Order          English      Italian               Spanish
                      Subject-Verb        Subject-Verb          +     No dominant order      No dominant order
                                          Verb-Subject          -     No dominant order      No dominant order
                      Adjective-Noun      Adjective-Noun        +             -                     -
                                          Noun-Adjective        -             +                     +
                                                                SSWL
                      Subject-Verb        Subject-Verb          +             +                      +
                                          Verb-Subject          -             +                      +
                      Adjective-Noun      Adjective-Noun        +             +                      +
                                          Noun-Adjective        +             +                      +

                             Table 1: Subject-Verb and Adjective-Noun order in WALS and SSWL.

Subject and Verb. Figure 3 reports different nsubj instances
occurring in the Italian UD treebank3 . The subject in a) (lit.
‘In this case, he answers within limits ...’) and c) (lit. ‘... the
rights not overtly granted by the licensor remain confiden-
tial’) occurs in the same position, i.e. pre-verbally. There
are however important differences worth noting here: in a)
a pronominal subject immediately precedes the verb, while
in c) a long–distance dependency relation links the nomi-
nal subject to its head. The question which naturally arises
here is how a) and c) relate to each other, and what are the
underlying properties explaining this difference, if any. On
                                                                      Figure 3: Different instances of the nsubj relation in the
the other hand, the sentence reported in b) (lit. ‘Here, from
                                                                      Italian UD treebank.
each corner of the world, arrive 300 thousand patients’) ex-
emplifies a different, less common, Verb-Subject order in-
volving a nominal subject. The question at this point is              ity assessment algorithm named LISCA (LInguiStically–
how a) and c), both with pre-verbal subjects, relate to b),           driven Selection of Correct Arcs) (Dell’Orletta et al., 2013).
with a post-verbal subject. Both questions cannot be an-              As illustrated in details in Section 3.1., the algorithm ex-
swered by simply considering finer-grained frequency dis-             ploits statistics about a wide range of linguistic features
tributions of different types of ordering. On the basis of            (covering different description levels, going from raw text
this, we can claim that a) and c) represent more likely verb-         to morpho-syntax and dependency syntax) extracted from
subject instances than b). But this may not be the case. To           a large reference corpus of automatically parsed sentences
answer questions like these, the properties underlying the            and uses them to assign a plausibility score to each de-
different word order patterns in a given language and cross-          pendency arc contained in a target corpus belonging to the
linguistically need to be investigated.                               same variety of use (i.e. textual genre) of the automatically
Thanks to the availability of corpora for different languages         parsed corpus. Accordingly, all the arcs contained in the
with manually revised linguistic annotation, the focus of             target corpus are ranked from those characterized by a high
studies on word order variation across languages moved                LISCA score to arcs with lower scores: the higher the score,
from discerning possible vs impossible word orders as in              the more similar the linguistic context of an arc with respect
the pioneering studies, to defining dominant vs rare word             to the statistics acquired from the large reference corpus.
order patterns based on actual frequencies attested in cor-           The underlying assumption is that syntactic structures that
pora. More recently, thanks to the wide variety of features           are more frequently generated by a parser are more likely
which can be tracked down and quantified in linguistically            to be plausible than less frequently generated ones.
annotated corpora, current explanations of word order vari-
ation can also aim at capturing finer-grained distinctions            3.1.   The LISCA Algorithm
able to predict the frequency distribution of attested word
orders in different languages. In what follows, we will try           LISCA takes as input a set of parsed sentences and it as-
to exploit the wide range of features which can be extracted          signs a plausibility score to each dependency, which is de-
from treebanks not only to characterize word order patterns           fined as a triple (d, h, t) where d is the dependent, h is
across languages, but also to identify and weight the factors         the head, and t is the type of dependency connecting d to
underlying them.                                                      h. The algorithm operates in two steps: 1) it collects statis-
                                                                      tics about a set of linguistically motivated features extracted
                 3.    Method and Data                                from a dependency annotated corpus obtained through au-
The methodology we devised to acquire typological evi-                tomatic dependency parsing, and 2) it combines the feature
dence from gold treebanks is based on the parse plausibil-            statistics extracted from the corpus used during the previous
                                                                      step. The final plausibility score associated with a given de-
   3                                                                  pendency arc results from the combination of the weights
     In the examples, the dependent is italicized and the head un-
derlined.                                                             associated with these features: the score is computed as a

                                                                  4542
by LISCA results in the following ranking: a) is assigned
                                                                  a higher score with respect to b), whose score in turn is
                                                                  higher than that assigned to c). This ranking follows from
                                                                  the combination of both local and global features taking
                                                                  into account the overall tree structure. From the result-
                                                                  ing LISCA score, it turned out e.g. that in Italian longer
                                                                  distance subjects are less prototypical than post–verbal and
                                                                  shorter ones.

                                                                             4.    Languages and Corpora
                                                                  For the specific concerns of this study we focused on two
                                                                  typologically close languages, namely Italian and Spanish,
                                                                  and a more distant one, English: for what concerns mor-
                                                                  phology, all three languages are fusional, although English
                                                                  has very few inflectional morphemes, which makes it rather
Figure 4: Features used by LISCA to measure arc(d, h, t)          similar to isolating languages. These properties imply that
plausibility.                                                     the prototypical sequence of the main constituents in En-
                                                                  glish strictly follows the linear order Subject-Verb-Object
                                                                  (SVO); Italian and Spanish are SVO languages too, but
simple product of the individual feature weights.                 show more syntactic freedom in the linear ordering of con-
Figure 4 illustrates the features taken into account by           stituents. Because of these properties, highly related to the
LISCA for measuring the plausibility of a given syntactic         linguistic type each language belongs to, we expect to ob-
dependency (d, h, t). For the purposes of the present study,      serve a similar behaviour for typologically close languages
LISCA has been used in its de–lexicalized version in or-          and, on the other hand, significant differences in case of
der to abstract away from variation resulting from lexical        typologically distant languages.
effects. In particular, two different types of features are       The corpora used to collect the statistics to build the LISCA
considered:                                                       models (step 1 in Section 3.1. above) are represented by
                                                                  the English, Italian and Spanish Wikipedia, for a total of
  • local features, corresponding to the characteristics of       around 40 million tokens for each language. The Spanish
    the syntactic arc considered, such as the distance in         and English corpora were morpho–syntactically annotated
    terms of tokens between d and h, or the associative           and parsed by the UDPipe pipeline (Straka et al., 2016)
    strength linking the grammatical categories (i.e. POSd        trained on the Universal Dependency treebanks, version
    and POSh ) involved in the relation, or the POS of the        2.0 (J. Nivre and A. Željko and A.Lars and et alii, 2017).
    head governor and the type of syntactic dependency            The Italian corpus was morpho–syntactically tagged using
    connecting it to h;                                           the ILC–POS–Tagger (Dell’Orletta, 2009), and then parsed
                                                                  with UDPipe.
  • global features, aimed at locating the arc considered
                                                                  LISCA, trained on the models we created earlier, was then
    within the overall syntactic structure of the sentence:
                                                                  applied to the Italian, English and Spanish UD Treebanks in
    for example, the distance of d from the root of the tree,
                                                                  order to assign a plausibility score to each dependency rela-
    or from the closest or most distant leaf node, or the
                                                                  tion. The English Web Treebank (Silveira et al., 2014) con-
    number of “brothers” and “children” nodes of d, oc-
                                                                  tains 16,624 sentences and 254,830 tokens, while the Italian
    curring respectively to its right or left in the linear or-
                                                                  Universal Dependency Treebank (Bosco et al., 2013) con-
    der of the sentence.
                                                                  tains 13,815 sentences corresponding to 325,816 tokens.
LISCA was successfully used against both the output of de-        The Spanish UD treebank (McDonald et al., 2013) is the
pendency parsers and gold treebanks. While in the first case      smallest one, with 4,000 sentences and 112,718 tokens. For
the plausibility score was meant to identify unreliable au-       all resources, most part of the sentences comes from blogs
tomatically produced dependency relations, in the second          and/or newspapers.
case it was used to detect shades of syntactic markedness
of syntactic constructions in manually annotated corpora.                           5.    Data Analysis
The latter is the case of Tusa et al. (2016), where the           For each treebank, the dependencies were first ordered by
LISCA ranking was used to investigate the linguistic no-          decreasing LISCA scores. The list of ordered dependen-
tion of “markedness” (Haspelmath, 2016): a given linguis-         cies was then subdivided into 10 groups, henceforth “bins”,
tic construction is considered “marked” when it deviates          each corresponding to 10% of the total. The distribution
from the “linguistic norm”, i.e. it is “abnormal”. Accord-        of syntactic dependencies in the LISCA bins was analyzed
ingly, unmarked constructions are expected to be character-       in order to investigate whether and to what extent it could
ized by higher LISCA scores and – conversely – construc-          be used to acquire typological trends, i.e. similarities and
tions characterized by increasing degrees of markedness are       differences across languages. This analysis has been car-
associated with lower scores.                                     ried out by comparing the dependency rankings by LISCA
Let us go back to the different instances of the nsubj rela-      (each subdivided into 10 bins) for the three languages taken
tion in Figure 3. The plausibility score assigned to them         into account. As reported by O’Horan et al. (2016), the

                                                              4543
Figure 5: Frequency distribution of right– vs left–headed                 Figure 6: Distribution of right– vs left–headed dependen-
dependencies in the three UD treebanks.                                   cies across the bins.

automatic learning of typological information is typically
carried out from parallel texts. In the case of our study,
parallelism is not concerned with texts, but rather with the
ranking of instances of dependendency relations by LISCA:
besides the fact that the LISCA score is based on the same
set of properties for the three languages, comparability is
guaranteed here by the same inventory of dependency rela-
tions and annotation guidelines shared by the UD treebanks
taken into account.                                                       Figure 7: Average length of dependencies across the bins.
In what follows, we will focus on word order patterns: Sec-
tion 5.1. focuses on a cross-lingual analysis of general
trends of word order formalized in terms of the direction
                                                                          the underlying structural properties of languages. On the
of the dependency link, whereas Section 5.2. reports the re-
                                                                          contrary, the direction-based distribution of dependencies
sults of an in–depth analysis of the frequency distribution of
                                                                          resulting from the LISCA ranking shows interesting sim-
two dependency relations, i.e. nominal subject (nsubj) and
                                                                          ilarities and differences across languages. All languages
adjectival modifier (amod), corresponding to widely inves-
                                                                          share the same trend, i.e. the top LISCA (namely 1-3) bins
tigated constructions in the typological literature.
                                                                          mainly contain right-headed dependencies, whose occur-
5.1.    Word Order Patterns: General Trends                               rence progressively decreases across the bins, until the last
As a first step, for each treebank we analyzed how the de-                two bins (9-10) where they represent around the 20-25% of
pendencies are distributed with respect to their direction.               the dependencies. Similar observations hold in the case of
For the specific purposes of this study, the order is defined             left-headed dependencies, which are characterized by the
by the right or left direction of the dependency that con-                reverse trend: their occurrence starts in the second bin and
nects d to h with respect to the linear order of words in the             progressively increases to cover about 80% of the relations
sentence. Figure 5 shows the frequency distribution of all                in the last two bins (9-10). Although this trend is shared by
dependencies in the three considered UD treebanks, distin-                the three languages taken into account, there are also sig-
guishing right–headed dependencies (i.e. d > h) vs left–                  nificant differences worth being pointed out. In fact, the
headed ones (i.e. h < d).4 Interestingly enough, in the                   trend reported in Figure 6 for English is slightly different if
three treebanks the frequency of the the two word orders                  compared with what observed for the other two Romance
is similar: whereas for Italian and Spanish they are equally              languages. For English, the lines representing the distri-
partitioned (50% both d > h and h < d), for English the                   bution of the left– and right–headed dependencies are less
ordering d > h covers 55% of the cases. Yet, if we focus on               steep and cross at the level of the 6th bin, while the Ital-
the distribution of dependencies across the LISCA bins in-                ian and Spanish lines are steeper and cross in the 5th one.
teresting differences across languages can be observed (see               This is to say that the distribution of dependencies in the
Figure 6). Despite their similar frequency in the three lan-              interval between the 4th and 9th bins is language-specific.
guages, right– vs left–headed dependencies are described                  A possible explanation for this is that English word order is
by opposite trends: whereas in the first bins d > h depen-                more rigid and shows a higher number of right–headed de-
dencies are more frequent, in the latter h < d dependencies               pendencies (including explicit subjects). On the contrary,
are mainly observed.                                                      Italian and Spanish, characterized by a more flexible word
This result confirms the intuition we started from, i.e. that             order, show a higher variability at the level of dependency
statistics about the frequency distribution of right- vs left-            direction.
headed dependencies in treebanks do not say much about                    We can now try to understand what distinguishes right–
                                                                          or left–headed dependencies occurring in the top vs bot-
   4
      In all figures left and right refer to the position of d relative   tom bins. Consider the distribution of dependencies with
to its governor h, respectively corresponding to right–headed (i.e.       respect to their length, as shown in Figure 7: all lan-
d > h) vs left–headed (i.e. h < d) dependencies.                          guages share the fact that longer dependencies are ranked

                                                                      4544
Figure 8: Distribution of the nsubj relation across the bins.

by LISCA in the last bins. This in line with the Depen-
dency Length Minimization principle (Temperly, 2007),
i.e. the idea that languages tend to place closely related
words close together since shortest dependency links re-
                                                                     Figure 9: Direction of the nsubj relation across the bins.
duce the human processing load and make the sentence
comprehension process easier (Gibson, 1998). If we con-
sider the average length of dependencies for each language,
we note that they are quite similar, i.e. they are about          gatorily requires an explicit subject. This is also reflected
three–token long on average. However, the LISCA rank-             by the frequency distribution of pronominal subjects, which
ing highlighted interesting differences between typologi-         correspond to 59% of the total amount of nominal subjects
cally different languages. The average length of Italian and      in the English treebank, while they cover only about 3% of
Spanish dependencies share a more similar trend with re-          the cases in both Italian and Spanish treebanks.
spect to the English ones: the confidence interval of the         Consider now the distribution of nsubj relations by de-
difference between means is lower for Italian and Spanish         pendency direction: in line with what reported in Section
(0.077 points) than both English/Italian (0.47 points), and       2., we focus now on lexical (i.e. non-pronominal) sub-
English/Spanish (0.54 points). On the basis of this we can        jects only. In Figure 1, it is shown that in all considered
hypothesize an interesting interplay between dependency           languages subjects are mostly right-headed, whereas sig-
direction and dependency length: among the underlying             nificant differences are recorded for left–headed subjects
properties of relations occurring in the last bins there are      whose percentage is much higher for Italian and Spanish
structural factors at work such as dependency length.             (i.e. 30% and 18% respectively) than for English (9%).
                                                                  Figure 9 reports, for the three languages, the distribution
5.2.   The case of nsubj and amod relations                       of right–headed subjects (i.e. d > h) vs left–headed ones
                                                                  (i.e. h < d) across the LISCA bins. Left–headed subjects
Let us focus now on specific dependency relations, namely         turned out to be mainly concentrated in the second half of
nominal subjects (nsubj) and adjectival modifiers (amod),         the LISCA bins, starting from the 5th one. Consider as an
and their distribution across the LISCA bins. These re-           example the following nsubj relation ranked in the 10th bin
lations correspond to constructions widely investigated in        for Italian: ‘Dalla rabbia dei valonesi non si salva niente
the typological literature, and for this reason they represent    e nessuno’ (lit. ‘From the rage of Valaisers not be saved
two challenging testbeds for our methodology of analysis.         nothing and nobody’, ‘From the rage of Valaisers nothing
Given the typology of features underlying LISCA, differ-          and nobody can be saved’). The frequency of left–headed
ent factors contribute to the distribution of the same relation   subjects for the English language is much lower than for
across the bins, concerning local features such as the linear     Romance languages, being they mainly restricted to paren-
ordering of words involved in the relation or their distance,     thetical clauses, such as for example [...], said Bush, and
to more global characteristics reflecting the position of the     existential clauses.
dependency arc within the overall dependency tree. In what        We have seen that the LISCA bins progressively gather oc-
follows, we try to reconstruct what are properties playing a      currences of rarer and less prototypical relation instances,
role in determining the distribution of different word order      such as left-headed subjects, whose occurrence mainly con-
patterns across the LISCA bins.                                   centrates in the last four bins. Yet, as already pointed out in
Nominal subjects (nsubj). Figure 8 reports the distribution       Section 2., the position with respect to the governing head
of nominal subjects (both lexical and pronominal) across          may be influenced by different linguistic properties. Con-
the LISCA bins. As it can be noted, the three languages are       sider the following examples for the three languages:
characterized by different distributions. First, whereas for
English nsubj relations already appear in the top positions           • Italian: ‘La proposta presentata dalla commissione
of the LISCA ranking, i.e. in the 2nd bin, the first occur-             conformente al suo mandato costituisce un punto di
rences of nsubj relations for Italian and Spanish are in the            partenza’ (lit. ‘The proposal presented by the commis-
4th and 5th bins respectively. Second, main differences can             sion in accordance with its mandate represents a start-
be observed at the level of frequency distributions. Both               ing point’), where the subject proposta (‘proposal’) is
differences can be explained by considering that whereas                modified by a participial phrase which creates a long-
Italian and Spanish are pro–drop languages English obli-                distance nsubj dependency;

                                                              4545
Figure 10: Average length of nsubj relations across the bins.

  • Spanish: ‘El descenso de la población indı́gena y la
    falta de mano de obra para los obrajes españoles
    originó el comercio de pobladores secuestrados ...’
    (lit. ‘The decline of the indigenous population and the
    lack of labor for the Spanish obrajes led to the trade       Figure 11: Average length of right– and left–headed lexical
    of kidnapped settlers ...’), where a coordinated subject     nsubj across the LISCA bins.
    determines a long-distance dependency link;

  • English: ‘Sergey Brin has actually a mathematical
    proof that the company’s self–driven research strat-         LISCA bins with respect to their length, to assess the corre-
    egy, which gives employees one day a week to do re-          lation, if any, between dependency length and word order.
    search projects on their own, is a good, respectable         As shown in Figure 11, for all languages the maximum av-
    idea’, where 22 tokens occur between the lexical sub-        erage length of right–headed nsubj relations is higher (i.e.
    ject strategy and its syntactic head.                        about 12 tokens for the Romance languages and about 8 for
                                                                 English) than the maximum average length of less frequent
All nsubj relations exemplified above have been ranked in        left–headed subjects (which is about 4 tokens for all lan-
the last (i.e. 10th) LISCA bin of each ordered list of de-       guages). Besides reported differences among languages, a
pendencies: they all represent long distance dependencies        similar trend can be reported here: namely, all languages
involving a right-headed subject. These examples suggest         tend to minimize the dependency length when an alterna-
an interesting and complex interplay between dependency          tive (with respect to the dominant, i.e. most frequent one)
length and the position of the subject with respect to the       word order is used. Dependency length minimization varies
governing head in influencing word order patterns, which         across languages: whereas in English the average depen-
is worth being investigated.                                     dency length of right–headed subjects is only slighly lower
In Section 5.1. we hypothesized a correlation between word       with respect to left–headed ones (i.e. 2.62 vs 2.76), for Ital-
order and dependency length. Let us now explore how the          ian and Spanish length minimization is more evident (i.e.
two interact for a specific dependency relation, nsubj. In       2.48 vs 4.64 and 2.38 vs 4.54 respectively). From what
Figure 10, it is reported that for all languages “heavier”       seen so far, we can put forward the hypothesis that length
(i.e. long distance) subjects (both right- and left-headed)      minimization plays a stronger role with less frequent, and
are ranked in the last bins. For most part of the bins, Ro-      therefore more marked, word orders: in other words, if on
mance languages show stronger similarities with respect          the one hand marked order is associated with minimized de-
to English. Despite these differences, each language ap-         pendency length, on the other hand the dominant unmarked
pears to follow the same trend, characterized by the fact that   order allows significantly longer dependencies.
the ordered bins contain increasingly longer dependencies.       Adjectival modifiers amod. Figure 12 reports the distribu-
Two questions arise at this juncture: i) whether dependency      tion of adjectival modifiers across the bins: it can be noticed
length is also influenced by the syntactic realization of the    that while for English they already appear in the 1st bin, for
subject, i.e. lexical or pronominal; ii) for lexical subjects,   both Romance languages the first occurrences of amod rela-
whether and to what extent dependency length influences          tions start in the 2nd bin. This difference can be explained
subject order patterns.                                          by considering that even in this area English is character-
Concerning the first question, our intuition is that longer      ized by a generally fixed order or at most slightly variable
dependencies are typically represented by lexical subjects,      structures, which are more easily predictable. Their distri-
i.e. by nsubj relations with a noun as dependent. This is        bution across the bins also varies significantly, with Italian
confirmed by the fact that for all languages nsubj relations     and Spanish sharing a similar trend as opposed to English.
ranked in the first LISCA bins mostly have a pronoun as a        Consider now the relative order of Adjective and Noun:
dependent. On the other hand, relations ranked in the last       in Figure 2 it was shown that the pre–nominal adjectives
bins are represented by long-distance dependencies with          are more common in the English UD treebank than in the
lexical subjects, as exemplified by the sample sentences re-     treebanks of the Romance languages. The distribution of
ported above for the three languages.                            the amod relation across the LISCA bins shows that right–
Consider now the second open issue above, in particular the      headed adjectives are more prototypical in English than in
distribution of left– and right–headed subjects across the       Italian and Spanish (see Figure 2). This is due to the fact

                                                             4546
6.   Conclusion
                                                                In this paper we presented a new methodology aimed at
                                                                acquiring typological evidence from “gold” treebanks for
                                                                different languages. In particular, we investigated whether
                                                                algorithms for measuring the plausibility of dependency re-
                                                                lations within the output of dependency parsers could be
                                                                exploited to acquire quantitative evidence from gold tree-
                                                                banks to shed new light in linguistic typological studies.
Figure 12: Distribution of the amod relation across the bins.   The methodology was tested in a case study carried out on
                                                                UD treebanks for different languages: English, Italian and
                                                                Spanish. By relying on a wide range of linguistic properties
                                                                aimed at weighting the plausibility of a given dependency
                                                                arc, it has been possible to reconstruct an articulated profile
                                                                of word order patterns attested in the languages considered,
                                                                in line with the literature and which has been enriched with
                                                                new interesting insights. Starting from the study of general
                                                                word order trends and their relationship with dependency
                                                                length, we focused on two dependency relations widely
                                                                investigated in the typological literature, nominal subjects
                                                                (nsubj) and adjectival modifiers (amod). For both of them,
                                                                word order similarities and differences were reported for
                                                                all languages, significantly enriching the picture emerging
                                                                from typological databases and showing the added value of
                                                                the LISCA-based methodology with respect to simple fre-
                                                                quency distributions. We also investigated the underlying
                                                                properties influencing the preference towards one word or-
                                                                der or the other. Among them, dependency length turned
        Figure 13: Direction of the amod relation.              out to play a significant role: its impact, however, appears
                                                                to vary according to whether a dominant or marked order
                                                                is used. Dependency length minimization seems to be at
                                                                work with less frequent order patterns, thus suggesting an
that, differently from English, a marked position of adjec-     interesting interaction between word order and dependency
tives is allowed in Italian and Spanish. The distribution of    length.
relations across the LISCA bins allows detecting adjectival     However, the potentialities of the method are not restricted
modifiers occurring in prototypical constructions.              to the area of typological studies. Nowadays, linguistic
Differently from what observed for subjects, the average        typology is starting to play a role in multilingual Natural
distance between d and h in amod relations remains ten-         Language Processing (O’Horan et al., 2016). While the
dentially constant through the bins (see Figure 14), ranging    growing importance of typological information in support-
between 1 and about 3. Some differences can be observed if      ing multilingual tasks has been recognized, existing typo-
we compare English and the two Romance languages: the           logical databases such as WALS have still a partial cover-
latter tend to be characterized by shorter relations. This      age, and most importantly here, do not always reflect real
may be explained in terms of adjacency to the nominal           language use. Methods for automatic induction of typolog-
head: i.e. in the left–headed position adjectival modifiers     ical information are still at the beginning: this paper rep-
are typically adjacent to the head, which is not necessarily    resents a promising attempt in this area. It is a widely ac-
the case in the case of prenominal position where the adjec-    knowledged fact that word-order affects the automatic anal-
tival modifier can be separated from the head by elements       ysis of sentences: free–order languages are harder to parse
that belong to the same subtree.                                (Gulordava and Merlo, 2015c). Acquired information from
                                                                real language usage can be used among the “selective shar-
                                                                ing parameters” in a cross–lingual transfer parsing scenario
                                                                (Naseem et al., 2012).
                                                                Further directions of research include: the application of
                                                                the methodology to other languages, including typolog-
                                                                ically distant ones, to reconstruct shades of typological
                                                                proximity starting from real language usage; the analysis
                                                                of other dependency relations as well as of more complex
                                                                structures such as dependency subtrees; the extension of the
                                                                range of properties expected to influence the preference to-
                                                                wards a given word order pattern. Last but not least, we are
       Figure 14: Average length of amod relations.             planning to test the effectiveness of acquired typological
                                                                information in a cross–lingual parsing scenario.

                                                            4547
Acknowledgements                                 the 19th Conference on Computational Natural Lan-
The work reported in the paper was partially supported by         guage Learning, CoNLL 2015, Beijing, China, July 30-
the 2–year project (2017-2019) UBIMOL, UBIquitous Mas-            31, 2015, pages 247–257.
sive Open Learning, funded by Regione Toscana (BANDO           Gulordava, K. and Merlo, P. (2015c). Structural and lex-
POR FESR 2014-2020).                                              ical factors in adjective placement in complex noun
                                                                  phrases across romance languages. In Proceedings of
         7.    Bibliographical References                         Conference on Computational Natural Language Learn-
                                                                  ing (CoNLL), pages 247–257, Beijing, China, July.
Alzetta, C., Dell’Orletta, F., Montemagni, S., and Ven-        Haspelmath, M. (2016). Against Markedness (and what to
  turi, G. (2018). Dangerous relations in dependency tree-        Replace it with). Journal of Linguistics, 42:25–70.
  banks. In Proceedings of the 16th International Work-
                                                               Kay, M. (2005). Acl lifetime achievement award: A life of
  shop on Treebanks and Linguistic Theories (TLT16),
                                                                  language. Computational Linguistics, 31(4).
  pages 201–210, Prague, Czech Republic, January.
                                                               Kučera, H. (1982). Markedness and frequency: a compu-
Baldwin, T. and Kordoni, V., (2011). Interaction between
                                                                  tational analysis. In Proceedings of COLING 82.
  Linguistics and Computational Linguistics. CSLI Publi-
  cations.                                                     McDonald, R., Nivre, J., Quirmbach-Brundage, Y., Gold-
Bosco, C., Montemagni, S., and Simi, M. (2013). Convert-          berg, Y., Das, D., Ganchev, K., Hall, K., Petrov, S.,
  ing italian treebanks: Towards an italian stanford depen-       Zhang, H., Täckström, O., Bedini, C., Castelló, N. B.,
  dency treebank. In Proceedings of the ACL Linguistic            and Lee, J. (2013). Universal dependency annotation for
  Annotation Workshop & Interoperability with Discourse,          multilingual parsing. In Proceedings of the 51st Annual
  Sofia, Bulgaria, August.                                        Meeting of the Association for Computational Linguis-
                                                                  tics, pages 92–97.
Che, W., Guo, J., and Liu, T. (2014). Reliable depen-
  dency arc recognition. Expert Systems with Applica-          Merlo, P. (2016). Quantitative computational syntax: some
  tions, 41(4):1716–1722.                                         initial result. Italian Journal of Computational Linguis-
Dell’Orletta, F., Venturi, G., and Montemagni, S. (2011).         tics, 2.
  ULISSE: an unsupervised algorithm for detecting reli-        Naseem, T., Barzilay, R., and Globerson, A. (2012). Se-
  able dependency parses. In Proceedings of the Fifteenth         lective sharing for multilingual dependency parsing. In
  Conference on Computational Natural Language Learn-             Proceedings of the 50th Annual Meeting of the Associa-
  ing (CoNLL 2011), pages 115–124, Portland, Oregon,              tion for Computational Linguistics, pages 629–637, Jeju,
  USA, June. Association for Computational Linguistics            Republic of Korea, July.
  Stroudsburg, PA, USA.                                        Nivre, J. (2015). Towards a universal grammar for natu-
Dell’Orletta, F., Venturi, G., and Montemagni, S. (2013).         ral language processing. In Computational Linguistics
  Linguistically-driven selection of correct arcs for depen-      and Intelligent Text Processing - Proceedings of the 16th
  dency parsing. Computaciòn y Sistemas, 2:125–136.              International Conference, CICLing 2015, Part I, pages
Dell’Orletta, F. (2009). Ensemble system for part-of-             3–16, Cairo, Egypt, April.
  speech tagging. In Proceedings of EVALITA 2009 - Eval-       O’Horan, H., Berzak, Y., Vulic, I., Reichart, R., and Ko-
  uation of NLP and Speech Tools for Italian 2009, Reggio         rhonen, A. (2016). Survey on the use of typological in-
  Emilia, Italy, December.                                        formation in natural language processing. In COLING
Futrell, R., Mahowald, K., and Gibson, E. (2015). Quan-           2016, 26th International Conference on Computational
  tifying word order freedom in dependency corpora. In            Linguistics, Proceedings of the Conference: Technical
  Proceedings of the Third International Conference on            Papers, December 11-16, 2016, Osaka, Japan, pages
  Dependency Linguistics, DepLing 2015, August 24-26              1297–1308.
  2015, Uppsala University, Uppsala, Sweden, pages 91–         Reichart, R. and Rappoport, A. (2009). Sample selection
  100.                                                            for statistical parsers: Cognitively driven algorithms and
Gibson, E. (1998). Linguistic complexity: Locality of syn-        evaluation measures. In Proceedings of the Conference
  tactic dependencies. Cognition, 68:1–76.                        on Computational Natural Language Learning (CoNLL
Greenberg, J. (1963). Some universals of grammar with             2009), pages 3–11. Association for Computational Lin-
  particular reference to the order of meaningful elements.       guistics Stroudsburg, PA, USA.
  In Joseph H. Greenberg, editor, Universals of Language,      Silveira, N., Dozat, T., de Marneffe, M., Bowman, S., Con-
  pages 73–113.                                                   nor, M., Bauer, J., and Manning, C. (2014). A gold
Gulordava, K. and Merlo, P. (2015a). Diachronic                   standard dependency corpus for english. In Proceedings
  trends in word order freedom and dependency length              of the Ninth International Conference on Language Re-
  in dependency-annotated corpora of latin and ancient            sources and Evaluation (LREC).
  greek. In Proceedings of the Third International Con-        Straka, M., Hajic, J., and Strakova, J. (2016). Ud-
  ference on Dependency Linguistics, DepLing 2015, Au-            pipe: Trainable pipeline for processing conll-u files per-
  gust 24-26 2015, Uppsala University, Uppsala, Sweden,           forming tokenization, morphological analysis, pos tag-
  pages 121–130.                                                  ging and parsing. In Proceedings of the Tenth Interna-
Gulordava, K. and Merlo, P. (2015b). Structural and lex-          tional Conference on Language Resources and Evalua-
  ical factors in adjective placement in complex noun             tion (LREC).
  phrases across romance languages. In Proceedings of          Temperly, D. (2007). Minimization of dependency length

                                                           4548
in written english. Cognition, 105(2):300–333.
Tusa, E., Dell’Orletta, F., Montemagni, S., and Venturi, G.
  (2016). Dieci sfumature di marcatezza sintattica: verso
  una nozione computazionale di complessitá. In Proceed-
  ings of the Third Italian Conference on Computational
  Linguistics (CLiC-it), pages 3–16, Napoli, Italy, Decem-
  ber.

      8.    Language Resource References
J. Nivre and A. Željko and A.Lars and et alii. (2017).
   Universal Dependencies 2.0. LINDAT/CLARIN dig-
   ital library at the Institute of Formal and Ap-
   plied Linguistics, Charles University, 2.0, ISLRN
   http://hdl.handle.net/11234/1-1983.
M. S. Dryer and M. Haspelmath. (2013). WALS Online.
   Max Planck Institute for Evolutionary Anthropology,
   ISLRN http://wals.info/.
M. S. Dryer. (2013a). Order of Adjective and Noun. Max
   Planck Institute for Evolutionary Anthropology, ISLRN
   http://wals.info/chapter/87.
M. S. Dryer. (2013b). Order of Subject and Verb. Max
   Planck Institute for Evolutionary Anthropology, ISLRN
   http://wals.info/chapter/82.
SSWL. (2009). Syntactic Structures of the World’s Lan-
   guages. ISLRN http://sswl.railsplayground.net/.

                                                          4549
You can also read